2024-05-21T21:28:12.905 INFO:root:teuthology version: 0.0.1.dev287+gb9e3da8 2024-05-21T21:28:12.905 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_leonidus@teuthology --archive /home/teuthworker/archive/leonidus-2024-05-21_21:13:35-fs-wip-lusov-quiesce-overdrive-export-distro-default-smithi/7719034 --verbose --name leonidus-2024-05-21_21:13:35-fs-wip-lusov-quiesce-overdrive-export-distro-default-smithi --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-quiesce/with-quiesce 6-workunit/suites/fsync-tester}} -- /home/teuthworker/archive/leonidus-2024-05-21_21:13:35-fs-wip-lusov-quiesce-overdrive-export-distro-default-smithi/7719034/orig.config.yaml 2024-05-21T21:28:12.967 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-21T21:28:13.055 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/leonidus-2024-05-21_21:13:35-fs-wip-lusov-quiesce-overdrive-export-distro-default-smithi/7719034 branch: wip-lusov-quiesce-overdrive-export description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-quiesce/with-quiesce 6-workunit/suites/fsync-tester}} email: null first_in_suite: false job_id: '7719034' kernel: client: branch: testing kdb: true sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule name: leonidus-2024-05-21_21:13:35-fs-wip-lusov-quiesce-overdrive-export-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-lusov-quiesce-overdrive-export ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request sha1: 3b748ea08891006c1152513d25ab4a03f7c9a995 subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount dry_run: true install: ceph: flavor: default sha1: 3b748ea08891006c1152513d25ab4a03f7c9a995 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=legacy - wsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-lusov-quiesce-overdrive-export sha1: 7956b5e965f6ee5669688b5643c11d881b72442c owner: scheduled_leonidus@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 3849 sha1: 3b748ea08891006c1152513d25ab4a03f7c9a995 sleep_before_teardown: 0 subset: 941/111111 suite: fs suite_branch: wip-lusov-quiesce-overdrive-export suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_7956b5e965f6ee5669688b5643c11d881b72442c/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 7956b5e965f6ee5669688b5643c11d881b72442c syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi151.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNGEdMvIHhOjJ4k9VKosMddoRPgB5+Pe0poeOYkfTB6RSt6uVC4U4eUkjslar2or/XZdXeXCSBmNOH/8seSyQ94= smithi164.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBK1Md+eimcaFfm6D3akWRuJP2hfXDDOl9Z6bg3nZG3CB8WsZfzgnDxjEe4V8CRKWng8sot0O4OPa7mdg9cGBk3w= smithi184.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJgRvLg3FarXIFUftI/MhFXxPAPHo7zZaD2WSchPiEH9RSwsQMNV5tu+TLN5HkmY8zFX7dCzzx+cXNgonOId7Ts= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph config set mgr mgr/snap_schedule/log_level debug - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - quiescer: null - workunit: clients: all: - suites/fsync-tester.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: b9e3da87b1ec7920937cb4902b7e3fd9eda86b4a timestamp: 2024-05-21_21:13:35 tube: smithi user: leonidus verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2698841 2024-05-21T21:28:13.055 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_7956b5e965f6ee5669688b5643c11d881b72442c/qa; will attempt to use it 2024-05-21T21:28:13.056 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_7956b5e965f6ee5669688b5643c11d881b72442c/qa/tasks 2024-05-21T21:28:13.056 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-21T21:28:13.058 INFO:teuthology.task.internal:Checking packages... 2024-05-21T21:28:13.079 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '3b748ea08891006c1152513d25ab4a03f7c9a995' 2024-05-21T21:28:13.079 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-21T21:28:13.079 INFO:teuthology.packaging:ref: None 2024-05-21T21:28:13.079 INFO:teuthology.packaging:tag: None 2024-05-21T21:28:13.079 INFO:teuthology.packaging:branch: wip-lusov-quiesce-overdrive-export 2024-05-21T21:28:13.080 INFO:teuthology.packaging:sha1: 3b748ea08891006c1152513d25ab4a03f7c9a995 2024-05-21T21:28:13.080 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-lusov-quiesce-overdrive-export 2024-05-21T21:28:13.226 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3857.g7956b5e9 2024-05-21T21:28:13.227 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-21T21:28:13.233 INFO:teuthology.task.internal:no buildpackages task found 2024-05-21T21:28:13.233 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-21T21:28:13.276 INFO:teuthology.task.internal:Saving configuration 2024-05-21T21:28:13.309 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-21T21:28:13.315 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-21T21:28:13.340 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi151.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-05-21_21:13:35-fs-wip-lusov-quiesce-overdrive-export-distro-default-smithi/7719034', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-21 21:23:07.838295', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNGEdMvIHhOjJ4k9VKosMddoRPgB5+Pe0poeOYkfTB6RSt6uVC4U4eUkjslar2or/XZdXeXCSBmNOH/8seSyQ94='} 2024-05-21T21:28:13.361 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi164.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-05-21_21:13:35-fs-wip-lusov-quiesce-overdrive-export-distro-default-smithi/7719034', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-21 21:23:07.839432', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBK1Md+eimcaFfm6D3akWRuJP2hfXDDOl9Z6bg3nZG3CB8WsZfzgnDxjEe4V8CRKWng8sot0O4OPa7mdg9cGBk3w='} 2024-05-21T21:28:13.386 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi184.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-05-21_21:13:35-fs-wip-lusov-quiesce-overdrive-export-distro-default-smithi/7719034', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-21 21:23:07.836814', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJgRvLg3FarXIFUftI/MhFXxPAPHo7zZaD2WSchPiEH9RSwsQMNV5tu+TLN5HkmY8zFX7dCzzx+cXNgonOId7Ts='} 2024-05-21T21:28:13.386 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-21T21:28:13.392 INFO:teuthology.task.internal:roles: ubuntu@smithi151.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-05-21T21:28:13.392 INFO:teuthology.task.internal:roles: ubuntu@smithi164.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-05-21T21:28:13.392 INFO:teuthology.task.internal:roles: ubuntu@smithi184.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-05-21T21:28:13.392 INFO:teuthology.run_tasks:Running task console_log... 2024-05-21T21:28:13.489 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f8c14c2d240>, signals=[15]) 2024-05-21T21:28:13.489 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-21T21:28:13.495 INFO:teuthology.task.internal:Opening connections... 2024-05-21T21:28:13.495 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi151.front.sepia.ceph.com 2024-05-21T21:28:13.496 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-21T21:28:13.575 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi164.front.sepia.ceph.com 2024-05-21T21:28:13.576 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-21T21:28:13.648 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi184.front.sepia.ceph.com 2024-05-21T21:28:13.649 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi184.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-21T21:28:13.725 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-21T21:28:13.731 DEBUG:teuthology.orchestra.run.smithi151:> uname -m 2024-05-21T21:28:13.749 INFO:teuthology.orchestra.run.smithi151.stdout:x86_64 2024-05-21T21:28:13.749 DEBUG:teuthology.orchestra.run.smithi151:> cat /etc/os-release 2024-05-21T21:28:13.807 INFO:teuthology.orchestra.run.smithi151.stdout:NAME="CentOS Stream" 2024-05-21T21:28:13.807 INFO:teuthology.orchestra.run.smithi151.stdout:VERSION="9" 2024-05-21T21:28:13.807 INFO:teuthology.orchestra.run.smithi151.stdout:ID="centos" 2024-05-21T21:28:13.807 INFO:teuthology.orchestra.run.smithi151.stdout:ID_LIKE="rhel fedora" 2024-05-21T21:28:13.807 INFO:teuthology.orchestra.run.smithi151.stdout:VERSION_ID="9" 2024-05-21T21:28:13.807 INFO:teuthology.orchestra.run.smithi151.stdout:PLATFORM_ID="platform:el9" 2024-05-21T21:28:13.807 INFO:teuthology.orchestra.run.smithi151.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-21T21:28:13.807 INFO:teuthology.orchestra.run.smithi151.stdout:ANSI_COLOR="0;31" 2024-05-21T21:28:13.807 INFO:teuthology.orchestra.run.smithi151.stdout:LOGO="fedora-logo-icon" 2024-05-21T21:28:13.807 INFO:teuthology.orchestra.run.smithi151.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-21T21:28:13.808 INFO:teuthology.orchestra.run.smithi151.stdout:HOME_URL="https://centos.org/" 2024-05-21T21:28:13.808 INFO:teuthology.orchestra.run.smithi151.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-21T21:28:13.808 INFO:teuthology.orchestra.run.smithi151.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-21T21:28:13.808 INFO:teuthology.orchestra.run.smithi151.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-21T21:28:13.808 INFO:teuthology.lock.ops:Updating smithi151.front.sepia.ceph.com on lock server 2024-05-21T21:28:13.830 DEBUG:teuthology.orchestra.run.smithi164:> uname -m 2024-05-21T21:28:13.848 INFO:teuthology.orchestra.run.smithi164.stdout:x86_64 2024-05-21T21:28:13.848 DEBUG:teuthology.orchestra.run.smithi164:> cat /etc/os-release 2024-05-21T21:28:13.905 INFO:teuthology.orchestra.run.smithi164.stdout:NAME="CentOS Stream" 2024-05-21T21:28:13.905 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION="9" 2024-05-21T21:28:13.905 INFO:teuthology.orchestra.run.smithi164.stdout:ID="centos" 2024-05-21T21:28:13.905 INFO:teuthology.orchestra.run.smithi164.stdout:ID_LIKE="rhel fedora" 2024-05-21T21:28:13.905 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION_ID="9" 2024-05-21T21:28:13.905 INFO:teuthology.orchestra.run.smithi164.stdout:PLATFORM_ID="platform:el9" 2024-05-21T21:28:13.905 INFO:teuthology.orchestra.run.smithi164.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-21T21:28:13.906 INFO:teuthology.orchestra.run.smithi164.stdout:ANSI_COLOR="0;31" 2024-05-21T21:28:13.906 INFO:teuthology.orchestra.run.smithi164.stdout:LOGO="fedora-logo-icon" 2024-05-21T21:28:13.906 INFO:teuthology.orchestra.run.smithi164.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-21T21:28:13.906 INFO:teuthology.orchestra.run.smithi164.stdout:HOME_URL="https://centos.org/" 2024-05-21T21:28:13.906 INFO:teuthology.orchestra.run.smithi164.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-21T21:28:13.906 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-21T21:28:13.906 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-21T21:28:13.906 INFO:teuthology.lock.ops:Updating smithi164.front.sepia.ceph.com on lock server 2024-05-21T21:28:13.931 DEBUG:teuthology.orchestra.run.smithi184:> uname -m 2024-05-21T21:28:13.951 INFO:teuthology.orchestra.run.smithi184.stdout:x86_64 2024-05-21T21:28:13.951 DEBUG:teuthology.orchestra.run.smithi184:> cat /etc/os-release 2024-05-21T21:28:14.008 INFO:teuthology.orchestra.run.smithi184.stdout:NAME="CentOS Stream" 2024-05-21T21:28:14.008 INFO:teuthology.orchestra.run.smithi184.stdout:VERSION="9" 2024-05-21T21:28:14.008 INFO:teuthology.orchestra.run.smithi184.stdout:ID="centos" 2024-05-21T21:28:14.008 INFO:teuthology.orchestra.run.smithi184.stdout:ID_LIKE="rhel fedora" 2024-05-21T21:28:14.008 INFO:teuthology.orchestra.run.smithi184.stdout:VERSION_ID="9" 2024-05-21T21:28:14.008 INFO:teuthology.orchestra.run.smithi184.stdout:PLATFORM_ID="platform:el9" 2024-05-21T21:28:14.008 INFO:teuthology.orchestra.run.smithi184.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-21T21:28:14.008 INFO:teuthology.orchestra.run.smithi184.stdout:ANSI_COLOR="0;31" 2024-05-21T21:28:14.008 INFO:teuthology.orchestra.run.smithi184.stdout:LOGO="fedora-logo-icon" 2024-05-21T21:28:14.008 INFO:teuthology.orchestra.run.smithi184.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-21T21:28:14.008 INFO:teuthology.orchestra.run.smithi184.stdout:HOME_URL="https://centos.org/" 2024-05-21T21:28:14.009 INFO:teuthology.orchestra.run.smithi184.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-21T21:28:14.009 INFO:teuthology.orchestra.run.smithi184.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-21T21:28:14.009 INFO:teuthology.orchestra.run.smithi184.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-21T21:28:14.009 INFO:teuthology.lock.ops:Updating smithi184.front.sepia.ceph.com on lock server 2024-05-21T21:28:14.030 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-21T21:28:14.039 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-21T21:28:14.068 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-21T21:28:14.068 DEBUG:teuthology.orchestra.run.smithi151:> test '!' -e /home/ubuntu/cephtest 2024-05-21T21:28:14.071 DEBUG:teuthology.orchestra.run.smithi164:> test '!' -e /home/ubuntu/cephtest 2024-05-21T21:28:14.073 DEBUG:teuthology.orchestra.run.smithi184:> test '!' -e /home/ubuntu/cephtest 2024-05-21T21:28:14.090 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-21T21:28:14.102 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-21T21:28:14.102 DEBUG:teuthology.orchestra.run.smithi151:> test -z $(ls -A /var/lib/ceph) 2024-05-21T21:28:14.128 DEBUG:teuthology.orchestra.run.smithi164:> test -z $(ls -A /var/lib/ceph) 2024-05-21T21:28:14.130 DEBUG:teuthology.orchestra.run.smithi184:> test -z $(ls -A /var/lib/ceph) 2024-05-21T21:28:14.226 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-21T21:28:14.294 INFO:teuthology.run_tasks:Running task kernel... 2024-05-21T21:28:14.308 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'} 2024-05-21T21:28:14.309 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-05-21T21:28:14.309 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-05-21T21:28:14.309 DEBUG:teuthology.orchestra.run.smithi151:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-21T21:28:14.325 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-21T21:28:14.326 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-05-21T21:28:14.613 DEBUG:teuthology.task.kernel:sha1 for client.0 is 4fed890e8ce3966dbe380e9eb4b0a796eddb841c 2024-05-21T21:28:14.613 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "4fed890e8ce3966dbe380e9eb4b0a796eddb841c"... 2024-05-21T21:28:14.613 DEBUG:teuthology.orchestra.run.smithi151:> uname -r 2024-05-21T21:28:14.630 INFO:teuthology.orchestra.run.smithi151.stdout:5.14.0-437.el9.x86_64 2024-05-21T21:28:14.631 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 4fed890e8ce3966dbe380e9eb4b0a796eddb841c 2024-05-21T21:28:14.631 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-05-21T21:28:14.631 DEBUG:teuthology.orchestra.run.smithi151:> sudo yum upgrade -y linux-firmware 2024-05-21T21:28:19.471 INFO:teuthology.orchestra.run.smithi151.stdout:CentOS Stream 9 - BaseOS 2.5 MB/s | 8.1 MB 00:03 2024-05-21T21:28:22.137 INFO:teuthology.orchestra.run.smithi151.stdout:CentOS Stream 9 - AppStream 12 MB/s | 19 MB 00:01 2024-05-21T21:28:26.613 INFO:teuthology.orchestra.run.smithi151.stdout:CentOS Stream 9 - CRB 7.1 MB/s | 6.2 MB 00:00 2024-05-21T21:28:28.047 INFO:teuthology.orchestra.run.smithi151.stdout:CentOS Stream 9 - Extras packages 54 kB/s | 16 kB 00:00 2024-05-21T21:28:29.107 INFO:teuthology.orchestra.run.smithi151.stdout:Extra Packages for Enterprise Linux 22 MB/s | 21 MB 00:00 2024-05-21T21:28:34.248 INFO:teuthology.orchestra.run.smithi151.stdout:lab-extras 18 kB/s | 1.7 kB 00:00 2024-05-21T21:28:36.222 INFO:teuthology.orchestra.run.smithi151.stdout:Dependencies resolved. 2024-05-21T21:28:36.223 INFO:teuthology.orchestra.run.smithi151.stdout:Nothing to do. 2024-05-21T21:28:36.224 INFO:teuthology.orchestra.run.smithi151.stdout:Complete! 2024-05-21T21:28:36.328 INFO:teuthology.task.kernel:Downloading kernel 4fed890e8ce3966dbe380e9eb4b0a796eddb841c on client.0... 2024-05-21T21:28:36.328 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=4fed890e8ce3966dbe380e9eb4b0a796eddb841c 2024-05-21T21:28:36.478 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/4fed890e8ce3966dbe380e9eb4b0a796eddb841c/centos/9/flavors/default/x86_64/ 2024-05-21T21:28:36.478 DEBUG:teuthology.orchestra.run.smithi151:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g4fed890e8ce3-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://2.chacra.ceph.com/r/kernel/testing/4fed890e8ce3966dbe380e9eb4b0a796eddb841c/centos/9/flavors/default/x86_64/ --input-file=- 2024-05-21T21:28:36.481 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi151.front.sepia.ceph.com to complete... 2024-05-21T21:28:49.823 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21 21:28:49 URL:https://2.chacra.ceph.com/r/kernel/testing/4fed890e8ce3966dbe380e9eb4b0a796eddb841c/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g4fed890e8ce3-1.x86_64.rpm [357350900/357350900] -> "/tmp/kernel.x86_64.rpm" [1] 2024-05-21T21:28:49.824 INFO:teuthology.orchestra.run.smithi151.stderr:FINISHED --2024-05-21 21:28:49-- 2024-05-21T21:28:49.824 INFO:teuthology.orchestra.run.smithi151.stderr:Total wall clock time: 13s 2024-05-21T21:28:49.824 INFO:teuthology.orchestra.run.smithi151.stderr:Downloaded: 1 files, 341M in 13s (26.2 MB/s) 2024-05-21T21:28:49.826 INFO:teuthology.task.kernel:Installing kernel 4fed890e8ce3966dbe380e9eb4b0a796eddb841c on client.0... 2024-05-21T21:28:49.827 DEBUG:teuthology.orchestra.run.smithi151:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-05-21T21:28:51.957 INFO:teuthology.orchestra.run.smithi151.stdout:Verifying... ######################################## 2024-05-21T21:28:52.141 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing... ######################################## 2024-05-21T21:28:52.269 INFO:teuthology.orchestra.run.smithi151.stdout:Updating / installing... 2024-05-21T21:29:00.241 INFO:teuthology.orchestra.run.smithi151.stdout:kernel-6.9.0_rc3_g4fed890e8ce3-1 ######################################## 2024-05-21T21:29:40.129 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi151.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-05-21T21:29:40.129 DEBUG:teuthology.orchestra.run.smithi151:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-05-21T21:29:40.211 INFO:teuthology.orchestra.run.smithi151.stdout:/boot/System.map-6.9.0-rc3-g4fed890e8ce3 2024-05-21T21:29:40.211 INFO:teuthology.orchestra.run.smithi151.stdout:/boot/config-6.9.0-rc3-g4fed890e8ce3 2024-05-21T21:29:40.211 INFO:teuthology.orchestra.run.smithi151.stdout:/boot/vmlinuz-6.9.0-rc3-g4fed890e8ce3 2024-05-21T21:29:40.211 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/System.map 2024-05-21T21:29:40.211 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/config 2024-05-21T21:29:40.211 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel 2024-05-21T21:29:40.211 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch 2024-05-21T21:29:40.211 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/x86 2024-05-21T21:29:40.211 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/x86/crypto 2024-05-21T21:29:40.212 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-05-21T21:29:40.212 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-05-21T21:29:40.212 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-05-21T21:29:40.212 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-05-21T21:29:40.212 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-05-21T21:29:40.212 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-05-21T21:29:40.212 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-05-21T21:29:40.212 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/x86/crypto/crc32c-intel.ko 2024-05-21T21:29:40.212 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-05-21T21:29:40.212 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-05-21T21:29:40.212 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-05-21T21:29:40.212 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-05-21T21:29:40.212 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/x86/crypto/serpent-avx2.ko 2024-05-21T21:29:40.213 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-05-21T21:29:40.213 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-05-21T21:29:40.213 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-05-21T21:29:40.213 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-05-21T21:29:40.213 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-05-21T21:29:40.213 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-05-21T21:29:40.213 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-05-21T21:29:40.213 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/x86/kernel 2024-05-21T21:29:40.213 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/x86/kernel/cpu 2024-05-21T21:29:40.213 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/x86/kernel/cpu/mce 2024-05-21T21:29:40.213 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-05-21T21:29:40.213 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/x86/kvm 2024-05-21T21:29:40.213 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/x86/kvm/kvm-amd.ko 2024-05-21T21:29:40.213 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/x86/kvm/kvm-intel.ko 2024-05-21T21:29:40.214 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/arch/x86/kvm/kvm.ko 2024-05-21T21:29:40.214 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto 2024-05-21T21:29:40.214 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/ansi_cprng.ko 2024-05-21T21:29:40.214 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/anubis.ko 2024-05-21T21:29:40.214 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/arc4.ko 2024-05-21T21:29:40.214 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/async_tx 2024-05-21T21:29:40.214 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/async_tx/async_memcpy.ko 2024-05-21T21:29:40.214 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/async_tx/async_pq.ko 2024-05-21T21:29:40.214 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/async_tx/async_raid6_recov.ko 2024-05-21T21:29:40.214 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/async_tx/async_tx.ko 2024-05-21T21:29:40.214 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/async_tx/async_xor.ko 2024-05-21T21:29:40.214 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/async_tx/raid6test.ko 2024-05-21T21:29:40.214 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/authenc.ko 2024-05-21T21:29:40.215 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/authencesn.ko 2024-05-21T21:29:40.215 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/blake2b_generic.ko 2024-05-21T21:29:40.215 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/blowfish_common.ko 2024-05-21T21:29:40.215 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/blowfish_generic.ko 2024-05-21T21:29:40.215 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/camellia_generic.ko 2024-05-21T21:29:40.215 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/cast5_generic.ko 2024-05-21T21:29:40.215 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/cast6_generic.ko 2024-05-21T21:29:40.215 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/cast_common.ko 2024-05-21T21:29:40.215 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/ccm.ko 2024-05-21T21:29:40.215 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/cmac.ko 2024-05-21T21:29:40.215 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/crc32_generic.ko 2024-05-21T21:29:40.215 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/crypto_engine.ko 2024-05-21T21:29:40.215 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/crypto_user.ko 2024-05-21T21:29:40.216 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/cts.ko 2024-05-21T21:29:40.216 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/des_generic.ko 2024-05-21T21:29:40.216 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/dh_generic.ko 2024-05-21T21:29:40.216 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/echainiv.ko 2024-05-21T21:29:40.216 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/essiv.ko 2024-05-21T21:29:40.216 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/fcrypt.ko 2024-05-21T21:29:40.216 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/khazad.ko 2024-05-21T21:29:40.216 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/lz4.ko 2024-05-21T21:29:40.216 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/lz4hc.ko 2024-05-21T21:29:40.216 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/md4.ko 2024-05-21T21:29:40.216 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/michael_mic.ko 2024-05-21T21:29:40.216 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/pcbc.ko 2024-05-21T21:29:40.216 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/pcrypt.ko 2024-05-21T21:29:40.217 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/rmd160.ko 2024-05-21T21:29:40.217 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/seed.ko 2024-05-21T21:29:40.217 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/serpent_generic.ko 2024-05-21T21:29:40.217 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/tcrypt.ko 2024-05-21T21:29:40.217 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/tea.ko 2024-05-21T21:29:40.217 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/twofish_common.ko 2024-05-21T21:29:40.217 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/twofish_generic.ko 2024-05-21T21:29:40.217 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/vmac.ko 2024-05-21T21:29:40.217 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/wp512.ko 2024-05-21T21:29:40.217 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/xcbc.ko 2024-05-21T21:29:40.217 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/xor.ko 2024-05-21T21:29:40.217 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/crypto/xxhash_generic.ko 2024-05-21T21:29:40.217 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers 2024-05-21T21:29:40.217 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/acpi 2024-05-21T21:29:40.218 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/acpi/acpi_ipmi.ko 2024-05-21T21:29:40.218 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/acpi/acpi_pad.ko 2024-05-21T21:29:40.218 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/acpi/ec_sys.ko 2024-05-21T21:29:40.218 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/acpi/platform_profile.ko 2024-05-21T21:29:40.218 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/acpi/sbs.ko 2024-05-21T21:29:40.218 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/acpi/sbshc.ko 2024-05-21T21:29:40.218 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/acpi/video.ko 2024-05-21T21:29:40.218 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata 2024-05-21T21:29:40.218 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/acard-ahci.ko 2024-05-21T21:29:40.218 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/ahci_platform.ko 2024-05-21T21:29:40.218 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/ata_generic.ko 2024-05-21T21:29:40.218 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/libahci_platform.ko 2024-05-21T21:29:40.218 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_acpi.ko 2024-05-21T21:29:40.219 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_ali.ko 2024-05-21T21:29:40.219 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_amd.ko 2024-05-21T21:29:40.219 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_artop.ko 2024-05-21T21:29:40.219 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_atiixp.ko 2024-05-21T21:29:40.219 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_atp867x.ko 2024-05-21T21:29:40.219 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_cmd640.ko 2024-05-21T21:29:40.219 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_cmd64x.ko 2024-05-21T21:29:40.219 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_cypress.ko 2024-05-21T21:29:40.219 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_efar.ko 2024-05-21T21:29:40.219 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_hpt366.ko 2024-05-21T21:29:40.219 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_hpt37x.ko 2024-05-21T21:29:40.219 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_hpt3x2n.ko 2024-05-21T21:29:40.219 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_hpt3x3.ko 2024-05-21T21:29:40.220 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_it8213.ko 2024-05-21T21:29:40.220 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_it821x.ko 2024-05-21T21:29:40.220 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_jmicron.ko 2024-05-21T21:29:40.220 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_marvell.ko 2024-05-21T21:29:40.220 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_mpiix.ko 2024-05-21T21:29:40.220 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_netcell.ko 2024-05-21T21:29:40.220 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_ninja32.ko 2024-05-21T21:29:40.220 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_ns87410.ko 2024-05-21T21:29:40.220 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_ns87415.ko 2024-05-21T21:29:40.220 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_oldpiix.ko 2024-05-21T21:29:40.220 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_opti.ko 2024-05-21T21:29:40.220 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_optidma.ko 2024-05-21T21:29:40.220 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_pcmcia.ko 2024-05-21T21:29:40.220 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_pdc2027x.ko 2024-05-21T21:29:40.221 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-05-21T21:29:40.221 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_piccolo.ko 2024-05-21T21:29:40.221 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_rdc.ko 2024-05-21T21:29:40.221 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_sch.ko 2024-05-21T21:29:40.221 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_serverworks.ko 2024-05-21T21:29:40.221 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_sil680.ko 2024-05-21T21:29:40.221 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_sis.ko 2024-05-21T21:29:40.221 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_sl82c105.ko 2024-05-21T21:29:40.221 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_triflex.ko 2024-05-21T21:29:40.221 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pata_via.ko 2024-05-21T21:29:40.221 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/pdc_adma.ko 2024-05-21T21:29:40.221 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/sata_inic162x.ko 2024-05-21T21:29:40.221 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/sata_mv.ko 2024-05-21T21:29:40.222 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/sata_nv.ko 2024-05-21T21:29:40.222 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/sata_promise.ko 2024-05-21T21:29:40.222 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/sata_qstor.ko 2024-05-21T21:29:40.222 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/sata_sil.ko 2024-05-21T21:29:40.222 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/sata_sil24.ko 2024-05-21T21:29:40.222 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/sata_sis.ko 2024-05-21T21:29:40.222 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/sata_svw.ko 2024-05-21T21:29:40.222 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/sata_sx4.ko 2024-05-21T21:29:40.222 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/sata_uli.ko 2024-05-21T21:29:40.222 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/sata_via.ko 2024-05-21T21:29:40.222 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ata/sata_vsc.ko 2024-05-21T21:29:40.222 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/auxdisplay 2024-05-21T21:29:40.222 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/auxdisplay/ks0108.ko 2024-05-21T21:29:40.223 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/base 2024-05-21T21:29:40.223 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/base/regmap 2024-05-21T21:29:40.223 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/base/regmap/regmap-i2c.ko 2024-05-21T21:29:40.223 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/base/regmap/regmap-mmio.ko 2024-05-21T21:29:40.223 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/bcma 2024-05-21T21:29:40.223 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/bcma/bcma.ko 2024-05-21T21:29:40.223 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/block 2024-05-21T21:29:40.223 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/block/aoe 2024-05-21T21:29:40.223 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/block/aoe/aoe.ko 2024-05-21T21:29:40.223 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/block/brd.ko 2024-05-21T21:29:40.223 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/block/drbd 2024-05-21T21:29:40.223 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/block/drbd/drbd.ko 2024-05-21T21:29:40.223 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/block/floppy.ko 2024-05-21T21:29:40.223 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/block/loop.ko 2024-05-21T21:29:40.224 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/block/mtip32xx 2024-05-21T21:29:40.224 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-05-21T21:29:40.224 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/block/nbd.ko 2024-05-21T21:29:40.224 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/block/null_blk 2024-05-21T21:29:40.224 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/block/null_blk/null_blk.ko 2024-05-21T21:29:40.224 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/block/pktcdvd.ko 2024-05-21T21:29:40.224 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/block/rbd.ko 2024-05-21T21:29:40.224 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/block/virtio_blk.ko 2024-05-21T21:29:40.224 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/block/xen-blkback 2024-05-21T21:29:40.224 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-05-21T21:29:40.224 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/block/xen-blkfront.ko 2024-05-21T21:29:40.224 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/block/zram 2024-05-21T21:29:40.224 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/block/zram/zram.ko 2024-05-21T21:29:40.225 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char 2024-05-21T21:29:40.225 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/hangcheck-timer.ko 2024-05-21T21:29:40.225 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/hw_random 2024-05-21T21:29:40.225 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/hw_random/amd-rng.ko 2024-05-21T21:29:40.225 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/hw_random/intel-rng.ko 2024-05-21T21:29:40.225 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-05-21T21:29:40.225 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/hw_random/via-rng.ko 2024-05-21T21:29:40.225 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/hw_random/virtio-rng.ko 2024-05-21T21:29:40.225 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/ipmi 2024-05-21T21:29:40.225 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-05-21T21:29:40.225 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-05-21T21:29:40.225 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-05-21T21:29:40.225 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/ipmi/ipmi_si.ko 2024-05-21T21:29:40.226 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-05-21T21:29:40.226 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-05-21T21:29:40.226 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/lp.ko 2024-05-21T21:29:40.226 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/mwave 2024-05-21T21:29:40.226 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/mwave/mwave.ko 2024-05-21T21:29:40.226 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/ppdev.ko 2024-05-21T21:29:40.226 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/tlclk.ko 2024-05-21T21:29:40.226 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/tpm 2024-05-21T21:29:40.226 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/tpm/tpm.ko 2024-05-21T21:29:40.226 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/tpm/tpm_atmel.ko 2024-05-21T21:29:40.226 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/tpm/tpm_infineon.ko 2024-05-21T21:29:40.226 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/tpm/tpm_nsc.ko 2024-05-21T21:29:40.226 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/tpm/tpm_tis.ko 2024-05-21T21:29:40.227 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-05-21T21:29:40.227 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/uv_mmtimer.ko 2024-05-21T21:29:40.227 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/char/virtio_console.ko 2024-05-21T21:29:40.227 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/cpufreq 2024-05-21T21:29:40.227 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-05-21T21:29:40.227 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-05-21T21:29:40.227 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/cpufreq/p4-clockmod.ko 2024-05-21T21:29:40.227 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-05-21T21:29:40.227 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/cpufreq/powernow-k8.ko 2024-05-21T21:29:40.227 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/cpufreq/speedstep-lib.ko 2024-05-21T21:29:40.227 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/crypto 2024-05-21T21:29:40.227 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/crypto/ccp 2024-05-21T21:29:40.227 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-05-21T21:29:40.227 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/crypto/ccp/ccp.ko 2024-05-21T21:29:40.228 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/crypto/intel 2024-05-21T21:29:40.228 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/crypto/intel/qat 2024-05-21T21:29:40.228 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/crypto/intel/qat/qat_common 2024-05-21T21:29:40.228 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-05-21T21:29:40.228 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-05-21T21:29:40.228 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-05-21T21:29:40.228 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/crypto/padlock-aes.ko 2024-05-21T21:29:40.228 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/crypto/padlock-sha.ko 2024-05-21T21:29:40.228 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/crypto/virtio 2024-05-21T21:29:40.228 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-05-21T21:29:40.228 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/dca 2024-05-21T21:29:40.228 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/dca/dca.ko 2024-05-21T21:29:40.228 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/devfreq 2024-05-21T21:29:40.228 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-05-21T21:29:40.229 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/dma 2024-05-21T21:29:40.229 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/dma/dw 2024-05-21T21:29:40.229 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/dma/dw/dw_dmac.ko 2024-05-21T21:29:40.229 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/dma/ioat 2024-05-21T21:29:40.229 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/dma/ioat/ioatdma.ko 2024-05-21T21:29:40.229 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/edac 2024-05-21T21:29:40.229 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/edac/amd64_edac.ko 2024-05-21T21:29:40.229 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/edac/e752x_edac.ko 2024-05-21T21:29:40.229 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/edac/edac_mce_amd.ko 2024-05-21T21:29:40.229 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/edac/i3000_edac.ko 2024-05-21T21:29:40.229 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/edac/i3200_edac.ko 2024-05-21T21:29:40.229 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/edac/i5100_edac.ko 2024-05-21T21:29:40.229 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/edac/i5400_edac.ko 2024-05-21T21:29:40.230 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/edac/i7300_edac.ko 2024-05-21T21:29:40.230 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/edac/i7core_edac.ko 2024-05-21T21:29:40.230 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/edac/i82975x_edac.ko 2024-05-21T21:29:40.230 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/edac/ie31200_edac.ko 2024-05-21T21:29:40.230 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/edac/sb_edac.ko 2024-05-21T21:29:40.230 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/edac/x38_edac.ko 2024-05-21T21:29:40.230 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/firmware 2024-05-21T21:29:40.230 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/firmware/edd.ko 2024-05-21T21:29:40.230 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/firmware/iscsi_ibft.ko 2024-05-21T21:29:40.230 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/gpio 2024-05-21T21:29:40.230 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/gpio/gpio-ich.ko 2024-05-21T21:29:40.230 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/gpio/gpio-viperboard.ko 2024-05-21T21:29:40.230 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid 2024-05-21T21:29:40.231 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-appleir.ko 2024-05-21T21:29:40.231 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-aureal.ko 2024-05-21T21:29:40.231 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-axff.ko 2024-05-21T21:29:40.231 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-dr.ko 2024-05-21T21:29:40.231 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-elecom.ko 2024-05-21T21:29:40.231 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-elo.ko 2024-05-21T21:29:40.231 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-emsff.ko 2024-05-21T21:29:40.231 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-gaff.ko 2024-05-21T21:29:40.231 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-gt683r.ko 2024-05-21T21:29:40.231 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-gyration.ko 2024-05-21T21:29:40.231 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-holtek-kbd.ko 2024-05-21T21:29:40.231 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-holtek-mouse.ko 2024-05-21T21:29:40.231 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-holtekff.ko 2024-05-21T21:29:40.232 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-hyperv.ko 2024-05-21T21:29:40.232 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-icade.ko 2024-05-21T21:29:40.232 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-keytouch.ko 2024-05-21T21:29:40.232 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-kye.ko 2024-05-21T21:29:40.232 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-lcpower.ko 2024-05-21T21:29:40.232 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-led.ko 2024-05-21T21:29:40.232 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-lenovo.ko 2024-05-21T21:29:40.232 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-logitech-dj.ko 2024-05-21T21:29:40.232 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-05-21T21:29:40.232 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-multitouch.ko 2024-05-21T21:29:40.232 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-ortek.ko 2024-05-21T21:29:40.232 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-penmount.ko 2024-05-21T21:29:40.232 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-petalynx.ko 2024-05-21T21:29:40.232 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-picolcd.ko 2024-05-21T21:29:40.233 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-pl.ko 2024-05-21T21:29:40.233 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-plantronics.ko 2024-05-21T21:29:40.233 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-primax.ko 2024-05-21T21:29:40.233 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-rmi.ko 2024-05-21T21:29:40.233 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-roccat-arvo.ko 2024-05-21T21:29:40.233 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-roccat-common.ko 2024-05-21T21:29:40.233 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-roccat-isku.ko 2024-05-21T21:29:40.233 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-roccat-kone.ko 2024-05-21T21:29:40.233 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-05-21T21:29:40.233 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-roccat-konepure.ko 2024-05-21T21:29:40.233 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-05-21T21:29:40.233 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-roccat-lua.ko 2024-05-21T21:29:40.233 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-roccat-pyra.ko 2024-05-21T21:29:40.234 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-roccat-ryos.ko 2024-05-21T21:29:40.234 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-roccat-savu.ko 2024-05-21T21:29:40.234 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-roccat.ko 2024-05-21T21:29:40.234 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-saitek.ko 2024-05-21T21:29:40.234 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-samsung.ko 2024-05-21T21:29:40.234 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-sensor-hub.ko 2024-05-21T21:29:40.234 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-sjoy.ko 2024-05-21T21:29:40.234 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-sony.ko 2024-05-21T21:29:40.234 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-speedlink.ko 2024-05-21T21:29:40.234 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-steelseries.ko 2024-05-21T21:29:40.234 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-sunplus.ko 2024-05-21T21:29:40.234 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-thrustmaster.ko 2024-05-21T21:29:40.234 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-tivo.ko 2024-05-21T21:29:40.235 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-tmff.ko 2024-05-21T21:29:40.235 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-topseed.ko 2024-05-21T21:29:40.235 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-twinhan.ko 2024-05-21T21:29:40.235 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-uclogic.ko 2024-05-21T21:29:40.235 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-waltop.ko 2024-05-21T21:29:40.235 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-wiimote.ko 2024-05-21T21:29:40.235 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-xinmo.ko 2024-05-21T21:29:40.235 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-zpff.ko 2024-05-21T21:29:40.235 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/hid-zydacron.ko 2024-05-21T21:29:40.235 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/uhid.ko 2024-05-21T21:29:40.235 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hid/wacom.ko 2024-05-21T21:29:40.235 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hv 2024-05-21T21:29:40.235 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hv/hv_balloon.ko 2024-05-21T21:29:40.235 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hv/hv_utils.ko 2024-05-21T21:29:40.236 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hv/hv_vmbus.ko 2024-05-21T21:29:40.236 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon 2024-05-21T21:29:40.236 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/abituguru.ko 2024-05-21T21:29:40.236 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/abituguru3.ko 2024-05-21T21:29:40.236 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/acpi_power_meter.ko 2024-05-21T21:29:40.236 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/ad7414.ko 2024-05-21T21:29:40.236 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/ad7418.ko 2024-05-21T21:29:40.236 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/adc128d818.ko 2024-05-21T21:29:40.236 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/adm1025.ko 2024-05-21T21:29:40.236 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/adm1026.ko 2024-05-21T21:29:40.236 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/adm1029.ko 2024-05-21T21:29:40.236 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/adm1031.ko 2024-05-21T21:29:40.236 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/adm9240.ko 2024-05-21T21:29:40.236 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/ads7828.ko 2024-05-21T21:29:40.237 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/adt7410.ko 2024-05-21T21:29:40.237 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/adt7411.ko 2024-05-21T21:29:40.237 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/adt7462.ko 2024-05-21T21:29:40.237 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/adt7470.ko 2024-05-21T21:29:40.237 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/adt7475.ko 2024-05-21T21:29:40.237 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/adt7x10.ko 2024-05-21T21:29:40.237 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/amc6821.ko 2024-05-21T21:29:40.237 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/applesmc.ko 2024-05-21T21:29:40.237 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/asb100.ko 2024-05-21T21:29:40.237 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/asc7621.ko 2024-05-21T21:29:40.237 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/asus_atk0110.ko 2024-05-21T21:29:40.237 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/atxp1.ko 2024-05-21T21:29:40.237 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/coretemp.ko 2024-05-21T21:29:40.238 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-05-21T21:29:40.238 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/dme1737.ko 2024-05-21T21:29:40.238 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/ds1621.ko 2024-05-21T21:29:40.238 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/ds620.ko 2024-05-21T21:29:40.238 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/emc1403.ko 2024-05-21T21:29:40.238 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/emc6w201.ko 2024-05-21T21:29:40.238 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/f71805f.ko 2024-05-21T21:29:40.238 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/f71882fg.ko 2024-05-21T21:29:40.238 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/f75375s.ko 2024-05-21T21:29:40.238 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/fam15h_power.ko 2024-05-21T21:29:40.238 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/fschmd.ko 2024-05-21T21:29:40.238 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/g760a.ko 2024-05-21T21:29:40.239 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/g762.ko 2024-05-21T21:29:40.239 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/gl518sm.ko 2024-05-21T21:29:40.239 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/gl520sm.ko 2024-05-21T21:29:40.239 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/hwmon-vid.ko 2024-05-21T21:29:40.239 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/i5500_temp.ko 2024-05-21T21:29:40.239 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/i5k_amb.ko 2024-05-21T21:29:40.239 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/ibmaem.ko 2024-05-21T21:29:40.239 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/ibmpex.ko 2024-05-21T21:29:40.239 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/ina209.ko 2024-05-21T21:29:40.239 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/ina2xx.ko 2024-05-21T21:29:40.239 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/it87.ko 2024-05-21T21:29:40.239 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/k10temp.ko 2024-05-21T21:29:40.239 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/k8temp.ko 2024-05-21T21:29:40.239 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/lineage-pem.ko 2024-05-21T21:29:40.240 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/lm63.ko 2024-05-21T21:29:40.240 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/lm73.ko 2024-05-21T21:29:40.240 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/lm75.ko 2024-05-21T21:29:40.240 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/lm77.ko 2024-05-21T21:29:40.240 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/lm78.ko 2024-05-21T21:29:40.240 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/lm80.ko 2024-05-21T21:29:40.240 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/lm83.ko 2024-05-21T21:29:40.240 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/lm85.ko 2024-05-21T21:29:40.240 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/lm87.ko 2024-05-21T21:29:40.240 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/lm90.ko 2024-05-21T21:29:40.240 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/lm92.ko 2024-05-21T21:29:40.240 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/lm93.ko 2024-05-21T21:29:40.240 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/lm95234.ko 2024-05-21T21:29:40.240 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/lm95241.ko 2024-05-21T21:29:40.241 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/lm95245.ko 2024-05-21T21:29:40.241 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/ltc2945.ko 2024-05-21T21:29:40.241 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/ltc4151.ko 2024-05-21T21:29:40.241 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/ltc4215.ko 2024-05-21T21:29:40.241 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/ltc4222.ko 2024-05-21T21:29:40.241 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/ltc4245.ko 2024-05-21T21:29:40.241 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/ltc4260.ko 2024-05-21T21:29:40.241 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/ltc4261.ko 2024-05-21T21:29:40.241 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/max16065.ko 2024-05-21T21:29:40.241 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/max1619.ko 2024-05-21T21:29:40.241 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/max1668.ko 2024-05-21T21:29:40.241 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/max197.ko 2024-05-21T21:29:40.241 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/max6639.ko 2024-05-21T21:29:40.242 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/max6650.ko 2024-05-21T21:29:40.242 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/max6697.ko 2024-05-21T21:29:40.242 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/mcp3021.ko 2024-05-21T21:29:40.242 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/nct6683.ko 2024-05-21T21:29:40.242 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/nct6775-core.ko 2024-05-21T21:29:40.242 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/nct6775.ko 2024-05-21T21:29:40.242 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/nct7802.ko 2024-05-21T21:29:40.242 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/pc87360.ko 2024-05-21T21:29:40.242 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/pc87427.ko 2024-05-21T21:29:40.242 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/pcf8591.ko 2024-05-21T21:29:40.242 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/pmbus 2024-05-21T21:29:40.242 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-05-21T21:29:40.242 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-05-21T21:29:40.243 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-05-21T21:29:40.243 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/pmbus/max16064.ko 2024-05-21T21:29:40.243 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/pmbus/max34440.ko 2024-05-21T21:29:40.243 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/pmbus/max8688.ko 2024-05-21T21:29:40.243 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-05-21T21:29:40.243 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-05-21T21:29:40.243 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-05-21T21:29:40.243 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-05-21T21:29:40.243 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-05-21T21:29:40.243 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-05-21T21:29:40.243 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/powr1220.ko 2024-05-21T21:29:40.243 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/sch5627.ko 2024-05-21T21:29:40.243 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/sch5636.ko 2024-05-21T21:29:40.244 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/sch56xx-common.ko 2024-05-21T21:29:40.244 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/sht15.ko 2024-05-21T21:29:40.244 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/sht21.ko 2024-05-21T21:29:40.244 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/shtc1.ko 2024-05-21T21:29:40.244 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/sis5595.ko 2024-05-21T21:29:40.244 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/smsc47b397.ko 2024-05-21T21:29:40.244 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/smsc47m1.ko 2024-05-21T21:29:40.244 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/smsc47m192.ko 2024-05-21T21:29:40.244 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/thmc50.ko 2024-05-21T21:29:40.244 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/tmp102.ko 2024-05-21T21:29:40.244 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/tmp103.ko 2024-05-21T21:29:40.244 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/tmp401.ko 2024-05-21T21:29:40.244 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/tmp421.ko 2024-05-21T21:29:40.244 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/via-cputemp.ko 2024-05-21T21:29:40.245 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/via686a.ko 2024-05-21T21:29:40.245 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/vt1211.ko 2024-05-21T21:29:40.245 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/vt8231.ko 2024-05-21T21:29:40.245 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/w83627ehf.ko 2024-05-21T21:29:40.245 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/w83627hf.ko 2024-05-21T21:29:40.245 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/w83781d.ko 2024-05-21T21:29:40.245 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/w83791d.ko 2024-05-21T21:29:40.245 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/w83792d.ko 2024-05-21T21:29:40.245 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/w83793.ko 2024-05-21T21:29:40.245 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/w83795.ko 2024-05-21T21:29:40.245 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/w83l785ts.ko 2024-05-21T21:29:40.245 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/hwmon/w83l786ng.ko 2024-05-21T21:29:40.245 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c 2024-05-21T21:29:40.245 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/algos 2024-05-21T21:29:40.246 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-05-21T21:29:40.246 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-05-21T21:29:40.246 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses 2024-05-21T21:29:40.246 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-05-21T21:29:40.246 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-05-21T21:29:40.246 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-05-21T21:29:40.246 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-05-21T21:29:40.246 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-05-21T21:29:40.246 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-05-21T21:29:40.246 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-05-21T21:29:40.246 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-05-21T21:29:40.246 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-i801.ko 2024-05-21T21:29:40.246 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-isch.ko 2024-05-21T21:29:40.247 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-05-21T21:29:40.247 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-05-21T21:29:40.247 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-05-21T21:29:40.247 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-parport.ko 2024-05-21T21:29:40.247 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-05-21T21:29:40.247 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-05-21T21:29:40.247 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-05-21T21:29:40.247 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-05-21T21:29:40.247 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-05-21T21:29:40.247 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-05-21T21:29:40.247 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-via.ko 2024-05-21T21:29:40.247 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-05-21T21:29:40.247 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-05-21T21:29:40.247 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/i2c-dev.ko 2024-05-21T21:29:40.248 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/i2c-mux.ko 2024-05-21T21:29:40.248 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-05-21T21:29:40.248 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/i2c-smbus.ko 2024-05-21T21:29:40.248 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/i2c/i2c-stub.ko 2024-05-21T21:29:40.248 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/infiniband 2024-05-21T21:29:40.248 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/infiniband/core 2024-05-21T21:29:40.248 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/infiniband/core/ib_cm.ko 2024-05-21T21:29:40.248 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/infiniband/core/ib_core.ko 2024-05-21T21:29:40.248 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/infiniband/core/ib_umad.ko 2024-05-21T21:29:40.248 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-05-21T21:29:40.248 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/infiniband/core/iw_cm.ko 2024-05-21T21:29:40.248 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/infiniband/core/rdma_cm.ko 2024-05-21T21:29:40.248 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-05-21T21:29:40.248 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/infiniband/hw 2024-05-21T21:29:40.249 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/infiniband/hw/cxgb4 2024-05-21T21:29:40.249 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-05-21T21:29:40.249 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/infiniband/hw/mlx4 2024-05-21T21:29:40.249 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-05-21T21:29:40.249 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/infiniband/hw/mlx5 2024-05-21T21:29:40.249 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-05-21T21:29:40.249 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/infiniband/hw/mthca 2024-05-21T21:29:40.249 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-05-21T21:29:40.249 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/infiniband/ulp 2024-05-21T21:29:40.249 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/infiniband/ulp/ipoib 2024-05-21T21:29:40.249 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-05-21T21:29:40.249 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/infiniband/ulp/iser 2024-05-21T21:29:40.249 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-05-21T21:29:40.250 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/infiniband/ulp/isert 2024-05-21T21:29:40.250 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-05-21T21:29:40.250 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/infiniband/ulp/srp 2024-05-21T21:29:40.250 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-05-21T21:29:40.250 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/infiniband/ulp/srpt 2024-05-21T21:29:40.250 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-05-21T21:29:40.250 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input 2024-05-21T21:29:40.250 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/gameport 2024-05-21T21:29:40.250 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-05-21T21:29:40.250 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/gameport/fm801-gp.ko 2024-05-21T21:29:40.250 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/gameport/gameport.ko 2024-05-21T21:29:40.250 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/joydev.ko 2024-05-21T21:29:40.250 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/keyboard 2024-05-21T21:29:40.251 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/keyboard/gpio_keys.ko 2024-05-21T21:29:40.251 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/mouse 2024-05-21T21:29:40.251 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/mouse/appletouch.ko 2024-05-21T21:29:40.251 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/mouse/bcm5974.ko 2024-05-21T21:29:40.251 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/mouse/cyapatp.ko 2024-05-21T21:29:40.251 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/mouse/elan_i2c.ko 2024-05-21T21:29:40.251 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/mouse/sermouse.ko 2024-05-21T21:29:40.251 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-05-21T21:29:40.251 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/mouse/synaptics_usb.ko 2024-05-21T21:29:40.251 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/mouse/vsxxxaa.ko 2024-05-21T21:29:40.251 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/rmi4 2024-05-21T21:29:40.251 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/rmi4/rmi_core.ko 2024-05-21T21:29:40.251 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/serio 2024-05-21T21:29:40.252 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/serio/altera_ps2.ko 2024-05-21T21:29:40.252 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/serio/arc_ps2.ko 2024-05-21T21:29:40.252 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-05-21T21:29:40.252 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/serio/serio_raw.ko 2024-05-21T21:29:40.252 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/input/sparse-keymap.ko 2024-05-21T21:29:40.252 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/leds 2024-05-21T21:29:40.252 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/leds/leds-blinkm.ko 2024-05-21T21:29:40.252 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/leds/leds-lm3530.ko 2024-05-21T21:29:40.252 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/leds/leds-lp3944.ko 2024-05-21T21:29:40.252 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/leds/leds-ss4200.ko 2024-05-21T21:29:40.252 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/leds/trigger 2024-05-21T21:29:40.252 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-05-21T21:29:40.252 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-05-21T21:29:40.252 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-05-21T21:29:40.253 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-05-21T21:29:40.253 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-05-21T21:29:40.253 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-05-21T21:29:40.253 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-05-21T21:29:40.253 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-05-21T21:29:40.253 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-05-21T21:29:40.253 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/md 2024-05-21T21:29:40.253 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/md/bcache 2024-05-21T21:29:40.253 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/md/bcache/bcache.ko 2024-05-21T21:29:40.253 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/md/dm-bio-prison.ko 2024-05-21T21:29:40.253 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/md/dm-cache-smq.ko 2024-05-21T21:29:40.253 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/md/dm-cache.ko 2024-05-21T21:29:40.253 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/md/dm-crypt.ko 2024-05-21T21:29:40.254 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/md/dm-delay.ko 2024-05-21T21:29:40.254 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/md/dm-flakey.ko 2024-05-21T21:29:40.254 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/md/dm-log-userspace.ko 2024-05-21T21:29:40.254 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/md/dm-multipath.ko 2024-05-21T21:29:40.254 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/md/dm-queue-length.ko 2024-05-21T21:29:40.254 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/md/dm-raid.ko 2024-05-21T21:29:40.254 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/md/dm-round-robin.ko 2024-05-21T21:29:40.254 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/md/dm-service-time.ko 2024-05-21T21:29:40.254 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/md/dm-switch.ko 2024-05-21T21:29:40.254 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/md/dm-thin-pool.ko 2024-05-21T21:29:40.254 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/md/dm-verity.ko 2024-05-21T21:29:40.254 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/md/persistent-data 2024-05-21T21:29:40.254 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-05-21T21:29:40.255 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/md/raid0.ko 2024-05-21T21:29:40.255 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/md/raid1.ko 2024-05-21T21:29:40.255 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/md/raid10.ko 2024-05-21T21:29:40.255 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/md/raid456.ko 2024-05-21T21:29:40.255 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/message 2024-05-21T21:29:40.255 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/message/fusion 2024-05-21T21:29:40.255 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/message/fusion/mptbase.ko 2024-05-21T21:29:40.255 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/message/fusion/mptctl.ko 2024-05-21T21:29:40.255 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/message/fusion/mptfc.ko 2024-05-21T21:29:40.255 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/message/fusion/mptlan.ko 2024-05-21T21:29:40.255 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/message/fusion/mptsas.ko 2024-05-21T21:29:40.255 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/message/fusion/mptscsih.ko 2024-05-21T21:29:40.255 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/message/fusion/mptspi.ko 2024-05-21T21:29:40.255 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/mfd 2024-05-21T21:29:40.256 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/mfd/lpc_ich.ko 2024-05-21T21:29:40.256 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/mfd/lpc_sch.ko 2024-05-21T21:29:40.256 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/mfd/mfd-core.ko 2024-05-21T21:29:40.256 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/mfd/sm501.ko 2024-05-21T21:29:40.256 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/mfd/viperboard.ko 2024-05-21T21:29:40.256 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/mfd/vx855.ko 2024-05-21T21:29:40.256 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/mfd/wl1273-core.ko 2024-05-21T21:29:40.256 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc 2024-05-21T21:29:40.256 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/altera-stapl 2024-05-21T21:29:40.256 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-05-21T21:29:40.256 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/apds9802als.ko 2024-05-21T21:29:40.256 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/apds990x.ko 2024-05-21T21:29:40.256 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/bh1770glc.ko 2024-05-21T21:29:40.257 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/cb710 2024-05-21T21:29:40.257 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/cb710/cb710.ko 2024-05-21T21:29:40.257 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/eeprom 2024-05-21T21:29:40.257 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/eeprom/at24.ko 2024-05-21T21:29:40.257 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-05-21T21:29:40.257 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/eeprom/max6875.ko 2024-05-21T21:29:40.257 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/enclosure.ko 2024-05-21T21:29:40.257 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/hpilo.ko 2024-05-21T21:29:40.257 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/isl29003.ko 2024-05-21T21:29:40.257 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/isl29020.ko 2024-05-21T21:29:40.257 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/lis3lv02d 2024-05-21T21:29:40.257 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-05-21T21:29:40.257 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-05-21T21:29:40.258 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/mei 2024-05-21T21:29:40.258 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/mei/mei-me.ko 2024-05-21T21:29:40.258 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/mei/mei-txe.ko 2024-05-21T21:29:40.258 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/mei/mei.ko 2024-05-21T21:29:40.258 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/sgi-gru 2024-05-21T21:29:40.258 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/sgi-gru/gru.ko 2024-05-21T21:29:40.258 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/sgi-xp 2024-05-21T21:29:40.258 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/sgi-xp/xp.ko 2024-05-21T21:29:40.258 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/sgi-xp/xpc.ko 2024-05-21T21:29:40.258 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-05-21T21:29:40.258 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/tifm_7xx1.ko 2024-05-21T21:29:40.258 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/tifm_core.ko 2024-05-21T21:29:40.259 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/tsl2550.ko 2024-05-21T21:29:40.259 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/vmw_balloon.ko 2024-05-21T21:29:40.259 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/vmw_vmci 2024-05-21T21:29:40.259 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-05-21T21:29:40.259 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net 2024-05-21T21:29:40.259 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/bonding 2024-05-21T21:29:40.259 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/bonding/bonding.ko 2024-05-21T21:29:40.259 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/dsa 2024-05-21T21:29:40.259 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/dsa/b53 2024-05-21T21:29:40.259 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/dsa/b53/b53_common.ko 2024-05-21T21:29:40.259 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/dsa/bcm-sf2.ko 2024-05-21T21:29:40.259 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/dsa/mv88e6060.ko 2024-05-21T21:29:40.259 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/dsa/mv88e6xxx 2024-05-21T21:29:40.260 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-05-21T21:29:40.260 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/dummy.ko 2024-05-21T21:29:40.260 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/eql.ko 2024-05-21T21:29:40.260 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet 2024-05-21T21:29:40.260 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/3com 2024-05-21T21:29:40.260 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-05-21T21:29:40.260 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-05-21T21:29:40.260 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-05-21T21:29:40.260 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-05-21T21:29:40.260 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/8390 2024-05-21T21:29:40.260 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/8390/8390.ko 2024-05-21T21:29:40.260 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-05-21T21:29:40.260 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-05-21T21:29:40.260 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-05-21T21:29:40.261 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/adaptec 2024-05-21T21:29:40.261 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-05-21T21:29:40.261 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/agere 2024-05-21T21:29:40.261 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/agere/et131x.ko 2024-05-21T21:29:40.261 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/alteon 2024-05-21T21:29:40.261 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-05-21T21:29:40.261 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/altera 2024-05-21T21:29:40.261 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-05-21T21:29:40.261 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/amd 2024-05-21T21:29:40.261 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-05-21T21:29:40.261 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-05-21T21:29:40.261 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-05-21T21:29:40.261 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/atheros 2024-05-21T21:29:40.262 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/atheros/alx 2024-05-21T21:29:40.262 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-05-21T21:29:40.262 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/atheros/atl1c 2024-05-21T21:29:40.262 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-05-21T21:29:40.262 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/atheros/atl1e 2024-05-21T21:29:40.262 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-05-21T21:29:40.262 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/atheros/atlx 2024-05-21T21:29:40.262 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-05-21T21:29:40.262 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-05-21T21:29:40.262 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/broadcom 2024-05-21T21:29:40.262 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-05-21T21:29:40.262 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-05-21T21:29:40.262 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-05-21T21:29:40.263 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-05-21T21:29:40.263 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-05-21T21:29:40.263 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/broadcom/genet 2024-05-21T21:29:40.263 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-05-21T21:29:40.263 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-05-21T21:29:40.263 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/brocade 2024-05-21T21:29:40.263 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/brocade/bna 2024-05-21T21:29:40.263 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-05-21T21:29:40.263 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/chelsio 2024-05-21T21:29:40.263 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/chelsio/cxgb 2024-05-21T21:29:40.263 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-05-21T21:29:40.263 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-05-21T21:29:40.263 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-05-21T21:29:40.264 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-05-21T21:29:40.264 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-05-21T21:29:40.264 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-05-21T21:29:40.264 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-05-21T21:29:40.264 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-05-21T21:29:40.264 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-05-21T21:29:40.264 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/cisco 2024-05-21T21:29:40.264 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/cisco/enic 2024-05-21T21:29:40.264 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-05-21T21:29:40.264 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/dec 2024-05-21T21:29:40.264 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/dec/tulip 2024-05-21T21:29:40.264 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-05-21T21:29:40.264 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-05-21T21:29:40.265 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-05-21T21:29:40.265 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-05-21T21:29:40.265 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-05-21T21:29:40.265 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-05-21T21:29:40.265 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/dlink 2024-05-21T21:29:40.265 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-05-21T21:29:40.265 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-05-21T21:29:40.265 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/dnet.ko 2024-05-21T21:29:40.265 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/emulex 2024-05-21T21:29:40.265 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/emulex/benet 2024-05-21T21:29:40.265 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-05-21T21:29:40.265 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/ethoc.ko 2024-05-21T21:29:40.265 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/fealnx.ko 2024-05-21T21:29:40.266 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/intel 2024-05-21T21:29:40.266 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/intel/e100.ko 2024-05-21T21:29:40.266 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/intel/e1000 2024-05-21T21:29:40.266 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-05-21T21:29:40.266 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/intel/e1000e 2024-05-21T21:29:40.266 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-05-21T21:29:40.266 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/intel/fm10k 2024-05-21T21:29:40.266 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-05-21T21:29:40.266 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/intel/i40e 2024-05-21T21:29:40.266 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-05-21T21:29:40.266 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/intel/iavf 2024-05-21T21:29:40.266 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-05-21T21:29:40.266 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/intel/igb 2024-05-21T21:29:40.267 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-05-21T21:29:40.267 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/intel/igbvf 2024-05-21T21:29:40.267 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-05-21T21:29:40.267 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/intel/ixgbe 2024-05-21T21:29:40.267 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-05-21T21:29:40.267 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/intel/ixgbevf 2024-05-21T21:29:40.267 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-05-21T21:29:40.267 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/jme.ko 2024-05-21T21:29:40.267 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/marvell 2024-05-21T21:29:40.267 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-05-21T21:29:40.267 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/marvell/skge.ko 2024-05-21T21:29:40.267 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-05-21T21:29:40.267 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/mellanox 2024-05-21T21:29:40.268 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/mellanox/mlx4 2024-05-21T21:29:40.268 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-05-21T21:29:40.268 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-05-21T21:29:40.268 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/mellanox/mlx5 2024-05-21T21:29:40.268 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-05-21T21:29:40.268 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-05-21T21:29:40.268 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/micrel 2024-05-21T21:29:40.268 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-05-21T21:29:40.268 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/myricom 2024-05-21T21:29:40.268 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/myricom/myri10ge 2024-05-21T21:29:40.268 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-05-21T21:29:40.268 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/natsemi 2024-05-21T21:29:40.268 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-05-21T21:29:40.268 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-05-21T21:29:40.269 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/neterion 2024-05-21T21:29:40.269 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-05-21T21:29:40.269 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/nvidia 2024-05-21T21:29:40.269 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-05-21T21:29:40.269 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/packetengines 2024-05-21T21:29:40.269 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-05-21T21:29:40.269 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-05-21T21:29:40.269 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/qlogic 2024-05-21T21:29:40.269 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/qlogic/netxen 2024-05-21T21:29:40.269 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-05-21T21:29:40.269 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-05-21T21:29:40.269 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-05-21T21:29:40.269 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-05-21T21:29:40.270 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/rdc 2024-05-21T21:29:40.270 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-05-21T21:29:40.270 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/realtek 2024-05-21T21:29:40.270 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-05-21T21:29:40.270 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-05-21T21:29:40.270 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/realtek/atp.ko 2024-05-21T21:29:40.270 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-05-21T21:29:40.270 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/rocker 2024-05-21T21:29:40.270 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-05-21T21:29:40.270 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/sfc 2024-05-21T21:29:40.270 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-05-21T21:29:40.270 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/silan 2024-05-21T21:29:40.270 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-05-21T21:29:40.271 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/sis 2024-05-21T21:29:40.271 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/sis/sis190.ko 2024-05-21T21:29:40.271 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/sis/sis900.ko 2024-05-21T21:29:40.271 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/smsc 2024-05-21T21:29:40.271 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-05-21T21:29:40.271 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-05-21T21:29:40.271 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-05-21T21:29:40.271 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-05-21T21:29:40.271 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/stmicro 2024-05-21T21:29:40.271 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/stmicro/stmmac 2024-05-21T21:29:40.271 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-05-21T21:29:40.271 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-05-21T21:29:40.271 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/sun 2024-05-21T21:29:40.272 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/sun/cassini.ko 2024-05-21T21:29:40.272 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/sun/niu.ko 2024-05-21T21:29:40.272 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/sun/sungem.ko 2024-05-21T21:29:40.272 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-05-21T21:29:40.272 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/tehuti 2024-05-21T21:29:40.272 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-05-21T21:29:40.272 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/ti 2024-05-21T21:29:40.272 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/ti/tlan.ko 2024-05-21T21:29:40.272 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/via 2024-05-21T21:29:40.272 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-05-21T21:29:40.272 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-05-21T21:29:40.272 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/wiznet 2024-05-21T21:29:40.272 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-05-21T21:29:40.273 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-05-21T21:29:40.273 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/xircom 2024-05-21T21:29:40.273 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-05-21T21:29:40.273 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/geneve.ko 2024-05-21T21:29:40.273 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/hyperv 2024-05-21T21:29:40.273 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-05-21T21:29:40.273 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ieee802154 2024-05-21T21:29:40.273 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ieee802154/fakelb.ko 2024-05-21T21:29:40.273 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ifb.ko 2024-05-21T21:29:40.273 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ipvlan 2024-05-21T21:29:40.273 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ipvlan/ipvlan.ko 2024-05-21T21:29:40.273 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/macvlan.ko 2024-05-21T21:29:40.273 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/macvtap.ko 2024-05-21T21:29:40.273 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/mdio 2024-05-21T21:29:40.274 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/mdio.ko 2024-05-21T21:29:40.274 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-05-21T21:29:40.274 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-05-21T21:29:40.274 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/mdio/mdio-regmap.ko 2024-05-21T21:29:40.274 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/mii.ko 2024-05-21T21:29:40.274 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/net_failover.ko 2024-05-21T21:29:40.274 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/netconsole.ko 2024-05-21T21:29:40.274 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/nlmon.ko 2024-05-21T21:29:40.274 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/pcs 2024-05-21T21:29:40.274 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/pcs/pcs-lynx.ko 2024-05-21T21:29:40.274 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-05-21T21:29:40.274 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/phy 2024-05-21T21:29:40.275 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/phy/amd.ko 2024-05-21T21:29:40.275 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-05-21T21:29:40.275 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-05-21T21:29:40.275 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/phy/bcm7xxx.ko 2024-05-21T21:29:40.275 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/phy/bcm87xx.ko 2024-05-21T21:29:40.275 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/phy/broadcom.ko 2024-05-21T21:29:40.275 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/phy/cicada.ko 2024-05-21T21:29:40.275 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/phy/davicom.ko 2024-05-21T21:29:40.275 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/phy/dp83640.ko 2024-05-21T21:29:40.275 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/phy/et1011c.ko 2024-05-21T21:29:40.275 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/phy/icplus.ko 2024-05-21T21:29:40.275 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/phy/lxt.ko 2024-05-21T21:29:40.275 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/phy/marvell.ko 2024-05-21T21:29:40.276 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/phy/micrel.ko 2024-05-21T21:29:40.276 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/phy/national.ko 2024-05-21T21:29:40.276 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/phy/phylink.ko 2024-05-21T21:29:40.276 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/phy/qsemi.ko 2024-05-21T21:29:40.276 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/phy/realtek.ko 2024-05-21T21:29:40.276 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/phy/smsc.ko 2024-05-21T21:29:40.276 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/phy/ste10Xp.ko 2024-05-21T21:29:40.276 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/phy/vitesse.ko 2024-05-21T21:29:40.276 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ppp 2024-05-21T21:29:40.276 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ppp/bsd_comp.ko 2024-05-21T21:29:40.276 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ppp/ppp_async.ko 2024-05-21T21:29:40.276 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ppp/ppp_deflate.ko 2024-05-21T21:29:40.276 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ppp/ppp_generic.ko 2024-05-21T21:29:40.276 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ppp/ppp_mppe.ko 2024-05-21T21:29:40.277 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ppp/ppp_synctty.ko 2024-05-21T21:29:40.277 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ppp/pppoe.ko 2024-05-21T21:29:40.277 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ppp/pppox.ko 2024-05-21T21:29:40.277 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/ppp/pptp.ko 2024-05-21T21:29:40.277 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/slip 2024-05-21T21:29:40.277 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/slip/slhc.ko 2024-05-21T21:29:40.277 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/slip/slip.ko 2024-05-21T21:29:40.277 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/sungem_phy.ko 2024-05-21T21:29:40.277 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/tap.ko 2024-05-21T21:29:40.277 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/team 2024-05-21T21:29:40.277 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/team/team.ko 2024-05-21T21:29:40.277 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/team/team_mode_activebackup.ko 2024-05-21T21:29:40.277 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/team/team_mode_broadcast.ko 2024-05-21T21:29:40.278 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-05-21T21:29:40.278 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/team/team_mode_random.ko 2024-05-21T21:29:40.278 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-05-21T21:29:40.278 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/tun.ko 2024-05-21T21:29:40.278 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/veth.ko 2024-05-21T21:29:40.278 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/virtio_net.ko 2024-05-21T21:29:40.278 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/vmxnet3 2024-05-21T21:29:40.278 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-05-21T21:29:40.278 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/vxlan 2024-05-21T21:29:40.278 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/vxlan/vxlan.ko 2024-05-21T21:29:40.278 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/xen-netback 2024-05-21T21:29:40.278 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/xen-netback/xen-netback.ko 2024-05-21T21:29:40.278 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/net/xen-netfront.ko 2024-05-21T21:29:40.279 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ntb 2024-05-21T21:29:40.279 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ntb/ntb.ko 2024-05-21T21:29:40.279 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/nvme 2024-05-21T21:29:40.279 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/nvme/host 2024-05-21T21:29:40.279 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/nvme/host/nvme-core.ko 2024-05-21T21:29:40.279 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/nvme/host/nvme.ko 2024-05-21T21:29:40.279 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/parport 2024-05-21T21:29:40.279 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/parport/parport.ko 2024-05-21T21:29:40.279 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/parport/parport_cs.ko 2024-05-21T21:29:40.279 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/parport/parport_pc.ko 2024-05-21T21:29:40.279 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/parport/parport_serial.ko 2024-05-21T21:29:40.279 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/pci 2024-05-21T21:29:40.279 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/pci/hotplug 2024-05-21T21:29:40.280 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-05-21T21:29:40.280 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/pci/pcie 2024-05-21T21:29:40.280 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/pci/pcie/aer_inject.ko 2024-05-21T21:29:40.280 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/pci/xen-pcifront.ko 2024-05-21T21:29:40.280 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/pcmcia 2024-05-21T21:29:40.280 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/pcmcia/i82092.ko 2024-05-21T21:29:40.280 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/pcmcia/pd6729.ko 2024-05-21T21:29:40.280 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/pcmcia/yenta_socket.ko 2024-05-21T21:29:40.280 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/pinctrl 2024-05-21T21:29:40.280 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/pinctrl/intel 2024-05-21T21:29:40.280 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-05-21T21:29:40.280 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform 2024-05-21T21:29:40.280 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/x86 2024-05-21T21:29:40.280 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/x86/acer-wmi.ko 2024-05-21T21:29:40.281 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/x86/acerhdf.ko 2024-05-21T21:29:40.281 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/x86/apple-gmux.ko 2024-05-21T21:29:40.281 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/x86/asus-laptop.ko 2024-05-21T21:29:40.281 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-05-21T21:29:40.281 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/x86/asus-wmi.ko 2024-05-21T21:29:40.281 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/x86/classmate-laptop.ko 2024-05-21T21:29:40.281 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-05-21T21:29:40.281 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-05-21T21:29:40.281 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-05-21T21:29:40.281 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-05-21T21:29:40.281 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/x86/hdaps.ko 2024-05-21T21:29:40.281 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/x86/intel 2024-05-21T21:29:40.281 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-05-21T21:29:40.282 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/x86/intel_ips.ko 2024-05-21T21:29:40.282 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/x86/msi-wmi.ko 2024-05-21T21:29:40.282 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/x86/mxm-wmi.ko 2024-05-21T21:29:40.282 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-05-21T21:29:40.282 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/x86/samsung-laptop.ko 2024-05-21T21:29:40.282 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/x86/samsung-q10.ko 2024-05-21T21:29:40.282 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/x86/topstar-laptop.ko 2024-05-21T21:29:40.282 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-05-21T21:29:40.282 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/x86/toshiba_haps.ko 2024-05-21T21:29:40.282 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/x86/wmi-bmof.ko 2024-05-21T21:29:40.282 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/platform/x86/wmi.ko 2024-05-21T21:29:40.282 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/powercap 2024-05-21T21:29:40.282 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/powercap/intel_rapl_common.ko 2024-05-21T21:29:40.283 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/powercap/intel_rapl_msr.ko 2024-05-21T21:29:40.283 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/pps 2024-05-21T21:29:40.283 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/pps/clients 2024-05-21T21:29:40.283 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/pps/clients/pps-gpio.ko 2024-05-21T21:29:40.283 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/pps/clients/pps-ldisc.ko 2024-05-21T21:29:40.283 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/pps/clients/pps_parport.ko 2024-05-21T21:29:40.283 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/pps/pps_core.ko 2024-05-21T21:29:40.283 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ptp 2024-05-21T21:29:40.283 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ptp/ptp.ko 2024-05-21T21:29:40.283 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ptp/ptp_kvm.ko 2024-05-21T21:29:40.283 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ras 2024-05-21T21:29:40.283 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ras/amd 2024-05-21T21:29:40.283 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ras/amd/atl 2024-05-21T21:29:40.283 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-05-21T21:29:40.284 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ras/amd/fmpm.ko 2024-05-21T21:29:40.284 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc 2024-05-21T21:29:40.284 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-bq32k.ko 2024-05-21T21:29:40.284 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-ds1286.ko 2024-05-21T21:29:40.284 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-ds1307.ko 2024-05-21T21:29:40.284 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-ds1374.ko 2024-05-21T21:29:40.284 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-ds1511.ko 2024-05-21T21:29:40.284 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-ds1553.ko 2024-05-21T21:29:40.284 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-ds1672.ko 2024-05-21T21:29:40.284 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-ds1742.ko 2024-05-21T21:29:40.284 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-ds2404.ko 2024-05-21T21:29:40.284 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-ds3232.ko 2024-05-21T21:29:40.284 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-em3027.ko 2024-05-21T21:29:40.285 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-fm3130.ko 2024-05-21T21:29:40.285 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-isl12022.ko 2024-05-21T21:29:40.285 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-isl1208.ko 2024-05-21T21:29:40.285 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-m41t80.ko 2024-05-21T21:29:40.285 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-m48t35.ko 2024-05-21T21:29:40.285 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-m48t59.ko 2024-05-21T21:29:40.285 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-max6900.ko 2024-05-21T21:29:40.285 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-msm6242.ko 2024-05-21T21:29:40.285 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-pcf2127.ko 2024-05-21T21:29:40.285 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-pcf85063.ko 2024-05-21T21:29:40.285 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-pcf8523.ko 2024-05-21T21:29:40.285 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-pcf8563.ko 2024-05-21T21:29:40.285 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-pcf8583.ko 2024-05-21T21:29:40.286 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-rp5c01.ko 2024-05-21T21:29:40.286 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-rs5c372.ko 2024-05-21T21:29:40.286 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-05-21T21:29:40.286 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-rx8025.ko 2024-05-21T21:29:40.286 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-rx8581.ko 2024-05-21T21:29:40.286 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-05-21T21:29:40.286 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/rtc/rtc-x1205.ko 2024-05-21T21:29:40.286 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi 2024-05-21T21:29:40.286 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/3w-9xxx.ko 2024-05-21T21:29:40.286 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/3w-sas.ko 2024-05-21T21:29:40.286 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/3w-xxxx.ko 2024-05-21T21:29:40.286 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/BusLogic.ko 2024-05-21T21:29:40.286 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/a100u2w.ko 2024-05-21T21:29:40.287 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/aacraid 2024-05-21T21:29:40.287 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/aacraid/aacraid.ko 2024-05-21T21:29:40.287 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/advansys.ko 2024-05-21T21:29:40.287 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/aic7xxx 2024-05-21T21:29:40.287 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-05-21T21:29:40.287 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-05-21T21:29:40.287 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/am53c974.ko 2024-05-21T21:29:40.287 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/arcmsr 2024-05-21T21:29:40.287 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-05-21T21:29:40.287 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/atp870u.ko 2024-05-21T21:29:40.287 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/be2iscsi 2024-05-21T21:29:40.287 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-05-21T21:29:40.287 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/bfa 2024-05-21T21:29:40.287 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/bfa/bfa.ko 2024-05-21T21:29:40.288 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/bnx2fc 2024-05-21T21:29:40.288 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-05-21T21:29:40.288 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/bnx2i 2024-05-21T21:29:40.288 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-05-21T21:29:40.288 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/ch.ko 2024-05-21T21:29:40.288 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/csiostor 2024-05-21T21:29:40.288 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/csiostor/csiostor.ko 2024-05-21T21:29:40.288 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/cxgbi 2024-05-21T21:29:40.288 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/cxgbi/cxgb3i 2024-05-21T21:29:40.288 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-05-21T21:29:40.288 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/cxgbi/cxgb4i 2024-05-21T21:29:40.288 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-05-21T21:29:40.288 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-05-21T21:29:40.289 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/dc395x.ko 2024-05-21T21:29:40.289 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/device_handler 2024-05-21T21:29:40.289 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-05-21T21:29:40.289 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-05-21T21:29:40.289 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-05-21T21:29:40.289 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-05-21T21:29:40.289 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/dmx3191d.ko 2024-05-21T21:29:40.289 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/esas2r 2024-05-21T21:29:40.289 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/esas2r/esas2r.ko 2024-05-21T21:29:40.289 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/esp_scsi.ko 2024-05-21T21:29:40.289 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/fcoe 2024-05-21T21:29:40.289 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/fcoe/fcoe.ko 2024-05-21T21:29:40.289 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-05-21T21:29:40.289 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/fnic 2024-05-21T21:29:40.289 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/fnic/fnic.ko 2024-05-21T21:29:40.290 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/hpsa.ko 2024-05-21T21:29:40.290 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/hptiop.ko 2024-05-21T21:29:40.290 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/hv_storvsc.ko 2024-05-21T21:29:40.290 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/initio.ko 2024-05-21T21:29:40.290 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/ipr.ko 2024-05-21T21:29:40.290 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/ips.ko 2024-05-21T21:29:40.290 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/isci 2024-05-21T21:29:40.290 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/isci/isci.ko 2024-05-21T21:29:40.290 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-05-21T21:29:40.290 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/iscsi_tcp.ko 2024-05-21T21:29:40.290 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/libfc 2024-05-21T21:29:40.290 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/libfc/libfc.ko 2024-05-21T21:29:40.290 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/libiscsi.ko 2024-05-21T21:29:40.291 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/libiscsi_tcp.ko 2024-05-21T21:29:40.291 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/libsas 2024-05-21T21:29:40.291 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/libsas/libsas.ko 2024-05-21T21:29:40.291 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/lpfc 2024-05-21T21:29:40.291 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/lpfc/lpfc.ko 2024-05-21T21:29:40.291 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/megaraid 2024-05-21T21:29:40.291 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/megaraid.ko 2024-05-21T21:29:40.291 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-05-21T21:29:40.291 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-05-21T21:29:40.291 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-05-21T21:29:40.291 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/mpt3sas 2024-05-21T21:29:40.291 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-05-21T21:29:40.291 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/mvsas 2024-05-21T21:29:40.291 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/mvsas/mvsas.ko 2024-05-21T21:29:40.292 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/mvumi.ko 2024-05-21T21:29:40.292 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/pm8001 2024-05-21T21:29:40.292 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-05-21T21:29:40.292 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/pmcraid.ko 2024-05-21T21:29:40.292 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/qla1280.ko 2024-05-21T21:29:40.292 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/qla2xxx 2024-05-21T21:29:40.292 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-05-21T21:29:40.292 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-05-21T21:29:40.292 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/qla4xxx 2024-05-21T21:29:40.292 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-05-21T21:29:40.292 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/raid_class.ko 2024-05-21T21:29:40.292 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/scsi_debug.ko 2024-05-21T21:29:40.292 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/scsi_transport_fc.ko 2024-05-21T21:29:40.293 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-05-21T21:29:40.293 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/scsi_transport_sas.ko 2024-05-21T21:29:40.293 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/scsi_transport_spi.ko 2024-05-21T21:29:40.293 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/scsi_transport_srp.ko 2024-05-21T21:29:40.293 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/ses.ko 2024-05-21T21:29:40.293 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/st.ko 2024-05-21T21:29:40.293 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/stex.ko 2024-05-21T21:29:40.293 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/sym53c8xx_2 2024-05-21T21:29:40.293 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-05-21T21:29:40.293 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/virtio_scsi.ko 2024-05-21T21:29:40.293 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/vmw_pvscsi.ko 2024-05-21T21:29:40.293 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/scsi/wd719x.ko 2024-05-21T21:29:40.293 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ssb 2024-05-21T21:29:40.293 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ssb/ssb.ko 2024-05-21T21:29:40.293 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/target 2024-05-21T21:29:40.294 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/target/iscsi 2024-05-21T21:29:40.294 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-05-21T21:29:40.294 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/target/loopback 2024-05-21T21:29:40.294 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/target/loopback/tcm_loop.ko 2024-05-21T21:29:40.294 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/target/target_core_file.ko 2024-05-21T21:29:40.294 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/target/target_core_iblock.ko 2024-05-21T21:29:40.294 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/target/target_core_mod.ko 2024-05-21T21:29:40.294 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/target/target_core_pscsi.ko 2024-05-21T21:29:40.294 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/target/target_core_user.ko 2024-05-21T21:29:40.294 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/target/tcm_fc 2024-05-21T21:29:40.294 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-05-21T21:29:40.294 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/thermal 2024-05-21T21:29:40.294 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/thermal/intel 2024-05-21T21:29:40.295 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/thermal/intel/int340x_thermal 2024-05-21T21:29:40.295 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-05-21T21:29:40.295 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-05-21T21:29:40.295 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-05-21T21:29:40.295 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-05-21T21:29:40.295 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-05-21T21:29:40.295 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-05-21T21:29:40.295 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-05-21T21:29:40.295 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-05-21T21:29:40.295 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-05-21T21:29:40.295 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-05-21T21:29:40.295 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-05-21T21:29:40.295 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-05-21T21:29:40.295 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-05-21T21:29:40.296 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-05-21T21:29:40.296 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-05-21T21:29:40.296 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-05-21T21:29:40.296 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-05-21T21:29:40.296 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-05-21T21:29:40.296 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/tty 2024-05-21T21:29:40.296 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/tty/ipwireless 2024-05-21T21:29:40.296 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-05-21T21:29:40.296 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/tty/n_gsm.ko 2024-05-21T21:29:40.296 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/tty/n_hdlc.ko 2024-05-21T21:29:40.296 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/tty/nozomi.ko 2024-05-21T21:29:40.296 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/tty/serial 2024-05-21T21:29:40.296 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/tty/serial/8250 2024-05-21T21:29:40.297 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-05-21T21:29:40.297 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/tty/serial/arc_uart.ko 2024-05-21T21:29:40.297 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/tty/serial/jsm 2024-05-21T21:29:40.297 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/tty/serial/jsm/jsm.ko 2024-05-21T21:29:40.297 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ufs 2024-05-21T21:29:40.297 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ufs/core 2024-05-21T21:29:40.297 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ufs/core/ufshcd-core.ko 2024-05-21T21:29:40.297 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ufs/host 2024-05-21T21:29:40.297 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-05-21T21:29:40.297 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/uio 2024-05-21T21:29:40.297 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/uio/uio.ko 2024-05-21T21:29:40.297 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/uio/uio_aec.ko 2024-05-21T21:29:40.297 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/uio/uio_cif.ko 2024-05-21T21:29:40.297 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/uio/uio_pci_generic.ko 2024-05-21T21:29:40.298 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/uio/uio_sercos3.ko 2024-05-21T21:29:40.298 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb 2024-05-21T21:29:40.298 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/class 2024-05-21T21:29:40.298 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/class/cdc-acm.ko 2024-05-21T21:29:40.298 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/class/cdc-wdm.ko 2024-05-21T21:29:40.298 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/class/usblp.ko 2024-05-21T21:29:40.298 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/class/usbtmc.ko 2024-05-21T21:29:40.298 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/host 2024-05-21T21:29:40.298 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/host/sl811-hcd.ko 2024-05-21T21:29:40.298 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/image 2024-05-21T21:29:40.298 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/image/mdc800.ko 2024-05-21T21:29:40.298 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/image/microtek.ko 2024-05-21T21:29:40.298 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/misc 2024-05-21T21:29:40.299 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/misc/adutux.ko 2024-05-21T21:29:40.299 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/misc/appledisplay.ko 2024-05-21T21:29:40.299 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/misc/emi26.ko 2024-05-21T21:29:40.299 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/misc/emi62.ko 2024-05-21T21:29:40.299 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/misc/ezusb.ko 2024-05-21T21:29:40.299 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/misc/idmouse.ko 2024-05-21T21:29:40.299 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/misc/iowarrior.ko 2024-05-21T21:29:40.299 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/misc/isight_firmware.ko 2024-05-21T21:29:40.299 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/misc/ldusb.ko 2024-05-21T21:29:40.299 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/misc/legousbtower.ko 2024-05-21T21:29:40.299 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/misc/sisusbvga 2024-05-21T21:29:40.299 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-05-21T21:29:40.299 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/misc/trancevibrator.ko 2024-05-21T21:29:40.299 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/misc/usb3503.ko 2024-05-21T21:29:40.300 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/misc/usblcd.ko 2024-05-21T21:29:40.300 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/misc/usbsevseg.ko 2024-05-21T21:29:40.300 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/misc/uss720.ko 2024-05-21T21:29:40.300 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/misc/yurex.ko 2024-05-21T21:29:40.300 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/phy 2024-05-21T21:29:40.300 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/phy/phy-generic.ko 2024-05-21T21:29:40.300 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial 2024-05-21T21:29:40.300 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/aircable.ko 2024-05-21T21:29:40.300 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/ark3116.ko 2024-05-21T21:29:40.300 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/belkin_sa.ko 2024-05-21T21:29:40.300 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/ch341.ko 2024-05-21T21:29:40.300 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/cp210x.ko 2024-05-21T21:29:40.300 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/cyberjack.ko 2024-05-21T21:29:40.300 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/cypress_m8.ko 2024-05-21T21:29:40.301 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/digi_acceleport.ko 2024-05-21T21:29:40.301 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/empeg.ko 2024-05-21T21:29:40.301 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/ftdi_sio.ko 2024-05-21T21:29:40.301 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/garmin_gps.ko 2024-05-21T21:29:40.301 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/io_edgeport.ko 2024-05-21T21:29:40.301 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/io_ti.ko 2024-05-21T21:29:40.301 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/ipaq.ko 2024-05-21T21:29:40.301 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/ipw.ko 2024-05-21T21:29:40.301 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/ir-usb.ko 2024-05-21T21:29:40.301 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-05-21T21:29:40.301 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/keyspan.ko 2024-05-21T21:29:40.301 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/keyspan_pda.ko 2024-05-21T21:29:40.301 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/kl5kusb105.ko 2024-05-21T21:29:40.301 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/kobil_sct.ko 2024-05-21T21:29:40.302 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/mct_u232.ko 2024-05-21T21:29:40.302 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/mos7720.ko 2024-05-21T21:29:40.302 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/mos7840.ko 2024-05-21T21:29:40.302 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/navman.ko 2024-05-21T21:29:40.302 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/omninet.ko 2024-05-21T21:29:40.302 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/opticon.ko 2024-05-21T21:29:40.302 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/option.ko 2024-05-21T21:29:40.302 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/oti6858.ko 2024-05-21T21:29:40.302 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/pl2303.ko 2024-05-21T21:29:40.302 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/qcaux.ko 2024-05-21T21:29:40.302 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/qcserial.ko 2024-05-21T21:29:40.302 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/quatech2.ko 2024-05-21T21:29:40.302 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/safe_serial.ko 2024-05-21T21:29:40.303 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/sierra.ko 2024-05-21T21:29:40.303 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/spcp8x5.ko 2024-05-21T21:29:40.303 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/ssu100.ko 2024-05-21T21:29:40.303 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/symbolserial.ko 2024-05-21T21:29:40.303 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-05-21T21:29:40.303 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-05-21T21:29:40.303 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/usb_debug.ko 2024-05-21T21:29:40.303 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/usb_wwan.ko 2024-05-21T21:29:40.303 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/visor.ko 2024-05-21T21:29:40.303 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/whiteheat.ko 2024-05-21T21:29:40.303 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/serial/xsens_mt.ko 2024-05-21T21:29:40.303 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/storage 2024-05-21T21:29:40.303 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/storage/uas.ko 2024-05-21T21:29:40.303 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/storage/ums-alauda.ko 2024-05-21T21:29:40.303 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/storage/ums-cypress.ko 2024-05-21T21:29:40.304 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/storage/ums-datafab.ko 2024-05-21T21:29:40.304 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-05-21T21:29:40.304 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/storage/ums-freecom.ko 2024-05-21T21:29:40.304 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/storage/ums-isd200.ko 2024-05-21T21:29:40.304 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-05-21T21:29:40.304 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/storage/ums-karma.ko 2024-05-21T21:29:40.304 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/storage/ums-onetouch.ko 2024-05-21T21:29:40.304 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/storage/ums-realtek.ko 2024-05-21T21:29:40.304 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/storage/ums-sddr09.ko 2024-05-21T21:29:40.304 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/storage/ums-sddr55.ko 2024-05-21T21:29:40.304 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/storage/ums-usbat.ko 2024-05-21T21:29:40.304 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/storage/usb-storage.ko 2024-05-21T21:29:40.305 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/usbip 2024-05-21T21:29:40.305 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/usbip/usbip-core.ko 2024-05-21T21:29:40.305 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/usbip/usbip-host.ko 2024-05-21T21:29:40.305 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-05-21T21:29:40.305 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/vfio 2024-05-21T21:29:40.305 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/vfio/pci 2024-05-21T21:29:40.305 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-05-21T21:29:40.305 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/vfio/pci/vfio-pci.ko 2024-05-21T21:29:40.305 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/vfio/vfio.ko 2024-05-21T21:29:40.305 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-05-21T21:29:40.305 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/vhost 2024-05-21T21:29:40.305 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/vhost/vhost.ko 2024-05-21T21:29:40.305 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/vhost/vhost_iotlb.ko 2024-05-21T21:29:40.306 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/vhost/vhost_net.ko 2024-05-21T21:29:40.306 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/vhost/vhost_scsi.ko 2024-05-21T21:29:40.306 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/video 2024-05-21T21:29:40.306 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/video/backlight 2024-05-21T21:29:40.306 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/video/backlight/apple_bl.ko 2024-05-21T21:29:40.306 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/video/backlight/lcd.ko 2024-05-21T21:29:40.306 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/video/backlight/platform_lcd.ko 2024-05-21T21:29:40.306 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/virtio 2024-05-21T21:29:40.306 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/virtio/virtio.ko 2024-05-21T21:29:40.306 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/virtio/virtio_balloon.ko 2024-05-21T21:29:40.306 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/virtio/virtio_mmio.ko 2024-05-21T21:29:40.306 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/virtio/virtio_pci.ko 2024-05-21T21:29:40.306 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-05-21T21:29:40.306 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-05-21T21:29:40.307 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/virtio/virtio_ring.ko 2024-05-21T21:29:40.307 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/w1 2024-05-21T21:29:40.307 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/w1/masters 2024-05-21T21:29:40.307 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/w1/masters/ds2482.ko 2024-05-21T21:29:40.307 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/w1/masters/ds2490.ko 2024-05-21T21:29:40.307 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/w1/slaves 2024-05-21T21:29:40.307 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-05-21T21:29:40.307 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-05-21T21:29:40.307 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-05-21T21:29:40.307 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-05-21T21:29:40.307 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-05-21T21:29:40.307 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-05-21T21:29:40.307 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-05-21T21:29:40.308 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-05-21T21:29:40.308 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-05-21T21:29:40.308 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/w1/slaves/w1_smem.ko 2024-05-21T21:29:40.308 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/w1/slaves/w1_therm.ko 2024-05-21T21:29:40.308 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/w1/wire.ko 2024-05-21T21:29:40.308 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog 2024-05-21T21:29:40.308 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/alim1535_wdt.ko 2024-05-21T21:29:40.308 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/alim7101_wdt.ko 2024-05-21T21:29:40.308 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/f71808e_wdt.ko 2024-05-21T21:29:40.308 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/hpwdt.ko 2024-05-21T21:29:40.308 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/i6300esb.ko 2024-05-21T21:29:40.308 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-05-21T21:29:40.308 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/iTCO_wdt.ko 2024-05-21T21:29:40.308 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/ib700wdt.ko 2024-05-21T21:29:40.309 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/ibmasr.ko 2024-05-21T21:29:40.309 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-05-21T21:29:40.309 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/it8712f_wdt.ko 2024-05-21T21:29:40.309 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/it87_wdt.ko 2024-05-21T21:29:40.309 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/machzwd.ko 2024-05-21T21:29:40.309 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/nv_tco.ko 2024-05-21T21:29:40.309 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/pcwd_pci.ko 2024-05-21T21:29:40.309 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/pcwd_usb.ko 2024-05-21T21:29:40.309 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-05-21T21:29:40.309 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/sch311x_wdt.ko 2024-05-21T21:29:40.309 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/softdog.ko 2024-05-21T21:29:40.309 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/sp5100_tco.ko 2024-05-21T21:29:40.309 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/via_wdt.ko 2024-05-21T21:29:40.310 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-05-21T21:29:40.310 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/w83877f_wdt.ko 2024-05-21T21:29:40.310 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/w83977f_wdt.ko 2024-05-21T21:29:40.310 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/wdt_pci.ko 2024-05-21T21:29:40.310 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/watchdog/xen_wdt.ko 2024-05-21T21:29:40.310 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/xen 2024-05-21T21:29:40.310 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/xen/xen-acpi-processor.ko 2024-05-21T21:29:40.310 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/xen/xen-evtchn.ko 2024-05-21T21:29:40.310 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/xen/xen-gntalloc.ko 2024-05-21T21:29:40.310 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/xen/xen-gntdev.ko 2024-05-21T21:29:40.310 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/xen/xen-pciback 2024-05-21T21:29:40.310 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-05-21T21:29:40.310 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/xen/xen-privcmd.ko 2024-05-21T21:29:40.310 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/xen/xenfs 2024-05-21T21:29:40.311 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/drivers/xen/xenfs/xenfs.ko 2024-05-21T21:29:40.311 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs 2024-05-21T21:29:40.311 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/9p 2024-05-21T21:29:40.311 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/9p/9p.ko 2024-05-21T21:29:40.311 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/affs 2024-05-21T21:29:40.311 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/affs/affs.ko 2024-05-21T21:29:40.311 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/befs 2024-05-21T21:29:40.311 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/befs/befs.ko 2024-05-21T21:29:40.311 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/binfmt_misc.ko 2024-05-21T21:29:40.311 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/btrfs 2024-05-21T21:29:40.311 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/btrfs/btrfs.ko 2024-05-21T21:29:40.311 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/cachefiles 2024-05-21T21:29:40.311 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/cachefiles/cachefiles.ko 2024-05-21T21:29:40.312 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/ceph 2024-05-21T21:29:40.312 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/ceph/ceph.ko 2024-05-21T21:29:40.312 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/coda 2024-05-21T21:29:40.312 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/coda/coda.ko 2024-05-21T21:29:40.312 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/cramfs 2024-05-21T21:29:40.312 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/cramfs/cramfs.ko 2024-05-21T21:29:40.312 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/dlm 2024-05-21T21:29:40.312 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/dlm/dlm.ko 2024-05-21T21:29:40.312 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/ecryptfs 2024-05-21T21:29:40.312 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/ecryptfs/ecryptfs.ko 2024-05-21T21:29:40.312 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/f2fs 2024-05-21T21:29:40.312 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/f2fs/f2fs.ko 2024-05-21T21:29:40.312 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/fat 2024-05-21T21:29:40.313 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/fat/fat.ko 2024-05-21T21:29:40.313 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/fat/msdos.ko 2024-05-21T21:29:40.313 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/fat/vfat.ko 2024-05-21T21:29:40.313 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/fuse 2024-05-21T21:29:40.313 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/fuse/cuse.ko 2024-05-21T21:29:40.314 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/fuse/fuse.ko 2024-05-21T21:29:40.314 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/gfs2 2024-05-21T21:29:40.314 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/gfs2/gfs2.ko 2024-05-21T21:29:40.314 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/hfs 2024-05-21T21:29:40.314 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/hfs/hfs.ko 2024-05-21T21:29:40.314 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/hfsplus 2024-05-21T21:29:40.314 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/hfsplus/hfsplus.ko 2024-05-21T21:29:40.314 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/isofs 2024-05-21T21:29:40.314 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/isofs/isofs.ko 2024-05-21T21:29:40.314 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/jfs 2024-05-21T21:29:40.314 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/jfs/jfs.ko 2024-05-21T21:29:40.314 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/lockd 2024-05-21T21:29:40.315 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/lockd/lockd.ko 2024-05-21T21:29:40.315 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/minix 2024-05-21T21:29:40.315 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/minix/minix.ko 2024-05-21T21:29:40.315 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/netfs 2024-05-21T21:29:40.315 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/netfs/netfs.ko 2024-05-21T21:29:40.315 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nfs 2024-05-21T21:29:40.315 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nfs/blocklayout 2024-05-21T21:29:40.315 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-05-21T21:29:40.315 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nfs/filelayout 2024-05-21T21:29:40.315 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-05-21T21:29:40.315 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nfs/flexfilelayout 2024-05-21T21:29:40.315 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-05-21T21:29:40.315 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nfs/nfs.ko 2024-05-21T21:29:40.316 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nfs/nfsv3.ko 2024-05-21T21:29:40.316 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nfs/nfsv4.ko 2024-05-21T21:29:40.316 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nfs_common 2024-05-21T21:29:40.316 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nfs_common/grace.ko 2024-05-21T21:29:40.316 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nfs_common/nfs_acl.ko 2024-05-21T21:29:40.316 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nfsd 2024-05-21T21:29:40.316 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nfsd/nfsd.ko 2024-05-21T21:29:40.316 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nilfs2 2024-05-21T21:29:40.316 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nilfs2/nilfs2.ko 2024-05-21T21:29:40.316 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls 2024-05-21T21:29:40.316 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/mac-celtic.ko 2024-05-21T21:29:40.316 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/mac-centeuro.ko 2024-05-21T21:29:40.316 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/mac-croatian.ko 2024-05-21T21:29:40.317 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/mac-cyrillic.ko 2024-05-21T21:29:40.317 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/mac-gaelic.ko 2024-05-21T21:29:40.317 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/mac-greek.ko 2024-05-21T21:29:40.317 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/mac-iceland.ko 2024-05-21T21:29:40.317 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/mac-inuit.ko 2024-05-21T21:29:40.317 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/mac-roman.ko 2024-05-21T21:29:40.317 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/mac-romanian.ko 2024-05-21T21:29:40.317 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/mac-turkish.ko 2024-05-21T21:29:40.317 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_cp1250.ko 2024-05-21T21:29:40.317 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_cp1251.ko 2024-05-21T21:29:40.317 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_cp1255.ko 2024-05-21T21:29:40.317 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_cp737.ko 2024-05-21T21:29:40.317 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_cp775.ko 2024-05-21T21:29:40.317 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_cp850.ko 2024-05-21T21:29:40.318 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_cp852.ko 2024-05-21T21:29:40.318 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_cp855.ko 2024-05-21T21:29:40.318 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_cp857.ko 2024-05-21T21:29:40.318 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_cp860.ko 2024-05-21T21:29:40.318 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_cp861.ko 2024-05-21T21:29:40.318 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_cp862.ko 2024-05-21T21:29:40.318 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_cp863.ko 2024-05-21T21:29:40.318 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_cp864.ko 2024-05-21T21:29:40.318 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_cp865.ko 2024-05-21T21:29:40.318 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_cp866.ko 2024-05-21T21:29:40.318 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_cp869.ko 2024-05-21T21:29:40.318 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_cp874.ko 2024-05-21T21:29:40.318 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_cp932.ko 2024-05-21T21:29:40.319 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_cp936.ko 2024-05-21T21:29:40.319 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_cp949.ko 2024-05-21T21:29:40.319 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_cp950.ko 2024-05-21T21:29:40.319 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_euc-jp.ko 2024-05-21T21:29:40.319 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_iso8859-1.ko 2024-05-21T21:29:40.319 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_iso8859-13.ko 2024-05-21T21:29:40.319 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_iso8859-14.ko 2024-05-21T21:29:40.319 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_iso8859-15.ko 2024-05-21T21:29:40.319 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_iso8859-2.ko 2024-05-21T21:29:40.319 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_iso8859-3.ko 2024-05-21T21:29:40.319 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_iso8859-4.ko 2024-05-21T21:29:40.319 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_iso8859-5.ko 2024-05-21T21:29:40.319 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_iso8859-6.ko 2024-05-21T21:29:40.320 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_iso8859-7.ko 2024-05-21T21:29:40.320 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_iso8859-9.ko 2024-05-21T21:29:40.320 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_koi8-r.ko 2024-05-21T21:29:40.320 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_koi8-ru.ko 2024-05-21T21:29:40.320 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_koi8-u.ko 2024-05-21T21:29:40.320 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_ucs2_utils.ko 2024-05-21T21:29:40.320 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/nls/nls_utf8.ko 2024-05-21T21:29:40.320 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/ocfs2 2024-05-21T21:29:40.320 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/ocfs2/cluster 2024-05-21T21:29:40.320 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-05-21T21:29:40.320 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/ocfs2/dlm 2024-05-21T21:29:40.320 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-05-21T21:29:40.320 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/ocfs2/dlmfs 2024-05-21T21:29:40.320 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-05-21T21:29:40.321 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/ocfs2/ocfs2.ko 2024-05-21T21:29:40.321 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-05-21T21:29:40.321 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-05-21T21:29:40.321 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-05-21T21:29:40.321 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/overlayfs 2024-05-21T21:29:40.321 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/overlayfs/overlay.ko 2024-05-21T21:29:40.321 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/pstore 2024-05-21T21:29:40.321 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/pstore/ramoops.ko 2024-05-21T21:29:40.321 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/reiserfs 2024-05-21T21:29:40.321 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/reiserfs/reiserfs.ko 2024-05-21T21:29:40.321 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/romfs 2024-05-21T21:29:40.321 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/romfs/romfs.ko 2024-05-21T21:29:40.321 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/smb 2024-05-21T21:29:40.322 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/smb/client 2024-05-21T21:29:40.322 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/smb/client/cifs.ko 2024-05-21T21:29:40.322 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/smb/common 2024-05-21T21:29:40.322 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/smb/common/cifs_arc4.ko 2024-05-21T21:29:40.322 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/smb/common/cifs_md4.ko 2024-05-21T21:29:40.322 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/squashfs 2024-05-21T21:29:40.322 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/squashfs/squashfs.ko 2024-05-21T21:29:40.322 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/sysv 2024-05-21T21:29:40.322 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/sysv/sysv.ko 2024-05-21T21:29:40.322 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/udf 2024-05-21T21:29:40.322 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/udf/udf.ko 2024-05-21T21:29:40.322 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/ufs 2024-05-21T21:29:40.322 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/ufs/ufs.ko 2024-05-21T21:29:40.323 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/xfs 2024-05-21T21:29:40.323 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/fs/xfs/xfs.ko 2024-05-21T21:29:40.323 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/kernel 2024-05-21T21:29:40.323 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/kernel/rcu 2024-05-21T21:29:40.323 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/kernel/rcu/rcutorture.ko 2024-05-21T21:29:40.323 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/kernel/torture.ko 2024-05-21T21:29:40.323 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/kernel/trace 2024-05-21T21:29:40.323 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-05-21T21:29:40.323 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/lib 2024-05-21T21:29:40.323 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/lib/asn1_encoder.ko 2024-05-21T21:29:40.323 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/lib/crc-itu-t.ko 2024-05-21T21:29:40.323 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/lib/crc8.ko 2024-05-21T21:29:40.323 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/lib/crypto 2024-05-21T21:29:40.324 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/lib/crypto/libarc4.ko 2024-05-21T21:29:40.324 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/lib/crypto/libdes.ko 2024-05-21T21:29:40.324 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/lib/libcrc32c.ko 2024-05-21T21:29:40.324 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/lib/lru_cache.ko 2024-05-21T21:29:40.324 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/lib/lz4 2024-05-21T21:29:40.324 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/lib/lz4/lz4_compress.ko 2024-05-21T21:29:40.324 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/lib/lz4/lz4hc_compress.ko 2024-05-21T21:29:40.324 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/lib/math 2024-05-21T21:29:40.324 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/lib/math/cordic.ko 2024-05-21T21:29:40.324 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/lib/raid6 2024-05-21T21:29:40.324 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/lib/raid6/raid6_pq.ko 2024-05-21T21:29:40.324 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/lib/reed_solomon 2024-05-21T21:29:40.324 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/lib/reed_solomon/reed_solomon.ko 2024-05-21T21:29:40.324 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/lib/ts_bm.ko 2024-05-21T21:29:40.325 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/lib/ts_fsm.ko 2024-05-21T21:29:40.325 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/lib/ts_kmp.ko 2024-05-21T21:29:40.325 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/lib/zstd 2024-05-21T21:29:40.325 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/lib/zstd/zstd_compress.ko 2024-05-21T21:29:40.325 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/mm 2024-05-21T21:29:40.325 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/mm/hwpoison-inject.ko 2024-05-21T21:29:40.325 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net 2024-05-21T21:29:40.325 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/6lowpan 2024-05-21T21:29:40.325 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/6lowpan/6lowpan.ko 2024-05-21T21:29:40.325 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/6lowpan/nhc_dest.ko 2024-05-21T21:29:40.325 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/6lowpan/nhc_fragment.ko 2024-05-21T21:29:40.325 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/6lowpan/nhc_hop.ko 2024-05-21T21:29:40.325 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/6lowpan/nhc_ipv6.ko 2024-05-21T21:29:40.326 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/6lowpan/nhc_mobility.ko 2024-05-21T21:29:40.326 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/6lowpan/nhc_routing.ko 2024-05-21T21:29:40.326 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/6lowpan/nhc_udp.ko 2024-05-21T21:29:40.326 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/802 2024-05-21T21:29:40.326 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/802/garp.ko 2024-05-21T21:29:40.326 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/802/mrp.ko 2024-05-21T21:29:40.326 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/802/p8022.ko 2024-05-21T21:29:40.326 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/802/psnap.ko 2024-05-21T21:29:40.326 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/802/stp.ko 2024-05-21T21:29:40.326 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/8021q 2024-05-21T21:29:40.326 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/8021q/8021q.ko 2024-05-21T21:29:40.326 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/9p 2024-05-21T21:29:40.326 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/9p/9pnet.ko 2024-05-21T21:29:40.327 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/9p/9pnet_fd.ko 2024-05-21T21:29:40.327 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/9p/9pnet_rdma.ko 2024-05-21T21:29:40.327 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/9p/9pnet_virtio.ko 2024-05-21T21:29:40.327 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/batman-adv 2024-05-21T21:29:40.327 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/batman-adv/batman-adv.ko 2024-05-21T21:29:40.327 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/bridge 2024-05-21T21:29:40.327 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/bridge/br_netfilter.ko 2024-05-21T21:29:40.327 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/bridge/bridge.ko 2024-05-21T21:29:40.327 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/bridge/netfilter 2024-05-21T21:29:40.327 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-05-21T21:29:40.327 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/bridge/netfilter/ebt_among.ko 2024-05-21T21:29:40.327 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/bridge/netfilter/ebt_arp.ko 2024-05-21T21:29:40.327 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-05-21T21:29:40.328 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-05-21T21:29:40.328 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/bridge/netfilter/ebt_ip.ko 2024-05-21T21:29:40.328 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-05-21T21:29:40.328 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/bridge/netfilter/ebt_limit.ko 2024-05-21T21:29:40.328 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/bridge/netfilter/ebt_log.ko 2024-05-21T21:29:40.328 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/bridge/netfilter/ebt_mark.ko 2024-05-21T21:29:40.328 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-05-21T21:29:40.328 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-05-21T21:29:40.328 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-05-21T21:29:40.328 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-05-21T21:29:40.328 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/bridge/netfilter/ebt_snat.ko 2024-05-21T21:29:40.328 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/bridge/netfilter/ebt_stp.ko 2024-05-21T21:29:40.328 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-05-21T21:29:40.329 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-05-21T21:29:40.329 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-05-21T21:29:40.329 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-05-21T21:29:40.329 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/bridge/netfilter/ebtables.ko 2024-05-21T21:29:40.329 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-05-21T21:29:40.329 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-05-21T21:29:40.329 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-05-21T21:29:40.329 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ceph 2024-05-21T21:29:40.329 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ceph/libceph.ko 2024-05-21T21:29:40.329 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/core 2024-05-21T21:29:40.329 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/core/failover.ko 2024-05-21T21:29:40.329 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/core/pktgen.ko 2024-05-21T21:29:40.329 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/dccp 2024-05-21T21:29:40.330 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/dccp/dccp.ko 2024-05-21T21:29:40.330 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/dccp/dccp_diag.ko 2024-05-21T21:29:40.330 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/dccp/dccp_ipv4.ko 2024-05-21T21:29:40.330 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/dccp/dccp_ipv6.ko 2024-05-21T21:29:40.330 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/dns_resolver 2024-05-21T21:29:40.330 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/dns_resolver/dns_resolver.ko 2024-05-21T21:29:40.330 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/dsa 2024-05-21T21:29:40.330 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/dsa/dsa_core.ko 2024-05-21T21:29:40.330 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/dsa/tag_brcm.ko 2024-05-21T21:29:40.330 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/dsa/tag_dsa.ko 2024-05-21T21:29:40.330 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/dsa/tag_none.ko 2024-05-21T21:29:40.330 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/dsa/tag_trailer.ko 2024-05-21T21:29:40.330 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ieee802154 2024-05-21T21:29:40.330 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ieee802154/6lowpan 2024-05-21T21:29:40.331 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-05-21T21:29:40.331 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ieee802154/ieee802154.ko 2024-05-21T21:29:40.331 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ieee802154/ieee802154_socket.ko 2024-05-21T21:29:40.331 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4 2024-05-21T21:29:40.331 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/ah4.ko 2024-05-21T21:29:40.331 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/esp4.ko 2024-05-21T21:29:40.331 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/fou.ko 2024-05-21T21:29:40.331 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/gre.ko 2024-05-21T21:29:40.331 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/inet_diag.ko 2024-05-21T21:29:40.331 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/ip_gre.ko 2024-05-21T21:29:40.331 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/ip_tunnel.ko 2024-05-21T21:29:40.331 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/ip_vti.ko 2024-05-21T21:29:40.331 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/ipcomp.ko 2024-05-21T21:29:40.332 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/ipip.ko 2024-05-21T21:29:40.332 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/netfilter 2024-05-21T21:29:40.332 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/netfilter/arp_tables.ko 2024-05-21T21:29:40.332 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-05-21T21:29:40.332 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-05-21T21:29:40.332 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-05-21T21:29:40.332 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-05-21T21:29:40.332 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-05-21T21:29:40.332 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-05-21T21:29:40.332 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-05-21T21:29:40.332 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-05-21T21:29:40.332 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-05-21T21:29:40.332 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/netfilter/iptable_security.ko 2024-05-21T21:29:40.332 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-05-21T21:29:40.333 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-05-21T21:29:40.333 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-05-21T21:29:40.333 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-05-21T21:29:40.333 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-05-21T21:29:40.333 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-05-21T21:29:40.333 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-05-21T21:29:40.333 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-05-21T21:29:40.333 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-05-21T21:29:40.333 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-05-21T21:29:40.333 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/tcp_bic.ko 2024-05-21T21:29:40.333 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/tcp_dctcp.ko 2024-05-21T21:29:40.333 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/tcp_diag.ko 2024-05-21T21:29:40.333 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/tcp_highspeed.ko 2024-05-21T21:29:40.333 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/tcp_htcp.ko 2024-05-21T21:29:40.334 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/tcp_hybla.ko 2024-05-21T21:29:40.334 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/tcp_illinois.ko 2024-05-21T21:29:40.334 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/tcp_lp.ko 2024-05-21T21:29:40.334 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/tcp_scalable.ko 2024-05-21T21:29:40.334 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/tcp_vegas.ko 2024-05-21T21:29:40.334 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/tcp_veno.ko 2024-05-21T21:29:40.334 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/tcp_westwood.ko 2024-05-21T21:29:40.334 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/tcp_yeah.ko 2024-05-21T21:29:40.334 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/tunnel4.ko 2024-05-21T21:29:40.334 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/udp_diag.ko 2024-05-21T21:29:40.334 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/udp_tunnel.ko 2024-05-21T21:29:40.334 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv4/xfrm4_tunnel.ko 2024-05-21T21:29:40.335 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6 2024-05-21T21:29:40.335 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/ah6.ko 2024-05-21T21:29:40.335 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/esp6.ko 2024-05-21T21:29:40.335 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/fou6.ko 2024-05-21T21:29:40.335 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/ip6_tunnel.ko 2024-05-21T21:29:40.335 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-05-21T21:29:40.335 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/ip6_vti.ko 2024-05-21T21:29:40.335 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/ipcomp6.ko 2024-05-21T21:29:40.335 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/netfilter 2024-05-21T21:29:40.335 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-05-21T21:29:40.335 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-05-21T21:29:40.335 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-05-21T21:29:40.335 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-05-21T21:29:40.335 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-05-21T21:29:40.336 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-05-21T21:29:40.336 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-05-21T21:29:40.336 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-05-21T21:29:40.336 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-05-21T21:29:40.336 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-05-21T21:29:40.336 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-05-21T21:29:40.336 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-05-21T21:29:40.336 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-05-21T21:29:40.336 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-05-21T21:29:40.336 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-05-21T21:29:40.336 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-05-21T21:29:40.336 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-05-21T21:29:40.336 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-05-21T21:29:40.336 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-05-21T21:29:40.337 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-05-21T21:29:40.337 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-05-21T21:29:40.337 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-05-21T21:29:40.337 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-05-21T21:29:40.337 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-05-21T21:29:40.337 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/sit.ko 2024-05-21T21:29:40.337 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/tunnel6.ko 2024-05-21T21:29:40.337 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/ipv6/xfrm6_tunnel.ko 2024-05-21T21:29:40.337 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/key 2024-05-21T21:29:40.337 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/key/af_key.ko 2024-05-21T21:29:40.337 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/l2tp 2024-05-21T21:29:40.337 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/l2tp/l2tp_core.ko 2024-05-21T21:29:40.337 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/l2tp/l2tp_debugfs.ko 2024-05-21T21:29:40.337 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/l2tp/l2tp_eth.ko 2024-05-21T21:29:40.338 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/l2tp/l2tp_ip.ko 2024-05-21T21:29:40.338 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/l2tp/l2tp_ip6.ko 2024-05-21T21:29:40.338 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/l2tp/l2tp_netlink.ko 2024-05-21T21:29:40.338 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/l2tp/l2tp_ppp.ko 2024-05-21T21:29:40.338 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/llc 2024-05-21T21:29:40.338 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/llc/llc.ko 2024-05-21T21:29:40.338 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/mac802154 2024-05-21T21:29:40.338 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/mac802154/mac802154.ko 2024-05-21T21:29:40.338 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/mpls 2024-05-21T21:29:40.338 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/mpls/mpls_gso.ko 2024-05-21T21:29:40.338 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter 2024-05-21T21:29:40.338 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipset 2024-05-21T21:29:40.339 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipset/ip_set.ko 2024-05-21T21:29:40.339 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-05-21T21:29:40.339 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-05-21T21:29:40.339 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-05-21T21:29:40.339 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-05-21T21:29:40.339 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-05-21T21:29:40.339 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-05-21T21:29:40.339 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-05-21T21:29:40.339 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-05-21T21:29:40.339 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-05-21T21:29:40.339 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-05-21T21:29:40.339 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-05-21T21:29:40.339 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-05-21T21:29:40.340 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-05-21T21:29:40.340 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-05-21T21:29:40.340 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-05-21T21:29:40.340 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipvs 2024-05-21T21:29:40.340 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipvs/ip_vs.ko 2024-05-21T21:29:40.340 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-05-21T21:29:40.340 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-05-21T21:29:40.340 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-05-21T21:29:40.340 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-05-21T21:29:40.340 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-05-21T21:29:40.340 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-05-21T21:29:40.340 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-05-21T21:29:40.340 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-05-21T21:29:40.340 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-05-21T21:29:40.341 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-05-21T21:29:40.341 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-05-21T21:29:40.341 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-05-21T21:29:40.341 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-05-21T21:29:40.341 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nf_conncount.ko 2024-05-21T21:29:40.341 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nf_conntrack.ko 2024-05-21T21:29:40.341 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-05-21T21:29:40.341 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-05-21T21:29:40.341 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-05-21T21:29:40.341 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nf_conntrack_h323.ko 2024-05-21T21:29:40.341 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nf_conntrack_irc.ko 2024-05-21T21:29:40.341 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-05-21T21:29:40.341 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-05-21T21:29:40.342 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-05-21T21:29:40.342 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nf_conntrack_sane.ko 2024-05-21T21:29:40.342 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nf_conntrack_sip.ko 2024-05-21T21:29:40.342 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-05-21T21:29:40.342 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-05-21T21:29:40.342 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nf_dup_netdev.ko 2024-05-21T21:29:40.342 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nf_flow_table.ko 2024-05-21T21:29:40.342 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nf_flow_table_inet.ko 2024-05-21T21:29:40.342 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nf_log_syslog.ko 2024-05-21T21:29:40.342 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nf_nat.ko 2024-05-21T21:29:40.342 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nf_nat_amanda.ko 2024-05-21T21:29:40.342 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nf_nat_ftp.ko 2024-05-21T21:29:40.343 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nf_nat_irc.ko 2024-05-21T21:29:40.343 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nf_nat_sip.ko 2024-05-21T21:29:40.343 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nf_nat_tftp.ko 2024-05-21T21:29:40.343 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nf_synproxy_core.ko 2024-05-21T21:29:40.343 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nf_tables.ko 2024-05-21T21:29:40.343 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nfnetlink.ko 2024-05-21T21:29:40.343 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nfnetlink_acct.ko 2024-05-21T21:29:40.343 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nfnetlink_log.ko 2024-05-21T21:29:40.343 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nfnetlink_osf.ko 2024-05-21T21:29:40.343 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nfnetlink_queue.ko 2024-05-21T21:29:40.343 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nft_chain_nat.ko 2024-05-21T21:29:40.343 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nft_compat.ko 2024-05-21T21:29:40.343 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nft_ct.ko 2024-05-21T21:29:40.344 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nft_dup_netdev.ko 2024-05-21T21:29:40.344 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nft_fib.ko 2024-05-21T21:29:40.344 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nft_fwd_netdev.ko 2024-05-21T21:29:40.344 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nft_hash.ko 2024-05-21T21:29:40.344 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nft_limit.ko 2024-05-21T21:29:40.344 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nft_log.ko 2024-05-21T21:29:40.344 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nft_masq.ko 2024-05-21T21:29:40.344 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nft_nat.ko 2024-05-21T21:29:40.344 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nft_numgen.ko 2024-05-21T21:29:40.344 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nft_queue.ko 2024-05-21T21:29:40.344 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nft_quota.ko 2024-05-21T21:29:40.344 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nft_redir.ko 2024-05-21T21:29:40.344 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nft_reject.ko 2024-05-21T21:29:40.344 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nft_reject_inet.ko 2024-05-21T21:29:40.345 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nft_socket.ko 2024-05-21T21:29:40.345 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/nft_tunnel.ko 2024-05-21T21:29:40.345 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_AUDIT.ko 2024-05-21T21:29:40.345 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_CHECKSUM.ko 2024-05-21T21:29:40.345 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_CLASSIFY.ko 2024-05-21T21:29:40.345 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-05-21T21:29:40.345 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_CT.ko 2024-05-21T21:29:40.345 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_DSCP.ko 2024-05-21T21:29:40.345 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_HL.ko 2024-05-21T21:29:40.345 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_HMARK.ko 2024-05-21T21:29:40.345 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_IDLETIMER.ko 2024-05-21T21:29:40.345 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_LED.ko 2024-05-21T21:29:40.345 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_LOG.ko 2024-05-21T21:29:40.345 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_MASQUERADE.ko 2024-05-21T21:29:40.346 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_NETMAP.ko 2024-05-21T21:29:40.346 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_NFLOG.ko 2024-05-21T21:29:40.346 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_NFQUEUE.ko 2024-05-21T21:29:40.346 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_RATEEST.ko 2024-05-21T21:29:40.346 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_REDIRECT.ko 2024-05-21T21:29:40.346 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_SECMARK.ko 2024-05-21T21:29:40.346 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_TCPMSS.ko 2024-05-21T21:29:40.346 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-05-21T21:29:40.346 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_TEE.ko 2024-05-21T21:29:40.346 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_TPROXY.ko 2024-05-21T21:29:40.346 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_TRACE.ko 2024-05-21T21:29:40.346 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_addrtype.ko 2024-05-21T21:29:40.346 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_bpf.ko 2024-05-21T21:29:40.346 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_cgroup.ko 2024-05-21T21:29:40.347 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_cluster.ko 2024-05-21T21:29:40.347 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_comment.ko 2024-05-21T21:29:40.347 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_connbytes.ko 2024-05-21T21:29:40.347 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_connlabel.ko 2024-05-21T21:29:40.347 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_connlimit.ko 2024-05-21T21:29:40.347 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_connmark.ko 2024-05-21T21:29:40.347 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_conntrack.ko 2024-05-21T21:29:40.347 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_cpu.ko 2024-05-21T21:29:40.347 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_dccp.ko 2024-05-21T21:29:40.347 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_devgroup.ko 2024-05-21T21:29:40.347 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_dscp.ko 2024-05-21T21:29:40.347 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_ecn.ko 2024-05-21T21:29:40.347 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_esp.ko 2024-05-21T21:29:40.348 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_hashlimit.ko 2024-05-21T21:29:40.348 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_helper.ko 2024-05-21T21:29:40.348 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_hl.ko 2024-05-21T21:29:40.348 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_ipcomp.ko 2024-05-21T21:29:40.348 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_iprange.ko 2024-05-21T21:29:40.348 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_ipvs.ko 2024-05-21T21:29:40.348 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_l2tp.ko 2024-05-21T21:29:40.348 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_length.ko 2024-05-21T21:29:40.348 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_limit.ko 2024-05-21T21:29:40.348 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_mac.ko 2024-05-21T21:29:40.348 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_mark.ko 2024-05-21T21:29:40.348 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_multiport.ko 2024-05-21T21:29:40.348 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_nat.ko 2024-05-21T21:29:40.348 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_nfacct.ko 2024-05-21T21:29:40.349 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_osf.ko 2024-05-21T21:29:40.349 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_owner.ko 2024-05-21T21:29:40.349 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_physdev.ko 2024-05-21T21:29:40.349 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_pkttype.ko 2024-05-21T21:29:40.349 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_policy.ko 2024-05-21T21:29:40.349 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_quota.ko 2024-05-21T21:29:40.349 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_rateest.ko 2024-05-21T21:29:40.349 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_realm.ko 2024-05-21T21:29:40.349 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_recent.ko 2024-05-21T21:29:40.349 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_sctp.ko 2024-05-21T21:29:40.349 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_set.ko 2024-05-21T21:29:40.349 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_socket.ko 2024-05-21T21:29:40.349 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_state.ko 2024-05-21T21:29:40.350 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_statistic.ko 2024-05-21T21:29:40.350 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_string.ko 2024-05-21T21:29:40.350 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_tcpmss.ko 2024-05-21T21:29:40.350 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_time.ko 2024-05-21T21:29:40.350 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netfilter/xt_u32.ko 2024-05-21T21:29:40.350 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netlink 2024-05-21T21:29:40.350 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/netlink/netlink_diag.ko 2024-05-21T21:29:40.350 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/nsh 2024-05-21T21:29:40.350 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/nsh/nsh.ko 2024-05-21T21:29:40.350 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/openvswitch 2024-05-21T21:29:40.350 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/openvswitch/openvswitch.ko 2024-05-21T21:29:40.350 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/openvswitch/vport-geneve.ko 2024-05-21T21:29:40.350 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/openvswitch/vport-gre.ko 2024-05-21T21:29:40.350 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/openvswitch/vport-vxlan.ko 2024-05-21T21:29:40.351 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/packet 2024-05-21T21:29:40.351 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/packet/af_packet_diag.ko 2024-05-21T21:29:40.351 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/rds 2024-05-21T21:29:40.351 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/rds/rds.ko 2024-05-21T21:29:40.351 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/rds/rds_rdma.ko 2024-05-21T21:29:40.351 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/rds/rds_tcp.ko 2024-05-21T21:29:40.351 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched 2024-05-21T21:29:40.351 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/act_csum.ko 2024-05-21T21:29:40.351 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/act_gact.ko 2024-05-21T21:29:40.351 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/act_mirred.ko 2024-05-21T21:29:40.351 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/act_nat.ko 2024-05-21T21:29:40.351 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/act_pedit.ko 2024-05-21T21:29:40.351 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/act_police.ko 2024-05-21T21:29:40.352 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/act_simple.ko 2024-05-21T21:29:40.352 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/act_skbedit.ko 2024-05-21T21:29:40.352 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/act_vlan.ko 2024-05-21T21:29:40.352 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/cls_basic.ko 2024-05-21T21:29:40.352 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/cls_bpf.ko 2024-05-21T21:29:40.352 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/cls_flow.ko 2024-05-21T21:29:40.352 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/cls_fw.ko 2024-05-21T21:29:40.352 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/cls_route.ko 2024-05-21T21:29:40.352 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/cls_u32.ko 2024-05-21T21:29:40.352 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/em_cmp.ko 2024-05-21T21:29:40.352 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/em_ipset.ko 2024-05-21T21:29:40.352 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/em_meta.ko 2024-05-21T21:29:40.352 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/em_nbyte.ko 2024-05-21T21:29:40.352 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/em_text.ko 2024-05-21T21:29:40.353 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/em_u32.ko 2024-05-21T21:29:40.353 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/sch_choke.ko 2024-05-21T21:29:40.353 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/sch_codel.ko 2024-05-21T21:29:40.353 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/sch_drr.ko 2024-05-21T21:29:40.353 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/sch_fq.ko 2024-05-21T21:29:40.353 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/sch_gred.ko 2024-05-21T21:29:40.353 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/sch_hfsc.ko 2024-05-21T21:29:40.353 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/sch_hhf.ko 2024-05-21T21:29:40.353 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/sch_htb.ko 2024-05-21T21:29:40.353 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/sch_ingress.ko 2024-05-21T21:29:40.353 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/sch_mqprio.ko 2024-05-21T21:29:40.353 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/sch_mqprio_lib.ko 2024-05-21T21:29:40.353 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/sch_multiq.ko 2024-05-21T21:29:40.354 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/sch_netem.ko 2024-05-21T21:29:40.354 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/sch_pie.ko 2024-05-21T21:29:40.354 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/sch_plug.ko 2024-05-21T21:29:40.354 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/sch_prio.ko 2024-05-21T21:29:40.354 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/sch_qfq.ko 2024-05-21T21:29:40.354 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/sch_red.ko 2024-05-21T21:29:40.354 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/sch_sfb.ko 2024-05-21T21:29:40.354 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/sch_sfq.ko 2024-05-21T21:29:40.354 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/sch_tbf.ko 2024-05-21T21:29:40.354 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sched/sch_teql.ko 2024-05-21T21:29:40.354 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sctp 2024-05-21T21:29:40.354 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sctp/sctp.ko 2024-05-21T21:29:40.354 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sctp/sctp_diag.ko 2024-05-21T21:29:40.355 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sunrpc 2024-05-21T21:29:40.355 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sunrpc/auth_gss 2024-05-21T21:29:40.355 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-05-21T21:29:40.355 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-05-21T21:29:40.355 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sunrpc/sunrpc.ko 2024-05-21T21:29:40.355 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sunrpc/xprtrdma 2024-05-21T21:29:40.355 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-05-21T21:29:40.355 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/tipc 2024-05-21T21:29:40.355 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/tipc/tipc.ko 2024-05-21T21:29:40.355 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/tipc/tipc_diag.ko 2024-05-21T21:29:40.355 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/unix 2024-05-21T21:29:40.355 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/unix/unix_diag.ko 2024-05-21T21:29:40.355 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/vmw_vsock 2024-05-21T21:29:40.355 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-05-21T21:29:40.356 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-05-21T21:29:40.356 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/vmw_vsock/vsock.ko 2024-05-21T21:29:40.356 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/vmw_vsock/vsock_diag.ko 2024-05-21T21:29:40.356 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/vmw_vsock/vsock_loopback.ko 2024-05-21T21:29:40.356 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/xfrm 2024-05-21T21:29:40.356 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/net/xfrm/xfrm_ipcomp.ko 2024-05-21T21:29:40.356 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/security 2024-05-21T21:29:40.356 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/security/keys 2024-05-21T21:29:40.356 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/security/keys/encrypted-keys 2024-05-21T21:29:40.356 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-05-21T21:29:40.356 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/security/keys/trusted-keys 2024-05-21T21:29:40.356 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/kernel/security/keys/trusted-keys/trusted.ko 2024-05-21T21:29:40.356 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/modules.alias 2024-05-21T21:29:40.357 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/modules.alias.bin 2024-05-21T21:29:40.357 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/modules.builtin 2024-05-21T21:29:40.357 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/modules.builtin.alias.bin 2024-05-21T21:29:40.357 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/modules.builtin.bin 2024-05-21T21:29:40.357 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/modules.builtin.modinfo 2024-05-21T21:29:40.357 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/modules.dep 2024-05-21T21:29:40.357 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/modules.dep.bin 2024-05-21T21:29:40.357 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/modules.devname 2024-05-21T21:29:40.357 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/modules.order 2024-05-21T21:29:40.357 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/modules.softdep 2024-05-21T21:29:40.357 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/modules.symbols 2024-05-21T21:29:40.357 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/modules.symbols.bin 2024-05-21T21:29:40.357 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.9.0-rc3-g4fed890e8ce3/vmlinuz 2024-05-21T21:29:40.361 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g4fed890e8ce3 2024-05-21T21:29:40.362 DEBUG:teuthology.orchestra.run.smithi151:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-05-21T21:29:40.384 INFO:teuthology.orchestra.run.smithi151.stdout:postinstall scriptlet (using /bin/sh): 2024-05-21T21:29:40.384 INFO:teuthology.orchestra.run.smithi151.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-05-21T21:29:40.385 INFO:teuthology.orchestra.run.smithi151.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g4fed890e8ce3 /lib/modules/6.9.0-rc3-g4fed890e8ce3/vmlinuz 2024-05-21T21:29:40.385 INFO:teuthology.orchestra.run.smithi151.stdout:fi 2024-05-21T21:29:40.385 INFO:teuthology.orchestra.run.smithi151.stdout:for file in vmlinuz System.map config; do 2024-05-21T21:29:40.385 INFO:teuthology.orchestra.run.smithi151.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g4fed890e8ce3/${file}" "/boot/${file}-6.9.0-rc3-g4fed890e8ce3"; then 2024-05-21T21:29:40.385 INFO:teuthology.orchestra.run.smithi151.stdout: cp "/lib/modules/6.9.0-rc3-g4fed890e8ce3/${file}" "/boot/${file}-6.9.0-rc3-g4fed890e8ce3" 2024-05-21T21:29:40.385 INFO:teuthology.orchestra.run.smithi151.stdout: fi 2024-05-21T21:29:40.385 INFO:teuthology.orchestra.run.smithi151.stdout:done 2024-05-21T21:29:40.385 INFO:teuthology.orchestra.run.smithi151.stdout:if [ -d "/lib/modules/6.9.0-rc3-g4fed890e8ce3/dtb" ] && \ 2024-05-21T21:29:40.385 INFO:teuthology.orchestra.run.smithi151.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g4fed890e8ce3/dtb" "/boot/dtb-6.9.0-rc3-g4fed890e8ce3" >/dev/null 2>&1; then 2024-05-21T21:29:40.385 INFO:teuthology.orchestra.run.smithi151.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g4fed890e8ce3" 2024-05-21T21:29:40.385 INFO:teuthology.orchestra.run.smithi151.stdout: cp -r "/lib/modules/6.9.0-rc3-g4fed890e8ce3/dtb" "/boot/dtb-6.9.0-rc3-g4fed890e8ce3" 2024-05-21T21:29:40.385 INFO:teuthology.orchestra.run.smithi151.stdout:fi 2024-05-21T21:29:40.385 INFO:teuthology.orchestra.run.smithi151.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g4fed890e8ce3/modules.dep" ]; then 2024-05-21T21:29:40.386 INFO:teuthology.orchestra.run.smithi151.stdout: /usr/sbin/depmod 6.9.0-rc3-g4fed890e8ce3 2024-05-21T21:29:40.386 INFO:teuthology.orchestra.run.smithi151.stdout:fi 2024-05-21T21:29:40.386 INFO:teuthology.orchestra.run.smithi151.stdout:preuninstall scriptlet (using /bin/sh): 2024-05-21T21:29:40.386 INFO:teuthology.orchestra.run.smithi151.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-05-21T21:29:40.386 INFO:teuthology.orchestra.run.smithi151.stdout:kernel-install remove 6.9.0-rc3-g4fed890e8ce3 2024-05-21T21:29:40.386 INFO:teuthology.orchestra.run.smithi151.stdout:fi 2024-05-21T21:29:40.387 DEBUG:teuthology.orchestra.run.smithi151:> sudo rpm -qi grub2-tools 2024-05-21T21:29:40.458 INFO:teuthology.orchestra.run.smithi151.stdout:Name : grub2-tools 2024-05-21T21:29:40.459 INFO:teuthology.orchestra.run.smithi151.stdout:Epoch : 1 2024-05-21T21:29:40.459 INFO:teuthology.orchestra.run.smithi151.stdout:Version : 2.06 2024-05-21T21:29:40.459 INFO:teuthology.orchestra.run.smithi151.stdout:Release : 68.el9 2024-05-21T21:29:40.459 INFO:teuthology.orchestra.run.smithi151.stdout:Architecture: x86_64 2024-05-21T21:29:40.459 INFO:teuthology.orchestra.run.smithi151.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-21T21:29:40.459 INFO:teuthology.orchestra.run.smithi151.stdout:Group : Unspecified 2024-05-21T21:29:40.459 INFO:teuthology.orchestra.run.smithi151.stdout:Size : 8264254 2024-05-21T21:29:40.459 INFO:teuthology.orchestra.run.smithi151.stdout:License : GPLv3+ 2024-05-21T21:29:40.459 INFO:teuthology.orchestra.run.smithi151.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-21T21:29:40.460 INFO:teuthology.orchestra.run.smithi151.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-21T21:29:40.460 INFO:teuthology.orchestra.run.smithi151.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-21T21:29:40.460 INFO:teuthology.orchestra.run.smithi151.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-21T21:29:40.460 INFO:teuthology.orchestra.run.smithi151.stdout:Packager : builder@centos.org 2024-05-21T21:29:40.460 INFO:teuthology.orchestra.run.smithi151.stdout:Vendor : CentOS 2024-05-21T21:29:40.460 INFO:teuthology.orchestra.run.smithi151.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-21T21:29:40.460 INFO:teuthology.orchestra.run.smithi151.stdout:Summary : Support tools for GRUB. 2024-05-21T21:29:40.461 INFO:teuthology.orchestra.run.smithi151.stdout:Description : 2024-05-21T21:29:40.461 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T21:29:40.461 INFO:teuthology.orchestra.run.smithi151.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-21T21:29:40.461 INFO:teuthology.orchestra.run.smithi151.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-21T21:29:40.461 INFO:teuthology.orchestra.run.smithi151.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-21T21:29:40.461 INFO:teuthology.orchestra.run.smithi151.stdout:hardware devices. 2024-05-21T21:29:40.461 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T21:29:40.461 INFO:teuthology.orchestra.run.smithi151.stdout:This subpackage provides tools for support of all platforms. 2024-05-21T21:29:40.462 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-21T21:29:40.462 INFO:teuthology.task.kernel:Updating grub on smithi151 to boot 6.9.0-rc3-g4fed890e8ce3 2024-05-21T21:29:40.463 DEBUG:teuthology.orchestra.run.smithi151:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-21T21:29:41.026 INFO:teuthology.orchestra.run.smithi151.stderr:Generating grub configuration file ... 2024-05-21T21:29:42.446 INFO:teuthology.orchestra.run.smithi151.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-21T21:29:42.479 INFO:teuthology.orchestra.run.smithi151.stderr:done 2024-05-21T21:29:42.481 DEBUG:teuthology.orchestra.run.smithi151:> mktemp 2024-05-21T21:29:42.496 INFO:teuthology.orchestra.run.smithi151.stdout:/tmp/tmp.dqfXZMkxoq 2024-05-21T21:29:42.496 DEBUG:teuthology.orchestra.run.smithi151:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.dqfXZMkxoq 2024-05-21T21:29:42.566 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod 0666 /tmp/tmp.dqfXZMkxoq 2024-05-21T21:29:42.709 DEBUG:teuthology.orchestra.remote:smithi151:/tmp/tmp.dqfXZMkxoq is 6KB 2024-05-21T21:29:42.728 DEBUG:teuthology.orchestra.run.smithi151:> rm -fr /tmp/tmp.dqfXZMkxoq 2024-05-21T21:29:42.743 DEBUG:teuthology.orchestra.run.smithi151:> sudo /bin/ls /boot/loader/entries || true 2024-05-21T21:29:42.804 INFO:teuthology.orchestra.run.smithi151.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-21T21:29:42.804 INFO:teuthology.orchestra.run.smithi151.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-21T21:29:42.804 INFO:teuthology.orchestra.run.smithi151.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-21T21:29:42.804 INFO:teuthology.orchestra.run.smithi151.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g4fed890e8ce3.conf 2024-05-21T21:29:42.805 DEBUG:teuthology.orchestra.run.smithi151:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g4fed890e8ce3 2024-05-21T21:29:42.953 DEBUG:teuthology.orchestra.run.smithi151:> sudo shutdown -r now 2024-05-21T21:30:12.956 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-05-21T21:30:12.957 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi151.front.sepia.ceph.com' 2024-05-21T21:30:12.958 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-21T21:30:31.330 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.151 2024-05-21T21:30:40.331 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi151.front.sepia.ceph.com' 2024-05-21T21:30:40.332 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-21T21:30:43.390 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.151 2024-05-21T21:30:55.391 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi151.front.sepia.ceph.com' 2024-05-21T21:30:55.392 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-21T21:31:55.398 DEBUG:teuthology.orchestra.remote:timed out 2024-05-21T21:32:10.398 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi151.front.sepia.ceph.com' 2024-05-21T21:32:10.400 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-21T21:32:10.698 DEBUG:teuthology.orchestra.run.smithi151:> true 2024-05-21T21:32:11.330 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi151.front.sepia.ceph.com' 2024-05-21T21:32:11.330 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.9.0-rc3-g4fed890e8ce3"... 2024-05-21T21:32:11.331 DEBUG:teuthology.orchestra.run.smithi151:> uname -r 2024-05-21T21:32:11.376 INFO:teuthology.orchestra.run.smithi151.stdout:6.9.0-rc3-g4fed890e8ce3 2024-05-21T21:32:11.377 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g4fed890e8ce3 vs 6.9.0-rc3-g4fed890e8ce3 2024-05-21T21:32:11.377 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-21T21:32:11.377 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-21T21:32:12.378 DEBUG:teuthology.parallel:result is None 2024-05-21T21:32:12.379 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-21T21:32:12.387 INFO:teuthology.task.internal:Creating test directory... 2024-05-21T21:32:12.387 DEBUG:teuthology.orchestra.run.smithi151:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-21T21:32:12.394 DEBUG:teuthology.orchestra.run.smithi164:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-21T21:32:12.397 DEBUG:teuthology.orchestra.run.smithi184:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-21T21:32:12.444 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-21T21:32:12.453 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-21T21:32:12.537 INFO:teuthology.task.internal:Creating archive directory... 2024-05-21T21:32:12.538 DEBUG:teuthology.orchestra.run.smithi151:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-21T21:32:12.545 DEBUG:teuthology.orchestra.run.smithi164:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-21T21:32:12.548 DEBUG:teuthology.orchestra.run.smithi184:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-21T21:32:12.666 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-21T21:32:12.675 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-21T21:32:12.675 DEBUG:teuthology.orchestra.run.smithi164:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-21T21:32:12.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-21T21:32:12.691 DEBUG:teuthology.orchestra.run.smithi184:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-21T21:32:12.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-21T21:32:12.709 DEBUG:teuthology.orchestra.run.smithi151:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-21T21:32:12.717 DEBUG:teuthology.orchestra.run.smithi164:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-21T21:32:12.734 DEBUG:teuthology.orchestra.run.smithi184:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-21T21:32:12.782 INFO:teuthology.orchestra.run.smithi164.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-21T21:32:12.788 INFO:teuthology.orchestra.run.smithi184.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-21T21:32:12.798 INFO:teuthology.orchestra.run.smithi164.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-21T21:32:12.809 INFO:teuthology.orchestra.run.smithi184.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-21T21:32:12.838 INFO:teuthology.orchestra.run.smithi151.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-21T21:32:12.904 INFO:teuthology.orchestra.run.smithi151.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-21T21:32:12.909 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-21T21:32:12.917 INFO:teuthology.task.internal:Configuring sudo... 2024-05-21T21:32:12.918 DEBUG:teuthology.orchestra.run.smithi151:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-21T21:32:12.923 DEBUG:teuthology.orchestra.run.smithi164:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-21T21:32:12.925 DEBUG:teuthology.orchestra.run.smithi184:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-21T21:32:13.000 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-21T21:32:13.015 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-21T21:32:13.016 DEBUG:teuthology.orchestra.run.smithi151:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-21T21:32:13.023 DEBUG:teuthology.orchestra.run.smithi164:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-21T21:32:13.027 DEBUG:teuthology.orchestra.run.smithi184:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-21T21:32:13.070 DEBUG:teuthology.orchestra.run.smithi151:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-21T21:32:13.154 DEBUG:teuthology.orchestra.run.smithi151:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-21T21:32:13.259 DEBUG:teuthology.orchestra.run.smithi151:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-21T21:32:13.340 DEBUG:teuthology.orchestra.run.smithi151:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-21T21:32:13.424 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-05-21T21:32:13.424 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-21T21:32:13.507 DEBUG:teuthology.orchestra.run.smithi164:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-21T21:32:13.554 DEBUG:teuthology.orchestra.run.smithi164:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-21T21:32:13.608 DEBUG:teuthology.orchestra.run.smithi164:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-21T21:32:13.692 DEBUG:teuthology.orchestra.run.smithi164:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-21T21:32:13.736 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-21T21:32:13.736 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-21T21:32:13.802 DEBUG:teuthology.orchestra.run.smithi184:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-21T21:32:13.847 DEBUG:teuthology.orchestra.run.smithi184:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-21T21:32:13.901 DEBUG:teuthology.orchestra.run.smithi184:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-21T21:32:13.984 DEBUG:teuthology.orchestra.run.smithi184:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-21T21:32:14.026 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-05-21T21:32:14.026 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-21T21:32:14.091 DEBUG:teuthology.orchestra.run.smithi151:> sudo service rsyslog restart 2024-05-21T21:32:14.098 DEBUG:teuthology.orchestra.run.smithi164:> sudo service rsyslog restart 2024-05-21T21:32:14.101 DEBUG:teuthology.orchestra.run.smithi184:> sudo service rsyslog restart 2024-05-21T21:32:14.135 INFO:teuthology.orchestra.run.smithi164.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-21T21:32:14.163 INFO:teuthology.orchestra.run.smithi184.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-21T21:32:14.208 INFO:teuthology.orchestra.run.smithi151.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-21T21:32:14.578 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-21T21:32:14.586 INFO:teuthology.task.internal:Starting timer... 2024-05-21T21:32:14.586 INFO:teuthology.run_tasks:Running task pcp... 2024-05-21T21:32:14.619 INFO:teuthology.run_tasks:Running task selinux... 2024-05-21T21:32:14.652 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-05-21T21:32:14.652 DEBUG:teuthology.orchestra.run.smithi151:> sudo service auditd rotate 2024-05-21T21:32:14.761 INFO:teuthology.orchestra.run.smithi151.stdout:Rotating logs: 2024-05-21T21:32:14.765 DEBUG:teuthology.orchestra.run.smithi164:> sudo service auditd rotate 2024-05-21T21:32:14.846 INFO:teuthology.orchestra.run.smithi164.stdout:Rotating logs: 2024-05-21T21:32:14.848 DEBUG:teuthology.orchestra.run.smithi184:> sudo service auditd rotate 2024-05-21T21:32:14.919 INFO:teuthology.orchestra.run.smithi184.stdout:Rotating logs: 2024-05-21T21:32:14.921 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-21T21:32:14.921 DEBUG:teuthology.orchestra.run.smithi151:> /usr/sbin/getenforce 2024-05-21T21:32:14.972 INFO:teuthology.orchestra.run.smithi151.stdout:Permissive 2024-05-21T21:32:14.972 DEBUG:teuthology.orchestra.run.smithi164:> /usr/sbin/getenforce 2024-05-21T21:32:14.994 INFO:teuthology.orchestra.run.smithi164.stdout:Permissive 2024-05-21T21:32:14.994 DEBUG:teuthology.orchestra.run.smithi184:> /usr/sbin/getenforce 2024-05-21T21:32:15.018 INFO:teuthology.orchestra.run.smithi184.stdout:Permissive 2024-05-21T21:32:15.019 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi151.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi164.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi184.front.sepia.ceph.com': 'permissive'} 2024-05-21T21:32:15.019 DEBUG:teuthology.orchestra.run.smithi151:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-21T21:32:15.090 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-21T21:32:15.090 DEBUG:teuthology.orchestra.run.smithi164:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-21T21:32:15.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-21T21:32:15.118 DEBUG:teuthology.orchestra.run.smithi184:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-21T21:32:15.147 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-21T21:32:15.147 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-21T21:32:15.147 DEBUG:teuthology.orchestra.run.smithi151:> sudo /usr/sbin/setenforce permissive 2024-05-21T21:32:15.215 DEBUG:teuthology.orchestra.run.smithi164:> sudo /usr/sbin/setenforce permissive 2024-05-21T21:32:15.241 DEBUG:teuthology.orchestra.run.smithi184:> sudo /usr/sbin/setenforce permissive 2024-05-21T21:32:15.266 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-21T21:32:15.276 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-21T21:32:15.288 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-21T21:32:15.289 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi151.front.sepia.ceph.com,smithi164.front.sepia.ceph.com,smithi184.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-21T21:38:27.184 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi151.front.sepia.ceph.com'), Remote(name='ubuntu@smithi164.front.sepia.ceph.com'), Remote(name='ubuntu@smithi184.front.sepia.ceph.com')] 2024-05-21T21:38:27.186 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi151.front.sepia.ceph.com' 2024-05-21T21:38:27.187 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-21T21:38:27.302 DEBUG:teuthology.orchestra.run.smithi151:> true 2024-05-21T21:38:27.472 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi151.front.sepia.ceph.com' 2024-05-21T21:38:27.473 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-05-21T21:38:27.473 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-21T21:38:27.548 DEBUG:teuthology.orchestra.run.smithi164:> true 2024-05-21T21:38:27.621 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-05-21T21:38:27.622 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi184.front.sepia.ceph.com' 2024-05-21T21:38:27.622 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi184.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-21T21:38:27.699 DEBUG:teuthology.orchestra.run.smithi184:> true 2024-05-21T21:38:27.787 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi184.front.sepia.ceph.com' 2024-05-21T21:38:27.788 INFO:teuthology.run_tasks:Running task clock... 2024-05-21T21:38:27.800 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-21T21:38:27.801 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-21T21:38:27.801 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-21T21:38:27.808 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-21T21:38:27.809 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-21T21:38:27.811 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-21T21:38:27.811 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-21T21:38:27.837 INFO:teuthology.orchestra.run.smithi164.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-21T21:38:27.851 INFO:teuthology.orchestra.run.smithi164.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-21T21:38:27.858 INFO:teuthology.orchestra.run.smithi184.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-21T21:38:27.873 INFO:teuthology.orchestra.run.smithi164.stderr:sudo: ntpd: command not found 2024-05-21T21:38:27.875 INFO:teuthology.orchestra.run.smithi184.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-21T21:38:27.883 INFO:teuthology.orchestra.run.smithi164.stdout:506 Cannot talk to daemon 2024-05-21T21:38:27.894 INFO:teuthology.orchestra.run.smithi151.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-21T21:38:27.897 INFO:teuthology.orchestra.run.smithi164.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-21T21:38:27.903 INFO:teuthology.orchestra.run.smithi184.stderr:sudo: ntpd: command not found 2024-05-21T21:38:27.910 INFO:teuthology.orchestra.run.smithi164.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-21T21:38:27.916 INFO:teuthology.orchestra.run.smithi184.stdout:506 Cannot talk to daemon 2024-05-21T21:38:27.932 INFO:teuthology.orchestra.run.smithi184.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-21T21:38:27.935 INFO:teuthology.orchestra.run.smithi151.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-21T21:38:27.947 INFO:teuthology.orchestra.run.smithi184.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-21T21:38:27.966 INFO:teuthology.orchestra.run.smithi164.stderr:bash: line 1: ntpq: command not found 2024-05-21T21:38:27.969 INFO:teuthology.orchestra.run.smithi164.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-21T21:38:27.969 INFO:teuthology.orchestra.run.smithi164.stdout:=============================================================================== 2024-05-21T21:38:27.969 INFO:teuthology.orchestra.run.smithi164.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-21T21:38:27.969 INFO:teuthology.orchestra.run.smithi164.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-21T21:38:27.969 INFO:teuthology.orchestra.run.smithi164.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-21T21:38:27.969 INFO:teuthology.orchestra.run.smithi164.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-21T21:38:28.005 INFO:teuthology.orchestra.run.smithi151.stderr:sudo: ntpd: command not found 2024-05-21T21:38:28.012 INFO:teuthology.orchestra.run.smithi184.stderr:bash: line 1: ntpq: command not found 2024-05-21T21:38:28.015 INFO:teuthology.orchestra.run.smithi184.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-21T21:38:28.015 INFO:teuthology.orchestra.run.smithi184.stdout:=============================================================================== 2024-05-21T21:38:28.015 INFO:teuthology.orchestra.run.smithi184.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-21T21:38:28.015 INFO:teuthology.orchestra.run.smithi184.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-21T21:38:28.015 INFO:teuthology.orchestra.run.smithi184.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-21T21:38:28.015 INFO:teuthology.orchestra.run.smithi184.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-21T21:38:28.036 INFO:teuthology.orchestra.run.smithi151.stdout:506 Cannot talk to daemon 2024-05-21T21:38:28.078 INFO:teuthology.orchestra.run.smithi151.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-21T21:38:28.118 INFO:teuthology.orchestra.run.smithi151.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-21T21:38:28.265 INFO:teuthology.orchestra.run.smithi151.stderr:bash: line 1: ntpq: command not found 2024-05-21T21:38:28.275 INFO:teuthology.orchestra.run.smithi151.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-21T21:38:28.275 INFO:teuthology.orchestra.run.smithi151.stdout:=============================================================================== 2024-05-21T21:38:28.275 INFO:teuthology.orchestra.run.smithi151.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-21T21:38:28.275 INFO:teuthology.orchestra.run.smithi151.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-21T21:38:28.275 INFO:teuthology.orchestra.run.smithi151.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-21T21:38:28.275 INFO:teuthology.orchestra.run.smithi151.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-21T21:38:28.277 INFO:teuthology.run_tasks:Running task install... 2024-05-21T21:38:28.285 DEBUG:teuthology.task.install:project ceph 2024-05-21T21:38:28.285 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '3b748ea08891006c1152513d25ab4a03f7c9a995'}} 2024-05-21T21:38:28.285 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '3b748ea08891006c1152513d25ab4a03f7c9a995'} 2024-05-21T21:38:28.285 INFO:teuthology.task.install:Using flavor: default 2024-05-21T21:38:28.291 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-21T21:38:28.291 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-05-21T21:38:28.292 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3b748ea08891006c1152513d25ab4a03f7c9a995', 'tag': None, 'wait_for_package': False} 2024-05-21T21:38:28.292 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3b748ea08891006c1152513d25ab4a03f7c9a995 2024-05-21T21:38:28.294 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3b748ea08891006c1152513d25ab4a03f7c9a995', 'tag': None, 'wait_for_package': False} 2024-05-21T21:38:28.294 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3b748ea08891006c1152513d25ab4a03f7c9a995 2024-05-21T21:38:28.295 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3b748ea08891006c1152513d25ab4a03f7c9a995', 'tag': None, 'wait_for_package': False} 2024-05-21T21:38:28.295 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3b748ea08891006c1152513d25ab4a03f7c9a995 2024-05-21T21:38:28.437 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce-overdrive-export/3b748ea08891006c1152513d25ab4a03f7c9a995/centos/9/flavors/default/ 2024-05-21T21:38:28.437 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3830.g3b748ea0 2024-05-21T21:38:28.444 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce-overdrive-export/3b748ea08891006c1152513d25ab4a03f7c9a995/centos/9/flavors/default/ 2024-05-21T21:38:28.444 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3830.g3b748ea0 2024-05-21T21:38:28.516 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce-overdrive-export/3b748ea08891006c1152513d25ab4a03f7c9a995/centos/9/flavors/default/ 2024-05-21T21:38:28.516 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3830.g3b748ea0 2024-05-21T21:38:28.660 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce-overdrive-export/3b748ea08891006c1152513d25ab4a03f7c9a995/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce-overdrive-export/3b748ea08891006c1152513d25ab4a03f7c9a995/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce-overdrive-export/3b748ea08891006c1152513d25ab4a03f7c9a995/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-21T21:38:28.660 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-05-21T21:38:28.660 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-21T21:38:28.666 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce-overdrive-export/3b748ea08891006c1152513d25ab4a03f7c9a995/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce-overdrive-export/3b748ea08891006c1152513d25ab4a03f7c9a995/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce-overdrive-export/3b748ea08891006c1152513d25ab4a03f7c9a995/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-21T21:38:28.666 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-05-21T21:38:28.666 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-21T21:38:28.672 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce-overdrive-export/3b748ea08891006c1152513d25ab4a03f7c9a995/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce-overdrive-export/3b748ea08891006c1152513d25ab4a03f7c9a995/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce-overdrive-export/3b748ea08891006c1152513d25ab4a03f7c9a995/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-21T21:38:28.672 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-21T21:38:28.672 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-21T21:38:28.689 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-21T21:38:28.689 DEBUG:teuthology.orchestra.run.smithi184:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3b748ea08891006c1152513d25ab4a03f7c9a995/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-21T21:38:28.697 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-21T21:38:28.697 DEBUG:teuthology.orchestra.run.smithi164:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3b748ea08891006c1152513d25ab4a03f7c9a995/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-21T21:38:28.756 DEBUG:teuthology.orchestra.run.smithi184:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-21T21:38:28.757 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-21T21:38:28.757 DEBUG:teuthology.orchestra.run.smithi151:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3b748ea08891006c1152513d25ab4a03f7c9a995/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-21T21:38:28.761 DEBUG:teuthology.orchestra.run.smithi164:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-21T21:38:28.831 DEBUG:teuthology.orchestra.run.smithi184:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-21T21:38:28.835 DEBUG:teuthology.orchestra.run.smithi164:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-21T21:38:28.887 DEBUG:teuthology.orchestra.run.smithi151:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-21T21:38:28.895 INFO:teuthology.orchestra.run.smithi184.stdout:check_obsoletes = 1 2024-05-21T21:38:28.897 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum clean all 2024-05-21T21:38:28.898 INFO:teuthology.orchestra.run.smithi164.stdout:check_obsoletes = 1 2024-05-21T21:38:28.900 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum clean all 2024-05-21T21:38:29.000 DEBUG:teuthology.orchestra.run.smithi151:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-21T21:38:29.078 INFO:teuthology.orchestra.run.smithi151.stdout:check_obsoletes = 1 2024-05-21T21:38:29.082 DEBUG:teuthology.orchestra.run.smithi151:> sudo yum clean all 2024-05-21T21:38:29.123 INFO:teuthology.orchestra.run.smithi184.stdout:45 files removed 2024-05-21T21:38:29.125 INFO:teuthology.orchestra.run.smithi164.stdout:45 files removed 2024-05-21T21:38:29.150 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-21T21:38:29.152 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-21T21:38:29.407 INFO:teuthology.orchestra.run.smithi151.stdout:45 files removed 2024-05-21T21:38:29.446 DEBUG:teuthology.orchestra.run.smithi151:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-21T21:38:29.897 INFO:teuthology.orchestra.run.smithi184.stdout:ceph packages for x86_64 174 kB/s | 84 kB 00:00 2024-05-21T21:38:29.934 INFO:teuthology.orchestra.run.smithi164.stdout:ceph packages for x86_64 161 kB/s | 84 kB 00:00 2024-05-21T21:38:30.298 INFO:teuthology.orchestra.run.smithi184.stdout:ceph noarch packages 33 kB/s | 12 kB 00:00 2024-05-21T21:38:30.301 INFO:teuthology.orchestra.run.smithi164.stdout:ceph noarch packages 36 kB/s | 12 kB 00:00 2024-05-21T21:38:30.318 INFO:teuthology.orchestra.run.smithi151.stdout:ceph packages for x86_64 183 kB/s | 84 kB 00:00 2024-05-21T21:38:30.707 INFO:teuthology.orchestra.run.smithi184.stdout:ceph source packages 5.1 kB/s | 1.9 kB 00:00 2024-05-21T21:38:30.752 INFO:teuthology.orchestra.run.smithi164.stdout:ceph source packages 4.6 kB/s | 1.9 kB 00:00 2024-05-21T21:38:30.827 INFO:teuthology.orchestra.run.smithi151.stdout:ceph noarch packages 29 kB/s | 12 kB 00:00 2024-05-21T21:38:31.402 INFO:teuthology.orchestra.run.smithi151.stdout:ceph source packages 3.9 kB/s | 1.9 kB 00:00 2024-05-21T21:38:32.067 INFO:teuthology.orchestra.run.smithi151.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.1 MB 00:00 2024-05-21T21:38:33.785 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - BaseOS 2.7 MB/s | 8.1 MB 00:03 2024-05-21T21:38:35.852 INFO:teuthology.orchestra.run.smithi151.stdout:CentOS Stream 9 - AppStream 7.5 MB/s | 19 MB 00:02 2024-05-21T21:38:39.348 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - BaseOS 961 kB/s | 8.1 MB 00:08 2024-05-21T21:38:40.661 INFO:teuthology.orchestra.run.smithi151.stdout:CentOS Stream 9 - CRB 7.4 MB/s | 6.2 MB 00:00 2024-05-21T21:38:41.774 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - AppStream 14 MB/s | 19 MB 00:01 2024-05-21T21:38:42.620 INFO:teuthology.orchestra.run.smithi151.stdout:CentOS Stream 9 - Extras packages 28 kB/s | 16 kB 00:00 2024-05-21T21:38:44.936 INFO:teuthology.orchestra.run.smithi151.stdout:Extra Packages for Enterprise Linux 9.9 MB/s | 21 MB 00:02 2024-05-21T21:38:46.183 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - CRB 7.3 MB/s | 6.2 MB 00:00 2024-05-21T21:38:47.742 INFO:teuthology.orchestra.run.smithi184.stdout:CentOS Stream 9 - Extras packages 39 kB/s | 16 kB 00:00 2024-05-21T21:38:49.127 INFO:teuthology.orchestra.run.smithi184.stdout:Extra Packages for Enterprise Linux 17 MB/s | 21 MB 00:01 2024-05-21T21:38:50.026 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - AppStream 1.3 MB/s | 19 MB 00:15 2024-05-21T21:38:51.147 INFO:teuthology.orchestra.run.smithi151.stdout:lab-extras 18 kB/s | 1.7 kB 00:00 2024-05-21T21:38:53.615 INFO:teuthology.orchestra.run.smithi151.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-21T21:38:53.616 INFO:teuthology.orchestra.run.smithi151.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-21T21:38:53.637 INFO:teuthology.orchestra.run.smithi151.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-21T21:38:53.639 INFO:teuthology.orchestra.run.smithi151.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-21T21:38:53.640 INFO:teuthology.orchestra.run.smithi151.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-21T21:38:53.641 INFO:teuthology.orchestra.run.smithi151.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-21T21:38:53.644 INFO:teuthology.orchestra.run.smithi151.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-21T21:38:53.647 INFO:teuthology.orchestra.run.smithi151.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-21T21:38:53.648 INFO:teuthology.orchestra.run.smithi151.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-21T21:38:53.722 INFO:teuthology.orchestra.run.smithi151.stdout:Dependencies resolved. 2024-05-21T21:38:53.747 INFO:teuthology.orchestra.run.smithi151.stdout:======================================================================================= 2024-05-21T21:38:53.748 INFO:teuthology.orchestra.run.smithi151.stdout: Package Arch Version Repository Size 2024-05-21T21:38:53.748 INFO:teuthology.orchestra.run.smithi151.stdout:======================================================================================= 2024-05-21T21:38:53.748 INFO:teuthology.orchestra.run.smithi151.stdout:Installing: 2024-05-21T21:38:53.748 INFO:teuthology.orchestra.run.smithi151.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-21T21:38:53.748 INFO:teuthology.orchestra.run.smithi151.stdout: ceph x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 6.6 k 2024-05-21T21:38:53.748 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-base x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 5.4 M 2024-05-21T21:38:53.748 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-fuse x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 891 k 2024-05-21T21:38:53.748 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 145 k 2024-05-21T21:38:53.748 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 1.5 M 2024-05-21T21:38:53.748 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3830.g3b748ea0.el9 ceph-noarch 143 k 2024-05-21T21:38:53.749 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3830.g3b748ea0.el9 ceph-noarch 3.5 M 2024-05-21T21:38:53.749 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3830.g3b748ea0.el9 ceph-noarch 7.4 M 2024-05-21T21:38:53.749 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-rook noarch 2:19.0.0-3830.g3b748ea0.el9 ceph-noarch 49 k 2024-05-21T21:38:53.749 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-radosgw x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 11 M 2024-05-21T21:38:53.749 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-test x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 49 M 2024-05-21T21:38:53.749 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-volume noarch 2:19.0.0-3830.g3b748ea0.el9 ceph-noarch 273 k 2024-05-21T21:38:53.749 INFO:teuthology.orchestra.run.smithi151.stdout: cephadm noarch 2:19.0.0-3830.g3b748ea0.el9 ceph-noarch 764 k 2024-05-21T21:38:53.749 INFO:teuthology.orchestra.run.smithi151.stdout: cephfs-mirror x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 225 k 2024-05-21T21:38:53.749 INFO:teuthology.orchestra.run.smithi151.stdout: cephfs-shell x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 27 k 2024-05-21T21:38:53.749 INFO:teuthology.orchestra.run.smithi151.stdout: cephfs-top noarch 2:19.0.0-3830.g3b748ea0.el9 ceph-noarch 18 k 2024-05-21T21:38:53.749 INFO:teuthology.orchestra.run.smithi151.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-05-21T21:38:53.749 INFO:teuthology.orchestra.run.smithi151.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-21T21:38:53.750 INFO:teuthology.orchestra.run.smithi151.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-21T21:38:53.750 INFO:teuthology.orchestra.run.smithi151.stdout: libcephfs-devel x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 32 k 2024-05-21T21:38:53.750 INFO:teuthology.orchestra.run.smithi151.stdout: libcephfs2 x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 746 k 2024-05-21T21:38:53.750 INFO:teuthology.orchestra.run.smithi151.stdout: librados-devel x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 127 k 2024-05-21T21:38:53.750 INFO:teuthology.orchestra.run.smithi151.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-21T21:38:53.750 INFO:teuthology.orchestra.run.smithi151.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-21T21:38:53.750 INFO:teuthology.orchestra.run.smithi151.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-21T21:38:53.750 INFO:teuthology.orchestra.run.smithi151.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-21T21:38:53.750 INFO:teuthology.orchestra.run.smithi151.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-21T21:38:53.750 INFO:teuthology.orchestra.run.smithi151.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-21T21:38:53.750 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cephfs x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 161 k 2024-05-21T21:38:53.750 INFO:teuthology.orchestra.run.smithi151.stdout: python3-rados x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 320 k 2024-05-21T21:38:53.751 INFO:teuthology.orchestra.run.smithi151.stdout: python3-rbd x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 300 k 2024-05-21T21:38:53.751 INFO:teuthology.orchestra.run.smithi151.stdout: python3-rgw x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 99 k 2024-05-21T21:38:53.751 INFO:teuthology.orchestra.run.smithi151.stdout: rbd-fuse x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 85 k 2024-05-21T21:38:53.751 INFO:teuthology.orchestra.run.smithi151.stdout: rbd-mirror x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 3.1 M 2024-05-21T21:38:53.751 INFO:teuthology.orchestra.run.smithi151.stdout: rbd-nbd x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 171 k 2024-05-21T21:38:53.751 INFO:teuthology.orchestra.run.smithi151.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-21T21:38:53.751 INFO:teuthology.orchestra.run.smithi151.stdout:Upgrading: 2024-05-21T21:38:53.751 INFO:teuthology.orchestra.run.smithi151.stdout: NetworkManager x86_64 1:1.47.90-1.el9 baseos 2.3 M 2024-05-21T21:38:53.751 INFO:teuthology.orchestra.run.smithi151.stdout: NetworkManager-libnm x86_64 1:1.47.90-1.el9 baseos 1.8 M 2024-05-21T21:38:53.751 INFO:teuthology.orchestra.run.smithi151.stdout: NetworkManager-team x86_64 1:1.47.90-1.el9 baseos 41 k 2024-05-21T21:38:53.751 INFO:teuthology.orchestra.run.smithi151.stdout: NetworkManager-tui x86_64 1:1.47.90-1.el9 baseos 248 k 2024-05-21T21:38:53.751 INFO:teuthology.orchestra.run.smithi151.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-05-21T21:38:53.751 INFO:teuthology.orchestra.run.smithi151.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-05-21T21:38:53.752 INFO:teuthology.orchestra.run.smithi151.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-05-21T21:38:53.752 INFO:teuthology.orchestra.run.smithi151.stdout: librados2 x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 3.4 M 2024-05-21T21:38:53.752 INFO:teuthology.orchestra.run.smithi151.stdout: librbd1 x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 3.1 M 2024-05-21T21:38:53.752 INFO:teuthology.orchestra.run.smithi151.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-21T21:38:53.752 INFO:teuthology.orchestra.run.smithi151.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-21T21:38:53.752 INFO:teuthology.orchestra.run.smithi151.stdout:Installing dependencies: 2024-05-21T21:38:53.752 INFO:teuthology.orchestra.run.smithi151.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-21T21:38:53.752 INFO:teuthology.orchestra.run.smithi151.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-21T21:38:53.752 INFO:teuthology.orchestra.run.smithi151.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-21T21:38:53.752 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-common x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 21 M 2024-05-21T21:38:53.752 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3830.g3b748ea0.el9 ceph-noarch 34 k 2024-05-21T21:38:53.752 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mds x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 2.4 M 2024-05-21T21:38:53.753 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3830.g3b748ea0.el9 ceph-noarch 274 k 2024-05-21T21:38:53.753 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mon x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 4.7 M 2024-05-21T21:38:53.753 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-osd x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 17 M 2024-05-21T21:38:53.753 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3830.g3b748ea0.el9 ceph-noarch 16 k 2024-05-21T21:38:53.753 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-selinux x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 25 k 2024-05-21T21:38:53.753 INFO:teuthology.orchestra.run.smithi151.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-21T21:38:53.753 INFO:teuthology.orchestra.run.smithi151.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-21T21:38:53.753 INFO:teuthology.orchestra.run.smithi151.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-21T21:38:53.753 INFO:teuthology.orchestra.run.smithi151.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-21T21:38:53.753 INFO:teuthology.orchestra.run.smithi151.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-21T21:38:53.753 INFO:teuthology.orchestra.run.smithi151.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-21T21:38:53.753 INFO:teuthology.orchestra.run.smithi151.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-21T21:38:53.753 INFO:teuthology.orchestra.run.smithi151.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-21T21:38:53.754 INFO:teuthology.orchestra.run.smithi151.stdout: libcephsqlite x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 163 k 2024-05-21T21:38:53.754 INFO:teuthology.orchestra.run.smithi151.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-21T21:38:53.754 INFO:teuthology.orchestra.run.smithi151.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-21T21:38:53.754 INFO:teuthology.orchestra.run.smithi151.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-21T21:38:53.754 INFO:teuthology.orchestra.run.smithi151.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-21T21:38:53.754 INFO:teuthology.orchestra.run.smithi151.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-21T21:38:53.754 INFO:teuthology.orchestra.run.smithi151.stdout: libradosstriper1 x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 501 k 2024-05-21T21:38:53.754 INFO:teuthology.orchestra.run.smithi151.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-21T21:38:53.754 INFO:teuthology.orchestra.run.smithi151.stdout: librgw2 x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 5.4 M 2024-05-21T21:38:53.754 INFO:teuthology.orchestra.run.smithi151.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-21T21:38:53.754 INFO:teuthology.orchestra.run.smithi151.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-21T21:38:53.754 INFO:teuthology.orchestra.run.smithi151.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-21T21:38:53.755 INFO:teuthology.orchestra.run.smithi151.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-21T21:38:53.755 INFO:teuthology.orchestra.run.smithi151.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-21T21:38:53.755 INFO:teuthology.orchestra.run.smithi151.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-05-21T21:38:53.755 INFO:teuthology.orchestra.run.smithi151.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-21T21:38:53.755 INFO:teuthology.orchestra.run.smithi151.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-21T21:38:53.755 INFO:teuthology.orchestra.run.smithi151.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-21T21:38:53.755 INFO:teuthology.orchestra.run.smithi151.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-21T21:38:53.755 INFO:teuthology.orchestra.run.smithi151.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-21T21:38:53.755 INFO:teuthology.orchestra.run.smithi151.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-21T21:38:53.755 INFO:teuthology.orchestra.run.smithi151.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-21T21:38:53.755 INFO:teuthology.orchestra.run.smithi151.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-21T21:38:53.755 INFO:teuthology.orchestra.run.smithi151.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-21T21:38:53.756 INFO:teuthology.orchestra.run.smithi151.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-21T21:38:53.756 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-21T21:38:53.756 INFO:teuthology.orchestra.run.smithi151.stdout: python3-ceph-argparse x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 45 k 2024-05-21T21:38:53.756 INFO:teuthology.orchestra.run.smithi151.stdout: python3-ceph-common x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 136 k 2024-05-21T21:38:53.756 INFO:teuthology.orchestra.run.smithi151.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-21T21:38:53.756 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-21T21:38:53.756 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-21T21:38:53.756 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-21T21:38:53.756 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-21T21:38:53.756 INFO:teuthology.orchestra.run.smithi151.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-05-21T21:38:53.756 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-21T21:38:53.756 INFO:teuthology.orchestra.run.smithi151.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-21T21:38:53.756 INFO:teuthology.orchestra.run.smithi151.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-21T21:38:53.757 INFO:teuthology.orchestra.run.smithi151.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-21T21:38:53.757 INFO:teuthology.orchestra.run.smithi151.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-21T21:38:53.757 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-21T21:38:53.757 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-21T21:38:53.757 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-21T21:38:53.757 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-21T21:38:53.757 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-21T21:38:53.757 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-05-21T21:38:53.757 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-21T21:38:53.757 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-21T21:38:53.757 INFO:teuthology.orchestra.run.smithi151.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-21T21:38:53.757 INFO:teuthology.orchestra.run.smithi151.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-21T21:38:53.757 INFO:teuthology.orchestra.run.smithi151.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-21T21:38:53.758 INFO:teuthology.orchestra.run.smithi151.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-21T21:38:53.758 INFO:teuthology.orchestra.run.smithi151.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-21T21:38:53.758 INFO:teuthology.orchestra.run.smithi151.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-21T21:38:53.758 INFO:teuthology.orchestra.run.smithi151.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-21T21:38:53.758 INFO:teuthology.orchestra.run.smithi151.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-21T21:38:53.758 INFO:teuthology.orchestra.run.smithi151.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-21T21:38:53.758 INFO:teuthology.orchestra.run.smithi151.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-21T21:38:53.758 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-21T21:38:53.758 INFO:teuthology.orchestra.run.smithi151.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-21T21:38:53.758 INFO:teuthology.orchestra.run.smithi151.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-21T21:38:53.758 INFO:teuthology.orchestra.run.smithi151.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-21T21:38:53.758 INFO:teuthology.orchestra.run.smithi151.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-21T21:38:53.758 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-21T21:38:53.759 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-21T21:38:53.759 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-21T21:38:53.759 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-21T21:38:53.759 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-21T21:38:53.759 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-21T21:38:53.759 INFO:teuthology.orchestra.run.smithi151.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-21T21:38:53.759 INFO:teuthology.orchestra.run.smithi151.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-21T21:38:53.759 INFO:teuthology.orchestra.run.smithi151.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-21T21:38:53.759 INFO:teuthology.orchestra.run.smithi151.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-21T21:38:53.759 INFO:teuthology.orchestra.run.smithi151.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-21T21:38:53.759 INFO:teuthology.orchestra.run.smithi151.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-21T21:38:53.759 INFO:teuthology.orchestra.run.smithi151.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-21T21:38:53.759 INFO:teuthology.orchestra.run.smithi151.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-21T21:38:53.760 INFO:teuthology.orchestra.run.smithi151.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-21T21:38:53.760 INFO:teuthology.orchestra.run.smithi151.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-21T21:38:53.760 INFO:teuthology.orchestra.run.smithi151.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-21T21:38:53.760 INFO:teuthology.orchestra.run.smithi151.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-21T21:38:53.760 INFO:teuthology.orchestra.run.smithi151.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-21T21:38:53.760 INFO:teuthology.orchestra.run.smithi151.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-21T21:38:53.760 INFO:teuthology.orchestra.run.smithi151.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-21T21:38:53.760 INFO:teuthology.orchestra.run.smithi151.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-21T21:38:53.760 INFO:teuthology.orchestra.run.smithi151.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-21T21:38:53.760 INFO:teuthology.orchestra.run.smithi151.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-21T21:38:53.760 INFO:teuthology.orchestra.run.smithi151.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-21T21:38:53.760 INFO:teuthology.orchestra.run.smithi151.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-21T21:38:53.760 INFO:teuthology.orchestra.run.smithi151.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-21T21:38:53.761 INFO:teuthology.orchestra.run.smithi151.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-21T21:38:53.761 INFO:teuthology.orchestra.run.smithi151.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-21T21:38:53.761 INFO:teuthology.orchestra.run.smithi151.stdout:Installing weak dependencies: 2024-05-21T21:38:53.761 INFO:teuthology.orchestra.run.smithi151.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-21T21:38:53.761 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T21:38:53.761 INFO:teuthology.orchestra.run.smithi151.stdout:Transaction Summary 2024-05-21T21:38:53.761 INFO:teuthology.orchestra.run.smithi151.stdout:======================================================================================= 2024-05-21T21:38:53.761 INFO:teuthology.orchestra.run.smithi151.stdout:Install 148 Packages 2024-05-21T21:38:53.761 INFO:teuthology.orchestra.run.smithi151.stdout:Upgrade 11 Packages 2024-05-21T21:38:53.761 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T21:38:53.762 INFO:teuthology.orchestra.run.smithi151.stdout:Total download size: 227 M 2024-05-21T21:38:53.762 INFO:teuthology.orchestra.run.smithi151.stdout:Downloading Packages: 2024-05-21T21:38:54.344 INFO:teuthology.orchestra.run.smithi184.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-05-21T21:38:54.757 INFO:teuthology.orchestra.run.smithi151.stdout:(1/159): ceph-19.0.0-3830.g3b748ea0.el9.x86_64. 20 kB/s | 6.6 kB 00:00 2024-05-21T21:38:55.093 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - CRB 4.3 MB/s | 6.2 MB 00:01 2024-05-21T21:38:55.348 INFO:teuthology.orchestra.run.smithi151.stdout:(2/159): ceph-fuse-19.0.0-3830.g3b748ea0.el9.x8 1.5 MB/s | 891 kB 00:00 2024-05-21T21:38:55.515 INFO:teuthology.orchestra.run.smithi151.stdout:(3/159): ceph-immutable-object-cache-19.0.0-383 871 kB/s | 145 kB 00:00 2024-05-21T21:38:56.374 INFO:teuthology.orchestra.run.smithi151.stdout:(4/159): ceph-mds-19.0.0-3830.g3b748ea0.el9.x86 2.8 MB/s | 2.4 MB 00:00 2024-05-21T21:38:56.512 INFO:teuthology.orchestra.run.smithi184.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-21T21:38:56.512 INFO:teuthology.orchestra.run.smithi184.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-21T21:38:56.523 INFO:teuthology.orchestra.run.smithi184.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-21T21:38:56.524 INFO:teuthology.orchestra.run.smithi184.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-21T21:38:56.525 INFO:teuthology.orchestra.run.smithi184.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-21T21:38:56.526 INFO:teuthology.orchestra.run.smithi184.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-21T21:38:56.527 INFO:teuthology.orchestra.run.smithi184.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-21T21:38:56.530 INFO:teuthology.orchestra.run.smithi184.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-21T21:38:56.530 INFO:teuthology.orchestra.run.smithi184.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-21T21:38:56.593 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2024-05-21T21:38:56.622 INFO:teuthology.orchestra.run.smithi184.stdout:======================================================================================= 2024-05-21T21:38:56.623 INFO:teuthology.orchestra.run.smithi184.stdout: Package Arch Version Repository Size 2024-05-21T21:38:56.623 INFO:teuthology.orchestra.run.smithi184.stdout:======================================================================================= 2024-05-21T21:38:56.623 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2024-05-21T21:38:56.623 INFO:teuthology.orchestra.run.smithi184.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-21T21:38:56.623 INFO:teuthology.orchestra.run.smithi184.stdout: ceph x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 6.6 k 2024-05-21T21:38:56.623 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-base x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 5.4 M 2024-05-21T21:38:56.623 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-fuse x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 891 k 2024-05-21T21:38:56.623 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 145 k 2024-05-21T21:38:56.623 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 1.5 M 2024-05-21T21:38:56.623 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3830.g3b748ea0.el9 ceph-noarch 143 k 2024-05-21T21:38:56.623 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3830.g3b748ea0.el9 ceph-noarch 3.5 M 2024-05-21T21:38:56.623 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3830.g3b748ea0.el9 ceph-noarch 7.4 M 2024-05-21T21:38:56.623 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-rook noarch 2:19.0.0-3830.g3b748ea0.el9 ceph-noarch 49 k 2024-05-21T21:38:56.624 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-radosgw x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 11 M 2024-05-21T21:38:56.624 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-test x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 49 M 2024-05-21T21:38:56.624 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-volume noarch 2:19.0.0-3830.g3b748ea0.el9 ceph-noarch 273 k 2024-05-21T21:38:56.624 INFO:teuthology.orchestra.run.smithi184.stdout: cephadm noarch 2:19.0.0-3830.g3b748ea0.el9 ceph-noarch 764 k 2024-05-21T21:38:56.624 INFO:teuthology.orchestra.run.smithi184.stdout: cephfs-mirror x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 225 k 2024-05-21T21:38:56.624 INFO:teuthology.orchestra.run.smithi184.stdout: cephfs-shell x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 27 k 2024-05-21T21:38:56.624 INFO:teuthology.orchestra.run.smithi184.stdout: cephfs-top noarch 2:19.0.0-3830.g3b748ea0.el9 ceph-noarch 18 k 2024-05-21T21:38:56.624 INFO:teuthology.orchestra.run.smithi184.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-05-21T21:38:56.624 INFO:teuthology.orchestra.run.smithi184.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-21T21:38:56.624 INFO:teuthology.orchestra.run.smithi184.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-21T21:38:56.624 INFO:teuthology.orchestra.run.smithi184.stdout: libcephfs-devel x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 32 k 2024-05-21T21:38:56.624 INFO:teuthology.orchestra.run.smithi184.stdout: libcephfs2 x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 746 k 2024-05-21T21:38:56.625 INFO:teuthology.orchestra.run.smithi184.stdout: librados-devel x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 127 k 2024-05-21T21:38:56.625 INFO:teuthology.orchestra.run.smithi184.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-21T21:38:56.625 INFO:teuthology.orchestra.run.smithi184.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-21T21:38:56.625 INFO:teuthology.orchestra.run.smithi184.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-21T21:38:56.625 INFO:teuthology.orchestra.run.smithi184.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-21T21:38:56.625 INFO:teuthology.orchestra.run.smithi184.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-21T21:38:56.625 INFO:teuthology.orchestra.run.smithi184.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-21T21:38:56.625 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cephfs x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 161 k 2024-05-21T21:38:56.625 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rados x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 320 k 2024-05-21T21:38:56.625 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rbd x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 300 k 2024-05-21T21:38:56.625 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rgw x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 99 k 2024-05-21T21:38:56.625 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-fuse x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 85 k 2024-05-21T21:38:56.625 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-mirror x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 3.1 M 2024-05-21T21:38:56.626 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-nbd x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 171 k 2024-05-21T21:38:56.626 INFO:teuthology.orchestra.run.smithi184.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-21T21:38:56.626 INFO:teuthology.orchestra.run.smithi184.stdout:Upgrading: 2024-05-21T21:38:56.626 INFO:teuthology.orchestra.run.smithi184.stdout: NetworkManager x86_64 1:1.47.90-1.el9 baseos 2.3 M 2024-05-21T21:38:56.626 INFO:teuthology.orchestra.run.smithi184.stdout: NetworkManager-libnm x86_64 1:1.47.90-1.el9 baseos 1.8 M 2024-05-21T21:38:56.626 INFO:teuthology.orchestra.run.smithi184.stdout: NetworkManager-team x86_64 1:1.47.90-1.el9 baseos 41 k 2024-05-21T21:38:56.626 INFO:teuthology.orchestra.run.smithi184.stdout: NetworkManager-tui x86_64 1:1.47.90-1.el9 baseos 248 k 2024-05-21T21:38:56.626 INFO:teuthology.orchestra.run.smithi184.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-05-21T21:38:56.626 INFO:teuthology.orchestra.run.smithi184.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-05-21T21:38:56.626 INFO:teuthology.orchestra.run.smithi184.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-05-21T21:38:56.626 INFO:teuthology.orchestra.run.smithi184.stdout: librados2 x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 3.4 M 2024-05-21T21:38:56.626 INFO:teuthology.orchestra.run.smithi184.stdout: librbd1 x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 3.1 M 2024-05-21T21:38:56.626 INFO:teuthology.orchestra.run.smithi184.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-21T21:38:56.627 INFO:teuthology.orchestra.run.smithi184.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-21T21:38:56.627 INFO:teuthology.orchestra.run.smithi184.stdout:Installing dependencies: 2024-05-21T21:38:56.627 INFO:teuthology.orchestra.run.smithi184.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-21T21:38:56.627 INFO:teuthology.orchestra.run.smithi184.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-21T21:38:56.627 INFO:teuthology.orchestra.run.smithi184.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-21T21:38:56.627 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-common x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 21 M 2024-05-21T21:38:56.627 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3830.g3b748ea0.el9 ceph-noarch 34 k 2024-05-21T21:38:56.627 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mds x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 2.4 M 2024-05-21T21:38:56.627 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3830.g3b748ea0.el9 ceph-noarch 274 k 2024-05-21T21:38:56.627 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mon x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 4.7 M 2024-05-21T21:38:56.627 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-osd x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 17 M 2024-05-21T21:38:56.627 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3830.g3b748ea0.el9 ceph-noarch 16 k 2024-05-21T21:38:56.627 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-selinux x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 25 k 2024-05-21T21:38:56.628 INFO:teuthology.orchestra.run.smithi184.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-21T21:38:56.628 INFO:teuthology.orchestra.run.smithi184.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-21T21:38:56.628 INFO:teuthology.orchestra.run.smithi184.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-21T21:38:56.628 INFO:teuthology.orchestra.run.smithi184.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-21T21:38:56.628 INFO:teuthology.orchestra.run.smithi184.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-21T21:38:56.628 INFO:teuthology.orchestra.run.smithi184.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-21T21:38:56.628 INFO:teuthology.orchestra.run.smithi184.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-21T21:38:56.628 INFO:teuthology.orchestra.run.smithi184.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-21T21:38:56.628 INFO:teuthology.orchestra.run.smithi184.stdout: libcephsqlite x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 163 k 2024-05-21T21:38:56.628 INFO:teuthology.orchestra.run.smithi184.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-21T21:38:56.628 INFO:teuthology.orchestra.run.smithi184.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-21T21:38:56.628 INFO:teuthology.orchestra.run.smithi184.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-21T21:38:56.628 INFO:teuthology.orchestra.run.smithi184.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-21T21:38:56.629 INFO:teuthology.orchestra.run.smithi184.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-21T21:38:56.629 INFO:teuthology.orchestra.run.smithi184.stdout: libradosstriper1 x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 501 k 2024-05-21T21:38:56.629 INFO:teuthology.orchestra.run.smithi184.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-21T21:38:56.629 INFO:teuthology.orchestra.run.smithi184.stdout: librgw2 x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 5.4 M 2024-05-21T21:38:56.629 INFO:teuthology.orchestra.run.smithi184.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-21T21:38:56.629 INFO:teuthology.orchestra.run.smithi184.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-21T21:38:56.629 INFO:teuthology.orchestra.run.smithi184.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-21T21:38:56.629 INFO:teuthology.orchestra.run.smithi184.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-21T21:38:56.629 INFO:teuthology.orchestra.run.smithi184.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-21T21:38:56.629 INFO:teuthology.orchestra.run.smithi184.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-05-21T21:38:56.629 INFO:teuthology.orchestra.run.smithi184.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-21T21:38:56.629 INFO:teuthology.orchestra.run.smithi184.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-21T21:38:56.629 INFO:teuthology.orchestra.run.smithi184.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-21T21:38:56.629 INFO:teuthology.orchestra.run.smithi184.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-21T21:38:56.630 INFO:teuthology.orchestra.run.smithi184.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-21T21:38:56.630 INFO:teuthology.orchestra.run.smithi184.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-21T21:38:56.630 INFO:teuthology.orchestra.run.smithi184.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-21T21:38:56.630 INFO:teuthology.orchestra.run.smithi184.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-21T21:38:56.630 INFO:teuthology.orchestra.run.smithi184.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-21T21:38:56.630 INFO:teuthology.orchestra.run.smithi184.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-21T21:38:56.630 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-21T21:38:56.630 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ceph-argparse x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 45 k 2024-05-21T21:38:56.630 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ceph-common x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 136 k 2024-05-21T21:38:56.630 INFO:teuthology.orchestra.run.smithi184.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-21T21:38:56.630 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-21T21:38:56.630 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-21T21:38:56.630 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-21T21:38:56.631 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-21T21:38:56.631 INFO:teuthology.orchestra.run.smithi184.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-05-21T21:38:56.631 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-21T21:38:56.631 INFO:teuthology.orchestra.run.smithi184.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-21T21:38:56.631 INFO:teuthology.orchestra.run.smithi184.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-21T21:38:56.631 INFO:teuthology.orchestra.run.smithi184.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-21T21:38:56.631 INFO:teuthology.orchestra.run.smithi184.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-21T21:38:56.631 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-21T21:38:56.631 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-21T21:38:56.631 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-21T21:38:56.631 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-21T21:38:56.631 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-21T21:38:56.631 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-05-21T21:38:56.631 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-21T21:38:56.632 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-21T21:38:56.632 INFO:teuthology.orchestra.run.smithi184.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-21T21:38:56.632 INFO:teuthology.orchestra.run.smithi184.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-21T21:38:56.632 INFO:teuthology.orchestra.run.smithi184.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-21T21:38:56.632 INFO:teuthology.orchestra.run.smithi184.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-21T21:38:56.632 INFO:teuthology.orchestra.run.smithi184.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-21T21:38:56.632 INFO:teuthology.orchestra.run.smithi184.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-21T21:38:56.632 INFO:teuthology.orchestra.run.smithi184.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-21T21:38:56.632 INFO:teuthology.orchestra.run.smithi184.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-21T21:38:56.632 INFO:teuthology.orchestra.run.smithi184.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-21T21:38:56.632 INFO:teuthology.orchestra.run.smithi184.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-21T21:38:56.632 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-21T21:38:56.632 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-21T21:38:56.633 INFO:teuthology.orchestra.run.smithi184.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-21T21:38:56.633 INFO:teuthology.orchestra.run.smithi184.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-21T21:38:56.633 INFO:teuthology.orchestra.run.smithi184.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-21T21:38:56.633 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-21T21:38:56.633 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-21T21:38:56.633 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-21T21:38:56.633 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-21T21:38:56.633 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-21T21:38:56.633 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-21T21:38:56.633 INFO:teuthology.orchestra.run.smithi184.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-21T21:38:56.633 INFO:teuthology.orchestra.run.smithi184.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-21T21:38:56.633 INFO:teuthology.orchestra.run.smithi184.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-21T21:38:56.633 INFO:teuthology.orchestra.run.smithi184.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-21T21:38:56.633 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-21T21:38:56.634 INFO:teuthology.orchestra.run.smithi184.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-21T21:38:56.634 INFO:teuthology.orchestra.run.smithi184.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-21T21:38:56.634 INFO:teuthology.orchestra.run.smithi184.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-21T21:38:56.634 INFO:teuthology.orchestra.run.smithi184.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-21T21:38:56.634 INFO:teuthology.orchestra.run.smithi184.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-21T21:38:56.634 INFO:teuthology.orchestra.run.smithi184.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-21T21:38:56.634 INFO:teuthology.orchestra.run.smithi184.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-21T21:38:56.634 INFO:teuthology.orchestra.run.smithi184.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-21T21:38:56.634 INFO:teuthology.orchestra.run.smithi184.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-21T21:38:56.634 INFO:teuthology.orchestra.run.smithi184.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-21T21:38:56.634 INFO:teuthology.orchestra.run.smithi184.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-21T21:38:56.634 INFO:teuthology.orchestra.run.smithi184.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-21T21:38:56.634 INFO:teuthology.orchestra.run.smithi184.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-21T21:38:56.634 INFO:teuthology.orchestra.run.smithi184.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-21T21:38:56.635 INFO:teuthology.orchestra.run.smithi184.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-21T21:38:56.635 INFO:teuthology.orchestra.run.smithi184.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-21T21:38:56.635 INFO:teuthology.orchestra.run.smithi184.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-21T21:38:56.635 INFO:teuthology.orchestra.run.smithi184.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-21T21:38:56.635 INFO:teuthology.orchestra.run.smithi184.stdout:Installing weak dependencies: 2024-05-21T21:38:56.635 INFO:teuthology.orchestra.run.smithi184.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-21T21:38:56.635 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-05-21T21:38:56.635 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2024-05-21T21:38:56.635 INFO:teuthology.orchestra.run.smithi184.stdout:======================================================================================= 2024-05-21T21:38:56.635 INFO:teuthology.orchestra.run.smithi184.stdout:Install 148 Packages 2024-05-21T21:38:56.635 INFO:teuthology.orchestra.run.smithi184.stdout:Upgrade 11 Packages 2024-05-21T21:38:56.635 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-05-21T21:38:56.636 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 227 M 2024-05-21T21:38:56.636 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2024-05-21T21:38:56.857 INFO:teuthology.orchestra.run.smithi151.stdout:(5/159): ceph-mgr-19.0.0-3830.g3b748ea0.el9.x86 3.1 MB/s | 1.5 MB 00:00 2024-05-21T21:38:57.452 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - Extras packages 13 kB/s | 16 kB 00:01 2024-05-21T21:38:57.983 INFO:teuthology.orchestra.run.smithi151.stdout:(6/159): ceph-base-19.0.0-3830.g3b748ea0.el9.x8 1.5 MB/s | 5.4 MB 00:03 2024-05-21T21:38:58.043 INFO:teuthology.orchestra.run.smithi184.stdout:(1/159): ceph-19.0.0-3830.g3b748ea0.el9.x86_64. 11 kB/s | 6.6 kB 00:00 2024-05-21T21:38:58.091 INFO:teuthology.orchestra.run.smithi151.stdout:(7/159): ceph-mon-19.0.0-3830.g3b748ea0.el9.x86 3.8 MB/s | 4.7 MB 00:01 2024-05-21T21:38:58.495 INFO:teuthology.orchestra.run.smithi184.stdout:(2/159): ceph-fuse-19.0.0-3830.g3b748ea0.el9.x8 1.7 MB/s | 891 kB 00:00 2024-05-21T21:38:58.515 INFO:teuthology.orchestra.run.smithi164.stdout:Extra Packages for Enterprise Linux 22 MB/s | 21 MB 00:00 2024-05-21T21:38:58.595 INFO:teuthology.orchestra.run.smithi184.stdout:(3/159): ceph-immutable-object-cache-19.0.0-383 1.4 MB/s | 145 kB 00:00 2024-05-21T21:38:59.371 INFO:teuthology.orchestra.run.smithi184.stdout:(4/159): ceph-mds-19.0.0-3830.g3b748ea0.el9.x86 3.1 MB/s | 2.4 MB 00:00 2024-05-21T21:38:59.921 INFO:teuthology.orchestra.run.smithi184.stdout:(5/159): ceph-mgr-19.0.0-3830.g3b748ea0.el9.x86 2.8 MB/s | 1.5 MB 00:00 2024-05-21T21:39:00.996 INFO:teuthology.orchestra.run.smithi184.stdout:(6/159): ceph-base-19.0.0-3830.g3b748ea0.el9.x8 1.5 MB/s | 5.4 MB 00:03 2024-05-21T21:39:01.525 INFO:teuthology.orchestra.run.smithi151.stdout:(8/159): ceph-radosgw-19.0.0-3830.g3b748ea0.el9 3.1 MB/s | 11 MB 00:03 2024-05-21T21:39:01.617 INFO:teuthology.orchestra.run.smithi151.stdout:(9/159): ceph-selinux-19.0.0-3830.g3b748ea0.el9 278 kB/s | 25 kB 00:00 2024-05-21T21:39:01.846 INFO:teuthology.orchestra.run.smithi184.stdout:(7/159): ceph-mon-19.0.0-3830.g3b748ea0.el9.x86 2.4 MB/s | 4.7 MB 00:01 2024-05-21T21:39:03.042 INFO:teuthology.orchestra.run.smithi151.stdout:(10/159): ceph-common-19.0.0-3830.g3b748ea0.el9 2.5 MB/s | 21 MB 00:08 2024-05-21T21:39:03.183 INFO:teuthology.orchestra.run.smithi151.stdout:(11/159): cephfs-mirror-19.0.0-3830.g3b748ea0.e 1.6 MB/s | 225 kB 00:00 2024-05-21T21:39:03.250 INFO:teuthology.orchestra.run.smithi151.stdout:(12/159): cephfs-shell-19.0.0-3830.g3b748ea0.el 411 kB/s | 27 kB 00:00 2024-05-21T21:39:03.309 INFO:teuthology.orchestra.run.smithi151.stdout:(13/159): libcephfs-devel-19.0.0-3830.g3b748ea0 550 kB/s | 32 kB 00:00 2024-05-21T21:39:03.568 INFO:teuthology.orchestra.run.smithi151.stdout:(14/159): libcephfs2-19.0.0-3830.g3b748ea0.el9. 2.8 MB/s | 746 kB 00:00 2024-05-21T21:39:03.643 INFO:teuthology.orchestra.run.smithi151.stdout:(15/159): libcephsqlite-19.0.0-3830.g3b748ea0.e 2.2 MB/s | 163 kB 00:00 2024-05-21T21:39:03.718 INFO:teuthology.orchestra.run.smithi151.stdout:(16/159): librados-devel-19.0.0-3830.g3b748ea0. 1.7 MB/s | 127 kB 00:00 2024-05-21T21:39:03.729 INFO:teuthology.orchestra.run.smithi164.stdout:lab-extras 17 kB/s | 1.7 kB 00:00 2024-05-21T21:39:03.885 INFO:teuthology.orchestra.run.smithi151.stdout:(17/159): libradosstriper1-19.0.0-3830.g3b748ea 3.0 MB/s | 501 kB 00:00 2024-05-21T21:39:05.229 INFO:teuthology.orchestra.run.smithi184.stdout:(8/159): ceph-radosgw-19.0.0-3830.g3b748ea0.el9 3.1 MB/s | 11 MB 00:03 2024-05-21T21:39:05.313 INFO:teuthology.orchestra.run.smithi184.stdout:(9/159): ceph-selinux-19.0.0-3830.g3b748ea0.el9 302 kB/s | 25 kB 00:00 2024-05-21T21:39:05.444 INFO:teuthology.orchestra.run.smithi151.stdout:(18/159): librgw2-19.0.0-3830.g3b748ea0.el9.x86 3.4 MB/s | 5.4 MB 00:01 2024-05-21T21:39:05.511 INFO:teuthology.orchestra.run.smithi151.stdout:(19/159): python3-ceph-argparse-19.0.0-3830.g3b 689 kB/s | 45 kB 00:00 2024-05-21T21:39:05.594 INFO:teuthology.orchestra.run.smithi151.stdout:(20/159): python3-ceph-common-19.0.0-3830.g3b74 1.6 MB/s | 136 kB 00:00 2024-05-21T21:39:05.678 INFO:teuthology.orchestra.run.smithi151.stdout:(21/159): python3-cephfs-19.0.0-3830.g3b748ea0. 1.9 MB/s | 161 kB 00:00 2024-05-21T21:39:05.828 INFO:teuthology.orchestra.run.smithi151.stdout:(22/159): python3-rados-19.0.0-3830.g3b748ea0.e 2.1 MB/s | 320 kB 00:00 2024-05-21T21:39:05.830 INFO:teuthology.orchestra.run.smithi164.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-21T21:39:05.831 INFO:teuthology.orchestra.run.smithi164.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-21T21:39:05.841 INFO:teuthology.orchestra.run.smithi164.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-21T21:39:05.842 INFO:teuthology.orchestra.run.smithi164.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-21T21:39:05.843 INFO:teuthology.orchestra.run.smithi164.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-21T21:39:05.844 INFO:teuthology.orchestra.run.smithi164.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-21T21:39:05.845 INFO:teuthology.orchestra.run.smithi164.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-21T21:39:05.848 INFO:teuthology.orchestra.run.smithi164.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-21T21:39:05.848 INFO:teuthology.orchestra.run.smithi164.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-21T21:39:05.911 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-05-21T21:39:05.932 INFO:teuthology.orchestra.run.smithi164.stdout:======================================================================================= 2024-05-21T21:39:05.932 INFO:teuthology.orchestra.run.smithi164.stdout: Package Arch Version Repository Size 2024-05-21T21:39:05.933 INFO:teuthology.orchestra.run.smithi164.stdout:======================================================================================= 2024-05-21T21:39:05.933 INFO:teuthology.orchestra.run.smithi164.stdout:Installing: 2024-05-21T21:39:05.933 INFO:teuthology.orchestra.run.smithi164.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-21T21:39:05.933 INFO:teuthology.orchestra.run.smithi164.stdout: ceph x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 6.6 k 2024-05-21T21:39:05.933 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-base x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 5.4 M 2024-05-21T21:39:05.933 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-fuse x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 891 k 2024-05-21T21:39:05.933 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 145 k 2024-05-21T21:39:05.933 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 1.5 M 2024-05-21T21:39:05.933 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3830.g3b748ea0.el9 ceph-noarch 143 k 2024-05-21T21:39:05.933 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3830.g3b748ea0.el9 ceph-noarch 3.5 M 2024-05-21T21:39:05.933 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3830.g3b748ea0.el9 ceph-noarch 7.4 M 2024-05-21T21:39:05.933 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-rook noarch 2:19.0.0-3830.g3b748ea0.el9 ceph-noarch 49 k 2024-05-21T21:39:05.933 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-radosgw x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 11 M 2024-05-21T21:39:05.934 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-test x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 49 M 2024-05-21T21:39:05.934 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-volume noarch 2:19.0.0-3830.g3b748ea0.el9 ceph-noarch 273 k 2024-05-21T21:39:05.934 INFO:teuthology.orchestra.run.smithi164.stdout: cephadm noarch 2:19.0.0-3830.g3b748ea0.el9 ceph-noarch 764 k 2024-05-21T21:39:05.934 INFO:teuthology.orchestra.run.smithi164.stdout: cephfs-mirror x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 225 k 2024-05-21T21:39:05.934 INFO:teuthology.orchestra.run.smithi164.stdout: cephfs-shell x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 27 k 2024-05-21T21:39:05.934 INFO:teuthology.orchestra.run.smithi164.stdout: cephfs-top noarch 2:19.0.0-3830.g3b748ea0.el9 ceph-noarch 18 k 2024-05-21T21:39:05.934 INFO:teuthology.orchestra.run.smithi164.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-05-21T21:39:05.934 INFO:teuthology.orchestra.run.smithi164.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-21T21:39:05.934 INFO:teuthology.orchestra.run.smithi164.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-21T21:39:05.934 INFO:teuthology.orchestra.run.smithi164.stdout: libcephfs-devel x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 32 k 2024-05-21T21:39:05.934 INFO:teuthology.orchestra.run.smithi164.stdout: libcephfs2 x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 746 k 2024-05-21T21:39:05.935 INFO:teuthology.orchestra.run.smithi164.stdout: librados-devel x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 127 k 2024-05-21T21:39:05.935 INFO:teuthology.orchestra.run.smithi164.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-21T21:39:05.935 INFO:teuthology.orchestra.run.smithi164.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-21T21:39:05.935 INFO:teuthology.orchestra.run.smithi164.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-21T21:39:05.935 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-21T21:39:05.935 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-21T21:39:05.935 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-21T21:39:05.935 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cephfs x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 161 k 2024-05-21T21:39:05.935 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rados x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 320 k 2024-05-21T21:39:05.935 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rbd x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 300 k 2024-05-21T21:39:05.935 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rgw x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 99 k 2024-05-21T21:39:05.935 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-fuse x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 85 k 2024-05-21T21:39:05.936 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-mirror x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 3.1 M 2024-05-21T21:39:05.936 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-nbd x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 171 k 2024-05-21T21:39:05.936 INFO:teuthology.orchestra.run.smithi164.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-21T21:39:05.936 INFO:teuthology.orchestra.run.smithi164.stdout:Upgrading: 2024-05-21T21:39:05.936 INFO:teuthology.orchestra.run.smithi164.stdout: NetworkManager x86_64 1:1.47.90-1.el9 baseos 2.3 M 2024-05-21T21:39:05.936 INFO:teuthology.orchestra.run.smithi164.stdout: NetworkManager-libnm x86_64 1:1.47.90-1.el9 baseos 1.8 M 2024-05-21T21:39:05.936 INFO:teuthology.orchestra.run.smithi164.stdout: NetworkManager-team x86_64 1:1.47.90-1.el9 baseos 41 k 2024-05-21T21:39:05.936 INFO:teuthology.orchestra.run.smithi164.stdout: NetworkManager-tui x86_64 1:1.47.90-1.el9 baseos 248 k 2024-05-21T21:39:05.936 INFO:teuthology.orchestra.run.smithi164.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-05-21T21:39:05.936 INFO:teuthology.orchestra.run.smithi164.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-05-21T21:39:05.936 INFO:teuthology.orchestra.run.smithi164.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-05-21T21:39:05.936 INFO:teuthology.orchestra.run.smithi164.stdout: librados2 x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 3.4 M 2024-05-21T21:39:05.936 INFO:teuthology.orchestra.run.smithi164.stdout: librbd1 x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 3.1 M 2024-05-21T21:39:05.937 INFO:teuthology.orchestra.run.smithi164.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-21T21:39:05.937 INFO:teuthology.orchestra.run.smithi164.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-21T21:39:05.937 INFO:teuthology.orchestra.run.smithi164.stdout:Installing dependencies: 2024-05-21T21:39:05.937 INFO:teuthology.orchestra.run.smithi164.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-21T21:39:05.937 INFO:teuthology.orchestra.run.smithi164.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-21T21:39:05.937 INFO:teuthology.orchestra.run.smithi164.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-21T21:39:05.937 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-common x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 21 M 2024-05-21T21:39:05.937 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3830.g3b748ea0.el9 ceph-noarch 34 k 2024-05-21T21:39:05.937 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mds x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 2.4 M 2024-05-21T21:39:05.937 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3830.g3b748ea0.el9 ceph-noarch 274 k 2024-05-21T21:39:05.937 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mon x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 4.7 M 2024-05-21T21:39:05.937 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-osd x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 17 M 2024-05-21T21:39:05.937 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3830.g3b748ea0.el9 ceph-noarch 16 k 2024-05-21T21:39:05.938 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-selinux x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 25 k 2024-05-21T21:39:05.938 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-21T21:39:05.938 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-21T21:39:05.938 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-21T21:39:05.938 INFO:teuthology.orchestra.run.smithi164.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-21T21:39:05.938 INFO:teuthology.orchestra.run.smithi164.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-21T21:39:05.938 INFO:teuthology.orchestra.run.smithi164.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-21T21:39:05.938 INFO:teuthology.orchestra.run.smithi164.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-21T21:39:05.938 INFO:teuthology.orchestra.run.smithi164.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-21T21:39:05.938 INFO:teuthology.orchestra.run.smithi164.stdout: libcephsqlite x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 163 k 2024-05-21T21:39:05.938 INFO:teuthology.orchestra.run.smithi164.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-21T21:39:05.938 INFO:teuthology.orchestra.run.smithi164.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-21T21:39:05.939 INFO:teuthology.orchestra.run.smithi164.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-21T21:39:05.939 INFO:teuthology.orchestra.run.smithi164.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-21T21:39:05.939 INFO:teuthology.orchestra.run.smithi164.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-21T21:39:05.939 INFO:teuthology.orchestra.run.smithi164.stdout: libradosstriper1 x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 501 k 2024-05-21T21:39:05.939 INFO:teuthology.orchestra.run.smithi164.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-21T21:39:05.939 INFO:teuthology.orchestra.run.smithi164.stdout: librgw2 x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 5.4 M 2024-05-21T21:39:05.939 INFO:teuthology.orchestra.run.smithi164.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-21T21:39:05.939 INFO:teuthology.orchestra.run.smithi164.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-21T21:39:05.939 INFO:teuthology.orchestra.run.smithi164.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-21T21:39:05.939 INFO:teuthology.orchestra.run.smithi164.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-21T21:39:05.939 INFO:teuthology.orchestra.run.smithi164.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-21T21:39:05.939 INFO:teuthology.orchestra.run.smithi164.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-05-21T21:39:05.940 INFO:teuthology.orchestra.run.smithi164.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-21T21:39:05.940 INFO:teuthology.orchestra.run.smithi164.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-21T21:39:05.940 INFO:teuthology.orchestra.run.smithi164.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-21T21:39:05.940 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-21T21:39:05.940 INFO:teuthology.orchestra.run.smithi164.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-21T21:39:05.940 INFO:teuthology.orchestra.run.smithi164.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-21T21:39:05.940 INFO:teuthology.orchestra.run.smithi164.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-21T21:39:05.940 INFO:teuthology.orchestra.run.smithi164.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-21T21:39:05.940 INFO:teuthology.orchestra.run.smithi164.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-21T21:39:05.940 INFO:teuthology.orchestra.run.smithi164.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-21T21:39:05.940 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-21T21:39:05.940 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ceph-argparse x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 45 k 2024-05-21T21:39:05.940 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ceph-common x86_64 2:19.0.0-3830.g3b748ea0.el9 ceph 136 k 2024-05-21T21:39:05.941 INFO:teuthology.orchestra.run.smithi164.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-21T21:39:05.941 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-21T21:39:05.941 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-21T21:39:05.941 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-21T21:39:05.941 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-21T21:39:05.941 INFO:teuthology.orchestra.run.smithi164.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-05-21T21:39:05.941 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-21T21:39:05.941 INFO:teuthology.orchestra.run.smithi164.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-21T21:39:05.941 INFO:teuthology.orchestra.run.smithi164.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-21T21:39:05.941 INFO:teuthology.orchestra.run.smithi164.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-21T21:39:05.941 INFO:teuthology.orchestra.run.smithi164.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-21T21:39:05.941 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-21T21:39:05.942 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-21T21:39:05.942 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-21T21:39:05.942 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-21T21:39:05.942 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-21T21:39:05.942 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-05-21T21:39:05.942 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-21T21:39:05.942 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-21T21:39:05.942 INFO:teuthology.orchestra.run.smithi164.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-21T21:39:05.942 INFO:teuthology.orchestra.run.smithi164.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-21T21:39:05.942 INFO:teuthology.orchestra.run.smithi164.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-21T21:39:05.942 INFO:teuthology.orchestra.run.smithi164.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-21T21:39:05.942 INFO:teuthology.orchestra.run.smithi164.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-21T21:39:05.943 INFO:teuthology.orchestra.run.smithi164.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-21T21:39:05.943 INFO:teuthology.orchestra.run.smithi164.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-21T21:39:05.943 INFO:teuthology.orchestra.run.smithi164.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-21T21:39:05.943 INFO:teuthology.orchestra.run.smithi164.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-21T21:39:05.943 INFO:teuthology.orchestra.run.smithi164.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-21T21:39:05.943 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-21T21:39:05.943 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-21T21:39:05.943 INFO:teuthology.orchestra.run.smithi164.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-21T21:39:05.943 INFO:teuthology.orchestra.run.smithi164.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-21T21:39:05.943 INFO:teuthology.orchestra.run.smithi164.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-21T21:39:05.943 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-21T21:39:05.943 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-21T21:39:05.943 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-21T21:39:05.944 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-21T21:39:05.944 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-21T21:39:05.944 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-21T21:39:05.944 INFO:teuthology.orchestra.run.smithi164.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-21T21:39:05.944 INFO:teuthology.orchestra.run.smithi164.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-21T21:39:05.944 INFO:teuthology.orchestra.run.smithi164.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-21T21:39:05.944 INFO:teuthology.orchestra.run.smithi164.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-21T21:39:05.944 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-21T21:39:05.944 INFO:teuthology.orchestra.run.smithi164.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-21T21:39:05.944 INFO:teuthology.orchestra.run.smithi164.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-21T21:39:05.944 INFO:teuthology.orchestra.run.smithi164.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-21T21:39:05.944 INFO:teuthology.orchestra.run.smithi164.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-21T21:39:05.944 INFO:teuthology.orchestra.run.smithi164.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-21T21:39:05.944 INFO:teuthology.orchestra.run.smithi164.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-21T21:39:05.944 INFO:teuthology.orchestra.run.smithi164.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-21T21:39:05.945 INFO:teuthology.orchestra.run.smithi164.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-21T21:39:05.945 INFO:teuthology.orchestra.run.smithi164.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-21T21:39:05.945 INFO:teuthology.orchestra.run.smithi164.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-21T21:39:05.945 INFO:teuthology.orchestra.run.smithi164.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-21T21:39:05.945 INFO:teuthology.orchestra.run.smithi164.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-21T21:39:05.945 INFO:teuthology.orchestra.run.smithi164.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-21T21:39:05.945 INFO:teuthology.orchestra.run.smithi164.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-21T21:39:05.945 INFO:teuthology.orchestra.run.smithi164.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-21T21:39:05.945 INFO:teuthology.orchestra.run.smithi164.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-21T21:39:05.945 INFO:teuthology.orchestra.run.smithi164.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-21T21:39:05.945 INFO:teuthology.orchestra.run.smithi164.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-21T21:39:05.945 INFO:teuthology.orchestra.run.smithi164.stdout:Installing weak dependencies: 2024-05-21T21:39:05.945 INFO:teuthology.orchestra.run.smithi164.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-21T21:39:05.946 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-21T21:39:05.946 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction Summary 2024-05-21T21:39:05.946 INFO:teuthology.orchestra.run.smithi164.stdout:======================================================================================= 2024-05-21T21:39:05.946 INFO:teuthology.orchestra.run.smithi164.stdout:Install 148 Packages 2024-05-21T21:39:05.946 INFO:teuthology.orchestra.run.smithi164.stdout:Upgrade 11 Packages 2024-05-21T21:39:05.946 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-21T21:39:05.946 INFO:teuthology.orchestra.run.smithi164.stdout:Total download size: 227 M 2024-05-21T21:39:05.947 INFO:teuthology.orchestra.run.smithi164.stdout:Downloading Packages: 2024-05-21T21:39:06.070 INFO:teuthology.orchestra.run.smithi151.stdout:(23/159): ceph-osd-19.0.0-3830.g3b748ea0.el9.x8 2.1 MB/s | 17 MB 00:08 2024-05-21T21:39:06.112 INFO:teuthology.orchestra.run.smithi151.stdout:(24/159): python3-rbd-19.0.0-3830.g3b748ea0.el9 1.0 MB/s | 300 kB 00:00 2024-05-21T21:39:06.196 INFO:teuthology.orchestra.run.smithi151.stdout:(25/159): rbd-fuse-19.0.0-3830.g3b748ea0.el9.x8 1.0 MB/s | 85 kB 00:00 2024-05-21T21:39:06.213 INFO:teuthology.orchestra.run.smithi151.stdout:(26/159): python3-rgw-19.0.0-3830.g3b748ea0.el9 704 kB/s | 99 kB 00:00 2024-05-21T21:39:06.296 INFO:teuthology.orchestra.run.smithi184.stdout:(10/159): ceph-osd-19.0.0-3830.g3b748ea0.el9.x8 3.2 MB/s | 17 MB 00:05 2024-05-21T21:39:06.371 INFO:teuthology.orchestra.run.smithi151.stdout:(27/159): rbd-nbd-19.0.0-3830.g3b748ea0.el9.x86 1.1 MB/s | 171 kB 00:00 2024-05-21T21:39:06.455 INFO:teuthology.orchestra.run.smithi151.stdout:(28/159): ceph-grafana-dashboards-19.0.0-3830.g 413 kB/s | 34 kB 00:00 2024-05-21T21:39:06.455 INFO:teuthology.orchestra.run.smithi184.stdout:(11/159): cephfs-mirror-19.0.0-3830.g3b748ea0.e 1.4 MB/s | 225 kB 00:00 2024-05-21T21:39:06.547 INFO:teuthology.orchestra.run.smithi184.stdout:(12/159): cephfs-shell-19.0.0-3830.g3b748ea0.el 295 kB/s | 27 kB 00:00 2024-05-21T21:39:06.547 INFO:teuthology.orchestra.run.smithi151.stdout:(29/159): ceph-mgr-cephadm-19.0.0-3830.g3b748ea 1.5 MB/s | 143 kB 00:00 2024-05-21T21:39:06.580 INFO:teuthology.orchestra.run.smithi164.stdout:(1/159): ceph-19.0.0-3830.g3b748ea0.el9.x86_64. 19 kB/s | 6.6 kB 00:00 2024-05-21T21:39:06.622 INFO:teuthology.orchestra.run.smithi184.stdout:(13/159): libcephfs-devel-19.0.0-3830.g3b748ea0 423 kB/s | 32 kB 00:00 2024-05-21T21:39:07.131 INFO:teuthology.orchestra.run.smithi184.stdout:(14/159): ceph-common-19.0.0-3830.g3b748ea0.el9 2.2 MB/s | 21 MB 00:09 2024-05-21T21:39:07.156 INFO:teuthology.orchestra.run.smithi184.stdout:(15/159): libcephfs2-19.0.0-3830.g3b748ea0.el9. 1.4 MB/s | 746 kB 00:00 2024-05-21T21:39:07.231 INFO:teuthology.orchestra.run.smithi184.stdout:(16/159): libcephsqlite-19.0.0-3830.g3b748ea0.e 1.6 MB/s | 163 kB 00:00 2024-05-21T21:39:07.239 INFO:teuthology.orchestra.run.smithi151.stdout:(30/159): rbd-mirror-19.0.0-3830.g3b748ea0.el9. 3.0 MB/s | 3.1 MB 00:01 2024-05-21T21:39:07.257 INFO:teuthology.orchestra.run.smithi184.stdout:(17/159): librados-devel-19.0.0-3830.g3b748ea0. 1.2 MB/s | 127 kB 00:00 2024-05-21T21:39:07.507 INFO:teuthology.orchestra.run.smithi184.stdout:(18/159): libradosstriper1-19.0.0-3830.g3b748ea 1.8 MB/s | 501 kB 00:00 2024-05-21T21:39:07.599 INFO:teuthology.orchestra.run.smithi184.stdout:(19/159): python3-ceph-argparse-19.0.0-3830.g3b 493 kB/s | 45 kB 00:00 2024-05-21T21:39:07.605 INFO:teuthology.orchestra.run.smithi164.stdout:(2/159): ceph-fuse-19.0.0-3830.g3b748ea0.el9.x8 870 kB/s | 891 kB 00:01 2024-05-21T21:39:07.703 INFO:teuthology.orchestra.run.smithi184.stdout:(20/159): python3-ceph-common-19.0.0-3830.g3b74 1.3 MB/s | 136 kB 00:00 2024-05-21T21:39:07.789 INFO:teuthology.orchestra.run.smithi164.stdout:(3/159): ceph-immutable-object-cache-19.0.0-383 790 kB/s | 145 kB 00:00 2024-05-21T21:39:07.807 INFO:teuthology.orchestra.run.smithi184.stdout:(21/159): python3-cephfs-19.0.0-3830.g3b748ea0. 1.5 MB/s | 161 kB 00:00 2024-05-21T21:39:07.983 INFO:teuthology.orchestra.run.smithi184.stdout:(22/159): python3-rados-19.0.0-3830.g3b748ea0.e 1.8 MB/s | 320 kB 00:00 2024-05-21T21:39:08.159 INFO:teuthology.orchestra.run.smithi184.stdout:(23/159): python3-rbd-19.0.0-3830.g3b748ea0.el9 1.7 MB/s | 300 kB 00:00 2024-05-21T21:39:08.259 INFO:teuthology.orchestra.run.smithi184.stdout:(24/159): python3-rgw-19.0.0-3830.g3b748ea0.el9 995 kB/s | 99 kB 00:00 2024-05-21T21:39:08.360 INFO:teuthology.orchestra.run.smithi184.stdout:(25/159): rbd-fuse-19.0.0-3830.g3b748ea0.el9.x8 852 kB/s | 85 kB 00:00 2024-05-21T21:39:08.398 INFO:teuthology.orchestra.run.smithi151.stdout:(31/159): ceph-mgr-dashboard-19.0.0-3830.g3b748 1.9 MB/s | 3.5 MB 00:01 2024-05-21T21:39:08.598 INFO:teuthology.orchestra.run.smithi151.stdout:(32/159): ceph-mgr-modules-core-19.0.0-3830.g3b 1.3 MB/s | 274 kB 00:00 2024-05-21T21:39:08.819 INFO:teuthology.orchestra.run.smithi151.stdout:(33/159): ceph-mgr-rook-19.0.0-3830.g3b748ea0.e 223 kB/s | 49 kB 00:00 2024-05-21T21:39:08.957 INFO:teuthology.orchestra.run.smithi151.stdout:(34/159): ceph-prometheus-alerts-19.0.0-3830.g3 121 kB/s | 16 kB 00:00 2024-05-21T21:39:09.157 INFO:teuthology.orchestra.run.smithi151.stdout:(35/159): ceph-volume-19.0.0-3830.g3b748ea0.el9 1.3 MB/s | 273 kB 00:00 2024-05-21T21:39:09.189 INFO:teuthology.orchestra.run.smithi164.stdout:(4/159): ceph-base-19.0.0-3830.g3b748ea0.el9.x8 1.8 MB/s | 5.4 MB 00:02 2024-05-21T21:39:09.239 INFO:teuthology.orchestra.run.smithi164.stdout:(5/159): ceph-mds-19.0.0-3830.g3b748ea0.el9.x86 1.6 MB/s | 2.4 MB 00:01 2024-05-21T21:39:09.259 INFO:teuthology.orchestra.run.smithi184.stdout:(26/159): librgw2-19.0.0-3830.g3b748ea0.el9.x86 2.7 MB/s | 5.4 MB 00:02 2024-05-21T21:39:09.607 INFO:teuthology.orchestra.run.smithi151.stdout:(36/159): cephadm-19.0.0-3830.g3b748ea0.el9.noa 1.7 MB/s | 764 kB 00:00 2024-05-21T21:39:09.750 INFO:teuthology.orchestra.run.smithi151.stdout:(37/159): ceph-mgr-diskprediction-local-19.0.0- 3.0 MB/s | 7.4 MB 00:02 2024-05-21T21:39:09.775 INFO:teuthology.orchestra.run.smithi151.stdout:(38/159): cephfs-top-19.0.0-3830.g3b748ea0.el9. 109 kB/s | 18 kB 00:00 2024-05-21T21:39:09.810 INFO:teuthology.orchestra.run.smithi184.stdout:(27/159): rbd-mirror-19.0.0-3830.g3b748ea0.el9. 2.1 MB/s | 3.1 MB 00:01 2024-05-21T21:39:09.885 INFO:teuthology.orchestra.run.smithi184.stdout:(28/159): ceph-grafana-dashboards-19.0.0-3830.g 458 kB/s | 34 kB 00:00 2024-05-21T21:39:09.900 INFO:teuthology.orchestra.run.smithi151.stdout:(39/159): fuse-2.9.9-15.el9.x86_64.rpm 539 kB/s | 80 kB 00:00 2024-05-21T21:39:09.964 INFO:teuthology.orchestra.run.smithi164.stdout:(6/159): ceph-common-19.0.0-3830.g3b748ea0.el9. 5.7 MB/s | 21 MB 00:03 2024-05-21T21:39:09.969 INFO:teuthology.orchestra.run.smithi184.stdout:(29/159): ceph-mgr-cephadm-19.0.0-3830.g3b748ea 1.7 MB/s | 143 kB 00:00 2024-05-21T21:39:10.006 INFO:teuthology.orchestra.run.smithi164.stdout:(7/159): ceph-mgr-19.0.0-3830.g3b748ea0.el9.x86 1.9 MB/s | 1.5 MB 00:00 2024-05-21T21:39:10.009 INFO:teuthology.orchestra.run.smithi151.stdout:(40/159): libgfortran-11.4.1-3.el9.x86_64.rpm 3.4 MB/s | 803 kB 00:00 2024-05-21T21:39:10.010 INFO:teuthology.orchestra.run.smithi184.stdout:(30/159): rbd-nbd-19.0.0-3830.g3b748ea0.el9.x86 228 kB/s | 171 kB 00:00 2024-05-21T21:39:10.034 INFO:teuthology.orchestra.run.smithi151.stdout:(41/159): libquadmath-11.4.1-3.el9.x86_64.rpm 1.4 MB/s | 195 kB 00:00 2024-05-21T21:39:10.093 INFO:teuthology.orchestra.run.smithi151.stdout:(42/159): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.0 MB/s | 253 kB 00:00 2024-05-21T21:39:10.151 INFO:teuthology.orchestra.run.smithi151.stdout:(43/159): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-05-21T21:39:10.201 INFO:teuthology.orchestra.run.smithi151.stdout:(44/159): python3-cryptography-36.0.1-4.el9.x86 7.4 MB/s | 1.2 MB 00:00 2024-05-21T21:39:10.235 INFO:teuthology.orchestra.run.smithi151.stdout:(45/159): python3-pycparser-2.20-6.el9.noarch.r 1.6 MB/s | 135 kB 00:00 2024-05-21T21:39:10.260 INFO:teuthology.orchestra.run.smithi151.stdout:(46/159): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-05-21T21:39:10.294 INFO:teuthology.orchestra.run.smithi151.stdout:(47/159): python3-urllib3-1.26.5-5.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-05-21T21:39:10.336 INFO:teuthology.orchestra.run.smithi151.stdout:(48/159): automake-1.16.2-8.el9.noarch.rpm 9.1 MB/s | 689 kB 00:00 2024-05-21T21:39:10.395 INFO:teuthology.orchestra.run.smithi151.stdout:(49/159): bison-3.7.4-5.el9.x86_64.rpm 9.3 MB/s | 943 kB 00:00 2024-05-21T21:39:10.428 INFO:teuthology.orchestra.run.smithi151.stdout:(50/159): boost-program-options-1.75.0-8.el9.x8 1.1 MB/s | 107 kB 00:00 2024-05-21T21:39:10.454 INFO:teuthology.orchestra.run.smithi151.stdout:(51/159): elfutils-libelf-devel-0.191-4.el9.x86 375 kB/s | 22 kB 00:00 2024-05-21T21:39:10.487 INFO:teuthology.orchestra.run.smithi151.stdout:(52/159): flex-2.6.4-9.el9.x86_64.rpm 5.3 MB/s | 313 kB 00:00 2024-05-21T21:39:10.512 INFO:teuthology.orchestra.run.smithi151.stdout:(53/159): flexiblas-3.0.4-8.el9.x86_64.rpm 571 kB/s | 33 kB 00:00 2024-05-21T21:39:10.554 INFO:teuthology.orchestra.run.smithi151.stdout:(54/159): flexiblas-openblas-openmp-3.0.4-8.el9 454 kB/s | 18 kB 00:00 2024-05-21T21:39:10.597 INFO:teuthology.orchestra.run.smithi151.stdout:(55/159): libaio-devel-0.3.111-13.el9.x86_64.rp 269 kB/s | 11 kB 00:00 2024-05-21T21:39:10.698 INFO:teuthology.orchestra.run.smithi151.stdout:(56/159): flexiblas-netlib-3.0.4-8.el9.x86_64.r 14 MB/s | 3.0 MB 00:00 2024-05-21T21:39:10.731 INFO:teuthology.orchestra.run.smithi151.stdout:(57/159): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.2 MB/s | 160 kB 00:00 2024-05-21T21:39:10.748 INFO:teuthology.orchestra.run.smithi151.stdout:(58/159): librabbitmq-0.11.0-7.el9.x86_64.rpm 917 kB/s | 45 kB 00:00 2024-05-21T21:39:10.798 INFO:teuthology.orchestra.run.smithi151.stdout:(59/159): librdkafka-1.6.1-102.el9.x86_64.rpm 9.8 MB/s | 662 kB 00:00 2024-05-21T21:39:10.839 INFO:teuthology.orchestra.run.smithi151.stdout:(60/159): libtool-2.4.6-45.el9.x86_64.rpm 6.2 MB/s | 582 kB 00:00 2024-05-21T21:39:10.857 INFO:teuthology.orchestra.run.smithi151.stdout:(61/159): libuuid-devel-2.37.4-18.el9.x86_64.rp 443 kB/s | 25 kB 00:00 2024-05-21T21:39:10.882 INFO:teuthology.orchestra.run.smithi151.stdout:(62/159): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-05-21T21:39:10.916 INFO:teuthology.orchestra.run.smithi151.stdout:(63/159): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.9 MB/s | 292 kB 00:00 2024-05-21T21:39:10.949 INFO:teuthology.orchestra.run.smithi151.stdout:(64/159): lua-5.4.4-4.el9.x86_64.rpm 2.8 MB/s | 188 kB 00:00 2024-05-21T21:39:10.976 INFO:teuthology.orchestra.run.smithi151.stdout:(65/159): openblas-0.3.21-2.el9.x86_64.rpm 595 kB/s | 35 kB 00:00 2024-05-21T21:39:11.142 INFO:teuthology.orchestra.run.smithi151.stdout:(66/159): openblas-openmp-0.3.21-2.el9.x86_64.r 25 MB/s | 4.7 MB 00:00 2024-05-21T21:39:11.242 INFO:teuthology.orchestra.run.smithi151.stdout:(67/159): openssl-devel-3.2.1-1.el9.x86_64.rpm 17 MB/s | 4.4 MB 00:00 2024-05-21T21:39:11.317 INFO:teuthology.orchestra.run.smithi151.stdout:(68/159): postgresql-13.14-1.el9.x86_64.rpm 9.1 MB/s | 1.6 MB 00:00 2024-05-21T21:39:11.359 INFO:teuthology.orchestra.run.smithi151.stdout:(69/159): postgresql-contrib-13.14-1.el9.x86_64 7.7 MB/s | 911 kB 00:00 2024-05-21T21:39:11.384 INFO:teuthology.orchestra.run.smithi151.stdout:(70/159): postgresql-private-libs-13.14-1.el9.x 2.0 MB/s | 133 kB 00:00 2024-05-21T21:39:11.473 INFO:teuthology.orchestra.run.smithi164.stdout:(8/159): ceph-osd-19.0.0-3830.g3b748ea0.el9.x86 11 MB/s | 17 MB 00:01 2024-05-21T21:39:11.476 INFO:teuthology.orchestra.run.smithi151.stdout:(71/159): protobuf-3.14.0-13.el9.x86_64.rpm 12 MB/s | 1.0 MB 00:00 2024-05-21T21:39:11.523 INFO:teuthology.orchestra.run.smithi164.stdout:(9/159): ceph-selinux-19.0.0-3830.g3b748ea0.el9 504 kB/s | 25 kB 00:00 2024-05-21T21:39:11.527 INFO:teuthology.orchestra.run.smithi151.stdout:(72/159): python3-attrs-20.3.0-7.el9.noarch.rpm 1.7 MB/s | 87 kB 00:00 2024-05-21T21:39:11.552 INFO:teuthology.orchestra.run.smithi184.stdout:(31/159): ceph-mgr-dashboard-19.0.0-3830.g3b748 2.2 MB/s | 3.5 MB 00:01 2024-05-21T21:39:11.660 INFO:teuthology.orchestra.run.smithi151.stdout:(73/159): postgresql-server-13.14-1.el9.x86_64. 19 MB/s | 5.8 MB 00:00 2024-05-21T21:39:11.710 INFO:teuthology.orchestra.run.smithi151.stdout:(74/159): python3-colorama-0.4.6-3.el9.noarch.r 1.2 MB/s | 61 kB 00:00 2024-05-21T21:39:11.744 INFO:teuthology.orchestra.run.smithi184.stdout:(32/159): ceph-mgr-modules-core-19.0.0-3830.g3b 1.4 MB/s | 274 kB 00:00 2024-05-21T21:39:11.769 INFO:teuthology.orchestra.run.smithi151.stdout:(75/159): python3-devel-3.9.18-3.el9.x86_64.rpm 4.2 MB/s | 244 kB 00:00 2024-05-21T21:39:11.828 INFO:teuthology.orchestra.run.smithi151.stdout:(76/159): python3-jinja2-2.11.3-6.el9.noarch.rp 4.2 MB/s | 248 kB 00:00 2024-05-21T21:39:11.836 INFO:teuthology.orchestra.run.smithi184.stdout:(33/159): ceph-mgr-rook-19.0.0-3830.g3b748ea0.e 539 kB/s | 49 kB 00:00 2024-05-21T21:39:11.865 INFO:teuthology.orchestra.run.smithi164.stdout:(10/159): ceph-mon-19.0.0-3830.g3b748ea0.el9.x8 1.8 MB/s | 4.7 MB 00:02 2024-05-21T21:39:11.878 INFO:teuthology.orchestra.run.smithi151.stdout:(77/159): python3-jsonpatch-1.21-16.el9.noarch. 523 kB/s | 26 kB 00:00 2024-05-21T21:39:11.920 INFO:teuthology.orchestra.run.smithi184.stdout:(34/159): ceph-prometheus-alerts-19.0.0-3830.g3 198 kB/s | 16 kB 00:00 2024-05-21T21:39:12.008 INFO:teuthology.orchestra.run.smithi151.stdout:(78/159): python3-babel-2.9.1-2.el9.noarch.rpm 12 MB/s | 6.0 MB 00:00 2024-05-21T21:39:12.037 INFO:teuthology.orchestra.run.smithi151.stdout:(79/159): python3-jsonpointer-2.0-4.el9.noarch. 121 kB/s | 19 kB 00:00 2024-05-21T21:39:12.049 INFO:teuthology.orchestra.run.smithi164.stdout:(11/159): cephfs-mirror-19.0.0-3830.g3b748ea0.e 1.2 MB/s | 225 kB 00:00 2024-05-21T21:39:12.071 INFO:teuthology.orchestra.run.smithi151.stdout:(80/159): python3-mako-1.1.4-6.el9.noarch.rpm 2.8 MB/s | 172 kB 00:00 2024-05-21T21:39:12.097 INFO:teuthology.orchestra.run.smithi151.stdout:(81/159): python3-markupsafe-1.1.1-12.el9.x86_6 592 kB/s | 35 kB 00:00 2024-05-21T21:39:12.112 INFO:teuthology.orchestra.run.smithi184.stdout:(35/159): ceph-volume-19.0.0-3830.g3b748ea0.el9 1.4 MB/s | 273 kB 00:00 2024-05-21T21:39:12.141 INFO:teuthology.orchestra.run.smithi164.stdout:(12/159): cephfs-shell-19.0.0-3830.g3b748ea0.el 295 kB/s | 27 kB 00:00 2024-05-21T21:39:12.155 INFO:teuthology.orchestra.run.smithi151.stdout:(82/159): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.5 MB/s | 250 kB 00:00 2024-05-21T21:39:12.213 INFO:teuthology.orchestra.run.smithi151.stdout:(83/159): python3-oauthlib-3.1.1-5.el9.noarch.r 3.8 MB/s | 222 kB 00:00 2024-05-21T21:39:12.241 INFO:teuthology.orchestra.run.smithi164.stdout:(13/159): libcephfs-devel-19.0.0-3830.g3b748ea0 318 kB/s | 32 kB 00:00 2024-05-21T21:39:12.264 INFO:teuthology.orchestra.run.smithi151.stdout:(84/159): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-05-21T21:39:12.372 INFO:teuthology.orchestra.run.smithi151.stdout:(85/159): python3-numpy-1.20.1-5.el9.x86_64.rpm 17 MB/s | 5.1 MB 00:00 2024-05-21T21:39:12.398 INFO:teuthology.orchestra.run.smithi151.stdout:(86/159): python3-prettytable-0.7.2-27.el9.noar 318 kB/s | 42 kB 00:00 2024-05-21T21:39:12.432 INFO:teuthology.orchestra.run.smithi151.stdout:(87/159): python3-protobuf-3.14.0-13.el9.noarch 4.6 MB/s | 269 kB 00:00 2024-05-21T21:39:12.465 INFO:teuthology.orchestra.run.smithi151.stdout:(88/159): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 159 kB 00:00 2024-05-21T21:39:12.499 INFO:teuthology.orchestra.run.smithi151.stdout:(89/159): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 279 kB 00:00 2024-05-21T21:39:12.512 INFO:teuthology.orchestra.run.smithi184.stdout:(36/159): cephadm-19.0.0-3830.g3b748ea0.el9.noa 1.9 MB/s | 764 kB 00:00 2024-05-21T21:39:12.516 INFO:teuthology.orchestra.run.smithi151.stdout:(90/159): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-05-21T21:39:12.541 INFO:teuthology.orchestra.run.smithi151.stdout:(91/159): python3-requests-oauthlib-1.3.0-12.el 1.3 MB/s | 54 kB 00:00 2024-05-21T21:39:12.583 INFO:teuthology.orchestra.run.smithi151.stdout:(92/159): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-05-21T21:39:12.625 INFO:teuthology.orchestra.run.smithi151.stdout:(93/159): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.1 MB/s | 44 kB 00:00 2024-05-21T21:39:12.629 INFO:teuthology.orchestra.run.smithi184.stdout:(37/159): cephfs-top-19.0.0-3830.g3b748ea0.el9. 156 kB/s | 18 kB 00:00 2024-05-21T21:39:12.658 INFO:teuthology.orchestra.run.smithi164.stdout:(14/159): libcephfs2-19.0.0-3830.g3b748ea0.el9. 1.7 MB/s | 746 kB 00:00 2024-05-21T21:39:12.683 INFO:teuthology.orchestra.run.smithi151.stdout:(94/159): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-05-21T21:39:12.725 INFO:teuthology.orchestra.run.smithi151.stdout:(95/159): qatlib-service-24.02.0-1.el9.x86_64.r 886 kB/s | 36 kB 00:00 2024-05-21T21:39:12.766 INFO:teuthology.orchestra.run.smithi164.stdout:(15/159): libcephsqlite-19.0.0-3830.g3b748ea0.e 1.5 MB/s | 163 kB 00:00 2024-05-21T21:39:12.767 INFO:teuthology.orchestra.run.smithi151.stdout:(96/159): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-05-21T21:39:12.787 INFO:teuthology.orchestra.run.smithi184.stdout:(38/159): fuse-2.9.9-15.el9.x86_64.rpm 508 kB/s | 80 kB 00:00 2024-05-21T21:39:12.826 INFO:teuthology.orchestra.run.smithi151.stdout:(97/159): socat-1.7.4.1-5.el9.x86_64.rpm 5.2 MB/s | 305 kB 00:00 2024-05-21T21:39:12.868 INFO:teuthology.orchestra.run.smithi151.stdout:(98/159): uuid-1.6.2-55.el9.x86_64.rpm 1.4 MB/s | 58 kB 00:00 2024-05-21T21:39:12.875 INFO:teuthology.orchestra.run.smithi164.stdout:(16/159): librados-devel-19.0.0-3830.g3b748ea0. 1.1 MB/s | 127 kB 00:00 2024-05-21T21:39:12.918 INFO:teuthology.orchestra.run.smithi151.stdout:(99/159): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.9 MB/s | 94 kB 00:00 2024-05-21T21:39:12.963 INFO:teuthology.orchestra.run.smithi184.stdout:(39/159): libgfortran-11.4.1-3.el9.x86_64.rpm 4.5 MB/s | 803 kB 00:00 2024-05-21T21:39:12.985 INFO:teuthology.orchestra.run.smithi151.stdout:(100/159): xmlstarlet-1.6.1-20.el9.x86_64.rpm 968 kB/s | 64 kB 00:00 2024-05-21T21:39:13.027 INFO:teuthology.orchestra.run.smithi151.stdout:(101/159): zlib-devel-1.2.11-41.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-05-21T21:39:13.047 INFO:teuthology.orchestra.run.smithi184.stdout:(40/159): ceph-mgr-diskprediction-local-19.0.0- 2.4 MB/s | 7.4 MB 00:03 2024-05-21T21:39:13.072 INFO:teuthology.orchestra.run.smithi184.stdout:(41/159): libquadmath-11.4.1-3.el9.x86_64.rpm 1.8 MB/s | 195 kB 00:00 2024-05-21T21:39:13.118 INFO:teuthology.orchestra.run.smithi164.stdout:(17/159): libradosstriper1-19.0.0-3830.g3b748ea 2.0 MB/s | 501 kB 00:00 2024-05-21T21:39:13.189 INFO:teuthology.orchestra.run.smithi184.stdout:(42/159): python3-cryptography-36.0.1-4.el9.x86 11 MB/s | 1.2 MB 00:00 2024-05-21T21:39:13.247 INFO:teuthology.orchestra.run.smithi184.stdout:(43/159): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-05-21T21:39:13.252 INFO:teuthology.orchestra.run.smithi151.stdout:(102/159): python3-scipy-1.6.2-8.el9.x86_64.rpm 22 MB/s | 16 MB 00:00 2024-05-21T21:39:13.278 INFO:teuthology.orchestra.run.smithi151.stdout:(103/159): lua-devel-5.4.4-4.el9.x86_64.rpm 89 kB/s | 22 kB 00:00 2024-05-21T21:39:13.306 INFO:teuthology.orchestra.run.smithi184.stdout:(44/159): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-05-21T21:39:13.331 INFO:teuthology.orchestra.run.smithi184.stdout:(45/159): python3-cffi-1.14.5-5.el9.x86_64.rpm 891 kB/s | 253 kB 00:00 2024-05-21T21:39:13.365 INFO:teuthology.orchestra.run.smithi184.stdout:(46/159): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-05-21T21:39:13.390 INFO:teuthology.orchestra.run.smithi184.stdout:(47/159): python3-urllib3-1.26.5-5.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-05-21T21:39:13.426 INFO:teuthology.orchestra.run.smithi164.stdout:(18/159): ceph-radosgw-19.0.0-3830.g3b748ea0.el 3.1 MB/s | 11 MB 00:03 2024-05-21T21:39:13.509 INFO:teuthology.orchestra.run.smithi164.stdout:(19/159): python3-ceph-argparse-19.0.0-3830.g3b 544 kB/s | 45 kB 00:00 2024-05-21T21:39:13.578 INFO:teuthology.orchestra.run.smithi151.stdout:(104/159): protobuf-compiler-3.14.0-13.el9.x86_ 2.6 MB/s | 863 kB 00:00 2024-05-21T21:39:13.610 INFO:teuthology.orchestra.run.smithi164.stdout:(20/159): python3-ceph-common-19.0.0-3830.g3b74 1.3 MB/s | 136 kB 00:00 2024-05-21T21:39:13.710 INFO:teuthology.orchestra.run.smithi164.stdout:(21/159): python3-cephfs-19.0.0-3830.g3b748ea0. 1.6 MB/s | 161 kB 00:00 2024-05-21T21:39:13.877 INFO:teuthology.orchestra.run.smithi164.stdout:(22/159): python3-rados-19.0.0-3830.g3b748ea0.e 1.9 MB/s | 320 kB 00:00 2024-05-21T21:39:14.044 INFO:teuthology.orchestra.run.smithi164.stdout:(23/159): python3-rbd-19.0.0-3830.g3b748ea0.el9 1.8 MB/s | 300 kB 00:00 2024-05-21T21:39:14.107 INFO:teuthology.orchestra.run.smithi184.stdout:(48/159): automake-1.16.2-8.el9.noarch.rpm 929 kB/s | 689 kB 00:00 2024-05-21T21:39:14.141 INFO:teuthology.orchestra.run.smithi184.stdout:(49/159): bison-3.7.4-5.el9.x86_64.rpm 1.2 MB/s | 943 kB 00:00 2024-05-21T21:39:14.144 INFO:teuthology.orchestra.run.smithi164.stdout:(24/159): python3-rgw-19.0.0-3830.g3b748ea0.el9 993 kB/s | 99 kB 00:00 2024-05-21T21:39:14.208 INFO:teuthology.orchestra.run.smithi184.stdout:(50/159): boost-program-options-1.75.0-8.el9.x8 1.0 MB/s | 107 kB 00:00 2024-05-21T21:39:14.241 INFO:teuthology.orchestra.run.smithi184.stdout:(51/159): elfutils-libelf-devel-0.191-4.el9.x86 216 kB/s | 22 kB 00:00 2024-05-21T21:39:14.261 INFO:teuthology.orchestra.run.smithi164.stdout:(25/159): rbd-fuse-19.0.0-3830.g3b748ea0.el9.x8 731 kB/s | 85 kB 00:00 2024-05-21T21:39:14.317 INFO:teuthology.orchestra.run.smithi184.stdout:(52/159): flex-2.6.4-9.el9.x86_64.rpm 2.8 MB/s | 313 kB 00:00 2024-05-21T21:39:14.342 INFO:teuthology.orchestra.run.smithi184.stdout:(53/159): flexiblas-3.0.4-8.el9.x86_64.rpm 326 kB/s | 33 kB 00:00 2024-05-21T21:39:14.459 INFO:teuthology.orchestra.run.smithi184.stdout:(54/159): flexiblas-openblas-openmp-3.0.4-8.el9 156 kB/s | 18 kB 00:00 2024-05-21T21:39:14.478 INFO:teuthology.orchestra.run.smithi151.stdout:(105/159): gperftools-libs-2.9.1-2.el9.x86_64.r 343 kB/s | 309 kB 00:00 2024-05-21T21:39:14.512 INFO:teuthology.orchestra.run.smithi151.stdout:(106/159): abseil-cpp-20211102.0-3.el9.x86_64.r 445 kB/s | 548 kB 00:01 2024-05-21T21:39:14.576 INFO:teuthology.orchestra.run.smithi184.stdout:(55/159): libaio-devel-0.3.111-13.el9.x86_64.rp 95 kB/s | 11 kB 00:00 2024-05-21T21:39:14.587 INFO:teuthology.orchestra.run.smithi151.stdout:(107/159): grpc-data-1.46.7-10.el9.noarch.rpm 180 kB/s | 19 kB 00:00 2024-05-21T21:39:14.685 INFO:teuthology.orchestra.run.smithi184.stdout:(56/159): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.4 MB/s | 160 kB 00:00 2024-05-21T21:39:14.712 INFO:teuthology.orchestra.run.smithi151.stdout:(108/159): libarrow-doc-9.0.0-11.el9.noarch.rpm 209 kB/s | 26 kB 00:00 2024-05-21T21:39:14.793 INFO:teuthology.orchestra.run.smithi184.stdout:(57/159): librabbitmq-0.11.0-7.el9.x86_64.rpm 418 kB/s | 45 kB 00:00 2024-05-21T21:39:14.844 INFO:teuthology.orchestra.run.smithi184.stdout:(58/159): flexiblas-netlib-3.0.4-8.el9.x86_64.r 5.7 MB/s | 3.0 MB 00:00 2024-05-21T21:39:14.911 INFO:teuthology.orchestra.run.smithi184.stdout:(59/159): librdkafka-1.6.1-102.el9.x86_64.rpm 5.5 MB/s | 662 kB 00:00 2024-05-21T21:39:14.921 INFO:teuthology.orchestra.run.smithi151.stdout:(109/159): liboath-2.6.7-2.el9.x86_64.rpm 234 kB/s | 49 kB 00:00 2024-05-21T21:39:14.953 INFO:teuthology.orchestra.run.smithi184.stdout:(60/159): libtool-2.4.6-45.el9.x86_64.rpm 5.2 MB/s | 582 kB 00:00 2024-05-21T21:39:15.011 INFO:teuthology.orchestra.run.smithi184.stdout:(61/159): libuuid-devel-2.37.4-18.el9.x86_64.rp 254 kB/s | 25 kB 00:00 2024-05-21T21:39:15.053 INFO:teuthology.orchestra.run.smithi184.stdout:(62/159): libzstd-devel-1.5.1-2.el9.x86_64.rpm 473 kB/s | 47 kB 00:00 2024-05-21T21:39:15.105 INFO:teuthology.orchestra.run.smithi151.stdout:(110/159): libunwind-1.6.2-1.el9.x86_64.rpm 369 kB/s | 67 kB 00:00 2024-05-21T21:39:15.262 INFO:teuthology.orchestra.run.smithi184.stdout:(63/159): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.1 MB/s | 292 kB 00:00 2024-05-21T21:39:15.288 INFO:teuthology.orchestra.run.smithi184.stdout:(64/159): lua-5.4.4-4.el9.x86_64.rpm 806 kB/s | 188 kB 00:00 2024-05-21T21:39:15.363 INFO:teuthology.orchestra.run.smithi184.stdout:(65/159): openblas-0.3.21-2.el9.x86_64.rpm 351 kB/s | 35 kB 00:00 2024-05-21T21:39:15.372 INFO:teuthology.orchestra.run.smithi151.stdout:(111/159): luarocks-3.9.2-5.el9.noarch.rpm 570 kB/s | 151 kB 00:00 2024-05-21T21:39:15.420 INFO:teuthology.orchestra.run.smithi164.stdout:(26/159): ceph-test-19.0.0-3830.g3b748ea0.el9.x 13 MB/s | 49 MB 00:03 2024-05-21T21:39:15.487 INFO:teuthology.orchestra.run.smithi164.stdout:(27/159): rbd-mirror-19.0.0-3830.g3b748ea0.el9. 2.5 MB/s | 3.1 MB 00:01 2024-05-21T21:39:15.512 INFO:teuthology.orchestra.run.smithi164.stdout:(28/159): rbd-nbd-19.0.0-3830.g3b748ea0.el9.x86 1.8 MB/s | 171 kB 00:00 2024-05-21T21:39:15.579 INFO:teuthology.orchestra.run.smithi164.stdout:(29/159): librgw2-19.0.0-3830.g3b748ea0.el9.x86 2.2 MB/s | 5.4 MB 00:02 2024-05-21T21:39:15.604 INFO:teuthology.orchestra.run.smithi164.stdout:(30/159): ceph-grafana-dashboards-19.0.0-3830.g 292 kB/s | 34 kB 00:00 2024-05-21T21:39:15.629 INFO:teuthology.orchestra.run.smithi164.stdout:(31/159): ceph-mgr-cephadm-19.0.0-3830.g3b748ea 1.2 MB/s | 143 kB 00:00 2024-05-21T21:39:15.688 INFO:teuthology.orchestra.run.smithi164.stdout:(32/159): ceph-mgr-modules-core-19.0.0-3830.g3b 4.6 MB/s | 274 kB 00:00 2024-05-21T21:39:15.738 INFO:teuthology.orchestra.run.smithi164.stdout:(33/159): ceph-mgr-rook-19.0.0-3830.g3b748ea0.e 986 kB/s | 49 kB 00:00 2024-05-21T21:39:15.788 INFO:teuthology.orchestra.run.smithi164.stdout:(34/159): ceph-prometheus-alerts-19.0.0-3830.g3 331 kB/s | 16 kB 00:00 2024-05-21T21:39:15.847 INFO:teuthology.orchestra.run.smithi164.stdout:(35/159): ceph-volume-19.0.0-3830.g3b748ea0.el9 4.6 MB/s | 273 kB 00:00 2024-05-21T21:39:15.931 INFO:teuthology.orchestra.run.smithi164.stdout:(36/159): cephadm-19.0.0-3830.g3b748ea0.el9.noa 9.0 MB/s | 764 kB 00:00 2024-05-21T21:39:15.981 INFO:teuthology.orchestra.run.smithi164.stdout:(37/159): cephfs-top-19.0.0-3830.g3b748ea0.el9. 365 kB/s | 18 kB 00:00 2024-05-21T21:39:16.088 INFO:teuthology.orchestra.run.smithi184.stdout:(66/159): ceph-test-19.0.0-3830.g3b748ea0.el9.x 4.6 MB/s | 49 MB 00:10 2024-05-21T21:39:16.123 INFO:teuthology.orchestra.run.smithi164.stdout:(38/159): fuse-2.9.9-15.el9.x86_64.rpm 567 kB/s | 80 kB 00:00 2024-05-21T21:39:16.155 INFO:teuthology.orchestra.run.smithi151.stdout:(112/159): ceph-test-19.0.0-3830.g3b748ea0.el9. 3.4 MB/s | 49 MB 00:14 2024-05-21T21:39:16.306 INFO:teuthology.orchestra.run.smithi164.stdout:(39/159): libgfortran-11.4.1-3.el9.x86_64.rpm 4.3 MB/s | 803 kB 00:00 2024-05-21T21:39:16.365 INFO:teuthology.orchestra.run.smithi164.stdout:(40/159): libquadmath-11.4.1-3.el9.x86_64.rpm 3.3 MB/s | 195 kB 00:00 2024-05-21T21:39:16.457 INFO:teuthology.orchestra.run.smithi164.stdout:(41/159): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.7 MB/s | 253 kB 00:00 2024-05-21T21:39:16.507 INFO:teuthology.orchestra.run.smithi164.stdout:(42/159): ceph-mgr-dashboard-19.0.0-3830.g3b748 3.8 MB/s | 3.5 MB 00:00 2024-05-21T21:39:16.733 INFO:teuthology.orchestra.run.smithi164.stdout:(43/159): python3-cryptography-36.0.1-4.el9.x86 4.5 MB/s | 1.2 MB 00:00 2024-05-21T21:39:16.758 INFO:teuthology.orchestra.run.smithi164.stdout:(44/159): python3-ply-3.11-14.el9.noarch.rpm 426 kB/s | 106 kB 00:00 2024-05-21T21:39:16.783 INFO:teuthology.orchestra.run.smithi164.stdout:(45/159): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-05-21T21:39:16.814 INFO:teuthology.orchestra.run.smithi151.stdout:(113/159): parquet-libs-9.0.0-11.el9.x86_64.rpm 582 kB/s | 839 kB 00:01 2024-05-21T21:39:16.884 INFO:teuthology.orchestra.run.smithi164.stdout:(46/159): ceph-mgr-diskprediction-local-19.0.0- 5.8 MB/s | 7.4 MB 00:01 2024-05-21T21:39:16.923 INFO:teuthology.orchestra.run.smithi151.stdout:(114/159): python3-bcrypt-3.2.2-1.el9.x86_64.rp 404 kB/s | 43 kB 00:00 2024-05-21T21:39:16.934 INFO:teuthology.orchestra.run.smithi164.stdout:(47/159): python3-requests-2.25.1-8.el9.noarch. 710 kB/s | 125 kB 00:00 2024-05-21T21:39:16.955 INFO:teuthology.orchestra.run.smithi184.stdout:(67/159): postgresql-13.14-1.el9.x86_64.rpm 1.8 MB/s | 1.6 MB 00:00 2024-05-21T21:39:16.959 INFO:teuthology.orchestra.run.smithi164.stdout:(48/159): python3-urllib3-1.26.5-5.el9.noarch.r 1.2 MB/s | 215 kB 00:00 2024-05-21T21:39:17.059 INFO:teuthology.orchestra.run.smithi151.stdout:(115/159): python3-cachetools-4.2.4-1.el9.noarc 237 kB/s | 32 kB 00:00 2024-05-21T21:39:17.071 INFO:teuthology.orchestra.run.smithi184.stdout:(68/159): postgresql-contrib-13.14-1.el9.x86_64 7.6 MB/s | 911 kB 00:00 2024-05-21T21:39:17.093 INFO:teuthology.orchestra.run.smithi164.stdout:(49/159): boost-program-options-1.75.0-8.el9.x8 802 kB/s | 107 kB 00:00 2024-05-21T21:39:17.135 INFO:teuthology.orchestra.run.smithi164.stdout:(50/159): elfutils-libelf-devel-0.191-4.el9.x86 523 kB/s | 22 kB 00:00 2024-05-21T21:39:17.160 INFO:teuthology.orchestra.run.smithi164.stdout:(51/159): automake-1.16.2-8.el9.noarch.rpm 2.4 MB/s | 689 kB 00:00 2024-05-21T21:39:17.165 INFO:teuthology.orchestra.run.smithi151.stdout:(116/159): python3-certifi-2021.10.8-2.el9.noar 143 kB/s | 15 kB 00:00 2024-05-21T21:39:17.188 INFO:teuthology.orchestra.run.smithi184.stdout:(69/159): postgresql-private-libs-13.14-1.el9.x 1.1 MB/s | 133 kB 00:00 2024-05-21T21:39:17.202 INFO:teuthology.orchestra.run.smithi164.stdout:(52/159): bison-3.7.4-5.el9.x86_64.rpm 3.4 MB/s | 943 kB 00:00 2024-05-21T21:39:17.227 INFO:teuthology.orchestra.run.smithi164.stdout:(53/159): flexiblas-3.0.4-8.el9.x86_64.rpm 489 kB/s | 33 kB 00:00 2024-05-21T21:39:17.253 INFO:teuthology.orchestra.run.smithi164.stdout:(54/159): flex-2.6.4-9.el9.x86_64.rpm 2.6 MB/s | 313 kB 00:00 2024-05-21T21:39:17.278 INFO:teuthology.orchestra.run.smithi164.stdout:(55/159): flexiblas-openblas-openmp-3.0.4-8.el9 363 kB/s | 18 kB 00:00 2024-05-21T21:39:17.303 INFO:teuthology.orchestra.run.smithi164.stdout:(56/159): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-05-21T21:39:17.329 INFO:teuthology.orchestra.run.smithi164.stdout:(57/159): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-05-21T21:39:17.354 INFO:teuthology.orchestra.run.smithi164.stdout:(58/159): librabbitmq-0.11.0-7.el9.x86_64.rpm 904 kB/s | 45 kB 00:00 2024-05-21T21:39:17.387 INFO:teuthology.orchestra.run.smithi164.stdout:(59/159): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-05-21T21:39:17.423 INFO:teuthology.orchestra.run.smithi151.stdout:(117/159): python3-asyncssh-2.13.2-5.el9.noarch 433 kB/s | 548 kB 00:01 2024-05-21T21:39:17.438 INFO:teuthology.orchestra.run.smithi164.stdout:(60/159): flexiblas-netlib-3.0.4-8.el9.x86_64.r 13 MB/s | 3.0 MB 00:00 2024-05-21T21:39:17.449 INFO:teuthology.orchestra.run.smithi151.stdout:(118/159): python3-cheroot-8.6.0-4.el9.noarch.r 607 kB/s | 172 kB 00:00 2024-05-21T21:39:17.463 INFO:teuthology.orchestra.run.smithi164.stdout:(61/159): libuuid-devel-2.37.4-18.el9.x86_64.rp 338 kB/s | 25 kB 00:00 2024-05-21T21:39:17.488 INFO:teuthology.orchestra.run.smithi164.stdout:(62/159): libzstd-devel-1.5.1-2.el9.x86_64.rpm 946 kB/s | 47 kB 00:00 2024-05-21T21:39:17.513 INFO:teuthology.orchestra.run.smithi164.stdout:(63/159): libtool-2.4.6-45.el9.x86_64.rpm 3.6 MB/s | 582 kB 00:00 2024-05-21T21:39:17.539 INFO:teuthology.orchestra.run.smithi164.stdout:(64/159): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.8 MB/s | 292 kB 00:00 2024-05-21T21:39:17.564 INFO:teuthology.orchestra.run.smithi164.stdout:(65/159): openblas-0.3.21-2.el9.x86_64.rpm 700 kB/s | 35 kB 00:00 2024-05-21T21:39:17.589 INFO:teuthology.orchestra.run.smithi164.stdout:(66/159): lua-5.4.4-4.el9.x86_64.rpm 1.8 MB/s | 188 kB 00:00 2024-05-21T21:39:17.681 INFO:teuthology.orchestra.run.smithi164.stdout:(67/159): postgresql-13.14-1.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-05-21T21:39:17.748 INFO:teuthology.orchestra.run.smithi164.stdout:(68/159): openblas-openmp-0.3.21-2.el9.x86_64.r 23 MB/s | 4.7 MB 00:00 2024-05-21T21:39:17.782 INFO:teuthology.orchestra.run.smithi164.stdout:(69/159): postgresql-contrib-13.14-1.el9.x86_64 8.9 MB/s | 911 kB 00:00 2024-05-21T21:39:17.807 INFO:teuthology.orchestra.run.smithi164.stdout:(70/159): postgresql-private-libs-13.14-1.el9.x 2.2 MB/s | 133 kB 00:00 2024-05-21T21:39:17.882 INFO:teuthology.orchestra.run.smithi164.stdout:(71/159): openssl-devel-3.2.1-1.el9.x86_64.rpm 14 MB/s | 4.4 MB 00:00 2024-05-21T21:39:17.905 INFO:teuthology.orchestra.run.smithi184.stdout:(70/159): openssl-devel-3.2.1-1.el9.x86_64.rpm 1.7 MB/s | 4.4 MB 00:02 2024-05-21T21:39:17.907 INFO:teuthology.orchestra.run.smithi151.stdout:(119/159): python3-cmd2-2.4.2-2.el9.noarch.rpm 662 kB/s | 303 kB 00:00 2024-05-21T21:39:17.924 INFO:teuthology.orchestra.run.smithi164.stdout:(72/159): protobuf-3.14.0-13.el9.x86_64.rpm 8.6 MB/s | 1.0 MB 00:00 2024-05-21T21:39:17.949 INFO:teuthology.orchestra.run.smithi164.stdout:(73/159): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-05-21T21:39:17.966 INFO:teuthology.orchestra.run.smithi151.stdout:(120/159): python3-cherrypy-18.6.1-2.el9.noarch 662 kB/s | 358 kB 00:00 2024-05-21T21:39:18.042 INFO:teuthology.orchestra.run.smithi164.stdout:(74/159): postgresql-server-13.14-1.el9.x86_64. 22 MB/s | 5.8 MB 00:00 2024-05-21T21:39:18.058 INFO:teuthology.orchestra.run.smithi164.stdout:(75/159): python3-colorama-0.4.6-3.el9.noarch.r 565 kB/s | 61 kB 00:00 2024-05-21T21:39:18.084 INFO:teuthology.orchestra.run.smithi164.stdout:(76/159): python3-devel-3.9.18-3.el9.x86_64.rpm 5.7 MB/s | 244 kB 00:00 2024-05-21T21:39:18.109 INFO:teuthology.orchestra.run.smithi164.stdout:(77/159): python3-jinja2-2.11.3-6.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-05-21T21:39:18.134 INFO:teuthology.orchestra.run.smithi164.stdout:(78/159): python3-jsonpatch-1.21-16.el9.noarch. 511 kB/s | 26 kB 00:00 2024-05-21T21:39:18.160 INFO:teuthology.orchestra.run.smithi164.stdout:(79/159): python3-jsonpointer-2.0-4.el9.noarch. 381 kB/s | 19 kB 00:00 2024-05-21T21:39:18.185 INFO:teuthology.orchestra.run.smithi164.stdout:(80/159): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-05-21T21:39:18.210 INFO:teuthology.orchestra.run.smithi164.stdout:(81/159): python3-markupsafe-1.1.1-12.el9.x86_6 693 kB/s | 35 kB 00:00 2024-05-21T21:39:18.266 INFO:teuthology.orchestra.run.smithi151.stdout:(121/159): python3-google-auth-2.29.0-1.el9.noa 574 kB/s | 206 kB 00:00 2024-05-21T21:39:18.294 INFO:teuthology.orchestra.run.smithi164.stdout:(82/159): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 6.0 MB 00:00 2024-05-21T21:39:18.319 INFO:teuthology.orchestra.run.smithi164.stdout:(83/159): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.3 MB/s | 250 kB 00:00 2024-05-21T21:39:18.344 INFO:teuthology.orchestra.run.smithi164.stdout:(84/159): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-05-21T21:39:18.370 INFO:teuthology.orchestra.run.smithi164.stdout:(85/159): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-05-21T21:39:18.395 INFO:teuthology.orchestra.run.smithi164.stdout:(86/159): python3-prettytable-0.7.2-27.el9.noar 841 kB/s | 42 kB 00:00 2024-05-21T21:39:18.420 INFO:teuthology.orchestra.run.smithi164.stdout:(87/159): python3-protobuf-3.14.0-13.el9.noarch 5.3 MB/s | 269 kB 00:00 2024-05-21T21:39:18.446 INFO:teuthology.orchestra.run.smithi164.stdout:(88/159): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-05-21T21:39:18.471 INFO:teuthology.orchestra.run.smithi164.stdout:(89/159): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-05-21T21:39:18.496 INFO:teuthology.orchestra.run.smithi164.stdout:(90/159): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-05-21T21:39:18.521 INFO:teuthology.orchestra.run.smithi164.stdout:(91/159): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-05-21T21:39:18.533 INFO:teuthology.orchestra.run.smithi151.stdout:(122/159): python3-grpcio-tools-1.46.7-10.el9.x 543 kB/s | 144 kB 00:00 2024-05-21T21:39:18.597 INFO:teuthology.orchestra.run.smithi164.stdout:(92/159): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-05-21T21:39:18.622 INFO:teuthology.orchestra.run.smithi164.stdout:(93/159): python3-toml-0.10.2-6.el9.noarch.rpm 415 kB/s | 42 kB 00:00 2024-05-21T21:39:18.647 INFO:teuthology.orchestra.run.smithi164.stdout:(94/159): python3-wcwidth-0.2.5-8.el9.noarch.rp 882 kB/s | 44 kB 00:00 2024-05-21T21:39:18.658 INFO:teuthology.orchestra.run.smithi151.stdout:(123/159): python3-jaraco-8.2.1-3.el9.noarch.rp 86 kB/s | 11 kB 00:00 2024-05-21T21:39:18.672 INFO:teuthology.orchestra.run.smithi164.stdout:(95/159): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-05-21T21:39:18.698 INFO:teuthology.orchestra.run.smithi164.stdout:(96/159): qatlib-service-24.02.0-1.el9.x86_64.r 714 kB/s | 36 kB 00:00 2024-05-21T21:39:18.723 INFO:teuthology.orchestra.run.smithi164.stdout:(97/159): qatzip-libs-1.2.0-1.el9.x86_64.rpm 943 kB/s | 47 kB 00:00 2024-05-21T21:39:18.765 INFO:teuthology.orchestra.run.smithi164.stdout:(98/159): socat-1.7.4.1-5.el9.x86_64.rpm 4.5 MB/s | 305 kB 00:00 2024-05-21T21:39:18.767 INFO:teuthology.orchestra.run.smithi151.stdout:(124/159): python3-jaraco-classes-3.2.1-5.el9.n 164 kB/s | 18 kB 00:00 2024-05-21T21:39:18.790 INFO:teuthology.orchestra.run.smithi164.stdout:(99/159): uuid-1.6.2-55.el9.x86_64.rpm 867 kB/s | 58 kB 00:00 2024-05-21T21:39:18.816 INFO:teuthology.orchestra.run.smithi164.stdout:(100/159): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.8 MB/s | 94 kB 00:00 2024-05-21T21:39:18.841 INFO:teuthology.orchestra.run.smithi164.stdout:(101/159): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-05-21T21:39:18.866 INFO:teuthology.orchestra.run.smithi164.stdout:(102/159): zlib-devel-1.2.11-41.el9.x86_64.rpm 894 kB/s | 45 kB 00:00 2024-05-21T21:39:18.876 INFO:teuthology.orchestra.run.smithi151.stdout:(125/159): python3-jaraco-collections-3.0.0-8.e 215 kB/s | 23 kB 00:00 2024-05-21T21:39:18.950 INFO:teuthology.orchestra.run.smithi164.stdout:(103/159): lua-devel-5.4.4-4.el9.x86_64.rpm 206 kB/s | 22 kB 00:00 2024-05-21T21:39:19.125 INFO:teuthology.orchestra.run.smithi164.stdout:(104/159): python3-scipy-1.6.2-8.el9.x86_64.rpm 26 MB/s | 16 MB 00:00 2024-05-21T21:39:19.168 INFO:teuthology.orchestra.run.smithi151.stdout:(126/159): python3-jaraco-functools-3.5.0-2.el9 67 kB/s | 19 kB 00:00 2024-05-21T21:39:19.217 INFO:teuthology.orchestra.run.smithi164.stdout:(105/159): protobuf-compiler-3.14.0-13.el9.x86_ 2.4 MB/s | 863 kB 00:00 2024-05-21T21:39:19.277 INFO:teuthology.orchestra.run.smithi151.stdout:(127/159): python3-jaraco-text-3.2.0-6.el9.noar 183 kB/s | 20 kB 00:00 2024-05-21T21:39:19.405 INFO:teuthology.orchestra.run.smithi184.stdout:(71/159): protobuf-3.14.0-13.el9.x86_64.rpm 689 kB/s | 1.0 MB 00:01 2024-05-21T21:39:19.472 INFO:teuthology.orchestra.run.smithi184.stdout:(72/159): openblas-openmp-0.3.21-2.el9.x86_64.r 1.1 MB/s | 4.7 MB 00:04 2024-05-21T21:39:19.484 INFO:teuthology.orchestra.run.smithi164.stdout:(106/159): grpc-data-1.46.7-10.el9.noarch.rpm 81 kB/s | 19 kB 00:00 2024-05-21T21:39:19.597 INFO:teuthology.orchestra.run.smithi184.stdout:(73/159): python3-attrs-20.3.0-7.el9.noarch.rpm 456 kB/s | 87 kB 00:00 2024-05-21T21:39:19.601 INFO:teuthology.orchestra.run.smithi164.stdout:(107/159): abseil-cpp-20211102.0-3.el9.x86_64.r 842 kB/s | 548 kB 00:00 2024-05-21T21:39:19.626 INFO:teuthology.orchestra.run.smithi164.stdout:(108/159): gperftools-libs-2.9.1-2.el9.x86_64.r 651 kB/s | 309 kB 00:00 2024-05-21T21:39:19.668 INFO:teuthology.orchestra.run.smithi164.stdout:(109/159): libarrow-doc-9.0.0-11.el9.noarch.rpm 389 kB/s | 26 kB 00:00 2024-05-21T21:39:19.698 INFO:teuthology.orchestra.run.smithi184.stdout:(74/159): python3-colorama-0.4.6-3.el9.noarch.r 612 kB/s | 61 kB 00:00 2024-05-21T21:39:19.702 INFO:teuthology.orchestra.run.smithi164.stdout:(110/159): liboath-2.6.7-2.el9.x86_64.rpm 647 kB/s | 49 kB 00:00 2024-05-21T21:39:19.752 INFO:teuthology.orchestra.run.smithi164.stdout:(111/159): libunwind-1.6.2-1.el9.x86_64.rpm 807 kB/s | 67 kB 00:00 2024-05-21T21:39:19.794 INFO:teuthology.orchestra.run.smithi164.stdout:(112/159): luarocks-3.9.2-5.el9.noarch.rpm 1.6 MB/s | 151 kB 00:00 2024-05-21T21:39:19.903 INFO:teuthology.orchestra.run.smithi164.stdout:(113/159): parquet-libs-9.0.0-11.el9.x86_64.rpm 5.5 MB/s | 839 kB 00:00 2024-05-21T21:39:19.928 INFO:teuthology.orchestra.run.smithi164.stdout:(114/159): python3-asyncssh-2.13.2-5.el9.noarch 4.0 MB/s | 548 kB 00:00 2024-05-21T21:39:19.995 INFO:teuthology.orchestra.run.smithi164.stdout:(115/159): libarrow-9.0.0-11.el9.x86_64.rpm 8.6 MB/s | 4.4 MB 00:00 2024-05-21T21:39:20.020 INFO:teuthology.orchestra.run.smithi164.stdout:(116/159): python3-bcrypt-3.2.2-1.el9.x86_64.rp 370 kB/s | 43 kB 00:00 2024-05-21T21:39:20.040 INFO:teuthology.orchestra.run.smithi184.stdout:(75/159): python3-devel-3.9.18-3.el9.x86_64.rpm 714 kB/s | 244 kB 00:00 2024-05-21T21:39:20.046 INFO:teuthology.orchestra.run.smithi164.stdout:(117/159): python3-cachetools-4.2.4-1.el9.noarc 274 kB/s | 32 kB 00:00 2024-05-21T21:39:20.071 INFO:teuthology.orchestra.run.smithi164.stdout:(118/159): python3-certifi-2021.10.8-2.el9.noar 199 kB/s | 15 kB 00:00 2024-05-21T21:39:20.121 INFO:teuthology.orchestra.run.smithi164.stdout:(119/159): python3-cheroot-8.6.0-4.el9.noarch.r 1.7 MB/s | 172 kB 00:00 2024-05-21T21:39:20.146 INFO:teuthology.orchestra.run.smithi164.stdout:(120/159): python3-cherrypy-18.6.1-2.el9.noarch 3.5 MB/s | 358 kB 00:00 2024-05-21T21:39:20.172 INFO:teuthology.orchestra.run.smithi164.stdout:(121/159): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.9 MB/s | 303 kB 00:00 2024-05-21T21:39:20.197 INFO:teuthology.orchestra.run.smithi164.stdout:(122/159): python3-google-auth-2.29.0-1.el9.noa 2.7 MB/s | 206 kB 00:00 2024-05-21T21:39:20.247 INFO:teuthology.orchestra.run.smithi164.stdout:(123/159): python3-grpcio-tools-1.46.7-10.el9.x 1.9 MB/s | 144 kB 00:00 2024-05-21T21:39:20.281 INFO:teuthology.orchestra.run.smithi164.stdout:(124/159): python3-jaraco-8.2.1-3.el9.noarch.rp 128 kB/s | 11 kB 00:00 2024-05-21T21:39:20.306 INFO:teuthology.orchestra.run.smithi164.stdout:(125/159): python3-jaraco-classes-3.2.1-5.el9.n 302 kB/s | 18 kB 00:00 2024-05-21T21:39:20.356 INFO:teuthology.orchestra.run.smithi164.stdout:(126/159): python3-grpcio-1.46.7-10.el9.x86_64. 9.7 MB/s | 2.0 MB 00:00 2024-05-21T21:39:20.373 INFO:teuthology.orchestra.run.smithi164.stdout:(127/159): python3-jaraco-collections-3.0.0-8.e 251 kB/s | 23 kB 00:00 2024-05-21T21:39:20.390 INFO:teuthology.orchestra.run.smithi164.stdout:(128/159): python3-jaraco-functools-3.5.0-2.el9 232 kB/s | 19 kB 00:00 2024-05-21T21:39:20.390 INFO:teuthology.orchestra.run.smithi184.stdout:(76/159): python3-jinja2-2.11.3-6.el9.noarch.rp 709 kB/s | 248 kB 00:00 2024-05-21T21:39:20.424 INFO:teuthology.orchestra.run.smithi164.stdout:(129/159): python3-jaraco-text-3.2.0-6.el9.noar 294 kB/s | 20 kB 00:00 2024-05-21T21:39:20.460 INFO:teuthology.orchestra.run.smithi151.stdout:(128/159): libarrow-9.0.0-11.el9.x86_64.rpm 760 kB/s | 4.4 MB 00:05 2024-05-21T21:39:20.465 INFO:teuthology.orchestra.run.smithi164.stdout:(130/159): python3-logutils-0.3.5-21.el9.noarch 617 kB/s | 46 kB 00:00 2024-05-21T21:39:20.499 INFO:teuthology.orchestra.run.smithi184.stdout:(77/159): python3-jsonpatch-1.21-16.el9.noarch. 236 kB/s | 26 kB 00:00 2024-05-21T21:39:20.499 INFO:teuthology.orchestra.run.smithi164.stdout:(131/159): python3-kubernetes-26.1.0-2.el9.noar 9.4 MB/s | 1.0 MB 00:00 2024-05-21T21:39:20.516 INFO:teuthology.orchestra.run.smithi164.stdout:(132/159): python3-more-itertools-8.12.0-2.el9. 856 kB/s | 79 kB 00:00 2024-05-21T21:39:20.527 INFO:teuthology.orchestra.run.smithi151.stdout:(129/159): python3-grpcio-1.46.7-10.el9.x86_64. 815 kB/s | 2.0 MB 00:02 2024-05-21T21:39:20.558 INFO:teuthology.orchestra.run.smithi164.stdout:(133/159): python3-natsort-7.1.1-5.el9.noarch.r 627 kB/s | 58 kB 00:00 2024-05-21T21:39:20.569 INFO:teuthology.orchestra.run.smithi151.stdout:(130/159): python3-logutils-0.3.5-21.el9.noarch 429 kB/s | 46 kB 00:00 2024-05-21T21:39:20.600 INFO:teuthology.orchestra.run.smithi164.stdout:(134/159): python3-pecan-1.4.2-3.el9.noarch.rpm 2.6 MB/s | 272 kB 00:00 2024-05-21T21:39:20.607 INFO:teuthology.orchestra.run.smithi184.stdout:(78/159): python3-jsonpointer-2.0-4.el9.noarch. 177 kB/s | 19 kB 00:00 2024-05-21T21:39:20.625 INFO:teuthology.orchestra.run.smithi164.stdout:(135/159): python3-portend-3.1.0-2.el9.noarch.r 150 kB/s | 16 kB 00:00 2024-05-21T21:39:20.651 INFO:teuthology.orchestra.run.smithi164.stdout:(136/159): python3-pyOpenSSL-21.0.0-1.el9.noarc 971 kB/s | 90 kB 00:00 2024-05-21T21:39:20.669 INFO:teuthology.orchestra.run.smithi151.stdout:(131/159): python3-kubernetes-26.1.0-2.el9.noar 751 kB/s | 1.0 MB 00:01 2024-05-21T21:39:20.676 INFO:teuthology.orchestra.run.smithi164.stdout:(137/159): python3-pyperclip-1.8.2-2.el9.noarch 373 kB/s | 28 kB 00:00 2024-05-21T21:39:20.710 INFO:teuthology.orchestra.run.smithi164.stdout:(138/159): python3-repoze-lru-0.7-16.el9.noarch 367 kB/s | 31 kB 00:00 2024-05-21T21:39:20.728 INFO:teuthology.orchestra.run.smithi151.stdout:(132/159): python3-more-itertools-8.12.0-2.el9. 394 kB/s | 79 kB 00:00 2024-05-21T21:39:20.735 INFO:teuthology.orchestra.run.smithi164.stdout:(139/159): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2024-05-21T21:39:20.760 INFO:teuthology.orchestra.run.smithi164.stdout:(140/159): python3-rsa-4.9-2.el9.noarch.rpm 704 kB/s | 59 kB 00:00 2024-05-21T21:39:20.778 INFO:teuthology.orchestra.run.smithi151.stdout:(133/159): python3-natsort-7.1.1-5.el9.noarch.r 277 kB/s | 58 kB 00:00 2024-05-21T21:39:20.794 INFO:teuthology.orchestra.run.smithi164.stdout:(141/159): python3-tempora-5.0.0-2.el9.noarch.r 426 kB/s | 36 kB 00:00 2024-05-21T21:39:20.819 INFO:teuthology.orchestra.run.smithi164.stdout:(142/159): python3-typing-extensions-4.4.0-2.el 608 kB/s | 51 kB 00:00 2024-05-21T21:39:20.837 INFO:teuthology.orchestra.run.smithi151.stdout:(134/159): python3-portend-3.1.0-2.el9.noarch.r 152 kB/s | 16 kB 00:00 2024-05-21T21:39:20.853 INFO:teuthology.orchestra.run.smithi164.stdout:(143/159): python3-webob-1.8.7-6.el9.noarch.rpm 2.4 MB/s | 230 kB 00:00 2024-05-21T21:39:20.875 INFO:teuthology.orchestra.run.smithi184.stdout:(79/159): python3-mako-1.1.4-6.el9.noarch.rpm 645 kB/s | 172 kB 00:00 2024-05-21T21:39:20.903 INFO:teuthology.orchestra.run.smithi164.stdout:(144/159): python3-websocket-client-1.2.3-2.el9 825 kB/s | 90 kB 00:00 2024-05-21T21:39:20.928 INFO:teuthology.orchestra.run.smithi164.stdout:(145/159): python3-werkzeug-2.0.3-3.el9.1.noarc 3.8 MB/s | 427 kB 00:00 2024-05-21T21:39:20.945 INFO:teuthology.orchestra.run.smithi151.stdout:(135/159): python3-pyperclip-1.8.2-2.el9.noarch 262 kB/s | 28 kB 00:00 2024-05-21T21:39:20.950 INFO:teuthology.orchestra.run.smithi184.stdout:(80/159): postgresql-server-13.14-1.el9.x86_64. 1.5 MB/s | 5.8 MB 00:03 2024-05-21T21:39:20.953 INFO:teuthology.orchestra.run.smithi164.stdout:(146/159): python3-zc-lockfile-2.0-10.el9.noarc 199 kB/s | 20 kB 00:00 2024-05-21T21:39:20.973 INFO:teuthology.orchestra.run.smithi151.stdout:(136/159): python3-pyOpenSSL-21.0.0-1.el9.noarc 463 kB/s | 90 kB 00:00 2024-05-21T21:39:20.983 INFO:teuthology.orchestra.run.smithi184.stdout:(81/159): python3-markupsafe-1.1.1-12.el9.x86_6 320 kB/s | 35 kB 00:00 2024-05-21T21:39:21.004 INFO:teuthology.orchestra.run.smithi164.stdout:(147/159): re2-20211101-3.el9.x86_64.rpm 1.9 MB/s | 192 kB 00:00 2024-05-21T21:39:21.054 INFO:teuthology.orchestra.run.smithi164.stdout:(148/159): thrift-0.15.0-2.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-05-21T21:39:21.073 INFO:teuthology.orchestra.run.smithi151.stdout:(137/159): python3-repoze-lru-0.7-16.el9.noarch 243 kB/s | 31 kB 00:00 2024-05-21T21:39:21.132 INFO:teuthology.orchestra.run.smithi151.stdout:(138/159): python3-pecan-1.4.2-3.el9.noarch.rpm 590 kB/s | 272 kB 00:00 2024-05-21T21:39:21.240 INFO:teuthology.orchestra.run.smithi151.stdout:(139/159): python3-tempora-5.0.0-2.el9.noarch.r 331 kB/s | 36 kB 00:00 2024-05-21T21:39:21.265 INFO:teuthology.orchestra.run.smithi151.stdout:(140/159): python3-rsa-4.9-2.el9.noarch.rpm 309 kB/s | 59 kB 00:00 2024-05-21T21:39:21.299 INFO:teuthology.orchestra.run.smithi151.stdout:(141/159): python3-routes-2.5.1-5.el9.noarch.rp 579 kB/s | 188 kB 00:00 2024-05-21T21:39:21.349 INFO:teuthology.orchestra.run.smithi151.stdout:(142/159): python3-typing-extensions-4.4.0-2.el 472 kB/s | 51 kB 00:00 2024-05-21T21:39:21.350 INFO:teuthology.orchestra.run.smithi184.stdout:(82/159): python3-numpy-f2py-1.20.1-5.el9.x86_6 683 kB/s | 250 kB 00:00 2024-05-21T21:39:21.488 INFO:teuthology.orchestra.run.smithi164.stdout:(149/159): NetworkManager-1.47.90-1.el9.x86_64. 5.3 MB/s | 2.3 MB 00:00 2024-05-21T21:39:21.491 INFO:teuthology.orchestra.run.smithi151.stdout:(143/159): python3-websocket-client-1.2.3-2.el9 469 kB/s | 90 kB 00:00 2024-05-21T21:39:21.600 INFO:teuthology.orchestra.run.smithi151.stdout:(144/159): python3-zc-lockfile-2.0-10.el9.noarc 185 kB/s | 20 kB 00:00 2024-05-21T21:39:21.633 INFO:teuthology.orchestra.run.smithi151.stdout:(145/159): python3-webob-1.8.7-6.el9.noarch.rpm 626 kB/s | 230 kB 00:00 2024-05-21T21:39:21.671 INFO:teuthology.orchestra.run.smithi164.stdout:(150/159): NetworkManager-libnm-1.47.90-1.el9.x 9.8 MB/s | 1.8 MB 00:00 2024-05-21T21:39:21.701 INFO:teuthology.orchestra.run.smithi184.stdout:(83/159): python3-oauthlib-3.1.1-5.el9.noarch.r 635 kB/s | 222 kB 00:00 2024-05-21T21:39:21.721 INFO:teuthology.orchestra.run.smithi164.stdout:(151/159): NetworkManager-team-1.47.90-1.el9.x8 835 kB/s | 41 kB 00:00 2024-05-21T21:39:21.805 INFO:teuthology.orchestra.run.smithi164.stdout:(152/159): NetworkManager-tui-1.47.90-1.el9.x86 2.9 MB/s | 248 kB 00:00 2024-05-21T21:39:21.864 INFO:teuthology.orchestra.run.smithi164.stdout:(153/159): elfutils-debuginfod-client-0.191-4.e 631 kB/s | 37 kB 00:00 2024-05-21T21:39:21.875 INFO:teuthology.orchestra.run.smithi151.stdout:(146/159): re2-20211101-3.el9.x86_64.rpm 698 kB/s | 192 kB 00:00 2024-05-21T21:39:21.884 INFO:teuthology.orchestra.run.smithi184.stdout:(84/159): python3-packaging-20.9-5.el9.noarch.r 421 kB/s | 77 kB 00:00 2024-05-21T21:39:21.914 INFO:teuthology.orchestra.run.smithi164.stdout:(154/159): librbd1-19.0.0-3830.g3b748ea0.el9.x8 3.5 MB/s | 3.1 MB 00:00 2024-05-21T21:39:21.939 INFO:teuthology.orchestra.run.smithi164.stdout:(155/159): elfutils-libelf-0.191-4.el9.x86_64.r 2.7 MB/s | 207 kB 00:00 2024-05-21T21:39:21.950 INFO:teuthology.orchestra.run.smithi151.stdout:(147/159): python3-werkzeug-2.0.3-3.el9.1.noarc 712 kB/s | 427 kB 00:00 2024-05-21T21:39:21.993 INFO:teuthology.orchestra.run.smithi184.stdout:(85/159): python3-prettytable-0.7.2-27.el9.noar 389 kB/s | 42 kB 00:00 2024-05-21T21:39:22.031 INFO:teuthology.orchestra.run.smithi164.stdout:(156/159): elfutils-libs-0.191-4.el9.x86_64.rpm 2.2 MB/s | 260 kB 00:00 2024-05-21T21:39:22.065 INFO:teuthology.orchestra.run.smithi164.stdout:(157/159): openssl-3.2.1-1.el9.x86_64.rpm 11 MB/s | 1.3 MB 00:00 2024-05-21T21:39:22.182 INFO:teuthology.orchestra.run.smithi164.stdout:(158/159): librados2-19.0.0-3830.g3b748ea0.el9. 2.8 MB/s | 3.4 MB 00:01 2024-05-21T21:39:22.418 INFO:teuthology.orchestra.run.smithi184.stdout:(86/159): python3-protobuf-3.14.0-13.el9.noarch 634 kB/s | 269 kB 00:00 2024-05-21T21:39:22.449 INFO:teuthology.orchestra.run.smithi164.stdout:(159/159): openssl-libs-3.2.1-1.el9.x86_64.rpm 5.7 MB/s | 2.4 MB 00:00 2024-05-21T21:39:22.455 INFO:teuthology.orchestra.run.smithi164.stdout:-------------------------------------------------------------------------------- 2024-05-21T21:39:22.455 INFO:teuthology.orchestra.run.smithi164.stdout:Total 14 MB/s | 227 MB 00:16 2024-05-21T21:39:22.618 INFO:teuthology.orchestra.run.smithi151.stdout:(148/159): librados2-19.0.0-3830.g3b748ea0.el9. 4.6 MB/s | 3.4 MB 00:00 2024-05-21T21:39:22.685 INFO:teuthology.orchestra.run.smithi184.stdout:(87/159): python3-pyasn1-0.4.8-6.el9.noarch.rpm 595 kB/s | 159 kB 00:00 2024-05-21T21:39:22.853 INFO:teuthology.orchestra.run.smithi151.stdout:(149/159): NetworkManager-1.47.90-1.el9.x86_64. 9.9 MB/s | 2.3 MB 00:00 2024-05-21T21:39:22.959 INFO:teuthology.orchestra.run.smithi151.stdout:(150/159): NetworkManager-libnm-1.47.90-1.el9.x 17 MB/s | 1.8 MB 00:00 2024-05-21T21:39:23.010 INFO:teuthology.orchestra.run.smithi151.stdout:(151/159): NetworkManager-team-1.47.90-1.el9.x8 840 kB/s | 41 kB 00:00 2024-05-21T21:39:23.069 INFO:teuthology.orchestra.run.smithi151.stdout:(152/159): NetworkManager-tui-1.47.90-1.el9.x86 4.2 MB/s | 248 kB 00:00 2024-05-21T21:39:23.110 INFO:teuthology.orchestra.run.smithi184.stdout:(88/159): python3-pyasn1-modules-0.4.8-6.el9.no 656 kB/s | 279 kB 00:00 2024-05-21T21:39:23.119 INFO:teuthology.orchestra.run.smithi151.stdout:(153/159): elfutils-debuginfod-client-0.191-4.e 747 kB/s | 37 kB 00:00 2024-05-21T21:39:23.169 INFO:teuthology.orchestra.run.smithi151.stdout:(154/159): elfutils-libelf-0.191-4.el9.x86_64.r 4.1 MB/s | 207 kB 00:00 2024-05-21T21:39:23.219 INFO:teuthology.orchestra.run.smithi184.stdout:(89/159): python3-pytz-2021.1-5.el9.noarch.rpm 470 kB/s | 51 kB 00:00 2024-05-21T21:39:23.245 INFO:teuthology.orchestra.run.smithi151.stdout:(155/159): librbd1-19.0.0-3830.g3b748ea0.el9.x8 2.4 MB/s | 3.1 MB 00:01 2024-05-21T21:39:23.278 INFO:teuthology.orchestra.run.smithi151.stdout:(156/159): elfutils-libs-0.191-4.el9.x86_64.rpm 2.4 MB/s | 260 kB 00:00 2024-05-21T21:39:23.348 INFO:teuthology.orchestra.run.smithi184.stdout:(90/159): python3-requests-oauthlib-1.3.0-12.el 418 kB/s | 54 kB 00:00 2024-05-21T21:39:23.445 INFO:teuthology.orchestra.run.smithi151.stdout:(157/159): openssl-libs-3.2.1-1.el9.x86_64.rpm 14 MB/s | 2.4 MB 00:00 2024-05-21T21:39:23.504 INFO:teuthology.orchestra.run.smithi151.stdout:(158/159): openssl-3.2.1-1.el9.x86_64.rpm 5.2 MB/s | 1.3 MB 00:00 2024-05-21T21:39:23.655 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction check 2024-05-21T21:39:23.729 INFO:teuthology.orchestra.run.smithi151.stdout:(159/159): thrift-0.15.0-2.el9.x86_64.rpm 773 kB/s | 1.6 MB 00:02 2024-05-21T21:39:23.741 INFO:teuthology.orchestra.run.smithi151.stdout:-------------------------------------------------------------------------------- 2024-05-21T21:39:23.741 INFO:teuthology.orchestra.run.smithi151.stdout:Total 7.6 MB/s | 227 MB 00:29 2024-05-21T21:39:23.815 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction check succeeded. 2024-05-21T21:39:23.815 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction test 2024-05-21T21:39:25.626 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction test succeeded. 2024-05-21T21:39:25.627 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction 2024-05-21T21:39:26.611 INFO:teuthology.orchestra.run.smithi184.stdout:(91/159): python3-numpy-1.20.1-5.el9.x86_64.rpm 922 kB/s | 5.1 MB 00:05 2024-05-21T21:39:26.741 INFO:teuthology.orchestra.run.smithi184.stdout:(92/159): python3-toml-0.10.2-6.el9.noarch.rpm 320 kB/s | 42 kB 00:00 2024-05-21T21:39:26.802 INFO:teuthology.orchestra.run.smithi151.stdout:Running transaction check 2024-05-21T21:39:26.845 INFO:teuthology.orchestra.run.smithi184.stdout:(93/159): python3-wcwidth-0.2.5-8.el9.noarch.rp 428 kB/s | 44 kB 00:00 2024-05-21T21:39:26.945 INFO:teuthology.orchestra.run.smithi184.stdout:(94/159): qatlib-24.02.0-1.el9.x86_64.rpm 2.2 MB/s | 221 kB 00:00 2024-05-21T21:39:27.045 INFO:teuthology.orchestra.run.smithi184.stdout:(95/159): qatlib-service-24.02.0-1.el9.x86_64.r 358 kB/s | 36 kB 00:00 2024-05-21T21:39:27.058 INFO:teuthology.orchestra.run.smithi151.stdout:Transaction check succeeded. 2024-05-21T21:39:27.058 INFO:teuthology.orchestra.run.smithi151.stdout:Running transaction test 2024-05-21T21:39:27.154 INFO:teuthology.orchestra.run.smithi184.stdout:(96/159): qatzip-libs-1.2.0-1.el9.x86_64.rpm 438 kB/s | 47 kB 00:00 2024-05-21T21:39:27.263 INFO:teuthology.orchestra.run.smithi184.stdout:(97/159): socat-1.7.4.1-5.el9.x86_64.rpm 2.7 MB/s | 305 kB 00:00 2024-05-21T21:39:27.363 INFO:teuthology.orchestra.run.smithi184.stdout:(98/159): uuid-1.6.2-55.el9.x86_64.rpm 578 kB/s | 58 kB 00:00 2024-05-21T21:39:27.472 INFO:teuthology.orchestra.run.smithi184.stdout:(99/159): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 864 kB/s | 94 kB 00:00 2024-05-21T21:39:27.631 INFO:teuthology.orchestra.run.smithi184.stdout:(100/159): python3-babel-2.9.1-2.el9.noarch.rpm 748 kB/s | 6.0 MB 00:08 2024-05-21T21:39:27.664 INFO:teuthology.orchestra.run.smithi184.stdout:(101/159): xmlstarlet-1.6.1-20.el9.x86_64.rpm 331 kB/s | 64 kB 00:00 2024-05-21T21:39:27.715 INFO:teuthology.orchestra.run.smithi184.stdout:(102/159): lua-devel-5.4.4-4.el9.x86_64.rpm 446 kB/s | 22 kB 00:00 2024-05-21T21:39:27.740 INFO:teuthology.orchestra.run.smithi184.stdout:(103/159): zlib-devel-1.2.11-41.el9.x86_64.rpm 412 kB/s | 45 kB 00:00 2024-05-21T21:39:27.840 INFO:teuthology.orchestra.run.smithi184.stdout:(104/159): protobuf-compiler-3.14.0-13.el9.x86_ 6.7 MB/s | 863 kB 00:00 2024-05-21T21:39:28.065 INFO:teuthology.orchestra.run.smithi184.stdout:(105/159): gperftools-libs-2.9.1-2.el9.x86_64.r 1.3 MB/s | 309 kB 00:00 2024-05-21T21:39:28.099 INFO:teuthology.orchestra.run.smithi184.stdout:(106/159): abseil-cpp-20211102.0-3.el9.x86_64.r 1.5 MB/s | 548 kB 00:00 2024-05-21T21:39:28.116 INFO:teuthology.orchestra.run.smithi184.stdout:(107/159): grpc-data-1.46.7-10.el9.noarch.rpm 387 kB/s | 19 kB 00:00 2024-05-21T21:39:28.149 INFO:teuthology.orchestra.run.smithi184.stdout:(108/159): libarrow-doc-9.0.0-11.el9.noarch.rpm 783 kB/s | 26 kB 00:00 2024-05-21T21:39:28.183 INFO:teuthology.orchestra.run.smithi184.stdout:(109/159): liboath-2.6.7-2.el9.x86_64.rpm 1.4 MB/s | 49 kB 00:00 2024-05-21T21:39:28.242 INFO:teuthology.orchestra.run.smithi184.stdout:(110/159): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-05-21T21:39:28.334 INFO:teuthology.orchestra.run.smithi184.stdout:(111/159): luarocks-3.9.2-5.el9.noarch.rpm 1.6 MB/s | 151 kB 00:00 2024-05-21T21:39:28.576 INFO:teuthology.orchestra.run.smithi184.stdout:(112/159): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.4 MB/s | 839 kB 00:00 2024-05-21T21:39:28.743 INFO:teuthology.orchestra.run.smithi184.stdout:(113/159): python3-asyncssh-2.13.2-5.el9.noarch 3.2 MB/s | 548 kB 00:00 2024-05-21T21:39:28.785 INFO:teuthology.orchestra.run.smithi184.stdout:(114/159): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.0 MB/s | 43 kB 00:00 2024-05-21T21:39:28.827 INFO:teuthology.orchestra.run.smithi184.stdout:(115/159): python3-cachetools-4.2.4-1.el9.noarc 773 kB/s | 32 kB 00:00 2024-05-21T21:39:28.869 INFO:teuthology.orchestra.run.smithi184.stdout:(116/159): python3-certifi-2021.10.8-2.el9.noar 361 kB/s | 15 kB 00:00 2024-05-21T21:39:28.944 INFO:teuthology.orchestra.run.smithi184.stdout:(117/159): python3-cheroot-8.6.0-4.el9.noarch.r 2.2 MB/s | 172 kB 00:00 2024-05-21T21:39:29.062 INFO:teuthology.orchestra.run.smithi184.stdout:(118/159): python3-cherrypy-18.6.1-2.el9.noarch 3.0 MB/s | 358 kB 00:00 2024-05-21T21:39:29.163 INFO:teuthology.orchestra.run.smithi184.stdout:(119/159): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.9 MB/s | 303 kB 00:00 2024-05-21T21:39:29.238 INFO:teuthology.orchestra.run.smithi184.stdout:(120/159): python3-google-auth-2.29.0-1.el9.noa 2.7 MB/s | 206 kB 00:00 2024-05-21T21:39:29.331 INFO:teuthology.orchestra.run.smithi184.stdout:(121/159): libarrow-9.0.0-11.el9.x86_64.rpm 3.6 MB/s | 4.4 MB 00:01 2024-05-21T21:39:29.410 INFO:teuthology.orchestra.run.smithi184.stdout:(122/159): python3-grpcio-tools-1.46.7-10.el9.x 1.9 MB/s | 144 kB 00:00 2024-05-21T21:39:29.449 INFO:teuthology.orchestra.run.smithi184.stdout:(123/159): python3-jaraco-8.2.1-3.el9.noarch.rp 256 kB/s | 11 kB 00:00 2024-05-21T21:39:29.491 INFO:teuthology.orchestra.run.smithi184.stdout:(124/159): python3-jaraco-classes-3.2.1-5.el9.n 423 kB/s | 18 kB 00:00 2024-05-21T21:39:29.533 INFO:teuthology.orchestra.run.smithi184.stdout:(125/159): python3-jaraco-collections-3.0.0-8.e 554 kB/s | 23 kB 00:00 2024-05-21T21:39:29.575 INFO:teuthology.orchestra.run.smithi184.stdout:(126/159): python3-jaraco-functools-3.5.0-2.el9 465 kB/s | 19 kB 00:00 2024-05-21T21:39:29.617 INFO:teuthology.orchestra.run.smithi184.stdout:(127/159): python3-jaraco-text-3.2.0-6.el9.noar 473 kB/s | 20 kB 00:00 2024-05-21T21:39:29.636 INFO:teuthology.orchestra.run.smithi151.stdout:Transaction test succeeded. 2024-05-21T21:39:29.637 INFO:teuthology.orchestra.run.smithi151.stdout:Running transaction 2024-05-21T21:39:29.909 INFO:teuthology.orchestra.run.smithi184.stdout:(128/159): python3-grpcio-1.46.7-10.el9.x86_64. 3.0 MB/s | 2.0 MB 00:00 2024-05-21T21:39:29.942 INFO:teuthology.orchestra.run.smithi184.stdout:(129/159): python3-kubernetes-26.1.0-2.el9.noar 3.1 MB/s | 1.0 MB 00:00 2024-05-21T21:39:29.959 INFO:teuthology.orchestra.run.smithi184.stdout:(130/159): python3-logutils-0.3.5-21.el9.noarch 926 kB/s | 46 kB 00:00 2024-05-21T21:39:29.985 INFO:teuthology.orchestra.run.smithi184.stdout:(131/159): python3-more-itertools-8.12.0-2.el9. 1.8 MB/s | 79 kB 00:00 2024-05-21T21:39:30.010 INFO:teuthology.orchestra.run.smithi184.stdout:(132/159): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-05-21T21:39:30.052 INFO:teuthology.orchestra.run.smithi184.stdout:(133/159): python3-portend-3.1.0-2.el9.noarch.r 392 kB/s | 16 kB 00:00 2024-05-21T21:39:30.136 INFO:teuthology.orchestra.run.smithi184.stdout:(134/159): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.1 MB/s | 90 kB 00:00 2024-05-21T21:39:30.169 INFO:teuthology.orchestra.run.smithi184.stdout:(135/159): python3-pecan-1.4.2-3.el9.noarch.rpm 1.4 MB/s | 272 kB 00:00 2024-05-21T21:39:30.194 INFO:teuthology.orchestra.run.smithi184.stdout:(136/159): python3-pyperclip-1.8.2-2.el9.noarch 482 kB/s | 28 kB 00:00 2024-05-21T21:39:30.220 INFO:teuthology.orchestra.run.smithi184.stdout:(137/159): python3-repoze-lru-0.7-16.el9.noarch 611 kB/s | 31 kB 00:00 2024-05-21T21:39:30.278 INFO:teuthology.orchestra.run.smithi184.stdout:(138/159): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2024-05-21T21:39:30.304 INFO:teuthology.orchestra.run.smithi184.stdout:(139/159): python3-rsa-4.9-2.el9.noarch.rpm 707 kB/s | 59 kB 00:00 2024-05-21T21:39:30.329 INFO:teuthology.orchestra.run.smithi184.stdout:(140/159): python3-tempora-5.0.0-2.el9.noarch.r 710 kB/s | 36 kB 00:00 2024-05-21T21:39:30.354 INFO:teuthology.orchestra.run.smithi184.stdout:(141/159): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2024-05-21T21:39:30.413 INFO:teuthology.orchestra.run.smithi184.stdout:(142/159): python3-websocket-client-1.2.3-2.el9 1.5 MB/s | 90 kB 00:00 2024-05-21T21:39:30.455 INFO:teuthology.orchestra.run.smithi184.stdout:(143/159): python3-webob-1.8.7-6.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2024-05-21T21:39:30.497 INFO:teuthology.orchestra.run.smithi184.stdout:(144/159): python3-zc-lockfile-2.0-10.el9.noarc 478 kB/s | 20 kB 00:00 2024-05-21T21:39:30.564 INFO:teuthology.orchestra.run.smithi184.stdout:(145/159): python3-werkzeug-2.0.3-3.el9.1.noarc 2.8 MB/s | 427 kB 00:00 2024-05-21T21:39:30.623 INFO:teuthology.orchestra.run.smithi184.stdout:(146/159): re2-20211101-3.el9.x86_64.rpm 1.5 MB/s | 192 kB 00:00 2024-05-21T21:39:30.948 INFO:teuthology.orchestra.run.smithi184.stdout:(147/159): thrift-0.15.0-2.el9.x86_64.rpm 4.1 MB/s | 1.6 MB 00:00 2024-05-21T21:39:31.082 INFO:teuthology.orchestra.run.smithi184.stdout:(148/159): librados2-19.0.0-3830.g3b748ea0.el9. 7.4 MB/s | 3.4 MB 00:00 2024-05-21T21:39:31.407 INFO:teuthology.orchestra.run.smithi184.stdout:(149/159): NetworkManager-1.47.90-1.el9.x86_64. 7.1 MB/s | 2.3 MB 00:00 2024-05-21T21:39:31.524 INFO:teuthology.orchestra.run.smithi184.stdout:(150/159): NetworkManager-libnm-1.47.90-1.el9.x 15 MB/s | 1.8 MB 00:00 2024-05-21T21:39:31.582 INFO:teuthology.orchestra.run.smithi184.stdout:(151/159): NetworkManager-team-1.47.90-1.el9.x8 711 kB/s | 41 kB 00:00 2024-05-21T21:39:31.641 INFO:teuthology.orchestra.run.smithi184.stdout:(152/159): NetworkManager-tui-1.47.90-1.el9.x86 4.2 MB/s | 248 kB 00:00 2024-05-21T21:39:31.700 INFO:teuthology.orchestra.run.smithi184.stdout:(153/159): elfutils-debuginfod-client-0.191-4.e 630 kB/s | 37 kB 00:00 2024-05-21T21:39:31.800 INFO:teuthology.orchestra.run.smithi184.stdout:(154/159): librbd1-19.0.0-3830.g3b748ea0.el9.x8 3.7 MB/s | 3.1 MB 00:00 2024-05-21T21:39:31.825 INFO:teuthology.orchestra.run.smithi184.stdout:(155/159): elfutils-libelf-0.191-4.el9.x86_64.r 1.6 MB/s | 207 kB 00:00 2024-05-21T21:39:31.839 INFO:teuthology.orchestra.run.smithi164.stdout: Preparing : 1/1 2024-05-21T21:39:31.919 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/170 2024-05-21T21:39:31.934 INFO:teuthology.orchestra.run.smithi184.stdout:(156/159): openssl-3.2.1-1.el9.x86_64.rpm 12 MB/s | 1.3 MB 00:00 2024-05-21T21:39:31.957 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/170 2024-05-21T21:39:32.020 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/170 2024-05-21T21:39:32.068 INFO:teuthology.orchestra.run.smithi184.stdout:(157/159): elfutils-libs-0.191-4.el9.x86_64.rpm 971 kB/s | 260 kB 00:00 2024-05-21T21:39:32.120 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/170 2024-05-21T21:39:32.126 INFO:teuthology.orchestra.run.smithi184.stdout:(158/159): openssl-libs-3.2.1-1.el9.x86_64.rpm 12 MB/s | 2.4 MB 00:00 2024-05-21T21:39:32.146 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/170 2024-05-21T21:39:32.180 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/170 2024-05-21T21:39:32.507 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/170 2024-05-21T21:39:32.622 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : librados2-2:19.0.0-3830.g3b748ea0.el9.x86_64 8/170 2024-05-21T21:39:32.743 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librados2-2:19.0.0-3830.g3b748ea0.el9.x86_64 8/170 2024-05-21T21:39:32.794 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libcephfs2-2:19.0.0-3830.g3b748ea0.el9.x86_64 9/170 2024-05-21T21:39:32.873 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: libcephfs2-2:19.0.0-3830.g3b748ea0.el9.x86_64 9/170 2024-05-21T21:39:32.902 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-rados-2:19.0.0-3830.g3b748ea0.el9.x86_64 10/170 2024-05-21T21:39:32.935 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/170 2024-05-21T21:39:33.128 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/170 2024-05-21T21:39:33.184 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/170 2024-05-21T21:39:33.220 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libcephsqlite-2:19.0.0-3830.g3b748ea0.el9.x86_64 14/170 2024-05-21T21:39:33.324 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3830.g3b748ea0.el9.x86_64 14/170 2024-05-21T21:39:33.354 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libradosstriper1-2:19.0.0-3830.g3b748ea0.el9.x86 15/170 2024-05-21T21:39:33.417 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3830.g3b748ea0.el9.x86 15/170 2024-05-21T21:39:33.439 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-21T21:39:33.461 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-21T21:39:33.646 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-21T21:39:33.773 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-21T21:39:33.913 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 18/170 2024-05-21T21:39:33.971 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 18/170 2024-05-21T21:39:34.068 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-21T21:39:34.139 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-21T21:39:34.334 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-21T21:39:34.436 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : re2-1:20211101-3.el9.x86_64 20/170 2024-05-21T21:39:34.460 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/170 2024-05-21T21:39:34.496 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/170 2024-05-21T21:39:34.569 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/170 2024-05-21T21:39:34.604 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/170 2024-05-21T21:39:34.648 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/170 2024-05-21T21:39:34.680 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 26/170 2024-05-21T21:39:34.708 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : lua-5.4.4-4.el9.x86_64 27/170 2024-05-21T21:39:34.769 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/170 2024-05-21T21:39:34.822 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 29/170 2024-05-21T21:39:34.853 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/170 2024-05-21T21:39:34.888 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 31/170 2024-05-21T21:39:34.964 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 32/170 2024-05-21T21:39:34.999 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-ceph-common-2:19.0.0-3830.g3b748ea0.el9. 33/170 2024-05-21T21:39:35.042 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-ceph-argparse-2:19.0.0-3830.g3b748ea0.el 34/170 2024-05-21T21:39:35.071 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cephfs-2:19.0.0-3830.g3b748ea0.el9.x86_6 35/170 2024-05-21T21:39:35.149 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 36/170 2024-05-21T21:39:35.203 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : luarocks-3.9.2-5.el9.noarch 37/170 2024-05-21T21:39:35.276 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 38/170 2024-05-21T21:39:35.408 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 39/170 2024-05-21T21:39:35.463 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 40/170 2024-05-21T21:39:35.985 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-rsa-4.9-2.el9.noarch 41/170 2024-05-21T21:39:36.066 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 42/170 2024-05-21T21:39:36.096 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 43/170 2024-05-21T21:39:36.132 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 44/170 2024-05-21T21:39:36.174 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 45/170 2024-05-21T21:39:36.195 INFO:teuthology.orchestra.run.smithi151.stdout: Preparing : 1/1 2024-05-21T21:39:36.211 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : librados-devel-2:19.0.0-3830.g3b748ea0.el9.x86_6 46/170 2024-05-21T21:39:36.253 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/170 2024-05-21T21:39:36.341 INFO:teuthology.orchestra.run.smithi151.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/170 2024-05-21T21:39:36.393 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 48/170 2024-05-21T21:39:36.415 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/170 2024-05-21T21:39:36.447 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 49/170 2024-05-21T21:39:36.528 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/170 2024-05-21T21:39:36.538 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/170 2024-05-21T21:39:36.581 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/170 2024-05-21T21:39:36.603 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/170 2024-05-21T21:39:36.612 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/170 2024-05-21T21:39:36.675 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/170 2024-05-21T21:39:36.707 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/170 2024-05-21T21:39:36.711 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/170 2024-05-21T21:39:36.772 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/170 2024-05-21T21:39:36.814 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/170 2024-05-21T21:39:36.851 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/170 2024-05-21T21:39:36.895 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/170 2024-05-21T21:39:36.936 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/170 2024-05-21T21:39:37.073 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/170 2024-05-21T21:39:37.107 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/170 2024-05-21T21:39:37.114 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/170 2024-05-21T21:39:37.259 INFO:teuthology.orchestra.run.smithi151.stdout: Upgrading : librados2-2:19.0.0-3830.g3b748ea0.el9.x86_64 8/170 2024-05-21T21:39:37.341 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 61/170 2024-05-21T21:39:37.442 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: librados2-2:19.0.0-3830.g3b748ea0.el9.x86_64 8/170 2024-05-21T21:39:37.469 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libcephfs2-2:19.0.0-3830.g3b748ea0.el9.x86_64 9/170 2024-05-21T21:39:37.579 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: libcephfs2-2:19.0.0-3830.g3b748ea0.el9.x86_64 9/170 2024-05-21T21:39:37.626 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-rados-2:19.0.0-3830.g3b748ea0.el9.x86_64 10/170 2024-05-21T21:39:37.641 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/170 2024-05-21T21:39:37.712 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/170 2024-05-21T21:39:37.747 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 63/170 2024-05-21T21:39:37.787 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/170 2024-05-21T21:39:37.813 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/170 2024-05-21T21:39:37.837 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/170 2024-05-21T21:39:37.913 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/170 2024-05-21T21:39:38.039 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/170 2024-05-21T21:39:38.078 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/170 2024-05-21T21:39:38.094 INFO:teuthology.orchestra.run.smithi151.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/170 2024-05-21T21:39:38.105 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/170 2024-05-21T21:39:38.139 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/170 2024-05-21T21:39:38.170 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/170 2024-05-21T21:39:38.178 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/170 2024-05-21T21:39:38.208 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libcephsqlite-2:19.0.0-3830.g3b748ea0.el9.x86_64 14/170 2024-05-21T21:39:38.221 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/170 2024-05-21T21:39:38.257 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/170 2024-05-21T21:39:38.292 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/170 2024-05-21T21:39:38.325 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/170 2024-05-21T21:39:38.366 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/170 2024-05-21T21:39:38.370 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3830.g3b748ea0.el9.x86_64 14/170 2024-05-21T21:39:38.401 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/170 2024-05-21T21:39:38.409 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libradosstriper1-2:19.0.0-3830.g3b748ea0.el9.x86 15/170 2024-05-21T21:39:38.495 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3830.g3b748ea0.el9.x86 15/170 2024-05-21T21:39:38.496 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/170 2024-05-21T21:39:38.526 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-21T21:39:38.548 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/170 2024-05-21T21:39:38.579 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/170 2024-05-21T21:39:38.601 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-21T21:39:38.627 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/170 2024-05-21T21:39:38.724 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-packaging-20.9-5.el9.noarch 83/170 2024-05-21T21:39:38.781 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 84/170 2024-05-21T21:39:39.047 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-21T21:39:39.341 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-21T21:39:39.509 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 85/170 2024-05-21T21:39:39.561 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 86/170 2024-05-21T21:39:39.594 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 87/170 2024-05-21T21:39:39.621 INFO:teuthology.orchestra.run.smithi151.stdout: Upgrading : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 18/170 2024-05-21T21:39:39.695 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 88/170 2024-05-21T21:39:39.750 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 89/170 2024-05-21T21:39:39.795 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 90/170 2024-05-21T21:39:39.882 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 91/170 2024-05-21T21:39:39.914 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 92/170 2024-05-21T21:39:39.945 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 18/170 2024-05-21T21:39:40.083 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/170 2024-05-21T21:39:40.111 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/170 2024-05-21T21:39:40.146 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-21T21:39:40.189 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/170 2024-05-21T21:39:40.253 INFO:teuthology.orchestra.run.smithi151.stdout: Upgrading : NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-21T21:39:40.576 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-21T21:39:40.772 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : re2-1:20211101-3.el9.x86_64 20/170 2024-05-21T21:39:40.808 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/170 2024-05-21T21:39:40.844 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/170 2024-05-21T21:39:40.861 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/170 2024-05-21T21:39:40.961 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/170 2024-05-21T21:39:40.968 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/170 2024-05-21T21:39:41.020 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/170 2024-05-21T21:39:41.118 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/170 2024-05-21T21:39:41.177 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 26/170 2024-05-21T21:39:41.216 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : lua-5.4.4-4.el9.x86_64 27/170 2024-05-21T21:39:41.336 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/170 2024-05-21T21:39:41.403 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 29/170 2024-05-21T21:39:41.441 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/170 2024-05-21T21:39:41.495 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 31/170 2024-05-21T21:39:41.606 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 32/170 2024-05-21T21:39:41.645 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-ceph-common-2:19.0.0-3830.g3b748ea0.el9. 33/170 2024-05-21T21:39:41.692 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-ceph-argparse-2:19.0.0-3830.g3b748ea0.el 34/170 2024-05-21T21:39:41.726 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-cephfs-2:19.0.0-3830.g3b748ea0.el9.x86_6 35/170 2024-05-21T21:39:41.840 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 36/170 2024-05-21T21:39:41.905 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : luarocks-3.9.2-5.el9.noarch 37/170 2024-05-21T21:39:42.006 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 38/170 2024-05-21T21:39:42.197 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 39/170 2024-05-21T21:39:42.216 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/170 2024-05-21T21:39:42.259 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 40/170 2024-05-21T21:39:42.307 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/170 2024-05-21T21:39:42.334 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 100/170 2024-05-21T21:39:42.368 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 101/170 2024-05-21T21:39:42.682 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 102/170 2024-05-21T21:39:42.730 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : librbd1-2:19.0.0-3830.g3b748ea0.el9.x86_64 103/170 2024-05-21T21:39:42.821 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librbd1-2:19.0.0-3830.g3b748ea0.el9.x86_64 103/170 2024-05-21T21:39:42.852 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-rbd-2:19.0.0-3830.g3b748ea0.el9.x86_64 104/170 2024-05-21T21:39:42.880 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 105/170 2024-05-21T21:39:42.980 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-rsa-4.9-2.el9.noarch 41/170 2024-05-21T21:39:43.089 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 42/170 2024-05-21T21:39:43.124 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 43/170 2024-05-21T21:39:43.166 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 44/170 2024-05-21T21:39:43.227 INFO:teuthology.orchestra.run.smithi151.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 45/170 2024-05-21T21:39:43.273 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : librados-devel-2:19.0.0-3830.g3b748ea0.el9.x86_6 46/170 2024-05-21T21:39:43.365 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/170 2024-05-21T21:39:43.365 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 106/170 2024-05-21T21:39:43.400 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : librgw2-2:19.0.0-3830.g3b748ea0.el9.x86_64 107/170 2024-05-21T21:39:43.462 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librgw2-2:19.0.0-3830.g3b748ea0.el9.x86_64 107/170 2024-05-21T21:39:43.485 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-rgw-2:19.0.0-3830.g3b748ea0.el9.x86_64 108/170 2024-05-21T21:39:43.589 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 48/170 2024-05-21T21:39:43.695 INFO:teuthology.orchestra.run.smithi151.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 49/170 2024-05-21T21:39:43.828 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/170 2024-05-21T21:39:43.909 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/170 2024-05-21T21:39:44.021 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/170 2024-05-21T21:39:44.072 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/170 2024-05-21T21:39:44.118 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/170 2024-05-21T21:39:44.187 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/170 2024-05-21T21:39:44.234 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/170 2024-05-21T21:39:44.298 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/170 2024-05-21T21:39:44.373 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/170 2024-05-21T21:39:44.583 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/170 2024-05-21T21:39:44.627 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/170 2024-05-21T21:39:44.680 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 61/170 2024-05-21T21:39:44.810 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/170 2024-05-21T21:39:44.859 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 63/170 2024-05-21T21:39:44.939 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/170 2024-05-21T21:39:44.974 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/170 2024-05-21T21:39:45.008 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/170 2024-05-21T21:39:45.175 INFO:teuthology.orchestra.run.smithi184.stdout:(159/159): python3-scipy-1.6.2-8.el9.x86_64.rpm 762 kB/s | 16 MB 00:21 2024-05-21T21:39:45.181 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2024-05-21T21:39:45.181 INFO:teuthology.orchestra.run.smithi184.stdout:Total 4.7 MB/s | 227 MB 00:48 2024-05-21T21:39:45.204 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/170 2024-05-21T21:39:45.387 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/170 2024-05-21T21:39:45.439 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/170 2024-05-21T21:39:45.488 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/170 2024-05-21T21:39:45.532 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/170 2024-05-21T21:39:45.589 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/170 2024-05-21T21:39:45.641 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/170 2024-05-21T21:39:45.683 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/170 2024-05-21T21:39:45.728 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/170 2024-05-21T21:39:45.777 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/170 2024-05-21T21:39:45.779 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-common-2:19.0.0-3830.g3b748ea0.el9.x86_64 109/170 2024-05-21T21:39:45.815 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-common-2:19.0.0-3830.g3b748ea0.el9.x86_64 109/170 2024-05-21T21:39:45.843 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/170 2024-05-21T21:39:45.893 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/170 2024-05-21T21:39:46.043 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/170 2024-05-21T21:39:46.103 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/170 2024-05-21T21:39:46.152 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/170 2024-05-21T21:39:46.221 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/170 2024-05-21T21:39:46.358 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-common-2:19.0.0-3830.g3b748ea0.el9.x86_64 109/170 2024-05-21T21:39:46.361 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-packaging-20.9-5.el9.noarch 83/170 2024-05-21T21:39:46.401 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-base-2:19.0.0-3830.g3b748ea0.el9.x86_64 110/170 2024-05-21T21:39:46.423 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 84/170 2024-05-21T21:39:46.428 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2024-05-21T21:39:46.490 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-base-2:19.0.0-3830.g3b748ea0.el9.x86_64 110/170 2024-05-21T21:39:46.491 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-21T21:39:46.491 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-21T21:39:46.491 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-21T21:39:46.519 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-selinux-2:19.0.0-3830.g3b748ea0.el9.x86_64 111/170 2024-05-21T21:39:46.586 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2024-05-21T21:39:46.587 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2024-05-21T21:39:47.380 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 85/170 2024-05-21T21:39:47.446 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 86/170 2024-05-21T21:39:47.480 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 87/170 2024-05-21T21:39:47.609 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 88/170 2024-05-21T21:39:47.672 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 89/170 2024-05-21T21:39:47.725 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 90/170 2024-05-21T21:39:47.828 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 91/170 2024-05-21T21:39:47.874 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 92/170 2024-05-21T21:39:48.108 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/170 2024-05-21T21:39:48.154 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/170 2024-05-21T21:39:48.276 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/170 2024-05-21T21:39:48.397 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2024-05-21T21:39:48.397 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2024-05-21T21:39:49.125 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/170 2024-05-21T21:39:49.268 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/170 2024-05-21T21:39:50.914 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/170 2024-05-21T21:39:51.009 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/170 2024-05-21T21:39:51.045 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 100/170 2024-05-21T21:39:51.080 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 101/170 2024-05-21T21:39:51.448 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 102/170 2024-05-21T21:39:51.488 INFO:teuthology.orchestra.run.smithi151.stdout: Upgrading : librbd1-2:19.0.0-3830.g3b748ea0.el9.x86_64 103/170 2024-05-21T21:39:51.617 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: librbd1-2:19.0.0-3830.g3b748ea0.el9.x86_64 103/170 2024-05-21T21:39:51.653 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-rbd-2:19.0.0-3830.g3b748ea0.el9.x86_64 104/170 2024-05-21T21:39:51.709 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 105/170 2024-05-21T21:39:52.243 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 106/170 2024-05-21T21:39:52.275 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : librgw2-2:19.0.0-3830.g3b748ea0.el9.x86_64 107/170 2024-05-21T21:39:52.358 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: librgw2-2:19.0.0-3830.g3b748ea0.el9.x86_64 107/170 2024-05-21T21:39:52.385 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-rgw-2:19.0.0-3830.g3b748ea0.el9.x86_64 108/170 2024-05-21T21:39:54.577 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2024-05-21T21:39:54.645 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/170 2024-05-21T21:39:54.684 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/170 2024-05-21T21:39:54.745 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/170 2024-05-21T21:39:54.780 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/170 2024-05-21T21:39:54.805 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/170 2024-05-21T21:39:54.848 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/170 2024-05-21T21:39:55.017 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-common-2:19.0.0-3830.g3b748ea0.el9.x86_64 109/170 2024-05-21T21:39:55.055 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-common-2:19.0.0-3830.g3b748ea0.el9.x86_64 109/170 2024-05-21T21:39:55.230 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/170 2024-05-21T21:39:55.281 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : librados2-2:19.0.0-3830.g3b748ea0.el9.x86_64 8/170 2024-05-21T21:39:55.402 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: librados2-2:19.0.0-3830.g3b748ea0.el9.x86_64 8/170 2024-05-21T21:39:55.424 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libcephfs2-2:19.0.0-3830.g3b748ea0.el9.x86_64 9/170 2024-05-21T21:39:55.515 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: libcephfs2-2:19.0.0-3830.g3b748ea0.el9.x86_64 9/170 2024-05-21T21:39:55.536 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-rados-2:19.0.0-3830.g3b748ea0.el9.x86_64 10/170 2024-05-21T21:39:55.569 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/170 2024-05-21T21:39:55.693 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-common-2:19.0.0-3830.g3b748ea0.el9.x86_64 109/170 2024-05-21T21:39:55.742 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-base-2:19.0.0-3830.g3b748ea0.el9.x86_64 110/170 2024-05-21T21:39:55.760 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/170 2024-05-21T21:39:55.820 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/170 2024-05-21T21:39:55.848 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libcephsqlite-2:19.0.0-3830.g3b748ea0.el9.x86_64 14/170 2024-05-21T21:39:55.952 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3830.g3b748ea0.el9.x86_64 14/170 2024-05-21T21:39:55.953 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-base-2:19.0.0-3830.g3b748ea0.el9.x86_64 110/170 2024-05-21T21:39:55.953 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-21T21:39:55.953 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-21T21:39:55.953 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T21:39:55.974 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libradosstriper1-2:19.0.0-3830.g3b748ea0.el9.x86 15/170 2024-05-21T21:39:55.984 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-selinux-2:19.0.0-3830.g3b748ea0.el9.x86_64 111/170 2024-05-21T21:39:56.037 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3830.g3b748ea0.el9.x86 15/170 2024-05-21T21:39:56.059 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-21T21:39:56.085 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-21T21:39:56.276 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-21T21:39:56.406 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-21T21:39:56.581 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 18/170 2024-05-21T21:39:56.641 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 18/170 2024-05-21T21:39:56.747 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-21T21:39:56.826 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-21T21:39:57.023 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-21T21:39:57.123 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : re2-1:20211101-3.el9.x86_64 20/170 2024-05-21T21:39:57.155 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/170 2024-05-21T21:39:57.190 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/170 2024-05-21T21:39:57.271 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/170 2024-05-21T21:39:57.319 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/170 2024-05-21T21:39:57.359 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/170 2024-05-21T21:39:57.391 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 26/170 2024-05-21T21:39:57.419 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : lua-5.4.4-4.el9.x86_64 27/170 2024-05-21T21:39:57.513 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/170 2024-05-21T21:39:57.566 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 29/170 2024-05-21T21:39:57.597 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/170 2024-05-21T21:39:57.632 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 31/170 2024-05-21T21:39:57.708 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 32/170 2024-05-21T21:39:57.743 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-ceph-common-2:19.0.0-3830.g3b748ea0.el9. 33/170 2024-05-21T21:39:57.786 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-ceph-argparse-2:19.0.0-3830.g3b748ea0.el 34/170 2024-05-21T21:39:57.815 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cephfs-2:19.0.0-3830.g3b748ea0.el9.x86_6 35/170 2024-05-21T21:39:57.884 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 36/170 2024-05-21T21:39:57.931 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : luarocks-3.9.2-5.el9.noarch 37/170 2024-05-21T21:39:58.005 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 38/170 2024-05-21T21:39:58.146 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 39/170 2024-05-21T21:39:58.158 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3830.g3b748ea0.el9.x86_64 111/170 2024-05-21T21:39:58.158 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /sys 2024-05-21T21:39:58.158 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /proc 2024-05-21T21:39:58.158 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /mnt 2024-05-21T21:39:58.158 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /var/tmp 2024-05-21T21:39:58.158 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /home 2024-05-21T21:39:58.158 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /root 2024-05-21T21:39:58.158 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /tmp 2024-05-21T21:39:58.159 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-21T21:39:58.199 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 40/170 2024-05-21T21:39:58.344 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-volume-2:19.0.0-3830.g3b748ea0.el9.noarch 112/170 2024-05-21T21:39:58.360 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-volume-2:19.0.0-3830.g3b748ea0.el9.noarch 112/170 2024-05-21T21:39:58.360 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T21:39:58.360 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-21T21:39:58.360 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-21T21:39:58.724 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-rsa-4.9-2.el9.noarch 41/170 2024-05-21T21:39:58.811 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 42/170 2024-05-21T21:39:58.841 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 43/170 2024-05-21T21:39:58.877 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 44/170 2024-05-21T21:39:58.926 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 45/170 2024-05-21T21:39:58.964 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : librados-devel-2:19.0.0-3830.g3b748ea0.el9.x86_6 46/170 2024-05-21T21:39:59.005 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/170 2024-05-21T21:39:59.162 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 48/170 2024-05-21T21:39:59.226 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 49/170 2024-05-21T21:39:59.318 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/170 2024-05-21T21:39:59.381 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/170 2024-05-21T21:39:59.455 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/170 2024-05-21T21:39:59.485 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/170 2024-05-21T21:39:59.522 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/170 2024-05-21T21:39:59.577 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/170 2024-05-21T21:39:59.614 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/170 2024-05-21T21:39:59.656 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/170 2024-05-21T21:39:59.707 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/170 2024-05-21T21:39:59.842 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/170 2024-05-21T21:39:59.853 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-osd-2:19.0.0-3830.g3b748ea0.el9.x86_64 113/170 2024-05-21T21:39:59.879 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-osd-2:19.0.0-3830.g3b748ea0.el9.x86_64 113/170 2024-05-21T21:39:59.879 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T21:39:59.879 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-21T21:39:59.879 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-21T21:39:59.880 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-21T21:39:59.880 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-21T21:39:59.883 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/170 2024-05-21T21:39:59.925 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 61/170 2024-05-21T21:40:00.019 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/170 2024-05-21T21:40:00.050 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 63/170 2024-05-21T21:40:00.090 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/170 2024-05-21T21:40:00.103 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mds-2:19.0.0-3830.g3b748ea0.el9.x86_64 114/170 2024-05-21T21:40:00.116 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/170 2024-05-21T21:40:00.139 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mds-2:19.0.0-3830.g3b748ea0.el9.x86_64 114/170 2024-05-21T21:40:00.139 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T21:40:00.139 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-21T21:40:00.139 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-21T21:40:00.140 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-21T21:40:00.140 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-21T21:40:00.141 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/170 2024-05-21T21:40:00.215 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/170 2024-05-21T21:40:00.342 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/170 2024-05-21T21:40:00.381 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/170 2024-05-21T21:40:00.409 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/170 2024-05-21T21:40:00.443 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/170 2024-05-21T21:40:00.482 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/170 2024-05-21T21:40:00.515 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/170 2024-05-21T21:40:00.544 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/170 2024-05-21T21:40:00.579 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/170 2024-05-21T21:40:00.588 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mon-2:19.0.0-3830.g3b748ea0.el9.x86_64 115/170 2024-05-21T21:40:00.611 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mon-2:19.0.0-3830.g3b748ea0.el9.x86_64 115/170 2024-05-21T21:40:00.611 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T21:40:00.611 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-21T21:40:00.611 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-21T21:40:00.611 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-21T21:40:00.611 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-21T21:40:00.612 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/170 2024-05-21T21:40:00.658 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/170 2024-05-21T21:40:00.718 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : automake-1.16.2-8.el9.noarch 116/170 2024-05-21T21:40:00.775 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-ply-3.11-14.el9.noarch 117/170 2024-05-21T21:40:00.830 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/170 2024-05-21T21:40:00.830 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 118/170 2024-05-21T21:40:01.014 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 119/170 2024-05-21T21:40:01.019 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/170 2024-05-21T21:40:01.064 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 120/170 2024-05-21T21:40:01.085 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/170 2024-05-21T21:40:01.116 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/170 2024-05-21T21:40:01.151 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/170 2024-05-21T21:40:01.163 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/170 2024-05-21T21:40:01.257 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 122/170 2024-05-21T21:40:01.260 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-packaging-20.9-5.el9.noarch 83/170 2024-05-21T21:40:01.318 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 84/170 2024-05-21T21:40:01.398 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 123/170 2024-05-21T21:40:01.440 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 124/170 2024-05-21T21:40:01.471 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 125/170 2024-05-21T21:40:01.499 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : fuse-2.9.9-15.el9.x86_64 126/170 2024-05-21T21:40:01.943 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: cephadm-2:19.0.0-3830.g3b748ea0.el9.noarch 127/170 2024-05-21T21:40:01.973 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : cephadm-2:19.0.0-3830.g3b748ea0.el9.noarch 127/170 2024-05-21T21:40:02.005 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3830.g3b748ea0.e 128/170 2024-05-21T21:40:02.035 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 85/170 2024-05-21T21:40:02.067 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3830.g3b748ea0. 129/170 2024-05-21T21:40:02.082 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 86/170 2024-05-21T21:40:02.096 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3830.g3b748ea0.el9.noa 130/170 2024-05-21T21:40:02.116 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 87/170 2024-05-21T21:40:02.214 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 88/170 2024-05-21T21:40:02.271 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 89/170 2024-05-21T21:40:02.315 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 90/170 2024-05-21T21:40:02.406 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 91/170 2024-05-21T21:40:02.435 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 92/170 2024-05-21T21:40:02.621 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/170 2024-05-21T21:40:02.649 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/170 2024-05-21T21:40:02.741 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/170 2024-05-21T21:40:03.149 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3830.g3b748ea0.el9.noa 130/170 2024-05-21T21:40:03.181 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3830.g3b748ea0.el9.n 131/170 2024-05-21T21:40:03.376 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/170 2024-05-21T21:40:03.497 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/170 2024-05-21T21:40:04.153 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3830.g3b748ea0.el9.n 131/170 2024-05-21T21:40:04.190 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3830.g3b7 132/170 2024-05-21T21:40:04.302 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3830.g3b7 132/170 2024-05-21T21:40:04.460 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3830.g3b748ea0.el 133/170 2024-05-21T21:40:04.484 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-2:19.0.0-3830.g3b748ea0.el9.x86_64 134/170 2024-05-21T21:40:04.507 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3830.g3b748ea0.el9.x86_64 134/170 2024-05-21T21:40:04.507 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T21:40:04.507 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-21T21:40:04.507 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-21T21:40:04.507 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-21T21:40:04.507 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-21T21:40:04.551 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-rook-2:19.0.0-3830.g3b748ea0.el9.noarch 135/170 2024-05-21T21:40:04.562 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3830.g3b748ea0.el9.noarch 135/170 2024-05-21T21:40:04.698 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-2:19.0.0-3830.g3b748ea0.el9.x86_64 136/170 2024-05-21T21:40:04.720 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/170 2024-05-21T21:40:04.758 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-fuse-2:19.0.0-3830.g3b748ea0.el9.x86_64 137/170 2024-05-21T21:40:04.796 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/170 2024-05-21T21:40:04.823 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 100/170 2024-05-21T21:40:04.857 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 101/170 2024-05-21T21:40:05.169 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 102/170 2024-05-21T21:40:05.220 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : librbd1-2:19.0.0-3830.g3b748ea0.el9.x86_64 103/170 2024-05-21T21:40:05.293 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: librbd1-2:19.0.0-3830.g3b748ea0.el9.x86_64 103/170 2024-05-21T21:40:05.325 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-rbd-2:19.0.0-3830.g3b748ea0.el9.x86_64 104/170 2024-05-21T21:40:05.361 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 105/170 2024-05-21T21:40:05.684 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libtool-2.4.6-45.el9.x86_64 138/170 2024-05-21T21:40:05.712 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-radosgw-2:19.0.0-3830.g3b748ea0.el9.x86_64 139/170 2024-05-21T21:40:05.735 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3830.g3b748ea0.el9.x86_64 139/170 2024-05-21T21:40:05.735 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T21:40:05.735 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-21T21:40:05.735 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-21T21:40:05.735 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-21T21:40:05.735 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-21T21:40:05.780 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3830.g3b748 140/170 2024-05-21T21:40:05.802 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3830.g3b748 140/170 2024-05-21T21:40:05.802 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T21:40:05.802 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-21T21:40:05.802 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-21T21:40:05.841 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 106/170 2024-05-21T21:40:05.847 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : cephfs-mirror-2:19.0.0-3830.g3b748ea0.el9.x86_64 141/170 2024-05-21T21:40:05.869 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3830.g3b748ea0.el9.x86_64 141/170 2024-05-21T21:40:05.869 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T21:40:05.869 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-21T21:40:05.870 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-21T21:40:05.870 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-21T21:40:05.870 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-21T21:40:05.881 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : librgw2-2:19.0.0-3830.g3b748ea0.el9.x86_64 107/170 2024-05-21T21:40:05.953 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: librgw2-2:19.0.0-3830.g3b748ea0.el9.x86_64 107/170 2024-05-21T21:40:05.983 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-rgw-2:19.0.0-3830.g3b748ea0.el9.x86_64 108/170 2024-05-21T21:40:06.181 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : rbd-mirror-2:19.0.0-3830.g3b748ea0.el9.x86_64 142/170 2024-05-21T21:40:06.204 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3830.g3b748ea0.el9.x86_64 142/170 2024-05-21T21:40:06.204 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T21:40:06.204 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-21T21:40:06.204 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-21T21:40:06.204 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-21T21:40:06.204 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-21T21:40:08.355 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-common-2:19.0.0-3830.g3b748ea0.el9.x86_64 109/170 2024-05-21T21:40:08.389 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-common-2:19.0.0-3830.g3b748ea0.el9.x86_64 109/170 2024-05-21T21:40:08.933 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-common-2:19.0.0-3830.g3b748ea0.el9.x86_64 109/170 2024-05-21T21:40:08.968 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-base-2:19.0.0-3830.g3b748ea0.el9.x86_64 110/170 2024-05-21T21:40:09.058 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-base-2:19.0.0-3830.g3b748ea0.el9.x86_64 110/170 2024-05-21T21:40:09.059 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-21T21:40:09.059 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-21T21:40:09.059 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-05-21T21:40:09.086 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-selinux-2:19.0.0-3830.g3b748ea0.el9.x86_64 111/170 2024-05-21T21:40:09.100 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3830.g3b748ea0.el9.x86_64 111/170 2024-05-21T21:40:09.100 INFO:teuthology.orchestra.run.smithi151.stdout:skipping the directory /sys 2024-05-21T21:40:09.100 INFO:teuthology.orchestra.run.smithi151.stdout:skipping the directory /proc 2024-05-21T21:40:09.100 INFO:teuthology.orchestra.run.smithi151.stdout:skipping the directory /mnt 2024-05-21T21:40:09.100 INFO:teuthology.orchestra.run.smithi151.stdout:skipping the directory /var/tmp 2024-05-21T21:40:09.101 INFO:teuthology.orchestra.run.smithi151.stdout:skipping the directory /home 2024-05-21T21:40:09.101 INFO:teuthology.orchestra.run.smithi151.stdout:skipping the directory /root 2024-05-21T21:40:09.101 INFO:teuthology.orchestra.run.smithi151.stdout:skipping the directory /tmp 2024-05-21T21:40:09.101 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T21:40:09.349 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-volume-2:19.0.0-3830.g3b748ea0.el9.noarch 112/170 2024-05-21T21:40:09.397 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-volume-2:19.0.0-3830.g3b748ea0.el9.noarch 112/170 2024-05-21T21:40:09.397 INFO:teuthology.orchestra.run.smithi151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T21:40:09.397 INFO:teuthology.orchestra.run.smithi151.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-21T21:40:09.397 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T21:40:10.801 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-test-2:19.0.0-3830.g3b748ea0.el9.x86_64 143/170 2024-05-21T21:40:10.843 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : rbd-fuse-2:19.0.0-3830.g3b748ea0.el9.x86_64 144/170 2024-05-21T21:40:10.892 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : rbd-nbd-2:19.0.0-3830.g3b748ea0.el9.x86_64 145/170 2024-05-21T21:40:10.932 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 146/170 2024-05-21T21:40:10.960 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 147/170 2024-05-21T21:40:11.025 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-osd-2:19.0.0-3830.g3b748ea0.el9.x86_64 113/170 2024-05-21T21:40:11.110 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-osd-2:19.0.0-3830.g3b748ea0.el9.x86_64 113/170 2024-05-21T21:40:11.110 INFO:teuthology.orchestra.run.smithi151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T21:40:11.110 INFO:teuthology.orchestra.run.smithi151.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-21T21:40:11.110 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-21T21:40:11.110 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-21T21:40:11.110 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T21:40:11.144 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : cephfs-shell-2:19.0.0-3830.g3b748ea0.el9.x86_64 148/170 2024-05-21T21:40:11.189 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 149/170 2024-05-21T21:40:11.230 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libcephfs-devel-2:19.0.0-3830.g3b748ea0.el9.x86_ 150/170 2024-05-21T21:40:11.270 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 151/170 2024-05-21T21:40:11.300 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : NetworkManager-team-1:1.47.90-1.el9.x86_64 152/170 2024-05-21T21:40:11.346 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : NetworkManager-tui-1:1.47.90-1.el9.x86_64 153/170 2024-05-21T21:40:11.359 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-mds-2:19.0.0-3830.g3b748ea0.el9.x86_64 114/170 2024-05-21T21:40:11.437 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-mds-2:19.0.0-3830.g3b748ea0.el9.x86_64 114/170 2024-05-21T21:40:11.437 INFO:teuthology.orchestra.run.smithi151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T21:40:11.437 INFO:teuthology.orchestra.run.smithi151.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-21T21:40:11.437 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-21T21:40:11.437 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-21T21:40:11.438 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T21:40:11.918 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-mon-2:19.0.0-3830.g3b748ea0.el9.x86_64 115/170 2024-05-21T21:40:11.937 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-21T21:40:11.978 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-21T21:40:11.987 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-mon-2:19.0.0-3830.g3b748ea0.el9.x86_64 115/170 2024-05-21T21:40:11.987 INFO:teuthology.orchestra.run.smithi151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T21:40:11.987 INFO:teuthology.orchestra.run.smithi151.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-21T21:40:11.987 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-21T21:40:11.987 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-21T21:40:11.987 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T21:40:11.997 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-21T21:40:12.119 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : automake-1.16.2-8.el9.noarch 116/170 2024-05-21T21:40:12.210 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-ply-3.11-14.el9.noarch 117/170 2024-05-21T21:40:12.290 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 118/170 2024-05-21T21:40:12.529 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 119/170 2024-05-21T21:40:12.589 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 120/170 2024-05-21T21:40:12.675 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/170 2024-05-21T21:40:12.776 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 122/170 2024-05-21T21:40:12.847 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : cephfs-top-2:19.0.0-3830.g3b748ea0.el9.noarch 155/170 2024-05-21T21:40:12.933 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 156/170 2024-05-21T21:40:12.952 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 123/170 2024-05-21T21:40:12.974 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 157/170 2024-05-21T21:40:13.005 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 124/170 2024-05-21T21:40:13.052 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 125/170 2024-05-21T21:40:13.064 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : flex-2.6.4-9.el9.x86_64 158/170 2024-05-21T21:40:13.086 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : fuse-2.9.9-15.el9.x86_64 126/170 2024-05-21T21:40:13.103 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : bison-3.7.4-5.el9.x86_64 159/170 2024-05-21T21:40:13.128 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-21T21:40:13.153 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 161/170 2024-05-21T21:40:13.180 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 162/170 2024-05-21T21:40:13.180 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-21T21:40:13.265 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-21T21:40:13.306 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 164/170 2024-05-21T21:40:13.306 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-21T21:40:13.390 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-21T21:40:13.415 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 166/170 2024-05-21T21:40:13.421 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-21T21:40:13.421 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-21T21:40:13.609 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-21T21:40:13.609 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-21T21:40:13.693 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-21T21:40:13.718 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 169/170 2024-05-21T21:40:13.782 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-21T21:40:13.953 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: cephadm-2:19.0.0-3830.g3b748ea0.el9.noarch 127/170 2024-05-21T21:40:13.992 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : cephadm-2:19.0.0-3830.g3b748ea0.el9.noarch 127/170 2024-05-21T21:40:14.049 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3830.g3b748ea0.e 128/170 2024-05-21T21:40:14.196 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3830.g3b748ea0. 129/170 2024-05-21T21:40:14.244 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3830.g3b748ea0.el9.noa 130/170 2024-05-21T21:40:15.589 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3830.g3b748ea0.el9.noa 130/170 2024-05-21T21:40:15.638 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3830.g3b748ea0.el9.n 131/170 2024-05-21T21:40:16.738 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3830.g3b748ea0.el9.n 131/170 2024-05-21T21:40:16.772 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3830.g3b7 132/170 2024-05-21T21:40:16.929 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3830.g3b7 132/170 2024-05-21T21:40:17.120 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3830.g3b748ea0.el 133/170 2024-05-21T21:40:17.150 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-mgr-2:19.0.0-3830.g3b748ea0.el9.x86_64 134/170 2024-05-21T21:40:17.220 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3830.g3b748ea0.el9.x86_64 134/170 2024-05-21T21:40:17.220 INFO:teuthology.orchestra.run.smithi151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T21:40:17.221 INFO:teuthology.orchestra.run.smithi151.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-21T21:40:17.221 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-21T21:40:17.221 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-21T21:40:17.221 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T21:40:17.292 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-mgr-rook-2:19.0.0-3830.g3b748ea0.el9.noarch 135/170 2024-05-21T21:40:17.329 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3830.g3b748ea0.el9.noarch 135/170 2024-05-21T21:40:17.442 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-2:19.0.0-3830.g3b748ea0.el9.x86_64 136/170 2024-05-21T21:40:17.500 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-fuse-2:19.0.0-3830.g3b748ea0.el9.x86_64 137/170 2024-05-21T21:40:18.501 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libtool-2.4.6-45.el9.x86_64 138/170 2024-05-21T21:40:18.537 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-radosgw-2:19.0.0-3830.g3b748ea0.el9.x86_64 139/170 2024-05-21T21:40:18.615 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3830.g3b748ea0.el9.x86_64 139/170 2024-05-21T21:40:18.615 INFO:teuthology.orchestra.run.smithi151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T21:40:18.615 INFO:teuthology.orchestra.run.smithi151.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-21T21:40:18.615 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-21T21:40:18.615 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-21T21:40:18.616 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T21:40:18.688 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3830.g3b748 140/170 2024-05-21T21:40:18.756 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3830.g3b748 140/170 2024-05-21T21:40:18.757 INFO:teuthology.orchestra.run.smithi151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T21:40:18.757 INFO:teuthology.orchestra.run.smithi151.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-21T21:40:18.757 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T21:40:18.822 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : cephfs-mirror-2:19.0.0-3830.g3b748ea0.el9.x86_64 141/170 2024-05-21T21:40:18.902 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3830.g3b748ea0.el9.x86_64 141/170 2024-05-21T21:40:18.903 INFO:teuthology.orchestra.run.smithi151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T21:40:18.903 INFO:teuthology.orchestra.run.smithi151.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-21T21:40:18.903 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-21T21:40:18.903 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-21T21:40:18.903 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T21:40:19.298 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : rbd-mirror-2:19.0.0-3830.g3b748ea0.el9.x86_64 142/170 2024-05-21T21:40:19.395 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3830.g3b748ea0.el9.x86_64 142/170 2024-05-21T21:40:19.396 INFO:teuthology.orchestra.run.smithi151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T21:40:19.396 INFO:teuthology.orchestra.run.smithi151.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-21T21:40:19.396 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-21T21:40:19.396 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-21T21:40:19.396 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T21:40:19.439 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-21T21:40:19.439 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-2:19.0.0-3830.g3b748ea0.el9.x86_64 1/170 2024-05-21T21:40:19.439 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-base-2:19.0.0-3830.g3b748ea0.el9.x86_64 2/170 2024-05-21T21:40:19.439 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-common-2:19.0.0-3830.g3b748ea0.el9.x86_64 3/170 2024-05-21T21:40:19.439 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-fuse-2:19.0.0-3830.g3b748ea0.el9.x86_64 4/170 2024-05-21T21:40:19.439 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3830.g3b748 5/170 2024-05-21T21:40:19.439 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mds-2:19.0.0-3830.g3b748ea0.el9.x86_64 6/170 2024-05-21T21:40:19.439 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-2:19.0.0-3830.g3b748ea0.el9.x86_64 7/170 2024-05-21T21:40:19.439 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mon-2:19.0.0-3830.g3b748ea0.el9.x86_64 8/170 2024-05-21T21:40:19.439 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-osd-2:19.0.0-3830.g3b748ea0.el9.x86_64 9/170 2024-05-21T21:40:19.440 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-radosgw-2:19.0.0-3830.g3b748ea0.el9.x86_64 10/170 2024-05-21T21:40:19.440 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-selinux-2:19.0.0-3830.g3b748ea0.el9.x86_64 11/170 2024-05-21T21:40:19.440 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-test-2:19.0.0-3830.g3b748ea0.el9.x86_64 12/170 2024-05-21T21:40:19.440 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : cephfs-mirror-2:19.0.0-3830.g3b748ea0.el9.x86_64 13/170 2024-05-21T21:40:19.440 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : cephfs-shell-2:19.0.0-3830.g3b748ea0.el9.x86_64 14/170 2024-05-21T21:40:19.440 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libcephfs-devel-2:19.0.0-3830.g3b748ea0.el9.x86_ 15/170 2024-05-21T21:40:19.440 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libcephfs2-2:19.0.0-3830.g3b748ea0.el9.x86_64 16/170 2024-05-21T21:40:19.440 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libcephsqlite-2:19.0.0-3830.g3b748ea0.el9.x86_64 17/170 2024-05-21T21:40:19.440 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librados-devel-2:19.0.0-3830.g3b748ea0.el9.x86_6 18/170 2024-05-21T21:40:19.440 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libradosstriper1-2:19.0.0-3830.g3b748ea0.el9.x86 19/170 2024-05-21T21:40:19.440 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librgw2-2:19.0.0-3830.g3b748ea0.el9.x86_64 20/170 2024-05-21T21:40:19.440 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3830.g3b748ea0.el 21/170 2024-05-21T21:40:19.440 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-ceph-common-2:19.0.0-3830.g3b748ea0.el9. 22/170 2024-05-21T21:40:19.441 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cephfs-2:19.0.0-3830.g3b748ea0.el9.x86_6 23/170 2024-05-21T21:40:19.441 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-rados-2:19.0.0-3830.g3b748ea0.el9.x86_64 24/170 2024-05-21T21:40:19.441 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-rbd-2:19.0.0-3830.g3b748ea0.el9.x86_64 25/170 2024-05-21T21:40:19.441 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-rgw-2:19.0.0-3830.g3b748ea0.el9.x86_64 26/170 2024-05-21T21:40:19.441 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : rbd-fuse-2:19.0.0-3830.g3b748ea0.el9.x86_64 27/170 2024-05-21T21:40:19.441 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : rbd-mirror-2:19.0.0-3830.g3b748ea0.el9.x86_64 28/170 2024-05-21T21:40:19.441 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : rbd-nbd-2:19.0.0-3830.g3b748ea0.el9.x86_64 29/170 2024-05-21T21:40:19.441 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3830.g3b748ea0. 30/170 2024-05-21T21:40:19.453 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3830.g3b748ea0.el9.noa 31/170 2024-05-21T21:40:19.454 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3830.g3b748ea0.el9.n 32/170 2024-05-21T21:40:19.454 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3830.g3b7 33/170 2024-05-21T21:40:19.454 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3830.g3b748ea0.el 34/170 2024-05-21T21:40:19.454 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3830.g3b748ea0.el9.noarch 35/170 2024-05-21T21:40:19.454 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3830.g3b748ea0.e 36/170 2024-05-21T21:40:19.454 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-volume-2:19.0.0-3830.g3b748ea0.el9.noarch 37/170 2024-05-21T21:40:19.454 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : cephadm-2:19.0.0-3830.g3b748ea0.el9.noarch 38/170 2024-05-21T21:40:19.454 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : cephfs-top-2:19.0.0-3830.g3b748ea0.el9.noarch 39/170 2024-05-21T21:40:19.454 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/170 2024-05-21T21:40:19.454 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/170 2024-05-21T21:40:19.454 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/170 2024-05-21T21:40:19.455 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/170 2024-05-21T21:40:19.455 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/170 2024-05-21T21:40:19.455 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/170 2024-05-21T21:40:19.455 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/170 2024-05-21T21:40:19.455 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/170 2024-05-21T21:40:19.455 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/170 2024-05-21T21:40:19.455 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/170 2024-05-21T21:40:19.455 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/170 2024-05-21T21:40:19.455 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/170 2024-05-21T21:40:19.455 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/170 2024-05-21T21:40:19.455 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/170 2024-05-21T21:40:19.456 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/170 2024-05-21T21:40:19.456 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/170 2024-05-21T21:40:19.456 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/170 2024-05-21T21:40:19.456 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/170 2024-05-21T21:40:19.456 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/170 2024-05-21T21:40:19.456 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/170 2024-05-21T21:40:19.456 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/170 2024-05-21T21:40:19.456 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/170 2024-05-21T21:40:19.456 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/170 2024-05-21T21:40:19.456 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/170 2024-05-21T21:40:19.456 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/170 2024-05-21T21:40:19.456 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/170 2024-05-21T21:40:19.457 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/170 2024-05-21T21:40:19.457 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/170 2024-05-21T21:40:19.457 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/170 2024-05-21T21:40:19.457 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/170 2024-05-21T21:40:19.457 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/170 2024-05-21T21:40:19.457 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/170 2024-05-21T21:40:19.457 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/170 2024-05-21T21:40:19.457 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/170 2024-05-21T21:40:19.457 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/170 2024-05-21T21:40:19.457 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/170 2024-05-21T21:40:19.457 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/170 2024-05-21T21:40:19.457 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/170 2024-05-21T21:40:19.458 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/170 2024-05-21T21:40:19.458 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/170 2024-05-21T21:40:19.458 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/170 2024-05-21T21:40:19.458 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/170 2024-05-21T21:40:19.458 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/170 2024-05-21T21:40:19.458 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/170 2024-05-21T21:40:19.458 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/170 2024-05-21T21:40:19.458 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/170 2024-05-21T21:40:19.458 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/170 2024-05-21T21:40:19.458 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/170 2024-05-21T21:40:19.458 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/170 2024-05-21T21:40:19.458 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/170 2024-05-21T21:40:19.459 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/170 2024-05-21T21:40:19.459 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/170 2024-05-21T21:40:19.459 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/170 2024-05-21T21:40:19.459 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/170 2024-05-21T21:40:19.459 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/170 2024-05-21T21:40:19.459 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/170 2024-05-21T21:40:19.459 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/170 2024-05-21T21:40:19.459 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/170 2024-05-21T21:40:19.459 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/170 2024-05-21T21:40:19.459 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 99/170 2024-05-21T21:40:19.459 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/170 2024-05-21T21:40:19.459 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 101/170 2024-05-21T21:40:19.459 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/170 2024-05-21T21:40:19.460 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/170 2024-05-21T21:40:19.460 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/170 2024-05-21T21:40:19.460 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/170 2024-05-21T21:40:19.460 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/170 2024-05-21T21:40:19.460 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/170 2024-05-21T21:40:19.460 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/170 2024-05-21T21:40:19.460 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/170 2024-05-21T21:40:19.460 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/170 2024-05-21T21:40:19.460 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/170 2024-05-21T21:40:19.460 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/170 2024-05-21T21:40:19.460 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/170 2024-05-21T21:40:19.460 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/170 2024-05-21T21:40:19.460 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/170 2024-05-21T21:40:19.461 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/170 2024-05-21T21:40:19.461 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/170 2024-05-21T21:40:19.461 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/170 2024-05-21T21:40:19.461 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/170 2024-05-21T21:40:19.461 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/170 2024-05-21T21:40:19.461 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/170 2024-05-21T21:40:19.461 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/170 2024-05-21T21:40:19.461 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/170 2024-05-21T21:40:19.461 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/170 2024-05-21T21:40:19.461 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/170 2024-05-21T21:40:19.461 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/170 2024-05-21T21:40:19.461 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/170 2024-05-21T21:40:19.461 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/170 2024-05-21T21:40:19.462 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/170 2024-05-21T21:40:19.462 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/170 2024-05-21T21:40:19.462 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/170 2024-05-21T21:40:19.462 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/170 2024-05-21T21:40:19.462 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/170 2024-05-21T21:40:19.462 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/170 2024-05-21T21:40:19.462 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/170 2024-05-21T21:40:19.462 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/170 2024-05-21T21:40:19.462 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/170 2024-05-21T21:40:19.462 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/170 2024-05-21T21:40:19.462 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/170 2024-05-21T21:40:19.462 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/170 2024-05-21T21:40:19.463 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/170 2024-05-21T21:40:19.463 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/170 2024-05-21T21:40:19.463 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/170 2024-05-21T21:40:19.463 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/170 2024-05-21T21:40:19.463 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/170 2024-05-21T21:40:19.463 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/170 2024-05-21T21:40:19.463 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/170 2024-05-21T21:40:19.463 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/170 2024-05-21T21:40:19.463 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librados2-2:19.0.0-3830.g3b748ea0.el9.x86_64 149/170 2024-05-21T21:40:19.463 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/170 2024-05-21T21:40:19.463 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librbd1-2:19.0.0-3830.g3b748ea0.el9.x86_64 151/170 2024-05-21T21:40:19.463 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/170 2024-05-21T21:40:19.463 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : NetworkManager-1:1.47.90-1.el9.x86_64 153/170 2024-05-21T21:40:19.464 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/170 2024-05-21T21:40:19.464 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 155/170 2024-05-21T21:40:19.464 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/170 2024-05-21T21:40:19.464 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : NetworkManager-team-1:1.47.90-1.el9.x86_64 157/170 2024-05-21T21:40:19.464 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/170 2024-05-21T21:40:19.464 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : NetworkManager-tui-1:1.47.90-1.el9.x86_64 159/170 2024-05-21T21:40:19.464 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-21T21:40:19.464 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/170 2024-05-21T21:40:19.464 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/170 2024-05-21T21:40:19.464 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/170 2024-05-21T21:40:19.464 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/170 2024-05-21T21:40:19.464 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/170 2024-05-21T21:40:19.464 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/170 2024-05-21T21:40:19.465 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 167/170 2024-05-21T21:40:19.465 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/170 2024-05-21T21:40:19.465 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 169/170 2024-05-21T21:40:20.892 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3830.g3b748ea0.el9.x86_64 111/170 2024-05-21T21:40:20.892 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /sys 2024-05-21T21:40:20.892 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /proc 2024-05-21T21:40:20.892 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /mnt 2024-05-21T21:40:20.892 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /var/tmp 2024-05-21T21:40:20.892 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /home 2024-05-21T21:40:20.892 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /root 2024-05-21T21:40:20.893 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /tmp 2024-05-21T21:40:20.893 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-05-21T21:40:21.055 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-volume-2:19.0.0-3830.g3b748ea0.el9.noarch 112/170 2024-05-21T21:40:21.071 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-volume-2:19.0.0-3830.g3b748ea0.el9.noarch 112/170 2024-05-21T21:40:21.071 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T21:40:21.071 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-21T21:40:21.071 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-05-21T21:40:22.326 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/170 2024-05-21T21:40:22.326 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-21T21:40:22.326 INFO:teuthology.orchestra.run.smithi164.stdout:Upgraded: 2024-05-21T21:40:22.326 INFO:teuthology.orchestra.run.smithi164.stdout: NetworkManager-1:1.47.90-1.el9.x86_64 2024-05-21T21:40:22.326 INFO:teuthology.orchestra.run.smithi164.stdout: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 2024-05-21T21:40:22.327 INFO:teuthology.orchestra.run.smithi164.stdout: NetworkManager-team-1:1.47.90-1.el9.x86_64 2024-05-21T21:40:22.327 INFO:teuthology.orchestra.run.smithi164.stdout: NetworkManager-tui-1:1.47.90-1.el9.x86_64 2024-05-21T21:40:22.327 INFO:teuthology.orchestra.run.smithi164.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-05-21T21:40:22.327 INFO:teuthology.orchestra.run.smithi164.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-05-21T21:40:22.327 INFO:teuthology.orchestra.run.smithi164.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-05-21T21:40:22.327 INFO:teuthology.orchestra.run.smithi164.stdout: librados2-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T21:40:22.327 INFO:teuthology.orchestra.run.smithi164.stdout: librbd1-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T21:40:22.327 INFO:teuthology.orchestra.run.smithi164.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-21T21:40:22.327 INFO:teuthology.orchestra.run.smithi164.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-21T21:40:22.327 INFO:teuthology.orchestra.run.smithi164.stdout:Installed: 2024-05-21T21:40:22.327 INFO:teuthology.orchestra.run.smithi164.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-21T21:40:22.327 INFO:teuthology.orchestra.run.smithi164.stdout: automake-1.16.2-8.el9.noarch 2024-05-21T21:40:22.327 INFO:teuthology.orchestra.run.smithi164.stdout: bison-3.7.4-5.el9.x86_64 2024-05-21T21:40:22.328 INFO:teuthology.orchestra.run.smithi164.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-21T21:40:22.328 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T21:40:22.328 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-base-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T21:40:22.328 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-common-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T21:40:22.328 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-fuse-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T21:40:22.328 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-grafana-dashboards-2:19.0.0-3830.g3b748ea0.el9.noarch 2024-05-21T21:40:22.328 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-immutable-object-cache-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T21:40:22.328 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mds-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T21:40:22.328 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T21:40:22.328 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-cephadm-2:19.0.0-3830.g3b748ea0.el9.noarch 2024-05-21T21:40:22.328 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-dashboard-2:19.0.0-3830.g3b748ea0.el9.noarch 2024-05-21T21:40:22.328 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3830.g3b748ea0.el9.noarch 2024-05-21T21:40:22.328 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-modules-core-2:19.0.0-3830.g3b748ea0.el9.noarch 2024-05-21T21:40:22.329 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-rook-2:19.0.0-3830.g3b748ea0.el9.noarch 2024-05-21T21:40:22.329 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mon-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T21:40:22.329 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-osd-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T21:40:22.329 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-prometheus-alerts-2:19.0.0-3830.g3b748ea0.el9.noarch 2024-05-21T21:40:22.329 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-radosgw-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T21:40:22.329 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-selinux-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T21:40:22.329 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-test-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T21:40:22.329 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-volume-2:19.0.0-3830.g3b748ea0.el9.noarch 2024-05-21T21:40:22.329 INFO:teuthology.orchestra.run.smithi164.stdout: cephadm-2:19.0.0-3830.g3b748ea0.el9.noarch 2024-05-21T21:40:22.329 INFO:teuthology.orchestra.run.smithi164.stdout: cephfs-mirror-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T21:40:22.329 INFO:teuthology.orchestra.run.smithi164.stdout: cephfs-shell-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T21:40:22.329 INFO:teuthology.orchestra.run.smithi164.stdout: cephfs-top-2:19.0.0-3830.g3b748ea0.el9.noarch 2024-05-21T21:40:22.329 INFO:teuthology.orchestra.run.smithi164.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-05-21T21:40:22.330 INFO:teuthology.orchestra.run.smithi164.stdout: flex-2.6.4-9.el9.x86_64 2024-05-21T21:40:22.330 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-21T21:40:22.330 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-21T21:40:22.330 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-21T21:40:22.330 INFO:teuthology.orchestra.run.smithi164.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-21T21:40:22.330 INFO:teuthology.orchestra.run.smithi164.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-21T21:40:22.330 INFO:teuthology.orchestra.run.smithi164.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-21T21:40:22.330 INFO:teuthology.orchestra.run.smithi164.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-21T21:40:22.330 INFO:teuthology.orchestra.run.smithi164.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-21T21:40:22.330 INFO:teuthology.orchestra.run.smithi164.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-21T21:40:22.330 INFO:teuthology.orchestra.run.smithi164.stdout: libcephfs-devel-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T21:40:22.330 INFO:teuthology.orchestra.run.smithi164.stdout: libcephfs2-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T21:40:22.330 INFO:teuthology.orchestra.run.smithi164.stdout: libcephsqlite-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T21:40:22.331 INFO:teuthology.orchestra.run.smithi164.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-21T21:40:22.331 INFO:teuthology.orchestra.run.smithi164.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-21T21:40:22.331 INFO:teuthology.orchestra.run.smithi164.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-21T21:40:22.331 INFO:teuthology.orchestra.run.smithi164.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-21T21:40:22.331 INFO:teuthology.orchestra.run.smithi164.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-21T21:40:22.331 INFO:teuthology.orchestra.run.smithi164.stdout: librados-devel-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T21:40:22.331 INFO:teuthology.orchestra.run.smithi164.stdout: libradosstriper1-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T21:40:22.331 INFO:teuthology.orchestra.run.smithi164.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-21T21:40:22.331 INFO:teuthology.orchestra.run.smithi164.stdout: librgw2-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T21:40:22.331 INFO:teuthology.orchestra.run.smithi164.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-21T21:40:22.331 INFO:teuthology.orchestra.run.smithi164.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-21T21:40:22.331 INFO:teuthology.orchestra.run.smithi164.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-21T21:40:22.331 INFO:teuthology.orchestra.run.smithi164.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-21T21:40:22.332 INFO:teuthology.orchestra.run.smithi164.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-21T21:40:22.332 INFO:teuthology.orchestra.run.smithi164.stdout: lua-5.4.4-4.el9.x86_64 2024-05-21T21:40:22.332 INFO:teuthology.orchestra.run.smithi164.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-21T21:40:22.332 INFO:teuthology.orchestra.run.smithi164.stdout: luarocks-3.9.2-5.el9.noarch 2024-05-21T21:40:22.332 INFO:teuthology.orchestra.run.smithi164.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-21T21:40:22.332 INFO:teuthology.orchestra.run.smithi164.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-21T21:40:22.332 INFO:teuthology.orchestra.run.smithi164.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-21T21:40:22.332 INFO:teuthology.orchestra.run.smithi164.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-21T21:40:22.332 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql-13.14-1.el9.x86_64 2024-05-21T21:40:22.332 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-21T21:40:22.332 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-21T21:40:22.332 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-21T21:40:22.332 INFO:teuthology.orchestra.run.smithi164.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-21T21:40:22.333 INFO:teuthology.orchestra.run.smithi164.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-21T21:40:22.333 INFO:teuthology.orchestra.run.smithi164.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-21T21:40:22.333 INFO:teuthology.orchestra.run.smithi164.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-21T21:40:22.333 INFO:teuthology.orchestra.run.smithi164.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-21T21:40:22.333 INFO:teuthology.orchestra.run.smithi164.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-21T21:40:22.333 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-21T21:40:22.333 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ceph-argparse-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T21:40:22.333 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ceph-common-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T21:40:22.333 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cephfs-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T21:40:22.333 INFO:teuthology.orchestra.run.smithi164.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-21T21:40:22.333 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-21T21:40:22.333 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-21T21:40:22.333 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-21T21:40:22.334 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-21T21:40:22.334 INFO:teuthology.orchestra.run.smithi164.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-05-21T21:40:22.334 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-21T21:40:22.334 INFO:teuthology.orchestra.run.smithi164.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-21T21:40:22.334 INFO:teuthology.orchestra.run.smithi164.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-21T21:40:22.335 INFO:teuthology.orchestra.run.smithi164.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-21T21:40:22.335 INFO:teuthology.orchestra.run.smithi164.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-21T21:40:22.335 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-21T21:40:22.335 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-21T21:40:22.335 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-21T21:40:22.335 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-21T21:40:22.335 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-21T21:40:22.335 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-05-21T21:40:22.336 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-21T21:40:22.336 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-21T21:40:22.336 INFO:teuthology.orchestra.run.smithi164.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-21T21:40:22.336 INFO:teuthology.orchestra.run.smithi164.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-21T21:40:22.336 INFO:teuthology.orchestra.run.smithi164.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-21T21:40:22.336 INFO:teuthology.orchestra.run.smithi164.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-21T21:40:22.336 INFO:teuthology.orchestra.run.smithi164.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-21T21:40:22.336 INFO:teuthology.orchestra.run.smithi164.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-21T21:40:22.336 INFO:teuthology.orchestra.run.smithi164.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-21T21:40:22.336 INFO:teuthology.orchestra.run.smithi164.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-21T21:40:22.336 INFO:teuthology.orchestra.run.smithi164.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-21T21:40:22.336 INFO:teuthology.orchestra.run.smithi164.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-21T21:40:22.337 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-21T21:40:22.337 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ply-3.11-14.el9.noarch 2024-05-21T21:40:22.337 INFO:teuthology.orchestra.run.smithi164.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-21T21:40:22.337 INFO:teuthology.orchestra.run.smithi164.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-21T21:40:22.337 INFO:teuthology.orchestra.run.smithi164.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-21T21:40:22.337 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-21T21:40:22.337 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-21T21:40:22.337 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-21T21:40:22.337 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-21T21:40:22.337 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-21T21:40:22.337 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-21T21:40:22.337 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rados-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T21:40:22.337 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rbd-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T21:40:22.337 INFO:teuthology.orchestra.run.smithi164.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-21T21:40:22.338 INFO:teuthology.orchestra.run.smithi164.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-21T21:40:22.338 INFO:teuthology.orchestra.run.smithi164.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-21T21:40:22.338 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rgw-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T21:40:22.338 INFO:teuthology.orchestra.run.smithi164.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-21T21:40:22.338 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-21T21:40:22.338 INFO:teuthology.orchestra.run.smithi164.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-21T21:40:22.338 INFO:teuthology.orchestra.run.smithi164.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-21T21:40:22.338 INFO:teuthology.orchestra.run.smithi164.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-21T21:40:22.338 INFO:teuthology.orchestra.run.smithi164.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-21T21:40:22.338 INFO:teuthology.orchestra.run.smithi164.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-21T21:40:22.338 INFO:teuthology.orchestra.run.smithi164.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-21T21:40:22.338 INFO:teuthology.orchestra.run.smithi164.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-21T21:40:22.338 INFO:teuthology.orchestra.run.smithi164.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-21T21:40:22.339 INFO:teuthology.orchestra.run.smithi164.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-21T21:40:22.339 INFO:teuthology.orchestra.run.smithi164.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-21T21:40:22.339 INFO:teuthology.orchestra.run.smithi164.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-21T21:40:22.339 INFO:teuthology.orchestra.run.smithi164.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-21T21:40:22.339 INFO:teuthology.orchestra.run.smithi164.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-21T21:40:22.339 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-fuse-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T21:40:22.339 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-mirror-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T21:40:22.339 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-nbd-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T21:40:22.339 INFO:teuthology.orchestra.run.smithi164.stdout: re2-1:20211101-3.el9.x86_64 2024-05-21T21:40:22.339 INFO:teuthology.orchestra.run.smithi164.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-21T21:40:22.339 INFO:teuthology.orchestra.run.smithi164.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-21T21:40:22.339 INFO:teuthology.orchestra.run.smithi164.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-21T21:40:22.339 INFO:teuthology.orchestra.run.smithi164.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-21T21:40:22.339 INFO:teuthology.orchestra.run.smithi164.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-21T21:40:22.340 INFO:teuthology.orchestra.run.smithi164.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-21T21:40:22.340 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-05-21T21:40:22.340 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-05-21T21:40:22.556 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-osd-2:19.0.0-3830.g3b748ea0.el9.x86_64 113/170 2024-05-21T21:40:22.583 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-osd-2:19.0.0-3830.g3b748ea0.el9.x86_64 113/170 2024-05-21T21:40:22.583 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T21:40:22.583 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-21T21:40:22.583 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-21T21:40:22.583 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-21T21:40:22.583 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-05-21T21:40:22.797 DEBUG:teuthology.parallel:result is None 2024-05-21T21:40:22.816 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mds-2:19.0.0-3830.g3b748ea0.el9.x86_64 114/170 2024-05-21T21:40:22.839 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mds-2:19.0.0-3830.g3b748ea0.el9.x86_64 114/170 2024-05-21T21:40:22.840 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T21:40:22.840 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-21T21:40:22.840 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-21T21:40:22.840 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-21T21:40:22.840 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-05-21T21:40:23.294 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mon-2:19.0.0-3830.g3b748ea0.el9.x86_64 115/170 2024-05-21T21:40:23.316 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mon-2:19.0.0-3830.g3b748ea0.el9.x86_64 115/170 2024-05-21T21:40:23.317 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T21:40:23.317 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-21T21:40:23.317 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-21T21:40:23.317 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-21T21:40:23.317 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-05-21T21:40:23.414 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : automake-1.16.2-8.el9.noarch 116/170 2024-05-21T21:40:23.470 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-ply-3.11-14.el9.noarch 117/170 2024-05-21T21:40:23.526 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 118/170 2024-05-21T21:40:23.722 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 119/170 2024-05-21T21:40:23.777 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 120/170 2024-05-21T21:40:23.854 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/170 2024-05-21T21:40:23.943 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 122/170 2024-05-21T21:40:24.091 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 123/170 2024-05-21T21:40:24.127 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 124/170 2024-05-21T21:40:24.158 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 125/170 2024-05-21T21:40:24.186 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : fuse-2.9.9-15.el9.x86_64 126/170 2024-05-21T21:40:24.438 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-test-2:19.0.0-3830.g3b748ea0.el9.x86_64 143/170 2024-05-21T21:40:24.490 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : rbd-fuse-2:19.0.0-3830.g3b748ea0.el9.x86_64 144/170 2024-05-21T21:40:24.548 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : rbd-nbd-2:19.0.0-3830.g3b748ea0.el9.x86_64 145/170 2024-05-21T21:40:24.588 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 146/170 2024-05-21T21:40:24.621 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: cephadm-2:19.0.0-3830.g3b748ea0.el9.noarch 127/170 2024-05-21T21:40:24.627 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 147/170 2024-05-21T21:40:24.651 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : cephadm-2:19.0.0-3830.g3b748ea0.el9.noarch 127/170 2024-05-21T21:40:24.683 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3830.g3b748ea0.e 128/170 2024-05-21T21:40:24.786 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3830.g3b748ea0. 129/170 2024-05-21T21:40:24.816 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3830.g3b748ea0.el9.noa 130/170 2024-05-21T21:40:24.950 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : cephfs-shell-2:19.0.0-3830.g3b748ea0.el9.x86_64 148/170 2024-05-21T21:40:25.020 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 149/170 2024-05-21T21:40:25.055 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libcephfs-devel-2:19.0.0-3830.g3b748ea0.el9.x86_ 150/170 2024-05-21T21:40:25.093 INFO:teuthology.orchestra.run.smithi151.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 151/170 2024-05-21T21:40:25.145 INFO:teuthology.orchestra.run.smithi151.stdout: Upgrading : NetworkManager-team-1:1.47.90-1.el9.x86_64 152/170 2024-05-21T21:40:25.195 INFO:teuthology.orchestra.run.smithi151.stdout: Upgrading : NetworkManager-tui-1:1.47.90-1.el9.x86_64 153/170 2024-05-21T21:40:25.854 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3830.g3b748ea0.el9.noa 130/170 2024-05-21T21:40:25.884 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3830.g3b748ea0.el9.n 131/170 2024-05-21T21:40:26.166 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-21T21:40:26.218 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-21T21:40:26.274 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-21T21:40:26.852 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3830.g3b748ea0.el9.n 131/170 2024-05-21T21:40:26.876 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3830.g3b7 132/170 2024-05-21T21:40:26.988 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3830.g3b7 132/170 2024-05-21T21:40:27.153 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3830.g3b748ea0.el 133/170 2024-05-21T21:40:27.178 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-2:19.0.0-3830.g3b748ea0.el9.x86_64 134/170 2024-05-21T21:40:27.201 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3830.g3b748ea0.el9.x86_64 134/170 2024-05-21T21:40:27.201 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T21:40:27.202 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-21T21:40:27.202 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-21T21:40:27.202 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-21T21:40:27.202 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-05-21T21:40:27.245 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-rook-2:19.0.0-3830.g3b748ea0.el9.noarch 135/170 2024-05-21T21:40:27.257 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3830.g3b748ea0.el9.noarch 135/170 2024-05-21T21:40:27.372 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-2:19.0.0-3830.g3b748ea0.el9.x86_64 136/170 2024-05-21T21:40:27.421 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-fuse-2:19.0.0-3830.g3b748ea0.el9.x86_64 137/170 2024-05-21T21:40:27.604 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : cephfs-top-2:19.0.0-3830.g3b748ea0.el9.noarch 155/170 2024-05-21T21:40:27.720 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 156/170 2024-05-21T21:40:27.765 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 157/170 2024-05-21T21:40:27.906 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : flex-2.6.4-9.el9.x86_64 158/170 2024-05-21T21:40:27.947 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : bison-3.7.4-5.el9.x86_64 159/170 2024-05-21T21:40:27.981 INFO:teuthology.orchestra.run.smithi151.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-21T21:40:28.016 INFO:teuthology.orchestra.run.smithi151.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 161/170 2024-05-21T21:40:28.057 INFO:teuthology.orchestra.run.smithi151.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 162/170 2024-05-21T21:40:28.057 INFO:teuthology.orchestra.run.smithi151.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-21T21:40:28.187 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-21T21:40:28.237 INFO:teuthology.orchestra.run.smithi151.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 164/170 2024-05-21T21:40:28.237 INFO:teuthology.orchestra.run.smithi151.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-21T21:40:28.352 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libtool-2.4.6-45.el9.x86_64 138/170 2024-05-21T21:40:28.381 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-radosgw-2:19.0.0-3830.g3b748ea0.el9.x86_64 139/170 2024-05-21T21:40:28.382 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-21T21:40:28.405 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3830.g3b748ea0.el9.x86_64 139/170 2024-05-21T21:40:28.405 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T21:40:28.405 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-21T21:40:28.405 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-21T21:40:28.405 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-21T21:40:28.405 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-05-21T21:40:28.421 INFO:teuthology.orchestra.run.smithi151.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 166/170 2024-05-21T21:40:28.449 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3830.g3b748 140/170 2024-05-21T21:40:28.457 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-21T21:40:28.458 INFO:teuthology.orchestra.run.smithi151.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-21T21:40:28.471 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3830.g3b748 140/170 2024-05-21T21:40:28.471 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T21:40:28.472 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-21T21:40:28.472 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-05-21T21:40:28.508 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : cephfs-mirror-2:19.0.0-3830.g3b748ea0.el9.x86_64 141/170 2024-05-21T21:40:28.531 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3830.g3b748ea0.el9.x86_64 141/170 2024-05-21T21:40:28.531 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T21:40:28.531 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-21T21:40:28.531 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-21T21:40:28.531 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-21T21:40:28.531 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-05-21T21:40:28.759 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-21T21:40:28.760 INFO:teuthology.orchestra.run.smithi151.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-21T21:40:28.842 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : rbd-mirror-2:19.0.0-3830.g3b748ea0.el9.x86_64 142/170 2024-05-21T21:40:28.864 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3830.g3b748ea0.el9.x86_64 142/170 2024-05-21T21:40:28.864 INFO:teuthology.orchestra.run.smithi184.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T21:40:28.864 INFO:teuthology.orchestra.run.smithi184.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-21T21:40:28.864 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-21T21:40:28.864 INFO:teuthology.orchestra.run.smithi184.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-21T21:40:28.865 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-05-21T21:40:28.904 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-21T21:40:28.958 INFO:teuthology.orchestra.run.smithi151.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 169/170 2024-05-21T21:40:29.087 INFO:teuthology.orchestra.run.smithi151.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-21T21:40:29.337 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-21T21:40:29.337 INFO:teuthology.orchestra.run.smithi151.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-05-21T21:40:29.337 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T21:40:33.454 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-test-2:19.0.0-3830.g3b748ea0.el9.x86_64 143/170 2024-05-21T21:40:33.495 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : rbd-fuse-2:19.0.0-3830.g3b748ea0.el9.x86_64 144/170 2024-05-21T21:40:33.536 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : rbd-nbd-2:19.0.0-3830.g3b748ea0.el9.x86_64 145/170 2024-05-21T21:40:33.568 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 146/170 2024-05-21T21:40:33.595 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 147/170 2024-05-21T21:40:33.777 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : cephfs-shell-2:19.0.0-3830.g3b748ea0.el9.x86_64 148/170 2024-05-21T21:40:33.833 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 149/170 2024-05-21T21:40:33.865 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libcephfs-devel-2:19.0.0-3830.g3b748ea0.el9.x86_ 150/170 2024-05-21T21:40:33.914 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 151/170 2024-05-21T21:40:33.944 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : NetworkManager-team-1:1.47.90-1.el9.x86_64 152/170 2024-05-21T21:40:33.990 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : NetworkManager-tui-1:1.47.90-1.el9.x86_64 153/170 2024-05-21T21:40:34.544 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-21T21:40:34.588 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-21T21:40:34.608 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-21T21:40:35.279 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-2:19.0.0-3830.g3b748ea0.el9.x86_64 1/170 2024-05-21T21:40:35.279 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-base-2:19.0.0-3830.g3b748ea0.el9.x86_64 2/170 2024-05-21T21:40:35.279 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-common-2:19.0.0-3830.g3b748ea0.el9.x86_64 3/170 2024-05-21T21:40:35.279 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-fuse-2:19.0.0-3830.g3b748ea0.el9.x86_64 4/170 2024-05-21T21:40:35.279 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3830.g3b748 5/170 2024-05-21T21:40:35.279 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-mds-2:19.0.0-3830.g3b748ea0.el9.x86_64 6/170 2024-05-21T21:40:35.279 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-mgr-2:19.0.0-3830.g3b748ea0.el9.x86_64 7/170 2024-05-21T21:40:35.279 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-mon-2:19.0.0-3830.g3b748ea0.el9.x86_64 8/170 2024-05-21T21:40:35.279 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-osd-2:19.0.0-3830.g3b748ea0.el9.x86_64 9/170 2024-05-21T21:40:35.280 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-radosgw-2:19.0.0-3830.g3b748ea0.el9.x86_64 10/170 2024-05-21T21:40:35.280 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-selinux-2:19.0.0-3830.g3b748ea0.el9.x86_64 11/170 2024-05-21T21:40:35.280 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-test-2:19.0.0-3830.g3b748ea0.el9.x86_64 12/170 2024-05-21T21:40:35.280 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : cephfs-mirror-2:19.0.0-3830.g3b748ea0.el9.x86_64 13/170 2024-05-21T21:40:35.280 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : cephfs-shell-2:19.0.0-3830.g3b748ea0.el9.x86_64 14/170 2024-05-21T21:40:35.280 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libcephfs-devel-2:19.0.0-3830.g3b748ea0.el9.x86_ 15/170 2024-05-21T21:40:35.280 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libcephfs2-2:19.0.0-3830.g3b748ea0.el9.x86_64 16/170 2024-05-21T21:40:35.280 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libcephsqlite-2:19.0.0-3830.g3b748ea0.el9.x86_64 17/170 2024-05-21T21:40:35.280 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : librados-devel-2:19.0.0-3830.g3b748ea0.el9.x86_6 18/170 2024-05-21T21:40:35.280 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libradosstriper1-2:19.0.0-3830.g3b748ea0.el9.x86 19/170 2024-05-21T21:40:35.280 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : librgw2-2:19.0.0-3830.g3b748ea0.el9.x86_64 20/170 2024-05-21T21:40:35.280 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3830.g3b748ea0.el 21/170 2024-05-21T21:40:35.280 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-ceph-common-2:19.0.0-3830.g3b748ea0.el9. 22/170 2024-05-21T21:40:35.281 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-cephfs-2:19.0.0-3830.g3b748ea0.el9.x86_6 23/170 2024-05-21T21:40:35.281 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-rados-2:19.0.0-3830.g3b748ea0.el9.x86_64 24/170 2024-05-21T21:40:35.281 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-rbd-2:19.0.0-3830.g3b748ea0.el9.x86_64 25/170 2024-05-21T21:40:35.281 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-rgw-2:19.0.0-3830.g3b748ea0.el9.x86_64 26/170 2024-05-21T21:40:35.281 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : rbd-fuse-2:19.0.0-3830.g3b748ea0.el9.x86_64 27/170 2024-05-21T21:40:35.281 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : rbd-mirror-2:19.0.0-3830.g3b748ea0.el9.x86_64 28/170 2024-05-21T21:40:35.281 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : rbd-nbd-2:19.0.0-3830.g3b748ea0.el9.x86_64 29/170 2024-05-21T21:40:35.281 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3830.g3b748ea0. 30/170 2024-05-21T21:40:35.281 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3830.g3b748ea0.el9.noa 31/170 2024-05-21T21:40:35.281 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3830.g3b748ea0.el9.n 32/170 2024-05-21T21:40:35.281 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3830.g3b7 33/170 2024-05-21T21:40:35.281 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3830.g3b748ea0.el 34/170 2024-05-21T21:40:35.281 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3830.g3b748ea0.el9.noarch 35/170 2024-05-21T21:40:35.282 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3830.g3b748ea0.e 36/170 2024-05-21T21:40:35.282 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-volume-2:19.0.0-3830.g3b748ea0.el9.noarch 37/170 2024-05-21T21:40:35.282 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : cephadm-2:19.0.0-3830.g3b748ea0.el9.noarch 38/170 2024-05-21T21:40:35.282 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : cephfs-top-2:19.0.0-3830.g3b748ea0.el9.noarch 39/170 2024-05-21T21:40:35.282 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/170 2024-05-21T21:40:35.282 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/170 2024-05-21T21:40:35.282 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/170 2024-05-21T21:40:35.282 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/170 2024-05-21T21:40:35.282 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/170 2024-05-21T21:40:35.282 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/170 2024-05-21T21:40:35.282 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/170 2024-05-21T21:40:35.282 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/170 2024-05-21T21:40:35.283 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/170 2024-05-21T21:40:35.283 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/170 2024-05-21T21:40:35.283 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/170 2024-05-21T21:40:35.283 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/170 2024-05-21T21:40:35.283 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/170 2024-05-21T21:40:35.283 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/170 2024-05-21T21:40:35.283 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/170 2024-05-21T21:40:35.283 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/170 2024-05-21T21:40:35.283 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/170 2024-05-21T21:40:35.283 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/170 2024-05-21T21:40:35.283 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/170 2024-05-21T21:40:35.283 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/170 2024-05-21T21:40:35.283 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/170 2024-05-21T21:40:35.284 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/170 2024-05-21T21:40:35.284 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/170 2024-05-21T21:40:35.284 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/170 2024-05-21T21:40:35.284 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/170 2024-05-21T21:40:35.284 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/170 2024-05-21T21:40:35.284 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/170 2024-05-21T21:40:35.284 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/170 2024-05-21T21:40:35.284 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/170 2024-05-21T21:40:35.284 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/170 2024-05-21T21:40:35.284 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/170 2024-05-21T21:40:35.284 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/170 2024-05-21T21:40:35.284 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/170 2024-05-21T21:40:35.284 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/170 2024-05-21T21:40:35.285 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/170 2024-05-21T21:40:35.285 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/170 2024-05-21T21:40:35.285 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/170 2024-05-21T21:40:35.285 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/170 2024-05-21T21:40:35.285 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/170 2024-05-21T21:40:35.285 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/170 2024-05-21T21:40:35.285 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/170 2024-05-21T21:40:35.285 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/170 2024-05-21T21:40:35.285 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/170 2024-05-21T21:40:35.285 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/170 2024-05-21T21:40:35.285 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/170 2024-05-21T21:40:35.285 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/170 2024-05-21T21:40:35.285 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/170 2024-05-21T21:40:35.286 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/170 2024-05-21T21:40:35.286 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/170 2024-05-21T21:40:35.286 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/170 2024-05-21T21:40:35.286 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/170 2024-05-21T21:40:35.286 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/170 2024-05-21T21:40:35.286 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/170 2024-05-21T21:40:35.286 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/170 2024-05-21T21:40:35.286 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/170 2024-05-21T21:40:35.286 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/170 2024-05-21T21:40:35.286 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/170 2024-05-21T21:40:35.286 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/170 2024-05-21T21:40:35.286 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/170 2024-05-21T21:40:35.287 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 99/170 2024-05-21T21:40:35.287 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/170 2024-05-21T21:40:35.287 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 101/170 2024-05-21T21:40:35.287 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/170 2024-05-21T21:40:35.287 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/170 2024-05-21T21:40:35.287 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/170 2024-05-21T21:40:35.287 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/170 2024-05-21T21:40:35.287 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/170 2024-05-21T21:40:35.287 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/170 2024-05-21T21:40:35.287 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/170 2024-05-21T21:40:35.287 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/170 2024-05-21T21:40:35.287 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/170 2024-05-21T21:40:35.288 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/170 2024-05-21T21:40:35.288 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/170 2024-05-21T21:40:35.288 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/170 2024-05-21T21:40:35.288 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/170 2024-05-21T21:40:35.288 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/170 2024-05-21T21:40:35.288 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/170 2024-05-21T21:40:35.288 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/170 2024-05-21T21:40:35.288 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/170 2024-05-21T21:40:35.288 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/170 2024-05-21T21:40:35.288 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/170 2024-05-21T21:40:35.288 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/170 2024-05-21T21:40:35.288 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/170 2024-05-21T21:40:35.288 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/170 2024-05-21T21:40:35.289 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/170 2024-05-21T21:40:35.289 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/170 2024-05-21T21:40:35.289 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/170 2024-05-21T21:40:35.289 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/170 2024-05-21T21:40:35.289 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/170 2024-05-21T21:40:35.289 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/170 2024-05-21T21:40:35.289 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/170 2024-05-21T21:40:35.289 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/170 2024-05-21T21:40:35.289 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/170 2024-05-21T21:40:35.289 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/170 2024-05-21T21:40:35.289 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/170 2024-05-21T21:40:35.289 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/170 2024-05-21T21:40:35.290 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/170 2024-05-21T21:40:35.290 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/170 2024-05-21T21:40:35.290 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/170 2024-05-21T21:40:35.290 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/170 2024-05-21T21:40:35.290 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/170 2024-05-21T21:40:35.290 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/170 2024-05-21T21:40:35.290 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/170 2024-05-21T21:40:35.290 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/170 2024-05-21T21:40:35.290 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/170 2024-05-21T21:40:35.290 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/170 2024-05-21T21:40:35.290 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/170 2024-05-21T21:40:35.290 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/170 2024-05-21T21:40:35.290 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/170 2024-05-21T21:40:35.291 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : librados2-2:19.0.0-3830.g3b748ea0.el9.x86_64 149/170 2024-05-21T21:40:35.291 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/170 2024-05-21T21:40:35.291 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : librbd1-2:19.0.0-3830.g3b748ea0.el9.x86_64 151/170 2024-05-21T21:40:35.291 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/170 2024-05-21T21:40:35.291 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : NetworkManager-1:1.47.90-1.el9.x86_64 153/170 2024-05-21T21:40:35.291 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/170 2024-05-21T21:40:35.291 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 155/170 2024-05-21T21:40:35.291 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/170 2024-05-21T21:40:35.291 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : NetworkManager-team-1:1.47.90-1.el9.x86_64 157/170 2024-05-21T21:40:35.291 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/170 2024-05-21T21:40:35.291 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : NetworkManager-tui-1:1.47.90-1.el9.x86_64 159/170 2024-05-21T21:40:35.291 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-21T21:40:35.291 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/170 2024-05-21T21:40:35.292 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/170 2024-05-21T21:40:35.292 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/170 2024-05-21T21:40:35.292 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/170 2024-05-21T21:40:35.292 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/170 2024-05-21T21:40:35.292 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/170 2024-05-21T21:40:35.292 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 167/170 2024-05-21T21:40:35.292 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/170 2024-05-21T21:40:35.292 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 169/170 2024-05-21T21:40:35.433 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : cephfs-top-2:19.0.0-3830.g3b748ea0.el9.noarch 155/170 2024-05-21T21:40:35.526 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 156/170 2024-05-21T21:40:35.566 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 157/170 2024-05-21T21:40:35.650 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : flex-2.6.4-9.el9.x86_64 158/170 2024-05-21T21:40:35.687 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : bison-3.7.4-5.el9.x86_64 159/170 2024-05-21T21:40:35.712 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-21T21:40:35.737 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 161/170 2024-05-21T21:40:35.764 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 162/170 2024-05-21T21:40:35.764 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-21T21:40:35.848 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-21T21:40:35.873 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 164/170 2024-05-21T21:40:35.873 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-21T21:40:35.957 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-21T21:40:35.982 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 166/170 2024-05-21T21:40:35.988 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-21T21:40:35.988 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-21T21:40:36.167 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-21T21:40:36.167 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-21T21:40:36.243 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-21T21:40:36.268 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 169/170 2024-05-21T21:40:36.332 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-21T21:40:37.516 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/170 2024-05-21T21:40:37.516 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T21:40:37.516 INFO:teuthology.orchestra.run.smithi151.stdout:Upgraded: 2024-05-21T21:40:37.516 INFO:teuthology.orchestra.run.smithi151.stdout: NetworkManager-1:1.47.90-1.el9.x86_64 2024-05-21T21:40:37.516 INFO:teuthology.orchestra.run.smithi151.stdout: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 2024-05-21T21:40:37.516 INFO:teuthology.orchestra.run.smithi151.stdout: NetworkManager-team-1:1.47.90-1.el9.x86_64 2024-05-21T21:40:37.516 INFO:teuthology.orchestra.run.smithi151.stdout: NetworkManager-tui-1:1.47.90-1.el9.x86_64 2024-05-21T21:40:37.517 INFO:teuthology.orchestra.run.smithi151.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-05-21T21:40:37.517 INFO:teuthology.orchestra.run.smithi151.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-05-21T21:40:37.517 INFO:teuthology.orchestra.run.smithi151.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-05-21T21:40:37.517 INFO:teuthology.orchestra.run.smithi151.stdout: librados2-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T21:40:37.517 INFO:teuthology.orchestra.run.smithi151.stdout: librbd1-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T21:40:37.517 INFO:teuthology.orchestra.run.smithi151.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-21T21:40:37.517 INFO:teuthology.orchestra.run.smithi151.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-21T21:40:37.517 INFO:teuthology.orchestra.run.smithi151.stdout:Installed: 2024-05-21T21:40:37.517 INFO:teuthology.orchestra.run.smithi151.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-21T21:40:37.517 INFO:teuthology.orchestra.run.smithi151.stdout: automake-1.16.2-8.el9.noarch 2024-05-21T21:40:37.518 INFO:teuthology.orchestra.run.smithi151.stdout: bison-3.7.4-5.el9.x86_64 2024-05-21T21:40:37.518 INFO:teuthology.orchestra.run.smithi151.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-21T21:40:37.518 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T21:40:37.518 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-base-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T21:40:37.518 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-common-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T21:40:37.518 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-fuse-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T21:40:37.518 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-grafana-dashboards-2:19.0.0-3830.g3b748ea0.el9.noarch 2024-05-21T21:40:37.518 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-immutable-object-cache-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T21:40:37.518 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mds-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T21:40:37.518 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T21:40:37.519 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-cephadm-2:19.0.0-3830.g3b748ea0.el9.noarch 2024-05-21T21:40:37.519 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-dashboard-2:19.0.0-3830.g3b748ea0.el9.noarch 2024-05-21T21:40:37.519 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3830.g3b748ea0.el9.noarch 2024-05-21T21:40:37.519 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-modules-core-2:19.0.0-3830.g3b748ea0.el9.noarch 2024-05-21T21:40:37.519 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-rook-2:19.0.0-3830.g3b748ea0.el9.noarch 2024-05-21T21:40:37.519 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mon-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T21:40:37.519 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-osd-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T21:40:37.519 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-prometheus-alerts-2:19.0.0-3830.g3b748ea0.el9.noarch 2024-05-21T21:40:37.519 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-radosgw-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T21:40:37.519 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-selinux-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T21:40:37.519 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-test-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T21:40:37.520 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-volume-2:19.0.0-3830.g3b748ea0.el9.noarch 2024-05-21T21:40:37.520 INFO:teuthology.orchestra.run.smithi151.stdout: cephadm-2:19.0.0-3830.g3b748ea0.el9.noarch 2024-05-21T21:40:37.520 INFO:teuthology.orchestra.run.smithi151.stdout: cephfs-mirror-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T21:40:37.520 INFO:teuthology.orchestra.run.smithi151.stdout: cephfs-shell-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T21:40:37.520 INFO:teuthology.orchestra.run.smithi151.stdout: cephfs-top-2:19.0.0-3830.g3b748ea0.el9.noarch 2024-05-21T21:40:37.520 INFO:teuthology.orchestra.run.smithi151.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-05-21T21:40:37.520 INFO:teuthology.orchestra.run.smithi151.stdout: flex-2.6.4-9.el9.x86_64 2024-05-21T21:40:37.520 INFO:teuthology.orchestra.run.smithi151.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-21T21:40:37.520 INFO:teuthology.orchestra.run.smithi151.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-21T21:40:37.520 INFO:teuthology.orchestra.run.smithi151.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-21T21:40:37.520 INFO:teuthology.orchestra.run.smithi151.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-21T21:40:37.521 INFO:teuthology.orchestra.run.smithi151.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-21T21:40:37.521 INFO:teuthology.orchestra.run.smithi151.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-21T21:40:37.521 INFO:teuthology.orchestra.run.smithi151.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-21T21:40:37.521 INFO:teuthology.orchestra.run.smithi151.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-21T21:40:37.521 INFO:teuthology.orchestra.run.smithi151.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-21T21:40:37.521 INFO:teuthology.orchestra.run.smithi151.stdout: libcephfs-devel-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T21:40:37.521 INFO:teuthology.orchestra.run.smithi151.stdout: libcephfs2-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T21:40:37.521 INFO:teuthology.orchestra.run.smithi151.stdout: libcephsqlite-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T21:40:37.521 INFO:teuthology.orchestra.run.smithi151.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-21T21:40:37.521 INFO:teuthology.orchestra.run.smithi151.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-21T21:40:37.522 INFO:teuthology.orchestra.run.smithi151.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-21T21:40:37.522 INFO:teuthology.orchestra.run.smithi151.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-21T21:40:37.522 INFO:teuthology.orchestra.run.smithi151.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-21T21:40:37.522 INFO:teuthology.orchestra.run.smithi151.stdout: librados-devel-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T21:40:37.522 INFO:teuthology.orchestra.run.smithi151.stdout: libradosstriper1-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T21:40:37.522 INFO:teuthology.orchestra.run.smithi151.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-21T21:40:37.522 INFO:teuthology.orchestra.run.smithi151.stdout: librgw2-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T21:40:37.522 INFO:teuthology.orchestra.run.smithi151.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-21T21:40:37.522 INFO:teuthology.orchestra.run.smithi151.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-21T21:40:37.522 INFO:teuthology.orchestra.run.smithi151.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-21T21:40:37.523 INFO:teuthology.orchestra.run.smithi151.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-21T21:40:37.523 INFO:teuthology.orchestra.run.smithi151.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-21T21:40:37.523 INFO:teuthology.orchestra.run.smithi151.stdout: lua-5.4.4-4.el9.x86_64 2024-05-21T21:40:37.523 INFO:teuthology.orchestra.run.smithi151.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-21T21:40:37.523 INFO:teuthology.orchestra.run.smithi151.stdout: luarocks-3.9.2-5.el9.noarch 2024-05-21T21:40:37.523 INFO:teuthology.orchestra.run.smithi151.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-21T21:40:37.523 INFO:teuthology.orchestra.run.smithi151.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-21T21:40:37.523 INFO:teuthology.orchestra.run.smithi151.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-21T21:40:37.523 INFO:teuthology.orchestra.run.smithi151.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-21T21:40:37.523 INFO:teuthology.orchestra.run.smithi151.stdout: postgresql-13.14-1.el9.x86_64 2024-05-21T21:40:37.523 INFO:teuthology.orchestra.run.smithi151.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-21T21:40:37.523 INFO:teuthology.orchestra.run.smithi151.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-21T21:40:37.524 INFO:teuthology.orchestra.run.smithi151.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-21T21:40:37.524 INFO:teuthology.orchestra.run.smithi151.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-21T21:40:37.524 INFO:teuthology.orchestra.run.smithi151.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-21T21:40:37.524 INFO:teuthology.orchestra.run.smithi151.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-21T21:40:37.524 INFO:teuthology.orchestra.run.smithi151.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-21T21:40:37.524 INFO:teuthology.orchestra.run.smithi151.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-21T21:40:37.524 INFO:teuthology.orchestra.run.smithi151.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-21T21:40:37.524 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-21T21:40:37.524 INFO:teuthology.orchestra.run.smithi151.stdout: python3-ceph-argparse-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T21:40:37.525 INFO:teuthology.orchestra.run.smithi151.stdout: python3-ceph-common-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T21:40:37.525 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cephfs-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T21:40:37.525 INFO:teuthology.orchestra.run.smithi151.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-21T21:40:37.525 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-21T21:40:37.525 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-21T21:40:37.525 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-21T21:40:37.525 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-21T21:40:37.525 INFO:teuthology.orchestra.run.smithi151.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-05-21T21:40:37.525 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-21T21:40:37.525 INFO:teuthology.orchestra.run.smithi151.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-21T21:40:37.525 INFO:teuthology.orchestra.run.smithi151.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-21T21:40:37.525 INFO:teuthology.orchestra.run.smithi151.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-21T21:40:37.525 INFO:teuthology.orchestra.run.smithi151.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-21T21:40:37.526 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-21T21:40:37.526 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-21T21:40:37.526 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-21T21:40:37.526 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-21T21:40:37.527 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-21T21:40:37.528 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-05-21T21:40:37.528 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-21T21:40:37.528 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-21T21:40:37.528 INFO:teuthology.orchestra.run.smithi151.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-21T21:40:37.528 INFO:teuthology.orchestra.run.smithi151.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-21T21:40:37.528 INFO:teuthology.orchestra.run.smithi151.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-21T21:40:37.528 INFO:teuthology.orchestra.run.smithi151.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-21T21:40:37.528 INFO:teuthology.orchestra.run.smithi151.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-21T21:40:37.529 INFO:teuthology.orchestra.run.smithi151.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-21T21:40:37.529 INFO:teuthology.orchestra.run.smithi151.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-21T21:40:37.529 INFO:teuthology.orchestra.run.smithi151.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-21T21:40:37.529 INFO:teuthology.orchestra.run.smithi151.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-21T21:40:37.529 INFO:teuthology.orchestra.run.smithi151.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-21T21:40:37.529 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-21T21:40:37.529 INFO:teuthology.orchestra.run.smithi151.stdout: python3-ply-3.11-14.el9.noarch 2024-05-21T21:40:37.529 INFO:teuthology.orchestra.run.smithi151.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-21T21:40:37.529 INFO:teuthology.orchestra.run.smithi151.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-21T21:40:37.530 INFO:teuthology.orchestra.run.smithi151.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-21T21:40:37.530 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-21T21:40:37.530 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-21T21:40:37.530 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-21T21:40:37.530 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-21T21:40:37.530 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-21T21:40:37.530 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-21T21:40:37.530 INFO:teuthology.orchestra.run.smithi151.stdout: python3-rados-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T21:40:37.530 INFO:teuthology.orchestra.run.smithi151.stdout: python3-rbd-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T21:40:37.530 INFO:teuthology.orchestra.run.smithi151.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-21T21:40:37.531 INFO:teuthology.orchestra.run.smithi151.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-21T21:40:37.531 INFO:teuthology.orchestra.run.smithi151.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-21T21:40:37.531 INFO:teuthology.orchestra.run.smithi151.stdout: python3-rgw-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T21:40:37.531 INFO:teuthology.orchestra.run.smithi151.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-21T21:40:37.531 INFO:teuthology.orchestra.run.smithi151.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-21T21:40:37.531 INFO:teuthology.orchestra.run.smithi151.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-21T21:40:37.531 INFO:teuthology.orchestra.run.smithi151.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-21T21:40:37.531 INFO:teuthology.orchestra.run.smithi151.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-21T21:40:37.531 INFO:teuthology.orchestra.run.smithi151.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-21T21:40:37.531 INFO:teuthology.orchestra.run.smithi151.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-21T21:40:37.532 INFO:teuthology.orchestra.run.smithi151.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-21T21:40:37.532 INFO:teuthology.orchestra.run.smithi151.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-21T21:40:37.532 INFO:teuthology.orchestra.run.smithi151.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-21T21:40:37.532 INFO:teuthology.orchestra.run.smithi151.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-21T21:40:37.532 INFO:teuthology.orchestra.run.smithi151.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-21T21:40:37.532 INFO:teuthology.orchestra.run.smithi151.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-21T21:40:37.532 INFO:teuthology.orchestra.run.smithi151.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-21T21:40:37.532 INFO:teuthology.orchestra.run.smithi151.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-21T21:40:37.532 INFO:teuthology.orchestra.run.smithi151.stdout: rbd-fuse-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T21:40:37.532 INFO:teuthology.orchestra.run.smithi151.stdout: rbd-mirror-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T21:40:37.532 INFO:teuthology.orchestra.run.smithi151.stdout: rbd-nbd-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T21:40:37.532 INFO:teuthology.orchestra.run.smithi151.stdout: re2-1:20211101-3.el9.x86_64 2024-05-21T21:40:37.532 INFO:teuthology.orchestra.run.smithi151.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-21T21:40:37.533 INFO:teuthology.orchestra.run.smithi151.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-21T21:40:37.533 INFO:teuthology.orchestra.run.smithi151.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-21T21:40:37.533 INFO:teuthology.orchestra.run.smithi151.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-21T21:40:37.533 INFO:teuthology.orchestra.run.smithi151.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-21T21:40:37.533 INFO:teuthology.orchestra.run.smithi151.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-21T21:40:37.533 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T21:40:37.533 INFO:teuthology.orchestra.run.smithi151.stdout:Complete! 2024-05-21T21:40:38.145 DEBUG:teuthology.parallel:result is None 2024-05-21T21:40:42.011 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-21T21:40:42.012 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-2:19.0.0-3830.g3b748ea0.el9.x86_64 1/170 2024-05-21T21:40:42.012 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-base-2:19.0.0-3830.g3b748ea0.el9.x86_64 2/170 2024-05-21T21:40:42.012 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-common-2:19.0.0-3830.g3b748ea0.el9.x86_64 3/170 2024-05-21T21:40:42.012 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-fuse-2:19.0.0-3830.g3b748ea0.el9.x86_64 4/170 2024-05-21T21:40:42.012 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3830.g3b748 5/170 2024-05-21T21:40:42.012 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mds-2:19.0.0-3830.g3b748ea0.el9.x86_64 6/170 2024-05-21T21:40:42.012 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-2:19.0.0-3830.g3b748ea0.el9.x86_64 7/170 2024-05-21T21:40:42.012 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mon-2:19.0.0-3830.g3b748ea0.el9.x86_64 8/170 2024-05-21T21:40:42.012 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-osd-2:19.0.0-3830.g3b748ea0.el9.x86_64 9/170 2024-05-21T21:40:42.012 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-radosgw-2:19.0.0-3830.g3b748ea0.el9.x86_64 10/170 2024-05-21T21:40:42.013 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-selinux-2:19.0.0-3830.g3b748ea0.el9.x86_64 11/170 2024-05-21T21:40:42.013 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-test-2:19.0.0-3830.g3b748ea0.el9.x86_64 12/170 2024-05-21T21:40:42.013 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : cephfs-mirror-2:19.0.0-3830.g3b748ea0.el9.x86_64 13/170 2024-05-21T21:40:42.013 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : cephfs-shell-2:19.0.0-3830.g3b748ea0.el9.x86_64 14/170 2024-05-21T21:40:42.013 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libcephfs-devel-2:19.0.0-3830.g3b748ea0.el9.x86_ 15/170 2024-05-21T21:40:42.013 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libcephfs2-2:19.0.0-3830.g3b748ea0.el9.x86_64 16/170 2024-05-21T21:40:42.013 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libcephsqlite-2:19.0.0-3830.g3b748ea0.el9.x86_64 17/170 2024-05-21T21:40:42.013 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librados-devel-2:19.0.0-3830.g3b748ea0.el9.x86_6 18/170 2024-05-21T21:40:42.013 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libradosstriper1-2:19.0.0-3830.g3b748ea0.el9.x86 19/170 2024-05-21T21:40:42.013 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librgw2-2:19.0.0-3830.g3b748ea0.el9.x86_64 20/170 2024-05-21T21:40:42.013 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3830.g3b748ea0.el 21/170 2024-05-21T21:40:42.013 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-ceph-common-2:19.0.0-3830.g3b748ea0.el9. 22/170 2024-05-21T21:40:42.013 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cephfs-2:19.0.0-3830.g3b748ea0.el9.x86_6 23/170 2024-05-21T21:40:42.014 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-rados-2:19.0.0-3830.g3b748ea0.el9.x86_64 24/170 2024-05-21T21:40:42.014 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-rbd-2:19.0.0-3830.g3b748ea0.el9.x86_64 25/170 2024-05-21T21:40:42.014 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-rgw-2:19.0.0-3830.g3b748ea0.el9.x86_64 26/170 2024-05-21T21:40:42.014 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : rbd-fuse-2:19.0.0-3830.g3b748ea0.el9.x86_64 27/170 2024-05-21T21:40:42.014 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : rbd-mirror-2:19.0.0-3830.g3b748ea0.el9.x86_64 28/170 2024-05-21T21:40:42.014 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : rbd-nbd-2:19.0.0-3830.g3b748ea0.el9.x86_64 29/170 2024-05-21T21:40:42.014 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3830.g3b748ea0. 30/170 2024-05-21T21:40:42.026 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3830.g3b748ea0.el9.noa 31/170 2024-05-21T21:40:42.026 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3830.g3b748ea0.el9.n 32/170 2024-05-21T21:40:42.026 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3830.g3b7 33/170 2024-05-21T21:40:42.026 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3830.g3b748ea0.el 34/170 2024-05-21T21:40:42.026 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3830.g3b748ea0.el9.noarch 35/170 2024-05-21T21:40:42.026 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3830.g3b748ea0.e 36/170 2024-05-21T21:40:42.027 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-volume-2:19.0.0-3830.g3b748ea0.el9.noarch 37/170 2024-05-21T21:40:42.027 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : cephadm-2:19.0.0-3830.g3b748ea0.el9.noarch 38/170 2024-05-21T21:40:42.027 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : cephfs-top-2:19.0.0-3830.g3b748ea0.el9.noarch 39/170 2024-05-21T21:40:42.027 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/170 2024-05-21T21:40:42.027 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/170 2024-05-21T21:40:42.027 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/170 2024-05-21T21:40:42.027 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/170 2024-05-21T21:40:42.027 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/170 2024-05-21T21:40:42.027 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/170 2024-05-21T21:40:42.027 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/170 2024-05-21T21:40:42.027 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/170 2024-05-21T21:40:42.027 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/170 2024-05-21T21:40:42.028 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/170 2024-05-21T21:40:42.028 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/170 2024-05-21T21:40:42.028 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/170 2024-05-21T21:40:42.028 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/170 2024-05-21T21:40:42.028 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/170 2024-05-21T21:40:42.028 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/170 2024-05-21T21:40:42.028 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/170 2024-05-21T21:40:42.028 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/170 2024-05-21T21:40:42.028 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/170 2024-05-21T21:40:42.028 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/170 2024-05-21T21:40:42.028 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/170 2024-05-21T21:40:42.028 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/170 2024-05-21T21:40:42.029 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/170 2024-05-21T21:40:42.029 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/170 2024-05-21T21:40:42.029 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/170 2024-05-21T21:40:42.029 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/170 2024-05-21T21:40:42.029 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/170 2024-05-21T21:40:42.029 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/170 2024-05-21T21:40:42.029 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/170 2024-05-21T21:40:42.029 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/170 2024-05-21T21:40:42.029 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/170 2024-05-21T21:40:42.029 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/170 2024-05-21T21:40:42.029 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/170 2024-05-21T21:40:42.029 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/170 2024-05-21T21:40:42.029 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/170 2024-05-21T21:40:42.030 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/170 2024-05-21T21:40:42.030 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/170 2024-05-21T21:40:42.030 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/170 2024-05-21T21:40:42.030 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/170 2024-05-21T21:40:42.030 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/170 2024-05-21T21:40:42.030 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/170 2024-05-21T21:40:42.030 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/170 2024-05-21T21:40:42.030 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/170 2024-05-21T21:40:42.030 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/170 2024-05-21T21:40:42.030 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/170 2024-05-21T21:40:42.030 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/170 2024-05-21T21:40:42.030 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/170 2024-05-21T21:40:42.031 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/170 2024-05-21T21:40:42.031 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/170 2024-05-21T21:40:42.031 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/170 2024-05-21T21:40:42.031 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/170 2024-05-21T21:40:42.031 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/170 2024-05-21T21:40:42.031 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/170 2024-05-21T21:40:42.031 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/170 2024-05-21T21:40:42.031 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/170 2024-05-21T21:40:42.031 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/170 2024-05-21T21:40:42.031 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/170 2024-05-21T21:40:42.031 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/170 2024-05-21T21:40:42.031 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/170 2024-05-21T21:40:42.031 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/170 2024-05-21T21:40:42.031 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 99/170 2024-05-21T21:40:42.031 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/170 2024-05-21T21:40:42.032 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 101/170 2024-05-21T21:40:42.032 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/170 2024-05-21T21:40:42.032 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/170 2024-05-21T21:40:42.032 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/170 2024-05-21T21:40:42.032 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/170 2024-05-21T21:40:42.032 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/170 2024-05-21T21:40:42.032 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/170 2024-05-21T21:40:42.032 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/170 2024-05-21T21:40:42.032 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/170 2024-05-21T21:40:42.032 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/170 2024-05-21T21:40:42.032 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/170 2024-05-21T21:40:42.032 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/170 2024-05-21T21:40:42.032 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/170 2024-05-21T21:40:42.033 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/170 2024-05-21T21:40:42.033 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/170 2024-05-21T21:40:42.033 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/170 2024-05-21T21:40:42.033 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/170 2024-05-21T21:40:42.033 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/170 2024-05-21T21:40:42.033 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/170 2024-05-21T21:40:42.033 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/170 2024-05-21T21:40:42.033 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/170 2024-05-21T21:40:42.033 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/170 2024-05-21T21:40:42.033 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/170 2024-05-21T21:40:42.033 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/170 2024-05-21T21:40:42.033 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/170 2024-05-21T21:40:42.033 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/170 2024-05-21T21:40:42.034 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/170 2024-05-21T21:40:42.034 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/170 2024-05-21T21:40:42.034 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/170 2024-05-21T21:40:42.034 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/170 2024-05-21T21:40:42.034 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/170 2024-05-21T21:40:42.034 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/170 2024-05-21T21:40:42.034 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/170 2024-05-21T21:40:42.034 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/170 2024-05-21T21:40:42.034 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/170 2024-05-21T21:40:42.034 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/170 2024-05-21T21:40:42.034 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/170 2024-05-21T21:40:42.034 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/170 2024-05-21T21:40:42.035 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/170 2024-05-21T21:40:42.035 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/170 2024-05-21T21:40:42.035 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/170 2024-05-21T21:40:42.035 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/170 2024-05-21T21:40:42.035 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/170 2024-05-21T21:40:42.035 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/170 2024-05-21T21:40:42.035 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/170 2024-05-21T21:40:42.035 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/170 2024-05-21T21:40:42.035 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/170 2024-05-21T21:40:42.035 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/170 2024-05-21T21:40:42.035 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librados2-2:19.0.0-3830.g3b748ea0.el9.x86_64 149/170 2024-05-21T21:40:42.035 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/170 2024-05-21T21:40:42.036 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librbd1-2:19.0.0-3830.g3b748ea0.el9.x86_64 151/170 2024-05-21T21:40:42.036 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/170 2024-05-21T21:40:42.036 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : NetworkManager-1:1.47.90-1.el9.x86_64 153/170 2024-05-21T21:40:42.036 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/170 2024-05-21T21:40:42.036 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 155/170 2024-05-21T21:40:42.036 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/170 2024-05-21T21:40:42.036 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : NetworkManager-team-1:1.47.90-1.el9.x86_64 157/170 2024-05-21T21:40:42.036 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/170 2024-05-21T21:40:42.036 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : NetworkManager-tui-1:1.47.90-1.el9.x86_64 159/170 2024-05-21T21:40:42.036 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-21T21:40:42.036 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/170 2024-05-21T21:40:42.036 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/170 2024-05-21T21:40:42.036 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/170 2024-05-21T21:40:42.037 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/170 2024-05-21T21:40:42.037 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/170 2024-05-21T21:40:42.037 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/170 2024-05-21T21:40:42.037 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 167/170 2024-05-21T21:40:42.037 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/170 2024-05-21T21:40:42.037 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 169/170 2024-05-21T21:40:44.729 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/170 2024-05-21T21:40:44.729 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-05-21T21:40:44.729 INFO:teuthology.orchestra.run.smithi184.stdout:Upgraded: 2024-05-21T21:40:44.730 INFO:teuthology.orchestra.run.smithi184.stdout: NetworkManager-1:1.47.90-1.el9.x86_64 2024-05-21T21:40:44.730 INFO:teuthology.orchestra.run.smithi184.stdout: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 2024-05-21T21:40:44.730 INFO:teuthology.orchestra.run.smithi184.stdout: NetworkManager-team-1:1.47.90-1.el9.x86_64 2024-05-21T21:40:44.730 INFO:teuthology.orchestra.run.smithi184.stdout: NetworkManager-tui-1:1.47.90-1.el9.x86_64 2024-05-21T21:40:44.730 INFO:teuthology.orchestra.run.smithi184.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-05-21T21:40:44.730 INFO:teuthology.orchestra.run.smithi184.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-05-21T21:40:44.730 INFO:teuthology.orchestra.run.smithi184.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-05-21T21:40:44.730 INFO:teuthology.orchestra.run.smithi184.stdout: librados2-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T21:40:44.730 INFO:teuthology.orchestra.run.smithi184.stdout: librbd1-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T21:40:44.730 INFO:teuthology.orchestra.run.smithi184.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-21T21:40:44.730 INFO:teuthology.orchestra.run.smithi184.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-21T21:40:44.730 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2024-05-21T21:40:44.731 INFO:teuthology.orchestra.run.smithi184.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-21T21:40:44.731 INFO:teuthology.orchestra.run.smithi184.stdout: automake-1.16.2-8.el9.noarch 2024-05-21T21:40:44.731 INFO:teuthology.orchestra.run.smithi184.stdout: bison-3.7.4-5.el9.x86_64 2024-05-21T21:40:44.731 INFO:teuthology.orchestra.run.smithi184.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-21T21:40:44.731 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T21:40:44.731 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-base-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T21:40:44.731 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-common-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T21:40:44.731 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-fuse-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T21:40:44.731 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-grafana-dashboards-2:19.0.0-3830.g3b748ea0.el9.noarch 2024-05-21T21:40:44.731 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-immutable-object-cache-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T21:40:44.731 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mds-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T21:40:44.732 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T21:40:44.732 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-cephadm-2:19.0.0-3830.g3b748ea0.el9.noarch 2024-05-21T21:40:44.732 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-dashboard-2:19.0.0-3830.g3b748ea0.el9.noarch 2024-05-21T21:40:44.732 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3830.g3b748ea0.el9.noarch 2024-05-21T21:40:44.732 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-modules-core-2:19.0.0-3830.g3b748ea0.el9.noarch 2024-05-21T21:40:44.732 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-rook-2:19.0.0-3830.g3b748ea0.el9.noarch 2024-05-21T21:40:44.732 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mon-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T21:40:44.732 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-osd-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T21:40:44.732 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-prometheus-alerts-2:19.0.0-3830.g3b748ea0.el9.noarch 2024-05-21T21:40:44.732 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-radosgw-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T21:40:44.732 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-selinux-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T21:40:44.733 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-test-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T21:40:44.733 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-volume-2:19.0.0-3830.g3b748ea0.el9.noarch 2024-05-21T21:40:44.733 INFO:teuthology.orchestra.run.smithi184.stdout: cephadm-2:19.0.0-3830.g3b748ea0.el9.noarch 2024-05-21T21:40:44.733 INFO:teuthology.orchestra.run.smithi184.stdout: cephfs-mirror-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T21:40:44.733 INFO:teuthology.orchestra.run.smithi184.stdout: cephfs-shell-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T21:40:44.733 INFO:teuthology.orchestra.run.smithi184.stdout: cephfs-top-2:19.0.0-3830.g3b748ea0.el9.noarch 2024-05-21T21:40:44.733 INFO:teuthology.orchestra.run.smithi184.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-05-21T21:40:44.733 INFO:teuthology.orchestra.run.smithi184.stdout: flex-2.6.4-9.el9.x86_64 2024-05-21T21:40:44.733 INFO:teuthology.orchestra.run.smithi184.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-21T21:40:44.733 INFO:teuthology.orchestra.run.smithi184.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-21T21:40:44.733 INFO:teuthology.orchestra.run.smithi184.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-21T21:40:44.733 INFO:teuthology.orchestra.run.smithi184.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-21T21:40:44.734 INFO:teuthology.orchestra.run.smithi184.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-21T21:40:44.734 INFO:teuthology.orchestra.run.smithi184.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-21T21:40:44.734 INFO:teuthology.orchestra.run.smithi184.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-21T21:40:44.734 INFO:teuthology.orchestra.run.smithi184.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-21T21:40:44.734 INFO:teuthology.orchestra.run.smithi184.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-21T21:40:44.734 INFO:teuthology.orchestra.run.smithi184.stdout: libcephfs-devel-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T21:40:44.734 INFO:teuthology.orchestra.run.smithi184.stdout: libcephfs2-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T21:40:44.734 INFO:teuthology.orchestra.run.smithi184.stdout: libcephsqlite-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T21:40:44.734 INFO:teuthology.orchestra.run.smithi184.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-21T21:40:44.734 INFO:teuthology.orchestra.run.smithi184.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-21T21:40:44.734 INFO:teuthology.orchestra.run.smithi184.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-21T21:40:44.735 INFO:teuthology.orchestra.run.smithi184.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-21T21:40:44.735 INFO:teuthology.orchestra.run.smithi184.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-21T21:40:44.735 INFO:teuthology.orchestra.run.smithi184.stdout: librados-devel-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T21:40:44.735 INFO:teuthology.orchestra.run.smithi184.stdout: libradosstriper1-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T21:40:44.735 INFO:teuthology.orchestra.run.smithi184.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-21T21:40:44.735 INFO:teuthology.orchestra.run.smithi184.stdout: librgw2-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T21:40:44.735 INFO:teuthology.orchestra.run.smithi184.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-21T21:40:44.735 INFO:teuthology.orchestra.run.smithi184.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-21T21:40:44.735 INFO:teuthology.orchestra.run.smithi184.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-21T21:40:44.735 INFO:teuthology.orchestra.run.smithi184.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-21T21:40:44.736 INFO:teuthology.orchestra.run.smithi184.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-21T21:40:44.736 INFO:teuthology.orchestra.run.smithi184.stdout: lua-5.4.4-4.el9.x86_64 2024-05-21T21:40:44.736 INFO:teuthology.orchestra.run.smithi184.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-21T21:40:44.736 INFO:teuthology.orchestra.run.smithi184.stdout: luarocks-3.9.2-5.el9.noarch 2024-05-21T21:40:44.736 INFO:teuthology.orchestra.run.smithi184.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-21T21:40:44.736 INFO:teuthology.orchestra.run.smithi184.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-21T21:40:44.736 INFO:teuthology.orchestra.run.smithi184.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-21T21:40:44.736 INFO:teuthology.orchestra.run.smithi184.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-21T21:40:44.736 INFO:teuthology.orchestra.run.smithi184.stdout: postgresql-13.14-1.el9.x86_64 2024-05-21T21:40:44.736 INFO:teuthology.orchestra.run.smithi184.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-21T21:40:44.736 INFO:teuthology.orchestra.run.smithi184.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-21T21:40:44.737 INFO:teuthology.orchestra.run.smithi184.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-21T21:40:44.737 INFO:teuthology.orchestra.run.smithi184.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-21T21:40:44.737 INFO:teuthology.orchestra.run.smithi184.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-21T21:40:44.737 INFO:teuthology.orchestra.run.smithi184.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-21T21:40:44.737 INFO:teuthology.orchestra.run.smithi184.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-21T21:40:44.737 INFO:teuthology.orchestra.run.smithi184.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-21T21:40:44.737 INFO:teuthology.orchestra.run.smithi184.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-21T21:40:44.737 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-21T21:40:44.737 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ceph-argparse-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T21:40:44.737 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ceph-common-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T21:40:44.737 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cephfs-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T21:40:44.737 INFO:teuthology.orchestra.run.smithi184.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-21T21:40:44.738 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-21T21:40:44.738 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-21T21:40:44.738 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-21T21:40:44.738 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-21T21:40:44.738 INFO:teuthology.orchestra.run.smithi184.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-05-21T21:40:44.738 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-21T21:40:44.738 INFO:teuthology.orchestra.run.smithi184.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-21T21:40:44.738 INFO:teuthology.orchestra.run.smithi184.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-21T21:40:44.738 INFO:teuthology.orchestra.run.smithi184.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-21T21:40:44.738 INFO:teuthology.orchestra.run.smithi184.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-21T21:40:44.738 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-21T21:40:44.738 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-21T21:40:44.739 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-21T21:40:44.739 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-21T21:40:44.739 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-21T21:40:44.739 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-05-21T21:40:44.739 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-21T21:40:44.739 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-21T21:40:44.739 INFO:teuthology.orchestra.run.smithi184.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-21T21:40:44.739 INFO:teuthology.orchestra.run.smithi184.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-21T21:40:44.739 INFO:teuthology.orchestra.run.smithi184.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-21T21:40:44.739 INFO:teuthology.orchestra.run.smithi184.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-21T21:40:44.739 INFO:teuthology.orchestra.run.smithi184.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-21T21:40:44.739 INFO:teuthology.orchestra.run.smithi184.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-21T21:40:44.740 INFO:teuthology.orchestra.run.smithi184.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-21T21:40:44.740 INFO:teuthology.orchestra.run.smithi184.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-21T21:40:44.740 INFO:teuthology.orchestra.run.smithi184.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-21T21:40:44.740 INFO:teuthology.orchestra.run.smithi184.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-21T21:40:44.740 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-21T21:40:44.740 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ply-3.11-14.el9.noarch 2024-05-21T21:40:44.740 INFO:teuthology.orchestra.run.smithi184.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-21T21:40:44.740 INFO:teuthology.orchestra.run.smithi184.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-21T21:40:44.740 INFO:teuthology.orchestra.run.smithi184.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-21T21:40:44.740 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-21T21:40:44.740 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-21T21:40:44.740 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-21T21:40:44.741 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-21T21:40:44.741 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-21T21:40:44.741 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-21T21:40:44.741 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rados-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T21:40:44.741 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rbd-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T21:40:44.741 INFO:teuthology.orchestra.run.smithi184.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-21T21:40:44.741 INFO:teuthology.orchestra.run.smithi184.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-21T21:40:44.741 INFO:teuthology.orchestra.run.smithi184.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-21T21:40:44.741 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rgw-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T21:40:44.741 INFO:teuthology.orchestra.run.smithi184.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-21T21:40:44.741 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-21T21:40:44.741 INFO:teuthology.orchestra.run.smithi184.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-21T21:40:44.742 INFO:teuthology.orchestra.run.smithi184.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-21T21:40:44.742 INFO:teuthology.orchestra.run.smithi184.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-21T21:40:44.742 INFO:teuthology.orchestra.run.smithi184.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-21T21:40:44.742 INFO:teuthology.orchestra.run.smithi184.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-21T21:40:44.742 INFO:teuthology.orchestra.run.smithi184.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-21T21:40:44.742 INFO:teuthology.orchestra.run.smithi184.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-21T21:40:44.742 INFO:teuthology.orchestra.run.smithi184.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-21T21:40:44.742 INFO:teuthology.orchestra.run.smithi184.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-21T21:40:44.742 INFO:teuthology.orchestra.run.smithi184.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-21T21:40:44.742 INFO:teuthology.orchestra.run.smithi184.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-21T21:40:44.742 INFO:teuthology.orchestra.run.smithi184.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-21T21:40:44.742 INFO:teuthology.orchestra.run.smithi184.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-21T21:40:44.743 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-fuse-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T21:40:44.743 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-mirror-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T21:40:44.743 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-nbd-2:19.0.0-3830.g3b748ea0.el9.x86_64 2024-05-21T21:40:44.743 INFO:teuthology.orchestra.run.smithi184.stdout: re2-1:20211101-3.el9.x86_64 2024-05-21T21:40:44.743 INFO:teuthology.orchestra.run.smithi184.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-21T21:40:44.743 INFO:teuthology.orchestra.run.smithi184.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-21T21:40:44.743 INFO:teuthology.orchestra.run.smithi184.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-21T21:40:44.743 INFO:teuthology.orchestra.run.smithi184.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-21T21:40:44.743 INFO:teuthology.orchestra.run.smithi184.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-21T21:40:44.744 INFO:teuthology.orchestra.run.smithi184.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-21T21:40:44.744 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-05-21T21:40:44.744 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2024-05-21T21:40:45.182 DEBUG:teuthology.parallel:result is None 2024-05-21T21:40:45.182 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3b748ea08891006c1152513d25ab4a03f7c9a995 2024-05-21T21:40:45.398 DEBUG:teuthology.orchestra.run.smithi151:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-21T21:40:45.471 INFO:teuthology.orchestra.run.smithi151.stdout:19.0.0-3830.g3b748ea0.el9 2024-05-21T21:40:45.471 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3830.g3b748ea0.el9 2024-05-21T21:40:45.471 INFO:teuthology.task.install:The correct ceph version 19.0.0-3830.g3b748ea0 is installed. 2024-05-21T21:40:45.473 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3b748ea08891006c1152513d25ab4a03f7c9a995 2024-05-21T21:40:45.610 DEBUG:teuthology.orchestra.run.smithi164:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-21T21:40:45.639 INFO:teuthology.orchestra.run.smithi164.stdout:19.0.0-3830.g3b748ea0.el9 2024-05-21T21:40:45.639 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3830.g3b748ea0.el9 2024-05-21T21:40:45.639 INFO:teuthology.task.install:The correct ceph version 19.0.0-3830.g3b748ea0 is installed. 2024-05-21T21:40:45.641 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3b748ea08891006c1152513d25ab4a03f7c9a995 2024-05-21T21:40:45.775 DEBUG:teuthology.orchestra.run.smithi184:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-21T21:40:45.798 INFO:teuthology.orchestra.run.smithi184.stdout:19.0.0-3830.g3b748ea0.el9 2024-05-21T21:40:45.799 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3830.g3b748ea0.el9 2024-05-21T21:40:45.799 INFO:teuthology.task.install:The correct ceph version 19.0.0-3830.g3b748ea0 is installed. 2024-05-21T21:40:45.801 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-21T21:40:45.801 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-05-21T21:40:45.801 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-21T21:40:45.888 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-21T21:40:45.888 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-21T21:40:45.920 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-05-21T21:40:45.920 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-21T21:40:45.949 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-21T21:40:45.949 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-05-21T21:40:45.949 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/usr/bin/daemon-helper 2024-05-21T21:40:46.037 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-21T21:40:46.115 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-21T21:40:46.115 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/usr/bin/daemon-helper 2024-05-21T21:40:46.143 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-21T21:40:46.211 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-05-21T21:40:46.211 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/usr/bin/daemon-helper 2024-05-21T21:40:46.239 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-21T21:40:46.306 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-21T21:40:46.306 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-05-21T21:40:46.306 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-21T21:40:46.395 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-21T21:40:46.527 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-21T21:40:46.527 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-21T21:40:46.552 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-21T21:40:46.615 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-05-21T21:40:46.615 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-21T21:40:46.643 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-21T21:40:46.713 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-21T21:40:46.713 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-05-21T21:40:46.713 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/usr/bin/stdin-killer 2024-05-21T21:40:46.802 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-21T21:40:46.884 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-21T21:40:46.884 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/usr/bin/stdin-killer 2024-05-21T21:40:46.909 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-21T21:40:46.973 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-05-21T21:40:46.973 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/usr/bin/stdin-killer 2024-05-21T21:40:47.001 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-21T21:40:47.068 INFO:teuthology.run_tasks:Running task cephadm... 2024-05-21T21:40:47.165 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '3b748ea08891006c1152513d25ab4a03f7c9a995', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-05-21T21:40:47.165 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 2024-05-21T21:40:47.165 INFO:tasks.cephadm:Cluster fsid is c8c7a2e6-17ba-11ef-bc9a-c7b262605968 2024-05-21T21:40:47.166 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-05-21T21:40:47.166 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.151', 'mon.b': '172.21.15.164', 'mon.c': '172.21.15.184'} 2024-05-21T21:40:47.166 INFO:tasks.cephadm:First mon is mon.a on smithi151 2024-05-21T21:40:47.166 INFO:tasks.cephadm:First mgr is x 2024-05-21T21:40:47.166 INFO:tasks.cephadm:Normalizing hostnames... 2024-05-21T21:40:47.166 DEBUG:teuthology.orchestra.run.smithi151:> sudo hostname $(hostname -s) 2024-05-21T21:40:47.257 DEBUG:teuthology.orchestra.run.smithi164:> sudo hostname $(hostname -s) 2024-05-21T21:40:47.281 DEBUG:teuthology.orchestra.run.smithi184:> sudo hostname $(hostname -s) 2024-05-21T21:40:47.309 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-05-21T21:40:47.309 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3b748ea08891006c1152513d25ab4a03f7c9a995 2024-05-21T21:40:47.445 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '3b748ea08891006c1152513d25ab4a03f7c9a995', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/79571/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3830-g3b748ea0', 'node_name': '172.21.5.35+adami05', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3830.g3b748ea0'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce-overdrive-export/3b748ea08891006c1152513d25ab4a03f7c9a995/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-05-20 22:39:51.349239', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-lusov-quiesce-overdrive-export', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-lusov-quiesce-overdrive-export/3b748ea08891006c1152513d25ab4a03f7c9a995/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-05-21T21:40:47.659 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref wip-lusov-quiesce-overdrive-export, sha1 3b748ea08891006c1152513d25ab4a03f7c9a995 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=3b748ea08891006c1152513d25ab4a03f7c9a995 2024-05-21T21:40:47.660 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/wip-lusov-quiesce-overdrive-export/3b748ea08891006c1152513d25ab4a03f7c9a995/centos/9/x86_64/flavors/default/cephadm 2024-05-21T21:40:47.660 INFO:tasks.cephadm:Downloading cephadm from url: https://3.chacra.ceph.com/binaries/ceph/wip-lusov-quiesce-overdrive-export/3b748ea08891006c1152513d25ab4a03f7c9a995/centos/9/x86_64/flavors/default/cephadm 2024-05-21T21:40:47.661 DEBUG:teuthology.orchestra.run.smithi151:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-lusov-quiesce-overdrive-export/3b748ea08891006c1152513d25ab4a03f7c9a995/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-21T21:40:48.357 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r--. 1 ubuntu ubuntu 782836 May 21 21:40 /home/ubuntu/cephtest/cephadm 2024-05-21T21:40:48.357 DEBUG:teuthology.orchestra.run.smithi164:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-lusov-quiesce-overdrive-export/3b748ea08891006c1152513d25ab4a03f7c9a995/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-21T21:40:49.214 INFO:teuthology.orchestra.run.smithi164.stdout:-rw-r--r--. 1 ubuntu ubuntu 782836 May 21 21:40 /home/ubuntu/cephtest/cephadm 2024-05-21T21:40:49.214 DEBUG:teuthology.orchestra.run.smithi184:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-lusov-quiesce-overdrive-export/3b748ea08891006c1152513d25ab4a03f7c9a995/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-21T21:40:50.073 INFO:teuthology.orchestra.run.smithi184.stdout:-rw-r--r--. 1 ubuntu ubuntu 782836 May 21 21:40 /home/ubuntu/cephtest/cephadm 2024-05-21T21:40:50.074 DEBUG:teuthology.orchestra.run.smithi151:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-21T21:40:50.134 DEBUG:teuthology.orchestra.run.smithi164:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-21T21:40:50.153 DEBUG:teuthology.orchestra.run.smithi184:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-21T21:40:50.186 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 on all hosts... 2024-05-21T21:40:50.186 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 pull 2024-05-21T21:40:50.192 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 pull 2024-05-21T21:40:50.196 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 pull 2024-05-21T21:40:51.007 INFO:teuthology.orchestra.run.smithi164.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995... 2024-05-21T21:40:51.136 INFO:teuthology.orchestra.run.smithi184.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995... 2024-05-21T21:40:51.765 INFO:teuthology.orchestra.run.smithi151.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995... 2024-05-21T21:41:32.618 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-05-21T21:41:32.619 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_version": "ceph version 19.0.0-3830-g3b748ea0 (3b748ea08891006c1152513d25ab4a03f7c9a995) squid (dev)", 2024-05-21T21:41:32.619 INFO:teuthology.orchestra.run.smithi164.stdout: "image_id": "d6bc8f31cbc1a7165ca248fe9ba0b06ea14b30f46e1ea33ced1a5238e247af1f", 2024-05-21T21:41:32.619 INFO:teuthology.orchestra.run.smithi164.stdout: "repo_digests": [ 2024-05-21T21:41:32.619 INFO:teuthology.orchestra.run.smithi164.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28" 2024-05-21T21:41:32.619 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-05-21T21:41:32.619 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-05-21T21:41:35.270 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-05-21T21:41:35.270 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_version": "ceph version 19.0.0-3830-g3b748ea0 (3b748ea08891006c1152513d25ab4a03f7c9a995) squid (dev)", 2024-05-21T21:41:35.270 INFO:teuthology.orchestra.run.smithi184.stdout: "image_id": "d6bc8f31cbc1a7165ca248fe9ba0b06ea14b30f46e1ea33ced1a5238e247af1f", 2024-05-21T21:41:35.270 INFO:teuthology.orchestra.run.smithi184.stdout: "repo_digests": [ 2024-05-21T21:41:35.271 INFO:teuthology.orchestra.run.smithi184.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28" 2024-05-21T21:41:35.271 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2024-05-21T21:41:35.271 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-05-21T21:41:37.802 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-05-21T21:41:37.802 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_version": "ceph version 19.0.0-3830-g3b748ea0 (3b748ea08891006c1152513d25ab4a03f7c9a995) squid (dev)", 2024-05-21T21:41:37.802 INFO:teuthology.orchestra.run.smithi151.stdout: "image_id": "d6bc8f31cbc1a7165ca248fe9ba0b06ea14b30f46e1ea33ced1a5238e247af1f", 2024-05-21T21:41:37.802 INFO:teuthology.orchestra.run.smithi151.stdout: "repo_digests": [ 2024-05-21T21:41:37.803 INFO:teuthology.orchestra.run.smithi151.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28" 2024-05-21T21:41:37.803 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-05-21T21:41:37.803 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-05-21T21:41:37.847 DEBUG:teuthology.orchestra.run.smithi151:> sudo mkdir -p /etc/ceph 2024-05-21T21:41:37.925 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /etc/ceph 2024-05-21T21:41:37.956 DEBUG:teuthology.orchestra.run.smithi184:> sudo mkdir -p /etc/ceph 2024-05-21T21:41:37.988 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod 777 /etc/ceph 2024-05-21T21:41:38.073 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod 777 /etc/ceph 2024-05-21T21:41:38.101 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod 777 /etc/ceph 2024-05-21T21:41:38.129 INFO:tasks.cephadm:Writing seed config... 2024-05-21T21:41:38.130 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-05-21T21:41:38.130 INFO:tasks.cephadm: override: [client] debug client = 20 2024-05-21T21:41:38.131 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-05-21T21:41:38.131 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-05-21T21:41:38.131 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-05-21T21:41:38.131 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-05-21T21:41:38.131 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-05-21T21:41:38.131 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-05-21T21:41:38.131 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-05-21T21:41:38.131 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-05-21T21:41:38.131 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-05-21T21:41:38.131 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-05-21T21:41:38.131 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-05-21T21:41:38.131 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-05-21T21:41:38.131 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-05-21T21:41:38.132 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-05-21T21:41:38.132 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-05-21T21:41:38.132 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-05-21T21:41:38.132 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-05-21T21:41:38.132 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-05-21T21:41:38.132 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-05-21T21:41:38.132 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-05-21T21:41:38.132 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-05-21T21:41:38.132 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-05-21T21:41:38.132 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-05-21T21:41:38.132 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-05-21T21:41:38.132 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-05-21T21:41:38.132 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-05-21T21:41:38.132 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-05-21T21:41:38.132 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-05-21T21:41:38.133 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-05-21T21:41:38.133 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-05-21T21:41:38.133 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-05-21T21:41:38.133 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-05-21T21:41:38.133 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-05-21T21:41:38.133 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-05-21T21:41:38.133 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-05-21T21:41:38.133 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-05-21T21:41:38.133 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-05-21T21:41:38.133 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-05-21T21:41:38.133 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-05-21T21:41:38.133 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-05-21T21:41:38.133 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-05-21T21:41:38.133 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-05-21T21:41:38.134 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-05-21T21:41:38.134 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-05-21T21:41:38.134 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-05-21T21:41:38.134 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-05-21T21:41:38.134 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-05-21T21:41:38.135 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-05-21T21:41:38.135 DEBUG:teuthology.orchestra.run.smithi151:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-05-21T21:41:38.191 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = c8c7a2e6-17ba-11ef-bc9a-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-05-21T21:41:38.192 DEBUG:teuthology.orchestra.run.smithi151:mon.a> sudo journalctl -f -n 0 -u ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@mon.a.service 2024-05-21T21:41:38.238 DEBUG:teuthology.orchestra.run.smithi151:mgr.x> sudo journalctl -f -n 0 -u ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@mgr.x.service 2024-05-21T21:41:38.284 INFO:tasks.cephadm:Bootstrapping... 2024-05-21T21:41:38.284 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 -v bootstrap --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.151 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-05-21T21:41:38.551 INFO:teuthology.orchestra.run.smithi151.stdout:-------------------------------------------------------------------------------- 2024-05-21T21:41:38.551 INFO:teuthology.orchestra.run.smithi151.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995', '-v', 'bootstrap', '--fsid', 'c8c7a2e6-17ba-11ef-bc9a-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.151', '--skip-admin-label'] 2024-05-21T21:41:38.552 INFO:teuthology.orchestra.run.smithi151.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-05-21T21:41:38.552 INFO:teuthology.orchestra.run.smithi151.stdout:Verifying podman|docker is present... 2024-05-21T21:41:38.612 INFO:teuthology.orchestra.run.smithi151.stdout:/bin/podman: stdout 5.0.2 2024-05-21T21:41:38.612 INFO:teuthology.orchestra.run.smithi151.stdout:Verifying lvm2 is present... 2024-05-21T21:41:38.612 INFO:teuthology.orchestra.run.smithi151.stdout:Verifying time synchronization is in place... 2024-05-21T21:41:38.632 INFO:teuthology.orchestra.run.smithi151.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-21T21:41:38.632 INFO:teuthology.orchestra.run.smithi151.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-21T21:41:38.650 INFO:teuthology.orchestra.run.smithi151.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-21T21:41:38.650 INFO:teuthology.orchestra.run.smithi151.stdout:systemctl: stdout inactive 2024-05-21T21:41:38.668 INFO:teuthology.orchestra.run.smithi151.stdout:systemctl: stdout enabled 2024-05-21T21:41:38.686 INFO:teuthology.orchestra.run.smithi151.stdout:systemctl: stdout active 2024-05-21T21:41:38.687 INFO:teuthology.orchestra.run.smithi151.stdout:Unit chronyd.service is enabled and running 2024-05-21T21:41:38.687 INFO:teuthology.orchestra.run.smithi151.stdout:Repeating the final host check... 2024-05-21T21:41:38.739 INFO:teuthology.orchestra.run.smithi151.stdout:/bin/podman: stdout 5.0.2 2024-05-21T21:41:38.739 INFO:teuthology.orchestra.run.smithi151.stdout:podman (/bin/podman) version 5.0.2 is present 2024-05-21T21:41:38.739 INFO:teuthology.orchestra.run.smithi151.stdout:systemctl is present 2024-05-21T21:41:38.739 INFO:teuthology.orchestra.run.smithi151.stdout:lvcreate is present 2024-05-21T21:41:38.753 INFO:teuthology.orchestra.run.smithi151.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-21T21:41:38.753 INFO:teuthology.orchestra.run.smithi151.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-21T21:41:38.765 INFO:teuthology.orchestra.run.smithi151.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-21T21:41:38.766 INFO:teuthology.orchestra.run.smithi151.stdout:systemctl: stdout inactive 2024-05-21T21:41:38.780 INFO:teuthology.orchestra.run.smithi151.stdout:systemctl: stdout enabled 2024-05-21T21:41:38.793 INFO:teuthology.orchestra.run.smithi151.stdout:systemctl: stdout active 2024-05-21T21:41:38.793 INFO:teuthology.orchestra.run.smithi151.stdout:Unit chronyd.service is enabled and running 2024-05-21T21:41:38.793 INFO:teuthology.orchestra.run.smithi151.stdout:Host looks OK 2024-05-21T21:41:38.793 INFO:teuthology.orchestra.run.smithi151.stdout:Cluster fsid: c8c7a2e6-17ba-11ef-bc9a-c7b262605968 2024-05-21T21:41:38.793 INFO:teuthology.orchestra.run.smithi151.stdout:Acquiring lock 140539267248912 on /run/cephadm/c8c7a2e6-17ba-11ef-bc9a-c7b262605968.lock 2024-05-21T21:41:38.794 INFO:teuthology.orchestra.run.smithi151.stdout:Lock 140539267248912 acquired on /run/cephadm/c8c7a2e6-17ba-11ef-bc9a-c7b262605968.lock 2024-05-21T21:41:38.794 INFO:teuthology.orchestra.run.smithi151.stdout:Verifying IP 172.21.15.151 port 3300 ... 2024-05-21T21:41:38.795 INFO:teuthology.orchestra.run.smithi151.stdout:Verifying IP 172.21.15.151 port 6789 ... 2024-05-21T21:41:38.795 INFO:teuthology.orchestra.run.smithi151.stdout:Base mon IP(s) is [172.21.15.151:3300, 172.21.15.151:6789], mon addrv is [v2:172.21.15.151:3300,v1:172.21.15.151:6789] 2024-05-21T21:41:38.802 INFO:teuthology.orchestra.run.smithi151.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.151 metric 100 2024-05-21T21:41:38.802 INFO:teuthology.orchestra.run.smithi151.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.151 metric 100 2024-05-21T21:41:38.808 INFO:teuthology.orchestra.run.smithi151.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-05-21T21:41:38.808 INFO:teuthology.orchestra.run.smithi151.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-05-21T21:41:38.808 INFO:teuthology.orchestra.run.smithi151.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1792sec hoplimit 64 pref medium 2024-05-21T21:41:38.814 INFO:teuthology.orchestra.run.smithi151.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-05-21T21:41:38.815 INFO:teuthology.orchestra.run.smithi151.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-05-21T21:41:38.815 INFO:teuthology.orchestra.run.smithi151.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-21T21:41:38.815 INFO:teuthology.orchestra.run.smithi151.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-05-21T21:41:38.815 INFO:teuthology.orchestra.run.smithi151.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe88:6f63/64 scope link 2024-05-21T21:41:38.815 INFO:teuthology.orchestra.run.smithi151.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-21T21:41:38.815 INFO:teuthology.orchestra.run.smithi151.stdout:Mon IP `172.21.15.151` is in CIDR network `172.21.0.0/20` 2024-05-21T21:41:38.816 INFO:teuthology.orchestra.run.smithi151.stdout:Mon IP `172.21.15.151` is in CIDR network `172.21.0.0/20` 2024-05-21T21:41:38.816 INFO:teuthology.orchestra.run.smithi151.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-05-21T21:41:38.816 INFO:teuthology.orchestra.run.smithi151.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-05-21T21:41:38.818 INFO:teuthology.orchestra.run.smithi151.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995... 2024-05-21T21:41:42.353 INFO:teuthology.orchestra.run.smithi151.stdout:/bin/podman: stdout d6bc8f31cbc1a7165ca248fe9ba0b06ea14b30f46e1ea33ced1a5238e247af1f 2024-05-21T21:41:42.354 INFO:teuthology.orchestra.run.smithi151.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995... 2024-05-21T21:41:42.354 INFO:teuthology.orchestra.run.smithi151.stdout:/bin/podman: stderr Getting image source signatures 2024-05-21T21:41:42.354 INFO:teuthology.orchestra.run.smithi151.stdout:/bin/podman: stderr Copying blob sha256:a5d06693b40edf6d7f15546e9c6c501dcf23f4c800cbc84dcc01d75832a05bb1 2024-05-21T21:41:42.354 INFO:teuthology.orchestra.run.smithi151.stdout:/bin/podman: stderr Copying blob sha256:d0e808cd455827ca3108db1b0125dcee9ffc148daa1d7dde963f550bc831d501 2024-05-21T21:41:42.354 INFO:teuthology.orchestra.run.smithi151.stdout:/bin/podman: stderr Copying config sha256:d6bc8f31cbc1a7165ca248fe9ba0b06ea14b30f46e1ea33ced1a5238e247af1f 2024-05-21T21:41:42.354 INFO:teuthology.orchestra.run.smithi151.stdout:/bin/podman: stderr Writing manifest to image destination 2024-05-21T21:41:43.518 INFO:teuthology.orchestra.run.smithi151.stdout:ceph: stdout ceph version 19.0.0-3830-g3b748ea0 (3b748ea08891006c1152513d25ab4a03f7c9a995) squid (dev) 2024-05-21T21:41:43.518 INFO:teuthology.orchestra.run.smithi151.stdout:Ceph version: ceph version 19.0.0-3830-g3b748ea0 (3b748ea08891006c1152513d25ab4a03f7c9a995) squid (dev) 2024-05-21T21:41:43.518 INFO:teuthology.orchestra.run.smithi151.stdout:Extracting ceph user uid/gid from container image... 2024-05-21T21:41:44.667 INFO:teuthology.orchestra.run.smithi151.stdout:stat: stdout 167 167 2024-05-21T21:41:44.667 INFO:teuthology.orchestra.run.smithi151.stdout:Creating initial keys... 2024-05-21T21:41:45.697 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph-authtool: stdout AQAZFU1mpza3BRAAPy7timcQXFkoSxKj3akM8g== 2024-05-21T21:41:46.739 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph-authtool: stdout AQAaFU1m7PXeBxAAwGqJr5i8YUyFCAGgEvMt0g== 2024-05-21T21:41:47.848 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph-authtool: stdout AQAbFU1mPexZCxAAWqEBi2/JH1/aWsWLFHfp4g== 2024-05-21T21:41:47.849 INFO:teuthology.orchestra.run.smithi151.stdout:Creating initial monmap... 2024-05-21T21:41:48.927 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-21T21:41:48.927 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-05-21T21:41:48.928 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to c8c7a2e6-17ba-11ef-bc9a-c7b262605968 2024-05-21T21:41:48.928 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-21T21:41:48.928 INFO:teuthology.orchestra.run.smithi151.stdout:monmaptool for a [v2:172.21.15.151:3300,v1:172.21.15.151:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-21T21:41:48.928 INFO:teuthology.orchestra.run.smithi151.stdout:setting min_mon_release = quincy 2024-05-21T21:41:48.928 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/monmaptool: set fsid to c8c7a2e6-17ba-11ef-bc9a-c7b262605968 2024-05-21T21:41:48.928 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-21T21:41:48.928 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T21:41:48.928 INFO:teuthology.orchestra.run.smithi151.stdout:Creating mon... 2024-05-21T21:41:50.049 INFO:teuthology.orchestra.run.smithi151.stdout:create mon.a on 2024-05-21T21:41:50.390 INFO:teuthology.orchestra.run.smithi151.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-05-21T21:41:50.624 INFO:teuthology.orchestra.run.smithi151.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-05-21T21:41:50.908 INFO:teuthology.orchestra.run.smithi151.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968.target → /etc/systemd/system/ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968.target. 2024-05-21T21:41:50.908 INFO:teuthology.orchestra.run.smithi151.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968.target → /etc/systemd/system/ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968.target. 2024-05-21T21:41:51.382 INFO:teuthology.orchestra.run.smithi151.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@mon.a 2024-05-21T21:41:51.383 INFO:teuthology.orchestra.run.smithi151.stdout:systemctl: stderr Failed to reset failed state of unit ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@mon.a.service: Unit ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@mon.a.service not loaded. 2024-05-21T21:41:51.683 INFO:teuthology.orchestra.run.smithi151.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968.target.wants/ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@mon.a.service → /etc/systemd/system/ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@.service. 2024-05-21T21:41:52.337 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:41:52 smithi151 podman[31227]: 2024-05-21 21:41:52.324817709 +0000 UTC m=+0.363384399 container init b7a96bd1085a11cc0a5a20a24818bc3b69eff10adc38df66b31dd9884cc662dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-21T21:41:52.337 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:41:52 smithi151 podman[31227]: 2024-05-21 21:41:52.335968536 +0000 UTC m=+0.374535230 container start b7a96bd1085a11cc0a5a20a24818bc3b69eff10adc38df66b31dd9884cc662dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mon-a, org.label-schema.build-date=20240514, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-05-21T21:41:52.449 INFO:teuthology.orchestra.run.smithi151.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-21T21:41:52.450 INFO:teuthology.orchestra.run.smithi151.stdout:systemctl: stdout disabled 2024-05-21T21:41:52.464 INFO:teuthology.orchestra.run.smithi151.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-21T21:41:52.464 INFO:teuthology.orchestra.run.smithi151.stdout:systemctl: stdout inactive 2024-05-21T21:41:52.464 INFO:teuthology.orchestra.run.smithi151.stdout:firewalld.service is not enabled 2024-05-21T21:41:52.464 INFO:teuthology.orchestra.run.smithi151.stdout:Not possible to enable service . firewalld.service is not available 2024-05-21T21:41:52.465 INFO:teuthology.orchestra.run.smithi151.stdout:Waiting for mon to start... 2024-05-21T21:41:52.465 INFO:teuthology.orchestra.run.smithi151.stdout:Waiting for mon... 2024-05-21T21:41:52.613 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:41:52 smithi151 bash[31227]: b7a96bd1085a11cc0a5a20a24818bc3b69eff10adc38df66b31dd9884cc662dd 2024-05-21T21:41:52.614 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:41:52 smithi151 ceph-mon[31246]: mkfs c8c7a2e6-17ba-11ef-bc9a-c7b262605968 2024-05-21T21:41:52.614 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:41:52 smithi151 ceph-mon[31246]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-21T21:41:52.614 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:41:52 smithi151 systemd[1]: Started Ceph mon.a for c8c7a2e6-17ba-11ef-bc9a-c7b262605968. 2024-05-21T21:41:53.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:41:53 smithi151 ceph-mon[31246]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-21T21:41:53.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:41:53 smithi151 ceph-mon[31246]: monmap epoch 1 2024-05-21T21:41:53.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:41:53 smithi151 ceph-mon[31246]: fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 2024-05-21T21:41:53.705 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:41:53 smithi151 ceph-mon[31246]: last_changed 2024-05-21T21:41:48.311542+0000 2024-05-21T21:41:53.705 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:41:53 smithi151 ceph-mon[31246]: created 2024-05-21T21:41:48.311542+0000 2024-05-21T21:41:53.705 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:41:53 smithi151 ceph-mon[31246]: min_mon_release 19 (squid) 2024-05-21T21:41:53.705 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:41:53 smithi151 ceph-mon[31246]: election_strategy: 1 2024-05-21T21:41:53.705 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:41:53 smithi151 ceph-mon[31246]: 0: [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] mon.a 2024-05-21T21:41:53.705 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:41:53 smithi151 ceph-mon[31246]: fsmap 2024-05-21T21:41:53.706 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:41:53 smithi151 ceph-mon[31246]: osdmap e1: 0 total, 0 up, 0 in 2024-05-21T21:41:53.706 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:41:53 smithi151 ceph-mon[31246]: mgrmap e1: no daemons active 2024-05-21T21:41:53.706 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:41:53 smithi151 ceph-mon[31246]: from='client.? 172.21.15.151:0/137427244' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-05-21T21:41:53.821 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout cluster: 2024-05-21T21:41:53.822 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout id: c8c7a2e6-17ba-11ef-bc9a-c7b262605968 2024-05-21T21:41:53.822 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-05-21T21:41:53.822 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout 2024-05-21T21:41:53.822 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout services: 2024-05-21T21:41:53.822 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.7377s) 2024-05-21T21:41:53.822 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-05-21T21:41:53.822 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-05-21T21:41:53.822 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout 2024-05-21T21:41:53.822 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout data: 2024-05-21T21:41:53.822 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-05-21T21:41:53.822 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-05-21T21:41:53.823 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-05-21T21:41:53.823 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout pgs: 2024-05-21T21:41:53.823 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout 2024-05-21T21:41:53.823 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:53.058+0000 7fb920000700 1 Processor -- start 2024-05-21T21:41:53.823 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:53.058+0000 7fb920000700 1 -- start start 2024-05-21T21:41:53.823 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:53.058+0000 7fb920000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb91810ff20 0x7fb918112310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:41:53.823 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:53.058+0000 7fb920000700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb9181128e0 con 0x7fb91810ff20 2024-05-21T21:41:53.823 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:53.059+0000 7fb91d600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb91810ff20 0x7fb918112310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:41:53.823 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:53.059+0000 7fb91d600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb91810ff20 0x7fb918112310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:51668/0 (socket says 172.21.15.151:51668) 2024-05-21T21:41:53.823 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:53.059+0000 7fb91d600700 1 -- 172.21.15.151:0/2925227713 learned_addr learned my addr 172.21.15.151:0/2925227713 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:41:53.823 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:53.060+0000 7fb91d600700 1 -- 172.21.15.151:0/2925227713 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb918112a60 con 0x7fb91810ff20 2024-05-21T21:41:53.824 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:53.060+0000 7fb91d600700 1 --2- 172.21.15.151:0/2925227713 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb91810ff20 0x7fb918112310 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb90c0098a0 tx=0x7fb90c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3021e38041373951 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:41:53.824 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:53.060+0000 7fb917e00700 1 -- 172.21.15.151:0/2925227713 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb90c004160 con 0x7fb91810ff20 2024-05-21T21:41:53.824 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:53.060+0000 7fb917e00700 1 -- 172.21.15.151:0/2925227713 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb90c004300 con 0x7fb91810ff20 2024-05-21T21:41:53.824 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:53.060+0000 7fb917e00700 1 -- 172.21.15.151:0/2925227713 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb90c0036a0 con 0x7fb91810ff20 2024-05-21T21:41:53.824 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:53.061+0000 7fb920000700 1 -- 172.21.15.151:0/2925227713 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb91810ff20 msgr2=0x7fb918112310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:41:53.824 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:53.061+0000 7fb920000700 1 --2- 172.21.15.151:0/2925227713 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb91810ff20 0x7fb918112310 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb90c0098a0 tx=0x7fb90c009bb0 comp rx=0 tx=0).stop 2024-05-21T21:41:53.824 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:53.061+0000 7fb920000700 1 -- 172.21.15.151:0/2925227713 shutdown_connections 2024-05-21T21:41:53.824 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:53.061+0000 7fb920000700 1 --2- 172.21.15.151:0/2925227713 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb91810ff20 0x7fb918112310 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:41:53.824 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:53.061+0000 7fb920000700 1 -- 172.21.15.151:0/2925227713 >> 172.21.15.151:0/2925227713 conn(0x7fb91807e890 msgr2=0x7fb918080cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:41:53.825 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:53.061+0000 7fb920000700 1 -- 172.21.15.151:0/2925227713 shutdown_connections 2024-05-21T21:41:53.825 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:53.061+0000 7fb920000700 1 -- 172.21.15.151:0/2925227713 wait complete. 2024-05-21T21:41:53.825 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:53.062+0000 7fb920000700 1 Processor -- start 2024-05-21T21:41:53.825 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:53.062+0000 7fb920000700 1 -- start start 2024-05-21T21:41:53.825 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:53.063+0000 7fb920000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb91810ff20 0x7fb9181a2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:41:53.825 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:53.063+0000 7fb920000700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb918112f90 con 0x7fb91810ff20 2024-05-21T21:41:53.826 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:53.063+0000 7fb91d600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb91810ff20 0x7fb9181a2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:41:53.826 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:53.063+0000 7fb91d600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb91810ff20 0x7fb9181a2e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:51680/0 (socket says 172.21.15.151:51680) 2024-05-21T21:41:53.826 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:53.063+0000 7fb91d600700 1 -- 172.21.15.151:0/137427244 learned_addr learned my addr 172.21.15.151:0/137427244 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:41:53.826 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:53.063+0000 7fb91d600700 1 -- 172.21.15.151:0/137427244 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9181a3360 con 0x7fb91810ff20 2024-05-21T21:41:53.826 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:53.064+0000 7fb91d600700 1 --2- 172.21.15.151:0/137427244 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb91810ff20 0x7fb9181a2e20 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fb90c000c00 tx=0x7fb90c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:41:53.827 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:53.064+0000 7fb916000700 1 -- 172.21.15.151:0/137427244 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb90c003db0 con 0x7fb91810ff20 2024-05-21T21:41:53.827 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:53.064+0000 7fb916000700 1 -- 172.21.15.151:0/137427244 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb90c011630 con 0x7fb91810ff20 2024-05-21T21:41:53.827 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:53.064+0000 7fb920000700 1 -- 172.21.15.151:0/137427244 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9181a35f0 con 0x7fb91810ff20 2024-05-21T21:41:53.827 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:53.064+0000 7fb916000700 1 -- 172.21.15.151:0/137427244 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb90c011950 con 0x7fb91810ff20 2024-05-21T21:41:53.827 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:53.064+0000 7fb920000700 1 -- 172.21.15.151:0/137427244 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9181a62e0 con 0x7fb91810ff20 2024-05-21T21:41:53.827 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:53.065+0000 7fb916000700 1 -- 172.21.15.151:0/137427244 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb90c011af0 con 0x7fb91810ff20 2024-05-21T21:41:53.827 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:53.065+0000 7fb916000700 1 -- 172.21.15.151:0/137427244 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb90c01c450 con 0x7fb91810ff20 2024-05-21T21:41:53.828 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:53.065+0000 7fb920000700 1 -- 172.21.15.151:0/137427244 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb918064ba0 con 0x7fb91810ff20 2024-05-21T21:41:53.828 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:53.068+0000 7fb916000700 1 -- 172.21.15.151:0/137427244 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fb90c032020 con 0x7fb91810ff20 2024-05-21T21:41:53.828 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:53.147+0000 7fb920000700 1 -- 172.21.15.151:0/137427244 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fb91802e770 con 0x7fb91810ff20 2024-05-21T21:41:53.828 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:53.147+0000 7fb916000700 1 -- 172.21.15.151:0/137427244 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+315 (secure 0 0 0) 0x7fb90c004260 con 0x7fb91810ff20 2024-05-21T21:41:53.828 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:53.149+0000 7fb920000700 1 -- 172.21.15.151:0/137427244 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb91810ff20 msgr2=0x7fb9181a2e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:41:53.828 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:53.149+0000 7fb920000700 1 --2- 172.21.15.151:0/137427244 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb91810ff20 0x7fb9181a2e20 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fb90c000c00 tx=0x7fb90c004750 comp rx=0 tx=0).stop 2024-05-21T21:41:53.829 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:53.149+0000 7fb920000700 1 -- 172.21.15.151:0/137427244 shutdown_connections 2024-05-21T21:41:53.829 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:53.149+0000 7fb920000700 1 --2- 172.21.15.151:0/137427244 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb91810ff20 0x7fb9181a2e20 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:41:53.829 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:53.149+0000 7fb920000700 1 -- 172.21.15.151:0/137427244 >> 172.21.15.151:0/137427244 conn(0x7fb91807e890 msgr2=0x7fb918081bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:41:53.829 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:53.149+0000 7fb920000700 1 -- 172.21.15.151:0/137427244 shutdown_connections 2024-05-21T21:41:53.829 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:53.149+0000 7fb920000700 1 -- 172.21.15.151:0/137427244 wait complete. 2024-05-21T21:41:53.829 INFO:teuthology.orchestra.run.smithi151.stdout:mon is available 2024-05-21T21:41:53.829 INFO:teuthology.orchestra.run.smithi151.stdout:Assimilating anything we can from ceph.conf... 2024-05-21T21:41:54.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:41:54 smithi151 ceph-mon[31246]: from='client.? 172.21.15.151:0/3380821874' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-21T21:41:54.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:41:54 smithi151 ceph-mon[31246]: from='client.? 172.21.15.151:0/3380821874' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-05-21T21:41:55.188 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout 2024-05-21T21:41:55.189 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout [global] 2024-05-21T21:41:55.189 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout fsid = c8c7a2e6-17ba-11ef-bc9a-c7b262605968 2024-05-21T21:41:55.189 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-05-21T21:41:55.189 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.151:3300,v1:172.21.15.151:6789] 2024-05-21T21:41:55.189 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-21T21:41:55.189 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-21T21:41:55.189 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-21T21:41:55.189 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-21T21:41:55.189 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout 2024-05-21T21:41:55.189 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-21T21:41:55.189 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-21T21:41:55.189 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout 2024-05-21T21:41:55.190 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout [osd] 2024-05-21T21:41:55.190 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-05-21T21:41:55.190 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-21T21:41:55.190 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-21T21:41:55.190 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-21T21:41:55.190 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-21T21:41:55.190 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-21T21:41:55.190 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:54.420+0000 7f6d09c00700 1 Processor -- start 2024-05-21T21:41:55.190 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:54.420+0000 7f6d09c00700 1 -- start start 2024-05-21T21:41:55.190 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:54.421+0000 7f6d09c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6d04108e40 0x7f6d04109260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:41:55.190 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:54.421+0000 7f6d09c00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6d041097a0 con 0x7f6d04108e40 2024-05-21T21:41:55.190 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:54.422+0000 7f6d03e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6d04108e40 0x7f6d04109260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:41:55.190 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:54.422+0000 7f6d03e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6d04108e40 0x7f6d04109260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:51688/0 (socket says 172.21.15.151:51688) 2024-05-21T21:41:55.191 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:54.422+0000 7f6d03e00700 1 -- 172.21.15.151:0/3972119356 learned_addr learned my addr 172.21.15.151:0/3972119356 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:41:55.191 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:54.423+0000 7f6d03e00700 1 -- 172.21.15.151:0/3972119356 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d04109fa0 con 0x7f6d04108e40 2024-05-21T21:41:55.191 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:54.423+0000 7f6d03e00700 1 --2- 172.21.15.151:0/3972119356 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6d04108e40 0x7f6d04109260 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f6cf40098a0 tx=0x7f6cf4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62551c28fa04cbf6 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:41:55.191 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:54.423+0000 7f6d02a00700 1 -- 172.21.15.151:0/3972119356 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6cf4004160 con 0x7f6d04108e40 2024-05-21T21:41:55.191 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:54.423+0000 7f6d02a00700 1 -- 172.21.15.151:0/3972119356 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cf4004300 con 0x7f6d04108e40 2024-05-21T21:41:55.191 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:54.423+0000 7f6d02a00700 1 -- 172.21.15.151:0/3972119356 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6cf40036a0 con 0x7f6d04108e40 2024-05-21T21:41:55.191 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:54.424+0000 7f6d09c00700 1 -- 172.21.15.151:0/3972119356 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6d04108e40 msgr2=0x7f6d04109260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:41:55.191 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:54.424+0000 7f6d09c00700 1 --2- 172.21.15.151:0/3972119356 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6d04108e40 0x7f6d04109260 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f6cf40098a0 tx=0x7f6cf4009bb0 comp rx=0 tx=0).stop 2024-05-21T21:41:55.191 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:54.424+0000 7f6d09c00700 1 -- 172.21.15.151:0/3972119356 shutdown_connections 2024-05-21T21:41:55.191 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:54.424+0000 7f6d09c00700 1 --2- 172.21.15.151:0/3972119356 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6d04108e40 0x7f6d04109260 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:41:55.191 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:54.424+0000 7f6d09c00700 1 -- 172.21.15.151:0/3972119356 >> 172.21.15.151:0/3972119356 conn(0x7f6d041043c0 msgr2=0x7f6d04106820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:41:55.191 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:54.424+0000 7f6d09c00700 1 -- 172.21.15.151:0/3972119356 shutdown_connections 2024-05-21T21:41:55.192 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:54.424+0000 7f6d09c00700 1 -- 172.21.15.151:0/3972119356 wait complete. 2024-05-21T21:41:55.192 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:54.425+0000 7f6d09c00700 1 Processor -- start 2024-05-21T21:41:55.192 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:54.425+0000 7f6d09c00700 1 -- start start 2024-05-21T21:41:55.192 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:54.426+0000 7f6d09c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6d04108e40 0x7f6d0407da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:41:55.192 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:54.426+0000 7f6d09c00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6d0410a470 con 0x7f6d04108e40 2024-05-21T21:41:55.192 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:54.426+0000 7f6d03e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6d04108e40 0x7f6d0407da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:41:55.192 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:54.426+0000 7f6d03e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6d04108e40 0x7f6d0407da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:51692/0 (socket says 172.21.15.151:51692) 2024-05-21T21:41:55.192 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:54.426+0000 7f6d03e00700 1 -- 172.21.15.151:0/3380821874 learned_addr learned my addr 172.21.15.151:0/3380821874 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:41:55.192 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:54.427+0000 7f6d03e00700 1 -- 172.21.15.151:0/3380821874 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d0407df90 con 0x7f6d04108e40 2024-05-21T21:41:55.192 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:54.427+0000 7f6d03e00700 1 --2- 172.21.15.151:0/3380821874 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6d04108e40 0x7f6d0407da50 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f6cf4000c00 tx=0x7f6cf4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:41:55.192 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:54.427+0000 7f6d00c00700 1 -- 172.21.15.151:0/3380821874 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6cf4003db0 con 0x7f6d04108e40 2024-05-21T21:41:55.192 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:54.427+0000 7f6d00c00700 1 -- 172.21.15.151:0/3380821874 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cf4011630 con 0x7f6d04108e40 2024-05-21T21:41:55.192 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:54.427+0000 7f6d09c00700 1 -- 172.21.15.151:0/3380821874 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d0407c180 con 0x7f6d04108e40 2024-05-21T21:41:55.192 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:54.427+0000 7f6d00c00700 1 -- 172.21.15.151:0/3380821874 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6cf4011950 con 0x7f6d04108e40 2024-05-21T21:41:55.193 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:54.427+0000 7f6d09c00700 1 -- 172.21.15.151:0/3380821874 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d0407c6c0 con 0x7f6d04108e40 2024-05-21T21:41:55.193 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:54.428+0000 7f6d00c00700 1 -- 172.21.15.151:0/3380821874 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6cf401a460 con 0x7f6d04108e40 2024-05-21T21:41:55.193 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:54.428+0000 7f6d00c00700 1 -- 172.21.15.151:0/3380821874 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f6cf401b780 con 0x7f6d04108e40 2024-05-21T21:41:55.193 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:54.429+0000 7f6d09c00700 1 -- 172.21.15.151:0/3380821874 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d0406a6a0 con 0x7f6d04108e40 2024-05-21T21:41:55.193 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:54.432+0000 7f6d00c00700 1 -- 172.21.15.151:0/3380821874 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f6cf401ba20 con 0x7f6d04108e40 2024-05-21T21:41:55.193 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:54.507+0000 7f6d09c00700 1 -- 172.21.15.151:0/3380821874 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f6d04195980 con 0x7f6d04108e40 2024-05-21T21:41:55.193 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:54.514+0000 7f6d00c00700 1 -- 172.21.15.151:0/3380821874 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+494 (secure 0 0 0) 0x7f6cf401bc00 con 0x7f6d04108e40 2024-05-21T21:41:55.193 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:54.515+0000 7f6d00c00700 1 -- 172.21.15.151:0/3380821874 <== mon.0 v2:172.21.15.151:3300/0 8 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6cf401bde0 con 0x7f6d04108e40 2024-05-21T21:41:55.193 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:54.516+0000 7f6d09c00700 1 -- 172.21.15.151:0/3380821874 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6d04108e40 msgr2=0x7f6d0407da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:41:55.193 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:54.516+0000 7f6d09c00700 1 --2- 172.21.15.151:0/3380821874 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6d04108e40 0x7f6d0407da50 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f6cf4000c00 tx=0x7f6cf4004750 comp rx=0 tx=0).stop 2024-05-21T21:41:55.193 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:54.516+0000 7f6d09c00700 1 -- 172.21.15.151:0/3380821874 shutdown_connections 2024-05-21T21:41:55.193 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:54.516+0000 7f6d09c00700 1 --2- 172.21.15.151:0/3380821874 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6d04108e40 0x7f6d0407da50 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:41:55.193 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:54.516+0000 7f6d09c00700 1 -- 172.21.15.151:0/3380821874 >> 172.21.15.151:0/3380821874 conn(0x7f6d041043c0 msgr2=0x7f6d041930e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:41:55.194 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:54.516+0000 7f6d09c00700 1 -- 172.21.15.151:0/3380821874 shutdown_connections 2024-05-21T21:41:55.194 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:54.516+0000 7f6d09c00700 1 -- 172.21.15.151:0/3380821874 wait complete. 2024-05-21T21:41:55.194 INFO:teuthology.orchestra.run.smithi151.stdout:Generating new minimal ceph.conf... 2024-05-21T21:41:56.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:41:55 smithi151 ceph-mon[31246]: from='client.? 172.21.15.151:0/682754738' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:42:06.236 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:55.766+0000 7fb713400700 1 Processor -- start 2024-05-21T21:42:06.236 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:55.766+0000 7fb713400700 1 -- start start 2024-05-21T21:42:06.236 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:55.767+0000 7fb713400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb70c07dd40 0x7fb70c07c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:42:06.236 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:55.767+0000 7fb713400700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb70c07c770 con 0x7fb70c07dd40 2024-05-21T21:42:06.236 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:55.767+0000 7fb710a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb70c07dd40 0x7fb70c07c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:42:06.236 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:55.767+0000 7fb710a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb70c07dd40 0x7fb70c07c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:51708/0 (socket says 172.21.15.151:51708) 2024-05-21T21:42:06.237 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:55.767+0000 7fb710a00700 1 -- 172.21.15.151:0/1259844994 learned_addr learned my addr 172.21.15.151:0/1259844994 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:42:06.237 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:55.768+0000 7fb710a00700 1 -- 172.21.15.151:0/1259844994 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb70c07c8f0 con 0x7fb70c07dd40 2024-05-21T21:42:06.237 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:55.768+0000 7fb710a00700 1 --2- 172.21.15.151:0/1259844994 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb70c07dd40 0x7fb70c07c1a0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb700009e80 tx=0x7fb70000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45625be6e376d1e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:42:06.237 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:55.768+0000 7fb70b400700 1 -- 172.21.15.151:0/1259844994 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb700017070 con 0x7fb70c07dd40 2024-05-21T21:42:06.237 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:55.768+0000 7fb70b400700 1 -- 172.21.15.151:0/1259844994 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb70000b8c0 con 0x7fb70c07dd40 2024-05-21T21:42:06.237 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:55.768+0000 7fb70b400700 1 -- 172.21.15.151:0/1259844994 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb700004490 con 0x7fb70c07dd40 2024-05-21T21:42:06.237 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:55.769+0000 7fb713400700 1 -- 172.21.15.151:0/1259844994 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb70c07dd40 msgr2=0x7fb70c07c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:42:06.237 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:55.769+0000 7fb713400700 1 --2- 172.21.15.151:0/1259844994 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb70c07dd40 0x7fb70c07c1a0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb700009e80 tx=0x7fb70000b330 comp rx=0 tx=0).stop 2024-05-21T21:42:06.237 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:55.769+0000 7fb713400700 1 -- 172.21.15.151:0/1259844994 shutdown_connections 2024-05-21T21:42:06.237 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:55.769+0000 7fb713400700 1 --2- 172.21.15.151:0/1259844994 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb70c07dd40 0x7fb70c07c1a0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:42:06.237 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:55.769+0000 7fb713400700 1 -- 172.21.15.151:0/1259844994 >> 172.21.15.151:0/1259844994 conn(0x7fb70c1045e0 msgr2=0x7fb70c106a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:42:06.237 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:55.769+0000 7fb713400700 1 -- 172.21.15.151:0/1259844994 shutdown_connections 2024-05-21T21:42:06.238 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:55.769+0000 7fb713400700 1 -- 172.21.15.151:0/1259844994 wait complete. 2024-05-21T21:42:06.238 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:55.770+0000 7fb713400700 1 Processor -- start 2024-05-21T21:42:06.238 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:55.771+0000 7fb713400700 1 -- start start 2024-05-21T21:42:06.238 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:55.771+0000 7fb713400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb70c07dd40 0x7fb70c19f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:42:06.238 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:55.771+0000 7fb713400700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb70c10f5d0 con 0x7fb70c07dd40 2024-05-21T21:42:06.238 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:55.771+0000 7fb710a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb70c07dd40 0x7fb70c19f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:42:06.238 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:55.771+0000 7fb710a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb70c07dd40 0x7fb70c19f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:51720/0 (socket says 172.21.15.151:51720) 2024-05-21T21:42:06.238 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:55.771+0000 7fb710a00700 1 -- 172.21.15.151:0/682754738 learned_addr learned my addr 172.21.15.151:0/682754738 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:42:06.238 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:55.772+0000 7fb710a00700 1 -- 172.21.15.151:0/682754738 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb70c19f6c0 con 0x7fb70c07dd40 2024-05-21T21:42:06.238 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:55.772+0000 7fb710a00700 1 --2- 172.21.15.151:0/682754738 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb70c07dd40 0x7fb70c19f180 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb70000b7b0 tx=0x7fb70000be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:42:06.238 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:55.772+0000 7fb709600700 1 -- 172.21.15.151:0/682754738 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb700020020 con 0x7fb70c07dd40 2024-05-21T21:42:06.238 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:55.772+0000 7fb709600700 1 -- 172.21.15.151:0/682754738 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb700003d10 con 0x7fb70c07dd40 2024-05-21T21:42:06.239 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:55.772+0000 7fb709600700 1 -- 172.21.15.151:0/682754738 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb700017040 con 0x7fb70c07dd40 2024-05-21T21:42:06.239 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:55.772+0000 7fb713400700 1 -- 172.21.15.151:0/682754738 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb70c19f950 con 0x7fb70c07dd40 2024-05-21T21:42:06.239 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:55.772+0000 7fb713400700 1 -- 172.21.15.151:0/682754738 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb70c1a2640 con 0x7fb70c07dd40 2024-05-21T21:42:06.239 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:55.773+0000 7fb709600700 1 -- 172.21.15.151:0/682754738 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb700023020 con 0x7fb70c07dd40 2024-05-21T21:42:06.239 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:55.773+0000 7fb709600700 1 -- 172.21.15.151:0/682754738 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb70000f050 con 0x7fb70c07dd40 2024-05-21T21:42:06.239 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:55.774+0000 7fb713400700 1 -- 172.21.15.151:0/682754738 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb70c064ba0 con 0x7fb70c07dd40 2024-05-21T21:42:06.239 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:55.776+0000 7fb709600700 1 -- 172.21.15.151:0/682754738 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fb70002a020 con 0x7fb70c07dd40 2024-05-21T21:42:06.239 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:55.854+0000 7fb713400700 1 -- 172.21.15.151:0/682754738 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fb70c1a2be0 con 0x7fb70c07dd40 2024-05-21T21:42:06.239 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:55.855+0000 7fb709600700 1 -- 172.21.15.151:0/682754738 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+177 (secure 0 0 0) 0x7fb700003ec0 con 0x7fb70c07dd40 2024-05-21T21:42:06.239 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:55.856+0000 7fb713400700 1 -- 172.21.15.151:0/682754738 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb70c07dd40 msgr2=0x7fb70c19f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:42:06.239 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:55.856+0000 7fb713400700 1 --2- 172.21.15.151:0/682754738 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb70c07dd40 0x7fb70c19f180 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb70000b7b0 tx=0x7fb70000be90 comp rx=0 tx=0).stop 2024-05-21T21:42:06.240 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:55.856+0000 7fb713400700 1 -- 172.21.15.151:0/682754738 shutdown_connections 2024-05-21T21:42:06.240 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:55.856+0000 7fb713400700 1 --2- 172.21.15.151:0/682754738 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb70c07dd40 0x7fb70c19f180 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:42:06.240 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:55.856+0000 7fb713400700 1 -- 172.21.15.151:0/682754738 >> 172.21.15.151:0/682754738 conn(0x7fb70c1045e0 msgr2=0x7fb70c1051a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:42:06.240 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:55.856+0000 7fb713400700 1 -- 172.21.15.151:0/682754738 shutdown_connections 2024-05-21T21:42:06.240 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:41:55.856+0000 7fb713400700 1 -- 172.21.15.151:0/682754738 wait complete. 2024-05-21T21:42:06.240 INFO:teuthology.orchestra.run.smithi151.stdout:Restarting the monitor... 2024-05-21T21:42:06.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:06 smithi151 systemd[1]: Stopping Ceph mon.a for c8c7a2e6-17ba-11ef-bc9a-c7b262605968... 2024-05-21T21:42:07.085 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:06 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mon-a[31242]: 2024-05-21T21:42:06.722+0000 7f0396c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T21:42:07.085 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:06 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mon-a[31242]: 2024-05-21T21:42:06.722+0000 7f0396c00700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-05-21T21:42:07.085 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:06 smithi151 podman[31489]: 2024-05-21 21:42:06.805107561 +0000 UTC m=+0.201635192 container died b7a96bd1085a11cc0a5a20a24818bc3b69eff10adc38df66b31dd9884cc662dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mon-a, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.schema-version=1.0) 2024-05-21T21:42:07.400 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:07 smithi151 podman[31489]: 2024-05-21 21:42:07.223253414 +0000 UTC m=+0.619781056 container cleanup b7a96bd1085a11cc0a5a20a24818bc3b69eff10adc38df66b31dd9884cc662dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mon-a, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-21T21:42:07.400 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:07 smithi151 bash[31489]: ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mon-a 2024-05-21T21:42:07.702 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:07 smithi151 podman[31503]: 2024-05-21 21:42:07.448767195 +0000 UTC m=+0.630048311 container remove b7a96bd1085a11cc0a5a20a24818bc3b69eff10adc38df66b31dd9884cc662dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mon-a, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-21T21:42:08.016 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:07 smithi151 systemd[1]: ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@mon.a.service: Deactivated successfully. 2024-05-21T21:42:08.016 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:07 smithi151 systemd[1]: Stopped Ceph mon.a for c8c7a2e6-17ba-11ef-bc9a-c7b262605968. 2024-05-21T21:42:08.016 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:07 smithi151 systemd[1]: ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@mon.a.service: Consumed 1.852s CPU time. 2024-05-21T21:42:08.016 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:07 smithi151 systemd[1]: Starting Ceph mon.a for c8c7a2e6-17ba-11ef-bc9a-c7b262605968... 2024-05-21T21:42:08.553 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 podman[31604]: 2024-05-21 21:42:08.299507353 +0000 UTC m=+0.123315158 container create 9cfe13f3b9612e0b654901b8e1252a003ced7683fbca5f515e236eedb8a1078e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mon-a, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True) 2024-05-21T21:42:08.553 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 podman[31604]: 2024-05-21 21:42:08.211223676 +0000 UTC m=+0.035031459 image pull d6bc8f31cbc1a7165ca248fe9ba0b06ea14b30f46e1ea33ced1a5238e247af1f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 2024-05-21T21:42:08.553 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 podman[31604]: 2024-05-21 21:42:08.552880754 +0000 UTC m=+0.376688566 container init 9cfe13f3b9612e0b654901b8e1252a003ced7683fbca5f515e236eedb8a1078e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0) 2024-05-21T21:42:08.644 INFO:teuthology.orchestra.run.smithi151.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-05-21T21:42:08.954 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 podman[31604]: 2024-05-21 21:42:08.563447137 +0000 UTC m=+0.387254916 container start 9cfe13f3b9612e0b654901b8e1252a003ced7683fbca5f515e236eedb8a1078e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mon-a, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-05-21T21:42:08.955 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: set uid:gid to 167:167 (ceph:ceph) 2024-05-21T21:42:08.955 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: ceph version 19.0.0-3830-g3b748ea0 (3b748ea08891006c1152513d25ab4a03f7c9a995) squid (dev), process ceph-mon, pid 2 2024-05-21T21:42:08.955 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: pidfile_write: ignore empty --pid-file 2024-05-21T21:42:08.955 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: load: jerasure load: lrc 2024-05-21T21:42:08.955 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: RocksDB version: 7.9.2 2024-05-21T21:42:08.955 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Git sha 0 2024-05-21T21:42:08.955 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Compile date 2024-05-20 21:53:18 2024-05-21T21:42:08.955 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: DB SUMMARY 2024-05-21T21:42:08.955 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: DB Session ID: 77S5N2I1JT2UZCU87CQ7 2024-05-21T21:42:08.955 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: CURRENT file: CURRENT 2024-05-21T21:42:08.956 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: IDENTITY file: IDENTITY 2024-05-21T21:42:08.956 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-05-21T21:42:08.956 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-05-21T21:42:08.956 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 105468 ; 2024-05-21T21:42:08.956 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.error_if_exists: 0 2024-05-21T21:42:08.956 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.create_if_missing: 0 2024-05-21T21:42:08.956 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.paranoid_checks: 1 2024-05-21T21:42:08.957 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-21T21:42:08.957 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-21T21:42:08.957 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-21T21:42:08.957 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.env: 0x55859f236cc0 2024-05-21T21:42:08.957 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.fs: PosixFileSystem 2024-05-21T21:42:08.957 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.info_log: 0x5585a0957de0 2024-05-21T21:42:08.957 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.max_file_opening_threads: 16 2024-05-21T21:42:08.958 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.statistics: (nil) 2024-05-21T21:42:08.958 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.use_fsync: 0 2024-05-21T21:42:08.958 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.max_log_file_size: 0 2024-05-21T21:42:08.958 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-21T21:42:08.958 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-21T21:42:08.958 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.keep_log_file_num: 1000 2024-05-21T21:42:08.958 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.recycle_log_file_num: 0 2024-05-21T21:42:08.958 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.allow_fallocate: 1 2024-05-21T21:42:08.959 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.allow_mmap_reads: 0 2024-05-21T21:42:08.959 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.allow_mmap_writes: 0 2024-05-21T21:42:08.959 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.use_direct_reads: 0 2024-05-21T21:42:08.960 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-21T21:42:08.960 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.create_missing_column_families: 0 2024-05-21T21:42:08.960 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.db_log_dir: 2024-05-21T21:42:08.960 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.wal_dir: 2024-05-21T21:42:08.960 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-21T21:42:08.960 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-21T21:42:08.960 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-21T21:42:08.960 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-21T21:42:08.960 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-21T21:42:08.960 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-21T21:42:08.961 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.advise_random_on_open: 1 2024-05-21T21:42:08.961 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.db_write_buffer_size: 0 2024-05-21T21:42:08.962 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.write_buffer_manager: 0x5585a09290e0 2024-05-21T21:42:08.962 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-21T21:42:08.962 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-21T21:42:08.962 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-21T21:42:08.962 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.rate_limiter: (nil) 2024-05-21T21:42:08.962 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-21T21:42:08.962 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.wal_recovery_mode: 2 2024-05-21T21:42:08.962 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.enable_thread_tracking: 0 2024-05-21T21:42:08.962 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.enable_pipelined_write: 0 2024-05-21T21:42:08.962 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.unordered_write: 0 2024-05-21T21:42:08.963 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-21T21:42:08.963 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-21T21:42:08.963 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-21T21:42:08.963 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-21T21:42:08.963 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.row_cache: None 2024-05-21T21:42:08.963 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.wal_filter: None 2024-05-21T21:42:08.963 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-21T21:42:08.963 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.allow_ingest_behind: 0 2024-05-21T21:42:08.963 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.two_write_queues: 0 2024-05-21T21:42:08.963 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.manual_wal_flush: 0 2024-05-21T21:42:08.963 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.wal_compression: 0 2024-05-21T21:42:08.963 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.atomic_flush: 0 2024-05-21T21:42:08.963 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-21T21:42:08.964 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-21T21:42:08.964 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-21T21:42:08.964 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.log_readahead_size: 0 2024-05-21T21:42:08.964 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-21T21:42:08.964 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.best_efforts_recovery: 0 2024-05-21T21:42:08.964 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-21T21:42:08.964 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-21T21:42:08.964 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.allow_data_in_errors: 0 2024-05-21T21:42:08.964 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.db_host_id: __hostname__ 2024-05-21T21:42:08.964 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-21T21:42:08.964 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.max_background_jobs: 2 2024-05-21T21:42:08.964 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.max_background_compactions: -1 2024-05-21T21:42:08.964 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.max_subcompactions: 1 2024-05-21T21:42:08.965 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-21T21:42:08.965 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-21T21:42:08.965 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-21T21:42:08.965 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.max_total_wal_size: 0 2024-05-21T21:42:08.965 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-21T21:42:08.965 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-21T21:42:08.965 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-21T21:42:08.965 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-21T21:42:08.965 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.max_open_files: -1 2024-05-21T21:42:08.965 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.bytes_per_sync: 0 2024-05-21T21:42:08.965 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-21T21:42:08.965 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-21T21:42:08.965 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.compaction_readahead_size: 0 2024-05-21T21:42:08.966 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.max_background_flushes: -1 2024-05-21T21:42:08.966 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Compression algorithms supported: 2024-05-21T21:42:08.966 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-21T21:42:08.966 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: kZSTD supported: 0 2024-05-21T21:42:08.966 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: kXpressCompression supported: 0 2024-05-21T21:42:08.966 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: kLZ4HCCompression supported: 1 2024-05-21T21:42:08.966 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: kZlibCompression supported: 1 2024-05-21T21:42:08.966 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: kSnappyCompression supported: 1 2024-05-21T21:42:08.966 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: kLZ4Compression supported: 1 2024-05-21T21:42:08.966 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: kBZip2Compression supported: 0 2024-05-21T21:42:08.966 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-21T21:42:08.966 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-21T21:42:08.966 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-05-21T21:42:08.966 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-21T21:42:08.967 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-21T21:42:08.967 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.merge_operator: 2024-05-21T21:42:08.967 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.compaction_filter: None 2024-05-21T21:42:08.967 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.compaction_filter_factory: None 2024-05-21T21:42:08.967 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.sst_partitioner_factory: None 2024-05-21T21:42:08.967 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-21T21:42:08.967 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-21T21:42:08.967 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5585a0957fa0) 2024-05-21T21:42:08.967 INFO:journalctl@ceph.mon.a.smithi151.stdout: cache_index_and_filter_blocks: 1 2024-05-21T21:42:08.967 INFO:journalctl@ceph.mon.a.smithi151.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-21T21:42:08.967 INFO:journalctl@ceph.mon.a.smithi151.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-21T21:42:08.967 INFO:journalctl@ceph.mon.a.smithi151.stdout: pin_top_level_index_and_filter: 1 2024-05-21T21:42:08.968 INFO:journalctl@ceph.mon.a.smithi151.stdout: index_type: 0 2024-05-21T21:42:08.968 INFO:journalctl@ceph.mon.a.smithi151.stdout: data_block_index_type: 0 2024-05-21T21:42:08.968 INFO:journalctl@ceph.mon.a.smithi151.stdout: index_shortening: 1 2024-05-21T21:42:08.968 INFO:journalctl@ceph.mon.a.smithi151.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-21T21:42:08.968 INFO:journalctl@ceph.mon.a.smithi151.stdout: checksum: 4 2024-05-21T21:42:08.968 INFO:journalctl@ceph.mon.a.smithi151.stdout: no_block_cache: 0 2024-05-21T21:42:08.968 INFO:journalctl@ceph.mon.a.smithi151.stdout: block_cache: 0x5585a095b1f0 2024-05-21T21:42:08.968 INFO:journalctl@ceph.mon.a.smithi151.stdout: block_cache_name: BinnedLRUCache 2024-05-21T21:42:08.968 INFO:journalctl@ceph.mon.a.smithi151.stdout: block_cache_options: 2024-05-21T21:42:08.968 INFO:journalctl@ceph.mon.a.smithi151.stdout: capacity : 536870912 2024-05-21T21:42:08.968 INFO:journalctl@ceph.mon.a.smithi151.stdout: num_shard_bits : 4 2024-05-21T21:42:08.968 INFO:journalctl@ceph.mon.a.smithi151.stdout: strict_capacity_limit : 0 2024-05-21T21:42:08.968 INFO:journalctl@ceph.mon.a.smithi151.stdout: high_pri_pool_ratio: 0.000 2024-05-21T21:42:08.969 INFO:journalctl@ceph.mon.a.smithi151.stdout: block_cache_compressed: (nil) 2024-05-21T21:42:08.969 INFO:journalctl@ceph.mon.a.smithi151.stdout: persistent_cache: (nil) 2024-05-21T21:42:08.969 INFO:journalctl@ceph.mon.a.smithi151.stdout: block_size: 4096 2024-05-21T21:42:08.969 INFO:journalctl@ceph.mon.a.smithi151.stdout: block_size_deviation: 10 2024-05-21T21:42:08.969 INFO:journalctl@ceph.mon.a.smithi151.stdout: block_restart_interval: 16 2024-05-21T21:42:08.969 INFO:journalctl@ceph.mon.a.smithi151.stdout: index_block_restart_interval: 1 2024-05-21T21:42:08.969 INFO:journalctl@ceph.mon.a.smithi151.stdout: metadata_block_size: 4096 2024-05-21T21:42:08.969 INFO:journalctl@ceph.mon.a.smithi151.stdout: partition_filters: 0 2024-05-21T21:42:08.969 INFO:journalctl@ceph.mon.a.smithi151.stdout: use_delta_encoding: 1 2024-05-21T21:42:08.969 INFO:journalctl@ceph.mon.a.smithi151.stdout: filter_policy: bloomfilter 2024-05-21T21:42:08.969 INFO:journalctl@ceph.mon.a.smithi151.stdout: whole_key_filtering: 1 2024-05-21T21:42:08.969 INFO:journalctl@ceph.mon.a.smithi151.stdout: verify_compression: 0 2024-05-21T21:42:08.969 INFO:journalctl@ceph.mon.a.smithi151.stdout: read_amp_bytes_per_bit: 0 2024-05-21T21:42:08.969 INFO:journalctl@ceph.mon.a.smithi151.stdout: format_version: 5 2024-05-21T21:42:08.970 INFO:journalctl@ceph.mon.a.smithi151.stdout: enable_index_compression: 1 2024-05-21T21:42:08.970 INFO:journalctl@ceph.mon.a.smithi151.stdout: block_align: 0 2024-05-21T21:42:08.970 INFO:journalctl@ceph.mon.a.smithi151.stdout: max_auto_readahead_size: 262144 2024-05-21T21:42:08.970 INFO:journalctl@ceph.mon.a.smithi151.stdout: prepopulate_block_cache: 0 2024-05-21T21:42:08.970 INFO:journalctl@ceph.mon.a.smithi151.stdout: initial_auto_readahead_size: 8192 2024-05-21T21:42:08.970 INFO:journalctl@ceph.mon.a.smithi151.stdout: num_file_reads_for_auto_readahead: 2 2024-05-21T21:42:08.970 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.write_buffer_size: 33554432 2024-05-21T21:42:08.970 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.max_write_buffer_number: 2 2024-05-21T21:42:08.970 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.compression: NoCompression 2024-05-21T21:42:08.970 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.bottommost_compression: Disabled 2024-05-21T21:42:08.970 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.prefix_extractor: nullptr 2024-05-21T21:42:08.970 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-21T21:42:08.970 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.num_levels: 7 2024-05-21T21:42:08.971 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-21T21:42:08.971 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-21T21:42:08.971 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-21T21:42:08.971 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-21T21:42:08.971 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-21T21:42:08.971 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-21T21:42:08.971 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-21T21:42:08.971 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-21T21:42:08.971 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-21T21:42:08.971 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-21T21:42:08.971 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-21T21:42:08.971 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-21T21:42:08.971 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-21T21:42:08.972 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.compression_opts.level: 32767 2024-05-21T21:42:08.972 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.compression_opts.strategy: 0 2024-05-21T21:42:08.972 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-21T21:42:08.972 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-21T21:42:08.972 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-21T21:42:08.972 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-21T21:42:08.972 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.compression_opts.enabled: false 2024-05-21T21:42:08.972 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-21T21:42:08.972 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-21T21:42:08.972 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-21T21:42:08.972 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-21T21:42:08.972 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.target_file_size_base: 67108864 2024-05-21T21:42:08.972 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-21T21:42:08.972 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-21T21:42:08.973 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-21T21:42:08.973 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-21T21:42:08.973 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-21T21:42:08.973 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-21T21:42:08.973 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-21T21:42:08.973 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-21T21:42:08.973 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-21T21:42:08.973 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-21T21:42:08.973 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-21T21:42:08.973 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-21T21:42:08.973 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-21T21:42:08.973 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-21T21:42:08.974 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.arena_block_size: 1048576 2024-05-21T21:42:08.974 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-21T21:42:08.974 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-21T21:42:08.975 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.disable_auto_compactions: 0 2024-05-21T21:42:08.975 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-21T21:42:08.975 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-21T21:42:08.975 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-21T21:42:08.975 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-21T21:42:08.975 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-21T21:42:08.975 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-21T21:42:08.975 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-21T21:42:08.975 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-21T21:42:08.975 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-21T21:42:08.975 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-21T21:42:08.975 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-05-21T21:42:08.975 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.inplace_update_support: 0 2024-05-21T21:42:08.976 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-21T21:42:08.976 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-21T21:42:08.976 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-21T21:42:08.976 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-21T21:42:08.976 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.bloom_locality: 0 2024-05-21T21:42:08.976 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.max_successive_merges: 0 2024-05-21T21:42:08.976 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-21T21:42:08.976 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.paranoid_file_checks: 0 2024-05-21T21:42:08.976 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.force_consistency_checks: 1 2024-05-21T21:42:08.976 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.report_bg_io_stats: 0 2024-05-21T21:42:08.976 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.ttl: 2592000 2024-05-21T21:42:08.976 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-21T21:42:08.976 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-21T21:42:08.976 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-21T21:42:08.977 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.enable_blob_files: false 2024-05-21T21:42:08.977 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.min_blob_size: 0 2024-05-21T21:42:08.977 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.blob_file_size: 268435456 2024-05-21T21:42:08.977 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-21T21:42:08.977 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-21T21:42:08.977 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-21T21:42:08.977 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-21T21:42:08.977 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-21T21:42:08.977 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.blob_file_starting_level: 0 2024-05-21T21:42:08.977 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-21T21:42:08.977 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-05-21T21:42:08.977 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-05-21T21:42:08.977 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: a1e56212-a159-4b49-bacc-d43dfaae65c4 2024-05-21T21:42:08.978 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716327728621244, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-05-21T21:42:08.978 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-05-21T21:42:08.978 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716327728622704, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100580, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 328, "table_properties": {"data_size": 98465, "index_size": 307, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14264, "raw_average_key_size": 49, "raw_value_size": 90719, "raw_average_value_size": 316, "num_data_blocks": 12, "num_entries": 287, "num_filter_entries": 287, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1716327728, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a1e56212-a159-4b49-bacc-d43dfaae65c4", "db_session_id": "77S5N2I1JT2UZCU87CQ7", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-05-21T21:42:08.978 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716327728622963, "job": 1, "event": "recovery_finished"} 2024-05-21T21:42:08.978 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-05-21T21:42:08.978 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-21T21:42:08.978 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5585a0a78000 2024-05-21T21:42:08.978 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: DB pointer 0x5585a0a64000 2024-05-21T21:42:08.978 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-21T21:42:08.978 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-21T21:42:08.978 INFO:journalctl@ceph.mon.a.smithi151.stdout: ** DB Stats ** 2024-05-21T21:42:08.978 INFO:journalctl@ceph.mon.a.smithi151.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-21T21:42:08.978 INFO:journalctl@ceph.mon.a.smithi151.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-21T21:42:08.979 INFO:journalctl@ceph.mon.a.smithi151.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-21T21:42:08.979 INFO:journalctl@ceph.mon.a.smithi151.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-21T21:42:08.979 INFO:journalctl@ceph.mon.a.smithi151.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-21T21:42:08.979 INFO:journalctl@ceph.mon.a.smithi151.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-21T21:42:08.979 INFO:journalctl@ceph.mon.a.smithi151.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-21T21:42:08.979 INFO:journalctl@ceph.mon.a.smithi151.stdout: 2024-05-21T21:42:08.979 INFO:journalctl@ceph.mon.a.smithi151.stdout: ** Compaction Stats [default] ** 2024-05-21T21:42:08.979 INFO:journalctl@ceph.mon.a.smithi151.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-21T21:42:08.979 INFO:journalctl@ceph.mon.a.smithi151.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-21T21:42:08.979 INFO:journalctl@ceph.mon.a.smithi151.stdout: L0 2/0 100.08 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 106.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-21T21:42:08.979 INFO:journalctl@ceph.mon.a.smithi151.stdout: Sum 2/0 100.08 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 106.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-21T21:42:08.979 INFO:journalctl@ceph.mon.a.smithi151.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 106.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-21T21:42:08.979 INFO:journalctl@ceph.mon.a.smithi151.stdout: 2024-05-21T21:42:08.980 INFO:journalctl@ceph.mon.a.smithi151.stdout: ** Compaction Stats [default] ** 2024-05-21T21:42:08.980 INFO:journalctl@ceph.mon.a.smithi151.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-21T21:42:08.980 INFO:journalctl@ceph.mon.a.smithi151.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-21T21:42:08.980 INFO:journalctl@ceph.mon.a.smithi151.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 106.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-21T21:42:08.980 INFO:journalctl@ceph.mon.a.smithi151.stdout: 2024-05-21T21:42:08.980 INFO:journalctl@ceph.mon.a.smithi151.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-21T21:42:08.980 INFO:journalctl@ceph.mon.a.smithi151.stdout: 2024-05-21T21:42:08.980 INFO:journalctl@ceph.mon.a.smithi151.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-21T21:42:08.980 INFO:journalctl@ceph.mon.a.smithi151.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-21T21:42:08.980 INFO:journalctl@ceph.mon.a.smithi151.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-21T21:42:08.980 INFO:journalctl@ceph.mon.a.smithi151.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-21T21:42:08.980 INFO:journalctl@ceph.mon.a.smithi151.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-21T21:42:08.980 INFO:journalctl@ceph.mon.a.smithi151.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-21T21:42:08.981 INFO:journalctl@ceph.mon.a.smithi151.stdout: Cumulative compaction: 0.00 GB write, 13.51 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-21T21:42:08.981 INFO:journalctl@ceph.mon.a.smithi151.stdout: Interval compaction: 0.00 GB write, 13.51 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-21T21:42:08.981 INFO:journalctl@ceph.mon.a.smithi151.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-21T21:42:08.981 INFO:journalctl@ceph.mon.a.smithi151.stdout: Block cache BinnedLRUCache@0x5585a095b1f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.2e-05 secs_since: 0 2024-05-21T21:42:08.981 INFO:journalctl@ceph.mon.a.smithi151.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-05-21T21:42:08.981 INFO:journalctl@ceph.mon.a.smithi151.stdout: 2024-05-21T21:42:08.981 INFO:journalctl@ceph.mon.a.smithi151.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-21T21:42:08.981 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: starting mon.a rank 0 at public addrs [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] at bind addrs [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 2024-05-21T21:42:08.981 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: mon.a@-1(???) e1 preinit fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 2024-05-21T21:42:08.981 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: mon.a@-1(???).mds e1 new map 2024-05-21T21:42:08.981 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: mon.a@-1(???).mds e1 print_map 2024-05-21T21:42:08.981 INFO:journalctl@ceph.mon.a.smithi151.stdout: e1 2024-05-21T21:42:08.981 INFO:journalctl@ceph.mon.a.smithi151.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-21T21:42:08.982 INFO:journalctl@ceph.mon.a.smithi151.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-21T21:42:08.982 INFO:journalctl@ceph.mon.a.smithi151.stdout: legacy client fscid: -1 2024-05-21T21:42:08.982 INFO:journalctl@ceph.mon.a.smithi151.stdout: 2024-05-21T21:42:08.982 INFO:journalctl@ceph.mon.a.smithi151.stdout: No filesystems configured 2024-05-21T21:42:08.982 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-21T21:42:08.982 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-21T21:42:08.982 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-21T21:42:08.982 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-21T21:42:08.982 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-05-21T21:42:08.982 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 bash[31604]: 9cfe13f3b9612e0b654901b8e1252a003ced7683fbca5f515e236eedb8a1078e 2024-05-21T21:42:08.982 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 systemd[1]: Started Ceph mon.a for c8c7a2e6-17ba-11ef-bc9a-c7b262605968. 2024-05-21T21:42:08.982 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-21T21:42:08.982 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: monmap epoch 1 2024-05-21T21:42:08.983 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 2024-05-21T21:42:08.983 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: last_changed 2024-05-21T21:41:48.311542+0000 2024-05-21T21:42:08.983 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: created 2024-05-21T21:41:48.311542+0000 2024-05-21T21:42:08.983 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: min_mon_release 19 (squid) 2024-05-21T21:42:08.983 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: election_strategy: 1 2024-05-21T21:42:08.983 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: 0: [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] mon.a 2024-05-21T21:42:08.983 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: fsmap 2024-05-21T21:42:08.983 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: osdmap e1: 0 total, 0 up, 0 in 2024-05-21T21:42:08.983 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:08 smithi151 ceph-mon[31623]: mgrmap e1: no daemons active 2024-05-21T21:42:09.991 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:09.282+0000 7ffb35c00700 1 Processor -- start 2024-05-21T21:42:09.996 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:09.282+0000 7ffb35c00700 1 -- start start 2024-05-21T21:42:09.996 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:09.282+0000 7ffb35c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffb3010c910 0x7ffb3010cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:42:09.996 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:09.282+0000 7ffb35c00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ffb3010aea0 con 0x7ffb3010c910 2024-05-21T21:42:09.996 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:09.283+0000 7ffb2ea00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffb3010c910 0x7ffb3010cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:42:09.996 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:09.283+0000 7ffb2ea00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffb3010c910 0x7ffb3010cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:50070/0 (socket says 172.21.15.151:50070) 2024-05-21T21:42:09.996 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:09.283+0000 7ffb2ea00700 1 -- 172.21.15.151:0/4073060385 learned_addr learned my addr 172.21.15.151:0/4073060385 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:42:09.996 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:09.284+0000 7ffb2ea00700 1 -- 172.21.15.151:0/4073060385 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb3010b020 con 0x7ffb3010c910 2024-05-21T21:42:09.996 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:09.284+0000 7ffb2ea00700 1 --2- 172.21.15.151:0/4073060385 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffb3010c910 0x7ffb3010cd30 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ffb200098a0 tx=0x7ffb20009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eab10032ea47ea9d server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:42:09.996 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:09.285+0000 7ffb2e000700 1 -- 172.21.15.151:0/4073060385 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffb20004160 con 0x7ffb3010c910 2024-05-21T21:42:09.997 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:09.285+0000 7ffb2e000700 1 -- 172.21.15.151:0/4073060385 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ffb20004300 con 0x7ffb3010c910 2024-05-21T21:42:09.997 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:09.285+0000 7ffb2e000700 1 -- 172.21.15.151:0/4073060385 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffb20011af0 con 0x7ffb3010c910 2024-05-21T21:42:09.997 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:09.286+0000 7ffb35c00700 1 -- 172.21.15.151:0/4073060385 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffb3010c910 msgr2=0x7ffb3010cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:42:09.997 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:09.286+0000 7ffb35c00700 1 --2- 172.21.15.151:0/4073060385 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffb3010c910 0x7ffb3010cd30 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ffb200098a0 tx=0x7ffb20009bb0 comp rx=0 tx=0).stop 2024-05-21T21:42:09.997 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:09.286+0000 7ffb35c00700 1 -- 172.21.15.151:0/4073060385 shutdown_connections 2024-05-21T21:42:09.997 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:09.286+0000 7ffb35c00700 1 --2- 172.21.15.151:0/4073060385 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffb3010c910 0x7ffb3010cd30 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:42:09.997 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:09.286+0000 7ffb35c00700 1 -- 172.21.15.151:0/4073060385 >> 172.21.15.151:0/4073060385 conn(0x7ffb3007e890 msgr2=0x7ffb301094c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:42:09.997 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:09.286+0000 7ffb35c00700 1 -- 172.21.15.151:0/4073060385 shutdown_connections 2024-05-21T21:42:09.997 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:09.286+0000 7ffb35c00700 1 -- 172.21.15.151:0/4073060385 wait complete. 2024-05-21T21:42:09.997 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:09.287+0000 7ffb35c00700 1 Processor -- start 2024-05-21T21:42:09.997 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:09.287+0000 7ffb35c00700 1 -- start start 2024-05-21T21:42:09.997 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:09.288+0000 7ffb35c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffb3010c910 0x7ffb3019f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:42:09.997 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:09.288+0000 7ffb35c00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ffb3010f7e0 con 0x7ffb3010c910 2024-05-21T21:42:09.998 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:09.288+0000 7ffb2ea00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffb3010c910 0x7ffb3019f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:42:09.998 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:09.288+0000 7ffb2ea00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffb3010c910 0x7ffb3019f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:50076/0 (socket says 172.21.15.151:50076) 2024-05-21T21:42:09.998 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:09.288+0000 7ffb2ea00700 1 -- 172.21.15.151:0/2143891815 learned_addr learned my addr 172.21.15.151:0/2143891815 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:42:09.998 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:09.288+0000 7ffb2ea00700 1 -- 172.21.15.151:0/2143891815 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb3019f7e0 con 0x7ffb3010c910 2024-05-21T21:42:09.998 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:09.289+0000 7ffb2ea00700 1 --2- 172.21.15.151:0/2143891815 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffb3010c910 0x7ffb3019f2a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ffb20000c00 tx=0x7ffb20011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:42:09.998 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:09.289+0000 7ffb27400700 1 -- 172.21.15.151:0/2143891815 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffb20003b20 con 0x7ffb3010c910 2024-05-21T21:42:09.998 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:09.289+0000 7ffb27400700 1 -- 172.21.15.151:0/2143891815 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ffb2001a8c0 con 0x7ffb3010c910 2024-05-21T21:42:09.998 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:09.289+0000 7ffb27400700 1 -- 172.21.15.151:0/2143891815 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffb20024940 con 0x7ffb3010c910 2024-05-21T21:42:09.998 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:09.289+0000 7ffb35c00700 1 -- 172.21.15.151:0/2143891815 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb3019fa70 con 0x7ffb3010c910 2024-05-21T21:42:09.998 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:09.289+0000 7ffb35c00700 1 -- 172.21.15.151:0/2143891815 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb301a2760 con 0x7ffb3010c910 2024-05-21T21:42:09.998 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:09.290+0000 7ffb27400700 1 -- 172.21.15.151:0/2143891815 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ffb20022020 con 0x7ffb3010c910 2024-05-21T21:42:09.998 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:09.290+0000 7ffb27400700 1 -- 172.21.15.151:0/2143891815 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ffb200193c0 con 0x7ffb3010c910 2024-05-21T21:42:09.998 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:09.290+0000 7ffb35c00700 1 -- 172.21.15.151:0/2143891815 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb3010f960 con 0x7ffb3010c910 2024-05-21T21:42:09.999 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:09.293+0000 7ffb27400700 1 -- 172.21.15.151:0/2143891815 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7ffb2002dde0 con 0x7ffb3010c910 2024-05-21T21:42:09.999 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:09.367+0000 7ffb35c00700 1 -- 172.21.15.151:0/2143891815 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7ffb301a2bf0 con 0x7ffb3010c910 2024-05-21T21:42:09.999 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:09.370+0000 7ffb27400700 1 -- 172.21.15.151:0/2143891815 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7ffb2001aa70 con 0x7ffb3010c910 2024-05-21T21:42:09.999 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:09.371+0000 7ffb35c00700 1 -- 172.21.15.151:0/2143891815 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffb3010c910 msgr2=0x7ffb3019f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:42:09.999 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:09.371+0000 7ffb35c00700 1 --2- 172.21.15.151:0/2143891815 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffb3010c910 0x7ffb3019f2a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ffb20000c00 tx=0x7ffb20011b70 comp rx=0 tx=0).stop 2024-05-21T21:42:09.999 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:09.372+0000 7ffb35c00700 1 -- 172.21.15.151:0/2143891815 shutdown_connections 2024-05-21T21:42:09.999 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:09.372+0000 7ffb35c00700 1 --2- 172.21.15.151:0/2143891815 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffb3010c910 0x7ffb3019f2a0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:42:09.999 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:09.372+0000 7ffb35c00700 1 -- 172.21.15.151:0/2143891815 >> 172.21.15.151:0/2143891815 conn(0x7ffb3007e890 msgr2=0x7ffb30108be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:42:09.999 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:09.372+0000 7ffb35c00700 1 -- 172.21.15.151:0/2143891815 shutdown_connections 2024-05-21T21:42:09.999 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:09.372+0000 7ffb35c00700 1 -- 172.21.15.151:0/2143891815 wait complete. 2024-05-21T21:42:10.000 INFO:teuthology.orchestra.run.smithi151.stdout:Wrote config to /etc/ceph/ceph.conf 2024-05-21T21:42:10.023 INFO:teuthology.orchestra.run.smithi151.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-05-21T21:42:10.023 INFO:teuthology.orchestra.run.smithi151.stdout:Creating mgr... 2024-05-21T21:42:10.024 INFO:teuthology.orchestra.run.smithi151.stdout:Verifying port 0.0.0.0:9283 ... 2024-05-21T21:42:10.026 INFO:teuthology.orchestra.run.smithi151.stdout:Verifying port 0.0.0.0:8765 ... 2024-05-21T21:42:10.461 INFO:teuthology.orchestra.run.smithi151.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@mgr.x 2024-05-21T21:42:10.461 INFO:teuthology.orchestra.run.smithi151.stdout:systemctl: stderr Failed to reset failed state of unit ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@mgr.x.service: Unit ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@mgr.x.service not loaded. 2024-05-21T21:42:10.702 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:10 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/2143891815' entity='client.admin' 2024-05-21T21:42:10.713 INFO:teuthology.orchestra.run.smithi151.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968.target.wants/ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@.service. 2024-05-21T21:42:11.136 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:42:10 smithi151 systemd[1]: Starting Ceph mgr.x for c8c7a2e6-17ba-11ef-bc9a-c7b262605968... 2024-05-21T21:42:11.453 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:42:11 smithi151 podman[31837]: 2024-05-21 21:42:11.039450154 +0000 UTC m=+0.041485780 image pull d6bc8f31cbc1a7165ca248fe9ba0b06ea14b30f46e1ea33ced1a5238e247af1f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 2024-05-21T21:42:11.796 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:42:11 smithi151 podman[31837]: 2024-05-21 21:42:11.542999068 +0000 UTC m=+0.545034723 container create 297d546f035d9044c7db4aa70103dc4b1b279122d03b50a982c43f618746edff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea) 2024-05-21T21:42:11.910 INFO:teuthology.orchestra.run.smithi151.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-21T21:42:11.910 INFO:teuthology.orchestra.run.smithi151.stdout:systemctl: stdout disabled 2024-05-21T21:42:11.927 INFO:teuthology.orchestra.run.smithi151.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-21T21:42:11.927 INFO:teuthology.orchestra.run.smithi151.stdout:systemctl: stdout inactive 2024-05-21T21:42:11.927 INFO:teuthology.orchestra.run.smithi151.stdout:firewalld.service is not enabled 2024-05-21T21:42:11.927 INFO:teuthology.orchestra.run.smithi151.stdout:Not possible to enable service . firewalld.service is not available 2024-05-21T21:42:11.949 INFO:teuthology.orchestra.run.smithi151.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-21T21:42:11.949 INFO:teuthology.orchestra.run.smithi151.stdout:systemctl: stdout disabled 2024-05-21T21:42:11.967 INFO:teuthology.orchestra.run.smithi151.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-21T21:42:11.967 INFO:teuthology.orchestra.run.smithi151.stdout:systemctl: stdout inactive 2024-05-21T21:42:11.968 INFO:teuthology.orchestra.run.smithi151.stdout:firewalld.service is not enabled 2024-05-21T21:42:11.968 INFO:teuthology.orchestra.run.smithi151.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-05-21T21:42:11.968 INFO:teuthology.orchestra.run.smithi151.stdout:Waiting for mgr to start... 2024-05-21T21:42:11.968 INFO:teuthology.orchestra.run.smithi151.stdout:Waiting for mgr... 2024-05-21T21:42:12.046 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:42:11 smithi151 podman[31837]: 2024-05-21 21:42:11.796270086 +0000 UTC m=+0.798305754 container init 297d546f035d9044c7db4aa70103dc4b1b279122d03b50a982c43f618746edff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea) 2024-05-21T21:42:12.046 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:42:11 smithi151 podman[31837]: 2024-05-21 21:42:11.806285889 +0000 UTC m=+0.808321554 container start 297d546f035d9044c7db4aa70103dc4b1b279122d03b50a982c43f618746edff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-21T21:42:12.046 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:42:11 smithi151 bash[31837]: 297d546f035d9044c7db4aa70103dc4b1b279122d03b50a982c43f618746edff 2024-05-21T21:42:12.046 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:42:11 smithi151 systemd[1]: Started Ceph mgr.x for c8c7a2e6-17ba-11ef-bc9a-c7b262605968. 2024-05-21T21:42:12.301 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:42:12 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:42:12.045+0000 7f3598803200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-21T21:42:12.301 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:42:12 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:42:12.300+0000 7f3598803200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-21T21:42:12.560 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:42:12 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:42:12.433+0000 7f3598803200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-21T21:42:12.881 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:12 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/1431789673' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-21T21:42:13.203 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:42:12 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:42:12.880+0000 7f3598803200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-21T21:42:13.203 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:42:13 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:42:13.005+0000 7f3598803200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-21T21:42:13.338 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout 2024-05-21T21:42:13.339 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout { 2024-05-21T21:42:13.339 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "fsid": "c8c7a2e6-17ba-11ef-bc9a-c7b262605968", 2024-05-21T21:42:13.339 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "health": { 2024-05-21T21:42:13.339 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-21T21:42:13.343 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-21T21:42:13.343 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-21T21:42:13.343 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout }, 2024-05-21T21:42:13.343 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-21T21:42:13.343 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-21T21:42:13.343 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout 0 2024-05-21T21:42:13.344 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout ], 2024-05-21T21:42:13.344 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-21T21:42:13.344 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "a" 2024-05-21T21:42:13.344 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout ], 2024-05-21T21:42:13.344 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "quorum_age": 4, 2024-05-21T21:42:13.344 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-21T21:42:13.344 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-21T21:42:13.344 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-21T21:42:13.344 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-21T21:42:13.344 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout }, 2024-05-21T21:42:13.344 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-21T21:42:13.344 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-21T21:42:13.344 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-21T21:42:13.345 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-21T21:42:13.345 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-21T21:42:13.345 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-21T21:42:13.345 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-21T21:42:13.345 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-21T21:42:13.345 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout }, 2024-05-21T21:42:13.345 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-21T21:42:13.345 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-21T21:42:13.345 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-21T21:42:13.345 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-21T21:42:13.345 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-21T21:42:13.345 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-21T21:42:13.345 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-21T21:42:13.346 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-21T21:42:13.346 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-21T21:42:13.346 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout }, 2024-05-21T21:42:13.346 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-21T21:42:13.346 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-21T21:42:13.352 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-21T21:42:13.352 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-21T21:42:13.352 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout }, 2024-05-21T21:42:13.353 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-21T21:42:13.353 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-21T21:42:13.353 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-21T21:42:13.353 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-21T21:42:13.353 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-21T21:42:13.353 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-21T21:42:13.353 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "restful" 2024-05-21T21:42:13.353 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout ], 2024-05-21T21:42:13.353 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-21T21:42:13.353 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout }, 2024-05-21T21:42:13.353 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-21T21:42:13.353 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-21T21:42:13.353 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "modified": "2024-05-21T21:41:52.413408+0000", 2024-05-21T21:42:13.353 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-21T21:42:13.353 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout }, 2024-05-21T21:42:13.354 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-21T21:42:13.354 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout } 2024-05-21T21:42:13.354 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:12.559+0000 7f7211600700 1 Processor -- start 2024-05-21T21:42:13.354 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:12.559+0000 7f7211600700 1 -- start start 2024-05-21T21:42:13.354 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:12.559+0000 7f7211600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f720c1099b0 0x7f720c10bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:42:13.354 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:12.560+0000 7f7211600700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f720c07ce60 con 0x7f720c1099b0 2024-05-21T21:42:13.354 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:12.560+0000 7f720be00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f720c1099b0 0x7f720c10bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:42:13.354 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:12.560+0000 7f720be00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f720c1099b0 0x7f720c10bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:50104/0 (socket says 172.21.15.151:50104) 2024-05-21T21:42:13.354 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:12.560+0000 7f720be00700 1 -- 172.21.15.151:0/4002493384 learned_addr learned my addr 172.21.15.151:0/4002493384 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:42:13.354 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:12.561+0000 7f720be00700 1 -- 172.21.15.151:0/4002493384 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f720c10c2e0 con 0x7f720c1099b0 2024-05-21T21:42:13.354 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:12.561+0000 7f720be00700 1 --2- 172.21.15.151:0/4002493384 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f720c1099b0 0x7f720c10bda0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f71fc0098a0 tx=0x7f71fc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84a5883f0c96c50 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:42:13.354 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:12.561+0000 7f720aa00700 1 -- 172.21.15.151:0/4002493384 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f71fc004160 con 0x7f720c1099b0 2024-05-21T21:42:13.354 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:12.561+0000 7f720aa00700 1 -- 172.21.15.151:0/4002493384 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f71fc004300 con 0x7f720c1099b0 2024-05-21T21:42:13.355 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:12.561+0000 7f720aa00700 1 -- 172.21.15.151:0/4002493384 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f71fc011af0 con 0x7f720c1099b0 2024-05-21T21:42:13.355 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:12.562+0000 7f7211600700 1 -- 172.21.15.151:0/4002493384 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f720c1099b0 msgr2=0x7f720c10bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:42:13.355 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:12.562+0000 7f7211600700 1 --2- 172.21.15.151:0/4002493384 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f720c1099b0 0x7f720c10bda0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f71fc0098a0 tx=0x7f71fc009bb0 comp rx=0 tx=0).stop 2024-05-21T21:42:13.355 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:12.562+0000 7f7211600700 1 -- 172.21.15.151:0/4002493384 shutdown_connections 2024-05-21T21:42:13.355 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:12.562+0000 7f7211600700 1 --2- 172.21.15.151:0/4002493384 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f720c1099b0 0x7f720c10bda0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:42:13.355 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:12.562+0000 7f7211600700 1 -- 172.21.15.151:0/4002493384 >> 172.21.15.151:0/4002493384 conn(0x7f720c1033a0 msgr2=0x7f720c1057e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:42:13.355 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:12.563+0000 7f7211600700 1 -- 172.21.15.151:0/4002493384 shutdown_connections 2024-05-21T21:42:13.355 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:12.563+0000 7f7211600700 1 -- 172.21.15.151:0/4002493384 wait complete. 2024-05-21T21:42:13.355 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:12.564+0000 7f7211600700 1 Processor -- start 2024-05-21T21:42:13.355 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:12.564+0000 7f7211600700 1 -- start start 2024-05-21T21:42:13.355 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:12.564+0000 7f7211600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f720c1099b0 0x7f720c19cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:42:13.355 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:12.564+0000 7f7211600700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f720c10d230 con 0x7f720c1099b0 2024-05-21T21:42:13.355 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:12.565+0000 7f720be00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f720c1099b0 0x7f720c19cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:42:13.355 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:12.565+0000 7f720be00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f720c1099b0 0x7f720c19cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:50108/0 (socket says 172.21.15.151:50108) 2024-05-21T21:42:13.356 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:12.565+0000 7f720be00700 1 -- 172.21.15.151:0/1431789673 learned_addr learned my addr 172.21.15.151:0/1431789673 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:42:13.356 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:12.565+0000 7f720be00700 1 -- 172.21.15.151:0/1431789673 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f720c19d310 con 0x7f720c1099b0 2024-05-21T21:42:13.356 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:12.565+0000 7f720be00700 1 --2- 172.21.15.151:0/1431789673 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f720c1099b0 0x7f720c19cdd0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f71fc011ac0 tx=0x7f71fc011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:42:13.356 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:12.565+0000 7f7208c00700 1 -- 172.21.15.151:0/1431789673 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f71fc003970 con 0x7f720c1099b0 2024-05-21T21:42:13.356 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:12.565+0000 7f7208c00700 1 -- 172.21.15.151:0/1431789673 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f71fc003b10 con 0x7f720c1099b0 2024-05-21T21:42:13.356 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:12.565+0000 7f7211600700 1 -- 172.21.15.151:0/1431789673 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f720c19d5a0 con 0x7f720c1099b0 2024-05-21T21:42:13.356 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:12.566+0000 7f7208c00700 1 -- 172.21.15.151:0/1431789673 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f71fc024610 con 0x7f720c1099b0 2024-05-21T21:42:13.356 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:12.566+0000 7f7211600700 1 -- 172.21.15.151:0/1431789673 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f720c1a0290 con 0x7f720c1099b0 2024-05-21T21:42:13.356 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:12.566+0000 7f7208c00700 1 -- 172.21.15.151:0/1431789673 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f71fc024bd0 con 0x7f720c1099b0 2024-05-21T21:42:13.356 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:12.566+0000 7f7208c00700 1 -- 172.21.15.151:0/1431789673 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f71fc019700 con 0x7f720c1099b0 2024-05-21T21:42:13.356 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:12.567+0000 7f7211600700 1 -- 172.21.15.151:0/1431789673 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f720c06a6a0 con 0x7f720c1099b0 2024-05-21T21:42:13.356 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:12.570+0000 7f7208c00700 1 -- 172.21.15.151:0/1431789673 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f71fc043ec0 con 0x7f720c1099b0 2024-05-21T21:42:13.357 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:12.659+0000 7f7211600700 1 -- 172.21.15.151:0/1431789673 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f720c1a0460 con 0x7f720c1099b0 2024-05-21T21:42:13.357 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:12.660+0000 7f7208c00700 1 -- 172.21.15.151:0/1431789673 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f71fc032030 con 0x7f720c1099b0 2024-05-21T21:42:13.357 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:12.661+0000 7f7211600700 1 -- 172.21.15.151:0/1431789673 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f720c1099b0 msgr2=0x7f720c19cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:42:13.357 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:12.661+0000 7f7211600700 1 --2- 172.21.15.151:0/1431789673 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f720c1099b0 0x7f720c19cdd0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f71fc011ac0 tx=0x7f71fc011ba0 comp rx=0 tx=0).stop 2024-05-21T21:42:13.357 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:12.661+0000 7f7211600700 1 -- 172.21.15.151:0/1431789673 shutdown_connections 2024-05-21T21:42:13.357 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:12.661+0000 7f7211600700 1 --2- 172.21.15.151:0/1431789673 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f720c1099b0 0x7f720c19cdd0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:42:13.357 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:12.662+0000 7f7211600700 1 -- 172.21.15.151:0/1431789673 >> 172.21.15.151:0/1431789673 conn(0x7f720c1033a0 msgr2=0x7f720c103f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:42:13.357 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:12.662+0000 7f7211600700 1 -- 172.21.15.151:0/1431789673 shutdown_connections 2024-05-21T21:42:13.357 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:12.662+0000 7f7211600700 1 -- 172.21.15.151:0/1431789673 wait complete. 2024-05-21T21:42:13.357 INFO:teuthology.orchestra.run.smithi151.stdout:mgr not available, waiting (1/15)... 2024-05-21T21:42:14.953 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:42:14 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:42:14.617+0000 7f3598803200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-21T21:42:14.953 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:42:14 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:42:14.840+0000 7f3598803200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-21T21:42:15.440 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:42:15 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:42:15.053+0000 7f3598803200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-21T21:42:15.441 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:42:15 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:42:15.440+0000 7f3598803200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-21T21:42:15.702 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:42:15 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:42:15.561+0000 7f3598803200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-21T21:42:15.953 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:42:15 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:42:15.821+0000 7f3598803200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-21T21:42:16.452 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:16 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/702078601' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-21T21:42:16.453 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:42:16 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:42:16.082+0000 7f3598803200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-21T21:42:16.816 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout 2024-05-21T21:42:16.816 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout { 2024-05-21T21:42:16.817 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "fsid": "c8c7a2e6-17ba-11ef-bc9a-c7b262605968", 2024-05-21T21:42:16.817 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "health": { 2024-05-21T21:42:16.817 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-21T21:42:16.817 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-21T21:42:16.817 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-21T21:42:16.817 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout }, 2024-05-21T21:42:16.817 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-21T21:42:16.817 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-21T21:42:16.817 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout 0 2024-05-21T21:42:16.817 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout ], 2024-05-21T21:42:16.817 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-21T21:42:16.817 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "a" 2024-05-21T21:42:16.817 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout ], 2024-05-21T21:42:16.818 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "quorum_age": 7, 2024-05-21T21:42:16.818 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-21T21:42:16.818 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-21T21:42:16.818 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-21T21:42:16.818 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-21T21:42:16.818 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout }, 2024-05-21T21:42:16.818 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-21T21:42:16.818 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-21T21:42:16.818 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-21T21:42:16.818 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-21T21:42:16.818 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-21T21:42:16.818 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-21T21:42:16.819 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-21T21:42:16.819 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-21T21:42:16.819 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout }, 2024-05-21T21:42:16.819 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-21T21:42:16.819 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-21T21:42:16.819 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-21T21:42:16.819 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-21T21:42:16.819 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-21T21:42:16.819 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-21T21:42:16.819 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-21T21:42:16.819 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-21T21:42:16.819 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-21T21:42:16.819 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout }, 2024-05-21T21:42:16.820 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-21T21:42:16.820 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-21T21:42:16.820 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-21T21:42:16.820 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-21T21:42:16.820 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout }, 2024-05-21T21:42:16.820 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-21T21:42:16.820 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-21T21:42:16.820 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-21T21:42:16.820 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-21T21:42:16.820 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-21T21:42:16.820 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-21T21:42:16.820 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "restful" 2024-05-21T21:42:16.820 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout ], 2024-05-21T21:42:16.820 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-21T21:42:16.821 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout }, 2024-05-21T21:42:16.821 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-21T21:42:16.821 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-21T21:42:16.821 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "modified": "2024-05-21T21:41:52.413408+0000", 2024-05-21T21:42:16.821 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-21T21:42:16.821 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout }, 2024-05-21T21:42:16.821 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-21T21:42:16.821 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout } 2024-05-21T21:42:16.821 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:15.984+0000 7f3ba9000700 1 Processor -- start 2024-05-21T21:42:16.821 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:15.984+0000 7f3ba9000700 1 -- start start 2024-05-21T21:42:16.821 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:15.984+0000 7f3ba9000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3ba4109040 0x7f3ba4109460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:42:16.821 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:15.984+0000 7f3ba9000700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba4109a30 con 0x7f3ba4109040 2024-05-21T21:42:16.821 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:15.985+0000 7f3ba1e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3ba4109040 0x7f3ba4109460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:42:16.822 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:15.985+0000 7f3ba1e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3ba4109040 0x7f3ba4109460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:46484/0 (socket says 172.21.15.151:46484) 2024-05-21T21:42:16.822 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:15.985+0000 7f3ba1e00700 1 -- 172.21.15.151:0/3234067016 learned_addr learned my addr 172.21.15.151:0/3234067016 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:42:16.822 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:15.985+0000 7f3ba1e00700 1 -- 172.21.15.151:0/3234067016 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ba410a240 con 0x7f3ba4109040 2024-05-21T21:42:16.822 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:15.985+0000 7f3ba1e00700 1 --2- 172.21.15.151:0/3234067016 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3ba4109040 0x7f3ba4109460 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3b940098a0 tx=0x7f3b94009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4c613bffd5fabb5 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:42:16.822 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:15.986+0000 7f3ba0a00700 1 -- 172.21.15.151:0/3234067016 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3b94004160 con 0x7f3ba4109040 2024-05-21T21:42:16.822 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:15.986+0000 7f3ba0a00700 1 -- 172.21.15.151:0/3234067016 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f3b94004300 con 0x7f3ba4109040 2024-05-21T21:42:16.822 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:15.986+0000 7f3ba0a00700 1 -- 172.21.15.151:0/3234067016 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3b94011af0 con 0x7f3ba4109040 2024-05-21T21:42:16.822 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:15.986+0000 7f3ba9000700 1 -- 172.21.15.151:0/3234067016 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3ba4109040 msgr2=0x7f3ba4109460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:42:16.822 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:15.986+0000 7f3ba9000700 1 --2- 172.21.15.151:0/3234067016 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3ba4109040 0x7f3ba4109460 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3b940098a0 tx=0x7f3b94009bb0 comp rx=0 tx=0).stop 2024-05-21T21:42:16.822 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:15.987+0000 7f3ba9000700 1 -- 172.21.15.151:0/3234067016 shutdown_connections 2024-05-21T21:42:16.822 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:15.987+0000 7f3ba9000700 1 --2- 172.21.15.151:0/3234067016 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3ba4109040 0x7f3ba4109460 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:42:16.822 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:15.987+0000 7f3ba9000700 1 -- 172.21.15.151:0/3234067016 >> 172.21.15.151:0/3234067016 conn(0x7f3ba41045e0 msgr2=0x7f3ba4106a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:42:16.822 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:15.987+0000 7f3ba9000700 1 -- 172.21.15.151:0/3234067016 shutdown_connections 2024-05-21T21:42:16.823 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:15.987+0000 7f3ba9000700 1 -- 172.21.15.151:0/3234067016 wait complete. 2024-05-21T21:42:16.823 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:15.988+0000 7f3ba9000700 1 Processor -- start 2024-05-21T21:42:16.823 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:15.988+0000 7f3ba9000700 1 -- start start 2024-05-21T21:42:16.823 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:15.988+0000 7f3ba9000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3ba4109040 0x7f3ba4195fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:42:16.823 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:15.988+0000 7f3ba9000700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba410d5f0 con 0x7f3ba4109040 2024-05-21T21:42:16.823 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:15.988+0000 7f3ba1e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3ba4109040 0x7f3ba4195fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:42:16.823 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:15.988+0000 7f3ba1e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3ba4109040 0x7f3ba4195fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:46492/0 (socket says 172.21.15.151:46492) 2024-05-21T21:42:16.823 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:15.988+0000 7f3ba1e00700 1 -- 172.21.15.151:0/702078601 learned_addr learned my addr 172.21.15.151:0/702078601 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:42:16.823 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:15.989+0000 7f3ba1e00700 1 -- 172.21.15.151:0/702078601 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ba41964f0 con 0x7f3ba4109040 2024-05-21T21:42:16.823 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:15.989+0000 7f3ba1e00700 1 --2- 172.21.15.151:0/702078601 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3ba4109040 0x7f3ba4195fb0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3b940117b0 tx=0x7f3b94011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:42:16.823 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:15.989+0000 7f3b9aa00700 1 -- 172.21.15.151:0/702078601 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3b94003af0 con 0x7f3ba4109040 2024-05-21T21:42:16.823 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:15.989+0000 7f3b9aa00700 1 -- 172.21.15.151:0/702078601 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f3b94020070 con 0x7f3ba4109040 2024-05-21T21:42:16.824 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:15.989+0000 7f3b9aa00700 1 -- 172.21.15.151:0/702078601 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3b94019630 con 0x7f3ba4109040 2024-05-21T21:42:16.824 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:15.989+0000 7f3ba9000700 1 -- 172.21.15.151:0/702078601 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ba4196780 con 0x7f3ba4109040 2024-05-21T21:42:16.824 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:15.990+0000 7f3ba9000700 1 -- 172.21.15.151:0/702078601 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ba4192d40 con 0x7f3ba4109040 2024-05-21T21:42:16.824 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:15.990+0000 7f3b9aa00700 1 -- 172.21.15.151:0/702078601 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3b94019c30 con 0x7f3ba4109040 2024-05-21T21:42:16.824 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:15.990+0000 7f3b9aa00700 1 -- 172.21.15.151:0/702078601 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f3b9400b720 con 0x7f3ba4109040 2024-05-21T21:42:16.824 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:15.991+0000 7f3ba9000700 1 -- 172.21.15.151:0/702078601 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ba4196950 con 0x7f3ba4109040 2024-05-21T21:42:16.824 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:15.995+0000 7f3b9aa00700 1 -- 172.21.15.151:0/702078601 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f3ba4196950 con 0x7f3ba4109040 2024-05-21T21:42:16.824 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:16.073+0000 7f3ba9000700 1 -- 172.21.15.151:0/702078601 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f3ba4193330 con 0x7f3ba4109040 2024-05-21T21:42:16.824 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:16.074+0000 7f3b9aa00700 1 -- 172.21.15.151:0/702078601 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f3b940042a0 con 0x7f3ba4109040 2024-05-21T21:42:16.824 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:16.076+0000 7f3ba9000700 1 -- 172.21.15.151:0/702078601 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3ba4109040 msgr2=0x7f3ba4195fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:42:16.824 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:16.076+0000 7f3ba9000700 1 --2- 172.21.15.151:0/702078601 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3ba4109040 0x7f3ba4195fb0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3b940117b0 tx=0x7f3b94011b70 comp rx=0 tx=0).stop 2024-05-21T21:42:16.824 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:16.076+0000 7f3ba9000700 1 -- 172.21.15.151:0/702078601 shutdown_connections 2024-05-21T21:42:16.824 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:16.076+0000 7f3ba9000700 1 --2- 172.21.15.151:0/702078601 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3ba4109040 0x7f3ba4195fb0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:42:16.825 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:16.076+0000 7f3ba9000700 1 -- 172.21.15.151:0/702078601 >> 172.21.15.151:0/702078601 conn(0x7f3ba41045e0 msgr2=0x7f3ba4105130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:42:16.825 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:16.076+0000 7f3ba9000700 1 -- 172.21.15.151:0/702078601 shutdown_connections 2024-05-21T21:42:16.825 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:16.076+0000 7f3ba9000700 1 -- 172.21.15.151:0/702078601 wait complete. 2024-05-21T21:42:16.825 INFO:teuthology.orchestra.run.smithi151.stdout:mgr not available, waiting (2/15)... 2024-05-21T21:42:17.185 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:42:16 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:42:16.903+0000 7f3598803200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-21T21:42:17.453 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:42:17 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:42:17.184+0000 7f3598803200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-21T21:42:18.203 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:42:17 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:42:17.745+0000 7f3598803200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-21T21:42:18.203 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:42:17 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:42:17.873+0000 7f3598803200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-21T21:42:18.703 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:42:18 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:42:18.338+0000 7f3598803200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-21T21:42:19.099 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:42:18 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:42:18.795+0000 7f3598803200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-21T21:42:19.100 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:42:18 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:42:18.921+0000 7f3598803200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-21T21:42:19.362 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:42:19 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:42:19.257+0000 7f3598803200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-21T21:42:19.702 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:19 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/1934585977' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-21T21:42:19.703 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:42:19 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:42:19.444+0000 7f3598803200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-21T21:42:19.703 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:42:19 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:42:19.576+0000 7f3598803200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-21T21:42:20.087 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:42:19 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:42:19.986+0000 7f3598803200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-21T21:42:20.095 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout 2024-05-21T21:42:20.095 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout { 2024-05-21T21:42:20.096 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "fsid": "c8c7a2e6-17ba-11ef-bc9a-c7b262605968", 2024-05-21T21:42:20.096 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "health": { 2024-05-21T21:42:20.096 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-21T21:42:20.096 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-21T21:42:20.096 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-21T21:42:20.096 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout }, 2024-05-21T21:42:20.096 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-21T21:42:20.096 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-21T21:42:20.096 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout 0 2024-05-21T21:42:20.096 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout ], 2024-05-21T21:42:20.096 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-21T21:42:20.096 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "a" 2024-05-21T21:42:20.096 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout ], 2024-05-21T21:42:20.096 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-05-21T21:42:20.097 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-21T21:42:20.097 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-21T21:42:20.097 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-21T21:42:20.097 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-21T21:42:20.097 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout }, 2024-05-21T21:42:20.097 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-21T21:42:20.097 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-21T21:42:20.097 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-21T21:42:20.097 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-21T21:42:20.105 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-21T21:42:20.105 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-21T21:42:20.105 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-21T21:42:20.105 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-21T21:42:20.105 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout }, 2024-05-21T21:42:20.105 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-21T21:42:20.105 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-21T21:42:20.105 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-21T21:42:20.105 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-21T21:42:20.106 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-21T21:42:20.106 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-21T21:42:20.106 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-21T21:42:20.106 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-21T21:42:20.106 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-21T21:42:20.106 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout }, 2024-05-21T21:42:20.106 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-21T21:42:20.106 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-21T21:42:20.106 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-21T21:42:20.106 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-21T21:42:20.106 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout }, 2024-05-21T21:42:20.106 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-21T21:42:20.107 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-21T21:42:20.107 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-21T21:42:20.107 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-21T21:42:20.107 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-21T21:42:20.107 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-21T21:42:20.107 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "restful" 2024-05-21T21:42:20.107 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout ], 2024-05-21T21:42:20.107 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-21T21:42:20.107 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout }, 2024-05-21T21:42:20.107 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-21T21:42:20.107 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-21T21:42:20.107 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "modified": "2024-05-21T21:41:52.413408+0000", 2024-05-21T21:42:20.107 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-21T21:42:20.108 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout }, 2024-05-21T21:42:20.108 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-21T21:42:20.108 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout } 2024-05-21T21:42:20.108 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:19.362+0000 7f54cf600700 1 Processor -- start 2024-05-21T21:42:20.108 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:19.362+0000 7f54cf600700 1 -- start start 2024-05-21T21:42:20.108 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:19.362+0000 7f54cf600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f54c8109050 0x7f54c8109470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:42:20.108 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:19.362+0000 7f54cf600700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f54c8109a40 con 0x7f54c8109050 2024-05-21T21:42:20.108 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:19.363+0000 7f54ccc00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f54c8109050 0x7f54c8109470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:42:20.108 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:19.363+0000 7f54ccc00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f54c8109050 0x7f54c8109470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:46500/0 (socket says 172.21.15.151:46500) 2024-05-21T21:42:20.108 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:19.363+0000 7f54ccc00700 1 -- 172.21.15.151:0/2354822249 learned_addr learned my addr 172.21.15.151:0/2354822249 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:42:20.108 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:19.364+0000 7f54ccc00700 1 -- 172.21.15.151:0/2354822249 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54c810a250 con 0x7f54c8109050 2024-05-21T21:42:20.108 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:19.364+0000 7f54ccc00700 1 --2- 172.21.15.151:0/2354822249 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f54c8109050 0x7f54c8109470 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f54bc0098a0 tx=0x7f54bc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4119a926f332047 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:42:20.108 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:19.364+0000 7f54c7400700 1 -- 172.21.15.151:0/2354822249 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f54bc004160 con 0x7f54c8109050 2024-05-21T21:42:20.109 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:19.364+0000 7f54c7400700 1 -- 172.21.15.151:0/2354822249 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f54bc004300 con 0x7f54c8109050 2024-05-21T21:42:20.109 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:19.364+0000 7f54c7400700 1 -- 172.21.15.151:0/2354822249 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f54bc011af0 con 0x7f54c8109050 2024-05-21T21:42:20.109 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:19.365+0000 7f54cf600700 1 -- 172.21.15.151:0/2354822249 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f54c8109050 msgr2=0x7f54c8109470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:42:20.109 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:19.365+0000 7f54cf600700 1 --2- 172.21.15.151:0/2354822249 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f54c8109050 0x7f54c8109470 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f54bc0098a0 tx=0x7f54bc009bb0 comp rx=0 tx=0).stop 2024-05-21T21:42:20.109 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:19.365+0000 7f54cf600700 1 -- 172.21.15.151:0/2354822249 shutdown_connections 2024-05-21T21:42:20.109 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:19.365+0000 7f54cf600700 1 --2- 172.21.15.151:0/2354822249 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f54c8109050 0x7f54c8109470 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:42:20.109 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:19.365+0000 7f54cf600700 1 -- 172.21.15.151:0/2354822249 >> 172.21.15.151:0/2354822249 conn(0x7f54c8104630 msgr2=0x7f54c8106a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:42:20.109 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:19.365+0000 7f54cf600700 1 -- 172.21.15.151:0/2354822249 shutdown_connections 2024-05-21T21:42:20.109 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:19.366+0000 7f54cf600700 1 -- 172.21.15.151:0/2354822249 wait complete. 2024-05-21T21:42:20.109 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:19.367+0000 7f54cf600700 1 Processor -- start 2024-05-21T21:42:20.109 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:19.367+0000 7f54cf600700 1 -- start start 2024-05-21T21:42:20.109 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:19.367+0000 7f54cf600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f54c8109050 0x7f54c819f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:42:20.109 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:19.367+0000 7f54cf600700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f54c810d600 con 0x7f54c8109050 2024-05-21T21:42:20.110 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:19.367+0000 7f54ccc00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f54c8109050 0x7f54c819f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:42:20.110 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:19.367+0000 7f54ccc00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f54c8109050 0x7f54c819f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:46510/0 (socket says 172.21.15.151:46510) 2024-05-21T21:42:20.110 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:19.367+0000 7f54ccc00700 1 -- 172.21.15.151:0/1934585977 learned_addr learned my addr 172.21.15.151:0/1934585977 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:42:20.110 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:19.368+0000 7f54ccc00700 1 -- 172.21.15.151:0/1934585977 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54c819f740 con 0x7f54c8109050 2024-05-21T21:42:20.110 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:19.368+0000 7f54ccc00700 1 --2- 172.21.15.151:0/1934585977 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f54c8109050 0x7f54c819f200 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f54bc0117b0 tx=0x7f54bc011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:42:20.110 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:19.368+0000 7f54c5600700 1 -- 172.21.15.151:0/1934585977 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f54bc003af0 con 0x7f54c8109050 2024-05-21T21:42:20.110 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:19.368+0000 7f54c5600700 1 -- 172.21.15.151:0/1934585977 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f54bc020070 con 0x7f54c8109050 2024-05-21T21:42:20.110 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:19.368+0000 7f54c5600700 1 -- 172.21.15.151:0/1934585977 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f54bc019630 con 0x7f54c8109050 2024-05-21T21:42:20.110 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:19.368+0000 7f54cf600700 1 -- 172.21.15.151:0/1934585977 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54c819f9d0 con 0x7f54c8109050 2024-05-21T21:42:20.110 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:19.368+0000 7f54cf600700 1 -- 172.21.15.151:0/1934585977 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54c81a26c0 con 0x7f54c8109050 2024-05-21T21:42:20.110 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:19.369+0000 7f54c5600700 1 -- 172.21.15.151:0/1934585977 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f54bc019c30 con 0x7f54c8109050 2024-05-21T21:42:20.110 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:19.369+0000 7f54c5600700 1 -- 172.21.15.151:0/1934585977 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f54bc00b720 con 0x7f54c8109050 2024-05-21T21:42:20.111 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:19.370+0000 7f54cf600700 1 -- 172.21.15.151:0/1934585977 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54c8064ba0 con 0x7f54c8109050 2024-05-21T21:42:20.111 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:19.372+0000 7f54c5600700 1 -- 172.21.15.151:0/1934585977 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f54bc01a9c0 con 0x7f54c8109050 2024-05-21T21:42:20.111 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:19.451+0000 7f54cf600700 1 -- 172.21.15.151:0/1934585977 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f54c81a2c00 con 0x7f54c8109050 2024-05-21T21:42:20.111 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:19.452+0000 7f54c5600700 1 -- 172.21.15.151:0/1934585977 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1240 (secure 0 0 0) 0x7f54bc01a430 con 0x7f54c8109050 2024-05-21T21:42:20.111 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:19.454+0000 7f54cf600700 1 -- 172.21.15.151:0/1934585977 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f54c8109050 msgr2=0x7f54c819f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:42:20.111 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:19.454+0000 7f54cf600700 1 --2- 172.21.15.151:0/1934585977 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f54c8109050 0x7f54c819f200 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f54bc0117b0 tx=0x7f54bc011b70 comp rx=0 tx=0).stop 2024-05-21T21:42:20.111 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:19.454+0000 7f54cf600700 1 -- 172.21.15.151:0/1934585977 shutdown_connections 2024-05-21T21:42:20.111 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:19.454+0000 7f54cf600700 1 --2- 172.21.15.151:0/1934585977 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f54c8109050 0x7f54c819f200 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:42:20.111 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:19.454+0000 7f54cf600700 1 -- 172.21.15.151:0/1934585977 >> 172.21.15.151:0/1934585977 conn(0x7f54c8104630 msgr2=0x7f54c8105120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:42:20.111 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:19.454+0000 7f54cf600700 1 -- 172.21.15.151:0/1934585977 shutdown_connections 2024-05-21T21:42:20.111 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:19.454+0000 7f54cf600700 1 -- 172.21.15.151:0/1934585977 wait complete. 2024-05-21T21:42:20.111 INFO:teuthology.orchestra.run.smithi151.stdout:mgr not available, waiting (3/15)... 2024-05-21T21:42:20.453 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:42:20 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:42:20.118+0000 7f3598803200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-21T21:42:21.163 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:42:20 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:42:20.878+0000 7f3598803200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-21T21:42:21.164 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:42:21 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:42:21.163+0000 7f3598803200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-21T21:42:21.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:21 smithi151 ceph-mon[31623]: Activating manager daemon x 2024-05-21T21:42:21.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:21 smithi151 ceph-mon[31623]: mgrmap e2: x(active, starting, since 0.0123923s) 2024-05-21T21:42:21.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:21 smithi151 ceph-mon[31623]: from='mgr.14100 172.21.15.151:0/2091242918' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-21T21:42:21.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:21 smithi151 ceph-mon[31623]: from='mgr.14100 172.21.15.151:0/2091242918' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-21T21:42:21.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:21 smithi151 ceph-mon[31623]: from='mgr.14100 172.21.15.151:0/2091242918' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-21T21:42:21.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:21 smithi151 ceph-mon[31623]: from='mgr.14100 172.21.15.151:0/2091242918' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-21T21:42:21.454 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:21 smithi151 ceph-mon[31623]: from='mgr.14100 172.21.15.151:0/2091242918' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-21T21:42:21.454 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:21 smithi151 ceph-mon[31623]: from='mgr.14100 172.21.15.151:0/2091242918' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-21T21:42:21.454 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:21 smithi151 ceph-mon[31623]: from='mgr.14100 172.21.15.151:0/2091242918' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-21T21:42:21.454 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:21 smithi151 ceph-mon[31623]: from='mgr.14100 172.21.15.151:0/2091242918' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-21T21:42:21.454 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:21 smithi151 ceph-mon[31623]: Manager daemon x is now available 2024-05-21T21:42:21.454 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:21 smithi151 ceph-mon[31623]: from='mgr.14100 172.21.15.151:0/2091242918' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-21T21:42:22.489 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:22 smithi151 ceph-mon[31623]: from='mgr.14100 172.21.15.151:0/2091242918' entity='mgr.x' 2024-05-21T21:42:22.489 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:22 smithi151 ceph-mon[31623]: from='mgr.14100 172.21.15.151:0/2091242918' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-21T21:42:22.489 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:22 smithi151 ceph-mon[31623]: from='mgr.14100 172.21.15.151:0/2091242918' entity='mgr.x' 2024-05-21T21:42:22.489 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:22 smithi151 ceph-mon[31623]: from='mgr.14100 172.21.15.151:0/2091242918' entity='mgr.x' 2024-05-21T21:42:22.489 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:22 smithi151 ceph-mon[31623]: mgrmap e3: x(active, since 1.01362s) 2024-05-21T21:42:23.447 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:23 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/635850280' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-21T21:42:23.758 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout 2024-05-21T21:42:23.758 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout { 2024-05-21T21:42:23.758 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "fsid": "c8c7a2e6-17ba-11ef-bc9a-c7b262605968", 2024-05-21T21:42:23.758 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "health": { 2024-05-21T21:42:23.758 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-21T21:42:23.758 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-21T21:42:23.758 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-21T21:42:23.758 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout }, 2024-05-21T21:42:23.759 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-21T21:42:23.759 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-21T21:42:23.759 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout 0 2024-05-21T21:42:23.759 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout ], 2024-05-21T21:42:23.759 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-21T21:42:23.759 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "a" 2024-05-21T21:42:23.759 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout ], 2024-05-21T21:42:23.759 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-05-21T21:42:23.760 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-21T21:42:23.760 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-21T21:42:23.760 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-21T21:42:23.760 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-21T21:42:23.761 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout }, 2024-05-21T21:42:23.761 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-21T21:42:23.761 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-21T21:42:23.761 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-21T21:42:23.761 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-21T21:42:23.761 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-21T21:42:23.771 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-21T21:42:23.771 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-21T21:42:23.771 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-21T21:42:23.771 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout }, 2024-05-21T21:42:23.771 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-21T21:42:23.771 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-21T21:42:23.771 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-21T21:42:23.772 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-21T21:42:23.772 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-21T21:42:23.772 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-21T21:42:23.772 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-21T21:42:23.772 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-21T21:42:23.772 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-21T21:42:23.772 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout }, 2024-05-21T21:42:23.772 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-21T21:42:23.772 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-21T21:42:23.772 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-21T21:42:23.772 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-21T21:42:23.772 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout }, 2024-05-21T21:42:23.772 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-21T21:42:23.773 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-21T21:42:23.773 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-21T21:42:23.773 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-21T21:42:23.773 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-21T21:42:23.773 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-21T21:42:23.773 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "restful" 2024-05-21T21:42:23.773 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout ], 2024-05-21T21:42:23.773 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-21T21:42:23.773 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout }, 2024-05-21T21:42:23.773 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-21T21:42:23.773 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-21T21:42:23.773 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "modified": "2024-05-21T21:41:52.413408+0000", 2024-05-21T21:42:23.773 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-21T21:42:23.773 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout }, 2024-05-21T21:42:23.774 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-21T21:42:23.774 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout } 2024-05-21T21:42:23.774 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:22.746+0000 7f4e97600700 1 Processor -- start 2024-05-21T21:42:23.774 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:22.747+0000 7f4e97600700 1 -- start start 2024-05-21T21:42:23.774 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:22.747+0000 7f4e97600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4e90109b00 0x7f4e9010bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:42:23.774 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:22.747+0000 7f4e97600700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4e9007cea0 con 0x7f4e90109b00 2024-05-21T21:42:23.774 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:22.748+0000 7f4e94c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4e90109b00 0x7f4e9010bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:42:23.774 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:22.748+0000 7f4e94c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4e90109b00 0x7f4e9010bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:46588/0 (socket says 172.21.15.151:46588) 2024-05-21T21:42:23.774 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:22.748+0000 7f4e94c00700 1 -- 172.21.15.151:0/996841507 learned_addr learned my addr 172.21.15.151:0/996841507 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:42:23.774 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:22.748+0000 7f4e94c00700 1 -- 172.21.15.151:0/996841507 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e9010c430 con 0x7f4e90109b00 2024-05-21T21:42:23.774 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:22.749+0000 7f4e94c00700 1 --2- 172.21.15.151:0/996841507 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4e90109b00 0x7f4e9010bef0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4e840098a0 tx=0x7f4e84009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51b0c70c12f607dc server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:42:23.774 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:22.749+0000 7f4e8f400700 1 -- 172.21.15.151:0/996841507 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e84004160 con 0x7f4e90109b00 2024-05-21T21:42:23.775 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:22.749+0000 7f4e8f400700 1 -- 172.21.15.151:0/996841507 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4e84004300 con 0x7f4e90109b00 2024-05-21T21:42:23.775 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:22.749+0000 7f4e8f400700 1 -- 172.21.15.151:0/996841507 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e84011af0 con 0x7f4e90109b00 2024-05-21T21:42:23.775 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:22.750+0000 7f4e97600700 1 -- 172.21.15.151:0/996841507 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4e90109b00 msgr2=0x7f4e9010bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:42:23.775 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:22.750+0000 7f4e97600700 1 --2- 172.21.15.151:0/996841507 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4e90109b00 0x7f4e9010bef0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4e840098a0 tx=0x7f4e84009bb0 comp rx=0 tx=0).stop 2024-05-21T21:42:23.775 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:22.750+0000 7f4e97600700 1 -- 172.21.15.151:0/996841507 shutdown_connections 2024-05-21T21:42:23.775 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:22.750+0000 7f4e97600700 1 --2- 172.21.15.151:0/996841507 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4e90109b00 0x7f4e9010bef0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:42:23.775 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:22.750+0000 7f4e97600700 1 -- 172.21.15.151:0/996841507 >> 172.21.15.151:0/996841507 conn(0x7f4e90103570 msgr2=0x7f4e90105990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:42:23.775 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:22.750+0000 7f4e97600700 1 -- 172.21.15.151:0/996841507 shutdown_connections 2024-05-21T21:42:23.775 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:22.750+0000 7f4e97600700 1 -- 172.21.15.151:0/996841507 wait complete. 2024-05-21T21:42:23.775 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:22.751+0000 7f4e97600700 1 Processor -- start 2024-05-21T21:42:23.775 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:22.751+0000 7f4e97600700 1 -- start start 2024-05-21T21:42:23.775 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:22.752+0000 7f4e97600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4e90109b00 0x7f4e9019f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:42:23.776 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:22.752+0000 7f4e97600700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4e9010d400 con 0x7f4e90109b00 2024-05-21T21:42:23.776 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:22.752+0000 7f4e94c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4e90109b00 0x7f4e9019f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:42:23.776 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:22.752+0000 7f4e94c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4e90109b00 0x7f4e9019f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:46592/0 (socket says 172.21.15.151:46592) 2024-05-21T21:42:23.776 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:22.752+0000 7f4e94c00700 1 -- 172.21.15.151:0/635850280 learned_addr learned my addr 172.21.15.151:0/635850280 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:42:23.776 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:22.752+0000 7f4e94c00700 1 -- 172.21.15.151:0/635850280 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e9019f730 con 0x7f4e90109b00 2024-05-21T21:42:23.776 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:22.753+0000 7f4e94c00700 1 --2- 172.21.15.151:0/635850280 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4e90109b00 0x7f4e9019f1f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4e84011ac0 tx=0x7f4e84011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:42:23.776 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:22.753+0000 7f4e8d600700 1 -- 172.21.15.151:0/635850280 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e84003970 con 0x7f4e90109b00 2024-05-21T21:42:23.776 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:22.753+0000 7f4e8d600700 1 -- 172.21.15.151:0/635850280 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4e84003b10 con 0x7f4e90109b00 2024-05-21T21:42:23.776 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:22.753+0000 7f4e97600700 1 -- 172.21.15.151:0/635850280 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e9019f9c0 con 0x7f4e90109b00 2024-05-21T21:42:23.776 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:22.753+0000 7f4e8d600700 1 -- 172.21.15.151:0/635850280 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e84024610 con 0x7f4e90109b00 2024-05-21T21:42:23.776 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:22.753+0000 7f4e97600700 1 -- 172.21.15.151:0/635850280 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e901a26b0 con 0x7f4e90109b00 2024-05-21T21:42:23.776 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:22.754+0000 7f4e97600700 1 -- 172.21.15.151:0/635850280 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e90064ba0 con 0x7f4e90109b00 2024-05-21T21:42:23.776 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:22.754+0000 7f4e8d600700 1 -- 172.21.15.151:0/635850280 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 3) ==== 50105+0+0 (secure 0 0 0) 0x7f4e8401a430 con 0x7f4e90109b00 2024-05-21T21:42:23.777 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:22.755+0000 7f4e8d600700 1 --2- 172.21.15.151:0/635850280 >> [v2:172.21.15.151:6800/1128885292,v1:172.21.15.151:6801/1128885292] conn(0x7f4e7803e300 0x7f4e780407c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:42:23.777 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:22.755+0000 7f4e8d600700 1 -- 172.21.15.151:0/635850280 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f4e840535e0 con 0x7f4e90109b00 2024-05-21T21:42:23.777 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:22.755+0000 7f4e8fe00700 1 --2- 172.21.15.151:0/635850280 >> [v2:172.21.15.151:6800/1128885292,v1:172.21.15.151:6801/1128885292] conn(0x7f4e7803e300 0x7f4e780407c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:42:23.777 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:22.756+0000 7f4e8fe00700 1 --2- 172.21.15.151:0/635850280 >> [v2:172.21.15.151:6800/1128885292,v1:172.21.15.151:6801/1128885292] conn(0x7f4e7803e300 0x7f4e780407c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c006fd0 tx=0x7f4e7c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:42:23.777 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:22.761+0000 7f4e8d600700 1 -- 172.21.15.151:0/635850280 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4e8400b3e0 con 0x7f4e90109b00 2024-05-21T21:42:23.777 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:23.083+0000 7f4e97600700 1 -- 172.21.15.151:0/635850280 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f4e901a2880 con 0x7f4e90109b00 2024-05-21T21:42:23.777 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:23.084+0000 7f4e8d600700 1 -- 172.21.15.151:0/635850280 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f4e901a2880 con 0x7f4e90109b00 2024-05-21T21:42:23.777 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:23.089+0000 7f4e97600700 1 -- 172.21.15.151:0/635850280 >> [v2:172.21.15.151:6800/1128885292,v1:172.21.15.151:6801/1128885292] conn(0x7f4e7803e300 msgr2=0x7f4e780407c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:42:23.777 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:23.089+0000 7f4e97600700 1 --2- 172.21.15.151:0/635850280 >> [v2:172.21.15.151:6800/1128885292,v1:172.21.15.151:6801/1128885292] conn(0x7f4e7803e300 0x7f4e780407c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c006fd0 tx=0x7f4e7c006e40 comp rx=0 tx=0).stop 2024-05-21T21:42:23.777 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:23.089+0000 7f4e97600700 1 -- 172.21.15.151:0/635850280 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4e90109b00 msgr2=0x7f4e9019f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:42:23.777 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:23.089+0000 7f4e97600700 1 --2- 172.21.15.151:0/635850280 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4e90109b00 0x7f4e9019f1f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4e84011ac0 tx=0x7f4e84011ba0 comp rx=0 tx=0).stop 2024-05-21T21:42:23.777 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:23.089+0000 7f4e97600700 1 -- 172.21.15.151:0/635850280 shutdown_connections 2024-05-21T21:42:23.778 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:23.089+0000 7f4e97600700 1 --2- 172.21.15.151:0/635850280 >> [v2:172.21.15.151:6800/1128885292,v1:172.21.15.151:6801/1128885292] conn(0x7f4e7803e300 0x7f4e780407c0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:42:23.778 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:23.089+0000 7f4e97600700 1 --2- 172.21.15.151:0/635850280 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4e90109b00 0x7f4e9019f1f0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:42:23.778 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:23.089+0000 7f4e97600700 1 -- 172.21.15.151:0/635850280 >> 172.21.15.151:0/635850280 conn(0x7f4e90103570 msgr2=0x7f4e90104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:42:23.778 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:23.090+0000 7f4e97600700 1 -- 172.21.15.151:0/635850280 shutdown_connections 2024-05-21T21:42:23.778 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:23.090+0000 7f4e97600700 1 -- 172.21.15.151:0/635850280 wait complete. 2024-05-21T21:42:23.778 INFO:teuthology.orchestra.run.smithi151.stdout:mgr is available 2024-05-21T21:42:24.304 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:24 smithi151 ceph-mon[31623]: mgrmap e4: x(active, since 2s) 2024-05-21T21:42:25.288 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout 2024-05-21T21:42:25.288 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout [global] 2024-05-21T21:42:25.288 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout fsid = c8c7a2e6-17ba-11ef-bc9a-c7b262605968 2024-05-21T21:42:25.288 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-05-21T21:42:25.288 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-21T21:42:25.288 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-21T21:42:25.288 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-21T21:42:25.288 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-21T21:42:25.288 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout 2024-05-21T21:42:25.288 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-21T21:42:25.288 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-21T21:42:25.289 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout 2024-05-21T21:42:25.289 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout [osd] 2024-05-21T21:42:25.289 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-05-21T21:42:25.289 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-21T21:42:25.289 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-21T21:42:25.289 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-21T21:42:25.289 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-21T21:42:25.289 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-21T21:42:25.289 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:24.368+0000 7f717f200700 1 Processor -- start 2024-05-21T21:42:25.289 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:24.368+0000 7f717f200700 1 -- start start 2024-05-21T21:42:25.289 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:24.369+0000 7f717f200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7180108e40 0x7f7180109260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:42:25.289 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:24.369+0000 7f717f200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f71801097a0 con 0x7f7180108e40 2024-05-21T21:42:25.289 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:24.369+0000 7f717de00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7180108e40 0x7f7180109260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:42:25.290 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:24.369+0000 7f717de00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7180108e40 0x7f7180109260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:46606/0 (socket says 172.21.15.151:46606) 2024-05-21T21:42:25.290 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:24.369+0000 7f717de00700 1 -- 172.21.15.151:0/3022503535 learned_addr learned my addr 172.21.15.151:0/3022503535 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:42:25.290 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:24.370+0000 7f717de00700 1 -- 172.21.15.151:0/3022503535 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7180109fa0 con 0x7f7180108e40 2024-05-21T21:42:25.290 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:24.370+0000 7f717de00700 1 --2- 172.21.15.151:0/3022503535 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7180108e40 0x7f7180109260 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f71700098a0 tx=0x7f7170009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec3acae8ded7a6cd server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:42:25.290 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:24.370+0000 7f717ca00700 1 -- 172.21.15.151:0/3022503535 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7170004160 con 0x7f7180108e40 2024-05-21T21:42:25.290 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:24.370+0000 7f717ca00700 1 -- 172.21.15.151:0/3022503535 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7170004300 con 0x7f7180108e40 2024-05-21T21:42:25.290 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:24.370+0000 7f717ca00700 1 -- 172.21.15.151:0/3022503535 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7170011af0 con 0x7f7180108e40 2024-05-21T21:42:25.290 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:24.371+0000 7f717f200700 1 -- 172.21.15.151:0/3022503535 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7180108e40 msgr2=0x7f7180109260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:42:25.290 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:24.371+0000 7f717f200700 1 --2- 172.21.15.151:0/3022503535 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7180108e40 0x7f7180109260 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f71700098a0 tx=0x7f7170009bb0 comp rx=0 tx=0).stop 2024-05-21T21:42:25.290 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:24.371+0000 7f717f200700 1 -- 172.21.15.151:0/3022503535 shutdown_connections 2024-05-21T21:42:25.290 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:24.371+0000 7f717f200700 1 --2- 172.21.15.151:0/3022503535 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7180108e40 0x7f7180109260 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:42:25.291 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:24.371+0000 7f717f200700 1 -- 172.21.15.151:0/3022503535 >> 172.21.15.151:0/3022503535 conn(0x7f71801043c0 msgr2=0x7f7180106820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:42:25.291 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:24.371+0000 7f717f200700 1 -- 172.21.15.151:0/3022503535 shutdown_connections 2024-05-21T21:42:25.291 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:24.371+0000 7f717f200700 1 -- 172.21.15.151:0/3022503535 wait complete. 2024-05-21T21:42:25.291 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:24.372+0000 7f717f200700 1 Processor -- start 2024-05-21T21:42:25.291 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:24.373+0000 7f717f200700 1 -- start start 2024-05-21T21:42:25.291 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:24.373+0000 7f717f200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7180108e40 0x7f718019ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:42:25.291 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:24.373+0000 7f717f200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f718010d3e0 con 0x7f7180108e40 2024-05-21T21:42:25.291 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:24.373+0000 7f717de00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7180108e40 0x7f718019ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:42:25.291 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:24.373+0000 7f717de00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7180108e40 0x7f718019ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:46620/0 (socket says 172.21.15.151:46620) 2024-05-21T21:42:25.291 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:24.373+0000 7f717de00700 1 -- 172.21.15.151:0/3891592587 learned_addr learned my addr 172.21.15.151:0/3891592587 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:42:25.291 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:24.374+0000 7f717de00700 1 -- 172.21.15.151:0/3891592587 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f718019d360 con 0x7f7180108e40 2024-05-21T21:42:25.291 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:24.374+0000 7f717de00700 1 --2- 172.21.15.151:0/3891592587 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7180108e40 0x7f718019ce20 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f7170011ac0 tx=0x7f7170011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:42:25.291 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:24.374+0000 7f7176a00700 1 -- 172.21.15.151:0/3891592587 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7170003970 con 0x7f7180108e40 2024-05-21T21:42:25.292 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:24.374+0000 7f7176a00700 1 -- 172.21.15.151:0/3891592587 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7170003b10 con 0x7f7180108e40 2024-05-21T21:42:25.292 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:24.374+0000 7f7176a00700 1 -- 172.21.15.151:0/3891592587 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7170024610 con 0x7f7180108e40 2024-05-21T21:42:25.292 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:24.374+0000 7f717f200700 1 -- 172.21.15.151:0/3891592587 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f718019d5f0 con 0x7f7180108e40 2024-05-21T21:42:25.292 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:24.374+0000 7f717f200700 1 -- 172.21.15.151:0/3891592587 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71801a02e0 con 0x7f7180108e40 2024-05-21T21:42:25.292 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:24.375+0000 7f7176a00700 1 -- 172.21.15.151:0/3891592587 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7f717001a9e0 con 0x7f7180108e40 2024-05-21T21:42:25.292 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:24.375+0000 7f717f200700 1 -- 172.21.15.151:0/3891592587 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f718006a6a0 con 0x7f7180108e40 2024-05-21T21:42:25.292 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:24.376+0000 7f7176a00700 1 --2- 172.21.15.151:0/3891592587 >> [v2:172.21.15.151:6800/1128885292,v1:172.21.15.151:6801/1128885292] conn(0x7f716403e470 0x7f7164040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:42:25.292 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:24.376+0000 7f7176a00700 1 -- 172.21.15.151:0/3891592587 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7170053840 con 0x7f7180108e40 2024-05-21T21:42:25.292 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:24.376+0000 7f717d400700 1 --2- 172.21.15.151:0/3891592587 >> [v2:172.21.15.151:6800/1128885292,v1:172.21.15.151:6801/1128885292] conn(0x7f716403e470 0x7f7164040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:42:25.292 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:24.377+0000 7f717d400700 1 --2- 172.21.15.151:0/3891592587 >> [v2:172.21.15.151:6800/1128885292,v1:172.21.15.151:6801/1128885292] conn(0x7f716403e470 0x7f7164040930 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f716c006fd0 tx=0x7f716c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:42:25.292 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:24.382+0000 7f7176a00700 1 -- 172.21.15.151:0/3891592587 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f71700247f0 con 0x7f7180108e40 2024-05-21T21:42:25.292 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:24.615+0000 7f717f200700 1 -- 172.21.15.151:0/3891592587 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f71801a04b0 con 0x7f7180108e40 2024-05-21T21:42:25.293 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:24.618+0000 7f7176a00700 1 -- 172.21.15.151:0/3891592587 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+436 (secure 0 0 0) 0x7f71801a04b0 con 0x7f7180108e40 2024-05-21T21:42:25.293 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:24.622+0000 7f717f200700 1 -- 172.21.15.151:0/3891592587 >> [v2:172.21.15.151:6800/1128885292,v1:172.21.15.151:6801/1128885292] conn(0x7f716403e470 msgr2=0x7f7164040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:42:25.293 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:24.622+0000 7f717f200700 1 --2- 172.21.15.151:0/3891592587 >> [v2:172.21.15.151:6800/1128885292,v1:172.21.15.151:6801/1128885292] conn(0x7f716403e470 0x7f7164040930 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f716c006fd0 tx=0x7f716c006e40 comp rx=0 tx=0).stop 2024-05-21T21:42:25.293 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:24.622+0000 7f717f200700 1 -- 172.21.15.151:0/3891592587 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7180108e40 msgr2=0x7f718019ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:42:25.293 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:24.622+0000 7f717f200700 1 --2- 172.21.15.151:0/3891592587 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7180108e40 0x7f718019ce20 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f7170011ac0 tx=0x7f7170011ba0 comp rx=0 tx=0).stop 2024-05-21T21:42:25.293 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:24.622+0000 7f717f200700 1 -- 172.21.15.151:0/3891592587 shutdown_connections 2024-05-21T21:42:25.293 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:24.622+0000 7f717f200700 1 --2- 172.21.15.151:0/3891592587 >> [v2:172.21.15.151:6800/1128885292,v1:172.21.15.151:6801/1128885292] conn(0x7f716403e470 0x7f7164040930 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:42:25.293 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:24.622+0000 7f717f200700 1 --2- 172.21.15.151:0/3891592587 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7180108e40 0x7f718019ce20 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:42:25.293 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:24.622+0000 7f717f200700 1 -- 172.21.15.151:0/3891592587 >> 172.21.15.151:0/3891592587 conn(0x7f71801043c0 msgr2=0x7f7180104fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:42:25.293 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:24.622+0000 7f717f200700 1 -- 172.21.15.151:0/3891592587 shutdown_connections 2024-05-21T21:42:25.293 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:24.622+0000 7f717f200700 1 -- 172.21.15.151:0/3891592587 wait complete. 2024-05-21T21:42:25.294 INFO:teuthology.orchestra.run.smithi151.stdout:Enabling cephadm module... 2024-05-21T21:42:25.563 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:25 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/3891592587' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-21T21:42:26.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:26 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/1168604837' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-05-21T21:42:26.453 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:42:26 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: ignoring --setuser ceph since I am not root 2024-05-21T21:42:26.453 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:42:26 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: ignoring --setgroup ceph since I am not root 2024-05-21T21:42:26.805 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:42:26 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:42:26.549+0000 7f7411685200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-21T21:42:26.805 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:42:26 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:42:26.796+0000 7f7411685200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-21T21:42:27.159 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:25.877+0000 7f5696400700 1 Processor -- start 2024-05-21T21:42:27.159 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:25.877+0000 7f5696400700 1 -- start start 2024-05-21T21:42:27.159 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:25.878+0000 7f5696400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5690108f10 0x7f5690109330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:42:27.160 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:25.878+0000 7f5696400700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5690109870 con 0x7f5690108f10 2024-05-21T21:42:27.160 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:25.878+0000 7f5695000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5690108f10 0x7f5690109330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:42:27.160 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:25.878+0000 7f5695000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5690108f10 0x7f5690109330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:39400/0 (socket says 172.21.15.151:39400) 2024-05-21T21:42:27.160 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:25.878+0000 7f5695000700 1 -- 172.21.15.151:0/164377625 learned_addr learned my addr 172.21.15.151:0/164377625 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:42:27.160 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:25.879+0000 7f5695000700 1 -- 172.21.15.151:0/164377625 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f569010a070 con 0x7f5690108f10 2024-05-21T21:42:27.160 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:25.879+0000 7f5695000700 1 --2- 172.21.15.151:0/164377625 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5690108f10 0x7f5690109330 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f56840098a0 tx=0x7f5684009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b209fe3093f90f55 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:42:27.160 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:25.879+0000 7f568f400700 1 -- 172.21.15.151:0/164377625 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5684004160 con 0x7f5690108f10 2024-05-21T21:42:27.160 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:25.879+0000 7f568f400700 1 -- 172.21.15.151:0/164377625 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f5684004300 con 0x7f5690108f10 2024-05-21T21:42:27.160 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:25.879+0000 7f568f400700 1 -- 172.21.15.151:0/164377625 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5684011af0 con 0x7f5690108f10 2024-05-21T21:42:27.160 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:25.880+0000 7f5696400700 1 -- 172.21.15.151:0/164377625 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5690108f10 msgr2=0x7f5690109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:42:27.160 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:25.880+0000 7f5696400700 1 --2- 172.21.15.151:0/164377625 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5690108f10 0x7f5690109330 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f56840098a0 tx=0x7f5684009bb0 comp rx=0 tx=0).stop 2024-05-21T21:42:27.160 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:25.880+0000 7f5696400700 1 -- 172.21.15.151:0/164377625 shutdown_connections 2024-05-21T21:42:27.160 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:25.880+0000 7f5696400700 1 --2- 172.21.15.151:0/164377625 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5690108f10 0x7f5690109330 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:42:27.161 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:25.880+0000 7f5696400700 1 -- 172.21.15.151:0/164377625 >> 172.21.15.151:0/164377625 conn(0x7f56901044d0 msgr2=0x7f56901068f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:42:27.161 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:25.880+0000 7f5696400700 1 -- 172.21.15.151:0/164377625 shutdown_connections 2024-05-21T21:42:27.161 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:25.880+0000 7f5696400700 1 -- 172.21.15.151:0/164377625 wait complete. 2024-05-21T21:42:27.161 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:25.881+0000 7f5696400700 1 Processor -- start 2024-05-21T21:42:27.161 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:25.881+0000 7f5696400700 1 -- start start 2024-05-21T21:42:27.161 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:25.882+0000 7f5696400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5690108f10 0x7f569019f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:42:27.161 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:25.882+0000 7f5696400700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f569010d330 con 0x7f5690108f10 2024-05-21T21:42:27.161 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:25.882+0000 7f5695000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5690108f10 0x7f569019f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:42:27.161 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:25.882+0000 7f5695000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5690108f10 0x7f569019f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:39412/0 (socket says 172.21.15.151:39412) 2024-05-21T21:42:27.161 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:25.882+0000 7f5695000700 1 -- 172.21.15.151:0/1168604837 learned_addr learned my addr 172.21.15.151:0/1168604837 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:42:27.161 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:25.882+0000 7f5695000700 1 -- 172.21.15.151:0/1168604837 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f569019f5c0 con 0x7f5690108f10 2024-05-21T21:42:27.161 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:25.882+0000 7f5695000700 1 --2- 172.21.15.151:0/1168604837 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5690108f10 0x7f569019f080 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f5684011ac0 tx=0x7f5684011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:42:27.162 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:25.883+0000 7f568d600700 1 -- 172.21.15.151:0/1168604837 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f56840039e0 con 0x7f5690108f10 2024-05-21T21:42:27.162 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:25.883+0000 7f568d600700 1 -- 172.21.15.151:0/1168604837 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f5684003b80 con 0x7f5690108f10 2024-05-21T21:42:27.162 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:25.883+0000 7f5696400700 1 -- 172.21.15.151:0/1168604837 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f569019f850 con 0x7f5690108f10 2024-05-21T21:42:27.162 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:25.883+0000 7f568d600700 1 -- 172.21.15.151:0/1168604837 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f56840246c0 con 0x7f5690108f10 2024-05-21T21:42:27.162 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:25.883+0000 7f5696400700 1 -- 172.21.15.151:0/1168604837 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56901a2540 con 0x7f5690108f10 2024-05-21T21:42:27.162 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:25.884+0000 7f568d600700 1 -- 172.21.15.151:0/1168604837 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7f5684022020 con 0x7f5690108f10 2024-05-21T21:42:27.162 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:25.884+0000 7f5696400700 1 -- 172.21.15.151:0/1168604837 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f569006a6a0 con 0x7f5690108f10 2024-05-21T21:42:27.162 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:25.884+0000 7f568d600700 1 --2- 172.21.15.151:0/1168604837 >> [v2:172.21.15.151:6800/1128885292,v1:172.21.15.151:6801/1128885292] conn(0x7f567803e420 0x7f56780408e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:42:27.162 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:25.885+0000 7f568d600700 1 -- 172.21.15.151:0/1168604837 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5684053450 con 0x7f5690108f10 2024-05-21T21:42:27.162 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:25.885+0000 7f568fe00700 1 --2- 172.21.15.151:0/1168604837 >> [v2:172.21.15.151:6800/1128885292,v1:172.21.15.151:6801/1128885292] conn(0x7f567803e420 0x7f56780408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:42:27.162 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:25.885+0000 7f568fe00700 1 --2- 172.21.15.151:0/1168604837 >> [v2:172.21.15.151:6800/1128885292,v1:172.21.15.151:6801/1128885292] conn(0x7f567803e420 0x7f56780408e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f567c006fd0 tx=0x7f567c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:42:27.162 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:25.890+0000 7f568d600700 1 -- 172.21.15.151:0/1168604837 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5684053130 con 0x7f5690108f10 2024-05-21T21:42:27.163 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:26.179+0000 7f5696400700 1 -- 172.21.15.151:0/1168604837 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f56901a2710 con 0x7f5690108f10 2024-05-21T21:42:27.163 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:26.245+0000 7f568d600700 1 -- 172.21.15.151:0/1168604837 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f56901a2710 con 0x7f5690108f10 2024-05-21T21:42:27.163 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:26.249+0000 7f568d600700 1 -- 172.21.15.151:0/1168604837 <== mon.0 v2:172.21.15.151:3300/0 8 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7f568401a430 con 0x7f5690108f10 2024-05-21T21:42:27.163 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:26.251+0000 7f5696400700 1 -- 172.21.15.151:0/1168604837 >> [v2:172.21.15.151:6800/1128885292,v1:172.21.15.151:6801/1128885292] conn(0x7f567803e420 msgr2=0x7f56780408e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:42:27.163 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:26.251+0000 7f5696400700 1 --2- 172.21.15.151:0/1168604837 >> [v2:172.21.15.151:6800/1128885292,v1:172.21.15.151:6801/1128885292] conn(0x7f567803e420 0x7f56780408e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f567c006fd0 tx=0x7f567c006e40 comp rx=0 tx=0).stop 2024-05-21T21:42:27.163 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:26.251+0000 7f5696400700 1 -- 172.21.15.151:0/1168604837 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5690108f10 msgr2=0x7f569019f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:42:27.163 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:26.251+0000 7f5696400700 1 --2- 172.21.15.151:0/1168604837 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5690108f10 0x7f569019f080 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f5684011ac0 tx=0x7f5684011ba0 comp rx=0 tx=0).stop 2024-05-21T21:42:27.163 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:26.252+0000 7f5696400700 1 -- 172.21.15.151:0/1168604837 shutdown_connections 2024-05-21T21:42:27.163 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:26.252+0000 7f5696400700 1 --2- 172.21.15.151:0/1168604837 >> [v2:172.21.15.151:6800/1128885292,v1:172.21.15.151:6801/1128885292] conn(0x7f567803e420 0x7f56780408e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:42:27.163 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:26.252+0000 7f5696400700 1 --2- 172.21.15.151:0/1168604837 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5690108f10 0x7f569019f080 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:42:27.163 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:26.252+0000 7f5696400700 1 -- 172.21.15.151:0/1168604837 >> 172.21.15.151:0/1168604837 conn(0x7f56901044d0 msgr2=0x7f5690104fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:42:27.163 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:26.252+0000 7f5696400700 1 -- 172.21.15.151:0/1168604837 shutdown_connections 2024-05-21T21:42:27.163 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:26.252+0000 7f5696400700 1 -- 172.21.15.151:0/1168604837 wait complete. 2024-05-21T21:42:27.202 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:42:26 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:42:26.926+0000 7f7411685200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-21T21:42:27.498 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:27 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/1168604837' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-05-21T21:42:27.498 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:27 smithi151 ceph-mon[31623]: mgrmap e5: x(active, since 5s) 2024-05-21T21:42:27.499 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:42:27 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:42:27.341+0000 7f7411685200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-21T21:42:27.499 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:42:27 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:42:27.453+0000 7f7411685200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-21T21:42:28.452 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:28 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/2509565033' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-21T21:42:28.789 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout { 2024-05-21T21:42:28.789 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-05-21T21:42:28.789 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-21T21:42:28.789 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-21T21:42:28.789 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-21T21:42:28.789 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout } 2024-05-21T21:42:28.789 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:27.778+0000 7f0ac2400700 1 Processor -- start 2024-05-21T21:42:28.789 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:27.779+0000 7f0ac2400700 1 -- start start 2024-05-21T21:42:28.789 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:27.779+0000 7f0ac2400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0abc07dd80 0x7f0abc07c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:42:28.789 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:27.779+0000 7f0ac2400700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0abc07c7b0 con 0x7f0abc07dd80 2024-05-21T21:42:28.790 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:27.780+0000 7f0ac1000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0abc07dd80 0x7f0abc07c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:42:28.790 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:27.780+0000 7f0ac1000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0abc07dd80 0x7f0abc07c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:39440/0 (socket says 172.21.15.151:39440) 2024-05-21T21:42:28.790 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:27.780+0000 7f0ac1000700 1 -- 172.21.15.151:0/969840940 learned_addr learned my addr 172.21.15.151:0/969840940 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:42:28.790 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:27.780+0000 7f0ac1000700 1 -- 172.21.15.151:0/969840940 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0abc07c930 con 0x7f0abc07dd80 2024-05-21T21:42:28.790 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:27.781+0000 7f0ac1000700 1 --2- 172.21.15.151:0/969840940 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0abc07dd80 0x7f0abc07c1e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0ab00098a0 tx=0x7f0ab0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59b03ab59afd7d35 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:42:28.790 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:27.781+0000 7f0abb400700 1 -- 172.21.15.151:0/969840940 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0ab0004160 con 0x7f0abc07dd80 2024-05-21T21:42:28.790 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:27.781+0000 7f0abb400700 1 -- 172.21.15.151:0/969840940 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0ab0004300 con 0x7f0abc07dd80 2024-05-21T21:42:28.790 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:27.781+0000 7f0abb400700 1 -- 172.21.15.151:0/969840940 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0ab0011af0 con 0x7f0abc07dd80 2024-05-21T21:42:28.790 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:27.781+0000 7f0ac2400700 1 -- 172.21.15.151:0/969840940 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0abc07dd80 msgr2=0x7f0abc07c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:42:28.790 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:27.781+0000 7f0ac2400700 1 --2- 172.21.15.151:0/969840940 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0abc07dd80 0x7f0abc07c1e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0ab00098a0 tx=0x7f0ab0009bb0 comp rx=0 tx=0).stop 2024-05-21T21:42:28.790 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:27.781+0000 7f0ac2400700 1 -- 172.21.15.151:0/969840940 shutdown_connections 2024-05-21T21:42:28.790 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:27.782+0000 7f0ac2400700 1 --2- 172.21.15.151:0/969840940 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0abc07dd80 0x7f0abc07c1e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:42:28.790 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:27.782+0000 7f0ac2400700 1 -- 172.21.15.151:0/969840940 >> 172.21.15.151:0/969840940 conn(0x7f0abc1043c0 msgr2=0x7f0abc106820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:42:28.791 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:27.782+0000 7f0ac2400700 1 -- 172.21.15.151:0/969840940 shutdown_connections 2024-05-21T21:42:28.791 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:27.782+0000 7f0ac2400700 1 -- 172.21.15.151:0/969840940 wait complete. 2024-05-21T21:42:28.791 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:27.783+0000 7f0ac2400700 1 Processor -- start 2024-05-21T21:42:28.791 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:27.783+0000 7f0ac2400700 1 -- start start 2024-05-21T21:42:28.791 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:27.783+0000 7f0ac2400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0abc07dd80 0x7f0abc19f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:42:28.791 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:27.783+0000 7f0ac2400700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0abc10f3f0 con 0x7f0abc07dd80 2024-05-21T21:42:28.791 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:27.784+0000 7f0ac1000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0abc07dd80 0x7f0abc19f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:42:28.791 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:27.784+0000 7f0ac1000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0abc07dd80 0x7f0abc19f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:39454/0 (socket says 172.21.15.151:39454) 2024-05-21T21:42:28.791 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:27.784+0000 7f0ac1000700 1 -- 172.21.15.151:0/2509565033 learned_addr learned my addr 172.21.15.151:0/2509565033 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:42:28.791 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:27.784+0000 7f0ac1000700 1 -- 172.21.15.151:0/2509565033 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0abc19f5c0 con 0x7f0abc07dd80 2024-05-21T21:42:28.791 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:27.784+0000 7f0ac1000700 1 --2- 172.21.15.151:0/2509565033 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0abc07dd80 0x7f0abc19f080 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0ab00117b0 tx=0x7f0ab0011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:42:28.791 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:27.785+0000 7f0ab9600700 1 -- 172.21.15.151:0/2509565033 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0ab0003af0 con 0x7f0abc07dd80 2024-05-21T21:42:28.791 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:27.785+0000 7f0ab9600700 1 -- 172.21.15.151:0/2509565033 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0ab0020070 con 0x7f0abc07dd80 2024-05-21T21:42:28.792 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:27.785+0000 7f0ac2400700 1 -- 172.21.15.151:0/2509565033 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0abc19f850 con 0x7f0abc07dd80 2024-05-21T21:42:28.792 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:27.785+0000 7f0ab9600700 1 -- 172.21.15.151:0/2509565033 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0ab0019630 con 0x7f0abc07dd80 2024-05-21T21:42:28.792 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:27.785+0000 7f0ac2400700 1 -- 172.21.15.151:0/2509565033 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0abc1a2540 con 0x7f0abc07dd80 2024-05-21T21:42:28.792 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:27.786+0000 7f0ab9600700 1 -- 172.21.15.151:0/2509565033 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7f0ab001a9c0 con 0x7f0abc07dd80 2024-05-21T21:42:28.792 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:27.786+0000 7f0ac2400700 1 -- 172.21.15.151:0/2509565033 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0abc06a6a0 con 0x7f0abc07dd80 2024-05-21T21:42:28.792 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:27.787+0000 7f0ab9600700 1 --2- 172.21.15.151:0/2509565033 >> [v2:172.21.15.151:6800/1128885292,v1:172.21.15.151:6801/1128885292] conn(0x7f0aa403e4c0 0x7f0aa4040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:42:28.792 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:27.787+0000 7f0ab9600700 1 -- 172.21.15.151:0/2509565033 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0ab0053a50 con 0x7f0abc07dd80 2024-05-21T21:42:28.792 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:27.787+0000 7f0abbe00700 1 -- 172.21.15.151:0/2509565033 >> [v2:172.21.15.151:6800/1128885292,v1:172.21.15.151:6801/1128885292] conn(0x7f0aa403e4c0 msgr2=0x7f0aa4040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.151:6800/1128885292 2024-05-21T21:42:28.792 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:27.787+0000 7f0abbe00700 1 --2- 172.21.15.151:0/2509565033 >> [v2:172.21.15.151:6800/1128885292,v1:172.21.15.151:6801/1128885292] conn(0x7f0aa403e4c0 0x7f0aa4040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-21T21:42:28.792 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:27.792+0000 7f0ab9600700 1 -- 172.21.15.151:0/2509565033 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0ab00042a0 con 0x7f0abc07dd80 2024-05-21T21:42:28.792 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:27.988+0000 7f0abbe00700 1 -- 172.21.15.151:0/2509565033 >> [v2:172.21.15.151:6800/1128885292,v1:172.21.15.151:6801/1128885292] conn(0x7f0aa403e4c0 msgr2=0x7f0aa4040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.151:6800/1128885292 2024-05-21T21:42:28.792 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:27.988+0000 7f0abbe00700 1 --2- 172.21.15.151:0/2509565033 >> [v2:172.21.15.151:6800/1128885292,v1:172.21.15.151:6801/1128885292] conn(0x7f0aa403e4c0 0x7f0aa4040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-21T21:42:28.792 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:28.117+0000 7f0ac2400700 1 -- 172.21.15.151:0/2509565033 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f0abc1a2710 con 0x7f0abc07dd80 2024-05-21T21:42:28.793 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:28.117+0000 7f0ab9600700 1 -- 172.21.15.151:0/2509565033 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f0abc1a2710 con 0x7f0abc07dd80 2024-05-21T21:42:28.793 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:28.122+0000 7f0ac2400700 1 -- 172.21.15.151:0/2509565033 >> [v2:172.21.15.151:6800/1128885292,v1:172.21.15.151:6801/1128885292] conn(0x7f0aa403e4c0 msgr2=0x7f0aa4040980 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-21T21:42:28.793 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:28.122+0000 7f0ac2400700 1 --2- 172.21.15.151:0/2509565033 >> [v2:172.21.15.151:6800/1128885292,v1:172.21.15.151:6801/1128885292] conn(0x7f0aa403e4c0 0x7f0aa4040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:42:28.793 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:28.122+0000 7f0ac2400700 1 -- 172.21.15.151:0/2509565033 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0abc07dd80 msgr2=0x7f0abc19f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:42:28.793 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:28.122+0000 7f0ac2400700 1 --2- 172.21.15.151:0/2509565033 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0abc07dd80 0x7f0abc19f080 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0ab00117b0 tx=0x7f0ab0011b70 comp rx=0 tx=0).stop 2024-05-21T21:42:28.793 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:28.122+0000 7f0ac2400700 1 -- 172.21.15.151:0/2509565033 shutdown_connections 2024-05-21T21:42:28.793 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:28.122+0000 7f0ac2400700 1 --2- 172.21.15.151:0/2509565033 >> [v2:172.21.15.151:6800/1128885292,v1:172.21.15.151:6801/1128885292] conn(0x7f0aa403e4c0 0x7f0aa4040980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:42:28.793 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:28.122+0000 7f0ac2400700 1 --2- 172.21.15.151:0/2509565033 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0abc07dd80 0x7f0abc19f080 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:42:28.793 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:28.122+0000 7f0ac2400700 1 -- 172.21.15.151:0/2509565033 >> 172.21.15.151:0/2509565033 conn(0x7f0abc1043c0 msgr2=0x7f0abc105050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:42:28.793 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:28.123+0000 7f0ac2400700 1 -- 172.21.15.151:0/2509565033 shutdown_connections 2024-05-21T21:42:28.793 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:28.123+0000 7f0ac2400700 1 -- 172.21.15.151:0/2509565033 wait complete. 2024-05-21T21:42:28.793 INFO:teuthology.orchestra.run.smithi151.stdout:Waiting for the mgr to restart... 2024-05-21T21:42:28.793 INFO:teuthology.orchestra.run.smithi151.stdout:Waiting for mgr epoch 5... 2024-05-21T21:42:29.320 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:42:29 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:42:29.048+0000 7f7411685200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-21T21:42:29.320 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:42:29 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:42:29.276+0000 7f7411685200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-21T21:42:29.605 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:42:29 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:42:29.496+0000 7f7411685200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-21T21:42:30.203 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:42:29 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:42:29.870+0000 7f7411685200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-21T21:42:30.203 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:42:29 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:42:29.990+0000 7f7411685200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-21T21:42:30.703 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:42:30 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:42:30.242+0000 7f7411685200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-21T21:42:30.703 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:42:30 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:42:30.492+0000 7f7411685200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-21T21:42:31.554 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:42:31 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:42:31.273+0000 7f7411685200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-21T21:42:31.555 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:42:31 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:42:31.554+0000 7f7411685200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-21T21:42:32.410 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:42:32 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:42:32.098+0000 7f7411685200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-21T21:42:32.410 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:42:32 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:42:32.226+0000 7f7411685200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-21T21:42:32.953 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:42:32 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:42:32.692+0000 7f7411685200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-21T21:42:33.453 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:42:33 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:42:33.140+0000 7f7411685200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-21T21:42:33.453 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:42:33 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:42:33.265+0000 7f7411685200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-21T21:42:33.895 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:42:33 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:42:33.584+0000 7f7411685200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-21T21:42:33.895 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:42:33 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:42:33.762+0000 7f7411685200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-21T21:42:34.203 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:42:33 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:42:33.895+0000 7f7411685200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-21T21:42:34.702 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:42:34 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:42:34.291+0000 7f7411685200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-21T21:42:34.703 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:42:34 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:42:34.412+0000 7f7411685200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-21T21:42:35.438 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:42:35 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:42:35.156+0000 7f7411685200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-21T21:42:35.703 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:42:35 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:42:35.437+0000 7f7411685200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-21T21:42:35.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:35 smithi151 ceph-mon[31623]: Active manager daemon x restarted 2024-05-21T21:42:35.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:35 smithi151 ceph-mon[31623]: Activating manager daemon x 2024-05-21T21:42:35.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:35 smithi151 ceph-mon[31623]: osdmap e2: 0 total, 0 up, 0 in 2024-05-21T21:42:35.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:35 smithi151 ceph-mon[31623]: mgrmap e6: x(active, starting, since 0.00909483s) 2024-05-21T21:42:35.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:35 smithi151 ceph-mon[31623]: from='mgr.14120 172.21.15.151:0/2204845274' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-21T21:42:35.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:35 smithi151 ceph-mon[31623]: from='mgr.14120 172.21.15.151:0/2204845274' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-21T21:42:35.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:35 smithi151 ceph-mon[31623]: from='mgr.14120 172.21.15.151:0/2204845274' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-21T21:42:35.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:35 smithi151 ceph-mon[31623]: from='mgr.14120 172.21.15.151:0/2204845274' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-21T21:42:35.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:35 smithi151 ceph-mon[31623]: from='mgr.14120 172.21.15.151:0/2204845274' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-21T21:42:35.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:35 smithi151 ceph-mon[31623]: Manager daemon x is now available 2024-05-21T21:42:36.846 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:36 smithi151 ceph-mon[31623]: Found migration_current of "None". Setting to last migration. 2024-05-21T21:42:36.846 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:36 smithi151 ceph-mon[31623]: from='mgr.14120 172.21.15.151:0/2204845274' entity='mgr.x' 2024-05-21T21:42:36.846 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:36 smithi151 ceph-mon[31623]: from='mgr.14120 172.21.15.151:0/2204845274' entity='mgr.x' 2024-05-21T21:42:36.846 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:36 smithi151 ceph-mon[31623]: from='mgr.14120 172.21.15.151:0/2204845274' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:42:36.846 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:36 smithi151 ceph-mon[31623]: from='mgr.14120 172.21.15.151:0/2204845274' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:42:36.846 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:36 smithi151 ceph-mon[31623]: from='mgr.14120 172.21.15.151:0/2204845274' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-21T21:42:36.846 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:36 smithi151 ceph-mon[31623]: from='mgr.14120 172.21.15.151:0/2204845274' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-21T21:42:36.846 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:36 smithi151 ceph-mon[31623]: from='mgr.14120 172.21.15.151:0/2204845274' entity='mgr.x' 2024-05-21T21:42:36.846 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:36 smithi151 ceph-mon[31623]: from='mgr.14120 172.21.15.151:0/2204845274' entity='mgr.x' 2024-05-21T21:42:36.846 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:36 smithi151 ceph-mon[31623]: [21/May/2024:21:42:36] ENGINE Bus STARTING 2024-05-21T21:42:36.846 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:36 smithi151 ceph-mon[31623]: [21/May/2024:21:42:36] ENGINE Serving on http://172.21.15.151:8765 2024-05-21T21:42:36.847 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:36 smithi151 ceph-mon[31623]: from='mgr.14120 172.21.15.151:0/2204845274' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:42:36.847 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:36 smithi151 ceph-mon[31623]: mgrmap e7: x(active, since 1.01936s) 2024-05-21T21:42:37.151 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout { 2024-05-21T21:42:37.151 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-05-21T21:42:37.151 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-21T21:42:37.151 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout } 2024-05-21T21:42:37.151 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:29.396+0000 7f53baa00700 1 Processor -- start 2024-05-21T21:42:37.151 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:29.396+0000 7f53baa00700 1 -- start start 2024-05-21T21:42:37.152 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:29.397+0000 7f53baa00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f53b407c1c0 0x7f53b407c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:42:37.152 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:29.397+0000 7f53baa00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f53b407def0 con 0x7f53b407c1c0 2024-05-21T21:42:37.152 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:29.397+0000 7f53b3e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f53b407c1c0 0x7f53b407c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:42:37.152 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:29.397+0000 7f53b3e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f53b407c1c0 0x7f53b407c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:39456/0 (socket says 172.21.15.151:39456) 2024-05-21T21:42:37.152 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:29.397+0000 7f53b3e00700 1 -- 172.21.15.151:0/212601782 learned_addr learned my addr 172.21.15.151:0/212601782 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:42:37.152 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:29.398+0000 7f53b3e00700 1 -- 172.21.15.151:0/212601782 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53b407cb20 con 0x7f53b407c1c0 2024-05-21T21:42:37.152 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:29.398+0000 7f53b3e00700 1 --2- 172.21.15.151:0/212601782 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f53b407c1c0 0x7f53b407c5e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f53a40098a0 tx=0x7f53a4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe2d65d5b43fc8b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:42:37.152 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:29.398+0000 7f53b2a00700 1 -- 172.21.15.151:0/212601782 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f53a4004160 con 0x7f53b407c1c0 2024-05-21T21:42:37.152 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:29.398+0000 7f53b2a00700 1 -- 172.21.15.151:0/212601782 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f53a4004300 con 0x7f53b407c1c0 2024-05-21T21:42:37.152 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:29.398+0000 7f53b2a00700 1 -- 172.21.15.151:0/212601782 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f53a4011af0 con 0x7f53b407c1c0 2024-05-21T21:42:37.153 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:29.399+0000 7f53baa00700 1 -- 172.21.15.151:0/212601782 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f53b407c1c0 msgr2=0x7f53b407c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:42:37.153 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:29.399+0000 7f53baa00700 1 --2- 172.21.15.151:0/212601782 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f53b407c1c0 0x7f53b407c5e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f53a40098a0 tx=0x7f53a4009bb0 comp rx=0 tx=0).stop 2024-05-21T21:42:37.153 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:29.399+0000 7f53baa00700 1 -- 172.21.15.151:0/212601782 shutdown_connections 2024-05-21T21:42:37.153 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:29.399+0000 7f53baa00700 1 --2- 172.21.15.151:0/212601782 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f53b407c1c0 0x7f53b407c5e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:42:37.153 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:29.399+0000 7f53baa00700 1 -- 172.21.15.151:0/212601782 >> 172.21.15.151:0/212601782 conn(0x7f53b41045a0 msgr2=0x7f53b4106a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:42:37.153 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:29.399+0000 7f53baa00700 1 -- 172.21.15.151:0/212601782 shutdown_connections 2024-05-21T21:42:37.153 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:29.399+0000 7f53baa00700 1 -- 172.21.15.151:0/212601782 wait complete. 2024-05-21T21:42:37.153 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:29.400+0000 7f53baa00700 1 Processor -- start 2024-05-21T21:42:37.153 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:29.400+0000 7f53baa00700 1 -- start start 2024-05-21T21:42:37.153 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:29.401+0000 7f53baa00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f53b407c1c0 0x7f53b419f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:42:37.153 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:29.401+0000 7f53baa00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f53b410f710 con 0x7f53b407c1c0 2024-05-21T21:42:37.153 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:29.401+0000 7f53b3e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f53b407c1c0 0x7f53b419f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:42:37.154 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:29.401+0000 7f53b3e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f53b407c1c0 0x7f53b419f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:39468/0 (socket says 172.21.15.151:39468) 2024-05-21T21:42:37.154 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:29.401+0000 7f53b3e00700 1 -- 172.21.15.151:0/3290539823 learned_addr learned my addr 172.21.15.151:0/3290539823 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:42:37.154 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:29.401+0000 7f53b3e00700 1 -- 172.21.15.151:0/3290539823 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53b419f7a0 con 0x7f53b407c1c0 2024-05-21T21:42:37.154 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:29.401+0000 7f53b3e00700 1 --2- 172.21.15.151:0/3290539823 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f53b407c1c0 0x7f53b419f260 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f53a4003f40 tx=0x7f53a4011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:42:37.154 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:29.402+0000 7f53b0c00700 1 -- 172.21.15.151:0/3290539823 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f53a40038b0 con 0x7f53b407c1c0 2024-05-21T21:42:37.154 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:29.402+0000 7f53b0c00700 1 -- 172.21.15.151:0/3290539823 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f53a4003a50 con 0x7f53b407c1c0 2024-05-21T21:42:37.154 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:29.402+0000 7f53baa00700 1 -- 172.21.15.151:0/3290539823 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53b419fa30 con 0x7f53b407c1c0 2024-05-21T21:42:37.154 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:29.402+0000 7f53b0c00700 1 -- 172.21.15.151:0/3290539823 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f53a401a5c0 con 0x7f53b407c1c0 2024-05-21T21:42:37.154 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:29.402+0000 7f53baa00700 1 -- 172.21.15.151:0/3290539823 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53b41a2720 con 0x7f53b407c1c0 2024-05-21T21:42:37.154 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:29.403+0000 7f53b0c00700 1 -- 172.21.15.151:0/3290539823 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7f53a4024430 con 0x7f53b407c1c0 2024-05-21T21:42:37.154 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:29.404+0000 7f53b0c00700 1 --2- 172.21.15.151:0/3290539823 >> [v2:172.21.15.151:6800/1128885292,v1:172.21.15.151:6801/1128885292] conn(0x7f539803e420 0x7f53980408e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:42:37.154 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:29.404+0000 7f53b0c00700 1 -- 172.21.15.151:0/3290539823 --> [v2:172.21.15.151:6800/1128885292,v1:172.21.15.151:6801/1128885292] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f5398040ff0 con 0x7f539803e420 2024-05-21T21:42:37.155 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:29.404+0000 7f53b0c00700 1 -- 172.21.15.151:0/3290539823 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f53a4053a50 con 0x7f53b407c1c0 2024-05-21T21:42:37.155 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:29.404+0000 7f53b3400700 1 -- 172.21.15.151:0/3290539823 >> [v2:172.21.15.151:6800/1128885292,v1:172.21.15.151:6801/1128885292] conn(0x7f539803e420 msgr2=0x7f53980408e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.151:6800/1128885292 2024-05-21T21:42:37.155 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:29.404+0000 7f53b3400700 1 --2- 172.21.15.151:0/3290539823 >> [v2:172.21.15.151:6800/1128885292,v1:172.21.15.151:6801/1128885292] conn(0x7f539803e420 0x7f53980408e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-21T21:42:37.155 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:29.605+0000 7f53b3400700 1 -- 172.21.15.151:0/3290539823 >> [v2:172.21.15.151:6800/1128885292,v1:172.21.15.151:6801/1128885292] conn(0x7f539803e420 msgr2=0x7f53980408e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.151:6800/1128885292 2024-05-21T21:42:37.155 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:29.605+0000 7f53b3400700 1 --2- 172.21.15.151:0/3290539823 >> [v2:172.21.15.151:6800/1128885292,v1:172.21.15.151:6801/1128885292] conn(0x7f539803e420 0x7f53980408e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-21T21:42:37.155 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:30.006+0000 7f53b3400700 1 -- 172.21.15.151:0/3290539823 >> [v2:172.21.15.151:6800/1128885292,v1:172.21.15.151:6801/1128885292] conn(0x7f539803e420 msgr2=0x7f53980408e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.151:6800/1128885292 2024-05-21T21:42:37.155 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:30.006+0000 7f53b3400700 1 --2- 172.21.15.151:0/3290539823 >> [v2:172.21.15.151:6800/1128885292,v1:172.21.15.151:6801/1128885292] conn(0x7f539803e420 0x7f53980408e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-21T21:42:37.155 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:30.807+0000 7f53b3400700 1 -- 172.21.15.151:0/3290539823 >> [v2:172.21.15.151:6800/1128885292,v1:172.21.15.151:6801/1128885292] conn(0x7f539803e420 msgr2=0x7f53980408e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.151:6800/1128885292 2024-05-21T21:42:37.155 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:30.807+0000 7f53b3400700 1 --2- 172.21.15.151:0/3290539823 >> [v2:172.21.15.151:6800/1128885292,v1:172.21.15.151:6801/1128885292] conn(0x7f539803e420 0x7f53980408e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-21T21:42:37.155 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:32.409+0000 7f53b3400700 1 -- 172.21.15.151:0/3290539823 >> [v2:172.21.15.151:6800/1128885292,v1:172.21.15.151:6801/1128885292] conn(0x7f539803e420 msgr2=0x7f53980408e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.151:6800/1128885292 2024-05-21T21:42:37.155 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:32.409+0000 7f53b3400700 1 --2- 172.21.15.151:0/3290539823 >> [v2:172.21.15.151:6800/1128885292,v1:172.21.15.151:6801/1128885292] conn(0x7f539803e420 0x7f53980408e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-21T21:42:37.156 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:35.446+0000 7f53b0c00700 1 -- 172.21.15.151:0/3290539823 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mgrmap(e 6) ==== 49989+0+0 (secure 0 0 0) 0x7f53a4024770 con 0x7f53b407c1c0 2024-05-21T21:42:37.156 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:35.446+0000 7f53b0c00700 1 -- 172.21.15.151:0/3290539823 >> [v2:172.21.15.151:6800/1128885292,v1:172.21.15.151:6801/1128885292] conn(0x7f539803e420 msgr2=0x7f53980408e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-21T21:42:37.156 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:35.446+0000 7f53b0c00700 1 --2- 172.21.15.151:0/3290539823 >> [v2:172.21.15.151:6800/1128885292,v1:172.21.15.151:6801/1128885292] conn(0x7f539803e420 0x7f53980408e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:42:37.156 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:36.453+0000 7f53b0c00700 1 -- 172.21.15.151:0/3290539823 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mgrmap(e 7) ==== 50116+0+0 (secure 0 0 0) 0x7f53a4055a10 con 0x7f53b407c1c0 2024-05-21T21:42:37.156 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:36.453+0000 7f53b0c00700 1 --2- 172.21.15.151:0/3290539823 >> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] conn(0x7f5398041d10 0x7f5398044100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:42:37.156 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:36.453+0000 7f53b0c00700 1 -- 172.21.15.151:0/3290539823 --> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f5398040ff0 con 0x7f5398041d10 2024-05-21T21:42:37.156 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:36.455+0000 7f53b3400700 1 --2- 172.21.15.151:0/3290539823 >> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] conn(0x7f5398041d10 0x7f5398044100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:42:37.156 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:36.457+0000 7f53b3400700 1 --2- 172.21.15.151:0/3290539823 >> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] conn(0x7f5398041d10 0x7f5398044100 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f53a0003a10 tx=0x7f53a0009290 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:42:37.156 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:36.459+0000 7f53b0c00700 1 -- 172.21.15.151:0/3290539823 <== mgr.14120 v2:172.21.15.151:6800/3428827300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f5398040ff0 con 0x7f5398041d10 2024-05-21T21:42:37.156 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:36.469+0000 7f53baa00700 1 -- 172.21.15.151:0/3290539823 --> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f53b406a6a0 con 0x7f5398041d10 2024-05-21T21:42:37.156 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:36.469+0000 7f53b0c00700 1 -- 172.21.15.151:0/3290539823 <== mgr.14120 v2:172.21.15.151:6800/3428827300 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f53b406a6a0 con 0x7f5398041d10 2024-05-21T21:42:37.156 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:36.470+0000 7f53baa00700 1 -- 172.21.15.151:0/3290539823 >> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] conn(0x7f5398041d10 msgr2=0x7f5398044100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:42:37.156 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:36.470+0000 7f53baa00700 1 --2- 172.21.15.151:0/3290539823 >> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] conn(0x7f5398041d10 0x7f5398044100 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f53a0003a10 tx=0x7f53a0009290 comp rx=0 tx=0).stop 2024-05-21T21:42:37.157 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:36.470+0000 7f53baa00700 1 -- 172.21.15.151:0/3290539823 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f53b407c1c0 msgr2=0x7f53b419f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:42:37.157 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:36.470+0000 7f53baa00700 1 --2- 172.21.15.151:0/3290539823 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f53b407c1c0 0x7f53b419f260 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f53a4003f40 tx=0x7f53a4011890 comp rx=0 tx=0).stop 2024-05-21T21:42:37.157 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:36.470+0000 7f53baa00700 1 -- 172.21.15.151:0/3290539823 shutdown_connections 2024-05-21T21:42:37.157 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:36.470+0000 7f53baa00700 1 --2- 172.21.15.151:0/3290539823 >> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] conn(0x7f5398041d10 0x7f5398044100 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:42:37.157 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:36.470+0000 7f53baa00700 1 --2- 172.21.15.151:0/3290539823 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f53b407c1c0 0x7f53b419f260 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:42:37.157 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:36.470+0000 7f53baa00700 1 --2- 172.21.15.151:0/3290539823 >> [v2:172.21.15.151:6800/1128885292,v1:172.21.15.151:6801/1128885292] conn(0x7f539803e420 0x7f53980408e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:42:37.157 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:36.470+0000 7f53baa00700 1 -- 172.21.15.151:0/3290539823 >> 172.21.15.151:0/3290539823 conn(0x7f53b41045a0 msgr2=0x7f53b4105140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:42:37.157 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:36.471+0000 7f53baa00700 1 -- 172.21.15.151:0/3290539823 shutdown_connections 2024-05-21T21:42:37.157 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:36.471+0000 7f53baa00700 1 -- 172.21.15.151:0/3290539823 wait complete. 2024-05-21T21:42:37.157 INFO:teuthology.orchestra.run.smithi151.stdout:mgr epoch 5 is available 2024-05-21T21:42:37.157 INFO:teuthology.orchestra.run.smithi151.stdout:Setting orchestrator backend to cephadm... 2024-05-21T21:42:37.701 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:37 smithi151 ceph-mon[31623]: [21/May/2024:21:42:36] ENGINE Serving on https://172.21.15.151:7150 2024-05-21T21:42:37.701 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:37 smithi151 ceph-mon[31623]: [21/May/2024:21:42:36] ENGINE Bus STARTED 2024-05-21T21:42:37.701 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:37 smithi151 ceph-mon[31623]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-21T21:42:37.701 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:37 smithi151 ceph-mon[31623]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-21T21:42:38.672 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:38 smithi151 ceph-mon[31623]: mgrmap e8: x(active, since 2s) 2024-05-21T21:42:38.673 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:38 smithi151 ceph-mon[31623]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:42:38.673 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:38 smithi151 ceph-mon[31623]: from='mgr.14120 172.21.15.151:0/2204845274' entity='mgr.x' 2024-05-21T21:42:38.673 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:38 smithi151 ceph-mon[31623]: from='mgr.14120 172.21.15.151:0/2204845274' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:42:38.724 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:37.700+0000 7fd28b200700 1 Processor -- start 2024-05-21T21:42:38.726 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:37.700+0000 7fd28b200700 1 -- start start 2024-05-21T21:42:38.726 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:37.701+0000 7fd28b200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd28c10b0a0 0x7fd28c10b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:42:38.726 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:37.701+0000 7fd28b200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd28c10ba90 con 0x7fd28c10b0a0 2024-05-21T21:42:38.726 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:37.701+0000 7fd289e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd28c10b0a0 0x7fd28c10b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:42:38.726 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:37.701+0000 7fd289e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd28c10b0a0 0x7fd28c10b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:36386/0 (socket says 172.21.15.151:36386) 2024-05-21T21:42:38.726 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:37.701+0000 7fd289e00700 1 -- 172.21.15.151:0/2765693539 learned_addr learned my addr 172.21.15.151:0/2765693539 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:42:38.726 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:37.702+0000 7fd289e00700 1 -- 172.21.15.151:0/2765693539 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd28c10c250 con 0x7fd28c10b0a0 2024-05-21T21:42:38.726 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:37.702+0000 7fd289e00700 1 --2- 172.21.15.151:0/2765693539 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd28c10b0a0 0x7fd28c10b4c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd27c0098a0 tx=0x7fd27c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a578f083380b45ca server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:42:38.726 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:37.702+0000 7fd288a00700 1 -- 172.21.15.151:0/2765693539 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd27c004160 con 0x7fd28c10b0a0 2024-05-21T21:42:38.726 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:37.703+0000 7fd288a00700 1 -- 172.21.15.151:0/2765693539 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd27c004300 con 0x7fd28c10b0a0 2024-05-21T21:42:38.726 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:37.703+0000 7fd288a00700 1 -- 172.21.15.151:0/2765693539 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd27c011af0 con 0x7fd28c10b0a0 2024-05-21T21:42:38.730 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:37.703+0000 7fd28b200700 1 -- 172.21.15.151:0/2765693539 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd28c10b0a0 msgr2=0x7fd28c10b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:42:38.731 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:37.703+0000 7fd28b200700 1 --2- 172.21.15.151:0/2765693539 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd28c10b0a0 0x7fd28c10b4c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd27c0098a0 tx=0x7fd27c009bb0 comp rx=0 tx=0).stop 2024-05-21T21:42:38.731 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:37.703+0000 7fd28b200700 1 -- 172.21.15.151:0/2765693539 shutdown_connections 2024-05-21T21:42:38.731 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:37.703+0000 7fd28b200700 1 --2- 172.21.15.151:0/2765693539 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd28c10b0a0 0x7fd28c10b4c0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:42:38.731 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:37.703+0000 7fd28b200700 1 -- 172.21.15.151:0/2765693539 >> 172.21.15.151:0/2765693539 conn(0x7fd28c07e670 msgr2=0x7fd28c07ea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:42:38.731 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:37.703+0000 7fd28b200700 1 -- 172.21.15.151:0/2765693539 shutdown_connections 2024-05-21T21:42:38.731 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:37.704+0000 7fd28b200700 1 -- 172.21.15.151:0/2765693539 wait complete. 2024-05-21T21:42:38.731 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:37.704+0000 7fd28b200700 1 Processor -- start 2024-05-21T21:42:38.731 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:37.705+0000 7fd28b200700 1 -- start start 2024-05-21T21:42:38.731 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:37.705+0000 7fd28b200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd28c10b0a0 0x7fd28c19efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:42:38.731 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:37.705+0000 7fd28b200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd28c10f600 con 0x7fd28c10b0a0 2024-05-21T21:42:38.732 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:37.705+0000 7fd289e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd28c10b0a0 0x7fd28c19efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:42:38.732 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:37.705+0000 7fd289e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd28c10b0a0 0x7fd28c19efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:36392/0 (socket says 172.21.15.151:36392) 2024-05-21T21:42:38.732 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:37.705+0000 7fd289e00700 1 -- 172.21.15.151:0/2924874305 learned_addr learned my addr 172.21.15.151:0/2924874305 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:42:38.732 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:37.706+0000 7fd289e00700 1 -- 172.21.15.151:0/2924874305 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd28c19f500 con 0x7fd28c10b0a0 2024-05-21T21:42:38.732 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:37.706+0000 7fd289e00700 1 --2- 172.21.15.151:0/2924874305 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd28c10b0a0 0x7fd28c19efc0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd27c003f40 tx=0x7fd27c011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:42:38.732 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:37.706+0000 7fd282a00700 1 -- 172.21.15.151:0/2924874305 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd27c0038b0 con 0x7fd28c10b0a0 2024-05-21T21:42:38.732 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:37.706+0000 7fd282a00700 1 -- 172.21.15.151:0/2924874305 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd27c003a50 con 0x7fd28c10b0a0 2024-05-21T21:42:38.732 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:37.706+0000 7fd28b200700 1 -- 172.21.15.151:0/2924874305 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd28c19f790 con 0x7fd28c10b0a0 2024-05-21T21:42:38.732 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:37.706+0000 7fd282a00700 1 -- 172.21.15.151:0/2924874305 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd27c01a5c0 con 0x7fd28c10b0a0 2024-05-21T21:42:38.732 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:37.706+0000 7fd28b200700 1 -- 172.21.15.151:0/2924874305 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd28c1a2480 con 0x7fd28c10b0a0 2024-05-21T21:42:38.732 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:37.708+0000 7fd282a00700 1 -- 172.21.15.151:0/2924874305 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fd27c024430 con 0x7fd28c10b0a0 2024-05-21T21:42:38.732 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:37.708+0000 7fd28b200700 1 -- 172.21.15.151:0/2924874305 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd28c06a6a0 con 0x7fd28c10b0a0 2024-05-21T21:42:38.733 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:37.708+0000 7fd282a00700 1 --2- 172.21.15.151:0/2924874305 >> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] conn(0x7fd27003e420 0x7fd2700408e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:42:38.733 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:37.708+0000 7fd282a00700 1 -- 172.21.15.151:0/2924874305 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd27c053b00 con 0x7fd28c10b0a0 2024-05-21T21:42:38.733 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:37.708+0000 7fd289400700 1 --2- 172.21.15.151:0/2924874305 >> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] conn(0x7fd27003e420 0x7fd2700408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:42:38.733 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:37.709+0000 7fd289400700 1 --2- 172.21.15.151:0/2924874305 >> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] conn(0x7fd27003e420 0x7fd2700408e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd278006fd0 tx=0x7fd278006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:42:38.733 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:37.714+0000 7fd282a00700 1 -- 172.21.15.151:0/2924874305 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd27c029df0 con 0x7fd28c10b0a0 2024-05-21T21:42:38.733 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:37.961+0000 7fd28b200700 1 -- 172.21.15.151:0/2924874305 --> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7fd28c108be0 con 0x7fd27003e420 2024-05-21T21:42:38.733 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:37.976+0000 7fd282a00700 1 -- 172.21.15.151:0/2924874305 <== mgr.14120 v2:172.21.15.151:6800/3428827300 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd28c108be0 con 0x7fd27003e420 2024-05-21T21:42:38.733 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:37.980+0000 7fd28b200700 1 -- 172.21.15.151:0/2924874305 >> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] conn(0x7fd27003e420 msgr2=0x7fd2700408e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:42:38.733 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:37.980+0000 7fd28b200700 1 --2- 172.21.15.151:0/2924874305 >> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] conn(0x7fd27003e420 0x7fd2700408e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd278006fd0 tx=0x7fd278006e40 comp rx=0 tx=0).stop 2024-05-21T21:42:38.733 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:37.980+0000 7fd28b200700 1 -- 172.21.15.151:0/2924874305 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd28c10b0a0 msgr2=0x7fd28c19efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:42:38.733 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:37.980+0000 7fd28b200700 1 --2- 172.21.15.151:0/2924874305 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd28c10b0a0 0x7fd28c19efc0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd27c003f40 tx=0x7fd27c011890 comp rx=0 tx=0).stop 2024-05-21T21:42:38.733 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:37.980+0000 7fd28b200700 1 -- 172.21.15.151:0/2924874305 shutdown_connections 2024-05-21T21:42:38.733 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:37.980+0000 7fd28b200700 1 --2- 172.21.15.151:0/2924874305 >> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] conn(0x7fd27003e420 0x7fd2700408e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:42:38.734 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:37.980+0000 7fd28b200700 1 --2- 172.21.15.151:0/2924874305 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd28c10b0a0 0x7fd28c19efc0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:42:38.734 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:37.980+0000 7fd28b200700 1 -- 172.21.15.151:0/2924874305 >> 172.21.15.151:0/2924874305 conn(0x7fd28c07e670 msgr2=0x7fd28c1084d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:42:38.734 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:37.981+0000 7fd28b200700 1 -- 172.21.15.151:0/2924874305 shutdown_connections 2024-05-21T21:42:38.734 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:37.981+0000 7fd28b200700 1 -- 172.21.15.151:0/2924874305 wait complete. 2024-05-21T21:42:40.143 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout value unchanged 2024-05-21T21:42:40.143 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:39.258+0000 7f051e000700 1 Processor -- start 2024-05-21T21:42:40.144 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:39.258+0000 7f051e000700 1 -- start start 2024-05-21T21:42:40.144 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:39.258+0000 7f051e000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f051810a700 0x7f051810ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:42:40.144 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:39.258+0000 7f051e000700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0518108c40 con 0x7f051810a700 2024-05-21T21:42:40.144 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:39.259+0000 7f0517400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f051810a700 0x7f051810ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:42:40.144 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:39.259+0000 7f0517400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f051810a700 0x7f051810ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:36396/0 (socket says 172.21.15.151:36396) 2024-05-21T21:42:40.144 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:39.259+0000 7f0517400700 1 -- 172.21.15.151:0/2449879059 learned_addr learned my addr 172.21.15.151:0/2449879059 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:42:40.144 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:39.259+0000 7f0517400700 1 -- 172.21.15.151:0/2449879059 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0518108dc0 con 0x7f051810a700 2024-05-21T21:42:40.144 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:39.259+0000 7f0517400700 1 --2- 172.21.15.151:0/2449879059 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f051810a700 0x7f051810ab20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f05080098a0 tx=0x7f0508009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2e936a4ce63a387 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:42:40.144 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:39.260+0000 7f0516000700 1 -- 172.21.15.151:0/2449879059 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0508004160 con 0x7f051810a700 2024-05-21T21:42:40.144 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:39.260+0000 7f0516000700 1 -- 172.21.15.151:0/2449879059 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0508004300 con 0x7f051810a700 2024-05-21T21:42:40.144 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:39.260+0000 7f0516000700 1 -- 172.21.15.151:0/2449879059 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0508011af0 con 0x7f051810a700 2024-05-21T21:42:40.144 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:39.260+0000 7f051e000700 1 -- 172.21.15.151:0/2449879059 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f051810a700 msgr2=0x7f051810ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:42:40.144 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:39.260+0000 7f051e000700 1 --2- 172.21.15.151:0/2449879059 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f051810a700 0x7f051810ab20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f05080098a0 tx=0x7f0508009bb0 comp rx=0 tx=0).stop 2024-05-21T21:42:40.145 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:39.261+0000 7f051e000700 1 -- 172.21.15.151:0/2449879059 shutdown_connections 2024-05-21T21:42:40.145 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:39.261+0000 7f051e000700 1 --2- 172.21.15.151:0/2449879059 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f051810a700 0x7f051810ab20 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:42:40.145 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:39.261+0000 7f051e000700 1 -- 172.21.15.151:0/2449879059 >> 172.21.15.151:0/2449879059 conn(0x7f05181045a0 msgr2=0x7f0518106a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:42:40.145 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:39.261+0000 7f051e000700 1 -- 172.21.15.151:0/2449879059 shutdown_connections 2024-05-21T21:42:40.145 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:39.261+0000 7f051e000700 1 -- 172.21.15.151:0/2449879059 wait complete. 2024-05-21T21:42:40.145 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:39.262+0000 7f051e000700 1 Processor -- start 2024-05-21T21:42:40.145 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:39.262+0000 7f051e000700 1 -- start start 2024-05-21T21:42:40.145 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:39.262+0000 7f051e000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f051810a700 0x7f051819f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:42:40.145 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:39.262+0000 7f051e000700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f051807c140 con 0x7f051810a700 2024-05-21T21:42:40.145 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:39.262+0000 7f0517400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f051810a700 0x7f051819f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:42:40.145 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:39.263+0000 7f0517400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f051810a700 0x7f051819f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:36400/0 (socket says 172.21.15.151:36400) 2024-05-21T21:42:40.145 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:39.263+0000 7f0517400700 1 -- 172.21.15.151:0/3109006252 learned_addr learned my addr 172.21.15.151:0/3109006252 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:42:40.145 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:39.263+0000 7f0517400700 1 -- 172.21.15.151:0/3109006252 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f051819f830 con 0x7f051810a700 2024-05-21T21:42:40.145 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:39.263+0000 7f0517400700 1 --2- 172.21.15.151:0/3109006252 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f051810a700 0x7f051819f2f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0508000c00 tx=0x7f0508011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:42:40.146 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:39.263+0000 7f050fe00700 1 -- 172.21.15.151:0/3109006252 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f05080039e0 con 0x7f051810a700 2024-05-21T21:42:40.146 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:39.263+0000 7f050fe00700 1 -- 172.21.15.151:0/3109006252 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0508003b80 con 0x7f051810a700 2024-05-21T21:42:40.146 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:39.263+0000 7f051e000700 1 -- 172.21.15.151:0/3109006252 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f051819fac0 con 0x7f051810a700 2024-05-21T21:42:40.146 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:39.263+0000 7f050fe00700 1 -- 172.21.15.151:0/3109006252 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0508024730 con 0x7f051810a700 2024-05-21T21:42:40.146 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:39.264+0000 7f051e000700 1 -- 172.21.15.151:0/3109006252 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05181a27b0 con 0x7f051810a700 2024-05-21T21:42:40.146 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:39.265+0000 7f050fe00700 1 -- 172.21.15.151:0/3109006252 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f0508022020 con 0x7f051810a700 2024-05-21T21:42:40.146 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:39.265+0000 7f051e000700 1 -- 172.21.15.151:0/3109006252 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f051806a6a0 con 0x7f051810a700 2024-05-21T21:42:40.146 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:39.265+0000 7f050fe00700 1 --2- 172.21.15.151:0/3109006252 >> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] conn(0x7f04fc03e4c0 0x7f04fc040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:42:40.146 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:39.265+0000 7f050fe00700 1 -- 172.21.15.151:0/3109006252 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f0508053790 con 0x7f051810a700 2024-05-21T21:42:40.146 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:39.266+0000 7f0516a00700 1 --2- 172.21.15.151:0/3109006252 >> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] conn(0x7f04fc03e4c0 0x7f04fc040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:42:40.146 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:39.266+0000 7f0516a00700 1 --2- 172.21.15.151:0/3109006252 >> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] conn(0x7f04fc03e4c0 0x7f04fc040980 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0504006fd0 tx=0x7f0504006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:42:40.146 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:39.271+0000 7f050fe00700 1 -- 172.21.15.151:0/3109006252 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f050801ac00 con 0x7f051810a700 2024-05-21T21:42:40.147 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:39.504+0000 7f051e000700 1 -- 172.21.15.151:0/3109006252 --> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f051802e790 con 0x7f04fc03e4c0 2024-05-21T21:42:40.147 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:39.506+0000 7f050fe00700 1 -- 172.21.15.151:0/3109006252 <== mgr.14120 v2:172.21.15.151:6800/3428827300 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f051802e790 con 0x7f04fc03e4c0 2024-05-21T21:42:40.147 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:39.510+0000 7f051e000700 1 -- 172.21.15.151:0/3109006252 >> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] conn(0x7f04fc03e4c0 msgr2=0x7f04fc040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:42:40.147 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:39.510+0000 7f051e000700 1 --2- 172.21.15.151:0/3109006252 >> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] conn(0x7f04fc03e4c0 0x7f04fc040980 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0504006fd0 tx=0x7f0504006e40 comp rx=0 tx=0).stop 2024-05-21T21:42:40.147 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:39.510+0000 7f051e000700 1 -- 172.21.15.151:0/3109006252 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f051810a700 msgr2=0x7f051819f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:42:40.147 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:39.510+0000 7f051e000700 1 --2- 172.21.15.151:0/3109006252 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f051810a700 0x7f051819f2f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0508000c00 tx=0x7f0508011b70 comp rx=0 tx=0).stop 2024-05-21T21:42:40.147 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:39.510+0000 7f051e000700 1 -- 172.21.15.151:0/3109006252 shutdown_connections 2024-05-21T21:42:40.147 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:39.510+0000 7f051e000700 1 --2- 172.21.15.151:0/3109006252 >> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] conn(0x7f04fc03e4c0 0x7f04fc040980 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:42:40.147 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:39.510+0000 7f051e000700 1 --2- 172.21.15.151:0/3109006252 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f051810a700 0x7f051819f2f0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:42:40.147 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:39.510+0000 7f051e000700 1 -- 172.21.15.151:0/3109006252 >> 172.21.15.151:0/3109006252 conn(0x7f05181045a0 msgr2=0x7f0518196570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:42:40.147 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:39.510+0000 7f051e000700 1 -- 172.21.15.151:0/3109006252 shutdown_connections 2024-05-21T21:42:40.147 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:39.510+0000 7f051e000700 1 -- 172.21.15.151:0/3109006252 wait complete. 2024-05-21T21:42:40.147 INFO:teuthology.orchestra.run.smithi151.stdout:Generating ssh key... 2024-05-21T21:42:40.688 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:40 smithi151 ceph-mon[31623]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:42:41.454 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:42:41 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: Generating public/private rsa key pair. 2024-05-21T21:42:41.454 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:42:41 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: Your identification has been saved in /tmp/tmppmova90r/key. 2024-05-21T21:42:41.455 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:42:41 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: Your public key has been saved in /tmp/tmppmova90r/key.pub. 2024-05-21T21:42:41.455 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:42:41 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: The key fingerprint is: 2024-05-21T21:42:41.455 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:42:41 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: SHA256:NIE50giHU574x0STUW3Eo3EktwSvNW6g5ictkIQMA/A ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968 2024-05-21T21:42:41.455 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:42:41 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: The key's randomart image is: 2024-05-21T21:42:41.455 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:42:41 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: +---[RSA 3072]----+ 2024-05-21T21:42:41.455 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:42:41 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: |B .+oo+*=*= | 2024-05-21T21:42:41.455 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:42:41 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: | =o=oo*..**. | 2024-05-21T21:42:41.455 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:42:41 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: | E.+...+==. | 2024-05-21T21:42:41.455 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:42:41 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: | o + o.* . | 2024-05-21T21:42:41.455 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:42:41 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: | + = S o | 2024-05-21T21:42:41.455 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:42:41 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: | = . . | 2024-05-21T21:42:41.455 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:42:41 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: | + o | 2024-05-21T21:42:41.455 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:42:41 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: | + | 2024-05-21T21:42:41.455 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:42:41 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: | | 2024-05-21T21:42:41.456 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:42:41 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: +----[SHA256]-----+ 2024-05-21T21:42:41.785 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:40.761+0000 7f2633800700 1 Processor -- start 2024-05-21T21:42:41.785 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:40.762+0000 7f2633800700 1 -- start start 2024-05-21T21:42:41.786 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:40.762+0000 7f2633800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f262c10ff20 0x7f262c112310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:42:41.786 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:40.762+0000 7f2633800700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f262c1128e0 con 0x7f262c10ff20 2024-05-21T21:42:41.786 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:40.762+0000 7f2630e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f262c10ff20 0x7f262c112310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:42:41.786 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:40.763+0000 7f2630e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f262c10ff20 0x7f262c112310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:36402/0 (socket says 172.21.15.151:36402) 2024-05-21T21:42:41.786 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:40.763+0000 7f2630e00700 1 -- 172.21.15.151:0/3565614919 learned_addr learned my addr 172.21.15.151:0/3565614919 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:42:41.786 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:40.763+0000 7f2630e00700 1 -- 172.21.15.151:0/3565614919 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f262c112a60 con 0x7f262c10ff20 2024-05-21T21:42:41.786 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:40.763+0000 7f2630e00700 1 --2- 172.21.15.151:0/3565614919 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f262c10ff20 0x7f262c112310 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f26200098a0 tx=0x7f2620009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc6e81380414d741 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:42:41.790 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:40.764+0000 7f262b400700 1 -- 172.21.15.151:0/3565614919 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2620004160 con 0x7f262c10ff20 2024-05-21T21:42:41.790 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:40.764+0000 7f262b400700 1 -- 172.21.15.151:0/3565614919 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2620004300 con 0x7f262c10ff20 2024-05-21T21:42:41.791 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:40.764+0000 7f262b400700 1 -- 172.21.15.151:0/3565614919 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2620011af0 con 0x7f262c10ff20 2024-05-21T21:42:41.791 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:40.764+0000 7f2633800700 1 -- 172.21.15.151:0/3565614919 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f262c10ff20 msgr2=0x7f262c112310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:42:41.791 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:40.764+0000 7f2633800700 1 --2- 172.21.15.151:0/3565614919 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f262c10ff20 0x7f262c112310 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f26200098a0 tx=0x7f2620009bb0 comp rx=0 tx=0).stop 2024-05-21T21:42:41.791 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:40.765+0000 7f2633800700 1 -- 172.21.15.151:0/3565614919 shutdown_connections 2024-05-21T21:42:41.791 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:40.765+0000 7f2633800700 1 --2- 172.21.15.151:0/3565614919 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f262c10ff20 0x7f262c112310 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:42:41.791 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:40.765+0000 7f2633800700 1 -- 172.21.15.151:0/3565614919 >> 172.21.15.151:0/3565614919 conn(0x7f262c07e890 msgr2=0x7f262c080cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:42:41.791 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:40.765+0000 7f2633800700 1 -- 172.21.15.151:0/3565614919 shutdown_connections 2024-05-21T21:42:41.791 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:40.765+0000 7f2633800700 1 -- 172.21.15.151:0/3565614919 wait complete. 2024-05-21T21:42:41.791 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:40.766+0000 7f2633800700 1 Processor -- start 2024-05-21T21:42:41.791 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:40.766+0000 7f2633800700 1 -- start start 2024-05-21T21:42:41.791 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:40.766+0000 7f2633800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f262c10ff20 0x7f262c1a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:42:41.791 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:40.766+0000 7f2633800700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f262c113a90 con 0x7f262c10ff20 2024-05-21T21:42:41.791 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:40.767+0000 7f2630e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f262c10ff20 0x7f262c1a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:42:41.792 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:40.767+0000 7f2630e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f262c10ff20 0x7f262c1a3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:36404/0 (socket says 172.21.15.151:36404) 2024-05-21T21:42:41.792 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:40.767+0000 7f2630e00700 1 -- 172.21.15.151:0/3885029058 learned_addr learned my addr 172.21.15.151:0/3885029058 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:42:41.792 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:40.767+0000 7f2630e00700 1 -- 172.21.15.151:0/3885029058 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f262c1a3b70 con 0x7f262c10ff20 2024-05-21T21:42:41.792 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:40.767+0000 7f2630e00700 1 --2- 172.21.15.151:0/3885029058 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f262c10ff20 0x7f262c1a3630 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2620011ac0 tx=0x7f2620011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:42:41.792 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:40.768+0000 7f2629600700 1 -- 172.21.15.151:0/3885029058 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2620003b20 con 0x7f262c10ff20 2024-05-21T21:42:41.792 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:40.768+0000 7f2629600700 1 -- 172.21.15.151:0/3885029058 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f262001a8c0 con 0x7f262c10ff20 2024-05-21T21:42:41.792 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:40.768+0000 7f2633800700 1 -- 172.21.15.151:0/3885029058 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f262c1a3e00 con 0x7f262c10ff20 2024-05-21T21:42:41.792 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:40.768+0000 7f2629600700 1 -- 172.21.15.151:0/3885029058 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f26200248d0 con 0x7f262c10ff20 2024-05-21T21:42:41.792 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:40.768+0000 7f2633800700 1 -- 172.21.15.151:0/3885029058 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f262c1a6af0 con 0x7f262c10ff20 2024-05-21T21:42:41.792 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:40.769+0000 7f2629600700 1 -- 172.21.15.151:0/3885029058 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f2620022020 con 0x7f262c10ff20 2024-05-21T21:42:41.792 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:40.769+0000 7f2633800700 1 -- 172.21.15.151:0/3885029058 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f262c064ba0 con 0x7f262c10ff20 2024-05-21T21:42:41.792 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:40.770+0000 7f2629600700 1 --2- 172.21.15.151:0/3885029058 >> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] conn(0x7f261403e470 0x7f2614040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:42:41.792 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:40.770+0000 7f2629600700 1 -- 172.21.15.151:0/3885029058 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f2620053860 con 0x7f262c10ff20 2024-05-21T21:42:41.792 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:40.770+0000 7f262be00700 1 --2- 172.21.15.151:0/3885029058 >> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] conn(0x7f261403e470 0x7f2614040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:42:41.793 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:40.771+0000 7f262be00700 1 --2- 172.21.15.151:0/3885029058 >> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] conn(0x7f261403e470 0x7f2614040930 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2618006fd0 tx=0x7f2618006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:42:41.793 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:40.775+0000 7f2629600700 1 -- 172.21.15.151:0/3885029058 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f262001ab20 con 0x7f262c10ff20 2024-05-21T21:42:41.793 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:41.005+0000 7f2633800700 1 -- 172.21.15.151:0/3885029058 --> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f262c080ac0 con 0x7f261403e470 2024-05-21T21:42:41.793 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:41.146+0000 7f2629600700 1 -- 172.21.15.151:0/3885029058 <== mgr.14120 v2:172.21.15.151:6800/3428827300 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f262c080ac0 con 0x7f261403e470 2024-05-21T21:42:41.793 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:41.149+0000 7f2633800700 1 -- 172.21.15.151:0/3885029058 >> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] conn(0x7f261403e470 msgr2=0x7f2614040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:42:41.793 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:41.149+0000 7f2633800700 1 --2- 172.21.15.151:0/3885029058 >> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] conn(0x7f261403e470 0x7f2614040930 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2618006fd0 tx=0x7f2618006e40 comp rx=0 tx=0).stop 2024-05-21T21:42:41.793 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:41.149+0000 7f2633800700 1 -- 172.21.15.151:0/3885029058 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f262c10ff20 msgr2=0x7f262c1a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:42:41.793 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:41.149+0000 7f2633800700 1 --2- 172.21.15.151:0/3885029058 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f262c10ff20 0x7f262c1a3630 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2620011ac0 tx=0x7f2620011ba0 comp rx=0 tx=0).stop 2024-05-21T21:42:41.793 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:41.150+0000 7f2633800700 1 -- 172.21.15.151:0/3885029058 shutdown_connections 2024-05-21T21:42:41.793 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:41.150+0000 7f2633800700 1 --2- 172.21.15.151:0/3885029058 >> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] conn(0x7f261403e470 0x7f2614040930 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:42:41.793 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:41.150+0000 7f2633800700 1 --2- 172.21.15.151:0/3885029058 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f262c10ff20 0x7f262c1a3630 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:42:41.793 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:41.150+0000 7f2633800700 1 -- 172.21.15.151:0/3885029058 >> 172.21.15.151:0/3885029058 conn(0x7f262c07e890 msgr2=0x7f262c0803b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:42:41.793 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:41.150+0000 7f2633800700 1 -- 172.21.15.151:0/3885029058 shutdown_connections 2024-05-21T21:42:41.793 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:41.150+0000 7f2633800700 1 -- 172.21.15.151:0/3885029058 wait complete. 2024-05-21T21:42:42.342 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:42 smithi151 ceph-mon[31623]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:42:42.342 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:42 smithi151 ceph-mon[31623]: Generating ssh key... 2024-05-21T21:42:42.342 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:42 smithi151 ceph-mon[31623]: from='mgr.14120 172.21.15.151:0/2204845274' entity='mgr.x' 2024-05-21T21:42:42.342 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:42 smithi151 ceph-mon[31623]: from='mgr.14120 172.21.15.151:0/2204845274' entity='mgr.x' 2024-05-21T21:42:43.197 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMwzrK6w5hxzJfOlJIl/zsewwuE1g7rOVKruibSweE6Q7QM303wVSPIN4PTeIffgAykZNYK85HCx0HEZzXuEOVDeB7O+VqOCQximU49wY6poD0pIbX8FOxeQS6m4CNBXVnk7Fdb9WHWlEFdg2d0OmKPlW9MnonGiMoILHm3fa7QoOBlO8YhmQ4/66gKN/TbqV6WQLmqvEkDcznRQvC4Bepq2E2m3LBmElw3J6Jiq+/fdQM1GQhBa6DDKg20wU+T7bxEmEZAsuV2wi8W2odsmULlPPj189IgArrJBUZkz0/baIIJqktWcbaUZ34Z5d9PFBEs6rfvmlRSX3QpNQ6muBkFJXm/DrgfgJ6it/+JmK+Do1By7v6y98HYD4nrV98PkC+S7rfx1Xzj+vPil0OTOaagfXvqRIxP8BFY9lb7b/AZy1TbZ4sx685siMN802ZQf8NRH3r/m2yYkqI03hDpI7SB8QbjIu8knzVfT/4+XLZWOaVlJEyY+2nll/QSYiJAl8= ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968 2024-05-21T21:42:43.198 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:42.341+0000 7fd65e200700 1 Processor -- start 2024-05-21T21:42:43.198 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:42.342+0000 7fd65e200700 1 -- start start 2024-05-21T21:42:43.198 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:42.342+0000 7fd65e200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd658108eb0 0x7fd6581092d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:42:43.198 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:42.342+0000 7fd65e200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd658109810 con 0x7fd658108eb0 2024-05-21T21:42:43.198 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:42.343+0000 7fd65ce00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd658108eb0 0x7fd6581092d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:42:43.198 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:42.343+0000 7fd65ce00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd658108eb0 0x7fd6581092d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:36408/0 (socket says 172.21.15.151:36408) 2024-05-21T21:42:43.198 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:42.343+0000 7fd65ce00700 1 -- 172.21.15.151:0/1064447472 learned_addr learned my addr 172.21.15.151:0/1064447472 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:42:43.198 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:42.344+0000 7fd65ce00700 1 -- 172.21.15.151:0/1064447472 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd658109990 con 0x7fd658108eb0 2024-05-21T21:42:43.198 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:42.344+0000 7fd65ce00700 1 --2- 172.21.15.151:0/1064447472 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd658108eb0 0x7fd6581092d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd64c0098a0 tx=0x7fd64c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca307a8bc0bc2eb5 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:42:43.198 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:42.344+0000 7fd657400700 1 -- 172.21.15.151:0/1064447472 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd64c004160 con 0x7fd658108eb0 2024-05-21T21:42:43.198 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:42.344+0000 7fd657400700 1 -- 172.21.15.151:0/1064447472 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd64c004300 con 0x7fd658108eb0 2024-05-21T21:42:43.198 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:42.344+0000 7fd657400700 1 -- 172.21.15.151:0/1064447472 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd64c011af0 con 0x7fd658108eb0 2024-05-21T21:42:43.199 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:42.345+0000 7fd65e200700 1 -- 172.21.15.151:0/1064447472 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd658108eb0 msgr2=0x7fd6581092d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:42:43.199 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:42.345+0000 7fd65e200700 1 --2- 172.21.15.151:0/1064447472 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd658108eb0 0x7fd6581092d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd64c0098a0 tx=0x7fd64c009bb0 comp rx=0 tx=0).stop 2024-05-21T21:42:43.199 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:42.345+0000 7fd65e200700 1 -- 172.21.15.151:0/1064447472 shutdown_connections 2024-05-21T21:42:43.199 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:42.345+0000 7fd65e200700 1 --2- 172.21.15.151:0/1064447472 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd658108eb0 0x7fd6581092d0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:42:43.199 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:42.345+0000 7fd65e200700 1 -- 172.21.15.151:0/1064447472 >> 172.21.15.151:0/1064447472 conn(0x7fd658104510 msgr2=0x7fd658106930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:42:43.199 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:42.345+0000 7fd65e200700 1 -- 172.21.15.151:0/1064447472 shutdown_connections 2024-05-21T21:42:43.201 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:42.345+0000 7fd65e200700 1 -- 172.21.15.151:0/1064447472 wait complete. 2024-05-21T21:42:43.201 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:42.346+0000 7fd65e200700 1 Processor -- start 2024-05-21T21:42:43.201 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:42.346+0000 7fd65e200700 1 -- start start 2024-05-21T21:42:43.201 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:42.347+0000 7fd65e200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd658108eb0 0x7fd65807eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:42:43.201 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:42.347+0000 7fd65e200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd65810ae40 con 0x7fd658108eb0 2024-05-21T21:42:43.201 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:42.347+0000 7fd65ce00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd658108eb0 0x7fd65807eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:42:43.202 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:42.347+0000 7fd65ce00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd658108eb0 0x7fd65807eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:36418/0 (socket says 172.21.15.151:36418) 2024-05-21T21:42:43.202 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:42.347+0000 7fd65ce00700 1 -- 172.21.15.151:0/3139135975 learned_addr learned my addr 172.21.15.151:0/3139135975 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:42:43.202 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:42.347+0000 7fd65ce00700 1 -- 172.21.15.151:0/3139135975 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd65807f0a0 con 0x7fd658108eb0 2024-05-21T21:42:43.202 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:42.347+0000 7fd65ce00700 1 --2- 172.21.15.151:0/3139135975 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd658108eb0 0x7fd65807eb60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd64c000c00 tx=0x7fd64c011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:42:43.202 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:42.348+0000 7fd655600700 1 -- 172.21.15.151:0/3139135975 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd64c003b20 con 0x7fd658108eb0 2024-05-21T21:42:43.202 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:42.348+0000 7fd655600700 1 -- 172.21.15.151:0/3139135975 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd64c01a8c0 con 0x7fd658108eb0 2024-05-21T21:42:43.202 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:42.348+0000 7fd655600700 1 -- 172.21.15.151:0/3139135975 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd64c024940 con 0x7fd658108eb0 2024-05-21T21:42:43.202 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:42.348+0000 7fd65e200700 1 -- 172.21.15.151:0/3139135975 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd65807d290 con 0x7fd658108eb0 2024-05-21T21:42:43.202 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:42.348+0000 7fd65e200700 1 -- 172.21.15.151:0/3139135975 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd65807d7d0 con 0x7fd658108eb0 2024-05-21T21:42:43.202 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:42.349+0000 7fd65e200700 1 -- 172.21.15.151:0/3139135975 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd65806a6a0 con 0x7fd658108eb0 2024-05-21T21:42:43.202 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:42.349+0000 7fd655600700 1 -- 172.21.15.151:0/3139135975 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fd64c022020 con 0x7fd658108eb0 2024-05-21T21:42:43.202 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:42.350+0000 7fd655600700 1 --2- 172.21.15.151:0/3139135975 >> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] conn(0x7fd64003e4c0 0x7fd640040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:42:43.202 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:42.350+0000 7fd655600700 1 -- 172.21.15.151:0/3139135975 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd64c053730 con 0x7fd658108eb0 2024-05-21T21:42:43.203 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:42.350+0000 7fd657e00700 1 --2- 172.21.15.151:0/3139135975 >> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] conn(0x7fd64003e4c0 0x7fd640040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:42:43.203 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:42.351+0000 7fd657e00700 1 --2- 172.21.15.151:0/3139135975 >> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] conn(0x7fd64003e4c0 0x7fd640040980 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd648006fd0 tx=0x7fd648006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:42:43.203 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:42.356+0000 7fd655600700 1 -- 172.21.15.151:0/3139135975 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd64c025e90 con 0x7fd658108eb0 2024-05-21T21:42:43.203 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:42.589+0000 7fd65e200700 1 -- 172.21.15.151:0/3139135975 --> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7fd65807e100 con 0x7fd64003e4c0 2024-05-21T21:42:43.203 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:42.590+0000 7fd655600700 1 -- 172.21.15.151:0/3139135975 <== mgr.14120 v2:172.21.15.151:6800/3428827300 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7fd65807e100 con 0x7fd64003e4c0 2024-05-21T21:42:43.203 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:42.593+0000 7fd65e200700 1 -- 172.21.15.151:0/3139135975 >> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] conn(0x7fd64003e4c0 msgr2=0x7fd640040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:42:43.203 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:42.593+0000 7fd65e200700 1 --2- 172.21.15.151:0/3139135975 >> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] conn(0x7fd64003e4c0 0x7fd640040980 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd648006fd0 tx=0x7fd648006e40 comp rx=0 tx=0).stop 2024-05-21T21:42:43.203 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:42.593+0000 7fd65e200700 1 -- 172.21.15.151:0/3139135975 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd658108eb0 msgr2=0x7fd65807eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:42:43.203 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:42.593+0000 7fd65e200700 1 --2- 172.21.15.151:0/3139135975 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd658108eb0 0x7fd65807eb60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd64c000c00 tx=0x7fd64c011b70 comp rx=0 tx=0).stop 2024-05-21T21:42:43.203 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:42.594+0000 7fd65e200700 1 -- 172.21.15.151:0/3139135975 shutdown_connections 2024-05-21T21:42:43.203 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:42.594+0000 7fd65e200700 1 --2- 172.21.15.151:0/3139135975 >> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] conn(0x7fd64003e4c0 0x7fd640040980 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:42:43.203 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:42.594+0000 7fd65e200700 1 --2- 172.21.15.151:0/3139135975 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd658108eb0 0x7fd65807eb60 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:42:43.203 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:42.594+0000 7fd65e200700 1 -- 172.21.15.151:0/3139135975 >> 172.21.15.151:0/3139135975 conn(0x7fd658104510 msgr2=0x7fd658105080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:42:43.204 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:42.594+0000 7fd65e200700 1 -- 172.21.15.151:0/3139135975 shutdown_connections 2024-05-21T21:42:43.204 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:42.594+0000 7fd65e200700 1 -- 172.21.15.151:0/3139135975 wait complete. 2024-05-21T21:42:43.204 INFO:teuthology.orchestra.run.smithi151.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-05-21T21:42:43.204 INFO:teuthology.orchestra.run.smithi151.stdout:Adding key to root@localhost authorized_keys... 2024-05-21T21:42:43.204 INFO:teuthology.orchestra.run.smithi151.stdout:Adding host smithi151... 2024-05-21T21:42:43.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:43 smithi151 ceph-mon[31623]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:42:44.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:44 smithi151 ceph-mon[31623]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi151", "addr": "172.21.15.151", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:42:45.641 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:45 smithi151 ceph-mon[31623]: Deploying cephadm binary to smithi151 2024-05-21T21:42:50.685 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout Added host 'smithi151' with addr '172.21.15.151' 2024-05-21T21:42:50.685 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:43.834+0000 7f2c5f200700 1 Processor -- start 2024-05-21T21:42:50.685 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:43.835+0000 7f2c5f200700 1 -- start start 2024-05-21T21:42:50.685 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:43.835+0000 7f2c5f200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c5810ff20 0x7f2c58112310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:42:50.685 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:43.835+0000 7f2c5f200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2c581128e0 con 0x7f2c5810ff20 2024-05-21T21:42:50.685 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:43.836+0000 7f2c57e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c5810ff20 0x7f2c58112310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:42:50.685 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:43.836+0000 7f2c57e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c5810ff20 0x7f2c58112310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:36430/0 (socket says 172.21.15.151:36430) 2024-05-21T21:42:50.686 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:43.836+0000 7f2c57e00700 1 -- 172.21.15.151:0/2986446761 learned_addr learned my addr 172.21.15.151:0/2986446761 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:42:50.686 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:43.836+0000 7f2c57e00700 1 -- 172.21.15.151:0/2986446761 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c58112a60 con 0x7f2c5810ff20 2024-05-21T21:42:50.686 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:43.837+0000 7f2c57e00700 1 --2- 172.21.15.151:0/2986446761 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c5810ff20 0x7f2c58112310 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2c480098a0 tx=0x7f2c48009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=995188430a4e152d server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:42:50.686 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:43.837+0000 7f2c56a00700 1 -- 172.21.15.151:0/2986446761 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c48004160 con 0x7f2c5810ff20 2024-05-21T21:42:50.686 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:43.837+0000 7f2c56a00700 1 -- 172.21.15.151:0/2986446761 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2c48004300 con 0x7f2c5810ff20 2024-05-21T21:42:50.686 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:43.837+0000 7f2c56a00700 1 -- 172.21.15.151:0/2986446761 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c48011af0 con 0x7f2c5810ff20 2024-05-21T21:42:50.686 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:43.838+0000 7f2c5f200700 1 -- 172.21.15.151:0/2986446761 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c5810ff20 msgr2=0x7f2c58112310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:42:50.686 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:43.838+0000 7f2c5f200700 1 --2- 172.21.15.151:0/2986446761 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c5810ff20 0x7f2c58112310 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2c480098a0 tx=0x7f2c48009bb0 comp rx=0 tx=0).stop 2024-05-21T21:42:50.686 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:43.838+0000 7f2c5f200700 1 -- 172.21.15.151:0/2986446761 shutdown_connections 2024-05-21T21:42:50.686 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:43.838+0000 7f2c5f200700 1 --2- 172.21.15.151:0/2986446761 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c5810ff20 0x7f2c58112310 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:42:50.687 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:43.838+0000 7f2c5f200700 1 -- 172.21.15.151:0/2986446761 >> 172.21.15.151:0/2986446761 conn(0x7f2c5807e890 msgr2=0x7f2c58080cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:42:50.687 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:43.838+0000 7f2c5f200700 1 -- 172.21.15.151:0/2986446761 shutdown_connections 2024-05-21T21:42:50.687 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:43.838+0000 7f2c5f200700 1 -- 172.21.15.151:0/2986446761 wait complete. 2024-05-21T21:42:50.687 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:43.839+0000 7f2c5f200700 1 Processor -- start 2024-05-21T21:42:50.687 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:43.839+0000 7f2c5f200700 1 -- start start 2024-05-21T21:42:50.687 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:43.839+0000 7f2c5f200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c5810ff20 0x7f2c581a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:42:50.687 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:43.839+0000 7f2c5f200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2c58113a90 con 0x7f2c5810ff20 2024-05-21T21:42:50.687 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:43.840+0000 7f2c57e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c5810ff20 0x7f2c581a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:42:50.687 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:43.840+0000 7f2c57e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c5810ff20 0x7f2c581a3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:36434/0 (socket says 172.21.15.151:36434) 2024-05-21T21:42:50.687 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:43.840+0000 7f2c57e00700 1 -- 172.21.15.151:0/3810669230 learned_addr learned my addr 172.21.15.151:0/3810669230 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:42:50.688 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:43.840+0000 7f2c57e00700 1 -- 172.21.15.151:0/3810669230 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c581a3b70 con 0x7f2c5810ff20 2024-05-21T21:42:50.688 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:43.840+0000 7f2c57e00700 1 --2- 172.21.15.151:0/3810669230 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c5810ff20 0x7f2c581a3630 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2c48011ac0 tx=0x7f2c48011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:42:50.688 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:43.840+0000 7f2c54c00700 1 -- 172.21.15.151:0/3810669230 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c48003b20 con 0x7f2c5810ff20 2024-05-21T21:42:50.688 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:43.841+0000 7f2c54c00700 1 -- 172.21.15.151:0/3810669230 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2c4801a8c0 con 0x7f2c5810ff20 2024-05-21T21:42:50.688 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:43.841+0000 7f2c5f200700 1 -- 172.21.15.151:0/3810669230 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c581a3e00 con 0x7f2c5810ff20 2024-05-21T21:42:50.688 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:43.841+0000 7f2c54c00700 1 -- 172.21.15.151:0/3810669230 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c480248d0 con 0x7f2c5810ff20 2024-05-21T21:42:50.688 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:43.841+0000 7f2c5f200700 1 -- 172.21.15.151:0/3810669230 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c581a6af0 con 0x7f2c5810ff20 2024-05-21T21:42:50.688 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:43.842+0000 7f2c54c00700 1 -- 172.21.15.151:0/3810669230 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f2c48022020 con 0x7f2c5810ff20 2024-05-21T21:42:50.688 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:43.842+0000 7f2c5f200700 1 -- 172.21.15.151:0/3810669230 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c58064ba0 con 0x7f2c5810ff20 2024-05-21T21:42:50.688 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:43.842+0000 7f2c54c00700 1 --2- 172.21.15.151:0/3810669230 >> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] conn(0x7f2c3c03e4c0 0x7f2c3c040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:42:50.688 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:43.842+0000 7f2c54c00700 1 -- 172.21.15.151:0/3810669230 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f2c48053810 con 0x7f2c5810ff20 2024-05-21T21:42:50.688 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:43.843+0000 7f2c57400700 1 --2- 172.21.15.151:0/3810669230 >> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] conn(0x7f2c3c03e4c0 0x7f2c3c040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:42:50.689 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:43.843+0000 7f2c57400700 1 --2- 172.21.15.151:0/3810669230 >> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] conn(0x7f2c3c03e4c0 0x7f2c3c040980 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2c44006fd0 tx=0x7f2c44006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:42:50.689 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:43.848+0000 7f2c54c00700 1 -- 172.21.15.151:0/3810669230 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2c4801ab20 con 0x7f2c5810ff20 2024-05-21T21:42:50.689 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:44.083+0000 7f2c5f200700 1 -- 172.21.15.151:0/3810669230 --> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi151", "addr": "172.21.15.151", "target": ["mon-mgr", ""]}) -- 0x7f2c58080ac0 con 0x7f2c3c03e4c0 2024-05-21T21:42:50.689 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:49.990+0000 7f2c54c00700 1 -- 172.21.15.151:0/3810669230 <== mgr.14120 v2:172.21.15.151:6800/3428827300 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f2c58080ac0 con 0x7f2c3c03e4c0 2024-05-21T21:42:50.689 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:49.995+0000 7f2c5f200700 1 -- 172.21.15.151:0/3810669230 >> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] conn(0x7f2c3c03e4c0 msgr2=0x7f2c3c040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:42:50.689 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:49.995+0000 7f2c5f200700 1 --2- 172.21.15.151:0/3810669230 >> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] conn(0x7f2c3c03e4c0 0x7f2c3c040980 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2c44006fd0 tx=0x7f2c44006e40 comp rx=0 tx=0).stop 2024-05-21T21:42:50.689 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:49.995+0000 7f2c5f200700 1 -- 172.21.15.151:0/3810669230 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c5810ff20 msgr2=0x7f2c581a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:42:50.689 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:49.995+0000 7f2c5f200700 1 --2- 172.21.15.151:0/3810669230 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c5810ff20 0x7f2c581a3630 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2c48011ac0 tx=0x7f2c48011ba0 comp rx=0 tx=0).stop 2024-05-21T21:42:50.689 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:49.995+0000 7f2c5f200700 1 -- 172.21.15.151:0/3810669230 shutdown_connections 2024-05-21T21:42:50.689 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:49.995+0000 7f2c5f200700 1 --2- 172.21.15.151:0/3810669230 >> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] conn(0x7f2c3c03e4c0 0x7f2c3c040980 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:42:50.689 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:49.995+0000 7f2c5f200700 1 --2- 172.21.15.151:0/3810669230 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c5810ff20 0x7f2c581a3630 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:42:50.690 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:49.995+0000 7f2c5f200700 1 -- 172.21.15.151:0/3810669230 >> 172.21.15.151:0/3810669230 conn(0x7f2c5807e890 msgr2=0x7f2c580803b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:42:50.690 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:49.995+0000 7f2c5f200700 1 -- 172.21.15.151:0/3810669230 shutdown_connections 2024-05-21T21:42:50.690 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:49.995+0000 7f2c5f200700 1 -- 172.21.15.151:0/3810669230 wait complete. 2024-05-21T21:42:50.690 INFO:teuthology.orchestra.run.smithi151.stdout:Deploying unmanaged mon service... 2024-05-21T21:42:51.319 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:50 smithi151 ceph-mon[31623]: from='mgr.14120 172.21.15.151:0/2204845274' entity='mgr.x' 2024-05-21T21:42:51.319 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:50 smithi151 ceph-mon[31623]: Added host smithi151 2024-05-21T21:42:51.319 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:50 smithi151 ceph-mon[31623]: from='mgr.14120 172.21.15.151:0/2204845274' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:42:52.747 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-05-21T21:42:52.747 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:51.341+0000 7fd94a200700 1 Processor -- start 2024-05-21T21:42:52.747 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:51.341+0000 7fd94a200700 1 -- start start 2024-05-21T21:42:52.747 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:51.342+0000 7fd94a200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd94410b270 0x7fd94410b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:42:52.748 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:51.342+0000 7fd94a200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd94410bc60 con 0x7fd94410b270 2024-05-21T21:42:52.748 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:51.342+0000 7fd943400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd94410b270 0x7fd94410b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:42:52.748 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:51.342+0000 7fd943400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd94410b270 0x7fd94410b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:38096/0 (socket says 172.21.15.151:38096) 2024-05-21T21:42:52.748 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:51.342+0000 7fd943400700 1 -- 172.21.15.151:0/666988860 learned_addr learned my addr 172.21.15.151:0/666988860 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:42:52.748 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:51.343+0000 7fd943400700 1 -- 172.21.15.151:0/666988860 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd94410c420 con 0x7fd94410b270 2024-05-21T21:42:52.748 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:51.343+0000 7fd943400700 1 --2- 172.21.15.151:0/666988860 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd94410b270 0x7fd94410b690 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd9340098a0 tx=0x7fd934009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f8e47ff6df8f060 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:42:52.748 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:51.343+0000 7fd942a00700 1 -- 172.21.15.151:0/666988860 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd934004160 con 0x7fd94410b270 2024-05-21T21:42:52.748 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:51.343+0000 7fd942a00700 1 -- 172.21.15.151:0/666988860 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd934004300 con 0x7fd94410b270 2024-05-21T21:42:52.748 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:51.343+0000 7fd942a00700 1 -- 172.21.15.151:0/666988860 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd934011af0 con 0x7fd94410b270 2024-05-21T21:42:52.748 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:51.344+0000 7fd94a200700 1 -- 172.21.15.151:0/666988860 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd94410b270 msgr2=0x7fd94410b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:42:52.748 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:51.344+0000 7fd94a200700 1 --2- 172.21.15.151:0/666988860 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd94410b270 0x7fd94410b690 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd9340098a0 tx=0x7fd934009bb0 comp rx=0 tx=0).stop 2024-05-21T21:42:52.748 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:51.344+0000 7fd94a200700 1 -- 172.21.15.151:0/666988860 shutdown_connections 2024-05-21T21:42:52.748 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:51.344+0000 7fd94a200700 1 --2- 172.21.15.151:0/666988860 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd94410b270 0x7fd94410b690 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:42:52.749 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:51.344+0000 7fd94a200700 1 -- 172.21.15.151:0/666988860 >> 172.21.15.151:0/666988860 conn(0x7fd944106910 msgr2=0x7fd944108d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:42:52.749 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:51.344+0000 7fd94a200700 1 -- 172.21.15.151:0/666988860 shutdown_connections 2024-05-21T21:42:52.749 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:51.344+0000 7fd94a200700 1 -- 172.21.15.151:0/666988860 wait complete. 2024-05-21T21:42:52.749 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:51.345+0000 7fd94a200700 1 Processor -- start 2024-05-21T21:42:52.749 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:51.345+0000 7fd94a200700 1 -- start start 2024-05-21T21:42:52.749 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:51.346+0000 7fd94a200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd94410b270 0x7fd94419f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:42:52.749 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:51.346+0000 7fd94a200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd94410f7d0 con 0x7fd94410b270 2024-05-21T21:42:52.749 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:51.346+0000 7fd943400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd94410b270 0x7fd94419f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:42:52.749 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:51.346+0000 7fd943400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd94410b270 0x7fd94419f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:38102/0 (socket says 172.21.15.151:38102) 2024-05-21T21:42:52.749 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:51.346+0000 7fd943400700 1 -- 172.21.15.151:0/3613723861 learned_addr learned my addr 172.21.15.151:0/3613723861 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:42:52.749 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:51.346+0000 7fd943400700 1 -- 172.21.15.151:0/3613723861 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd94419f740 con 0x7fd94410b270 2024-05-21T21:42:52.749 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:51.347+0000 7fd943400700 1 --2- 172.21.15.151:0/3613723861 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd94410b270 0x7fd94419f200 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd9340117b0 tx=0x7fd934011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:42:52.749 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:51.347+0000 7fd940c00700 1 -- 172.21.15.151:0/3613723861 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd934003af0 con 0x7fd94410b270 2024-05-21T21:42:52.749 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:51.347+0000 7fd940c00700 1 -- 172.21.15.151:0/3613723861 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd934020070 con 0x7fd94410b270 2024-05-21T21:42:52.750 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:51.347+0000 7fd940c00700 1 -- 172.21.15.151:0/3613723861 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd934019630 con 0x7fd94410b270 2024-05-21T21:42:52.750 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:51.347+0000 7fd94a200700 1 -- 172.21.15.151:0/3613723861 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd94419f9d0 con 0x7fd94410b270 2024-05-21T21:42:52.750 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:51.347+0000 7fd94a200700 1 -- 172.21.15.151:0/3613723861 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9441a26c0 con 0x7fd94410b270 2024-05-21T21:42:52.750 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:51.348+0000 7fd940c00700 1 -- 172.21.15.151:0/3613723861 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fd93401a9c0 con 0x7fd94410b270 2024-05-21T21:42:52.750 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:51.349+0000 7fd94a200700 1 -- 172.21.15.151:0/3613723861 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd944064ba0 con 0x7fd94410b270 2024-05-21T21:42:52.750 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:51.349+0000 7fd940c00700 1 --2- 172.21.15.151:0/3613723861 >> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] conn(0x7fd92803e4c0 0x7fd928040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:42:52.750 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:51.349+0000 7fd940c00700 1 -- 172.21.15.151:0/3613723861 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd934053a60 con 0x7fd94410b270 2024-05-21T21:42:52.750 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:51.349+0000 7fd93be00700 1 --2- 172.21.15.151:0/3613723861 >> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] conn(0x7fd92803e4c0 0x7fd928040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:42:52.750 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:51.350+0000 7fd93be00700 1 --2- 172.21.15.151:0/3613723861 >> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] conn(0x7fd92803e4c0 0x7fd928040980 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd92c006fd0 tx=0x7fd92c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:42:52.750 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:51.355+0000 7fd940c00700 1 -- 172.21.15.151:0/3613723861 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd9340042a0 con 0x7fd94410b270 2024-05-21T21:42:52.750 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:51.612+0000 7fd94a200700 1 -- 172.21.15.151:0/3613723861 --> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fd944107bd0 con 0x7fd92803e4c0 2024-05-21T21:42:52.750 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:51.621+0000 7fd940c00700 1 -- 172.21.15.151:0/3613723861 <== mgr.14120 v2:172.21.15.151:6800/3428827300 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fd944107bd0 con 0x7fd92803e4c0 2024-05-21T21:42:52.751 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:51.624+0000 7fd94a200700 1 -- 172.21.15.151:0/3613723861 >> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] conn(0x7fd92803e4c0 msgr2=0x7fd928040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:42:52.751 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:51.624+0000 7fd94a200700 1 --2- 172.21.15.151:0/3613723861 >> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] conn(0x7fd92803e4c0 0x7fd928040980 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd92c006fd0 tx=0x7fd92c006e40 comp rx=0 tx=0).stop 2024-05-21T21:42:52.751 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:51.624+0000 7fd94a200700 1 -- 172.21.15.151:0/3613723861 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd94410b270 msgr2=0x7fd94419f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:42:52.751 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:51.624+0000 7fd94a200700 1 --2- 172.21.15.151:0/3613723861 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd94410b270 0x7fd94419f200 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd9340117b0 tx=0x7fd934011b70 comp rx=0 tx=0).stop 2024-05-21T21:42:52.751 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:51.624+0000 7fd94a200700 1 -- 172.21.15.151:0/3613723861 shutdown_connections 2024-05-21T21:42:52.751 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:51.624+0000 7fd94a200700 1 --2- 172.21.15.151:0/3613723861 >> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] conn(0x7fd92803e4c0 0x7fd928040980 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:42:52.751 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:51.624+0000 7fd94a200700 1 --2- 172.21.15.151:0/3613723861 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd94410b270 0x7fd94419f200 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:42:52.751 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:51.624+0000 7fd94a200700 1 -- 172.21.15.151:0/3613723861 >> 172.21.15.151:0/3613723861 conn(0x7fd944106910 msgr2=0x7fd9441074c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:42:52.751 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:51.624+0000 7fd94a200700 1 -- 172.21.15.151:0/3613723861 shutdown_connections 2024-05-21T21:42:52.751 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:51.625+0000 7fd94a200700 1 -- 172.21.15.151:0/3613723861 wait complete. 2024-05-21T21:42:52.751 INFO:teuthology.orchestra.run.smithi151.stdout:Deploying unmanaged mgr service... 2024-05-21T21:42:52.894 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:52 smithi151 ceph-mon[31623]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:42:52.894 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:52 smithi151 ceph-mon[31623]: Saving service mon spec with placement count:5 2024-05-21T21:42:52.894 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:52 smithi151 ceph-mon[31623]: from='mgr.14120 172.21.15.151:0/2204845274' entity='mgr.x' 2024-05-21T21:42:54.386 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-05-21T21:42:54.386 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:53.387+0000 7f3f67400700 1 Processor -- start 2024-05-21T21:42:54.386 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:53.388+0000 7f3f67400700 1 -- start start 2024-05-21T21:42:54.386 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:53.388+0000 7f3f67400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3f6007ddc0 0x7f3f6007c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:42:54.386 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:53.388+0000 7f3f67400700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3f6007c7f0 con 0x7f3f6007ddc0 2024-05-21T21:42:54.386 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:53.389+0000 7f3f64a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3f6007ddc0 0x7f3f6007c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:42:54.386 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:53.389+0000 7f3f64a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3f6007ddc0 0x7f3f6007c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:38108/0 (socket says 172.21.15.151:38108) 2024-05-21T21:42:54.387 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:53.389+0000 7f3f64a00700 1 -- 172.21.15.151:0/1023951172 learned_addr learned my addr 172.21.15.151:0/1023951172 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:42:54.387 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:53.389+0000 7f3f64a00700 1 -- 172.21.15.151:0/1023951172 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f6007c970 con 0x7f3f6007ddc0 2024-05-21T21:42:54.387 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:53.389+0000 7f3f64a00700 1 --2- 172.21.15.151:0/1023951172 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3f6007ddc0 0x7f3f6007c220 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3f540098a0 tx=0x7f3f54009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=231d3b349bbfa1fe server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:42:54.387 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:53.390+0000 7f3f5f400700 1 -- 172.21.15.151:0/1023951172 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f54004160 con 0x7f3f6007ddc0 2024-05-21T21:42:54.387 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:53.390+0000 7f3f5f400700 1 -- 172.21.15.151:0/1023951172 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f3f54004300 con 0x7f3f6007ddc0 2024-05-21T21:42:54.387 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:53.390+0000 7f3f5f400700 1 -- 172.21.15.151:0/1023951172 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f54011af0 con 0x7f3f6007ddc0 2024-05-21T21:42:54.387 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:53.390+0000 7f3f67400700 1 -- 172.21.15.151:0/1023951172 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3f6007ddc0 msgr2=0x7f3f6007c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:42:54.387 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:53.390+0000 7f3f67400700 1 --2- 172.21.15.151:0/1023951172 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3f6007ddc0 0x7f3f6007c220 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3f540098a0 tx=0x7f3f54009bb0 comp rx=0 tx=0).stop 2024-05-21T21:42:54.387 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:53.391+0000 7f3f67400700 1 -- 172.21.15.151:0/1023951172 shutdown_connections 2024-05-21T21:42:54.387 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:53.391+0000 7f3f67400700 1 --2- 172.21.15.151:0/1023951172 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3f6007ddc0 0x7f3f6007c220 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:42:54.387 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:53.391+0000 7f3f67400700 1 -- 172.21.15.151:0/1023951172 >> 172.21.15.151:0/1023951172 conn(0x7f3f60104630 msgr2=0x7f3f60106a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:42:54.388 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:53.391+0000 7f3f67400700 1 -- 172.21.15.151:0/1023951172 shutdown_connections 2024-05-21T21:42:54.388 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:53.391+0000 7f3f67400700 1 -- 172.21.15.151:0/1023951172 wait complete. 2024-05-21T21:42:54.388 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:53.392+0000 7f3f67400700 1 Processor -- start 2024-05-21T21:42:54.388 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:53.392+0000 7f3f67400700 1 -- start start 2024-05-21T21:42:54.388 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:53.392+0000 7f3f67400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3f6007ddc0 0x7f3f6019f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:42:54.388 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:53.392+0000 7f3f67400700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3f6010f610 con 0x7f3f6007ddc0 2024-05-21T21:42:54.388 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:53.393+0000 7f3f64a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3f6007ddc0 0x7f3f6019f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:42:54.388 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:53.393+0000 7f3f64a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3f6007ddc0 0x7f3f6019f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:38124/0 (socket says 172.21.15.151:38124) 2024-05-21T21:42:54.388 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:53.393+0000 7f3f64a00700 1 -- 172.21.15.151:0/1497676478 learned_addr learned my addr 172.21.15.151:0/1497676478 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:42:54.388 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:53.393+0000 7f3f64a00700 1 -- 172.21.15.151:0/1497676478 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f6019f750 con 0x7f3f6007ddc0 2024-05-21T21:42:54.388 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:53.393+0000 7f3f64a00700 1 --2- 172.21.15.151:0/1497676478 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3f6007ddc0 0x7f3f6019f210 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3f540117b0 tx=0x7f3f54011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:42:54.388 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:53.393+0000 7f3f5d600700 1 -- 172.21.15.151:0/1497676478 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f54003af0 con 0x7f3f6007ddc0 2024-05-21T21:42:54.388 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:53.394+0000 7f3f5d600700 1 -- 172.21.15.151:0/1497676478 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f3f54020070 con 0x7f3f6007ddc0 2024-05-21T21:42:54.389 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:53.394+0000 7f3f67400700 1 -- 172.21.15.151:0/1497676478 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f6019f9e0 con 0x7f3f6007ddc0 2024-05-21T21:42:54.389 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:53.394+0000 7f3f5d600700 1 -- 172.21.15.151:0/1497676478 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f54019630 con 0x7f3f6007ddc0 2024-05-21T21:42:54.389 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:53.394+0000 7f3f67400700 1 -- 172.21.15.151:0/1497676478 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f601a26d0 con 0x7f3f6007ddc0 2024-05-21T21:42:54.389 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:53.395+0000 7f3f67400700 1 -- 172.21.15.151:0/1497676478 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f60064ba0 con 0x7f3f6007ddc0 2024-05-21T21:42:54.389 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:53.395+0000 7f3f5d600700 1 -- 172.21.15.151:0/1497676478 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f3f5401a9c0 con 0x7f3f6007ddc0 2024-05-21T21:42:54.389 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:53.395+0000 7f3f5d600700 1 --2- 172.21.15.151:0/1497676478 >> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] conn(0x7f3f4803e4c0 0x7f3f48040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:42:54.389 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:53.396+0000 7f3f5d600700 1 -- 172.21.15.151:0/1497676478 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f3f54053a60 con 0x7f3f6007ddc0 2024-05-21T21:42:54.389 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:53.396+0000 7f3f5fe00700 1 --2- 172.21.15.151:0/1497676478 >> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] conn(0x7f3f4803e4c0 0x7f3f48040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:42:54.389 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:53.396+0000 7f3f5fe00700 1 --2- 172.21.15.151:0/1497676478 >> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] conn(0x7f3f4803e4c0 0x7f3f48040980 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c006fd0 tx=0x7f3f4c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:42:54.389 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:53.401+0000 7f3f5d600700 1 -- 172.21.15.151:0/1497676478 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3f540042a0 con 0x7f3f6007ddc0 2024-05-21T21:42:54.389 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:53.650+0000 7f3f67400700 1 -- 172.21.15.151:0/1497676478 --> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f3f601058f0 con 0x7f3f4803e4c0 2024-05-21T21:42:54.389 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:53.659+0000 7f3f5d600700 1 -- 172.21.15.151:0/1497676478 <== mgr.14120 v2:172.21.15.151:6800/3428827300 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f3f601058f0 con 0x7f3f4803e4c0 2024-05-21T21:42:54.389 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:53.662+0000 7f3f67400700 1 -- 172.21.15.151:0/1497676478 >> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] conn(0x7f3f4803e4c0 msgr2=0x7f3f48040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:42:54.389 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:53.662+0000 7f3f67400700 1 --2- 172.21.15.151:0/1497676478 >> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] conn(0x7f3f4803e4c0 0x7f3f48040980 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c006fd0 tx=0x7f3f4c006e40 comp rx=0 tx=0).stop 2024-05-21T21:42:54.390 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:53.662+0000 7f3f67400700 1 -- 172.21.15.151:0/1497676478 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3f6007ddc0 msgr2=0x7f3f6019f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:42:54.390 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:53.662+0000 7f3f67400700 1 --2- 172.21.15.151:0/1497676478 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3f6007ddc0 0x7f3f6019f210 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3f540117b0 tx=0x7f3f54011b70 comp rx=0 tx=0).stop 2024-05-21T21:42:54.390 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:53.662+0000 7f3f67400700 1 -- 172.21.15.151:0/1497676478 shutdown_connections 2024-05-21T21:42:54.390 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:53.662+0000 7f3f67400700 1 --2- 172.21.15.151:0/1497676478 >> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] conn(0x7f3f4803e4c0 0x7f3f48040980 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:42:54.390 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:53.662+0000 7f3f67400700 1 --2- 172.21.15.151:0/1497676478 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3f6007ddc0 0x7f3f6019f210 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:42:54.390 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:53.662+0000 7f3f67400700 1 -- 172.21.15.151:0/1497676478 >> 172.21.15.151:0/1497676478 conn(0x7f3f60104630 msgr2=0x7f3f601051e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:42:54.390 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:53.663+0000 7f3f67400700 1 -- 172.21.15.151:0/1497676478 shutdown_connections 2024-05-21T21:42:54.390 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:53.663+0000 7f3f67400700 1 -- 172.21.15.151:0/1497676478 wait complete. 2024-05-21T21:42:54.952 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:54 smithi151 ceph-mon[31623]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:42:54.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:54 smithi151 ceph-mon[31623]: Saving service mgr spec with placement count:2 2024-05-21T21:42:54.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:54 smithi151 ceph-mon[31623]: from='mgr.14120 172.21.15.151:0/2204845274' entity='mgr.x' 2024-05-21T21:42:54.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:54 smithi151 ceph-mon[31623]: from='mgr.14120 172.21.15.151:0/2204845274' entity='mgr.x' 2024-05-21T21:42:55.911 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:55 smithi151 ceph-mon[31623]: from='mgr.14120 172.21.15.151:0/2204845274' entity='mgr.x' 2024-05-21T21:42:55.912 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:55 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/4149861364' entity='client.admin' 2024-05-21T21:42:55.927 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:55.010+0000 7fed0b200700 1 Processor -- start 2024-05-21T21:42:55.928 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:55.010+0000 7fed0b200700 1 -- start start 2024-05-21T21:42:55.928 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:55.010+0000 7fed0b200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fed0c108e40 0x7fed0c109260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:42:55.928 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:55.010+0000 7fed0b200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fed0c1097a0 con 0x7fed0c108e40 2024-05-21T21:42:55.928 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:55.011+0000 7fed09e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fed0c108e40 0x7fed0c109260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:42:55.928 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:55.011+0000 7fed09e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fed0c108e40 0x7fed0c109260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:38132/0 (socket says 172.21.15.151:38132) 2024-05-21T21:42:55.928 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:55.011+0000 7fed09e00700 1 -- 172.21.15.151:0/3811226894 learned_addr learned my addr 172.21.15.151:0/3811226894 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:42:55.928 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:55.012+0000 7fed09e00700 1 -- 172.21.15.151:0/3811226894 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed0c109fa0 con 0x7fed0c108e40 2024-05-21T21:42:55.928 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:55.012+0000 7fed09e00700 1 --2- 172.21.15.151:0/3811226894 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fed0c108e40 0x7fed0c109260 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fecfc0098a0 tx=0x7fecfc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe76cf0a5a4e0fb0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:42:55.928 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:55.012+0000 7fed08a00700 1 -- 172.21.15.151:0/3811226894 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fecfc004160 con 0x7fed0c108e40 2024-05-21T21:42:55.928 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:55.012+0000 7fed08a00700 1 -- 172.21.15.151:0/3811226894 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fecfc004300 con 0x7fed0c108e40 2024-05-21T21:42:55.929 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:55.012+0000 7fed08a00700 1 -- 172.21.15.151:0/3811226894 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fecfc011a20 con 0x7fed0c108e40 2024-05-21T21:42:55.929 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:55.013+0000 7fed0b200700 1 -- 172.21.15.151:0/3811226894 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fed0c108e40 msgr2=0x7fed0c109260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:42:55.929 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:55.013+0000 7fed0b200700 1 --2- 172.21.15.151:0/3811226894 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fed0c108e40 0x7fed0c109260 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fecfc0098a0 tx=0x7fecfc009bb0 comp rx=0 tx=0).stop 2024-05-21T21:42:55.929 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:55.013+0000 7fed0b200700 1 -- 172.21.15.151:0/3811226894 shutdown_connections 2024-05-21T21:42:55.929 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:55.013+0000 7fed0b200700 1 --2- 172.21.15.151:0/3811226894 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fed0c108e40 0x7fed0c109260 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:42:55.929 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:55.013+0000 7fed0b200700 1 -- 172.21.15.151:0/3811226894 >> 172.21.15.151:0/3811226894 conn(0x7fed0c1043c0 msgr2=0x7fed0c106820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:42:55.929 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:55.014+0000 7fed0b200700 1 -- 172.21.15.151:0/3811226894 shutdown_connections 2024-05-21T21:42:55.929 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:55.014+0000 7fed0b200700 1 -- 172.21.15.151:0/3811226894 wait complete. 2024-05-21T21:42:55.929 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:55.015+0000 7fed0b200700 1 Processor -- start 2024-05-21T21:42:55.929 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:55.015+0000 7fed0b200700 1 -- start start 2024-05-21T21:42:55.929 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:55.015+0000 7fed0b200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fed0c108e40 0x7fed0c19ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:42:55.930 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:55.015+0000 7fed0b200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fed0c10d3e0 con 0x7fed0c108e40 2024-05-21T21:42:55.930 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:55.015+0000 7fed09e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fed0c108e40 0x7fed0c19ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:42:55.930 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:55.015+0000 7fed09e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fed0c108e40 0x7fed0c19ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:38140/0 (socket says 172.21.15.151:38140) 2024-05-21T21:42:55.930 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:55.015+0000 7fed09e00700 1 -- 172.21.15.151:0/4149861364 learned_addr learned my addr 172.21.15.151:0/4149861364 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:42:55.930 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:55.016+0000 7fed09e00700 1 -- 172.21.15.151:0/4149861364 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed0c19f4d0 con 0x7fed0c108e40 2024-05-21T21:42:55.930 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:55.016+0000 7fed09e00700 1 --2- 172.21.15.151:0/4149861364 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fed0c108e40 0x7fed0c19ef90 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fecfc004660 tx=0x7fecfc011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:42:55.930 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:55.016+0000 7fed02a00700 1 -- 172.21.15.151:0/4149861364 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fecfc003aa0 con 0x7fed0c108e40 2024-05-21T21:42:55.930 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:55.016+0000 7fed02a00700 1 -- 172.21.15.151:0/4149861364 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fecfc01b8f0 con 0x7fed0c108e40 2024-05-21T21:42:55.930 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:55.016+0000 7fed0b200700 1 -- 172.21.15.151:0/4149861364 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed0c19f760 con 0x7fed0c108e40 2024-05-21T21:42:55.930 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:55.017+0000 7fed02a00700 1 -- 172.21.15.151:0/4149861364 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fecfc023900 con 0x7fed0c108e40 2024-05-21T21:42:55.930 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:55.017+0000 7fed0b200700 1 -- 172.21.15.151:0/4149861364 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed0c1a2450 con 0x7fed0c108e40 2024-05-21T21:42:55.931 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:55.018+0000 7fed0b200700 1 -- 172.21.15.151:0/4149861364 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed0c10d560 con 0x7fed0c108e40 2024-05-21T21:42:55.931 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:55.018+0000 7fed02a00700 1 -- 172.21.15.151:0/4149861364 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fecfc023b70 con 0x7fed0c108e40 2024-05-21T21:42:55.931 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:55.018+0000 7fed02a00700 1 --2- 172.21.15.151:0/4149861364 >> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] conn(0x7fecf003e4c0 0x7fecf0040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:42:55.931 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:55.019+0000 7fed02a00700 1 -- 172.21.15.151:0/4149861364 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fecfc053ef0 con 0x7fed0c108e40 2024-05-21T21:42:55.931 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:55.019+0000 7fed09400700 1 --2- 172.21.15.151:0/4149861364 >> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] conn(0x7fecf003e4c0 0x7fecf0040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:42:55.931 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:55.019+0000 7fed09400700 1 --2- 172.21.15.151:0/4149861364 >> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] conn(0x7fecf003e4c0 0x7fecf0040980 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fecf8006fd0 tx=0x7fecf8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:42:55.931 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:55.025+0000 7fed02a00700 1 -- 172.21.15.151:0/4149861364 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fecfc01f030 con 0x7fed0c108e40 2024-05-21T21:42:55.931 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:55.255+0000 7fed0b200700 1 -- 172.21.15.151:0/4149861364 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7fed0c1a2a30 con 0x7fed0c108e40 2024-05-21T21:42:55.931 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:55.262+0000 7fed02a00700 1 -- 172.21.15.151:0/4149861364 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7fecfc01bb30 con 0x7fed0c108e40 2024-05-21T21:42:55.931 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:55.266+0000 7fed0b200700 1 -- 172.21.15.151:0/4149861364 >> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] conn(0x7fecf003e4c0 msgr2=0x7fecf0040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:42:55.931 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:55.266+0000 7fed0b200700 1 --2- 172.21.15.151:0/4149861364 >> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] conn(0x7fecf003e4c0 0x7fecf0040980 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fecf8006fd0 tx=0x7fecf8006e40 comp rx=0 tx=0).stop 2024-05-21T21:42:55.931 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:55.266+0000 7fed0b200700 1 -- 172.21.15.151:0/4149861364 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fed0c108e40 msgr2=0x7fed0c19ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:42:55.932 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:55.266+0000 7fed0b200700 1 --2- 172.21.15.151:0/4149861364 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fed0c108e40 0x7fed0c19ef90 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fecfc004660 tx=0x7fecfc011aa0 comp rx=0 tx=0).stop 2024-05-21T21:42:55.932 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:55.266+0000 7fed0b200700 1 -- 172.21.15.151:0/4149861364 shutdown_connections 2024-05-21T21:42:55.932 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:55.266+0000 7fed0b200700 1 --2- 172.21.15.151:0/4149861364 >> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] conn(0x7fecf003e4c0 0x7fecf0040980 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:42:55.932 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:55.266+0000 7fed0b200700 1 --2- 172.21.15.151:0/4149861364 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fed0c108e40 0x7fed0c19ef90 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:42:55.932 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:55.266+0000 7fed0b200700 1 -- 172.21.15.151:0/4149861364 >> 172.21.15.151:0/4149861364 conn(0x7fed0c1043c0 msgr2=0x7fed0c104e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:42:55.932 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:55.267+0000 7fed0b200700 1 -- 172.21.15.151:0/4149861364 shutdown_connections 2024-05-21T21:42:55.932 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:55.267+0000 7fed0b200700 1 -- 172.21.15.151:0/4149861364 wait complete. 2024-05-21T21:42:57.023 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:56 smithi151 ceph-mon[31623]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:42:57.023 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:56 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/35668627' entity='client.admin' 2024-05-21T21:42:57.901 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:56.483+0000 7fbfb2e00700 1 Processor -- start 2024-05-21T21:42:57.901 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:56.484+0000 7fbfb2e00700 1 -- start start 2024-05-21T21:42:57.901 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:56.484+0000 7fbfb2e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbfac109050 0x7fbfac109470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:42:57.901 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:56.484+0000 7fbfb2e00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbfac109a40 con 0x7fbfac109050 2024-05-21T21:42:57.901 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:56.485+0000 7fbfabe00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbfac109050 0x7fbfac109470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:42:57.901 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:56.485+0000 7fbfabe00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbfac109050 0x7fbfac109470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:33414/0 (socket says 172.21.15.151:33414) 2024-05-21T21:42:57.902 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:56.485+0000 7fbfabe00700 1 -- 172.21.15.151:0/2768166630 learned_addr learned my addr 172.21.15.151:0/2768166630 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:42:57.905 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:56.485+0000 7fbfabe00700 1 -- 172.21.15.151:0/2768166630 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfac10a250 con 0x7fbfac109050 2024-05-21T21:42:57.905 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:56.485+0000 7fbfabe00700 1 --2- 172.21.15.151:0/2768166630 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbfac109050 0x7fbfac109470 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c0098a0 tx=0x7fbf9c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=138f18db6115d7aa server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:42:57.905 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:56.486+0000 7fbfaaa00700 1 -- 172.21.15.151:0/2768166630 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbf9c004160 con 0x7fbfac109050 2024-05-21T21:42:57.905 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:56.486+0000 7fbfaaa00700 1 -- 172.21.15.151:0/2768166630 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf9c004300 con 0x7fbfac109050 2024-05-21T21:42:57.905 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:56.486+0000 7fbfaaa00700 1 -- 172.21.15.151:0/2768166630 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbf9c011a20 con 0x7fbfac109050 2024-05-21T21:42:57.905 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:56.486+0000 7fbfb2e00700 1 -- 172.21.15.151:0/2768166630 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbfac109050 msgr2=0x7fbfac109470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:42:57.905 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:56.486+0000 7fbfb2e00700 1 --2- 172.21.15.151:0/2768166630 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbfac109050 0x7fbfac109470 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c0098a0 tx=0x7fbf9c009bb0 comp rx=0 tx=0).stop 2024-05-21T21:42:57.905 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:56.487+0000 7fbfb2e00700 1 -- 172.21.15.151:0/2768166630 shutdown_connections 2024-05-21T21:42:57.906 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:56.487+0000 7fbfb2e00700 1 --2- 172.21.15.151:0/2768166630 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbfac109050 0x7fbfac109470 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:42:57.906 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:56.487+0000 7fbfb2e00700 1 -- 172.21.15.151:0/2768166630 >> 172.21.15.151:0/2768166630 conn(0x7fbfac104630 msgr2=0x7fbfac106a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:42:57.906 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:56.487+0000 7fbfb2e00700 1 -- 172.21.15.151:0/2768166630 shutdown_connections 2024-05-21T21:42:57.906 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:56.487+0000 7fbfb2e00700 1 -- 172.21.15.151:0/2768166630 wait complete. 2024-05-21T21:42:57.906 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:56.488+0000 7fbfb2e00700 1 Processor -- start 2024-05-21T21:42:57.906 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:56.488+0000 7fbfb2e00700 1 -- start start 2024-05-21T21:42:57.906 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:56.488+0000 7fbfb2e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbfac109050 0x7fbfac19f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:42:57.906 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:56.488+0000 7fbfb2e00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbfac10d600 con 0x7fbfac109050 2024-05-21T21:42:57.906 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:56.489+0000 7fbfabe00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbfac109050 0x7fbfac19f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:42:57.906 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:56.489+0000 7fbfabe00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbfac109050 0x7fbfac19f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:33422/0 (socket says 172.21.15.151:33422) 2024-05-21T21:42:57.906 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:56.489+0000 7fbfabe00700 1 -- 172.21.15.151:0/35668627 learned_addr learned my addr 172.21.15.151:0/35668627 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:42:57.906 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:56.489+0000 7fbfabe00700 1 -- 172.21.15.151:0/35668627 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfac19f720 con 0x7fbfac109050 2024-05-21T21:42:57.906 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:56.489+0000 7fbfabe00700 1 --2- 172.21.15.151:0/35668627 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbfac109050 0x7fbfac19f1e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c000c00 tx=0x7fbf9c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:42:57.907 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:56.490+0000 7fbfa8c00700 1 -- 172.21.15.151:0/35668627 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbf9c003aa0 con 0x7fbfac109050 2024-05-21T21:42:57.907 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:56.490+0000 7fbfa8c00700 1 -- 172.21.15.151:0/35668627 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf9c01b8f0 con 0x7fbfac109050 2024-05-21T21:42:57.907 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:56.490+0000 7fbfa8c00700 1 -- 172.21.15.151:0/35668627 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbf9c023890 con 0x7fbfac109050 2024-05-21T21:42:57.907 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:56.490+0000 7fbfb2e00700 1 -- 172.21.15.151:0/35668627 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfac19f9b0 con 0x7fbfac109050 2024-05-21T21:42:57.907 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:56.490+0000 7fbfb2e00700 1 -- 172.21.15.151:0/35668627 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfac1a26a0 con 0x7fbfac109050 2024-05-21T21:42:57.907 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:56.491+0000 7fbfa8c00700 1 -- 172.21.15.151:0/35668627 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fbf9c01b440 con 0x7fbfac109050 2024-05-21T21:42:57.907 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:56.491+0000 7fbfb2e00700 1 -- 172.21.15.151:0/35668627 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfac064ba0 con 0x7fbfac109050 2024-05-21T21:42:57.907 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:56.492+0000 7fbfa8c00700 1 --2- 172.21.15.151:0/35668627 >> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] conn(0x7fbf9003e4c0 0x7fbf90040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:42:57.907 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:56.492+0000 7fbfa8c00700 1 -- 172.21.15.151:0/35668627 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fbf9c053e00 con 0x7fbfac109050 2024-05-21T21:42:57.907 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:56.492+0000 7fbfab400700 1 --2- 172.21.15.151:0/35668627 >> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] conn(0x7fbf9003e4c0 0x7fbf90040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:42:57.909 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:56.493+0000 7fbfab400700 1 --2- 172.21.15.151:0/35668627 >> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] conn(0x7fbf9003e4c0 0x7fbf90040980 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fbf98006fd0 tx=0x7fbf98006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:42:57.909 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:56.498+0000 7fbfa8c00700 1 -- 172.21.15.151:0/35668627 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbf9c01f030 con 0x7fbfac109050 2024-05-21T21:42:57.909 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:56.734+0000 7fbfb2e00700 1 -- 172.21.15.151:0/35668627 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7fbfac1a2d00 con 0x7fbfac109050 2024-05-21T21:42:57.909 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:56.738+0000 7fbfa8c00700 1 -- 172.21.15.151:0/35668627 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7fbf9c01bda0 con 0x7fbfac109050 2024-05-21T21:42:57.909 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:56.741+0000 7fbfb2e00700 1 -- 172.21.15.151:0/35668627 >> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] conn(0x7fbf9003e4c0 msgr2=0x7fbf90040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:42:57.909 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:56.741+0000 7fbfb2e00700 1 --2- 172.21.15.151:0/35668627 >> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] conn(0x7fbf9003e4c0 0x7fbf90040980 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fbf98006fd0 tx=0x7fbf98006e40 comp rx=0 tx=0).stop 2024-05-21T21:42:57.909 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:56.742+0000 7fbfb2e00700 1 -- 172.21.15.151:0/35668627 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbfac109050 msgr2=0x7fbfac19f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:42:57.909 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:56.742+0000 7fbfb2e00700 1 --2- 172.21.15.151:0/35668627 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbfac109050 0x7fbfac19f1e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c000c00 tx=0x7fbf9c011aa0 comp rx=0 tx=0).stop 2024-05-21T21:42:57.909 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:56.742+0000 7fbfb2e00700 1 -- 172.21.15.151:0/35668627 shutdown_connections 2024-05-21T21:42:57.910 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:56.742+0000 7fbfb2e00700 1 --2- 172.21.15.151:0/35668627 >> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] conn(0x7fbf9003e4c0 0x7fbf90040980 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:42:57.910 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:56.742+0000 7fbfb2e00700 1 --2- 172.21.15.151:0/35668627 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbfac109050 0x7fbfac19f1e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:42:57.910 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:56.742+0000 7fbfb2e00700 1 -- 172.21.15.151:0/35668627 >> 172.21.15.151:0/35668627 conn(0x7fbfac104630 msgr2=0x7fbfac1050a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:42:57.910 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:56.742+0000 7fbfb2e00700 1 -- 172.21.15.151:0/35668627 shutdown_connections 2024-05-21T21:42:57.910 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:56.743+0000 7fbfb2e00700 1 -- 172.21.15.151:0/35668627 wait complete. 2024-05-21T21:42:57.910 INFO:teuthology.orchestra.run.smithi151.stdout:Enabling the dashboard module... 2024-05-21T21:42:58.844 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:58 smithi151 ceph-mon[31623]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:42:58.845 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:58 smithi151 ceph-mon[31623]: from='mgr.14120 172.21.15.151:0/2204845274' entity='mgr.x' 2024-05-21T21:42:59.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:42:59 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/2152116346' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-05-21T21:42:59.953 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:42:59 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: ignoring --setuser ceph since I am not root 2024-05-21T21:42:59.953 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:42:59 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: ignoring --setgroup ceph since I am not root 2024-05-21T21:43:00.390 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:43:00 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:43:00.017+0000 7ff3f0290200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-21T21:43:00.390 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:43:00 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:43:00.262+0000 7ff3f0290200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-21T21:43:00.397 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:58.507+0000 7f8cb0a00700 1 Processor -- start 2024-05-21T21:43:00.398 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:58.507+0000 7f8cb0a00700 1 -- start start 2024-05-21T21:43:00.398 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:58.508+0000 7f8cb0a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8cac07dd80 0x7f8cac07c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:43:00.398 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:58.508+0000 7f8cb0a00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8cac07c7b0 con 0x7f8cac07dd80 2024-05-21T21:43:00.398 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:58.508+0000 7f8cab400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8cac07dd80 0x7f8cac07c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:43:00.399 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:58.508+0000 7f8cab400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8cac07dd80 0x7f8cac07c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:33434/0 (socket says 172.21.15.151:33434) 2024-05-21T21:43:00.399 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:58.508+0000 7f8cab400700 1 -- 172.21.15.151:0/1652371133 learned_addr learned my addr 172.21.15.151:0/1652371133 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:43:00.399 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:58.509+0000 7f8cab400700 1 -- 172.21.15.151:0/1652371133 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cac07c930 con 0x7f8cac07dd80 2024-05-21T21:43:00.399 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:58.509+0000 7f8cab400700 1 --2- 172.21.15.151:0/1652371133 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8cac07dd80 0x7f8cac07c1e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c0098a0 tx=0x7f8c9c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=616b8ae5b12b4303 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:43:00.399 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:58.509+0000 7f8caa000700 1 -- 172.21.15.151:0/1652371133 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8c9c004160 con 0x7f8cac07dd80 2024-05-21T21:43:00.400 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:58.509+0000 7f8caa000700 1 -- 172.21.15.151:0/1652371133 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c9c004300 con 0x7f8cac07dd80 2024-05-21T21:43:00.400 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:58.509+0000 7f8caa000700 1 -- 172.21.15.151:0/1652371133 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8c9c011a20 con 0x7f8cac07dd80 2024-05-21T21:43:00.401 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:58.510+0000 7f8cb0a00700 1 -- 172.21.15.151:0/1652371133 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8cac07dd80 msgr2=0x7f8cac07c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:43:00.401 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:58.510+0000 7f8cb0a00700 1 --2- 172.21.15.151:0/1652371133 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8cac07dd80 0x7f8cac07c1e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c0098a0 tx=0x7f8c9c009bb0 comp rx=0 tx=0).stop 2024-05-21T21:43:00.401 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:58.510+0000 7f8cb0a00700 1 -- 172.21.15.151:0/1652371133 shutdown_connections 2024-05-21T21:43:00.401 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:58.510+0000 7f8cb0a00700 1 --2- 172.21.15.151:0/1652371133 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8cac07dd80 0x7f8cac07c1e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:43:00.401 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:58.510+0000 7f8cb0a00700 1 -- 172.21.15.151:0/1652371133 >> 172.21.15.151:0/1652371133 conn(0x7f8cac1043c0 msgr2=0x7f8cac106820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:43:00.402 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:58.510+0000 7f8cb0a00700 1 -- 172.21.15.151:0/1652371133 shutdown_connections 2024-05-21T21:43:00.402 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:58.510+0000 7f8cb0a00700 1 -- 172.21.15.151:0/1652371133 wait complete. 2024-05-21T21:43:00.402 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:58.511+0000 7f8cb0a00700 1 Processor -- start 2024-05-21T21:43:00.402 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:58.511+0000 7f8cb0a00700 1 -- start start 2024-05-21T21:43:00.402 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:58.512+0000 7f8cb0a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8cac07dd80 0x7f8cac19f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:43:00.402 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:58.512+0000 7f8cb0a00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8cac10f440 con 0x7f8cac07dd80 2024-05-21T21:43:00.402 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:58.512+0000 7f8cab400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8cac07dd80 0x7f8cac19f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:43:00.402 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:58.512+0000 7f8cab400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8cac07dd80 0x7f8cac19f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:33444/0 (socket says 172.21.15.151:33444) 2024-05-21T21:43:00.405 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:58.512+0000 7f8cab400700 1 -- 172.21.15.151:0/2152116346 learned_addr learned my addr 172.21.15.151:0/2152116346 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:43:00.405 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:58.512+0000 7f8cab400700 1 -- 172.21.15.151:0/2152116346 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cac19f540 con 0x7f8cac07dd80 2024-05-21T21:43:00.405 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:58.513+0000 7f8cab400700 1 --2- 172.21.15.151:0/2152116346 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8cac07dd80 0x7f8cac19f000 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c000c00 tx=0x7f8c9c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:43:00.405 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:58.513+0000 7f8ca3e00700 1 -- 172.21.15.151:0/2152116346 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8c9c003aa0 con 0x7f8cac07dd80 2024-05-21T21:43:00.405 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:58.513+0000 7f8ca3e00700 1 -- 172.21.15.151:0/2152116346 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c9c01b8f0 con 0x7f8cac07dd80 2024-05-21T21:43:00.405 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:58.513+0000 7f8ca3e00700 1 -- 172.21.15.151:0/2152116346 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8c9c023890 con 0x7f8cac07dd80 2024-05-21T21:43:00.405 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:58.513+0000 7f8cb0a00700 1 -- 172.21.15.151:0/2152116346 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cac19f7d0 con 0x7f8cac07dd80 2024-05-21T21:43:00.406 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:58.513+0000 7f8cb0a00700 1 -- 172.21.15.151:0/2152116346 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cac1a24c0 con 0x7f8cac07dd80 2024-05-21T21:43:00.406 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:58.515+0000 7f8cb0a00700 1 -- 172.21.15.151:0/2152116346 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cac06a6a0 con 0x7f8cac07dd80 2024-05-21T21:43:00.406 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:58.515+0000 7f8ca3e00700 1 -- 172.21.15.151:0/2152116346 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f8c9c01b440 con 0x7f8cac07dd80 2024-05-21T21:43:00.406 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:58.515+0000 7f8ca3e00700 1 --2- 172.21.15.151:0/2152116346 >> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] conn(0x7f8c9003e4c0 0x7f8c90040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:43:00.406 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:58.515+0000 7f8ca3e00700 1 -- 172.21.15.151:0/2152116346 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f8c9c053db0 con 0x7f8cac07dd80 2024-05-21T21:43:00.406 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:58.515+0000 7f8caaa00700 1 --2- 172.21.15.151:0/2152116346 >> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] conn(0x7f8c9003e4c0 0x7f8c90040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:43:00.406 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:58.516+0000 7f8caaa00700 1 --2- 172.21.15.151:0/2152116346 >> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] conn(0x7f8c9003e4c0 0x7f8c90040980 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8c98006fd0 tx=0x7f8c98006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:43:00.406 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:58.521+0000 7f8ca3e00700 1 -- 172.21.15.151:0/2152116346 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8c9c01f030 con 0x7f8cac07dd80 2024-05-21T21:43:00.406 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:58.843+0000 7f8cb0a00700 1 -- 172.21.15.151:0/2152116346 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f8cac1a2690 con 0x7f8cac07dd80 2024-05-21T21:43:00.406 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:59.685+0000 7f8ca3e00700 1 -- 172.21.15.151:0/2152116346 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f8c9c016260 con 0x7f8cac07dd80 2024-05-21T21:43:00.406 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:59.689+0000 7f8cb0a00700 1 -- 172.21.15.151:0/2152116346 >> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] conn(0x7f8c9003e4c0 msgr2=0x7f8c90040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:43:00.406 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:59.689+0000 7f8cb0a00700 1 --2- 172.21.15.151:0/2152116346 >> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] conn(0x7f8c9003e4c0 0x7f8c90040980 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8c98006fd0 tx=0x7f8c98006e40 comp rx=0 tx=0).stop 2024-05-21T21:43:00.407 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:59.689+0000 7f8cb0a00700 1 -- 172.21.15.151:0/2152116346 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8cac07dd80 msgr2=0x7f8cac19f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:43:00.407 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:59.689+0000 7f8cb0a00700 1 --2- 172.21.15.151:0/2152116346 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8cac07dd80 0x7f8cac19f000 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c000c00 tx=0x7f8c9c011aa0 comp rx=0 tx=0).stop 2024-05-21T21:43:00.407 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:59.689+0000 7f8cb0a00700 1 -- 172.21.15.151:0/2152116346 shutdown_connections 2024-05-21T21:43:00.407 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:59.689+0000 7f8cb0a00700 1 --2- 172.21.15.151:0/2152116346 >> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] conn(0x7f8c9003e4c0 0x7f8c90040980 secure :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8c98006fd0 tx=0x7f8c98006e40 comp rx=0 tx=0).stop 2024-05-21T21:43:00.407 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:59.689+0000 7f8cb0a00700 1 --2- 172.21.15.151:0/2152116346 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8cac07dd80 0x7f8cac19f000 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:43:00.407 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:59.689+0000 7f8cb0a00700 1 -- 172.21.15.151:0/2152116346 >> 172.21.15.151:0/2152116346 conn(0x7f8cac1043c0 msgr2=0x7f8cac104fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:43:00.407 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:59.691+0000 7f8cb0a00700 1 -- 172.21.15.151:0/2152116346 shutdown_connections 2024-05-21T21:43:00.407 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:42:59.691+0000 7f8cb0a00700 1 -- 172.21.15.151:0/2152116346 wait complete. 2024-05-21T21:43:00.687 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:43:00 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:43:00.397+0000 7ff3f0290200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-21T21:43:00.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:00 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/2152116346' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-05-21T21:43:00.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:00 smithi151 ceph-mon[31623]: mgrmap e9: x(active, since 24s) 2024-05-21T21:43:00.953 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:43:00 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:43:00.807+0000 7ff3f0290200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-21T21:43:00.954 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:43:00 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:43:00.925+0000 7ff3f0290200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-21T21:43:01.952 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:01 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/625770533' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-21T21:43:01.980 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout { 2024-05-21T21:43:01.981 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-05-21T21:43:01.981 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-21T21:43:01.981 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-21T21:43:01.981 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-21T21:43:01.981 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout } 2024-05-21T21:43:01.983 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:00.987+0000 7f7607e00700 1 Processor -- start 2024-05-21T21:43:01.983 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:00.987+0000 7f7607e00700 1 -- start start 2024-05-21T21:43:01.983 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:00.987+0000 7f7607e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7600109040 0x7f7600109460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:43:01.983 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:00.987+0000 7f7607e00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7600109a30 con 0x7f7600109040 2024-05-21T21:43:01.983 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:00.988+0000 7f7605400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7600109040 0x7f7600109460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:43:01.984 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:00.988+0000 7f7605400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7600109040 0x7f7600109460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:33474/0 (socket says 172.21.15.151:33474) 2024-05-21T21:43:01.984 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:00.988+0000 7f7605400700 1 -- 172.21.15.151:0/1399871332 learned_addr learned my addr 172.21.15.151:0/1399871332 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:43:01.984 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:00.988+0000 7f7605400700 1 -- 172.21.15.151:0/1399871332 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f760010a240 con 0x7f7600109040 2024-05-21T21:43:01.984 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:00.988+0000 7f7605400700 1 --2- 172.21.15.151:0/1399871332 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7600109040 0x7f7600109460 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f75f40098a0 tx=0x7f75f4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aac05943c94f770 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:43:01.984 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:00.989+0000 7f75ffe00700 1 -- 172.21.15.151:0/1399871332 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f75f4004160 con 0x7f7600109040 2024-05-21T21:43:01.984 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:00.989+0000 7f75ffe00700 1 -- 172.21.15.151:0/1399871332 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75f4004300 con 0x7f7600109040 2024-05-21T21:43:01.984 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:00.989+0000 7f75ffe00700 1 -- 172.21.15.151:0/1399871332 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f75f4011a20 con 0x7f7600109040 2024-05-21T21:43:01.984 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:00.989+0000 7f7607e00700 1 -- 172.21.15.151:0/1399871332 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7600109040 msgr2=0x7f7600109460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:43:01.984 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:00.989+0000 7f7607e00700 1 --2- 172.21.15.151:0/1399871332 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7600109040 0x7f7600109460 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f75f40098a0 tx=0x7f75f4009bb0 comp rx=0 tx=0).stop 2024-05-21T21:43:01.984 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:00.990+0000 7f7607e00700 1 -- 172.21.15.151:0/1399871332 shutdown_connections 2024-05-21T21:43:01.984 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:00.990+0000 7f7607e00700 1 --2- 172.21.15.151:0/1399871332 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7600109040 0x7f7600109460 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:43:01.988 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:00.990+0000 7f7607e00700 1 -- 172.21.15.151:0/1399871332 >> 172.21.15.151:0/1399871332 conn(0x7f76001045e0 msgr2=0x7f7600106a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:43:01.988 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:00.990+0000 7f7607e00700 1 -- 172.21.15.151:0/1399871332 shutdown_connections 2024-05-21T21:43:01.989 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:00.990+0000 7f7607e00700 1 -- 172.21.15.151:0/1399871332 wait complete. 2024-05-21T21:43:01.989 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:00.991+0000 7f7607e00700 1 Processor -- start 2024-05-21T21:43:01.989 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:00.991+0000 7f7607e00700 1 -- start start 2024-05-21T21:43:01.989 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:00.991+0000 7f7607e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7600109040 0x7f760019f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:43:01.989 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:00.991+0000 7f7607e00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f760010d5f0 con 0x7f7600109040 2024-05-21T21:43:01.989 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:00.992+0000 7f7605400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7600109040 0x7f760019f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:43:01.989 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:00.992+0000 7f7605400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7600109040 0x7f760019f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:33490/0 (socket says 172.21.15.151:33490) 2024-05-21T21:43:01.989 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:00.992+0000 7f7605400700 1 -- 172.21.15.151:0/625770533 learned_addr learned my addr 172.21.15.151:0/625770533 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:43:01.989 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:00.992+0000 7f7605400700 1 -- 172.21.15.151:0/625770533 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f760019f730 con 0x7f7600109040 2024-05-21T21:43:01.989 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:00.992+0000 7f7605400700 1 --2- 172.21.15.151:0/625770533 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7600109040 0x7f760019f1f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f75f4000c00 tx=0x7f75f4011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:43:01.989 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:00.992+0000 7f75fe000700 1 -- 172.21.15.151:0/625770533 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f75f4003aa0 con 0x7f7600109040 2024-05-21T21:43:01.990 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:00.992+0000 7f75fe000700 1 -- 172.21.15.151:0/625770533 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75f401b8f0 con 0x7f7600109040 2024-05-21T21:43:01.990 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:00.992+0000 7f75fe000700 1 -- 172.21.15.151:0/625770533 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f75f4023890 con 0x7f7600109040 2024-05-21T21:43:01.990 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:00.993+0000 7f7607e00700 1 -- 172.21.15.151:0/625770533 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f760019f9c0 con 0x7f7600109040 2024-05-21T21:43:01.990 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:00.993+0000 7f7607e00700 1 -- 172.21.15.151:0/625770533 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76001a26b0 con 0x7f7600109040 2024-05-21T21:43:01.990 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:00.994+0000 7f75fe000700 1 -- 172.21.15.151:0/625770533 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7f75f401b440 con 0x7f7600109040 2024-05-21T21:43:01.990 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:00.994+0000 7f7607e00700 1 -- 172.21.15.151:0/625770533 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7600064ba0 con 0x7f7600109040 2024-05-21T21:43:01.990 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:00.994+0000 7f75fe000700 1 --2- 172.21.15.151:0/625770533 >> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] conn(0x7f75e803e520 0x7f75e80409e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:43:01.990 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:00.994+0000 7f75fe000700 1 -- 172.21.15.151:0/625770533 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f75f4053df0 con 0x7f7600109040 2024-05-21T21:43:01.990 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:00.994+0000 7f7604a00700 1 -- 172.21.15.151:0/625770533 >> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] conn(0x7f75e803e520 msgr2=0x7f75e80409e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.151:6800/3428827300 2024-05-21T21:43:01.990 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:00.994+0000 7f7604a00700 1 --2- 172.21.15.151:0/625770533 >> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] conn(0x7f75e803e520 0x7f75e80409e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-21T21:43:01.990 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:01.005+0000 7f75fe000700 1 -- 172.21.15.151:0/625770533 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f75f401f030 con 0x7f7600109040 2024-05-21T21:43:01.990 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:01.195+0000 7f7604a00700 1 -- 172.21.15.151:0/625770533 >> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] conn(0x7f75e803e520 msgr2=0x7f75e80409e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.151:6800/3428827300 2024-05-21T21:43:01.991 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:01.195+0000 7f7604a00700 1 --2- 172.21.15.151:0/625770533 >> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] conn(0x7f75e803e520 0x7f75e80409e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-21T21:43:01.991 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:01.299+0000 7f7607e00700 1 -- 172.21.15.151:0/625770533 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f76001a2ca0 con 0x7f7600109040 2024-05-21T21:43:01.991 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:01.300+0000 7f75fe000700 1 -- 172.21.15.151:0/625770533 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f75f401bda0 con 0x7f7600109040 2024-05-21T21:43:01.991 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:01.305+0000 7f7607e00700 1 -- 172.21.15.151:0/625770533 >> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] conn(0x7f75e803e520 msgr2=0x7f75e80409e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-21T21:43:01.991 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:01.305+0000 7f7607e00700 1 --2- 172.21.15.151:0/625770533 >> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] conn(0x7f75e803e520 0x7f75e80409e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:43:01.991 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:01.305+0000 7f7607e00700 1 -- 172.21.15.151:0/625770533 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7600109040 msgr2=0x7f760019f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:43:01.991 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:01.305+0000 7f7607e00700 1 --2- 172.21.15.151:0/625770533 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7600109040 0x7f760019f1f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f75f4000c00 tx=0x7f75f4011aa0 comp rx=0 tx=0).stop 2024-05-21T21:43:01.991 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:01.305+0000 7f7607e00700 1 -- 172.21.15.151:0/625770533 shutdown_connections 2024-05-21T21:43:01.991 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:01.305+0000 7f7607e00700 1 --2- 172.21.15.151:0/625770533 >> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] conn(0x7f75e803e520 0x7f75e80409e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:43:01.991 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:01.305+0000 7f7607e00700 1 --2- 172.21.15.151:0/625770533 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7600109040 0x7f760019f1f0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:43:01.991 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:01.305+0000 7f7607e00700 1 -- 172.21.15.151:0/625770533 >> 172.21.15.151:0/625770533 conn(0x7f76001045e0 msgr2=0x7f76001050a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:43:01.991 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:01.305+0000 7f7607e00700 1 -- 172.21.15.151:0/625770533 shutdown_connections 2024-05-21T21:43:01.991 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:01.305+0000 7f7607e00700 1 -- 172.21.15.151:0/625770533 wait complete. 2024-05-21T21:43:01.992 INFO:teuthology.orchestra.run.smithi151.stdout:Waiting for the mgr to restart... 2024-05-21T21:43:01.992 INFO:teuthology.orchestra.run.smithi151.stdout:Waiting for mgr epoch 9... 2024-05-21T21:43:02.565 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:43:02 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:43:02.518+0000 7ff3f0290200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-21T21:43:02.953 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:43:02 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:43:02.744+0000 7ff3f0290200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-21T21:43:03.343 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:43:02 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:43:02.959+0000 7ff3f0290200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-21T21:43:03.343 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:43:03 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:43:03.341+0000 7ff3f0290200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-21T21:43:03.702 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:43:03 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:43:03.462+0000 7ff3f0290200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-21T21:43:03.970 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:43:03 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:43:03.715+0000 7ff3f0290200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-21T21:43:03.970 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:43:03 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:43:03.968+0000 7ff3f0290200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-21T21:43:05.045 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:43:04 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:43:04.752+0000 7ff3f0290200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-21T21:43:05.045 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:43:05 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:43:05.042+0000 7ff3f0290200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-21T21:43:05.953 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:43:05 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:43:05.598+0000 7ff3f0290200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-21T21:43:05.953 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:43:05 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:43:05.727+0000 7ff3f0290200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-21T21:43:06.453 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:43:06 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:43:06.197+0000 7ff3f0290200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-21T21:43:06.788 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:43:06 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:43:06.655+0000 7ff3f0290200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-21T21:43:06.788 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:43:06 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:43:06.786+0000 7ff3f0290200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-21T21:43:07.449 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:43:07 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:43:07.125+0000 7ff3f0290200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-21T21:43:07.449 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:43:07 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:43:07.313+0000 7ff3f0290200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-21T21:43:07.449 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:43:07 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:43:07.447+0000 7ff3f0290200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-21T21:43:08.203 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:43:07 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:43:07.861+0000 7ff3f0290200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-21T21:43:08.203 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:43:07 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:43:07.987+0000 7ff3f0290200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-21T21:43:09.037 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:43:08 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:43:08.748+0000 7ff3f0290200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-21T21:43:09.038 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:43:09 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:43:09.035+0000 7ff3f0290200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-21T21:43:09.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:09 smithi151 ceph-mon[31623]: Active manager daemon x restarted 2024-05-21T21:43:09.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:09 smithi151 ceph-mon[31623]: Activating manager daemon x 2024-05-21T21:43:09.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:09 smithi151 ceph-mon[31623]: osdmap e3: 0 total, 0 up, 0 in 2024-05-21T21:43:09.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:09 smithi151 ceph-mon[31623]: mgrmap e10: x(active, starting, since 0.0114607s) 2024-05-21T21:43:09.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:09 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-21T21:43:09.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:09 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-21T21:43:09.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:09 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-21T21:43:09.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:09 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-21T21:43:09.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:09 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-21T21:43:09.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:09 smithi151 ceph-mon[31623]: Manager daemon x is now available 2024-05-21T21:43:10.432 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:10 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:43:10.432 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:10 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-21T21:43:10.432 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:10 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-21T21:43:10.432 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:10 smithi151 ceph-mon[31623]: mgrmap e11: x(active, since 1.01329s) 2024-05-21T21:43:10.945 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout { 2024-05-21T21:43:10.945 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-05-21T21:43:10.945 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-21T21:43:10.945 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout } 2024-05-21T21:43:10.945 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:02.563+0000 7f792d000700 1 Processor -- start 2024-05-21T21:43:10.945 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:02.564+0000 7f792d000700 1 -- start start 2024-05-21T21:43:10.945 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:02.564+0000 7f792d000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f792807edc0 0x7f792807d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:43:10.945 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:02.564+0000 7f792d000700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f792807d870 con 0x7f792807edc0 2024-05-21T21:43:10.945 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:02.565+0000 7f7925e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f792807edc0 0x7f792807d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:43:10.945 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:02.565+0000 7f7925e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f792807edc0 0x7f792807d330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:33502/0 (socket says 172.21.15.151:33502) 2024-05-21T21:43:10.946 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:02.565+0000 7f7925e00700 1 -- 172.21.15.151:0/3639603995 learned_addr learned my addr 172.21.15.151:0/3639603995 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:43:10.946 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:02.565+0000 7f7925e00700 1 -- 172.21.15.151:0/3639603995 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f792807d9f0 con 0x7f792807edc0 2024-05-21T21:43:10.946 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:02.565+0000 7f7925e00700 1 --2- 172.21.15.151:0/3639603995 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f792807edc0 0x7f792807d330 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f79180098a0 tx=0x7f7918009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aaeafe09a805da8 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:43:10.946 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:02.566+0000 7f7924a00700 1 -- 172.21.15.151:0/3639603995 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7918004160 con 0x7f792807edc0 2024-05-21T21:43:10.946 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:02.566+0000 7f7924a00700 1 -- 172.21.15.151:0/3639603995 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7918004300 con 0x7f792807edc0 2024-05-21T21:43:10.946 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:02.566+0000 7f7924a00700 1 -- 172.21.15.151:0/3639603995 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7918011a20 con 0x7f792807edc0 2024-05-21T21:43:10.946 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:02.566+0000 7f792d000700 1 -- 172.21.15.151:0/3639603995 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f792807edc0 msgr2=0x7f792807d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:43:10.946 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:02.566+0000 7f792d000700 1 --2- 172.21.15.151:0/3639603995 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f792807edc0 0x7f792807d330 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f79180098a0 tx=0x7f7918009bb0 comp rx=0 tx=0).stop 2024-05-21T21:43:10.946 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:02.567+0000 7f792d000700 1 -- 172.21.15.151:0/3639603995 shutdown_connections 2024-05-21T21:43:10.946 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:02.567+0000 7f792d000700 1 --2- 172.21.15.151:0/3639603995 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f792807edc0 0x7f792807d330 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:43:10.946 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:02.567+0000 7f792d000700 1 -- 172.21.15.151:0/3639603995 >> 172.21.15.151:0/3639603995 conn(0x7f7928104730 msgr2=0x7f7928106b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:43:10.946 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:02.567+0000 7f792d000700 1 -- 172.21.15.151:0/3639603995 shutdown_connections 2024-05-21T21:43:10.947 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:02.567+0000 7f792d000700 1 -- 172.21.15.151:0/3639603995 wait complete. 2024-05-21T21:43:10.947 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:02.568+0000 7f792d000700 1 Processor -- start 2024-05-21T21:43:10.947 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:02.568+0000 7f792d000700 1 -- start start 2024-05-21T21:43:10.947 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:02.568+0000 7f792d000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f792807edc0 0x7f792819f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:43:10.947 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:02.568+0000 7f792d000700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f792810f5a0 con 0x7f792807edc0 2024-05-21T21:43:10.947 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:02.569+0000 7f7925e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f792807edc0 0x7f792819f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:43:10.947 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:02.569+0000 7f7925e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f792807edc0 0x7f792819f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:33516/0 (socket says 172.21.15.151:33516) 2024-05-21T21:43:10.947 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:02.569+0000 7f7925e00700 1 -- 172.21.15.151:0/2179841605 learned_addr learned my addr 172.21.15.151:0/2179841605 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:43:10.947 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:02.569+0000 7f7925e00700 1 -- 172.21.15.151:0/2179841605 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f792819f600 con 0x7f792807edc0 2024-05-21T21:43:10.947 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:02.569+0000 7f7925e00700 1 --2- 172.21.15.151:0/2179841605 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f792807edc0 0x7f792819f0c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7918000c00 tx=0x7f7918011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:43:10.947 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:02.570+0000 7f791ea00700 1 -- 172.21.15.151:0/2179841605 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7918003aa0 con 0x7f792807edc0 2024-05-21T21:43:10.947 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:02.570+0000 7f791ea00700 1 -- 172.21.15.151:0/2179841605 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f791801b8f0 con 0x7f792807edc0 2024-05-21T21:43:10.947 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:02.570+0000 7f792d000700 1 -- 172.21.15.151:0/2179841605 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f792819f890 con 0x7f792807edc0 2024-05-21T21:43:10.948 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:02.570+0000 7f791ea00700 1 -- 172.21.15.151:0/2179841605 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7918023870 con 0x7f792807edc0 2024-05-21T21:43:10.948 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:02.570+0000 7f792d000700 1 -- 172.21.15.151:0/2179841605 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79281a2580 con 0x7f792807edc0 2024-05-21T21:43:10.948 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:02.571+0000 7f791ea00700 1 -- 172.21.15.151:0/2179841605 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7f7918023b70 con 0x7f792807edc0 2024-05-21T21:43:10.948 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:02.572+0000 7f791ea00700 1 --2- 172.21.15.151:0/2179841605 >> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] conn(0x7f790c03e4c0 0x7f790c040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:43:10.948 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:02.572+0000 7f791ea00700 1 -- 172.21.15.151:0/2179841605 --> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f790c041090 con 0x7f790c03e4c0 2024-05-21T21:43:10.948 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:02.572+0000 7f791ea00700 1 -- 172.21.15.151:0/2179841605 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f7918053f20 con 0x7f792807edc0 2024-05-21T21:43:10.948 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:02.572+0000 7f7925400700 1 -- 172.21.15.151:0/2179841605 >> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] conn(0x7f790c03e4c0 msgr2=0x7f790c040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.151:6800/3428827300 2024-05-21T21:43:10.948 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:02.572+0000 7f7925400700 1 --2- 172.21.15.151:0/2179841605 >> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] conn(0x7f790c03e4c0 0x7f790c040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-21T21:43:10.948 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:02.773+0000 7f7925400700 1 -- 172.21.15.151:0/2179841605 >> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] conn(0x7f790c03e4c0 msgr2=0x7f790c040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.151:6800/3428827300 2024-05-21T21:43:10.948 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:02.773+0000 7f7925400700 1 --2- 172.21.15.151:0/2179841605 >> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] conn(0x7f790c03e4c0 0x7f790c040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-21T21:43:10.948 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:03.174+0000 7f7925400700 1 -- 172.21.15.151:0/2179841605 >> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] conn(0x7f790c03e4c0 msgr2=0x7f790c040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.151:6800/3428827300 2024-05-21T21:43:10.948 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:03.174+0000 7f7925400700 1 --2- 172.21.15.151:0/2179841605 >> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] conn(0x7f790c03e4c0 0x7f790c040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-21T21:43:10.949 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:03.975+0000 7f7925400700 1 -- 172.21.15.151:0/2179841605 >> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] conn(0x7f790c03e4c0 msgr2=0x7f790c040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.151:6800/3428827300 2024-05-21T21:43:10.949 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:03.975+0000 7f7925400700 1 --2- 172.21.15.151:0/2179841605 >> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] conn(0x7f790c03e4c0 0x7f790c040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-21T21:43:10.949 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:05.577+0000 7f7925400700 1 -- 172.21.15.151:0/2179841605 >> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] conn(0x7f790c03e4c0 msgr2=0x7f790c040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.151:6800/3428827300 2024-05-21T21:43:10.949 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:05.577+0000 7f7925400700 1 --2- 172.21.15.151:0/2179841605 >> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] conn(0x7f790c03e4c0 0x7f790c040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-21T21:43:10.949 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:08.781+0000 7f7925400700 1 -- 172.21.15.151:0/2179841605 >> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] conn(0x7f790c03e4c0 msgr2=0x7f790c040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.151:6800/3428827300 2024-05-21T21:43:10.949 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:08.781+0000 7f7925400700 1 --2- 172.21.15.151:0/2179841605 >> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] conn(0x7f790c03e4c0 0x7f790c040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-05-21T21:43:10.949 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:09.048+0000 7f791ea00700 1 -- 172.21.15.151:0/2179841605 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mgrmap(e 10) ==== 50002+0+0 (secure 0 0 0) 0x7f791802c3f0 con 0x7f792807edc0 2024-05-21T21:43:10.949 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:09.048+0000 7f791ea00700 1 -- 172.21.15.151:0/2179841605 >> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] conn(0x7f790c03e4c0 msgr2=0x7f790c040980 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-21T21:43:10.949 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:09.048+0000 7f791ea00700 1 --2- 172.21.15.151:0/2179841605 >> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] conn(0x7f790c03e4c0 0x7f790c040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:43:10.949 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:10.048+0000 7f791ea00700 1 -- 172.21.15.151:0/2179841605 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mgrmap(e 11) ==== 50129+0+0 (secure 0 0 0) 0x7f7918055d90 con 0x7f792807edc0 2024-05-21T21:43:10.951 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:10.048+0000 7f791ea00700 1 --2- 172.21.15.151:0/2179841605 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f790c041f40 0x7f790c044330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:43:10.952 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:10.048+0000 7f791ea00700 1 -- 172.21.15.151:0/2179841605 --> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f791800bd30 con 0x7f790c041f40 2024-05-21T21:43:10.952 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:10.049+0000 7f7925400700 1 --2- 172.21.15.151:0/2179841605 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f790c041f40 0x7f790c044330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:43:10.952 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:10.050+0000 7f7925400700 1 --2- 172.21.15.151:0/2179841605 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f790c041f40 0x7f790c044330 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7914003a10 tx=0x7f79140092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:43:10.952 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:10.051+0000 7f791ea00700 1 -- 172.21.15.151:0/2179841605 <== mgr.14152 v2:172.21.15.151:6800/2796622483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f791800bd30 con 0x7f790c041f40 2024-05-21T21:43:10.952 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:10.061+0000 7f792d000700 1 -- 172.21.15.151:0/2179841605 --> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f792806a6a0 con 0x7f790c041f40 2024-05-21T21:43:10.952 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:10.061+0000 7f791ea00700 1 -- 172.21.15.151:0/2179841605 <== mgr.14152 v2:172.21.15.151:6800/2796622483 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f792806a6a0 con 0x7f790c041f40 2024-05-21T21:43:10.952 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:10.062+0000 7f792d000700 1 -- 172.21.15.151:0/2179841605 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f790c041f40 msgr2=0x7f790c044330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:43:10.952 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:10.062+0000 7f792d000700 1 --2- 172.21.15.151:0/2179841605 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f790c041f40 0x7f790c044330 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7914003a10 tx=0x7f79140092b0 comp rx=0 tx=0).stop 2024-05-21T21:43:10.952 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:10.062+0000 7f792d000700 1 -- 172.21.15.151:0/2179841605 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f792807edc0 msgr2=0x7f792819f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:43:10.952 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:10.062+0000 7f792d000700 1 --2- 172.21.15.151:0/2179841605 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f792807edc0 0x7f792819f0c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7918000c00 tx=0x7f7918011aa0 comp rx=0 tx=0).stop 2024-05-21T21:43:10.952 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:10.062+0000 7f792d000700 1 -- 172.21.15.151:0/2179841605 shutdown_connections 2024-05-21T21:43:10.952 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:10.062+0000 7f792d000700 1 --2- 172.21.15.151:0/2179841605 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f790c041f40 0x7f790c044330 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:43:10.953 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:10.062+0000 7f792d000700 1 --2- 172.21.15.151:0/2179841605 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f792807edc0 0x7f792819f0c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:43:10.953 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:10.062+0000 7f792d000700 1 --2- 172.21.15.151:0/2179841605 >> [v2:172.21.15.151:6800/3428827300,v1:172.21.15.151:6801/3428827300] conn(0x7f790c03e4c0 0x7f790c040980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:43:10.953 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:10.062+0000 7f792d000700 1 -- 172.21.15.151:0/2179841605 >> 172.21.15.151:0/2179841605 conn(0x7f7928104730 msgr2=0x7f7928105200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:43:10.953 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:10.062+0000 7f792d000700 1 -- 172.21.15.151:0/2179841605 shutdown_connections 2024-05-21T21:43:10.953 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:10.062+0000 7f792d000700 1 -- 172.21.15.151:0/2179841605 wait complete. 2024-05-21T21:43:10.953 INFO:teuthology.orchestra.run.smithi151.stdout:mgr epoch 9 is available 2024-05-21T21:43:10.953 INFO:teuthology.orchestra.run.smithi151.stdout:Generating a dashboard self-signed certificate... 2024-05-21T21:43:11.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:11 smithi151 ceph-mon[31623]: [21/May/2024:21:43:10] ENGINE Bus STARTING 2024-05-21T21:43:11.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:11 smithi151 ceph-mon[31623]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-21T21:43:11.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:11 smithi151 ceph-mon[31623]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-21T21:43:11.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:11 smithi151 ceph-mon[31623]: [21/May/2024:21:43:10] ENGINE Serving on http://172.21.15.151:8765 2024-05-21T21:43:11.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:11 smithi151 ceph-mon[31623]: [21/May/2024:21:43:10] ENGINE Serving on https://172.21.15.151:7150 2024-05-21T21:43:11.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:11 smithi151 ceph-mon[31623]: [21/May/2024:21:43:10] ENGINE Bus STARTED 2024-05-21T21:43:12.384 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:12 smithi151 ceph-mon[31623]: mgrmap e12: x(active, since 2s) 2024-05-21T21:43:12.384 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:12 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:43:13.046 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-05-21T21:43:13.046 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:11.985+0000 7fece8200700 1 Processor -- start 2024-05-21T21:43:13.046 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:11.985+0000 7fece8200700 1 -- start start 2024-05-21T21:43:13.046 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:11.985+0000 7fece8200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fece007dd60 0x7fece007c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:43:13.046 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:11.985+0000 7fece8200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fece007c790 con 0x7fece007dd60 2024-05-21T21:43:13.046 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:11.986+0000 7fece5800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fece007dd60 0x7fece007c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:43:13.047 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:11.986+0000 7fece5800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fece007dd60 0x7fece007c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:33734/0 (socket says 172.21.15.151:33734) 2024-05-21T21:43:13.047 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:11.986+0000 7fece5800700 1 -- 172.21.15.151:0/3376838260 learned_addr learned my addr 172.21.15.151:0/3376838260 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:43:13.047 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:11.986+0000 7fece5800700 1 -- 172.21.15.151:0/3376838260 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fece007c910 con 0x7fece007dd60 2024-05-21T21:43:13.047 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:11.987+0000 7fece5800700 1 --2- 172.21.15.151:0/3376838260 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fece007dd60 0x7fece007c1c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fecd40098a0 tx=0x7fecd4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dfa4dd1202ece97 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:43:13.047 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:11.987+0000 7fecdfe00700 1 -- 172.21.15.151:0/3376838260 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fecd4004160 con 0x7fece007dd60 2024-05-21T21:43:13.047 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:11.987+0000 7fecdfe00700 1 -- 172.21.15.151:0/3376838260 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fecd4004300 con 0x7fece007dd60 2024-05-21T21:43:13.047 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:11.987+0000 7fecdfe00700 1 -- 172.21.15.151:0/3376838260 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fecd4011a20 con 0x7fece007dd60 2024-05-21T21:43:13.047 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:11.988+0000 7fece8200700 1 -- 172.21.15.151:0/3376838260 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fece007dd60 msgr2=0x7fece007c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:43:13.052 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:11.988+0000 7fece8200700 1 --2- 172.21.15.151:0/3376838260 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fece007dd60 0x7fece007c1c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fecd40098a0 tx=0x7fecd4009bb0 comp rx=0 tx=0).stop 2024-05-21T21:43:13.052 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:11.988+0000 7fece8200700 1 -- 172.21.15.151:0/3376838260 shutdown_connections 2024-05-21T21:43:13.052 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:11.988+0000 7fece8200700 1 --2- 172.21.15.151:0/3376838260 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fece007dd60 0x7fece007c1c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:43:13.052 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:11.988+0000 7fece8200700 1 -- 172.21.15.151:0/3376838260 >> 172.21.15.151:0/3376838260 conn(0x7fece01045e0 msgr2=0x7fece0106a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:43:13.052 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:11.988+0000 7fece8200700 1 -- 172.21.15.151:0/3376838260 shutdown_connections 2024-05-21T21:43:13.052 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:11.988+0000 7fece8200700 1 -- 172.21.15.151:0/3376838260 wait complete. 2024-05-21T21:43:13.052 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:11.989+0000 7fece8200700 1 Processor -- start 2024-05-21T21:43:13.052 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:11.989+0000 7fece8200700 1 -- start start 2024-05-21T21:43:13.052 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:11.990+0000 7fece8200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fece007dd60 0x7fece019f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:43:13.052 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:11.990+0000 7fece8200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fece010f660 con 0x7fece007dd60 2024-05-21T21:43:13.052 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:11.990+0000 7fece5800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fece007dd60 0x7fece019f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:43:13.052 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:11.990+0000 7fece5800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fece007dd60 0x7fece019f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:33748/0 (socket says 172.21.15.151:33748) 2024-05-21T21:43:13.053 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:11.990+0000 7fece5800700 1 -- 172.21.15.151:0/817228166 learned_addr learned my addr 172.21.15.151:0/817228166 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:43:13.053 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:11.990+0000 7fece5800700 1 -- 172.21.15.151:0/817228166 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fece019f6b0 con 0x7fece007dd60 2024-05-21T21:43:13.053 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:11.991+0000 7fece5800700 1 --2- 172.21.15.151:0/817228166 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fece007dd60 0x7fece019f170 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fecd4004660 tx=0x7fecd4011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:43:13.053 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:11.991+0000 7fecde000700 1 -- 172.21.15.151:0/817228166 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fecd4003aa0 con 0x7fece007dd60 2024-05-21T21:43:13.053 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:11.991+0000 7fecde000700 1 -- 172.21.15.151:0/817228166 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fecd401b8f0 con 0x7fece007dd60 2024-05-21T21:43:13.053 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:11.991+0000 7fecde000700 1 -- 172.21.15.151:0/817228166 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fecd4023900 con 0x7fece007dd60 2024-05-21T21:43:13.053 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:11.991+0000 7fece8200700 1 -- 172.21.15.151:0/817228166 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fece019f940 con 0x7fece007dd60 2024-05-21T21:43:13.053 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:11.991+0000 7fece8200700 1 -- 172.21.15.151:0/817228166 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fece01a2630 con 0x7fece007dd60 2024-05-21T21:43:13.053 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:11.992+0000 7fece8200700 1 -- 172.21.15.151:0/817228166 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fece0064ba0 con 0x7fece007dd60 2024-05-21T21:43:13.053 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:11.993+0000 7fecde000700 1 -- 172.21.15.151:0/817228166 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7fecd401baa0 con 0x7fece007dd60 2024-05-21T21:43:13.053 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:11.993+0000 7fecde000700 1 --2- 172.21.15.151:0/817228166 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fecc803e4c0 0x7fecc8040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:43:13.053 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:11.994+0000 7fecde000700 1 -- 172.21.15.151:0/817228166 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fecd4053ff0 con 0x7fece007dd60 2024-05-21T21:43:13.054 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:11.994+0000 7fece4e00700 1 --2- 172.21.15.151:0/817228166 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fecc803e4c0 0x7fecc8040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:43:13.054 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:11.995+0000 7fece4e00700 1 --2- 172.21.15.151:0/817228166 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fecc803e4c0 0x7fecc8040980 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7feccc006fd0 tx=0x7feccc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:43:13.054 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:12.004+0000 7fecde000700 1 -- 172.21.15.151:0/817228166 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fecd401f030 con 0x7fece007dd60 2024-05-21T21:43:13.054 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:12.246+0000 7fece8200700 1 -- 172.21.15.151:0/817228166 --> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7fece0105830 con 0x7fecc803e4c0 2024-05-21T21:43:13.054 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:12.382+0000 7fecde000700 1 -- 172.21.15.151:0/817228166 <== mgr.14152 v2:172.21.15.151:6800/2796622483 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7fece0105830 con 0x7fecc803e4c0 2024-05-21T21:43:13.054 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:12.387+0000 7fece8200700 1 -- 172.21.15.151:0/817228166 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fecc803e4c0 msgr2=0x7fecc8040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:43:13.054 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:12.387+0000 7fece8200700 1 --2- 172.21.15.151:0/817228166 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fecc803e4c0 0x7fecc8040980 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7feccc006fd0 tx=0x7feccc006e40 comp rx=0 tx=0).stop 2024-05-21T21:43:13.054 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:12.387+0000 7fece8200700 1 -- 172.21.15.151:0/817228166 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fece007dd60 msgr2=0x7fece019f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:43:13.054 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:12.388+0000 7fece8200700 1 --2- 172.21.15.151:0/817228166 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fece007dd60 0x7fece019f170 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fecd4004660 tx=0x7fecd4011aa0 comp rx=0 tx=0).stop 2024-05-21T21:43:13.054 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:12.388+0000 7fece8200700 1 -- 172.21.15.151:0/817228166 shutdown_connections 2024-05-21T21:43:13.054 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:12.388+0000 7fece8200700 1 --2- 172.21.15.151:0/817228166 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fecc803e4c0 0x7fecc8040980 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:43:13.054 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:12.388+0000 7fece8200700 1 --2- 172.21.15.151:0/817228166 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fece007dd60 0x7fece019f170 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:43:13.054 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:12.388+0000 7fece8200700 1 -- 172.21.15.151:0/817228166 >> 172.21.15.151:0/817228166 conn(0x7fece01045e0 msgr2=0x7fece0105120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:43:13.055 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:12.388+0000 7fece8200700 1 -- 172.21.15.151:0/817228166 shutdown_connections 2024-05-21T21:43:13.055 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:12.388+0000 7fece8200700 1 -- 172.21.15.151:0/817228166 wait complete. 2024-05-21T21:43:13.055 INFO:teuthology.orchestra.run.smithi151.stdout:Creating initial admin user... 2024-05-21T21:43:13.680 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:13 smithi151 ceph-mon[31623]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:43:13.681 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:13 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:43:13.681 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:13 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:43:14.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:14 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:43:14.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:14 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:43:14.986 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$I9p.GnSi.7XbNeq0J5OS/O8mcBddUGS7E.x.txT4XSr55eT9q1jvO", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1716327794, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-05-21T21:43:14.987 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:13.768+0000 7f907c400700 1 Processor -- start 2024-05-21T21:43:14.987 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:13.768+0000 7f907c400700 1 -- start start 2024-05-21T21:43:14.987 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:13.769+0000 7f907c400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f907407dd40 0x7f907407c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:43:14.988 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:13.769+0000 7f907c400700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f907407c770 con 0x7f907407dd40 2024-05-21T21:43:14.988 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:13.769+0000 7f9079a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f907407dd40 0x7f907407c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:43:14.988 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:13.769+0000 7f9079a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f907407dd40 0x7f907407c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:33762/0 (socket says 172.21.15.151:33762) 2024-05-21T21:43:14.988 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:13.769+0000 7f9079a00700 1 -- 172.21.15.151:0/2801695226 learned_addr learned my addr 172.21.15.151:0/2801695226 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:43:14.988 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:13.770+0000 7f9079a00700 1 -- 172.21.15.151:0/2801695226 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f907407c8f0 con 0x7f907407dd40 2024-05-21T21:43:14.989 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:13.770+0000 7f9079a00700 1 --2- 172.21.15.151:0/2801695226 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f907407dd40 0x7f907407c1a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9068009e80 tx=0x7f906800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df6104e3378f49e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:43:14.989 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:13.770+0000 7f9073e00700 1 -- 172.21.15.151:0/2801695226 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9068017070 con 0x7f907407dd40 2024-05-21T21:43:14.989 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:13.771+0000 7f9073e00700 1 -- 172.21.15.151:0/2801695226 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f906800b8c0 con 0x7f907407dd40 2024-05-21T21:43:14.989 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:13.771+0000 7f9073e00700 1 -- 172.21.15.151:0/2801695226 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9068004490 con 0x7f907407dd40 2024-05-21T21:43:14.989 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:13.771+0000 7f907c400700 1 -- 172.21.15.151:0/2801695226 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f907407dd40 msgr2=0x7f907407c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:43:14.989 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:13.771+0000 7f907c400700 1 --2- 172.21.15.151:0/2801695226 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f907407dd40 0x7f907407c1a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9068009e80 tx=0x7f906800b330 comp rx=0 tx=0).stop 2024-05-21T21:43:14.989 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:13.771+0000 7f907c400700 1 -- 172.21.15.151:0/2801695226 shutdown_connections 2024-05-21T21:43:14.992 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:13.771+0000 7f907c400700 1 --2- 172.21.15.151:0/2801695226 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f907407dd40 0x7f907407c1a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:43:14.992 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:13.771+0000 7f907c400700 1 -- 172.21.15.151:0/2801695226 >> 172.21.15.151:0/2801695226 conn(0x7f90741045e0 msgr2=0x7f9074106a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:43:14.992 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:13.771+0000 7f907c400700 1 -- 172.21.15.151:0/2801695226 shutdown_connections 2024-05-21T21:43:14.992 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:13.771+0000 7f907c400700 1 -- 172.21.15.151:0/2801695226 wait complete. 2024-05-21T21:43:14.992 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:13.772+0000 7f907c400700 1 Processor -- start 2024-05-21T21:43:14.992 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:13.773+0000 7f907c400700 1 -- start start 2024-05-21T21:43:14.992 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:13.773+0000 7f907c400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f907407dd40 0x7f907419f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:43:14.992 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:13.773+0000 7f907c400700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f907410f620 con 0x7f907407dd40 2024-05-21T21:43:14.992 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:13.773+0000 7f9079a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f907407dd40 0x7f907419f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:43:14.992 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:13.773+0000 7f9079a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f907407dd40 0x7f907419f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:33778/0 (socket says 172.21.15.151:33778) 2024-05-21T21:43:14.992 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:13.773+0000 7f9079a00700 1 -- 172.21.15.151:0/1432802105 learned_addr learned my addr 172.21.15.151:0/1432802105 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:43:14.992 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:13.774+0000 7f9079a00700 1 -- 172.21.15.151:0/1432802105 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f907419f710 con 0x7f907407dd40 2024-05-21T21:43:14.993 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:13.774+0000 7f9079a00700 1 --2- 172.21.15.151:0/1432802105 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f907407dd40 0x7f907419f1d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f906800b7b0 tx=0x7f906800bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:43:14.993 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:13.774+0000 7f9072000700 1 -- 172.21.15.151:0/1432802105 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9068020020 con 0x7f907407dd40 2024-05-21T21:43:14.993 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:13.774+0000 7f9072000700 1 -- 172.21.15.151:0/1432802105 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9068003c60 con 0x7f907407dd40 2024-05-21T21:43:14.993 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:13.774+0000 7f907c400700 1 -- 172.21.15.151:0/1432802105 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f907419f9a0 con 0x7f907407dd40 2024-05-21T21:43:14.993 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:13.774+0000 7f9072000700 1 -- 172.21.15.151:0/1432802105 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9068017040 con 0x7f907407dd40 2024-05-21T21:43:14.993 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:13.774+0000 7f907c400700 1 -- 172.21.15.151:0/1432802105 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90741a2690 con 0x7f907407dd40 2024-05-21T21:43:14.993 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:13.776+0000 7f907c400700 1 -- 172.21.15.151:0/1432802105 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9074064ba0 con 0x7f907407dd40 2024-05-21T21:43:14.993 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:13.776+0000 7f9072000700 1 -- 172.21.15.151:0/1432802105 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7f9068023020 con 0x7f907407dd40 2024-05-21T21:43:14.993 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:13.776+0000 7f9072000700 1 --2- 172.21.15.151:0/1432802105 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f905c03e4c0 0x7f905c040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:43:14.993 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:13.776+0000 7f9072000700 1 -- 172.21.15.151:0/1432802105 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f906800f050 con 0x7f907407dd40 2024-05-21T21:43:14.993 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:13.776+0000 7f9079000700 1 --2- 172.21.15.151:0/1432802105 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f905c03e4c0 0x7f905c040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:43:14.993 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:13.777+0000 7f9079000700 1 --2- 172.21.15.151:0/1432802105 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f905c03e4c0 0x7f905c040980 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9060006fd0 tx=0x7f9060006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:43:14.993 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:13.782+0000 7f9072000700 1 -- 172.21.15.151:0/1432802105 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f906802a020 con 0x7f907407dd40 2024-05-21T21:43:14.993 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:14.029+0000 7f907c400700 1 -- 172.21.15.151:0/1432802105 --> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f9074105870 con 0x7f905c03e4c0 2024-05-21T21:43:14.994 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:14.320+0000 7f9072000700 1 -- 172.21.15.151:0/1432802105 <== mgr.14152 v2:172.21.15.151:6800/2796622483 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f9074105870 con 0x7f905c03e4c0 2024-05-21T21:43:14.994 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:14.323+0000 7f907c400700 1 -- 172.21.15.151:0/1432802105 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f905c03e4c0 msgr2=0x7f905c040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:43:14.994 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:14.323+0000 7f907c400700 1 --2- 172.21.15.151:0/1432802105 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f905c03e4c0 0x7f905c040980 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9060006fd0 tx=0x7f9060006e40 comp rx=0 tx=0).stop 2024-05-21T21:43:14.994 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:14.323+0000 7f907c400700 1 -- 172.21.15.151:0/1432802105 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f907407dd40 msgr2=0x7f907419f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:43:14.994 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:14.323+0000 7f907c400700 1 --2- 172.21.15.151:0/1432802105 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f907407dd40 0x7f907419f1d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f906800b7b0 tx=0x7f906800bd50 comp rx=0 tx=0).stop 2024-05-21T21:43:14.994 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:14.324+0000 7f907c400700 1 -- 172.21.15.151:0/1432802105 shutdown_connections 2024-05-21T21:43:14.994 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:14.324+0000 7f907c400700 1 --2- 172.21.15.151:0/1432802105 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f905c03e4c0 0x7f905c040980 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:43:14.994 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:14.324+0000 7f907c400700 1 --2- 172.21.15.151:0/1432802105 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f907407dd40 0x7f907419f1d0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:43:14.994 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:14.324+0000 7f907c400700 1 -- 172.21.15.151:0/1432802105 >> 172.21.15.151:0/1432802105 conn(0x7f90741045e0 msgr2=0x7f9074105160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:43:14.994 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:14.324+0000 7f907c400700 1 -- 172.21.15.151:0/1432802105 shutdown_connections 2024-05-21T21:43:14.994 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:14.324+0000 7f907c400700 1 -- 172.21.15.151:0/1432802105 wait complete. 2024-05-21T21:43:14.994 INFO:teuthology.orchestra.run.smithi151.stdout:Fetching dashboard port number... 2024-05-21T21:43:15.865 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:15 smithi151 ceph-mon[31623]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:43:16.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:16 smithi151 ceph-mon[31623]: mgrmap e13: x(active, since 6s) 2024-05-21T21:43:16.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:16 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/3662392588' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-05-21T21:43:17.320 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stdout 8443 2024-05-21T21:43:17.320 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:15.831+0000 7f8a8e000700 1 Processor -- start 2024-05-21T21:43:17.320 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:15.831+0000 7f8a8e000700 1 -- start start 2024-05-21T21:43:17.320 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:15.831+0000 7f8a8e000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a88109560 0x7f8a881099a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:43:17.320 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:15.831+0000 7f8a8e000700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8a88109ee0 con 0x7f8a88109560 2024-05-21T21:43:17.320 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:15.832+0000 7f8a87400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a88109560 0x7f8a881099a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:43:17.320 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:15.832+0000 7f8a87400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a88109560 0x7f8a881099a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:35980/0 (socket says 172.21.15.151:35980) 2024-05-21T21:43:17.320 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:15.832+0000 7f8a87400700 1 -- 172.21.15.151:0/39556514 learned_addr learned my addr 172.21.15.151:0/39556514 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:43:17.321 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:15.832+0000 7f8a87400700 1 -- 172.21.15.151:0/39556514 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a88107b60 con 0x7f8a88109560 2024-05-21T21:43:17.321 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:15.833+0000 7f8a87400700 1 --2- 172.21.15.151:0/39556514 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a88109560 0x7f8a881099a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8a780098a0 tx=0x7f8a78009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32926cc00bfa66ac server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:43:17.321 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:15.833+0000 7f8a86000700 1 -- 172.21.15.151:0/39556514 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8a78004160 con 0x7f8a88109560 2024-05-21T21:43:17.321 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:15.833+0000 7f8a86000700 1 -- 172.21.15.151:0/39556514 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a78004300 con 0x7f8a88109560 2024-05-21T21:43:17.321 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:15.833+0000 7f8a86000700 1 -- 172.21.15.151:0/39556514 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8a78011a20 con 0x7f8a88109560 2024-05-21T21:43:17.321 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:15.833+0000 7f8a8e000700 1 -- 172.21.15.151:0/39556514 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a88109560 msgr2=0x7f8a881099a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:43:17.321 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:15.833+0000 7f8a8e000700 1 --2- 172.21.15.151:0/39556514 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a88109560 0x7f8a881099a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8a780098a0 tx=0x7f8a78009bb0 comp rx=0 tx=0).stop 2024-05-21T21:43:17.321 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:15.834+0000 7f8a8e000700 1 -- 172.21.15.151:0/39556514 shutdown_connections 2024-05-21T21:43:17.321 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:15.834+0000 7f8a8e000700 1 --2- 172.21.15.151:0/39556514 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a88109560 0x7f8a881099a0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:43:17.321 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:15.834+0000 7f8a8e000700 1 -- 172.21.15.151:0/39556514 >> 172.21.15.151:0/39556514 conn(0x7f8a88103500 msgr2=0x7f8a88105920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:43:17.321 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:15.834+0000 7f8a8e000700 1 -- 172.21.15.151:0/39556514 shutdown_connections 2024-05-21T21:43:17.321 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:15.834+0000 7f8a8e000700 1 -- 172.21.15.151:0/39556514 wait complete. 2024-05-21T21:43:17.322 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:15.835+0000 7f8a8e000700 1 Processor -- start 2024-05-21T21:43:17.322 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:15.835+0000 7f8a8e000700 1 -- start start 2024-05-21T21:43:17.322 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:15.835+0000 7f8a8e000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a88109560 0x7f8a8819f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:43:17.322 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:15.835+0000 7f8a8e000700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8a8810c510 con 0x7f8a88109560 2024-05-21T21:43:17.322 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:15.835+0000 7f8a87400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a88109560 0x7f8a8819f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:43:17.322 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:15.836+0000 7f8a87400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a88109560 0x7f8a8819f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:35996/0 (socket says 172.21.15.151:35996) 2024-05-21T21:43:17.322 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:15.836+0000 7f8a87400700 1 -- 172.21.15.151:0/3662392588 learned_addr learned my addr 172.21.15.151:0/3662392588 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:43:17.322 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:15.836+0000 7f8a87400700 1 -- 172.21.15.151:0/3662392588 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a8819f850 con 0x7f8a88109560 2024-05-21T21:43:17.322 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:15.836+0000 7f8a87400700 1 --2- 172.21.15.151:0/3662392588 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a88109560 0x7f8a8819f310 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8a78009520 tx=0x7f8a78011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:43:17.322 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:15.836+0000 7f8a7fe00700 1 -- 172.21.15.151:0/3662392588 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8a780039e0 con 0x7f8a88109560 2024-05-21T21:43:17.322 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:15.836+0000 7f8a7fe00700 1 -- 172.21.15.151:0/3662392588 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a7801b8f0 con 0x7f8a88109560 2024-05-21T21:43:17.322 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:15.836+0000 7f8a8e000700 1 -- 172.21.15.151:0/3662392588 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a8819fae0 con 0x7f8a88109560 2024-05-21T21:43:17.323 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:15.836+0000 7f8a7fe00700 1 -- 172.21.15.151:0/3662392588 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8a78019770 con 0x7f8a88109560 2024-05-21T21:43:17.323 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:15.837+0000 7f8a8e000700 1 -- 172.21.15.151:0/3662392588 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a8819ff40 con 0x7f8a88109560 2024-05-21T21:43:17.323 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:15.838+0000 7f8a7fe00700 1 -- 172.21.15.151:0/3662392588 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7f8a7801b440 con 0x7f8a88109560 2024-05-21T21:43:17.323 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:15.838+0000 7f8a8e000700 1 -- 172.21.15.151:0/3662392588 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a8810c690 con 0x7f8a88109560 2024-05-21T21:43:17.323 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:15.838+0000 7f8a7fe00700 1 --2- 172.21.15.151:0/3662392588 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f8a6c03e560 0x7f8a6c040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:43:17.323 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:15.838+0000 7f8a7fe00700 1 -- 172.21.15.151:0/3662392588 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f8a78054110 con 0x7f8a88109560 2024-05-21T21:43:17.323 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:15.839+0000 7f8a86a00700 1 --2- 172.21.15.151:0/3662392588 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f8a6c03e560 0x7f8a6c040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:43:17.323 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:15.839+0000 7f8a86a00700 1 --2- 172.21.15.151:0/3662392588 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f8a6c03e560 0x7f8a6c040a20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8a74006fd0 tx=0x7f8a74006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:43:17.323 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:15.844+0000 7f8a7fe00700 1 -- 172.21.15.151:0/3662392588 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8a7801f030 con 0x7f8a88109560 2024-05-21T21:43:17.323 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:16.077+0000 7f8a8e000700 1 -- 172.21.15.151:0/3662392588 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f8a8802e860 con 0x7f8a88109560 2024-05-21T21:43:17.323 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:16.078+0000 7f8a7fe00700 1 -- 172.21.15.151:0/3662392588 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f8a78056150 con 0x7f8a88109560 2024-05-21T21:43:17.323 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:16.081+0000 7f8a8e000700 1 -- 172.21.15.151:0/3662392588 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f8a6c03e560 msgr2=0x7f8a6c040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:43:17.324 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:16.081+0000 7f8a8e000700 1 --2- 172.21.15.151:0/3662392588 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f8a6c03e560 0x7f8a6c040a20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8a74006fd0 tx=0x7f8a74006e40 comp rx=0 tx=0).stop 2024-05-21T21:43:17.324 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:16.081+0000 7f8a8e000700 1 -- 172.21.15.151:0/3662392588 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a88109560 msgr2=0x7f8a8819f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:43:17.324 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:16.081+0000 7f8a8e000700 1 --2- 172.21.15.151:0/3662392588 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a88109560 0x7f8a8819f310 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8a78009520 tx=0x7f8a78011aa0 comp rx=0 tx=0).stop 2024-05-21T21:43:17.324 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:16.081+0000 7f8a8e000700 1 -- 172.21.15.151:0/3662392588 shutdown_connections 2024-05-21T21:43:17.324 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:16.081+0000 7f8a8e000700 1 --2- 172.21.15.151:0/3662392588 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f8a6c03e560 0x7f8a6c040a20 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:43:17.324 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:16.081+0000 7f8a8e000700 1 --2- 172.21.15.151:0/3662392588 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a88109560 0x7f8a8819f310 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:43:17.324 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:16.081+0000 7f8a8e000700 1 -- 172.21.15.151:0/3662392588 >> 172.21.15.151:0/3662392588 conn(0x7f8a88103500 msgr2=0x7f8a88104300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:43:17.324 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:16.082+0000 7f8a8e000700 1 -- 172.21.15.151:0/3662392588 shutdown_connections 2024-05-21T21:43:17.324 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:16.082+0000 7f8a8e000700 1 -- 172.21.15.151:0/3662392588 wait complete. 2024-05-21T21:43:17.341 INFO:teuthology.orchestra.run.smithi151.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-21T21:43:17.341 INFO:teuthology.orchestra.run.smithi151.stdout:systemctl: stdout disabled 2024-05-21T21:43:17.356 INFO:teuthology.orchestra.run.smithi151.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-21T21:43:17.356 INFO:teuthology.orchestra.run.smithi151.stdout:systemctl: stdout inactive 2024-05-21T21:43:17.356 INFO:teuthology.orchestra.run.smithi151.stdout:firewalld.service is not enabled 2024-05-21T21:43:17.356 INFO:teuthology.orchestra.run.smithi151.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-05-21T21:43:17.359 INFO:teuthology.orchestra.run.smithi151.stdout:Ceph Dashboard is now available at: 2024-05-21T21:43:17.360 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T21:43:17.360 INFO:teuthology.orchestra.run.smithi151.stdout: URL: https://smithi151.front.sepia.ceph.com:8443/ 2024-05-21T21:43:17.360 INFO:teuthology.orchestra.run.smithi151.stdout: User: admin 2024-05-21T21:43:17.360 INFO:teuthology.orchestra.run.smithi151.stdout: Password: ugh17xd0x7 2024-05-21T21:43:17.360 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T21:43:17.360 INFO:teuthology.orchestra.run.smithi151.stdout:Saving cluster configuration to /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/config directory 2024-05-21T21:43:17.361 INFO:teuthology.orchestra.run.smithi151.stdout:Enabling autotune for osd_memory_target 2024-05-21T21:43:18.879 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:17.967+0000 7f4f78600700 1 Processor -- start 2024-05-21T21:43:18.879 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:17.967+0000 7f4f78600700 1 -- start start 2024-05-21T21:43:18.879 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:17.968+0000 7f4f78600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4f7010c910 0x7f4f7010cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:43:18.880 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:17.968+0000 7f4f78600700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4f7010aea0 con 0x7f4f7010c910 2024-05-21T21:43:18.880 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:17.968+0000 7f4f75c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4f7010c910 0x7f4f7010cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:43:18.880 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:17.968+0000 7f4f75c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4f7010c910 0x7f4f7010cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:36004/0 (socket says 172.21.15.151:36004) 2024-05-21T21:43:18.880 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:17.968+0000 7f4f75c00700 1 -- 172.21.15.151:0/291210271 learned_addr learned my addr 172.21.15.151:0/291210271 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:43:18.880 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:17.969+0000 7f4f75c00700 1 -- 172.21.15.151:0/291210271 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f7010b020 con 0x7f4f7010c910 2024-05-21T21:43:18.880 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:17.969+0000 7f4f75c00700 1 --2- 172.21.15.151:0/291210271 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4f7010c910 0x7f4f7010cd30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4f640098a0 tx=0x7f4f64009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d747b231eaf22a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:43:18.880 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:17.969+0000 7f4f6fe00700 1 -- 172.21.15.151:0/291210271 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f64004160 con 0x7f4f7010c910 2024-05-21T21:43:18.880 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:17.969+0000 7f4f6fe00700 1 -- 172.21.15.151:0/291210271 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f64004300 con 0x7f4f7010c910 2024-05-21T21:43:18.880 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:17.969+0000 7f4f6fe00700 1 -- 172.21.15.151:0/291210271 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f64011a20 con 0x7f4f7010c910 2024-05-21T21:43:18.880 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:17.970+0000 7f4f78600700 1 -- 172.21.15.151:0/291210271 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4f7010c910 msgr2=0x7f4f7010cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:43:18.880 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:17.970+0000 7f4f78600700 1 --2- 172.21.15.151:0/291210271 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4f7010c910 0x7f4f7010cd30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4f640098a0 tx=0x7f4f64009bb0 comp rx=0 tx=0).stop 2024-05-21T21:43:18.880 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:17.970+0000 7f4f78600700 1 -- 172.21.15.151:0/291210271 shutdown_connections 2024-05-21T21:43:18.881 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:17.970+0000 7f4f78600700 1 --2- 172.21.15.151:0/291210271 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4f7010c910 0x7f4f7010cd30 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:43:18.881 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:17.970+0000 7f4f78600700 1 -- 172.21.15.151:0/291210271 >> 172.21.15.151:0/291210271 conn(0x7f4f7007e890 msgr2=0x7f4f701094c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:43:18.881 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:17.970+0000 7f4f78600700 1 -- 172.21.15.151:0/291210271 shutdown_connections 2024-05-21T21:43:18.881 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:17.970+0000 7f4f78600700 1 -- 172.21.15.151:0/291210271 wait complete. 2024-05-21T21:43:18.881 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:17.972+0000 7f4f78600700 1 Processor -- start 2024-05-21T21:43:18.881 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:17.972+0000 7f4f78600700 1 -- start start 2024-05-21T21:43:18.881 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:17.972+0000 7f4f78600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4f7010c910 0x7f4f7019f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:43:18.881 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:17.972+0000 7f4f78600700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4f7010f7e0 con 0x7f4f7010c910 2024-05-21T21:43:18.884 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:17.972+0000 7f4f75c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4f7010c910 0x7f4f7019f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:43:18.884 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:17.972+0000 7f4f75c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4f7010c910 0x7f4f7019f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:36020/0 (socket says 172.21.15.151:36020) 2024-05-21T21:43:18.884 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:17.972+0000 7f4f75c00700 1 -- 172.21.15.151:0/2387628177 learned_addr learned my addr 172.21.15.151:0/2387628177 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:43:18.884 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:17.973+0000 7f4f75c00700 1 -- 172.21.15.151:0/2387628177 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f7019f760 con 0x7f4f7010c910 2024-05-21T21:43:18.884 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:17.973+0000 7f4f75c00700 1 --2- 172.21.15.151:0/2387628177 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4f7010c910 0x7f4f7019f220 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4f64009520 tx=0x7f4f64011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:43:18.885 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:17.973+0000 7f4f6e000700 1 -- 172.21.15.151:0/2387628177 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f640039e0 con 0x7f4f7010c910 2024-05-21T21:43:18.885 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:17.973+0000 7f4f6e000700 1 -- 172.21.15.151:0/2387628177 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f6401b8f0 con 0x7f4f7010c910 2024-05-21T21:43:18.885 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:17.973+0000 7f4f78600700 1 -- 172.21.15.151:0/2387628177 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f7019f9f0 con 0x7f4f7010c910 2024-05-21T21:43:18.885 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:17.973+0000 7f4f6e000700 1 -- 172.21.15.151:0/2387628177 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f64019770 con 0x7f4f7010c910 2024-05-21T21:43:18.885 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:17.974+0000 7f4f78600700 1 -- 172.21.15.151:0/2387628177 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f701a26e0 con 0x7f4f7010c910 2024-05-21T21:43:18.885 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:17.975+0000 7f4f78600700 1 -- 172.21.15.151:0/2387628177 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f7010f960 con 0x7f4f7010c910 2024-05-21T21:43:18.885 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:17.975+0000 7f4f6e000700 1 -- 172.21.15.151:0/2387628177 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7f4f6401b440 con 0x7f4f7010c910 2024-05-21T21:43:18.885 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:17.975+0000 7f4f6e000700 1 --2- 172.21.15.151:0/2387628177 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f4f5803e5b0 0x7f4f58040a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:43:18.885 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:17.975+0000 7f4f6e000700 1 -- 172.21.15.151:0/2387628177 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f4f64054090 con 0x7f4f7010c910 2024-05-21T21:43:18.885 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:17.976+0000 7f4f75200700 1 --2- 172.21.15.151:0/2387628177 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f4f5803e5b0 0x7f4f58040a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:43:18.885 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:17.976+0000 7f4f75200700 1 --2- 172.21.15.151:0/2387628177 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f4f5803e5b0 0x7f4f58040a70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c006fd0 tx=0x7f4f5c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:43:18.885 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:17.981+0000 7f4f6e000700 1 -- 172.21.15.151:0/2387628177 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4f6401f030 con 0x7f4f7010c910 2024-05-21T21:43:18.886 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:18.213+0000 7f4f78600700 1 -- 172.21.15.151:0/2387628177 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f4f7019fed0 con 0x7f4f7010c910 2024-05-21T21:43:18.886 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:18.214+0000 7f4f6e000700 1 -- 172.21.15.151:0/2387628177 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f4f7019fed0 con 0x7f4f7010c910 2024-05-21T21:43:18.886 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:18.217+0000 7f4f78600700 1 -- 172.21.15.151:0/2387628177 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f4f5803e5b0 msgr2=0x7f4f58040a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:43:18.886 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:18.217+0000 7f4f78600700 1 --2- 172.21.15.151:0/2387628177 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f4f5803e5b0 0x7f4f58040a70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c006fd0 tx=0x7f4f5c006e40 comp rx=0 tx=0).stop 2024-05-21T21:43:18.886 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:18.217+0000 7f4f78600700 1 -- 172.21.15.151:0/2387628177 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4f7010c910 msgr2=0x7f4f7019f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:43:18.886 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:18.217+0000 7f4f78600700 1 --2- 172.21.15.151:0/2387628177 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4f7010c910 0x7f4f7019f220 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4f64009520 tx=0x7f4f64011aa0 comp rx=0 tx=0).stop 2024-05-21T21:43:18.886 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:18.217+0000 7f4f78600700 1 -- 172.21.15.151:0/2387628177 shutdown_connections 2024-05-21T21:43:18.886 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:18.217+0000 7f4f78600700 1 --2- 172.21.15.151:0/2387628177 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f4f5803e5b0 0x7f4f58040a70 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:43:18.886 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:18.217+0000 7f4f78600700 1 --2- 172.21.15.151:0/2387628177 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4f7010c910 0x7f4f7019f220 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:43:18.886 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:18.217+0000 7f4f78600700 1 -- 172.21.15.151:0/2387628177 >> 172.21.15.151:0/2387628177 conn(0x7f4f7007e890 msgr2=0x7f4f70108b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:43:18.886 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:18.218+0000 7f4f78600700 1 -- 172.21.15.151:0/2387628177 shutdown_connections 2024-05-21T21:43:18.886 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:18.218+0000 7f4f78600700 1 -- 172.21.15.151:0/2387628177 wait complete. 2024-05-21T21:43:20.521 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:19.490+0000 7f4803e00700 1 Processor -- start 2024-05-21T21:43:20.521 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:19.490+0000 7f4803e00700 1 -- start start 2024-05-21T21:43:20.522 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:19.490+0000 7f4803e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f480410fcc0 0x7f48041120b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:43:20.522 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:19.491+0000 7f4803e00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f48041125f0 con 0x7f480410fcc0 2024-05-21T21:43:20.522 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:19.491+0000 7f4802a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f480410fcc0 0x7f48041120b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:43:20.522 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:19.491+0000 7f4802a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f480410fcc0 0x7f48041120b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:36028/0 (socket says 172.21.15.151:36028) 2024-05-21T21:43:20.524 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:19.491+0000 7f4802a00700 1 -- 172.21.15.151:0/460735849 learned_addr learned my addr 172.21.15.151:0/460735849 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:43:20.524 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:19.492+0000 7f4802a00700 1 -- 172.21.15.151:0/460735849 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4804112770 con 0x7f480410fcc0 2024-05-21T21:43:20.524 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:19.492+0000 7f4802a00700 1 --2- 172.21.15.151:0/460735849 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f480410fcc0 0x7f48041120b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f47f40098a0 tx=0x7f47f4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ffd38e0334e6965 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:43:20.524 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:19.492+0000 7f4801600700 1 -- 172.21.15.151:0/460735849 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f47f4004160 con 0x7f480410fcc0 2024-05-21T21:43:20.524 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:19.492+0000 7f4801600700 1 -- 172.21.15.151:0/460735849 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47f4004300 con 0x7f480410fcc0 2024-05-21T21:43:20.524 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:19.492+0000 7f4801600700 1 -- 172.21.15.151:0/460735849 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f47f4011a20 con 0x7f480410fcc0 2024-05-21T21:43:20.524 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:19.493+0000 7f4803e00700 1 -- 172.21.15.151:0/460735849 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f480410fcc0 msgr2=0x7f48041120b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:43:20.524 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:19.493+0000 7f4803e00700 1 --2- 172.21.15.151:0/460735849 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f480410fcc0 0x7f48041120b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f47f40098a0 tx=0x7f47f4009bb0 comp rx=0 tx=0).stop 2024-05-21T21:43:20.524 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:19.493+0000 7f4803e00700 1 -- 172.21.15.151:0/460735849 shutdown_connections 2024-05-21T21:43:20.524 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:19.493+0000 7f4803e00700 1 --2- 172.21.15.151:0/460735849 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f480410fcc0 0x7f48041120b0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:43:20.524 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:19.493+0000 7f4803e00700 1 -- 172.21.15.151:0/460735849 >> 172.21.15.151:0/460735849 conn(0x7f480407e510 msgr2=0x7f4804080930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:43:20.524 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:19.493+0000 7f4803e00700 1 -- 172.21.15.151:0/460735849 shutdown_connections 2024-05-21T21:43:20.525 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:19.493+0000 7f4803e00700 1 -- 172.21.15.151:0/460735849 wait complete. 2024-05-21T21:43:20.525 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:19.494+0000 7f4803e00700 1 Processor -- start 2024-05-21T21:43:20.525 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:19.494+0000 7f4803e00700 1 -- start start 2024-05-21T21:43:20.525 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:19.494+0000 7f4803e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f480410fcc0 0x7f48041a3350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:43:20.525 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:19.494+0000 7f4803e00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4804113830 con 0x7f480410fcc0 2024-05-21T21:43:20.525 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:19.495+0000 7f4802a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f480410fcc0 0x7f48041a3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:43:20.528 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:19.495+0000 7f4802a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f480410fcc0 0x7f48041a3350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:36034/0 (socket says 172.21.15.151:36034) 2024-05-21T21:43:20.528 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:19.495+0000 7f4802a00700 1 -- 172.21.15.151:0/3929893526 learned_addr learned my addr 172.21.15.151:0/3929893526 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:43:20.528 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:19.495+0000 7f4802a00700 1 -- 172.21.15.151:0/3929893526 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48041a3890 con 0x7f480410fcc0 2024-05-21T21:43:20.528 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:19.495+0000 7f4802a00700 1 --2- 172.21.15.151:0/3929893526 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f480410fcc0 0x7f48041a3350 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f47f4000c00 tx=0x7f47f4011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:43:20.528 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:19.496+0000 7f47fb400700 1 -- 172.21.15.151:0/3929893526 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f47f4003aa0 con 0x7f480410fcc0 2024-05-21T21:43:20.528 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:19.496+0000 7f47fb400700 1 -- 172.21.15.151:0/3929893526 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47f401b8f0 con 0x7f480410fcc0 2024-05-21T21:43:20.528 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:19.496+0000 7f47fb400700 1 -- 172.21.15.151:0/3929893526 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f47f4023870 con 0x7f480410fcc0 2024-05-21T21:43:20.528 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:19.496+0000 7f4803e00700 1 -- 172.21.15.151:0/3929893526 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48041a3b20 con 0x7f480410fcc0 2024-05-21T21:43:20.528 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:19.496+0000 7f4803e00700 1 -- 172.21.15.151:0/3929893526 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48041a6810 con 0x7f480410fcc0 2024-05-21T21:43:20.528 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:19.497+0000 7f4803e00700 1 -- 172.21.15.151:0/3929893526 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f480406a6a0 con 0x7f480410fcc0 2024-05-21T21:43:20.529 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:19.497+0000 7f47fb400700 1 -- 172.21.15.151:0/3929893526 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7f47f401b440 con 0x7f480410fcc0 2024-05-21T21:43:20.529 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:19.498+0000 7f47fb400700 1 --2- 172.21.15.151:0/3929893526 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f47e803e5c0 0x7f47e8040a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:43:20.529 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:19.498+0000 7f47fb400700 1 -- 172.21.15.151:0/3929893526 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f47f4053f80 con 0x7f480410fcc0 2024-05-21T21:43:20.529 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:19.498+0000 7f4802000700 1 --2- 172.21.15.151:0/3929893526 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f47e803e5c0 0x7f47e8040a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:43:20.529 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:19.499+0000 7f4802000700 1 --2- 172.21.15.151:0/3929893526 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f47e803e5c0 0x7f47e8040a80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f47f0006fd0 tx=0x7f47f0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:43:20.529 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:19.503+0000 7f47fb400700 1 -- 172.21.15.151:0/3929893526 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f47f401f030 con 0x7f480410fcc0 2024-05-21T21:43:20.529 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:19.857+0000 7f4803e00700 1 -- 172.21.15.151:0/3929893526 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f48041a69e0 con 0x7f480410fcc0 2024-05-21T21:43:20.529 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:19.861+0000 7f47fb400700 1 -- 172.21.15.151:0/3929893526 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) ==== 153+0+0 (secure 0 0 0) 0x7f48041a69e0 con 0x7f480410fcc0 2024-05-21T21:43:20.529 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:19.865+0000 7f4803e00700 1 -- 172.21.15.151:0/3929893526 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f47e803e5c0 msgr2=0x7f47e8040a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:43:20.529 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:19.865+0000 7f4803e00700 1 --2- 172.21.15.151:0/3929893526 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f47e803e5c0 0x7f47e8040a80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f47f0006fd0 tx=0x7f47f0006e40 comp rx=0 tx=0).stop 2024-05-21T21:43:20.529 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:19.865+0000 7f4803e00700 1 -- 172.21.15.151:0/3929893526 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f480410fcc0 msgr2=0x7f48041a3350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:43:20.529 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:19.865+0000 7f4803e00700 1 --2- 172.21.15.151:0/3929893526 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f480410fcc0 0x7f48041a3350 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f47f4000c00 tx=0x7f47f4011aa0 comp rx=0 tx=0).stop 2024-05-21T21:43:20.529 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:19.865+0000 7f4803e00700 1 -- 172.21.15.151:0/3929893526 shutdown_connections 2024-05-21T21:43:20.529 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:19.865+0000 7f4803e00700 1 --2- 172.21.15.151:0/3929893526 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f47e803e5c0 0x7f47e8040a80 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:43:20.529 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:19.865+0000 7f4803e00700 1 --2- 172.21.15.151:0/3929893526 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f480410fcc0 0x7f48041a3350 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:43:20.530 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:19.865+0000 7f4803e00700 1 -- 172.21.15.151:0/3929893526 >> 172.21.15.151:0/3929893526 conn(0x7f480407e510 msgr2=0x7f480407efd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:43:20.530 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:19.865+0000 7f4803e00700 1 -- 172.21.15.151:0/3929893526 shutdown_connections 2024-05-21T21:43:20.530 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr 2024-05-21T21:43:19.865+0000 7f4803e00700 1 -- 172.21.15.151:0/3929893526 wait complete. 2024-05-21T21:43:20.530 INFO:teuthology.orchestra.run.smithi151.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-05-21T21:43:20.530 INFO:teuthology.orchestra.run.smithi151.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-05-21T21:43:20.530 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T21:43:20.530 INFO:teuthology.orchestra.run.smithi151.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-05-21T21:43:20.530 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T21:43:20.531 INFO:teuthology.orchestra.run.smithi151.stdout:Or, if you are only running a single cluster on this host: 2024-05-21T21:43:20.531 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T21:43:20.531 INFO:teuthology.orchestra.run.smithi151.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-05-21T21:43:20.531 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T21:43:20.531 INFO:teuthology.orchestra.run.smithi151.stdout:Please consider enabling telemetry to help improve Ceph: 2024-05-21T21:43:20.531 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T21:43:20.531 INFO:teuthology.orchestra.run.smithi151.stdout: ceph telemetry on 2024-05-21T21:43:20.531 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T21:43:20.531 INFO:teuthology.orchestra.run.smithi151.stdout:For more information see: 2024-05-21T21:43:20.531 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T21:43:20.531 INFO:teuthology.orchestra.run.smithi151.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-05-21T21:43:20.531 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T21:43:20.531 INFO:teuthology.orchestra.run.smithi151.stdout:Bootstrap complete. 2024-05-21T21:43:20.639 INFO:tasks.cephadm:Fetching config... 2024-05-21T21:43:20.639 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-05-21T21:43:20.639 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-05-21T21:43:20.687 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-05-21T21:43:20.687 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-05-21T21:43:20.687 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-05-21T21:43:20.769 INFO:tasks.cephadm:Fetching mon keyring... 2024-05-21T21:43:20.769 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-05-21T21:43:20.769 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/keyring of=/dev/stdout 2024-05-21T21:43:20.882 INFO:tasks.cephadm:Fetching pub ssh key... 2024-05-21T21:43:20.882 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-05-21T21:43:20.882 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-05-21T21:43:20.976 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-05-21T21:43:20.977 DEBUG:teuthology.orchestra.run.smithi151:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMwzrK6w5hxzJfOlJIl/zsewwuE1g7rOVKruibSweE6Q7QM303wVSPIN4PTeIffgAykZNYK85HCx0HEZzXuEOVDeB7O+VqOCQximU49wY6poD0pIbX8FOxeQS6m4CNBXVnk7Fdb9WHWlEFdg2d0OmKPlW9MnonGiMoILHm3fa7QoOBlO8YhmQ4/66gKN/TbqV6WQLmqvEkDcznRQvC4Bepq2E2m3LBmElw3J6Jiq+/fdQM1GQhBa6DDKg20wU+T7bxEmEZAsuV2wi8W2odsmULlPPj189IgArrJBUZkz0/baIIJqktWcbaUZ34Z5d9PFBEs6rfvmlRSX3QpNQ6muBkFJXm/DrgfgJ6it/+JmK+Do1By7v6y98HYD4nrV98PkC+S7rfx1Xzj+vPil0OTOaagfXvqRIxP8BFY9lb7b/AZy1TbZ4sx685siMN802ZQf8NRH3r/m2yYkqI03hDpI7SB8QbjIu8knzVfT/4+XLZWOaVlJEyY+2nll/QSYiJAl8= ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-21T21:43:21.078 INFO:teuthology.orchestra.run.smithi151.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMwzrK6w5hxzJfOlJIl/zsewwuE1g7rOVKruibSweE6Q7QM303wVSPIN4PTeIffgAykZNYK85HCx0HEZzXuEOVDeB7O+VqOCQximU49wY6poD0pIbX8FOxeQS6m4CNBXVnk7Fdb9WHWlEFdg2d0OmKPlW9MnonGiMoILHm3fa7QoOBlO8YhmQ4/66gKN/TbqV6WQLmqvEkDcznRQvC4Bepq2E2m3LBmElw3J6Jiq+/fdQM1GQhBa6DDKg20wU+T7bxEmEZAsuV2wi8W2odsmULlPPj189IgArrJBUZkz0/baIIJqktWcbaUZ34Z5d9PFBEs6rfvmlRSX3QpNQ6muBkFJXm/DrgfgJ6it/+JmK+Do1By7v6y98HYD4nrV98PkC+S7rfx1Xzj+vPil0OTOaagfXvqRIxP8BFY9lb7b/AZy1TbZ4sx685siMN802ZQf8NRH3r/m2yYkqI03hDpI7SB8QbjIu8knzVfT/4+XLZWOaVlJEyY+2nll/QSYiJAl8= ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968 2024-05-21T21:43:21.110 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:20 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/3929893526' entity='client.admin' 2024-05-21T21:43:21.126 DEBUG:teuthology.orchestra.run.smithi164:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMwzrK6w5hxzJfOlJIl/zsewwuE1g7rOVKruibSweE6Q7QM303wVSPIN4PTeIffgAykZNYK85HCx0HEZzXuEOVDeB7O+VqOCQximU49wY6poD0pIbX8FOxeQS6m4CNBXVnk7Fdb9WHWlEFdg2d0OmKPlW9MnonGiMoILHm3fa7QoOBlO8YhmQ4/66gKN/TbqV6WQLmqvEkDcznRQvC4Bepq2E2m3LBmElw3J6Jiq+/fdQM1GQhBa6DDKg20wU+T7bxEmEZAsuV2wi8W2odsmULlPPj189IgArrJBUZkz0/baIIJqktWcbaUZ34Z5d9PFBEs6rfvmlRSX3QpNQ6muBkFJXm/DrgfgJ6it/+JmK+Do1By7v6y98HYD4nrV98PkC+S7rfx1Xzj+vPil0OTOaagfXvqRIxP8BFY9lb7b/AZy1TbZ4sx685siMN802ZQf8NRH3r/m2yYkqI03hDpI7SB8QbjIu8knzVfT/4+XLZWOaVlJEyY+2nll/QSYiJAl8= ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-21T21:43:21.168 INFO:teuthology.orchestra.run.smithi164.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMwzrK6w5hxzJfOlJIl/zsewwuE1g7rOVKruibSweE6Q7QM303wVSPIN4PTeIffgAykZNYK85HCx0HEZzXuEOVDeB7O+VqOCQximU49wY6poD0pIbX8FOxeQS6m4CNBXVnk7Fdb9WHWlEFdg2d0OmKPlW9MnonGiMoILHm3fa7QoOBlO8YhmQ4/66gKN/TbqV6WQLmqvEkDcznRQvC4Bepq2E2m3LBmElw3J6Jiq+/fdQM1GQhBa6DDKg20wU+T7bxEmEZAsuV2wi8W2odsmULlPPj189IgArrJBUZkz0/baIIJqktWcbaUZ34Z5d9PFBEs6rfvmlRSX3QpNQ6muBkFJXm/DrgfgJ6it/+JmK+Do1By7v6y98HYD4nrV98PkC+S7rfx1Xzj+vPil0OTOaagfXvqRIxP8BFY9lb7b/AZy1TbZ4sx685siMN802ZQf8NRH3r/m2yYkqI03hDpI7SB8QbjIu8knzVfT/4+XLZWOaVlJEyY+2nll/QSYiJAl8= ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968 2024-05-21T21:43:21.180 DEBUG:teuthology.orchestra.run.smithi184:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMwzrK6w5hxzJfOlJIl/zsewwuE1g7rOVKruibSweE6Q7QM303wVSPIN4PTeIffgAykZNYK85HCx0HEZzXuEOVDeB7O+VqOCQximU49wY6poD0pIbX8FOxeQS6m4CNBXVnk7Fdb9WHWlEFdg2d0OmKPlW9MnonGiMoILHm3fa7QoOBlO8YhmQ4/66gKN/TbqV6WQLmqvEkDcznRQvC4Bepq2E2m3LBmElw3J6Jiq+/fdQM1GQhBa6DDKg20wU+T7bxEmEZAsuV2wi8W2odsmULlPPj189IgArrJBUZkz0/baIIJqktWcbaUZ34Z5d9PFBEs6rfvmlRSX3QpNQ6muBkFJXm/DrgfgJ6it/+JmK+Do1By7v6y98HYD4nrV98PkC+S7rfx1Xzj+vPil0OTOaagfXvqRIxP8BFY9lb7b/AZy1TbZ4sx685siMN802ZQf8NRH3r/m2yYkqI03hDpI7SB8QbjIu8knzVfT/4+XLZWOaVlJEyY+2nll/QSYiJAl8= ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-21T21:43:21.223 INFO:teuthology.orchestra.run.smithi184.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMwzrK6w5hxzJfOlJIl/zsewwuE1g7rOVKruibSweE6Q7QM303wVSPIN4PTeIffgAykZNYK85HCx0HEZzXuEOVDeB7O+VqOCQximU49wY6poD0pIbX8FOxeQS6m4CNBXVnk7Fdb9WHWlEFdg2d0OmKPlW9MnonGiMoILHm3fa7QoOBlO8YhmQ4/66gKN/TbqV6WQLmqvEkDcznRQvC4Bepq2E2m3LBmElw3J6Jiq+/fdQM1GQhBa6DDKg20wU+T7bxEmEZAsuV2wi8W2odsmULlPPj189IgArrJBUZkz0/baIIJqktWcbaUZ34Z5d9PFBEs6rfvmlRSX3QpNQ6muBkFJXm/DrgfgJ6it/+JmK+Do1By7v6y98HYD4nrV98PkC+S7rfx1Xzj+vPil0OTOaagfXvqRIxP8BFY9lb7b/AZy1TbZ4sx685siMN802ZQf8NRH3r/m2yYkqI03hDpI7SB8QbjIu8knzVfT/4+XLZWOaVlJEyY+2nll/QSYiJAl8= ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968 2024-05-21T21:43:21.235 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-05-21T21:43:21.659 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:43:24.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:24.261+0000 7f8d59000700 1 -- 172.21.15.151:0/2421341841 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8d54101e90 msgr2=0x7f8d541022b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:43:24.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:24.261+0000 7f8d59000700 1 --2- 172.21.15.151:0/2421341841 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8d54101e90 0x7f8d541022b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8d44009910 tx=0x7f8d44009c20 comp rx=0 tx=0).stop 2024-05-21T21:43:24.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:24.261+0000 7f8d59000700 1 -- 172.21.15.151:0/2421341841 shutdown_connections 2024-05-21T21:43:24.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:24.261+0000 7f8d59000700 1 --2- 172.21.15.151:0/2421341841 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8d54101e90 0x7f8d541022b0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:43:24.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:24.261+0000 7f8d59000700 1 -- 172.21.15.151:0/2421341841 >> 172.21.15.151:0/2421341841 conn(0x7f8d540fd450 msgr2=0x7f8d540ff870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:43:24.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:24.262+0000 7f8d59000700 1 -- 172.21.15.151:0/2421341841 shutdown_connections 2024-05-21T21:43:24.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:24.262+0000 7f8d59000700 1 -- 172.21.15.151:0/2421341841 wait complete. 2024-05-21T21:43:24.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:24.263+0000 7f8d59000700 1 Processor -- start 2024-05-21T21:43:24.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:24.264+0000 7f8d59000700 1 -- start start 2024-05-21T21:43:24.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:24.264+0000 7f8d59000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8d54101e90 0x7f8d5406bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:43:24.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:24.264+0000 7f8d59000700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8d54107d50 con 0x7f8d54101e90 2024-05-21T21:43:24.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:24.264+0000 7f8d51e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8d54101e90 0x7f8d5406bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:43:24.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:24.265+0000 7f8d51e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8d54101e90 0x7f8d5406bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:36054/0 (socket says 172.21.15.151:36054) 2024-05-21T21:43:24.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:24.265+0000 7f8d51e00700 1 -- 172.21.15.151:0/2914278544 learned_addr learned my addr 172.21.15.151:0/2914278544 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:43:24.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:24.265+0000 7f8d51e00700 1 -- 172.21.15.151:0/2914278544 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d5419b280 con 0x7f8d54101e90 2024-05-21T21:43:24.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:24.265+0000 7f8d51e00700 1 --2- 172.21.15.151:0/2914278544 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8d54101e90 0x7f8d5406bce0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8d44005190 tx=0x7f8d44004f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:43:24.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:24.266+0000 7f8d4aa00700 1 -- 172.21.15.151:0/2914278544 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8d44027070 con 0x7f8d54101e90 2024-05-21T21:43:24.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:24.266+0000 7f8d4aa00700 1 -- 172.21.15.151:0/2914278544 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d44022470 con 0x7f8d54101e90 2024-05-21T21:43:24.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:24.266+0000 7f8d59000700 1 -- 172.21.15.151:0/2914278544 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d5419b450 con 0x7f8d54101e90 2024-05-21T21:43:24.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:24.266+0000 7f8d4aa00700 1 -- 172.21.15.151:0/2914278544 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8d4400f5f0 con 0x7f8d54101e90 2024-05-21T21:43:24.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:24.266+0000 7f8d59000700 1 -- 172.21.15.151:0/2914278544 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d5419b870 con 0x7f8d54101e90 2024-05-21T21:43:24.269 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:24.267+0000 7f8d59000700 1 -- 172.21.15.151:0/2914278544 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d541084d0 con 0x7f8d54101e90 2024-05-21T21:43:24.269 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:24.267+0000 7f8d4aa00700 1 -- 172.21.15.151:0/2914278544 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7f8d4402a070 con 0x7f8d54101e90 2024-05-21T21:43:24.269 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:24.268+0000 7f8d4aa00700 1 --2- 172.21.15.151:0/2914278544 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f8d3803e510 0x7f8d380409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:43:24.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:24.268+0000 7f8d4aa00700 1 -- 172.21.15.151:0/2914278544 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f8d4401e030 con 0x7f8d54101e90 2024-05-21T21:43:24.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:24.268+0000 7f8d51400700 1 --2- 172.21.15.151:0/2914278544 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f8d3803e510 0x7f8d380409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:43:24.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:24.269+0000 7f8d51400700 1 --2- 172.21.15.151:0/2914278544 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f8d3803e510 0x7f8d380409d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8d40006fd0 tx=0x7f8d40006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:43:24.275 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:24.273+0000 7f8d4aa00700 1 -- 172.21.15.151:0/2914278544 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8d44013070 con 0x7f8d54101e90 2024-05-21T21:43:24.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:24.502+0000 7f8d59000700 1 -- 172.21.15.151:0/2914278544 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f8d5402e720 con 0x7f8d54101e90 2024-05-21T21:43:24.507 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:24.506+0000 7f8d4aa00700 1 -- 172.21.15.151:0/2914278544 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f8d440310c0 con 0x7f8d54101e90 2024-05-21T21:43:24.510 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:24.509+0000 7f8d59000700 1 -- 172.21.15.151:0/2914278544 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f8d3803e510 msgr2=0x7f8d380409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:43:24.510 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:24.509+0000 7f8d59000700 1 --2- 172.21.15.151:0/2914278544 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f8d3803e510 0x7f8d380409d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8d40006fd0 tx=0x7f8d40006e40 comp rx=0 tx=0).stop 2024-05-21T21:43:24.510 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:24.509+0000 7f8d59000700 1 -- 172.21.15.151:0/2914278544 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8d54101e90 msgr2=0x7f8d5406bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:43:24.510 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:24.509+0000 7f8d59000700 1 --2- 172.21.15.151:0/2914278544 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8d54101e90 0x7f8d5406bce0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8d44005190 tx=0x7f8d44004f80 comp rx=0 tx=0).stop 2024-05-21T21:43:24.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:24.510+0000 7f8d59000700 1 -- 172.21.15.151:0/2914278544 shutdown_connections 2024-05-21T21:43:24.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:24.510+0000 7f8d59000700 1 --2- 172.21.15.151:0/2914278544 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f8d3803e510 0x7f8d380409d0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:43:24.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:24.510+0000 7f8d59000700 1 --2- 172.21.15.151:0/2914278544 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8d54101e90 0x7f8d5406bce0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:43:24.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:24.510+0000 7f8d59000700 1 -- 172.21.15.151:0/2914278544 >> 172.21.15.151:0/2914278544 conn(0x7f8d540fd450 msgr2=0x7f8d540fded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:43:24.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:24.510+0000 7f8d59000700 1 -- 172.21.15.151:0/2914278544 shutdown_connections 2024-05-21T21:43:24.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:24.510+0000 7f8d59000700 1 -- 172.21.15.151:0/2914278544 wait complete. 2024-05-21T21:43:24.918 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:24 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:43:24.919 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:24 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:43:24.919 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:24 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi151", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:43:24.919 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:24 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:43:24.919 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:24 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:43:24.919 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:24 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:43:24.919 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:24 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/2914278544' entity='client.admin' 2024-05-21T21:43:24.919 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:24 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:43:24.919 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:24 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:43:25.324 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-05-21T21:43:25.324 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-05-21T21:43:25.762 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:43:27.622 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:27.621+0000 7fe488200700 1 -- 172.21.15.151:0/216386877 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe4801051c0 msgr2=0x7fe4801055e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:43:27.623 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:27.621+0000 7fe488200700 1 --2- 172.21.15.151:0/216386877 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe4801051c0 0x7fe4801055e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe474009910 tx=0x7fe474009c20 comp rx=0 tx=0).stop 2024-05-21T21:43:27.623 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:27.621+0000 7fe488200700 1 -- 172.21.15.151:0/216386877 shutdown_connections 2024-05-21T21:43:27.623 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:27.621+0000 7fe488200700 1 --2- 172.21.15.151:0/216386877 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe4801051c0 0x7fe4801055e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:43:27.623 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:27.621+0000 7fe488200700 1 -- 172.21.15.151:0/216386877 >> 172.21.15.151:0/216386877 conn(0x7fe480100760 msgr2=0x7fe480102ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:43:27.623 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:27.621+0000 7fe488200700 1 -- 172.21.15.151:0/216386877 shutdown_connections 2024-05-21T21:43:27.623 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:27.621+0000 7fe488200700 1 -- 172.21.15.151:0/216386877 wait complete. 2024-05-21T21:43:27.624 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:27.622+0000 7fe488200700 1 Processor -- start 2024-05-21T21:43:27.624 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:27.622+0000 7fe488200700 1 -- start start 2024-05-21T21:43:27.624 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:27.622+0000 7fe488200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe4801051c0 0x7fe48019a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:43:27.624 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:27.622+0000 7fe488200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe48010b100 con 0x7fe4801051c0 2024-05-21T21:43:27.624 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:27.623+0000 7fe485800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe4801051c0 0x7fe48019a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:43:27.624 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:27.623+0000 7fe485800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe4801051c0 0x7fe48019a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:35740/0 (socket says 172.21.15.151:35740) 2024-05-21T21:43:27.624 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:27.623+0000 7fe485800700 1 -- 172.21.15.151:0/3013244656 learned_addr learned my addr 172.21.15.151:0/3013244656 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:43:27.624 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:27.623+0000 7fe485800700 1 -- 172.21.15.151:0/3013244656 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe48019ad30 con 0x7fe4801051c0 2024-05-21T21:43:27.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:27.623+0000 7fe485800700 1 --2- 172.21.15.151:0/3013244656 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe4801051c0 0x7fe48019a7f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe474005950 tx=0x7fe474004fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:43:27.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:27.624+0000 7fe47e000700 1 -- 172.21.15.151:0/3013244656 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe474026070 con 0x7fe4801051c0 2024-05-21T21:43:27.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:27.624+0000 7fe47e000700 1 -- 172.21.15.151:0/3013244656 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe47400bc00 con 0x7fe4801051c0 2024-05-21T21:43:27.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:27.624+0000 7fe488200700 1 -- 172.21.15.151:0/3013244656 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe48019afc0 con 0x7fe4801051c0 2024-05-21T21:43:27.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:27.624+0000 7fe47e000700 1 -- 172.21.15.151:0/3013244656 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe474003a10 con 0x7fe4801051c0 2024-05-21T21:43:27.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:27.624+0000 7fe488200700 1 -- 172.21.15.151:0/3013244656 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe48019dcb0 con 0x7fe4801051c0 2024-05-21T21:43:27.626 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:27.625+0000 7fe488200700 1 -- 172.21.15.151:0/3013244656 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe48010b880 con 0x7fe4801051c0 2024-05-21T21:43:27.626 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:27.625+0000 7fe47e000700 1 -- 172.21.15.151:0/3013244656 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7fe474029070 con 0x7fe4801051c0 2024-05-21T21:43:27.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:27.626+0000 7fe47e000700 1 --2- 172.21.15.151:0/3013244656 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fe46803e570 0x7fe468040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:43:27.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:27.626+0000 7fe47e000700 1 -- 172.21.15.151:0/3013244656 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe47401e070 con 0x7fe4801051c0 2024-05-21T21:43:27.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:27.626+0000 7fe484e00700 1 --2- 172.21.15.151:0/3013244656 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fe46803e570 0x7fe468040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:43:27.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:27.627+0000 7fe484e00700 1 --2- 172.21.15.151:0/3013244656 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fe46803e570 0x7fe468040a30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe46c006fd0 tx=0x7fe46c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:43:27.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:27.631+0000 7fe47e000700 1 -- 172.21.15.151:0/3013244656 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe474013070 con 0x7fe4801051c0 2024-05-21T21:43:27.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:27.866+0000 7fe488200700 1 -- 172.21.15.151:0/3013244656 --> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7fe48002e670 con 0x7fe46803e570 2024-05-21T21:43:27.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:27.874+0000 7fe47e000700 1 -- 172.21.15.151:0/3013244656 <== mgr.14152 v2:172.21.15.151:6800/2796622483 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe48002e670 con 0x7fe46803e570 2024-05-21T21:43:27.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:27.877+0000 7fe488200700 1 -- 172.21.15.151:0/3013244656 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fe46803e570 msgr2=0x7fe468040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:43:27.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:27.877+0000 7fe488200700 1 --2- 172.21.15.151:0/3013244656 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fe46803e570 0x7fe468040a30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe46c006fd0 tx=0x7fe46c006e40 comp rx=0 tx=0).stop 2024-05-21T21:43:27.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:27.877+0000 7fe488200700 1 -- 172.21.15.151:0/3013244656 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe4801051c0 msgr2=0x7fe48019a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:43:27.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:27.877+0000 7fe488200700 1 --2- 172.21.15.151:0/3013244656 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe4801051c0 0x7fe48019a7f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe474005950 tx=0x7fe474004fa0 comp rx=0 tx=0).stop 2024-05-21T21:43:27.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:27.877+0000 7fe488200700 1 -- 172.21.15.151:0/3013244656 shutdown_connections 2024-05-21T21:43:27.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:27.877+0000 7fe488200700 1 --2- 172.21.15.151:0/3013244656 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fe46803e570 0x7fe468040a30 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:43:27.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:27.877+0000 7fe488200700 1 --2- 172.21.15.151:0/3013244656 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe4801051c0 0x7fe48019a7f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:43:27.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:27.877+0000 7fe488200700 1 -- 172.21.15.151:0/3013244656 >> 172.21.15.151:0/3013244656 conn(0x7fe480100760 msgr2=0x7fe480101230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:43:27.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:27.877+0000 7fe488200700 1 -- 172.21.15.151:0/3013244656 shutdown_connections 2024-05-21T21:43:27.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:27.877+0000 7fe488200700 1 -- 172.21.15.151:0/3013244656 wait complete. 2024-05-21T21:43:28.660 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi164 2024-05-21T21:43:28.660 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-21T21:43:28.660 DEBUG:teuthology.orchestra.run.smithi164:> dd of=/etc/ceph/ceph.conf 2024-05-21T21:43:28.679 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-21T21:43:28.679 DEBUG:teuthology.orchestra.run.smithi164:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-21T21:43:28.736 INFO:tasks.cephadm:Adding host smithi164 to orchestrator... 2024-05-21T21:43:28.736 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph orch host add smithi164 2024-05-21T21:43:29.138 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:43:29.202 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:28 smithi151 ceph-mon[31623]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:43:29.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:28 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:43:29.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:28 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:43:29.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:28 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:43:29.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:28 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:43:29.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:28 smithi151 ceph-mon[31623]: Updating smithi151:/etc/ceph/ceph.conf 2024-05-21T21:43:29.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:28 smithi151 ceph-mon[31623]: Updating smithi151:/var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T21:43:30.190 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:29 smithi151 ceph-mon[31623]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:43:30.190 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:29 smithi151 ceph-mon[31623]: Updating smithi151:/etc/ceph/ceph.client.admin.keyring 2024-05-21T21:43:30.190 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:29 smithi151 ceph-mon[31623]: Updating smithi151:/var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-21T21:43:31.283 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:31.282+0000 7f32dae00700 1 -- 172.21.15.151:0/2486189789 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f32d410e530 msgr2=0x7f32d410e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:43:31.283 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:31.282+0000 7f32dae00700 1 --2- 172.21.15.151:0/2486189789 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f32d410e530 0x7f32d410e910 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f32c4009910 tx=0x7f32c4009c20 comp rx=0 tx=0).stop 2024-05-21T21:43:31.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:31.283+0000 7f32dae00700 1 -- 172.21.15.151:0/2486189789 shutdown_connections 2024-05-21T21:43:31.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:31.283+0000 7f32dae00700 1 --2- 172.21.15.151:0/2486189789 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f32d410e530 0x7f32d410e910 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:43:31.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:31.283+0000 7f32dae00700 1 -- 172.21.15.151:0/2486189789 >> 172.21.15.151:0/2486189789 conn(0x7f32d4078da0 msgr2=0x7f32d407b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:43:31.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:31.283+0000 7f32dae00700 1 -- 172.21.15.151:0/2486189789 shutdown_connections 2024-05-21T21:43:31.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:31.283+0000 7f32dae00700 1 -- 172.21.15.151:0/2486189789 wait complete. 2024-05-21T21:43:31.285 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:31.284+0000 7f32dae00700 1 Processor -- start 2024-05-21T21:43:31.285 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:31.284+0000 7f32dae00700 1 -- start start 2024-05-21T21:43:31.285 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:31.284+0000 7f32dae00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f32d410e530 0x7f32d419efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:43:31.285 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:31.284+0000 7f32dae00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f32d4113a60 con 0x7f32d410e530 2024-05-21T21:43:31.285 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:31.285+0000 7f32d3e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f32d410e530 0x7f32d419efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:43:31.285 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:31.285+0000 7f32d3e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f32d410e530 0x7f32d419efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:35764/0 (socket says 172.21.15.151:35764) 2024-05-21T21:43:31.285 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:31.285+0000 7f32d3e00700 1 -- 172.21.15.151:0/571465643 learned_addr learned my addr 172.21.15.151:0/571465643 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:43:31.286 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:31.285+0000 7f32d3e00700 1 -- 172.21.15.151:0/571465643 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32d419f4f0 con 0x7f32d410e530 2024-05-21T21:43:31.286 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:31.285+0000 7f32d3e00700 1 --2- 172.21.15.151:0/571465643 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f32d410e530 0x7f32d419efb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f32c40092e0 tx=0x7f32c4005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:43:31.286 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:31.286+0000 7f32d0c00700 1 -- 172.21.15.151:0/571465643 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f32c4026070 con 0x7f32d410e530 2024-05-21T21:43:31.286 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:31.286+0000 7f32d0c00700 1 -- 172.21.15.151:0/571465643 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32c400b6f0 con 0x7f32d410e530 2024-05-21T21:43:31.286 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:31.286+0000 7f32d0c00700 1 -- 172.21.15.151:0/571465643 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f32c4003880 con 0x7f32d410e530 2024-05-21T21:43:31.286 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:31.286+0000 7f32dae00700 1 -- 172.21.15.151:0/571465643 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32d419f7e0 con 0x7f32d410e530 2024-05-21T21:43:31.286 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:31.286+0000 7f32dae00700 1 -- 172.21.15.151:0/571465643 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32d41a3880 con 0x7f32d410e530 2024-05-21T21:43:31.287 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:31.287+0000 7f32dae00700 1 -- 172.21.15.151:0/571465643 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32d410ac70 con 0x7f32d410e530 2024-05-21T21:43:31.287 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:31.287+0000 7f32d0c00700 1 -- 172.21.15.151:0/571465643 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7f32c4029070 con 0x7f32d410e530 2024-05-21T21:43:31.287 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:31.288+0000 7f32d0c00700 1 --2- 172.21.15.151:0/571465643 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f32b803e570 0x7f32b8040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:43:31.287 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:31.288+0000 7f32d0c00700 1 -- 172.21.15.151:0/571465643 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f32c401e070 con 0x7f32d410e530 2024-05-21T21:43:31.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:31.288+0000 7f32d3400700 1 --2- 172.21.15.151:0/571465643 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f32b803e570 0x7f32b8040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:43:31.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:31.289+0000 7f32d3400700 1 --2- 172.21.15.151:0/571465643 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f32b803e570 0x7f32b8040a30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f32c0006fd0 tx=0x7f32c0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:43:31.293 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:31.294+0000 7f32d0c00700 1 -- 172.21.15.151:0/571465643 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f32c4013070 con 0x7f32d410e530 2024-05-21T21:43:31.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:31.528+0000 7f32dae00700 1 -- 172.21.15.151:0/571465643 --> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi164", "target": ["mon-mgr", ""]}) -- 0x7f32d41a0310 con 0x7f32b803e570 2024-05-21T21:43:31.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:31 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:43:31.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:31 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:43:31.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:31 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:43:32.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:32 smithi151 ceph-mon[31623]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:43:32.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:32 smithi151 ceph-mon[31623]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi164", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:43:33.952 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:33 smithi151 ceph-mon[31623]: Deploying cephadm binary to smithi164 2024-05-21T21:43:34.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:34 smithi151 ceph-mon[31623]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:43:36.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:36.299+0000 7f32d0c00700 1 -- 172.21.15.151:0/571465643 <== mgr.14152 v2:172.21.15.151:6800/2796622483 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f32d41a0310 con 0x7f32b803e570 2024-05-21T21:43:36.301 INFO:teuthology.orchestra.run.smithi151.stdout:Added host 'smithi164' with addr '172.21.15.164' 2024-05-21T21:43:36.303 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:36.303+0000 7f32dae00700 1 -- 172.21.15.151:0/571465643 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f32b803e570 msgr2=0x7f32b8040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:43:36.303 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:36.303+0000 7f32dae00700 1 --2- 172.21.15.151:0/571465643 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f32b803e570 0x7f32b8040a30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f32c0006fd0 tx=0x7f32c0006e40 comp rx=0 tx=0).stop 2024-05-21T21:43:36.303 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:36.303+0000 7f32dae00700 1 -- 172.21.15.151:0/571465643 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f32d410e530 msgr2=0x7f32d419efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:43:36.303 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:36.303+0000 7f32dae00700 1 --2- 172.21.15.151:0/571465643 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f32d410e530 0x7f32d419efb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f32c40092e0 tx=0x7f32c4005780 comp rx=0 tx=0).stop 2024-05-21T21:43:36.304 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:36.303+0000 7f32dae00700 1 -- 172.21.15.151:0/571465643 shutdown_connections 2024-05-21T21:43:36.304 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:36.303+0000 7f32dae00700 1 --2- 172.21.15.151:0/571465643 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f32b803e570 0x7f32b8040a30 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:43:36.304 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:36.303+0000 7f32dae00700 1 --2- 172.21.15.151:0/571465643 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f32d410e530 0x7f32d419efb0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:43:36.304 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:36.303+0000 7f32dae00700 1 -- 172.21.15.151:0/571465643 >> 172.21.15.151:0/571465643 conn(0x7f32d4078da0 msgr2=0x7f32d4111140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:43:36.304 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:36.303+0000 7f32dae00700 1 -- 172.21.15.151:0/571465643 shutdown_connections 2024-05-21T21:43:36.304 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:36.303+0000 7f32dae00700 1 -- 172.21.15.151:0/571465643 wait complete. 2024-05-21T21:43:36.620 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:36 smithi151 ceph-mon[31623]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:43:36.620 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:36 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:43:36.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:36 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:43:36.990 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph orch host ls --format=json 2024-05-21T21:43:37.436 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:43:37.861 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:37 smithi151 ceph-mon[31623]: Added host smithi164 2024-05-21T21:43:37.861 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:37 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:43:37.861 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:37 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:43:39.183 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:38 smithi151 ceph-mon[31623]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:43:39.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:39.352+0000 7f3aab600700 1 -- 172.21.15.151:0/205839920 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3aa41040e0 msgr2=0x7f3aa41044c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:43:39.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:39.352+0000 7f3aab600700 1 --2- 172.21.15.151:0/205839920 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3aa41040e0 0x7f3aa41044c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3a9400b0a0 tx=0x7f3a9400b3b0 comp rx=0 tx=0).stop 2024-05-21T21:43:39.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:39.353+0000 7f3aab600700 1 -- 172.21.15.151:0/205839920 shutdown_connections 2024-05-21T21:43:39.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:39.353+0000 7f3aab600700 1 --2- 172.21.15.151:0/205839920 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3aa41040e0 0x7f3aa41044c0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:43:39.354 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:39.353+0000 7f3aab600700 1 -- 172.21.15.151:0/205839920 >> 172.21.15.151:0/205839920 conn(0x7f3aa4078b10 msgr2=0x7f3aa40ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:43:39.354 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:39.353+0000 7f3aab600700 1 -- 172.21.15.151:0/205839920 shutdown_connections 2024-05-21T21:43:39.354 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:39.353+0000 7f3aab600700 1 -- 172.21.15.151:0/205839920 wait complete. 2024-05-21T21:43:39.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:39.354+0000 7f3aab600700 1 Processor -- start 2024-05-21T21:43:39.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:39.354+0000 7f3aab600700 1 -- start start 2024-05-21T21:43:39.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:39.354+0000 7f3aab600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3aa41040e0 0x7f3aa419e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:43:39.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:39.354+0000 7f3aab600700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa410d480 con 0x7f3aa41040e0 2024-05-21T21:43:39.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:39.355+0000 7f3aa8c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3aa41040e0 0x7f3aa419e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:43:39.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:39.355+0000 7f3aa8c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3aa41040e0 0x7f3aa419e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:40882/0 (socket says 172.21.15.151:40882) 2024-05-21T21:43:39.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:39.355+0000 7f3aa8c00700 1 -- 172.21.15.151:0/2682917205 learned_addr learned my addr 172.21.15.151:0/2682917205 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:43:39.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:39.355+0000 7f3aa8c00700 1 -- 172.21.15.151:0/2682917205 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3aa419ebe0 con 0x7f3aa41040e0 2024-05-21T21:43:39.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:39.355+0000 7f3aa8c00700 1 --2- 172.21.15.151:0/2682917205 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3aa41040e0 0x7f3aa419e6a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3a9400d010 tx=0x7f3a94009790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:43:39.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:39.356+0000 7f3aa2000700 1 -- 172.21.15.151:0/2682917205 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a94021020 con 0x7f3aa41040e0 2024-05-21T21:43:39.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:39.356+0000 7f3aa2000700 1 -- 172.21.15.151:0/2682917205 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a94004000 con 0x7f3aa41040e0 2024-05-21T21:43:39.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:39.356+0000 7f3aa2000700 1 -- 172.21.15.151:0/2682917205 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a94017040 con 0x7f3aa41040e0 2024-05-21T21:43:39.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:39.356+0000 7f3aab600700 1 -- 172.21.15.151:0/2682917205 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3aa419ee70 con 0x7f3aa41040e0 2024-05-21T21:43:39.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:39.356+0000 7f3aab600700 1 -- 172.21.15.151:0/2682917205 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3aa4198770 con 0x7f3aa41040e0 2024-05-21T21:43:39.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:39.357+0000 7f3aa2000700 1 -- 172.21.15.151:0/2682917205 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7f3a94024020 con 0x7f3aa41040e0 2024-05-21T21:43:39.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:39.357+0000 7f3aab600700 1 -- 172.21.15.151:0/2682917205 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3aa406b250 con 0x7f3aa41040e0 2024-05-21T21:43:39.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:39.358+0000 7f3aa2000700 1 --2- 172.21.15.151:0/2682917205 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f3a8803e5b0 0x7f3a88040a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:43:39.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:39.358+0000 7f3aa2000700 1 -- 172.21.15.151:0/2682917205 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f3a9400f050 con 0x7f3aa41040e0 2024-05-21T21:43:39.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:39.358+0000 7f3a9be00700 1 --2- 172.21.15.151:0/2682917205 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f3a8803e5b0 0x7f3a88040a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:43:39.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:39.359+0000 7f3a9be00700 1 --2- 172.21.15.151:0/2682917205 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f3a8803e5b0 0x7f3a88040a70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c006fd0 tx=0x7f3a8c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:43:39.363 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:39.363+0000 7f3aa2000700 1 -- 172.21.15.151:0/2682917205 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3a9402b020 con 0x7f3aa41040e0 2024-05-21T21:43:39.613 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:39.612+0000 7f3aab600700 1 -- 172.21.15.151:0/2682917205 --> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3aa4199ca0 con 0x7f3a8803e5b0 2024-05-21T21:43:39.614 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:39.614+0000 7f3aa2000700 1 -- 172.21.15.151:0/2682917205 <== mgr.14152 v2:172.21.15.151:6800/2796622483 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+161 (secure 0 0 0) 0x7f3aa4199ca0 con 0x7f3a8803e5b0 2024-05-21T21:43:39.614 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T21:43:39.615 INFO:teuthology.orchestra.run.smithi151.stdout:[{"addr": "172.21.15.151", "hostname": "smithi151", "labels": [], "status": ""}, {"addr": "172.21.15.164", "hostname": "smithi164", "labels": [], "status": ""}] 2024-05-21T21:43:39.617 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:39.617+0000 7f3aab600700 1 -- 172.21.15.151:0/2682917205 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f3a8803e5b0 msgr2=0x7f3a88040a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:43:39.617 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:39.617+0000 7f3aab600700 1 --2- 172.21.15.151:0/2682917205 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f3a8803e5b0 0x7f3a88040a70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c006fd0 tx=0x7f3a8c006e40 comp rx=0 tx=0).stop 2024-05-21T21:43:39.618 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:39.617+0000 7f3aab600700 1 -- 172.21.15.151:0/2682917205 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3aa41040e0 msgr2=0x7f3aa419e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:43:39.618 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:39.617+0000 7f3aab600700 1 --2- 172.21.15.151:0/2682917205 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3aa41040e0 0x7f3aa419e6a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3a9400d010 tx=0x7f3a94009790 comp rx=0 tx=0).stop 2024-05-21T21:43:39.618 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:39.617+0000 7f3aab600700 1 -- 172.21.15.151:0/2682917205 shutdown_connections 2024-05-21T21:43:39.618 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:39.617+0000 7f3aab600700 1 --2- 172.21.15.151:0/2682917205 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f3a8803e5b0 0x7f3a88040a70 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:43:39.618 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:39.617+0000 7f3aab600700 1 --2- 172.21.15.151:0/2682917205 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3aa41040e0 0x7f3aa419e6a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:43:39.618 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:39.617+0000 7f3aab600700 1 -- 172.21.15.151:0/2682917205 >> 172.21.15.151:0/2682917205 conn(0x7f3aa4078b10 msgr2=0x7f3aa4104e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:43:39.618 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:39.617+0000 7f3aab600700 1 -- 172.21.15.151:0/2682917205 shutdown_connections 2024-05-21T21:43:39.618 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:39.618+0000 7f3aab600700 1 -- 172.21.15.151:0/2682917205 wait complete. 2024-05-21T21:43:40.311 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:40 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:43:40.312 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:40 smithi151 ceph-mon[31623]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:43:40.312 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:40 smithi151 ceph-mon[31623]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T21:43:40.360 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi184 2024-05-21T21:43:40.360 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-05-21T21:43:40.360 DEBUG:teuthology.orchestra.run.smithi184:> dd of=/etc/ceph/ceph.conf 2024-05-21T21:43:40.378 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-05-21T21:43:40.378 DEBUG:teuthology.orchestra.run.smithi184:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-21T21:43:40.436 INFO:tasks.cephadm:Adding host smithi184 to orchestrator... 2024-05-21T21:43:40.436 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph orch host add smithi184 2024-05-21T21:43:40.871 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:43:42.364 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:42 smithi151 ceph-mon[31623]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:43:42.858 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:42.858+0000 7f66a1400700 1 -- 172.21.15.151:0/806309947 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f669c1051b0 msgr2=0x7f669c105590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:43:42.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:42.858+0000 7f66a1400700 1 --2- 172.21.15.151:0/806309947 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f669c1051b0 0x7f669c105590 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f668c009910 tx=0x7f668c009c20 comp rx=0 tx=0).stop 2024-05-21T21:43:42.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:42.858+0000 7f66a1400700 1 -- 172.21.15.151:0/806309947 shutdown_connections 2024-05-21T21:43:42.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:42.858+0000 7f66a1400700 1 --2- 172.21.15.151:0/806309947 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f669c1051b0 0x7f669c105590 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:43:42.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:42.858+0000 7f66a1400700 1 -- 172.21.15.151:0/806309947 >> 172.21.15.151:0/806309947 conn(0x7f669c100a40 msgr2=0x7f669c102e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:43:42.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:42.858+0000 7f66a1400700 1 -- 172.21.15.151:0/806309947 shutdown_connections 2024-05-21T21:43:42.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:42.858+0000 7f66a1400700 1 -- 172.21.15.151:0/806309947 wait complete. 2024-05-21T21:43:42.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:42.859+0000 7f66a1400700 1 Processor -- start 2024-05-21T21:43:42.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:42.859+0000 7f66a1400700 1 -- start start 2024-05-21T21:43:42.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:42.859+0000 7f66a1400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f669c1051b0 0x7f669c1a0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:43:42.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:42.859+0000 7f66a1400700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f669c10f430 con 0x7f669c1051b0 2024-05-21T21:43:42.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:42.860+0000 7f669be00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f669c1051b0 0x7f669c1a0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:43:42.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:42.860+0000 7f669be00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f669c1051b0 0x7f669c1a0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:40902/0 (socket says 172.21.15.151:40902) 2024-05-21T21:43:42.861 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:42.860+0000 7f669be00700 1 -- 172.21.15.151:0/1628572646 learned_addr learned my addr 172.21.15.151:0/1628572646 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:43:42.861 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:42.860+0000 7f669be00700 1 -- 172.21.15.151:0/1628572646 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f669c1a0c40 con 0x7f669c1051b0 2024-05-21T21:43:42.861 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:42.860+0000 7f669be00700 1 --2- 172.21.15.151:0/1628572646 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f669c1051b0 0x7f669c1a0700 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f668c0056d0 tx=0x7f668c0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:43:42.861 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:42.861+0000 7f6698c00700 1 -- 172.21.15.151:0/1628572646 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f668c026070 con 0x7f669c1051b0 2024-05-21T21:43:42.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:42.861+0000 7f6698c00700 1 -- 172.21.15.151:0/1628572646 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f668c00bba0 con 0x7f669c1051b0 2024-05-21T21:43:42.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:42.861+0000 7f66a1400700 1 -- 172.21.15.151:0/1628572646 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f669c1a0f30 con 0x7f669c1051b0 2024-05-21T21:43:42.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:42.861+0000 7f6698c00700 1 -- 172.21.15.151:0/1628572646 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f668c003920 con 0x7f669c1051b0 2024-05-21T21:43:42.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:42.861+0000 7f66a1400700 1 -- 172.21.15.151:0/1628572646 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f669c19a8c0 con 0x7f669c1051b0 2024-05-21T21:43:42.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:42.862+0000 7f66a1400700 1 -- 172.21.15.151:0/1628572646 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f669c106690 con 0x7f669c1051b0 2024-05-21T21:43:42.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:42.862+0000 7f6698c00700 1 -- 172.21.15.151:0/1628572646 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7f668c029070 con 0x7f669c1051b0 2024-05-21T21:43:42.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:42.863+0000 7f6698c00700 1 --2- 172.21.15.151:0/1628572646 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f668003e570 0x7f6680040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:43:42.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:42.863+0000 7f6698c00700 1 -- 172.21.15.151:0/1628572646 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f668c01e070 con 0x7f669c1051b0 2024-05-21T21:43:42.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:42.863+0000 7f669b400700 1 --2- 172.21.15.151:0/1628572646 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f668003e570 0x7f6680040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:43:42.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:42.864+0000 7f669b400700 1 --2- 172.21.15.151:0/1628572646 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f668003e570 0x7f6680040a30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6688006fd0 tx=0x7f6688006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:43:42.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:42.868+0000 7f6698c00700 1 -- 172.21.15.151:0/1628572646 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f668c013070 con 0x7f669c1051b0 2024-05-21T21:43:43.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:43.109+0000 7f66a1400700 1 -- 172.21.15.151:0/1628572646 --> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi184", "target": ["mon-mgr", ""]}) -- 0x7f669c19bdf0 con 0x7f668003e570 2024-05-21T21:43:43.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:43 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:43:43.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:43 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:43:43.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:43 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:43:43.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:43 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:43:43.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:43 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi164", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:43:43.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:43 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:43:43.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:43 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:43:43.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:43 smithi151 ceph-mon[31623]: Updating smithi164:/etc/ceph/ceph.conf 2024-05-21T21:43:43.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:43 smithi151 ceph-mon[31623]: Updating smithi164:/var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T21:43:43.454 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:43 smithi151 ceph-mon[31623]: Updating smithi164:/etc/ceph/ceph.client.admin.keyring 2024-05-21T21:43:44.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:44 smithi151 ceph-mon[31623]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:43:44.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:44 smithi151 ceph-mon[31623]: Updating smithi164:/var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-21T21:43:44.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:44 smithi151 ceph-mon[31623]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi184", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:43:44.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:44 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:43:44.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:44 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:43:44.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:44 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:43:45.702 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:45 smithi151 ceph-mon[31623]: Deploying cephadm binary to smithi184 2024-05-21T21:43:46.702 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:46 smithi151 ceph-mon[31623]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:43:47.757 INFO:teuthology.orchestra.run.smithi151.stdout:Added host 'smithi184' with addr '172.21.15.184' 2024-05-21T21:43:47.757 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:47.755+0000 7f6698c00700 1 -- 172.21.15.151:0/1628572646 <== mgr.14152 v2:172.21.15.151:6800/2796622483 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f669c19bdf0 con 0x7f668003e570 2024-05-21T21:43:47.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:47.759+0000 7f66a1400700 1 -- 172.21.15.151:0/1628572646 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f668003e570 msgr2=0x7f6680040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:43:47.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:47.759+0000 7f66a1400700 1 --2- 172.21.15.151:0/1628572646 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f668003e570 0x7f6680040a30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6688006fd0 tx=0x7f6688006e40 comp rx=0 tx=0).stop 2024-05-21T21:43:47.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:47.759+0000 7f66a1400700 1 -- 172.21.15.151:0/1628572646 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f669c1051b0 msgr2=0x7f669c1a0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:43:47.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:47.759+0000 7f66a1400700 1 --2- 172.21.15.151:0/1628572646 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f669c1051b0 0x7f669c1a0700 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f668c0056d0 tx=0x7f668c0057b0 comp rx=0 tx=0).stop 2024-05-21T21:43:47.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:47.760+0000 7f66a1400700 1 -- 172.21.15.151:0/1628572646 shutdown_connections 2024-05-21T21:43:47.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:47.760+0000 7f66a1400700 1 --2- 172.21.15.151:0/1628572646 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f668003e570 0x7f6680040a30 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:43:47.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:47.760+0000 7f66a1400700 1 --2- 172.21.15.151:0/1628572646 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f669c1051b0 0x7f669c1a0700 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:43:47.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:47.760+0000 7f66a1400700 1 -- 172.21.15.151:0/1628572646 >> 172.21.15.151:0/1628572646 conn(0x7f669c100a40 msgr2=0x7f669c106b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:43:47.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:47.760+0000 7f66a1400700 1 -- 172.21.15.151:0/1628572646 shutdown_connections 2024-05-21T21:43:47.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:47.760+0000 7f66a1400700 1 -- 172.21.15.151:0/1628572646 wait complete. 2024-05-21T21:43:48.487 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph orch host ls --format=json 2024-05-21T21:43:48.932 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:43:49.081 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:48 smithi151 ceph-mon[31623]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:43:49.081 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:48 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:43:49.081 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:48 smithi151 ceph-mon[31623]: Added host smithi184 2024-05-21T21:43:49.081 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:48 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:43:49.081 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:48 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:43:50.149 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:49 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:43:50.149 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:49 smithi151 ceph-mon[31623]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:43:50.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:50.929+0000 7fe0c1c00700 1 -- 172.21.15.151:0/2348655476 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe0bc105390 msgr2=0x7fe0bc105770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:43:50.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:50.930+0000 7fe0c1c00700 1 --2- 172.21.15.151:0/2348655476 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe0bc105390 0x7fe0bc105770 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac009910 tx=0x7fe0ac009c20 comp rx=0 tx=0).stop 2024-05-21T21:43:50.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:50.930+0000 7fe0c1c00700 1 -- 172.21.15.151:0/2348655476 shutdown_connections 2024-05-21T21:43:50.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:50.930+0000 7fe0c1c00700 1 --2- 172.21.15.151:0/2348655476 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe0bc105390 0x7fe0bc105770 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:43:50.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:50.930+0000 7fe0c1c00700 1 -- 172.21.15.151:0/2348655476 >> 172.21.15.151:0/2348655476 conn(0x7fe0bc100c20 msgr2=0x7fe0bc103040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:43:50.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:50.930+0000 7fe0c1c00700 1 -- 172.21.15.151:0/2348655476 shutdown_connections 2024-05-21T21:43:50.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:50.930+0000 7fe0c1c00700 1 -- 172.21.15.151:0/2348655476 wait complete. 2024-05-21T21:43:50.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:50.931+0000 7fe0c1c00700 1 Processor -- start 2024-05-21T21:43:50.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:50.931+0000 7fe0c1c00700 1 -- start start 2024-05-21T21:43:50.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:50.931+0000 7fe0c1c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe0bc105390 0x7fe0bc1a0800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:43:50.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:50.931+0000 7fe0c1c00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe0bc10f610 con 0x7fe0bc105390 2024-05-21T21:43:50.932 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:50.932+0000 7fe0baa00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe0bc105390 0x7fe0bc1a0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:43:50.932 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:50.932+0000 7fe0baa00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe0bc105390 0x7fe0bc1a0800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:53788/0 (socket says 172.21.15.151:53788) 2024-05-21T21:43:50.932 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:50.932+0000 7fe0baa00700 1 -- 172.21.15.151:0/765471811 learned_addr learned my addr 172.21.15.151:0/765471811 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:43:50.932 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:50.932+0000 7fe0baa00700 1 -- 172.21.15.151:0/765471811 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0bc1a0d40 con 0x7fe0bc105390 2024-05-21T21:43:50.932 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:50.932+0000 7fe0baa00700 1 --2- 172.21.15.151:0/765471811 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe0bc105390 0x7fe0bc1a0800 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac006010 tx=0x7fe0ac005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:43:50.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:50.933+0000 7fe0b3400700 1 -- 172.21.15.151:0/765471811 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe0ac026070 con 0x7fe0bc105390 2024-05-21T21:43:50.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:50.933+0000 7fe0b3400700 1 -- 172.21.15.151:0/765471811 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0ac00bba0 con 0x7fe0bc105390 2024-05-21T21:43:50.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:50.933+0000 7fe0c1c00700 1 -- 172.21.15.151:0/765471811 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0bc1a1030 con 0x7fe0bc105390 2024-05-21T21:43:50.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:50.933+0000 7fe0c1c00700 1 -- 172.21.15.151:0/765471811 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0bc19a9c0 con 0x7fe0bc105390 2024-05-21T21:43:50.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:50.933+0000 7fe0b3400700 1 -- 172.21.15.151:0/765471811 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe0ac003910 con 0x7fe0bc105390 2024-05-21T21:43:50.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:50.934+0000 7fe0b3400700 1 -- 172.21.15.151:0/765471811 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7fe0ac029070 con 0x7fe0bc105390 2024-05-21T21:43:50.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:50.934+0000 7fe0c1c00700 1 -- 172.21.15.151:0/765471811 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0bc106870 con 0x7fe0bc105390 2024-05-21T21:43:50.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:50.935+0000 7fe0b3400700 1 --2- 172.21.15.151:0/765471811 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fe0a003e510 0x7fe0a00409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:43:50.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:50.935+0000 7fe0b3400700 1 -- 172.21.15.151:0/765471811 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe0ac01e070 con 0x7fe0bc105390 2024-05-21T21:43:50.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:50.935+0000 7fe0ba000700 1 --2- 172.21.15.151:0/765471811 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fe0a003e510 0x7fe0a00409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:43:50.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:50.936+0000 7fe0ba000700 1 --2- 172.21.15.151:0/765471811 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fe0a003e510 0x7fe0a00409d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe0a8006fd0 tx=0x7fe0a8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:43:50.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:50.941+0000 7fe0b3400700 1 -- 172.21.15.151:0/765471811 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe0ac013070 con 0x7fe0bc105390 2024-05-21T21:43:51.178 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:51.178+0000 7fe0c1c00700 1 -- 172.21.15.151:0/765471811 --> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe0bc19bf60 con 0x7fe0a003e510 2024-05-21T21:43:51.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:51.180+0000 7fe0b3400700 1 -- 172.21.15.151:0/765471811 <== mgr.14152 v2:172.21.15.151:6800/2796622483 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+241 (secure 0 0 0) 0x7fe0bc19bf60 con 0x7fe0a003e510 2024-05-21T21:43:51.181 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T21:43:51.181 INFO:teuthology.orchestra.run.smithi151.stdout:[{"addr": "172.21.15.151", "hostname": "smithi151", "labels": [], "status": ""}, {"addr": "172.21.15.164", "hostname": "smithi164", "labels": [], "status": ""}, {"addr": "172.21.15.184", "hostname": "smithi184", "labels": [], "status": ""}] 2024-05-21T21:43:51.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:51.183+0000 7fe0c1c00700 1 -- 172.21.15.151:0/765471811 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fe0a003e510 msgr2=0x7fe0a00409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:43:51.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:51.183+0000 7fe0c1c00700 1 --2- 172.21.15.151:0/765471811 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fe0a003e510 0x7fe0a00409d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe0a8006fd0 tx=0x7fe0a8006e40 comp rx=0 tx=0).stop 2024-05-21T21:43:51.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:51.183+0000 7fe0c1c00700 1 -- 172.21.15.151:0/765471811 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe0bc105390 msgr2=0x7fe0bc1a0800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:43:51.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:51.183+0000 7fe0c1c00700 1 --2- 172.21.15.151:0/765471811 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe0bc105390 0x7fe0bc1a0800 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac006010 tx=0x7fe0ac005780 comp rx=0 tx=0).stop 2024-05-21T21:43:51.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:51.183+0000 7fe0c1c00700 1 -- 172.21.15.151:0/765471811 shutdown_connections 2024-05-21T21:43:51.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:51.183+0000 7fe0c1c00700 1 --2- 172.21.15.151:0/765471811 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fe0a003e510 0x7fe0a00409d0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:43:51.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:51.183+0000 7fe0c1c00700 1 --2- 172.21.15.151:0/765471811 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe0bc105390 0x7fe0bc1a0800 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:43:51.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:51.183+0000 7fe0c1c00700 1 -- 172.21.15.151:0/765471811 >> 172.21.15.151:0/765471811 conn(0x7fe0bc100c20 msgr2=0x7fe0bc106d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:43:51.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:51.184+0000 7fe0c1c00700 1 -- 172.21.15.151:0/765471811 shutdown_connections 2024-05-21T21:43:51.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:51.184+0000 7fe0c1c00700 1 -- 172.21.15.151:0/765471811 wait complete. 2024-05-21T21:43:51.823 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:51 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:43:51.897 INFO:tasks.cephadm:Setting crush tunables to default 2024-05-21T21:43:51.898 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph osd crush tunables default 2024-05-21T21:43:52.349 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:43:52.749 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:52 smithi151 ceph-mon[31623]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:43:52.749 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:52 smithi151 ceph-mon[31623]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T21:43:54.324 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:54.324+0000 7f404e400700 1 -- 172.21.15.151:0/2566702617 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4048103f30 msgr2=0x7f4048104310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:43:54.324 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:54.324+0000 7f404e400700 1 --2- 172.21.15.151:0/2566702617 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4048103f30 0x7f4048104310 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4038009910 tx=0x7f4038009c20 comp rx=0 tx=0).stop 2024-05-21T21:43:54.324 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:54.324+0000 7f404e400700 1 -- 172.21.15.151:0/2566702617 shutdown_connections 2024-05-21T21:43:54.324 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:54.324+0000 7f404e400700 1 --2- 172.21.15.151:0/2566702617 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4048103f30 0x7f4048104310 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:43:54.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:54.324+0000 7f404e400700 1 -- 172.21.15.151:0/2566702617 >> 172.21.15.151:0/2566702617 conn(0x7f40480fd780 msgr2=0x7f40480ffba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:43:54.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:54.324+0000 7f404e400700 1 -- 172.21.15.151:0/2566702617 shutdown_connections 2024-05-21T21:43:54.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:54.324+0000 7f404e400700 1 -- 172.21.15.151:0/2566702617 wait complete. 2024-05-21T21:43:54.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:54.325+0000 7f404e400700 1 Processor -- start 2024-05-21T21:43:54.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:54.325+0000 7f404e400700 1 -- start start 2024-05-21T21:43:54.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:54.326+0000 7f404e400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4048103f30 0x7f404819e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:43:54.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:54.326+0000 7f404e400700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f404810d2d0 con 0x7f4048103f30 2024-05-21T21:43:54.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:54.326+0000 7f404d000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4048103f30 0x7f404819e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:43:54.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:54.326+0000 7f404d000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4048103f30 0x7f404819e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:53806/0 (socket says 172.21.15.151:53806) 2024-05-21T21:43:54.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:54.326+0000 7f404d000700 1 -- 172.21.15.151:0/1411402349 learned_addr learned my addr 172.21.15.151:0/1411402349 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:43:54.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:54.327+0000 7f404d000700 1 -- 172.21.15.151:0/1411402349 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f404819eb20 con 0x7f4048103f30 2024-05-21T21:43:54.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:54.327+0000 7f404d000700 1 --2- 172.21.15.151:0/1411402349 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4048103f30 0x7f404819e5e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4038006010 tx=0x7f4038005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:43:54.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:54.327+0000 7f4046000700 1 -- 172.21.15.151:0/1411402349 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4038026070 con 0x7f4048103f30 2024-05-21T21:43:54.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:54.327+0000 7f4046000700 1 -- 172.21.15.151:0/1411402349 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f403800bc00 con 0x7f4048103f30 2024-05-21T21:43:54.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:54.327+0000 7f404e400700 1 -- 172.21.15.151:0/1411402349 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f404819edb0 con 0x7f4048103f30 2024-05-21T21:43:54.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:54.327+0000 7f4046000700 1 -- 172.21.15.151:0/1411402349 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4038003910 con 0x7f4048103f30 2024-05-21T21:43:54.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:54.327+0000 7f404e400700 1 -- 172.21.15.151:0/1411402349 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40481986b0 con 0x7f4048103f30 2024-05-21T21:43:54.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:54.329+0000 7f404e400700 1 -- 172.21.15.151:0/1411402349 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f404806bae0 con 0x7f4048103f30 2024-05-21T21:43:54.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:54.329+0000 7f4046000700 1 -- 172.21.15.151:0/1411402349 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7f4038029070 con 0x7f4048103f30 2024-05-21T21:43:54.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:54.329+0000 7f4046000700 1 --2- 172.21.15.151:0/1411402349 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f402c03e510 0x7f402c0409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:43:54.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:54.330+0000 7f4046000700 1 -- 172.21.15.151:0/1411402349 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f403801e070 con 0x7f4048103f30 2024-05-21T21:43:54.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:54.330+0000 7f403fe00700 1 --2- 172.21.15.151:0/1411402349 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f402c03e510 0x7f402c0409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:43:54.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:54.331+0000 7f403fe00700 1 --2- 172.21.15.151:0/1411402349 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f402c03e510 0x7f402c0409d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4030006fd0 tx=0x7f4030006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:43:54.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:54.335+0000 7f4046000700 1 -- 172.21.15.151:0/1411402349 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4038013070 con 0x7f4048103f30 2024-05-21T21:43:54.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:54.575+0000 7f404e400700 1 -- 172.21.15.151:0/1411402349 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f4048064ba0 con 0x7f4048103f30 2024-05-21T21:43:54.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:54 smithi151 ceph-mon[31623]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:43:54.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:54 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:43:54.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:54 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:43:54.954 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:54 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:43:54.954 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:54 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:43:54.954 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:54 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi184", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:43:54.954 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:54 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:43:54.954 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:54 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:43:55.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:55.557+0000 7f4046000700 1 -- 172.21.15.151:0/1411402349 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f403800d340 con 0x7f4048103f30 2024-05-21T21:43:55.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:55.563+0000 7f404e400700 1 -- 172.21.15.151:0/1411402349 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f402c03e510 msgr2=0x7f402c0409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:43:55.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:55.563+0000 7f404e400700 1 --2- 172.21.15.151:0/1411402349 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f402c03e510 0x7f402c0409d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4030006fd0 tx=0x7f4030006e40 comp rx=0 tx=0).stop 2024-05-21T21:43:55.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:55.563+0000 7f404e400700 1 -- 172.21.15.151:0/1411402349 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4048103f30 msgr2=0x7f404819e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:43:55.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:55.563+0000 7f404e400700 1 --2- 172.21.15.151:0/1411402349 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4048103f30 0x7f404819e5e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4038006010 tx=0x7f4038005780 comp rx=0 tx=0).stop 2024-05-21T21:43:55.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:55.563+0000 7f404e400700 1 -- 172.21.15.151:0/1411402349 shutdown_connections 2024-05-21T21:43:55.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:55.563+0000 7f404e400700 1 --2- 172.21.15.151:0/1411402349 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f402c03e510 0x7f402c0409d0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:43:55.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:55.563+0000 7f404e400700 1 --2- 172.21.15.151:0/1411402349 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4048103f30 0x7f404819e5e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:43:55.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:55.563+0000 7f404e400700 1 -- 172.21.15.151:0/1411402349 >> 172.21.15.151:0/1411402349 conn(0x7f40480fd780 msgr2=0x7f4048104ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:43:55.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:55.564+0000 7f404e400700 1 -- 172.21.15.151:0/1411402349 shutdown_connections 2024-05-21T21:43:55.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:43:55.564+0000 7f404e400700 1 -- 172.21.15.151:0/1411402349 wait complete. 2024-05-21T21:43:55.565 INFO:teuthology.orchestra.run.smithi151.stderr:adjusted tunables profile to default 2024-05-21T21:43:55.952 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:55 smithi151 ceph-mon[31623]: Updating smithi184:/etc/ceph/ceph.conf 2024-05-21T21:43:55.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:55 smithi151 ceph-mon[31623]: Updating smithi184:/var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T21:43:55.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:55 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/1411402349' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-21T21:43:55.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:55 smithi151 ceph-mon[31623]: Updating smithi184:/etc/ceph/ceph.client.admin.keyring 2024-05-21T21:43:55.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:55 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:43:55.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:55 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:43:55.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:55 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:43:56.318 INFO:tasks.cephadm:Adding mon.a on smithi151 2024-05-21T21:43:56.319 INFO:tasks.cephadm:Adding mon.b on smithi164 2024-05-21T21:43:56.319 INFO:tasks.cephadm:Adding mon.c on smithi184 2024-05-21T21:43:56.319 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph orch apply mon '3;smithi151:172.21.15.151=a;smithi164:172.21.15.164=b;smithi184:172.21.15.184=c' 2024-05-21T21:43:56.556 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T21:43:56.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:56 smithi151 ceph-mon[31623]: Updating smithi184:/var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-21T21:43:56.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:56 smithi151 ceph-mon[31623]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:43:56.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:56 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/1411402349' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-21T21:43:56.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:56 smithi151 ceph-mon[31623]: osdmap e4: 0 total, 0 up, 0 in 2024-05-21T21:43:58.057 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:43:58.056+0000 7f02ed170700 1 -- 172.21.15.184:0/1527565637 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f02e8104110 msgr2=0x7f02e81044f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:43:58.057 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:43:58.056+0000 7f02ed170700 1 --2- 172.21.15.184:0/1527565637 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f02e8104110 0x7f02e81044f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f02d800b0a0 tx=0x7f02d800b3b0 comp rx=0 tx=0).stop 2024-05-21T21:43:58.057 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:43:58.056+0000 7f02ed170700 1 -- 172.21.15.184:0/1527565637 shutdown_connections 2024-05-21T21:43:58.057 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:43:58.056+0000 7f02ed170700 1 --2- 172.21.15.184:0/1527565637 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f02e8104110 0x7f02e81044f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:43:58.057 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:43:58.056+0000 7f02ed170700 1 -- 172.21.15.184:0/1527565637 >> 172.21.15.184:0/1527565637 conn(0x7f02e80fd980 msgr2=0x7f02e80ffda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:43:58.057 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:43:58.056+0000 7f02ed170700 1 -- 172.21.15.184:0/1527565637 shutdown_connections 2024-05-21T21:43:58.057 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:43:58.056+0000 7f02ed170700 1 -- 172.21.15.184:0/1527565637 wait complete. 2024-05-21T21:43:58.057 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:43:58.057+0000 7f02ed170700 1 Processor -- start 2024-05-21T21:43:58.058 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:43:58.057+0000 7f02ed170700 1 -- start start 2024-05-21T21:43:58.058 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:43:58.057+0000 7f02ed170700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f02e8104110 0x7f02e819e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:43:58.058 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:43:58.057+0000 7f02ed170700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f02e810d4b0 con 0x7f02e8104110 2024-05-21T21:43:58.059 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:43:58.058+0000 7f02e6d9d700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f02e8104110 0x7f02e819e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:43:58.059 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:43:58.058+0000 7f02e6d9d700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f02e8104110 0x7f02e819e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.184:42140/0 (socket says 172.21.15.184:42140) 2024-05-21T21:43:58.059 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:43:58.058+0000 7f02e6d9d700 1 -- 172.21.15.184:0/4263541758 learned_addr learned my addr 172.21.15.184:0/4263541758 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-05-21T21:43:58.059 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:43:58.058+0000 7f02e6d9d700 1 -- 172.21.15.184:0/4263541758 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02e819eba0 con 0x7f02e8104110 2024-05-21T21:43:58.059 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:43:58.059+0000 7f02e6d9d700 1 --2- 172.21.15.184:0/4263541758 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f02e8104110 0x7f02e819e660 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f02d8015040 tx=0x7f02d8009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:43:58.060 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:43:58.059+0000 7f02dffff700 1 -- 172.21.15.184:0/4263541758 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f02d8021020 con 0x7f02e8104110 2024-05-21T21:43:58.060 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:43:58.059+0000 7f02ed170700 1 -- 172.21.15.184:0/4263541758 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02e819ee30 con 0x7f02e8104110 2024-05-21T21:43:58.060 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:43:58.059+0000 7f02ed170700 1 -- 172.21.15.184:0/4263541758 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02e81987e0 con 0x7f02e8104110 2024-05-21T21:43:58.060 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:43:58.059+0000 7f02dffff700 1 -- 172.21.15.184:0/4263541758 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02d8009410 con 0x7f02e8104110 2024-05-21T21:43:58.060 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:43:58.059+0000 7f02dffff700 1 -- 172.21.15.184:0/4263541758 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f02d8017040 con 0x7f02e8104110 2024-05-21T21:43:58.060 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:43:58.060+0000 7f02ed170700 1 -- 172.21.15.184:0/4263541758 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02c8005330 con 0x7f02e8104110 2024-05-21T21:43:58.061 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:43:58.060+0000 7f02dffff700 1 -- 172.21.15.184:0/4263541758 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7f02d8024020 con 0x7f02e8104110 2024-05-21T21:43:58.061 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:43:58.061+0000 7f02dffff700 1 --2- 172.21.15.184:0/4263541758 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f02d403e510 0x7f02d40409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:43:58.061 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:43:58.061+0000 7f02dffff700 1 -- 172.21.15.184:0/4263541758 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f02d800f050 con 0x7f02e8104110 2024-05-21T21:43:58.061 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:43:58.061+0000 7f02e659c700 1 --2- 172.21.15.184:0/4263541758 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f02d403e510 0x7f02d40409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:43:58.062 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:43:58.062+0000 7f02e659c700 1 --2- 172.21.15.184:0/4263541758 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f02d403e510 0x7f02d40409d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f02d0006fd0 tx=0x7f02d0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:43:58.076 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:43:58.076+0000 7f02dffff700 1 -- 172.21.15.184:0/4263541758 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f02d802b030 con 0x7f02e8104110 2024-05-21T21:43:58.291 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:43:58.290+0000 7f02ed170700 1 -- 172.21.15.184:0/4263541758 --> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi151:172.21.15.151=a;smithi164:172.21.15.164=b;smithi184:172.21.15.184=c", "target": ["mon-mgr", ""]}) -- 0x7f02c8000c80 con 0x7f02d403e510 2024-05-21T21:43:58.307 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:43:58.307+0000 7f02dffff700 1 -- 172.21.15.184:0/4263541758 <== mgr.14152 v2:172.21.15.151:6800/2796622483 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f02c8000c80 con 0x7f02d403e510 2024-05-21T21:43:58.307 INFO:teuthology.orchestra.run.smithi184.stdout:Scheduled mon update... 2024-05-21T21:43:58.309 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:43:58.309+0000 7f02ed170700 1 -- 172.21.15.184:0/4263541758 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f02d403e510 msgr2=0x7f02d40409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:43:58.309 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:43:58.309+0000 7f02ed170700 1 --2- 172.21.15.184:0/4263541758 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f02d403e510 0x7f02d40409d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f02d0006fd0 tx=0x7f02d0006e40 comp rx=0 tx=0).stop 2024-05-21T21:43:58.310 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:43:58.309+0000 7f02ed170700 1 -- 172.21.15.184:0/4263541758 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f02e8104110 msgr2=0x7f02e819e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:43:58.310 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:43:58.309+0000 7f02ed170700 1 --2- 172.21.15.184:0/4263541758 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f02e8104110 0x7f02e819e660 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f02d8015040 tx=0x7f02d8009800 comp rx=0 tx=0).stop 2024-05-21T21:43:58.310 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:43:58.309+0000 7f02ed170700 1 -- 172.21.15.184:0/4263541758 shutdown_connections 2024-05-21T21:43:58.310 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:43:58.309+0000 7f02ed170700 1 --2- 172.21.15.184:0/4263541758 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f02d403e510 0x7f02d40409d0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:43:58.310 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:43:58.309+0000 7f02ed170700 1 --2- 172.21.15.184:0/4263541758 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f02e8104110 0x7f02e819e660 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:43:58.310 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:43:58.309+0000 7f02ed170700 1 -- 172.21.15.184:0/4263541758 >> 172.21.15.184:0/4263541758 conn(0x7f02e80fd980 msgr2=0x7f02e8106ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:43:58.311 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:43:58.309+0000 7f02ed170700 1 -- 172.21.15.184:0/4263541758 shutdown_connections 2024-05-21T21:43:58.311 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:43:58.309+0000 7f02ed170700 1 -- 172.21.15.184:0/4263541758 wait complete. 2024-05-21T21:43:58.838 DEBUG:teuthology.orchestra.run.smithi164:mon.b> sudo journalctl -f -n 0 -u ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@mon.b.service 2024-05-21T21:43:58.842 DEBUG:teuthology.orchestra.run.smithi184:mon.c> sudo journalctl -f -n 0 -u ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@mon.c.service 2024-05-21T21:43:58.844 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-21T21:43:58.844 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph mon dump -f json 2024-05-21T21:43:58.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:58 smithi151 ceph-mon[31623]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:43:58.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:58 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:43:58.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:58 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:43:58.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:58 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:43:58.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:58 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:43:58.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:58 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:43:58.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:58 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-21T21:43:58.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:58 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:43:59.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:59 smithi151 ceph-mon[31623]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi151:172.21.15.151=a;smithi164:172.21.15.164=b;smithi184:172.21.15.184=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:43:59.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:59 smithi151 ceph-mon[31623]: Saving service mon spec with placement smithi151:172.21.15.151=a;smithi164:172.21.15.164=b;smithi184:172.21.15.184=c;count:3 2024-05-21T21:43:59.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:43:59 smithi151 ceph-mon[31623]: Deploying daemon mon.c on smithi184 2024-05-21T21:44:00.001 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T21:44:00.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:00 smithi151 ceph-mon[31623]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:02.664 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:02.659+0000 7faef7a18700 1 -- 172.21.15.184:0/2179954133 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faef01020b0 msgr2=0x7faef0102490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:44:02.664 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:02.659+0000 7faef7a18700 1 --2- 172.21.15.184:0/2179954133 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faef01020b0 0x7faef0102490 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7faeec009910 tx=0x7faeec009c20 comp rx=0 tx=0).stop 2024-05-21T21:44:02.664 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:02.659+0000 7faef7a18700 1 -- 172.21.15.184:0/2179954133 shutdown_connections 2024-05-21T21:44:02.664 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:02.659+0000 7faef7a18700 1 --2- 172.21.15.184:0/2179954133 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faef01020b0 0x7faef0102490 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:44:02.665 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:02.659+0000 7faef7a18700 1 -- 172.21.15.184:0/2179954133 >> 172.21.15.184:0/2179954133 conn(0x7faef00fd980 msgr2=0x7faef00ffda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:44:02.665 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:02.659+0000 7faef7a18700 1 -- 172.21.15.184:0/2179954133 shutdown_connections 2024-05-21T21:44:02.665 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:02.659+0000 7faef7a18700 1 -- 172.21.15.184:0/2179954133 wait complete. 2024-05-21T21:44:02.665 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:02.659+0000 7faef7a18700 1 Processor -- start 2024-05-21T21:44:02.665 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:02.659+0000 7faef7a18700 1 -- start start 2024-05-21T21:44:02.665 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:02.660+0000 7faef7a18700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faef01020b0 0x7faef019e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:44:02.665 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:02.660+0000 7faef7a18700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7faef010c330 con 0x7faef01020b0 2024-05-21T21:44:02.665 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:02.660+0000 7faef57b4700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faef01020b0 0x7faef019e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:44:02.665 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:02.660+0000 7faef57b4700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faef01020b0 0x7faef019e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.184:48068/0 (socket says 172.21.15.184:48068) 2024-05-21T21:44:02.665 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:02.660+0000 7faef57b4700 1 -- 172.21.15.184:0/3177859394 learned_addr learned my addr 172.21.15.184:0/3177859394 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-05-21T21:44:02.665 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:02.661+0000 7faef57b4700 1 -- 172.21.15.184:0/3177859394 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faef019eb90 con 0x7faef01020b0 2024-05-21T21:44:02.666 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:02.661+0000 7faef57b4700 1 --2- 172.21.15.184:0/3177859394 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faef01020b0 0x7faef019e650 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7faeec0056d0 tx=0x7faeec0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:44:02.666 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:02.661+0000 7faee67fc700 1 -- 172.21.15.184:0/3177859394 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faeec026070 con 0x7faef01020b0 2024-05-21T21:44:02.666 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:02.661+0000 7faef7a18700 1 -- 172.21.15.184:0/3177859394 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faef019ee80 con 0x7faef01020b0 2024-05-21T21:44:02.666 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:02.661+0000 7faef7a18700 1 -- 172.21.15.184:0/3177859394 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faef01987d0 con 0x7faef01020b0 2024-05-21T21:44:02.666 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:02.662+0000 7faee67fc700 1 -- 172.21.15.184:0/3177859394 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faeec00bba0 con 0x7faef01020b0 2024-05-21T21:44:02.666 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:02.662+0000 7faee67fc700 1 -- 172.21.15.184:0/3177859394 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faeec030060 con 0x7faef01020b0 2024-05-21T21:44:02.666 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:02.662+0000 7faef7a18700 1 -- 172.21.15.184:0/3177859394 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faef0103590 con 0x7faef01020b0 2024-05-21T21:44:02.666 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:02.663+0000 7faee67fc700 1 -- 172.21.15.184:0/3177859394 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7faeec029070 con 0x7faef01020b0 2024-05-21T21:44:02.667 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:02.663+0000 7faee67fc700 1 --2- 172.21.15.184:0/3177859394 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7faedc03e510 0x7faedc0409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:44:02.667 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:02.663+0000 7faee67fc700 1 -- 172.21.15.184:0/3177859394 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7faeec01e070 con 0x7faef01020b0 2024-05-21T21:44:02.667 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:02.664+0000 7faef4fb3700 1 --2- 172.21.15.184:0/3177859394 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7faedc03e510 0x7faedc0409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:44:02.667 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:02.664+0000 7faef4fb3700 1 --2- 172.21.15.184:0/3177859394 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7faedc03e510 0x7faedc0409d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7faee0006fd0 tx=0x7faee0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:44:02.679 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:02.679+0000 7faee67fc700 1 -- 172.21.15.184:0/3177859394 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7faeec013070 con 0x7faef01020b0 2024-05-21T21:44:02.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:02 smithi151 ceph-mon[31623]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:02.962 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:02.961+0000 7faee67fc700 1 -- 172.21.15.184:0/3177859394 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7faeec013250 con 0x7faef01020b0 2024-05-21T21:44:02.978 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:02.978+0000 7faef7a18700 1 -- 172.21.15.184:0/3177859394 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7faef0064bd0 con 0x7faef01020b0 2024-05-21T21:44:03.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:02 smithi184 systemd[1]: Started Ceph mon.c for c8c7a2e6-17ba-11ef-bc9a-c7b262605968. 2024-05-21T21:44:07.990 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-05-21T21:44:07.990 INFO:teuthology.orchestra.run.smithi184.stdout:{"epoch":2,"fsid":"c8c7a2e6-17ba-11ef-bc9a-c7b262605968","modified":"2024-05-21T21:44:02.958032Z","created":"2024-05-21T21:41:48.311542Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:3300","nonce":0},{"type":"v1","addr":"172.21.15.151:6789","nonce":0}]},"addr":"172.21.15.151:6789/0","public_addr":"172.21.15.151:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:3300","nonce":0},{"type":"v1","addr":"172.21.15.184:6789","nonce":0}]},"addr":"172.21.15.184:6789/0","public_addr":"172.21.15.184:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-21T21:44:07.990 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:07.988+0000 7faee67fc700 1 -- 172.21.15.184:0/3177859394 <== mon.0 v2:172.21.15.151:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1019 (secure 0 0 0) 0x7faef0064bd0 con 0x7faef01020b0 2024-05-21T21:44:07.994 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:07.992+0000 7faef7a18700 1 -- 172.21.15.184:0/3177859394 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7faedc03e510 msgr2=0x7faedc0409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:44:07.994 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:07.992+0000 7faef7a18700 1 --2- 172.21.15.184:0/3177859394 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7faedc03e510 0x7faedc0409d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7faee0006fd0 tx=0x7faee0006e40 comp rx=0 tx=0).stop 2024-05-21T21:44:07.994 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:07.992+0000 7faef7a18700 1 -- 172.21.15.184:0/3177859394 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faef01020b0 msgr2=0x7faef019e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:44:07.994 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:07.992+0000 7faef7a18700 1 --2- 172.21.15.184:0/3177859394 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faef01020b0 0x7faef019e650 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7faeec0056d0 tx=0x7faeec0057b0 comp rx=0 tx=0).stop 2024-05-21T21:44:07.994 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:07.992+0000 7faef7a18700 1 -- 172.21.15.184:0/3177859394 shutdown_connections 2024-05-21T21:44:07.994 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:07.992+0000 7faef7a18700 1 --2- 172.21.15.184:0/3177859394 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7faedc03e510 0x7faedc0409d0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:44:07.995 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:07.992+0000 7faef7a18700 1 --2- 172.21.15.184:0/3177859394 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faef01020b0 0x7faef019e650 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:44:07.995 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:07.992+0000 7faef7a18700 1 -- 172.21.15.184:0/3177859394 >> 172.21.15.184:0/3177859394 conn(0x7faef00fd980 msgr2=0x7faef006b200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:44:07.995 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:07.992+0000 7faef7a18700 1 -- 172.21.15.184:0/3177859394 shutdown_connections 2024-05-21T21:44:07.995 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:07.992+0000 7faef7a18700 1 -- 172.21.15.184:0/3177859394 wait complete. 2024-05-21T21:44:07.995 INFO:teuthology.orchestra.run.smithi184.stderr:dumped monmap epoch 2 2024-05-21T21:44:08.296 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:08 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-21T21:44:08.296 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:08 smithi184 ceph-mon[36942]: mon.a calling monitor election 2024-05-21T21:44:08.296 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:08 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T21:44:08.296 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:08 smithi184 ceph-mon[36942]: from='client.? 172.21.15.184:0/3177859394' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-21T21:44:08.296 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:08 smithi184 ceph-mon[36942]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:08.296 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:08 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T21:44:08.296 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:08 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T21:44:08.296 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:08 smithi184 ceph-mon[36942]: mon.c calling monitor election 2024-05-21T21:44:08.297 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:08 smithi184 ceph-mon[36942]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:08.297 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:08 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T21:44:08.297 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:08 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T21:44:08.297 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:08 smithi184 ceph-mon[36942]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:08.297 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:08 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T21:44:08.297 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:08 smithi184 ceph-mon[36942]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-21T21:44:08.297 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:08 smithi184 ceph-mon[36942]: monmap epoch 2 2024-05-21T21:44:08.297 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:08 smithi184 ceph-mon[36942]: fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 2024-05-21T21:44:08.297 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:08 smithi184 ceph-mon[36942]: last_changed 2024-05-21T21:44:02.958032+0000 2024-05-21T21:44:08.297 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:08 smithi184 ceph-mon[36942]: created 2024-05-21T21:41:48.311542+0000 2024-05-21T21:44:08.297 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:08 smithi184 ceph-mon[36942]: min_mon_release 19 (squid) 2024-05-21T21:44:08.297 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:08 smithi184 ceph-mon[36942]: election_strategy: 1 2024-05-21T21:44:08.298 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:08 smithi184 ceph-mon[36942]: 0: [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] mon.a 2024-05-21T21:44:08.298 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:08 smithi184 ceph-mon[36942]: 1: [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] mon.c 2024-05-21T21:44:08.298 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:08 smithi184 ceph-mon[36942]: fsmap 2024-05-21T21:44:08.298 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:08 smithi184 ceph-mon[36942]: osdmap e4: 0 total, 0 up, 0 in 2024-05-21T21:44:08.298 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:08 smithi184 ceph-mon[36942]: mgrmap e13: x(active, since 58s) 2024-05-21T21:44:08.298 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:08 smithi184 ceph-mon[36942]: overall HEALTH_OK 2024-05-21T21:44:08.298 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:08 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:08.298 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:08 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:08.298 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:08 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:08.298 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:08 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-21T21:44:08.298 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:08 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:44:08.360 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:08 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-21T21:44:08.360 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:08 smithi151 ceph-mon[31623]: mon.a calling monitor election 2024-05-21T21:44:08.360 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:08 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T21:44:08.361 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:08 smithi151 ceph-mon[31623]: from='client.? 172.21.15.184:0/3177859394' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-21T21:44:08.361 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:08 smithi151 ceph-mon[31623]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:08.361 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:08 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T21:44:08.361 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:08 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T21:44:08.361 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:08 smithi151 ceph-mon[31623]: mon.c calling monitor election 2024-05-21T21:44:08.361 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:08 smithi151 ceph-mon[31623]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:08.361 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:08 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T21:44:08.361 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:08 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T21:44:08.361 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:08 smithi151 ceph-mon[31623]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:08.361 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:08 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T21:44:08.361 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:08 smithi151 ceph-mon[31623]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-21T21:44:08.361 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:08 smithi151 ceph-mon[31623]: monmap epoch 2 2024-05-21T21:44:08.361 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:08 smithi151 ceph-mon[31623]: fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 2024-05-21T21:44:08.361 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:08 smithi151 ceph-mon[31623]: last_changed 2024-05-21T21:44:02.958032+0000 2024-05-21T21:44:08.362 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:08 smithi151 ceph-mon[31623]: created 2024-05-21T21:41:48.311542+0000 2024-05-21T21:44:08.362 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:08 smithi151 ceph-mon[31623]: min_mon_release 19 (squid) 2024-05-21T21:44:08.362 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:08 smithi151 ceph-mon[31623]: election_strategy: 1 2024-05-21T21:44:08.362 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:08 smithi151 ceph-mon[31623]: 0: [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] mon.a 2024-05-21T21:44:08.362 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:08 smithi151 ceph-mon[31623]: 1: [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] mon.c 2024-05-21T21:44:08.362 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:08 smithi151 ceph-mon[31623]: fsmap 2024-05-21T21:44:08.362 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:08 smithi151 ceph-mon[31623]: osdmap e4: 0 total, 0 up, 0 in 2024-05-21T21:44:08.362 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:08 smithi151 ceph-mon[31623]: mgrmap e13: x(active, since 58s) 2024-05-21T21:44:08.362 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:08 smithi151 ceph-mon[31623]: overall HEALTH_OK 2024-05-21T21:44:08.362 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:08 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:08.362 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:08 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:08.362 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:08 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:08.363 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:08 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-21T21:44:08.363 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:08 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:44:09.392 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:09 smithi151 ceph-mon[31623]: Deploying daemon mon.b on smithi164 2024-05-21T21:44:09.392 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:09 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T21:44:09.485 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:09 smithi184 ceph-mon[36942]: Deploying daemon mon.b on smithi164 2024-05-21T21:44:09.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:09 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T21:44:09.580 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-21T21:44:09.581 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph mon dump -f json 2024-05-21T21:44:09.828 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.c/config 2024-05-21T21:44:10.235 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:10 smithi184 ceph-mon[36942]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:10.452 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:10 smithi151 ceph-mon[31623]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:11.315 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:11.313+0000 7fab027ca700 1 -- 172.21.15.184:0/1899915628 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faafc1032c0 msgr2=0x7faafc1036a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:44:11.315 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:11.313+0000 7fab027ca700 1 --2- 172.21.15.184:0/1899915628 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faafc1032c0 0x7faafc1036a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7faaec009910 tx=0x7faaec009c20 comp rx=0 tx=0).stop 2024-05-21T21:44:11.315 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:11.313+0000 7fab027ca700 1 -- 172.21.15.184:0/1899915628 shutdown_connections 2024-05-21T21:44:11.315 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:11.313+0000 7fab027ca700 1 --2- 172.21.15.184:0/1899915628 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faafc1032c0 0x7faafc1036a0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:44:11.315 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:11.313+0000 7fab027ca700 1 -- 172.21.15.184:0/1899915628 >> 172.21.15.184:0/1899915628 conn(0x7faafc0feb50 msgr2=0x7faafc100f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:44:11.316 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:11.313+0000 7fab027ca700 1 -- 172.21.15.184:0/1899915628 shutdown_connections 2024-05-21T21:44:11.316 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:11.313+0000 7fab027ca700 1 -- 172.21.15.184:0/1899915628 wait complete. 2024-05-21T21:44:11.316 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:11.314+0000 7fab027ca700 1 Processor -- start 2024-05-21T21:44:11.316 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:11.314+0000 7fab027ca700 1 -- start start 2024-05-21T21:44:11.317 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:11.314+0000 7fab027ca700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faafc1032c0 0x7faafc19e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:44:11.318 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:11.314+0000 7fab027ca700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faafc19ee10 0x7faafc198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:44:11.318 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:11.314+0000 7fab027ca700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7faafc10d540 con 0x7faafc19ee10 2024-05-21T21:44:11.318 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:11.314+0000 7fab027ca700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7faafc10d3c0 con 0x7faafc1032c0 2024-05-21T21:44:11.318 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:11.315+0000 7faafbfff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faafc1032c0 0x7faafc19e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:44:11.318 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:11.315+0000 7faafbfff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faafc1032c0 0x7faafc19e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:59732/0 (socket says 172.21.15.184:59732) 2024-05-21T21:44:11.318 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:11.315+0000 7faafbfff700 1 -- 172.21.15.184:0/1417005022 learned_addr learned my addr 172.21.15.184:0/1417005022 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-05-21T21:44:11.318 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:11.315+0000 7faafbfff700 1 -- 172.21.15.184:0/1417005022 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faafc1032c0 msgr2=0x7faafc19e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-05-21T21:44:11.318 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:11.315+0000 7faafbfff700 1 -- 172.21.15.184:0/1417005022 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faafc1032c0 msgr2=0x7faafc19e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-21T21:44:11.318 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:11.315+0000 7faafbfff700 1 --2- 172.21.15.184:0/1417005022 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faafc1032c0 0x7faafc19e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-21T21:44:11.318 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:11.315+0000 7faafbfff700 1 --2- 172.21.15.184:0/1417005022 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faafc1032c0 0x7faafc19e8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-21T21:44:11.318 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:11.315+0000 7faafb7fe700 1 --2- 172.21.15.184:0/1417005022 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faafc19ee10 0x7faafc198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:44:11.319 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:11.315+0000 7faafb7fe700 1 -- 172.21.15.184:0/1417005022 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faafc1032c0 msgr2=0x7faafc19e8d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-21T21:44:11.319 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:11.315+0000 7faafb7fe700 1 --2- 172.21.15.184:0/1417005022 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faafc1032c0 0x7faafc19e8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:44:11.319 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:11.315+0000 7faafb7fe700 1 -- 172.21.15.184:0/1417005022 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faafc19f450 con 0x7faafc19ee10 2024-05-21T21:44:11.319 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:11.316+0000 7faafb7fe700 1 --2- 172.21.15.184:0/1417005022 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faafc19ee10 0x7faafc198a90 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7faaf000b4d0 tx=0x7faaf000b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:44:11.319 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:11.316+0000 7faaf97fa700 1 -- 172.21.15.184:0/1417005022 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7faaf0011630 con 0x7faafc19ee10 2024-05-21T21:44:11.319 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:11.316+0000 7fab027ca700 1 -- 172.21.15.184:0/1417005022 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faafc198fd0 con 0x7faafc19ee10 2024-05-21T21:44:11.319 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:11.316+0000 7fab027ca700 1 -- 172.21.15.184:0/1417005022 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faafc199560 con 0x7faafc19ee10 2024-05-21T21:44:11.319 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:11.316+0000 7faaf97fa700 1 -- 172.21.15.184:0/1417005022 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faaf0011c80 con 0x7faafc19ee10 2024-05-21T21:44:11.319 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:11.316+0000 7faaf97fa700 1 -- 172.21.15.184:0/1417005022 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7faaf001adc0 con 0x7faafc19ee10 2024-05-21T21:44:11.320 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:11.317+0000 7fab027ca700 1 -- 172.21.15.184:0/1417005022 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faae8005330 con 0x7faafc19ee10 2024-05-21T21:44:11.320 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:11.318+0000 7faaf97fa700 1 -- 172.21.15.184:0/1417005022 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7faaf0018020 con 0x7faafc19ee10 2024-05-21T21:44:11.320 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:11.318+0000 7faaf97fa700 1 --2- 172.21.15.184:0/1417005022 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7faae403e2b0 0x7faae4040770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:44:11.320 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:11.318+0000 7faaf97fa700 1 -- 172.21.15.184:0/1417005022 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7faaf0052420 con 0x7faafc19ee10 2024-05-21T21:44:11.320 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:11.318+0000 7faafbfff700 1 --2- 172.21.15.184:0/1417005022 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7faae403e2b0 0x7faae4040770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:44:11.320 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:11.319+0000 7faafbfff700 1 --2- 172.21.15.184:0/1417005022 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7faae403e2b0 0x7faae4040770 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7faaec009750 tx=0x7faaec01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:44:11.336 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:11.336+0000 7faaf97fa700 1 -- 172.21.15.184:0/1417005022 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7faaf001f020 con 0x7faafc19ee10 2024-05-21T21:44:11.623 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:11.622+0000 7fab027ca700 1 -- 172.21.15.184:0/1417005022 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7faae80058f0 con 0x7faafc19ee10 2024-05-21T21:44:11.624 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:11.623+0000 7faaf97fa700 1 -- 172.21.15.184:0/1417005022 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1019 (secure 0 0 0) 0x7faaf00105e0 con 0x7faafc19ee10 2024-05-21T21:44:11.624 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-05-21T21:44:11.624 INFO:teuthology.orchestra.run.smithi184.stdout:{"epoch":2,"fsid":"c8c7a2e6-17ba-11ef-bc9a-c7b262605968","modified":"2024-05-21T21:44:02.958032Z","created":"2024-05-21T21:41:48.311542Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:3300","nonce":0},{"type":"v1","addr":"172.21.15.151:6789","nonce":0}]},"addr":"172.21.15.151:6789/0","public_addr":"172.21.15.151:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:3300","nonce":0},{"type":"v1","addr":"172.21.15.184:6789","nonce":0}]},"addr":"172.21.15.184:6789/0","public_addr":"172.21.15.184:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-21T21:44:11.626 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:11.625+0000 7fab027ca700 1 -- 172.21.15.184:0/1417005022 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7faae403e2b0 msgr2=0x7faae4040770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:44:11.627 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:11.625+0000 7fab027ca700 1 --2- 172.21.15.184:0/1417005022 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7faae403e2b0 0x7faae4040770 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7faaec009750 tx=0x7faaec01a040 comp rx=0 tx=0).stop 2024-05-21T21:44:11.627 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:11.625+0000 7fab027ca700 1 -- 172.21.15.184:0/1417005022 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faafc19ee10 msgr2=0x7faafc198a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:44:11.627 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:11.625+0000 7fab027ca700 1 --2- 172.21.15.184:0/1417005022 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faafc19ee10 0x7faafc198a90 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7faaf000b4d0 tx=0x7faaf000b890 comp rx=0 tx=0).stop 2024-05-21T21:44:11.627 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:11.625+0000 7fab027ca700 1 -- 172.21.15.184:0/1417005022 shutdown_connections 2024-05-21T21:44:11.627 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:11.625+0000 7fab027ca700 1 --2- 172.21.15.184:0/1417005022 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7faae403e2b0 0x7faae4040770 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:44:11.627 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:11.625+0000 7fab027ca700 1 --2- 172.21.15.184:0/1417005022 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faafc1032c0 0x7faafc19e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:44:11.627 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:11.625+0000 7fab027ca700 1 --2- 172.21.15.184:0/1417005022 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faafc19ee10 0x7faafc198a90 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:44:11.627 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:11.625+0000 7fab027ca700 1 -- 172.21.15.184:0/1417005022 >> 172.21.15.184:0/1417005022 conn(0x7faafc0feb50 msgr2=0x7faafc104c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:44:11.627 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:11.625+0000 7fab027ca700 1 -- 172.21.15.184:0/1417005022 shutdown_connections 2024-05-21T21:44:11.627 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:11.625+0000 7fab027ca700 1 -- 172.21.15.184:0/1417005022 wait complete. 2024-05-21T21:44:11.628 INFO:teuthology.orchestra.run.smithi184.stderr:dumped monmap epoch 2 2024-05-21T21:44:11.794 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:11 smithi164 podman[36250]: 2024-05-21 21:44:11.794059066 +0000 UTC m=+0.133000337 container create 921034200eb5664d9bfa4bbe36821e5483583cfc91526f16e6499d118ba2ac61 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mon-b, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, ceph=True, CEPH_POINT_RELEASE=) 2024-05-21T21:44:12.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 podman[36250]: 2024-05-21 21:44:12.013760829 +0000 UTC m=+0.352702115 container init 921034200eb5664d9bfa4bbe36821e5483583cfc91526f16e6499d118ba2ac61 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mon-b, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-05-21T21:44:12.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 podman[36250]: 2024-05-21 21:44:12.017012441 +0000 UTC m=+0.355953720 container start 921034200eb5664d9bfa4bbe36821e5483583cfc91526f16e6499d118ba2ac61 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mon-b, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-05-21T21:44:12.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: set uid:gid to 167:167 (ceph:ceph) 2024-05-21T21:44:12.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: ceph version 19.0.0-3830-g3b748ea0 (3b748ea08891006c1152513d25ab4a03f7c9a995) squid (dev), process ceph-mon, pid 2 2024-05-21T21:44:12.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: pidfile_write: ignore empty --pid-file 2024-05-21T21:44:12.045 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: load: jerasure load: lrc 2024-05-21T21:44:12.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: RocksDB version: 7.9.2 2024-05-21T21:44:12.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Git sha 0 2024-05-21T21:44:12.537 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Compile date 2024-05-20 21:53:18 2024-05-21T21:44:12.537 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: DB SUMMARY 2024-05-21T21:44:12.537 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: DB Session ID: II3CRQW0TA9B7084I9VE 2024-05-21T21:44:12.537 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: CURRENT file: CURRENT 2024-05-21T21:44:12.537 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: IDENTITY file: IDENTITY 2024-05-21T21:44:12.537 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-05-21T21:44:12.537 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-05-21T21:44:12.537 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-05-21T21:44:12.537 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.error_if_exists: 0 2024-05-21T21:44:12.537 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.create_if_missing: 0 2024-05-21T21:44:12.537 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.paranoid_checks: 1 2024-05-21T21:44:12.537 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-21T21:44:12.538 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-21T21:44:12.538 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-21T21:44:12.538 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.env: 0x55d6a3a57cc0 2024-05-21T21:44:12.538 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.fs: PosixFileSystem 2024-05-21T21:44:12.538 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.info_log: 0x55d6a4e3bda0 2024-05-21T21:44:12.538 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.max_file_opening_threads: 16 2024-05-21T21:44:12.538 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.statistics: (nil) 2024-05-21T21:44:12.538 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.use_fsync: 0 2024-05-21T21:44:12.538 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.max_log_file_size: 0 2024-05-21T21:44:12.538 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-21T21:44:12.538 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-21T21:44:12.538 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.keep_log_file_num: 1000 2024-05-21T21:44:12.539 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.recycle_log_file_num: 0 2024-05-21T21:44:12.539 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.allow_fallocate: 1 2024-05-21T21:44:12.539 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.allow_mmap_reads: 0 2024-05-21T21:44:12.539 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.allow_mmap_writes: 0 2024-05-21T21:44:12.539 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.use_direct_reads: 0 2024-05-21T21:44:12.539 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-21T21:44:12.539 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.create_missing_column_families: 0 2024-05-21T21:44:12.539 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.db_log_dir: 2024-05-21T21:44:12.539 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.wal_dir: 2024-05-21T21:44:12.539 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-21T21:44:12.539 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-21T21:44:12.539 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-21T21:44:12.539 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-21T21:44:12.540 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-21T21:44:12.540 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-21T21:44:12.540 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.advise_random_on_open: 1 2024-05-21T21:44:12.540 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.db_write_buffer_size: 0 2024-05-21T21:44:12.540 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.write_buffer_manager: 0x55d6a4e0d0e0 2024-05-21T21:44:12.540 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-21T21:44:12.540 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-21T21:44:12.540 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-21T21:44:12.540 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.rate_limiter: (nil) 2024-05-21T21:44:12.540 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-21T21:44:12.540 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.wal_recovery_mode: 2 2024-05-21T21:44:12.540 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.enable_thread_tracking: 0 2024-05-21T21:44:12.540 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.enable_pipelined_write: 0 2024-05-21T21:44:12.540 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.unordered_write: 0 2024-05-21T21:44:12.541 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-21T21:44:12.541 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-21T21:44:12.541 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-21T21:44:12.541 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-21T21:44:12.541 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.row_cache: None 2024-05-21T21:44:12.541 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.wal_filter: None 2024-05-21T21:44:12.541 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-21T21:44:12.541 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.allow_ingest_behind: 0 2024-05-21T21:44:12.541 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.two_write_queues: 0 2024-05-21T21:44:12.541 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.manual_wal_flush: 0 2024-05-21T21:44:12.541 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.wal_compression: 0 2024-05-21T21:44:12.541 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.atomic_flush: 0 2024-05-21T21:44:12.541 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-21T21:44:12.541 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-21T21:44:12.542 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-21T21:44:12.542 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.log_readahead_size: 0 2024-05-21T21:44:12.542 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-21T21:44:12.542 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.best_efforts_recovery: 0 2024-05-21T21:44:12.542 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-21T21:44:12.542 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-21T21:44:12.542 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.allow_data_in_errors: 0 2024-05-21T21:44:12.542 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.db_host_id: __hostname__ 2024-05-21T21:44:12.542 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-21T21:44:12.542 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.max_background_jobs: 2 2024-05-21T21:44:12.542 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.max_background_compactions: -1 2024-05-21T21:44:12.542 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.max_subcompactions: 1 2024-05-21T21:44:12.542 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-21T21:44:12.544 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-21T21:44:12.544 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-21T21:44:12.544 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.max_total_wal_size: 0 2024-05-21T21:44:12.544 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-21T21:44:12.544 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-21T21:44:12.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-21T21:44:12.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-21T21:44:12.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.max_open_files: -1 2024-05-21T21:44:12.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.bytes_per_sync: 0 2024-05-21T21:44:12.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-21T21:44:12.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-21T21:44:12.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.compaction_readahead_size: 0 2024-05-21T21:44:12.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.max_background_flushes: -1 2024-05-21T21:44:12.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Compression algorithms supported: 2024-05-21T21:44:12.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-21T21:44:12.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: kZSTD supported: 0 2024-05-21T21:44:12.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: kXpressCompression supported: 0 2024-05-21T21:44:12.545 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: kLZ4HCCompression supported: 1 2024-05-21T21:44:12.546 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: kZlibCompression supported: 1 2024-05-21T21:44:12.546 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: kSnappyCompression supported: 1 2024-05-21T21:44:12.546 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: kLZ4Compression supported: 1 2024-05-21T21:44:12.546 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: kBZip2Compression supported: 0 2024-05-21T21:44:12.546 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-21T21:44:12.546 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-21T21:44:12.546 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-05-21T21:44:12.546 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-21T21:44:12.546 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-21T21:44:12.546 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.merge_operator: 2024-05-21T21:44:12.546 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.compaction_filter: None 2024-05-21T21:44:12.546 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.compaction_filter_factory: None 2024-05-21T21:44:12.546 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.sst_partitioner_factory: None 2024-05-21T21:44:12.547 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-21T21:44:12.547 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-21T21:44:12.547 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55d6a4e3bf60) 2024-05-21T21:44:12.547 INFO:journalctl@ceph.mon.b.smithi164.stdout: cache_index_and_filter_blocks: 1 2024-05-21T21:44:12.547 INFO:journalctl@ceph.mon.b.smithi164.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-21T21:44:12.547 INFO:journalctl@ceph.mon.b.smithi164.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-21T21:44:12.547 INFO:journalctl@ceph.mon.b.smithi164.stdout: pin_top_level_index_and_filter: 1 2024-05-21T21:44:12.547 INFO:journalctl@ceph.mon.b.smithi164.stdout: index_type: 0 2024-05-21T21:44:12.547 INFO:journalctl@ceph.mon.b.smithi164.stdout: data_block_index_type: 0 2024-05-21T21:44:12.547 INFO:journalctl@ceph.mon.b.smithi164.stdout: index_shortening: 1 2024-05-21T21:44:12.547 INFO:journalctl@ceph.mon.b.smithi164.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-21T21:44:12.547 INFO:journalctl@ceph.mon.b.smithi164.stdout: checksum: 4 2024-05-21T21:44:12.547 INFO:journalctl@ceph.mon.b.smithi164.stdout: no_block_cache: 0 2024-05-21T21:44:12.547 INFO:journalctl@ceph.mon.b.smithi164.stdout: block_cache: 0x55d6a4e3f1f0 2024-05-21T21:44:12.547 INFO:journalctl@ceph.mon.b.smithi164.stdout: block_cache_name: BinnedLRUCache 2024-05-21T21:44:12.548 INFO:journalctl@ceph.mon.b.smithi164.stdout: block_cache_options: 2024-05-21T21:44:12.548 INFO:journalctl@ceph.mon.b.smithi164.stdout: capacity : 536870912 2024-05-21T21:44:12.548 INFO:journalctl@ceph.mon.b.smithi164.stdout: num_shard_bits : 4 2024-05-21T21:44:12.548 INFO:journalctl@ceph.mon.b.smithi164.stdout: strict_capacity_limit : 0 2024-05-21T21:44:12.548 INFO:journalctl@ceph.mon.b.smithi164.stdout: high_pri_pool_ratio: 0.000 2024-05-21T21:44:12.548 INFO:journalctl@ceph.mon.b.smithi164.stdout: block_cache_compressed: (nil) 2024-05-21T21:44:12.548 INFO:journalctl@ceph.mon.b.smithi164.stdout: persistent_cache: (nil) 2024-05-21T21:44:12.548 INFO:journalctl@ceph.mon.b.smithi164.stdout: block_size: 4096 2024-05-21T21:44:12.548 INFO:journalctl@ceph.mon.b.smithi164.stdout: block_size_deviation: 10 2024-05-21T21:44:12.548 INFO:journalctl@ceph.mon.b.smithi164.stdout: block_restart_interval: 16 2024-05-21T21:44:12.548 INFO:journalctl@ceph.mon.b.smithi164.stdout: index_block_restart_interval: 1 2024-05-21T21:44:12.548 INFO:journalctl@ceph.mon.b.smithi164.stdout: metadata_block_size: 4096 2024-05-21T21:44:12.548 INFO:journalctl@ceph.mon.b.smithi164.stdout: partition_filters: 0 2024-05-21T21:44:12.548 INFO:journalctl@ceph.mon.b.smithi164.stdout: use_delta_encoding: 1 2024-05-21T21:44:12.549 INFO:journalctl@ceph.mon.b.smithi164.stdout: filter_policy: bloomfilter 2024-05-21T21:44:12.549 INFO:journalctl@ceph.mon.b.smithi164.stdout: whole_key_filtering: 1 2024-05-21T21:44:12.549 INFO:journalctl@ceph.mon.b.smithi164.stdout: verify_compression: 0 2024-05-21T21:44:12.549 INFO:journalctl@ceph.mon.b.smithi164.stdout: read_amp_bytes_per_bit: 0 2024-05-21T21:44:12.549 INFO:journalctl@ceph.mon.b.smithi164.stdout: format_version: 5 2024-05-21T21:44:12.549 INFO:journalctl@ceph.mon.b.smithi164.stdout: enable_index_compression: 1 2024-05-21T21:44:12.549 INFO:journalctl@ceph.mon.b.smithi164.stdout: block_align: 0 2024-05-21T21:44:12.549 INFO:journalctl@ceph.mon.b.smithi164.stdout: max_auto_readahead_size: 262144 2024-05-21T21:44:12.549 INFO:journalctl@ceph.mon.b.smithi164.stdout: prepopulate_block_cache: 0 2024-05-21T21:44:12.549 INFO:journalctl@ceph.mon.b.smithi164.stdout: initial_auto_readahead_size: 8192 2024-05-21T21:44:12.549 INFO:journalctl@ceph.mon.b.smithi164.stdout: num_file_reads_for_auto_readahead: 2 2024-05-21T21:44:12.549 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.write_buffer_size: 33554432 2024-05-21T21:44:12.549 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.max_write_buffer_number: 2 2024-05-21T21:44:12.549 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.compression: NoCompression 2024-05-21T21:44:12.549 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.bottommost_compression: Disabled 2024-05-21T21:44:12.550 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.prefix_extractor: nullptr 2024-05-21T21:44:12.550 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-21T21:44:12.550 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.num_levels: 7 2024-05-21T21:44:12.550 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-21T21:44:12.550 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-21T21:44:12.550 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-21T21:44:12.550 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-21T21:44:12.550 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-21T21:44:12.550 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-21T21:44:12.550 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-21T21:44:12.550 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-21T21:44:12.550 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-21T21:44:12.550 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-21T21:44:12.550 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-21T21:44:12.551 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-21T21:44:12.551 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-21T21:44:12.551 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.compression_opts.level: 32767 2024-05-21T21:44:12.551 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.compression_opts.strategy: 0 2024-05-21T21:44:12.551 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-21T21:44:12.551 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-21T21:44:12.551 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-21T21:44:12.551 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-21T21:44:12.551 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.compression_opts.enabled: false 2024-05-21T21:44:12.551 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-21T21:44:12.551 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-21T21:44:12.551 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-21T21:44:12.551 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-21T21:44:12.551 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.target_file_size_base: 67108864 2024-05-21T21:44:12.552 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-21T21:44:12.552 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-21T21:44:12.552 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-21T21:44:12.552 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-21T21:44:12.552 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-21T21:44:12.552 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-21T21:44:12.552 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-21T21:44:12.552 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-21T21:44:12.552 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-21T21:44:12.552 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-21T21:44:12.552 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-21T21:44:12.552 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-21T21:44:12.552 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-21T21:44:12.552 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-21T21:44:12.552 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.arena_block_size: 1048576 2024-05-21T21:44:12.553 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-21T21:44:12.553 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-21T21:44:12.553 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.disable_auto_compactions: 0 2024-05-21T21:44:12.553 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-21T21:44:12.553 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-21T21:44:12.553 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-21T21:44:12.553 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-21T21:44:12.553 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-21T21:44:12.553 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-21T21:44:12.553 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-21T21:44:12.553 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-21T21:44:12.553 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-21T21:44:12.553 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-21T21:44:12.553 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-05-21T21:44:12.554 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.inplace_update_support: 0 2024-05-21T21:44:12.554 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-21T21:44:12.554 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-21T21:44:12.554 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-21T21:44:12.554 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-21T21:44:12.554 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.bloom_locality: 0 2024-05-21T21:44:12.554 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.max_successive_merges: 0 2024-05-21T21:44:12.554 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-21T21:44:12.554 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.paranoid_file_checks: 0 2024-05-21T21:44:12.554 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.force_consistency_checks: 1 2024-05-21T21:44:12.554 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.report_bg_io_stats: 0 2024-05-21T21:44:12.554 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.ttl: 2592000 2024-05-21T21:44:12.554 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-21T21:44:12.555 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-21T21:44:12.555 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-21T21:44:12.555 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.enable_blob_files: false 2024-05-21T21:44:12.555 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.min_blob_size: 0 2024-05-21T21:44:12.555 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.blob_file_size: 268435456 2024-05-21T21:44:12.555 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-21T21:44:12.555 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-21T21:44:12.555 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-21T21:44:12.555 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-21T21:44:12.555 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-21T21:44:12.555 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.blob_file_starting_level: 0 2024-05-21T21:44:12.555 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-21T21:44:12.555 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-05-21T21:44:12.555 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-05-21T21:44:12.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 3f5269e8-fc46-4f79-a72d-d051341944c6 2024-05-21T21:44:12.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716327852047297, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-05-21T21:44:12.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-05-21T21:44:12.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716327852047830, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1716327852, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "3f5269e8-fc46-4f79-a72d-d051341944c6", "db_session_id": "II3CRQW0TA9B7084I9VE", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-05-21T21:44:12.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716327852047927, "job": 1, "event": "recovery_finished"} 2024-05-21T21:44:12.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-05-21T21:44:12.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-21T21:44:12.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55d6a4f5c000 2024-05-21T21:44:12.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: DB pointer 0x55d6a4f48000 2024-05-21T21:44:12.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-21T21:44:12.556 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-21T21:44:12.556 INFO:journalctl@ceph.mon.b.smithi164.stdout: ** DB Stats ** 2024-05-21T21:44:12.556 INFO:journalctl@ceph.mon.b.smithi164.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-21T21:44:12.557 INFO:journalctl@ceph.mon.b.smithi164.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-21T21:44:12.557 INFO:journalctl@ceph.mon.b.smithi164.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-21T21:44:12.557 INFO:journalctl@ceph.mon.b.smithi164.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-21T21:44:12.557 INFO:journalctl@ceph.mon.b.smithi164.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-21T21:44:12.557 INFO:journalctl@ceph.mon.b.smithi164.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-21T21:44:12.557 INFO:journalctl@ceph.mon.b.smithi164.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-21T21:44:12.557 INFO:journalctl@ceph.mon.b.smithi164.stdout: 2024-05-21T21:44:12.557 INFO:journalctl@ceph.mon.b.smithi164.stdout: ** Compaction Stats [default] ** 2024-05-21T21:44:12.557 INFO:journalctl@ceph.mon.b.smithi164.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-21T21:44:12.557 INFO:journalctl@ceph.mon.b.smithi164.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-21T21:44:12.557 INFO:journalctl@ceph.mon.b.smithi164.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-21T21:44:12.557 INFO:journalctl@ceph.mon.b.smithi164.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-21T21:44:12.557 INFO:journalctl@ceph.mon.b.smithi164.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-21T21:44:12.558 INFO:journalctl@ceph.mon.b.smithi164.stdout: 2024-05-21T21:44:12.558 INFO:journalctl@ceph.mon.b.smithi164.stdout: ** Compaction Stats [default] ** 2024-05-21T21:44:12.558 INFO:journalctl@ceph.mon.b.smithi164.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-21T21:44:12.558 INFO:journalctl@ceph.mon.b.smithi164.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-21T21:44:12.558 INFO:journalctl@ceph.mon.b.smithi164.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-21T21:44:12.558 INFO:journalctl@ceph.mon.b.smithi164.stdout: 2024-05-21T21:44:12.558 INFO:journalctl@ceph.mon.b.smithi164.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-21T21:44:12.558 INFO:journalctl@ceph.mon.b.smithi164.stdout: 2024-05-21T21:44:12.558 INFO:journalctl@ceph.mon.b.smithi164.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-21T21:44:12.558 INFO:journalctl@ceph.mon.b.smithi164.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-21T21:44:12.558 INFO:journalctl@ceph.mon.b.smithi164.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-21T21:44:12.558 INFO:journalctl@ceph.mon.b.smithi164.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-21T21:44:12.558 INFO:journalctl@ceph.mon.b.smithi164.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-21T21:44:12.559 INFO:journalctl@ceph.mon.b.smithi164.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-21T21:44:12.559 INFO:journalctl@ceph.mon.b.smithi164.stdout: Cumulative compaction: 0.00 GB write, 0.56 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-21T21:44:12.559 INFO:journalctl@ceph.mon.b.smithi164.stdout: Interval compaction: 0.00 GB write, 0.56 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-21T21:44:12.559 INFO:journalctl@ceph.mon.b.smithi164.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-21T21:44:12.559 INFO:journalctl@ceph.mon.b.smithi164.stdout: Block cache BinnedLRUCache@0x55d6a4e3f1f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-05-21T21:44:12.559 INFO:journalctl@ceph.mon.b.smithi164.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-05-21T21:44:12.559 INFO:journalctl@ceph.mon.b.smithi164.stdout: 2024-05-21T21:44:12.559 INFO:journalctl@ceph.mon.b.smithi164.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-21T21:44:12.559 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-05-21T21:44:12.559 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: using public_addr v2:172.21.15.164:0/0 -> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] 2024-05-21T21:44:12.559 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: starting mon.b rank -1 at public addrs [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] at bind addrs [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 2024-05-21T21:44:12.559 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: mon.b@-1(???) e0 preinit fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 2024-05-21T21:44:12.559 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: mon.b@-1(synchronizing).mds e1 new map 2024-05-21T21:44:12.560 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: mon.b@-1(synchronizing).mds e1 print_map 2024-05-21T21:44:12.560 INFO:journalctl@ceph.mon.b.smithi164.stdout: e1 2024-05-21T21:44:12.560 INFO:journalctl@ceph.mon.b.smithi164.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-21T21:44:12.560 INFO:journalctl@ceph.mon.b.smithi164.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-21T21:44:12.560 INFO:journalctl@ceph.mon.b.smithi164.stdout: legacy client fscid: -1 2024-05-21T21:44:12.560 INFO:journalctl@ceph.mon.b.smithi164.stdout: 2024-05-21T21:44:12.560 INFO:journalctl@ceph.mon.b.smithi164.stdout: No filesystems configured 2024-05-21T21:44:12.560 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-05-21T21:44:12.560 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-05-21T21:44:12.560 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-05-21T21:44:12.560 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-05-21T21:44:12.560 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-05-21T21:44:12.560 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-05-21T21:44:12.561 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-21T21:44:12.561 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-21T21:44:12.561 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-21T21:44:12.561 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-21T21:44:12.561 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:12.561 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:12.561 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:12.561 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:44:12.561 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: Added host smithi164 2024-05-21T21:44:12.561 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:12.561 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:12.561 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:12.561 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:12.561 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:12.562 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T21:44:12.562 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:12.562 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:12.562 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:12.562 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:12.562 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:12.562 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi164", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:44:12.562 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:44:12.562 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:44:12.562 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: Updating smithi164:/etc/ceph/ceph.conf 2024-05-21T21:44:12.562 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: Updating smithi164:/var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T21:44:12.562 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: Updating smithi164:/etc/ceph/ceph.client.admin.keyring 2024-05-21T21:44:12.563 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:12.563 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: Updating smithi164:/var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-21T21:44:12.563 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi184", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:44:12.563 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:12.563 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:12.563 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:12.563 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: Deploying cephadm binary to smithi184 2024-05-21T21:44:12.563 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:12.563 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:12.563 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:12.563 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: Added host smithi184 2024-05-21T21:44:12.563 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:44:12.563 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:12.563 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:12.563 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:12.564 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:12.564 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:12.564 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T21:44:12.564 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:12.564 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:12.564 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:12.564 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:12.564 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:12.564 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi184", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:44:12.564 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:44:12.564 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:44:12.564 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: Updating smithi184:/etc/ceph/ceph.conf 2024-05-21T21:44:12.564 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: Updating smithi184:/var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T21:44:12.564 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/1411402349' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-21T21:44:12.565 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: Updating smithi184:/etc/ceph/ceph.client.admin.keyring 2024-05-21T21:44:12.565 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:12.565 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:12.565 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:12.565 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: Updating smithi184:/var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-21T21:44:12.565 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:12.565 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/1411402349' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-21T21:44:12.565 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: osdmap e4: 0 total, 0 up, 0 in 2024-05-21T21:44:12.565 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:12.565 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:12.565 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:44:12.565 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:44:12.565 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:44:12.566 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:12.566 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-21T21:44:12.566 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:44:12.566 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi151:172.21.15.151=a;smithi164:172.21.15.164=b;smithi184:172.21.15.184=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:44:12.566 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: Saving service mon spec with placement smithi151:172.21.15.151=a;smithi164:172.21.15.164=b;smithi184:172.21.15.184=c;count:3 2024-05-21T21:44:12.566 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: Deploying daemon mon.c on smithi184 2024-05-21T21:44:12.566 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:12.566 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:12.566 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-21T21:44:12.566 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: mon.a calling monitor election 2024-05-21T21:44:12.566 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T21:44:12.566 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: from='client.? 172.21.15.184:0/3177859394' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-21T21:44:12.566 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:12.567 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T21:44:12.567 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T21:44:12.567 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: mon.c calling monitor election 2024-05-21T21:44:12.567 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:12.567 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T21:44:12.567 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T21:44:12.567 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:12.567 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T21:44:12.567 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-21T21:44:12.567 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: monmap epoch 2 2024-05-21T21:44:12.567 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 2024-05-21T21:44:12.567 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: last_changed 2024-05-21T21:44:02.958032+0000 2024-05-21T21:44:12.567 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: created 2024-05-21T21:41:48.311542+0000 2024-05-21T21:44:12.568 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: min_mon_release 19 (squid) 2024-05-21T21:44:12.568 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: election_strategy: 1 2024-05-21T21:44:12.568 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: 0: [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] mon.a 2024-05-21T21:44:12.568 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: 1: [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] mon.c 2024-05-21T21:44:12.568 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: fsmap 2024-05-21T21:44:12.568 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: osdmap e4: 0 total, 0 up, 0 in 2024-05-21T21:44:12.568 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: mgrmap e13: x(active, since 58s) 2024-05-21T21:44:12.568 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: overall HEALTH_OK 2024-05-21T21:44:12.568 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:12.568 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:12.568 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:12.568 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-21T21:44:12.568 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:44:12.568 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: Deploying daemon mon.b on smithi164 2024-05-21T21:44:12.568 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T21:44:12.569 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:12.569 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 ceph-mon[36269]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-05-21T21:44:12.569 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 bash[36250]: 921034200eb5664d9bfa4bbe36821e5483583cfc91526f16e6499d118ba2ac61 2024-05-21T21:44:12.569 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:12 smithi164 systemd[1]: Started Ceph mon.b for c8c7a2e6-17ba-11ef-bc9a-c7b262605968. 2024-05-21T21:44:13.204 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-21T21:44:13.204 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph mon dump -f json 2024-05-21T21:44:13.446 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.c/config 2024-05-21T21:44:17.454 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:17 smithi151 ceph-mon[31623]: mon.c calling monitor election 2024-05-21T21:44:17.454 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:17 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-21T21:44:17.454 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:17 smithi151 ceph-mon[31623]: mon.a calling monitor election 2024-05-21T21:44:17.454 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:17 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T21:44:17.454 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:17 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T21:44:17.454 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:17 smithi151 ceph-mon[31623]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:17.454 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:17 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T21:44:17.455 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:17 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T21:44:17.455 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:17 smithi151 ceph-mon[31623]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:17.455 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:17 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T21:44:17.455 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:17 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T21:44:17.455 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:17 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T21:44:17.455 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:17 smithi151 ceph-mon[31623]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-21T21:44:17.455 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:17 smithi151 ceph-mon[31623]: monmap epoch 3 2024-05-21T21:44:17.455 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:17 smithi151 ceph-mon[31623]: fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 2024-05-21T21:44:17.455 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:17 smithi151 ceph-mon[31623]: last_changed 2024-05-21T21:44:12.072469+0000 2024-05-21T21:44:17.455 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:17 smithi151 ceph-mon[31623]: created 2024-05-21T21:41:48.311542+0000 2024-05-21T21:44:17.455 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:17 smithi151 ceph-mon[31623]: min_mon_release 19 (squid) 2024-05-21T21:44:17.455 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:17 smithi151 ceph-mon[31623]: election_strategy: 1 2024-05-21T21:44:17.455 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:17 smithi151 ceph-mon[31623]: 0: [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] mon.a 2024-05-21T21:44:17.456 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:17 smithi151 ceph-mon[31623]: 1: [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] mon.c 2024-05-21T21:44:17.456 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:17 smithi151 ceph-mon[31623]: 2: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.b 2024-05-21T21:44:17.456 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:17 smithi151 ceph-mon[31623]: fsmap 2024-05-21T21:44:17.456 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:17 smithi151 ceph-mon[31623]: osdmap e4: 0 total, 0 up, 0 in 2024-05-21T21:44:17.456 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:17 smithi151 ceph-mon[31623]: mgrmap e13: x(active, since 68s) 2024-05-21T21:44:17.456 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:17 smithi151 ceph-mon[31623]: overall HEALTH_OK 2024-05-21T21:44:17.456 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:17 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:17.456 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:17 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:17.456 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:17 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:17.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:17 smithi184 ceph-mon[36942]: mon.c calling monitor election 2024-05-21T21:44:17.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:17 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-21T21:44:17.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:17 smithi184 ceph-mon[36942]: mon.a calling monitor election 2024-05-21T21:44:17.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:17 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T21:44:17.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:17 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T21:44:17.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:17 smithi184 ceph-mon[36942]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:17.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:17 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T21:44:17.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:17 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T21:44:17.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:17 smithi184 ceph-mon[36942]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:17.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:17 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T21:44:17.487 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:17 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T21:44:17.487 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:17 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T21:44:17.487 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:17 smithi184 ceph-mon[36942]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-21T21:44:17.487 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:17 smithi184 ceph-mon[36942]: monmap epoch 3 2024-05-21T21:44:17.487 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:17 smithi184 ceph-mon[36942]: fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 2024-05-21T21:44:17.487 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:17 smithi184 ceph-mon[36942]: last_changed 2024-05-21T21:44:12.072469+0000 2024-05-21T21:44:17.487 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:17 smithi184 ceph-mon[36942]: created 2024-05-21T21:41:48.311542+0000 2024-05-21T21:44:17.487 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:17 smithi184 ceph-mon[36942]: min_mon_release 19 (squid) 2024-05-21T21:44:17.487 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:17 smithi184 ceph-mon[36942]: election_strategy: 1 2024-05-21T21:44:17.487 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:17 smithi184 ceph-mon[36942]: 0: [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] mon.a 2024-05-21T21:44:17.487 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:17 smithi184 ceph-mon[36942]: 1: [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] mon.c 2024-05-21T21:44:17.487 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:17 smithi184 ceph-mon[36942]: 2: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.b 2024-05-21T21:44:17.487 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:17 smithi184 ceph-mon[36942]: fsmap 2024-05-21T21:44:17.487 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:17 smithi184 ceph-mon[36942]: osdmap e4: 0 total, 0 up, 0 in 2024-05-21T21:44:17.488 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:17 smithi184 ceph-mon[36942]: mgrmap e13: x(active, since 68s) 2024-05-21T21:44:17.488 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:17 smithi184 ceph-mon[36942]: overall HEALTH_OK 2024-05-21T21:44:17.488 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:17 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:17.488 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:17 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:17.488 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:17 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:17.818 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:17.817+0000 7f0fc0c8a700 1 -- 172.21.15.184:0/4225960409 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fac011470 con 0x7f0fa80035c0 2024-05-21T21:44:17.824 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:17.818+0000 7f0fc3ef0700 1 -- 172.21.15.184:0/4225960409 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0fa80035c0 msgr2=0x7f0fa8005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:44:17.824 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:17.818+0000 7f0fc3ef0700 1 --2- 172.21.15.184:0/4225960409 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0fa80035c0 0x7f0fa8005a50 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc102f90 tx=0x7f0fac00feb0 comp rx=0 tx=0).stop 2024-05-21T21:44:17.824 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:17.818+0000 7f0fc3ef0700 1 -- 172.21.15.184:0/4225960409 shutdown_connections 2024-05-21T21:44:17.824 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:17.818+0000 7f0fc3ef0700 1 --2- 172.21.15.184:0/4225960409 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0fa80035c0 0x7f0fa8005a50 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:44:17.824 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:17.818+0000 7f0fc3ef0700 1 -- 172.21.15.184:0/4225960409 >> 172.21.15.184:0/4225960409 conn(0x7f0fbc0feb50 msgr2=0x7f0fbc100f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:44:17.824 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:17.818+0000 7f0fc3ef0700 1 -- 172.21.15.184:0/4225960409 shutdown_connections 2024-05-21T21:44:17.824 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:17.818+0000 7f0fc3ef0700 1 -- 172.21.15.184:0/4225960409 wait complete. 2024-05-21T21:44:17.824 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:17.819+0000 7f0fc3ef0700 1 Processor -- start 2024-05-21T21:44:17.824 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:17.819+0000 7f0fc3ef0700 1 -- start start 2024-05-21T21:44:17.824 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:17.820+0000 7f0fc3ef0700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0fa80035c0 0x7f0fbc19e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:44:17.824 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:17.820+0000 7f0fc3ef0700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0fbc19edf0 0x7f0fbc198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:44:17.824 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:17.820+0000 7f0fc3ef0700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0fbc199000 0x7f0fbc1994b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:44:17.824 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:17.820+0000 7f0fc3ef0700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0fbc10af50 con 0x7f0fbc199000 2024-05-21T21:44:17.825 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:17.820+0000 7f0fc3ef0700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0fbc10add0 con 0x7f0fbc19edf0 2024-05-21T21:44:17.825 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:17.820+0000 7f0fc3ef0700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0fbc10b0d0 con 0x7f0fa80035c0 2024-05-21T21:44:17.825 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:17.820+0000 7f0fc1c8c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0fa80035c0 0x7f0fbc19e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:44:17.825 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:17.820+0000 7f0fc1c8c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0fa80035c0 0x7f0fbc19e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:59734/0 (socket says 172.21.15.184:59734) 2024-05-21T21:44:17.825 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:17.820+0000 7f0fc1c8c700 1 -- 172.21.15.184:0/2344664664 learned_addr learned my addr 172.21.15.184:0/2344664664 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-05-21T21:44:17.825 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:17.820+0000 7f0fc1c8c700 1 -- 172.21.15.184:0/2344664664 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0fa80035c0 msgr2=0x7f0fbc19e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-05-21T21:44:17.825 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:17.820+0000 7f0fc1c8c700 1 -- 172.21.15.184:0/2344664664 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0fa80035c0 msgr2=0x7f0fbc19e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-21T21:44:17.825 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:17.820+0000 7f0fc1c8c700 1 --2- 172.21.15.184:0/2344664664 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0fa80035c0 0x7f0fbc19e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-21T21:44:17.825 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:17.820+0000 7f0fc148b700 1 --2- 172.21.15.184:0/2344664664 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0fbc19edf0 0x7f0fbc198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:44:17.825 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:17.820+0000 7f0fc1c8c700 1 --2- 172.21.15.184:0/2344664664 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0fa80035c0 0x7f0fbc19e8b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-21T21:44:17.825 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:17.820+0000 7f0fc248d700 1 --2- 172.21.15.184:0/2344664664 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0fbc199000 0x7f0fbc1994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:44:17.825 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:17.820+0000 7f0fc148b700 1 -- 172.21.15.184:0/2344664664 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0fbc19edf0 msgr2=0x7f0fbc198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-05-21T21:44:17.825 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:17.820+0000 7f0fc148b700 1 -- 172.21.15.184:0/2344664664 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0fbc19edf0 msgr2=0x7f0fbc198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-21T21:44:17.826 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:17.820+0000 7f0fc148b700 1 --2- 172.21.15.184:0/2344664664 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0fbc19edf0 0x7f0fbc198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-21T21:44:17.826 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:17.820+0000 7f0fc148b700 1 --2- 172.21.15.184:0/2344664664 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0fbc19edf0 0x7f0fbc198ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-21T21:44:17.826 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:17.821+0000 7f0fc248d700 1 -- 172.21.15.184:0/2344664664 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0fa80035c0 msgr2=0x7f0fbc19e8b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-21T21:44:17.826 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:17.821+0000 7f0fc248d700 1 --2- 172.21.15.184:0/2344664664 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0fa80035c0 0x7f0fbc19e8b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:44:17.826 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:17.821+0000 7f0fc248d700 1 -- 172.21.15.184:0/2344664664 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0fbc19edf0 msgr2=0x7f0fbc198ac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-21T21:44:17.826 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:17.821+0000 7f0fc248d700 1 --2- 172.21.15.184:0/2344664664 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0fbc19edf0 0x7f0fbc198ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:44:17.826 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:17.821+0000 7f0fc248d700 1 -- 172.21.15.184:0/2344664664 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fbc1a5780 con 0x7f0fbc199000 2024-05-21T21:44:17.826 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:17.821+0000 7f0fc248d700 1 --2- 172.21.15.184:0/2344664664 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0fbc199000 0x7f0fbc1994b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0fb400bd50 tx=0x7f0fb400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:44:17.826 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:17.821+0000 7f0fb2ffd700 1 -- 172.21.15.184:0/2344664664 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb4019070 con 0x7f0fbc199000 2024-05-21T21:44:17.826 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:17.821+0000 7f0fc3ef0700 1 -- 172.21.15.184:0/2344664664 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fbc1a59b0 con 0x7f0fbc199000 2024-05-21T21:44:17.826 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:17.821+0000 7f0fc3ef0700 1 -- 172.21.15.184:0/2344664664 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fbc1a5f40 con 0x7f0fbc199000 2024-05-21T21:44:17.826 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:17.821+0000 7f0fb2ffd700 1 -- 172.21.15.184:0/2344664664 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0fb400cd10 con 0x7f0fbc199000 2024-05-21T21:44:17.826 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:17.821+0000 7f0fb2ffd700 1 -- 172.21.15.184:0/2344664664 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb4023060 con 0x7f0fbc199000 2024-05-21T21:44:17.827 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:17.823+0000 7f0fc3ef0700 1 -- 172.21.15.184:0/2344664664 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fa8002160 con 0x7f0fbc199000 2024-05-21T21:44:17.827 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:17.823+0000 7f0fb2ffd700 1 -- 172.21.15.184:0/2344664664 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7f0fb401c070 con 0x7f0fbc199000 2024-05-21T21:44:17.827 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:17.823+0000 7f0fb2ffd700 1 --2- 172.21.15.184:0/2344664664 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f0fa003e9a0 0x7f0fa0040e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:44:17.827 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:17.823+0000 7f0fb2ffd700 1 -- 172.21.15.184:0/2344664664 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f0fb4051f80 con 0x7f0fbc199000 2024-05-21T21:44:17.827 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:17.823+0000 7f0fc1c8c700 1 --2- 172.21.15.184:0/2344664664 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f0fa003e9a0 0x7f0fa0040e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:44:17.827 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:17.824+0000 7f0fc1c8c700 1 --2- 172.21.15.184:0/2344664664 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f0fa003e9a0 0x7f0fa0040e60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0fac005b40 tx=0x7f0fac005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:44:17.839 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:17.839+0000 7f0fb2ffd700 1 -- 172.21.15.184:0/2344664664 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0fb4014bf0 con 0x7f0fbc199000 2024-05-21T21:44:18.224 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:18.224+0000 7f0fc3ef0700 1 -- 172.21.15.184:0/2344664664 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f0fa8002c50 con 0x7f0fbc199000 2024-05-21T21:44:18.225 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:18.225+0000 7f0fb2ffd700 1 -- 172.21.15.184:0/2344664664 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1287 (secure 0 0 0) 0x7f0fb4010200 con 0x7f0fbc199000 2024-05-21T21:44:18.226 INFO:teuthology.orchestra.run.smithi184.stdout: 2024-05-21T21:44:18.226 INFO:teuthology.orchestra.run.smithi184.stdout:{"epoch":3,"fsid":"c8c7a2e6-17ba-11ef-bc9a-c7b262605968","modified":"2024-05-21T21:44:12.072469Z","created":"2024-05-21T21:41:48.311542Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:3300","nonce":0},{"type":"v1","addr":"172.21.15.151:6789","nonce":0}]},"addr":"172.21.15.151:6789/0","public_addr":"172.21.15.151:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:3300","nonce":0},{"type":"v1","addr":"172.21.15.184:6789","nonce":0}]},"addr":"172.21.15.184:6789/0","public_addr":"172.21.15.184:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:3300","nonce":0},{"type":"v1","addr":"172.21.15.164:6789","nonce":0}]},"addr":"172.21.15.164:6789/0","public_addr":"172.21.15.164:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-21T21:44:18.228 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:18.228+0000 7f0fc3ef0700 1 -- 172.21.15.184:0/2344664664 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f0fa003e9a0 msgr2=0x7f0fa0040e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:44:18.228 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:18.228+0000 7f0fc3ef0700 1 --2- 172.21.15.184:0/2344664664 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f0fa003e9a0 0x7f0fa0040e60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0fac005b40 tx=0x7f0fac005a90 comp rx=0 tx=0).stop 2024-05-21T21:44:18.228 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:18.228+0000 7f0fc3ef0700 1 -- 172.21.15.184:0/2344664664 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0fbc199000 msgr2=0x7f0fbc1994b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:44:18.228 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:18.228+0000 7f0fc3ef0700 1 --2- 172.21.15.184:0/2344664664 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0fbc199000 0x7f0fbc1994b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0fb400bd50 tx=0x7f0fb400be30 comp rx=0 tx=0).stop 2024-05-21T21:44:18.228 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:18.228+0000 7f0fc3ef0700 1 -- 172.21.15.184:0/2344664664 shutdown_connections 2024-05-21T21:44:18.229 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:18.228+0000 7f0fc3ef0700 1 --2- 172.21.15.184:0/2344664664 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f0fa003e9a0 0x7f0fa0040e60 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:44:18.229 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:18.228+0000 7f0fc3ef0700 1 --2- 172.21.15.184:0/2344664664 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0fbc199000 0x7f0fbc1994b0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:44:18.229 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:18.228+0000 7f0fc3ef0700 1 --2- 172.21.15.184:0/2344664664 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0fa80035c0 0x7f0fbc19e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:44:18.229 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:18.228+0000 7f0fc3ef0700 1 --2- 172.21.15.184:0/2344664664 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0fbc19edf0 0x7f0fbc198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:44:18.229 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:18.228+0000 7f0fc3ef0700 1 -- 172.21.15.184:0/2344664664 >> 172.21.15.184:0/2344664664 conn(0x7f0fbc0feb50 msgr2=0x7f0fbc0ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:44:18.229 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:18.228+0000 7f0fc3ef0700 1 -- 172.21.15.184:0/2344664664 shutdown_connections 2024-05-21T21:44:18.229 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:18.228+0000 7f0fc3ef0700 1 -- 172.21.15.184:0/2344664664 wait complete. 2024-05-21T21:44:18.229 INFO:teuthology.orchestra.run.smithi184.stderr:dumped monmap epoch 3 2024-05-21T21:44:18.418 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:18 smithi184 ceph-mon[36942]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:18.418 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:18 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:18.418 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:18 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:44:18.418 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:18 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T21:44:18.452 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:18 smithi151 ceph-mon[31623]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:18.452 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:18 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:18.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:18 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:44:18.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:18 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T21:44:19.370 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-05-21T21:44:19.371 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph config generate-minimal-conf 2024-05-21T21:44:19.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:19 smithi164 ceph-mon[36269]: mon.c calling monitor election 2024-05-21T21:44:19.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:19 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-21T21:44:19.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:19 smithi164 ceph-mon[36269]: mon.a calling monitor election 2024-05-21T21:44:19.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:19 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T21:44:19.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:19 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T21:44:19.537 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:19 smithi164 ceph-mon[36269]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:19.537 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:19 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T21:44:19.537 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:19 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T21:44:19.537 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:19 smithi164 ceph-mon[36269]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:19.537 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:19 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T21:44:19.537 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:19 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T21:44:19.537 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:19 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T21:44:19.537 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:19 smithi164 ceph-mon[36269]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-21T21:44:19.537 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:19 smithi164 ceph-mon[36269]: monmap epoch 3 2024-05-21T21:44:19.537 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:19 smithi164 ceph-mon[36269]: fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 2024-05-21T21:44:19.537 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:19 smithi164 ceph-mon[36269]: last_changed 2024-05-21T21:44:12.072469+0000 2024-05-21T21:44:19.537 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:19 smithi164 ceph-mon[36269]: created 2024-05-21T21:41:48.311542+0000 2024-05-21T21:44:19.537 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:19 smithi164 ceph-mon[36269]: min_mon_release 19 (squid) 2024-05-21T21:44:19.538 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:19 smithi164 ceph-mon[36269]: election_strategy: 1 2024-05-21T21:44:19.538 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:19 smithi164 ceph-mon[36269]: 0: [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] mon.a 2024-05-21T21:44:19.538 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:19 smithi164 ceph-mon[36269]: 1: [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] mon.c 2024-05-21T21:44:19.538 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:19 smithi164 ceph-mon[36269]: 2: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.b 2024-05-21T21:44:19.538 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:19 smithi164 ceph-mon[36269]: fsmap 2024-05-21T21:44:19.538 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:19 smithi164 ceph-mon[36269]: osdmap e4: 0 total, 0 up, 0 in 2024-05-21T21:44:19.538 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:19 smithi164 ceph-mon[36269]: mgrmap e13: x(active, since 68s) 2024-05-21T21:44:19.538 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:19 smithi164 ceph-mon[36269]: overall HEALTH_OK 2024-05-21T21:44:19.538 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:19 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:19.538 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:19 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:19.538 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:19 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:19.538 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:19 smithi164 ceph-mon[36269]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:19.538 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:19 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:19.539 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:19 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:44:19.539 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:19 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T21:44:19.788 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:44:20.155 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:19 smithi151 ceph-mon[31623]: mon.b calling monitor election 2024-05-21T21:44:20.156 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:19 smithi151 ceph-mon[31623]: Updating smithi151:/etc/ceph/ceph.conf 2024-05-21T21:44:20.156 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:19 smithi151 ceph-mon[31623]: Updating smithi164:/etc/ceph/ceph.conf 2024-05-21T21:44:20.156 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:19 smithi151 ceph-mon[31623]: Updating smithi184:/etc/ceph/ceph.conf 2024-05-21T21:44:20.156 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:19 smithi151 ceph-mon[31623]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:20.156 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:19 smithi151 ceph-mon[31623]: mon.b calling monitor election 2024-05-21T21:44:20.156 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:19 smithi151 ceph-mon[31623]: mon.c calling monitor election 2024-05-21T21:44:20.156 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:19 smithi151 ceph-mon[31623]: mon.a calling monitor election 2024-05-21T21:44:20.156 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:19 smithi151 ceph-mon[31623]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-21T21:44:20.156 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:19 smithi151 ceph-mon[31623]: monmap epoch 3 2024-05-21T21:44:20.156 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:19 smithi151 ceph-mon[31623]: fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 2024-05-21T21:44:20.156 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:19 smithi151 ceph-mon[31623]: last_changed 2024-05-21T21:44:12.072469+0000 2024-05-21T21:44:20.157 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:19 smithi151 ceph-mon[31623]: created 2024-05-21T21:41:48.311542+0000 2024-05-21T21:44:20.157 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:19 smithi151 ceph-mon[31623]: min_mon_release 19 (squid) 2024-05-21T21:44:20.157 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:19 smithi151 ceph-mon[31623]: election_strategy: 1 2024-05-21T21:44:20.157 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:19 smithi151 ceph-mon[31623]: 0: [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] mon.a 2024-05-21T21:44:20.157 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:19 smithi151 ceph-mon[31623]: 1: [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] mon.c 2024-05-21T21:44:20.157 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:19 smithi151 ceph-mon[31623]: 2: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.b 2024-05-21T21:44:20.157 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:19 smithi151 ceph-mon[31623]: fsmap 2024-05-21T21:44:20.157 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:19 smithi151 ceph-mon[31623]: osdmap e4: 0 total, 0 up, 0 in 2024-05-21T21:44:20.157 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:19 smithi151 ceph-mon[31623]: mgrmap e13: x(active, since 70s) 2024-05-21T21:44:20.157 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:19 smithi151 ceph-mon[31623]: overall HEALTH_OK 2024-05-21T21:44:20.157 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:19 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:20.157 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:19 smithi151 ceph-mon[31623]: Updating smithi164:/var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T21:44:20.157 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:19 smithi151 ceph-mon[31623]: Updating smithi184:/var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T21:44:20.157 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:19 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:20.158 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:19 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:20.158 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:19 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:20.158 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:19 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:20.158 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:19 smithi151 ceph-mon[31623]: Updating smithi151:/var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T21:44:20.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:19 smithi184 ceph-mon[36942]: mon.b calling monitor election 2024-05-21T21:44:20.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:19 smithi184 ceph-mon[36942]: Updating smithi151:/etc/ceph/ceph.conf 2024-05-21T21:44:20.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:19 smithi184 ceph-mon[36942]: Updating smithi164:/etc/ceph/ceph.conf 2024-05-21T21:44:20.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:19 smithi184 ceph-mon[36942]: Updating smithi184:/etc/ceph/ceph.conf 2024-05-21T21:44:20.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:19 smithi184 ceph-mon[36942]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:20.237 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:19 smithi184 ceph-mon[36942]: mon.b calling monitor election 2024-05-21T21:44:20.237 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:19 smithi184 ceph-mon[36942]: mon.c calling monitor election 2024-05-21T21:44:20.237 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:19 smithi184 ceph-mon[36942]: mon.a calling monitor election 2024-05-21T21:44:20.237 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:19 smithi184 ceph-mon[36942]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-21T21:44:20.237 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:19 smithi184 ceph-mon[36942]: monmap epoch 3 2024-05-21T21:44:20.237 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:19 smithi184 ceph-mon[36942]: fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 2024-05-21T21:44:20.237 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:19 smithi184 ceph-mon[36942]: last_changed 2024-05-21T21:44:12.072469+0000 2024-05-21T21:44:20.237 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:19 smithi184 ceph-mon[36942]: created 2024-05-21T21:41:48.311542+0000 2024-05-21T21:44:20.237 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:19 smithi184 ceph-mon[36942]: min_mon_release 19 (squid) 2024-05-21T21:44:20.237 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:19 smithi184 ceph-mon[36942]: election_strategy: 1 2024-05-21T21:44:20.237 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:19 smithi184 ceph-mon[36942]: 0: [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] mon.a 2024-05-21T21:44:20.237 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:19 smithi184 ceph-mon[36942]: 1: [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] mon.c 2024-05-21T21:44:20.237 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:19 smithi184 ceph-mon[36942]: 2: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.b 2024-05-21T21:44:20.238 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:19 smithi184 ceph-mon[36942]: fsmap 2024-05-21T21:44:20.238 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:19 smithi184 ceph-mon[36942]: osdmap e4: 0 total, 0 up, 0 in 2024-05-21T21:44:20.238 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:19 smithi184 ceph-mon[36942]: mgrmap e13: x(active, since 70s) 2024-05-21T21:44:20.238 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:19 smithi184 ceph-mon[36942]: overall HEALTH_OK 2024-05-21T21:44:20.238 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:19 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:20.238 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:19 smithi184 ceph-mon[36942]: Updating smithi164:/var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T21:44:20.238 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:19 smithi184 ceph-mon[36942]: Updating smithi184:/var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T21:44:20.238 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:19 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:20.238 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:19 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:20.238 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:19 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:20.238 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:19 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:20.238 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:19 smithi184 ceph-mon[36942]: Updating smithi151:/var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T21:44:20.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:19 smithi164 ceph-mon[36269]: mon.b calling monitor election 2024-05-21T21:44:20.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:19 smithi164 ceph-mon[36269]: Updating smithi151:/etc/ceph/ceph.conf 2024-05-21T21:44:20.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:19 smithi164 ceph-mon[36269]: Updating smithi164:/etc/ceph/ceph.conf 2024-05-21T21:44:20.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:19 smithi164 ceph-mon[36269]: Updating smithi184:/etc/ceph/ceph.conf 2024-05-21T21:44:20.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:19 smithi164 ceph-mon[36269]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:20.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:19 smithi164 ceph-mon[36269]: mon.b calling monitor election 2024-05-21T21:44:20.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:19 smithi164 ceph-mon[36269]: mon.c calling monitor election 2024-05-21T21:44:20.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:19 smithi164 ceph-mon[36269]: mon.a calling monitor election 2024-05-21T21:44:20.287 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:19 smithi164 ceph-mon[36269]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-21T21:44:20.287 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:19 smithi164 ceph-mon[36269]: monmap epoch 3 2024-05-21T21:44:20.287 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:19 smithi164 ceph-mon[36269]: fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 2024-05-21T21:44:20.287 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:19 smithi164 ceph-mon[36269]: last_changed 2024-05-21T21:44:12.072469+0000 2024-05-21T21:44:20.287 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:19 smithi164 ceph-mon[36269]: created 2024-05-21T21:41:48.311542+0000 2024-05-21T21:44:20.287 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:19 smithi164 ceph-mon[36269]: min_mon_release 19 (squid) 2024-05-21T21:44:20.287 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:19 smithi164 ceph-mon[36269]: election_strategy: 1 2024-05-21T21:44:20.287 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:19 smithi164 ceph-mon[36269]: 0: [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] mon.a 2024-05-21T21:44:20.287 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:19 smithi164 ceph-mon[36269]: 1: [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] mon.c 2024-05-21T21:44:20.287 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:19 smithi164 ceph-mon[36269]: 2: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.b 2024-05-21T21:44:20.287 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:19 smithi164 ceph-mon[36269]: fsmap 2024-05-21T21:44:20.287 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:19 smithi164 ceph-mon[36269]: osdmap e4: 0 total, 0 up, 0 in 2024-05-21T21:44:20.287 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:19 smithi164 ceph-mon[36269]: mgrmap e13: x(active, since 70s) 2024-05-21T21:44:20.288 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:19 smithi164 ceph-mon[36269]: overall HEALTH_OK 2024-05-21T21:44:20.288 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:19 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:20.288 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:19 smithi164 ceph-mon[36269]: Updating smithi164:/var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T21:44:20.288 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:19 smithi164 ceph-mon[36269]: Updating smithi184:/var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T21:44:20.288 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:19 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:20.288 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:19 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:20.288 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:19 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:20.288 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:19 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:20.288 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:19 smithi164 ceph-mon[36269]: Updating smithi151:/var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T21:44:21.062 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:20 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T21:44:21.062 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:20 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:21.062 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:20 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:21.063 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:20 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:21.063 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:20 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:21.063 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:20 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:21.063 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:20 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:21.063 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:20 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:21.063 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:20 smithi151 ceph-mon[31623]: Reconfiguring mon.a (unknown last config time)... 2024-05-21T21:44:21.063 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:20 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-21T21:44:21.063 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:20 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-21T21:44:21.063 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:20 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:44:21.063 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:20 smithi151 ceph-mon[31623]: Reconfiguring daemon mon.a on smithi151 2024-05-21T21:44:21.140 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:20 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T21:44:21.140 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:20 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:21.140 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:20 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:21.140 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:20 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:21.140 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:20 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:21.140 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:20 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:21.140 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:20 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:21.141 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:20 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:21.141 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:20 smithi184 ceph-mon[36942]: Reconfiguring mon.a (unknown last config time)... 2024-05-21T21:44:21.141 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:20 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-21T21:44:21.141 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:20 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-21T21:44:21.141 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:20 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:44:21.141 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:20 smithi184 ceph-mon[36942]: Reconfiguring daemon mon.a on smithi151 2024-05-21T21:44:21.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:20 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T21:44:21.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:20 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:21.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:20 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:21.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:20 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:21.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:20 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:21.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:20 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:21.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:20 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:21.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:20 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:21.287 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:20 smithi164 ceph-mon[36269]: Reconfiguring mon.a (unknown last config time)... 2024-05-21T21:44:21.287 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:20 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-21T21:44:21.287 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:20 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-21T21:44:21.287 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:20 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:44:21.287 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:20 smithi164 ceph-mon[36269]: Reconfiguring daemon mon.a on smithi151 2024-05-21T21:44:21.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:21.917+0000 7f23cda00700 1 -- 172.21.15.151:0/641570306 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f23c8075970 msgr2=0x7f23c8075d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:44:21.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:21.917+0000 7f23cda00700 1 --2- 172.21.15.151:0/641570306 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f23c8075970 0x7f23c8075d50 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f23b8009910 tx=0x7f23b8009c20 comp rx=0 tx=0).stop 2024-05-21T21:44:21.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:21.918+0000 7f23cda00700 1 -- 172.21.15.151:0/641570306 shutdown_connections 2024-05-21T21:44:21.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:21.918+0000 7f23cda00700 1 --2- 172.21.15.151:0/641570306 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f23c8075970 0x7f23c8075d50 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:44:21.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:21.918+0000 7f23cda00700 1 -- 172.21.15.151:0/641570306 >> 172.21.15.151:0/641570306 conn(0x7f23c80fea30 msgr2=0x7f23c8100e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:44:21.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:21.918+0000 7f23cda00700 1 -- 172.21.15.151:0/641570306 shutdown_connections 2024-05-21T21:44:21.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:21.918+0000 7f23cda00700 1 -- 172.21.15.151:0/641570306 wait complete. 2024-05-21T21:44:21.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:21.919+0000 7f23cda00700 1 Processor -- start 2024-05-21T21:44:21.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:21.919+0000 7f23cda00700 1 -- start start 2024-05-21T21:44:21.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:21.919+0000 7f23cda00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f23c8075970 0x7f23c819aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:44:21.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:21.919+0000 7f23cda00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f23c819b410 0x7f23c819f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:44:21.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:21.919+0000 7f23cda00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f23c819fc90 0x7f23c81a0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:44:21.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:21.919+0000 7f23cda00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f23c810f680 con 0x7f23c819b410 2024-05-21T21:44:21.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:21.919+0000 7f23cda00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f23c810f500 con 0x7f23c819fc90 2024-05-21T21:44:21.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:21.919+0000 7f23cda00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f23c810f800 con 0x7f23c8075970 2024-05-21T21:44:21.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:21.919+0000 7f23c6a00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f23c8075970 0x7f23c819aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:44:21.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:21.919+0000 7f23c6000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f23c819b410 0x7f23c819f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:44:21.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:21.919+0000 7f23c6a00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f23c8075970 0x7f23c819aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:39900/0 (socket says 172.21.15.151:39900) 2024-05-21T21:44:21.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:21.919+0000 7f23c6a00700 1 -- 172.21.15.151:0/948793130 learned_addr learned my addr 172.21.15.151:0/948793130 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:44:21.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:21.920+0000 7f23c7400700 1 --2- 172.21.15.151:0/948793130 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f23c819fc90 0x7f23c81a0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:44:21.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:21.920+0000 7f23c6a00700 1 -- 172.21.15.151:0/948793130 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f23c819fc90 msgr2=0x7f23c81a0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:44:21.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:21.920+0000 7f23c6a00700 1 --2- 172.21.15.151:0/948793130 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f23c819fc90 0x7f23c81a0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:44:21.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:21.920+0000 7f23c6a00700 1 -- 172.21.15.151:0/948793130 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f23c819b410 msgr2=0x7f23c819f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:44:21.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:21.920+0000 7f23c6a00700 1 --2- 172.21.15.151:0/948793130 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f23c819b410 0x7f23c819f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:44:21.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:21.920+0000 7f23c6a00700 1 -- 172.21.15.151:0/948793130 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23c81a7a00 con 0x7f23c8075970 2024-05-21T21:44:21.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:21.920+0000 7f23c6a00700 1 --2- 172.21.15.151:0/948793130 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f23c8075970 0x7f23c819aed0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f23b8004f80 tx=0x7f23b8004fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:44:21.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:21.920+0000 7f23bf400700 1 -- 172.21.15.151:0/948793130 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b8026070 con 0x7f23c8075970 2024-05-21T21:44:21.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:21.920+0000 7f23bf400700 1 -- 172.21.15.151:0/948793130 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23b800b750 con 0x7f23c8075970 2024-05-21T21:44:21.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:21.920+0000 7f23bf400700 1 -- 172.21.15.151:0/948793130 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b8003950 con 0x7f23c8075970 2024-05-21T21:44:21.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:21.920+0000 7f23cda00700 1 -- 172.21.15.151:0/948793130 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23c81a7bd0 con 0x7f23c8075970 2024-05-21T21:44:21.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:21.921+0000 7f23cda00700 1 -- 172.21.15.151:0/948793130 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23c81a8060 con 0x7f23c8075970 2024-05-21T21:44:21.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:21.921+0000 7f23bf400700 1 -- 172.21.15.151:0/948793130 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7f23b8029070 con 0x7f23c8075970 2024-05-21T21:44:21.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:21.922+0000 7f23cda00700 1 -- 172.21.15.151:0/948793130 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23c8064ba0 con 0x7f23c8075970 2024-05-21T21:44:21.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:21.922+0000 7f23bf400700 1 --2- 172.21.15.151:0/948793130 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f23ac03e9a0 0x7f23ac040e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:44:21.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:21.922+0000 7f23bf400700 1 -- 172.21.15.151:0/948793130 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f23b80533c0 con 0x7f23c8075970 2024-05-21T21:44:21.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:21.922+0000 7f23c6000700 1 --2- 172.21.15.151:0/948793130 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f23ac03e9a0 0x7f23ac040e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:44:21.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:21.923+0000 7f23c6000700 1 --2- 172.21.15.151:0/948793130 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f23ac03e9a0 0x7f23ac040e60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f23b4009ad0 tx=0x7f23b4009210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:44:21.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:21.928+0000 7f23bf400700 1 -- 172.21.15.151:0/948793130 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f23b8013070 con 0x7f23c8075970 2024-05-21T21:44:22.139 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:21 smithi151 ceph-mon[31623]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:22.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:22.165+0000 7f23cda00700 1 -- 172.21.15.151:0/948793130 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f23c81a8390 con 0x7f23c8075970 2024-05-21T21:44:22.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:22.166+0000 7f23bf400700 1 -- 172.21.15.151:0/948793130 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+277 (secure 0 0 0) 0x7f23b8030130 con 0x7f23c8075970 2024-05-21T21:44:22.166 INFO:teuthology.orchestra.run.smithi151.stdout:# minimal ceph.conf for c8c7a2e6-17ba-11ef-bc9a-c7b262605968 2024-05-21T21:44:22.166 INFO:teuthology.orchestra.run.smithi151.stdout:[global] 2024-05-21T21:44:22.166 INFO:teuthology.orchestra.run.smithi151.stdout: fsid = c8c7a2e6-17ba-11ef-bc9a-c7b262605968 2024-05-21T21:44:22.166 INFO:teuthology.orchestra.run.smithi151.stdout: mon_host = [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] 2024-05-21T21:44:22.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:22.169+0000 7f23cda00700 1 -- 172.21.15.151:0/948793130 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f23ac03e9a0 msgr2=0x7f23ac040e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:44:22.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:22.169+0000 7f23cda00700 1 --2- 172.21.15.151:0/948793130 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f23ac03e9a0 0x7f23ac040e60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f23b4009ad0 tx=0x7f23b4009210 comp rx=0 tx=0).stop 2024-05-21T21:44:22.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:22.169+0000 7f23cda00700 1 -- 172.21.15.151:0/948793130 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f23c8075970 msgr2=0x7f23c819aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:44:22.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:22.169+0000 7f23cda00700 1 --2- 172.21.15.151:0/948793130 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f23c8075970 0x7f23c819aed0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f23b8004f80 tx=0x7f23b8004fb0 comp rx=0 tx=0).stop 2024-05-21T21:44:22.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:22.169+0000 7f23cda00700 1 -- 172.21.15.151:0/948793130 shutdown_connections 2024-05-21T21:44:22.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:22.169+0000 7f23cda00700 1 --2- 172.21.15.151:0/948793130 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f23ac03e9a0 0x7f23ac040e60 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:44:22.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:22.169+0000 7f23cda00700 1 --2- 172.21.15.151:0/948793130 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f23c819fc90 0x7f23c81a0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:44:22.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:22.169+0000 7f23cda00700 1 --2- 172.21.15.151:0/948793130 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f23c8075970 0x7f23c819aed0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:44:22.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:22.169+0000 7f23cda00700 1 --2- 172.21.15.151:0/948793130 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f23c819b410 0x7f23c819f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:44:22.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:22.169+0000 7f23cda00700 1 -- 172.21.15.151:0/948793130 >> 172.21.15.151:0/948793130 conn(0x7f23c80fea30 msgr2=0x7f23c81091d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:44:22.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:22.169+0000 7f23cda00700 1 -- 172.21.15.151:0/948793130 shutdown_connections 2024-05-21T21:44:22.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:22.169+0000 7f23cda00700 1 -- 172.21.15.151:0/948793130 wait complete. 2024-05-21T21:44:22.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:21 smithi184 ceph-mon[36942]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:22.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:21 smithi164 ceph-mon[36269]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:22.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:22 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/948793130' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:44:23.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:22 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/948793130' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:44:23.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:22 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/948793130' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:44:23.471 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-05-21T21:44:23.471 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-05-21T21:44:23.471 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/etc/ceph/ceph.conf 2024-05-21T21:44:23.546 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-05-21T21:44:23.546 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-21T21:44:23.655 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-21T21:44:23.655 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/etc/ceph/ceph.conf 2024-05-21T21:44:23.681 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-21T21:44:23.682 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-21T21:44:23.746 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-05-21T21:44:23.746 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/etc/ceph/ceph.conf 2024-05-21T21:44:23.779 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-05-21T21:44:23.779 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-21T21:44:23.847 INFO:tasks.cephadm:Adding mgr.x on smithi151 2024-05-21T21:44:23.847 INFO:tasks.cephadm:Adding mgr.y on smithi164 2024-05-21T21:44:23.847 INFO:tasks.cephadm:Adding mgr.z on smithi184 2024-05-21T21:44:23.847 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph orch apply mgr '3;smithi151=x;smithi164=y;smithi184=z' 2024-05-21T21:44:24.119 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.c/config 2024-05-21T21:44:24.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:24 smithi164 ceph-mon[36269]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:24.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:24 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:24.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:24 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:24.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:24 smithi164 ceph-mon[36269]: Reconfiguring mon.b (monmap changed)... 2024-05-21T21:44:24.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:24 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-21T21:44:24.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:24 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-21T21:44:24.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:24 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:44:24.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:24 smithi164 ceph-mon[36269]: Reconfiguring daemon mon.b on smithi164 2024-05-21T21:44:24.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:24 smithi151 ceph-mon[31623]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:24.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:24 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:24.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:24 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:24.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:24 smithi151 ceph-mon[31623]: Reconfiguring mon.b (monmap changed)... 2024-05-21T21:44:24.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:24 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-21T21:44:24.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:24 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-21T21:44:24.454 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:24 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:44:24.454 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:24 smithi151 ceph-mon[31623]: Reconfiguring daemon mon.b on smithi164 2024-05-21T21:44:24.454 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:24 smithi184 ceph-mon[36942]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:24.454 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:24 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:24.454 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:24 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:24.454 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:24 smithi184 ceph-mon[36942]: Reconfiguring mon.b (monmap changed)... 2024-05-21T21:44:24.454 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:24 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-21T21:44:24.454 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:24 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-21T21:44:24.455 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:24 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:44:24.455 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:24 smithi184 ceph-mon[36942]: Reconfiguring daemon mon.b on smithi164 2024-05-21T21:44:25.837 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:25 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:25.837 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:25 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:25.837 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:25 smithi184 ceph-mon[36942]: Reconfiguring mon.c (monmap changed)... 2024-05-21T21:44:25.837 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:25 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-21T21:44:25.837 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:25 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-21T21:44:25.838 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:25 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:44:25.838 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:25 smithi184 ceph-mon[36942]: Reconfiguring daemon mon.c on smithi184 2024-05-21T21:44:25.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:25 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:25.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:25 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:25.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:25 smithi151 ceph-mon[31623]: Reconfiguring mon.c (monmap changed)... 2024-05-21T21:44:25.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:25 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-21T21:44:25.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:25 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-21T21:44:25.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:25 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:44:25.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:25 smithi151 ceph-mon[31623]: Reconfiguring daemon mon.c on smithi184 2024-05-21T21:44:26.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:25 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:26.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:25 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:26.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:25 smithi164 ceph-mon[36269]: Reconfiguring mon.c (monmap changed)... 2024-05-21T21:44:26.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:25 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-21T21:44:26.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:25 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-21T21:44:26.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:25 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:44:26.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:25 smithi164 ceph-mon[36269]: Reconfiguring daemon mon.c on smithi184 2024-05-21T21:44:26.659 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:25.943+0000 7f2383e61700 1 -- 172.21.15.184:0/1848898439 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f237c101ff0 msgr2=0x7f237c102410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:44:26.659 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:25.943+0000 7f2383e61700 1 --2- 172.21.15.184:0/1848898439 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f237c101ff0 0x7f237c102410 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2378009910 tx=0x7f2378009c20 comp rx=0 tx=0).stop 2024-05-21T21:44:26.659 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:25.943+0000 7f2383e61700 1 -- 172.21.15.184:0/1848898439 shutdown_connections 2024-05-21T21:44:26.659 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:25.943+0000 7f2383e61700 1 --2- 172.21.15.184:0/1848898439 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f237c101ff0 0x7f237c102410 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:44:26.659 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:25.943+0000 7f2383e61700 1 -- 172.21.15.184:0/1848898439 >> 172.21.15.184:0/1848898439 conn(0x7f237c0fd570 msgr2=0x7f237c0ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:44:26.659 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:25.943+0000 7f2383e61700 1 -- 172.21.15.184:0/1848898439 shutdown_connections 2024-05-21T21:44:26.659 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:25.943+0000 7f2383e61700 1 -- 172.21.15.184:0/1848898439 wait complete. 2024-05-21T21:44:26.659 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:25.944+0000 7f2383e61700 1 Processor -- start 2024-05-21T21:44:26.659 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:25.944+0000 7f2383e61700 1 -- start start 2024-05-21T21:44:26.659 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:25.944+0000 7f2383e61700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f237c101ff0 0x7f237c0ff530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:44:26.659 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:25.944+0000 7f2383e61700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f237c0fdb80 0x7f237c0fe000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:44:26.660 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:25.944+0000 7f2383e61700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f237c0fe540 0x7f237c1a28a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:44:26.660 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:25.944+0000 7f2383e61700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f237c107f30 con 0x7f237c0fe540 2024-05-21T21:44:26.660 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:25.944+0000 7f2383e61700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f237c107db0 con 0x7f237c0fdb80 2024-05-21T21:44:26.660 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:25.944+0000 7f2383e61700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f237c1080b0 con 0x7f237c101ff0 2024-05-21T21:44:26.660 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:25.944+0000 7f2381bfd700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f237c101ff0 0x7f237c0ff530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:44:26.660 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:25.944+0000 7f2381bfd700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f237c101ff0 0x7f237c0ff530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:55672/0 (socket says 172.21.15.184:55672) 2024-05-21T21:44:26.660 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:25.944+0000 7f2381bfd700 1 -- 172.21.15.184:0/896041273 learned_addr learned my addr 172.21.15.184:0/896041273 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-05-21T21:44:26.660 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:25.944+0000 7f23813fc700 1 --2- 172.21.15.184:0/896041273 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f237c0fdb80 0x7f237c0fe000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:44:26.660 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:25.944+0000 7f2381bfd700 1 -- 172.21.15.184:0/896041273 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f237c0fdb80 msgr2=0x7f237c0fe000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:44:26.660 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:25.944+0000 7f2381bfd700 1 --2- 172.21.15.184:0/896041273 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f237c0fdb80 0x7f237c0fe000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:44:26.660 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:25.944+0000 7f2381bfd700 1 -- 172.21.15.184:0/896041273 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f237c0fe540 msgr2=0x7f237c1a28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:44:26.660 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:25.944+0000 7f2381bfd700 1 --2- 172.21.15.184:0/896041273 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f237c0fe540 0x7f237c1a28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:44:26.661 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:25.944+0000 7f2381bfd700 1 -- 172.21.15.184:0/896041273 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f237c1a2e60 con 0x7f237c101ff0 2024-05-21T21:44:26.661 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:25.944+0000 7f23813fc700 1 --2- 172.21.15.184:0/896041273 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f237c0fdb80 0x7f237c0fe000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T21:44:26.661 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:25.944+0000 7f2381bfd700 1 --2- 172.21.15.184:0/896041273 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f237c101ff0 0x7f237c0ff530 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f2378005e00 tx=0x7f2378009e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:44:26.661 INFO:teuthology.orchestra.run.smithi184.stdout:Scheduled mgr update... 2024-05-21T21:44:26.662 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:25.945+0000 7f2372ffd700 1 -- 172.21.15.184:0/896041273 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2378027070 con 0x7f237c101ff0 2024-05-21T21:44:26.662 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:25.945+0000 7f2383e61700 1 -- 172.21.15.184:0/896041273 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f237c1a30f0 con 0x7f237c101ff0 2024-05-21T21:44:26.662 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:25.945+0000 7f2372ffd700 1 -- 172.21.15.184:0/896041273 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f237800be10 con 0x7f237c101ff0 2024-05-21T21:44:26.662 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:25.945+0000 7f2372ffd700 1 -- 172.21.15.184:0/896041273 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f237800f460 con 0x7f237c101ff0 2024-05-21T21:44:26.662 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:25.945+0000 7f2383e61700 1 -- 172.21.15.184:0/896041273 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f237c1a36a0 con 0x7f237c101ff0 2024-05-21T21:44:26.662 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:25.945+0000 7f2372ffd700 1 -- 172.21.15.184:0/896041273 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7f237802a070 con 0x7f237c101ff0 2024-05-21T21:44:26.662 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:25.946+0000 7f2372ffd700 1 --2- 172.21.15.184:0/896041273 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f236803e9a0 0x7f2368040e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:44:26.662 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:25.946+0000 7f2372ffd700 1 -- 172.21.15.184:0/896041273 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f23780548a0 con 0x7f237c101ff0 2024-05-21T21:44:26.662 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:25.946+0000 7f2383e61700 1 -- 172.21.15.184:0/896041273 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f237c064ba0 con 0x7f237c101ff0 2024-05-21T21:44:26.662 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:25.946+0000 7f23813fc700 1 --2- 172.21.15.184:0/896041273 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f236803e9a0 0x7f2368040e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:44:26.662 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:25.947+0000 7f23813fc700 1 --2- 172.21.15.184:0/896041273 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f236803e9a0 0x7f2368040e60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f236c009910 tx=0x7f236c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:44:26.663 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:25.952+0000 7f2372ffd700 1 -- 172.21.15.184:0/896041273 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2378031070 con 0x7f237c101ff0 2024-05-21T21:44:26.663 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:26.170+0000 7f2383e61700 1 -- 172.21.15.184:0/896041273 --> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi151=x;smithi164=y;smithi184=z", "target": ["mon-mgr", ""]}) -- 0x7f237c063980 con 0x7f236803e9a0 2024-05-21T21:44:26.663 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:26.184+0000 7f2372ffd700 1 -- 172.21.15.184:0/896041273 <== mgr.14152 v2:172.21.15.151:6800/2796622483 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f237c063980 con 0x7f236803e9a0 2024-05-21T21:44:26.663 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:26.187+0000 7f2383e61700 1 -- 172.21.15.184:0/896041273 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f236803e9a0 msgr2=0x7f2368040e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:44:26.663 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:26.187+0000 7f2383e61700 1 --2- 172.21.15.184:0/896041273 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f236803e9a0 0x7f2368040e60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f236c009910 tx=0x7f236c008040 comp rx=0 tx=0).stop 2024-05-21T21:44:26.663 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:26.187+0000 7f2383e61700 1 -- 172.21.15.184:0/896041273 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f237c101ff0 msgr2=0x7f237c0ff530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:44:26.663 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:26.187+0000 7f2383e61700 1 --2- 172.21.15.184:0/896041273 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f237c101ff0 0x7f237c0ff530 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f2378005e00 tx=0x7f2378009e60 comp rx=0 tx=0).stop 2024-05-21T21:44:26.663 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:26.187+0000 7f2383e61700 1 -- 172.21.15.184:0/896041273 shutdown_connections 2024-05-21T21:44:26.663 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:26.187+0000 7f2383e61700 1 --2- 172.21.15.184:0/896041273 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f236803e9a0 0x7f2368040e60 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:44:26.663 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:26.187+0000 7f2383e61700 1 --2- 172.21.15.184:0/896041273 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f237c0fe540 0x7f237c1a28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:44:26.663 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:26.187+0000 7f2383e61700 1 --2- 172.21.15.184:0/896041273 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f237c101ff0 0x7f237c0ff530 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:44:26.663 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:26.187+0000 7f2383e61700 1 --2- 172.21.15.184:0/896041273 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f237c0fdb80 0x7f237c0fe000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:44:26.663 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:26.187+0000 7f2383e61700 1 -- 172.21.15.184:0/896041273 >> 172.21.15.184:0/896041273 conn(0x7f237c0fd570 msgr2=0x7f237c06bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:44:26.664 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:26.187+0000 7f2383e61700 1 -- 172.21.15.184:0/896041273 shutdown_connections 2024-05-21T21:44:26.664 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:44:26.187+0000 7f2383e61700 1 -- 172.21.15.184:0/896041273 wait complete. 2024-05-21T21:44:26.945 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:26 smithi184 ceph-mon[36942]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:26.945 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:26 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:26.952 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:26 smithi151 ceph-mon[31623]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:26.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:26 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:27.035 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:26 smithi164 ceph-mon[36269]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:27.035 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:26 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:27.205 DEBUG:teuthology.orchestra.run.smithi164:mgr.y> sudo journalctl -f -n 0 -u ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@mgr.y.service 2024-05-21T21:44:27.207 DEBUG:teuthology.orchestra.run.smithi184:mgr.z> sudo journalctl -f -n 0 -u ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@mgr.z.service 2024-05-21T21:44:27.209 INFO:tasks.cephadm:Deploying OSDs... 2024-05-21T21:44:27.210 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-05-21T21:44:27.210 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/scratch_devs of=/dev/stdout 2024-05-21T21:44:27.266 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-21T21:44:27.266 DEBUG:teuthology.orchestra.run.smithi151:> stat /dev/vg_nvme/lv_1 2024-05-21T21:44:27.315 INFO:teuthology.orchestra.run.smithi151.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-21T21:44:27.315 INFO:teuthology.orchestra.run.smithi151.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-21T21:44:27.315 INFO:teuthology.orchestra.run.smithi151.stdout:Device: 6h/6d Inode: 640 Links: 1 2024-05-21T21:44:27.315 INFO:teuthology.orchestra.run.smithi151.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-21T21:44:27.315 INFO:teuthology.orchestra.run.smithi151.stdout:Context: system_u:object_r:device_t:s0 2024-05-21T21:44:27.316 INFO:teuthology.orchestra.run.smithi151.stdout:Access: 2024-05-21 21:43:22.171738060 +0000 2024-05-21T21:44:27.316 INFO:teuthology.orchestra.run.smithi151.stdout:Modify: 2024-05-21 21:37:38.101166922 +0000 2024-05-21T21:44:27.316 INFO:teuthology.orchestra.run.smithi151.stdout:Change: 2024-05-21 21:37:38.101166922 +0000 2024-05-21T21:44:27.316 INFO:teuthology.orchestra.run.smithi151.stdout: Birth: 2024-05-21 21:37:38.101166922 +0000 2024-05-21T21:44:27.316 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-21T21:44:27.394 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records in 2024-05-21T21:44:27.394 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records out 2024-05-21T21:44:27.394 INFO:teuthology.orchestra.run.smithi151.stderr:512 bytes copied, 0.000382525 s, 1.3 MB/s 2024-05-21T21:44:27.397 DEBUG:teuthology.orchestra.run.smithi151:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-21T21:44:27.451 DEBUG:teuthology.orchestra.run.smithi151:> stat /dev/vg_nvme/lv_2 2024-05-21T21:44:27.503 INFO:teuthology.orchestra.run.smithi151.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-21T21:44:27.503 INFO:teuthology.orchestra.run.smithi151.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-21T21:44:27.504 INFO:teuthology.orchestra.run.smithi151.stdout:Device: 6h/6d Inode: 656 Links: 1 2024-05-21T21:44:27.504 INFO:teuthology.orchestra.run.smithi151.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-21T21:44:27.504 INFO:teuthology.orchestra.run.smithi151.stdout:Context: system_u:object_r:device_t:s0 2024-05-21T21:44:27.504 INFO:teuthology.orchestra.run.smithi151.stdout:Access: 2024-05-21 21:43:22.230737358 +0000 2024-05-21T21:44:27.504 INFO:teuthology.orchestra.run.smithi151.stdout:Modify: 2024-05-21 21:37:38.685160011 +0000 2024-05-21T21:44:27.504 INFO:teuthology.orchestra.run.smithi151.stdout:Change: 2024-05-21 21:37:38.685160011 +0000 2024-05-21T21:44:27.505 INFO:teuthology.orchestra.run.smithi151.stdout: Birth: 2024-05-21 21:37:38.685160011 +0000 2024-05-21T21:44:27.505 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-21T21:44:27.582 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records in 2024-05-21T21:44:27.582 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records out 2024-05-21T21:44:27.582 INFO:teuthology.orchestra.run.smithi151.stderr:512 bytes copied, 0.000385499 s, 1.3 MB/s 2024-05-21T21:44:27.585 DEBUG:teuthology.orchestra.run.smithi151:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-21T21:44:27.639 DEBUG:teuthology.orchestra.run.smithi151:> stat /dev/vg_nvme/lv_3 2024-05-21T21:44:27.688 INFO:teuthology.orchestra.run.smithi151.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-21T21:44:27.688 INFO:teuthology.orchestra.run.smithi151.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-21T21:44:27.688 INFO:teuthology.orchestra.run.smithi151.stdout:Device: 6h/6d Inode: 676 Links: 1 2024-05-21T21:44:27.688 INFO:teuthology.orchestra.run.smithi151.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-21T21:44:27.688 INFO:teuthology.orchestra.run.smithi151.stdout:Context: system_u:object_r:device_t:s0 2024-05-21T21:44:27.688 INFO:teuthology.orchestra.run.smithi151.stdout:Access: 2024-05-21 21:43:22.293736607 +0000 2024-05-21T21:44:27.688 INFO:teuthology.orchestra.run.smithi151.stdout:Modify: 2024-05-21 21:37:39.288152875 +0000 2024-05-21T21:44:27.689 INFO:teuthology.orchestra.run.smithi151.stdout:Change: 2024-05-21 21:37:39.288152875 +0000 2024-05-21T21:44:27.689 INFO:teuthology.orchestra.run.smithi151.stdout: Birth: 2024-05-21 21:37:39.288152875 +0000 2024-05-21T21:44:27.689 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-21T21:44:27.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:27 smithi151 ceph-mon[31623]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi151=x;smithi164=y;smithi184=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:44:27.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:27 smithi151 ceph-mon[31623]: Saving service mgr spec with placement smithi151=x;smithi164=y;smithi184=z;count:3 2024-05-21T21:44:27.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:27 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:27.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:27 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:27.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:27 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:44:27.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:27 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:44:27.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:27 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:44:27.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:27 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:27.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:27 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-21T21:44:27.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:27 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-21T21:44:27.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:27 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-21T21:44:27.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:27 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:44:27.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:27 smithi151 ceph-mon[31623]: Deploying daemon mgr.z on smithi184 2024-05-21T21:44:27.766 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records in 2024-05-21T21:44:27.766 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records out 2024-05-21T21:44:27.766 INFO:teuthology.orchestra.run.smithi151.stderr:512 bytes copied, 0.000355207 s, 1.4 MB/s 2024-05-21T21:44:27.769 DEBUG:teuthology.orchestra.run.smithi151:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-21T21:44:27.869 DEBUG:teuthology.orchestra.run.smithi151:> stat /dev/vg_nvme/lv_4 2024-05-21T21:44:27.922 INFO:teuthology.orchestra.run.smithi151.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-21T21:44:27.922 INFO:teuthology.orchestra.run.smithi151.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-21T21:44:27.922 INFO:teuthology.orchestra.run.smithi151.stdout:Device: 6h/6d Inode: 697 Links: 1 2024-05-21T21:44:27.922 INFO:teuthology.orchestra.run.smithi151.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-21T21:44:27.922 INFO:teuthology.orchestra.run.smithi151.stdout:Context: system_u:object_r:device_t:s0 2024-05-21T21:44:27.922 INFO:teuthology.orchestra.run.smithi151.stdout:Access: 2024-05-21 21:43:22.368735714 +0000 2024-05-21T21:44:27.922 INFO:teuthology.orchestra.run.smithi151.stdout:Modify: 2024-05-21 21:37:39.883145833 +0000 2024-05-21T21:44:27.923 INFO:teuthology.orchestra.run.smithi151.stdout:Change: 2024-05-21 21:37:39.883145833 +0000 2024-05-21T21:44:27.923 INFO:teuthology.orchestra.run.smithi151.stdout: Birth: 2024-05-21 21:37:39.883145833 +0000 2024-05-21T21:44:27.923 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-21T21:44:27.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:27 smithi184 ceph-mon[36942]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi151=x;smithi164=y;smithi184=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:44:27.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:27 smithi184 ceph-mon[36942]: Saving service mgr spec with placement smithi151=x;smithi164=y;smithi184=z;count:3 2024-05-21T21:44:27.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:27 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:27.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:27 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:27.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:27 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:44:27.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:27 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:44:27.987 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:27 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:44:27.987 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:27 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:27.987 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:27 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-21T21:44:27.987 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:27 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-21T21:44:27.987 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:27 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-21T21:44:27.987 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:27 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:44:27.987 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:27 smithi184 ceph-mon[36942]: Deploying daemon mgr.z on smithi184 2024-05-21T21:44:28.005 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records in 2024-05-21T21:44:28.005 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records out 2024-05-21T21:44:28.005 INFO:teuthology.orchestra.run.smithi151.stderr:512 bytes copied, 0.000368944 s, 1.4 MB/s 2024-05-21T21:44:28.009 DEBUG:teuthology.orchestra.run.smithi151:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-21T21:44:28.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:27 smithi164 ceph-mon[36269]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi151=x;smithi164=y;smithi184=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:44:28.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:27 smithi164 ceph-mon[36269]: Saving service mgr spec with placement smithi151=x;smithi164=y;smithi184=z;count:3 2024-05-21T21:44:28.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:27 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:28.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:27 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:28.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:27 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:44:28.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:27 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:44:28.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:27 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:44:28.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:27 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:28.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:27 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-21T21:44:28.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:27 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-21T21:44:28.037 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:27 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-21T21:44:28.037 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:27 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:44:28.037 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:27 smithi164 ceph-mon[36269]: Deploying daemon mgr.z on smithi184 2024-05-21T21:44:28.069 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-05-21T21:44:28.069 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/scratch_devs of=/dev/stdout 2024-05-21T21:44:28.088 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-21T21:44:28.088 DEBUG:teuthology.orchestra.run.smithi164:> stat /dev/vg_nvme/lv_1 2024-05-21T21:44:28.146 INFO:teuthology.orchestra.run.smithi164.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-21T21:44:28.146 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-21T21:44:28.146 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-05-21T21:44:28.146 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-21T21:44:28.146 INFO:teuthology.orchestra.run.smithi164.stdout:Context: system_u:object_r:device_t:s0 2024-05-21T21:44:28.147 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-05-21 21:44:11.463940494 +0000 2024-05-21T21:44:28.147 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-05-21 21:36:04.673379277 +0000 2024-05-21T21:44:28.147 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-05-21 21:36:04.673379277 +0000 2024-05-21T21:44:28.147 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-05-21 21:36:04.673379277 +0000 2024-05-21T21:44:28.147 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-21T21:44:28.211 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records in 2024-05-21T21:44:28.212 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records out 2024-05-21T21:44:28.212 INFO:teuthology.orchestra.run.smithi164.stderr:512 bytes copied, 0.00022677 s, 2.3 MB/s 2024-05-21T21:44:28.213 DEBUG:teuthology.orchestra.run.smithi164:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-21T21:44:28.271 DEBUG:teuthology.orchestra.run.smithi164:> stat /dev/vg_nvme/lv_2 2024-05-21T21:44:28.327 INFO:teuthology.orchestra.run.smithi164.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-21T21:44:28.328 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-21T21:44:28.328 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-05-21T21:44:28.328 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-21T21:44:28.328 INFO:teuthology.orchestra.run.smithi164.stdout:Context: system_u:object_r:device_t:s0 2024-05-21T21:44:28.328 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-05-21 21:44:11.464940495 +0000 2024-05-21T21:44:28.328 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-05-21 21:36:05.042379777 +0000 2024-05-21T21:44:28.328 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-05-21 21:36:05.042379777 +0000 2024-05-21T21:44:28.328 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-05-21 21:36:05.042379777 +0000 2024-05-21T21:44:28.328 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-21T21:44:28.392 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records in 2024-05-21T21:44:28.392 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records out 2024-05-21T21:44:28.392 INFO:teuthology.orchestra.run.smithi164.stderr:512 bytes copied, 0.00021125 s, 2.4 MB/s 2024-05-21T21:44:28.394 DEBUG:teuthology.orchestra.run.smithi164:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-21T21:44:28.451 DEBUG:teuthology.orchestra.run.smithi164:> stat /dev/vg_nvme/lv_3 2024-05-21T21:44:28.507 INFO:teuthology.orchestra.run.smithi164.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-21T21:44:28.508 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-21T21:44:28.508 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-05-21T21:44:28.508 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-21T21:44:28.508 INFO:teuthology.orchestra.run.smithi164.stdout:Context: system_u:object_r:device_t:s0 2024-05-21T21:44:28.508 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-05-21 21:44:11.464940495 +0000 2024-05-21T21:44:28.508 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-05-21 21:36:05.374380226 +0000 2024-05-21T21:44:28.508 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-05-21 21:36:05.374380226 +0000 2024-05-21T21:44:28.508 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-05-21 21:36:05.374380226 +0000 2024-05-21T21:44:28.508 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-21T21:44:28.571 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records in 2024-05-21T21:44:28.571 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records out 2024-05-21T21:44:28.571 INFO:teuthology.orchestra.run.smithi164.stderr:512 bytes copied, 0.000204803 s, 2.5 MB/s 2024-05-21T21:44:28.572 DEBUG:teuthology.orchestra.run.smithi164:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-21T21:44:28.629 DEBUG:teuthology.orchestra.run.smithi164:> stat /dev/vg_nvme/lv_4 2024-05-21T21:44:28.684 INFO:teuthology.orchestra.run.smithi164.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-21T21:44:28.684 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-21T21:44:28.684 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-05-21T21:44:28.684 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-21T21:44:28.684 INFO:teuthology.orchestra.run.smithi164.stdout:Context: system_u:object_r:device_t:s0 2024-05-21T21:44:28.684 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-05-21 21:44:11.464940495 +0000 2024-05-21T21:44:28.684 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-05-21 21:36:05.706380676 +0000 2024-05-21T21:44:28.684 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-05-21 21:36:05.706380676 +0000 2024-05-21T21:44:28.684 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-05-21 21:36:05.706380676 +0000 2024-05-21T21:44:28.685 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-21T21:44:28.747 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records in 2024-05-21T21:44:28.747 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records out 2024-05-21T21:44:28.747 INFO:teuthology.orchestra.run.smithi164.stderr:512 bytes copied, 0.000179181 s, 2.9 MB/s 2024-05-21T21:44:28.748 DEBUG:teuthology.orchestra.run.smithi164:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-21T21:44:28.804 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2024-05-21T21:44:28.805 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/scratch_devs of=/dev/stdout 2024-05-21T21:44:28.821 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-21T21:44:28.821 DEBUG:teuthology.orchestra.run.smithi184:> stat /dev/vg_nvme/lv_1 2024-05-21T21:44:28.876 INFO:teuthology.orchestra.run.smithi184.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-21T21:44:28.876 INFO:teuthology.orchestra.run.smithi184.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-21T21:44:28.877 INFO:teuthology.orchestra.run.smithi184.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-05-21T21:44:28.877 INFO:teuthology.orchestra.run.smithi184.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-21T21:44:28.877 INFO:teuthology.orchestra.run.smithi184.stdout:Context: system_u:object_r:device_t:s0 2024-05-21T21:44:28.877 INFO:teuthology.orchestra.run.smithi184.stdout:Access: 2024-05-21 21:44:28.670612468 +0000 2024-05-21T21:44:28.877 INFO:teuthology.orchestra.run.smithi184.stdout:Modify: 2024-05-21 21:36:16.484539819 +0000 2024-05-21T21:44:28.877 INFO:teuthology.orchestra.run.smithi184.stdout:Change: 2024-05-21 21:36:16.484539819 +0000 2024-05-21T21:44:28.877 INFO:teuthology.orchestra.run.smithi184.stdout: Birth: 2024-05-21 21:36:16.484539819 +0000 2024-05-21T21:44:28.877 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-21T21:44:28.938 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:28 smithi184 ceph-mon[36942]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:28.939 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 21 21:44:28 smithi184 systemd[1]: Starting Ceph mgr.z for c8c7a2e6-17ba-11ef-bc9a-c7b262605968... 2024-05-21T21:44:28.941 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records in 2024-05-21T21:44:28.941 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records out 2024-05-21T21:44:28.942 INFO:teuthology.orchestra.run.smithi184.stderr:512 bytes copied, 0.000238844 s, 2.1 MB/s 2024-05-21T21:44:28.943 DEBUG:teuthology.orchestra.run.smithi184:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-21T21:44:28.952 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:28 smithi151 ceph-mon[31623]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:28.999 DEBUG:teuthology.orchestra.run.smithi184:> stat /dev/vg_nvme/lv_2 2024-05-21T21:44:29.035 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:28 smithi164 ceph-mon[36269]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:29.053 INFO:teuthology.orchestra.run.smithi184.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-21T21:44:29.053 INFO:teuthology.orchestra.run.smithi184.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-21T21:44:29.054 INFO:teuthology.orchestra.run.smithi184.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-05-21T21:44:29.054 INFO:teuthology.orchestra.run.smithi184.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-21T21:44:29.054 INFO:teuthology.orchestra.run.smithi184.stdout:Context: system_u:object_r:device_t:s0 2024-05-21T21:44:29.054 INFO:teuthology.orchestra.run.smithi184.stdout:Access: 2024-05-21 21:44:28.670612468 +0000 2024-05-21T21:44:29.054 INFO:teuthology.orchestra.run.smithi184.stdout:Modify: 2024-05-21 21:36:16.822532939 +0000 2024-05-21T21:44:29.054 INFO:teuthology.orchestra.run.smithi184.stdout:Change: 2024-05-21 21:36:16.822532939 +0000 2024-05-21T21:44:29.054 INFO:teuthology.orchestra.run.smithi184.stdout: Birth: 2024-05-21 21:36:16.822532939 +0000 2024-05-21T21:44:29.055 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-21T21:44:29.127 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records in 2024-05-21T21:44:29.127 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records out 2024-05-21T21:44:29.128 INFO:teuthology.orchestra.run.smithi184.stderr:512 bytes copied, 0.00954035 s, 53.7 kB/s 2024-05-21T21:44:29.128 DEBUG:teuthology.orchestra.run.smithi184:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-21T21:44:29.184 DEBUG:teuthology.orchestra.run.smithi184:> stat /dev/vg_nvme/lv_3 2024-05-21T21:44:29.215 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 21 21:44:28 smithi184 podman[38575]: 2024-05-21 21:44:28.848446637 +0000 UTC m=+0.017264616 image pull d6bc8f31cbc1a7165ca248fe9ba0b06ea14b30f46e1ea33ced1a5238e247af1f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28 2024-05-21T21:44:29.215 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 21 21:44:28 smithi184 podman[38575]: 2024-05-21 21:44:28.95922158 +0000 UTC m=+0.128039544 container create ed05f69e3229a9e798932e71a5880eb38990e58ffe5f8a51e73f115c6d61355b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-z, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea) 2024-05-21T21:44:29.215 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 21 21:44:29 smithi184 podman[38575]: 2024-05-21 21:44:29.215713558 +0000 UTC m=+0.384531530 container init ed05f69e3229a9e798932e71a5880eb38990e58ffe5f8a51e73f115c6d61355b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-z, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-05-21T21:44:29.230 INFO:teuthology.orchestra.run.smithi184.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-21T21:44:29.230 INFO:teuthology.orchestra.run.smithi184.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-21T21:44:29.230 INFO:teuthology.orchestra.run.smithi184.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-05-21T21:44:29.230 INFO:teuthology.orchestra.run.smithi184.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-21T21:44:29.230 INFO:teuthology.orchestra.run.smithi184.stdout:Context: system_u:object_r:device_t:s0 2024-05-21T21:44:29.230 INFO:teuthology.orchestra.run.smithi184.stdout:Access: 2024-05-21 21:44:28.670612468 +0000 2024-05-21T21:44:29.230 INFO:teuthology.orchestra.run.smithi184.stdout:Modify: 2024-05-21 21:36:17.164525977 +0000 2024-05-21T21:44:29.230 INFO:teuthology.orchestra.run.smithi184.stdout:Change: 2024-05-21 21:36:17.164525977 +0000 2024-05-21T21:44:29.231 INFO:teuthology.orchestra.run.smithi184.stdout: Birth: 2024-05-21 21:36:17.164525977 +0000 2024-05-21T21:44:29.231 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-21T21:44:29.293 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records in 2024-05-21T21:44:29.293 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records out 2024-05-21T21:44:29.293 INFO:teuthology.orchestra.run.smithi184.stderr:512 bytes copied, 0.000208166 s, 2.5 MB/s 2024-05-21T21:44:29.295 DEBUG:teuthology.orchestra.run.smithi184:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-21T21:44:29.351 DEBUG:teuthology.orchestra.run.smithi184:> stat /dev/vg_nvme/lv_4 2024-05-21T21:44:29.406 INFO:teuthology.orchestra.run.smithi184.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-21T21:44:29.406 INFO:teuthology.orchestra.run.smithi184.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-21T21:44:29.406 INFO:teuthology.orchestra.run.smithi184.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-05-21T21:44:29.406 INFO:teuthology.orchestra.run.smithi184.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-21T21:44:29.406 INFO:teuthology.orchestra.run.smithi184.stdout:Context: system_u:object_r:device_t:s0 2024-05-21T21:44:29.406 INFO:teuthology.orchestra.run.smithi184.stdout:Access: 2024-05-21 21:44:28.671612448 +0000 2024-05-21T21:44:29.406 INFO:teuthology.orchestra.run.smithi184.stdout:Modify: 2024-05-21 21:36:17.501519117 +0000 2024-05-21T21:44:29.407 INFO:teuthology.orchestra.run.smithi184.stdout:Change: 2024-05-21 21:36:17.501519117 +0000 2024-05-21T21:44:29.407 INFO:teuthology.orchestra.run.smithi184.stdout: Birth: 2024-05-21 21:36:17.501519117 +0000 2024-05-21T21:44:29.407 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-21T21:44:29.466 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 21 21:44:29 smithi184 podman[38575]: 2024-05-21 21:44:29.218798779 +0000 UTC m=+0.387616752 container start ed05f69e3229a9e798932e71a5880eb38990e58ffe5f8a51e73f115c6d61355b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-z, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-05-21T21:44:29.466 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 21 21:44:29 smithi184 ceph-mgr[38701]: -- 172.21.15.184:0/649057745 <== mon.1 v2:172.21.15.184:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558d1e1385a0 con 0x558d1e10d000 2024-05-21T21:44:29.467 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 21 21:44:29 smithi184 bash[38575]: ed05f69e3229a9e798932e71a5880eb38990e58ffe5f8a51e73f115c6d61355b 2024-05-21T21:44:29.467 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 21 21:44:29 smithi184 systemd[1]: Started Ceph mgr.z for c8c7a2e6-17ba-11ef-bc9a-c7b262605968. 2024-05-21T21:44:29.467 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 21 21:44:29 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-z[38659]: 2024-05-21T21:44:29.379+0000 7f7df830d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-21T21:44:29.469 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records in 2024-05-21T21:44:29.469 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records out 2024-05-21T21:44:29.469 INFO:teuthology.orchestra.run.smithi184.stderr:512 bytes copied, 0.000303187 s, 1.7 MB/s 2024-05-21T21:44:29.473 DEBUG:teuthology.orchestra.run.smithi184:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-21T21:44:29.527 INFO:tasks.cephadm:Deploying osd.0 on smithi151 with /dev/vg_nvme/lv_4... 2024-05-21T21:44:29.528 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-21T21:44:29.735 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 21 21:44:29 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-z[38659]: 2024-05-21T21:44:29.576+0000 7f7df830d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-21T21:44:29.736 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 21 21:44:29 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-z[38659]: 2024-05-21T21:44:29.677+0000 7f7df830d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-21T21:44:29.962 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:44:30.236 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 21 21:44:29 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-z[38659]: 2024-05-21T21:44:29.984+0000 7f7df830d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-21T21:44:30.236 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 21 21:44:30 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-z[38659]: 2024-05-21T21:44:30.069+0000 7f7df830d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-21T21:44:30.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:30 smithi151 ceph-mon[31623]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:30.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:30 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:30.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:30 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:30.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:30 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:30.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:30 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-21T21:44:30.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:30 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-21T21:44:30.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:30 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-21T21:44:30.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:30 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:44:30.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:30 smithi151 ceph-mon[31623]: Deploying daemon mgr.y on smithi164 2024-05-21T21:44:30.723 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:30 smithi164 ceph-mon[36269]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:30.723 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:30 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:30.723 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:30 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:30.724 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:30 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:30.724 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:30 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-21T21:44:30.724 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:30 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-21T21:44:30.724 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:30 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-21T21:44:30.724 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:30 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:44:30.724 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:30 smithi164 ceph-mon[36269]: Deploying daemon mgr.y on smithi164 2024-05-21T21:44:30.735 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:30 smithi184 ceph-mon[36942]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:30.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:30 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:30.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:30 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:30.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:30 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:30.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:30 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-21T21:44:30.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:30 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-21T21:44:30.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:30 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-21T21:44:30.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:30 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:44:30.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:30 smithi184 ceph-mon[36942]: Deploying daemon mgr.y on smithi164 2024-05-21T21:44:31.536 INFO:journalctl@ceph.mgr.y.smithi164.stdout:May 21 21:44:31 smithi164 systemd[1]: Starting Ceph mgr.y for c8c7a2e6-17ba-11ef-bc9a-c7b262605968... 2024-05-21T21:44:31.599 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 21 21:44:31 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-z[38659]: 2024-05-21T21:44:31.260+0000 7f7df830d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-21T21:44:31.599 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 21 21:44:31 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-z[38659]: 2024-05-21T21:44:31.428+0000 7f7df830d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-21T21:44:31.599 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 21 21:44:31 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-z[38659]: 2024-05-21T21:44:31.598+0000 7f7df830d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-21T21:44:31.927 INFO:journalctl@ceph.mgr.y.smithi164.stdout:May 21 21:44:31 smithi164 podman[37635]: 2024-05-21 21:44:31.561506878 +0000 UTC m=+0.016868211 image pull d6bc8f31cbc1a7165ca248fe9ba0b06ea14b30f46e1ea33ced1a5238e247af1f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28 2024-05-21T21:44:31.927 INFO:journalctl@ceph.mgr.y.smithi164.stdout:May 21 21:44:31 smithi164 podman[37635]: 2024-05-21 21:44:31.671806184 +0000 UTC m=+0.127167493 container create 64dc68d1e6d00c60eaf06d621145bc56f0fec52932d5668990a4db17e19998e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-05-21T21:44:31.927 INFO:journalctl@ceph.mgr.y.smithi164.stdout:May 21 21:44:31 smithi164 podman[37635]: 2024-05-21 21:44:31.869068934 +0000 UTC m=+0.324430249 container init 64dc68d1e6d00c60eaf06d621145bc56f0fec52932d5668990a4db17e19998e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-y, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-05-21T21:44:31.927 INFO:journalctl@ceph.mgr.y.smithi164.stdout:May 21 21:44:31 smithi164 podman[37635]: 2024-05-21 21:44:31.871610837 +0000 UTC m=+0.326972152 container start 64dc68d1e6d00c60eaf06d621145bc56f0fec52932d5668990a4db17e19998e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-y, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-21T21:44:31.928 INFO:journalctl@ceph.mgr.y.smithi164.stdout:May 21 21:44:31 smithi164 ceph-mgr[37654]: -- 172.21.15.164:0/1710284841 <== mon.2 v2:172.21.15.164:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5587a8ad65a0 con 0x5587a8aab000 2024-05-21T21:44:31.928 INFO:journalctl@ceph.mgr.y.smithi164.stdout:May 21 21:44:31 smithi164 bash[37635]: 64dc68d1e6d00c60eaf06d621145bc56f0fec52932d5668990a4db17e19998e5 2024-05-21T21:44:32.181 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 21 21:44:31 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-z[38659]: 2024-05-21T21:44:31.884+0000 7f7df830d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-21T21:44:32.181 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 21 21:44:31 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-z[38659]: 2024-05-21T21:44:31.975+0000 7f7df830d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-21T21:44:32.181 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 21 21:44:32 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-z[38659]: 2024-05-21T21:44:32.180+0000 7f7df830d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-21T21:44:32.214 INFO:journalctl@ceph.mgr.y.smithi164.stdout:May 21 21:44:31 smithi164 systemd[1]: Started Ceph mgr.y for c8c7a2e6-17ba-11ef-bc9a-c7b262605968. 2024-05-21T21:44:32.214 INFO:journalctl@ceph.mgr.y.smithi164.stdout:May 21 21:44:32 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-y[37650]: 2024-05-21T21:44:32.024+0000 7f18ed4f5200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-21T21:44:32.214 INFO:journalctl@ceph.mgr.y.smithi164.stdout:May 21 21:44:32 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-y[37650]: 2024-05-21T21:44:32.213+0000 7f18ed4f5200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-21T21:44:32.486 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 21 21:44:32 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-z[38659]: 2024-05-21T21:44:32.376+0000 7f7df830d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-21T21:44:32.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:32 smithi184 ceph-mon[36942]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:32.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:32 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:32.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:32 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:32.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:32 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:32.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:32 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:32.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:32 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:44:32.536 INFO:journalctl@ceph.mgr.y.smithi164.stdout:May 21 21:44:32 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-y[37650]: 2024-05-21T21:44:32.316+0000 7f18ed4f5200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-21T21:44:32.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:32 smithi164 ceph-mon[36269]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:32.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:32 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:32.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:32 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:32.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:32 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:32.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:32 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:32.537 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:32 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:44:32.702 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:32 smithi151 ceph-mon[31623]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:32.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:32 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:32.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:32 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:32.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:32 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:32.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:32 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:32.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:32 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:44:32.703 INFO:teuthology.orchestra.run.smithi151.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-21T21:44:32.703 INFO:teuthology.orchestra.run.smithi151.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-21T21:44:32.704 INFO:teuthology.orchestra.run.smithi151.stderr: stderr: 10+0 records in 2024-05-21T21:44:32.704 INFO:teuthology.orchestra.run.smithi151.stderr:10+0 records out 2024-05-21T21:44:32.704 INFO:teuthology.orchestra.run.smithi151.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0330947 s, 317 MB/s 2024-05-21T21:44:32.704 INFO:teuthology.orchestra.run.smithi151.stderr:--> Zapping successful for: 2024-05-21T21:44:33.036 INFO:journalctl@ceph.mgr.y.smithi164.stdout:May 21 21:44:32 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-y[37650]: 2024-05-21T21:44:32.645+0000 7f18ed4f5200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-21T21:44:33.036 INFO:journalctl@ceph.mgr.y.smithi164.stdout:May 21 21:44:32 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-y[37650]: 2024-05-21T21:44:32.734+0000 7f18ed4f5200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-21T21:44:33.486 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 21 21:44:33 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-z[38659]: 2024-05-21T21:44:33.010+0000 7f7df830d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-21T21:44:33.486 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 21 21:44:33 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-z[38659]: 2024-05-21T21:44:33.237+0000 7f7df830d200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-21T21:44:33.515 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi151:vg_nvme/lv_4 2024-05-21T21:44:33.968 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:44:33.985 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 21 21:44:33 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-z[38659]: 2024-05-21T21:44:33.670+0000 7f7df830d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-21T21:44:33.985 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 21 21:44:33 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-z[38659]: 2024-05-21T21:44:33.766+0000 7f7df830d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-21T21:44:34.286 INFO:journalctl@ceph.mgr.y.smithi164.stdout:May 21 21:44:33 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-y[37650]: 2024-05-21T21:44:33.985+0000 7f18ed4f5200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-21T21:44:34.286 INFO:journalctl@ceph.mgr.y.smithi164.stdout:May 21 21:44:34 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-y[37650]: 2024-05-21T21:44:34.155+0000 7f18ed4f5200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-21T21:44:34.486 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 21 21:44:34 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-z[38659]: 2024-05-21T21:44:34.143+0000 7f7df830d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-21T21:44:34.619 INFO:journalctl@ceph.mgr.y.smithi164.stdout:May 21 21:44:34 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-y[37650]: 2024-05-21T21:44:34.328+0000 7f18ed4f5200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-21T21:44:34.619 INFO:journalctl@ceph.mgr.y.smithi164.stdout:May 21 21:44:34 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-y[37650]: 2024-05-21T21:44:34.617+0000 7f18ed4f5200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-21T21:44:34.619 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:34 smithi164 ceph-mon[36269]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:34.619 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:34 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:34.619 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:34 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:34.619 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:34 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:34.619 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:34 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:34.619 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:34 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:44:34.619 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:34 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:44:34.619 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:34 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:34.620 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:34 smithi164 ceph-mon[36269]: Reconfiguring mgr.x (unknown last config time)... 2024-05-21T21:44:34.620 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:34 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-21T21:44:34.620 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:34 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-21T21:44:34.620 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:34 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:44:34.620 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:34 smithi164 ceph-mon[36269]: Reconfiguring daemon mgr.x on smithi151 2024-05-21T21:44:34.620 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:34 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:34.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:34 smithi151 ceph-mon[31623]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:34.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:34 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:34.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:34 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:34.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:34 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:34.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:34 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:34.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:34 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:44:34.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:34 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:44:34.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:34 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:34.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:34 smithi151 ceph-mon[31623]: Reconfiguring mgr.x (unknown last config time)... 2024-05-21T21:44:34.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:34 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-21T21:44:34.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:34 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-21T21:44:34.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:34 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:44:34.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:34 smithi151 ceph-mon[31623]: Reconfiguring daemon mgr.x on smithi151 2024-05-21T21:44:34.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:34 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:34.826 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 21 21:44:34 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-z[38659]: 2024-05-21T21:44:34.492+0000 7f7df830d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-21T21:44:34.826 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 21 21:44:34 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-z[38659]: 2024-05-21T21:44:34.584+0000 7f7df830d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-21T21:44:34.827 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:34 smithi184 ceph-mon[36942]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:34.827 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:34 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:34.827 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:34 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:34.827 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:34 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:34.827 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:34 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:34.827 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:34 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:44:34.827 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:34 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:44:34.827 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:34 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:34.827 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:34 smithi184 ceph-mon[36942]: Reconfiguring mgr.x (unknown last config time)... 2024-05-21T21:44:34.827 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:34 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-21T21:44:34.827 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:34 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-21T21:44:34.828 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:34 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:44:34.828 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:34 smithi184 ceph-mon[36942]: Reconfiguring daemon mgr.x on smithi151 2024-05-21T21:44:34.828 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:34 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:34.916 INFO:journalctl@ceph.mgr.y.smithi164.stdout:May 21 21:44:34 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-y[37650]: 2024-05-21T21:44:34.709+0000 7f18ed4f5200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-21T21:44:34.916 INFO:journalctl@ceph.mgr.y.smithi164.stdout:May 21 21:44:34 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-y[37650]: 2024-05-21T21:44:34.914+0000 7f18ed4f5200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-21T21:44:35.236 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 21 21:44:34 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-z[38659]: 2024-05-21T21:44:34.825+0000 7f7df830d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-21T21:44:35.236 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 21 21:44:34 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-z[38659]: 2024-05-21T21:44:34.958+0000 7f7df830d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-21T21:44:35.236 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 21 21:44:35 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-z[38659]: 2024-05-21T21:44:35.053+0000 7f7df830d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-21T21:44:35.285 INFO:journalctl@ceph.mgr.y.smithi164.stdout:May 21 21:44:35 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-y[37650]: 2024-05-21T21:44:35.105+0000 7f18ed4f5200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-21T21:44:35.736 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 21 21:44:35 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-z[38659]: 2024-05-21T21:44:35.384+0000 7f7df830d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-21T21:44:35.736 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 21 21:44:35 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-z[38659]: 2024-05-21T21:44:35.475+0000 7f7df830d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-21T21:44:36.036 INFO:journalctl@ceph.mgr.y.smithi164.stdout:May 21 21:44:35 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-y[37650]: 2024-05-21T21:44:35.711+0000 7f18ed4f5200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-21T21:44:36.036 INFO:journalctl@ceph.mgr.y.smithi164.stdout:May 21 21:44:35 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-y[37650]: 2024-05-21T21:44:35.931+0000 7f18ed4f5200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-21T21:44:36.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:36.163+0000 7f9d34a00700 1 -- 172.21.15.151:0/2119792304 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9d30105140 msgr2=0x7f9d3010b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:44:36.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:36.163+0000 7f9d34a00700 1 --2- 172.21.15.151:0/2119792304 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9d30105140 0x7f9d3010b9f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9d28009fc0 tx=0x7f9d2800b330 comp rx=0 tx=0).stop 2024-05-21T21:44:36.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:36.163+0000 7f9d34a00700 1 -- 172.21.15.151:0/2119792304 shutdown_connections 2024-05-21T21:44:36.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:36.163+0000 7f9d34a00700 1 --2- 172.21.15.151:0/2119792304 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9d30105140 0x7f9d3010b9f0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:44:36.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:36.163+0000 7f9d34a00700 1 --2- 172.21.15.151:0/2119792304 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9d301035a0 0x7f9d301039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:44:36.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:36.163+0000 7f9d34a00700 1 --2- 172.21.15.151:0/2119792304 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d301047a0 0x7f9d30104c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:44:36.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:36.163+0000 7f9d34a00700 1 -- 172.21.15.151:0/2119792304 >> 172.21.15.151:0/2119792304 conn(0x7f9d300feb40 msgr2=0x7f9d30100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:44:36.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:36.163+0000 7f9d34a00700 1 -- 172.21.15.151:0/2119792304 shutdown_connections 2024-05-21T21:44:36.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:36.164+0000 7f9d34a00700 1 -- 172.21.15.151:0/2119792304 wait complete. 2024-05-21T21:44:36.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:36.165+0000 7f9d34a00700 1 Processor -- start 2024-05-21T21:44:36.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:36.165+0000 7f9d34a00700 1 -- start start 2024-05-21T21:44:36.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:36.165+0000 7f9d34a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9d301035a0 0x7f9d3019b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:44:36.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:36.165+0000 7f9d34a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9d301047a0 0x7f9d3019be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:44:36.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:36.165+0000 7f9d34a00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d30105140 0x7f9d301a2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:44:36.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:36.165+0000 7f9d34a00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9d3010f090 con 0x7f9d301047a0 2024-05-21T21:44:36.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:36.165+0000 7f9d34a00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9d3010ef10 con 0x7f9d301035a0 2024-05-21T21:44:36.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:36.165+0000 7f9d34a00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9d3010f210 con 0x7f9d30105140 2024-05-21T21:44:36.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:36.165+0000 7f9d2de00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9d301035a0 0x7f9d3019b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:44:36.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:36.166+0000 7f9d2de00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9d301035a0 0x7f9d3019b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:53648/0 (socket says 172.21.15.151:53648) 2024-05-21T21:44:36.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:36.166+0000 7f9d2e800700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d30105140 0x7f9d301a2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:44:36.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:36.166+0000 7f9d2de00700 1 -- 172.21.15.151:0/3352725711 learned_addr learned my addr 172.21.15.151:0/3352725711 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:44:36.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:36.166+0000 7f9d2d400700 1 --2- 172.21.15.151:0/3352725711 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9d301047a0 0x7f9d3019be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:44:36.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:36.166+0000 7f9d2de00700 1 -- 172.21.15.151:0/3352725711 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d30105140 msgr2=0x7f9d301a2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:44:36.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:36.166+0000 7f9d2de00700 1 --2- 172.21.15.151:0/3352725711 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d30105140 0x7f9d301a2ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:44:36.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:36.166+0000 7f9d2de00700 1 -- 172.21.15.151:0/3352725711 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9d301047a0 msgr2=0x7f9d3019be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:44:36.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:36.166+0000 7f9d2de00700 1 --2- 172.21.15.151:0/3352725711 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9d301047a0 0x7f9d3019be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:44:36.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:36.166+0000 7f9d2de00700 1 -- 172.21.15.151:0/3352725711 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d301a3540 con 0x7f9d301035a0 2024-05-21T21:44:36.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:36.166+0000 7f9d2e800700 1 --2- 172.21.15.151:0/3352725711 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d30105140 0x7f9d301a2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:44:36.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:36.166+0000 7f9d2de00700 1 --2- 172.21.15.151:0/3352725711 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9d301035a0 0x7f9d3019b8e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9d2000e970 tx=0x7f9d2000ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:44:36.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:36.167+0000 7f9d26a00700 1 -- 172.21.15.151:0/3352725711 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d20004220 con 0x7f9d301035a0 2024-05-21T21:44:36.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:36.167+0000 7f9d26a00700 1 -- 172.21.15.151:0/3352725711 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d200043c0 con 0x7f9d301035a0 2024-05-21T21:44:36.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:36.167+0000 7f9d34a00700 1 -- 172.21.15.151:0/3352725711 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d301a3830 con 0x7f9d301035a0 2024-05-21T21:44:36.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:36.167+0000 7f9d26a00700 1 -- 172.21.15.151:0/3352725711 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d20003cb0 con 0x7f9d301035a0 2024-05-21T21:44:36.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:36.167+0000 7f9d34a00700 1 -- 172.21.15.151:0/3352725711 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d301a3d70 con 0x7f9d301035a0 2024-05-21T21:44:36.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:36.168+0000 7f9d34a00700 1 -- 172.21.15.151:0/3352725711 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d3006a230 con 0x7f9d301035a0 2024-05-21T21:44:36.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:36.168+0000 7f9d26a00700 1 -- 172.21.15.151:0/3352725711 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 13) ==== 50279+0+0 (secure 0 0 0) 0x7f9d2000c7c0 con 0x7f9d301035a0 2024-05-21T21:44:36.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:36.169+0000 7f9d26a00700 1 --2- 172.21.15.151:0/3352725711 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f9d1403e750 0x7f9d14040c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:44:36.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:36.169+0000 7f9d26a00700 1 -- 172.21.15.151:0/3352725711 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f9d20052b40 con 0x7f9d301035a0 2024-05-21T21:44:36.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:36.169+0000 7f9d2d400700 1 --2- 172.21.15.151:0/3352725711 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f9d1403e750 0x7f9d14040c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:44:36.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:36.170+0000 7f9d2d400700 1 --2- 172.21.15.151:0/3352725711 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f9d1403e750 0x7f9d14040c10 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c005fd0 tx=0x7f9d1c009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:44:36.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:36.174+0000 7f9d26a00700 1 -- 172.21.15.151:0/3352725711 <== mon.2 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9d200071c0 con 0x7f9d301035a0 2024-05-21T21:44:36.423 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:36.422+0000 7f9d34a00700 1 -- 172.21.15.151:0/3352725711 --> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f9d30076e90 con 0x7f9d1403e750 2024-05-21T21:44:36.486 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 21 21:44:36 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-z[38659]: 2024-05-21T21:44:36.074+0000 7f7df830d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-21T21:44:36.486 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 21 21:44:36 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-z[38659]: 2024-05-21T21:44:36.289+0000 7f7df830d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-21T21:44:36.547 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:36.545+0000 7f9d26a00700 1 -- 172.21.15.151:0/3352725711 <== mon.2 v2:172.21.15.164:3300/0 7 ==== mgrmap(e 14) ==== 99938+0+0 (secure 0 0 0) 0x7f9d2000f3c0 con 0x7f9d301035a0 2024-05-21T21:44:36.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:36 smithi151 ceph-mon[31623]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:36.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:36 smithi151 ceph-mon[31623]: Standby manager daemon z started 2024-05-21T21:44:36.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:36 smithi151 ceph-mon[31623]: from='mgr.? 172.21.15.184:0/1623240176' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-21T21:44:36.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:36 smithi151 ceph-mon[31623]: from='mgr.? 172.21.15.184:0/1623240176' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-21T21:44:36.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:36 smithi151 ceph-mon[31623]: from='mgr.? 172.21.15.184:0/1623240176' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-21T21:44:36.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:36 smithi151 ceph-mon[31623]: from='mgr.? 172.21.15.184:0/1623240176' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-21T21:44:36.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:36 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T21:44:36.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:36 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T21:44:36.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:36 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:44:36.786 INFO:journalctl@ceph.mgr.y.smithi164.stdout:May 21 21:44:36 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-y[37650]: 2024-05-21T21:44:36.353+0000 7f18ed4f5200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-21T21:44:36.786 INFO:journalctl@ceph.mgr.y.smithi164.stdout:May 21 21:44:36 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-y[37650]: 2024-05-21T21:44:36.449+0000 7f18ed4f5200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-21T21:44:36.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:36 smithi164 ceph-mon[36269]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:36.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:36 smithi164 ceph-mon[36269]: Standby manager daemon z started 2024-05-21T21:44:36.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:36 smithi164 ceph-mon[36269]: from='mgr.? 172.21.15.184:0/1623240176' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-21T21:44:36.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:36 smithi164 ceph-mon[36269]: from='mgr.? 172.21.15.184:0/1623240176' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-21T21:44:36.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:36 smithi164 ceph-mon[36269]: from='mgr.? 172.21.15.184:0/1623240176' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-21T21:44:36.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:36 smithi164 ceph-mon[36269]: from='mgr.? 172.21.15.184:0/1623240176' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-21T21:44:36.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:36 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T21:44:36.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:36 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T21:44:36.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:36 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:44:36.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:36 smithi184 ceph-mon[36942]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:36.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:36 smithi184 ceph-mon[36942]: Standby manager daemon z started 2024-05-21T21:44:36.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:36 smithi184 ceph-mon[36942]: from='mgr.? 172.21.15.184:0/1623240176' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-21T21:44:36.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:36 smithi184 ceph-mon[36942]: from='mgr.? 172.21.15.184:0/1623240176' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-21T21:44:36.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:36 smithi184 ceph-mon[36942]: from='mgr.? 172.21.15.184:0/1623240176' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-21T21:44:36.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:36 smithi184 ceph-mon[36942]: from='mgr.? 172.21.15.184:0/1623240176' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-21T21:44:36.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:36 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T21:44:36.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:36 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T21:44:36.987 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:36 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:44:37.182 INFO:journalctl@ceph.mgr.y.smithi164.stdout:May 21 21:44:36 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-y[37650]: 2024-05-21T21:44:36.822+0000 7f18ed4f5200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-21T21:44:37.183 INFO:journalctl@ceph.mgr.y.smithi164.stdout:May 21 21:44:37 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-y[37650]: 2024-05-21T21:44:37.181+0000 7f18ed4f5200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-21T21:44:37.517 INFO:journalctl@ceph.mgr.y.smithi164.stdout:May 21 21:44:37 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-y[37650]: 2024-05-21T21:44:37.275+0000 7f18ed4f5200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-21T21:44:37.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:37 smithi151 ceph-mon[31623]: from='client.24122 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:44:37.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:37 smithi151 ceph-mon[31623]: mgrmap e14: x(active, since 87s), standbys: z 2024-05-21T21:44:37.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:37 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-21T21:44:37.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:37 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:37.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:37 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:37.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:37 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:44:37.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:37 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:44:37.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:37 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:44:37.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:37 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:37.786 INFO:journalctl@ceph.mgr.y.smithi164.stdout:May 21 21:44:37 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-y[37650]: 2024-05-21T21:44:37.516+0000 7f18ed4f5200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-21T21:44:37.786 INFO:journalctl@ceph.mgr.y.smithi164.stdout:May 21 21:44:37 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-y[37650]: 2024-05-21T21:44:37.650+0000 7f18ed4f5200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-21T21:44:37.786 INFO:journalctl@ceph.mgr.y.smithi164.stdout:May 21 21:44:37 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-y[37650]: 2024-05-21T21:44:37.745+0000 7f18ed4f5200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-21T21:44:37.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:37 smithi164 ceph-mon[36269]: from='client.24122 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:44:37.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:37 smithi164 ceph-mon[36269]: mgrmap e14: x(active, since 87s), standbys: z 2024-05-21T21:44:37.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:37 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-21T21:44:37.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:37 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:37.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:37 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:37.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:37 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:44:37.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:37 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:44:37.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:37 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:44:37.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:37 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:37.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:37 smithi184 ceph-mon[36942]: from='client.24122 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:44:37.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:37 smithi184 ceph-mon[36942]: mgrmap e14: x(active, since 87s), standbys: z 2024-05-21T21:44:37.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:37 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-21T21:44:37.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:37 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:37.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:37 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:37.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:37 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:44:37.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:37 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:44:37.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:37 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:44:37.987 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:37 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:38.536 INFO:journalctl@ceph.mgr.y.smithi164.stdout:May 21 21:44:38 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-y[37650]: 2024-05-21T21:44:38.077+0000 7f18ed4f5200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-21T21:44:38.536 INFO:journalctl@ceph.mgr.y.smithi164.stdout:May 21 21:44:38 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-y[37650]: 2024-05-21T21:44:38.168+0000 7f18ed4f5200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-21T21:44:38.795 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:38 smithi151 ceph-mon[31623]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:38.985 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:38 smithi184 ceph-mon[36942]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:38.986 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:38 smithi164 ceph-mon[36269]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:38.986 INFO:journalctl@ceph.mgr.y.smithi164.stdout:May 21 21:44:38 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-y[37650]: 2024-05-21T21:44:38.770+0000 7f18ed4f5200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-21T21:44:39.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:44:39.578+0000 7f9d26a00700 1 -- 172.21.15.151:0/3352725711 <== mon.2 v2:172.21.15.164:3300/0 8 ==== mgrmap(e 15) ==== 149597+0+0 (secure 0 0 0) 0x7f9d20018c50 con 0x7f9d301035a0 2024-05-21T21:44:39.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:39 smithi151 ceph-mon[31623]: Standby manager daemon y started 2024-05-21T21:44:39.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:39 smithi151 ceph-mon[31623]: from='mgr.? 172.21.15.164:0/1708224984' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-21T21:44:39.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:39 smithi151 ceph-mon[31623]: from='mgr.? 172.21.15.164:0/1708224984' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-21T21:44:39.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:39 smithi151 ceph-mon[31623]: from='mgr.? 172.21.15.164:0/1708224984' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-21T21:44:39.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:39 smithi151 ceph-mon[31623]: from='mgr.? 172.21.15.164:0/1708224984' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-21T21:44:39.985 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:39 smithi184 ceph-mon[36942]: Standby manager daemon y started 2024-05-21T21:44:39.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:39 smithi184 ceph-mon[36942]: from='mgr.? 172.21.15.164:0/1708224984' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-21T21:44:39.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:39 smithi184 ceph-mon[36942]: from='mgr.? 172.21.15.164:0/1708224984' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-21T21:44:39.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:39 smithi184 ceph-mon[36942]: from='mgr.? 172.21.15.164:0/1708224984' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-21T21:44:39.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:39 smithi184 ceph-mon[36942]: from='mgr.? 172.21.15.164:0/1708224984' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-21T21:44:40.036 INFO:journalctl@ceph.mgr.y.smithi164.stdout:May 21 21:44:38 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-y[37650]: 2024-05-21T21:44:38.984+0000 7f18ed4f5200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-21T21:44:40.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:39 smithi164 ceph-mon[36269]: Standby manager daemon y started 2024-05-21T21:44:40.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:39 smithi164 ceph-mon[36269]: from='mgr.? 172.21.15.164:0/1708224984' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-21T21:44:40.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:39 smithi164 ceph-mon[36269]: from='mgr.? 172.21.15.164:0/1708224984' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-21T21:44:40.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:39 smithi164 ceph-mon[36269]: from='mgr.? 172.21.15.164:0/1708224984' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-21T21:44:40.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:39 smithi164 ceph-mon[36269]: from='mgr.? 172.21.15.164:0/1708224984' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-21T21:44:40.660 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:40 smithi151 ceph-mon[31623]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:40.660 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:40 smithi151 ceph-mon[31623]: mgrmap e15: x(active, since 90s), standbys: z, y 2024-05-21T21:44:40.660 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:40 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-21T21:44:40.661 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:40 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/2497408078' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "18db8e54-0cfc-4e12-9c84-6f72a99cc989"}]: dispatch 2024-05-21T21:44:40.661 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:40 smithi151 ceph-mon[31623]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "18db8e54-0cfc-4e12-9c84-6f72a99cc989"}]: dispatch 2024-05-21T21:44:40.661 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:40 smithi151 ceph-mon[31623]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "18db8e54-0cfc-4e12-9c84-6f72a99cc989"}]': finished 2024-05-21T21:44:40.661 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:40 smithi151 ceph-mon[31623]: osdmap e5: 1 total, 0 up, 1 in 2024-05-21T21:44:40.661 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:40 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-21T21:44:40.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:40 smithi184 ceph-mon[36942]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:40.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:40 smithi184 ceph-mon[36942]: mgrmap e15: x(active, since 90s), standbys: z, y 2024-05-21T21:44:40.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:40 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-21T21:44:40.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:40 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/2497408078' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "18db8e54-0cfc-4e12-9c84-6f72a99cc989"}]: dispatch 2024-05-21T21:44:40.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:40 smithi184 ceph-mon[36942]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "18db8e54-0cfc-4e12-9c84-6f72a99cc989"}]: dispatch 2024-05-21T21:44:40.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:40 smithi184 ceph-mon[36942]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "18db8e54-0cfc-4e12-9c84-6f72a99cc989"}]': finished 2024-05-21T21:44:40.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:40 smithi184 ceph-mon[36942]: osdmap e5: 1 total, 0 up, 1 in 2024-05-21T21:44:40.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:40 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-21T21:44:41.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:40 smithi164 ceph-mon[36269]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:41.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:40 smithi164 ceph-mon[36269]: mgrmap e15: x(active, since 90s), standbys: z, y 2024-05-21T21:44:41.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:40 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-21T21:44:41.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:40 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/2497408078' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "18db8e54-0cfc-4e12-9c84-6f72a99cc989"}]: dispatch 2024-05-21T21:44:41.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:40 smithi164 ceph-mon[36269]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "18db8e54-0cfc-4e12-9c84-6f72a99cc989"}]: dispatch 2024-05-21T21:44:41.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:40 smithi164 ceph-mon[36269]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "18db8e54-0cfc-4e12-9c84-6f72a99cc989"}]': finished 2024-05-21T21:44:41.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:40 smithi164 ceph-mon[36269]: osdmap e5: 1 total, 0 up, 1 in 2024-05-21T21:44:41.037 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:40 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-21T21:44:41.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:41 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/2023987294' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T21:44:41.985 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:41 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/2023987294' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T21:44:42.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:41 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/2023987294' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T21:44:42.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:42 smithi151 ceph-mon[31623]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:42.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:42 smithi184 ceph-mon[36942]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:43.035 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:42 smithi164 ceph-mon[36269]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:44.907 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:44 smithi151 ceph-mon[31623]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:44.985 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:44 smithi184 ceph-mon[36942]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:45.035 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:44 smithi164 ceph-mon[36269]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:46.932 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:46 smithi151 ceph-mon[31623]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:46.985 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:46 smithi184 ceph-mon[36942]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:47.035 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:46 smithi164 ceph-mon[36269]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:48.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:48 smithi151 ceph-mon[31623]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:48.985 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:48 smithi184 ceph-mon[36942]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:49.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:48 smithi164 ceph-mon[36269]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:50.914 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:50 smithi151 ceph-mon[31623]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:50.914 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:50 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-21T21:44:50.914 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:50 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:44:50.914 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:50 smithi151 ceph-mon[31623]: Deploying daemon osd.0 on smithi151 2024-05-21T21:44:50.985 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:50 smithi184 ceph-mon[36942]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:50.985 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:50 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-21T21:44:50.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:50 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:44:50.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:50 smithi184 ceph-mon[36942]: Deploying daemon osd.0 on smithi151 2024-05-21T21:44:51.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:50 smithi164 ceph-mon[36269]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:51.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:50 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-21T21:44:51.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:50 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:44:51.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:50 smithi164 ceph-mon[36269]: Deploying daemon osd.0 on smithi151 2024-05-21T21:44:52.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:52 smithi151 ceph-mon[31623]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:52.985 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:52 smithi184 ceph-mon[36942]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:53.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:52 smithi164 ceph-mon[36269]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:54.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:54 smithi151 ceph-mon[31623]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:54.985 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:54 smithi184 ceph-mon[36942]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:55.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:54 smithi164 ceph-mon[36269]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:56.950 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:56 smithi151 ceph-mon[31623]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:56.985 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:56 smithi184 ceph-mon[36942]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:57.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:56 smithi164 ceph-mon[36269]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:59.076 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:58 smithi151 ceph-mon[31623]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:59.076 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:58 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:59.076 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:58 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:59.076 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:44:58 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:44:59.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:58 smithi184 ceph-mon[36942]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:59.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:58 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:59.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:58 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:59.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:44:58 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:44:59.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:58 smithi164 ceph-mon[36269]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:44:59.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:58 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:59.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:58 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:44:59.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:44:58 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:45:01.088 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:00 smithi164 ceph-mon[36269]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:45:01.088 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:00 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:01.088 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:00 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:01.089 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:00 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:45:01.089 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:00 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:45:01.089 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:00 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:01.132 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:00 smithi151 ceph-mon[31623]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:45:01.132 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:00 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:01.132 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:00 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:01.132 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:00 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:45:01.132 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:00 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:45:01.133 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:00 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:01.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:00 smithi184 ceph-mon[36942]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:45:01.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:00 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:01.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:00 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:01.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:00 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:45:01.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:00 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:45:01.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:00 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:02.672 INFO:teuthology.orchestra.run.smithi151.stdout:Created osd(s) 0 on host 'smithi151' 2024-05-21T21:45:02.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:45:02.672+0000 7f9d26a00700 1 -- 172.21.15.151:0/3352725711 <== mgr.14152 v2:172.21.15.151:6800/2796622483 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9d30076e90 con 0x7f9d1403e750 2024-05-21T21:45:02.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:45:02.676+0000 7f9d34a00700 1 -- 172.21.15.151:0/3352725711 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f9d1403e750 msgr2=0x7f9d14040c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:45:02.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:45:02.676+0000 7f9d34a00700 1 --2- 172.21.15.151:0/3352725711 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f9d1403e750 0x7f9d14040c10 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c005fd0 tx=0x7f9d1c009310 comp rx=0 tx=0).stop 2024-05-21T21:45:02.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:45:02.676+0000 7f9d34a00700 1 -- 172.21.15.151:0/3352725711 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9d301035a0 msgr2=0x7f9d3019b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:45:02.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:45:02.676+0000 7f9d34a00700 1 --2- 172.21.15.151:0/3352725711 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9d301035a0 0x7f9d3019b8e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9d2000e970 tx=0x7f9d2000ec80 comp rx=0 tx=0).stop 2024-05-21T21:45:02.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:45:02.676+0000 7f9d34a00700 1 -- 172.21.15.151:0/3352725711 shutdown_connections 2024-05-21T21:45:02.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:45:02.676+0000 7f9d34a00700 1 --2- 172.21.15.151:0/3352725711 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f9d1403e750 0x7f9d14040c10 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:45:02.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:45:02.676+0000 7f9d34a00700 1 --2- 172.21.15.151:0/3352725711 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9d301035a0 0x7f9d3019b8e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:45:02.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:45:02.676+0000 7f9d34a00700 1 --2- 172.21.15.151:0/3352725711 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d30105140 0x7f9d301a2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:45:02.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:45:02.676+0000 7f9d34a00700 1 --2- 172.21.15.151:0/3352725711 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9d301047a0 0x7f9d3019be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:45:02.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:45:02.676+0000 7f9d34a00700 1 -- 172.21.15.151:0/3352725711 >> 172.21.15.151:0/3352725711 conn(0x7f9d300feb40 msgr2=0x7f9d30075770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:45:02.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:45:02.676+0000 7f9d34a00700 1 -- 172.21.15.151:0/3352725711 shutdown_connections 2024-05-21T21:45:02.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:45:02.677+0000 7f9d34a00700 1 -- 172.21.15.151:0/3352725711 wait complete. 2024-05-21T21:45:03.047 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:02 smithi151 ceph-mon[31623]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:45:03.047 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:02 smithi151 ceph-mon[31623]: from='osd.0 [v2:172.21.15.151:6802/2420400473,v1:172.21.15.151:6803/2420400473]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-21T21:45:03.047 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:02 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:03.048 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:02 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:03.048 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:02 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:45:03.238 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:02 smithi184 ceph-mon[36942]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:45:03.238 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:02 smithi184 ceph-mon[36942]: from='osd.0 [v2:172.21.15.151:6802/2420400473,v1:172.21.15.151:6803/2420400473]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-21T21:45:03.238 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:02 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:03.238 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:02 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:03.238 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:02 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:45:03.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:02 smithi164 ceph-mon[36269]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:45:03.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:02 smithi164 ceph-mon[36269]: from='osd.0 [v2:172.21.15.151:6802/2420400473,v1:172.21.15.151:6803/2420400473]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-21T21:45:03.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:02 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:03.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:02 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:03.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:02 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:45:03.407 DEBUG:teuthology.orchestra.run.smithi151:osd.0> sudo journalctl -f -n 0 -u ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@osd.0.service 2024-05-21T21:45:03.412 INFO:tasks.cephadm:Deploying osd.1 on smithi164 with /dev/vg_nvme/lv_4... 2024-05-21T21:45:03.412 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-21T21:45:03.657 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.b/config 2024-05-21T21:45:03.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:03 smithi151 ceph-mon[31623]: from='osd.0 [v2:172.21.15.151:6802/2420400473,v1:172.21.15.151:6803/2420400473]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-21T21:45:03.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:03 smithi151 ceph-mon[31623]: osdmap e6: 1 total, 0 up, 1 in 2024-05-21T21:45:03.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:03 smithi151 ceph-mon[31623]: from='osd.0 [v2:172.21.15.151:6802/2420400473,v1:172.21.15.151:6803/2420400473]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-05-21T21:45:03.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:03 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-21T21:45:04.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:03 smithi164 ceph-mon[36269]: from='osd.0 [v2:172.21.15.151:6802/2420400473,v1:172.21.15.151:6803/2420400473]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-21T21:45:04.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:03 smithi164 ceph-mon[36269]: osdmap e6: 1 total, 0 up, 1 in 2024-05-21T21:45:04.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:03 smithi164 ceph-mon[36269]: from='osd.0 [v2:172.21.15.151:6802/2420400473,v1:172.21.15.151:6803/2420400473]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-05-21T21:45:04.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:03 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-21T21:45:04.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:03 smithi184 ceph-mon[36942]: from='osd.0 [v2:172.21.15.151:6802/2420400473,v1:172.21.15.151:6803/2420400473]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-21T21:45:04.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:03 smithi184 ceph-mon[36942]: osdmap e6: 1 total, 0 up, 1 in 2024-05-21T21:45:04.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:03 smithi184 ceph-mon[36942]: from='osd.0 [v2:172.21.15.151:6802/2420400473,v1:172.21.15.151:6803/2420400473]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-05-21T21:45:04.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:03 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-21T21:45:04.911 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:04 smithi164 ceph-mon[36269]: purged_snaps scrub starts 2024-05-21T21:45:04.911 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:04 smithi164 ceph-mon[36269]: purged_snaps scrub ok 2024-05-21T21:45:04.911 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:04 smithi164 ceph-mon[36269]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:45:04.911 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:04 smithi164 ceph-mon[36269]: from='osd.0 [v2:172.21.15.151:6802/2420400473,v1:172.21.15.151:6803/2420400473]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-05-21T21:45:04.911 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:04 smithi164 ceph-mon[36269]: osdmap e7: 1 total, 0 up, 1 in 2024-05-21T21:45:04.912 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:04 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-21T21:45:04.912 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:04 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-21T21:45:04.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:04 smithi151 ceph-mon[31623]: purged_snaps scrub starts 2024-05-21T21:45:04.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:04 smithi151 ceph-mon[31623]: purged_snaps scrub ok 2024-05-21T21:45:04.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:04 smithi151 ceph-mon[31623]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:45:04.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:04 smithi151 ceph-mon[31623]: from='osd.0 [v2:172.21.15.151:6802/2420400473,v1:172.21.15.151:6803/2420400473]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-05-21T21:45:04.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:04 smithi151 ceph-mon[31623]: osdmap e7: 1 total, 0 up, 1 in 2024-05-21T21:45:04.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:04 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-21T21:45:04.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:04 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-21T21:45:04.954 INFO:journalctl@ceph.osd.0.smithi151.stdout:May 21 21:45:04 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-0[44060]: 2024-05-21T21:45:04.861+0000 7f8013800700 -1 osd.0 0 waiting for initial osdmap 2024-05-21T21:45:04.954 INFO:journalctl@ceph.osd.0.smithi151.stdout:May 21 21:45:04 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-0[44060]: 2024-05-21T21:45:04.895+0000 7f8009600700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-21T21:45:05.235 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:04 smithi184 ceph-mon[36942]: purged_snaps scrub starts 2024-05-21T21:45:05.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:04 smithi184 ceph-mon[36942]: purged_snaps scrub ok 2024-05-21T21:45:05.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:04 smithi184 ceph-mon[36942]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:45:05.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:04 smithi184 ceph-mon[36942]: from='osd.0 [v2:172.21.15.151:6802/2420400473,v1:172.21.15.151:6803/2420400473]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-05-21T21:45:05.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:04 smithi184 ceph-mon[36942]: osdmap e7: 1 total, 0 up, 1 in 2024-05-21T21:45:05.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:04 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-21T21:45:05.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:04 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-21T21:45:05.350 INFO:teuthology.orchestra.run.smithi164.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-21T21:45:05.350 INFO:teuthology.orchestra.run.smithi164.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-21T21:45:05.351 INFO:teuthology.orchestra.run.smithi164.stderr: stderr: 10+0 records in 2024-05-21T21:45:05.351 INFO:teuthology.orchestra.run.smithi164.stderr:10+0 records out 2024-05-21T21:45:05.351 INFO:teuthology.orchestra.run.smithi164.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0114555 s, 915 MB/s 2024-05-21T21:45:05.351 INFO:teuthology.orchestra.run.smithi164.stderr: stderr: 2024-05-21T21:45:05.351 INFO:teuthology.orchestra.run.smithi164.stderr:--> Zapping successful for: 2024-05-21T21:45:05.933 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi164:vg_nvme/lv_4 2024-05-21T21:45:06.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:05 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-21T21:45:06.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:05 smithi164 ceph-mon[36269]: from='osd.0 [v2:172.21.15.151:6802/2420400473,v1:172.21.15.151:6803/2420400473]' entity='osd.0' 2024-05-21T21:45:06.165 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.b/config 2024-05-21T21:45:06.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:05 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-21T21:45:06.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:05 smithi151 ceph-mon[31623]: from='osd.0 [v2:172.21.15.151:6802/2420400473,v1:172.21.15.151:6803/2420400473]' entity='osd.0' 2024-05-21T21:45:06.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:05 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-21T21:45:06.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:05 smithi184 ceph-mon[36942]: from='osd.0 [v2:172.21.15.151:6802/2420400473,v1:172.21.15.151:6803/2420400473]' entity='osd.0' 2024-05-21T21:45:07.086 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:06 smithi151 ceph-mon[31623]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:45:07.086 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:06 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-21T21:45:07.086 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:06 smithi151 ceph-mon[31623]: osd.0 [v2:172.21.15.151:6802/2420400473,v1:172.21.15.151:6803/2420400473] boot 2024-05-21T21:45:07.086 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:06 smithi151 ceph-mon[31623]: osdmap e8: 1 total, 1 up, 1 in 2024-05-21T21:45:07.086 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:06 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-21T21:45:07.103 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:06 smithi164 ceph-mon[36269]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:45:07.104 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:06 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-21T21:45:07.104 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:06 smithi164 ceph-mon[36269]: osd.0 [v2:172.21.15.151:6802/2420400473,v1:172.21.15.151:6803/2420400473] boot 2024-05-21T21:45:07.104 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:06 smithi164 ceph-mon[36269]: osdmap e8: 1 total, 1 up, 1 in 2024-05-21T21:45:07.104 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:06 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-21T21:45:07.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:06 smithi184 ceph-mon[36942]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-21T21:45:07.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:06 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-21T21:45:07.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:06 smithi184 ceph-mon[36942]: osd.0 [v2:172.21.15.151:6802/2420400473,v1:172.21.15.151:6803/2420400473] boot 2024-05-21T21:45:07.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:06 smithi184 ceph-mon[36942]: osdmap e8: 1 total, 1 up, 1 in 2024-05-21T21:45:07.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:06 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-21T21:45:07.646 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:45:07.644+0000 7f0628342700 1 -- 172.21.15.164:0/2696894088 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0620105140 msgr2=0x7f062010b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:45:07.646 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:45:07.644+0000 7f0628342700 1 --2- 172.21.15.164:0/2696894088 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0620105140 0x7f062010b9f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f061800b0d0 tx=0x7f061800b3e0 comp rx=0 tx=0).stop 2024-05-21T21:45:07.646 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:45:07.645+0000 7f0628342700 1 -- 172.21.15.164:0/2696894088 shutdown_connections 2024-05-21T21:45:07.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:45:07.645+0000 7f0628342700 1 --2- 172.21.15.164:0/2696894088 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0620105140 0x7f062010b9f0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:45:07.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:45:07.645+0000 7f0628342700 1 --2- 172.21.15.164:0/2696894088 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f06201035a0 0x7f06201039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:45:07.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:45:07.645+0000 7f0628342700 1 --2- 172.21.15.164:0/2696894088 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f06201047a0 0x7f0620104c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:45:07.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:45:07.645+0000 7f0628342700 1 -- 172.21.15.164:0/2696894088 >> 172.21.15.164:0/2696894088 conn(0x7f06200feb40 msgr2=0x7f0620100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:45:07.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:45:07.645+0000 7f0628342700 1 -- 172.21.15.164:0/2696894088 shutdown_connections 2024-05-21T21:45:07.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:45:07.645+0000 7f0628342700 1 -- 172.21.15.164:0/2696894088 wait complete. 2024-05-21T21:45:07.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:45:07.645+0000 7f0628342700 1 Processor -- start 2024-05-21T21:45:07.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:45:07.645+0000 7f0628342700 1 -- start start 2024-05-21T21:45:07.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:45:07.645+0000 7f0628342700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f06201035a0 0x7f06200772e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:45:07.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:45:07.645+0000 7f0628342700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f06201047a0 0x7f0620075930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:45:07.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:45:07.645+0000 7f0628342700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0620105140 0x7f0620075e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:45:07.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:45:07.645+0000 7f0628342700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f062010f0b0 con 0x7f06201035a0 2024-05-21T21:45:07.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:45:07.645+0000 7f0628342700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f062010ef30 con 0x7f06201047a0 2024-05-21T21:45:07.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:45:07.645+0000 7f0628342700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f062010f230 con 0x7f0620105140 2024-05-21T21:45:07.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:45:07.646+0000 7f06258dd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f06201047a0 0x7f0620075930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:45:07.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:45:07.646+0000 7f06258dd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f06201047a0 0x7f0620075930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59726/0 (socket says 172.21.15.164:59726) 2024-05-21T21:45:07.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:45:07.646+0000 7f06258dd700 1 -- 172.21.15.164:0/2575767352 learned_addr learned my addr 172.21.15.164:0/2575767352 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-21T21:45:07.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:45:07.646+0000 7f06268df700 1 --2- 172.21.15.164:0/2575767352 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0620105140 0x7f0620075e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:45:07.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:45:07.646+0000 7f06258dd700 1 -- 172.21.15.164:0/2575767352 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0620105140 msgr2=0x7f0620075e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:45:07.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:45:07.646+0000 7f06258dd700 1 --2- 172.21.15.164:0/2575767352 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0620105140 0x7f0620075e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:45:07.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:45:07.646+0000 7f06258dd700 1 -- 172.21.15.164:0/2575767352 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f06201035a0 msgr2=0x7f06200772e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:45:07.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:45:07.646+0000 7f06258dd700 1 --2- 172.21.15.164:0/2575767352 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f06201035a0 0x7f06200772e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:45:07.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:45:07.646+0000 7f06258dd700 1 -- 172.21.15.164:0/2575767352 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0620076720 con 0x7f06201047a0 2024-05-21T21:45:07.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:45:07.646+0000 7f06258dd700 1 --2- 172.21.15.164:0/2575767352 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f06201047a0 0x7f0620075930 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f061c00b530 tx=0x7f061c00b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:45:07.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:45:07.646+0000 7f06177fe700 1 -- 172.21.15.164:0/2575767352 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f061c015020 con 0x7f06201047a0 2024-05-21T21:45:07.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:45:07.646+0000 7f0628342700 1 -- 172.21.15.164:0/2575767352 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06201a3890 con 0x7f06201047a0 2024-05-21T21:45:07.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:45:07.646+0000 7f06177fe700 1 -- 172.21.15.164:0/2575767352 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f061c011ce0 con 0x7f06201047a0 2024-05-21T21:45:07.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:45:07.646+0000 7f06177fe700 1 -- 172.21.15.164:0/2575767352 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f061c010d20 con 0x7f06201047a0 2024-05-21T21:45:07.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:45:07.646+0000 7f0628342700 1 -- 172.21.15.164:0/2575767352 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06201a3e50 con 0x7f06201047a0 2024-05-21T21:45:07.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:45:07.647+0000 7f0628342700 1 -- 172.21.15.164:0/2575767352 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f062006a230 con 0x7f06201047a0 2024-05-21T21:45:07.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:45:07.648+0000 7f06177fe700 1 -- 172.21.15.164:0/2575767352 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 15) ==== 149597+0+0 (secure 0 0 0) 0x7f061c018020 con 0x7f06201047a0 2024-05-21T21:45:07.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:45:07.649+0000 7f06177fe700 1 --2- 172.21.15.164:0/2575767352 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f060c0b27f0 0x7f060c0b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:45:07.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:45:07.649+0000 7f06177fe700 1 -- 172.21.15.164:0/2575767352 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7f061c0dffa0 con 0x7f06201047a0 2024-05-21T21:45:07.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:45:07.650+0000 7f06260de700 1 --2- 172.21.15.164:0/2575767352 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f060c0b27f0 0x7f060c0b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:45:07.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:45:07.651+0000 7f06260de700 1 --2- 172.21.15.164:0/2575767352 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f060c0b27f0 0x7f060c0b4cb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0620104600 tx=0x7f0610005ca0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:45:07.654 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:45:07.654+0000 7f06177fe700 1 -- 172.21.15.164:0/2575767352 <== mon.2 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f061c01f020 con 0x7f06201047a0 2024-05-21T21:45:07.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:45:07.867+0000 7f0628342700 1 -- 172.21.15.164:0/2575767352 --> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f0620063980 con 0x7f060c0b27f0 2024-05-21T21:45:08.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:07 smithi151 ceph-mon[31623]: osdmap e9: 1 total, 1 up, 1 in 2024-05-21T21:45:08.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:07 smithi151 ceph-mon[31623]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-21T21:45:08.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:07 smithi151 ceph-mon[31623]: Detected new or changed devices on smithi151 2024-05-21T21:45:08.204 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:07 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:08.204 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:07 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:08.204 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:07 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:45:08.204 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:07 smithi151 ceph-mon[31623]: Adjusting osd_memory_target on smithi151 to 17089M 2024-05-21T21:45:08.204 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:07 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:08.204 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:07 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:45:08.204 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:07 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:45:08.204 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:07 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:08.204 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:07 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T21:45:08.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:07 smithi184 ceph-mon[36942]: osdmap e9: 1 total, 1 up, 1 in 2024-05-21T21:45:08.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:07 smithi184 ceph-mon[36942]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-21T21:45:08.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:07 smithi184 ceph-mon[36942]: Detected new or changed devices on smithi151 2024-05-21T21:45:08.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:07 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:08.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:07 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:08.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:07 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:45:08.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:07 smithi184 ceph-mon[36942]: Adjusting osd_memory_target on smithi151 to 17089M 2024-05-21T21:45:08.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:07 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:08.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:07 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:45:08.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:07 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:45:08.237 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:07 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:08.237 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:07 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T21:45:08.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:07 smithi164 ceph-mon[36269]: osdmap e9: 1 total, 1 up, 1 in 2024-05-21T21:45:08.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:07 smithi164 ceph-mon[36269]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-21T21:45:08.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:07 smithi164 ceph-mon[36269]: Detected new or changed devices on smithi151 2024-05-21T21:45:08.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:07 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:08.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:07 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:08.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:07 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:45:08.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:07 smithi164 ceph-mon[36269]: Adjusting osd_memory_target on smithi151 to 17089M 2024-05-21T21:45:08.287 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:07 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:08.287 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:07 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:45:08.287 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:07 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:45:08.287 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:07 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:08.287 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:07 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T21:45:09.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:08 smithi164 ceph-mon[36269]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:45:09.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:08 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T21:45:09.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:08 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:45:09.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:08 smithi151 ceph-mon[31623]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:45:09.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:08 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T21:45:09.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:08 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:45:09.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:08 smithi184 ceph-mon[36942]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:45:09.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:08 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T21:45:09.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:08 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:45:10.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:09 smithi164 ceph-mon[36269]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-21T21:45:10.202 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:09 smithi151 ceph-mon[31623]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-21T21:45:10.235 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:09 smithi184 ceph-mon[36942]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-21T21:45:11.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:10 smithi164 ceph-mon[36269]: from='client.? 172.21.15.164:0/274849218' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bac384c7-1102-4815-81a8-10cc7661d230"}]: dispatch 2024-05-21T21:45:11.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:10 smithi164 ceph-mon[36269]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bac384c7-1102-4815-81a8-10cc7661d230"}]: dispatch 2024-05-21T21:45:11.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:10 smithi164 ceph-mon[36269]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bac384c7-1102-4815-81a8-10cc7661d230"}]': finished 2024-05-21T21:45:11.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:10 smithi164 ceph-mon[36269]: osdmap e10: 2 total, 1 up, 2 in 2024-05-21T21:45:11.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:10 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-21T21:45:11.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:10 smithi151 ceph-mon[31623]: from='client.? 172.21.15.164:0/274849218' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bac384c7-1102-4815-81a8-10cc7661d230"}]: dispatch 2024-05-21T21:45:11.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:10 smithi151 ceph-mon[31623]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bac384c7-1102-4815-81a8-10cc7661d230"}]: dispatch 2024-05-21T21:45:11.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:10 smithi151 ceph-mon[31623]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bac384c7-1102-4815-81a8-10cc7661d230"}]': finished 2024-05-21T21:45:11.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:10 smithi151 ceph-mon[31623]: osdmap e10: 2 total, 1 up, 2 in 2024-05-21T21:45:11.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:10 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-21T21:45:11.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:10 smithi184 ceph-mon[36942]: from='client.? 172.21.15.164:0/274849218' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bac384c7-1102-4815-81a8-10cc7661d230"}]: dispatch 2024-05-21T21:45:11.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:10 smithi184 ceph-mon[36942]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bac384c7-1102-4815-81a8-10cc7661d230"}]: dispatch 2024-05-21T21:45:11.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:10 smithi184 ceph-mon[36942]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bac384c7-1102-4815-81a8-10cc7661d230"}]': finished 2024-05-21T21:45:11.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:10 smithi184 ceph-mon[36942]: osdmap e10: 2 total, 1 up, 2 in 2024-05-21T21:45:11.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:10 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-21T21:45:12.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:11 smithi151 ceph-mon[31623]: from='client.? 172.21.15.164:0/494398337' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T21:45:12.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:11 smithi151 ceph-mon[31623]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-21T21:45:12.238 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:11 smithi184 ceph-mon[36942]: from='client.? 172.21.15.164:0/494398337' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T21:45:12.238 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:11 smithi184 ceph-mon[36942]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-21T21:45:12.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:11 smithi164 ceph-mon[36269]: from='client.? 172.21.15.164:0/494398337' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T21:45:12.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:11 smithi164 ceph-mon[36269]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-21T21:45:14.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:14 smithi151 ceph-mon[31623]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-21T21:45:14.485 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:14 smithi184 ceph-mon[36942]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-21T21:45:14.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:14 smithi164 ceph-mon[36269]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-21T21:45:16.431 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:16 smithi164 ceph-mon[36269]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-21T21:45:16.452 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:16 smithi151 ceph-mon[31623]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-21T21:45:16.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:16 smithi184 ceph-mon[36942]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-21T21:45:18.436 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:18 smithi164 ceph-mon[36269]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-21T21:45:18.437 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:18 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-21T21:45:18.437 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:18 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:45:18.437 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:18 smithi164 ceph-mon[36269]: Deploying daemon osd.1 on smithi164 2024-05-21T21:45:18.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:18 smithi151 ceph-mon[31623]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-21T21:45:18.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:18 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-21T21:45:18.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:18 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:45:18.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:18 smithi151 ceph-mon[31623]: Deploying daemon osd.1 on smithi164 2024-05-21T21:45:18.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:18 smithi184 ceph-mon[36942]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-21T21:45:18.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:18 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-21T21:45:18.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:18 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:45:18.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:18 smithi184 ceph-mon[36942]: Deploying daemon osd.1 on smithi164 2024-05-21T21:45:20.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:20 smithi151 ceph-mon[31623]: pgmap v65: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-21T21:45:20.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:20 smithi184 ceph-mon[36942]: pgmap v65: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-21T21:45:20.503 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:20 smithi164 ceph-mon[36269]: pgmap v65: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-21T21:45:22.429 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:22 smithi164 ceph-mon[36269]: pgmap v66: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-21T21:45:22.452 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:22 smithi151 ceph-mon[31623]: pgmap v66: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-21T21:45:22.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:22 smithi184 ceph-mon[36942]: pgmap v66: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-21T21:45:24.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:23 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:24.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:23 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:24.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:23 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:45:24.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:23 smithi151 ceph-mon[31623]: pgmap v67: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-21T21:45:24.235 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:23 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:24.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:23 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:24.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:23 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:45:24.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:23 smithi184 ceph-mon[36942]: pgmap v67: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-21T21:45:24.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:23 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:24.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:23 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:24.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:23 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:45:24.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:23 smithi164 ceph-mon[36269]: pgmap v67: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-21T21:45:26.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:26 smithi151 ceph-mon[31623]: pgmap v68: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-21T21:45:26.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:26 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:26.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:26 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:26.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:26 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:45:26.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:26 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:45:26.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:26 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:26.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:45:26.455+0000 7f06177fe700 1 -- 172.21.15.164:0/2575767352 <== mgr.14152 v2:172.21.15.151:6800/2796622483 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0620063980 con 0x7f060c0b27f0 2024-05-21T21:45:26.455 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:26 smithi164 ceph-mon[36269]: pgmap v68: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-21T21:45:26.455 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:26 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:26.456 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:26 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:26.456 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:26 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:45:26.456 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:26 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:45:26.456 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:26 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:26.456 INFO:teuthology.orchestra.run.smithi164.stdout:Created osd(s) 1 on host 'smithi164' 2024-05-21T21:45:26.458 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:45:26.457+0000 7f0628342700 1 -- 172.21.15.164:0/2575767352 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f060c0b27f0 msgr2=0x7f060c0b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:45:26.458 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:45:26.457+0000 7f0628342700 1 --2- 172.21.15.164:0/2575767352 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f060c0b27f0 0x7f060c0b4cb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0620104600 tx=0x7f0610005ca0 comp rx=0 tx=0).stop 2024-05-21T21:45:26.458 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:45:26.457+0000 7f0628342700 1 -- 172.21.15.164:0/2575767352 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f06201047a0 msgr2=0x7f0620075930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:45:26.458 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:45:26.457+0000 7f0628342700 1 --2- 172.21.15.164:0/2575767352 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f06201047a0 0x7f0620075930 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f061c00b530 tx=0x7f061c00b8f0 comp rx=0 tx=0).stop 2024-05-21T21:45:26.458 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:45:26.457+0000 7f0628342700 1 -- 172.21.15.164:0/2575767352 shutdown_connections 2024-05-21T21:45:26.458 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:45:26.457+0000 7f0628342700 1 --2- 172.21.15.164:0/2575767352 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f060c0b27f0 0x7f060c0b4cb0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:45:26.458 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:45:26.457+0000 7f0628342700 1 --2- 172.21.15.164:0/2575767352 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0620105140 0x7f0620075e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:45:26.458 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:45:26.457+0000 7f0628342700 1 --2- 172.21.15.164:0/2575767352 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f06201035a0 0x7f06200772e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:45:26.458 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:45:26.457+0000 7f0628342700 1 --2- 172.21.15.164:0/2575767352 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f06201047a0 0x7f0620075930 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:45:26.458 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:45:26.457+0000 7f0628342700 1 -- 172.21.15.164:0/2575767352 >> 172.21.15.164:0/2575767352 conn(0x7f06200feb40 msgr2=0x7f0620107b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:45:26.458 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:45:26.457+0000 7f0628342700 1 -- 172.21.15.164:0/2575767352 shutdown_connections 2024-05-21T21:45:26.458 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:45:26.458+0000 7f0628342700 1 -- 172.21.15.164:0/2575767352 wait complete. 2024-05-21T21:45:26.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:26 smithi184 ceph-mon[36942]: pgmap v68: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-21T21:45:26.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:26 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:26.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:26 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:26.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:26 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:45:26.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:26 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:45:26.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:26 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:27.098 DEBUG:teuthology.orchestra.run.smithi164:osd.1> sudo journalctl -f -n 0 -u ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@osd.1.service 2024-05-21T21:45:27.101 INFO:tasks.cephadm:Deploying osd.2 on smithi184 with /dev/vg_nvme/lv_4... 2024-05-21T21:45:27.101 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-21T21:45:27.345 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.c/config 2024-05-21T21:45:27.680 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:27 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:27.680 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:27 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:45:27.681 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:27 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:27.681 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:27 smithi184 ceph-mon[36942]: from='osd.1 [v2:172.21.15.164:6800/3900240447,v1:172.21.15.164:6801/3900240447]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-21T21:45:27.681 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:27 smithi184 ceph-mon[36942]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-21T21:45:27.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:27 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:27.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:27 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:45:27.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:27 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:27.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:27 smithi151 ceph-mon[31623]: from='osd.1 [v2:172.21.15.164:6800/3900240447,v1:172.21.15.164:6801/3900240447]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-21T21:45:27.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:27 smithi151 ceph-mon[31623]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-21T21:45:27.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:27 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:27.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:27 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:45:27.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:27 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:27.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:27 smithi164 ceph-mon[36269]: from='osd.1 [v2:172.21.15.164:6800/3900240447,v1:172.21.15.164:6801/3900240447]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-21T21:45:27.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:27 smithi164 ceph-mon[36269]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-21T21:45:28.697 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:28 smithi184 ceph-mon[36942]: pgmap v69: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-21T21:45:28.697 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:28 smithi184 ceph-mon[36942]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-21T21:45:28.697 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:28 smithi184 ceph-mon[36942]: from='osd.1 [v2:172.21.15.164:6800/3900240447,v1:172.21.15.164:6801/3900240447]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-05-21T21:45:28.698 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:28 smithi184 ceph-mon[36942]: osdmap e11: 2 total, 1 up, 2 in 2024-05-21T21:45:28.698 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:28 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-21T21:45:28.698 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:28 smithi184 ceph-mon[36942]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-05-21T21:45:28.722 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:28 smithi164 ceph-mon[36269]: pgmap v69: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-21T21:45:28.722 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:28 smithi164 ceph-mon[36269]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-21T21:45:28.722 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:28 smithi164 ceph-mon[36269]: from='osd.1 [v2:172.21.15.164:6800/3900240447,v1:172.21.15.164:6801/3900240447]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-05-21T21:45:28.723 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:28 smithi164 ceph-mon[36269]: osdmap e11: 2 total, 1 up, 2 in 2024-05-21T21:45:28.723 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:28 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-21T21:45:28.723 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:28 smithi164 ceph-mon[36269]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-05-21T21:45:28.723 INFO:journalctl@ceph.osd.1.smithi164.stdout:May 21 21:45:28 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-1[40856]: 2024-05-21T21:45:28.456+0000 7f5f53c73700 -1 osd.1 0 waiting for initial osdmap 2024-05-21T21:45:28.723 INFO:journalctl@ceph.osd.1.smithi164.stdout:May 21 21:45:28 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-1[40856]: 2024-05-21T21:45:28.459+0000 7f5f4fa66700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-21T21:45:28.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:28 smithi151 ceph-mon[31623]: pgmap v69: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-21T21:45:28.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:28 smithi151 ceph-mon[31623]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-21T21:45:28.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:28 smithi151 ceph-mon[31623]: from='osd.1 [v2:172.21.15.164:6800/3900240447,v1:172.21.15.164:6801/3900240447]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-05-21T21:45:28.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:28 smithi151 ceph-mon[31623]: osdmap e11: 2 total, 1 up, 2 in 2024-05-21T21:45:28.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:28 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-21T21:45:28.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:28 smithi151 ceph-mon[31623]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-05-21T21:45:29.243 INFO:teuthology.orchestra.run.smithi184.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-21T21:45:29.243 INFO:teuthology.orchestra.run.smithi184.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-21T21:45:29.243 INFO:teuthology.orchestra.run.smithi184.stderr: stderr: 10+0 records in 2024-05-21T21:45:29.243 INFO:teuthology.orchestra.run.smithi184.stderr:10+0 records out 2024-05-21T21:45:29.243 INFO:teuthology.orchestra.run.smithi184.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.01174 s, 893 MB/s 2024-05-21T21:45:29.243 INFO:teuthology.orchestra.run.smithi184.stderr:--> Zapping successful for: 2024-05-21T21:45:29.518 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:29 smithi184 ceph-mon[36942]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-05-21T21:45:29.518 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:29 smithi184 ceph-mon[36942]: osdmap e12: 2 total, 1 up, 2 in 2024-05-21T21:45:29.519 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:29 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-21T21:45:29.519 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:29 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-21T21:45:29.755 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:29 smithi164 ceph-mon[36269]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-05-21T21:45:29.756 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:29 smithi164 ceph-mon[36269]: osdmap e12: 2 total, 1 up, 2 in 2024-05-21T21:45:29.756 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:29 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-21T21:45:29.756 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:29 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-21T21:45:29.782 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi184:vg_nvme/lv_4 2024-05-21T21:45:29.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:29 smithi151 ceph-mon[31623]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-05-21T21:45:29.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:29 smithi151 ceph-mon[31623]: osdmap e12: 2 total, 1 up, 2 in 2024-05-21T21:45:29.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:29 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-21T21:45:29.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:29 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-21T21:45:30.025 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.c/config 2024-05-21T21:45:30.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:30 smithi184 ceph-mon[36942]: purged_snaps scrub starts 2024-05-21T21:45:30.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:30 smithi184 ceph-mon[36942]: purged_snaps scrub ok 2024-05-21T21:45:30.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:30 smithi184 ceph-mon[36942]: pgmap v72: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-21T21:45:30.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:30 smithi184 ceph-mon[36942]: osd.1 [v2:172.21.15.164:6800/3900240447,v1:172.21.15.164:6801/3900240447] boot 2024-05-21T21:45:30.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:30 smithi184 ceph-mon[36942]: osdmap e13: 2 total, 2 up, 2 in 2024-05-21T21:45:30.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:30 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-21T21:45:30.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:30 smithi184 ceph-mon[36942]: Detected new or changed devices on smithi164 2024-05-21T21:45:30.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:30 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:30.737 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:30 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:30.737 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:30 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:45:30.737 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:30 smithi184 ceph-mon[36942]: Adjusting osd_memory_target on smithi164 to 17088M 2024-05-21T21:45:30.737 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:30 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:30.737 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:30 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:45:30.737 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:30 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:45:30.737 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:30 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:30.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:30 smithi164 ceph-mon[36269]: purged_snaps scrub starts 2024-05-21T21:45:30.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:30 smithi164 ceph-mon[36269]: purged_snaps scrub ok 2024-05-21T21:45:30.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:30 smithi164 ceph-mon[36269]: pgmap v72: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-21T21:45:30.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:30 smithi164 ceph-mon[36269]: osd.1 [v2:172.21.15.164:6800/3900240447,v1:172.21.15.164:6801/3900240447] boot 2024-05-21T21:45:30.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:30 smithi164 ceph-mon[36269]: osdmap e13: 2 total, 2 up, 2 in 2024-05-21T21:45:30.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:30 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-21T21:45:30.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:30 smithi164 ceph-mon[36269]: Detected new or changed devices on smithi164 2024-05-21T21:45:30.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:30 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:30.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:30 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:30.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:30 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:45:30.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:30 smithi164 ceph-mon[36269]: Adjusting osd_memory_target on smithi164 to 17088M 2024-05-21T21:45:30.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:30 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:30.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:30 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:45:30.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:30 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:45:30.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:30 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:30.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:30 smithi151 ceph-mon[31623]: purged_snaps scrub starts 2024-05-21T21:45:30.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:30 smithi151 ceph-mon[31623]: purged_snaps scrub ok 2024-05-21T21:45:30.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:30 smithi151 ceph-mon[31623]: pgmap v72: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-21T21:45:30.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:30 smithi151 ceph-mon[31623]: osd.1 [v2:172.21.15.164:6800/3900240447,v1:172.21.15.164:6801/3900240447] boot 2024-05-21T21:45:30.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:30 smithi151 ceph-mon[31623]: osdmap e13: 2 total, 2 up, 2 in 2024-05-21T21:45:30.954 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:30 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-21T21:45:30.954 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:30 smithi151 ceph-mon[31623]: Detected new or changed devices on smithi164 2024-05-21T21:45:30.954 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:30 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:30.954 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:30 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:30.954 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:30 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:45:30.954 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:30 smithi151 ceph-mon[31623]: Adjusting osd_memory_target on smithi164 to 17088M 2024-05-21T21:45:30.954 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:30 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:30.954 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:30 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:45:30.954 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:30 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:45:30.954 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:30 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:31.509 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:45:31.507+0000 7fd4debba700 1 -- 172.21.15.184:0/4031802137 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4d8105140 msgr2=0x7fd4d810b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:45:31.509 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:45:31.507+0000 7fd4debba700 1 --2- 172.21.15.184:0/4031802137 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4d8105140 0x7fd4d810b9f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd4d000b0a0 tx=0x7fd4d000b3b0 comp rx=0 tx=0).stop 2024-05-21T21:45:31.509 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:45:31.507+0000 7fd4debba700 1 -- 172.21.15.184:0/4031802137 shutdown_connections 2024-05-21T21:45:31.509 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:45:31.507+0000 7fd4debba700 1 --2- 172.21.15.184:0/4031802137 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4d81035a0 0x7fd4d81039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:45:31.509 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:45:31.507+0000 7fd4debba700 1 --2- 172.21.15.184:0/4031802137 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4d8105140 0x7fd4d810b9f0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:45:31.509 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:45:31.507+0000 7fd4debba700 1 --2- 172.21.15.184:0/4031802137 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd4d81047a0 0x7fd4d8104c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:45:31.510 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:45:31.507+0000 7fd4debba700 1 -- 172.21.15.184:0/4031802137 >> 172.21.15.184:0/4031802137 conn(0x7fd4d80feb40 msgr2=0x7fd4d8100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:45:31.510 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:45:31.507+0000 7fd4debba700 1 -- 172.21.15.184:0/4031802137 shutdown_connections 2024-05-21T21:45:31.510 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:45:31.507+0000 7fd4debba700 1 -- 172.21.15.184:0/4031802137 wait complete. 2024-05-21T21:45:31.510 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:45:31.507+0000 7fd4debba700 1 Processor -- start 2024-05-21T21:45:31.510 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:45:31.507+0000 7fd4debba700 1 -- start start 2024-05-21T21:45:31.510 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:45:31.507+0000 7fd4debba700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd4d81035a0 0x7fd4d8199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:45:31.510 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:45:31.507+0000 7fd4debba700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4d81047a0 0x7fd4d8199d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:45:31.510 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:45:31.507+0000 7fd4debba700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4d8105140 0x7fd4d81a0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:45:31.510 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:45:31.508+0000 7fd4debba700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d810f130 con 0x7fd4d81035a0 2024-05-21T21:45:31.510 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:45:31.508+0000 7fd4debba700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d810efb0 con 0x7fd4d8105140 2024-05-21T21:45:31.510 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:45:31.508+0000 7fd4debba700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d810f2b0 con 0x7fd4d81047a0 2024-05-21T21:45:31.510 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:45:31.508+0000 7fd4d7fff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4d81047a0 0x7fd4d8199d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:45:31.511 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:45:31.508+0000 7fd4d7fff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4d81047a0 0x7fd4d8199d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:48612/0 (socket says 172.21.15.184:48612) 2024-05-21T21:45:31.511 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:45:31.508+0000 7fd4d7fff700 1 -- 172.21.15.184:0/1742796981 learned_addr learned my addr 172.21.15.184:0/1742796981 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-05-21T21:45:31.511 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:45:31.508+0000 7fd4dd157700 1 --2- 172.21.15.184:0/1742796981 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4d8105140 0x7fd4d81a0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:45:31.511 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:45:31.508+0000 7fd4d7fff700 1 -- 172.21.15.184:0/1742796981 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4d8105140 msgr2=0x7fd4d81a0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:45:31.511 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:45:31.508+0000 7fd4d7fff700 1 --2- 172.21.15.184:0/1742796981 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4d8105140 0x7fd4d81a0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:45:31.511 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:45:31.508+0000 7fd4d7fff700 1 -- 172.21.15.184:0/1742796981 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd4d81035a0 msgr2=0x7fd4d8199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:45:31.511 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:45:31.508+0000 7fd4d7fff700 1 --2- 172.21.15.184:0/1742796981 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd4d81035a0 0x7fd4d8199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:45:31.511 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:45:31.508+0000 7fd4d7fff700 1 -- 172.21.15.184:0/1742796981 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4d81a1490 con 0x7fd4d81047a0 2024-05-21T21:45:31.511 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:45:31.508+0000 7fd4d7fff700 1 --2- 172.21.15.184:0/1742796981 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4d81047a0 0x7fd4d8199d70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd4c400b560 tx=0x7fd4c400b870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:45:31.511 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:45:31.508+0000 7fd4d5ffb700 1 -- 172.21.15.184:0/1742796981 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c4015020 con 0x7fd4d81047a0 2024-05-21T21:45:31.511 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:45:31.508+0000 7fd4d5ffb700 1 -- 172.21.15.184:0/1742796981 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4c4011ce0 con 0x7fd4d81047a0 2024-05-21T21:45:31.511 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:45:31.508+0000 7fd4debba700 1 -- 172.21.15.184:0/1742796981 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4d80757d0 con 0x7fd4d81047a0 2024-05-21T21:45:31.511 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:45:31.508+0000 7fd4d5ffb700 1 -- 172.21.15.184:0/1742796981 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c4010d20 con 0x7fd4d81047a0 2024-05-21T21:45:31.511 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:45:31.508+0000 7fd4debba700 1 -- 172.21.15.184:0/1742796981 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4d8075d90 con 0x7fd4d81047a0 2024-05-21T21:45:31.512 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:45:31.509+0000 7fd4debba700 1 -- 172.21.15.184:0/1742796981 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4d806a230 con 0x7fd4d81047a0 2024-05-21T21:45:31.512 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:45:31.510+0000 7fd4d5ffb700 1 -- 172.21.15.184:0/1742796981 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 15) ==== 149597+0+0 (secure 0 0 0) 0x7fd4c4018020 con 0x7fd4d81047a0 2024-05-21T21:45:31.513 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:45:31.511+0000 7fd4d5ffb700 1 --2- 172.21.15.184:0/1742796981 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fd4c80b27f0 0x7fd4c80b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:45:31.513 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:45:31.511+0000 7fd4d5ffb700 1 -- 172.21.15.184:0/1742796981 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7fd4c40e00e0 con 0x7fd4d81047a0 2024-05-21T21:45:31.513 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:45:31.512+0000 7fd4dc956700 1 --2- 172.21.15.184:0/1742796981 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fd4c80b27f0 0x7fd4c80b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:45:31.513 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:45:31.512+0000 7fd4dc956700 1 --2- 172.21.15.184:0/1742796981 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fd4c80b27f0 0x7fd4c80b4cb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd4d8104600 tx=0x7fd4cc006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:45:31.516 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:45:31.516+0000 7fd4d5ffb700 1 -- 172.21.15.184:0/1742796981 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd4c401f020 con 0x7fd4d81047a0 2024-05-21T21:45:31.726 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:45:31.723+0000 7fd4debba700 1 -- 172.21.15.184:0/1742796981 --> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fd4d8063980 con 0x7fd4c80b27f0 2024-05-21T21:45:31.726 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:31 smithi184 ceph-mon[36942]: osdmap e14: 2 total, 2 up, 2 in 2024-05-21T21:45:31.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:31 smithi164 ceph-mon[36269]: osdmap e14: 2 total, 2 up, 2 in 2024-05-21T21:45:31.952 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:31 smithi151 ceph-mon[31623]: osdmap e14: 2 total, 2 up, 2 in 2024-05-21T21:45:32.656 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:32 smithi184 ceph-mon[36942]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T21:45:32.656 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:32 smithi184 ceph-mon[36942]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:45:32.656 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:32 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T21:45:32.656 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:32 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T21:45:32.657 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:32 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:45:32.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:32 smithi164 ceph-mon[36269]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T21:45:32.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:32 smithi164 ceph-mon[36269]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:45:32.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:32 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T21:45:32.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:32 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T21:45:32.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:32 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:45:32.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:32 smithi151 ceph-mon[31623]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T21:45:32.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:32 smithi151 ceph-mon[31623]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:45:32.954 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:32 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T21:45:32.954 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:32 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T21:45:32.954 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:32 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:45:34.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:34 smithi164 ceph-mon[36269]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T21:45:34.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:34 smithi164 ceph-mon[36269]: from='client.? 172.21.15.184:0/2550692392' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b7e4a782-1518-4d43-bee9-7fd768ee5cb4"}]: dispatch 2024-05-21T21:45:34.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:34 smithi164 ceph-mon[36269]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b7e4a782-1518-4d43-bee9-7fd768ee5cb4"}]: dispatch 2024-05-21T21:45:34.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:34 smithi164 ceph-mon[36269]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b7e4a782-1518-4d43-bee9-7fd768ee5cb4"}]': finished 2024-05-21T21:45:34.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:34 smithi164 ceph-mon[36269]: osdmap e15: 3 total, 2 up, 3 in 2024-05-21T21:45:34.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:34 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-21T21:45:34.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:34 smithi151 ceph-mon[31623]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T21:45:34.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:34 smithi151 ceph-mon[31623]: from='client.? 172.21.15.184:0/2550692392' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b7e4a782-1518-4d43-bee9-7fd768ee5cb4"}]: dispatch 2024-05-21T21:45:34.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:34 smithi151 ceph-mon[31623]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b7e4a782-1518-4d43-bee9-7fd768ee5cb4"}]: dispatch 2024-05-21T21:45:34.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:34 smithi151 ceph-mon[31623]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b7e4a782-1518-4d43-bee9-7fd768ee5cb4"}]': finished 2024-05-21T21:45:34.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:34 smithi151 ceph-mon[31623]: osdmap e15: 3 total, 2 up, 3 in 2024-05-21T21:45:34.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:34 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-21T21:45:34.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:34 smithi184 ceph-mon[36942]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T21:45:34.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:34 smithi184 ceph-mon[36942]: from='client.? 172.21.15.184:0/2550692392' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b7e4a782-1518-4d43-bee9-7fd768ee5cb4"}]: dispatch 2024-05-21T21:45:34.987 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:34 smithi184 ceph-mon[36942]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b7e4a782-1518-4d43-bee9-7fd768ee5cb4"}]: dispatch 2024-05-21T21:45:34.987 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:34 smithi184 ceph-mon[36942]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b7e4a782-1518-4d43-bee9-7fd768ee5cb4"}]': finished 2024-05-21T21:45:34.987 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:34 smithi184 ceph-mon[36942]: osdmap e15: 3 total, 2 up, 3 in 2024-05-21T21:45:34.987 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:34 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-21T21:45:35.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:35 smithi164 ceph-mon[36269]: from='client.? 172.21.15.184:0/858385858' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T21:45:35.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:35 smithi151 ceph-mon[31623]: from='client.? 172.21.15.184:0/858385858' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T21:45:35.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:35 smithi184 ceph-mon[36942]: from='client.? 172.21.15.184:0/858385858' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T21:45:36.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:36 smithi164 ceph-mon[36269]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T21:45:36.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:36 smithi151 ceph-mon[31623]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T21:45:36.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:36 smithi184 ceph-mon[36942]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T21:45:38.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:38 smithi184 ceph-mon[36942]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T21:45:38.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:38 smithi164 ceph-mon[36269]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T21:45:38.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:38 smithi151 ceph-mon[31623]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T21:45:40.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:40 smithi184 ceph-mon[36942]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T21:45:40.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:40 smithi164 ceph-mon[36269]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T21:45:40.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:40 smithi151 ceph-mon[31623]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T21:45:42.642 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:42 smithi184 ceph-mon[36942]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T21:45:42.642 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:42 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-21T21:45:42.643 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:42 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:45:42.643 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:42 smithi184 ceph-mon[36942]: Deploying daemon osd.2 on smithi184 2024-05-21T21:45:42.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:42 smithi164 ceph-mon[36269]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T21:45:42.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:42 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-21T21:45:42.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:42 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:45:42.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:42 smithi164 ceph-mon[36269]: Deploying daemon osd.2 on smithi184 2024-05-21T21:45:42.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:42 smithi151 ceph-mon[31623]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T21:45:42.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:42 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-21T21:45:42.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:42 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:45:42.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:42 smithi151 ceph-mon[31623]: Deploying daemon osd.2 on smithi184 2024-05-21T21:45:44.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:44 smithi184 ceph-mon[36942]: pgmap v82: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T21:45:44.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:44 smithi164 ceph-mon[36269]: pgmap v82: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T21:45:44.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:44 smithi151 ceph-mon[31623]: pgmap v82: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T21:45:46.718 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:46 smithi184 ceph-mon[36942]: pgmap v83: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T21:45:46.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:46 smithi151 ceph-mon[31623]: pgmap v83: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T21:45:47.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:46 smithi164 ceph-mon[36269]: pgmap v83: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T21:45:47.671 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:47 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:45:47.671 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:47 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:47.671 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:47 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:47.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:47 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:45:47.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:47 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:47.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:47 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:48.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:47 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:45:48.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:47 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:48.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:47 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:48.706 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:48 smithi184 ceph-mon[36942]: pgmap v84: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T21:45:48.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:48 smithi151 ceph-mon[31623]: pgmap v84: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T21:45:49.035 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:48 smithi164 ceph-mon[36269]: pgmap v84: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T21:45:50.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:50 smithi151 ceph-mon[31623]: pgmap v85: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T21:45:50.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:50 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:50.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:50 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:50.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:50 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:45:50.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:50 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:45:50.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:50 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:50.455 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:50 smithi184 ceph-mon[36942]: pgmap v85: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T21:45:50.455 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:50 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:50.455 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:50 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:50.456 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:50 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:45:50.456 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:50 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:45:50.456 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:50 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:50.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:50 smithi164 ceph-mon[36269]: pgmap v85: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T21:45:50.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:50 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:50.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:50 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:50.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:50 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:45:50.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:50 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:45:50.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:50 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:50.556 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:45:50.555+0000 7fd4d5ffb700 1 -- 172.21.15.184:0/1742796981 <== mgr.14152 v2:172.21.15.151:6800/2796622483 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd4d8063980 con 0x7fd4c80b27f0 2024-05-21T21:45:50.557 INFO:teuthology.orchestra.run.smithi184.stdout:Created osd(s) 2 on host 'smithi184' 2024-05-21T21:45:50.559 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:45:50.558+0000 7fd4debba700 1 -- 172.21.15.184:0/1742796981 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fd4c80b27f0 msgr2=0x7fd4c80b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:45:50.559 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:45:50.558+0000 7fd4debba700 1 --2- 172.21.15.184:0/1742796981 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fd4c80b27f0 0x7fd4c80b4cb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd4d8104600 tx=0x7fd4cc006d20 comp rx=0 tx=0).stop 2024-05-21T21:45:50.559 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:45:50.558+0000 7fd4debba700 1 -- 172.21.15.184:0/1742796981 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4d81047a0 msgr2=0x7fd4d8199d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:45:50.559 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:45:50.558+0000 7fd4debba700 1 --2- 172.21.15.184:0/1742796981 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4d81047a0 0x7fd4d8199d70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd4c400b560 tx=0x7fd4c400b870 comp rx=0 tx=0).stop 2024-05-21T21:45:50.559 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:45:50.558+0000 7fd4debba700 1 -- 172.21.15.184:0/1742796981 shutdown_connections 2024-05-21T21:45:50.559 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:45:50.558+0000 7fd4debba700 1 --2- 172.21.15.184:0/1742796981 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fd4c80b27f0 0x7fd4c80b4cb0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:45:50.559 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:45:50.558+0000 7fd4debba700 1 --2- 172.21.15.184:0/1742796981 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4d8105140 0x7fd4d81a0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:45:50.559 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:45:50.558+0000 7fd4debba700 1 --2- 172.21.15.184:0/1742796981 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd4d81035a0 0x7fd4d8199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:45:50.559 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:45:50.558+0000 7fd4debba700 1 --2- 172.21.15.184:0/1742796981 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4d81047a0 0x7fd4d8199d70 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:45:50.560 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:45:50.558+0000 7fd4debba700 1 -- 172.21.15.184:0/1742796981 >> 172.21.15.184:0/1742796981 conn(0x7fd4d80feb40 msgr2=0x7fd4d8107b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:45:50.560 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:45:50.558+0000 7fd4debba700 1 -- 172.21.15.184:0/1742796981 shutdown_connections 2024-05-21T21:45:50.560 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:45:50.558+0000 7fd4debba700 1 -- 172.21.15.184:0/1742796981 wait complete. 2024-05-21T21:45:51.151 DEBUG:teuthology.orchestra.run.smithi184:osd.2> sudo journalctl -f -n 0 -u ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@osd.2.service 2024-05-21T21:45:51.153 INFO:tasks.cephadm:Deploying osd.3 on smithi151 with /dev/vg_nvme/lv_3... 2024-05-21T21:45:51.153 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-21T21:45:51.588 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:45:51.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:51 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:51.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:51 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:51.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:51 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:45:51.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:51 smithi184 ceph-mon[36942]: from='osd.2 [v2:172.21.15.184:6800/3971150737,v1:172.21.15.184:6801/3971150737]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-21T21:45:51.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:51 smithi184 ceph-mon[36942]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-21T21:45:51.772 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:51 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:51.772 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:51 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:51.772 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:51 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:45:51.773 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:51 smithi151 ceph-mon[31623]: from='osd.2 [v2:172.21.15.184:6800/3971150737,v1:172.21.15.184:6801/3971150737]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-21T21:45:51.773 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:51 smithi151 ceph-mon[31623]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-21T21:45:51.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:51 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:51.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:51 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:51.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:51 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:45:51.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:51 smithi164 ceph-mon[36269]: from='osd.2 [v2:172.21.15.184:6800/3971150737,v1:172.21.15.184:6801/3971150737]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-21T21:45:51.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:51 smithi164 ceph-mon[36269]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-21T21:45:52.669 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:52 smithi184 ceph-mon[36942]: pgmap v86: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T21:45:52.669 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:52 smithi184 ceph-mon[36942]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-21T21:45:52.669 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:52 smithi184 ceph-mon[36942]: from='osd.2 [v2:172.21.15.184:6800/3971150737,v1:172.21.15.184:6801/3971150737]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-05-21T21:45:52.670 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:52 smithi184 ceph-mon[36942]: osdmap e16: 3 total, 2 up, 3 in 2024-05-21T21:45:52.670 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:52 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-21T21:45:52.670 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:52 smithi184 ceph-mon[36942]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-05-21T21:45:52.782 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:52 smithi151 ceph-mon[31623]: pgmap v86: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T21:45:52.782 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:52 smithi151 ceph-mon[31623]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-21T21:45:52.782 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:52 smithi151 ceph-mon[31623]: from='osd.2 [v2:172.21.15.184:6800/3971150737,v1:172.21.15.184:6801/3971150737]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-05-21T21:45:52.782 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:52 smithi151 ceph-mon[31623]: osdmap e16: 3 total, 2 up, 3 in 2024-05-21T21:45:52.782 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:52 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-21T21:45:52.783 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:52 smithi151 ceph-mon[31623]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-05-21T21:45:52.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:52 smithi164 ceph-mon[36269]: pgmap v86: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T21:45:52.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:52 smithi164 ceph-mon[36269]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-21T21:45:52.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:52 smithi164 ceph-mon[36269]: from='osd.2 [v2:172.21.15.184:6800/3971150737,v1:172.21.15.184:6801/3971150737]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-05-21T21:45:52.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:52 smithi164 ceph-mon[36269]: osdmap e16: 3 total, 2 up, 3 in 2024-05-21T21:45:52.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:52 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-21T21:45:52.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:52 smithi164 ceph-mon[36269]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-05-21T21:45:53.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:53 smithi151 ceph-mon[31623]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2024-05-21T21:45:53.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:53 smithi151 ceph-mon[31623]: osdmap e17: 3 total, 2 up, 3 in 2024-05-21T21:45:53.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:53 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-21T21:45:53.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:53 smithi151 ceph-mon[31623]: from='osd.2 ' entity='osd.2' 2024-05-21T21:45:53.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:53 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-21T21:45:53.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:53 smithi184 ceph-mon[36942]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2024-05-21T21:45:53.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:53 smithi184 ceph-mon[36942]: osdmap e17: 3 total, 2 up, 3 in 2024-05-21T21:45:53.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:53 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-21T21:45:53.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:53 smithi184 ceph-mon[36942]: from='osd.2 ' entity='osd.2' 2024-05-21T21:45:53.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:53 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-21T21:45:53.736 INFO:journalctl@ceph.osd.2.smithi184.stdout:May 21 21:45:53 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-2[42060]: 2024-05-21T21:45:53.442+0000 7fe498519700 -1 osd.2 0 waiting for initial osdmap 2024-05-21T21:45:53.736 INFO:journalctl@ceph.osd.2.smithi184.stdout:May 21 21:45:53 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-2[42060]: 2024-05-21T21:45:53.600+0000 7fe490304700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-21T21:45:54.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:53 smithi164 ceph-mon[36269]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2024-05-21T21:45:54.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:53 smithi164 ceph-mon[36269]: osdmap e17: 3 total, 2 up, 3 in 2024-05-21T21:45:54.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:53 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-21T21:45:54.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:53 smithi164 ceph-mon[36269]: from='osd.2 ' entity='osd.2' 2024-05-21T21:45:54.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:53 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-21T21:45:54.324 INFO:teuthology.orchestra.run.smithi151.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-21T21:45:54.325 INFO:teuthology.orchestra.run.smithi151.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-21T21:45:54.325 INFO:teuthology.orchestra.run.smithi151.stderr: stderr: 10+0 records in 2024-05-21T21:45:54.325 INFO:teuthology.orchestra.run.smithi151.stderr:10+0 records out 2024-05-21T21:45:54.325 INFO:teuthology.orchestra.run.smithi151.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0524949 s, 200 MB/s 2024-05-21T21:45:54.325 INFO:teuthology.orchestra.run.smithi151.stderr:--> Zapping successful for: 2024-05-21T21:45:54.910 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:54 smithi151 ceph-mon[31623]: purged_snaps scrub starts 2024-05-21T21:45:54.910 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:54 smithi151 ceph-mon[31623]: purged_snaps scrub ok 2024-05-21T21:45:54.910 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:54 smithi151 ceph-mon[31623]: pgmap v89: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T21:45:54.910 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:54 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:54.910 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:54 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:54.910 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:54 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:45:54.911 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:54 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:54.911 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:54 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:45:54.911 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:54 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:45:54.911 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:54 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:54.911 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:54 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-21T21:45:54.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:54 smithi184 ceph-mon[36942]: purged_snaps scrub starts 2024-05-21T21:45:54.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:54 smithi184 ceph-mon[36942]: purged_snaps scrub ok 2024-05-21T21:45:54.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:54 smithi184 ceph-mon[36942]: pgmap v89: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T21:45:54.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:54 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:54.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:54 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:54.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:54 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:45:54.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:54 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:54.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:54 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:45:54.987 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:54 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:45:54.987 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:54 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:54.987 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:54 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-21T21:45:55.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:54 smithi164 ceph-mon[36269]: purged_snaps scrub starts 2024-05-21T21:45:55.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:54 smithi164 ceph-mon[36269]: purged_snaps scrub ok 2024-05-21T21:45:55.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:54 smithi164 ceph-mon[36269]: pgmap v89: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T21:45:55.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:54 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:55.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:54 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:55.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:54 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:45:55.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:54 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:55.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:54 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:45:55.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:54 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:45:55.037 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:54 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:55.037 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:54 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-21T21:45:55.145 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi151:vg_nvme/lv_3 2024-05-21T21:45:55.545 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:45:55.560 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:55 smithi151 ceph-mon[31623]: Detected new or changed devices on smithi184 2024-05-21T21:45:55.560 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:55 smithi151 ceph-mon[31623]: Adjusting osd_memory_target on smithi184 to 17088M 2024-05-21T21:45:55.560 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:55 smithi151 ceph-mon[31623]: osd.2 [v2:172.21.15.184:6800/3971150737,v1:172.21.15.184:6801/3971150737] boot 2024-05-21T21:45:55.560 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:55 smithi151 ceph-mon[31623]: osdmap e18: 3 total, 3 up, 3 in 2024-05-21T21:45:55.560 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:55 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-21T21:45:55.561 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:55 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-21T21:45:55.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:55 smithi184 ceph-mon[36942]: Detected new or changed devices on smithi184 2024-05-21T21:45:55.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:55 smithi184 ceph-mon[36942]: Adjusting osd_memory_target on smithi184 to 17088M 2024-05-21T21:45:55.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:55 smithi184 ceph-mon[36942]: osd.2 [v2:172.21.15.184:6800/3971150737,v1:172.21.15.184:6801/3971150737] boot 2024-05-21T21:45:55.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:55 smithi184 ceph-mon[36942]: osdmap e18: 3 total, 3 up, 3 in 2024-05-21T21:45:55.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:55 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-21T21:45:55.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:55 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-21T21:45:56.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:55 smithi164 ceph-mon[36269]: Detected new or changed devices on smithi184 2024-05-21T21:45:56.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:55 smithi164 ceph-mon[36269]: Adjusting osd_memory_target on smithi184 to 17088M 2024-05-21T21:45:56.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:55 smithi164 ceph-mon[36269]: osd.2 [v2:172.21.15.184:6800/3971150737,v1:172.21.15.184:6801/3971150737] boot 2024-05-21T21:45:56.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:55 smithi164 ceph-mon[36269]: osdmap e18: 3 total, 3 up, 3 in 2024-05-21T21:45:56.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:55 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-21T21:45:56.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:55 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-21T21:45:56.778 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:56 smithi151 ceph-mon[31623]: pgmap v91: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T21:45:56.778 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:56 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-21T21:45:56.778 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:56 smithi151 ceph-mon[31623]: osdmap e19: 3 total, 3 up, 3 in 2024-05-21T21:45:56.778 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:56 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-21T21:45:56.778 INFO:journalctl@ceph.osd.0.smithi151.stdout:May 21 21:45:56 smithi151 sudo[48226]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-21T21:45:56.779 INFO:journalctl@ceph.osd.0.smithi151.stdout:May 21 21:45:56 smithi151 sudo[48226]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-21T21:45:56.779 INFO:journalctl@ceph.osd.0.smithi151.stdout:May 21 21:45:56 smithi151 sudo[48226]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-21T21:45:56.779 INFO:journalctl@ceph.osd.0.smithi151.stdout:May 21 21:45:56 smithi151 sudo[48226]: pam_unix(sudo:session): session closed for user root 2024-05-21T21:45:56.842 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:56 smithi164 ceph-mon[36269]: pgmap v91: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T21:45:56.843 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:56 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-21T21:45:56.843 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:56 smithi164 ceph-mon[36269]: osdmap e19: 3 total, 3 up, 3 in 2024-05-21T21:45:56.843 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:56 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-21T21:45:56.930 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:56 smithi184 ceph-mon[36942]: pgmap v91: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-21T21:45:56.930 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:56 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-21T21:45:56.930 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:56 smithi184 ceph-mon[36942]: osdmap e19: 3 total, 3 up, 3 in 2024-05-21T21:45:56.930 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:56 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-21T21:45:57.033 INFO:journalctl@ceph.osd.0.smithi151.stdout:May 21 21:45:56 smithi151 sudo[48231]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-21T21:45:57.034 INFO:journalctl@ceph.osd.0.smithi151.stdout:May 21 21:45:56 smithi151 sudo[48231]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-21T21:45:57.034 INFO:journalctl@ceph.osd.0.smithi151.stdout:May 21 21:45:56 smithi151 sudo[48231]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-21T21:45:57.034 INFO:journalctl@ceph.osd.0.smithi151.stdout:May 21 21:45:56 smithi151 sudo[48231]: pam_unix(sudo:session): session closed for user root 2024-05-21T21:45:57.146 INFO:journalctl@ceph.osd.1.smithi164.stdout:May 21 21:45:56 smithi164 sudo[44698]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-21T21:45:57.146 INFO:journalctl@ceph.osd.1.smithi164.stdout:May 21 21:45:56 smithi164 sudo[44698]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-21T21:45:57.146 INFO:journalctl@ceph.osd.1.smithi164.stdout:May 21 21:45:56 smithi164 sudo[44698]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-21T21:45:57.146 INFO:journalctl@ceph.osd.1.smithi164.stdout:May 21 21:45:56 smithi164 sudo[44698]: pam_unix(sudo:session): session closed for user root 2024-05-21T21:45:57.147 INFO:journalctl@ceph.osd.1.smithi164.stdout:May 21 21:45:56 smithi164 sudo[44701]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-21T21:45:57.147 INFO:journalctl@ceph.osd.1.smithi164.stdout:May 21 21:45:56 smithi164 sudo[44701]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-21T21:45:57.147 INFO:journalctl@ceph.osd.1.smithi164.stdout:May 21 21:45:56 smithi164 sudo[44701]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-21T21:45:57.147 INFO:journalctl@ceph.osd.1.smithi164.stdout:May 21 21:45:56 smithi164 sudo[44701]: pam_unix(sudo:session): session closed for user root 2024-05-21T21:45:57.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:57 smithi184 sudo[46003]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-21T21:45:57.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:57 smithi184 sudo[46003]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-21T21:45:57.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:57 smithi184 sudo[46003]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-21T21:45:57.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:57 smithi184 sudo[46003]: pam_unix(sudo:session): session closed for user root 2024-05-21T21:45:57.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:57 smithi184 sudo[46006]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-21T21:45:57.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:57 smithi184 sudo[46006]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-21T21:45:57.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:57 smithi184 sudo[46006]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-21T21:45:57.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:57 smithi184 sudo[46006]: pam_unix(sudo:session): session closed for user root 2024-05-21T21:45:57.236 INFO:journalctl@ceph.osd.2.smithi184.stdout:May 21 21:45:56 smithi184 sudo[45997]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-21T21:45:57.237 INFO:journalctl@ceph.osd.2.smithi184.stdout:May 21 21:45:56 smithi184 sudo[45997]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-21T21:45:57.237 INFO:journalctl@ceph.osd.2.smithi184.stdout:May 21 21:45:56 smithi184 sudo[45997]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-21T21:45:57.237 INFO:journalctl@ceph.osd.2.smithi184.stdout:May 21 21:45:56 smithi184 sudo[45997]: pam_unix(sudo:session): session closed for user root 2024-05-21T21:45:57.237 INFO:journalctl@ceph.osd.2.smithi184.stdout:May 21 21:45:56 smithi184 sudo[46000]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-21T21:45:57.237 INFO:journalctl@ceph.osd.2.smithi184.stdout:May 21 21:45:56 smithi184 sudo[46000]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-21T21:45:57.237 INFO:journalctl@ceph.osd.2.smithi184.stdout:May 21 21:45:56 smithi184 sudo[46000]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-21T21:45:57.237 INFO:journalctl@ceph.osd.2.smithi184.stdout:May 21 21:45:56 smithi184 sudo[46000]: pam_unix(sudo:session): session closed for user root 2024-05-21T21:45:57.331 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:57 smithi151 sudo[48241]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-21T21:45:57.331 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:57 smithi151 sudo[48241]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-21T21:45:57.331 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:57 smithi151 sudo[48241]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-21T21:45:57.331 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:57 smithi151 sudo[48241]: pam_unix(sudo:session): session closed for user root 2024-05-21T21:45:57.331 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:57 smithi151 sudo[48255]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-21T21:45:57.331 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:57 smithi151 sudo[48255]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-21T21:45:57.331 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:57 smithi151 sudo[48255]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-21T21:45:57.331 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:57 smithi151 sudo[48255]: pam_unix(sudo:session): session closed for user root 2024-05-21T21:45:57.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:57 smithi164 sudo[44704]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-21T21:45:57.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:57 smithi164 sudo[44704]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-21T21:45:57.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:57 smithi164 sudo[44704]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-21T21:45:57.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:57 smithi164 sudo[44704]: pam_unix(sudo:session): session closed for user root 2024-05-21T21:45:57.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:57 smithi164 sudo[44707]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-21T21:45:57.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:57 smithi164 sudo[44707]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-21T21:45:57.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:57 smithi164 sudo[44707]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-21T21:45:57.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:57 smithi164 sudo[44707]: pam_unix(sudo:session): session closed for user root 2024-05-21T21:45:57.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:45:57.626+0000 7f9292400700 1 -- 172.21.15.151:0/442245164 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f928c075f40 msgr2=0x7f928c0763c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:45:57.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:45:57.626+0000 7f9292400700 1 --2- 172.21.15.151:0/442245164 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f928c075f40 0x7f928c0763c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f92780098f0 tx=0x7f9278009c00 comp rx=0 tx=0).stop 2024-05-21T21:45:57.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:45:57.626+0000 7f9292400700 1 -- 172.21.15.151:0/442245164 shutdown_connections 2024-05-21T21:45:57.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:45:57.626+0000 7f9292400700 1 --2- 172.21.15.151:0/442245164 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f928c10b480 0x7f928c10d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:45:57.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:45:57.626+0000 7f9292400700 1 --2- 172.21.15.151:0/442245164 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f928c077510 0x7f928c075970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:45:57.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:45:57.626+0000 7f9292400700 1 --2- 172.21.15.151:0/442245164 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f928c075f40 0x7f928c0763c0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:45:57.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:45:57.626+0000 7f9292400700 1 -- 172.21.15.151:0/442245164 >> 172.21.15.151:0/442245164 conn(0x7f928c0feb40 msgr2=0x7f928c100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:45:57.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:45:57.626+0000 7f9292400700 1 -- 172.21.15.151:0/442245164 shutdown_connections 2024-05-21T21:45:57.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:45:57.626+0000 7f9292400700 1 -- 172.21.15.151:0/442245164 wait complete. 2024-05-21T21:45:57.628 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:57 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-21T21:45:57.628 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:57 smithi151 ceph-mon[31623]: osdmap e20: 3 total, 3 up, 3 in 2024-05-21T21:45:57.629 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:57 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:57.629 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:57 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:57.629 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:57 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:57.629 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:57 smithi151 ceph-mon[31623]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-21T21:45:57.629 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:57 smithi151 ceph-mon[31623]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-21T21:45:57.629 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:57 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-21T21:45:57.629 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:57 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T21:45:57.629 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:57 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T21:45:57.629 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:57 smithi151 ceph-mon[31623]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-21T21:45:57.629 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:57 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-21T21:45:57.629 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:57 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T21:45:57.629 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:57 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T21:45:57.630 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:57 smithi151 ceph-mon[31623]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-21T21:45:57.630 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:57 smithi151 ceph-mon[31623]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-21T21:45:57.630 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:57 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-21T21:45:57.630 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:57 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T21:45:57.630 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:57 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T21:45:57.630 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:57 smithi151 ceph-mon[31623]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-21T21:45:57.630 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:57 smithi151 ceph-mon[31623]: osdmap e21: 3 total, 3 up, 3 in 2024-05-21T21:45:57.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:45:57.627+0000 7f9292400700 1 Processor -- start 2024-05-21T21:45:57.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:45:57.628+0000 7f9292400700 1 -- start start 2024-05-21T21:45:57.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:45:57.628+0000 7f9292400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f928c075f40 0x7f928c19b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:45:57.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:45:57.628+0000 7f9292400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f928c077510 0x7f928c19bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:45:57.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:45:57.628+0000 7f9292400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f928c10b480 0x7f928c1a2f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:45:57.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:45:57.628+0000 7f9292400700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f928c111270 con 0x7f928c075f40 2024-05-21T21:45:57.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:45:57.628+0000 7f9292400700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f928c1110f0 con 0x7f928c10b480 2024-05-21T21:45:57.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:45:57.628+0000 7f9292400700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f928c1113f0 con 0x7f928c077510 2024-05-21T21:45:57.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:45:57.629+0000 7f928b400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f928c075f40 0x7f928c19b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:45:57.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:45:57.629+0000 7f928b400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f928c075f40 0x7f928c19b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41314/0 (socket says 172.21.15.151:41314) 2024-05-21T21:45:57.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:45:57.629+0000 7f928b400700 1 -- 172.21.15.151:0/1611481354 learned_addr learned my addr 172.21.15.151:0/1611481354 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:45:57.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:45:57.629+0000 7f928aa00700 1 --2- 172.21.15.151:0/1611481354 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f928c077510 0x7f928c19bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:45:57.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:45:57.629+0000 7f928be00700 1 --2- 172.21.15.151:0/1611481354 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f928c10b480 0x7f928c1a2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:45:57.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:45:57.629+0000 7f928aa00700 1 -- 172.21.15.151:0/1611481354 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f928c10b480 msgr2=0x7f928c1a2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:45:57.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:45:57.629+0000 7f928aa00700 1 --2- 172.21.15.151:0/1611481354 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f928c10b480 0x7f928c1a2f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:45:57.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:45:57.629+0000 7f928aa00700 1 -- 172.21.15.151:0/1611481354 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f928c075f40 msgr2=0x7f928c19b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:45:57.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:45:57.629+0000 7f928aa00700 1 --2- 172.21.15.151:0/1611481354 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f928c075f40 0x7f928c19b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:45:57.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:45:57.629+0000 7f928aa00700 1 -- 172.21.15.151:0/1611481354 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f928c1a35b0 con 0x7f928c077510 2024-05-21T21:45:57.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:45:57.629+0000 7f928b400700 1 --2- 172.21.15.151:0/1611481354 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f928c075f40 0x7f928c19b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:45:57.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:45:57.629+0000 7f928aa00700 1 --2- 172.21.15.151:0/1611481354 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f928c077510 0x7f928c19bee0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9278000c00 tx=0x7f9278005270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:45:57.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:45:57.630+0000 7f9283e00700 1 -- 172.21.15.151:0/1611481354 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9278025070 con 0x7f928c077510 2024-05-21T21:45:57.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:45:57.630+0000 7f9292400700 1 -- 172.21.15.151:0/1611481354 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f928c1a3840 con 0x7f928c077510 2024-05-21T21:45:57.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:45:57.630+0000 7f9283e00700 1 -- 172.21.15.151:0/1611481354 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9278005670 con 0x7f928c077510 2024-05-21T21:45:57.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:45:57.630+0000 7f9283e00700 1 -- 172.21.15.151:0/1611481354 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9278003ae0 con 0x7f928c077510 2024-05-21T21:45:57.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:45:57.630+0000 7f9292400700 1 -- 172.21.15.151:0/1611481354 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f928c1a3cf0 con 0x7f928c077510 2024-05-21T21:45:57.634 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:45:57.631+0000 7f9292400700 1 -- 172.21.15.151:0/1611481354 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f928c06a230 con 0x7f928c077510 2024-05-21T21:45:57.634 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:45:57.633+0000 7f9283e00700 1 -- 172.21.15.151:0/1611481354 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 15) ==== 149597+0+0 (secure 0 0 0) 0x7f9278028070 con 0x7f928c077510 2024-05-21T21:45:57.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:45:57.635+0000 7f9283e00700 1 --2- 172.21.15.151:0/1611481354 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f92700b27f0 0x7f92700b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:45:57.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:45:57.635+0000 7f9283e00700 1 -- 172.21.15.151:0/1611481354 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 3237+0+0 (secure 0 0 0) 0x7f92780e1510 con 0x7f928c077510 2024-05-21T21:45:57.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:45:57.635+0000 7f928b400700 1 --2- 172.21.15.151:0/1611481354 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f92700b27f0 0x7f92700b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:45:57.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:45:57.636+0000 7f928b400700 1 --2- 172.21.15.151:0/1611481354 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f92700b27f0 0x7f92700b4cb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f927c005950 tx=0x7f927c009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:45:57.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:45:57.639+0000 7f9283e00700 1 -- 172.21.15.151:0/1611481354 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9278013030 con 0x7f928c077510 2024-05-21T21:45:57.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:45:57.885+0000 7f9292400700 1 -- 172.21.15.151:0/1611481354 --> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f928c063980 con 0x7f92700b27f0 2024-05-21T21:45:57.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:57 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-21T21:45:57.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:57 smithi184 ceph-mon[36942]: osdmap e20: 3 total, 3 up, 3 in 2024-05-21T21:45:57.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:57 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:57.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:57 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:57.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:57 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:57.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:57 smithi184 ceph-mon[36942]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-21T21:45:57.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:57 smithi184 ceph-mon[36942]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-21T21:45:57.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:57 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-21T21:45:57.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:57 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T21:45:57.987 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:57 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T21:45:57.987 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:57 smithi184 ceph-mon[36942]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-21T21:45:57.987 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:57 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-21T21:45:57.987 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:57 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T21:45:57.987 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:57 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T21:45:57.987 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:57 smithi184 ceph-mon[36942]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-21T21:45:57.987 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:57 smithi184 ceph-mon[36942]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-21T21:45:57.987 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:57 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-21T21:45:57.987 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:57 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T21:45:57.987 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:57 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T21:45:57.987 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:57 smithi184 ceph-mon[36942]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-21T21:45:57.987 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:57 smithi184 ceph-mon[36942]: osdmap e21: 3 total, 3 up, 3 in 2024-05-21T21:45:58.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:45:58.010+0000 7f9283e00700 1 -- 172.21.15.151:0/1611481354 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f9278003ce0 con 0x7f928c077510 2024-05-21T21:45:58.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:57 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-21T21:45:58.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:57 smithi164 ceph-mon[36269]: osdmap e20: 3 total, 3 up, 3 in 2024-05-21T21:45:58.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:57 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:58.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:57 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:58.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:57 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:45:58.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:57 smithi164 ceph-mon[36269]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-21T21:45:58.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:57 smithi164 ceph-mon[36269]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-21T21:45:58.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:57 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-21T21:45:58.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:57 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T21:45:58.037 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:57 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T21:45:58.037 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:57 smithi164 ceph-mon[36269]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-21T21:45:58.037 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:57 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-21T21:45:58.037 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:57 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T21:45:58.037 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:57 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T21:45:58.037 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:57 smithi164 ceph-mon[36269]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-21T21:45:58.037 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:57 smithi164 ceph-mon[36269]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-21T21:45:58.037 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:57 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-21T21:45:58.037 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:57 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T21:45:58.037 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:57 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T21:45:58.037 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:57 smithi164 ceph-mon[36269]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-21T21:45:58.037 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:57 smithi164 ceph-mon[36269]: osdmap e21: 3 total, 3 up, 3 in 2024-05-21T21:45:58.952 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:58 smithi151 ceph-mon[31623]: pgmap v94: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-21T21:45:58.952 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:58 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T21:45:58.952 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:58 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T21:45:58.952 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:58 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:45:58.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:58 smithi151 ceph-mon[31623]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-21T21:45:58.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:58 smithi184 ceph-mon[36942]: pgmap v94: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-21T21:45:58.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:58 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T21:45:58.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:58 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T21:45:58.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:58 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:45:58.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:58 smithi184 ceph-mon[36942]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-21T21:45:59.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:58 smithi164 ceph-mon[36269]: pgmap v94: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-21T21:45:59.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:58 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T21:45:59.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:58 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T21:45:59.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:58 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:45:59.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:58 smithi164 ceph-mon[36269]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-21T21:45:59.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:45:59 smithi151 ceph-mon[31623]: from='client.24205 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:45:59.985 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:45:59 smithi184 ceph-mon[36942]: from='client.24205 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:46:00.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:45:59 smithi164 ceph-mon[36269]: from='client.24205 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:46:00.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:00 smithi151 ceph-mon[31623]: pgmap v96: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-21T21:46:00.985 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:00 smithi184 ceph-mon[36942]: pgmap v96: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-21T21:46:01.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:00 smithi164 ceph-mon[36269]: pgmap v96: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-21T21:46:02.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:02 smithi151 ceph-mon[31623]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-21T21:46:02.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:02 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/2627502023' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ab3df826-39e3-4635-a49c-2b5ffdbf12f3"}]: dispatch 2024-05-21T21:46:02.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:02 smithi151 ceph-mon[31623]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ab3df826-39e3-4635-a49c-2b5ffdbf12f3"}]: dispatch 2024-05-21T21:46:02.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:02 smithi151 ceph-mon[31623]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ab3df826-39e3-4635-a49c-2b5ffdbf12f3"}]': finished 2024-05-21T21:46:02.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:02 smithi151 ceph-mon[31623]: osdmap e22: 4 total, 3 up, 4 in 2024-05-21T21:46:02.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:02 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-21T21:46:02.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:02 smithi184 ceph-mon[36942]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-21T21:46:02.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:02 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/2627502023' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ab3df826-39e3-4635-a49c-2b5ffdbf12f3"}]: dispatch 2024-05-21T21:46:02.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:02 smithi184 ceph-mon[36942]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ab3df826-39e3-4635-a49c-2b5ffdbf12f3"}]: dispatch 2024-05-21T21:46:02.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:02 smithi184 ceph-mon[36942]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ab3df826-39e3-4635-a49c-2b5ffdbf12f3"}]': finished 2024-05-21T21:46:02.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:02 smithi184 ceph-mon[36942]: osdmap e22: 4 total, 3 up, 4 in 2024-05-21T21:46:02.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:02 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-21T21:46:03.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:02 smithi164 ceph-mon[36269]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-21T21:46:03.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:02 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/2627502023' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ab3df826-39e3-4635-a49c-2b5ffdbf12f3"}]: dispatch 2024-05-21T21:46:03.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:02 smithi164 ceph-mon[36269]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ab3df826-39e3-4635-a49c-2b5ffdbf12f3"}]: dispatch 2024-05-21T21:46:03.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:02 smithi164 ceph-mon[36269]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ab3df826-39e3-4635-a49c-2b5ffdbf12f3"}]': finished 2024-05-21T21:46:03.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:02 smithi164 ceph-mon[36269]: osdmap e22: 4 total, 3 up, 4 in 2024-05-21T21:46:03.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:02 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-21T21:46:03.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:03 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/2566808523' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T21:46:03.985 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:03 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/2566808523' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T21:46:04.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:03 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/2566808523' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T21:46:04.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:04 smithi151 ceph-mon[31623]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-21T21:46:04.985 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:04 smithi184 ceph-mon[36942]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-21T21:46:05.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:04 smithi164 ceph-mon[36269]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-21T21:46:06.892 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:06 smithi151 ceph-mon[31623]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-21T21:46:06.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:06 smithi184 ceph-mon[36942]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-21T21:46:07.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:06 smithi164 ceph-mon[36269]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-21T21:46:08.913 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:08 smithi151 ceph-mon[31623]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-21T21:46:08.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:08 smithi184 ceph-mon[36942]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-21T21:46:09.035 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:08 smithi164 ceph-mon[36269]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-21T21:46:10.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:10 smithi151 ceph-mon[31623]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-21T21:46:10.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:10 smithi184 ceph-mon[36942]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-21T21:46:11.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:10 smithi164 ceph-mon[36269]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-21T21:46:11.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:11 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-21T21:46:11.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:11 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:46:11.985 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:11 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-21T21:46:11.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:11 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:46:12.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:11 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-21T21:46:12.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:11 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:46:12.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:12 smithi151 ceph-mon[31623]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-21T21:46:12.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:12 smithi151 ceph-mon[31623]: Deploying daemon osd.3 on smithi151 2024-05-21T21:46:12.985 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:12 smithi184 ceph-mon[36942]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-21T21:46:12.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:12 smithi184 ceph-mon[36942]: Deploying daemon osd.3 on smithi151 2024-05-21T21:46:13.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:12 smithi164 ceph-mon[36269]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-21T21:46:13.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:12 smithi164 ceph-mon[36269]: Deploying daemon osd.3 on smithi151 2024-05-21T21:46:14.901 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:14 smithi151 ceph-mon[31623]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-21T21:46:14.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:14 smithi184 ceph-mon[36942]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-21T21:46:15.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:14 smithi164 ceph-mon[36269]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-21T21:46:16.909 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:16 smithi151 ceph-mon[31623]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-21T21:46:16.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:16 smithi184 ceph-mon[36942]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-21T21:46:17.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:16 smithi164 ceph-mon[36269]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-21T21:46:18.867 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:18 smithi151 ceph-mon[31623]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-21T21:46:18.985 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:18 smithi184 ceph-mon[36942]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-21T21:46:19.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:18 smithi164 ceph-mon[36269]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-21T21:46:20.527 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:20 smithi151 ceph-mon[31623]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-21T21:46:20.528 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:20 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:46:20.528 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:20 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:46:20.528 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:20 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:46:20.735 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:20 smithi184 ceph-mon[36942]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-21T21:46:20.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:20 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:46:20.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:20 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:46:20.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:20 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:46:20.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:20 smithi164 ceph-mon[36269]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-21T21:46:20.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:20 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:46:20.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:20 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:46:20.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:20 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:46:22.677 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:22 smithi151 ceph-mon[31623]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-21T21:46:22.735 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:22 smithi184 ceph-mon[36942]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-21T21:46:22.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:22 smithi164 ceph-mon[36269]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-21T21:46:24.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:24 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:46:24.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:24 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:46:24.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:24 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:46:24.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:24 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:46:24.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:24 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:46:24.121 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:24 smithi151 ceph-mon[31623]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-21T21:46:24.122 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:24 smithi151 ceph-mon[31623]: from='osd.3 [v2:172.21.15.151:6810/556594400,v1:172.21.15.151:6811/556594400]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-21T21:46:24.122 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:24 smithi151 ceph-mon[31623]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-21T21:46:24.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:46:24.226+0000 7f9283e00700 1 -- 172.21.15.151:0/1611481354 <== mgr.14152 v2:172.21.15.151:6800/2796622483 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f928c063980 con 0x7f92700b27f0 2024-05-21T21:46:24.229 INFO:teuthology.orchestra.run.smithi151.stdout:Created osd(s) 3 on host 'smithi151' 2024-05-21T21:46:24.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:46:24.230+0000 7f9292400700 1 -- 172.21.15.151:0/1611481354 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f92700b27f0 msgr2=0x7f92700b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:46:24.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:46:24.230+0000 7f9292400700 1 --2- 172.21.15.151:0/1611481354 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f92700b27f0 0x7f92700b4cb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f927c005950 tx=0x7f927c009310 comp rx=0 tx=0).stop 2024-05-21T21:46:24.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:46:24.230+0000 7f9292400700 1 -- 172.21.15.151:0/1611481354 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f928c077510 msgr2=0x7f928c19bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:46:24.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:46:24.230+0000 7f9292400700 1 --2- 172.21.15.151:0/1611481354 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f928c077510 0x7f928c19bee0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9278000c00 tx=0x7f9278005270 comp rx=0 tx=0).stop 2024-05-21T21:46:24.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:46:24.230+0000 7f9292400700 1 -- 172.21.15.151:0/1611481354 shutdown_connections 2024-05-21T21:46:24.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:46:24.230+0000 7f9292400700 1 --2- 172.21.15.151:0/1611481354 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f92700b27f0 0x7f92700b4cb0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:46:24.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:46:24.230+0000 7f9292400700 1 --2- 172.21.15.151:0/1611481354 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f928c10b480 0x7f928c1a2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:46:24.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:46:24.230+0000 7f9292400700 1 --2- 172.21.15.151:0/1611481354 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f928c075f40 0x7f928c19b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:46:24.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:46:24.230+0000 7f9292400700 1 --2- 172.21.15.151:0/1611481354 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f928c077510 0x7f928c19bee0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:46:24.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:46:24.230+0000 7f9292400700 1 -- 172.21.15.151:0/1611481354 >> 172.21.15.151:0/1611481354 conn(0x7f928c0feb40 msgr2=0x7f928c1073d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:46:24.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:46:24.231+0000 7f9292400700 1 -- 172.21.15.151:0/1611481354 shutdown_connections 2024-05-21T21:46:24.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:46:24.231+0000 7f9292400700 1 -- 172.21.15.151:0/1611481354 wait complete. 2024-05-21T21:46:24.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:24 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:46:24.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:24 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:46:24.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:24 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:46:24.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:24 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:46:24.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:24 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:46:24.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:24 smithi164 ceph-mon[36269]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-21T21:46:24.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:24 smithi164 ceph-mon[36269]: from='osd.3 [v2:172.21.15.151:6810/556594400,v1:172.21.15.151:6811/556594400]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-21T21:46:24.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:24 smithi164 ceph-mon[36269]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-21T21:46:24.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:24 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:46:24.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:24 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:46:24.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:24 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:46:24.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:24 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:46:24.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:24 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:46:24.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:24 smithi184 ceph-mon[36942]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-21T21:46:24.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:24 smithi184 ceph-mon[36942]: from='osd.3 [v2:172.21.15.151:6810/556594400,v1:172.21.15.151:6811/556594400]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-21T21:46:24.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:24 smithi184 ceph-mon[36942]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-21T21:46:24.908 DEBUG:teuthology.orchestra.run.smithi151:osd.3> sudo journalctl -f -n 0 -u ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@osd.3.service 2024-05-21T21:46:24.913 INFO:tasks.cephadm:Deploying osd.4 on smithi164 with /dev/vg_nvme/lv_3... 2024-05-21T21:46:24.913 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-21T21:46:25.164 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.b/config 2024-05-21T21:46:25.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:25 smithi151 ceph-mon[31623]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-21T21:46:25.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:25 smithi151 ceph-mon[31623]: osdmap e23: 4 total, 3 up, 4 in 2024-05-21T21:46:25.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:25 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-21T21:46:25.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:25 smithi151 ceph-mon[31623]: from='osd.3 [v2:172.21.15.151:6810/556594400,v1:172.21.15.151:6811/556594400]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-05-21T21:46:25.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:25 smithi151 ceph-mon[31623]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-05-21T21:46:25.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:25 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:46:25.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:25 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:46:25.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:25 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:46:25.285 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:25 smithi164 ceph-mon[36269]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-21T21:46:25.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:25 smithi164 ceph-mon[36269]: osdmap e23: 4 total, 3 up, 4 in 2024-05-21T21:46:25.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:25 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-21T21:46:25.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:25 smithi164 ceph-mon[36269]: from='osd.3 [v2:172.21.15.151:6810/556594400,v1:172.21.15.151:6811/556594400]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-05-21T21:46:25.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:25 smithi164 ceph-mon[36269]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-05-21T21:46:25.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:25 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:46:25.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:25 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:46:25.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:25 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:46:25.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:25 smithi184 ceph-mon[36942]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-21T21:46:25.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:25 smithi184 ceph-mon[36942]: osdmap e23: 4 total, 3 up, 4 in 2024-05-21T21:46:25.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:25 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-21T21:46:25.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:25 smithi184 ceph-mon[36942]: from='osd.3 [v2:172.21.15.151:6810/556594400,v1:172.21.15.151:6811/556594400]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-05-21T21:46:25.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:25 smithi184 ceph-mon[36942]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-05-21T21:46:25.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:25 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:46:25.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:25 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:46:25.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:25 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:46:26.205 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:26 smithi151 ceph-mon[31623]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-05-21T21:46:26.205 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:26 smithi151 ceph-mon[31623]: osdmap e24: 4 total, 3 up, 4 in 2024-05-21T21:46:26.205 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:26 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-21T21:46:26.205 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:26 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-21T21:46:26.205 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:26 smithi151 ceph-mon[31623]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-21T21:46:26.206 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:26 smithi151 ceph-mon[31623]: osdmap e25: 4 total, 3 up, 4 in 2024-05-21T21:46:26.206 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:26 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-21T21:46:26.206 INFO:journalctl@ceph.osd.3.smithi151.stdout:May 21 21:46:26 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-3[50544]: 2024-05-21T21:46:26.136+0000 7f1892400700 -1 osd.3 0 waiting for initial osdmap 2024-05-21T21:46:26.206 INFO:journalctl@ceph.osd.3.smithi151.stdout:May 21 21:46:26 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-3[50544]: 2024-05-21T21:46:26.171+0000 7f1888200700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-21T21:46:26.313 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:26 smithi164 ceph-mon[36269]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-05-21T21:46:26.313 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:26 smithi164 ceph-mon[36269]: osdmap e24: 4 total, 3 up, 4 in 2024-05-21T21:46:26.313 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:26 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-21T21:46:26.313 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:26 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-21T21:46:26.313 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:26 smithi164 ceph-mon[36269]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-21T21:46:26.313 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:26 smithi164 ceph-mon[36269]: osdmap e25: 4 total, 3 up, 4 in 2024-05-21T21:46:26.314 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:26 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-21T21:46:26.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:26 smithi184 ceph-mon[36942]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-05-21T21:46:26.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:26 smithi184 ceph-mon[36942]: osdmap e24: 4 total, 3 up, 4 in 2024-05-21T21:46:26.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:26 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-21T21:46:26.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:26 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-21T21:46:26.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:26 smithi184 ceph-mon[36942]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-21T21:46:26.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:26 smithi184 ceph-mon[36942]: osdmap e25: 4 total, 3 up, 4 in 2024-05-21T21:46:26.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:26 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-21T21:46:27.060 INFO:teuthology.orchestra.run.smithi164.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-21T21:46:27.060 INFO:teuthology.orchestra.run.smithi164.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-21T21:46:27.060 INFO:teuthology.orchestra.run.smithi164.stderr: stderr: 10+0 records in 2024-05-21T21:46:27.061 INFO:teuthology.orchestra.run.smithi164.stderr:10+0 records out 2024-05-21T21:46:27.061 INFO:teuthology.orchestra.run.smithi164.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.01189 s, 882 MB/s 2024-05-21T21:46:27.061 INFO:teuthology.orchestra.run.smithi164.stderr:--> Zapping successful for: 2024-05-21T21:46:27.360 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:27 smithi164 ceph-mon[36269]: purged_snaps scrub starts 2024-05-21T21:46:27.361 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:27 smithi164 ceph-mon[36269]: purged_snaps scrub ok 2024-05-21T21:46:27.361 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:27 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-21T21:46:27.361 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:27 smithi164 ceph-mon[36269]: from='osd.3 ' entity='osd.3' 2024-05-21T21:46:27.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:27 smithi151 ceph-mon[31623]: purged_snaps scrub starts 2024-05-21T21:46:27.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:27 smithi151 ceph-mon[31623]: purged_snaps scrub ok 2024-05-21T21:46:27.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:27 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-21T21:46:27.454 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:27 smithi151 ceph-mon[31623]: from='osd.3 ' entity='osd.3' 2024-05-21T21:46:27.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:27 smithi184 ceph-mon[36942]: purged_snaps scrub starts 2024-05-21T21:46:27.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:27 smithi184 ceph-mon[36942]: purged_snaps scrub ok 2024-05-21T21:46:27.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:27 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-21T21:46:27.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:27 smithi184 ceph-mon[36942]: from='osd.3 ' entity='osd.3' 2024-05-21T21:46:27.650 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi164:vg_nvme/lv_3 2024-05-21T21:46:27.893 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.b/config 2024-05-21T21:46:28.176 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:28 smithi151 ceph-mon[31623]: osd.3 [v2:172.21.15.151:6810/556594400,v1:172.21.15.151:6811/556594400] boot 2024-05-21T21:46:28.176 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:28 smithi151 ceph-mon[31623]: osdmap e26: 4 total, 4 up, 4 in 2024-05-21T21:46:28.176 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:28 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-21T21:46:28.177 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:28 smithi151 ceph-mon[31623]: pgmap v115: 1 pgs: 1 peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-05-21T21:46:28.177 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:28 smithi151 ceph-mon[31623]: osdmap e27: 4 total, 4 up, 4 in 2024-05-21T21:46:28.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:28 smithi164 ceph-mon[36269]: osd.3 [v2:172.21.15.151:6810/556594400,v1:172.21.15.151:6811/556594400] boot 2024-05-21T21:46:28.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:28 smithi164 ceph-mon[36269]: osdmap e26: 4 total, 4 up, 4 in 2024-05-21T21:46:28.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:28 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-21T21:46:28.287 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:28 smithi164 ceph-mon[36269]: pgmap v115: 1 pgs: 1 peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-05-21T21:46:28.287 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:28 smithi164 ceph-mon[36269]: osdmap e27: 4 total, 4 up, 4 in 2024-05-21T21:46:28.485 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:28 smithi184 ceph-mon[36942]: osd.3 [v2:172.21.15.151:6810/556594400,v1:172.21.15.151:6811/556594400] boot 2024-05-21T21:46:28.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:28 smithi184 ceph-mon[36942]: osdmap e26: 4 total, 4 up, 4 in 2024-05-21T21:46:28.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:28 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-21T21:46:28.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:28 smithi184 ceph-mon[36942]: pgmap v115: 1 pgs: 1 peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-05-21T21:46:28.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:28 smithi184 ceph-mon[36942]: osdmap e27: 4 total, 4 up, 4 in 2024-05-21T21:46:29.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:46:29.405+0000 7fdec5162700 1 -- 172.21.15.164:0/2990114187 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdec01024a0 msgr2=0x7fdec01028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:46:29.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:46:29.405+0000 7fdec5162700 1 --2- 172.21.15.164:0/2990114187 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdec01024a0 0x7fdec01028c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fdeb00099c0 tx=0x7fdeb0009cd0 comp rx=0 tx=0).stop 2024-05-21T21:46:29.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:46:29.405+0000 7fdec5162700 1 -- 172.21.15.164:0/2990114187 shutdown_connections 2024-05-21T21:46:29.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:46:29.405+0000 7fdec5162700 1 --2- 172.21.15.164:0/2990114187 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdec01024a0 0x7fdec01028c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:46:29.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:46:29.405+0000 7fdec5162700 1 --2- 172.21.15.164:0/2990114187 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdec0104040 0x7fdec010a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:46:29.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:46:29.405+0000 7fdec5162700 1 --2- 172.21.15.164:0/2990114187 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdec01036a0 0x7fdec0103b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:46:29.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:46:29.405+0000 7fdec5162700 1 -- 172.21.15.164:0/2990114187 >> 172.21.15.164:0/2990114187 conn(0x7fdec00fda40 msgr2=0x7fdec00ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:46:29.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:46:29.405+0000 7fdec5162700 1 -- 172.21.15.164:0/2990114187 shutdown_connections 2024-05-21T21:46:29.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:46:29.405+0000 7fdec5162700 1 -- 172.21.15.164:0/2990114187 wait complete. 2024-05-21T21:46:29.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:46:29.406+0000 7fdec5162700 1 Processor -- start 2024-05-21T21:46:29.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:46:29.406+0000 7fdec5162700 1 -- start start 2024-05-21T21:46:29.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:46:29.406+0000 7fdec5162700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdec01024a0 0x7fdec006bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:46:29.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:46:29.406+0000 7fdec5162700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdec01036a0 0x7fdec019e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:46:29.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:46:29.406+0000 7fdec5162700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdec0104040 0x7fdec01a0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:46:29.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:46:29.406+0000 7fdec5162700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdec010df50 con 0x7fdec0104040 2024-05-21T21:46:29.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:46:29.406+0000 7fdec5162700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdec010ddd0 con 0x7fdec01036a0 2024-05-21T21:46:29.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:46:29.406+0000 7fdec5162700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fdec010e0d0 con 0x7fdec01024a0 2024-05-21T21:46:29.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:46:29.406+0000 7fdebe59c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdec01036a0 0x7fdec019e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:46:29.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:46:29.406+0000 7fdebe59c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdec01036a0 0x7fdec019e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:58126/0 (socket says 172.21.15.164:58126) 2024-05-21T21:46:29.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:46:29.406+0000 7fdebe59c700 1 -- 172.21.15.164:0/1386530440 learned_addr learned my addr 172.21.15.164:0/1386530440 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-21T21:46:29.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:46:29.406+0000 7fdebed9d700 1 --2- 172.21.15.164:0/1386530440 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdec01024a0 0x7fdec006bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:46:29.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:46:29.406+0000 7fdebe59c700 1 -- 172.21.15.164:0/1386530440 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdec01024a0 msgr2=0x7fdec006bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:46:29.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:46:29.406+0000 7fdebf59e700 1 --2- 172.21.15.164:0/1386530440 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdec0104040 0x7fdec01a0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:46:29.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:46:29.406+0000 7fdebe59c700 1 --2- 172.21.15.164:0/1386530440 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdec01024a0 0x7fdec006bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:46:29.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:46:29.406+0000 7fdebe59c700 1 -- 172.21.15.164:0/1386530440 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdec0104040 msgr2=0x7fdec01a0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:46:29.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:46:29.406+0000 7fdebe59c700 1 --2- 172.21.15.164:0/1386530440 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdec0104040 0x7fdec01a0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:46:29.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:46:29.407+0000 7fdebe59c700 1 -- 172.21.15.164:0/1386530440 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdec01a13d0 con 0x7fdec01036a0 2024-05-21T21:46:29.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:46:29.407+0000 7fdebed9d700 1 --2- 172.21.15.164:0/1386530440 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdec01024a0 0x7fdec006bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T21:46:29.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:46:29.407+0000 7fdebe59c700 1 --2- 172.21.15.164:0/1386530440 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdec01036a0 0x7fdec019e4c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fdeb400d980 tx=0x7fdeb400dd40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:46:29.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:46:29.407+0000 7fdea7fff700 1 -- 172.21.15.164:0/1386530440 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb4017070 con 0x7fdec01036a0 2024-05-21T21:46:29.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:46:29.407+0000 7fdec5162700 1 -- 172.21.15.164:0/1386530440 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdec01a16c0 con 0x7fdec01036a0 2024-05-21T21:46:29.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:46:29.407+0000 7fdea7fff700 1 -- 172.21.15.164:0/1386530440 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdeb4004830 con 0x7fdec01036a0 2024-05-21T21:46:29.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:46:29.407+0000 7fdea7fff700 1 -- 172.21.15.164:0/1386530440 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb4005440 con 0x7fdec01036a0 2024-05-21T21:46:29.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:46:29.407+0000 7fdec5162700 1 -- 172.21.15.164:0/1386530440 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdec01a1bb0 con 0x7fdec01036a0 2024-05-21T21:46:29.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:46:29.408+0000 7fdec5162700 1 -- 172.21.15.164:0/1386530440 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdec006a230 con 0x7fdec01036a0 2024-05-21T21:46:29.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:46:29.410+0000 7fdea7fff700 1 -- 172.21.15.164:0/1386530440 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7fdeb40055e0 con 0x7fdec01036a0 2024-05-21T21:46:29.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:46:29.410+0000 7fdea7fff700 1 --2- 172.21.15.164:0/1386530440 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fdea80b2880 0x7fdea80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:46:29.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:46:29.410+0000 7fdea7fff700 1 -- 172.21.15.164:0/1386530440 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 3669+0+0 (secure 0 0 0) 0x7fdeb40e1370 con 0x7fdec01036a0 2024-05-21T21:46:29.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:46:29.411+0000 7fdebed9d700 1 --2- 172.21.15.164:0/1386530440 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fdea80b2880 0x7fdea80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:46:29.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:46:29.411+0000 7fdebed9d700 1 --2- 172.21.15.164:0/1386530440 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fdea80b2880 0x7fdea80b4d40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdeb0006010 tx=0x7fdeb0005c40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:46:29.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:46:29.414+0000 7fdea7fff700 1 -- 172.21.15.164:0/1386530440 <== mon.2 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdeb4020020 con 0x7fdec01036a0 2024-05-21T21:46:29.625 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:46:29.625+0000 7fdec5162700 1 -- 172.21.15.164:0/1386530440 --> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fdec0063980 con 0x7fdea80b2880 2024-05-21T21:46:30.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:30 smithi151 ceph-mon[31623]: osdmap e28: 4 total, 4 up, 4 in 2024-05-21T21:46:30.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:30 smithi151 ceph-mon[31623]: pgmap v118: 1 pgs: 1 peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-05-21T21:46:30.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:30 smithi151 ceph-mon[31623]: from='client.24230 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:46:30.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:30 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T21:46:30.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:30 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T21:46:30.204 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:30 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:46:30.350 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:30 smithi164 ceph-mon[36269]: osdmap e28: 4 total, 4 up, 4 in 2024-05-21T21:46:30.350 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:30 smithi164 ceph-mon[36269]: pgmap v118: 1 pgs: 1 peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-05-21T21:46:30.350 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:30 smithi164 ceph-mon[36269]: from='client.24230 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:46:30.350 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:30 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T21:46:30.350 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:30 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T21:46:30.350 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:30 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:46:30.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:30 smithi184 ceph-mon[36942]: osdmap e28: 4 total, 4 up, 4 in 2024-05-21T21:46:30.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:30 smithi184 ceph-mon[36942]: pgmap v118: 1 pgs: 1 peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-05-21T21:46:30.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:30 smithi184 ceph-mon[36942]: from='client.24230 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:46:30.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:30 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T21:46:30.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:30 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T21:46:30.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:30 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:46:31.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:31 smithi151 ceph-mon[31623]: Detected new or changed devices on smithi151 2024-05-21T21:46:31.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:31 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:46:31.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:31 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:46:31.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:31 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:46:31.954 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:31 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:46:31.954 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:31 smithi151 ceph-mon[31623]: Adjusting osd_memory_target on smithi151 to 8544M 2024-05-21T21:46:31.954 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:31 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:46:31.954 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:31 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:46:31.954 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:31 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:46:31.954 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:31 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:46:31.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:31 smithi184 ceph-mon[36942]: Detected new or changed devices on smithi151 2024-05-21T21:46:31.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:31 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:46:31.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:31 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:46:31.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:31 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:46:31.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:31 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:46:31.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:31 smithi184 ceph-mon[36942]: Adjusting osd_memory_target on smithi151 to 8544M 2024-05-21T21:46:31.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:31 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:46:31.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:31 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:46:31.987 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:31 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:46:31.987 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:31 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:46:32.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:31 smithi164 ceph-mon[36269]: Detected new or changed devices on smithi151 2024-05-21T21:46:32.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:31 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:46:32.037 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:31 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:46:32.037 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:31 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:46:32.037 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:31 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:46:32.037 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:31 smithi164 ceph-mon[36269]: Adjusting osd_memory_target on smithi151 to 8544M 2024-05-21T21:46:32.037 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:31 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:46:32.037 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:31 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:46:32.037 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:31 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:46:32.037 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:31 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:46:32.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:32 smithi151 ceph-mon[31623]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-05-21T21:46:32.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:32 smithi151 ceph-mon[31623]: from='client.? 172.21.15.164:0/3246948933' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "00cbf6ae-0f42-4685-a77d-d803c6467aa0"}]: dispatch 2024-05-21T21:46:32.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:32 smithi151 ceph-mon[31623]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "00cbf6ae-0f42-4685-a77d-d803c6467aa0"}]: dispatch 2024-05-21T21:46:32.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:32 smithi151 ceph-mon[31623]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "00cbf6ae-0f42-4685-a77d-d803c6467aa0"}]': finished 2024-05-21T21:46:32.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:32 smithi151 ceph-mon[31623]: osdmap e29: 5 total, 4 up, 5 in 2024-05-21T21:46:32.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:32 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-21T21:46:32.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:32 smithi184 ceph-mon[36942]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-05-21T21:46:32.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:32 smithi184 ceph-mon[36942]: from='client.? 172.21.15.164:0/3246948933' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "00cbf6ae-0f42-4685-a77d-d803c6467aa0"}]: dispatch 2024-05-21T21:46:32.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:32 smithi184 ceph-mon[36942]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "00cbf6ae-0f42-4685-a77d-d803c6467aa0"}]: dispatch 2024-05-21T21:46:32.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:32 smithi184 ceph-mon[36942]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "00cbf6ae-0f42-4685-a77d-d803c6467aa0"}]': finished 2024-05-21T21:46:32.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:32 smithi184 ceph-mon[36942]: osdmap e29: 5 total, 4 up, 5 in 2024-05-21T21:46:32.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:32 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-21T21:46:33.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:32 smithi164 ceph-mon[36269]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-05-21T21:46:33.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:32 smithi164 ceph-mon[36269]: from='client.? 172.21.15.164:0/3246948933' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "00cbf6ae-0f42-4685-a77d-d803c6467aa0"}]: dispatch 2024-05-21T21:46:33.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:32 smithi164 ceph-mon[36269]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "00cbf6ae-0f42-4685-a77d-d803c6467aa0"}]: dispatch 2024-05-21T21:46:33.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:32 smithi164 ceph-mon[36269]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "00cbf6ae-0f42-4685-a77d-d803c6467aa0"}]': finished 2024-05-21T21:46:33.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:32 smithi164 ceph-mon[36269]: osdmap e29: 5 total, 4 up, 5 in 2024-05-21T21:46:33.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:32 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-21T21:46:33.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:33 smithi151 ceph-mon[31623]: from='client.? 172.21.15.164:0/2794473436' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T21:46:33.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:33 smithi184 ceph-mon[36942]: from='client.? 172.21.15.164:0/2794473436' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T21:46:34.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:33 smithi164 ceph-mon[36269]: from='client.? 172.21.15.164:0/2794473436' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T21:46:34.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:34 smithi151 ceph-mon[31623]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-21T21:46:34.985 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:34 smithi184 ceph-mon[36942]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-21T21:46:35.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:34 smithi164 ceph-mon[36269]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-21T21:46:36.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:36 smithi164 ceph-mon[36269]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-05-21T21:46:36.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:36 smithi151 ceph-mon[31623]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-05-21T21:46:36.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:36 smithi184 ceph-mon[36942]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-05-21T21:46:38.734 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:38 smithi164 ceph-mon[36269]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-21T21:46:38.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:38 smithi151 ceph-mon[31623]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-21T21:46:38.985 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:38 smithi184 ceph-mon[36942]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-21T21:46:40.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:40 smithi151 ceph-mon[31623]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-21T21:46:40.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:40 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-21T21:46:40.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:40 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:46:40.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:40 smithi151 ceph-mon[31623]: Deploying daemon osd.4 on smithi164 2024-05-21T21:46:40.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:40 smithi184 ceph-mon[36942]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-21T21:46:40.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:40 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-21T21:46:40.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:40 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:46:40.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:40 smithi184 ceph-mon[36942]: Deploying daemon osd.4 on smithi164 2024-05-21T21:46:41.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:40 smithi164 ceph-mon[36269]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-21T21:46:41.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:40 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-21T21:46:41.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:40 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:46:41.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:40 smithi164 ceph-mon[36269]: Deploying daemon osd.4 on smithi164 2024-05-21T21:46:42.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:42 smithi151 ceph-mon[31623]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-05-21T21:46:42.985 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:42 smithi184 ceph-mon[36942]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-05-21T21:46:43.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:42 smithi164 ceph-mon[36269]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-05-21T21:46:44.974 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:44 smithi164 ceph-mon[36269]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-05-21T21:46:44.985 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:44 smithi184 ceph-mon[36942]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-05-21T21:46:45.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:44 smithi151 ceph-mon[31623]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-05-21T21:46:46.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:46 smithi151 ceph-mon[31623]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-05-21T21:46:46.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:46 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:46:46.454 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:46 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:46:46.454 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:46 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:46:46.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:46 smithi184 ceph-mon[36942]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-05-21T21:46:46.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:46 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:46:46.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:46 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:46:46.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:46 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:46:46.504 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:46 smithi164 ceph-mon[36269]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-05-21T21:46:46.505 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:46 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:46:46.505 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:46 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:46:46.505 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:46 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:46:48.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:48 smithi151 ceph-mon[31623]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-05-21T21:46:48.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:48 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:46:48.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:48 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:46:48.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:48 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:46:48.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:48 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:46:48.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:48 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:46:48.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:48 smithi184 ceph-mon[36942]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-05-21T21:46:48.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:48 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:46:48.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:48 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:46:48.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:48 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:46:48.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:48 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:46:48.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:48 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:46:48.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:48 smithi164 ceph-mon[36269]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-05-21T21:46:48.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:48 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:46:48.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:48 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:46:48.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:48 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:46:48.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:48 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:46:48.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:48 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:46:48.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:46:48.810+0000 7fdea7fff700 1 -- 172.21.15.164:0/1386530440 <== mgr.14152 v2:172.21.15.151:6800/2796622483 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fdec0063980 con 0x7fdea80b2880 2024-05-21T21:46:48.810 INFO:teuthology.orchestra.run.smithi164.stdout:Created osd(s) 4 on host 'smithi164' 2024-05-21T21:46:48.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:46:48.812+0000 7fdec5162700 1 -- 172.21.15.164:0/1386530440 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fdea80b2880 msgr2=0x7fdea80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:46:48.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:46:48.812+0000 7fdec5162700 1 --2- 172.21.15.164:0/1386530440 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fdea80b2880 0x7fdea80b4d40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdeb0006010 tx=0x7fdeb0005c40 comp rx=0 tx=0).stop 2024-05-21T21:46:48.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:46:48.812+0000 7fdec5162700 1 -- 172.21.15.164:0/1386530440 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdec01036a0 msgr2=0x7fdec019e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:46:48.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:46:48.812+0000 7fdec5162700 1 --2- 172.21.15.164:0/1386530440 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdec01036a0 0x7fdec019e4c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fdeb400d980 tx=0x7fdeb400dd40 comp rx=0 tx=0).stop 2024-05-21T21:46:48.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:46:48.812+0000 7fdec5162700 1 -- 172.21.15.164:0/1386530440 shutdown_connections 2024-05-21T21:46:48.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:46:48.812+0000 7fdec5162700 1 --2- 172.21.15.164:0/1386530440 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fdea80b2880 0x7fdea80b4d40 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:46:48.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:46:48.812+0000 7fdec5162700 1 --2- 172.21.15.164:0/1386530440 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdec0104040 0x7fdec01a0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:46:48.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:46:48.812+0000 7fdec5162700 1 --2- 172.21.15.164:0/1386530440 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdec01024a0 0x7fdec006bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:46:48.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:46:48.812+0000 7fdec5162700 1 --2- 172.21.15.164:0/1386530440 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdec01036a0 0x7fdec019e4c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:46:48.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:46:48.812+0000 7fdec5162700 1 -- 172.21.15.164:0/1386530440 >> 172.21.15.164:0/1386530440 conn(0x7fdec00fda40 msgr2=0x7fdec0106ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:46:48.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:46:48.812+0000 7fdec5162700 1 -- 172.21.15.164:0/1386530440 shutdown_connections 2024-05-21T21:46:48.814 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:46:48.812+0000 7fdec5162700 1 -- 172.21.15.164:0/1386530440 wait complete. 2024-05-21T21:46:49.436 DEBUG:teuthology.orchestra.run.smithi164:osd.4> sudo journalctl -f -n 0 -u ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@osd.4.service 2024-05-21T21:46:49.438 INFO:tasks.cephadm:Deploying osd.5 on smithi184 with /dev/vg_nvme/lv_3... 2024-05-21T21:46:49.438 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-21T21:46:49.683 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.c/config 2024-05-21T21:46:50.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:49 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:46:50.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:49 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:46:50.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:49 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:46:50.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:49 smithi164 ceph-mon[36269]: from='osd.4 [v2:172.21.15.164:6808/988881973,v1:172.21.15.164:6809/988881973]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-21T21:46:50.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:49 smithi164 ceph-mon[36269]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-21T21:46:50.122 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:49 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:46:50.122 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:49 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:46:50.122 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:49 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:46:50.122 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:49 smithi184 ceph-mon[36942]: from='osd.4 [v2:172.21.15.164:6808/988881973,v1:172.21.15.164:6809/988881973]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-21T21:46:50.122 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:49 smithi184 ceph-mon[36942]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-21T21:46:50.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:49 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:46:50.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:49 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:46:50.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:49 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:46:50.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:49 smithi151 ceph-mon[31623]: from='osd.4 [v2:172.21.15.164:6808/988881973,v1:172.21.15.164:6809/988881973]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-21T21:46:50.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:49 smithi151 ceph-mon[31623]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-21T21:46:50.919 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:50 smithi164 ceph-mon[36269]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-05-21T21:46:50.919 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:50 smithi164 ceph-mon[36269]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-21T21:46:50.919 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:50 smithi164 ceph-mon[36269]: from='osd.4 [v2:172.21.15.164:6808/988881973,v1:172.21.15.164:6809/988881973]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-05-21T21:46:50.920 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:50 smithi164 ceph-mon[36269]: osdmap e30: 5 total, 4 up, 5 in 2024-05-21T21:46:50.920 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:50 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-21T21:46:50.920 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:50 smithi164 ceph-mon[36269]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-05-21T21:46:50.920 INFO:journalctl@ceph.osd.4.smithi164.stdout:May 21 21:46:50 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-4[47510]: 2024-05-21T21:46:50.822+0000 7f4b9be9f700 -1 osd.4 0 waiting for initial osdmap 2024-05-21T21:46:50.920 INFO:journalctl@ceph.osd.4.smithi164.stdout:May 21 21:46:50 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-4[47510]: 2024-05-21T21:46:50.829+0000 7f4b97c92700 -1 osd.4 31 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-21T21:46:51.103 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:50 smithi184 ceph-mon[36942]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-05-21T21:46:51.103 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:50 smithi184 ceph-mon[36942]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-21T21:46:51.103 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:50 smithi184 ceph-mon[36942]: from='osd.4 [v2:172.21.15.164:6808/988881973,v1:172.21.15.164:6809/988881973]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-05-21T21:46:51.104 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:50 smithi184 ceph-mon[36942]: osdmap e30: 5 total, 4 up, 5 in 2024-05-21T21:46:51.104 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:50 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-21T21:46:51.104 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:50 smithi184 ceph-mon[36942]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-05-21T21:46:51.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:50 smithi151 ceph-mon[31623]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-05-21T21:46:51.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:50 smithi151 ceph-mon[31623]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-21T21:46:51.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:50 smithi151 ceph-mon[31623]: from='osd.4 [v2:172.21.15.164:6808/988881973,v1:172.21.15.164:6809/988881973]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-05-21T21:46:51.204 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:50 smithi151 ceph-mon[31623]: osdmap e30: 5 total, 4 up, 5 in 2024-05-21T21:46:51.204 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:50 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-21T21:46:51.204 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:50 smithi151 ceph-mon[31623]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-05-21T21:46:51.677 INFO:teuthology.orchestra.run.smithi184.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-21T21:46:51.677 INFO:teuthology.orchestra.run.smithi184.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-21T21:46:51.677 INFO:teuthology.orchestra.run.smithi184.stderr: stderr: 10+0 records in 2024-05-21T21:46:51.677 INFO:teuthology.orchestra.run.smithi184.stderr:10+0 records out 2024-05-21T21:46:51.677 INFO:teuthology.orchestra.run.smithi184.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0113714 s, 922 MB/s 2024-05-21T21:46:51.678 INFO:teuthology.orchestra.run.smithi184.stderr:--> Zapping successful for: 2024-05-21T21:46:51.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:51 smithi151 ceph-mon[31623]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-05-21T21:46:51.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:51 smithi151 ceph-mon[31623]: osdmap e31: 5 total, 4 up, 5 in 2024-05-21T21:46:51.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:51 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-21T21:46:52.124 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:51 smithi164 ceph-mon[36269]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-05-21T21:46:52.124 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:51 smithi164 ceph-mon[36269]: osdmap e31: 5 total, 4 up, 5 in 2024-05-21T21:46:52.124 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:51 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-21T21:46:52.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:51 smithi184 ceph-mon[36942]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-05-21T21:46:52.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:51 smithi184 ceph-mon[36942]: osdmap e31: 5 total, 4 up, 5 in 2024-05-21T21:46:52.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:51 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-21T21:46:52.345 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi184:vg_nvme/lv_3 2024-05-21T21:46:52.590 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.c/config 2024-05-21T21:46:53.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:52 smithi151 ceph-mon[31623]: purged_snaps scrub starts 2024-05-21T21:46:53.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:52 smithi151 ceph-mon[31623]: purged_snaps scrub ok 2024-05-21T21:46:53.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:52 smithi151 ceph-mon[31623]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-05-21T21:46:53.204 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:52 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-21T21:46:53.204 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:52 smithi151 ceph-mon[31623]: osd.4 [v2:172.21.15.164:6808/988881973,v1:172.21.15.164:6809/988881973] boot 2024-05-21T21:46:53.204 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:52 smithi151 ceph-mon[31623]: osdmap e32: 5 total, 5 up, 5 in 2024-05-21T21:46:53.204 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:52 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-21T21:46:53.204 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:52 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:46:53.204 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:52 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:46:53.204 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:52 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:46:53.204 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:52 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:46:53.204 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:52 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:46:53.204 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:52 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:46:53.204 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:52 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:46:53.204 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:52 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:46:53.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:52 smithi184 ceph-mon[36942]: purged_snaps scrub starts 2024-05-21T21:46:53.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:52 smithi184 ceph-mon[36942]: purged_snaps scrub ok 2024-05-21T21:46:53.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:52 smithi184 ceph-mon[36942]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-05-21T21:46:53.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:52 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-21T21:46:53.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:52 smithi184 ceph-mon[36942]: osd.4 [v2:172.21.15.164:6808/988881973,v1:172.21.15.164:6809/988881973] boot 2024-05-21T21:46:53.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:52 smithi184 ceph-mon[36942]: osdmap e32: 5 total, 5 up, 5 in 2024-05-21T21:46:53.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:52 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-21T21:46:53.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:52 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:46:53.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:52 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:46:53.237 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:52 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:46:53.237 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:52 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:46:53.237 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:52 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:46:53.237 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:52 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:46:53.237 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:52 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:46:53.237 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:52 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:46:53.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:52 smithi164 ceph-mon[36269]: purged_snaps scrub starts 2024-05-21T21:46:53.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:52 smithi164 ceph-mon[36269]: purged_snaps scrub ok 2024-05-21T21:46:53.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:52 smithi164 ceph-mon[36269]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-05-21T21:46:53.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:52 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-21T21:46:53.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:52 smithi164 ceph-mon[36269]: osd.4 [v2:172.21.15.164:6808/988881973,v1:172.21.15.164:6809/988881973] boot 2024-05-21T21:46:53.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:52 smithi164 ceph-mon[36269]: osdmap e32: 5 total, 5 up, 5 in 2024-05-21T21:46:53.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:52 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-21T21:46:53.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:52 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:46:53.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:52 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:46:53.287 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:52 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:46:53.287 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:52 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:46:53.287 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:52 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:46:53.287 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:52 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:46:53.287 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:52 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:46:53.287 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:52 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:46:53.953 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:53 smithi184 ceph-mon[36942]: Detected new or changed devices on smithi164 2024-05-21T21:46:53.954 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:53 smithi184 ceph-mon[36942]: Adjusting osd_memory_target on smithi164 to 8544M 2024-05-21T21:46:53.954 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:53 smithi184 ceph-mon[36942]: osdmap e33: 5 total, 5 up, 5 in 2024-05-21T21:46:54.191 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:46:54.189+0000 7fe5a191f700 1 -- 172.21.15.184:0/957223974 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe59c105140 msgr2=0x7fe59c10b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:46:54.191 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:46:54.189+0000 7fe5a191f700 1 --2- 172.21.15.184:0/957223974 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe59c105140 0x7fe59c10b9f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe59400b0a0 tx=0x7fe59400b3b0 comp rx=0 tx=0).stop 2024-05-21T21:46:54.191 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:46:54.189+0000 7fe5a191f700 1 -- 172.21.15.184:0/957223974 shutdown_connections 2024-05-21T21:46:54.191 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:46:54.189+0000 7fe5a191f700 1 --2- 172.21.15.184:0/957223974 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe59c105140 0x7fe59c10b9f0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:46:54.191 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:46:54.189+0000 7fe5a191f700 1 --2- 172.21.15.184:0/957223974 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe59c1035a0 0x7fe59c1039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:46:54.191 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:46:54.189+0000 7fe5a191f700 1 --2- 172.21.15.184:0/957223974 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe59c1047a0 0x7fe59c104c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:46:54.192 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:46:54.189+0000 7fe5a191f700 1 -- 172.21.15.184:0/957223974 >> 172.21.15.184:0/957223974 conn(0x7fe59c0feb40 msgr2=0x7fe59c100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:46:54.192 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:46:54.189+0000 7fe5a191f700 1 -- 172.21.15.184:0/957223974 shutdown_connections 2024-05-21T21:46:54.192 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:46:54.189+0000 7fe5a191f700 1 -- 172.21.15.184:0/957223974 wait complete. 2024-05-21T21:46:54.192 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:46:54.189+0000 7fe5a191f700 1 Processor -- start 2024-05-21T21:46:54.192 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:46:54.190+0000 7fe5a191f700 1 -- start start 2024-05-21T21:46:54.192 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:46:54.190+0000 7fe5a191f700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe59c1035a0 0x7fe59c199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:46:54.192 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:46:54.190+0000 7fe5a191f700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe59c1047a0 0x7fe59c199c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:46:54.192 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:46:54.190+0000 7fe5a191f700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe59c105140 0x7fe59c1a0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:46:54.192 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:46:54.190+0000 7fe5a191f700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe59c10f100 con 0x7fe59c1035a0 2024-05-21T21:46:54.192 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:46:54.190+0000 7fe5a191f700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe59c10ef80 con 0x7fe59c105140 2024-05-21T21:46:54.192 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:46:54.190+0000 7fe5a191f700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe59c10f280 con 0x7fe59c1047a0 2024-05-21T21:46:54.192 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:46:54.190+0000 7fe5927fc700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe59c1047a0 0x7fe59c199c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:46:54.192 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:46:54.190+0000 7fe5927fc700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe59c1047a0 0x7fe59c199c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:32770/0 (socket says 172.21.15.184:32770) 2024-05-21T21:46:54.193 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:46:54.190+0000 7fe5927fc700 1 -- 172.21.15.184:0/1045034713 learned_addr learned my addr 172.21.15.184:0/1045034713 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-05-21T21:46:54.193 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:46:54.190+0000 7fe59b7fe700 1 --2- 172.21.15.184:0/1045034713 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe59c105140 0x7fe59c1a0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:46:54.193 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:46:54.190+0000 7fe5927fc700 1 -- 172.21.15.184:0/1045034713 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe59c105140 msgr2=0x7fe59c1a0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:46:54.193 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:46:54.190+0000 7fe5927fc700 1 --2- 172.21.15.184:0/1045034713 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe59c105140 0x7fe59c1a0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:46:54.193 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:46:54.190+0000 7fe5927fc700 1 -- 172.21.15.184:0/1045034713 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe59c1035a0 msgr2=0x7fe59c199750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:46:54.193 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:46:54.190+0000 7fe5927fc700 1 --2- 172.21.15.184:0/1045034713 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe59c1035a0 0x7fe59c199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:46:54.193 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:46:54.190+0000 7fe5927fc700 1 -- 172.21.15.184:0/1045034713 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe59c1a13b0 con 0x7fe59c1047a0 2024-05-21T21:46:54.193 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:46:54.190+0000 7fe5927fc700 1 --2- 172.21.15.184:0/1045034713 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe59c1047a0 0x7fe59c199c90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe58400d980 tx=0x7fe58400dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:46:54.193 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:46:54.190+0000 7fe5a091d700 1 -- 172.21.15.184:0/1045034713 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe584016070 con 0x7fe59c1047a0 2024-05-21T21:46:54.193 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:46:54.191+0000 7fe5a091d700 1 -- 172.21.15.184:0/1045034713 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe58400de70 con 0x7fe59c1047a0 2024-05-21T21:46:54.193 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:46:54.191+0000 7fe5a191f700 1 -- 172.21.15.184:0/1045034713 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe59c1a16a0 con 0x7fe59c1047a0 2024-05-21T21:46:54.193 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:46:54.191+0000 7fe5a091d700 1 -- 172.21.15.184:0/1045034713 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe58400f3c0 con 0x7fe59c1047a0 2024-05-21T21:46:54.193 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:46:54.191+0000 7fe5a191f700 1 -- 172.21.15.184:0/1045034713 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe59c1a1c60 con 0x7fe59c1047a0 2024-05-21T21:46:54.194 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:46:54.192+0000 7fe5a191f700 1 -- 172.21.15.184:0/1045034713 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe59c06a230 con 0x7fe59c1047a0 2024-05-21T21:46:54.195 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:46:54.193+0000 7fe5a091d700 1 -- 172.21.15.184:0/1045034713 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7fe584019070 con 0x7fe59c1047a0 2024-05-21T21:46:54.195 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:46:54.193+0000 7fe5a091d700 1 --2- 172.21.15.184:0/1045034713 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fe5880b2880 0x7fe5880b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:46:54.195 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:46:54.193+0000 7fe5a091d700 1 -- 172.21.15.184:0/1045034713 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 4101+0+0 (secure 0 0 0) 0x7fe5840e0ce0 con 0x7fe59c1047a0 2024-05-21T21:46:54.195 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:46:54.194+0000 7fe59affd700 1 --2- 172.21.15.184:0/1045034713 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fe5880b2880 0x7fe5880b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:46:54.195 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:46:54.195+0000 7fe59affd700 1 --2- 172.21.15.184:0/1045034713 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fe5880b2880 0x7fe5880b4d40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe59c104600 tx=0x7fe58c006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:46:54.198 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:46:54.198+0000 7fe5a091d700 1 -- 172.21.15.184:0/1045034713 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe5840a9a70 con 0x7fe59c1047a0 2024-05-21T21:46:54.202 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:53 smithi151 ceph-mon[31623]: Detected new or changed devices on smithi164 2024-05-21T21:46:54.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:53 smithi151 ceph-mon[31623]: Adjusting osd_memory_target on smithi164 to 8544M 2024-05-21T21:46:54.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:53 smithi151 ceph-mon[31623]: osdmap e33: 5 total, 5 up, 5 in 2024-05-21T21:46:54.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:53 smithi164 ceph-mon[36269]: Detected new or changed devices on smithi164 2024-05-21T21:46:54.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:53 smithi164 ceph-mon[36269]: Adjusting osd_memory_target on smithi164 to 8544M 2024-05-21T21:46:54.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:53 smithi164 ceph-mon[36269]: osdmap e33: 5 total, 5 up, 5 in 2024-05-21T21:46:54.410 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:46:54.410+0000 7fe5a191f700 1 -- 172.21.15.184:0/1045034713 --> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fe59c063980 con 0x7fe5880b2880 2024-05-21T21:46:55.192 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:54 smithi184 ceph-mon[36942]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-21T21:46:55.193 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:54 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T21:46:55.193 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:54 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T21:46:55.193 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:54 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:46:55.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:54 smithi151 ceph-mon[31623]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-21T21:46:55.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:54 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T21:46:55.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:54 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T21:46:55.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:54 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:46:55.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:54 smithi164 ceph-mon[36269]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-21T21:46:55.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:54 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T21:46:55.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:54 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T21:46:55.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:54 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:46:56.202 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:55 smithi151 ceph-mon[31623]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:46:56.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:55 smithi184 ceph-mon[36942]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:46:56.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:55 smithi164 ceph-mon[36269]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:46:57.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:56 smithi151 ceph-mon[31623]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-21T21:46:57.235 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:56 smithi184 ceph-mon[36942]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-21T21:46:57.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:56 smithi164 ceph-mon[36269]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-21T21:46:58.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:57 smithi151 ceph-mon[31623]: from='client.? 172.21.15.184:0/1841505181' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fad14221-0ef0-4d40-b414-c6c9f1f9718c"}]: dispatch 2024-05-21T21:46:58.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:57 smithi151 ceph-mon[31623]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fad14221-0ef0-4d40-b414-c6c9f1f9718c"}]: dispatch 2024-05-21T21:46:58.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:57 smithi151 ceph-mon[31623]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fad14221-0ef0-4d40-b414-c6c9f1f9718c"}]': finished 2024-05-21T21:46:58.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:57 smithi151 ceph-mon[31623]: osdmap e34: 6 total, 5 up, 6 in 2024-05-21T21:46:58.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:57 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-21T21:46:58.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:57 smithi184 ceph-mon[36942]: from='client.? 172.21.15.184:0/1841505181' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fad14221-0ef0-4d40-b414-c6c9f1f9718c"}]: dispatch 2024-05-21T21:46:58.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:57 smithi184 ceph-mon[36942]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fad14221-0ef0-4d40-b414-c6c9f1f9718c"}]: dispatch 2024-05-21T21:46:58.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:57 smithi184 ceph-mon[36942]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fad14221-0ef0-4d40-b414-c6c9f1f9718c"}]': finished 2024-05-21T21:46:58.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:57 smithi184 ceph-mon[36942]: osdmap e34: 6 total, 5 up, 6 in 2024-05-21T21:46:58.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:57 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-21T21:46:58.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:57 smithi164 ceph-mon[36269]: from='client.? 172.21.15.184:0/1841505181' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fad14221-0ef0-4d40-b414-c6c9f1f9718c"}]: dispatch 2024-05-21T21:46:58.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:57 smithi164 ceph-mon[36269]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fad14221-0ef0-4d40-b414-c6c9f1f9718c"}]: dispatch 2024-05-21T21:46:58.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:57 smithi164 ceph-mon[36269]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fad14221-0ef0-4d40-b414-c6c9f1f9718c"}]': finished 2024-05-21T21:46:58.287 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:57 smithi164 ceph-mon[36269]: osdmap e34: 6 total, 5 up, 6 in 2024-05-21T21:46:58.287 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:57 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-21T21:46:59.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:58 smithi151 ceph-mon[31623]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-21T21:46:59.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:58 smithi151 ceph-mon[31623]: from='client.? 172.21.15.184:0/2105612771' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T21:46:59.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:58 smithi184 ceph-mon[36942]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-21T21:46:59.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:58 smithi184 ceph-mon[36942]: from='client.? 172.21.15.184:0/2105612771' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T21:46:59.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:58 smithi164 ceph-mon[36269]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-21T21:46:59.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:58 smithi164 ceph-mon[36269]: from='client.? 172.21.15.184:0/2105612771' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T21:47:00.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:46:59 smithi151 ceph-mon[31623]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-21T21:47:00.235 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:46:59 smithi184 ceph-mon[36942]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-21T21:47:00.285 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:46:59 smithi164 ceph-mon[36269]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-21T21:47:02.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:02 smithi151 ceph-mon[31623]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-21T21:47:02.485 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:02 smithi184 ceph-mon[36942]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-21T21:47:02.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:02 smithi164 ceph-mon[36269]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-21T21:47:04.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:04 smithi151 ceph-mon[31623]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-21T21:47:04.481 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:04 smithi184 ceph-mon[36942]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-21T21:47:04.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:04 smithi164 ceph-mon[36269]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-21T21:47:05.417 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:05 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-21T21:47:05.417 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:05 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:47:05.417 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:05 smithi184 ceph-mon[36942]: Deploying daemon osd.5 on smithi184 2024-05-21T21:47:05.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:05 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-21T21:47:05.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:05 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:47:05.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:05 smithi151 ceph-mon[31623]: Deploying daemon osd.5 on smithi184 2024-05-21T21:47:05.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:05 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-21T21:47:05.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:05 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:47:05.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:05 smithi164 ceph-mon[36269]: Deploying daemon osd.5 on smithi184 2024-05-21T21:47:06.292 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:06 smithi184 ceph-mon[36942]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-21T21:47:06.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:06 smithi151 ceph-mon[31623]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-21T21:47:06.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:06 smithi164 ceph-mon[36269]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-21T21:47:08.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:08 smithi151 ceph-mon[31623]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-21T21:47:08.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:08 smithi184 ceph-mon[36942]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-21T21:47:08.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:08 smithi164 ceph-mon[36269]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-21T21:47:11.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:10 smithi151 ceph-mon[31623]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-21T21:47:11.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:10 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:47:11.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:10 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:47:11.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:10 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:47:11.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:10 smithi184 ceph-mon[36942]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-21T21:47:11.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:10 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:47:11.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:10 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:47:11.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:10 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:47:11.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:10 smithi164 ceph-mon[36269]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-21T21:47:11.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:10 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:47:11.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:10 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:47:11.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:10 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:47:13.091 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:12 smithi184 ceph-mon[36942]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-21T21:47:13.149 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:47:13.148+0000 7fe5a091d700 1 -- 172.21.15.184:0/1045034713 <== mgr.14152 v2:172.21.15.151:6800/2796622483 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe59c063980 con 0x7fe5880b2880 2024-05-21T21:47:13.149 INFO:teuthology.orchestra.run.smithi184.stdout:Created osd(s) 5 on host 'smithi184' 2024-05-21T21:47:13.151 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:47:13.151+0000 7fe5a191f700 1 -- 172.21.15.184:0/1045034713 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fe5880b2880 msgr2=0x7fe5880b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:47:13.151 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:47:13.151+0000 7fe5a191f700 1 --2- 172.21.15.184:0/1045034713 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fe5880b2880 0x7fe5880b4d40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe59c104600 tx=0x7fe58c006d20 comp rx=0 tx=0).stop 2024-05-21T21:47:13.152 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:47:13.151+0000 7fe5a191f700 1 -- 172.21.15.184:0/1045034713 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe59c1047a0 msgr2=0x7fe59c199c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:47:13.152 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:47:13.151+0000 7fe5a191f700 1 --2- 172.21.15.184:0/1045034713 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe59c1047a0 0x7fe59c199c90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe58400d980 tx=0x7fe58400dc90 comp rx=0 tx=0).stop 2024-05-21T21:47:13.152 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:47:13.151+0000 7fe5a191f700 1 -- 172.21.15.184:0/1045034713 shutdown_connections 2024-05-21T21:47:13.152 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:47:13.151+0000 7fe5a191f700 1 --2- 172.21.15.184:0/1045034713 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fe5880b2880 0x7fe5880b4d40 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:47:13.152 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:47:13.151+0000 7fe5a191f700 1 --2- 172.21.15.184:0/1045034713 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe59c105140 0x7fe59c1a0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:47:13.152 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:47:13.151+0000 7fe5a191f700 1 --2- 172.21.15.184:0/1045034713 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe59c1035a0 0x7fe59c199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:47:13.152 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:47:13.151+0000 7fe5a191f700 1 --2- 172.21.15.184:0/1045034713 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe59c1047a0 0x7fe59c199c90 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:47:13.152 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:47:13.151+0000 7fe5a191f700 1 -- 172.21.15.184:0/1045034713 >> 172.21.15.184:0/1045034713 conn(0x7fe59c0feb40 msgr2=0x7fe59c107b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:47:13.152 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:47:13.151+0000 7fe5a191f700 1 -- 172.21.15.184:0/1045034713 shutdown_connections 2024-05-21T21:47:13.152 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:47:13.151+0000 7fe5a191f700 1 -- 172.21.15.184:0/1045034713 wait complete. 2024-05-21T21:47:13.202 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:12 smithi151 ceph-mon[31623]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-21T21:47:13.242 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:12 smithi164 ceph-mon[36269]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-21T21:47:13.748 DEBUG:teuthology.orchestra.run.smithi184:osd.5> sudo journalctl -f -n 0 -u ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@osd.5.service 2024-05-21T21:47:13.750 INFO:tasks.cephadm:Deploying osd.6 on smithi151 with /dev/vg_nvme/lv_2... 2024-05-21T21:47:13.750 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-21T21:47:14.198 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:47:14.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:14 smithi151 ceph-mon[31623]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-21T21:47:14.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:14 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:47:14.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:14 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:47:14.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:14 smithi151 ceph-mon[31623]: from='osd.5 [v2:172.21.15.184:6808/3293311106,v1:172.21.15.184:6809/3293311106]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-21T21:47:14.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:14 smithi151 ceph-mon[31623]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-21T21:47:14.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:14 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:47:14.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:14 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:47:14.470 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:14 smithi184 ceph-mon[36942]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-21T21:47:14.470 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:14 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:47:14.470 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:14 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:47:14.470 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:14 smithi184 ceph-mon[36942]: from='osd.5 [v2:172.21.15.184:6808/3293311106,v1:172.21.15.184:6809/3293311106]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-21T21:47:14.471 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:14 smithi184 ceph-mon[36942]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-21T21:47:14.471 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:14 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:47:14.471 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:14 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:47:14.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:14 smithi164 ceph-mon[36269]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-21T21:47:14.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:14 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:47:14.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:14 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:47:14.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:14 smithi164 ceph-mon[36269]: from='osd.5 [v2:172.21.15.184:6808/3293311106,v1:172.21.15.184:6809/3293311106]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-21T21:47:14.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:14 smithi164 ceph-mon[36269]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-21T21:47:14.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:14 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:47:14.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:14 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:47:15.375 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:15 smithi151 ceph-mon[31623]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-21T21:47:15.375 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:15 smithi151 ceph-mon[31623]: from='osd.5 [v2:172.21.15.184:6808/3293311106,v1:172.21.15.184:6809/3293311106]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-05-21T21:47:15.376 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:15 smithi151 ceph-mon[31623]: osdmap e35: 6 total, 5 up, 6 in 2024-05-21T21:47:15.376 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:15 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-21T21:47:15.376 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:15 smithi151 ceph-mon[31623]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-05-21T21:47:15.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:15 smithi184 ceph-mon[36942]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-21T21:47:15.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:15 smithi184 ceph-mon[36942]: from='osd.5 [v2:172.21.15.184:6808/3293311106,v1:172.21.15.184:6809/3293311106]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-05-21T21:47:15.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:15 smithi184 ceph-mon[36942]: osdmap e35: 6 total, 5 up, 6 in 2024-05-21T21:47:15.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:15 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-21T21:47:15.473 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:15 smithi184 ceph-mon[36942]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-05-21T21:47:15.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:15 smithi164 ceph-mon[36269]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-21T21:47:15.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:15 smithi164 ceph-mon[36269]: from='osd.5 [v2:172.21.15.184:6808/3293311106,v1:172.21.15.184:6809/3293311106]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-05-21T21:47:15.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:15 smithi164 ceph-mon[36269]: osdmap e35: 6 total, 5 up, 6 in 2024-05-21T21:47:15.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:15 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-21T21:47:15.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:15 smithi164 ceph-mon[36269]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-05-21T21:47:16.416 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:16 smithi151 ceph-mon[31623]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-21T21:47:16.416 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:16 smithi151 ceph-mon[31623]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2024-05-21T21:47:16.416 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:16 smithi151 ceph-mon[31623]: osdmap e36: 6 total, 5 up, 6 in 2024-05-21T21:47:16.416 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:16 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-21T21:47:16.416 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:16 smithi151 ceph-mon[31623]: from='osd.5 ' entity='osd.5' 2024-05-21T21:47:16.417 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:16 smithi151 ceph-mon[31623]: osdmap e37: 6 total, 5 up, 6 in 2024-05-21T21:47:16.417 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:16 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-21T21:47:16.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:16 smithi184 ceph-mon[36942]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-21T21:47:16.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:16 smithi184 ceph-mon[36942]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2024-05-21T21:47:16.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:16 smithi184 ceph-mon[36942]: osdmap e36: 6 total, 5 up, 6 in 2024-05-21T21:47:16.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:16 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-21T21:47:16.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:16 smithi184 ceph-mon[36942]: from='osd.5 ' entity='osd.5' 2024-05-21T21:47:16.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:16 smithi184 ceph-mon[36942]: osdmap e37: 6 total, 5 up, 6 in 2024-05-21T21:47:16.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:16 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-21T21:47:16.487 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 21 21:47:16 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-5[48868]: 2024-05-21T21:47:16.079+0000 7f58f2404700 -1 osd.5 0 waiting for initial osdmap 2024-05-21T21:47:16.487 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 21 21:47:16 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-5[48868]: 2024-05-21T21:47:16.274+0000 7f58ea1ef700 -1 osd.5 36 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-21T21:47:16.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:16 smithi164 ceph-mon[36269]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-21T21:47:16.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:16 smithi164 ceph-mon[36269]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2024-05-21T21:47:16.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:16 smithi164 ceph-mon[36269]: osdmap e36: 6 total, 5 up, 6 in 2024-05-21T21:47:16.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:16 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-21T21:47:16.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:16 smithi164 ceph-mon[36269]: from='osd.5 ' entity='osd.5' 2024-05-21T21:47:16.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:16 smithi164 ceph-mon[36269]: osdmap e37: 6 total, 5 up, 6 in 2024-05-21T21:47:16.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:16 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-21T21:47:17.004 INFO:teuthology.orchestra.run.smithi151.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-21T21:47:17.004 INFO:teuthology.orchestra.run.smithi151.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-21T21:47:17.004 INFO:teuthology.orchestra.run.smithi151.stderr: stderr: 10+0 records in 2024-05-21T21:47:17.004 INFO:teuthology.orchestra.run.smithi151.stderr:10+0 records out 2024-05-21T21:47:17.004 INFO:teuthology.orchestra.run.smithi151.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0319036 s, 329 MB/s 2024-05-21T21:47:17.004 INFO:teuthology.orchestra.run.smithi151.stderr:--> Zapping successful for: 2024-05-21T21:47:17.363 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:17 smithi151 ceph-mon[31623]: purged_snaps scrub starts 2024-05-21T21:47:17.364 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:17 smithi151 ceph-mon[31623]: purged_snaps scrub ok 2024-05-21T21:47:17.364 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:17 smithi151 ceph-mon[31623]: osd.5 [v2:172.21.15.184:6808/3293311106,v1:172.21.15.184:6809/3293311106] boot 2024-05-21T21:47:17.364 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:17 smithi151 ceph-mon[31623]: osdmap e38: 6 total, 6 up, 6 in 2024-05-21T21:47:17.364 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:17 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-21T21:47:17.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:17 smithi184 ceph-mon[36942]: purged_snaps scrub starts 2024-05-21T21:47:17.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:17 smithi184 ceph-mon[36942]: purged_snaps scrub ok 2024-05-21T21:47:17.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:17 smithi184 ceph-mon[36942]: osd.5 [v2:172.21.15.184:6808/3293311106,v1:172.21.15.184:6809/3293311106] boot 2024-05-21T21:47:17.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:17 smithi184 ceph-mon[36942]: osdmap e38: 6 total, 6 up, 6 in 2024-05-21T21:47:17.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:17 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-21T21:47:17.535 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:17 smithi164 ceph-mon[36269]: purged_snaps scrub starts 2024-05-21T21:47:17.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:17 smithi164 ceph-mon[36269]: purged_snaps scrub ok 2024-05-21T21:47:17.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:17 smithi164 ceph-mon[36269]: osd.5 [v2:172.21.15.184:6808/3293311106,v1:172.21.15.184:6809/3293311106] boot 2024-05-21T21:47:17.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:17 smithi164 ceph-mon[36269]: osdmap e38: 6 total, 6 up, 6 in 2024-05-21T21:47:17.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:17 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-21T21:47:17.886 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi151:vg_nvme/lv_2 2024-05-21T21:47:18.318 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:47:18.624 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:18 smithi151 ceph-mon[31623]: pgmap v151: 1 pgs: 1 peering; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-21T21:47:18.624 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:18 smithi151 ceph-mon[31623]: Detected new or changed devices on smithi184 2024-05-21T21:47:18.624 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:18 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:47:18.625 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:18 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:47:18.625 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:18 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:47:18.625 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:18 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:47:18.625 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:18 smithi151 ceph-mon[31623]: Adjusting osd_memory_target on smithi184 to 8544M 2024-05-21T21:47:18.625 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:18 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:47:18.625 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:18 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:47:18.625 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:18 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:47:18.625 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:18 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:47:18.625 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:18 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:47:18.625 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:18 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:47:18.625 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:18 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:47:18.625 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:18 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:47:18.625 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:18 smithi151 ceph-mon[31623]: osdmap e39: 6 total, 6 up, 6 in 2024-05-21T21:47:18.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:18 smithi184 ceph-mon[36942]: pgmap v151: 1 pgs: 1 peering; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-21T21:47:18.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:18 smithi184 ceph-mon[36942]: Detected new or changed devices on smithi184 2024-05-21T21:47:18.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:18 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:47:18.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:18 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:47:18.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:18 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:47:18.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:18 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:47:18.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:18 smithi184 ceph-mon[36942]: Adjusting osd_memory_target on smithi184 to 8544M 2024-05-21T21:47:18.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:18 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:47:18.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:18 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:47:18.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:18 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:47:18.737 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:18 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:47:18.737 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:18 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:47:18.737 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:18 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:47:18.737 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:18 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:47:18.737 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:18 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:47:18.737 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:18 smithi184 ceph-mon[36942]: osdmap e39: 6 total, 6 up, 6 in 2024-05-21T21:47:18.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:18 smithi164 ceph-mon[36269]: pgmap v151: 1 pgs: 1 peering; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-05-21T21:47:18.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:18 smithi164 ceph-mon[36269]: Detected new or changed devices on smithi184 2024-05-21T21:47:18.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:18 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:47:18.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:18 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:47:18.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:18 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:47:18.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:18 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:47:18.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:18 smithi164 ceph-mon[36269]: Adjusting osd_memory_target on smithi184 to 8544M 2024-05-21T21:47:18.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:18 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:47:18.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:18 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:47:18.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:18 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:47:18.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:18 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:47:18.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:18 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:47:18.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:18 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:47:18.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:18 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:47:18.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:18 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:47:18.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:18 smithi164 ceph-mon[36269]: osdmap e39: 6 total, 6 up, 6 in 2024-05-21T21:47:20.279 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:20 smithi151 ceph-mon[31623]: pgmap v154: 1 pgs: 1 peering; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-05-21T21:47:20.280 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:20 smithi151 ceph-mon[31623]: osdmap e40: 6 total, 6 up, 6 in 2024-05-21T21:47:20.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:47:20.449+0000 7fd953400700 1 -- 172.21.15.151:0/3663004245 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd94c103140 msgr2=0x7fd94c1035c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:47:20.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:47:20.449+0000 7fd953400700 1 --2- 172.21.15.151:0/3663004245 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd94c103140 0x7fd94c1035c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd944009f90 tx=0x7fd94400b330 comp rx=0 tx=0).stop 2024-05-21T21:47:20.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:47:20.449+0000 7fd953400700 1 -- 172.21.15.151:0/3663004245 shutdown_connections 2024-05-21T21:47:20.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:47:20.449+0000 7fd953400700 1 --2- 172.21.15.151:0/3663004245 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd94c103140 0x7fd94c1035c0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:47:20.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:47:20.449+0000 7fd953400700 1 --2- 172.21.15.151:0/3663004245 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd94c101e20 0x7fd94c102240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:47:20.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:47:20.449+0000 7fd953400700 1 --2- 172.21.15.151:0/3663004245 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd94c102780 0x7fd94c102c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:47:20.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:47:20.449+0000 7fd953400700 1 -- 172.21.15.151:0/3663004245 >> 172.21.15.151:0/3663004245 conn(0x7fd94c0fda40 msgr2=0x7fd94c0ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:47:20.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:47:20.450+0000 7fd953400700 1 -- 172.21.15.151:0/3663004245 shutdown_connections 2024-05-21T21:47:20.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:47:20.450+0000 7fd953400700 1 -- 172.21.15.151:0/3663004245 wait complete. 2024-05-21T21:47:20.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:47:20.451+0000 7fd953400700 1 Processor -- start 2024-05-21T21:47:20.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:47:20.451+0000 7fd953400700 1 -- start start 2024-05-21T21:47:20.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:47:20.451+0000 7fd953400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd94c101e20 0x7fd94c19b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:47:20.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:47:20.451+0000 7fd953400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd94c102780 0x7fd94c19be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:47:20.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:47:20.451+0000 7fd953400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd94c103140 0x7fd94c1a2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:47:20.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:47:20.451+0000 7fd953400700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd94c111270 con 0x7fd94c101e20 2024-05-21T21:47:20.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:47:20.451+0000 7fd953400700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd94c1110f0 con 0x7fd94c103140 2024-05-21T21:47:20.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:47:20.451+0000 7fd953400700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd94c1113f0 con 0x7fd94c102780 2024-05-21T21:47:20.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:47:20.452+0000 7fd950a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd94c101e20 0x7fd94c19b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:47:20.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:47:20.452+0000 7fd950a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd94c101e20 0x7fd94c19b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:40390/0 (socket says 172.21.15.151:40390) 2024-05-21T21:47:20.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:47:20.452+0000 7fd950a00700 1 -- 172.21.15.151:0/3174374825 learned_addr learned my addr 172.21.15.151:0/3174374825 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:47:20.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:47:20.452+0000 7fd94be00700 1 --2- 172.21.15.151:0/3174374825 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd94c102780 0x7fd94c19be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:47:20.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:47:20.452+0000 7fd951400700 1 --2- 172.21.15.151:0/3174374825 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd94c103140 0x7fd94c1a2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:47:20.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:47:20.452+0000 7fd950a00700 1 -- 172.21.15.151:0/3174374825 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd94c102780 msgr2=0x7fd94c19be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:47:20.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:47:20.452+0000 7fd950a00700 1 --2- 172.21.15.151:0/3174374825 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd94c102780 0x7fd94c19be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:47:20.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:47:20.452+0000 7fd950a00700 1 -- 172.21.15.151:0/3174374825 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd94c103140 msgr2=0x7fd94c1a2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:47:20.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:47:20.452+0000 7fd950a00700 1 --2- 172.21.15.151:0/3174374825 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd94c103140 0x7fd94c1a2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:47:20.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:47:20.452+0000 7fd950a00700 1 -- 172.21.15.151:0/3174374825 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd94c1a3590 con 0x7fd94c101e20 2024-05-21T21:47:20.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:47:20.452+0000 7fd951400700 1 --2- 172.21.15.151:0/3174374825 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd94c103140 0x7fd94c1a2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:47:20.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:47:20.452+0000 7fd94be00700 1 --2- 172.21.15.151:0/3174374825 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd94c102780 0x7fd94c19be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:47:20.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:47:20.453+0000 7fd950a00700 1 --2- 172.21.15.151:0/3174374825 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd94c101e20 0x7fd94c19b930 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd94000e9f0 tx=0x7fd94000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:47:20.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:47:20.453+0000 7fd949600700 1 -- 172.21.15.151:0/3174374825 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd940004a30 con 0x7fd94c101e20 2024-05-21T21:47:20.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:47:20.453+0000 7fd949600700 1 -- 172.21.15.151:0/3174374825 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9400054d0 con 0x7fd94c101e20 2024-05-21T21:47:20.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:47:20.453+0000 7fd949600700 1 -- 172.21.15.151:0/3174374825 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd94000cc30 con 0x7fd94c101e20 2024-05-21T21:47:20.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:47:20.453+0000 7fd953400700 1 -- 172.21.15.151:0/3174374825 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd94c1a3880 con 0x7fd94c101e20 2024-05-21T21:47:20.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:47:20.453+0000 7fd953400700 1 -- 172.21.15.151:0/3174374825 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd94c1a3d70 con 0x7fd94c101e20 2024-05-21T21:47:20.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:47:20.454+0000 7fd953400700 1 -- 172.21.15.151:0/3174374825 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd94c06a230 con 0x7fd94c101e20 2024-05-21T21:47:20.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:47:20.457+0000 7fd949600700 1 -- 172.21.15.151:0/3174374825 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7fd940004bd0 con 0x7fd94c101e20 2024-05-21T21:47:20.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:47:20.459+0000 7fd949600700 1 --2- 172.21.15.151:0/3174374825 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fd9340b2900 0x7fd9340b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:47:20.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:47:20.459+0000 7fd949600700 1 -- 172.21.15.151:0/3174374825 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4533+0+0 (secure 0 0 0) 0x7fd9400e1490 con 0x7fd94c101e20 2024-05-21T21:47:20.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:47:20.460+0000 7fd94be00700 1 --2- 172.21.15.151:0/3174374825 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fd9340b2900 0x7fd9340b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:47:20.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:47:20.460+0000 7fd94be00700 1 --2- 172.21.15.151:0/3174374825 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fd9340b2900 0x7fd9340b4dc0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd938011490 tx=0x7fd938009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:47:20.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:47:20.464+0000 7fd949600700 1 -- 172.21.15.151:0/3174374825 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd9400a9fd0 con 0x7fd94c101e20 2024-05-21T21:47:20.485 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:20 smithi184 ceph-mon[36942]: pgmap v154: 1 pgs: 1 peering; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-05-21T21:47:20.485 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:20 smithi184 ceph-mon[36942]: osdmap e40: 6 total, 6 up, 6 in 2024-05-21T21:47:20.535 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:20 smithi164 ceph-mon[36269]: pgmap v154: 1 pgs: 1 peering; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-05-21T21:47:20.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:20 smithi164 ceph-mon[36269]: osdmap e40: 6 total, 6 up, 6 in 2024-05-21T21:47:20.705 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:47:20.704+0000 7fd953400700 1 -- 172.21.15.151:0/3174374825 --> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fd94c063980 con 0x7fd9340b2900 2024-05-21T21:47:21.452 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:21 smithi151 ceph-mon[31623]: from='client.14274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:47:21.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:21 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T21:47:21.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:21 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T21:47:21.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:21 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:47:21.485 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:21 smithi184 ceph-mon[36942]: from='client.14274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:47:21.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:21 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T21:47:21.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:21 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T21:47:21.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:21 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:47:21.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:21 smithi164 ceph-mon[36269]: from='client.14274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:47:21.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:21 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T21:47:21.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:21 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T21:47:21.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:21 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:47:22.407 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:22 smithi151 ceph-mon[31623]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 116 KiB/s, 0 objects/s recovering 2024-05-21T21:47:22.485 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:22 smithi184 ceph-mon[36942]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 116 KiB/s, 0 objects/s recovering 2024-05-21T21:47:22.535 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:22 smithi164 ceph-mon[36269]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 116 KiB/s, 0 objects/s recovering 2024-05-21T21:47:24.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:24 smithi151 ceph-mon[31623]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-21T21:47:24.485 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:24 smithi184 ceph-mon[36942]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-21T21:47:24.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:24 smithi164 ceph-mon[36269]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-21T21:47:25.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:25 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/1525972603' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ab8a60f6-0dec-4c1c-89db-f2e2f638f1ff"}]: dispatch 2024-05-21T21:47:25.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:25 smithi151 ceph-mon[31623]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ab8a60f6-0dec-4c1c-89db-f2e2f638f1ff"}]: dispatch 2024-05-21T21:47:25.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:25 smithi151 ceph-mon[31623]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ab8a60f6-0dec-4c1c-89db-f2e2f638f1ff"}]': finished 2024-05-21T21:47:25.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:25 smithi151 ceph-mon[31623]: osdmap e41: 7 total, 6 up, 7 in 2024-05-21T21:47:25.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:25 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-21T21:47:25.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:25 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/1525972603' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ab8a60f6-0dec-4c1c-89db-f2e2f638f1ff"}]: dispatch 2024-05-21T21:47:25.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:25 smithi184 ceph-mon[36942]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ab8a60f6-0dec-4c1c-89db-f2e2f638f1ff"}]: dispatch 2024-05-21T21:47:25.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:25 smithi184 ceph-mon[36942]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ab8a60f6-0dec-4c1c-89db-f2e2f638f1ff"}]': finished 2024-05-21T21:47:25.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:25 smithi184 ceph-mon[36942]: osdmap e41: 7 total, 6 up, 7 in 2024-05-21T21:47:25.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:25 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-21T21:47:25.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:25 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/1525972603' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ab8a60f6-0dec-4c1c-89db-f2e2f638f1ff"}]: dispatch 2024-05-21T21:47:25.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:25 smithi164 ceph-mon[36269]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ab8a60f6-0dec-4c1c-89db-f2e2f638f1ff"}]: dispatch 2024-05-21T21:47:25.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:25 smithi164 ceph-mon[36269]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ab8a60f6-0dec-4c1c-89db-f2e2f638f1ff"}]': finished 2024-05-21T21:47:25.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:25 smithi164 ceph-mon[36269]: osdmap e41: 7 total, 6 up, 7 in 2024-05-21T21:47:25.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:25 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-21T21:47:26.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:26 smithi151 ceph-mon[31623]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-05-21T21:47:26.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:26 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/4060659611' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T21:47:26.485 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:26 smithi184 ceph-mon[36942]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-05-21T21:47:26.485 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:26 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/4060659611' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T21:47:26.535 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:26 smithi164 ceph-mon[36269]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-05-21T21:47:26.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:26 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/4060659611' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T21:47:28.452 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:28 smithi151 ceph-mon[31623]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-21T21:47:28.485 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:28 smithi184 ceph-mon[36942]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-21T21:47:28.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:28 smithi164 ceph-mon[36269]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-21T21:47:30.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:30 smithi151 ceph-mon[31623]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-21T21:47:30.485 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:30 smithi184 ceph-mon[36942]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-21T21:47:30.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:30 smithi164 ceph-mon[36269]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-21T21:47:32.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:32 smithi151 ceph-mon[31623]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-05-21T21:47:32.485 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:32 smithi184 ceph-mon[36942]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-05-21T21:47:32.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:32 smithi164 ceph-mon[36269]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-05-21T21:47:34.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:34 smithi151 ceph-mon[31623]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-05-21T21:47:34.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:34 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-21T21:47:34.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:34 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:47:34.485 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:34 smithi184 ceph-mon[36942]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-05-21T21:47:34.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:34 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-21T21:47:34.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:34 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:47:34.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:34 smithi164 ceph-mon[36269]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-05-21T21:47:34.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:34 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-21T21:47:34.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:34 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:47:35.405 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:35 smithi151 ceph-mon[31623]: Deploying daemon osd.6 on smithi151 2024-05-21T21:47:35.485 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:35 smithi184 ceph-mon[36942]: Deploying daemon osd.6 on smithi151 2024-05-21T21:47:35.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:35 smithi164 ceph-mon[36269]: Deploying daemon osd.6 on smithi151 2024-05-21T21:47:36.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:36 smithi184 ceph-mon[36942]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-05-21T21:47:36.535 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:36 smithi164 ceph-mon[36269]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-05-21T21:47:36.702 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:36 smithi151 ceph-mon[31623]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-05-21T21:47:38.485 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:38 smithi184 ceph-mon[36942]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-05-21T21:47:38.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:38 smithi164 ceph-mon[36269]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-05-21T21:47:38.669 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:38 smithi151 ceph-mon[31623]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-05-21T21:47:40.485 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:40 smithi184 ceph-mon[36942]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-05-21T21:47:40.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:40 smithi164 ceph-mon[36269]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-05-21T21:47:40.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:40 smithi151 ceph-mon[31623]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-05-21T21:47:42.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:42 smithi184 ceph-mon[36942]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-05-21T21:47:42.535 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:42 smithi164 ceph-mon[36269]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-05-21T21:47:42.702 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:42 smithi151 ceph-mon[31623]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-05-21T21:47:43.657 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:43 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:47:43.657 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:43 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:47:43.657 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:43 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:47:43.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:43 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:47:43.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:43 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:47:43.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:43 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:47:43.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:43 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:47:43.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:43 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:47:43.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:43 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:47:44.702 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:44 smithi151 ceph-mon[31623]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-05-21T21:47:44.735 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:44 smithi184 ceph-mon[36942]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-05-21T21:47:44.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:44 smithi164 ceph-mon[36269]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-05-21T21:47:46.673 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:46 smithi151 ceph-mon[31623]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-05-21T21:47:46.673 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:46 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:47:46.673 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:46 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:47:46.673 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:46 smithi151 ceph-mon[31623]: from='osd.6 [v2:172.21.15.151:6818/3377550762,v1:172.21.15.151:6819/3377550762]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-21T21:47:46.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:46 smithi184 ceph-mon[36942]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-05-21T21:47:46.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:46 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:47:46.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:46 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:47:46.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:46 smithi184 ceph-mon[36942]: from='osd.6 [v2:172.21.15.151:6818/3377550762,v1:172.21.15.151:6819/3377550762]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-21T21:47:46.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:46 smithi164 ceph-mon[36269]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-05-21T21:47:46.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:46 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:47:46.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:46 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:47:46.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:46 smithi164 ceph-mon[36269]: from='osd.6 [v2:172.21.15.151:6818/3377550762,v1:172.21.15.151:6819/3377550762]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-21T21:47:47.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:47:47.557+0000 7fd949600700 1 -- 172.21.15.151:0/3174374825 <== mgr.14152 v2:172.21.15.151:6800/2796622483 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd94c063980 con 0x7fd9340b2900 2024-05-21T21:47:47.558 INFO:teuthology.orchestra.run.smithi151.stdout:Created osd(s) 6 on host 'smithi151' 2024-05-21T21:47:47.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:47:47.560+0000 7fd953400700 1 -- 172.21.15.151:0/3174374825 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fd9340b2900 msgr2=0x7fd9340b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:47:47.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:47:47.560+0000 7fd953400700 1 --2- 172.21.15.151:0/3174374825 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fd9340b2900 0x7fd9340b4dc0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd938011490 tx=0x7fd938009260 comp rx=0 tx=0).stop 2024-05-21T21:47:47.561 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:47:47.560+0000 7fd953400700 1 -- 172.21.15.151:0/3174374825 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd94c101e20 msgr2=0x7fd94c19b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:47:47.561 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:47:47.560+0000 7fd953400700 1 --2- 172.21.15.151:0/3174374825 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd94c101e20 0x7fd94c19b930 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd94000e9f0 tx=0x7fd94000ed00 comp rx=0 tx=0).stop 2024-05-21T21:47:47.561 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:47:47.560+0000 7fd953400700 1 -- 172.21.15.151:0/3174374825 shutdown_connections 2024-05-21T21:47:47.561 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:47:47.560+0000 7fd953400700 1 --2- 172.21.15.151:0/3174374825 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fd9340b2900 0x7fd9340b4dc0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:47:47.561 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:47:47.560+0000 7fd953400700 1 --2- 172.21.15.151:0/3174374825 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd94c103140 0x7fd94c1a2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:47:47.561 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:47:47.560+0000 7fd953400700 1 --2- 172.21.15.151:0/3174374825 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd94c101e20 0x7fd94c19b930 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:47:47.561 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:47:47.560+0000 7fd953400700 1 --2- 172.21.15.151:0/3174374825 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd94c102780 0x7fd94c19be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:47:47.561 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:47:47.560+0000 7fd953400700 1 -- 172.21.15.151:0/3174374825 >> 172.21.15.151:0/3174374825 conn(0x7fd94c0fda40 msgr2=0x7fd94c100da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:47:47.561 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:47:47.561+0000 7fd953400700 1 -- 172.21.15.151:0/3174374825 shutdown_connections 2024-05-21T21:47:47.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:47:47.561+0000 7fd953400700 1 -- 172.21.15.151:0/3174374825 wait complete. 2024-05-21T21:47:48.324 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:48 smithi151 ceph-mon[31623]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-05-21T21:47:48.324 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:48 smithi151 ceph-mon[31623]: from='osd.6 [v2:172.21.15.151:6818/3377550762,v1:172.21.15.151:6819/3377550762]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-21T21:47:48.324 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:48 smithi151 ceph-mon[31623]: osdmap e42: 7 total, 6 up, 7 in 2024-05-21T21:47:48.324 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:48 smithi151 ceph-mon[31623]: from='osd.6 [v2:172.21.15.151:6818/3377550762,v1:172.21.15.151:6819/3377550762]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-05-21T21:47:48.324 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:48 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-21T21:47:48.324 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:48 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:47:48.324 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:48 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:47:48.379 DEBUG:teuthology.orchestra.run.smithi151:osd.6> sudo journalctl -f -n 0 -u ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@osd.6.service 2024-05-21T21:47:48.384 INFO:tasks.cephadm:Deploying osd.7 on smithi164 with /dev/vg_nvme/lv_2... 2024-05-21T21:47:48.384 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-21T21:47:48.411 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:48 smithi164 ceph-mon[36269]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-05-21T21:47:48.411 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:48 smithi164 ceph-mon[36269]: from='osd.6 [v2:172.21.15.151:6818/3377550762,v1:172.21.15.151:6819/3377550762]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-21T21:47:48.411 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:48 smithi164 ceph-mon[36269]: osdmap e42: 7 total, 6 up, 7 in 2024-05-21T21:47:48.411 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:48 smithi164 ceph-mon[36269]: from='osd.6 [v2:172.21.15.151:6818/3377550762,v1:172.21.15.151:6819/3377550762]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-05-21T21:47:48.411 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:48 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-21T21:47:48.411 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:48 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:47:48.412 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:48 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:47:48.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:48 smithi184 ceph-mon[36942]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-05-21T21:47:48.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:48 smithi184 ceph-mon[36942]: from='osd.6 [v2:172.21.15.151:6818/3377550762,v1:172.21.15.151:6819/3377550762]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-21T21:47:48.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:48 smithi184 ceph-mon[36942]: osdmap e42: 7 total, 6 up, 7 in 2024-05-21T21:47:48.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:48 smithi184 ceph-mon[36942]: from='osd.6 [v2:172.21.15.151:6818/3377550762,v1:172.21.15.151:6819/3377550762]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-05-21T21:47:48.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:48 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-21T21:47:48.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:48 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:47:48.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:48 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:47:48.632 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.b/config 2024-05-21T21:47:49.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:49 smithi151 ceph-mon[31623]: from='osd.6 [v2:172.21.15.151:6818/3377550762,v1:172.21.15.151:6819/3377550762]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-05-21T21:47:49.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:49 smithi151 ceph-mon[31623]: osdmap e43: 7 total, 6 up, 7 in 2024-05-21T21:47:49.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:49 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-21T21:47:49.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:49 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-21T21:47:49.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:49 smithi184 ceph-mon[36942]: from='osd.6 [v2:172.21.15.151:6818/3377550762,v1:172.21.15.151:6819/3377550762]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-05-21T21:47:49.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:49 smithi184 ceph-mon[36942]: osdmap e43: 7 total, 6 up, 7 in 2024-05-21T21:47:49.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:49 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-21T21:47:49.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:49 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-21T21:47:49.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:49 smithi164 ceph-mon[36269]: from='osd.6 [v2:172.21.15.151:6818/3377550762,v1:172.21.15.151:6819/3377550762]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-05-21T21:47:49.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:49 smithi164 ceph-mon[36269]: osdmap e43: 7 total, 6 up, 7 in 2024-05-21T21:47:49.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:49 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-21T21:47:49.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:49 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-21T21:47:50.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:50 smithi164 ceph-mon[36269]: purged_snaps scrub starts 2024-05-21T21:47:50.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:50 smithi164 ceph-mon[36269]: purged_snaps scrub ok 2024-05-21T21:47:50.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:50 smithi164 ceph-mon[36269]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-05-21T21:47:50.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:50 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-21T21:47:50.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:50 smithi164 ceph-mon[36269]: osd.6 [v2:172.21.15.151:6818/3377550762,v1:172.21.15.151:6819/3377550762] boot 2024-05-21T21:47:50.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:50 smithi164 ceph-mon[36269]: osdmap e44: 7 total, 7 up, 7 in 2024-05-21T21:47:50.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:50 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-21T21:47:50.296 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:50 smithi151 ceph-mon[31623]: purged_snaps scrub starts 2024-05-21T21:47:50.296 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:50 smithi151 ceph-mon[31623]: purged_snaps scrub ok 2024-05-21T21:47:50.297 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:50 smithi151 ceph-mon[31623]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-05-21T21:47:50.297 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:50 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-21T21:47:50.297 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:50 smithi151 ceph-mon[31623]: osd.6 [v2:172.21.15.151:6818/3377550762,v1:172.21.15.151:6819/3377550762] boot 2024-05-21T21:47:50.297 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:50 smithi151 ceph-mon[31623]: osdmap e44: 7 total, 7 up, 7 in 2024-05-21T21:47:50.297 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:50 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-21T21:47:50.437 INFO:teuthology.orchestra.run.smithi164.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-21T21:47:50.437 INFO:teuthology.orchestra.run.smithi164.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-21T21:47:50.437 INFO:teuthology.orchestra.run.smithi164.stderr: stderr: 10+0 records in 2024-05-21T21:47:50.437 INFO:teuthology.orchestra.run.smithi164.stderr:10+0 records out 2024-05-21T21:47:50.437 INFO:teuthology.orchestra.run.smithi164.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0115331 s, 909 MB/s 2024-05-21T21:47:50.437 INFO:teuthology.orchestra.run.smithi164.stderr:--> Zapping successful for: 2024-05-21T21:47:50.485 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:50 smithi184 ceph-mon[36942]: purged_snaps scrub starts 2024-05-21T21:47:50.485 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:50 smithi184 ceph-mon[36942]: purged_snaps scrub ok 2024-05-21T21:47:50.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:50 smithi184 ceph-mon[36942]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-05-21T21:47:50.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:50 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-21T21:47:50.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:50 smithi184 ceph-mon[36942]: osd.6 [v2:172.21.15.151:6818/3377550762,v1:172.21.15.151:6819/3377550762] boot 2024-05-21T21:47:50.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:50 smithi184 ceph-mon[36942]: osdmap e44: 7 total, 7 up, 7 in 2024-05-21T21:47:50.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:50 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-21T21:47:51.029 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi164:vg_nvme/lv_2 2024-05-21T21:47:51.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:51 smithi164 ceph-mon[36269]: osdmap e45: 7 total, 7 up, 7 in 2024-05-21T21:47:51.308 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.b/config 2024-05-21T21:47:51.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:51 smithi151 ceph-mon[31623]: osdmap e45: 7 total, 7 up, 7 in 2024-05-21T21:47:51.485 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:51 smithi184 ceph-mon[36942]: osdmap e45: 7 total, 7 up, 7 in 2024-05-21T21:47:52.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:52 smithi151 ceph-mon[31623]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 213 MiB used, 626 GiB / 626 GiB avail 2024-05-21T21:47:52.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:52 smithi184 ceph-mon[36942]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 213 MiB used, 626 GiB / 626 GiB avail 2024-05-21T21:47:52.525 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:52 smithi164 ceph-mon[36269]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 213 MiB used, 626 GiB / 626 GiB avail 2024-05-21T21:47:52.831 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:47:52.828+0000 7f17597ca700 1 -- 172.21.15.164:0/1358594405 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f17541036a0 msgr2=0x7f1754103b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:47:52.831 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:47:52.828+0000 7f17597ca700 1 --2- 172.21.15.164:0/1358594405 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f17541036a0 0x7f1754103b00 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f17480099c0 tx=0x7f1748009cd0 comp rx=0 tx=0).stop 2024-05-21T21:47:52.831 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:47:52.828+0000 7f17597ca700 1 -- 172.21.15.164:0/1358594405 shutdown_connections 2024-05-21T21:47:52.831 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:47:52.828+0000 7f17597ca700 1 --2- 172.21.15.164:0/1358594405 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1754104040 0x7f175410a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:47:52.831 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:47:52.828+0000 7f17597ca700 1 --2- 172.21.15.164:0/1358594405 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17541024a0 0x7f17541028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:47:52.831 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:47:52.828+0000 7f17597ca700 1 --2- 172.21.15.164:0/1358594405 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f17541036a0 0x7f1754103b00 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:47:52.831 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:47:52.828+0000 7f17597ca700 1 -- 172.21.15.164:0/1358594405 >> 172.21.15.164:0/1358594405 conn(0x7f17540fda60 msgr2=0x7f17540ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:47:52.831 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:47:52.828+0000 7f17597ca700 1 -- 172.21.15.164:0/1358594405 shutdown_connections 2024-05-21T21:47:52.832 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:47:52.828+0000 7f17597ca700 1 -- 172.21.15.164:0/1358594405 wait complete. 2024-05-21T21:47:52.832 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:47:52.829+0000 7f17597ca700 1 Processor -- start 2024-05-21T21:47:52.832 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:47:52.829+0000 7f17597ca700 1 -- start start 2024-05-21T21:47:52.832 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:47:52.829+0000 7f17597ca700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f17541024a0 0x7f1754106430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:47:52.832 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:47:52.829+0000 7f17597ca700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17541036a0 0x7f1754104a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:47:52.832 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:47:52.829+0000 7f17597ca700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1754104040 0x7f1754104fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:47:52.832 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:47:52.829+0000 7f17597ca700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f175410df90 con 0x7f1754104040 2024-05-21T21:47:52.832 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:47:52.829+0000 7f17597ca700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f175410de10 con 0x7f17541024a0 2024-05-21T21:47:52.832 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:47:52.829+0000 7f17597ca700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f175410e110 con 0x7f17541036a0 2024-05-21T21:47:52.832 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:47:52.829+0000 7f1752ffd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f17541024a0 0x7f1754106430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:47:52.832 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:47:52.829+0000 7f1752ffd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f17541024a0 0x7f1754106430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:37782/0 (socket says 172.21.15.164:37782) 2024-05-21T21:47:52.832 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:47:52.829+0000 7f1752ffd700 1 -- 172.21.15.164:0/1168059208 learned_addr learned my addr 172.21.15.164:0/1168059208 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-21T21:47:52.833 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:47:52.830+0000 7f17527fc700 1 --2- 172.21.15.164:0/1168059208 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17541036a0 0x7f1754104a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:47:52.833 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:47:52.830+0000 7f1752ffd700 1 -- 172.21.15.164:0/1168059208 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17541036a0 msgr2=0x7f1754104a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:47:52.833 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:47:52.830+0000 7f1752ffd700 1 --2- 172.21.15.164:0/1168059208 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17541036a0 0x7f1754104a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:47:52.833 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:47:52.830+0000 7f1752ffd700 1 -- 172.21.15.164:0/1168059208 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1754104040 msgr2=0x7f1754104fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:47:52.833 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:47:52.830+0000 7f1752ffd700 1 --2- 172.21.15.164:0/1168059208 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1754104040 0x7f1754104fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:47:52.833 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:47:52.830+0000 7f1752ffd700 1 -- 172.21.15.164:0/1168059208 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1754105870 con 0x7f17541024a0 2024-05-21T21:47:52.833 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:47:52.830+0000 7f1752ffd700 1 --2- 172.21.15.164:0/1168059208 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f17541024a0 0x7f1754106430 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f174400c990 tx=0x7f174400cd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:47:52.833 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:47:52.830+0000 7f173bfff700 1 -- 172.21.15.164:0/1168059208 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1744007b00 con 0x7f17541024a0 2024-05-21T21:47:52.833 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:47:52.830+0000 7f17597ca700 1 -- 172.21.15.164:0/1168059208 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17541a3850 con 0x7f17541024a0 2024-05-21T21:47:52.833 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:47:52.830+0000 7f173bfff700 1 -- 172.21.15.164:0/1168059208 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1744007ca0 con 0x7f17541024a0 2024-05-21T21:47:52.833 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:47:52.830+0000 7f173bfff700 1 -- 172.21.15.164:0/1168059208 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1744018660 con 0x7f17541024a0 2024-05-21T21:47:52.833 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:47:52.830+0000 7f17597ca700 1 -- 172.21.15.164:0/1168059208 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17541a3d90 con 0x7f17541024a0 2024-05-21T21:47:52.833 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:47:52.831+0000 7f17597ca700 1 -- 172.21.15.164:0/1168059208 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f175406a230 con 0x7f17541024a0 2024-05-21T21:47:52.834 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:47:52.832+0000 7f173bfff700 1 -- 172.21.15.164:0/1168059208 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f1744004030 con 0x7f17541024a0 2024-05-21T21:47:52.834 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:47:52.833+0000 7f173bfff700 1 --2- 172.21.15.164:0/1168059208 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f173c0b2880 0x7f173c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:47:52.834 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:47:52.833+0000 7f173bfff700 1 -- 172.21.15.164:0/1168059208 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 4965+0+0 (secure 0 0 0) 0x7f17440e15f0 con 0x7f17541024a0 2024-05-21T21:47:52.834 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:47:52.833+0000 7f17527fc700 1 --2- 172.21.15.164:0/1168059208 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f173c0b2880 0x7f173c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:47:52.835 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:47:52.834+0000 7f17527fc700 1 --2- 172.21.15.164:0/1168059208 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f173c0b2880 0x7f173c0b4d40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1748006010 tx=0x7f1748009f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:47:52.836 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:47:52.837+0000 7f173bfff700 1 -- 172.21.15.164:0/1168059208 <== mon.2 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1744014030 con 0x7f17541024a0 2024-05-21T21:47:53.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:47:53.051+0000 7f17597ca700 1 -- 172.21.15.164:0/1168059208 --> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f17540ffc30 con 0x7f173c0b2880 2024-05-21T21:47:54.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:54 smithi164 ceph-mon[36269]: Detected new or changed devices on smithi151 2024-05-21T21:47:54.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:54 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:47:54.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:54 smithi164 ceph-mon[36269]: from='client.24299 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:47:54.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:54 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:47:54.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:54 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T21:47:54.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:54 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:47:54.287 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:54 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T21:47:54.287 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:54 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:47:54.287 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:54 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:47:54.287 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:54 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:47:54.287 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:54 smithi164 ceph-mon[36269]: Adjusting osd_memory_target on smithi151 to 5696M 2024-05-21T21:47:54.287 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:54 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:47:54.287 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:54 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:47:54.287 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:54 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:47:54.287 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:54 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:47:54.287 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:54 smithi164 ceph-mon[36269]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-21T21:47:54.287 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:54 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:47:54.287 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:54 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:47:54.287 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:54 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:47:54.288 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:54 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:47:54.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:54 smithi151 ceph-mon[31623]: Detected new or changed devices on smithi151 2024-05-21T21:47:54.454 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:54 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:47:54.454 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:54 smithi151 ceph-mon[31623]: from='client.24299 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:47:54.454 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:54 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:47:54.454 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:54 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T21:47:54.454 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:54 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:47:54.454 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:54 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T21:47:54.454 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:54 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:47:54.454 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:54 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:47:54.454 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:54 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:47:54.454 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:54 smithi151 ceph-mon[31623]: Adjusting osd_memory_target on smithi151 to 5696M 2024-05-21T21:47:54.454 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:54 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:47:54.454 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:54 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:47:54.454 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:54 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:47:54.455 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:54 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:47:54.455 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:54 smithi151 ceph-mon[31623]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-21T21:47:54.455 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:54 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:47:54.455 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:54 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:47:54.455 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:54 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:47:54.455 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:54 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:47:54.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:54 smithi184 ceph-mon[36942]: Detected new or changed devices on smithi151 2024-05-21T21:47:54.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:54 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:47:54.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:54 smithi184 ceph-mon[36942]: from='client.24299 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:47:54.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:54 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:47:54.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:54 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T21:47:54.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:54 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:47:54.487 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:54 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T21:47:54.487 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:54 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:47:54.487 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:54 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:47:54.487 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:54 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:47:54.487 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:54 smithi184 ceph-mon[36942]: Adjusting osd_memory_target on smithi151 to 5696M 2024-05-21T21:47:54.487 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:54 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:47:54.487 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:54 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:47:54.488 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:54 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:47:54.488 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:54 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:47:54.488 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:54 smithi184 ceph-mon[36942]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-21T21:47:54.488 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:54 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:47:54.488 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:54 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:47:54.488 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:54 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:47:54.488 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:54 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:47:56.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:56 smithi151 ceph-mon[31623]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-21T21:47:56.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:56 smithi151 ceph-mon[31623]: from='client.? 172.21.15.164:0/1627666' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "db332268-9972-4ed7-a307-822b83a0fbb1"}]: dispatch 2024-05-21T21:47:56.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:56 smithi151 ceph-mon[31623]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "db332268-9972-4ed7-a307-822b83a0fbb1"}]: dispatch 2024-05-21T21:47:56.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:56 smithi151 ceph-mon[31623]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "db332268-9972-4ed7-a307-822b83a0fbb1"}]': finished 2024-05-21T21:47:56.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:56 smithi151 ceph-mon[31623]: osdmap e46: 8 total, 7 up, 8 in 2024-05-21T21:47:56.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:56 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-21T21:47:56.485 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:56 smithi184 ceph-mon[36942]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-21T21:47:56.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:56 smithi184 ceph-mon[36942]: from='client.? 172.21.15.164:0/1627666' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "db332268-9972-4ed7-a307-822b83a0fbb1"}]: dispatch 2024-05-21T21:47:56.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:56 smithi184 ceph-mon[36942]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "db332268-9972-4ed7-a307-822b83a0fbb1"}]: dispatch 2024-05-21T21:47:56.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:56 smithi184 ceph-mon[36942]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "db332268-9972-4ed7-a307-822b83a0fbb1"}]': finished 2024-05-21T21:47:56.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:56 smithi184 ceph-mon[36942]: osdmap e46: 8 total, 7 up, 8 in 2024-05-21T21:47:56.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:56 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-21T21:47:56.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:56 smithi164 ceph-mon[36269]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-21T21:47:56.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:56 smithi164 ceph-mon[36269]: from='client.? 172.21.15.164:0/1627666' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "db332268-9972-4ed7-a307-822b83a0fbb1"}]: dispatch 2024-05-21T21:47:56.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:56 smithi164 ceph-mon[36269]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "db332268-9972-4ed7-a307-822b83a0fbb1"}]: dispatch 2024-05-21T21:47:56.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:56 smithi164 ceph-mon[36269]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "db332268-9972-4ed7-a307-822b83a0fbb1"}]': finished 2024-05-21T21:47:56.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:56 smithi164 ceph-mon[36269]: osdmap e46: 8 total, 7 up, 8 in 2024-05-21T21:47:56.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:56 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-21T21:47:57.452 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:57 smithi151 ceph-mon[31623]: from='client.? 172.21.15.164:0/3005558030' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T21:47:57.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:57 smithi184 ceph-mon[36942]: from='client.? 172.21.15.164:0/3005558030' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T21:47:57.535 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:57 smithi164 ceph-mon[36269]: from='client.? 172.21.15.164:0/3005558030' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T21:47:58.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:47:58 smithi151 ceph-mon[31623]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-21T21:47:58.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:47:58 smithi184 ceph-mon[36942]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-21T21:47:58.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:47:58 smithi164 ceph-mon[36269]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-21T21:48:00.301 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:00 smithi164 ceph-mon[36269]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-21T21:48:00.452 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:00 smithi151 ceph-mon[31623]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-21T21:48:00.485 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:00 smithi184 ceph-mon[36942]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-21T21:48:02.485 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:02 smithi184 ceph-mon[36942]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-21T21:48:02.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:02 smithi164 ceph-mon[36269]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-21T21:48:02.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:02 smithi151 ceph-mon[31623]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-21T21:48:03.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:03 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-21T21:48:03.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:03 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:48:03.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:03 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-21T21:48:03.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:03 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:48:03.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:03 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-21T21:48:03.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:03 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:48:04.444 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:04 smithi164 ceph-mon[36269]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-21T21:48:04.445 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:04 smithi164 ceph-mon[36269]: Deploying daemon osd.7 on smithi164 2024-05-21T21:48:04.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:04 smithi184 ceph-mon[36942]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-21T21:48:04.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:04 smithi184 ceph-mon[36942]: Deploying daemon osd.7 on smithi164 2024-05-21T21:48:04.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:04 smithi151 ceph-mon[31623]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-21T21:48:04.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:04 smithi151 ceph-mon[31623]: Deploying daemon osd.7 on smithi164 2024-05-21T21:48:06.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:06 smithi164 ceph-mon[36269]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-21T21:48:06.485 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:06 smithi184 ceph-mon[36942]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-21T21:48:06.702 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:06 smithi151 ceph-mon[31623]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-21T21:48:08.433 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:08 smithi164 ceph-mon[36269]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-21T21:48:08.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:08 smithi184 ceph-mon[36942]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-21T21:48:08.505 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:08 smithi151 ceph-mon[31623]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-21T21:48:09.883 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:09 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:48:09.883 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:09 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:48:09.883 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:09 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:48:09.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:09 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:48:09.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:09 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:48:09.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:09 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:48:10.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:09 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:48:10.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:09 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:48:10.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:09 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:48:10.902 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:10 smithi164 ceph-mon[36269]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-21T21:48:10.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:10 smithi151 ceph-mon[31623]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-21T21:48:10.985 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:10 smithi184 ceph-mon[36942]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-21T21:48:12.492 INFO:teuthology.orchestra.run.smithi164.stdout:Created osd(s) 7 on host 'smithi164' 2024-05-21T21:48:12.492 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:48:12.491+0000 7f173bfff700 1 -- 172.21.15.164:0/1168059208 <== mgr.14152 v2:172.21.15.151:6800/2796622483 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f17540ffc30 con 0x7f173c0b2880 2024-05-21T21:48:12.494 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:48:12.494+0000 7f17597ca700 1 -- 172.21.15.164:0/1168059208 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f173c0b2880 msgr2=0x7f173c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:48:12.494 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:48:12.494+0000 7f17597ca700 1 --2- 172.21.15.164:0/1168059208 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f173c0b2880 0x7f173c0b4d40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1748006010 tx=0x7f1748009f10 comp rx=0 tx=0).stop 2024-05-21T21:48:12.494 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:48:12.494+0000 7f17597ca700 1 -- 172.21.15.164:0/1168059208 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f17541024a0 msgr2=0x7f1754106430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:48:12.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:48:12.494+0000 7f17597ca700 1 --2- 172.21.15.164:0/1168059208 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f17541024a0 0x7f1754106430 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f174400c990 tx=0x7f174400cd50 comp rx=0 tx=0).stop 2024-05-21T21:48:12.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:48:12.494+0000 7f17597ca700 1 -- 172.21.15.164:0/1168059208 shutdown_connections 2024-05-21T21:48:12.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:48:12.494+0000 7f17597ca700 1 --2- 172.21.15.164:0/1168059208 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f173c0b2880 0x7f173c0b4d40 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:48:12.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:48:12.494+0000 7f17597ca700 1 --2- 172.21.15.164:0/1168059208 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f17541024a0 0x7f1754106430 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:48:12.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:48:12.494+0000 7f17597ca700 1 --2- 172.21.15.164:0/1168059208 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1754104040 0x7f1754104fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:48:12.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:48:12.494+0000 7f17597ca700 1 --2- 172.21.15.164:0/1168059208 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17541036a0 0x7f1754104a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:48:12.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:48:12.494+0000 7f17597ca700 1 -- 172.21.15.164:0/1168059208 >> 172.21.15.164:0/1168059208 conn(0x7f17540fda60 msgr2=0x7f17540ff520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:48:12.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:48:12.494+0000 7f17597ca700 1 -- 172.21.15.164:0/1168059208 shutdown_connections 2024-05-21T21:48:12.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:48:12.494+0000 7f17597ca700 1 -- 172.21.15.164:0/1168059208 wait complete. 2024-05-21T21:48:12.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:12 smithi164 ceph-mon[36269]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-21T21:48:12.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:12 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:48:12.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:12 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:48:12.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:12 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:48:12.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:12 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:48:12.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:12 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:48:12.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:12 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:48:12.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:12 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:48:12.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:12 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:48:12.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:12 smithi151 ceph-mon[31623]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-21T21:48:12.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:12 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:48:12.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:12 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:48:12.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:12 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:48:12.954 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:12 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:48:12.954 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:12 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:48:12.954 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:12 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:48:12.954 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:12 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:48:12.954 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:12 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:48:12.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:12 smithi184 ceph-mon[36942]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-21T21:48:12.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:12 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:48:12.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:12 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:48:12.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:12 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:48:12.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:12 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:48:12.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:12 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:48:12.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:12 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:48:12.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:12 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:48:12.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:12 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:48:13.099 DEBUG:teuthology.orchestra.run.smithi164:osd.7> sudo journalctl -f -n 0 -u ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@osd.7.service 2024-05-21T21:48:13.101 INFO:tasks.cephadm:Deploying osd.8 on smithi184 with /dev/vg_nvme/lv_2... 2024-05-21T21:48:13.101 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-21T21:48:13.347 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.c/config 2024-05-21T21:48:13.707 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:13 smithi184 ceph-mon[36942]: from='osd.7 [v2:172.21.15.164:6816/3869696936,v1:172.21.15.164:6817/3869696936]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-21T21:48:13.707 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:13 smithi184 ceph-mon[36942]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-21T21:48:13.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:13 smithi164 ceph-mon[36269]: from='osd.7 [v2:172.21.15.164:6816/3869696936,v1:172.21.15.164:6817/3869696936]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-21T21:48:13.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:13 smithi164 ceph-mon[36269]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-21T21:48:13.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:13 smithi151 ceph-mon[31623]: from='osd.7 [v2:172.21.15.164:6816/3869696936,v1:172.21.15.164:6817/3869696936]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-21T21:48:13.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:13 smithi151 ceph-mon[31623]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-21T21:48:14.583 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:14 smithi184 ceph-mon[36942]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-21T21:48:14.584 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:14 smithi184 ceph-mon[36942]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-21T21:48:14.584 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:14 smithi184 ceph-mon[36942]: from='osd.7 [v2:172.21.15.164:6816/3869696936,v1:172.21.15.164:6817/3869696936]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-05-21T21:48:14.584 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:14 smithi184 ceph-mon[36942]: osdmap e47: 8 total, 7 up, 8 in 2024-05-21T21:48:14.584 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:14 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-21T21:48:14.584 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:14 smithi184 ceph-mon[36942]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-05-21T21:48:14.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:14 smithi164 ceph-mon[36269]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-21T21:48:14.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:14 smithi164 ceph-mon[36269]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-21T21:48:14.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:14 smithi164 ceph-mon[36269]: from='osd.7 [v2:172.21.15.164:6816/3869696936,v1:172.21.15.164:6817/3869696936]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-05-21T21:48:14.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:14 smithi164 ceph-mon[36269]: osdmap e47: 8 total, 7 up, 8 in 2024-05-21T21:48:14.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:14 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-21T21:48:14.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:14 smithi164 ceph-mon[36269]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-05-21T21:48:14.787 INFO:journalctl@ceph.osd.7.smithi164.stdout:May 21 21:48:14 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-7[54180]: 2024-05-21T21:48:14.547+0000 7f8d5041b700 -1 osd.7 0 waiting for initial osdmap 2024-05-21T21:48:14.787 INFO:journalctl@ceph.osd.7.smithi164.stdout:May 21 21:48:14 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-7[54180]: 2024-05-21T21:48:14.558+0000 7f8d4c20e700 -1 osd.7 48 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-21T21:48:14.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:14 smithi151 ceph-mon[31623]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-21T21:48:14.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:14 smithi151 ceph-mon[31623]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-21T21:48:14.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:14 smithi151 ceph-mon[31623]: from='osd.7 [v2:172.21.15.164:6816/3869696936,v1:172.21.15.164:6817/3869696936]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-05-21T21:48:14.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:14 smithi151 ceph-mon[31623]: osdmap e47: 8 total, 7 up, 8 in 2024-05-21T21:48:14.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:14 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-21T21:48:14.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:14 smithi151 ceph-mon[31623]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-05-21T21:48:15.126 INFO:teuthology.orchestra.run.smithi184.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-21T21:48:15.126 INFO:teuthology.orchestra.run.smithi184.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-21T21:48:15.126 INFO:teuthology.orchestra.run.smithi184.stderr: stderr: 10+0 records in 2024-05-21T21:48:15.126 INFO:teuthology.orchestra.run.smithi184.stderr:10+0 records out 2024-05-21T21:48:15.126 INFO:teuthology.orchestra.run.smithi184.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0118389 s, 886 MB/s 2024-05-21T21:48:15.126 INFO:teuthology.orchestra.run.smithi184.stderr:--> Zapping successful for: 2024-05-21T21:48:15.718 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:15 smithi184 ceph-mon[36942]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-05-21T21:48:15.718 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:15 smithi184 ceph-mon[36942]: osdmap e48: 8 total, 7 up, 8 in 2024-05-21T21:48:15.718 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:15 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-21T21:48:15.718 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:15 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-21T21:48:15.718 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:15 smithi184 ceph-mon[36942]: osd.7 [v2:172.21.15.164:6816/3869696936,v1:172.21.15.164:6817/3869696936] boot 2024-05-21T21:48:15.718 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:15 smithi184 ceph-mon[36942]: osdmap e49: 8 total, 8 up, 8 in 2024-05-21T21:48:15.719 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:15 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-21T21:48:15.746 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi184:vg_nvme/lv_2 2024-05-21T21:48:15.777 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:15 smithi164 ceph-mon[36269]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-05-21T21:48:15.778 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:15 smithi164 ceph-mon[36269]: osdmap e48: 8 total, 7 up, 8 in 2024-05-21T21:48:15.778 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:15 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-21T21:48:15.778 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:15 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-21T21:48:15.778 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:15 smithi164 ceph-mon[36269]: osd.7 [v2:172.21.15.164:6816/3869696936,v1:172.21.15.164:6817/3869696936] boot 2024-05-21T21:48:15.778 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:15 smithi164 ceph-mon[36269]: osdmap e49: 8 total, 8 up, 8 in 2024-05-21T21:48:15.778 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:15 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-21T21:48:15.952 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:15 smithi151 ceph-mon[31623]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-05-21T21:48:15.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:15 smithi151 ceph-mon[31623]: osdmap e48: 8 total, 7 up, 8 in 2024-05-21T21:48:15.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:15 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-21T21:48:15.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:15 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-21T21:48:15.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:15 smithi151 ceph-mon[31623]: osd.7 [v2:172.21.15.164:6816/3869696936,v1:172.21.15.164:6817/3869696936] boot 2024-05-21T21:48:15.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:15 smithi151 ceph-mon[31623]: osdmap e49: 8 total, 8 up, 8 in 2024-05-21T21:48:15.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:15 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-21T21:48:15.995 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.c/config 2024-05-21T21:48:16.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:16 smithi184 ceph-mon[36942]: purged_snaps scrub starts 2024-05-21T21:48:16.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:16 smithi184 ceph-mon[36942]: purged_snaps scrub ok 2024-05-21T21:48:16.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:16 smithi184 ceph-mon[36942]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-21T21:48:16.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:16 smithi184 ceph-mon[36942]: Detected new or changed devices on smithi164 2024-05-21T21:48:16.737 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:16 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:48:16.737 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:16 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:48:16.737 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:16 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:48:16.737 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:16 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:48:16.737 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:16 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:48:16.737 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:16 smithi184 ceph-mon[36942]: Adjusting osd_memory_target on smithi164 to 5696M 2024-05-21T21:48:16.737 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:16 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:48:16.737 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:16 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:48:16.737 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:16 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:48:16.737 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:16 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:48:16.737 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:16 smithi184 ceph-mon[36942]: osdmap e50: 8 total, 8 up, 8 in 2024-05-21T21:48:16.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:16 smithi151 ceph-mon[31623]: purged_snaps scrub starts 2024-05-21T21:48:16.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:16 smithi151 ceph-mon[31623]: purged_snaps scrub ok 2024-05-21T21:48:16.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:16 smithi151 ceph-mon[31623]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-21T21:48:16.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:16 smithi151 ceph-mon[31623]: Detected new or changed devices on smithi164 2024-05-21T21:48:16.954 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:16 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:48:16.954 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:16 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:48:16.954 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:16 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:48:16.954 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:16 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:48:16.954 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:16 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:48:16.954 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:16 smithi151 ceph-mon[31623]: Adjusting osd_memory_target on smithi164 to 5696M 2024-05-21T21:48:16.954 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:16 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:48:16.954 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:16 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:48:16.954 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:16 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:48:16.954 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:16 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:48:16.954 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:16 smithi151 ceph-mon[31623]: osdmap e50: 8 total, 8 up, 8 in 2024-05-21T21:48:17.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:16 smithi164 ceph-mon[36269]: purged_snaps scrub starts 2024-05-21T21:48:17.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:16 smithi164 ceph-mon[36269]: purged_snaps scrub ok 2024-05-21T21:48:17.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:16 smithi164 ceph-mon[36269]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-05-21T21:48:17.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:16 smithi164 ceph-mon[36269]: Detected new or changed devices on smithi164 2024-05-21T21:48:17.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:16 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:48:17.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:16 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:48:17.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:16 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:48:17.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:16 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:48:17.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:16 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:48:17.037 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:16 smithi164 ceph-mon[36269]: Adjusting osd_memory_target on smithi164 to 5696M 2024-05-21T21:48:17.037 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:16 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:48:17.037 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:16 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:48:17.037 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:16 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:48:17.037 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:16 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:48:17.037 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:16 smithi164 ceph-mon[36269]: osdmap e50: 8 total, 8 up, 8 in 2024-05-21T21:48:17.488 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:48:17.486+0000 7f77ba0d3700 1 -- 172.21.15.184:0/4208159194 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77b41036b0 msgr2=0x7f77b4103b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:48:17.488 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:48:17.486+0000 7f77ba0d3700 1 --2- 172.21.15.184:0/4208159194 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77b41036b0 0x7f77b4103b10 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f77a8009990 tx=0x7f77a8009ca0 comp rx=0 tx=0).stop 2024-05-21T21:48:17.488 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:48:17.486+0000 7f77ba0d3700 1 -- 172.21.15.184:0/4208159194 shutdown_connections 2024-05-21T21:48:17.488 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:48:17.486+0000 7f77ba0d3700 1 --2- 172.21.15.184:0/4208159194 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f77b41024b0 0x7f77b41028d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:48:17.488 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:48:17.486+0000 7f77ba0d3700 1 --2- 172.21.15.184:0/4208159194 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f77b4104050 0x7f77b410a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:48:17.488 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:48:17.486+0000 7f77ba0d3700 1 --2- 172.21.15.184:0/4208159194 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77b41036b0 0x7f77b4103b10 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:48:17.489 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:48:17.486+0000 7f77ba0d3700 1 -- 172.21.15.184:0/4208159194 >> 172.21.15.184:0/4208159194 conn(0x7f77b40fda70 msgr2=0x7f77b40ffe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:48:17.489 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:48:17.486+0000 7f77ba0d3700 1 -- 172.21.15.184:0/4208159194 shutdown_connections 2024-05-21T21:48:17.489 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:48:17.486+0000 7f77ba0d3700 1 -- 172.21.15.184:0/4208159194 wait complete. 2024-05-21T21:48:17.489 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:48:17.486+0000 7f77ba0d3700 1 Processor -- start 2024-05-21T21:48:17.489 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:48:17.486+0000 7f77ba0d3700 1 -- start start 2024-05-21T21:48:17.489 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:48:17.487+0000 7f77ba0d3700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f77b41024b0 0x7f77b406c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:48:17.489 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:48:17.487+0000 7f77ba0d3700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77b41036b0 0x7f77b41a0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:48:17.489 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:48:17.487+0000 7f77ba0d3700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f77b4104050 0x7f77b41a2e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:48:17.489 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:48:17.487+0000 7f77ba0d3700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f77b410dfa0 con 0x7f77b41024b0 2024-05-21T21:48:17.489 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:48:17.487+0000 7f77ba0d3700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f77b410de20 con 0x7f77b4104050 2024-05-21T21:48:17.489 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:48:17.487+0000 7f77ba0d3700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f77b410e120 con 0x7f77b41036b0 2024-05-21T21:48:17.489 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:48:17.487+0000 7f77b2ffd700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77b41036b0 0x7f77b41a0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:48:17.489 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:48:17.487+0000 7f77b2ffd700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77b41036b0 0x7f77b41a0930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:49732/0 (socket says 172.21.15.184:49732) 2024-05-21T21:48:17.490 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:48:17.487+0000 7f77b2ffd700 1 -- 172.21.15.184:0/2928191386 learned_addr learned my addr 172.21.15.184:0/2928191386 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-05-21T21:48:17.490 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:48:17.487+0000 7f77b2ffd700 1 -- 172.21.15.184:0/2928191386 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f77b4104050 msgr2=0x7f77b41a2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:48:17.490 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:48:17.487+0000 7f77b2ffd700 1 --2- 172.21.15.184:0/2928191386 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f77b4104050 0x7f77b41a2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:48:17.490 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:48:17.487+0000 7f77b2ffd700 1 -- 172.21.15.184:0/2928191386 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f77b41024b0 msgr2=0x7f77b406c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:48:17.490 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:48:17.487+0000 7f77b2ffd700 1 --2- 172.21.15.184:0/2928191386 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f77b41024b0 0x7f77b406c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:48:17.490 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:48:17.487+0000 7f77b2ffd700 1 -- 172.21.15.184:0/2928191386 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77b41a35d0 con 0x7f77b41036b0 2024-05-21T21:48:17.490 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:48:17.487+0000 7f77b2ffd700 1 --2- 172.21.15.184:0/2928191386 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77b41036b0 0x7f77b41a0930 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f77a8006010 tx=0x7f77a80039f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:48:17.490 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:48:17.487+0000 7f77b0ff9700 1 -- 172.21.15.184:0/2928191386 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77a8027070 con 0x7f77b41036b0 2024-05-21T21:48:17.490 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:48:17.487+0000 7f77b0ff9700 1 -- 172.21.15.184:0/2928191386 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77a8022470 con 0x7f77b41036b0 2024-05-21T21:48:17.490 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:48:17.487+0000 7f77ba0d3700 1 -- 172.21.15.184:0/2928191386 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77b41a3860 con 0x7f77b41036b0 2024-05-21T21:48:17.490 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:48:17.487+0000 7f77b0ff9700 1 -- 172.21.15.184:0/2928191386 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77a800f6c0 con 0x7f77b41036b0 2024-05-21T21:48:17.490 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:48:17.487+0000 7f77ba0d3700 1 -- 172.21.15.184:0/2928191386 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77b41a3d10 con 0x7f77b41036b0 2024-05-21T21:48:17.491 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:48:17.488+0000 7f77ba0d3700 1 -- 172.21.15.184:0/2928191386 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77b406a230 con 0x7f77b41036b0 2024-05-21T21:48:17.491 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:48:17.490+0000 7f77b0ff9700 1 -- 172.21.15.184:0/2928191386 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f77a802a070 con 0x7f77b41036b0 2024-05-21T21:48:17.492 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:48:17.490+0000 7f77b0ff9700 1 --2- 172.21.15.184:0/2928191386 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f779c0b2900 0x7f779c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:48:17.492 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:48:17.491+0000 7f77b0ff9700 1 -- 172.21.15.184:0/2928191386 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5397+0+0 (secure 0 0 0) 0x7f77a80e2770 con 0x7f77b41036b0 2024-05-21T21:48:17.492 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:48:17.491+0000 7f77b37fe700 1 --2- 172.21.15.184:0/2928191386 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f779c0b2900 0x7f779c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:48:17.492 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:48:17.492+0000 7f77b37fe700 1 --2- 172.21.15.184:0/2928191386 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f779c0b2900 0x7f779c0b4dc0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f77b4103510 tx=0x7f77a4005ca0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:48:17.494 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:48:17.494+0000 7f77b0ff9700 1 -- 172.21.15.184:0/2928191386 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f77a8013070 con 0x7f77b41036b0 2024-05-21T21:48:17.711 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:48:17.710+0000 7f77ba0d3700 1 -- 172.21.15.184:0/2928191386 --> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f77b4063980 con 0x7f779c0b2900 2024-05-21T21:48:18.586 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:18 smithi184 ceph-mon[36942]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-05-21T21:48:18.586 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:18 smithi184 ceph-mon[36942]: osdmap e51: 8 total, 8 up, 8 in 2024-05-21T21:48:18.587 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:18 smithi184 ceph-mon[36942]: from='client.24316 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:48:18.587 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:18 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T21:48:18.587 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:18 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T21:48:18.587 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:18 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:48:18.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:18 smithi151 ceph-mon[31623]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-05-21T21:48:18.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:18 smithi151 ceph-mon[31623]: osdmap e51: 8 total, 8 up, 8 in 2024-05-21T21:48:18.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:18 smithi151 ceph-mon[31623]: from='client.24316 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:48:18.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:18 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T21:48:18.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:18 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T21:48:18.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:18 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:48:19.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:18 smithi164 ceph-mon[36269]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-05-21T21:48:19.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:18 smithi164 ceph-mon[36269]: osdmap e51: 8 total, 8 up, 8 in 2024-05-21T21:48:19.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:18 smithi164 ceph-mon[36269]: from='client.24316 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:48:19.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:18 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T21:48:19.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:18 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T21:48:19.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:18 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:48:20.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:20 smithi151 ceph-mon[31623]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-05-21T21:48:20.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:20 smithi151 ceph-mon[31623]: from='client.? 172.21.15.184:0/3567208166' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fe2fbe1c-549a-4947-a188-426dfca04d10"}]: dispatch 2024-05-21T21:48:20.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:20 smithi151 ceph-mon[31623]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fe2fbe1c-549a-4947-a188-426dfca04d10"}]: dispatch 2024-05-21T21:48:20.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:20 smithi151 ceph-mon[31623]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fe2fbe1c-549a-4947-a188-426dfca04d10"}]': finished 2024-05-21T21:48:20.954 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:20 smithi151 ceph-mon[31623]: osdmap e52: 9 total, 8 up, 9 in 2024-05-21T21:48:20.954 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:20 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-21T21:48:20.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:20 smithi184 ceph-mon[36942]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-05-21T21:48:20.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:20 smithi184 ceph-mon[36942]: from='client.? 172.21.15.184:0/3567208166' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fe2fbe1c-549a-4947-a188-426dfca04d10"}]: dispatch 2024-05-21T21:48:20.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:20 smithi184 ceph-mon[36942]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fe2fbe1c-549a-4947-a188-426dfca04d10"}]: dispatch 2024-05-21T21:48:20.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:20 smithi184 ceph-mon[36942]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fe2fbe1c-549a-4947-a188-426dfca04d10"}]': finished 2024-05-21T21:48:20.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:20 smithi184 ceph-mon[36942]: osdmap e52: 9 total, 8 up, 9 in 2024-05-21T21:48:20.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:20 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-21T21:48:21.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:20 smithi164 ceph-mon[36269]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-05-21T21:48:21.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:20 smithi164 ceph-mon[36269]: from='client.? 172.21.15.184:0/3567208166' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fe2fbe1c-549a-4947-a188-426dfca04d10"}]: dispatch 2024-05-21T21:48:21.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:20 smithi164 ceph-mon[36269]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fe2fbe1c-549a-4947-a188-426dfca04d10"}]: dispatch 2024-05-21T21:48:21.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:20 smithi164 ceph-mon[36269]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fe2fbe1c-549a-4947-a188-426dfca04d10"}]': finished 2024-05-21T21:48:21.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:20 smithi164 ceph-mon[36269]: osdmap e52: 9 total, 8 up, 9 in 2024-05-21T21:48:21.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:20 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-21T21:48:21.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:21 smithi151 ceph-mon[31623]: from='client.? 172.21.15.184:0/3717569687' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T21:48:21.985 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:21 smithi184 ceph-mon[36942]: from='client.? 172.21.15.184:0/3717569687' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T21:48:22.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:21 smithi164 ceph-mon[36269]: from='client.? 172.21.15.184:0/3717569687' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T21:48:22.887 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:22 smithi184 ceph-mon[36942]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-05-21T21:48:22.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:22 smithi151 ceph-mon[31623]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-05-21T21:48:23.035 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:22 smithi164 ceph-mon[36269]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-05-21T21:48:24.882 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:24 smithi184 ceph-mon[36942]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-05-21T21:48:24.952 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:24 smithi151 ceph-mon[31623]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-05-21T21:48:25.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:24 smithi164 ceph-mon[36269]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-05-21T21:48:26.700 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:26 smithi184 ceph-mon[36942]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-21T21:48:26.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:26 smithi151 ceph-mon[31623]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-21T21:48:27.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:26 smithi164 ceph-mon[36269]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-21T21:48:28.677 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:28 smithi184 ceph-mon[36942]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-05-21T21:48:28.678 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:28 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-21T21:48:28.678 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:28 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:48:28.678 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:28 smithi184 ceph-mon[36942]: Deploying daemon osd.8 on smithi184 2024-05-21T21:48:28.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:28 smithi151 ceph-mon[31623]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-05-21T21:48:28.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:28 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-21T21:48:28.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:28 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:48:28.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:28 smithi151 ceph-mon[31623]: Deploying daemon osd.8 on smithi184 2024-05-21T21:48:29.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:28 smithi164 ceph-mon[36269]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-05-21T21:48:29.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:28 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-21T21:48:29.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:28 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:48:29.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:28 smithi164 ceph-mon[36269]: Deploying daemon osd.8 on smithi184 2024-05-21T21:48:30.922 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:30 smithi184 ceph-mon[36942]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-21T21:48:30.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:30 smithi151 ceph-mon[31623]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-21T21:48:31.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:30 smithi164 ceph-mon[36269]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-21T21:48:32.660 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:32 smithi184 ceph-mon[36942]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-05-21T21:48:32.853 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:32 smithi164 ceph-mon[36269]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-05-21T21:48:32.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:32 smithi151 ceph-mon[31623]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-05-21T21:48:34.485 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:34 smithi184 ceph-mon[36942]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-05-21T21:48:34.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:34 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:48:34.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:34 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:48:34.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:34 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:48:34.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:34 smithi164 ceph-mon[36269]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-05-21T21:48:34.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:34 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:48:34.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:34 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:48:34.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:34 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:48:34.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:34 smithi151 ceph-mon[31623]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-05-21T21:48:34.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:34 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:48:34.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:34 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:48:34.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:34 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:48:36.437 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:36 smithi184 ceph-mon[36942]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-05-21T21:48:36.437 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:36 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:48:36.437 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:36 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:48:36.437 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:36 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:48:36.437 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:36 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:48:36.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:36 smithi164 ceph-mon[36269]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-05-21T21:48:36.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:36 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:48:36.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:36 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:48:36.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:36 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:48:36.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:36 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:48:36.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:36 smithi151 ceph-mon[31623]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-05-21T21:48:36.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:36 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:48:36.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:36 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:48:36.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:36 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:48:36.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:36 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:48:37.095 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:48:37.095+0000 7f77b0ff9700 1 -- 172.21.15.184:0/2928191386 <== mgr.14152 v2:172.21.15.151:6800/2796622483 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f77b4063980 con 0x7f779c0b2900 2024-05-21T21:48:37.096 INFO:teuthology.orchestra.run.smithi184.stdout:Created osd(s) 8 on host 'smithi184' 2024-05-21T21:48:37.098 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:48:37.097+0000 7f77ba0d3700 1 -- 172.21.15.184:0/2928191386 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f779c0b2900 msgr2=0x7f779c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:48:37.098 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:48:37.097+0000 7f77ba0d3700 1 --2- 172.21.15.184:0/2928191386 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f779c0b2900 0x7f779c0b4dc0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f77b4103510 tx=0x7f77a4005ca0 comp rx=0 tx=0).stop 2024-05-21T21:48:37.098 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:48:37.097+0000 7f77ba0d3700 1 -- 172.21.15.184:0/2928191386 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77b41036b0 msgr2=0x7f77b41a0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:48:37.098 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:48:37.097+0000 7f77ba0d3700 1 --2- 172.21.15.184:0/2928191386 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77b41036b0 0x7f77b41a0930 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f77a8006010 tx=0x7f77a80039f0 comp rx=0 tx=0).stop 2024-05-21T21:48:37.098 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:48:37.097+0000 7f77ba0d3700 1 -- 172.21.15.184:0/2928191386 shutdown_connections 2024-05-21T21:48:37.098 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:48:37.097+0000 7f77ba0d3700 1 --2- 172.21.15.184:0/2928191386 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f779c0b2900 0x7f779c0b4dc0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:48:37.098 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:48:37.097+0000 7f77ba0d3700 1 --2- 172.21.15.184:0/2928191386 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f77b4104050 0x7f77b41a2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:48:37.098 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:48:37.097+0000 7f77ba0d3700 1 --2- 172.21.15.184:0/2928191386 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f77b41024b0 0x7f77b406c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:48:37.099 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:48:37.097+0000 7f77ba0d3700 1 --2- 172.21.15.184:0/2928191386 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77b41036b0 0x7f77b41a0930 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:48:37.099 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:48:37.097+0000 7f77ba0d3700 1 -- 172.21.15.184:0/2928191386 >> 172.21.15.184:0/2928191386 conn(0x7f77b40fda70 msgr2=0x7f77b4106ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:48:37.099 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:48:37.097+0000 7f77ba0d3700 1 -- 172.21.15.184:0/2928191386 shutdown_connections 2024-05-21T21:48:37.099 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:48:37.097+0000 7f77ba0d3700 1 -- 172.21.15.184:0/2928191386 wait complete. 2024-05-21T21:48:37.392 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:37 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:48:37.393 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:37 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:48:37.393 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:37 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:48:37.393 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:37 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:48:37.393 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:37 smithi184 ceph-mon[36942]: from='osd.8 [v2:172.21.15.184:6816/575273541,v1:172.21.15.184:6817/575273541]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-21T21:48:37.393 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:37 smithi184 ceph-mon[36942]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-21T21:48:37.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:37 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:48:37.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:37 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:48:37.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:37 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:48:37.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:37 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:48:37.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:37 smithi164 ceph-mon[36269]: from='osd.8 [v2:172.21.15.184:6816/575273541,v1:172.21.15.184:6817/575273541]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-21T21:48:37.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:37 smithi164 ceph-mon[36269]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-21T21:48:37.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:37 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:48:37.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:37 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:48:37.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:37 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:48:37.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:37 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:48:37.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:37 smithi151 ceph-mon[31623]: from='osd.8 [v2:172.21.15.184:6816/575273541,v1:172.21.15.184:6817/575273541]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-21T21:48:37.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:37 smithi151 ceph-mon[31623]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-21T21:48:37.713 DEBUG:teuthology.orchestra.run.smithi184:osd.8> sudo journalctl -f -n 0 -u ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@osd.8.service 2024-05-21T21:48:37.715 INFO:tasks.cephadm:Deploying osd.9 on smithi151 with /dev/vg_nvme/lv_1... 2024-05-21T21:48:37.715 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-21T21:48:38.163 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:48:38.489 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:38 smithi151 ceph-mon[31623]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-05-21T21:48:38.489 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:38 smithi151 ceph-mon[31623]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-21T21:48:38.489 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:38 smithi151 ceph-mon[31623]: osdmap e53: 9 total, 8 up, 9 in 2024-05-21T21:48:38.489 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:38 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-21T21:48:38.489 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:38 smithi151 ceph-mon[31623]: from='osd.8 [v2:172.21.15.184:6816/575273541,v1:172.21.15.184:6817/575273541]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-05-21T21:48:38.489 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:38 smithi151 ceph-mon[31623]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-05-21T21:48:38.507 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:38 smithi184 ceph-mon[36942]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-05-21T21:48:38.507 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:38 smithi184 ceph-mon[36942]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-21T21:48:38.507 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:38 smithi184 ceph-mon[36942]: osdmap e53: 9 total, 8 up, 9 in 2024-05-21T21:48:38.507 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:38 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-21T21:48:38.507 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:38 smithi184 ceph-mon[36942]: from='osd.8 [v2:172.21.15.184:6816/575273541,v1:172.21.15.184:6817/575273541]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-05-21T21:48:38.507 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:38 smithi184 ceph-mon[36942]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-05-21T21:48:38.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:38 smithi164 ceph-mon[36269]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-05-21T21:48:38.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:38 smithi164 ceph-mon[36269]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-21T21:48:38.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:38 smithi164 ceph-mon[36269]: osdmap e53: 9 total, 8 up, 9 in 2024-05-21T21:48:38.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:38 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-21T21:48:38.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:38 smithi164 ceph-mon[36269]: from='osd.8 [v2:172.21.15.184:6816/575273541,v1:172.21.15.184:6817/575273541]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-05-21T21:48:38.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:38 smithi164 ceph-mon[36269]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-05-21T21:48:39.486 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 21:48:39 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T21:48:39.072+0000 7f452e720700 -1 osd.8 0 waiting for initial osdmap 2024-05-21T21:48:39.486 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 21:48:39 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T21:48:39.083+0000 7f452a513700 -1 osd.8 54 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-21T21:48:40.198 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:40 smithi184 ceph-mon[36942]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2024-05-21T21:48:40.198 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:40 smithi184 ceph-mon[36942]: osdmap e54: 9 total, 8 up, 9 in 2024-05-21T21:48:40.199 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:40 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-21T21:48:40.199 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:40 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-21T21:48:40.199 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:40 smithi184 ceph-mon[36942]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-05-21T21:48:40.202 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:40 smithi151 ceph-mon[31623]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2024-05-21T21:48:40.202 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:40 smithi151 ceph-mon[31623]: osdmap e54: 9 total, 8 up, 9 in 2024-05-21T21:48:40.202 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:40 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-21T21:48:40.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:40 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-21T21:48:40.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:40 smithi151 ceph-mon[31623]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-05-21T21:48:40.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:40 smithi164 ceph-mon[36269]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2024-05-21T21:48:40.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:40 smithi164 ceph-mon[36269]: osdmap e54: 9 total, 8 up, 9 in 2024-05-21T21:48:40.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:40 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-21T21:48:40.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:40 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-21T21:48:40.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:40 smithi164 ceph-mon[36269]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-05-21T21:48:41.079 INFO:teuthology.orchestra.run.smithi151.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-21T21:48:41.079 INFO:teuthology.orchestra.run.smithi151.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-21T21:48:41.079 INFO:teuthology.orchestra.run.smithi151.stderr: stderr: 10+0 records in 2024-05-21T21:48:41.079 INFO:teuthology.orchestra.run.smithi151.stderr:10+0 records out 2024-05-21T21:48:41.079 INFO:teuthology.orchestra.run.smithi151.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0350207 s, 299 MB/s 2024-05-21T21:48:41.079 INFO:teuthology.orchestra.run.smithi151.stderr:--> Zapping successful for: 2024-05-21T21:48:41.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:41 smithi151 ceph-mon[31623]: purged_snaps scrub starts 2024-05-21T21:48:41.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:41 smithi151 ceph-mon[31623]: purged_snaps scrub ok 2024-05-21T21:48:41.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:41 smithi151 ceph-mon[31623]: osd.8 [v2:172.21.15.184:6816/575273541,v1:172.21.15.184:6817/575273541] boot 2024-05-21T21:48:41.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:41 smithi151 ceph-mon[31623]: osdmap e55: 9 total, 9 up, 9 in 2024-05-21T21:48:41.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:41 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-21T21:48:41.454 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:41 smithi151 ceph-mon[31623]: Detected new or changed devices on smithi184 2024-05-21T21:48:41.454 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:41 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:48:41.454 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:41 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:48:41.454 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:41 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:48:41.454 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:41 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:48:41.454 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:41 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:48:41.454 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:41 smithi151 ceph-mon[31623]: Adjusting osd_memory_target on smithi184 to 5696M 2024-05-21T21:48:41.454 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:41 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:48:41.454 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:41 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:48:41.454 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:41 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:48:41.454 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:41 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:48:41.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:41 smithi184 ceph-mon[36942]: purged_snaps scrub starts 2024-05-21T21:48:41.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:41 smithi184 ceph-mon[36942]: purged_snaps scrub ok 2024-05-21T21:48:41.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:41 smithi184 ceph-mon[36942]: osd.8 [v2:172.21.15.184:6816/575273541,v1:172.21.15.184:6817/575273541] boot 2024-05-21T21:48:41.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:41 smithi184 ceph-mon[36942]: osdmap e55: 9 total, 9 up, 9 in 2024-05-21T21:48:41.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:41 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-21T21:48:41.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:41 smithi184 ceph-mon[36942]: Detected new or changed devices on smithi184 2024-05-21T21:48:41.487 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:41 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:48:41.487 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:41 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:48:41.487 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:41 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:48:41.487 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:41 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:48:41.487 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:41 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:48:41.487 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:41 smithi184 ceph-mon[36942]: Adjusting osd_memory_target on smithi184 to 5696M 2024-05-21T21:48:41.487 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:41 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:48:41.487 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:41 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:48:41.487 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:41 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:48:41.487 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:41 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:48:41.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:41 smithi164 ceph-mon[36269]: purged_snaps scrub starts 2024-05-21T21:48:41.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:41 smithi164 ceph-mon[36269]: purged_snaps scrub ok 2024-05-21T21:48:41.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:41 smithi164 ceph-mon[36269]: osd.8 [v2:172.21.15.184:6816/575273541,v1:172.21.15.184:6817/575273541] boot 2024-05-21T21:48:41.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:41 smithi164 ceph-mon[36269]: osdmap e55: 9 total, 9 up, 9 in 2024-05-21T21:48:41.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:41 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-21T21:48:41.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:41 smithi164 ceph-mon[36269]: Detected new or changed devices on smithi184 2024-05-21T21:48:41.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:41 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:48:41.537 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:41 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:48:41.537 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:41 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:48:41.537 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:41 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:48:41.537 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:41 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:48:41.537 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:41 smithi164 ceph-mon[36269]: Adjusting osd_memory_target on smithi184 to 5696M 2024-05-21T21:48:41.537 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:41 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:48:41.537 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:41 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:48:41.537 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:41 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:48:41.537 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:41 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:48:41.996 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi151:vg_nvme/lv_1 2024-05-21T21:48:42.202 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:42 smithi151 ceph-mon[31623]: osdmap e56: 9 total, 9 up, 9 in 2024-05-21T21:48:42.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:42 smithi151 ceph-mon[31623]: pgmap v212: 1 pgs: 1 remapped+peering; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-05-21T21:48:42.430 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:48:42.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:42 smithi184 ceph-mon[36942]: osdmap e56: 9 total, 9 up, 9 in 2024-05-21T21:48:42.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:42 smithi184 ceph-mon[36942]: pgmap v212: 1 pgs: 1 remapped+peering; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-05-21T21:48:42.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:42 smithi164 ceph-mon[36269]: osdmap e56: 9 total, 9 up, 9 in 2024-05-21T21:48:42.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:42 smithi164 ceph-mon[36269]: pgmap v212: 1 pgs: 1 remapped+peering; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-05-21T21:48:43.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:43 smithi151 ceph-mon[31623]: osdmap e57: 9 total, 9 up, 9 in 2024-05-21T21:48:43.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:43 smithi184 ceph-mon[36942]: osdmap e57: 9 total, 9 up, 9 in 2024-05-21T21:48:43.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:43 smithi164 ceph-mon[36269]: osdmap e57: 9 total, 9 up, 9 in 2024-05-21T21:48:44.387 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:44 smithi151 ceph-mon[31623]: pgmap v214: 1 pgs: 1 remapped+peering; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-05-21T21:48:44.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:44 smithi184 ceph-mon[36942]: pgmap v214: 1 pgs: 1 remapped+peering; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-05-21T21:48:44.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:44 smithi164 ceph-mon[36269]: pgmap v214: 1 pgs: 1 remapped+peering; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-05-21T21:48:44.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:48:44.562+0000 7f71db200700 1 -- 172.21.15.151:0/1426561248 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f71d41068e0 msgr2=0x7f71d4106d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:48:44.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:48:44.562+0000 7f71db200700 1 --2- 172.21.15.151:0/1426561248 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f71d41068e0 0x7f71d4106d40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f71c40099c0 tx=0x7f71c4009cd0 comp rx=0 tx=0).stop 2024-05-21T21:48:44.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:48:44.562+0000 7f71db200700 1 -- 172.21.15.151:0/1426561248 shutdown_connections 2024-05-21T21:48:44.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:48:44.562+0000 7f71db200700 1 --2- 172.21.15.151:0/1426561248 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71d4107280 0x7f71d410db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:48:44.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:48:44.562+0000 7f71db200700 1 --2- 172.21.15.151:0/1426561248 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f71d41056e0 0x7f71d4105b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:48:44.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:48:44.562+0000 7f71db200700 1 --2- 172.21.15.151:0/1426561248 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f71d41068e0 0x7f71d4106d40 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:48:44.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:48:44.562+0000 7f71db200700 1 -- 172.21.15.151:0/1426561248 >> 172.21.15.151:0/1426561248 conn(0x7f71d4100c80 msgr2=0x7f71d41030c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:48:44.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:48:44.562+0000 7f71db200700 1 -- 172.21.15.151:0/1426561248 shutdown_connections 2024-05-21T21:48:44.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:48:44.562+0000 7f71db200700 1 -- 172.21.15.151:0/1426561248 wait complete. 2024-05-21T21:48:44.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:48:44.563+0000 7f71db200700 1 Processor -- start 2024-05-21T21:48:44.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:48:44.563+0000 7f71db200700 1 -- start start 2024-05-21T21:48:44.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:48:44.564+0000 7f71db200700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71d41056e0 0x7f71d419b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:48:44.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:48:44.564+0000 7f71db200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f71d41068e0 0x7f71d419bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:48:44.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:48:44.564+0000 7f71db200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f71d4107280 0x7f71d41a2db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:48:44.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:48:44.564+0000 7f71db200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f71d41111d0 con 0x7f71d4107280 2024-05-21T21:48:44.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:48:44.564+0000 7f71db200700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f71d4111050 con 0x7f71d41068e0 2024-05-21T21:48:44.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:48:44.564+0000 7f71db200700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f71d4111350 con 0x7f71d41056e0 2024-05-21T21:48:44.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:48:44.564+0000 7f71d9e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71d41056e0 0x7f71d419b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:48:44.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:48:44.564+0000 7f71d9e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71d41056e0 0x7f71d419b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:41264/0 (socket says 172.21.15.151:41264) 2024-05-21T21:48:44.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:48:44.564+0000 7f71d9e00700 1 -- 172.21.15.151:0/3967506525 learned_addr learned my addr 172.21.15.151:0/3967506525 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:48:44.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:48:44.564+0000 7f71da800700 1 --2- 172.21.15.151:0/3967506525 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f71d4107280 0x7f71d41a2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:48:44.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:48:44.564+0000 7f71d9400700 1 --2- 172.21.15.151:0/3967506525 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f71d41068e0 0x7f71d419bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:48:44.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:48:44.565+0000 7f71da800700 1 -- 172.21.15.151:0/3967506525 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71d41056e0 msgr2=0x7f71d419b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:48:44.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:48:44.565+0000 7f71da800700 1 --2- 172.21.15.151:0/3967506525 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71d41056e0 0x7f71d419b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:48:44.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:48:44.565+0000 7f71da800700 1 -- 172.21.15.151:0/3967506525 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f71d41068e0 msgr2=0x7f71d419bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:48:44.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:48:44.565+0000 7f71da800700 1 --2- 172.21.15.151:0/3967506525 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f71d41068e0 0x7f71d419bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:48:44.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:48:44.565+0000 7f71da800700 1 -- 172.21.15.151:0/3967506525 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71d41a3450 con 0x7f71d4107280 2024-05-21T21:48:44.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:48:44.565+0000 7f71d9e00700 1 --2- 172.21.15.151:0/3967506525 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71d41056e0 0x7f71d419b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:48:44.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:48:44.565+0000 7f71da800700 1 --2- 172.21.15.151:0/3967506525 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f71d4107280 0x7f71d41a2db0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f71d0009f60 tx=0x7f71d0009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:48:44.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:48:44.565+0000 7f71cea00700 1 -- 172.21.15.151:0/3967506525 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d00092e0 con 0x7f71d4107280 2024-05-21T21:48:44.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:48:44.565+0000 7f71db200700 1 -- 172.21.15.151:0/3967506525 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71d41a36e0 con 0x7f71d4107280 2024-05-21T21:48:44.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:48:44.565+0000 7f71cea00700 1 -- 172.21.15.151:0/3967506525 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71d0004bb0 con 0x7f71d4107280 2024-05-21T21:48:44.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:48:44.565+0000 7f71cea00700 1 -- 172.21.15.151:0/3967506525 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d0004020 con 0x7f71d4107280 2024-05-21T21:48:44.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:48:44.566+0000 7f71db200700 1 -- 172.21.15.151:0/3967506525 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71d41a3c20 con 0x7f71d4107280 2024-05-21T21:48:44.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:48:44.567+0000 7f71db200700 1 -- 172.21.15.151:0/3967506525 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71d406a230 con 0x7f71d4107280 2024-05-21T21:48:44.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:48:44.569+0000 7f71cea00700 1 -- 172.21.15.151:0/3967506525 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f71d001b030 con 0x7f71d4107280 2024-05-21T21:48:44.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:48:44.571+0000 7f71cea00700 1 --2- 172.21.15.151:0/3967506525 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f71bc0b2900 0x7f71bc0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:48:44.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:48:44.571+0000 7f71cea00700 1 -- 172.21.15.151:0/3967506525 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5829+0+0 (secure 0 0 0) 0x7f71d00e1c80 con 0x7f71d4107280 2024-05-21T21:48:44.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:48:44.571+0000 7f71d9e00700 1 --2- 172.21.15.151:0/3967506525 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f71bc0b2900 0x7f71bc0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:48:44.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:48:44.572+0000 7f71d9e00700 1 --2- 172.21.15.151:0/3967506525 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f71bc0b2900 0x7f71bc0b4dc0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f71c8005950 tx=0x7f71c8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:48:44.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:48:44.574+0000 7f71cea00700 1 -- 172.21.15.151:0/3967506525 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f71d0022020 con 0x7f71d4107280 2024-05-21T21:48:44.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:48:44.815+0000 7f71db200700 1 -- 172.21.15.151:0/3967506525 --> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f71d4063980 con 0x7f71bc0b2900 2024-05-21T21:48:45.452 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:45 smithi151 ceph-mon[31623]: from='client.14301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:48:45.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:45 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T21:48:45.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:45 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T21:48:45.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:45 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:48:45.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:45 smithi184 ceph-mon[36942]: from='client.14301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:48:45.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:45 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T21:48:45.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:45 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T21:48:45.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:45 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:48:45.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:45 smithi164 ceph-mon[36269]: from='client.14301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:48:45.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:45 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T21:48:45.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:45 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T21:48:45.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:45 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:48:46.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:46 smithi151 ceph-mon[31623]: pgmap v215: 1 pgs: 1 remapped+peering; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-05-21T21:48:46.485 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:46 smithi184 ceph-mon[36942]: pgmap v215: 1 pgs: 1 remapped+peering; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-05-21T21:48:46.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:46 smithi164 ceph-mon[36269]: pgmap v215: 1 pgs: 1 remapped+peering; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-05-21T21:48:48.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:48 smithi151 ceph-mon[31623]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-05-21T21:48:48.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:48 smithi184 ceph-mon[36942]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-05-21T21:48:48.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:48 smithi164 ceph-mon[36269]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-05-21T21:48:49.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:49 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/2073546147' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3a7b450f-8004-4adf-bae3-970918f98f50"}]: dispatch 2024-05-21T21:48:49.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:49 smithi151 ceph-mon[31623]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3a7b450f-8004-4adf-bae3-970918f98f50"}]: dispatch 2024-05-21T21:48:49.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:49 smithi151 ceph-mon[31623]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3a7b450f-8004-4adf-bae3-970918f98f50"}]': finished 2024-05-21T21:48:49.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:49 smithi151 ceph-mon[31623]: osdmap e58: 10 total, 9 up, 10 in 2024-05-21T21:48:49.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:49 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-21T21:48:49.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:49 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/2073546147' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3a7b450f-8004-4adf-bae3-970918f98f50"}]: dispatch 2024-05-21T21:48:49.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:49 smithi184 ceph-mon[36942]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3a7b450f-8004-4adf-bae3-970918f98f50"}]: dispatch 2024-05-21T21:48:49.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:49 smithi184 ceph-mon[36942]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3a7b450f-8004-4adf-bae3-970918f98f50"}]': finished 2024-05-21T21:48:49.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:49 smithi184 ceph-mon[36942]: osdmap e58: 10 total, 9 up, 10 in 2024-05-21T21:48:49.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:49 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-21T21:48:49.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:49 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/2073546147' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3a7b450f-8004-4adf-bae3-970918f98f50"}]: dispatch 2024-05-21T21:48:49.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:49 smithi164 ceph-mon[36269]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3a7b450f-8004-4adf-bae3-970918f98f50"}]: dispatch 2024-05-21T21:48:49.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:49 smithi164 ceph-mon[36269]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3a7b450f-8004-4adf-bae3-970918f98f50"}]': finished 2024-05-21T21:48:49.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:49 smithi164 ceph-mon[36269]: osdmap e58: 10 total, 9 up, 10 in 2024-05-21T21:48:49.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:49 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-21T21:48:50.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:50 smithi184 ceph-mon[36942]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-21T21:48:50.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:50 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/2017960615' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T21:48:50.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:50 smithi164 ceph-mon[36269]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-21T21:48:50.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:50 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/2017960615' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T21:48:50.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:50 smithi151 ceph-mon[31623]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-21T21:48:50.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:50 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/2017960615' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T21:48:52.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:52 smithi184 ceph-mon[36942]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-05-21T21:48:52.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:52 smithi164 ceph-mon[36269]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-05-21T21:48:52.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:52 smithi151 ceph-mon[31623]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-05-21T21:48:54.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:54 smithi184 ceph-mon[36942]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-21T21:48:54.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:54 smithi164 ceph-mon[36269]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-21T21:48:54.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:54 smithi151 ceph-mon[31623]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-21T21:48:56.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:56 smithi184 ceph-mon[36942]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-21T21:48:56.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:56 smithi164 ceph-mon[36269]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-21T21:48:56.623 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:56 smithi151 ceph-mon[31623]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-21T21:48:58.485 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:58 smithi184 ceph-mon[36942]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-05-21T21:48:58.537 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:58 smithi164 ceph-mon[36269]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-05-21T21:48:58.633 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:58 smithi151 ceph-mon[31623]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-05-21T21:48:59.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:59 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-21T21:48:59.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:59 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:48:59.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:48:59 smithi164 ceph-mon[36269]: Deploying daemon osd.9 on smithi151 2024-05-21T21:48:59.673 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:59 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-21T21:48:59.673 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:59 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:48:59.673 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:48:59 smithi151 ceph-mon[31623]: Deploying daemon osd.9 on smithi151 2024-05-21T21:48:59.735 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:59 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-21T21:48:59.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:59 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:48:59.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:48:59 smithi184 ceph-mon[36942]: Deploying daemon osd.9 on smithi151 2024-05-21T21:49:00.354 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:00 smithi151 ceph-mon[31623]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-05-21T21:49:00.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:00 smithi164 ceph-mon[36269]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-05-21T21:49:00.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:00 smithi184 ceph-mon[36942]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-05-21T21:49:02.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:02 smithi164 ceph-mon[36269]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-05-21T21:49:02.700 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:02 smithi151 ceph-mon[31623]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-05-21T21:49:02.735 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:02 smithi184 ceph-mon[36942]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-05-21T21:49:04.450 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:04 smithi151 ceph-mon[31623]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-05-21T21:49:04.533 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:04 smithi164 ceph-mon[36269]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-05-21T21:49:04.735 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:04 smithi184 ceph-mon[36942]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-05-21T21:49:06.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:06 smithi151 ceph-mon[31623]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-05-21T21:49:06.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:06 smithi164 ceph-mon[36269]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-05-21T21:49:06.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:06 smithi184 ceph-mon[36942]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-05-21T21:49:08.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:07 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:49:08.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:07 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:49:08.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:07 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:49:08.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:07 smithi151 ceph-mon[31623]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-05-21T21:49:08.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:07 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:49:08.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:07 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:49:08.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:07 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:49:08.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:07 smithi184 ceph-mon[36942]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-05-21T21:49:08.285 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:07 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:49:08.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:07 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:49:08.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:07 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:49:08.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:07 smithi164 ceph-mon[36269]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-05-21T21:49:10.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:10 smithi151 ceph-mon[31623]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-05-21T21:49:10.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:10 smithi184 ceph-mon[36942]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-05-21T21:49:10.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:10 smithi164 ceph-mon[36269]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-05-21T21:49:11.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:49:11.759+0000 7f71cea00700 1 -- 172.21.15.151:0/3967506525 <== mgr.14152 v2:172.21.15.151:6800/2796622483 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f71d4063980 con 0x7f71bc0b2900 2024-05-21T21:49:11.759 INFO:teuthology.orchestra.run.smithi151.stdout:Created osd(s) 9 on host 'smithi151' 2024-05-21T21:49:11.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:49:11.762+0000 7f71db200700 1 -- 172.21.15.151:0/3967506525 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f71bc0b2900 msgr2=0x7f71bc0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:49:11.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:49:11.762+0000 7f71db200700 1 --2- 172.21.15.151:0/3967506525 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f71bc0b2900 0x7f71bc0b4dc0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f71c8005950 tx=0x7f71c8009260 comp rx=0 tx=0).stop 2024-05-21T21:49:11.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:49:11.762+0000 7f71db200700 1 -- 172.21.15.151:0/3967506525 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f71d4107280 msgr2=0x7f71d41a2db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:49:11.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:49:11.762+0000 7f71db200700 1 --2- 172.21.15.151:0/3967506525 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f71d4107280 0x7f71d41a2db0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f71d0009f60 tx=0x7f71d0009f90 comp rx=0 tx=0).stop 2024-05-21T21:49:11.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:49:11.762+0000 7f71db200700 1 -- 172.21.15.151:0/3967506525 shutdown_connections 2024-05-21T21:49:11.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:49:11.762+0000 7f71db200700 1 --2- 172.21.15.151:0/3967506525 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f71bc0b2900 0x7f71bc0b4dc0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:49:11.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:49:11.762+0000 7f71db200700 1 --2- 172.21.15.151:0/3967506525 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f71d4107280 0x7f71d41a2db0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:49:11.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:49:11.762+0000 7f71db200700 1 --2- 172.21.15.151:0/3967506525 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71d41056e0 0x7f71d419b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:49:11.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:49:11.762+0000 7f71db200700 1 --2- 172.21.15.151:0/3967506525 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f71d41068e0 0x7f71d419bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:49:11.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:49:11.763+0000 7f71db200700 1 -- 172.21.15.151:0/3967506525 >> 172.21.15.151:0/3967506525 conn(0x7f71d4100c80 msgr2=0x7f71d4109be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:49:11.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:49:11.765+0000 7f71db200700 1 -- 172.21.15.151:0/3967506525 shutdown_connections 2024-05-21T21:49:11.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:49:11.765+0000 7f71db200700 1 -- 172.21.15.151:0/3967506525 wait complete. 2024-05-21T21:49:11.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:11 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:49:11.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:11 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:49:11.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:11 smithi151 ceph-mon[31623]: from='osd.9 [v2:172.21.15.151:6826/337128533,v1:172.21.15.151:6827/337128533]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-21T21:49:11.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:11 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:49:11.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:11 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:49:12.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:11 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:49:12.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:11 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:49:12.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:11 smithi164 ceph-mon[36269]: from='osd.9 [v2:172.21.15.151:6826/337128533,v1:172.21.15.151:6827/337128533]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-21T21:49:12.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:11 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:49:12.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:11 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:49:12.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:11 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:49:12.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:11 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:49:12.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:11 smithi184 ceph-mon[36942]: from='osd.9 [v2:172.21.15.151:6826/337128533,v1:172.21.15.151:6827/337128533]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-21T21:49:12.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:11 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:49:12.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:11 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:49:12.593 DEBUG:teuthology.orchestra.run.smithi151:osd.9> sudo journalctl -f -n 0 -u ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@osd.9.service 2024-05-21T21:49:12.600 INFO:tasks.cephadm:Deploying osd.10 on smithi164 with /dev/vg_nvme/lv_1... 2024-05-21T21:49:12.600 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-21T21:49:12.784 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:12 smithi151 ceph-mon[31623]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-05-21T21:49:12.784 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:12 smithi151 ceph-mon[31623]: from='osd.9 [v2:172.21.15.151:6826/337128533,v1:172.21.15.151:6827/337128533]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-21T21:49:12.784 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:12 smithi151 ceph-mon[31623]: osdmap e59: 10 total, 9 up, 10 in 2024-05-21T21:49:12.784 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:12 smithi151 ceph-mon[31623]: from='osd.9 [v2:172.21.15.151:6826/337128533,v1:172.21.15.151:6827/337128533]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-05-21T21:49:12.784 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:12 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-21T21:49:12.848 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.b/config 2024-05-21T21:49:12.971 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:12 smithi164 ceph-mon[36269]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-05-21T21:49:12.971 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:12 smithi164 ceph-mon[36269]: from='osd.9 [v2:172.21.15.151:6826/337128533,v1:172.21.15.151:6827/337128533]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-21T21:49:12.971 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:12 smithi164 ceph-mon[36269]: osdmap e59: 10 total, 9 up, 10 in 2024-05-21T21:49:12.971 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:12 smithi164 ceph-mon[36269]: from='osd.9 [v2:172.21.15.151:6826/337128533,v1:172.21.15.151:6827/337128533]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-05-21T21:49:12.971 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:12 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-21T21:49:13.098 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 21:49:12 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T21:49:12.783+0000 7f5971c00700 -1 osd.9 0 waiting for initial osdmap 2024-05-21T21:49:13.099 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 21:49:12 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T21:49:12.801+0000 7f596ca00700 -1 osd.9 60 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-21T21:49:13.235 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:12 smithi184 ceph-mon[36942]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-05-21T21:49:13.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:12 smithi184 ceph-mon[36942]: from='osd.9 [v2:172.21.15.151:6826/337128533,v1:172.21.15.151:6827/337128533]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-21T21:49:13.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:12 smithi184 ceph-mon[36942]: osdmap e59: 10 total, 9 up, 10 in 2024-05-21T21:49:13.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:12 smithi184 ceph-mon[36942]: from='osd.9 [v2:172.21.15.151:6826/337128533,v1:172.21.15.151:6827/337128533]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-05-21T21:49:13.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:12 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-21T21:49:13.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:13 smithi151 ceph-mon[31623]: from='osd.9 [v2:172.21.15.151:6826/337128533,v1:172.21.15.151:6827/337128533]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-05-21T21:49:13.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:13 smithi151 ceph-mon[31623]: osdmap e60: 10 total, 9 up, 10 in 2024-05-21T21:49:13.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:13 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-21T21:49:13.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:13 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-21T21:49:14.026 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:13 smithi164 ceph-mon[36269]: from='osd.9 [v2:172.21.15.151:6826/337128533,v1:172.21.15.151:6827/337128533]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-05-21T21:49:14.027 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:13 smithi164 ceph-mon[36269]: osdmap e60: 10 total, 9 up, 10 in 2024-05-21T21:49:14.027 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:13 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-21T21:49:14.027 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:13 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-21T21:49:14.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:13 smithi184 ceph-mon[36942]: from='osd.9 [v2:172.21.15.151:6826/337128533,v1:172.21.15.151:6827/337128533]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-05-21T21:49:14.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:13 smithi184 ceph-mon[36942]: osdmap e60: 10 total, 9 up, 10 in 2024-05-21T21:49:14.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:13 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-21T21:49:14.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:13 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-21T21:49:14.929 INFO:teuthology.orchestra.run.smithi164.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-21T21:49:14.929 INFO:teuthology.orchestra.run.smithi164.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-21T21:49:14.929 INFO:teuthology.orchestra.run.smithi164.stderr: stderr: 10+0 records in 2024-05-21T21:49:14.929 INFO:teuthology.orchestra.run.smithi164.stderr:10+0 records out 2024-05-21T21:49:14.929 INFO:teuthology.orchestra.run.smithi164.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0117867 s, 890 MB/s 2024-05-21T21:49:14.929 INFO:teuthology.orchestra.run.smithi164.stderr:--> Zapping successful for: 2024-05-21T21:49:14.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:14 smithi151 ceph-mon[31623]: purged_snaps scrub starts 2024-05-21T21:49:14.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:14 smithi151 ceph-mon[31623]: purged_snaps scrub ok 2024-05-21T21:49:14.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:14 smithi151 ceph-mon[31623]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-05-21T21:49:14.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:14 smithi151 ceph-mon[31623]: osd.9 [v2:172.21.15.151:6826/337128533,v1:172.21.15.151:6827/337128533] boot 2024-05-21T21:49:14.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:14 smithi151 ceph-mon[31623]: osdmap e61: 10 total, 10 up, 10 in 2024-05-21T21:49:14.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:14 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-21T21:49:15.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:14 smithi164 ceph-mon[36269]: purged_snaps scrub starts 2024-05-21T21:49:15.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:14 smithi164 ceph-mon[36269]: purged_snaps scrub ok 2024-05-21T21:49:15.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:14 smithi164 ceph-mon[36269]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-05-21T21:49:15.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:14 smithi164 ceph-mon[36269]: osd.9 [v2:172.21.15.151:6826/337128533,v1:172.21.15.151:6827/337128533] boot 2024-05-21T21:49:15.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:14 smithi164 ceph-mon[36269]: osdmap e61: 10 total, 10 up, 10 in 2024-05-21T21:49:15.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:14 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-21T21:49:15.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:14 smithi184 ceph-mon[36942]: purged_snaps scrub starts 2024-05-21T21:49:15.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:14 smithi184 ceph-mon[36942]: purged_snaps scrub ok 2024-05-21T21:49:15.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:14 smithi184 ceph-mon[36942]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-05-21T21:49:15.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:14 smithi184 ceph-mon[36942]: osd.9 [v2:172.21.15.151:6826/337128533,v1:172.21.15.151:6827/337128533] boot 2024-05-21T21:49:15.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:14 smithi184 ceph-mon[36942]: osdmap e61: 10 total, 10 up, 10 in 2024-05-21T21:49:15.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:14 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-21T21:49:15.541 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi164:vg_nvme/lv_1 2024-05-21T21:49:15.786 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.b/config 2024-05-21T21:49:16.052 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:15 smithi164 ceph-mon[36269]: osdmap e62: 10 total, 10 up, 10 in 2024-05-21T21:49:16.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:15 smithi151 ceph-mon[31623]: osdmap e62: 10 total, 10 up, 10 in 2024-05-21T21:49:16.235 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:15 smithi184 ceph-mon[36942]: osdmap e62: 10 total, 10 up, 10 in 2024-05-21T21:49:17.032 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:16 smithi164 ceph-mon[36269]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-21T21:49:17.155 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:16 smithi151 ceph-mon[31623]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-21T21:49:17.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:49:17.235+0000 7f7c7db5e700 1 -- 172.21.15.164:0/752351023 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7c781036e0 msgr2=0x7f7c78103b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:49:17.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:49:17.235+0000 7f7c7db5e700 1 --2- 172.21.15.164:0/752351023 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7c781036e0 0x7f7c78103b40 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c0099c0 tx=0x7f7c6c009cd0 comp rx=0 tx=0).stop 2024-05-21T21:49:17.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:49:17.235+0000 7f7c7db5e700 1 -- 172.21.15.164:0/752351023 shutdown_connections 2024-05-21T21:49:17.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:49:17.235+0000 7f7c7db5e700 1 --2- 172.21.15.164:0/752351023 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7c78104080 0x7f7c7810a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:49:17.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:49:17.235+0000 7f7c7db5e700 1 --2- 172.21.15.164:0/752351023 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7c781024e0 0x7f7c78102900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:49:17.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:49:17.235+0000 7f7c7db5e700 1 --2- 172.21.15.164:0/752351023 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7c781036e0 0x7f7c78103b40 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:49:17.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:49:17.235+0000 7f7c7db5e700 1 -- 172.21.15.164:0/752351023 >> 172.21.15.164:0/752351023 conn(0x7f7c780fdaa0 msgr2=0x7f7c780ffec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:49:17.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:49:17.235+0000 7f7c7db5e700 1 -- 172.21.15.164:0/752351023 shutdown_connections 2024-05-21T21:49:17.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:49:17.235+0000 7f7c7db5e700 1 -- 172.21.15.164:0/752351023 wait complete. 2024-05-21T21:49:17.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:49:17.235+0000 7f7c7db5e700 1 Processor -- start 2024-05-21T21:49:17.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:49:17.236+0000 7f7c7db5e700 1 -- start start 2024-05-21T21:49:17.237 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:16 smithi184 ceph-mon[36942]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-21T21:49:17.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:49:17.236+0000 7f7c7db5e700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7c781024e0 0x7f7c7806c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:49:17.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:49:17.236+0000 7f7c7db5e700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7c781036e0 0x7f7c7819e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:49:17.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:49:17.236+0000 7f7c7db5e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7c78104080 0x7f7c781a0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:49:17.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:49:17.236+0000 7f7c7db5e700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7c7810e070 con 0x7f7c781024e0 2024-05-21T21:49:17.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:49:17.236+0000 7f7c7db5e700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7c7810def0 con 0x7f7c781036e0 2024-05-21T21:49:17.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:49:17.236+0000 7f7c7db5e700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7c7810e1f0 con 0x7f7c78104080 2024-05-21T21:49:17.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:49:17.236+0000 7f7c76ffd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7c781036e0 0x7f7c7819e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:49:17.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:49:17.236+0000 7f7c76ffd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7c781036e0 0x7f7c7819e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:39270/0 (socket says 172.21.15.164:39270) 2024-05-21T21:49:17.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:49:17.236+0000 7f7c76ffd700 1 -- 172.21.15.164:0/1632639510 learned_addr learned my addr 172.21.15.164:0/1632639510 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-05-21T21:49:17.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:49:17.236+0000 7f7c77fff700 1 --2- 172.21.15.164:0/1632639510 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7c78104080 0x7f7c781a0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:49:17.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:49:17.236+0000 7f7c777fe700 1 --2- 172.21.15.164:0/1632639510 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7c781024e0 0x7f7c7806c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:49:17.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:49:17.236+0000 7f7c76ffd700 1 -- 172.21.15.164:0/1632639510 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7c78104080 msgr2=0x7f7c781a0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:49:17.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:49:17.236+0000 7f7c76ffd700 1 --2- 172.21.15.164:0/1632639510 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7c78104080 0x7f7c781a0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:49:17.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:49:17.236+0000 7f7c76ffd700 1 -- 172.21.15.164:0/1632639510 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7c781024e0 msgr2=0x7f7c7806c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:49:17.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:49:17.236+0000 7f7c76ffd700 1 --2- 172.21.15.164:0/1632639510 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7c781024e0 0x7f7c7806c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:49:17.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:49:17.236+0000 7f7c76ffd700 1 -- 172.21.15.164:0/1632639510 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c781a13f0 con 0x7f7c781036e0 2024-05-21T21:49:17.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:49:17.236+0000 7f7c777fe700 1 --2- 172.21.15.164:0/1632639510 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7c781024e0 0x7f7c7806c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T21:49:17.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:49:17.236+0000 7f7c76ffd700 1 --2- 172.21.15.164:0/1632639510 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7c781036e0 0x7f7c7819e4d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c003790 tx=0x7f7c6c005ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:49:17.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:49:17.237+0000 7f7c74ff9700 1 -- 172.21.15.164:0/1632639510 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c6c027070 con 0x7f7c781036e0 2024-05-21T21:49:17.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:49:17.237+0000 7f7c74ff9700 1 -- 172.21.15.164:0/1632639510 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c6c022470 con 0x7f7c781036e0 2024-05-21T21:49:17.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:49:17.237+0000 7f7c7db5e700 1 -- 172.21.15.164:0/1632639510 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c781a1680 con 0x7f7c781036e0 2024-05-21T21:49:17.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:49:17.237+0000 7f7c74ff9700 1 -- 172.21.15.164:0/1632639510 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c6c00f6e0 con 0x7f7c781036e0 2024-05-21T21:49:17.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:49:17.237+0000 7f7c7db5e700 1 -- 172.21.15.164:0/1632639510 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c781a1c30 con 0x7f7c781036e0 2024-05-21T21:49:17.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:49:17.238+0000 7f7c7db5e700 1 -- 172.21.15.164:0/1632639510 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c7806a230 con 0x7f7c781036e0 2024-05-21T21:49:17.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:49:17.239+0000 7f7c74ff9700 1 -- 172.21.15.164:0/1632639510 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f7c6c02a070 con 0x7f7c781036e0 2024-05-21T21:49:17.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:49:17.239+0000 7f7c74ff9700 1 --2- 172.21.15.164:0/1632639510 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f7c600b2900 0x7f7c600b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:49:17.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:49:17.240+0000 7f7c74ff9700 1 -- 172.21.15.164:0/1632639510 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 6261+0+0 (secure 0 0 0) 0x7f7c6c0e27e0 con 0x7f7c781036e0 2024-05-21T21:49:17.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:49:17.240+0000 7f7c777fe700 1 --2- 172.21.15.164:0/1632639510 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f7c600b2900 0x7f7c600b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:49:17.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:49:17.241+0000 7f7c777fe700 1 --2- 172.21.15.164:0/1632639510 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f7c600b2900 0x7f7c600b4dc0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7c78103540 tx=0x7f7c680073c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:49:17.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:49:17.244+0000 7f7c74ff9700 1 -- 172.21.15.164:0/1632639510 <== mon.2 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7c6c013070 con 0x7f7c781036e0 2024-05-21T21:49:17.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:49:17.462+0000 7f7c7db5e700 1 -- 172.21.15.164:0/1632639510 --> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f7c78063980 con 0x7f7c600b2900 2024-05-21T21:49:18.363 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:18 smithi164 ceph-mon[36269]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-21T21:49:18.363 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:18 smithi164 ceph-mon[36269]: Detected new or changed devices on smithi151 2024-05-21T21:49:18.364 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:18 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:49:18.364 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:18 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:49:18.364 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:18 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:49:18.364 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:18 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:49:18.364 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:18 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:49:18.364 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:18 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:49:18.364 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:18 smithi164 ceph-mon[36269]: Adjusting osd_memory_target on smithi151 to 4272M 2024-05-21T21:49:18.364 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:18 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:49:18.364 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:18 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:49:18.364 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:18 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:49:18.364 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:18 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:49:18.364 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:18 smithi164 ceph-mon[36269]: from='client.24371 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:49:18.364 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:18 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T21:49:18.364 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:18 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T21:49:18.365 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:18 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:49:18.365 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:18 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:49:18.365 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:18 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:49:18.365 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:18 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:49:18.365 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:18 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:49:18.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:18 smithi151 ceph-mon[31623]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-21T21:49:18.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:18 smithi151 ceph-mon[31623]: Detected new or changed devices on smithi151 2024-05-21T21:49:18.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:18 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:49:18.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:18 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:49:18.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:18 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:49:18.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:18 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:49:18.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:18 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:49:18.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:18 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:49:18.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:18 smithi151 ceph-mon[31623]: Adjusting osd_memory_target on smithi151 to 4272M 2024-05-21T21:49:18.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:18 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:49:18.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:18 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:49:18.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:18 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:49:18.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:18 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:49:18.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:18 smithi151 ceph-mon[31623]: from='client.24371 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:49:18.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:18 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T21:49:18.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:18 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T21:49:18.705 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:18 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:49:18.705 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:18 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:49:18.705 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:18 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:49:18.705 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:18 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:49:18.705 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:18 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:49:18.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:18 smithi184 ceph-mon[36942]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-21T21:49:18.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:18 smithi184 ceph-mon[36942]: Detected new or changed devices on smithi151 2024-05-21T21:49:18.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:18 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:49:18.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:18 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:49:18.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:18 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:49:18.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:18 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:49:18.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:18 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:49:18.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:18 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:49:18.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:18 smithi184 ceph-mon[36942]: Adjusting osd_memory_target on smithi151 to 4272M 2024-05-21T21:49:18.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:18 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:49:18.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:18 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:49:18.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:18 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:49:18.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:18 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:49:18.737 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:18 smithi184 ceph-mon[36942]: from='client.24371 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi164:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:49:18.737 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:18 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T21:49:18.737 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:18 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T21:49:18.737 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:18 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:49:18.737 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:18 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:49:18.737 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:18 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:49:18.737 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:18 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:49:18.737 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:18 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:49:20.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:20 smithi164 ceph-mon[36269]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-21T21:49:20.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:20 smithi164 ceph-mon[36269]: from='client.? 172.21.15.164:0/544050932' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8fcdf271-90c4-4db3-be69-b0ca716dfd80"}]: dispatch 2024-05-21T21:49:20.537 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:20 smithi164 ceph-mon[36269]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8fcdf271-90c4-4db3-be69-b0ca716dfd80"}]: dispatch 2024-05-21T21:49:20.537 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:20 smithi164 ceph-mon[36269]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8fcdf271-90c4-4db3-be69-b0ca716dfd80"}]': finished 2024-05-21T21:49:20.537 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:20 smithi164 ceph-mon[36269]: osdmap e63: 11 total, 10 up, 11 in 2024-05-21T21:49:20.537 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:20 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-21T21:49:20.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:20 smithi151 ceph-mon[31623]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-21T21:49:20.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:20 smithi151 ceph-mon[31623]: from='client.? 172.21.15.164:0/544050932' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8fcdf271-90c4-4db3-be69-b0ca716dfd80"}]: dispatch 2024-05-21T21:49:20.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:20 smithi151 ceph-mon[31623]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8fcdf271-90c4-4db3-be69-b0ca716dfd80"}]: dispatch 2024-05-21T21:49:20.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:20 smithi151 ceph-mon[31623]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8fcdf271-90c4-4db3-be69-b0ca716dfd80"}]': finished 2024-05-21T21:49:20.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:20 smithi151 ceph-mon[31623]: osdmap e63: 11 total, 10 up, 11 in 2024-05-21T21:49:20.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:20 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-21T21:49:20.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:20 smithi184 ceph-mon[36942]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-21T21:49:20.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:20 smithi184 ceph-mon[36942]: from='client.? 172.21.15.164:0/544050932' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8fcdf271-90c4-4db3-be69-b0ca716dfd80"}]: dispatch 2024-05-21T21:49:20.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:20 smithi184 ceph-mon[36942]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8fcdf271-90c4-4db3-be69-b0ca716dfd80"}]: dispatch 2024-05-21T21:49:20.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:20 smithi184 ceph-mon[36942]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8fcdf271-90c4-4db3-be69-b0ca716dfd80"}]': finished 2024-05-21T21:49:20.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:20 smithi184 ceph-mon[36942]: osdmap e63: 11 total, 10 up, 11 in 2024-05-21T21:49:20.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:20 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-21T21:49:21.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:21 smithi151 ceph-mon[31623]: from='client.? 172.21.15.164:0/1037400046' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T21:49:21.735 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:21 smithi184 ceph-mon[36942]: from='client.? 172.21.15.164:0/1037400046' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T21:49:21.785 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:21 smithi164 ceph-mon[36269]: from='client.? 172.21.15.164:0/1037400046' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T21:49:22.637 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:22 smithi184 ceph-mon[36942]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-21T21:49:22.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:22 smithi151 ceph-mon[31623]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-21T21:49:22.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:22 smithi164 ceph-mon[36269]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-21T21:49:24.496 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:24 smithi164 ceph-mon[36269]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-21T21:49:24.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:24 smithi151 ceph-mon[31623]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-21T21:49:24.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:24 smithi184 ceph-mon[36942]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-21T21:49:26.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:26 smithi164 ceph-mon[36269]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-21T21:49:26.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:26 smithi151 ceph-mon[31623]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-21T21:49:26.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:26 smithi184 ceph-mon[36942]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-21T21:49:28.578 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:28 smithi164 ceph-mon[36269]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-21T21:49:28.578 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:28 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-21T21:49:28.578 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:28 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:49:28.578 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:28 smithi164 ceph-mon[36269]: Deploying daemon osd.10 on smithi164 2024-05-21T21:49:28.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:28 smithi151 ceph-mon[31623]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-21T21:49:28.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:28 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-21T21:49:28.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:28 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:49:28.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:28 smithi151 ceph-mon[31623]: Deploying daemon osd.10 on smithi164 2024-05-21T21:49:28.735 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:28 smithi184 ceph-mon[36942]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-21T21:49:28.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:28 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-21T21:49:28.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:28 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:49:28.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:28 smithi184 ceph-mon[36942]: Deploying daemon osd.10 on smithi164 2024-05-21T21:49:30.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:30 smithi151 ceph-mon[31623]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-21T21:49:30.735 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:30 smithi184 ceph-mon[36942]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-21T21:49:30.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:30 smithi164 ceph-mon[36269]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-21T21:49:32.690 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:32 smithi164 ceph-mon[36269]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-21T21:49:32.702 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:32 smithi151 ceph-mon[31623]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-21T21:49:32.735 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:32 smithi184 ceph-mon[36942]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-21T21:49:34.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:34 smithi151 ceph-mon[31623]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-21T21:49:34.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:34 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:49:34.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:34 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:49:34.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:34 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:49:34.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:34 smithi184 ceph-mon[36942]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-21T21:49:34.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:34 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:49:34.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:34 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:49:34.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:34 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:49:35.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:34 smithi164 ceph-mon[36269]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-21T21:49:35.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:34 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:49:35.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:34 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:49:35.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:34 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:49:36.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:36 smithi151 ceph-mon[31623]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-21T21:49:36.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:36 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:49:36.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:36 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:49:36.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:36 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:49:36.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:36 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:49:36.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:36 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:49:36.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:36 smithi184 ceph-mon[36942]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-21T21:49:36.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:36 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:49:36.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:36 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:49:36.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:36 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:49:36.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:36 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:49:36.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:36 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:49:37.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:36 smithi164 ceph-mon[36269]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-21T21:49:37.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:36 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:49:37.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:36 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:49:37.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:36 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:49:37.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:36 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:49:37.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:36 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:49:37.507 INFO:teuthology.orchestra.run.smithi164.stdout:Created osd(s) 10 on host 'smithi164' 2024-05-21T21:49:37.508 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:49:37.507+0000 7f7c74ff9700 1 -- 172.21.15.164:0/1632639510 <== mgr.14152 v2:172.21.15.151:6800/2796622483 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f7c78063980 con 0x7f7c600b2900 2024-05-21T21:49:37.510 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:49:37.509+0000 7f7c7db5e700 1 -- 172.21.15.164:0/1632639510 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f7c600b2900 msgr2=0x7f7c600b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:49:37.510 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:49:37.509+0000 7f7c7db5e700 1 --2- 172.21.15.164:0/1632639510 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f7c600b2900 0x7f7c600b4dc0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7c78103540 tx=0x7f7c680073c0 comp rx=0 tx=0).stop 2024-05-21T21:49:37.510 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:49:37.509+0000 7f7c7db5e700 1 -- 172.21.15.164:0/1632639510 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7c781036e0 msgr2=0x7f7c7819e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:49:37.510 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:49:37.510+0000 7f7c7db5e700 1 --2- 172.21.15.164:0/1632639510 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7c781036e0 0x7f7c7819e4d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c003790 tx=0x7f7c6c005ef0 comp rx=0 tx=0).stop 2024-05-21T21:49:37.510 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:49:37.510+0000 7f7c7db5e700 1 -- 172.21.15.164:0/1632639510 shutdown_connections 2024-05-21T21:49:37.510 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:49:37.510+0000 7f7c7db5e700 1 --2- 172.21.15.164:0/1632639510 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f7c600b2900 0x7f7c600b4dc0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:49:37.510 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:49:37.510+0000 7f7c7db5e700 1 --2- 172.21.15.164:0/1632639510 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7c78104080 0x7f7c781a0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:49:37.510 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:49:37.510+0000 7f7c7db5e700 1 --2- 172.21.15.164:0/1632639510 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7c781024e0 0x7f7c7806c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:49:37.511 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:49:37.510+0000 7f7c7db5e700 1 --2- 172.21.15.164:0/1632639510 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7c781036e0 0x7f7c7819e4d0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:49:37.511 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:49:37.510+0000 7f7c7db5e700 1 -- 172.21.15.164:0/1632639510 >> 172.21.15.164:0/1632639510 conn(0x7f7c780fdaa0 msgr2=0x7f7c781069e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:49:37.511 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:49:37.510+0000 7f7c7db5e700 1 -- 172.21.15.164:0/1632639510 shutdown_connections 2024-05-21T21:49:37.511 INFO:teuthology.orchestra.run.smithi164.stderr:2024-05-21T21:49:37.510+0000 7f7c7db5e700 1 -- 172.21.15.164:0/1632639510 wait complete. 2024-05-21T21:49:37.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:37 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:49:37.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:37 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:49:37.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:37 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:49:37.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:37 smithi164 ceph-mon[36269]: from='osd.10 [v2:172.21.15.164:6824/1823448782,v1:172.21.15.164:6825/1823448782]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-21T21:49:37.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:37 smithi164 ceph-mon[36269]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-21T21:49:37.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:37 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:49:37.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:37 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:49:37.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:37 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:49:37.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:37 smithi151 ceph-mon[31623]: from='osd.10 [v2:172.21.15.164:6824/1823448782,v1:172.21.15.164:6825/1823448782]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-21T21:49:37.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:37 smithi151 ceph-mon[31623]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-21T21:49:37.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:37 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:49:37.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:37 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:49:37.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:37 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:49:37.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:37 smithi184 ceph-mon[36942]: from='osd.10 [v2:172.21.15.164:6824/1823448782,v1:172.21.15.164:6825/1823448782]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-21T21:49:37.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:37 smithi184 ceph-mon[36942]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-21T21:49:38.046 DEBUG:teuthology.orchestra.run.smithi164:osd.10> sudo journalctl -f -n 0 -u ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@osd.10.service 2024-05-21T21:49:38.048 INFO:tasks.cephadm:Deploying osd.11 on smithi184 with /dev/vg_nvme/lv_1... 2024-05-21T21:49:38.048 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-21T21:49:38.294 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.c/config 2024-05-21T21:49:38.681 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:38 smithi184 ceph-mon[36942]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-21T21:49:38.681 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:38 smithi184 ceph-mon[36942]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-21T21:49:38.682 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:38 smithi184 ceph-mon[36942]: from='osd.10 [v2:172.21.15.164:6824/1823448782,v1:172.21.15.164:6825/1823448782]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-05-21T21:49:38.682 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:38 smithi184 ceph-mon[36942]: osdmap e64: 11 total, 10 up, 11 in 2024-05-21T21:49:38.682 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:38 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-21T21:49:38.682 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:38 smithi184 ceph-mon[36942]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-05-21T21:49:38.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:38 smithi151 ceph-mon[31623]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-21T21:49:38.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:38 smithi151 ceph-mon[31623]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-21T21:49:38.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:38 smithi151 ceph-mon[31623]: from='osd.10 [v2:172.21.15.164:6824/1823448782,v1:172.21.15.164:6825/1823448782]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-05-21T21:49:38.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:38 smithi151 ceph-mon[31623]: osdmap e64: 11 total, 10 up, 11 in 2024-05-21T21:49:38.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:38 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-21T21:49:38.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:38 smithi151 ceph-mon[31623]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-05-21T21:49:39.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:38 smithi164 ceph-mon[36269]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-21T21:49:39.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:38 smithi164 ceph-mon[36269]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-21T21:49:39.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:38 smithi164 ceph-mon[36269]: from='osd.10 [v2:172.21.15.164:6824/1823448782,v1:172.21.15.164:6825/1823448782]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-05-21T21:49:39.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:38 smithi164 ceph-mon[36269]: osdmap e64: 11 total, 10 up, 11 in 2024-05-21T21:49:39.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:38 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-21T21:49:39.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:38 smithi164 ceph-mon[36269]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]: dispatch 2024-05-21T21:49:39.786 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 21:49:39 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T21:49:39.488+0000 7f0b434fd700 -1 osd.10 0 waiting for initial osdmap 2024-05-21T21:49:39.787 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 21:49:39 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T21:49:39.500+0000 7f0b3f2f0700 -1 osd.10 65 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-21T21:49:40.182 INFO:teuthology.orchestra.run.smithi184.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-21T21:49:40.182 INFO:teuthology.orchestra.run.smithi184.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-21T21:49:40.182 INFO:teuthology.orchestra.run.smithi184.stderr: stderr: 10+0 records in 2024-05-21T21:49:40.182 INFO:teuthology.orchestra.run.smithi184.stderr:10+0 records out 2024-05-21T21:49:40.182 INFO:teuthology.orchestra.run.smithi184.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0120194 s, 872 MB/s 2024-05-21T21:49:40.182 INFO:teuthology.orchestra.run.smithi184.stderr:--> Zapping successful for: 2024-05-21T21:49:40.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:40 smithi184 ceph-mon[36942]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-21T21:49:40.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:40 smithi184 ceph-mon[36942]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-05-21T21:49:40.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:40 smithi184 ceph-mon[36942]: osdmap e65: 11 total, 10 up, 11 in 2024-05-21T21:49:40.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:40 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-21T21:49:40.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:40 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-21T21:49:40.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:40 smithi164 ceph-mon[36269]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-21T21:49:40.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:40 smithi164 ceph-mon[36269]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-05-21T21:49:40.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:40 smithi164 ceph-mon[36269]: osdmap e65: 11 total, 10 up, 11 in 2024-05-21T21:49:40.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:40 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-21T21:49:40.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:40 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-21T21:49:40.854 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph orch daemon add osd smithi184:vg_nvme/lv_1 2024-05-21T21:49:40.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:40 smithi151 ceph-mon[31623]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-21T21:49:40.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:40 smithi151 ceph-mon[31623]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi164", "root=default"]}]': finished 2024-05-21T21:49:40.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:40 smithi151 ceph-mon[31623]: osdmap e65: 11 total, 10 up, 11 in 2024-05-21T21:49:40.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:40 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-21T21:49:40.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:40 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-21T21:49:41.103 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.c/config 2024-05-21T21:49:41.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:41 smithi184 ceph-mon[36942]: purged_snaps scrub starts 2024-05-21T21:49:41.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:41 smithi184 ceph-mon[36942]: purged_snaps scrub ok 2024-05-21T21:49:41.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:41 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-21T21:49:41.737 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:41 smithi184 ceph-mon[36942]: osd.10 [v2:172.21.15.164:6824/1823448782,v1:172.21.15.164:6825/1823448782] boot 2024-05-21T21:49:41.737 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:41 smithi184 ceph-mon[36942]: osdmap e66: 11 total, 11 up, 11 in 2024-05-21T21:49:41.737 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:41 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-21T21:49:41.737 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:41 smithi184 ceph-mon[36942]: Detected new or changed devices on smithi164 2024-05-21T21:49:41.737 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:41 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:49:41.737 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:41 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:49:41.737 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:41 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:49:41.737 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:41 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:49:41.737 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:41 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:49:41.737 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:41 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:49:41.737 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:41 smithi184 ceph-mon[36942]: Adjusting osd_memory_target on smithi164 to 4272M 2024-05-21T21:49:41.737 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:41 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:49:41.737 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:41 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:49:41.737 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:41 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:49:41.738 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:41 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:49:41.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:41 smithi164 ceph-mon[36269]: purged_snaps scrub starts 2024-05-21T21:49:41.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:41 smithi164 ceph-mon[36269]: purged_snaps scrub ok 2024-05-21T21:49:41.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:41 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-21T21:49:41.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:41 smithi164 ceph-mon[36269]: osd.10 [v2:172.21.15.164:6824/1823448782,v1:172.21.15.164:6825/1823448782] boot 2024-05-21T21:49:41.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:41 smithi164 ceph-mon[36269]: osdmap e66: 11 total, 11 up, 11 in 2024-05-21T21:49:41.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:41 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-21T21:49:41.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:41 smithi164 ceph-mon[36269]: Detected new or changed devices on smithi164 2024-05-21T21:49:41.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:41 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:49:41.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:41 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:49:41.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:41 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:49:41.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:41 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:49:41.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:41 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:49:41.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:41 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:49:41.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:41 smithi164 ceph-mon[36269]: Adjusting osd_memory_target on smithi164 to 4272M 2024-05-21T21:49:41.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:41 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:49:41.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:41 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:49:41.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:41 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:49:41.788 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:41 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:49:41.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:41 smithi151 ceph-mon[31623]: purged_snaps scrub starts 2024-05-21T21:49:41.954 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:41 smithi151 ceph-mon[31623]: purged_snaps scrub ok 2024-05-21T21:49:41.954 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:41 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-21T21:49:41.954 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:41 smithi151 ceph-mon[31623]: osd.10 [v2:172.21.15.164:6824/1823448782,v1:172.21.15.164:6825/1823448782] boot 2024-05-21T21:49:41.954 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:41 smithi151 ceph-mon[31623]: osdmap e66: 11 total, 11 up, 11 in 2024-05-21T21:49:41.954 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:41 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-21T21:49:41.954 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:41 smithi151 ceph-mon[31623]: Detected new or changed devices on smithi164 2024-05-21T21:49:41.954 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:41 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:49:41.954 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:41 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:49:41.954 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:41 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:49:41.954 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:41 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:49:41.955 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:41 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:49:41.955 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:41 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:49:41.955 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:41 smithi151 ceph-mon[31623]: Adjusting osd_memory_target on smithi164 to 4272M 2024-05-21T21:49:41.955 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:41 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:49:41.955 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:41 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:49:41.955 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:41 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:49:41.955 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:41 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:49:42.630 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:49:42.629+0000 7fd40f78a700 1 -- 172.21.15.184:0/1110355875 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd408105160 msgr2=0x7fd40810ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:49:42.631 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:49:42.629+0000 7fd40f78a700 1 --2- 172.21.15.184:0/1110355875 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd408105160 0x7fd40810ba10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd40000b0a0 tx=0x7fd40000b3b0 comp rx=0 tx=0).stop 2024-05-21T21:49:42.631 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:49:42.629+0000 7fd40f78a700 1 -- 172.21.15.184:0/1110355875 shutdown_connections 2024-05-21T21:49:42.631 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:49:42.629+0000 7fd40f78a700 1 --2- 172.21.15.184:0/1110355875 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd408105160 0x7fd40810ba10 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:49:42.631 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:49:42.629+0000 7fd40f78a700 1 --2- 172.21.15.184:0/1110355875 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd4081035c0 0x7fd4081039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:49:42.631 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:49:42.629+0000 7fd40f78a700 1 --2- 172.21.15.184:0/1110355875 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4081047c0 0x7fd408104c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:49:42.631 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:49:42.629+0000 7fd40f78a700 1 -- 172.21.15.184:0/1110355875 >> 172.21.15.184:0/1110355875 conn(0x7fd4080feb60 msgr2=0x7fd408100fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:49:42.631 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:49:42.630+0000 7fd40f78a700 1 -- 172.21.15.184:0/1110355875 shutdown_connections 2024-05-21T21:49:42.631 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:49:42.630+0000 7fd40f78a700 1 -- 172.21.15.184:0/1110355875 wait complete. 2024-05-21T21:49:42.631 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:49:42.630+0000 7fd40f78a700 1 Processor -- start 2024-05-21T21:49:42.631 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:49:42.630+0000 7fd40f78a700 1 -- start start 2024-05-21T21:49:42.633 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:49:42.631+0000 7fd40f78a700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4081035c0 0x7fd408199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:49:42.633 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:49:42.631+0000 7fd40f78a700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4081047c0 0x7fd408199d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:49:42.633 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:49:42.631+0000 7fd40f78a700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd408105160 0x7fd4081a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:49:42.633 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:49:42.631+0000 7fd40f78a700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd40810f1a0 con 0x7fd408105160 2024-05-21T21:49:42.633 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:49:42.631+0000 7fd40f78a700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd40810f020 con 0x7fd4081035c0 2024-05-21T21:49:42.633 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:49:42.631+0000 7fd40f78a700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd40810f320 con 0x7fd4081047c0 2024-05-21T21:49:42.634 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:49:42.631+0000 7fd40cd25700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4081047c0 0x7fd408199d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:49:42.634 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:49:42.631+0000 7fd40cd25700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4081047c0 0x7fd408199d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:48000/0 (socket says 172.21.15.184:48000) 2024-05-21T21:49:42.634 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:49:42.631+0000 7fd40cd25700 1 -- 172.21.15.184:0/315446258 learned_addr learned my addr 172.21.15.184:0/315446258 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-05-21T21:49:42.634 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:49:42.631+0000 7fd40d526700 1 --2- 172.21.15.184:0/315446258 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4081035c0 0x7fd408199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:49:42.634 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:49:42.631+0000 7fd40cd25700 1 -- 172.21.15.184:0/315446258 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4081035c0 msgr2=0x7fd408199840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:49:42.634 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:49:42.631+0000 7fd40cd25700 1 --2- 172.21.15.184:0/315446258 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4081035c0 0x7fd408199840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:49:42.634 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:49:42.631+0000 7fd40cd25700 1 -- 172.21.15.184:0/315446258 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd408105160 msgr2=0x7fd4081a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:49:42.634 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:49:42.631+0000 7fd40cd25700 1 --2- 172.21.15.184:0/315446258 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd408105160 0x7fd4081a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:49:42.634 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:49:42.631+0000 7fd40cd25700 1 -- 172.21.15.184:0/315446258 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4081a14a0 con 0x7fd4081047c0 2024-05-21T21:49:42.634 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:49:42.631+0000 7fd40cd25700 1 --2- 172.21.15.184:0/315446258 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4081047c0 0x7fd408199d80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd3f800b560 tx=0x7fd3f800b870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:49:42.634 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:49:42.631+0000 7fd3fe7fc700 1 -- 172.21.15.184:0/315446258 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f8015020 con 0x7fd4081047c0 2024-05-21T21:49:42.635 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:49:42.631+0000 7fd3fe7fc700 1 -- 172.21.15.184:0/315446258 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3f8011ce0 con 0x7fd4081047c0 2024-05-21T21:49:42.635 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:49:42.631+0000 7fd40f78a700 1 -- 172.21.15.184:0/315446258 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4081a1790 con 0x7fd4081047c0 2024-05-21T21:49:42.635 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:49:42.631+0000 7fd3fe7fc700 1 -- 172.21.15.184:0/315446258 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f8010d20 con 0x7fd4081047c0 2024-05-21T21:49:42.635 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:49:42.631+0000 7fd40f78a700 1 -- 172.21.15.184:0/315446258 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4081a1d50 con 0x7fd4081047c0 2024-05-21T21:49:42.635 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:49:42.632+0000 7fd40f78a700 1 -- 172.21.15.184:0/315446258 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd40806a230 con 0x7fd4081047c0 2024-05-21T21:49:42.636 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:49:42.634+0000 7fd3fe7fc700 1 -- 172.21.15.184:0/315446258 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7fd3f8018020 con 0x7fd4081047c0 2024-05-21T21:49:42.636 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:49:42.634+0000 7fd3fe7fc700 1 --2- 172.21.15.184:0/315446258 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fd3f40b2900 0x7fd3f40b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:49:42.636 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:49:42.634+0000 7fd3fe7fc700 1 -- 172.21.15.184:0/315446258 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6694+0+0 (secure 0 0 0) 0x7fd3f80e12f0 con 0x7fd4081047c0 2024-05-21T21:49:42.636 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:49:42.635+0000 7fd40d526700 1 --2- 172.21.15.184:0/315446258 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fd3f40b2900 0x7fd3f40b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:49:42.636 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:49:42.636+0000 7fd40d526700 1 --2- 172.21.15.184:0/315446258 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fd3f40b2900 0x7fd3f40b4dc0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd408104620 tx=0x7fd404009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:49:42.639 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:49:42.639+0000 7fd3fe7fc700 1 -- 172.21.15.184:0/315446258 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd3f801f020 con 0x7fd4081047c0 2024-05-21T21:49:42.735 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:42 smithi184 ceph-mon[36942]: pgmap v252: 1 pgs: 1 remapped+peering; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-21T21:49:42.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:42 smithi184 ceph-mon[36942]: osdmap e67: 11 total, 11 up, 11 in 2024-05-21T21:49:42.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:42 smithi164 ceph-mon[36269]: pgmap v252: 1 pgs: 1 remapped+peering; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-21T21:49:42.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:42 smithi164 ceph-mon[36269]: osdmap e67: 11 total, 11 up, 11 in 2024-05-21T21:49:42.853 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:49:42.853+0000 7fd40f78a700 1 -- 172.21.15.184:0/315446258 --> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fd408063980 con 0x7fd3f40b2900 2024-05-21T21:49:42.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:42 smithi151 ceph-mon[31623]: pgmap v252: 1 pgs: 1 remapped+peering; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-05-21T21:49:42.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:42 smithi151 ceph-mon[31623]: osdmap e67: 11 total, 11 up, 11 in 2024-05-21T21:49:43.780 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:43 smithi184 ceph-mon[36942]: osdmap e68: 11 total, 11 up, 11 in 2024-05-21T21:49:43.781 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:43 smithi184 ceph-mon[36942]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:49:43.781 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:43 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T21:49:43.781 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:43 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T21:49:43.781 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:43 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:49:43.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:43 smithi164 ceph-mon[36269]: osdmap e68: 11 total, 11 up, 11 in 2024-05-21T21:49:43.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:43 smithi164 ceph-mon[36269]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:49:43.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:43 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T21:49:43.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:43 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T21:49:43.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:43 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:49:43.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:43 smithi151 ceph-mon[31623]: osdmap e68: 11 total, 11 up, 11 in 2024-05-21T21:49:43.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:43 smithi151 ceph-mon[31623]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:49:43.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:43 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-21T21:49:43.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:43 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-21T21:49:43.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:43 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:49:44.756 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:44 smithi184 ceph-mon[36942]: pgmap v255: 1 pgs: 1 remapped+peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-05-21T21:49:44.785 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:44 smithi164 ceph-mon[36269]: pgmap v255: 1 pgs: 1 remapped+peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-05-21T21:49:44.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:44 smithi151 ceph-mon[31623]: pgmap v255: 1 pgs: 1 remapped+peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-05-21T21:49:46.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:46 smithi164 ceph-mon[36269]: pgmap v256: 1 pgs: 1 remapped+peering; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-21T21:49:46.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:46 smithi164 ceph-mon[36269]: from='client.? 172.21.15.184:0/1251408335' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e661477b-0c09-409f-a2fe-6ebd34642a8c"}]: dispatch 2024-05-21T21:49:46.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:46 smithi164 ceph-mon[36269]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e661477b-0c09-409f-a2fe-6ebd34642a8c"}]: dispatch 2024-05-21T21:49:46.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:46 smithi164 ceph-mon[36269]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e661477b-0c09-409f-a2fe-6ebd34642a8c"}]': finished 2024-05-21T21:49:46.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:46 smithi164 ceph-mon[36269]: osdmap e69: 12 total, 11 up, 12 in 2024-05-21T21:49:46.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:46 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-21T21:49:46.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:46 smithi164 ceph-mon[36269]: from='client.? 172.21.15.184:0/519568016' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T21:49:46.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:46 smithi151 ceph-mon[31623]: pgmap v256: 1 pgs: 1 remapped+peering; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-21T21:49:46.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:46 smithi151 ceph-mon[31623]: from='client.? 172.21.15.184:0/1251408335' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e661477b-0c09-409f-a2fe-6ebd34642a8c"}]: dispatch 2024-05-21T21:49:46.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:46 smithi151 ceph-mon[31623]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e661477b-0c09-409f-a2fe-6ebd34642a8c"}]: dispatch 2024-05-21T21:49:46.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:46 smithi151 ceph-mon[31623]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e661477b-0c09-409f-a2fe-6ebd34642a8c"}]': finished 2024-05-21T21:49:46.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:46 smithi151 ceph-mon[31623]: osdmap e69: 12 total, 11 up, 12 in 2024-05-21T21:49:46.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:46 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-21T21:49:46.954 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:46 smithi151 ceph-mon[31623]: from='client.? 172.21.15.184:0/519568016' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T21:49:46.985 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:46 smithi184 ceph-mon[36942]: pgmap v256: 1 pgs: 1 remapped+peering; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-21T21:49:46.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:46 smithi184 ceph-mon[36942]: from='client.? 172.21.15.184:0/1251408335' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e661477b-0c09-409f-a2fe-6ebd34642a8c"}]: dispatch 2024-05-21T21:49:46.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:46 smithi184 ceph-mon[36942]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e661477b-0c09-409f-a2fe-6ebd34642a8c"}]: dispatch 2024-05-21T21:49:46.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:46 smithi184 ceph-mon[36942]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e661477b-0c09-409f-a2fe-6ebd34642a8c"}]': finished 2024-05-21T21:49:46.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:46 smithi184 ceph-mon[36942]: osdmap e69: 12 total, 11 up, 12 in 2024-05-21T21:49:46.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:46 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-21T21:49:46.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:46 smithi184 ceph-mon[36942]: from='client.? 172.21.15.184:0/519568016' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-21T21:49:48.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:48 smithi151 ceph-mon[31623]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-21T21:49:48.985 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:48 smithi184 ceph-mon[36942]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-21T21:49:49.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:48 smithi164 ceph-mon[36269]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-21T21:49:50.877 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:50 smithi184 ceph-mon[36942]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-05-21T21:49:50.933 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:50 smithi151 ceph-mon[31623]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-05-21T21:49:51.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:50 smithi164 ceph-mon[36269]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-05-21T21:49:52.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:52 smithi184 ceph-mon[36942]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-05-21T21:49:52.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:52 smithi151 ceph-mon[31623]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-05-21T21:49:53.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:52 smithi164 ceph-mon[36269]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-05-21T21:49:53.717 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:53 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-21T21:49:53.717 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:53 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:49:53.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:53 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-21T21:49:53.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:53 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:49:54.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:53 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-21T21:49:54.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:53 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:49:54.980 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:54 smithi184 ceph-mon[36942]: Deploying daemon osd.11 on smithi184 2024-05-21T21:49:54.980 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:54 smithi184 ceph-mon[36942]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-21T21:49:55.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:54 smithi164 ceph-mon[36269]: Deploying daemon osd.11 on smithi184 2024-05-21T21:49:55.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:54 smithi164 ceph-mon[36269]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-21T21:49:55.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:54 smithi151 ceph-mon[31623]: Deploying daemon osd.11 on smithi184 2024-05-21T21:49:55.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:54 smithi151 ceph-mon[31623]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-21T21:49:56.881 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:56 smithi184 ceph-mon[36942]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-21T21:49:57.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:56 smithi151 ceph-mon[31623]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-21T21:49:57.285 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:56 smithi164 ceph-mon[36269]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-21T21:49:59.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:58 smithi151 ceph-mon[31623]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-21T21:49:59.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:58 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:49:59.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:58 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:49:59.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:49:58 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:49:59.235 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:58 smithi184 ceph-mon[36942]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-21T21:49:59.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:58 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:49:59.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:58 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:49:59.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:49:58 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:49:59.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:58 smithi164 ceph-mon[36269]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-21T21:49:59.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:58 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:49:59.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:58 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:49:59.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:49:58 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:50:00.985 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:00 smithi184 ceph-mon[36942]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-21T21:50:00.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:00 smithi184 ceph-mon[36942]: overall HEALTH_OK 2024-05-21T21:50:01.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:00 smithi151 ceph-mon[31623]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-21T21:50:01.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:00 smithi151 ceph-mon[31623]: overall HEALTH_OK 2024-05-21T21:50:01.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:00 smithi164 ceph-mon[36269]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-21T21:50:01.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:00 smithi164 ceph-mon[36269]: overall HEALTH_OK 2024-05-21T21:50:02.395 INFO:teuthology.orchestra.run.smithi184.stdout:Created osd(s) 11 on host 'smithi184' 2024-05-21T21:50:02.395 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:50:02.394+0000 7fd3fe7fc700 1 -- 172.21.15.184:0/315446258 <== mgr.14152 v2:172.21.15.151:6800/2796622483 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fd408063980 con 0x7fd3f40b2900 2024-05-21T21:50:02.398 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:50:02.397+0000 7fd40f78a700 1 -- 172.21.15.184:0/315446258 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fd3f40b2900 msgr2=0x7fd3f40b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:50:02.398 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:50:02.397+0000 7fd40f78a700 1 --2- 172.21.15.184:0/315446258 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fd3f40b2900 0x7fd3f40b4dc0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd408104620 tx=0x7fd404009190 comp rx=0 tx=0).stop 2024-05-21T21:50:02.399 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:50:02.398+0000 7fd40f78a700 1 -- 172.21.15.184:0/315446258 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4081047c0 msgr2=0x7fd408199d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:50:02.399 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:50:02.398+0000 7fd40f78a700 1 --2- 172.21.15.184:0/315446258 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4081047c0 0x7fd408199d80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd3f800b560 tx=0x7fd3f800b870 comp rx=0 tx=0).stop 2024-05-21T21:50:02.399 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:50:02.398+0000 7fd40f78a700 1 -- 172.21.15.184:0/315446258 shutdown_connections 2024-05-21T21:50:02.399 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:50:02.398+0000 7fd40f78a700 1 --2- 172.21.15.184:0/315446258 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fd3f40b2900 0x7fd3f40b4dc0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:02.399 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:50:02.398+0000 7fd40f78a700 1 --2- 172.21.15.184:0/315446258 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4081035c0 0x7fd408199840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:02.399 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:50:02.398+0000 7fd40f78a700 1 --2- 172.21.15.184:0/315446258 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd408105160 0x7fd4081a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:02.399 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:50:02.398+0000 7fd40f78a700 1 --2- 172.21.15.184:0/315446258 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4081047c0 0x7fd408199d80 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:02.399 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:50:02.398+0000 7fd40f78a700 1 -- 172.21.15.184:0/315446258 >> 172.21.15.184:0/315446258 conn(0x7fd4080feb60 msgr2=0x7fd408107b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:50:02.399 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:50:02.398+0000 7fd40f78a700 1 -- 172.21.15.184:0/315446258 shutdown_connections 2024-05-21T21:50:02.399 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:50:02.398+0000 7fd40f78a700 1 -- 172.21.15.184:0/315446258 wait complete. 2024-05-21T21:50:02.697 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:02 smithi184 ceph-mon[36942]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-21T21:50:02.697 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:02 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:02.697 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:02 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:02.697 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:02 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:50:02.697 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:02 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:50:02.697 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:02 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:02.697 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:02 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:02.698 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:02 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:02.698 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:02 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:50:02.698 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:02 smithi184 ceph-mon[36942]: from='osd.11 [v2:172.21.15.184:6824/1563340128,v1:172.21.15.184:6825/1563340128]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-21T21:50:02.698 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:02 smithi184 ceph-mon[36942]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-21T21:50:02.954 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:02 smithi151 ceph-mon[31623]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-21T21:50:02.954 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:02 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:02.954 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:02 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:02.954 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:02 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:50:02.954 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:02 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:50:02.954 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:02 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:02.954 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:02 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:02.954 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:02 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:02.954 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:02 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:50:02.954 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:02 smithi151 ceph-mon[31623]: from='osd.11 [v2:172.21.15.184:6824/1563340128,v1:172.21.15.184:6825/1563340128]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-21T21:50:02.954 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:02 smithi151 ceph-mon[31623]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-21T21:50:03.023 DEBUG:teuthology.orchestra.run.smithi184:osd.11> sudo journalctl -f -n 0 -u ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@osd.11.service 2024-05-21T21:50:03.025 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-05-21T21:50:03.026 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph osd stat -f json 2024-05-21T21:50:03.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:02 smithi164 ceph-mon[36269]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-21T21:50:03.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:02 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:03.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:02 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:03.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:02 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:50:03.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:02 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:50:03.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:02 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:03.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:02 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:03.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:02 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:03.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:02 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:50:03.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:02 smithi164 ceph-mon[36269]: from='osd.11 [v2:172.21.15.184:6824/1563340128,v1:172.21.15.184:6825/1563340128]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-21T21:50:03.037 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:02 smithi164 ceph-mon[36269]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-21T21:50:03.485 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:50:04.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:04 smithi184 ceph-mon[36942]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-21T21:50:04.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:04 smithi184 ceph-mon[36942]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-21T21:50:04.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:04 smithi184 ceph-mon[36942]: osdmap e70: 12 total, 11 up, 12 in 2024-05-21T21:50:04.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:04 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-21T21:50:04.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:04 smithi184 ceph-mon[36942]: from='osd.11 [v2:172.21.15.184:6824/1563340128,v1:172.21.15.184:6825/1563340128]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-05-21T21:50:04.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:04 smithi184 ceph-mon[36942]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-05-21T21:50:04.487 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 21:50:04 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T21:50:04.397+0000 7f61ea0e8700 -1 osd.11 0 waiting for initial osdmap 2024-05-21T21:50:04.487 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 21:50:04 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T21:50:04.411+0000 7f61e5edb700 -1 osd.11 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-21T21:50:04.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:04 smithi164 ceph-mon[36269]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-21T21:50:04.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:04 smithi164 ceph-mon[36269]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-21T21:50:04.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:04 smithi164 ceph-mon[36269]: osdmap e70: 12 total, 11 up, 12 in 2024-05-21T21:50:04.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:04 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-21T21:50:04.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:04 smithi164 ceph-mon[36269]: from='osd.11 [v2:172.21.15.184:6824/1563340128,v1:172.21.15.184:6825/1563340128]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-05-21T21:50:04.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:04 smithi164 ceph-mon[36269]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-05-21T21:50:04.631 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:04 smithi151 ceph-mon[31623]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-21T21:50:04.632 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:04 smithi151 ceph-mon[31623]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-21T21:50:04.632 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:04 smithi151 ceph-mon[31623]: osdmap e70: 12 total, 11 up, 12 in 2024-05-21T21:50:04.632 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:04 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-21T21:50:04.632 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:04 smithi151 ceph-mon[31623]: from='osd.11 [v2:172.21.15.184:6824/1563340128,v1:172.21.15.184:6825/1563340128]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-05-21T21:50:04.632 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:04 smithi151 ceph-mon[31623]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2024-05-21T21:50:05.561 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:05.559+0000 7f2f0ec00700 1 -- 172.21.15.151:0/2157656250 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2f08105190 msgr2=0x7f2f0810ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:50:05.561 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:05.559+0000 7f2f0ec00700 1 --2- 172.21.15.151:0/2157656250 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2f08105190 0x7f2f0810ba40 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2f00009fc0 tx=0x7f2f0000b330 comp rx=0 tx=0).stop 2024-05-21T21:50:05.561 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:05.560+0000 7f2f0ec00700 1 -- 172.21.15.151:0/2157656250 shutdown_connections 2024-05-21T21:50:05.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:05.560+0000 7f2f0ec00700 1 --2- 172.21.15.151:0/2157656250 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2f08105190 0x7f2f0810ba40 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:05.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:05.560+0000 7f2f0ec00700 1 --2- 172.21.15.151:0/2157656250 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2f081035f0 0x7f2f08103a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:05.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:05.560+0000 7f2f0ec00700 1 --2- 172.21.15.151:0/2157656250 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f081047f0 0x7f2f08104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:05.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:05.560+0000 7f2f0ec00700 1 -- 172.21.15.151:0/2157656250 >> 172.21.15.151:0/2157656250 conn(0x7f2f080feb90 msgr2=0x7f2f08100fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:50:05.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:05.560+0000 7f2f0ec00700 1 -- 172.21.15.151:0/2157656250 shutdown_connections 2024-05-21T21:50:05.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:05.560+0000 7f2f0ec00700 1 -- 172.21.15.151:0/2157656250 wait complete. 2024-05-21T21:50:05.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:05.561+0000 7f2f0ec00700 1 Processor -- start 2024-05-21T21:50:05.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:05.561+0000 7f2f0ec00700 1 -- start start 2024-05-21T21:50:05.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:05.562+0000 7f2f0ec00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f081035f0 0x7f2f0819b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:50:05.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:05.562+0000 7f2f0ec00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2f081047f0 0x7f2f0819be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:50:05.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:05.562+0000 7f2f0ec00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2f08105190 0x7f2f081a2e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:50:05.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:05.562+0000 7f2f0ec00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2f0810f0b0 con 0x7f2f08105190 2024-05-21T21:50:05.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:05.562+0000 7f2f0ec00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2f0810ef30 con 0x7f2f081035f0 2024-05-21T21:50:05.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:05.562+0000 7f2f0ec00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2f0810f230 con 0x7f2f081047f0 2024-05-21T21:50:05.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:05.562+0000 7f2f07e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f081035f0 0x7f2f0819b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:50:05.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:05.562+0000 7f2f0cc00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2f08105190 0x7f2f081a2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:50:05.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:05.562+0000 7f2effe00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2f081047f0 0x7f2f0819be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:50:05.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:05.562+0000 7f2f0cc00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2f08105190 0x7f2f081a2e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:51294/0 (socket says 172.21.15.151:51294) 2024-05-21T21:50:05.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:05.562+0000 7f2f0cc00700 1 -- 172.21.15.151:0/1408145010 learned_addr learned my addr 172.21.15.151:0/1408145010 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:50:05.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:05.562+0000 7f2f07e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f081035f0 0x7f2f0819b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:55676/0 (socket says 172.21.15.151:55676) 2024-05-21T21:50:05.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:05.563+0000 7f2effe00700 1 -- 172.21.15.151:0/1408145010 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f081035f0 msgr2=0x7f2f0819b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:50:05.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:05.563+0000 7f2effe00700 1 --2- 172.21.15.151:0/1408145010 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f081035f0 0x7f2f0819b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:05.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:05.563+0000 7f2effe00700 1 -- 172.21.15.151:0/1408145010 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2f08105190 msgr2=0x7f2f081a2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:50:05.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:05.563+0000 7f2effe00700 1 --2- 172.21.15.151:0/1408145010 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2f08105190 0x7f2f081a2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:05.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:05.563+0000 7f2effe00700 1 -- 172.21.15.151:0/1408145010 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f081a34f0 con 0x7f2f081047f0 2024-05-21T21:50:05.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:05.563+0000 7f2effe00700 1 --2- 172.21.15.151:0/1408145010 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2f081047f0 0x7f2f0819be20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2ef000ea20 tx=0x7f2ef000ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:50:05.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:05.563+0000 7f2f05600700 1 -- 172.21.15.151:0/1408145010 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef000cbd0 con 0x7f2f081047f0 2024-05-21T21:50:05.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:05.563+0000 7f2f05600700 1 -- 172.21.15.151:0/1408145010 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ef000ee60 con 0x7f2f081047f0 2024-05-21T21:50:05.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:05.563+0000 7f2f0ec00700 1 -- 172.21.15.151:0/1408145010 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f081a37e0 con 0x7f2f081047f0 2024-05-21T21:50:05.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:05.563+0000 7f2f05600700 1 -- 172.21.15.151:0/1408145010 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef0018680 con 0x7f2f081047f0 2024-05-21T21:50:05.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:05.563+0000 7f2f0ec00700 1 -- 172.21.15.151:0/1408145010 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f081a3d20 con 0x7f2f081047f0 2024-05-21T21:50:05.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:05.565+0000 7f2f0ec00700 1 -- 172.21.15.151:0/1408145010 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f0806a230 con 0x7f2f081047f0 2024-05-21T21:50:05.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:05.567+0000 7f2f05600700 1 -- 172.21.15.151:0/1408145010 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f2ef004aab0 con 0x7f2f081047f0 2024-05-21T21:50:05.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:05.568+0000 7f2f05600700 1 --2- 172.21.15.151:0/1408145010 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f2eec0b2880 0x7f2eec0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:50:05.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:05.568+0000 7f2f05600700 1 -- 172.21.15.151:0/1408145010 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f2ef00e1ee0 con 0x7f2f081047f0 2024-05-21T21:50:05.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:05.569+0000 7f2f07e00700 1 --2- 172.21.15.151:0/1408145010 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f2eec0b2880 0x7f2eec0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:50:05.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:05.569+0000 7f2f07e00700 1 --2- 172.21.15.151:0/1408145010 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f2eec0b2880 0x7f2eec0b4d40 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2ef8009750 tx=0x7f2ef8009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:50:05.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:05.570+0000 7f2f05600700 1 -- 172.21.15.151:0/1408145010 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2ef00aa000 con 0x7f2f081047f0 2024-05-21T21:50:05.681 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:05 smithi151 ceph-mon[31623]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2024-05-21T21:50:05.681 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:05 smithi151 ceph-mon[31623]: osdmap e71: 12 total, 11 up, 12 in 2024-05-21T21:50:05.681 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:05 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-21T21:50:05.681 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:05 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-21T21:50:05.720 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:05 smithi184 ceph-mon[36942]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2024-05-21T21:50:05.720 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:05 smithi184 ceph-mon[36942]: osdmap e71: 12 total, 11 up, 12 in 2024-05-21T21:50:05.720 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:05 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-21T21:50:05.720 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:05 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-21T21:50:05.785 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:05 smithi164 ceph-mon[36269]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2024-05-21T21:50:05.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:05 smithi164 ceph-mon[36269]: osdmap e71: 12 total, 11 up, 12 in 2024-05-21T21:50:05.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:05 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-21T21:50:05.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:05 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-21T21:50:05.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:05.811+0000 7f2f0ec00700 1 -- 172.21.15.151:0/1408145010 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f2f081a4140 con 0x7f2f081047f0 2024-05-21T21:50:05.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:05.813+0000 7f2f05600700 1 -- 172.21.15.151:0/1408145010 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v72) ==== 74+0+133 (secure 0 0 0) 0x7f2ef00b2080 con 0x7f2f081047f0 2024-05-21T21:50:05.814 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T21:50:05.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:05.816+0000 7f2f0ec00700 1 -- 172.21.15.151:0/1408145010 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f2eec0b2880 msgr2=0x7f2eec0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:50:05.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:05.816+0000 7f2f0ec00700 1 --2- 172.21.15.151:0/1408145010 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f2eec0b2880 0x7f2eec0b4d40 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2ef8009750 tx=0x7f2ef8009310 comp rx=0 tx=0).stop 2024-05-21T21:50:05.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:05.816+0000 7f2f0ec00700 1 -- 172.21.15.151:0/1408145010 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2f081047f0 msgr2=0x7f2f0819be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:50:05.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:05.816+0000 7f2f0ec00700 1 --2- 172.21.15.151:0/1408145010 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2f081047f0 0x7f2f0819be20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2ef000ea20 tx=0x7f2ef000ed30 comp rx=0 tx=0).stop 2024-05-21T21:50:05.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:05.817+0000 7f2f0ec00700 1 -- 172.21.15.151:0/1408145010 shutdown_connections 2024-05-21T21:50:05.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:05.817+0000 7f2f0ec00700 1 --2- 172.21.15.151:0/1408145010 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f2eec0b2880 0x7f2eec0b4d40 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:05.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:05.817+0000 7f2f0ec00700 1 --2- 172.21.15.151:0/1408145010 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f081035f0 0x7f2f0819b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:05.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:05.817+0000 7f2f0ec00700 1 --2- 172.21.15.151:0/1408145010 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2f08105190 0x7f2f081a2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:05.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:05.817+0000 7f2f0ec00700 1 --2- 172.21.15.151:0/1408145010 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2f081047f0 0x7f2f0819be20 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:05.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:05.817+0000 7f2f0ec00700 1 -- 172.21.15.151:0/1408145010 >> 172.21.15.151:0/1408145010 conn(0x7f2f080feb90 msgr2=0x7f2f08075770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:50:05.819 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:05.817+0000 7f2f0ec00700 1 -- 172.21.15.151:0/1408145010 shutdown_connections 2024-05-21T21:50:05.819 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:05.817+0000 7f2f0ec00700 1 -- 172.21.15.151:0/1408145010 wait complete. 2024-05-21T21:50:06.532 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:06 smithi151 ceph-mon[31623]: purged_snaps scrub starts 2024-05-21T21:50:06.533 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:06 smithi151 ceph-mon[31623]: purged_snaps scrub ok 2024-05-21T21:50:06.533 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:06 smithi151 ceph-mon[31623]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-21T21:50:06.533 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:06 smithi151 ceph-mon[31623]: osd.11 [v2:172.21.15.184:6824/1563340128,v1:172.21.15.184:6825/1563340128] boot 2024-05-21T21:50:06.533 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:06 smithi151 ceph-mon[31623]: osdmap e72: 12 total, 12 up, 12 in 2024-05-21T21:50:06.533 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:06 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-21T21:50:06.533 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:06 smithi151 ceph-mon[31623]: Detected new or changed devices on smithi184 2024-05-21T21:50:06.533 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:06 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:06.533 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:06 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:06.533 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:06 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:50:06.533 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:06 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:50:06.533 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:06 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:50:06.534 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:06 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:50:06.534 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:06 smithi151 ceph-mon[31623]: Adjusting osd_memory_target on smithi184 to 4272M 2024-05-21T21:50:06.534 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:06 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:06.534 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:06 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:50:06.534 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:06 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:50:06.534 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:06 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/1408145010' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-21T21:50:06.534 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:06 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:06.593 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":72,"num_osds":12,"num_up_osds":12,"osd_up_since":1716328205,"num_in_osds":12,"osd_in_since":1716328185,"num_remapped_pgs":0} 2024-05-21T21:50:06.594 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph osd dump --format=json 2024-05-21T21:50:06.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:06 smithi184 ceph-mon[36942]: purged_snaps scrub starts 2024-05-21T21:50:06.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:06 smithi184 ceph-mon[36942]: purged_snaps scrub ok 2024-05-21T21:50:06.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:06 smithi184 ceph-mon[36942]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-21T21:50:06.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:06 smithi184 ceph-mon[36942]: osd.11 [v2:172.21.15.184:6824/1563340128,v1:172.21.15.184:6825/1563340128] boot 2024-05-21T21:50:06.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:06 smithi184 ceph-mon[36942]: osdmap e72: 12 total, 12 up, 12 in 2024-05-21T21:50:06.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:06 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-21T21:50:06.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:06 smithi184 ceph-mon[36942]: Detected new or changed devices on smithi184 2024-05-21T21:50:06.737 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:06 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:06.737 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:06 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:06.737 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:06 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:50:06.737 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:06 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:50:06.737 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:06 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:50:06.737 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:06 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:50:06.737 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:06 smithi184 ceph-mon[36942]: Adjusting osd_memory_target on smithi184 to 4272M 2024-05-21T21:50:06.737 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:06 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:06.737 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:06 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:50:06.737 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:06 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:50:06.737 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:06 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/1408145010' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-21T21:50:06.737 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:06 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:06.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:06 smithi164 ceph-mon[36269]: purged_snaps scrub starts 2024-05-21T21:50:06.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:06 smithi164 ceph-mon[36269]: purged_snaps scrub ok 2024-05-21T21:50:06.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:06 smithi164 ceph-mon[36269]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-21T21:50:06.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:06 smithi164 ceph-mon[36269]: osd.11 [v2:172.21.15.184:6824/1563340128,v1:172.21.15.184:6825/1563340128] boot 2024-05-21T21:50:06.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:06 smithi164 ceph-mon[36269]: osdmap e72: 12 total, 12 up, 12 in 2024-05-21T21:50:06.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:06 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-21T21:50:06.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:06 smithi164 ceph-mon[36269]: Detected new or changed devices on smithi184 2024-05-21T21:50:06.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:06 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:06.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:06 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:06.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:06 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:50:06.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:06 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:50:06.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:06 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:50:06.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:06 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:50:06.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:06 smithi164 ceph-mon[36269]: Adjusting osd_memory_target on smithi184 to 4272M 2024-05-21T21:50:06.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:06 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:06.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:06 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:50:06.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:06 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:50:06.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:06 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/1408145010' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-21T21:50:06.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:06 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:07.015 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:50:07.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:07 smithi151 ceph-mon[31623]: osdmap e73: 12 total, 12 up, 12 in 2024-05-21T21:50:07.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:07 smithi184 ceph-mon[36942]: osdmap e73: 12 total, 12 up, 12 in 2024-05-21T21:50:07.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:07 smithi164 ceph-mon[36269]: osdmap e73: 12 total, 12 up, 12 in 2024-05-21T21:50:08.666 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:08 smithi151 ceph-mon[31623]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:50:08.735 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:08 smithi184 ceph-mon[36942]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:50:08.785 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:08 smithi164 ceph-mon[36269]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:50:09.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:09.163+0000 7f04ca800700 1 -- 172.21.15.151:0/2276105377 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f04c4102fd0 msgr2=0x7f04c4107020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:50:09.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:09.163+0000 7f04ca800700 1 --2- 172.21.15.151:0/2276105377 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f04c4102fd0 0x7f04c4107020 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f04b00098f0 tx=0x7f04b0009c00 comp rx=0 tx=0).stop 2024-05-21T21:50:09.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:09.163+0000 7f04ca800700 1 -- 172.21.15.151:0/2276105377 shutdown_connections 2024-05-21T21:50:09.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:09.163+0000 7f04ca800700 1 --2- 172.21.15.151:0/2276105377 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f04c4107700 0x7f04c410f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:09.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:09.163+0000 7f04ca800700 1 --2- 172.21.15.151:0/2276105377 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04c4102620 0x7f04c4102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:09.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:09.163+0000 7f04ca800700 1 --2- 172.21.15.151:0/2276105377 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f04c4102fd0 0x7f04c4107020 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:09.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:09.163+0000 7f04ca800700 1 -- 172.21.15.151:0/2276105377 >> 172.21.15.151:0/2276105377 conn(0x7f04c40fded0 msgr2=0x7f04c41002f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:50:09.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:09.163+0000 7f04ca800700 1 -- 172.21.15.151:0/2276105377 shutdown_connections 2024-05-21T21:50:09.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:09.163+0000 7f04ca800700 1 -- 172.21.15.151:0/2276105377 wait complete. 2024-05-21T21:50:09.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:09.164+0000 7f04ca800700 1 Processor -- start 2024-05-21T21:50:09.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:09.164+0000 7f04ca800700 1 -- start start 2024-05-21T21:50:09.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:09.164+0000 7f04ca800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f04c4102620 0x7f04c419be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:50:09.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:09.165+0000 7f04ca800700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04c4102fd0 0x7f04c419c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:50:09.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:09.165+0000 7f04ca800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f04c4107700 0x7f04c41a0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:50:09.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:09.165+0000 7f04ca800700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f04c4112570 con 0x7f04c4102620 2024-05-21T21:50:09.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:09.165+0000 7f04ca800700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f04c41123f0 con 0x7f04c4107700 2024-05-21T21:50:09.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:09.165+0000 7f04ca800700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f04c41126f0 con 0x7f04c4102fd0 2024-05-21T21:50:09.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:09.165+0000 7f04c9e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f04c4107700 0x7f04c41a0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:50:09.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:09.165+0000 7f04c8a00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04c4102fd0 0x7f04c419c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:50:09.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:09.165+0000 7f04c9e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f04c4107700 0x7f04c41a0740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:57024/0 (socket says 172.21.15.151:57024) 2024-05-21T21:50:09.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:09.165+0000 7f04c9e00700 1 -- 172.21.15.151:0/390582933 learned_addr learned my addr 172.21.15.151:0/390582933 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:50:09.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:09.165+0000 7f04c9400700 1 --2- 172.21.15.151:0/390582933 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f04c4102620 0x7f04c419be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:50:09.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:09.165+0000 7f04c8a00700 1 -- 172.21.15.151:0/390582933 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f04c4107700 msgr2=0x7f04c41a0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:50:09.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:09.165+0000 7f04c8a00700 1 --2- 172.21.15.151:0/390582933 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f04c4107700 0x7f04c41a0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:09.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:09.165+0000 7f04c8a00700 1 -- 172.21.15.151:0/390582933 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f04c4102620 msgr2=0x7f04c419be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:50:09.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:09.165+0000 7f04c8a00700 1 --2- 172.21.15.151:0/390582933 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f04c4102620 0x7f04c419be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:09.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:09.165+0000 7f04c8a00700 1 -- 172.21.15.151:0/390582933 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04c41a0e80 con 0x7f04c4102fd0 2024-05-21T21:50:09.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:09.166+0000 7f04c8a00700 1 --2- 172.21.15.151:0/390582933 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04c4102fd0 0x7f04c419c3b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f04b0000c00 tx=0x7f04b000fa20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:50:09.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:09.166+0000 7f04c2000700 1 -- 172.21.15.151:0/390582933 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b0005040 con 0x7f04c4102fd0 2024-05-21T21:50:09.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:09.166+0000 7f04c2000700 1 -- 172.21.15.151:0/390582933 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04b0005690 con 0x7f04c4102fd0 2024-05-21T21:50:09.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:09.166+0000 7f04c2000700 1 -- 172.21.15.151:0/390582933 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b00038e0 con 0x7f04c4102fd0 2024-05-21T21:50:09.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:09.166+0000 7f04ca800700 1 -- 172.21.15.151:0/390582933 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04c41a1110 con 0x7f04c4102fd0 2024-05-21T21:50:09.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:09.166+0000 7f04ca800700 1 -- 172.21.15.151:0/390582933 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04c41a89f0 con 0x7f04c4102fd0 2024-05-21T21:50:09.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:09.167+0000 7f04ca800700 1 -- 172.21.15.151:0/390582933 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04c406a230 con 0x7f04c4102fd0 2024-05-21T21:50:09.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:09.170+0000 7f04c2000700 1 -- 172.21.15.151:0/390582933 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f04b000b6f0 con 0x7f04c4102fd0 2024-05-21T21:50:09.172 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:09.171+0000 7f04c2000700 1 --2- 172.21.15.151:0/390582933 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f04ac0b2900 0x7f04ac0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:50:09.173 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:09.172+0000 7f04c2000700 1 -- 172.21.15.151:0/390582933 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f04b00e29b0 con 0x7f04c4102fd0 2024-05-21T21:50:09.173 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:09.172+0000 7f04c9400700 1 --2- 172.21.15.151:0/390582933 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f04ac0b2900 0x7f04ac0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:50:09.173 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:09.172+0000 7f04c9400700 1 --2- 172.21.15.151:0/390582933 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f04ac0b2900 0x7f04ac0b4dc0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f04b8009c60 tx=0x7f04b8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:50:09.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:09.174+0000 7f04c2000700 1 -- 172.21.15.151:0/390582933 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f04b0013030 con 0x7f04c4102fd0 2024-05-21T21:50:09.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:09.406+0000 7f04ca800700 1 -- 172.21.15.151:0/390582933 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f04c41a8bc0 con 0x7f04c4102fd0 2024-05-21T21:50:09.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:09.407+0000 7f04c2000700 1 -- 172.21.15.151:0/390582933 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) ==== 74+0+18501 (secure 0 0 0) 0x7f04b00aabf0 con 0x7f04c4102fd0 2024-05-21T21:50:09.409 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T21:50:09.409 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":73,"fsid":"c8c7a2e6-17ba-11ef-bc9a-c7b262605968","created":"2024-05-21T21:41:52.412833+0000","modified":"2024-05-21T21:50:06.411320+0000","last_up_change":"2024-05-21T21:50:05.401314+0000","last_in_change":"2024-05-21T21:49:45.684136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-21T21:45:55.196046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"18db8e54-0cfc-4e12-9c84-6f72a99cc989","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":2420400473},{"type":"v1","addr":"172.21.15.151:6803","nonce":2420400473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":2420400473},{"type":"v1","addr":"172.21.15.151:6805","nonce":2420400473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":2420400473},{"type":"v1","addr":"172.21.15.151:6809","nonce":2420400473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":2420400473},{"type":"v1","addr":"172.21.15.151:6807","nonce":2420400473}]},"public_addr":"172.21.15.151:6803/2420400473","cluster_addr":"172.21.15.151:6805/2420400473","heartbeat_back_addr":"172.21.15.151:6809/2420400473","heartbeat_front_addr":"172.21.15.151:6807/2420400473","state":["exists","up"]},{"osd":1,"uuid":"bac384c7-1102-4815-81a8-10cc7661d230","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":3900240447},{"type":"v1","addr":"172.21.15.164:6801","nonce":3900240447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":3900240447},{"type":"v1","addr":"172.21.15.164:6803","nonce":3900240447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":3900240447},{"type":"v1","addr":"172.21.15.164:6807","nonce":3900240447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":3900240447},{"type":"v1","addr":"172.21.15.164:6805","nonce":3900240447}]},"public_addr":"172.21.15.164:6801/3900240447","cluster_addr":"172.21.15.164:6803/3900240447","heartbeat_back_addr":"172.21.15.164:6807/3900240447","heartbeat_front_addr":"172.21.15.164:6805/3900240447","state":["exists","up"]},{"osd":2,"uuid":"b7e4a782-1518-4d43-bee9-7fd768ee5cb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":3971150737},{"type":"v1","addr":"172.21.15.184:6801","nonce":3971150737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3971150737},{"type":"v1","addr":"172.21.15.184:6803","nonce":3971150737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3971150737},{"type":"v1","addr":"172.21.15.184:6807","nonce":3971150737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3971150737},{"type":"v1","addr":"172.21.15.184:6805","nonce":3971150737}]},"public_addr":"172.21.15.184:6801/3971150737","cluster_addr":"172.21.15.184:6803/3971150737","heartbeat_back_addr":"172.21.15.184:6807/3971150737","heartbeat_front_addr":"172.21.15.184:6805/3971150737","state":["exists","up"]},{"osd":3,"uuid":"ab3df826-39e3-4635-a49c-2b5ffdbf12f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":556594400},{"type":"v1","addr":"172.21.15.151:6811","nonce":556594400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":556594400},{"type":"v1","addr":"172.21.15.151:6813","nonce":556594400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":556594400},{"type":"v1","addr":"172.21.15.151:6817","nonce":556594400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":556594400},{"type":"v1","addr":"172.21.15.151:6815","nonce":556594400}]},"public_addr":"172.21.15.151:6811/556594400","cluster_addr":"172.21.15.151:6813/556594400","heartbeat_back_addr":"172.21.15.151:6817/556594400","heartbeat_front_addr":"172.21.15.151:6815/556594400","state":["exists","up"]},{"osd":4,"uuid":"00cbf6ae-0f42-4685-a77d-d803c6467aa0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":988881973},{"type":"v1","addr":"172.21.15.164:6809","nonce":988881973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":988881973},{"type":"v1","addr":"172.21.15.164:6811","nonce":988881973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":988881973},{"type":"v1","addr":"172.21.15.164:6815","nonce":988881973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":988881973},{"type":"v1","addr":"172.21.15.164:6813","nonce":988881973}]},"public_addr":"172.21.15.164:6809/988881973","cluster_addr":"172.21.15.164:6811/988881973","heartbeat_back_addr":"172.21.15.164:6815/988881973","heartbeat_front_addr":"172.21.15.164:6813/988881973","state":["exists","up"]},{"osd":5,"uuid":"fad14221-0ef0-4d40-b414-c6c9f1f9718c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3293311106},{"type":"v1","addr":"172.21.15.184:6809","nonce":3293311106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":3293311106},{"type":"v1","addr":"172.21.15.184:6811","nonce":3293311106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":3293311106},{"type":"v1","addr":"172.21.15.184:6815","nonce":3293311106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":3293311106},{"type":"v1","addr":"172.21.15.184:6813","nonce":3293311106}]},"public_addr":"172.21.15.184:6809/3293311106","cluster_addr":"172.21.15.184:6811/3293311106","heartbeat_back_addr":"172.21.15.184:6815/3293311106","heartbeat_front_addr":"172.21.15.184:6813/3293311106","state":["exists","up"]},{"osd":6,"uuid":"ab8a60f6-0dec-4c1c-89db-f2e2f638f1ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":3377550762},{"type":"v1","addr":"172.21.15.151:6819","nonce":3377550762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":3377550762},{"type":"v1","addr":"172.21.15.151:6821","nonce":3377550762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":3377550762},{"type":"v1","addr":"172.21.15.151:6825","nonce":3377550762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":3377550762},{"type":"v1","addr":"172.21.15.151:6823","nonce":3377550762}]},"public_addr":"172.21.15.151:6819/3377550762","cluster_addr":"172.21.15.151:6821/3377550762","heartbeat_back_addr":"172.21.15.151:6825/3377550762","heartbeat_front_addr":"172.21.15.151:6823/3377550762","state":["exists","up"]},{"osd":7,"uuid":"db332268-9972-4ed7-a307-822b83a0fbb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":3869696936},{"type":"v1","addr":"172.21.15.164:6817","nonce":3869696936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":3869696936},{"type":"v1","addr":"172.21.15.164:6819","nonce":3869696936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":3869696936},{"type":"v1","addr":"172.21.15.164:6823","nonce":3869696936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":3869696936},{"type":"v1","addr":"172.21.15.164:6821","nonce":3869696936}]},"public_addr":"172.21.15.164:6817/3869696936","cluster_addr":"172.21.15.164:6819/3869696936","heartbeat_back_addr":"172.21.15.164:6823/3869696936","heartbeat_front_addr":"172.21.15.164:6821/3869696936","state":["exists","up"]},{"osd":8,"uuid":"fe2fbe1c-549a-4947-a188-426dfca04d10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":575273541},{"type":"v1","addr":"172.21.15.184:6817","nonce":575273541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":575273541},{"type":"v1","addr":"172.21.15.184:6819","nonce":575273541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":575273541},{"type":"v1","addr":"172.21.15.184:6823","nonce":575273541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":575273541},{"type":"v1","addr":"172.21.15.184:6821","nonce":575273541}]},"public_addr":"172.21.15.184:6817/575273541","cluster_addr":"172.21.15.184:6819/575273541","heartbeat_back_addr":"172.21.15.184:6823/575273541","heartbeat_front_addr":"172.21.15.184:6821/575273541","state":["exists","up"]},{"osd":9,"uuid":"3a7b450f-8004-4adf-bae3-970918f98f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":337128533},{"type":"v1","addr":"172.21.15.151:6827","nonce":337128533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":337128533},{"type":"v1","addr":"172.21.15.151:6829","nonce":337128533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":337128533},{"type":"v1","addr":"172.21.15.151:6833","nonce":337128533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":337128533},{"type":"v1","addr":"172.21.15.151:6831","nonce":337128533}]},"public_addr":"172.21.15.151:6827/337128533","cluster_addr":"172.21.15.151:6829/337128533","heartbeat_back_addr":"172.21.15.151:6833/337128533","heartbeat_front_addr":"172.21.15.151:6831/337128533","state":["exists","up"]},{"osd":10,"uuid":"8fcdf271-90c4-4db3-be69-b0ca716dfd80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":1823448782},{"type":"v1","addr":"172.21.15.164:6825","nonce":1823448782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":1823448782},{"type":"v1","addr":"172.21.15.164:6827","nonce":1823448782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":1823448782},{"type":"v1","addr":"172.21.15.164:6831","nonce":1823448782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":1823448782},{"type":"v1","addr":"172.21.15.164:6829","nonce":1823448782}]},"public_addr":"172.21.15.164:6825/1823448782","cluster_addr":"172.21.15.164:6827/1823448782","heartbeat_back_addr":"172.21.15.164:6831/1823448782","heartbeat_front_addr":"172.21.15.164:6829/1823448782","state":["exists","up"]},{"osd":11,"uuid":"e661477b-0c09-409f-a2fe-6ebd34642a8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":1563340128},{"type":"v1","addr":"172.21.15.184:6825","nonce":1563340128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":1563340128},{"type":"v1","addr":"172.21.15.184:6827","nonce":1563340128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":1563340128},{"type":"v1","addr":"172.21.15.184:6831","nonce":1563340128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":1563340128},{"type":"v1","addr":"172.21.15.184:6829","nonce":1563340128}]},"public_addr":"172.21.15.184:6825/1563340128","cluster_addr":"172.21.15.184:6827/1563340128","heartbeat_back_addr":"172.21.15.184:6831/1563340128","heartbeat_front_addr":"172.21.15.184:6829/1563340128","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:45:02.821160+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:45:27.844564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:45:51.904981+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:46:24.376245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:46:50.036621+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:47:14.748422+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:47:47.396383+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:48:13.543174+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:48:38.213285+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:49:11.980042+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:49:38.519761+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:50:03.439898+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.151:0/31963128":"2024-05-22T21:43:09.041455+0000","172.21.15.151:6800/3428827300":"2024-05-22T21:43:09.041455+0000","172.21.15.151:6801/3428827300":"2024-05-22T21:43:09.041455+0000","172.21.15.151:0/131989141":"2024-05-22T21:43:09.041455+0000","172.21.15.151:6801/1128885292":"2024-05-22T21:42:35.440147+0000","172.21.15.151:0/2703664519":"2024-05-22T21:43:09.041455+0000","172.21.15.151:6800/1128885292":"2024-05-22T21:42:35.440147+0000","172.21.15.151:0/3052943699":"2024-05-22T21:42:35.440147+0000","172.21.15.151:0/3929452511":"2024-05-22T21:42:35.440147+0000","172.21.15.151:0/3762889700":"2024-05-22T21:42:35.440147+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-21T21:50:09.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:09.410+0000 7f04ca800700 1 -- 172.21.15.151:0/390582933 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f04ac0b2900 msgr2=0x7f04ac0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:50:09.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:09.410+0000 7f04ca800700 1 --2- 172.21.15.151:0/390582933 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f04ac0b2900 0x7f04ac0b4dc0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f04b8009c60 tx=0x7f04b8009260 comp rx=0 tx=0).stop 2024-05-21T21:50:09.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:09.410+0000 7f04ca800700 1 -- 172.21.15.151:0/390582933 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04c4102fd0 msgr2=0x7f04c419c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:50:09.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:09.410+0000 7f04ca800700 1 --2- 172.21.15.151:0/390582933 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04c4102fd0 0x7f04c419c3b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f04b0000c00 tx=0x7f04b000fa20 comp rx=0 tx=0).stop 2024-05-21T21:50:09.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:09.411+0000 7f04ca800700 1 -- 172.21.15.151:0/390582933 shutdown_connections 2024-05-21T21:50:09.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:09.411+0000 7f04ca800700 1 --2- 172.21.15.151:0/390582933 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f04ac0b2900 0x7f04ac0b4dc0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:09.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:09.411+0000 7f04ca800700 1 --2- 172.21.15.151:0/390582933 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f04c4107700 0x7f04c41a0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:09.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:09.411+0000 7f04ca800700 1 --2- 172.21.15.151:0/390582933 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f04c4102620 0x7f04c419be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:09.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:09.411+0000 7f04ca800700 1 --2- 172.21.15.151:0/390582933 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04c4102fd0 0x7f04c419c3b0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:09.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:09.411+0000 7f04ca800700 1 -- 172.21.15.151:0/390582933 >> 172.21.15.151:0/390582933 conn(0x7f04c40fded0 msgr2=0x7f04c406bcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:50:09.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:09.411+0000 7f04ca800700 1 -- 172.21.15.151:0/390582933 shutdown_connections 2024-05-21T21:50:09.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:09.411+0000 7f04ca800700 1 -- 172.21.15.151:0/390582933 wait complete. 2024-05-21T21:50:09.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:09 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/390582933' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T21:50:09.735 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:09 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/390582933' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T21:50:09.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:09 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/390582933' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T21:50:10.211 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-21T21:45:55.196046+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-21T21:50:10.211 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-05-21T21:50:10.652 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:50:10.735 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:10 smithi184 ceph-mon[36942]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:50:10.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:10 smithi164 ceph-mon[36269]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:50:10.799 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:10 smithi151 ceph-mon[31623]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:50:12.555 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:12 smithi151 ceph-mon[31623]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:50:12.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:12.727+0000 7f8fc3200700 1 -- 172.21.15.151:0/3908448456 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8fc4104680 msgr2=0x7f8fc4104a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:50:12.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:12.727+0000 7f8fc3200700 1 --2- 172.21.15.151:0/3908448456 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8fc4104680 0x7f8fc4104a60 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4009990 tx=0x7f8fb4009ca0 comp rx=0 tx=0).stop 2024-05-21T21:50:12.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:12.728+0000 7f8fc3200700 1 -- 172.21.15.151:0/3908448456 shutdown_connections 2024-05-21T21:50:12.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:12.728+0000 7f8fc3200700 1 --2- 172.21.15.151:0/3908448456 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8fc410dab0 0x7f8fc410ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:12.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:12.728+0000 7f8fc3200700 1 --2- 172.21.15.151:0/3908448456 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8fc4104680 0x7f8fc4104a60 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:12.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:12.728+0000 7f8fc3200700 1 --2- 172.21.15.151:0/3908448456 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8fc4104fa0 0x7f8fc410d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:12.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:12.728+0000 7f8fc3200700 1 -- 172.21.15.151:0/3908448456 >> 172.21.15.151:0/3908448456 conn(0x7f8fc40fded0 msgr2=0x7f8fc41002f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:50:12.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:12.728+0000 7f8fc3200700 1 -- 172.21.15.151:0/3908448456 shutdown_connections 2024-05-21T21:50:12.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:12.728+0000 7f8fc3200700 1 -- 172.21.15.151:0/3908448456 wait complete. 2024-05-21T21:50:12.732 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:12.729+0000 7f8fc3200700 1 Processor -- start 2024-05-21T21:50:12.732 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:12.729+0000 7f8fc3200700 1 -- start start 2024-05-21T21:50:12.732 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:12.729+0000 7f8fc3200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8fc4104680 0x7f8fc419be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:50:12.732 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:12.729+0000 7f8fc3200700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8fc4104fa0 0x7f8fc419c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:50:12.732 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:12.729+0000 7f8fc3200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8fc410dab0 0x7f8fc41a0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:50:12.732 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:12.729+0000 7f8fc3200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc4113720 con 0x7f8fc410dab0 2024-05-21T21:50:12.732 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:12.729+0000 7f8fc3200700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc41135a0 con 0x7f8fc4104680 2024-05-21T21:50:12.732 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:12.729+0000 7f8fc3200700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc41138a0 con 0x7f8fc4104fa0 2024-05-21T21:50:12.732 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:12.730+0000 7f8fc2800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8fc410dab0 0x7f8fc41a0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:50:12.732 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:12.730+0000 7f8fc2800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8fc410dab0 0x7f8fc41a0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:35526/0 (socket says 172.21.15.151:35526) 2024-05-21T21:50:12.732 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:12.730+0000 7f8fc2800700 1 -- 172.21.15.151:0/1874058553 learned_addr learned my addr 172.21.15.151:0/1874058553 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:50:12.732 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:12.730+0000 7f8fc1e00700 1 --2- 172.21.15.151:0/1874058553 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8fc4104680 0x7f8fc419be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:50:12.733 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:12.730+0000 7f8fc1400700 1 --2- 172.21.15.151:0/1874058553 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8fc4104fa0 0x7f8fc419c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:50:12.733 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:12.730+0000 7f8fc1e00700 1 -- 172.21.15.151:0/1874058553 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8fc4104fa0 msgr2=0x7f8fc419c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:50:12.733 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:12.730+0000 7f8fc1e00700 1 --2- 172.21.15.151:0/1874058553 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8fc4104fa0 0x7f8fc419c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:12.733 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:12.730+0000 7f8fc1e00700 1 -- 172.21.15.151:0/1874058553 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8fc410dab0 msgr2=0x7f8fc41a0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:50:12.733 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:12.730+0000 7f8fc1e00700 1 --2- 172.21.15.151:0/1874058553 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8fc410dab0 0x7f8fc41a0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:12.733 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:12.730+0000 7f8fc1e00700 1 -- 172.21.15.151:0/1874058553 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fc41a0e60 con 0x7f8fc4104680 2024-05-21T21:50:12.733 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:12.730+0000 7f8fc2800700 1 --2- 172.21.15.151:0/1874058553 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8fc410dab0 0x7f8fc41a0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:50:12.733 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:12.731+0000 7f8fc1e00700 1 --2- 172.21.15.151:0/1874058553 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8fc4104680 0x7f8fc419be50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4000c00 tx=0x7f8fb40047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:50:12.733 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:12.731+0000 7f8fbaa00700 1 -- 172.21.15.151:0/1874058553 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb4026070 con 0x7f8fc4104680 2024-05-21T21:50:12.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:12.731+0000 7f8fc3200700 1 -- 172.21.15.151:0/1874058553 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fc41a10f0 con 0x7f8fc4104680 2024-05-21T21:50:12.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:12.731+0000 7f8fbaa00700 1 -- 172.21.15.151:0/1874058553 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fb4021410 con 0x7f8fc4104680 2024-05-21T21:50:12.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:12.731+0000 7f8fbaa00700 1 -- 172.21.15.151:0/1874058553 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb4017650 con 0x7f8fc4104680 2024-05-21T21:50:12.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:12.731+0000 7f8fc3200700 1 -- 172.21.15.151:0/1874058553 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fc41a89d0 con 0x7f8fc4104680 2024-05-21T21:50:12.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:12.732+0000 7f8fc3200700 1 -- 172.21.15.151:0/1874058553 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fc406a230 con 0x7f8fc4104680 2024-05-21T21:50:12.735 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:12.734+0000 7f8fbaa00700 1 -- 172.21.15.151:0/1874058553 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f8fb4029070 con 0x7f8fc4104680 2024-05-21T21:50:12.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:12 smithi184 ceph-mon[36942]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:50:12.737 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:12.736+0000 7f8fbaa00700 1 --2- 172.21.15.151:0/1874058553 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f8fa80b2880 0x7f8fa80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:50:12.737 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:12.736+0000 7f8fbaa00700 1 -- 172.21.15.151:0/1874058553 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f8fb40e31d0 con 0x7f8fc4104680 2024-05-21T21:50:12.737 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:12.736+0000 7f8fc1400700 1 --2- 172.21.15.151:0/1874058553 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f8fa80b2880 0x7f8fa80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:50:12.738 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:12.737+0000 7f8fc1400700 1 --2- 172.21.15.151:0/1874058553 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f8fa80b2880 0x7f8fa80b4d40 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8fc410f2b0 tx=0x7f8fb0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:50:12.739 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:12.738+0000 7f8fbaa00700 1 -- 172.21.15.151:0/1874058553 <== mon.2 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8fb4030070 con 0x7f8fc4104680 2024-05-21T21:50:12.785 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:12 smithi164 ceph-mon[36269]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:50:12.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:12.967+0000 7f8fc3200700 1 -- 172.21.15.151:0/1874058553 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8fc41a8ba0 con 0x7f8fc4104680 2024-05-21T21:50:12.970 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:12.968+0000 7f8fbaa00700 1 -- 172.21.15.151:0/1874058553 <== mon.2 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v73) ==== 93+0+10 (secure 0 0 0) 0x7f8fb4013070 con 0x7f8fc4104680 2024-05-21T21:50:12.970 INFO:teuthology.orchestra.run.smithi151.stdout:pg_num: 1 2024-05-21T21:50:12.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:12.971+0000 7f8fc3200700 1 -- 172.21.15.151:0/1874058553 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f8fa80b2880 msgr2=0x7f8fa80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:50:12.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:12.972+0000 7f8fc3200700 1 --2- 172.21.15.151:0/1874058553 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f8fa80b2880 0x7f8fa80b4d40 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8fc410f2b0 tx=0x7f8fb0009260 comp rx=0 tx=0).stop 2024-05-21T21:50:12.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:12.972+0000 7f8fc3200700 1 -- 172.21.15.151:0/1874058553 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8fc4104680 msgr2=0x7f8fc419be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:50:12.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:12.972+0000 7f8fc3200700 1 --2- 172.21.15.151:0/1874058553 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8fc4104680 0x7f8fc419be50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4000c00 tx=0x7f8fb40047b0 comp rx=0 tx=0).stop 2024-05-21T21:50:12.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:12.972+0000 7f8fc3200700 1 -- 172.21.15.151:0/1874058553 shutdown_connections 2024-05-21T21:50:12.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:12.972+0000 7f8fc3200700 1 --2- 172.21.15.151:0/1874058553 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f8fa80b2880 0x7f8fa80b4d40 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:12.974 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:12.972+0000 7f8fc3200700 1 --2- 172.21.15.151:0/1874058553 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8fc4104680 0x7f8fc419be50 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:12.974 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:12.972+0000 7f8fc3200700 1 --2- 172.21.15.151:0/1874058553 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8fc410dab0 0x7f8fc41a0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:12.974 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:12.972+0000 7f8fc3200700 1 --2- 172.21.15.151:0/1874058553 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8fc4104fa0 0x7f8fc419c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:12.974 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:12.972+0000 7f8fc3200700 1 -- 172.21.15.151:0/1874058553 >> 172.21.15.151:0/1874058553 conn(0x7f8fc40fded0 msgr2=0x7f8fc41002c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:50:12.974 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:12.972+0000 7f8fc3200700 1 -- 172.21.15.151:0/1874058553 shutdown_connections 2024-05-21T21:50:12.974 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:12.972+0000 7f8fc3200700 1 -- 172.21.15.151:0/1874058553 wait complete. 2024-05-21T21:50:13.584 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:13 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/1874058553' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-21T21:50:13.657 INFO:tasks.cephadm:Adding mds.a on smithi151 2024-05-21T21:50:13.657 INFO:tasks.cephadm:Adding mds.d on smithi151 2024-05-21T21:50:13.657 INFO:tasks.cephadm:Adding mds.g on smithi151 2024-05-21T21:50:13.657 INFO:tasks.cephadm:Adding mds.j on smithi151 2024-05-21T21:50:13.657 INFO:tasks.cephadm:Adding mds.b on smithi164 2024-05-21T21:50:13.657 INFO:tasks.cephadm:Adding mds.e on smithi164 2024-05-21T21:50:13.657 INFO:tasks.cephadm:Adding mds.h on smithi164 2024-05-21T21:50:13.657 INFO:tasks.cephadm:Adding mds.k on smithi164 2024-05-21T21:50:13.657 INFO:tasks.cephadm:Adding mds.c on smithi184 2024-05-21T21:50:13.657 INFO:tasks.cephadm:Adding mds.f on smithi184 2024-05-21T21:50:13.657 INFO:tasks.cephadm:Adding mds.i on smithi184 2024-05-21T21:50:13.657 INFO:tasks.cephadm:Adding mds.l on smithi184 2024-05-21T21:50:13.658 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph orch apply mds all '12;smithi151=a;smithi151=d;smithi151=g;smithi151=j;smithi164=b;smithi164=e;smithi164=h;smithi164=k;smithi184=c;smithi184=f;smithi184=i;smithi184=l' 2024-05-21T21:50:13.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:13 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/1874058553' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-21T21:50:13.752 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:13 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/1874058553' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-21T21:50:13.907 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.c/config 2024-05-21T21:50:14.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:14 smithi184 ceph-mon[36942]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:50:14.785 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:14 smithi164 ceph-mon[36269]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:50:14.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:14 smithi151 ceph-mon[31623]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:50:15.428 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:50:15.425+0000 7feb48743700 1 -- 172.21.15.184:0/4236205746 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feb401035a0 msgr2=0x7feb401039c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:50:15.428 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:50:15.425+0000 7feb48743700 1 --2- 172.21.15.184:0/4236205746 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feb401035a0 0x7feb401039c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7feb3c009990 tx=0x7feb3c009ca0 comp rx=0 tx=0).stop 2024-05-21T21:50:15.429 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:50:15.425+0000 7feb48743700 1 -- 172.21.15.184:0/4236205746 shutdown_connections 2024-05-21T21:50:15.429 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:50:15.425+0000 7feb48743700 1 --2- 172.21.15.184:0/4236205746 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb40105140 0x7feb4010b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:15.429 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:50:15.425+0000 7feb48743700 1 --2- 172.21.15.184:0/4236205746 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feb401035a0 0x7feb401039c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:15.429 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:50:15.425+0000 7feb48743700 1 --2- 172.21.15.184:0/4236205746 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feb401047a0 0x7feb40104c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:15.429 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:50:15.425+0000 7feb48743700 1 -- 172.21.15.184:0/4236205746 >> 172.21.15.184:0/4236205746 conn(0x7feb400feb40 msgr2=0x7feb40100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:50:15.429 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:50:15.425+0000 7feb48743700 1 -- 172.21.15.184:0/4236205746 shutdown_connections 2024-05-21T21:50:15.429 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:50:15.426+0000 7feb48743700 1 -- 172.21.15.184:0/4236205746 wait complete. 2024-05-21T21:50:15.429 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:50:15.426+0000 7feb48743700 1 Processor -- start 2024-05-21T21:50:15.429 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:50:15.426+0000 7feb48743700 1 -- start start 2024-05-21T21:50:15.429 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:50:15.426+0000 7feb48743700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb401035a0 0x7feb40199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:50:15.429 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:50:15.426+0000 7feb48743700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feb401047a0 0x7feb40199d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:50:15.429 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:50:15.426+0000 7feb48743700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feb40105140 0x7feb401a0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:50:15.430 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:50:15.426+0000 7feb48743700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7feb4010f110 con 0x7feb40105140 2024-05-21T21:50:15.430 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:50:15.426+0000 7feb48743700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7feb4010ef90 con 0x7feb401035a0 2024-05-21T21:50:15.430 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:50:15.426+0000 7feb48743700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7feb4010f290 con 0x7feb401047a0 2024-05-21T21:50:15.430 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:50:15.426+0000 7feb45cde700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feb401047a0 0x7feb40199d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:50:15.430 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:50:15.427+0000 7feb45cde700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feb401047a0 0x7feb40199d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:35896/0 (socket says 172.21.15.184:35896) 2024-05-21T21:50:15.430 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:50:15.427+0000 7feb45cde700 1 -- 172.21.15.184:0/3319877681 learned_addr learned my addr 172.21.15.184:0/3319877681 (peer_addr_for_me v2:172.21.15.184:0/0) 2024-05-21T21:50:15.430 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:50:15.427+0000 7feb464df700 1 --2- 172.21.15.184:0/3319877681 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb401035a0 0x7feb40199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:50:15.430 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:50:15.427+0000 7feb45cde700 1 -- 172.21.15.184:0/3319877681 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb401035a0 msgr2=0x7feb40199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:50:15.430 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:50:15.427+0000 7feb45cde700 1 --2- 172.21.15.184:0/3319877681 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb401035a0 0x7feb40199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:15.430 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:50:15.427+0000 7feb45cde700 1 -- 172.21.15.184:0/3319877681 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feb40105140 msgr2=0x7feb401a0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:50:15.430 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:50:15.427+0000 7feb45cde700 1 --2- 172.21.15.184:0/3319877681 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feb40105140 0x7feb401a0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:15.431 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:50:15.427+0000 7feb45cde700 1 -- 172.21.15.184:0/3319877681 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb401a1490 con 0x7feb401047a0 2024-05-21T21:50:15.431 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:50:15.427+0000 7feb464df700 1 --2- 172.21.15.184:0/3319877681 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb401035a0 0x7feb40199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T21:50:15.431 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:50:15.427+0000 7feb45cde700 1 --2- 172.21.15.184:0/3319877681 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feb401047a0 0x7feb40199d70 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7feb3000b560 tx=0x7feb3000b920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:50:15.431 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:50:15.427+0000 7feb377fe700 1 -- 172.21.15.184:0/3319877681 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb30015020 con 0x7feb401047a0 2024-05-21T21:50:15.431 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:50:15.427+0000 7feb377fe700 1 -- 172.21.15.184:0/3319877681 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb30011ce0 con 0x7feb401047a0 2024-05-21T21:50:15.431 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:50:15.427+0000 7feb377fe700 1 -- 172.21.15.184:0/3319877681 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb30010da0 con 0x7feb401047a0 2024-05-21T21:50:15.431 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:50:15.427+0000 7feb48743700 1 -- 172.21.15.184:0/3319877681 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb401a1780 con 0x7feb401047a0 2024-05-21T21:50:15.431 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:50:15.427+0000 7feb48743700 1 -- 172.21.15.184:0/3319877681 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb401a1d40 con 0x7feb401047a0 2024-05-21T21:50:15.431 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:50:15.428+0000 7feb48743700 1 -- 172.21.15.184:0/3319877681 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb4006a230 con 0x7feb401047a0 2024-05-21T21:50:15.432 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:50:15.429+0000 7feb377fe700 1 -- 172.21.15.184:0/3319877681 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7feb30018020 con 0x7feb401047a0 2024-05-21T21:50:15.432 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:50:15.430+0000 7feb377fe700 1 --2- 172.21.15.184:0/3319877681 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7feb2c0b2880 0x7feb2c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:50:15.432 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:50:15.430+0000 7feb377fe700 1 -- 172.21.15.184:0/3319877681 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7feb300e1b50 con 0x7feb401047a0 2024-05-21T21:50:15.432 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:50:15.431+0000 7feb464df700 1 --2- 172.21.15.184:0/3319877681 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7feb2c0b2880 0x7feb2c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:50:15.432 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:50:15.431+0000 7feb464df700 1 --2- 172.21.15.184:0/3319877681 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7feb2c0b2880 0x7feb2c0b4d40 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7feb3c0038a0 tx=0x7feb3c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:50:15.434 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:50:15.434+0000 7feb377fe700 1 -- 172.21.15.184:0/3319877681 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7feb3001f020 con 0x7feb401047a0 2024-05-21T21:50:15.649 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:50:15.648+0000 7feb48743700 1 -- 172.21.15.184:0/3319877681 --> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi151=a;smithi151=d;smithi151=g;smithi151=j;smithi164=b;smithi164=e;smithi164=h;smithi164=k;smithi184=c;smithi184=f;smithi184=i;smithi184=l", "target": ["mon-mgr", ""]}) -- 0x7feb40063a80 con 0x7feb2c0b2880 2024-05-21T21:50:15.666 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:50:15.666+0000 7feb377fe700 1 -- 172.21.15.184:0/3319877681 <== mgr.14152 v2:172.21.15.151:6800/2796622483 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7feb40063a80 con 0x7feb2c0b2880 2024-05-21T21:50:15.667 INFO:teuthology.orchestra.run.smithi184.stdout:Scheduled mds.all update... 2024-05-21T21:50:15.669 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:50:15.668+0000 7feb48743700 1 -- 172.21.15.184:0/3319877681 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7feb2c0b2880 msgr2=0x7feb2c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:50:15.669 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:50:15.668+0000 7feb48743700 1 --2- 172.21.15.184:0/3319877681 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7feb2c0b2880 0x7feb2c0b4d40 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7feb3c0038a0 tx=0x7feb3c005fd0 comp rx=0 tx=0).stop 2024-05-21T21:50:15.669 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:50:15.668+0000 7feb48743700 1 -- 172.21.15.184:0/3319877681 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feb401047a0 msgr2=0x7feb40199d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:50:15.669 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:50:15.668+0000 7feb48743700 1 --2- 172.21.15.184:0/3319877681 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feb401047a0 0x7feb40199d70 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7feb3000b560 tx=0x7feb3000b920 comp rx=0 tx=0).stop 2024-05-21T21:50:15.669 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:50:15.668+0000 7feb48743700 1 -- 172.21.15.184:0/3319877681 shutdown_connections 2024-05-21T21:50:15.669 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:50:15.668+0000 7feb48743700 1 --2- 172.21.15.184:0/3319877681 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7feb2c0b2880 0x7feb2c0b4d40 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:15.669 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:50:15.668+0000 7feb48743700 1 --2- 172.21.15.184:0/3319877681 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb401035a0 0x7feb40199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:15.669 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:50:15.668+0000 7feb48743700 1 --2- 172.21.15.184:0/3319877681 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feb40105140 0x7feb401a0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:15.669 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:50:15.668+0000 7feb48743700 1 --2- 172.21.15.184:0/3319877681 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feb401047a0 0x7feb40199d70 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:15.670 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:50:15.669+0000 7feb48743700 1 -- 172.21.15.184:0/3319877681 >> 172.21.15.184:0/3319877681 conn(0x7feb400feb40 msgr2=0x7feb40107bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:50:15.670 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:50:15.669+0000 7feb48743700 1 -- 172.21.15.184:0/3319877681 shutdown_connections 2024-05-21T21:50:15.670 INFO:teuthology.orchestra.run.smithi184.stderr:2024-05-21T21:50:15.669+0000 7feb48743700 1 -- 172.21.15.184:0/3319877681 wait complete. 2024-05-21T21:50:16.280 DEBUG:teuthology.orchestra.run.smithi151:mds.a> sudo journalctl -f -n 0 -u ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@mds.a.service 2024-05-21T21:50:16.287 DEBUG:teuthology.orchestra.run.smithi151:mds.d> sudo journalctl -f -n 0 -u ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@mds.d.service 2024-05-21T21:50:16.338 DEBUG:teuthology.orchestra.run.smithi151:mds.g> sudo journalctl -f -n 0 -u ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@mds.g.service 2024-05-21T21:50:16.385 DEBUG:teuthology.orchestra.run.smithi151:mds.j> sudo journalctl -f -n 0 -u ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@mds.j.service 2024-05-21T21:50:16.430 DEBUG:teuthology.orchestra.run.smithi164:mds.b> sudo journalctl -f -n 0 -u ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@mds.b.service 2024-05-21T21:50:16.432 DEBUG:teuthology.orchestra.run.smithi164:mds.e> sudo journalctl -f -n 0 -u ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@mds.e.service 2024-05-21T21:50:16.475 DEBUG:teuthology.orchestra.run.smithi164:mds.h> sudo journalctl -f -n 0 -u ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@mds.h.service 2024-05-21T21:50:16.518 DEBUG:teuthology.orchestra.run.smithi164:mds.k> sudo journalctl -f -n 0 -u ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@mds.k.service 2024-05-21T21:50:16.562 DEBUG:teuthology.orchestra.run.smithi184:mds.c> sudo journalctl -f -n 0 -u ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@mds.c.service 2024-05-21T21:50:16.564 DEBUG:teuthology.orchestra.run.smithi184:mds.f> sudo journalctl -f -n 0 -u ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@mds.f.service 2024-05-21T21:50:16.607 DEBUG:teuthology.orchestra.run.smithi184:mds.i> sudo journalctl -f -n 0 -u ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@mds.i.service 2024-05-21T21:50:16.650 DEBUG:teuthology.orchestra.run.smithi184:mds.l> sudo journalctl -f -n 0 -u ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@mds.l.service 2024-05-21T21:50:16.694 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-05-21T21:50:16.694 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-21T21:50:16.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:16 smithi164 ceph-mon[36269]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:50:16.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:16 smithi164 ceph-mon[36269]: from='client.24424 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi151=a;smithi151=d;smithi151=g;smithi151=j;smithi164=b;smithi164=e;smithi164=h;smithi164=k;smithi184=c;smithi184=f;smithi184=i;smithi184=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:50:16.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:16 smithi164 ceph-mon[36269]: Saving service mds.all spec with placement smithi151=a;smithi151=d;smithi151=g;smithi151=j;smithi164=b;smithi164=e;smithi164=h;smithi164=k;smithi184=c;smithi184=f;smithi184=i;smithi184=l;count:12 2024-05-21T21:50:16.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:16 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:16.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:16 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:50:16.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:16 smithi151 ceph-mon[31623]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:50:16.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:16 smithi151 ceph-mon[31623]: from='client.24424 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi151=a;smithi151=d;smithi151=g;smithi151=j;smithi164=b;smithi164=e;smithi164=h;smithi164=k;smithi184=c;smithi184=f;smithi184=i;smithi184=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:50:16.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:16 smithi151 ceph-mon[31623]: Saving service mds.all spec with placement smithi151=a;smithi151=d;smithi151=g;smithi151=j;smithi164=b;smithi164=e;smithi164=h;smithi164=k;smithi184=c;smithi184=f;smithi184=i;smithi184=l;count:12 2024-05-21T21:50:16.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:16 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:16.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:16 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:50:16.985 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:16 smithi184 ceph-mon[36942]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:50:16.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:16 smithi184 ceph-mon[36942]: from='client.24424 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi151=a;smithi151=d;smithi151=g;smithi151=j;smithi164=b;smithi164=e;smithi164=h;smithi164=k;smithi184=c;smithi184=f;smithi184=i;smithi184=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:50:16.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:16 smithi184 ceph-mon[36942]: Saving service mds.all spec with placement smithi151=a;smithi151=d;smithi151=g;smithi151=j;smithi164=b;smithi164=e;smithi164=h;smithi164=k;smithi184=c;smithi184=f;smithi184=i;smithi184=l;count:12 2024-05-21T21:50:16.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:16 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:16.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:16 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:50:17.129 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:50:17.779 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:17 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:50:17.780 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:17 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:50:17.780 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:17 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:17.780 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:17 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T21:50:17.780 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:17 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T21:50:17.780 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:17 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:50:17.780 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:17 smithi184 ceph-mon[36942]: Deploying daemon mds.l on smithi184 2024-05-21T21:50:17.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:17 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:50:17.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:17 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:50:17.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:17 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:17.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:17 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T21:50:17.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:17 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T21:50:17.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:17 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:50:17.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:17 smithi151 ceph-mon[31623]: Deploying daemon mds.l on smithi184 2024-05-21T21:50:18.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:17 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:50:18.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:17 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:50:18.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:17 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:18.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:17 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T21:50:18.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:17 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T21:50:18.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:17 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:50:18.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:17 smithi164 ceph-mon[36269]: Deploying daemon mds.l on smithi184 2024-05-21T21:50:19.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:19.164+0000 7f3ce9400700 1 -- 172.21.15.151:0/1449164327 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3ce4102e00 msgr2=0x7f3ce410f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:50:19.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:19.164+0000 7f3ce9400700 1 --2- 172.21.15.151:0/1449164327 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3ce4102e00 0x7f3ce410f2f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc00b0a0 tx=0x7f3cdc00b3b0 comp rx=0 tx=0).stop 2024-05-21T21:50:19.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:19.164+0000 7f3ce9400700 1 -- 172.21.15.151:0/1449164327 shutdown_connections 2024-05-21T21:50:19.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:19.164+0000 7f3ce9400700 1 --2- 172.21.15.151:0/1449164327 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3ce4108520 0x7f3ce4108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:19.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:19.164+0000 7f3ce9400700 1 --2- 172.21.15.151:0/1449164327 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3ce4102e00 0x7f3ce410f2f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:19.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:19.164+0000 7f3ce9400700 1 --2- 172.21.15.151:0/1449164327 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3ce4102440 0x7f3ce41028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:19.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:19.164+0000 7f3ce9400700 1 -- 172.21.15.151:0/1449164327 >> 172.21.15.151:0/1449164327 conn(0x7f3ce40793f0 msgr2=0x7f3ce4079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:50:19.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:19.164+0000 7f3ce9400700 1 -- 172.21.15.151:0/1449164327 shutdown_connections 2024-05-21T21:50:19.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:19.164+0000 7f3ce9400700 1 -- 172.21.15.151:0/1449164327 wait complete. 2024-05-21T21:50:19.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:19.165+0000 7f3ce9400700 1 Processor -- start 2024-05-21T21:50:19.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:19.166+0000 7f3ce9400700 1 -- start start 2024-05-21T21:50:19.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:19.166+0000 7f3ce9400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3ce4102440 0x7f3ce4104080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:50:19.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:19.166+0000 7f3ce9400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3ce4102e00 0x7f3ce41045c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:50:19.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:19.166+0000 7f3ce9400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3ce4108520 0x7f3ce4104b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:50:19.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:19.166+0000 7f3ce9400700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3ce41125d0 con 0x7f3ce4102e00 2024-05-21T21:50:19.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:19.166+0000 7f3ce9400700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3ce4112450 con 0x7f3ce4108520 2024-05-21T21:50:19.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:19.166+0000 7f3ce9400700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3ce4112750 con 0x7f3ce4102440 2024-05-21T21:50:19.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:19.166+0000 7f3ce1e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3ce4102e00 0x7f3ce41045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:50:19.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:19.166+0000 7f3ce2800700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3ce4102440 0x7f3ce4104080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:50:19.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:19.166+0000 7f3ce1e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3ce4102e00 0x7f3ce41045c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:58830/0 (socket says 172.21.15.151:58830) 2024-05-21T21:50:19.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:19.166+0000 7f3ce2800700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3ce4102440 0x7f3ce4104080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:54534/0 (socket says 172.21.15.151:54534) 2024-05-21T21:50:19.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:19.166+0000 7f3ce1e00700 1 -- 172.21.15.151:0/2415903888 learned_addr learned my addr 172.21.15.151:0/2415903888 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:50:19.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:19.166+0000 7f3ce3200700 1 --2- 172.21.15.151:0/2415903888 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3ce4108520 0x7f3ce4104b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:50:19.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:19.167+0000 7f3ce2800700 1 -- 172.21.15.151:0/2415903888 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3ce4108520 msgr2=0x7f3ce4104b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:50:19.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:19.167+0000 7f3ce2800700 1 --2- 172.21.15.151:0/2415903888 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3ce4108520 0x7f3ce4104b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:19.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:19.167+0000 7f3ce2800700 1 -- 172.21.15.151:0/2415903888 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3ce4102e00 msgr2=0x7f3ce41045c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:50:19.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:19.167+0000 7f3ce2800700 1 --2- 172.21.15.151:0/2415903888 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3ce4102e00 0x7f3ce41045c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:19.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:19.167+0000 7f3ce2800700 1 -- 172.21.15.151:0/2415903888 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ce41a80e0 con 0x7f3ce4102440 2024-05-21T21:50:19.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:19.167+0000 7f3ce1e00700 1 --2- 172.21.15.151:0/2415903888 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3ce4102e00 0x7f3ce41045c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:50:19.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:19.167+0000 7f3ce2800700 1 --2- 172.21.15.151:0/2415903888 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3ce4102440 0x7f3ce4104080 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3cd400e9f0 tx=0x7f3cd400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:50:19.170 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:18 smithi151 ceph-mon[31623]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:50:19.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:19.168+0000 7f3cdb400700 1 -- 172.21.15.151:0/2415903888 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd400c9c0 con 0x7f3ce4102440 2024-05-21T21:50:19.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:19.168+0000 7f3cdb400700 1 -- 172.21.15.151:0/2415903888 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3cd400cb60 con 0x7f3ce4102440 2024-05-21T21:50:19.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:19.168+0000 7f3cdb400700 1 -- 172.21.15.151:0/2415903888 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd4018680 con 0x7f3ce4102440 2024-05-21T21:50:19.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:19.168+0000 7f3ce9400700 1 -- 172.21.15.151:0/2415903888 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ce41a83d0 con 0x7f3ce4102440 2024-05-21T21:50:19.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:19.168+0000 7f3ce9400700 1 -- 172.21.15.151:0/2415903888 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ce41a8960 con 0x7f3ce4102440 2024-05-21T21:50:19.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:19.169+0000 7f3ce9400700 1 -- 172.21.15.151:0/2415903888 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ce406a230 con 0x7f3ce4102440 2024-05-21T21:50:19.172 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:19.171+0000 7f3cdb400700 1 -- 172.21.15.151:0/2415903888 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f3cd4010430 con 0x7f3ce4102440 2024-05-21T21:50:19.174 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:19.173+0000 7f3cdb400700 1 --2- 172.21.15.151:0/2415903888 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f3cc80b2900 0x7f3cc80b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:50:19.174 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:19.173+0000 7f3cdb400700 1 -- 172.21.15.151:0/2415903888 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f3cd40e1e50 con 0x7f3ce4102440 2024-05-21T21:50:19.174 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:19.173+0000 7f3ce1e00700 1 --2- 172.21.15.151:0/2415903888 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f3cc80b2900 0x7f3cc80b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:50:19.174 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:19.174+0000 7f3ce1e00700 1 --2- 172.21.15.151:0/2415903888 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f3cc80b2900 0x7f3cc80b4dc0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3ce4079b40 tx=0x7f3cd0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:50:19.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:19.176+0000 7f3cdb400700 1 -- 172.21.15.151:0/2415903888 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3cd40a9f70 con 0x7f3ce4102440 2024-05-21T21:50:19.189 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:18 smithi184 ceph-mon[36942]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:50:19.189 INFO:journalctl@ceph.mds.l.smithi184.stdout:May 21 21:50:18 smithi184 systemd[1]: Starting Ceph mds.l for c8c7a2e6-17ba-11ef-bc9a-c7b262605968... 2024-05-21T21:50:19.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:18 smithi164 ceph-mon[36269]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:50:19.440 INFO:journalctl@ceph.mds.l.smithi184.stdout:May 21 21:50:19 smithi184 podman[66892]: 2024-05-21 21:50:19.091866313 +0000 UTC m=+0.017289716 image pull d6bc8f31cbc1a7165ca248fe9ba0b06ea14b30f46e1ea33ced1a5238e247af1f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28 2024-05-21T21:50:19.440 INFO:journalctl@ceph.mds.l.smithi184.stdout:May 21 21:50:19 smithi184 podman[66892]: 2024-05-21 21:50:19.319968181 +0000 UTC m=+0.245391598 container create 75cdbe32c0e3c375f95eec372eec98ad8d33d7530451d3986c3502918b941f42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-l, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True) 2024-05-21T21:50:19.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:19.458+0000 7f3ce9400700 1 -- 172.21.15.151:0/2415903888 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3ce402e560 con 0x7f3ce4102440 2024-05-21T21:50:19.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:19.458+0000 7f3cdb400700 1 -- 172.21.15.151:0/2415903888 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7f3cd40a9870 con 0x7f3ce4102440 2024-05-21T21:50:19.460 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T21:50:19.460 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-05-21T21:50:19.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:19.462+0000 7f3ce9400700 1 -- 172.21.15.151:0/2415903888 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f3cc80b2900 msgr2=0x7f3cc80b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:50:19.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:19.462+0000 7f3ce9400700 1 --2- 172.21.15.151:0/2415903888 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f3cc80b2900 0x7f3cc80b4dc0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3ce4079b40 tx=0x7f3cd0009260 comp rx=0 tx=0).stop 2024-05-21T21:50:19.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:19.462+0000 7f3ce9400700 1 -- 172.21.15.151:0/2415903888 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3ce4102440 msgr2=0x7f3ce4104080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:50:19.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:19.462+0000 7f3ce9400700 1 --2- 172.21.15.151:0/2415903888 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3ce4102440 0x7f3ce4104080 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3cd400e9f0 tx=0x7f3cd400ed00 comp rx=0 tx=0).stop 2024-05-21T21:50:19.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:19.462+0000 7f3ce9400700 1 -- 172.21.15.151:0/2415903888 shutdown_connections 2024-05-21T21:50:19.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:19.462+0000 7f3ce9400700 1 --2- 172.21.15.151:0/2415903888 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f3cc80b2900 0x7f3cc80b4dc0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:19.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:19.462+0000 7f3ce9400700 1 --2- 172.21.15.151:0/2415903888 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3ce4108520 0x7f3ce4104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:19.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:19.462+0000 7f3ce9400700 1 --2- 172.21.15.151:0/2415903888 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3ce4102440 0x7f3ce4104080 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:19.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:19.462+0000 7f3ce9400700 1 --2- 172.21.15.151:0/2415903888 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3ce4102e00 0x7f3ce41045c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:19.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:19.462+0000 7f3ce9400700 1 -- 172.21.15.151:0/2415903888 >> 172.21.15.151:0/2415903888 conn(0x7f3ce40793f0 msgr2=0x7f3ce406b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:50:19.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:19.462+0000 7f3ce9400700 1 -- 172.21.15.151:0/2415903888 shutdown_connections 2024-05-21T21:50:19.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:19.462+0000 7f3ce9400700 1 -- 172.21.15.151:0/2415903888 wait complete. 2024-05-21T21:50:19.465 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 1 2024-05-21T21:50:19.929 INFO:journalctl@ceph.mds.l.smithi184.stdout:May 21 21:50:19 smithi184 podman[66892]: 2024-05-21 21:50:19.556184538 +0000 UTC m=+0.481607936 container init 75cdbe32c0e3c375f95eec372eec98ad8d33d7530451d3986c3502918b941f42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-l, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea) 2024-05-21T21:50:19.929 INFO:journalctl@ceph.mds.l.smithi184.stdout:May 21 21:50:19 smithi184 podman[66892]: 2024-05-21 21:50:19.558722387 +0000 UTC m=+0.484145781 container start 75cdbe32c0e3c375f95eec372eec98ad8d33d7530451d3986c3502918b941f42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-l, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-21T21:50:19.929 INFO:journalctl@ceph.mds.l.smithi184.stdout:May 21 21:50:19 smithi184 ceph-mds[66911]: -- 172.21.15.184:0/4022398928 <== mon.1 v2:172.21.15.184:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563c36ef3a40 con 0x563c36ec6c00 2024-05-21T21:50:19.929 INFO:journalctl@ceph.mds.l.smithi184.stdout:May 21 21:50:19 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-l[66907]: starting mds.l at 2024-05-21T21:50:19.929 INFO:journalctl@ceph.mds.l.smithi184.stdout:May 21 21:50:19 smithi184 bash[66892]: 75cdbe32c0e3c375f95eec372eec98ad8d33d7530451d3986c3502918b941f42 2024-05-21T21:50:19.929 INFO:journalctl@ceph.mds.l.smithi184.stdout:May 21 21:50:19 smithi184 systemd[1]: Started Ceph mds.l for c8c7a2e6-17ba-11ef-bc9a-c7b262605968. 2024-05-21T21:50:20.204 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:19 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/2415903888' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T21:50:20.204 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:19 smithi151 ceph-mon[31623]: mds.? [v2:172.21.15.184:6832/2026266114,v1:172.21.15.184:6833/2026266114] up:boot 2024-05-21T21:50:20.204 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:19 smithi151 ceph-mon[31623]: fsmap 1 up:standby 2024-05-21T21:50:20.204 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:19 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-21T21:50:20.204 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:19 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-21T21:50:20.204 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:19 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:20.204 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:19 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:20.205 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:19 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:20.205 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:19 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T21:50:20.205 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:19 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T21:50:20.205 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:19 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:50:20.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:19 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/2415903888' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T21:50:20.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:19 smithi184 ceph-mon[36942]: mds.? [v2:172.21.15.184:6832/2026266114,v1:172.21.15.184:6833/2026266114] up:boot 2024-05-21T21:50:20.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:19 smithi184 ceph-mon[36942]: fsmap 1 up:standby 2024-05-21T21:50:20.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:19 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-21T21:50:20.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:19 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-21T21:50:20.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:19 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:20.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:19 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:20.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:19 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:20.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:19 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T21:50:20.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:19 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T21:50:20.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:19 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:50:20.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:19 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/2415903888' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T21:50:20.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:19 smithi164 ceph-mon[36269]: mds.? [v2:172.21.15.184:6832/2026266114,v1:172.21.15.184:6833/2026266114] up:boot 2024-05-21T21:50:20.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:19 smithi164 ceph-mon[36269]: fsmap 1 up:standby 2024-05-21T21:50:20.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:19 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-21T21:50:20.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:19 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-21T21:50:20.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:19 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:20.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:19 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:20.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:19 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:20.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:19 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T21:50:20.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:19 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T21:50:20.287 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:19 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:50:21.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:20 smithi151 ceph-mon[31623]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:50:21.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:20 smithi151 ceph-mon[31623]: Deploying daemon mds.k on smithi164 2024-05-21T21:50:21.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:20 smithi184 ceph-mon[36942]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:50:21.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:20 smithi184 ceph-mon[36942]: Deploying daemon mds.k on smithi164 2024-05-21T21:50:21.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:20 smithi164 ceph-mon[36269]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:50:21.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:20 smithi164 ceph-mon[36269]: Deploying daemon mds.k on smithi164 2024-05-21T21:50:22.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:21 smithi164 ceph-mon[36269]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:50:22.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:21 smithi151 ceph-mon[31623]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:50:22.235 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:21 smithi184 ceph-mon[36942]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:50:22.267 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-21T21:50:22.461 INFO:journalctl@ceph.mds.k.smithi164.stdout:May 21 21:50:22 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-k[65312]: starting mds.k at 2024-05-21T21:50:22.710 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:50:22.786 INFO:journalctl@ceph.mds.k.smithi164.stdout:May 21 21:50:22 smithi164 bash[65297]: 7865d2590247b9bd0a85bcf77fc76cf86eff856c6fdd1e942dcbb03e8f93977c 2024-05-21T21:50:22.786 INFO:journalctl@ceph.mds.k.smithi164.stdout:May 21 21:50:22 smithi164 systemd[1]: Started Ceph mds.k for c8c7a2e6-17ba-11ef-bc9a-c7b262605968. 2024-05-21T21:50:23.775 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:23 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:23.775 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:23 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:23.775 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:23 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:23.775 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:23 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T21:50:23.776 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:23 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T21:50:23.776 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:23 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:50:23.776 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:23 smithi164 ceph-mon[36269]: Deploying daemon mds.j on smithi151 2024-05-21T21:50:23.776 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:23 smithi164 ceph-mon[36269]: mds.? [v2:172.21.15.164:6832/2467900844,v1:172.21.15.164:6833/2467900844] up:boot 2024-05-21T21:50:23.776 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:23 smithi164 ceph-mon[36269]: fsmap 2 up:standby 2024-05-21T21:50:23.776 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:23 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-21T21:50:23.865 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:23 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:23.865 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:23 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:23.865 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:23 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:23.866 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:23 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T21:50:23.866 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:23 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T21:50:23.866 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:23 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:50:23.866 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:23 smithi151 ceph-mon[31623]: Deploying daemon mds.j on smithi151 2024-05-21T21:50:23.866 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:23 smithi151 ceph-mon[31623]: mds.? [v2:172.21.15.164:6832/2467900844,v1:172.21.15.164:6833/2467900844] up:boot 2024-05-21T21:50:23.866 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:23 smithi151 ceph-mon[31623]: fsmap 2 up:standby 2024-05-21T21:50:23.866 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:23 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-21T21:50:23.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:23 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:23.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:23 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:23.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:23 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:23.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:23 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T21:50:23.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:23 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T21:50:23.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:23 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:50:23.987 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:23 smithi184 ceph-mon[36942]: Deploying daemon mds.j on smithi151 2024-05-21T21:50:23.987 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:23 smithi184 ceph-mon[36942]: mds.? [v2:172.21.15.164:6832/2467900844,v1:172.21.15.164:6833/2467900844] up:boot 2024-05-21T21:50:23.987 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:23 smithi184 ceph-mon[36942]: fsmap 2 up:standby 2024-05-21T21:50:23.987 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:23 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-21T21:50:24.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:24 smithi164 ceph-mon[36269]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:50:24.918 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:24 smithi151 ceph-mon[31623]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:50:24.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:24 smithi184 ceph-mon[36942]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:50:26.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:26.036+0000 7f93f2a00700 1 -- 172.21.15.151:0/1078674142 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f93ec06b220 msgr2=0x7f93ec06b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:50:26.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:26.036+0000 7f93f2a00700 1 --2- 172.21.15.151:0/1078674142 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f93ec06b220 0x7f93ec06b6a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f93d4009990 tx=0x7f93d4009ca0 comp rx=0 tx=0).stop 2024-05-21T21:50:26.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:26.037+0000 7f93f2a00700 1 -- 172.21.15.151:0/1078674142 shutdown_connections 2024-05-21T21:50:26.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:26.037+0000 7f93f2a00700 1 --2- 172.21.15.151:0/1078674142 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93ec1095b0 0x7f93ec109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:26.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:26.037+0000 7f93f2a00700 1 --2- 172.21.15.151:0/1078674142 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f93ec06bbe0 0x7f93ec1103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:26.040 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:26.037+0000 7f93f2a00700 1 --2- 172.21.15.151:0/1078674142 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f93ec06b220 0x7f93ec06b6a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:26.040 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:26.037+0000 7f93f2a00700 1 -- 172.21.15.151:0/1078674142 >> 172.21.15.151:0/1078674142 conn(0x7f93ec0793f0 msgr2=0x7f93ec079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:50:26.040 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:26.037+0000 7f93f2a00700 1 -- 172.21.15.151:0/1078674142 shutdown_connections 2024-05-21T21:50:26.040 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:26.037+0000 7f93f2a00700 1 -- 172.21.15.151:0/1078674142 wait complete. 2024-05-21T21:50:26.040 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:26.039+0000 7f93f2a00700 1 Processor -- start 2024-05-21T21:50:26.040 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:26.039+0000 7f93f2a00700 1 -- start start 2024-05-21T21:50:26.041 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:26.039+0000 7f93f2a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93ec06b220 0x7f93ec102510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:50:26.041 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:26.039+0000 7f93f2a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f93ec06bbe0 0x7f93ec102a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:50:26.041 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:26.039+0000 7f93f2a00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f93ec1095b0 0x7f93ec102f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:50:26.041 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:26.039+0000 7f93f2a00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f93ec113730 con 0x7f93ec06bbe0 2024-05-21T21:50:26.041 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:26.039+0000 7f93f2a00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f93ec1135b0 con 0x7f93ec06b220 2024-05-21T21:50:26.041 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:26.039+0000 7f93f2a00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f93ec1138b0 con 0x7f93ec1095b0 2024-05-21T21:50:26.042 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:26.040+0000 7f93e3e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f93ec06bbe0 0x7f93ec102a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:50:26.042 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:26.040+0000 7f93e3e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f93ec06bbe0 0x7f93ec102a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:35206/0 (socket says 172.21.15.151:35206) 2024-05-21T21:50:26.042 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:26.040+0000 7f93f0a00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f93ec1095b0 0x7f93ec102f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:50:26.042 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:26.040+0000 7f93e3e00700 1 -- 172.21.15.151:0/3953178792 learned_addr learned my addr 172.21.15.151:0/3953178792 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:50:26.042 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:26.040+0000 7f93ebe00700 1 --2- 172.21.15.151:0/3953178792 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93ec06b220 0x7f93ec102510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:50:26.042 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:26.040+0000 7f93f0a00700 1 -- 172.21.15.151:0/3953178792 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93ec06b220 msgr2=0x7f93ec102510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:50:26.042 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:26.040+0000 7f93f0a00700 1 --2- 172.21.15.151:0/3953178792 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93ec06b220 0x7f93ec102510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:26.042 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:26.040+0000 7f93f0a00700 1 -- 172.21.15.151:0/3953178792 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f93ec06bbe0 msgr2=0x7f93ec102a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:50:26.043 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:26.040+0000 7f93f0a00700 1 --2- 172.21.15.151:0/3953178792 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f93ec06bbe0 0x7f93ec102a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:26.043 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:26.040+0000 7f93f0a00700 1 -- 172.21.15.151:0/3953178792 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93ec1a81b0 con 0x7f93ec1095b0 2024-05-21T21:50:26.043 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:26.040+0000 7f93e3e00700 1 --2- 172.21.15.151:0/3953178792 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f93ec06bbe0 0x7f93ec102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:50:26.043 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:26.041+0000 7f93f0a00700 1 --2- 172.21.15.151:0/3953178792 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f93ec1095b0 0x7f93ec102f90 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f93e400efb0 tx=0x7f93e400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:50:26.043 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:26.041+0000 7f93e9600700 1 -- 172.21.15.151:0/3953178792 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e4003a00 con 0x7f93ec1095b0 2024-05-21T21:50:26.043 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:26.041+0000 7f93e9600700 1 -- 172.21.15.151:0/3953178792 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93e400ce50 con 0x7f93ec1095b0 2024-05-21T21:50:26.043 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:26.041+0000 7f93f2a00700 1 -- 172.21.15.151:0/3953178792 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93ec1a84a0 con 0x7f93ec1095b0 2024-05-21T21:50:26.043 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:26.041+0000 7f93f2a00700 1 -- 172.21.15.151:0/3953178792 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93ec1a8990 con 0x7f93ec1095b0 2024-05-21T21:50:26.044 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:26.041+0000 7f93e9600700 1 -- 172.21.15.151:0/3953178792 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e40047c0 con 0x7f93ec1095b0 2024-05-21T21:50:26.044 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:26.043+0000 7f93f2a00700 1 -- 172.21.15.151:0/3953178792 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93ec10ba90 con 0x7f93ec1095b0 2024-05-21T21:50:26.046 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:26.045+0000 7f93e9600700 1 -- 172.21.15.151:0/3953178792 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f93e4012050 con 0x7f93ec1095b0 2024-05-21T21:50:26.047 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:26.046+0000 7f93e9600700 1 --2- 172.21.15.151:0/3953178792 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f93d00b2880 0x7f93d00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:50:26.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:26.047+0000 7f93e9600700 1 -- 172.21.15.151:0/3953178792 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f93e40e16c0 con 0x7f93ec1095b0 2024-05-21T21:50:26.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:26.047+0000 7f93ebe00700 1 --2- 172.21.15.151:0/3953178792 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f93d00b2880 0x7f93d00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:50:26.049 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:26.048+0000 7f93ebe00700 1 --2- 172.21.15.151:0/3953178792 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f93d00b2880 0x7f93d00b4d40 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f93dc009be0 tx=0x7f93dc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:50:26.051 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:26.051+0000 7f93e9600700 1 -- 172.21.15.151:0/3953178792 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f93e4022060 con 0x7f93ec1095b0 2024-05-21T21:50:26.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:26.363+0000 7f93f2a00700 1 -- 172.21.15.151:0/3953178792 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f93ec10b470 con 0x7f93ec1095b0 2024-05-21T21:50:26.365 INFO:journalctl@ceph.mds.j.smithi151.stdout:May 21 21:50:26 smithi151 systemd[1]: Starting Ceph mds.j for c8c7a2e6-17ba-11ef-bc9a-c7b262605968... 2024-05-21T21:50:26.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:26.364+0000 7f93e9600700 1 -- 172.21.15.151:0/3953178792 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2224 (secure 0 0 0) 0x7f93e4010040 con 0x7f93ec1095b0 2024-05-21T21:50:26.366 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T21:50:26.366 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/2026266114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":2026266114},{"type":"v1","addr":"172.21.15.184:6833","nonce":2026266114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/2467900844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":2467900844},{"type":"v1","addr":"172.21.15.164:6833","nonce":2467900844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-05-21T21:50:26.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:26.367+0000 7f93f2a00700 1 -- 172.21.15.151:0/3953178792 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f93d00b2880 msgr2=0x7f93d00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:50:26.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:26.367+0000 7f93f2a00700 1 --2- 172.21.15.151:0/3953178792 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f93d00b2880 0x7f93d00b4d40 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f93dc009be0 tx=0x7f93dc009260 comp rx=0 tx=0).stop 2024-05-21T21:50:26.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:26.367+0000 7f93f2a00700 1 -- 172.21.15.151:0/3953178792 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f93ec1095b0 msgr2=0x7f93ec102f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:50:26.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:26.367+0000 7f93f2a00700 1 --2- 172.21.15.151:0/3953178792 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f93ec1095b0 0x7f93ec102f90 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f93e400efb0 tx=0x7f93e400c3a0 comp rx=0 tx=0).stop 2024-05-21T21:50:26.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:26.367+0000 7f93f2a00700 1 -- 172.21.15.151:0/3953178792 shutdown_connections 2024-05-21T21:50:26.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:26.367+0000 7f93f2a00700 1 --2- 172.21.15.151:0/3953178792 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f93d00b2880 0x7f93d00b4d40 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:26.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:26.367+0000 7f93f2a00700 1 --2- 172.21.15.151:0/3953178792 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93ec06b220 0x7f93ec102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:26.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:26.367+0000 7f93f2a00700 1 --2- 172.21.15.151:0/3953178792 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f93ec1095b0 0x7f93ec102f90 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:26.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:26.367+0000 7f93f2a00700 1 --2- 172.21.15.151:0/3953178792 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f93ec06bbe0 0x7f93ec102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:26.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:26.367+0000 7f93f2a00700 1 -- 172.21.15.151:0/3953178792 >> 172.21.15.151:0/3953178792 conn(0x7f93ec0793f0 msgr2=0x7f93ec101240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:50:26.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:26.367+0000 7f93f2a00700 1 -- 172.21.15.151:0/3953178792 shutdown_connections 2024-05-21T21:50:26.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:26.367+0000 7f93f2a00700 1 -- 172.21.15.151:0/3953178792 wait complete. 2024-05-21T21:50:26.371 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 3 2024-05-21T21:50:26.617 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:26 smithi151 ceph-mon[31623]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:50:26.617 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:26 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/3953178792' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T21:50:26.618 INFO:journalctl@ceph.mds.j.smithi151.stdout:May 21 21:50:26 smithi151 podman[69694]: 2024-05-21 21:50:26.342510815 +0000 UTC m=+0.036970033 image pull d6bc8f31cbc1a7165ca248fe9ba0b06ea14b30f46e1ea33ced1a5238e247af1f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28 2024-05-21T21:50:26.618 INFO:journalctl@ceph.mds.j.smithi151.stdout:May 21 21:50:26 smithi151 podman[69694]: 2024-05-21 21:50:26.467793574 +0000 UTC m=+0.162252819 container create d9eec75ffa5a3f1e595e2ba8321f3be8100eb7a90a9b6338f0b592ab524197a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-j, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea) 2024-05-21T21:50:26.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:26 smithi164 ceph-mon[36269]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:50:26.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:26 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/3953178792' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T21:50:26.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:26 smithi184 ceph-mon[36942]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:50:26.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:26 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/3953178792' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T21:50:27.703 INFO:journalctl@ceph.mds.j.smithi151.stdout:May 21 21:50:27 smithi151 podman[69694]: 2024-05-21 21:50:27.377306267 +0000 UTC m=+1.071765536 container init d9eec75ffa5a3f1e595e2ba8321f3be8100eb7a90a9b6338f0b592ab524197a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-j, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-05-21T21:50:27.703 INFO:journalctl@ceph.mds.j.smithi151.stdout:May 21 21:50:27 smithi151 podman[69694]: 2024-05-21 21:50:27.387959541 +0000 UTC m=+1.082418820 container start d9eec75ffa5a3f1e595e2ba8321f3be8100eb7a90a9b6338f0b592ab524197a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-j, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-21T21:50:27.703 INFO:journalctl@ceph.mds.j.smithi151.stdout:May 21 21:50:27 smithi151 ceph-mds[69729]: -- 172.21.15.151:0/3862737353 <== mon.1 v2:172.21.15.184:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a15aad7a40 con 0x55a15aaaa800 2024-05-21T21:50:27.703 INFO:journalctl@ceph.mds.j.smithi151.stdout:May 21 21:50:27 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-j[69722]: starting mds.j at 2024-05-21T21:50:27.704 INFO:journalctl@ceph.mds.j.smithi151.stdout:May 21 21:50:27 smithi151 bash[69694]: d9eec75ffa5a3f1e595e2ba8321f3be8100eb7a90a9b6338f0b592ab524197a6 2024-05-21T21:50:27.704 INFO:journalctl@ceph.mds.j.smithi151.stdout:May 21 21:50:27 smithi151 systemd[1]: Started Ceph mds.j for c8c7a2e6-17ba-11ef-bc9a-c7b262605968. 2024-05-21T21:50:28.822 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:28 smithi151 ceph-mon[31623]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:50:28.822 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:28 smithi151 ceph-mon[31623]: mds.? [v2:172.21.15.151:6834/158010529,v1:172.21.15.151:6835/158010529] up:boot 2024-05-21T21:50:28.822 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:28 smithi151 ceph-mon[31623]: fsmap 3 up:standby 2024-05-21T21:50:28.822 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:28 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-21T21:50:28.822 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:28 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:28.822 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:28 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:28.822 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:28 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:28.822 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:28 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T21:50:28.822 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:28 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T21:50:28.823 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:28 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:50:28.823 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:28 smithi151 ceph-mon[31623]: Deploying daemon mds.d on smithi151 2024-05-21T21:50:28.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:28 smithi184 ceph-mon[36942]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:50:28.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:28 smithi184 ceph-mon[36942]: mds.? [v2:172.21.15.151:6834/158010529,v1:172.21.15.151:6835/158010529] up:boot 2024-05-21T21:50:28.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:28 smithi184 ceph-mon[36942]: fsmap 3 up:standby 2024-05-21T21:50:28.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:28 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-21T21:50:28.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:28 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:28.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:28 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:28.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:28 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:28.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:28 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T21:50:28.987 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:28 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T21:50:28.987 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:28 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:50:28.987 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:28 smithi184 ceph-mon[36942]: Deploying daemon mds.d on smithi151 2024-05-21T21:50:29.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:28 smithi164 ceph-mon[36269]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:50:29.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:28 smithi164 ceph-mon[36269]: mds.? [v2:172.21.15.151:6834/158010529,v1:172.21.15.151:6835/158010529] up:boot 2024-05-21T21:50:29.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:28 smithi164 ceph-mon[36269]: fsmap 3 up:standby 2024-05-21T21:50:29.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:28 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-21T21:50:29.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:28 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:29.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:28 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:29.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:28 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:29.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:28 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T21:50:29.037 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:28 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T21:50:29.037 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:28 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:50:29.037 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:28 smithi164 ceph-mon[36269]: Deploying daemon mds.d on smithi151 2024-05-21T21:50:29.302 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-21T21:50:30.123 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:50:30.665 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:30 smithi151 ceph-mon[31623]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:50:30.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:30 smithi184 ceph-mon[36942]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:50:31.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:30 smithi164 ceph-mon[36269]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:50:31.831 INFO:journalctl@ceph.mds.d.smithi151.stdout:May 21 21:50:31 smithi151 podman[70102]: 2024-05-21 21:50:31.725997373 +0000 UTC m=+0.034402610 image pull d6bc8f31cbc1a7165ca248fe9ba0b06ea14b30f46e1ea33ced1a5238e247af1f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28 2024-05-21T21:50:31.832 INFO:journalctl@ceph.mds.d.smithi151.stdout:May 21 21:50:31 smithi151 podman[70102]: 2024-05-21 21:50:31.829930706 +0000 UTC m=+0.138335970 container create 3dfebcc084a3ba55d4ad88c1d1bd9fcbfc2c3a700368fd851fa4160809e2ddb7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-d, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-21T21:50:32.369 INFO:journalctl@ceph.mds.d.smithi151.stdout:May 21 21:50:32 smithi151 podman[70102]: 2024-05-21 21:50:32.313944409 +0000 UTC m=+0.622349689 container init 3dfebcc084a3ba55d4ad88c1d1bd9fcbfc2c3a700368fd851fa4160809e2ddb7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-d, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea) 2024-05-21T21:50:32.369 INFO:journalctl@ceph.mds.d.smithi151.stdout:May 21 21:50:32 smithi151 podman[70102]: 2024-05-21 21:50:32.324414258 +0000 UTC m=+0.632819541 container start 3dfebcc084a3ba55d4ad88c1d1bd9fcbfc2c3a700368fd851fa4160809e2ddb7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-21T21:50:32.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:32.691+0000 7f6056a00700 1 -- 172.21.15.151:0/1153783903 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f60501088f0 msgr2=0x7f6050110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:50:32.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:32.691+0000 7f6056a00700 1 --2- 172.21.15.151:0/1153783903 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f60501088f0 0x7f6050110470 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f604800b0d0 tx=0x7f604800b3e0 comp rx=0 tx=0).stop 2024-05-21T21:50:32.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:32.691+0000 7f6056a00700 1 -- 172.21.15.151:0/1153783903 shutdown_connections 2024-05-21T21:50:32.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:32.691+0000 7f6056a00700 1 --2- 172.21.15.151:0/1153783903 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f60501037f0 0x7f6050103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:32.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:32.691+0000 7f6056a00700 1 --2- 172.21.15.151:0/1153783903 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f60501088f0 0x7f6050110470 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:32.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:32.691+0000 7f6056a00700 1 --2- 172.21.15.151:0/1153783903 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f60501041a0 0x7f60501081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:32.693 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:32.691+0000 7f6056a00700 1 -- 172.21.15.151:0/1153783903 >> 172.21.15.151:0/1153783903 conn(0x7f605007a6a0 msgr2=0x7f605007aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:50:32.693 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:32.691+0000 7f6056a00700 1 -- 172.21.15.151:0/1153783903 shutdown_connections 2024-05-21T21:50:32.693 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:32.691+0000 7f6056a00700 1 -- 172.21.15.151:0/1153783903 wait complete. 2024-05-21T21:50:32.693 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:32 smithi151 ceph-mon[31623]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:50:32.693 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:32 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:32.693 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:32 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:32.693 INFO:journalctl@ceph.mds.d.smithi151.stdout:May 21 21:50:32 smithi151 ceph-mds[70143]: -- 172.21.15.151:0/1967361022 <== mon.1 v2:172.21.15.184:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5621475dda40 con 0x5621475b1000 2024-05-21T21:50:32.694 INFO:journalctl@ceph.mds.d.smithi151.stdout:May 21 21:50:32 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-d[70133]: starting mds.d at 2024-05-21T21:50:32.694 INFO:journalctl@ceph.mds.d.smithi151.stdout:May 21 21:50:32 smithi151 bash[70102]: 3dfebcc084a3ba55d4ad88c1d1bd9fcbfc2c3a700368fd851fa4160809e2ddb7 2024-05-21T21:50:32.694 INFO:journalctl@ceph.mds.d.smithi151.stdout:May 21 21:50:32 smithi151 systemd[1]: Started Ceph mds.d for c8c7a2e6-17ba-11ef-bc9a-c7b262605968. 2024-05-21T21:50:32.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:32.692+0000 7f6056a00700 1 Processor -- start 2024-05-21T21:50:32.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:32.692+0000 7f6056a00700 1 -- start start 2024-05-21T21:50:32.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:32.693+0000 7f6056a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f60501037f0 0x7f605019f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:50:32.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:32.693+0000 7f6056a00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f60501041a0 0x7f605019fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:50:32.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:32.693+0000 7f6056a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f60501088f0 0x7f60501999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:50:32.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:32.693+0000 7f6056a00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f60501136e0 con 0x7f60501088f0 2024-05-21T21:50:32.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:32.693+0000 7f6056a00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6050113560 con 0x7f60501037f0 2024-05-21T21:50:32.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:32.693+0000 7f6056a00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6050113860 con 0x7f60501041a0 2024-05-21T21:50:32.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:32.693+0000 7f604fe00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f60501037f0 0x7f605019f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:50:32.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:32.693+0000 7f6054a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f60501088f0 0x7f60501999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:50:32.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:32.693+0000 7f604fe00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f60501037f0 0x7f605019f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:36826/0 (socket says 172.21.15.151:36826) 2024-05-21T21:50:32.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:32.693+0000 7f604fe00700 1 -- 172.21.15.151:0/2654560448 learned_addr learned my addr 172.21.15.151:0/2654560448 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:50:32.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:32.694+0000 7f604fe00700 1 -- 172.21.15.151:0/2654560448 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f60501041a0 msgr2=0x7f605019fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:50:32.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:32.694+0000 7f604fe00700 1 --2- 172.21.15.151:0/2654560448 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f60501041a0 0x7f605019fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:32.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:32.694+0000 7f604fe00700 1 -- 172.21.15.151:0/2654560448 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f60501088f0 msgr2=0x7f60501999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:50:32.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:32.694+0000 7f604fe00700 1 --2- 172.21.15.151:0/2654560448 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f60501088f0 0x7f60501999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:32.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:32.694+0000 7f604fe00700 1 -- 172.21.15.151:0/2654560448 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f605019a100 con 0x7f60501037f0 2024-05-21T21:50:32.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:32.694+0000 7f604fe00700 1 --2- 172.21.15.151:0/2654560448 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f60501037f0 0x7f605019f7e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f604000e9a0 tx=0x7f604000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:50:32.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:32.694+0000 7f604cc00700 1 -- 172.21.15.151:0/2654560448 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f604000cbd0 con 0x7f60501037f0 2024-05-21T21:50:32.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:32.694+0000 7f604cc00700 1 -- 172.21.15.151:0/2654560448 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f604000cd70 con 0x7f60501037f0 2024-05-21T21:50:32.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:32.694+0000 7f6056a00700 1 -- 172.21.15.151:0/2654560448 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f605019a3f0 con 0x7f60501037f0 2024-05-21T21:50:32.697 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:32.694+0000 7f604cc00700 1 -- 172.21.15.151:0/2654560448 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6040018680 con 0x7f60501037f0 2024-05-21T21:50:32.697 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:32.695+0000 7f6056a00700 1 -- 172.21.15.151:0/2654560448 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60501a66b0 con 0x7f60501037f0 2024-05-21T21:50:32.697 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:32.696+0000 7f6056a00700 1 -- 172.21.15.151:0/2654560448 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f605006a230 con 0x7f60501037f0 2024-05-21T21:50:32.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:32.699+0000 7f604cc00700 1 -- 172.21.15.151:0/2654560448 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f6040018820 con 0x7f60501037f0 2024-05-21T21:50:32.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:32.700+0000 7f604cc00700 1 --2- 172.21.15.151:0/2654560448 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f60340b2900 0x7f60340b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:50:32.701 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:32.700+0000 7f604cc00700 1 -- 172.21.15.151:0/2654560448 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f60400e2000 con 0x7f60501037f0 2024-05-21T21:50:32.701 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:32.701+0000 7f604f400700 1 --2- 172.21.15.151:0/2654560448 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f60340b2900 0x7f60340b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:50:32.701 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:32.701+0000 7f604f400700 1 --2- 172.21.15.151:0/2654560448 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f60340b2900 0x7f60340b4dc0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6050108170 tx=0x7f603c009620 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:50:32.705 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:32.705+0000 7f604cc00700 1 -- 172.21.15.151:0/2654560448 <== mon.2 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f60400aa120 con 0x7f60501037f0 2024-05-21T21:50:32.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:32 smithi184 ceph-mon[36942]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:50:32.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:32 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:32.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:32 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:32.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:32.986+0000 7f6056a00700 1 -- 172.21.15.151:0/2654560448 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f605002e560 con 0x7f60501037f0 2024-05-21T21:50:32.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:32.987+0000 7f604cc00700 1 -- 172.21.15.151:0/2654560448 <== mon.2 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3877 (secure 0 0 0) 0x7f60400b21a0 con 0x7f60501037f0 2024-05-21T21:50:32.987 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T21:50:32.987 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14358,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/158010529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":158010529},{"type":"v1","addr":"172.21.15.151:6835","nonce":158010529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/2026266114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":2026266114},{"type":"v1","addr":"172.21.15.184:6833","nonce":2026266114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/2467900844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":2467900844},{"type":"v1","addr":"172.21.15.164:6833","nonce":2467900844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24461,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6837/3367640960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3367640960},{"type":"v1","addr":"172.21.15.151:6837","nonce":3367640960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[]} 2024-05-21T21:50:32.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:32.990+0000 7f6056a00700 1 -- 172.21.15.151:0/2654560448 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f60340b2900 msgr2=0x7f60340b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:50:32.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:32.990+0000 7f6056a00700 1 --2- 172.21.15.151:0/2654560448 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f60340b2900 0x7f60340b4dc0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6050108170 tx=0x7f603c009620 comp rx=0 tx=0).stop 2024-05-21T21:50:32.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:32.990+0000 7f6056a00700 1 -- 172.21.15.151:0/2654560448 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f60501037f0 msgr2=0x7f605019f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:50:32.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:32.990+0000 7f6056a00700 1 --2- 172.21.15.151:0/2654560448 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f60501037f0 0x7f605019f7e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f604000e9a0 tx=0x7f604000ecb0 comp rx=0 tx=0).stop 2024-05-21T21:50:32.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:32.990+0000 7f6056a00700 1 -- 172.21.15.151:0/2654560448 shutdown_connections 2024-05-21T21:50:32.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:32.990+0000 7f6056a00700 1 --2- 172.21.15.151:0/2654560448 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f60340b2900 0x7f60340b4dc0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:32.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:32.990+0000 7f6056a00700 1 --2- 172.21.15.151:0/2654560448 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f60501037f0 0x7f605019f7e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:32.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:32.990+0000 7f6056a00700 1 --2- 172.21.15.151:0/2654560448 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f60501088f0 0x7f60501999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:32.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:32.990+0000 7f6056a00700 1 --2- 172.21.15.151:0/2654560448 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f60501041a0 0x7f605019fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:32.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:32.990+0000 7f6056a00700 1 -- 172.21.15.151:0/2654560448 >> 172.21.15.151:0/2654560448 conn(0x7f605007a6a0 msgr2=0x7f60501015e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:50:32.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:32.990+0000 7f6056a00700 1 -- 172.21.15.151:0/2654560448 shutdown_connections 2024-05-21T21:50:32.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:32.990+0000 7f6056a00700 1 -- 172.21.15.151:0/2654560448 wait complete. 2024-05-21T21:50:32.991 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 5 2024-05-21T21:50:33.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:32 smithi164 ceph-mon[36269]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:50:33.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:32 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:33.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:32 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:33.678 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:33 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:33.678 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:33 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T21:50:33.678 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:33 smithi151 ceph-mon[31623]: mds.? [v2:172.21.15.151:6836/3367640960,v1:172.21.15.151:6837/3367640960] up:boot 2024-05-21T21:50:33.678 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:33 smithi151 ceph-mon[31623]: fsmap 4 up:standby 2024-05-21T21:50:33.678 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:33 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-21T21:50:33.679 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:33 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T21:50:33.679 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:33 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:50:33.679 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:33 smithi151 ceph-mon[31623]: Deploying daemon mds.e on smithi164 2024-05-21T21:50:33.679 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:33 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/2654560448' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T21:50:33.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:33 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:33.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:33 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T21:50:33.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:33 smithi164 ceph-mon[36269]: mds.? [v2:172.21.15.151:6836/3367640960,v1:172.21.15.151:6837/3367640960] up:boot 2024-05-21T21:50:33.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:33 smithi164 ceph-mon[36269]: fsmap 4 up:standby 2024-05-21T21:50:33.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:33 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-21T21:50:33.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:33 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T21:50:33.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:33 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:50:33.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:33 smithi164 ceph-mon[36269]: Deploying daemon mds.e on smithi164 2024-05-21T21:50:33.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:33 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/2654560448' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T21:50:33.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:33 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:33.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:33 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T21:50:33.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:33 smithi184 ceph-mon[36942]: mds.? [v2:172.21.15.151:6836/3367640960,v1:172.21.15.151:6837/3367640960] up:boot 2024-05-21T21:50:33.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:33 smithi184 ceph-mon[36942]: fsmap 4 up:standby 2024-05-21T21:50:33.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:33 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-21T21:50:33.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:33 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T21:50:33.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:33 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:50:33.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:33 smithi184 ceph-mon[36942]: Deploying daemon mds.e on smithi164 2024-05-21T21:50:33.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:33 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/2654560448' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T21:50:34.951 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:34 smithi164 ceph-mon[36269]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:50:34.952 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:34 smithi151 ceph-mon[31623]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:50:34.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:34 smithi184 ceph-mon[36942]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:50:35.218 INFO:journalctl@ceph.mds.e.smithi164.stdout:May 21 21:50:35 smithi164 podman[65608]: 2024-05-21 21:50:35.195049386 +0000 UTC m=+0.358103249 container start 0bfde453384df9e536c07290e67ae58ce0063add83907a05e05c3039ac7b7b76 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-e, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514) 2024-05-21T21:50:35.218 INFO:journalctl@ceph.mds.e.smithi164.stdout:May 21 21:50:35 smithi164 ceph-mds[65628]: -- 172.21.15.164:0/1170902656 <== mon.2 v2:172.21.15.164:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ca8fc9da40 con 0x55ca8fc71000 2024-05-21T21:50:35.496 INFO:journalctl@ceph.mds.e.smithi164.stdout:May 21 21:50:35 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-e[65623]: starting mds.e at 2024-05-21T21:50:35.496 INFO:journalctl@ceph.mds.e.smithi164.stdout:May 21 21:50:35 smithi164 bash[65608]: 0bfde453384df9e536c07290e67ae58ce0063add83907a05e05c3039ac7b7b76 2024-05-21T21:50:35.496 INFO:journalctl@ceph.mds.e.smithi164.stdout:May 21 21:50:35 smithi164 systemd[1]: Started Ceph mds.e for c8c7a2e6-17ba-11ef-bc9a-c7b262605968. 2024-05-21T21:50:35.740 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-21T21:50:36.202 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:50:36.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:36 smithi164 ceph-mon[36269]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:50:36.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:36 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:36.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:36 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:36.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:36 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:36.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:36 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T21:50:36.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:36 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T21:50:36.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:36 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:50:36.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:36 smithi164 ceph-mon[36269]: Deploying daemon mds.f on smithi184 2024-05-21T21:50:36.537 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:36 smithi164 ceph-mon[36269]: mds.? [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] up:boot 2024-05-21T21:50:36.537 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:36 smithi164 ceph-mon[36269]: fsmap 5 up:standby 2024-05-21T21:50:36.537 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:36 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-21T21:50:36.684 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:36 smithi151 ceph-mon[31623]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:50:36.684 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:36 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:36.684 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:36 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:36.684 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:36 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:36.684 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:36 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T21:50:36.684 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:36 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T21:50:36.685 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:36 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:50:36.685 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:36 smithi151 ceph-mon[31623]: Deploying daemon mds.f on smithi184 2024-05-21T21:50:36.685 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:36 smithi151 ceph-mon[31623]: mds.? [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] up:boot 2024-05-21T21:50:36.685 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:36 smithi151 ceph-mon[31623]: fsmap 5 up:standby 2024-05-21T21:50:36.685 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:36 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-21T21:50:36.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:36 smithi184 ceph-mon[36942]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:50:36.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:36 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:36.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:36 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:36.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:36 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:36.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:36 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T21:50:36.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:36 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T21:50:36.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:36 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:50:36.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:36 smithi184 ceph-mon[36942]: Deploying daemon mds.f on smithi184 2024-05-21T21:50:36.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:36 smithi184 ceph-mon[36942]: mds.? [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] up:boot 2024-05-21T21:50:36.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:36 smithi184 ceph-mon[36942]: fsmap 5 up:standby 2024-05-21T21:50:36.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:36 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-21T21:50:37.736 INFO:journalctl@ceph.mds.f.smithi184.stdout:May 21 21:50:37 smithi184 systemd[1]: Starting Ceph mds.f for c8c7a2e6-17ba-11ef-bc9a-c7b262605968... 2024-05-21T21:50:38.084 INFO:journalctl@ceph.mds.f.smithi184.stdout:May 21 21:50:37 smithi184 podman[67193]: 2024-05-21 21:50:37.721739511 +0000 UTC m=+0.016793684 image pull d6bc8f31cbc1a7165ca248fe9ba0b06ea14b30f46e1ea33ced1a5238e247af1f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28 2024-05-21T21:50:38.084 INFO:journalctl@ceph.mds.f.smithi184.stdout:May 21 21:50:37 smithi184 podman[67193]: 2024-05-21 21:50:37.831211663 +0000 UTC m=+0.126265821 container create e13fed9be1b512cee5c2dc5487666b6feb2ecf2193c40e0cc9bbc2e24309839e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-f, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-05-21T21:50:38.084 INFO:journalctl@ceph.mds.f.smithi184.stdout:May 21 21:50:38 smithi184 podman[67193]: 2024-05-21 21:50:38.058568671 +0000 UTC m=+0.353622832 container init e13fed9be1b512cee5c2dc5487666b6feb2ecf2193c40e0cc9bbc2e24309839e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-f, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux ) 2024-05-21T21:50:38.084 INFO:journalctl@ceph.mds.f.smithi184.stdout:May 21 21:50:38 smithi184 podman[67193]: 2024-05-21 21:50:38.0612021 +0000 UTC m=+0.356256260 container start e13fed9be1b512cee5c2dc5487666b6feb2ecf2193c40e0cc9bbc2e24309839e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-f, GIT_CLEAN=True, org.label-schema.build-date=20240514, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=) 2024-05-21T21:50:38.084 INFO:journalctl@ceph.mds.f.smithi184.stdout:May 21 21:50:38 smithi184 ceph-mds[67212]: -- 172.21.15.184:0/2889473530 <== mon.1 v2:172.21.15.184:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559774777a40 con 0x55977474ac00 2024-05-21T21:50:38.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:38.286+0000 7f8c24200700 1 -- 172.21.15.151:0/1539536144 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c1c06c560 msgr2=0x7f8c1c10fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:50:38.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:38.286+0000 7f8c24200700 1 --2- 172.21.15.151:0/1539536144 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c1c06c560 0x7f8c1c10fae0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8c080098f0 tx=0x7f8c08009c00 comp rx=0 tx=0).stop 2024-05-21T21:50:38.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:38.287+0000 7f8c24200700 1 -- 172.21.15.151:0/1539536144 shutdown_connections 2024-05-21T21:50:38.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:38.287+0000 7f8c24200700 1 --2- 172.21.15.151:0/1539536144 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8c1c110020 0x7f8c1c112410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:38.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:38.287+0000 7f8c24200700 1 --2- 172.21.15.151:0/1539536144 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8c1c06bc40 0x7f8c1c06c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:38.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:38.287+0000 7f8c24200700 1 --2- 172.21.15.151:0/1539536144 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c1c06c560 0x7f8c1c10fae0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:38.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:38.287+0000 7f8c24200700 1 -- 172.21.15.151:0/1539536144 >> 172.21.15.151:0/1539536144 conn(0x7f8c1c079480 msgr2=0x7f8c1c079890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:50:38.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:38.287+0000 7f8c24200700 1 -- 172.21.15.151:0/1539536144 shutdown_connections 2024-05-21T21:50:38.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:38.287+0000 7f8c24200700 1 -- 172.21.15.151:0/1539536144 wait complete. 2024-05-21T21:50:38.289 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:38.288+0000 7f8c24200700 1 Processor -- start 2024-05-21T21:50:38.289 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:38.288+0000 7f8c24200700 1 -- start start 2024-05-21T21:50:38.290 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:38.288+0000 7f8c24200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8c1c06bc40 0x7f8c1c19bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:50:38.290 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:38.288+0000 7f8c24200700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c1c06c560 0x7f8c1c19c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:50:38.290 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:38.288+0000 7f8c24200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8c1c110020 0x7f8c1c1a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:50:38.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:38.288+0000 7f8c24200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1c1159a0 con 0x7f8c1c06bc40 2024-05-21T21:50:38.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:38.288+0000 7f8c24200700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1c115820 con 0x7f8c1c110020 2024-05-21T21:50:38.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:38.288+0000 7f8c24200700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1c115b20 con 0x7f8c1c06c560 2024-05-21T21:50:38.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:38.289+0000 7f8c21800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8c1c06bc40 0x7f8c1c19bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:50:38.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:38.289+0000 7f8c21800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8c1c06bc40 0x7f8c1c19bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:60800/0 (socket says 172.21.15.151:60800) 2024-05-21T21:50:38.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:38.289+0000 7f8c21800700 1 -- 172.21.15.151:0/1827944752 learned_addr learned my addr 172.21.15.151:0/1827944752 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:50:38.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:38.289+0000 7f8c20e00700 1 --2- 172.21.15.151:0/1827944752 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c1c06c560 0x7f8c1c19c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:50:38.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:38.289+0000 7f8c22200700 1 --2- 172.21.15.151:0/1827944752 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8c1c110020 0x7f8c1c1a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:50:38.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:38.289+0000 7f8c20e00700 1 -- 172.21.15.151:0/1827944752 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8c1c110020 msgr2=0x7f8c1c1a07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:50:38.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:38.289+0000 7f8c20e00700 1 --2- 172.21.15.151:0/1827944752 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8c1c110020 0x7f8c1c1a07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:38.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:38.289+0000 7f8c20e00700 1 -- 172.21.15.151:0/1827944752 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8c1c06bc40 msgr2=0x7f8c1c19bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:50:38.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:38.289+0000 7f8c20e00700 1 --2- 172.21.15.151:0/1827944752 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8c1c06bc40 0x7f8c1c19bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:38.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:38.289+0000 7f8c20e00700 1 -- 172.21.15.151:0/1827944752 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c1c1a0f20 con 0x7f8c1c06c560 2024-05-21T21:50:38.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:38.289+0000 7f8c20e00700 1 --2- 172.21.15.151:0/1827944752 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c1c06c560 0x7f8c1c19c450 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8c0800c010 tx=0x7f8c0800fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:50:38.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:38.290+0000 7f8c1a000700 1 -- 172.21.15.151:0/1827944752 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c080050c0 con 0x7f8c1c06c560 2024-05-21T21:50:38.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:38.290+0000 7f8c1a000700 1 -- 172.21.15.151:0/1827944752 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c0800b560 con 0x7f8c1c06c560 2024-05-21T21:50:38.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:38.290+0000 7f8c24200700 1 -- 172.21.15.151:0/1827944752 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c1c1a11b0 con 0x7f8c1c06c560 2024-05-21T21:50:38.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:38.290+0000 7f8c1a000700 1 -- 172.21.15.151:0/1827944752 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c08003a00 con 0x7f8c1c06c560 2024-05-21T21:50:38.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:38.290+0000 7f8c24200700 1 -- 172.21.15.151:0/1827944752 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c1c1a8a90 con 0x7f8c1c06c560 2024-05-21T21:50:38.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:38.291+0000 7f8c24200700 1 -- 172.21.15.151:0/1827944752 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c1c102a90 con 0x7f8c1c06c560 2024-05-21T21:50:38.293 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:38.293+0000 7f8c1a000700 1 -- 172.21.15.151:0/1827944752 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f8c0800b710 con 0x7f8c1c06c560 2024-05-21T21:50:38.294 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:38.295+0000 7f8c1a000700 1 --2- 172.21.15.151:0/1827944752 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f8c040b2880 0x7f8c040b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:50:38.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:38.295+0000 7f8c1a000700 1 -- 172.21.15.151:0/1827944752 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f8c080e2bd0 con 0x7f8c1c06c560 2024-05-21T21:50:38.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:38.295+0000 7f8c21800700 1 --2- 172.21.15.151:0/1827944752 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f8c040b2880 0x7f8c040b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:50:38.296 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:38.296+0000 7f8c21800700 1 --2- 172.21.15.151:0/1827944752 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f8c040b2880 0x7f8c040b4d40 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8c10005950 tx=0x7f8c10009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:50:38.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:38.299+0000 7f8c1a000700 1 -- 172.21.15.151:0/1827944752 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8c08013030 con 0x7f8c1c06c560 2024-05-21T21:50:38.348 INFO:journalctl@ceph.mds.f.smithi184.stdout:May 21 21:50:38 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-f[67208]: starting mds.f at 2024-05-21T21:50:38.348 INFO:journalctl@ceph.mds.f.smithi184.stdout:May 21 21:50:38 smithi184 bash[67193]: e13fed9be1b512cee5c2dc5487666b6feb2ecf2193c40e0cc9bbc2e24309839e 2024-05-21T21:50:38.348 INFO:journalctl@ceph.mds.f.smithi184.stdout:May 21 21:50:38 smithi184 systemd[1]: Started Ceph mds.f for c8c7a2e6-17ba-11ef-bc9a-c7b262605968. 2024-05-21T21:50:38.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:38.585+0000 7f8c24200700 1 -- 172.21.15.151:0/1827944752 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8c1c19d270 con 0x7f8c1c06c560 2024-05-21T21:50:38.586 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:38 smithi151 ceph-mon[31623]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:50:38.586 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:38 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:38.586 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:38 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:38.586 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:38 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:38.586 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:38 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T21:50:38.586 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:38 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T21:50:38.586 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:38 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:50:38.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:38.586+0000 7f8c1a000700 1 -- 172.21.15.151:0/1827944752 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5533 (secure 0 0 0) 0x7f8c080aad80 con 0x7f8c1c06c560 2024-05-21T21:50:38.587 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T21:50:38.587 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14358,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/158010529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":158010529},{"type":"v1","addr":"172.21.15.151:6835","nonce":158010529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/2026266114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":2026266114},{"type":"v1","addr":"172.21.15.184:6833","nonce":2026266114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/2467900844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":2467900844},{"type":"v1","addr":"172.21.15.164:6833","nonce":2467900844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24461,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6837/3367640960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3367640960},{"type":"v1","addr":"172.21.15.151:6837","nonce":3367640960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/3264142621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3264142621},{"type":"v1","addr":"172.21.15.184:6835","nonce":3264142621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24473,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2340428470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2340428470},{"type":"v1","addr":"172.21.15.164:6835","nonce":2340428470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-05-21T21:50:38.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:38.590+0000 7f8c24200700 1 -- 172.21.15.151:0/1827944752 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f8c040b2880 msgr2=0x7f8c040b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:50:38.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:38.590+0000 7f8c24200700 1 --2- 172.21.15.151:0/1827944752 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f8c040b2880 0x7f8c040b4d40 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8c10005950 tx=0x7f8c10009310 comp rx=0 tx=0).stop 2024-05-21T21:50:38.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:38.590+0000 7f8c24200700 1 -- 172.21.15.151:0/1827944752 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c1c06c560 msgr2=0x7f8c1c19c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:50:38.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:38.590+0000 7f8c24200700 1 --2- 172.21.15.151:0/1827944752 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c1c06c560 0x7f8c1c19c450 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8c0800c010 tx=0x7f8c0800fa00 comp rx=0 tx=0).stop 2024-05-21T21:50:38.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:38.591+0000 7f8c24200700 1 -- 172.21.15.151:0/1827944752 shutdown_connections 2024-05-21T21:50:38.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:38.591+0000 7f8c24200700 1 --2- 172.21.15.151:0/1827944752 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f8c040b2880 0x7f8c040b4d40 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:38.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:38.591+0000 7f8c24200700 1 --2- 172.21.15.151:0/1827944752 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8c1c110020 0x7f8c1c1a07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:38.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:38.591+0000 7f8c24200700 1 --2- 172.21.15.151:0/1827944752 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8c1c06bc40 0x7f8c1c19bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:38.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:38.591+0000 7f8c24200700 1 --2- 172.21.15.151:0/1827944752 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c1c06c560 0x7f8c1c19c450 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:38.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:38.591+0000 7f8c24200700 1 -- 172.21.15.151:0/1827944752 >> 172.21.15.151:0/1827944752 conn(0x7f8c1c079480 msgr2=0x7f8c1c111510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:50:38.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:38.591+0000 7f8c24200700 1 -- 172.21.15.151:0/1827944752 shutdown_connections 2024-05-21T21:50:38.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:38.591+0000 7f8c24200700 1 -- 172.21.15.151:0/1827944752 wait complete. 2024-05-21T21:50:38.592 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 7 2024-05-21T21:50:38.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:38 smithi184 ceph-mon[36942]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:50:38.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:38 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:38.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:38 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:38.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:38 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:38.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:38 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T21:50:38.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:38 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T21:50:38.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:38 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:50:38.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:38 smithi164 ceph-mon[36269]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:50:38.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:38 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:38.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:38 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:38.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:38 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:38.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:38 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T21:50:38.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:38 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T21:50:38.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:38 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:50:39.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:39 smithi184 ceph-mon[36942]: Deploying daemon mds.c on smithi184 2024-05-21T21:50:39.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:39 smithi184 ceph-mon[36942]: mds.? [v2:172.21.15.184:6834/3264142621,v1:172.21.15.184:6835/3264142621] up:boot 2024-05-21T21:50:39.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:39 smithi184 ceph-mon[36942]: fsmap 6 up:standby 2024-05-21T21:50:39.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:39 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-21T21:50:39.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:39 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/1827944752' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T21:50:39.600 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:39 smithi151 ceph-mon[31623]: Deploying daemon mds.c on smithi184 2024-05-21T21:50:39.601 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:39 smithi151 ceph-mon[31623]: mds.? [v2:172.21.15.184:6834/3264142621,v1:172.21.15.184:6835/3264142621] up:boot 2024-05-21T21:50:39.601 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:39 smithi151 ceph-mon[31623]: fsmap 6 up:standby 2024-05-21T21:50:39.601 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:39 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-21T21:50:39.601 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:39 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/1827944752' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T21:50:39.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:39 smithi164 ceph-mon[36269]: Deploying daemon mds.c on smithi184 2024-05-21T21:50:39.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:39 smithi164 ceph-mon[36269]: mds.? [v2:172.21.15.184:6834/3264142621,v1:172.21.15.184:6835/3264142621] up:boot 2024-05-21T21:50:39.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:39 smithi164 ceph-mon[36269]: fsmap 6 up:standby 2024-05-21T21:50:39.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:39 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-21T21:50:39.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:39 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/1827944752' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T21:50:40.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:40 smithi151 ceph-mon[31623]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:50:40.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:40 smithi184 ceph-mon[36942]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:50:40.736 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 21 21:50:40 smithi184 systemd[1]: Starting Ceph mds.c for c8c7a2e6-17ba-11ef-bc9a-c7b262605968... 2024-05-21T21:50:40.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:40 smithi164 ceph-mon[36269]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:50:41.096 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 21 21:50:40 smithi184 podman[67497]: 2024-05-21 21:50:40.71154186 +0000 UTC m=+0.017376970 image pull d6bc8f31cbc1a7165ca248fe9ba0b06ea14b30f46e1ea33ced1a5238e247af1f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28 2024-05-21T21:50:41.097 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 21 21:50:40 smithi184 podman[67497]: 2024-05-21 21:50:40.835446727 +0000 UTC m=+0.141281826 container create 6364be5c643d85b7a0a3cc1969a2574ab74c8457d3bec2769f05cabbd13d84d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-05-21T21:50:41.097 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 21 21:50:41 smithi184 podman[67497]: 2024-05-21 21:50:41.096901267 +0000 UTC m=+0.402736378 container init 6364be5c643d85b7a0a3cc1969a2574ab74c8457d3bec2769f05cabbd13d84d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-c, org.label-schema.build-date=20240514, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-21T21:50:41.351 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-21T21:50:41.485 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 21 21:50:41 smithi184 podman[67497]: 2024-05-21 21:50:41.099996325 +0000 UTC m=+0.405831431 container start 6364be5c643d85b7a0a3cc1969a2574ab74c8457d3bec2769f05cabbd13d84d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-c, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-21T21:50:41.486 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 21 21:50:41 smithi184 ceph-mds[67515]: -- 172.21.15.184:0/3094420379 <== mon.1 v2:172.21.15.184:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561f8ebafa40 con 0x561f8eb82c00 2024-05-21T21:50:41.486 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 21 21:50:41 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-c[67511]: starting mds.c at 2024-05-21T21:50:41.486 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 21 21:50:41 smithi184 bash[67497]: 6364be5c643d85b7a0a3cc1969a2574ab74c8457d3bec2769f05cabbd13d84d3 2024-05-21T21:50:41.486 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 21 21:50:41 smithi184 systemd[1]: Started Ceph mds.c for c8c7a2e6-17ba-11ef-bc9a-c7b262605968. 2024-05-21T21:50:41.802 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:50:42.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:42 smithi184 ceph-mon[36942]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:50:42.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:42 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:42.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:42 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:42.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:42 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:42.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:42 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T21:50:42.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:42 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T21:50:42.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:42 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:50:42.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:42 smithi184 ceph-mon[36942]: Deploying daemon mds.h on smithi164 2024-05-21T21:50:42.487 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:42 smithi184 ceph-mon[36942]: mds.? [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] up:boot 2024-05-21T21:50:42.487 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:42 smithi184 ceph-mon[36942]: fsmap 7 up:standby 2024-05-21T21:50:42.487 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:42 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-21T21:50:42.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:42 smithi164 ceph-mon[36269]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:50:42.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:42 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:42.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:42 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:42.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:42 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:42.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:42 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T21:50:42.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:42 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T21:50:42.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:42 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:50:42.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:42 smithi164 ceph-mon[36269]: Deploying daemon mds.h on smithi164 2024-05-21T21:50:42.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:42 smithi164 ceph-mon[36269]: mds.? [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] up:boot 2024-05-21T21:50:42.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:42 smithi164 ceph-mon[36269]: fsmap 7 up:standby 2024-05-21T21:50:42.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:42 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-21T21:50:42.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:42 smithi151 ceph-mon[31623]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:50:42.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:42 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:42.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:42 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:42.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:42 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:42.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:42 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T21:50:42.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:42 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T21:50:42.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:42 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:50:42.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:42 smithi151 ceph-mon[31623]: Deploying daemon mds.h on smithi164 2024-05-21T21:50:42.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:42 smithi151 ceph-mon[31623]: mds.? [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] up:boot 2024-05-21T21:50:42.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:42 smithi151 ceph-mon[31623]: fsmap 7 up:standby 2024-05-21T21:50:42.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:42 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-21T21:50:43.536 INFO:journalctl@ceph.mds.h.smithi164.stdout:May 21 21:50:43 smithi164 systemd[1]: Starting Ceph mds.h for c8c7a2e6-17ba-11ef-bc9a-c7b262605968... 2024-05-21T21:50:43.861 INFO:journalctl@ceph.mds.h.smithi164.stdout:May 21 21:50:43 smithi164 podman[65924]: 2024-05-21 21:50:43.510663419 +0000 UTC m=+0.016756811 image pull d6bc8f31cbc1a7165ca248fe9ba0b06ea14b30f46e1ea33ced1a5238e247af1f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28 2024-05-21T21:50:43.861 INFO:journalctl@ceph.mds.h.smithi164.stdout:May 21 21:50:43 smithi164 podman[65924]: 2024-05-21 21:50:43.618084475 +0000 UTC m=+0.124177878 container create 8738418e32525b423defbafc15f0a15c2bc9923b5e5a6a4134a880b2f7ce728e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-h, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, ceph=True, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-05-21T21:50:43.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:43.943+0000 7fd342e00700 1 -- 172.21.15.151:0/464603320 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd33c10ff40 msgr2=0x7fd33c112330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:50:43.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:43.943+0000 7fd342e00700 1 --2- 172.21.15.151:0/464603320 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd33c10ff40 0x7fd33c112330 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd33400b0a0 tx=0x7fd33400b3b0 comp rx=0 tx=0).stop 2024-05-21T21:50:43.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:43.943+0000 7fd342e00700 1 -- 172.21.15.151:0/464603320 shutdown_connections 2024-05-21T21:50:43.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:43.943+0000 7fd342e00700 1 --2- 172.21.15.151:0/464603320 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd33c10ff40 0x7fd33c112330 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:43.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:43.943+0000 7fd342e00700 1 --2- 172.21.15.151:0/464603320 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd33c075770 0x7fd33c075b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:43.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:43.943+0000 7fd342e00700 1 --2- 172.21.15.151:0/464603320 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd33c076090 0x7fd33c10fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:43.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:43.943+0000 7fd342e00700 1 -- 172.21.15.151:0/464603320 >> 172.21.15.151:0/464603320 conn(0x7fd33c07a6a0 msgr2=0x7fd33c07aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:50:43.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:43.943+0000 7fd342e00700 1 -- 172.21.15.151:0/464603320 shutdown_connections 2024-05-21T21:50:43.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:43.944+0000 7fd342e00700 1 -- 172.21.15.151:0/464603320 wait complete. 2024-05-21T21:50:43.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:43.945+0000 7fd342e00700 1 Processor -- start 2024-05-21T21:50:43.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:43.945+0000 7fd342e00700 1 -- start start 2024-05-21T21:50:43.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:43.945+0000 7fd342e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd33c075770 0x7fd33c19bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:50:43.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:43.945+0000 7fd342e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd33c076090 0x7fd33c19c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:50:43.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:43.945+0000 7fd342e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd33c10ff40 0x7fd33c1a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:50:43.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:43.945+0000 7fd342e00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd33c115930 con 0x7fd33c075770 2024-05-21T21:50:43.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:43.945+0000 7fd342e00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd33c1157b0 con 0x7fd33c076090 2024-05-21T21:50:43.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:43.945+0000 7fd342e00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd33c115ab0 con 0x7fd33c10ff40 2024-05-21T21:50:43.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:43.945+0000 7fd33be00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd33c075770 0x7fd33c19bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:50:43.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:43.945+0000 7fd33be00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd33c075770 0x7fd33c19bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:60818/0 (socket says 172.21.15.151:60818) 2024-05-21T21:50:43.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:43.945+0000 7fd33be00700 1 -- 172.21.15.151:0/3223446459 learned_addr learned my addr 172.21.15.151:0/3223446459 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:50:43.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:43.945+0000 7fd333400700 1 --2- 172.21.15.151:0/3223446459 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd33c076090 0x7fd33c19c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:50:43.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:43.946+0000 7fd340e00700 1 --2- 172.21.15.151:0/3223446459 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd33c10ff40 0x7fd33c1a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:50:43.948 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:43.946+0000 7fd33be00700 1 -- 172.21.15.151:0/3223446459 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd33c10ff40 msgr2=0x7fd33c1a0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:50:43.948 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:43.946+0000 7fd33be00700 1 --2- 172.21.15.151:0/3223446459 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd33c10ff40 0x7fd33c1a0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:43.948 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:43.946+0000 7fd33be00700 1 -- 172.21.15.151:0/3223446459 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd33c076090 msgr2=0x7fd33c19c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:50:43.948 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:43.946+0000 7fd33be00700 1 --2- 172.21.15.151:0/3223446459 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd33c076090 0x7fd33c19c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:43.948 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:43.946+0000 7fd33be00700 1 -- 172.21.15.151:0/3223446459 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd33c1a0d80 con 0x7fd33c075770 2024-05-21T21:50:43.948 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:43.946+0000 7fd340e00700 1 --2- 172.21.15.151:0/3223446459 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd33c10ff40 0x7fd33c1a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T21:50:43.948 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:43.946+0000 7fd33be00700 1 --2- 172.21.15.151:0/3223446459 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd33c075770 0x7fd33c19bdc0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd32c00d980 tx=0x7fd32c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:50:43.948 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:43.947+0000 7fd339600700 1 -- 172.21.15.151:0/3223446459 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd32c016070 con 0x7fd33c075770 2024-05-21T21:50:43.948 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:43.947+0000 7fd342e00700 1 -- 172.21.15.151:0/3223446459 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd33c1a1070 con 0x7fd33c075770 2024-05-21T21:50:43.948 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:43.947+0000 7fd339600700 1 -- 172.21.15.151:0/3223446459 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd32c004830 con 0x7fd33c075770 2024-05-21T21:50:43.948 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:43.947+0000 7fd339600700 1 -- 172.21.15.151:0/3223446459 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd32c0054c0 con 0x7fd33c075770 2024-05-21T21:50:43.948 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:43.947+0000 7fd342e00700 1 -- 172.21.15.151:0/3223446459 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd33c1a88f0 con 0x7fd33c075770 2024-05-21T21:50:43.949 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:43.948+0000 7fd342e00700 1 -- 172.21.15.151:0/3223446459 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd33c06a230 con 0x7fd33c075770 2024-05-21T21:50:43.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:43.950+0000 7fd339600700 1 -- 172.21.15.151:0/3223446459 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7fd32c004220 con 0x7fd33c075770 2024-05-21T21:50:43.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:43.951+0000 7fd339600700 1 --2- 172.21.15.151:0/3223446459 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fd3200b2880 0x7fd3200b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:50:43.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:43.952+0000 7fd339600700 1 -- 172.21.15.151:0/3223446459 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7fd32c0e1ce0 con 0x7fd33c075770 2024-05-21T21:50:43.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:43.952+0000 7fd333400700 1 --2- 172.21.15.151:0/3223446459 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fd3200b2880 0x7fd3200b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:50:43.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:43.952+0000 7fd333400700 1 --2- 172.21.15.151:0/3223446459 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fd3200b2880 0x7fd3200b4d40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd33c111bd0 tx=0x7fd324009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:50:43.956 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:43.956+0000 7fd339600700 1 -- 172.21.15.151:0/3223446459 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd32c01f020 con 0x7fd33c075770 2024-05-21T21:50:44.247 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:44.246+0000 7fd342e00700 1 -- 172.21.15.151:0/3223446459 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd33c1a8af0 con 0x7fd33c075770 2024-05-21T21:50:44.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:44.248+0000 7fd339600700 1 -- 172.21.15.151:0/3223446459 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7189 (secure 0 0 0) 0x7fd32c0a3e20 con 0x7fd33c075770 2024-05-21T21:50:44.248 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T21:50:44.249 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14358,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/158010529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":158010529},{"type":"v1","addr":"172.21.15.151:6835","nonce":158010529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/2026266114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":2026266114},{"type":"v1","addr":"172.21.15.184:6833","nonce":2026266114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/2467900844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":2467900844},{"type":"v1","addr":"172.21.15.164:6833","nonce":2467900844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24461,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6837/3367640960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3367640960},{"type":"v1","addr":"172.21.15.151:6837","nonce":3367640960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/3264142621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3264142621},{"type":"v1","addr":"172.21.15.184:6835","nonce":3264142621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24473,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2340428470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2340428470},{"type":"v1","addr":"172.21.15.164:6835","nonce":2340428470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/2988194888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":2988194888},{"type":"v1","addr":"172.21.15.184:6837","nonce":2988194888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24491,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/1554609572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1554609572},{"type":"v1","addr":"172.21.15.164:6837","nonce":1554609572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-05-21T21:50:44.249 INFO:journalctl@ceph.mds.h.smithi164.stdout:May 21 21:50:43 smithi164 podman[65924]: 2024-05-21 21:50:43.862002886 +0000 UTC m=+0.368096262 container init 8738418e32525b423defbafc15f0a15c2bc9923b5e5a6a4134a880b2f7ce728e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-h, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2) 2024-05-21T21:50:44.249 INFO:journalctl@ceph.mds.h.smithi164.stdout:May 21 21:50:43 smithi164 podman[65924]: 2024-05-21 21:50:43.864844841 +0000 UTC m=+0.370938228 container start 8738418e32525b423defbafc15f0a15c2bc9923b5e5a6a4134a880b2f7ce728e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-h, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True) 2024-05-21T21:50:44.249 INFO:journalctl@ceph.mds.h.smithi164.stdout:May 21 21:50:43 smithi164 ceph-mds[65943]: -- 172.21.15.164:0/2162164870 <== mon.2 v2:172.21.15.164:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5628f01d5a40 con 0x5628f01a8c00 2024-05-21T21:50:44.249 INFO:journalctl@ceph.mds.h.smithi164.stdout:May 21 21:50:43 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-h[65939]: starting mds.h at 2024-05-21T21:50:44.249 INFO:journalctl@ceph.mds.h.smithi164.stdout:May 21 21:50:43 smithi164 bash[65924]: 8738418e32525b423defbafc15f0a15c2bc9923b5e5a6a4134a880b2f7ce728e 2024-05-21T21:50:44.249 INFO:journalctl@ceph.mds.h.smithi164.stdout:May 21 21:50:43 smithi164 systemd[1]: Started Ceph mds.h for c8c7a2e6-17ba-11ef-bc9a-c7b262605968. 2024-05-21T21:50:44.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:44.251+0000 7fd342e00700 1 -- 172.21.15.151:0/3223446459 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fd3200b2880 msgr2=0x7fd3200b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:50:44.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:44.251+0000 7fd342e00700 1 --2- 172.21.15.151:0/3223446459 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fd3200b2880 0x7fd3200b4d40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd33c111bd0 tx=0x7fd324009260 comp rx=0 tx=0).stop 2024-05-21T21:50:44.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:44.251+0000 7fd342e00700 1 -- 172.21.15.151:0/3223446459 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd33c075770 msgr2=0x7fd33c19bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:50:44.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:44.251+0000 7fd342e00700 1 --2- 172.21.15.151:0/3223446459 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd33c075770 0x7fd33c19bdc0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd32c00d980 tx=0x7fd32c00dc90 comp rx=0 tx=0).stop 2024-05-21T21:50:44.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:44.251+0000 7fd342e00700 1 -- 172.21.15.151:0/3223446459 shutdown_connections 2024-05-21T21:50:44.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:44.251+0000 7fd342e00700 1 --2- 172.21.15.151:0/3223446459 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fd3200b2880 0x7fd3200b4d40 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:44.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:44.251+0000 7fd342e00700 1 --2- 172.21.15.151:0/3223446459 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd33c10ff40 0x7fd33c1a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:44.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:44.251+0000 7fd342e00700 1 --2- 172.21.15.151:0/3223446459 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd33c075770 0x7fd33c19bdc0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:44.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:44.251+0000 7fd342e00700 1 --2- 172.21.15.151:0/3223446459 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd33c076090 0x7fd33c19c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:44.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:44.251+0000 7fd342e00700 1 -- 172.21.15.151:0/3223446459 >> 172.21.15.151:0/3223446459 conn(0x7fd33c07a6a0 msgr2=0x7fd33c1006f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:50:44.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:44.251+0000 7fd342e00700 1 -- 172.21.15.151:0/3223446459 shutdown_connections 2024-05-21T21:50:44.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:44.252+0000 7fd342e00700 1 -- 172.21.15.151:0/3223446459 wait complete. 2024-05-21T21:50:44.253 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 9 2024-05-21T21:50:44.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:44 smithi164 ceph-mon[36269]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:50:44.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:44 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:44.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:44 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:44.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:44 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:44.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:44 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T21:50:44.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:44 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T21:50:44.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:44 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:50:44.669 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:44 smithi151 ceph-mon[31623]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:50:44.669 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:44 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:44.669 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:44 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:44.670 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:44 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:44.670 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:44 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T21:50:44.670 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:44 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T21:50:44.670 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:44 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:50:44.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:44 smithi184 ceph-mon[36942]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:50:44.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:44 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:44.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:44 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:44.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:44 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:44.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:44 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T21:50:44.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:44 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T21:50:44.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:44 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:50:45.428 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:45 smithi184 ceph-mon[36942]: Deploying daemon mds.i on smithi184 2024-05-21T21:50:45.428 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:45 smithi184 ceph-mon[36942]: mds.? [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] up:boot 2024-05-21T21:50:45.428 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:45 smithi184 ceph-mon[36942]: fsmap 8 up:standby 2024-05-21T21:50:45.428 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:45 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-21T21:50:45.428 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:45 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/3223446459' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T21:50:45.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:45 smithi151 ceph-mon[31623]: Deploying daemon mds.i on smithi184 2024-05-21T21:50:45.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:45 smithi151 ceph-mon[31623]: mds.? [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] up:boot 2024-05-21T21:50:45.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:45 smithi151 ceph-mon[31623]: fsmap 8 up:standby 2024-05-21T21:50:45.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:45 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-21T21:50:45.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:45 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/3223446459' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T21:50:45.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:45 smithi164 ceph-mon[36269]: Deploying daemon mds.i on smithi184 2024-05-21T21:50:45.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:45 smithi164 ceph-mon[36269]: mds.? [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] up:boot 2024-05-21T21:50:45.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:45 smithi164 ceph-mon[36269]: fsmap 8 up:standby 2024-05-21T21:50:45.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:45 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-21T21:50:45.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:45 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/3223446459' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T21:50:46.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:46 smithi164 ceph-mon[36269]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:50:46.641 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:46 smithi184 ceph-mon[36942]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:50:46.641 INFO:journalctl@ceph.mds.i.smithi184.stdout:May 21 21:50:46 smithi184 systemd[1]: Starting Ceph mds.i for c8c7a2e6-17ba-11ef-bc9a-c7b262605968... 2024-05-21T21:50:46.702 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:46 smithi151 ceph-mon[31623]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:50:46.902 INFO:journalctl@ceph.mds.i.smithi184.stdout:May 21 21:50:46 smithi184 podman[67796]: 2024-05-21 21:50:46.543521095 +0000 UTC m=+0.017476350 image pull d6bc8f31cbc1a7165ca248fe9ba0b06ea14b30f46e1ea33ced1a5238e247af1f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28 2024-05-21T21:50:46.902 INFO:journalctl@ceph.mds.i.smithi184.stdout:May 21 21:50:46 smithi184 podman[67796]: 2024-05-21 21:50:46.648184342 +0000 UTC m=+0.122139587 container create 1fa7a2f12571eda44f1771cbf7d4e32e7114a6b87a3f6ed9f7db2012a366f6e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-i, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, ceph=True, maintainer=Guillaume Abrioux ) 2024-05-21T21:50:46.902 INFO:journalctl@ceph.mds.i.smithi184.stdout:May 21 21:50:46 smithi184 podman[67796]: 2024-05-21 21:50:46.902347837 +0000 UTC m=+0.376303082 container init 1fa7a2f12571eda44f1771cbf7d4e32e7114a6b87a3f6ed9f7db2012a366f6e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-05-21T21:50:47.080 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-21T21:50:47.236 INFO:journalctl@ceph.mds.i.smithi184.stdout:May 21 21:50:46 smithi184 podman[67796]: 2024-05-21 21:50:46.905101971 +0000 UTC m=+0.379057217 container start 1fa7a2f12571eda44f1771cbf7d4e32e7114a6b87a3f6ed9f7db2012a366f6e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-i, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-05-21T21:50:47.236 INFO:journalctl@ceph.mds.i.smithi184.stdout:May 21 21:50:46 smithi184 ceph-mds[67815]: -- 172.21.15.184:0/2813256346 <== mon.1 v2:172.21.15.184:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5560df4c3a40 con 0x5560df496c00 2024-05-21T21:50:47.236 INFO:journalctl@ceph.mds.i.smithi184.stdout:May 21 21:50:46 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-i[67811]: starting mds.i at 2024-05-21T21:50:47.236 INFO:journalctl@ceph.mds.i.smithi184.stdout:May 21 21:50:46 smithi184 bash[67796]: 1fa7a2f12571eda44f1771cbf7d4e32e7114a6b87a3f6ed9f7db2012a366f6e5 2024-05-21T21:50:47.236 INFO:journalctl@ceph.mds.i.smithi184.stdout:May 21 21:50:46 smithi184 systemd[1]: Started Ceph mds.i for c8c7a2e6-17ba-11ef-bc9a-c7b262605968. 2024-05-21T21:50:47.518 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:50:48.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:48 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:48.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:48 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:48.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:48 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:48.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:48 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T21:50:48.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:48 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T21:50:48.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:48 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:50:48.287 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:48 smithi164 ceph-mon[36269]: Deploying daemon mds.g on smithi151 2024-05-21T21:50:48.287 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:48 smithi164 ceph-mon[36269]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:50:48.287 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:48 smithi164 ceph-mon[36269]: mds.? [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] up:boot 2024-05-21T21:50:48.287 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:48 smithi164 ceph-mon[36269]: fsmap 9 up:standby 2024-05-21T21:50:48.287 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:48 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-21T21:50:48.314 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:48 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:48.315 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:48 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:48.315 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:48 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:48.315 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:48 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T21:50:48.315 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:48 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T21:50:48.315 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:48 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:50:48.315 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:48 smithi184 ceph-mon[36942]: Deploying daemon mds.g on smithi151 2024-05-21T21:50:48.315 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:48 smithi184 ceph-mon[36942]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:50:48.315 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:48 smithi184 ceph-mon[36942]: mds.? [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] up:boot 2024-05-21T21:50:48.315 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:48 smithi184 ceph-mon[36942]: fsmap 9 up:standby 2024-05-21T21:50:48.315 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:48 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-21T21:50:48.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:48 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:48.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:48 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:48.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:48 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:48.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:48 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T21:50:48.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:48 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T21:50:48.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:48 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:50:48.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:48 smithi151 ceph-mon[31623]: Deploying daemon mds.g on smithi151 2024-05-21T21:50:48.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:48 smithi151 ceph-mon[31623]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:50:48.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:48 smithi151 ceph-mon[31623]: mds.? [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] up:boot 2024-05-21T21:50:48.454 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:48 smithi151 ceph-mon[31623]: fsmap 9 up:standby 2024-05-21T21:50:48.454 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:48 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-21T21:50:50.216 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:50.052+0000 7ffb2e400700 1 -- 172.21.15.151:0/1509466567 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffb2810ff40 msgr2=0x7ffb28112330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:50:50.216 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:50.052+0000 7ffb2e400700 1 --2- 172.21.15.151:0/1509466567 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffb2810ff40 0x7ffb28112330 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ffb20009f90 tx=0x7ffb2000b330 comp rx=0 tx=0).stop 2024-05-21T21:50:50.216 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:50.052+0000 7ffb2e400700 1 -- 172.21.15.151:0/1509466567 shutdown_connections 2024-05-21T21:50:50.216 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:50.052+0000 7ffb2e400700 1 --2- 172.21.15.151:0/1509466567 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffb28075770 0x7ffb28075b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:50.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:50.052+0000 7ffb2e400700 1 --2- 172.21.15.151:0/1509466567 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffb2810ff40 0x7ffb28112330 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:50.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:50.052+0000 7ffb2e400700 1 --2- 172.21.15.151:0/1509466567 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffb28076090 0x7ffb2810fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:50.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:50.052+0000 7ffb2e400700 1 -- 172.21.15.151:0/1509466567 >> 172.21.15.151:0/1509466567 conn(0x7ffb2807a6a0 msgr2=0x7ffb2807aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:50:50.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:50.052+0000 7ffb2e400700 1 -- 172.21.15.151:0/1509466567 shutdown_connections 2024-05-21T21:50:50.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:50.052+0000 7ffb2e400700 1 -- 172.21.15.151:0/1509466567 wait complete. 2024-05-21T21:50:50.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:50.054+0000 7ffb2e400700 1 Processor -- start 2024-05-21T21:50:50.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:50.054+0000 7ffb2e400700 1 -- start start 2024-05-21T21:50:50.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:50.054+0000 7ffb2e400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffb28075770 0x7ffb2819bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:50:50.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:50.054+0000 7ffb2e400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffb28076090 0x7ffb2819c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:50:50.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:50.054+0000 7ffb2e400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffb2810ff40 0x7ffb281a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:50:50.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:50.055+0000 7ffb2e400700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ffb28115970 con 0x7ffb2810ff40 2024-05-21T21:50:50.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:50.055+0000 7ffb2e400700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ffb281157f0 con 0x7ffb28075770 2024-05-21T21:50:50.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:50.055+0000 7ffb2e400700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ffb28115af0 con 0x7ffb28076090 2024-05-21T21:50:50.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:50.055+0000 7ffb27e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffb2810ff40 0x7ffb281a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:50:50.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:50.055+0000 7ffb27400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffb28075770 0x7ffb2819bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:50:50.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:50.055+0000 7ffb27e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffb2810ff40 0x7ffb281a0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41150/0 (socket says 172.21.15.151:41150) 2024-05-21T21:50:50.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:50.055+0000 7ffb27e00700 1 -- 172.21.15.151:0/406851664 learned_addr learned my addr 172.21.15.151:0/406851664 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:50:50.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:50.055+0000 7ffb26a00700 1 --2- 172.21.15.151:0/406851664 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffb28076090 0x7ffb2819c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:50:50.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:50.055+0000 7ffb27e00700 1 -- 172.21.15.151:0/406851664 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffb28076090 msgr2=0x7ffb2819c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:50:50.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:50.055+0000 7ffb27e00700 1 --2- 172.21.15.151:0/406851664 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffb28076090 0x7ffb2819c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:50.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:50.055+0000 7ffb27e00700 1 -- 172.21.15.151:0/406851664 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffb28075770 msgr2=0x7ffb2819bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:50:50.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:50.055+0000 7ffb27e00700 1 --2- 172.21.15.151:0/406851664 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffb28075770 0x7ffb2819bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:50.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:50.055+0000 7ffb27e00700 1 -- 172.21.15.151:0/406851664 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb281a0db0 con 0x7ffb2810ff40 2024-05-21T21:50:50.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:50.055+0000 7ffb27400700 1 --2- 172.21.15.151:0/406851664 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffb28075770 0x7ffb2819bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:50:50.219 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:50.056+0000 7ffb27e00700 1 --2- 172.21.15.151:0/406851664 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffb2810ff40 0x7ffb281a0670 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ffb20009c10 tx=0x7ffb2000cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:50:50.219 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:50.056+0000 7ffb1fe00700 1 -- 172.21.15.151:0/406851664 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb20021070 con 0x7ffb2810ff40 2024-05-21T21:50:50.219 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:50.056+0000 7ffb1fe00700 1 -- 172.21.15.151:0/406851664 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb2000ce90 con 0x7ffb2810ff40 2024-05-21T21:50:50.219 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:50.056+0000 7ffb2e400700 1 -- 172.21.15.151:0/406851664 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb281a1040 con 0x7ffb2810ff40 2024-05-21T21:50:50.219 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:50.056+0000 7ffb1fe00700 1 -- 172.21.15.151:0/406851664 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb20005040 con 0x7ffb2810ff40 2024-05-21T21:50:50.219 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:50.056+0000 7ffb2e400700 1 -- 172.21.15.151:0/406851664 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb281a8920 con 0x7ffb2810ff40 2024-05-21T21:50:50.219 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:50.057+0000 7ffb2e400700 1 -- 172.21.15.151:0/406851664 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb2806a230 con 0x7ffb2810ff40 2024-05-21T21:50:50.219 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:50.060+0000 7ffb1fe00700 1 -- 172.21.15.151:0/406851664 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7ffb20024070 con 0x7ffb2810ff40 2024-05-21T21:50:50.219 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:50.062+0000 7ffb1fe00700 1 --2- 172.21.15.151:0/406851664 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7ffb0c0b2880 0x7ffb0c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:50:50.219 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:50.062+0000 7ffb1fe00700 1 -- 172.21.15.151:0/406851664 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7ffb200e2fe0 con 0x7ffb2810ff40 2024-05-21T21:50:50.219 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:50.062+0000 7ffb27400700 1 --2- 172.21.15.151:0/406851664 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7ffb0c0b2880 0x7ffb0c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:50:50.219 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:50.063+0000 7ffb27400700 1 --2- 172.21.15.151:0/406851664 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7ffb0c0b2880 0x7ffb0c0b4d40 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ffb18011490 tx=0x7ffb18009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:50:50.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:50.064+0000 7ffb1fe00700 1 -- 172.21.15.151:0/406851664 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ffb200ab230 con 0x7ffb2810ff40 2024-05-21T21:50:50.333 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:50 smithi151 ceph-mon[31623]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:50:50.354 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:50.354+0000 7ffb2e400700 1 -- 172.21.15.151:0/406851664 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffb2819d100 con 0x7ffb2810ff40 2024-05-21T21:50:50.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:50.355+0000 7ffb1fe00700 1 -- 172.21.15.151:0/406851664 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8019 (secure 0 0 0) 0x7ffb200aab30 con 0x7ffb2810ff40 2024-05-21T21:50:50.356 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T21:50:50.356 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14358,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/158010529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":158010529},{"type":"v1","addr":"172.21.15.151:6835","nonce":158010529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/2026266114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":2026266114},{"type":"v1","addr":"172.21.15.184:6833","nonce":2026266114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/2467900844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":2467900844},{"type":"v1","addr":"172.21.15.164:6833","nonce":2467900844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24461,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6837/3367640960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3367640960},{"type":"v1","addr":"172.21.15.151:6837","nonce":3367640960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/3264142621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3264142621},{"type":"v1","addr":"172.21.15.184:6835","nonce":3264142621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24473,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2340428470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2340428470},{"type":"v1","addr":"172.21.15.164:6835","nonce":2340428470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/2988194888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":2988194888},{"type":"v1","addr":"172.21.15.184:6837","nonce":2988194888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24491,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/1554609572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1554609572},{"type":"v1","addr":"172.21.15.164:6837","nonce":1554609572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24493,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6839/3059935143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":3059935143},{"type":"v1","addr":"172.21.15.184:6839","nonce":3059935143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-05-21T21:50:50.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:50.359+0000 7ffb2e400700 1 -- 172.21.15.151:0/406851664 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7ffb0c0b2880 msgr2=0x7ffb0c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:50:50.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:50.359+0000 7ffb2e400700 1 --2- 172.21.15.151:0/406851664 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7ffb0c0b2880 0x7ffb0c0b4d40 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ffb18011490 tx=0x7ffb18009260 comp rx=0 tx=0).stop 2024-05-21T21:50:50.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:50.359+0000 7ffb2e400700 1 -- 172.21.15.151:0/406851664 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffb2810ff40 msgr2=0x7ffb281a0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:50:50.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:50.359+0000 7ffb2e400700 1 --2- 172.21.15.151:0/406851664 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffb2810ff40 0x7ffb281a0670 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ffb20009c10 tx=0x7ffb2000cca0 comp rx=0 tx=0).stop 2024-05-21T21:50:50.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:50.359+0000 7ffb2e400700 1 -- 172.21.15.151:0/406851664 shutdown_connections 2024-05-21T21:50:50.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:50.359+0000 7ffb2e400700 1 --2- 172.21.15.151:0/406851664 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7ffb0c0b2880 0x7ffb0c0b4d40 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:50.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:50.359+0000 7ffb2e400700 1 --2- 172.21.15.151:0/406851664 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffb28075770 0x7ffb2819bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:50.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:50.359+0000 7ffb2e400700 1 --2- 172.21.15.151:0/406851664 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffb2810ff40 0x7ffb281a0670 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:50.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:50.359+0000 7ffb2e400700 1 --2- 172.21.15.151:0/406851664 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffb28076090 0x7ffb2819c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:50.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:50.359+0000 7ffb2e400700 1 -- 172.21.15.151:0/406851664 >> 172.21.15.151:0/406851664 conn(0x7ffb2807a6a0 msgr2=0x7ffb28100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:50:50.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:50.359+0000 7ffb2e400700 1 -- 172.21.15.151:0/406851664 shutdown_connections 2024-05-21T21:50:50.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:50.359+0000 7ffb2e400700 1 -- 172.21.15.151:0/406851664 wait complete. 2024-05-21T21:50:50.361 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 10 2024-05-21T21:50:50.485 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:50 smithi184 ceph-mon[36942]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:50:50.535 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:50 smithi164 ceph-mon[36269]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:50:51.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:51 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/406851664' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T21:50:51.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:51 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/406851664' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T21:50:51.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:51 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/406851664' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T21:50:51.858 INFO:journalctl@ceph.mds.g.smithi151.stdout:May 21 21:50:51 smithi151 podman[70940]: 2024-05-21 21:50:51.572162748 +0000 UTC m=+0.154804153 container create d3e49631a0d408dd159d6e2d6f2f636fbaa457a01825abcb3f5c1db2ebc2abe8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-g, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_CLEAN=True, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-05-21T21:50:51.859 INFO:journalctl@ceph.mds.g.smithi151.stdout:May 21 21:50:51 smithi151 podman[70940]: 2024-05-21 21:50:51.857369513 +0000 UTC m=+0.440010941 container init d3e49631a0d408dd159d6e2d6f2f636fbaa457a01825abcb3f5c1db2ebc2abe8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-g, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-21T21:50:52.203 INFO:journalctl@ceph.mds.g.smithi151.stdout:May 21 21:50:51 smithi151 podman[70940]: 2024-05-21 21:50:51.868268545 +0000 UTC m=+0.450909947 container start d3e49631a0d408dd159d6e2d6f2f636fbaa457a01825abcb3f5c1db2ebc2abe8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-g, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2) 2024-05-21T21:50:52.203 INFO:journalctl@ceph.mds.g.smithi151.stdout:May 21 21:50:51 smithi151 ceph-mds[70959]: -- 172.21.15.151:0/3368930344 <== mon.2 v2:172.21.15.164:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55df5bf89a40 con 0x55df5bf5c800 2024-05-21T21:50:52.203 INFO:journalctl@ceph.mds.g.smithi151.stdout:May 21 21:50:51 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-g[70955]: starting mds.g at 2024-05-21T21:50:52.203 INFO:journalctl@ceph.mds.g.smithi151.stdout:May 21 21:50:51 smithi151 bash[70940]: d3e49631a0d408dd159d6e2d6f2f636fbaa457a01825abcb3f5c1db2ebc2abe8 2024-05-21T21:50:52.203 INFO:journalctl@ceph.mds.g.smithi151.stdout:May 21 21:50:51 smithi151 systemd[1]: Started Ceph mds.g for c8c7a2e6-17ba-11ef-bc9a-c7b262605968. 2024-05-21T21:50:52.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:52 smithi164 ceph-mon[36269]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:50:52.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:52 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:52.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:52 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:52.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:52 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:52.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:52 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T21:50:52.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:52 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T21:50:52.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:52 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:50:52.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:52 smithi184 ceph-mon[36942]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:50:52.637 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:52 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:52.637 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:52 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:52.637 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:52 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:52.637 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:52 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T21:50:52.637 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:52 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T21:50:52.637 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:52 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:50:52.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:52 smithi151 ceph-mon[31623]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:50:52.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:52 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:52.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:52 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:52.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:52 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:52.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:52 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T21:50:52.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:52 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T21:50:52.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:52 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:50:53.253 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-21T21:50:53.435 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:53 smithi151 ceph-mon[31623]: Deploying daemon mds.a on smithi151 2024-05-21T21:50:53.436 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:53 smithi151 ceph-mon[31623]: mds.? [v2:172.21.15.151:6838/2069058115,v1:172.21.15.151:6839/2069058115] up:boot 2024-05-21T21:50:53.436 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:53 smithi151 ceph-mon[31623]: fsmap 10 up:standby 2024-05-21T21:50:53.436 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:53 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-21T21:50:53.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:53 smithi164 ceph-mon[36269]: Deploying daemon mds.a on smithi151 2024-05-21T21:50:53.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:53 smithi164 ceph-mon[36269]: mds.? [v2:172.21.15.151:6838/2069058115,v1:172.21.15.151:6839/2069058115] up:boot 2024-05-21T21:50:53.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:53 smithi164 ceph-mon[36269]: fsmap 10 up:standby 2024-05-21T21:50:53.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:53 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-21T21:50:53.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:53 smithi184 ceph-mon[36942]: Deploying daemon mds.a on smithi151 2024-05-21T21:50:53.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:53 smithi184 ceph-mon[36942]: mds.? [v2:172.21.15.151:6838/2069058115,v1:172.21.15.151:6839/2069058115] up:boot 2024-05-21T21:50:53.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:53 smithi184 ceph-mon[36942]: fsmap 10 up:standby 2024-05-21T21:50:53.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:53 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-21T21:50:54.378 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:50:54.459 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:54 smithi151 ceph-mon[31623]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:50:54.535 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:54 smithi164 ceph-mon[36269]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:50:54.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:54 smithi184 ceph-mon[36942]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:50:55.203 INFO:journalctl@ceph.mds.a.smithi151.stdout:May 21 21:50:55 smithi151 systemd[1]: Starting Ceph mds.a for c8c7a2e6-17ba-11ef-bc9a-c7b262605968... 2024-05-21T21:50:56.071 INFO:journalctl@ceph.mds.a.smithi151.stdout:May 21 21:50:55 smithi151 podman[71335]: 2024-05-21 21:50:55.810742735 +0000 UTC m=+0.033772631 image pull d6bc8f31cbc1a7165ca248fe9ba0b06ea14b30f46e1ea33ced1a5238e247af1f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28 2024-05-21T21:50:56.071 INFO:journalctl@ceph.mds.a.smithi151.stdout:May 21 21:50:55 smithi151 podman[71335]: 2024-05-21 21:50:55.946032019 +0000 UTC m=+0.169061952 container create 257d6d88cf359fe1af275777577778de79cd6c14ba68e0f7de4aa079c280fc5e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-a, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-21T21:50:56.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:56 smithi151 ceph-mon[31623]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:50:56.453 INFO:journalctl@ceph.mds.a.smithi151.stdout:May 21 21:50:56 smithi151 podman[71335]: 2024-05-21 21:50:56.175271811 +0000 UTC m=+0.398301751 container init 257d6d88cf359fe1af275777577778de79cd6c14ba68e0f7de4aa079c280fc5e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-05-21T21:50:56.453 INFO:journalctl@ceph.mds.a.smithi151.stdout:May 21 21:50:56 smithi151 podman[71335]: 2024-05-21 21:50:56.185522535 +0000 UTC m=+0.408552451 container start 257d6d88cf359fe1af275777577778de79cd6c14ba68e0f7de4aa079c280fc5e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-a, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-21T21:50:56.453 INFO:journalctl@ceph.mds.a.smithi151.stdout:May 21 21:50:56 smithi151 ceph-mds[71371]: -- 172.21.15.151:0/1515896196 <== mon.1 v2:172.21.15.184:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564be5fd7a40 con 0x564be5faa800 2024-05-21T21:50:56.453 INFO:journalctl@ceph.mds.a.smithi151.stdout:May 21 21:50:56 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-a[71364]: starting mds.a at 2024-05-21T21:50:56.454 INFO:journalctl@ceph.mds.a.smithi151.stdout:May 21 21:50:56 smithi151 bash[71335]: 257d6d88cf359fe1af275777577778de79cd6c14ba68e0f7de4aa079c280fc5e 2024-05-21T21:50:56.454 INFO:journalctl@ceph.mds.a.smithi151.stdout:May 21 21:50:56 smithi151 systemd[1]: Started Ceph mds.a for c8c7a2e6-17ba-11ef-bc9a-c7b262605968. 2024-05-21T21:50:56.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:56 smithi164 ceph-mon[36269]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:50:56.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:56 smithi184 ceph-mon[36942]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:50:56.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:56.861+0000 7f6690600700 1 -- 172.21.15.151:0/1942965687 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6688078390 msgr2=0x7f6688113ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:50:56.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:56.861+0000 7f6690600700 1 --2- 172.21.15.151:0/1942965687 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6688078390 0x7f6688113ec0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6674009990 tx=0x7f6674009ca0 comp rx=0 tx=0).stop 2024-05-21T21:50:56.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:56.861+0000 7f6690600700 1 -- 172.21.15.151:0/1942965687 shutdown_connections 2024-05-21T21:50:56.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:56.861+0000 7f6690600700 1 --2- 172.21.15.151:0/1942965687 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6688077a70 0x7f6688077e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:56.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:56.861+0000 7f6690600700 1 --2- 172.21.15.151:0/1942965687 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6688114400 0x7f66881167f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:56.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:56.861+0000 7f6690600700 1 --2- 172.21.15.151:0/1942965687 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6688078390 0x7f6688113ec0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:56.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:56.861+0000 7f6690600700 1 -- 172.21.15.151:0/1942965687 >> 172.21.15.151:0/1942965687 conn(0x7f6688101270 msgr2=0x7f6688103690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:50:56.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:56.861+0000 7f6690600700 1 -- 172.21.15.151:0/1942965687 shutdown_connections 2024-05-21T21:50:56.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:56.861+0000 7f6690600700 1 -- 172.21.15.151:0/1942965687 wait complete. 2024-05-21T21:50:56.865 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:56.862+0000 7f6690600700 1 Processor -- start 2024-05-21T21:50:56.865 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:56.863+0000 7f6690600700 1 -- start start 2024-05-21T21:50:56.865 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:56.863+0000 7f6690600700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6688077a70 0x7f66881a0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:50:56.865 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:56.863+0000 7f6690600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6688078390 0x7f66881a0870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:50:56.865 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:56.863+0000 7f6690600700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6688114400 0x7f66881a4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:50:56.865 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:56.863+0000 7f6690600700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6688119de0 con 0x7f6688078390 2024-05-21T21:50:56.865 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:56.863+0000 7f6690600700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6688119c60 con 0x7f6688077a70 2024-05-21T21:50:56.865 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:56.863+0000 7f6690600700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6688119f60 con 0x7f6688114400 2024-05-21T21:50:56.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:56.863+0000 7f668dc00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6688077a70 0x7f66881a0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:50:56.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:56.863+0000 7f668d200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6688078390 0x7f66881a0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:50:56.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:56.863+0000 7f668dc00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6688077a70 0x7f66881a0330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:53470/0 (socket says 172.21.15.151:53470) 2024-05-21T21:50:56.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:56.863+0000 7f668dc00700 1 -- 172.21.15.151:0/3592124076 learned_addr learned my addr 172.21.15.151:0/3592124076 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:50:56.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:56.863+0000 7f668e600700 1 --2- 172.21.15.151:0/3592124076 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6688114400 0x7f66881a4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:50:56.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:56.864+0000 7f668dc00700 1 -- 172.21.15.151:0/3592124076 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6688114400 msgr2=0x7f66881a4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:50:56.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:56.864+0000 7f668dc00700 1 --2- 172.21.15.151:0/3592124076 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6688114400 0x7f66881a4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:56.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:56.864+0000 7f668dc00700 1 -- 172.21.15.151:0/3592124076 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6688078390 msgr2=0x7f66881a0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:50:56.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:56.864+0000 7f668dc00700 1 --2- 172.21.15.151:0/3592124076 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6688078390 0x7f66881a0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:56.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:56.864+0000 7f668dc00700 1 -- 172.21.15.151:0/3592124076 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66881a5340 con 0x7f6688077a70 2024-05-21T21:50:56.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:56.864+0000 7f668dc00700 1 --2- 172.21.15.151:0/3592124076 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6688077a70 0x7f66881a0330 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f667c00e9a0 tx=0x7f667c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:50:56.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:56.864+0000 7f6686000700 1 -- 172.21.15.151:0/3592124076 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f667c00ca10 con 0x7f6688077a70 2024-05-21T21:50:56.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:56.864+0000 7f6686000700 1 -- 172.21.15.151:0/3592124076 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f667c00cbb0 con 0x7f6688077a70 2024-05-21T21:50:56.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:56.864+0000 7f6686000700 1 -- 172.21.15.151:0/3592124076 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f667c018680 con 0x7f6688077a70 2024-05-21T21:50:56.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:56.864+0000 7f6690600700 1 -- 172.21.15.151:0/3592124076 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66881a5630 con 0x7f6688077a70 2024-05-21T21:50:56.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:56.865+0000 7f6690600700 1 -- 172.21.15.151:0/3592124076 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66881aceb0 con 0x7f6688077a70 2024-05-21T21:50:56.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:56.866+0000 7f6690600700 1 -- 172.21.15.151:0/3592124076 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f668806a230 con 0x7f6688077a70 2024-05-21T21:50:56.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:56.868+0000 7f6686000700 1 -- 172.21.15.151:0/3592124076 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f667c010430 con 0x7f6688077a70 2024-05-21T21:50:56.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:56.869+0000 7f6686000700 1 --2- 172.21.15.151:0/3592124076 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f66700b2880 0x7f66700b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:50:56.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:56.869+0000 7f6686000700 1 -- 172.21.15.151:0/3592124076 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f667c0e1f80 con 0x7f6688077a70 2024-05-21T21:50:56.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:56.869+0000 7f668d200700 1 --2- 172.21.15.151:0/3592124076 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f66700b2880 0x7f66700b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:50:56.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:56.870+0000 7f668d200700 1 --2- 172.21.15.151:0/3592124076 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f66700b2880 0x7f66700b4d40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f668807a870 tx=0x7f6674005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:50:56.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:56.873+0000 7f6686000700 1 -- 172.21.15.151:0/3592124076 <== mon.2 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f667c0aa0a0 con 0x7f6688077a70 2024-05-21T21:50:57.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:57.151+0000 7f6690600700 1 -- 172.21.15.151:0/3592124076 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f668802e560 con 0x7f6688077a70 2024-05-21T21:50:57.152 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:57.152+0000 7f6686000700 1 -- 172.21.15.151:0/3592124076 <== mon.2 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9677 (secure 0 0 0) 0x7f667c0b2120 con 0x7f6688077a70 2024-05-21T21:50:57.152 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T21:50:57.153 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14358,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/158010529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":158010529},{"type":"v1","addr":"172.21.15.151:6835","nonce":158010529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6839/2069058115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":2069058115},{"type":"v1","addr":"172.21.15.151:6839","nonce":2069058115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14403,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6841/1749723794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6840","nonce":1749723794},{"type":"v1","addr":"172.21.15.151:6841","nonce":1749723794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/2026266114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":2026266114},{"type":"v1","addr":"172.21.15.184:6833","nonce":2026266114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/2467900844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":2467900844},{"type":"v1","addr":"172.21.15.164:6833","nonce":2467900844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24461,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6837/3367640960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3367640960},{"type":"v1","addr":"172.21.15.151:6837","nonce":3367640960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/3264142621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3264142621},{"type":"v1","addr":"172.21.15.184:6835","nonce":3264142621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24473,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2340428470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2340428470},{"type":"v1","addr":"172.21.15.164:6835","nonce":2340428470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/2988194888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":2988194888},{"type":"v1","addr":"172.21.15.184:6837","nonce":2988194888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24491,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/1554609572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1554609572},{"type":"v1","addr":"172.21.15.164:6837","nonce":1554609572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24493,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6839/3059935143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":3059935143},{"type":"v1","addr":"172.21.15.184:6839","nonce":3059935143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-05-21T21:50:57.155 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:57.155+0000 7f6690600700 1 -- 172.21.15.151:0/3592124076 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f66700b2880 msgr2=0x7f66700b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:50:57.155 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:57.155+0000 7f6690600700 1 --2- 172.21.15.151:0/3592124076 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f66700b2880 0x7f66700b4d40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f668807a870 tx=0x7f6674005fd0 comp rx=0 tx=0).stop 2024-05-21T21:50:57.155 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:57.155+0000 7f6690600700 1 -- 172.21.15.151:0/3592124076 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6688077a70 msgr2=0x7f66881a0330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:50:57.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:57.155+0000 7f6690600700 1 --2- 172.21.15.151:0/3592124076 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6688077a70 0x7f66881a0330 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f667c00e9a0 tx=0x7f667c00ecb0 comp rx=0 tx=0).stop 2024-05-21T21:50:57.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:57.155+0000 7f6690600700 1 -- 172.21.15.151:0/3592124076 shutdown_connections 2024-05-21T21:50:57.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:57.155+0000 7f6690600700 1 --2- 172.21.15.151:0/3592124076 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f66700b2880 0x7f66700b4d40 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:57.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:57.155+0000 7f6690600700 1 --2- 172.21.15.151:0/3592124076 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6688077a70 0x7f66881a0330 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:57.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:57.155+0000 7f6690600700 1 --2- 172.21.15.151:0/3592124076 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6688114400 0x7f66881a4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:57.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:57.155+0000 7f6690600700 1 --2- 172.21.15.151:0/3592124076 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6688078390 0x7f66881a0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:50:57.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:57.155+0000 7f6690600700 1 -- 172.21.15.151:0/3592124076 >> 172.21.15.151:0/3592124076 conn(0x7f6688101270 msgr2=0x7f6688103660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:50:57.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:57.155+0000 7f6690600700 1 -- 172.21.15.151:0/3592124076 shutdown_connections 2024-05-21T21:50:57.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:50:57.155+0000 7f6690600700 1 -- 172.21.15.151:0/3592124076 wait complete. 2024-05-21T21:50:57.157 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 12 2024-05-21T21:50:57.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:57 smithi151 ceph-mon[31623]: mds.? [v2:172.21.15.151:6840/1749723794,v1:172.21.15.151:6841/1749723794] up:boot 2024-05-21T21:50:57.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:57 smithi151 ceph-mon[31623]: fsmap 11 up:standby 2024-05-21T21:50:57.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:57 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-21T21:50:57.454 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:57 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:57.454 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:57 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:57.454 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:57 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:57.454 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:57 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T21:50:57.454 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:57 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T21:50:57.454 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:57 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:50:57.454 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:57 smithi151 ceph-mon[31623]: Deploying daemon mds.b on smithi164 2024-05-21T21:50:57.454 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:57 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/3592124076' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T21:50:57.473 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:57 smithi164 ceph-mon[36269]: mds.? [v2:172.21.15.151:6840/1749723794,v1:172.21.15.151:6841/1749723794] up:boot 2024-05-21T21:50:57.473 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:57 smithi164 ceph-mon[36269]: fsmap 11 up:standby 2024-05-21T21:50:57.473 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:57 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-21T21:50:57.473 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:57 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:57.473 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:57 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:57.473 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:57 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:57.473 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:57 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T21:50:57.473 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:57 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T21:50:57.473 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:57 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:50:57.473 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:57 smithi164 ceph-mon[36269]: Deploying daemon mds.b on smithi164 2024-05-21T21:50:57.473 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:57 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/3592124076' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T21:50:57.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:57 smithi184 ceph-mon[36942]: mds.? [v2:172.21.15.151:6840/1749723794,v1:172.21.15.151:6841/1749723794] up:boot 2024-05-21T21:50:57.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:57 smithi184 ceph-mon[36942]: fsmap 11 up:standby 2024-05-21T21:50:57.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:57 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-21T21:50:57.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:57 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:57.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:57 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:57.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:57 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:50:57.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:57 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-21T21:50:57.737 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:57 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-21T21:50:57.737 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:57 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:50:57.737 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:57 smithi184 ceph-mon[36942]: Deploying daemon mds.b on smithi164 2024-05-21T21:50:57.737 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:57 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/3592124076' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T21:50:58.762 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:50:58 smithi164 ceph-mon[36269]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:50:58.762 INFO:journalctl@ceph.mds.b.smithi164.stdout:May 21 21:50:58 smithi164 systemd[1]: Starting Ceph mds.b for c8c7a2e6-17ba-11ef-bc9a-c7b262605968... 2024-05-21T21:50:58.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:50:58 smithi151 ceph-mon[31623]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:50:58.985 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:50:58 smithi184 ceph-mon[36942]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:50:59.037 INFO:journalctl@ceph.mds.b.smithi164.stdout:May 21 21:50:58 smithi164 podman[66226]: 2024-05-21 21:50:58.761662916 +0000 UTC m=+0.106241074 container create 8598cc3b89fc922a00957b9961e3deb1bb83bcd9ff7cce7f4b4ca855d4223593 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2) 2024-05-21T21:50:59.037 INFO:journalctl@ceph.mds.b.smithi164.stdout:May 21 21:50:58 smithi164 podman[66226]: 2024-05-21 21:50:58.672424938 +0000 UTC m=+0.017003116 image pull d6bc8f31cbc1a7165ca248fe9ba0b06ea14b30f46e1ea33ced1a5238e247af1f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28 2024-05-21T21:50:59.037 INFO:journalctl@ceph.mds.b.smithi164.stdout:May 21 21:50:58 smithi164 podman[66226]: 2024-05-21 21:50:58.964214194 +0000 UTC m=+0.308792361 container init 8598cc3b89fc922a00957b9961e3deb1bb83bcd9ff7cce7f4b4ca855d4223593 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-b, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True) 2024-05-21T21:50:59.037 INFO:journalctl@ceph.mds.b.smithi164.stdout:May 21 21:50:58 smithi164 podman[66226]: 2024-05-21 21:50:58.967020701 +0000 UTC m=+0.311598874 container start 8598cc3b89fc922a00957b9961e3deb1bb83bcd9ff7cce7f4b4ca855d4223593 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-b, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-05-21T21:50:59.037 INFO:journalctl@ceph.mds.b.smithi164.stdout:May 21 21:50:58 smithi164 ceph-mds[66245]: -- 172.21.15.164:0/3110743864 <== mon.2 v2:172.21.15.164:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b20df21a40 con 0x55b20def5000 2024-05-21T21:50:59.037 INFO:journalctl@ceph.mds.b.smithi164.stdout:May 21 21:50:58 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-b[66241]: starting mds.b at 2024-05-21T21:50:59.536 INFO:journalctl@ceph.mds.b.smithi164.stdout:May 21 21:50:59 smithi164 bash[66226]: 8598cc3b89fc922a00957b9961e3deb1bb83bcd9ff7cce7f4b4ca855d4223593 2024-05-21T21:50:59.536 INFO:journalctl@ceph.mds.b.smithi164.stdout:May 21 21:50:59 smithi164 systemd[1]: Started Ceph mds.b for c8c7a2e6-17ba-11ef-bc9a-c7b262605968. 2024-05-21T21:51:00.117 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-21T21:51:00.371 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:00 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:51:00.371 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:00 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:51:00.371 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:00 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:51:00.371 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:00 smithi164 ceph-mon[36269]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:51:00.371 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:00 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:51:00.371 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:00 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:51:00.371 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:00 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:51:00.371 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:00 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:51:00.371 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:00 smithi164 ceph-mon[36269]: mds.? [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] up:boot 2024-05-21T21:51:00.372 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:00 smithi164 ceph-mon[36269]: fsmap 12 up:standby 2024-05-21T21:51:00.372 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:00 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-21T21:51:00.421 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:00 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:51:00.454 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:00 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:51:00.454 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:00 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:51:00.454 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:00 smithi184 ceph-mon[36942]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:51:00.454 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:00 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:51:00.454 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:00 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:51:00.454 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:00 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:51:00.454 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:00 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:51:00.454 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:00 smithi184 ceph-mon[36942]: mds.? [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] up:boot 2024-05-21T21:51:00.454 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:00 smithi184 ceph-mon[36942]: fsmap 12 up:standby 2024-05-21T21:51:00.454 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:00 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-21T21:51:00.455 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:00 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:51:00.455 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:00 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:51:00.455 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:00 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:51:00.455 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:00 smithi151 ceph-mon[31623]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:51:00.455 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:00 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:51:00.455 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:00 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:51:00.455 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:00 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:51:00.456 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:00 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:51:00.456 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:00 smithi151 ceph-mon[31623]: mds.? [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] up:boot 2024-05-21T21:51:00.456 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:00 smithi151 ceph-mon[31623]: fsmap 12 up:standby 2024-05-21T21:51:00.456 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:00 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-21T21:51:01.321 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:51:02.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:02 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:51:02.454 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:02 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:51:02.454 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:02 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:51:02.454 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:02 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:51:02.454 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:02 smithi151 ceph-mon[31623]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:51:02.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:02 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:51:02.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:02 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:51:02.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:02 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:51:02.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:02 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:51:02.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:02 smithi184 ceph-mon[36942]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:51:02.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:02 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:51:02.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:02 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:51:02.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:02 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:51:02.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:02 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:51:02.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:02 smithi164 ceph-mon[36269]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:51:03.309 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:03.309+0000 7f687e600700 1 -- 172.21.15.151:0/1054565742 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f68781095b0 msgr2=0x7f6878109990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:51:03.310 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:03.309+0000 7f687e600700 1 --2- 172.21.15.151:0/1054565742 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f68781095b0 0x7f6878109990 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6868009990 tx=0x7f6868009ca0 comp rx=0 tx=0).stop 2024-05-21T21:51:03.310 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:03.309+0000 7f687e600700 1 -- 172.21.15.151:0/1054565742 shutdown_connections 2024-05-21T21:51:03.310 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:03.309+0000 7f687e600700 1 --2- 172.21.15.151:0/1054565742 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f687806bbe0 0x7f68781103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:03.310 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:03.309+0000 7f687e600700 1 --2- 172.21.15.151:0/1054565742 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f68781095b0 0x7f6878109990 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:03.310 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:03.309+0000 7f687e600700 1 --2- 172.21.15.151:0/1054565742 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f687806b220 0x7f687806b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:03.310 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:03.309+0000 7f687e600700 1 -- 172.21.15.151:0/1054565742 >> 172.21.15.151:0/1054565742 conn(0x7f68780793f0 msgr2=0x7f6878079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:51:03.310 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:03.309+0000 7f687e600700 1 -- 172.21.15.151:0/1054565742 shutdown_connections 2024-05-21T21:51:03.310 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:03.309+0000 7f687e600700 1 -- 172.21.15.151:0/1054565742 wait complete. 2024-05-21T21:51:03.311 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:03.311+0000 7f687e600700 1 Processor -- start 2024-05-21T21:51:03.311 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:03.311+0000 7f687e600700 1 -- start start 2024-05-21T21:51:03.312 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:03.311+0000 7f687e600700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f687806b220 0x7f687819beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:51:03.312 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:03.311+0000 7f687e600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f687806bbe0 0x7f687819c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:51:03.312 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:03.311+0000 7f687e600700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f68781095b0 0x7f68781a0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:51:03.312 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:03.311+0000 7f687e600700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6878113720 con 0x7f687806bbe0 2024-05-21T21:51:03.312 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:03.311+0000 7f687e600700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f68781135a0 con 0x7f68781095b0 2024-05-21T21:51:03.312 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:03.311+0000 7f687e600700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f68781138a0 con 0x7f687806b220 2024-05-21T21:51:03.312 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:03.312+0000 7f6876a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f687806bbe0 0x7f687819c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:51:03.312 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:03.312+0000 7f6877400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f687806b220 0x7f687819beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:51:03.312 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:03.312+0000 7f6877400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f687806b220 0x7f687819beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:50134/0 (socket says 172.21.15.151:50134) 2024-05-21T21:51:03.312 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:03.312+0000 7f6876a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f687806bbe0 0x7f687819c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:43342/0 (socket says 172.21.15.151:43342) 2024-05-21T21:51:03.312 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:03.312+0000 7f6877400700 1 -- 172.21.15.151:0/3230135313 learned_addr learned my addr 172.21.15.151:0/3230135313 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:51:03.313 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:03.312+0000 7f6877e00700 1 --2- 172.21.15.151:0/3230135313 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f68781095b0 0x7f68781a0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:51:03.314 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:03.312+0000 7f6877400700 1 -- 172.21.15.151:0/3230135313 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f68781095b0 msgr2=0x7f68781a0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:51:03.314 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:03.312+0000 7f6877400700 1 --2- 172.21.15.151:0/3230135313 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f68781095b0 0x7f68781a0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:03.314 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:03.312+0000 7f6877400700 1 -- 172.21.15.151:0/3230135313 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f687806bbe0 msgr2=0x7f687819c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:51:03.314 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:03.312+0000 7f6877400700 1 --2- 172.21.15.151:0/3230135313 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f687806bbe0 0x7f687819c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:03.314 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:03.312+0000 7f6877400700 1 -- 172.21.15.151:0/3230135313 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68781a0e70 con 0x7f687806b220 2024-05-21T21:51:03.314 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:03.313+0000 7f6876a00700 1 --2- 172.21.15.151:0/3230135313 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f687806bbe0 0x7f687819c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:51:03.314 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:03.313+0000 7f6877400700 1 --2- 172.21.15.151:0/3230135313 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f687806b220 0x7f687819beb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6868000c00 tx=0x7f68680057c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:51:03.314 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:03.313+0000 7f686fe00700 1 -- 172.21.15.151:0/3230135313 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6868027070 con 0x7f687806b220 2024-05-21T21:51:03.314 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:03.313+0000 7f686fe00700 1 -- 172.21.15.151:0/3230135313 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6868022470 con 0x7f687806b220 2024-05-21T21:51:03.314 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:03.313+0000 7f687e600700 1 -- 172.21.15.151:0/3230135313 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68781a1100 con 0x7f687806b220 2024-05-21T21:51:03.314 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:03.313+0000 7f686fe00700 1 -- 172.21.15.151:0/3230135313 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6868017650 con 0x7f687806b220 2024-05-21T21:51:03.314 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:03.314+0000 7f687e600700 1 -- 172.21.15.151:0/3230135313 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68781a89e0 con 0x7f687806b220 2024-05-21T21:51:03.315 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:03.315+0000 7f687e600700 1 -- 172.21.15.151:0/3230135313 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f687806a230 con 0x7f687806b220 2024-05-21T21:51:03.317 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:03.317+0000 7f686fe00700 1 -- 172.21.15.151:0/3230135313 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f686802a070 con 0x7f687806b220 2024-05-21T21:51:03.318 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:03.318+0000 7f686fe00700 1 --2- 172.21.15.151:0/3230135313 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f685c0b2880 0x7f685c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:51:03.318 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:03.318+0000 7f686fe00700 1 -- 172.21.15.151:0/3230135313 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f68680e2b30 con 0x7f687806b220 2024-05-21T21:51:03.319 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:03.319+0000 7f6876a00700 1 --2- 172.21.15.151:0/3230135313 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f685c0b2880 0x7f685c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:51:03.319 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:03.319+0000 7f6876a00700 1 --2- 172.21.15.151:0/3230135313 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f685c0b2880 0x7f685c0b4d40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f68781107a0 tx=0x7f686400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:51:03.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:03.322+0000 7f686fe00700 1 -- 172.21.15.151:0/3230135313 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6868013070 con 0x7f687806b220 2024-05-21T21:51:03.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:03.625+0000 7f687e600700 1 -- 172.21.15.151:0/3230135313 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f68781a8bb0 con 0x7f687806b220 2024-05-21T21:51:03.626 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:03.626+0000 7f686fe00700 1 -- 172.21.15.151:0/3230135313 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10506 (secure 0 0 0) 0x7f68680aace0 con 0x7f687806b220 2024-05-21T21:51:03.626 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T21:51:03.626 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14358,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/158010529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":158010529},{"type":"v1","addr":"172.21.15.151:6835","nonce":158010529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6839/2069058115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":2069058115},{"type":"v1","addr":"172.21.15.151:6839","nonce":2069058115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14403,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6841/1749723794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6840","nonce":1749723794},{"type":"v1","addr":"172.21.15.151:6841","nonce":1749723794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/2026266114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":2026266114},{"type":"v1","addr":"172.21.15.184:6833","nonce":2026266114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6833/2467900844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":2467900844},{"type":"v1","addr":"172.21.15.164:6833","nonce":2467900844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24461,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6837/3367640960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3367640960},{"type":"v1","addr":"172.21.15.151:6837","nonce":3367640960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/3264142621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3264142621},{"type":"v1","addr":"172.21.15.184:6835","nonce":3264142621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24473,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/2340428470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2340428470},{"type":"v1","addr":"172.21.15.164:6835","nonce":2340428470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/2988194888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":2988194888},{"type":"v1","addr":"172.21.15.184:6837","nonce":2988194888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24491,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/1554609572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1554609572},{"type":"v1","addr":"172.21.15.164:6837","nonce":1554609572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24493,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6839/3059935143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":3059935143},{"type":"v1","addr":"172.21.15.184:6839","nonce":3059935143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24530,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6839/2646401247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":2646401247},{"type":"v1","addr":"172.21.15.164:6839","nonce":2646401247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-05-21T21:51:03.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:03.629+0000 7f687e600700 1 -- 172.21.15.151:0/3230135313 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f685c0b2880 msgr2=0x7f685c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:51:03.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:03.629+0000 7f687e600700 1 --2- 172.21.15.151:0/3230135313 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f685c0b2880 0x7f685c0b4d40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f68781107a0 tx=0x7f686400a380 comp rx=0 tx=0).stop 2024-05-21T21:51:03.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:03.629+0000 7f687e600700 1 -- 172.21.15.151:0/3230135313 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f687806b220 msgr2=0x7f687819beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:51:03.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:03.629+0000 7f687e600700 1 --2- 172.21.15.151:0/3230135313 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f687806b220 0x7f687819beb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6868000c00 tx=0x7f68680057c0 comp rx=0 tx=0).stop 2024-05-21T21:51:03.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:03.629+0000 7f687e600700 1 -- 172.21.15.151:0/3230135313 shutdown_connections 2024-05-21T21:51:03.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:03.629+0000 7f687e600700 1 --2- 172.21.15.151:0/3230135313 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f685c0b2880 0x7f685c0b4d40 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:03.630 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:03.629+0000 7f687e600700 1 --2- 172.21.15.151:0/3230135313 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f68781095b0 0x7f68781a0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:03.630 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:03.629+0000 7f687e600700 1 --2- 172.21.15.151:0/3230135313 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f687806b220 0x7f687819beb0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:03.630 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:03.629+0000 7f687e600700 1 --2- 172.21.15.151:0/3230135313 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f687806bbe0 0x7f687819c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:03.630 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:03.629+0000 7f687e600700 1 -- 172.21.15.151:0/3230135313 >> 172.21.15.151:0/3230135313 conn(0x7f68780793f0 msgr2=0x7f6878100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:51:03.630 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:03.629+0000 7f687e600700 1 -- 172.21.15.151:0/3230135313 shutdown_connections 2024-05-21T21:51:03.630 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:03.629+0000 7f687e600700 1 -- 172.21.15.151:0/3230135313 wait complete. 2024-05-21T21:51:03.630 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 13 2024-05-21T21:51:04.430 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-05-21T21:51:04.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:04 smithi151 ceph-mon[31623]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:51:04.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:04 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:51:04.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:04 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:51:04.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:04 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:51:04.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:04 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:51:04.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:04 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:51:04.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:04 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/3230135313' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T21:51:04.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:04 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:51:04.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:04 smithi164 ceph-mon[36269]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:51:04.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:04 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:51:04.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:04 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:51:04.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:04 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:51:04.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:04 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:51:04.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:04 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:51:04.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:04 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/3230135313' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T21:51:04.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:04 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:51:04.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:04 smithi184 ceph-mon[36942]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:51:04.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:04 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:51:04.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:04 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:51:04.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:04 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:51:04.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:04 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:51:04.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:04 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:51:04.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:04 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/3230135313' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T21:51:04.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:04 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:51:05.498 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:51:06.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:06 smithi184 ceph-mon[36942]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:51:06.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:06 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:51:06.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:06 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:51:06.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:06 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:51:06.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:06 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:51:06.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:06 smithi164 ceph-mon[36269]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:51:06.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:06 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:51:06.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:06 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:51:06.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:06 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:51:06.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:06 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:51:06.917 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:06 smithi151 ceph-mon[31623]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:51:06.917 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:06 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:51:06.917 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:06 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:51:06.917 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:06 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:51:06.917 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:06 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:51:07.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:07.637+0000 7f0f57200700 1 -- 172.21.15.151:0/1018846869 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0f58103e20 msgr2=0x7f0f58110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:51:07.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:07.637+0000 7f0f57200700 1 --2- 172.21.15.151:0/1018846869 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0f58103e20 0x7f0f58110330 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0f5000b0a0 tx=0x7f0f5000b3b0 comp rx=0 tx=0).stop 2024-05-21T21:51:07.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:07.637+0000 7f0f57200700 1 -- 172.21.15.151:0/1018846869 shutdown_connections 2024-05-21T21:51:07.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:07.637+0000 7f0f57200700 1 --2- 172.21.15.151:0/1018846869 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0f58103e20 0x7f0f58110330 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:07.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:07.637+0000 7f0f57200700 1 --2- 172.21.15.151:0/1018846869 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f581094f0 0x7f0f581098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:07.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:07.637+0000 7f0f57200700 1 --2- 172.21.15.151:0/1018846869 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f58103460 0x7f0f581038e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:07.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:07.637+0000 7f0f57200700 1 -- 172.21.15.151:0/1018846869 >> 172.21.15.151:0/1018846869 conn(0x7f0f580fef50 msgr2=0x7f0f58101370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:51:07.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:07.637+0000 7f0f57200700 1 -- 172.21.15.151:0/1018846869 shutdown_connections 2024-05-21T21:51:07.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:07.637+0000 7f0f57200700 1 -- 172.21.15.151:0/1018846869 wait complete. 2024-05-21T21:51:07.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:07.638+0000 7f0f57200700 1 Processor -- start 2024-05-21T21:51:07.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:07.639+0000 7f0f57200700 1 -- start start 2024-05-21T21:51:07.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:07.639+0000 7f0f57200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f58103460 0x7f0f581a1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:51:07.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:07.639+0000 7f0f57200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0f58103e20 0x7f0f581a1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:51:07.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:07.639+0000 7f0f57200700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f581094f0 0x7f0f5819bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:51:07.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:07.639+0000 7f0f57200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0f58113670 con 0x7f0f58103e20 2024-05-21T21:51:07.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:07.639+0000 7f0f57200700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0f581134f0 con 0x7f0f58103460 2024-05-21T21:51:07.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:07.639+0000 7f0f57200700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0f581137f0 con 0x7f0f581094f0 2024-05-21T21:51:07.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:07.640+0000 7f0f55400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0f58103e20 0x7f0f581a1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:51:07.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:07.640+0000 7f0f55e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f58103460 0x7f0f581a1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:51:07.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:07.640+0000 7f0f55e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f58103460 0x7f0f581a1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:38814/0 (socket says 172.21.15.151:38814) 2024-05-21T21:51:07.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:07.640+0000 7f0f55e00700 1 -- 172.21.15.151:0/2591873544 learned_addr learned my addr 172.21.15.151:0/2591873544 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:51:07.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:07.640+0000 7f0f56800700 1 --2- 172.21.15.151:0/2591873544 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f581094f0 0x7f0f5819bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:51:07.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:07.640+0000 7f0f55e00700 1 -- 172.21.15.151:0/2591873544 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f581094f0 msgr2=0x7f0f5819bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:51:07.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:07.640+0000 7f0f55e00700 1 --2- 172.21.15.151:0/2591873544 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f581094f0 0x7f0f5819bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:07.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:07.640+0000 7f0f55e00700 1 -- 172.21.15.151:0/2591873544 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0f58103e20 msgr2=0x7f0f581a1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:51:07.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:07.640+0000 7f0f55e00700 1 --2- 172.21.15.151:0/2591873544 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0f58103e20 0x7f0f581a1f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:07.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:07.640+0000 7f0f55e00700 1 -- 172.21.15.151:0/2591873544 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f5819c450 con 0x7f0f58103460 2024-05-21T21:51:07.643 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:07.640+0000 7f0f56800700 1 --2- 172.21.15.151:0/2591873544 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f581094f0 0x7f0f5819bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:51:07.643 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:07.640+0000 7f0f55400700 1 --2- 172.21.15.151:0/2591873544 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0f58103e20 0x7f0f581a1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:51:07.643 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:07.641+0000 7f0f55e00700 1 --2- 172.21.15.151:0/2591873544 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f58103460 0x7f0f581a1a50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0f4800e9c0 tx=0x7f0f4800ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:51:07.643 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:07.641+0000 7f0f4ea00700 1 -- 172.21.15.151:0/2591873544 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f4800cbd0 con 0x7f0f58103460 2024-05-21T21:51:07.643 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:07.641+0000 7f0f4ea00700 1 -- 172.21.15.151:0/2591873544 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f4800ee60 con 0x7f0f58103460 2024-05-21T21:51:07.643 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:07.641+0000 7f0f57200700 1 -- 172.21.15.151:0/2591873544 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f5819c740 con 0x7f0f58103460 2024-05-21T21:51:07.643 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:07.641+0000 7f0f4ea00700 1 -- 172.21.15.151:0/2591873544 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f480055c0 con 0x7f0f58103460 2024-05-21T21:51:07.644 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:07.641+0000 7f0f57200700 1 -- 172.21.15.151:0/2591873544 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f581a89d0 con 0x7f0f58103460 2024-05-21T21:51:07.644 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:07.642+0000 7f0f57200700 1 -- 172.21.15.151:0/2591873544 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f58104ba0 con 0x7f0f58103460 2024-05-21T21:51:07.644 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:07.644+0000 7f0f4ea00700 1 -- 172.21.15.151:0/2591873544 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f0f48004750 con 0x7f0f58103460 2024-05-21T21:51:07.646 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:07.646+0000 7f0f4ea00700 1 --2- 172.21.15.151:0/2591873544 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f0f3c0b2900 0x7f0f3c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:51:07.646 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:07.646+0000 7f0f4ea00700 1 -- 172.21.15.151:0/2591873544 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f0f480e1f90 con 0x7f0f58103460 2024-05-21T21:51:07.646 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:07.646+0000 7f0f55400700 1 --2- 172.21.15.151:0/2591873544 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f0f3c0b2900 0x7f0f3c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:51:07.647 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:07.647+0000 7f0f55400700 1 --2- 172.21.15.151:0/2591873544 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f0f3c0b2900 0x7f0f3c0b4dc0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0f5819d270 tx=0x7f0f44009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:51:07.651 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:07.651+0000 7f0f4ea00700 1 -- 172.21.15.151:0/2591873544 <== mon.2 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0f480aa0b0 con 0x7f0f58103460 2024-05-21T21:51:07.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:07.898+0000 7f0f57200700 1 -- 172.21.15.151:0/2591873544 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f0f5819cfe0 con 0x7f0f58103460 2024-05-21T21:51:07.899 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:07.899+0000 7f0f4ea00700 1 -- 172.21.15.151:0/2591873544 <== mon.2 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v73) ==== 84+0+65 (secure 0 0 0) 0x7f0f480a99b0 con 0x7f0f58103460 2024-05-21T21:51:07.899 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T21:51:07.899 INFO:teuthology.orchestra.run.smithi151.stdout:[ 2024-05-21T21:51:07.900 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T21:51:07.900 INFO:teuthology.orchestra.run.smithi151.stdout: "poolnum": 1, 2024-05-21T21:51:07.900 INFO:teuthology.orchestra.run.smithi151.stdout: "poolname": ".mgr" 2024-05-21T21:51:07.900 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T21:51:07.900 INFO:teuthology.orchestra.run.smithi151.stdout:] 2024-05-21T21:51:07.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:07.902+0000 7f0f57200700 1 -- 172.21.15.151:0/2591873544 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f0f3c0b2900 msgr2=0x7f0f3c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:51:07.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:07.902+0000 7f0f57200700 1 --2- 172.21.15.151:0/2591873544 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f0f3c0b2900 0x7f0f3c0b4dc0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0f5819d270 tx=0x7f0f44009260 comp rx=0 tx=0).stop 2024-05-21T21:51:07.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:07.902+0000 7f0f57200700 1 -- 172.21.15.151:0/2591873544 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f58103460 msgr2=0x7f0f581a1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:51:07.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:07.902+0000 7f0f57200700 1 --2- 172.21.15.151:0/2591873544 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f58103460 0x7f0f581a1a50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0f4800e9c0 tx=0x7f0f4800ecd0 comp rx=0 tx=0).stop 2024-05-21T21:51:07.903 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:07.902+0000 7f0f57200700 1 -- 172.21.15.151:0/2591873544 shutdown_connections 2024-05-21T21:51:07.903 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:07.902+0000 7f0f57200700 1 --2- 172.21.15.151:0/2591873544 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f0f3c0b2900 0x7f0f3c0b4dc0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:07.903 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:07.902+0000 7f0f57200700 1 --2- 172.21.15.151:0/2591873544 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f58103460 0x7f0f581a1a50 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:07.903 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:07.902+0000 7f0f57200700 1 --2- 172.21.15.151:0/2591873544 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f581094f0 0x7f0f5819bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:07.903 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:07.902+0000 7f0f57200700 1 --2- 172.21.15.151:0/2591873544 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0f58103e20 0x7f0f581a1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:07.903 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:07.902+0000 7f0f57200700 1 -- 172.21.15.151:0/2591873544 >> 172.21.15.151:0/2591873544 conn(0x7f0f580fef50 msgr2=0x7f0f5810eb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:51:07.903 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:07.902+0000 7f0f57200700 1 -- 172.21.15.151:0/2591873544 shutdown_connections 2024-05-21T21:51:07.903 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:07.902+0000 7f0f57200700 1 -- 172.21.15.151:0/2591873544 wait complete. 2024-05-21T21:51:08.758 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-21T21:51:08.759 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-21T21:51:08.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:08 smithi151 ceph-mon[31623]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:51:08.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:08 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:51:08.954 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:08 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:51:08.954 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:08 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:51:08.954 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:08 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:51:08.954 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:08 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:51:08.954 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:08 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/2591873544' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-21T21:51:09.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:08 smithi164 ceph-mon[36269]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:51:09.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:08 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:51:09.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:08 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:51:09.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:08 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:51:09.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:08 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:51:09.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:08 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:51:09.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:08 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/2591873544' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-21T21:51:09.203 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:51:09.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:08 smithi184 ceph-mon[36942]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:51:09.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:08 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:51:09.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:08 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:51:09.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:08 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:51:09.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:08 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:51:09.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:08 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:51:09.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:08 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/2591873544' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-21T21:51:11.043 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:10 smithi151 ceph-mon[31623]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:51:11.205 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:11.205+0000 7fe461800700 1 -- 172.21.15.151:0/32760042 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe45c06bbe0 msgr2=0x7fe45c1103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:51:11.205 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:11.205+0000 7fe461800700 1 --2- 172.21.15.151:0/32760042 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe45c06bbe0 0x7fe45c1103d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe45400b0a0 tx=0x7fe45400b3b0 comp rx=0 tx=0).stop 2024-05-21T21:51:11.206 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:11.205+0000 7fe461800700 1 -- 172.21.15.151:0/32760042 shutdown_connections 2024-05-21T21:51:11.206 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:11.205+0000 7fe461800700 1 --2- 172.21.15.151:0/32760042 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe45c06bbe0 0x7fe45c1103d0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:11.206 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:11.205+0000 7fe461800700 1 --2- 172.21.15.151:0/32760042 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe45c1095b0 0x7fe45c109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:11.206 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:11.205+0000 7fe461800700 1 --2- 172.21.15.151:0/32760042 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe45c06b220 0x7fe45c06b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:11.206 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:11.205+0000 7fe461800700 1 -- 172.21.15.151:0/32760042 >> 172.21.15.151:0/32760042 conn(0x7fe45c0793f0 msgr2=0x7fe45c079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:51:11.206 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:11.205+0000 7fe461800700 1 -- 172.21.15.151:0/32760042 shutdown_connections 2024-05-21T21:51:11.206 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:11.205+0000 7fe461800700 1 -- 172.21.15.151:0/32760042 wait complete. 2024-05-21T21:51:11.207 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:11.206+0000 7fe461800700 1 Processor -- start 2024-05-21T21:51:11.207 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:11.206+0000 7fe461800700 1 -- start start 2024-05-21T21:51:11.208 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:11.207+0000 7fe461800700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe45c06b220 0x7fe45c19be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:51:11.208 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:11.207+0000 7fe461800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe45c06bbe0 0x7fe45c19c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:51:11.208 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:11.207+0000 7fe461800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe45c1095b0 0x7fe45c1a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:51:11.208 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:11.207+0000 7fe461800700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe45c113720 con 0x7fe45c06bbe0 2024-05-21T21:51:11.208 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:11.207+0000 7fe461800700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe45c1135a0 con 0x7fe45c1095b0 2024-05-21T21:51:11.208 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:11.207+0000 7fe461800700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe45c1138a0 con 0x7fe45c06b220 2024-05-21T21:51:11.208 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:11.207+0000 7fe45aa00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe45c06b220 0x7fe45c19be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:51:11.208 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:11.207+0000 7fe45a000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe45c06bbe0 0x7fe45c19c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:51:11.208 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:11.207+0000 7fe45aa00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe45c06b220 0x7fe45c19be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:59504/0 (socket says 172.21.15.151:59504) 2024-05-21T21:51:11.208 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:11.207+0000 7fe45aa00700 1 -- 172.21.15.151:0/880490739 learned_addr learned my addr 172.21.15.151:0/880490739 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:51:11.208 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:11.207+0000 7fe45b400700 1 --2- 172.21.15.151:0/880490739 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe45c1095b0 0x7fe45c1a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:51:11.208 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:11.207+0000 7fe45aa00700 1 -- 172.21.15.151:0/880490739 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe45c1095b0 msgr2=0x7fe45c1a06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:51:11.208 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:11.207+0000 7fe45aa00700 1 --2- 172.21.15.151:0/880490739 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe45c1095b0 0x7fe45c1a06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:11.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:11.207+0000 7fe45aa00700 1 -- 172.21.15.151:0/880490739 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe45c06bbe0 msgr2=0x7fe45c19c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:51:11.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:11.207+0000 7fe45aa00700 1 --2- 172.21.15.151:0/880490739 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe45c06bbe0 0x7fe45c19c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:11.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:11.208+0000 7fe45aa00700 1 -- 172.21.15.151:0/880490739 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe45c1a0e20 con 0x7fe45c06b220 2024-05-21T21:51:11.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:11.208+0000 7fe45aa00700 1 --2- 172.21.15.151:0/880490739 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe45c06b220 0x7fe45c19be10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe44c00e9a0 tx=0x7fe44c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:51:11.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:11.208+0000 7fe453400700 1 -- 172.21.15.151:0/880490739 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe44c00ca10 con 0x7fe45c06b220 2024-05-21T21:51:11.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:11.208+0000 7fe453400700 1 -- 172.21.15.151:0/880490739 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe44c00cbb0 con 0x7fe45c06b220 2024-05-21T21:51:11.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:11.208+0000 7fe453400700 1 -- 172.21.15.151:0/880490739 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe44c018680 con 0x7fe45c06b220 2024-05-21T21:51:11.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:11.208+0000 7fe461800700 1 -- 172.21.15.151:0/880490739 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe45c1a1110 con 0x7fe45c06b220 2024-05-21T21:51:11.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:11.208+0000 7fe461800700 1 -- 172.21.15.151:0/880490739 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe45c1a8990 con 0x7fe45c06b220 2024-05-21T21:51:11.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:11.209+0000 7fe461800700 1 -- 172.21.15.151:0/880490739 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe45c06a230 con 0x7fe45c06b220 2024-05-21T21:51:11.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:11.212+0000 7fe453400700 1 -- 172.21.15.151:0/880490739 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7fe44c010430 con 0x7fe45c06b220 2024-05-21T21:51:11.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:11.213+0000 7fe453400700 1 --2- 172.21.15.151:0/880490739 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fe4400b2880 0x7fe4400b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:51:11.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:11.213+0000 7fe453400700 1 -- 172.21.15.151:0/880490739 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7fe44c0e1f20 con 0x7fe45c06b220 2024-05-21T21:51:11.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:11.213+0000 7fe45a000700 1 --2- 172.21.15.151:0/880490739 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fe4400b2880 0x7fe4400b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:51:11.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:11.214+0000 7fe45a000700 1 --2- 172.21.15.151:0/880490739 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fe4400b2880 0x7fe4400b4d40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe45c1045d0 tx=0x7fe448009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:51:11.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:11.217+0000 7fe453400700 1 -- 172.21.15.151:0/880490739 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe44c0aa040 con 0x7fe45c06b220 2024-05-21T21:51:11.235 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:10 smithi184 ceph-mon[36942]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:51:11.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:10 smithi164 ceph-mon[36269]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:51:11.433 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:11.432+0000 7fe461800700 1 -- 172.21.15.151:0/880490739 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fe45c1a8b60 con 0x7fe45c06b220 2024-05-21T21:51:11.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:11.837+0000 7fe453400700 1 -- 172.21.15.151:0/880490739 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v74) ==== 138+0+0 (secure 0 0 0) 0x7fe44c0b20c0 con 0x7fe45c06b220 2024-05-21T21:51:11.844 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:11.843+0000 7fe461800700 1 -- 172.21.15.151:0/880490739 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fe4400b2880 msgr2=0x7fe4400b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:51:11.844 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:11.843+0000 7fe461800700 1 --2- 172.21.15.151:0/880490739 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fe4400b2880 0x7fe4400b4d40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe45c1045d0 tx=0x7fe448009260 comp rx=0 tx=0).stop 2024-05-21T21:51:11.844 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:11.843+0000 7fe461800700 1 -- 172.21.15.151:0/880490739 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe45c06b220 msgr2=0x7fe45c19be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:51:11.844 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:11.843+0000 7fe461800700 1 --2- 172.21.15.151:0/880490739 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe45c06b220 0x7fe45c19be10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe44c00e9a0 tx=0x7fe44c00ecb0 comp rx=0 tx=0).stop 2024-05-21T21:51:11.844 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:11.844+0000 7fe461800700 1 -- 172.21.15.151:0/880490739 shutdown_connections 2024-05-21T21:51:11.844 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:11.844+0000 7fe461800700 1 --2- 172.21.15.151:0/880490739 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fe4400b2880 0x7fe4400b4d40 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:11.844 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:11.844+0000 7fe461800700 1 --2- 172.21.15.151:0/880490739 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe45c1095b0 0x7fe45c1a06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:11.845 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:11.844+0000 7fe461800700 1 --2- 172.21.15.151:0/880490739 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe45c06b220 0x7fe45c19be10 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:11.845 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:11.844+0000 7fe461800700 1 --2- 172.21.15.151:0/880490739 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe45c06bbe0 0x7fe45c19c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:11.845 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:11.844+0000 7fe461800700 1 -- 172.21.15.151:0/880490739 >> 172.21.15.151:0/880490739 conn(0x7fe45c0793f0 msgr2=0x7fe45c10e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:51:11.845 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:11.844+0000 7fe461800700 1 -- 172.21.15.151:0/880490739 shutdown_connections 2024-05-21T21:51:11.845 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:11.844+0000 7fe461800700 1 -- 172.21.15.151:0/880490739 wait complete. 2024-05-21T21:51:11.845 INFO:teuthology.orchestra.run.smithi151.stderr:pool 'cephfs_metadata' created 2024-05-21T21:51:12.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:11 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/880490739' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-21T21:51:12.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:11 smithi151 ceph-mon[31623]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-21T21:51:12.235 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:11 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/880490739' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-21T21:51:12.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:11 smithi184 ceph-mon[36942]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-21T21:51:12.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:11 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/880490739' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-21T21:51:12.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:11 smithi164 ceph-mon[36269]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-21T21:51:12.719 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-21T21:51:12.952 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:12 smithi151 ceph-mon[31623]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:51:12.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:12 smithi151 ceph-mon[31623]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-21T21:51:12.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:12 smithi151 ceph-mon[31623]: osdmap e74: 12 total, 12 up, 12 in 2024-05-21T21:51:13.183 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:51:13.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:12 smithi184 ceph-mon[36942]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:51:13.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:12 smithi184 ceph-mon[36942]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-21T21:51:13.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:12 smithi184 ceph-mon[36942]: osdmap e74: 12 total, 12 up, 12 in 2024-05-21T21:51:13.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:12 smithi164 ceph-mon[36269]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:51:13.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:12 smithi164 ceph-mon[36269]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-21T21:51:13.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:12 smithi164 ceph-mon[36269]: osdmap e74: 12 total, 12 up, 12 in 2024-05-21T21:51:13.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:13 smithi151 ceph-mon[31623]: osdmap e75: 12 total, 12 up, 12 in 2024-05-21T21:51:14.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:13 smithi184 ceph-mon[36942]: osdmap e75: 12 total, 12 up, 12 in 2024-05-21T21:51:14.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:13 smithi164 ceph-mon[36269]: osdmap e75: 12 total, 12 up, 12 in 2024-05-21T21:51:15.176 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:14 smithi151 ceph-mon[31623]: pgmap v307: 33 pgs: 3 creating+activating, 2 creating+peering, 6 active+clean, 22 unknown; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:51:15.176 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:14 smithi151 ceph-mon[31623]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-21T21:51:15.176 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:14 smithi151 ceph-mon[31623]: osdmap e76: 12 total, 12 up, 12 in 2024-05-21T21:51:15.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:14 smithi184 ceph-mon[36942]: pgmap v307: 33 pgs: 3 creating+activating, 2 creating+peering, 6 active+clean, 22 unknown; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:51:15.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:14 smithi184 ceph-mon[36942]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-21T21:51:15.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:14 smithi184 ceph-mon[36942]: osdmap e76: 12 total, 12 up, 12 in 2024-05-21T21:51:15.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:14 smithi164 ceph-mon[36269]: pgmap v307: 33 pgs: 3 creating+activating, 2 creating+peering, 6 active+clean, 22 unknown; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:51:15.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:14 smithi164 ceph-mon[36269]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-21T21:51:15.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:14 smithi164 ceph-mon[36269]: osdmap e76: 12 total, 12 up, 12 in 2024-05-21T21:51:15.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:15.354+0000 7f8ae7e00700 1 -- 172.21.15.151:0/2392556996 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ae8075970 msgr2=0x7f8ae8075d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:51:15.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:15.354+0000 7f8ae7e00700 1 --2- 172.21.15.151:0/2392556996 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ae8075970 0x7f8ae8075d50 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8ad8009990 tx=0x7f8ad8009ca0 comp rx=0 tx=0).stop 2024-05-21T21:51:15.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:15.354+0000 7f8ae7e00700 1 -- 172.21.15.151:0/2392556996 shutdown_connections 2024-05-21T21:51:15.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:15.354+0000 7f8ae7e00700 1 --2- 172.21.15.151:0/2392556996 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8ae810fc90 0x7f8ae8112100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:15.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:15.354+0000 7f8ae7e00700 1 --2- 172.21.15.151:0/2392556996 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ae8075970 0x7f8ae8075d50 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:15.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:15.354+0000 7f8ae7e00700 1 --2- 172.21.15.151:0/2392556996 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8ae8076290 0x7f8ae810f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:15.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:15.354+0000 7f8ae7e00700 1 -- 172.21.15.151:0/2392556996 >> 172.21.15.151:0/2392556996 conn(0x7f8ae80fef60 msgr2=0x7f8ae8101380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:51:15.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:15.354+0000 7f8ae7e00700 1 -- 172.21.15.151:0/2392556996 shutdown_connections 2024-05-21T21:51:15.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:15.355+0000 7f8ae7e00700 1 -- 172.21.15.151:0/2392556996 wait complete. 2024-05-21T21:51:15.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:15.355+0000 7f8ae7e00700 1 Processor -- start 2024-05-21T21:51:15.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:15.356+0000 7f8ae7e00700 1 -- start start 2024-05-21T21:51:15.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:15.356+0000 7f8ae7e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8ae8075970 0x7f8ae819bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:51:15.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:15.356+0000 7f8ae7e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ae8076290 0x7f8ae819c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:51:15.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:15.356+0000 7f8ae7e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8ae810fc90 0x7f8ae81a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:51:15.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:15.356+0000 7f8ae7e00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8ae8115860 con 0x7f8ae8076290 2024-05-21T21:51:15.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:15.356+0000 7f8ae7e00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8ae81156e0 con 0x7f8ae8075970 2024-05-21T21:51:15.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:15.356+0000 7f8ae7e00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8ae81159e0 con 0x7f8ae810fc90 2024-05-21T21:51:15.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:15.356+0000 7f8ae6000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ae8076290 0x7f8ae819c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:51:15.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:15.356+0000 7f8ae6a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8ae8075970 0x7f8ae819bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:51:15.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:15.356+0000 7f8ae6000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ae8076290 0x7f8ae819c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:55264/0 (socket says 172.21.15.151:55264) 2024-05-21T21:51:15.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:15.356+0000 7f8ae6a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8ae8075970 0x7f8ae819bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:38856/0 (socket says 172.21.15.151:38856) 2024-05-21T21:51:15.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:15.356+0000 7f8ae6000700 1 -- 172.21.15.151:0/593795560 learned_addr learned my addr 172.21.15.151:0/593795560 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:51:15.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:15.356+0000 7f8ae7400700 1 --2- 172.21.15.151:0/593795560 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8ae810fc90 0x7f8ae81a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:51:15.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:15.357+0000 7f8ae6a00700 1 -- 172.21.15.151:0/593795560 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8ae810fc90 msgr2=0x7f8ae81a0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:51:15.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:15.357+0000 7f8ae6a00700 1 --2- 172.21.15.151:0/593795560 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8ae810fc90 0x7f8ae81a0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:15.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:15.357+0000 7f8ae6a00700 1 -- 172.21.15.151:0/593795560 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ae8076290 msgr2=0x7f8ae819c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:51:15.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:15.357+0000 7f8ae6a00700 1 --2- 172.21.15.151:0/593795560 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ae8076290 0x7f8ae819c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:15.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:15.357+0000 7f8ae6a00700 1 -- 172.21.15.151:0/593795560 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ae81a0d80 con 0x7f8ae8075970 2024-05-21T21:51:15.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:15.357+0000 7f8ae6000700 1 --2- 172.21.15.151:0/593795560 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ae8076290 0x7f8ae819c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:51:15.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:15.357+0000 7f8ae6a00700 1 --2- 172.21.15.151:0/593795560 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8ae8075970 0x7f8ae819bd70 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8ad8000c00 tx=0x7f8ad80047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:51:15.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:15.358+0000 7f8adf400700 1 -- 172.21.15.151:0/593795560 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ad8026070 con 0x7f8ae8075970 2024-05-21T21:51:15.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:15.358+0000 7f8ae7e00700 1 -- 172.21.15.151:0/593795560 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ae81a1010 con 0x7f8ae8075970 2024-05-21T21:51:15.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:15.358+0000 7f8adf400700 1 -- 172.21.15.151:0/593795560 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ad8021410 con 0x7f8ae8075970 2024-05-21T21:51:15.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:15.358+0000 7f8ae7e00700 1 -- 172.21.15.151:0/593795560 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ae81a88f0 con 0x7f8ae8075970 2024-05-21T21:51:15.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:15.358+0000 7f8adf400700 1 -- 172.21.15.151:0/593795560 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ad8017650 con 0x7f8ae8075970 2024-05-21T21:51:15.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:15.359+0000 7f8ae7e00700 1 -- 172.21.15.151:0/593795560 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ae806a230 con 0x7f8ae8075970 2024-05-21T21:51:15.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:15.361+0000 7f8adf400700 1 -- 172.21.15.151:0/593795560 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f8ad8029070 con 0x7f8ae8075970 2024-05-21T21:51:15.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:15.363+0000 7f8adf400700 1 --2- 172.21.15.151:0/593795560 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f8acc0b2900 0x7f8acc0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:51:15.363 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:15.363+0000 7f8adf400700 1 -- 172.21.15.151:0/593795560 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7497+0+0 (secure 0 0 0) 0x7f8ad80e2db0 con 0x7f8ae8075970 2024-05-21T21:51:15.363 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:15.363+0000 7f8ae6000700 1 --2- 172.21.15.151:0/593795560 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f8acc0b2900 0x7f8acc0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:51:15.363 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:15.364+0000 7f8ae6000700 1 --2- 172.21.15.151:0/593795560 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f8acc0b2900 0x7f8acc0b4dc0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8ae81110c0 tx=0x7f8ad4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:51:15.364 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:15.365+0000 7f8adf400700 1 -- 172.21.15.151:0/593795560 <== mon.2 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8ad8030080 con 0x7f8ae8075970 2024-05-21T21:51:15.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:15.601+0000 7f8ae7e00700 1 -- 172.21.15.151:0/593795560 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f8ae802e560 con 0x7f8ae8075970 2024-05-21T21:51:15.899 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:15.897+0000 7f8adf400700 1 -- 172.21.15.151:0/593795560 <== mon.2 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v77) ==== 170+0+0 (secure 0 0 0) 0x7f8ad8013070 con 0x7f8ae8075970 2024-05-21T21:51:15.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:15.907+0000 7f8ae7e00700 1 -- 172.21.15.151:0/593795560 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f8acc0b2900 msgr2=0x7f8acc0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:51:15.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:15.907+0000 7f8ae7e00700 1 --2- 172.21.15.151:0/593795560 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f8acc0b2900 0x7f8acc0b4dc0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8ae81110c0 tx=0x7f8ad4009260 comp rx=0 tx=0).stop 2024-05-21T21:51:15.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:15.907+0000 7f8ae7e00700 1 -- 172.21.15.151:0/593795560 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8ae8075970 msgr2=0x7f8ae819bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:51:15.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:15.907+0000 7f8ae7e00700 1 --2- 172.21.15.151:0/593795560 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8ae8075970 0x7f8ae819bd70 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8ad8000c00 tx=0x7f8ad80047b0 comp rx=0 tx=0).stop 2024-05-21T21:51:15.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:15.908+0000 7f8ae7e00700 1 -- 172.21.15.151:0/593795560 shutdown_connections 2024-05-21T21:51:15.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:15.908+0000 7f8ae7e00700 1 --2- 172.21.15.151:0/593795560 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f8acc0b2900 0x7f8acc0b4dc0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:15.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:15.908+0000 7f8ae7e00700 1 --2- 172.21.15.151:0/593795560 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8ae8075970 0x7f8ae819bd70 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:15.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:15.908+0000 7f8ae7e00700 1 --2- 172.21.15.151:0/593795560 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8ae810fc90 0x7f8ae81a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:15.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:15.908+0000 7f8ae7e00700 1 --2- 172.21.15.151:0/593795560 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ae8076290 0x7f8ae819c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:15.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:15.908+0000 7f8ae7e00700 1 -- 172.21.15.151:0/593795560 >> 172.21.15.151:0/593795560 conn(0x7f8ae80fef60 msgr2=0x7f8ae80ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:51:15.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:15.910+0000 7f8ae7e00700 1 -- 172.21.15.151:0/593795560 shutdown_connections 2024-05-21T21:51:15.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:15.911+0000 7f8ae7e00700 1 -- 172.21.15.151:0/593795560 wait complete. 2024-05-21T21:51:15.914 INFO:teuthology.orchestra.run.smithi151.stderr:pool 'cephfs_data' created 2024-05-21T21:51:16.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:15 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/593795560' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-21T21:51:16.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:15 smithi151 ceph-mon[31623]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-21T21:51:16.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:15 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/593795560' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-21T21:51:16.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:15 smithi184 ceph-mon[36942]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-21T21:51:16.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:15 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/593795560' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-21T21:51:16.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:15 smithi164 ceph-mon[36269]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-21T21:51:16.778 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-21T21:51:16.952 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:16 smithi151 ceph-mon[31623]: pgmap v309: 33 pgs: 3 creating+activating, 2 creating+peering, 12 active+clean, 16 unknown; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:51:16.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:16 smithi151 ceph-mon[31623]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-21T21:51:16.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:16 smithi151 ceph-mon[31623]: osdmap e77: 12 total, 12 up, 12 in 2024-05-21T21:51:17.235 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:16 smithi184 ceph-mon[36942]: pgmap v309: 33 pgs: 3 creating+activating, 2 creating+peering, 12 active+clean, 16 unknown; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:51:17.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:16 smithi184 ceph-mon[36942]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-21T21:51:17.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:16 smithi184 ceph-mon[36942]: osdmap e77: 12 total, 12 up, 12 in 2024-05-21T21:51:17.247 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:51:17.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:16 smithi164 ceph-mon[36269]: pgmap v309: 33 pgs: 3 creating+activating, 2 creating+peering, 12 active+clean, 16 unknown; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:51:17.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:16 smithi164 ceph-mon[36269]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-21T21:51:17.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:16 smithi164 ceph-mon[36269]: osdmap e77: 12 total, 12 up, 12 in 2024-05-21T21:51:18.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:17 smithi151 ceph-mon[31623]: osdmap e78: 12 total, 12 up, 12 in 2024-05-21T21:51:18.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:17 smithi184 ceph-mon[36942]: osdmap e78: 12 total, 12 up, 12 in 2024-05-21T21:51:18.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:17 smithi164 ceph-mon[36269]: osdmap e78: 12 total, 12 up, 12 in 2024-05-21T21:51:19.087 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:18 smithi151 ceph-mon[31623]: pgmap v312: 97 pgs: 3 creating+activating, 23 creating+peering, 38 active+clean, 33 unknown; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:51:19.087 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:18 smithi151 ceph-mon[31623]: osdmap e79: 12 total, 12 up, 12 in 2024-05-21T21:51:19.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:18 smithi184 ceph-mon[36942]: pgmap v312: 97 pgs: 3 creating+activating, 23 creating+peering, 38 active+clean, 33 unknown; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:51:19.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:18 smithi184 ceph-mon[36942]: osdmap e79: 12 total, 12 up, 12 in 2024-05-21T21:51:19.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:19.258+0000 7ff7ece00700 1 -- 172.21.15.151:0/697681597 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff7e806bbe0 msgr2=0x7ff7e81103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:51:19.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:19.258+0000 7ff7ece00700 1 --2- 172.21.15.151:0/697681597 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff7e806bbe0 0x7ff7e81103d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff7e000b0a0 tx=0x7ff7e000b3b0 comp rx=0 tx=0).stop 2024-05-21T21:51:19.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:19.259+0000 7ff7ece00700 1 -- 172.21.15.151:0/697681597 shutdown_connections 2024-05-21T21:51:19.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:19.259+0000 7ff7ece00700 1 --2- 172.21.15.151:0/697681597 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff7e81095b0 0x7ff7e8109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:19.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:19.259+0000 7ff7ece00700 1 --2- 172.21.15.151:0/697681597 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff7e806bbe0 0x7ff7e81103d0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:19.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:19.259+0000 7ff7ece00700 1 --2- 172.21.15.151:0/697681597 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff7e806b220 0x7ff7e806b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:19.260 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:19.259+0000 7ff7ece00700 1 -- 172.21.15.151:0/697681597 >> 172.21.15.151:0/697681597 conn(0x7ff7e80793f0 msgr2=0x7ff7e8079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:51:19.260 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:19.259+0000 7ff7ece00700 1 -- 172.21.15.151:0/697681597 shutdown_connections 2024-05-21T21:51:19.260 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:19.259+0000 7ff7ece00700 1 -- 172.21.15.151:0/697681597 wait complete. 2024-05-21T21:51:19.260 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:19.260+0000 7ff7ece00700 1 Processor -- start 2024-05-21T21:51:19.260 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:19.260+0000 7ff7ece00700 1 -- start start 2024-05-21T21:51:19.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:19.261+0000 7ff7ece00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff7e806b220 0x7ff7e819f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:51:19.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:19.261+0000 7ff7ece00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff7e806bbe0 0x7ff7e819fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:51:19.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:19.261+0000 7ff7ece00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff7e81095b0 0x7ff7e8199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:51:19.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:19.261+0000 7ff7ece00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e8113730 con 0x7ff7e806b220 2024-05-21T21:51:19.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:19.261+0000 7ff7ece00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e81135b0 con 0x7ff7e81095b0 2024-05-21T21:51:19.262 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:19.261+0000 7ff7ece00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e81138b0 con 0x7ff7e806bbe0 2024-05-21T21:51:19.262 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:19.261+0000 7ff7e5e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff7e806b220 0x7ff7e819f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:51:19.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:19.261+0000 7ff7e5e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff7e806b220 0x7ff7e819f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:42118/0 (socket says 172.21.15.151:42118) 2024-05-21T21:51:19.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:19.261+0000 7ff7e5400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff7e806bbe0 0x7ff7e819fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:51:19.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:19.261+0000 7ff7e5e00700 1 -- 172.21.15.151:0/771364888 learned_addr learned my addr 172.21.15.151:0/771364888 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:51:19.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:19.262+0000 7ff7e6800700 1 --2- 172.21.15.151:0/771364888 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff7e81095b0 0x7ff7e8199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:51:19.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:19.262+0000 7ff7e5e00700 1 -- 172.21.15.151:0/771364888 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff7e806bbe0 msgr2=0x7ff7e819fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:51:19.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:19.262+0000 7ff7e5e00700 1 --2- 172.21.15.151:0/771364888 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff7e806bbe0 0x7ff7e819fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:19.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:19.262+0000 7ff7e5e00700 1 -- 172.21.15.151:0/771364888 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff7e81095b0 msgr2=0x7ff7e8199a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:51:19.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:19.262+0000 7ff7e5e00700 1 --2- 172.21.15.151:0/771364888 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff7e81095b0 0x7ff7e8199a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:19.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:19.262+0000 7ff7e5e00700 1 -- 172.21.15.151:0/771364888 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7e819a170 con 0x7ff7e806b220 2024-05-21T21:51:19.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:19.262+0000 7ff7e5e00700 1 --2- 172.21.15.151:0/771364888 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff7e806b220 0x7ff7e819f850 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff7d800e9c0 tx=0x7ff7d800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:51:19.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:19.262+0000 7ff7dea00700 1 -- 172.21.15.151:0/771364888 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d800cbd0 con 0x7ff7e806b220 2024-05-21T21:51:19.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:19.263+0000 7ff7dea00700 1 -- 172.21.15.151:0/771364888 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7d800ee60 con 0x7ff7e806b220 2024-05-21T21:51:19.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:19.263+0000 7ff7ece00700 1 -- 172.21.15.151:0/771364888 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7e819a460 con 0x7ff7e806b220 2024-05-21T21:51:19.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:19.263+0000 7ff7dea00700 1 -- 172.21.15.151:0/771364888 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d80055c0 con 0x7ff7e806b220 2024-05-21T21:51:19.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:19.263+0000 7ff7ece00700 1 -- 172.21.15.151:0/771364888 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7e81a6720 con 0x7ff7e806b220 2024-05-21T21:51:19.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:19.264+0000 7ff7ece00700 1 -- 172.21.15.151:0/771364888 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7e806a230 con 0x7ff7e806b220 2024-05-21T21:51:19.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:19.266+0000 7ff7dea00700 1 -- 172.21.15.151:0/771364888 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7ff7d8004750 con 0x7ff7e806b220 2024-05-21T21:51:19.269 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:19.269+0000 7ff7dea00700 1 --2- 172.21.15.151:0/771364888 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7ff7cc0b2880 0x7ff7cc0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:51:19.269 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:19.269+0000 7ff7dea00700 1 -- 172.21.15.151:0/771364888 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7879+0+0 (secure 0 0 0) 0x7ff7d80e24e0 con 0x7ff7e806b220 2024-05-21T21:51:19.269 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:19.269+0000 7ff7e5400700 1 --2- 172.21.15.151:0/771364888 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7ff7cc0b2880 0x7ff7cc0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:51:19.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:19.270+0000 7ff7e5400700 1 --2- 172.21.15.151:0/771364888 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7ff7cc0b2880 0x7ff7cc0b4d40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff7e80799d0 tx=0x7ff7d4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:51:19.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:19.272+0000 7ff7dea00700 1 -- 172.21.15.151:0/771364888 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff7d8054df0 con 0x7ff7e806b220 2024-05-21T21:51:19.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:18 smithi164 ceph-mon[36269]: pgmap v312: 97 pgs: 3 creating+activating, 23 creating+peering, 38 active+clean, 33 unknown; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:51:19.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:18 smithi164 ceph-mon[36269]: osdmap e79: 12 total, 12 up, 12 in 2024-05-21T21:51:19.550 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:19.550+0000 7ff7ece00700 1 -- 172.21.15.151:0/771364888 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7ff7e802e560 con 0x7ff7e806b220 2024-05-21T21:51:19.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:19.934+0000 7ff7dea00700 1 -- 172.21.15.151:0/771364888 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7ff7d80aa3b0 con 0x7ff7e806b220 2024-05-21T21:51:19.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:19.939+0000 7ff7ece00700 1 -- 172.21.15.151:0/771364888 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7ff7cc0b2880 msgr2=0x7ff7cc0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:51:19.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:19.939+0000 7ff7ece00700 1 --2- 172.21.15.151:0/771364888 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7ff7cc0b2880 0x7ff7cc0b4d40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff7e80799d0 tx=0x7ff7d4009260 comp rx=0 tx=0).stop 2024-05-21T21:51:19.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:19.939+0000 7ff7ece00700 1 -- 172.21.15.151:0/771364888 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff7e806b220 msgr2=0x7ff7e819f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:51:19.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:19.939+0000 7ff7ece00700 1 --2- 172.21.15.151:0/771364888 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff7e806b220 0x7ff7e819f850 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff7d800e9c0 tx=0x7ff7d800ecd0 comp rx=0 tx=0).stop 2024-05-21T21:51:19.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:19.940+0000 7ff7ece00700 1 -- 172.21.15.151:0/771364888 shutdown_connections 2024-05-21T21:51:19.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:19.940+0000 7ff7ece00700 1 --2- 172.21.15.151:0/771364888 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7ff7cc0b2880 0x7ff7cc0b4d40 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:19.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:19.940+0000 7ff7ece00700 1 --2- 172.21.15.151:0/771364888 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff7e81095b0 0x7ff7e8199a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:19.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:19.940+0000 7ff7ece00700 1 --2- 172.21.15.151:0/771364888 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff7e806b220 0x7ff7e819f850 secure :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff7d800e9c0 tx=0x7ff7d800ecd0 comp rx=0 tx=0).stop 2024-05-21T21:51:19.942 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:19.940+0000 7ff7ece00700 1 --2- 172.21.15.151:0/771364888 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff7e806bbe0 0x7ff7e819fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:19.942 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:19.940+0000 7ff7ece00700 1 -- 172.21.15.151:0/771364888 >> 172.21.15.151:0/771364888 conn(0x7ff7e80793f0 msgr2=0x7ff7e8101080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:51:19.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:19.942+0000 7ff7ece00700 1 -- 172.21.15.151:0/771364888 shutdown_connections 2024-05-21T21:51:19.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:19.942+0000 7ff7ece00700 1 -- 172.21.15.151:0/771364888 wait complete. 2024-05-21T21:51:19.946 INFO:teuthology.orchestra.run.smithi151.stderr:new fs with metadata pool 2 and data pool 3 2024-05-21T21:51:20.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:19 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/771364888' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-21T21:51:20.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:19 smithi151 ceph-mon[31623]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-21T21:51:20.238 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:19 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/771364888' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-21T21:51:20.238 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:19 smithi184 ceph-mon[36942]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-21T21:51:20.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:19 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/771364888' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-21T21:51:20.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:19 smithi164 ceph-mon[36269]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-21T21:51:20.828 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph osd dump --format=json 2024-05-21T21:51:21.202 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:20 smithi151 ceph-mon[31623]: pgmap v314: 97 pgs: 23 creating+peering, 48 active+clean, 26 unknown; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:51:21.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:20 smithi151 ceph-mon[31623]: osdmap e80: 12 total, 12 up, 12 in 2024-05-21T21:51:21.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:20 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/771364888' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-21T21:51:21.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:20 smithi151 ceph-mon[31623]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-21T21:51:21.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:20 smithi151 ceph-mon[31623]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-21T21:51:21.235 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:20 smithi184 ceph-mon[36942]: pgmap v314: 97 pgs: 23 creating+peering, 48 active+clean, 26 unknown; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:51:21.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:20 smithi184 ceph-mon[36942]: osdmap e80: 12 total, 12 up, 12 in 2024-05-21T21:51:21.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:20 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/771364888' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-21T21:51:21.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:20 smithi184 ceph-mon[36942]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-21T21:51:21.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:20 smithi184 ceph-mon[36942]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-21T21:51:21.283 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:51:21.285 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:20 smithi164 ceph-mon[36269]: pgmap v314: 97 pgs: 23 creating+peering, 48 active+clean, 26 unknown; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:51:21.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:20 smithi164 ceph-mon[36269]: osdmap e80: 12 total, 12 up, 12 in 2024-05-21T21:51:21.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:20 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/771364888' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-21T21:51:21.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:20 smithi164 ceph-mon[36269]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-21T21:51:21.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:20 smithi164 ceph-mon[36269]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-21T21:51:22.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:21 smithi151 ceph-mon[31623]: mds.? [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] up:active 2024-05-21T21:51:22.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:21 smithi151 ceph-mon[31623]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-21T21:51:22.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:21 smithi151 ceph-mon[31623]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-21T21:51:22.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:21 smithi151 ceph-mon[31623]: pgmap v316: 97 pgs: 19 creating+peering, 78 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-05-21T21:51:22.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:21 smithi184 ceph-mon[36942]: mds.? [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] up:active 2024-05-21T21:51:22.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:21 smithi184 ceph-mon[36942]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-21T21:51:22.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:21 smithi184 ceph-mon[36942]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-21T21:51:22.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:21 smithi184 ceph-mon[36942]: pgmap v316: 97 pgs: 19 creating+peering, 78 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-05-21T21:51:22.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:21 smithi164 ceph-mon[36269]: mds.? [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] up:active 2024-05-21T21:51:22.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:21 smithi164 ceph-mon[36269]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-21T21:51:22.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:21 smithi164 ceph-mon[36269]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-21T21:51:22.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:21 smithi164 ceph-mon[36269]: pgmap v316: 97 pgs: 19 creating+peering, 78 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-05-21T21:51:23.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:22 smithi151 ceph-mon[31623]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-21T21:51:23.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:22 smithi151 ceph-mon[31623]: Cluster is now healthy 2024-05-21T21:51:23.235 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:22 smithi184 ceph-mon[36942]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-21T21:51:23.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:22 smithi184 ceph-mon[36942]: Cluster is now healthy 2024-05-21T21:51:23.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:22 smithi164 ceph-mon[36269]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-21T21:51:23.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:22 smithi164 ceph-mon[36269]: Cluster is now healthy 2024-05-21T21:51:23.394 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:23.392+0000 7f92f4a00700 1 -- 172.21.15.151:0/1758439036 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f92f0106120 msgr2=0x7f92f0112630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:51:23.394 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:23.392+0000 7f92f4a00700 1 --2- 172.21.15.151:0/1758439036 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f92f0106120 0x7f92f0112630 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f92e800b0a0 tx=0x7f92e800b3b0 comp rx=0 tx=0).stop 2024-05-21T21:51:23.394 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:23.393+0000 7f92f4a00700 1 -- 172.21.15.151:0/1758439036 shutdown_connections 2024-05-21T21:51:23.394 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:23.393+0000 7f92f4a00700 1 --2- 172.21.15.151:0/1758439036 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f92f0106120 0x7f92f0112630 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:23.394 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:23.393+0000 7f92f4a00700 1 --2- 172.21.15.151:0/1758439036 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f92f010b760 0x7f92f010bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:23.394 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:23.393+0000 7f92f4a00700 1 --2- 172.21.15.151:0/1758439036 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f92f0105760 0x7f92f0105be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:23.394 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:23.393+0000 7f92f4a00700 1 -- 172.21.15.151:0/1758439036 >> 172.21.15.151:0/1758439036 conn(0x7f92f0101250 msgr2=0x7f92f0103670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:51:23.394 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:23.393+0000 7f92f4a00700 1 -- 172.21.15.151:0/1758439036 shutdown_connections 2024-05-21T21:51:23.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:23.393+0000 7f92f4a00700 1 -- 172.21.15.151:0/1758439036 wait complete. 2024-05-21T21:51:23.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:23.394+0000 7f92f4a00700 1 Processor -- start 2024-05-21T21:51:23.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:23.394+0000 7f92f4a00700 1 -- start start 2024-05-21T21:51:23.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:23.395+0000 7f92f4a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f92f0105760 0x7f92f019be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:51:23.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:23.395+0000 7f92f4a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f92f0106120 0x7f92f019c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:51:23.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:23.395+0000 7f92f4a00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f92f010b760 0x7f92f01a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:51:23.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:23.395+0000 7f92f4a00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f92f01159f0 con 0x7f92f0106120 2024-05-21T21:51:23.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:23.395+0000 7f92f4a00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f92f0115870 con 0x7f92f0105760 2024-05-21T21:51:23.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:23.395+0000 7f92f4a00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f92f0115b70 con 0x7f92f010b760 2024-05-21T21:51:23.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:23.395+0000 7f92ed400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f92f0106120 0x7f92f019c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:51:23.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:23.395+0000 7f92ee800700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f92f010b760 0x7f92f01a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:51:23.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:23.395+0000 7f92ee800700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f92f010b760 0x7f92f01a06d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:36212/0 (socket says 172.21.15.151:36212) 2024-05-21T21:51:23.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:23.395+0000 7f92ee800700 1 -- 172.21.15.151:0/1816541368 learned_addr learned my addr 172.21.15.151:0/1816541368 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:51:23.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:23.395+0000 7f92ede00700 1 --2- 172.21.15.151:0/1816541368 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f92f0105760 0x7f92f019be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:51:23.398 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:23.396+0000 7f92ee800700 1 -- 172.21.15.151:0/1816541368 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f92f0105760 msgr2=0x7f92f019be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:51:23.398 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:23.396+0000 7f92ee800700 1 --2- 172.21.15.151:0/1816541368 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f92f0105760 0x7f92f019be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:23.398 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:23.396+0000 7f92ee800700 1 -- 172.21.15.151:0/1816541368 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f92f0106120 msgr2=0x7f92f019c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:51:23.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:23.396+0000 7f92ee800700 1 --2- 172.21.15.151:0/1816541368 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f92f0106120 0x7f92f019c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:23.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:23.396+0000 7f92ee800700 1 -- 172.21.15.151:0/1816541368 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92f01a0e10 con 0x7f92f010b760 2024-05-21T21:51:23.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:23.396+0000 7f92ed400700 1 --2- 172.21.15.151:0/1816541368 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f92f0106120 0x7f92f019c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:51:23.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:23.396+0000 7f92ee800700 1 --2- 172.21.15.151:0/1816541368 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f92f010b760 0x7f92f01a06d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f92e800b070 tx=0x7f92e8009c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:51:23.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:23.396+0000 7f92e6a00700 1 -- 172.21.15.151:0/1816541368 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e8021070 con 0x7f92f010b760 2024-05-21T21:51:23.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:23.396+0000 7f92f4a00700 1 -- 172.21.15.151:0/1816541368 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92f01a10a0 con 0x7f92f010b760 2024-05-21T21:51:23.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:23.396+0000 7f92e6a00700 1 -- 172.21.15.151:0/1816541368 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92e8004e40 con 0x7f92f010b760 2024-05-21T21:51:23.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:23.396+0000 7f92e6a00700 1 -- 172.21.15.151:0/1816541368 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e8004630 con 0x7f92f010b760 2024-05-21T21:51:23.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:23.396+0000 7f92f4a00700 1 -- 172.21.15.151:0/1816541368 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92f01a8980 con 0x7f92f010b760 2024-05-21T21:51:23.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:23.397+0000 7f92f4a00700 1 -- 172.21.15.151:0/1816541368 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92f006a230 con 0x7f92f010b760 2024-05-21T21:51:23.401 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:23.400+0000 7f92e6a00700 1 -- 172.21.15.151:0/1816541368 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f92e8007980 con 0x7f92f010b760 2024-05-21T21:51:23.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:23.401+0000 7f92e6a00700 1 --2- 172.21.15.151:0/1816541368 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f92d40b2900 0x7f92d40b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:51:23.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:23.401+0000 7f92e6a00700 1 -- 172.21.15.151:0/1816541368 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f92e800f020 con 0x7f92f010b760 2024-05-21T21:51:23.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:23.402+0000 7f92ede00700 1 --2- 172.21.15.151:0/1816541368 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f92d40b2900 0x7f92d40b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:51:23.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:23.402+0000 7f92ede00700 1 --2- 172.21.15.151:0/1816541368 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f92d40b2900 0x7f92d40b4dc0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f92e0009be0 tx=0x7f92e0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:51:23.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:23.404+0000 7f92e6a00700 1 -- 172.21.15.151:0/1816541368 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f92e802a020 con 0x7f92f010b760 2024-05-21T21:51:23.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:23.634+0000 7f92f4a00700 1 -- 172.21.15.151:0/1816541368 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f92f019d160 con 0x7f92f010b760 2024-05-21T21:51:23.637 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:23.636+0000 7f92e6a00700 1 -- 172.21.15.151:0/1816541368 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22264 (secure 0 0 0) 0x7f92e80ab590 con 0x7f92f010b760 2024-05-21T21:51:23.637 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T21:51:23.638 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":80,"fsid":"c8c7a2e6-17ba-11ef-bc9a-c7b262605968","created":"2024-05-21T21:41:52.412833+0000","modified":"2024-05-21T21:51:19.552159+0000","last_up_change":"2024-05-21T21:50:05.401314+0000","last_in_change":"2024-05-21T21:49:45.684136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-21T21:45:55.196046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-21T21:51:11.447428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-21T21:51:15.618605+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"18db8e54-0cfc-4e12-9c84-6f72a99cc989","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":2420400473},{"type":"v1","addr":"172.21.15.151:6803","nonce":2420400473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":2420400473},{"type":"v1","addr":"172.21.15.151:6805","nonce":2420400473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":2420400473},{"type":"v1","addr":"172.21.15.151:6809","nonce":2420400473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":2420400473},{"type":"v1","addr":"172.21.15.151:6807","nonce":2420400473}]},"public_addr":"172.21.15.151:6803/2420400473","cluster_addr":"172.21.15.151:6805/2420400473","heartbeat_back_addr":"172.21.15.151:6809/2420400473","heartbeat_front_addr":"172.21.15.151:6807/2420400473","state":["exists","up"]},{"osd":1,"uuid":"bac384c7-1102-4815-81a8-10cc7661d230","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":3900240447},{"type":"v1","addr":"172.21.15.164:6801","nonce":3900240447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":3900240447},{"type":"v1","addr":"172.21.15.164:6803","nonce":3900240447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":3900240447},{"type":"v1","addr":"172.21.15.164:6807","nonce":3900240447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":3900240447},{"type":"v1","addr":"172.21.15.164:6805","nonce":3900240447}]},"public_addr":"172.21.15.164:6801/3900240447","cluster_addr":"172.21.15.164:6803/3900240447","heartbeat_back_addr":"172.21.15.164:6807/3900240447","heartbeat_front_addr":"172.21.15.164:6805/3900240447","state":["exists","up"]},{"osd":2,"uuid":"b7e4a782-1518-4d43-bee9-7fd768ee5cb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":3971150737},{"type":"v1","addr":"172.21.15.184:6801","nonce":3971150737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3971150737},{"type":"v1","addr":"172.21.15.184:6803","nonce":3971150737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3971150737},{"type":"v1","addr":"172.21.15.184:6807","nonce":3971150737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3971150737},{"type":"v1","addr":"172.21.15.184:6805","nonce":3971150737}]},"public_addr":"172.21.15.184:6801/3971150737","cluster_addr":"172.21.15.184:6803/3971150737","heartbeat_back_addr":"172.21.15.184:6807/3971150737","heartbeat_front_addr":"172.21.15.184:6805/3971150737","state":["exists","up"]},{"osd":3,"uuid":"ab3df826-39e3-4635-a49c-2b5ffdbf12f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":556594400},{"type":"v1","addr":"172.21.15.151:6811","nonce":556594400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":556594400},{"type":"v1","addr":"172.21.15.151:6813","nonce":556594400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":556594400},{"type":"v1","addr":"172.21.15.151:6817","nonce":556594400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":556594400},{"type":"v1","addr":"172.21.15.151:6815","nonce":556594400}]},"public_addr":"172.21.15.151:6811/556594400","cluster_addr":"172.21.15.151:6813/556594400","heartbeat_back_addr":"172.21.15.151:6817/556594400","heartbeat_front_addr":"172.21.15.151:6815/556594400","state":["exists","up"]},{"osd":4,"uuid":"00cbf6ae-0f42-4685-a77d-d803c6467aa0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":988881973},{"type":"v1","addr":"172.21.15.164:6809","nonce":988881973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":988881973},{"type":"v1","addr":"172.21.15.164:6811","nonce":988881973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":988881973},{"type":"v1","addr":"172.21.15.164:6815","nonce":988881973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":988881973},{"type":"v1","addr":"172.21.15.164:6813","nonce":988881973}]},"public_addr":"172.21.15.164:6809/988881973","cluster_addr":"172.21.15.164:6811/988881973","heartbeat_back_addr":"172.21.15.164:6815/988881973","heartbeat_front_addr":"172.21.15.164:6813/988881973","state":["exists","up"]},{"osd":5,"uuid":"fad14221-0ef0-4d40-b414-c6c9f1f9718c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3293311106},{"type":"v1","addr":"172.21.15.184:6809","nonce":3293311106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":3293311106},{"type":"v1","addr":"172.21.15.184:6811","nonce":3293311106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":3293311106},{"type":"v1","addr":"172.21.15.184:6815","nonce":3293311106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":3293311106},{"type":"v1","addr":"172.21.15.184:6813","nonce":3293311106}]},"public_addr":"172.21.15.184:6809/3293311106","cluster_addr":"172.21.15.184:6811/3293311106","heartbeat_back_addr":"172.21.15.184:6815/3293311106","heartbeat_front_addr":"172.21.15.184:6813/3293311106","state":["exists","up"]},{"osd":6,"uuid":"ab8a60f6-0dec-4c1c-89db-f2e2f638f1ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":3377550762},{"type":"v1","addr":"172.21.15.151:6819","nonce":3377550762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":3377550762},{"type":"v1","addr":"172.21.15.151:6821","nonce":3377550762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":3377550762},{"type":"v1","addr":"172.21.15.151:6825","nonce":3377550762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":3377550762},{"type":"v1","addr":"172.21.15.151:6823","nonce":3377550762}]},"public_addr":"172.21.15.151:6819/3377550762","cluster_addr":"172.21.15.151:6821/3377550762","heartbeat_back_addr":"172.21.15.151:6825/3377550762","heartbeat_front_addr":"172.21.15.151:6823/3377550762","state":["exists","up"]},{"osd":7,"uuid":"db332268-9972-4ed7-a307-822b83a0fbb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":3869696936},{"type":"v1","addr":"172.21.15.164:6817","nonce":3869696936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":3869696936},{"type":"v1","addr":"172.21.15.164:6819","nonce":3869696936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":3869696936},{"type":"v1","addr":"172.21.15.164:6823","nonce":3869696936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":3869696936},{"type":"v1","addr":"172.21.15.164:6821","nonce":3869696936}]},"public_addr":"172.21.15.164:6817/3869696936","cluster_addr":"172.21.15.164:6819/3869696936","heartbeat_back_addr":"172.21.15.164:6823/3869696936","heartbeat_front_addr":"172.21.15.164:6821/3869696936","state":["exists","up"]},{"osd":8,"uuid":"fe2fbe1c-549a-4947-a188-426dfca04d10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":575273541},{"type":"v1","addr":"172.21.15.184:6817","nonce":575273541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":575273541},{"type":"v1","addr":"172.21.15.184:6819","nonce":575273541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":575273541},{"type":"v1","addr":"172.21.15.184:6823","nonce":575273541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":575273541},{"type":"v1","addr":"172.21.15.184:6821","nonce":575273541}]},"public_addr":"172.21.15.184:6817/575273541","cluster_addr":"172.21.15.184:6819/575273541","heartbeat_back_addr":"172.21.15.184:6823/575273541","heartbeat_front_addr":"172.21.15.184:6821/575273541","state":["exists","up"]},{"osd":9,"uuid":"3a7b450f-8004-4adf-bae3-970918f98f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":337128533},{"type":"v1","addr":"172.21.15.151:6827","nonce":337128533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":337128533},{"type":"v1","addr":"172.21.15.151:6829","nonce":337128533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":337128533},{"type":"v1","addr":"172.21.15.151:6833","nonce":337128533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":337128533},{"type":"v1","addr":"172.21.15.151:6831","nonce":337128533}]},"public_addr":"172.21.15.151:6827/337128533","cluster_addr":"172.21.15.151:6829/337128533","heartbeat_back_addr":"172.21.15.151:6833/337128533","heartbeat_front_addr":"172.21.15.151:6831/337128533","state":["exists","up"]},{"osd":10,"uuid":"8fcdf271-90c4-4db3-be69-b0ca716dfd80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":1823448782},{"type":"v1","addr":"172.21.15.164:6825","nonce":1823448782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":1823448782},{"type":"v1","addr":"172.21.15.164:6827","nonce":1823448782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":1823448782},{"type":"v1","addr":"172.21.15.164:6831","nonce":1823448782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":1823448782},{"type":"v1","addr":"172.21.15.164:6829","nonce":1823448782}]},"public_addr":"172.21.15.164:6825/1823448782","cluster_addr":"172.21.15.164:6827/1823448782","heartbeat_back_addr":"172.21.15.164:6831/1823448782","heartbeat_front_addr":"172.21.15.164:6829/1823448782","state":["exists","up"]},{"osd":11,"uuid":"e661477b-0c09-409f-a2fe-6ebd34642a8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":1563340128},{"type":"v1","addr":"172.21.15.184:6825","nonce":1563340128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":1563340128},{"type":"v1","addr":"172.21.15.184:6827","nonce":1563340128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":1563340128},{"type":"v1","addr":"172.21.15.184:6831","nonce":1563340128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":1563340128},{"type":"v1","addr":"172.21.15.184:6829","nonce":1563340128}]},"public_addr":"172.21.15.184:6825/1563340128","cluster_addr":"172.21.15.184:6827/1563340128","heartbeat_back_addr":"172.21.15.184:6831/1563340128","heartbeat_front_addr":"172.21.15.184:6829/1563340128","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:45:02.821160+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:45:27.844564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:45:51.904981+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:46:24.376245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:46:50.036621+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:47:14.748422+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:47:47.396383+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:48:13.543174+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:48:38.213285+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:49:11.980042+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:49:38.519761+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:50:03.439898+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.151:0/31963128":"2024-05-22T21:43:09.041455+0000","172.21.15.151:6800/3428827300":"2024-05-22T21:43:09.041455+0000","172.21.15.151:6801/3428827300":"2024-05-22T21:43:09.041455+0000","172.21.15.151:0/131989141":"2024-05-22T21:43:09.041455+0000","172.21.15.151:6801/1128885292":"2024-05-22T21:42:35.440147+0000","172.21.15.151:0/2703664519":"2024-05-22T21:43:09.041455+0000","172.21.15.151:6800/1128885292":"2024-05-22T21:42:35.440147+0000","172.21.15.151:0/3052943699":"2024-05-22T21:42:35.440147+0000","172.21.15.151:0/3929452511":"2024-05-22T21:42:35.440147+0000","172.21.15.151:0/3762889700":"2024-05-22T21:42:35.440147+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-21T21:51:23.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:23.639+0000 7f92f4a00700 1 -- 172.21.15.151:0/1816541368 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f92d40b2900 msgr2=0x7f92d40b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:51:23.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:23.639+0000 7f92f4a00700 1 --2- 172.21.15.151:0/1816541368 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f92d40b2900 0x7f92d40b4dc0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f92e0009be0 tx=0x7f92e0009260 comp rx=0 tx=0).stop 2024-05-21T21:51:23.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:23.639+0000 7f92f4a00700 1 -- 172.21.15.151:0/1816541368 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f92f010b760 msgr2=0x7f92f01a06d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:51:23.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:23.639+0000 7f92f4a00700 1 --2- 172.21.15.151:0/1816541368 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f92f010b760 0x7f92f01a06d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f92e800b070 tx=0x7f92e8009c90 comp rx=0 tx=0).stop 2024-05-21T21:51:23.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:23.639+0000 7f92f4a00700 1 -- 172.21.15.151:0/1816541368 shutdown_connections 2024-05-21T21:51:23.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:23.639+0000 7f92f4a00700 1 --2- 172.21.15.151:0/1816541368 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f92d40b2900 0x7f92d40b4dc0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:23.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:23.639+0000 7f92f4a00700 1 --2- 172.21.15.151:0/1816541368 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f92f0105760 0x7f92f019be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:23.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:23.639+0000 7f92f4a00700 1 --2- 172.21.15.151:0/1816541368 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f92f010b760 0x7f92f01a06d0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:23.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:23.639+0000 7f92f4a00700 1 --2- 172.21.15.151:0/1816541368 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f92f0106120 0x7f92f019c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:23.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:23.639+0000 7f92f4a00700 1 -- 172.21.15.151:0/1816541368 >> 172.21.15.151:0/1816541368 conn(0x7f92f0101250 msgr2=0x7f92f0101790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:51:23.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:23.640+0000 7f92f4a00700 1 -- 172.21.15.151:0/1816541368 shutdown_connections 2024-05-21T21:51:23.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:23.640+0000 7f92f4a00700 1 -- 172.21.15.151:0/1816541368 wait complete. 2024-05-21T21:51:24.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:23 smithi184 ceph-mon[36942]: pgmap v317: 97 pgs: 97 active+clean; 578 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 6 op/s 2024-05-21T21:51:24.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:23 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/1816541368' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T21:51:24.285 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:23 smithi164 ceph-mon[36269]: pgmap v317: 97 pgs: 97 active+clean; 578 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 6 op/s 2024-05-21T21:51:24.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:23 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/1816541368' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T21:51:24.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:23 smithi151 ceph-mon[31623]: pgmap v317: 97 pgs: 97 active+clean; 578 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 6 op/s 2024-05-21T21:51:24.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:23 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/1816541368' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T21:51:24.530 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph osd dump --format=json 2024-05-21T21:51:24.980 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:51:25.036 INFO:journalctl@ceph.mds.b.smithi164.stdout:May 21 21:51:24 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-b[66241]: 2024-05-21T21:51:24.555+0000 7f79f0f0f700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-21T21:51:26.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:26 smithi164 ceph-mon[36269]: pgmap v318: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-21T21:51:26.661 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:26 smithi151 ceph-mon[31623]: pgmap v318: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-21T21:51:26.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:26 smithi184 ceph-mon[36942]: pgmap v318: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-21T21:51:27.150 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:27.148+0000 7f700e400700 1 -- 172.21.15.151:0/3349757933 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7008075770 msgr2=0x7f7008075b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:51:27.150 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:27.148+0000 7f700e400700 1 --2- 172.21.15.151:0/3349757933 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7008075770 0x7f7008075b50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6ff8009990 tx=0x7f6ff8009ca0 comp rx=0 tx=0).stop 2024-05-21T21:51:27.150 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:27.148+0000 7f700e400700 1 -- 172.21.15.151:0/3349757933 shutdown_connections 2024-05-21T21:51:27.150 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:27.148+0000 7f700e400700 1 --2- 172.21.15.151:0/3349757933 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f700810ff40 0x7f7008112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:27.150 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:27.148+0000 7f700e400700 1 --2- 172.21.15.151:0/3349757933 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7008075770 0x7f7008075b50 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:27.150 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:27.148+0000 7f700e400700 1 --2- 172.21.15.151:0/3349757933 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7008076090 0x7f700810fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:27.150 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:27.148+0000 7f700e400700 1 -- 172.21.15.151:0/3349757933 >> 172.21.15.151:0/3349757933 conn(0x7f700807a6a0 msgr2=0x7f700807aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:51:27.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:27.149+0000 7f700e400700 1 -- 172.21.15.151:0/3349757933 shutdown_connections 2024-05-21T21:51:27.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:27.149+0000 7f700e400700 1 -- 172.21.15.151:0/3349757933 wait complete. 2024-05-21T21:51:27.152 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:27.150+0000 7f700e400700 1 Processor -- start 2024-05-21T21:51:27.152 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:27.150+0000 7f700e400700 1 -- start start 2024-05-21T21:51:27.152 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:27.150+0000 7f700e400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7008075770 0x7f700819be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:51:27.152 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:27.150+0000 7f700e400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7008076090 0x7f700819c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:51:27.152 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:27.150+0000 7f700e400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f700810ff40 0x7f70081a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:51:27.152 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:27.150+0000 7f700e400700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7008115910 con 0x7f7008076090 2024-05-21T21:51:27.152 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:27.150+0000 7f700e400700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7008115790 con 0x7f700810ff40 2024-05-21T21:51:27.152 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:27.150+0000 7f700e400700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7008115a90 con 0x7f7008075770 2024-05-21T21:51:27.152 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:27.151+0000 7f7007400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7008075770 0x7f700819be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:51:27.152 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:27.151+0000 7f7006a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7008076090 0x7f700819c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:51:27.153 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:27.151+0000 7f7007400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7008075770 0x7f700819be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:36044/0 (socket says 172.21.15.151:36044) 2024-05-21T21:51:27.153 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:27.151+0000 7f7007400700 1 -- 172.21.15.151:0/2089539304 learned_addr learned my addr 172.21.15.151:0/2089539304 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:51:27.153 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:27.151+0000 7f7007e00700 1 --2- 172.21.15.151:0/2089539304 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f700810ff40 0x7f70081a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:51:27.153 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:27.151+0000 7f7007400700 1 -- 172.21.15.151:0/2089539304 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f700810ff40 msgr2=0x7f70081a06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:51:27.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:27.151+0000 7f7007400700 1 --2- 172.21.15.151:0/2089539304 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f700810ff40 0x7f70081a06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:27.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:27.151+0000 7f7007400700 1 -- 172.21.15.151:0/2089539304 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7008076090 msgr2=0x7f700819c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:51:27.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:27.151+0000 7f7007400700 1 --2- 172.21.15.151:0/2089539304 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7008076090 0x7f700819c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:27.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:27.151+0000 7f7007400700 1 -- 172.21.15.151:0/2089539304 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70081a0e00 con 0x7f7008075770 2024-05-21T21:51:27.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:27.151+0000 7f7007e00700 1 --2- 172.21.15.151:0/2089539304 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f700810ff40 0x7f70081a06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T21:51:27.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:27.151+0000 7f7007400700 1 --2- 172.21.15.151:0/2089539304 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7008075770 0x7f700819be40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6ff8000c00 tx=0x7f6ff800bfd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:51:27.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:27.152+0000 7f6fffe00700 1 -- 172.21.15.151:0/2089539304 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff8027070 con 0x7f7008075770 2024-05-21T21:51:27.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:27.152+0000 7f6fffe00700 1 -- 172.21.15.151:0/2089539304 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ff8022470 con 0x7f7008075770 2024-05-21T21:51:27.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:27.152+0000 7f6fffe00700 1 -- 172.21.15.151:0/2089539304 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff8017650 con 0x7f7008075770 2024-05-21T21:51:27.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:27.152+0000 7f700e400700 1 -- 172.21.15.151:0/2089539304 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70081a1090 con 0x7f7008075770 2024-05-21T21:51:27.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:27.152+0000 7f700e400700 1 -- 172.21.15.151:0/2089539304 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70081a8970 con 0x7f7008075770 2024-05-21T21:51:27.155 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:27.153+0000 7f700e400700 1 -- 172.21.15.151:0/2089539304 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f700806a230 con 0x7f7008075770 2024-05-21T21:51:27.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:27.156+0000 7f6fffe00700 1 -- 172.21.15.151:0/2089539304 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f6ff802a070 con 0x7f7008075770 2024-05-21T21:51:27.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:27.157+0000 7f6fffe00700 1 --2- 172.21.15.151:0/2089539304 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f6fec0b2900 0x7f6fec0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:51:27.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:27.157+0000 7f6fffe00700 1 -- 172.21.15.151:0/2089539304 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f6ff80e3070 con 0x7f7008075770 2024-05-21T21:51:27.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:27.157+0000 7f7006a00700 1 --2- 172.21.15.151:0/2089539304 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f6fec0b2900 0x7f6fec0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:51:27.159 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:27.158+0000 7f7006a00700 1 --2- 172.21.15.151:0/2089539304 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f6fec0b2900 0x7f6fec0b4dc0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7008111c00 tx=0x7f6ff4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:51:27.160 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:27.159+0000 7f6fffe00700 1 -- 172.21.15.151:0/2089539304 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6ff8013070 con 0x7f7008075770 2024-05-21T21:51:27.394 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:27.393+0000 7f700e400700 1 -- 172.21.15.151:0/2089539304 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f700819d150 con 0x7f7008075770 2024-05-21T21:51:27.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:27.395+0000 7f6fffe00700 1 -- 172.21.15.151:0/2089539304 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22264 (secure 0 0 0) 0x7f6ff80aaed0 con 0x7f7008075770 2024-05-21T21:51:27.397 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T21:51:27.397 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":80,"fsid":"c8c7a2e6-17ba-11ef-bc9a-c7b262605968","created":"2024-05-21T21:41:52.412833+0000","modified":"2024-05-21T21:51:19.552159+0000","last_up_change":"2024-05-21T21:50:05.401314+0000","last_in_change":"2024-05-21T21:49:45.684136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-21T21:45:55.196046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-21T21:51:11.447428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-21T21:51:15.618605+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"18db8e54-0cfc-4e12-9c84-6f72a99cc989","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":2420400473},{"type":"v1","addr":"172.21.15.151:6803","nonce":2420400473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":2420400473},{"type":"v1","addr":"172.21.15.151:6805","nonce":2420400473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":2420400473},{"type":"v1","addr":"172.21.15.151:6809","nonce":2420400473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":2420400473},{"type":"v1","addr":"172.21.15.151:6807","nonce":2420400473}]},"public_addr":"172.21.15.151:6803/2420400473","cluster_addr":"172.21.15.151:6805/2420400473","heartbeat_back_addr":"172.21.15.151:6809/2420400473","heartbeat_front_addr":"172.21.15.151:6807/2420400473","state":["exists","up"]},{"osd":1,"uuid":"bac384c7-1102-4815-81a8-10cc7661d230","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":3900240447},{"type":"v1","addr":"172.21.15.164:6801","nonce":3900240447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":3900240447},{"type":"v1","addr":"172.21.15.164:6803","nonce":3900240447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":3900240447},{"type":"v1","addr":"172.21.15.164:6807","nonce":3900240447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":3900240447},{"type":"v1","addr":"172.21.15.164:6805","nonce":3900240447}]},"public_addr":"172.21.15.164:6801/3900240447","cluster_addr":"172.21.15.164:6803/3900240447","heartbeat_back_addr":"172.21.15.164:6807/3900240447","heartbeat_front_addr":"172.21.15.164:6805/3900240447","state":["exists","up"]},{"osd":2,"uuid":"b7e4a782-1518-4d43-bee9-7fd768ee5cb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":3971150737},{"type":"v1","addr":"172.21.15.184:6801","nonce":3971150737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3971150737},{"type":"v1","addr":"172.21.15.184:6803","nonce":3971150737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3971150737},{"type":"v1","addr":"172.21.15.184:6807","nonce":3971150737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3971150737},{"type":"v1","addr":"172.21.15.184:6805","nonce":3971150737}]},"public_addr":"172.21.15.184:6801/3971150737","cluster_addr":"172.21.15.184:6803/3971150737","heartbeat_back_addr":"172.21.15.184:6807/3971150737","heartbeat_front_addr":"172.21.15.184:6805/3971150737","state":["exists","up"]},{"osd":3,"uuid":"ab3df826-39e3-4635-a49c-2b5ffdbf12f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":556594400},{"type":"v1","addr":"172.21.15.151:6811","nonce":556594400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":556594400},{"type":"v1","addr":"172.21.15.151:6813","nonce":556594400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":556594400},{"type":"v1","addr":"172.21.15.151:6817","nonce":556594400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":556594400},{"type":"v1","addr":"172.21.15.151:6815","nonce":556594400}]},"public_addr":"172.21.15.151:6811/556594400","cluster_addr":"172.21.15.151:6813/556594400","heartbeat_back_addr":"172.21.15.151:6817/556594400","heartbeat_front_addr":"172.21.15.151:6815/556594400","state":["exists","up"]},{"osd":4,"uuid":"00cbf6ae-0f42-4685-a77d-d803c6467aa0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":988881973},{"type":"v1","addr":"172.21.15.164:6809","nonce":988881973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":988881973},{"type":"v1","addr":"172.21.15.164:6811","nonce":988881973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":988881973},{"type":"v1","addr":"172.21.15.164:6815","nonce":988881973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":988881973},{"type":"v1","addr":"172.21.15.164:6813","nonce":988881973}]},"public_addr":"172.21.15.164:6809/988881973","cluster_addr":"172.21.15.164:6811/988881973","heartbeat_back_addr":"172.21.15.164:6815/988881973","heartbeat_front_addr":"172.21.15.164:6813/988881973","state":["exists","up"]},{"osd":5,"uuid":"fad14221-0ef0-4d40-b414-c6c9f1f9718c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3293311106},{"type":"v1","addr":"172.21.15.184:6809","nonce":3293311106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":3293311106},{"type":"v1","addr":"172.21.15.184:6811","nonce":3293311106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":3293311106},{"type":"v1","addr":"172.21.15.184:6815","nonce":3293311106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":3293311106},{"type":"v1","addr":"172.21.15.184:6813","nonce":3293311106}]},"public_addr":"172.21.15.184:6809/3293311106","cluster_addr":"172.21.15.184:6811/3293311106","heartbeat_back_addr":"172.21.15.184:6815/3293311106","heartbeat_front_addr":"172.21.15.184:6813/3293311106","state":["exists","up"]},{"osd":6,"uuid":"ab8a60f6-0dec-4c1c-89db-f2e2f638f1ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":3377550762},{"type":"v1","addr":"172.21.15.151:6819","nonce":3377550762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":3377550762},{"type":"v1","addr":"172.21.15.151:6821","nonce":3377550762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":3377550762},{"type":"v1","addr":"172.21.15.151:6825","nonce":3377550762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":3377550762},{"type":"v1","addr":"172.21.15.151:6823","nonce":3377550762}]},"public_addr":"172.21.15.151:6819/3377550762","cluster_addr":"172.21.15.151:6821/3377550762","heartbeat_back_addr":"172.21.15.151:6825/3377550762","heartbeat_front_addr":"172.21.15.151:6823/3377550762","state":["exists","up"]},{"osd":7,"uuid":"db332268-9972-4ed7-a307-822b83a0fbb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":3869696936},{"type":"v1","addr":"172.21.15.164:6817","nonce":3869696936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":3869696936},{"type":"v1","addr":"172.21.15.164:6819","nonce":3869696936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":3869696936},{"type":"v1","addr":"172.21.15.164:6823","nonce":3869696936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":3869696936},{"type":"v1","addr":"172.21.15.164:6821","nonce":3869696936}]},"public_addr":"172.21.15.164:6817/3869696936","cluster_addr":"172.21.15.164:6819/3869696936","heartbeat_back_addr":"172.21.15.164:6823/3869696936","heartbeat_front_addr":"172.21.15.164:6821/3869696936","state":["exists","up"]},{"osd":8,"uuid":"fe2fbe1c-549a-4947-a188-426dfca04d10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":575273541},{"type":"v1","addr":"172.21.15.184:6817","nonce":575273541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":575273541},{"type":"v1","addr":"172.21.15.184:6819","nonce":575273541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":575273541},{"type":"v1","addr":"172.21.15.184:6823","nonce":575273541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":575273541},{"type":"v1","addr":"172.21.15.184:6821","nonce":575273541}]},"public_addr":"172.21.15.184:6817/575273541","cluster_addr":"172.21.15.184:6819/575273541","heartbeat_back_addr":"172.21.15.184:6823/575273541","heartbeat_front_addr":"172.21.15.184:6821/575273541","state":["exists","up"]},{"osd":9,"uuid":"3a7b450f-8004-4adf-bae3-970918f98f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":337128533},{"type":"v1","addr":"172.21.15.151:6827","nonce":337128533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":337128533},{"type":"v1","addr":"172.21.15.151:6829","nonce":337128533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":337128533},{"type":"v1","addr":"172.21.15.151:6833","nonce":337128533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":337128533},{"type":"v1","addr":"172.21.15.151:6831","nonce":337128533}]},"public_addr":"172.21.15.151:6827/337128533","cluster_addr":"172.21.15.151:6829/337128533","heartbeat_back_addr":"172.21.15.151:6833/337128533","heartbeat_front_addr":"172.21.15.151:6831/337128533","state":["exists","up"]},{"osd":10,"uuid":"8fcdf271-90c4-4db3-be69-b0ca716dfd80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":1823448782},{"type":"v1","addr":"172.21.15.164:6825","nonce":1823448782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":1823448782},{"type":"v1","addr":"172.21.15.164:6827","nonce":1823448782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":1823448782},{"type":"v1","addr":"172.21.15.164:6831","nonce":1823448782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":1823448782},{"type":"v1","addr":"172.21.15.164:6829","nonce":1823448782}]},"public_addr":"172.21.15.164:6825/1823448782","cluster_addr":"172.21.15.164:6827/1823448782","heartbeat_back_addr":"172.21.15.164:6831/1823448782","heartbeat_front_addr":"172.21.15.164:6829/1823448782","state":["exists","up"]},{"osd":11,"uuid":"e661477b-0c09-409f-a2fe-6ebd34642a8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":1563340128},{"type":"v1","addr":"172.21.15.184:6825","nonce":1563340128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":1563340128},{"type":"v1","addr":"172.21.15.184:6827","nonce":1563340128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":1563340128},{"type":"v1","addr":"172.21.15.184:6831","nonce":1563340128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":1563340128},{"type":"v1","addr":"172.21.15.184:6829","nonce":1563340128}]},"public_addr":"172.21.15.184:6825/1563340128","cluster_addr":"172.21.15.184:6827/1563340128","heartbeat_back_addr":"172.21.15.184:6831/1563340128","heartbeat_front_addr":"172.21.15.184:6829/1563340128","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:45:02.821160+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:45:27.844564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:45:51.904981+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:46:24.376245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:46:50.036621+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:47:14.748422+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:47:47.396383+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:48:13.543174+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:48:38.213285+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:49:11.980042+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:49:38.519761+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:50:03.439898+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.151:0/31963128":"2024-05-22T21:43:09.041455+0000","172.21.15.151:6800/3428827300":"2024-05-22T21:43:09.041455+0000","172.21.15.151:6801/3428827300":"2024-05-22T21:43:09.041455+0000","172.21.15.151:0/131989141":"2024-05-22T21:43:09.041455+0000","172.21.15.151:6801/1128885292":"2024-05-22T21:42:35.440147+0000","172.21.15.151:0/2703664519":"2024-05-22T21:43:09.041455+0000","172.21.15.151:6800/1128885292":"2024-05-22T21:42:35.440147+0000","172.21.15.151:0/3052943699":"2024-05-22T21:42:35.440147+0000","172.21.15.151:0/3929452511":"2024-05-22T21:42:35.440147+0000","172.21.15.151:0/3762889700":"2024-05-22T21:42:35.440147+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-21T21:51:27.401 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:27.399+0000 7f700e400700 1 -- 172.21.15.151:0/2089539304 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f6fec0b2900 msgr2=0x7f6fec0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:51:27.401 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:27.400+0000 7f700e400700 1 --2- 172.21.15.151:0/2089539304 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f6fec0b2900 0x7f6fec0b4dc0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7008111c00 tx=0x7f6ff4009260 comp rx=0 tx=0).stop 2024-05-21T21:51:27.401 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:27.400+0000 7f700e400700 1 -- 172.21.15.151:0/2089539304 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7008075770 msgr2=0x7f700819be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:51:27.401 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:27.400+0000 7f700e400700 1 --2- 172.21.15.151:0/2089539304 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7008075770 0x7f700819be40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6ff8000c00 tx=0x7f6ff800bfd0 comp rx=0 tx=0).stop 2024-05-21T21:51:27.401 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:27.400+0000 7f700e400700 1 -- 172.21.15.151:0/2089539304 shutdown_connections 2024-05-21T21:51:27.401 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:27.400+0000 7f700e400700 1 --2- 172.21.15.151:0/2089539304 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f6fec0b2900 0x7f6fec0b4dc0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:27.401 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:27.400+0000 7f700e400700 1 --2- 172.21.15.151:0/2089539304 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f700810ff40 0x7f70081a06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:27.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:27.400+0000 7f700e400700 1 --2- 172.21.15.151:0/2089539304 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7008075770 0x7f700819be40 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:27.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:27.400+0000 7f700e400700 1 --2- 172.21.15.151:0/2089539304 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7008076090 0x7f700819c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:27.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:27.400+0000 7f700e400700 1 -- 172.21.15.151:0/2089539304 >> 172.21.15.151:0/2089539304 conn(0x7f700807a6a0 msgr2=0x7f70081007f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:51:27.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:27.400+0000 7f700e400700 1 -- 172.21.15.151:0/2089539304 shutdown_connections 2024-05-21T21:51:27.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:27.400+0000 7f700e400700 1 -- 172.21.15.151:0/2089539304 wait complete. 2024-05-21T21:51:28.234 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-05-21T21:51:28.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:28 smithi151 ceph-mon[31623]: pgmap v319: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-05-21T21:51:28.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:28 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/2089539304' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T21:51:28.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:28 smithi164 ceph-mon[36269]: pgmap v319: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-05-21T21:51:28.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:28 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/2089539304' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T21:51:28.696 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:51:28.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:28 smithi184 ceph-mon[36942]: pgmap v319: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-05-21T21:51:28.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:28 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/2089539304' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T21:51:30.367 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:30 smithi151 ceph-mon[31623]: pgmap v320: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-05-21T21:51:30.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:30 smithi164 ceph-mon[36269]: pgmap v320: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-05-21T21:51:30.735 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:30 smithi184 ceph-mon[36942]: pgmap v320: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-05-21T21:51:30.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:30.848+0000 7f27eae00700 1 -- 172.21.15.151:0/2835549509 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f27e4106010 msgr2=0x7f27e4112520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:51:30.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:30.848+0000 7f27eae00700 1 --2- 172.21.15.151:0/2835549509 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f27e4106010 0x7f27e4112520 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f27dc009990 tx=0x7f27dc009ca0 comp rx=0 tx=0).stop 2024-05-21T21:51:30.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:30.849+0000 7f27eae00700 1 -- 172.21.15.151:0/2835549509 shutdown_connections 2024-05-21T21:51:30.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:30.849+0000 7f27eae00700 1 --2- 172.21.15.151:0/2835549509 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f27e4106010 0x7f27e4112520 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:30.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:30.849+0000 7f27eae00700 1 --2- 172.21.15.151:0/2835549509 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f27e410b650 0x7f27e410ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:30.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:30.849+0000 7f27eae00700 1 --2- 172.21.15.151:0/2835549509 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f27e4105650 0x7f27e4105ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:30.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:30.849+0000 7f27eae00700 1 -- 172.21.15.151:0/2835549509 >> 172.21.15.151:0/2835549509 conn(0x7f27e4101180 msgr2=0x7f27e41035a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:51:30.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:30.849+0000 7f27eae00700 1 -- 172.21.15.151:0/2835549509 shutdown_connections 2024-05-21T21:51:30.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:30.849+0000 7f27eae00700 1 -- 172.21.15.151:0/2835549509 wait complete. 2024-05-21T21:51:30.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:30.850+0000 7f27eae00700 1 Processor -- start 2024-05-21T21:51:30.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:30.850+0000 7f27eae00700 1 -- start start 2024-05-21T21:51:30.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:30.851+0000 7f27eae00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f27e4105650 0x7f27e41a1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:51:30.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:30.851+0000 7f27eae00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f27e4106010 0x7f27e41a1e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:51:30.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:30.851+0000 7f27eae00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f27e410b650 0x7f27e419bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:51:30.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:30.851+0000 7f27eae00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f27e4115850 con 0x7f27e4106010 2024-05-21T21:51:30.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:30.851+0000 7f27eae00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f27e41156d0 con 0x7f27e4105650 2024-05-21T21:51:30.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:30.851+0000 7f27eae00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f27e41159d0 con 0x7f27e410b650 2024-05-21T21:51:30.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:30.851+0000 7f27e9000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f27e4106010 0x7f27e41a1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:51:30.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:30.851+0000 7f27e9a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f27e4105650 0x7f27e41a1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:51:30.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:30.851+0000 7f27e9000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f27e4106010 0x7f27e41a1e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:60444/0 (socket says 172.21.15.151:60444) 2024-05-21T21:51:30.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:30.851+0000 7f27e9000700 1 -- 172.21.15.151:0/3944288868 learned_addr learned my addr 172.21.15.151:0/3944288868 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:51:30.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:30.851+0000 7f27e9a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f27e4105650 0x7f27e41a1910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:40762/0 (socket says 172.21.15.151:40762) 2024-05-21T21:51:30.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:30.851+0000 7f27ea400700 1 --2- 172.21.15.151:0/3944288868 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f27e410b650 0x7f27e419bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:51:30.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:30.852+0000 7f27e9a00700 1 -- 172.21.15.151:0/3944288868 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f27e410b650 msgr2=0x7f27e419bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:51:30.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:30.852+0000 7f27e9a00700 1 --2- 172.21.15.151:0/3944288868 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f27e410b650 0x7f27e419bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:30.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:30.852+0000 7f27e9a00700 1 -- 172.21.15.151:0/3944288868 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f27e4106010 msgr2=0x7f27e41a1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:51:30.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:30.852+0000 7f27e9a00700 1 --2- 172.21.15.151:0/3944288868 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f27e4106010 0x7f27e41a1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:30.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:30.852+0000 7f27e9a00700 1 -- 172.21.15.151:0/3944288868 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27e419c230 con 0x7f27e4105650 2024-05-21T21:51:30.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:30.852+0000 7f27e9a00700 1 --2- 172.21.15.151:0/3944288868 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f27e4105650 0x7f27e41a1910 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f27d800ef10 tx=0x7f27d800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:51:30.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:30.852+0000 7f27e2000700 1 -- 172.21.15.151:0/3944288868 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d800c9c0 con 0x7f27e4105650 2024-05-21T21:51:30.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:30.852+0000 7f27e2000700 1 -- 172.21.15.151:0/3944288868 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27d8019070 con 0x7f27e4105650 2024-05-21T21:51:30.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:30.852+0000 7f27eae00700 1 -- 172.21.15.151:0/3944288868 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27e419c4c0 con 0x7f27e4105650 2024-05-21T21:51:30.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:30.852+0000 7f27e2000700 1 -- 172.21.15.151:0/3944288868 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d8014480 con 0x7f27e4105650 2024-05-21T21:51:30.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:30.852+0000 7f27eae00700 1 -- 172.21.15.151:0/3944288868 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27e41a87e0 con 0x7f27e4105650 2024-05-21T21:51:30.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:30.853+0000 7f27eae00700 1 -- 172.21.15.151:0/3944288868 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27e406a230 con 0x7f27e4105650 2024-05-21T21:51:30.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:30.856+0000 7f27e2000700 1 -- 172.21.15.151:0/3944288868 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f27d8012050 con 0x7f27e4105650 2024-05-21T21:51:30.858 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:30.857+0000 7f27e2000700 1 --2- 172.21.15.151:0/3944288868 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f27cc0b2880 0x7f27cc0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:51:30.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:30.857+0000 7f27e2000700 1 -- 172.21.15.151:0/3944288868 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f27d80e26a0 con 0x7f27e4105650 2024-05-21T21:51:30.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:30.857+0000 7f27e9000700 1 --2- 172.21.15.151:0/3944288868 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f27cc0b2880 0x7f27cc0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:51:30.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:30.858+0000 7f27e9000700 1 --2- 172.21.15.151:0/3944288868 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f27cc0b2880 0x7f27cc0b4d40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f27e410ac80 tx=0x7f27d0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:51:30.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:30.860+0000 7f27e2000700 1 -- 172.21.15.151:0/3944288868 <== mon.2 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f27d8010040 con 0x7f27e4105650 2024-05-21T21:51:31.146 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:31.145+0000 7f27eae00700 1 -- 172.21.15.151:0/3944288868 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f27e402e560 con 0x7f27e4105650 2024-05-21T21:51:31.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:31.269+0000 7f27e2000700 1 -- 172.21.15.151:0/3944288868 <== mon.2 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f27d80aa470 con 0x7f27e4105650 2024-05-21T21:51:31.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:31.273+0000 7f27eae00700 1 -- 172.21.15.151:0/3944288868 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f27cc0b2880 msgr2=0x7f27cc0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:51:31.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:31.273+0000 7f27eae00700 1 --2- 172.21.15.151:0/3944288868 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f27cc0b2880 0x7f27cc0b4d40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f27e410ac80 tx=0x7f27d0009260 comp rx=0 tx=0).stop 2024-05-21T21:51:31.275 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:31.273+0000 7f27eae00700 1 -- 172.21.15.151:0/3944288868 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f27e4105650 msgr2=0x7f27e41a1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:51:31.275 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:31.273+0000 7f27eae00700 1 --2- 172.21.15.151:0/3944288868 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f27e4105650 0x7f27e41a1910 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f27d800ef10 tx=0x7f27d800c3a0 comp rx=0 tx=0).stop 2024-05-21T21:51:31.275 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:31.274+0000 7f27eae00700 1 -- 172.21.15.151:0/3944288868 shutdown_connections 2024-05-21T21:51:31.275 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:31.274+0000 7f27eae00700 1 --2- 172.21.15.151:0/3944288868 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f27cc0b2880 0x7f27cc0b4d40 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:31.276 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:31.274+0000 7f27eae00700 1 --2- 172.21.15.151:0/3944288868 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f27e4105650 0x7f27e41a1910 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:31.276 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:31.274+0000 7f27eae00700 1 --2- 172.21.15.151:0/3944288868 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f27e410b650 0x7f27e419bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:31.276 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:31.274+0000 7f27eae00700 1 --2- 172.21.15.151:0/3944288868 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f27e4106010 0x7f27e41a1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:31.276 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:31.274+0000 7f27eae00700 1 -- 172.21.15.151:0/3944288868 >> 172.21.15.151:0/3944288868 conn(0x7f27e4101180 msgr2=0x7f27e4101760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:51:31.276 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:31.274+0000 7f27eae00700 1 -- 172.21.15.151:0/3944288868 shutdown_connections 2024-05-21T21:51:31.276 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:31.274+0000 7f27eae00700 1 -- 172.21.15.151:0/3944288868 wait complete. 2024-05-21T21:51:31.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:31 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/3944288868' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-21T21:51:31.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:31 smithi151 ceph-mon[31623]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-21T21:51:31.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:31 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/3944288868' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-21T21:51:31.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:31 smithi164 ceph-mon[36269]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-21T21:51:31.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:31 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/3944288868' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-21T21:51:31.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:31 smithi184 ceph-mon[36942]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-21T21:51:32.079 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-05-21T21:51:32.079 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph fs set cephfs max_mds 5 2024-05-21T21:51:32.534 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:51:32.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:32 smithi164 ceph-mon[36269]: pgmap v321: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-05-21T21:51:32.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:32 smithi164 ceph-mon[36269]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-21T21:51:32.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:32 smithi164 ceph-mon[36269]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-21T21:51:32.684 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:32 smithi151 ceph-mon[31623]: pgmap v321: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-05-21T21:51:32.684 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:32 smithi151 ceph-mon[31623]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-21T21:51:32.684 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:32 smithi151 ceph-mon[31623]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-21T21:51:32.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:32 smithi184 ceph-mon[36942]: pgmap v321: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-05-21T21:51:32.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:32 smithi184 ceph-mon[36942]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-21T21:51:32.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:32 smithi184 ceph-mon[36942]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-21T21:51:34.369 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:34 smithi151 ceph-mon[31623]: pgmap v322: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2024-05-21T21:51:34.535 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:34.534+0000 7efc4c800700 1 -- 172.21.15.151:0/3918501272 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc4410b760 msgr2=0x7efc4410bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:51:34.535 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:34.534+0000 7efc4c800700 1 --2- 172.21.15.151:0/3918501272 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc4410b760 0x7efc4410bb40 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7efc38009990 tx=0x7efc38009ca0 comp rx=0 tx=0).stop 2024-05-21T21:51:34.535 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:34.534+0000 7efc4c800700 1 -- 172.21.15.151:0/3918501272 shutdown_connections 2024-05-21T21:51:34.536 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:34.534+0000 7efc4c800700 1 --2- 172.21.15.151:0/3918501272 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efc44106120 0x7efc44112630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:34.536 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:34.534+0000 7efc4c800700 1 --2- 172.21.15.151:0/3918501272 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc4410b760 0x7efc4410bb40 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:34.536 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:34.534+0000 7efc4c800700 1 --2- 172.21.15.151:0/3918501272 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc44105760 0x7efc44105be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:34.536 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:34.534+0000 7efc4c800700 1 -- 172.21.15.151:0/3918501272 >> 172.21.15.151:0/3918501272 conn(0x7efc44101250 msgr2=0x7efc44103670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:51:34.536 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:34.534+0000 7efc4c800700 1 -- 172.21.15.151:0/3918501272 shutdown_connections 2024-05-21T21:51:34.536 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:34.534+0000 7efc4c800700 1 -- 172.21.15.151:0/3918501272 wait complete. 2024-05-21T21:51:34.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:34 smithi164 ceph-mon[36269]: pgmap v322: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2024-05-21T21:51:34.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:34.535+0000 7efc4c800700 1 Processor -- start 2024-05-21T21:51:34.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:34.536+0000 7efc4c800700 1 -- start start 2024-05-21T21:51:34.538 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:34.536+0000 7efc4c800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efc44105760 0x7efc4419be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:51:34.538 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:34.536+0000 7efc4c800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc44106120 0x7efc4419c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:51:34.538 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:34.536+0000 7efc4c800700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc4410b760 0x7efc441a0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:51:34.538 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:34.536+0000 7efc4c800700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7efc44115990 con 0x7efc44106120 2024-05-21T21:51:34.538 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:34.536+0000 7efc4c800700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7efc44115810 con 0x7efc44105760 2024-05-21T21:51:34.538 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:34.536+0000 7efc4c800700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7efc44115b10 con 0x7efc4410b760 2024-05-21T21:51:34.539 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:34.536+0000 7efc49e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efc44105760 0x7efc4419be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:51:34.539 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:34.536+0000 7efc49400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc44106120 0x7efc4419c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:51:34.539 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:34.537+0000 7efc49e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efc44105760 0x7efc4419be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:40780/0 (socket says 172.21.15.151:40780) 2024-05-21T21:51:34.539 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:34.537+0000 7efc49e00700 1 -- 172.21.15.151:0/4046700077 learned_addr learned my addr 172.21.15.151:0/4046700077 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:51:34.539 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:34.537+0000 7efc4a800700 1 --2- 172.21.15.151:0/4046700077 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc4410b760 0x7efc441a0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:51:34.539 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:34.537+0000 7efc49e00700 1 -- 172.21.15.151:0/4046700077 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc4410b760 msgr2=0x7efc441a0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:51:34.539 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:34.537+0000 7efc49e00700 1 --2- 172.21.15.151:0/4046700077 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc4410b760 0x7efc441a0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:34.539 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:34.537+0000 7efc49e00700 1 -- 172.21.15.151:0/4046700077 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc44106120 msgr2=0x7efc4419c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:51:34.539 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:34.537+0000 7efc49e00700 1 --2- 172.21.15.151:0/4046700077 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc44106120 0x7efc4419c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:34.539 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:34.537+0000 7efc49e00700 1 -- 172.21.15.151:0/4046700077 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc441a0e70 con 0x7efc44105760 2024-05-21T21:51:34.539 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:34.537+0000 7efc4a800700 1 --2- 172.21.15.151:0/4046700077 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc4410b760 0x7efc441a0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T21:51:34.539 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:34.537+0000 7efc49e00700 1 --2- 172.21.15.151:0/4046700077 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efc44105760 0x7efc4419be60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7efc38005910 tx=0x7efc3800b670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:51:34.540 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:34.537+0000 7efc3ea00700 1 -- 172.21.15.151:0/4046700077 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc38027070 con 0x7efc44105760 2024-05-21T21:51:34.540 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:34.538+0000 7efc3ea00700 1 -- 172.21.15.151:0/4046700077 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc38022470 con 0x7efc44105760 2024-05-21T21:51:34.540 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:34.538+0000 7efc3ea00700 1 -- 172.21.15.151:0/4046700077 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc38017650 con 0x7efc44105760 2024-05-21T21:51:34.540 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:34.538+0000 7efc4c800700 1 -- 172.21.15.151:0/4046700077 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc441a1100 con 0x7efc44105760 2024-05-21T21:51:34.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:34.538+0000 7efc4c800700 1 -- 172.21.15.151:0/4046700077 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc441a89e0 con 0x7efc44105760 2024-05-21T21:51:34.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:34.539+0000 7efc4c800700 1 -- 172.21.15.151:0/4046700077 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc44106ea0 con 0x7efc44105760 2024-05-21T21:51:34.542 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:34.541+0000 7efc3ea00700 1 -- 172.21.15.151:0/4046700077 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7efc3802a070 con 0x7efc44105760 2024-05-21T21:51:34.543 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:34.542+0000 7efc3ea00700 1 --2- 172.21.15.151:0/4046700077 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7efc2c0b2900 0x7efc2c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:51:34.544 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:34.543+0000 7efc3ea00700 1 -- 172.21.15.151:0/4046700077 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7efc380e2e80 con 0x7efc44105760 2024-05-21T21:51:34.544 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:34.543+0000 7efc49400700 1 --2- 172.21.15.151:0/4046700077 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7efc2c0b2900 0x7efc2c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:51:34.544 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:34.543+0000 7efc49400700 1 --2- 172.21.15.151:0/4046700077 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7efc2c0b2900 0x7efc2c0b4dc0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7efc441038b0 tx=0x7efc34009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:51:34.546 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:34.545+0000 7efc3ea00700 1 -- 172.21.15.151:0/4046700077 <== mon.2 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7efc38013070 con 0x7efc44105760 2024-05-21T21:51:34.735 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:34 smithi184 ceph-mon[36942]: pgmap v322: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2024-05-21T21:51:34.833 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:34.832+0000 7efc4c800700 1 -- 172.21.15.151:0/4046700077 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7efc4402e560 con 0x7efc44105760 2024-05-21T21:51:35.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:35.290+0000 7efc3ea00700 1 -- 172.21.15.151:0/4046700077 <== mon.2 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7efc380aace0 con 0x7efc44105760 2024-05-21T21:51:35.296 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:35.295+0000 7efc4c800700 1 -- 172.21.15.151:0/4046700077 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7efc2c0b2900 msgr2=0x7efc2c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:51:35.296 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:35.295+0000 7efc4c800700 1 --2- 172.21.15.151:0/4046700077 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7efc2c0b2900 0x7efc2c0b4dc0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7efc441038b0 tx=0x7efc34009260 comp rx=0 tx=0).stop 2024-05-21T21:51:35.296 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:35.295+0000 7efc4c800700 1 -- 172.21.15.151:0/4046700077 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efc44105760 msgr2=0x7efc4419be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:51:35.297 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:35.295+0000 7efc4c800700 1 --2- 172.21.15.151:0/4046700077 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efc44105760 0x7efc4419be60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7efc38005910 tx=0x7efc3800b670 comp rx=0 tx=0).stop 2024-05-21T21:51:35.297 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:35.295+0000 7efc4c800700 1 -- 172.21.15.151:0/4046700077 shutdown_connections 2024-05-21T21:51:35.297 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:35.295+0000 7efc4c800700 1 --2- 172.21.15.151:0/4046700077 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7efc2c0b2900 0x7efc2c0b4dc0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:35.297 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:35.295+0000 7efc4c800700 1 --2- 172.21.15.151:0/4046700077 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efc44105760 0x7efc4419be60 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:35.297 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:35.295+0000 7efc4c800700 1 --2- 172.21.15.151:0/4046700077 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc4410b760 0x7efc441a0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:35.297 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:35.295+0000 7efc4c800700 1 --2- 172.21.15.151:0/4046700077 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc44106120 0x7efc4419c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:35.297 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:35.295+0000 7efc4c800700 1 -- 172.21.15.151:0/4046700077 >> 172.21.15.151:0/4046700077 conn(0x7efc44101250 msgr2=0x7efc44101a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:51:35.297 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:35.295+0000 7efc4c800700 1 -- 172.21.15.151:0/4046700077 shutdown_connections 2024-05-21T21:51:35.297 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:35.295+0000 7efc4c800700 1 -- 172.21.15.151:0/4046700077 wait complete. 2024-05-21T21:51:35.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:35 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/4046700077' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-21T21:51:35.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:35 smithi164 ceph-mon[36269]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-21T21:51:35.639 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:35 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/4046700077' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-21T21:51:35.639 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:35 smithi151 ceph-mon[31623]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-21T21:51:35.735 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:35 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/4046700077' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-21T21:51:35.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:35 smithi184 ceph-mon[36942]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-21T21:51:36.008 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-05-21T21:51:36.445 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:51:36.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:36 smithi164 ceph-mon[36269]: pgmap v323: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-21T21:51:36.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:36 smithi164 ceph-mon[36269]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-21T21:51:36.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:36 smithi164 ceph-mon[36269]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-05-21T21:51:36.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:36 smithi164 ceph-mon[36269]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-21T21:51:36.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:36 smithi164 ceph-mon[36269]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-21T21:51:36.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:36 smithi164 ceph-mon[36269]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-05-21T21:51:36.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:36 smithi164 ceph-mon[36269]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-05-21T21:51:36.587 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:36 smithi151 ceph-mon[31623]: pgmap v323: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-21T21:51:36.587 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:36 smithi151 ceph-mon[31623]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-21T21:51:36.587 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:36 smithi151 ceph-mon[31623]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-05-21T21:51:36.587 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:36 smithi151 ceph-mon[31623]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-21T21:51:36.587 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:36 smithi151 ceph-mon[31623]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-21T21:51:36.587 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:36 smithi151 ceph-mon[31623]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-05-21T21:51:36.587 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:36 smithi151 ceph-mon[31623]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-05-21T21:51:36.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:36 smithi184 ceph-mon[36942]: pgmap v323: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-21T21:51:36.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:36 smithi184 ceph-mon[36942]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-21T21:51:36.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:36 smithi184 ceph-mon[36942]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-05-21T21:51:36.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:36 smithi184 ceph-mon[36942]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-21T21:51:36.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:36 smithi184 ceph-mon[36942]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-21T21:51:36.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:36 smithi184 ceph-mon[36942]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-05-21T21:51:36.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:36 smithi184 ceph-mon[36942]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-05-21T21:51:37.600 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:37 smithi151 ceph-mon[31623]: mds.? [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] up:active 2024-05-21T21:51:37.600 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:37 smithi151 ceph-mon[31623]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-21T21:51:37.600 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:37 smithi151 ceph-mon[31623]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-05-21T21:51:37.600 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:37 smithi151 ceph-mon[31623]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-05-21T21:51:37.600 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:37 smithi151 ceph-mon[31623]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-05-21T21:51:37.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:37 smithi184 ceph-mon[36942]: mds.? [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] up:active 2024-05-21T21:51:37.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:37 smithi184 ceph-mon[36942]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-21T21:51:37.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:37 smithi184 ceph-mon[36942]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-05-21T21:51:37.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:37 smithi184 ceph-mon[36942]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-05-21T21:51:37.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:37 smithi184 ceph-mon[36942]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-05-21T21:51:37.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:37 smithi164 ceph-mon[36269]: mds.? [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] up:active 2024-05-21T21:51:37.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:37 smithi164 ceph-mon[36269]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-21T21:51:37.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:37 smithi164 ceph-mon[36269]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-05-21T21:51:37.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:37 smithi164 ceph-mon[36269]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-05-21T21:51:37.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:37 smithi164 ceph-mon[36269]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-05-21T21:51:38.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:38.613+0000 7f3965400700 1 -- 172.21.15.151:0/3991998963 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f396010eeb0 msgr2=0x7f3960116a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:51:38.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:38.613+0000 7f3965400700 1 --2- 172.21.15.151:0/3991998963 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f396010eeb0 0x7f3960116a80 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f395800b0d0 tx=0x7f395800b3e0 comp rx=0 tx=0).stop 2024-05-21T21:51:38.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:38.613+0000 7f3965400700 1 -- 172.21.15.151:0/3991998963 shutdown_connections 2024-05-21T21:51:38.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:38.613+0000 7f3965400700 1 --2- 172.21.15.151:0/3991998963 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3960109dd0 0x7f396010a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:38.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:38.613+0000 7f3965400700 1 --2- 172.21.15.151:0/3991998963 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f396010eeb0 0x7f3960116a80 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:38.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:38.613+0000 7f3965400700 1 --2- 172.21.15.151:0/3991998963 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f396010a780 0x7f396010e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:38.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:38.613+0000 7f3965400700 1 -- 172.21.15.151:0/3991998963 >> 172.21.15.151:0/3991998963 conn(0x7f3960076d10 msgr2=0x7f3960108360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:51:38.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:38.613+0000 7f3965400700 1 -- 172.21.15.151:0/3991998963 shutdown_connections 2024-05-21T21:51:38.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:38.613+0000 7f3965400700 1 -- 172.21.15.151:0/3991998963 wait complete. 2024-05-21T21:51:38.616 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:38 smithi151 ceph-mon[31623]: pgmap v324: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 2 op/s 2024-05-21T21:51:38.616 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:38 smithi151 ceph-mon[31623]: mds.? [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] up:active 2024-05-21T21:51:38.616 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:38 smithi151 ceph-mon[31623]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-05-21T21:51:38.616 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:38 smithi151 ceph-mon[31623]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-05-21T21:51:38.616 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:38 smithi151 ceph-mon[31623]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2024-05-21T21:51:38.616 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:38 smithi151 ceph-mon[31623]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-05-21T21:51:38.617 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:38.614+0000 7f3965400700 1 Processor -- start 2024-05-21T21:51:38.617 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:38.615+0000 7f3965400700 1 -- start start 2024-05-21T21:51:38.617 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:38.615+0000 7f3965400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3960109dd0 0x7f39601a0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:51:38.618 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:38.615+0000 7f3965400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f396010a780 0x7f39601a07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:51:38.618 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:38.615+0000 7f3965400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f396010eeb0 0x7f39601a4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:51:38.618 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:38.615+0000 7f3965400700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3960119db0 con 0x7f3960109dd0 2024-05-21T21:51:38.618 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:38.615+0000 7f3965400700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3960119c30 con 0x7f396010a780 2024-05-21T21:51:38.618 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:38.615+0000 7f3965400700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3960119f30 con 0x7f396010eeb0 2024-05-21T21:51:38.618 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:38.615+0000 7f395e800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3960109dd0 0x7f39601a0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:51:38.618 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:38.615+0000 7f395de00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f396010a780 0x7f39601a07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:51:38.618 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:38.616+0000 7f395f200700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f396010eeb0 0x7f39601a4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:51:38.618 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:38.616+0000 7f395de00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f396010a780 0x7f39601a07c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:54162/0 (socket says 172.21.15.151:54162) 2024-05-21T21:51:38.618 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:38.616+0000 7f395de00700 1 -- 172.21.15.151:0/704793360 learned_addr learned my addr 172.21.15.151:0/704793360 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:51:38.618 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:38.616+0000 7f395de00700 1 -- 172.21.15.151:0/704793360 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f396010eeb0 msgr2=0x7f39601a4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:51:38.618 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:38.616+0000 7f395de00700 1 --2- 172.21.15.151:0/704793360 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f396010eeb0 0x7f39601a4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:38.619 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:38.616+0000 7f395de00700 1 -- 172.21.15.151:0/704793360 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3960109dd0 msgr2=0x7f39601a0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:51:38.619 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:38.616+0000 7f395de00700 1 --2- 172.21.15.151:0/704793360 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3960109dd0 0x7f39601a0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:38.619 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:38.616+0000 7f395de00700 1 -- 172.21.15.151:0/704793360 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39601a5240 con 0x7f396010a780 2024-05-21T21:51:38.619 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:38.616+0000 7f395f200700 1 --2- 172.21.15.151:0/704793360 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f396010eeb0 0x7f39601a4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:51:38.619 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:38.616+0000 7f395de00700 1 --2- 172.21.15.151:0/704793360 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f396010a780 0x7f39601a07c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f394c00e9f0 tx=0x7f394c00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:51:38.620 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:38.617+0000 7f3957400700 1 -- 172.21.15.151:0/704793360 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f394c004a30 con 0x7f396010a780 2024-05-21T21:51:38.620 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:38.617+0000 7f3957400700 1 -- 172.21.15.151:0/704793360 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f394c00cc70 con 0x7f396010a780 2024-05-21T21:51:38.620 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:38.617+0000 7f3965400700 1 -- 172.21.15.151:0/704793360 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39601a5530 con 0x7f396010a780 2024-05-21T21:51:38.620 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:38.617+0000 7f3957400700 1 -- 172.21.15.151:0/704793360 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f394c005530 con 0x7f396010a780 2024-05-21T21:51:38.620 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:38.617+0000 7f3965400700 1 -- 172.21.15.151:0/704793360 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39601acdb0 con 0x7f396010a780 2024-05-21T21:51:38.620 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:38.618+0000 7f3965400700 1 -- 172.21.15.151:0/704793360 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f396006a230 con 0x7f396010a780 2024-05-21T21:51:38.622 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:38.621+0000 7f3957400700 1 -- 172.21.15.151:0/704793360 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f394c004bd0 con 0x7f396010a780 2024-05-21T21:51:38.624 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:38.623+0000 7f3957400700 1 --2- 172.21.15.151:0/704793360 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f39440b2900 0x7f39440b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:51:38.624 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:38.623+0000 7f3957400700 1 -- 172.21.15.151:0/704793360 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f394c0e2250 con 0x7f396010a780 2024-05-21T21:51:38.624 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:38.623+0000 7f395e800700 1 --2- 172.21.15.151:0/704793360 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f39440b2900 0x7f39440b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:51:38.624 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:38.624+0000 7f395e800700 1 --2- 172.21.15.151:0/704793360 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f39440b2900 0x7f39440b4dc0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3950007be0 tx=0x7f39500073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:51:38.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:38.627+0000 7f3957400700 1 -- 172.21.15.151:0/704793360 <== mon.2 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f394c00ce20 con 0x7f396010a780 2024-05-21T21:51:38.735 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:38 smithi184 ceph-mon[36942]: pgmap v324: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 2 op/s 2024-05-21T21:51:38.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:38 smithi184 ceph-mon[36942]: mds.? [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] up:active 2024-05-21T21:51:38.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:38 smithi184 ceph-mon[36942]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-05-21T21:51:38.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:38 smithi184 ceph-mon[36942]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-05-21T21:51:38.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:38 smithi184 ceph-mon[36942]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2024-05-21T21:51:38.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:38 smithi184 ceph-mon[36942]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-05-21T21:51:38.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:38 smithi164 ceph-mon[36269]: pgmap v324: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 2 op/s 2024-05-21T21:51:38.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:38 smithi164 ceph-mon[36269]: mds.? [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] up:active 2024-05-21T21:51:38.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:38 smithi164 ceph-mon[36269]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-05-21T21:51:38.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:38 smithi164 ceph-mon[36269]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-05-21T21:51:38.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:38 smithi164 ceph-mon[36269]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2024-05-21T21:51:38.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:38 smithi164 ceph-mon[36269]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-05-21T21:51:38.916 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:38.914+0000 7f3965400700 1 -- 172.21.15.151:0/704793360 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f396002e560 con 0x7f396010a780 2024-05-21T21:51:39.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:39.349+0000 7f3957400700 1 -- 172.21.15.151:0/704793360 <== mon.2 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v26) ==== 121+0+0 (secure 0 0 0) 0x7f394c0aa020 con 0x7f396010a780 2024-05-21T21:51:39.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:39.354+0000 7f3965400700 1 -- 172.21.15.151:0/704793360 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f39440b2900 msgr2=0x7f39440b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:51:39.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:39.354+0000 7f3965400700 1 --2- 172.21.15.151:0/704793360 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f39440b2900 0x7f39440b4dc0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3950007be0 tx=0x7f39500073d0 comp rx=0 tx=0).stop 2024-05-21T21:51:39.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:39.355+0000 7f3965400700 1 -- 172.21.15.151:0/704793360 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f396010a780 msgr2=0x7f39601a07c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:51:39.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:39.355+0000 7f3965400700 1 --2- 172.21.15.151:0/704793360 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f396010a780 0x7f39601a07c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f394c00e9f0 tx=0x7f394c00ed00 comp rx=0 tx=0).stop 2024-05-21T21:51:39.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:39.355+0000 7f3965400700 1 -- 172.21.15.151:0/704793360 shutdown_connections 2024-05-21T21:51:39.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:39.355+0000 7f3965400700 1 --2- 172.21.15.151:0/704793360 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f39440b2900 0x7f39440b4dc0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:39.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:39.355+0000 7f3965400700 1 --2- 172.21.15.151:0/704793360 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f396010eeb0 0x7f39601a4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:39.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:39.355+0000 7f3965400700 1 --2- 172.21.15.151:0/704793360 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3960109dd0 0x7f39601a0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:39.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:39.355+0000 7f3965400700 1 --2- 172.21.15.151:0/704793360 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f396010a780 0x7f39601a07c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:39.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:39.355+0000 7f3965400700 1 -- 172.21.15.151:0/704793360 >> 172.21.15.151:0/704793360 conn(0x7f3960076d10 msgr2=0x7f39600772d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:51:39.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:39.355+0000 7f3965400700 1 -- 172.21.15.151:0/704793360 shutdown_connections 2024-05-21T21:51:39.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:39.355+0000 7f3965400700 1 -- 172.21.15.151:0/704793360 wait complete. 2024-05-21T21:51:39.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:39 smithi151 ceph-mon[31623]: mds.? [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] up:active 2024-05-21T21:51:39.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:39 smithi151 ceph-mon[31623]: daemon mds.e assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-05-21T21:51:39.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:39 smithi151 ceph-mon[31623]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-21T21:51:39.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:39 smithi151 ceph-mon[31623]: Cluster is now healthy 2024-05-21T21:51:39.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:39 smithi151 ceph-mon[31623]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 8 up:standby 2024-05-21T21:51:39.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:39 smithi151 ceph-mon[31623]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-05-21T21:51:39.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:39 smithi151 ceph-mon[31623]: daemon mds.e is now active in filesystem cephfs as rank 4 2024-05-21T21:51:39.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:39 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/704793360' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-21T21:51:39.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:39 smithi151 ceph-mon[31623]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-21T21:51:39.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:39 smithi184 ceph-mon[36942]: mds.? [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] up:active 2024-05-21T21:51:39.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:39 smithi184 ceph-mon[36942]: daemon mds.e assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-05-21T21:51:39.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:39 smithi184 ceph-mon[36942]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-21T21:51:39.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:39 smithi184 ceph-mon[36942]: Cluster is now healthy 2024-05-21T21:51:39.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:39 smithi184 ceph-mon[36942]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 8 up:standby 2024-05-21T21:51:39.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:39 smithi184 ceph-mon[36942]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-05-21T21:51:39.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:39 smithi184 ceph-mon[36942]: daemon mds.e is now active in filesystem cephfs as rank 4 2024-05-21T21:51:39.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:39 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/704793360' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-21T21:51:39.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:39 smithi184 ceph-mon[36942]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-21T21:51:39.786 INFO:journalctl@ceph.mds.b.smithi164.stdout:May 21 21:51:39 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-b[66241]: 2024-05-21T21:51:39.556+0000 7f79f0f0f700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-05-21T21:51:39.786 INFO:journalctl@ceph.mds.b.smithi164.stdout:May 21 21:51:39 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-b[66241]: 2024-05-21T21:51:39.556+0000 7f79f0f0f700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-05-21T21:51:39.786 INFO:journalctl@ceph.mds.b.smithi164.stdout:May 21 21:51:39 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-b[66241]: 2024-05-21T21:51:39.556+0000 7f79f0f0f700 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-05-21T21:51:39.786 INFO:journalctl@ceph.mds.b.smithi164.stdout:May 21 21:51:39 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-b[66241]: 2024-05-21T21:51:39.556+0000 7f79f0f0f700 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-05-21T21:51:39.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:39 smithi164 ceph-mon[36269]: mds.? [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] up:active 2024-05-21T21:51:39.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:39 smithi164 ceph-mon[36269]: daemon mds.e assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-05-21T21:51:39.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:39 smithi164 ceph-mon[36269]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-21T21:51:39.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:39 smithi164 ceph-mon[36269]: Cluster is now healthy 2024-05-21T21:51:39.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:39 smithi164 ceph-mon[36269]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 8 up:standby 2024-05-21T21:51:39.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:39 smithi164 ceph-mon[36269]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-05-21T21:51:39.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:39 smithi164 ceph-mon[36269]: daemon mds.e is now active in filesystem cephfs as rank 4 2024-05-21T21:51:39.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:39 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/704793360' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-21T21:51:39.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:39 smithi164 ceph-mon[36269]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-21T21:51:40.189 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-05-21T21:51:40.627 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:51:40.702 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:40 smithi151 ceph-mon[31623]: pgmap v325: 97 pgs: 97 active+clean; 582 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 4 op/s 2024-05-21T21:51:40.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:40 smithi151 ceph-mon[31623]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-21T21:51:40.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:40 smithi151 ceph-mon[31623]: mds.? [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] up:active 2024-05-21T21:51:40.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:40 smithi151 ceph-mon[31623]: fsmap cephfs:5 7 up:standby 5 up:active 2024-05-21T21:51:40.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:40 smithi151 ceph-mon[31623]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T21:51:40.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:40 smithi184 ceph-mon[36942]: pgmap v325: 97 pgs: 97 active+clean; 582 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 4 op/s 2024-05-21T21:51:40.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:40 smithi184 ceph-mon[36942]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-21T21:51:40.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:40 smithi184 ceph-mon[36942]: mds.? [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] up:active 2024-05-21T21:51:40.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:40 smithi184 ceph-mon[36942]: fsmap cephfs:5 7 up:standby 5 up:active 2024-05-21T21:51:40.737 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:40 smithi184 ceph-mon[36942]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T21:51:40.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:40 smithi164 ceph-mon[36269]: pgmap v325: 97 pgs: 97 active+clean; 582 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 4 op/s 2024-05-21T21:51:40.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:40 smithi164 ceph-mon[36269]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-21T21:51:40.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:40 smithi164 ceph-mon[36269]: mds.? [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] up:active 2024-05-21T21:51:40.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:40 smithi164 ceph-mon[36269]: fsmap cephfs:5 7 up:standby 5 up:active 2024-05-21T21:51:40.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:40 smithi164 ceph-mon[36269]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T21:51:42.535 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:42 smithi151 ceph-mon[31623]: pgmap v326: 97 pgs: 97 active+clean; 584 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 3.8 KiB/s wr, 13 op/s 2024-05-21T21:51:42.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:42.705+0000 7fc18a200700 1 -- 172.21.15.151:0/3344887361 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc184106030 msgr2=0x7fc184112540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:51:42.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:42.705+0000 7fc18a200700 1 --2- 172.21.15.151:0/3344887361 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc184106030 0x7fc184112540 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc17c00b0a0 tx=0x7fc17c00b3b0 comp rx=0 tx=0).stop 2024-05-21T21:51:42.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:42.705+0000 7fc18a200700 1 -- 172.21.15.151:0/3344887361 shutdown_connections 2024-05-21T21:51:42.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:42.705+0000 7fc18a200700 1 --2- 172.21.15.151:0/3344887361 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc184106030 0x7fc184112540 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:42.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:42.705+0000 7fc18a200700 1 --2- 172.21.15.151:0/3344887361 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc18410b670 0x7fc18410ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:42.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:42.705+0000 7fc18a200700 1 --2- 172.21.15.151:0/3344887361 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc184105670 0x7fc184105af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:42.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:42.705+0000 7fc18a200700 1 -- 172.21.15.151:0/3344887361 >> 172.21.15.151:0/3344887361 conn(0x7fc184101160 msgr2=0x7fc184103580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:51:42.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:42.706+0000 7fc18a200700 1 -- 172.21.15.151:0/3344887361 shutdown_connections 2024-05-21T21:51:42.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:42.706+0000 7fc18a200700 1 -- 172.21.15.151:0/3344887361 wait complete. 2024-05-21T21:51:42.708 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:42.707+0000 7fc18a200700 1 Processor -- start 2024-05-21T21:51:42.708 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:42.707+0000 7fc18a200700 1 -- start start 2024-05-21T21:51:42.708 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:42.707+0000 7fc18a200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc184105670 0x7fc1841a1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:51:42.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:42.707+0000 7fc18a200700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc184106030 0x7fc1841a1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:51:42.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:42.707+0000 7fc18a200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc18410b670 0x7fc18419bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:51:42.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:42.707+0000 7fc18a200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc184115900 con 0x7fc18410b670 2024-05-21T21:51:42.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:42.707+0000 7fc18a200700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc184115780 con 0x7fc184105670 2024-05-21T21:51:42.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:42.707+0000 7fc18a200700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc184115a80 con 0x7fc184106030 2024-05-21T21:51:42.710 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:42.708+0000 7fc183e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc184106030 0x7fc1841a1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:51:42.710 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:42.708+0000 7fc189800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc18410b670 0x7fc18419bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:51:42.710 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:42.708+0000 7fc183e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc184106030 0x7fc1841a1f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:33212/0 (socket says 172.21.15.151:33212) 2024-05-21T21:51:42.711 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:42.708+0000 7fc183e00700 1 -- 172.21.15.151:0/1772237002 learned_addr learned my addr 172.21.15.151:0/1772237002 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:51:42.711 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:42.708+0000 7fc188e00700 1 --2- 172.21.15.151:0/1772237002 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc184105670 0x7fc1841a1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:51:42.711 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:42.708+0000 7fc183e00700 1 -- 172.21.15.151:0/1772237002 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc184105670 msgr2=0x7fc1841a1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:51:42.711 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:42.708+0000 7fc183e00700 1 --2- 172.21.15.151:0/1772237002 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc184105670 0x7fc1841a1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:42.711 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:42.708+0000 7fc183e00700 1 -- 172.21.15.151:0/1772237002 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc18410b670 msgr2=0x7fc18419bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:51:42.711 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:42.708+0000 7fc183e00700 1 --2- 172.21.15.151:0/1772237002 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc18410b670 0x7fc18419bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:42.711 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:42.708+0000 7fc183e00700 1 -- 172.21.15.151:0/1772237002 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc18419c400 con 0x7fc184106030 2024-05-21T21:51:42.711 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:42.708+0000 7fc189800700 1 --2- 172.21.15.151:0/1772237002 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc18410b670 0x7fc18419bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:51:42.711 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:42.708+0000 7fc183e00700 1 --2- 172.21.15.151:0/1772237002 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc184106030 0x7fc1841a1f90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc17000e9a0 tx=0x7fc17000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:51:42.711 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:42.709+0000 7fc181600700 1 -- 172.21.15.151:0/1772237002 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc17000ca10 con 0x7fc184106030 2024-05-21T21:51:42.711 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:42.709+0000 7fc18a200700 1 -- 172.21.15.151:0/1772237002 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc18419c6f0 con 0x7fc184106030 2024-05-21T21:51:42.711 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:42.709+0000 7fc181600700 1 -- 172.21.15.151:0/1772237002 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc17000cbb0 con 0x7fc184106030 2024-05-21T21:51:42.711 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:42.709+0000 7fc181600700 1 -- 172.21.15.151:0/1772237002 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc170018680 con 0x7fc184106030 2024-05-21T21:51:42.711 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:42.709+0000 7fc18a200700 1 -- 172.21.15.151:0/1772237002 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1841a89d0 con 0x7fc184106030 2024-05-21T21:51:42.712 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:42.710+0000 7fc18a200700 1 -- 172.21.15.151:0/1772237002 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc184106db0 con 0x7fc184106030 2024-05-21T21:51:42.713 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:42.712+0000 7fc181600700 1 -- 172.21.15.151:0/1772237002 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7fc170010430 con 0x7fc184106030 2024-05-21T21:51:42.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:42.714+0000 7fc181600700 1 --2- 172.21.15.151:0/1772237002 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fc16c0b2900 0x7fc16c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:51:42.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:42.714+0000 7fc181600700 1 -- 172.21.15.151:0/1772237002 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fc1700e2610 con 0x7fc184106030 2024-05-21T21:51:42.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:42.714+0000 7fc188e00700 1 --2- 172.21.15.151:0/1772237002 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fc16c0b2900 0x7fc16c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:51:42.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:42.715+0000 7fc188e00700 1 --2- 172.21.15.151:0/1772237002 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fc16c0b2900 0x7fc16c0b4dc0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc178009be0 tx=0x7fc178009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:51:42.717 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:42.716+0000 7fc181600700 1 -- 172.21.15.151:0/1772237002 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc1700aa3e0 con 0x7fc184106030 2024-05-21T21:51:42.735 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:42 smithi184 ceph-mon[36942]: pgmap v326: 97 pgs: 97 active+clean; 584 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 3.8 KiB/s wr, 13 op/s 2024-05-21T21:51:42.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:42 smithi164 ceph-mon[36269]: pgmap v326: 97 pgs: 97 active+clean; 584 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 3.8 KiB/s wr, 13 op/s 2024-05-21T21:51:43.000 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:42.999+0000 7fc18a200700 1 -- 172.21.15.151:0/1772237002 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fc18419cfe0 con 0x7fc184106030 2024-05-21T21:51:43.363 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:43.361+0000 7fc181600700 1 -- 172.21.15.151:0/1772237002 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v28) ==== 115+0+0 (secure 0 0 0) 0x7fc1700a9ce0 con 0x7fc184106030 2024-05-21T21:51:43.368 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:43.367+0000 7fc18a200700 1 -- 172.21.15.151:0/1772237002 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fc16c0b2900 msgr2=0x7fc16c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:51:43.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:43.367+0000 7fc18a200700 1 --2- 172.21.15.151:0/1772237002 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fc16c0b2900 0x7fc16c0b4dc0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc178009be0 tx=0x7fc178009260 comp rx=0 tx=0).stop 2024-05-21T21:51:43.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:43.367+0000 7fc18a200700 1 -- 172.21.15.151:0/1772237002 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc184106030 msgr2=0x7fc1841a1f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:51:43.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:43.367+0000 7fc18a200700 1 --2- 172.21.15.151:0/1772237002 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc184106030 0x7fc1841a1f90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc17000e9a0 tx=0x7fc17000ecb0 comp rx=0 tx=0).stop 2024-05-21T21:51:43.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:43.367+0000 7fc18a200700 1 -- 172.21.15.151:0/1772237002 shutdown_connections 2024-05-21T21:51:43.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:43.367+0000 7fc18a200700 1 --2- 172.21.15.151:0/1772237002 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fc16c0b2900 0x7fc16c0b4dc0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:43.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:43.367+0000 7fc18a200700 1 --2- 172.21.15.151:0/1772237002 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc184105670 0x7fc1841a1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:43.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:43.367+0000 7fc18a200700 1 --2- 172.21.15.151:0/1772237002 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc18410b670 0x7fc18419bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:43.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:43.367+0000 7fc18a200700 1 --2- 172.21.15.151:0/1772237002 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc184106030 0x7fc1841a1f90 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:43.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:43.367+0000 7fc18a200700 1 -- 172.21.15.151:0/1772237002 >> 172.21.15.151:0/1772237002 conn(0x7fc184101160 msgr2=0x7fc1841017c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:51:43.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:43.367+0000 7fc18a200700 1 -- 172.21.15.151:0/1772237002 shutdown_connections 2024-05-21T21:51:43.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:43.367+0000 7fc18a200700 1 -- 172.21.15.151:0/1772237002 wait complete. 2024-05-21T21:51:43.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:43 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/1772237002' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-21T21:51:43.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:43 smithi151 ceph-mon[31623]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-21T21:51:43.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:43 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/1772237002' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-21T21:51:43.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:43 smithi184 ceph-mon[36942]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-21T21:51:43.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:43 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/1772237002' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-21T21:51:43.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:43 smithi164 ceph-mon[36269]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-21T21:51:44.122 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2024-05-21T21:51:44.123 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-05-21T21:51:44.566 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:51:44.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:44 smithi151 ceph-mon[31623]: pgmap v327: 97 pgs: 97 active+clean; 584 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 4.0 KiB/s wr, 18 op/s 2024-05-21T21:51:44.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:44 smithi151 ceph-mon[31623]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-21T21:51:44.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:44 smithi151 ceph-mon[31623]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T21:51:44.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:44 smithi184 ceph-mon[36942]: pgmap v327: 97 pgs: 97 active+clean; 584 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 4.0 KiB/s wr, 18 op/s 2024-05-21T21:51:44.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:44 smithi184 ceph-mon[36942]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-21T21:51:44.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:44 smithi184 ceph-mon[36942]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T21:51:44.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:44 smithi164 ceph-mon[36269]: pgmap v327: 97 pgs: 97 active+clean; 584 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 4.0 KiB/s wr, 18 op/s 2024-05-21T21:51:44.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:44 smithi164 ceph-mon[36269]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-21T21:51:44.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:44 smithi164 ceph-mon[36269]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T21:51:46.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:46 smithi151 ceph-mon[31623]: pgmap v328: 97 pgs: 97 active+clean; 584 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 4.0 KiB/s wr, 18 op/s 2024-05-21T21:51:46.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:46.637+0000 7f342a000700 1 -- 172.21.15.151:0/3222369209 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f342410ff40 msgr2=0x7f3424112330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:51:46.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:46.637+0000 7f342a000700 1 --2- 172.21.15.151:0/3222369209 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f342410ff40 0x7f3424112330 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f341c00b0a0 tx=0x7f341c00b3b0 comp rx=0 tx=0).stop 2024-05-21T21:51:46.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:46.637+0000 7f342a000700 1 -- 172.21.15.151:0/3222369209 shutdown_connections 2024-05-21T21:51:46.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:46.637+0000 7f342a000700 1 --2- 172.21.15.151:0/3222369209 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f342410ff40 0x7f3424112330 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:46.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:46.637+0000 7f342a000700 1 --2- 172.21.15.151:0/3222369209 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3424075770 0x7f3424075b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:46.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:46.637+0000 7f342a000700 1 --2- 172.21.15.151:0/3222369209 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3424076090 0x7f342410fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:46.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:46.637+0000 7f342a000700 1 -- 172.21.15.151:0/3222369209 >> 172.21.15.151:0/3222369209 conn(0x7f342407a6a0 msgr2=0x7f342407aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:51:46.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:46.637+0000 7f342a000700 1 -- 172.21.15.151:0/3222369209 shutdown_connections 2024-05-21T21:51:46.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:46.637+0000 7f342a000700 1 -- 172.21.15.151:0/3222369209 wait complete. 2024-05-21T21:51:46.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:46.638+0000 7f342a000700 1 Processor -- start 2024-05-21T21:51:46.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:46.638+0000 7f342a000700 1 -- start start 2024-05-21T21:51:46.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:46.639+0000 7f342a000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3424075770 0x7f342419be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:51:46.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:46.639+0000 7f342a000700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3424076090 0x7f342419c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:51:46.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:46.639+0000 7f342a000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f342410ff40 0x7f34241a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:51:46.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:46.639+0000 7f342a000700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3424115970 con 0x7f342410ff40 2024-05-21T21:51:46.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:46.639+0000 7f342a000700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f34241157f0 con 0x7f3424075770 2024-05-21T21:51:46.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:46.639+0000 7f342a000700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3424115af0 con 0x7f3424076090 2024-05-21T21:51:46.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:46.639+0000 7f3423400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3424075770 0x7f342419be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:51:46.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:46.639+0000 7f3423400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3424075770 0x7f342419be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:54238/0 (socket says 172.21.15.151:54238) 2024-05-21T21:51:46.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:46.639+0000 7f3423400700 1 -- 172.21.15.151:0/2428768357 learned_addr learned my addr 172.21.15.151:0/2428768357 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:51:46.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:46.639+0000 7f341be00700 1 --2- 172.21.15.151:0/2428768357 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3424076090 0x7f342419c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:51:46.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:46.639+0000 7f3423e00700 1 --2- 172.21.15.151:0/2428768357 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f342410ff40 0x7f34241a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:51:46.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:46.640+0000 7f3423400700 1 -- 172.21.15.151:0/2428768357 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3424076090 msgr2=0x7f342419c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:51:46.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:46.640+0000 7f3423400700 1 --2- 172.21.15.151:0/2428768357 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3424076090 0x7f342419c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:46.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:46.640+0000 7f3423400700 1 -- 172.21.15.151:0/2428768357 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f342410ff40 msgr2=0x7f34241a06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:51:46.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:46.640+0000 7f3423400700 1 --2- 172.21.15.151:0/2428768357 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f342410ff40 0x7f34241a06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:46.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:46.640+0000 7f3423400700 1 -- 172.21.15.151:0/2428768357 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34241a0e00 con 0x7f3424075770 2024-05-21T21:51:46.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:46.640+0000 7f3423400700 1 --2- 172.21.15.151:0/2428768357 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3424075770 0x7f342419be40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f341400e9a0 tx=0x7f341400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:51:46.643 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:46.640+0000 7f3420c00700 1 -- 172.21.15.151:0/2428768357 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341400ca10 con 0x7f3424075770 2024-05-21T21:51:46.643 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:46.640+0000 7f3420c00700 1 -- 172.21.15.151:0/2428768357 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f341400cbb0 con 0x7f3424075770 2024-05-21T21:51:46.643 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:46.640+0000 7f3420c00700 1 -- 172.21.15.151:0/2428768357 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3414018680 con 0x7f3424075770 2024-05-21T21:51:46.643 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:46.640+0000 7f342a000700 1 -- 172.21.15.151:0/2428768357 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34241a10f0 con 0x7f3424075770 2024-05-21T21:51:46.643 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:46.640+0000 7f342a000700 1 -- 172.21.15.151:0/2428768357 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34241a8970 con 0x7f3424075770 2024-05-21T21:51:46.643 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:46.641+0000 7f342a000700 1 -- 172.21.15.151:0/2428768357 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f342406a230 con 0x7f3424075770 2024-05-21T21:51:46.645 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:46.644+0000 7f3420c00700 1 -- 172.21.15.151:0/2428768357 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 16) ==== 149653+0+0 (secure 0 0 0) 0x7f3414010430 con 0x7f3424075770 2024-05-21T21:51:46.647 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:46.645+0000 7f3420c00700 1 --2- 172.21.15.151:0/2428768357 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f34080b2880 0x7f34080b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:51:46.647 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:46.646+0000 7f3420c00700 1 -- 172.21.15.151:0/2428768357 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f34140e2640 con 0x7f3424075770 2024-05-21T21:51:46.647 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:46.646+0000 7f341be00700 1 --2- 172.21.15.151:0/2428768357 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f34080b2880 0x7f34080b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:51:46.647 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:46.646+0000 7f341be00700 1 --2- 172.21.15.151:0/2428768357 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f34080b2880 0x7f34080b4d40 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f3424111c60 tx=0x7f340c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:51:46.648 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:46.647+0000 7f3420c00700 1 -- 172.21.15.151:0/2428768357 <== mon.2 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f34140aa410 con 0x7f3424075770 2024-05-21T21:51:46.735 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:46 smithi184 ceph-mon[36942]: pgmap v328: 97 pgs: 97 active+clean; 584 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 4.0 KiB/s wr, 18 op/s 2024-05-21T21:51:46.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:46 smithi164 ceph-mon[36269]: pgmap v328: 97 pgs: 97 active+clean; 584 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 4.0 KiB/s wr, 18 op/s 2024-05-21T21:51:46.915 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:46.914+0000 7f342a000700 1 -- 172.21.15.151:0/2428768357 --> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f3424063980 con 0x7f34080b2880 2024-05-21T21:51:46.954 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:46.953+0000 7f3420c00700 1 -- 172.21.15.151:0/2428768357 <== mgr.14152 v2:172.21.15.151:6800/2796622483 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3424063980 con 0x7f34080b2880 2024-05-21T21:51:46.957 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:46.956+0000 7f342a000700 1 -- 172.21.15.151:0/2428768357 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f34080b2880 msgr2=0x7f34080b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:51:46.957 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:46.956+0000 7f342a000700 1 --2- 172.21.15.151:0/2428768357 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f34080b2880 0x7f34080b4d40 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f3424111c60 tx=0x7f340c009260 comp rx=0 tx=0).stop 2024-05-21T21:51:46.958 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:46.956+0000 7f342a000700 1 -- 172.21.15.151:0/2428768357 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3424075770 msgr2=0x7f342419be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:51:46.958 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:46.956+0000 7f342a000700 1 --2- 172.21.15.151:0/2428768357 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3424075770 0x7f342419be40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f341400e9a0 tx=0x7f341400ecb0 comp rx=0 tx=0).stop 2024-05-21T21:51:46.958 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:46.956+0000 7f342a000700 1 -- 172.21.15.151:0/2428768357 shutdown_connections 2024-05-21T21:51:46.958 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:46.956+0000 7f342a000700 1 --2- 172.21.15.151:0/2428768357 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f34080b2880 0x7f34080b4d40 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:46.958 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:46.956+0000 7f342a000700 1 --2- 172.21.15.151:0/2428768357 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3424075770 0x7f342419be40 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:46.958 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:46.956+0000 7f342a000700 1 --2- 172.21.15.151:0/2428768357 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f342410ff40 0x7f34241a06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:46.958 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:46.956+0000 7f342a000700 1 --2- 172.21.15.151:0/2428768357 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3424076090 0x7f342419c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:46.958 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:46.956+0000 7f342a000700 1 -- 172.21.15.151:0/2428768357 >> 172.21.15.151:0/2428768357 conn(0x7f342407a6a0 msgr2=0x7f34241007f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:51:46.958 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:46.956+0000 7f342a000700 1 -- 172.21.15.151:0/2428768357 shutdown_connections 2024-05-21T21:51:46.958 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:46.956+0000 7f342a000700 1 -- 172.21.15.151:0/2428768357 wait complete. 2024-05-21T21:51:47.203 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:51:46 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:51:46.926+0000 7ff344a00700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-21T21:51:47.203 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:51:46 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:51:46.926+0000 7ff344a00700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-21T21:51:47.203 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:51:46 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:51:46.926+0000 7ff344a00700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-21T21:51:47.203 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:51:46 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:51:46.926+0000 7ff344a00700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-21T21:51:47.204 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:51:46 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:51:46.926+0000 7ff344a00700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-21T21:51:47.660 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:47 smithi151 ceph-mon[31623]: from='client.24602 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:51:47.719 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-05-21T21:51:47.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:47 smithi184 ceph-mon[36942]: from='client.24602 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:51:47.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:47 smithi164 ceph-mon[36269]: from='client.24602 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:51:48.159 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:51:48.639 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:48 smithi151 ceph-mon[31623]: pgmap v329: 97 pgs: 97 active+clean; 584 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.0 KiB/s wr, 21 op/s 2024-05-21T21:51:48.639 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:48 smithi151 ceph-mon[31623]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-05-21T21:51:48.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:48 smithi184 ceph-mon[36942]: pgmap v329: 97 pgs: 97 active+clean; 584 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.0 KiB/s wr, 21 op/s 2024-05-21T21:51:48.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:48 smithi184 ceph-mon[36942]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-05-21T21:51:48.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:48 smithi164 ceph-mon[36269]: pgmap v329: 97 pgs: 97 active+clean; 584 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.0 KiB/s wr, 21 op/s 2024-05-21T21:51:48.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:48 smithi164 ceph-mon[36269]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-05-21T21:51:50.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:50.250+0000 7f0420600700 1 -- 172.21.15.151:0/696324094 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f041810b760 msgr2=0x7f041810bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:51:50.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:50.250+0000 7f0420600700 1 --2- 172.21.15.151:0/696324094 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f041810b760 0x7f041810bb40 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f040c009990 tx=0x7f040c009ca0 comp rx=0 tx=0).stop 2024-05-21T21:51:50.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:50.250+0000 7f0420600700 1 -- 172.21.15.151:0/696324094 shutdown_connections 2024-05-21T21:51:50.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:50.250+0000 7f0420600700 1 --2- 172.21.15.151:0/696324094 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0418106120 0x7f0418112630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:50.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:50.250+0000 7f0420600700 1 --2- 172.21.15.151:0/696324094 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f041810b760 0x7f041810bb40 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:50.253 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:50.250+0000 7f0420600700 1 --2- 172.21.15.151:0/696324094 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0418105760 0x7f0418105be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:50.253 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:50.250+0000 7f0420600700 1 -- 172.21.15.151:0/696324094 >> 172.21.15.151:0/696324094 conn(0x7f0418101210 msgr2=0x7f0418103630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:51:50.253 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:50.251+0000 7f0420600700 1 -- 172.21.15.151:0/696324094 shutdown_connections 2024-05-21T21:51:50.253 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:50.251+0000 7f0420600700 1 -- 172.21.15.151:0/696324094 wait complete. 2024-05-21T21:51:50.253 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:50.252+0000 7f0420600700 1 Processor -- start 2024-05-21T21:51:50.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:50.252+0000 7f0420600700 1 -- start start 2024-05-21T21:51:50.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:50.253+0000 7f0420600700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0418105760 0x7f041819bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:51:50.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:50.253+0000 7f0420600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0418106120 0x7f041819c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:51:50.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:50.253+0000 7f0420600700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f041810b760 0x7f04181a07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:51:50.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:50.253+0000 7f0420600700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f04181159b0 con 0x7f0418106120 2024-05-21T21:51:50.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:50.253+0000 7f0420600700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0418115830 con 0x7f0418105760 2024-05-21T21:51:50.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:50.253+0000 7f0420600700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0418115b30 con 0x7f041810b760 2024-05-21T21:51:50.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:50.253+0000 7f041dc00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0418105760 0x7f041819bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:51:50.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:50.253+0000 7f041d200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0418106120 0x7f041819c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:51:50.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:50.253+0000 7f041dc00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0418105760 0x7f041819bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:54280/0 (socket says 172.21.15.151:54280) 2024-05-21T21:51:50.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:50.253+0000 7f041dc00700 1 -- 172.21.15.151:0/1285455158 learned_addr learned my addr 172.21.15.151:0/1285455158 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:51:50.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:50.253+0000 7f041e600700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f041810b760 0x7f04181a07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:51:50.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:50.253+0000 7f041dc00700 1 -- 172.21.15.151:0/1285455158 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f041810b760 msgr2=0x7f04181a07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:51:50.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:50.253+0000 7f041dc00700 1 --2- 172.21.15.151:0/1285455158 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f041810b760 0x7f04181a07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:50.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:50.253+0000 7f041dc00700 1 -- 172.21.15.151:0/1285455158 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0418106120 msgr2=0x7f041819c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:51:50.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:50.253+0000 7f041dc00700 1 --2- 172.21.15.151:0/1285455158 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0418106120 0x7f041819c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:50.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:50.253+0000 7f041dc00700 1 -- 172.21.15.151:0/1285455158 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04181a0ef0 con 0x7f0418105760 2024-05-21T21:51:50.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:50.254+0000 7f041dc00700 1 --2- 172.21.15.151:0/1285455158 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0418105760 0x7f041819bee0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f040c000c00 tx=0x7f040c0057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:51:50.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:50.254+0000 7f0416000700 1 -- 172.21.15.151:0/1285455158 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f040c027070 con 0x7f0418105760 2024-05-21T21:51:50.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:50.254+0000 7f0416000700 1 -- 172.21.15.151:0/1285455158 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f040c022470 con 0x7f0418105760 2024-05-21T21:51:50.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:50.254+0000 7f0420600700 1 -- 172.21.15.151:0/1285455158 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04181a1180 con 0x7f0418105760 2024-05-21T21:51:50.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:50.254+0000 7f0416000700 1 -- 172.21.15.151:0/1285455158 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f040c017650 con 0x7f0418105760 2024-05-21T21:51:50.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:50.254+0000 7f0420600700 1 -- 172.21.15.151:0/1285455158 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04181a8a60 con 0x7f0418105760 2024-05-21T21:51:50.257 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:50.255+0000 7f0420600700 1 -- 172.21.15.151:0/1285455158 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0418106ea0 con 0x7f0418105760 2024-05-21T21:51:50.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:50.258+0000 7f0416000700 1 -- 172.21.15.151:0/1285455158 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f040c02a070 con 0x7f0418105760 2024-05-21T21:51:50.260 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:50.259+0000 7f0416000700 1 --2- 172.21.15.151:0/1285455158 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f04000b2910 0x7f04000b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:51:50.260 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:50.259+0000 7f0416000700 1 -- 172.21.15.151:0/1285455158 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f040c0e3080 con 0x7f0418105760 2024-05-21T21:51:50.260 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:50.259+0000 7f041d200700 1 --2- 172.21.15.151:0/1285455158 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f04000b2910 0x7f04000b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:51:50.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:50.260+0000 7f041d200700 1 --2- 172.21.15.151:0/1285455158 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f04000b2910 0x7f04000b4dd0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f041810ae60 tx=0x7f0404009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:51:50.262 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:50.262+0000 7f0416000700 1 -- 172.21.15.151:0/1285455158 <== mon.2 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f040c013070 con 0x7f0418105760 2024-05-21T21:51:50.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:50.529+0000 7f0420600700 1 -- 172.21.15.151:0/1285455158 --> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f041819d240 con 0x7f04000b2910 2024-05-21T21:51:50.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:50.572+0000 7f0416000700 1 -- 172.21.15.151:0/1285455158 <== mgr.14152 v2:172.21.15.151:6800/2796622483 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f041819d240 con 0x7f04000b2910 2024-05-21T21:51:50.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:50.576+0000 7f0420600700 1 -- 172.21.15.151:0/1285455158 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f04000b2910 msgr2=0x7f04000b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:51:50.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:50.576+0000 7f0420600700 1 --2- 172.21.15.151:0/1285455158 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f04000b2910 0x7f04000b4dd0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f041810ae60 tx=0x7f0404009260 comp rx=0 tx=0).stop 2024-05-21T21:51:50.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:50.576+0000 7f0420600700 1 -- 172.21.15.151:0/1285455158 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0418105760 msgr2=0x7f041819bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:51:50.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:50.576+0000 7f0420600700 1 --2- 172.21.15.151:0/1285455158 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0418105760 0x7f041819bee0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f040c000c00 tx=0x7f040c0057c0 comp rx=0 tx=0).stop 2024-05-21T21:51:50.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:50.576+0000 7f0420600700 1 -- 172.21.15.151:0/1285455158 shutdown_connections 2024-05-21T21:51:50.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:50.576+0000 7f0420600700 1 --2- 172.21.15.151:0/1285455158 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f04000b2910 0x7f04000b4dd0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:50.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:50.576+0000 7f0420600700 1 --2- 172.21.15.151:0/1285455158 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0418105760 0x7f041819bee0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:50.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:50.576+0000 7f0420600700 1 --2- 172.21.15.151:0/1285455158 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f041810b760 0x7f04181a07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:50.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:50.576+0000 7f0420600700 1 --2- 172.21.15.151:0/1285455158 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0418106120 0x7f041819c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:50.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:50.576+0000 7f0420600700 1 -- 172.21.15.151:0/1285455158 >> 172.21.15.151:0/1285455158 conn(0x7f0418101210 msgr2=0x7f0418101a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:51:50.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:50.576+0000 7f0420600700 1 -- 172.21.15.151:0/1285455158 shutdown_connections 2024-05-21T21:51:50.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:50.576+0000 7f0420600700 1 -- 172.21.15.151:0/1285455158 wait complete. 2024-05-21T21:51:50.702 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:50 smithi151 ceph-mon[31623]: pgmap v330: 97 pgs: 97 active+clean; 585 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.9 KiB/s wr, 20 op/s 2024-05-21T21:51:50.738 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:50 smithi184 ceph-mon[36942]: pgmap v330: 97 pgs: 97 active+clean; 585 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.9 KiB/s wr, 20 op/s 2024-05-21T21:51:50.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:50 smithi164 ceph-mon[36269]: pgmap v330: 97 pgs: 97 active+clean; 585 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.9 KiB/s wr, 20 op/s 2024-05-21T21:51:51.325 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-05-21T21:51:51.702 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:51 smithi151 ceph-mon[31623]: from='client.24614 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:51:51.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:51 smithi184 ceph-mon[36942]: from='client.24614 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:51:51.770 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:51:51.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:51 smithi164 ceph-mon[36269]: from='client.24614 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:51:52.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:52 smithi151 ceph-mon[31623]: pgmap v331: 97 pgs: 97 active+clean; 585 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.3 KiB/s wr, 19 op/s 2024-05-21T21:51:52.735 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:52 smithi184 ceph-mon[36942]: pgmap v331: 97 pgs: 97 active+clean; 585 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.3 KiB/s wr, 19 op/s 2024-05-21T21:51:52.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:52 smithi164 ceph-mon[36269]: pgmap v331: 97 pgs: 97 active+clean; 585 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.3 KiB/s wr, 19 op/s 2024-05-21T21:51:53.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:53.801+0000 7f74ed400700 1 -- 172.21.15.151:0/669475090 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74e81046a0 msgr2=0x7f74e8104a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:51:53.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:53.801+0000 7f74ed400700 1 --2- 172.21.15.151:0/669475090 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74e81046a0 0x7f74e8104a80 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f74d8009990 tx=0x7f74d8009ca0 comp rx=0 tx=0).stop 2024-05-21T21:51:53.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:53.802+0000 7f74ed400700 1 -- 172.21.15.151:0/669475090 shutdown_connections 2024-05-21T21:51:53.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:53.802+0000 7f74ed400700 1 --2- 172.21.15.151:0/669475090 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74e810dad0 0x7f74e810ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:53.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:53.802+0000 7f74ed400700 1 --2- 172.21.15.151:0/669475090 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74e81046a0 0x7f74e8104a80 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:53.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:53.802+0000 7f74ed400700 1 --2- 172.21.15.151:0/669475090 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74e8104fc0 0x7f74e810d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:53.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:53.802+0000 7f74ed400700 1 -- 172.21.15.151:0/669475090 >> 172.21.15.151:0/669475090 conn(0x7f74e80fdef0 msgr2=0x7f74e8100310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:51:53.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:53.802+0000 7f74ed400700 1 -- 172.21.15.151:0/669475090 shutdown_connections 2024-05-21T21:51:53.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:53.802+0000 7f74ed400700 1 -- 172.21.15.151:0/669475090 wait complete. 2024-05-21T21:51:53.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:53.803+0000 7f74ed400700 1 Processor -- start 2024-05-21T21:51:53.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:53.803+0000 7f74ed400700 1 -- start start 2024-05-21T21:51:53.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:53.804+0000 7f74ed400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74e81046a0 0x7f74e8102300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:51:53.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:53.804+0000 7f74ed400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74e8104fc0 0x7f74e8102840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:51:53.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:53.804+0000 7f74ed400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74e810dad0 0x7f74e8102d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:51:53.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:53.804+0000 7f74ed400700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f74e81136e0 con 0x7f74e810dad0 2024-05-21T21:51:53.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:53.804+0000 7f74ed400700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f74e8113560 con 0x7f74e81046a0 2024-05-21T21:51:53.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:53.804+0000 7f74ed400700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f74e8113860 con 0x7f74e8104fc0 2024-05-21T21:51:53.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:53.804+0000 7f74e7400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74e8104fc0 0x7f74e8102840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:51:53.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:53.804+0000 7f74eca00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74e810dad0 0x7f74e8102d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:51:53.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:53.804+0000 7f74e7400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74e8104fc0 0x7f74e8102840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:52882/0 (socket says 172.21.15.151:52882) 2024-05-21T21:51:53.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:53.804+0000 7f74e7e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74e81046a0 0x7f74e8102300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:51:53.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:53.804+0000 7f74e7400700 1 -- 172.21.15.151:0/1325559293 learned_addr learned my addr 172.21.15.151:0/1325559293 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:51:53.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:53.805+0000 7f74e7400700 1 -- 172.21.15.151:0/1325559293 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74e81046a0 msgr2=0x7f74e8102300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:51:53.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:53.805+0000 7f74e7400700 1 --2- 172.21.15.151:0/1325559293 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74e81046a0 0x7f74e8102300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:53.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:53.805+0000 7f74e7400700 1 -- 172.21.15.151:0/1325559293 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74e810dad0 msgr2=0x7f74e8102d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:51:53.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:53.805+0000 7f74e7400700 1 --2- 172.21.15.151:0/1325559293 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74e810dad0 0x7f74e8102d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:53.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:53.805+0000 7f74e7400700 1 -- 172.21.15.151:0/1325559293 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74e81a81d0 con 0x7f74e8104fc0 2024-05-21T21:51:53.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:53.805+0000 7f74e7400700 1 --2- 172.21.15.151:0/1325559293 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74e8104fc0 0x7f74e8102840 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f74d400e9f0 tx=0x7f74d400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:51:53.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:53.805+0000 7f74e4c00700 1 -- 172.21.15.151:0/1325559293 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d4004a30 con 0x7f74e8104fc0 2024-05-21T21:51:53.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:53.805+0000 7f74ed400700 1 -- 172.21.15.151:0/1325559293 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74e81a84c0 con 0x7f74e8104fc0 2024-05-21T21:51:53.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:53.805+0000 7f74e4c00700 1 -- 172.21.15.151:0/1325559293 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74d400cc70 con 0x7f74e8104fc0 2024-05-21T21:51:53.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:53.805+0000 7f74e4c00700 1 -- 172.21.15.151:0/1325559293 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d4005530 con 0x7f74e8104fc0 2024-05-21T21:51:53.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:53.805+0000 7f74ed400700 1 -- 172.21.15.151:0/1325559293 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74e81a8a00 con 0x7f74e8104fc0 2024-05-21T21:51:53.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:53.806+0000 7f74ed400700 1 -- 172.21.15.151:0/1325559293 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74e806a230 con 0x7f74e8104fc0 2024-05-21T21:51:53.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:53.809+0000 7f74e4c00700 1 -- 172.21.15.151:0/1325559293 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f74d400c7c0 con 0x7f74e8104fc0 2024-05-21T21:51:53.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:53.810+0000 7f74e4c00700 1 --2- 172.21.15.151:0/1325559293 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f74cc0b2990 0x7f74cc0b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:51:53.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:53.811+0000 7f74e4c00700 1 -- 172.21.15.151:0/1325559293 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f74d40e2390 con 0x7f74e8104fc0 2024-05-21T21:51:53.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:53.811+0000 7f74e7e00700 1 --2- 172.21.15.151:0/1325559293 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f74cc0b2990 0x7f74cc0b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:51:53.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:53.811+0000 7f74e7e00700 1 --2- 172.21.15.151:0/1325559293 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f74cc0b2990 0x7f74cc0b4e50 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f74d80098e0 tx=0x7f74d8005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:51:53.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:53.812+0000 7f74e4c00700 1 -- 172.21.15.151:0/1325559293 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f74d400ce20 con 0x7f74e8104fc0 2024-05-21T21:51:54.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:54.084+0000 7f74ed400700 1 -- 172.21.15.151:0/1325559293 --> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f74e81037e0 con 0x7f74cc0b2990 2024-05-21T21:51:54.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:54.092+0000 7f74e4c00700 1 -- 172.21.15.151:0/1325559293 <== mgr.14152 v2:172.21.15.151:6800/2796622483 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f74e81037e0 con 0x7f74cc0b2990 2024-05-21T21:51:54.092 INFO:teuthology.orchestra.run.smithi151.stdout:/volumes/qa/sv_0/130cd7e1-798f-4ed7-9efd-359de99ddab4 2024-05-21T21:51:54.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:54.095+0000 7f74ed400700 1 -- 172.21.15.151:0/1325559293 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f74cc0b2990 msgr2=0x7f74cc0b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:51:54.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:54.095+0000 7f74ed400700 1 --2- 172.21.15.151:0/1325559293 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f74cc0b2990 0x7f74cc0b4e50 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f74d80098e0 tx=0x7f74d8005960 comp rx=0 tx=0).stop 2024-05-21T21:51:54.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:54.095+0000 7f74ed400700 1 -- 172.21.15.151:0/1325559293 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74e8104fc0 msgr2=0x7f74e8102840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:51:54.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:54.095+0000 7f74ed400700 1 --2- 172.21.15.151:0/1325559293 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74e8104fc0 0x7f74e8102840 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f74d400e9f0 tx=0x7f74d400ed00 comp rx=0 tx=0).stop 2024-05-21T21:51:54.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:54.095+0000 7f74ed400700 1 -- 172.21.15.151:0/1325559293 shutdown_connections 2024-05-21T21:51:54.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:54.095+0000 7f74ed400700 1 --2- 172.21.15.151:0/1325559293 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f74cc0b2990 0x7f74cc0b4e50 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:54.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:54.095+0000 7f74ed400700 1 --2- 172.21.15.151:0/1325559293 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74e81046a0 0x7f74e8102300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:54.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:54.095+0000 7f74ed400700 1 --2- 172.21.15.151:0/1325559293 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74e810dad0 0x7f74e8102d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:54.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:54.095+0000 7f74ed400700 1 --2- 172.21.15.151:0/1325559293 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74e8104fc0 0x7f74e8102840 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:54.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:54.095+0000 7f74ed400700 1 -- 172.21.15.151:0/1325559293 >> 172.21.15.151:0/1325559293 conn(0x7f74e80fdef0 msgr2=0x7f74e80fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:51:54.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:54.095+0000 7f74ed400700 1 -- 172.21.15.151:0/1325559293 shutdown_connections 2024-05-21T21:51:54.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:54.095+0000 7f74ed400700 1 -- 172.21.15.151:0/1325559293 wait complete. 2024-05-21T21:51:54.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:54 smithi151 ceph-mon[31623]: pgmap v332: 97 pgs: 97 active+clean; 605 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 2.4 KiB/s wr, 14 op/s 2024-05-21T21:51:54.735 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:54 smithi184 ceph-mon[36942]: pgmap v332: 97 pgs: 97 active+clean; 605 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 2.4 KiB/s wr, 14 op/s 2024-05-21T21:51:54.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:54 smithi164 ceph-mon[36269]: pgmap v332: 97 pgs: 97 active+clean; 605 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 2.4 KiB/s wr, 14 op/s 2024-05-21T21:51:54.830 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-05-21T21:51:55.273 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:51:55.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:55 smithi151 ceph-mon[31623]: from='client.24607 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:51:55.735 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:55 smithi184 ceph-mon[36942]: from='client.24607 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:51:55.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:55 smithi164 ceph-mon[36269]: from='client.24607 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:51:56.503 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:56 smithi151 ceph-mon[31623]: pgmap v333: 97 pgs: 97 active+clean; 605 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.3 KiB/s wr, 10 op/s 2024-05-21T21:51:56.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:56 smithi184 ceph-mon[36942]: pgmap v333: 97 pgs: 97 active+clean; 605 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.3 KiB/s wr, 10 op/s 2024-05-21T21:51:56.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:56 smithi164 ceph-mon[36269]: pgmap v333: 97 pgs: 97 active+clean; 605 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.3 KiB/s wr, 10 op/s 2024-05-21T21:51:57.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:57.410+0000 7fb290600700 1 -- 172.21.15.151:0/396671851 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb288105780 msgr2=0x7fb288105c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:51:57.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:57.410+0000 7fb290600700 1 --2- 172.21.15.151:0/396671851 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb288105780 0x7fb288105c00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb2740098f0 tx=0x7fb274009c00 comp rx=0 tx=0).stop 2024-05-21T21:51:57.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:57.410+0000 7fb290600700 1 -- 172.21.15.151:0/396671851 shutdown_connections 2024-05-21T21:51:57.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:57.410+0000 7fb290600700 1 --2- 172.21.15.151:0/396671851 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb288106140 0x7fb288112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:57.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:57.410+0000 7fb290600700 1 --2- 172.21.15.151:0/396671851 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb28810b780 0x7fb28810bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:57.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:57.410+0000 7fb290600700 1 --2- 172.21.15.151:0/396671851 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb288105780 0x7fb288105c00 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:57.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:57.410+0000 7fb290600700 1 -- 172.21.15.151:0/396671851 >> 172.21.15.151:0/396671851 conn(0x7fb288101270 msgr2=0x7fb288103690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:51:57.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:57.411+0000 7fb290600700 1 -- 172.21.15.151:0/396671851 shutdown_connections 2024-05-21T21:51:57.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:57.411+0000 7fb290600700 1 -- 172.21.15.151:0/396671851 wait complete. 2024-05-21T21:51:57.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:57.412+0000 7fb290600700 1 Processor -- start 2024-05-21T21:51:57.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:57.412+0000 7fb290600700 1 -- start start 2024-05-21T21:51:57.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:57.413+0000 7fb290600700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb288105780 0x7fb28819be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:51:57.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:57.413+0000 7fb290600700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb288106140 0x7fb28819c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:51:57.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:57.413+0000 7fb290600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb28810b780 0x7fb2881a0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:51:57.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:57.413+0000 7fb290600700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb288115a10 con 0x7fb28810b780 2024-05-21T21:51:57.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:57.413+0000 7fb290600700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb288115890 con 0x7fb288106140 2024-05-21T21:51:57.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:57.413+0000 7fb290600700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb288115b90 con 0x7fb288105780 2024-05-21T21:51:57.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:57.413+0000 7fb28d200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb288106140 0x7fb28819c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:51:57.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:57.413+0000 7fb28dc00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb288105780 0x7fb28819be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:51:57.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:57.413+0000 7fb28d200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb288106140 0x7fb28819c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:40144/0 (socket says 172.21.15.151:40144) 2024-05-21T21:51:57.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:57.413+0000 7fb28d200700 1 -- 172.21.15.151:0/53579496 learned_addr learned my addr 172.21.15.151:0/53579496 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:51:57.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:57.413+0000 7fb28e600700 1 --2- 172.21.15.151:0/53579496 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb28810b780 0x7fb2881a0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:51:57.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:57.413+0000 7fb28d200700 1 -- 172.21.15.151:0/53579496 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb288105780 msgr2=0x7fb28819be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:51:57.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:57.414+0000 7fb28d200700 1 --2- 172.21.15.151:0/53579496 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb288105780 0x7fb28819be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:57.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:57.414+0000 7fb28d200700 1 -- 172.21.15.151:0/53579496 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb28810b780 msgr2=0x7fb2881a0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:51:57.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:57.414+0000 7fb28d200700 1 --2- 172.21.15.151:0/53579496 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb28810b780 0x7fb2881a0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:57.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:57.414+0000 7fb28d200700 1 -- 172.21.15.151:0/53579496 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2881a0e90 con 0x7fb288106140 2024-05-21T21:51:57.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:57.414+0000 7fb28dc00700 1 --2- 172.21.15.151:0/53579496 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb288105780 0x7fb28819be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:51:57.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:57.414+0000 7fb28d200700 1 --2- 172.21.15.151:0/53579496 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb288106140 0x7fb28819c3c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb274009230 tx=0x7fb27400fa70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:51:57.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:57.414+0000 7fb286000700 1 -- 172.21.15.151:0/53579496 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2740050c0 con 0x7fb288106140 2024-05-21T21:51:57.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:57.414+0000 7fb286000700 1 -- 172.21.15.151:0/53579496 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb27400b560 con 0x7fb288106140 2024-05-21T21:51:57.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:57.414+0000 7fb286000700 1 -- 172.21.15.151:0/53579496 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb274003a00 con 0x7fb288106140 2024-05-21T21:51:57.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:57.414+0000 7fb290600700 1 -- 172.21.15.151:0/53579496 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2881a1120 con 0x7fb288106140 2024-05-21T21:51:57.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:57.414+0000 7fb290600700 1 -- 172.21.15.151:0/53579496 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2881a8a00 con 0x7fb288106140 2024-05-21T21:51:57.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:57.415+0000 7fb290600700 1 -- 172.21.15.151:0/53579496 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb28806a230 con 0x7fb288106140 2024-05-21T21:51:57.417 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:57.418+0000 7fb286000700 1 -- 172.21.15.151:0/53579496 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fb27400b710 con 0x7fb288106140 2024-05-21T21:51:57.419 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:57.419+0000 7fb286000700 1 --2- 172.21.15.151:0/53579496 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fb2700b2910 0x7fb2700b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:51:57.419 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:57.419+0000 7fb286000700 1 -- 172.21.15.151:0/53579496 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fb2740e33a0 con 0x7fb288106140 2024-05-21T21:51:57.419 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:57.420+0000 7fb28dc00700 1 --2- 172.21.15.151:0/53579496 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fb2700b2910 0x7fb2700b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:51:57.420 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:57.420+0000 7fb28dc00700 1 --2- 172.21.15.151:0/53579496 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fb2700b2910 0x7fb2700b4dd0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb27c009fd0 tx=0x7fb27c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:51:57.421 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:57.422+0000 7fb286000700 1 -- 172.21.15.151:0/53579496 <== mon.2 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb274013030 con 0x7fb288106140 2024-05-21T21:51:57.690 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:57.690+0000 7fb290600700 1 -- 172.21.15.151:0/53579496 --> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fb28819d1e0 con 0x7fb2700b2910 2024-05-21T21:51:57.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:57.730+0000 7fb286000700 1 -- 172.21.15.151:0/53579496 <== mgr.14152 v2:172.21.15.151:6800/2796622483 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb28819d1e0 con 0x7fb2700b2910 2024-05-21T21:51:57.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:57.733+0000 7fb290600700 1 -- 172.21.15.151:0/53579496 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fb2700b2910 msgr2=0x7fb2700b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:51:57.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:57.733+0000 7fb290600700 1 --2- 172.21.15.151:0/53579496 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fb2700b2910 0x7fb2700b4dd0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb27c009fd0 tx=0x7fb27c009260 comp rx=0 tx=0).stop 2024-05-21T21:51:57.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:57.733+0000 7fb290600700 1 -- 172.21.15.151:0/53579496 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb288106140 msgr2=0x7fb28819c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:51:57.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:57.733+0000 7fb290600700 1 --2- 172.21.15.151:0/53579496 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb288106140 0x7fb28819c3c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb274009230 tx=0x7fb27400fa70 comp rx=0 tx=0).stop 2024-05-21T21:51:57.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:57.733+0000 7fb290600700 1 -- 172.21.15.151:0/53579496 shutdown_connections 2024-05-21T21:51:57.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:57.733+0000 7fb290600700 1 --2- 172.21.15.151:0/53579496 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fb2700b2910 0x7fb2700b4dd0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:57.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:57.733+0000 7fb290600700 1 --2- 172.21.15.151:0/53579496 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb28810b780 0x7fb2881a0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:57.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:57.733+0000 7fb290600700 1 --2- 172.21.15.151:0/53579496 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb288105780 0x7fb28819be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:57.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:57.733+0000 7fb290600700 1 --2- 172.21.15.151:0/53579496 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb288106140 0x7fb28819c3c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:51:57.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:57.733+0000 7fb290600700 1 -- 172.21.15.151:0/53579496 >> 172.21.15.151:0/53579496 conn(0x7fb288101270 msgr2=0x7fb288101a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:51:57.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:57.733+0000 7fb290600700 1 -- 172.21.15.151:0/53579496 shutdown_connections 2024-05-21T21:51:57.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:51:57.734+0000 7fb290600700 1 -- 172.21.15.151:0/53579496 wait complete. 2024-05-21T21:51:58.520 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-05-21T21:51:58.702 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:58 smithi151 ceph-mon[31623]: pgmap v334: 97 pgs: 97 active+clean; 611 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 2.9 KiB/s wr, 12 op/s 2024-05-21T21:51:58.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:51:58 smithi151 ceph-mon[31623]: from='client.24626 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:51:58.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:58 smithi184 ceph-mon[36942]: pgmap v334: 97 pgs: 97 active+clean; 611 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 2.9 KiB/s wr, 12 op/s 2024-05-21T21:51:58.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:51:58 smithi184 ceph-mon[36942]: from='client.24626 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:51:58.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:58 smithi164 ceph-mon[36269]: pgmap v334: 97 pgs: 97 active+clean; 611 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 2.9 KiB/s wr, 12 op/s 2024-05-21T21:51:58.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:51:58 smithi164 ceph-mon[36269]: from='client.24626 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:51:58.975 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:52:00.605 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:52:00 smithi151 ceph-mon[31623]: pgmap v335: 97 pgs: 97 active+clean; 611 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 2.9 KiB/s wr, 10 op/s 2024-05-21T21:52:00.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:52:00 smithi184 ceph-mon[36942]: pgmap v335: 97 pgs: 97 active+clean; 611 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 2.9 KiB/s wr, 10 op/s 2024-05-21T21:52:00.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:52:00 smithi164 ceph-mon[36269]: pgmap v335: 97 pgs: 97 active+clean; 611 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 2.9 KiB/s wr, 10 op/s 2024-05-21T21:52:01.064 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:01.063+0000 7f931b200700 1 -- 172.21.15.151:0/3713000710 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f931c106010 msgr2=0x7f931c112520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:52:01.064 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:01.063+0000 7f931b200700 1 --2- 172.21.15.151:0/3713000710 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f931c106010 0x7f931c112520 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f931400b0a0 tx=0x7f931400b3b0 comp rx=0 tx=0).stop 2024-05-21T21:52:01.064 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:01.063+0000 7f931b200700 1 -- 172.21.15.151:0/3713000710 shutdown_connections 2024-05-21T21:52:01.065 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:01.063+0000 7f931b200700 1 --2- 172.21.15.151:0/3713000710 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f931c106010 0x7f931c112520 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:01.065 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:01.063+0000 7f931b200700 1 --2- 172.21.15.151:0/3713000710 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f931c10b650 0x7f931c10ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:01.065 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:01.063+0000 7f931b200700 1 --2- 172.21.15.151:0/3713000710 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f931c105650 0x7f931c105ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:01.065 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:01.063+0000 7f931b200700 1 -- 172.21.15.151:0/3713000710 >> 172.21.15.151:0/3713000710 conn(0x7f931c101180 msgr2=0x7f931c1035a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:52:01.065 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:01.063+0000 7f931b200700 1 -- 172.21.15.151:0/3713000710 shutdown_connections 2024-05-21T21:52:01.065 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:01.063+0000 7f931b200700 1 -- 172.21.15.151:0/3713000710 wait complete. 2024-05-21T21:52:01.067 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:01.064+0000 7f931b200700 1 Processor -- start 2024-05-21T21:52:01.067 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:01.065+0000 7f931b200700 1 -- start start 2024-05-21T21:52:01.067 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:01.065+0000 7f931b200700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f931c105650 0x7f931c1a1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:52:01.067 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:01.065+0000 7f931b200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f931c106010 0x7f931c1a1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:52:01.067 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:01.065+0000 7f931b200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f931c10b650 0x7f931c19bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:52:01.067 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:01.065+0000 7f931b200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f931c115880 con 0x7f931c106010 2024-05-21T21:52:01.067 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:01.065+0000 7f931b200700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f931c115700 con 0x7f931c10b650 2024-05-21T21:52:01.067 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:01.065+0000 7f931b200700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f931c115a00 con 0x7f931c105650 2024-05-21T21:52:01.067 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:01.065+0000 7f9319e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f931c105650 0x7f931c1a1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:52:01.068 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:01.065+0000 7f9319400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f931c106010 0x7f931c1a1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:52:01.068 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:01.065+0000 7f9319e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f931c105650 0x7f931c1a1a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:35984/0 (socket says 172.21.15.151:35984) 2024-05-21T21:52:01.068 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:01.065+0000 7f9319e00700 1 -- 172.21.15.151:0/3651263321 learned_addr learned my addr 172.21.15.151:0/3651263321 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:52:01.068 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:01.066+0000 7f931a800700 1 --2- 172.21.15.151:0/3651263321 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f931c10b650 0x7f931c19bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:52:01.068 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:01.066+0000 7f9319e00700 1 -- 172.21.15.151:0/3651263321 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f931c10b650 msgr2=0x7f931c19bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:52:01.068 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:01.066+0000 7f9319e00700 1 --2- 172.21.15.151:0/3651263321 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f931c10b650 0x7f931c19bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:01.068 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:01.066+0000 7f9319e00700 1 -- 172.21.15.151:0/3651263321 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f931c106010 msgr2=0x7f931c1a1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:52:01.068 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:01.066+0000 7f9319e00700 1 --2- 172.21.15.151:0/3651263321 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f931c106010 0x7f931c1a1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:01.068 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:01.066+0000 7f9319e00700 1 -- 172.21.15.151:0/3651263321 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f931c19c460 con 0x7f931c105650 2024-05-21T21:52:01.068 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:01.066+0000 7f9319e00700 1 --2- 172.21.15.151:0/3651263321 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f931c105650 0x7f931c1a1a60 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f930c00d950 tx=0x7f930c00dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:52:01.069 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:01.067+0000 7f9312a00700 1 -- 172.21.15.151:0/3651263321 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f930c016070 con 0x7f931c105650 2024-05-21T21:52:01.069 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:01.067+0000 7f931b200700 1 -- 172.21.15.151:0/3651263321 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f931c19c750 con 0x7f931c105650 2024-05-21T21:52:01.069 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:01.067+0000 7f9312a00700 1 -- 172.21.15.151:0/3651263321 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f930c00fd40 con 0x7f931c105650 2024-05-21T21:52:01.069 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:01.067+0000 7f9312a00700 1 -- 172.21.15.151:0/3651263321 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f930c00e3b0 con 0x7f931c105650 2024-05-21T21:52:01.069 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:01.067+0000 7f931b200700 1 -- 172.21.15.151:0/3651263321 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f931c1a89e0 con 0x7f931c105650 2024-05-21T21:52:01.069 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:01.068+0000 7f931b200700 1 -- 172.21.15.151:0/3651263321 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f931c106d90 con 0x7f931c105650 2024-05-21T21:52:01.070 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:01.070+0000 7f9312a00700 1 -- 172.21.15.151:0/3651263321 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f930c019070 con 0x7f931c105650 2024-05-21T21:52:01.072 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:01.072+0000 7f9312a00700 1 --2- 172.21.15.151:0/3651263321 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f93000b2990 0x7f93000b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:52:01.072 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:01.072+0000 7f9312a00700 1 -- 172.21.15.151:0/3651263321 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f930c0e1c50 con 0x7f931c105650 2024-05-21T21:52:01.072 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:01.072+0000 7f9319400700 1 --2- 172.21.15.151:0/3651263321 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f93000b2990 0x7f93000b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:52:01.073 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:01.073+0000 7f9319400700 1 --2- 172.21.15.151:0/3651263321 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f93000b2990 0x7f93000b4e50 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f931c19d280 tx=0x7f930800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:52:01.074 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:01.074+0000 7f9312a00700 1 -- 172.21.15.151:0/3651263321 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f930c0a98c0 con 0x7f931c105650 2024-05-21T21:52:01.345 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:01.344+0000 7f931b200700 1 -- 172.21.15.151:0/3651263321 --> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f931c19cff0 con 0x7f93000b2990 2024-05-21T21:52:01.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:01.356+0000 7f9312a00700 1 -- 172.21.15.151:0/3651263321 <== mgr.14152 v2:172.21.15.151:6800/2796622483 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f931c19cff0 con 0x7f93000b2990 2024-05-21T21:52:01.356 INFO:teuthology.orchestra.run.smithi151.stdout:/volumes/qa/sv_1/219d07b0-9264-4d1a-9d62-bff6dec196ce 2024-05-21T21:52:01.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:01.359+0000 7f931b200700 1 -- 172.21.15.151:0/3651263321 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f93000b2990 msgr2=0x7f93000b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:52:01.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:01.359+0000 7f931b200700 1 --2- 172.21.15.151:0/3651263321 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f93000b2990 0x7f93000b4e50 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f931c19d280 tx=0x7f930800b410 comp rx=0 tx=0).stop 2024-05-21T21:52:01.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:01.359+0000 7f931b200700 1 -- 172.21.15.151:0/3651263321 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f931c105650 msgr2=0x7f931c1a1a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:52:01.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:01.359+0000 7f931b200700 1 --2- 172.21.15.151:0/3651263321 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f931c105650 0x7f931c1a1a60 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f930c00d950 tx=0x7f930c00dc60 comp rx=0 tx=0).stop 2024-05-21T21:52:01.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:01.359+0000 7f931b200700 1 -- 172.21.15.151:0/3651263321 shutdown_connections 2024-05-21T21:52:01.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:01.359+0000 7f931b200700 1 --2- 172.21.15.151:0/3651263321 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f93000b2990 0x7f93000b4e50 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:01.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:01.359+0000 7f931b200700 1 --2- 172.21.15.151:0/3651263321 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f931c10b650 0x7f931c19bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:01.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:01.359+0000 7f931b200700 1 --2- 172.21.15.151:0/3651263321 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f931c105650 0x7f931c1a1a60 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:01.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:01.359+0000 7f931b200700 1 --2- 172.21.15.151:0/3651263321 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f931c106010 0x7f931c1a1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:01.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:01.359+0000 7f931b200700 1 -- 172.21.15.151:0/3651263321 >> 172.21.15.151:0/3651263321 conn(0x7f931c101180 msgr2=0x7f931c101830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:52:01.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:01.359+0000 7f931b200700 1 -- 172.21.15.151:0/3651263321 shutdown_connections 2024-05-21T21:52:01.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:01.359+0000 7f931b200700 1 -- 172.21.15.151:0/3651263321 wait complete. 2024-05-21T21:52:02.008 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-21T21:52:02.459 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:52:02.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:52:02 smithi151 ceph-mon[31623]: pgmap v336: 97 pgs: 97 active+clean; 611 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.0 KiB/s wr, 12 op/s 2024-05-21T21:52:02.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:52:02 smithi151 ceph-mon[31623]: from='client.24619 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:52:02.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:52:02 smithi184 ceph-mon[36942]: pgmap v336: 97 pgs: 97 active+clean; 611 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.0 KiB/s wr, 12 op/s 2024-05-21T21:52:02.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:52:02 smithi184 ceph-mon[36942]: from='client.24619 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:52:02.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:52:02 smithi164 ceph-mon[36269]: pgmap v336: 97 pgs: 97 active+clean; 611 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.0 KiB/s wr, 12 op/s 2024-05-21T21:52:02.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:52:02 smithi164 ceph-mon[36269]: from='client.24619 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:52:04.449 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:04.449+0000 7f33c9e00700 1 -- 172.21.15.151:0/3719733230 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f33c410eeb0 msgr2=0x7f33c4116a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:52:04.449 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:04.449+0000 7f33c9e00700 1 --2- 172.21.15.151:0/3719733230 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f33c410eeb0 0x7f33c4116a80 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f33bc00b0d0 tx=0x7f33bc00b3e0 comp rx=0 tx=0).stop 2024-05-21T21:52:04.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:04.449+0000 7f33c9e00700 1 -- 172.21.15.151:0/3719733230 shutdown_connections 2024-05-21T21:52:04.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:04.449+0000 7f33c9e00700 1 --2- 172.21.15.151:0/3719733230 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f33c4109dd0 0x7f33c410a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:04.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:04.449+0000 7f33c9e00700 1 --2- 172.21.15.151:0/3719733230 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f33c410eeb0 0x7f33c4116a80 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:04.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:04.449+0000 7f33c9e00700 1 --2- 172.21.15.151:0/3719733230 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f33c410a780 0x7f33c410e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:04.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:04.449+0000 7f33c9e00700 1 -- 172.21.15.151:0/3719733230 >> 172.21.15.151:0/3719733230 conn(0x7f33c4076d10 msgr2=0x7f33c4108360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:52:04.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:04.449+0000 7f33c9e00700 1 -- 172.21.15.151:0/3719733230 shutdown_connections 2024-05-21T21:52:04.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:04.449+0000 7f33c9e00700 1 -- 172.21.15.151:0/3719733230 wait complete. 2024-05-21T21:52:04.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:04.450+0000 7f33c9e00700 1 Processor -- start 2024-05-21T21:52:04.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:04.450+0000 7f33c9e00700 1 -- start start 2024-05-21T21:52:04.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:04.450+0000 7f33c9e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f33c4109dd0 0x7f33c41a01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:52:04.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:04.451+0000 7f33c9e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f33c410a780 0x7f33c41a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:52:04.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:04.451+0000 7f33c9e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f33c410eeb0 0x7f33c41a4a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:52:04.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:04.451+0000 7f33c9e00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f33c4119d60 con 0x7f33c410eeb0 2024-05-21T21:52:04.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:04.451+0000 7f33c9e00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f33c4119be0 con 0x7f33c410a780 2024-05-21T21:52:04.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:04.451+0000 7f33c9e00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f33c4119ee0 con 0x7f33c4109dd0 2024-05-21T21:52:04.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:04.451+0000 7f33c2000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f33c410a780 0x7f33c41a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:52:04.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:04.451+0000 7f33c3400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f33c410eeb0 0x7f33c41a4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:52:04.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:04.451+0000 7f33c2000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f33c410a780 0x7f33c41a06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:40192/0 (socket says 172.21.15.151:40192) 2024-05-21T21:52:04.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:04.451+0000 7f33c2000700 1 -- 172.21.15.151:0/3150155948 learned_addr learned my addr 172.21.15.151:0/3150155948 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:52:04.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:04.451+0000 7f33c2a00700 1 --2- 172.21.15.151:0/3150155948 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f33c4109dd0 0x7f33c41a01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:52:04.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:04.451+0000 7f33c2000700 1 -- 172.21.15.151:0/3150155948 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f33c4109dd0 msgr2=0x7f33c41a01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:52:04.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:04.451+0000 7f33c2000700 1 --2- 172.21.15.151:0/3150155948 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f33c4109dd0 0x7f33c41a01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:04.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:04.451+0000 7f33c2000700 1 -- 172.21.15.151:0/3150155948 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f33c410eeb0 msgr2=0x7f33c41a4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:52:04.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:04.451+0000 7f33c2000700 1 --2- 172.21.15.151:0/3150155948 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f33c410eeb0 0x7f33c41a4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:04.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:04.451+0000 7f33c2000700 1 -- 172.21.15.151:0/3150155948 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33c41a51c0 con 0x7f33c410a780 2024-05-21T21:52:04.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:04.451+0000 7f33c3400700 1 --2- 172.21.15.151:0/3150155948 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f33c410eeb0 0x7f33c41a4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:52:04.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:04.452+0000 7f33c2000700 1 --2- 172.21.15.151:0/3150155948 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f33c410a780 0x7f33c41a06f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f33b000e9f0 tx=0x7f33b000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:52:04.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:04.452+0000 7f33c8a00700 1 -- 172.21.15.151:0/3150155948 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b0004a30 con 0x7f33c410a780 2024-05-21T21:52:04.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:04.452+0000 7f33c8a00700 1 -- 172.21.15.151:0/3150155948 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33b000cc70 con 0x7f33c410a780 2024-05-21T21:52:04.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:04.452+0000 7f33c9e00700 1 -- 172.21.15.151:0/3150155948 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33c41a54b0 con 0x7f33c410a780 2024-05-21T21:52:04.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:04.452+0000 7f33c8a00700 1 -- 172.21.15.151:0/3150155948 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b0005530 con 0x7f33c410a780 2024-05-21T21:52:04.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:04.452+0000 7f33c9e00700 1 -- 172.21.15.151:0/3150155948 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33c41acd30 con 0x7f33c410a780 2024-05-21T21:52:04.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:04.453+0000 7f33c9e00700 1 -- 172.21.15.151:0/3150155948 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33c406a230 con 0x7f33c410a780 2024-05-21T21:52:04.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:04.456+0000 7f33c8a00700 1 -- 172.21.15.151:0/3150155948 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f33b0004bd0 con 0x7f33c410a780 2024-05-21T21:52:04.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:04.457+0000 7f33c8a00700 1 --2- 172.21.15.151:0/3150155948 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f33a80b2910 0x7f33a80b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:52:04.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:04.457+0000 7f33c8a00700 1 -- 172.21.15.151:0/3150155948 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f33b00e2660 con 0x7f33c410a780 2024-05-21T21:52:04.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:04.457+0000 7f33c2a00700 1 --2- 172.21.15.151:0/3150155948 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f33a80b2910 0x7f33a80b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:52:04.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:04.458+0000 7f33c2a00700 1 --2- 172.21.15.151:0/3150155948 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f33a80b2910 0x7f33a80b4dd0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f33b4009c60 tx=0x7f33b4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:52:04.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:04.459+0000 7f33c8a00700 1 -- 172.21.15.151:0/3150155948 <== mon.2 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f33b0054df0 con 0x7f33c410a780 2024-05-21T21:52:04.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:04.740+0000 7f33c9e00700 1 -- 172.21.15.151:0/3150155948 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f33c402e560 con 0x7f33c410a780 2024-05-21T21:52:04.742 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:04.742+0000 7f33c8a00700 1 -- 172.21.15.151:0/3150155948 <== mon.2 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11958 (secure 0 0 0) 0x7f33b000ce20 con 0x7f33c410a780 2024-05-21T21:52:04.742 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T21:52:04.743 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14358,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/158010529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":158010529},{"type":"v1","addr":"172.21.15.151:6835","nonce":158010529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6839/2069058115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":2069058115},{"type":"v1","addr":"172.21.15.151:6839","nonce":2069058115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T21:51:19.552069+0000","modified":"2024-05-21T21:51:43.002737+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24530,"mds_1":24493,"mds_2":24491,"mds_3":24478,"mds_4":24473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14403":{"gid":14403,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6841/1749723794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6840","nonce":1749723794},{"type":"v1","addr":"172.21.15.151:6841","nonce":1749723794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/2026266114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":2026266114},{"type":"v1","addr":"172.21.15.184:6833","nonce":2026266114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/2467900844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":2467900844},{"type":"v1","addr":"172.21.15.164:6833","nonce":2467900844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3367640960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3367640960},{"type":"v1","addr":"172.21.15.151:6837","nonce":3367640960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/3264142621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3264142621},{"type":"v1","addr":"172.21.15.184:6835","nonce":3264142621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":17,"addr":"172.21.15.164:6835/2340428470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2340428470},{"type":"v1","addr":"172.21.15.164:6835","nonce":2340428470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":16,"addr":"172.21.15.184:6837/2988194888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":2988194888},{"type":"v1","addr":"172.21.15.184:6837","nonce":2988194888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.164:6837/1554609572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1554609572},{"type":"v1","addr":"172.21.15.164:6837","nonce":1554609572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.184:6839/3059935143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":3059935143},{"type":"v1","addr":"172.21.15.184:6839","nonce":3059935143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.164:6839/2646401247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":2646401247},{"type":"v1","addr":"172.21.15.164:6839","nonce":2646401247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24530,"qdb_cluster":[24473,24478,24491,24493,24530]},"id":1}]} 2024-05-21T21:52:04.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:04.745+0000 7f33c9e00700 1 -- 172.21.15.151:0/3150155948 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f33a80b2910 msgr2=0x7f33a80b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:52:04.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:04.745+0000 7f33c9e00700 1 --2- 172.21.15.151:0/3150155948 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f33a80b2910 0x7f33a80b4dd0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f33b4009c60 tx=0x7f33b4009260 comp rx=0 tx=0).stop 2024-05-21T21:52:04.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:04.745+0000 7f33c9e00700 1 -- 172.21.15.151:0/3150155948 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f33c410a780 msgr2=0x7f33c41a06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:52:04.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:04.745+0000 7f33c9e00700 1 --2- 172.21.15.151:0/3150155948 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f33c410a780 0x7f33c41a06f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f33b000e9f0 tx=0x7f33b000ed00 comp rx=0 tx=0).stop 2024-05-21T21:52:04.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:04.745+0000 7f33c9e00700 1 -- 172.21.15.151:0/3150155948 shutdown_connections 2024-05-21T21:52:04.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:04.745+0000 7f33c9e00700 1 --2- 172.21.15.151:0/3150155948 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f33a80b2910 0x7f33a80b4dd0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:04.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:04.745+0000 7f33c9e00700 1 --2- 172.21.15.151:0/3150155948 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f33c410eeb0 0x7f33c41a4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:04.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:04.745+0000 7f33c9e00700 1 --2- 172.21.15.151:0/3150155948 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f33c4109dd0 0x7f33c41a01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:04.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:04.745+0000 7f33c9e00700 1 --2- 172.21.15.151:0/3150155948 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f33c410a780 0x7f33c41a06f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:04.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:04.745+0000 7f33c9e00700 1 -- 172.21.15.151:0/3150155948 >> 172.21.15.151:0/3150155948 conn(0x7f33c4076d10 msgr2=0x7f33c4077290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:52:04.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:04.746+0000 7f33c9e00700 1 -- 172.21.15.151:0/3150155948 shutdown_connections 2024-05-21T21:52:04.747 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:04.746+0000 7f33c9e00700 1 -- 172.21.15.151:0/3150155948 wait complete. 2024-05-21T21:52:04.747 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 28 2024-05-21T21:52:05.115 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:52:04 smithi151 ceph-mon[31623]: pgmap v337: 97 pgs: 97 active+clean; 633 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 4.9 KiB/s wr, 13 op/s 2024-05-21T21:52:05.115 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:52:04 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/3150155948' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T21:52:05.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:52:04 smithi184 ceph-mon[36942]: pgmap v337: 97 pgs: 97 active+clean; 633 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 4.9 KiB/s wr, 13 op/s 2024-05-21T21:52:05.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:52:04 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/3150155948' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T21:52:05.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:52:04 smithi164 ceph-mon[36269]: pgmap v337: 97 pgs: 97 active+clean; 633 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 4.9 KiB/s wr, 13 op/s 2024-05-21T21:52:05.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:52:04 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/3150155948' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T21:52:05.586 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph osd dump --format=json 2024-05-21T21:52:06.044 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:52:06.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:52:06 smithi151 ceph-mon[31623]: pgmap v338: 97 pgs: 97 active+clean; 633 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 3.0 KiB/s wr, 11 op/s 2024-05-21T21:52:07.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:52:06 smithi184 ceph-mon[36942]: pgmap v338: 97 pgs: 97 active+clean; 633 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 3.0 KiB/s wr, 11 op/s 2024-05-21T21:52:07.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:52:06 smithi164 ceph-mon[36269]: pgmap v338: 97 pgs: 97 active+clean; 633 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 3.0 KiB/s wr, 11 op/s 2024-05-21T21:52:08.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:08.300+0000 7fa64a000700 1 -- 172.21.15.151:0/4061610920 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa644076090 msgr2=0x7fa6441141d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:52:08.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:08.300+0000 7fa64a000700 1 --2- 172.21.15.151:0/4061610920 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa644076090 0x7fa6441141d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa630009990 tx=0x7fa630009ca0 comp rx=0 tx=0).stop 2024-05-21T21:52:08.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:08.301+0000 7fa64a000700 1 -- 172.21.15.151:0/4061610920 shutdown_connections 2024-05-21T21:52:08.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:08.301+0000 7fa64a000700 1 --2- 172.21.15.151:0/4061610920 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa644114710 0x7fa644116b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:08.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:08.301+0000 7fa64a000700 1 --2- 172.21.15.151:0/4061610920 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa644075770 0x7fa644075b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:08.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:08.301+0000 7fa64a000700 1 --2- 172.21.15.151:0/4061610920 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa644076090 0x7fa6441141d0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:08.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:08.301+0000 7fa64a000700 1 -- 172.21.15.151:0/4061610920 >> 172.21.15.151:0/4061610920 conn(0x7fa64407a6a0 msgr2=0x7fa64407aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:52:08.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:08.301+0000 7fa64a000700 1 -- 172.21.15.151:0/4061610920 shutdown_connections 2024-05-21T21:52:08.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:08.301+0000 7fa64a000700 1 -- 172.21.15.151:0/4061610920 wait complete. 2024-05-21T21:52:08.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:08.302+0000 7fa64a000700 1 Processor -- start 2024-05-21T21:52:08.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:08.302+0000 7fa64a000700 1 -- start start 2024-05-21T21:52:08.303 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:08.303+0000 7fa64a000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa644075770 0x7fa6441a02c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:52:08.303 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:08.303+0000 7fa64a000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa644076090 0x7fa6441a0800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:52:08.303 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:08.303+0000 7fa64a000700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa644114710 0x7fa6441a4b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:52:08.303 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:08.303+0000 7fa64a000700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa644119d50 con 0x7fa644076090 2024-05-21T21:52:08.303 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:08.303+0000 7fa64a000700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa644119bd0 con 0x7fa644075770 2024-05-21T21:52:08.303 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:08.303+0000 7fa64a000700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa644119ed0 con 0x7fa644114710 2024-05-21T21:52:08.305 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:08.303+0000 7fa643400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa644075770 0x7fa6441a02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:52:08.305 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:08.303+0000 7fa643400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa644075770 0x7fa6441a02c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:42760/0 (socket says 172.21.15.151:42760) 2024-05-21T21:52:08.305 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:08.303+0000 7fa643400700 1 -- 172.21.15.151:0/1207161529 learned_addr learned my addr 172.21.15.151:0/1207161529 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:52:08.305 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:08.303+0000 7fa643e00700 1 --2- 172.21.15.151:0/1207161529 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa644114710 0x7fa6441a4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:52:08.305 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:08.304+0000 7fa642a00700 1 --2- 172.21.15.151:0/1207161529 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa644076090 0x7fa6441a0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:52:08.305 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:08.304+0000 7fa643e00700 1 -- 172.21.15.151:0/1207161529 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa644075770 msgr2=0x7fa6441a02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:52:08.305 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:08.304+0000 7fa643e00700 1 --2- 172.21.15.151:0/1207161529 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa644075770 0x7fa6441a02c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:08.305 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:08.304+0000 7fa643e00700 1 -- 172.21.15.151:0/1207161529 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa644076090 msgr2=0x7fa6441a0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:52:08.305 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:08.304+0000 7fa643e00700 1 --2- 172.21.15.151:0/1207161529 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa644076090 0x7fa6441a0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:08.305 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:08.304+0000 7fa643e00700 1 -- 172.21.15.151:0/1207161529 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6441a52d0 con 0x7fa644114710 2024-05-21T21:52:08.306 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:08.304+0000 7fa642a00700 1 --2- 172.21.15.151:0/1207161529 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa644076090 0x7fa6441a0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T21:52:08.306 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:08.304+0000 7fa643e00700 1 --2- 172.21.15.151:0/1207161529 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa644114710 0x7fa6441a4b90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa63c00ef90 tx=0x7fa63c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:52:08.306 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:08.304+0000 7fa63be00700 1 -- 172.21.15.151:0/1207161529 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa63c0092e0 con 0x7fa644114710 2024-05-21T21:52:08.306 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:08.304+0000 7fa63be00700 1 -- 172.21.15.151:0/1207161529 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa63c004bb0 con 0x7fa644114710 2024-05-21T21:52:08.306 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:08.305+0000 7fa64a000700 1 -- 172.21.15.151:0/1207161529 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6441a5560 con 0x7fa644114710 2024-05-21T21:52:08.306 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:08.305+0000 7fa64a000700 1 -- 172.21.15.151:0/1207161529 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6441ace40 con 0x7fa644114710 2024-05-21T21:52:08.306 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:08.305+0000 7fa63be00700 1 -- 172.21.15.151:0/1207161529 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa63c004280 con 0x7fa644114710 2024-05-21T21:52:08.306 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:08.306+0000 7fa64a000700 1 -- 172.21.15.151:0/1207161529 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa624005330 con 0x7fa644114710 2024-05-21T21:52:08.308 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:08.308+0000 7fa63be00700 1 -- 172.21.15.151:0/1207161529 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fa63c01b030 con 0x7fa644114710 2024-05-21T21:52:08.311 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:08.311+0000 7fa63be00700 1 --2- 172.21.15.151:0/1207161529 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fa6280b2990 0x7fa6280b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:52:08.311 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:08.311+0000 7fa63be00700 1 -- 172.21.15.151:0/1207161529 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fa63c0e1e30 con 0x7fa644114710 2024-05-21T21:52:08.311 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:08.311+0000 7fa643400700 1 --2- 172.21.15.151:0/1207161529 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fa6280b2990 0x7fa6280b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:52:08.312 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:08.312+0000 7fa63be00700 1 -- 172.21.15.151:0/1207161529 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa63c022020 con 0x7fa644114710 2024-05-21T21:52:08.312 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:08.312+0000 7fa643400700 1 --2- 172.21.15.151:0/1207161529 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fa6280b2990 0x7fa6280b4e50 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa634009680 tx=0x7fa634009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:52:08.554 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:08.554+0000 7fa64a000700 1 -- 172.21.15.151:0/1207161529 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa6240058f0 con 0x7fa644114710 2024-05-21T21:52:08.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:08.555+0000 7fa63be00700 1 -- 172.21.15.151:0/1207161529 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22264 (secure 0 0 0) 0x7fa63c010040 con 0x7fa644114710 2024-05-21T21:52:08.556 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T21:52:08.556 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":80,"fsid":"c8c7a2e6-17ba-11ef-bc9a-c7b262605968","created":"2024-05-21T21:41:52.412833+0000","modified":"2024-05-21T21:51:19.552159+0000","last_up_change":"2024-05-21T21:50:05.401314+0000","last_in_change":"2024-05-21T21:49:45.684136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-21T21:45:55.196046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-21T21:51:11.447428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-21T21:51:15.618605+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"18db8e54-0cfc-4e12-9c84-6f72a99cc989","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":2420400473},{"type":"v1","addr":"172.21.15.151:6803","nonce":2420400473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":2420400473},{"type":"v1","addr":"172.21.15.151:6805","nonce":2420400473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":2420400473},{"type":"v1","addr":"172.21.15.151:6809","nonce":2420400473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":2420400473},{"type":"v1","addr":"172.21.15.151:6807","nonce":2420400473}]},"public_addr":"172.21.15.151:6803/2420400473","cluster_addr":"172.21.15.151:6805/2420400473","heartbeat_back_addr":"172.21.15.151:6809/2420400473","heartbeat_front_addr":"172.21.15.151:6807/2420400473","state":["exists","up"]},{"osd":1,"uuid":"bac384c7-1102-4815-81a8-10cc7661d230","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":3900240447},{"type":"v1","addr":"172.21.15.164:6801","nonce":3900240447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":3900240447},{"type":"v1","addr":"172.21.15.164:6803","nonce":3900240447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":3900240447},{"type":"v1","addr":"172.21.15.164:6807","nonce":3900240447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":3900240447},{"type":"v1","addr":"172.21.15.164:6805","nonce":3900240447}]},"public_addr":"172.21.15.164:6801/3900240447","cluster_addr":"172.21.15.164:6803/3900240447","heartbeat_back_addr":"172.21.15.164:6807/3900240447","heartbeat_front_addr":"172.21.15.164:6805/3900240447","state":["exists","up"]},{"osd":2,"uuid":"b7e4a782-1518-4d43-bee9-7fd768ee5cb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":3971150737},{"type":"v1","addr":"172.21.15.184:6801","nonce":3971150737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3971150737},{"type":"v1","addr":"172.21.15.184:6803","nonce":3971150737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3971150737},{"type":"v1","addr":"172.21.15.184:6807","nonce":3971150737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3971150737},{"type":"v1","addr":"172.21.15.184:6805","nonce":3971150737}]},"public_addr":"172.21.15.184:6801/3971150737","cluster_addr":"172.21.15.184:6803/3971150737","heartbeat_back_addr":"172.21.15.184:6807/3971150737","heartbeat_front_addr":"172.21.15.184:6805/3971150737","state":["exists","up"]},{"osd":3,"uuid":"ab3df826-39e3-4635-a49c-2b5ffdbf12f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":556594400},{"type":"v1","addr":"172.21.15.151:6811","nonce":556594400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":556594400},{"type":"v1","addr":"172.21.15.151:6813","nonce":556594400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":556594400},{"type":"v1","addr":"172.21.15.151:6817","nonce":556594400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":556594400},{"type":"v1","addr":"172.21.15.151:6815","nonce":556594400}]},"public_addr":"172.21.15.151:6811/556594400","cluster_addr":"172.21.15.151:6813/556594400","heartbeat_back_addr":"172.21.15.151:6817/556594400","heartbeat_front_addr":"172.21.15.151:6815/556594400","state":["exists","up"]},{"osd":4,"uuid":"00cbf6ae-0f42-4685-a77d-d803c6467aa0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":988881973},{"type":"v1","addr":"172.21.15.164:6809","nonce":988881973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":988881973},{"type":"v1","addr":"172.21.15.164:6811","nonce":988881973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":988881973},{"type":"v1","addr":"172.21.15.164:6815","nonce":988881973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":988881973},{"type":"v1","addr":"172.21.15.164:6813","nonce":988881973}]},"public_addr":"172.21.15.164:6809/988881973","cluster_addr":"172.21.15.164:6811/988881973","heartbeat_back_addr":"172.21.15.164:6815/988881973","heartbeat_front_addr":"172.21.15.164:6813/988881973","state":["exists","up"]},{"osd":5,"uuid":"fad14221-0ef0-4d40-b414-c6c9f1f9718c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3293311106},{"type":"v1","addr":"172.21.15.184:6809","nonce":3293311106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":3293311106},{"type":"v1","addr":"172.21.15.184:6811","nonce":3293311106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":3293311106},{"type":"v1","addr":"172.21.15.184:6815","nonce":3293311106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":3293311106},{"type":"v1","addr":"172.21.15.184:6813","nonce":3293311106}]},"public_addr":"172.21.15.184:6809/3293311106","cluster_addr":"172.21.15.184:6811/3293311106","heartbeat_back_addr":"172.21.15.184:6815/3293311106","heartbeat_front_addr":"172.21.15.184:6813/3293311106","state":["exists","up"]},{"osd":6,"uuid":"ab8a60f6-0dec-4c1c-89db-f2e2f638f1ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":3377550762},{"type":"v1","addr":"172.21.15.151:6819","nonce":3377550762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":3377550762},{"type":"v1","addr":"172.21.15.151:6821","nonce":3377550762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":3377550762},{"type":"v1","addr":"172.21.15.151:6825","nonce":3377550762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":3377550762},{"type":"v1","addr":"172.21.15.151:6823","nonce":3377550762}]},"public_addr":"172.21.15.151:6819/3377550762","cluster_addr":"172.21.15.151:6821/3377550762","heartbeat_back_addr":"172.21.15.151:6825/3377550762","heartbeat_front_addr":"172.21.15.151:6823/3377550762","state":["exists","up"]},{"osd":7,"uuid":"db332268-9972-4ed7-a307-822b83a0fbb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":3869696936},{"type":"v1","addr":"172.21.15.164:6817","nonce":3869696936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":3869696936},{"type":"v1","addr":"172.21.15.164:6819","nonce":3869696936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":3869696936},{"type":"v1","addr":"172.21.15.164:6823","nonce":3869696936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":3869696936},{"type":"v1","addr":"172.21.15.164:6821","nonce":3869696936}]},"public_addr":"172.21.15.164:6817/3869696936","cluster_addr":"172.21.15.164:6819/3869696936","heartbeat_back_addr":"172.21.15.164:6823/3869696936","heartbeat_front_addr":"172.21.15.164:6821/3869696936","state":["exists","up"]},{"osd":8,"uuid":"fe2fbe1c-549a-4947-a188-426dfca04d10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":575273541},{"type":"v1","addr":"172.21.15.184:6817","nonce":575273541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":575273541},{"type":"v1","addr":"172.21.15.184:6819","nonce":575273541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":575273541},{"type":"v1","addr":"172.21.15.184:6823","nonce":575273541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":575273541},{"type":"v1","addr":"172.21.15.184:6821","nonce":575273541}]},"public_addr":"172.21.15.184:6817/575273541","cluster_addr":"172.21.15.184:6819/575273541","heartbeat_back_addr":"172.21.15.184:6823/575273541","heartbeat_front_addr":"172.21.15.184:6821/575273541","state":["exists","up"]},{"osd":9,"uuid":"3a7b450f-8004-4adf-bae3-970918f98f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":337128533},{"type":"v1","addr":"172.21.15.151:6827","nonce":337128533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":337128533},{"type":"v1","addr":"172.21.15.151:6829","nonce":337128533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":337128533},{"type":"v1","addr":"172.21.15.151:6833","nonce":337128533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":337128533},{"type":"v1","addr":"172.21.15.151:6831","nonce":337128533}]},"public_addr":"172.21.15.151:6827/337128533","cluster_addr":"172.21.15.151:6829/337128533","heartbeat_back_addr":"172.21.15.151:6833/337128533","heartbeat_front_addr":"172.21.15.151:6831/337128533","state":["exists","up"]},{"osd":10,"uuid":"8fcdf271-90c4-4db3-be69-b0ca716dfd80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":1823448782},{"type":"v1","addr":"172.21.15.164:6825","nonce":1823448782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":1823448782},{"type":"v1","addr":"172.21.15.164:6827","nonce":1823448782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":1823448782},{"type":"v1","addr":"172.21.15.164:6831","nonce":1823448782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":1823448782},{"type":"v1","addr":"172.21.15.164:6829","nonce":1823448782}]},"public_addr":"172.21.15.164:6825/1823448782","cluster_addr":"172.21.15.164:6827/1823448782","heartbeat_back_addr":"172.21.15.164:6831/1823448782","heartbeat_front_addr":"172.21.15.164:6829/1823448782","state":["exists","up"]},{"osd":11,"uuid":"e661477b-0c09-409f-a2fe-6ebd34642a8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":1563340128},{"type":"v1","addr":"172.21.15.184:6825","nonce":1563340128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":1563340128},{"type":"v1","addr":"172.21.15.184:6827","nonce":1563340128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":1563340128},{"type":"v1","addr":"172.21.15.184:6831","nonce":1563340128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":1563340128},{"type":"v1","addr":"172.21.15.184:6829","nonce":1563340128}]},"public_addr":"172.21.15.184:6825/1563340128","cluster_addr":"172.21.15.184:6827/1563340128","heartbeat_back_addr":"172.21.15.184:6831/1563340128","heartbeat_front_addr":"172.21.15.184:6829/1563340128","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:45:02.821160+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:45:27.844564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:45:51.904981+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:46:24.376245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:46:50.036621+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:47:14.748422+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:47:47.396383+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:48:13.543174+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:48:38.213285+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:49:11.980042+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:49:38.519761+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:50:03.439898+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.151:0/31963128":"2024-05-22T21:43:09.041455+0000","172.21.15.151:6800/3428827300":"2024-05-22T21:43:09.041455+0000","172.21.15.151:6801/3428827300":"2024-05-22T21:43:09.041455+0000","172.21.15.151:0/131989141":"2024-05-22T21:43:09.041455+0000","172.21.15.151:6801/1128885292":"2024-05-22T21:42:35.440147+0000","172.21.15.151:0/2703664519":"2024-05-22T21:43:09.041455+0000","172.21.15.151:6800/1128885292":"2024-05-22T21:42:35.440147+0000","172.21.15.151:0/3052943699":"2024-05-22T21:42:35.440147+0000","172.21.15.151:0/3929452511":"2024-05-22T21:42:35.440147+0000","172.21.15.151:0/3762889700":"2024-05-22T21:42:35.440147+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-21T21:52:08.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:08.559+0000 7fa64a000700 1 -- 172.21.15.151:0/1207161529 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fa6280b2990 msgr2=0x7fa6280b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:52:08.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:08.559+0000 7fa64a000700 1 --2- 172.21.15.151:0/1207161529 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fa6280b2990 0x7fa6280b4e50 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa634009680 tx=0x7fa634009260 comp rx=0 tx=0).stop 2024-05-21T21:52:08.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:08.559+0000 7fa64a000700 1 -- 172.21.15.151:0/1207161529 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa644114710 msgr2=0x7fa6441a4b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:52:08.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:08.559+0000 7fa64a000700 1 --2- 172.21.15.151:0/1207161529 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa644114710 0x7fa6441a4b90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa63c00ef90 tx=0x7fa63c00c3a0 comp rx=0 tx=0).stop 2024-05-21T21:52:08.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:08.559+0000 7fa64a000700 1 -- 172.21.15.151:0/1207161529 shutdown_connections 2024-05-21T21:52:08.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:08.559+0000 7fa64a000700 1 --2- 172.21.15.151:0/1207161529 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fa6280b2990 0x7fa6280b4e50 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:08.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:08.559+0000 7fa64a000700 1 --2- 172.21.15.151:0/1207161529 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa644075770 0x7fa6441a02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:08.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:08.559+0000 7fa64a000700 1 --2- 172.21.15.151:0/1207161529 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa644114710 0x7fa6441a4b90 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:08.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:08.559+0000 7fa64a000700 1 --2- 172.21.15.151:0/1207161529 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa644076090 0x7fa6441a0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:08.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:08.559+0000 7fa64a000700 1 -- 172.21.15.151:0/1207161529 >> 172.21.15.151:0/1207161529 conn(0x7fa64407a6a0 msgr2=0x7fa6441101b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:52:08.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:08.559+0000 7fa64a000700 1 -- 172.21.15.151:0/1207161529 shutdown_connections 2024-05-21T21:52:08.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:08.559+0000 7fa64a000700 1 -- 172.21.15.151:0/1207161529 wait complete. 2024-05-21T21:52:09.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:52:08 smithi151 ceph-mon[31623]: pgmap v339: 97 pgs: 97 active+clean; 638 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 3.5 KiB/s wr, 13 op/s 2024-05-21T21:52:09.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:52:08 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:52:09.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:52:08 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/1207161529' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T21:52:09.235 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:52:08 smithi184 ceph-mon[36942]: pgmap v339: 97 pgs: 97 active+clean; 638 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 3.5 KiB/s wr, 13 op/s 2024-05-21T21:52:09.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:52:08 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:52:09.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:52:08 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/1207161529' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T21:52:09.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:52:08 smithi164 ceph-mon[36269]: pgmap v339: 97 pgs: 97 active+clean; 638 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 3.5 KiB/s wr, 13 op/s 2024-05-21T21:52:09.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:52:08 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:52:09.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:52:08 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/1207161529' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T21:52:09.369 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-21T21:52:09.369 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph pg dump --format=json 2024-05-21T21:52:10.056 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:52:09 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-05-21T21:52:10.056 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:52:09 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2024-05-21T21:52:10.056 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:52:09 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]}]: dispatch 2024-05-21T21:52:10.056 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:52:09 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-05-21T21:52:10.056 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:52:09 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.5", "id": [3, 9]}]: dispatch 2024-05-21T21:52:10.056 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:52:09 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-05-21T21:52:10.056 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:52:09 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]: dispatch 2024-05-21T21:52:10.056 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:52:09 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 8]}]: dispatch 2024-05-21T21:52:10.057 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:52:09 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-05-21T21:52:10.057 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:52:09 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:52:10.057 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:52:09 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:52:10.057 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:52:09 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:52:10.057 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:52:09 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:52:10.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:52:09 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-05-21T21:52:10.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:52:09 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2024-05-21T21:52:10.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:52:09 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]}]: dispatch 2024-05-21T21:52:10.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:52:09 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-05-21T21:52:10.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:52:09 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.5", "id": [3, 9]}]: dispatch 2024-05-21T21:52:10.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:52:09 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-05-21T21:52:10.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:52:09 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]: dispatch 2024-05-21T21:52:10.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:52:09 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 8]}]: dispatch 2024-05-21T21:52:10.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:52:09 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-05-21T21:52:10.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:52:09 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:52:10.237 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:52:09 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:52:10.237 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:52:09 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:52:10.237 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:52:09 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:52:10.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:52:09 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-05-21T21:52:10.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:52:09 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2024-05-21T21:52:10.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:52:09 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]}]: dispatch 2024-05-21T21:52:10.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:52:09 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-05-21T21:52:10.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:52:09 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.5", "id": [3, 9]}]: dispatch 2024-05-21T21:52:10.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:52:09 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-05-21T21:52:10.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:52:09 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]: dispatch 2024-05-21T21:52:10.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:52:09 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 8]}]: dispatch 2024-05-21T21:52:10.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:52:09 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-05-21T21:52:10.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:52:09 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:52:10.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:52:09 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:52:10.287 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:52:09 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:52:10.287 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:52:09 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:52:10.421 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:52:11.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:52:10 smithi151 ceph-mon[31623]: pgmap v340: 97 pgs: 97 active+clean; 638 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.9 KiB/s wr, 11 op/s 2024-05-21T21:52:11.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:52:10 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-05-21T21:52:11.204 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:52:10 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-05-21T21:52:11.204 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:52:10 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]}]': finished 2024-05-21T21:52:11.204 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:52:10 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-05-21T21:52:11.204 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:52:10 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.5", "id": [3, 9]}]': finished 2024-05-21T21:52:11.204 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:52:10 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-05-21T21:52:11.204 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:52:10 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]': finished 2024-05-21T21:52:11.204 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:52:10 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 8]}]': finished 2024-05-21T21:52:11.204 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:52:10 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-05-21T21:52:11.204 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:52:10 smithi151 ceph-mon[31623]: osdmap e81: 12 total, 12 up, 12 in 2024-05-21T21:52:11.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:52:10 smithi184 ceph-mon[36942]: pgmap v340: 97 pgs: 97 active+clean; 638 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.9 KiB/s wr, 11 op/s 2024-05-21T21:52:11.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:52:10 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-05-21T21:52:11.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:52:10 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-05-21T21:52:11.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:52:10 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]}]': finished 2024-05-21T21:52:11.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:52:10 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-05-21T21:52:11.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:52:10 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.5", "id": [3, 9]}]': finished 2024-05-21T21:52:11.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:52:10 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-05-21T21:52:11.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:52:10 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]': finished 2024-05-21T21:52:11.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:52:10 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 8]}]': finished 2024-05-21T21:52:11.237 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:52:10 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-05-21T21:52:11.237 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:52:10 smithi184 ceph-mon[36942]: osdmap e81: 12 total, 12 up, 12 in 2024-05-21T21:52:11.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:52:10 smithi164 ceph-mon[36269]: pgmap v340: 97 pgs: 97 active+clean; 638 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.9 KiB/s wr, 11 op/s 2024-05-21T21:52:11.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:52:10 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-05-21T21:52:11.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:52:10 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-05-21T21:52:11.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:52:10 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]}]': finished 2024-05-21T21:52:11.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:52:10 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-05-21T21:52:11.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:52:10 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.5", "id": [3, 9]}]': finished 2024-05-21T21:52:11.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:52:10 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-05-21T21:52:11.287 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:52:10 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]': finished 2024-05-21T21:52:11.287 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:52:10 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 8]}]': finished 2024-05-21T21:52:11.287 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:52:10 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-05-21T21:52:11.287 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:52:10 smithi164 ceph-mon[36269]: osdmap e81: 12 total, 12 up, 12 in 2024-05-21T21:52:12.132 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:52:11 smithi151 ceph-mon[31623]: osdmap e82: 12 total, 12 up, 12 in 2024-05-21T21:52:12.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:52:11 smithi184 ceph-mon[36942]: osdmap e82: 12 total, 12 up, 12 in 2024-05-21T21:52:12.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:52:11 smithi164 ceph-mon[36269]: osdmap e82: 12 total, 12 up, 12 in 2024-05-21T21:52:12.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:12.390+0000 7f57d4000700 1 -- 172.21.15.151:0/637257287 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f57cc1037f0 msgr2=0x7f57cc103bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:52:12.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:12.390+0000 7f57d4000700 1 --2- 172.21.15.151:0/637257287 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f57cc1037f0 0x7f57cc103bd0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f57c00099c0 tx=0x7f57c0009cd0 comp rx=0 tx=0).stop 2024-05-21T21:52:12.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:12.391+0000 7f57d4000700 1 -- 172.21.15.151:0/637257287 shutdown_connections 2024-05-21T21:52:12.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:12.391+0000 7f57d4000700 1 --2- 172.21.15.151:0/637257287 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f57cc1088f0 0x7f57cc110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:12.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:12.391+0000 7f57d4000700 1 --2- 172.21.15.151:0/637257287 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f57cc1037f0 0x7f57cc103bd0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:12.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:12.391+0000 7f57d4000700 1 --2- 172.21.15.151:0/637257287 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57cc1041a0 0x7f57cc1081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:12.392 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:12.391+0000 7f57d4000700 1 -- 172.21.15.151:0/637257287 >> 172.21.15.151:0/637257287 conn(0x7f57cc07a6a0 msgr2=0x7f57cc07aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:52:12.392 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:12.391+0000 7f57d4000700 1 -- 172.21.15.151:0/637257287 shutdown_connections 2024-05-21T21:52:12.392 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:12.391+0000 7f57d4000700 1 -- 172.21.15.151:0/637257287 wait complete. 2024-05-21T21:52:12.392 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:12.392+0000 7f57d4000700 1 Processor -- start 2024-05-21T21:52:12.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:12.393+0000 7f57d4000700 1 -- start start 2024-05-21T21:52:12.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:12.393+0000 7f57d4000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f57cc1037f0 0x7f57cc19bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:52:12.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:12.393+0000 7f57d4000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57cc1041a0 0x7f57cc19c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:52:12.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:12.393+0000 7f57d4000700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f57cc1088f0 0x7f57cc1a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:52:12.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:12.393+0000 7f57d4000700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f57cc113710 con 0x7f57cc1037f0 2024-05-21T21:52:12.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:12.393+0000 7f57d4000700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f57cc113590 con 0x7f57cc1041a0 2024-05-21T21:52:12.394 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:12.393+0000 7f57d4000700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f57cc113890 con 0x7f57cc1088f0 2024-05-21T21:52:12.394 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:12.393+0000 7f57d1600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f57cc1037f0 0x7f57cc19bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:52:12.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:12.393+0000 7f57d1600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f57cc1037f0 0x7f57cc19bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:54188/0 (socket says 172.21.15.151:54188) 2024-05-21T21:52:12.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:12.394+0000 7f57d1600700 1 -- 172.21.15.151:0/4051745778 learned_addr learned my addr 172.21.15.151:0/4051745778 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:52:12.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:12.394+0000 7f57d2000700 1 --2- 172.21.15.151:0/4051745778 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f57cc1088f0 0x7f57cc1a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:52:12.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:12.394+0000 7f57d0c00700 1 --2- 172.21.15.151:0/4051745778 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57cc1041a0 0x7f57cc19c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:52:12.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:12.394+0000 7f57d2000700 1 -- 172.21.15.151:0/4051745778 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57cc1041a0 msgr2=0x7f57cc19c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:52:12.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:12.394+0000 7f57d2000700 1 --2- 172.21.15.151:0/4051745778 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57cc1041a0 0x7f57cc19c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:12.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:12.394+0000 7f57d2000700 1 -- 172.21.15.151:0/4051745778 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f57cc1037f0 msgr2=0x7f57cc19bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:52:12.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:12.394+0000 7f57d2000700 1 --2- 172.21.15.151:0/4051745778 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f57cc1037f0 0x7f57cc19bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:12.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:12.394+0000 7f57d2000700 1 -- 172.21.15.151:0/4051745778 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57cc1a0db0 con 0x7f57cc1088f0 2024-05-21T21:52:12.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:12.394+0000 7f57d2000700 1 --2- 172.21.15.151:0/4051745778 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f57cc1088f0 0x7f57cc1a0670 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f57c400efb0 tx=0x7f57c400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:52:12.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:12.395+0000 7f57ca000700 1 -- 172.21.15.151:0/4051745778 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c4003a00 con 0x7f57cc1088f0 2024-05-21T21:52:12.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:12.395+0000 7f57ca000700 1 -- 172.21.15.151:0/4051745778 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57c400ce50 con 0x7f57cc1088f0 2024-05-21T21:52:12.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:12.395+0000 7f57d4000700 1 -- 172.21.15.151:0/4051745778 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57cc1a10a0 con 0x7f57cc1088f0 2024-05-21T21:52:12.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:12.395+0000 7f57ca000700 1 -- 172.21.15.151:0/4051745778 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c40047c0 con 0x7f57cc1088f0 2024-05-21T21:52:12.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:12.395+0000 7f57d4000700 1 -- 172.21.15.151:0/4051745778 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57cc1a8920 con 0x7f57cc1088f0 2024-05-21T21:52:12.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:12.396+0000 7f57d4000700 1 -- 172.21.15.151:0/4051745778 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57cc06a230 con 0x7f57cc1088f0 2024-05-21T21:52:12.398 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:12.398+0000 7f57ca000700 1 -- 172.21.15.151:0/4051745778 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f57c4012050 con 0x7f57cc1088f0 2024-05-21T21:52:12.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:12.400+0000 7f57ca000700 1 --2- 172.21.15.151:0/4051745778 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f57b40b2910 0x7f57b40b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:52:12.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:12.400+0000 7f57ca000700 1 -- 172.21.15.151:0/4051745778 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f57c40e2090 con 0x7f57cc1088f0 2024-05-21T21:52:12.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:12.400+0000 7f57d1600700 1 --2- 172.21.15.151:0/4051745778 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f57b40b2910 0x7f57b40b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:52:12.401 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:12.401+0000 7f57d1600700 1 --2- 172.21.15.151:0/4051745778 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f57b40b2910 0x7f57b40b4dd0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f57c0009990 tx=0x7f57c0005ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:52:12.401 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:12.401+0000 7f57ca000700 1 -- 172.21.15.151:0/4051745778 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f57c4022020 con 0x7f57cc1088f0 2024-05-21T21:52:12.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:12.635+0000 7f57d4000700 1 -- 172.21.15.151:0/4051745778 --> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f57cc076780 con 0x7f57b40b2910 2024-05-21T21:52:12.646 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:12.646+0000 7f57ca000700 1 -- 172.21.15.151:0/4051745778 <== mgr.14152 v2:172.21.15.151:6800/2796622483 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258707 (secure 0 0 0) 0x7f57cc076780 con 0x7f57b40b2910 2024-05-21T21:52:12.647 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T21:52:12.654 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:12.653+0000 7f57d4000700 1 -- 172.21.15.151:0/4051745778 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f57b40b2910 msgr2=0x7f57b40b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:52:12.654 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:12.653+0000 7f57d4000700 1 --2- 172.21.15.151:0/4051745778 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f57b40b2910 0x7f57b40b4dd0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f57c0009990 tx=0x7f57c0005ab0 comp rx=0 tx=0).stop 2024-05-21T21:52:12.654 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:12.653+0000 7f57d4000700 1 -- 172.21.15.151:0/4051745778 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f57cc1088f0 msgr2=0x7f57cc1a0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:52:12.654 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:12.653+0000 7f57d4000700 1 --2- 172.21.15.151:0/4051745778 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f57cc1088f0 0x7f57cc1a0670 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f57c400efb0 tx=0x7f57c400c3a0 comp rx=0 tx=0).stop 2024-05-21T21:52:12.654 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:12.653+0000 7f57d4000700 1 -- 172.21.15.151:0/4051745778 shutdown_connections 2024-05-21T21:52:12.654 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:12.653+0000 7f57d4000700 1 --2- 172.21.15.151:0/4051745778 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f57b40b2910 0x7f57b40b4dd0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:12.654 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:12.653+0000 7f57d4000700 1 --2- 172.21.15.151:0/4051745778 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f57cc1088f0 0x7f57cc1a0670 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:12.654 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:12.653+0000 7f57d4000700 1 --2- 172.21.15.151:0/4051745778 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f57cc1037f0 0x7f57cc19bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:12.654 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:12.653+0000 7f57d4000700 1 --2- 172.21.15.151:0/4051745778 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57cc1041a0 0x7f57cc19c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:12.654 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:12.653+0000 7f57d4000700 1 -- 172.21.15.151:0/4051745778 >> 172.21.15.151:0/4051745778 conn(0x7f57cc07a6a0 msgr2=0x7f57cc101920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:52:12.655 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:12.653+0000 7f57d4000700 1 -- 172.21.15.151:0/4051745778 shutdown_connections 2024-05-21T21:52:12.655 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:12.653+0000 7f57d4000700 1 -- 172.21.15.151:0/4051745778 wait complete. 2024-05-21T21:52:12.655 INFO:teuthology.orchestra.run.smithi151.stderr:dumped all 2024-05-21T21:52:12.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:52:12 smithi151 ceph-mon[31623]: pgmap v343: 97 pgs: 2 peering, 95 active+clean; 638 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1023 B/s wr, 10 op/s; 13 B/s, 0 objects/s recovering 2024-05-21T21:52:12.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:52:12 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:52:12.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:52:12 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:52:13.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:52:12 smithi184 ceph-mon[36942]: pgmap v343: 97 pgs: 2 peering, 95 active+clean; 638 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1023 B/s wr, 10 op/s; 13 B/s, 0 objects/s recovering 2024-05-21T21:52:13.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:52:12 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:52:13.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:52:12 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:52:13.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:52:12 smithi164 ceph-mon[36269]: pgmap v343: 97 pgs: 2 peering, 95 active+clean; 638 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1023 B/s wr, 10 op/s; 13 B/s, 0 objects/s recovering 2024-05-21T21:52:13.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:52:12 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:52:13.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:52:12 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:52:13.455 INFO:teuthology.orchestra.run.smithi151.stdout:{"pg_ready":true,"pg_map":{"version":343,"stamp":"2024-05-21T21:52:11.211549+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":653311,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":417,"num_read_kb":304,"num_write":369,"num_write_kb":2282,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":2951952,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":303,"ondisk_log_size":303,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":292,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":365752,"kb_used_data":12424,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124576072,"statfs":{"total":1151940427776,"available":1151565897728,"internally_reserved":0,"allocated":12722176,"data_stored":6998587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19140,"internal_metadata":361674044},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4810,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":71,"num_read_kb":40,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003608"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T21:51:19.948298+0000","last_change":"2024-05-21T21:51:16.945132+0000","last_active":"2024-05-21T21:51:19.948298+0000","last_peered":"2024-05-21T21:51:19.948298+0000","last_clean":"2024-05-21T21:51:19.948298+0000","last_became_active":"2024-05-21T21:51:16.944766+0000","last_became_peered":"2024-05-21T21:51:16.944766+0000","last_unstale":"2024-05-21T21:51:19.948298+0000","last_undegraded":"2024-05-21T21:51:19.948298+0000","last_fullsized":"2024-05-21T21:51:19.948298+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:54:04.185095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T21:51:19.951009+0000","last_change":"2024-05-21T21:51:16.939291+0000","last_active":"2024-05-21T21:51:19.951009+0000","last_peered":"2024-05-21T21:51:19.951009+0000","last_clean":"2024-05-21T21:51:19.951009+0000","last_became_active":"2024-05-21T21:51:16.938730+0000","last_became_peered":"2024-05-21T21:51:16.938730+0000","last_unstale":"2024-05-21T21:51:19.951009+0000","last_undegraded":"2024-05-21T21:51:19.951009+0000","last_fullsized":"2024-05-21T21:51:19.951009+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:05:30.623744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T21:51:19.950520+0000","last_change":"2024-05-21T21:51:16.941565+0000","last_active":"2024-05-21T21:51:19.950520+0000","last_peered":"2024-05-21T21:51:19.950520+0000","last_clean":"2024-05-21T21:51:19.950520+0000","last_became_active":"2024-05-21T21:51:16.941033+0000","last_became_peered":"2024-05-21T21:51:16.941033+0000","last_unstale":"2024-05-21T21:51:19.950520+0000","last_undegraded":"2024-05-21T21:51:19.950520+0000","last_fullsized":"2024-05-21T21:51:19.950520+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:04:55.789479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T21:51:19.943677+0000","last_change":"2024-05-21T21:51:16.928455+0000","last_active":"2024-05-21T21:51:19.943677+0000","last_peered":"2024-05-21T21:51:19.943677+0000","last_clean":"2024-05-21T21:51:19.943677+0000","last_became_active":"2024-05-21T21:51:16.927874+0000","last_became_peered":"2024-05-21T21:51:16.927874+0000","last_unstale":"2024-05-21T21:51:19.943677+0000","last_undegraded":"2024-05-21T21:51:19.943677+0000","last_fullsized":"2024-05-21T21:51:19.943677+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:25:16.483175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.863776+0000","last_change":"2024-05-21T21:51:16.937773+0000","last_active":"2024-05-21T21:52:10.863776+0000","last_peered":"2024-05-21T21:52:10.863776+0000","last_clean":"2024-05-21T21:52:10.863776+0000","last_became_active":"2024-05-21T21:51:16.937306+0000","last_became_peered":"2024-05-21T21:51:16.937306+0000","last_unstale":"2024-05-21T21:52:10.863776+0000","last_undegraded":"2024-05-21T21:52:10.863776+0000","last_fullsized":"2024-05-21T21:52:10.863776+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:05:03.762390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T21:51:19.927500+0000","last_change":"2024-05-21T21:51:16.948275+0000","last_active":"2024-05-21T21:51:19.927500+0000","last_peered":"2024-05-21T21:51:19.927500+0000","last_clean":"2024-05-21T21:51:19.927500+0000","last_became_active":"2024-05-21T21:51:16.947890+0000","last_became_peered":"2024-05-21T21:51:16.947890+0000","last_unstale":"2024-05-21T21:51:19.927500+0000","last_undegraded":"2024-05-21T21:51:19.927500+0000","last_fullsized":"2024-05-21T21:51:19.927500+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:08:38.572177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-21T21:52:09.867065+0000","last_change":"2024-05-21T21:51:16.946821+0000","last_active":"2024-05-21T21:52:09.867065+0000","last_peered":"2024-05-21T21:52:09.867065+0000","last_clean":"2024-05-21T21:52:09.867065+0000","last_became_active":"2024-05-21T21:51:16.946515+0000","last_became_peered":"2024-05-21T21:51:16.946515+0000","last_unstale":"2024-05-21T21:52:09.867065+0000","last_undegraded":"2024-05-21T21:52:09.867065+0000","last_fullsized":"2024-05-21T21:52:09.867065+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T09:26:25.867895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T21:51:19.946888+0000","last_change":"2024-05-21T21:51:16.929228+0000","last_active":"2024-05-21T21:51:19.946888+0000","last_peered":"2024-05-21T21:51:19.946888+0000","last_clean":"2024-05-21T21:51:19.946888+0000","last_became_active":"2024-05-21T21:51:16.928875+0000","last_became_peered":"2024-05-21T21:51:16.928875+0000","last_unstale":"2024-05-21T21:51:19.946888+0000","last_undegraded":"2024-05-21T21:51:19.946888+0000","last_fullsized":"2024-05-21T21:51:19.946888+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:50:56.431310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-21T21:52:09.870499+0000","last_change":"2024-05-21T21:51:16.932360+0000","last_active":"2024-05-21T21:52:09.870499+0000","last_peered":"2024-05-21T21:52:09.870499+0000","last_clean":"2024-05-21T21:52:09.870499+0000","last_became_active":"2024-05-21T21:51:16.932036+0000","last_became_peered":"2024-05-21T21:51:16.932036+0000","last_unstale":"2024-05-21T21:52:09.870499+0000","last_undegraded":"2024-05-21T21:52:09.870499+0000","last_fullsized":"2024-05-21T21:52:09.870499+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T09:06:37.257323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T21:51:19.927918+0000","last_change":"2024-05-21T21:51:16.926057+0000","last_active":"2024-05-21T21:51:19.927918+0000","last_peered":"2024-05-21T21:51:19.927918+0000","last_clean":"2024-05-21T21:51:19.927918+0000","last_became_active":"2024-05-21T21:51:16.925693+0000","last_became_peered":"2024-05-21T21:51:16.925693+0000","last_unstale":"2024-05-21T21:51:19.927918+0000","last_undegraded":"2024-05-21T21:51:19.927918+0000","last_fullsized":"2024-05-21T21:51:19.927918+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T07:31:11.069530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-21T21:52:09.867342+0000","last_change":"2024-05-21T21:51:16.933425+0000","last_active":"2024-05-21T21:52:09.867342+0000","last_peered":"2024-05-21T21:52:09.867342+0000","last_clean":"2024-05-21T21:52:09.867342+0000","last_became_active":"2024-05-21T21:51:16.933057+0000","last_became_peered":"2024-05-21T21:51:16.933057+0000","last_unstale":"2024-05-21T21:52:09.867342+0000","last_undegraded":"2024-05-21T21:52:09.867342+0000","last_fullsized":"2024-05-21T21:52:09.867342+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:04:57.237841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T21:51:19.943540+0000","last_change":"2024-05-21T21:51:16.927014+0000","last_active":"2024-05-21T21:51:19.943540+0000","last_peered":"2024-05-21T21:51:19.943540+0000","last_clean":"2024-05-21T21:51:19.943540+0000","last_became_active":"2024-05-21T21:51:16.925954+0000","last_became_peered":"2024-05-21T21:51:16.925954+0000","last_unstale":"2024-05-21T21:51:19.943540+0000","last_undegraded":"2024-05-21T21:51:19.943540+0000","last_fullsized":"2024-05-21T21:51:19.943540+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:19:20.530727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-21T21:52:09.864058+0000","last_change":"2024-05-21T21:51:16.930909+0000","last_active":"2024-05-21T21:52:09.864058+0000","last_peered":"2024-05-21T21:52:09.864058+0000","last_clean":"2024-05-21T21:52:09.864058+0000","last_became_active":"2024-05-21T21:51:16.927110+0000","last_became_peered":"2024-05-21T21:51:16.927110+0000","last_unstale":"2024-05-21T21:52:09.864058+0000","last_undegraded":"2024-05-21T21:52:09.864058+0000","last_fullsized":"2024-05-21T21:52:09.864058+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T06:23:55.386047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T21:51:19.947843+0000","last_change":"2024-05-21T21:51:16.938797+0000","last_active":"2024-05-21T21:51:19.947843+0000","last_peered":"2024-05-21T21:51:19.947843+0000","last_clean":"2024-05-21T21:51:19.947843+0000","last_became_active":"2024-05-21T21:51:16.938263+0000","last_became_peered":"2024-05-21T21:51:16.938263+0000","last_unstale":"2024-05-21T21:51:19.947843+0000","last_undegraded":"2024-05-21T21:51:19.947843+0000","last_fullsized":"2024-05-21T21:51:19.947843+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:37:40.521118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T21:51:19.948007+0000","last_change":"2024-05-21T21:51:16.944160+0000","last_active":"2024-05-21T21:51:19.948007+0000","last_peered":"2024-05-21T21:51:19.948007+0000","last_clean":"2024-05-21T21:51:19.948007+0000","last_became_active":"2024-05-21T21:51:16.940046+0000","last_became_peered":"2024-05-21T21:51:16.940046+0000","last_unstale":"2024-05-21T21:51:19.948007+0000","last_undegraded":"2024-05-21T21:51:19.948007+0000","last_fullsized":"2024-05-21T21:51:19.948007+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:14:13.278799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-21T21:52:09.870033+0000","last_change":"2024-05-21T21:51:16.945599+0000","last_active":"2024-05-21T21:52:09.870033+0000","last_peered":"2024-05-21T21:52:09.870033+0000","last_clean":"2024-05-21T21:52:09.870033+0000","last_became_active":"2024-05-21T21:51:16.945196+0000","last_became_peered":"2024-05-21T21:51:16.945196+0000","last_unstale":"2024-05-21T21:52:09.870033+0000","last_undegraded":"2024-05-21T21:52:09.870033+0000","last_fullsized":"2024-05-21T21:52:09.870033+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:59:24.453615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T21:51:19.948661+0000","last_change":"2024-05-21T21:51:16.944895+0000","last_active":"2024-05-21T21:51:19.948661+0000","last_peered":"2024-05-21T21:51:19.948661+0000","last_clean":"2024-05-21T21:51:19.948661+0000","last_became_active":"2024-05-21T21:51:16.944560+0000","last_became_peered":"2024-05-21T21:51:16.944560+0000","last_unstale":"2024-05-21T21:51:19.948661+0000","last_undegraded":"2024-05-21T21:51:19.948661+0000","last_fullsized":"2024-05-21T21:51:19.948661+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:53:13.264701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T21:51:19.953849+0000","last_change":"2024-05-21T21:51:16.943861+0000","last_active":"2024-05-21T21:51:19.953849+0000","last_peered":"2024-05-21T21:51:19.953849+0000","last_clean":"2024-05-21T21:51:19.953849+0000","last_became_active":"2024-05-21T21:51:16.943123+0000","last_became_peered":"2024-05-21T21:51:16.943123+0000","last_unstale":"2024-05-21T21:51:19.953849+0000","last_undegraded":"2024-05-21T21:51:19.953849+0000","last_fullsized":"2024-05-21T21:51:19.953849+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:35:37.732855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-21T21:52:09.865357+0000","last_change":"2024-05-21T21:51:16.940580+0000","last_active":"2024-05-21T21:52:09.865357+0000","last_peered":"2024-05-21T21:52:09.865357+0000","last_clean":"2024-05-21T21:52:09.865357+0000","last_became_active":"2024-05-21T21:51:16.940264+0000","last_became_peered":"2024-05-21T21:51:16.940264+0000","last_unstale":"2024-05-21T21:52:09.865357+0000","last_undegraded":"2024-05-21T21:52:09.865357+0000","last_fullsized":"2024-05-21T21:52:09.865357+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:54:22.915541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"80'13","reported_seq":67,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T21:52:08.437619+0000","last_change":"2024-05-21T21:51:12.852935+0000","last_active":"2024-05-21T21:52:08.437619+0000","last_peered":"2024-05-21T21:52:08.437619+0000","last_clean":"2024-05-21T21:52:08.437619+0000","last_became_active":"2024-05-21T21:51:12.851802+0000","last_became_peered":"2024-05-21T21:51:12.851802+0000","last_unstale":"2024-05-21T21:52:08.437619+0000","last_undegraded":"2024-05-21T21:52:08.437619+0000","last_fullsized":"2024-05-21T21:52:08.437619+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:13:24.643408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":32,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T21:51:19.949744+0000","last_change":"2024-05-21T21:51:16.939461+0000","last_active":"2024-05-21T21:51:19.949744+0000","last_peered":"2024-05-21T21:51:19.949744+0000","last_clean":"2024-05-21T21:51:19.949744+0000","last_became_active":"2024-05-21T21:51:16.936322+0000","last_became_peered":"2024-05-21T21:51:16.936322+0000","last_unstale":"2024-05-21T21:51:19.949744+0000","last_undegraded":"2024-05-21T21:51:19.949744+0000","last_fullsized":"2024-05-21T21:51:19.949744+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:37:26.153963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T21:51:19.953470+0000","last_change":"2024-05-21T21:51:12.941838+0000","last_active":"2024-05-21T21:51:19.953470+0000","last_peered":"2024-05-21T21:51:19.953470+0000","last_clean":"2024-05-21T21:51:19.953470+0000","last_became_active":"2024-05-21T21:51:12.940818+0000","last_became_peered":"2024-05-21T21:51:12.940818+0000","last_unstale":"2024-05-21T21:51:19.953470+0000","last_undegraded":"2024-05-21T21:51:19.953470+0000","last_fullsized":"2024-05-21T21:51:19.953470+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:30:42.349241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T21:51:19.951097+0000","last_change":"2024-05-21T21:51:16.950252+0000","last_active":"2024-05-21T21:51:19.951097+0000","last_peered":"2024-05-21T21:51:19.951097+0000","last_clean":"2024-05-21T21:51:19.951097+0000","last_became_active":"2024-05-21T21:51:16.947375+0000","last_became_peered":"2024-05-21T21:51:16.947375+0000","last_unstale":"2024-05-21T21:51:19.951097+0000","last_undegraded":"2024-05-21T21:51:19.951097+0000","last_fullsized":"2024-05-21T21:51:19.951097+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:27:30.462799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"80'12","reported_seq":35,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T21:51:39.364569+0000","last_change":"2024-05-21T21:51:12.850707+0000","last_active":"2024-05-21T21:51:39.364569+0000","last_peered":"2024-05-21T21:51:39.364569+0000","last_clean":"2024-05-21T21:51:39.364569+0000","last_became_active":"2024-05-21T21:51:12.850330+0000","last_became_peered":"2024-05-21T21:51:12.850330+0000","last_unstale":"2024-05-21T21:51:39.364569+0000","last_undegraded":"2024-05-21T21:51:39.364569+0000","last_fullsized":"2024-05-21T21:51:39.364569+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:21:40.942128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-21T21:52:09.870192+0000","last_change":"2024-05-21T21:51:16.926380+0000","last_active":"2024-05-21T21:52:09.870192+0000","last_peered":"2024-05-21T21:52:09.870192+0000","last_clean":"2024-05-21T21:52:09.870192+0000","last_became_active":"2024-05-21T21:51:16.925821+0000","last_became_peered":"2024-05-21T21:51:16.925821+0000","last_unstale":"2024-05-21T21:52:09.870192+0000","last_undegraded":"2024-05-21T21:52:09.870192+0000","last_fullsized":"2024-05-21T21:52:09.870192+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:42:14.887155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'5","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.864272+0000","last_change":"2024-05-21T21:51:12.854342+0000","last_active":"2024-05-21T21:52:10.864272+0000","last_peered":"2024-05-21T21:52:10.864272+0000","last_clean":"2024-05-21T21:52:10.864272+0000","last_became_active":"2024-05-21T21:51:12.853908+0000","last_became_peered":"2024-05-21T21:51:12.853908+0000","last_unstale":"2024-05-21T21:52:10.864272+0000","last_undegraded":"2024-05-21T21:52:10.864272+0000","last_fullsized":"2024-05-21T21:52:10.864272+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:35:10.374749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T21:51:19.928904+0000","last_change":"2024-05-21T21:51:16.934891+0000","last_active":"2024-05-21T21:51:19.928904+0000","last_peered":"2024-05-21T21:51:19.928904+0000","last_clean":"2024-05-21T21:51:19.928904+0000","last_became_active":"2024-05-21T21:51:16.934527+0000","last_became_peered":"2024-05-21T21:51:16.934527+0000","last_unstale":"2024-05-21T21:51:19.928904+0000","last_undegraded":"2024-05-21T21:51:19.928904+0000","last_fullsized":"2024-05-21T21:51:19.928904+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T06:42:00.244657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"80'6","reported_seq":60,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T21:52:06.424545+0000","last_change":"2024-05-21T21:51:12.855658+0000","last_active":"2024-05-21T21:52:06.424545+0000","last_peered":"2024-05-21T21:52:06.424545+0000","last_clean":"2024-05-21T21:52:06.424545+0000","last_became_active":"2024-05-21T21:51:12.855281+0000","last_became_peered":"2024-05-21T21:51:12.855281+0000","last_unstale":"2024-05-21T21:52:06.424545+0000","last_undegraded":"2024-05-21T21:52:06.424545+0000","last_fullsized":"2024-05-21T21:52:06.424545+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:55:08.836045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":32,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.863544+0000","last_change":"2024-05-21T21:51:16.946977+0000","last_active":"2024-05-21T21:52:10.863544+0000","last_peered":"2024-05-21T21:52:10.863544+0000","last_clean":"2024-05-21T21:52:10.863544+0000","last_became_active":"2024-05-21T21:51:16.946562+0000","last_became_peered":"2024-05-21T21:51:16.946562+0000","last_unstale":"2024-05-21T21:52:10.863544+0000","last_undegraded":"2024-05-21T21:52:10.863544+0000","last_fullsized":"2024-05-21T21:52:10.863544+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:43:33.370265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T21:51:37.335446+0000","last_change":"2024-05-21T21:51:12.852574+0000","last_active":"2024-05-21T21:51:37.335446+0000","last_peered":"2024-05-21T21:51:37.335446+0000","last_clean":"2024-05-21T21:51:37.335446+0000","last_became_active":"2024-05-21T21:51:12.852258+0000","last_became_peered":"2024-05-21T21:51:12.852258+0000","last_unstale":"2024-05-21T21:51:37.335446+0000","last_undegraded":"2024-05-21T21:51:37.335446+0000","last_fullsized":"2024-05-21T21:51:37.335446+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:09:40.280821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T21:51:19.928624+0000","last_change":"2024-05-21T21:51:16.934768+0000","last_active":"2024-05-21T21:51:19.928624+0000","last_peered":"2024-05-21T21:51:19.928624+0000","last_clean":"2024-05-21T21:51:19.928624+0000","last_became_active":"2024-05-21T21:51:16.934275+0000","last_became_peered":"2024-05-21T21:51:16.934275+0000","last_unstale":"2024-05-21T21:51:19.928624+0000","last_undegraded":"2024-05-21T21:51:19.928624+0000","last_fullsized":"2024-05-21T21:51:19.928624+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:31:54.425069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"80'3","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-21T21:52:09.864130+0000","last_change":"2024-05-21T21:51:12.854166+0000","last_active":"2024-05-21T21:52:09.864130+0000","last_peered":"2024-05-21T21:52:09.864130+0000","last_clean":"2024-05-21T21:52:09.864130+0000","last_became_active":"2024-05-21T21:51:12.853508+0000","last_became_peered":"2024-05-21T21:51:12.853508+0000","last_unstale":"2024-05-21T21:52:09.864130+0000","last_undegraded":"2024-05-21T21:52:09.864130+0000","last_fullsized":"2024-05-21T21:52:09.864130+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:29:41.858655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T21:51:19.946690+0000","last_change":"2024-05-21T21:51:16.938293+0000","last_active":"2024-05-21T21:51:19.946690+0000","last_peered":"2024-05-21T21:51:19.946690+0000","last_clean":"2024-05-21T21:51:19.946690+0000","last_became_active":"2024-05-21T21:51:16.937970+0000","last_became_peered":"2024-05-21T21:51:16.937970+0000","last_unstale":"2024-05-21T21:51:19.946690+0000","last_undegraded":"2024-05-21T21:51:19.946690+0000","last_fullsized":"2024-05-21T21:51:19.946690+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:18:27.473098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'3","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T21:51:38.342136+0000","last_change":"2024-05-21T21:51:12.853366+0000","last_active":"2024-05-21T21:51:38.342136+0000","last_peered":"2024-05-21T21:51:38.342136+0000","last_clean":"2024-05-21T21:51:38.342136+0000","last_became_active":"2024-05-21T21:51:12.853028+0000","last_became_peered":"2024-05-21T21:51:12.853028+0000","last_unstale":"2024-05-21T21:51:38.342136+0000","last_undegraded":"2024-05-21T21:51:38.342136+0000","last_fullsized":"2024-05-21T21:51:38.342136+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T09:15:52.593028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T21:51:19.953083+0000","last_change":"2024-05-21T21:51:16.947903+0000","last_active":"2024-05-21T21:51:19.953083+0000","last_peered":"2024-05-21T21:51:19.953083+0000","last_clean":"2024-05-21T21:51:19.953083+0000","last_became_active":"2024-05-21T21:51:16.946677+0000","last_became_peered":"2024-05-21T21:51:16.946677+0000","last_unstale":"2024-05-21T21:51:19.953083+0000","last_undegraded":"2024-05-21T21:51:19.953083+0000","last_fullsized":"2024-05-21T21:51:19.953083+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T06:12:56.514284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'4","reported_seq":82,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T21:52:06.411004+0000","last_change":"2024-05-21T21:51:12.854873+0000","last_active":"2024-05-21T21:52:06.411004+0000","last_peered":"2024-05-21T21:52:06.411004+0000","last_clean":"2024-05-21T21:52:06.411004+0000","last_became_active":"2024-05-21T21:51:12.854393+0000","last_became_peered":"2024-05-21T21:51:12.854393+0000","last_unstale":"2024-05-21T21:52:06.411004+0000","last_undegraded":"2024-05-21T21:52:06.411004+0000","last_fullsized":"2024-05-21T21:52:06.411004+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:58:52.411164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":56,"num_read_kb":56,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-21T21:52:09.870477+0000","last_change":"2024-05-21T21:51:16.951094+0000","last_active":"2024-05-21T21:52:09.870477+0000","last_peered":"2024-05-21T21:52:09.870477+0000","last_clean":"2024-05-21T21:52:09.870477+0000","last_became_active":"2024-05-21T21:51:16.950724+0000","last_became_peered":"2024-05-21T21:51:16.950724+0000","last_unstale":"2024-05-21T21:52:09.870477+0000","last_undegraded":"2024-05-21T21:52:09.870477+0000","last_fullsized":"2024-05-21T21:52:09.870477+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:54:02.835426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'4","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-21T21:52:09.867386+0000","last_change":"2024-05-21T21:51:12.850427+0000","last_active":"2024-05-21T21:52:09.867386+0000","last_peered":"2024-05-21T21:52:09.867386+0000","last_clean":"2024-05-21T21:52:09.867386+0000","last_became_active":"2024-05-21T21:51:12.850096+0000","last_became_peered":"2024-05-21T21:51:12.850096+0000","last_unstale":"2024-05-21T21:52:09.867386+0000","last_undegraded":"2024-05-21T21:52:09.867386+0000","last_fullsized":"2024-05-21T21:52:09.867386+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:43:22.611043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T21:51:19.951120+0000","last_change":"2024-05-21T21:51:16.942330+0000","last_active":"2024-05-21T21:51:19.951120+0000","last_peered":"2024-05-21T21:51:19.951120+0000","last_clean":"2024-05-21T21:51:19.951120+0000","last_became_active":"2024-05-21T21:51:16.941718+0000","last_became_peered":"2024-05-21T21:51:16.941718+0000","last_unstale":"2024-05-21T21:51:19.951120+0000","last_undegraded":"2024-05-21T21:51:19.951120+0000","last_fullsized":"2024-05-21T21:51:19.951120+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:54:17.176411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'5","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T21:51:37.336487+0000","last_change":"2024-05-21T21:51:12.941100+0000","last_active":"2024-05-21T21:51:37.336487+0000","last_peered":"2024-05-21T21:51:37.336487+0000","last_clean":"2024-05-21T21:51:37.336487+0000","last_became_active":"2024-05-21T21:51:12.940597+0000","last_became_peered":"2024-05-21T21:51:12.940597+0000","last_unstale":"2024-05-21T21:51:37.336487+0000","last_undegraded":"2024-05-21T21:51:37.336487+0000","last_fullsized":"2024-05-21T21:51:37.336487+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:30:58.457108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-21T21:52:09.870315+0000","last_change":"2024-05-21T21:51:16.921973+0000","last_active":"2024-05-21T21:52:09.870315+0000","last_peered":"2024-05-21T21:52:09.870315+0000","last_clean":"2024-05-21T21:52:09.870315+0000","last_became_active":"2024-05-21T21:51:16.921597+0000","last_became_peered":"2024-05-21T21:51:16.921597+0000","last_unstale":"2024-05-21T21:52:09.870315+0000","last_undegraded":"2024-05-21T21:52:09.870315+0000","last_fullsized":"2024-05-21T21:52:09.870315+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:25:32.395940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'30","reported_seq":116,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T21:52:06.411074+0000","last_change":"2024-05-21T21:51:12.850312+0000","last_active":"2024-05-21T21:52:06.411074+0000","last_peered":"2024-05-21T21:52:06.411074+0000","last_clean":"2024-05-21T21:52:06.411074+0000","last_became_active":"2024-05-21T21:51:12.849837+0000","last_became_peered":"2024-05-21T21:51:12.849837+0000","last_unstale":"2024-05-21T21:52:06.411074+0000","last_undegraded":"2024-05-21T21:52:06.411074+0000","last_fullsized":"2024-05-21T21:52:06.411074+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T07:09:13.339554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":55505,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":64,"num_read_kb":85,"num_write":32,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.863568+0000","last_change":"2024-05-21T21:51:16.926625+0000","last_active":"2024-05-21T21:52:10.863568+0000","last_peered":"2024-05-21T21:52:10.863568+0000","last_clean":"2024-05-21T21:52:10.863568+0000","last_became_active":"2024-05-21T21:51:16.925889+0000","last_became_peered":"2024-05-21T21:51:16.925889+0000","last_unstale":"2024-05-21T21:52:10.863568+0000","last_undegraded":"2024-05-21T21:52:10.863568+0000","last_fullsized":"2024-05-21T21:52:10.863568+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:35:23.464747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'9","reported_seq":35,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-21T21:52:09.865277+0000","last_change":"2024-05-21T21:51:12.859772+0000","last_active":"2024-05-21T21:52:09.865277+0000","last_peered":"2024-05-21T21:52:09.865277+0000","last_clean":"2024-05-21T21:52:09.865277+0000","last_became_active":"2024-05-21T21:51:12.859290+0000","last_became_peered":"2024-05-21T21:51:12.859290+0000","last_unstale":"2024-05-21T21:52:09.865277+0000","last_undegraded":"2024-05-21T21:52:09.865277+0000","last_fullsized":"2024-05-21T21:52:09.865277+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T05:28:29.953381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-21T21:52:09.863631+0000","last_change":"2024-05-21T21:51:16.925803+0000","last_active":"2024-05-21T21:52:09.863631+0000","last_peered":"2024-05-21T21:52:09.863631+0000","last_clean":"2024-05-21T21:52:09.863631+0000","last_became_active":"2024-05-21T21:51:16.925111+0000","last_became_peered":"2024-05-21T21:51:16.925111+0000","last_unstale":"2024-05-21T21:52:09.863631+0000","last_undegraded":"2024-05-21T21:52:09.863631+0000","last_fullsized":"2024-05-21T21:52:09.863631+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T06:22:11.353379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'5","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T21:51:38.343627+0000","last_change":"2024-05-21T21:51:12.859593+0000","last_active":"2024-05-21T21:51:38.343627+0000","last_peered":"2024-05-21T21:51:38.343627+0000","last_clean":"2024-05-21T21:51:38.343627+0000","last_became_active":"2024-05-21T21:51:12.859197+0000","last_became_peered":"2024-05-21T21:51:12.859197+0000","last_unstale":"2024-05-21T21:51:38.343627+0000","last_undegraded":"2024-05-21T21:51:38.343627+0000","last_fullsized":"2024-05-21T21:51:38.343627+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:13:12.311496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"peering","last_fresh":"2024-05-21T21:51:20.571325+0000","last_change":"2024-05-21T21:52:09.867385+0000","last_active":"2024-05-21T21:52:09.867385+0000","last_peered":"2024-05-21T21:51:20.571325+0000","last_clean":"2024-05-21T21:51:20.571325+0000","last_became_active":"2024-05-21T21:51:16.931709+0000","last_became_peered":"2024-05-21T21:51:16.931709+0000","last_unstale":"2024-05-21T21:51:20.571325+0000","last_undegraded":"2024-05-21T21:51:20.571325+0000","last_fullsized":"2024-05-21T21:51:20.571325+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,6,9],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'6","reported_seq":60,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T21:52:06.419006+0000","last_change":"2024-05-21T21:51:12.852169+0000","last_active":"2024-05-21T21:52:06.419006+0000","last_peered":"2024-05-21T21:52:06.419006+0000","last_clean":"2024-05-21T21:52:06.419006+0000","last_became_active":"2024-05-21T21:51:12.851782+0000","last_became_peered":"2024-05-21T21:51:12.851782+0000","last_unstale":"2024-05-21T21:52:06.419006+0000","last_undegraded":"2024-05-21T21:52:06.419006+0000","last_fullsized":"2024-05-21T21:52:06.419006+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:46:09.692158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":33,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-21T21:52:09.866991+0000","last_change":"2024-05-21T21:51:16.929264+0000","last_active":"2024-05-21T21:52:09.866991+0000","last_peered":"2024-05-21T21:52:09.866991+0000","last_clean":"2024-05-21T21:52:09.866991+0000","last_became_active":"2024-05-21T21:51:16.928900+0000","last_became_peered":"2024-05-21T21:51:16.928900+0000","last_unstale":"2024-05-21T21:52:09.866991+0000","last_undegraded":"2024-05-21T21:52:09.866991+0000","last_fullsized":"2024-05-21T21:52:09.866991+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:42:28.040939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'17","reported_seq":68,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.984451+0000","last_change":"2024-05-21T21:52:10.984451+0000","last_active":"2024-05-21T21:52:10.984451+0000","last_peered":"2024-05-21T21:52:10.984451+0000","last_clean":"2024-05-21T21:52:10.984451+0000","last_became_active":"2024-05-21T21:52:10.974050+0000","last_became_peered":"2024-05-21T21:52:10.974050+0000","last_unstale":"2024-05-21T21:52:10.984451+0000","last_undegraded":"2024-05-21T21:52:10.984451+0000","last_fullsized":"2024-05-21T21:52:10.984451+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T07:08:18.307022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T21:51:19.928744+0000","last_change":"2024-05-21T21:51:16.935464+0000","last_active":"2024-05-21T21:51:19.928744+0000","last_peered":"2024-05-21T21:51:19.928744+0000","last_clean":"2024-05-21T21:51:19.928744+0000","last_became_active":"2024-05-21T21:51:16.935091+0000","last_became_peered":"2024-05-21T21:51:16.935091+0000","last_unstale":"2024-05-21T21:51:19.928744+0000","last_undegraded":"2024-05-21T21:51:19.928744+0000","last_fullsized":"2024-05-21T21:51:19.928744+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T06:59:16.330649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.864530+0000","last_change":"2024-05-21T21:51:12.853327+0000","last_active":"2024-05-21T21:52:10.864530+0000","last_peered":"2024-05-21T21:52:10.864530+0000","last_clean":"2024-05-21T21:52:10.864530+0000","last_became_active":"2024-05-21T21:51:12.852946+0000","last_became_peered":"2024-05-21T21:51:12.852946+0000","last_unstale":"2024-05-21T21:52:10.864530+0000","last_undegraded":"2024-05-21T21:52:10.864530+0000","last_fullsized":"2024-05-21T21:52:10.864530+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:00:34.256858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T21:51:19.947439+0000","last_change":"2024-05-21T21:51:16.929341+0000","last_active":"2024-05-21T21:51:19.947439+0000","last_peered":"2024-05-21T21:51:19.947439+0000","last_clean":"2024-05-21T21:51:19.947439+0000","last_became_active":"2024-05-21T21:51:16.929008+0000","last_became_peered":"2024-05-21T21:51:16.929008+0000","last_unstale":"2024-05-21T21:51:19.947439+0000","last_undegraded":"2024-05-21T21:51:19.947439+0000","last_fullsized":"2024-05-21T21:51:19.947439+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:26:26.297136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"80'1","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T21:51:19.953880+0000","last_change":"2024-05-21T21:51:12.945433+0000","last_active":"2024-05-21T21:51:19.953880+0000","last_peered":"2024-05-21T21:51:19.953880+0000","last_clean":"2024-05-21T21:51:19.953880+0000","last_became_active":"2024-05-21T21:51:12.944463+0000","last_became_peered":"2024-05-21T21:51:12.944463+0000","last_unstale":"2024-05-21T21:51:19.953880+0000","last_undegraded":"2024-05-21T21:51:19.953880+0000","last_fullsized":"2024-05-21T21:51:19.953880+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:29:52.014583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'1","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T21:51:19.927880+0000","last_change":"2024-05-21T21:51:16.937066+0000","last_active":"2024-05-21T21:51:19.927880+0000","last_peered":"2024-05-21T21:51:19.927880+0000","last_clean":"2024-05-21T21:51:19.927880+0000","last_became_active":"2024-05-21T21:51:16.936708+0000","last_became_peered":"2024-05-21T21:51:16.936708+0000","last_unstale":"2024-05-21T21:51:19.927880+0000","last_undegraded":"2024-05-21T21:51:19.927880+0000","last_fullsized":"2024-05-21T21:51:19.927880+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T07:36:59.538351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"80'5","reported_seq":62,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-21T21:52:09.867234+0000","last_change":"2024-05-21T21:51:12.852883+0000","last_active":"2024-05-21T21:52:09.867234+0000","last_peered":"2024-05-21T21:52:09.867234+0000","last_clean":"2024-05-21T21:52:09.867234+0000","last_became_active":"2024-05-21T21:51:12.852549+0000","last_became_peered":"2024-05-21T21:51:12.852549+0000","last_unstale":"2024-05-21T21:52:09.867234+0000","last_undegraded":"2024-05-21T21:52:09.867234+0000","last_fullsized":"2024-05-21T21:52:09.867234+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:05:53.002286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":33,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T21:51:19.927495+0000","last_change":"2024-05-21T21:51:16.944425+0000","last_active":"2024-05-21T21:51:19.927495+0000","last_peered":"2024-05-21T21:51:19.927495+0000","last_clean":"2024-05-21T21:51:19.927495+0000","last_became_active":"2024-05-21T21:51:16.944033+0000","last_became_peered":"2024-05-21T21:51:16.944033+0000","last_unstale":"2024-05-21T21:51:19.927495+0000","last_undegraded":"2024-05-21T21:51:19.927495+0000","last_fullsized":"2024-05-21T21:51:19.927495+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T07:16:26.802329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T21:51:19.928626+0000","last_change":"2024-05-21T21:51:16.933264+0000","last_active":"2024-05-21T21:51:19.928626+0000","last_peered":"2024-05-21T21:51:19.928626+0000","last_clean":"2024-05-21T21:51:19.928626+0000","last_became_active":"2024-05-21T21:51:16.932911+0000","last_became_peered":"2024-05-21T21:51:16.932911+0000","last_unstale":"2024-05-21T21:51:19.928626+0000","last_undegraded":"2024-05-21T21:51:19.928626+0000","last_fullsized":"2024-05-21T21:51:19.928626+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:25:54.006367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"80'5","reported_seq":61,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-21T21:52:09.863719+0000","last_change":"2024-05-21T21:51:12.942900+0000","last_active":"2024-05-21T21:52:09.863719+0000","last_peered":"2024-05-21T21:52:09.863719+0000","last_clean":"2024-05-21T21:52:09.863719+0000","last_became_active":"2024-05-21T21:51:12.941038+0000","last_became_peered":"2024-05-21T21:51:12.941038+0000","last_unstale":"2024-05-21T21:52:09.863719+0000","last_undegraded":"2024-05-21T21:52:09.863719+0000","last_fullsized":"2024-05-21T21:52:09.863719+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:38:43.240011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":32,"num_read_kb":32,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T21:51:19.948861+0000","last_change":"2024-05-21T21:51:16.933391+0000","last_active":"2024-05-21T21:51:19.948861+0000","last_peered":"2024-05-21T21:51:19.948861+0000","last_clean":"2024-05-21T21:51:19.948861+0000","last_became_active":"2024-05-21T21:51:16.932320+0000","last_became_peered":"2024-05-21T21:51:16.932320+0000","last_unstale":"2024-05-21T21:51:19.948861+0000","last_undegraded":"2024-05-21T21:51:19.948861+0000","last_fullsized":"2024-05-21T21:51:19.948861+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T09:06:59.126748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'6","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T21:51:39.366139+0000","last_change":"2024-05-21T21:51:12.856342+0000","last_active":"2024-05-21T21:51:39.366139+0000","last_peered":"2024-05-21T21:51:39.366139+0000","last_clean":"2024-05-21T21:51:39.366139+0000","last_became_active":"2024-05-21T21:51:12.855952+0000","last_became_peered":"2024-05-21T21:51:12.855952+0000","last_unstale":"2024-05-21T21:51:39.366139+0000","last_undegraded":"2024-05-21T21:51:39.366139+0000","last_fullsized":"2024-05-21T21:51:39.366139+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:34:40.734034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":97,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-21T21:52:09.867081+0000","last_change":"2024-05-21T21:49:42.797187+0000","last_active":"2024-05-21T21:52:09.867081+0000","last_peered":"2024-05-21T21:52:09.867081+0000","last_clean":"2024-05-21T21:52:09.867081+0000","last_became_active":"2024-05-21T21:49:42.791291+0000","last_became_peered":"2024-05-21T21:49:42.791291+0000","last_unstale":"2024-05-21T21:52:09.867081+0000","last_undegraded":"2024-05-21T21:52:09.867081+0000","last_fullsized":"2024-05-21T21:52:09.867081+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:45:55.552293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:45:55.552293+0000","last_clean_scrub_stamp":"2024-05-21T21:45:55.552293+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T07:24:18.302523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-21T21:52:09.863401+0000","last_change":"2024-05-21T21:51:16.926446+0000","last_active":"2024-05-21T21:52:09.863401+0000","last_peered":"2024-05-21T21:52:09.863401+0000","last_clean":"2024-05-21T21:52:09.863401+0000","last_became_active":"2024-05-21T21:51:16.926070+0000","last_became_peered":"2024-05-21T21:51:16.926070+0000","last_unstale":"2024-05-21T21:52:09.863401+0000","last_undegraded":"2024-05-21T21:52:09.863401+0000","last_fullsized":"2024-05-21T21:52:09.863401+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:44:27.366393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'4","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-21T21:52:09.867193+0000","last_change":"2024-05-21T21:51:12.854371+0000","last_active":"2024-05-21T21:52:09.867193+0000","last_peered":"2024-05-21T21:52:09.867193+0000","last_clean":"2024-05-21T21:52:09.867193+0000","last_became_active":"2024-05-21T21:51:12.853847+0000","last_became_peered":"2024-05-21T21:51:12.853847+0000","last_unstale":"2024-05-21T21:52:09.867193+0000","last_undegraded":"2024-05-21T21:52:09.867193+0000","last_fullsized":"2024-05-21T21:52:09.867193+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:14:10.549901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"80'4","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.863685+0000","last_change":"2024-05-21T21:51:12.941227+0000","last_active":"2024-05-21T21:52:10.863685+0000","last_peered":"2024-05-21T21:52:10.863685+0000","last_clean":"2024-05-21T21:52:10.863685+0000","last_became_active":"2024-05-21T21:51:12.940702+0000","last_became_peered":"2024-05-21T21:51:12.940702+0000","last_unstale":"2024-05-21T21:52:10.863685+0000","last_undegraded":"2024-05-21T21:52:10.863685+0000","last_fullsized":"2024-05-21T21:52:10.863685+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:40:34.124921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-21T21:52:09.864369+0000","last_change":"2024-05-21T21:51:16.945627+0000","last_active":"2024-05-21T21:52:09.864369+0000","last_peered":"2024-05-21T21:52:09.864369+0000","last_clean":"2024-05-21T21:52:09.864369+0000","last_became_active":"2024-05-21T21:51:16.945297+0000","last_became_peered":"2024-05-21T21:51:16.945297+0000","last_unstale":"2024-05-21T21:52:09.864369+0000","last_undegraded":"2024-05-21T21:52:09.864369+0000","last_fullsized":"2024-05-21T21:52:09.864369+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:54:38.898774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'15","reported_seq":37,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T21:52:02.949182+0000","last_change":"2024-05-21T21:51:12.860580+0000","last_active":"2024-05-21T21:52:02.949182+0000","last_peered":"2024-05-21T21:52:02.949182+0000","last_clean":"2024-05-21T21:52:02.949182+0000","last_became_active":"2024-05-21T21:51:12.860010+0000","last_became_peered":"2024-05-21T21:51:12.860010+0000","last_unstale":"2024-05-21T21:52:02.949182+0000","last_undegraded":"2024-05-21T21:52:02.949182+0000","last_fullsized":"2024-05-21T21:52:02.949182+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:12:13.916560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T21:51:19.950016+0000","last_change":"2024-05-21T21:51:16.929807+0000","last_active":"2024-05-21T21:51:19.950016+0000","last_peered":"2024-05-21T21:51:19.950016+0000","last_clean":"2024-05-21T21:51:19.950016+0000","last_became_active":"2024-05-21T21:51:16.928859+0000","last_became_peered":"2024-05-21T21:51:16.928859+0000","last_unstale":"2024-05-21T21:51:19.950016+0000","last_undegraded":"2024-05-21T21:51:19.950016+0000","last_fullsized":"2024-05-21T21:51:19.950016+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:41:32.830892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'7","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T21:51:38.347610+0000","last_change":"2024-05-21T21:51:12.850843+0000","last_active":"2024-05-21T21:51:38.347610+0000","last_peered":"2024-05-21T21:51:38.347610+0000","last_clean":"2024-05-21T21:51:38.347610+0000","last_became_active":"2024-05-21T21:51:12.850516+0000","last_became_peered":"2024-05-21T21:51:12.850516+0000","last_unstale":"2024-05-21T21:51:38.347610+0000","last_undegraded":"2024-05-21T21:51:38.347610+0000","last_fullsized":"2024-05-21T21:51:38.347610+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:37:05.191626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T21:51:19.951302+0000","last_change":"2024-05-21T21:51:16.949017+0000","last_active":"2024-05-21T21:51:19.951302+0000","last_peered":"2024-05-21T21:51:19.951302+0000","last_clean":"2024-05-21T21:51:19.951302+0000","last_became_active":"2024-05-21T21:51:16.948691+0000","last_became_peered":"2024-05-21T21:51:16.948691+0000","last_unstale":"2024-05-21T21:51:19.951302+0000","last_undegraded":"2024-05-21T21:51:19.951302+0000","last_fullsized":"2024-05-21T21:51:19.951302+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:32:06.606581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-21T21:52:09.870043+0000","last_change":"2024-05-21T21:51:12.939203+0000","last_active":"2024-05-21T21:52:09.870043+0000","last_peered":"2024-05-21T21:52:09.870043+0000","last_clean":"2024-05-21T21:52:09.870043+0000","last_became_active":"2024-05-21T21:51:12.938936+0000","last_became_peered":"2024-05-21T21:51:12.938936+0000","last_unstale":"2024-05-21T21:52:09.870043+0000","last_undegraded":"2024-05-21T21:52:09.870043+0000","last_fullsized":"2024-05-21T21:52:09.870043+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:55:50.336362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.863329+0000","last_change":"2024-05-21T21:51:16.926216+0000","last_active":"2024-05-21T21:52:10.863329+0000","last_peered":"2024-05-21T21:52:10.863329+0000","last_clean":"2024-05-21T21:52:10.863329+0000","last_became_active":"2024-05-21T21:51:16.925834+0000","last_became_peered":"2024-05-21T21:51:16.925834+0000","last_unstale":"2024-05-21T21:52:10.863329+0000","last_undegraded":"2024-05-21T21:52:10.863329+0000","last_fullsized":"2024-05-21T21:52:10.863329+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:13:30.192908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'7","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T21:51:38.349287+0000","last_change":"2024-05-21T21:51:12.855306+0000","last_active":"2024-05-21T21:51:38.349287+0000","last_peered":"2024-05-21T21:51:38.349287+0000","last_clean":"2024-05-21T21:51:38.349287+0000","last_became_active":"2024-05-21T21:51:12.854722+0000","last_became_peered":"2024-05-21T21:51:12.854722+0000","last_unstale":"2024-05-21T21:51:38.349287+0000","last_undegraded":"2024-05-21T21:51:38.349287+0000","last_fullsized":"2024-05-21T21:51:38.349287+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:39:41.583219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T21:51:19.947366+0000","last_change":"2024-05-21T21:51:16.944758+0000","last_active":"2024-05-21T21:51:19.947366+0000","last_peered":"2024-05-21T21:51:19.947366+0000","last_clean":"2024-05-21T21:51:19.947366+0000","last_became_active":"2024-05-21T21:51:16.944393+0000","last_became_peered":"2024-05-21T21:51:16.944393+0000","last_unstale":"2024-05-21T21:51:19.947366+0000","last_undegraded":"2024-05-21T21:51:19.947366+0000","last_fullsized":"2024-05-21T21:51:19.947366+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:01:23.048355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'3","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-21T21:52:09.865240+0000","last_change":"2024-05-21T21:51:12.860378+0000","last_active":"2024-05-21T21:52:09.865240+0000","last_peered":"2024-05-21T21:52:09.865240+0000","last_clean":"2024-05-21T21:52:09.865240+0000","last_became_active":"2024-05-21T21:51:12.859978+0000","last_became_peered":"2024-05-21T21:51:12.859978+0000","last_unstale":"2024-05-21T21:52:09.865240+0000","last_undegraded":"2024-05-21T21:52:09.865240+0000","last_fullsized":"2024-05-21T21:52:09.865240+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T07:52:26.902232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T21:51:19.949340+0000","last_change":"2024-05-21T21:51:16.943194+0000","last_active":"2024-05-21T21:51:19.949340+0000","last_peered":"2024-05-21T21:51:19.949340+0000","last_clean":"2024-05-21T21:51:19.949340+0000","last_became_active":"2024-05-21T21:51:16.942884+0000","last_became_peered":"2024-05-21T21:51:16.942884+0000","last_unstale":"2024-05-21T21:51:19.949340+0000","last_undegraded":"2024-05-21T21:51:19.949340+0000","last_fullsized":"2024-05-21T21:51:19.949340+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T05:58:17.436995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.863880+0000","last_change":"2024-05-21T21:51:12.853390+0000","last_active":"2024-05-21T21:52:10.863880+0000","last_peered":"2024-05-21T21:52:10.863880+0000","last_clean":"2024-05-21T21:52:10.863880+0000","last_became_active":"2024-05-21T21:51:12.852990+0000","last_became_peered":"2024-05-21T21:51:12.852990+0000","last_unstale":"2024-05-21T21:52:10.863880+0000","last_undegraded":"2024-05-21T21:52:10.863880+0000","last_fullsized":"2024-05-21T21:52:10.863880+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:32:16.392609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-21T21:52:09.867247+0000","last_change":"2024-05-21T21:51:16.933430+0000","last_active":"2024-05-21T21:52:09.867247+0000","last_peered":"2024-05-21T21:52:09.867247+0000","last_clean":"2024-05-21T21:52:09.867247+0000","last_became_active":"2024-05-21T21:51:16.933053+0000","last_became_peered":"2024-05-21T21:51:16.933053+0000","last_unstale":"2024-05-21T21:52:09.867247+0000","last_undegraded":"2024-05-21T21:52:09.867247+0000","last_fullsized":"2024-05-21T21:52:09.867247+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T06:14:57.468727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'5","reported_seq":63,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-21T21:52:09.865363+0000","last_change":"2024-05-21T21:51:12.946113+0000","last_active":"2024-05-21T21:52:09.865363+0000","last_peered":"2024-05-21T21:52:09.865363+0000","last_clean":"2024-05-21T21:52:09.865363+0000","last_became_active":"2024-05-21T21:51:12.945650+0000","last_became_peered":"2024-05-21T21:51:12.945650+0000","last_unstale":"2024-05-21T21:52:09.865363+0000","last_undegraded":"2024-05-21T21:52:09.865363+0000","last_fullsized":"2024-05-21T21:52:09.865363+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T09:40:18.209400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":34,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-21T21:52:09.867452+0000","last_change":"2024-05-21T21:51:16.932632+0000","last_active":"2024-05-21T21:52:09.867452+0000","last_peered":"2024-05-21T21:52:09.867452+0000","last_clean":"2024-05-21T21:52:09.867452+0000","last_became_active":"2024-05-21T21:51:16.932256+0000","last_became_peered":"2024-05-21T21:51:16.932256+0000","last_unstale":"2024-05-21T21:52:09.867452+0000","last_undegraded":"2024-05-21T21:52:09.867452+0000","last_fullsized":"2024-05-21T21:52:09.867452+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:04:48.146485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'3","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T21:51:38.345123+0000","last_change":"2024-05-21T21:51:12.860602+0000","last_active":"2024-05-21T21:51:38.345123+0000","last_peered":"2024-05-21T21:51:38.345123+0000","last_clean":"2024-05-21T21:51:38.345123+0000","last_became_active":"2024-05-21T21:51:12.860228+0000","last_became_peered":"2024-05-21T21:51:12.860228+0000","last_unstale":"2024-05-21T21:51:38.345123+0000","last_undegraded":"2024-05-21T21:51:38.345123+0000","last_fullsized":"2024-05-21T21:51:38.345123+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:33:39.230872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'1","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-21T21:52:09.867258+0000","last_change":"2024-05-21T21:51:16.929616+0000","last_active":"2024-05-21T21:52:09.867258+0000","last_peered":"2024-05-21T21:52:09.867258+0000","last_clean":"2024-05-21T21:52:09.867258+0000","last_became_active":"2024-05-21T21:51:16.928569+0000","last_became_peered":"2024-05-21T21:51:16.928569+0000","last_unstale":"2024-05-21T21:52:09.867258+0000","last_undegraded":"2024-05-21T21:52:09.867258+0000","last_fullsized":"2024-05-21T21:52:09.867258+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:15:41.790344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'5","reported_seq":28,"reported_epoch":80,"state":"peering","last_fresh":"2024-05-21T21:51:38.348403+0000","last_change":"2024-05-21T21:52:09.866155+0000","last_active":"2024-05-21T21:52:09.866155+0000","last_peered":"2024-05-21T21:51:38.348403+0000","last_clean":"2024-05-21T21:51:38.348403+0000","last_became_active":"2024-05-21T21:51:12.859641+0000","last_became_peered":"2024-05-21T21:51:12.859641+0000","last_unstale":"2024-05-21T21:51:38.348403+0000","last_undegraded":"2024-05-21T21:51:38.348403+0000","last_fullsized":"2024-05-21T21:51:38.348403+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,3,4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-21T21:52:09.867474+0000","last_change":"2024-05-21T21:51:16.926496+0000","last_active":"2024-05-21T21:52:09.867474+0000","last_peered":"2024-05-21T21:52:09.867474+0000","last_clean":"2024-05-21T21:52:09.867474+0000","last_became_active":"2024-05-21T21:51:16.926112+0000","last_became_peered":"2024-05-21T21:51:16.926112+0000","last_unstale":"2024-05-21T21:52:09.867474+0000","last_undegraded":"2024-05-21T21:52:09.867474+0000","last_fullsized":"2024-05-21T21:52:09.867474+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:26:08.254421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T21:51:19.947566+0000","last_change":"2024-05-21T21:51:16.930522+0000","last_active":"2024-05-21T21:51:19.947566+0000","last_peered":"2024-05-21T21:51:19.947566+0000","last_clean":"2024-05-21T21:51:19.947566+0000","last_became_active":"2024-05-21T21:51:16.929853+0000","last_became_peered":"2024-05-21T21:51:16.929853+0000","last_unstale":"2024-05-21T21:51:19.947566+0000","last_undegraded":"2024-05-21T21:51:19.947566+0000","last_fullsized":"2024-05-21T21:51:19.947566+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:10:57.940009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-21T21:52:09.865072+0000","last_change":"2024-05-21T21:51:16.931751+0000","last_active":"2024-05-21T21:52:09.865072+0000","last_peered":"2024-05-21T21:52:09.865072+0000","last_clean":"2024-05-21T21:52:09.865072+0000","last_became_active":"2024-05-21T21:51:16.931353+0000","last_became_peered":"2024-05-21T21:51:16.931353+0000","last_unstale":"2024-05-21T21:52:09.865072+0000","last_undegraded":"2024-05-21T21:52:09.865072+0000","last_fullsized":"2024-05-21T21:52:09.865072+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:33:45.297850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-21T21:52:09.865073+0000","last_change":"2024-05-21T21:51:16.940096+0000","last_active":"2024-05-21T21:52:09.865073+0000","last_peered":"2024-05-21T21:52:09.865073+0000","last_clean":"2024-05-21T21:52:09.865073+0000","last_became_active":"2024-05-21T21:51:16.939740+0000","last_became_peered":"2024-05-21T21:51:16.939740+0000","last_unstale":"2024-05-21T21:52:09.865073+0000","last_undegraded":"2024-05-21T21:52:09.865073+0000","last_fullsized":"2024-05-21T21:52:09.865073+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:59:52.569526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T21:51:19.948822+0000","last_change":"2024-05-21T21:51:16.946488+0000","last_active":"2024-05-21T21:51:19.948822+0000","last_peered":"2024-05-21T21:51:19.948822+0000","last_clean":"2024-05-21T21:51:19.948822+0000","last_became_active":"2024-05-21T21:51:16.945957+0000","last_became_peered":"2024-05-21T21:51:16.945957+0000","last_unstale":"2024-05-21T21:51:19.948822+0000","last_undegraded":"2024-05-21T21:51:19.948822+0000","last_fullsized":"2024-05-21T21:51:19.948822+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:40:36.708344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-21T21:52:09.864097+0000","last_change":"2024-05-21T21:51:16.949349+0000","last_active":"2024-05-21T21:52:09.864097+0000","last_peered":"2024-05-21T21:52:09.864097+0000","last_clean":"2024-05-21T21:52:09.864097+0000","last_became_active":"2024-05-21T21:51:16.946986+0000","last_became_peered":"2024-05-21T21:51:16.946986+0000","last_unstale":"2024-05-21T21:52:09.864097+0000","last_undegraded":"2024-05-21T21:52:09.864097+0000","last_fullsized":"2024-05-21T21:52:09.864097+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:01:14.877110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T21:51:19.948098+0000","last_change":"2024-05-21T21:51:16.929172+0000","last_active":"2024-05-21T21:51:19.948098+0000","last_peered":"2024-05-21T21:51:19.948098+0000","last_clean":"2024-05-21T21:51:19.948098+0000","last_became_active":"2024-05-21T21:51:16.928823+0000","last_became_peered":"2024-05-21T21:51:16.928823+0000","last_unstale":"2024-05-21T21:51:19.948098+0000","last_undegraded":"2024-05-21T21:51:19.948098+0000","last_fullsized":"2024-05-21T21:51:19.948098+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:09:13.719272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-21T21:52:09.865587+0000","last_change":"2024-05-21T21:51:16.930332+0000","last_active":"2024-05-21T21:52:09.865587+0000","last_peered":"2024-05-21T21:52:09.865587+0000","last_clean":"2024-05-21T21:52:09.865587+0000","last_became_active":"2024-05-21T21:51:16.930024+0000","last_became_peered":"2024-05-21T21:51:16.930024+0000","last_unstale":"2024-05-21T21:52:09.865587+0000","last_undegraded":"2024-05-21T21:52:09.865587+0000","last_fullsized":"2024-05-21T21:52:09.865587+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:47:41.272725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T21:51:19.949814+0000","last_change":"2024-05-21T21:51:16.930102+0000","last_active":"2024-05-21T21:51:19.949814+0000","last_peered":"2024-05-21T21:51:19.949814+0000","last_clean":"2024-05-21T21:51:19.949814+0000","last_became_active":"2024-05-21T21:51:16.929410+0000","last_became_peered":"2024-05-21T21:51:16.929410+0000","last_unstale":"2024-05-21T21:51:19.949814+0000","last_undegraded":"2024-05-21T21:51:19.949814+0000","last_fullsized":"2024-05-21T21:51:19.949814+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:11:46.296288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.873829+0000","last_change":"2024-05-21T21:52:10.873424+0000","last_active":"2024-05-21T21:52:10.873829+0000","last_peered":"2024-05-21T21:52:10.873829+0000","last_clean":"2024-05-21T21:52:10.873829+0000","last_became_active":"2024-05-21T21:52:10.873118+0000","last_became_peered":"2024-05-21T21:52:10.873118+0000","last_unstale":"2024-05-21T21:52:10.873829+0000","last_undegraded":"2024-05-21T21:52:10.873829+0000","last_fullsized":"2024-05-21T21:52:10.873829+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:05:00.731013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,3],"acting":[8,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T21:51:19.948466+0000","last_change":"2024-05-21T21:51:16.934442+0000","last_active":"2024-05-21T21:51:19.948466+0000","last_peered":"2024-05-21T21:51:19.948466+0000","last_clean":"2024-05-21T21:51:19.948466+0000","last_became_active":"2024-05-21T21:51:16.934086+0000","last_became_peered":"2024-05-21T21:51:16.934086+0000","last_unstale":"2024-05-21T21:51:19.948466+0000","last_undegraded":"2024-05-21T21:51:19.948466+0000","last_fullsized":"2024-05-21T21:51:19.948466+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:36:44.302532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-21T21:52:09.870214+0000","last_change":"2024-05-21T21:51:16.924479+0000","last_active":"2024-05-21T21:52:09.870214+0000","last_peered":"2024-05-21T21:52:09.870214+0000","last_clean":"2024-05-21T21:52:09.870214+0000","last_became_active":"2024-05-21T21:51:16.923998+0000","last_became_peered":"2024-05-21T21:51:16.923998+0000","last_unstale":"2024-05-21T21:52:09.870214+0000","last_undegraded":"2024-05-21T21:52:09.870214+0000","last_fullsized":"2024-05-21T21:52:09.870214+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:23:07.931731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-21T21:52:09.864311+0000","last_change":"2024-05-21T21:51:16.947271+0000","last_active":"2024-05-21T21:52:09.864311+0000","last_peered":"2024-05-21T21:52:09.864311+0000","last_clean":"2024-05-21T21:52:09.864311+0000","last_became_active":"2024-05-21T21:51:16.946912+0000","last_became_peered":"2024-05-21T21:51:16.946912+0000","last_unstale":"2024-05-21T21:52:09.864311+0000","last_undegraded":"2024-05-21T21:52:09.864311+0000","last_fullsized":"2024-05-21T21:52:09.864311+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:51:29.646915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-21T21:51:19.947638+0000","last_change":"2024-05-21T21:51:16.928852+0000","last_active":"2024-05-21T21:51:19.947638+0000","last_peered":"2024-05-21T21:51:19.947638+0000","last_clean":"2024-05-21T21:51:19.947638+0000","last_became_active":"2024-05-21T21:51:16.928480+0000","last_became_peered":"2024-05-21T21:51:16.928480+0000","last_unstale":"2024-05-21T21:51:19.947638+0000","last_undegraded":"2024-05-21T21:51:19.947638+0000","last_fullsized":"2024-05-21T21:51:19.947638+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T09:12:30.691254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":62709,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":336,"num_read_kb":235,"num_write":244,"num_write_kb":130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":561152,"data_stored":196641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":209,"ondisk_log_size":209,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645339,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":868352,"data_stored":416762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841567,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28544,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716608,"statfs":{"total":95995035648,"available":95965806592,"internally_reserved":0,"allocated":1564672,"data_stored":1063687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005093,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":892928,"data_stored":416692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201324,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":921600,"data_stored":434267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397552,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28488,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716664,"statfs":{"total":95995035648,"available":95965863936,"internally_reserved":0,"allocated":1503232,"data_stored":1018680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561078,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":876544,"data_stored":416409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574930,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":954368,"data_stored":485785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738455,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36212,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708940,"statfs":{"total":95995035648,"available":95957954560,"internally_reserved":0,"allocated":897024,"data_stored":417748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411406,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36292,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708860,"statfs":{"total":95995035648,"available":95957872640,"internally_reserved":0,"allocated":974848,"data_stored":486396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149766,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36208,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708944,"statfs":{"total":95995035648,"available":95957958656,"internally_reserved":0,"allocated":892928,"data_stored":416234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953537,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":876544,"data_stored":416661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757308,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32640,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712512,"statfs":{"total":95995035648,"available":95961612288,"internally_reserved":0,"allocated":1499136,"data_stored":1009266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":61514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":57632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-21T21:52:13.461 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph pg dump --format=json 2024-05-21T21:52:13.937 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:52:14.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:52:13 smithi151 ceph-mon[31623]: from='client.24637 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T21:52:14.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:52:13 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:52:14.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:52:13 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:52:14.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:52:13 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:52:14.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:52:13 smithi184 ceph-mon[36942]: from='client.24637 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T21:52:14.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:52:13 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:52:14.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:52:13 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:52:14.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:52:13 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:52:14.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:52:13 smithi164 ceph-mon[36269]: from='client.24637 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T21:52:14.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:52:13 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:52:14.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:52:13 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:52:14.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:52:13 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:52:15.089 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:52:14 smithi151 ceph-mon[31623]: pgmap v344: 97 pgs: 2 peering, 95 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.5 KiB/s wr, 11 op/s; 28 B/s, 1 keys/s, 1 objects/s recovering 2024-05-21T21:52:15.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:52:14 smithi184 ceph-mon[36942]: pgmap v344: 97 pgs: 2 peering, 95 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.5 KiB/s wr, 11 op/s; 28 B/s, 1 keys/s, 1 objects/s recovering 2024-05-21T21:52:15.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:52:14 smithi164 ceph-mon[36269]: pgmap v344: 97 pgs: 2 peering, 95 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.5 KiB/s wr, 11 op/s; 28 B/s, 1 keys/s, 1 objects/s recovering 2024-05-21T21:52:16.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:16.009+0000 7f6244600700 1 -- 172.21.15.151:0/1347348928 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f623c105760 msgr2=0x7f623c105be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:52:16.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:16.009+0000 7f6244600700 1 --2- 172.21.15.151:0/1347348928 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f623c105760 0x7f623c105be0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6228009990 tx=0x7f6228009ca0 comp rx=0 tx=0).stop 2024-05-21T21:52:16.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:16.009+0000 7f6244600700 1 -- 172.21.15.151:0/1347348928 shutdown_connections 2024-05-21T21:52:16.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:16.010+0000 7f6244600700 1 --2- 172.21.15.151:0/1347348928 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f623c106120 0x7f623c112630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:16.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:16.010+0000 7f6244600700 1 --2- 172.21.15.151:0/1347348928 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f623c10b760 0x7f623c10bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:16.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:16.010+0000 7f6244600700 1 --2- 172.21.15.151:0/1347348928 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f623c105760 0x7f623c105be0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:16.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:16.010+0000 7f6244600700 1 -- 172.21.15.151:0/1347348928 >> 172.21.15.151:0/1347348928 conn(0x7f623c101210 msgr2=0x7f623c103630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:52:16.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:16.010+0000 7f6244600700 1 -- 172.21.15.151:0/1347348928 shutdown_connections 2024-05-21T21:52:16.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:16.010+0000 7f6244600700 1 -- 172.21.15.151:0/1347348928 wait complete. 2024-05-21T21:52:16.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:16.011+0000 7f6244600700 1 Processor -- start 2024-05-21T21:52:16.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:16.011+0000 7f6244600700 1 -- start start 2024-05-21T21:52:16.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:16.011+0000 7f6244600700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f623c105760 0x7f623c19bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:52:16.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:16.011+0000 7f6244600700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f623c106120 0x7f623c19c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:52:16.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:16.011+0000 7f6244600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f623c10b760 0x7f623c1a0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:52:16.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:16.011+0000 7f6244600700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f623c1159f0 con 0x7f623c10b760 2024-05-21T21:52:16.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:16.011+0000 7f6244600700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f623c115870 con 0x7f623c106120 2024-05-21T21:52:16.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:16.011+0000 7f6244600700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f623c115b70 con 0x7f623c105760 2024-05-21T21:52:16.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:16.011+0000 7f6242600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f623c10b760 0x7f623c1a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:52:16.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:16.012+0000 7f6242600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f623c10b760 0x7f623c1a0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:45304/0 (socket says 172.21.15.151:45304) 2024-05-21T21:52:16.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:16.012+0000 7f6242600700 1 -- 172.21.15.151:0/59492692 learned_addr learned my addr 172.21.15.151:0/59492692 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:52:16.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:16.012+0000 7f6241c00700 1 --2- 172.21.15.151:0/59492692 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f623c105760 0x7f623c19bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:52:16.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:16.012+0000 7f6241200700 1 --2- 172.21.15.151:0/59492692 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f623c106120 0x7f623c19c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:52:16.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:16.012+0000 7f6241c00700 1 -- 172.21.15.151:0/59492692 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f623c106120 msgr2=0x7f623c19c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:52:16.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:16.012+0000 7f6241c00700 1 --2- 172.21.15.151:0/59492692 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f623c106120 0x7f623c19c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:16.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:16.012+0000 7f6241c00700 1 -- 172.21.15.151:0/59492692 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f623c10b760 msgr2=0x7f623c1a0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:52:16.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:16.012+0000 7f6241c00700 1 --2- 172.21.15.151:0/59492692 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f623c10b760 0x7f623c1a0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:16.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:16.012+0000 7f6241c00700 1 -- 172.21.15.151:0/59492692 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f623c1a0eb0 con 0x7f623c105760 2024-05-21T21:52:16.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:16.012+0000 7f6241c00700 1 --2- 172.21.15.151:0/59492692 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f623c105760 0x7f623c19bea0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f623000e9f0 tx=0x7f623000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:52:16.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:16.013+0000 7f623a000700 1 -- 172.21.15.151:0/59492692 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f623000c9c0 con 0x7f623c105760 2024-05-21T21:52:16.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:16.013+0000 7f623a000700 1 -- 172.21.15.151:0/59492692 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f623000cb60 con 0x7f623c105760 2024-05-21T21:52:16.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:16.013+0000 7f623a000700 1 -- 172.21.15.151:0/59492692 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6230018680 con 0x7f623c105760 2024-05-21T21:52:16.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:16.013+0000 7f6244600700 1 -- 172.21.15.151:0/59492692 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f623c1a11a0 con 0x7f623c105760 2024-05-21T21:52:16.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:16.013+0000 7f6244600700 1 -- 172.21.15.151:0/59492692 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f623c1a8a20 con 0x7f623c105760 2024-05-21T21:52:16.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:16.014+0000 7f6244600700 1 -- 172.21.15.151:0/59492692 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f623c06a230 con 0x7f623c105760 2024-05-21T21:52:16.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:16.016+0000 7f623a000700 1 -- 172.21.15.151:0/59492692 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f6230010430 con 0x7f623c105760 2024-05-21T21:52:16.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:16.018+0000 7f623a000700 1 --2- 172.21.15.151:0/59492692 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f62240b2910 0x7f62240b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:52:16.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:16.018+0000 7f623a000700 1 -- 172.21.15.151:0/59492692 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f62300e2840 con 0x7f623c105760 2024-05-21T21:52:16.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:16.018+0000 7f6241200700 1 --2- 172.21.15.151:0/59492692 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f62240b2910 0x7f62240b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:52:16.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:16.019+0000 7f6241200700 1 --2- 172.21.15.151:0/59492692 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f62240b2910 0x7f62240b4dd0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f623c103910 tx=0x7f6228005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:52:16.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:16.020+0000 7f623a000700 1 -- 172.21.15.151:0/59492692 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f62300aa3c0 con 0x7f623c105760 2024-05-21T21:52:16.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:16.246+0000 7f6244600700 1 -- 172.21.15.151:0/59492692 --> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f623c063980 con 0x7f62240b2910 2024-05-21T21:52:16.257 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:16.257+0000 7f623a000700 1 -- 172.21.15.151:0/59492692 <== mgr.14152 v2:172.21.15.151:6800/2796622483 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258789 (secure 0 0 0) 0x7f6228067300 con 0x7f62240b2910 2024-05-21T21:52:16.263 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T21:52:16.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:16.264+0000 7f6244600700 1 -- 172.21.15.151:0/59492692 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f62240b2910 msgr2=0x7f62240b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:52:16.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:16.264+0000 7f6244600700 1 --2- 172.21.15.151:0/59492692 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f62240b2910 0x7f62240b4dd0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f623c103910 tx=0x7f6228005fd0 comp rx=0 tx=0).stop 2024-05-21T21:52:16.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:16.265+0000 7f6244600700 1 -- 172.21.15.151:0/59492692 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f623c105760 msgr2=0x7f623c19bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:52:16.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:16.265+0000 7f6244600700 1 --2- 172.21.15.151:0/59492692 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f623c105760 0x7f623c19bea0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f623000e9f0 tx=0x7f623000ed00 comp rx=0 tx=0).stop 2024-05-21T21:52:16.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:16.265+0000 7f6244600700 1 -- 172.21.15.151:0/59492692 shutdown_connections 2024-05-21T21:52:16.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:16.265+0000 7f6244600700 1 --2- 172.21.15.151:0/59492692 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f62240b2910 0x7f62240b4dd0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:16.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:16.265+0000 7f6244600700 1 --2- 172.21.15.151:0/59492692 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f623c10b760 0x7f623c1a0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:16.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:16.265+0000 7f6244600700 1 --2- 172.21.15.151:0/59492692 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f623c105760 0x7f623c19bea0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:16.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:16.265+0000 7f6244600700 1 --2- 172.21.15.151:0/59492692 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f623c106120 0x7f623c19c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:16.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:16.265+0000 7f6244600700 1 -- 172.21.15.151:0/59492692 >> 172.21.15.151:0/59492692 conn(0x7f623c101210 msgr2=0x7f623c101aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:52:16.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:16.265+0000 7f6244600700 1 -- 172.21.15.151:0/59492692 shutdown_connections 2024-05-21T21:52:16.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:16.265+0000 7f6244600700 1 -- 172.21.15.151:0/59492692 wait complete. 2024-05-21T21:52:16.266 INFO:teuthology.orchestra.run.smithi151.stderr:dumped all 2024-05-21T21:52:17.009 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:52:16 smithi151 ceph-mon[31623]: pgmap v345: 97 pgs: 1 peering, 96 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 639 B/s wr, 8 op/s; 28 B/s, 2 keys/s, 2 objects/s recovering 2024-05-21T21:52:17.073 INFO:teuthology.orchestra.run.smithi151.stdout:{"pg_ready":true,"pg_map":{"version":345,"stamp":"2024-05-21T21:52:15.213152+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657568,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":456,"num_read_kb":332,"num_write":370,"num_write_kb":2287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":2952069,"num_keys_recovered":17,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":304,"ondisk_log_size":304,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":293,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":366048,"kb_used_data":12668,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124575776,"statfs":{"total":1151940427776,"available":1151565594624,"internally_reserved":0,"allocated":12972032,"data_stored":7131924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19140,"internal_metadata":361674044},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4257,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":69,"num_read_kb":38,"num_write":3,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":229,"num_keys_recovered":17,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003372"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.861691+0000","last_change":"2024-05-21T21:51:16.945132+0000","last_active":"2024-05-21T21:52:10.861691+0000","last_peered":"2024-05-21T21:52:10.861691+0000","last_clean":"2024-05-21T21:52:10.861691+0000","last_became_active":"2024-05-21T21:51:16.944766+0000","last_became_peered":"2024-05-21T21:51:16.944766+0000","last_unstale":"2024-05-21T21:52:10.861691+0000","last_undegraded":"2024-05-21T21:52:10.861691+0000","last_fullsized":"2024-05-21T21:52:10.861691+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:54:04.185095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.873699+0000","last_change":"2024-05-21T21:51:16.939291+0000","last_active":"2024-05-21T21:52:10.873699+0000","last_peered":"2024-05-21T21:52:10.873699+0000","last_clean":"2024-05-21T21:52:10.873699+0000","last_became_active":"2024-05-21T21:51:16.938730+0000","last_became_peered":"2024-05-21T21:51:16.938730+0000","last_unstale":"2024-05-21T21:52:10.873699+0000","last_undegraded":"2024-05-21T21:52:10.873699+0000","last_fullsized":"2024-05-21T21:52:10.873699+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:05:30.623744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.873014+0000","last_change":"2024-05-21T21:51:16.941565+0000","last_active":"2024-05-21T21:52:10.873014+0000","last_peered":"2024-05-21T21:52:10.873014+0000","last_clean":"2024-05-21T21:52:10.873014+0000","last_became_active":"2024-05-21T21:51:16.941033+0000","last_became_peered":"2024-05-21T21:51:16.941033+0000","last_unstale":"2024-05-21T21:52:10.873014+0000","last_undegraded":"2024-05-21T21:52:10.873014+0000","last_fullsized":"2024-05-21T21:52:10.873014+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:04:55.789479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.865221+0000","last_change":"2024-05-21T21:51:16.928455+0000","last_active":"2024-05-21T21:52:10.865221+0000","last_peered":"2024-05-21T21:52:10.865221+0000","last_clean":"2024-05-21T21:52:10.865221+0000","last_became_active":"2024-05-21T21:51:16.927874+0000","last_became_peered":"2024-05-21T21:51:16.927874+0000","last_unstale":"2024-05-21T21:52:10.865221+0000","last_undegraded":"2024-05-21T21:52:10.865221+0000","last_fullsized":"2024-05-21T21:52:10.865221+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:25:16.483175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.863776+0000","last_change":"2024-05-21T21:51:16.937773+0000","last_active":"2024-05-21T21:52:10.863776+0000","last_peered":"2024-05-21T21:52:10.863776+0000","last_clean":"2024-05-21T21:52:10.863776+0000","last_became_active":"2024-05-21T21:51:16.937306+0000","last_became_peered":"2024-05-21T21:51:16.937306+0000","last_unstale":"2024-05-21T21:52:10.863776+0000","last_undegraded":"2024-05-21T21:52:10.863776+0000","last_fullsized":"2024-05-21T21:52:10.863776+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:05:03.762390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.865632+0000","last_change":"2024-05-21T21:51:16.948275+0000","last_active":"2024-05-21T21:52:10.865632+0000","last_peered":"2024-05-21T21:52:10.865632+0000","last_clean":"2024-05-21T21:52:10.865632+0000","last_became_active":"2024-05-21T21:51:16.947890+0000","last_became_peered":"2024-05-21T21:51:16.947890+0000","last_unstale":"2024-05-21T21:52:10.865632+0000","last_undegraded":"2024-05-21T21:52:10.865632+0000","last_fullsized":"2024-05-21T21:52:10.865632+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:08:38.572177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-21T21:52:09.867065+0000","last_change":"2024-05-21T21:51:16.946821+0000","last_active":"2024-05-21T21:52:09.867065+0000","last_peered":"2024-05-21T21:52:09.867065+0000","last_clean":"2024-05-21T21:52:09.867065+0000","last_became_active":"2024-05-21T21:51:16.946515+0000","last_became_peered":"2024-05-21T21:51:16.946515+0000","last_unstale":"2024-05-21T21:52:09.867065+0000","last_undegraded":"2024-05-21T21:52:09.867065+0000","last_fullsized":"2024-05-21T21:52:09.867065+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T09:26:25.867895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.870476+0000","last_change":"2024-05-21T21:51:16.929228+0000","last_active":"2024-05-21T21:52:10.870476+0000","last_peered":"2024-05-21T21:52:10.870476+0000","last_clean":"2024-05-21T21:52:10.870476+0000","last_became_active":"2024-05-21T21:51:16.928875+0000","last_became_peered":"2024-05-21T21:51:16.928875+0000","last_unstale":"2024-05-21T21:52:10.870476+0000","last_undegraded":"2024-05-21T21:52:10.870476+0000","last_fullsized":"2024-05-21T21:52:10.870476+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:50:56.431310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-21T21:52:09.870499+0000","last_change":"2024-05-21T21:51:16.932360+0000","last_active":"2024-05-21T21:52:09.870499+0000","last_peered":"2024-05-21T21:52:09.870499+0000","last_clean":"2024-05-21T21:52:09.870499+0000","last_became_active":"2024-05-21T21:51:16.932036+0000","last_became_peered":"2024-05-21T21:51:16.932036+0000","last_unstale":"2024-05-21T21:52:09.870499+0000","last_undegraded":"2024-05-21T21:52:09.870499+0000","last_fullsized":"2024-05-21T21:52:09.870499+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T09:06:37.257323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.876556+0000","last_change":"2024-05-21T21:52:10.876556+0000","last_active":"2024-05-21T21:52:10.876556+0000","last_peered":"2024-05-21T21:52:10.876556+0000","last_clean":"2024-05-21T21:52:10.876556+0000","last_became_active":"2024-05-21T21:52:10.876273+0000","last_became_peered":"2024-05-21T21:52:10.876273+0000","last_unstale":"2024-05-21T21:52:10.876556+0000","last_undegraded":"2024-05-21T21:52:10.876556+0000","last_fullsized":"2024-05-21T21:52:10.876556+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T05:09:40.037731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-21T21:52:09.867342+0000","last_change":"2024-05-21T21:51:16.933425+0000","last_active":"2024-05-21T21:52:09.867342+0000","last_peered":"2024-05-21T21:52:09.867342+0000","last_clean":"2024-05-21T21:52:09.867342+0000","last_became_active":"2024-05-21T21:51:16.933057+0000","last_became_peered":"2024-05-21T21:51:16.933057+0000","last_unstale":"2024-05-21T21:52:09.867342+0000","last_undegraded":"2024-05-21T21:52:09.867342+0000","last_fullsized":"2024-05-21T21:52:09.867342+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:04:57.237841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.865179+0000","last_change":"2024-05-21T21:51:16.927014+0000","last_active":"2024-05-21T21:52:10.865179+0000","last_peered":"2024-05-21T21:52:10.865179+0000","last_clean":"2024-05-21T21:52:10.865179+0000","last_became_active":"2024-05-21T21:51:16.925954+0000","last_became_peered":"2024-05-21T21:51:16.925954+0000","last_unstale":"2024-05-21T21:52:10.865179+0000","last_undegraded":"2024-05-21T21:52:10.865179+0000","last_fullsized":"2024-05-21T21:52:10.865179+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:19:20.530727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.862761+0000","last_change":"2024-05-21T21:51:16.930909+0000","last_active":"2024-05-21T21:52:10.862761+0000","last_peered":"2024-05-21T21:52:10.862761+0000","last_clean":"2024-05-21T21:52:10.862761+0000","last_became_active":"2024-05-21T21:51:16.927110+0000","last_became_peered":"2024-05-21T21:51:16.927110+0000","last_unstale":"2024-05-21T21:52:10.862761+0000","last_undegraded":"2024-05-21T21:52:10.862761+0000","last_fullsized":"2024-05-21T21:52:10.862761+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T06:23:55.386047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.872086+0000","last_change":"2024-05-21T21:51:16.938797+0000","last_active":"2024-05-21T21:52:10.872086+0000","last_peered":"2024-05-21T21:52:10.872086+0000","last_clean":"2024-05-21T21:52:10.872086+0000","last_became_active":"2024-05-21T21:51:16.938263+0000","last_became_peered":"2024-05-21T21:51:16.938263+0000","last_unstale":"2024-05-21T21:52:10.872086+0000","last_undegraded":"2024-05-21T21:52:10.872086+0000","last_fullsized":"2024-05-21T21:52:10.872086+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:37:40.521118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.871620+0000","last_change":"2024-05-21T21:51:16.944160+0000","last_active":"2024-05-21T21:52:10.871620+0000","last_peered":"2024-05-21T21:52:10.871620+0000","last_clean":"2024-05-21T21:52:10.871620+0000","last_became_active":"2024-05-21T21:51:16.940046+0000","last_became_peered":"2024-05-21T21:51:16.940046+0000","last_unstale":"2024-05-21T21:52:10.871620+0000","last_undegraded":"2024-05-21T21:52:10.871620+0000","last_fullsized":"2024-05-21T21:52:10.871620+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:14:13.278799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-21T21:52:09.870033+0000","last_change":"2024-05-21T21:51:16.945599+0000","last_active":"2024-05-21T21:52:09.870033+0000","last_peered":"2024-05-21T21:52:09.870033+0000","last_clean":"2024-05-21T21:52:09.870033+0000","last_became_active":"2024-05-21T21:51:16.945196+0000","last_became_peered":"2024-05-21T21:51:16.945196+0000","last_unstale":"2024-05-21T21:52:09.870033+0000","last_undegraded":"2024-05-21T21:52:09.870033+0000","last_fullsized":"2024-05-21T21:52:09.870033+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:59:24.453615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.861779+0000","last_change":"2024-05-21T21:51:16.944895+0000","last_active":"2024-05-21T21:52:10.861779+0000","last_peered":"2024-05-21T21:52:10.861779+0000","last_clean":"2024-05-21T21:52:10.861779+0000","last_became_active":"2024-05-21T21:51:16.944560+0000","last_became_peered":"2024-05-21T21:51:16.944560+0000","last_unstale":"2024-05-21T21:52:10.861779+0000","last_undegraded":"2024-05-21T21:52:10.861779+0000","last_fullsized":"2024-05-21T21:52:10.861779+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:53:13.264701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.877457+0000","last_change":"2024-05-21T21:51:16.943861+0000","last_active":"2024-05-21T21:52:10.877457+0000","last_peered":"2024-05-21T21:52:10.877457+0000","last_clean":"2024-05-21T21:52:10.877457+0000","last_became_active":"2024-05-21T21:51:16.943123+0000","last_became_peered":"2024-05-21T21:51:16.943123+0000","last_unstale":"2024-05-21T21:52:10.877457+0000","last_undegraded":"2024-05-21T21:52:10.877457+0000","last_fullsized":"2024-05-21T21:52:10.877457+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:35:37.732855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-21T21:52:09.865357+0000","last_change":"2024-05-21T21:51:16.940580+0000","last_active":"2024-05-21T21:52:09.865357+0000","last_peered":"2024-05-21T21:52:09.865357+0000","last_clean":"2024-05-21T21:52:09.865357+0000","last_became_active":"2024-05-21T21:51:16.940264+0000","last_became_peered":"2024-05-21T21:51:16.940264+0000","last_unstale":"2024-05-21T21:52:09.865357+0000","last_undegraded":"2024-05-21T21:52:09.865357+0000","last_fullsized":"2024-05-21T21:52:09.865357+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:54:22.915541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"80'13","reported_seq":76,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:13.445747+0000","last_change":"2024-05-21T21:51:12.852935+0000","last_active":"2024-05-21T21:52:13.445747+0000","last_peered":"2024-05-21T21:52:13.445747+0000","last_clean":"2024-05-21T21:52:13.445747+0000","last_became_active":"2024-05-21T21:51:12.851802+0000","last_became_peered":"2024-05-21T21:51:12.851802+0000","last_unstale":"2024-05-21T21:52:13.445747+0000","last_undegraded":"2024-05-21T21:52:13.445747+0000","last_fullsized":"2024-05-21T21:52:13.445747+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:13:24.643408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":37,"num_read_kb":37,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.868253+0000","last_change":"2024-05-21T21:51:16.939461+0000","last_active":"2024-05-21T21:52:10.868253+0000","last_peered":"2024-05-21T21:52:10.868253+0000","last_clean":"2024-05-21T21:52:10.868253+0000","last_became_active":"2024-05-21T21:51:16.936322+0000","last_became_peered":"2024-05-21T21:51:16.936322+0000","last_unstale":"2024-05-21T21:52:10.868253+0000","last_undegraded":"2024-05-21T21:52:10.868253+0000","last_fullsized":"2024-05-21T21:52:10.868253+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:37:26.153963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.868674+0000","last_change":"2024-05-21T21:51:12.941838+0000","last_active":"2024-05-21T21:52:10.868674+0000","last_peered":"2024-05-21T21:52:10.868674+0000","last_clean":"2024-05-21T21:52:10.868674+0000","last_became_active":"2024-05-21T21:51:12.940818+0000","last_became_peered":"2024-05-21T21:51:12.940818+0000","last_unstale":"2024-05-21T21:52:10.868674+0000","last_undegraded":"2024-05-21T21:52:10.868674+0000","last_fullsized":"2024-05-21T21:52:10.868674+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:30:42.349241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.873994+0000","last_change":"2024-05-21T21:51:16.950252+0000","last_active":"2024-05-21T21:52:10.873994+0000","last_peered":"2024-05-21T21:52:10.873994+0000","last_clean":"2024-05-21T21:52:10.873994+0000","last_became_active":"2024-05-21T21:51:16.947375+0000","last_became_peered":"2024-05-21T21:51:16.947375+0000","last_unstale":"2024-05-21T21:52:10.873994+0000","last_undegraded":"2024-05-21T21:52:10.873994+0000","last_fullsized":"2024-05-21T21:52:10.873994+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:27:30.462799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"80'12","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.876122+0000","last_change":"2024-05-21T21:52:10.875739+0000","last_active":"2024-05-21T21:52:10.876122+0000","last_peered":"2024-05-21T21:52:10.876122+0000","last_clean":"2024-05-21T21:52:10.876122+0000","last_became_active":"2024-05-21T21:52:10.865846+0000","last_became_peered":"2024-05-21T21:52:10.865846+0000","last_unstale":"2024-05-21T21:52:10.876122+0000","last_undegraded":"2024-05-21T21:52:10.876122+0000","last_fullsized":"2024-05-21T21:52:10.876122+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T06:48:06.465757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-21T21:52:09.870192+0000","last_change":"2024-05-21T21:51:16.926380+0000","last_active":"2024-05-21T21:52:09.870192+0000","last_peered":"2024-05-21T21:52:09.870192+0000","last_clean":"2024-05-21T21:52:09.870192+0000","last_became_active":"2024-05-21T21:51:16.925821+0000","last_became_peered":"2024-05-21T21:51:16.925821+0000","last_unstale":"2024-05-21T21:52:09.870192+0000","last_undegraded":"2024-05-21T21:52:09.870192+0000","last_fullsized":"2024-05-21T21:52:09.870192+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:42:14.887155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'5","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.864272+0000","last_change":"2024-05-21T21:51:12.854342+0000","last_active":"2024-05-21T21:52:10.864272+0000","last_peered":"2024-05-21T21:52:10.864272+0000","last_clean":"2024-05-21T21:52:10.864272+0000","last_became_active":"2024-05-21T21:51:12.853908+0000","last_became_peered":"2024-05-21T21:51:12.853908+0000","last_unstale":"2024-05-21T21:52:10.864272+0000","last_undegraded":"2024-05-21T21:52:10.864272+0000","last_fullsized":"2024-05-21T21:52:10.864272+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:35:10.374749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.973816+0000","last_change":"2024-05-21T21:52:10.973816+0000","last_active":"2024-05-21T21:52:10.973816+0000","last_peered":"2024-05-21T21:52:10.973816+0000","last_clean":"2024-05-21T21:52:10.973816+0000","last_became_active":"2024-05-21T21:52:10.973457+0000","last_became_peered":"2024-05-21T21:52:10.973457+0000","last_unstale":"2024-05-21T21:52:10.973816+0000","last_undegraded":"2024-05-21T21:52:10.973816+0000","last_fullsized":"2024-05-21T21:52:10.973816+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T05:01:41.915158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"80'6","reported_seq":69,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:11.433184+0000","last_change":"2024-05-21T21:51:12.855658+0000","last_active":"2024-05-21T21:52:11.433184+0000","last_peered":"2024-05-21T21:52:11.433184+0000","last_clean":"2024-05-21T21:52:11.433184+0000","last_became_active":"2024-05-21T21:51:12.855281+0000","last_became_peered":"2024-05-21T21:51:12.855281+0000","last_unstale":"2024-05-21T21:52:11.433184+0000","last_undegraded":"2024-05-21T21:52:11.433184+0000","last_fullsized":"2024-05-21T21:52:11.433184+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:55:08.836045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":37,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.863544+0000","last_change":"2024-05-21T21:51:16.946977+0000","last_active":"2024-05-21T21:52:10.863544+0000","last_peered":"2024-05-21T21:52:10.863544+0000","last_clean":"2024-05-21T21:52:10.863544+0000","last_became_active":"2024-05-21T21:51:16.946562+0000","last_became_peered":"2024-05-21T21:51:16.946562+0000","last_unstale":"2024-05-21T21:52:10.863544+0000","last_undegraded":"2024-05-21T21:52:10.863544+0000","last_fullsized":"2024-05-21T21:52:10.863544+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:43:33.370265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"80'7","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.865540+0000","last_change":"2024-05-21T21:51:12.852574+0000","last_active":"2024-05-21T21:52:10.865540+0000","last_peered":"2024-05-21T21:52:10.865540+0000","last_clean":"2024-05-21T21:52:10.865540+0000","last_became_active":"2024-05-21T21:51:12.852258+0000","last_became_peered":"2024-05-21T21:51:12.852258+0000","last_unstale":"2024-05-21T21:52:10.865540+0000","last_undegraded":"2024-05-21T21:52:10.865540+0000","last_fullsized":"2024-05-21T21:52:10.865540+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:09:40.280821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.867344+0000","last_change":"2024-05-21T21:51:16.934768+0000","last_active":"2024-05-21T21:52:10.867344+0000","last_peered":"2024-05-21T21:52:10.867344+0000","last_clean":"2024-05-21T21:52:10.867344+0000","last_became_active":"2024-05-21T21:51:16.934275+0000","last_became_peered":"2024-05-21T21:51:16.934275+0000","last_unstale":"2024-05-21T21:52:10.867344+0000","last_undegraded":"2024-05-21T21:52:10.867344+0000","last_fullsized":"2024-05-21T21:52:10.867344+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:31:54.425069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"80'3","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.861882+0000","last_change":"2024-05-21T21:51:12.854166+0000","last_active":"2024-05-21T21:52:10.861882+0000","last_peered":"2024-05-21T21:52:10.861882+0000","last_clean":"2024-05-21T21:52:10.861882+0000","last_became_active":"2024-05-21T21:51:12.853508+0000","last_became_peered":"2024-05-21T21:51:12.853508+0000","last_unstale":"2024-05-21T21:52:10.861882+0000","last_undegraded":"2024-05-21T21:52:10.861882+0000","last_fullsized":"2024-05-21T21:52:10.861882+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:29:41.858655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.870323+0000","last_change":"2024-05-21T21:51:16.938293+0000","last_active":"2024-05-21T21:52:10.870323+0000","last_peered":"2024-05-21T21:52:10.870323+0000","last_clean":"2024-05-21T21:52:10.870323+0000","last_became_active":"2024-05-21T21:51:16.937970+0000","last_became_peered":"2024-05-21T21:51:16.937970+0000","last_unstale":"2024-05-21T21:52:10.870323+0000","last_undegraded":"2024-05-21T21:52:10.870323+0000","last_fullsized":"2024-05-21T21:52:10.870323+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:18:27.473098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'3","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.866723+0000","last_change":"2024-05-21T21:51:12.853366+0000","last_active":"2024-05-21T21:52:10.866723+0000","last_peered":"2024-05-21T21:52:10.866723+0000","last_clean":"2024-05-21T21:52:10.866723+0000","last_became_active":"2024-05-21T21:51:12.853028+0000","last_became_peered":"2024-05-21T21:51:12.853028+0000","last_unstale":"2024-05-21T21:52:10.866723+0000","last_undegraded":"2024-05-21T21:52:10.866723+0000","last_fullsized":"2024-05-21T21:52:10.866723+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T09:15:52.593028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.874880+0000","last_change":"2024-05-21T21:51:16.947903+0000","last_active":"2024-05-21T21:52:10.874880+0000","last_peered":"2024-05-21T21:52:10.874880+0000","last_clean":"2024-05-21T21:52:10.874880+0000","last_became_active":"2024-05-21T21:51:16.946677+0000","last_became_peered":"2024-05-21T21:51:16.946677+0000","last_unstale":"2024-05-21T21:52:10.874880+0000","last_undegraded":"2024-05-21T21:52:10.874880+0000","last_fullsized":"2024-05-21T21:52:10.874880+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T06:12:56.514284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'4","reported_seq":96,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:11.418160+0000","last_change":"2024-05-21T21:51:12.854873+0000","last_active":"2024-05-21T21:52:11.418160+0000","last_peered":"2024-05-21T21:52:11.418160+0000","last_clean":"2024-05-21T21:52:11.418160+0000","last_became_active":"2024-05-21T21:51:12.854393+0000","last_became_peered":"2024-05-21T21:51:12.854393+0000","last_unstale":"2024-05-21T21:52:11.418160+0000","last_undegraded":"2024-05-21T21:52:11.418160+0000","last_fullsized":"2024-05-21T21:52:11.418160+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:58:52.411164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":66,"num_read_kb":66,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-21T21:52:09.870477+0000","last_change":"2024-05-21T21:51:16.951094+0000","last_active":"2024-05-21T21:52:09.870477+0000","last_peered":"2024-05-21T21:52:09.870477+0000","last_clean":"2024-05-21T21:52:09.870477+0000","last_became_active":"2024-05-21T21:51:16.950724+0000","last_became_peered":"2024-05-21T21:51:16.950724+0000","last_unstale":"2024-05-21T21:52:09.870477+0000","last_undegraded":"2024-05-21T21:52:09.870477+0000","last_fullsized":"2024-05-21T21:52:09.870477+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:54:02.835426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'4","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-21T21:52:09.867386+0000","last_change":"2024-05-21T21:51:12.850427+0000","last_active":"2024-05-21T21:52:09.867386+0000","last_peered":"2024-05-21T21:52:09.867386+0000","last_clean":"2024-05-21T21:52:09.867386+0000","last_became_active":"2024-05-21T21:51:12.850096+0000","last_became_peered":"2024-05-21T21:51:12.850096+0000","last_unstale":"2024-05-21T21:52:09.867386+0000","last_undegraded":"2024-05-21T21:52:09.867386+0000","last_fullsized":"2024-05-21T21:52:09.867386+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:43:22.611043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.874050+0000","last_change":"2024-05-21T21:51:16.942330+0000","last_active":"2024-05-21T21:52:10.874050+0000","last_peered":"2024-05-21T21:52:10.874050+0000","last_clean":"2024-05-21T21:52:10.874050+0000","last_became_active":"2024-05-21T21:51:16.941718+0000","last_became_peered":"2024-05-21T21:51:16.941718+0000","last_unstale":"2024-05-21T21:52:10.874050+0000","last_undegraded":"2024-05-21T21:52:10.874050+0000","last_fullsized":"2024-05-21T21:52:10.874050+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:54:17.176411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'5","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.867578+0000","last_change":"2024-05-21T21:51:12.941100+0000","last_active":"2024-05-21T21:52:10.867578+0000","last_peered":"2024-05-21T21:52:10.867578+0000","last_clean":"2024-05-21T21:52:10.867578+0000","last_became_active":"2024-05-21T21:51:12.940597+0000","last_became_peered":"2024-05-21T21:51:12.940597+0000","last_unstale":"2024-05-21T21:52:10.867578+0000","last_undegraded":"2024-05-21T21:52:10.867578+0000","last_fullsized":"2024-05-21T21:52:10.867578+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:30:58.457108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-21T21:52:09.870315+0000","last_change":"2024-05-21T21:51:16.921973+0000","last_active":"2024-05-21T21:52:09.870315+0000","last_peered":"2024-05-21T21:52:09.870315+0000","last_clean":"2024-05-21T21:52:09.870315+0000","last_became_active":"2024-05-21T21:51:16.921597+0000","last_became_peered":"2024-05-21T21:51:16.921597+0000","last_unstale":"2024-05-21T21:52:09.870315+0000","last_undegraded":"2024-05-21T21:52:09.870315+0000","last_fullsized":"2024-05-21T21:52:09.870315+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:25:32.395940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'31","reported_seq":132,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:11.418085+0000","last_change":"2024-05-21T21:51:12.850312+0000","last_active":"2024-05-21T21:52:11.418085+0000","last_peered":"2024-05-21T21:52:11.418085+0000","last_clean":"2024-05-21T21:52:11.418085+0000","last_became_active":"2024-05-21T21:51:12.849837+0000","last_became_peered":"2024-05-21T21:51:12.849837+0000","last_unstale":"2024-05-21T21:52:11.418085+0000","last_undegraded":"2024-05-21T21:52:11.418085+0000","last_fullsized":"2024-05-21T21:52:11.418085+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T07:09:13.339554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59762,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":75,"num_read_kb":95,"num_write":33,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.863568+0000","last_change":"2024-05-21T21:51:16.926625+0000","last_active":"2024-05-21T21:52:10.863568+0000","last_peered":"2024-05-21T21:52:10.863568+0000","last_clean":"2024-05-21T21:52:10.863568+0000","last_became_active":"2024-05-21T21:51:16.925889+0000","last_became_peered":"2024-05-21T21:51:16.925889+0000","last_unstale":"2024-05-21T21:52:10.863568+0000","last_undegraded":"2024-05-21T21:52:10.863568+0000","last_fullsized":"2024-05-21T21:52:10.863568+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:35:23.464747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'9","reported_seq":35,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-21T21:52:09.865277+0000","last_change":"2024-05-21T21:51:12.859772+0000","last_active":"2024-05-21T21:52:09.865277+0000","last_peered":"2024-05-21T21:52:09.865277+0000","last_clean":"2024-05-21T21:52:09.865277+0000","last_became_active":"2024-05-21T21:51:12.859290+0000","last_became_peered":"2024-05-21T21:51:12.859290+0000","last_unstale":"2024-05-21T21:52:09.865277+0000","last_undegraded":"2024-05-21T21:52:09.865277+0000","last_fullsized":"2024-05-21T21:52:09.865277+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T05:28:29.953381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.861761+0000","last_change":"2024-05-21T21:51:16.925803+0000","last_active":"2024-05-21T21:52:10.861761+0000","last_peered":"2024-05-21T21:52:10.861761+0000","last_clean":"2024-05-21T21:52:10.861761+0000","last_became_active":"2024-05-21T21:51:16.925111+0000","last_became_peered":"2024-05-21T21:51:16.925111+0000","last_unstale":"2024-05-21T21:52:10.861761+0000","last_undegraded":"2024-05-21T21:52:10.861761+0000","last_fullsized":"2024-05-21T21:52:10.861761+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T06:22:11.353379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'5","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.867329+0000","last_change":"2024-05-21T21:51:12.859593+0000","last_active":"2024-05-21T21:52:10.867329+0000","last_peered":"2024-05-21T21:52:10.867329+0000","last_clean":"2024-05-21T21:52:10.867329+0000","last_became_active":"2024-05-21T21:51:12.859197+0000","last_became_peered":"2024-05-21T21:51:12.859197+0000","last_unstale":"2024-05-21T21:52:10.867329+0000","last_undegraded":"2024-05-21T21:52:10.867329+0000","last_fullsized":"2024-05-21T21:52:10.867329+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:13:12.311496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"peering","last_fresh":"2024-05-21T21:51:20.571325+0000","last_change":"2024-05-21T21:52:09.867385+0000","last_active":"2024-05-21T21:52:09.867385+0000","last_peered":"2024-05-21T21:51:20.571325+0000","last_clean":"2024-05-21T21:51:20.571325+0000","last_became_active":"2024-05-21T21:51:16.931709+0000","last_became_peered":"2024-05-21T21:51:16.931709+0000","last_unstale":"2024-05-21T21:51:20.571325+0000","last_undegraded":"2024-05-21T21:51:20.571325+0000","last_fullsized":"2024-05-21T21:51:20.571325+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,6,9],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'6","reported_seq":69,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:11.427457+0000","last_change":"2024-05-21T21:51:12.852169+0000","last_active":"2024-05-21T21:52:11.427457+0000","last_peered":"2024-05-21T21:52:11.427457+0000","last_clean":"2024-05-21T21:52:11.427457+0000","last_became_active":"2024-05-21T21:51:12.851782+0000","last_became_peered":"2024-05-21T21:51:12.851782+0000","last_unstale":"2024-05-21T21:52:11.427457+0000","last_undegraded":"2024-05-21T21:52:11.427457+0000","last_fullsized":"2024-05-21T21:52:11.427457+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:46:09.692158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":38,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-21T21:52:09.866991+0000","last_change":"2024-05-21T21:51:16.929264+0000","last_active":"2024-05-21T21:52:09.866991+0000","last_peered":"2024-05-21T21:52:09.866991+0000","last_clean":"2024-05-21T21:52:09.866991+0000","last_became_active":"2024-05-21T21:51:16.928900+0000","last_became_peered":"2024-05-21T21:51:16.928900+0000","last_unstale":"2024-05-21T21:52:09.866991+0000","last_undegraded":"2024-05-21T21:52:09.866991+0000","last_fullsized":"2024-05-21T21:52:09.866991+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:42:28.040939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'17","reported_seq":68,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.984451+0000","last_change":"2024-05-21T21:52:10.984451+0000","last_active":"2024-05-21T21:52:10.984451+0000","last_peered":"2024-05-21T21:52:10.984451+0000","last_clean":"2024-05-21T21:52:10.984451+0000","last_became_active":"2024-05-21T21:52:10.974050+0000","last_became_peered":"2024-05-21T21:52:10.974050+0000","last_unstale":"2024-05-21T21:52:10.984451+0000","last_undegraded":"2024-05-21T21:52:10.984451+0000","last_fullsized":"2024-05-21T21:52:10.984451+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T07:08:18.307022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.867149+0000","last_change":"2024-05-21T21:51:16.935464+0000","last_active":"2024-05-21T21:52:10.867149+0000","last_peered":"2024-05-21T21:52:10.867149+0000","last_clean":"2024-05-21T21:52:10.867149+0000","last_became_active":"2024-05-21T21:51:16.935091+0000","last_became_peered":"2024-05-21T21:51:16.935091+0000","last_unstale":"2024-05-21T21:52:10.867149+0000","last_undegraded":"2024-05-21T21:52:10.867149+0000","last_fullsized":"2024-05-21T21:52:10.867149+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T06:59:16.330649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.864530+0000","last_change":"2024-05-21T21:51:12.853327+0000","last_active":"2024-05-21T21:52:10.864530+0000","last_peered":"2024-05-21T21:52:10.864530+0000","last_clean":"2024-05-21T21:52:10.864530+0000","last_became_active":"2024-05-21T21:51:12.852946+0000","last_became_peered":"2024-05-21T21:51:12.852946+0000","last_unstale":"2024-05-21T21:52:10.864530+0000","last_undegraded":"2024-05-21T21:52:10.864530+0000","last_fullsized":"2024-05-21T21:52:10.864530+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:00:34.256858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.861583+0000","last_change":"2024-05-21T21:51:16.929341+0000","last_active":"2024-05-21T21:52:10.861583+0000","last_peered":"2024-05-21T21:52:10.861583+0000","last_clean":"2024-05-21T21:52:10.861583+0000","last_became_active":"2024-05-21T21:51:16.929008+0000","last_became_peered":"2024-05-21T21:51:16.929008+0000","last_unstale":"2024-05-21T21:52:10.861583+0000","last_undegraded":"2024-05-21T21:52:10.861583+0000","last_fullsized":"2024-05-21T21:52:10.861583+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:26:26.297136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"80'1","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.875960+0000","last_change":"2024-05-21T21:51:12.945433+0000","last_active":"2024-05-21T21:52:10.875960+0000","last_peered":"2024-05-21T21:52:10.875960+0000","last_clean":"2024-05-21T21:52:10.875960+0000","last_became_active":"2024-05-21T21:51:12.944463+0000","last_became_peered":"2024-05-21T21:51:12.944463+0000","last_unstale":"2024-05-21T21:52:10.875960+0000","last_undegraded":"2024-05-21T21:52:10.875960+0000","last_fullsized":"2024-05-21T21:52:10.875960+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:29:52.014583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'1","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.883554+0000","last_change":"2024-05-21T21:52:10.883554+0000","last_active":"2024-05-21T21:52:10.883554+0000","last_peered":"2024-05-21T21:52:10.883554+0000","last_clean":"2024-05-21T21:52:10.883554+0000","last_became_active":"2024-05-21T21:52:10.877789+0000","last_became_peered":"2024-05-21T21:52:10.877789+0000","last_unstale":"2024-05-21T21:52:10.883554+0000","last_undegraded":"2024-05-21T21:52:10.883554+0000","last_fullsized":"2024-05-21T21:52:10.883554+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:52:37.623551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"80'5","reported_seq":62,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-21T21:52:09.867234+0000","last_change":"2024-05-21T21:51:12.852883+0000","last_active":"2024-05-21T21:52:09.867234+0000","last_peered":"2024-05-21T21:52:09.867234+0000","last_clean":"2024-05-21T21:52:09.867234+0000","last_became_active":"2024-05-21T21:51:12.852549+0000","last_became_peered":"2024-05-21T21:51:12.852549+0000","last_unstale":"2024-05-21T21:52:09.867234+0000","last_undegraded":"2024-05-21T21:52:09.867234+0000","last_fullsized":"2024-05-21T21:52:09.867234+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:05:53.002286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":33,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.865777+0000","last_change":"2024-05-21T21:51:16.944425+0000","last_active":"2024-05-21T21:52:10.865777+0000","last_peered":"2024-05-21T21:52:10.865777+0000","last_clean":"2024-05-21T21:52:10.865777+0000","last_became_active":"2024-05-21T21:51:16.944033+0000","last_became_peered":"2024-05-21T21:51:16.944033+0000","last_unstale":"2024-05-21T21:52:10.865777+0000","last_undegraded":"2024-05-21T21:52:10.865777+0000","last_fullsized":"2024-05-21T21:52:10.865777+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T07:16:26.802329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.866923+0000","last_change":"2024-05-21T21:51:16.933264+0000","last_active":"2024-05-21T21:52:10.866923+0000","last_peered":"2024-05-21T21:52:10.866923+0000","last_clean":"2024-05-21T21:52:10.866923+0000","last_became_active":"2024-05-21T21:51:16.932911+0000","last_became_peered":"2024-05-21T21:51:16.932911+0000","last_unstale":"2024-05-21T21:52:10.866923+0000","last_undegraded":"2024-05-21T21:52:10.866923+0000","last_fullsized":"2024-05-21T21:52:10.866923+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:25:54.006367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"80'5","reported_seq":68,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:13.429840+0000","last_change":"2024-05-21T21:51:12.942900+0000","last_active":"2024-05-21T21:52:13.429840+0000","last_peered":"2024-05-21T21:52:13.429840+0000","last_clean":"2024-05-21T21:52:13.429840+0000","last_became_active":"2024-05-21T21:51:12.941038+0000","last_became_peered":"2024-05-21T21:51:12.941038+0000","last_unstale":"2024-05-21T21:52:13.429840+0000","last_undegraded":"2024-05-21T21:52:13.429840+0000","last_fullsized":"2024-05-21T21:52:13.429840+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:38:43.240011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":37,"num_read_kb":37,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.867131+0000","last_change":"2024-05-21T21:51:16.933391+0000","last_active":"2024-05-21T21:52:10.867131+0000","last_peered":"2024-05-21T21:52:10.867131+0000","last_clean":"2024-05-21T21:52:10.867131+0000","last_became_active":"2024-05-21T21:51:16.932320+0000","last_became_peered":"2024-05-21T21:51:16.932320+0000","last_unstale":"2024-05-21T21:52:10.867131+0000","last_undegraded":"2024-05-21T21:52:10.867131+0000","last_fullsized":"2024-05-21T21:52:10.867131+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T09:06:59.126748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'6","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.865206+0000","last_change":"2024-05-21T21:51:12.856342+0000","last_active":"2024-05-21T21:52:10.865206+0000","last_peered":"2024-05-21T21:52:10.865206+0000","last_clean":"2024-05-21T21:52:10.865206+0000","last_became_active":"2024-05-21T21:51:12.855952+0000","last_became_peered":"2024-05-21T21:51:12.855952+0000","last_unstale":"2024-05-21T21:52:10.865206+0000","last_undegraded":"2024-05-21T21:52:10.865206+0000","last_fullsized":"2024-05-21T21:52:10.865206+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:34:40.734034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":97,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-21T21:52:09.867081+0000","last_change":"2024-05-21T21:49:42.797187+0000","last_active":"2024-05-21T21:52:09.867081+0000","last_peered":"2024-05-21T21:52:09.867081+0000","last_clean":"2024-05-21T21:52:09.867081+0000","last_became_active":"2024-05-21T21:49:42.791291+0000","last_became_peered":"2024-05-21T21:49:42.791291+0000","last_unstale":"2024-05-21T21:52:09.867081+0000","last_undegraded":"2024-05-21T21:52:09.867081+0000","last_fullsized":"2024-05-21T21:52:09.867081+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:45:55.552293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:45:55.552293+0000","last_clean_scrub_stamp":"2024-05-21T21:45:55.552293+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T07:24:18.302523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.861715+0000","last_change":"2024-05-21T21:51:16.926446+0000","last_active":"2024-05-21T21:52:10.861715+0000","last_peered":"2024-05-21T21:52:10.861715+0000","last_clean":"2024-05-21T21:52:10.861715+0000","last_became_active":"2024-05-21T21:51:16.926070+0000","last_became_peered":"2024-05-21T21:51:16.926070+0000","last_unstale":"2024-05-21T21:52:10.861715+0000","last_undegraded":"2024-05-21T21:52:10.861715+0000","last_fullsized":"2024-05-21T21:52:10.861715+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:44:27.366393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'4","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-21T21:52:09.867193+0000","last_change":"2024-05-21T21:51:12.854371+0000","last_active":"2024-05-21T21:52:09.867193+0000","last_peered":"2024-05-21T21:52:09.867193+0000","last_clean":"2024-05-21T21:52:09.867193+0000","last_became_active":"2024-05-21T21:51:12.853847+0000","last_became_peered":"2024-05-21T21:51:12.853847+0000","last_unstale":"2024-05-21T21:52:09.867193+0000","last_undegraded":"2024-05-21T21:52:09.867193+0000","last_fullsized":"2024-05-21T21:52:09.867193+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:14:10.549901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"80'4","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.863685+0000","last_change":"2024-05-21T21:51:12.941227+0000","last_active":"2024-05-21T21:52:10.863685+0000","last_peered":"2024-05-21T21:52:10.863685+0000","last_clean":"2024-05-21T21:52:10.863685+0000","last_became_active":"2024-05-21T21:51:12.940702+0000","last_became_peered":"2024-05-21T21:51:12.940702+0000","last_unstale":"2024-05-21T21:52:10.863685+0000","last_undegraded":"2024-05-21T21:52:10.863685+0000","last_fullsized":"2024-05-21T21:52:10.863685+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:40:34.124921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.862247+0000","last_change":"2024-05-21T21:51:16.945627+0000","last_active":"2024-05-21T21:52:10.862247+0000","last_peered":"2024-05-21T21:52:10.862247+0000","last_clean":"2024-05-21T21:52:10.862247+0000","last_became_active":"2024-05-21T21:51:16.945297+0000","last_became_peered":"2024-05-21T21:51:16.945297+0000","last_unstale":"2024-05-21T21:52:10.862247+0000","last_undegraded":"2024-05-21T21:52:10.862247+0000","last_fullsized":"2024-05-21T21:52:10.862247+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:54:38.898774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'15","reported_seq":57,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.888076+0000","last_change":"2024-05-21T21:52:10.888076+0000","last_active":"2024-05-21T21:52:10.888076+0000","last_peered":"2024-05-21T21:52:10.888076+0000","last_clean":"2024-05-21T21:52:10.888076+0000","last_became_active":"2024-05-21T21:52:10.879368+0000","last_became_peered":"2024-05-21T21:52:10.879368+0000","last_unstale":"2024-05-21T21:52:10.888076+0000","last_undegraded":"2024-05-21T21:52:10.888076+0000","last_fullsized":"2024-05-21T21:52:10.888076+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:46:43.190411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.868811+0000","last_change":"2024-05-21T21:51:16.929807+0000","last_active":"2024-05-21T21:52:10.868811+0000","last_peered":"2024-05-21T21:52:10.868811+0000","last_clean":"2024-05-21T21:52:10.868811+0000","last_became_active":"2024-05-21T21:51:16.928859+0000","last_became_peered":"2024-05-21T21:51:16.928859+0000","last_unstale":"2024-05-21T21:52:10.868811+0000","last_undegraded":"2024-05-21T21:52:10.868811+0000","last_fullsized":"2024-05-21T21:52:10.868811+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:41:32.830892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'7","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.865597+0000","last_change":"2024-05-21T21:51:12.850843+0000","last_active":"2024-05-21T21:52:10.865597+0000","last_peered":"2024-05-21T21:52:10.865597+0000","last_clean":"2024-05-21T21:52:10.865597+0000","last_became_active":"2024-05-21T21:51:12.850516+0000","last_became_peered":"2024-05-21T21:51:12.850516+0000","last_unstale":"2024-05-21T21:52:10.865597+0000","last_undegraded":"2024-05-21T21:52:10.865597+0000","last_fullsized":"2024-05-21T21:52:10.865597+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:37:05.191626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.873845+0000","last_change":"2024-05-21T21:51:16.949017+0000","last_active":"2024-05-21T21:52:10.873845+0000","last_peered":"2024-05-21T21:52:10.873845+0000","last_clean":"2024-05-21T21:52:10.873845+0000","last_became_active":"2024-05-21T21:51:16.948691+0000","last_became_peered":"2024-05-21T21:51:16.948691+0000","last_unstale":"2024-05-21T21:52:10.873845+0000","last_undegraded":"2024-05-21T21:52:10.873845+0000","last_fullsized":"2024-05-21T21:52:10.873845+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:32:06.606581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-21T21:52:09.870043+0000","last_change":"2024-05-21T21:51:12.939203+0000","last_active":"2024-05-21T21:52:09.870043+0000","last_peered":"2024-05-21T21:52:09.870043+0000","last_clean":"2024-05-21T21:52:09.870043+0000","last_became_active":"2024-05-21T21:51:12.938936+0000","last_became_peered":"2024-05-21T21:51:12.938936+0000","last_unstale":"2024-05-21T21:52:09.870043+0000","last_undegraded":"2024-05-21T21:52:09.870043+0000","last_fullsized":"2024-05-21T21:52:09.870043+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:55:50.336362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.863329+0000","last_change":"2024-05-21T21:51:16.926216+0000","last_active":"2024-05-21T21:52:10.863329+0000","last_peered":"2024-05-21T21:52:10.863329+0000","last_clean":"2024-05-21T21:52:10.863329+0000","last_became_active":"2024-05-21T21:51:16.925834+0000","last_became_peered":"2024-05-21T21:51:16.925834+0000","last_unstale":"2024-05-21T21:52:10.863329+0000","last_undegraded":"2024-05-21T21:52:10.863329+0000","last_fullsized":"2024-05-21T21:52:10.863329+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:13:30.192908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'7","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.866548+0000","last_change":"2024-05-21T21:51:12.855306+0000","last_active":"2024-05-21T21:52:10.866548+0000","last_peered":"2024-05-21T21:52:10.866548+0000","last_clean":"2024-05-21T21:52:10.866548+0000","last_became_active":"2024-05-21T21:51:12.854722+0000","last_became_peered":"2024-05-21T21:51:12.854722+0000","last_unstale":"2024-05-21T21:52:10.866548+0000","last_undegraded":"2024-05-21T21:52:10.866548+0000","last_fullsized":"2024-05-21T21:52:10.866548+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:39:41.583219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.861450+0000","last_change":"2024-05-21T21:51:16.944758+0000","last_active":"2024-05-21T21:52:10.861450+0000","last_peered":"2024-05-21T21:52:10.861450+0000","last_clean":"2024-05-21T21:52:10.861450+0000","last_became_active":"2024-05-21T21:51:16.944393+0000","last_became_peered":"2024-05-21T21:51:16.944393+0000","last_unstale":"2024-05-21T21:52:10.861450+0000","last_undegraded":"2024-05-21T21:52:10.861450+0000","last_fullsized":"2024-05-21T21:52:10.861450+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:01:23.048355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'3","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-21T21:52:09.865240+0000","last_change":"2024-05-21T21:51:12.860378+0000","last_active":"2024-05-21T21:52:09.865240+0000","last_peered":"2024-05-21T21:52:09.865240+0000","last_clean":"2024-05-21T21:52:09.865240+0000","last_became_active":"2024-05-21T21:51:12.859978+0000","last_became_peered":"2024-05-21T21:51:12.859978+0000","last_unstale":"2024-05-21T21:52:09.865240+0000","last_undegraded":"2024-05-21T21:52:09.865240+0000","last_fullsized":"2024-05-21T21:52:09.865240+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T07:52:26.902232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.867609+0000","last_change":"2024-05-21T21:51:16.943194+0000","last_active":"2024-05-21T21:52:10.867609+0000","last_peered":"2024-05-21T21:52:10.867609+0000","last_clean":"2024-05-21T21:52:10.867609+0000","last_became_active":"2024-05-21T21:51:16.942884+0000","last_became_peered":"2024-05-21T21:51:16.942884+0000","last_unstale":"2024-05-21T21:52:10.867609+0000","last_undegraded":"2024-05-21T21:52:10.867609+0000","last_fullsized":"2024-05-21T21:52:10.867609+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T05:58:17.436995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.863880+0000","last_change":"2024-05-21T21:51:12.853390+0000","last_active":"2024-05-21T21:52:10.863880+0000","last_peered":"2024-05-21T21:52:10.863880+0000","last_clean":"2024-05-21T21:52:10.863880+0000","last_became_active":"2024-05-21T21:51:12.852990+0000","last_became_peered":"2024-05-21T21:51:12.852990+0000","last_unstale":"2024-05-21T21:52:10.863880+0000","last_undegraded":"2024-05-21T21:52:10.863880+0000","last_fullsized":"2024-05-21T21:52:10.863880+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:32:16.392609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-21T21:52:09.867247+0000","last_change":"2024-05-21T21:51:16.933430+0000","last_active":"2024-05-21T21:52:09.867247+0000","last_peered":"2024-05-21T21:52:09.867247+0000","last_clean":"2024-05-21T21:52:09.867247+0000","last_became_active":"2024-05-21T21:51:16.933053+0000","last_became_peered":"2024-05-21T21:51:16.933053+0000","last_unstale":"2024-05-21T21:52:09.867247+0000","last_undegraded":"2024-05-21T21:52:09.867247+0000","last_fullsized":"2024-05-21T21:52:09.867247+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T06:14:57.468727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'5","reported_seq":63,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-21T21:52:09.865363+0000","last_change":"2024-05-21T21:51:12.946113+0000","last_active":"2024-05-21T21:52:09.865363+0000","last_peered":"2024-05-21T21:52:09.865363+0000","last_clean":"2024-05-21T21:52:09.865363+0000","last_became_active":"2024-05-21T21:51:12.945650+0000","last_became_peered":"2024-05-21T21:51:12.945650+0000","last_unstale":"2024-05-21T21:52:09.865363+0000","last_undegraded":"2024-05-21T21:52:09.865363+0000","last_fullsized":"2024-05-21T21:52:09.865363+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T09:40:18.209400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":34,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-21T21:52:09.867452+0000","last_change":"2024-05-21T21:51:16.932632+0000","last_active":"2024-05-21T21:52:09.867452+0000","last_peered":"2024-05-21T21:52:09.867452+0000","last_clean":"2024-05-21T21:52:09.867452+0000","last_became_active":"2024-05-21T21:51:16.932256+0000","last_became_peered":"2024-05-21T21:51:16.932256+0000","last_unstale":"2024-05-21T21:52:09.867452+0000","last_undegraded":"2024-05-21T21:52:09.867452+0000","last_fullsized":"2024-05-21T21:52:09.867452+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:04:48.146485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'3","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.867556+0000","last_change":"2024-05-21T21:51:12.860602+0000","last_active":"2024-05-21T21:52:10.867556+0000","last_peered":"2024-05-21T21:52:10.867556+0000","last_clean":"2024-05-21T21:52:10.867556+0000","last_became_active":"2024-05-21T21:51:12.860228+0000","last_became_peered":"2024-05-21T21:51:12.860228+0000","last_unstale":"2024-05-21T21:52:10.867556+0000","last_undegraded":"2024-05-21T21:52:10.867556+0000","last_fullsized":"2024-05-21T21:52:10.867556+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:33:39.230872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'1","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-21T21:52:09.867258+0000","last_change":"2024-05-21T21:51:16.929616+0000","last_active":"2024-05-21T21:52:09.867258+0000","last_peered":"2024-05-21T21:52:09.867258+0000","last_clean":"2024-05-21T21:52:09.867258+0000","last_became_active":"2024-05-21T21:51:16.928569+0000","last_became_peered":"2024-05-21T21:51:16.928569+0000","last_unstale":"2024-05-21T21:52:09.867258+0000","last_undegraded":"2024-05-21T21:52:09.867258+0000","last_fullsized":"2024-05-21T21:52:09.867258+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:15:41.790344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'5","reported_seq":43,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.881114+0000","last_change":"2024-05-21T21:52:10.881114+0000","last_active":"2024-05-21T21:52:10.881114+0000","last_peered":"2024-05-21T21:52:10.881114+0000","last_clean":"2024-05-21T21:52:10.881114+0000","last_became_active":"2024-05-21T21:52:10.872022+0000","last_became_peered":"2024-05-21T21:52:10.872022+0000","last_unstale":"2024-05-21T21:52:10.881114+0000","last_undegraded":"2024-05-21T21:52:10.881114+0000","last_fullsized":"2024-05-21T21:52:10.881114+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T06:05:51.419156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-21T21:52:09.867474+0000","last_change":"2024-05-21T21:51:16.926496+0000","last_active":"2024-05-21T21:52:09.867474+0000","last_peered":"2024-05-21T21:52:09.867474+0000","last_clean":"2024-05-21T21:52:09.867474+0000","last_became_active":"2024-05-21T21:51:16.926112+0000","last_became_peered":"2024-05-21T21:51:16.926112+0000","last_unstale":"2024-05-21T21:52:09.867474+0000","last_undegraded":"2024-05-21T21:52:09.867474+0000","last_fullsized":"2024-05-21T21:52:09.867474+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:26:08.254421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.861508+0000","last_change":"2024-05-21T21:51:16.930522+0000","last_active":"2024-05-21T21:52:10.861508+0000","last_peered":"2024-05-21T21:52:10.861508+0000","last_clean":"2024-05-21T21:52:10.861508+0000","last_became_active":"2024-05-21T21:51:16.929853+0000","last_became_peered":"2024-05-21T21:51:16.929853+0000","last_unstale":"2024-05-21T21:52:10.861508+0000","last_undegraded":"2024-05-21T21:52:10.861508+0000","last_fullsized":"2024-05-21T21:52:10.861508+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:10:57.940009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-21T21:52:09.865072+0000","last_change":"2024-05-21T21:51:16.931751+0000","last_active":"2024-05-21T21:52:09.865072+0000","last_peered":"2024-05-21T21:52:09.865072+0000","last_clean":"2024-05-21T21:52:09.865072+0000","last_became_active":"2024-05-21T21:51:16.931353+0000","last_became_peered":"2024-05-21T21:51:16.931353+0000","last_unstale":"2024-05-21T21:52:09.865072+0000","last_undegraded":"2024-05-21T21:52:09.865072+0000","last_fullsized":"2024-05-21T21:52:09.865072+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:33:45.297850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-21T21:52:09.865073+0000","last_change":"2024-05-21T21:51:16.940096+0000","last_active":"2024-05-21T21:52:09.865073+0000","last_peered":"2024-05-21T21:52:09.865073+0000","last_clean":"2024-05-21T21:52:09.865073+0000","last_became_active":"2024-05-21T21:51:16.939740+0000","last_became_peered":"2024-05-21T21:51:16.939740+0000","last_unstale":"2024-05-21T21:52:09.865073+0000","last_undegraded":"2024-05-21T21:52:09.865073+0000","last_fullsized":"2024-05-21T21:52:09.865073+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:59:52.569526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.872739+0000","last_change":"2024-05-21T21:51:16.946488+0000","last_active":"2024-05-21T21:52:10.872739+0000","last_peered":"2024-05-21T21:52:10.872739+0000","last_clean":"2024-05-21T21:52:10.872739+0000","last_became_active":"2024-05-21T21:51:16.945957+0000","last_became_peered":"2024-05-21T21:51:16.945957+0000","last_unstale":"2024-05-21T21:52:10.872739+0000","last_undegraded":"2024-05-21T21:52:10.872739+0000","last_fullsized":"2024-05-21T21:52:10.872739+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:40:36.708344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.862107+0000","last_change":"2024-05-21T21:51:16.949349+0000","last_active":"2024-05-21T21:52:10.862107+0000","last_peered":"2024-05-21T21:52:10.862107+0000","last_clean":"2024-05-21T21:52:10.862107+0000","last_became_active":"2024-05-21T21:51:16.946986+0000","last_became_peered":"2024-05-21T21:51:16.946986+0000","last_unstale":"2024-05-21T21:52:10.862107+0000","last_undegraded":"2024-05-21T21:52:10.862107+0000","last_fullsized":"2024-05-21T21:52:10.862107+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:01:14.877110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.861685+0000","last_change":"2024-05-21T21:51:16.929172+0000","last_active":"2024-05-21T21:52:10.861685+0000","last_peered":"2024-05-21T21:52:10.861685+0000","last_clean":"2024-05-21T21:52:10.861685+0000","last_became_active":"2024-05-21T21:51:16.928823+0000","last_became_peered":"2024-05-21T21:51:16.928823+0000","last_unstale":"2024-05-21T21:52:10.861685+0000","last_undegraded":"2024-05-21T21:52:10.861685+0000","last_fullsized":"2024-05-21T21:52:10.861685+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:09:13.719272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-21T21:52:09.865587+0000","last_change":"2024-05-21T21:51:16.930332+0000","last_active":"2024-05-21T21:52:09.865587+0000","last_peered":"2024-05-21T21:52:09.865587+0000","last_clean":"2024-05-21T21:52:09.865587+0000","last_became_active":"2024-05-21T21:51:16.930024+0000","last_became_peered":"2024-05-21T21:51:16.930024+0000","last_unstale":"2024-05-21T21:52:09.865587+0000","last_undegraded":"2024-05-21T21:52:09.865587+0000","last_fullsized":"2024-05-21T21:52:09.865587+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:47:41.272725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.868573+0000","last_change":"2024-05-21T21:51:16.930102+0000","last_active":"2024-05-21T21:52:10.868573+0000","last_peered":"2024-05-21T21:52:10.868573+0000","last_clean":"2024-05-21T21:52:10.868573+0000","last_became_active":"2024-05-21T21:51:16.929410+0000","last_became_peered":"2024-05-21T21:51:16.929410+0000","last_unstale":"2024-05-21T21:52:10.868573+0000","last_undegraded":"2024-05-21T21:52:10.868573+0000","last_fullsized":"2024-05-21T21:52:10.868573+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:11:46.296288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.873829+0000","last_change":"2024-05-21T21:52:10.873424+0000","last_active":"2024-05-21T21:52:10.873829+0000","last_peered":"2024-05-21T21:52:10.873829+0000","last_clean":"2024-05-21T21:52:10.873829+0000","last_became_active":"2024-05-21T21:52:10.873118+0000","last_became_peered":"2024-05-21T21:52:10.873118+0000","last_unstale":"2024-05-21T21:52:10.873829+0000","last_undegraded":"2024-05-21T21:52:10.873829+0000","last_fullsized":"2024-05-21T21:52:10.873829+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:05:00.731013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,3],"acting":[8,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.866413+0000","last_change":"2024-05-21T21:51:16.934442+0000","last_active":"2024-05-21T21:52:10.866413+0000","last_peered":"2024-05-21T21:52:10.866413+0000","last_clean":"2024-05-21T21:52:10.866413+0000","last_became_active":"2024-05-21T21:51:16.934086+0000","last_became_peered":"2024-05-21T21:51:16.934086+0000","last_unstale":"2024-05-21T21:52:10.866413+0000","last_undegraded":"2024-05-21T21:52:10.866413+0000","last_fullsized":"2024-05-21T21:52:10.866413+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:36:44.302532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-21T21:52:09.870214+0000","last_change":"2024-05-21T21:51:16.924479+0000","last_active":"2024-05-21T21:52:09.870214+0000","last_peered":"2024-05-21T21:52:09.870214+0000","last_clean":"2024-05-21T21:52:09.870214+0000","last_became_active":"2024-05-21T21:51:16.923998+0000","last_became_peered":"2024-05-21T21:51:16.923998+0000","last_unstale":"2024-05-21T21:52:09.870214+0000","last_undegraded":"2024-05-21T21:52:09.870214+0000","last_fullsized":"2024-05-21T21:52:09.870214+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:23:07.931731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.862486+0000","last_change":"2024-05-21T21:51:16.947271+0000","last_active":"2024-05-21T21:52:10.862486+0000","last_peered":"2024-05-21T21:52:10.862486+0000","last_clean":"2024-05-21T21:52:10.862486+0000","last_became_active":"2024-05-21T21:51:16.946912+0000","last_became_peered":"2024-05-21T21:51:16.946912+0000","last_unstale":"2024-05-21T21:52:10.862486+0000","last_undegraded":"2024-05-21T21:52:10.862486+0000","last_fullsized":"2024-05-21T21:52:10.862486+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:51:29.646915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.861507+0000","last_change":"2024-05-21T21:51:16.928852+0000","last_active":"2024-05-21T21:52:10.861507+0000","last_peered":"2024-05-21T21:52:10.861507+0000","last_clean":"2024-05-21T21:52:10.861507+0000","last_became_active":"2024-05-21T21:51:16.928480+0000","last_became_peered":"2024-05-21T21:51:16.928480+0000","last_unstale":"2024-05-21T21:52:10.861507+0000","last_undegraded":"2024-05-21T21:52:10.861507+0000","last_fullsized":"2024-05-21T21:52:10.861507+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T09:12:30.691254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66966,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":375,"num_read_kb":263,"num_write":245,"num_write_kb":135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":112,"num_keys_recovered":17,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":200898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":210,"ondisk_log_size":210,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645340,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":884736,"data_stored":425463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841568,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28588,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716564,"statfs":{"total":95995035648,"available":95965761536,"internally_reserved":0,"allocated":1601536,"data_stored":1085141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005094,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":929792,"data_stored":434006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201324,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":921600,"data_stored":434267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397552,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28488,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716664,"statfs":{"total":95995035648,"available":95965863936,"internally_reserved":0,"allocated":1503232,"data_stored":1018680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561079,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":909312,"data_stored":433606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574930,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":954368,"data_stored":485785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738456,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36252,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708900,"statfs":{"total":95995035648,"available":95957913600,"internally_reserved":0,"allocated":929792,"data_stored":434945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411406,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36292,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708860,"statfs":{"total":95995035648,"available":95957872640,"internally_reserved":0,"allocated":974848,"data_stored":486396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149767,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36244,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708908,"statfs":{"total":95995035648,"available":95957921792,"internally_reserved":0,"allocated":921600,"data_stored":433314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953538,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":909312,"data_stored":433858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757309,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32676,"kb_used_data":1496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712476,"statfs":{"total":95995035648,"available":95961575424,"internally_reserved":0,"allocated":1531904,"data_stored":1026463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":61514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":61889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-21T21:52:17.077 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph pg dump --format=json 2024-05-21T21:52:17.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:52:16 smithi184 ceph-mon[36942]: pgmap v345: 97 pgs: 1 peering, 96 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 639 B/s wr, 8 op/s; 28 B/s, 2 keys/s, 2 objects/s recovering 2024-05-21T21:52:17.285 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:52:16 smithi164 ceph-mon[36269]: pgmap v345: 97 pgs: 1 peering, 96 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 639 B/s wr, 8 op/s; 28 B/s, 2 keys/s, 2 objects/s recovering 2024-05-21T21:52:17.576 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:52:18.054 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:52:17 smithi151 ceph-mon[31623]: from='client.24643 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T21:52:18.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:52:17 smithi184 ceph-mon[36942]: from='client.24643 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T21:52:18.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:52:17 smithi164 ceph-mon[36269]: from='client.24643 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T21:52:19.202 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:52:18 smithi151 ceph-mon[31623]: pgmap v346: 97 pgs: 97 active+clean; 642 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 639 B/s wr, 11 op/s; 28 B/s, 2 keys/s, 2 objects/s recovering 2024-05-21T21:52:19.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:52:18 smithi184 ceph-mon[36942]: pgmap v346: 97 pgs: 97 active+clean; 642 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 639 B/s wr, 11 op/s; 28 B/s, 2 keys/s, 2 objects/s recovering 2024-05-21T21:52:19.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:52:18 smithi164 ceph-mon[36269]: pgmap v346: 97 pgs: 97 active+clean; 642 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 639 B/s wr, 11 op/s; 28 B/s, 2 keys/s, 2 objects/s recovering 2024-05-21T21:52:19.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:19.781+0000 7f99a1600700 1 -- 172.21.15.151:0/3728153472 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f999c076090 msgr2=0x7f999c10fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:52:19.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:19.781+0000 7f99a1600700 1 --2- 172.21.15.151:0/3728153472 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f999c076090 0x7f999c10fa00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f99880098f0 tx=0x7f9988009c00 comp rx=0 tx=0).stop 2024-05-21T21:52:19.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:19.781+0000 7f99a1600700 1 -- 172.21.15.151:0/3728153472 shutdown_connections 2024-05-21T21:52:19.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:19.781+0000 7f99a1600700 1 --2- 172.21.15.151:0/3728153472 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f999c10ff40 0x7f999c112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:19.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:19.781+0000 7f99a1600700 1 --2- 172.21.15.151:0/3728153472 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f999c075770 0x7f999c075b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:19.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:19.781+0000 7f99a1600700 1 --2- 172.21.15.151:0/3728153472 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f999c076090 0x7f999c10fa00 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:19.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:19.781+0000 7f99a1600700 1 -- 172.21.15.151:0/3728153472 >> 172.21.15.151:0/3728153472 conn(0x7f999c07a6a0 msgr2=0x7f999c07aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:52:19.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:19.781+0000 7f99a1600700 1 -- 172.21.15.151:0/3728153472 shutdown_connections 2024-05-21T21:52:19.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:19.782+0000 7f99a1600700 1 -- 172.21.15.151:0/3728153472 wait complete. 2024-05-21T21:52:19.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:19.783+0000 7f99a1600700 1 Processor -- start 2024-05-21T21:52:19.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:19.783+0000 7f99a1600700 1 -- start start 2024-05-21T21:52:19.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:19.783+0000 7f99a1600700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f999c075770 0x7f999c19be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:52:19.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:19.783+0000 7f99a1600700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f999c076090 0x7f999c19c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:52:19.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:19.784+0000 7f99a1600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f999c10ff40 0x7f999c1a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:52:19.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:19.784+0000 7f99a1600700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f999c115920 con 0x7f999c10ff40 2024-05-21T21:52:19.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:19.784+0000 7f99a1600700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f999c1157a0 con 0x7f999c075770 2024-05-21T21:52:19.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:19.784+0000 7f99a1600700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f999c115aa0 con 0x7f999c076090 2024-05-21T21:52:19.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:19.784+0000 7f999aa00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f999c075770 0x7f999c19be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:52:19.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:19.784+0000 7f999aa00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f999c075770 0x7f999c19be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:46016/0 (socket says 172.21.15.151:46016) 2024-05-21T21:52:19.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:19.784+0000 7f999aa00700 1 -- 172.21.15.151:0/148202045 learned_addr learned my addr 172.21.15.151:0/148202045 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:52:19.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:19.784+0000 7f999a000700 1 --2- 172.21.15.151:0/148202045 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f999c076090 0x7f999c19c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:52:19.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:19.784+0000 7f999b400700 1 --2- 172.21.15.151:0/148202045 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f999c10ff40 0x7f999c1a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:52:19.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:19.784+0000 7f999aa00700 1 -- 172.21.15.151:0/148202045 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f999c076090 msgr2=0x7f999c19c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:52:19.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:19.784+0000 7f999aa00700 1 --2- 172.21.15.151:0/148202045 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f999c076090 0x7f999c19c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:19.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:19.784+0000 7f999aa00700 1 -- 172.21.15.151:0/148202045 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f999c10ff40 msgr2=0x7f999c1a06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:52:19.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:19.784+0000 7f999aa00700 1 --2- 172.21.15.151:0/148202045 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f999c10ff40 0x7f999c1a06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:19.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:19.784+0000 7f999aa00700 1 -- 172.21.15.151:0/148202045 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f999c1a0e30 con 0x7f999c075770 2024-05-21T21:52:19.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:19.784+0000 7f999b400700 1 --2- 172.21.15.151:0/148202045 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f999c10ff40 0x7f999c1a06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T21:52:19.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:19.784+0000 7f999a000700 1 --2- 172.21.15.151:0/148202045 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f999c076090 0x7f999c19c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:52:19.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:19.785+0000 7f999aa00700 1 --2- 172.21.15.151:0/148202045 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f999c075770 0x7f999c19be70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f998c00e9c0 tx=0x7f998c00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:52:19.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:19.785+0000 7f9993400700 1 -- 172.21.15.151:0/148202045 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f998c00cbd0 con 0x7f999c075770 2024-05-21T21:52:19.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:19.785+0000 7f9993400700 1 -- 172.21.15.151:0/148202045 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f998c00cd70 con 0x7f999c075770 2024-05-21T21:52:19.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:19.785+0000 7f99a1600700 1 -- 172.21.15.151:0/148202045 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f999c1a1120 con 0x7f999c075770 2024-05-21T21:52:19.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:19.785+0000 7f9993400700 1 -- 172.21.15.151:0/148202045 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f998c005580 con 0x7f999c075770 2024-05-21T21:52:19.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:19.785+0000 7f99a1600700 1 -- 172.21.15.151:0/148202045 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f999c1a89a0 con 0x7f999c075770 2024-05-21T21:52:19.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:19.786+0000 7f99a1600700 1 -- 172.21.15.151:0/148202045 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f999c06a230 con 0x7f999c075770 2024-05-21T21:52:19.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:19.789+0000 7f9993400700 1 -- 172.21.15.151:0/148202045 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f998c004750 con 0x7f999c075770 2024-05-21T21:52:19.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:19.790+0000 7f9993400700 1 --2- 172.21.15.151:0/148202045 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f99800b2910 0x7f99800b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:52:19.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:19.790+0000 7f9993400700 1 -- 172.21.15.151:0/148202045 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f998c0e27f0 con 0x7f999c075770 2024-05-21T21:52:19.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:19.790+0000 7f999a000700 1 --2- 172.21.15.151:0/148202045 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f99800b2910 0x7f99800b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:52:19.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:19.791+0000 7f999a000700 1 --2- 172.21.15.151:0/148202045 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f99800b2910 0x7f99800b4dd0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9988009fd0 tx=0x7f9988019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:52:19.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:19.792+0000 7f9993400700 1 -- 172.21.15.151:0/148202045 <== mon.2 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f998c054df0 con 0x7f999c075770 2024-05-21T21:52:20.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:20.019+0000 7f99a1600700 1 -- 172.21.15.151:0/148202045 --> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f999c063980 con 0x7f99800b2910 2024-05-21T21:52:20.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:20.031+0000 7f9993400700 1 -- 172.21.15.151:0/148202045 <== mgr.14152 v2:172.21.15.151:6800/2796622483 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258803 (secure 0 0 0) 0x7f999c063980 con 0x7f99800b2910 2024-05-21T21:52:20.034 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T21:52:20.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:20.036+0000 7f99a1600700 1 -- 172.21.15.151:0/148202045 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f99800b2910 msgr2=0x7f99800b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:52:20.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:20.036+0000 7f99a1600700 1 --2- 172.21.15.151:0/148202045 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f99800b2910 0x7f99800b4dd0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9988009fd0 tx=0x7f9988019040 comp rx=0 tx=0).stop 2024-05-21T21:52:20.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:20.036+0000 7f99a1600700 1 -- 172.21.15.151:0/148202045 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f999c075770 msgr2=0x7f999c19be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:52:20.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:20.036+0000 7f99a1600700 1 --2- 172.21.15.151:0/148202045 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f999c075770 0x7f999c19be70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f998c00e9c0 tx=0x7f998c00ecd0 comp rx=0 tx=0).stop 2024-05-21T21:52:20.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:20.036+0000 7f99a1600700 1 -- 172.21.15.151:0/148202045 shutdown_connections 2024-05-21T21:52:20.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:20.036+0000 7f99a1600700 1 --2- 172.21.15.151:0/148202045 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f99800b2910 0x7f99800b4dd0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:20.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:20.036+0000 7f99a1600700 1 --2- 172.21.15.151:0/148202045 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f999c075770 0x7f999c19be70 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:20.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:20.036+0000 7f99a1600700 1 --2- 172.21.15.151:0/148202045 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f999c10ff40 0x7f999c1a06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:20.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:20.036+0000 7f99a1600700 1 --2- 172.21.15.151:0/148202045 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f999c076090 0x7f999c19c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:20.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:20.036+0000 7f99a1600700 1 -- 172.21.15.151:0/148202045 >> 172.21.15.151:0/148202045 conn(0x7f999c07a6a0 msgr2=0x7f999c1007f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:52:20.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:20.036+0000 7f99a1600700 1 -- 172.21.15.151:0/148202045 shutdown_connections 2024-05-21T21:52:20.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:20.036+0000 7f99a1600700 1 -- 172.21.15.151:0/148202045 wait complete. 2024-05-21T21:52:20.039 INFO:teuthology.orchestra.run.smithi151.stderr:dumped all 2024-05-21T21:52:20.767 INFO:teuthology.orchestra.run.smithi151.stdout:{"pg_ready":true,"pg_map":{"version":347,"stamp":"2024-05-21T21:52:19.214834+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657568,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":496,"num_read_kb":347,"num_write":370,"num_write_kb":2287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":2952069,"num_keys_recovered":17,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":304,"ondisk_log_size":304,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":366112,"kb_used_data":12732,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124575712,"statfs":{"total":1151940427776,"available":1151565529088,"internally_reserved":0,"allocated":13037568,"data_stored":7165908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19140,"internal_metadata":361674044},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4257,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":94,"num_read_kb":48,"num_write":1,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":229,"num_keys_recovered":17,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.354944"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.861691+0000","last_change":"2024-05-21T21:51:16.945132+0000","last_active":"2024-05-21T21:52:10.861691+0000","last_peered":"2024-05-21T21:52:10.861691+0000","last_clean":"2024-05-21T21:52:10.861691+0000","last_became_active":"2024-05-21T21:51:16.944766+0000","last_became_peered":"2024-05-21T21:51:16.944766+0000","last_unstale":"2024-05-21T21:52:10.861691+0000","last_undegraded":"2024-05-21T21:52:10.861691+0000","last_fullsized":"2024-05-21T21:52:10.861691+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:54:04.185095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.873699+0000","last_change":"2024-05-21T21:51:16.939291+0000","last_active":"2024-05-21T21:52:10.873699+0000","last_peered":"2024-05-21T21:52:10.873699+0000","last_clean":"2024-05-21T21:52:10.873699+0000","last_became_active":"2024-05-21T21:51:16.938730+0000","last_became_peered":"2024-05-21T21:51:16.938730+0000","last_unstale":"2024-05-21T21:52:10.873699+0000","last_undegraded":"2024-05-21T21:52:10.873699+0000","last_fullsized":"2024-05-21T21:52:10.873699+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:05:30.623744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.873014+0000","last_change":"2024-05-21T21:51:16.941565+0000","last_active":"2024-05-21T21:52:10.873014+0000","last_peered":"2024-05-21T21:52:10.873014+0000","last_clean":"2024-05-21T21:52:10.873014+0000","last_became_active":"2024-05-21T21:51:16.941033+0000","last_became_peered":"2024-05-21T21:51:16.941033+0000","last_unstale":"2024-05-21T21:52:10.873014+0000","last_undegraded":"2024-05-21T21:52:10.873014+0000","last_fullsized":"2024-05-21T21:52:10.873014+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:04:55.789479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.865221+0000","last_change":"2024-05-21T21:51:16.928455+0000","last_active":"2024-05-21T21:52:10.865221+0000","last_peered":"2024-05-21T21:52:10.865221+0000","last_clean":"2024-05-21T21:52:10.865221+0000","last_became_active":"2024-05-21T21:51:16.927874+0000","last_became_peered":"2024-05-21T21:51:16.927874+0000","last_unstale":"2024-05-21T21:52:10.865221+0000","last_undegraded":"2024-05-21T21:52:10.865221+0000","last_fullsized":"2024-05-21T21:52:10.865221+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:25:16.483175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.863776+0000","last_change":"2024-05-21T21:51:16.937773+0000","last_active":"2024-05-21T21:52:10.863776+0000","last_peered":"2024-05-21T21:52:10.863776+0000","last_clean":"2024-05-21T21:52:10.863776+0000","last_became_active":"2024-05-21T21:51:16.937306+0000","last_became_peered":"2024-05-21T21:51:16.937306+0000","last_unstale":"2024-05-21T21:52:10.863776+0000","last_undegraded":"2024-05-21T21:52:10.863776+0000","last_fullsized":"2024-05-21T21:52:10.863776+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:05:03.762390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.865632+0000","last_change":"2024-05-21T21:51:16.948275+0000","last_active":"2024-05-21T21:52:10.865632+0000","last_peered":"2024-05-21T21:52:10.865632+0000","last_clean":"2024-05-21T21:52:10.865632+0000","last_became_active":"2024-05-21T21:51:16.947890+0000","last_became_peered":"2024-05-21T21:51:16.947890+0000","last_unstale":"2024-05-21T21:52:10.865632+0000","last_undegraded":"2024-05-21T21:52:10.865632+0000","last_fullsized":"2024-05-21T21:52:10.865632+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:08:38.572177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.864396+0000","last_change":"2024-05-21T21:51:16.946821+0000","last_active":"2024-05-21T21:52:10.864396+0000","last_peered":"2024-05-21T21:52:10.864396+0000","last_clean":"2024-05-21T21:52:10.864396+0000","last_became_active":"2024-05-21T21:51:16.946515+0000","last_became_peered":"2024-05-21T21:51:16.946515+0000","last_unstale":"2024-05-21T21:52:10.864396+0000","last_undegraded":"2024-05-21T21:52:10.864396+0000","last_fullsized":"2024-05-21T21:52:10.864396+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T09:26:25.867895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.870476+0000","last_change":"2024-05-21T21:51:16.929228+0000","last_active":"2024-05-21T21:52:10.870476+0000","last_peered":"2024-05-21T21:52:10.870476+0000","last_clean":"2024-05-21T21:52:10.870476+0000","last_became_active":"2024-05-21T21:51:16.928875+0000","last_became_peered":"2024-05-21T21:51:16.928875+0000","last_unstale":"2024-05-21T21:52:10.870476+0000","last_undegraded":"2024-05-21T21:52:10.870476+0000","last_fullsized":"2024-05-21T21:52:10.870476+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:50:56.431310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.973145+0000","last_change":"2024-05-21T21:51:16.932360+0000","last_active":"2024-05-21T21:52:10.973145+0000","last_peered":"2024-05-21T21:52:10.973145+0000","last_clean":"2024-05-21T21:52:10.973145+0000","last_became_active":"2024-05-21T21:51:16.932036+0000","last_became_peered":"2024-05-21T21:51:16.932036+0000","last_unstale":"2024-05-21T21:52:10.973145+0000","last_undegraded":"2024-05-21T21:52:10.973145+0000","last_fullsized":"2024-05-21T21:52:10.973145+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T09:06:37.257323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.876556+0000","last_change":"2024-05-21T21:52:10.876556+0000","last_active":"2024-05-21T21:52:10.876556+0000","last_peered":"2024-05-21T21:52:10.876556+0000","last_clean":"2024-05-21T21:52:10.876556+0000","last_became_active":"2024-05-21T21:52:10.876273+0000","last_became_peered":"2024-05-21T21:52:10.876273+0000","last_unstale":"2024-05-21T21:52:10.876556+0000","last_undegraded":"2024-05-21T21:52:10.876556+0000","last_fullsized":"2024-05-21T21:52:10.876556+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T05:09:40.037731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.864226+0000","last_change":"2024-05-21T21:51:16.933425+0000","last_active":"2024-05-21T21:52:10.864226+0000","last_peered":"2024-05-21T21:52:10.864226+0000","last_clean":"2024-05-21T21:52:10.864226+0000","last_became_active":"2024-05-21T21:51:16.933057+0000","last_became_peered":"2024-05-21T21:51:16.933057+0000","last_unstale":"2024-05-21T21:52:10.864226+0000","last_undegraded":"2024-05-21T21:52:10.864226+0000","last_fullsized":"2024-05-21T21:52:10.864226+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:04:57.237841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.865179+0000","last_change":"2024-05-21T21:51:16.927014+0000","last_active":"2024-05-21T21:52:10.865179+0000","last_peered":"2024-05-21T21:52:10.865179+0000","last_clean":"2024-05-21T21:52:10.865179+0000","last_became_active":"2024-05-21T21:51:16.925954+0000","last_became_peered":"2024-05-21T21:51:16.925954+0000","last_unstale":"2024-05-21T21:52:10.865179+0000","last_undegraded":"2024-05-21T21:52:10.865179+0000","last_fullsized":"2024-05-21T21:52:10.865179+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:19:20.530727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.862761+0000","last_change":"2024-05-21T21:51:16.930909+0000","last_active":"2024-05-21T21:52:10.862761+0000","last_peered":"2024-05-21T21:52:10.862761+0000","last_clean":"2024-05-21T21:52:10.862761+0000","last_became_active":"2024-05-21T21:51:16.927110+0000","last_became_peered":"2024-05-21T21:51:16.927110+0000","last_unstale":"2024-05-21T21:52:10.862761+0000","last_undegraded":"2024-05-21T21:52:10.862761+0000","last_fullsized":"2024-05-21T21:52:10.862761+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T06:23:55.386047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.872086+0000","last_change":"2024-05-21T21:51:16.938797+0000","last_active":"2024-05-21T21:52:10.872086+0000","last_peered":"2024-05-21T21:52:10.872086+0000","last_clean":"2024-05-21T21:52:10.872086+0000","last_became_active":"2024-05-21T21:51:16.938263+0000","last_became_peered":"2024-05-21T21:51:16.938263+0000","last_unstale":"2024-05-21T21:52:10.872086+0000","last_undegraded":"2024-05-21T21:52:10.872086+0000","last_fullsized":"2024-05-21T21:52:10.872086+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:37:40.521118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.871620+0000","last_change":"2024-05-21T21:51:16.944160+0000","last_active":"2024-05-21T21:52:10.871620+0000","last_peered":"2024-05-21T21:52:10.871620+0000","last_clean":"2024-05-21T21:52:10.871620+0000","last_became_active":"2024-05-21T21:51:16.940046+0000","last_became_peered":"2024-05-21T21:51:16.940046+0000","last_unstale":"2024-05-21T21:52:10.871620+0000","last_undegraded":"2024-05-21T21:52:10.871620+0000","last_fullsized":"2024-05-21T21:52:10.871620+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:14:13.278799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.973152+0000","last_change":"2024-05-21T21:51:16.945599+0000","last_active":"2024-05-21T21:52:10.973152+0000","last_peered":"2024-05-21T21:52:10.973152+0000","last_clean":"2024-05-21T21:52:10.973152+0000","last_became_active":"2024-05-21T21:51:16.945196+0000","last_became_peered":"2024-05-21T21:51:16.945196+0000","last_unstale":"2024-05-21T21:52:10.973152+0000","last_undegraded":"2024-05-21T21:52:10.973152+0000","last_fullsized":"2024-05-21T21:52:10.973152+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:59:24.453615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.861779+0000","last_change":"2024-05-21T21:51:16.944895+0000","last_active":"2024-05-21T21:52:10.861779+0000","last_peered":"2024-05-21T21:52:10.861779+0000","last_clean":"2024-05-21T21:52:10.861779+0000","last_became_active":"2024-05-21T21:51:16.944560+0000","last_became_peered":"2024-05-21T21:51:16.944560+0000","last_unstale":"2024-05-21T21:52:10.861779+0000","last_undegraded":"2024-05-21T21:52:10.861779+0000","last_fullsized":"2024-05-21T21:52:10.861779+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:53:13.264701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.877457+0000","last_change":"2024-05-21T21:51:16.943861+0000","last_active":"2024-05-21T21:52:10.877457+0000","last_peered":"2024-05-21T21:52:10.877457+0000","last_clean":"2024-05-21T21:52:10.877457+0000","last_became_active":"2024-05-21T21:51:16.943123+0000","last_became_peered":"2024-05-21T21:51:16.943123+0000","last_unstale":"2024-05-21T21:52:10.877457+0000","last_undegraded":"2024-05-21T21:52:10.877457+0000","last_fullsized":"2024-05-21T21:52:10.877457+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:35:37.732855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.864479+0000","last_change":"2024-05-21T21:51:16.940580+0000","last_active":"2024-05-21T21:52:10.864479+0000","last_peered":"2024-05-21T21:52:10.864479+0000","last_clean":"2024-05-21T21:52:10.864479+0000","last_became_active":"2024-05-21T21:51:16.940264+0000","last_became_peered":"2024-05-21T21:51:16.940264+0000","last_unstale":"2024-05-21T21:52:10.864479+0000","last_undegraded":"2024-05-21T21:52:10.864479+0000","last_fullsized":"2024-05-21T21:52:10.864479+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:54:22.915541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"80'13","reported_seq":76,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:13.445747+0000","last_change":"2024-05-21T21:51:12.852935+0000","last_active":"2024-05-21T21:52:13.445747+0000","last_peered":"2024-05-21T21:52:13.445747+0000","last_clean":"2024-05-21T21:52:13.445747+0000","last_became_active":"2024-05-21T21:51:12.851802+0000","last_became_peered":"2024-05-21T21:51:12.851802+0000","last_unstale":"2024-05-21T21:52:13.445747+0000","last_undegraded":"2024-05-21T21:52:13.445747+0000","last_fullsized":"2024-05-21T21:52:13.445747+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:13:24.643408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":37,"num_read_kb":37,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.868253+0000","last_change":"2024-05-21T21:51:16.939461+0000","last_active":"2024-05-21T21:52:10.868253+0000","last_peered":"2024-05-21T21:52:10.868253+0000","last_clean":"2024-05-21T21:52:10.868253+0000","last_became_active":"2024-05-21T21:51:16.936322+0000","last_became_peered":"2024-05-21T21:51:16.936322+0000","last_unstale":"2024-05-21T21:52:10.868253+0000","last_undegraded":"2024-05-21T21:52:10.868253+0000","last_fullsized":"2024-05-21T21:52:10.868253+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:37:26.153963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.868674+0000","last_change":"2024-05-21T21:51:12.941838+0000","last_active":"2024-05-21T21:52:10.868674+0000","last_peered":"2024-05-21T21:52:10.868674+0000","last_clean":"2024-05-21T21:52:10.868674+0000","last_became_active":"2024-05-21T21:51:12.940818+0000","last_became_peered":"2024-05-21T21:51:12.940818+0000","last_unstale":"2024-05-21T21:52:10.868674+0000","last_undegraded":"2024-05-21T21:52:10.868674+0000","last_fullsized":"2024-05-21T21:52:10.868674+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:30:42.349241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.873994+0000","last_change":"2024-05-21T21:51:16.950252+0000","last_active":"2024-05-21T21:52:10.873994+0000","last_peered":"2024-05-21T21:52:10.873994+0000","last_clean":"2024-05-21T21:52:10.873994+0000","last_became_active":"2024-05-21T21:51:16.947375+0000","last_became_peered":"2024-05-21T21:51:16.947375+0000","last_unstale":"2024-05-21T21:52:10.873994+0000","last_undegraded":"2024-05-21T21:52:10.873994+0000","last_fullsized":"2024-05-21T21:52:10.873994+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:27:30.462799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"80'12","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.876122+0000","last_change":"2024-05-21T21:52:10.875739+0000","last_active":"2024-05-21T21:52:10.876122+0000","last_peered":"2024-05-21T21:52:10.876122+0000","last_clean":"2024-05-21T21:52:10.876122+0000","last_became_active":"2024-05-21T21:52:10.865846+0000","last_became_peered":"2024-05-21T21:52:10.865846+0000","last_unstale":"2024-05-21T21:52:10.876122+0000","last_undegraded":"2024-05-21T21:52:10.876122+0000","last_fullsized":"2024-05-21T21:52:10.876122+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T06:48:06.465757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.972759+0000","last_change":"2024-05-21T21:51:16.926380+0000","last_active":"2024-05-21T21:52:10.972759+0000","last_peered":"2024-05-21T21:52:10.972759+0000","last_clean":"2024-05-21T21:52:10.972759+0000","last_became_active":"2024-05-21T21:51:16.925821+0000","last_became_peered":"2024-05-21T21:51:16.925821+0000","last_unstale":"2024-05-21T21:52:10.972759+0000","last_undegraded":"2024-05-21T21:52:10.972759+0000","last_fullsized":"2024-05-21T21:52:10.972759+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:42:14.887155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'5","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.864272+0000","last_change":"2024-05-21T21:51:12.854342+0000","last_active":"2024-05-21T21:52:10.864272+0000","last_peered":"2024-05-21T21:52:10.864272+0000","last_clean":"2024-05-21T21:52:10.864272+0000","last_became_active":"2024-05-21T21:51:12.853908+0000","last_became_peered":"2024-05-21T21:51:12.853908+0000","last_unstale":"2024-05-21T21:52:10.864272+0000","last_undegraded":"2024-05-21T21:52:10.864272+0000","last_fullsized":"2024-05-21T21:52:10.864272+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:35:10.374749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.973816+0000","last_change":"2024-05-21T21:52:10.973816+0000","last_active":"2024-05-21T21:52:10.973816+0000","last_peered":"2024-05-21T21:52:10.973816+0000","last_clean":"2024-05-21T21:52:10.973816+0000","last_became_active":"2024-05-21T21:52:10.973457+0000","last_became_peered":"2024-05-21T21:52:10.973457+0000","last_unstale":"2024-05-21T21:52:10.973816+0000","last_undegraded":"2024-05-21T21:52:10.973816+0000","last_fullsized":"2024-05-21T21:52:10.973816+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T05:01:41.915158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"80'6","reported_seq":74,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:16.443294+0000","last_change":"2024-05-21T21:51:12.855658+0000","last_active":"2024-05-21T21:52:16.443294+0000","last_peered":"2024-05-21T21:52:16.443294+0000","last_clean":"2024-05-21T21:52:16.443294+0000","last_became_active":"2024-05-21T21:51:12.855281+0000","last_became_peered":"2024-05-21T21:51:12.855281+0000","last_unstale":"2024-05-21T21:52:16.443294+0000","last_undegraded":"2024-05-21T21:52:16.443294+0000","last_fullsized":"2024-05-21T21:52:16.443294+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:55:08.836045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":42,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.863544+0000","last_change":"2024-05-21T21:51:16.946977+0000","last_active":"2024-05-21T21:52:10.863544+0000","last_peered":"2024-05-21T21:52:10.863544+0000","last_clean":"2024-05-21T21:52:10.863544+0000","last_became_active":"2024-05-21T21:51:16.946562+0000","last_became_peered":"2024-05-21T21:51:16.946562+0000","last_unstale":"2024-05-21T21:52:10.863544+0000","last_undegraded":"2024-05-21T21:52:10.863544+0000","last_fullsized":"2024-05-21T21:52:10.863544+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:43:33.370265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"80'7","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.865540+0000","last_change":"2024-05-21T21:51:12.852574+0000","last_active":"2024-05-21T21:52:10.865540+0000","last_peered":"2024-05-21T21:52:10.865540+0000","last_clean":"2024-05-21T21:52:10.865540+0000","last_became_active":"2024-05-21T21:51:12.852258+0000","last_became_peered":"2024-05-21T21:51:12.852258+0000","last_unstale":"2024-05-21T21:52:10.865540+0000","last_undegraded":"2024-05-21T21:52:10.865540+0000","last_fullsized":"2024-05-21T21:52:10.865540+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:09:40.280821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.867344+0000","last_change":"2024-05-21T21:51:16.934768+0000","last_active":"2024-05-21T21:52:10.867344+0000","last_peered":"2024-05-21T21:52:10.867344+0000","last_clean":"2024-05-21T21:52:10.867344+0000","last_became_active":"2024-05-21T21:51:16.934275+0000","last_became_peered":"2024-05-21T21:51:16.934275+0000","last_unstale":"2024-05-21T21:52:10.867344+0000","last_undegraded":"2024-05-21T21:52:10.867344+0000","last_fullsized":"2024-05-21T21:52:10.867344+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:31:54.425069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"80'3","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.861882+0000","last_change":"2024-05-21T21:51:12.854166+0000","last_active":"2024-05-21T21:52:10.861882+0000","last_peered":"2024-05-21T21:52:10.861882+0000","last_clean":"2024-05-21T21:52:10.861882+0000","last_became_active":"2024-05-21T21:51:12.853508+0000","last_became_peered":"2024-05-21T21:51:12.853508+0000","last_unstale":"2024-05-21T21:52:10.861882+0000","last_undegraded":"2024-05-21T21:52:10.861882+0000","last_fullsized":"2024-05-21T21:52:10.861882+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:29:41.858655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.870323+0000","last_change":"2024-05-21T21:51:16.938293+0000","last_active":"2024-05-21T21:52:10.870323+0000","last_peered":"2024-05-21T21:52:10.870323+0000","last_clean":"2024-05-21T21:52:10.870323+0000","last_became_active":"2024-05-21T21:51:16.937970+0000","last_became_peered":"2024-05-21T21:51:16.937970+0000","last_unstale":"2024-05-21T21:52:10.870323+0000","last_undegraded":"2024-05-21T21:52:10.870323+0000","last_fullsized":"2024-05-21T21:52:10.870323+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:18:27.473098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'3","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.866723+0000","last_change":"2024-05-21T21:51:12.853366+0000","last_active":"2024-05-21T21:52:10.866723+0000","last_peered":"2024-05-21T21:52:10.866723+0000","last_clean":"2024-05-21T21:52:10.866723+0000","last_became_active":"2024-05-21T21:51:12.853028+0000","last_became_peered":"2024-05-21T21:51:12.853028+0000","last_unstale":"2024-05-21T21:52:10.866723+0000","last_undegraded":"2024-05-21T21:52:10.866723+0000","last_fullsized":"2024-05-21T21:52:10.866723+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T09:15:52.593028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.874880+0000","last_change":"2024-05-21T21:51:16.947903+0000","last_active":"2024-05-21T21:52:10.874880+0000","last_peered":"2024-05-21T21:52:10.874880+0000","last_clean":"2024-05-21T21:52:10.874880+0000","last_became_active":"2024-05-21T21:51:16.946677+0000","last_became_peered":"2024-05-21T21:51:16.946677+0000","last_unstale":"2024-05-21T21:52:10.874880+0000","last_undegraded":"2024-05-21T21:52:10.874880+0000","last_fullsized":"2024-05-21T21:52:10.874880+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T06:12:56.514284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'4","reported_seq":106,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:16.426637+0000","last_change":"2024-05-21T21:51:12.854873+0000","last_active":"2024-05-21T21:52:16.426637+0000","last_peered":"2024-05-21T21:52:16.426637+0000","last_clean":"2024-05-21T21:52:16.426637+0000","last_became_active":"2024-05-21T21:51:12.854393+0000","last_became_peered":"2024-05-21T21:51:12.854393+0000","last_unstale":"2024-05-21T21:52:16.426637+0000","last_undegraded":"2024-05-21T21:52:16.426637+0000","last_fullsized":"2024-05-21T21:52:16.426637+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:58:52.411164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":76,"num_read_kb":76,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.973117+0000","last_change":"2024-05-21T21:51:16.951094+0000","last_active":"2024-05-21T21:52:10.973117+0000","last_peered":"2024-05-21T21:52:10.973117+0000","last_clean":"2024-05-21T21:52:10.973117+0000","last_became_active":"2024-05-21T21:51:16.950724+0000","last_became_peered":"2024-05-21T21:51:16.950724+0000","last_unstale":"2024-05-21T21:52:10.973117+0000","last_undegraded":"2024-05-21T21:52:10.973117+0000","last_fullsized":"2024-05-21T21:52:10.973117+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:54:02.835426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'4","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.864127+0000","last_change":"2024-05-21T21:51:12.850427+0000","last_active":"2024-05-21T21:52:10.864127+0000","last_peered":"2024-05-21T21:52:10.864127+0000","last_clean":"2024-05-21T21:52:10.864127+0000","last_became_active":"2024-05-21T21:51:12.850096+0000","last_became_peered":"2024-05-21T21:51:12.850096+0000","last_unstale":"2024-05-21T21:52:10.864127+0000","last_undegraded":"2024-05-21T21:52:10.864127+0000","last_fullsized":"2024-05-21T21:52:10.864127+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:43:22.611043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.874050+0000","last_change":"2024-05-21T21:51:16.942330+0000","last_active":"2024-05-21T21:52:10.874050+0000","last_peered":"2024-05-21T21:52:10.874050+0000","last_clean":"2024-05-21T21:52:10.874050+0000","last_became_active":"2024-05-21T21:51:16.941718+0000","last_became_peered":"2024-05-21T21:51:16.941718+0000","last_unstale":"2024-05-21T21:52:10.874050+0000","last_undegraded":"2024-05-21T21:52:10.874050+0000","last_fullsized":"2024-05-21T21:52:10.874050+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:54:17.176411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'5","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.867578+0000","last_change":"2024-05-21T21:51:12.941100+0000","last_active":"2024-05-21T21:52:10.867578+0000","last_peered":"2024-05-21T21:52:10.867578+0000","last_clean":"2024-05-21T21:52:10.867578+0000","last_became_active":"2024-05-21T21:51:12.940597+0000","last_became_peered":"2024-05-21T21:51:12.940597+0000","last_unstale":"2024-05-21T21:52:10.867578+0000","last_undegraded":"2024-05-21T21:52:10.867578+0000","last_fullsized":"2024-05-21T21:52:10.867578+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:30:58.457108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.972855+0000","last_change":"2024-05-21T21:51:16.921973+0000","last_active":"2024-05-21T21:52:10.972855+0000","last_peered":"2024-05-21T21:52:10.972855+0000","last_clean":"2024-05-21T21:52:10.972855+0000","last_became_active":"2024-05-21T21:51:16.921597+0000","last_became_peered":"2024-05-21T21:51:16.921597+0000","last_unstale":"2024-05-21T21:52:10.972855+0000","last_undegraded":"2024-05-21T21:52:10.972855+0000","last_fullsized":"2024-05-21T21:52:10.972855+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:25:32.395940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'31","reported_seq":142,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:16.426636+0000","last_change":"2024-05-21T21:51:12.850312+0000","last_active":"2024-05-21T21:52:16.426636+0000","last_peered":"2024-05-21T21:52:16.426636+0000","last_clean":"2024-05-21T21:52:16.426636+0000","last_became_active":"2024-05-21T21:51:12.849837+0000","last_became_peered":"2024-05-21T21:51:12.849837+0000","last_unstale":"2024-05-21T21:52:16.426636+0000","last_undegraded":"2024-05-21T21:52:16.426636+0000","last_fullsized":"2024-05-21T21:52:16.426636+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T07:09:13.339554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59762,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":85,"num_read_kb":100,"num_write":33,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.863568+0000","last_change":"2024-05-21T21:51:16.926625+0000","last_active":"2024-05-21T21:52:10.863568+0000","last_peered":"2024-05-21T21:52:10.863568+0000","last_clean":"2024-05-21T21:52:10.863568+0000","last_became_active":"2024-05-21T21:51:16.925889+0000","last_became_peered":"2024-05-21T21:51:16.925889+0000","last_unstale":"2024-05-21T21:52:10.863568+0000","last_undegraded":"2024-05-21T21:52:10.863568+0000","last_fullsized":"2024-05-21T21:52:10.863568+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:35:23.464747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'9","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.863891+0000","last_change":"2024-05-21T21:51:12.859772+0000","last_active":"2024-05-21T21:52:10.863891+0000","last_peered":"2024-05-21T21:52:10.863891+0000","last_clean":"2024-05-21T21:52:10.863891+0000","last_became_active":"2024-05-21T21:51:12.859290+0000","last_became_peered":"2024-05-21T21:51:12.859290+0000","last_unstale":"2024-05-21T21:52:10.863891+0000","last_undegraded":"2024-05-21T21:52:10.863891+0000","last_fullsized":"2024-05-21T21:52:10.863891+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T05:28:29.953381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.861761+0000","last_change":"2024-05-21T21:51:16.925803+0000","last_active":"2024-05-21T21:52:10.861761+0000","last_peered":"2024-05-21T21:52:10.861761+0000","last_clean":"2024-05-21T21:52:10.861761+0000","last_became_active":"2024-05-21T21:51:16.925111+0000","last_became_peered":"2024-05-21T21:51:16.925111+0000","last_unstale":"2024-05-21T21:52:10.861761+0000","last_undegraded":"2024-05-21T21:52:10.861761+0000","last_fullsized":"2024-05-21T21:52:10.861761+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T06:22:11.353379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'5","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.867329+0000","last_change":"2024-05-21T21:51:12.859593+0000","last_active":"2024-05-21T21:52:10.867329+0000","last_peered":"2024-05-21T21:52:10.867329+0000","last_clean":"2024-05-21T21:52:10.867329+0000","last_became_active":"2024-05-21T21:51:12.859197+0000","last_became_peered":"2024-05-21T21:51:12.859197+0000","last_unstale":"2024-05-21T21:52:10.867329+0000","last_undegraded":"2024-05-21T21:52:10.867329+0000","last_fullsized":"2024-05-21T21:52:10.867329+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:13:12.311496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.879028+0000","last_change":"2024-05-21T21:52:10.879028+0000","last_active":"2024-05-21T21:52:10.879028+0000","last_peered":"2024-05-21T21:52:10.879028+0000","last_clean":"2024-05-21T21:52:10.879028+0000","last_became_active":"2024-05-21T21:52:10.878664+0000","last_became_peered":"2024-05-21T21:52:10.878664+0000","last_unstale":"2024-05-21T21:52:10.879028+0000","last_undegraded":"2024-05-21T21:52:10.879028+0000","last_fullsized":"2024-05-21T21:52:10.879028+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:05:53.953546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'6","reported_seq":74,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:16.435599+0000","last_change":"2024-05-21T21:51:12.852169+0000","last_active":"2024-05-21T21:52:16.435599+0000","last_peered":"2024-05-21T21:52:16.435599+0000","last_clean":"2024-05-21T21:52:16.435599+0000","last_became_active":"2024-05-21T21:51:12.851782+0000","last_became_peered":"2024-05-21T21:51:12.851782+0000","last_unstale":"2024-05-21T21:52:16.435599+0000","last_undegraded":"2024-05-21T21:52:16.435599+0000","last_fullsized":"2024-05-21T21:52:16.435599+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:46:09.692158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":43,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.863791+0000","last_change":"2024-05-21T21:51:16.929264+0000","last_active":"2024-05-21T21:52:10.863791+0000","last_peered":"2024-05-21T21:52:10.863791+0000","last_clean":"2024-05-21T21:52:10.863791+0000","last_became_active":"2024-05-21T21:51:16.928900+0000","last_became_peered":"2024-05-21T21:51:16.928900+0000","last_unstale":"2024-05-21T21:52:10.863791+0000","last_undegraded":"2024-05-21T21:52:10.863791+0000","last_fullsized":"2024-05-21T21:52:10.863791+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:42:28.040939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'17","reported_seq":68,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.984451+0000","last_change":"2024-05-21T21:52:10.984451+0000","last_active":"2024-05-21T21:52:10.984451+0000","last_peered":"2024-05-21T21:52:10.984451+0000","last_clean":"2024-05-21T21:52:10.984451+0000","last_became_active":"2024-05-21T21:52:10.974050+0000","last_became_peered":"2024-05-21T21:52:10.974050+0000","last_unstale":"2024-05-21T21:52:10.984451+0000","last_undegraded":"2024-05-21T21:52:10.984451+0000","last_fullsized":"2024-05-21T21:52:10.984451+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T07:08:18.307022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.867149+0000","last_change":"2024-05-21T21:51:16.935464+0000","last_active":"2024-05-21T21:52:10.867149+0000","last_peered":"2024-05-21T21:52:10.867149+0000","last_clean":"2024-05-21T21:52:10.867149+0000","last_became_active":"2024-05-21T21:51:16.935091+0000","last_became_peered":"2024-05-21T21:51:16.935091+0000","last_unstale":"2024-05-21T21:52:10.867149+0000","last_undegraded":"2024-05-21T21:52:10.867149+0000","last_fullsized":"2024-05-21T21:52:10.867149+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T06:59:16.330649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.864530+0000","last_change":"2024-05-21T21:51:12.853327+0000","last_active":"2024-05-21T21:52:10.864530+0000","last_peered":"2024-05-21T21:52:10.864530+0000","last_clean":"2024-05-21T21:52:10.864530+0000","last_became_active":"2024-05-21T21:51:12.852946+0000","last_became_peered":"2024-05-21T21:51:12.852946+0000","last_unstale":"2024-05-21T21:52:10.864530+0000","last_undegraded":"2024-05-21T21:52:10.864530+0000","last_fullsized":"2024-05-21T21:52:10.864530+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:00:34.256858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.861583+0000","last_change":"2024-05-21T21:51:16.929341+0000","last_active":"2024-05-21T21:52:10.861583+0000","last_peered":"2024-05-21T21:52:10.861583+0000","last_clean":"2024-05-21T21:52:10.861583+0000","last_became_active":"2024-05-21T21:51:16.929008+0000","last_became_peered":"2024-05-21T21:51:16.929008+0000","last_unstale":"2024-05-21T21:52:10.861583+0000","last_undegraded":"2024-05-21T21:52:10.861583+0000","last_fullsized":"2024-05-21T21:52:10.861583+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:26:26.297136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"80'1","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.875960+0000","last_change":"2024-05-21T21:51:12.945433+0000","last_active":"2024-05-21T21:52:10.875960+0000","last_peered":"2024-05-21T21:52:10.875960+0000","last_clean":"2024-05-21T21:52:10.875960+0000","last_became_active":"2024-05-21T21:51:12.944463+0000","last_became_peered":"2024-05-21T21:51:12.944463+0000","last_unstale":"2024-05-21T21:52:10.875960+0000","last_undegraded":"2024-05-21T21:52:10.875960+0000","last_fullsized":"2024-05-21T21:52:10.875960+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:29:52.014583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'1","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.883554+0000","last_change":"2024-05-21T21:52:10.883554+0000","last_active":"2024-05-21T21:52:10.883554+0000","last_peered":"2024-05-21T21:52:10.883554+0000","last_clean":"2024-05-21T21:52:10.883554+0000","last_became_active":"2024-05-21T21:52:10.877789+0000","last_became_peered":"2024-05-21T21:52:10.877789+0000","last_unstale":"2024-05-21T21:52:10.883554+0000","last_undegraded":"2024-05-21T21:52:10.883554+0000","last_fullsized":"2024-05-21T21:52:10.883554+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:52:37.623551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"80'5","reported_seq":69,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:14.447918+0000","last_change":"2024-05-21T21:51:12.852883+0000","last_active":"2024-05-21T21:52:14.447918+0000","last_peered":"2024-05-21T21:52:14.447918+0000","last_clean":"2024-05-21T21:52:14.447918+0000","last_became_active":"2024-05-21T21:51:12.852549+0000","last_became_peered":"2024-05-21T21:51:12.852549+0000","last_unstale":"2024-05-21T21:52:14.447918+0000","last_undegraded":"2024-05-21T21:52:14.447918+0000","last_fullsized":"2024-05-21T21:52:14.447918+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:05:53.002286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":38,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.865777+0000","last_change":"2024-05-21T21:51:16.944425+0000","last_active":"2024-05-21T21:52:10.865777+0000","last_peered":"2024-05-21T21:52:10.865777+0000","last_clean":"2024-05-21T21:52:10.865777+0000","last_became_active":"2024-05-21T21:51:16.944033+0000","last_became_peered":"2024-05-21T21:51:16.944033+0000","last_unstale":"2024-05-21T21:52:10.865777+0000","last_undegraded":"2024-05-21T21:52:10.865777+0000","last_fullsized":"2024-05-21T21:52:10.865777+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T07:16:26.802329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.866923+0000","last_change":"2024-05-21T21:51:16.933264+0000","last_active":"2024-05-21T21:52:10.866923+0000","last_peered":"2024-05-21T21:52:10.866923+0000","last_clean":"2024-05-21T21:52:10.866923+0000","last_became_active":"2024-05-21T21:51:16.932911+0000","last_became_peered":"2024-05-21T21:51:16.932911+0000","last_unstale":"2024-05-21T21:52:10.866923+0000","last_undegraded":"2024-05-21T21:52:10.866923+0000","last_fullsized":"2024-05-21T21:52:10.866923+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:25:54.006367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"80'5","reported_seq":68,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:13.429840+0000","last_change":"2024-05-21T21:51:12.942900+0000","last_active":"2024-05-21T21:52:13.429840+0000","last_peered":"2024-05-21T21:52:13.429840+0000","last_clean":"2024-05-21T21:52:13.429840+0000","last_became_active":"2024-05-21T21:51:12.941038+0000","last_became_peered":"2024-05-21T21:51:12.941038+0000","last_unstale":"2024-05-21T21:52:13.429840+0000","last_undegraded":"2024-05-21T21:52:13.429840+0000","last_fullsized":"2024-05-21T21:52:13.429840+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:38:43.240011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":37,"num_read_kb":37,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.867131+0000","last_change":"2024-05-21T21:51:16.933391+0000","last_active":"2024-05-21T21:52:10.867131+0000","last_peered":"2024-05-21T21:52:10.867131+0000","last_clean":"2024-05-21T21:52:10.867131+0000","last_became_active":"2024-05-21T21:51:16.932320+0000","last_became_peered":"2024-05-21T21:51:16.932320+0000","last_unstale":"2024-05-21T21:52:10.867131+0000","last_undegraded":"2024-05-21T21:52:10.867131+0000","last_fullsized":"2024-05-21T21:52:10.867131+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T09:06:59.126748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'6","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.865206+0000","last_change":"2024-05-21T21:51:12.856342+0000","last_active":"2024-05-21T21:52:10.865206+0000","last_peered":"2024-05-21T21:52:10.865206+0000","last_clean":"2024-05-21T21:52:10.865206+0000","last_became_active":"2024-05-21T21:51:12.855952+0000","last_became_peered":"2024-05-21T21:51:12.855952+0000","last_unstale":"2024-05-21T21:52:10.865206+0000","last_undegraded":"2024-05-21T21:52:10.865206+0000","last_fullsized":"2024-05-21T21:52:10.865206+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:34:40.734034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":99,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.863915+0000","last_change":"2024-05-21T21:49:42.797187+0000","last_active":"2024-05-21T21:52:10.863915+0000","last_peered":"2024-05-21T21:52:10.863915+0000","last_clean":"2024-05-21T21:52:10.863915+0000","last_became_active":"2024-05-21T21:49:42.791291+0000","last_became_peered":"2024-05-21T21:49:42.791291+0000","last_unstale":"2024-05-21T21:52:10.863915+0000","last_undegraded":"2024-05-21T21:52:10.863915+0000","last_fullsized":"2024-05-21T21:52:10.863915+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:45:55.552293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:45:55.552293+0000","last_clean_scrub_stamp":"2024-05-21T21:45:55.552293+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T07:24:18.302523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.861715+0000","last_change":"2024-05-21T21:51:16.926446+0000","last_active":"2024-05-21T21:52:10.861715+0000","last_peered":"2024-05-21T21:52:10.861715+0000","last_clean":"2024-05-21T21:52:10.861715+0000","last_became_active":"2024-05-21T21:51:16.926070+0000","last_became_peered":"2024-05-21T21:51:16.926070+0000","last_unstale":"2024-05-21T21:52:10.861715+0000","last_undegraded":"2024-05-21T21:52:10.861715+0000","last_fullsized":"2024-05-21T21:52:10.861715+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:44:27.366393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'4","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.864417+0000","last_change":"2024-05-21T21:51:12.854371+0000","last_active":"2024-05-21T21:52:10.864417+0000","last_peered":"2024-05-21T21:52:10.864417+0000","last_clean":"2024-05-21T21:52:10.864417+0000","last_became_active":"2024-05-21T21:51:12.853847+0000","last_became_peered":"2024-05-21T21:51:12.853847+0000","last_unstale":"2024-05-21T21:52:10.864417+0000","last_undegraded":"2024-05-21T21:52:10.864417+0000","last_fullsized":"2024-05-21T21:52:10.864417+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:14:10.549901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"80'4","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.863685+0000","last_change":"2024-05-21T21:51:12.941227+0000","last_active":"2024-05-21T21:52:10.863685+0000","last_peered":"2024-05-21T21:52:10.863685+0000","last_clean":"2024-05-21T21:52:10.863685+0000","last_became_active":"2024-05-21T21:51:12.940702+0000","last_became_peered":"2024-05-21T21:51:12.940702+0000","last_unstale":"2024-05-21T21:52:10.863685+0000","last_undegraded":"2024-05-21T21:52:10.863685+0000","last_fullsized":"2024-05-21T21:52:10.863685+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:40:34.124921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.862247+0000","last_change":"2024-05-21T21:51:16.945627+0000","last_active":"2024-05-21T21:52:10.862247+0000","last_peered":"2024-05-21T21:52:10.862247+0000","last_clean":"2024-05-21T21:52:10.862247+0000","last_became_active":"2024-05-21T21:51:16.945297+0000","last_became_peered":"2024-05-21T21:51:16.945297+0000","last_unstale":"2024-05-21T21:52:10.862247+0000","last_undegraded":"2024-05-21T21:52:10.862247+0000","last_fullsized":"2024-05-21T21:52:10.862247+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:54:38.898774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'15","reported_seq":57,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.888076+0000","last_change":"2024-05-21T21:52:10.888076+0000","last_active":"2024-05-21T21:52:10.888076+0000","last_peered":"2024-05-21T21:52:10.888076+0000","last_clean":"2024-05-21T21:52:10.888076+0000","last_became_active":"2024-05-21T21:52:10.879368+0000","last_became_peered":"2024-05-21T21:52:10.879368+0000","last_unstale":"2024-05-21T21:52:10.888076+0000","last_undegraded":"2024-05-21T21:52:10.888076+0000","last_fullsized":"2024-05-21T21:52:10.888076+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:46:43.190411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.868811+0000","last_change":"2024-05-21T21:51:16.929807+0000","last_active":"2024-05-21T21:52:10.868811+0000","last_peered":"2024-05-21T21:52:10.868811+0000","last_clean":"2024-05-21T21:52:10.868811+0000","last_became_active":"2024-05-21T21:51:16.928859+0000","last_became_peered":"2024-05-21T21:51:16.928859+0000","last_unstale":"2024-05-21T21:52:10.868811+0000","last_undegraded":"2024-05-21T21:52:10.868811+0000","last_fullsized":"2024-05-21T21:52:10.868811+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:41:32.830892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'7","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.865597+0000","last_change":"2024-05-21T21:51:12.850843+0000","last_active":"2024-05-21T21:52:10.865597+0000","last_peered":"2024-05-21T21:52:10.865597+0000","last_clean":"2024-05-21T21:52:10.865597+0000","last_became_active":"2024-05-21T21:51:12.850516+0000","last_became_peered":"2024-05-21T21:51:12.850516+0000","last_unstale":"2024-05-21T21:52:10.865597+0000","last_undegraded":"2024-05-21T21:52:10.865597+0000","last_fullsized":"2024-05-21T21:52:10.865597+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:37:05.191626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.873845+0000","last_change":"2024-05-21T21:51:16.949017+0000","last_active":"2024-05-21T21:52:10.873845+0000","last_peered":"2024-05-21T21:52:10.873845+0000","last_clean":"2024-05-21T21:52:10.873845+0000","last_became_active":"2024-05-21T21:51:16.948691+0000","last_became_peered":"2024-05-21T21:51:16.948691+0000","last_unstale":"2024-05-21T21:52:10.873845+0000","last_undegraded":"2024-05-21T21:52:10.873845+0000","last_fullsized":"2024-05-21T21:52:10.873845+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:32:06.606581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.973164+0000","last_change":"2024-05-21T21:51:12.939203+0000","last_active":"2024-05-21T21:52:10.973164+0000","last_peered":"2024-05-21T21:52:10.973164+0000","last_clean":"2024-05-21T21:52:10.973164+0000","last_became_active":"2024-05-21T21:51:12.938936+0000","last_became_peered":"2024-05-21T21:51:12.938936+0000","last_unstale":"2024-05-21T21:52:10.973164+0000","last_undegraded":"2024-05-21T21:52:10.973164+0000","last_fullsized":"2024-05-21T21:52:10.973164+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:55:50.336362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.863329+0000","last_change":"2024-05-21T21:51:16.926216+0000","last_active":"2024-05-21T21:52:10.863329+0000","last_peered":"2024-05-21T21:52:10.863329+0000","last_clean":"2024-05-21T21:52:10.863329+0000","last_became_active":"2024-05-21T21:51:16.925834+0000","last_became_peered":"2024-05-21T21:51:16.925834+0000","last_unstale":"2024-05-21T21:52:10.863329+0000","last_undegraded":"2024-05-21T21:52:10.863329+0000","last_fullsized":"2024-05-21T21:52:10.863329+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:13:30.192908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'7","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.866548+0000","last_change":"2024-05-21T21:51:12.855306+0000","last_active":"2024-05-21T21:52:10.866548+0000","last_peered":"2024-05-21T21:52:10.866548+0000","last_clean":"2024-05-21T21:52:10.866548+0000","last_became_active":"2024-05-21T21:51:12.854722+0000","last_became_peered":"2024-05-21T21:51:12.854722+0000","last_unstale":"2024-05-21T21:52:10.866548+0000","last_undegraded":"2024-05-21T21:52:10.866548+0000","last_fullsized":"2024-05-21T21:52:10.866548+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:39:41.583219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.861450+0000","last_change":"2024-05-21T21:51:16.944758+0000","last_active":"2024-05-21T21:52:10.861450+0000","last_peered":"2024-05-21T21:52:10.861450+0000","last_clean":"2024-05-21T21:52:10.861450+0000","last_became_active":"2024-05-21T21:51:16.944393+0000","last_became_peered":"2024-05-21T21:51:16.944393+0000","last_unstale":"2024-05-21T21:52:10.861450+0000","last_undegraded":"2024-05-21T21:52:10.861450+0000","last_fullsized":"2024-05-21T21:52:10.861450+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:01:23.048355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'3","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.863710+0000","last_change":"2024-05-21T21:51:12.860378+0000","last_active":"2024-05-21T21:52:10.863710+0000","last_peered":"2024-05-21T21:52:10.863710+0000","last_clean":"2024-05-21T21:52:10.863710+0000","last_became_active":"2024-05-21T21:51:12.859978+0000","last_became_peered":"2024-05-21T21:51:12.859978+0000","last_unstale":"2024-05-21T21:52:10.863710+0000","last_undegraded":"2024-05-21T21:52:10.863710+0000","last_fullsized":"2024-05-21T21:52:10.863710+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T07:52:26.902232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.867609+0000","last_change":"2024-05-21T21:51:16.943194+0000","last_active":"2024-05-21T21:52:10.867609+0000","last_peered":"2024-05-21T21:52:10.867609+0000","last_clean":"2024-05-21T21:52:10.867609+0000","last_became_active":"2024-05-21T21:51:16.942884+0000","last_became_peered":"2024-05-21T21:51:16.942884+0000","last_unstale":"2024-05-21T21:52:10.867609+0000","last_undegraded":"2024-05-21T21:52:10.867609+0000","last_fullsized":"2024-05-21T21:52:10.867609+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T05:58:17.436995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.863880+0000","last_change":"2024-05-21T21:51:12.853390+0000","last_active":"2024-05-21T21:52:10.863880+0000","last_peered":"2024-05-21T21:52:10.863880+0000","last_clean":"2024-05-21T21:52:10.863880+0000","last_became_active":"2024-05-21T21:51:12.852990+0000","last_became_peered":"2024-05-21T21:51:12.852990+0000","last_unstale":"2024-05-21T21:52:10.863880+0000","last_undegraded":"2024-05-21T21:52:10.863880+0000","last_fullsized":"2024-05-21T21:52:10.863880+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:32:16.392609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.864540+0000","last_change":"2024-05-21T21:51:16.933430+0000","last_active":"2024-05-21T21:52:10.864540+0000","last_peered":"2024-05-21T21:52:10.864540+0000","last_clean":"2024-05-21T21:52:10.864540+0000","last_became_active":"2024-05-21T21:51:16.933053+0000","last_became_peered":"2024-05-21T21:51:16.933053+0000","last_unstale":"2024-05-21T21:52:10.864540+0000","last_undegraded":"2024-05-21T21:52:10.864540+0000","last_fullsized":"2024-05-21T21:52:10.864540+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T06:14:57.468727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'5","reported_seq":70,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:14.423930+0000","last_change":"2024-05-21T21:51:12.946113+0000","last_active":"2024-05-21T21:52:14.423930+0000","last_peered":"2024-05-21T21:52:14.423930+0000","last_clean":"2024-05-21T21:52:14.423930+0000","last_became_active":"2024-05-21T21:51:12.945650+0000","last_became_peered":"2024-05-21T21:51:12.945650+0000","last_unstale":"2024-05-21T21:52:14.423930+0000","last_undegraded":"2024-05-21T21:52:14.423930+0000","last_fullsized":"2024-05-21T21:52:14.423930+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T09:40:18.209400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":39,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.864300+0000","last_change":"2024-05-21T21:51:16.932632+0000","last_active":"2024-05-21T21:52:10.864300+0000","last_peered":"2024-05-21T21:52:10.864300+0000","last_clean":"2024-05-21T21:52:10.864300+0000","last_became_active":"2024-05-21T21:51:16.932256+0000","last_became_peered":"2024-05-21T21:51:16.932256+0000","last_unstale":"2024-05-21T21:52:10.864300+0000","last_undegraded":"2024-05-21T21:52:10.864300+0000","last_fullsized":"2024-05-21T21:52:10.864300+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:04:48.146485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'3","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.867556+0000","last_change":"2024-05-21T21:51:12.860602+0000","last_active":"2024-05-21T21:52:10.867556+0000","last_peered":"2024-05-21T21:52:10.867556+0000","last_clean":"2024-05-21T21:52:10.867556+0000","last_became_active":"2024-05-21T21:51:12.860228+0000","last_became_peered":"2024-05-21T21:51:12.860228+0000","last_unstale":"2024-05-21T21:52:10.867556+0000","last_undegraded":"2024-05-21T21:52:10.867556+0000","last_fullsized":"2024-05-21T21:52:10.867556+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:33:39.230872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'1","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.863947+0000","last_change":"2024-05-21T21:51:16.929616+0000","last_active":"2024-05-21T21:52:10.863947+0000","last_peered":"2024-05-21T21:52:10.863947+0000","last_clean":"2024-05-21T21:52:10.863947+0000","last_became_active":"2024-05-21T21:51:16.928569+0000","last_became_peered":"2024-05-21T21:51:16.928569+0000","last_unstale":"2024-05-21T21:52:10.863947+0000","last_undegraded":"2024-05-21T21:52:10.863947+0000","last_fullsized":"2024-05-21T21:52:10.863947+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:15:41.790344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'5","reported_seq":43,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.881114+0000","last_change":"2024-05-21T21:52:10.881114+0000","last_active":"2024-05-21T21:52:10.881114+0000","last_peered":"2024-05-21T21:52:10.881114+0000","last_clean":"2024-05-21T21:52:10.881114+0000","last_became_active":"2024-05-21T21:52:10.872022+0000","last_became_peered":"2024-05-21T21:52:10.872022+0000","last_unstale":"2024-05-21T21:52:10.881114+0000","last_undegraded":"2024-05-21T21:52:10.881114+0000","last_fullsized":"2024-05-21T21:52:10.881114+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T06:05:51.419156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.864314+0000","last_change":"2024-05-21T21:51:16.926496+0000","last_active":"2024-05-21T21:52:10.864314+0000","last_peered":"2024-05-21T21:52:10.864314+0000","last_clean":"2024-05-21T21:52:10.864314+0000","last_became_active":"2024-05-21T21:51:16.926112+0000","last_became_peered":"2024-05-21T21:51:16.926112+0000","last_unstale":"2024-05-21T21:52:10.864314+0000","last_undegraded":"2024-05-21T21:52:10.864314+0000","last_fullsized":"2024-05-21T21:52:10.864314+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:26:08.254421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.861508+0000","last_change":"2024-05-21T21:51:16.930522+0000","last_active":"2024-05-21T21:52:10.861508+0000","last_peered":"2024-05-21T21:52:10.861508+0000","last_clean":"2024-05-21T21:52:10.861508+0000","last_became_active":"2024-05-21T21:51:16.929853+0000","last_became_peered":"2024-05-21T21:51:16.929853+0000","last_unstale":"2024-05-21T21:52:10.861508+0000","last_undegraded":"2024-05-21T21:52:10.861508+0000","last_fullsized":"2024-05-21T21:52:10.861508+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:10:57.940009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.863593+0000","last_change":"2024-05-21T21:51:16.931751+0000","last_active":"2024-05-21T21:52:10.863593+0000","last_peered":"2024-05-21T21:52:10.863593+0000","last_clean":"2024-05-21T21:52:10.863593+0000","last_became_active":"2024-05-21T21:51:16.931353+0000","last_became_peered":"2024-05-21T21:51:16.931353+0000","last_unstale":"2024-05-21T21:52:10.863593+0000","last_undegraded":"2024-05-21T21:52:10.863593+0000","last_fullsized":"2024-05-21T21:52:10.863593+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:33:45.297850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.863516+0000","last_change":"2024-05-21T21:51:16.940096+0000","last_active":"2024-05-21T21:52:10.863516+0000","last_peered":"2024-05-21T21:52:10.863516+0000","last_clean":"2024-05-21T21:52:10.863516+0000","last_became_active":"2024-05-21T21:51:16.939740+0000","last_became_peered":"2024-05-21T21:51:16.939740+0000","last_unstale":"2024-05-21T21:52:10.863516+0000","last_undegraded":"2024-05-21T21:52:10.863516+0000","last_fullsized":"2024-05-21T21:52:10.863516+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:59:52.569526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.872739+0000","last_change":"2024-05-21T21:51:16.946488+0000","last_active":"2024-05-21T21:52:10.872739+0000","last_peered":"2024-05-21T21:52:10.872739+0000","last_clean":"2024-05-21T21:52:10.872739+0000","last_became_active":"2024-05-21T21:51:16.945957+0000","last_became_peered":"2024-05-21T21:51:16.945957+0000","last_unstale":"2024-05-21T21:52:10.872739+0000","last_undegraded":"2024-05-21T21:52:10.872739+0000","last_fullsized":"2024-05-21T21:52:10.872739+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:40:36.708344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.862107+0000","last_change":"2024-05-21T21:51:16.949349+0000","last_active":"2024-05-21T21:52:10.862107+0000","last_peered":"2024-05-21T21:52:10.862107+0000","last_clean":"2024-05-21T21:52:10.862107+0000","last_became_active":"2024-05-21T21:51:16.946986+0000","last_became_peered":"2024-05-21T21:51:16.946986+0000","last_unstale":"2024-05-21T21:52:10.862107+0000","last_undegraded":"2024-05-21T21:52:10.862107+0000","last_fullsized":"2024-05-21T21:52:10.862107+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:01:14.877110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.861685+0000","last_change":"2024-05-21T21:51:16.929172+0000","last_active":"2024-05-21T21:52:10.861685+0000","last_peered":"2024-05-21T21:52:10.861685+0000","last_clean":"2024-05-21T21:52:10.861685+0000","last_became_active":"2024-05-21T21:51:16.928823+0000","last_became_peered":"2024-05-21T21:51:16.928823+0000","last_unstale":"2024-05-21T21:52:10.861685+0000","last_undegraded":"2024-05-21T21:52:10.861685+0000","last_fullsized":"2024-05-21T21:52:10.861685+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:09:13.719272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.863847+0000","last_change":"2024-05-21T21:51:16.930332+0000","last_active":"2024-05-21T21:52:10.863847+0000","last_peered":"2024-05-21T21:52:10.863847+0000","last_clean":"2024-05-21T21:52:10.863847+0000","last_became_active":"2024-05-21T21:51:16.930024+0000","last_became_peered":"2024-05-21T21:51:16.930024+0000","last_unstale":"2024-05-21T21:52:10.863847+0000","last_undegraded":"2024-05-21T21:52:10.863847+0000","last_fullsized":"2024-05-21T21:52:10.863847+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:47:41.272725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.868573+0000","last_change":"2024-05-21T21:51:16.930102+0000","last_active":"2024-05-21T21:52:10.868573+0000","last_peered":"2024-05-21T21:52:10.868573+0000","last_clean":"2024-05-21T21:52:10.868573+0000","last_became_active":"2024-05-21T21:51:16.929410+0000","last_became_peered":"2024-05-21T21:51:16.929410+0000","last_unstale":"2024-05-21T21:52:10.868573+0000","last_undegraded":"2024-05-21T21:52:10.868573+0000","last_fullsized":"2024-05-21T21:52:10.868573+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:11:46.296288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.873829+0000","last_change":"2024-05-21T21:52:10.873424+0000","last_active":"2024-05-21T21:52:10.873829+0000","last_peered":"2024-05-21T21:52:10.873829+0000","last_clean":"2024-05-21T21:52:10.873829+0000","last_became_active":"2024-05-21T21:52:10.873118+0000","last_became_peered":"2024-05-21T21:52:10.873118+0000","last_unstale":"2024-05-21T21:52:10.873829+0000","last_undegraded":"2024-05-21T21:52:10.873829+0000","last_fullsized":"2024-05-21T21:52:10.873829+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:05:00.731013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,3],"acting":[8,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.866413+0000","last_change":"2024-05-21T21:51:16.934442+0000","last_active":"2024-05-21T21:52:10.866413+0000","last_peered":"2024-05-21T21:52:10.866413+0000","last_clean":"2024-05-21T21:52:10.866413+0000","last_became_active":"2024-05-21T21:51:16.934086+0000","last_became_peered":"2024-05-21T21:51:16.934086+0000","last_unstale":"2024-05-21T21:52:10.866413+0000","last_undegraded":"2024-05-21T21:52:10.866413+0000","last_fullsized":"2024-05-21T21:52:10.866413+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:36:44.302532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.972750+0000","last_change":"2024-05-21T21:51:16.924479+0000","last_active":"2024-05-21T21:52:10.972750+0000","last_peered":"2024-05-21T21:52:10.972750+0000","last_clean":"2024-05-21T21:52:10.972750+0000","last_became_active":"2024-05-21T21:51:16.923998+0000","last_became_peered":"2024-05-21T21:51:16.923998+0000","last_unstale":"2024-05-21T21:52:10.972750+0000","last_undegraded":"2024-05-21T21:52:10.972750+0000","last_fullsized":"2024-05-21T21:52:10.972750+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:23:07.931731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.862486+0000","last_change":"2024-05-21T21:51:16.947271+0000","last_active":"2024-05-21T21:52:10.862486+0000","last_peered":"2024-05-21T21:52:10.862486+0000","last_clean":"2024-05-21T21:52:10.862486+0000","last_became_active":"2024-05-21T21:51:16.946912+0000","last_became_peered":"2024-05-21T21:51:16.946912+0000","last_unstale":"2024-05-21T21:52:10.862486+0000","last_undegraded":"2024-05-21T21:52:10.862486+0000","last_fullsized":"2024-05-21T21:52:10.862486+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:51:29.646915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.861507+0000","last_change":"2024-05-21T21:51:16.928852+0000","last_active":"2024-05-21T21:52:10.861507+0000","last_peered":"2024-05-21T21:52:10.861507+0000","last_clean":"2024-05-21T21:52:10.861507+0000","last_became_active":"2024-05-21T21:51:16.928480+0000","last_became_peered":"2024-05-21T21:51:16.928480+0000","last_unstale":"2024-05-21T21:52:10.861507+0000","last_undegraded":"2024-05-21T21:52:10.861507+0000","last_fullsized":"2024-05-21T21:52:10.861507+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T09:12:30.691254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66966,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":415,"num_read_kb":278,"num_write":245,"num_write_kb":135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":112,"num_keys_recovered":17,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":200898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":210,"ondisk_log_size":210,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645341,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":433959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841569,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28588,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716564,"statfs":{"total":95995035648,"available":95965761536,"internally_reserved":0,"allocated":1601536,"data_stored":1085141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005094,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":929792,"data_stored":434006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201325,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":921600,"data_stored":434267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397553,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28504,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716648,"statfs":{"total":95995035648,"available":95965847552,"internally_reserved":0,"allocated":1519616,"data_stored":1027176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561079,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":909312,"data_stored":433606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":978944,"data_stored":494393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738457,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36252,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708900,"statfs":{"total":95995035648,"available":95957913600,"internally_reserved":0,"allocated":929792,"data_stored":434945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411407,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36296,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708856,"statfs":{"total":95995035648,"available":95957868544,"internally_reserved":0,"allocated":983040,"data_stored":494780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149768,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36244,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708908,"statfs":{"total":95995035648,"available":95957921792,"internally_reserved":0,"allocated":921600,"data_stored":433314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953539,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":909312,"data_stored":433858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757310,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32676,"kb_used_data":1496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712476,"statfs":{"total":95995035648,"available":95961575424,"internally_reserved":0,"allocated":1531904,"data_stored":1026463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":61626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":62013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":61889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-21T21:52:20.771 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph status --format=json 2024-05-21T21:52:20.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:52:20 smithi151 ceph-mon[31623]: pgmap v347: 97 pgs: 97 active+clean; 642 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 547 B/s wr, 10 op/s; 24 B/s, 1 keys/s, 2 objects/s recovering 2024-05-21T21:52:21.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:52:20 smithi184 ceph-mon[36942]: pgmap v347: 97 pgs: 97 active+clean; 642 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 547 B/s wr, 10 op/s; 24 B/s, 1 keys/s, 2 objects/s recovering 2024-05-21T21:52:21.272 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:52:21.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:52:20 smithi164 ceph-mon[36269]: pgmap v347: 97 pgs: 97 active+clean; 642 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 547 B/s wr, 10 op/s; 24 B/s, 1 keys/s, 2 objects/s recovering 2024-05-21T21:52:22.192 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:52:21 smithi151 ceph-mon[31623]: from='client.24662 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T21:52:22.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:52:21 smithi184 ceph-mon[36942]: from='client.24662 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T21:52:22.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:52:21 smithi164 ceph-mon[36269]: from='client.24662 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T21:52:23.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:52:22 smithi151 ceph-mon[31623]: pgmap v348: 97 pgs: 97 active+clean; 642 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 494 B/s wr, 9 op/s; 22 B/s, 1 keys/s, 2 objects/s recovering 2024-05-21T21:52:23.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:52:22 smithi184 ceph-mon[36942]: pgmap v348: 97 pgs: 97 active+clean; 642 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 494 B/s wr, 9 op/s; 22 B/s, 1 keys/s, 2 objects/s recovering 2024-05-21T21:52:23.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:52:22 smithi164 ceph-mon[36269]: pgmap v348: 97 pgs: 97 active+clean; 642 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 494 B/s wr, 9 op/s; 22 B/s, 1 keys/s, 2 objects/s recovering 2024-05-21T21:52:23.447 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:23.446+0000 7fccf7e00700 1 -- 172.21.15.151:0/2083959863 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fccf8106030 msgr2=0x7fccf8112540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:52:23.447 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:23.446+0000 7fccf7e00700 1 --2- 172.21.15.151:0/2083959863 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fccf8106030 0x7fccf8112540 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fccf000b0a0 tx=0x7fccf000b3b0 comp rx=0 tx=0).stop 2024-05-21T21:52:23.447 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:23.446+0000 7fccf7e00700 1 -- 172.21.15.151:0/2083959863 shutdown_connections 2024-05-21T21:52:23.447 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:23.446+0000 7fccf7e00700 1 --2- 172.21.15.151:0/2083959863 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fccf8106030 0x7fccf8112540 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:23.447 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:23.446+0000 7fccf7e00700 1 --2- 172.21.15.151:0/2083959863 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fccf810b670 0x7fccf810ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:23.447 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:23.446+0000 7fccf7e00700 1 --2- 172.21.15.151:0/2083959863 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fccf8105670 0x7fccf8105af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:23.447 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:23.446+0000 7fccf7e00700 1 -- 172.21.15.151:0/2083959863 >> 172.21.15.151:0/2083959863 conn(0x7fccf8101160 msgr2=0x7fccf8103580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:52:23.447 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:23.446+0000 7fccf7e00700 1 -- 172.21.15.151:0/2083959863 shutdown_connections 2024-05-21T21:52:23.447 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:23.447+0000 7fccf7e00700 1 -- 172.21.15.151:0/2083959863 wait complete. 2024-05-21T21:52:23.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:23.448+0000 7fccf7e00700 1 Processor -- start 2024-05-21T21:52:23.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:23.448+0000 7fccf7e00700 1 -- start start 2024-05-21T21:52:23.449 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:23.449+0000 7fccf7e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fccf8105670 0x7fccf819bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:52:23.449 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:23.449+0000 7fccf7e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fccf8106030 0x7fccf819c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:52:23.449 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:23.449+0000 7fccf7e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fccf810b670 0x7fccf81a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:52:23.449 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:23.449+0000 7fccf7e00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fccf81158c0 con 0x7fccf8106030 2024-05-21T21:52:23.449 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:23.449+0000 7fccf7e00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fccf8115740 con 0x7fccf8105670 2024-05-21T21:52:23.449 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:23.449+0000 7fccf7e00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fccf8115a40 con 0x7fccf810b670 2024-05-21T21:52:23.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:23.449+0000 7fccf6a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fccf8105670 0x7fccf819bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:52:23.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:23.449+0000 7fccf6000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fccf8106030 0x7fccf819c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:52:23.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:23.449+0000 7fccf7400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fccf810b670 0x7fccf81a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:52:23.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:23.449+0000 7fccf6a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fccf8105670 0x7fccf819bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:46046/0 (socket says 172.21.15.151:46046) 2024-05-21T21:52:23.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:23.449+0000 7fccf6a00700 1 -- 172.21.15.151:0/2459207648 learned_addr learned my addr 172.21.15.151:0/2459207648 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:52:23.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:23.450+0000 7fccf6a00700 1 -- 172.21.15.151:0/2459207648 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fccf810b670 msgr2=0x7fccf81a05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:52:23.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:23.450+0000 7fccf6a00700 1 --2- 172.21.15.151:0/2459207648 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fccf810b670 0x7fccf81a05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:23.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:23.450+0000 7fccf6a00700 1 -- 172.21.15.151:0/2459207648 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fccf8106030 msgr2=0x7fccf819c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:52:23.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:23.450+0000 7fccf6a00700 1 --2- 172.21.15.151:0/2459207648 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fccf8106030 0x7fccf819c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:23.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:23.450+0000 7fccf6a00700 1 -- 172.21.15.151:0/2459207648 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccf81a0d20 con 0x7fccf8105670 2024-05-21T21:52:23.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:23.450+0000 7fccf6a00700 1 --2- 172.21.15.151:0/2459207648 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fccf8105670 0x7fccf819bd10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fcce800e9a0 tx=0x7fcce800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:52:23.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:23.450+0000 7fccef400700 1 -- 172.21.15.151:0/2459207648 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce800ca10 con 0x7fccf8105670 2024-05-21T21:52:23.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:23.450+0000 7fccef400700 1 -- 172.21.15.151:0/2459207648 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcce800cbb0 con 0x7fccf8105670 2024-05-21T21:52:23.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:23.450+0000 7fccef400700 1 -- 172.21.15.151:0/2459207648 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce8018680 con 0x7fccf8105670 2024-05-21T21:52:23.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:23.450+0000 7fccf7e00700 1 -- 172.21.15.151:0/2459207648 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccf81a1010 con 0x7fccf8105670 2024-05-21T21:52:23.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:23.450+0000 7fccf7e00700 1 -- 172.21.15.151:0/2459207648 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccf81a8890 con 0x7fccf8105670 2024-05-21T21:52:23.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:23.451+0000 7fccf7e00700 1 -- 172.21.15.151:0/2459207648 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccf806a230 con 0x7fccf8105670 2024-05-21T21:52:23.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:23.454+0000 7fccef400700 1 -- 172.21.15.151:0/2459207648 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fcce8010430 con 0x7fccf8105670 2024-05-21T21:52:23.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:23.456+0000 7fccef400700 1 --2- 172.21.15.151:0/2459207648 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fccdc0b2910 0x7fccdc0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:52:23.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:23.456+0000 7fccef400700 1 -- 172.21.15.151:0/2459207648 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fcce80e27f0 con 0x7fccf8105670 2024-05-21T21:52:23.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:23.456+0000 7fccf6000700 1 --2- 172.21.15.151:0/2459207648 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fccdc0b2910 0x7fccdc0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:52:23.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:23.457+0000 7fccf6000700 1 --2- 172.21.15.151:0/2459207648 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fccdc0b2910 0x7fccdc0b4dd0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fccf810ac10 tx=0x7fcce4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:52:23.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:23.458+0000 7fccef400700 1 -- 172.21.15.151:0/2459207648 <== mon.2 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcce80aa370 con 0x7fccf8105670 2024-05-21T21:52:23.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:23.785+0000 7fccf7e00700 1 -- 172.21.15.151:0/2459207648 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fccf802e560 con 0x7fccf8105670 2024-05-21T21:52:23.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:23.786+0000 7fccef400700 1 -- 172.21.15.151:0/2459207648 <== mon.2 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+3291 (secure 0 0 0) 0x7fcce80a9c70 con 0x7fccf8105670 2024-05-21T21:52:23.787 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T21:52:23.787 INFO:teuthology.orchestra.run.smithi151.stdout:{"fsid":"c8c7a2e6-17ba-11ef-bc9a-c7b262605968","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","c","b"],"quorum_age":484,"monmap":{"epoch":3,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":82,"num_osds":12,"num_up_osds":12,"osd_up_since":1716328205,"num_in_osds":12,"osd_in_since":1716328185,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":102,"data_bytes":657568,"bytes_used":374898688,"bytes_avail":1151565529088,"bytes_total":1151940427776,"recovering_objects_per_sec":2,"recovering_bytes_per_sec":22,"recovering_keys_per_sec":1,"num_objects_recovered":22,"num_bytes_recovered":229,"num_keys_recovered":17,"read_bytes_sec":5241,"write_bytes_sec":494,"read_op_per_sec":9,"write_op_per_sec":0},"fsmap":{"epoch":28,"id":1,"up":5,"in":5,"max":5,"by_rank":[{"filesystem_id":1,"rank":4,"name":"a","status":"up:standby-replay","gid":14403},{"filesystem_id":1,"rank":3,"name":"l","status":"up:standby-replay","gid":24436},{"filesystem_id":1,"rank":2,"name":"k","status":"up:standby-replay","gid":24443},{"filesystem_id":1,"rank":1,"name":"d","status":"up:standby-replay","gid":24461},{"filesystem_id":1,"rank":0,"name":"f","status":"up:standby-replay","gid":24466},{"filesystem_id":1,"rank":4,"name":"e","status":"up:active","gid":24473},{"filesystem_id":1,"rank":3,"name":"c","status":"up:active","gid":24478},{"filesystem_id":1,"rank":2,"name":"h","status":"up:active","gid":24491},{"filesystem_id":1,"rank":1,"name":"i","status":"up:active","gid":24493},{"filesystem_id":1,"rank":0,"name":"b","status":"up:active","gid":24530}],"up:standby":2},"mgrmap":{"available":true,"num_standbys":2,"modules":["cephadm","dashboard","iostat","nfs","restful"],"services":{"dashboard":"https://172.21.15.151:8443/"}},"servicemap":{"epoch":25,"modified":"2024-05-21T21:51:53.203624+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"e":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"f":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"h":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"i":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"k":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"l":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-05-21T21:52:23.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:23.789+0000 7fccf7e00700 1 -- 172.21.15.151:0/2459207648 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fccdc0b2910 msgr2=0x7fccdc0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:52:23.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:23.789+0000 7fccf7e00700 1 --2- 172.21.15.151:0/2459207648 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fccdc0b2910 0x7fccdc0b4dd0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fccf810ac10 tx=0x7fcce4009260 comp rx=0 tx=0).stop 2024-05-21T21:52:23.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:23.789+0000 7fccf7e00700 1 -- 172.21.15.151:0/2459207648 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fccf8105670 msgr2=0x7fccf819bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:52:23.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:23.789+0000 7fccf7e00700 1 --2- 172.21.15.151:0/2459207648 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fccf8105670 0x7fccf819bd10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fcce800e9a0 tx=0x7fcce800ecb0 comp rx=0 tx=0).stop 2024-05-21T21:52:23.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:23.790+0000 7fccf7e00700 1 -- 172.21.15.151:0/2459207648 shutdown_connections 2024-05-21T21:52:23.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:23.790+0000 7fccf7e00700 1 --2- 172.21.15.151:0/2459207648 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fccdc0b2910 0x7fccdc0b4dd0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:23.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:23.790+0000 7fccf7e00700 1 --2- 172.21.15.151:0/2459207648 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fccf8105670 0x7fccf819bd10 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:23.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:23.790+0000 7fccf7e00700 1 --2- 172.21.15.151:0/2459207648 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fccf810b670 0x7fccf81a05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:23.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:23.790+0000 7fccf7e00700 1 --2- 172.21.15.151:0/2459207648 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fccf8106030 0x7fccf819c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:23.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:23.790+0000 7fccf7e00700 1 -- 172.21.15.151:0/2459207648 >> 172.21.15.151:0/2459207648 conn(0x7fccf8101160 msgr2=0x7fccf81016a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:52:23.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:23.790+0000 7fccf7e00700 1 -- 172.21.15.151:0/2459207648 shutdown_connections 2024-05-21T21:52:23.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:23.790+0000 7fccf7e00700 1 -- 172.21.15.151:0/2459207648 wait complete. 2024-05-21T21:52:24.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:52:23 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/2459207648' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-05-21T21:52:24.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:52:23 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/2459207648' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-05-21T21:52:24.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:52:23 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/2459207648' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-05-21T21:52:24.611 INFO:tasks.cephadm.ceph_manager.ceph:making progress, resetting timeout 2024-05-21T21:52:24.611 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph pg dump --format=json 2024-05-21T21:52:25.071 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:52:25.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:52:24 smithi151 ceph-mon[31623]: pgmap v349: 97 pgs: 97 active+clean; 642 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 426 B/s wr, 10 op/s; 9 B/s, 1 keys/s, 1 objects/s recovering 2024-05-21T21:52:25.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:52:24 smithi184 ceph-mon[36942]: pgmap v349: 97 pgs: 97 active+clean; 642 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 426 B/s wr, 10 op/s; 9 B/s, 1 keys/s, 1 objects/s recovering 2024-05-21T21:52:25.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:52:24 smithi164 ceph-mon[36269]: pgmap v349: 97 pgs: 97 active+clean; 642 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 426 B/s wr, 10 op/s; 9 B/s, 1 keys/s, 1 objects/s recovering 2024-05-21T21:52:27.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:27.079+0000 7fa1ee600700 1 -- 172.21.15.151:0/1079773543 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa1e8105670 msgr2=0x7fa1e8105af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:52:27.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:27.079+0000 7fa1ee600700 1 --2- 172.21.15.151:0/1079773543 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa1e8105670 0x7fa1e8105af0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa1d0009990 tx=0x7fa1d0009ca0 comp rx=0 tx=0).stop 2024-05-21T21:52:27.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:27.079+0000 7fa1ee600700 1 -- 172.21.15.151:0/1079773543 shutdown_connections 2024-05-21T21:52:27.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:27.079+0000 7fa1ee600700 1 --2- 172.21.15.151:0/1079773543 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa1e810b670 0x7fa1e810ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:27.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:27.079+0000 7fa1ee600700 1 --2- 172.21.15.151:0/1079773543 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1e8106030 0x7fa1e8112540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:27.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:27.079+0000 7fa1ee600700 1 --2- 172.21.15.151:0/1079773543 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa1e8105670 0x7fa1e8105af0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:27.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:27.079+0000 7fa1ee600700 1 -- 172.21.15.151:0/1079773543 >> 172.21.15.151:0/1079773543 conn(0x7fa1e8101160 msgr2=0x7fa1e8103580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:52:27.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:27.079+0000 7fa1ee600700 1 -- 172.21.15.151:0/1079773543 shutdown_connections 2024-05-21T21:52:27.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:27.080+0000 7fa1ee600700 1 -- 172.21.15.151:0/1079773543 wait complete. 2024-05-21T21:52:27.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:27.081+0000 7fa1ee600700 1 Processor -- start 2024-05-21T21:52:27.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:27.081+0000 7fa1ee600700 1 -- start start 2024-05-21T21:52:27.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:27.081+0000 7fa1ee600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa1e8105670 0x7fa1e819bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:52:27.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:27.081+0000 7fa1ee600700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa1e8106030 0x7fa1e819c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:52:27.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:27.081+0000 7fa1ee600700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1e810b670 0x7fa1e81a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:52:27.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:27.081+0000 7fa1ee600700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e81158b0 con 0x7fa1e8105670 2024-05-21T21:52:27.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:27.081+0000 7fa1ee600700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e8115730 con 0x7fa1e8106030 2024-05-21T21:52:27.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:27.081+0000 7fa1ee600700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e8115a30 con 0x7fa1e810b670 2024-05-21T21:52:27.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:27.081+0000 7fa1ed200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa1e8105670 0x7fa1e819bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:52:27.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:27.081+0000 7fa1ed200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa1e8105670 0x7fa1e819bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:47144/0 (socket says 172.21.15.151:47144) 2024-05-21T21:52:27.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:27.081+0000 7fa1ed200700 1 -- 172.21.15.151:0/596384247 learned_addr learned my addr 172.21.15.151:0/596384247 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:52:27.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:27.082+0000 7fa1dfe00700 1 --2- 172.21.15.151:0/596384247 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa1e8106030 0x7fa1e819c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:52:27.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:27.082+0000 7fa1edc00700 1 --2- 172.21.15.151:0/596384247 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1e810b670 0x7fa1e81a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:52:27.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:27.082+0000 7fa1ed200700 1 -- 172.21.15.151:0/596384247 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1e810b670 msgr2=0x7fa1e81a05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:52:27.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:27.082+0000 7fa1ed200700 1 --2- 172.21.15.151:0/596384247 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1e810b670 0x7fa1e81a05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:27.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:27.082+0000 7fa1ed200700 1 -- 172.21.15.151:0/596384247 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa1e8106030 msgr2=0x7fa1e819c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:52:27.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:27.082+0000 7fa1ed200700 1 --2- 172.21.15.151:0/596384247 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa1e8106030 0x7fa1e819c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:27.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:27.082+0000 7fa1ed200700 1 -- 172.21.15.151:0/596384247 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1e81a0d30 con 0x7fa1e8105670 2024-05-21T21:52:27.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:27.082+0000 7fa1dfe00700 1 --2- 172.21.15.151:0/596384247 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa1e8106030 0x7fa1e819c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:52:27.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:27.082+0000 7fa1ed200700 1 --2- 172.21.15.151:0/596384247 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa1e8105670 0x7fa1e819bd20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa1d800e9a0 tx=0x7fa1d800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:52:27.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:27.083+0000 7fa1e6000700 1 -- 172.21.15.151:0/596384247 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d800ca10 con 0x7fa1e8105670 2024-05-21T21:52:27.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:27.083+0000 7fa1e6000700 1 -- 172.21.15.151:0/596384247 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1d800cbb0 con 0x7fa1e8105670 2024-05-21T21:52:27.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:27.083+0000 7fa1ee600700 1 -- 172.21.15.151:0/596384247 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1e81a1020 con 0x7fa1e8105670 2024-05-21T21:52:27.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:27.083+0000 7fa1e6000700 1 -- 172.21.15.151:0/596384247 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d8018680 con 0x7fa1e8105670 2024-05-21T21:52:27.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:27.083+0000 7fa1ee600700 1 -- 172.21.15.151:0/596384247 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1e81a88a0 con 0x7fa1e8105670 2024-05-21T21:52:27.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:27.084+0000 7fa1ee600700 1 -- 172.21.15.151:0/596384247 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1e806a230 con 0x7fa1e8105670 2024-05-21T21:52:27.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:27.086+0000 7fa1e6000700 1 -- 172.21.15.151:0/596384247 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fa1d8010430 con 0x7fa1e8105670 2024-05-21T21:52:27.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:27.088+0000 7fa1e6000700 1 --2- 172.21.15.151:0/596384247 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fa1cc0b2990 0x7fa1cc0b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:52:27.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:27.088+0000 7fa1e6000700 1 -- 172.21.15.151:0/596384247 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fa1d80e2460 con 0x7fa1e8105670 2024-05-21T21:52:27.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:27.088+0000 7fa1dfe00700 1 --2- 172.21.15.151:0/596384247 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fa1cc0b2990 0x7fa1cc0b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:52:27.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:27.089+0000 7fa1dfe00700 1 --2- 172.21.15.151:0/596384247 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fa1cc0b2990 0x7fa1cc0b4e50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa1d00098e0 tx=0x7fa1d0005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:52:27.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:27.092+0000 7fa1e6000700 1 -- 172.21.15.151:0/596384247 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa1d80a9fe0 con 0x7fa1e8105670 2024-05-21T21:52:27.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:52:26 smithi151 ceph-mon[31623]: pgmap v350: 97 pgs: 97 active+clean; 642 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-21T21:52:27.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:52:26 smithi184 ceph-mon[36942]: pgmap v350: 97 pgs: 97 active+clean; 642 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-21T21:52:27.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:52:26 smithi164 ceph-mon[36269]: pgmap v350: 97 pgs: 97 active+clean; 642 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-21T21:52:27.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:27.323+0000 7fa1ee600700 1 -- 172.21.15.151:0/596384247 --> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa1e819d080 con 0x7fa1cc0b2990 2024-05-21T21:52:27.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:27.343+0000 7fa1e6000700 1 -- 172.21.15.151:0/596384247 <== mgr.14152 v2:172.21.15.151:6800/2796622483 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258764 (secure 0 0 0) 0x7fa1d00652e0 con 0x7fa1cc0b2990 2024-05-21T21:52:27.345 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T21:52:27.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:27.349+0000 7fa1ee600700 1 -- 172.21.15.151:0/596384247 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fa1cc0b2990 msgr2=0x7fa1cc0b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:52:27.350 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:27.349+0000 7fa1ee600700 1 --2- 172.21.15.151:0/596384247 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fa1cc0b2990 0x7fa1cc0b4e50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa1d00098e0 tx=0x7fa1d0005960 comp rx=0 tx=0).stop 2024-05-21T21:52:27.350 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:27.349+0000 7fa1ee600700 1 -- 172.21.15.151:0/596384247 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa1e8105670 msgr2=0x7fa1e819bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:52:27.350 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:27.349+0000 7fa1ee600700 1 --2- 172.21.15.151:0/596384247 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa1e8105670 0x7fa1e819bd20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa1d800e9a0 tx=0x7fa1d800ecb0 comp rx=0 tx=0).stop 2024-05-21T21:52:27.350 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:27.350+0000 7fa1ee600700 1 -- 172.21.15.151:0/596384247 shutdown_connections 2024-05-21T21:52:27.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:27.350+0000 7fa1ee600700 1 --2- 172.21.15.151:0/596384247 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fa1cc0b2990 0x7fa1cc0b4e50 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:27.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:27.350+0000 7fa1ee600700 1 --2- 172.21.15.151:0/596384247 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1e810b670 0x7fa1e81a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:27.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:27.350+0000 7fa1ee600700 1 --2- 172.21.15.151:0/596384247 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa1e8105670 0x7fa1e819bd20 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:27.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:27.350+0000 7fa1ee600700 1 --2- 172.21.15.151:0/596384247 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa1e8106030 0x7fa1e819c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:27.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:27.350+0000 7fa1ee600700 1 -- 172.21.15.151:0/596384247 >> 172.21.15.151:0/596384247 conn(0x7fa1e8101160 msgr2=0x7fa1e8101930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:52:27.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:27.350+0000 7fa1ee600700 1 -- 172.21.15.151:0/596384247 shutdown_connections 2024-05-21T21:52:27.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:27.350+0000 7fa1ee600700 1 -- 172.21.15.151:0/596384247 wait complete. 2024-05-21T21:52:27.352 INFO:teuthology.orchestra.run.smithi151.stderr:dumped all 2024-05-21T21:52:28.100 INFO:teuthology.orchestra.run.smithi151.stdout:{"pg_ready":true,"pg_map":{"version":351,"stamp":"2024-05-21T21:52:27.217951+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657568,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":586,"num_read_kb":397,"num_write":370,"num_write_kb":2287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":2952069,"num_keys_recovered":17,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":304,"ondisk_log_size":304,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":366112,"kb_used_data":12732,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124575712,"statfs":{"total":1151940427776,"available":1151565529088,"internally_reserved":0,"allocated":13037568,"data_stored":7165908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19140,"internal_metadata":361674044},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":130,"num_read_kb":65,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004798"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.861691+0000","last_change":"2024-05-21T21:51:16.945132+0000","last_active":"2024-05-21T21:52:10.861691+0000","last_peered":"2024-05-21T21:52:10.861691+0000","last_clean":"2024-05-21T21:52:10.861691+0000","last_became_active":"2024-05-21T21:51:16.944766+0000","last_became_peered":"2024-05-21T21:51:16.944766+0000","last_unstale":"2024-05-21T21:52:10.861691+0000","last_undegraded":"2024-05-21T21:52:10.861691+0000","last_fullsized":"2024-05-21T21:52:10.861691+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:54:04.185095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.873699+0000","last_change":"2024-05-21T21:51:16.939291+0000","last_active":"2024-05-21T21:52:10.873699+0000","last_peered":"2024-05-21T21:52:10.873699+0000","last_clean":"2024-05-21T21:52:10.873699+0000","last_became_active":"2024-05-21T21:51:16.938730+0000","last_became_peered":"2024-05-21T21:51:16.938730+0000","last_unstale":"2024-05-21T21:52:10.873699+0000","last_undegraded":"2024-05-21T21:52:10.873699+0000","last_fullsized":"2024-05-21T21:52:10.873699+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:05:30.623744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.873014+0000","last_change":"2024-05-21T21:51:16.941565+0000","last_active":"2024-05-21T21:52:10.873014+0000","last_peered":"2024-05-21T21:52:10.873014+0000","last_clean":"2024-05-21T21:52:10.873014+0000","last_became_active":"2024-05-21T21:51:16.941033+0000","last_became_peered":"2024-05-21T21:51:16.941033+0000","last_unstale":"2024-05-21T21:52:10.873014+0000","last_undegraded":"2024-05-21T21:52:10.873014+0000","last_fullsized":"2024-05-21T21:52:10.873014+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:04:55.789479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.865221+0000","last_change":"2024-05-21T21:51:16.928455+0000","last_active":"2024-05-21T21:52:10.865221+0000","last_peered":"2024-05-21T21:52:10.865221+0000","last_clean":"2024-05-21T21:52:10.865221+0000","last_became_active":"2024-05-21T21:51:16.927874+0000","last_became_peered":"2024-05-21T21:51:16.927874+0000","last_unstale":"2024-05-21T21:52:10.865221+0000","last_undegraded":"2024-05-21T21:52:10.865221+0000","last_fullsized":"2024-05-21T21:52:10.865221+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:25:16.483175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.863776+0000","last_change":"2024-05-21T21:51:16.937773+0000","last_active":"2024-05-21T21:52:10.863776+0000","last_peered":"2024-05-21T21:52:10.863776+0000","last_clean":"2024-05-21T21:52:10.863776+0000","last_became_active":"2024-05-21T21:51:16.937306+0000","last_became_peered":"2024-05-21T21:51:16.937306+0000","last_unstale":"2024-05-21T21:52:10.863776+0000","last_undegraded":"2024-05-21T21:52:10.863776+0000","last_fullsized":"2024-05-21T21:52:10.863776+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:05:03.762390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.865632+0000","last_change":"2024-05-21T21:51:16.948275+0000","last_active":"2024-05-21T21:52:10.865632+0000","last_peered":"2024-05-21T21:52:10.865632+0000","last_clean":"2024-05-21T21:52:10.865632+0000","last_became_active":"2024-05-21T21:51:16.947890+0000","last_became_peered":"2024-05-21T21:51:16.947890+0000","last_unstale":"2024-05-21T21:52:10.865632+0000","last_undegraded":"2024-05-21T21:52:10.865632+0000","last_fullsized":"2024-05-21T21:52:10.865632+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:08:38.572177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.864396+0000","last_change":"2024-05-21T21:51:16.946821+0000","last_active":"2024-05-21T21:52:10.864396+0000","last_peered":"2024-05-21T21:52:10.864396+0000","last_clean":"2024-05-21T21:52:10.864396+0000","last_became_active":"2024-05-21T21:51:16.946515+0000","last_became_peered":"2024-05-21T21:51:16.946515+0000","last_unstale":"2024-05-21T21:52:10.864396+0000","last_undegraded":"2024-05-21T21:52:10.864396+0000","last_fullsized":"2024-05-21T21:52:10.864396+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T09:26:25.867895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.870476+0000","last_change":"2024-05-21T21:51:16.929228+0000","last_active":"2024-05-21T21:52:10.870476+0000","last_peered":"2024-05-21T21:52:10.870476+0000","last_clean":"2024-05-21T21:52:10.870476+0000","last_became_active":"2024-05-21T21:51:16.928875+0000","last_became_peered":"2024-05-21T21:51:16.928875+0000","last_unstale":"2024-05-21T21:52:10.870476+0000","last_undegraded":"2024-05-21T21:52:10.870476+0000","last_fullsized":"2024-05-21T21:52:10.870476+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:50:56.431310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.973145+0000","last_change":"2024-05-21T21:51:16.932360+0000","last_active":"2024-05-21T21:52:10.973145+0000","last_peered":"2024-05-21T21:52:10.973145+0000","last_clean":"2024-05-21T21:52:10.973145+0000","last_became_active":"2024-05-21T21:51:16.932036+0000","last_became_peered":"2024-05-21T21:51:16.932036+0000","last_unstale":"2024-05-21T21:52:10.973145+0000","last_undegraded":"2024-05-21T21:52:10.973145+0000","last_fullsized":"2024-05-21T21:52:10.973145+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T09:06:37.257323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.876556+0000","last_change":"2024-05-21T21:52:10.876556+0000","last_active":"2024-05-21T21:52:10.876556+0000","last_peered":"2024-05-21T21:52:10.876556+0000","last_clean":"2024-05-21T21:52:10.876556+0000","last_became_active":"2024-05-21T21:52:10.876273+0000","last_became_peered":"2024-05-21T21:52:10.876273+0000","last_unstale":"2024-05-21T21:52:10.876556+0000","last_undegraded":"2024-05-21T21:52:10.876556+0000","last_fullsized":"2024-05-21T21:52:10.876556+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T05:09:40.037731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.864226+0000","last_change":"2024-05-21T21:51:16.933425+0000","last_active":"2024-05-21T21:52:10.864226+0000","last_peered":"2024-05-21T21:52:10.864226+0000","last_clean":"2024-05-21T21:52:10.864226+0000","last_became_active":"2024-05-21T21:51:16.933057+0000","last_became_peered":"2024-05-21T21:51:16.933057+0000","last_unstale":"2024-05-21T21:52:10.864226+0000","last_undegraded":"2024-05-21T21:52:10.864226+0000","last_fullsized":"2024-05-21T21:52:10.864226+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:04:57.237841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.865179+0000","last_change":"2024-05-21T21:51:16.927014+0000","last_active":"2024-05-21T21:52:10.865179+0000","last_peered":"2024-05-21T21:52:10.865179+0000","last_clean":"2024-05-21T21:52:10.865179+0000","last_became_active":"2024-05-21T21:51:16.925954+0000","last_became_peered":"2024-05-21T21:51:16.925954+0000","last_unstale":"2024-05-21T21:52:10.865179+0000","last_undegraded":"2024-05-21T21:52:10.865179+0000","last_fullsized":"2024-05-21T21:52:10.865179+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:19:20.530727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.862761+0000","last_change":"2024-05-21T21:51:16.930909+0000","last_active":"2024-05-21T21:52:10.862761+0000","last_peered":"2024-05-21T21:52:10.862761+0000","last_clean":"2024-05-21T21:52:10.862761+0000","last_became_active":"2024-05-21T21:51:16.927110+0000","last_became_peered":"2024-05-21T21:51:16.927110+0000","last_unstale":"2024-05-21T21:52:10.862761+0000","last_undegraded":"2024-05-21T21:52:10.862761+0000","last_fullsized":"2024-05-21T21:52:10.862761+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T06:23:55.386047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.872086+0000","last_change":"2024-05-21T21:51:16.938797+0000","last_active":"2024-05-21T21:52:10.872086+0000","last_peered":"2024-05-21T21:52:10.872086+0000","last_clean":"2024-05-21T21:52:10.872086+0000","last_became_active":"2024-05-21T21:51:16.938263+0000","last_became_peered":"2024-05-21T21:51:16.938263+0000","last_unstale":"2024-05-21T21:52:10.872086+0000","last_undegraded":"2024-05-21T21:52:10.872086+0000","last_fullsized":"2024-05-21T21:52:10.872086+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:37:40.521118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.871620+0000","last_change":"2024-05-21T21:51:16.944160+0000","last_active":"2024-05-21T21:52:10.871620+0000","last_peered":"2024-05-21T21:52:10.871620+0000","last_clean":"2024-05-21T21:52:10.871620+0000","last_became_active":"2024-05-21T21:51:16.940046+0000","last_became_peered":"2024-05-21T21:51:16.940046+0000","last_unstale":"2024-05-21T21:52:10.871620+0000","last_undegraded":"2024-05-21T21:52:10.871620+0000","last_fullsized":"2024-05-21T21:52:10.871620+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:14:13.278799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.973152+0000","last_change":"2024-05-21T21:51:16.945599+0000","last_active":"2024-05-21T21:52:10.973152+0000","last_peered":"2024-05-21T21:52:10.973152+0000","last_clean":"2024-05-21T21:52:10.973152+0000","last_became_active":"2024-05-21T21:51:16.945196+0000","last_became_peered":"2024-05-21T21:51:16.945196+0000","last_unstale":"2024-05-21T21:52:10.973152+0000","last_undegraded":"2024-05-21T21:52:10.973152+0000","last_fullsized":"2024-05-21T21:52:10.973152+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:59:24.453615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.861779+0000","last_change":"2024-05-21T21:51:16.944895+0000","last_active":"2024-05-21T21:52:10.861779+0000","last_peered":"2024-05-21T21:52:10.861779+0000","last_clean":"2024-05-21T21:52:10.861779+0000","last_became_active":"2024-05-21T21:51:16.944560+0000","last_became_peered":"2024-05-21T21:51:16.944560+0000","last_unstale":"2024-05-21T21:52:10.861779+0000","last_undegraded":"2024-05-21T21:52:10.861779+0000","last_fullsized":"2024-05-21T21:52:10.861779+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:53:13.264701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.877457+0000","last_change":"2024-05-21T21:51:16.943861+0000","last_active":"2024-05-21T21:52:10.877457+0000","last_peered":"2024-05-21T21:52:10.877457+0000","last_clean":"2024-05-21T21:52:10.877457+0000","last_became_active":"2024-05-21T21:51:16.943123+0000","last_became_peered":"2024-05-21T21:51:16.943123+0000","last_unstale":"2024-05-21T21:52:10.877457+0000","last_undegraded":"2024-05-21T21:52:10.877457+0000","last_fullsized":"2024-05-21T21:52:10.877457+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:35:37.732855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.864479+0000","last_change":"2024-05-21T21:51:16.940580+0000","last_active":"2024-05-21T21:52:10.864479+0000","last_peered":"2024-05-21T21:52:10.864479+0000","last_clean":"2024-05-21T21:52:10.864479+0000","last_became_active":"2024-05-21T21:51:16.940264+0000","last_became_peered":"2024-05-21T21:51:16.940264+0000","last_unstale":"2024-05-21T21:52:10.864479+0000","last_undegraded":"2024-05-21T21:52:10.864479+0000","last_fullsized":"2024-05-21T21:52:10.864479+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:54:22.915541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"80'13","reported_seq":86,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:23.467605+0000","last_change":"2024-05-21T21:51:12.852935+0000","last_active":"2024-05-21T21:52:23.467605+0000","last_peered":"2024-05-21T21:52:23.467605+0000","last_clean":"2024-05-21T21:52:23.467605+0000","last_became_active":"2024-05-21T21:51:12.851802+0000","last_became_peered":"2024-05-21T21:51:12.851802+0000","last_unstale":"2024-05-21T21:52:23.467605+0000","last_undegraded":"2024-05-21T21:52:23.467605+0000","last_fullsized":"2024-05-21T21:52:23.467605+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:13:24.643408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":47,"num_read_kb":47,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.868253+0000","last_change":"2024-05-21T21:51:16.939461+0000","last_active":"2024-05-21T21:52:10.868253+0000","last_peered":"2024-05-21T21:52:10.868253+0000","last_clean":"2024-05-21T21:52:10.868253+0000","last_became_active":"2024-05-21T21:51:16.936322+0000","last_became_peered":"2024-05-21T21:51:16.936322+0000","last_unstale":"2024-05-21T21:52:10.868253+0000","last_undegraded":"2024-05-21T21:52:10.868253+0000","last_fullsized":"2024-05-21T21:52:10.868253+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:37:26.153963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.868674+0000","last_change":"2024-05-21T21:51:12.941838+0000","last_active":"2024-05-21T21:52:10.868674+0000","last_peered":"2024-05-21T21:52:10.868674+0000","last_clean":"2024-05-21T21:52:10.868674+0000","last_became_active":"2024-05-21T21:51:12.940818+0000","last_became_peered":"2024-05-21T21:51:12.940818+0000","last_unstale":"2024-05-21T21:52:10.868674+0000","last_undegraded":"2024-05-21T21:52:10.868674+0000","last_fullsized":"2024-05-21T21:52:10.868674+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:30:42.349241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.873994+0000","last_change":"2024-05-21T21:51:16.950252+0000","last_active":"2024-05-21T21:52:10.873994+0000","last_peered":"2024-05-21T21:52:10.873994+0000","last_clean":"2024-05-21T21:52:10.873994+0000","last_became_active":"2024-05-21T21:51:16.947375+0000","last_became_peered":"2024-05-21T21:51:16.947375+0000","last_unstale":"2024-05-21T21:52:10.873994+0000","last_undegraded":"2024-05-21T21:52:10.873994+0000","last_fullsized":"2024-05-21T21:52:10.873994+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:27:30.462799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"80'12","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.876122+0000","last_change":"2024-05-21T21:52:10.875739+0000","last_active":"2024-05-21T21:52:10.876122+0000","last_peered":"2024-05-21T21:52:10.876122+0000","last_clean":"2024-05-21T21:52:10.876122+0000","last_became_active":"2024-05-21T21:52:10.865846+0000","last_became_peered":"2024-05-21T21:52:10.865846+0000","last_unstale":"2024-05-21T21:52:10.876122+0000","last_undegraded":"2024-05-21T21:52:10.876122+0000","last_fullsized":"2024-05-21T21:52:10.876122+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T06:48:06.465757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.972759+0000","last_change":"2024-05-21T21:51:16.926380+0000","last_active":"2024-05-21T21:52:10.972759+0000","last_peered":"2024-05-21T21:52:10.972759+0000","last_clean":"2024-05-21T21:52:10.972759+0000","last_became_active":"2024-05-21T21:51:16.925821+0000","last_became_peered":"2024-05-21T21:51:16.925821+0000","last_unstale":"2024-05-21T21:52:10.972759+0000","last_undegraded":"2024-05-21T21:52:10.972759+0000","last_fullsized":"2024-05-21T21:52:10.972759+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:42:14.887155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'5","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.864272+0000","last_change":"2024-05-21T21:51:12.854342+0000","last_active":"2024-05-21T21:52:10.864272+0000","last_peered":"2024-05-21T21:52:10.864272+0000","last_clean":"2024-05-21T21:52:10.864272+0000","last_became_active":"2024-05-21T21:51:12.853908+0000","last_became_peered":"2024-05-21T21:51:12.853908+0000","last_unstale":"2024-05-21T21:52:10.864272+0000","last_undegraded":"2024-05-21T21:52:10.864272+0000","last_fullsized":"2024-05-21T21:52:10.864272+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:35:10.374749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.973816+0000","last_change":"2024-05-21T21:52:10.973816+0000","last_active":"2024-05-21T21:52:10.973816+0000","last_peered":"2024-05-21T21:52:10.973816+0000","last_clean":"2024-05-21T21:52:10.973816+0000","last_became_active":"2024-05-21T21:52:10.973457+0000","last_became_peered":"2024-05-21T21:52:10.973457+0000","last_unstale":"2024-05-21T21:52:10.973816+0000","last_undegraded":"2024-05-21T21:52:10.973816+0000","last_fullsized":"2024-05-21T21:52:10.973816+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T05:01:41.915158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"80'6","reported_seq":79,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:21.453398+0000","last_change":"2024-05-21T21:51:12.855658+0000","last_active":"2024-05-21T21:52:21.453398+0000","last_peered":"2024-05-21T21:52:21.453398+0000","last_clean":"2024-05-21T21:52:21.453398+0000","last_became_active":"2024-05-21T21:51:12.855281+0000","last_became_peered":"2024-05-21T21:51:12.855281+0000","last_unstale":"2024-05-21T21:52:21.453398+0000","last_undegraded":"2024-05-21T21:52:21.453398+0000","last_fullsized":"2024-05-21T21:52:21.453398+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:55:08.836045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":47,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.863544+0000","last_change":"2024-05-21T21:51:16.946977+0000","last_active":"2024-05-21T21:52:10.863544+0000","last_peered":"2024-05-21T21:52:10.863544+0000","last_clean":"2024-05-21T21:52:10.863544+0000","last_became_active":"2024-05-21T21:51:16.946562+0000","last_became_peered":"2024-05-21T21:51:16.946562+0000","last_unstale":"2024-05-21T21:52:10.863544+0000","last_undegraded":"2024-05-21T21:52:10.863544+0000","last_fullsized":"2024-05-21T21:52:10.863544+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:43:33.370265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"80'7","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.865540+0000","last_change":"2024-05-21T21:51:12.852574+0000","last_active":"2024-05-21T21:52:10.865540+0000","last_peered":"2024-05-21T21:52:10.865540+0000","last_clean":"2024-05-21T21:52:10.865540+0000","last_became_active":"2024-05-21T21:51:12.852258+0000","last_became_peered":"2024-05-21T21:51:12.852258+0000","last_unstale":"2024-05-21T21:52:10.865540+0000","last_undegraded":"2024-05-21T21:52:10.865540+0000","last_fullsized":"2024-05-21T21:52:10.865540+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:09:40.280821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.867344+0000","last_change":"2024-05-21T21:51:16.934768+0000","last_active":"2024-05-21T21:52:10.867344+0000","last_peered":"2024-05-21T21:52:10.867344+0000","last_clean":"2024-05-21T21:52:10.867344+0000","last_became_active":"2024-05-21T21:51:16.934275+0000","last_became_peered":"2024-05-21T21:51:16.934275+0000","last_unstale":"2024-05-21T21:52:10.867344+0000","last_undegraded":"2024-05-21T21:52:10.867344+0000","last_fullsized":"2024-05-21T21:52:10.867344+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:31:54.425069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"80'3","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.861882+0000","last_change":"2024-05-21T21:51:12.854166+0000","last_active":"2024-05-21T21:52:10.861882+0000","last_peered":"2024-05-21T21:52:10.861882+0000","last_clean":"2024-05-21T21:52:10.861882+0000","last_became_active":"2024-05-21T21:51:12.853508+0000","last_became_peered":"2024-05-21T21:51:12.853508+0000","last_unstale":"2024-05-21T21:52:10.861882+0000","last_undegraded":"2024-05-21T21:52:10.861882+0000","last_fullsized":"2024-05-21T21:52:10.861882+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:29:41.858655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.870323+0000","last_change":"2024-05-21T21:51:16.938293+0000","last_active":"2024-05-21T21:52:10.870323+0000","last_peered":"2024-05-21T21:52:10.870323+0000","last_clean":"2024-05-21T21:52:10.870323+0000","last_became_active":"2024-05-21T21:51:16.937970+0000","last_became_peered":"2024-05-21T21:51:16.937970+0000","last_unstale":"2024-05-21T21:52:10.870323+0000","last_undegraded":"2024-05-21T21:52:10.870323+0000","last_fullsized":"2024-05-21T21:52:10.870323+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:18:27.473098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'3","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.866723+0000","last_change":"2024-05-21T21:51:12.853366+0000","last_active":"2024-05-21T21:52:10.866723+0000","last_peered":"2024-05-21T21:52:10.866723+0000","last_clean":"2024-05-21T21:52:10.866723+0000","last_became_active":"2024-05-21T21:51:12.853028+0000","last_became_peered":"2024-05-21T21:51:12.853028+0000","last_unstale":"2024-05-21T21:52:10.866723+0000","last_undegraded":"2024-05-21T21:52:10.866723+0000","last_fullsized":"2024-05-21T21:52:10.866723+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T09:15:52.593028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.874880+0000","last_change":"2024-05-21T21:51:16.947903+0000","last_active":"2024-05-21T21:52:10.874880+0000","last_peered":"2024-05-21T21:52:10.874880+0000","last_clean":"2024-05-21T21:52:10.874880+0000","last_became_active":"2024-05-21T21:51:16.946677+0000","last_became_peered":"2024-05-21T21:51:16.946677+0000","last_unstale":"2024-05-21T21:52:10.874880+0000","last_undegraded":"2024-05-21T21:52:10.874880+0000","last_fullsized":"2024-05-21T21:52:10.874880+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T06:12:56.514284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'4","reported_seq":126,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:26.443320+0000","last_change":"2024-05-21T21:51:12.854873+0000","last_active":"2024-05-21T21:52:26.443320+0000","last_peered":"2024-05-21T21:52:26.443320+0000","last_clean":"2024-05-21T21:52:26.443320+0000","last_became_active":"2024-05-21T21:51:12.854393+0000","last_became_peered":"2024-05-21T21:51:12.854393+0000","last_unstale":"2024-05-21T21:52:26.443320+0000","last_undegraded":"2024-05-21T21:52:26.443320+0000","last_fullsized":"2024-05-21T21:52:26.443320+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:58:52.411164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":96,"num_read_kb":96,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.973117+0000","last_change":"2024-05-21T21:51:16.951094+0000","last_active":"2024-05-21T21:52:10.973117+0000","last_peered":"2024-05-21T21:52:10.973117+0000","last_clean":"2024-05-21T21:52:10.973117+0000","last_became_active":"2024-05-21T21:51:16.950724+0000","last_became_peered":"2024-05-21T21:51:16.950724+0000","last_unstale":"2024-05-21T21:52:10.973117+0000","last_undegraded":"2024-05-21T21:52:10.973117+0000","last_fullsized":"2024-05-21T21:52:10.973117+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:54:02.835426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'4","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.864127+0000","last_change":"2024-05-21T21:51:12.850427+0000","last_active":"2024-05-21T21:52:10.864127+0000","last_peered":"2024-05-21T21:52:10.864127+0000","last_clean":"2024-05-21T21:52:10.864127+0000","last_became_active":"2024-05-21T21:51:12.850096+0000","last_became_peered":"2024-05-21T21:51:12.850096+0000","last_unstale":"2024-05-21T21:52:10.864127+0000","last_undegraded":"2024-05-21T21:52:10.864127+0000","last_fullsized":"2024-05-21T21:52:10.864127+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:43:22.611043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.874050+0000","last_change":"2024-05-21T21:51:16.942330+0000","last_active":"2024-05-21T21:52:10.874050+0000","last_peered":"2024-05-21T21:52:10.874050+0000","last_clean":"2024-05-21T21:52:10.874050+0000","last_became_active":"2024-05-21T21:51:16.941718+0000","last_became_peered":"2024-05-21T21:51:16.941718+0000","last_unstale":"2024-05-21T21:52:10.874050+0000","last_undegraded":"2024-05-21T21:52:10.874050+0000","last_fullsized":"2024-05-21T21:52:10.874050+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:54:17.176411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'5","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.867578+0000","last_change":"2024-05-21T21:51:12.941100+0000","last_active":"2024-05-21T21:52:10.867578+0000","last_peered":"2024-05-21T21:52:10.867578+0000","last_clean":"2024-05-21T21:52:10.867578+0000","last_became_active":"2024-05-21T21:51:12.940597+0000","last_became_peered":"2024-05-21T21:51:12.940597+0000","last_unstale":"2024-05-21T21:52:10.867578+0000","last_undegraded":"2024-05-21T21:52:10.867578+0000","last_fullsized":"2024-05-21T21:52:10.867578+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:30:58.457108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.972855+0000","last_change":"2024-05-21T21:51:16.921973+0000","last_active":"2024-05-21T21:52:10.972855+0000","last_peered":"2024-05-21T21:52:10.972855+0000","last_clean":"2024-05-21T21:52:10.972855+0000","last_became_active":"2024-05-21T21:51:16.921597+0000","last_became_peered":"2024-05-21T21:51:16.921597+0000","last_unstale":"2024-05-21T21:52:10.972855+0000","last_undegraded":"2024-05-21T21:52:10.972855+0000","last_fullsized":"2024-05-21T21:52:10.972855+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:25:32.395940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'31","reported_seq":162,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:26.443388+0000","last_change":"2024-05-21T21:51:12.850312+0000","last_active":"2024-05-21T21:52:26.443388+0000","last_peered":"2024-05-21T21:52:26.443388+0000","last_clean":"2024-05-21T21:52:26.443388+0000","last_became_active":"2024-05-21T21:51:12.849837+0000","last_became_peered":"2024-05-21T21:51:12.849837+0000","last_unstale":"2024-05-21T21:52:26.443388+0000","last_undegraded":"2024-05-21T21:52:26.443388+0000","last_fullsized":"2024-05-21T21:52:26.443388+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T07:09:13.339554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59762,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":105,"num_read_kb":110,"num_write":33,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.863568+0000","last_change":"2024-05-21T21:51:16.926625+0000","last_active":"2024-05-21T21:52:10.863568+0000","last_peered":"2024-05-21T21:52:10.863568+0000","last_clean":"2024-05-21T21:52:10.863568+0000","last_became_active":"2024-05-21T21:51:16.925889+0000","last_became_peered":"2024-05-21T21:51:16.925889+0000","last_unstale":"2024-05-21T21:52:10.863568+0000","last_undegraded":"2024-05-21T21:52:10.863568+0000","last_fullsized":"2024-05-21T21:52:10.863568+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:35:23.464747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'9","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.863891+0000","last_change":"2024-05-21T21:51:12.859772+0000","last_active":"2024-05-21T21:52:10.863891+0000","last_peered":"2024-05-21T21:52:10.863891+0000","last_clean":"2024-05-21T21:52:10.863891+0000","last_became_active":"2024-05-21T21:51:12.859290+0000","last_became_peered":"2024-05-21T21:51:12.859290+0000","last_unstale":"2024-05-21T21:52:10.863891+0000","last_undegraded":"2024-05-21T21:52:10.863891+0000","last_fullsized":"2024-05-21T21:52:10.863891+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T05:28:29.953381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.861761+0000","last_change":"2024-05-21T21:51:16.925803+0000","last_active":"2024-05-21T21:52:10.861761+0000","last_peered":"2024-05-21T21:52:10.861761+0000","last_clean":"2024-05-21T21:52:10.861761+0000","last_became_active":"2024-05-21T21:51:16.925111+0000","last_became_peered":"2024-05-21T21:51:16.925111+0000","last_unstale":"2024-05-21T21:52:10.861761+0000","last_undegraded":"2024-05-21T21:52:10.861761+0000","last_fullsized":"2024-05-21T21:52:10.861761+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T06:22:11.353379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'5","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.867329+0000","last_change":"2024-05-21T21:51:12.859593+0000","last_active":"2024-05-21T21:52:10.867329+0000","last_peered":"2024-05-21T21:52:10.867329+0000","last_clean":"2024-05-21T21:52:10.867329+0000","last_became_active":"2024-05-21T21:51:12.859197+0000","last_became_peered":"2024-05-21T21:51:12.859197+0000","last_unstale":"2024-05-21T21:52:10.867329+0000","last_undegraded":"2024-05-21T21:52:10.867329+0000","last_fullsized":"2024-05-21T21:52:10.867329+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:13:12.311496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.879028+0000","last_change":"2024-05-21T21:52:10.879028+0000","last_active":"2024-05-21T21:52:10.879028+0000","last_peered":"2024-05-21T21:52:10.879028+0000","last_clean":"2024-05-21T21:52:10.879028+0000","last_became_active":"2024-05-21T21:52:10.878664+0000","last_became_peered":"2024-05-21T21:52:10.878664+0000","last_unstale":"2024-05-21T21:52:10.879028+0000","last_undegraded":"2024-05-21T21:52:10.879028+0000","last_fullsized":"2024-05-21T21:52:10.879028+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:05:53.953546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'6","reported_seq":79,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:21.445661+0000","last_change":"2024-05-21T21:51:12.852169+0000","last_active":"2024-05-21T21:52:21.445661+0000","last_peered":"2024-05-21T21:52:21.445661+0000","last_clean":"2024-05-21T21:52:21.445661+0000","last_became_active":"2024-05-21T21:51:12.851782+0000","last_became_peered":"2024-05-21T21:51:12.851782+0000","last_unstale":"2024-05-21T21:52:21.445661+0000","last_undegraded":"2024-05-21T21:52:21.445661+0000","last_fullsized":"2024-05-21T21:52:21.445661+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:46:09.692158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":48,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.863791+0000","last_change":"2024-05-21T21:51:16.929264+0000","last_active":"2024-05-21T21:52:10.863791+0000","last_peered":"2024-05-21T21:52:10.863791+0000","last_clean":"2024-05-21T21:52:10.863791+0000","last_became_active":"2024-05-21T21:51:16.928900+0000","last_became_peered":"2024-05-21T21:51:16.928900+0000","last_unstale":"2024-05-21T21:52:10.863791+0000","last_undegraded":"2024-05-21T21:52:10.863791+0000","last_fullsized":"2024-05-21T21:52:10.863791+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:42:28.040939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'17","reported_seq":68,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.984451+0000","last_change":"2024-05-21T21:52:10.984451+0000","last_active":"2024-05-21T21:52:10.984451+0000","last_peered":"2024-05-21T21:52:10.984451+0000","last_clean":"2024-05-21T21:52:10.984451+0000","last_became_active":"2024-05-21T21:52:10.974050+0000","last_became_peered":"2024-05-21T21:52:10.974050+0000","last_unstale":"2024-05-21T21:52:10.984451+0000","last_undegraded":"2024-05-21T21:52:10.984451+0000","last_fullsized":"2024-05-21T21:52:10.984451+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T07:08:18.307022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.867149+0000","last_change":"2024-05-21T21:51:16.935464+0000","last_active":"2024-05-21T21:52:10.867149+0000","last_peered":"2024-05-21T21:52:10.867149+0000","last_clean":"2024-05-21T21:52:10.867149+0000","last_became_active":"2024-05-21T21:51:16.935091+0000","last_became_peered":"2024-05-21T21:51:16.935091+0000","last_unstale":"2024-05-21T21:52:10.867149+0000","last_undegraded":"2024-05-21T21:52:10.867149+0000","last_fullsized":"2024-05-21T21:52:10.867149+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T06:59:16.330649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.864530+0000","last_change":"2024-05-21T21:51:12.853327+0000","last_active":"2024-05-21T21:52:10.864530+0000","last_peered":"2024-05-21T21:52:10.864530+0000","last_clean":"2024-05-21T21:52:10.864530+0000","last_became_active":"2024-05-21T21:51:12.852946+0000","last_became_peered":"2024-05-21T21:51:12.852946+0000","last_unstale":"2024-05-21T21:52:10.864530+0000","last_undegraded":"2024-05-21T21:52:10.864530+0000","last_fullsized":"2024-05-21T21:52:10.864530+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:00:34.256858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.861583+0000","last_change":"2024-05-21T21:51:16.929341+0000","last_active":"2024-05-21T21:52:10.861583+0000","last_peered":"2024-05-21T21:52:10.861583+0000","last_clean":"2024-05-21T21:52:10.861583+0000","last_became_active":"2024-05-21T21:51:16.929008+0000","last_became_peered":"2024-05-21T21:51:16.929008+0000","last_unstale":"2024-05-21T21:52:10.861583+0000","last_undegraded":"2024-05-21T21:52:10.861583+0000","last_fullsized":"2024-05-21T21:52:10.861583+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:26:26.297136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"80'1","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.875960+0000","last_change":"2024-05-21T21:51:12.945433+0000","last_active":"2024-05-21T21:52:10.875960+0000","last_peered":"2024-05-21T21:52:10.875960+0000","last_clean":"2024-05-21T21:52:10.875960+0000","last_became_active":"2024-05-21T21:51:12.944463+0000","last_became_peered":"2024-05-21T21:51:12.944463+0000","last_unstale":"2024-05-21T21:52:10.875960+0000","last_undegraded":"2024-05-21T21:52:10.875960+0000","last_fullsized":"2024-05-21T21:52:10.875960+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:29:52.014583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'1","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.883554+0000","last_change":"2024-05-21T21:52:10.883554+0000","last_active":"2024-05-21T21:52:10.883554+0000","last_peered":"2024-05-21T21:52:10.883554+0000","last_clean":"2024-05-21T21:52:10.883554+0000","last_became_active":"2024-05-21T21:52:10.877789+0000","last_became_peered":"2024-05-21T21:52:10.877789+0000","last_unstale":"2024-05-21T21:52:10.883554+0000","last_undegraded":"2024-05-21T21:52:10.883554+0000","last_fullsized":"2024-05-21T21:52:10.883554+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:52:37.623551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"80'5","reported_seq":79,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:24.470681+0000","last_change":"2024-05-21T21:51:12.852883+0000","last_active":"2024-05-21T21:52:24.470681+0000","last_peered":"2024-05-21T21:52:24.470681+0000","last_clean":"2024-05-21T21:52:24.470681+0000","last_became_active":"2024-05-21T21:51:12.852549+0000","last_became_peered":"2024-05-21T21:51:12.852549+0000","last_unstale":"2024-05-21T21:52:24.470681+0000","last_undegraded":"2024-05-21T21:52:24.470681+0000","last_fullsized":"2024-05-21T21:52:24.470681+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:05:53.002286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":48,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.865777+0000","last_change":"2024-05-21T21:51:16.944425+0000","last_active":"2024-05-21T21:52:10.865777+0000","last_peered":"2024-05-21T21:52:10.865777+0000","last_clean":"2024-05-21T21:52:10.865777+0000","last_became_active":"2024-05-21T21:51:16.944033+0000","last_became_peered":"2024-05-21T21:51:16.944033+0000","last_unstale":"2024-05-21T21:52:10.865777+0000","last_undegraded":"2024-05-21T21:52:10.865777+0000","last_fullsized":"2024-05-21T21:52:10.865777+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T07:16:26.802329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.866923+0000","last_change":"2024-05-21T21:51:16.933264+0000","last_active":"2024-05-21T21:52:10.866923+0000","last_peered":"2024-05-21T21:52:10.866923+0000","last_clean":"2024-05-21T21:52:10.866923+0000","last_became_active":"2024-05-21T21:51:16.932911+0000","last_became_peered":"2024-05-21T21:51:16.932911+0000","last_unstale":"2024-05-21T21:52:10.866923+0000","last_undegraded":"2024-05-21T21:52:10.866923+0000","last_fullsized":"2024-05-21T21:52:10.866923+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:25:54.006367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"80'5","reported_seq":78,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:23.448429+0000","last_change":"2024-05-21T21:51:12.942900+0000","last_active":"2024-05-21T21:52:23.448429+0000","last_peered":"2024-05-21T21:52:23.448429+0000","last_clean":"2024-05-21T21:52:23.448429+0000","last_became_active":"2024-05-21T21:51:12.941038+0000","last_became_peered":"2024-05-21T21:51:12.941038+0000","last_unstale":"2024-05-21T21:52:23.448429+0000","last_undegraded":"2024-05-21T21:52:23.448429+0000","last_fullsized":"2024-05-21T21:52:23.448429+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:38:43.240011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":47,"num_read_kb":47,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.867131+0000","last_change":"2024-05-21T21:51:16.933391+0000","last_active":"2024-05-21T21:52:10.867131+0000","last_peered":"2024-05-21T21:52:10.867131+0000","last_clean":"2024-05-21T21:52:10.867131+0000","last_became_active":"2024-05-21T21:51:16.932320+0000","last_became_peered":"2024-05-21T21:51:16.932320+0000","last_unstale":"2024-05-21T21:52:10.867131+0000","last_undegraded":"2024-05-21T21:52:10.867131+0000","last_fullsized":"2024-05-21T21:52:10.867131+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T09:06:59.126748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'6","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.865206+0000","last_change":"2024-05-21T21:51:12.856342+0000","last_active":"2024-05-21T21:52:10.865206+0000","last_peered":"2024-05-21T21:52:10.865206+0000","last_clean":"2024-05-21T21:52:10.865206+0000","last_became_active":"2024-05-21T21:51:12.855952+0000","last_became_peered":"2024-05-21T21:51:12.855952+0000","last_unstale":"2024-05-21T21:52:10.865206+0000","last_undegraded":"2024-05-21T21:52:10.865206+0000","last_fullsized":"2024-05-21T21:52:10.865206+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:34:40.734034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":100,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.863960+0000","last_change":"2024-05-21T21:49:42.797187+0000","last_active":"2024-05-21T21:52:10.863960+0000","last_peered":"2024-05-21T21:52:10.863960+0000","last_clean":"2024-05-21T21:52:10.863960+0000","last_became_active":"2024-05-21T21:49:42.791291+0000","last_became_peered":"2024-05-21T21:49:42.791291+0000","last_unstale":"2024-05-21T21:52:10.863960+0000","last_undegraded":"2024-05-21T21:52:10.863960+0000","last_fullsized":"2024-05-21T21:52:10.863960+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:45:55.552293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:45:55.552293+0000","last_clean_scrub_stamp":"2024-05-21T21:45:55.552293+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T07:24:18.302523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.861715+0000","last_change":"2024-05-21T21:51:16.926446+0000","last_active":"2024-05-21T21:52:10.861715+0000","last_peered":"2024-05-21T21:52:10.861715+0000","last_clean":"2024-05-21T21:52:10.861715+0000","last_became_active":"2024-05-21T21:51:16.926070+0000","last_became_peered":"2024-05-21T21:51:16.926070+0000","last_unstale":"2024-05-21T21:52:10.861715+0000","last_undegraded":"2024-05-21T21:52:10.861715+0000","last_fullsized":"2024-05-21T21:52:10.861715+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:44:27.366393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'4","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.864417+0000","last_change":"2024-05-21T21:51:12.854371+0000","last_active":"2024-05-21T21:52:10.864417+0000","last_peered":"2024-05-21T21:52:10.864417+0000","last_clean":"2024-05-21T21:52:10.864417+0000","last_became_active":"2024-05-21T21:51:12.853847+0000","last_became_peered":"2024-05-21T21:51:12.853847+0000","last_unstale":"2024-05-21T21:52:10.864417+0000","last_undegraded":"2024-05-21T21:52:10.864417+0000","last_fullsized":"2024-05-21T21:52:10.864417+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:14:10.549901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"80'4","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.863685+0000","last_change":"2024-05-21T21:51:12.941227+0000","last_active":"2024-05-21T21:52:10.863685+0000","last_peered":"2024-05-21T21:52:10.863685+0000","last_clean":"2024-05-21T21:52:10.863685+0000","last_became_active":"2024-05-21T21:51:12.940702+0000","last_became_peered":"2024-05-21T21:51:12.940702+0000","last_unstale":"2024-05-21T21:52:10.863685+0000","last_undegraded":"2024-05-21T21:52:10.863685+0000","last_fullsized":"2024-05-21T21:52:10.863685+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:40:34.124921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.862247+0000","last_change":"2024-05-21T21:51:16.945627+0000","last_active":"2024-05-21T21:52:10.862247+0000","last_peered":"2024-05-21T21:52:10.862247+0000","last_clean":"2024-05-21T21:52:10.862247+0000","last_became_active":"2024-05-21T21:51:16.945297+0000","last_became_peered":"2024-05-21T21:51:16.945297+0000","last_unstale":"2024-05-21T21:52:10.862247+0000","last_undegraded":"2024-05-21T21:52:10.862247+0000","last_fullsized":"2024-05-21T21:52:10.862247+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:54:38.898774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'15","reported_seq":57,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.888076+0000","last_change":"2024-05-21T21:52:10.888076+0000","last_active":"2024-05-21T21:52:10.888076+0000","last_peered":"2024-05-21T21:52:10.888076+0000","last_clean":"2024-05-21T21:52:10.888076+0000","last_became_active":"2024-05-21T21:52:10.879368+0000","last_became_peered":"2024-05-21T21:52:10.879368+0000","last_unstale":"2024-05-21T21:52:10.888076+0000","last_undegraded":"2024-05-21T21:52:10.888076+0000","last_fullsized":"2024-05-21T21:52:10.888076+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:46:43.190411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.868811+0000","last_change":"2024-05-21T21:51:16.929807+0000","last_active":"2024-05-21T21:52:10.868811+0000","last_peered":"2024-05-21T21:52:10.868811+0000","last_clean":"2024-05-21T21:52:10.868811+0000","last_became_active":"2024-05-21T21:51:16.928859+0000","last_became_peered":"2024-05-21T21:51:16.928859+0000","last_unstale":"2024-05-21T21:52:10.868811+0000","last_undegraded":"2024-05-21T21:52:10.868811+0000","last_fullsized":"2024-05-21T21:52:10.868811+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:41:32.830892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'7","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.865597+0000","last_change":"2024-05-21T21:51:12.850843+0000","last_active":"2024-05-21T21:52:10.865597+0000","last_peered":"2024-05-21T21:52:10.865597+0000","last_clean":"2024-05-21T21:52:10.865597+0000","last_became_active":"2024-05-21T21:51:12.850516+0000","last_became_peered":"2024-05-21T21:51:12.850516+0000","last_unstale":"2024-05-21T21:52:10.865597+0000","last_undegraded":"2024-05-21T21:52:10.865597+0000","last_fullsized":"2024-05-21T21:52:10.865597+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:37:05.191626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.873845+0000","last_change":"2024-05-21T21:51:16.949017+0000","last_active":"2024-05-21T21:52:10.873845+0000","last_peered":"2024-05-21T21:52:10.873845+0000","last_clean":"2024-05-21T21:52:10.873845+0000","last_became_active":"2024-05-21T21:51:16.948691+0000","last_became_peered":"2024-05-21T21:51:16.948691+0000","last_unstale":"2024-05-21T21:52:10.873845+0000","last_undegraded":"2024-05-21T21:52:10.873845+0000","last_fullsized":"2024-05-21T21:52:10.873845+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:32:06.606581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.973164+0000","last_change":"2024-05-21T21:51:12.939203+0000","last_active":"2024-05-21T21:52:10.973164+0000","last_peered":"2024-05-21T21:52:10.973164+0000","last_clean":"2024-05-21T21:52:10.973164+0000","last_became_active":"2024-05-21T21:51:12.938936+0000","last_became_peered":"2024-05-21T21:51:12.938936+0000","last_unstale":"2024-05-21T21:52:10.973164+0000","last_undegraded":"2024-05-21T21:52:10.973164+0000","last_fullsized":"2024-05-21T21:52:10.973164+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:55:50.336362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.863329+0000","last_change":"2024-05-21T21:51:16.926216+0000","last_active":"2024-05-21T21:52:10.863329+0000","last_peered":"2024-05-21T21:52:10.863329+0000","last_clean":"2024-05-21T21:52:10.863329+0000","last_became_active":"2024-05-21T21:51:16.925834+0000","last_became_peered":"2024-05-21T21:51:16.925834+0000","last_unstale":"2024-05-21T21:52:10.863329+0000","last_undegraded":"2024-05-21T21:52:10.863329+0000","last_fullsized":"2024-05-21T21:52:10.863329+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:13:30.192908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'7","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.866548+0000","last_change":"2024-05-21T21:51:12.855306+0000","last_active":"2024-05-21T21:52:10.866548+0000","last_peered":"2024-05-21T21:52:10.866548+0000","last_clean":"2024-05-21T21:52:10.866548+0000","last_became_active":"2024-05-21T21:51:12.854722+0000","last_became_peered":"2024-05-21T21:51:12.854722+0000","last_unstale":"2024-05-21T21:52:10.866548+0000","last_undegraded":"2024-05-21T21:52:10.866548+0000","last_fullsized":"2024-05-21T21:52:10.866548+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:39:41.583219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.861450+0000","last_change":"2024-05-21T21:51:16.944758+0000","last_active":"2024-05-21T21:52:10.861450+0000","last_peered":"2024-05-21T21:52:10.861450+0000","last_clean":"2024-05-21T21:52:10.861450+0000","last_became_active":"2024-05-21T21:51:16.944393+0000","last_became_peered":"2024-05-21T21:51:16.944393+0000","last_unstale":"2024-05-21T21:52:10.861450+0000","last_undegraded":"2024-05-21T21:52:10.861450+0000","last_fullsized":"2024-05-21T21:52:10.861450+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:01:23.048355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'3","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.863710+0000","last_change":"2024-05-21T21:51:12.860378+0000","last_active":"2024-05-21T21:52:10.863710+0000","last_peered":"2024-05-21T21:52:10.863710+0000","last_clean":"2024-05-21T21:52:10.863710+0000","last_became_active":"2024-05-21T21:51:12.859978+0000","last_became_peered":"2024-05-21T21:51:12.859978+0000","last_unstale":"2024-05-21T21:52:10.863710+0000","last_undegraded":"2024-05-21T21:52:10.863710+0000","last_fullsized":"2024-05-21T21:52:10.863710+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T07:52:26.902232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.867609+0000","last_change":"2024-05-21T21:51:16.943194+0000","last_active":"2024-05-21T21:52:10.867609+0000","last_peered":"2024-05-21T21:52:10.867609+0000","last_clean":"2024-05-21T21:52:10.867609+0000","last_became_active":"2024-05-21T21:51:16.942884+0000","last_became_peered":"2024-05-21T21:51:16.942884+0000","last_unstale":"2024-05-21T21:52:10.867609+0000","last_undegraded":"2024-05-21T21:52:10.867609+0000","last_fullsized":"2024-05-21T21:52:10.867609+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T05:58:17.436995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.863880+0000","last_change":"2024-05-21T21:51:12.853390+0000","last_active":"2024-05-21T21:52:10.863880+0000","last_peered":"2024-05-21T21:52:10.863880+0000","last_clean":"2024-05-21T21:52:10.863880+0000","last_became_active":"2024-05-21T21:51:12.852990+0000","last_became_peered":"2024-05-21T21:51:12.852990+0000","last_unstale":"2024-05-21T21:52:10.863880+0000","last_undegraded":"2024-05-21T21:52:10.863880+0000","last_fullsized":"2024-05-21T21:52:10.863880+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:32:16.392609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.864540+0000","last_change":"2024-05-21T21:51:16.933430+0000","last_active":"2024-05-21T21:52:10.864540+0000","last_peered":"2024-05-21T21:52:10.864540+0000","last_clean":"2024-05-21T21:52:10.864540+0000","last_became_active":"2024-05-21T21:51:16.933053+0000","last_became_peered":"2024-05-21T21:51:16.933053+0000","last_unstale":"2024-05-21T21:52:10.864540+0000","last_undegraded":"2024-05-21T21:52:10.864540+0000","last_fullsized":"2024-05-21T21:52:10.864540+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T06:14:57.468727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'5","reported_seq":80,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:24.440375+0000","last_change":"2024-05-21T21:51:12.946113+0000","last_active":"2024-05-21T21:52:24.440375+0000","last_peered":"2024-05-21T21:52:24.440375+0000","last_clean":"2024-05-21T21:52:24.440375+0000","last_became_active":"2024-05-21T21:51:12.945650+0000","last_became_peered":"2024-05-21T21:51:12.945650+0000","last_unstale":"2024-05-21T21:52:24.440375+0000","last_undegraded":"2024-05-21T21:52:24.440375+0000","last_fullsized":"2024-05-21T21:52:24.440375+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T09:40:18.209400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":49,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.864300+0000","last_change":"2024-05-21T21:51:16.932632+0000","last_active":"2024-05-21T21:52:10.864300+0000","last_peered":"2024-05-21T21:52:10.864300+0000","last_clean":"2024-05-21T21:52:10.864300+0000","last_became_active":"2024-05-21T21:51:16.932256+0000","last_became_peered":"2024-05-21T21:51:16.932256+0000","last_unstale":"2024-05-21T21:52:10.864300+0000","last_undegraded":"2024-05-21T21:52:10.864300+0000","last_fullsized":"2024-05-21T21:52:10.864300+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:04:48.146485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'3","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.867556+0000","last_change":"2024-05-21T21:51:12.860602+0000","last_active":"2024-05-21T21:52:10.867556+0000","last_peered":"2024-05-21T21:52:10.867556+0000","last_clean":"2024-05-21T21:52:10.867556+0000","last_became_active":"2024-05-21T21:51:12.860228+0000","last_became_peered":"2024-05-21T21:51:12.860228+0000","last_unstale":"2024-05-21T21:52:10.867556+0000","last_undegraded":"2024-05-21T21:52:10.867556+0000","last_fullsized":"2024-05-21T21:52:10.867556+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:33:39.230872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'1","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.863947+0000","last_change":"2024-05-21T21:51:16.929616+0000","last_active":"2024-05-21T21:52:10.863947+0000","last_peered":"2024-05-21T21:52:10.863947+0000","last_clean":"2024-05-21T21:52:10.863947+0000","last_became_active":"2024-05-21T21:51:16.928569+0000","last_became_peered":"2024-05-21T21:51:16.928569+0000","last_unstale":"2024-05-21T21:52:10.863947+0000","last_undegraded":"2024-05-21T21:52:10.863947+0000","last_fullsized":"2024-05-21T21:52:10.863947+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:15:41.790344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'5","reported_seq":43,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.881114+0000","last_change":"2024-05-21T21:52:10.881114+0000","last_active":"2024-05-21T21:52:10.881114+0000","last_peered":"2024-05-21T21:52:10.881114+0000","last_clean":"2024-05-21T21:52:10.881114+0000","last_became_active":"2024-05-21T21:52:10.872022+0000","last_became_peered":"2024-05-21T21:52:10.872022+0000","last_unstale":"2024-05-21T21:52:10.881114+0000","last_undegraded":"2024-05-21T21:52:10.881114+0000","last_fullsized":"2024-05-21T21:52:10.881114+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T06:05:51.419156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.864314+0000","last_change":"2024-05-21T21:51:16.926496+0000","last_active":"2024-05-21T21:52:10.864314+0000","last_peered":"2024-05-21T21:52:10.864314+0000","last_clean":"2024-05-21T21:52:10.864314+0000","last_became_active":"2024-05-21T21:51:16.926112+0000","last_became_peered":"2024-05-21T21:51:16.926112+0000","last_unstale":"2024-05-21T21:52:10.864314+0000","last_undegraded":"2024-05-21T21:52:10.864314+0000","last_fullsized":"2024-05-21T21:52:10.864314+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:26:08.254421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.861508+0000","last_change":"2024-05-21T21:51:16.930522+0000","last_active":"2024-05-21T21:52:10.861508+0000","last_peered":"2024-05-21T21:52:10.861508+0000","last_clean":"2024-05-21T21:52:10.861508+0000","last_became_active":"2024-05-21T21:51:16.929853+0000","last_became_peered":"2024-05-21T21:51:16.929853+0000","last_unstale":"2024-05-21T21:52:10.861508+0000","last_undegraded":"2024-05-21T21:52:10.861508+0000","last_fullsized":"2024-05-21T21:52:10.861508+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:10:57.940009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.863593+0000","last_change":"2024-05-21T21:51:16.931751+0000","last_active":"2024-05-21T21:52:10.863593+0000","last_peered":"2024-05-21T21:52:10.863593+0000","last_clean":"2024-05-21T21:52:10.863593+0000","last_became_active":"2024-05-21T21:51:16.931353+0000","last_became_peered":"2024-05-21T21:51:16.931353+0000","last_unstale":"2024-05-21T21:52:10.863593+0000","last_undegraded":"2024-05-21T21:52:10.863593+0000","last_fullsized":"2024-05-21T21:52:10.863593+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:33:45.297850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.863516+0000","last_change":"2024-05-21T21:51:16.940096+0000","last_active":"2024-05-21T21:52:10.863516+0000","last_peered":"2024-05-21T21:52:10.863516+0000","last_clean":"2024-05-21T21:52:10.863516+0000","last_became_active":"2024-05-21T21:51:16.939740+0000","last_became_peered":"2024-05-21T21:51:16.939740+0000","last_unstale":"2024-05-21T21:52:10.863516+0000","last_undegraded":"2024-05-21T21:52:10.863516+0000","last_fullsized":"2024-05-21T21:52:10.863516+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:59:52.569526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.872739+0000","last_change":"2024-05-21T21:51:16.946488+0000","last_active":"2024-05-21T21:52:10.872739+0000","last_peered":"2024-05-21T21:52:10.872739+0000","last_clean":"2024-05-21T21:52:10.872739+0000","last_became_active":"2024-05-21T21:51:16.945957+0000","last_became_peered":"2024-05-21T21:51:16.945957+0000","last_unstale":"2024-05-21T21:52:10.872739+0000","last_undegraded":"2024-05-21T21:52:10.872739+0000","last_fullsized":"2024-05-21T21:52:10.872739+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:40:36.708344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.862107+0000","last_change":"2024-05-21T21:51:16.949349+0000","last_active":"2024-05-21T21:52:10.862107+0000","last_peered":"2024-05-21T21:52:10.862107+0000","last_clean":"2024-05-21T21:52:10.862107+0000","last_became_active":"2024-05-21T21:51:16.946986+0000","last_became_peered":"2024-05-21T21:51:16.946986+0000","last_unstale":"2024-05-21T21:52:10.862107+0000","last_undegraded":"2024-05-21T21:52:10.862107+0000","last_fullsized":"2024-05-21T21:52:10.862107+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:01:14.877110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.861685+0000","last_change":"2024-05-21T21:51:16.929172+0000","last_active":"2024-05-21T21:52:10.861685+0000","last_peered":"2024-05-21T21:52:10.861685+0000","last_clean":"2024-05-21T21:52:10.861685+0000","last_became_active":"2024-05-21T21:51:16.928823+0000","last_became_peered":"2024-05-21T21:51:16.928823+0000","last_unstale":"2024-05-21T21:52:10.861685+0000","last_undegraded":"2024-05-21T21:52:10.861685+0000","last_fullsized":"2024-05-21T21:52:10.861685+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:09:13.719272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.863847+0000","last_change":"2024-05-21T21:51:16.930332+0000","last_active":"2024-05-21T21:52:10.863847+0000","last_peered":"2024-05-21T21:52:10.863847+0000","last_clean":"2024-05-21T21:52:10.863847+0000","last_became_active":"2024-05-21T21:51:16.930024+0000","last_became_peered":"2024-05-21T21:51:16.930024+0000","last_unstale":"2024-05-21T21:52:10.863847+0000","last_undegraded":"2024-05-21T21:52:10.863847+0000","last_fullsized":"2024-05-21T21:52:10.863847+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:47:41.272725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.868573+0000","last_change":"2024-05-21T21:51:16.930102+0000","last_active":"2024-05-21T21:52:10.868573+0000","last_peered":"2024-05-21T21:52:10.868573+0000","last_clean":"2024-05-21T21:52:10.868573+0000","last_became_active":"2024-05-21T21:51:16.929410+0000","last_became_peered":"2024-05-21T21:51:16.929410+0000","last_unstale":"2024-05-21T21:52:10.868573+0000","last_undegraded":"2024-05-21T21:52:10.868573+0000","last_fullsized":"2024-05-21T21:52:10.868573+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:11:46.296288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.873829+0000","last_change":"2024-05-21T21:52:10.873424+0000","last_active":"2024-05-21T21:52:10.873829+0000","last_peered":"2024-05-21T21:52:10.873829+0000","last_clean":"2024-05-21T21:52:10.873829+0000","last_became_active":"2024-05-21T21:52:10.873118+0000","last_became_peered":"2024-05-21T21:52:10.873118+0000","last_unstale":"2024-05-21T21:52:10.873829+0000","last_undegraded":"2024-05-21T21:52:10.873829+0000","last_fullsized":"2024-05-21T21:52:10.873829+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:05:00.731013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,3],"acting":[8,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.866413+0000","last_change":"2024-05-21T21:51:16.934442+0000","last_active":"2024-05-21T21:52:10.866413+0000","last_peered":"2024-05-21T21:52:10.866413+0000","last_clean":"2024-05-21T21:52:10.866413+0000","last_became_active":"2024-05-21T21:51:16.934086+0000","last_became_peered":"2024-05-21T21:51:16.934086+0000","last_unstale":"2024-05-21T21:52:10.866413+0000","last_undegraded":"2024-05-21T21:52:10.866413+0000","last_fullsized":"2024-05-21T21:52:10.866413+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:36:44.302532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.972750+0000","last_change":"2024-05-21T21:51:16.924479+0000","last_active":"2024-05-21T21:52:10.972750+0000","last_peered":"2024-05-21T21:52:10.972750+0000","last_clean":"2024-05-21T21:52:10.972750+0000","last_became_active":"2024-05-21T21:51:16.923998+0000","last_became_peered":"2024-05-21T21:51:16.923998+0000","last_unstale":"2024-05-21T21:52:10.972750+0000","last_undegraded":"2024-05-21T21:52:10.972750+0000","last_fullsized":"2024-05-21T21:52:10.972750+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:23:07.931731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.862486+0000","last_change":"2024-05-21T21:51:16.947271+0000","last_active":"2024-05-21T21:52:10.862486+0000","last_peered":"2024-05-21T21:52:10.862486+0000","last_clean":"2024-05-21T21:52:10.862486+0000","last_became_active":"2024-05-21T21:51:16.946912+0000","last_became_peered":"2024-05-21T21:51:16.946912+0000","last_unstale":"2024-05-21T21:52:10.862486+0000","last_undegraded":"2024-05-21T21:52:10.862486+0000","last_fullsized":"2024-05-21T21:52:10.862486+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:51:29.646915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.861507+0000","last_change":"2024-05-21T21:51:16.928852+0000","last_active":"2024-05-21T21:52:10.861507+0000","last_peered":"2024-05-21T21:52:10.861507+0000","last_clean":"2024-05-21T21:52:10.861507+0000","last_became_active":"2024-05-21T21:51:16.928480+0000","last_became_peered":"2024-05-21T21:51:16.928480+0000","last_unstale":"2024-05-21T21:52:10.861507+0000","last_undegraded":"2024-05-21T21:52:10.861507+0000","last_fullsized":"2024-05-21T21:52:10.861507+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T09:12:30.691254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66966,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":505,"num_read_kb":328,"num_write":245,"num_write_kb":135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":112,"num_keys_recovered":17,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":200898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":210,"ondisk_log_size":210,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645343,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":433959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841571,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28588,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716564,"statfs":{"total":95995035648,"available":95965761536,"internally_reserved":0,"allocated":1601536,"data_stored":1085141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005096,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":929792,"data_stored":434006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201327,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":921600,"data_stored":434267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397555,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28504,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716648,"statfs":{"total":95995035648,"available":95965847552,"internally_reserved":0,"allocated":1519616,"data_stored":1027176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561081,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":909312,"data_stored":433606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":978944,"data_stored":494393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738459,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36252,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708900,"statfs":{"total":95995035648,"available":95957913600,"internally_reserved":0,"allocated":929792,"data_stored":434945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411409,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36296,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708856,"statfs":{"total":95995035648,"available":95957868544,"internally_reserved":0,"allocated":983040,"data_stored":494780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149770,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36244,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708908,"statfs":{"total":95995035648,"available":95957921792,"internally_reserved":0,"allocated":921600,"data_stored":433314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953540,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":909312,"data_stored":433858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757311,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32676,"kb_used_data":1496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712476,"statfs":{"total":95995035648,"available":95961575424,"internally_reserved":0,"allocated":1531904,"data_stored":1026463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":61626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":62013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":61889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-21T21:52:29.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:52:28 smithi151 ceph-mon[31623]: pgmap v351: 97 pgs: 97 active+clean; 642 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-21T21:52:29.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:52:28 smithi151 ceph-mon[31623]: from='client.14553 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T21:52:29.238 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:52:28 smithi184 ceph-mon[36942]: pgmap v351: 97 pgs: 97 active+clean; 642 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-21T21:52:29.238 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:52:28 smithi184 ceph-mon[36942]: from='client.14553 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T21:52:29.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:52:28 smithi164 ceph-mon[36269]: pgmap v351: 97 pgs: 97 active+clean; 642 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-21T21:52:29.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:52:28 smithi164 ceph-mon[36269]: from='client.14553 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T21:52:30.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:52:29 smithi184 ceph-mon[36942]: pgmap v352: 97 pgs: 97 active+clean; 642 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-21T21:52:30.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:52:29 smithi164 ceph-mon[36269]: pgmap v352: 97 pgs: 97 active+clean; 642 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-21T21:52:30.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:52:29 smithi151 ceph-mon[31623]: pgmap v352: 97 pgs: 97 active+clean; 642 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-21T21:52:31.105 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph pg dump --format=json 2024-05-21T21:52:31.573 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:52:32.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:52:32 smithi164 ceph-mon[36269]: pgmap v353: 97 pgs: 97 active+clean; 642 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-21T21:52:32.598 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:52:32 smithi151 ceph-mon[31623]: pgmap v353: 97 pgs: 97 active+clean; 642 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-21T21:52:32.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:52:32 smithi184 ceph-mon[36942]: pgmap v353: 97 pgs: 97 active+clean; 642 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-21T21:52:33.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:33.807+0000 7fe72ea00700 1 -- 172.21.15.151:0/2008694455 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe72806b220 msgr2=0x7fe72806b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:52:33.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:33.807+0000 7fe72ea00700 1 --2- 172.21.15.151:0/2008694455 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe72806b220 0x7fe72806b6a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe714009990 tx=0x7fe714009ca0 comp rx=0 tx=0).stop 2024-05-21T21:52:33.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:33.807+0000 7fe72ea00700 1 -- 172.21.15.151:0/2008694455 shutdown_connections 2024-05-21T21:52:33.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:33.807+0000 7fe72ea00700 1 --2- 172.21.15.151:0/2008694455 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe7281095b0 0x7fe728109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:33.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:33.807+0000 7fe72ea00700 1 --2- 172.21.15.151:0/2008694455 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe72806bbe0 0x7fe7281103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:33.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:33.807+0000 7fe72ea00700 1 --2- 172.21.15.151:0/2008694455 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe72806b220 0x7fe72806b6a0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:33.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:33.807+0000 7fe72ea00700 1 -- 172.21.15.151:0/2008694455 >> 172.21.15.151:0/2008694455 conn(0x7fe7280793f0 msgr2=0x7fe728079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:52:33.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:33.807+0000 7fe72ea00700 1 -- 172.21.15.151:0/2008694455 shutdown_connections 2024-05-21T21:52:33.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:33.807+0000 7fe72ea00700 1 -- 172.21.15.151:0/2008694455 wait complete. 2024-05-21T21:52:33.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:33.808+0000 7fe72ea00700 1 Processor -- start 2024-05-21T21:52:33.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:33.809+0000 7fe72ea00700 1 -- start start 2024-05-21T21:52:33.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:33.809+0000 7fe72ea00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe72806b220 0x7fe72810a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:52:33.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:33.809+0000 7fe72ea00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe72806bbe0 0x7fe72810a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:52:33.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:33.809+0000 7fe72ea00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe7281095b0 0x7fe72810ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:52:33.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:33.809+0000 7fe72ea00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe728113710 con 0x7fe72806bbe0 2024-05-21T21:52:33.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:33.809+0000 7fe72ea00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe728113590 con 0x7fe72806b220 2024-05-21T21:52:33.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:33.809+0000 7fe72ea00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe728113890 con 0x7fe7281095b0 2024-05-21T21:52:33.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:33.809+0000 7fe727400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe72806bbe0 0x7fe72810a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:52:33.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:33.809+0000 7fe727e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe72806b220 0x7fe72810a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:52:33.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:33.809+0000 7fe727400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe72806bbe0 0x7fe72810a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:47166/0 (socket says 172.21.15.151:47166) 2024-05-21T21:52:33.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:33.810+0000 7fe72ca00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe7281095b0 0x7fe72810ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:52:33.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:33.810+0000 7fe727e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe72806b220 0x7fe72810a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:36858/0 (socket says 172.21.15.151:36858) 2024-05-21T21:52:33.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:33.810+0000 7fe727400700 1 -- 172.21.15.151:0/656267688 learned_addr learned my addr 172.21.15.151:0/656267688 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:52:33.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:33.810+0000 7fe72ca00700 1 -- 172.21.15.151:0/656267688 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe72806b220 msgr2=0x7fe72810a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:52:33.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:33.810+0000 7fe72ca00700 1 --2- 172.21.15.151:0/656267688 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe72806b220 0x7fe72810a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:33.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:33.810+0000 7fe72ca00700 1 -- 172.21.15.151:0/656267688 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe72806bbe0 msgr2=0x7fe72810a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:52:33.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:33.810+0000 7fe72ca00700 1 --2- 172.21.15.151:0/656267688 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe72806bbe0 0x7fe72810a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:33.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:33.810+0000 7fe72ca00700 1 -- 172.21.15.151:0/656267688 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7281ac510 con 0x7fe7281095b0 2024-05-21T21:52:33.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:33.810+0000 7fe72ca00700 1 --2- 172.21.15.151:0/656267688 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe7281095b0 0x7fe72810ae40 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe720009f60 tx=0x7fe720009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:52:33.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:33.811+0000 7fe724c00700 1 -- 172.21.15.151:0/656267688 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7200092e0 con 0x7fe7281095b0 2024-05-21T21:52:33.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:33.811+0000 7fe724c00700 1 -- 172.21.15.151:0/656267688 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe720004bb0 con 0x7fe7281095b0 2024-05-21T21:52:33.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:33.811+0000 7fe724c00700 1 -- 172.21.15.151:0/656267688 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe720004020 con 0x7fe7281095b0 2024-05-21T21:52:33.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:33.811+0000 7fe72ea00700 1 -- 172.21.15.151:0/656267688 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7281ac7a0 con 0x7fe7281095b0 2024-05-21T21:52:33.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:33.811+0000 7fe72ea00700 1 -- 172.21.15.151:0/656267688 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7281acc90 con 0x7fe7281095b0 2024-05-21T21:52:33.813 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:33.812+0000 7fe72ea00700 1 -- 172.21.15.151:0/656267688 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe728104c70 con 0x7fe7281095b0 2024-05-21T21:52:33.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:33.814+0000 7fe724c00700 1 -- 172.21.15.151:0/656267688 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fe72001b030 con 0x7fe7281095b0 2024-05-21T21:52:33.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:33.816+0000 7fe724c00700 1 --2- 172.21.15.151:0/656267688 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fe70c0b2990 0x7fe70c0b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:52:33.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:33.816+0000 7fe724c00700 1 -- 172.21.15.151:0/656267688 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fe7200e2750 con 0x7fe7281095b0 2024-05-21T21:52:33.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:33.816+0000 7fe727e00700 1 --2- 172.21.15.151:0/656267688 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fe70c0b2990 0x7fe70c0b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:52:33.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:33.817+0000 7fe727e00700 1 --2- 172.21.15.151:0/656267688 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fe70c0b2990 0x7fe70c0b4e50 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe718009be0 tx=0x7fe718009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:52:33.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:33.818+0000 7fe724c00700 1 -- 172.21.15.151:0/656267688 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe720022020 con 0x7fe7281095b0 2024-05-21T21:52:34.047 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:34.047+0000 7fe72ea00700 1 -- 172.21.15.151:0/656267688 --> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe72810ee10 con 0x7fe70c0b2990 2024-05-21T21:52:34.069 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:34.069+0000 7fe724c00700 1 -- 172.21.15.151:0/656267688 <== mgr.14152 v2:172.21.15.151:6800/2796622483 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258766 (secure 0 0 0) 0x7fe72810ee10 con 0x7fe70c0b2990 2024-05-21T21:52:34.075 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T21:52:34.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:34.076+0000 7fe72ea00700 1 -- 172.21.15.151:0/656267688 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fe70c0b2990 msgr2=0x7fe70c0b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:52:34.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:34.077+0000 7fe72ea00700 1 --2- 172.21.15.151:0/656267688 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fe70c0b2990 0x7fe70c0b4e50 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe718009be0 tx=0x7fe718009260 comp rx=0 tx=0).stop 2024-05-21T21:52:34.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:34.077+0000 7fe72ea00700 1 -- 172.21.15.151:0/656267688 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe7281095b0 msgr2=0x7fe72810ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:52:34.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:34.077+0000 7fe72ea00700 1 --2- 172.21.15.151:0/656267688 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe7281095b0 0x7fe72810ae40 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe720009f60 tx=0x7fe720009f90 comp rx=0 tx=0).stop 2024-05-21T21:52:34.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:34.077+0000 7fe72ea00700 1 -- 172.21.15.151:0/656267688 shutdown_connections 2024-05-21T21:52:34.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:34.077+0000 7fe72ea00700 1 --2- 172.21.15.151:0/656267688 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fe70c0b2990 0x7fe70c0b4e50 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:34.078 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:34.077+0000 7fe72ea00700 1 --2- 172.21.15.151:0/656267688 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe72806b220 0x7fe72810a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:34.078 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:34.077+0000 7fe72ea00700 1 --2- 172.21.15.151:0/656267688 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe7281095b0 0x7fe72810ae40 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:34.078 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:34.077+0000 7fe72ea00700 1 --2- 172.21.15.151:0/656267688 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe72806bbe0 0x7fe72810a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:34.078 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:34.077+0000 7fe72ea00700 1 -- 172.21.15.151:0/656267688 >> 172.21.15.151:0/656267688 conn(0x7fe7280793f0 msgr2=0x7fe72810e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:52:34.078 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:34.077+0000 7fe72ea00700 1 -- 172.21.15.151:0/656267688 shutdown_connections 2024-05-21T21:52:34.078 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:34.077+0000 7fe72ea00700 1 -- 172.21.15.151:0/656267688 wait complete. 2024-05-21T21:52:34.078 INFO:teuthology.orchestra.run.smithi151.stderr:dumped all 2024-05-21T21:52:34.447 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:52:34 smithi151 ceph-mon[31623]: pgmap v354: 97 pgs: 97 active+clean; 642 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-21T21:52:34.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:52:34 smithi164 ceph-mon[36269]: pgmap v354: 97 pgs: 97 active+clean; 642 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-21T21:52:34.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:52:34 smithi184 ceph-mon[36942]: pgmap v354: 97 pgs: 97 active+clean; 642 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-21T21:52:34.979 INFO:teuthology.orchestra.run.smithi151.stdout:{"pg_ready":true,"pg_map":{"version":354,"stamp":"2024-05-21T21:52:33.220618+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657568,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":646,"num_read_kb":422,"num_write":370,"num_write_kb":2287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":2952069,"num_keys_recovered":17,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":304,"ondisk_log_size":304,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":366112,"kb_used_data":12732,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124575712,"statfs":{"total":1151940427776,"available":1151565529088,"internally_reserved":0,"allocated":13037568,"data_stored":7165908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19140,"internal_metadata":361674044},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":130,"num_read_kb":65,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005326"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.861691+0000","last_change":"2024-05-21T21:51:16.945132+0000","last_active":"2024-05-21T21:52:10.861691+0000","last_peered":"2024-05-21T21:52:10.861691+0000","last_clean":"2024-05-21T21:52:10.861691+0000","last_became_active":"2024-05-21T21:51:16.944766+0000","last_became_peered":"2024-05-21T21:51:16.944766+0000","last_unstale":"2024-05-21T21:52:10.861691+0000","last_undegraded":"2024-05-21T21:52:10.861691+0000","last_fullsized":"2024-05-21T21:52:10.861691+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:54:04.185095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.873699+0000","last_change":"2024-05-21T21:51:16.939291+0000","last_active":"2024-05-21T21:52:10.873699+0000","last_peered":"2024-05-21T21:52:10.873699+0000","last_clean":"2024-05-21T21:52:10.873699+0000","last_became_active":"2024-05-21T21:51:16.938730+0000","last_became_peered":"2024-05-21T21:51:16.938730+0000","last_unstale":"2024-05-21T21:52:10.873699+0000","last_undegraded":"2024-05-21T21:52:10.873699+0000","last_fullsized":"2024-05-21T21:52:10.873699+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:05:30.623744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.873014+0000","last_change":"2024-05-21T21:51:16.941565+0000","last_active":"2024-05-21T21:52:10.873014+0000","last_peered":"2024-05-21T21:52:10.873014+0000","last_clean":"2024-05-21T21:52:10.873014+0000","last_became_active":"2024-05-21T21:51:16.941033+0000","last_became_peered":"2024-05-21T21:51:16.941033+0000","last_unstale":"2024-05-21T21:52:10.873014+0000","last_undegraded":"2024-05-21T21:52:10.873014+0000","last_fullsized":"2024-05-21T21:52:10.873014+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:04:55.789479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.865221+0000","last_change":"2024-05-21T21:51:16.928455+0000","last_active":"2024-05-21T21:52:10.865221+0000","last_peered":"2024-05-21T21:52:10.865221+0000","last_clean":"2024-05-21T21:52:10.865221+0000","last_became_active":"2024-05-21T21:51:16.927874+0000","last_became_peered":"2024-05-21T21:51:16.927874+0000","last_unstale":"2024-05-21T21:52:10.865221+0000","last_undegraded":"2024-05-21T21:52:10.865221+0000","last_fullsized":"2024-05-21T21:52:10.865221+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:25:16.483175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.863776+0000","last_change":"2024-05-21T21:51:16.937773+0000","last_active":"2024-05-21T21:52:10.863776+0000","last_peered":"2024-05-21T21:52:10.863776+0000","last_clean":"2024-05-21T21:52:10.863776+0000","last_became_active":"2024-05-21T21:51:16.937306+0000","last_became_peered":"2024-05-21T21:51:16.937306+0000","last_unstale":"2024-05-21T21:52:10.863776+0000","last_undegraded":"2024-05-21T21:52:10.863776+0000","last_fullsized":"2024-05-21T21:52:10.863776+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:05:03.762390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.865632+0000","last_change":"2024-05-21T21:51:16.948275+0000","last_active":"2024-05-21T21:52:10.865632+0000","last_peered":"2024-05-21T21:52:10.865632+0000","last_clean":"2024-05-21T21:52:10.865632+0000","last_became_active":"2024-05-21T21:51:16.947890+0000","last_became_peered":"2024-05-21T21:51:16.947890+0000","last_unstale":"2024-05-21T21:52:10.865632+0000","last_undegraded":"2024-05-21T21:52:10.865632+0000","last_fullsized":"2024-05-21T21:52:10.865632+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:08:38.572177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.864396+0000","last_change":"2024-05-21T21:51:16.946821+0000","last_active":"2024-05-21T21:52:10.864396+0000","last_peered":"2024-05-21T21:52:10.864396+0000","last_clean":"2024-05-21T21:52:10.864396+0000","last_became_active":"2024-05-21T21:51:16.946515+0000","last_became_peered":"2024-05-21T21:51:16.946515+0000","last_unstale":"2024-05-21T21:52:10.864396+0000","last_undegraded":"2024-05-21T21:52:10.864396+0000","last_fullsized":"2024-05-21T21:52:10.864396+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T09:26:25.867895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.870476+0000","last_change":"2024-05-21T21:51:16.929228+0000","last_active":"2024-05-21T21:52:10.870476+0000","last_peered":"2024-05-21T21:52:10.870476+0000","last_clean":"2024-05-21T21:52:10.870476+0000","last_became_active":"2024-05-21T21:51:16.928875+0000","last_became_peered":"2024-05-21T21:51:16.928875+0000","last_unstale":"2024-05-21T21:52:10.870476+0000","last_undegraded":"2024-05-21T21:52:10.870476+0000","last_fullsized":"2024-05-21T21:52:10.870476+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:50:56.431310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.973145+0000","last_change":"2024-05-21T21:51:16.932360+0000","last_active":"2024-05-21T21:52:10.973145+0000","last_peered":"2024-05-21T21:52:10.973145+0000","last_clean":"2024-05-21T21:52:10.973145+0000","last_became_active":"2024-05-21T21:51:16.932036+0000","last_became_peered":"2024-05-21T21:51:16.932036+0000","last_unstale":"2024-05-21T21:52:10.973145+0000","last_undegraded":"2024-05-21T21:52:10.973145+0000","last_fullsized":"2024-05-21T21:52:10.973145+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T09:06:37.257323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.876556+0000","last_change":"2024-05-21T21:52:10.876556+0000","last_active":"2024-05-21T21:52:10.876556+0000","last_peered":"2024-05-21T21:52:10.876556+0000","last_clean":"2024-05-21T21:52:10.876556+0000","last_became_active":"2024-05-21T21:52:10.876273+0000","last_became_peered":"2024-05-21T21:52:10.876273+0000","last_unstale":"2024-05-21T21:52:10.876556+0000","last_undegraded":"2024-05-21T21:52:10.876556+0000","last_fullsized":"2024-05-21T21:52:10.876556+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T05:09:40.037731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.864226+0000","last_change":"2024-05-21T21:51:16.933425+0000","last_active":"2024-05-21T21:52:10.864226+0000","last_peered":"2024-05-21T21:52:10.864226+0000","last_clean":"2024-05-21T21:52:10.864226+0000","last_became_active":"2024-05-21T21:51:16.933057+0000","last_became_peered":"2024-05-21T21:51:16.933057+0000","last_unstale":"2024-05-21T21:52:10.864226+0000","last_undegraded":"2024-05-21T21:52:10.864226+0000","last_fullsized":"2024-05-21T21:52:10.864226+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:04:57.237841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.865179+0000","last_change":"2024-05-21T21:51:16.927014+0000","last_active":"2024-05-21T21:52:10.865179+0000","last_peered":"2024-05-21T21:52:10.865179+0000","last_clean":"2024-05-21T21:52:10.865179+0000","last_became_active":"2024-05-21T21:51:16.925954+0000","last_became_peered":"2024-05-21T21:51:16.925954+0000","last_unstale":"2024-05-21T21:52:10.865179+0000","last_undegraded":"2024-05-21T21:52:10.865179+0000","last_fullsized":"2024-05-21T21:52:10.865179+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:19:20.530727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.862761+0000","last_change":"2024-05-21T21:51:16.930909+0000","last_active":"2024-05-21T21:52:10.862761+0000","last_peered":"2024-05-21T21:52:10.862761+0000","last_clean":"2024-05-21T21:52:10.862761+0000","last_became_active":"2024-05-21T21:51:16.927110+0000","last_became_peered":"2024-05-21T21:51:16.927110+0000","last_unstale":"2024-05-21T21:52:10.862761+0000","last_undegraded":"2024-05-21T21:52:10.862761+0000","last_fullsized":"2024-05-21T21:52:10.862761+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T06:23:55.386047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.872086+0000","last_change":"2024-05-21T21:51:16.938797+0000","last_active":"2024-05-21T21:52:10.872086+0000","last_peered":"2024-05-21T21:52:10.872086+0000","last_clean":"2024-05-21T21:52:10.872086+0000","last_became_active":"2024-05-21T21:51:16.938263+0000","last_became_peered":"2024-05-21T21:51:16.938263+0000","last_unstale":"2024-05-21T21:52:10.872086+0000","last_undegraded":"2024-05-21T21:52:10.872086+0000","last_fullsized":"2024-05-21T21:52:10.872086+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:37:40.521118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.871620+0000","last_change":"2024-05-21T21:51:16.944160+0000","last_active":"2024-05-21T21:52:10.871620+0000","last_peered":"2024-05-21T21:52:10.871620+0000","last_clean":"2024-05-21T21:52:10.871620+0000","last_became_active":"2024-05-21T21:51:16.940046+0000","last_became_peered":"2024-05-21T21:51:16.940046+0000","last_unstale":"2024-05-21T21:52:10.871620+0000","last_undegraded":"2024-05-21T21:52:10.871620+0000","last_fullsized":"2024-05-21T21:52:10.871620+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:14:13.278799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.973152+0000","last_change":"2024-05-21T21:51:16.945599+0000","last_active":"2024-05-21T21:52:10.973152+0000","last_peered":"2024-05-21T21:52:10.973152+0000","last_clean":"2024-05-21T21:52:10.973152+0000","last_became_active":"2024-05-21T21:51:16.945196+0000","last_became_peered":"2024-05-21T21:51:16.945196+0000","last_unstale":"2024-05-21T21:52:10.973152+0000","last_undegraded":"2024-05-21T21:52:10.973152+0000","last_fullsized":"2024-05-21T21:52:10.973152+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:59:24.453615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.861779+0000","last_change":"2024-05-21T21:51:16.944895+0000","last_active":"2024-05-21T21:52:10.861779+0000","last_peered":"2024-05-21T21:52:10.861779+0000","last_clean":"2024-05-21T21:52:10.861779+0000","last_became_active":"2024-05-21T21:51:16.944560+0000","last_became_peered":"2024-05-21T21:51:16.944560+0000","last_unstale":"2024-05-21T21:52:10.861779+0000","last_undegraded":"2024-05-21T21:52:10.861779+0000","last_fullsized":"2024-05-21T21:52:10.861779+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:53:13.264701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.877457+0000","last_change":"2024-05-21T21:51:16.943861+0000","last_active":"2024-05-21T21:52:10.877457+0000","last_peered":"2024-05-21T21:52:10.877457+0000","last_clean":"2024-05-21T21:52:10.877457+0000","last_became_active":"2024-05-21T21:51:16.943123+0000","last_became_peered":"2024-05-21T21:51:16.943123+0000","last_unstale":"2024-05-21T21:52:10.877457+0000","last_undegraded":"2024-05-21T21:52:10.877457+0000","last_fullsized":"2024-05-21T21:52:10.877457+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:35:37.732855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.864479+0000","last_change":"2024-05-21T21:51:16.940580+0000","last_active":"2024-05-21T21:52:10.864479+0000","last_peered":"2024-05-21T21:52:10.864479+0000","last_clean":"2024-05-21T21:52:10.864479+0000","last_became_active":"2024-05-21T21:51:16.940264+0000","last_became_peered":"2024-05-21T21:51:16.940264+0000","last_unstale":"2024-05-21T21:52:10.864479+0000","last_undegraded":"2024-05-21T21:52:10.864479+0000","last_fullsized":"2024-05-21T21:52:10.864479+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:54:22.915541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"80'13","reported_seq":91,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:28.479532+0000","last_change":"2024-05-21T21:51:12.852935+0000","last_active":"2024-05-21T21:52:28.479532+0000","last_peered":"2024-05-21T21:52:28.479532+0000","last_clean":"2024-05-21T21:52:28.479532+0000","last_became_active":"2024-05-21T21:51:12.851802+0000","last_became_peered":"2024-05-21T21:51:12.851802+0000","last_unstale":"2024-05-21T21:52:28.479532+0000","last_undegraded":"2024-05-21T21:52:28.479532+0000","last_fullsized":"2024-05-21T21:52:28.479532+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:13:24.643408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":52,"num_read_kb":52,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.868253+0000","last_change":"2024-05-21T21:51:16.939461+0000","last_active":"2024-05-21T21:52:10.868253+0000","last_peered":"2024-05-21T21:52:10.868253+0000","last_clean":"2024-05-21T21:52:10.868253+0000","last_became_active":"2024-05-21T21:51:16.936322+0000","last_became_peered":"2024-05-21T21:51:16.936322+0000","last_unstale":"2024-05-21T21:52:10.868253+0000","last_undegraded":"2024-05-21T21:52:10.868253+0000","last_fullsized":"2024-05-21T21:52:10.868253+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:37:26.153963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.868674+0000","last_change":"2024-05-21T21:51:12.941838+0000","last_active":"2024-05-21T21:52:10.868674+0000","last_peered":"2024-05-21T21:52:10.868674+0000","last_clean":"2024-05-21T21:52:10.868674+0000","last_became_active":"2024-05-21T21:51:12.940818+0000","last_became_peered":"2024-05-21T21:51:12.940818+0000","last_unstale":"2024-05-21T21:52:10.868674+0000","last_undegraded":"2024-05-21T21:52:10.868674+0000","last_fullsized":"2024-05-21T21:52:10.868674+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:30:42.349241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.873994+0000","last_change":"2024-05-21T21:51:16.950252+0000","last_active":"2024-05-21T21:52:10.873994+0000","last_peered":"2024-05-21T21:52:10.873994+0000","last_clean":"2024-05-21T21:52:10.873994+0000","last_became_active":"2024-05-21T21:51:16.947375+0000","last_became_peered":"2024-05-21T21:51:16.947375+0000","last_unstale":"2024-05-21T21:52:10.873994+0000","last_undegraded":"2024-05-21T21:52:10.873994+0000","last_fullsized":"2024-05-21T21:52:10.873994+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:27:30.462799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"80'12","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.876122+0000","last_change":"2024-05-21T21:52:10.875739+0000","last_active":"2024-05-21T21:52:10.876122+0000","last_peered":"2024-05-21T21:52:10.876122+0000","last_clean":"2024-05-21T21:52:10.876122+0000","last_became_active":"2024-05-21T21:52:10.865846+0000","last_became_peered":"2024-05-21T21:52:10.865846+0000","last_unstale":"2024-05-21T21:52:10.876122+0000","last_undegraded":"2024-05-21T21:52:10.876122+0000","last_fullsized":"2024-05-21T21:52:10.876122+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T06:48:06.465757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.972759+0000","last_change":"2024-05-21T21:51:16.926380+0000","last_active":"2024-05-21T21:52:10.972759+0000","last_peered":"2024-05-21T21:52:10.972759+0000","last_clean":"2024-05-21T21:52:10.972759+0000","last_became_active":"2024-05-21T21:51:16.925821+0000","last_became_peered":"2024-05-21T21:51:16.925821+0000","last_unstale":"2024-05-21T21:52:10.972759+0000","last_undegraded":"2024-05-21T21:52:10.972759+0000","last_fullsized":"2024-05-21T21:52:10.972759+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:42:14.887155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'5","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.864272+0000","last_change":"2024-05-21T21:51:12.854342+0000","last_active":"2024-05-21T21:52:10.864272+0000","last_peered":"2024-05-21T21:52:10.864272+0000","last_clean":"2024-05-21T21:52:10.864272+0000","last_became_active":"2024-05-21T21:51:12.853908+0000","last_became_peered":"2024-05-21T21:51:12.853908+0000","last_unstale":"2024-05-21T21:52:10.864272+0000","last_undegraded":"2024-05-21T21:52:10.864272+0000","last_fullsized":"2024-05-21T21:52:10.864272+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:35:10.374749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.973816+0000","last_change":"2024-05-21T21:52:10.973816+0000","last_active":"2024-05-21T21:52:10.973816+0000","last_peered":"2024-05-21T21:52:10.973816+0000","last_clean":"2024-05-21T21:52:10.973816+0000","last_became_active":"2024-05-21T21:52:10.973457+0000","last_became_peered":"2024-05-21T21:52:10.973457+0000","last_unstale":"2024-05-21T21:52:10.973816+0000","last_undegraded":"2024-05-21T21:52:10.973816+0000","last_fullsized":"2024-05-21T21:52:10.973816+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T05:01:41.915158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"80'6","reported_seq":89,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:31.473827+0000","last_change":"2024-05-21T21:51:12.855658+0000","last_active":"2024-05-21T21:52:31.473827+0000","last_peered":"2024-05-21T21:52:31.473827+0000","last_clean":"2024-05-21T21:52:31.473827+0000","last_became_active":"2024-05-21T21:51:12.855281+0000","last_became_peered":"2024-05-21T21:51:12.855281+0000","last_unstale":"2024-05-21T21:52:31.473827+0000","last_undegraded":"2024-05-21T21:52:31.473827+0000","last_fullsized":"2024-05-21T21:52:31.473827+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:55:08.836045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":57,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.863544+0000","last_change":"2024-05-21T21:51:16.946977+0000","last_active":"2024-05-21T21:52:10.863544+0000","last_peered":"2024-05-21T21:52:10.863544+0000","last_clean":"2024-05-21T21:52:10.863544+0000","last_became_active":"2024-05-21T21:51:16.946562+0000","last_became_peered":"2024-05-21T21:51:16.946562+0000","last_unstale":"2024-05-21T21:52:10.863544+0000","last_undegraded":"2024-05-21T21:52:10.863544+0000","last_fullsized":"2024-05-21T21:52:10.863544+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:43:33.370265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"80'7","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.865540+0000","last_change":"2024-05-21T21:51:12.852574+0000","last_active":"2024-05-21T21:52:10.865540+0000","last_peered":"2024-05-21T21:52:10.865540+0000","last_clean":"2024-05-21T21:52:10.865540+0000","last_became_active":"2024-05-21T21:51:12.852258+0000","last_became_peered":"2024-05-21T21:51:12.852258+0000","last_unstale":"2024-05-21T21:52:10.865540+0000","last_undegraded":"2024-05-21T21:52:10.865540+0000","last_fullsized":"2024-05-21T21:52:10.865540+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:09:40.280821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.867344+0000","last_change":"2024-05-21T21:51:16.934768+0000","last_active":"2024-05-21T21:52:10.867344+0000","last_peered":"2024-05-21T21:52:10.867344+0000","last_clean":"2024-05-21T21:52:10.867344+0000","last_became_active":"2024-05-21T21:51:16.934275+0000","last_became_peered":"2024-05-21T21:51:16.934275+0000","last_unstale":"2024-05-21T21:52:10.867344+0000","last_undegraded":"2024-05-21T21:52:10.867344+0000","last_fullsized":"2024-05-21T21:52:10.867344+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:31:54.425069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"80'3","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.861882+0000","last_change":"2024-05-21T21:51:12.854166+0000","last_active":"2024-05-21T21:52:10.861882+0000","last_peered":"2024-05-21T21:52:10.861882+0000","last_clean":"2024-05-21T21:52:10.861882+0000","last_became_active":"2024-05-21T21:51:12.853508+0000","last_became_peered":"2024-05-21T21:51:12.853508+0000","last_unstale":"2024-05-21T21:52:10.861882+0000","last_undegraded":"2024-05-21T21:52:10.861882+0000","last_fullsized":"2024-05-21T21:52:10.861882+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:29:41.858655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.870323+0000","last_change":"2024-05-21T21:51:16.938293+0000","last_active":"2024-05-21T21:52:10.870323+0000","last_peered":"2024-05-21T21:52:10.870323+0000","last_clean":"2024-05-21T21:52:10.870323+0000","last_became_active":"2024-05-21T21:51:16.937970+0000","last_became_peered":"2024-05-21T21:51:16.937970+0000","last_unstale":"2024-05-21T21:52:10.870323+0000","last_undegraded":"2024-05-21T21:52:10.870323+0000","last_fullsized":"2024-05-21T21:52:10.870323+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:18:27.473098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'3","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.866723+0000","last_change":"2024-05-21T21:51:12.853366+0000","last_active":"2024-05-21T21:52:10.866723+0000","last_peered":"2024-05-21T21:52:10.866723+0000","last_clean":"2024-05-21T21:52:10.866723+0000","last_became_active":"2024-05-21T21:51:12.853028+0000","last_became_peered":"2024-05-21T21:51:12.853028+0000","last_unstale":"2024-05-21T21:52:10.866723+0000","last_undegraded":"2024-05-21T21:52:10.866723+0000","last_fullsized":"2024-05-21T21:52:10.866723+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T09:15:52.593028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.874880+0000","last_change":"2024-05-21T21:51:16.947903+0000","last_active":"2024-05-21T21:52:10.874880+0000","last_peered":"2024-05-21T21:52:10.874880+0000","last_clean":"2024-05-21T21:52:10.874880+0000","last_became_active":"2024-05-21T21:51:16.946677+0000","last_became_peered":"2024-05-21T21:51:16.946677+0000","last_unstale":"2024-05-21T21:52:10.874880+0000","last_undegraded":"2024-05-21T21:52:10.874880+0000","last_fullsized":"2024-05-21T21:52:10.874880+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T06:12:56.514284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'4","reported_seq":136,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:31.451417+0000","last_change":"2024-05-21T21:51:12.854873+0000","last_active":"2024-05-21T21:52:31.451417+0000","last_peered":"2024-05-21T21:52:31.451417+0000","last_clean":"2024-05-21T21:52:31.451417+0000","last_became_active":"2024-05-21T21:51:12.854393+0000","last_became_peered":"2024-05-21T21:51:12.854393+0000","last_unstale":"2024-05-21T21:52:31.451417+0000","last_undegraded":"2024-05-21T21:52:31.451417+0000","last_fullsized":"2024-05-21T21:52:31.451417+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:58:52.411164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":106,"num_read_kb":106,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.973117+0000","last_change":"2024-05-21T21:51:16.951094+0000","last_active":"2024-05-21T21:52:10.973117+0000","last_peered":"2024-05-21T21:52:10.973117+0000","last_clean":"2024-05-21T21:52:10.973117+0000","last_became_active":"2024-05-21T21:51:16.950724+0000","last_became_peered":"2024-05-21T21:51:16.950724+0000","last_unstale":"2024-05-21T21:52:10.973117+0000","last_undegraded":"2024-05-21T21:52:10.973117+0000","last_fullsized":"2024-05-21T21:52:10.973117+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:54:02.835426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'4","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.864127+0000","last_change":"2024-05-21T21:51:12.850427+0000","last_active":"2024-05-21T21:52:10.864127+0000","last_peered":"2024-05-21T21:52:10.864127+0000","last_clean":"2024-05-21T21:52:10.864127+0000","last_became_active":"2024-05-21T21:51:12.850096+0000","last_became_peered":"2024-05-21T21:51:12.850096+0000","last_unstale":"2024-05-21T21:52:10.864127+0000","last_undegraded":"2024-05-21T21:52:10.864127+0000","last_fullsized":"2024-05-21T21:52:10.864127+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:43:22.611043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.874050+0000","last_change":"2024-05-21T21:51:16.942330+0000","last_active":"2024-05-21T21:52:10.874050+0000","last_peered":"2024-05-21T21:52:10.874050+0000","last_clean":"2024-05-21T21:52:10.874050+0000","last_became_active":"2024-05-21T21:51:16.941718+0000","last_became_peered":"2024-05-21T21:51:16.941718+0000","last_unstale":"2024-05-21T21:52:10.874050+0000","last_undegraded":"2024-05-21T21:52:10.874050+0000","last_fullsized":"2024-05-21T21:52:10.874050+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:54:17.176411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'5","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.867578+0000","last_change":"2024-05-21T21:51:12.941100+0000","last_active":"2024-05-21T21:52:10.867578+0000","last_peered":"2024-05-21T21:52:10.867578+0000","last_clean":"2024-05-21T21:52:10.867578+0000","last_became_active":"2024-05-21T21:51:12.940597+0000","last_became_peered":"2024-05-21T21:51:12.940597+0000","last_unstale":"2024-05-21T21:52:10.867578+0000","last_undegraded":"2024-05-21T21:52:10.867578+0000","last_fullsized":"2024-05-21T21:52:10.867578+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:30:58.457108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.972855+0000","last_change":"2024-05-21T21:51:16.921973+0000","last_active":"2024-05-21T21:52:10.972855+0000","last_peered":"2024-05-21T21:52:10.972855+0000","last_clean":"2024-05-21T21:52:10.972855+0000","last_became_active":"2024-05-21T21:51:16.921597+0000","last_became_peered":"2024-05-21T21:51:16.921597+0000","last_unstale":"2024-05-21T21:52:10.972855+0000","last_undegraded":"2024-05-21T21:52:10.972855+0000","last_fullsized":"2024-05-21T21:52:10.972855+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:25:32.395940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'31","reported_seq":172,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:31.451420+0000","last_change":"2024-05-21T21:51:12.850312+0000","last_active":"2024-05-21T21:52:31.451420+0000","last_peered":"2024-05-21T21:52:31.451420+0000","last_clean":"2024-05-21T21:52:31.451420+0000","last_became_active":"2024-05-21T21:51:12.849837+0000","last_became_peered":"2024-05-21T21:51:12.849837+0000","last_unstale":"2024-05-21T21:52:31.451420+0000","last_undegraded":"2024-05-21T21:52:31.451420+0000","last_fullsized":"2024-05-21T21:52:31.451420+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T07:09:13.339554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59762,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":115,"num_read_kb":115,"num_write":33,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.863568+0000","last_change":"2024-05-21T21:51:16.926625+0000","last_active":"2024-05-21T21:52:10.863568+0000","last_peered":"2024-05-21T21:52:10.863568+0000","last_clean":"2024-05-21T21:52:10.863568+0000","last_became_active":"2024-05-21T21:51:16.925889+0000","last_became_peered":"2024-05-21T21:51:16.925889+0000","last_unstale":"2024-05-21T21:52:10.863568+0000","last_undegraded":"2024-05-21T21:52:10.863568+0000","last_fullsized":"2024-05-21T21:52:10.863568+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:35:23.464747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'9","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.863891+0000","last_change":"2024-05-21T21:51:12.859772+0000","last_active":"2024-05-21T21:52:10.863891+0000","last_peered":"2024-05-21T21:52:10.863891+0000","last_clean":"2024-05-21T21:52:10.863891+0000","last_became_active":"2024-05-21T21:51:12.859290+0000","last_became_peered":"2024-05-21T21:51:12.859290+0000","last_unstale":"2024-05-21T21:52:10.863891+0000","last_undegraded":"2024-05-21T21:52:10.863891+0000","last_fullsized":"2024-05-21T21:52:10.863891+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T05:28:29.953381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.861761+0000","last_change":"2024-05-21T21:51:16.925803+0000","last_active":"2024-05-21T21:52:10.861761+0000","last_peered":"2024-05-21T21:52:10.861761+0000","last_clean":"2024-05-21T21:52:10.861761+0000","last_became_active":"2024-05-21T21:51:16.925111+0000","last_became_peered":"2024-05-21T21:51:16.925111+0000","last_unstale":"2024-05-21T21:52:10.861761+0000","last_undegraded":"2024-05-21T21:52:10.861761+0000","last_fullsized":"2024-05-21T21:52:10.861761+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T06:22:11.353379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'5","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.867329+0000","last_change":"2024-05-21T21:51:12.859593+0000","last_active":"2024-05-21T21:52:10.867329+0000","last_peered":"2024-05-21T21:52:10.867329+0000","last_clean":"2024-05-21T21:52:10.867329+0000","last_became_active":"2024-05-21T21:51:12.859197+0000","last_became_peered":"2024-05-21T21:51:12.859197+0000","last_unstale":"2024-05-21T21:52:10.867329+0000","last_undegraded":"2024-05-21T21:52:10.867329+0000","last_fullsized":"2024-05-21T21:52:10.867329+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:13:12.311496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.879028+0000","last_change":"2024-05-21T21:52:10.879028+0000","last_active":"2024-05-21T21:52:10.879028+0000","last_peered":"2024-05-21T21:52:10.879028+0000","last_clean":"2024-05-21T21:52:10.879028+0000","last_became_active":"2024-05-21T21:52:10.878664+0000","last_became_peered":"2024-05-21T21:52:10.878664+0000","last_unstale":"2024-05-21T21:52:10.879028+0000","last_undegraded":"2024-05-21T21:52:10.879028+0000","last_fullsized":"2024-05-21T21:52:10.879028+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:05:53.953546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'6","reported_seq":89,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:31.464826+0000","last_change":"2024-05-21T21:51:12.852169+0000","last_active":"2024-05-21T21:52:31.464826+0000","last_peered":"2024-05-21T21:52:31.464826+0000","last_clean":"2024-05-21T21:52:31.464826+0000","last_became_active":"2024-05-21T21:51:12.851782+0000","last_became_peered":"2024-05-21T21:51:12.851782+0000","last_unstale":"2024-05-21T21:52:31.464826+0000","last_undegraded":"2024-05-21T21:52:31.464826+0000","last_fullsized":"2024-05-21T21:52:31.464826+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:46:09.692158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":58,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.863791+0000","last_change":"2024-05-21T21:51:16.929264+0000","last_active":"2024-05-21T21:52:10.863791+0000","last_peered":"2024-05-21T21:52:10.863791+0000","last_clean":"2024-05-21T21:52:10.863791+0000","last_became_active":"2024-05-21T21:51:16.928900+0000","last_became_peered":"2024-05-21T21:51:16.928900+0000","last_unstale":"2024-05-21T21:52:10.863791+0000","last_undegraded":"2024-05-21T21:52:10.863791+0000","last_fullsized":"2024-05-21T21:52:10.863791+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:42:28.040939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'17","reported_seq":68,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.984451+0000","last_change":"2024-05-21T21:52:10.984451+0000","last_active":"2024-05-21T21:52:10.984451+0000","last_peered":"2024-05-21T21:52:10.984451+0000","last_clean":"2024-05-21T21:52:10.984451+0000","last_became_active":"2024-05-21T21:52:10.974050+0000","last_became_peered":"2024-05-21T21:52:10.974050+0000","last_unstale":"2024-05-21T21:52:10.984451+0000","last_undegraded":"2024-05-21T21:52:10.984451+0000","last_fullsized":"2024-05-21T21:52:10.984451+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T07:08:18.307022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.867149+0000","last_change":"2024-05-21T21:51:16.935464+0000","last_active":"2024-05-21T21:52:10.867149+0000","last_peered":"2024-05-21T21:52:10.867149+0000","last_clean":"2024-05-21T21:52:10.867149+0000","last_became_active":"2024-05-21T21:51:16.935091+0000","last_became_peered":"2024-05-21T21:51:16.935091+0000","last_unstale":"2024-05-21T21:52:10.867149+0000","last_undegraded":"2024-05-21T21:52:10.867149+0000","last_fullsized":"2024-05-21T21:52:10.867149+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T06:59:16.330649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.864530+0000","last_change":"2024-05-21T21:51:12.853327+0000","last_active":"2024-05-21T21:52:10.864530+0000","last_peered":"2024-05-21T21:52:10.864530+0000","last_clean":"2024-05-21T21:52:10.864530+0000","last_became_active":"2024-05-21T21:51:12.852946+0000","last_became_peered":"2024-05-21T21:51:12.852946+0000","last_unstale":"2024-05-21T21:52:10.864530+0000","last_undegraded":"2024-05-21T21:52:10.864530+0000","last_fullsized":"2024-05-21T21:52:10.864530+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:00:34.256858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.861583+0000","last_change":"2024-05-21T21:51:16.929341+0000","last_active":"2024-05-21T21:52:10.861583+0000","last_peered":"2024-05-21T21:52:10.861583+0000","last_clean":"2024-05-21T21:52:10.861583+0000","last_became_active":"2024-05-21T21:51:16.929008+0000","last_became_peered":"2024-05-21T21:51:16.929008+0000","last_unstale":"2024-05-21T21:52:10.861583+0000","last_undegraded":"2024-05-21T21:52:10.861583+0000","last_fullsized":"2024-05-21T21:52:10.861583+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:26:26.297136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"80'1","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.875960+0000","last_change":"2024-05-21T21:51:12.945433+0000","last_active":"2024-05-21T21:52:10.875960+0000","last_peered":"2024-05-21T21:52:10.875960+0000","last_clean":"2024-05-21T21:52:10.875960+0000","last_became_active":"2024-05-21T21:51:12.944463+0000","last_became_peered":"2024-05-21T21:51:12.944463+0000","last_unstale":"2024-05-21T21:52:10.875960+0000","last_undegraded":"2024-05-21T21:52:10.875960+0000","last_fullsized":"2024-05-21T21:52:10.875960+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:29:52.014583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'1","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.883554+0000","last_change":"2024-05-21T21:52:10.883554+0000","last_active":"2024-05-21T21:52:10.883554+0000","last_peered":"2024-05-21T21:52:10.883554+0000","last_clean":"2024-05-21T21:52:10.883554+0000","last_became_active":"2024-05-21T21:52:10.877789+0000","last_became_peered":"2024-05-21T21:52:10.877789+0000","last_unstale":"2024-05-21T21:52:10.883554+0000","last_undegraded":"2024-05-21T21:52:10.883554+0000","last_fullsized":"2024-05-21T21:52:10.883554+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:52:37.623551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"80'5","reported_seq":84,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:29.482938+0000","last_change":"2024-05-21T21:51:12.852883+0000","last_active":"2024-05-21T21:52:29.482938+0000","last_peered":"2024-05-21T21:52:29.482938+0000","last_clean":"2024-05-21T21:52:29.482938+0000","last_became_active":"2024-05-21T21:51:12.852549+0000","last_became_peered":"2024-05-21T21:51:12.852549+0000","last_unstale":"2024-05-21T21:52:29.482938+0000","last_undegraded":"2024-05-21T21:52:29.482938+0000","last_fullsized":"2024-05-21T21:52:29.482938+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:05:53.002286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":53,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.865777+0000","last_change":"2024-05-21T21:51:16.944425+0000","last_active":"2024-05-21T21:52:10.865777+0000","last_peered":"2024-05-21T21:52:10.865777+0000","last_clean":"2024-05-21T21:52:10.865777+0000","last_became_active":"2024-05-21T21:51:16.944033+0000","last_became_peered":"2024-05-21T21:51:16.944033+0000","last_unstale":"2024-05-21T21:52:10.865777+0000","last_undegraded":"2024-05-21T21:52:10.865777+0000","last_fullsized":"2024-05-21T21:52:10.865777+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T07:16:26.802329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.866923+0000","last_change":"2024-05-21T21:51:16.933264+0000","last_active":"2024-05-21T21:52:10.866923+0000","last_peered":"2024-05-21T21:52:10.866923+0000","last_clean":"2024-05-21T21:52:10.866923+0000","last_became_active":"2024-05-21T21:51:16.932911+0000","last_became_peered":"2024-05-21T21:51:16.932911+0000","last_unstale":"2024-05-21T21:52:10.866923+0000","last_undegraded":"2024-05-21T21:52:10.866923+0000","last_fullsized":"2024-05-21T21:52:10.866923+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:25:54.006367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"80'5","reported_seq":83,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:28.458359+0000","last_change":"2024-05-21T21:51:12.942900+0000","last_active":"2024-05-21T21:52:28.458359+0000","last_peered":"2024-05-21T21:52:28.458359+0000","last_clean":"2024-05-21T21:52:28.458359+0000","last_became_active":"2024-05-21T21:51:12.941038+0000","last_became_peered":"2024-05-21T21:51:12.941038+0000","last_unstale":"2024-05-21T21:52:28.458359+0000","last_undegraded":"2024-05-21T21:52:28.458359+0000","last_fullsized":"2024-05-21T21:52:28.458359+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:38:43.240011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":52,"num_read_kb":52,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.867131+0000","last_change":"2024-05-21T21:51:16.933391+0000","last_active":"2024-05-21T21:52:10.867131+0000","last_peered":"2024-05-21T21:52:10.867131+0000","last_clean":"2024-05-21T21:52:10.867131+0000","last_became_active":"2024-05-21T21:51:16.932320+0000","last_became_peered":"2024-05-21T21:51:16.932320+0000","last_unstale":"2024-05-21T21:52:10.867131+0000","last_undegraded":"2024-05-21T21:52:10.867131+0000","last_fullsized":"2024-05-21T21:52:10.867131+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T09:06:59.126748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'6","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.865206+0000","last_change":"2024-05-21T21:51:12.856342+0000","last_active":"2024-05-21T21:52:10.865206+0000","last_peered":"2024-05-21T21:52:10.865206+0000","last_clean":"2024-05-21T21:52:10.865206+0000","last_became_active":"2024-05-21T21:51:12.855952+0000","last_became_peered":"2024-05-21T21:51:12.855952+0000","last_unstale":"2024-05-21T21:52:10.865206+0000","last_undegraded":"2024-05-21T21:52:10.865206+0000","last_fullsized":"2024-05-21T21:52:10.865206+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:34:40.734034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":100,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.863960+0000","last_change":"2024-05-21T21:49:42.797187+0000","last_active":"2024-05-21T21:52:10.863960+0000","last_peered":"2024-05-21T21:52:10.863960+0000","last_clean":"2024-05-21T21:52:10.863960+0000","last_became_active":"2024-05-21T21:49:42.791291+0000","last_became_peered":"2024-05-21T21:49:42.791291+0000","last_unstale":"2024-05-21T21:52:10.863960+0000","last_undegraded":"2024-05-21T21:52:10.863960+0000","last_fullsized":"2024-05-21T21:52:10.863960+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:45:55.552293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:45:55.552293+0000","last_clean_scrub_stamp":"2024-05-21T21:45:55.552293+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T07:24:18.302523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.861715+0000","last_change":"2024-05-21T21:51:16.926446+0000","last_active":"2024-05-21T21:52:10.861715+0000","last_peered":"2024-05-21T21:52:10.861715+0000","last_clean":"2024-05-21T21:52:10.861715+0000","last_became_active":"2024-05-21T21:51:16.926070+0000","last_became_peered":"2024-05-21T21:51:16.926070+0000","last_unstale":"2024-05-21T21:52:10.861715+0000","last_undegraded":"2024-05-21T21:52:10.861715+0000","last_fullsized":"2024-05-21T21:52:10.861715+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:44:27.366393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'4","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.864417+0000","last_change":"2024-05-21T21:51:12.854371+0000","last_active":"2024-05-21T21:52:10.864417+0000","last_peered":"2024-05-21T21:52:10.864417+0000","last_clean":"2024-05-21T21:52:10.864417+0000","last_became_active":"2024-05-21T21:51:12.853847+0000","last_became_peered":"2024-05-21T21:51:12.853847+0000","last_unstale":"2024-05-21T21:52:10.864417+0000","last_undegraded":"2024-05-21T21:52:10.864417+0000","last_fullsized":"2024-05-21T21:52:10.864417+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:14:10.549901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"80'4","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.863685+0000","last_change":"2024-05-21T21:51:12.941227+0000","last_active":"2024-05-21T21:52:10.863685+0000","last_peered":"2024-05-21T21:52:10.863685+0000","last_clean":"2024-05-21T21:52:10.863685+0000","last_became_active":"2024-05-21T21:51:12.940702+0000","last_became_peered":"2024-05-21T21:51:12.940702+0000","last_unstale":"2024-05-21T21:52:10.863685+0000","last_undegraded":"2024-05-21T21:52:10.863685+0000","last_fullsized":"2024-05-21T21:52:10.863685+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:40:34.124921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.862247+0000","last_change":"2024-05-21T21:51:16.945627+0000","last_active":"2024-05-21T21:52:10.862247+0000","last_peered":"2024-05-21T21:52:10.862247+0000","last_clean":"2024-05-21T21:52:10.862247+0000","last_became_active":"2024-05-21T21:51:16.945297+0000","last_became_peered":"2024-05-21T21:51:16.945297+0000","last_unstale":"2024-05-21T21:52:10.862247+0000","last_undegraded":"2024-05-21T21:52:10.862247+0000","last_fullsized":"2024-05-21T21:52:10.862247+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:54:38.898774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'15","reported_seq":57,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.888076+0000","last_change":"2024-05-21T21:52:10.888076+0000","last_active":"2024-05-21T21:52:10.888076+0000","last_peered":"2024-05-21T21:52:10.888076+0000","last_clean":"2024-05-21T21:52:10.888076+0000","last_became_active":"2024-05-21T21:52:10.879368+0000","last_became_peered":"2024-05-21T21:52:10.879368+0000","last_unstale":"2024-05-21T21:52:10.888076+0000","last_undegraded":"2024-05-21T21:52:10.888076+0000","last_fullsized":"2024-05-21T21:52:10.888076+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:46:43.190411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.868811+0000","last_change":"2024-05-21T21:51:16.929807+0000","last_active":"2024-05-21T21:52:10.868811+0000","last_peered":"2024-05-21T21:52:10.868811+0000","last_clean":"2024-05-21T21:52:10.868811+0000","last_became_active":"2024-05-21T21:51:16.928859+0000","last_became_peered":"2024-05-21T21:51:16.928859+0000","last_unstale":"2024-05-21T21:52:10.868811+0000","last_undegraded":"2024-05-21T21:52:10.868811+0000","last_fullsized":"2024-05-21T21:52:10.868811+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:41:32.830892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'7","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.865597+0000","last_change":"2024-05-21T21:51:12.850843+0000","last_active":"2024-05-21T21:52:10.865597+0000","last_peered":"2024-05-21T21:52:10.865597+0000","last_clean":"2024-05-21T21:52:10.865597+0000","last_became_active":"2024-05-21T21:51:12.850516+0000","last_became_peered":"2024-05-21T21:51:12.850516+0000","last_unstale":"2024-05-21T21:52:10.865597+0000","last_undegraded":"2024-05-21T21:52:10.865597+0000","last_fullsized":"2024-05-21T21:52:10.865597+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:37:05.191626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.873845+0000","last_change":"2024-05-21T21:51:16.949017+0000","last_active":"2024-05-21T21:52:10.873845+0000","last_peered":"2024-05-21T21:52:10.873845+0000","last_clean":"2024-05-21T21:52:10.873845+0000","last_became_active":"2024-05-21T21:51:16.948691+0000","last_became_peered":"2024-05-21T21:51:16.948691+0000","last_unstale":"2024-05-21T21:52:10.873845+0000","last_undegraded":"2024-05-21T21:52:10.873845+0000","last_fullsized":"2024-05-21T21:52:10.873845+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:32:06.606581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.973164+0000","last_change":"2024-05-21T21:51:12.939203+0000","last_active":"2024-05-21T21:52:10.973164+0000","last_peered":"2024-05-21T21:52:10.973164+0000","last_clean":"2024-05-21T21:52:10.973164+0000","last_became_active":"2024-05-21T21:51:12.938936+0000","last_became_peered":"2024-05-21T21:51:12.938936+0000","last_unstale":"2024-05-21T21:52:10.973164+0000","last_undegraded":"2024-05-21T21:52:10.973164+0000","last_fullsized":"2024-05-21T21:52:10.973164+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:55:50.336362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.863329+0000","last_change":"2024-05-21T21:51:16.926216+0000","last_active":"2024-05-21T21:52:10.863329+0000","last_peered":"2024-05-21T21:52:10.863329+0000","last_clean":"2024-05-21T21:52:10.863329+0000","last_became_active":"2024-05-21T21:51:16.925834+0000","last_became_peered":"2024-05-21T21:51:16.925834+0000","last_unstale":"2024-05-21T21:52:10.863329+0000","last_undegraded":"2024-05-21T21:52:10.863329+0000","last_fullsized":"2024-05-21T21:52:10.863329+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:13:30.192908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'7","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.866548+0000","last_change":"2024-05-21T21:51:12.855306+0000","last_active":"2024-05-21T21:52:10.866548+0000","last_peered":"2024-05-21T21:52:10.866548+0000","last_clean":"2024-05-21T21:52:10.866548+0000","last_became_active":"2024-05-21T21:51:12.854722+0000","last_became_peered":"2024-05-21T21:51:12.854722+0000","last_unstale":"2024-05-21T21:52:10.866548+0000","last_undegraded":"2024-05-21T21:52:10.866548+0000","last_fullsized":"2024-05-21T21:52:10.866548+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:39:41.583219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.861450+0000","last_change":"2024-05-21T21:51:16.944758+0000","last_active":"2024-05-21T21:52:10.861450+0000","last_peered":"2024-05-21T21:52:10.861450+0000","last_clean":"2024-05-21T21:52:10.861450+0000","last_became_active":"2024-05-21T21:51:16.944393+0000","last_became_peered":"2024-05-21T21:51:16.944393+0000","last_unstale":"2024-05-21T21:52:10.861450+0000","last_undegraded":"2024-05-21T21:52:10.861450+0000","last_fullsized":"2024-05-21T21:52:10.861450+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:01:23.048355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'3","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.863710+0000","last_change":"2024-05-21T21:51:12.860378+0000","last_active":"2024-05-21T21:52:10.863710+0000","last_peered":"2024-05-21T21:52:10.863710+0000","last_clean":"2024-05-21T21:52:10.863710+0000","last_became_active":"2024-05-21T21:51:12.859978+0000","last_became_peered":"2024-05-21T21:51:12.859978+0000","last_unstale":"2024-05-21T21:52:10.863710+0000","last_undegraded":"2024-05-21T21:52:10.863710+0000","last_fullsized":"2024-05-21T21:52:10.863710+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T07:52:26.902232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.867609+0000","last_change":"2024-05-21T21:51:16.943194+0000","last_active":"2024-05-21T21:52:10.867609+0000","last_peered":"2024-05-21T21:52:10.867609+0000","last_clean":"2024-05-21T21:52:10.867609+0000","last_became_active":"2024-05-21T21:51:16.942884+0000","last_became_peered":"2024-05-21T21:51:16.942884+0000","last_unstale":"2024-05-21T21:52:10.867609+0000","last_undegraded":"2024-05-21T21:52:10.867609+0000","last_fullsized":"2024-05-21T21:52:10.867609+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T05:58:17.436995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.863880+0000","last_change":"2024-05-21T21:51:12.853390+0000","last_active":"2024-05-21T21:52:10.863880+0000","last_peered":"2024-05-21T21:52:10.863880+0000","last_clean":"2024-05-21T21:52:10.863880+0000","last_became_active":"2024-05-21T21:51:12.852990+0000","last_became_peered":"2024-05-21T21:51:12.852990+0000","last_unstale":"2024-05-21T21:52:10.863880+0000","last_undegraded":"2024-05-21T21:52:10.863880+0000","last_fullsized":"2024-05-21T21:52:10.863880+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:32:16.392609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.864540+0000","last_change":"2024-05-21T21:51:16.933430+0000","last_active":"2024-05-21T21:52:10.864540+0000","last_peered":"2024-05-21T21:52:10.864540+0000","last_clean":"2024-05-21T21:52:10.864540+0000","last_became_active":"2024-05-21T21:51:16.933053+0000","last_became_peered":"2024-05-21T21:51:16.933053+0000","last_unstale":"2024-05-21T21:52:10.864540+0000","last_undegraded":"2024-05-21T21:52:10.864540+0000","last_fullsized":"2024-05-21T21:52:10.864540+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T06:14:57.468727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'5","reported_seq":85,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:29.448771+0000","last_change":"2024-05-21T21:51:12.946113+0000","last_active":"2024-05-21T21:52:29.448771+0000","last_peered":"2024-05-21T21:52:29.448771+0000","last_clean":"2024-05-21T21:52:29.448771+0000","last_became_active":"2024-05-21T21:51:12.945650+0000","last_became_peered":"2024-05-21T21:51:12.945650+0000","last_unstale":"2024-05-21T21:52:29.448771+0000","last_undegraded":"2024-05-21T21:52:29.448771+0000","last_fullsized":"2024-05-21T21:52:29.448771+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T09:40:18.209400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":54,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.864300+0000","last_change":"2024-05-21T21:51:16.932632+0000","last_active":"2024-05-21T21:52:10.864300+0000","last_peered":"2024-05-21T21:52:10.864300+0000","last_clean":"2024-05-21T21:52:10.864300+0000","last_became_active":"2024-05-21T21:51:16.932256+0000","last_became_peered":"2024-05-21T21:51:16.932256+0000","last_unstale":"2024-05-21T21:52:10.864300+0000","last_undegraded":"2024-05-21T21:52:10.864300+0000","last_fullsized":"2024-05-21T21:52:10.864300+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:04:48.146485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'3","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.867556+0000","last_change":"2024-05-21T21:51:12.860602+0000","last_active":"2024-05-21T21:52:10.867556+0000","last_peered":"2024-05-21T21:52:10.867556+0000","last_clean":"2024-05-21T21:52:10.867556+0000","last_became_active":"2024-05-21T21:51:12.860228+0000","last_became_peered":"2024-05-21T21:51:12.860228+0000","last_unstale":"2024-05-21T21:52:10.867556+0000","last_undegraded":"2024-05-21T21:52:10.867556+0000","last_fullsized":"2024-05-21T21:52:10.867556+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:33:39.230872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'1","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.863947+0000","last_change":"2024-05-21T21:51:16.929616+0000","last_active":"2024-05-21T21:52:10.863947+0000","last_peered":"2024-05-21T21:52:10.863947+0000","last_clean":"2024-05-21T21:52:10.863947+0000","last_became_active":"2024-05-21T21:51:16.928569+0000","last_became_peered":"2024-05-21T21:51:16.928569+0000","last_unstale":"2024-05-21T21:52:10.863947+0000","last_undegraded":"2024-05-21T21:52:10.863947+0000","last_fullsized":"2024-05-21T21:52:10.863947+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:15:41.790344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'5","reported_seq":43,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.881114+0000","last_change":"2024-05-21T21:52:10.881114+0000","last_active":"2024-05-21T21:52:10.881114+0000","last_peered":"2024-05-21T21:52:10.881114+0000","last_clean":"2024-05-21T21:52:10.881114+0000","last_became_active":"2024-05-21T21:52:10.872022+0000","last_became_peered":"2024-05-21T21:52:10.872022+0000","last_unstale":"2024-05-21T21:52:10.881114+0000","last_undegraded":"2024-05-21T21:52:10.881114+0000","last_fullsized":"2024-05-21T21:52:10.881114+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T06:05:51.419156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.864314+0000","last_change":"2024-05-21T21:51:16.926496+0000","last_active":"2024-05-21T21:52:10.864314+0000","last_peered":"2024-05-21T21:52:10.864314+0000","last_clean":"2024-05-21T21:52:10.864314+0000","last_became_active":"2024-05-21T21:51:16.926112+0000","last_became_peered":"2024-05-21T21:51:16.926112+0000","last_unstale":"2024-05-21T21:52:10.864314+0000","last_undegraded":"2024-05-21T21:52:10.864314+0000","last_fullsized":"2024-05-21T21:52:10.864314+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:26:08.254421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.861508+0000","last_change":"2024-05-21T21:51:16.930522+0000","last_active":"2024-05-21T21:52:10.861508+0000","last_peered":"2024-05-21T21:52:10.861508+0000","last_clean":"2024-05-21T21:52:10.861508+0000","last_became_active":"2024-05-21T21:51:16.929853+0000","last_became_peered":"2024-05-21T21:51:16.929853+0000","last_unstale":"2024-05-21T21:52:10.861508+0000","last_undegraded":"2024-05-21T21:52:10.861508+0000","last_fullsized":"2024-05-21T21:52:10.861508+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:10:57.940009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.863593+0000","last_change":"2024-05-21T21:51:16.931751+0000","last_active":"2024-05-21T21:52:10.863593+0000","last_peered":"2024-05-21T21:52:10.863593+0000","last_clean":"2024-05-21T21:52:10.863593+0000","last_became_active":"2024-05-21T21:51:16.931353+0000","last_became_peered":"2024-05-21T21:51:16.931353+0000","last_unstale":"2024-05-21T21:52:10.863593+0000","last_undegraded":"2024-05-21T21:52:10.863593+0000","last_fullsized":"2024-05-21T21:52:10.863593+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:33:45.297850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.863516+0000","last_change":"2024-05-21T21:51:16.940096+0000","last_active":"2024-05-21T21:52:10.863516+0000","last_peered":"2024-05-21T21:52:10.863516+0000","last_clean":"2024-05-21T21:52:10.863516+0000","last_became_active":"2024-05-21T21:51:16.939740+0000","last_became_peered":"2024-05-21T21:51:16.939740+0000","last_unstale":"2024-05-21T21:52:10.863516+0000","last_undegraded":"2024-05-21T21:52:10.863516+0000","last_fullsized":"2024-05-21T21:52:10.863516+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:59:52.569526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.872739+0000","last_change":"2024-05-21T21:51:16.946488+0000","last_active":"2024-05-21T21:52:10.872739+0000","last_peered":"2024-05-21T21:52:10.872739+0000","last_clean":"2024-05-21T21:52:10.872739+0000","last_became_active":"2024-05-21T21:51:16.945957+0000","last_became_peered":"2024-05-21T21:51:16.945957+0000","last_unstale":"2024-05-21T21:52:10.872739+0000","last_undegraded":"2024-05-21T21:52:10.872739+0000","last_fullsized":"2024-05-21T21:52:10.872739+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:40:36.708344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.862107+0000","last_change":"2024-05-21T21:51:16.949349+0000","last_active":"2024-05-21T21:52:10.862107+0000","last_peered":"2024-05-21T21:52:10.862107+0000","last_clean":"2024-05-21T21:52:10.862107+0000","last_became_active":"2024-05-21T21:51:16.946986+0000","last_became_peered":"2024-05-21T21:51:16.946986+0000","last_unstale":"2024-05-21T21:52:10.862107+0000","last_undegraded":"2024-05-21T21:52:10.862107+0000","last_fullsized":"2024-05-21T21:52:10.862107+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:01:14.877110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.861685+0000","last_change":"2024-05-21T21:51:16.929172+0000","last_active":"2024-05-21T21:52:10.861685+0000","last_peered":"2024-05-21T21:52:10.861685+0000","last_clean":"2024-05-21T21:52:10.861685+0000","last_became_active":"2024-05-21T21:51:16.928823+0000","last_became_peered":"2024-05-21T21:51:16.928823+0000","last_unstale":"2024-05-21T21:52:10.861685+0000","last_undegraded":"2024-05-21T21:52:10.861685+0000","last_fullsized":"2024-05-21T21:52:10.861685+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:09:13.719272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.863847+0000","last_change":"2024-05-21T21:51:16.930332+0000","last_active":"2024-05-21T21:52:10.863847+0000","last_peered":"2024-05-21T21:52:10.863847+0000","last_clean":"2024-05-21T21:52:10.863847+0000","last_became_active":"2024-05-21T21:51:16.930024+0000","last_became_peered":"2024-05-21T21:51:16.930024+0000","last_unstale":"2024-05-21T21:52:10.863847+0000","last_undegraded":"2024-05-21T21:52:10.863847+0000","last_fullsized":"2024-05-21T21:52:10.863847+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:47:41.272725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.868573+0000","last_change":"2024-05-21T21:51:16.930102+0000","last_active":"2024-05-21T21:52:10.868573+0000","last_peered":"2024-05-21T21:52:10.868573+0000","last_clean":"2024-05-21T21:52:10.868573+0000","last_became_active":"2024-05-21T21:51:16.929410+0000","last_became_peered":"2024-05-21T21:51:16.929410+0000","last_unstale":"2024-05-21T21:52:10.868573+0000","last_undegraded":"2024-05-21T21:52:10.868573+0000","last_fullsized":"2024-05-21T21:52:10.868573+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:11:46.296288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.873829+0000","last_change":"2024-05-21T21:52:10.873424+0000","last_active":"2024-05-21T21:52:10.873829+0000","last_peered":"2024-05-21T21:52:10.873829+0000","last_clean":"2024-05-21T21:52:10.873829+0000","last_became_active":"2024-05-21T21:52:10.873118+0000","last_became_peered":"2024-05-21T21:52:10.873118+0000","last_unstale":"2024-05-21T21:52:10.873829+0000","last_undegraded":"2024-05-21T21:52:10.873829+0000","last_fullsized":"2024-05-21T21:52:10.873829+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:05:00.731013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,3],"acting":[8,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.866413+0000","last_change":"2024-05-21T21:51:16.934442+0000","last_active":"2024-05-21T21:52:10.866413+0000","last_peered":"2024-05-21T21:52:10.866413+0000","last_clean":"2024-05-21T21:52:10.866413+0000","last_became_active":"2024-05-21T21:51:16.934086+0000","last_became_peered":"2024-05-21T21:51:16.934086+0000","last_unstale":"2024-05-21T21:52:10.866413+0000","last_undegraded":"2024-05-21T21:52:10.866413+0000","last_fullsized":"2024-05-21T21:52:10.866413+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:36:44.302532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.972750+0000","last_change":"2024-05-21T21:51:16.924479+0000","last_active":"2024-05-21T21:52:10.972750+0000","last_peered":"2024-05-21T21:52:10.972750+0000","last_clean":"2024-05-21T21:52:10.972750+0000","last_became_active":"2024-05-21T21:51:16.923998+0000","last_became_peered":"2024-05-21T21:51:16.923998+0000","last_unstale":"2024-05-21T21:52:10.972750+0000","last_undegraded":"2024-05-21T21:52:10.972750+0000","last_fullsized":"2024-05-21T21:52:10.972750+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:23:07.931731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.862486+0000","last_change":"2024-05-21T21:51:16.947271+0000","last_active":"2024-05-21T21:52:10.862486+0000","last_peered":"2024-05-21T21:52:10.862486+0000","last_clean":"2024-05-21T21:52:10.862486+0000","last_became_active":"2024-05-21T21:51:16.946912+0000","last_became_peered":"2024-05-21T21:51:16.946912+0000","last_unstale":"2024-05-21T21:52:10.862486+0000","last_undegraded":"2024-05-21T21:52:10.862486+0000","last_fullsized":"2024-05-21T21:52:10.862486+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:51:29.646915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.861507+0000","last_change":"2024-05-21T21:51:16.928852+0000","last_active":"2024-05-21T21:52:10.861507+0000","last_peered":"2024-05-21T21:52:10.861507+0000","last_clean":"2024-05-21T21:52:10.861507+0000","last_became_active":"2024-05-21T21:51:16.928480+0000","last_became_peered":"2024-05-21T21:51:16.928480+0000","last_unstale":"2024-05-21T21:52:10.861507+0000","last_undegraded":"2024-05-21T21:52:10.861507+0000","last_fullsized":"2024-05-21T21:52:10.861507+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T09:12:30.691254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66966,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":565,"num_read_kb":353,"num_write":245,"num_write_kb":135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":112,"num_keys_recovered":17,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":200898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":210,"ondisk_log_size":210,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645344,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":433959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841572,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28588,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716564,"statfs":{"total":95995035648,"available":95965761536,"internally_reserved":0,"allocated":1601536,"data_stored":1085141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005097,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":929792,"data_stored":434006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201328,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":921600,"data_stored":434267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397556,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28504,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716648,"statfs":{"total":95995035648,"available":95965847552,"internally_reserved":0,"allocated":1519616,"data_stored":1027176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561082,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":909312,"data_stored":433606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574934,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":978944,"data_stored":494393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738460,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36252,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708900,"statfs":{"total":95995035648,"available":95957913600,"internally_reserved":0,"allocated":929792,"data_stored":434945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411410,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36296,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708856,"statfs":{"total":95995035648,"available":95957868544,"internally_reserved":0,"allocated":983040,"data_stored":494780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149771,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36244,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708908,"statfs":{"total":95995035648,"available":95957921792,"internally_reserved":0,"allocated":921600,"data_stored":433314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953542,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":909312,"data_stored":433858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757313,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32676,"kb_used_data":1496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712476,"statfs":{"total":95995035648,"available":95961575424,"internally_reserved":0,"allocated":1531904,"data_stored":1026463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":61626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":62013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":61889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-21T21:52:34.984 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-21T21:52:34.984 INFO:tasks.cephadm:Setting up client nodes... 2024-05-21T21:52:34.985 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-05-21T21:52:35.469 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:52:35.620 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:52:35 smithi151 ceph-mon[31623]: from='client.24667 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T21:52:35.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:52:35 smithi184 ceph-mon[36942]: from='client.24667 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T21:52:35.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:52:35 smithi164 ceph-mon[36269]: from='client.24667 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T21:52:36.652 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:52:36 smithi151 ceph-mon[31623]: pgmap v355: 97 pgs: 97 active+clean; 642 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-21T21:52:36.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:52:36 smithi184 ceph-mon[36942]: pgmap v355: 97 pgs: 97 active+clean; 642 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-21T21:52:36.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:52:36 smithi164 ceph-mon[36269]: pgmap v355: 97 pgs: 97 active+clean; 642 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-21T21:52:37.667 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:37.666+0000 7fa9cf200700 1 -- 172.21.15.151:0/1358249966 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa9c81047d0 msgr2=0x7fa9c8104c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:52:37.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:37.666+0000 7fa9cf200700 1 --2- 172.21.15.151:0/1358249966 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa9c81047d0 0x7fa9c8104c30 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa9b40099c0 tx=0x7fa9b4009cd0 comp rx=0 tx=0).stop 2024-05-21T21:52:37.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:37.666+0000 7fa9cf200700 1 -- 172.21.15.151:0/1358249966 shutdown_connections 2024-05-21T21:52:37.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:37.666+0000 7fa9cf200700 1 --2- 172.21.15.151:0/1358249966 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa9c81035d0 0x7fa9c81039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:37.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:37.666+0000 7fa9cf200700 1 --2- 172.21.15.151:0/1358249966 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa9c8105170 0x7fa9c810ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:37.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:37.666+0000 7fa9cf200700 1 --2- 172.21.15.151:0/1358249966 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa9c81047d0 0x7fa9c8104c30 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:37.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:37.666+0000 7fa9cf200700 1 -- 172.21.15.151:0/1358249966 >> 172.21.15.151:0/1358249966 conn(0x7fa9c80feb70 msgr2=0x7fa9c8100fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:52:37.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:37.667+0000 7fa9cf200700 1 -- 172.21.15.151:0/1358249966 shutdown_connections 2024-05-21T21:52:37.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:37.667+0000 7fa9cf200700 1 -- 172.21.15.151:0/1358249966 wait complete. 2024-05-21T21:52:37.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:37.668+0000 7fa9cf200700 1 Processor -- start 2024-05-21T21:52:37.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:37.668+0000 7fa9cf200700 1 -- start start 2024-05-21T21:52:37.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:37.668+0000 7fa9cf200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa9c81035d0 0x7fa9c819b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:52:37.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:37.668+0000 7fa9cf200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa9c81047d0 0x7fa9c819bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:52:37.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:37.668+0000 7fa9cf200700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa9c8105170 0x7fa9c81a2e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:52:37.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:37.668+0000 7fa9cf200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c810f0b0 con 0x7fa9c81035d0 2024-05-21T21:52:37.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:37.668+0000 7fa9cf200700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c810ef30 con 0x7fa9c81047d0 2024-05-21T21:52:37.708 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:37.668+0000 7fa9cf200700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c810f230 con 0x7fa9c8105170 2024-05-21T21:52:37.708 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:37.668+0000 7fa9c7e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa9c81035d0 0x7fa9c819b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:52:37.708 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:37.668+0000 7fa9c7e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa9c81035d0 0x7fa9c819b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:47910/0 (socket says 172.21.15.151:47910) 2024-05-21T21:52:37.708 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:37.668+0000 7fa9c7e00700 1 -- 172.21.15.151:0/768699808 learned_addr learned my addr 172.21.15.151:0/768699808 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:52:37.708 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:37.668+0000 7fa9cd200700 1 --2- 172.21.15.151:0/768699808 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa9c8105170 0x7fa9c81a2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:52:37.708 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:37.668+0000 7fa9c7400700 1 --2- 172.21.15.151:0/768699808 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa9c81047d0 0x7fa9c819bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:52:37.708 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:37.669+0000 7fa9c7e00700 1 -- 172.21.15.151:0/768699808 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa9c8105170 msgr2=0x7fa9c81a2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:52:37.708 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:37.669+0000 7fa9c7e00700 1 --2- 172.21.15.151:0/768699808 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa9c8105170 0x7fa9c81a2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:37.708 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:37.669+0000 7fa9c7e00700 1 -- 172.21.15.151:0/768699808 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa9c81047d0 msgr2=0x7fa9c819bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:52:37.708 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:37.669+0000 7fa9c7e00700 1 --2- 172.21.15.151:0/768699808 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa9c81047d0 0x7fa9c819bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:37.708 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:37.669+0000 7fa9c7e00700 1 -- 172.21.15.151:0/768699808 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9c81a3500 con 0x7fa9c81035d0 2024-05-21T21:52:37.708 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:37.669+0000 7fa9cd200700 1 --2- 172.21.15.151:0/768699808 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa9c8105170 0x7fa9c81a2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:52:37.708 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:37.669+0000 7fa9c7400700 1 --2- 172.21.15.151:0/768699808 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa9c81047d0 0x7fa9c819bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:52:37.708 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:37.669+0000 7fa9c7e00700 1 --2- 172.21.15.151:0/768699808 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa9c81035d0 0x7fa9c819b8a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa9b800e9a0 tx=0x7fa9b800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:52:37.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:37.670+0000 7fa9c4c00700 1 -- 172.21.15.151:0/768699808 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b8004220 con 0x7fa9c81035d0 2024-05-21T21:52:37.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:37.670+0000 7fa9c4c00700 1 -- 172.21.15.151:0/768699808 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9b80043c0 con 0x7fa9c81035d0 2024-05-21T21:52:37.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:37.670+0000 7fa9c4c00700 1 -- 172.21.15.151:0/768699808 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b8003cc0 con 0x7fa9c81035d0 2024-05-21T21:52:37.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:37.670+0000 7fa9cf200700 1 -- 172.21.15.151:0/768699808 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9c81a37f0 con 0x7fa9c81035d0 2024-05-21T21:52:37.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:37.670+0000 7fa9cf200700 1 -- 172.21.15.151:0/768699808 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9c81a3d80 con 0x7fa9c81035d0 2024-05-21T21:52:37.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:37.671+0000 7fa9cf200700 1 -- 172.21.15.151:0/768699808 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9c8051a80 con 0x7fa9c81035d0 2024-05-21T21:52:37.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:37.674+0000 7fa9c4c00700 1 -- 172.21.15.151:0/768699808 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fa9b800c810 con 0x7fa9c81035d0 2024-05-21T21:52:37.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:37.675+0000 7fa9c4c00700 1 --2- 172.21.15.151:0/768699808 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fa9ac0b2a30 0x7fa9ac0b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:52:37.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:37.675+0000 7fa9c4c00700 1 -- 172.21.15.151:0/768699808 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fa9b80e25e0 con 0x7fa9c81035d0 2024-05-21T21:52:37.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:37.676+0000 7fa9c7400700 1 --2- 172.21.15.151:0/768699808 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fa9ac0b2a30 0x7fa9ac0b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:52:37.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:37.676+0000 7fa9c7400700 1 --2- 172.21.15.151:0/768699808 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fa9ac0b2a30 0x7fa9ac0b4ef0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa9b4003820 tx=0x7fa9b4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:52:37.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:37.678+0000 7fa9c4c00700 1 -- 172.21.15.151:0/768699808 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa9b80aa1f0 con 0x7fa9c81035d0 2024-05-21T21:52:38.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:38.004+0000 7fa9cf200700 1 -- 172.21.15.151:0/768699808 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7fa9c81a4040 con 0x7fa9c81035d0 2024-05-21T21:52:38.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:38.007+0000 7fa9c4c00700 1 -- 172.21.15.151:0/768699808 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7fa9b80a9af0 con 0x7fa9c81035d0 2024-05-21T21:52:38.008 INFO:teuthology.orchestra.run.smithi151.stdout:[client.0] 2024-05-21T21:52:38.008 INFO:teuthology.orchestra.run.smithi151.stdout: key = AQCmF01mrkFTABAAQ/RGt2Tp1yP85aBT21wMGg== 2024-05-21T21:52:38.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:38.010+0000 7fa9cf200700 1 -- 172.21.15.151:0/768699808 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fa9ac0b2a30 msgr2=0x7fa9ac0b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:52:38.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:38.010+0000 7fa9cf200700 1 --2- 172.21.15.151:0/768699808 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fa9ac0b2a30 0x7fa9ac0b4ef0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa9b4003820 tx=0x7fa9b4005fb0 comp rx=0 tx=0).stop 2024-05-21T21:52:38.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:38.010+0000 7fa9cf200700 1 -- 172.21.15.151:0/768699808 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa9c81035d0 msgr2=0x7fa9c819b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:52:38.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:38.010+0000 7fa9cf200700 1 --2- 172.21.15.151:0/768699808 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa9c81035d0 0x7fa9c819b8a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa9b800e9a0 tx=0x7fa9b800ecb0 comp rx=0 tx=0).stop 2024-05-21T21:52:38.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:38.011+0000 7fa9cf200700 1 -- 172.21.15.151:0/768699808 shutdown_connections 2024-05-21T21:52:38.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:38.011+0000 7fa9cf200700 1 --2- 172.21.15.151:0/768699808 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fa9ac0b2a30 0x7fa9ac0b4ef0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:38.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:38.011+0000 7fa9cf200700 1 --2- 172.21.15.151:0/768699808 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa9c8105170 0x7fa9c81a2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:38.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:38.011+0000 7fa9cf200700 1 --2- 172.21.15.151:0/768699808 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa9c81035d0 0x7fa9c819b8a0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:38.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:38.011+0000 7fa9cf200700 1 --2- 172.21.15.151:0/768699808 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa9c81047d0 0x7fa9c819bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:38.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:38.011+0000 7fa9cf200700 1 -- 172.21.15.151:0/768699808 >> 172.21.15.151:0/768699808 conn(0x7fa9c80feb70 msgr2=0x7fa9c8075770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:52:38.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:38.011+0000 7fa9cf200700 1 -- 172.21.15.151:0/768699808 shutdown_connections 2024-05-21T21:52:38.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:38.011+0000 7fa9cf200700 1 -- 172.21.15.151:0/768699808 wait complete. 2024-05-21T21:52:38.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:52:38 smithi151 ceph-mon[31623]: pgmap v356: 97 pgs: 97 active+clean; 642 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-21T21:52:38.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:52:38 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/768699808' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-21T21:52:38.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:52:38 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/768699808' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-21T21:52:38.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:52:38 smithi184 ceph-mon[36942]: pgmap v356: 97 pgs: 97 active+clean; 642 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-21T21:52:38.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:52:38 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/768699808' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-21T21:52:38.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:52:38 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/768699808' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-21T21:52:38.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:52:38 smithi164 ceph-mon[36269]: pgmap v356: 97 pgs: 97 active+clean; 642 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-21T21:52:38.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:52:38 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/768699808' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-21T21:52:38.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:52:38 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/768699808' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-21T21:52:38.879 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-05-21T21:52:38.880 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-05-21T21:52:38.880 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-21T21:52:39.000 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-21T21:52:39.000 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-05-21T21:52:39.000 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph mgr dump --format=json 2024-05-21T21:52:39.481 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:52:40.669 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:52:40 smithi151 ceph-mon[31623]: pgmap v357: 97 pgs: 97 active+clean; 642 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-21T21:52:40.735 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:52:40 smithi184 ceph-mon[36942]: pgmap v357: 97 pgs: 97 active+clean; 642 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-21T21:52:40.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:52:40 smithi164 ceph-mon[36269]: pgmap v357: 97 pgs: 97 active+clean; 642 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-21T21:52:41.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:41.674+0000 7efd91600700 1 -- 172.21.15.151:0/1197964129 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efd8c10b760 msgr2=0x7efd8c10bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:52:41.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:41.674+0000 7efd91600700 1 --2- 172.21.15.151:0/1197964129 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efd8c10b760 0x7efd8c10bb40 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7efd7c0098f0 tx=0x7efd7c009c00 comp rx=0 tx=0).stop 2024-05-21T21:52:41.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:41.674+0000 7efd91600700 1 -- 172.21.15.151:0/1197964129 shutdown_connections 2024-05-21T21:52:41.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:41.674+0000 7efd91600700 1 --2- 172.21.15.151:0/1197964129 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efd8c10b760 0x7efd8c10bb40 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:41.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:41.674+0000 7efd91600700 1 --2- 172.21.15.151:0/1197964129 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd8c106120 0x7efd8c112630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:41.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:41.674+0000 7efd91600700 1 --2- 172.21.15.151:0/1197964129 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efd8c105760 0x7efd8c105be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:41.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:41.674+0000 7efd91600700 1 -- 172.21.15.151:0/1197964129 >> 172.21.15.151:0/1197964129 conn(0x7efd8c101250 msgr2=0x7efd8c103670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:52:41.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:41.674+0000 7efd91600700 1 -- 172.21.15.151:0/1197964129 shutdown_connections 2024-05-21T21:52:41.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:41.674+0000 7efd91600700 1 -- 172.21.15.151:0/1197964129 wait complete. 2024-05-21T21:52:41.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:41.675+0000 7efd91600700 1 Processor -- start 2024-05-21T21:52:41.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:41.675+0000 7efd91600700 1 -- start start 2024-05-21T21:52:41.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:41.676+0000 7efd91600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efd8c105760 0x7efd8c19bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:52:41.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:41.676+0000 7efd91600700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd8c106120 0x7efd8c19c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:52:41.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:41.676+0000 7efd91600700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efd8c10b760 0x7efd8c1a0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:52:41.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:41.676+0000 7efd91600700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7efd8c1159b0 con 0x7efd8c105760 2024-05-21T21:52:41.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:41.676+0000 7efd91600700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7efd8c115830 con 0x7efd8c10b760 2024-05-21T21:52:41.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:41.676+0000 7efd91600700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7efd8c115b30 con 0x7efd8c106120 2024-05-21T21:52:41.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:41.676+0000 7efd8aa00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efd8c105760 0x7efd8c19bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:52:41.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:41.676+0000 7efd8aa00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efd8c105760 0x7efd8c19bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:47922/0 (socket says 172.21.15.151:47922) 2024-05-21T21:52:41.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:41.676+0000 7efd8aa00700 1 -- 172.21.15.151:0/1869791363 learned_addr learned my addr 172.21.15.151:0/1869791363 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:52:41.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:41.676+0000 7efd8b400700 1 --2- 172.21.15.151:0/1869791363 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efd8c10b760 0x7efd8c1a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:52:41.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:41.676+0000 7efd8a000700 1 --2- 172.21.15.151:0/1869791363 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd8c106120 0x7efd8c19c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:52:41.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:41.676+0000 7efd8aa00700 1 -- 172.21.15.151:0/1869791363 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd8c106120 msgr2=0x7efd8c19c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:52:41.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:41.676+0000 7efd8aa00700 1 --2- 172.21.15.151:0/1869791363 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd8c106120 0x7efd8c19c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:41.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:41.676+0000 7efd8aa00700 1 -- 172.21.15.151:0/1869791363 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efd8c10b760 msgr2=0x7efd8c1a0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:52:41.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:41.676+0000 7efd8aa00700 1 --2- 172.21.15.151:0/1869791363 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efd8c10b760 0x7efd8c1a0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:41.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:41.676+0000 7efd8aa00700 1 -- 172.21.15.151:0/1869791363 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd8c1a0eb0 con 0x7efd8c105760 2024-05-21T21:52:41.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:41.676+0000 7efd8b400700 1 --2- 172.21.15.151:0/1869791363 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efd8c10b760 0x7efd8c1a0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:52:41.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:41.677+0000 7efd8aa00700 1 --2- 172.21.15.151:0/1869791363 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efd8c105760 0x7efd8c19bea0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7efd7c009230 tx=0x7efd7c00fa70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:52:41.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:41.677+0000 7efd83400700 1 -- 172.21.15.151:0/1869791363 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd7c005140 con 0x7efd8c105760 2024-05-21T21:52:41.679 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:41.677+0000 7efd83400700 1 -- 172.21.15.151:0/1869791363 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd7c00b560 con 0x7efd8c105760 2024-05-21T21:52:41.679 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:41.677+0000 7efd83400700 1 -- 172.21.15.151:0/1869791363 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd7c003970 con 0x7efd8c105760 2024-05-21T21:52:41.679 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:41.677+0000 7efd91600700 1 -- 172.21.15.151:0/1869791363 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd8c1a1140 con 0x7efd8c105760 2024-05-21T21:52:41.679 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:41.677+0000 7efd91600700 1 -- 172.21.15.151:0/1869791363 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd8c1a8a20 con 0x7efd8c105760 2024-05-21T21:52:41.679 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:41.678+0000 7efd91600700 1 -- 172.21.15.151:0/1869791363 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd8c10dcf0 con 0x7efd8c105760 2024-05-21T21:52:41.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:41.681+0000 7efd83400700 1 -- 172.21.15.151:0/1869791363 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7efd7c00ba50 con 0x7efd8c105760 2024-05-21T21:52:41.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:41.683+0000 7efd83400700 1 --2- 172.21.15.151:0/1869791363 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7efd700b2990 0x7efd700b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:52:41.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:41.684+0000 7efd83400700 1 -- 172.21.15.151:0/1869791363 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7efd7c0e3290 con 0x7efd8c105760 2024-05-21T21:52:41.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:41.684+0000 7efd8a000700 1 --2- 172.21.15.151:0/1869791363 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7efd700b2990 0x7efd700b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:52:41.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:41.684+0000 7efd8a000700 1 --2- 172.21.15.151:0/1869791363 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7efd700b2990 0x7efd700b4e50 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7efd8c103950 tx=0x7efd78009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:52:41.688 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:41.688+0000 7efd83400700 1 -- 172.21.15.151:0/1869791363 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7efd7c013030 con 0x7efd8c105760 2024-05-21T21:52:41.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:41.966+0000 7efd91600700 1 -- 172.21.15.151:0/1869791363 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7efd8c19d200 con 0x7efd8c105760 2024-05-21T21:52:41.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:41.977+0000 7efd83400700 1 -- 172.21.15.151:0/1869791363 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+288654 (secure 0 0 0) 0x7efd7c0aaea0 con 0x7efd8c105760 2024-05-21T21:52:41.980 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T21:52:41.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:41.986+0000 7efd91600700 1 -- 172.21.15.151:0/1869791363 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7efd700b2990 msgr2=0x7efd700b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:52:41.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:41.986+0000 7efd91600700 1 --2- 172.21.15.151:0/1869791363 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7efd700b2990 0x7efd700b4e50 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7efd8c103950 tx=0x7efd78009310 comp rx=0 tx=0).stop 2024-05-21T21:52:41.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:41.986+0000 7efd91600700 1 -- 172.21.15.151:0/1869791363 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efd8c105760 msgr2=0x7efd8c19bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:52:41.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:41.986+0000 7efd91600700 1 --2- 172.21.15.151:0/1869791363 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efd8c105760 0x7efd8c19bea0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7efd7c009230 tx=0x7efd7c00fa70 comp rx=0 tx=0).stop 2024-05-21T21:52:41.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:41.986+0000 7efd91600700 1 -- 172.21.15.151:0/1869791363 shutdown_connections 2024-05-21T21:52:41.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:41.986+0000 7efd91600700 1 --2- 172.21.15.151:0/1869791363 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7efd700b2990 0x7efd700b4e50 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:41.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:41.986+0000 7efd91600700 1 --2- 172.21.15.151:0/1869791363 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efd8c10b760 0x7efd8c1a0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:41.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:41.986+0000 7efd91600700 1 --2- 172.21.15.151:0/1869791363 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efd8c105760 0x7efd8c19bea0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:41.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:41.986+0000 7efd91600700 1 --2- 172.21.15.151:0/1869791363 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd8c106120 0x7efd8c19c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:41.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:41.986+0000 7efd91600700 1 -- 172.21.15.151:0/1869791363 >> 172.21.15.151:0/1869791363 conn(0x7efd8c101250 msgr2=0x7efd8c101ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:52:41.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:41.986+0000 7efd91600700 1 -- 172.21.15.151:0/1869791363 shutdown_connections 2024-05-21T21:52:41.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:41.986+0000 7efd91600700 1 -- 172.21.15.151:0/1869791363 wait complete. 2024-05-21T21:52:42.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:52:42 smithi151 ceph-mon[31623]: pgmap v358: 97 pgs: 97 active+clean; 642 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-21T21:52:42.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:52:42 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/1869791363' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-21T21:52:42.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:52:42 smithi184 ceph-mon[36942]: pgmap v358: 97 pgs: 97 active+clean; 642 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-21T21:52:42.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:52:42 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/1869791363' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-21T21:52:42.785 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":2796622483},{"type":"v1","addr":"172.21.15.151:6801","nonce":2796622483}]},"active_addr":"172.21.15.151:6801/2796622483","active_change":"2024-05-21T21:43:09.041814+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24116,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.151:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.151:0","nonce":3232339048}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.151:0","nonce":1946920088}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.151:0","nonce":2415279581}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.151:0","nonce":3481112063}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.151:0","nonce":40669060}]}]} 2024-05-21T21:52:42.791 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-05-21T21:52:42.791 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-05-21T21:52:42.792 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph osd dump --format=json 2024-05-21T21:52:42.792 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:52:42 smithi164 ceph-mon[36269]: pgmap v358: 97 pgs: 97 active+clean; 642 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-21T21:52:42.792 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:52:42 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/1869791363' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-21T21:52:43.266 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:52:44.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:52:44 smithi151 ceph-mon[31623]: pgmap v359: 97 pgs: 97 active+clean; 642 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-21T21:52:44.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:52:44 smithi184 ceph-mon[36942]: pgmap v359: 97 pgs: 97 active+clean; 642 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-21T21:52:44.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:52:44 smithi164 ceph-mon[36269]: pgmap v359: 97 pgs: 97 active+clean; 642 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-21T21:52:45.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:45.578+0000 7fc8ebe00700 1 -- 172.21.15.151:0/2962263031 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc8ec109540 msgr2=0x7fc8ec109920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:52:45.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:45.578+0000 7fc8ebe00700 1 --2- 172.21.15.151:0/2962263031 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc8ec109540 0x7fc8ec109920 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc0099c0 tx=0x7fc8dc009cd0 comp rx=0 tx=0).stop 2024-05-21T21:52:45.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:45.578+0000 7fc8ebe00700 1 -- 172.21.15.151:0/2962263031 shutdown_connections 2024-05-21T21:52:45.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:45.578+0000 7fc8ebe00700 1 --2- 172.21.15.151:0/2962263031 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8ec103e50 0x7fc8ec110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:45.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:45.578+0000 7fc8ebe00700 1 --2- 172.21.15.151:0/2962263031 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc8ec109540 0x7fc8ec109920 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:45.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:45.578+0000 7fc8ebe00700 1 --2- 172.21.15.151:0/2962263031 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc8ec103490 0x7fc8ec103910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:45.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:45.578+0000 7fc8ebe00700 1 -- 172.21.15.151:0/2962263031 >> 172.21.15.151:0/2962263031 conn(0x7fc8ec0fefc0 msgr2=0x7fc8ec1013e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:52:45.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:45.578+0000 7fc8ebe00700 1 -- 172.21.15.151:0/2962263031 shutdown_connections 2024-05-21T21:52:45.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:45.578+0000 7fc8ebe00700 1 -- 172.21.15.151:0/2962263031 wait complete. 2024-05-21T21:52:45.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:45.579+0000 7fc8ebe00700 1 Processor -- start 2024-05-21T21:52:45.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:45.580+0000 7fc8ebe00700 1 -- start start 2024-05-21T21:52:45.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:45.580+0000 7fc8ebe00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc8ec103490 0x7fc8ec075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:52:45.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:45.580+0000 7fc8ebe00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8ec103e50 0x7fc8ec075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:52:45.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:45.580+0000 7fc8ebe00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc8ec109540 0x7fc8ec0762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:52:45.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:45.580+0000 7fc8ebe00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc8ec113670 con 0x7fc8ec103490 2024-05-21T21:52:45.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:45.580+0000 7fc8ebe00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc8ec1134f0 con 0x7fc8ec109540 2024-05-21T21:52:45.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:45.580+0000 7fc8ebe00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc8ec1137f0 con 0x7fc8ec103e50 2024-05-21T21:52:45.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:45.580+0000 7fc8eaa00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc8ec103490 0x7fc8ec075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:52:45.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:45.580+0000 7fc8eaa00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc8ec103490 0x7fc8ec075840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:47926/0 (socket says 172.21.15.151:47926) 2024-05-21T21:52:45.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:45.580+0000 7fc8eaa00700 1 -- 172.21.15.151:0/4123916194 learned_addr learned my addr 172.21.15.151:0/4123916194 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:52:45.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:45.580+0000 7fc8e3e00700 1 --2- 172.21.15.151:0/4123916194 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8ec103e50 0x7fc8ec075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:52:45.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:45.580+0000 7fc8eb400700 1 --2- 172.21.15.151:0/4123916194 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc8ec109540 0x7fc8ec0762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:52:45.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:45.581+0000 7fc8e3e00700 1 -- 172.21.15.151:0/4123916194 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc8ec109540 msgr2=0x7fc8ec0762c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:52:45.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:45.581+0000 7fc8e3e00700 1 --2- 172.21.15.151:0/4123916194 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc8ec109540 0x7fc8ec0762c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:45.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:45.581+0000 7fc8e3e00700 1 -- 172.21.15.151:0/4123916194 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc8ec103490 msgr2=0x7fc8ec075840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:52:45.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:45.581+0000 7fc8e3e00700 1 --2- 172.21.15.151:0/4123916194 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc8ec103490 0x7fc8ec075840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:45.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:45.581+0000 7fc8e3e00700 1 -- 172.21.15.151:0/4123916194 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8ec1a7fd0 con 0x7fc8ec103e50 2024-05-21T21:52:45.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:45.581+0000 7fc8eaa00700 1 --2- 172.21.15.151:0/4123916194 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc8ec103490 0x7fc8ec075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:52:45.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:45.581+0000 7fc8e3e00700 1 --2- 172.21.15.151:0/4123916194 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8ec103e50 0x7fc8ec075d80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc8d400e9f0 tx=0x7fc8d400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:52:45.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:45.581+0000 7fc8e3400700 1 -- 172.21.15.151:0/4123916194 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d400c9c0 con 0x7fc8ec103e50 2024-05-21T21:52:45.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:45.581+0000 7fc8e3400700 1 -- 172.21.15.151:0/4123916194 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8d400cb60 con 0x7fc8ec103e50 2024-05-21T21:52:45.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:45.581+0000 7fc8e3400700 1 -- 172.21.15.151:0/4123916194 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d4018680 con 0x7fc8ec103e50 2024-05-21T21:52:45.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:45.582+0000 7fc8ebe00700 1 -- 172.21.15.151:0/4123916194 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8ec1a82c0 con 0x7fc8ec103e50 2024-05-21T21:52:45.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:45.582+0000 7fc8ebe00700 1 -- 172.21.15.151:0/4123916194 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8ec1a8800 con 0x7fc8ec103e50 2024-05-21T21:52:45.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:45.583+0000 7fc8ebe00700 1 -- 172.21.15.151:0/4123916194 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8ec10b9d0 con 0x7fc8ec103e50 2024-05-21T21:52:45.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:45.585+0000 7fc8e3400700 1 -- 172.21.15.151:0/4123916194 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fc8d4010430 con 0x7fc8ec103e50 2024-05-21T21:52:45.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:45.586+0000 7fc8e3400700 1 --2- 172.21.15.151:0/4123916194 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fc8d00b2910 0x7fc8d00b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:52:45.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:45.587+0000 7fc8e3400700 1 -- 172.21.15.151:0/4123916194 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fc8d40e28b0 con 0x7fc8ec103e50 2024-05-21T21:52:45.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:45.587+0000 7fc8eaa00700 1 --2- 172.21.15.151:0/4123916194 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fc8d00b2910 0x7fc8d00b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:52:45.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:45.587+0000 7fc8eaa00700 1 --2- 172.21.15.151:0/4123916194 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fc8d00b2910 0x7fc8d00b4dd0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc000c00 tx=0x7fc8dc005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:52:45.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:45.589+0000 7fc8e3400700 1 -- 172.21.15.151:0/4123916194 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc8d40aa430 con 0x7fc8ec103e50 2024-05-21T21:52:45.826 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:45.825+0000 7fc8ebe00700 1 -- 172.21.15.151:0/4123916194 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc8ec10b3b0 con 0x7fc8ec103e50 2024-05-21T21:52:45.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:45.827+0000 7fc8e3400700 1 -- 172.21.15.151:0/4123916194 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22744 (secure 0 0 0) 0x7fc8d40a9d30 con 0x7fc8ec103e50 2024-05-21T21:52:45.829 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T21:52:45.829 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":82,"fsid":"c8c7a2e6-17ba-11ef-bc9a-c7b262605968","created":"2024-05-21T21:41:52.412833+0000","modified":"2024-05-21T21:52:10.856587+0000","last_up_change":"2024-05-21T21:50:05.401314+0000","last_in_change":"2024-05-21T21:49:45.684136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-21T21:45:55.196046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-21T21:51:11.447428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-21T21:51:15.618605+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"18db8e54-0cfc-4e12-9c84-6f72a99cc989","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":2420400473},{"type":"v1","addr":"172.21.15.151:6803","nonce":2420400473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":2420400473},{"type":"v1","addr":"172.21.15.151:6805","nonce":2420400473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":2420400473},{"type":"v1","addr":"172.21.15.151:6809","nonce":2420400473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":2420400473},{"type":"v1","addr":"172.21.15.151:6807","nonce":2420400473}]},"public_addr":"172.21.15.151:6803/2420400473","cluster_addr":"172.21.15.151:6805/2420400473","heartbeat_back_addr":"172.21.15.151:6809/2420400473","heartbeat_front_addr":"172.21.15.151:6807/2420400473","state":["exists","up"]},{"osd":1,"uuid":"bac384c7-1102-4815-81a8-10cc7661d230","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":3900240447},{"type":"v1","addr":"172.21.15.164:6801","nonce":3900240447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":3900240447},{"type":"v1","addr":"172.21.15.164:6803","nonce":3900240447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":3900240447},{"type":"v1","addr":"172.21.15.164:6807","nonce":3900240447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":3900240447},{"type":"v1","addr":"172.21.15.164:6805","nonce":3900240447}]},"public_addr":"172.21.15.164:6801/3900240447","cluster_addr":"172.21.15.164:6803/3900240447","heartbeat_back_addr":"172.21.15.164:6807/3900240447","heartbeat_front_addr":"172.21.15.164:6805/3900240447","state":["exists","up"]},{"osd":2,"uuid":"b7e4a782-1518-4d43-bee9-7fd768ee5cb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":3971150737},{"type":"v1","addr":"172.21.15.184:6801","nonce":3971150737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3971150737},{"type":"v1","addr":"172.21.15.184:6803","nonce":3971150737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3971150737},{"type":"v1","addr":"172.21.15.184:6807","nonce":3971150737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3971150737},{"type":"v1","addr":"172.21.15.184:6805","nonce":3971150737}]},"public_addr":"172.21.15.184:6801/3971150737","cluster_addr":"172.21.15.184:6803/3971150737","heartbeat_back_addr":"172.21.15.184:6807/3971150737","heartbeat_front_addr":"172.21.15.184:6805/3971150737","state":["exists","up"]},{"osd":3,"uuid":"ab3df826-39e3-4635-a49c-2b5ffdbf12f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":556594400},{"type":"v1","addr":"172.21.15.151:6811","nonce":556594400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":556594400},{"type":"v1","addr":"172.21.15.151:6813","nonce":556594400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":556594400},{"type":"v1","addr":"172.21.15.151:6817","nonce":556594400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":556594400},{"type":"v1","addr":"172.21.15.151:6815","nonce":556594400}]},"public_addr":"172.21.15.151:6811/556594400","cluster_addr":"172.21.15.151:6813/556594400","heartbeat_back_addr":"172.21.15.151:6817/556594400","heartbeat_front_addr":"172.21.15.151:6815/556594400","state":["exists","up"]},{"osd":4,"uuid":"00cbf6ae-0f42-4685-a77d-d803c6467aa0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":988881973},{"type":"v1","addr":"172.21.15.164:6809","nonce":988881973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":988881973},{"type":"v1","addr":"172.21.15.164:6811","nonce":988881973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":988881973},{"type":"v1","addr":"172.21.15.164:6815","nonce":988881973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":988881973},{"type":"v1","addr":"172.21.15.164:6813","nonce":988881973}]},"public_addr":"172.21.15.164:6809/988881973","cluster_addr":"172.21.15.164:6811/988881973","heartbeat_back_addr":"172.21.15.164:6815/988881973","heartbeat_front_addr":"172.21.15.164:6813/988881973","state":["exists","up"]},{"osd":5,"uuid":"fad14221-0ef0-4d40-b414-c6c9f1f9718c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3293311106},{"type":"v1","addr":"172.21.15.184:6809","nonce":3293311106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":3293311106},{"type":"v1","addr":"172.21.15.184:6811","nonce":3293311106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":3293311106},{"type":"v1","addr":"172.21.15.184:6815","nonce":3293311106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":3293311106},{"type":"v1","addr":"172.21.15.184:6813","nonce":3293311106}]},"public_addr":"172.21.15.184:6809/3293311106","cluster_addr":"172.21.15.184:6811/3293311106","heartbeat_back_addr":"172.21.15.184:6815/3293311106","heartbeat_front_addr":"172.21.15.184:6813/3293311106","state":["exists","up"]},{"osd":6,"uuid":"ab8a60f6-0dec-4c1c-89db-f2e2f638f1ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":3377550762},{"type":"v1","addr":"172.21.15.151:6819","nonce":3377550762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":3377550762},{"type":"v1","addr":"172.21.15.151:6821","nonce":3377550762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":3377550762},{"type":"v1","addr":"172.21.15.151:6825","nonce":3377550762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":3377550762},{"type":"v1","addr":"172.21.15.151:6823","nonce":3377550762}]},"public_addr":"172.21.15.151:6819/3377550762","cluster_addr":"172.21.15.151:6821/3377550762","heartbeat_back_addr":"172.21.15.151:6825/3377550762","heartbeat_front_addr":"172.21.15.151:6823/3377550762","state":["exists","up"]},{"osd":7,"uuid":"db332268-9972-4ed7-a307-822b83a0fbb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":3869696936},{"type":"v1","addr":"172.21.15.164:6817","nonce":3869696936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":3869696936},{"type":"v1","addr":"172.21.15.164:6819","nonce":3869696936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":3869696936},{"type":"v1","addr":"172.21.15.164:6823","nonce":3869696936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":3869696936},{"type":"v1","addr":"172.21.15.164:6821","nonce":3869696936}]},"public_addr":"172.21.15.164:6817/3869696936","cluster_addr":"172.21.15.164:6819/3869696936","heartbeat_back_addr":"172.21.15.164:6823/3869696936","heartbeat_front_addr":"172.21.15.164:6821/3869696936","state":["exists","up"]},{"osd":8,"uuid":"fe2fbe1c-549a-4947-a188-426dfca04d10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":575273541},{"type":"v1","addr":"172.21.15.184:6817","nonce":575273541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":575273541},{"type":"v1","addr":"172.21.15.184:6819","nonce":575273541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":575273541},{"type":"v1","addr":"172.21.15.184:6823","nonce":575273541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":575273541},{"type":"v1","addr":"172.21.15.184:6821","nonce":575273541}]},"public_addr":"172.21.15.184:6817/575273541","cluster_addr":"172.21.15.184:6819/575273541","heartbeat_back_addr":"172.21.15.184:6823/575273541","heartbeat_front_addr":"172.21.15.184:6821/575273541","state":["exists","up"]},{"osd":9,"uuid":"3a7b450f-8004-4adf-bae3-970918f98f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":337128533},{"type":"v1","addr":"172.21.15.151:6827","nonce":337128533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":337128533},{"type":"v1","addr":"172.21.15.151:6829","nonce":337128533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":337128533},{"type":"v1","addr":"172.21.15.151:6833","nonce":337128533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":337128533},{"type":"v1","addr":"172.21.15.151:6831","nonce":337128533}]},"public_addr":"172.21.15.151:6827/337128533","cluster_addr":"172.21.15.151:6829/337128533","heartbeat_back_addr":"172.21.15.151:6833/337128533","heartbeat_front_addr":"172.21.15.151:6831/337128533","state":["exists","up"]},{"osd":10,"uuid":"8fcdf271-90c4-4db3-be69-b0ca716dfd80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":1823448782},{"type":"v1","addr":"172.21.15.164:6825","nonce":1823448782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":1823448782},{"type":"v1","addr":"172.21.15.164:6827","nonce":1823448782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":1823448782},{"type":"v1","addr":"172.21.15.164:6831","nonce":1823448782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":1823448782},{"type":"v1","addr":"172.21.15.164:6829","nonce":1823448782}]},"public_addr":"172.21.15.164:6825/1823448782","cluster_addr":"172.21.15.164:6827/1823448782","heartbeat_back_addr":"172.21.15.164:6831/1823448782","heartbeat_front_addr":"172.21.15.164:6829/1823448782","state":["exists","up"]},{"osd":11,"uuid":"e661477b-0c09-409f-a2fe-6ebd34642a8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":1563340128},{"type":"v1","addr":"172.21.15.184:6825","nonce":1563340128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":1563340128},{"type":"v1","addr":"172.21.15.184:6827","nonce":1563340128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":1563340128},{"type":"v1","addr":"172.21.15.184:6831","nonce":1563340128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":1563340128},{"type":"v1","addr":"172.21.15.184:6829","nonce":1563340128}]},"public_addr":"172.21.15.184:6825/1563340128","cluster_addr":"172.21.15.184:6827/1563340128","heartbeat_back_addr":"172.21.15.184:6831/1563340128","heartbeat_front_addr":"172.21.15.184:6829/1563340128","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:45:02.821160+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:45:27.844564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:45:51.904981+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:46:24.376245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:46:50.036621+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:47:14.748422+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:47:47.396383+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:48:13.543174+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:48:38.213285+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:49:11.980042+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:49:38.519761+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:50:03.439898+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.151:0/31963128":"2024-05-22T21:43:09.041455+0000","172.21.15.151:6800/3428827300":"2024-05-22T21:43:09.041455+0000","172.21.15.151:6801/3428827300":"2024-05-22T21:43:09.041455+0000","172.21.15.151:0/131989141":"2024-05-22T21:43:09.041455+0000","172.21.15.151:6801/1128885292":"2024-05-22T21:42:35.440147+0000","172.21.15.151:0/2703664519":"2024-05-22T21:43:09.041455+0000","172.21.15.151:6800/1128885292":"2024-05-22T21:42:35.440147+0000","172.21.15.151:0/3052943699":"2024-05-22T21:42:35.440147+0000","172.21.15.151:0/3929452511":"2024-05-22T21:42:35.440147+0000","172.21.15.151:0/3762889700":"2024-05-22T21:42:35.440147+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-21T21:52:45.833 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:45.832+0000 7fc8ebe00700 1 -- 172.21.15.151:0/4123916194 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fc8d00b2910 msgr2=0x7fc8d00b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:52:45.833 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:45.832+0000 7fc8ebe00700 1 --2- 172.21.15.151:0/4123916194 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fc8d00b2910 0x7fc8d00b4dd0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc000c00 tx=0x7fc8dc005fd0 comp rx=0 tx=0).stop 2024-05-21T21:52:45.833 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:45.832+0000 7fc8ebe00700 1 -- 172.21.15.151:0/4123916194 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8ec103e50 msgr2=0x7fc8ec075d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:52:45.833 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:45.832+0000 7fc8ebe00700 1 --2- 172.21.15.151:0/4123916194 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8ec103e50 0x7fc8ec075d80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc8d400e9f0 tx=0x7fc8d400ed00 comp rx=0 tx=0).stop 2024-05-21T21:52:45.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:45.832+0000 7fc8ebe00700 1 -- 172.21.15.151:0/4123916194 shutdown_connections 2024-05-21T21:52:45.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:45.832+0000 7fc8ebe00700 1 --2- 172.21.15.151:0/4123916194 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fc8d00b2910 0x7fc8d00b4dd0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:45.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:45.832+0000 7fc8ebe00700 1 --2- 172.21.15.151:0/4123916194 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc8ec109540 0x7fc8ec0762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:45.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:45.832+0000 7fc8ebe00700 1 --2- 172.21.15.151:0/4123916194 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc8ec103490 0x7fc8ec075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:45.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:45.832+0000 7fc8ebe00700 1 --2- 172.21.15.151:0/4123916194 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8ec103e50 0x7fc8ec075d80 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:45.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:45.832+0000 7fc8ebe00700 1 -- 172.21.15.151:0/4123916194 >> 172.21.15.151:0/4123916194 conn(0x7fc8ec0fefc0 msgr2=0x7fc8ec1006c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:52:45.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:45.833+0000 7fc8ebe00700 1 -- 172.21.15.151:0/4123916194 shutdown_connections 2024-05-21T21:52:45.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:45.833+0000 7fc8ebe00700 1 -- 172.21.15.151:0/4123916194 wait complete. 2024-05-21T21:52:46.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:52:46 smithi151 ceph-mon[31623]: pgmap v360: 97 pgs: 97 active+clean; 642 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-21T21:52:46.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:52:46 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/4123916194' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T21:52:46.670 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-05-21T21:52:46.671 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph osd dump --format=json 2024-05-21T21:52:46.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:52:46 smithi184 ceph-mon[36942]: pgmap v360: 97 pgs: 97 active+clean; 642 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-21T21:52:46.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:52:46 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/4123916194' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T21:52:46.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:52:46 smithi164 ceph-mon[36269]: pgmap v360: 97 pgs: 97 active+clean; 642 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-21T21:52:46.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:52:46 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/4123916194' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T21:52:47.122 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:52:48.702 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:52:48 smithi151 ceph-mon[31623]: pgmap v361: 97 pgs: 97 active+clean; 642 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-05-21T21:52:48.735 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:52:48 smithi184 ceph-mon[36942]: pgmap v361: 97 pgs: 97 active+clean; 642 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-05-21T21:52:48.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:52:48 smithi164 ceph-mon[36269]: pgmap v361: 97 pgs: 97 active+clean; 642 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-05-21T21:52:49.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:49.251+0000 7fa2d9c00700 1 -- 172.21.15.151:0/3991657612 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa2d410b670 msgr2=0x7fa2d410ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:52:49.253 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:49.251+0000 7fa2d9c00700 1 --2- 172.21.15.151:0/3991657612 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa2d410b670 0x7fa2d410ba50 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa2c4009990 tx=0x7fa2c4009ca0 comp rx=0 tx=0).stop 2024-05-21T21:52:49.253 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:49.251+0000 7fa2d9c00700 1 -- 172.21.15.151:0/3991657612 shutdown_connections 2024-05-21T21:52:49.253 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:49.251+0000 7fa2d9c00700 1 --2- 172.21.15.151:0/3991657612 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa2d4106030 0x7fa2d4112540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:49.253 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:49.251+0000 7fa2d9c00700 1 --2- 172.21.15.151:0/3991657612 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa2d410b670 0x7fa2d410ba50 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:49.253 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:49.251+0000 7fa2d9c00700 1 --2- 172.21.15.151:0/3991657612 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2d4105670 0x7fa2d4105af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:49.253 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:49.251+0000 7fa2d9c00700 1 -- 172.21.15.151:0/3991657612 >> 172.21.15.151:0/3991657612 conn(0x7fa2d4101160 msgr2=0x7fa2d4103580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:52:49.253 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:49.251+0000 7fa2d9c00700 1 -- 172.21.15.151:0/3991657612 shutdown_connections 2024-05-21T21:52:49.253 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:49.251+0000 7fa2d9c00700 1 -- 172.21.15.151:0/3991657612 wait complete. 2024-05-21T21:52:49.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:49.252+0000 7fa2d9c00700 1 Processor -- start 2024-05-21T21:52:49.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:49.252+0000 7fa2d9c00700 1 -- start start 2024-05-21T21:52:49.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:49.253+0000 7fa2d9c00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2d4105670 0x7fa2d419bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:52:49.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:49.253+0000 7fa2d9c00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa2d4106030 0x7fa2d419c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:52:49.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:49.253+0000 7fa2d9c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa2d410b670 0x7fa2d41a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:52:49.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:49.253+0000 7fa2d9c00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d41158c0 con 0x7fa2d410b670 2024-05-21T21:52:49.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:49.253+0000 7fa2d9c00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d4115740 con 0x7fa2d4106030 2024-05-21T21:52:49.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:49.253+0000 7fa2d9c00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d4115a40 con 0x7fa2d4105670 2024-05-21T21:52:49.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:49.253+0000 7fa2d3e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2d4105670 0x7fa2d419bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:52:49.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:49.253+0000 7fa2d3400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa2d4106030 0x7fa2d419c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:52:49.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:49.253+0000 7fa2d3400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa2d4106030 0x7fa2d419c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:54744/0 (socket says 172.21.15.151:54744) 2024-05-21T21:52:49.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:49.253+0000 7fa2d3e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2d4105670 0x7fa2d419bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:41052/0 (socket says 172.21.15.151:41052) 2024-05-21T21:52:49.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:49.253+0000 7fa2d3400700 1 -- 172.21.15.151:0/2088445316 learned_addr learned my addr 172.21.15.151:0/2088445316 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:52:49.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:49.253+0000 7fa2d9200700 1 --2- 172.21.15.151:0/2088445316 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa2d410b670 0x7fa2d41a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:52:49.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:49.254+0000 7fa2d3400700 1 -- 172.21.15.151:0/2088445316 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2d4105670 msgr2=0x7fa2d419bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:52:49.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:49.254+0000 7fa2d3400700 1 --2- 172.21.15.151:0/2088445316 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2d4105670 0x7fa2d419bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:49.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:49.254+0000 7fa2d3400700 1 -- 172.21.15.151:0/2088445316 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa2d410b670 msgr2=0x7fa2d41a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:52:49.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:49.254+0000 7fa2d3400700 1 --2- 172.21.15.151:0/2088445316 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa2d410b670 0x7fa2d41a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:49.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:49.254+0000 7fa2d3400700 1 -- 172.21.15.151:0/2088445316 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2d41a0d90 con 0x7fa2d4106030 2024-05-21T21:52:49.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:49.254+0000 7fa2d3400700 1 --2- 172.21.15.151:0/2088445316 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa2d4106030 0x7fa2d419c310 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa2c000e9c0 tx=0x7fa2c000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:52:49.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:49.254+0000 7fa2d0c00700 1 -- 172.21.15.151:0/2088445316 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c000cbd0 con 0x7fa2d4106030 2024-05-21T21:52:49.257 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:49.254+0000 7fa2d9c00700 1 -- 172.21.15.151:0/2088445316 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2d41a1080 con 0x7fa2d4106030 2024-05-21T21:52:49.257 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:49.254+0000 7fa2d0c00700 1 -- 172.21.15.151:0/2088445316 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2c000ee60 con 0x7fa2d4106030 2024-05-21T21:52:49.257 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:49.254+0000 7fa2d0c00700 1 -- 172.21.15.151:0/2088445316 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c00055c0 con 0x7fa2d4106030 2024-05-21T21:52:49.257 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:49.254+0000 7fa2d9c00700 1 -- 172.21.15.151:0/2088445316 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2d41a8900 con 0x7fa2d4106030 2024-05-21T21:52:49.257 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:49.256+0000 7fa2d9c00700 1 -- 172.21.15.151:0/2088445316 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2d406a230 con 0x7fa2d4106030 2024-05-21T21:52:49.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:49.259+0000 7fa2d0c00700 1 -- 172.21.15.151:0/2088445316 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fa2c0047990 con 0x7fa2d4106030 2024-05-21T21:52:49.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:49.260+0000 7fa2d0c00700 1 --2- 172.21.15.151:0/2088445316 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fa2b80b2910 0x7fa2b80b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:52:49.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:49.260+0000 7fa2d0c00700 1 -- 172.21.15.151:0/2088445316 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fa2c00e26c0 con 0x7fa2d4106030 2024-05-21T21:52:49.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:49.260+0000 7fa2d3e00700 1 --2- 172.21.15.151:0/2088445316 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fa2b80b2910 0x7fa2b80b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:52:49.262 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:49.261+0000 7fa2d3e00700 1 --2- 172.21.15.151:0/2088445316 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fa2b80b2910 0x7fa2b80b4dd0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa2c4006010 tx=0x7fa2c400b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:52:49.262 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:49.262+0000 7fa2d0c00700 1 -- 172.21.15.151:0/2088445316 <== mon.2 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa2c00aa240 con 0x7fa2d4106030 2024-05-21T21:52:49.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:49.503+0000 7fa2d9c00700 1 -- 172.21.15.151:0/2088445316 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa2d419d0e0 con 0x7fa2d4106030 2024-05-21T21:52:49.507 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:49.506+0000 7fa2d0c00700 1 -- 172.21.15.151:0/2088445316 <== mon.2 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22744 (secure 0 0 0) 0x7fa2c00a9b40 con 0x7fa2d4106030 2024-05-21T21:52:49.507 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T21:52:49.507 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":82,"fsid":"c8c7a2e6-17ba-11ef-bc9a-c7b262605968","created":"2024-05-21T21:41:52.412833+0000","modified":"2024-05-21T21:52:10.856587+0000","last_up_change":"2024-05-21T21:50:05.401314+0000","last_in_change":"2024-05-21T21:49:45.684136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-21T21:45:55.196046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-21T21:51:11.447428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-21T21:51:15.618605+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"18db8e54-0cfc-4e12-9c84-6f72a99cc989","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":2420400473},{"type":"v1","addr":"172.21.15.151:6803","nonce":2420400473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":2420400473},{"type":"v1","addr":"172.21.15.151:6805","nonce":2420400473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":2420400473},{"type":"v1","addr":"172.21.15.151:6809","nonce":2420400473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":2420400473},{"type":"v1","addr":"172.21.15.151:6807","nonce":2420400473}]},"public_addr":"172.21.15.151:6803/2420400473","cluster_addr":"172.21.15.151:6805/2420400473","heartbeat_back_addr":"172.21.15.151:6809/2420400473","heartbeat_front_addr":"172.21.15.151:6807/2420400473","state":["exists","up"]},{"osd":1,"uuid":"bac384c7-1102-4815-81a8-10cc7661d230","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":3900240447},{"type":"v1","addr":"172.21.15.164:6801","nonce":3900240447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":3900240447},{"type":"v1","addr":"172.21.15.164:6803","nonce":3900240447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":3900240447},{"type":"v1","addr":"172.21.15.164:6807","nonce":3900240447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":3900240447},{"type":"v1","addr":"172.21.15.164:6805","nonce":3900240447}]},"public_addr":"172.21.15.164:6801/3900240447","cluster_addr":"172.21.15.164:6803/3900240447","heartbeat_back_addr":"172.21.15.164:6807/3900240447","heartbeat_front_addr":"172.21.15.164:6805/3900240447","state":["exists","up"]},{"osd":2,"uuid":"b7e4a782-1518-4d43-bee9-7fd768ee5cb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":3971150737},{"type":"v1","addr":"172.21.15.184:6801","nonce":3971150737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3971150737},{"type":"v1","addr":"172.21.15.184:6803","nonce":3971150737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3971150737},{"type":"v1","addr":"172.21.15.184:6807","nonce":3971150737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3971150737},{"type":"v1","addr":"172.21.15.184:6805","nonce":3971150737}]},"public_addr":"172.21.15.184:6801/3971150737","cluster_addr":"172.21.15.184:6803/3971150737","heartbeat_back_addr":"172.21.15.184:6807/3971150737","heartbeat_front_addr":"172.21.15.184:6805/3971150737","state":["exists","up"]},{"osd":3,"uuid":"ab3df826-39e3-4635-a49c-2b5ffdbf12f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":556594400},{"type":"v1","addr":"172.21.15.151:6811","nonce":556594400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":556594400},{"type":"v1","addr":"172.21.15.151:6813","nonce":556594400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":556594400},{"type":"v1","addr":"172.21.15.151:6817","nonce":556594400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":556594400},{"type":"v1","addr":"172.21.15.151:6815","nonce":556594400}]},"public_addr":"172.21.15.151:6811/556594400","cluster_addr":"172.21.15.151:6813/556594400","heartbeat_back_addr":"172.21.15.151:6817/556594400","heartbeat_front_addr":"172.21.15.151:6815/556594400","state":["exists","up"]},{"osd":4,"uuid":"00cbf6ae-0f42-4685-a77d-d803c6467aa0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":988881973},{"type":"v1","addr":"172.21.15.164:6809","nonce":988881973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":988881973},{"type":"v1","addr":"172.21.15.164:6811","nonce":988881973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":988881973},{"type":"v1","addr":"172.21.15.164:6815","nonce":988881973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":988881973},{"type":"v1","addr":"172.21.15.164:6813","nonce":988881973}]},"public_addr":"172.21.15.164:6809/988881973","cluster_addr":"172.21.15.164:6811/988881973","heartbeat_back_addr":"172.21.15.164:6815/988881973","heartbeat_front_addr":"172.21.15.164:6813/988881973","state":["exists","up"]},{"osd":5,"uuid":"fad14221-0ef0-4d40-b414-c6c9f1f9718c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3293311106},{"type":"v1","addr":"172.21.15.184:6809","nonce":3293311106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":3293311106},{"type":"v1","addr":"172.21.15.184:6811","nonce":3293311106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":3293311106},{"type":"v1","addr":"172.21.15.184:6815","nonce":3293311106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":3293311106},{"type":"v1","addr":"172.21.15.184:6813","nonce":3293311106}]},"public_addr":"172.21.15.184:6809/3293311106","cluster_addr":"172.21.15.184:6811/3293311106","heartbeat_back_addr":"172.21.15.184:6815/3293311106","heartbeat_front_addr":"172.21.15.184:6813/3293311106","state":["exists","up"]},{"osd":6,"uuid":"ab8a60f6-0dec-4c1c-89db-f2e2f638f1ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":3377550762},{"type":"v1","addr":"172.21.15.151:6819","nonce":3377550762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":3377550762},{"type":"v1","addr":"172.21.15.151:6821","nonce":3377550762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":3377550762},{"type":"v1","addr":"172.21.15.151:6825","nonce":3377550762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":3377550762},{"type":"v1","addr":"172.21.15.151:6823","nonce":3377550762}]},"public_addr":"172.21.15.151:6819/3377550762","cluster_addr":"172.21.15.151:6821/3377550762","heartbeat_back_addr":"172.21.15.151:6825/3377550762","heartbeat_front_addr":"172.21.15.151:6823/3377550762","state":["exists","up"]},{"osd":7,"uuid":"db332268-9972-4ed7-a307-822b83a0fbb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":3869696936},{"type":"v1","addr":"172.21.15.164:6817","nonce":3869696936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":3869696936},{"type":"v1","addr":"172.21.15.164:6819","nonce":3869696936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":3869696936},{"type":"v1","addr":"172.21.15.164:6823","nonce":3869696936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":3869696936},{"type":"v1","addr":"172.21.15.164:6821","nonce":3869696936}]},"public_addr":"172.21.15.164:6817/3869696936","cluster_addr":"172.21.15.164:6819/3869696936","heartbeat_back_addr":"172.21.15.164:6823/3869696936","heartbeat_front_addr":"172.21.15.164:6821/3869696936","state":["exists","up"]},{"osd":8,"uuid":"fe2fbe1c-549a-4947-a188-426dfca04d10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":575273541},{"type":"v1","addr":"172.21.15.184:6817","nonce":575273541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":575273541},{"type":"v1","addr":"172.21.15.184:6819","nonce":575273541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":575273541},{"type":"v1","addr":"172.21.15.184:6823","nonce":575273541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":575273541},{"type":"v1","addr":"172.21.15.184:6821","nonce":575273541}]},"public_addr":"172.21.15.184:6817/575273541","cluster_addr":"172.21.15.184:6819/575273541","heartbeat_back_addr":"172.21.15.184:6823/575273541","heartbeat_front_addr":"172.21.15.184:6821/575273541","state":["exists","up"]},{"osd":9,"uuid":"3a7b450f-8004-4adf-bae3-970918f98f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":337128533},{"type":"v1","addr":"172.21.15.151:6827","nonce":337128533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":337128533},{"type":"v1","addr":"172.21.15.151:6829","nonce":337128533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":337128533},{"type":"v1","addr":"172.21.15.151:6833","nonce":337128533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":337128533},{"type":"v1","addr":"172.21.15.151:6831","nonce":337128533}]},"public_addr":"172.21.15.151:6827/337128533","cluster_addr":"172.21.15.151:6829/337128533","heartbeat_back_addr":"172.21.15.151:6833/337128533","heartbeat_front_addr":"172.21.15.151:6831/337128533","state":["exists","up"]},{"osd":10,"uuid":"8fcdf271-90c4-4db3-be69-b0ca716dfd80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":1823448782},{"type":"v1","addr":"172.21.15.164:6825","nonce":1823448782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":1823448782},{"type":"v1","addr":"172.21.15.164:6827","nonce":1823448782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":1823448782},{"type":"v1","addr":"172.21.15.164:6831","nonce":1823448782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":1823448782},{"type":"v1","addr":"172.21.15.164:6829","nonce":1823448782}]},"public_addr":"172.21.15.164:6825/1823448782","cluster_addr":"172.21.15.164:6827/1823448782","heartbeat_back_addr":"172.21.15.164:6831/1823448782","heartbeat_front_addr":"172.21.15.164:6829/1823448782","state":["exists","up"]},{"osd":11,"uuid":"e661477b-0c09-409f-a2fe-6ebd34642a8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":1563340128},{"type":"v1","addr":"172.21.15.184:6825","nonce":1563340128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":1563340128},{"type":"v1","addr":"172.21.15.184:6827","nonce":1563340128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":1563340128},{"type":"v1","addr":"172.21.15.184:6831","nonce":1563340128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":1563340128},{"type":"v1","addr":"172.21.15.184:6829","nonce":1563340128}]},"public_addr":"172.21.15.184:6825/1563340128","cluster_addr":"172.21.15.184:6827/1563340128","heartbeat_back_addr":"172.21.15.184:6831/1563340128","heartbeat_front_addr":"172.21.15.184:6829/1563340128","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:45:02.821160+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:45:27.844564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:45:51.904981+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:46:24.376245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:46:50.036621+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:47:14.748422+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:47:47.396383+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:48:13.543174+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:48:38.213285+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:49:11.980042+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:49:38.519761+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:50:03.439898+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.151:0/31963128":"2024-05-22T21:43:09.041455+0000","172.21.15.151:6800/3428827300":"2024-05-22T21:43:09.041455+0000","172.21.15.151:6801/3428827300":"2024-05-22T21:43:09.041455+0000","172.21.15.151:0/131989141":"2024-05-22T21:43:09.041455+0000","172.21.15.151:6801/1128885292":"2024-05-22T21:42:35.440147+0000","172.21.15.151:0/2703664519":"2024-05-22T21:43:09.041455+0000","172.21.15.151:6800/1128885292":"2024-05-22T21:42:35.440147+0000","172.21.15.151:0/3052943699":"2024-05-22T21:42:35.440147+0000","172.21.15.151:0/3929452511":"2024-05-22T21:42:35.440147+0000","172.21.15.151:0/3762889700":"2024-05-22T21:42:35.440147+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-21T21:52:49.510 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:49.509+0000 7fa2d9c00700 1 -- 172.21.15.151:0/2088445316 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fa2b80b2910 msgr2=0x7fa2b80b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:52:49.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:49.509+0000 7fa2d9c00700 1 --2- 172.21.15.151:0/2088445316 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fa2b80b2910 0x7fa2b80b4dd0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa2c4006010 tx=0x7fa2c400b260 comp rx=0 tx=0).stop 2024-05-21T21:52:49.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:49.509+0000 7fa2d9c00700 1 -- 172.21.15.151:0/2088445316 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa2d4106030 msgr2=0x7fa2d419c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:52:49.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:49.509+0000 7fa2d9c00700 1 --2- 172.21.15.151:0/2088445316 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa2d4106030 0x7fa2d419c310 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa2c000e9c0 tx=0x7fa2c000ecd0 comp rx=0 tx=0).stop 2024-05-21T21:52:49.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:49.510+0000 7fa2d9c00700 1 -- 172.21.15.151:0/2088445316 shutdown_connections 2024-05-21T21:52:49.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:49.510+0000 7fa2d9c00700 1 --2- 172.21.15.151:0/2088445316 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fa2b80b2910 0x7fa2b80b4dd0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:49.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:49.510+0000 7fa2d9c00700 1 --2- 172.21.15.151:0/2088445316 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa2d410b670 0x7fa2d41a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:49.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:49.510+0000 7fa2d9c00700 1 --2- 172.21.15.151:0/2088445316 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2d4105670 0x7fa2d419bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:49.512 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:49.510+0000 7fa2d9c00700 1 --2- 172.21.15.151:0/2088445316 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa2d4106030 0x7fa2d419c310 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:49.512 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:49.510+0000 7fa2d9c00700 1 -- 172.21.15.151:0/2088445316 >> 172.21.15.151:0/2088445316 conn(0x7fa2d4101160 msgr2=0x7fa2d4101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:52:49.512 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:49.510+0000 7fa2d9c00700 1 -- 172.21.15.151:0/2088445316 shutdown_connections 2024-05-21T21:52:49.512 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:49.510+0000 7fa2d9c00700 1 -- 172.21.15.151:0/2088445316 wait complete. 2024-05-21T21:52:50.449 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-05-21T21:52:50.449 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-05-21T21:52:50.450 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-05-21T21:52:50.450 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-05-21T21:52:50.451 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-05-21T21:52:50.451 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-05-21T21:52:50.451 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-05-21T21:52:50.452 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-05-21T21:52:50.452 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-05-21T21:52:50.453 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-05-21T21:52:50.453 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-05-21T21:52:50.453 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-05-21T21:52:50.584 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:52:50 smithi151 ceph-mon[31623]: pgmap v362: 97 pgs: 97 active+clean; 642 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-05-21T21:52:50.584 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:52:50 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/2088445316' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T21:52:50.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:52:50 smithi184 ceph-mon[36942]: pgmap v362: 97 pgs: 97 active+clean; 642 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-05-21T21:52:50.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:52:50 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/2088445316' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T21:52:50.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:52:50 smithi164 ceph-mon[36269]: pgmap v362: 97 pgs: 97 active+clean; 642 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-05-21T21:52:50.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:52:50 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/2088445316' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T21:52:51.556 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:52:51.595 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:52:51.596 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:52:51.602 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:52:51.603 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:52:51.606 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:52:51.613 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:52:51.618 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:52:51.618 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:52:51.632 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:52:51.637 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:52:52.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:52:52 smithi151 ceph-mon[31623]: pgmap v363: 97 pgs: 97 active+clean; 642 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-05-21T21:52:52.735 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:52:52 smithi184 ceph-mon[36942]: pgmap v363: 97 pgs: 97 active+clean; 642 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-05-21T21:52:52.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:52:52 smithi164 ceph-mon[36269]: pgmap v363: 97 pgs: 97 active+clean; 642 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-05-21T21:52:54.426 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:52:54 smithi151 ceph-mon[31623]: pgmap v364: 97 pgs: 97 active+clean; 642 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-05-21T21:52:54.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:52:54 smithi184 ceph-mon[36942]: pgmap v364: 97 pgs: 97 active+clean; 642 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-05-21T21:52:54.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:52:54 smithi164 ceph-mon[36269]: pgmap v364: 97 pgs: 97 active+clean; 642 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-05-21T21:52:56.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:56.096+0000 7fac10a00700 1 -- 172.21.15.151:0/1670113579 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac0c075970 msgr2=0x7fac0c075d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:52:56.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:56.096+0000 7fac10a00700 1 --2- 172.21.15.151:0/1670113579 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac0c075970 0x7fac0c075d50 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fabfc009990 tx=0x7fabfc009ca0 comp rx=0 tx=0).stop 2024-05-21T21:52:56.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:56.096+0000 7fac10a00700 1 -- 172.21.15.151:0/1670113579 shutdown_connections 2024-05-21T21:52:56.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:56.096+0000 7fac10a00700 1 --2- 172.21.15.151:0/1670113579 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fac0c10fc90 0x7fac0c112100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:56.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:56.096+0000 7fac10a00700 1 --2- 172.21.15.151:0/1670113579 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac0c075970 0x7fac0c075d50 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:56.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:56.096+0000 7fac10a00700 1 --2- 172.21.15.151:0/1670113579 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac0c076290 0x7fac0c10f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:56.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:56.096+0000 7fac10a00700 1 -- 172.21.15.151:0/1670113579 >> 172.21.15.151:0/1670113579 conn(0x7fac0c0fef60 msgr2=0x7fac0c101380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:52:56.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:56.096+0000 7fac10a00700 1 -- 172.21.15.151:0/1670113579 shutdown_connections 2024-05-21T21:52:56.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:56.096+0000 7fac10a00700 1 -- 172.21.15.151:0/1670113579 wait complete. 2024-05-21T21:52:56.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:56.097+0000 7fac10a00700 1 Processor -- start 2024-05-21T21:52:56.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:56.098+0000 7fac10a00700 1 -- start start 2024-05-21T21:52:56.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:56.098+0000 7fac10a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac0c075970 0x7fac0c19bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:52:56.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:56.098+0000 7fac10a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fac0c076290 0x7fac0c19c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:52:56.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:56.098+0000 7fac10a00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac0c10fc90 0x7fac0c1a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:52:56.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:56.098+0000 7fac10a00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fac0c115840 con 0x7fac0c075970 2024-05-21T21:52:56.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:56.098+0000 7fac10a00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fac0c1156c0 con 0x7fac0c076290 2024-05-21T21:52:56.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:56.098+0000 7fac10a00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fac0c1159c0 con 0x7fac0c10fc90 2024-05-21T21:52:56.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:56.098+0000 7fac0a800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac0c075970 0x7fac0c19bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:52:56.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:56.098+0000 7fac0a800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac0c075970 0x7fac0c19bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:43106/0 (socket says 172.21.15.151:43106) 2024-05-21T21:52:56.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:56.098+0000 7fac0a800700 1 -- 172.21.15.151:0/873041993 learned_addr learned my addr 172.21.15.151:0/873041993 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:52:56.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:56.098+0000 7fac0b200700 1 --2- 172.21.15.151:0/873041993 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac0c10fc90 0x7fac0c1a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:52:56.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:56.098+0000 7fac09e00700 1 --2- 172.21.15.151:0/873041993 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fac0c076290 0x7fac0c19c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:52:56.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:56.099+0000 7fac0b200700 1 -- 172.21.15.151:0/873041993 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fac0c076290 msgr2=0x7fac0c19c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:52:56.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:56.099+0000 7fac0b200700 1 --2- 172.21.15.151:0/873041993 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fac0c076290 0x7fac0c19c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:56.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:56.099+0000 7fac0b200700 1 -- 172.21.15.151:0/873041993 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac0c075970 msgr2=0x7fac0c19bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:52:56.350 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:56.099+0000 7fac0b200700 1 --2- 172.21.15.151:0/873041993 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac0c075970 0x7fac0c19bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:56.350 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:56.099+0000 7fac0b200700 1 -- 172.21.15.151:0/873041993 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac0c1a0dc0 con 0x7fac0c10fc90 2024-05-21T21:52:56.350 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:56.099+0000 7fac0a800700 1 --2- 172.21.15.151:0/873041993 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac0c075970 0x7fac0c19bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:52:56.350 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:56.099+0000 7fac09e00700 1 --2- 172.21.15.151:0/873041993 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fac0c076290 0x7fac0c19c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:52:56.350 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:56.099+0000 7fac0b200700 1 --2- 172.21.15.151:0/873041993 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac0c10fc90 0x7fac0c1a0680 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fac04007f10 tx=0x7fac0400d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:52:56.350 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:56.099+0000 7fac03400700 1 -- 172.21.15.151:0/873041993 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac04004db0 con 0x7fac0c10fc90 2024-05-21T21:52:56.350 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:56.099+0000 7fac03400700 1 -- 172.21.15.151:0/873041993 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac04003a00 con 0x7fac0c10fc90 2024-05-21T21:52:56.350 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:56.099+0000 7fac03400700 1 -- 172.21.15.151:0/873041993 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac04004580 con 0x7fac0c10fc90 2024-05-21T21:52:56.350 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:56.099+0000 7fac10a00700 1 -- 172.21.15.151:0/873041993 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac0c1a10b0 con 0x7fac0c10fc90 2024-05-21T21:52:56.350 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:56.100+0000 7fac10a00700 1 -- 172.21.15.151:0/873041993 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac0c1a8930 con 0x7fac0c10fc90 2024-05-21T21:52:56.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:56.100+0000 7fac10a00700 1 -- 172.21.15.151:0/873041993 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fac0c063cd0 con 0x7fac0c10fc90 2024-05-21T21:52:56.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:56.103+0000 7fac03400700 1 -- 172.21.15.151:0/873041993 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fac0401c020 con 0x7fac0c10fc90 2024-05-21T21:52:56.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:56.104+0000 7fac03400700 1 --2- 172.21.15.151:0/873041993 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fabf00b2990 0x7fabf00b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:52:56.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:56.105+0000 7fac03400700 1 -- 172.21.15.151:0/873041993 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fac040e27a0 con 0x7fac0c10fc90 2024-05-21T21:52:56.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:56.105+0000 7fac0a800700 1 --2- 172.21.15.151:0/873041993 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fabf00b2990 0x7fabf00b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:52:56.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:56.105+0000 7fac03400700 1 --2- 172.21.15.151:0/873041993 >> [v2:172.21.15.184:6816/575273541,v1:172.21.15.184:6817/575273541] conn(0x7fabf00bc550 0x7fabf00be9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:52:56.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:56.105+0000 7fac03400700 1 -- 172.21.15.151:0/873041993 --> [v2:172.21.15.184:6816/575273541,v1:172.21.15.184:6817/575273541] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fac04020e70 con 0x7fabf00bc550 2024-05-21T21:52:56.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:56.105+0000 7fac03400700 1 -- 172.21.15.151:0/873041993 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7fac040e2b50 con 0x7fac0c10fc90 2024-05-21T21:52:56.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:56.105+0000 7fac09e00700 1 --2- 172.21.15.151:0/873041993 >> [v2:172.21.15.184:6816/575273541,v1:172.21.15.184:6817/575273541] conn(0x7fabf00bc550 0x7fabf00be9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:52:56.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:56.106+0000 7fac0a800700 1 --2- 172.21.15.151:0/873041993 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fabf00b2990 0x7fabf00b4e50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fabfc009750 tx=0x7fabfc01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:52:56.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:56.106+0000 7fac09e00700 1 --2- 172.21.15.151:0/873041993 >> [v2:172.21.15.184:6816/575273541,v1:172.21.15.184:6817/575273541] conn(0x7fabf00bc550 0x7fabf00be9f0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:52:56.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:56.107+0000 7fac03400700 1 -- 172.21.15.151:0/873041993 <== osd.8 v2:172.21.15.184:6816/575273541 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fac04020e70 con 0x7fabf00bc550 2024-05-21T21:52:56.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:56.126+0000 7fac10a00700 1 -- 172.21.15.151:0/873041993 --> [v2:172.21.15.184:6816/575273541,v1:172.21.15.184:6817/575273541] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fac0c064150 con 0x7fabf00bc550 2024-05-21T21:52:56.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:56.127+0000 7fac03400700 1 -- 172.21.15.151:0/873041993 <== osd.8 v2:172.21.15.184:6816/575273541 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fac0c064150 con 0x7fabf00bc550 2024-05-21T21:52:56.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:56.127+0000 7fac10a00700 1 -- 172.21.15.151:0/873041993 >> [v2:172.21.15.184:6816/575273541,v1:172.21.15.184:6817/575273541] conn(0x7fabf00bc550 msgr2=0x7fabf00be9f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:52:56.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:56.127+0000 7fac10a00700 1 --2- 172.21.15.151:0/873041993 >> [v2:172.21.15.184:6816/575273541,v1:172.21.15.184:6817/575273541] conn(0x7fabf00bc550 0x7fabf00be9f0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:52:56.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:56.128+0000 7fac10a00700 1 -- 172.21.15.151:0/873041993 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fabf00b2990 msgr2=0x7fabf00b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:52:56.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:56.128+0000 7fac10a00700 1 --2- 172.21.15.151:0/873041993 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fabf00b2990 0x7fabf00b4e50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fabfc009750 tx=0x7fabfc01a040 comp rx=0 tx=0).stop 2024-05-21T21:52:56.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:56.128+0000 7fac10a00700 1 -- 172.21.15.151:0/873041993 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac0c10fc90 msgr2=0x7fac0c1a0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:52:56.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:56.128+0000 7fac10a00700 1 --2- 172.21.15.151:0/873041993 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac0c10fc90 0x7fac0c1a0680 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fac04007f10 tx=0x7fac0400d3b0 comp rx=0 tx=0).stop 2024-05-21T21:52:56.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:56.128+0000 7fac0b200700 1 -- 172.21.15.151:0/873041993 reap_dead start 2024-05-21T21:52:56.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:56.128+0000 7fac10a00700 1 -- 172.21.15.151:0/873041993 shutdown_connections 2024-05-21T21:52:56.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:56.128+0000 7fac10a00700 1 -- 172.21.15.151:0/873041993 >> 172.21.15.151:0/873041993 conn(0x7fac0c0fef60 msgr2=0x7fac0c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:52:56.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:56.128+0000 7fac10a00700 1 -- 172.21.15.151:0/873041993 shutdown_connections 2024-05-21T21:52:56.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:56.128+0000 7fac10a00700 1 -- 172.21.15.151:0/873041993 wait complete. 2024-05-21T21:52:56.353 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:52:56 smithi151 ceph-mon[31623]: pgmap v365: 97 pgs: 97 active+clean; 642 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-05-21T21:52:56.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:52:56 smithi184 ceph-mon[36942]: pgmap v365: 97 pgs: 97 active+clean; 642 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-05-21T21:52:56.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:52:56 smithi164 ceph-mon[36269]: pgmap v365: 97 pgs: 97 active+clean; 642 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-05-21T21:52:58.539 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:52:58 smithi151 ceph-mon[31623]: pgmap v366: 97 pgs: 97 active+clean; 642 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-05-21T21:52:58.735 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:52:58 smithi184 ceph-mon[36942]: pgmap v366: 97 pgs: 97 active+clean; 642 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-05-21T21:52:58.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:52:58 smithi164 ceph-mon[36269]: pgmap v366: 97 pgs: 97 active+clean; 642 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-05-21T21:52:59.206 INFO:teuthology.orchestra.run.smithi151.stdout:236223201334 2024-05-21T21:52:59.207 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-05-21T21:53:00.589 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:53:00 smithi151 ceph-mon[31623]: pgmap v367: 97 pgs: 97 active+clean; 642 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-21T21:53:00.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:53:00 smithi184 ceph-mon[36942]: pgmap v367: 97 pgs: 97 active+clean; 642 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-21T21:53:00.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:53:00 smithi164 ceph-mon[36269]: pgmap v367: 97 pgs: 97 active+clean; 642 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-21T21:53:02.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:58.780+0000 7fac22800700 1 -- 172.21.15.151:0/2670555531 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac1c10fff0 msgr2=0x7fac1c1123e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:02.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:58.780+0000 7fac22800700 1 --2- 172.21.15.151:0/2670555531 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac1c10fff0 0x7fac1c1123e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fac1400b0a0 tx=0x7fac1400b3b0 comp rx=0 tx=0).stop 2024-05-21T21:53:02.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:58.780+0000 7fac22800700 1 -- 172.21.15.151:0/2670555531 shutdown_connections 2024-05-21T21:53:02.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:58.780+0000 7fac22800700 1 --2- 172.21.15.151:0/2670555531 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac1c10fff0 0x7fac1c1123e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:02.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:58.780+0000 7fac22800700 1 --2- 172.21.15.151:0/2670555531 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac1c06b250 0x7fac1c06b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:02.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:58.780+0000 7fac22800700 1 --2- 172.21.15.151:0/2670555531 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fac1c06bb70 0x7fac1c10fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:02.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:58.780+0000 7fac22800700 1 -- 172.21.15.151:0/2670555531 >> 172.21.15.151:0/2670555531 conn(0x7fac1c0793f0 msgr2=0x7fac1c079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:53:02.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:58.780+0000 7fac22800700 1 -- 172.21.15.151:0/2670555531 shutdown_connections 2024-05-21T21:53:02.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:58.780+0000 7fac22800700 1 -- 172.21.15.151:0/2670555531 wait complete. 2024-05-21T21:53:02.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:58.781+0000 7fac22800700 1 Processor -- start 2024-05-21T21:53:02.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:58.781+0000 7fac22800700 1 -- start start 2024-05-21T21:53:02.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:58.782+0000 7fac22800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fac1c06b250 0x7fac1c19be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:02.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:58.782+0000 7fac22800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac1c06bb70 0x7fac1c19c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:02.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:58.782+0000 7fac22800700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac1c10fff0 0x7fac1c1a0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:02.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:58.782+0000 7fac22800700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fac1c115960 con 0x7fac1c06bb70 2024-05-21T21:53:02.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:58.782+0000 7fac22800700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fac1c1157e0 con 0x7fac1c06b250 2024-05-21T21:53:02.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:58.782+0000 7fac22800700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fac1c115ae0 con 0x7fac1c10fff0 2024-05-21T21:53:02.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:58.782+0000 7fac13e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac1c06bb70 0x7fac1c19c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:02.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:58.782+0000 7fac1b400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fac1c06b250 0x7fac1c19be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:02.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:58.782+0000 7fac13e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac1c06bb70 0x7fac1c19c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:43126/0 (socket says 172.21.15.151:43126) 2024-05-21T21:53:02.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:58.782+0000 7fac13e00700 1 -- 172.21.15.151:0/2473355257 learned_addr learned my addr 172.21.15.151:0/2473355257 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:53:02.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:58.782+0000 7fac1be00700 1 --2- 172.21.15.151:0/2473355257 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac1c10fff0 0x7fac1c1a0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:02.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:58.783+0000 7fac1b400700 1 -- 172.21.15.151:0/2473355257 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac1c10fff0 msgr2=0x7fac1c1a0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:02.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:58.783+0000 7fac1b400700 1 --2- 172.21.15.151:0/2473355257 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac1c10fff0 0x7fac1c1a0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:02.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:58.783+0000 7fac1b400700 1 -- 172.21.15.151:0/2473355257 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac1c06bb70 msgr2=0x7fac1c19c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:02.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:58.783+0000 7fac1b400700 1 --2- 172.21.15.151:0/2473355257 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac1c06bb70 0x7fac1c19c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:02.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:58.783+0000 7fac1b400700 1 -- 172.21.15.151:0/2473355257 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac1c1a0e50 con 0x7fac1c06b250 2024-05-21T21:53:02.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:58.783+0000 7fac13e00700 1 --2- 172.21.15.151:0/2473355257 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac1c06bb70 0x7fac1c19c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:53:02.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:58.783+0000 7fac1b400700 1 --2- 172.21.15.151:0/2473355257 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fac1c06b250 0x7fac1c19be90 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fac0c00d750 tx=0x7fac0c00da60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:53:02.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:58.783+0000 7fac18c00700 1 -- 172.21.15.151:0/2473355257 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac0c014030 con 0x7fac1c06b250 2024-05-21T21:53:02.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:58.783+0000 7fac22800700 1 -- 172.21.15.151:0/2473355257 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac1c1a1140 con 0x7fac1c06b250 2024-05-21T21:53:02.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:58.783+0000 7fac22800700 1 -- 172.21.15.151:0/2473355257 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac1c1a89c0 con 0x7fac1c06b250 2024-05-21T21:53:02.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:58.783+0000 7fac18c00700 1 -- 172.21.15.151:0/2473355257 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac0c0044d0 con 0x7fac1c06b250 2024-05-21T21:53:02.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:58.784+0000 7fac18c00700 1 -- 172.21.15.151:0/2473355257 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac0c005420 con 0x7fac1c06b250 2024-05-21T21:53:02.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:58.784+0000 7fac22800700 1 -- 172.21.15.151:0/2473355257 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fabf8001030 con 0x7fac1c06b250 2024-05-21T21:53:02.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:58.787+0000 7fac18c00700 1 -- 172.21.15.151:0/2473355257 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fac0c0055c0 con 0x7fac1c06b250 2024-05-21T21:53:02.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:58.788+0000 7fac18c00700 1 --2- 172.21.15.151:0/2473355257 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fac000b2a10 0x7fac000b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:02.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:58.788+0000 7fac18c00700 1 -- 172.21.15.151:0/2473355257 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fac0c0e2000 con 0x7fac1c06b250 2024-05-21T21:53:02.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:58.789+0000 7fac13e00700 1 --2- 172.21.15.151:0/2473355257 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fac000b2a10 0x7fac000b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:02.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:58.789+0000 7fac18c00700 1 --2- 172.21.15.151:0/2473355257 >> [v2:172.21.15.184:6808/3293311106,v1:172.21.15.184:6809/3293311106] conn(0x7fac000bc5d0 0x7fac000bea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:02.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:58.789+0000 7fac18c00700 1 -- 172.21.15.151:0/2473355257 --> [v2:172.21.15.184:6808/3293311106,v1:172.21.15.184:6809/3293311106] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fac0c022e50 con 0x7fac000bc5d0 2024-05-21T21:53:02.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:58.789+0000 7fac18c00700 1 -- 172.21.15.151:0/2473355257 <== mon.2 v2:172.21.15.164:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7fac0c0e23f0 con 0x7fac1c06b250 2024-05-21T21:53:02.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:58.789+0000 7fac1be00700 1 --2- 172.21.15.151:0/2473355257 >> [v2:172.21.15.184:6808/3293311106,v1:172.21.15.184:6809/3293311106] conn(0x7fac000bc5d0 0x7fac000bea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:02.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:58.789+0000 7fac13e00700 1 --2- 172.21.15.151:0/2473355257 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fac000b2a10 0x7fac000b4ed0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fac1c103a00 tx=0x7fac04009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:53:02.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:58.790+0000 7fac1be00700 1 --2- 172.21.15.151:0/2473355257 >> [v2:172.21.15.184:6808/3293311106,v1:172.21.15.184:6809/3293311106] conn(0x7fac000bc5d0 0x7fac000bea70 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:53:02.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:58.791+0000 7fac18c00700 1 -- 172.21.15.151:0/2473355257 <== osd.5 v2:172.21.15.184:6808/3293311106 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fac0c022e50 con 0x7fac000bc5d0 2024-05-21T21:53:02.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:58.808+0000 7fac22800700 1 -- 172.21.15.151:0/2473355257 --> [v2:172.21.15.184:6808/3293311106,v1:172.21.15.184:6809/3293311106] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fabf8002e60 con 0x7fac000bc5d0 2024-05-21T21:53:02.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:58.809+0000 7fac18c00700 1 -- 172.21.15.151:0/2473355257 <== osd.5 v2:172.21.15.184:6808/3293311106 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fabf8002e60 con 0x7fac000bc5d0 2024-05-21T21:53:02.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:58.809+0000 7fac22800700 1 -- 172.21.15.151:0/2473355257 >> [v2:172.21.15.184:6808/3293311106,v1:172.21.15.184:6809/3293311106] conn(0x7fac000bc5d0 msgr2=0x7fac000bea70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:02.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:58.809+0000 7fac22800700 1 --2- 172.21.15.151:0/2473355257 >> [v2:172.21.15.184:6808/3293311106,v1:172.21.15.184:6809/3293311106] conn(0x7fac000bc5d0 0x7fac000bea70 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:02.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:58.809+0000 7fac22800700 1 -- 172.21.15.151:0/2473355257 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fac000b2a10 msgr2=0x7fac000b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:02.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:58.809+0000 7fac22800700 1 --2- 172.21.15.151:0/2473355257 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fac000b2a10 0x7fac000b4ed0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fac1c103a00 tx=0x7fac04009260 comp rx=0 tx=0).stop 2024-05-21T21:53:02.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:58.809+0000 7fac22800700 1 -- 172.21.15.151:0/2473355257 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fac1c06b250 msgr2=0x7fac1c19be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:02.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:58.809+0000 7fac22800700 1 --2- 172.21.15.151:0/2473355257 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fac1c06b250 0x7fac1c19be90 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fac0c00d750 tx=0x7fac0c00da60 comp rx=0 tx=0).stop 2024-05-21T21:53:02.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:58.809+0000 7fac1be00700 1 -- 172.21.15.151:0/2473355257 reap_dead start 2024-05-21T21:53:02.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:58.809+0000 7fac22800700 1 -- 172.21.15.151:0/2473355257 shutdown_connections 2024-05-21T21:53:02.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:58.809+0000 7fac22800700 1 -- 172.21.15.151:0/2473355257 >> 172.21.15.151:0/2473355257 conn(0x7fac1c0793f0 msgr2=0x7fac1c1114e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:53:02.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:58.809+0000 7fac22800700 1 -- 172.21.15.151:0/2473355257 shutdown_connections 2024-05-21T21:53:02.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:52:58.809+0000 7fac22800700 1 -- 172.21.15.151:0/2473355257 wait complete. 2024-05-21T21:53:02.702 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:53:02 smithi151 ceph-mon[31623]: pgmap v368: 97 pgs: 97 active+clean; 642 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-05-21T21:53:02.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:53:02 smithi184 ceph-mon[36942]: pgmap v368: 97 pgs: 97 active+clean; 642 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-05-21T21:53:02.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:53:02 smithi164 ceph-mon[36269]: pgmap v368: 97 pgs: 97 active+clean; 642 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-05-21T21:53:04.702 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:53:04 smithi151 ceph-mon[31623]: pgmap v369: 97 pgs: 97 active+clean; 642 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-21T21:53:04.735 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:53:04 smithi184 ceph-mon[36942]: pgmap v369: 97 pgs: 97 active+clean; 642 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-21T21:53:04.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:53:04 smithi164 ceph-mon[36269]: pgmap v369: 97 pgs: 97 active+clean; 642 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-21T21:53:05.920 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:05.901+0000 7fef8fc00700 1 -- 172.21.15.151:0/100088128 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fef881037f0 msgr2=0x7fef88103bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:05.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:05.901+0000 7fef8fc00700 1 --2- 172.21.15.151:0/100088128 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fef881037f0 0x7fef88103bd0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fef7c0099c0 tx=0x7fef7c009cd0 comp rx=0 tx=0).stop 2024-05-21T21:53:05.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:05.901+0000 7fef8fc00700 1 -- 172.21.15.151:0/100088128 shutdown_connections 2024-05-21T21:53:05.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:05.901+0000 7fef8fc00700 1 --2- 172.21.15.151:0/100088128 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fef881088f0 0x7fef88110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:05.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:05.901+0000 7fef8fc00700 1 --2- 172.21.15.151:0/100088128 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fef881037f0 0x7fef88103bd0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:05.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:05.901+0000 7fef8fc00700 1 --2- 172.21.15.151:0/100088128 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fef881041a0 0x7fef881081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:05.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:05.901+0000 7fef8fc00700 1 -- 172.21.15.151:0/100088128 >> 172.21.15.151:0/100088128 conn(0x7fef8807a6a0 msgr2=0x7fef8807aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:53:05.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:05.901+0000 7fef8fc00700 1 -- 172.21.15.151:0/100088128 shutdown_connections 2024-05-21T21:53:05.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:05.901+0000 7fef8fc00700 1 -- 172.21.15.151:0/100088128 wait complete. 2024-05-21T21:53:05.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:05.902+0000 7fef8fc00700 1 Processor -- start 2024-05-21T21:53:05.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:05.903+0000 7fef8fc00700 1 -- start start 2024-05-21T21:53:05.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:05.903+0000 7fef8fc00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fef881037f0 0x7fef8819bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:05.922 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:05.903+0000 7fef8fc00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fef881041a0 0x7fef8819c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:05.922 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:05.903+0000 7fef8fc00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fef881088f0 0x7fef881a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:05.922 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:05.903+0000 7fef8fc00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fef88113710 con 0x7fef881037f0 2024-05-21T21:53:05.922 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:05.903+0000 7fef8fc00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fef88113590 con 0x7fef881088f0 2024-05-21T21:53:05.922 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:05.903+0000 7fef8fc00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fef88113890 con 0x7fef881041a0 2024-05-21T21:53:05.922 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:05.903+0000 7fef8d200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fef881037f0 0x7fef8819bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:05.922 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:05.903+0000 7fef8d200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fef881037f0 0x7fef8819bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:60852/0 (socket says 172.21.15.151:60852) 2024-05-21T21:53:05.922 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:05.903+0000 7fef8d200700 1 -- 172.21.15.151:0/3493662155 learned_addr learned my addr 172.21.15.151:0/3493662155 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:53:05.922 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:05.903+0000 7fef87e00700 1 --2- 172.21.15.151:0/3493662155 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fef881041a0 0x7fef8819c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:05.922 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:05.903+0000 7fef8dc00700 1 --2- 172.21.15.151:0/3493662155 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fef881088f0 0x7fef881a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:05.922 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:05.904+0000 7fef8d200700 1 -- 172.21.15.151:0/3493662155 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fef881041a0 msgr2=0x7fef8819c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:05.922 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:05.904+0000 7fef8d200700 1 --2- 172.21.15.151:0/3493662155 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fef881041a0 0x7fef8819c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:05.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:05.904+0000 7fef8d200700 1 -- 172.21.15.151:0/3493662155 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fef881088f0 msgr2=0x7fef881a06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:05.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:05.904+0000 7fef8d200700 1 --2- 172.21.15.151:0/3493662155 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fef881088f0 0x7fef881a06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:05.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:05.904+0000 7fef8d200700 1 -- 172.21.15.151:0/3493662155 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef881a0e00 con 0x7fef881037f0 2024-05-21T21:53:05.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:05.904+0000 7fef8dc00700 1 --2- 172.21.15.151:0/3493662155 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fef881088f0 0x7fef881a06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:53:05.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:05.904+0000 7fef8d200700 1 --2- 172.21.15.151:0/3493662155 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fef881037f0 0x7fef8819bdf0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fef7c005910 tx=0x7fef7c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:53:05.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:05.904+0000 7fef85600700 1 -- 172.21.15.151:0/3493662155 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef7c026070 con 0x7fef881037f0 2024-05-21T21:53:05.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:05.904+0000 7fef8fc00700 1 -- 172.21.15.151:0/3493662155 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef881a1090 con 0x7fef881037f0 2024-05-21T21:53:05.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:05.904+0000 7fef85600700 1 -- 172.21.15.151:0/3493662155 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef7c003c70 con 0x7fef881037f0 2024-05-21T21:53:05.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:05.904+0000 7fef85600700 1 -- 172.21.15.151:0/3493662155 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef7c00fae0 con 0x7fef881037f0 2024-05-21T21:53:05.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:05.905+0000 7fef8fc00700 1 -- 172.21.15.151:0/3493662155 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef881a8970 con 0x7fef881037f0 2024-05-21T21:53:05.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:05.906+0000 7fef8fc00700 1 -- 172.21.15.151:0/3493662155 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fef88101fd0 con 0x7fef881037f0 2024-05-21T21:53:05.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:05.908+0000 7fef85600700 1 -- 172.21.15.151:0/3493662155 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7fef7c029070 con 0x7fef881037f0 2024-05-21T21:53:05.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:05.910+0000 7fef85600700 1 --2- 172.21.15.151:0/3493662155 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fef700b2990 0x7fef700b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:05.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:05.910+0000 7fef85600700 1 -- 172.21.15.151:0/3493662155 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fef7c0e2c00 con 0x7fef881037f0 2024-05-21T21:53:05.924 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:05.910+0000 7fef87e00700 1 --2- 172.21.15.151:0/3493662155 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fef700b2990 0x7fef700b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:05.924 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:05.910+0000 7fef85600700 1 --2- 172.21.15.151:0/3493662155 >> [v2:172.21.15.164:6808/988881973,v1:172.21.15.164:6809/988881973] conn(0x7fef700bc550 0x7fef700be9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:05.924 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:05.910+0000 7fef85600700 1 -- 172.21.15.151:0/3493662155 --> [v2:172.21.15.164:6808/988881973,v1:172.21.15.164:6809/988881973] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fef700bf0c0 con 0x7fef700bc550 2024-05-21T21:53:05.924 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:05.910+0000 7fef85600700 1 -- 172.21.15.151:0/3493662155 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7fef7c0e9020 con 0x7fef881037f0 2024-05-21T21:53:05.924 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:05.911+0000 7fef8dc00700 1 --2- 172.21.15.151:0/3493662155 >> [v2:172.21.15.164:6808/988881973,v1:172.21.15.164:6809/988881973] conn(0x7fef700bc550 0x7fef700be9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:05.924 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:05.911+0000 7fef87e00700 1 --2- 172.21.15.151:0/3493662155 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fef700b2990 0x7fef700b4e50 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fef881108b0 tx=0x7fef74009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:53:05.924 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:05.911+0000 7fef8dc00700 1 --2- 172.21.15.151:0/3493662155 >> [v2:172.21.15.164:6808/988881973,v1:172.21.15.164:6809/988881973] conn(0x7fef700bc550 0x7fef700be9f0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:53:05.924 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:05.912+0000 7fef85600700 1 -- 172.21.15.151:0/3493662155 <== osd.4 v2:172.21.15.164:6808/988881973 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fef700bf0c0 con 0x7fef700bc550 2024-05-21T21:53:05.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:05.930+0000 7fef8fc00700 1 -- 172.21.15.151:0/3493662155 --> [v2:172.21.15.164:6808/988881973,v1:172.21.15.164:6809/988881973] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fef88104c80 con 0x7fef700bc550 2024-05-21T21:53:05.932 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:05.931+0000 7fef85600700 1 -- 172.21.15.151:0/3493662155 <== osd.4 v2:172.21.15.164:6808/988881973 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fef88104c80 con 0x7fef700bc550 2024-05-21T21:53:05.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:05.931+0000 7fef8fc00700 1 -- 172.21.15.151:0/3493662155 >> [v2:172.21.15.164:6808/988881973,v1:172.21.15.164:6809/988881973] conn(0x7fef700bc550 msgr2=0x7fef700be9f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:05.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:05.931+0000 7fef8fc00700 1 --2- 172.21.15.151:0/3493662155 >> [v2:172.21.15.164:6808/988881973,v1:172.21.15.164:6809/988881973] conn(0x7fef700bc550 0x7fef700be9f0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:05.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:05.931+0000 7fef8fc00700 1 -- 172.21.15.151:0/3493662155 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fef700b2990 msgr2=0x7fef700b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:05.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:05.931+0000 7fef8fc00700 1 --2- 172.21.15.151:0/3493662155 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fef700b2990 0x7fef700b4e50 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fef881108b0 tx=0x7fef74009260 comp rx=0 tx=0).stop 2024-05-21T21:53:05.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:05.931+0000 7fef8fc00700 1 -- 172.21.15.151:0/3493662155 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fef881037f0 msgr2=0x7fef8819bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:05.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:05.931+0000 7fef8fc00700 1 --2- 172.21.15.151:0/3493662155 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fef881037f0 0x7fef8819bdf0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fef7c005910 tx=0x7fef7c004270 comp rx=0 tx=0).stop 2024-05-21T21:53:05.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:05.931+0000 7fef8dc00700 1 -- 172.21.15.151:0/3493662155 reap_dead start 2024-05-21T21:53:05.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:05.932+0000 7fef8fc00700 1 -- 172.21.15.151:0/3493662155 shutdown_connections 2024-05-21T21:53:05.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:05.932+0000 7fef8fc00700 1 -- 172.21.15.151:0/3493662155 >> 172.21.15.151:0/3493662155 conn(0x7fef8807a6a0 msgr2=0x7fef88101570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:53:05.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:05.932+0000 7fef8fc00700 1 -- 172.21.15.151:0/3493662155 shutdown_connections 2024-05-21T21:53:05.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:05.932+0000 7fef8fc00700 1 -- 172.21.15.151:0/3493662155 wait complete. 2024-05-21T21:53:06.183 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:06.182+0000 7f4678000700 1 -- 172.21.15.151:0/1226310543 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4670106120 msgr2=0x7f4670112630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:06.183 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:06.182+0000 7f4678000700 1 --2- 172.21.15.151:0/1226310543 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4670106120 0x7f4670112630 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f466800b0a0 tx=0x7f466800b3b0 comp rx=0 tx=0).stop 2024-05-21T21:53:06.183 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:06.182+0000 7f4678000700 1 -- 172.21.15.151:0/1226310543 shutdown_connections 2024-05-21T21:53:06.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:06.182+0000 7f4678000700 1 --2- 172.21.15.151:0/1226310543 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4670106120 0x7f4670112630 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:06.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:06.182+0000 7f4678000700 1 --2- 172.21.15.151:0/1226310543 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f467010b760 0x7f467010bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:06.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:06.182+0000 7f4678000700 1 --2- 172.21.15.151:0/1226310543 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4670105760 0x7f4670105be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:06.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:06.182+0000 7f4678000700 1 -- 172.21.15.151:0/1226310543 >> 172.21.15.151:0/1226310543 conn(0x7f4670101210 msgr2=0x7f4670103630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:53:06.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:06.182+0000 7f4678000700 1 -- 172.21.15.151:0/1226310543 shutdown_connections 2024-05-21T21:53:06.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:06.182+0000 7f4678000700 1 -- 172.21.15.151:0/1226310543 wait complete. 2024-05-21T21:53:06.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:06.183+0000 7f4678000700 1 Processor -- start 2024-05-21T21:53:06.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:06.183+0000 7f4678000700 1 -- start start 2024-05-21T21:53:06.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:06.184+0000 7f4678000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4670105760 0x7f467019bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:06.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:06.184+0000 7f4678000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4670106120 0x7f467019c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:06.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:06.184+0000 7f4678000700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f467010b760 0x7f46701a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:06.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:06.184+0000 7f4678000700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4670115990 con 0x7f4670106120 2024-05-21T21:53:06.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:06.184+0000 7f4678000700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4670115810 con 0x7f4670105760 2024-05-21T21:53:06.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:06.184+0000 7f4678000700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4670115b10 con 0x7f467010b760 2024-05-21T21:53:06.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:06.184+0000 7f4674c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4670106120 0x7f467019c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:06.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:06.184+0000 7f4674c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4670106120 0x7f467019c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:60872/0 (socket says 172.21.15.151:60872) 2024-05-21T21:53:06.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:06.184+0000 7f4674c00700 1 -- 172.21.15.151:0/379362741 learned_addr learned my addr 172.21.15.151:0/379362741 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:53:06.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:06.184+0000 7f4675600700 1 --2- 172.21.15.151:0/379362741 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4670105760 0x7f467019bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:06.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:06.184+0000 7f4676000700 1 --2- 172.21.15.151:0/379362741 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f467010b760 0x7f46701a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:06.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:06.184+0000 7f4674c00700 1 -- 172.21.15.151:0/379362741 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f467010b760 msgr2=0x7f46701a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:06.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:06.184+0000 7f4674c00700 1 --2- 172.21.15.151:0/379362741 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f467010b760 0x7f46701a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:06.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:06.184+0000 7f4674c00700 1 -- 172.21.15.151:0/379362741 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4670105760 msgr2=0x7f467019bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:06.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:06.184+0000 7f4674c00700 1 --2- 172.21.15.151:0/379362741 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4670105760 0x7f467019bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:06.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:06.185+0000 7f4674c00700 1 -- 172.21.15.151:0/379362741 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46701a0df0 con 0x7f4670106120 2024-05-21T21:53:06.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:06.185+0000 7f4675600700 1 --2- 172.21.15.151:0/379362741 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4670105760 0x7f467019bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:53:06.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:06.185+0000 7f4674c00700 1 --2- 172.21.15.151:0/379362741 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4670106120 0x7f467019c320 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f465c00e9a0 tx=0x7f465c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:53:06.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:06.185+0000 7f466e000700 1 -- 172.21.15.151:0/379362741 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f465c00ca10 con 0x7f4670106120 2024-05-21T21:53:06.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:06.185+0000 7f466e000700 1 -- 172.21.15.151:0/379362741 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f465c00cbb0 con 0x7f4670106120 2024-05-21T21:53:06.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:06.185+0000 7f4678000700 1 -- 172.21.15.151:0/379362741 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46701a10e0 con 0x7f4670106120 2024-05-21T21:53:06.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:06.185+0000 7f466e000700 1 -- 172.21.15.151:0/379362741 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f465c018680 con 0x7f4670106120 2024-05-21T21:53:06.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:06.185+0000 7f4678000700 1 -- 172.21.15.151:0/379362741 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46701a8960 con 0x7f4670106120 2024-05-21T21:53:06.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:06.186+0000 7f4678000700 1 -- 172.21.15.151:0/379362741 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4670063cd0 con 0x7f4670106120 2024-05-21T21:53:06.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:06.189+0000 7f466e000700 1 -- 172.21.15.151:0/379362741 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f465c010430 con 0x7f4670106120 2024-05-21T21:53:06.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:06.190+0000 7f466e000700 1 --2- 172.21.15.151:0/379362741 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f46580b2910 0x7f46580b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:06.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:06.191+0000 7f466e000700 1 -- 172.21.15.151:0/379362741 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f465c0e2b60 con 0x7f4670106120 2024-05-21T21:53:06.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:06.191+0000 7f4675600700 1 --2- 172.21.15.151:0/379362741 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f46580b2910 0x7f46580b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:06.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:06.191+0000 7f466e000700 1 --2- 172.21.15.151:0/379362741 >> [v2:172.21.15.151:6810/556594400,v1:172.21.15.151:6811/556594400] conn(0x7f46580bc4d0 0x7f46580be970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:06.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:06.191+0000 7f466e000700 1 -- 172.21.15.151:0/379362741 --> [v2:172.21.15.151:6810/556594400,v1:172.21.15.151:6811/556594400] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f465c024e70 con 0x7f46580bc4d0 2024-05-21T21:53:06.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:06.191+0000 7f466e000700 1 -- 172.21.15.151:0/379362741 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f465c0e2f10 con 0x7f4670106120 2024-05-21T21:53:06.193 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:06.191+0000 7f4676000700 1 --2- 172.21.15.151:0/379362741 >> [v2:172.21.15.151:6810/556594400,v1:172.21.15.151:6811/556594400] conn(0x7f46580bc4d0 0x7f46580be970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:06.193 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:06.191+0000 7f4675600700 1 --2- 172.21.15.151:0/379362741 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f46580b2910 0x7f46580b4dd0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4664009680 tx=0x7f4664009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:53:06.193 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:06.192+0000 7f4676000700 1 --2- 172.21.15.151:0/379362741 >> [v2:172.21.15.151:6810/556594400,v1:172.21.15.151:6811/556594400] conn(0x7f46580bc4d0 0x7f46580be970 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:53:06.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:06.193+0000 7f466e000700 1 -- 172.21.15.151:0/379362741 <== osd.3 v2:172.21.15.151:6810/556594400 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f465c024e70 con 0x7f46580bc4d0 2024-05-21T21:53:06.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:06.211+0000 7f4678000700 1 -- 172.21.15.151:0/379362741 --> [v2:172.21.15.151:6810/556594400,v1:172.21.15.151:6811/556594400] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4670064150 con 0x7f46580bc4d0 2024-05-21T21:53:06.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:06.211+0000 7f466e000700 1 -- 172.21.15.151:0/379362741 <== osd.3 v2:172.21.15.151:6810/556594400 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4670064150 con 0x7f46580bc4d0 2024-05-21T21:53:06.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:06.211+0000 7f4678000700 1 -- 172.21.15.151:0/379362741 >> [v2:172.21.15.151:6810/556594400,v1:172.21.15.151:6811/556594400] conn(0x7f46580bc4d0 msgr2=0x7f46580be970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:06.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:06.211+0000 7f4678000700 1 --2- 172.21.15.151:0/379362741 >> [v2:172.21.15.151:6810/556594400,v1:172.21.15.151:6811/556594400] conn(0x7f46580bc4d0 0x7f46580be970 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:06.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:06.212+0000 7f4678000700 1 -- 172.21.15.151:0/379362741 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f46580b2910 msgr2=0x7f46580b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:06.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:06.212+0000 7f4678000700 1 --2- 172.21.15.151:0/379362741 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f46580b2910 0x7f46580b4dd0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4664009680 tx=0x7f4664009260 comp rx=0 tx=0).stop 2024-05-21T21:53:06.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:06.212+0000 7f4678000700 1 -- 172.21.15.151:0/379362741 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4670106120 msgr2=0x7f467019c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:06.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:06.212+0000 7f4678000700 1 --2- 172.21.15.151:0/379362741 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4670106120 0x7f467019c320 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f465c00e9a0 tx=0x7f465c00ecb0 comp rx=0 tx=0).stop 2024-05-21T21:53:06.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:06.212+0000 7f4676000700 1 -- 172.21.15.151:0/379362741 reap_dead start 2024-05-21T21:53:06.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:06.212+0000 7f4678000700 1 -- 172.21.15.151:0/379362741 shutdown_connections 2024-05-21T21:53:06.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:06.212+0000 7f4678000700 1 -- 172.21.15.151:0/379362741 >> 172.21.15.151:0/379362741 conn(0x7f4670101210 msgr2=0x7f4670101750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:53:06.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:06.212+0000 7f4678000700 1 -- 172.21.15.151:0/379362741 shutdown_connections 2024-05-21T21:53:06.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:06.212+0000 7f4678000700 1 -- 172.21.15.151:0/379362741 wait complete. 2024-05-21T21:53:06.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:53:06 smithi151 ceph-mon[31623]: pgmap v370: 97 pgs: 97 active+clean; 642 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-21T21:53:06.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:53:06 smithi184 ceph-mon[36942]: pgmap v370: 97 pgs: 97 active+clean; 642 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-21T21:53:06.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:53:06 smithi164 ceph-mon[36269]: pgmap v370: 97 pgs: 97 active+clean; 642 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-21T21:53:07.233 INFO:teuthology.orchestra.run.smithi151.stdout:163208757319 2024-05-21T21:53:07.234 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-05-21T21:53:07.613 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:07.610+0000 7f8fe8c00700 1 -- 172.21.15.151:0/1444198739 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8fe4075770 msgr2=0x7f8fe4075b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:07.613 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:07.610+0000 7f8fe8c00700 1 --2- 172.21.15.151:0/1444198739 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8fe4075770 0x7f8fe4075b50 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8fd400b0a0 tx=0x7f8fd400b3b0 comp rx=0 tx=0).stop 2024-05-21T21:53:07.613 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:07.610+0000 7f8fe8c00700 1 -- 172.21.15.151:0/1444198739 shutdown_connections 2024-05-21T21:53:07.613 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:07.610+0000 7f8fe8c00700 1 --2- 172.21.15.151:0/1444198739 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8fe410ffa0 0x7f8fe4112390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:07.613 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:07.610+0000 7f8fe8c00700 1 --2- 172.21.15.151:0/1444198739 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8fe4075770 0x7f8fe4075b50 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:07.613 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:07.610+0000 7f8fe8c00700 1 --2- 172.21.15.151:0/1444198739 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8fe4076090 0x7f8fe410fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:07.613 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:07.610+0000 7f8fe8c00700 1 -- 172.21.15.151:0/1444198739 >> 172.21.15.151:0/1444198739 conn(0x7f8fe407a6a0 msgr2=0x7f8fe407aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:53:07.613 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:07.610+0000 7f8fe8c00700 1 -- 172.21.15.151:0/1444198739 shutdown_connections 2024-05-21T21:53:07.613 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:07.610+0000 7f8fe8c00700 1 -- 172.21.15.151:0/1444198739 wait complete. 2024-05-21T21:53:07.613 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:07.611+0000 7f8fe8c00700 1 Processor -- start 2024-05-21T21:53:07.613 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:07.612+0000 7f8fe8c00700 1 -- start start 2024-05-21T21:53:07.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:07.612+0000 7f8fe8c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8fe4075770 0x7f8fe419bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:07.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:07.612+0000 7f8fe8c00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8fe4076090 0x7f8fe419c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:07.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:07.612+0000 7f8fe8c00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8fe410ffa0 0x7f8fe41a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:07.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:07.612+0000 7f8fe8c00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe4115980 con 0x7f8fe4075770 2024-05-21T21:53:07.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:07.612+0000 7f8fe8c00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe4115800 con 0x7f8fe410ffa0 2024-05-21T21:53:07.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:07.612+0000 7f8fe8c00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe4115b00 con 0x7f8fe4076090 2024-05-21T21:53:07.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:07.612+0000 7f8fe1e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8fe4075770 0x7f8fe419bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:07.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:07.612+0000 7f8fe1e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8fe4075770 0x7f8fe419bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:60890/0 (socket says 172.21.15.151:60890) 2024-05-21T21:53:07.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:07.612+0000 7f8fe1e00700 1 -- 172.21.15.151:0/3736944492 learned_addr learned my addr 172.21.15.151:0/3736944492 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:53:07.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:07.612+0000 7f8fe1400700 1 --2- 172.21.15.151:0/3736944492 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8fe4076090 0x7f8fe419c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:07.616 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:07.612+0000 7f8fe2800700 1 --2- 172.21.15.151:0/3736944492 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8fe410ffa0 0x7f8fe41a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:07.616 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:07.613+0000 7f8fe1e00700 1 -- 172.21.15.151:0/3736944492 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8fe4076090 msgr2=0x7f8fe419c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:07.616 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:07.613+0000 7f8fe1e00700 1 --2- 172.21.15.151:0/3736944492 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8fe4076090 0x7f8fe419c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:07.616 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:07.613+0000 7f8fe1e00700 1 -- 172.21.15.151:0/3736944492 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8fe410ffa0 msgr2=0x7f8fe41a06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:07.616 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:07.613+0000 7f8fe1e00700 1 --2- 172.21.15.151:0/3736944492 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8fe410ffa0 0x7f8fe41a06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:07.616 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:07.613+0000 7f8fe1e00700 1 -- 172.21.15.151:0/3736944492 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fe41a0de0 con 0x7f8fe4075770 2024-05-21T21:53:07.616 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:07.613+0000 7f8fe2800700 1 --2- 172.21.15.151:0/3736944492 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8fe410ffa0 0x7f8fe41a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T21:53:07.616 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:07.613+0000 7f8fe1e00700 1 --2- 172.21.15.151:0/3736944492 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8fe4075770 0x7f8fe419bdd0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8fd40062a0 tx=0x7f8fd4004e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:53:07.616 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:07.613+0000 7f8fdaa00700 1 -- 172.21.15.151:0/3736944492 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd4021070 con 0x7f8fe4075770 2024-05-21T21:53:07.616 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:07.613+0000 7f8fdaa00700 1 -- 172.21.15.151:0/3736944492 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fd4007cc0 con 0x7f8fe4075770 2024-05-21T21:53:07.617 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:07.613+0000 7f8fe8c00700 1 -- 172.21.15.151:0/3736944492 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fe41a1070 con 0x7f8fe4075770 2024-05-21T21:53:07.617 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:07.613+0000 7f8fdaa00700 1 -- 172.21.15.151:0/3736944492 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd4004700 con 0x7f8fe4075770 2024-05-21T21:53:07.617 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:07.614+0000 7f8fe8c00700 1 -- 172.21.15.151:0/3736944492 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fe41a8950 con 0x7f8fe4075770 2024-05-21T21:53:07.617 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:07.615+0000 7f8fe8c00700 1 -- 172.21.15.151:0/3736944492 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8fe4063cd0 con 0x7f8fe4075770 2024-05-21T21:53:07.618 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:07.617+0000 7f8fdaa00700 1 -- 172.21.15.151:0/3736944492 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f8fd4003a00 con 0x7f8fe4075770 2024-05-21T21:53:07.619 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:07.618+0000 7f8fdaa00700 1 --2- 172.21.15.151:0/3736944492 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f8fc80b2990 0x7f8fc80b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:07.620 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:07.619+0000 7f8fdaa00700 1 -- 172.21.15.151:0/3736944492 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f8fd400f020 con 0x7f8fe4075770 2024-05-21T21:53:07.620 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:07.619+0000 7f8fe1400700 1 --2- 172.21.15.151:0/3736944492 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f8fc80b2990 0x7f8fc80b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:07.620 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:07.619+0000 7f8fdaa00700 1 --2- 172.21.15.151:0/3736944492 >> [v2:172.21.15.164:6800/3900240447,v1:172.21.15.164:6801/3900240447] conn(0x7f8fc80bc550 0x7f8fc80be9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:07.620 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:07.619+0000 7f8fdaa00700 1 -- 172.21.15.151:0/3736944492 --> [v2:172.21.15.164:6800/3900240447,v1:172.21.15.164:6801/3900240447] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8fd40048a0 con 0x7f8fc80bc550 2024-05-21T21:53:07.620 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:07.619+0000 7f8fdaa00700 1 -- 172.21.15.151:0/3736944492 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f8fd40e3e80 con 0x7f8fe4075770 2024-05-21T21:53:07.620 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:07.619+0000 7f8fe2800700 1 --2- 172.21.15.151:0/3736944492 >> [v2:172.21.15.164:6800/3900240447,v1:172.21.15.164:6801/3900240447] conn(0x7f8fc80bc550 0x7f8fc80be9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:07.621 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:07.619+0000 7f8fe1400700 1 --2- 172.21.15.151:0/3736944492 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f8fc80b2990 0x7f8fc80b4e50 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4111b80 tx=0x7f8fd00073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:53:07.621 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:07.620+0000 7f8fe2800700 1 --2- 172.21.15.151:0/3736944492 >> [v2:172.21.15.164:6800/3900240447,v1:172.21.15.164:6801/3900240447] conn(0x7f8fc80bc550 0x7f8fc80be9f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:53:07.622 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:07.621+0000 7f8fdaa00700 1 -- 172.21.15.151:0/3736944492 <== osd.1 v2:172.21.15.164:6800/3900240447 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8fd40048a0 con 0x7f8fc80bc550 2024-05-21T21:53:07.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:07.639+0000 7f8fe8c00700 1 -- 172.21.15.151:0/3736944492 --> [v2:172.21.15.164:6800/3900240447,v1:172.21.15.164:6801/3900240447] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8fe4064100 con 0x7f8fc80bc550 2024-05-21T21:53:07.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:07.639+0000 7f8fdaa00700 1 -- 172.21.15.151:0/3736944492 <== osd.1 v2:172.21.15.164:6800/3900240447 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f8fe4064100 con 0x7f8fc80bc550 2024-05-21T21:53:07.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:07.640+0000 7f8fe8c00700 1 -- 172.21.15.151:0/3736944492 >> [v2:172.21.15.164:6800/3900240447,v1:172.21.15.164:6801/3900240447] conn(0x7f8fc80bc550 msgr2=0x7f8fc80be9f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:07.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:07.640+0000 7f8fe8c00700 1 --2- 172.21.15.151:0/3736944492 >> [v2:172.21.15.164:6800/3900240447,v1:172.21.15.164:6801/3900240447] conn(0x7f8fc80bc550 0x7f8fc80be9f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:07.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:07.640+0000 7f8fe8c00700 1 -- 172.21.15.151:0/3736944492 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f8fc80b2990 msgr2=0x7f8fc80b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:07.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:07.640+0000 7f8fe8c00700 1 --2- 172.21.15.151:0/3736944492 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f8fc80b2990 0x7f8fc80b4e50 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4111b80 tx=0x7f8fd00073d0 comp rx=0 tx=0).stop 2024-05-21T21:53:07.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:07.640+0000 7f8fe8c00700 1 -- 172.21.15.151:0/3736944492 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8fe4075770 msgr2=0x7f8fe419bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:07.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:07.640+0000 7f8fe8c00700 1 --2- 172.21.15.151:0/3736944492 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8fe4075770 0x7f8fe419bdd0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8fd40062a0 tx=0x7f8fd4004e60 comp rx=0 tx=0).stop 2024-05-21T21:53:07.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:07.640+0000 7f8fe2800700 1 -- 172.21.15.151:0/3736944492 reap_dead start 2024-05-21T21:53:07.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:07.640+0000 7f8fe8c00700 1 -- 172.21.15.151:0/3736944492 shutdown_connections 2024-05-21T21:53:07.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:07.640+0000 7f8fe8c00700 1 -- 172.21.15.151:0/3736944492 >> 172.21.15.151:0/3736944492 conn(0x7f8fe407a6a0 msgr2=0x7f8fe4100770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:53:07.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:07.640+0000 7f8fe8c00700 1 -- 172.21.15.151:0/3736944492 shutdown_connections 2024-05-21T21:53:07.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:07.640+0000 7f8fe8c00700 1 -- 172.21.15.151:0/3736944492 wait complete. 2024-05-21T21:53:08.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:53:08 smithi151 ceph-mon[31623]: pgmap v371: 97 pgs: 97 active+clean; 642 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-21T21:53:08.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:53:08 smithi184 ceph-mon[36942]: pgmap v371: 97 pgs: 97 active+clean; 642 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-21T21:53:08.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:53:08 smithi164 ceph-mon[36269]: pgmap v371: 97 pgs: 97 active+clean; 642 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-21T21:53:08.948 INFO:teuthology.orchestra.run.smithi151.stdout:137438953549 2024-05-21T21:53:08.949 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-05-21T21:53:09.792 INFO:teuthology.orchestra.run.smithi151.stdout:55834574942 2024-05-21T21:53:09.792 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-05-21T21:53:10.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:10.026+0000 7f7f74c00700 1 -- 172.21.15.151:0/1123213760 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7f70076200 msgr2=0x7f7f70113f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:10.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:10.026+0000 7f7f74c00700 1 --2- 172.21.15.151:0/1123213760 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7f70076200 0x7f7f70113f40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c009990 tx=0x7f7f5c009ca0 comp rx=0 tx=0).stop 2024-05-21T21:53:10.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:10.026+0000 7f7f74c00700 1 -- 172.21.15.151:0/1123213760 shutdown_connections 2024-05-21T21:53:10.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:10.026+0000 7f7f74c00700 1 --2- 172.21.15.151:0/1123213760 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7f70114480 0x7f7f70116870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:10.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:10.026+0000 7f7f74c00700 1 --2- 172.21.15.151:0/1123213760 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7f700758e0 0x7f7f70075cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:10.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:10.026+0000 7f7f74c00700 1 --2- 172.21.15.151:0/1123213760 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7f70076200 0x7f7f70113f40 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:10.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:10.026+0000 7f7f74c00700 1 -- 172.21.15.151:0/1123213760 >> 172.21.15.151:0/1123213760 conn(0x7f7f700fefc0 msgr2=0x7f7f701013e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:53:10.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:10.026+0000 7f7f74c00700 1 -- 172.21.15.151:0/1123213760 shutdown_connections 2024-05-21T21:53:10.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:10.027+0000 7f7f74c00700 1 -- 172.21.15.151:0/1123213760 wait complete. 2024-05-21T21:53:10.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:10.028+0000 7f7f74c00700 1 Processor -- start 2024-05-21T21:53:10.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:10.028+0000 7f7f74c00700 1 -- start start 2024-05-21T21:53:10.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:10.029+0000 7f7f74c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7f700758e0 0x7f7f701a01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:10.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:10.029+0000 7f7f74c00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7f70076200 0x7f7f701a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:10.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:10.029+0000 7f7f74c00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7f70114480 0x7f7f701a4a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:10.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:10.029+0000 7f7f74c00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7f70119cb0 con 0x7f7f700758e0 2024-05-21T21:53:10.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:10.029+0000 7f7f74c00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7f70119b30 con 0x7f7f70076200 2024-05-21T21:53:10.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:10.029+0000 7f7f74c00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7f70119e30 con 0x7f7f70114480 2024-05-21T21:53:10.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:10.029+0000 7f7f6ea00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7f70076200 0x7f7f701a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:10.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:10.029+0000 7f7f6ea00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7f70076200 0x7f7f701a06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:47806/0 (socket says 172.21.15.151:47806) 2024-05-21T21:53:10.061 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:10.029+0000 7f7f6ea00700 1 -- 172.21.15.151:0/2174458572 learned_addr learned my addr 172.21.15.151:0/2174458572 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:53:10.061 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:10.029+0000 7f7f6fe00700 1 --2- 172.21.15.151:0/2174458572 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7f70114480 0x7f7f701a4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:10.061 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:10.030+0000 7f7f6ea00700 1 -- 172.21.15.151:0/2174458572 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7f70114480 msgr2=0x7f7f701a4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:10.061 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:10.030+0000 7f7f6ea00700 1 --2- 172.21.15.151:0/2174458572 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7f70114480 0x7f7f701a4a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:10.061 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:10.030+0000 7f7f6f400700 1 --2- 172.21.15.151:0/2174458572 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7f700758e0 0x7f7f701a01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:10.061 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:10.030+0000 7f7f6ea00700 1 -- 172.21.15.151:0/2174458572 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7f700758e0 msgr2=0x7f7f701a01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:10.061 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:10.030+0000 7f7f6ea00700 1 --2- 172.21.15.151:0/2174458572 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7f700758e0 0x7f7f701a01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:10.061 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:10.030+0000 7f7f6ea00700 1 -- 172.21.15.151:0/2174458572 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f701a51b0 con 0x7f7f70076200 2024-05-21T21:53:10.061 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:10.030+0000 7f7f6f400700 1 --2- 172.21.15.151:0/2174458572 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7f700758e0 0x7f7f701a01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T21:53:10.061 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:10.030+0000 7f7f6ea00700 1 --2- 172.21.15.151:0/2174458572 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7f70076200 0x7f7f701a06e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c000c00 tx=0x7f7f5c0057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:53:10.061 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:10.030+0000 7f7f67e00700 1 -- 172.21.15.151:0/2174458572 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c027070 con 0x7f7f70076200 2024-05-21T21:53:10.062 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:10.030+0000 7f7f74c00700 1 -- 172.21.15.151:0/2174458572 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f701a5440 con 0x7f7f70076200 2024-05-21T21:53:10.062 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:10.031+0000 7f7f67e00700 1 -- 172.21.15.151:0/2174458572 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f5c022470 con 0x7f7f70076200 2024-05-21T21:53:10.062 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:10.031+0000 7f7f67e00700 1 -- 172.21.15.151:0/2174458572 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c017650 con 0x7f7f70076200 2024-05-21T21:53:10.062 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:10.031+0000 7f7f74c00700 1 -- 172.21.15.151:0/2174458572 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f701acd20 con 0x7f7f70076200 2024-05-21T21:53:10.062 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:10.031+0000 7f7f74c00700 1 -- 172.21.15.151:0/2174458572 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7f70063cd0 con 0x7f7f70076200 2024-05-21T21:53:10.062 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:10.036+0000 7f7f67e00700 1 -- 172.21.15.151:0/2174458572 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 17) ==== 149704+0+0 (secure 0 0 0) 0x7f7f5c02a070 con 0x7f7f70076200 2024-05-21T21:53:10.062 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:10.037+0000 7f7f67e00700 1 --2- 172.21.15.151:0/2174458572 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f7f540b2910 0x7f7f540b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:10.062 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:10.038+0000 7f7f67e00700 1 -- 172.21.15.151:0/2174458572 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f7f5c0e3a40 con 0x7f7f70076200 2024-05-21T21:53:10.062 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:10.038+0000 7f7f6f400700 1 --2- 172.21.15.151:0/2174458572 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f7f540b2910 0x7f7f540b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:10.062 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:10.038+0000 7f7f67e00700 1 --2- 172.21.15.151:0/2174458572 >> [v2:172.21.15.151:6802/2420400473,v1:172.21.15.151:6803/2420400473] conn(0x7f7f540bc4d0 0x7f7f540be970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:10.062 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:10.038+0000 7f7f67e00700 1 -- 172.21.15.151:0/2174458572 --> [v2:172.21.15.151:6802/2420400473,v1:172.21.15.151:6803/2420400473] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7f540bf040 con 0x7f7f540bc4d0 2024-05-21T21:53:10.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:10.038+0000 7f7f67e00700 1 -- 172.21.15.151:0/2174458572 <== mon.2 v2:172.21.15.164:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f7f5c0e3df0 con 0x7f7f70076200 2024-05-21T21:53:10.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:10.039+0000 7f7f6fe00700 1 --2- 172.21.15.151:0/2174458572 >> [v2:172.21.15.151:6802/2420400473,v1:172.21.15.151:6803/2420400473] conn(0x7f7f540bc4d0 0x7f7f540be970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:10.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:10.039+0000 7f7f6f400700 1 --2- 172.21.15.151:0/2174458572 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f7f540b2910 0x7f7f540b4dd0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7f60009be0 tx=0x7f7f60009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:53:10.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:10.039+0000 7f7f6fe00700 1 --2- 172.21.15.151:0/2174458572 >> [v2:172.21.15.151:6802/2420400473,v1:172.21.15.151:6803/2420400473] conn(0x7f7f540bc4d0 0x7f7f540be970 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:53:10.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:10.040+0000 7f7f67e00700 1 -- 172.21.15.151:0/2174458572 <== osd.0 v2:172.21.15.151:6802/2420400473 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7f540bf040 con 0x7f7f540bc4d0 2024-05-21T21:53:10.064 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:10.060+0000 7f7f74c00700 1 -- 172.21.15.151:0/2174458572 --> [v2:172.21.15.151:6802/2420400473,v1:172.21.15.151:6803/2420400473] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7f700640d0 con 0x7f7f540bc4d0 2024-05-21T21:53:10.064 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:10.061+0000 7f7f67e00700 1 -- 172.21.15.151:0/2174458572 <== osd.0 v2:172.21.15.151:6802/2420400473 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f7f700640d0 con 0x7f7f540bc4d0 2024-05-21T21:53:10.065 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:10.062+0000 7f7f74c00700 1 -- 172.21.15.151:0/2174458572 >> [v2:172.21.15.151:6802/2420400473,v1:172.21.15.151:6803/2420400473] conn(0x7f7f540bc4d0 msgr2=0x7f7f540be970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:10.065 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:10.062+0000 7f7f74c00700 1 --2- 172.21.15.151:0/2174458572 >> [v2:172.21.15.151:6802/2420400473,v1:172.21.15.151:6803/2420400473] conn(0x7f7f540bc4d0 0x7f7f540be970 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:10.065 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:10.062+0000 7f7f74c00700 1 -- 172.21.15.151:0/2174458572 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f7f540b2910 msgr2=0x7f7f540b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:10.065 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:10.062+0000 7f7f74c00700 1 --2- 172.21.15.151:0/2174458572 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f7f540b2910 0x7f7f540b4dd0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7f60009be0 tx=0x7f7f60009260 comp rx=0 tx=0).stop 2024-05-21T21:53:10.065 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:10.062+0000 7f7f74c00700 1 -- 172.21.15.151:0/2174458572 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7f70076200 msgr2=0x7f7f701a06e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:10.065 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:10.062+0000 7f7f74c00700 1 --2- 172.21.15.151:0/2174458572 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7f70076200 0x7f7f701a06e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c000c00 tx=0x7f7f5c0057c0 comp rx=0 tx=0).stop 2024-05-21T21:53:10.065 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:10.062+0000 7f7f6fe00700 1 -- 172.21.15.151:0/2174458572 reap_dead start 2024-05-21T21:53:10.065 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:10.062+0000 7f7f74c00700 1 -- 172.21.15.151:0/2174458572 shutdown_connections 2024-05-21T21:53:10.065 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:10.062+0000 7f7f74c00700 1 -- 172.21.15.151:0/2174458572 >> 172.21.15.151:0/2174458572 conn(0x7f7f700fefc0 msgr2=0x7f7f70100820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:53:10.065 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:10.063+0000 7f7f74c00700 1 -- 172.21.15.151:0/2174458572 shutdown_connections 2024-05-21T21:53:10.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:10.063+0000 7f7f74c00700 1 -- 172.21.15.151:0/2174458572 wait complete. 2024-05-21T21:53:10.561 INFO:teuthology.orchestra.run.smithi151.stdout:111669149778 2024-05-21T21:53:10.561 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-05-21T21:53:10.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:53:10 smithi151 ceph-mon[31623]: pgmap v372: 97 pgs: 97 active+clean; 642 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-21T21:53:10.735 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:53:10 smithi184 ceph-mon[36942]: pgmap v372: 97 pgs: 97 active+clean; 642 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-21T21:53:10.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:53:10 smithi164 ceph-mon[36269]: pgmap v372: 97 pgs: 97 active+clean; 642 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-21T21:53:11.233 INFO:teuthology.orchestra.run.smithi151.stdout:34359738468 2024-05-21T21:53:11.233 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-05-21T21:53:11.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:53:11 smithi151 ceph-mon[31623]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-05-21T21:53:11.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:53:11 smithi184 ceph-mon[36942]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-05-21T21:53:11.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:53:11 smithi164 ceph-mon[36269]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-05-21T21:53:12.026 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.025+0000 7f918fc00700 1 -- 172.21.15.151:0/1364060177 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f918810ef00 msgr2=0x7f9188116a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:12.026 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.025+0000 7f918fc00700 1 --2- 172.21.15.151:0/1364060177 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f918810ef00 0x7f9188116a90 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9180009fc0 tx=0x7f918000b330 comp rx=0 tx=0).stop 2024-05-21T21:53:12.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.025+0000 7f918fc00700 1 -- 172.21.15.151:0/1364060177 shutdown_connections 2024-05-21T21:53:12.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.025+0000 7f918fc00700 1 --2- 172.21.15.151:0/1364060177 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f918810ef00 0x7f9188116a90 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:12.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.025+0000 7f918fc00700 1 --2- 172.21.15.151:0/1364060177 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f918810a5e0 0x7f918810a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:12.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.025+0000 7f918fc00700 1 --2- 172.21.15.151:0/1364060177 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f918810af90 0x7f918810e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:12.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.025+0000 7f918fc00700 1 -- 172.21.15.151:0/1364060177 >> 172.21.15.151:0/1364060177 conn(0x7f9188079410 msgr2=0x7f918807b830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:53:12.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.026+0000 7f918fc00700 1 -- 172.21.15.151:0/1364060177 shutdown_connections 2024-05-21T21:53:12.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.026+0000 7f918fc00700 1 -- 172.21.15.151:0/1364060177 wait complete. 2024-05-21T21:53:12.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.027+0000 7f918fc00700 1 Processor -- start 2024-05-21T21:53:12.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.028+0000 7f918fc00700 1 -- start start 2024-05-21T21:53:12.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.028+0000 7f918fc00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f918810a5e0 0x7f91881a0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:12.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.028+0000 7f918fc00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f918810af90 0x7f91881a0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:12.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.028+0000 7f918fc00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f918810ef00 0x7f91881a4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:12.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.028+0000 7f918fc00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9188119db0 con 0x7f918810af90 2024-05-21T21:53:12.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.028+0000 7f918fc00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9188119c30 con 0x7f918810ef00 2024-05-21T21:53:12.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.028+0000 7f918fc00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9188119f30 con 0x7f918810a5e0 2024-05-21T21:53:12.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.028+0000 7f918d200700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f918810a5e0 0x7f91881a0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:12.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.028+0000 7f918d200700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f918810a5e0 0x7f91881a0220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:50260/0 (socket says 172.21.15.151:50260) 2024-05-21T21:53:12.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.028+0000 7f918d200700 1 -- 172.21.15.151:0/2199642616 learned_addr learned my addr 172.21.15.151:0/2199642616 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:53:12.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.029+0000 7f918dc00700 1 --2- 172.21.15.151:0/2199642616 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f918810ef00 0x7f91881a4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:12.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.029+0000 7f9187e00700 1 --2- 172.21.15.151:0/2199642616 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f918810af90 0x7f91881a0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:12.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.029+0000 7f918dc00700 1 -- 172.21.15.151:0/2199642616 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f918810a5e0 msgr2=0x7f91881a0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:12.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.029+0000 7f918dc00700 1 --2- 172.21.15.151:0/2199642616 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f918810a5e0 0x7f91881a0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:12.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.029+0000 7f918dc00700 1 -- 172.21.15.151:0/2199642616 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f918810af90 msgr2=0x7f91881a0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:12.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.029+0000 7f918dc00700 1 --2- 172.21.15.151:0/2199642616 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f918810af90 0x7f91881a0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:12.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.029+0000 7f918dc00700 1 -- 172.21.15.151:0/2199642616 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91881a51e0 con 0x7f918810ef00 2024-05-21T21:53:12.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.029+0000 7f9187e00700 1 --2- 172.21.15.151:0/2199642616 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f918810af90 0x7f91881a0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T21:53:12.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.030+0000 7f918dc00700 1 --2- 172.21.15.151:0/2199642616 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f918810ef00 0x7f91881a4aa0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9180015000 tx=0x7f918000cc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:53:12.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.030+0000 7f9185600700 1 -- 172.21.15.151:0/2199642616 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9180021070 con 0x7f918810ef00 2024-05-21T21:53:12.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.030+0000 7f918fc00700 1 -- 172.21.15.151:0/2199642616 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91881a5470 con 0x7f918810ef00 2024-05-21T21:53:12.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.030+0000 7f9185600700 1 -- 172.21.15.151:0/2199642616 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9180007ac0 con 0x7f918810ef00 2024-05-21T21:53:12.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.030+0000 7f9185600700 1 -- 172.21.15.151:0/2199642616 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9180005040 con 0x7f918810ef00 2024-05-21T21:53:12.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.030+0000 7f918fc00700 1 -- 172.21.15.151:0/2199642616 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91881acd50 con 0x7f918810ef00 2024-05-21T21:53:12.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.031+0000 7f918fc00700 1 -- 172.21.15.151:0/2199642616 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9188063c90 con 0x7f918810ef00 2024-05-21T21:53:12.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.034+0000 7f9185600700 1 -- 172.21.15.151:0/2199642616 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f9180024070 con 0x7f918810ef00 2024-05-21T21:53:12.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.036+0000 7f9185600700 1 --2- 172.21.15.151:0/2199642616 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f91700b2940 0x7f91700b4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:12.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.036+0000 7f9185600700 1 -- 172.21.15.151:0/2199642616 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f91800e3410 con 0x7f918810ef00 2024-05-21T21:53:12.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.036+0000 7f918d200700 1 --2- 172.21.15.151:0/2199642616 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f91700b2940 0x7f91700b4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:12.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.036+0000 7f9185600700 1 --2- 172.21.15.151:0/2199642616 >> [v2:172.21.15.151:6826/337128533,v1:172.21.15.151:6827/337128533] conn(0x7f91700bc500 0x7f91700be9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:12.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.036+0000 7f9185600700 1 -- 172.21.15.151:0/2199642616 --> [v2:172.21.15.151:6826/337128533,v1:172.21.15.151:6827/337128533] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f91700bf070 con 0x7f91700bc500 2024-05-21T21:53:12.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.036+0000 7f9185600700 1 -- 172.21.15.151:0/2199642616 <== mon.2 v2:172.21.15.164:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f91800e37c0 con 0x7f918810ef00 2024-05-21T21:53:12.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.037+0000 7f9187e00700 1 --2- 172.21.15.151:0/2199642616 >> [v2:172.21.15.151:6826/337128533,v1:172.21.15.151:6827/337128533] conn(0x7f91700bc500 0x7f91700be9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:12.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.037+0000 7f918d200700 1 --2- 172.21.15.151:0/2199642616 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f91700b2940 0x7f91700b4e00 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f917c009d20 tx=0x7f917c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:53:12.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.037+0000 7f9187e00700 1 --2- 172.21.15.151:0/2199642616 >> [v2:172.21.15.151:6826/337128533,v1:172.21.15.151:6827/337128533] conn(0x7f91700bc500 0x7f91700be9a0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:53:12.040 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.039+0000 7f9185600700 1 -- 172.21.15.151:0/2199642616 <== osd.9 v2:172.21.15.151:6826/337128533 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f91700bf070 con 0x7f91700bc500 2024-05-21T21:53:12.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.058+0000 7f918fc00700 1 -- 172.21.15.151:0/2199642616 --> [v2:172.21.15.151:6826/337128533,v1:172.21.15.151:6827/337128533] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9188064110 con 0x7f91700bc500 2024-05-21T21:53:12.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.058+0000 7f9185600700 1 -- 172.21.15.151:0/2199642616 <== osd.9 v2:172.21.15.151:6826/337128533 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9188064110 con 0x7f91700bc500 2024-05-21T21:53:12.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.059+0000 7f918fc00700 1 -- 172.21.15.151:0/2199642616 >> [v2:172.21.15.151:6826/337128533,v1:172.21.15.151:6827/337128533] conn(0x7f91700bc500 msgr2=0x7f91700be9a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:12.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.059+0000 7f918fc00700 1 --2- 172.21.15.151:0/2199642616 >> [v2:172.21.15.151:6826/337128533,v1:172.21.15.151:6827/337128533] conn(0x7f91700bc500 0x7f91700be9a0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:12.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.059+0000 7f918fc00700 1 -- 172.21.15.151:0/2199642616 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f91700b2940 msgr2=0x7f91700b4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:12.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.059+0000 7f918fc00700 1 --2- 172.21.15.151:0/2199642616 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f91700b2940 0x7f91700b4e00 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f917c009d20 tx=0x7f917c009260 comp rx=0 tx=0).stop 2024-05-21T21:53:12.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.059+0000 7f918fc00700 1 -- 172.21.15.151:0/2199642616 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f918810ef00 msgr2=0x7f91881a4aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:12.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.059+0000 7f918fc00700 1 --2- 172.21.15.151:0/2199642616 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f918810ef00 0x7f91881a4aa0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9180015000 tx=0x7f918000cc30 comp rx=0 tx=0).stop 2024-05-21T21:53:12.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.059+0000 7f918dc00700 1 -- 172.21.15.151:0/2199642616 reap_dead start 2024-05-21T21:53:12.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.059+0000 7f918fc00700 1 -- 172.21.15.151:0/2199642616 shutdown_connections 2024-05-21T21:53:12.061 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.059+0000 7f918fc00700 1 -- 172.21.15.151:0/2199642616 >> 172.21.15.151:0/2199642616 conn(0x7f9188079410 msgr2=0x7f918810d690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:53:12.061 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.059+0000 7f918fc00700 1 -- 172.21.15.151:0/2199642616 shutdown_connections 2024-05-21T21:53:12.061 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.059+0000 7f918fc00700 1 -- 172.21.15.151:0/2199642616 wait complete. 2024-05-21T21:53:12.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.935+0000 7fdc87800700 1 -- 172.21.15.151:0/2054180418 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdc801088f0 msgr2=0x7fdc80110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:12.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.935+0000 7fdc87800700 1 --2- 172.21.15.151:0/2054180418 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdc801088f0 0x7fdc80110470 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fdc7800b0d0 tx=0x7fdc7800b3e0 comp rx=0 tx=0).stop 2024-05-21T21:53:12.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.936+0000 7fdc87800700 1 -- 172.21.15.151:0/2054180418 shutdown_connections 2024-05-21T21:53:12.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.936+0000 7fdc87800700 1 --2- 172.21.15.151:0/2054180418 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdc801088f0 0x7fdc80110470 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:12.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.936+0000 7fdc87800700 1 --2- 172.21.15.151:0/2054180418 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc801037f0 0x7fdc80103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:12.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.936+0000 7fdc87800700 1 --2- 172.21.15.151:0/2054180418 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdc801041a0 0x7fdc801081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:12.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.936+0000 7fdc87800700 1 -- 172.21.15.151:0/2054180418 >> 172.21.15.151:0/2054180418 conn(0x7fdc8007a6a0 msgr2=0x7fdc8007aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:53:12.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.936+0000 7fdc87800700 1 -- 172.21.15.151:0/2054180418 shutdown_connections 2024-05-21T21:53:12.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.936+0000 7fdc87800700 1 -- 172.21.15.151:0/2054180418 wait complete. 2024-05-21T21:53:12.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.937+0000 7fdc87800700 1 Processor -- start 2024-05-21T21:53:12.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.937+0000 7fdc87800700 1 -- start start 2024-05-21T21:53:12.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.937+0000 7fdc87800700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc801037f0 0x7fdc80075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:12.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.937+0000 7fdc87800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdc801041a0 0x7fdc80075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:12.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.937+0000 7fdc87800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdc801088f0 0x7fdc800762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:12.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.937+0000 7fdc87800700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdc80113710 con 0x7fdc801041a0 2024-05-21T21:53:12.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.937+0000 7fdc87800700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdc80113590 con 0x7fdc801088f0 2024-05-21T21:53:12.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.937+0000 7fdc87800700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fdc80113890 con 0x7fdc801037f0 2024-05-21T21:53:12.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.938+0000 7fdc7fe00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdc801041a0 0x7fdc80075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:12.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.938+0000 7fdc7fe00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdc801041a0 0x7fdc80075d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:60928/0 (socket says 172.21.15.151:60928) 2024-05-21T21:53:12.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.938+0000 7fdc84e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc801037f0 0x7fdc80075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:12.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.938+0000 7fdc7fe00700 1 -- 172.21.15.151:0/2840695807 learned_addr learned my addr 172.21.15.151:0/2840695807 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:53:12.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.938+0000 7fdc85800700 1 --2- 172.21.15.151:0/2840695807 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdc801088f0 0x7fdc800762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:12.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.938+0000 7fdc7fe00700 1 -- 172.21.15.151:0/2840695807 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc801037f0 msgr2=0x7fdc80075840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:12.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.938+0000 7fdc7fe00700 1 --2- 172.21.15.151:0/2840695807 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc801037f0 0x7fdc80075840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:12.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.938+0000 7fdc7fe00700 1 -- 172.21.15.151:0/2840695807 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdc801088f0 msgr2=0x7fdc800762c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:12.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.938+0000 7fdc7fe00700 1 --2- 172.21.15.151:0/2840695807 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdc801088f0 0x7fdc800762c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:12.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.938+0000 7fdc7fe00700 1 -- 172.21.15.151:0/2840695807 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc801a80b0 con 0x7fdc801041a0 2024-05-21T21:53:12.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.938+0000 7fdc85800700 1 --2- 172.21.15.151:0/2840695807 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdc801088f0 0x7fdc800762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:53:12.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.939+0000 7fdc7fe00700 1 --2- 172.21.15.151:0/2840695807 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdc801041a0 0x7fdc80075d80 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c00e9a0 tx=0x7fdc6c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:53:12.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.939+0000 7fdc7d600700 1 -- 172.21.15.151:0/2840695807 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc6c00ca10 con 0x7fdc801041a0 2024-05-21T21:53:12.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.939+0000 7fdc7d600700 1 -- 172.21.15.151:0/2840695807 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc6c00cbb0 con 0x7fdc801041a0 2024-05-21T21:53:12.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.939+0000 7fdc7d600700 1 -- 172.21.15.151:0/2840695807 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc6c018680 con 0x7fdc801041a0 2024-05-21T21:53:12.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.939+0000 7fdc87800700 1 -- 172.21.15.151:0/2840695807 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc801a83a0 con 0x7fdc801041a0 2024-05-21T21:53:12.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.939+0000 7fdc87800700 1 -- 172.21.15.151:0/2840695807 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc801a8890 con 0x7fdc801041a0 2024-05-21T21:53:12.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.941+0000 7fdc87800700 1 -- 172.21.15.151:0/2840695807 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdc80063cd0 con 0x7fdc801041a0 2024-05-21T21:53:12.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.943+0000 7fdc7d600700 1 -- 172.21.15.151:0/2840695807 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fdc6c010430 con 0x7fdc801041a0 2024-05-21T21:53:12.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.944+0000 7fdc7d600700 1 --2- 172.21.15.151:0/2840695807 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fdc680b2900 0x7fdc680b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:12.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.944+0000 7fdc7d600700 1 -- 172.21.15.151:0/2840695807 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fdc6c0e2290 con 0x7fdc801041a0 2024-05-21T21:53:12.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.945+0000 7fdc84e00700 1 --2- 172.21.15.151:0/2840695807 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fdc680b2900 0x7fdc680b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:12.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.945+0000 7fdc7d600700 1 --2- 172.21.15.151:0/2840695807 >> [v2:172.21.15.164:6816/3869696936,v1:172.21.15.164:6817/3869696936] conn(0x7fdc680bc4c0 0x7fdc680be960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:12.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.945+0000 7fdc7d600700 1 -- 172.21.15.151:0/2840695807 --> [v2:172.21.15.164:6816/3869696936,v1:172.21.15.164:6817/3869696936] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdc680bf030 con 0x7fdc680bc4c0 2024-05-21T21:53:12.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.945+0000 7fdc7d600700 1 -- 172.21.15.151:0/2840695807 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7fdc6c0e9050 con 0x7fdc801041a0 2024-05-21T21:53:12.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.945+0000 7fdc84e00700 1 --2- 172.21.15.151:0/2840695807 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fdc680b2900 0x7fdc680b4dc0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fdc74009c60 tx=0x7fdc74009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:53:12.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.945+0000 7fdc85800700 1 --2- 172.21.15.151:0/2840695807 >> [v2:172.21.15.164:6816/3869696936,v1:172.21.15.164:6817/3869696936] conn(0x7fdc680bc4c0 0x7fdc680be960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:12.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.946+0000 7fdc85800700 1 --2- 172.21.15.151:0/2840695807 >> [v2:172.21.15.164:6816/3869696936,v1:172.21.15.164:6817/3869696936] conn(0x7fdc680bc4c0 0x7fdc680be960 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:53:12.102 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.947+0000 7fdc7d600700 1 -- 172.21.15.151:0/2840695807 <== osd.7 v2:172.21.15.164:6816/3869696936 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdc680bf030 con 0x7fdc680bc4c0 2024-05-21T21:53:12.102 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.965+0000 7fdc87800700 1 -- 172.21.15.151:0/2840695807 --> [v2:172.21.15.164:6816/3869696936,v1:172.21.15.164:6817/3869696936] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fdc800641a0 con 0x7fdc680bc4c0 2024-05-21T21:53:12.102 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.966+0000 7fdc7d600700 1 -- 172.21.15.151:0/2840695807 <== osd.7 v2:172.21.15.164:6816/3869696936 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fdc800641a0 con 0x7fdc680bc4c0 2024-05-21T21:53:12.102 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.966+0000 7fdc87800700 1 -- 172.21.15.151:0/2840695807 >> [v2:172.21.15.164:6816/3869696936,v1:172.21.15.164:6817/3869696936] conn(0x7fdc680bc4c0 msgr2=0x7fdc680be960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:12.102 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.966+0000 7fdc87800700 1 --2- 172.21.15.151:0/2840695807 >> [v2:172.21.15.164:6816/3869696936,v1:172.21.15.164:6817/3869696936] conn(0x7fdc680bc4c0 0x7fdc680be960 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:12.102 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.966+0000 7fdc87800700 1 -- 172.21.15.151:0/2840695807 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fdc680b2900 msgr2=0x7fdc680b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:12.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.966+0000 7fdc87800700 1 --2- 172.21.15.151:0/2840695807 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fdc680b2900 0x7fdc680b4dc0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fdc74009c60 tx=0x7fdc74009260 comp rx=0 tx=0).stop 2024-05-21T21:53:12.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.966+0000 7fdc87800700 1 -- 172.21.15.151:0/2840695807 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdc801041a0 msgr2=0x7fdc80075d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:12.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.966+0000 7fdc87800700 1 --2- 172.21.15.151:0/2840695807 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdc801041a0 0x7fdc80075d80 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c00e9a0 tx=0x7fdc6c00ecb0 comp rx=0 tx=0).stop 2024-05-21T21:53:12.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.966+0000 7fdc85800700 1 -- 172.21.15.151:0/2840695807 reap_dead start 2024-05-21T21:53:12.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.967+0000 7fdc87800700 1 -- 172.21.15.151:0/2840695807 shutdown_connections 2024-05-21T21:53:12.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.967+0000 7fdc87800700 1 -- 172.21.15.151:0/2840695807 >> 172.21.15.151:0/2840695807 conn(0x7fdc8007a6a0 msgr2=0x7fdc80101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:53:12.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.967+0000 7fdc87800700 1 -- 172.21.15.151:0/2840695807 shutdown_connections 2024-05-21T21:53:12.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.967+0000 7fdc87800700 1 -- 172.21.15.151:0/2840695807 wait complete. 2024-05-21T21:53:12.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.637+0000 7fae14800700 1 -- 172.21.15.151:0/1617731395 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fae0c109a00 msgr2=0x7fae0c110430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:12.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.637+0000 7fae14800700 1 --2- 172.21.15.151:0/1617731395 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fae0c109a00 0x7fae0c110430 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fae0800b0d0 tx=0x7fae0800b3e0 comp rx=0 tx=0).stop 2024-05-21T21:53:12.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.637+0000 7fae14800700 1 -- 172.21.15.151:0/1617731395 shutdown_connections 2024-05-21T21:53:12.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.637+0000 7fae14800700 1 --2- 172.21.15.151:0/1617731395 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fae0c101920 0x7fae0c101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:12.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.637+0000 7fae14800700 1 --2- 172.21.15.151:0/1617731395 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fae0c109a00 0x7fae0c110430 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:12.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.637+0000 7fae14800700 1 --2- 172.21.15.151:0/1617731395 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae0c1022d0 0x7fae0c109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:12.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.637+0000 7fae14800700 1 -- 172.21.15.151:0/1617731395 >> 172.21.15.151:0/1617731395 conn(0x7fae0c078640 msgr2=0x7fae0c0ff9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:53:12.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.637+0000 7fae14800700 1 -- 172.21.15.151:0/1617731395 shutdown_connections 2024-05-21T21:53:12.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.638+0000 7fae14800700 1 -- 172.21.15.151:0/1617731395 wait complete. 2024-05-21T21:53:12.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.639+0000 7fae14800700 1 Processor -- start 2024-05-21T21:53:12.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.639+0000 7fae14800700 1 -- start start 2024-05-21T21:53:12.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.639+0000 7fae14800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fae0c101920 0x7fae0c19bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:12.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.639+0000 7fae14800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fae0c1022d0 0x7fae0c19c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:12.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.639+0000 7fae14800700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae0c109a00 0x7fae0c1a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:12.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.639+0000 7fae14800700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fae0c113710 con 0x7fae0c1022d0 2024-05-21T21:53:12.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.639+0000 7fae14800700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fae0c113590 con 0x7fae0c101920 2024-05-21T21:53:12.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.639+0000 7fae14800700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fae0c113890 con 0x7fae0c109a00 2024-05-21T21:53:12.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.639+0000 7fae11400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fae0c1022d0 0x7fae0c19c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:12.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.640+0000 7fae11400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fae0c1022d0 0x7fae0c19c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:60958/0 (socket says 172.21.15.151:60958) 2024-05-21T21:53:12.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.640+0000 7fae11400700 1 -- 172.21.15.151:0/260116925 learned_addr learned my addr 172.21.15.151:0/260116925 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:53:12.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.640+0000 7fae11e00700 1 --2- 172.21.15.151:0/260116925 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fae0c101920 0x7fae0c19bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:12.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.640+0000 7fae12800700 1 --2- 172.21.15.151:0/260116925 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae0c109a00 0x7fae0c1a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:12.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.640+0000 7fae11400700 1 -- 172.21.15.151:0/260116925 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae0c109a00 msgr2=0x7fae0c1a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:12.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.640+0000 7fae11400700 1 --2- 172.21.15.151:0/260116925 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae0c109a00 0x7fae0c1a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:12.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.640+0000 7fae11400700 1 -- 172.21.15.151:0/260116925 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fae0c101920 msgr2=0x7fae0c19bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:12.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.640+0000 7fae11400700 1 --2- 172.21.15.151:0/260116925 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fae0c101920 0x7fae0c19bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:12.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.640+0000 7fae11400700 1 -- 172.21.15.151:0/260116925 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae0c1a0d90 con 0x7fae0c1022d0 2024-05-21T21:53:12.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.640+0000 7fae11400700 1 --2- 172.21.15.151:0/260116925 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fae0c1022d0 0x7fae0c19c2c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fadfc00e9f0 tx=0x7fadfc00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:53:12.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.641+0000 7fae06a00700 1 -- 172.21.15.151:0/260116925 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadfc004a30 con 0x7fae0c1022d0 2024-05-21T21:53:12.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.641+0000 7fae14800700 1 -- 172.21.15.151:0/260116925 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae0c1a1080 con 0x7fae0c1022d0 2024-05-21T21:53:12.643 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.641+0000 7fae06a00700 1 -- 172.21.15.151:0/260116925 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadfc00cc70 con 0x7fae0c1022d0 2024-05-21T21:53:12.643 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.641+0000 7fae06a00700 1 -- 172.21.15.151:0/260116925 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadfc005530 con 0x7fae0c1022d0 2024-05-21T21:53:12.643 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.641+0000 7fae14800700 1 -- 172.21.15.151:0/260116925 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae0c1a8900 con 0x7fae0c1022d0 2024-05-21T21:53:12.643 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.642+0000 7fae14800700 1 -- 172.21.15.151:0/260116925 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fae0c100ab0 con 0x7fae0c1022d0 2024-05-21T21:53:12.645 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.645+0000 7fae06a00700 1 -- 172.21.15.151:0/260116925 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fadfc004bd0 con 0x7fae0c1022d0 2024-05-21T21:53:12.647 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.647+0000 7fae06a00700 1 --2- 172.21.15.151:0/260116925 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fadf40b2900 0x7fadf40b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:12.647 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.647+0000 7fae06a00700 1 -- 172.21.15.151:0/260116925 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fadfc0e2760 con 0x7fae0c1022d0 2024-05-21T21:53:12.647 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.647+0000 7fae11e00700 1 --2- 172.21.15.151:0/260116925 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fadf40b2900 0x7fadf40b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:12.647 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.647+0000 7fae06a00700 1 --2- 172.21.15.151:0/260116925 >> [v2:172.21.15.184:6800/3971150737,v1:172.21.15.184:6801/3971150737] conn(0x7fadf40bc4c0 0x7fadf40be960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:12.648 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.647+0000 7fae06a00700 1 -- 172.21.15.151:0/260116925 --> [v2:172.21.15.184:6800/3971150737,v1:172.21.15.184:6801/3971150737] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fadfc016ec0 con 0x7fadf40bc4c0 2024-05-21T21:53:12.648 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.647+0000 7fae06a00700 1 -- 172.21.15.151:0/260116925 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7fadfc0e2b10 con 0x7fae0c1022d0 2024-05-21T21:53:12.648 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.648+0000 7fae12800700 1 --2- 172.21.15.151:0/260116925 >> [v2:172.21.15.184:6800/3971150737,v1:172.21.15.184:6801/3971150737] conn(0x7fadf40bc4c0 0x7fadf40be960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:12.648 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.648+0000 7fae11e00700 1 --2- 172.21.15.151:0/260116925 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fadf40b2900 0x7fadf40b4dc0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fae00009c60 tx=0x7fae00009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:53:12.648 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.648+0000 7fae12800700 1 --2- 172.21.15.151:0/260116925 >> [v2:172.21.15.184:6800/3971150737,v1:172.21.15.184:6801/3971150737] conn(0x7fadf40bc4c0 0x7fadf40be960 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:53:12.649 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.649+0000 7fae06a00700 1 -- 172.21.15.151:0/260116925 <== osd.2 v2:172.21.15.184:6800/3971150737 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fadfc016ec0 con 0x7fadf40bc4c0 2024-05-21T21:53:12.667 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.667+0000 7fae14800700 1 -- 172.21.15.151:0/260116925 --> [v2:172.21.15.184:6800/3971150737,v1:172.21.15.184:6801/3971150737] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fae0c102a50 con 0x7fadf40bc4c0 2024-05-21T21:53:12.668 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:53:12 smithi151 ceph-mon[31623]: pgmap v373: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 85 B/s wr, 9 op/s 2024-05-21T21:53:12.668 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.668+0000 7fae06a00700 1 -- 172.21.15.151:0/260116925 <== osd.2 v2:172.21.15.184:6800/3971150737 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fae0c102a50 con 0x7fadf40bc4c0 2024-05-21T21:53:12.669 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.668+0000 7fae14800700 1 -- 172.21.15.151:0/260116925 >> [v2:172.21.15.184:6800/3971150737,v1:172.21.15.184:6801/3971150737] conn(0x7fadf40bc4c0 msgr2=0x7fadf40be960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:12.669 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.668+0000 7fae14800700 1 --2- 172.21.15.151:0/260116925 >> [v2:172.21.15.184:6800/3971150737,v1:172.21.15.184:6801/3971150737] conn(0x7fadf40bc4c0 0x7fadf40be960 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:12.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.668+0000 7fae14800700 1 -- 172.21.15.151:0/260116925 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fadf40b2900 msgr2=0x7fadf40b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:12.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.668+0000 7fae14800700 1 --2- 172.21.15.151:0/260116925 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fadf40b2900 0x7fadf40b4dc0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fae00009c60 tx=0x7fae00009260 comp rx=0 tx=0).stop 2024-05-21T21:53:12.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.668+0000 7fae14800700 1 -- 172.21.15.151:0/260116925 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fae0c1022d0 msgr2=0x7fae0c19c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:12.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.668+0000 7fae14800700 1 --2- 172.21.15.151:0/260116925 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fae0c1022d0 0x7fae0c19c2c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fadfc00e9f0 tx=0x7fadfc00ed00 comp rx=0 tx=0).stop 2024-05-21T21:53:12.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.669+0000 7fae12800700 1 -- 172.21.15.151:0/260116925 reap_dead start 2024-05-21T21:53:12.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.669+0000 7fae14800700 1 -- 172.21.15.151:0/260116925 shutdown_connections 2024-05-21T21:53:12.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.669+0000 7fae14800700 1 -- 172.21.15.151:0/260116925 >> 172.21.15.151:0/260116925 conn(0x7fae0c078640 msgr2=0x7fae0c100050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:53:12.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.669+0000 7fae14800700 1 -- 172.21.15.151:0/260116925 shutdown_connections 2024-05-21T21:53:12.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:12.669+0000 7fae14800700 1 -- 172.21.15.151:0/260116925 wait complete. 2024-05-21T21:53:12.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:53:12 smithi184 ceph-mon[36942]: pgmap v373: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 85 B/s wr, 9 op/s 2024-05-21T21:53:12.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:53:12 smithi164 ceph-mon[36269]: pgmap v373: 97 pgs: 97 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 85 B/s wr, 9 op/s 2024-05-21T21:53:13.376 INFO:teuthology.orchestra.run.smithi151.stdout:210453397565 2024-05-21T21:53:13.376 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-05-21T21:53:13.706 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:53:14.305 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.828+0000 7f9dffc00700 1 -- 172.21.15.151:0/650157256 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9df81037f0 msgr2=0x7f9df8103bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:14.305 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.828+0000 7f9dffc00700 1 --2- 172.21.15.151:0/650157256 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9df81037f0 0x7f9df8103bd0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9dec0099c0 tx=0x7f9dec009cd0 comp rx=0 tx=0).stop 2024-05-21T21:53:14.306 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.829+0000 7f9dffc00700 1 -- 172.21.15.151:0/650157256 shutdown_connections 2024-05-21T21:53:14.306 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.829+0000 7f9dffc00700 1 --2- 172.21.15.151:0/650157256 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9df81088f0 0x7f9df8110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:14.306 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.829+0000 7f9dffc00700 1 --2- 172.21.15.151:0/650157256 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9df81037f0 0x7f9df8103bd0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:14.306 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.829+0000 7f9dffc00700 1 --2- 172.21.15.151:0/650157256 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9df81041a0 0x7f9df81081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:14.306 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.829+0000 7f9dffc00700 1 -- 172.21.15.151:0/650157256 >> 172.21.15.151:0/650157256 conn(0x7f9df807a6a0 msgr2=0x7f9df807aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:53:14.306 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.829+0000 7f9dffc00700 1 -- 172.21.15.151:0/650157256 shutdown_connections 2024-05-21T21:53:14.306 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.829+0000 7f9dffc00700 1 -- 172.21.15.151:0/650157256 wait complete. 2024-05-21T21:53:14.306 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.830+0000 7f9dffc00700 1 Processor -- start 2024-05-21T21:53:14.306 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.830+0000 7f9dffc00700 1 -- start start 2024-05-21T21:53:14.306 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.830+0000 7f9dffc00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9df81037f0 0x7f9df8075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:14.306 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.830+0000 7f9dffc00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9df81041a0 0x7f9df8075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:14.307 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.830+0000 7f9dffc00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9df81088f0 0x7f9df80762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:14.307 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.830+0000 7f9dffc00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9df8113710 con 0x7f9df81037f0 2024-05-21T21:53:14.307 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.830+0000 7f9dffc00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9df8113590 con 0x7f9df81088f0 2024-05-21T21:53:14.307 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.830+0000 7f9dffc00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9df8113890 con 0x7f9df81041a0 2024-05-21T21:53:14.307 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.831+0000 7f9dfd200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9df81037f0 0x7f9df8075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:14.307 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.831+0000 7f9dfd200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9df81037f0 0x7f9df8075840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:60922/0 (socket says 172.21.15.151:60922) 2024-05-21T21:53:14.307 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.831+0000 7f9dfd200700 1 -- 172.21.15.151:0/2355512116 learned_addr learned my addr 172.21.15.151:0/2355512116 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:53:14.307 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.831+0000 7f9dfdc00700 1 --2- 172.21.15.151:0/2355512116 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9df81088f0 0x7f9df80762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:14.307 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.831+0000 7f9df7e00700 1 --2- 172.21.15.151:0/2355512116 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9df81041a0 0x7f9df8075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:14.307 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.831+0000 7f9dfd200700 1 -- 172.21.15.151:0/2355512116 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9df81041a0 msgr2=0x7f9df8075d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:14.307 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.831+0000 7f9dfd200700 1 --2- 172.21.15.151:0/2355512116 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9df81041a0 0x7f9df8075d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:14.307 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.831+0000 7f9dfd200700 1 -- 172.21.15.151:0/2355512116 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9df81088f0 msgr2=0x7f9df80762c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:14.308 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.831+0000 7f9dfd200700 1 --2- 172.21.15.151:0/2355512116 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9df81088f0 0x7f9df80762c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:14.308 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.831+0000 7f9dfd200700 1 -- 172.21.15.151:0/2355512116 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9df81a8130 con 0x7f9df81037f0 2024-05-21T21:53:14.308 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.831+0000 7f9dfd200700 1 --2- 172.21.15.151:0/2355512116 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9df81037f0 0x7f9df8075840 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9dec006010 tx=0x7f9dec0056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:53:14.308 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.832+0000 7f9df5600700 1 -- 172.21.15.151:0/2355512116 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dec026070 con 0x7f9df81037f0 2024-05-21T21:53:14.308 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.832+0000 7f9df5600700 1 -- 172.21.15.151:0/2355512116 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9dec00f460 con 0x7f9df81037f0 2024-05-21T21:53:14.308 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.832+0000 7f9dffc00700 1 -- 172.21.15.151:0/2355512116 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9df81a83c0 con 0x7f9df81037f0 2024-05-21T21:53:14.308 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.832+0000 7f9df5600700 1 -- 172.21.15.151:0/2355512116 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dec009e60 con 0x7f9df81037f0 2024-05-21T21:53:14.308 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.832+0000 7f9dffc00700 1 -- 172.21.15.151:0/2355512116 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9df81a8810 con 0x7f9df81037f0 2024-05-21T21:53:14.308 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.833+0000 7f9dffc00700 1 -- 172.21.15.151:0/2355512116 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9df8063cd0 con 0x7f9df81037f0 2024-05-21T21:53:14.308 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.835+0000 7f9df5600700 1 -- 172.21.15.151:0/2355512116 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f9dec029070 con 0x7f9df81037f0 2024-05-21T21:53:14.308 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.838+0000 7f9df5600700 1 --2- 172.21.15.151:0/2355512116 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f9de00b2880 0x7f9de00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:14.308 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.838+0000 7f9df5600700 1 -- 172.21.15.151:0/2355512116 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f9dec0e3170 con 0x7f9df81037f0 2024-05-21T21:53:14.308 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.838+0000 7f9df7e00700 1 --2- 172.21.15.151:0/2355512116 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f9de00b2880 0x7f9de00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:14.309 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.839+0000 7f9df5600700 1 --2- 172.21.15.151:0/2355512116 >> [v2:172.21.15.184:6824/1563340128,v1:172.21.15.184:6825/1563340128] conn(0x7f9de00bc440 0x7f9de00be8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:14.309 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.839+0000 7f9df5600700 1 -- 172.21.15.151:0/2355512116 --> [v2:172.21.15.184:6824/1563340128,v1:172.21.15.184:6825/1563340128] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9de00befb0 con 0x7f9de00bc440 2024-05-21T21:53:14.309 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.839+0000 7f9df5600700 1 -- 172.21.15.151:0/2355512116 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f9dec0e3520 con 0x7f9df81037f0 2024-05-21T21:53:14.309 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.839+0000 7f9df7e00700 1 --2- 172.21.15.151:0/2355512116 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f9de00b2880 0x7f9de00b4d40 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9de4005fd0 tx=0x7f9de4005f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:53:14.309 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.839+0000 7f9dfdc00700 1 --2- 172.21.15.151:0/2355512116 >> [v2:172.21.15.184:6824/1563340128,v1:172.21.15.184:6825/1563340128] conn(0x7f9de00bc440 0x7f9de00be8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:14.309 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.840+0000 7f9dfdc00700 1 --2- 172.21.15.151:0/2355512116 >> [v2:172.21.15.184:6824/1563340128,v1:172.21.15.184:6825/1563340128] conn(0x7f9de00bc440 0x7f9de00be8e0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:53:14.309 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.841+0000 7f9df5600700 1 -- 172.21.15.151:0/2355512116 <== osd.11 v2:172.21.15.184:6824/1563340128 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9de00befb0 con 0x7f9de00bc440 2024-05-21T21:53:14.309 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.861+0000 7f9dffc00700 1 -- 172.21.15.151:0/2355512116 --> [v2:172.21.15.184:6824/1563340128,v1:172.21.15.184:6825/1563340128] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9df80640d0 con 0x7f9de00bc440 2024-05-21T21:53:14.309 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.862+0000 7f9df5600700 1 -- 172.21.15.151:0/2355512116 <== osd.11 v2:172.21.15.184:6824/1563340128 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9df80640d0 con 0x7f9de00bc440 2024-05-21T21:53:14.310 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.863+0000 7f9dffc00700 1 -- 172.21.15.151:0/2355512116 >> [v2:172.21.15.184:6824/1563340128,v1:172.21.15.184:6825/1563340128] conn(0x7f9de00bc440 msgr2=0x7f9de00be8e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:14.310 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.863+0000 7f9dffc00700 1 --2- 172.21.15.151:0/2355512116 >> [v2:172.21.15.184:6824/1563340128,v1:172.21.15.184:6825/1563340128] conn(0x7f9de00bc440 0x7f9de00be8e0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:14.310 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.863+0000 7f9dffc00700 1 -- 172.21.15.151:0/2355512116 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f9de00b2880 msgr2=0x7f9de00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:14.310 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.863+0000 7f9dffc00700 1 --2- 172.21.15.151:0/2355512116 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f9de00b2880 0x7f9de00b4d40 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9de4005fd0 tx=0x7f9de4005f60 comp rx=0 tx=0).stop 2024-05-21T21:53:14.310 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.863+0000 7f9dffc00700 1 -- 172.21.15.151:0/2355512116 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9df81037f0 msgr2=0x7f9df8075840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:14.310 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.863+0000 7f9dffc00700 1 --2- 172.21.15.151:0/2355512116 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9df81037f0 0x7f9df8075840 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9dec006010 tx=0x7f9dec0056a0 comp rx=0 tx=0).stop 2024-05-21T21:53:14.310 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.863+0000 7f9dfdc00700 1 -- 172.21.15.151:0/2355512116 reap_dead start 2024-05-21T21:53:14.310 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.863+0000 7f9dffc00700 1 -- 172.21.15.151:0/2355512116 shutdown_connections 2024-05-21T21:53:14.310 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.863+0000 7f9dffc00700 1 -- 172.21.15.151:0/2355512116 >> 172.21.15.151:0/2355512116 conn(0x7f9df807a6a0 msgr2=0x7f9df8101810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:53:14.310 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.863+0000 7f9dffc00700 1 -- 172.21.15.151:0/2355512116 shutdown_connections 2024-05-21T21:53:14.310 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:11.863+0000 7f9dffc00700 1 -- 172.21.15.151:0/2355512116 wait complete. 2024-05-21T21:53:14.613 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:53:14 smithi151 ceph-mon[31623]: pgmap v374: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 11 op/s 2024-05-21T21:53:14.613 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:53:14 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:53:14.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:53:14 smithi184 ceph-mon[36942]: pgmap v374: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 11 op/s 2024-05-21T21:53:14.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:53:14 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:53:14.775 INFO:teuthology.orchestra.run.smithi151.stdout:261993005106 2024-05-21T21:53:14.775 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-05-21T21:53:14.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:53:14 smithi164 ceph-mon[36269]: pgmap v374: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 11 op/s 2024-05-21T21:53:14.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:53:14 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:53:15.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:15.163+0000 7f6bf8e00700 1 -- 172.21.15.151:0/692590898 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6bf410eea0 msgr2=0x7f6bf4116a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:15.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:15.163+0000 7f6bf8e00700 1 --2- 172.21.15.151:0/692590898 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6bf410eea0 0x7f6bf4116a70 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6bec00b0a0 tx=0x7f6bec00b3b0 comp rx=0 tx=0).stop 2024-05-21T21:53:15.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:15.163+0000 7f6bf8e00700 1 -- 172.21.15.151:0/692590898 shutdown_connections 2024-05-21T21:53:15.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:15.163+0000 7f6bf8e00700 1 --2- 172.21.15.151:0/692590898 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6bf410eea0 0x7f6bf4116a70 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:15.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:15.163+0000 7f6bf8e00700 1 --2- 172.21.15.151:0/692590898 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6bf4109e10 0x7f6bf410a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:15.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:15.163+0000 7f6bf8e00700 1 --2- 172.21.15.151:0/692590898 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6bf410a7c0 0x7f6bf410e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:15.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:15.163+0000 7f6bf8e00700 1 -- 172.21.15.151:0/692590898 >> 172.21.15.151:0/692590898 conn(0x7f6bf4076d10 msgr2=0x7f6bf4079130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:53:15.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:15.163+0000 7f6bf8e00700 1 -- 172.21.15.151:0/692590898 shutdown_connections 2024-05-21T21:53:15.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:15.163+0000 7f6bf8e00700 1 -- 172.21.15.151:0/692590898 wait complete. 2024-05-21T21:53:15.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:15.165+0000 7f6bf8e00700 1 Processor -- start 2024-05-21T21:53:15.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:15.165+0000 7f6bf8e00700 1 -- start start 2024-05-21T21:53:15.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:15.165+0000 7f6bf8e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6bf4109e10 0x7f6bf41a01e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:15.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:15.165+0000 7f6bf8e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6bf410a7c0 0x7f6bf41a0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:15.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:15.165+0000 7f6bf8e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6bf410eea0 0x7f6bf41a4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:15.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:15.165+0000 7f6bf8e00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf4119d30 con 0x7f6bf4109e10 2024-05-21T21:53:15.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:15.165+0000 7f6bf8e00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf4119bb0 con 0x7f6bf410eea0 2024-05-21T21:53:15.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:15.165+0000 7f6bf8e00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf4119eb0 con 0x7f6bf410a7c0 2024-05-21T21:53:15.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:15.166+0000 7f6bf1e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6bf4109e10 0x7f6bf41a01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:15.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:15.166+0000 7f6bf1e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6bf4109e10 0x7f6bf41a01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:60974/0 (socket says 172.21.15.151:60974) 2024-05-21T21:53:15.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:15.166+0000 7f6bf1e00700 1 -- 172.21.15.151:0/4030895691 learned_addr learned my addr 172.21.15.151:0/4030895691 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:53:15.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:15.166+0000 7f6bf1400700 1 --2- 172.21.15.151:0/4030895691 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6bf410a7c0 0x7f6bf41a0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:15.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:15.166+0000 7f6bf2800700 1 --2- 172.21.15.151:0/4030895691 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6bf410eea0 0x7f6bf41a4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:15.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:15.166+0000 7f6bf1e00700 1 -- 172.21.15.151:0/4030895691 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6bf410a7c0 msgr2=0x7f6bf41a0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:15.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:15.166+0000 7f6bf1e00700 1 --2- 172.21.15.151:0/4030895691 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6bf410a7c0 0x7f6bf41a0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:15.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:15.166+0000 7f6bf1e00700 1 -- 172.21.15.151:0/4030895691 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6bf410eea0 msgr2=0x7f6bf41a4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:15.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:15.166+0000 7f6bf1e00700 1 --2- 172.21.15.151:0/4030895691 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6bf410eea0 0x7f6bf41a4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:15.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:15.166+0000 7f6bf1e00700 1 -- 172.21.15.151:0/4030895691 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bf41a51f0 con 0x7f6bf4109e10 2024-05-21T21:53:15.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:15.166+0000 7f6bf1e00700 1 --2- 172.21.15.151:0/4030895691 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6bf4109e10 0x7f6bf41a01e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f6be400e9a0 tx=0x7f6be400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:53:15.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:15.167+0000 7f6beaa00700 1 -- 172.21.15.151:0/4030895691 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6be400ca10 con 0x7f6bf4109e10 2024-05-21T21:53:15.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:15.167+0000 7f6bf8e00700 1 -- 172.21.15.151:0/4030895691 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bf41a54e0 con 0x7f6bf4109e10 2024-05-21T21:53:15.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:15.167+0000 7f6beaa00700 1 -- 172.21.15.151:0/4030895691 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6be400cbb0 con 0x7f6bf4109e10 2024-05-21T21:53:15.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:15.167+0000 7f6beaa00700 1 -- 172.21.15.151:0/4030895691 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6be4018680 con 0x7f6bf4109e10 2024-05-21T21:53:15.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:15.167+0000 7f6bf8e00700 1 -- 172.21.15.151:0/4030895691 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bf41acd60 con 0x7f6bf4109e10 2024-05-21T21:53:15.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:15.168+0000 7f6bf8e00700 1 -- 172.21.15.151:0/4030895691 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6bf4063cd0 con 0x7f6bf4109e10 2024-05-21T21:53:15.172 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:15.172+0000 7f6beaa00700 1 -- 172.21.15.151:0/4030895691 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f6be4010430 con 0x7f6bf4109e10 2024-05-21T21:53:15.174 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:15.174+0000 7f6beaa00700 1 --2- 172.21.15.151:0/4030895691 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f6bd80b2900 0x7f6bd80b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:15.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:15.175+0000 7f6beaa00700 1 -- 172.21.15.151:0/4030895691 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f6be40e2950 con 0x7f6bf4109e10 2024-05-21T21:53:15.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:15.175+0000 7f6bf1400700 1 --2- 172.21.15.151:0/4030895691 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f6bd80b2900 0x7f6bd80b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:15.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:15.175+0000 7f6beaa00700 1 --2- 172.21.15.151:0/4030895691 >> [v2:172.21.15.151:6818/3377550762,v1:172.21.15.151:6819/3377550762] conn(0x7f6bd80bc4c0 0x7f6bd80be960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:15.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:15.175+0000 7f6beaa00700 1 -- 172.21.15.151:0/4030895691 --> [v2:172.21.15.151:6818/3377550762,v1:172.21.15.151:6819/3377550762] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6be4024e70 con 0x7f6bd80bc4c0 2024-05-21T21:53:15.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:15.175+0000 7f6beaa00700 1 -- 172.21.15.151:0/4030895691 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f6be40e2d00 con 0x7f6bf4109e10 2024-05-21T21:53:15.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:15.175+0000 7f6bf2800700 1 --2- 172.21.15.151:0/4030895691 >> [v2:172.21.15.151:6818/3377550762,v1:172.21.15.151:6819/3377550762] conn(0x7f6bd80bc4c0 0x7f6bd80be960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:15.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:15.175+0000 7f6bf1400700 1 --2- 172.21.15.151:0/4030895691 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f6bd80b2900 0x7f6bd80b4dc0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6bf4108de0 tx=0x7f6be0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:53:15.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:15.176+0000 7f6bf2800700 1 --2- 172.21.15.151:0/4030895691 >> [v2:172.21.15.151:6818/3377550762,v1:172.21.15.151:6819/3377550762] conn(0x7f6bd80bc4c0 0x7f6bd80be960 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:53:15.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:15.177+0000 7f6beaa00700 1 -- 172.21.15.151:0/4030895691 <== osd.6 v2:172.21.15.151:6818/3377550762 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6be4024e70 con 0x7f6bd80bc4c0 2024-05-21T21:53:15.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:15.197+0000 7f6bf8e00700 1 -- 172.21.15.151:0/4030895691 --> [v2:172.21.15.151:6818/3377550762,v1:172.21.15.151:6819/3377550762] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6bf4064100 con 0x7f6bd80bc4c0 2024-05-21T21:53:15.198 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:15.198+0000 7f6beaa00700 1 -- 172.21.15.151:0/4030895691 <== osd.6 v2:172.21.15.151:6818/3377550762 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6bf4064100 con 0x7f6bd80bc4c0 2024-05-21T21:53:15.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:15.198+0000 7f6bf8e00700 1 -- 172.21.15.151:0/4030895691 >> [v2:172.21.15.151:6818/3377550762,v1:172.21.15.151:6819/3377550762] conn(0x7f6bd80bc4c0 msgr2=0x7f6bd80be960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:15.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:15.198+0000 7f6bf8e00700 1 --2- 172.21.15.151:0/4030895691 >> [v2:172.21.15.151:6818/3377550762,v1:172.21.15.151:6819/3377550762] conn(0x7f6bd80bc4c0 0x7f6bd80be960 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:15.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:15.199+0000 7f6bf8e00700 1 -- 172.21.15.151:0/4030895691 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f6bd80b2900 msgr2=0x7f6bd80b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:15.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:15.199+0000 7f6bf8e00700 1 --2- 172.21.15.151:0/4030895691 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f6bd80b2900 0x7f6bd80b4dc0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6bf4108de0 tx=0x7f6be0009260 comp rx=0 tx=0).stop 2024-05-21T21:53:15.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:15.199+0000 7f6bf8e00700 1 -- 172.21.15.151:0/4030895691 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6bf4109e10 msgr2=0x7f6bf41a01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:15.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:15.199+0000 7f6bf8e00700 1 --2- 172.21.15.151:0/4030895691 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6bf4109e10 0x7f6bf41a01e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f6be400e9a0 tx=0x7f6be400ecb0 comp rx=0 tx=0).stop 2024-05-21T21:53:15.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:15.199+0000 7f6bf2800700 1 -- 172.21.15.151:0/4030895691 reap_dead start 2024-05-21T21:53:15.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:15.199+0000 7f6bf8e00700 1 -- 172.21.15.151:0/4030895691 shutdown_connections 2024-05-21T21:53:15.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:15.199+0000 7f6bf8e00700 1 -- 172.21.15.151:0/4030895691 >> 172.21.15.151:0/4030895691 conn(0x7f6bf4076d10 msgr2=0x7f6bf4079010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:53:15.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:15.199+0000 7f6bf8e00700 1 -- 172.21.15.151:0/4030895691 shutdown_connections 2024-05-21T21:53:15.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:15.199+0000 7f6bf8e00700 1 -- 172.21.15.151:0/4030895691 wait complete. 2024-05-21T21:53:15.665 INFO:teuthology.orchestra.run.smithi151.stdout:309237645353 2024-05-21T21:53:15.666 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-05-21T21:53:15.990 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:53:16.011 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:53:16.049 INFO:teuthology.orchestra.run.smithi151.stdout:77309411419 2024-05-21T21:53:16.050 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-05-21T21:53:16.262 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:53:16 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:53:16.262 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:53:16 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:53:16.262 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:53:16 smithi151 ceph-mon[31623]: pgmap v375: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 597 B/s wr, 10 op/s 2024-05-21T21:53:16.262 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:53:16 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:53:16.262 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:53:16 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:53:16.263 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:53:16 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:53:16.263 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:53:16 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:53:16.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:53:16 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:53:16.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:53:16 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:53:16.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:53:16 smithi184 ceph-mon[36942]: pgmap v375: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 597 B/s wr, 10 op/s 2024-05-21T21:53:16.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:53:16 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:53:16.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:53:16 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:53:16.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:53:16 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:53:16.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:53:16 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:53:16.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:53:16 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:53:16.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:53:16 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:53:16.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:53:16 smithi164 ceph-mon[36269]: pgmap v375: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 597 B/s wr, 10 op/s 2024-05-21T21:53:16.537 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:53:16 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:53:16.537 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:53:16 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:53:16.537 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:53:16 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:53:16.537 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:53:16 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:53:18.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:53:18 smithi151 ceph-mon[31623]: pgmap v376: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 597 B/s wr, 11 op/s 2024-05-21T21:53:18.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:53:18 smithi184 ceph-mon[36942]: pgmap v376: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 597 B/s wr, 11 op/s 2024-05-21T21:53:18.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:53:18 smithi164 ceph-mon[36269]: pgmap v376: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 597 B/s wr, 11 op/s 2024-05-21T21:53:19.433 INFO:teuthology.orchestra.run.smithi151.stdout:188978561092 2024-05-21T21:53:19.433 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-05-21T21:53:20.141 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:53:20.141 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:53:20.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:53:20 smithi151 ceph-mon[31623]: pgmap v377: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 9 op/s 2024-05-21T21:53:20.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:53:20 smithi184 ceph-mon[36942]: pgmap v377: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 9 op/s 2024-05-21T21:53:20.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:53:20 smithi164 ceph-mon[36269]: pgmap v377: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 9 op/s 2024-05-21T21:53:21.564 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:53:21 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/1614409617' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-21T21:53:21.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:53:21 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/1614409617' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-21T21:53:21.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:53:21 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/1614409617' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-21T21:53:21.872 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:20.770+0000 7fda56000700 1 -- 172.21.15.151:0/3822843041 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fda5010da60 msgr2=0x7fda5010ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:21.872 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:20.770+0000 7fda56000700 1 --2- 172.21.15.151:0/3822843041 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fda5010da60 0x7fda5010ff20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fda4800b0a0 tx=0x7fda4800b3b0 comp rx=0 tx=0).stop 2024-05-21T21:53:21.872 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:20.770+0000 7fda56000700 1 -- 172.21.15.151:0/3822843041 shutdown_connections 2024-05-21T21:53:21.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:20.770+0000 7fda56000700 1 --2- 172.21.15.151:0/3822843041 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fda5010da60 0x7fda5010ff20 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:21.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:20.770+0000 7fda56000700 1 --2- 172.21.15.151:0/3822843041 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda50104630 0x7fda50104a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:21.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:20.770+0000 7fda56000700 1 --2- 172.21.15.151:0/3822843041 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fda50104f50 0x7fda5010d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:21.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:20.770+0000 7fda56000700 1 -- 172.21.15.151:0/3822843041 >> 172.21.15.151:0/3822843041 conn(0x7fda500fde60 msgr2=0x7fda50100280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:53:21.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:20.770+0000 7fda56000700 1 -- 172.21.15.151:0/3822843041 shutdown_connections 2024-05-21T21:53:21.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:20.770+0000 7fda56000700 1 -- 172.21.15.151:0/3822843041 wait complete. 2024-05-21T21:53:21.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:20.771+0000 7fda56000700 1 Processor -- start 2024-05-21T21:53:21.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:20.771+0000 7fda56000700 1 -- start start 2024-05-21T21:53:21.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:20.772+0000 7fda56000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fda50104630 0x7fda501a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:21.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:20.772+0000 7fda56000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fda50104f50 0x7fda501a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:21.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:20.772+0000 7fda56000700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda5010da60 0x7fda5010c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:21.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:20.772+0000 7fda56000700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fda50113670 con 0x7fda50104630 2024-05-21T21:53:21.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:20.772+0000 7fda56000700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fda501134f0 con 0x7fda50104f50 2024-05-21T21:53:21.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:20.772+0000 7fda56000700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fda501137f0 con 0x7fda5010da60 2024-05-21T21:53:21.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:20.772+0000 7fda4fe00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fda50104f50 0x7fda501a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:21.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:20.772+0000 7fda54c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fda50104630 0x7fda501a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:21.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:20.772+0000 7fda4fe00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fda50104f50 0x7fda501a0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:33624/0 (socket says 172.21.15.151:33624) 2024-05-21T21:53:21.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:20.772+0000 7fda4fe00700 1 -- 172.21.15.151:0/1614409617 learned_addr learned my addr 172.21.15.151:0/1614409617 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:53:21.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:20.772+0000 7fda55600700 1 --2- 172.21.15.151:0/1614409617 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda5010da60 0x7fda5010c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:21.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:20.772+0000 7fda4fe00700 1 -- 172.21.15.151:0/1614409617 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda5010da60 msgr2=0x7fda5010c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:21.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:20.772+0000 7fda4fe00700 1 --2- 172.21.15.151:0/1614409617 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda5010da60 0x7fda5010c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:21.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:20.772+0000 7fda4fe00700 1 -- 172.21.15.151:0/1614409617 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fda50104630 msgr2=0x7fda501a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:21.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:20.772+0000 7fda4fe00700 1 --2- 172.21.15.151:0/1614409617 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fda50104630 0x7fda501a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:21.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:20.772+0000 7fda4fe00700 1 -- 172.21.15.151:0/1614409617 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda5010ce50 con 0x7fda50104f50 2024-05-21T21:53:21.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:20.773+0000 7fda4fe00700 1 --2- 172.21.15.151:0/1614409617 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fda50104f50 0x7fda501a0bb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fda3c00e980 tx=0x7fda3c00ec90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:53:21.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:20.773+0000 7fda4d600700 1 -- 172.21.15.151:0/1614409617 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda3c004220 con 0x7fda50104f50 2024-05-21T21:53:21.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:20.773+0000 7fda56000700 1 -- 172.21.15.151:0/1614409617 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda5010d140 con 0x7fda50104f50 2024-05-21T21:53:21.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:20.773+0000 7fda4d600700 1 -- 172.21.15.151:0/1614409617 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda3c0043c0 con 0x7fda50104f50 2024-05-21T21:53:21.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:20.773+0000 7fda4d600700 1 -- 172.21.15.151:0/1614409617 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda3c003d60 con 0x7fda50104f50 2024-05-21T21:53:21.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:20.773+0000 7fda56000700 1 -- 172.21.15.151:0/1614409617 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda5019c380 con 0x7fda50104f50 2024-05-21T21:53:21.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:20.774+0000 7fda56000700 1 -- 172.21.15.151:0/1614409617 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda5006a230 con 0x7fda50104f50 2024-05-21T21:53:21.876 INFO:teuthology.orchestra.run.smithi151.stdout:236223201338 2024-05-21T21:53:21.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:20.779+0000 7fda4d600700 1 -- 172.21.15.151:0/1614409617 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fda3c00c810 con 0x7fda50104f50 2024-05-21T21:53:21.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:20.780+0000 7fda4d600700 1 --2- 172.21.15.151:0/1614409617 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fda380b2900 0x7fda380b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:21.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:20.780+0000 7fda4d600700 1 -- 172.21.15.151:0/1614409617 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fda3c0e23b0 con 0x7fda50104f50 2024-05-21T21:53:21.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:20.780+0000 7fda54c00700 1 --2- 172.21.15.151:0/1614409617 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fda380b2900 0x7fda380b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:21.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:20.780+0000 7fda4d600700 1 -- 172.21.15.151:0/1614409617 <== mon.2 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fda3c0aa120 con 0x7fda50104f50 2024-05-21T21:53:21.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:20.781+0000 7fda54c00700 1 --2- 172.21.15.151:0/1614409617 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fda380b2900 0x7fda380b4dc0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fda44009c60 tx=0x7fda44009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:53:21.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:21.045+0000 7fda56000700 1 -- 172.21.15.151:0/1614409617 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7fda50198c50 con 0x7fda50104f50 2024-05-21T21:53:21.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:21.046+0000 7fda4d600700 1 -- 172.21.15.151:0/1614409617 <== mon.2 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fda50198c50 con 0x7fda50104f50 2024-05-21T21:53:21.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:21.048+0000 7fda56000700 1 -- 172.21.15.151:0/1614409617 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fda380b2900 msgr2=0x7fda380b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:21.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:21.049+0000 7fda56000700 1 --2- 172.21.15.151:0/1614409617 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fda380b2900 0x7fda380b4dc0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fda44009c60 tx=0x7fda44009260 comp rx=0 tx=0).stop 2024-05-21T21:53:21.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:21.049+0000 7fda56000700 1 -- 172.21.15.151:0/1614409617 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fda50104f50 msgr2=0x7fda501a0bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:21.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:21.049+0000 7fda56000700 1 --2- 172.21.15.151:0/1614409617 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fda50104f50 0x7fda501a0bb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fda3c00e980 tx=0x7fda3c00ec90 comp rx=0 tx=0).stop 2024-05-21T21:53:21.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:21.049+0000 7fda56000700 1 -- 172.21.15.151:0/1614409617 shutdown_connections 2024-05-21T21:53:21.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:21.049+0000 7fda56000700 1 --2- 172.21.15.151:0/1614409617 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fda380b2900 0x7fda380b4dc0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:21.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:21.049+0000 7fda56000700 1 --2- 172.21.15.151:0/1614409617 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda5010da60 0x7fda5010c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:21.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:21.049+0000 7fda56000700 1 --2- 172.21.15.151:0/1614409617 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fda50104630 0x7fda501a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:21.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:21.049+0000 7fda56000700 1 --2- 172.21.15.151:0/1614409617 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fda50104f50 0x7fda501a0bb0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:21.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:21.049+0000 7fda56000700 1 -- 172.21.15.151:0/1614409617 >> 172.21.15.151:0/1614409617 conn(0x7fda500fde60 msgr2=0x7fda5010fd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:53:21.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:21.049+0000 7fda56000700 1 -- 172.21.15.151:0/1614409617 shutdown_connections 2024-05-21T21:53:21.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:21.049+0000 7fda56000700 1 -- 172.21.15.151:0/1614409617 wait complete. 2024-05-21T21:53:22.073 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:53:22.202 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:53:22.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:53:22 smithi151 ceph-mon[31623]: pgmap v378: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 597 B/s wr, 10 op/s 2024-05-21T21:53:22.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:53:22 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/668470061' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-21T21:53:22.637 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:53:22 smithi184 ceph-mon[36942]: pgmap v378: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 597 B/s wr, 10 op/s 2024-05-21T21:53:22.637 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:53:22 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/668470061' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-21T21:53:22.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:53:22 smithi164 ceph-mon[36269]: pgmap v378: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 597 B/s wr, 10 op/s 2024-05-21T21:53:22.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:53:22 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/668470061' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-21T21:53:22.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:22.821+0000 7f42f0600700 1 -- 172.21.15.151:0/425234391 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42e8076090 msgr2=0x7f42e810fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:22.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:22.821+0000 7f42f0600700 1 --2- 172.21.15.151:0/425234391 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42e8076090 0x7f42e810fa00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f42d40098f0 tx=0x7f42d4009c00 comp rx=0 tx=0).stop 2024-05-21T21:53:22.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:22.821+0000 7f42f0600700 1 -- 172.21.15.151:0/425234391 shutdown_connections 2024-05-21T21:53:22.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:22.821+0000 7f42f0600700 1 --2- 172.21.15.151:0/425234391 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f42e810ff40 0x7f42e8112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:22.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:22.821+0000 7f42f0600700 1 --2- 172.21.15.151:0/425234391 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f42e8075770 0x7f42e8075b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:22.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:22.821+0000 7f42f0600700 1 --2- 172.21.15.151:0/425234391 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42e8076090 0x7f42e810fa00 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:22.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:22.821+0000 7f42f0600700 1 -- 172.21.15.151:0/425234391 >> 172.21.15.151:0/425234391 conn(0x7f42e807a6a0 msgr2=0x7f42e807aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:53:22.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:22.821+0000 7f42f0600700 1 -- 172.21.15.151:0/425234391 shutdown_connections 2024-05-21T21:53:22.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:22.821+0000 7f42f0600700 1 -- 172.21.15.151:0/425234391 wait complete. 2024-05-21T21:53:22.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:22.822+0000 7f42f0600700 1 Processor -- start 2024-05-21T21:53:22.823 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:22.822+0000 7f42f0600700 1 -- start start 2024-05-21T21:53:22.823 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:22.823+0000 7f42f0600700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42e8075770 0x7f42e819be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:22.823 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:22.823+0000 7f42f0600700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f42e8076090 0x7f42e819c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:22.823 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:22.823+0000 7f42f0600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f42e810ff40 0x7f42e81a0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:22.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:22.823+0000 7f42f0600700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f42e8115970 con 0x7f42e810ff40 2024-05-21T21:53:22.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:22.823+0000 7f42f0600700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f42e81157f0 con 0x7f42e8075770 2024-05-21T21:53:22.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:22.823+0000 7f42f0600700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f42e8115af0 con 0x7f42e8076090 2024-05-21T21:53:22.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:22.823+0000 7f42edc00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42e8075770 0x7f42e819be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:22.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:22.823+0000 7f42ed200700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f42e8076090 0x7f42e819c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:22.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:22.823+0000 7f42edc00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42e8075770 0x7f42e819be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:33672/0 (socket says 172.21.15.151:33672) 2024-05-21T21:53:22.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:22.823+0000 7f42edc00700 1 -- 172.21.15.151:0/2857770259 learned_addr learned my addr 172.21.15.151:0/2857770259 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:53:22.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:22.823+0000 7f42ee600700 1 --2- 172.21.15.151:0/2857770259 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f42e810ff40 0x7f42e81a0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:22.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:22.824+0000 7f42edc00700 1 -- 172.21.15.151:0/2857770259 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f42e8076090 msgr2=0x7f42e819c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:22.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:22.824+0000 7f42edc00700 1 --2- 172.21.15.151:0/2857770259 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f42e8076090 0x7f42e819c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:22.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:22.824+0000 7f42edc00700 1 -- 172.21.15.151:0/2857770259 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f42e810ff40 msgr2=0x7f42e81a0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:22.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:22.824+0000 7f42edc00700 1 --2- 172.21.15.151:0/2857770259 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f42e810ff40 0x7f42e81a0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:22.826 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:22.824+0000 7f42edc00700 1 -- 172.21.15.151:0/2857770259 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42e81a0e90 con 0x7f42e8075770 2024-05-21T21:53:22.826 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:22.824+0000 7f42ee600700 1 --2- 172.21.15.151:0/2857770259 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f42e810ff40 0x7f42e81a0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T21:53:22.826 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:22.824+0000 7f42edc00700 1 --2- 172.21.15.151:0/2857770259 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42e8075770 0x7f42e819be80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f42dc00e9a0 tx=0x7f42dc00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:53:22.826 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:22.824+0000 7f42e6000700 1 -- 172.21.15.151:0/2857770259 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42dc00cbd0 con 0x7f42e8075770 2024-05-21T21:53:22.826 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:22.824+0000 7f42e6000700 1 -- 172.21.15.151:0/2857770259 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42dc00cd70 con 0x7f42e8075770 2024-05-21T21:53:22.826 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:22.824+0000 7f42e6000700 1 -- 172.21.15.151:0/2857770259 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42dc018680 con 0x7f42e8075770 2024-05-21T21:53:22.826 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:22.824+0000 7f42f0600700 1 -- 172.21.15.151:0/2857770259 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42e81a1180 con 0x7f42e8075770 2024-05-21T21:53:22.826 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:22.825+0000 7f42f0600700 1 -- 172.21.15.151:0/2857770259 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42e81a8a00 con 0x7f42e8075770 2024-05-21T21:53:22.826 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:22.825+0000 7f42f0600700 1 -- 172.21.15.151:0/2857770259 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f42e8063cd0 con 0x7f42e8075770 2024-05-21T21:53:22.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:22.828+0000 7f42e6000700 1 -- 172.21.15.151:0/2857770259 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f42dc010430 con 0x7f42e8075770 2024-05-21T21:53:22.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:22.829+0000 7f42e6000700 1 --2- 172.21.15.151:0/2857770259 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f42d00b2900 0x7f42d00b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:22.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:22.830+0000 7f42e6000700 1 -- 172.21.15.151:0/2857770259 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f42dc0e2c60 con 0x7f42e8075770 2024-05-21T21:53:22.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:22.830+0000 7f42ed200700 1 --2- 172.21.15.151:0/2857770259 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f42d00b2900 0x7f42d00b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:22.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:22.830+0000 7f42e6000700 1 --2- 172.21.15.151:0/2857770259 >> [v2:172.21.15.164:6824/1823448782,v1:172.21.15.164:6825/1823448782] conn(0x7f42d00bc4c0 0x7f42d00be960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:22.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:22.830+0000 7f42e6000700 1 -- 172.21.15.151:0/2857770259 --> [v2:172.21.15.164:6824/1823448782,v1:172.21.15.164:6825/1823448782] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f42d00bf030 con 0x7f42d00bc4c0 2024-05-21T21:53:22.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:22.830+0000 7f42e6000700 1 -- 172.21.15.151:0/2857770259 <== mon.2 v2:172.21.15.164:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f42dc0e3010 con 0x7f42e8075770 2024-05-21T21:53:22.831 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:22.830+0000 7f42ed200700 1 --2- 172.21.15.151:0/2857770259 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f42d00b2900 0x7f42d00b4dc0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f42d40098c0 tx=0x7f42d4019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:53:22.831 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:22.830+0000 7f42ee600700 1 --2- 172.21.15.151:0/2857770259 >> [v2:172.21.15.164:6824/1823448782,v1:172.21.15.164:6825/1823448782] conn(0x7f42d00bc4c0 0x7f42d00be960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:22.831 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:22.831+0000 7f42ee600700 1 --2- 172.21.15.151:0/2857770259 >> [v2:172.21.15.164:6824/1823448782,v1:172.21.15.164:6825/1823448782] conn(0x7f42d00bc4c0 0x7f42d00be960 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:53:22.831 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:22.832+0000 7f42e6000700 1 -- 172.21.15.151:0/2857770259 <== osd.10 v2:172.21.15.164:6824/1823448782 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f42d00bf030 con 0x7f42d00bc4c0 2024-05-21T21:53:22.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:22.849+0000 7f42f0600700 1 -- 172.21.15.151:0/2857770259 --> [v2:172.21.15.164:6824/1823448782,v1:172.21.15.164:6825/1823448782] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f42e80641a0 con 0x7f42d00bc4c0 2024-05-21T21:53:22.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:22.850+0000 7f42e6000700 1 -- 172.21.15.151:0/2857770259 <== osd.10 v2:172.21.15.164:6824/1823448782 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f42e80641a0 con 0x7f42d00bc4c0 2024-05-21T21:53:22.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:22.851+0000 7f42f0600700 1 -- 172.21.15.151:0/2857770259 >> [v2:172.21.15.164:6824/1823448782,v1:172.21.15.164:6825/1823448782] conn(0x7f42d00bc4c0 msgr2=0x7f42d00be960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:22.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:22.851+0000 7f42f0600700 1 --2- 172.21.15.151:0/2857770259 >> [v2:172.21.15.164:6824/1823448782,v1:172.21.15.164:6825/1823448782] conn(0x7f42d00bc4c0 0x7f42d00be960 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:22.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:22.851+0000 7f42f0600700 1 -- 172.21.15.151:0/2857770259 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f42d00b2900 msgr2=0x7f42d00b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:22.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:22.851+0000 7f42f0600700 1 --2- 172.21.15.151:0/2857770259 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f42d00b2900 0x7f42d00b4dc0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f42d40098c0 tx=0x7f42d4019040 comp rx=0 tx=0).stop 2024-05-21T21:53:22.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:22.851+0000 7f42f0600700 1 -- 172.21.15.151:0/2857770259 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42e8075770 msgr2=0x7f42e819be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:22.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:22.851+0000 7f42f0600700 1 --2- 172.21.15.151:0/2857770259 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42e8075770 0x7f42e819be80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f42dc00e9a0 tx=0x7f42dc00ecb0 comp rx=0 tx=0).stop 2024-05-21T21:53:22.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:22.851+0000 7f42ee600700 1 -- 172.21.15.151:0/2857770259 reap_dead start 2024-05-21T21:53:22.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:22.851+0000 7f42f0600700 1 -- 172.21.15.151:0/2857770259 shutdown_connections 2024-05-21T21:53:22.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:22.851+0000 7f42f0600700 1 -- 172.21.15.151:0/2857770259 >> 172.21.15.151:0/2857770259 conn(0x7f42e807a6a0 msgr2=0x7f42e8100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:53:22.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:22.851+0000 7f42f0600700 1 -- 172.21.15.151:0/2857770259 shutdown_connections 2024-05-21T21:53:22.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:22.852+0000 7f42f0600700 1 -- 172.21.15.151:0/2857770259 wait complete. 2024-05-21T21:53:23.193 INFO:tasks.cephadm.ceph_manager.ceph:need seq 236223201334 got 236223201338 for osd.8 2024-05-21T21:53:23.193 DEBUG:teuthology.parallel:result is None 2024-05-21T21:53:23.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:21.909+0000 7feffd400700 1 -- 172.21.15.151:0/1967458995 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feff81094f0 msgr2=0x7feff81098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:23.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:21.909+0000 7feffd400700 1 --2- 172.21.15.151:0/1967458995 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feff81094f0 0x7feff81098d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fefe8009990 tx=0x7fefe8009ca0 comp rx=0 tx=0).stop 2024-05-21T21:53:23.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:21.909+0000 7feffd400700 1 -- 172.21.15.151:0/1967458995 shutdown_connections 2024-05-21T21:53:23.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:21.909+0000 7feffd400700 1 --2- 172.21.15.151:0/1967458995 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feff81094f0 0x7feff81098d0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:23.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:21.909+0000 7feffd400700 1 --2- 172.21.15.151:0/1967458995 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feff8103e20 0x7feff8110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:23.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:21.909+0000 7feffd400700 1 --2- 172.21.15.151:0/1967458995 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feff8103460 0x7feff81038e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:23.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:21.909+0000 7feffd400700 1 -- 172.21.15.151:0/1967458995 >> 172.21.15.151:0/1967458995 conn(0x7feff80fef50 msgr2=0x7feff8101370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:53:23.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:21.909+0000 7feffd400700 1 -- 172.21.15.151:0/1967458995 shutdown_connections 2024-05-21T21:53:23.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:21.909+0000 7feffd400700 1 -- 172.21.15.151:0/1967458995 wait complete. 2024-05-21T21:53:23.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:21.910+0000 7feffd400700 1 Processor -- start 2024-05-21T21:53:23.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:21.910+0000 7feffd400700 1 -- start start 2024-05-21T21:53:23.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:21.911+0000 7feffd400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feff8103460 0x7feff8075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:23.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:21.911+0000 7feffd400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feff8103e20 0x7feff8075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:23.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:21.911+0000 7feffd400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feff81094f0 0x7feff80762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:23.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:21.911+0000 7feffd400700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7feff8113670 con 0x7feff8103460 2024-05-21T21:53:23.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:21.911+0000 7feffd400700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7feff81134f0 con 0x7feff81094f0 2024-05-21T21:53:23.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:21.911+0000 7feffd400700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7feff81137f0 con 0x7feff8103e20 2024-05-21T21:53:23.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:21.911+0000 7feff7e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feff8103460 0x7feff8075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:23.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:21.911+0000 7feff7e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feff8103460 0x7feff8075840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:47000/0 (socket says 172.21.15.151:47000) 2024-05-21T21:53:23.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:21.911+0000 7feff7e00700 1 -- 172.21.15.151:0/668470061 learned_addr learned my addr 172.21.15.151:0/668470061 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:53:23.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:21.911+0000 7feffca00700 1 --2- 172.21.15.151:0/668470061 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feff81094f0 0x7feff80762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:23.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:21.911+0000 7feff7400700 1 --2- 172.21.15.151:0/668470061 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feff8103e20 0x7feff8075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:23.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:21.911+0000 7feff7e00700 1 -- 172.21.15.151:0/668470061 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feff8103e20 msgr2=0x7feff8075d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:23.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:21.911+0000 7feff7e00700 1 --2- 172.21.15.151:0/668470061 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feff8103e20 0x7feff8075d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:23.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:21.911+0000 7feff7e00700 1 -- 172.21.15.151:0/668470061 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feff81094f0 msgr2=0x7feff80762c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:23.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:21.911+0000 7feff7e00700 1 --2- 172.21.15.151:0/668470061 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feff81094f0 0x7feff80762c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:23.578 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:21.911+0000 7feff7e00700 1 -- 172.21.15.151:0/668470061 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feff81a80b0 con 0x7feff8103460 2024-05-21T21:53:23.578 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:21.912+0000 7feff7e00700 1 --2- 172.21.15.151:0/668470061 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feff8103460 0x7feff8075840 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fefe8000c00 tx=0x7fefe80057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:53:23.578 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:21.912+0000 7feff4c00700 1 -- 172.21.15.151:0/668470061 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe8027070 con 0x7feff8103460 2024-05-21T21:53:23.578 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:21.912+0000 7feff4c00700 1 -- 172.21.15.151:0/668470061 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fefe8022470 con 0x7feff8103460 2024-05-21T21:53:23.578 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:21.912+0000 7feffd400700 1 -- 172.21.15.151:0/668470061 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feff81a8340 con 0x7feff8103460 2024-05-21T21:53:23.578 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:21.912+0000 7feff4c00700 1 -- 172.21.15.151:0/668470061 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe8017650 con 0x7feff8103460 2024-05-21T21:53:23.578 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:21.912+0000 7feffd400700 1 -- 172.21.15.151:0/668470061 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feff81a8790 con 0x7feff8103460 2024-05-21T21:53:23.578 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:21.913+0000 7feffd400700 1 -- 172.21.15.151:0/668470061 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feff806a230 con 0x7feff8103460 2024-05-21T21:53:23.578 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:21.916+0000 7feff4c00700 1 -- 172.21.15.151:0/668470061 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fefe802a070 con 0x7feff8103460 2024-05-21T21:53:23.578 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:21.918+0000 7feff4c00700 1 --2- 172.21.15.151:0/668470061 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fefdc0b2880 0x7fefdc0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:23.578 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:21.919+0000 7feff4c00700 1 -- 172.21.15.151:0/668470061 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fefe80e2fe0 con 0x7feff8103460 2024-05-21T21:53:23.578 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:21.919+0000 7feff7400700 1 --2- 172.21.15.151:0/668470061 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fefdc0b2880 0x7fefdc0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:23.578 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:21.919+0000 7feff7400700 1 --2- 172.21.15.151:0/668470061 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fefdc0b2880 0x7fefdc0b4d40 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7feff8101570 tx=0x7fefe4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:53:23.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:21.921+0000 7feff4c00700 1 -- 172.21.15.151:0/668470061 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fefe8013070 con 0x7feff8103460 2024-05-21T21:53:23.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:22.156+0000 7feffd400700 1 -- 172.21.15.151:0/668470061 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7feff802e560 con 0x7feff8103460 2024-05-21T21:53:23.579 INFO:teuthology.orchestra.run.smithi151.stdout:163208757323 2024-05-21T21:53:23.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:22.156+0000 7feff4c00700 1 -- 172.21.15.151:0/668470061 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fefe80aad50 con 0x7feff8103460 2024-05-21T21:53:23.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:22.160+0000 7feffd400700 1 -- 172.21.15.151:0/668470061 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fefdc0b2880 msgr2=0x7fefdc0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:23.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:22.160+0000 7feffd400700 1 --2- 172.21.15.151:0/668470061 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fefdc0b2880 0x7fefdc0b4d40 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7feff8101570 tx=0x7fefe4009260 comp rx=0 tx=0).stop 2024-05-21T21:53:23.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:22.160+0000 7feffd400700 1 -- 172.21.15.151:0/668470061 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feff8103460 msgr2=0x7feff8075840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:23.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:22.160+0000 7feffd400700 1 --2- 172.21.15.151:0/668470061 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feff8103460 0x7feff8075840 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fefe8000c00 tx=0x7fefe80057c0 comp rx=0 tx=0).stop 2024-05-21T21:53:23.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:22.160+0000 7feffd400700 1 -- 172.21.15.151:0/668470061 shutdown_connections 2024-05-21T21:53:23.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:22.160+0000 7feffd400700 1 --2- 172.21.15.151:0/668470061 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fefdc0b2880 0x7fefdc0b4d40 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:23.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:22.160+0000 7feffd400700 1 --2- 172.21.15.151:0/668470061 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feff81094f0 0x7feff80762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:23.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:22.160+0000 7feffd400700 1 --2- 172.21.15.151:0/668470061 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feff8103460 0x7feff8075840 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:23.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:22.160+0000 7feffd400700 1 --2- 172.21.15.151:0/668470061 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feff8103e20 0x7feff8075d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:23.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:22.160+0000 7feffd400700 1 -- 172.21.15.151:0/668470061 >> 172.21.15.151:0/668470061 conn(0x7feff80fef50 msgr2=0x7feff80ff710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:53:23.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:22.160+0000 7feffd400700 1 -- 172.21.15.151:0/668470061 shutdown_connections 2024-05-21T21:53:23.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:22.160+0000 7feffd400700 1 -- 172.21.15.151:0/668470061 wait complete. 2024-05-21T21:53:24.425 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:53:24 smithi151 ceph-mon[31623]: pgmap v379: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 511 B/s wr, 12 op/s 2024-05-21T21:53:24.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:53:24 smithi184 ceph-mon[36942]: pgmap v379: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 511 B/s wr, 12 op/s 2024-05-21T21:53:24.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:53:24 smithi164 ceph-mon[36269]: pgmap v379: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 511 B/s wr, 12 op/s 2024-05-21T21:53:25.853 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:53:25.854 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:53:25.858 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:53:26.375 INFO:teuthology.orchestra.run.smithi151.stdout:283467841583 2024-05-21T21:53:26.376 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-05-21T21:53:26.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:53:26 smithi151 ceph-mon[31623]: pgmap v380: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-05-21T21:53:26.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:53:26 smithi184 ceph-mon[36942]: pgmap v380: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-05-21T21:53:26.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:53:26 smithi164 ceph-mon[36269]: pgmap v380: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-05-21T21:53:28.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:28.455+0000 7f8faae00700 1 -- 172.21.15.151:0/230963606 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8fa41037f0 msgr2=0x7f8fa4103bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:28.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:28.455+0000 7f8faae00700 1 --2- 172.21.15.151:0/230963606 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8fa41037f0 0x7f8fa4103bd0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8f940099c0 tx=0x7f8f94009cd0 comp rx=0 tx=0).stop 2024-05-21T21:53:28.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:28.455+0000 7f8faae00700 1 -- 172.21.15.151:0/230963606 shutdown_connections 2024-05-21T21:53:28.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:28.455+0000 7f8faae00700 1 --2- 172.21.15.151:0/230963606 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8fa41088f0 0x7f8fa4110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:28.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:28.455+0000 7f8faae00700 1 --2- 172.21.15.151:0/230963606 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8fa41037f0 0x7f8fa4103bd0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:28.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:28.455+0000 7f8faae00700 1 --2- 172.21.15.151:0/230963606 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8fa41041a0 0x7f8fa41081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:28.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:28.455+0000 7f8faae00700 1 -- 172.21.15.151:0/230963606 >> 172.21.15.151:0/230963606 conn(0x7f8fa407a6a0 msgr2=0x7f8fa407aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:53:28.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:28.455+0000 7f8faae00700 1 -- 172.21.15.151:0/230963606 shutdown_connections 2024-05-21T21:53:28.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:28.455+0000 7f8faae00700 1 -- 172.21.15.151:0/230963606 wait complete. 2024-05-21T21:53:28.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:28.457+0000 7f8faae00700 1 Processor -- start 2024-05-21T21:53:28.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:28.457+0000 7f8faae00700 1 -- start start 2024-05-21T21:53:28.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:28.457+0000 7f8faae00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8fa41037f0 0x7f8fa419bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:28.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:28.457+0000 7f8faae00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8fa41041a0 0x7f8fa419c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:28.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:28.457+0000 7f8faae00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8fa41088f0 0x7f8fa41a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:28.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:28.457+0000 7f8faae00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8fa4113730 con 0x7f8fa41088f0 2024-05-21T21:53:28.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:28.457+0000 7f8faae00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8fa41135b0 con 0x7f8fa41037f0 2024-05-21T21:53:28.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:28.457+0000 7f8faae00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8fa41138b0 con 0x7f8fa41041a0 2024-05-21T21:53:28.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:28.457+0000 7f8fa3e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8fa41037f0 0x7f8fa419bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:28.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:28.458+0000 7f8fa3e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8fa41037f0 0x7f8fa419bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:34740/0 (socket says 172.21.15.151:34740) 2024-05-21T21:53:28.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:28.458+0000 7f8fa3e00700 1 -- 172.21.15.151:0/2467308613 learned_addr learned my addr 172.21.15.151:0/2467308613 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:53:28.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:28.458+0000 7f8fa3400700 1 --2- 172.21.15.151:0/2467308613 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8fa41041a0 0x7f8fa419c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:28.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:28.458+0000 7f8fa8e00700 1 --2- 172.21.15.151:0/2467308613 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8fa41088f0 0x7f8fa41a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:28.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:28.458+0000 7f8fa3400700 1 -- 172.21.15.151:0/2467308613 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8fa41037f0 msgr2=0x7f8fa419bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:28.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:28.458+0000 7f8fa3400700 1 --2- 172.21.15.151:0/2467308613 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8fa41037f0 0x7f8fa419bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:28.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:28.458+0000 7f8fa3400700 1 -- 172.21.15.151:0/2467308613 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8fa41088f0 msgr2=0x7f8fa41a06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:28.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:28.458+0000 7f8fa3400700 1 --2- 172.21.15.151:0/2467308613 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8fa41088f0 0x7f8fa41a06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:28.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:28.458+0000 7f8fa3400700 1 -- 172.21.15.151:0/2467308613 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fa41a0de0 con 0x7f8fa41041a0 2024-05-21T21:53:28.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:28.458+0000 7f8fa3e00700 1 --2- 172.21.15.151:0/2467308613 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8fa41037f0 0x7f8fa419bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:53:28.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:28.458+0000 7f8fa3400700 1 --2- 172.21.15.151:0/2467308613 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8fa41041a0 0x7f8fa419c310 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8f9000e9f0 tx=0x7f8f9000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:53:28.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:28.459+0000 7f8fa0c00700 1 -- 172.21.15.151:0/2467308613 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f90004220 con 0x7f8fa41041a0 2024-05-21T21:53:28.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:28.459+0000 7f8fa0c00700 1 -- 172.21.15.151:0/2467308613 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f900043c0 con 0x7f8fa41041a0 2024-05-21T21:53:28.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:28.459+0000 7f8faae00700 1 -- 172.21.15.151:0/2467308613 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fa41a10d0 con 0x7f8fa41041a0 2024-05-21T21:53:28.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:28.459+0000 7f8fa0c00700 1 -- 172.21.15.151:0/2467308613 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f90005020 con 0x7f8fa41041a0 2024-05-21T21:53:28.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:28.459+0000 7f8faae00700 1 -- 172.21.15.151:0/2467308613 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fa41a8950 con 0x7f8fa41041a0 2024-05-21T21:53:28.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:28.460+0000 7f8faae00700 1 -- 172.21.15.151:0/2467308613 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fa406a230 con 0x7f8fa41041a0 2024-05-21T21:53:28.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:28.462+0000 7f8fa0c00700 1 -- 172.21.15.151:0/2467308613 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f8f90004750 con 0x7f8fa41041a0 2024-05-21T21:53:28.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:28.464+0000 7f8fa0c00700 1 --2- 172.21.15.151:0/2467308613 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f8f880b2900 0x7f8f880b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:28.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:28.464+0000 7f8fa0c00700 1 -- 172.21.15.151:0/2467308613 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f8f900e20d0 con 0x7f8fa41041a0 2024-05-21T21:53:28.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:28.464+0000 7f8fa3e00700 1 --2- 172.21.15.151:0/2467308613 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f8f880b2900 0x7f8f880b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:28.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:28.465+0000 7f8fa3e00700 1 --2- 172.21.15.151:0/2467308613 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f8f880b2900 0x7f8f880b4dc0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8f94009990 tx=0x7f8f94005ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:53:28.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:28.468+0000 7f8fa0c00700 1 -- 172.21.15.151:0/2467308613 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8f900a9df0 con 0x7f8fa41041a0 2024-05-21T21:53:28.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:53:28 smithi151 ceph-mon[31623]: pgmap v381: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-21T21:53:28.708 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:28.708+0000 7f8faae00700 1 -- 172.21.15.151:0/2467308613 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f8fa419d130 con 0x7f8fa41041a0 2024-05-21T21:53:28.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:28.709+0000 7f8fa0c00700 1 -- 172.21.15.151:0/2467308613 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f8fa419d130 con 0x7f8fa41041a0 2024-05-21T21:53:28.710 INFO:teuthology.orchestra.run.smithi151.stdout:137438953553 2024-05-21T21:53:28.713 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:28.713+0000 7f8faae00700 1 -- 172.21.15.151:0/2467308613 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f8f880b2900 msgr2=0x7f8f880b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:28.713 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:28.713+0000 7f8faae00700 1 --2- 172.21.15.151:0/2467308613 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f8f880b2900 0x7f8f880b4dc0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8f94009990 tx=0x7f8f94005ab0 comp rx=0 tx=0).stop 2024-05-21T21:53:28.714 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:28.713+0000 7f8faae00700 1 -- 172.21.15.151:0/2467308613 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8fa41041a0 msgr2=0x7f8fa419c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:28.714 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:28.713+0000 7f8faae00700 1 --2- 172.21.15.151:0/2467308613 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8fa41041a0 0x7f8fa419c310 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8f9000e9f0 tx=0x7f8f9000ed00 comp rx=0 tx=0).stop 2024-05-21T21:53:28.714 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:28.713+0000 7f8faae00700 1 -- 172.21.15.151:0/2467308613 shutdown_connections 2024-05-21T21:53:28.714 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:28.713+0000 7f8faae00700 1 --2- 172.21.15.151:0/2467308613 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f8f880b2900 0x7f8f880b4dc0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:28.714 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:28.713+0000 7f8faae00700 1 --2- 172.21.15.151:0/2467308613 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8fa41037f0 0x7f8fa419bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:28.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:28.713+0000 7f8faae00700 1 --2- 172.21.15.151:0/2467308613 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8fa41088f0 0x7f8fa41a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:28.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:28.713+0000 7f8faae00700 1 --2- 172.21.15.151:0/2467308613 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8fa41041a0 0x7f8fa419c310 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:28.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:28.713+0000 7f8faae00700 1 -- 172.21.15.151:0/2467308613 >> 172.21.15.151:0/2467308613 conn(0x7f8fa407a6a0 msgr2=0x7f8fa41018d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:53:28.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:28.713+0000 7f8faae00700 1 -- 172.21.15.151:0/2467308613 shutdown_connections 2024-05-21T21:53:28.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:28.713+0000 7f8faae00700 1 -- 172.21.15.151:0/2467308613 wait complete. 2024-05-21T21:53:28.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:53:28 smithi184 ceph-mon[36942]: pgmap v381: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-21T21:53:28.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:53:28 smithi164 ceph-mon[36269]: pgmap v381: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-21T21:53:29.546 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:29.546+0000 7f9b1e600700 1 -- 172.21.15.151:0/4227595144 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b1806bc30 msgr2=0x7f9b1806c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:29.547 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:29.546+0000 7f9b1e600700 1 --2- 172.21.15.151:0/4227595144 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b1806bc30 0x7f9b1806c0b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9b04009990 tx=0x7f9b04009ca0 comp rx=0 tx=0).stop 2024-05-21T21:53:29.547 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:29.546+0000 7f9b1e600700 1 -- 172.21.15.151:0/4227595144 shutdown_connections 2024-05-21T21:53:29.547 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:29.546+0000 7f9b1e600700 1 --2- 172.21.15.151:0/4227595144 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b181095a0 0x7f9b18109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:29.547 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:29.546+0000 7f9b1e600700 1 --2- 172.21.15.151:0/4227595144 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9b1806c5f0 0x7f9b181103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:29.547 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:29.546+0000 7f9b1e600700 1 --2- 172.21.15.151:0/4227595144 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b1806bc30 0x7f9b1806c0b0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:29.547 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:29.546+0000 7f9b1e600700 1 -- 172.21.15.151:0/4227595144 >> 172.21.15.151:0/4227595144 conn(0x7f9b180793b0 msgr2=0x7f9b180797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:53:29.547 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:29.546+0000 7f9b1e600700 1 -- 172.21.15.151:0/4227595144 shutdown_connections 2024-05-21T21:53:29.547 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:29.546+0000 7f9b1e600700 1 -- 172.21.15.151:0/4227595144 wait complete. 2024-05-21T21:53:29.548 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:29.547+0000 7f9b1e600700 1 Processor -- start 2024-05-21T21:53:29.548 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:29.547+0000 7f9b1e600700 1 -- start start 2024-05-21T21:53:29.548 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:29.548+0000 7f9b1e600700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b1806bc30 0x7f9b18102500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:29.548 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:29.548+0000 7f9b1e600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9b1806c5f0 0x7f9b18102a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:29.548 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:29.548+0000 7f9b1e600700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b181095a0 0x7f9b18102f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:29.548 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:29.548+0000 7f9b1e600700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9b18113700 con 0x7f9b1806c5f0 2024-05-21T21:53:29.548 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:29.548+0000 7f9b1e600700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9b18113580 con 0x7f9b1806bc30 2024-05-21T21:53:29.548 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:29.548+0000 7f9b1e600700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9b18113880 con 0x7f9b181095a0 2024-05-21T21:53:29.549 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:29.548+0000 7f9b16a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9b1806c5f0 0x7f9b18102a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:29.550 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:29.548+0000 7f9b17400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b1806bc30 0x7f9b18102500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:29.550 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:29.548+0000 7f9b17400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b1806bc30 0x7f9b18102500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:34762/0 (socket says 172.21.15.151:34762) 2024-05-21T21:53:29.550 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:29.548+0000 7f9b16a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9b1806c5f0 0x7f9b18102a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:39874/0 (socket says 172.21.15.151:39874) 2024-05-21T21:53:29.550 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:29.548+0000 7f9b17400700 1 -- 172.21.15.151:0/3919187780 learned_addr learned my addr 172.21.15.151:0/3919187780 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:53:29.550 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:29.548+0000 7f9b17e00700 1 --2- 172.21.15.151:0/3919187780 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b181095a0 0x7f9b18102f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:29.550 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:29.549+0000 7f9b17400700 1 -- 172.21.15.151:0/3919187780 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b181095a0 msgr2=0x7f9b18102f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:29.550 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:29.549+0000 7f9b17400700 1 --2- 172.21.15.151:0/3919187780 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b181095a0 0x7f9b18102f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:29.550 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:29.549+0000 7f9b17400700 1 -- 172.21.15.151:0/3919187780 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9b1806c5f0 msgr2=0x7f9b18102a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:29.550 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:29.549+0000 7f9b17400700 1 --2- 172.21.15.151:0/3919187780 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9b1806c5f0 0x7f9b18102a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:29.550 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:29.549+0000 7f9b17400700 1 -- 172.21.15.151:0/3919187780 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b181a81a0 con 0x7f9b1806bc30 2024-05-21T21:53:29.550 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:29.549+0000 7f9b17400700 1 --2- 172.21.15.151:0/3919187780 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b1806bc30 0x7f9b18102500 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9b0800d950 tx=0x7f9b0800dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:53:29.550 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:29.549+0000 7f9b0fe00700 1 -- 172.21.15.151:0/3919187780 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b08016070 con 0x7f9b1806bc30 2024-05-21T21:53:29.551 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:29.549+0000 7f9b0fe00700 1 -- 172.21.15.151:0/3919187780 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b0800fd40 con 0x7f9b1806bc30 2024-05-21T21:53:29.551 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:29.549+0000 7f9b1e600700 1 -- 172.21.15.151:0/3919187780 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b181a8490 con 0x7f9b1806bc30 2024-05-21T21:53:29.551 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:29.549+0000 7f9b0fe00700 1 -- 172.21.15.151:0/3919187780 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b0800e3b0 con 0x7f9b1806bc30 2024-05-21T21:53:29.551 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:29.549+0000 7f9b1e600700 1 -- 172.21.15.151:0/3919187780 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b181a89d0 con 0x7f9b1806bc30 2024-05-21T21:53:29.551 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:29.551+0000 7f9b1e600700 1 -- 172.21.15.151:0/3919187780 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b1806a230 con 0x7f9b1806bc30 2024-05-21T21:53:29.552 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:29.553+0000 7f9b0fe00700 1 -- 172.21.15.151:0/3919187780 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f9b08019070 con 0x7f9b1806bc30 2024-05-21T21:53:29.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:29.554+0000 7f9b0fe00700 1 --2- 172.21.15.151:0/3919187780 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f9afc0b2880 0x7f9afc0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:29.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:29.554+0000 7f9b0fe00700 1 -- 172.21.15.151:0/3919187780 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f9b080e1c90 con 0x7f9b1806bc30 2024-05-21T21:53:29.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:29.554+0000 7f9b16a00700 1 --2- 172.21.15.151:0/3919187780 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f9afc0b2880 0x7f9afc0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:29.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:29.555+0000 7f9b16a00700 1 --2- 172.21.15.151:0/3919187780 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f9afc0b2880 0x7f9afc0b4d40 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9b18079af0 tx=0x7f9b0400b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:53:29.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:29.558+0000 7f9b0fe00700 1 -- 172.21.15.151:0/3919187780 <== mon.2 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9b080a9970 con 0x7f9b1806bc30 2024-05-21T21:53:29.617 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:53:29 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/2467308613' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-21T21:53:29.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:53:29 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/2467308613' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-21T21:53:29.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:53:29 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/2467308613' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-21T21:53:29.797 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:29.796+0000 7f9b1e600700 1 -- 172.21.15.151:0/3919187780 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f9b1802e560 con 0x7f9b1806bc30 2024-05-21T21:53:29.797 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:29.797+0000 7f9b0fe00700 1 -- 172.21.15.151:0/3919187780 <== mon.2 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f9b080a9270 con 0x7f9b1806bc30 2024-05-21T21:53:29.797 INFO:teuthology.orchestra.run.smithi151.stdout:111669149783 2024-05-21T21:53:29.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:29.800+0000 7f9b1e600700 1 -- 172.21.15.151:0/3919187780 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f9afc0b2880 msgr2=0x7f9afc0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:29.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:29.800+0000 7f9b1e600700 1 --2- 172.21.15.151:0/3919187780 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f9afc0b2880 0x7f9afc0b4d40 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9b18079af0 tx=0x7f9b0400b540 comp rx=0 tx=0).stop 2024-05-21T21:53:29.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:29.800+0000 7f9b1e600700 1 -- 172.21.15.151:0/3919187780 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b1806bc30 msgr2=0x7f9b18102500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:29.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:29.800+0000 7f9b1e600700 1 --2- 172.21.15.151:0/3919187780 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b1806bc30 0x7f9b18102500 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9b0800d950 tx=0x7f9b0800dc60 comp rx=0 tx=0).stop 2024-05-21T21:53:29.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:29.800+0000 7f9b1e600700 1 -- 172.21.15.151:0/3919187780 shutdown_connections 2024-05-21T21:53:29.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:29.800+0000 7f9b1e600700 1 --2- 172.21.15.151:0/3919187780 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f9afc0b2880 0x7f9afc0b4d40 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:29.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:29.800+0000 7f9b1e600700 1 --2- 172.21.15.151:0/3919187780 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b1806bc30 0x7f9b18102500 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:29.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:29.800+0000 7f9b1e600700 1 --2- 172.21.15.151:0/3919187780 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b181095a0 0x7f9b18102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:29.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:29.800+0000 7f9b1e600700 1 --2- 172.21.15.151:0/3919187780 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9b1806c5f0 0x7f9b18102a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:29.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:29.800+0000 7f9b1e600700 1 -- 172.21.15.151:0/3919187780 >> 172.21.15.151:0/3919187780 conn(0x7f9b180793b0 msgr2=0x7f9b181012d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:53:29.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:29.801+0000 7f9b1e600700 1 -- 172.21.15.151:0/3919187780 shutdown_connections 2024-05-21T21:53:29.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:29.801+0000 7f9b1e600700 1 -- 172.21.15.151:0/3919187780 wait complete. 2024-05-21T21:53:30.605 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:53:30 smithi151 ceph-mon[31623]: pgmap v382: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-21T21:53:30.605 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:53:30 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/3919187780' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-21T21:53:30.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:53:30 smithi184 ceph-mon[36942]: pgmap v382: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-21T21:53:30.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:53:30 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/3919187780' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-21T21:53:30.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:53:30 smithi164 ceph-mon[36269]: pgmap v382: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-21T21:53:30.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:53:30 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/3919187780' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-21T21:53:31.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:31.266+0000 7f1b72000700 1 -- 172.21.15.151:0/2678696624 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1b6c1096a0 msgr2=0x7f1b6c109a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:31.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:31.266+0000 7f1b72000700 1 --2- 172.21.15.151:0/2678696624 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1b6c1096a0 0x7f1b6c109a80 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c009990 tx=0x7f1b5c009ca0 comp rx=0 tx=0).stop 2024-05-21T21:53:31.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:31.266+0000 7f1b72000700 1 -- 172.21.15.151:0/2678696624 shutdown_connections 2024-05-21T21:53:31.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:31.266+0000 7f1b72000700 1 --2- 172.21.15.151:0/2678696624 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b6c103f50 0x7f1b6c110430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:31.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:31.266+0000 7f1b72000700 1 --2- 172.21.15.151:0/2678696624 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1b6c1096a0 0x7f1b6c109a80 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:31.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:31.266+0000 7f1b72000700 1 --2- 172.21.15.151:0/2678696624 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b6c103590 0x7f1b6c103a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:31.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:31.266+0000 7f1b72000700 1 -- 172.21.15.151:0/2678696624 >> 172.21.15.151:0/2678696624 conn(0x7f1b6c07a6f0 msgr2=0x7f1b6c07ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:53:31.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:31.266+0000 7f1b72000700 1 -- 172.21.15.151:0/2678696624 shutdown_connections 2024-05-21T21:53:31.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:31.266+0000 7f1b72000700 1 -- 172.21.15.151:0/2678696624 wait complete. 2024-05-21T21:53:31.269 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:31.267+0000 7f1b72000700 1 Processor -- start 2024-05-21T21:53:31.269 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:31.267+0000 7f1b72000700 1 -- start start 2024-05-21T21:53:31.269 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:31.268+0000 7f1b72000700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b6c103590 0x7f1b6c19bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:31.269 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:31.268+0000 7f1b72000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b6c103f50 0x7f1b6c19c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:31.269 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:31.268+0000 7f1b72000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1b6c1096a0 0x7f1b6c1a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:31.269 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:31.268+0000 7f1b72000700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1b6c113770 con 0x7f1b6c1096a0 2024-05-21T21:53:31.269 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:31.268+0000 7f1b72000700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1b6c1135f0 con 0x7f1b6c103f50 2024-05-21T21:53:31.269 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:31.268+0000 7f1b72000700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1b6c1138f0 con 0x7f1b6c103590 2024-05-21T21:53:31.269 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:31.268+0000 7f1b6b400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b6c103590 0x7f1b6c19bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:31.269 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:31.268+0000 7f1b6aa00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b6c103f50 0x7f1b6c19c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:31.269 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:31.268+0000 7f1b6b400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b6c103590 0x7f1b6c19bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:47826/0 (socket says 172.21.15.151:47826) 2024-05-21T21:53:31.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:31.268+0000 7f1b6b400700 1 -- 172.21.15.151:0/180849389 learned_addr learned my addr 172.21.15.151:0/180849389 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:53:31.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:31.268+0000 7f1b6be00700 1 --2- 172.21.15.151:0/180849389 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1b6c1096a0 0x7f1b6c1a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:31.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:31.269+0000 7f1b6aa00700 1 -- 172.21.15.151:0/180849389 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b6c103590 msgr2=0x7f1b6c19bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:31.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:31.269+0000 7f1b6aa00700 1 --2- 172.21.15.151:0/180849389 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b6c103590 0x7f1b6c19bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:31.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:31.269+0000 7f1b6aa00700 1 -- 172.21.15.151:0/180849389 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1b6c1096a0 msgr2=0x7f1b6c1a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:31.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:31.269+0000 7f1b6aa00700 1 --2- 172.21.15.151:0/180849389 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1b6c1096a0 0x7f1b6c1a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:31.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:31.269+0000 7f1b6aa00700 1 -- 172.21.15.151:0/180849389 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b6c1a0dd0 con 0x7f1b6c103f50 2024-05-21T21:53:31.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:31.269+0000 7f1b6aa00700 1 --2- 172.21.15.151:0/180849389 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b6c103f50 0x7f1b6c19c300 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1b5800e9f0 tx=0x7f1b5800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:53:31.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:31.269+0000 7f1b63e00700 1 -- 172.21.15.151:0/180849389 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b5800cbd0 con 0x7f1b6c103f50 2024-05-21T21:53:31.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:31.269+0000 7f1b63e00700 1 -- 172.21.15.151:0/180849389 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b5800cd70 con 0x7f1b6c103f50 2024-05-21T21:53:31.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:31.269+0000 7f1b72000700 1 -- 172.21.15.151:0/180849389 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b6c1a10c0 con 0x7f1b6c103f50 2024-05-21T21:53:31.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:31.269+0000 7f1b63e00700 1 -- 172.21.15.151:0/180849389 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b58018680 con 0x7f1b6c103f50 2024-05-21T21:53:31.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:31.270+0000 7f1b72000700 1 -- 172.21.15.151:0/180849389 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b6c1a8940 con 0x7f1b6c103f50 2024-05-21T21:53:31.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:31.271+0000 7f1b72000700 1 -- 172.21.15.151:0/180849389 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b6c06a230 con 0x7f1b6c103f50 2024-05-21T21:53:31.276 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:31.276+0000 7f1b63e00700 1 -- 172.21.15.151:0/180849389 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f1b5804aab0 con 0x7f1b6c103f50 2024-05-21T21:53:31.279 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:31.278+0000 7f1b63e00700 1 --2- 172.21.15.151:0/180849389 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f1b500b2900 0x7f1b500b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:31.279 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:31.279+0000 7f1b63e00700 1 -- 172.21.15.151:0/180849389 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f1b580e2320 con 0x7f1b6c103f50 2024-05-21T21:53:31.279 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:31.279+0000 7f1b6b400700 1 --2- 172.21.15.151:0/180849389 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f1b500b2900 0x7f1b500b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:31.279 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:31.279+0000 7f1b63e00700 1 -- 172.21.15.151:0/180849389 <== mon.2 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1b580e27e0 con 0x7f1b6c103f50 2024-05-21T21:53:31.279 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:31.280+0000 7f1b6b400700 1 --2- 172.21.15.151:0/180849389 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f1b500b2900 0x7f1b500b4dc0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c006010 tx=0x7f1b5c00b280 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:53:31.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:31.568+0000 7f1b72000700 1 -- 172.21.15.151:0/180849389 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f1b6c1a8ba0 con 0x7f1b6c103f50 2024-05-21T21:53:31.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:31.568+0000 7f1b63e00700 1 -- 172.21.15.151:0/180849389 <== mon.2 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f1b580aa040 con 0x7f1b6c103f50 2024-05-21T21:53:31.569 INFO:teuthology.orchestra.run.smithi151.stdout:261993005110 2024-05-21T21:53:31.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:31.572+0000 7f1b72000700 1 -- 172.21.15.151:0/180849389 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f1b500b2900 msgr2=0x7f1b500b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:31.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:31.572+0000 7f1b72000700 1 --2- 172.21.15.151:0/180849389 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f1b500b2900 0x7f1b500b4dc0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c006010 tx=0x7f1b5c00b280 comp rx=0 tx=0).stop 2024-05-21T21:53:31.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:31.572+0000 7f1b72000700 1 -- 172.21.15.151:0/180849389 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b6c103f50 msgr2=0x7f1b6c19c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:31.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:31.572+0000 7f1b72000700 1 --2- 172.21.15.151:0/180849389 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b6c103f50 0x7f1b6c19c300 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1b5800e9f0 tx=0x7f1b5800ed00 comp rx=0 tx=0).stop 2024-05-21T21:53:31.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:31.572+0000 7f1b72000700 1 -- 172.21.15.151:0/180849389 shutdown_connections 2024-05-21T21:53:31.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:31.572+0000 7f1b72000700 1 --2- 172.21.15.151:0/180849389 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f1b500b2900 0x7f1b500b4dc0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:31.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:31.572+0000 7f1b72000700 1 --2- 172.21.15.151:0/180849389 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1b6c1096a0 0x7f1b6c1a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:31.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:31.572+0000 7f1b72000700 1 --2- 172.21.15.151:0/180849389 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b6c103590 0x7f1b6c19bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:31.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:31.572+0000 7f1b72000700 1 --2- 172.21.15.151:0/180849389 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b6c103f50 0x7f1b6c19c300 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:31.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:31.572+0000 7f1b72000700 1 -- 172.21.15.151:0/180849389 >> 172.21.15.151:0/180849389 conn(0x7f1b6c07a6f0 msgr2=0x7f1b6c101f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:53:31.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:31.572+0000 7f1b72000700 1 -- 172.21.15.151:0/180849389 shutdown_connections 2024-05-21T21:53:31.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:31.572+0000 7f1b72000700 1 -- 172.21.15.151:0/180849389 wait complete. 2024-05-21T21:53:32.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:53:32 smithi151 ceph-mon[31623]: pgmap v383: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-21T21:53:32.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:53:32 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/180849389' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-21T21:53:32.735 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:53:32 smithi184 ceph-mon[36942]: pgmap v383: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-21T21:53:32.737 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:53:32 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/180849389' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-21T21:53:32.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:53:32 smithi164 ceph-mon[36269]: pgmap v383: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-21T21:53:32.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:53:32 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/180849389' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-21T21:53:33.633 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:53:33 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/60292425' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-21T21:53:33.636 INFO:tasks.cephadm.ceph_manager.ceph:need seq 261993005106 got 261993005110 for osd.9 2024-05-21T21:53:33.636 DEBUG:teuthology.parallel:result is None 2024-05-21T21:53:33.702 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:32.114+0000 7fe8f0c00700 1 -- 172.21.15.151:0/3052181751 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe8ec06b220 msgr2=0x7fe8ec06b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:33.702 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:32.114+0000 7fe8f0c00700 1 --2- 172.21.15.151:0/3052181751 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe8ec06b220 0x7fe8ec06b6a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe8d8009990 tx=0x7fe8d8009ca0 comp rx=0 tx=0).stop 2024-05-21T21:53:33.702 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:32.114+0000 7fe8f0c00700 1 -- 172.21.15.151:0/3052181751 shutdown_connections 2024-05-21T21:53:33.703 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:32.114+0000 7fe8f0c00700 1 --2- 172.21.15.151:0/3052181751 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe8ec06bbe0 0x7fe8ec1103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:33.703 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:32.114+0000 7fe8f0c00700 1 --2- 172.21.15.151:0/3052181751 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8ec1095b0 0x7fe8ec109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:33.703 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:32.114+0000 7fe8f0c00700 1 --2- 172.21.15.151:0/3052181751 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe8ec06b220 0x7fe8ec06b6a0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:33.703 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:32.114+0000 7fe8f0c00700 1 -- 172.21.15.151:0/3052181751 >> 172.21.15.151:0/3052181751 conn(0x7fe8ec0793f0 msgr2=0x7fe8ec079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:53:33.703 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:32.114+0000 7fe8f0c00700 1 -- 172.21.15.151:0/3052181751 shutdown_connections 2024-05-21T21:53:33.703 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:32.114+0000 7fe8f0c00700 1 -- 172.21.15.151:0/3052181751 wait complete. 2024-05-21T21:53:33.703 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:32.116+0000 7fe8f0c00700 1 Processor -- start 2024-05-21T21:53:33.703 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:32.116+0000 7fe8f0c00700 1 -- start start 2024-05-21T21:53:33.703 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:32.116+0000 7fe8f0c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe8ec06b220 0x7fe8ec19be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:33.703 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:32.116+0000 7fe8f0c00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe8ec06bbe0 0x7fe8ec19c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:33.703 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:32.116+0000 7fe8f0c00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8ec1095b0 0x7fe8ec1a0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:33.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:32.116+0000 7fe8f0c00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe8ec113720 con 0x7fe8ec06b220 2024-05-21T21:53:33.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:32.116+0000 7fe8f0c00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe8ec1135a0 con 0x7fe8ec06bbe0 2024-05-21T21:53:33.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:32.116+0000 7fe8f0c00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe8ec1138a0 con 0x7fe8ec1095b0 2024-05-21T21:53:33.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:32.117+0000 7fe8e9e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe8ec06b220 0x7fe8ec19be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:33.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:32.117+0000 7fe8e9e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe8ec06b220 0x7fe8ec19be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:39932/0 (socket says 172.21.15.151:39932) 2024-05-21T21:53:33.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:32.117+0000 7fe8e9e00700 1 -- 172.21.15.151:0/60292425 learned_addr learned my addr 172.21.15.151:0/60292425 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:53:33.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:32.117+0000 7fe8e9400700 1 --2- 172.21.15.151:0/60292425 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe8ec06bbe0 0x7fe8ec19c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:33.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:32.117+0000 7fe8ea800700 1 --2- 172.21.15.151:0/60292425 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8ec1095b0 0x7fe8ec1a0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:33.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:32.117+0000 7fe8e9e00700 1 -- 172.21.15.151:0/60292425 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8ec1095b0 msgr2=0x7fe8ec1a0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:33.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:32.117+0000 7fe8e9e00700 1 --2- 172.21.15.151:0/60292425 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8ec1095b0 0x7fe8ec1a0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:33.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:32.117+0000 7fe8e9e00700 1 -- 172.21.15.151:0/60292425 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe8ec06bbe0 msgr2=0x7fe8ec19c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:33.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:32.117+0000 7fe8e9e00700 1 --2- 172.21.15.151:0/60292425 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe8ec06bbe0 0x7fe8ec19c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:33.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:32.117+0000 7fe8e9e00700 1 -- 172.21.15.151:0/60292425 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8ec1a0e60 con 0x7fe8ec06b220 2024-05-21T21:53:33.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:32.117+0000 7fe8e9e00700 1 --2- 172.21.15.151:0/60292425 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe8ec06b220 0x7fe8ec19be50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc00e9a0 tx=0x7fe8dc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:53:33.705 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:32.118+0000 7fe8e2a00700 1 -- 172.21.15.151:0/60292425 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc00ca10 con 0x7fe8ec06b220 2024-05-21T21:53:33.705 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:32.118+0000 7fe8f0c00700 1 -- 172.21.15.151:0/60292425 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8ec1a1150 con 0x7fe8ec06b220 2024-05-21T21:53:33.705 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:32.118+0000 7fe8e2a00700 1 -- 172.21.15.151:0/60292425 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8dc00cbb0 con 0x7fe8ec06b220 2024-05-21T21:53:33.705 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:32.118+0000 7fe8e2a00700 1 -- 172.21.15.151:0/60292425 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc018680 con 0x7fe8ec06b220 2024-05-21T21:53:33.705 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:32.118+0000 7fe8f0c00700 1 -- 172.21.15.151:0/60292425 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8ec1a89d0 con 0x7fe8ec06b220 2024-05-21T21:53:33.705 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:32.119+0000 7fe8f0c00700 1 -- 172.21.15.151:0/60292425 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8ec06a230 con 0x7fe8ec06b220 2024-05-21T21:53:33.705 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:32.121+0000 7fe8e2a00700 1 -- 172.21.15.151:0/60292425 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fe8dc010430 con 0x7fe8ec06b220 2024-05-21T21:53:33.705 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:32.123+0000 7fe8e2a00700 1 --2- 172.21.15.151:0/60292425 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fe8d00b2880 0x7fe8d00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:33.705 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:32.123+0000 7fe8e2a00700 1 -- 172.21.15.151:0/60292425 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fe8dc0e2360 con 0x7fe8ec06b220 2024-05-21T21:53:33.705 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:32.123+0000 7fe8e9400700 1 --2- 172.21.15.151:0/60292425 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fe8d00b2880 0x7fe8d00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:33.706 INFO:teuthology.orchestra.run.smithi151.stdout:55834574947 2024-05-21T21:53:33.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:32.124+0000 7fe8e9400700 1 --2- 172.21.15.151:0/60292425 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fe8d00b2880 0x7fe8d00b4d40 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec104530 tx=0x7fe8d8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:53:33.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:32.127+0000 7fe8e2a00700 1 -- 172.21.15.151:0/60292425 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe8dc0aa040 con 0x7fe8ec06b220 2024-05-21T21:53:33.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:32.372+0000 7fe8f0c00700 1 -- 172.21.15.151:0/60292425 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fe8ec1a8ba0 con 0x7fe8ec06b220 2024-05-21T21:53:33.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:32.373+0000 7fe8e2a00700 1 -- 172.21.15.151:0/60292425 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe8dc0b20c0 con 0x7fe8ec06b220 2024-05-21T21:53:33.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:32.375+0000 7fe8f0c00700 1 -- 172.21.15.151:0/60292425 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fe8d00b2880 msgr2=0x7fe8d00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:33.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:32.375+0000 7fe8f0c00700 1 --2- 172.21.15.151:0/60292425 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fe8d00b2880 0x7fe8d00b4d40 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec104530 tx=0x7fe8d8005fd0 comp rx=0 tx=0).stop 2024-05-21T21:53:33.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:32.375+0000 7fe8f0c00700 1 -- 172.21.15.151:0/60292425 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe8ec06b220 msgr2=0x7fe8ec19be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:33.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:32.375+0000 7fe8f0c00700 1 --2- 172.21.15.151:0/60292425 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe8ec06b220 0x7fe8ec19be50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc00e9a0 tx=0x7fe8dc00ecb0 comp rx=0 tx=0).stop 2024-05-21T21:53:33.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:32.376+0000 7fe8f0c00700 1 -- 172.21.15.151:0/60292425 shutdown_connections 2024-05-21T21:53:33.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:32.376+0000 7fe8f0c00700 1 --2- 172.21.15.151:0/60292425 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fe8d00b2880 0x7fe8d00b4d40 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:33.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:32.376+0000 7fe8f0c00700 1 --2- 172.21.15.151:0/60292425 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8ec1095b0 0x7fe8ec1a0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:33.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:32.376+0000 7fe8f0c00700 1 --2- 172.21.15.151:0/60292425 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe8ec06b220 0x7fe8ec19be50 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:33.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:32.376+0000 7fe8f0c00700 1 --2- 172.21.15.151:0/60292425 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe8ec06bbe0 0x7fe8ec19c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:33.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:32.376+0000 7fe8f0c00700 1 -- 172.21.15.151:0/60292425 >> 172.21.15.151:0/60292425 conn(0x7fe8ec0793f0 msgr2=0x7fe8ec10e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:53:33.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:32.376+0000 7fe8f0c00700 1 -- 172.21.15.151:0/60292425 shutdown_connections 2024-05-21T21:53:33.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:32.376+0000 7fe8f0c00700 1 -- 172.21.15.151:0/60292425 wait complete. 2024-05-21T21:53:33.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:53:33 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/60292425' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-21T21:53:33.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:53:33 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/60292425' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-21T21:53:34.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:53:34 smithi151 ceph-mon[31623]: pgmap v384: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-21T21:53:34.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:53:34 smithi184 ceph-mon[36942]: pgmap v384: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-21T21:53:34.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:53:34 smithi164 ceph-mon[36269]: pgmap v384: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-21T21:53:35.047 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:53:35.047 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:53:35.091 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757319 got 163208757323 for osd.5 2024-05-21T21:53:35.091 DEBUG:teuthology.parallel:result is None 2024-05-21T21:53:35.582 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149778 got 111669149783 for osd.3 2024-05-21T21:53:35.582 DEBUG:teuthology.parallel:result is None 2024-05-21T21:53:36.587 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:53:36 smithi151 ceph-mon[31623]: pgmap v385: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-05-21T21:53:36.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:53:36 smithi184 ceph-mon[36942]: pgmap v385: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-05-21T21:53:36.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:53:36 smithi164 ceph-mon[36269]: pgmap v385: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-05-21T21:53:36.840 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.467+0000 7f9147c00700 1 -- 172.21.15.151:0/3398354200 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9140108500 msgr2=0x7f91401088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:36.840 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.467+0000 7f9147c00700 1 --2- 172.21.15.151:0/3398354200 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9140108500 0x7f91401088e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f91340098f0 tx=0x7f9134009c00 comp rx=0 tx=0).stop 2024-05-21T21:53:36.840 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.467+0000 7f9147c00700 1 -- 172.21.15.151:0/3398354200 shutdown_connections 2024-05-21T21:53:36.840 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.467+0000 7f9147c00700 1 --2- 172.21.15.151:0/3398354200 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9140108500 0x7f91401088e0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:36.840 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.467+0000 7f9147c00700 1 --2- 172.21.15.151:0/3398354200 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9140103f90 0x7f9140110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:36.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.467+0000 7f9147c00700 1 --2- 172.21.15.151:0/3398354200 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f91401035f0 0x7f9140103a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:36.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.467+0000 7f9147c00700 1 -- 172.21.15.151:0/3398354200 >> 172.21.15.151:0/3398354200 conn(0x7f914007a6a0 msgr2=0x7f914007aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:53:36.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.467+0000 7f9147c00700 1 -- 172.21.15.151:0/3398354200 shutdown_connections 2024-05-21T21:53:36.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.467+0000 7f9147c00700 1 -- 172.21.15.151:0/3398354200 wait complete. 2024-05-21T21:53:36.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.468+0000 7f9147c00700 1 Processor -- start 2024-05-21T21:53:36.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.469+0000 7f9147c00700 1 -- start start 2024-05-21T21:53:36.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.469+0000 7f9147c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f91401035f0 0x7f914019bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:36.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.469+0000 7f9147c00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9140103f90 0x7f914019c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:36.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.469+0000 7f9147c00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9140108500 0x7f91401a0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:36.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.469+0000 7f9147c00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9140113790 con 0x7f91401035f0 2024-05-21T21:53:36.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.469+0000 7f9147c00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9140113610 con 0x7f9140103f90 2024-05-21T21:53:36.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.469+0000 7f9147c00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9140113910 con 0x7f9140108500 2024-05-21T21:53:36.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.469+0000 7f9145200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f91401035f0 0x7f914019bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:36.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.469+0000 7f9145200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f91401035f0 0x7f914019bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:58154/0 (socket says 172.21.15.151:58154) 2024-05-21T21:53:36.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.469+0000 7f9145200700 1 -- 172.21.15.151:0/1482219959 learned_addr learned my addr 172.21.15.151:0/1482219959 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:53:36.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.469+0000 7f9145c00700 1 --2- 172.21.15.151:0/1482219959 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9140108500 0x7f91401a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:36.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.469+0000 7f913fe00700 1 --2- 172.21.15.151:0/1482219959 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9140103f90 0x7f914019c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:36.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.470+0000 7f9145200700 1 -- 172.21.15.151:0/1482219959 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9140108500 msgr2=0x7f91401a0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:36.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.470+0000 7f9145200700 1 --2- 172.21.15.151:0/1482219959 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9140108500 0x7f91401a0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:36.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.470+0000 7f9145200700 1 -- 172.21.15.151:0/1482219959 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9140103f90 msgr2=0x7f914019c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:36.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.470+0000 7f9145200700 1 --2- 172.21.15.151:0/1482219959 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9140103f90 0x7f914019c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:36.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.470+0000 7f9145200700 1 -- 172.21.15.151:0/1482219959 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91401a0eb0 con 0x7f91401035f0 2024-05-21T21:53:36.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.470+0000 7f9145200700 1 --2- 172.21.15.151:0/1482219959 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f91401035f0 0x7f914019bea0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9134009230 tx=0x7f913400f990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:53:36.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.470+0000 7f913d600700 1 -- 172.21.15.151:0/1482219959 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9134027070 con 0x7f91401035f0 2024-05-21T21:53:36.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.470+0000 7f913d600700 1 -- 172.21.15.151:0/1482219959 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91340055f0 con 0x7f91401035f0 2024-05-21T21:53:36.843 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.470+0000 7f913d600700 1 -- 172.21.15.151:0/1482219959 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9134004930 con 0x7f91401035f0 2024-05-21T21:53:36.843 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.470+0000 7f9147c00700 1 -- 172.21.15.151:0/1482219959 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91401a1140 con 0x7f91401035f0 2024-05-21T21:53:36.843 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.471+0000 7f9147c00700 1 -- 172.21.15.151:0/1482219959 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91401a8a20 con 0x7f91401035f0 2024-05-21T21:53:36.845 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.472+0000 7f9147c00700 1 -- 172.21.15.151:0/1482219959 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f914006a230 con 0x7f91401035f0 2024-05-21T21:53:36.845 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.474+0000 7f913d600700 1 -- 172.21.15.151:0/1482219959 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f913402a070 con 0x7f91401035f0 2024-05-21T21:53:36.845 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.475+0000 7f913d600700 1 --2- 172.21.15.151:0/1482219959 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f91280b2900 0x7f91280b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:36.845 INFO:teuthology.orchestra.run.smithi151.stdout:34359738473 2024-05-21T21:53:36.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.476+0000 7f913d600700 1 -- 172.21.15.151:0/1482219959 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f91340e2ec0 con 0x7f91401035f0 2024-05-21T21:53:36.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.476+0000 7f913fe00700 1 --2- 172.21.15.151:0/1482219959 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f91280b2900 0x7f91280b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:36.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.477+0000 7f913fe00700 1 --2- 172.21.15.151:0/1482219959 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f91280b2900 0x7f91280b4dc0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f91401108b0 tx=0x7f912c009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:53:36.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.478+0000 7f913d600700 1 -- 172.21.15.151:0/1482219959 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9134031020 con 0x7f91401035f0 2024-05-21T21:53:36.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.730+0000 7f9147c00700 1 -- 172.21.15.151:0/1482219959 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f91401a8bf0 con 0x7f91401035f0 2024-05-21T21:53:36.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.730+0000 7f913d600700 1 -- 172.21.15.151:0/1482219959 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9134013030 con 0x7f91401035f0 2024-05-21T21:53:36.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.733+0000 7f9147c00700 1 -- 172.21.15.151:0/1482219959 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f91280b2900 msgr2=0x7f91280b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:36.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.733+0000 7f9147c00700 1 --2- 172.21.15.151:0/1482219959 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f91280b2900 0x7f91280b4dc0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f91401108b0 tx=0x7f912c009310 comp rx=0 tx=0).stop 2024-05-21T21:53:36.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.733+0000 7f9147c00700 1 -- 172.21.15.151:0/1482219959 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f91401035f0 msgr2=0x7f914019bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:36.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.733+0000 7f9147c00700 1 --2- 172.21.15.151:0/1482219959 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f91401035f0 0x7f914019bea0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9134009230 tx=0x7f913400f990 comp rx=0 tx=0).stop 2024-05-21T21:53:36.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.733+0000 7f9147c00700 1 -- 172.21.15.151:0/1482219959 shutdown_connections 2024-05-21T21:53:36.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.733+0000 7f9147c00700 1 --2- 172.21.15.151:0/1482219959 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f91280b2900 0x7f91280b4dc0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:36.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.733+0000 7f9147c00700 1 --2- 172.21.15.151:0/1482219959 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9140108500 0x7f91401a0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:36.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.733+0000 7f9147c00700 1 --2- 172.21.15.151:0/1482219959 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f91401035f0 0x7f914019bea0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:36.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.733+0000 7f9147c00700 1 --2- 172.21.15.151:0/1482219959 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9140103f90 0x7f914019c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:36.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.733+0000 7f9147c00700 1 -- 172.21.15.151:0/1482219959 >> 172.21.15.151:0/1482219959 conn(0x7f914007a6a0 msgr2=0x7f9140101460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:53:36.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.733+0000 7f9147c00700 1 -- 172.21.15.151:0/1482219959 shutdown_connections 2024-05-21T21:53:36.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.733+0000 7f9147c00700 1 -- 172.21.15.151:0/1482219959 wait complete. 2024-05-21T21:53:37.153 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.386+0000 7f5284c00700 1 -- 172.21.15.151:0/2065493655 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5280075770 msgr2=0x7f5280075b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:37.153 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.386+0000 7f5284c00700 1 --2- 172.21.15.151:0/2065493655 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5280075770 0x7f5280075b50 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5270009990 tx=0x7f5270009ca0 comp rx=0 tx=0).stop 2024-05-21T21:53:37.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.386+0000 7f5284c00700 1 -- 172.21.15.151:0/2065493655 shutdown_connections 2024-05-21T21:53:37.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.386+0000 7f5284c00700 1 --2- 172.21.15.151:0/2065493655 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5280075770 0x7f5280075b50 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:37.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.386+0000 7f5284c00700 1 --2- 172.21.15.151:0/2065493655 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5280114710 0x7f5280116b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:37.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.386+0000 7f5284c00700 1 --2- 172.21.15.151:0/2065493655 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5280076090 0x7f52801141d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:37.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.386+0000 7f5284c00700 1 -- 172.21.15.151:0/2065493655 >> 172.21.15.151:0/2065493655 conn(0x7f528007a6a0 msgr2=0x7f528007aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:53:37.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.386+0000 7f5284c00700 1 -- 172.21.15.151:0/2065493655 shutdown_connections 2024-05-21T21:53:37.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.386+0000 7f5284c00700 1 -- 172.21.15.151:0/2065493655 wait complete. 2024-05-21T21:53:37.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.387+0000 7f5284c00700 1 Processor -- start 2024-05-21T21:53:37.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.387+0000 7f5284c00700 1 -- start start 2024-05-21T21:53:37.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.387+0000 7f5284c00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5280075770 0x7f52801a01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:37.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.387+0000 7f5284c00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5280076090 0x7f52801a0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:37.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.388+0000 7f5284c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5280114710 0x7f52801a4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:37.155 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.388+0000 7f5284c00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5280119d20 con 0x7f5280114710 2024-05-21T21:53:37.155 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.388+0000 7f5284c00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5280119ba0 con 0x7f5280076090 2024-05-21T21:53:37.155 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.388+0000 7f5284c00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5280119ea0 con 0x7f5280075770 2024-05-21T21:53:37.155 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.388+0000 7f527e800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5280114710 0x7f52801a4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:37.155 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.388+0000 7f527d400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5280076090 0x7f52801a0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:37.155 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.388+0000 7f527d400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5280076090 0x7f52801a0710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:51568/0 (socket says 172.21.15.151:51568) 2024-05-21T21:53:37.155 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.388+0000 7f527d400700 1 -- 172.21.15.151:0/213883570 learned_addr learned my addr 172.21.15.151:0/213883570 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:53:37.155 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.388+0000 7f527de00700 1 --2- 172.21.15.151:0/213883570 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5280075770 0x7f52801a01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:37.155 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.388+0000 7f527d400700 1 -- 172.21.15.151:0/213883570 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5280075770 msgr2=0x7f52801a01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:37.155 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.388+0000 7f527d400700 1 --2- 172.21.15.151:0/213883570 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5280075770 0x7f52801a01d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:37.155 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.388+0000 7f527d400700 1 -- 172.21.15.151:0/213883570 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5280114710 msgr2=0x7f52801a4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:37.155 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.388+0000 7f527d400700 1 --2- 172.21.15.151:0/213883570 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5280114710 0x7f52801a4aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:37.155 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.388+0000 7f527d400700 1 -- 172.21.15.151:0/213883570 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52801a51e0 con 0x7f5280076090 2024-05-21T21:53:37.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.388+0000 7f527e800700 1 --2- 172.21.15.151:0/213883570 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5280114710 0x7f52801a4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:53:37.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.389+0000 7f527d400700 1 --2- 172.21.15.151:0/213883570 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5280076090 0x7f52801a0710 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f526c00d950 tx=0x7f526c00dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:53:37.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.389+0000 7f5276a00700 1 -- 172.21.15.151:0/213883570 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f526c016070 con 0x7f5280076090 2024-05-21T21:53:37.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.389+0000 7f5276a00700 1 -- 172.21.15.151:0/213883570 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f526c00fd40 con 0x7f5280076090 2024-05-21T21:53:37.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.389+0000 7f5284c00700 1 -- 172.21.15.151:0/213883570 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52801a54d0 con 0x7f5280076090 2024-05-21T21:53:37.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.389+0000 7f5276a00700 1 -- 172.21.15.151:0/213883570 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f526c00e3b0 con 0x7f5280076090 2024-05-21T21:53:37.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.389+0000 7f5284c00700 1 -- 172.21.15.151:0/213883570 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52801acd50 con 0x7f5280076090 2024-05-21T21:53:37.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.390+0000 7f5284c00700 1 -- 172.21.15.151:0/213883570 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f528006a230 con 0x7f5280076090 2024-05-21T21:53:37.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.393+0000 7f5276a00700 1 -- 172.21.15.151:0/213883570 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f526c019070 con 0x7f5280076090 2024-05-21T21:53:37.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.395+0000 7f5276a00700 1 --2- 172.21.15.151:0/213883570 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f52640b2880 0x7f52640b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:37.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.395+0000 7f5276a00700 1 -- 172.21.15.151:0/213883570 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f526c0e1d40 con 0x7f5280076090 2024-05-21T21:53:37.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.395+0000 7f527de00700 1 --2- 172.21.15.151:0/213883570 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f52640b2880 0x7f52640b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:37.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.396+0000 7f527de00700 1 --2- 172.21.15.151:0/213883570 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f52640b2880 0x7f52640b4d40 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f527000b2e0 tx=0x7f5270005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:53:37.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.399+0000 7f5276a00700 1 -- 172.21.15.151:0/213883570 <== mon.2 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f526c0a9a20 con 0x7f5280076090 2024-05-21T21:53:37.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.648+0000 7f5284c00700 1 -- 172.21.15.151:0/213883570 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f52801acf50 con 0x7f5280076090 2024-05-21T21:53:37.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.649+0000 7f5276a00700 1 -- 172.21.15.151:0/213883570 <== mon.2 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f526c0a9320 con 0x7f5280076090 2024-05-21T21:53:37.157 INFO:teuthology.orchestra.run.smithi151.stdout:309237645357 2024-05-21T21:53:37.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.652+0000 7f5284c00700 1 -- 172.21.15.151:0/213883570 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f52640b2880 msgr2=0x7f52640b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:37.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.652+0000 7f5284c00700 1 --2- 172.21.15.151:0/213883570 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f52640b2880 0x7f52640b4d40 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f527000b2e0 tx=0x7f5270005960 comp rx=0 tx=0).stop 2024-05-21T21:53:37.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.652+0000 7f5284c00700 1 -- 172.21.15.151:0/213883570 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5280076090 msgr2=0x7f52801a0710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:37.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.652+0000 7f5284c00700 1 --2- 172.21.15.151:0/213883570 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5280076090 0x7f52801a0710 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f526c00d950 tx=0x7f526c00dc60 comp rx=0 tx=0).stop 2024-05-21T21:53:37.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.652+0000 7f5284c00700 1 -- 172.21.15.151:0/213883570 shutdown_connections 2024-05-21T21:53:37.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.652+0000 7f5284c00700 1 --2- 172.21.15.151:0/213883570 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f52640b2880 0x7f52640b4d40 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:37.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.652+0000 7f5284c00700 1 --2- 172.21.15.151:0/213883570 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5280114710 0x7f52801a4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:37.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.652+0000 7f5284c00700 1 --2- 172.21.15.151:0/213883570 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5280075770 0x7f52801a01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:37.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.652+0000 7f5284c00700 1 --2- 172.21.15.151:0/213883570 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5280076090 0x7f52801a0710 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:37.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.652+0000 7f5284c00700 1 -- 172.21.15.151:0/213883570 >> 172.21.15.151:0/213883570 conn(0x7f528007a6a0 msgr2=0x7f52801135b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:53:37.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.652+0000 7f5284c00700 1 -- 172.21.15.151:0/213883570 shutdown_connections 2024-05-21T21:53:37.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:36.652+0000 7f5284c00700 1 -- 172.21.15.151:0/213883570 wait complete. 2024-05-21T21:53:37.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:53:37 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/213883570' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-21T21:53:37.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:53:37 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/1482219959' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-21T21:53:37.737 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:53:37 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/213883570' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-21T21:53:37.737 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:53:37 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/1482219959' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-21T21:53:37.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:53:37 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/213883570' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-21T21:53:37.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:53:37 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/1482219959' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-21T21:53:38.300 INFO:tasks.cephadm.ceph_manager.ceph:need seq 137438953549 got 137438953553 for osd.4 2024-05-21T21:53:38.300 DEBUG:teuthology.parallel:result is None 2024-05-21T21:53:38.440 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574942 got 55834574947 for osd.1 2024-05-21T21:53:38.440 DEBUG:teuthology.parallel:result is None 2024-05-21T21:53:38.514 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:53:38 smithi151 ceph-mon[31623]: pgmap v386: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 10 op/s 2024-05-21T21:53:38.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:53:38 smithi184 ceph-mon[36942]: pgmap v386: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 10 op/s 2024-05-21T21:53:38.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:53:38 smithi164 ceph-mon[36269]: pgmap v386: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 10 op/s 2024-05-21T21:53:40.225 INFO:tasks.cephadm.ceph_manager.ceph:need seq 309237645353 got 309237645357 for osd.11 2024-05-21T21:53:40.226 DEBUG:teuthology.parallel:result is None 2024-05-21T21:53:40.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:53:40 smithi151 ceph-mon[31623]: pgmap v387: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-21T21:53:40.735 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:53:40 smithi184 ceph-mon[36942]: pgmap v387: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-21T21:53:40.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:53:40 smithi164 ceph-mon[36269]: pgmap v387: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-21T21:53:41.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:41.313+0000 7f6355800700 1 -- 172.21.15.151:0/1521805713 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f63501041a0 msgr2=0x7f63501081c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:41.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:41.313+0000 7f6355800700 1 --2- 172.21.15.151:0/1521805713 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f63501041a0 0x7f63501081c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f633c0099c0 tx=0x7f633c009cd0 comp rx=0 tx=0).stop 2024-05-21T21:53:41.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:41.313+0000 7f6355800700 1 -- 172.21.15.151:0/1521805713 shutdown_connections 2024-05-21T21:53:41.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:41.313+0000 7f6355800700 1 --2- 172.21.15.151:0/1521805713 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f63501088f0 0x7f6350110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:41.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:41.313+0000 7f6355800700 1 --2- 172.21.15.151:0/1521805713 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f63501037f0 0x7f6350103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:41.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:41.313+0000 7f6355800700 1 --2- 172.21.15.151:0/1521805713 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f63501041a0 0x7f63501081c0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:41.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:41.313+0000 7f6355800700 1 -- 172.21.15.151:0/1521805713 >> 172.21.15.151:0/1521805713 conn(0x7f635007a6a0 msgr2=0x7f635007aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:53:41.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:41.313+0000 7f6355800700 1 -- 172.21.15.151:0/1521805713 shutdown_connections 2024-05-21T21:53:41.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:41.313+0000 7f6355800700 1 -- 172.21.15.151:0/1521805713 wait complete. 2024-05-21T21:53:41.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:41.315+0000 7f6355800700 1 Processor -- start 2024-05-21T21:53:41.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:41.315+0000 7f6355800700 1 -- start start 2024-05-21T21:53:41.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:41.315+0000 7f6355800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f63501037f0 0x7f635019f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:41.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:41.315+0000 7f6355800700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f63501041a0 0x7f635019fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:41.604 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:41.315+0000 7f6355800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f63501088f0 0x7f63501999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:41.604 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:41.315+0000 7f6355800700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6350113720 con 0x7f63501037f0 2024-05-21T21:53:41.604 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:41.315+0000 7f6355800700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f63501135a0 con 0x7f63501088f0 2024-05-21T21:53:41.604 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:41.315+0000 7f6355800700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f63501138a0 con 0x7f63501041a0 2024-05-21T21:53:41.604 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:41.316+0000 7f634ea00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f63501037f0 0x7f635019f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:41.604 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:41.316+0000 7f634e000700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f63501041a0 0x7f635019fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:41.604 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:41.316+0000 7f634ea00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f63501037f0 0x7f635019f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:58168/0 (socket says 172.21.15.151:58168) 2024-05-21T21:53:41.604 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:41.316+0000 7f634ea00700 1 -- 172.21.15.151:0/620965011 learned_addr learned my addr 172.21.15.151:0/620965011 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:53:41.604 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:41.316+0000 7f634e000700 1 --2- 172.21.15.151:0/620965011 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f63501041a0 0x7f635019fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:55320/0 (socket says 172.21.15.151:55320) 2024-05-21T21:53:41.604 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:41.316+0000 7f634f400700 1 --2- 172.21.15.151:0/620965011 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f63501088f0 0x7f63501999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:41.604 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:41.316+0000 7f634e000700 1 -- 172.21.15.151:0/620965011 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f63501088f0 msgr2=0x7f63501999e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:41.604 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:41.316+0000 7f634e000700 1 --2- 172.21.15.151:0/620965011 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f63501088f0 0x7f63501999e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:41.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:41.316+0000 7f634e000700 1 -- 172.21.15.151:0/620965011 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f63501037f0 msgr2=0x7f635019f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:41.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:41.316+0000 7f634e000700 1 --2- 172.21.15.151:0/620965011 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f63501037f0 0x7f635019f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:41.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:41.316+0000 7f634e000700 1 -- 172.21.15.151:0/620965011 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f635019a140 con 0x7f63501041a0 2024-05-21T21:53:41.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:41.316+0000 7f634e000700 1 --2- 172.21.15.151:0/620965011 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f63501041a0 0x7f635019fd60 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f633c0098e0 tx=0x7f633c005280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:53:41.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:41.317+0000 7f6347400700 1 -- 172.21.15.151:0/620965011 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f633c026070 con 0x7f63501041a0 2024-05-21T21:53:41.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:41.317+0000 7f6347400700 1 -- 172.21.15.151:0/620965011 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f633c0056c0 con 0x7f63501041a0 2024-05-21T21:53:41.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:41.317+0000 7f6355800700 1 -- 172.21.15.151:0/620965011 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f635019a3d0 con 0x7f63501041a0 2024-05-21T21:53:41.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:41.317+0000 7f6347400700 1 -- 172.21.15.151:0/620965011 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f633c00f990 con 0x7f63501041a0 2024-05-21T21:53:41.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:41.317+0000 7f6355800700 1 -- 172.21.15.151:0/620965011 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63501a66f0 con 0x7f63501041a0 2024-05-21T21:53:41.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:41.318+0000 7f6355800700 1 -- 172.21.15.151:0/620965011 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f635010bac0 con 0x7f63501041a0 2024-05-21T21:53:41.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:41.320+0000 7f6347400700 1 -- 172.21.15.151:0/620965011 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f633c029070 con 0x7f63501041a0 2024-05-21T21:53:41.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:41.323+0000 7f6347400700 1 --2- 172.21.15.151:0/620965011 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f63340b2880 0x7f63340b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:41.606 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:41.323+0000 7f6347400700 1 -- 172.21.15.151:0/620965011 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f633c0e29d0 con 0x7f63501041a0 2024-05-21T21:53:41.606 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:41.323+0000 7f634ea00700 1 --2- 172.21.15.151:0/620965011 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f63340b2880 0x7f63340b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:41.606 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:41.324+0000 7f634ea00700 1 --2- 172.21.15.151:0/620965011 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f63340b2880 0x7f63340b4d40 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f635019ab20 tx=0x7f6340009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:53:41.606 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:41.324+0000 7f6347400700 1 -- 172.21.15.151:0/620965011 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f633c013070 con 0x7f63501041a0 2024-05-21T21:53:41.606 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:41.563+0000 7f6355800700 1 -- 172.21.15.151:0/620965011 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f635019af10 con 0x7f63501041a0 2024-05-21T21:53:41.606 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:41.564+0000 7f6347400700 1 -- 172.21.15.151:0/620965011 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f633c0aa740 con 0x7f63501041a0 2024-05-21T21:53:41.606 INFO:teuthology.orchestra.run.smithi151.stdout:210453397570 2024-05-21T21:53:41.607 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:41.567+0000 7f6355800700 1 -- 172.21.15.151:0/620965011 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f63340b2880 msgr2=0x7f63340b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:41.607 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:41.567+0000 7f6355800700 1 --2- 172.21.15.151:0/620965011 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f63340b2880 0x7f63340b4d40 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f635019ab20 tx=0x7f6340009260 comp rx=0 tx=0).stop 2024-05-21T21:53:41.607 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:41.567+0000 7f6355800700 1 -- 172.21.15.151:0/620965011 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f63501041a0 msgr2=0x7f635019fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:41.607 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:41.567+0000 7f6355800700 1 --2- 172.21.15.151:0/620965011 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f63501041a0 0x7f635019fd60 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f633c0098e0 tx=0x7f633c005280 comp rx=0 tx=0).stop 2024-05-21T21:53:41.607 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:41.567+0000 7f6355800700 1 -- 172.21.15.151:0/620965011 shutdown_connections 2024-05-21T21:53:41.607 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:41.567+0000 7f6355800700 1 --2- 172.21.15.151:0/620965011 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f63340b2880 0x7f63340b4d40 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:41.607 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:41.567+0000 7f6355800700 1 --2- 172.21.15.151:0/620965011 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f63501088f0 0x7f63501999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:41.607 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:41.567+0000 7f6355800700 1 --2- 172.21.15.151:0/620965011 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f63501037f0 0x7f635019f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:41.607 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:41.567+0000 7f6355800700 1 --2- 172.21.15.151:0/620965011 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f63501041a0 0x7f635019fd60 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:41.607 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:41.568+0000 7f6355800700 1 -- 172.21.15.151:0/620965011 >> 172.21.15.151:0/620965011 conn(0x7f635007a6a0 msgr2=0x7f6350101960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:53:41.607 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:41.568+0000 7f6355800700 1 -- 172.21.15.151:0/620965011 shutdown_connections 2024-05-21T21:53:41.608 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:41.568+0000 7f6355800700 1 -- 172.21.15.151:0/620965011 wait complete. 2024-05-21T21:53:42.100 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738468 got 34359738473 for osd.0 2024-05-21T21:53:42.100 DEBUG:teuthology.parallel:result is None 2024-05-21T21:53:42.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:53:42 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:53:42.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:53:42 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:53:42.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:53:42 smithi151 ceph-mon[31623]: pgmap v388: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-21T21:53:42.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:53:42 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/620965011' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-21T21:53:42.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:53:42 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:53:42.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:53:42 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:53:42.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:53:42 smithi164 ceph-mon[36269]: pgmap v388: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-21T21:53:42.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:53:42 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/620965011' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-21T21:53:42.419 INFO:tasks.cephadm.ceph_manager.ceph:need seq 210453397565 got 210453397570 for osd.7 2024-05-21T21:53:42.419 DEBUG:teuthology.parallel:result is None 2024-05-21T21:53:42.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:53:42 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:53:42.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:53:42 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:53:42.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:53:42 smithi184 ceph-mon[36942]: pgmap v388: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-21T21:53:42.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:53:42 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/620965011' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-21T21:53:42.672 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:53:43.058 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:43.058+0000 7f0d5b200700 1 -- 172.21.15.151:0/3626187594 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0d5c10b670 msgr2=0x7f0d5c10ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:43.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:43.058+0000 7f0d5b200700 1 --2- 172.21.15.151:0/3626187594 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0d5c10b670 0x7f0d5c10ba50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c009990 tx=0x7f0d4c009ca0 comp rx=0 tx=0).stop 2024-05-21T21:53:43.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:43.058+0000 7f0d5b200700 1 -- 172.21.15.151:0/3626187594 shutdown_connections 2024-05-21T21:53:43.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:43.058+0000 7f0d5b200700 1 --2- 172.21.15.151:0/3626187594 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0d5c10b670 0x7f0d5c10ba50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:43.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:43.058+0000 7f0d5b200700 1 --2- 172.21.15.151:0/3626187594 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d5c106030 0x7f0d5c112540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:43.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:43.058+0000 7f0d5b200700 1 --2- 172.21.15.151:0/3626187594 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0d5c105670 0x7f0d5c105af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:43.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:43.058+0000 7f0d5b200700 1 -- 172.21.15.151:0/3626187594 >> 172.21.15.151:0/3626187594 conn(0x7f0d5c101160 msgr2=0x7f0d5c103580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:53:43.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:43.058+0000 7f0d5b200700 1 -- 172.21.15.151:0/3626187594 shutdown_connections 2024-05-21T21:53:43.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:43.058+0000 7f0d5b200700 1 -- 172.21.15.151:0/3626187594 wait complete. 2024-05-21T21:53:43.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:43.059+0000 7f0d5b200700 1 Processor -- start 2024-05-21T21:53:43.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:43.059+0000 7f0d5b200700 1 -- start start 2024-05-21T21:53:43.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:43.060+0000 7f0d5b200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0d5c105670 0x7f0d5c1a1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:43.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:43.060+0000 7f0d5b200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0d5c106030 0x7f0d5c1a1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:43.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:43.060+0000 7f0d5b200700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d5c10b670 0x7f0d5c19baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:43.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:43.060+0000 7f0d5b200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0d5c115870 con 0x7f0d5c105670 2024-05-21T21:53:43.061 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:43.060+0000 7f0d5b200700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0d5c1156f0 con 0x7f0d5c106030 2024-05-21T21:53:43.061 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:43.060+0000 7f0d5b200700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0d5c1159f0 con 0x7f0d5c10b670 2024-05-21T21:53:43.061 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:43.060+0000 7f0d59e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0d5c105670 0x7f0d5c1a1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:43.061 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:43.060+0000 7f0d59e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0d5c105670 0x7f0d5c1a1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:58186/0 (socket says 172.21.15.151:58186) 2024-05-21T21:53:43.061 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:43.060+0000 7f0d59e00700 1 -- 172.21.15.151:0/2179191424 learned_addr learned my addr 172.21.15.151:0/2179191424 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:53:43.061 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:43.060+0000 7f0d59400700 1 --2- 172.21.15.151:0/2179191424 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0d5c106030 0x7f0d5c1a1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:43.061 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:43.060+0000 7f0d5a800700 1 --2- 172.21.15.151:0/2179191424 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d5c10b670 0x7f0d5c19baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:43.062 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:43.060+0000 7f0d59e00700 1 -- 172.21.15.151:0/2179191424 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d5c10b670 msgr2=0x7f0d5c19baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:43.062 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:43.060+0000 7f0d59e00700 1 --2- 172.21.15.151:0/2179191424 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d5c10b670 0x7f0d5c19baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:43.062 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:43.061+0000 7f0d59e00700 1 -- 172.21.15.151:0/2179191424 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0d5c106030 msgr2=0x7f0d5c1a1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:43.062 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:43.061+0000 7f0d59e00700 1 --2- 172.21.15.151:0/2179191424 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0d5c106030 0x7f0d5c1a1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:43.062 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:43.061+0000 7f0d59e00700 1 -- 172.21.15.151:0/2179191424 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d5c19c250 con 0x7f0d5c105670 2024-05-21T21:53:43.062 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:43.061+0000 7f0d59400700 1 --2- 172.21.15.151:0/2179191424 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0d5c106030 0x7f0d5c1a1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:53:43.062 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:43.061+0000 7f0d59e00700 1 --2- 172.21.15.151:0/2179191424 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0d5c105670 0x7f0d5c1a1930 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c000c00 tx=0x7f0d4c00bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:53:43.062 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:43.061+0000 7f0d52a00700 1 -- 172.21.15.151:0/2179191424 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d4c027070 con 0x7f0d5c105670 2024-05-21T21:53:43.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:43.061+0000 7f0d52a00700 1 -- 172.21.15.151:0/2179191424 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d4c022470 con 0x7f0d5c105670 2024-05-21T21:53:43.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:43.061+0000 7f0d5b200700 1 -- 172.21.15.151:0/2179191424 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d5c19c4e0 con 0x7f0d5c105670 2024-05-21T21:53:43.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:43.061+0000 7f0d52a00700 1 -- 172.21.15.151:0/2179191424 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d4c017650 con 0x7f0d5c105670 2024-05-21T21:53:43.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:43.061+0000 7f0d5b200700 1 -- 172.21.15.151:0/2179191424 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d5c1a8800 con 0x7f0d5c105670 2024-05-21T21:53:43.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:43.062+0000 7f0d5b200700 1 -- 172.21.15.151:0/2179191424 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d5c10dc00 con 0x7f0d5c105670 2024-05-21T21:53:43.065 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:43.065+0000 7f0d52a00700 1 -- 172.21.15.151:0/2179191424 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f0d4c02a070 con 0x7f0d5c105670 2024-05-21T21:53:43.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:43.066+0000 7f0d52a00700 1 --2- 172.21.15.151:0/2179191424 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f0d400b2880 0x7f0d400b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:43.067 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:43.067+0000 7f0d52a00700 1 -- 172.21.15.151:0/2179191424 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f0d4c0e33a0 con 0x7f0d5c105670 2024-05-21T21:53:43.067 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:43.067+0000 7f0d59400700 1 --2- 172.21.15.151:0/2179191424 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f0d400b2880 0x7f0d400b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:43.067 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:53:43 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:53:43.067 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:53:43 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:53:43.067 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:53:43 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:53:43.068 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:43.068+0000 7f0d59400700 1 --2- 172.21.15.151:0/2179191424 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f0d400b2880 0x7f0d400b4d40 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c1037c0 tx=0x7f0d48009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:53:43.070 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:43.069+0000 7f0d52a00700 1 -- 172.21.15.151:0/2179191424 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0d4c013070 con 0x7f0d5c105670 2024-05-21T21:53:43.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:53:43 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:53:43.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:53:43 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:53:43.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:53:43 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:53:43.306 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:43.305+0000 7f0d5b200700 1 -- 172.21.15.151:0/2179191424 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f0d5c19cfe0 con 0x7f0d5c105670 2024-05-21T21:53:43.306 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:43.306+0000 7f0d52a00700 1 -- 172.21.15.151:0/2179191424 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f0d4c0ab110 con 0x7f0d5c105670 2024-05-21T21:53:43.306 INFO:teuthology.orchestra.run.smithi151.stdout:77309411425 2024-05-21T21:53:43.309 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:43.309+0000 7f0d5b200700 1 -- 172.21.15.151:0/2179191424 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f0d400b2880 msgr2=0x7f0d400b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:43.309 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:43.309+0000 7f0d5b200700 1 --2- 172.21.15.151:0/2179191424 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f0d400b2880 0x7f0d400b4d40 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c1037c0 tx=0x7f0d48009260 comp rx=0 tx=0).stop 2024-05-21T21:53:43.309 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:43.309+0000 7f0d5b200700 1 -- 172.21.15.151:0/2179191424 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0d5c105670 msgr2=0x7f0d5c1a1930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:43.309 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:43.309+0000 7f0d5b200700 1 --2- 172.21.15.151:0/2179191424 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0d5c105670 0x7f0d5c1a1930 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c000c00 tx=0x7f0d4c00bfd0 comp rx=0 tx=0).stop 2024-05-21T21:53:43.310 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:43.309+0000 7f0d5b200700 1 -- 172.21.15.151:0/2179191424 shutdown_connections 2024-05-21T21:53:43.310 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:43.309+0000 7f0d5b200700 1 --2- 172.21.15.151:0/2179191424 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f0d400b2880 0x7f0d400b4d40 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:43.310 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:43.309+0000 7f0d5b200700 1 --2- 172.21.15.151:0/2179191424 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d5c10b670 0x7f0d5c19baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:43.310 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:43.309+0000 7f0d5b200700 1 --2- 172.21.15.151:0/2179191424 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0d5c105670 0x7f0d5c1a1930 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:43.310 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:43.309+0000 7f0d5b200700 1 --2- 172.21.15.151:0/2179191424 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0d5c106030 0x7f0d5c1a1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:43.310 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:43.309+0000 7f0d5b200700 1 -- 172.21.15.151:0/2179191424 >> 172.21.15.151:0/2179191424 conn(0x7f0d5c101160 msgr2=0x7f0d5c1018b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:53:43.310 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:43.309+0000 7f0d5b200700 1 -- 172.21.15.151:0/2179191424 shutdown_connections 2024-05-21T21:53:43.310 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:43.309+0000 7f0d5b200700 1 -- 172.21.15.151:0/2179191424 wait complete. 2024-05-21T21:53:43.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:53:43 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:53:43.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:53:43 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:53:43.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:53:43 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:53:44.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:53:44 smithi164 ceph-mon[36269]: pgmap v389: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-21T21:53:44.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:53:44 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/2179191424' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-21T21:53:44.288 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:53:44 smithi151 ceph-mon[31623]: pgmap v389: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-21T21:53:44.289 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:53:44 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/2179191424' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-21T21:53:44.342 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411419 got 77309411425 for osd.2 2024-05-21T21:53:44.343 DEBUG:teuthology.parallel:result is None 2024-05-21T21:53:44.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:44.190+0000 7f1175c00700 1 -- 172.21.15.151:0/1391902090 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1170105780 msgr2=0x7f1170105c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:44.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:44.190+0000 7f1175c00700 1 --2- 172.21.15.151:0/1391902090 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1170105780 0x7f1170105c00 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1158009990 tx=0x7f1158009ca0 comp rx=0 tx=0).stop 2024-05-21T21:53:44.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:44.190+0000 7f1175c00700 1 -- 172.21.15.151:0/1391902090 shutdown_connections 2024-05-21T21:53:44.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:44.190+0000 7f1175c00700 1 --2- 172.21.15.151:0/1391902090 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1170106140 0x7f1170112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:44.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:44.190+0000 7f1175c00700 1 --2- 172.21.15.151:0/1391902090 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f117010b780 0x7f117010bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:44.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:44.190+0000 7f1175c00700 1 --2- 172.21.15.151:0/1391902090 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1170105780 0x7f1170105c00 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:44.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:44.190+0000 7f1175c00700 1 -- 172.21.15.151:0/1391902090 >> 172.21.15.151:0/1391902090 conn(0x7f1170101270 msgr2=0x7f1170103690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:53:44.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:44.191+0000 7f1175c00700 1 -- 172.21.15.151:0/1391902090 shutdown_connections 2024-05-21T21:53:44.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:44.191+0000 7f1175c00700 1 -- 172.21.15.151:0/1391902090 wait complete. 2024-05-21T21:53:44.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:44.192+0000 7f1175c00700 1 Processor -- start 2024-05-21T21:53:44.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:44.192+0000 7f1175c00700 1 -- start start 2024-05-21T21:53:44.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:44.192+0000 7f1175c00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1170105780 0x7f117019bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:44.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:44.192+0000 7f1175c00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1170106140 0x7f117019c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:44.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:44.192+0000 7f1175c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f117010b780 0x7f11701a0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:44.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:44.192+0000 7f1175c00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f11701159d0 con 0x7f117010b780 2024-05-21T21:53:44.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:44.192+0000 7f1175c00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1170115850 con 0x7f1170105780 2024-05-21T21:53:44.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:44.192+0000 7f1175c00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1170115b50 con 0x7f1170106140 2024-05-21T21:53:44.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:44.193+0000 7f116f400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f117010b780 0x7f11701a0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:44.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:44.193+0000 7f116f400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f117010b780 0x7f11701a0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:58210/0 (socket says 172.21.15.151:58210) 2024-05-21T21:53:44.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:44.193+0000 7f116ea00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1170105780 0x7f117019bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:44.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:44.193+0000 7f116f400700 1 -- 172.21.15.151:0/1672424213 learned_addr learned my addr 172.21.15.151:0/1672424213 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:53:44.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:44.193+0000 7f1167e00700 1 --2- 172.21.15.151:0/1672424213 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1170106140 0x7f117019c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:44.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:44.193+0000 7f116f400700 1 -- 172.21.15.151:0/1672424213 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1170106140 msgr2=0x7f117019c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:44.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:44.193+0000 7f116f400700 1 --2- 172.21.15.151:0/1672424213 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1170106140 0x7f117019c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:44.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:44.193+0000 7f116f400700 1 -- 172.21.15.151:0/1672424213 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1170105780 msgr2=0x7f117019bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:44.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:44.193+0000 7f116f400700 1 --2- 172.21.15.151:0/1672424213 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1170105780 0x7f117019bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:44.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:44.193+0000 7f116f400700 1 -- 172.21.15.151:0/1672424213 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11701a0e60 con 0x7f117010b780 2024-05-21T21:53:44.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:44.193+0000 7f116ea00700 1 --2- 172.21.15.151:0/1672424213 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1170105780 0x7f117019bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:53:44.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:44.193+0000 7f116f400700 1 --2- 172.21.15.151:0/1672424213 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f117010b780 0x7f11701a0720 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f116800ef30 tx=0x7f116800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:53:44.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:44.194+0000 7f1167400700 1 -- 172.21.15.151:0/1672424213 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11680092e0 con 0x7f117010b780 2024-05-21T21:53:44.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:44.194+0000 7f1167400700 1 -- 172.21.15.151:0/1672424213 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1168004bb0 con 0x7f117010b780 2024-05-21T21:53:44.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:44.194+0000 7f1175c00700 1 -- 172.21.15.151:0/1672424213 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11701a10f0 con 0x7f117010b780 2024-05-21T21:53:44.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:44.194+0000 7f1167400700 1 -- 172.21.15.151:0/1672424213 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1168004020 con 0x7f117010b780 2024-05-21T21:53:44.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:44.194+0000 7f1175c00700 1 -- 172.21.15.151:0/1672424213 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11701a89d0 con 0x7f117010b780 2024-05-21T21:53:44.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:44.195+0000 7f1175c00700 1 -- 172.21.15.151:0/1672424213 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f117006a230 con 0x7f117010b780 2024-05-21T21:53:44.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:44.197+0000 7f1167400700 1 -- 172.21.15.151:0/1672424213 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f116801b030 con 0x7f117010b780 2024-05-21T21:53:44.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:44.199+0000 7f1167400700 1 --2- 172.21.15.151:0/1672424213 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f11540b2900 0x7f11540b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:44.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:44.199+0000 7f1167400700 1 -- 172.21.15.151:0/1672424213 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f11680e26b0 con 0x7f117010b780 2024-05-21T21:53:44.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:44.199+0000 7f116ea00700 1 --2- 172.21.15.151:0/1672424213 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f11540b2900 0x7f11540b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:44.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:44.200+0000 7f116ea00700 1 --2- 172.21.15.151:0/1672424213 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f11540b2900 0x7f11540b4dc0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1160009fd0 tx=0x7f1160009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:53:44.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:44.201+0000 7f1167400700 1 -- 172.21.15.151:0/1672424213 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1168022020 con 0x7f117010b780 2024-05-21T21:53:44.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:44.437+0000 7f1175c00700 1 -- 172.21.15.151:0/1672424213 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f117002e560 con 0x7f117010b780 2024-05-21T21:53:44.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:44.437+0000 7f1167400700 1 -- 172.21.15.151:0/1672424213 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f1168010040 con 0x7f117010b780 2024-05-21T21:53:44.460 INFO:teuthology.orchestra.run.smithi151.stdout:188978561097 2024-05-21T21:53:44.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:44.440+0000 7f1175c00700 1 -- 172.21.15.151:0/1672424213 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f11540b2900 msgr2=0x7f11540b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:44.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:44.440+0000 7f1175c00700 1 --2- 172.21.15.151:0/1672424213 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f11540b2900 0x7f11540b4dc0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1160009fd0 tx=0x7f1160009260 comp rx=0 tx=0).stop 2024-05-21T21:53:44.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:44.440+0000 7f1175c00700 1 -- 172.21.15.151:0/1672424213 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f117010b780 msgr2=0x7f11701a0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:44.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:44.440+0000 7f1175c00700 1 --2- 172.21.15.151:0/1672424213 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f117010b780 0x7f11701a0720 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f116800ef30 tx=0x7f116800c3a0 comp rx=0 tx=0).stop 2024-05-21T21:53:44.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:44.440+0000 7f1175c00700 1 -- 172.21.15.151:0/1672424213 shutdown_connections 2024-05-21T21:53:44.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:44.440+0000 7f1175c00700 1 --2- 172.21.15.151:0/1672424213 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f11540b2900 0x7f11540b4dc0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:44.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:44.440+0000 7f1175c00700 1 --2- 172.21.15.151:0/1672424213 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1170105780 0x7f117019bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:44.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:44.440+0000 7f1175c00700 1 --2- 172.21.15.151:0/1672424213 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f117010b780 0x7f11701a0720 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:44.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:44.440+0000 7f1175c00700 1 --2- 172.21.15.151:0/1672424213 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1170106140 0x7f117019c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:44.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:44.440+0000 7f1175c00700 1 -- 172.21.15.151:0/1672424213 >> 172.21.15.151:0/1672424213 conn(0x7f1170101270 msgr2=0x7f1170101870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:53:44.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:44.441+0000 7f1175c00700 1 -- 172.21.15.151:0/1672424213 shutdown_connections 2024-05-21T21:53:44.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:44.441+0000 7f1175c00700 1 -- 172.21.15.151:0/1672424213 wait complete. 2024-05-21T21:53:44.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:53:44 smithi184 ceph-mon[36942]: pgmap v389: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-21T21:53:44.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:53:44 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/2179191424' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-21T21:53:45.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:53:45 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/1672424213' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-21T21:53:45.485 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:53:45 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/1672424213' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-21T21:53:45.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:53:45 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/1672424213' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-21T21:53:46.184 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:53:46 smithi151 ceph-mon[31623]: pgmap v390: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-21T21:53:46.250 INFO:tasks.cephadm.ceph_manager.ceph:need seq 188978561092 got 188978561097 for osd.6 2024-05-21T21:53:46.250 DEBUG:teuthology.parallel:result is None 2024-05-21T21:53:46.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:53:46 smithi184 ceph-mon[36942]: pgmap v390: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-21T21:53:46.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:53:46 smithi164 ceph-mon[36269]: pgmap v390: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-21T21:53:46.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:46.795+0000 7fa533600700 1 -- 172.21.15.151:0/3306809592 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa52c10a780 msgr2=0x7fa52c10e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:46.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:46.795+0000 7fa533600700 1 --2- 172.21.15.151:0/3306809592 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa52c10a780 0x7fa52c10e7d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa518009990 tx=0x7fa518009ca0 comp rx=0 tx=0).stop 2024-05-21T21:53:46.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:46.795+0000 7fa533600700 1 -- 172.21.15.151:0/3306809592 shutdown_connections 2024-05-21T21:53:46.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:46.795+0000 7fa533600700 1 --2- 172.21.15.151:0/3306809592 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa52c10eeb0 0x7fa52c116a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:46.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:46.795+0000 7fa533600700 1 --2- 172.21.15.151:0/3306809592 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa52c109dd0 0x7fa52c10a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:46.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:46.795+0000 7fa533600700 1 --2- 172.21.15.151:0/3306809592 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa52c10a780 0x7fa52c10e7d0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:46.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:46.795+0000 7fa533600700 1 -- 172.21.15.151:0/3306809592 >> 172.21.15.151:0/3306809592 conn(0x7fa52c076d10 msgr2=0x7fa52c108360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:53:46.797 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:46.796+0000 7fa533600700 1 -- 172.21.15.151:0/3306809592 shutdown_connections 2024-05-21T21:53:46.797 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:46.796+0000 7fa533600700 1 -- 172.21.15.151:0/3306809592 wait complete. 2024-05-21T21:53:46.797 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:46.797+0000 7fa533600700 1 Processor -- start 2024-05-21T21:53:46.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:46.797+0000 7fa533600700 1 -- start start 2024-05-21T21:53:46.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:46.797+0000 7fa533600700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa52c109dd0 0x7fa52c1a02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:46.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:46.797+0000 7fa533600700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa52c10a780 0x7fa52c1a0830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:46.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:46.797+0000 7fa533600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa52c10eeb0 0x7fa52c1a4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:46.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:46.797+0000 7fa533600700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa52c119da0 con 0x7fa52c10eeb0 2024-05-21T21:53:46.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:46.797+0000 7fa533600700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa52c119c20 con 0x7fa52c109dd0 2024-05-21T21:53:46.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:46.797+0000 7fa533600700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa52c119f20 con 0x7fa52c10a780 2024-05-21T21:53:46.799 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:46.798+0000 7fa531600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa52c10eeb0 0x7fa52c1a4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:46.799 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:46.798+0000 7fa531600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa52c10eeb0 0x7fa52c1a4bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:40506/0 (socket says 172.21.15.151:40506) 2024-05-21T21:53:46.799 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:46.798+0000 7fa531600700 1 -- 172.21.15.151:0/2118264727 learned_addr learned my addr 172.21.15.151:0/2118264727 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:53:46.799 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:46.798+0000 7fa52be00700 1 --2- 172.21.15.151:0/2118264727 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa52c10a780 0x7fa52c1a0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:46.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:46.798+0000 7fa530c00700 1 --2- 172.21.15.151:0/2118264727 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa52c109dd0 0x7fa52c1a02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:46.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:46.798+0000 7fa531600700 1 -- 172.21.15.151:0/2118264727 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa52c10a780 msgr2=0x7fa52c1a0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:46.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:46.798+0000 7fa531600700 1 --2- 172.21.15.151:0/2118264727 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa52c10a780 0x7fa52c1a0830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:46.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:46.798+0000 7fa531600700 1 -- 172.21.15.151:0/2118264727 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa52c109dd0 msgr2=0x7fa52c1a02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:46.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:46.798+0000 7fa531600700 1 --2- 172.21.15.151:0/2118264727 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa52c109dd0 0x7fa52c1a02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:46.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:46.798+0000 7fa531600700 1 -- 172.21.15.151:0/2118264727 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa52c1a5300 con 0x7fa52c10eeb0 2024-05-21T21:53:46.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:46.798+0000 7fa530c00700 1 --2- 172.21.15.151:0/2118264727 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa52c109dd0 0x7fa52c1a02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:53:46.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:46.799+0000 7fa531600700 1 --2- 172.21.15.151:0/2118264727 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa52c10eeb0 0x7fa52c1a4bc0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa52400ef30 tx=0x7fa52400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:53:46.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:46.799+0000 7fa529600700 1 -- 172.21.15.151:0/2118264727 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5240092e0 con 0x7fa52c10eeb0 2024-05-21T21:53:46.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:46.799+0000 7fa533600700 1 -- 172.21.15.151:0/2118264727 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa52c1a5590 con 0x7fa52c10eeb0 2024-05-21T21:53:46.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:46.799+0000 7fa529600700 1 -- 172.21.15.151:0/2118264727 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa524004bb0 con 0x7fa52c10eeb0 2024-05-21T21:53:46.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:46.799+0000 7fa529600700 1 -- 172.21.15.151:0/2118264727 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa524004020 con 0x7fa52c10eeb0 2024-05-21T21:53:46.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:46.799+0000 7fa533600700 1 -- 172.21.15.151:0/2118264727 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa52c1ace70 con 0x7fa52c10eeb0 2024-05-21T21:53:46.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:46.800+0000 7fa533600700 1 -- 172.21.15.151:0/2118264727 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa52c06a230 con 0x7fa52c10eeb0 2024-05-21T21:53:46.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:46.802+0000 7fa529600700 1 -- 172.21.15.151:0/2118264727 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fa52401b030 con 0x7fa52c10eeb0 2024-05-21T21:53:46.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:46.804+0000 7fa529600700 1 --2- 172.21.15.151:0/2118264727 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fa5140b2900 0x7fa5140b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:46.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:46.804+0000 7fa529600700 1 -- 172.21.15.151:0/2118264727 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fa5240e1be0 con 0x7fa52c10eeb0 2024-05-21T21:53:46.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:46.804+0000 7fa530c00700 1 --2- 172.21.15.151:0/2118264727 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fa5140b2900 0x7fa5140b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:46.805 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:46.805+0000 7fa530c00700 1 --2- 172.21.15.151:0/2118264727 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fa5140b2900 0x7fa5140b4dc0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa520011490 tx=0x7fa520009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:53:46.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:46.808+0000 7fa529600700 1 -- 172.21.15.151:0/2118264727 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa524022020 con 0x7fa52c10eeb0 2024-05-21T21:53:47.044 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:47.044+0000 7fa533600700 1 -- 172.21.15.151:0/2118264727 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7fa52c02e560 con 0x7fa52c10eeb0 2024-05-21T21:53:47.045 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:47.045+0000 7fa529600700 1 -- 172.21.15.151:0/2118264727 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fa524010040 con 0x7fa52c10eeb0 2024-05-21T21:53:47.045 INFO:teuthology.orchestra.run.smithi151.stdout:283467841587 2024-05-21T21:53:47.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:47.048+0000 7fa533600700 1 -- 172.21.15.151:0/2118264727 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fa5140b2900 msgr2=0x7fa5140b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:47.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:47.048+0000 7fa533600700 1 --2- 172.21.15.151:0/2118264727 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fa5140b2900 0x7fa5140b4dc0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa520011490 tx=0x7fa520009260 comp rx=0 tx=0).stop 2024-05-21T21:53:47.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:47.048+0000 7fa533600700 1 -- 172.21.15.151:0/2118264727 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa52c10eeb0 msgr2=0x7fa52c1a4bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:47.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:47.048+0000 7fa533600700 1 --2- 172.21.15.151:0/2118264727 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa52c10eeb0 0x7fa52c1a4bc0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa52400ef30 tx=0x7fa52400c3a0 comp rx=0 tx=0).stop 2024-05-21T21:53:47.049 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:47.048+0000 7fa533600700 1 -- 172.21.15.151:0/2118264727 shutdown_connections 2024-05-21T21:53:47.049 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:47.048+0000 7fa533600700 1 --2- 172.21.15.151:0/2118264727 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fa5140b2900 0x7fa5140b4dc0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:47.049 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:47.048+0000 7fa533600700 1 --2- 172.21.15.151:0/2118264727 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa52c109dd0 0x7fa52c1a02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:47.049 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:47.048+0000 7fa533600700 1 --2- 172.21.15.151:0/2118264727 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa52c10eeb0 0x7fa52c1a4bc0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:47.049 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:47.048+0000 7fa533600700 1 --2- 172.21.15.151:0/2118264727 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa52c10a780 0x7fa52c1a0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:47.049 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:47.048+0000 7fa533600700 1 -- 172.21.15.151:0/2118264727 >> 172.21.15.151:0/2118264727 conn(0x7fa52c076d10 msgr2=0x7fa52c0788d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:53:47.049 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:47.049+0000 7fa533600700 1 -- 172.21.15.151:0/2118264727 shutdown_connections 2024-05-21T21:53:47.049 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:47.049+0000 7fa533600700 1 -- 172.21.15.151:0/2118264727 wait complete. 2024-05-21T21:53:47.421 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:53:47 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/2118264727' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-21T21:53:47.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:53:47 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/2118264727' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-21T21:53:47.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:53:47 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/2118264727' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-21T21:53:47.970 INFO:tasks.cephadm.ceph_manager.ceph:need seq 283467841583 got 283467841587 for osd.10 2024-05-21T21:53:47.970 DEBUG:teuthology.parallel:result is None 2024-05-21T21:53:47.970 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-21T21:53:47.971 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph pg dump --format=json 2024-05-21T21:53:48.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:53:48 smithi151 ceph-mon[31623]: pgmap v391: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-21T21:53:48.415 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:53:48.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:53:48 smithi184 ceph-mon[36942]: pgmap v391: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-21T21:53:48.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:53:48 smithi164 ceph-mon[36269]: pgmap v391: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-21T21:53:50.410 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:53:50 smithi151 ceph-mon[31623]: pgmap v392: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-21T21:53:50.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:50.581+0000 7f32b8200700 1 -- 172.21.15.151:0/1256547988 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f32b006c470 msgr2=0x7f32b01103c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:50.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:50.581+0000 7f32b8200700 1 --2- 172.21.15.151:0/1256547988 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f32b006c470 0x7f32b01103c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f32a800b0a0 tx=0x7f32a800b3b0 comp rx=0 tx=0).stop 2024-05-21T21:53:50.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:50.581+0000 7f32b8200700 1 -- 172.21.15.151:0/1256547988 shutdown_connections 2024-05-21T21:53:50.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:50.581+0000 7f32b8200700 1 --2- 172.21.15.151:0/1256547988 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f32b006c470 0x7f32b01103c0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:50.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:50.581+0000 7f32b8200700 1 --2- 172.21.15.151:0/1256547988 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f32b01095a0 0x7f32b0109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:50.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:50.581+0000 7f32b8200700 1 --2- 172.21.15.151:0/1256547988 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f32b006bab0 0x7f32b006bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:50.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:50.581+0000 7f32b8200700 1 -- 172.21.15.151:0/1256547988 >> 172.21.15.151:0/1256547988 conn(0x7f32b00793b0 msgr2=0x7f32b00797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:53:50.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:50.581+0000 7f32b8200700 1 -- 172.21.15.151:0/1256547988 shutdown_connections 2024-05-21T21:53:50.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:50.581+0000 7f32b8200700 1 -- 172.21.15.151:0/1256547988 wait complete. 2024-05-21T21:53:50.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:50.582+0000 7f32b8200700 1 Processor -- start 2024-05-21T21:53:50.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:50.583+0000 7f32b8200700 1 -- start start 2024-05-21T21:53:50.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:50.583+0000 7f32b8200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f32b006bab0 0x7f32b019bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:50.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:50.583+0000 7f32b8200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f32b006c470 0x7f32b019c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:50.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:50.583+0000 7f32b8200700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f32b01095a0 0x7f32b01a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:50.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:50.583+0000 7f32b8200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f32b0113730 con 0x7f32b006bab0 2024-05-21T21:53:50.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:50.583+0000 7f32b8200700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f32b01135b0 con 0x7f32b006c470 2024-05-21T21:53:50.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:50.583+0000 7f32b8200700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f32b01138b0 con 0x7f32b01095a0 2024-05-21T21:53:50.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:50.583+0000 7f32b5800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f32b006bab0 0x7f32b019bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:50.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:50.583+0000 7f32b5800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f32b006bab0 0x7f32b019bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:40522/0 (socket says 172.21.15.151:40522) 2024-05-21T21:53:50.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:50.584+0000 7f32b5800700 1 -- 172.21.15.151:0/1702157414 learned_addr learned my addr 172.21.15.151:0/1702157414 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:53:50.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:50.584+0000 7f32b4e00700 1 --2- 172.21.15.151:0/1702157414 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f32b006c470 0x7f32b019c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:50.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:50.584+0000 7f32b6200700 1 --2- 172.21.15.151:0/1702157414 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f32b01095a0 0x7f32b01a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:50.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:50.584+0000 7f32b5800700 1 -- 172.21.15.151:0/1702157414 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f32b01095a0 msgr2=0x7f32b01a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:50.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:50.584+0000 7f32b5800700 1 --2- 172.21.15.151:0/1702157414 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f32b01095a0 0x7f32b01a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:50.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:50.584+0000 7f32b5800700 1 -- 172.21.15.151:0/1702157414 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f32b006c470 msgr2=0x7f32b019c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:50.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:50.584+0000 7f32b5800700 1 --2- 172.21.15.151:0/1702157414 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f32b006c470 0x7f32b019c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:50.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:50.584+0000 7f32b5800700 1 -- 172.21.15.151:0/1702157414 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32b01a0df0 con 0x7f32b006bab0 2024-05-21T21:53:50.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:50.584+0000 7f32b4e00700 1 --2- 172.21.15.151:0/1702157414 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f32b006c470 0x7f32b019c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:53:50.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:50.584+0000 7f32b5800700 1 --2- 172.21.15.151:0/1702157414 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f32b006bab0 0x7f32b019bde0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f32a400e9a0 tx=0x7f32a400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:53:50.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:50.585+0000 7f32ae000700 1 -- 172.21.15.151:0/1702157414 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a400ca10 con 0x7f32b006bab0 2024-05-21T21:53:50.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:50.585+0000 7f32ae000700 1 -- 172.21.15.151:0/1702157414 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32a400cbb0 con 0x7f32b006bab0 2024-05-21T21:53:50.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:50.585+0000 7f32b8200700 1 -- 172.21.15.151:0/1702157414 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32b01a10e0 con 0x7f32b006bab0 2024-05-21T21:53:50.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:50.585+0000 7f32ae000700 1 -- 172.21.15.151:0/1702157414 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a4018680 con 0x7f32b006bab0 2024-05-21T21:53:50.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:50.585+0000 7f32b8200700 1 -- 172.21.15.151:0/1702157414 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32b01a8960 con 0x7f32b006bab0 2024-05-21T21:53:50.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:50.586+0000 7f32b8200700 1 -- 172.21.15.151:0/1702157414 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3290005330 con 0x7f32b006bab0 2024-05-21T21:53:50.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:50.588+0000 7f32ae000700 1 -- 172.21.15.151:0/1702157414 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f32a4018820 con 0x7f32b006bab0 2024-05-21T21:53:50.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:50.591+0000 7f32ae000700 1 --2- 172.21.15.151:0/1702157414 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f32980b2900 0x7f32980b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:50.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:50.591+0000 7f32ae000700 1 -- 172.21.15.151:0/1702157414 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f32a40e21b0 con 0x7f32b006bab0 2024-05-21T21:53:50.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:50.591+0000 7f32b4e00700 1 --2- 172.21.15.151:0/1702157414 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f32980b2900 0x7f32980b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:50.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:50.592+0000 7f32b4e00700 1 --2- 172.21.15.151:0/1702157414 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f32980b2900 0x7f32980b4dc0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f32b0104530 tx=0x7f329c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:53:50.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:50.595+0000 7f32ae000700 1 -- 172.21.15.151:0/1702157414 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f32a40a9e90 con 0x7f32b006bab0 2024-05-21T21:53:50.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:53:50 smithi184 ceph-mon[36942]: pgmap v392: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-21T21:53:50.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:53:50 smithi164 ceph-mon[36269]: pgmap v392: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-21T21:53:50.826 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:50.826+0000 7f32b8200700 1 -- 172.21.15.151:0/1702157414 --> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3290000bc0 con 0x7f32980b2900 2024-05-21T21:53:50.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:50.834+0000 7f32ae000700 1 -- 172.21.15.151:0/1702157414 <== mgr.14152 v2:172.21.15.151:6800/2796622483 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258888 (secure 0 0 0) 0x7f3290000bc0 con 0x7f32980b2900 2024-05-21T21:53:50.836 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T21:53:50.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:50.841+0000 7f32b8200700 1 -- 172.21.15.151:0/1702157414 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f32980b2900 msgr2=0x7f32980b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:50.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:50.841+0000 7f32b8200700 1 --2- 172.21.15.151:0/1702157414 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f32980b2900 0x7f32980b4dc0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f32b0104530 tx=0x7f329c009260 comp rx=0 tx=0).stop 2024-05-21T21:53:50.843 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:50.841+0000 7f32b8200700 1 -- 172.21.15.151:0/1702157414 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f32b006bab0 msgr2=0x7f32b019bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:50.843 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:50.841+0000 7f32b8200700 1 --2- 172.21.15.151:0/1702157414 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f32b006bab0 0x7f32b019bde0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f32a400e9a0 tx=0x7f32a400ecb0 comp rx=0 tx=0).stop 2024-05-21T21:53:50.843 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:50.841+0000 7f32b8200700 1 -- 172.21.15.151:0/1702157414 shutdown_connections 2024-05-21T21:53:50.843 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:50.841+0000 7f32b8200700 1 --2- 172.21.15.151:0/1702157414 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f32980b2900 0x7f32980b4dc0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:50.843 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:50.841+0000 7f32b8200700 1 --2- 172.21.15.151:0/1702157414 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f32b01095a0 0x7f32b01a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:50.843 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:50.841+0000 7f32b8200700 1 --2- 172.21.15.151:0/1702157414 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f32b006bab0 0x7f32b019bde0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:50.843 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:50.841+0000 7f32b8200700 1 --2- 172.21.15.151:0/1702157414 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f32b006c470 0x7f32b019c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:50.843 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:50.841+0000 7f32b8200700 1 -- 172.21.15.151:0/1702157414 >> 172.21.15.151:0/1702157414 conn(0x7f32b00793b0 msgr2=0x7f32b010e6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:53:50.843 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:50.842+0000 7f32b8200700 1 -- 172.21.15.151:0/1702157414 shutdown_connections 2024-05-21T21:53:50.843 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:50.842+0000 7f32b8200700 1 -- 172.21.15.151:0/1702157414 wait complete. 2024-05-21T21:53:50.844 INFO:teuthology.orchestra.run.smithi151.stderr:dumped all 2024-05-21T21:53:51.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:53:51 smithi151 ceph-mon[31623]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T21:53:51.559 INFO:teuthology.orchestra.run.smithi151.stdout:{"pg_ready":true,"pg_map":{"version":392,"stamp":"2024-05-21T21:53:49.252562+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658099,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1393,"num_read_kb":797,"num_write":379,"num_write_kb":2294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":2952069,"num_keys_recovered":17,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":812,"ondisk_log_size":812,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":366044,"kb_used_data":12732,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124575780,"statfs":{"total":1151940427776,"available":1151565598720,"internally_reserved":0,"allocated":13037568,"data_stored":7167501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19140,"internal_metadata":361674044},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":109,"num_read_kb":50,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005281"},"pg_stats":[{"pgid":"3.3f","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.190291+0000","last_change":"2024-05-21T21:51:16.945132+0000","last_active":"2024-05-21T21:53:09.190291+0000","last_peered":"2024-05-21T21:53:09.190291+0000","last_clean":"2024-05-21T21:53:09.190291+0000","last_became_active":"2024-05-21T21:51:16.944766+0000","last_became_peered":"2024-05-21T21:51:16.944766+0000","last_unstale":"2024-05-21T21:53:09.190291+0000","last_undegraded":"2024-05-21T21:53:09.190291+0000","last_fullsized":"2024-05-21T21:53:09.190291+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:54:04.185095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"82'4","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.233872+0000","last_change":"2024-05-21T21:51:16.939291+0000","last_active":"2024-05-21T21:53:09.233872+0000","last_peered":"2024-05-21T21:53:09.233872+0000","last_clean":"2024-05-21T21:53:09.233872+0000","last_became_active":"2024-05-21T21:51:16.938730+0000","last_became_peered":"2024-05-21T21:51:16.938730+0000","last_unstale":"2024-05-21T21:53:09.233872+0000","last_undegraded":"2024-05-21T21:53:09.233872+0000","last_fullsized":"2024-05-21T21:53:09.233872+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:05:30.623744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.232965+0000","last_change":"2024-05-21T21:51:16.941565+0000","last_active":"2024-05-21T21:53:09.232965+0000","last_peered":"2024-05-21T21:53:09.232965+0000","last_clean":"2024-05-21T21:53:09.232965+0000","last_became_active":"2024-05-21T21:51:16.941033+0000","last_became_peered":"2024-05-21T21:51:16.941033+0000","last_unstale":"2024-05-21T21:53:09.232965+0000","last_undegraded":"2024-05-21T21:53:09.232965+0000","last_fullsized":"2024-05-21T21:53:09.232965+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:04:55.789479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.183697+0000","last_change":"2024-05-21T21:51:16.928455+0000","last_active":"2024-05-21T21:53:09.183697+0000","last_peered":"2024-05-21T21:53:09.183697+0000","last_clean":"2024-05-21T21:53:09.183697+0000","last_became_active":"2024-05-21T21:51:16.927874+0000","last_became_peered":"2024-05-21T21:51:16.927874+0000","last_unstale":"2024-05-21T21:53:09.183697+0000","last_undegraded":"2024-05-21T21:53:09.183697+0000","last_fullsized":"2024-05-21T21:53:09.183697+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:25:16.483175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"82'4","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.215489+0000","last_change":"2024-05-21T21:51:16.937773+0000","last_active":"2024-05-21T21:53:09.215489+0000","last_peered":"2024-05-21T21:53:09.215489+0000","last_clean":"2024-05-21T21:53:09.215489+0000","last_became_active":"2024-05-21T21:51:16.937306+0000","last_became_peered":"2024-05-21T21:51:16.937306+0000","last_unstale":"2024-05-21T21:53:09.215489+0000","last_undegraded":"2024-05-21T21:53:09.215489+0000","last_fullsized":"2024-05-21T21:53:09.215489+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:05:03.762390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.197708+0000","last_change":"2024-05-21T21:51:16.948275+0000","last_active":"2024-05-21T21:53:09.197708+0000","last_peered":"2024-05-21T21:53:09.197708+0000","last_clean":"2024-05-21T21:53:09.197708+0000","last_became_active":"2024-05-21T21:51:16.947890+0000","last_became_peered":"2024-05-21T21:51:16.947890+0000","last_unstale":"2024-05-21T21:53:09.197708+0000","last_undegraded":"2024-05-21T21:53:09.197708+0000","last_fullsized":"2024-05-21T21:53:09.197708+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:08:38.572177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"82'5","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.197835+0000","last_change":"2024-05-21T21:51:16.946821+0000","last_active":"2024-05-21T21:53:09.197835+0000","last_peered":"2024-05-21T21:53:09.197835+0000","last_clean":"2024-05-21T21:53:09.197835+0000","last_became_active":"2024-05-21T21:51:16.946515+0000","last_became_peered":"2024-05-21T21:51:16.946515+0000","last_unstale":"2024-05-21T21:53:09.197835+0000","last_undegraded":"2024-05-21T21:53:09.197835+0000","last_fullsized":"2024-05-21T21:53:09.197835+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T09:26:25.867895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.194071+0000","last_change":"2024-05-21T21:51:16.929228+0000","last_active":"2024-05-21T21:53:09.194071+0000","last_peered":"2024-05-21T21:53:09.194071+0000","last_clean":"2024-05-21T21:53:09.194071+0000","last_became_active":"2024-05-21T21:51:16.928875+0000","last_became_peered":"2024-05-21T21:51:16.928875+0000","last_unstale":"2024-05-21T21:53:09.194071+0000","last_undegraded":"2024-05-21T21:53:09.194071+0000","last_fullsized":"2024-05-21T21:53:09.194071+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:50:56.431310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"82'11","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.193550+0000","last_change":"2024-05-21T21:51:16.932360+0000","last_active":"2024-05-21T21:53:09.193550+0000","last_peered":"2024-05-21T21:53:09.193550+0000","last_clean":"2024-05-21T21:53:09.193550+0000","last_became_active":"2024-05-21T21:51:16.932036+0000","last_became_peered":"2024-05-21T21:51:16.932036+0000","last_unstale":"2024-05-21T21:53:09.193550+0000","last_undegraded":"2024-05-21T21:53:09.193550+0000","last_fullsized":"2024-05-21T21:53:09.193550+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T09:06:37.257323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"82'16","reported_seq":43,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.217531+0000","last_change":"2024-05-21T21:52:10.876556+0000","last_active":"2024-05-21T21:53:09.217531+0000","last_peered":"2024-05-21T21:53:09.217531+0000","last_clean":"2024-05-21T21:53:09.217531+0000","last_became_active":"2024-05-21T21:52:10.876273+0000","last_became_peered":"2024-05-21T21:52:10.876273+0000","last_unstale":"2024-05-21T21:53:09.217531+0000","last_undegraded":"2024-05-21T21:53:09.217531+0000","last_fullsized":"2024-05-21T21:53:09.217531+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T05:09:40.037731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.195209+0000","last_change":"2024-05-21T21:51:16.933425+0000","last_active":"2024-05-21T21:53:09.195209+0000","last_peered":"2024-05-21T21:53:09.195209+0000","last_clean":"2024-05-21T21:53:09.195209+0000","last_became_active":"2024-05-21T21:51:16.933057+0000","last_became_peered":"2024-05-21T21:51:16.933057+0000","last_unstale":"2024-05-21T21:53:09.195209+0000","last_undegraded":"2024-05-21T21:53:09.195209+0000","last_fullsized":"2024-05-21T21:53:09.195209+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:04:57.237841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.184643+0000","last_change":"2024-05-21T21:51:16.927014+0000","last_active":"2024-05-21T21:53:09.184643+0000","last_peered":"2024-05-21T21:53:09.184643+0000","last_clean":"2024-05-21T21:53:09.184643+0000","last_became_active":"2024-05-21T21:51:16.925954+0000","last_became_peered":"2024-05-21T21:51:16.925954+0000","last_unstale":"2024-05-21T21:53:09.184643+0000","last_undegraded":"2024-05-21T21:53:09.184643+0000","last_fullsized":"2024-05-21T21:53:09.184643+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:19:20.530727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"82'10","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.228367+0000","last_change":"2024-05-21T21:51:16.930909+0000","last_active":"2024-05-21T21:53:09.228367+0000","last_peered":"2024-05-21T21:53:09.228367+0000","last_clean":"2024-05-21T21:53:09.228367+0000","last_became_active":"2024-05-21T21:51:16.927110+0000","last_became_peered":"2024-05-21T21:51:16.927110+0000","last_unstale":"2024-05-21T21:53:09.228367+0000","last_undegraded":"2024-05-21T21:53:09.228367+0000","last_fullsized":"2024-05-21T21:53:09.228367+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T06:23:55.386047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.232153+0000","last_change":"2024-05-21T21:51:16.938797+0000","last_active":"2024-05-21T21:53:09.232153+0000","last_peered":"2024-05-21T21:53:09.232153+0000","last_clean":"2024-05-21T21:53:09.232153+0000","last_became_active":"2024-05-21T21:51:16.938263+0000","last_became_peered":"2024-05-21T21:51:16.938263+0000","last_unstale":"2024-05-21T21:53:09.232153+0000","last_undegraded":"2024-05-21T21:53:09.232153+0000","last_fullsized":"2024-05-21T21:53:09.232153+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:37:40.521118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.233950+0000","last_change":"2024-05-21T21:51:16.944160+0000","last_active":"2024-05-21T21:53:09.233950+0000","last_peered":"2024-05-21T21:53:09.233950+0000","last_clean":"2024-05-21T21:53:09.233950+0000","last_became_active":"2024-05-21T21:51:16.940046+0000","last_became_peered":"2024-05-21T21:51:16.940046+0000","last_unstale":"2024-05-21T21:53:09.233950+0000","last_undegraded":"2024-05-21T21:53:09.233950+0000","last_fullsized":"2024-05-21T21:53:09.233950+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:14:13.278799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.199500+0000","last_change":"2024-05-21T21:51:16.945599+0000","last_active":"2024-05-21T21:53:09.199500+0000","last_peered":"2024-05-21T21:53:09.199500+0000","last_clean":"2024-05-21T21:53:09.199500+0000","last_became_active":"2024-05-21T21:51:16.945196+0000","last_became_peered":"2024-05-21T21:51:16.945196+0000","last_unstale":"2024-05-21T21:53:09.199500+0000","last_undegraded":"2024-05-21T21:53:09.199500+0000","last_fullsized":"2024-05-21T21:53:09.199500+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:59:24.453615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.199215+0000","last_change":"2024-05-21T21:51:16.944895+0000","last_active":"2024-05-21T21:53:09.199215+0000","last_peered":"2024-05-21T21:53:09.199215+0000","last_clean":"2024-05-21T21:53:09.199215+0000","last_became_active":"2024-05-21T21:51:16.944560+0000","last_became_peered":"2024-05-21T21:51:16.944560+0000","last_unstale":"2024-05-21T21:53:09.199215+0000","last_undegraded":"2024-05-21T21:53:09.199215+0000","last_fullsized":"2024-05-21T21:53:09.199215+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:53:13.264701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"82'10","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.221764+0000","last_change":"2024-05-21T21:51:16.943861+0000","last_active":"2024-05-21T21:53:09.221764+0000","last_peered":"2024-05-21T21:53:09.221764+0000","last_clean":"2024-05-21T21:53:09.221764+0000","last_became_active":"2024-05-21T21:51:16.943123+0000","last_became_peered":"2024-05-21T21:51:16.943123+0000","last_unstale":"2024-05-21T21:53:09.221764+0000","last_undegraded":"2024-05-21T21:53:09.221764+0000","last_fullsized":"2024-05-21T21:53:09.221764+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:35:37.732855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.206209+0000","last_change":"2024-05-21T21:51:16.940580+0000","last_active":"2024-05-21T21:53:09.206209+0000","last_peered":"2024-05-21T21:53:09.206209+0000","last_clean":"2024-05-21T21:53:09.206209+0000","last_became_active":"2024-05-21T21:51:16.940264+0000","last_became_peered":"2024-05-21T21:51:16.940264+0000","last_unstale":"2024-05-21T21:53:09.206209+0000","last_undegraded":"2024-05-21T21:53:09.206209+0000","last_fullsized":"2024-05-21T21:53:09.206209+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:54:22.915541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"80'13","reported_seq":167,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:43.640889+0000","last_change":"2024-05-21T21:51:12.852935+0000","last_active":"2024-05-21T21:53:43.640889+0000","last_peered":"2024-05-21T21:53:43.640889+0000","last_clean":"2024-05-21T21:53:43.640889+0000","last_became_active":"2024-05-21T21:51:12.851802+0000","last_became_peered":"2024-05-21T21:51:12.851802+0000","last_unstale":"2024-05-21T21:53:43.640889+0000","last_undegraded":"2024-05-21T21:53:43.640889+0000","last_fullsized":"2024-05-21T21:53:43.640889+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:13:24.643408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":127,"num_read_kb":127,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"82'5","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.198288+0000","last_change":"2024-05-21T21:51:16.939461+0000","last_active":"2024-05-21T21:53:09.198288+0000","last_peered":"2024-05-21T21:53:09.198288+0000","last_clean":"2024-05-21T21:53:09.198288+0000","last_became_active":"2024-05-21T21:51:16.936322+0000","last_became_peered":"2024-05-21T21:51:16.936322+0000","last_unstale":"2024-05-21T21:53:09.198288+0000","last_undegraded":"2024-05-21T21:53:09.198288+0000","last_fullsized":"2024-05-21T21:53:09.198288+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:37:26.153963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.868750+0000","last_change":"2024-05-21T21:51:12.941838+0000","last_active":"2024-05-21T21:52:10.868750+0000","last_peered":"2024-05-21T21:52:10.868750+0000","last_clean":"2024-05-21T21:52:10.868750+0000","last_became_active":"2024-05-21T21:51:12.940818+0000","last_became_peered":"2024-05-21T21:51:12.940818+0000","last_unstale":"2024-05-21T21:52:10.868750+0000","last_undegraded":"2024-05-21T21:52:10.868750+0000","last_fullsized":"2024-05-21T21:52:10.868750+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:30:42.349241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.238219+0000","last_change":"2024-05-21T21:51:16.950252+0000","last_active":"2024-05-21T21:53:09.238219+0000","last_peered":"2024-05-21T21:53:09.238219+0000","last_clean":"2024-05-21T21:53:09.238219+0000","last_became_active":"2024-05-21T21:51:16.947375+0000","last_became_peered":"2024-05-21T21:51:16.947375+0000","last_unstale":"2024-05-21T21:53:09.238219+0000","last_undegraded":"2024-05-21T21:53:09.238219+0000","last_fullsized":"2024-05-21T21:53:09.238219+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:27:30.462799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"80'12","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.876122+0000","last_change":"2024-05-21T21:52:10.875739+0000","last_active":"2024-05-21T21:52:10.876122+0000","last_peered":"2024-05-21T21:52:10.876122+0000","last_clean":"2024-05-21T21:52:10.876122+0000","last_became_active":"2024-05-21T21:52:10.865846+0000","last_became_peered":"2024-05-21T21:52:10.865846+0000","last_unstale":"2024-05-21T21:52:10.876122+0000","last_undegraded":"2024-05-21T21:52:10.876122+0000","last_fullsized":"2024-05-21T21:52:10.876122+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T06:48:06.465757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"82'9","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.215982+0000","last_change":"2024-05-21T21:51:16.926380+0000","last_active":"2024-05-21T21:53:09.215982+0000","last_peered":"2024-05-21T21:53:09.215982+0000","last_clean":"2024-05-21T21:53:09.215982+0000","last_became_active":"2024-05-21T21:51:16.925821+0000","last_became_peered":"2024-05-21T21:51:16.925821+0000","last_unstale":"2024-05-21T21:53:09.215982+0000","last_undegraded":"2024-05-21T21:53:09.215982+0000","last_fullsized":"2024-05-21T21:53:09.215982+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:42:14.887155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'5","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.864315+0000","last_change":"2024-05-21T21:51:12.854342+0000","last_active":"2024-05-21T21:52:10.864315+0000","last_peered":"2024-05-21T21:52:10.864315+0000","last_clean":"2024-05-21T21:52:10.864315+0000","last_became_active":"2024-05-21T21:51:12.853908+0000","last_became_peered":"2024-05-21T21:51:12.853908+0000","last_unstale":"2024-05-21T21:52:10.864315+0000","last_undegraded":"2024-05-21T21:52:10.864315+0000","last_fullsized":"2024-05-21T21:52:10.864315+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:35:10.374749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"82'15","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.236662+0000","last_change":"2024-05-21T21:52:10.973816+0000","last_active":"2024-05-21T21:53:09.236662+0000","last_peered":"2024-05-21T21:53:09.236662+0000","last_clean":"2024-05-21T21:53:09.236662+0000","last_became_active":"2024-05-21T21:52:10.973457+0000","last_became_peered":"2024-05-21T21:52:10.973457+0000","last_unstale":"2024-05-21T21:53:09.236662+0000","last_undegraded":"2024-05-21T21:53:09.236662+0000","last_fullsized":"2024-05-21T21:53:09.236662+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T05:01:41.915158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"82'7","reported_seq":167,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:46.637753+0000","last_change":"2024-05-21T21:51:12.855658+0000","last_active":"2024-05-21T21:53:46.637753+0000","last_peered":"2024-05-21T21:53:46.637753+0000","last_clean":"2024-05-21T21:53:46.637753+0000","last_became_active":"2024-05-21T21:51:12.855281+0000","last_became_peered":"2024-05-21T21:51:12.855281+0000","last_unstale":"2024-05-21T21:53:46.637753+0000","last_undegraded":"2024-05-21T21:53:46.637753+0000","last_fullsized":"2024-05-21T21:53:46.637753+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:55:08.836045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":133,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.198092+0000","last_change":"2024-05-21T21:51:16.946977+0000","last_active":"2024-05-21T21:53:09.198092+0000","last_peered":"2024-05-21T21:53:09.198092+0000","last_clean":"2024-05-21T21:53:09.198092+0000","last_became_active":"2024-05-21T21:51:16.946562+0000","last_became_peered":"2024-05-21T21:51:16.946562+0000","last_unstale":"2024-05-21T21:53:09.198092+0000","last_undegraded":"2024-05-21T21:53:09.198092+0000","last_fullsized":"2024-05-21T21:53:09.198092+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:43:33.370265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"80'7","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.865598+0000","last_change":"2024-05-21T21:51:12.852574+0000","last_active":"2024-05-21T21:52:10.865598+0000","last_peered":"2024-05-21T21:52:10.865598+0000","last_clean":"2024-05-21T21:52:10.865598+0000","last_became_active":"2024-05-21T21:51:12.852258+0000","last_became_peered":"2024-05-21T21:51:12.852258+0000","last_unstale":"2024-05-21T21:52:10.865598+0000","last_undegraded":"2024-05-21T21:52:10.865598+0000","last_fullsized":"2024-05-21T21:52:10.865598+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:09:40.280821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.192658+0000","last_change":"2024-05-21T21:51:16.934768+0000","last_active":"2024-05-21T21:53:09.192658+0000","last_peered":"2024-05-21T21:53:09.192658+0000","last_clean":"2024-05-21T21:53:09.192658+0000","last_became_active":"2024-05-21T21:51:16.934275+0000","last_became_peered":"2024-05-21T21:51:16.934275+0000","last_unstale":"2024-05-21T21:53:09.192658+0000","last_undegraded":"2024-05-21T21:53:09.192658+0000","last_fullsized":"2024-05-21T21:53:09.192658+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:31:54.425069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"80'3","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.861975+0000","last_change":"2024-05-21T21:51:12.854166+0000","last_active":"2024-05-21T21:52:10.861975+0000","last_peered":"2024-05-21T21:52:10.861975+0000","last_clean":"2024-05-21T21:52:10.861975+0000","last_became_active":"2024-05-21T21:51:12.853508+0000","last_became_peered":"2024-05-21T21:51:12.853508+0000","last_unstale":"2024-05-21T21:52:10.861975+0000","last_undegraded":"2024-05-21T21:52:10.861975+0000","last_fullsized":"2024-05-21T21:52:10.861975+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:29:41.858655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.218781+0000","last_change":"2024-05-21T21:51:16.938293+0000","last_active":"2024-05-21T21:53:09.218781+0000","last_peered":"2024-05-21T21:53:09.218781+0000","last_clean":"2024-05-21T21:53:09.218781+0000","last_became_active":"2024-05-21T21:51:16.937970+0000","last_became_peered":"2024-05-21T21:51:16.937970+0000","last_unstale":"2024-05-21T21:53:09.218781+0000","last_undegraded":"2024-05-21T21:53:09.218781+0000","last_fullsized":"2024-05-21T21:53:09.218781+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:18:27.473098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'3","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.866779+0000","last_change":"2024-05-21T21:51:12.853366+0000","last_active":"2024-05-21T21:52:10.866779+0000","last_peered":"2024-05-21T21:52:10.866779+0000","last_clean":"2024-05-21T21:52:10.866779+0000","last_became_active":"2024-05-21T21:51:12.853028+0000","last_became_peered":"2024-05-21T21:51:12.853028+0000","last_unstale":"2024-05-21T21:52:10.866779+0000","last_undegraded":"2024-05-21T21:52:10.866779+0000","last_fullsized":"2024-05-21T21:52:10.866779+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T09:15:52.593028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"82'11","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.205727+0000","last_change":"2024-05-21T21:51:16.947903+0000","last_active":"2024-05-21T21:53:09.205727+0000","last_peered":"2024-05-21T21:53:09.205727+0000","last_clean":"2024-05-21T21:53:09.205727+0000","last_became_active":"2024-05-21T21:51:16.946677+0000","last_became_peered":"2024-05-21T21:51:16.946677+0000","last_unstale":"2024-05-21T21:53:09.205727+0000","last_undegraded":"2024-05-21T21:53:09.205727+0000","last_fullsized":"2024-05-21T21:53:09.205727+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T06:12:56.514284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'5","reported_seq":286,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:45.567836+0000","last_change":"2024-05-21T21:51:12.854873+0000","last_active":"2024-05-21T21:53:45.567836+0000","last_peered":"2024-05-21T21:53:45.567836+0000","last_clean":"2024-05-21T21:53:45.567836+0000","last_became_active":"2024-05-21T21:51:12.854393+0000","last_became_peered":"2024-05-21T21:51:12.854393+0000","last_unstale":"2024-05-21T21:53:45.567836+0000","last_undegraded":"2024-05-21T21:53:45.567836+0000","last_fullsized":"2024-05-21T21:53:45.567836+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:58:52.411164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":254,"num_read_kb":254,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"82'10","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.219978+0000","last_change":"2024-05-21T21:51:16.951094+0000","last_active":"2024-05-21T21:53:09.219978+0000","last_peered":"2024-05-21T21:53:09.219978+0000","last_clean":"2024-05-21T21:53:09.219978+0000","last_became_active":"2024-05-21T21:51:16.950724+0000","last_became_peered":"2024-05-21T21:51:16.950724+0000","last_unstale":"2024-05-21T21:53:09.219978+0000","last_undegraded":"2024-05-21T21:53:09.219978+0000","last_fullsized":"2024-05-21T21:53:09.219978+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:54:02.835426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'4","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.864180+0000","last_change":"2024-05-21T21:51:12.850427+0000","last_active":"2024-05-21T21:52:10.864180+0000","last_peered":"2024-05-21T21:52:10.864180+0000","last_clean":"2024-05-21T21:52:10.864180+0000","last_became_active":"2024-05-21T21:51:12.850096+0000","last_became_peered":"2024-05-21T21:51:12.850096+0000","last_unstale":"2024-05-21T21:52:10.864180+0000","last_undegraded":"2024-05-21T21:52:10.864180+0000","last_fullsized":"2024-05-21T21:52:10.864180+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:43:22.611043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.238035+0000","last_change":"2024-05-21T21:51:16.942330+0000","last_active":"2024-05-21T21:53:09.238035+0000","last_peered":"2024-05-21T21:53:09.238035+0000","last_clean":"2024-05-21T21:53:09.238035+0000","last_became_active":"2024-05-21T21:51:16.941718+0000","last_became_peered":"2024-05-21T21:51:16.941718+0000","last_unstale":"2024-05-21T21:53:09.238035+0000","last_undegraded":"2024-05-21T21:53:09.238035+0000","last_fullsized":"2024-05-21T21:53:09.238035+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:54:17.176411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'5","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.867630+0000","last_change":"2024-05-21T21:51:12.941100+0000","last_active":"2024-05-21T21:52:10.867630+0000","last_peered":"2024-05-21T21:52:10.867630+0000","last_clean":"2024-05-21T21:52:10.867630+0000","last_became_active":"2024-05-21T21:51:12.940597+0000","last_became_peered":"2024-05-21T21:51:12.940597+0000","last_unstale":"2024-05-21T21:52:10.867630+0000","last_undegraded":"2024-05-21T21:52:10.867630+0000","last_fullsized":"2024-05-21T21:52:10.867630+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:30:58.457108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"82'11","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.193400+0000","last_change":"2024-05-21T21:51:16.921973+0000","last_active":"2024-05-21T21:53:09.193400+0000","last_peered":"2024-05-21T21:53:09.193400+0000","last_clean":"2024-05-21T21:53:09.193400+0000","last_became_active":"2024-05-21T21:51:16.921597+0000","last_became_peered":"2024-05-21T21:51:16.921597+0000","last_unstale":"2024-05-21T21:53:09.193400+0000","last_undegraded":"2024-05-21T21:53:09.193400+0000","last_fullsized":"2024-05-21T21:53:09.193400+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:25:32.395940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'34","reported_seq":325,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:45.567836+0000","last_change":"2024-05-21T21:51:12.850312+0000","last_active":"2024-05-21T21:53:45.567836+0000","last_peered":"2024-05-21T21:53:45.567836+0000","last_clean":"2024-05-21T21:53:45.567836+0000","last_became_active":"2024-05-21T21:51:12.849837+0000","last_became_peered":"2024-05-21T21:51:12.849837+0000","last_unstale":"2024-05-21T21:53:45.567836+0000","last_undegraded":"2024-05-21T21:53:45.567836+0000","last_fullsized":"2024-05-21T21:53:45.567836+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T07:09:13.339554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60011,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":264,"num_read_kb":190,"num_write":36,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.192679+0000","last_change":"2024-05-21T21:51:16.926625+0000","last_active":"2024-05-21T21:53:09.192679+0000","last_peered":"2024-05-21T21:53:09.192679+0000","last_clean":"2024-05-21T21:53:09.192679+0000","last_became_active":"2024-05-21T21:51:16.925889+0000","last_became_peered":"2024-05-21T21:51:16.925889+0000","last_unstale":"2024-05-21T21:53:09.192679+0000","last_undegraded":"2024-05-21T21:53:09.192679+0000","last_fullsized":"2024-05-21T21:53:09.192679+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:35:23.464747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'9","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.863955+0000","last_change":"2024-05-21T21:51:12.859772+0000","last_active":"2024-05-21T21:52:10.863955+0000","last_peered":"2024-05-21T21:52:10.863955+0000","last_clean":"2024-05-21T21:52:10.863955+0000","last_became_active":"2024-05-21T21:51:12.859290+0000","last_became_peered":"2024-05-21T21:51:12.859290+0000","last_unstale":"2024-05-21T21:52:10.863955+0000","last_undegraded":"2024-05-21T21:52:10.863955+0000","last_fullsized":"2024-05-21T21:52:10.863955+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T05:28:29.953381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.229561+0000","last_change":"2024-05-21T21:51:16.925803+0000","last_active":"2024-05-21T21:53:09.229561+0000","last_peered":"2024-05-21T21:53:09.229561+0000","last_clean":"2024-05-21T21:53:09.229561+0000","last_became_active":"2024-05-21T21:51:16.925111+0000","last_became_peered":"2024-05-21T21:51:16.925111+0000","last_unstale":"2024-05-21T21:53:09.229561+0000","last_undegraded":"2024-05-21T21:53:09.229561+0000","last_fullsized":"2024-05-21T21:53:09.229561+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T06:22:11.353379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'5","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.867387+0000","last_change":"2024-05-21T21:51:12.859593+0000","last_active":"2024-05-21T21:52:10.867387+0000","last_peered":"2024-05-21T21:52:10.867387+0000","last_clean":"2024-05-21T21:52:10.867387+0000","last_became_active":"2024-05-21T21:51:12.859197+0000","last_became_peered":"2024-05-21T21:51:12.859197+0000","last_unstale":"2024-05-21T21:52:10.867387+0000","last_undegraded":"2024-05-21T21:52:10.867387+0000","last_fullsized":"2024-05-21T21:52:10.867387+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:13:12.311496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"82'15","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.236923+0000","last_change":"2024-05-21T21:52:10.879028+0000","last_active":"2024-05-21T21:53:09.236923+0000","last_peered":"2024-05-21T21:53:09.236923+0000","last_clean":"2024-05-21T21:53:09.236923+0000","last_became_active":"2024-05-21T21:52:10.878664+0000","last_became_peered":"2024-05-21T21:52:10.878664+0000","last_unstale":"2024-05-21T21:53:09.236923+0000","last_undegraded":"2024-05-21T21:53:09.236923+0000","last_fullsized":"2024-05-21T21:53:09.236923+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:05:53.953546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'7","reported_seq":167,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:46.620116+0000","last_change":"2024-05-21T21:51:12.852169+0000","last_active":"2024-05-21T21:53:46.620116+0000","last_peered":"2024-05-21T21:53:46.620116+0000","last_clean":"2024-05-21T21:53:46.620116+0000","last_became_active":"2024-05-21T21:51:12.851782+0000","last_became_peered":"2024-05-21T21:51:12.851782+0000","last_unstale":"2024-05-21T21:53:46.620116+0000","last_undegraded":"2024-05-21T21:53:46.620116+0000","last_fullsized":"2024-05-21T21:53:46.620116+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:46:09.692158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":134,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.194105+0000","last_change":"2024-05-21T21:51:16.929264+0000","last_active":"2024-05-21T21:53:09.194105+0000","last_peered":"2024-05-21T21:53:09.194105+0000","last_clean":"2024-05-21T21:53:09.194105+0000","last_became_active":"2024-05-21T21:51:16.928900+0000","last_became_peered":"2024-05-21T21:51:16.928900+0000","last_unstale":"2024-05-21T21:53:09.194105+0000","last_undegraded":"2024-05-21T21:53:09.194105+0000","last_fullsized":"2024-05-21T21:53:09.194105+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:42:28.040939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'17","reported_seq":69,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.984839+0000","last_change":"2024-05-21T21:52:10.984451+0000","last_active":"2024-05-21T21:52:10.984839+0000","last_peered":"2024-05-21T21:52:10.984839+0000","last_clean":"2024-05-21T21:52:10.984839+0000","last_became_active":"2024-05-21T21:52:10.974050+0000","last_became_peered":"2024-05-21T21:52:10.974050+0000","last_unstale":"2024-05-21T21:52:10.984839+0000","last_undegraded":"2024-05-21T21:52:10.984839+0000","last_fullsized":"2024-05-21T21:52:10.984839+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T07:08:18.307022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.189550+0000","last_change":"2024-05-21T21:51:16.935464+0000","last_active":"2024-05-21T21:53:09.189550+0000","last_peered":"2024-05-21T21:53:09.189550+0000","last_clean":"2024-05-21T21:53:09.189550+0000","last_became_active":"2024-05-21T21:51:16.935091+0000","last_became_peered":"2024-05-21T21:51:16.935091+0000","last_unstale":"2024-05-21T21:53:09.189550+0000","last_undegraded":"2024-05-21T21:53:09.189550+0000","last_fullsized":"2024-05-21T21:53:09.189550+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T06:59:16.330649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.864584+0000","last_change":"2024-05-21T21:51:12.853327+0000","last_active":"2024-05-21T21:52:10.864584+0000","last_peered":"2024-05-21T21:52:10.864584+0000","last_clean":"2024-05-21T21:52:10.864584+0000","last_became_active":"2024-05-21T21:51:12.852946+0000","last_became_peered":"2024-05-21T21:51:12.852946+0000","last_unstale":"2024-05-21T21:52:10.864584+0000","last_undegraded":"2024-05-21T21:52:10.864584+0000","last_fullsized":"2024-05-21T21:52:10.864584+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:00:34.256858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"82'5","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.184045+0000","last_change":"2024-05-21T21:51:16.929341+0000","last_active":"2024-05-21T21:53:09.184045+0000","last_peered":"2024-05-21T21:53:09.184045+0000","last_clean":"2024-05-21T21:53:09.184045+0000","last_became_active":"2024-05-21T21:51:16.929008+0000","last_became_peered":"2024-05-21T21:51:16.929008+0000","last_unstale":"2024-05-21T21:53:09.184045+0000","last_undegraded":"2024-05-21T21:53:09.184045+0000","last_fullsized":"2024-05-21T21:53:09.184045+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:26:26.297136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"80'1","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.876017+0000","last_change":"2024-05-21T21:51:12.945433+0000","last_active":"2024-05-21T21:52:10.876017+0000","last_peered":"2024-05-21T21:52:10.876017+0000","last_clean":"2024-05-21T21:52:10.876017+0000","last_became_active":"2024-05-21T21:51:12.944463+0000","last_became_peered":"2024-05-21T21:51:12.944463+0000","last_unstale":"2024-05-21T21:52:10.876017+0000","last_undegraded":"2024-05-21T21:52:10.876017+0000","last_fullsized":"2024-05-21T21:52:10.876017+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:29:52.014583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'5","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.215224+0000","last_change":"2024-05-21T21:52:10.883554+0000","last_active":"2024-05-21T21:53:09.215224+0000","last_peered":"2024-05-21T21:53:09.215224+0000","last_clean":"2024-05-21T21:53:09.215224+0000","last_became_active":"2024-05-21T21:52:10.877789+0000","last_became_peered":"2024-05-21T21:52:10.877789+0000","last_unstale":"2024-05-21T21:53:09.215224+0000","last_undegraded":"2024-05-21T21:53:09.215224+0000","last_fullsized":"2024-05-21T21:53:09.215224+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:52:37.623551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"80'5","reported_seq":159,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:43.641688+0000","last_change":"2024-05-21T21:51:12.852883+0000","last_active":"2024-05-21T21:53:43.641688+0000","last_peered":"2024-05-21T21:53:43.641688+0000","last_clean":"2024-05-21T21:53:43.641688+0000","last_became_active":"2024-05-21T21:51:12.852549+0000","last_became_peered":"2024-05-21T21:51:12.852549+0000","last_unstale":"2024-05-21T21:53:43.641688+0000","last_undegraded":"2024-05-21T21:53:43.641688+0000","last_fullsized":"2024-05-21T21:53:43.641688+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:05:53.002286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":127,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.199218+0000","last_change":"2024-05-21T21:51:16.944425+0000","last_active":"2024-05-21T21:53:09.199218+0000","last_peered":"2024-05-21T21:53:09.199218+0000","last_clean":"2024-05-21T21:53:09.199218+0000","last_became_active":"2024-05-21T21:51:16.944033+0000","last_became_peered":"2024-05-21T21:51:16.944033+0000","last_unstale":"2024-05-21T21:53:09.199218+0000","last_undegraded":"2024-05-21T21:53:09.199218+0000","last_fullsized":"2024-05-21T21:53:09.199218+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T07:16:26.802329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.186645+0000","last_change":"2024-05-21T21:51:16.933264+0000","last_active":"2024-05-21T21:53:09.186645+0000","last_peered":"2024-05-21T21:53:09.186645+0000","last_clean":"2024-05-21T21:53:09.186645+0000","last_became_active":"2024-05-21T21:51:16.932911+0000","last_became_peered":"2024-05-21T21:51:16.932911+0000","last_unstale":"2024-05-21T21:53:09.186645+0000","last_undegraded":"2024-05-21T21:53:09.186645+0000","last_fullsized":"2024-05-21T21:53:09.186645+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:25:54.006367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"82'6","reported_seq":160,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:43.612325+0000","last_change":"2024-05-21T21:51:12.942900+0000","last_active":"2024-05-21T21:53:43.612325+0000","last_peered":"2024-05-21T21:53:43.612325+0000","last_clean":"2024-05-21T21:53:43.612325+0000","last_became_active":"2024-05-21T21:51:12.941038+0000","last_became_peered":"2024-05-21T21:51:12.941038+0000","last_unstale":"2024-05-21T21:53:43.612325+0000","last_undegraded":"2024-05-21T21:53:43.612325+0000","last_fullsized":"2024-05-21T21:53:43.612325+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:38:43.240011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":127,"num_read_kb":127,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.186935+0000","last_change":"2024-05-21T21:51:16.933391+0000","last_active":"2024-05-21T21:53:09.186935+0000","last_peered":"2024-05-21T21:53:09.186935+0000","last_clean":"2024-05-21T21:53:09.186935+0000","last_became_active":"2024-05-21T21:51:16.932320+0000","last_became_peered":"2024-05-21T21:51:16.932320+0000","last_unstale":"2024-05-21T21:53:09.186935+0000","last_undegraded":"2024-05-21T21:53:09.186935+0000","last_fullsized":"2024-05-21T21:53:09.186935+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T09:06:59.126748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'6","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.865253+0000","last_change":"2024-05-21T21:51:12.856342+0000","last_active":"2024-05-21T21:52:10.865253+0000","last_peered":"2024-05-21T21:52:10.865253+0000","last_clean":"2024-05-21T21:52:10.865253+0000","last_became_active":"2024-05-21T21:51:12.855952+0000","last_became_peered":"2024-05-21T21:51:12.855952+0000","last_unstale":"2024-05-21T21:52:10.865253+0000","last_undegraded":"2024-05-21T21:52:10.865253+0000","last_fullsized":"2024-05-21T21:52:10.865253+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:34:40.734034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":100,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.863960+0000","last_change":"2024-05-21T21:49:42.797187+0000","last_active":"2024-05-21T21:52:10.863960+0000","last_peered":"2024-05-21T21:52:10.863960+0000","last_clean":"2024-05-21T21:52:10.863960+0000","last_became_active":"2024-05-21T21:49:42.791291+0000","last_became_peered":"2024-05-21T21:49:42.791291+0000","last_unstale":"2024-05-21T21:52:10.863960+0000","last_undegraded":"2024-05-21T21:52:10.863960+0000","last_fullsized":"2024-05-21T21:52:10.863960+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:45:55.552293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:45:55.552293+0000","last_clean_scrub_stamp":"2024-05-21T21:45:55.552293+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T07:24:18.302523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"82'5","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.216448+0000","last_change":"2024-05-21T21:51:16.926446+0000","last_active":"2024-05-21T21:53:09.216448+0000","last_peered":"2024-05-21T21:53:09.216448+0000","last_clean":"2024-05-21T21:53:09.216448+0000","last_became_active":"2024-05-21T21:51:16.926070+0000","last_became_peered":"2024-05-21T21:51:16.926070+0000","last_unstale":"2024-05-21T21:53:09.216448+0000","last_undegraded":"2024-05-21T21:53:09.216448+0000","last_fullsized":"2024-05-21T21:53:09.216448+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:44:27.366393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'4","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.864466+0000","last_change":"2024-05-21T21:51:12.854371+0000","last_active":"2024-05-21T21:52:10.864466+0000","last_peered":"2024-05-21T21:52:10.864466+0000","last_clean":"2024-05-21T21:52:10.864466+0000","last_became_active":"2024-05-21T21:51:12.853847+0000","last_became_peered":"2024-05-21T21:51:12.853847+0000","last_unstale":"2024-05-21T21:52:10.864466+0000","last_undegraded":"2024-05-21T21:52:10.864466+0000","last_fullsized":"2024-05-21T21:52:10.864466+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:14:10.549901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"80'4","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.863735+0000","last_change":"2024-05-21T21:51:12.941227+0000","last_active":"2024-05-21T21:52:10.863735+0000","last_peered":"2024-05-21T21:52:10.863735+0000","last_clean":"2024-05-21T21:52:10.863735+0000","last_became_active":"2024-05-21T21:51:12.940702+0000","last_became_peered":"2024-05-21T21:51:12.940702+0000","last_unstale":"2024-05-21T21:52:10.863735+0000","last_undegraded":"2024-05-21T21:52:10.863735+0000","last_fullsized":"2024-05-21T21:52:10.863735+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:40:34.124921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.205536+0000","last_change":"2024-05-21T21:51:16.945627+0000","last_active":"2024-05-21T21:53:09.205536+0000","last_peered":"2024-05-21T21:53:09.205536+0000","last_clean":"2024-05-21T21:53:09.205536+0000","last_became_active":"2024-05-21T21:51:16.945297+0000","last_became_peered":"2024-05-21T21:51:16.945297+0000","last_unstale":"2024-05-21T21:53:09.205536+0000","last_undegraded":"2024-05-21T21:53:09.205536+0000","last_fullsized":"2024-05-21T21:53:09.205536+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:54:38.898774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'16","reported_seq":59,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.957822+0000","last_change":"2024-05-21T21:52:10.888076+0000","last_active":"2024-05-21T21:53:09.957822+0000","last_peered":"2024-05-21T21:53:09.957822+0000","last_clean":"2024-05-21T21:53:09.957822+0000","last_became_active":"2024-05-21T21:52:10.879368+0000","last_became_peered":"2024-05-21T21:52:10.879368+0000","last_unstale":"2024-05-21T21:53:09.957822+0000","last_undegraded":"2024-05-21T21:53:09.957822+0000","last_fullsized":"2024-05-21T21:53:09.957822+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:46:43.190411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"82'5","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.194548+0000","last_change":"2024-05-21T21:51:16.929807+0000","last_active":"2024-05-21T21:53:09.194548+0000","last_peered":"2024-05-21T21:53:09.194548+0000","last_clean":"2024-05-21T21:53:09.194548+0000","last_became_active":"2024-05-21T21:51:16.928859+0000","last_became_peered":"2024-05-21T21:51:16.928859+0000","last_unstale":"2024-05-21T21:53:09.194548+0000","last_undegraded":"2024-05-21T21:53:09.194548+0000","last_fullsized":"2024-05-21T21:53:09.194548+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:41:32.830892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'7","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.865650+0000","last_change":"2024-05-21T21:51:12.850843+0000","last_active":"2024-05-21T21:52:10.865650+0000","last_peered":"2024-05-21T21:52:10.865650+0000","last_clean":"2024-05-21T21:52:10.865650+0000","last_became_active":"2024-05-21T21:51:12.850516+0000","last_became_peered":"2024-05-21T21:51:12.850516+0000","last_unstale":"2024-05-21T21:52:10.865650+0000","last_undegraded":"2024-05-21T21:52:10.865650+0000","last_fullsized":"2024-05-21T21:52:10.865650+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:37:05.191626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.237987+0000","last_change":"2024-05-21T21:51:16.949017+0000","last_active":"2024-05-21T21:53:09.237987+0000","last_peered":"2024-05-21T21:53:09.237987+0000","last_clean":"2024-05-21T21:53:09.237987+0000","last_became_active":"2024-05-21T21:51:16.948691+0000","last_became_peered":"2024-05-21T21:51:16.948691+0000","last_unstale":"2024-05-21T21:53:09.237987+0000","last_undegraded":"2024-05-21T21:53:09.237987+0000","last_fullsized":"2024-05-21T21:53:09.237987+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:32:06.606581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.973226+0000","last_change":"2024-05-21T21:51:12.939203+0000","last_active":"2024-05-21T21:52:10.973226+0000","last_peered":"2024-05-21T21:52:10.973226+0000","last_clean":"2024-05-21T21:52:10.973226+0000","last_became_active":"2024-05-21T21:51:12.938936+0000","last_became_peered":"2024-05-21T21:51:12.938936+0000","last_unstale":"2024-05-21T21:52:10.973226+0000","last_undegraded":"2024-05-21T21:52:10.973226+0000","last_fullsized":"2024-05-21T21:52:10.973226+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:55:50.336362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"82'2","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.174421+0000","last_change":"2024-05-21T21:51:16.926216+0000","last_active":"2024-05-21T21:53:09.174421+0000","last_peered":"2024-05-21T21:53:09.174421+0000","last_clean":"2024-05-21T21:53:09.174421+0000","last_became_active":"2024-05-21T21:51:16.925834+0000","last_became_peered":"2024-05-21T21:51:16.925834+0000","last_unstale":"2024-05-21T21:53:09.174421+0000","last_undegraded":"2024-05-21T21:53:09.174421+0000","last_fullsized":"2024-05-21T21:53:09.174421+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:13:30.192908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'7","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.866611+0000","last_change":"2024-05-21T21:51:12.855306+0000","last_active":"2024-05-21T21:52:10.866611+0000","last_peered":"2024-05-21T21:52:10.866611+0000","last_clean":"2024-05-21T21:52:10.866611+0000","last_became_active":"2024-05-21T21:51:12.854722+0000","last_became_peered":"2024-05-21T21:51:12.854722+0000","last_unstale":"2024-05-21T21:52:10.866611+0000","last_undegraded":"2024-05-21T21:52:10.866611+0000","last_fullsized":"2024-05-21T21:52:10.866611+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:39:41.583219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.199172+0000","last_change":"2024-05-21T21:51:16.944758+0000","last_active":"2024-05-21T21:53:09.199172+0000","last_peered":"2024-05-21T21:53:09.199172+0000","last_clean":"2024-05-21T21:53:09.199172+0000","last_became_active":"2024-05-21T21:51:16.944393+0000","last_became_peered":"2024-05-21T21:51:16.944393+0000","last_unstale":"2024-05-21T21:53:09.199172+0000","last_undegraded":"2024-05-21T21:53:09.199172+0000","last_fullsized":"2024-05-21T21:53:09.199172+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:01:23.048355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'3","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.864860+0000","last_change":"2024-05-21T21:51:12.860378+0000","last_active":"2024-05-21T21:52:10.864860+0000","last_peered":"2024-05-21T21:52:10.864860+0000","last_clean":"2024-05-21T21:52:10.864860+0000","last_became_active":"2024-05-21T21:51:12.859978+0000","last_became_peered":"2024-05-21T21:51:12.859978+0000","last_unstale":"2024-05-21T21:52:10.864860+0000","last_undegraded":"2024-05-21T21:52:10.864860+0000","last_fullsized":"2024-05-21T21:52:10.864860+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T07:52:26.902232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.229331+0000","last_change":"2024-05-21T21:51:16.943194+0000","last_active":"2024-05-21T21:53:09.229331+0000","last_peered":"2024-05-21T21:53:09.229331+0000","last_clean":"2024-05-21T21:53:09.229331+0000","last_became_active":"2024-05-21T21:51:16.942884+0000","last_became_peered":"2024-05-21T21:51:16.942884+0000","last_unstale":"2024-05-21T21:53:09.229331+0000","last_undegraded":"2024-05-21T21:53:09.229331+0000","last_fullsized":"2024-05-21T21:53:09.229331+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T05:58:17.436995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.863937+0000","last_change":"2024-05-21T21:51:12.853390+0000","last_active":"2024-05-21T21:52:10.863937+0000","last_peered":"2024-05-21T21:52:10.863937+0000","last_clean":"2024-05-21T21:52:10.863937+0000","last_became_active":"2024-05-21T21:51:12.852990+0000","last_became_peered":"2024-05-21T21:51:12.852990+0000","last_unstale":"2024-05-21T21:52:10.863937+0000","last_undegraded":"2024-05-21T21:52:10.863937+0000","last_fullsized":"2024-05-21T21:52:10.863937+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:32:16.392609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"82'5","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.235699+0000","last_change":"2024-05-21T21:51:16.933430+0000","last_active":"2024-05-21T21:53:09.235699+0000","last_peered":"2024-05-21T21:53:09.235699+0000","last_clean":"2024-05-21T21:53:09.235699+0000","last_became_active":"2024-05-21T21:51:16.933053+0000","last_became_peered":"2024-05-21T21:51:16.933053+0000","last_unstale":"2024-05-21T21:53:09.235699+0000","last_undegraded":"2024-05-21T21:53:09.235699+0000","last_fullsized":"2024-05-21T21:53:09.235699+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T06:14:57.468727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'5","reported_seq":160,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:43.565162+0000","last_change":"2024-05-21T21:51:12.946113+0000","last_active":"2024-05-21T21:53:43.565162+0000","last_peered":"2024-05-21T21:53:43.565162+0000","last_clean":"2024-05-21T21:53:43.565162+0000","last_became_active":"2024-05-21T21:51:12.945650+0000","last_became_peered":"2024-05-21T21:51:12.945650+0000","last_unstale":"2024-05-21T21:53:43.565162+0000","last_undegraded":"2024-05-21T21:53:43.565162+0000","last_fullsized":"2024-05-21T21:53:43.565162+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T09:40:18.209400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":128,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.195332+0000","last_change":"2024-05-21T21:51:16.932632+0000","last_active":"2024-05-21T21:53:09.195332+0000","last_peered":"2024-05-21T21:53:09.195332+0000","last_clean":"2024-05-21T21:53:09.195332+0000","last_became_active":"2024-05-21T21:51:16.932256+0000","last_became_peered":"2024-05-21T21:51:16.932256+0000","last_unstale":"2024-05-21T21:53:09.195332+0000","last_undegraded":"2024-05-21T21:53:09.195332+0000","last_fullsized":"2024-05-21T21:53:09.195332+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:04:48.146485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'3","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.867614+0000","last_change":"2024-05-21T21:51:12.860602+0000","last_active":"2024-05-21T21:52:10.867614+0000","last_peered":"2024-05-21T21:52:10.867614+0000","last_clean":"2024-05-21T21:52:10.867614+0000","last_became_active":"2024-05-21T21:51:12.860228+0000","last_became_peered":"2024-05-21T21:51:12.860228+0000","last_unstale":"2024-05-21T21:52:10.867614+0000","last_undegraded":"2024-05-21T21:52:10.867614+0000","last_fullsized":"2024-05-21T21:52:10.867614+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:33:39.230872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.196150+0000","last_change":"2024-05-21T21:51:16.929616+0000","last_active":"2024-05-21T21:53:09.196150+0000","last_peered":"2024-05-21T21:53:09.196150+0000","last_clean":"2024-05-21T21:53:09.196150+0000","last_became_active":"2024-05-21T21:51:16.928569+0000","last_became_peered":"2024-05-21T21:51:16.928569+0000","last_unstale":"2024-05-21T21:53:09.196150+0000","last_undegraded":"2024-05-21T21:53:09.196150+0000","last_fullsized":"2024-05-21T21:53:09.196150+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:15:41.790344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'5","reported_seq":44,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.881746+0000","last_change":"2024-05-21T21:52:10.881114+0000","last_active":"2024-05-21T21:52:10.881746+0000","last_peered":"2024-05-21T21:52:10.881746+0000","last_clean":"2024-05-21T21:52:10.881746+0000","last_became_active":"2024-05-21T21:52:10.872022+0000","last_became_peered":"2024-05-21T21:52:10.872022+0000","last_unstale":"2024-05-21T21:52:10.881746+0000","last_undegraded":"2024-05-21T21:52:10.881746+0000","last_fullsized":"2024-05-21T21:52:10.881746+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T06:05:51.419156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.193224+0000","last_change":"2024-05-21T21:51:16.926496+0000","last_active":"2024-05-21T21:53:09.193224+0000","last_peered":"2024-05-21T21:53:09.193224+0000","last_clean":"2024-05-21T21:53:09.193224+0000","last_became_active":"2024-05-21T21:51:16.926112+0000","last_became_peered":"2024-05-21T21:51:16.926112+0000","last_unstale":"2024-05-21T21:53:09.193224+0000","last_undegraded":"2024-05-21T21:53:09.193224+0000","last_fullsized":"2024-05-21T21:53:09.193224+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:26:08.254421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.185320+0000","last_change":"2024-05-21T21:51:16.930522+0000","last_active":"2024-05-21T21:53:09.185320+0000","last_peered":"2024-05-21T21:53:09.185320+0000","last_clean":"2024-05-21T21:53:09.185320+0000","last_became_active":"2024-05-21T21:51:16.929853+0000","last_became_peered":"2024-05-21T21:51:16.929853+0000","last_unstale":"2024-05-21T21:53:09.185320+0000","last_undegraded":"2024-05-21T21:53:09.185320+0000","last_fullsized":"2024-05-21T21:53:09.185320+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:10:57.940009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"82'12","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.185369+0000","last_change":"2024-05-21T21:51:16.931751+0000","last_active":"2024-05-21T21:53:09.185369+0000","last_peered":"2024-05-21T21:53:09.185369+0000","last_clean":"2024-05-21T21:53:09.185369+0000","last_became_active":"2024-05-21T21:51:16.931353+0000","last_became_peered":"2024-05-21T21:51:16.931353+0000","last_unstale":"2024-05-21T21:53:09.185369+0000","last_undegraded":"2024-05-21T21:53:09.185369+0000","last_fullsized":"2024-05-21T21:53:09.185369+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:33:45.297850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.206238+0000","last_change":"2024-05-21T21:51:16.940096+0000","last_active":"2024-05-21T21:53:09.206238+0000","last_peered":"2024-05-21T21:53:09.206238+0000","last_clean":"2024-05-21T21:53:09.206238+0000","last_became_active":"2024-05-21T21:51:16.939740+0000","last_became_peered":"2024-05-21T21:51:16.939740+0000","last_unstale":"2024-05-21T21:53:09.206238+0000","last_undegraded":"2024-05-21T21:53:09.206238+0000","last_fullsized":"2024-05-21T21:53:09.206238+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:59:52.569526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.238448+0000","last_change":"2024-05-21T21:51:16.946488+0000","last_active":"2024-05-21T21:53:09.238448+0000","last_peered":"2024-05-21T21:53:09.238448+0000","last_clean":"2024-05-21T21:53:09.238448+0000","last_became_active":"2024-05-21T21:51:16.945957+0000","last_became_peered":"2024-05-21T21:51:16.945957+0000","last_unstale":"2024-05-21T21:53:09.238448+0000","last_undegraded":"2024-05-21T21:53:09.238448+0000","last_fullsized":"2024-05-21T21:53:09.238448+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:40:36.708344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"82'4","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.231934+0000","last_change":"2024-05-21T21:51:16.949349+0000","last_active":"2024-05-21T21:53:09.231934+0000","last_peered":"2024-05-21T21:53:09.231934+0000","last_clean":"2024-05-21T21:53:09.231934+0000","last_became_active":"2024-05-21T21:51:16.946986+0000","last_became_peered":"2024-05-21T21:51:16.946986+0000","last_unstale":"2024-05-21T21:53:09.231934+0000","last_undegraded":"2024-05-21T21:53:09.231934+0000","last_fullsized":"2024-05-21T21:53:09.231934+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:01:14.877110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.181433+0000","last_change":"2024-05-21T21:51:16.929172+0000","last_active":"2024-05-21T21:53:09.181433+0000","last_peered":"2024-05-21T21:53:09.181433+0000","last_clean":"2024-05-21T21:53:09.181433+0000","last_became_active":"2024-05-21T21:51:16.928823+0000","last_became_peered":"2024-05-21T21:51:16.928823+0000","last_unstale":"2024-05-21T21:53:09.181433+0000","last_undegraded":"2024-05-21T21:53:09.181433+0000","last_fullsized":"2024-05-21T21:53:09.181433+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:09:13.719272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"82'15","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.186944+0000","last_change":"2024-05-21T21:51:16.930332+0000","last_active":"2024-05-21T21:53:09.186944+0000","last_peered":"2024-05-21T21:53:09.186944+0000","last_clean":"2024-05-21T21:53:09.186944+0000","last_became_active":"2024-05-21T21:51:16.930024+0000","last_became_peered":"2024-05-21T21:51:16.930024+0000","last_unstale":"2024-05-21T21:53:09.186944+0000","last_undegraded":"2024-05-21T21:53:09.186944+0000","last_fullsized":"2024-05-21T21:53:09.186944+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:47:41.272725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"82'2","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.198573+0000","last_change":"2024-05-21T21:51:16.930102+0000","last_active":"2024-05-21T21:53:09.198573+0000","last_peered":"2024-05-21T21:53:09.198573+0000","last_clean":"2024-05-21T21:53:09.198573+0000","last_became_active":"2024-05-21T21:51:16.929410+0000","last_became_peered":"2024-05-21T21:51:16.929410+0000","last_unstale":"2024-05-21T21:53:09.198573+0000","last_undegraded":"2024-05-21T21:53:09.198573+0000","last_fullsized":"2024-05-21T21:53:09.198573+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:11:46.296288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"82'6","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.199470+0000","last_change":"2024-05-21T21:52:10.873424+0000","last_active":"2024-05-21T21:53:09.199470+0000","last_peered":"2024-05-21T21:53:09.199470+0000","last_clean":"2024-05-21T21:53:09.199470+0000","last_became_active":"2024-05-21T21:52:10.873118+0000","last_became_peered":"2024-05-21T21:52:10.873118+0000","last_unstale":"2024-05-21T21:53:09.199470+0000","last_undegraded":"2024-05-21T21:53:09.199470+0000","last_fullsized":"2024-05-21T21:53:09.199470+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:05:00.731013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,3],"acting":[8,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.29","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.197868+0000","last_change":"2024-05-21T21:51:16.934442+0000","last_active":"2024-05-21T21:53:09.197868+0000","last_peered":"2024-05-21T21:53:09.197868+0000","last_clean":"2024-05-21T21:53:09.197868+0000","last_became_active":"2024-05-21T21:51:16.934086+0000","last_became_peered":"2024-05-21T21:51:16.934086+0000","last_unstale":"2024-05-21T21:53:09.197868+0000","last_undegraded":"2024-05-21T21:53:09.197868+0000","last_fullsized":"2024-05-21T21:53:09.197868+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:36:44.302532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"82'10","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.199678+0000","last_change":"2024-05-21T21:51:16.924479+0000","last_active":"2024-05-21T21:53:09.199678+0000","last_peered":"2024-05-21T21:53:09.199678+0000","last_clean":"2024-05-21T21:53:09.199678+0000","last_became_active":"2024-05-21T21:51:16.923998+0000","last_became_peered":"2024-05-21T21:51:16.923998+0000","last_unstale":"2024-05-21T21:53:09.199678+0000","last_undegraded":"2024-05-21T21:53:09.199678+0000","last_fullsized":"2024-05-21T21:53:09.199678+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:23:07.931731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"82'11","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.235733+0000","last_change":"2024-05-21T21:51:16.947271+0000","last_active":"2024-05-21T21:53:09.235733+0000","last_peered":"2024-05-21T21:53:09.235733+0000","last_clean":"2024-05-21T21:53:09.235733+0000","last_became_active":"2024-05-21T21:51:16.946912+0000","last_became_peered":"2024-05-21T21:51:16.946912+0000","last_unstale":"2024-05-21T21:53:09.235733+0000","last_undegraded":"2024-05-21T21:53:09.235733+0000","last_fullsized":"2024-05-21T21:53:09.235733+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:51:29.646915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.184712+0000","last_change":"2024-05-21T21:51:16.928852+0000","last_active":"2024-05-21T21:53:09.184712+0000","last_peered":"2024-05-21T21:53:09.184712+0000","last_clean":"2024-05-21T21:53:09.184712+0000","last_became_active":"2024-05-21T21:51:16.928480+0000","last_became_peered":"2024-05-21T21:51:16.928480+0000","last_unstale":"2024-05-21T21:53:09.184712+0000","last_undegraded":"2024-05-21T21:53:09.184712+0000","last_fullsized":"2024-05-21T21:53:09.184712+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T09:12:30.691254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67497,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1312,"num_read_kb":728,"num_write":254,"num_write_kb":142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":112,"num_keys_recovered":17,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":202491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":218,"ondisk_log_size":218,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645360,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":901120,"data_stored":433959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841588,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28580,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716572,"statfs":{"total":95995035648,"available":95965769728,"internally_reserved":0,"allocated":1601536,"data_stored":1085531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005113,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":929792,"data_stored":434006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201344,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":921600,"data_stored":434267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397572,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28500,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716652,"statfs":{"total":95995035648,"available":95965851648,"internally_reserved":0,"allocated":1519616,"data_stored":1027458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561098,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":909312,"data_stored":433606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574950,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":494642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738476,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36244,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708908,"statfs":{"total":95995035648,"available":95957921792,"internally_reserved":0,"allocated":929792,"data_stored":434945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411426,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36296,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708856,"statfs":{"total":95995035648,"available":95957868544,"internally_reserved":0,"allocated":983040,"data_stored":495170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149787,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36236,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708916,"statfs":{"total":95995035648,"available":95957929984,"internally_reserved":0,"allocated":921600,"data_stored":433314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953558,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":909312,"data_stored":433999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757329,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32672,"kb_used_data":1496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712480,"statfs":{"total":95995035648,"available":95961579520,"internally_reserved":0,"allocated":1531904,"data_stored":1026604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":61875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":62403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":62279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-21T21:53:51.565 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph pg dump --format=json 2024-05-21T21:53:51.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:53:51 smithi184 ceph-mon[36942]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T21:53:51.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:53:51 smithi164 ceph-mon[36269]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T21:53:52.056 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:53:52.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:53:52 smithi151 ceph-mon[31623]: pgmap v393: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-21T21:53:52.637 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:53:52 smithi184 ceph-mon[36942]: pgmap v393: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-21T21:53:52.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:53:52 smithi164 ceph-mon[36269]: pgmap v393: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-21T21:53:54.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:54.193+0000 7f747dc00700 1 -- 172.21.15.151:0/517199052 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7478105670 msgr2=0x7f7478105af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:54.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:54.193+0000 7f747dc00700 1 --2- 172.21.15.151:0/517199052 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7478105670 0x7f7478105af0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7460009990 tx=0x7f7460009ca0 comp rx=0 tx=0).stop 2024-05-21T21:53:54.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:54.193+0000 7f747dc00700 1 -- 172.21.15.151:0/517199052 shutdown_connections 2024-05-21T21:53:54.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:54.193+0000 7f747dc00700 1 --2- 172.21.15.151:0/517199052 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f747810b670 0x7f747810ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:54.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:54.193+0000 7f747dc00700 1 --2- 172.21.15.151:0/517199052 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7478106030 0x7f7478112540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:54.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:54.193+0000 7f747dc00700 1 --2- 172.21.15.151:0/517199052 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7478105670 0x7f7478105af0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:54.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:54.193+0000 7f747dc00700 1 -- 172.21.15.151:0/517199052 >> 172.21.15.151:0/517199052 conn(0x7f7478101160 msgr2=0x7f7478103580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:53:54.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:54.193+0000 7f747dc00700 1 -- 172.21.15.151:0/517199052 shutdown_connections 2024-05-21T21:53:54.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:54.193+0000 7f747dc00700 1 -- 172.21.15.151:0/517199052 wait complete. 2024-05-21T21:53:54.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:54.194+0000 7f747dc00700 1 Processor -- start 2024-05-21T21:53:54.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:54.195+0000 7f747dc00700 1 -- start start 2024-05-21T21:53:54.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:54.195+0000 7f747dc00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7478105670 0x7f747819bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:54.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:54.195+0000 7f747dc00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7478106030 0x7f747819c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:54.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:54.195+0000 7f747dc00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f747810b670 0x7f74781a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:54.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:54.195+0000 7f747dc00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7478115900 con 0x7f747810b670 2024-05-21T21:53:54.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:54.195+0000 7f747dc00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7478115780 con 0x7f7478106030 2024-05-21T21:53:54.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:54.195+0000 7f747dc00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7478115a80 con 0x7f7478105670 2024-05-21T21:53:54.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:54.196+0000 7f7477e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7478105670 0x7f747819bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:54.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:54.196+0000 7f7477e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7478105670 0x7f747819bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:49942/0 (socket says 172.21.15.151:49942) 2024-05-21T21:53:54.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:54.196+0000 7f7477e00700 1 -- 172.21.15.151:0/4276382784 learned_addr learned my addr 172.21.15.151:0/4276382784 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:53:54.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:54.196+0000 7f747d200700 1 --2- 172.21.15.151:0/4276382784 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f747810b670 0x7f74781a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:54.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:54.196+0000 7f746fe00700 1 --2- 172.21.15.151:0/4276382784 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7478106030 0x7f747819c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:54.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:54.196+0000 7f7477e00700 1 -- 172.21.15.151:0/4276382784 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7478106030 msgr2=0x7f747819c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:54.198 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:54.196+0000 7f7477e00700 1 --2- 172.21.15.151:0/4276382784 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7478106030 0x7f747819c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:54.198 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:54.196+0000 7f7477e00700 1 -- 172.21.15.151:0/4276382784 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f747810b670 msgr2=0x7f74781a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:54.198 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:54.196+0000 7f7477e00700 1 --2- 172.21.15.151:0/4276382784 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f747810b670 0x7f74781a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:54.198 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:54.196+0000 7f7477e00700 1 -- 172.21.15.151:0/4276382784 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74781a0d60 con 0x7f7478105670 2024-05-21T21:53:54.198 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:54.196+0000 7f7477e00700 1 --2- 172.21.15.151:0/4276382784 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7478105670 0x7f747819bd50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f746800eb30 tx=0x7f746800ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:53:54.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:54.197+0000 7f7475600700 1 -- 172.21.15.151:0/4276382784 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f746800cbd0 con 0x7f7478105670 2024-05-21T21:53:54.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:54.197+0000 7f7475600700 1 -- 172.21.15.151:0/4276382784 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74680044d0 con 0x7f7478105670 2024-05-21T21:53:54.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:54.197+0000 7f747dc00700 1 -- 172.21.15.151:0/4276382784 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74781a1050 con 0x7f7478105670 2024-05-21T21:53:54.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:54.197+0000 7f7475600700 1 -- 172.21.15.151:0/4276382784 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7468003e30 con 0x7f7478105670 2024-05-21T21:53:54.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:54.197+0000 7f747dc00700 1 -- 172.21.15.151:0/4276382784 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74781a88d0 con 0x7f7478105670 2024-05-21T21:53:54.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:54.198+0000 7f747dc00700 1 -- 172.21.15.151:0/4276382784 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f747806a230 con 0x7f7478105670 2024-05-21T21:53:54.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:54.200+0000 7f7475600700 1 -- 172.21.15.151:0/4276382784 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f7468004000 con 0x7f7478105670 2024-05-21T21:53:54.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:54.202+0000 7f7475600700 1 --2- 172.21.15.151:0/4276382784 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f745c0b2900 0x7f745c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:54.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:54.202+0000 7f7475600700 1 -- 172.21.15.151:0/4276382784 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f74680e24e0 con 0x7f7478105670 2024-05-21T21:53:54.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:54.202+0000 7f746fe00700 1 --2- 172.21.15.151:0/4276382784 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f745c0b2900 0x7f745c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:54.203 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:54.203+0000 7f746fe00700 1 --2- 172.21.15.151:0/4276382784 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f745c0b2900 0x7f745c0b4dc0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f74781037e0 tx=0x7f746000b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:53:54.206 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:54.206+0000 7f7475600700 1 -- 172.21.15.151:0/4276382784 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f74680042f0 con 0x7f7478105670 2024-05-21T21:53:54.433 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:54.433+0000 7f747dc00700 1 -- 172.21.15.151:0/4276382784 --> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7478063980 con 0x7f745c0b2900 2024-05-21T21:53:54.447 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:54.447+0000 7f7475600700 1 -- 172.21.15.151:0/4276382784 <== mgr.14152 v2:172.21.15.151:6800/2796622483 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258888 (secure 0 0 0) 0x7f7460067360 con 0x7f745c0b2900 2024-05-21T21:53:54.459 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T21:53:54.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:54.454+0000 7f747dc00700 1 -- 172.21.15.151:0/4276382784 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f745c0b2900 msgr2=0x7f745c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:54.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:54.454+0000 7f747dc00700 1 --2- 172.21.15.151:0/4276382784 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f745c0b2900 0x7f745c0b4dc0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f74781037e0 tx=0x7f746000b260 comp rx=0 tx=0).stop 2024-05-21T21:53:54.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:54.454+0000 7f747dc00700 1 -- 172.21.15.151:0/4276382784 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7478105670 msgr2=0x7f747819bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:54.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:54.454+0000 7f747dc00700 1 --2- 172.21.15.151:0/4276382784 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7478105670 0x7f747819bd50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f746800eb30 tx=0x7f746800ee40 comp rx=0 tx=0).stop 2024-05-21T21:53:54.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:54.454+0000 7f747dc00700 1 -- 172.21.15.151:0/4276382784 shutdown_connections 2024-05-21T21:53:54.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:54.454+0000 7f747dc00700 1 --2- 172.21.15.151:0/4276382784 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f745c0b2900 0x7f745c0b4dc0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:54.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:54.454+0000 7f747dc00700 1 --2- 172.21.15.151:0/4276382784 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f747810b670 0x7f74781a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:54.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:54.454+0000 7f747dc00700 1 --2- 172.21.15.151:0/4276382784 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7478105670 0x7f747819bd50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:54.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:54.454+0000 7f747dc00700 1 --2- 172.21.15.151:0/4276382784 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7478106030 0x7f747819c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:54.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:54.454+0000 7f747dc00700 1 -- 172.21.15.151:0/4276382784 >> 172.21.15.151:0/4276382784 conn(0x7f7478101160 msgr2=0x7f7478101960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:53:54.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:54.454+0000 7f747dc00700 1 -- 172.21.15.151:0/4276382784 shutdown_connections 2024-05-21T21:53:54.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:54.454+0000 7f747dc00700 1 -- 172.21.15.151:0/4276382784 wait complete. 2024-05-21T21:53:54.462 INFO:teuthology.orchestra.run.smithi151.stderr:dumped all 2024-05-21T21:53:55.185 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:53:54 smithi151 ceph-mon[31623]: pgmap v394: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-21T21:53:55.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:53:54 smithi184 ceph-mon[36942]: pgmap v394: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-21T21:53:55.255 INFO:teuthology.orchestra.run.smithi151.stdout:{"pg_ready":true,"pg_map":{"version":394,"stamp":"2024-05-21T21:53:53.254337+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658099,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1443,"num_read_kb":822,"num_write":379,"num_write_kb":2294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":2952069,"num_keys_recovered":17,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":812,"ondisk_log_size":812,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":366044,"kb_used_data":12732,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124575780,"statfs":{"total":1151940427776,"available":1151565598720,"internally_reserved":0,"allocated":13037568,"data_stored":7167501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19140,"internal_metadata":361674044},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":129,"num_read_kb":65,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005370"},"pg_stats":[{"pgid":"3.3f","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.190291+0000","last_change":"2024-05-21T21:51:16.945132+0000","last_active":"2024-05-21T21:53:09.190291+0000","last_peered":"2024-05-21T21:53:09.190291+0000","last_clean":"2024-05-21T21:53:09.190291+0000","last_became_active":"2024-05-21T21:51:16.944766+0000","last_became_peered":"2024-05-21T21:51:16.944766+0000","last_unstale":"2024-05-21T21:53:09.190291+0000","last_undegraded":"2024-05-21T21:53:09.190291+0000","last_fullsized":"2024-05-21T21:53:09.190291+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:54:04.185095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"82'4","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.233872+0000","last_change":"2024-05-21T21:51:16.939291+0000","last_active":"2024-05-21T21:53:09.233872+0000","last_peered":"2024-05-21T21:53:09.233872+0000","last_clean":"2024-05-21T21:53:09.233872+0000","last_became_active":"2024-05-21T21:51:16.938730+0000","last_became_peered":"2024-05-21T21:51:16.938730+0000","last_unstale":"2024-05-21T21:53:09.233872+0000","last_undegraded":"2024-05-21T21:53:09.233872+0000","last_fullsized":"2024-05-21T21:53:09.233872+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:05:30.623744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.232965+0000","last_change":"2024-05-21T21:51:16.941565+0000","last_active":"2024-05-21T21:53:09.232965+0000","last_peered":"2024-05-21T21:53:09.232965+0000","last_clean":"2024-05-21T21:53:09.232965+0000","last_became_active":"2024-05-21T21:51:16.941033+0000","last_became_peered":"2024-05-21T21:51:16.941033+0000","last_unstale":"2024-05-21T21:53:09.232965+0000","last_undegraded":"2024-05-21T21:53:09.232965+0000","last_fullsized":"2024-05-21T21:53:09.232965+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:04:55.789479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.183697+0000","last_change":"2024-05-21T21:51:16.928455+0000","last_active":"2024-05-21T21:53:09.183697+0000","last_peered":"2024-05-21T21:53:09.183697+0000","last_clean":"2024-05-21T21:53:09.183697+0000","last_became_active":"2024-05-21T21:51:16.927874+0000","last_became_peered":"2024-05-21T21:51:16.927874+0000","last_unstale":"2024-05-21T21:53:09.183697+0000","last_undegraded":"2024-05-21T21:53:09.183697+0000","last_fullsized":"2024-05-21T21:53:09.183697+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:25:16.483175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"82'4","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.215489+0000","last_change":"2024-05-21T21:51:16.937773+0000","last_active":"2024-05-21T21:53:09.215489+0000","last_peered":"2024-05-21T21:53:09.215489+0000","last_clean":"2024-05-21T21:53:09.215489+0000","last_became_active":"2024-05-21T21:51:16.937306+0000","last_became_peered":"2024-05-21T21:51:16.937306+0000","last_unstale":"2024-05-21T21:53:09.215489+0000","last_undegraded":"2024-05-21T21:53:09.215489+0000","last_fullsized":"2024-05-21T21:53:09.215489+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:05:03.762390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.197708+0000","last_change":"2024-05-21T21:51:16.948275+0000","last_active":"2024-05-21T21:53:09.197708+0000","last_peered":"2024-05-21T21:53:09.197708+0000","last_clean":"2024-05-21T21:53:09.197708+0000","last_became_active":"2024-05-21T21:51:16.947890+0000","last_became_peered":"2024-05-21T21:51:16.947890+0000","last_unstale":"2024-05-21T21:53:09.197708+0000","last_undegraded":"2024-05-21T21:53:09.197708+0000","last_fullsized":"2024-05-21T21:53:09.197708+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:08:38.572177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"82'5","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.197953+0000","last_change":"2024-05-21T21:51:16.946821+0000","last_active":"2024-05-21T21:53:09.197953+0000","last_peered":"2024-05-21T21:53:09.197953+0000","last_clean":"2024-05-21T21:53:09.197953+0000","last_became_active":"2024-05-21T21:51:16.946515+0000","last_became_peered":"2024-05-21T21:51:16.946515+0000","last_unstale":"2024-05-21T21:53:09.197953+0000","last_undegraded":"2024-05-21T21:53:09.197953+0000","last_fullsized":"2024-05-21T21:53:09.197953+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T09:26:25.867895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.194071+0000","last_change":"2024-05-21T21:51:16.929228+0000","last_active":"2024-05-21T21:53:09.194071+0000","last_peered":"2024-05-21T21:53:09.194071+0000","last_clean":"2024-05-21T21:53:09.194071+0000","last_became_active":"2024-05-21T21:51:16.928875+0000","last_became_peered":"2024-05-21T21:51:16.928875+0000","last_unstale":"2024-05-21T21:53:09.194071+0000","last_undegraded":"2024-05-21T21:53:09.194071+0000","last_fullsized":"2024-05-21T21:53:09.194071+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:50:56.431310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"82'11","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.193550+0000","last_change":"2024-05-21T21:51:16.932360+0000","last_active":"2024-05-21T21:53:09.193550+0000","last_peered":"2024-05-21T21:53:09.193550+0000","last_clean":"2024-05-21T21:53:09.193550+0000","last_became_active":"2024-05-21T21:51:16.932036+0000","last_became_peered":"2024-05-21T21:51:16.932036+0000","last_unstale":"2024-05-21T21:53:09.193550+0000","last_undegraded":"2024-05-21T21:53:09.193550+0000","last_fullsized":"2024-05-21T21:53:09.193550+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T09:06:37.257323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"82'16","reported_seq":43,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.217531+0000","last_change":"2024-05-21T21:52:10.876556+0000","last_active":"2024-05-21T21:53:09.217531+0000","last_peered":"2024-05-21T21:53:09.217531+0000","last_clean":"2024-05-21T21:53:09.217531+0000","last_became_active":"2024-05-21T21:52:10.876273+0000","last_became_peered":"2024-05-21T21:52:10.876273+0000","last_unstale":"2024-05-21T21:53:09.217531+0000","last_undegraded":"2024-05-21T21:53:09.217531+0000","last_fullsized":"2024-05-21T21:53:09.217531+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T05:09:40.037731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.195209+0000","last_change":"2024-05-21T21:51:16.933425+0000","last_active":"2024-05-21T21:53:09.195209+0000","last_peered":"2024-05-21T21:53:09.195209+0000","last_clean":"2024-05-21T21:53:09.195209+0000","last_became_active":"2024-05-21T21:51:16.933057+0000","last_became_peered":"2024-05-21T21:51:16.933057+0000","last_unstale":"2024-05-21T21:53:09.195209+0000","last_undegraded":"2024-05-21T21:53:09.195209+0000","last_fullsized":"2024-05-21T21:53:09.195209+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:04:57.237841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.184643+0000","last_change":"2024-05-21T21:51:16.927014+0000","last_active":"2024-05-21T21:53:09.184643+0000","last_peered":"2024-05-21T21:53:09.184643+0000","last_clean":"2024-05-21T21:53:09.184643+0000","last_became_active":"2024-05-21T21:51:16.925954+0000","last_became_peered":"2024-05-21T21:51:16.925954+0000","last_unstale":"2024-05-21T21:53:09.184643+0000","last_undegraded":"2024-05-21T21:53:09.184643+0000","last_fullsized":"2024-05-21T21:53:09.184643+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:19:20.530727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"82'10","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.228367+0000","last_change":"2024-05-21T21:51:16.930909+0000","last_active":"2024-05-21T21:53:09.228367+0000","last_peered":"2024-05-21T21:53:09.228367+0000","last_clean":"2024-05-21T21:53:09.228367+0000","last_became_active":"2024-05-21T21:51:16.927110+0000","last_became_peered":"2024-05-21T21:51:16.927110+0000","last_unstale":"2024-05-21T21:53:09.228367+0000","last_undegraded":"2024-05-21T21:53:09.228367+0000","last_fullsized":"2024-05-21T21:53:09.228367+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T06:23:55.386047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.232153+0000","last_change":"2024-05-21T21:51:16.938797+0000","last_active":"2024-05-21T21:53:09.232153+0000","last_peered":"2024-05-21T21:53:09.232153+0000","last_clean":"2024-05-21T21:53:09.232153+0000","last_became_active":"2024-05-21T21:51:16.938263+0000","last_became_peered":"2024-05-21T21:51:16.938263+0000","last_unstale":"2024-05-21T21:53:09.232153+0000","last_undegraded":"2024-05-21T21:53:09.232153+0000","last_fullsized":"2024-05-21T21:53:09.232153+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:37:40.521118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.233950+0000","last_change":"2024-05-21T21:51:16.944160+0000","last_active":"2024-05-21T21:53:09.233950+0000","last_peered":"2024-05-21T21:53:09.233950+0000","last_clean":"2024-05-21T21:53:09.233950+0000","last_became_active":"2024-05-21T21:51:16.940046+0000","last_became_peered":"2024-05-21T21:51:16.940046+0000","last_unstale":"2024-05-21T21:53:09.233950+0000","last_undegraded":"2024-05-21T21:53:09.233950+0000","last_fullsized":"2024-05-21T21:53:09.233950+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:14:13.278799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.199500+0000","last_change":"2024-05-21T21:51:16.945599+0000","last_active":"2024-05-21T21:53:09.199500+0000","last_peered":"2024-05-21T21:53:09.199500+0000","last_clean":"2024-05-21T21:53:09.199500+0000","last_became_active":"2024-05-21T21:51:16.945196+0000","last_became_peered":"2024-05-21T21:51:16.945196+0000","last_unstale":"2024-05-21T21:53:09.199500+0000","last_undegraded":"2024-05-21T21:53:09.199500+0000","last_fullsized":"2024-05-21T21:53:09.199500+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:59:24.453615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.199215+0000","last_change":"2024-05-21T21:51:16.944895+0000","last_active":"2024-05-21T21:53:09.199215+0000","last_peered":"2024-05-21T21:53:09.199215+0000","last_clean":"2024-05-21T21:53:09.199215+0000","last_became_active":"2024-05-21T21:51:16.944560+0000","last_became_peered":"2024-05-21T21:51:16.944560+0000","last_unstale":"2024-05-21T21:53:09.199215+0000","last_undegraded":"2024-05-21T21:53:09.199215+0000","last_fullsized":"2024-05-21T21:53:09.199215+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:53:13.264701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"82'10","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.221764+0000","last_change":"2024-05-21T21:51:16.943861+0000","last_active":"2024-05-21T21:53:09.221764+0000","last_peered":"2024-05-21T21:53:09.221764+0000","last_clean":"2024-05-21T21:53:09.221764+0000","last_became_active":"2024-05-21T21:51:16.943123+0000","last_became_peered":"2024-05-21T21:51:16.943123+0000","last_unstale":"2024-05-21T21:53:09.221764+0000","last_undegraded":"2024-05-21T21:53:09.221764+0000","last_fullsized":"2024-05-21T21:53:09.221764+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:35:37.732855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.206209+0000","last_change":"2024-05-21T21:51:16.940580+0000","last_active":"2024-05-21T21:53:09.206209+0000","last_peered":"2024-05-21T21:53:09.206209+0000","last_clean":"2024-05-21T21:53:09.206209+0000","last_became_active":"2024-05-21T21:51:16.940264+0000","last_became_peered":"2024-05-21T21:51:16.940264+0000","last_unstale":"2024-05-21T21:53:09.206209+0000","last_undegraded":"2024-05-21T21:53:09.206209+0000","last_fullsized":"2024-05-21T21:53:09.206209+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:54:22.915541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"80'13","reported_seq":172,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:48.652994+0000","last_change":"2024-05-21T21:51:12.852935+0000","last_active":"2024-05-21T21:53:48.652994+0000","last_peered":"2024-05-21T21:53:48.652994+0000","last_clean":"2024-05-21T21:53:48.652994+0000","last_became_active":"2024-05-21T21:51:12.851802+0000","last_became_peered":"2024-05-21T21:51:12.851802+0000","last_unstale":"2024-05-21T21:53:48.652994+0000","last_undegraded":"2024-05-21T21:53:48.652994+0000","last_fullsized":"2024-05-21T21:53:48.652994+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:13:24.643408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":132,"num_read_kb":132,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"82'5","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.198288+0000","last_change":"2024-05-21T21:51:16.939461+0000","last_active":"2024-05-21T21:53:09.198288+0000","last_peered":"2024-05-21T21:53:09.198288+0000","last_clean":"2024-05-21T21:53:09.198288+0000","last_became_active":"2024-05-21T21:51:16.936322+0000","last_became_peered":"2024-05-21T21:51:16.936322+0000","last_unstale":"2024-05-21T21:53:09.198288+0000","last_undegraded":"2024-05-21T21:53:09.198288+0000","last_fullsized":"2024-05-21T21:53:09.198288+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:37:26.153963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.868750+0000","last_change":"2024-05-21T21:51:12.941838+0000","last_active":"2024-05-21T21:52:10.868750+0000","last_peered":"2024-05-21T21:52:10.868750+0000","last_clean":"2024-05-21T21:52:10.868750+0000","last_became_active":"2024-05-21T21:51:12.940818+0000","last_became_peered":"2024-05-21T21:51:12.940818+0000","last_unstale":"2024-05-21T21:52:10.868750+0000","last_undegraded":"2024-05-21T21:52:10.868750+0000","last_fullsized":"2024-05-21T21:52:10.868750+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:30:42.349241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.238219+0000","last_change":"2024-05-21T21:51:16.950252+0000","last_active":"2024-05-21T21:53:09.238219+0000","last_peered":"2024-05-21T21:53:09.238219+0000","last_clean":"2024-05-21T21:53:09.238219+0000","last_became_active":"2024-05-21T21:51:16.947375+0000","last_became_peered":"2024-05-21T21:51:16.947375+0000","last_unstale":"2024-05-21T21:53:09.238219+0000","last_undegraded":"2024-05-21T21:53:09.238219+0000","last_fullsized":"2024-05-21T21:53:09.238219+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:27:30.462799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"80'12","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.876122+0000","last_change":"2024-05-21T21:52:10.875739+0000","last_active":"2024-05-21T21:52:10.876122+0000","last_peered":"2024-05-21T21:52:10.876122+0000","last_clean":"2024-05-21T21:52:10.876122+0000","last_became_active":"2024-05-21T21:52:10.865846+0000","last_became_peered":"2024-05-21T21:52:10.865846+0000","last_unstale":"2024-05-21T21:52:10.876122+0000","last_undegraded":"2024-05-21T21:52:10.876122+0000","last_fullsized":"2024-05-21T21:52:10.876122+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T06:48:06.465757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.216100+0000","last_change":"2024-05-21T21:51:16.926380+0000","last_active":"2024-05-21T21:53:09.216100+0000","last_peered":"2024-05-21T21:53:09.216100+0000","last_clean":"2024-05-21T21:53:09.216100+0000","last_became_active":"2024-05-21T21:51:16.925821+0000","last_became_peered":"2024-05-21T21:51:16.925821+0000","last_unstale":"2024-05-21T21:53:09.216100+0000","last_undegraded":"2024-05-21T21:53:09.216100+0000","last_fullsized":"2024-05-21T21:53:09.216100+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:42:14.887155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'5","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.864315+0000","last_change":"2024-05-21T21:51:12.854342+0000","last_active":"2024-05-21T21:52:10.864315+0000","last_peered":"2024-05-21T21:52:10.864315+0000","last_clean":"2024-05-21T21:52:10.864315+0000","last_became_active":"2024-05-21T21:51:12.853908+0000","last_became_peered":"2024-05-21T21:51:12.853908+0000","last_unstale":"2024-05-21T21:52:10.864315+0000","last_undegraded":"2024-05-21T21:52:10.864315+0000","last_fullsized":"2024-05-21T21:52:10.864315+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:35:10.374749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"82'15","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.236662+0000","last_change":"2024-05-21T21:52:10.973816+0000","last_active":"2024-05-21T21:53:09.236662+0000","last_peered":"2024-05-21T21:53:09.236662+0000","last_clean":"2024-05-21T21:53:09.236662+0000","last_became_active":"2024-05-21T21:52:10.973457+0000","last_became_peered":"2024-05-21T21:52:10.973457+0000","last_unstale":"2024-05-21T21:53:09.236662+0000","last_undegraded":"2024-05-21T21:53:09.236662+0000","last_fullsized":"2024-05-21T21:53:09.236662+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T05:01:41.915158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"82'7","reported_seq":172,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:51.649543+0000","last_change":"2024-05-21T21:51:12.855658+0000","last_active":"2024-05-21T21:53:51.649543+0000","last_peered":"2024-05-21T21:53:51.649543+0000","last_clean":"2024-05-21T21:53:51.649543+0000","last_became_active":"2024-05-21T21:51:12.855281+0000","last_became_peered":"2024-05-21T21:51:12.855281+0000","last_unstale":"2024-05-21T21:53:51.649543+0000","last_undegraded":"2024-05-21T21:53:51.649543+0000","last_fullsized":"2024-05-21T21:53:51.649543+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:55:08.836045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":138,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.198092+0000","last_change":"2024-05-21T21:51:16.946977+0000","last_active":"2024-05-21T21:53:09.198092+0000","last_peered":"2024-05-21T21:53:09.198092+0000","last_clean":"2024-05-21T21:53:09.198092+0000","last_became_active":"2024-05-21T21:51:16.946562+0000","last_became_peered":"2024-05-21T21:51:16.946562+0000","last_unstale":"2024-05-21T21:53:09.198092+0000","last_undegraded":"2024-05-21T21:53:09.198092+0000","last_fullsized":"2024-05-21T21:53:09.198092+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:43:33.370265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"80'7","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.865598+0000","last_change":"2024-05-21T21:51:12.852574+0000","last_active":"2024-05-21T21:52:10.865598+0000","last_peered":"2024-05-21T21:52:10.865598+0000","last_clean":"2024-05-21T21:52:10.865598+0000","last_became_active":"2024-05-21T21:51:12.852258+0000","last_became_peered":"2024-05-21T21:51:12.852258+0000","last_unstale":"2024-05-21T21:52:10.865598+0000","last_undegraded":"2024-05-21T21:52:10.865598+0000","last_fullsized":"2024-05-21T21:52:10.865598+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:09:40.280821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.192658+0000","last_change":"2024-05-21T21:51:16.934768+0000","last_active":"2024-05-21T21:53:09.192658+0000","last_peered":"2024-05-21T21:53:09.192658+0000","last_clean":"2024-05-21T21:53:09.192658+0000","last_became_active":"2024-05-21T21:51:16.934275+0000","last_became_peered":"2024-05-21T21:51:16.934275+0000","last_unstale":"2024-05-21T21:53:09.192658+0000","last_undegraded":"2024-05-21T21:53:09.192658+0000","last_fullsized":"2024-05-21T21:53:09.192658+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:31:54.425069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"80'3","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.861975+0000","last_change":"2024-05-21T21:51:12.854166+0000","last_active":"2024-05-21T21:52:10.861975+0000","last_peered":"2024-05-21T21:52:10.861975+0000","last_clean":"2024-05-21T21:52:10.861975+0000","last_became_active":"2024-05-21T21:51:12.853508+0000","last_became_peered":"2024-05-21T21:51:12.853508+0000","last_unstale":"2024-05-21T21:52:10.861975+0000","last_undegraded":"2024-05-21T21:52:10.861975+0000","last_fullsized":"2024-05-21T21:52:10.861975+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:29:41.858655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.218781+0000","last_change":"2024-05-21T21:51:16.938293+0000","last_active":"2024-05-21T21:53:09.218781+0000","last_peered":"2024-05-21T21:53:09.218781+0000","last_clean":"2024-05-21T21:53:09.218781+0000","last_became_active":"2024-05-21T21:51:16.937970+0000","last_became_peered":"2024-05-21T21:51:16.937970+0000","last_unstale":"2024-05-21T21:53:09.218781+0000","last_undegraded":"2024-05-21T21:53:09.218781+0000","last_fullsized":"2024-05-21T21:53:09.218781+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:18:27.473098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'3","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.866779+0000","last_change":"2024-05-21T21:51:12.853366+0000","last_active":"2024-05-21T21:52:10.866779+0000","last_peered":"2024-05-21T21:52:10.866779+0000","last_clean":"2024-05-21T21:52:10.866779+0000","last_became_active":"2024-05-21T21:51:12.853028+0000","last_became_peered":"2024-05-21T21:51:12.853028+0000","last_unstale":"2024-05-21T21:52:10.866779+0000","last_undegraded":"2024-05-21T21:52:10.866779+0000","last_fullsized":"2024-05-21T21:52:10.866779+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T09:15:52.593028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"82'11","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.205727+0000","last_change":"2024-05-21T21:51:16.947903+0000","last_active":"2024-05-21T21:53:09.205727+0000","last_peered":"2024-05-21T21:53:09.205727+0000","last_clean":"2024-05-21T21:53:09.205727+0000","last_became_active":"2024-05-21T21:51:16.946677+0000","last_became_peered":"2024-05-21T21:51:16.946677+0000","last_unstale":"2024-05-21T21:53:09.205727+0000","last_undegraded":"2024-05-21T21:53:09.205727+0000","last_fullsized":"2024-05-21T21:53:09.205727+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T06:12:56.514284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'5","reported_seq":296,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:50.576033+0000","last_change":"2024-05-21T21:51:12.854873+0000","last_active":"2024-05-21T21:53:50.576033+0000","last_peered":"2024-05-21T21:53:50.576033+0000","last_clean":"2024-05-21T21:53:50.576033+0000","last_became_active":"2024-05-21T21:51:12.854393+0000","last_became_peered":"2024-05-21T21:51:12.854393+0000","last_unstale":"2024-05-21T21:53:50.576033+0000","last_undegraded":"2024-05-21T21:53:50.576033+0000","last_fullsized":"2024-05-21T21:53:50.576033+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:58:52.411164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":264,"num_read_kb":264,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"82'10","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.219978+0000","last_change":"2024-05-21T21:51:16.951094+0000","last_active":"2024-05-21T21:53:09.219978+0000","last_peered":"2024-05-21T21:53:09.219978+0000","last_clean":"2024-05-21T21:53:09.219978+0000","last_became_active":"2024-05-21T21:51:16.950724+0000","last_became_peered":"2024-05-21T21:51:16.950724+0000","last_unstale":"2024-05-21T21:53:09.219978+0000","last_undegraded":"2024-05-21T21:53:09.219978+0000","last_fullsized":"2024-05-21T21:53:09.219978+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:54:02.835426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'4","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.864180+0000","last_change":"2024-05-21T21:51:12.850427+0000","last_active":"2024-05-21T21:52:10.864180+0000","last_peered":"2024-05-21T21:52:10.864180+0000","last_clean":"2024-05-21T21:52:10.864180+0000","last_became_active":"2024-05-21T21:51:12.850096+0000","last_became_peered":"2024-05-21T21:51:12.850096+0000","last_unstale":"2024-05-21T21:52:10.864180+0000","last_undegraded":"2024-05-21T21:52:10.864180+0000","last_fullsized":"2024-05-21T21:52:10.864180+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:43:22.611043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.238035+0000","last_change":"2024-05-21T21:51:16.942330+0000","last_active":"2024-05-21T21:53:09.238035+0000","last_peered":"2024-05-21T21:53:09.238035+0000","last_clean":"2024-05-21T21:53:09.238035+0000","last_became_active":"2024-05-21T21:51:16.941718+0000","last_became_peered":"2024-05-21T21:51:16.941718+0000","last_unstale":"2024-05-21T21:53:09.238035+0000","last_undegraded":"2024-05-21T21:53:09.238035+0000","last_fullsized":"2024-05-21T21:53:09.238035+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:54:17.176411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'5","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.867630+0000","last_change":"2024-05-21T21:51:12.941100+0000","last_active":"2024-05-21T21:52:10.867630+0000","last_peered":"2024-05-21T21:52:10.867630+0000","last_clean":"2024-05-21T21:52:10.867630+0000","last_became_active":"2024-05-21T21:51:12.940597+0000","last_became_peered":"2024-05-21T21:51:12.940597+0000","last_unstale":"2024-05-21T21:52:10.867630+0000","last_undegraded":"2024-05-21T21:52:10.867630+0000","last_fullsized":"2024-05-21T21:52:10.867630+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:30:58.457108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"82'11","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.193400+0000","last_change":"2024-05-21T21:51:16.921973+0000","last_active":"2024-05-21T21:53:09.193400+0000","last_peered":"2024-05-21T21:53:09.193400+0000","last_clean":"2024-05-21T21:53:09.193400+0000","last_became_active":"2024-05-21T21:51:16.921597+0000","last_became_peered":"2024-05-21T21:51:16.921597+0000","last_unstale":"2024-05-21T21:53:09.193400+0000","last_undegraded":"2024-05-21T21:53:09.193400+0000","last_fullsized":"2024-05-21T21:53:09.193400+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:25:32.395940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'34","reported_seq":335,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:50.576078+0000","last_change":"2024-05-21T21:51:12.850312+0000","last_active":"2024-05-21T21:53:50.576078+0000","last_peered":"2024-05-21T21:53:50.576078+0000","last_clean":"2024-05-21T21:53:50.576078+0000","last_became_active":"2024-05-21T21:51:12.849837+0000","last_became_peered":"2024-05-21T21:51:12.849837+0000","last_unstale":"2024-05-21T21:53:50.576078+0000","last_undegraded":"2024-05-21T21:53:50.576078+0000","last_fullsized":"2024-05-21T21:53:50.576078+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T07:09:13.339554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60011,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":274,"num_read_kb":195,"num_write":36,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.192679+0000","last_change":"2024-05-21T21:51:16.926625+0000","last_active":"2024-05-21T21:53:09.192679+0000","last_peered":"2024-05-21T21:53:09.192679+0000","last_clean":"2024-05-21T21:53:09.192679+0000","last_became_active":"2024-05-21T21:51:16.925889+0000","last_became_peered":"2024-05-21T21:51:16.925889+0000","last_unstale":"2024-05-21T21:53:09.192679+0000","last_undegraded":"2024-05-21T21:53:09.192679+0000","last_fullsized":"2024-05-21T21:53:09.192679+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:35:23.464747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'9","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.863955+0000","last_change":"2024-05-21T21:51:12.859772+0000","last_active":"2024-05-21T21:52:10.863955+0000","last_peered":"2024-05-21T21:52:10.863955+0000","last_clean":"2024-05-21T21:52:10.863955+0000","last_became_active":"2024-05-21T21:51:12.859290+0000","last_became_peered":"2024-05-21T21:51:12.859290+0000","last_unstale":"2024-05-21T21:52:10.863955+0000","last_undegraded":"2024-05-21T21:52:10.863955+0000","last_fullsized":"2024-05-21T21:52:10.863955+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T05:28:29.953381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.229561+0000","last_change":"2024-05-21T21:51:16.925803+0000","last_active":"2024-05-21T21:53:09.229561+0000","last_peered":"2024-05-21T21:53:09.229561+0000","last_clean":"2024-05-21T21:53:09.229561+0000","last_became_active":"2024-05-21T21:51:16.925111+0000","last_became_peered":"2024-05-21T21:51:16.925111+0000","last_unstale":"2024-05-21T21:53:09.229561+0000","last_undegraded":"2024-05-21T21:53:09.229561+0000","last_fullsized":"2024-05-21T21:53:09.229561+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T06:22:11.353379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'5","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.867387+0000","last_change":"2024-05-21T21:51:12.859593+0000","last_active":"2024-05-21T21:52:10.867387+0000","last_peered":"2024-05-21T21:52:10.867387+0000","last_clean":"2024-05-21T21:52:10.867387+0000","last_became_active":"2024-05-21T21:51:12.859197+0000","last_became_peered":"2024-05-21T21:51:12.859197+0000","last_unstale":"2024-05-21T21:52:10.867387+0000","last_undegraded":"2024-05-21T21:52:10.867387+0000","last_fullsized":"2024-05-21T21:52:10.867387+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:13:12.311496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"82'15","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.236923+0000","last_change":"2024-05-21T21:52:10.879028+0000","last_active":"2024-05-21T21:53:09.236923+0000","last_peered":"2024-05-21T21:53:09.236923+0000","last_clean":"2024-05-21T21:53:09.236923+0000","last_became_active":"2024-05-21T21:52:10.878664+0000","last_became_peered":"2024-05-21T21:52:10.878664+0000","last_unstale":"2024-05-21T21:53:09.236923+0000","last_undegraded":"2024-05-21T21:53:09.236923+0000","last_fullsized":"2024-05-21T21:53:09.236923+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:05:53.953546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'7","reported_seq":172,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:51.631695+0000","last_change":"2024-05-21T21:51:12.852169+0000","last_active":"2024-05-21T21:53:51.631695+0000","last_peered":"2024-05-21T21:53:51.631695+0000","last_clean":"2024-05-21T21:53:51.631695+0000","last_became_active":"2024-05-21T21:51:12.851782+0000","last_became_peered":"2024-05-21T21:51:12.851782+0000","last_unstale":"2024-05-21T21:53:51.631695+0000","last_undegraded":"2024-05-21T21:53:51.631695+0000","last_fullsized":"2024-05-21T21:53:51.631695+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:46:09.692158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":139,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.194105+0000","last_change":"2024-05-21T21:51:16.929264+0000","last_active":"2024-05-21T21:53:09.194105+0000","last_peered":"2024-05-21T21:53:09.194105+0000","last_clean":"2024-05-21T21:53:09.194105+0000","last_became_active":"2024-05-21T21:51:16.928900+0000","last_became_peered":"2024-05-21T21:51:16.928900+0000","last_unstale":"2024-05-21T21:53:09.194105+0000","last_undegraded":"2024-05-21T21:53:09.194105+0000","last_fullsized":"2024-05-21T21:53:09.194105+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:42:28.040939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'17","reported_seq":69,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.984839+0000","last_change":"2024-05-21T21:52:10.984451+0000","last_active":"2024-05-21T21:52:10.984839+0000","last_peered":"2024-05-21T21:52:10.984839+0000","last_clean":"2024-05-21T21:52:10.984839+0000","last_became_active":"2024-05-21T21:52:10.974050+0000","last_became_peered":"2024-05-21T21:52:10.974050+0000","last_unstale":"2024-05-21T21:52:10.984839+0000","last_undegraded":"2024-05-21T21:52:10.984839+0000","last_fullsized":"2024-05-21T21:52:10.984839+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T07:08:18.307022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.189550+0000","last_change":"2024-05-21T21:51:16.935464+0000","last_active":"2024-05-21T21:53:09.189550+0000","last_peered":"2024-05-21T21:53:09.189550+0000","last_clean":"2024-05-21T21:53:09.189550+0000","last_became_active":"2024-05-21T21:51:16.935091+0000","last_became_peered":"2024-05-21T21:51:16.935091+0000","last_unstale":"2024-05-21T21:53:09.189550+0000","last_undegraded":"2024-05-21T21:53:09.189550+0000","last_fullsized":"2024-05-21T21:53:09.189550+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T06:59:16.330649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.864584+0000","last_change":"2024-05-21T21:51:12.853327+0000","last_active":"2024-05-21T21:52:10.864584+0000","last_peered":"2024-05-21T21:52:10.864584+0000","last_clean":"2024-05-21T21:52:10.864584+0000","last_became_active":"2024-05-21T21:51:12.852946+0000","last_became_peered":"2024-05-21T21:51:12.852946+0000","last_unstale":"2024-05-21T21:52:10.864584+0000","last_undegraded":"2024-05-21T21:52:10.864584+0000","last_fullsized":"2024-05-21T21:52:10.864584+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:00:34.256858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"82'5","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.184045+0000","last_change":"2024-05-21T21:51:16.929341+0000","last_active":"2024-05-21T21:53:09.184045+0000","last_peered":"2024-05-21T21:53:09.184045+0000","last_clean":"2024-05-21T21:53:09.184045+0000","last_became_active":"2024-05-21T21:51:16.929008+0000","last_became_peered":"2024-05-21T21:51:16.929008+0000","last_unstale":"2024-05-21T21:53:09.184045+0000","last_undegraded":"2024-05-21T21:53:09.184045+0000","last_fullsized":"2024-05-21T21:53:09.184045+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:26:26.297136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"80'1","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.876017+0000","last_change":"2024-05-21T21:51:12.945433+0000","last_active":"2024-05-21T21:52:10.876017+0000","last_peered":"2024-05-21T21:52:10.876017+0000","last_clean":"2024-05-21T21:52:10.876017+0000","last_became_active":"2024-05-21T21:51:12.944463+0000","last_became_peered":"2024-05-21T21:51:12.944463+0000","last_unstale":"2024-05-21T21:52:10.876017+0000","last_undegraded":"2024-05-21T21:52:10.876017+0000","last_fullsized":"2024-05-21T21:52:10.876017+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:29:52.014583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'5","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.215224+0000","last_change":"2024-05-21T21:52:10.883554+0000","last_active":"2024-05-21T21:53:09.215224+0000","last_peered":"2024-05-21T21:53:09.215224+0000","last_clean":"2024-05-21T21:53:09.215224+0000","last_became_active":"2024-05-21T21:52:10.877789+0000","last_became_peered":"2024-05-21T21:52:10.877789+0000","last_unstale":"2024-05-21T21:53:09.215224+0000","last_undegraded":"2024-05-21T21:53:09.215224+0000","last_fullsized":"2024-05-21T21:53:09.215224+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:52:37.623551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"80'5","reported_seq":164,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:48.654021+0000","last_change":"2024-05-21T21:51:12.852883+0000","last_active":"2024-05-21T21:53:48.654021+0000","last_peered":"2024-05-21T21:53:48.654021+0000","last_clean":"2024-05-21T21:53:48.654021+0000","last_became_active":"2024-05-21T21:51:12.852549+0000","last_became_peered":"2024-05-21T21:51:12.852549+0000","last_unstale":"2024-05-21T21:53:48.654021+0000","last_undegraded":"2024-05-21T21:53:48.654021+0000","last_fullsized":"2024-05-21T21:53:48.654021+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:05:53.002286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":132,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.199218+0000","last_change":"2024-05-21T21:51:16.944425+0000","last_active":"2024-05-21T21:53:09.199218+0000","last_peered":"2024-05-21T21:53:09.199218+0000","last_clean":"2024-05-21T21:53:09.199218+0000","last_became_active":"2024-05-21T21:51:16.944033+0000","last_became_peered":"2024-05-21T21:51:16.944033+0000","last_unstale":"2024-05-21T21:53:09.199218+0000","last_undegraded":"2024-05-21T21:53:09.199218+0000","last_fullsized":"2024-05-21T21:53:09.199218+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T07:16:26.802329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.186645+0000","last_change":"2024-05-21T21:51:16.933264+0000","last_active":"2024-05-21T21:53:09.186645+0000","last_peered":"2024-05-21T21:53:09.186645+0000","last_clean":"2024-05-21T21:53:09.186645+0000","last_became_active":"2024-05-21T21:51:16.932911+0000","last_became_peered":"2024-05-21T21:51:16.932911+0000","last_unstale":"2024-05-21T21:53:09.186645+0000","last_undegraded":"2024-05-21T21:53:09.186645+0000","last_fullsized":"2024-05-21T21:53:09.186645+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:25:54.006367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"82'6","reported_seq":165,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:48.624717+0000","last_change":"2024-05-21T21:51:12.942900+0000","last_active":"2024-05-21T21:53:48.624717+0000","last_peered":"2024-05-21T21:53:48.624717+0000","last_clean":"2024-05-21T21:53:48.624717+0000","last_became_active":"2024-05-21T21:51:12.941038+0000","last_became_peered":"2024-05-21T21:51:12.941038+0000","last_unstale":"2024-05-21T21:53:48.624717+0000","last_undegraded":"2024-05-21T21:53:48.624717+0000","last_fullsized":"2024-05-21T21:53:48.624717+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:38:43.240011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":132,"num_read_kb":132,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.186935+0000","last_change":"2024-05-21T21:51:16.933391+0000","last_active":"2024-05-21T21:53:09.186935+0000","last_peered":"2024-05-21T21:53:09.186935+0000","last_clean":"2024-05-21T21:53:09.186935+0000","last_became_active":"2024-05-21T21:51:16.932320+0000","last_became_peered":"2024-05-21T21:51:16.932320+0000","last_unstale":"2024-05-21T21:53:09.186935+0000","last_undegraded":"2024-05-21T21:53:09.186935+0000","last_fullsized":"2024-05-21T21:53:09.186935+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T09:06:59.126748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'6","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.865253+0000","last_change":"2024-05-21T21:51:12.856342+0000","last_active":"2024-05-21T21:52:10.865253+0000","last_peered":"2024-05-21T21:52:10.865253+0000","last_clean":"2024-05-21T21:52:10.865253+0000","last_became_active":"2024-05-21T21:51:12.855952+0000","last_became_peered":"2024-05-21T21:51:12.855952+0000","last_unstale":"2024-05-21T21:52:10.865253+0000","last_undegraded":"2024-05-21T21:52:10.865253+0000","last_fullsized":"2024-05-21T21:52:10.865253+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:34:40.734034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":100,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.863960+0000","last_change":"2024-05-21T21:49:42.797187+0000","last_active":"2024-05-21T21:52:10.863960+0000","last_peered":"2024-05-21T21:52:10.863960+0000","last_clean":"2024-05-21T21:52:10.863960+0000","last_became_active":"2024-05-21T21:49:42.791291+0000","last_became_peered":"2024-05-21T21:49:42.791291+0000","last_unstale":"2024-05-21T21:52:10.863960+0000","last_undegraded":"2024-05-21T21:52:10.863960+0000","last_fullsized":"2024-05-21T21:52:10.863960+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:45:55.552293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:45:55.552293+0000","last_clean_scrub_stamp":"2024-05-21T21:45:55.552293+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T07:24:18.302523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"82'5","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.216448+0000","last_change":"2024-05-21T21:51:16.926446+0000","last_active":"2024-05-21T21:53:09.216448+0000","last_peered":"2024-05-21T21:53:09.216448+0000","last_clean":"2024-05-21T21:53:09.216448+0000","last_became_active":"2024-05-21T21:51:16.926070+0000","last_became_peered":"2024-05-21T21:51:16.926070+0000","last_unstale":"2024-05-21T21:53:09.216448+0000","last_undegraded":"2024-05-21T21:53:09.216448+0000","last_fullsized":"2024-05-21T21:53:09.216448+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:44:27.366393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'4","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.864466+0000","last_change":"2024-05-21T21:51:12.854371+0000","last_active":"2024-05-21T21:52:10.864466+0000","last_peered":"2024-05-21T21:52:10.864466+0000","last_clean":"2024-05-21T21:52:10.864466+0000","last_became_active":"2024-05-21T21:51:12.853847+0000","last_became_peered":"2024-05-21T21:51:12.853847+0000","last_unstale":"2024-05-21T21:52:10.864466+0000","last_undegraded":"2024-05-21T21:52:10.864466+0000","last_fullsized":"2024-05-21T21:52:10.864466+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:14:10.549901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"80'4","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.863735+0000","last_change":"2024-05-21T21:51:12.941227+0000","last_active":"2024-05-21T21:52:10.863735+0000","last_peered":"2024-05-21T21:52:10.863735+0000","last_clean":"2024-05-21T21:52:10.863735+0000","last_became_active":"2024-05-21T21:51:12.940702+0000","last_became_peered":"2024-05-21T21:51:12.940702+0000","last_unstale":"2024-05-21T21:52:10.863735+0000","last_undegraded":"2024-05-21T21:52:10.863735+0000","last_fullsized":"2024-05-21T21:52:10.863735+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:40:34.124921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.205536+0000","last_change":"2024-05-21T21:51:16.945627+0000","last_active":"2024-05-21T21:53:09.205536+0000","last_peered":"2024-05-21T21:53:09.205536+0000","last_clean":"2024-05-21T21:53:09.205536+0000","last_became_active":"2024-05-21T21:51:16.945297+0000","last_became_peered":"2024-05-21T21:51:16.945297+0000","last_unstale":"2024-05-21T21:53:09.205536+0000","last_undegraded":"2024-05-21T21:53:09.205536+0000","last_fullsized":"2024-05-21T21:53:09.205536+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:54:38.898774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'16","reported_seq":59,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.957822+0000","last_change":"2024-05-21T21:52:10.888076+0000","last_active":"2024-05-21T21:53:09.957822+0000","last_peered":"2024-05-21T21:53:09.957822+0000","last_clean":"2024-05-21T21:53:09.957822+0000","last_became_active":"2024-05-21T21:52:10.879368+0000","last_became_peered":"2024-05-21T21:52:10.879368+0000","last_unstale":"2024-05-21T21:53:09.957822+0000","last_undegraded":"2024-05-21T21:53:09.957822+0000","last_fullsized":"2024-05-21T21:53:09.957822+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:46:43.190411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"82'5","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.194548+0000","last_change":"2024-05-21T21:51:16.929807+0000","last_active":"2024-05-21T21:53:09.194548+0000","last_peered":"2024-05-21T21:53:09.194548+0000","last_clean":"2024-05-21T21:53:09.194548+0000","last_became_active":"2024-05-21T21:51:16.928859+0000","last_became_peered":"2024-05-21T21:51:16.928859+0000","last_unstale":"2024-05-21T21:53:09.194548+0000","last_undegraded":"2024-05-21T21:53:09.194548+0000","last_fullsized":"2024-05-21T21:53:09.194548+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:41:32.830892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'7","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.865650+0000","last_change":"2024-05-21T21:51:12.850843+0000","last_active":"2024-05-21T21:52:10.865650+0000","last_peered":"2024-05-21T21:52:10.865650+0000","last_clean":"2024-05-21T21:52:10.865650+0000","last_became_active":"2024-05-21T21:51:12.850516+0000","last_became_peered":"2024-05-21T21:51:12.850516+0000","last_unstale":"2024-05-21T21:52:10.865650+0000","last_undegraded":"2024-05-21T21:52:10.865650+0000","last_fullsized":"2024-05-21T21:52:10.865650+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:37:05.191626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.237987+0000","last_change":"2024-05-21T21:51:16.949017+0000","last_active":"2024-05-21T21:53:09.237987+0000","last_peered":"2024-05-21T21:53:09.237987+0000","last_clean":"2024-05-21T21:53:09.237987+0000","last_became_active":"2024-05-21T21:51:16.948691+0000","last_became_peered":"2024-05-21T21:51:16.948691+0000","last_unstale":"2024-05-21T21:53:09.237987+0000","last_undegraded":"2024-05-21T21:53:09.237987+0000","last_fullsized":"2024-05-21T21:53:09.237987+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:32:06.606581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.973226+0000","last_change":"2024-05-21T21:51:12.939203+0000","last_active":"2024-05-21T21:52:10.973226+0000","last_peered":"2024-05-21T21:52:10.973226+0000","last_clean":"2024-05-21T21:52:10.973226+0000","last_became_active":"2024-05-21T21:51:12.938936+0000","last_became_peered":"2024-05-21T21:51:12.938936+0000","last_unstale":"2024-05-21T21:52:10.973226+0000","last_undegraded":"2024-05-21T21:52:10.973226+0000","last_fullsized":"2024-05-21T21:52:10.973226+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:55:50.336362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"82'2","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.174421+0000","last_change":"2024-05-21T21:51:16.926216+0000","last_active":"2024-05-21T21:53:09.174421+0000","last_peered":"2024-05-21T21:53:09.174421+0000","last_clean":"2024-05-21T21:53:09.174421+0000","last_became_active":"2024-05-21T21:51:16.925834+0000","last_became_peered":"2024-05-21T21:51:16.925834+0000","last_unstale":"2024-05-21T21:53:09.174421+0000","last_undegraded":"2024-05-21T21:53:09.174421+0000","last_fullsized":"2024-05-21T21:53:09.174421+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:13:30.192908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'7","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.866611+0000","last_change":"2024-05-21T21:51:12.855306+0000","last_active":"2024-05-21T21:52:10.866611+0000","last_peered":"2024-05-21T21:52:10.866611+0000","last_clean":"2024-05-21T21:52:10.866611+0000","last_became_active":"2024-05-21T21:51:12.854722+0000","last_became_peered":"2024-05-21T21:51:12.854722+0000","last_unstale":"2024-05-21T21:52:10.866611+0000","last_undegraded":"2024-05-21T21:52:10.866611+0000","last_fullsized":"2024-05-21T21:52:10.866611+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:39:41.583219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.199172+0000","last_change":"2024-05-21T21:51:16.944758+0000","last_active":"2024-05-21T21:53:09.199172+0000","last_peered":"2024-05-21T21:53:09.199172+0000","last_clean":"2024-05-21T21:53:09.199172+0000","last_became_active":"2024-05-21T21:51:16.944393+0000","last_became_peered":"2024-05-21T21:51:16.944393+0000","last_unstale":"2024-05-21T21:53:09.199172+0000","last_undegraded":"2024-05-21T21:53:09.199172+0000","last_fullsized":"2024-05-21T21:53:09.199172+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:01:23.048355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'3","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.864860+0000","last_change":"2024-05-21T21:51:12.860378+0000","last_active":"2024-05-21T21:52:10.864860+0000","last_peered":"2024-05-21T21:52:10.864860+0000","last_clean":"2024-05-21T21:52:10.864860+0000","last_became_active":"2024-05-21T21:51:12.859978+0000","last_became_peered":"2024-05-21T21:51:12.859978+0000","last_unstale":"2024-05-21T21:52:10.864860+0000","last_undegraded":"2024-05-21T21:52:10.864860+0000","last_fullsized":"2024-05-21T21:52:10.864860+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T07:52:26.902232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.229331+0000","last_change":"2024-05-21T21:51:16.943194+0000","last_active":"2024-05-21T21:53:09.229331+0000","last_peered":"2024-05-21T21:53:09.229331+0000","last_clean":"2024-05-21T21:53:09.229331+0000","last_became_active":"2024-05-21T21:51:16.942884+0000","last_became_peered":"2024-05-21T21:51:16.942884+0000","last_unstale":"2024-05-21T21:53:09.229331+0000","last_undegraded":"2024-05-21T21:53:09.229331+0000","last_fullsized":"2024-05-21T21:53:09.229331+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T05:58:17.436995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.863937+0000","last_change":"2024-05-21T21:51:12.853390+0000","last_active":"2024-05-21T21:52:10.863937+0000","last_peered":"2024-05-21T21:52:10.863937+0000","last_clean":"2024-05-21T21:52:10.863937+0000","last_became_active":"2024-05-21T21:51:12.852990+0000","last_became_peered":"2024-05-21T21:51:12.852990+0000","last_unstale":"2024-05-21T21:52:10.863937+0000","last_undegraded":"2024-05-21T21:52:10.863937+0000","last_fullsized":"2024-05-21T21:52:10.863937+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:32:16.392609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"82'5","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.235699+0000","last_change":"2024-05-21T21:51:16.933430+0000","last_active":"2024-05-21T21:53:09.235699+0000","last_peered":"2024-05-21T21:53:09.235699+0000","last_clean":"2024-05-21T21:53:09.235699+0000","last_became_active":"2024-05-21T21:51:16.933053+0000","last_became_peered":"2024-05-21T21:51:16.933053+0000","last_unstale":"2024-05-21T21:53:09.235699+0000","last_undegraded":"2024-05-21T21:53:09.235699+0000","last_fullsized":"2024-05-21T21:53:09.235699+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T06:14:57.468727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'5","reported_seq":165,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:48.573457+0000","last_change":"2024-05-21T21:51:12.946113+0000","last_active":"2024-05-21T21:53:48.573457+0000","last_peered":"2024-05-21T21:53:48.573457+0000","last_clean":"2024-05-21T21:53:48.573457+0000","last_became_active":"2024-05-21T21:51:12.945650+0000","last_became_peered":"2024-05-21T21:51:12.945650+0000","last_unstale":"2024-05-21T21:53:48.573457+0000","last_undegraded":"2024-05-21T21:53:48.573457+0000","last_fullsized":"2024-05-21T21:53:48.573457+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T09:40:18.209400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":133,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.195332+0000","last_change":"2024-05-21T21:51:16.932632+0000","last_active":"2024-05-21T21:53:09.195332+0000","last_peered":"2024-05-21T21:53:09.195332+0000","last_clean":"2024-05-21T21:53:09.195332+0000","last_became_active":"2024-05-21T21:51:16.932256+0000","last_became_peered":"2024-05-21T21:51:16.932256+0000","last_unstale":"2024-05-21T21:53:09.195332+0000","last_undegraded":"2024-05-21T21:53:09.195332+0000","last_fullsized":"2024-05-21T21:53:09.195332+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:04:48.146485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'3","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.867614+0000","last_change":"2024-05-21T21:51:12.860602+0000","last_active":"2024-05-21T21:52:10.867614+0000","last_peered":"2024-05-21T21:52:10.867614+0000","last_clean":"2024-05-21T21:52:10.867614+0000","last_became_active":"2024-05-21T21:51:12.860228+0000","last_became_peered":"2024-05-21T21:51:12.860228+0000","last_unstale":"2024-05-21T21:52:10.867614+0000","last_undegraded":"2024-05-21T21:52:10.867614+0000","last_fullsized":"2024-05-21T21:52:10.867614+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:33:39.230872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.196150+0000","last_change":"2024-05-21T21:51:16.929616+0000","last_active":"2024-05-21T21:53:09.196150+0000","last_peered":"2024-05-21T21:53:09.196150+0000","last_clean":"2024-05-21T21:53:09.196150+0000","last_became_active":"2024-05-21T21:51:16.928569+0000","last_became_peered":"2024-05-21T21:51:16.928569+0000","last_unstale":"2024-05-21T21:53:09.196150+0000","last_undegraded":"2024-05-21T21:53:09.196150+0000","last_fullsized":"2024-05-21T21:53:09.196150+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:15:41.790344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'5","reported_seq":44,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:52:10.881746+0000","last_change":"2024-05-21T21:52:10.881114+0000","last_active":"2024-05-21T21:52:10.881746+0000","last_peered":"2024-05-21T21:52:10.881746+0000","last_clean":"2024-05-21T21:52:10.881746+0000","last_became_active":"2024-05-21T21:52:10.872022+0000","last_became_peered":"2024-05-21T21:52:10.872022+0000","last_unstale":"2024-05-21T21:52:10.881746+0000","last_undegraded":"2024-05-21T21:52:10.881746+0000","last_fullsized":"2024-05-21T21:52:10.881746+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T06:05:51.419156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.193224+0000","last_change":"2024-05-21T21:51:16.926496+0000","last_active":"2024-05-21T21:53:09.193224+0000","last_peered":"2024-05-21T21:53:09.193224+0000","last_clean":"2024-05-21T21:53:09.193224+0000","last_became_active":"2024-05-21T21:51:16.926112+0000","last_became_peered":"2024-05-21T21:51:16.926112+0000","last_unstale":"2024-05-21T21:53:09.193224+0000","last_undegraded":"2024-05-21T21:53:09.193224+0000","last_fullsized":"2024-05-21T21:53:09.193224+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:26:08.254421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.185320+0000","last_change":"2024-05-21T21:51:16.930522+0000","last_active":"2024-05-21T21:53:09.185320+0000","last_peered":"2024-05-21T21:53:09.185320+0000","last_clean":"2024-05-21T21:53:09.185320+0000","last_became_active":"2024-05-21T21:51:16.929853+0000","last_became_peered":"2024-05-21T21:51:16.929853+0000","last_unstale":"2024-05-21T21:53:09.185320+0000","last_undegraded":"2024-05-21T21:53:09.185320+0000","last_fullsized":"2024-05-21T21:53:09.185320+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:10:57.940009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"82'12","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.185369+0000","last_change":"2024-05-21T21:51:16.931751+0000","last_active":"2024-05-21T21:53:09.185369+0000","last_peered":"2024-05-21T21:53:09.185369+0000","last_clean":"2024-05-21T21:53:09.185369+0000","last_became_active":"2024-05-21T21:51:16.931353+0000","last_became_peered":"2024-05-21T21:51:16.931353+0000","last_unstale":"2024-05-21T21:53:09.185369+0000","last_undegraded":"2024-05-21T21:53:09.185369+0000","last_fullsized":"2024-05-21T21:53:09.185369+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:33:45.297850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.206238+0000","last_change":"2024-05-21T21:51:16.940096+0000","last_active":"2024-05-21T21:53:09.206238+0000","last_peered":"2024-05-21T21:53:09.206238+0000","last_clean":"2024-05-21T21:53:09.206238+0000","last_became_active":"2024-05-21T21:51:16.939740+0000","last_became_peered":"2024-05-21T21:51:16.939740+0000","last_unstale":"2024-05-21T21:53:09.206238+0000","last_undegraded":"2024-05-21T21:53:09.206238+0000","last_fullsized":"2024-05-21T21:53:09.206238+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:59:52.569526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"82'7","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.238448+0000","last_change":"2024-05-21T21:51:16.946488+0000","last_active":"2024-05-21T21:53:09.238448+0000","last_peered":"2024-05-21T21:53:09.238448+0000","last_clean":"2024-05-21T21:53:09.238448+0000","last_became_active":"2024-05-21T21:51:16.945957+0000","last_became_peered":"2024-05-21T21:51:16.945957+0000","last_unstale":"2024-05-21T21:53:09.238448+0000","last_undegraded":"2024-05-21T21:53:09.238448+0000","last_fullsized":"2024-05-21T21:53:09.238448+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:40:36.708344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"82'4","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.231934+0000","last_change":"2024-05-21T21:51:16.949349+0000","last_active":"2024-05-21T21:53:09.231934+0000","last_peered":"2024-05-21T21:53:09.231934+0000","last_clean":"2024-05-21T21:53:09.231934+0000","last_became_active":"2024-05-21T21:51:16.946986+0000","last_became_peered":"2024-05-21T21:51:16.946986+0000","last_unstale":"2024-05-21T21:53:09.231934+0000","last_undegraded":"2024-05-21T21:53:09.231934+0000","last_fullsized":"2024-05-21T21:53:09.231934+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:01:14.877110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.181433+0000","last_change":"2024-05-21T21:51:16.929172+0000","last_active":"2024-05-21T21:53:09.181433+0000","last_peered":"2024-05-21T21:53:09.181433+0000","last_clean":"2024-05-21T21:53:09.181433+0000","last_became_active":"2024-05-21T21:51:16.928823+0000","last_became_peered":"2024-05-21T21:51:16.928823+0000","last_unstale":"2024-05-21T21:53:09.181433+0000","last_undegraded":"2024-05-21T21:53:09.181433+0000","last_fullsized":"2024-05-21T21:53:09.181433+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:09:13.719272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"82'15","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.186944+0000","last_change":"2024-05-21T21:51:16.930332+0000","last_active":"2024-05-21T21:53:09.186944+0000","last_peered":"2024-05-21T21:53:09.186944+0000","last_clean":"2024-05-21T21:53:09.186944+0000","last_became_active":"2024-05-21T21:51:16.930024+0000","last_became_peered":"2024-05-21T21:51:16.930024+0000","last_unstale":"2024-05-21T21:53:09.186944+0000","last_undegraded":"2024-05-21T21:53:09.186944+0000","last_fullsized":"2024-05-21T21:53:09.186944+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:47:41.272725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"82'2","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.198573+0000","last_change":"2024-05-21T21:51:16.930102+0000","last_active":"2024-05-21T21:53:09.198573+0000","last_peered":"2024-05-21T21:53:09.198573+0000","last_clean":"2024-05-21T21:53:09.198573+0000","last_became_active":"2024-05-21T21:51:16.929410+0000","last_became_peered":"2024-05-21T21:51:16.929410+0000","last_unstale":"2024-05-21T21:53:09.198573+0000","last_undegraded":"2024-05-21T21:53:09.198573+0000","last_fullsized":"2024-05-21T21:53:09.198573+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:11:46.296288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"82'6","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.199470+0000","last_change":"2024-05-21T21:52:10.873424+0000","last_active":"2024-05-21T21:53:09.199470+0000","last_peered":"2024-05-21T21:53:09.199470+0000","last_clean":"2024-05-21T21:53:09.199470+0000","last_became_active":"2024-05-21T21:52:10.873118+0000","last_became_peered":"2024-05-21T21:52:10.873118+0000","last_unstale":"2024-05-21T21:53:09.199470+0000","last_undegraded":"2024-05-21T21:53:09.199470+0000","last_fullsized":"2024-05-21T21:53:09.199470+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:05:00.731013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,3],"acting":[8,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.29","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.197868+0000","last_change":"2024-05-21T21:51:16.934442+0000","last_active":"2024-05-21T21:53:09.197868+0000","last_peered":"2024-05-21T21:53:09.197868+0000","last_clean":"2024-05-21T21:53:09.197868+0000","last_became_active":"2024-05-21T21:51:16.934086+0000","last_became_peered":"2024-05-21T21:51:16.934086+0000","last_unstale":"2024-05-21T21:53:09.197868+0000","last_undegraded":"2024-05-21T21:53:09.197868+0000","last_fullsized":"2024-05-21T21:53:09.197868+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:36:44.302532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"82'10","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.199678+0000","last_change":"2024-05-21T21:51:16.924479+0000","last_active":"2024-05-21T21:53:09.199678+0000","last_peered":"2024-05-21T21:53:09.199678+0000","last_clean":"2024-05-21T21:53:09.199678+0000","last_became_active":"2024-05-21T21:51:16.923998+0000","last_became_peered":"2024-05-21T21:51:16.923998+0000","last_unstale":"2024-05-21T21:53:09.199678+0000","last_undegraded":"2024-05-21T21:53:09.199678+0000","last_fullsized":"2024-05-21T21:53:09.199678+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:23:07.931731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"82'11","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.235733+0000","last_change":"2024-05-21T21:51:16.947271+0000","last_active":"2024-05-21T21:53:09.235733+0000","last_peered":"2024-05-21T21:53:09.235733+0000","last_clean":"2024-05-21T21:53:09.235733+0000","last_became_active":"2024-05-21T21:51:16.946912+0000","last_became_peered":"2024-05-21T21:51:16.946912+0000","last_unstale":"2024-05-21T21:53:09.235733+0000","last_undegraded":"2024-05-21T21:53:09.235733+0000","last_fullsized":"2024-05-21T21:53:09.235733+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:51:29.646915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"82'8","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-21T21:53:09.184712+0000","last_change":"2024-05-21T21:51:16.928852+0000","last_active":"2024-05-21T21:53:09.184712+0000","last_peered":"2024-05-21T21:53:09.184712+0000","last_clean":"2024-05-21T21:53:09.184712+0000","last_became_active":"2024-05-21T21:51:16.928480+0000","last_became_peered":"2024-05-21T21:51:16.928480+0000","last_unstale":"2024-05-21T21:53:09.184712+0000","last_undegraded":"2024-05-21T21:53:09.184712+0000","last_fullsized":"2024-05-21T21:53:09.184712+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T09:12:30.691254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67497,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1362,"num_read_kb":753,"num_write":254,"num_write_kb":142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":112,"num_keys_recovered":17,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":202491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":218,"ondisk_log_size":218,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645361,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":901120,"data_stored":433959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841589,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28580,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716572,"statfs":{"total":95995035648,"available":95965769728,"internally_reserved":0,"allocated":1601536,"data_stored":1085531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005114,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":929792,"data_stored":434006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201345,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":921600,"data_stored":434267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397573,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28500,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716652,"statfs":{"total":95995035648,"available":95965851648,"internally_reserved":0,"allocated":1519616,"data_stored":1027458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561099,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":909312,"data_stored":433606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574951,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":494642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738477,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36244,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708908,"statfs":{"total":95995035648,"available":95957921792,"internally_reserved":0,"allocated":929792,"data_stored":434945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411427,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36296,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708856,"statfs":{"total":95995035648,"available":95957868544,"internally_reserved":0,"allocated":983040,"data_stored":495170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149788,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36236,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708916,"statfs":{"total":95995035648,"available":95957929984,"internally_reserved":0,"allocated":921600,"data_stored":433314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953559,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":909312,"data_stored":433999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757330,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32672,"kb_used_data":1496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712480,"statfs":{"total":95995035648,"available":95961579520,"internally_reserved":0,"allocated":1531904,"data_stored":1026604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":61875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":62403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":62279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-21T21:53:55.259 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-21T21:53:55.259 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-21T21:53:55.259 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-05-21T21:53:55.259 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph health --format=json 2024-05-21T21:53:55.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:53:54 smithi164 ceph-mon[36269]: pgmap v394: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-21T21:53:55.708 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:53:56.148 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:53:55 smithi151 ceph-mon[31623]: from='client.24847 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T21:53:56.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:53:55 smithi184 ceph-mon[36942]: from='client.24847 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T21:53:56.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:53:55 smithi164 ceph-mon[36269]: from='client.24847 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T21:53:56.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:53:56 smithi151 ceph-mon[31623]: pgmap v395: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-21T21:53:57.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:53:56 smithi184 ceph-mon[36942]: pgmap v395: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-21T21:53:57.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:53:56 smithi164 ceph-mon[36269]: pgmap v395: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-21T21:53:57.916 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:57.915+0000 7f06ab400700 1 -- 172.21.15.151:0/2679237916 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f06a410eeb0 msgr2=0x7f06a4116a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:57.916 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:57.915+0000 7f06ab400700 1 --2- 172.21.15.151:0/2679237916 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f06a410eeb0 0x7f06a4116a80 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f069c00b0d0 tx=0x7f069c00b3e0 comp rx=0 tx=0).stop 2024-05-21T21:53:57.916 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:57.915+0000 7f06ab400700 1 -- 172.21.15.151:0/2679237916 shutdown_connections 2024-05-21T21:53:57.917 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:57.915+0000 7f06ab400700 1 --2- 172.21.15.151:0/2679237916 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f06a4109dd0 0x7f06a410a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:57.917 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:57.915+0000 7f06ab400700 1 --2- 172.21.15.151:0/2679237916 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f06a410eeb0 0x7f06a4116a80 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:57.917 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:57.915+0000 7f06ab400700 1 --2- 172.21.15.151:0/2679237916 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f06a410a780 0x7f06a410e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:57.917 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:57.915+0000 7f06ab400700 1 -- 172.21.15.151:0/2679237916 >> 172.21.15.151:0/2679237916 conn(0x7f06a4076d10 msgr2=0x7f06a4108360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:53:57.917 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:57.915+0000 7f06ab400700 1 -- 172.21.15.151:0/2679237916 shutdown_connections 2024-05-21T21:53:57.918 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:57.915+0000 7f06ab400700 1 -- 172.21.15.151:0/2679237916 wait complete. 2024-05-21T21:53:57.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:57.916+0000 7f06ab400700 1 Processor -- start 2024-05-21T21:53:57.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:57.917+0000 7f06ab400700 1 -- start start 2024-05-21T21:53:57.922 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:57.917+0000 7f06ab400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f06a4109dd0 0x7f06a41a0200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:57.922 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:57.917+0000 7f06ab400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f06a410a780 0x7f06a41a0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:57.922 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:57.917+0000 7f06ab400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f06a410eeb0 0x7f06a41a4a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:57.922 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:57.917+0000 7f06ab400700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f06a4119d90 con 0x7f06a410eeb0 2024-05-21T21:53:57.922 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:57.917+0000 7f06ab400700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f06a4119c10 con 0x7f06a410a780 2024-05-21T21:53:57.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:57.917+0000 7f06ab400700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f06a4119f10 con 0x7f06a4109dd0 2024-05-21T21:53:57.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:57.917+0000 7f06a8a00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f06a4109dd0 0x7f06a41a0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:57.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:57.918+0000 7f06a8a00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f06a4109dd0 0x7f06a41a0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:38852/0 (socket says 172.21.15.151:38852) 2024-05-21T21:53:57.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:57.918+0000 7f06a8a00700 1 -- 172.21.15.151:0/599052989 learned_addr learned my addr 172.21.15.151:0/599052989 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:53:57.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:57.918+0000 7f06a3e00700 1 --2- 172.21.15.151:0/599052989 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f06a410a780 0x7f06a41a0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:57.924 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:57.918+0000 7f06a9400700 1 --2- 172.21.15.151:0/599052989 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f06a410eeb0 0x7f06a41a4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:57.924 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:57.918+0000 7f06a8a00700 1 -- 172.21.15.151:0/599052989 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f06a410a780 msgr2=0x7f06a41a0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:57.924 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:57.918+0000 7f06a8a00700 1 --2- 172.21.15.151:0/599052989 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f06a410a780 0x7f06a41a0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:57.924 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:57.918+0000 7f06a8a00700 1 -- 172.21.15.151:0/599052989 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f06a410eeb0 msgr2=0x7f06a41a4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:57.924 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:57.918+0000 7f06a8a00700 1 --2- 172.21.15.151:0/599052989 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f06a410eeb0 0x7f06a41a4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:57.925 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:57.918+0000 7f06a8a00700 1 -- 172.21.15.151:0/599052989 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06a41a51c0 con 0x7f06a4109dd0 2024-05-21T21:53:57.925 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:57.918+0000 7f06a3e00700 1 --2- 172.21.15.151:0/599052989 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f06a410a780 0x7f06a41a0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:53:57.925 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:57.918+0000 7f06a9400700 1 --2- 172.21.15.151:0/599052989 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f06a410eeb0 0x7f06a41a4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:53:57.925 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:57.918+0000 7f06a8a00700 1 --2- 172.21.15.151:0/599052989 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f06a4109dd0 0x7f06a41a0200 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f069800e9a0 tx=0x7f069800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:53:57.925 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:57.919+0000 7f06a1600700 1 -- 172.21.15.151:0/599052989 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f069800ca10 con 0x7f06a4109dd0 2024-05-21T21:53:57.925 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:57.919+0000 7f06a1600700 1 -- 172.21.15.151:0/599052989 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f069800cbb0 con 0x7f06a4109dd0 2024-05-21T21:53:57.926 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:57.919+0000 7f06ab400700 1 -- 172.21.15.151:0/599052989 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06a41a54b0 con 0x7f06a4109dd0 2024-05-21T21:53:57.926 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:57.919+0000 7f06a1600700 1 -- 172.21.15.151:0/599052989 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0698018680 con 0x7f06a4109dd0 2024-05-21T21:53:57.926 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:57.919+0000 7f06ab400700 1 -- 172.21.15.151:0/599052989 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06a41acd30 con 0x7f06a4109dd0 2024-05-21T21:53:57.926 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:57.920+0000 7f06ab400700 1 -- 172.21.15.151:0/599052989 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06a406a230 con 0x7f06a4109dd0 2024-05-21T21:53:57.928 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:57.922+0000 7f06a1600700 1 -- 172.21.15.151:0/599052989 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f0698010430 con 0x7f06a4109dd0 2024-05-21T21:53:57.928 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:57.923+0000 7f06a1600700 1 --2- 172.21.15.151:0/599052989 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f068c0b2880 0x7f068c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:53:57.928 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:57.924+0000 7f06a1600700 1 -- 172.21.15.151:0/599052989 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f06980e23a0 con 0x7f06a4109dd0 2024-05-21T21:53:57.928 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:57.924+0000 7f06a3e00700 1 --2- 172.21.15.151:0/599052989 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f068c0b2880 0x7f068c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:53:57.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:57.924+0000 7f06a3e00700 1 --2- 172.21.15.151:0/599052989 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f068c0b2880 0x7f068c0b4d40 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0690009680 tx=0x7f0690009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:53:57.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:57.928+0000 7f06a1600700 1 -- 172.21.15.151:0/599052989 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f06980aa080 con 0x7f06a4109dd0 2024-05-21T21:53:58.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:58.211+0000 7f06ab400700 1 -- 172.21.15.151:0/599052989 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f06a402e560 con 0x7f06a4109dd0 2024-05-21T21:53:58.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:58.211+0000 7f06a1600700 1 -- 172.21.15.151:0/599052989 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f06980a9980 con 0x7f06a4109dd0 2024-05-21T21:53:58.212 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T21:53:58.212 INFO:teuthology.orchestra.run.smithi151.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-21T21:53:58.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:58.214+0000 7f06ab400700 1 -- 172.21.15.151:0/599052989 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f068c0b2880 msgr2=0x7f068c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:58.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:58.214+0000 7f06ab400700 1 --2- 172.21.15.151:0/599052989 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f068c0b2880 0x7f068c0b4d40 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0690009680 tx=0x7f0690009260 comp rx=0 tx=0).stop 2024-05-21T21:53:58.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:58.214+0000 7f06ab400700 1 -- 172.21.15.151:0/599052989 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f06a4109dd0 msgr2=0x7f06a41a0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:53:58.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:58.214+0000 7f06ab400700 1 --2- 172.21.15.151:0/599052989 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f06a4109dd0 0x7f06a41a0200 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f069800e9a0 tx=0x7f069800ecb0 comp rx=0 tx=0).stop 2024-05-21T21:53:58.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:58.214+0000 7f06ab400700 1 -- 172.21.15.151:0/599052989 shutdown_connections 2024-05-21T21:53:58.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:58.214+0000 7f06ab400700 1 --2- 172.21.15.151:0/599052989 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f068c0b2880 0x7f068c0b4d40 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:58.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:58.214+0000 7f06ab400700 1 --2- 172.21.15.151:0/599052989 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f06a410eeb0 0x7f06a41a4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:58.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:58.214+0000 7f06ab400700 1 --2- 172.21.15.151:0/599052989 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f06a4109dd0 0x7f06a41a0200 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:58.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:58.214+0000 7f06ab400700 1 --2- 172.21.15.151:0/599052989 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f06a410a780 0x7f06a41a0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:53:58.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:58.214+0000 7f06ab400700 1 -- 172.21.15.151:0/599052989 >> 172.21.15.151:0/599052989 conn(0x7f06a4076d10 msgr2=0x7f06a40772d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:53:58.216 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:58.215+0000 7f06ab400700 1 -- 172.21.15.151:0/599052989 shutdown_connections 2024-05-21T21:53:58.216 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:53:58.215+0000 7f06ab400700 1 -- 172.21.15.151:0/599052989 wait complete. 2024-05-21T21:53:59.068 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-05-21T21:53:59.069 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-21T21:53:59.152 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:53:58 smithi151 ceph-mon[31623]: pgmap v396: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-21T21:53:59.152 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:53:58 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/599052989' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-21T21:53:59.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:53:58 smithi184 ceph-mon[36942]: pgmap v396: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-21T21:53:59.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:53:58 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/599052989' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-21T21:53:59.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:53:58 smithi164 ceph-mon[36269]: pgmap v396: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-21T21:53:59.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:53:58 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/599052989' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-21T21:53:59.521 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:54:01.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:54:00 smithi151 ceph-mon[31623]: pgmap v397: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-21T21:54:01.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:54:00 smithi184 ceph-mon[36942]: pgmap v397: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-21T21:54:01.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:54:00 smithi164 ceph-mon[36269]: pgmap v397: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-21T21:54:01.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:01.791+0000 7f6c77000700 1 -- 172.21.15.151:0/1901733908 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6c7010eeb0 msgr2=0x7f6c70116a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:01.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:01.791+0000 7f6c77000700 1 --2- 172.21.15.151:0/1901733908 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6c7010eeb0 0x7f6c70116a80 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6c6800b0d0 tx=0x7f6c6800b3e0 comp rx=0 tx=0).stop 2024-05-21T21:54:01.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:01.791+0000 7f6c77000700 1 -- 172.21.15.151:0/1901733908 shutdown_connections 2024-05-21T21:54:01.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:01.791+0000 7f6c77000700 1 --2- 172.21.15.151:0/1901733908 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c70109dd0 0x7f6c7010a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:01.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:01.791+0000 7f6c77000700 1 --2- 172.21.15.151:0/1901733908 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6c7010eeb0 0x7f6c70116a80 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:01.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:01.791+0000 7f6c77000700 1 --2- 172.21.15.151:0/1901733908 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6c7010a780 0x7f6c7010e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:01.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:01.791+0000 7f6c77000700 1 -- 172.21.15.151:0/1901733908 >> 172.21.15.151:0/1901733908 conn(0x7f6c70076d10 msgr2=0x7f6c70108360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:54:01.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:01.791+0000 7f6c77000700 1 -- 172.21.15.151:0/1901733908 shutdown_connections 2024-05-21T21:54:01.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:01.791+0000 7f6c77000700 1 -- 172.21.15.151:0/1901733908 wait complete. 2024-05-21T21:54:01.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:01.792+0000 7f6c77000700 1 Processor -- start 2024-05-21T21:54:01.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:01.792+0000 7f6c77000700 1 -- start start 2024-05-21T21:54:01.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:01.792+0000 7f6c77000700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6c70109dd0 0x7f6c701a0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:01.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:01.793+0000 7f6c77000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c7010a780 0x7f6c701a07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:01.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:01.793+0000 7f6c77000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6c7010eeb0 0x7f6c701a4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:01.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:01.793+0000 7f6c77000700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6c70119db0 con 0x7f6c7010eeb0 2024-05-21T21:54:01.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:01.793+0000 7f6c77000700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6c70119c30 con 0x7f6c7010a780 2024-05-21T21:54:01.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:01.793+0000 7f6c77000700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6c70119f30 con 0x7f6c70109dd0 2024-05-21T21:54:01.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:01.793+0000 7f6c6fe00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6c70109dd0 0x7f6c701a0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:01.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:01.793+0000 7f6c6f400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c7010a780 0x7f6c701a07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:01.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:01.793+0000 7f6c75000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6c7010eeb0 0x7f6c701a4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:01.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:01.793+0000 7f6c6fe00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6c70109dd0 0x7f6c701a0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:38880/0 (socket says 172.21.15.151:38880) 2024-05-21T21:54:01.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:01.793+0000 7f6c6fe00700 1 -- 172.21.15.151:0/4282774799 learned_addr learned my addr 172.21.15.151:0/4282774799 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:54:01.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:01.793+0000 7f6c6f400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c7010a780 0x7f6c701a07c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:60928/0 (socket says 172.21.15.151:60928) 2024-05-21T21:54:01.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:01.793+0000 7f6c6f400700 1 -- 172.21.15.151:0/4282774799 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6c70109dd0 msgr2=0x7f6c701a0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:01.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:01.793+0000 7f6c6f400700 1 --2- 172.21.15.151:0/4282774799 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6c70109dd0 0x7f6c701a0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:01.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:01.793+0000 7f6c6f400700 1 -- 172.21.15.151:0/4282774799 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6c7010eeb0 msgr2=0x7f6c701a4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:01.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:01.793+0000 7f6c6f400700 1 --2- 172.21.15.151:0/4282774799 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6c7010eeb0 0x7f6c701a4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:01.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:01.793+0000 7f6c6f400700 1 -- 172.21.15.151:0/4282774799 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c701a5240 con 0x7f6c7010a780 2024-05-21T21:54:01.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:01.794+0000 7f6c6f400700 1 --2- 172.21.15.151:0/4282774799 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c7010a780 0x7f6c701a07c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c00e9c0 tx=0x7f6c5c00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:54:01.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:01.794+0000 7f6c6cc00700 1 -- 172.21.15.151:0/4282774799 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c5c004220 con 0x7f6c7010a780 2024-05-21T21:54:01.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:01.794+0000 7f6c6cc00700 1 -- 172.21.15.151:0/4282774799 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c5c0043c0 con 0x7f6c7010a780 2024-05-21T21:54:01.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:01.794+0000 7f6c77000700 1 -- 172.21.15.151:0/4282774799 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c701a5530 con 0x7f6c7010a780 2024-05-21T21:54:01.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:01.794+0000 7f6c6cc00700 1 -- 172.21.15.151:0/4282774799 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c5c005020 con 0x7f6c7010a780 2024-05-21T21:54:01.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:01.794+0000 7f6c77000700 1 -- 172.21.15.151:0/4282774799 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c701acdb0 con 0x7f6c7010a780 2024-05-21T21:54:01.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:01.795+0000 7f6c77000700 1 -- 172.21.15.151:0/4282774799 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c7006a230 con 0x7f6c7010a780 2024-05-21T21:54:01.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:01.798+0000 7f6c6cc00700 1 -- 172.21.15.151:0/4282774799 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f6c5c004750 con 0x7f6c7010a780 2024-05-21T21:54:01.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:01.800+0000 7f6c6cc00700 1 --2- 172.21.15.151:0/4282774799 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f6c540b2880 0x7f6c540b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:01.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:01.800+0000 7f6c6cc00700 1 -- 172.21.15.151:0/4282774799 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f6c5c0e2120 con 0x7f6c7010a780 2024-05-21T21:54:01.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:01.800+0000 7f6c6fe00700 1 --2- 172.21.15.151:0/4282774799 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f6c540b2880 0x7f6c540b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:01.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:01.801+0000 7f6c6fe00700 1 --2- 172.21.15.151:0/4282774799 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f6c540b2880 0x7f6c540b4d40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6c60009d20 tx=0x7f6c60009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:54:01.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:01.802+0000 7f6c6cc00700 1 -- 172.21.15.151:0/4282774799 <== mon.2 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6c5c0a9e00 con 0x7f6c7010a780 2024-05-21T21:54:02.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:02.080+0000 7f6c77000700 1 -- 172.21.15.151:0/4282774799 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6c7002e560 con 0x7f6c7010a780 2024-05-21T21:54:02.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:02.082+0000 7f6c6cc00700 1 -- 172.21.15.151:0/4282774799 <== mon.2 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11958 (secure 0 0 0) 0x7f6c5c0a3e20 con 0x7f6c7010a780 2024-05-21T21:54:02.083 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T21:54:02.083 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14358,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/158010529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":158010529},{"type":"v1","addr":"172.21.15.151:6835","nonce":158010529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6839/2069058115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":2069058115},{"type":"v1","addr":"172.21.15.151:6839","nonce":2069058115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T21:51:19.552069+0000","modified":"2024-05-21T21:51:43.002737+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24530,"mds_1":24493,"mds_2":24491,"mds_3":24478,"mds_4":24473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14403":{"gid":14403,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6841/1749723794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6840","nonce":1749723794},{"type":"v1","addr":"172.21.15.151:6841","nonce":1749723794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/2026266114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":2026266114},{"type":"v1","addr":"172.21.15.184:6833","nonce":2026266114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/2467900844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":2467900844},{"type":"v1","addr":"172.21.15.164:6833","nonce":2467900844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3367640960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3367640960},{"type":"v1","addr":"172.21.15.151:6837","nonce":3367640960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/3264142621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3264142621},{"type":"v1","addr":"172.21.15.184:6835","nonce":3264142621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":17,"addr":"172.21.15.164:6835/2340428470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2340428470},{"type":"v1","addr":"172.21.15.164:6835","nonce":2340428470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":16,"addr":"172.21.15.184:6837/2988194888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":2988194888},{"type":"v1","addr":"172.21.15.184:6837","nonce":2988194888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.164:6837/1554609572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1554609572},{"type":"v1","addr":"172.21.15.164:6837","nonce":1554609572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.184:6839/3059935143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":3059935143},{"type":"v1","addr":"172.21.15.184:6839","nonce":3059935143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.164:6839/2646401247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":2646401247},{"type":"v1","addr":"172.21.15.164:6839","nonce":2646401247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24530,"qdb_cluster":[24473,24478,24491,24493,24530]},"id":1}]} 2024-05-21T21:54:02.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:02.085+0000 7f6c77000700 1 -- 172.21.15.151:0/4282774799 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f6c540b2880 msgr2=0x7f6c540b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:02.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:02.085+0000 7f6c77000700 1 --2- 172.21.15.151:0/4282774799 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f6c540b2880 0x7f6c540b4d40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6c60009d20 tx=0x7f6c60009260 comp rx=0 tx=0).stop 2024-05-21T21:54:02.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:02.085+0000 7f6c77000700 1 -- 172.21.15.151:0/4282774799 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c7010a780 msgr2=0x7f6c701a07c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:02.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:02.085+0000 7f6c77000700 1 --2- 172.21.15.151:0/4282774799 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c7010a780 0x7f6c701a07c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c00e9c0 tx=0x7f6c5c00ecd0 comp rx=0 tx=0).stop 2024-05-21T21:54:02.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:02.085+0000 7f6c77000700 1 -- 172.21.15.151:0/4282774799 shutdown_connections 2024-05-21T21:54:02.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:02.085+0000 7f6c77000700 1 --2- 172.21.15.151:0/4282774799 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f6c540b2880 0x7f6c540b4d40 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:02.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:02.085+0000 7f6c77000700 1 --2- 172.21.15.151:0/4282774799 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6c7010eeb0 0x7f6c701a4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:02.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:02.085+0000 7f6c77000700 1 --2- 172.21.15.151:0/4282774799 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6c70109dd0 0x7f6c701a0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:02.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:02.086+0000 7f6c77000700 1 --2- 172.21.15.151:0/4282774799 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c7010a780 0x7f6c701a07c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:02.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:02.086+0000 7f6c77000700 1 -- 172.21.15.151:0/4282774799 >> 172.21.15.151:0/4282774799 conn(0x7f6c70076d10 msgr2=0x7f6c700772d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:54:02.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:02.086+0000 7f6c77000700 1 -- 172.21.15.151:0/4282774799 shutdown_connections 2024-05-21T21:54:02.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:02.086+0000 7f6c77000700 1 -- 172.21.15.151:0/4282774799 wait complete. 2024-05-21T21:54:02.087 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 28 2024-05-21T21:54:03.028 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-21T21:54:03.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:54:02 smithi151 ceph-mon[31623]: pgmap v398: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-21T21:54:03.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:54:02 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/4282774799' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T21:54:03.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:54:02 smithi184 ceph-mon[36942]: pgmap v398: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-21T21:54:03.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:54:02 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/4282774799' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T21:54:03.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:54:02 smithi164 ceph-mon[36269]: pgmap v398: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-21T21:54:03.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:54:02 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/4282774799' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T21:54:03.472 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:54:05.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:54:04 smithi151 ceph-mon[31623]: pgmap v399: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-21T21:54:05.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:54:04 smithi184 ceph-mon[36942]: pgmap v399: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-21T21:54:05.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:54:04 smithi164 ceph-mon[36269]: pgmap v399: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-21T21:54:05.762 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:05.762+0000 7fb764000700 1 -- 172.21.15.151:0/3093706647 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb75c10fff0 msgr2=0x7fb75c1123e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:05.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:05.762+0000 7fb764000700 1 --2- 172.21.15.151:0/3093706647 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb75c10fff0 0x7fb75c1123e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb75400b0a0 tx=0x7fb75400b3b0 comp rx=0 tx=0).stop 2024-05-21T21:54:05.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:05.762+0000 7fb764000700 1 -- 172.21.15.151:0/3093706647 shutdown_connections 2024-05-21T21:54:05.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:05.762+0000 7fb764000700 1 --2- 172.21.15.151:0/3093706647 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb75c06b250 0x7fb75c06b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:05.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:05.762+0000 7fb764000700 1 --2- 172.21.15.151:0/3093706647 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb75c10fff0 0x7fb75c1123e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:05.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:05.762+0000 7fb764000700 1 --2- 172.21.15.151:0/3093706647 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb75c06bb70 0x7fb75c10fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:05.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:05.762+0000 7fb764000700 1 -- 172.21.15.151:0/3093706647 >> 172.21.15.151:0/3093706647 conn(0x7fb75c0793f0 msgr2=0x7fb75c079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:54:05.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:05.762+0000 7fb764000700 1 -- 172.21.15.151:0/3093706647 shutdown_connections 2024-05-21T21:54:05.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:05.762+0000 7fb764000700 1 -- 172.21.15.151:0/3093706647 wait complete. 2024-05-21T21:54:05.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:05.763+0000 7fb764000700 1 Processor -- start 2024-05-21T21:54:05.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:05.763+0000 7fb764000700 1 -- start start 2024-05-21T21:54:05.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:05.764+0000 7fb764000700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb75c06b250 0x7fb75c19bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:05.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:05.764+0000 7fb764000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb75c06bb70 0x7fb75c19c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:05.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:05.764+0000 7fb764000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb75c10fff0 0x7fb75c1a07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:05.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:05.764+0000 7fb764000700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb75c115960 con 0x7fb75c10fff0 2024-05-21T21:54:05.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:05.764+0000 7fb764000700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb75c1157e0 con 0x7fb75c06bb70 2024-05-21T21:54:05.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:05.764+0000 7fb764000700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb75c115ae0 con 0x7fb75c06b250 2024-05-21T21:54:05.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:05.764+0000 7fb761600700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb75c06b250 0x7fb75c19bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:05.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:05.764+0000 7fb760c00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb75c06bb70 0x7fb75c19c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:05.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:05.764+0000 7fb761600700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb75c06b250 0x7fb75c19bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:38894/0 (socket says 172.21.15.151:38894) 2024-05-21T21:54:05.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:05.764+0000 7fb761600700 1 -- 172.21.15.151:0/3112285974 learned_addr learned my addr 172.21.15.151:0/3112285974 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:54:05.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:05.764+0000 7fb762000700 1 --2- 172.21.15.151:0/3112285974 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb75c10fff0 0x7fb75c1a07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:05.766 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:05.764+0000 7fb760c00700 1 -- 172.21.15.151:0/3112285974 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb75c06b250 msgr2=0x7fb75c19bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:05.766 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:05.764+0000 7fb760c00700 1 --2- 172.21.15.151:0/3112285974 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb75c06b250 0x7fb75c19bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:05.766 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:05.764+0000 7fb760c00700 1 -- 172.21.15.151:0/3112285974 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb75c10fff0 msgr2=0x7fb75c1a07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:05.766 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:05.764+0000 7fb760c00700 1 --2- 172.21.15.151:0/3112285974 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb75c10fff0 0x7fb75c1a07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:05.766 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:05.764+0000 7fb760c00700 1 -- 172.21.15.151:0/3112285974 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb75c1a0ee0 con 0x7fb75c06bb70 2024-05-21T21:54:05.766 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:05.765+0000 7fb760c00700 1 --2- 172.21.15.151:0/3112285974 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb75c06bb70 0x7fb75c19c410 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb74800e9f0 tx=0x7fb74800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:54:05.766 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:05.765+0000 7fb75a000700 1 -- 172.21.15.151:0/3112285974 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb74800c9c0 con 0x7fb75c06bb70 2024-05-21T21:54:05.766 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:05.765+0000 7fb75a000700 1 -- 172.21.15.151:0/3112285974 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb74800cb60 con 0x7fb75c06bb70 2024-05-21T21:54:05.767 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:05.765+0000 7fb764000700 1 -- 172.21.15.151:0/3112285974 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb75c1a11d0 con 0x7fb75c06bb70 2024-05-21T21:54:05.767 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:05.765+0000 7fb75a000700 1 -- 172.21.15.151:0/3112285974 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb748018680 con 0x7fb75c06bb70 2024-05-21T21:54:05.767 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:05.765+0000 7fb764000700 1 -- 172.21.15.151:0/3112285974 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb75c1a8a50 con 0x7fb75c06bb70 2024-05-21T21:54:05.767 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:05.766+0000 7fb764000700 1 -- 172.21.15.151:0/3112285974 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb75c06a230 con 0x7fb75c06bb70 2024-05-21T21:54:05.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:05.769+0000 7fb75a000700 1 -- 172.21.15.151:0/3112285974 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fb748010430 con 0x7fb75c06bb70 2024-05-21T21:54:05.770 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:05.770+0000 7fb75a000700 1 --2- 172.21.15.151:0/3112285974 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fb7440b2880 0x7fb7440b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:05.770 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:05.770+0000 7fb75a000700 1 -- 172.21.15.151:0/3112285974 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fb7480e26b0 con 0x7fb75c06bb70 2024-05-21T21:54:05.770 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:05.770+0000 7fb761600700 1 --2- 172.21.15.151:0/3112285974 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fb7440b2880 0x7fb7440b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:05.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:05.771+0000 7fb761600700 1 --2- 172.21.15.151:0/3112285974 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fb7440b2880 0x7fb7440b4d40 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb750009d20 tx=0x7fb750009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:54:05.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:05.772+0000 7fb75a000700 1 -- 172.21.15.151:0/3112285974 <== mon.2 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb7480aa390 con 0x7fb75c06bb70 2024-05-21T21:54:06.056 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:06.055+0000 7fb764000700 1 -- 172.21.15.151:0/3112285974 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb75c02e560 con 0x7fb75c06bb70 2024-05-21T21:54:06.057 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:06.056+0000 7fb75a000700 1 -- 172.21.15.151:0/3112285974 <== mon.2 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11958 (secure 0 0 0) 0x7fb7480b2410 con 0x7fb75c06bb70 2024-05-21T21:54:06.058 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T21:54:06.059 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14358,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/158010529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":158010529},{"type":"v1","addr":"172.21.15.151:6835","nonce":158010529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6839/2069058115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":2069058115},{"type":"v1","addr":"172.21.15.151:6839","nonce":2069058115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T21:51:19.552069+0000","modified":"2024-05-21T21:51:43.002737+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24530,"mds_1":24493,"mds_2":24491,"mds_3":24478,"mds_4":24473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14403":{"gid":14403,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6841/1749723794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6840","nonce":1749723794},{"type":"v1","addr":"172.21.15.151:6841","nonce":1749723794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/2026266114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":2026266114},{"type":"v1","addr":"172.21.15.184:6833","nonce":2026266114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/2467900844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":2467900844},{"type":"v1","addr":"172.21.15.164:6833","nonce":2467900844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3367640960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3367640960},{"type":"v1","addr":"172.21.15.151:6837","nonce":3367640960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/3264142621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3264142621},{"type":"v1","addr":"172.21.15.184:6835","nonce":3264142621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":17,"addr":"172.21.15.164:6835/2340428470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2340428470},{"type":"v1","addr":"172.21.15.164:6835","nonce":2340428470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":16,"addr":"172.21.15.184:6837/2988194888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":2988194888},{"type":"v1","addr":"172.21.15.184:6837","nonce":2988194888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.164:6837/1554609572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1554609572},{"type":"v1","addr":"172.21.15.164:6837","nonce":1554609572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.184:6839/3059935143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":3059935143},{"type":"v1","addr":"172.21.15.184:6839","nonce":3059935143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.164:6839/2646401247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":2646401247},{"type":"v1","addr":"172.21.15.164:6839","nonce":2646401247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24530,"qdb_cluster":[24473,24478,24491,24493,24530]},"id":1}]} 2024-05-21T21:54:06.061 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:06.060+0000 7fb764000700 1 -- 172.21.15.151:0/3112285974 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fb7440b2880 msgr2=0x7fb7440b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:06.061 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:06.060+0000 7fb764000700 1 --2- 172.21.15.151:0/3112285974 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fb7440b2880 0x7fb7440b4d40 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb750009d20 tx=0x7fb750009260 comp rx=0 tx=0).stop 2024-05-21T21:54:06.061 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:06.060+0000 7fb764000700 1 -- 172.21.15.151:0/3112285974 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb75c06bb70 msgr2=0x7fb75c19c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:06.061 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:06.060+0000 7fb764000700 1 --2- 172.21.15.151:0/3112285974 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb75c06bb70 0x7fb75c19c410 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb74800e9f0 tx=0x7fb74800ed00 comp rx=0 tx=0).stop 2024-05-21T21:54:06.062 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:06.060+0000 7fb764000700 1 -- 172.21.15.151:0/3112285974 shutdown_connections 2024-05-21T21:54:06.062 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:06.060+0000 7fb764000700 1 --2- 172.21.15.151:0/3112285974 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fb7440b2880 0x7fb7440b4d40 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:06.062 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:06.060+0000 7fb764000700 1 --2- 172.21.15.151:0/3112285974 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb75c10fff0 0x7fb75c1a07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:06.062 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:06.060+0000 7fb764000700 1 --2- 172.21.15.151:0/3112285974 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb75c06b250 0x7fb75c19bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:06.062 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:06.060+0000 7fb764000700 1 --2- 172.21.15.151:0/3112285974 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb75c06bb70 0x7fb75c19c410 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:06.062 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:06.060+0000 7fb764000700 1 -- 172.21.15.151:0/3112285974 >> 172.21.15.151:0/3112285974 conn(0x7fb75c0793f0 msgr2=0x7fb75c111500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:54:06.062 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:06.060+0000 7fb764000700 1 -- 172.21.15.151:0/3112285974 shutdown_connections 2024-05-21T21:54:06.062 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:06.060+0000 7fb764000700 1 -- 172.21.15.151:0/3112285974 wait complete. 2024-05-21T21:54:06.063 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 28 2024-05-21T21:54:06.871 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph osd dump --format=json 2024-05-21T21:54:07.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:54:06 smithi151 ceph-mon[31623]: pgmap v400: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-21T21:54:07.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:54:06 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/3112285974' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T21:54:07.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:54:06 smithi184 ceph-mon[36942]: pgmap v400: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-21T21:54:07.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:54:06 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/3112285974' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T21:54:07.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:54:06 smithi164 ceph-mon[36269]: pgmap v400: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-21T21:54:07.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:54:06 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/3112285974' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T21:54:07.313 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:54:09.174 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:54:08 smithi151 ceph-mon[31623]: pgmap v401: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-21T21:54:09.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:54:08 smithi184 ceph-mon[36942]: pgmap v401: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-21T21:54:09.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:54:08 smithi164 ceph-mon[36269]: pgmap v401: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-21T21:54:09.505 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:09.503+0000 7f210a000700 1 -- 172.21.15.151:0/3202909669 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f21041041a0 msgr2=0x7f21041081c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:09.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:09.503+0000 7f210a000700 1 --2- 172.21.15.151:0/3202909669 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f21041041a0 0x7f21041081c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f20ec0099c0 tx=0x7f20ec009cd0 comp rx=0 tx=0).stop 2024-05-21T21:54:09.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:09.504+0000 7f210a000700 1 -- 172.21.15.151:0/3202909669 shutdown_connections 2024-05-21T21:54:09.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:09.504+0000 7f210a000700 1 --2- 172.21.15.151:0/3202909669 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21041037f0 0x7f2104103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:09.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:09.504+0000 7f210a000700 1 --2- 172.21.15.151:0/3202909669 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21041088f0 0x7f2104110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:09.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:09.504+0000 7f210a000700 1 --2- 172.21.15.151:0/3202909669 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f21041041a0 0x7f21041081c0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:09.507 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:09.504+0000 7f210a000700 1 -- 172.21.15.151:0/3202909669 >> 172.21.15.151:0/3202909669 conn(0x7f210407a6a0 msgr2=0x7f210407aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:54:09.507 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:09.504+0000 7f210a000700 1 -- 172.21.15.151:0/3202909669 shutdown_connections 2024-05-21T21:54:09.507 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:09.504+0000 7f210a000700 1 -- 172.21.15.151:0/3202909669 wait complete. 2024-05-21T21:54:09.509 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:09.505+0000 7f210a000700 1 Processor -- start 2024-05-21T21:54:09.510 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:09.505+0000 7f210a000700 1 -- start start 2024-05-21T21:54:09.510 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:09.505+0000 7f210a000700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21041037f0 0x7f210419bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:09.510 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:09.505+0000 7f210a000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f21041041a0 0x7f210419c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:09.510 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:09.506+0000 7f210a000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21041088f0 0x7f21041a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:09.510 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:09.506+0000 7f210a000700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2104113720 con 0x7f21041041a0 2024-05-21T21:54:09.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:09.506+0000 7f210a000700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f21041135a0 con 0x7f21041088f0 2024-05-21T21:54:09.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:09.506+0000 7f210a000700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f21041138a0 con 0x7f21041037f0 2024-05-21T21:54:09.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:09.506+0000 7f2103400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21041037f0 0x7f210419bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:09.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:09.506+0000 7f2103400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21041037f0 0x7f210419bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:58994/0 (socket says 172.21.15.151:58994) 2024-05-21T21:54:09.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:09.506+0000 7f2103400700 1 -- 172.21.15.151:0/1825960273 learned_addr learned my addr 172.21.15.151:0/1825960273 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:54:09.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:09.506+0000 7f2103e00700 1 --2- 172.21.15.151:0/1825960273 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21041088f0 0x7f21041a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:09.512 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:09.506+0000 7f2103400700 1 -- 172.21.15.151:0/1825960273 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21041088f0 msgr2=0x7f21041a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:09.512 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:09.506+0000 7f20fbe00700 1 --2- 172.21.15.151:0/1825960273 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f21041041a0 0x7f210419c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:09.512 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:09.506+0000 7f2103400700 1 --2- 172.21.15.151:0/1825960273 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21041088f0 0x7f21041a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:09.512 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:09.506+0000 7f2103400700 1 -- 172.21.15.151:0/1825960273 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f21041041a0 msgr2=0x7f210419c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:09.512 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:09.506+0000 7f2103400700 1 --2- 172.21.15.151:0/1825960273 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f21041041a0 0x7f210419c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:09.512 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:09.506+0000 7f2103400700 1 -- 172.21.15.151:0/1825960273 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21041a0db0 con 0x7f21041037f0 2024-05-21T21:54:09.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:09.506+0000 7f20fbe00700 1 --2- 172.21.15.151:0/1825960273 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f21041041a0 0x7f210419c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T21:54:09.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:09.507+0000 7f2103400700 1 --2- 172.21.15.151:0/1825960273 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21041037f0 0x7f210419bda0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f20f400e9a0 tx=0x7f20f400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:54:09.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:09.507+0000 7f2100c00700 1 -- 172.21.15.151:0/1825960273 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f400cbd0 con 0x7f21041037f0 2024-05-21T21:54:09.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:09.507+0000 7f2100c00700 1 -- 172.21.15.151:0/1825960273 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20f400cd70 con 0x7f21041037f0 2024-05-21T21:54:09.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:09.507+0000 7f210a000700 1 -- 172.21.15.151:0/1825960273 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21041a10a0 con 0x7f21041037f0 2024-05-21T21:54:09.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:09.507+0000 7f2100c00700 1 -- 172.21.15.151:0/1825960273 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f4018680 con 0x7f21041037f0 2024-05-21T21:54:09.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:09.507+0000 7f210a000700 1 -- 172.21.15.151:0/1825960273 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21041a8920 con 0x7f21041037f0 2024-05-21T21:54:09.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:09.508+0000 7f210a000700 1 -- 172.21.15.151:0/1825960273 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f210406a230 con 0x7f21041037f0 2024-05-21T21:54:09.514 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:09.511+0000 7f2100c00700 1 -- 172.21.15.151:0/1825960273 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f20f4010430 con 0x7f21041037f0 2024-05-21T21:54:09.514 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:09.512+0000 7f2100c00700 1 --2- 172.21.15.151:0/1825960273 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f20e80b2880 0x7f20e80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:09.514 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:09.512+0000 7f2100c00700 1 -- 172.21.15.151:0/1825960273 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f20f40e2980 con 0x7f21041037f0 2024-05-21T21:54:09.514 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:09.512+0000 7f20fbe00700 1 --2- 172.21.15.151:0/1825960273 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f20e80b2880 0x7f20e80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:09.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:09.513+0000 7f20fbe00700 1 --2- 172.21.15.151:0/1825960273 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f20e80b2880 0x7f20e80b4d40 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f20ec006010 tx=0x7f20ec003680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:54:09.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:09.514+0000 7f2100c00700 1 -- 172.21.15.151:0/1825960273 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f20f40aa660 con 0x7f21041037f0 2024-05-21T21:54:09.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:09.750+0000 7f210a000700 1 -- 172.21.15.151:0/1825960273 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f21041a8af0 con 0x7f21041037f0 2024-05-21T21:54:09.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:09.752+0000 7f2100c00700 1 -- 172.21.15.151:0/1825960273 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22744 (secure 0 0 0) 0x7f20f40a9f60 con 0x7f21041037f0 2024-05-21T21:54:09.754 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T21:54:09.754 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":82,"fsid":"c8c7a2e6-17ba-11ef-bc9a-c7b262605968","created":"2024-05-21T21:41:52.412833+0000","modified":"2024-05-21T21:52:10.856587+0000","last_up_change":"2024-05-21T21:50:05.401314+0000","last_in_change":"2024-05-21T21:49:45.684136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-21T21:45:55.196046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-21T21:51:11.447428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-21T21:51:15.618605+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"18db8e54-0cfc-4e12-9c84-6f72a99cc989","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":2420400473},{"type":"v1","addr":"172.21.15.151:6803","nonce":2420400473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":2420400473},{"type":"v1","addr":"172.21.15.151:6805","nonce":2420400473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":2420400473},{"type":"v1","addr":"172.21.15.151:6809","nonce":2420400473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":2420400473},{"type":"v1","addr":"172.21.15.151:6807","nonce":2420400473}]},"public_addr":"172.21.15.151:6803/2420400473","cluster_addr":"172.21.15.151:6805/2420400473","heartbeat_back_addr":"172.21.15.151:6809/2420400473","heartbeat_front_addr":"172.21.15.151:6807/2420400473","state":["exists","up"]},{"osd":1,"uuid":"bac384c7-1102-4815-81a8-10cc7661d230","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":3900240447},{"type":"v1","addr":"172.21.15.164:6801","nonce":3900240447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":3900240447},{"type":"v1","addr":"172.21.15.164:6803","nonce":3900240447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":3900240447},{"type":"v1","addr":"172.21.15.164:6807","nonce":3900240447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":3900240447},{"type":"v1","addr":"172.21.15.164:6805","nonce":3900240447}]},"public_addr":"172.21.15.164:6801/3900240447","cluster_addr":"172.21.15.164:6803/3900240447","heartbeat_back_addr":"172.21.15.164:6807/3900240447","heartbeat_front_addr":"172.21.15.164:6805/3900240447","state":["exists","up"]},{"osd":2,"uuid":"b7e4a782-1518-4d43-bee9-7fd768ee5cb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":3971150737},{"type":"v1","addr":"172.21.15.184:6801","nonce":3971150737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3971150737},{"type":"v1","addr":"172.21.15.184:6803","nonce":3971150737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3971150737},{"type":"v1","addr":"172.21.15.184:6807","nonce":3971150737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3971150737},{"type":"v1","addr":"172.21.15.184:6805","nonce":3971150737}]},"public_addr":"172.21.15.184:6801/3971150737","cluster_addr":"172.21.15.184:6803/3971150737","heartbeat_back_addr":"172.21.15.184:6807/3971150737","heartbeat_front_addr":"172.21.15.184:6805/3971150737","state":["exists","up"]},{"osd":3,"uuid":"ab3df826-39e3-4635-a49c-2b5ffdbf12f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":556594400},{"type":"v1","addr":"172.21.15.151:6811","nonce":556594400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":556594400},{"type":"v1","addr":"172.21.15.151:6813","nonce":556594400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":556594400},{"type":"v1","addr":"172.21.15.151:6817","nonce":556594400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":556594400},{"type":"v1","addr":"172.21.15.151:6815","nonce":556594400}]},"public_addr":"172.21.15.151:6811/556594400","cluster_addr":"172.21.15.151:6813/556594400","heartbeat_back_addr":"172.21.15.151:6817/556594400","heartbeat_front_addr":"172.21.15.151:6815/556594400","state":["exists","up"]},{"osd":4,"uuid":"00cbf6ae-0f42-4685-a77d-d803c6467aa0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":988881973},{"type":"v1","addr":"172.21.15.164:6809","nonce":988881973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":988881973},{"type":"v1","addr":"172.21.15.164:6811","nonce":988881973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":988881973},{"type":"v1","addr":"172.21.15.164:6815","nonce":988881973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":988881973},{"type":"v1","addr":"172.21.15.164:6813","nonce":988881973}]},"public_addr":"172.21.15.164:6809/988881973","cluster_addr":"172.21.15.164:6811/988881973","heartbeat_back_addr":"172.21.15.164:6815/988881973","heartbeat_front_addr":"172.21.15.164:6813/988881973","state":["exists","up"]},{"osd":5,"uuid":"fad14221-0ef0-4d40-b414-c6c9f1f9718c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3293311106},{"type":"v1","addr":"172.21.15.184:6809","nonce":3293311106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":3293311106},{"type":"v1","addr":"172.21.15.184:6811","nonce":3293311106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":3293311106},{"type":"v1","addr":"172.21.15.184:6815","nonce":3293311106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":3293311106},{"type":"v1","addr":"172.21.15.184:6813","nonce":3293311106}]},"public_addr":"172.21.15.184:6809/3293311106","cluster_addr":"172.21.15.184:6811/3293311106","heartbeat_back_addr":"172.21.15.184:6815/3293311106","heartbeat_front_addr":"172.21.15.184:6813/3293311106","state":["exists","up"]},{"osd":6,"uuid":"ab8a60f6-0dec-4c1c-89db-f2e2f638f1ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":3377550762},{"type":"v1","addr":"172.21.15.151:6819","nonce":3377550762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":3377550762},{"type":"v1","addr":"172.21.15.151:6821","nonce":3377550762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":3377550762},{"type":"v1","addr":"172.21.15.151:6825","nonce":3377550762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":3377550762},{"type":"v1","addr":"172.21.15.151:6823","nonce":3377550762}]},"public_addr":"172.21.15.151:6819/3377550762","cluster_addr":"172.21.15.151:6821/3377550762","heartbeat_back_addr":"172.21.15.151:6825/3377550762","heartbeat_front_addr":"172.21.15.151:6823/3377550762","state":["exists","up"]},{"osd":7,"uuid":"db332268-9972-4ed7-a307-822b83a0fbb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":3869696936},{"type":"v1","addr":"172.21.15.164:6817","nonce":3869696936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":3869696936},{"type":"v1","addr":"172.21.15.164:6819","nonce":3869696936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":3869696936},{"type":"v1","addr":"172.21.15.164:6823","nonce":3869696936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":3869696936},{"type":"v1","addr":"172.21.15.164:6821","nonce":3869696936}]},"public_addr":"172.21.15.164:6817/3869696936","cluster_addr":"172.21.15.164:6819/3869696936","heartbeat_back_addr":"172.21.15.164:6823/3869696936","heartbeat_front_addr":"172.21.15.164:6821/3869696936","state":["exists","up"]},{"osd":8,"uuid":"fe2fbe1c-549a-4947-a188-426dfca04d10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":575273541},{"type":"v1","addr":"172.21.15.184:6817","nonce":575273541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":575273541},{"type":"v1","addr":"172.21.15.184:6819","nonce":575273541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":575273541},{"type":"v1","addr":"172.21.15.184:6823","nonce":575273541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":575273541},{"type":"v1","addr":"172.21.15.184:6821","nonce":575273541}]},"public_addr":"172.21.15.184:6817/575273541","cluster_addr":"172.21.15.184:6819/575273541","heartbeat_back_addr":"172.21.15.184:6823/575273541","heartbeat_front_addr":"172.21.15.184:6821/575273541","state":["exists","up"]},{"osd":9,"uuid":"3a7b450f-8004-4adf-bae3-970918f98f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":337128533},{"type":"v1","addr":"172.21.15.151:6827","nonce":337128533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":337128533},{"type":"v1","addr":"172.21.15.151:6829","nonce":337128533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":337128533},{"type":"v1","addr":"172.21.15.151:6833","nonce":337128533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":337128533},{"type":"v1","addr":"172.21.15.151:6831","nonce":337128533}]},"public_addr":"172.21.15.151:6827/337128533","cluster_addr":"172.21.15.151:6829/337128533","heartbeat_back_addr":"172.21.15.151:6833/337128533","heartbeat_front_addr":"172.21.15.151:6831/337128533","state":["exists","up"]},{"osd":10,"uuid":"8fcdf271-90c4-4db3-be69-b0ca716dfd80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":1823448782},{"type":"v1","addr":"172.21.15.164:6825","nonce":1823448782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":1823448782},{"type":"v1","addr":"172.21.15.164:6827","nonce":1823448782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":1823448782},{"type":"v1","addr":"172.21.15.164:6831","nonce":1823448782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":1823448782},{"type":"v1","addr":"172.21.15.164:6829","nonce":1823448782}]},"public_addr":"172.21.15.164:6825/1823448782","cluster_addr":"172.21.15.164:6827/1823448782","heartbeat_back_addr":"172.21.15.164:6831/1823448782","heartbeat_front_addr":"172.21.15.164:6829/1823448782","state":["exists","up"]},{"osd":11,"uuid":"e661477b-0c09-409f-a2fe-6ebd34642a8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":1563340128},{"type":"v1","addr":"172.21.15.184:6825","nonce":1563340128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":1563340128},{"type":"v1","addr":"172.21.15.184:6827","nonce":1563340128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":1563340128},{"type":"v1","addr":"172.21.15.184:6831","nonce":1563340128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":1563340128},{"type":"v1","addr":"172.21.15.184:6829","nonce":1563340128}]},"public_addr":"172.21.15.184:6825/1563340128","cluster_addr":"172.21.15.184:6827/1563340128","heartbeat_back_addr":"172.21.15.184:6831/1563340128","heartbeat_front_addr":"172.21.15.184:6829/1563340128","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:45:02.821160+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:45:27.844564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:45:51.904981+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:46:24.376245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:46:50.036621+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:47:14.748422+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:47:47.396383+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:48:13.543174+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:48:38.213285+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:49:11.980042+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:49:38.519761+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:50:03.439898+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.151:0/31963128":"2024-05-22T21:43:09.041455+0000","172.21.15.151:6800/3428827300":"2024-05-22T21:43:09.041455+0000","172.21.15.151:6801/3428827300":"2024-05-22T21:43:09.041455+0000","172.21.15.151:0/131989141":"2024-05-22T21:43:09.041455+0000","172.21.15.151:6801/1128885292":"2024-05-22T21:42:35.440147+0000","172.21.15.151:0/2703664519":"2024-05-22T21:43:09.041455+0000","172.21.15.151:6800/1128885292":"2024-05-22T21:42:35.440147+0000","172.21.15.151:0/3052943699":"2024-05-22T21:42:35.440147+0000","172.21.15.151:0/3929452511":"2024-05-22T21:42:35.440147+0000","172.21.15.151:0/3762889700":"2024-05-22T21:42:35.440147+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-21T21:54:09.756 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:09.755+0000 7f210a000700 1 -- 172.21.15.151:0/1825960273 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f20e80b2880 msgr2=0x7f20e80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:09.756 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:09.755+0000 7f210a000700 1 --2- 172.21.15.151:0/1825960273 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f20e80b2880 0x7f20e80b4d40 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f20ec006010 tx=0x7f20ec003680 comp rx=0 tx=0).stop 2024-05-21T21:54:09.756 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:09.755+0000 7f210a000700 1 -- 172.21.15.151:0/1825960273 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21041037f0 msgr2=0x7f210419bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:09.757 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:09.755+0000 7f210a000700 1 --2- 172.21.15.151:0/1825960273 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21041037f0 0x7f210419bda0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f20f400e9a0 tx=0x7f20f400ecb0 comp rx=0 tx=0).stop 2024-05-21T21:54:09.757 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:09.755+0000 7f210a000700 1 -- 172.21.15.151:0/1825960273 shutdown_connections 2024-05-21T21:54:09.757 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:09.755+0000 7f210a000700 1 --2- 172.21.15.151:0/1825960273 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f20e80b2880 0x7f20e80b4d40 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:09.757 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:09.755+0000 7f210a000700 1 --2- 172.21.15.151:0/1825960273 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21041088f0 0x7f21041a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:09.757 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:09.755+0000 7f210a000700 1 --2- 172.21.15.151:0/1825960273 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21041037f0 0x7f210419bda0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:09.757 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:09.755+0000 7f210a000700 1 --2- 172.21.15.151:0/1825960273 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f21041041a0 0x7f210419c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:09.757 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:09.755+0000 7f210a000700 1 -- 172.21.15.151:0/1825960273 >> 172.21.15.151:0/1825960273 conn(0x7f210407a6a0 msgr2=0x7f21041018f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:54:09.757 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:09.755+0000 7f210a000700 1 -- 172.21.15.151:0/1825960273 shutdown_connections 2024-05-21T21:54:09.757 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:09.755+0000 7f210a000700 1 -- 172.21.15.151:0/1825960273 wait complete. 2024-05-21T21:54:10.164 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:54:09 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/1825960273' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T21:54:10.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:54:09 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/1825960273' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T21:54:10.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:54:09 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/1825960273' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T21:54:10.657 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-21T21:54:11.102 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:54:11.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:54:10 smithi151 ceph-mon[31623]: pgmap v402: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-21T21:54:11.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:54:10 smithi184 ceph-mon[36942]: pgmap v402: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-21T21:54:11.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:54:10 smithi164 ceph-mon[36269]: pgmap v402: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-21T21:54:13.068 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:54:12 smithi151 ceph-mon[31623]: pgmap v403: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-21T21:54:13.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:54:12 smithi184 ceph-mon[36942]: pgmap v403: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-21T21:54:13.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:54:12 smithi164 ceph-mon[36269]: pgmap v403: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-21T21:54:13.336 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:13.334+0000 7fd1dec00700 1 -- 172.21.15.151:0/3441775946 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd1d8106140 msgr2=0x7fd1d8112650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:13.336 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:13.334+0000 7fd1dec00700 1 --2- 172.21.15.151:0/3441775946 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd1d8106140 0x7fd1d8112650 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd1d000b0a0 tx=0x7fd1d000b3b0 comp rx=0 tx=0).stop 2024-05-21T21:54:13.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:13.334+0000 7fd1dec00700 1 -- 172.21.15.151:0/3441775946 shutdown_connections 2024-05-21T21:54:13.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:13.334+0000 7fd1dec00700 1 --2- 172.21.15.151:0/3441775946 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd1d8106140 0x7fd1d8112650 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:13.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:13.334+0000 7fd1dec00700 1 --2- 172.21.15.151:0/3441775946 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1d810b780 0x7fd1d810bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:13.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:13.334+0000 7fd1dec00700 1 --2- 172.21.15.151:0/3441775946 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1d8105780 0x7fd1d8105c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:13.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:13.334+0000 7fd1dec00700 1 -- 172.21.15.151:0/3441775946 >> 172.21.15.151:0/3441775946 conn(0x7fd1d8101270 msgr2=0x7fd1d8103690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:54:13.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:13.334+0000 7fd1dec00700 1 -- 172.21.15.151:0/3441775946 shutdown_connections 2024-05-21T21:54:13.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:13.334+0000 7fd1dec00700 1 -- 172.21.15.151:0/3441775946 wait complete. 2024-05-21T21:54:13.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:13.335+0000 7fd1dec00700 1 Processor -- start 2024-05-21T21:54:13.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:13.336+0000 7fd1dec00700 1 -- start start 2024-05-21T21:54:13.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:13.336+0000 7fd1dec00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd1d8105780 0x7fd1d819bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:13.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:13.336+0000 7fd1dec00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1d8106140 0x7fd1d819c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:13.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:13.336+0000 7fd1dec00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1d810b780 0x7fd1d81a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:13.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:13.336+0000 7fd1dec00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d81159c0 con 0x7fd1d8105780 2024-05-21T21:54:13.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:13.336+0000 7fd1dec00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d8115840 con 0x7fd1d810b780 2024-05-21T21:54:13.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:13.336+0000 7fd1dec00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d8115b40 con 0x7fd1d8106140 2024-05-21T21:54:13.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:13.336+0000 7fd1d7e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd1d8105780 0x7fd1d819bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:13.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:13.337+0000 7fd1d7e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd1d8105780 0x7fd1d819bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:58352/0 (socket says 172.21.15.151:58352) 2024-05-21T21:54:13.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:13.337+0000 7fd1d7e00700 1 -- 172.21.15.151:0/1096849034 learned_addr learned my addr 172.21.15.151:0/1096849034 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:54:13.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:13.337+0000 7fd1d7400700 1 --2- 172.21.15.151:0/1096849034 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1d8106140 0x7fd1d819c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:13.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:13.337+0000 7fd1dcc00700 1 --2- 172.21.15.151:0/1096849034 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1d810b780 0x7fd1d81a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:13.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:13.337+0000 7fd1d7e00700 1 -- 172.21.15.151:0/1096849034 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1d8106140 msgr2=0x7fd1d819c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:13.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:13.337+0000 7fd1d7e00700 1 --2- 172.21.15.151:0/1096849034 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1d8106140 0x7fd1d819c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:13.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:13.337+0000 7fd1d7e00700 1 -- 172.21.15.151:0/1096849034 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1d810b780 msgr2=0x7fd1d81a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:13.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:13.337+0000 7fd1d7e00700 1 --2- 172.21.15.151:0/1096849034 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1d810b780 0x7fd1d81a0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:13.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:13.337+0000 7fd1d7e00700 1 -- 172.21.15.151:0/1096849034 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1d81a0da0 con 0x7fd1d8105780 2024-05-21T21:54:13.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:13.337+0000 7fd1d7e00700 1 --2- 172.21.15.151:0/1096849034 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd1d8105780 0x7fd1d819bd90 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd1c800d980 tx=0x7fd1c800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:54:13.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:13.338+0000 7fd1d4c00700 1 -- 172.21.15.151:0/1096849034 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c8016070 con 0x7fd1d8105780 2024-05-21T21:54:13.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:13.338+0000 7fd1d4c00700 1 -- 172.21.15.151:0/1096849034 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1c8004830 con 0x7fd1d8105780 2024-05-21T21:54:13.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:13.338+0000 7fd1dec00700 1 -- 172.21.15.151:0/1096849034 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1d81a1090 con 0x7fd1d8105780 2024-05-21T21:54:13.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:13.338+0000 7fd1d4c00700 1 -- 172.21.15.151:0/1096849034 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c8005540 con 0x7fd1d8105780 2024-05-21T21:54:13.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:13.338+0000 7fd1dec00700 1 -- 172.21.15.151:0/1096849034 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1d81a8910 con 0x7fd1d8105780 2024-05-21T21:54:13.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:13.339+0000 7fd1dec00700 1 -- 172.21.15.151:0/1096849034 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1d806a230 con 0x7fd1d8105780 2024-05-21T21:54:13.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:13.341+0000 7fd1d4c00700 1 -- 172.21.15.151:0/1096849034 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fd1c8004220 con 0x7fd1d8105780 2024-05-21T21:54:13.345 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:13.344+0000 7fd1d4c00700 1 --2- 172.21.15.151:0/1096849034 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fd1bc0b2900 0x7fd1bc0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:13.345 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:13.344+0000 7fd1d4c00700 1 -- 172.21.15.151:0/1096849034 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fd1c80e2450 con 0x7fd1d8105780 2024-05-21T21:54:13.345 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:13.344+0000 7fd1d7400700 1 --2- 172.21.15.151:0/1096849034 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fd1bc0b2900 0x7fd1bc0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:13.346 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:13.345+0000 7fd1d7400700 1 --2- 172.21.15.151:0/1096849034 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fd1bc0b2900 0x7fd1bc0b4dc0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd1d8103840 tx=0x7fd1c400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:54:13.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:13.347+0000 7fd1d4c00700 1 -- 172.21.15.151:0/1096849034 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd1c801f020 con 0x7fd1d8105780 2024-05-21T21:54:13.630 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:13.629+0000 7fd1dec00700 1 -- 172.21.15.151:0/1096849034 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd1d802e560 con 0x7fd1d8105780 2024-05-21T21:54:13.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:13.630+0000 7fd1d4c00700 1 -- 172.21.15.151:0/1096849034 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11958 (secure 0 0 0) 0x7fd1c80aa130 con 0x7fd1d8105780 2024-05-21T21:54:13.632 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T21:54:13.632 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14358,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/158010529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":158010529},{"type":"v1","addr":"172.21.15.151:6835","nonce":158010529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6839/2069058115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":2069058115},{"type":"v1","addr":"172.21.15.151:6839","nonce":2069058115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T21:51:19.552069+0000","modified":"2024-05-21T21:51:43.002737+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24530,"mds_1":24493,"mds_2":24491,"mds_3":24478,"mds_4":24473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14403":{"gid":14403,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6841/1749723794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6840","nonce":1749723794},{"type":"v1","addr":"172.21.15.151:6841","nonce":1749723794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/2026266114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":2026266114},{"type":"v1","addr":"172.21.15.184:6833","nonce":2026266114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/2467900844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":2467900844},{"type":"v1","addr":"172.21.15.164:6833","nonce":2467900844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3367640960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3367640960},{"type":"v1","addr":"172.21.15.151:6837","nonce":3367640960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/3264142621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3264142621},{"type":"v1","addr":"172.21.15.184:6835","nonce":3264142621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":17,"addr":"172.21.15.164:6835/2340428470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2340428470},{"type":"v1","addr":"172.21.15.164:6835","nonce":2340428470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":16,"addr":"172.21.15.184:6837/2988194888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":2988194888},{"type":"v1","addr":"172.21.15.184:6837","nonce":2988194888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.164:6837/1554609572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1554609572},{"type":"v1","addr":"172.21.15.164:6837","nonce":1554609572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.184:6839/3059935143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":3059935143},{"type":"v1","addr":"172.21.15.184:6839","nonce":3059935143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.164:6839/2646401247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":2646401247},{"type":"v1","addr":"172.21.15.164:6839","nonce":2646401247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24530,"qdb_cluster":[24473,24478,24491,24493,24530]},"id":1}]} 2024-05-21T21:54:13.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:13.635+0000 7fd1dec00700 1 -- 172.21.15.151:0/1096849034 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fd1bc0b2900 msgr2=0x7fd1bc0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:13.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:13.635+0000 7fd1dec00700 1 --2- 172.21.15.151:0/1096849034 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fd1bc0b2900 0x7fd1bc0b4dc0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd1d8103840 tx=0x7fd1c400b410 comp rx=0 tx=0).stop 2024-05-21T21:54:13.637 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:13.635+0000 7fd1dec00700 1 -- 172.21.15.151:0/1096849034 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd1d8105780 msgr2=0x7fd1d819bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:13.637 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:13.635+0000 7fd1dec00700 1 --2- 172.21.15.151:0/1096849034 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd1d8105780 0x7fd1d819bd90 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd1c800d980 tx=0x7fd1c800dc90 comp rx=0 tx=0).stop 2024-05-21T21:54:13.637 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:13.635+0000 7fd1dec00700 1 -- 172.21.15.151:0/1096849034 shutdown_connections 2024-05-21T21:54:13.637 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:13.635+0000 7fd1dec00700 1 --2- 172.21.15.151:0/1096849034 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fd1bc0b2900 0x7fd1bc0b4dc0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:13.637 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:13.635+0000 7fd1dec00700 1 --2- 172.21.15.151:0/1096849034 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd1d810b780 0x7fd1d81a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:13.637 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:13.635+0000 7fd1dec00700 1 --2- 172.21.15.151:0/1096849034 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd1d8105780 0x7fd1d819bd90 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:13.637 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:13.635+0000 7fd1dec00700 1 --2- 172.21.15.151:0/1096849034 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1d8106140 0x7fd1d819c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:13.637 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:13.635+0000 7fd1dec00700 1 -- 172.21.15.151:0/1096849034 >> 172.21.15.151:0/1096849034 conn(0x7fd1d8101270 msgr2=0x7fd1d8101880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:54:13.637 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:13.635+0000 7fd1dec00700 1 -- 172.21.15.151:0/1096849034 shutdown_connections 2024-05-21T21:54:13.637 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:13.635+0000 7fd1dec00700 1 -- 172.21.15.151:0/1096849034 wait complete. 2024-05-21T21:54:13.638 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 28 2024-05-21T21:54:14.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:54:13 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/1096849034' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T21:54:14.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:54:13 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/1096849034' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T21:54:14.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:54:13 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/1096849034' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T21:54:14.505 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-21T21:51:19.552069+0000', 'modified': '2024-05-21T21:51:43.002737+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24530, 'mds_1': 24493, 'mds_2': 24491, 'mds_3': 24478, 'mds_4': 24473}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14403': {'gid': 14403, 'name': 'a', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.151:6841/1749723794', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.151:6840', 'nonce': 1749723794}, {'type': 'v1', 'addr': '172.21.15.151:6841', 'nonce': 1749723794}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24436': {'gid': 24436, 'name': 'l', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.184:6833/2026266114', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6832', 'nonce': 2026266114}, {'type': 'v1', 'addr': '172.21.15.184:6833', 'nonce': 2026266114}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24443': {'gid': 24443, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.164:6833/2467900844', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6832', 'nonce': 2467900844}, {'type': 'v1', 'addr': '172.21.15.164:6833', 'nonce': 2467900844}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24461': {'gid': 24461, 'name': 'd', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.151:6837/3367640960', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.151:6836', 'nonce': 3367640960}, {'type': 'v1', 'addr': '172.21.15.151:6837', 'nonce': 3367640960}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24466': {'gid': 24466, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.184:6835/3264142621', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6834', 'nonce': 3264142621}, {'type': 'v1', 'addr': '172.21.15.184:6835', 'nonce': 3264142621}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24473': {'gid': 24473, 'name': 'e', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.164:6835/2340428470', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6834', 'nonce': 2340428470}, {'type': 'v1', 'addr': '172.21.15.164:6835', 'nonce': 2340428470}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24478': {'gid': 24478, 'name': 'c', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 16, 'addr': '172.21.15.184:6837/2988194888', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6836', 'nonce': 2988194888}, {'type': 'v1', 'addr': '172.21.15.184:6837', 'nonce': 2988194888}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24491': {'gid': 24491, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.164:6837/1554609572', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6836', 'nonce': 1554609572}, {'type': 'v1', 'addr': '172.21.15.164:6837', 'nonce': 1554609572}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24493': {'gid': 24493, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.184:6839/3059935143', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6838', 'nonce': 3059935143}, {'type': 'v1', 'addr': '172.21.15.184:6839', 'nonce': 3059935143}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24530': {'gid': 24530, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.164:6839/2646401247', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6838', 'nonce': 2646401247}, {'type': 'v1', 'addr': '172.21.15.164:6839', 'nonce': 2646401247}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24530, 'qdb_cluster': [24473, 24478, 24491, 24493, 24530]} 2024-05-21T21:54:14.505 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-05-21T21:54:14.505 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.e status 2024-05-21T21:54:14.966 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:54:15.233 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:54:14 smithi151 ceph-mon[31623]: pgmap v404: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-21T21:54:15.235 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:54:14 smithi184 ceph-mon[36942]: pgmap v404: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-21T21:54:15.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:54:14 smithi164 ceph-mon[36269]: pgmap v404: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-21T21:54:17.002 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.000+0000 7fb223e00700 1 -- 172.21.15.151:0/613982920 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb21c076090 msgr2=0x7fb21c10fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:17.002 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.000+0000 7fb223e00700 1 --2- 172.21.15.151:0/613982920 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb21c076090 0x7fb21c10fa00 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb2080098f0 tx=0x7fb208009c00 comp rx=0 tx=0).stop 2024-05-21T21:54:17.002 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.000+0000 7fb223e00700 1 -- 172.21.15.151:0/613982920 shutdown_connections 2024-05-21T21:54:17.002 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.000+0000 7fb223e00700 1 --2- 172.21.15.151:0/613982920 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb21c10ff40 0x7fb21c112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:17.002 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.000+0000 7fb223e00700 1 --2- 172.21.15.151:0/613982920 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb21c075770 0x7fb21c075b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:17.002 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.000+0000 7fb223e00700 1 --2- 172.21.15.151:0/613982920 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb21c076090 0x7fb21c10fa00 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:17.002 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.000+0000 7fb223e00700 1 -- 172.21.15.151:0/613982920 >> 172.21.15.151:0/613982920 conn(0x7fb21c07a6a0 msgr2=0x7fb21c07aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:54:17.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.000+0000 7fb223e00700 1 -- 172.21.15.151:0/613982920 shutdown_connections 2024-05-21T21:54:17.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.000+0000 7fb223e00700 1 -- 172.21.15.151:0/613982920 wait complete. 2024-05-21T21:54:17.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.001+0000 7fb223e00700 1 Processor -- start 2024-05-21T21:54:17.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.002+0000 7fb223e00700 1 -- start start 2024-05-21T21:54:17.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.002+0000 7fb223e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb21c075770 0x7fb21c19bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:17.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.002+0000 7fb223e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb21c076090 0x7fb21c19c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:17.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.002+0000 7fb223e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb21c10ff40 0x7fb21c1a0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:17.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.002+0000 7fb223e00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb21c115950 con 0x7fb21c075770 2024-05-21T21:54:17.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.002+0000 7fb223e00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb21c1157d0 con 0x7fb21c076090 2024-05-21T21:54:17.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.002+0000 7fb223e00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb21c115ad0 con 0x7fb21c10ff40 2024-05-21T21:54:17.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.002+0000 7fb221400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb21c075770 0x7fb21c19bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:17.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.003+0000 7fb220a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb21c076090 0x7fb21c19c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:17.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.003+0000 7fb221e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb21c10ff40 0x7fb21c1a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:17.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.003+0000 7fb221400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb21c075770 0x7fb21c19bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:34478/0 (socket says 172.21.15.151:34478) 2024-05-21T21:54:17.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.003+0000 7fb221400700 1 -- 172.21.15.151:0/3339799597 learned_addr learned my addr 172.21.15.151:0/3339799597 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:54:17.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.003+0000 7fb221e00700 1 -- 172.21.15.151:0/3339799597 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb21c076090 msgr2=0x7fb21c19c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:17.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.003+0000 7fb221e00700 1 --2- 172.21.15.151:0/3339799597 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb21c076090 0x7fb21c19c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:17.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.003+0000 7fb221e00700 1 -- 172.21.15.151:0/3339799597 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb21c075770 msgr2=0x7fb21c19bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:17.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.003+0000 7fb221e00700 1 --2- 172.21.15.151:0/3339799597 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb21c075770 0x7fb21c19bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:17.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.003+0000 7fb221e00700 1 -- 172.21.15.151:0/3339799597 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb21c1a0eb0 con 0x7fb21c10ff40 2024-05-21T21:54:17.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.003+0000 7fb220a00700 1 --2- 172.21.15.151:0/3339799597 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb21c076090 0x7fb21c19c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:54:17.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.003+0000 7fb221e00700 1 --2- 172.21.15.151:0/3339799597 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb21c10ff40 0x7fb21c1a0770 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb21400e940 tx=0x7fb21400ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:54:17.006 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.004+0000 7fb21a000700 1 -- 172.21.15.151:0/3339799597 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb214004220 con 0x7fb21c10ff40 2024-05-21T21:54:17.006 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.004+0000 7fb21a000700 1 -- 172.21.15.151:0/3339799597 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2140043c0 con 0x7fb21c10ff40 2024-05-21T21:54:17.006 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.004+0000 7fb223e00700 1 -- 172.21.15.151:0/3339799597 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb21c1a11a0 con 0x7fb21c10ff40 2024-05-21T21:54:17.006 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.004+0000 7fb21a000700 1 -- 172.21.15.151:0/3339799597 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb214010700 con 0x7fb21c10ff40 2024-05-21T21:54:17.006 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.004+0000 7fb223e00700 1 -- 172.21.15.151:0/3339799597 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb21c1a8a20 con 0x7fb21c10ff40 2024-05-21T21:54:17.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.009+0000 7fb21a000700 1 -- 172.21.15.151:0/3339799597 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fb214004750 con 0x7fb21c10ff40 2024-05-21T21:54:17.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.010+0000 7fb22e6f8b80 1 Processor -- start 2024-05-21T21:54:17.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.010+0000 7fb22e6f8b80 1 -- start start 2024-05-21T21:54:17.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.010+0000 7fb22e6f8b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d77f8d9e60 0x55d77f8da240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:17.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.010+0000 7fb22e6f8b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d77f8da780 0x55d77f8d4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:17.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.010+0000 7fb22e6f8b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d77f8d4830 0x55d77f8d4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:17.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.010+0000 7fb22e6f8b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d77f832df0 con 0x55d77f8d9e60 2024-05-21T21:54:17.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.010+0000 7fb22e6f8b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55d77f822990 con 0x55d77f8da780 2024-05-21T21:54:17.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.010+0000 7fb22e6f8b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55d77f8269d0 con 0x55d77f8d4830 2024-05-21T21:54:17.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.011+0000 7fb21a000700 1 --2- 172.21.15.151:0/3339799597 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fb2000b2900 0x7fb2000b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:17.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.011+0000 7fb220a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d77f8d9e60 0x55d77f8da240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:17.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.011+0000 7fb220a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d77f8d9e60 0x55d77f8da240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:34492/0 (socket says 172.21.15.151:34492) 2024-05-21T21:54:17.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.011+0000 7fb220a00700 1 -- 172.21.15.151:0/2127594777 learned_addr learned my addr 172.21.15.151:0/2127594777 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:54:17.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.011+0000 7fb220a00700 1 --2- 172.21.15.151:0/2127594777 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d77f8d4830 0x55d77f8d4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:17.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.011+0000 7fb221400700 1 --2- 172.21.15.151:0/2127594777 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d77f8da780 0x55d77f8d4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:17.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.011+0000 7fb21a000700 1 -- 172.21.15.151:0/3339799597 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fb2140e1f10 con 0x7fb21c10ff40 2024-05-21T21:54:17.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.011+0000 7fb221400700 1 -- 172.21.15.151:0/2127594777 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d77f8d4830 msgr2=0x55d77f8d4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:54:17.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.011+0000 7fb221400700 1 --2- 172.21.15.151:0/2127594777 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d77f8d4830 0x55d77f8d4c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:17.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.011+0000 7fb221400700 1 -- 172.21.15.151:0/2127594777 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d77f8d9e60 msgr2=0x55d77f8da240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:54:17.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.011+0000 7fb221400700 1 --2- 172.21.15.151:0/2127594777 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d77f8d9e60 0x55d77f8da240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:17.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.011+0000 7fb221400700 1 -- 172.21.15.151:0/2127594777 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d77f7eb3d0 con 0x55d77f8da780 2024-05-21T21:54:17.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.011+0000 7fb221e00700 1 --2- 172.21.15.151:0/3339799597 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fb2000b2900 0x7fb2000b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:17.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.012+0000 7fb221400700 1 --2- 172.21.15.151:0/2127594777 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d77f8da780 0x55d77f8d4210 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb20c00c590 tx=0x7fb20c0099e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e29a92f419e5111b server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:54:17.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.012+0000 7fb212a00700 1 -- 172.21.15.151:0/2127594777 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb20c003a00 con 0x55d77f8da780 2024-05-21T21:54:17.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.012+0000 7fb212a00700 1 -- 172.21.15.151:0/2127594777 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb20c01a070 con 0x55d77f8da780 2024-05-21T21:54:17.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.012+0000 7fb212a00700 1 -- 172.21.15.151:0/2127594777 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb20c007e30 con 0x55d77f8da780 2024-05-21T21:54:17.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.013+0000 7fb22e6f8b80 1 -- 172.21.15.151:0/2127594777 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d77f8da780 msgr2=0x55d77f8d4210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:17.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.013+0000 7fb22e6f8b80 1 --2- 172.21.15.151:0/2127594777 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d77f8da780 0x55d77f8d4210 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb20c00c590 tx=0x7fb20c0099e0 comp rx=0 tx=0).stop 2024-05-21T21:54:17.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.013+0000 7fb221e00700 1 --2- 172.21.15.151:0/3339799597 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fb2000b2900 0x7fb2000b4dc0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb21401a040 tx=0x7fb2140051c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:54:17.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.013+0000 7fb22e6f8b80 1 -- 172.21.15.151:0/2127594777 shutdown_connections 2024-05-21T21:54:17.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.013+0000 7fb22e6f8b80 1 --2- 172.21.15.151:0/2127594777 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d77f8d4830 0x55d77f8d4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:17.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.013+0000 7fb22e6f8b80 1 --2- 172.21.15.151:0/2127594777 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d77f8d9e60 0x55d77f8da240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:17.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.013+0000 7fb22e6f8b80 1 --2- 172.21.15.151:0/2127594777 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d77f8da780 0x55d77f8d4210 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:17.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.013+0000 7fb22e6f8b80 1 -- 172.21.15.151:0/2127594777 >> 172.21.15.151:0/2127594777 conn(0x55d77f802b90 msgr2=0x55d77f8cefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:54:17.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.013+0000 7fb22e6f8b80 1 -- 172.21.15.151:0/2127594777 shutdown_connections 2024-05-21T21:54:17.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.013+0000 7fb22e6f8b80 1 -- 172.21.15.151:0/2127594777 wait complete. 2024-05-21T21:54:17.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.014+0000 7fb22e6f8b80 1 -- start start 2024-05-21T21:54:17.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.014+0000 7fb22e6f8b80 1 Processor -- start 2024-05-21T21:54:17.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.014+0000 7fb22e6f8b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d77f8d4830 0x55d77f8e03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:17.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.014+0000 7fb22e6f8b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d77f8d9e60 0x55d77f8dd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:17.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.014+0000 7fb22e6f8b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d77f8da780 0x55d77f8ddd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:17.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.014+0000 7fb22e6f8b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d77f728f50 con 0x55d77f8da780 2024-05-21T21:54:17.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.014+0000 7fb22e6f8b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55d77f7c3710 con 0x55d77f8d4830 2024-05-21T21:54:17.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.014+0000 7fb22e6f8b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55d77f8e99a0 con 0x55d77f8d9e60 2024-05-21T21:54:17.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.015+0000 7fb22e6f8b80 10 client.0 _ll_register_callbacks cb 0x55d77f7ccc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:54:17.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.015+0000 7fb220a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d77f8d4830 0x55d77f8e03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:17.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.015+0000 7fb221400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d77f8d9e60 0x55d77f8dd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:17.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.015+0000 7fb220a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d77f8d4830 0x55d77f8e03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:55002/0 (socket says 172.21.15.151:55002) 2024-05-21T21:54:17.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.015+0000 7fb220a00700 1 -- 172.21.15.151:0/1425706512 learned_addr learned my addr 172.21.15.151:0/1425706512 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:54:17.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.015+0000 7fb220a00700 1 --2- 172.21.15.151:0/1425706512 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d77f8da780 0x55d77f8ddd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:17.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.015+0000 7fb221400700 1 -- 172.21.15.151:0/1425706512 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d77f8d4830 msgr2=0x55d77f8e03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:54:17.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.015+0000 7fb221400700 1 --2- 172.21.15.151:0/1425706512 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d77f8d4830 0x55d77f8e03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:17.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.015+0000 7fb221400700 1 -- 172.21.15.151:0/1425706512 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d77f8da780 msgr2=0x55d77f8ddd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:54:17.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.015+0000 7fb221400700 1 --2- 172.21.15.151:0/1425706512 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d77f8da780 0x55d77f8ddd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:17.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.015+0000 7fb221400700 1 -- 172.21.15.151:0/1425706512 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d77f772880 con 0x55d77f8d9e60 2024-05-21T21:54:17.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.016+0000 7fb221400700 1 --2- 172.21.15.151:0/1425706512 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d77f8d9e60 0x55d77f8dd7e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb20c00c590 tx=0x7fb20c009510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7623c9c876d0828 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:54:17.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.016+0000 7fb207400700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-21T21:54:17.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.016+0000 7fb207400700 1 -- 172.21.15.151:0/1425706512 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb20c0075c0 con 0x55d77f8d9e60 2024-05-21T21:54:17.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.016+0000 7fb207400700 1 -- 172.21.15.151:0/1425706512 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fb20c007760 con 0x55d77f8d9e60 2024-05-21T21:54:17.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.016+0000 7fb22e6f8b80 1 -- 172.21.15.151:0/1425706512 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb20c01a070 con 0x55d77f8d9e60 2024-05-21T21:54:17.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.016+0000 7fb207400700 1 -- 172.21.15.151:0/1425706512 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb20c007960 con 0x55d77f8d9e60 2024-05-21T21:54:17.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.016+0000 7fb22e6f8b80 10 client.24889 fetch_fsmap learned FSMap version 28 2024-05-21T21:54:17.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.016+0000 7fb22e6f8b80 1 -- 172.21.15.151:0/1425706512 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d77f7f3010 con 0x55d77f8d9e60 2024-05-21T21:54:17.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.017+0000 7fb207400700 1 -- 172.21.15.151:0/1425706512 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb20c01a070 con 0x55d77f8d9e60 2024-05-21T21:54:17.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.017+0000 7fb207400700 1 -- 172.21.15.151:0/1425706512 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fb20c004720 con 0x55d77f8d9e60 2024-05-21T21:54:17.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.017+0000 7fb22e6f8b80 10 client.24889 fetch_fsmap finished waiting for FSMap version 28 2024-05-21T21:54:17.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.017+0000 7fb22e6f8b80 10 client.24889 resolve_mds: resolved name 'e' to daemon mds.e 2024-05-21T21:54:17.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.017+0000 7fb22e6f8b80 20 client.24889 populate_metadata read hostname 'smithi151' 2024-05-21T21:54:17.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.017+0000 7fb22e6f8b80 1 --2- 172.21.15.151:0/1425706512 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x55d77f8f6230 0x55d77f8f76a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:17.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.017+0000 7fb22e6f8b80 4 client.24889 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T21:54:17.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.017+0000 7fb22e6f8b80 1 -- 172.21.15.151:0/1425706512 --> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d77f834d40 con 0x55d77f8f6230 2024-05-21T21:54:17.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.017+0000 7fb220a00700 1 --2- 172.21.15.151:0/1425706512 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x55d77f8f6230 0x55d77f8f76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:17.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.018+0000 7fb220a00700 1 --2- 172.21.15.151:0/1425706512 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x55d77f8f6230 0x55d77f8f76a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a4b66c37f61ea655 server_cookie=e90e1791942605c0 in_seq=0 out_seq=0 2024-05-21T21:54:17.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.018+0000 7fb207400700 10 client.24889 ms_handle_connect on v2:172.21.15.164:6834/2340428470 2024-05-21T21:54:17.023 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.019+0000 7fb207400700 1 -- 172.21.15.151:0/1425706512 <== mds.4 v2:172.21.15.164:6834/2340428470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55d77f834d40 con 0x55d77f8f6230 2024-05-21T21:54:17.023 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.019+0000 7fb207400700 10 client.24889 handle_command_reply: tid=0 2024-05-21T21:54:17.023 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.019+0000 7fb22e6f8b80 1 client.24889 shutdown 2024-05-21T21:54:17.023 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.019+0000 7fb22e6f8b80 1 -- 172.21.15.151:0/1425706512 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d77f8d9e60 msgr2=0x55d77f8dd7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:17.023 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.019+0000 7fb22e6f8b80 1 --2- 172.21.15.151:0/1425706512 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d77f8d9e60 0x55d77f8dd7e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb20c00c590 tx=0x7fb20c009510 comp rx=0 tx=0).stop 2024-05-21T21:54:17.023 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.019+0000 7fb22e6f8b80 1 -- 172.21.15.151:0/1425706512 shutdown_connections 2024-05-21T21:54:17.023 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.019+0000 7fb22e6f8b80 1 --2- 172.21.15.151:0/1425706512 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x55d77f8f6230 0x55d77f8f76a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:17.023 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.019+0000 7fb22e6f8b80 1 --2- 172.21.15.151:0/1425706512 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d77f8d4830 0x55d77f8e03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:17.023 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.019+0000 7fb22e6f8b80 1 --2- 172.21.15.151:0/1425706512 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d77f8da780 0x55d77f8ddd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:17.023 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.019+0000 7fb22e6f8b80 1 --2- 172.21.15.151:0/1425706512 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d77f8d9e60 0x55d77f8dd7e0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:17.023 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.019+0000 7fb22e6f8b80 1 -- 172.21.15.151:0/1425706512 >> 172.21.15.151:0/1425706512 conn(0x55d77f802b90 msgr2=0x55d77f8d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:54:17.023 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.019+0000 7fb207400700 0 client.24889 ms_handle_reset on v2:172.21.15.164:6834/2340428470 2024-05-21T21:54:17.023 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.019+0000 7fb22e6f8b80 1 -- 172.21.15.151:0/1425706512 shutdown_connections 2024-05-21T21:54:17.024 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.019+0000 7fb22e6f8b80 1 -- 172.21.15.151:0/1425706512 wait complete. 2024-05-21T21:54:17.024 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.020+0000 7fb22e6f8b80 20 client.24889 trim_cache size 0 max 16384 2024-05-21T21:54:17.044 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.043+0000 7fb22e6f8b80 1 Processor -- start 2024-05-21T21:54:17.044 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.043+0000 7fb22e6f8b80 1 -- start start 2024-05-21T21:54:17.044 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.043+0000 7fb22e6f8b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d77f8d4830 0x55d77f8d4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:17.044 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.043+0000 7fb22e6f8b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d77f8d9e60 0x55d77f8d06b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:17.044 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.043+0000 7fb22e6f8b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d77f8da780 0x55d77f8dc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:17.044 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.043+0000 7fb22e6f8b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d77f8f59e0 con 0x55d77f8d9e60 2024-05-21T21:54:17.044 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.043+0000 7fb22e6f8b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55d77f8f60b0 con 0x55d77f8da780 2024-05-21T21:54:17.044 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.043+0000 7fb22e6f8b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55d77f8f6780 con 0x55d77f8d4830 2024-05-21T21:54:17.046 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.043+0000 7fb220a00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d77f8d4830 0x55d77f8d4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:17.046 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.044+0000 7fb220a00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d77f8d4830 0x55d77f8d4210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:60454/0 (socket says 172.21.15.151:60454) 2024-05-21T21:54:17.046 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.044+0000 7fb220a00700 1 -- 172.21.15.151:0/3863889561 learned_addr learned my addr 172.21.15.151:0/3863889561 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:54:17.046 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.044+0000 7fb221400700 1 --2- 172.21.15.151:0/3863889561 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d77f8d9e60 0x55d77f8d06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:17.046 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.044+0000 7fb220a00700 1 --2- 172.21.15.151:0/3863889561 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d77f8da780 0x55d77f8dc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:17.046 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.044+0000 7fb221400700 1 -- 172.21.15.151:0/3863889561 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d77f8d4830 msgr2=0x55d77f8d4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:54:17.046 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.044+0000 7fb221400700 1 --2- 172.21.15.151:0/3863889561 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d77f8d4830 0x55d77f8d4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:17.046 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.044+0000 7fb221400700 1 -- 172.21.15.151:0/3863889561 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d77f8da780 msgr2=0x55d77f8dc0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:54:17.046 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.044+0000 7fb221400700 1 --2- 172.21.15.151:0/3863889561 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d77f8da780 0x55d77f8dc0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:17.046 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.044+0000 7fb221400700 1 -- 172.21.15.151:0/3863889561 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d77f8f8600 con 0x55d77f8d9e60 2024-05-21T21:54:17.046 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.044+0000 7fb221400700 1 --2- 172.21.15.151:0/3863889561 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d77f8d9e60 0x55d77f8d06b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb20c00cb80 tx=0x7fb20c006260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7152eb2b3050b7ea server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:54:17.047 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.045+0000 7fb212000700 1 -- 172.21.15.151:0/3863889561 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb20c0077c0 con 0x55d77f8d9e60 2024-05-21T21:54:17.047 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.045+0000 7fb212000700 1 -- 172.21.15.151:0/3863889561 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb20c01a070 con 0x55d77f8d9e60 2024-05-21T21:54:17.047 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.045+0000 7fb212000700 1 -- 172.21.15.151:0/3863889561 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb20c02ae30 con 0x55d77f8d9e60 2024-05-21T21:54:17.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.045+0000 7fb22e6f8b80 1 -- 172.21.15.151:0/3863889561 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d77f8d9e60 msgr2=0x55d77f8d06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:17.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.045+0000 7fb22e6f8b80 1 --2- 172.21.15.151:0/3863889561 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d77f8d9e60 0x55d77f8d06b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb20c00cb80 tx=0x7fb20c006260 comp rx=0 tx=0).stop 2024-05-21T21:54:17.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.045+0000 7fb22e6f8b80 1 -- 172.21.15.151:0/3863889561 shutdown_connections 2024-05-21T21:54:17.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.045+0000 7fb22e6f8b80 1 --2- 172.21.15.151:0/3863889561 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d77f8da780 0x55d77f8dc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:17.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.045+0000 7fb22e6f8b80 1 --2- 172.21.15.151:0/3863889561 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d77f8d4830 0x55d77f8d4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:17.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.045+0000 7fb22e6f8b80 1 --2- 172.21.15.151:0/3863889561 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d77f8d9e60 0x55d77f8d06b0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:17.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.046+0000 7fb22e6f8b80 1 -- 172.21.15.151:0/3863889561 >> 172.21.15.151:0/3863889561 conn(0x55d77f802b90 msgr2=0x55d77f8f5250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:54:17.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.046+0000 7fb22e6f8b80 1 -- 172.21.15.151:0/3863889561 shutdown_connections 2024-05-21T21:54:17.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.046+0000 7fb22e6f8b80 1 -- 172.21.15.151:0/3863889561 wait complete. 2024-05-21T21:54:17.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.046+0000 7fb22e6f8b80 1 -- start start 2024-05-21T21:54:17.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.046+0000 7fb22e6f8b80 1 Processor -- start 2024-05-21T21:54:17.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.047+0000 7fb22e6f8b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d77f8d4830 0x55d77f8d3620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:17.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.047+0000 7fb22e6f8b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d77f8d9e60 0x55d77f8d3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:17.049 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.047+0000 7fb22e6f8b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d77f8da780 0x55d77f8d40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:17.049 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.047+0000 7fb22e6f8b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d77f8f7520 con 0x55d77f8da780 2024-05-21T21:54:17.049 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.047+0000 7fb22e6f8b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55d77f8f6e50 con 0x55d77f8d4830 2024-05-21T21:54:17.049 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.047+0000 7fb22e6f8b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55d77f8f7bf0 con 0x55d77f8d9e60 2024-05-21T21:54:17.049 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.047+0000 7fb22e6f8b80 10 client.0 _ll_register_callbacks cb 0x55d77f7ccc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:54:17.051 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.047+0000 7fb221400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d77f8d9e60 0x55d77f8d3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:17.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.047+0000 7fb220a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d77f8da780 0x55d77f8d40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:17.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.047+0000 7fb221400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d77f8d9e60 0x55d77f8d3b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:60468/0 (socket says 172.21.15.151:60468) 2024-05-21T21:54:17.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.047+0000 7fb220a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d77f8d4830 0x55d77f8d3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:17.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.047+0000 7fb221400700 1 -- 172.21.15.151:0/3480186054 learned_addr learned my addr 172.21.15.151:0/3480186054 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:54:17.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.047+0000 7fb221400700 1 -- 172.21.15.151:0/3480186054 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d77f8d4830 msgr2=0x55d77f8d3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:54:17.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.047+0000 7fb221400700 1 --2- 172.21.15.151:0/3480186054 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d77f8d4830 0x55d77f8d3620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:17.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.047+0000 7fb221400700 1 -- 172.21.15.151:0/3480186054 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d77f8da780 msgr2=0x55d77f8d40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:54:17.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.047+0000 7fb221400700 1 --2- 172.21.15.151:0/3480186054 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d77f8da780 0x55d77f8d40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:17.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.047+0000 7fb221400700 1 -- 172.21.15.151:0/3480186054 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d77f8ff1f0 con 0x55d77f8d9e60 2024-05-21T21:54:17.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.047+0000 7fb220a00700 1 --2- 172.21.15.151:0/3480186054 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d77f8d4830 0x55d77f8d3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:54:17.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.048+0000 7fb221400700 1 --2- 172.21.15.151:0/3480186054 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d77f8d9e60 0x55d77f8d3b60 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb20c003d50 tx=0x7fb20c009960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=102bedd927691df7 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:54:17.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.048+0000 7fb207400700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-21T21:54:17.053 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.048+0000 7fb207400700 1 -- 172.21.15.151:0/3480186054 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb20c007d00 con 0x55d77f8d9e60 2024-05-21T21:54:17.053 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.048+0000 7fb22e6f8b80 1 -- 172.21.15.151:0/3480186054 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb20c01a070 con 0x55d77f8d9e60 2024-05-21T21:54:17.053 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.048+0000 7fb207400700 1 -- 172.21.15.151:0/3480186054 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fb20c02a770 con 0x55d77f8d9e60 2024-05-21T21:54:17.053 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.048+0000 7fb207400700 1 -- 172.21.15.151:0/3480186054 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb20c02a990 con 0x55d77f8d9e60 2024-05-21T21:54:17.053 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.048+0000 7fb22e6f8b80 10 client.24895 fetch_fsmap learned FSMap version 28 2024-05-21T21:54:17.053 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.048+0000 7fb22e6f8b80 1 -- 172.21.15.151:0/3480186054 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d77f961d70 con 0x55d77f8d9e60 2024-05-21T21:54:17.053 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.049+0000 7fb207400700 1 -- 172.21.15.151:0/3480186054 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb20c01a070 con 0x55d77f8d9e60 2024-05-21T21:54:17.053 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.049+0000 7fb207400700 1 -- 172.21.15.151:0/3480186054 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fb20c004ce0 con 0x55d77f8d9e60 2024-05-21T21:54:17.053 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.049+0000 7fb22e6f8b80 10 client.24895 fetch_fsmap finished waiting for FSMap version 28 2024-05-21T21:54:17.053 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.049+0000 7fb22e6f8b80 10 client.24895 resolve_mds: resolved name 'e' to daemon mds.e 2024-05-21T21:54:17.053 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.049+0000 7fb22e6f8b80 20 client.24895 populate_metadata read hostname 'smithi151' 2024-05-21T21:54:17.053 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.049+0000 7fb22e6f8b80 1 --2- 172.21.15.151:0/3480186054 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x55d77f9665e0 0x55d77f9652f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:17.053 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.049+0000 7fb22e6f8b80 4 client.24895 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-21T21:54:17.053 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.049+0000 7fb22e6f8b80 1 -- 172.21.15.151:0/3480186054 --> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] -- command(tid 0: {"prefix": "status"}) -- 0x55d77f82e520 con 0x55d77f9665e0 2024-05-21T21:54:17.054 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.049+0000 7fb220a00700 1 --2- 172.21.15.151:0/3480186054 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x55d77f9665e0 0x55d77f9652f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:17.054 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.050+0000 7fb220a00700 1 --2- 172.21.15.151:0/3480186054 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x55d77f9665e0 0x55d77f9652f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=445d3bd278914f12 server_cookie=b1d5718820ce2cea in_seq=0 out_seq=0 2024-05-21T21:54:17.054 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.050+0000 7fb207400700 10 client.24895 ms_handle_connect on v2:172.21.15.164:6834/2340428470 2024-05-21T21:54:17.054 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.050+0000 7fb207400700 1 -- 172.21.15.151:0/3480186054 <== mds.4 v2:172.21.15.164:6834/2340428470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x55d77f82e520 con 0x55d77f9665e0 2024-05-21T21:54:17.054 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.050+0000 7fb207400700 10 client.24895 handle_command_reply: tid=0 2024-05-21T21:54:17.054 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.050+0000 7fb22e6f8b80 1 client.24895 shutdown 2024-05-21T21:54:17.054 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.050+0000 7fb22e6f8b80 1 -- 172.21.15.151:0/3480186054 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d77f8d9e60 msgr2=0x55d77f8d3b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:17.054 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.050+0000 7fb22e6f8b80 1 --2- 172.21.15.151:0/3480186054 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d77f8d9e60 0x55d77f8d3b60 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb20c003d50 tx=0x7fb20c009960 comp rx=0 tx=0).stop 2024-05-21T21:54:17.054 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.050+0000 7fb22e6f8b80 1 -- 172.21.15.151:0/3480186054 shutdown_connections 2024-05-21T21:54:17.054 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.050+0000 7fb22e6f8b80 1 --2- 172.21.15.151:0/3480186054 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x55d77f9665e0 0x55d77f9652f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:17.054 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.050+0000 7fb22e6f8b80 1 --2- 172.21.15.151:0/3480186054 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d77f8d4830 0x55d77f8d3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:17.054 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.050+0000 7fb207400700 0 client.24895 ms_handle_reset on v2:172.21.15.164:6834/2340428470 2024-05-21T21:54:17.054 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.050+0000 7fb22e6f8b80 1 --2- 172.21.15.151:0/3480186054 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d77f8da780 0x55d77f8d40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:17.055 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.050+0000 7fb22e6f8b80 1 --2- 172.21.15.151:0/3480186054 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d77f8d9e60 0x55d77f8d3b60 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:17.055 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.050+0000 7fb22e6f8b80 1 -- 172.21.15.151:0/3480186054 >> 172.21.15.151:0/3480186054 conn(0x55d77f802b90 msgr2=0x55d77f8d06b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:54:17.055 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.050+0000 7fb22e6f8b80 1 -- 172.21.15.151:0/3480186054 shutdown_connections 2024-05-21T21:54:17.055 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.050+0000 7fb22e6f8b80 1 -- 172.21.15.151:0/3480186054 wait complete. 2024-05-21T21:54:17.055 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.051+0000 7fb22e6f8b80 20 client.24895 trim_cache size 0 max 16384 2024-05-21T21:54:17.055 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-05-21T21:54:17.055 INFO:teuthology.orchestra.run.smithi151.stdout: "cluster_fsid": "c8c7a2e6-17ba-11ef-bc9a-c7b262605968", 2024-05-21T21:54:17.055 INFO:teuthology.orchestra.run.smithi151.stdout: "whoami": 4, 2024-05-21T21:54:17.055 INFO:teuthology.orchestra.run.smithi151.stdout: "id": 24473, 2024-05-21T21:54:17.055 INFO:teuthology.orchestra.run.smithi151.stdout: "want_state": "up:active", 2024-05-21T21:54:17.055 INFO:teuthology.orchestra.run.smithi151.stdout: "state": "up:active", 2024-05-21T21:54:17.056 INFO:teuthology.orchestra.run.smithi151.stdout: "fs_name": "cephfs", 2024-05-21T21:54:17.056 INFO:teuthology.orchestra.run.smithi151.stdout: "rank_uptime": 158.71855796, 2024-05-21T21:54:17.056 INFO:teuthology.orchestra.run.smithi151.stdout: "mdsmap_epoch": 28, 2024-05-21T21:54:17.056 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_epoch": 82, 2024-05-21T21:54:17.056 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_epoch_barrier": 80, 2024-05-21T21:54:17.056 INFO:teuthology.orchestra.run.smithi151.stdout: "uptime": 221.83037699299999 2024-05-21T21:54:17.056 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-05-21T21:54:17.057 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.051+0000 7fb223e00700 1 -- 172.21.15.151:0/3339799597 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fb2000b2900 msgr2=0x7fb2000b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:17.057 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.051+0000 7fb223e00700 1 --2- 172.21.15.151:0/3339799597 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fb2000b2900 0x7fb2000b4dc0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb21401a040 tx=0x7fb2140051c0 comp rx=0 tx=0).stop 2024-05-21T21:54:17.057 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.051+0000 7fb223e00700 1 -- 172.21.15.151:0/3339799597 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb21c10ff40 msgr2=0x7fb21c1a0770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:17.057 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.051+0000 7fb223e00700 1 --2- 172.21.15.151:0/3339799597 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb21c10ff40 0x7fb21c1a0770 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb21400e940 tx=0x7fb21400ec50 comp rx=0 tx=0).stop 2024-05-21T21:54:17.057 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.051+0000 7fb223e00700 1 -- 172.21.15.151:0/3339799597 shutdown_connections 2024-05-21T21:54:17.057 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.051+0000 7fb223e00700 1 --2- 172.21.15.151:0/3339799597 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fb2000b2900 0x7fb2000b4dc0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:17.057 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.051+0000 7fb223e00700 1 --2- 172.21.15.151:0/3339799597 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb21c10ff40 0x7fb21c1a0770 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:17.057 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.051+0000 7fb223e00700 1 --2- 172.21.15.151:0/3339799597 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb21c075770 0x7fb21c19bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:17.057 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.051+0000 7fb223e00700 1 --2- 172.21.15.151:0/3339799597 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb21c076090 0x7fb21c19c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:17.057 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.051+0000 7fb223e00700 1 -- 172.21.15.151:0/3339799597 >> 172.21.15.151:0/3339799597 conn(0x7fb21c07a6a0 msgr2=0x7fb21c100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:54:17.057 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.051+0000 7fb223e00700 1 -- 172.21.15.151:0/3339799597 shutdown_connections 2024-05-21T21:54:17.057 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:17.052+0000 7fb223e00700 1 -- 172.21.15.151:0/3339799597 wait complete. 2024-05-21T21:54:17.120 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:54:16 smithi151 ceph-mon[31623]: pgmap v405: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-21T21:54:17.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:54:16 smithi184 ceph-mon[36942]: pgmap v405: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-21T21:54:17.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:54:16 smithi164 ceph-mon[36269]: pgmap v405: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-21T21:54:17.997 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.c status 2024-05-21T21:54:18.437 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:54:19.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:54:18 smithi151 ceph-mon[31623]: pgmap v406: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-21T21:54:19.235 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:54:18 smithi184 ceph-mon[36942]: pgmap v406: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-21T21:54:19.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:54:18 smithi164 ceph-mon[36269]: pgmap v406: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-21T21:54:20.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.477+0000 7ffa1e000700 1 -- 172.21.15.151:0/29908981 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa18076090 msgr2=0x7ffa1810fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:20.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.477+0000 7ffa1e000700 1 --2- 172.21.15.151:0/29908981 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa18076090 0x7ffa1810fa00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ffa04009990 tx=0x7ffa04009ca0 comp rx=0 tx=0).stop 2024-05-21T21:54:20.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.477+0000 7ffa1e000700 1 -- 172.21.15.151:0/29908981 shutdown_connections 2024-05-21T21:54:20.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.477+0000 7ffa1e000700 1 --2- 172.21.15.151:0/29908981 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffa18075770 0x7ffa18075b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:20.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.477+0000 7ffa1e000700 1 --2- 172.21.15.151:0/29908981 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffa1810ff40 0x7ffa18112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:20.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.477+0000 7ffa1e000700 1 --2- 172.21.15.151:0/29908981 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa18076090 0x7ffa1810fa00 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:20.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.477+0000 7ffa1e000700 1 -- 172.21.15.151:0/29908981 >> 172.21.15.151:0/29908981 conn(0x7ffa1807a6a0 msgr2=0x7ffa1807aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:54:20.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.477+0000 7ffa1e000700 1 -- 172.21.15.151:0/29908981 shutdown_connections 2024-05-21T21:54:20.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.477+0000 7ffa1e000700 1 -- 172.21.15.151:0/29908981 wait complete. 2024-05-21T21:54:20.481 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.478+0000 7ffa1e000700 1 Processor -- start 2024-05-21T21:54:20.481 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.478+0000 7ffa1e000700 1 -- start start 2024-05-21T21:54:20.481 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.479+0000 7ffa1e000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffa18075770 0x7ffa18103660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:20.481 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.479+0000 7ffa1e000700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffa18076090 0x7ffa18103ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:20.481 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.479+0000 7ffa1e000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa1810ff40 0x7ffa181040e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:20.481 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.479+0000 7ffa1e000700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ffa18115920 con 0x7ffa1810ff40 2024-05-21T21:54:20.481 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.479+0000 7ffa1e000700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ffa181157a0 con 0x7ffa18075770 2024-05-21T21:54:20.481 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.479+0000 7ffa1e000700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ffa18115aa0 con 0x7ffa18076090 2024-05-21T21:54:20.481 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.479+0000 7ffa17400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffa18075770 0x7ffa18103660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:20.481 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.479+0000 7ffa16a00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffa18076090 0x7ffa18103ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:20.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.479+0000 7ffa17400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffa18075770 0x7ffa18103660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:55030/0 (socket says 172.21.15.151:55030) 2024-05-21T21:54:20.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.479+0000 7ffa17400700 1 -- 172.21.15.151:0/3155565742 learned_addr learned my addr 172.21.15.151:0/3155565742 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:54:20.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.479+0000 7ffa17e00700 1 --2- 172.21.15.151:0/3155565742 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa1810ff40 0x7ffa181040e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:20.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.480+0000 7ffa17400700 1 -- 172.21.15.151:0/3155565742 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffa18076090 msgr2=0x7ffa18103ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:20.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.480+0000 7ffa17400700 1 --2- 172.21.15.151:0/3155565742 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffa18076090 0x7ffa18103ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:20.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.480+0000 7ffa17400700 1 -- 172.21.15.151:0/3155565742 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa1810ff40 msgr2=0x7ffa181040e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:20.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.480+0000 7ffa17400700 1 --2- 172.21.15.151:0/3155565742 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa1810ff40 0x7ffa181040e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:20.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.480+0000 7ffa17400700 1 -- 172.21.15.151:0/3155565742 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa181ac4e0 con 0x7ffa18075770 2024-05-21T21:54:20.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.480+0000 7ffa17e00700 1 --2- 172.21.15.151:0/3155565742 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa1810ff40 0x7ffa181040e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:54:20.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.480+0000 7ffa17400700 1 --2- 172.21.15.151:0/3155565742 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffa18075770 0x7ffa18103660 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ffa0800d980 tx=0x7ffa0800dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:54:20.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.480+0000 7ffa0fe00700 1 -- 172.21.15.151:0/3155565742 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa08016070 con 0x7ffa18075770 2024-05-21T21:54:20.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.480+0000 7ffa0fe00700 1 -- 172.21.15.151:0/3155565742 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa08004830 con 0x7ffa18075770 2024-05-21T21:54:20.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.480+0000 7ffa1e000700 1 -- 172.21.15.151:0/3155565742 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa181ac7d0 con 0x7ffa18075770 2024-05-21T21:54:20.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.480+0000 7ffa0fe00700 1 -- 172.21.15.151:0/3155565742 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa08005540 con 0x7ffa18075770 2024-05-21T21:54:20.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.480+0000 7ffa1e000700 1 -- 172.21.15.151:0/3155565742 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa181acce0 con 0x7ffa18075770 2024-05-21T21:54:20.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.485+0000 7ffa0fe00700 1 -- 172.21.15.151:0/3155565742 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7ffa08004220 con 0x7ffa18075770 2024-05-21T21:54:20.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.485+0000 7ffa289cfb80 1 Processor -- start 2024-05-21T21:54:20.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.485+0000 7ffa289cfb80 1 -- start start 2024-05-21T21:54:20.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.485+0000 7ffa289cfb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x565295a1c5c0 0x565295a1c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:20.495 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.485+0000 7ffa289cfb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565295a1cee0 0x565295a16ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:20.495 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.485+0000 7ffa289cfb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565295a170f0 0x565295a17570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:20.495 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.486+0000 7ffa289cfb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x565295974df0 con 0x565295a1c5c0 2024-05-21T21:54:20.495 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.486+0000 7ffa289cfb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x565295964990 con 0x565295a170f0 2024-05-21T21:54:20.495 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.486+0000 7ffa289cfb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5652959689d0 con 0x565295a1cee0 2024-05-21T21:54:20.495 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.486+0000 7ffa17e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x565295a1c5c0 0x565295a1c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:20.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.486+0000 7ffa17e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x565295a1c5c0 0x565295a1c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:34542/0 (socket says 172.21.15.151:34542) 2024-05-21T21:54:20.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.486+0000 7ffa17e00700 1 -- 172.21.15.151:0/3241087783 learned_addr learned my addr 172.21.15.151:0/3241087783 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:54:20.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.486+0000 7ffa17400700 1 --2- 172.21.15.151:0/3241087783 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565295a1cee0 0x565295a16ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:20.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.486+0000 7ffa16a00700 1 --2- 172.21.15.151:0/3241087783 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565295a170f0 0x565295a17570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:20.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.486+0000 7ffa17e00700 1 -- 172.21.15.151:0/3241087783 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565295a1cee0 msgr2=0x565295a16ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:54:20.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.486+0000 7ffa17e00700 1 --2- 172.21.15.151:0/3241087783 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565295a1cee0 0x565295a16ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:20.497 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.486+0000 7ffa17e00700 1 -- 172.21.15.151:0/3241087783 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565295a170f0 msgr2=0x565295a17570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:54:20.497 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.486+0000 7ffa17e00700 1 --2- 172.21.15.151:0/3241087783 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565295a170f0 0x565295a17570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:20.497 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.486+0000 7ffa17e00700 1 -- 172.21.15.151:0/3241087783 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56529592d3d0 con 0x565295a1c5c0 2024-05-21T21:54:20.497 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.487+0000 7ffa17e00700 1 --2- 172.21.15.151:0/3241087783 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x565295a1c5c0 0x565295a1c9a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ffa10007740 tx=0x7ffa10009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10b5f646719f96ac server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:54:20.497 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.487+0000 7ffa0cc00700 1 -- 172.21.15.151:0/3241087783 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa10019020 con 0x565295a1c5c0 2024-05-21T21:54:20.498 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.487+0000 7ffa0cc00700 1 -- 172.21.15.151:0/3241087783 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa10015d00 con 0x565295a1c5c0 2024-05-21T21:54:20.498 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.487+0000 7ffa0cc00700 1 -- 172.21.15.151:0/3241087783 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa10014d80 con 0x565295a1c5c0 2024-05-21T21:54:20.498 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.487+0000 7ffa0fe00700 1 --2- 172.21.15.151:0/3155565742 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7ff9fc0b28b0 0x7ff9fc0b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:20.498 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.487+0000 7ffa0fe00700 1 -- 172.21.15.151:0/3155565742 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7ffa080e1fa0 con 0x7ffa18075770 2024-05-21T21:54:20.498 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.487+0000 7ffa17400700 1 --2- 172.21.15.151:0/3155565742 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7ff9fc0b28b0 0x7ff9fc0b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:20.498 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.488+0000 7ffa289cfb80 1 -- 172.21.15.151:0/3241087783 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x565295a1c5c0 msgr2=0x565295a1c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:20.499 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.488+0000 7ffa289cfb80 1 --2- 172.21.15.151:0/3241087783 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x565295a1c5c0 0x565295a1c9a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ffa10007740 tx=0x7ffa10009f90 comp rx=0 tx=0).stop 2024-05-21T21:54:20.499 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.488+0000 7ffa289cfb80 1 -- 172.21.15.151:0/3241087783 shutdown_connections 2024-05-21T21:54:20.499 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.488+0000 7ffa289cfb80 1 --2- 172.21.15.151:0/3241087783 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565295a170f0 0x565295a17570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:20.499 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.488+0000 7ffa289cfb80 1 --2- 172.21.15.151:0/3241087783 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x565295a1c5c0 0x565295a1c9a0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:20.499 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.488+0000 7ffa289cfb80 1 --2- 172.21.15.151:0/3241087783 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565295a1cee0 0x565295a16ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:20.500 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.488+0000 7ffa289cfb80 1 -- 172.21.15.151:0/3241087783 >> 172.21.15.151:0/3241087783 conn(0x565295944b90 msgr2=0x565295a11640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:54:20.500 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.488+0000 7ffa289cfb80 1 -- 172.21.15.151:0/3241087783 shutdown_connections 2024-05-21T21:54:20.500 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.488+0000 7ffa289cfb80 1 -- 172.21.15.151:0/3241087783 wait complete. 2024-05-21T21:54:20.500 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.488+0000 7ffa17400700 1 --2- 172.21.15.151:0/3155565742 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7ff9fc0b28b0 0x7ff9fc0b4d70 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ffa08000c00 tx=0x7ffa080e5a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:54:20.500 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.489+0000 7ffa289cfb80 1 -- start start 2024-05-21T21:54:20.500 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.489+0000 7ffa289cfb80 1 Processor -- start 2024-05-21T21:54:20.501 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.489+0000 7ffa289cfb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x565295a170f0 0x565295a22b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:20.501 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.490+0000 7ffa289cfb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565295a1c5c0 0x565295a1ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:20.501 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.490+0000 7ffa289cfb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565295a1cee0 0x565295a20440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:20.501 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.490+0000 7ffa289cfb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56529586af50 con 0x565295a170f0 2024-05-21T21:54:20.501 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.490+0000 7ffa289cfb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x565295905710 con 0x565295a1cee0 2024-05-21T21:54:20.501 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.490+0000 7ffa289cfb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x565295a23990 con 0x565295a1c5c0 2024-05-21T21:54:20.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.490+0000 7ffa289cfb80 10 client.0 _ll_register_callbacks cb 0x56529590ec20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:54:20.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.490+0000 7ffa17e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x565295a170f0 0x565295a22b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:20.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.490+0000 7ffa17e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x565295a170f0 0x565295a22b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:34544/0 (socket says 172.21.15.151:34544) 2024-05-21T21:54:20.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.490+0000 7ffa17e00700 1 -- 172.21.15.151:0/2725768721 learned_addr learned my addr 172.21.15.151:0/2725768721 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:54:20.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.490+0000 7ffa16a00700 1 --2- 172.21.15.151:0/2725768721 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565295a1c5c0 0x565295a1ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:20.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.490+0000 7ffa17e00700 1 --2- 172.21.15.151:0/2725768721 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565295a1cee0 0x565295a20440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:20.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.490+0000 7ffa17e00700 1 -- 172.21.15.151:0/2725768721 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565295a1c5c0 msgr2=0x565295a1ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:54:20.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.490+0000 7ffa17e00700 1 --2- 172.21.15.151:0/2725768721 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565295a1c5c0 0x565295a1ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:20.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.490+0000 7ffa17e00700 1 -- 172.21.15.151:0/2725768721 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565295a1cee0 msgr2=0x565295a20440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:54:20.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.490+0000 7ffa17e00700 1 --2- 172.21.15.151:0/2725768721 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565295a1cee0 0x565295a20440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:20.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.490+0000 7ffa17e00700 1 -- 172.21.15.151:0/2725768721 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652958b4880 con 0x565295a170f0 2024-05-21T21:54:20.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.490+0000 7ffa17e00700 1 --2- 172.21.15.151:0/2725768721 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x565295a170f0 0x565295a22b60 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ffa100096d0 tx=0x7ffa10009640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f33906c47d09cb4 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:54:20.504 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.490+0000 7ffa01600700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-21T21:54:20.504 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.491+0000 7ffa01600700 1 -- 172.21.15.151:0/2725768721 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa10019070 con 0x565295a170f0 2024-05-21T21:54:20.504 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.491+0000 7ffa01600700 1 -- 172.21.15.151:0/2725768721 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7ffa10003a00 con 0x565295a170f0 2024-05-21T21:54:20.504 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.491+0000 7ffa289cfb80 1 -- 172.21.15.151:0/2725768721 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa10015d00 con 0x565295a170f0 2024-05-21T21:54:20.504 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.491+0000 7ffa01600700 1 -- 172.21.15.151:0/2725768721 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ffa10030050 con 0x565295a170f0 2024-05-21T21:54:20.504 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.491+0000 7ffa289cfb80 10 client.14793 fetch_fsmap learned FSMap version 28 2024-05-21T21:54:20.504 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.491+0000 7ffa289cfb80 1 -- 172.21.15.151:0/2725768721 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x565295935010 con 0x565295a170f0 2024-05-21T21:54:20.504 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.491+0000 7ffa01600700 1 -- 172.21.15.151:0/2725768721 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa10015d00 con 0x565295a170f0 2024-05-21T21:54:20.505 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.492+0000 7ffa01600700 1 -- 172.21.15.151:0/2725768721 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7ffa100041a0 con 0x565295a170f0 2024-05-21T21:54:20.505 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.492+0000 7ffa289cfb80 10 client.14793 fetch_fsmap finished waiting for FSMap version 28 2024-05-21T21:54:20.505 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.492+0000 7ffa289cfb80 10 client.14793 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-21T21:54:20.505 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.492+0000 7ffa289cfb80 20 client.14793 populate_metadata read hostname 'smithi151' 2024-05-21T21:54:20.505 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.492+0000 7ffa289cfb80 1 --2- 172.21.15.151:0/2725768721 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x565295a2ff50 0x565295a313c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:20.505 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.492+0000 7ffa289cfb80 4 client.14793 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T21:54:20.505 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.492+0000 7ffa289cfb80 1 -- 172.21.15.151:0/2725768721 --> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565295976d40 con 0x565295a2ff50 2024-05-21T21:54:20.505 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.492+0000 7ffa16a00700 1 --2- 172.21.15.151:0/2725768721 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x565295a2ff50 0x565295a313c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:20.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.493+0000 7ffa16a00700 1 --2- 172.21.15.151:0/2725768721 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x565295a2ff50 0x565295a313c0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6ac362a3d7f6252a server_cookie=f36ae3468f3691b3 in_seq=0 out_seq=0 2024-05-21T21:54:20.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.493+0000 7ffa01600700 10 client.14793 ms_handle_connect on v2:172.21.15.184:6836/2988194888 2024-05-21T21:54:20.510 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.494+0000 7ffa01600700 1 -- 172.21.15.151:0/2725768721 <== mds.3 v2:172.21.15.184:6836/2988194888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x565295976d40 con 0x565295a2ff50 2024-05-21T21:54:20.510 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.494+0000 7ffa01600700 10 client.14793 handle_command_reply: tid=0 2024-05-21T21:54:20.510 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.494+0000 7ffa289cfb80 1 client.14793 shutdown 2024-05-21T21:54:20.510 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.494+0000 7ffa289cfb80 1 -- 172.21.15.151:0/2725768721 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x565295a170f0 msgr2=0x565295a22b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:20.510 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.494+0000 7ffa289cfb80 1 --2- 172.21.15.151:0/2725768721 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x565295a170f0 0x565295a22b60 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ffa100096d0 tx=0x7ffa10009640 comp rx=0 tx=0).stop 2024-05-21T21:54:20.510 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.494+0000 7ffa289cfb80 1 -- 172.21.15.151:0/2725768721 shutdown_connections 2024-05-21T21:54:20.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.494+0000 7ffa289cfb80 1 --2- 172.21.15.151:0/2725768721 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x565295a2ff50 0x565295a313c0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:20.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.494+0000 7ffa289cfb80 1 --2- 172.21.15.151:0/2725768721 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565295a1cee0 0x565295a20440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:20.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.494+0000 7ffa289cfb80 1 --2- 172.21.15.151:0/2725768721 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x565295a170f0 0x565295a22b60 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:20.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.494+0000 7ffa289cfb80 1 --2- 172.21.15.151:0/2725768721 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565295a1c5c0 0x565295a1ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:20.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.494+0000 7ffa289cfb80 1 -- 172.21.15.151:0/2725768721 >> 172.21.15.151:0/2725768721 conn(0x565295944b90 msgr2=0x565295a10d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:54:20.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.494+0000 7ffa01600700 0 client.14793 ms_handle_reset on v2:172.21.15.184:6836/2988194888 2024-05-21T21:54:20.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.494+0000 7ffa289cfb80 1 -- 172.21.15.151:0/2725768721 shutdown_connections 2024-05-21T21:54:20.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.494+0000 7ffa289cfb80 1 -- 172.21.15.151:0/2725768721 wait complete. 2024-05-21T21:54:20.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.494+0000 7ffa289cfb80 20 client.14793 trim_cache size 0 max 16384 2024-05-21T21:54:20.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.518+0000 7ffa289cfb80 1 Processor -- start 2024-05-21T21:54:20.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.518+0000 7ffa289cfb80 1 -- start start 2024-05-21T21:54:20.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.518+0000 7ffa289cfb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565295a170f0 0x565295a16ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:20.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.518+0000 7ffa289cfb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565295a1c5c0 0x565295a2f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:20.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.518+0000 7ffa289cfb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x565295a1cee0 0x565295a114a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:20.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.518+0000 7ffa289cfb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x565295a2f700 con 0x565295a1cee0 2024-05-21T21:54:20.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.518+0000 7ffa289cfb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x565295a2fdd0 con 0x565295a1c5c0 2024-05-21T21:54:20.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.518+0000 7ffa289cfb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x565295a304a0 con 0x565295a170f0 2024-05-21T21:54:20.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.519+0000 7ffa16a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565295a1c5c0 0x565295a2f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:20.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.519+0000 7ffa17e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565295a170f0 0x565295a16ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:20.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.519+0000 7ffa16a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565295a1c5c0 0x565295a2f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:55062/0 (socket says 172.21.15.151:55062) 2024-05-21T21:54:20.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.519+0000 7ffa16a00700 1 -- 172.21.15.151:0/272122044 learned_addr learned my addr 172.21.15.151:0/272122044 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:54:20.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.519+0000 7ffa17e00700 1 --2- 172.21.15.151:0/272122044 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x565295a1cee0 0x565295a114a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:20.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.519+0000 7ffa16a00700 1 -- 172.21.15.151:0/272122044 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565295a170f0 msgr2=0x565295a16ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:54:20.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.519+0000 7ffa16a00700 1 --2- 172.21.15.151:0/272122044 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565295a170f0 0x565295a16ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:20.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.519+0000 7ffa16a00700 1 -- 172.21.15.151:0/272122044 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x565295a1cee0 msgr2=0x565295a114a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:54:20.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.519+0000 7ffa16a00700 1 --2- 172.21.15.151:0/272122044 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x565295a1cee0 0x565295a114a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:20.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.519+0000 7ffa16a00700 1 -- 172.21.15.151:0/272122044 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565295a1c030 con 0x565295a1c5c0 2024-05-21T21:54:20.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.519+0000 7ffa17e00700 1 --2- 172.21.15.151:0/272122044 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x565295a1cee0 0x565295a114a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T21:54:20.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.519+0000 7ffa16a00700 1 --2- 172.21.15.151:0/272122044 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565295a1c5c0 0x565295a2f880 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ffa04005f50 tx=0x7ffa04005960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42be8bdd4ca7cff1 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:54:20.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.520+0000 7ffa03e00700 1 -- 172.21.15.151:0/272122044 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa0400bd10 con 0x565295a1c5c0 2024-05-21T21:54:20.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.520+0000 7ffa03e00700 1 -- 172.21.15.151:0/272122044 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa04003d10 con 0x565295a1c5c0 2024-05-21T21:54:20.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.520+0000 7ffa03e00700 1 -- 172.21.15.151:0/272122044 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa0400f460 con 0x565295a1c5c0 2024-05-21T21:54:20.529 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.520+0000 7ffa289cfb80 1 -- 172.21.15.151:0/272122044 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565295a1c5c0 msgr2=0x565295a2f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:20.529 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.520+0000 7ffa289cfb80 1 --2- 172.21.15.151:0/272122044 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565295a1c5c0 0x565295a2f880 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ffa04005f50 tx=0x7ffa04005960 comp rx=0 tx=0).stop 2024-05-21T21:54:20.529 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.521+0000 7ffa289cfb80 1 -- 172.21.15.151:0/272122044 shutdown_connections 2024-05-21T21:54:20.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.521+0000 7ffa289cfb80 1 --2- 172.21.15.151:0/272122044 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x565295a1cee0 0x565295a114a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:20.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.521+0000 7ffa289cfb80 1 --2- 172.21.15.151:0/272122044 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565295a170f0 0x565295a16ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:20.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.521+0000 7ffa289cfb80 1 --2- 172.21.15.151:0/272122044 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565295a1c5c0 0x565295a2f880 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:20.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.521+0000 7ffa289cfb80 1 -- 172.21.15.151:0/272122044 >> 172.21.15.151:0/272122044 conn(0x565295944b90 msgr2=0x565295a1e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:54:20.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.521+0000 7ffa289cfb80 1 -- 172.21.15.151:0/272122044 shutdown_connections 2024-05-21T21:54:20.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.521+0000 7ffa289cfb80 1 -- 172.21.15.151:0/272122044 wait complete. 2024-05-21T21:54:20.531 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.521+0000 7ffa289cfb80 1 -- start start 2024-05-21T21:54:20.531 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.521+0000 7ffa289cfb80 1 Processor -- start 2024-05-21T21:54:20.531 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.522+0000 7ffa289cfb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x565295a170f0 0x565295a10b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:20.531 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.522+0000 7ffa289cfb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565295a1c5c0 0x565295a110c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:20.531 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.522+0000 7ffa289cfb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565295a1cee0 0x565295a11600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:20.531 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.522+0000 7ffa289cfb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x565295a31240 con 0x565295a170f0 2024-05-21T21:54:20.531 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.522+0000 7ffa289cfb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x565295a30b70 con 0x565295a1c5c0 2024-05-21T21:54:20.532 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.522+0000 7ffa289cfb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x565295a31910 con 0x565295a1cee0 2024-05-21T21:54:20.532 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.522+0000 7ffa289cfb80 10 client.0 _ll_register_callbacks cb 0x56529590ec20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:54:20.532 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.522+0000 7ffa16a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565295a1c5c0 0x565295a110c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:20.532 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.522+0000 7ffa16a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565295a1c5c0 0x565295a110c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:55068/0 (socket says 172.21.15.151:55068) 2024-05-21T21:54:20.532 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.522+0000 7ffa16a00700 1 -- 172.21.15.151:0/4198254089 learned_addr learned my addr 172.21.15.151:0/4198254089 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:54:20.532 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.522+0000 7ffa17e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x565295a170f0 0x565295a10b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:20.532 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.522+0000 7ffa17e00700 1 --2- 172.21.15.151:0/4198254089 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565295a1cee0 0x565295a11600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:20.533 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.522+0000 7ffa16a00700 1 -- 172.21.15.151:0/4198254089 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565295a1cee0 msgr2=0x565295a11600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:54:20.533 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.522+0000 7ffa16a00700 1 --2- 172.21.15.151:0/4198254089 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565295a1cee0 0x565295a11600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:20.533 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.522+0000 7ffa16a00700 1 -- 172.21.15.151:0/4198254089 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x565295a170f0 msgr2=0x565295a10b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:54:20.533 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.523+0000 7ffa16a00700 1 --2- 172.21.15.151:0/4198254089 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x565295a170f0 0x565295a10b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:20.533 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.523+0000 7ffa16a00700 1 -- 172.21.15.151:0/4198254089 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565295a38ec0 con 0x565295a1c5c0 2024-05-21T21:54:20.534 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.523+0000 7ffa16a00700 1 --2- 172.21.15.151:0/4198254089 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565295a1c5c0 0x565295a110c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ffa04009660 tx=0x7ffa04004ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4fbe3a7b64eb7298 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:54:20.534 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.523+0000 7ffa01600700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-21T21:54:20.534 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.523+0000 7ffa01600700 1 -- 172.21.15.151:0/4198254089 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa0403b060 con 0x565295a1c5c0 2024-05-21T21:54:20.534 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.523+0000 7ffa289cfb80 1 -- 172.21.15.151:0/4198254089 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa04003d10 con 0x565295a1c5c0 2024-05-21T21:54:20.534 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.523+0000 7ffa01600700 1 -- 172.21.15.151:0/4198254089 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7ffa04003860 con 0x565295a1c5c0 2024-05-21T21:54:20.534 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.523+0000 7ffa01600700 1 -- 172.21.15.151:0/4198254089 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ffa040056d0 con 0x565295a1c5c0 2024-05-21T21:54:20.535 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.523+0000 7ffa289cfb80 10 client.24923 fetch_fsmap learned FSMap version 28 2024-05-21T21:54:20.535 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.523+0000 7ffa289cfb80 1 -- 172.21.15.151:0/4198254089 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x565295a9baf0 con 0x565295a1c5c0 2024-05-21T21:54:20.535 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.524+0000 7ffa01600700 1 -- 172.21.15.151:0/4198254089 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa04003d10 con 0x565295a1c5c0 2024-05-21T21:54:20.535 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.524+0000 7ffa01600700 1 -- 172.21.15.151:0/4198254089 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7ffa04040070 con 0x565295a1c5c0 2024-05-21T21:54:20.535 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.524+0000 7ffa289cfb80 10 client.24923 fetch_fsmap finished waiting for FSMap version 28 2024-05-21T21:54:20.535 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.524+0000 7ffa289cfb80 10 client.24923 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-21T21:54:20.536 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.524+0000 7ffa289cfb80 20 client.24923 populate_metadata read hostname 'smithi151' 2024-05-21T21:54:20.536 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.524+0000 7ffa289cfb80 1 --2- 172.21.15.151:0/4198254089 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x565295aa00c0 0x565295a9edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:20.536 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.524+0000 7ffa289cfb80 4 client.24923 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-21T21:54:20.536 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.524+0000 7ffa289cfb80 1 -- 172.21.15.151:0/4198254089 --> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] -- command(tid 0: {"prefix": "status"}) -- 0x565295970520 con 0x565295aa00c0 2024-05-21T21:54:20.536 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.524+0000 7ffa17e00700 1 --2- 172.21.15.151:0/4198254089 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x565295aa00c0 0x565295a9edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:20.536 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.525+0000 7ffa17e00700 1 --2- 172.21.15.151:0/4198254089 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x565295aa00c0 0x565295a9edd0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=52be162c304c4830 server_cookie=1993316ab6d665d1 in_seq=0 out_seq=0 2024-05-21T21:54:20.536 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.525+0000 7ffa01600700 10 client.24923 ms_handle_connect on v2:172.21.15.184:6836/2988194888 2024-05-21T21:54:20.536 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.525+0000 7ffa01600700 1 -- 172.21.15.151:0/4198254089 <== mds.3 v2:172.21.15.184:6836/2988194888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x565295970520 con 0x565295aa00c0 2024-05-21T21:54:20.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.525+0000 7ffa01600700 10 client.24923 handle_command_reply: tid=0 2024-05-21T21:54:20.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.525+0000 7ffa289cfb80 1 client.24923 shutdown 2024-05-21T21:54:20.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.525+0000 7ffa289cfb80 1 -- 172.21.15.151:0/4198254089 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565295a1c5c0 msgr2=0x565295a110c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:20.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.525+0000 7ffa289cfb80 1 --2- 172.21.15.151:0/4198254089 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565295a1c5c0 0x565295a110c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ffa04009660 tx=0x7ffa04004ed0 comp rx=0 tx=0).stop 2024-05-21T21:54:20.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.525+0000 7ffa289cfb80 1 -- 172.21.15.151:0/4198254089 shutdown_connections 2024-05-21T21:54:20.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.525+0000 7ffa289cfb80 1 --2- 172.21.15.151:0/4198254089 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x565295aa00c0 0x565295a9edd0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:20.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.525+0000 7ffa289cfb80 1 --2- 172.21.15.151:0/4198254089 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565295a1cee0 0x565295a11600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:20.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.525+0000 7ffa289cfb80 1 --2- 172.21.15.151:0/4198254089 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x565295a170f0 0x565295a10b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:20.538 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.525+0000 7ffa289cfb80 1 --2- 172.21.15.151:0/4198254089 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565295a1c5c0 0x565295a110c0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:20.538 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.525+0000 7ffa289cfb80 1 -- 172.21.15.151:0/4198254089 >> 172.21.15.151:0/4198254089 conn(0x565295944b90 msgr2=0x565295a1d2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:54:20.538 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.525+0000 7ffa01600700 0 client.24923 ms_handle_reset on v2:172.21.15.184:6836/2988194888 2024-05-21T21:54:20.538 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.526+0000 7ffa289cfb80 1 -- 172.21.15.151:0/4198254089 shutdown_connections 2024-05-21T21:54:20.538 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.526+0000 7ffa289cfb80 1 -- 172.21.15.151:0/4198254089 wait complete. 2024-05-21T21:54:20.538 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.526+0000 7ffa289cfb80 20 client.24923 trim_cache size 0 max 16384 2024-05-21T21:54:20.539 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-05-21T21:54:20.539 INFO:teuthology.orchestra.run.smithi151.stdout: "cluster_fsid": "c8c7a2e6-17ba-11ef-bc9a-c7b262605968", 2024-05-21T21:54:20.539 INFO:teuthology.orchestra.run.smithi151.stdout: "whoami": 3, 2024-05-21T21:54:20.539 INFO:teuthology.orchestra.run.smithi151.stdout: "id": 24478, 2024-05-21T21:54:20.539 INFO:teuthology.orchestra.run.smithi151.stdout: "want_state": "up:active", 2024-05-21T21:54:20.540 INFO:teuthology.orchestra.run.smithi151.stdout: "state": "up:active", 2024-05-21T21:54:20.540 INFO:teuthology.orchestra.run.smithi151.stdout: "fs_name": "cephfs", 2024-05-21T21:54:20.540 INFO:teuthology.orchestra.run.smithi151.stdout: "rank_uptime": 163.20354735199999, 2024-05-21T21:54:20.540 INFO:teuthology.orchestra.run.smithi151.stdout: "mdsmap_epoch": 28, 2024-05-21T21:54:20.540 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_epoch": 82, 2024-05-21T21:54:20.540 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_epoch_barrier": 80, 2024-05-21T21:54:20.540 INFO:teuthology.orchestra.run.smithi151.stdout: "uptime": 219.40187023199999 2024-05-21T21:54:20.541 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-05-21T21:54:20.542 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.526+0000 7ffa1e000700 1 -- 172.21.15.151:0/3155565742 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7ff9fc0b28b0 msgr2=0x7ff9fc0b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:20.542 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.526+0000 7ffa1e000700 1 --2- 172.21.15.151:0/3155565742 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7ff9fc0b28b0 0x7ff9fc0b4d70 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ffa08000c00 tx=0x7ffa080e5a20 comp rx=0 tx=0).stop 2024-05-21T21:54:20.542 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.526+0000 7ffa1e000700 1 -- 172.21.15.151:0/3155565742 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffa18075770 msgr2=0x7ffa18103660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:20.542 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.526+0000 7ffa1e000700 1 --2- 172.21.15.151:0/3155565742 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffa18075770 0x7ffa18103660 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ffa0800d980 tx=0x7ffa0800dc90 comp rx=0 tx=0).stop 2024-05-21T21:54:20.543 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.527+0000 7ffa1e000700 1 -- 172.21.15.151:0/3155565742 shutdown_connections 2024-05-21T21:54:20.543 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.527+0000 7ffa1e000700 1 --2- 172.21.15.151:0/3155565742 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7ff9fc0b28b0 0x7ff9fc0b4d70 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:20.543 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.527+0000 7ffa1e000700 1 --2- 172.21.15.151:0/3155565742 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffa18075770 0x7ffa18103660 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:20.543 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.527+0000 7ffa1e000700 1 --2- 172.21.15.151:0/3155565742 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa1810ff40 0x7ffa181040e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:20.543 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.527+0000 7ffa1e000700 1 --2- 172.21.15.151:0/3155565742 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffa18076090 0x7ffa18103ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:20.543 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.527+0000 7ffa1e000700 1 -- 172.21.15.151:0/3155565742 >> 172.21.15.151:0/3155565742 conn(0x7ffa1807a6a0 msgr2=0x7ffa181007f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:54:20.543 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.527+0000 7ffa1e000700 1 -- 172.21.15.151:0/3155565742 shutdown_connections 2024-05-21T21:54:20.544 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:20.527+0000 7ffa1e000700 1 -- 172.21.15.151:0/3155565742 wait complete. 2024-05-21T21:54:21.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:54:20 smithi151 ceph-mon[31623]: pgmap v407: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-21T21:54:21.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:54:20 smithi184 ceph-mon[36942]: pgmap v407: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-21T21:54:21.269 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.h status 2024-05-21T21:54:21.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:54:20 smithi164 ceph-mon[36269]: pgmap v407: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-21T21:54:21.724 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:54:22.167 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:54:21 smithi151 ceph-mon[31623]: pgmap v408: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-21T21:54:22.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:54:21 smithi184 ceph-mon[36942]: pgmap v408: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-21T21:54:22.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:54:21 smithi164 ceph-mon[36269]: pgmap v408: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-21T21:54:23.831 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.830+0000 7f15c8a00700 1 -- 172.21.15.151:0/3834245068 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f15c406b250 msgr2=0x7f15c406b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:23.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.830+0000 7f15c8a00700 1 --2- 172.21.15.151:0/3834245068 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f15c406b250 0x7f15c406b630 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f15b400b0a0 tx=0x7f15b400b3b0 comp rx=0 tx=0).stop 2024-05-21T21:54:23.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.830+0000 7f15c8a00700 1 -- 172.21.15.151:0/3834245068 shutdown_connections 2024-05-21T21:54:23.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.830+0000 7f15c8a00700 1 --2- 172.21.15.151:0/3834245068 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f15c410fff0 0x7f15c41123e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:23.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.830+0000 7f15c8a00700 1 --2- 172.21.15.151:0/3834245068 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f15c406b250 0x7f15c406b630 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:23.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.830+0000 7f15c8a00700 1 --2- 172.21.15.151:0/3834245068 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f15c406bb70 0x7f15c410fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:23.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.830+0000 7f15c8a00700 1 -- 172.21.15.151:0/3834245068 >> 172.21.15.151:0/3834245068 conn(0x7f15c40793f0 msgr2=0x7f15c4079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:54:23.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.830+0000 7f15c8a00700 1 -- 172.21.15.151:0/3834245068 shutdown_connections 2024-05-21T21:54:23.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.830+0000 7f15c8a00700 1 -- 172.21.15.151:0/3834245068 wait complete. 2024-05-21T21:54:23.833 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.831+0000 7f15c8a00700 1 Processor -- start 2024-05-21T21:54:23.833 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.831+0000 7f15c8a00700 1 -- start start 2024-05-21T21:54:23.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.832+0000 7f15c8a00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f15c406b250 0x7f15c419be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:23.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.832+0000 7f15c8a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f15c406bb70 0x7f15c419c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:23.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.832+0000 7f15c8a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f15c410fff0 0x7f15c41a0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:23.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.832+0000 7f15c8a00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f15c4115950 con 0x7f15c406bb70 2024-05-21T21:54:23.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.832+0000 7f15c8a00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f15c41157d0 con 0x7f15c410fff0 2024-05-21T21:54:23.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.832+0000 7f15c8a00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f15c4115ad0 con 0x7f15c406b250 2024-05-21T21:54:23.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.832+0000 7f15b9400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f15c406bb70 0x7f15c419c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:23.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.832+0000 7f15b9400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f15c406bb70 0x7f15c419c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:34584/0 (socket says 172.21.15.151:34584) 2024-05-21T21:54:23.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.832+0000 7f15b9400700 1 -- 172.21.15.151:0/3536427224 learned_addr learned my addr 172.21.15.151:0/3536427224 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:54:23.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.832+0000 7f15c1e00700 1 --2- 172.21.15.151:0/3536427224 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f15c406b250 0x7f15c419be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:23.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.832+0000 7f15c2800700 1 --2- 172.21.15.151:0/3536427224 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f15c410fff0 0x7f15c41a0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:23.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.832+0000 7f15b9400700 1 -- 172.21.15.151:0/3536427224 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f15c406b250 msgr2=0x7f15c419be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:23.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.832+0000 7f15b9400700 1 --2- 172.21.15.151:0/3536427224 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f15c406b250 0x7f15c419be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:23.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.832+0000 7f15b9400700 1 -- 172.21.15.151:0/3536427224 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f15c410fff0 msgr2=0x7f15c41a0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:23.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.832+0000 7f15b9400700 1 --2- 172.21.15.151:0/3536427224 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f15c410fff0 0x7f15c41a0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:23.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.833+0000 7f15b9400700 1 -- 172.21.15.151:0/3536427224 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15c41a0e70 con 0x7f15c406bb70 2024-05-21T21:54:23.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.833+0000 7f15b9400700 1 --2- 172.21.15.151:0/3536427224 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f15c406bb70 0x7f15c419c3a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f15ac00eb30 tx=0x7f15ac00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:54:23.836 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.833+0000 7f15bb400700 1 -- 172.21.15.151:0/3536427224 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15ac00cbd0 con 0x7f15c406bb70 2024-05-21T21:54:23.836 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.833+0000 7f15bb400700 1 -- 172.21.15.151:0/3536427224 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15ac0044d0 con 0x7f15c406bb70 2024-05-21T21:54:23.836 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.833+0000 7f15c8a00700 1 -- 172.21.15.151:0/3536427224 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15c41a1160 con 0x7f15c406bb70 2024-05-21T21:54:23.836 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.833+0000 7f15bb400700 1 -- 172.21.15.151:0/3536427224 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15ac003df0 con 0x7f15c406bb70 2024-05-21T21:54:23.836 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.833+0000 7f15c8a00700 1 -- 172.21.15.151:0/3536427224 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15c41a89e0 con 0x7f15c406bb70 2024-05-21T21:54:23.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.837+0000 7f15bb400700 1 -- 172.21.15.151:0/3536427224 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f15ac004000 con 0x7f15c406bb70 2024-05-21T21:54:23.839 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.838+0000 7f15d332fb80 1 Processor -- start 2024-05-21T21:54:23.839 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.838+0000 7f15d332fb80 1 -- start start 2024-05-21T21:54:23.840 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.839+0000 7f15d332fb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e9c35255c0 0x55e9c35259a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:23.840 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.839+0000 7f15d332fb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e9c3525ee0 0x55e9c351fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:23.840 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.839+0000 7f15d332fb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e9c35200f0 0x55e9c3520570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:23.840 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.839+0000 7f15d332fb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e9c347ddf0 con 0x55e9c3525ee0 2024-05-21T21:54:23.840 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.839+0000 7f15d332fb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55e9c346d990 con 0x55e9c35255c0 2024-05-21T21:54:23.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.839+0000 7f15d332fb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55e9c34719d0 con 0x55e9c35200f0 2024-05-21T21:54:23.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.839+0000 7f15bb400700 1 --2- 172.21.15.151:0/3536427224 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f15a80b2880 0x7f15a80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:23.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.839+0000 7f15c1e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e9c3525ee0 0x55e9c351fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:23.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.840+0000 7f15c1e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e9c3525ee0 0x55e9c351fad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:34586/0 (socket says 172.21.15.151:34586) 2024-05-21T21:54:23.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.839+0000 7f15c2800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e9c35255c0 0x55e9c35259a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:23.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.840+0000 7f15c1e00700 1 -- 172.21.15.151:0/3916765076 learned_addr learned my addr 172.21.15.151:0/3916765076 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:54:23.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.840+0000 7f15b9400700 1 --2- 172.21.15.151:0/3916765076 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e9c35200f0 0x55e9c3520570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:23.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.840+0000 7f15bb400700 1 -- 172.21.15.151:0/3536427224 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f15ac0e2520 con 0x7f15c406bb70 2024-05-21T21:54:23.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.840+0000 7f15c2800700 1 --2- 172.21.15.151:0/3536427224 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f15a80b2880 0x7f15a80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:23.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.840+0000 7f15c1e00700 1 -- 172.21.15.151:0/3916765076 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e9c35200f0 msgr2=0x55e9c3520570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:54:23.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.840+0000 7f15c1e00700 1 --2- 172.21.15.151:0/3916765076 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e9c35200f0 0x55e9c3520570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:23.843 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.840+0000 7f15c1e00700 1 -- 172.21.15.151:0/3916765076 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e9c35255c0 msgr2=0x55e9c35259a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:54:23.843 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.840+0000 7f15c1e00700 1 --2- 172.21.15.151:0/3916765076 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e9c35255c0 0x55e9c35259a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:23.843 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.840+0000 7f15c1e00700 1 -- 172.21.15.151:0/3916765076 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9c34363d0 con 0x55e9c3525ee0 2024-05-21T21:54:23.843 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.840+0000 7f15c1e00700 1 --2- 172.21.15.151:0/3916765076 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e9c3525ee0 0x55e9c351fad0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f15b4000f80 tx=0x7f15b4009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff2b7053fd0d4c28 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:54:23.843 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.841+0000 7f15b3400700 1 -- 172.21.15.151:0/3916765076 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b400bcb0 con 0x55e9c3525ee0 2024-05-21T21:54:23.843 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.841+0000 7f15b3400700 1 -- 172.21.15.151:0/3916765076 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15b40074e0 con 0x55e9c3525ee0 2024-05-21T21:54:23.843 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.841+0000 7f15b3400700 1 -- 172.21.15.151:0/3916765076 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b4005240 con 0x55e9c3525ee0 2024-05-21T21:54:23.843 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.841+0000 7f15c2800700 1 --2- 172.21.15.151:0/3536427224 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f15a80b2880 0x7f15a80b4d40 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f15c406ae10 tx=0x7f15bc0047c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:54:23.844 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.842+0000 7f15d332fb80 1 -- 172.21.15.151:0/3916765076 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e9c3525ee0 msgr2=0x55e9c351fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:23.844 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.842+0000 7f15d332fb80 1 --2- 172.21.15.151:0/3916765076 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e9c3525ee0 0x55e9c351fad0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f15b4000f80 tx=0x7f15b4009360 comp rx=0 tx=0).stop 2024-05-21T21:54:23.844 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.842+0000 7f15d332fb80 1 -- 172.21.15.151:0/3916765076 shutdown_connections 2024-05-21T21:54:23.844 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.842+0000 7f15d332fb80 1 --2- 172.21.15.151:0/3916765076 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e9c35255c0 0x55e9c35259a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:23.844 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.842+0000 7f15d332fb80 1 --2- 172.21.15.151:0/3916765076 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e9c35200f0 0x55e9c3520570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:23.844 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.842+0000 7f15d332fb80 1 --2- 172.21.15.151:0/3916765076 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e9c3525ee0 0x55e9c351fad0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:23.844 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.842+0000 7f15d332fb80 1 -- 172.21.15.151:0/3916765076 >> 172.21.15.151:0/3916765076 conn(0x55e9c344db90 msgr2=0x55e9c351a640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:54:23.844 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.842+0000 7f15d332fb80 1 -- 172.21.15.151:0/3916765076 shutdown_connections 2024-05-21T21:54:23.844 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.842+0000 7f15d332fb80 1 -- 172.21.15.151:0/3916765076 wait complete. 2024-05-21T21:54:23.844 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.843+0000 7f15d332fb80 1 -- start start 2024-05-21T21:54:23.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.843+0000 7f15d332fb80 1 Processor -- start 2024-05-21T21:54:23.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.844+0000 7f15d332fb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e9c35200f0 0x55e9c352bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:23.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.844+0000 7f15d332fb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e9c35255c0 0x55e9c3528ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:23.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.844+0000 7f15d332fb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e9c3525ee0 0x55e9c3529410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:23.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.844+0000 7f15d332fb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e9c3373f50 con 0x55e9c35200f0 2024-05-21T21:54:23.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.844+0000 7f15d332fb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55e9c340e710 con 0x55e9c35255c0 2024-05-21T21:54:23.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.844+0000 7f15d332fb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55e9c352c930 con 0x55e9c3525ee0 2024-05-21T21:54:23.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.844+0000 7f15d332fb80 10 client.0 _ll_register_callbacks cb 0x55e9c3417c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:54:23.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.844+0000 7f15c1e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e9c35200f0 0x55e9c352bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:23.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.844+0000 7f15c1e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e9c35200f0 0x55e9c352bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:34600/0 (socket says 172.21.15.151:34600) 2024-05-21T21:54:23.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.844+0000 7f15b9400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e9c35255c0 0x55e9c3528ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:23.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.844+0000 7f15c1e00700 1 -- 172.21.15.151:0/1984267494 learned_addr learned my addr 172.21.15.151:0/1984267494 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:54:23.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.844+0000 7f15c2800700 1 --2- 172.21.15.151:0/1984267494 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e9c3525ee0 0x55e9c3529410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:23.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.845+0000 7f15c1e00700 1 -- 172.21.15.151:0/1984267494 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e9c3525ee0 msgr2=0x55e9c3529410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:54:23.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.845+0000 7f15c1e00700 1 --2- 172.21.15.151:0/1984267494 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e9c3525ee0 0x55e9c3529410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:23.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.845+0000 7f15c1e00700 1 -- 172.21.15.151:0/1984267494 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e9c35255c0 msgr2=0x55e9c3528ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:54:23.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.845+0000 7f15c1e00700 1 --2- 172.21.15.151:0/1984267494 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e9c35255c0 0x55e9c3528ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:23.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.845+0000 7f15c1e00700 1 -- 172.21.15.151:0/1984267494 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9c33bd880 con 0x55e9c35200f0 2024-05-21T21:54:23.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.845+0000 7f15c2800700 1 --2- 172.21.15.151:0/1984267494 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e9c3525ee0 0x55e9c3529410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T21:54:23.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.845+0000 7f15b9400700 1 --2- 172.21.15.151:0/1984267494 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e9c35255c0 0x55e9c3528ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:54:23.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.845+0000 7f15c1e00700 1 --2- 172.21.15.151:0/1984267494 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e9c35200f0 0x55e9c352bb30 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f15b400b780 tx=0x7f15b4009550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0298b962e72dd3f server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:54:23.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.845+0000 7f15a7e00700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-21T21:54:23.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.845+0000 7f15a7e00700 1 -- 172.21.15.151:0/1984267494 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b402a070 con 0x55e9c35200f0 2024-05-21T21:54:23.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.845+0000 7f15a7e00700 1 -- 172.21.15.151:0/1984267494 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f15b4003dd0 con 0x55e9c35200f0 2024-05-21T21:54:23.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.845+0000 7f15d332fb80 1 -- 172.21.15.151:0/1984267494 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15b40074e0 con 0x55e9c35200f0 2024-05-21T21:54:23.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.846+0000 7f15a7e00700 1 -- 172.21.15.151:0/1984267494 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f15b4007690 con 0x55e9c35200f0 2024-05-21T21:54:23.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.846+0000 7f15d332fb80 10 client.14811 fetch_fsmap learned FSMap version 28 2024-05-21T21:54:23.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.846+0000 7f15d332fb80 1 -- 172.21.15.151:0/1984267494 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e9c343e010 con 0x55e9c35200f0 2024-05-21T21:54:23.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.846+0000 7f15a7e00700 1 -- 172.21.15.151:0/1984267494 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15b40074e0 con 0x55e9c35200f0 2024-05-21T21:54:23.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.846+0000 7f15a7e00700 1 -- 172.21.15.151:0/1984267494 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f15b4011070 con 0x55e9c35200f0 2024-05-21T21:54:23.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.846+0000 7f15d332fb80 10 client.14811 fetch_fsmap finished waiting for FSMap version 28 2024-05-21T21:54:23.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.846+0000 7f15d332fb80 10 client.14811 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-21T21:54:23.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.846+0000 7f15d332fb80 20 client.14811 populate_metadata read hostname 'smithi151' 2024-05-21T21:54:23.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.846+0000 7f15d332fb80 1 --2- 172.21.15.151:0/1984267494 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x55e9c35380e0 0x55e9c353a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:23.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.846+0000 7f15d332fb80 4 client.14811 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T21:54:23.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.846+0000 7f15d332fb80 1 -- 172.21.15.151:0/1984267494 --> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e9c347fd40 con 0x55e9c35380e0 2024-05-21T21:54:23.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.847+0000 7f15b9400700 1 --2- 172.21.15.151:0/1984267494 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x55e9c35380e0 0x55e9c353a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:23.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.847+0000 7f15b9400700 1 --2- 172.21.15.151:0/1984267494 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x55e9c35380e0 0x55e9c353a290 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=77f8f3bf12b2f584 server_cookie=31c0fd93a38c7d78 in_seq=0 out_seq=0 2024-05-21T21:54:23.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.847+0000 7f15a7e00700 10 client.14811 ms_handle_connect on v2:172.21.15.164:6836/1554609572 2024-05-21T21:54:23.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.848+0000 7f15a7e00700 1 -- 172.21.15.151:0/1984267494 <== mds.2 v2:172.21.15.164:6836/1554609572 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55e9c347fd40 con 0x55e9c35380e0 2024-05-21T21:54:23.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.848+0000 7f15a7e00700 10 client.14811 handle_command_reply: tid=0 2024-05-21T21:54:23.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.848+0000 7f15d332fb80 1 client.14811 shutdown 2024-05-21T21:54:23.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.849+0000 7f15d332fb80 1 -- 172.21.15.151:0/1984267494 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e9c35200f0 msgr2=0x55e9c352bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:23.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.849+0000 7f15d332fb80 1 --2- 172.21.15.151:0/1984267494 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e9c35200f0 0x55e9c352bb30 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f15b400b780 tx=0x7f15b4009550 comp rx=0 tx=0).stop 2024-05-21T21:54:23.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.849+0000 7f15d332fb80 1 -- 172.21.15.151:0/1984267494 shutdown_connections 2024-05-21T21:54:23.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.849+0000 7f15d332fb80 1 --2- 172.21.15.151:0/1984267494 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x55e9c35380e0 0x55e9c353a290 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:23.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.849+0000 7f15d332fb80 1 --2- 172.21.15.151:0/1984267494 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e9c3525ee0 0x55e9c3529410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:23.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.849+0000 7f15d332fb80 1 --2- 172.21.15.151:0/1984267494 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e9c35200f0 0x55e9c352bb30 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:23.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.849+0000 7f15d332fb80 1 --2- 172.21.15.151:0/1984267494 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e9c35255c0 0x55e9c3528ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:23.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.849+0000 7f15a7e00700 0 client.14811 ms_handle_reset on v2:172.21.15.164:6836/1554609572 2024-05-21T21:54:23.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.849+0000 7f15d332fb80 1 -- 172.21.15.151:0/1984267494 >> 172.21.15.151:0/1984267494 conn(0x55e9c344db90 msgr2=0x55e9c3519d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:54:23.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.849+0000 7f15d332fb80 1 -- 172.21.15.151:0/1984267494 shutdown_connections 2024-05-21T21:54:23.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.849+0000 7f15d332fb80 1 -- 172.21.15.151:0/1984267494 wait complete. 2024-05-21T21:54:23.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.849+0000 7f15d332fb80 20 client.14811 trim_cache size 0 max 16384 2024-05-21T21:54:23.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.872+0000 7f15d332fb80 1 Processor -- start 2024-05-21T21:54:23.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.872+0000 7f15d332fb80 1 -- start start 2024-05-21T21:54:23.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.872+0000 7f15d332fb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e9c35200f0 0x55e9c35293c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:23.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.872+0000 7f15d332fb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e9c35255c0 0x55e9c35204d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:23.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.872+0000 7f15d332fb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e9c3525ee0 0x55e9c351a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:23.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.872+0000 7f15d332fb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e9c35385d0 con 0x55e9c35200f0 2024-05-21T21:54:23.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.872+0000 7f15d332fb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55e9c3538ca0 con 0x55e9c3525ee0 2024-05-21T21:54:23.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.872+0000 7f15d332fb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55e9c3539370 con 0x55e9c35255c0 2024-05-21T21:54:23.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.872+0000 7f15c1e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e9c35200f0 0x55e9c35293c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:23.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.873+0000 7f15c1e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e9c35200f0 0x55e9c35293c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:34606/0 (socket says 172.21.15.151:34606) 2024-05-21T21:54:23.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.873+0000 7f15c1e00700 1 -- 172.21.15.151:0/1173657058 learned_addr learned my addr 172.21.15.151:0/1173657058 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:54:23.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.873+0000 7f15c2800700 1 --2- 172.21.15.151:0/1173657058 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e9c3525ee0 0x55e9c351a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:23.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.873+0000 7f15b9400700 1 --2- 172.21.15.151:0/1173657058 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e9c35255c0 0x55e9c35204d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:23.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.873+0000 7f15c1e00700 1 -- 172.21.15.151:0/1173657058 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e9c35255c0 msgr2=0x55e9c35204d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:54:23.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.873+0000 7f15c1e00700 1 --2- 172.21.15.151:0/1173657058 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e9c35255c0 0x55e9c35204d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:23.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.873+0000 7f15c1e00700 1 -- 172.21.15.151:0/1173657058 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e9c3525ee0 msgr2=0x55e9c351a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:54:23.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.873+0000 7f15c1e00700 1 --2- 172.21.15.151:0/1173657058 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e9c3525ee0 0x55e9c351a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:23.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.873+0000 7f15c1e00700 1 -- 172.21.15.151:0/1173657058 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9c352c330 con 0x55e9c35200f0 2024-05-21T21:54:23.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.873+0000 7f15c1e00700 1 --2- 172.21.15.151:0/1173657058 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e9c35200f0 0x55e9c35293c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f15b4006250 tx=0x7f15b4003a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8584b9e2037eb5d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:54:23.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.873+0000 7f15b2a00700 1 -- 172.21.15.151:0/1173657058 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b402a040 con 0x55e9c35200f0 2024-05-21T21:54:23.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.873+0000 7f15b2a00700 1 -- 172.21.15.151:0/1173657058 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15b40055a0 con 0x55e9c35200f0 2024-05-21T21:54:23.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.873+0000 7f15b2a00700 1 -- 172.21.15.151:0/1173657058 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b402f9a0 con 0x55e9c35200f0 2024-05-21T21:54:23.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.874+0000 7f15d332fb80 1 -- 172.21.15.151:0/1173657058 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e9c35200f0 msgr2=0x55e9c35293c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:23.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.874+0000 7f15d332fb80 1 --2- 172.21.15.151:0/1173657058 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e9c35200f0 0x55e9c35293c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f15b4006250 tx=0x7f15b4003a00 comp rx=0 tx=0).stop 2024-05-21T21:54:23.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.874+0000 7f15d332fb80 1 -- 172.21.15.151:0/1173657058 shutdown_connections 2024-05-21T21:54:23.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.874+0000 7f15d332fb80 1 --2- 172.21.15.151:0/1173657058 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e9c3525ee0 0x55e9c351a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:23.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.874+0000 7f15d332fb80 1 --2- 172.21.15.151:0/1173657058 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e9c35200f0 0x55e9c35293c0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:23.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.874+0000 7f15d332fb80 1 --2- 172.21.15.151:0/1173657058 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e9c35255c0 0x55e9c35204d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:23.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.874+0000 7f15d332fb80 1 -- 172.21.15.151:0/1173657058 >> 172.21.15.151:0/1173657058 conn(0x55e9c344db90 msgr2=0x55e9c3528cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:54:23.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.874+0000 7f15d332fb80 1 -- 172.21.15.151:0/1173657058 shutdown_connections 2024-05-21T21:54:23.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.874+0000 7f15d332fb80 1 -- 172.21.15.151:0/1173657058 wait complete. 2024-05-21T21:54:23.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.875+0000 7f15d332fb80 1 -- start start 2024-05-21T21:54:23.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.875+0000 7f15d332fb80 1 Processor -- start 2024-05-21T21:54:23.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.875+0000 7f15d332fb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e9c35200f0 0x55e9c3543b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:23.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.875+0000 7f15d332fb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e9c35255c0 0x55e9c353e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:23.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.875+0000 7f15d332fb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e9c3525ee0 0x55e9c353ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:23.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.875+0000 7f15d332fb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e9c353a110 con 0x55e9c3525ee0 2024-05-21T21:54:23.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.875+0000 7f15d332fb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55e9c3539a40 con 0x55e9c35200f0 2024-05-21T21:54:23.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.875+0000 7f15d332fb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55e9c353a7e0 con 0x55e9c35255c0 2024-05-21T21:54:23.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.876+0000 7f15d332fb80 10 client.0 _ll_register_callbacks cb 0x55e9c3417c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:54:23.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.876+0000 7f15c1e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e9c35200f0 0x55e9c3543b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:23.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.876+0000 7f15b9400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e9c35255c0 0x55e9c353e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:23.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.876+0000 7f15c1e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e9c35200f0 0x55e9c3543b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:55132/0 (socket says 172.21.15.151:55132) 2024-05-21T21:54:23.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.876+0000 7f15c1e00700 1 -- 172.21.15.151:0/1885684539 learned_addr learned my addr 172.21.15.151:0/1885684539 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:54:23.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.876+0000 7f15c2800700 1 --2- 172.21.15.151:0/1885684539 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e9c3525ee0 0x55e9c353ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:23.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.876+0000 7f15b9400700 1 -- 172.21.15.151:0/1885684539 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e9c35200f0 msgr2=0x55e9c3543b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:54:23.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.876+0000 7f15b9400700 1 --2- 172.21.15.151:0/1885684539 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e9c35200f0 0x55e9c3543b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:23.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.876+0000 7f15b9400700 1 -- 172.21.15.151:0/1885684539 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e9c3525ee0 msgr2=0x55e9c353ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:54:23.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.876+0000 7f15b9400700 1 --2- 172.21.15.151:0/1885684539 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e9c3525ee0 0x55e9c353ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:23.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.876+0000 7f15b9400700 1 -- 172.21.15.151:0/1885684539 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9c353f170 con 0x55e9c35255c0 2024-05-21T21:54:23.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.876+0000 7f15c1e00700 1 --2- 172.21.15.151:0/1885684539 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e9c35200f0 0x55e9c3543b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:54:23.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.876+0000 7f15b9400700 1 --2- 172.21.15.151:0/1885684539 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e9c35255c0 0x55e9c353e6c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x55e9c3520ab0 tx=0x7f15ac0733f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4df1831f1e261c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:54:23.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.877+0000 7f15a7e00700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-21T21:54:23.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.877+0000 7f15a7e00700 1 -- 172.21.15.151:0/1885684539 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15ac0a9270 con 0x55e9c35255c0 2024-05-21T21:54:23.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.877+0000 7f15d332fb80 1 -- 172.21.15.151:0/1885684539 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15b40055a0 con 0x55e9c35255c0 2024-05-21T21:54:23.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.877+0000 7f15a7e00700 1 -- 172.21.15.151:0/1885684539 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f15ac0a9f70 con 0x55e9c35255c0 2024-05-21T21:54:23.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.877+0000 7f15a7e00700 1 -- 172.21.15.151:0/1885684539 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f15ac0a5290 con 0x55e9c35255c0 2024-05-21T21:54:23.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.877+0000 7f15d332fb80 10 client.24931 fetch_fsmap learned FSMap version 28 2024-05-21T21:54:23.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.877+0000 7f15d332fb80 1 -- 172.21.15.151:0/1885684539 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e9c35a5350 con 0x55e9c35255c0 2024-05-21T21:54:23.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.877+0000 7f15a7e00700 1 -- 172.21.15.151:0/1885684539 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15b40055a0 con 0x55e9c35255c0 2024-05-21T21:54:23.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.878+0000 7f15a7e00700 1 -- 172.21.15.151:0/1885684539 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f15ac0b24d0 con 0x55e9c35255c0 2024-05-21T21:54:23.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.878+0000 7f15d332fb80 10 client.24931 fetch_fsmap finished waiting for FSMap version 28 2024-05-21T21:54:23.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.878+0000 7f15d332fb80 10 client.24931 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-21T21:54:23.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.878+0000 7f15d332fb80 20 client.24931 populate_metadata read hostname 'smithi151' 2024-05-21T21:54:23.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.878+0000 7f15d332fb80 1 --2- 172.21.15.151:0/1885684539 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x55e9c35a9ec0 0x55e9c35a8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:23.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.878+0000 7f15d332fb80 4 client.24931 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-21T21:54:23.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.878+0000 7f15d332fb80 1 -- 172.21.15.151:0/1885684539 --> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] -- command(tid 0: {"prefix": "status"}) -- 0x55e9c3479520 con 0x55e9c35a9ec0 2024-05-21T21:54:23.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.878+0000 7f15c1e00700 1 --2- 172.21.15.151:0/1885684539 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x55e9c35a9ec0 0x55e9c35a8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:23.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.878+0000 7f15c1e00700 1 --2- 172.21.15.151:0/1885684539 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x55e9c35a9ec0 0x55e9c35a8bd0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=289bd9647d4bf181 server_cookie=71b65c272fe05528 in_seq=0 out_seq=0 2024-05-21T21:54:23.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.878+0000 7f15a7e00700 10 client.24931 ms_handle_connect on v2:172.21.15.164:6836/1554609572 2024-05-21T21:54:23.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.879+0000 7f15a7e00700 1 -- 172.21.15.151:0/1885684539 <== mds.2 v2:172.21.15.164:6836/1554609572 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55e9c3479520 con 0x55e9c35a9ec0 2024-05-21T21:54:23.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.879+0000 7f15a7e00700 10 client.24931 handle_command_reply: tid=0 2024-05-21T21:54:23.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.879+0000 7f15d332fb80 1 client.24931 shutdown 2024-05-21T21:54:23.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.879+0000 7f15d332fb80 1 -- 172.21.15.151:0/1885684539 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e9c35255c0 msgr2=0x55e9c353e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:23.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.879+0000 7f15d332fb80 1 --2- 172.21.15.151:0/1885684539 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e9c35255c0 0x55e9c353e6c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x55e9c3520ab0 tx=0x7f15ac0733f0 comp rx=0 tx=0).stop 2024-05-21T21:54:23.884 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.879+0000 7f15d332fb80 1 -- 172.21.15.151:0/1885684539 shutdown_connections 2024-05-21T21:54:23.884 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.879+0000 7f15d332fb80 1 --2- 172.21.15.151:0/1885684539 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x55e9c35a9ec0 0x55e9c35a8bd0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:23.884 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.879+0000 7f15d332fb80 1 --2- 172.21.15.151:0/1885684539 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e9c35200f0 0x55e9c3543b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:23.884 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.879+0000 7f15d332fb80 1 --2- 172.21.15.151:0/1885684539 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e9c3525ee0 0x55e9c353ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:23.884 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.879+0000 7f15d332fb80 1 --2- 172.21.15.151:0/1885684539 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e9c35255c0 0x55e9c353e6c0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:23.884 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.879+0000 7f15d332fb80 1 -- 172.21.15.151:0/1885684539 >> 172.21.15.151:0/1885684539 conn(0x55e9c344db90 msgr2=0x55e9c35262c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:54:23.884 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.879+0000 7f15a7e00700 0 client.24931 ms_handle_reset on v2:172.21.15.164:6836/1554609572 2024-05-21T21:54:23.884 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.879+0000 7f15d332fb80 1 -- 172.21.15.151:0/1885684539 shutdown_connections 2024-05-21T21:54:23.884 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.879+0000 7f15d332fb80 1 -- 172.21.15.151:0/1885684539 wait complete. 2024-05-21T21:54:23.884 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.879+0000 7f15d332fb80 20 client.24931 trim_cache size 0 max 16384 2024-05-21T21:54:23.885 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-05-21T21:54:23.885 INFO:teuthology.orchestra.run.smithi151.stdout: "cluster_fsid": "c8c7a2e6-17ba-11ef-bc9a-c7b262605968", 2024-05-21T21:54:23.885 INFO:teuthology.orchestra.run.smithi151.stdout: "whoami": 2, 2024-05-21T21:54:23.885 INFO:teuthology.orchestra.run.smithi151.stdout: "id": 24491, 2024-05-21T21:54:23.885 INFO:teuthology.orchestra.run.smithi151.stdout: "want_state": "up:active", 2024-05-21T21:54:23.885 INFO:teuthology.orchestra.run.smithi151.stdout: "state": "up:active", 2024-05-21T21:54:23.885 INFO:teuthology.orchestra.run.smithi151.stdout: "fs_name": "cephfs", 2024-05-21T21:54:23.885 INFO:teuthology.orchestra.run.smithi151.stdout: "rank_uptime": 167.57359291500001, 2024-05-21T21:54:23.885 INFO:teuthology.orchestra.run.smithi151.stdout: "mdsmap_epoch": 28, 2024-05-21T21:54:23.885 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_epoch": 82, 2024-05-21T21:54:23.885 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_epoch_barrier": 80, 2024-05-21T21:54:23.886 INFO:teuthology.orchestra.run.smithi151.stdout: "uptime": 219.99028960800001 2024-05-21T21:54:23.886 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-05-21T21:54:23.886 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.880+0000 7f15c8a00700 1 -- 172.21.15.151:0/3536427224 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f15a80b2880 msgr2=0x7f15a80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:23.886 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.880+0000 7f15c8a00700 1 --2- 172.21.15.151:0/3536427224 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f15a80b2880 0x7f15a80b4d40 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f15c406ae10 tx=0x7f15bc0047c0 comp rx=0 tx=0).stop 2024-05-21T21:54:23.886 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.880+0000 7f15c8a00700 1 -- 172.21.15.151:0/3536427224 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f15c406bb70 msgr2=0x7f15c419c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:23.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.880+0000 7f15c8a00700 1 --2- 172.21.15.151:0/3536427224 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f15c406bb70 0x7f15c419c3a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f15ac00eb30 tx=0x7f15ac00ee40 comp rx=0 tx=0).stop 2024-05-21T21:54:23.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.880+0000 7f15c8a00700 1 -- 172.21.15.151:0/3536427224 shutdown_connections 2024-05-21T21:54:23.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.880+0000 7f15c8a00700 1 --2- 172.21.15.151:0/3536427224 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f15a80b2880 0x7f15a80b4d40 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:23.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.880+0000 7f15c8a00700 1 --2- 172.21.15.151:0/3536427224 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f15c410fff0 0x7f15c41a0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:23.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.880+0000 7f15c8a00700 1 --2- 172.21.15.151:0/3536427224 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f15c406b250 0x7f15c419be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:23.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.880+0000 7f15c8a00700 1 --2- 172.21.15.151:0/3536427224 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f15c406bb70 0x7f15c419c3a0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:23.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.880+0000 7f15c8a00700 1 -- 172.21.15.151:0/3536427224 >> 172.21.15.151:0/3536427224 conn(0x7f15c40793f0 msgr2=0x7f15c41114c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:54:23.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.880+0000 7f15c8a00700 1 -- 172.21.15.151:0/3536427224 shutdown_connections 2024-05-21T21:54:23.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:23.880+0000 7f15c8a00700 1 -- 172.21.15.151:0/3536427224 wait complete. 2024-05-21T21:54:24.599 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:54:24 smithi151 ceph-mon[31623]: pgmap v409: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-21T21:54:24.651 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.i status 2024-05-21T21:54:24.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:54:24 smithi184 ceph-mon[36942]: pgmap v409: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-21T21:54:24.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:54:24 smithi164 ceph-mon[36269]: pgmap v409: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-21T21:54:25.090 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:54:26.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:54:26 smithi151 ceph-mon[31623]: pgmap v410: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-21T21:54:26.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:54:26 smithi184 ceph-mon[36942]: pgmap v410: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-21T21:54:26.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:54:26 smithi164 ceph-mon[36269]: pgmap v410: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-21T21:54:27.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.348+0000 7f209f200700 1 -- 172.21.15.151:0/3917026373 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f209806bbe0 msgr2=0x7f20981103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:27.350 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.348+0000 7f209f200700 1 --2- 172.21.15.151:0/3917026373 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f209806bbe0 0x7f20981103d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f209000b0a0 tx=0x7f209000b3b0 comp rx=0 tx=0).stop 2024-05-21T21:54:27.350 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.348+0000 7f209f200700 1 -- 172.21.15.151:0/3917026373 shutdown_connections 2024-05-21T21:54:27.350 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.348+0000 7f209f200700 1 --2- 172.21.15.151:0/3917026373 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f209806bbe0 0x7f20981103d0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:27.350 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.348+0000 7f209f200700 1 --2- 172.21.15.151:0/3917026373 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20981095b0 0x7f2098109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:27.350 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.348+0000 7f209f200700 1 --2- 172.21.15.151:0/3917026373 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f209806b220 0x7f209806b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:27.350 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.348+0000 7f209f200700 1 -- 172.21.15.151:0/3917026373 >> 172.21.15.151:0/3917026373 conn(0x7f20980793f0 msgr2=0x7f2098079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:54:27.350 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.348+0000 7f209f200700 1 -- 172.21.15.151:0/3917026373 shutdown_connections 2024-05-21T21:54:27.350 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.348+0000 7f209f200700 1 -- 172.21.15.151:0/3917026373 wait complete. 2024-05-21T21:54:27.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.350+0000 7f209f200700 1 Processor -- start 2024-05-21T21:54:27.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.350+0000 7f209f200700 1 -- start start 2024-05-21T21:54:27.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.350+0000 7f209f200700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f209806b220 0x7f209819be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:27.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.350+0000 7f209f200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f209806bbe0 0x7f209819c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:27.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.350+0000 7f209f200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f20981095b0 0x7f20981a0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:27.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.350+0000 7f209f200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2098113740 con 0x7f20981095b0 2024-05-21T21:54:27.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.350+0000 7f209f200700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f20981135c0 con 0x7f209806bbe0 2024-05-21T21:54:27.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.350+0000 7f209f200700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f20981138c0 con 0x7f209806b220 2024-05-21T21:54:27.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.350+0000 7f209d200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f20981095b0 0x7f20981a0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:27.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.351+0000 7f2097e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f209806b220 0x7f209819be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:27.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.351+0000 7f2097400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f209806bbe0 0x7f209819c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:27.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.351+0000 7f2097e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f209806b220 0x7f209819be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:52492/0 (socket says 172.21.15.151:52492) 2024-05-21T21:54:27.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.351+0000 7f2097400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f209806bbe0 0x7f209819c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:52188/0 (socket says 172.21.15.151:52188) 2024-05-21T21:54:27.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.351+0000 7f2097e00700 1 -- 172.21.15.151:0/3328263652 learned_addr learned my addr 172.21.15.151:0/3328263652 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:54:27.354 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.351+0000 7f2097400700 1 -- 172.21.15.151:0/3328263652 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f209806b220 msgr2=0x7f209819be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:27.354 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.351+0000 7f2097400700 1 --2- 172.21.15.151:0/3328263652 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f209806b220 0x7f209819be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:27.354 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.351+0000 7f2097400700 1 -- 172.21.15.151:0/3328263652 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f20981095b0 msgr2=0x7f20981a0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:27.354 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.351+0000 7f2097400700 1 --2- 172.21.15.151:0/3328263652 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f20981095b0 0x7f20981a0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:27.354 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.351+0000 7f2097400700 1 -- 172.21.15.151:0/3328263652 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20981a0ea0 con 0x7f209806bbe0 2024-05-21T21:54:27.354 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.351+0000 7f209d200700 1 --2- 172.21.15.151:0/3328263652 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f20981095b0 0x7f20981a0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:54:27.354 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.351+0000 7f2097e00700 1 --2- 172.21.15.151:0/3328263652 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f209806b220 0x7f209819be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:54:27.354 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.351+0000 7f2097400700 1 --2- 172.21.15.151:0/3328263652 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f209806bbe0 0x7f209819c3d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f208400d750 tx=0x7f208400da60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:54:27.354 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.352+0000 7f2094c00700 1 -- 172.21.15.151:0/3328263652 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2084014030 con 0x7f209806bbe0 2024-05-21T21:54:27.354 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.352+0000 7f2094c00700 1 -- 172.21.15.151:0/3328263652 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20840044d0 con 0x7f209806bbe0 2024-05-21T21:54:27.354 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.352+0000 7f2094c00700 1 -- 172.21.15.151:0/3328263652 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2084005420 con 0x7f209806bbe0 2024-05-21T21:54:27.354 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.352+0000 7f209f200700 1 -- 172.21.15.151:0/3328263652 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20981a1190 con 0x7f209806bbe0 2024-05-21T21:54:27.354 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.352+0000 7f209f200700 1 -- 172.21.15.151:0/3328263652 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20981a8a10 con 0x7f209806bbe0 2024-05-21T21:54:27.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.356+0000 7f2094c00700 1 -- 172.21.15.151:0/3328263652 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f2084004a50 con 0x7f209806bbe0 2024-05-21T21:54:27.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.356+0000 7f20a9bdfb80 1 Processor -- start 2024-05-21T21:54:27.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.357+0000 7f20a9bdfb80 1 -- start start 2024-05-21T21:54:27.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.357+0000 7f20a9bdfb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cd51e44e60 0x55cd51e45240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:27.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.357+0000 7f20a9bdfb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd51e45780 0x55cd51e3f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:27.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.357+0000 7f20a9bdfb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cd51e3f830 0x55cd51e3fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:27.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.357+0000 7f20a9bdfb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55cd51d9ddf0 con 0x55cd51e3f830 2024-05-21T21:54:27.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.357+0000 7f20a9bdfb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55cd51d8d990 con 0x55cd51e45780 2024-05-21T21:54:27.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.357+0000 7f20a9bdfb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55cd51d919d0 con 0x55cd51e44e60 2024-05-21T21:54:27.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.357+0000 7f209d200700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cd51e44e60 0x55cd51e45240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:27.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.357+0000 7f209d200700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cd51e44e60 0x55cd51e45240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:52496/0 (socket says 172.21.15.151:52496) 2024-05-21T21:54:27.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.357+0000 7f2097400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cd51e3f830 0x55cd51e3fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:27.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.357+0000 7f209d200700 1 -- 172.21.15.151:0/89162824 learned_addr learned my addr 172.21.15.151:0/89162824 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:54:27.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.357+0000 7f2097e00700 1 --2- 172.21.15.151:0/89162824 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd51e45780 0x55cd51e3f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:27.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.358+0000 7f2097e00700 1 -- 172.21.15.151:0/89162824 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cd51e44e60 msgr2=0x55cd51e45240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:54:27.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.358+0000 7f2097e00700 1 --2- 172.21.15.151:0/89162824 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cd51e44e60 0x55cd51e45240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:27.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.358+0000 7f2097e00700 1 -- 172.21.15.151:0/89162824 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cd51e3f830 msgr2=0x55cd51e3fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:54:27.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.358+0000 7f2097e00700 1 --2- 172.21.15.151:0/89162824 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cd51e3f830 0x55cd51e3fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:27.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.358+0000 7f2097e00700 1 -- 172.21.15.151:0/89162824 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd51d563d0 con 0x55cd51e45780 2024-05-21T21:54:27.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.358+0000 7f209d200700 1 --2- 172.21.15.151:0/89162824 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cd51e44e60 0x55cd51e45240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:54:27.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.358+0000 7f2094c00700 1 --2- 172.21.15.151:0/3328263652 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f207c0b2900 0x7f207c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:27.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.358+0000 7f209d200700 1 --2- 172.21.15.151:0/3328263652 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f207c0b2900 0x7f207c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:27.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.358+0000 7f2097e00700 1 --2- 172.21.15.151:0/89162824 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd51e45780 0x55cd51e3f210 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2088011540 tx=0x7f2088011850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e05194ef16cf1251 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:54:27.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.358+0000 7f2094c00700 1 -- 172.21.15.151:0/3328263652 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f20840e1e40 con 0x7f209806bbe0 2024-05-21T21:54:27.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.359+0000 7f208d600700 1 -- 172.21.15.151:0/89162824 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f208801e070 con 0x55cd51e45780 2024-05-21T21:54:27.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.359+0000 7f208d600700 1 -- 172.21.15.151:0/89162824 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f208800fb30 con 0x55cd51e45780 2024-05-21T21:54:27.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.359+0000 7f208d600700 1 -- 172.21.15.151:0/89162824 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2088019c60 con 0x55cd51e45780 2024-05-21T21:54:27.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.359+0000 7f20a9bdfb80 1 -- 172.21.15.151:0/89162824 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd51e45780 msgr2=0x55cd51e3f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:27.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.359+0000 7f20a9bdfb80 1 --2- 172.21.15.151:0/89162824 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd51e45780 0x55cd51e3f210 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2088011540 tx=0x7f2088011850 comp rx=0 tx=0).stop 2024-05-21T21:54:27.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.359+0000 7f209d200700 1 --2- 172.21.15.151:0/3328263652 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f207c0b2900 0x7f207c0b4dc0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f209000bfa0 tx=0x7f209001d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:54:27.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.360+0000 7f20a9bdfb80 1 -- 172.21.15.151:0/89162824 shutdown_connections 2024-05-21T21:54:27.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.360+0000 7f20a9bdfb80 1 --2- 172.21.15.151:0/89162824 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cd51e3f830 0x55cd51e3fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:27.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.360+0000 7f20a9bdfb80 1 --2- 172.21.15.151:0/89162824 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cd51e44e60 0x55cd51e45240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:27.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.360+0000 7f20a9bdfb80 1 --2- 172.21.15.151:0/89162824 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd51e45780 0x55cd51e3f210 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:27.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.360+0000 7f20a9bdfb80 1 -- 172.21.15.151:0/89162824 >> 172.21.15.151:0/89162824 conn(0x55cd51d6db90 msgr2=0x55cd51e39fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:54:27.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.360+0000 7f20a9bdfb80 1 -- 172.21.15.151:0/89162824 shutdown_connections 2024-05-21T21:54:27.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.360+0000 7f20a9bdfb80 1 -- 172.21.15.151:0/89162824 wait complete. 2024-05-21T21:54:27.363 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.360+0000 7f20a9bdfb80 1 -- start start 2024-05-21T21:54:27.363 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.360+0000 7f20a9bdfb80 1 Processor -- start 2024-05-21T21:54:27.364 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.361+0000 7f20a9bdfb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cd51e3f830 0x55cd51e4b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:27.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.361+0000 7f20a9bdfb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cd51e44e60 0x55cd51e48850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:27.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.361+0000 7f20a9bdfb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd51e45780 0x55cd51e48d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:27.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.361+0000 7f20a9bdfb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55cd51c93f50 con 0x55cd51e3f830 2024-05-21T21:54:27.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.361+0000 7f20a9bdfb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55cd51d2e710 con 0x55cd51e45780 2024-05-21T21:54:27.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.361+0000 7f20a9bdfb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55cd51e4c1f0 con 0x55cd51e44e60 2024-05-21T21:54:27.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.361+0000 7f20a9bdfb80 10 client.0 _ll_register_callbacks cb 0x55cd51d37c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:54:27.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.361+0000 7f2097400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cd51e44e60 0x55cd51e48850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:27.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.361+0000 7f2097e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cd51e3f830 0x55cd51e4b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:27.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.361+0000 7f2097400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cd51e44e60 0x55cd51e48850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:52510/0 (socket says 172.21.15.151:52510) 2024-05-21T21:54:27.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.361+0000 7f2097400700 1 -- 172.21.15.151:0/3866084373 learned_addr learned my addr 172.21.15.151:0/3866084373 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:54:27.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.361+0000 7f2097e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cd51e3f830 0x55cd51e4b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:39848/0 (socket says 172.21.15.151:39848) 2024-05-21T21:54:27.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.361+0000 7f209d200700 1 --2- 172.21.15.151:0/3866084373 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd51e45780 0x55cd51e48d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:27.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.362+0000 7f2097400700 1 -- 172.21.15.151:0/3866084373 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd51e45780 msgr2=0x55cd51e48d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:54:27.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.362+0000 7f2097400700 1 --2- 172.21.15.151:0/3866084373 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd51e45780 0x55cd51e48d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:27.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.362+0000 7f2097400700 1 -- 172.21.15.151:0/3866084373 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cd51e3f830 msgr2=0x55cd51e4b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:54:27.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.362+0000 7f2097400700 1 --2- 172.21.15.151:0/3866084373 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cd51e3f830 0x55cd51e4b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:27.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.362+0000 7f2097400700 1 -- 172.21.15.151:0/3866084373 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd51cdd880 con 0x55cd51e44e60 2024-05-21T21:54:27.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.362+0000 7f2097e00700 1 --2- 172.21.15.151:0/3866084373 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cd51e3f830 0x55cd51e4b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:54:27.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.362+0000 7f2097400700 1 --2- 172.21.15.151:0/3866084373 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cd51e44e60 0x55cd51e48850 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f20840a6690 tx=0x7f2084004fa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f387d784e8a4631 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:54:27.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.362+0000 7f2082000700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-21T21:54:27.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.362+0000 7f2082000700 1 -- 172.21.15.151:0/3866084373 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f208401d040 con 0x55cd51e44e60 2024-05-21T21:54:27.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.362+0000 7f2082000700 1 -- 172.21.15.151:0/3866084373 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f20840055c0 con 0x55cd51e44e60 2024-05-21T21:54:27.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.362+0000 7f20a9bdfb80 1 -- 172.21.15.151:0/3866084373 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f208800fb30 con 0x55cd51e44e60 2024-05-21T21:54:27.367 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.362+0000 7f2082000700 1 -- 172.21.15.151:0/3866084373 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f20840a8d70 con 0x55cd51e44e60 2024-05-21T21:54:27.367 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.362+0000 7f20a9bdfb80 10 client.24943 fetch_fsmap learned FSMap version 28 2024-05-21T21:54:27.367 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.362+0000 7f20a9bdfb80 1 -- 172.21.15.151:0/3866084373 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cd51d5e010 con 0x55cd51e44e60 2024-05-21T21:54:27.367 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.363+0000 7f2082000700 1 -- 172.21.15.151:0/3866084373 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f208800fb30 con 0x55cd51e44e60 2024-05-21T21:54:27.367 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.363+0000 7f2082000700 1 -- 172.21.15.151:0/3866084373 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f20840a9680 con 0x55cd51e44e60 2024-05-21T21:54:27.367 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.363+0000 7f20a9bdfb80 10 client.24943 fetch_fsmap finished waiting for FSMap version 28 2024-05-21T21:54:27.367 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.363+0000 7f20a9bdfb80 10 client.24943 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-21T21:54:27.367 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.363+0000 7f20a9bdfb80 20 client.24943 populate_metadata read hostname 'smithi151' 2024-05-21T21:54:27.367 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.363+0000 7f20a9bdfb80 1 --2- 172.21.15.151:0/3866084373 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x55cd51e61440 0x55cd51e628b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:27.367 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.363+0000 7f20a9bdfb80 4 client.24943 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T21:54:27.367 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.363+0000 7f20a9bdfb80 1 -- 172.21.15.151:0/3866084373 --> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cd51d9fd40 con 0x55cd51e61440 2024-05-21T21:54:27.368 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.363+0000 7f2097e00700 1 --2- 172.21.15.151:0/3866084373 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x55cd51e61440 0x55cd51e628b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:27.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.364+0000 7f2097e00700 1 --2- 172.21.15.151:0/3866084373 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x55cd51e61440 0x55cd51e628b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e56d6fcf51dcfc4e server_cookie=3154e55bb3e8d4d9 in_seq=0 out_seq=0 2024-05-21T21:54:27.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.364+0000 7f2082000700 10 client.24943 ms_handle_connect on v2:172.21.15.184:6838/3059935143 2024-05-21T21:54:27.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.365+0000 7f2082000700 1 -- 172.21.15.151:0/3866084373 <== mds.1 v2:172.21.15.184:6838/3059935143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55cd51d9fd40 con 0x55cd51e61440 2024-05-21T21:54:27.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.365+0000 7f2082000700 10 client.24943 handle_command_reply: tid=0 2024-05-21T21:54:27.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.365+0000 7f20a9bdfb80 1 client.24943 shutdown 2024-05-21T21:54:27.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.365+0000 7f20a9bdfb80 1 -- 172.21.15.151:0/3866084373 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cd51e44e60 msgr2=0x55cd51e48850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:27.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.365+0000 7f20a9bdfb80 1 --2- 172.21.15.151:0/3866084373 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cd51e44e60 0x55cd51e48850 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f20840a6690 tx=0x7f2084004fa0 comp rx=0 tx=0).stop 2024-05-21T21:54:27.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.365+0000 7f20a9bdfb80 1 -- 172.21.15.151:0/3866084373 shutdown_connections 2024-05-21T21:54:27.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.365+0000 7f20a9bdfb80 1 --2- 172.21.15.151:0/3866084373 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x55cd51e61440 0x55cd51e628b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:27.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.365+0000 7f20a9bdfb80 1 --2- 172.21.15.151:0/3866084373 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd51e45780 0x55cd51e48d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:27.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.365+0000 7f20a9bdfb80 1 --2- 172.21.15.151:0/3866084373 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cd51e3f830 0x55cd51e4b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:27.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.365+0000 7f20a9bdfb80 1 --2- 172.21.15.151:0/3866084373 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cd51e44e60 0x55cd51e48850 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:27.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.365+0000 7f2082000700 0 client.24943 ms_handle_reset on v2:172.21.15.184:6838/3059935143 2024-05-21T21:54:27.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.365+0000 7f20a9bdfb80 1 -- 172.21.15.151:0/3866084373 >> 172.21.15.151:0/3866084373 conn(0x55cd51d6db90 msgr2=0x55cd51e3e200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:54:27.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.366+0000 7f20a9bdfb80 1 -- 172.21.15.151:0/3866084373 shutdown_connections 2024-05-21T21:54:27.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.366+0000 7f20a9bdfb80 1 -- 172.21.15.151:0/3866084373 wait complete. 2024-05-21T21:54:27.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.366+0000 7f20a9bdfb80 20 client.24943 trim_cache size 0 max 16384 2024-05-21T21:54:27.390 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.389+0000 7f20a9bdfb80 1 Processor -- start 2024-05-21T21:54:27.390 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.389+0000 7f20a9bdfb80 1 -- start start 2024-05-21T21:54:27.390 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.389+0000 7f20a9bdfb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd51e3f830 0x55cd51e3f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:27.390 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.389+0000 7f20a9bdfb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cd51e44e60 0x55cd51e42970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:27.390 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.389+0000 7f20a9bdfb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cd51e45780 0x55cd51e42eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:27.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.389+0000 7f20a9bdfb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55cd51e60bf0 con 0x55cd51e44e60 2024-05-21T21:54:27.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.389+0000 7f20a9bdfb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55cd51e612c0 con 0x55cd51e3f830 2024-05-21T21:54:27.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.389+0000 7f20a9bdfb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55cd51e61990 con 0x55cd51e45780 2024-05-21T21:54:27.392 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.389+0000 7f2097400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cd51e44e60 0x55cd51e42970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:27.392 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.390+0000 7f209d200700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cd51e45780 0x55cd51e42eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:27.392 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.390+0000 7f2097e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd51e3f830 0x55cd51e3f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:27.392 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.390+0000 7f209d200700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cd51e45780 0x55cd51e42eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:52526/0 (socket says 172.21.15.151:52526) 2024-05-21T21:54:27.392 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.390+0000 7f209d200700 1 -- 172.21.15.151:0/438436418 learned_addr learned my addr 172.21.15.151:0/438436418 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:54:27.392 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.390+0000 7f209d200700 1 -- 172.21.15.151:0/438436418 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd51e3f830 msgr2=0x55cd51e3f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:54:27.392 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.390+0000 7f209d200700 1 --2- 172.21.15.151:0/438436418 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd51e3f830 0x55cd51e3f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:27.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.390+0000 7f209d200700 1 -- 172.21.15.151:0/438436418 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cd51e44e60 msgr2=0x55cd51e42970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:54:27.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.390+0000 7f209d200700 1 --2- 172.21.15.151:0/438436418 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cd51e44e60 0x55cd51e42970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:27.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.390+0000 7f209d200700 1 -- 172.21.15.151:0/438436418 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd51e62550 con 0x55cd51e45780 2024-05-21T21:54:27.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.390+0000 7f2097e00700 1 --2- 172.21.15.151:0/438436418 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd51e3f830 0x55cd51e3f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:54:27.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.390+0000 7f2097400700 1 --2- 172.21.15.151:0/438436418 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cd51e44e60 0x55cd51e42970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:54:27.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.390+0000 7f209d200700 1 --2- 172.21.15.151:0/438436418 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cd51e45780 0x55cd51e42eb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f209000b0a0 tx=0x7f209001f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3fc0a61802daa7e server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:54:27.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.391+0000 7f208cc00700 1 -- 172.21.15.151:0/438436418 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20900078c0 con 0x55cd51e45780 2024-05-21T21:54:27.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.391+0000 7f208cc00700 1 -- 172.21.15.151:0/438436418 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2090005040 con 0x55cd51e45780 2024-05-21T21:54:27.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.391+0000 7f208cc00700 1 -- 172.21.15.151:0/438436418 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2090027410 con 0x55cd51e45780 2024-05-21T21:54:27.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.391+0000 7f20a9bdfb80 1 -- 172.21.15.151:0/438436418 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cd51e45780 msgr2=0x55cd51e42eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:27.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.391+0000 7f20a9bdfb80 1 --2- 172.21.15.151:0/438436418 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cd51e45780 0x55cd51e42eb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f209000b0a0 tx=0x7f209001f000 comp rx=0 tx=0).stop 2024-05-21T21:54:27.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.391+0000 7f20a9bdfb80 1 -- 172.21.15.151:0/438436418 shutdown_connections 2024-05-21T21:54:27.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.391+0000 7f20a9bdfb80 1 --2- 172.21.15.151:0/438436418 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd51e3f830 0x55cd51e3f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:27.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.391+0000 7f20a9bdfb80 1 --2- 172.21.15.151:0/438436418 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cd51e45780 0x55cd51e42eb0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:27.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.391+0000 7f20a9bdfb80 1 --2- 172.21.15.151:0/438436418 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cd51e44e60 0x55cd51e42970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:27.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.391+0000 7f20a9bdfb80 1 -- 172.21.15.151:0/438436418 >> 172.21.15.151:0/438436418 conn(0x55cd51d6db90 msgr2=0x55cd51e39c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:54:27.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.392+0000 7f20a9bdfb80 1 -- 172.21.15.151:0/438436418 shutdown_connections 2024-05-21T21:54:27.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.392+0000 7f20a9bdfb80 1 -- 172.21.15.151:0/438436418 wait complete. 2024-05-21T21:54:27.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.392+0000 7f20a9bdfb80 1 -- start start 2024-05-21T21:54:27.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.392+0000 7f20a9bdfb80 1 Processor -- start 2024-05-21T21:54:27.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.393+0000 7f20a9bdfb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cd51e3f830 0x55cd51e6c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:27.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.393+0000 7f20a9bdfb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cd51e44e60 0x55cd51e66ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:27.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.393+0000 7f20a9bdfb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd51e45780 0x55cd51e67220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:27.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.393+0000 7f20a9bdfb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55cd51e62730 con 0x55cd51e3f830 2024-05-21T21:54:27.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.393+0000 7f20a9bdfb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55cd51e62060 con 0x55cd51e45780 2024-05-21T21:54:27.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.393+0000 7f20a9bdfb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55cd51e62e00 con 0x55cd51e44e60 2024-05-21T21:54:27.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.393+0000 7f20a9bdfb80 10 client.0 _ll_register_callbacks cb 0x55cd51d37c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:54:27.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.393+0000 7f2097400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cd51e44e60 0x55cd51e66ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:27.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.393+0000 7f2097e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cd51e3f830 0x55cd51e6c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:27.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.393+0000 7f2097400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cd51e44e60 0x55cd51e66ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:52530/0 (socket says 172.21.15.151:52530) 2024-05-21T21:54:27.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.393+0000 7f2097400700 1 -- 172.21.15.151:0/2328899571 learned_addr learned my addr 172.21.15.151:0/2328899571 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:54:27.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.393+0000 7f209d200700 1 --2- 172.21.15.151:0/2328899571 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd51e45780 0x55cd51e67220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:27.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.393+0000 7f2097e00700 1 -- 172.21.15.151:0/2328899571 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cd51e44e60 msgr2=0x55cd51e66ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:54:27.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.393+0000 7f2097e00700 1 --2- 172.21.15.151:0/2328899571 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cd51e44e60 0x55cd51e66ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:27.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.393+0000 7f2097e00700 1 -- 172.21.15.151:0/2328899571 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd51e45780 msgr2=0x55cd51e67220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:54:27.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.394+0000 7f2097e00700 1 --2- 172.21.15.151:0/2328899571 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd51e45780 0x55cd51e67220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:27.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.394+0000 7f2097e00700 1 -- 172.21.15.151:0/2328899571 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd51e67860 con 0x55cd51e3f830 2024-05-21T21:54:27.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.394+0000 7f2097400700 1 --2- 172.21.15.151:0/2328899571 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cd51e44e60 0x55cd51e66ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:54:27.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.394+0000 7f209d200700 1 --2- 172.21.15.151:0/2328899571 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd51e45780 0x55cd51e67220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:54:27.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.394+0000 7f2097e00700 1 --2- 172.21.15.151:0/2328899571 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cd51e3f830 0x55cd51e6c1f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x55cd51e49330 tx=0x7f2088011270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bb31b9d08e96ba server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:54:27.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.394+0000 7f2082000700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-21T21:54:27.398 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.394+0000 7f2082000700 1 -- 172.21.15.151:0/2328899571 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f208801e040 con 0x55cd51e3f830 2024-05-21T21:54:27.398 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.394+0000 7f20a9bdfb80 1 -- 172.21.15.151:0/2328899571 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2090005040 con 0x55cd51e3f830 2024-05-21T21:54:27.398 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.394+0000 7f2082000700 1 -- 172.21.15.151:0/2328899571 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f208800fa30 con 0x55cd51e3f830 2024-05-21T21:54:27.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.394+0000 7f2082000700 1 -- 172.21.15.151:0/2328899571 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2088005ce0 con 0x55cd51e3f830 2024-05-21T21:54:27.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.394+0000 7f20a9bdfb80 10 client.14835 fetch_fsmap learned FSMap version 28 2024-05-21T21:54:27.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.394+0000 7f20a9bdfb80 1 -- 172.21.15.151:0/2328899571 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cd51ecda20 con 0x55cd51e3f830 2024-05-21T21:54:27.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.395+0000 7f2082000700 1 -- 172.21.15.151:0/2328899571 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2090005040 con 0x55cd51e3f830 2024-05-21T21:54:27.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.395+0000 7f2082000700 1 -- 172.21.15.151:0/2328899571 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f2088005140 con 0x55cd51e3f830 2024-05-21T21:54:27.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.395+0000 7f20a9bdfb80 10 client.14835 fetch_fsmap finished waiting for FSMap version 28 2024-05-21T21:54:27.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.395+0000 7f20a9bdfb80 10 client.14835 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-21T21:54:27.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.395+0000 7f20a9bdfb80 20 client.14835 populate_metadata read hostname 'smithi151' 2024-05-21T21:54:27.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.395+0000 7f20a9bdfb80 1 --2- 172.21.15.151:0/2328899571 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x55cd51ed2450 0x55cd51ed1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:27.401 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.395+0000 7f20a9bdfb80 4 client.14835 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-21T21:54:27.401 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.395+0000 7f20a9bdfb80 1 -- 172.21.15.151:0/2328899571 --> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] -- command(tid 0: {"prefix": "status"}) -- 0x55cd51d99520 con 0x55cd51ed2450 2024-05-21T21:54:27.401 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.396+0000 7f2097400700 1 --2- 172.21.15.151:0/2328899571 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x55cd51ed2450 0x55cd51ed1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:27.401 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.396+0000 7f2097400700 1 --2- 172.21.15.151:0/2328899571 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x55cd51ed2450 0x55cd51ed1160 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6d73d8e86b7d6539 server_cookie=100e5b8afcfee94d in_seq=0 out_seq=0 2024-05-21T21:54:27.401 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.396+0000 7f2082000700 10 client.14835 ms_handle_connect on v2:172.21.15.184:6838/3059935143 2024-05-21T21:54:27.401 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.396+0000 7f2082000700 1 -- 172.21.15.151:0/2328899571 <== mds.1 v2:172.21.15.184:6838/3059935143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55cd51d99520 con 0x55cd51ed2450 2024-05-21T21:54:27.401 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.396+0000 7f2082000700 10 client.14835 handle_command_reply: tid=0 2024-05-21T21:54:27.401 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.396+0000 7f20a9bdfb80 1 client.14835 shutdown 2024-05-21T21:54:27.401 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.396+0000 7f20a9bdfb80 1 -- 172.21.15.151:0/2328899571 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cd51e3f830 msgr2=0x55cd51e6c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:27.401 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.396+0000 7f20a9bdfb80 1 --2- 172.21.15.151:0/2328899571 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cd51e3f830 0x55cd51e6c1f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x55cd51e49330 tx=0x7f2088011270 comp rx=0 tx=0).stop 2024-05-21T21:54:27.401 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.397+0000 7f20a9bdfb80 1 -- 172.21.15.151:0/2328899571 shutdown_connections 2024-05-21T21:54:27.401 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.397+0000 7f20a9bdfb80 1 --2- 172.21.15.151:0/2328899571 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x55cd51ed2450 0x55cd51ed1160 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:27.401 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.397+0000 7f20a9bdfb80 1 --2- 172.21.15.151:0/2328899571 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cd51e45780 0x55cd51e67220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:27.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.397+0000 7f20a9bdfb80 1 --2- 172.21.15.151:0/2328899571 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cd51e3f830 0x55cd51e6c1f0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:27.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.397+0000 7f20a9bdfb80 1 --2- 172.21.15.151:0/2328899571 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cd51e44e60 0x55cd51e66ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:27.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.397+0000 7f20a9bdfb80 1 -- 172.21.15.151:0/2328899571 >> 172.21.15.151:0/2328899571 conn(0x55cd51d6db90 msgr2=0x55cd51e42970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:54:27.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.397+0000 7f2082000700 0 client.14835 ms_handle_reset on v2:172.21.15.184:6838/3059935143 2024-05-21T21:54:27.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.397+0000 7f20a9bdfb80 1 -- 172.21.15.151:0/2328899571 shutdown_connections 2024-05-21T21:54:27.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.397+0000 7f20a9bdfb80 1 -- 172.21.15.151:0/2328899571 wait complete. 2024-05-21T21:54:27.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.397+0000 7f20a9bdfb80 20 client.14835 trim_cache size 0 max 16384 2024-05-21T21:54:27.402 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-05-21T21:54:27.402 INFO:teuthology.orchestra.run.smithi151.stdout: "cluster_fsid": "c8c7a2e6-17ba-11ef-bc9a-c7b262605968", 2024-05-21T21:54:27.402 INFO:teuthology.orchestra.run.smithi151.stdout: "whoami": 1, 2024-05-21T21:54:27.403 INFO:teuthology.orchestra.run.smithi151.stdout: "id": 24493, 2024-05-21T21:54:27.403 INFO:teuthology.orchestra.run.smithi151.stdout: "want_state": "up:active", 2024-05-21T21:54:27.403 INFO:teuthology.orchestra.run.smithi151.stdout: "state": "up:active", 2024-05-21T21:54:27.403 INFO:teuthology.orchestra.run.smithi151.stdout: "fs_name": "cephfs", 2024-05-21T21:54:27.403 INFO:teuthology.orchestra.run.smithi151.stdout: "rank_uptime": 172.099131478, 2024-05-21T21:54:27.403 INFO:teuthology.orchestra.run.smithi151.stdout: "mdsmap_epoch": 28, 2024-05-21T21:54:27.403 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_epoch": 82, 2024-05-21T21:54:27.403 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_epoch_barrier": 80, 2024-05-21T21:54:27.403 INFO:teuthology.orchestra.run.smithi151.stdout: "uptime": 220.46684267399999 2024-05-21T21:54:27.403 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-05-21T21:54:27.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.398+0000 7f209f200700 1 -- 172.21.15.151:0/3328263652 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f207c0b2900 msgr2=0x7f207c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:27.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.398+0000 7f209f200700 1 --2- 172.21.15.151:0/3328263652 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f207c0b2900 0x7f207c0b4dc0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f209000bfa0 tx=0x7f209001d040 comp rx=0 tx=0).stop 2024-05-21T21:54:27.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.398+0000 7f209f200700 1 -- 172.21.15.151:0/3328263652 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f209806bbe0 msgr2=0x7f209819c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:27.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.398+0000 7f209f200700 1 --2- 172.21.15.151:0/3328263652 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f209806bbe0 0x7f209819c3d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f208400d750 tx=0x7f208400da60 comp rx=0 tx=0).stop 2024-05-21T21:54:27.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.398+0000 7f209f200700 1 -- 172.21.15.151:0/3328263652 shutdown_connections 2024-05-21T21:54:27.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.398+0000 7f209f200700 1 --2- 172.21.15.151:0/3328263652 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f207c0b2900 0x7f207c0b4dc0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:27.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.398+0000 7f209f200700 1 --2- 172.21.15.151:0/3328263652 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f20981095b0 0x7f20981a0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:27.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.398+0000 7f209f200700 1 --2- 172.21.15.151:0/3328263652 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f209806b220 0x7f209819be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:27.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.398+0000 7f209f200700 1 --2- 172.21.15.151:0/3328263652 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f209806bbe0 0x7f209819c3d0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:27.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.398+0000 7f209f200700 1 -- 172.21.15.151:0/3328263652 >> 172.21.15.151:0/3328263652 conn(0x7f20980793f0 msgr2=0x7f209810e720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:54:27.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.398+0000 7f209f200700 1 -- 172.21.15.151:0/3328263652 shutdown_connections 2024-05-21T21:54:27.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:27.398+0000 7f209f200700 1 -- 172.21.15.151:0/3328263652 wait complete. 2024-05-21T21:54:28.265 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.b status 2024-05-21T21:54:28.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:54:28 smithi151 ceph-mon[31623]: pgmap v411: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-21T21:54:28.705 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:54:28.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:54:28 smithi184 ceph-mon[36942]: pgmap v411: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-21T21:54:28.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:54:28 smithi164 ceph-mon[36269]: pgmap v411: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-21T21:54:30.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:54:30 smithi151 ceph-mon[31623]: pgmap v412: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-21T21:54:30.710 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.707+0000 7f0e3ea00700 1 -- 172.21.15.151:0/3811815617 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e3810eeb0 msgr2=0x7f0e38116a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:30.710 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.707+0000 7f0e3ea00700 1 --2- 172.21.15.151:0/3811815617 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e3810eeb0 0x7f0e38116a80 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0e3000b0a0 tx=0x7f0e3000b3b0 comp rx=0 tx=0).stop 2024-05-21T21:54:30.710 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.707+0000 7f0e3ea00700 1 -- 172.21.15.151:0/3811815617 shutdown_connections 2024-05-21T21:54:30.710 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.707+0000 7f0e3ea00700 1 --2- 172.21.15.151:0/3811815617 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e3810eeb0 0x7f0e38116a80 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:30.710 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.707+0000 7f0e3ea00700 1 --2- 172.21.15.151:0/3811815617 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0e38109dd0 0x7f0e3810a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:30.710 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.707+0000 7f0e3ea00700 1 --2- 172.21.15.151:0/3811815617 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0e3810a780 0x7f0e3810e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:30.710 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.707+0000 7f0e3ea00700 1 -- 172.21.15.151:0/3811815617 >> 172.21.15.151:0/3811815617 conn(0x7f0e38076d10 msgr2=0x7f0e38108360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:54:30.710 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.708+0000 7f0e3ea00700 1 -- 172.21.15.151:0/3811815617 shutdown_connections 2024-05-21T21:54:30.710 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.708+0000 7f0e3ea00700 1 -- 172.21.15.151:0/3811815617 wait complete. 2024-05-21T21:54:30.711 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.709+0000 7f0e3ea00700 1 Processor -- start 2024-05-21T21:54:30.711 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.709+0000 7f0e3ea00700 1 -- start start 2024-05-21T21:54:30.712 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.709+0000 7f0e3ea00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e38109dd0 0x7f0e381a0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:30.712 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.709+0000 7f0e3ea00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0e3810a780 0x7f0e381a07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:30.712 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.709+0000 7f0e3ea00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0e3810eeb0 0x7f0e381a4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:30.712 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.709+0000 7f0e3ea00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0e38119d90 con 0x7f0e3810eeb0 2024-05-21T21:54:30.712 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.709+0000 7f0e3ea00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0e38119c10 con 0x7f0e38109dd0 2024-05-21T21:54:30.712 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.709+0000 7f0e3ea00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0e38119f10 con 0x7f0e3810a780 2024-05-21T21:54:30.713 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.709+0000 7f0e3ca00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0e3810eeb0 0x7f0e381a4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:30.713 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.709+0000 7f0e3ca00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0e3810eeb0 0x7f0e381a4ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:39884/0 (socket says 172.21.15.151:39884) 2024-05-21T21:54:30.713 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.710+0000 7f0e3ca00700 1 -- 172.21.15.151:0/305172725 learned_addr learned my addr 172.21.15.151:0/305172725 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:54:30.713 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.710+0000 7f0e37e00700 1 --2- 172.21.15.151:0/305172725 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e38109dd0 0x7f0e381a0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:30.713 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.710+0000 7f0e37400700 1 --2- 172.21.15.151:0/305172725 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0e3810a780 0x7f0e381a07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:30.713 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.710+0000 7f0e3ca00700 1 -- 172.21.15.151:0/305172725 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0e3810a780 msgr2=0x7f0e381a07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:30.713 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.710+0000 7f0e3ca00700 1 --2- 172.21.15.151:0/305172725 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0e3810a780 0x7f0e381a07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:30.713 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.710+0000 7f0e3ca00700 1 -- 172.21.15.151:0/305172725 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e38109dd0 msgr2=0x7f0e381a0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:30.713 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.710+0000 7f0e3ca00700 1 --2- 172.21.15.151:0/305172725 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e38109dd0 0x7f0e381a0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:30.713 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.710+0000 7f0e3ca00700 1 -- 172.21.15.151:0/305172725 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e381a5220 con 0x7f0e3810eeb0 2024-05-21T21:54:30.713 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.710+0000 7f0e37e00700 1 --2- 172.21.15.151:0/305172725 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e38109dd0 0x7f0e381a0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:54:30.713 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.710+0000 7f0e3ca00700 1 --2- 172.21.15.151:0/305172725 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0e3810eeb0 0x7f0e381a4ae0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0e3000b070 tx=0x7f0e30004c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:54:30.713 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.711+0000 7f0e34c00700 1 -- 172.21.15.151:0/305172725 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e30021070 con 0x7f0e3810eeb0 2024-05-21T21:54:30.714 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.711+0000 7f0e34c00700 1 -- 172.21.15.151:0/305172725 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e30003c00 con 0x7f0e3810eeb0 2024-05-21T21:54:30.714 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.711+0000 7f0e3ea00700 1 -- 172.21.15.151:0/305172725 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e381a54b0 con 0x7f0e3810eeb0 2024-05-21T21:54:30.714 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.711+0000 7f0e34c00700 1 -- 172.21.15.151:0/305172725 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e30004640 con 0x7f0e3810eeb0 2024-05-21T21:54:30.714 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.711+0000 7f0e3ea00700 1 -- 172.21.15.151:0/305172725 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e381acd90 con 0x7f0e3810eeb0 2024-05-21T21:54:30.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.714+0000 7f0e34c00700 1 -- 172.21.15.151:0/305172725 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f0e30007980 con 0x7f0e3810eeb0 2024-05-21T21:54:30.717 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.716+0000 7f0e493a4b80 1 Processor -- start 2024-05-21T21:54:30.717 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.716+0000 7f0e493a4b80 1 -- start start 2024-05-21T21:54:30.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.717+0000 7f0e493a4b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bef2e31e60 0x55bef2e32240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:30.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.717+0000 7f0e493a4b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bef2e32780 0x55bef2e2c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:30.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.717+0000 7f0e493a4b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bef2e2c830 0x55bef2e2cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:30.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.717+0000 7f0e493a4b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55bef2d8adf0 con 0x55bef2e2c830 2024-05-21T21:54:30.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.717+0000 7f0e493a4b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55bef2d7a990 con 0x55bef2e31e60 2024-05-21T21:54:30.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.717+0000 7f0e493a4b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55bef2d7e9d0 con 0x55bef2e32780 2024-05-21T21:54:30.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.717+0000 7f0e34c00700 1 --2- 172.21.15.151:0/305172725 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f0e1c0b2880 0x7f0e1c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:30.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.717+0000 7f0e37400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bef2e31e60 0x55bef2e32240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:30.719 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.717+0000 7f0e34c00700 1 -- 172.21.15.151:0/305172725 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f0e300e33f0 con 0x7f0e3810eeb0 2024-05-21T21:54:30.719 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.717+0000 7f0e37e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bef2e32780 0x55bef2e2c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:30.720 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.717+0000 7f0e37e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bef2e32780 0x55bef2e2c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:52558/0 (socket says 172.21.15.151:52558) 2024-05-21T21:54:30.720 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.717+0000 7f0e37e00700 1 -- 172.21.15.151:0/1170142526 learned_addr learned my addr 172.21.15.151:0/1170142526 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:54:30.720 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.717+0000 7f0e37400700 1 --2- 172.21.15.151:0/1170142526 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bef2e2c830 0x55bef2e2cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:30.720 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.717+0000 7f0e3ca00700 1 --2- 172.21.15.151:0/305172725 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f0e1c0b2880 0x7f0e1c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:30.720 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.718+0000 7f0e37e00700 1 -- 172.21.15.151:0/1170142526 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bef2e31e60 msgr2=0x55bef2e32240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:54:30.720 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.718+0000 7f0e37e00700 1 --2- 172.21.15.151:0/1170142526 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bef2e31e60 0x55bef2e32240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:30.720 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.718+0000 7f0e37e00700 1 -- 172.21.15.151:0/1170142526 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bef2e2c830 msgr2=0x55bef2e2cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:54:30.720 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.718+0000 7f0e37e00700 1 --2- 172.21.15.151:0/1170142526 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bef2e2c830 0x55bef2e2cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:30.720 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.718+0000 7f0e37e00700 1 -- 172.21.15.151:0/1170142526 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bef2d433d0 con 0x55bef2e32780 2024-05-21T21:54:30.720 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.718+0000 7f0e37e00700 1 --2- 172.21.15.151:0/1170142526 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bef2e32780 0x55bef2e2c210 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0e280115f0 tx=0x7f0e28011900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f81924dc95254ddc server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:54:30.720 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.718+0000 7f0e3ca00700 1 --2- 172.21.15.151:0/305172725 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f0e1c0b2880 0x7f0e1c0b4d40 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0e300a7cd0 tx=0x7f0e300099e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:54:30.720 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.718+0000 7f0e2d600700 1 -- 172.21.15.151:0/1170142526 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e2801e070 con 0x55bef2e32780 2024-05-21T21:54:30.720 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.718+0000 7f0e2d600700 1 -- 172.21.15.151:0/1170142526 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e28019ae0 con 0x55bef2e32780 2024-05-21T21:54:30.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.718+0000 7f0e2d600700 1 -- 172.21.15.151:0/1170142526 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e2800ea00 con 0x55bef2e32780 2024-05-21T21:54:30.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.719+0000 7f0e493a4b80 1 -- 172.21.15.151:0/1170142526 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bef2e32780 msgr2=0x55bef2e2c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:30.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.719+0000 7f0e493a4b80 1 --2- 172.21.15.151:0/1170142526 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bef2e32780 0x55bef2e2c210 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0e280115f0 tx=0x7f0e28011900 comp rx=0 tx=0).stop 2024-05-21T21:54:30.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.719+0000 7f0e493a4b80 1 -- 172.21.15.151:0/1170142526 shutdown_connections 2024-05-21T21:54:30.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.719+0000 7f0e493a4b80 1 --2- 172.21.15.151:0/1170142526 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bef2e31e60 0x55bef2e32240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:30.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.719+0000 7f0e493a4b80 1 --2- 172.21.15.151:0/1170142526 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bef2e2c830 0x55bef2e2cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:30.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.719+0000 7f0e493a4b80 1 --2- 172.21.15.151:0/1170142526 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bef2e32780 0x55bef2e2c210 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:30.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.719+0000 7f0e493a4b80 1 -- 172.21.15.151:0/1170142526 >> 172.21.15.151:0/1170142526 conn(0x55bef2d5ab90 msgr2=0x55bef2e26fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:54:30.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.720+0000 7f0e493a4b80 1 -- 172.21.15.151:0/1170142526 shutdown_connections 2024-05-21T21:54:30.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.720+0000 7f0e493a4b80 1 -- 172.21.15.151:0/1170142526 wait complete. 2024-05-21T21:54:30.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.720+0000 7f0e493a4b80 1 -- start start 2024-05-21T21:54:30.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.720+0000 7f0e493a4b80 1 Processor -- start 2024-05-21T21:54:30.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.721+0000 7f0e493a4b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bef2e2c830 0x55bef2e38460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:30.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.721+0000 7f0e493a4b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bef2e31e60 0x55bef2e35850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:30.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.721+0000 7f0e493a4b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bef2e32780 0x55bef2e35d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:30.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.721+0000 7f0e493a4b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55bef2c80f50 con 0x55bef2e31e60 2024-05-21T21:54:30.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.721+0000 7f0e493a4b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55bef2d1b710 con 0x55bef2e32780 2024-05-21T21:54:30.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.721+0000 7f0e493a4b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55bef2e391f0 con 0x55bef2e2c830 2024-05-21T21:54:30.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.721+0000 7f0e493a4b80 10 client.0 _ll_register_callbacks cb 0x55bef2d24c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:54:30.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.721+0000 7f0e37400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bef2e2c830 0x55bef2e38460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:30.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.721+0000 7f0e37e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bef2e31e60 0x55bef2e35850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:30.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.721+0000 7f0e37400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bef2e2c830 0x55bef2e38460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:52570/0 (socket says 172.21.15.151:52570) 2024-05-21T21:54:30.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.721+0000 7f0e37400700 1 -- 172.21.15.151:0/3214302139 learned_addr learned my addr 172.21.15.151:0/3214302139 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:54:30.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.721+0000 7f0e37400700 1 --2- 172.21.15.151:0/3214302139 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bef2e32780 0x55bef2e35d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:30.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.722+0000 7f0e37400700 1 -- 172.21.15.151:0/3214302139 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bef2e32780 msgr2=0x55bef2e35d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:54:30.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.722+0000 7f0e37400700 1 --2- 172.21.15.151:0/3214302139 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bef2e32780 0x55bef2e35d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:30.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.722+0000 7f0e37400700 1 -- 172.21.15.151:0/3214302139 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bef2e31e60 msgr2=0x55bef2e35850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:54:30.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.722+0000 7f0e37400700 1 --2- 172.21.15.151:0/3214302139 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bef2e31e60 0x55bef2e35850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:30.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.722+0000 7f0e37400700 1 -- 172.21.15.151:0/3214302139 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bef2cca880 con 0x55bef2e2c830 2024-05-21T21:54:30.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.722+0000 7f0e37400700 1 --2- 172.21.15.151:0/3214302139 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bef2e2c830 0x55bef2e38460 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0e24013ed0 tx=0x7f0e24009bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=299cc5e0c093c260 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:54:30.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.722+0000 7f0e22000700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-21T21:54:30.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.722+0000 7f0e22000700 1 -- 172.21.15.151:0/3214302139 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e24011c40 con 0x55bef2e2c830 2024-05-21T21:54:30.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.722+0000 7f0e22000700 1 -- 172.21.15.151:0/3214302139 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f0e24009040 con 0x55bef2e2c830 2024-05-21T21:54:30.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.722+0000 7f0e493a4b80 1 -- 172.21.15.151:0/3214302139 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e28019ae0 con 0x55bef2e2c830 2024-05-21T21:54:30.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.723+0000 7f0e22000700 1 -- 172.21.15.151:0/3214302139 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0e24011de0 con 0x55bef2e2c830 2024-05-21T21:54:30.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.723+0000 7f0e493a4b80 10 client.24961 fetch_fsmap learned FSMap version 28 2024-05-21T21:54:30.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.723+0000 7f0e493a4b80 1 -- 172.21.15.151:0/3214302139 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bef2d4b010 con 0x55bef2e2c830 2024-05-21T21:54:30.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.723+0000 7f0e22000700 1 -- 172.21.15.151:0/3214302139 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e28019ae0 con 0x55bef2e2c830 2024-05-21T21:54:30.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.723+0000 7f0e22000700 1 -- 172.21.15.151:0/3214302139 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f0e24005380 con 0x55bef2e2c830 2024-05-21T21:54:30.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.723+0000 7f0e493a4b80 10 client.24961 fetch_fsmap finished waiting for FSMap version 28 2024-05-21T21:54:30.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.723+0000 7f0e493a4b80 10 client.24961 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-21T21:54:30.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.723+0000 7f0e493a4b80 20 client.24961 populate_metadata read hostname 'smithi151' 2024-05-21T21:54:30.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.723+0000 7f0e493a4b80 1 --2- 172.21.15.151:0/3214302139 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55bef2e45c90 0x55bef2e47100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:30.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.723+0000 7f0e493a4b80 4 client.24961 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T21:54:30.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.723+0000 7f0e493a4b80 1 -- 172.21.15.151:0/3214302139 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bef2d8cd40 con 0x55bef2e45c90 2024-05-21T21:54:30.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.724+0000 7f0e37e00700 1 --2- 172.21.15.151:0/3214302139 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55bef2e45c90 0x55bef2e47100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:30.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.724+0000 7f0e37e00700 1 --2- 172.21.15.151:0/3214302139 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55bef2e45c90 0x55bef2e47100 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48ac0ddb8cd4a517 server_cookie=8c125ee0e3deeb43 in_seq=0 out_seq=0 2024-05-21T21:54:30.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.724+0000 7f0e22000700 10 client.24961 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T21:54:30.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.725+0000 7f0e22000700 1 -- 172.21.15.151:0/3214302139 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55bef2d8cd40 con 0x55bef2e45c90 2024-05-21T21:54:30.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.725+0000 7f0e22000700 10 client.24961 handle_command_reply: tid=0 2024-05-21T21:54:30.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.725+0000 7f0e493a4b80 1 client.24961 shutdown 2024-05-21T21:54:30.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.726+0000 7f0e493a4b80 1 -- 172.21.15.151:0/3214302139 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bef2e2c830 msgr2=0x55bef2e38460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:30.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.726+0000 7f0e493a4b80 1 --2- 172.21.15.151:0/3214302139 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bef2e2c830 0x55bef2e38460 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0e24013ed0 tx=0x7f0e24009bb0 comp rx=0 tx=0).stop 2024-05-21T21:54:30.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.726+0000 7f0e493a4b80 1 -- 172.21.15.151:0/3214302139 shutdown_connections 2024-05-21T21:54:30.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.726+0000 7f0e493a4b80 1 --2- 172.21.15.151:0/3214302139 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55bef2e45c90 0x55bef2e47100 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:30.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.726+0000 7f0e493a4b80 1 --2- 172.21.15.151:0/3214302139 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bef2e32780 0x55bef2e35d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:30.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.726+0000 7f0e22000700 0 client.24961 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T21:54:30.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.726+0000 7f0e493a4b80 1 --2- 172.21.15.151:0/3214302139 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bef2e2c830 0x55bef2e38460 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:30.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.726+0000 7f0e493a4b80 1 --2- 172.21.15.151:0/3214302139 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bef2e31e60 0x55bef2e35850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:30.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.726+0000 7f0e493a4b80 1 -- 172.21.15.151:0/3214302139 >> 172.21.15.151:0/3214302139 conn(0x55bef2d5ab90 msgr2=0x55bef2e2b200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:54:30.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.726+0000 7f0e493a4b80 1 -- 172.21.15.151:0/3214302139 shutdown_connections 2024-05-21T21:54:30.731 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.726+0000 7f0e493a4b80 1 -- 172.21.15.151:0/3214302139 wait complete. 2024-05-21T21:54:30.731 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.726+0000 7f0e493a4b80 20 client.24961 trim_cache size 0 max 16384 2024-05-21T21:54:30.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:54:30 smithi184 ceph-mon[36942]: pgmap v412: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-21T21:54:30.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.749+0000 7f0e493a4b80 1 Processor -- start 2024-05-21T21:54:30.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.749+0000 7f0e493a4b80 1 -- start start 2024-05-21T21:54:30.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.750+0000 7f0e493a4b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bef2e2c830 0x55bef2e2c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:30.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.750+0000 7f0e493a4b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bef2e31e60 0x55bef2e455c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:30.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.750+0000 7f0e493a4b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bef2e32780 0x55bef2e37ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:30.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.750+0000 7f0e493a4b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55bef2e45440 con 0x55bef2e31e60 2024-05-21T21:54:30.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.750+0000 7f0e493a4b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55bef2e45b10 con 0x55bef2e32780 2024-05-21T21:54:30.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.750+0000 7f0e493a4b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55bef2e461e0 con 0x55bef2e2c830 2024-05-21T21:54:30.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.750+0000 7f0e37400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bef2e32780 0x55bef2e37ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:30.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.750+0000 7f0e37400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bef2e32780 0x55bef2e37ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:52282/0 (socket says 172.21.15.151:52282) 2024-05-21T21:54:30.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.750+0000 7f0e37400700 1 -- 172.21.15.151:0/224333695 learned_addr learned my addr 172.21.15.151:0/224333695 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:54:30.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.750+0000 7f0e37400700 1 --2- 172.21.15.151:0/224333695 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bef2e2c830 0x55bef2e2c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:30.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.750+0000 7f0e37e00700 1 --2- 172.21.15.151:0/224333695 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bef2e31e60 0x55bef2e455c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:30.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.751+0000 7f0e37400700 1 -- 172.21.15.151:0/224333695 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bef2e2c830 msgr2=0x55bef2e2c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:54:30.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.751+0000 7f0e37400700 1 --2- 172.21.15.151:0/224333695 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bef2e2c830 0x55bef2e2c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:30.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.751+0000 7f0e37400700 1 -- 172.21.15.151:0/224333695 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bef2e31e60 msgr2=0x55bef2e455c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:54:30.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.751+0000 7f0e37400700 1 --2- 172.21.15.151:0/224333695 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bef2e31e60 0x55bef2e455c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:30.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.751+0000 7f0e37400700 1 -- 172.21.15.151:0/224333695 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bef2e27ef0 con 0x55bef2e32780 2024-05-21T21:54:30.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.751+0000 7f0e37e00700 1 --2- 172.21.15.151:0/224333695 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bef2e31e60 0x55bef2e455c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T21:54:30.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.751+0000 7f0e37400700 1 --2- 172.21.15.151:0/224333695 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bef2e32780 0x55bef2e37ca0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0e24009770 tx=0x7f0e24001e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac25a81b07eb2693 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:54:30.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.751+0000 7f0e2cc00700 1 -- 172.21.15.151:0/224333695 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e24022ac0 con 0x55bef2e32780 2024-05-21T21:54:30.754 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.751+0000 7f0e2cc00700 1 -- 172.21.15.151:0/224333695 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e24013440 con 0x55bef2e32780 2024-05-21T21:54:30.754 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.751+0000 7f0e2cc00700 1 -- 172.21.15.151:0/224333695 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e2402dc90 con 0x55bef2e32780 2024-05-21T21:54:30.756 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.752+0000 7f0e493a4b80 1 -- 172.21.15.151:0/224333695 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bef2e32780 msgr2=0x55bef2e37ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:30.756 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.752+0000 7f0e493a4b80 1 --2- 172.21.15.151:0/224333695 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bef2e32780 0x55bef2e37ca0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0e24009770 tx=0x7f0e24001e70 comp rx=0 tx=0).stop 2024-05-21T21:54:30.756 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.752+0000 7f0e493a4b80 1 -- 172.21.15.151:0/224333695 shutdown_connections 2024-05-21T21:54:30.756 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.752+0000 7f0e493a4b80 1 --2- 172.21.15.151:0/224333695 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bef2e32780 0x55bef2e37ca0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:30.757 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.752+0000 7f0e493a4b80 1 --2- 172.21.15.151:0/224333695 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bef2e2c830 0x55bef2e2c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:30.757 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.752+0000 7f0e493a4b80 1 --2- 172.21.15.151:0/224333695 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bef2e31e60 0x55bef2e455c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:30.757 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.752+0000 7f0e493a4b80 1 -- 172.21.15.151:0/224333695 >> 172.21.15.151:0/224333695 conn(0x55bef2d5ab90 msgr2=0x55bef2e2f970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:54:30.757 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.752+0000 7f0e493a4b80 1 -- 172.21.15.151:0/224333695 shutdown_connections 2024-05-21T21:54:30.757 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.752+0000 7f0e493a4b80 1 -- 172.21.15.151:0/224333695 wait complete. 2024-05-21T21:54:30.757 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.753+0000 7f0e493a4b80 1 -- start start 2024-05-21T21:54:30.757 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.753+0000 7f0e493a4b80 1 Processor -- start 2024-05-21T21:54:30.757 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.754+0000 7f0e493a4b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bef2e2c830 0x55bef2e50580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:30.757 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.754+0000 7f0e493a4b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bef2e31e60 0x55bef2e50ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:30.757 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.754+0000 7f0e493a4b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bef2e32780 0x55bef2e4b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:30.757 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.754+0000 7f0e493a4b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55bef2e46f80 con 0x55bef2e31e60 2024-05-21T21:54:30.757 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.754+0000 7f0e493a4b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55bef2e468b0 con 0x55bef2e32780 2024-05-21T21:54:30.757 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.754+0000 7f0e493a4b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55bef2e47650 con 0x55bef2e2c830 2024-05-21T21:54:30.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.754+0000 7f0e493a4b80 10 client.0 _ll_register_callbacks cb 0x55bef2d24c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:54:30.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.754+0000 7f0e37e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bef2e31e60 0x55bef2e50ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:30.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.754+0000 7f0e37e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bef2e31e60 0x55bef2e50ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:39918/0 (socket says 172.21.15.151:39918) 2024-05-21T21:54:30.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.754+0000 7f0e37e00700 1 -- 172.21.15.151:0/3188740147 learned_addr learned my addr 172.21.15.151:0/3188740147 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:54:30.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.754+0000 7f0e37400700 1 --2- 172.21.15.151:0/3188740147 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bef2e2c830 0x55bef2e50580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:30.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.754+0000 7f0e37400700 1 --2- 172.21.15.151:0/3188740147 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bef2e32780 0x55bef2e4b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:30.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.754+0000 7f0e37e00700 1 -- 172.21.15.151:0/3188740147 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bef2e2c830 msgr2=0x55bef2e50580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:54:30.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.754+0000 7f0e37e00700 1 --2- 172.21.15.151:0/3188740147 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bef2e2c830 0x55bef2e50580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:30.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.754+0000 7f0e37e00700 1 -- 172.21.15.151:0/3188740147 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bef2e32780 msgr2=0x55bef2e4b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:54:30.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.754+0000 7f0e37e00700 1 --2- 172.21.15.151:0/3188740147 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bef2e32780 0x55bef2e4b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:30.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.754+0000 7f0e37e00700 1 -- 172.21.15.151:0/3188740147 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bef2e4bac0 con 0x55bef2e31e60 2024-05-21T21:54:30.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.754+0000 7f0e37400700 1 --2- 172.21.15.151:0/3188740147 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bef2e32780 0x55bef2e4b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T21:54:30.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.755+0000 7f0e37e00700 1 --2- 172.21.15.151:0/3188740147 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bef2e31e60 0x55bef2e50ac0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0e28000c00 tx=0x7f0e28005e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2753fb5ad7e6a34c server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:54:30.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.755+0000 7f0e22000700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-21T21:54:30.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.755+0000 7f0e22000700 1 -- 172.21.15.151:0/3188740147 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e2801e040 con 0x55bef2e31e60 2024-05-21T21:54:30.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.755+0000 7f0e493a4b80 1 -- 172.21.15.151:0/3188740147 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55bef2d86520 con 0x55bef2e31e60 2024-05-21T21:54:30.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.755+0000 7f0e22000700 1 -- 172.21.15.151:0/3188740147 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f0e28005250 con 0x55bef2e31e60 2024-05-21T21:54:30.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.755+0000 7f0e22000700 1 -- 172.21.15.151:0/3188740147 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0e28005490 con 0x55bef2e31e60 2024-05-21T21:54:30.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.755+0000 7f0e493a4b80 10 client.14853 fetch_fsmap learned FSMap version 28 2024-05-21T21:54:30.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.755+0000 7f0e493a4b80 1 -- 172.21.15.151:0/3188740147 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bef2eb1ca0 con 0x55bef2e31e60 2024-05-21T21:54:30.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.756+0000 7f0e22000700 1 -- 172.21.15.151:0/3188740147 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55bef2d86520 con 0x55bef2e31e60 2024-05-21T21:54:30.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.756+0000 7f0e22000700 1 -- 172.21.15.151:0/3188740147 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f0e28038050 con 0x55bef2e31e60 2024-05-21T21:54:30.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.756+0000 7f0e493a4b80 10 client.14853 fetch_fsmap finished waiting for FSMap version 28 2024-05-21T21:54:30.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.756+0000 7f0e493a4b80 10 client.14853 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-21T21:54:30.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.756+0000 7f0e493a4b80 20 client.14853 populate_metadata read hostname 'smithi151' 2024-05-21T21:54:30.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.756+0000 7f0e493a4b80 1 --2- 172.21.15.151:0/3188740147 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55bef2eb66d0 0x55bef2eb53e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:30.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.756+0000 7f0e493a4b80 4 client.14853 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-21T21:54:30.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.756+0000 7f0e493a4b80 1 -- 172.21.15.151:0/3188740147 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "status"}) -- 0x55bef2bfd200 con 0x55bef2eb66d0 2024-05-21T21:54:30.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.756+0000 7f0e37400700 1 --2- 172.21.15.151:0/3188740147 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55bef2eb66d0 0x55bef2eb53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:30.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.757+0000 7f0e37400700 1 --2- 172.21.15.151:0/3188740147 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55bef2eb66d0 0x55bef2eb53e0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=890132fbb771c746 server_cookie=dc304140575a507e in_seq=0 out_seq=0 2024-05-21T21:54:30.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.757+0000 7f0e22000700 10 client.14853 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T21:54:30.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.757+0000 7f0e22000700 1 -- 172.21.15.151:0/3188740147 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55bef2bfd200 con 0x55bef2eb66d0 2024-05-21T21:54:30.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.757+0000 7f0e22000700 10 client.14853 handle_command_reply: tid=0 2024-05-21T21:54:30.762 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.757+0000 7f0e493a4b80 1 client.14853 shutdown 2024-05-21T21:54:30.762 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.757+0000 7f0e493a4b80 1 -- 172.21.15.151:0/3188740147 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bef2e31e60 msgr2=0x55bef2e50ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:30.762 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.757+0000 7f0e493a4b80 1 --2- 172.21.15.151:0/3188740147 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bef2e31e60 0x55bef2e50ac0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0e28000c00 tx=0x7f0e28005e00 comp rx=0 tx=0).stop 2024-05-21T21:54:30.762 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.757+0000 7f0e493a4b80 1 -- 172.21.15.151:0/3188740147 shutdown_connections 2024-05-21T21:54:30.762 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.757+0000 7f0e493a4b80 1 --2- 172.21.15.151:0/3188740147 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55bef2eb66d0 0x55bef2eb53e0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:30.766 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.757+0000 7f0e493a4b80 1 --2- 172.21.15.151:0/3188740147 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bef2e32780 0x55bef2e4b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:30.766 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.757+0000 7f0e493a4b80 1 --2- 172.21.15.151:0/3188740147 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bef2e2c830 0x55bef2e50580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:30.766 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.757+0000 7f0e493a4b80 1 --2- 172.21.15.151:0/3188740147 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bef2e31e60 0x55bef2e50ac0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:30.766 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.757+0000 7f0e493a4b80 1 -- 172.21.15.151:0/3188740147 >> 172.21.15.151:0/3188740147 conn(0x55bef2d5ab90 msgr2=0x55bef2e33fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:54:30.766 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.757+0000 7f0e22000700 0 client.14853 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T21:54:30.766 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.758+0000 7f0e493a4b80 1 -- 172.21.15.151:0/3188740147 shutdown_connections 2024-05-21T21:54:30.766 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.758+0000 7f0e493a4b80 1 -- 172.21.15.151:0/3188740147 wait complete. 2024-05-21T21:54:30.766 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.758+0000 7f0e493a4b80 20 client.14853 trim_cache size 0 max 16384 2024-05-21T21:54:30.767 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-05-21T21:54:30.767 INFO:teuthology.orchestra.run.smithi151.stdout: "cluster_fsid": "c8c7a2e6-17ba-11ef-bc9a-c7b262605968", 2024-05-21T21:54:30.767 INFO:teuthology.orchestra.run.smithi151.stdout: "whoami": 0, 2024-05-21T21:54:30.767 INFO:teuthology.orchestra.run.smithi151.stdout: "id": 24530, 2024-05-21T21:54:30.767 INFO:teuthology.orchestra.run.smithi151.stdout: "want_state": "up:active", 2024-05-21T21:54:30.767 INFO:teuthology.orchestra.run.smithi151.stdout: "state": "up:active", 2024-05-21T21:54:30.767 INFO:teuthology.orchestra.run.smithi151.stdout: "fs_name": "cephfs", 2024-05-21T21:54:30.767 INFO:teuthology.orchestra.run.smithi151.stdout: "rank_uptime": 190.82223810900001, 2024-05-21T21:54:30.767 INFO:teuthology.orchestra.run.smithi151.stdout: "mdsmap_epoch": 28, 2024-05-21T21:54:30.767 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_epoch": 82, 2024-05-21T21:54:30.768 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_epoch_barrier": 80, 2024-05-21T21:54:30.768 INFO:teuthology.orchestra.run.smithi151.stdout: "uptime": 211.76639936699999 2024-05-21T21:54:30.768 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-05-21T21:54:30.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.758+0000 7f0e3ea00700 1 -- 172.21.15.151:0/305172725 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f0e1c0b2880 msgr2=0x7f0e1c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:30.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.758+0000 7f0e3ea00700 1 --2- 172.21.15.151:0/305172725 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f0e1c0b2880 0x7f0e1c0b4d40 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0e300a7cd0 tx=0x7f0e300099e0 comp rx=0 tx=0).stop 2024-05-21T21:54:30.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.758+0000 7f0e3ea00700 1 -- 172.21.15.151:0/305172725 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0e3810eeb0 msgr2=0x7f0e381a4ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:30.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.758+0000 7f0e3ea00700 1 --2- 172.21.15.151:0/305172725 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0e3810eeb0 0x7f0e381a4ae0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0e3000b070 tx=0x7f0e30004c60 comp rx=0 tx=0).stop 2024-05-21T21:54:30.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.759+0000 7f0e3ea00700 1 -- 172.21.15.151:0/305172725 shutdown_connections 2024-05-21T21:54:30.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.759+0000 7f0e3ea00700 1 --2- 172.21.15.151:0/305172725 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f0e1c0b2880 0x7f0e1c0b4d40 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:30.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.759+0000 7f0e3ea00700 1 --2- 172.21.15.151:0/305172725 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e38109dd0 0x7f0e381a0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:30.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.759+0000 7f0e3ea00700 1 --2- 172.21.15.151:0/305172725 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0e3810eeb0 0x7f0e381a4ae0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:30.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.759+0000 7f0e3ea00700 1 --2- 172.21.15.151:0/305172725 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0e3810a780 0x7f0e381a07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:30.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.759+0000 7f0e3ea00700 1 -- 172.21.15.151:0/305172725 >> 172.21.15.151:0/305172725 conn(0x7f0e38076d10 msgr2=0x7f0e380772d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:54:30.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.759+0000 7f0e3ea00700 1 -- 172.21.15.151:0/305172725 shutdown_connections 2024-05-21T21:54:30.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:30.759+0000 7f0e3ea00700 1 -- 172.21.15.151:0/305172725 wait complete. 2024-05-21T21:54:30.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:54:30 smithi164 ceph-mon[36269]: pgmap v412: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-21T21:54:31.602 INFO:tasks.cephadm:Setup complete, yielding 2024-05-21T21:54:31.602 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-21T21:54:31.612 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi151.front.sepia.ceph.com 2024-05-21T21:54:31.612 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- bash -c 'ceph orch status' 2024-05-21T21:54:32.080 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:54:32.518 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:54:32 smithi151 ceph-mon[31623]: pgmap v413: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-21T21:54:32.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:54:32 smithi184 ceph-mon[36942]: pgmap v413: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-21T21:54:32.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:54:32 smithi164 ceph-mon[36269]: pgmap v413: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-21T21:54:34.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:34.117+0000 7f8a0be00700 1 -- 172.21.15.151:0/1056021031 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a041047f0 msgr2=0x7f8a04104c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:34.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:34.117+0000 7f8a0be00700 1 --2- 172.21.15.151:0/1056021031 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a041047f0 0x7f8a04104c50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f89f00098f0 tx=0x7f89f0009c00 comp rx=0 tx=0).stop 2024-05-21T21:54:34.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:34.117+0000 7f8a0be00700 1 -- 172.21.15.151:0/1056021031 shutdown_connections 2024-05-21T21:54:34.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:34.117+0000 7f8a0be00700 1 --2- 172.21.15.151:0/1056021031 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a041035f0 0x7f8a04103a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:34.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:34.117+0000 7f8a0be00700 1 --2- 172.21.15.151:0/1056021031 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a04105190 0x7f8a0410ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:34.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:34.117+0000 7f8a0be00700 1 --2- 172.21.15.151:0/1056021031 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a041047f0 0x7f8a04104c50 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:34.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:34.118+0000 7f8a0be00700 1 -- 172.21.15.151:0/1056021031 >> 172.21.15.151:0/1056021031 conn(0x7f8a040feb90 msgr2=0x7f8a04100fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:54:34.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:34.118+0000 7f8a0be00700 1 -- 172.21.15.151:0/1056021031 shutdown_connections 2024-05-21T21:54:34.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:34.118+0000 7f8a0be00700 1 -- 172.21.15.151:0/1056021031 wait complete. 2024-05-21T21:54:34.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:34.119+0000 7f8a0be00700 1 Processor -- start 2024-05-21T21:54:34.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:34.119+0000 7f8a0be00700 1 -- start start 2024-05-21T21:54:34.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:34.120+0000 7f8a0be00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a041035f0 0x7f8a0419b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:34.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:34.120+0000 7f8a0be00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a041047f0 0x7f8a0419bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:34.121 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:34.120+0000 7f8a0be00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a04105190 0x7f8a041a2f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:34.121 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:34.120+0000 7f8a0be00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0410f0e0 con 0x7f8a04105190 2024-05-21T21:54:34.121 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:34.120+0000 7f8a0be00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0410ef60 con 0x7f8a041035f0 2024-05-21T21:54:34.121 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:34.120+0000 7f8a0be00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0410f260 con 0x7f8a041047f0 2024-05-21T21:54:34.121 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:34.120+0000 7f8a08a00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a041047f0 0x7f8a0419bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:34.121 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:34.120+0000 7f8a08a00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a041047f0 0x7f8a0419bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:52606/0 (socket says 172.21.15.151:52606) 2024-05-21T21:54:34.121 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:34.120+0000 7f8a08a00700 1 -- 172.21.15.151:0/2232108130 learned_addr learned my addr 172.21.15.151:0/2232108130 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:54:34.121 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:34.120+0000 7f8a09400700 1 --2- 172.21.15.151:0/2232108130 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a041035f0 0x7f8a0419b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:34.121 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:34.120+0000 7f8a09e00700 1 --2- 172.21.15.151:0/2232108130 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a04105190 0x7f8a041a2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:34.121 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:34.120+0000 7f8a08a00700 1 -- 172.21.15.151:0/2232108130 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a041035f0 msgr2=0x7f8a0419b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:34.121 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:34.120+0000 7f8a08a00700 1 --2- 172.21.15.151:0/2232108130 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a041035f0 0x7f8a0419b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:34.121 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:34.120+0000 7f8a08a00700 1 -- 172.21.15.151:0/2232108130 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a04105190 msgr2=0x7f8a041a2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:34.122 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:34.120+0000 7f8a08a00700 1 --2- 172.21.15.151:0/2232108130 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a04105190 0x7f8a041a2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:34.122 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:34.121+0000 7f8a08a00700 1 -- 172.21.15.151:0/2232108130 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a041a3620 con 0x7f8a041047f0 2024-05-21T21:54:34.122 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:34.121+0000 7f8a09400700 1 --2- 172.21.15.151:0/2232108130 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a041035f0 0x7f8a0419b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:54:34.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:34.121+0000 7f8a08a00700 1 --2- 172.21.15.151:0/2232108130 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a041047f0 0x7f8a0419bf00 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f89f0009810 tx=0x7f89f0005270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:54:34.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:34.121+0000 7f8a02000700 1 -- 172.21.15.151:0/2232108130 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f0025070 con 0x7f8a041047f0 2024-05-21T21:54:34.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:34.121+0000 7f8a02000700 1 -- 172.21.15.151:0/2232108130 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89f0005670 con 0x7f8a041047f0 2024-05-21T21:54:34.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:34.121+0000 7f8a02000700 1 -- 172.21.15.151:0/2232108130 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f0003c20 con 0x7f8a041047f0 2024-05-21T21:54:34.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:34.121+0000 7f8a0be00700 1 -- 172.21.15.151:0/2232108130 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a041a38b0 con 0x7f8a041047f0 2024-05-21T21:54:34.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:34.121+0000 7f8a0be00700 1 -- 172.21.15.151:0/2232108130 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a041a3d00 con 0x7f8a041047f0 2024-05-21T21:54:34.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:34.122+0000 7f8a0be00700 1 -- 172.21.15.151:0/2232108130 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a0406a230 con 0x7f8a041047f0 2024-05-21T21:54:34.125 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:34.125+0000 7f8a02000700 1 -- 172.21.15.151:0/2232108130 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f89f0028070 con 0x7f8a041047f0 2024-05-21T21:54:34.126 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:34.126+0000 7f8a02000700 1 --2- 172.21.15.151:0/2232108130 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f89ec0b2880 0x7f89ec0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:34.127 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:34.126+0000 7f8a02000700 1 -- 172.21.15.151:0/2232108130 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f89f00e2ee0 con 0x7f8a041047f0 2024-05-21T21:54:34.127 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:34.127+0000 7f8a09400700 1 --2- 172.21.15.151:0/2232108130 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f89ec0b2880 0x7f89ec0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:34.127 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:34.127+0000 7f8a09400700 1 --2- 172.21.15.151:0/2232108130 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f89ec0b2880 0x7f89ec0b4d40 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8a04104650 tx=0x7f89f8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:54:34.128 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:34.128+0000 7f8a02000700 1 -- 172.21.15.151:0/2232108130 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f89f002f020 con 0x7f8a041047f0 2024-05-21T21:54:34.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:34.388+0000 7f8a0be00700 1 -- 172.21.15.151:0/2232108130 --> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f8a04076e90 con 0x7f89ec0b2880 2024-05-21T21:54:34.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:34.391+0000 7f8a02000700 1 -- 172.21.15.151:0/2232108130 <== mgr.14152 v2:172.21.15.151:6800/2796622483 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f8a04076e90 con 0x7f89ec0b2880 2024-05-21T21:54:34.391 INFO:teuthology.orchestra.run.smithi151.stdout:Backend: cephadm 2024-05-21T21:54:34.392 INFO:teuthology.orchestra.run.smithi151.stdout:Available: Yes 2024-05-21T21:54:34.392 INFO:teuthology.orchestra.run.smithi151.stdout:Paused: No 2024-05-21T21:54:34.394 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:34.394+0000 7f8a0be00700 1 -- 172.21.15.151:0/2232108130 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f89ec0b2880 msgr2=0x7f89ec0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:34.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:34.394+0000 7f8a0be00700 1 --2- 172.21.15.151:0/2232108130 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f89ec0b2880 0x7f89ec0b4d40 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8a04104650 tx=0x7f89f8009260 comp rx=0 tx=0).stop 2024-05-21T21:54:34.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:34.394+0000 7f8a0be00700 1 -- 172.21.15.151:0/2232108130 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a041047f0 msgr2=0x7f8a0419bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:34.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:34.394+0000 7f8a0be00700 1 --2- 172.21.15.151:0/2232108130 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a041047f0 0x7f8a0419bf00 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f89f0009810 tx=0x7f89f0005270 comp rx=0 tx=0).stop 2024-05-21T21:54:34.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:34.394+0000 7f8a0be00700 1 -- 172.21.15.151:0/2232108130 shutdown_connections 2024-05-21T21:54:34.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:34.394+0000 7f8a0be00700 1 --2- 172.21.15.151:0/2232108130 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f89ec0b2880 0x7f89ec0b4d40 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:34.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:34.394+0000 7f8a0be00700 1 --2- 172.21.15.151:0/2232108130 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a041035f0 0x7f8a0419b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:34.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:34.394+0000 7f8a0be00700 1 --2- 172.21.15.151:0/2232108130 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a04105190 0x7f8a041a2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:34.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:34.394+0000 7f8a0be00700 1 --2- 172.21.15.151:0/2232108130 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a041047f0 0x7f8a0419bf00 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:34.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:34.395+0000 7f8a0be00700 1 -- 172.21.15.151:0/2232108130 >> 172.21.15.151:0/2232108130 conn(0x7f8a040feb90 msgr2=0x7f8a04075770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:54:34.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:34.395+0000 7f8a0be00700 1 -- 172.21.15.151:0/2232108130 shutdown_connections 2024-05-21T21:54:34.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:34.395+0000 7f8a0be00700 1 -- 172.21.15.151:0/2232108130 wait complete. 2024-05-21T21:54:34.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:54:34 smithi151 ceph-mon[31623]: pgmap v414: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-21T21:54:34.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:54:34 smithi184 ceph-mon[36942]: pgmap v414: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-21T21:54:34.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:54:34 smithi164 ceph-mon[36269]: pgmap v414: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-21T21:54:35.293 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- bash -c 'ceph orch ps' 2024-05-21T21:54:35.702 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:54:35 smithi151 ceph-mon[31623]: from='client.24973 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:54:35.730 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:54:35.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:54:35 smithi184 ceph-mon[36942]: from='client.24973 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:54:35.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:54:35 smithi164 ceph-mon[36269]: from='client.24973 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:54:36.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:54:36 smithi151 ceph-mon[31623]: pgmap v415: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-21T21:54:36.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:54:36 smithi184 ceph-mon[36942]: pgmap v415: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-21T21:54:36.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:54:36 smithi164 ceph-mon[36269]: pgmap v415: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-21T21:54:37.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:37.876+0000 7f2721200700 1 -- 172.21.15.151:0/333920296 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f271c10b490 msgr2=0x7f271c10d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:37.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:37.876+0000 7f2721200700 1 --2- 172.21.15.151:0/333920296 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f271c10b490 0x7f271c10d920 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f271400b0a0 tx=0x7f271400b3b0 comp rx=0 tx=0).stop 2024-05-21T21:54:37.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:37.876+0000 7f2721200700 1 -- 172.21.15.151:0/333920296 shutdown_connections 2024-05-21T21:54:37.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:37.876+0000 7f2721200700 1 --2- 172.21.15.151:0/333920296 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f271c10b490 0x7f271c10d920 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:37.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:37.876+0000 7f2721200700 1 --2- 172.21.15.151:0/333920296 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f271c077570 0x7f271c0759d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:37.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:37.876+0000 7f2721200700 1 --2- 172.21.15.151:0/333920296 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f271c075fa0 0x7f271c076420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:37.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:37.876+0000 7f2721200700 1 -- 172.21.15.151:0/333920296 >> 172.21.15.151:0/333920296 conn(0x7f271c0feb60 msgr2=0x7f271c100fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:54:37.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:37.877+0000 7f2721200700 1 -- 172.21.15.151:0/333920296 shutdown_connections 2024-05-21T21:54:37.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:37.877+0000 7f2721200700 1 -- 172.21.15.151:0/333920296 wait complete. 2024-05-21T21:54:37.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:37.878+0000 7f2721200700 1 Processor -- start 2024-05-21T21:54:37.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:37.878+0000 7f2721200700 1 -- start start 2024-05-21T21:54:37.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:37.878+0000 7f2721200700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f271c075fa0 0x7f271c19b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:37.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:37.879+0000 7f2721200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f271c077570 0x7f271c19be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:37.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:37.879+0000 7f2721200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f271c10b490 0x7f271c1a2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:37.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:37.879+0000 7f2721200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f271c1112a0 con 0x7f271c10b490 2024-05-21T21:54:37.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:37.879+0000 7f2721200700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f271c111120 con 0x7f271c077570 2024-05-21T21:54:37.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:37.879+0000 7f2721200700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f271c111420 con 0x7f271c075fa0 2024-05-21T21:54:37.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:37.879+0000 7f271a800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f271c10b490 0x7f271c1a2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:37.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:37.879+0000 7f2719e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f271c075fa0 0x7f271c19b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:37.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:37.879+0000 7f271a800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f271c10b490 0x7f271c1a2ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:40922/0 (socket says 172.21.15.151:40922) 2024-05-21T21:54:37.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:37.879+0000 7f2719400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f271c077570 0x7f271c19be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:37.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:37.879+0000 7f2719e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f271c075fa0 0x7f271c19b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:49006/0 (socket says 172.21.15.151:49006) 2024-05-21T21:54:37.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:37.879+0000 7f271a800700 1 -- 172.21.15.151:0/2723151765 learned_addr learned my addr 172.21.15.151:0/2723151765 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:54:37.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:37.879+0000 7f271a800700 1 -- 172.21.15.151:0/2723151765 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f271c075fa0 msgr2=0x7f271c19b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:37.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:37.879+0000 7f271a800700 1 --2- 172.21.15.151:0/2723151765 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f271c075fa0 0x7f271c19b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:37.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:37.879+0000 7f271a800700 1 -- 172.21.15.151:0/2723151765 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f271c077570 msgr2=0x7f271c19be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:37.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:37.879+0000 7f271a800700 1 --2- 172.21.15.151:0/2723151765 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f271c077570 0x7f271c19be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:37.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:37.879+0000 7f271a800700 1 -- 172.21.15.151:0/2723151765 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f271c1a3560 con 0x7f271c10b490 2024-05-21T21:54:37.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:37.880+0000 7f271a800700 1 --2- 172.21.15.151:0/2723151765 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f271c10b490 0x7f271c1a2ec0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2714009be0 tx=0x7f2714004cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:54:37.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:37.880+0000 7f2712a00700 1 -- 172.21.15.151:0/2723151765 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2714021070 con 0x7f271c10b490 2024-05-21T21:54:37.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:37.880+0000 7f2712a00700 1 -- 172.21.15.151:0/2723151765 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2714007b80 con 0x7f271c10b490 2024-05-21T21:54:37.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:37.880+0000 7f2721200700 1 -- 172.21.15.151:0/2723151765 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f271c1a37f0 con 0x7f271c10b490 2024-05-21T21:54:37.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:37.880+0000 7f2712a00700 1 -- 172.21.15.151:0/2723151765 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2714004640 con 0x7f271c10b490 2024-05-21T21:54:37.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:37.880+0000 7f2721200700 1 -- 172.21.15.151:0/2723151765 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f271c1a3ce0 con 0x7f271c10b490 2024-05-21T21:54:37.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:37.881+0000 7f2721200700 1 -- 172.21.15.151:0/2723151765 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f271c06a230 con 0x7f271c10b490 2024-05-21T21:54:37.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:37.884+0000 7f2712a00700 1 -- 172.21.15.151:0/2723151765 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f2714003a00 con 0x7f271c10b490 2024-05-21T21:54:37.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:37.885+0000 7f2712a00700 1 --2- 172.21.15.151:0/2723151765 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f27000b2880 0x7f27000b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:37.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:37.885+0000 7f2712a00700 1 -- 172.21.15.151:0/2723151765 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f27140e3250 con 0x7f271c10b490 2024-05-21T21:54:37.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:37.885+0000 7f2719e00700 1 --2- 172.21.15.151:0/2723151765 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f27000b2880 0x7f27000b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:37.886 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:37.886+0000 7f2719e00700 1 --2- 172.21.15.151:0/2723151765 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f27000b2880 0x7f27000b4d40 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f271c077950 tx=0x7f270c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:54:37.890 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:37.890+0000 7f2712a00700 1 -- 172.21.15.151:0/2723151765 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f27140aafc0 con 0x7f271c10b490 2024-05-21T21:54:38.130 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:38.129+0000 7f2721200700 1 -- 172.21.15.151:0/2723151765 --> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f271c063980 con 0x7f27000b2880 2024-05-21T21:54:38.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:38.179+0000 7f2712a00700 1 -- 172.21.15.151:0/2723151765 <== mgr.14152 v2:172.21.15.151:6800/2796622483 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7f271c063980 con 0x7f27000b2880 2024-05-21T21:54:38.180 INFO:teuthology.orchestra.run.smithi151.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-21T21:54:38.180 INFO:teuthology.orchestra.run.smithi151.stdout:mds.a smithi151 running (3m) 57s ago 3m 17.7M - 19.0.0-3830-g3b748ea0 d6bc8f31cbc1 257d6d88cf35 2024-05-21T21:54:38.180 INFO:teuthology.orchestra.run.smithi151.stdout:mds.b smithi164 running (3m) 82s ago 3m 26.8M - 19.0.0-3830-g3b748ea0 d6bc8f31cbc1 8598cc3b89fc 2024-05-21T21:54:38.181 INFO:teuthology.orchestra.run.smithi151.stdout:mds.c smithi184 running (3m) 82s ago 3m 19.5M - 19.0.0-3830-g3b748ea0 d6bc8f31cbc1 6364be5c643d 2024-05-21T21:54:38.181 INFO:teuthology.orchestra.run.smithi151.stdout:mds.d smithi151 running (4m) 57s ago 4m 17.9M - 19.0.0-3830-g3b748ea0 d6bc8f31cbc1 3dfebcc084a3 2024-05-21T21:54:38.181 INFO:teuthology.orchestra.run.smithi151.stdout:mds.e smithi164 running (4m) 82s ago 4m 21.2M - 19.0.0-3830-g3b748ea0 d6bc8f31cbc1 0bfde453384d 2024-05-21T21:54:38.181 INFO:teuthology.orchestra.run.smithi151.stdout:mds.f smithi184 running (4m) 82s ago 3m 20.7M - 19.0.0-3830-g3b748ea0 d6bc8f31cbc1 e13fed9be1b5 2024-05-21T21:54:38.181 INFO:teuthology.orchestra.run.smithi151.stdout:mds.g smithi151 running (3m) 57s ago 3m 13.2M - 19.0.0-3830-g3b748ea0 d6bc8f31cbc1 d3e49631a0d4 2024-05-21T21:54:38.181 INFO:teuthology.orchestra.run.smithi151.stdout:mds.h smithi164 running (3m) 82s ago 3m 22.2M - 19.0.0-3830-g3b748ea0 d6bc8f31cbc1 8738418e3252 2024-05-21T21:54:38.181 INFO:teuthology.orchestra.run.smithi151.stdout:mds.i smithi184 running (3m) 82s ago 3m 21.2M - 19.0.0-3830-g3b748ea0 d6bc8f31cbc1 1fa7a2f12571 2024-05-21T21:54:38.181 INFO:teuthology.orchestra.run.smithi151.stdout:mds.j smithi151 running (4m) 57s ago 4m 13.3M - 19.0.0-3830-g3b748ea0 d6bc8f31cbc1 d9eec75ffa5a 2024-05-21T21:54:38.181 INFO:teuthology.orchestra.run.smithi151.stdout:mds.k smithi164 running (4m) 82s ago 4m 19.6M - 19.0.0-3830-g3b748ea0 d6bc8f31cbc1 7865d2590247 2024-05-21T21:54:38.181 INFO:teuthology.orchestra.run.smithi151.stdout:mds.l smithi184 running (4m) 82s ago 4m 18.7M - 19.0.0-3830-g3b748ea0 d6bc8f31cbc1 75cdbe32c0e3 2024-05-21T21:54:38.181 INFO:teuthology.orchestra.run.smithi151.stdout:mgr.x smithi151 *:9283,8765 running (12m) 57s ago 12m 530M - 19.0.0-3830-g3b748ea0 d6bc8f31cbc1 297d546f035d 2024-05-21T21:54:38.181 INFO:teuthology.orchestra.run.smithi151.stdout:mgr.y smithi164 *:8443,8765 running (10m) 82s ago 10m 469M - 19.0.0-3830-g3b748ea0 d6bc8f31cbc1 64dc68d1e6d0 2024-05-21T21:54:38.181 INFO:teuthology.orchestra.run.smithi151.stdout:mgr.z smithi184 *:8443,8765 running (10m) 82s ago 10m 468M - 19.0.0-3830-g3b748ea0 d6bc8f31cbc1 ed05f69e3229 2024-05-21T21:54:38.182 INFO:teuthology.orchestra.run.smithi151.stdout:mon.a smithi151 running (12m) 57s ago 12m 48.8M 2048M 19.0.0-3830-g3b748ea0 d6bc8f31cbc1 9cfe13f3b961 2024-05-21T21:54:38.182 INFO:teuthology.orchestra.run.smithi151.stdout:mon.b smithi164 running (10m) 82s ago 10m 44.9M 2048M 19.0.0-3830-g3b748ea0 d6bc8f31cbc1 921034200eb5 2024-05-21T21:54:38.182 INFO:teuthology.orchestra.run.smithi151.stdout:mon.c smithi184 running (10m) 82s ago 10m 42.3M 2048M 19.0.0-3830-g3b748ea0 d6bc8f31cbc1 a586a91da0af 2024-05-21T21:54:38.182 INFO:teuthology.orchestra.run.smithi151.stdout:osd.0 smithi151 running (9m) 57s ago 9m 55.0M 4272M 19.0.0-3830-g3b748ea0 d6bc8f31cbc1 c2162c111912 2024-05-21T21:54:38.182 INFO:teuthology.orchestra.run.smithi151.stdout:osd.1 smithi164 running (9m) 82s ago 9m 53.2M 4272M 19.0.0-3830-g3b748ea0 d6bc8f31cbc1 55d1bf230b7a 2024-05-21T21:54:38.182 INFO:teuthology.orchestra.run.smithi151.stdout:osd.2 smithi184 running (8m) 82s ago 8m 60.8M 4272M 19.0.0-3830-g3b748ea0 d6bc8f31cbc1 209e64b2a5d6 2024-05-21T21:54:38.182 INFO:teuthology.orchestra.run.smithi151.stdout:osd.3 smithi151 running (8m) 57s ago 8m 53.9M 4272M 19.0.0-3830-g3b748ea0 d6bc8f31cbc1 90351da10fba 2024-05-21T21:54:38.182 INFO:teuthology.orchestra.run.smithi151.stdout:osd.4 smithi164 running (7m) 82s ago 7m 47.0M 4272M 19.0.0-3830-g3b748ea0 d6bc8f31cbc1 c937f40e2c94 2024-05-21T21:54:38.182 INFO:teuthology.orchestra.run.smithi151.stdout:osd.5 smithi184 running (7m) 82s ago 7m 57.2M 4272M 19.0.0-3830-g3b748ea0 d6bc8f31cbc1 44a6b9505c01 2024-05-21T21:54:38.182 INFO:teuthology.orchestra.run.smithi151.stdout:osd.6 smithi151 running (6m) 57s ago 6m 44.5M 4272M 19.0.0-3830-g3b748ea0 d6bc8f31cbc1 1283ff154bd1 2024-05-21T21:54:38.182 INFO:teuthology.orchestra.run.smithi151.stdout:osd.7 smithi164 running (6m) 82s ago 6m 50.7M 4272M 19.0.0-3830-g3b748ea0 d6bc8f31cbc1 a53d6eb75ee2 2024-05-21T21:54:38.182 INFO:teuthology.orchestra.run.smithi151.stdout:osd.8 smithi184 running (6m) 82s ago 6m 49.6M 4272M 19.0.0-3830-g3b748ea0 d6bc8f31cbc1 4eade7a10472 2024-05-21T21:54:38.182 INFO:teuthology.orchestra.run.smithi151.stdout:osd.9 smithi151 running (5m) 57s ago 5m 45.4M 4272M 19.0.0-3830-g3b748ea0 d6bc8f31cbc1 c0d77e7b9a7a 2024-05-21T21:54:38.183 INFO:teuthology.orchestra.run.smithi151.stdout:osd.10 smithi164 running (5m) 82s ago 5m 47.8M 4272M 19.0.0-3830-g3b748ea0 d6bc8f31cbc1 05e2afdedadc 2024-05-21T21:54:38.183 INFO:teuthology.orchestra.run.smithi151.stdout:osd.11 smithi184 running (4m) 82s ago 4m 46.0M 4272M 19.0.0-3830-g3b748ea0 d6bc8f31cbc1 72df127bc3cf 2024-05-21T21:54:38.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:38.183+0000 7f2721200700 1 -- 172.21.15.151:0/2723151765 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f27000b2880 msgr2=0x7f27000b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:38.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:38.183+0000 7f2721200700 1 --2- 172.21.15.151:0/2723151765 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f27000b2880 0x7f27000b4d40 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f271c077950 tx=0x7f270c009260 comp rx=0 tx=0).stop 2024-05-21T21:54:38.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:38.183+0000 7f2721200700 1 -- 172.21.15.151:0/2723151765 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f271c10b490 msgr2=0x7f271c1a2ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:38.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:38.183+0000 7f2721200700 1 --2- 172.21.15.151:0/2723151765 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f271c10b490 0x7f271c1a2ec0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2714009be0 tx=0x7f2714004cf0 comp rx=0 tx=0).stop 2024-05-21T21:54:38.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:38.184+0000 7f2721200700 1 -- 172.21.15.151:0/2723151765 shutdown_connections 2024-05-21T21:54:38.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:38.184+0000 7f2721200700 1 --2- 172.21.15.151:0/2723151765 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f27000b2880 0x7f27000b4d40 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:38.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:38.184+0000 7f2721200700 1 --2- 172.21.15.151:0/2723151765 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f271c10b490 0x7f271c1a2ec0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:38.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:38.184+0000 7f2721200700 1 --2- 172.21.15.151:0/2723151765 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f271c075fa0 0x7f271c19b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:38.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:38.184+0000 7f2721200700 1 --2- 172.21.15.151:0/2723151765 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f271c077570 0x7f271c19be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:38.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:38.184+0000 7f2721200700 1 -- 172.21.15.151:0/2723151765 >> 172.21.15.151:0/2723151765 conn(0x7f271c0feb60 msgr2=0x7f271c107390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:54:38.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:38.184+0000 7f2721200700 1 -- 172.21.15.151:0/2723151765 shutdown_connections 2024-05-21T21:54:38.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:38.184+0000 7f2721200700 1 -- 172.21.15.151:0/2723151765 wait complete. 2024-05-21T21:54:38.454 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:54:38 smithi151 ceph-mon[31623]: pgmap v416: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-21T21:54:38.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:54:38 smithi184 ceph-mon[36942]: pgmap v416: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-21T21:54:38.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:54:38 smithi164 ceph-mon[36269]: pgmap v416: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-21T21:54:38.994 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- bash -c 'ceph orch ls' 2024-05-21T21:54:39.442 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:54:39.669 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:54:39 smithi151 ceph-mon[31623]: from='client.14865 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:54:39.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:54:39 smithi184 ceph-mon[36942]: from='client.14865 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:54:39.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:54:39 smithi164 ceph-mon[36269]: from='client.14865 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:54:40.678 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:54:40 smithi151 ceph-mon[31623]: pgmap v417: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-21T21:54:40.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:54:40 smithi184 ceph-mon[36942]: pgmap v417: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-21T21:54:40.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:54:40 smithi164 ceph-mon[36269]: pgmap v417: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-21T21:54:41.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:41.519+0000 7f12d6600700 1 -- 172.21.15.151:0/263042371 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12d0107280 msgr2=0x7f12d010db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:41.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:41.519+0000 7f12d6600700 1 --2- 172.21.15.151:0/263042371 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12d0107280 0x7f12d010db30 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f12c800b0d0 tx=0x7f12c800b3e0 comp rx=0 tx=0).stop 2024-05-21T21:54:41.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:41.519+0000 7f12d6600700 1 -- 172.21.15.151:0/263042371 shutdown_connections 2024-05-21T21:54:41.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:41.519+0000 7f12d6600700 1 --2- 172.21.15.151:0/263042371 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12d0107280 0x7f12d010db30 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:41.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:41.519+0000 7f12d6600700 1 --2- 172.21.15.151:0/263042371 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f12d01056e0 0x7f12d0105b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:41.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:41.519+0000 7f12d6600700 1 --2- 172.21.15.151:0/263042371 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f12d01068e0 0x7f12d0106d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:41.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:41.519+0000 7f12d6600700 1 -- 172.21.15.151:0/263042371 >> 172.21.15.151:0/263042371 conn(0x7f12d0100c80 msgr2=0x7f12d01030c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:54:41.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:41.519+0000 7f12d6600700 1 -- 172.21.15.151:0/263042371 shutdown_connections 2024-05-21T21:54:41.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:41.519+0000 7f12d6600700 1 -- 172.21.15.151:0/263042371 wait complete. 2024-05-21T21:54:41.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:41.521+0000 7f12d6600700 1 Processor -- start 2024-05-21T21:54:41.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:41.521+0000 7f12d6600700 1 -- start start 2024-05-21T21:54:41.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:41.521+0000 7f12d6600700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f12d01056e0 0x7f12d019b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:41.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:41.521+0000 7f12d6600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12d01068e0 0x7f12d019bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:41.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:41.521+0000 7f12d6600700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f12d0107280 0x7f12d01a2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:41.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:41.521+0000 7f12d6600700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f12d0111160 con 0x7f12d01068e0 2024-05-21T21:54:41.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:41.521+0000 7f12d6600700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f12d0110fe0 con 0x7f12d01056e0 2024-05-21T21:54:41.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:41.521+0000 7f12d6600700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f12d01112e0 con 0x7f12d0107280 2024-05-21T21:54:41.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:41.522+0000 7f12cfe00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12d01068e0 0x7f12d019bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:41.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:41.522+0000 7f12d5200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f12d01056e0 0x7f12d019b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:41.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:41.522+0000 7f12d5200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f12d01056e0 0x7f12d019b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:41836/0 (socket says 172.21.15.151:41836) 2024-05-21T21:54:41.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:41.522+0000 7f12d5200700 1 -- 172.21.15.151:0/1127668922 learned_addr learned my addr 172.21.15.151:0/1127668922 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:54:41.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:41.522+0000 7f12d5c00700 1 --2- 172.21.15.151:0/1127668922 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f12d0107280 0x7f12d01a2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:41.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:41.522+0000 7f12d5200700 1 -- 172.21.15.151:0/1127668922 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f12d0107280 msgr2=0x7f12d01a2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:41.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:41.522+0000 7f12d5200700 1 --2- 172.21.15.151:0/1127668922 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f12d0107280 0x7f12d01a2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:41.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:41.522+0000 7f12d5200700 1 -- 172.21.15.151:0/1127668922 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12d01068e0 msgr2=0x7f12d019bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:41.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:41.522+0000 7f12d5200700 1 --2- 172.21.15.151:0/1127668922 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12d01068e0 0x7f12d019bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:41.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:41.522+0000 7f12d5200700 1 -- 172.21.15.151:0/1127668922 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12d01a3370 con 0x7f12d01056e0 2024-05-21T21:54:41.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:41.523+0000 7f12d5200700 1 --2- 172.21.15.151:0/1127668922 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f12d01056e0 0x7f12d019b710 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f12c400e970 tx=0x7f12c400ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:54:41.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:41.523+0000 7f12cd600700 1 -- 172.21.15.151:0/1127668922 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12c4004220 con 0x7f12d01056e0 2024-05-21T21:54:41.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:41.523+0000 7f12cd600700 1 -- 172.21.15.151:0/1127668922 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12c40043c0 con 0x7f12d01056e0 2024-05-21T21:54:41.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:41.523+0000 7f12cd600700 1 -- 172.21.15.151:0/1127668922 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12c4003ce0 con 0x7f12d01056e0 2024-05-21T21:54:41.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:41.523+0000 7f12d6600700 1 -- 172.21.15.151:0/1127668922 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12d01a3660 con 0x7f12d01056e0 2024-05-21T21:54:41.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:41.523+0000 7f12d6600700 1 -- 172.21.15.151:0/1127668922 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12d01a3b50 con 0x7f12d01056e0 2024-05-21T21:54:41.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:41.525+0000 7f12d6600700 1 -- 172.21.15.151:0/1127668922 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12d006a230 con 0x7f12d01056e0 2024-05-21T21:54:41.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:41.527+0000 7f12cd600700 1 -- 172.21.15.151:0/1127668922 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f12c400c7e0 con 0x7f12d01056e0 2024-05-21T21:54:41.529 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:41.528+0000 7f12cd600700 1 --2- 172.21.15.151:0/1127668922 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f12b80b2900 0x7f12b80b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:41.529 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:41.529+0000 7f12cd600700 1 -- 172.21.15.151:0/1127668922 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f12c40e2430 con 0x7f12d01056e0 2024-05-21T21:54:41.529 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:41.529+0000 7f12cfe00700 1 --2- 172.21.15.151:0/1127668922 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f12b80b2900 0x7f12b80b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:41.529 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:41.530+0000 7f12cfe00700 1 --2- 172.21.15.151:0/1127668922 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f12b80b2900 0x7f12b80b4dc0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f12bc009c60 tx=0x7f12bc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:54:41.533 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:41.533+0000 7f12cd600700 1 -- 172.21.15.151:0/1127668922 <== mon.2 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f12c40aa110 con 0x7f12d01056e0 2024-05-21T21:54:41.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:41.780+0000 7f12d6600700 1 -- 172.21.15.151:0/1127668922 --> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f12d0063980 con 0x7f12b80b2900 2024-05-21T21:54:41.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:41.787+0000 7f12cd600700 1 -- 172.21.15.151:0/1127668922 <== mgr.14152 v2:172.21.15.151:6800/2796622483 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f12d0063980 con 0x7f12b80b2900 2024-05-21T21:54:41.787 INFO:teuthology.orchestra.run.smithi151.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-05-21T21:54:41.787 INFO:teuthology.orchestra.run.smithi151.stdout:mds.all 12/12 86s ago 4m smithi151=a;smithi151=d;smithi151=g;smithi151=j;smithi164=b;smithi164=e;smithi164=h;smithi164=k;smithi184=c;smithi184=f;smithi184=i;smithi184=l;count:12 2024-05-21T21:54:41.787 INFO:teuthology.orchestra.run.smithi151.stdout:mgr 3/3 86s ago 10m smithi151=x;smithi164=y;smithi184=z;count:3 2024-05-21T21:54:41.787 INFO:teuthology.orchestra.run.smithi151.stdout:mon 3/3 86s ago 10m smithi151:172.21.15.151=a;smithi164:172.21.15.164=b;smithi184:172.21.15.184=c;count:3 2024-05-21T21:54:41.787 INFO:teuthology.orchestra.run.smithi151.stdout:osd 12 86s ago - 2024-05-21T21:54:41.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:41.789+0000 7f12d6600700 1 -- 172.21.15.151:0/1127668922 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f12b80b2900 msgr2=0x7f12b80b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:41.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:41.789+0000 7f12d6600700 1 --2- 172.21.15.151:0/1127668922 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f12b80b2900 0x7f12b80b4dc0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f12bc009c60 tx=0x7f12bc009260 comp rx=0 tx=0).stop 2024-05-21T21:54:41.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:41.789+0000 7f12d6600700 1 -- 172.21.15.151:0/1127668922 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f12d01056e0 msgr2=0x7f12d019b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:41.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:41.789+0000 7f12d6600700 1 --2- 172.21.15.151:0/1127668922 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f12d01056e0 0x7f12d019b710 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f12c400e970 tx=0x7f12c400ec80 comp rx=0 tx=0).stop 2024-05-21T21:54:41.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:41.790+0000 7f12d6600700 1 -- 172.21.15.151:0/1127668922 shutdown_connections 2024-05-21T21:54:41.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:41.790+0000 7f12d6600700 1 --2- 172.21.15.151:0/1127668922 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f12b80b2900 0x7f12b80b4dc0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:41.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:41.790+0000 7f12d6600700 1 --2- 172.21.15.151:0/1127668922 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f12d01056e0 0x7f12d019b710 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:41.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:41.790+0000 7f12d6600700 1 --2- 172.21.15.151:0/1127668922 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f12d0107280 0x7f12d01a2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:41.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:41.790+0000 7f12d6600700 1 --2- 172.21.15.151:0/1127668922 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12d01068e0 0x7f12d019bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:41.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:41.790+0000 7f12d6600700 1 -- 172.21.15.151:0/1127668922 >> 172.21.15.151:0/1127668922 conn(0x7f12d0100c80 msgr2=0x7f12d0109c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:54:41.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:41.790+0000 7f12d6600700 1 -- 172.21.15.151:0/1127668922 shutdown_connections 2024-05-21T21:54:41.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:41.790+0000 7f12d6600700 1 -- 172.21.15.151:0/1127668922 wait complete. 2024-05-21T21:54:42.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:54:42 smithi151 ceph-mon[31623]: pgmap v418: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-21T21:54:42.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:54:42 smithi151 ceph-mon[31623]: from='client.24995 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:54:42.627 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- bash -c 'ceph orch host ls' 2024-05-21T21:54:42.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:54:42 smithi184 ceph-mon[36942]: pgmap v418: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-21T21:54:42.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:54:42 smithi184 ceph-mon[36942]: from='client.24995 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:54:42.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:54:42 smithi164 ceph-mon[36269]: pgmap v418: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-21T21:54:42.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:54:42 smithi164 ceph-mon[36269]: from='client.24995 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:54:43.082 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:54:43.501 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:54:43 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:54:43.501 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:54:43 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:54:43.501 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:54:43 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:54:43.502 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:54:43 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:54:43.502 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:54:43 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:54:43.735 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:54:43 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:54:43.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:54:43 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:54:43.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:54:43 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:54:43.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:54:43 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:54:43.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:54:43 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:54:43.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:54:43 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:54:43.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:54:43 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:54:43.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:54:43 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:54:43.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:54:43 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:54:43.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:54:43 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:54:44.699 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:54:44 smithi151 ceph-mon[31623]: pgmap v419: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-21T21:54:44.699 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:54:44 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:54:44.699 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:54:44 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:54:44.699 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:54:44 smithi151 ceph-mon[31623]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:54:44.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:54:44 smithi184 ceph-mon[36942]: pgmap v419: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-21T21:54:44.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:54:44 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:54:44.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:54:44 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:54:44.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:54:44 smithi184 ceph-mon[36942]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:54:44.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:54:44 smithi164 ceph-mon[36269]: pgmap v419: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-21T21:54:44.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:54:44 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:54:44.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:54:44 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:54:44.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:54:44 smithi164 ceph-mon[36269]: from='mgr.14152 172.21.15.151:0/1034694240' entity='mgr.x' 2024-05-21T21:54:45.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:45.209+0000 7fd87ba00700 1 -- 172.21.15.151:0/3329003514 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd87410b780 msgr2=0x7fd874112030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:45.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:45.209+0000 7fd87ba00700 1 --2- 172.21.15.151:0/3329003514 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd87410b780 0x7fd874112030 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd86c009f90 tx=0x7fd86c00b330 comp rx=0 tx=0).stop 2024-05-21T21:54:45.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:45.209+0000 7fd87ba00700 1 -- 172.21.15.151:0/3329003514 shutdown_connections 2024-05-21T21:54:45.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:45.209+0000 7fd87ba00700 1 --2- 172.21.15.151:0/3329003514 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd874109be0 0x7fd87410a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:45.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:45.209+0000 7fd87ba00700 1 --2- 172.21.15.151:0/3329003514 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd87410b780 0x7fd874112030 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:45.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:45.209+0000 7fd87ba00700 1 --2- 172.21.15.151:0/3329003514 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd87410ade0 0x7fd87410b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:45.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:45.209+0000 7fd87ba00700 1 -- 172.21.15.151:0/3329003514 >> 172.21.15.151:0/3329003514 conn(0x7fd874078bb0 msgr2=0x7fd87407aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:54:45.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:45.210+0000 7fd87ba00700 1 -- 172.21.15.151:0/3329003514 shutdown_connections 2024-05-21T21:54:45.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:45.210+0000 7fd87ba00700 1 -- 172.21.15.151:0/3329003514 wait complete. 2024-05-21T21:54:45.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:45.211+0000 7fd87ba00700 1 Processor -- start 2024-05-21T21:54:45.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:45.211+0000 7fd87ba00700 1 -- start start 2024-05-21T21:54:45.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:45.212+0000 7fd87ba00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd874109be0 0x7fd87419fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:45.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:45.212+0000 7fd87ba00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd87410ade0 0x7fd8741a0170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:45.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:45.212+0000 7fd87ba00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd87410b780 0x7fd8741a71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:45.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:45.212+0000 7fd87ba00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd874115660 con 0x7fd87410b780 2024-05-21T21:54:45.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:45.212+0000 7fd87ba00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd8741154e0 con 0x7fd87410ade0 2024-05-21T21:54:45.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:45.212+0000 7fd87ba00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd8741157e0 con 0x7fd874109be0 2024-05-21T21:54:45.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:45.212+0000 7fd873e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd87410ade0 0x7fd8741a0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:45.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:45.212+0000 7fd879000700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd874109be0 0x7fd87419fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:45.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:45.212+0000 7fd873e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd87410ade0 0x7fd8741a0170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:41850/0 (socket says 172.21.15.151:41850) 2024-05-21T21:54:45.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:45.212+0000 7fd873e00700 1 -- 172.21.15.151:0/2734084200 learned_addr learned my addr 172.21.15.151:0/2734084200 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:54:45.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:45.213+0000 7fd879a00700 1 --2- 172.21.15.151:0/2734084200 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd87410b780 0x7fd8741a71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:45.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:45.213+0000 7fd879000700 1 -- 172.21.15.151:0/2734084200 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd87410ade0 msgr2=0x7fd8741a0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:45.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:45.213+0000 7fd879000700 1 --2- 172.21.15.151:0/2734084200 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd87410ade0 0x7fd8741a0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:45.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:45.213+0000 7fd879000700 1 -- 172.21.15.151:0/2734084200 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd87410b780 msgr2=0x7fd8741a71b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:45.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:45.213+0000 7fd879000700 1 --2- 172.21.15.151:0/2734084200 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd87410b780 0x7fd8741a71b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:45.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:45.213+0000 7fd879000700 1 -- 172.21.15.151:0/2734084200 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8741a7880 con 0x7fd874109be0 2024-05-21T21:54:45.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:45.213+0000 7fd873e00700 1 --2- 172.21.15.151:0/2734084200 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd87410ade0 0x7fd8741a0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:54:45.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:45.213+0000 7fd879a00700 1 --2- 172.21.15.151:0/2734084200 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd87410b780 0x7fd8741a71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:54:45.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:45.213+0000 7fd879000700 1 --2- 172.21.15.151:0/2734084200 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd874109be0 0x7fd87419fc30 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd86800d720 tx=0x7fd86800da30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:54:45.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:45.214+0000 7fd871600700 1 -- 172.21.15.151:0/2734084200 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd868014030 con 0x7fd874109be0 2024-05-21T21:54:45.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:45.214+0000 7fd871600700 1 -- 172.21.15.151:0/2734084200 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd86800fb40 con 0x7fd874109be0 2024-05-21T21:54:45.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:45.214+0000 7fd87ba00700 1 -- 172.21.15.151:0/2734084200 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8741a7b70 con 0x7fd874109be0 2024-05-21T21:54:45.216 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:45.214+0000 7fd871600700 1 -- 172.21.15.151:0/2734084200 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd868010d30 con 0x7fd874109be0 2024-05-21T21:54:45.216 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:45.214+0000 7fd87ba00700 1 -- 172.21.15.151:0/2734084200 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8741a8100 con 0x7fd874109be0 2024-05-21T21:54:45.216 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:45.215+0000 7fd87ba00700 1 -- 172.21.15.151:0/2734084200 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd87406a230 con 0x7fd874109be0 2024-05-21T21:54:45.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:45.217+0000 7fd871600700 1 -- 172.21.15.151:0/2734084200 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fd868017030 con 0x7fd874109be0 2024-05-21T21:54:45.219 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:45.219+0000 7fd871600700 1 --2- 172.21.15.151:0/2734084200 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fd85c0b2900 0x7fd85c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:45.219 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:45.219+0000 7fd871600700 1 -- 172.21.15.151:0/2734084200 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fd8680e1ec0 con 0x7fd874109be0 2024-05-21T21:54:45.219 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:45.219+0000 7fd873e00700 1 --2- 172.21.15.151:0/2734084200 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fd85c0b2900 0x7fd85c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:45.219 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:45.220+0000 7fd873e00700 1 --2- 172.21.15.151:0/2734084200 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fd85c0b2900 0x7fd85c0b4dc0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd860009750 tx=0x7fd860009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:54:45.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:45.221+0000 7fd871600700 1 -- 172.21.15.151:0/2734084200 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd8680a9ba0 con 0x7fd874109be0 2024-05-21T21:54:45.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:45.463+0000 7fd87ba00700 1 -- 172.21.15.151:0/2734084200 --> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7fd874063980 con 0x7fd85c0b2900 2024-05-21T21:54:45.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:45.465+0000 7fd871600700 1 -- 172.21.15.151:0/2734084200 <== mgr.14152 v2:172.21.15.151:6800/2796622483 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7fd874063980 con 0x7fd85c0b2900 2024-05-21T21:54:45.466 INFO:teuthology.orchestra.run.smithi151.stdout:HOST ADDR LABELS STATUS 2024-05-21T21:54:45.466 INFO:teuthology.orchestra.run.smithi151.stdout:smithi151 172.21.15.151 2024-05-21T21:54:45.466 INFO:teuthology.orchestra.run.smithi151.stdout:smithi164 172.21.15.164 2024-05-21T21:54:45.466 INFO:teuthology.orchestra.run.smithi151.stdout:smithi184 172.21.15.184 2024-05-21T21:54:45.466 INFO:teuthology.orchestra.run.smithi151.stdout:3 hosts in cluster 2024-05-21T21:54:45.469 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:45.468+0000 7fd87ba00700 1 -- 172.21.15.151:0/2734084200 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fd85c0b2900 msgr2=0x7fd85c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:45.469 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:45.468+0000 7fd87ba00700 1 --2- 172.21.15.151:0/2734084200 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fd85c0b2900 0x7fd85c0b4dc0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd860009750 tx=0x7fd860009310 comp rx=0 tx=0).stop 2024-05-21T21:54:45.469 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:45.468+0000 7fd87ba00700 1 -- 172.21.15.151:0/2734084200 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd874109be0 msgr2=0x7fd87419fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:45.469 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:45.468+0000 7fd87ba00700 1 --2- 172.21.15.151:0/2734084200 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd874109be0 0x7fd87419fc30 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd86800d720 tx=0x7fd86800da30 comp rx=0 tx=0).stop 2024-05-21T21:54:45.469 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:45.469+0000 7fd87ba00700 1 -- 172.21.15.151:0/2734084200 shutdown_connections 2024-05-21T21:54:45.469 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:45.469+0000 7fd87ba00700 1 --2- 172.21.15.151:0/2734084200 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fd85c0b2900 0x7fd85c0b4dc0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:45.469 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:45.469+0000 7fd87ba00700 1 --2- 172.21.15.151:0/2734084200 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd87410b780 0x7fd8741a71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:45.469 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:45.469+0000 7fd87ba00700 1 --2- 172.21.15.151:0/2734084200 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd874109be0 0x7fd87419fc30 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:45.470 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:45.469+0000 7fd87ba00700 1 --2- 172.21.15.151:0/2734084200 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd87410ade0 0x7fd8741a0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:45.470 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:45.469+0000 7fd87ba00700 1 -- 172.21.15.151:0/2734084200 >> 172.21.15.151:0/2734084200 conn(0x7fd874078bb0 msgr2=0x7fd87410e0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:54:45.470 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:45.469+0000 7fd87ba00700 1 -- 172.21.15.151:0/2734084200 shutdown_connections 2024-05-21T21:54:45.470 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:45.469+0000 7fd87ba00700 1 -- 172.21.15.151:0/2734084200 wait complete. 2024-05-21T21:54:46.201 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- bash -c 'ceph orch device ls' 2024-05-21T21:54:46.659 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:54:46.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:54:46 smithi151 ceph-mon[31623]: pgmap v420: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-21T21:54:46.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:54:46 smithi151 ceph-mon[31623]: from='client.24991 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:54:46.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:54:46 smithi184 ceph-mon[36942]: pgmap v420: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-21T21:54:46.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:54:46 smithi184 ceph-mon[36942]: from='client.24991 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:54:46.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:54:46 smithi164 ceph-mon[36269]: pgmap v420: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-21T21:54:46.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:54:46 smithi164 ceph-mon[36269]: from='client.24991 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:54:48.488 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:54:48 smithi151 ceph-mon[31623]: pgmap v421: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-21T21:54:48.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:54:48 smithi184 ceph-mon[36942]: pgmap v421: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-21T21:54:48.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:54:48 smithi164 ceph-mon[36269]: pgmap v421: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-21T21:54:48.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:48.785+0000 7fca75000700 1 -- 172.21.15.151:0/2660509653 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca70104760 msgr2=0x7fca70104bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:48.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:48.785+0000 7fca75000700 1 --2- 172.21.15.151:0/2660509653 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca70104760 0x7fca70104bc0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fca58009990 tx=0x7fca58009ca0 comp rx=0 tx=0).stop 2024-05-21T21:54:48.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:48.786+0000 7fca75000700 1 -- 172.21.15.151:0/2660509653 shutdown_connections 2024-05-21T21:54:48.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:48.786+0000 7fca75000700 1 --2- 172.21.15.151:0/2660509653 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fca70105100 0x7fca7010b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:48.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:48.786+0000 7fca75000700 1 --2- 172.21.15.151:0/2660509653 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca70101780 0x7fca70101ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:48.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:48.786+0000 7fca75000700 1 --2- 172.21.15.151:0/2660509653 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca70104760 0x7fca70104bc0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:48.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:48.786+0000 7fca75000700 1 -- 172.21.15.151:0/2660509653 >> 172.21.15.151:0/2660509653 conn(0x7fca700fd0e0 msgr2=0x7fca700ff520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:54:48.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:48.786+0000 7fca75000700 1 -- 172.21.15.151:0/2660509653 shutdown_connections 2024-05-21T21:54:48.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:48.786+0000 7fca75000700 1 -- 172.21.15.151:0/2660509653 wait complete. 2024-05-21T21:54:48.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:48.787+0000 7fca75000700 1 Processor -- start 2024-05-21T21:54:48.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:48.787+0000 7fca75000700 1 -- start start 2024-05-21T21:54:48.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:48.788+0000 7fca75000700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca70101780 0x7fca7019b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:48.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:48.788+0000 7fca75000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca70104760 0x7fca7019bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:48.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:48.788+0000 7fca75000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fca70105100 0x7fca701a2db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:48.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:48.788+0000 7fca75000700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fca7010efa0 con 0x7fca70104760 2024-05-21T21:54:48.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:48.788+0000 7fca75000700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fca7010ee20 con 0x7fca70105100 2024-05-21T21:54:48.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:48.788+0000 7fca75000700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fca7010f120 con 0x7fca70101780 2024-05-21T21:54:48.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:48.788+0000 7fca66a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca70104760 0x7fca7019bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:48.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:48.788+0000 7fca6f400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca70101780 0x7fca7019b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:48.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:48.788+0000 7fca6f400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca70101780 0x7fca7019b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:60428/0 (socket says 172.21.15.151:60428) 2024-05-21T21:54:48.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:48.788+0000 7fca6f400700 1 -- 172.21.15.151:0/2676629004 learned_addr learned my addr 172.21.15.151:0/2676629004 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:54:48.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:48.788+0000 7fca6fe00700 1 --2- 172.21.15.151:0/2676629004 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fca70105100 0x7fca701a2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:48.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:48.789+0000 7fca66a00700 1 -- 172.21.15.151:0/2676629004 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca70101780 msgr2=0x7fca7019b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:48.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:48.789+0000 7fca66a00700 1 --2- 172.21.15.151:0/2676629004 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca70101780 0x7fca7019b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:48.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:48.789+0000 7fca66a00700 1 -- 172.21.15.151:0/2676629004 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fca70105100 msgr2=0x7fca701a2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:48.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:48.789+0000 7fca66a00700 1 --2- 172.21.15.151:0/2676629004 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fca70105100 0x7fca701a2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:48.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:48.789+0000 7fca66a00700 1 -- 172.21.15.151:0/2676629004 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca701a3450 con 0x7fca70104760 2024-05-21T21:54:48.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:48.789+0000 7fca6fe00700 1 --2- 172.21.15.151:0/2676629004 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fca70105100 0x7fca701a2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:54:48.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:48.789+0000 7fca66a00700 1 --2- 172.21.15.151:0/2676629004 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca70104760 0x7fca7019bd30 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fca58009990 tx=0x7fca580047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:54:48.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:48.790+0000 7fca6cc00700 1 -- 172.21.15.151:0/2676629004 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca58026070 con 0x7fca70104760 2024-05-21T21:54:48.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:48.790+0000 7fca6cc00700 1 -- 172.21.15.151:0/2676629004 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca58021410 con 0x7fca70104760 2024-05-21T21:54:48.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:48.790+0000 7fca75000700 1 -- 172.21.15.151:0/2676629004 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca701a36e0 con 0x7fca70104760 2024-05-21T21:54:48.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:48.790+0000 7fca6cc00700 1 -- 172.21.15.151:0/2676629004 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca58017650 con 0x7fca70104760 2024-05-21T21:54:48.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:48.790+0000 7fca75000700 1 -- 172.21.15.151:0/2676629004 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca701a3b30 con 0x7fca70104760 2024-05-21T21:54:48.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:48.791+0000 7fca75000700 1 -- 172.21.15.151:0/2676629004 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca7006a230 con 0x7fca70104760 2024-05-21T21:54:48.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:48.793+0000 7fca6cc00700 1 -- 172.21.15.151:0/2676629004 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fca58029070 con 0x7fca70104760 2024-05-21T21:54:48.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:48.795+0000 7fca6cc00700 1 --2- 172.21.15.151:0/2676629004 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fca540b2880 0x7fca540b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:48.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:48.795+0000 7fca6cc00700 1 -- 172.21.15.151:0/2676629004 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fca580e3350 con 0x7fca70104760 2024-05-21T21:54:48.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:48.795+0000 7fca6f400700 1 --2- 172.21.15.151:0/2676629004 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fca540b2880 0x7fca540b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:48.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:48.796+0000 7fca6f400700 1 --2- 172.21.15.151:0/2676629004 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fca540b2880 0x7fca540b4d40 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fca60005950 tx=0x7fca60009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:54:48.797 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:48.798+0000 7fca6cc00700 1 -- 172.21.15.151:0/2676629004 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fca58013070 con 0x7fca70104760 2024-05-21T21:54:49.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:49.038+0000 7fca75000700 1 -- 172.21.15.151:0/2676629004 --> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fca701a3e50 con 0x7fca540b2880 2024-05-21T21:54:49.040 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:49.040+0000 7fca6cc00700 1 -- 172.21.15.151:0/2676629004 <== mgr.14152 v2:172.21.15.151:6800/2796622483 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7fca701a3e50 con 0x7fca540b2880 2024-05-21T21:54:49.040 INFO:teuthology.orchestra.run.smithi151.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-05-21T21:54:49.040 INFO:teuthology.orchestra.run.smithi151.stdout:smithi151 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300JJ400BGN 372G No 5m ago Has a FileSystem, LVM detected 2024-05-21T21:54:49.040 INFO:teuthology.orchestra.run.smithi151.stdout:smithi164 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300H0400BGN 372G No 5m ago Has a FileSystem, LVM detected 2024-05-21T21:54:49.041 INFO:teuthology.orchestra.run.smithi151.stdout:smithi184 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT626500CE400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-05-21T21:54:49.043 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:49.043+0000 7fca75000700 1 -- 172.21.15.151:0/2676629004 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fca540b2880 msgr2=0x7fca540b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:49.043 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:49.043+0000 7fca75000700 1 --2- 172.21.15.151:0/2676629004 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fca540b2880 0x7fca540b4d40 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fca60005950 tx=0x7fca60009260 comp rx=0 tx=0).stop 2024-05-21T21:54:49.044 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:49.043+0000 7fca75000700 1 -- 172.21.15.151:0/2676629004 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca70104760 msgr2=0x7fca7019bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:49.044 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:49.043+0000 7fca75000700 1 --2- 172.21.15.151:0/2676629004 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca70104760 0x7fca7019bd30 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fca58009990 tx=0x7fca580047b0 comp rx=0 tx=0).stop 2024-05-21T21:54:49.044 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:49.043+0000 7fca75000700 1 -- 172.21.15.151:0/2676629004 shutdown_connections 2024-05-21T21:54:49.044 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:49.043+0000 7fca75000700 1 --2- 172.21.15.151:0/2676629004 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fca540b2880 0x7fca540b4d40 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:49.044 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:49.043+0000 7fca75000700 1 --2- 172.21.15.151:0/2676629004 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fca70105100 0x7fca701a2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:49.044 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:49.043+0000 7fca75000700 1 --2- 172.21.15.151:0/2676629004 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fca70101780 0x7fca7019b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:49.044 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:49.043+0000 7fca75000700 1 --2- 172.21.15.151:0/2676629004 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca70104760 0x7fca7019bd30 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:49.044 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:49.043+0000 7fca75000700 1 -- 172.21.15.151:0/2676629004 >> 172.21.15.151:0/2676629004 conn(0x7fca700fd0e0 msgr2=0x7fca70102550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:54:49.044 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:49.043+0000 7fca75000700 1 -- 172.21.15.151:0/2676629004 shutdown_connections 2024-05-21T21:54:49.044 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:49.044+0000 7fca75000700 1 -- 172.21.15.151:0/2676629004 wait complete. 2024-05-21T21:54:49.867 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-21T21:54:49.877 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi151.front.sepia.ceph.com 2024-05-21T21:54:49.877 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- bash -c 'ceph fs dump' 2024-05-21T21:54:50.324 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:54:50.691 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:54:50 smithi151 ceph-mon[31623]: from='client.14883 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:54:50.692 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:54:50 smithi151 ceph-mon[31623]: pgmap v422: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-21T21:54:50.735 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:54:50 smithi184 ceph-mon[36942]: from='client.14883 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:54:50.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:54:50 smithi184 ceph-mon[36942]: pgmap v422: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-21T21:54:50.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:54:50 smithi164 ceph-mon[36269]: from='client.14883 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:54:50.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:54:50 smithi164 ceph-mon[36269]: pgmap v422: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-21T21:54:52.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:52.394+0000 7ff8e0800700 1 -- 172.21.15.151:0/2451294490 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff8d8101e10 msgr2=0x7ff8d8102230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:52.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:52.394+0000 7ff8e0800700 1 --2- 172.21.15.151:0/2451294490 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff8d8101e10 0x7ff8d8102230 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc009990 tx=0x7ff8cc009ca0 comp rx=0 tx=0).stop 2024-05-21T21:54:52.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:52.394+0000 7ff8e0800700 1 -- 172.21.15.151:0/2451294490 shutdown_connections 2024-05-21T21:54:52.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:52.394+0000 7ff8e0800700 1 --2- 172.21.15.151:0/2451294490 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff8d8103130 0x7ff8d81035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:52.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:52.394+0000 7ff8e0800700 1 --2- 172.21.15.151:0/2451294490 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff8d8101e10 0x7ff8d8102230 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:52.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:52.394+0000 7ff8e0800700 1 --2- 172.21.15.151:0/2451294490 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8d8102770 0x7ff8d8102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:52.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:52.394+0000 7ff8e0800700 1 -- 172.21.15.151:0/2451294490 >> 172.21.15.151:0/2451294490 conn(0x7ff8d80fda30 msgr2=0x7ff8d80ffe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:54:52.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:52.394+0000 7ff8e0800700 1 -- 172.21.15.151:0/2451294490 shutdown_connections 2024-05-21T21:54:52.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:52.394+0000 7ff8e0800700 1 -- 172.21.15.151:0/2451294490 wait complete. 2024-05-21T21:54:52.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:52.396+0000 7ff8e0800700 1 Processor -- start 2024-05-21T21:54:52.398 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:52.396+0000 7ff8e0800700 1 -- start start 2024-05-21T21:54:52.398 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:52.396+0000 7ff8e0800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8d8101e10 0x7ff8d819b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:52.398 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:52.397+0000 7ff8e0800700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff8d8102770 0x7ff8d819bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:52.398 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:52.397+0000 7ff8e0800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff8d8103130 0x7ff8d81a2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:52.398 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:52.397+0000 7ff8e0800700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff8d8111200 con 0x7ff8d8103130 2024-05-21T21:54:52.398 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:52.397+0000 7ff8e0800700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff8d8111080 con 0x7ff8d8101e10 2024-05-21T21:54:52.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:52.397+0000 7ff8e0800700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff8d8111380 con 0x7ff8d8102770 2024-05-21T21:54:52.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:52.397+0000 7ff8dde00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8d8101e10 0x7ff8d819b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:52.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:52.397+0000 7ff8dde00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8d8101e10 0x7ff8d819b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:37584/0 (socket says 172.21.15.151:37584) 2024-05-21T21:54:52.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:52.397+0000 7ff8dde00700 1 -- 172.21.15.151:0/3531630693 learned_addr learned my addr 172.21.15.151:0/3531630693 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:54:52.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:52.397+0000 7ff8dd400700 1 --2- 172.21.15.151:0/3531630693 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff8d8102770 0x7ff8d819bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:52.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:52.397+0000 7ff8de800700 1 --2- 172.21.15.151:0/3531630693 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff8d8103130 0x7ff8d81a2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:52.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:52.397+0000 7ff8dde00700 1 -- 172.21.15.151:0/3531630693 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff8d8102770 msgr2=0x7ff8d819bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:52.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:52.397+0000 7ff8dde00700 1 --2- 172.21.15.151:0/3531630693 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff8d8102770 0x7ff8d819bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:52.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:52.397+0000 7ff8dde00700 1 -- 172.21.15.151:0/3531630693 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff8d8103130 msgr2=0x7ff8d81a2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:52.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:52.397+0000 7ff8dde00700 1 --2- 172.21.15.151:0/3531630693 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff8d8103130 0x7ff8d81a2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:52.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:52.397+0000 7ff8dde00700 1 -- 172.21.15.151:0/3531630693 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8d81a34d0 con 0x7ff8d8101e10 2024-05-21T21:54:52.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:52.398+0000 7ff8dde00700 1 --2- 172.21.15.151:0/3531630693 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8d8101e10 0x7ff8d819b870 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc006010 tx=0x7ff8cc0057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:54:52.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:52.398+0000 7ff8d2a00700 1 -- 172.21.15.151:0/3531630693 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8cc027070 con 0x7ff8d8101e10 2024-05-21T21:54:52.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:52.398+0000 7ff8d2a00700 1 -- 172.21.15.151:0/3531630693 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8cc00fb90 con 0x7ff8d8101e10 2024-05-21T21:54:52.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:52.398+0000 7ff8d2a00700 1 -- 172.21.15.151:0/3531630693 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8cc017650 con 0x7ff8d8101e10 2024-05-21T21:54:52.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:52.398+0000 7ff8e0800700 1 -- 172.21.15.151:0/3531630693 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8d81a3760 con 0x7ff8d8101e10 2024-05-21T21:54:52.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:52.398+0000 7ff8e0800700 1 -- 172.21.15.151:0/3531630693 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8d81a3b60 con 0x7ff8d8101e10 2024-05-21T21:54:52.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:52.399+0000 7ff8e0800700 1 -- 172.21.15.151:0/3531630693 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8d806a270 con 0x7ff8d8101e10 2024-05-21T21:54:52.401 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:52.401+0000 7ff8d2a00700 1 -- 172.21.15.151:0/3531630693 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7ff8cc02a070 con 0x7ff8d8101e10 2024-05-21T21:54:52.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:52.403+0000 7ff8d2a00700 1 --2- 172.21.15.151:0/3531630693 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7ff8c00b2900 0x7ff8c00b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:52.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:52.403+0000 7ff8d2a00700 1 -- 172.21.15.151:0/3531630693 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7ff8cc0e3530 con 0x7ff8d8101e10 2024-05-21T21:54:52.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:52.403+0000 7ff8dd400700 1 --2- 172.21.15.151:0/3531630693 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7ff8c00b2900 0x7ff8c00b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:52.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:52.404+0000 7ff8dd400700 1 --2- 172.21.15.151:0/3531630693 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7ff8c00b2900 0x7ff8c00b4dc0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff8c8009c60 tx=0x7ff8c8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:54:52.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:52.405+0000 7ff8d2a00700 1 -- 172.21.15.151:0/3531630693 <== mon.2 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff8cc031080 con 0x7ff8d8101e10 2024-05-21T21:54:52.686 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:52.685+0000 7ff8e0800700 1 -- 172.21.15.151:0/3531630693 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7ff8d802e770 con 0x7ff8d8101e10 2024-05-21T21:54:52.686 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:54:52 smithi151 ceph-mon[31623]: pgmap v423: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-21T21:54:52.687 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:52.686+0000 7ff8d2a00700 1 -- 172.21.15.151:0/3531630693 <== mon.2 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) ==== 76+0+3108 (secure 0 0 0) 0x7ff8d802e770 con 0x7ff8d8101e10 2024-05-21T21:54:52.687 INFO:teuthology.orchestra.run.smithi151.stdout:e28 2024-05-21T21:54:52.687 INFO:teuthology.orchestra.run.smithi151.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-21T21:54:52.687 INFO:teuthology.orchestra.run.smithi151.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-21T21:54:52.687 INFO:teuthology.orchestra.run.smithi151.stdout:legacy client fscid: 1 2024-05-21T21:54:52.687 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T21:54:52.687 INFO:teuthology.orchestra.run.smithi151.stdout:Filesystem 'cephfs' (1) 2024-05-21T21:54:52.688 INFO:teuthology.orchestra.run.smithi151.stdout:fs_name cephfs 2024-05-21T21:54:52.688 INFO:teuthology.orchestra.run.smithi151.stdout:epoch 28 2024-05-21T21:54:52.688 INFO:teuthology.orchestra.run.smithi151.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-21T21:54:52.688 INFO:teuthology.orchestra.run.smithi151.stdout:created 2024-05-21T21:51:19.552069+0000 2024-05-21T21:54:52.688 INFO:teuthology.orchestra.run.smithi151.stdout:modified 2024-05-21T21:51:43.002737+0000 2024-05-21T21:54:52.688 INFO:teuthology.orchestra.run.smithi151.stdout:tableserver 0 2024-05-21T21:54:52.688 INFO:teuthology.orchestra.run.smithi151.stdout:root 0 2024-05-21T21:54:52.688 INFO:teuthology.orchestra.run.smithi151.stdout:session_timeout 300 2024-05-21T21:54:52.688 INFO:teuthology.orchestra.run.smithi151.stdout:session_autoclose 300 2024-05-21T21:54:52.688 INFO:teuthology.orchestra.run.smithi151.stdout:max_file_size 1099511627776 2024-05-21T21:54:52.688 INFO:teuthology.orchestra.run.smithi151.stdout:max_xattr_size 65536 2024-05-21T21:54:52.688 INFO:teuthology.orchestra.run.smithi151.stdout:required_client_features {} 2024-05-21T21:54:52.688 INFO:teuthology.orchestra.run.smithi151.stdout:last_failure 0 2024-05-21T21:54:52.688 INFO:teuthology.orchestra.run.smithi151.stdout:last_failure_osd_epoch 0 2024-05-21T21:54:52.689 INFO:teuthology.orchestra.run.smithi151.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-21T21:54:52.689 INFO:teuthology.orchestra.run.smithi151.stdout:max_mds 5 2024-05-21T21:54:52.689 INFO:teuthology.orchestra.run.smithi151.stdout:in 0,1,2,3,4 2024-05-21T21:54:52.689 INFO:teuthology.orchestra.run.smithi151.stdout:up {0=24530,1=24493,2=24491,3=24478,4=24473} 2024-05-21T21:54:52.689 INFO:teuthology.orchestra.run.smithi151.stdout:failed 2024-05-21T21:54:52.689 INFO:teuthology.orchestra.run.smithi151.stdout:damaged 2024-05-21T21:54:52.689 INFO:teuthology.orchestra.run.smithi151.stdout:stopped 2024-05-21T21:54:52.689 INFO:teuthology.orchestra.run.smithi151.stdout:data_pools [3] 2024-05-21T21:54:52.689 INFO:teuthology.orchestra.run.smithi151.stdout:metadata_pool 2 2024-05-21T21:54:52.689 INFO:teuthology.orchestra.run.smithi151.stdout:inline_data disabled 2024-05-21T21:54:52.689 INFO:teuthology.orchestra.run.smithi151.stdout:balancer 2024-05-21T21:54:52.689 INFO:teuthology.orchestra.run.smithi151.stdout:bal_rank_mask -1 2024-05-21T21:54:52.689 INFO:teuthology.orchestra.run.smithi151.stdout:standby_count_wanted 0 2024-05-21T21:54:52.689 INFO:teuthology.orchestra.run.smithi151.stdout:qdb_cluster leader: 24530 members: 24473,24478,24491,24493,24530 2024-05-21T21:54:52.690 INFO:teuthology.orchestra.run.smithi151.stdout:[mds.b{0:24530} state up:active seq 7 addr [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] compat {c=[1],r=[1],i=[1fff]}] 2024-05-21T21:54:52.690 INFO:teuthology.orchestra.run.smithi151.stdout:[mds.f{0:24466} state up:standby-replay seq 1 addr [v2:172.21.15.184:6834/3264142621,v1:172.21.15.184:6835/3264142621] compat {c=[1],r=[1],i=[1fff]}] 2024-05-21T21:54:52.690 INFO:teuthology.orchestra.run.smithi151.stdout:[mds.i{1:24493} state up:active seq 14 addr [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] compat {c=[1],r=[1],i=[1fff]}] 2024-05-21T21:54:52.690 INFO:teuthology.orchestra.run.smithi151.stdout:[mds.d{1:24461} state up:standby-replay seq 1 addr [v2:172.21.15.151:6836/3367640960,v1:172.21.15.151:6837/3367640960] compat {c=[1],r=[1],i=[1fff]}] 2024-05-21T21:54:52.690 INFO:teuthology.orchestra.run.smithi151.stdout:[mds.h{2:24491} state up:active seq 15 addr [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] compat {c=[1],r=[1],i=[1fff]}] 2024-05-21T21:54:52.690 INFO:teuthology.orchestra.run.smithi151.stdout:[mds.k{2:24443} state up:standby-replay seq 1 addr [v2:172.21.15.164:6832/2467900844,v1:172.21.15.164:6833/2467900844] compat {c=[1],r=[1],i=[1fff]}] 2024-05-21T21:54:52.690 INFO:teuthology.orchestra.run.smithi151.stdout:[mds.c{3:24478} state up:active seq 16 addr [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] compat {c=[1],r=[1],i=[1fff]}] 2024-05-21T21:54:52.690 INFO:teuthology.orchestra.run.smithi151.stdout:[mds.l{3:24436} state up:standby-replay seq 1 addr [v2:172.21.15.184:6832/2026266114,v1:172.21.15.184:6833/2026266114] compat {c=[1],r=[1],i=[1fff]}] 2024-05-21T21:54:52.690 INFO:teuthology.orchestra.run.smithi151.stdout:[mds.e{4:24473} state up:active seq 17 addr [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] compat {c=[1],r=[1],i=[1fff]}] 2024-05-21T21:54:52.690 INFO:teuthology.orchestra.run.smithi151.stdout:[mds.a{4:14403} state up:standby-replay seq 1 addr [v2:172.21.15.151:6840/1749723794,v1:172.21.15.151:6841/1749723794] compat {c=[1],r=[1],i=[1fff]}] 2024-05-21T21:54:52.690 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T21:54:52.690 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T21:54:52.690 INFO:teuthology.orchestra.run.smithi151.stdout:Standby daemons: 2024-05-21T21:54:52.691 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T21:54:52.691 INFO:teuthology.orchestra.run.smithi151.stdout:[mds.j{-1:14358} state up:standby seq 1 addr [v2:172.21.15.151:6834/158010529,v1:172.21.15.151:6835/158010529] compat {c=[1],r=[1],i=[1fff]}] 2024-05-21T21:54:52.691 INFO:teuthology.orchestra.run.smithi151.stdout:[mds.g{-1:14397} state up:standby seq 1 addr [v2:172.21.15.151:6838/2069058115,v1:172.21.15.151:6839/2069058115] compat {c=[1],r=[1],i=[1fff]}] 2024-05-21T21:54:52.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:52.689+0000 7ff8e0800700 1 -- 172.21.15.151:0/3531630693 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7ff8c00b2900 msgr2=0x7ff8c00b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:52.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:52.689+0000 7ff8e0800700 1 --2- 172.21.15.151:0/3531630693 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7ff8c00b2900 0x7ff8c00b4dc0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff8c8009c60 tx=0x7ff8c8009260 comp rx=0 tx=0).stop 2024-05-21T21:54:52.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:52.689+0000 7ff8e0800700 1 -- 172.21.15.151:0/3531630693 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8d8101e10 msgr2=0x7ff8d819b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:52.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:52.689+0000 7ff8e0800700 1 --2- 172.21.15.151:0/3531630693 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8d8101e10 0x7ff8d819b870 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc006010 tx=0x7ff8cc0057c0 comp rx=0 tx=0).stop 2024-05-21T21:54:52.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:52.690+0000 7ff8e0800700 1 -- 172.21.15.151:0/3531630693 shutdown_connections 2024-05-21T21:54:52.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:52.690+0000 7ff8e0800700 1 --2- 172.21.15.151:0/3531630693 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7ff8c00b2900 0x7ff8c00b4dc0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:52.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:52.690+0000 7ff8e0800700 1 --2- 172.21.15.151:0/3531630693 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8d8101e10 0x7ff8d819b870 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:52.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:52.690+0000 7ff8e0800700 1 --2- 172.21.15.151:0/3531630693 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff8d8103130 0x7ff8d81a2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:52.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:52.690+0000 7ff8e0800700 1 --2- 172.21.15.151:0/3531630693 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff8d8102770 0x7ff8d819bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:52.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:52.690+0000 7ff8e0800700 1 -- 172.21.15.151:0/3531630693 >> 172.21.15.151:0/3531630693 conn(0x7ff8d80fda30 msgr2=0x7ff8d80ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:54:52.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:52.690+0000 7ff8e0800700 1 -- 172.21.15.151:0/3531630693 shutdown_connections 2024-05-21T21:54:52.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:52.690+0000 7ff8e0800700 1 -- 172.21.15.151:0/3531630693 wait complete. 2024-05-21T21:54:52.692 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 28 2024-05-21T21:54:52.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:54:52 smithi184 ceph-mon[36942]: pgmap v423: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-21T21:54:52.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:54:52 smithi164 ceph-mon[36269]: pgmap v423: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-21T21:54:53.451 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- bash -c 'ceph osd dump' 2024-05-21T21:54:53.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:54:53 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/3531630693' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-21T21:54:53.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:54:53 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/3531630693' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-21T21:54:53.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:54:53 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/3531630693' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-21T21:54:53.896 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:54:54.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:54:54 smithi151 ceph-mon[31623]: pgmap v424: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-21T21:54:54.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:54:54 smithi184 ceph-mon[36942]: pgmap v424: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-21T21:54:54.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:54:54 smithi164 ceph-mon[36269]: pgmap v424: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-21T21:54:55.974 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:55.973+0000 7ff868e00700 1 -- 172.21.15.151:0/92215227 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff8641024a0 msgr2=0x7ff8641028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:55.974 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:55.973+0000 7ff868e00700 1 --2- 172.21.15.151:0/92215227 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff8641024a0 0x7ff8641028c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff85400b0d0 tx=0x7ff85400b3e0 comp rx=0 tx=0).stop 2024-05-21T21:54:55.974 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:55.973+0000 7ff868e00700 1 -- 172.21.15.151:0/92215227 shutdown_connections 2024-05-21T21:54:55.974 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:55.973+0000 7ff868e00700 1 --2- 172.21.15.151:0/92215227 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff864104040 0x7ff86410a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:55.975 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:55.973+0000 7ff868e00700 1 --2- 172.21.15.151:0/92215227 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff8641024a0 0x7ff8641028c0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:55.975 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:55.973+0000 7ff868e00700 1 --2- 172.21.15.151:0/92215227 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff8641036a0 0x7ff864103b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:55.975 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:55.973+0000 7ff868e00700 1 -- 172.21.15.151:0/92215227 >> 172.21.15.151:0/92215227 conn(0x7ff8640fda40 msgr2=0x7ff8640ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:54:55.975 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:55.973+0000 7ff868e00700 1 -- 172.21.15.151:0/92215227 shutdown_connections 2024-05-21T21:54:55.975 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:55.973+0000 7ff868e00700 1 -- 172.21.15.151:0/92215227 wait complete. 2024-05-21T21:54:55.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:55.974+0000 7ff868e00700 1 Processor -- start 2024-05-21T21:54:55.979 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:55.974+0000 7ff868e00700 1 -- start start 2024-05-21T21:54:55.979 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:55.975+0000 7ff868e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff8641024a0 0x7ff864106430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:55.979 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:55.975+0000 7ff868e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff8641036a0 0x7ff864104a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:55.979 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:55.975+0000 7ff868e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff864104040 0x7ff864104fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:55.979 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:55.975+0000 7ff868e00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff86410df80 con 0x7ff8641024a0 2024-05-21T21:54:55.979 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:55.975+0000 7ff868e00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff86410de00 con 0x7ff864104040 2024-05-21T21:54:55.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:55.975+0000 7ff868e00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff86410e100 con 0x7ff8641036a0 2024-05-21T21:54:55.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:55.975+0000 7ff861e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff8641024a0 0x7ff864106430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:55.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:55.975+0000 7ff861e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff8641024a0 0x7ff864106430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:43000/0 (socket says 172.21.15.151:43000) 2024-05-21T21:54:55.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:55.975+0000 7ff861e00700 1 -- 172.21.15.151:0/3715465869 learned_addr learned my addr 172.21.15.151:0/3715465869 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:54:55.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:55.975+0000 7ff862800700 1 --2- 172.21.15.151:0/3715465869 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff864104040 0x7ff864104fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:55.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:55.975+0000 7ff859400700 1 --2- 172.21.15.151:0/3715465869 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff8641036a0 0x7ff864104a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:55.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:55.976+0000 7ff861e00700 1 -- 172.21.15.151:0/3715465869 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff8641036a0 msgr2=0x7ff864104a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:55.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:55.976+0000 7ff861e00700 1 --2- 172.21.15.151:0/3715465869 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff8641036a0 0x7ff864104a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:55.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:55.976+0000 7ff861e00700 1 -- 172.21.15.151:0/3715465869 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff864104040 msgr2=0x7ff864104fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:55.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:55.976+0000 7ff861e00700 1 --2- 172.21.15.151:0/3715465869 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff864104040 0x7ff864104fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:55.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:55.976+0000 7ff861e00700 1 -- 172.21.15.151:0/3715465869 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff864105870 con 0x7ff8641024a0 2024-05-21T21:54:55.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:55.976+0000 7ff862800700 1 --2- 172.21.15.151:0/3715465869 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff864104040 0x7ff864104fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:54:55.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:55.976+0000 7ff861e00700 1 --2- 172.21.15.151:0/3715465869 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff8641024a0 0x7ff864106430 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff854009040 tx=0x7ff854004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:54:55.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:55.976+0000 7ff85b400700 1 -- 172.21.15.151:0/3715465869 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff85401f030 con 0x7ff8641024a0 2024-05-21T21:54:55.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:55.976+0000 7ff85b400700 1 -- 172.21.15.151:0/3715465869 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff854005040 con 0x7ff8641024a0 2024-05-21T21:54:55.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:55.976+0000 7ff85b400700 1 -- 172.21.15.151:0/3715465869 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff85401b690 con 0x7ff8641024a0 2024-05-21T21:54:55.982 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:55.976+0000 7ff868e00700 1 -- 172.21.15.151:0/3715465869 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8641a38f0 con 0x7ff8641024a0 2024-05-21T21:54:55.982 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:55.977+0000 7ff868e00700 1 -- 172.21.15.151:0/3715465869 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8641a3cf0 con 0x7ff8641024a0 2024-05-21T21:54:55.982 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:55.978+0000 7ff868e00700 1 -- 172.21.15.151:0/3715465869 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff86406a230 con 0x7ff8641024a0 2024-05-21T21:54:55.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:55.980+0000 7ff85b400700 1 -- 172.21.15.151:0/3715465869 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7ff854022030 con 0x7ff8641024a0 2024-05-21T21:54:55.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:55.981+0000 7ff85b400700 1 --2- 172.21.15.151:0/3715465869 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7ff8480b2900 0x7ff8480b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:55.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:55.981+0000 7ff85b400700 1 -- 172.21.15.151:0/3715465869 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7ff85400f020 con 0x7ff8641024a0 2024-05-21T21:54:55.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:55.982+0000 7ff859400700 1 --2- 172.21.15.151:0/3715465869 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7ff8480b2900 0x7ff8480b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:55.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:55.982+0000 7ff859400700 1 --2- 172.21.15.151:0/3715465869 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7ff8480b2900 0x7ff8480b4dc0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff84c009c60 tx=0x7ff84c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:54:55.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:55.986+0000 7ff85b400700 1 -- 172.21.15.151:0/3715465869 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff854029020 con 0x7ff8641024a0 2024-05-21T21:54:56.216 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:56.216+0000 7ff868e00700 1 -- 172.21.15.151:0/3715465869 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7ff8641a46c0 con 0x7ff8641024a0 2024-05-21T21:54:56.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:56.217+0000 7ff85b400700 1 -- 172.21.15.151:0/3715465869 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v82) ==== 56+0+5330 (secure 0 0 0) 0x7ff8540aac40 con 0x7ff8641024a0 2024-05-21T21:54:56.218 INFO:teuthology.orchestra.run.smithi151.stdout:epoch 82 2024-05-21T21:54:56.218 INFO:teuthology.orchestra.run.smithi151.stdout:fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 2024-05-21T21:54:56.218 INFO:teuthology.orchestra.run.smithi151.stdout:created 2024-05-21T21:41:52.412833+0000 2024-05-21T21:54:56.218 INFO:teuthology.orchestra.run.smithi151.stdout:modified 2024-05-21T21:52:10.856587+0000 2024-05-21T21:54:56.218 INFO:teuthology.orchestra.run.smithi151.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-05-21T21:54:56.219 INFO:teuthology.orchestra.run.smithi151.stdout:crush_version 26 2024-05-21T21:54:56.219 INFO:teuthology.orchestra.run.smithi151.stdout:full_ratio 0.95 2024-05-21T21:54:56.219 INFO:teuthology.orchestra.run.smithi151.stdout:backfillfull_ratio 0.9 2024-05-21T21:54:56.219 INFO:teuthology.orchestra.run.smithi151.stdout:nearfull_ratio 0.85 2024-05-21T21:54:56.219 INFO:teuthology.orchestra.run.smithi151.stdout:require_min_compat_client luminous 2024-05-21T21:54:56.219 INFO:teuthology.orchestra.run.smithi151.stdout:min_compat_client luminous 2024-05-21T21:54:56.219 INFO:teuthology.orchestra.run.smithi151.stdout:require_osd_release squid 2024-05-21T21:54:56.219 INFO:teuthology.orchestra.run.smithi151.stdout:stretch_mode_enabled false 2024-05-21T21:54:56.219 INFO:teuthology.orchestra.run.smithi151.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-05-21T21:54:56.219 INFO:teuthology.orchestra.run.smithi151.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 80 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-05-21T21:54:56.219 INFO:teuthology.orchestra.run.smithi151.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 80 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-05-21T21:54:56.219 INFO:teuthology.orchestra.run.smithi151.stdout:max_osd 12 2024-05-21T21:54:56.219 INFO:teuthology.orchestra.run.smithi151.stdout:osd.0 up in weight 1 up_from 8 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.151:6802/2420400473,v1:172.21.15.151:6803/2420400473] [v2:172.21.15.151:6804/2420400473,v1:172.21.15.151:6805/2420400473] exists,up 18db8e54-0cfc-4e12-9c84-6f72a99cc989 2024-05-21T21:54:56.220 INFO:teuthology.orchestra.run.smithi151.stdout:osd.1 up in weight 1 up_from 13 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.164:6800/3900240447,v1:172.21.15.164:6801/3900240447] [v2:172.21.15.164:6802/3900240447,v1:172.21.15.164:6803/3900240447] exists,up bac384c7-1102-4815-81a8-10cc7661d230 2024-05-21T21:54:56.220 INFO:teuthology.orchestra.run.smithi151.stdout:osd.2 up in weight 1 up_from 18 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.184:6800/3971150737,v1:172.21.15.184:6801/3971150737] [v2:172.21.15.184:6802/3971150737,v1:172.21.15.184:6803/3971150737] exists,up b7e4a782-1518-4d43-bee9-7fd768ee5cb4 2024-05-21T21:54:56.220 INFO:teuthology.orchestra.run.smithi151.stdout:osd.3 up in weight 1 up_from 26 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.151:6810/556594400,v1:172.21.15.151:6811/556594400] [v2:172.21.15.151:6812/556594400,v1:172.21.15.151:6813/556594400] exists,up ab3df826-39e3-4635-a49c-2b5ffdbf12f3 2024-05-21T21:54:56.220 INFO:teuthology.orchestra.run.smithi151.stdout:osd.4 up in weight 1 up_from 32 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.164:6808/988881973,v1:172.21.15.164:6809/988881973] [v2:172.21.15.164:6810/988881973,v1:172.21.15.164:6811/988881973] exists,up 00cbf6ae-0f42-4685-a77d-d803c6467aa0 2024-05-21T21:54:56.220 INFO:teuthology.orchestra.run.smithi151.stdout:osd.5 up in weight 1 up_from 38 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.184:6808/3293311106,v1:172.21.15.184:6809/3293311106] [v2:172.21.15.184:6810/3293311106,v1:172.21.15.184:6811/3293311106] exists,up fad14221-0ef0-4d40-b414-c6c9f1f9718c 2024-05-21T21:54:56.220 INFO:teuthology.orchestra.run.smithi151.stdout:osd.6 up in weight 1 up_from 44 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.151:6818/3377550762,v1:172.21.15.151:6819/3377550762] [v2:172.21.15.151:6820/3377550762,v1:172.21.15.151:6821/3377550762] exists,up ab8a60f6-0dec-4c1c-89db-f2e2f638f1ff 2024-05-21T21:54:56.220 INFO:teuthology.orchestra.run.smithi151.stdout:osd.7 up in weight 1 up_from 49 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.164:6816/3869696936,v1:172.21.15.164:6817/3869696936] [v2:172.21.15.164:6818/3869696936,v1:172.21.15.164:6819/3869696936] exists,up db332268-9972-4ed7-a307-822b83a0fbb1 2024-05-21T21:54:56.220 INFO:teuthology.orchestra.run.smithi151.stdout:osd.8 up in weight 1 up_from 55 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.184:6816/575273541,v1:172.21.15.184:6817/575273541] [v2:172.21.15.184:6818/575273541,v1:172.21.15.184:6819/575273541] exists,up fe2fbe1c-549a-4947-a188-426dfca04d10 2024-05-21T21:54:56.220 INFO:teuthology.orchestra.run.smithi151.stdout:osd.9 up in weight 1 up_from 61 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.151:6826/337128533,v1:172.21.15.151:6827/337128533] [v2:172.21.15.151:6828/337128533,v1:172.21.15.151:6829/337128533] exists,up 3a7b450f-8004-4adf-bae3-970918f98f50 2024-05-21T21:54:56.220 INFO:teuthology.orchestra.run.smithi151.stdout:osd.10 up in weight 1 up_from 66 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.164:6824/1823448782,v1:172.21.15.164:6825/1823448782] [v2:172.21.15.164:6826/1823448782,v1:172.21.15.164:6827/1823448782] exists,up 8fcdf271-90c4-4db3-be69-b0ca716dfd80 2024-05-21T21:54:56.220 INFO:teuthology.orchestra.run.smithi151.stdout:osd.11 up in weight 1 up_from 72 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.184:6824/1563340128,v1:172.21.15.184:6825/1563340128] [v2:172.21.15.184:6826/1563340128,v1:172.21.15.184:6827/1563340128] exists,up e661477b-0c09-409f-a2fe-6ebd34642a8c 2024-05-21T21:54:56.220 INFO:teuthology.orchestra.run.smithi151.stdout:pg_upmap_items 2.6 [8,11] 2024-05-21T21:54:56.220 INFO:teuthology.orchestra.run.smithi151.stdout:pg_upmap_items 2.e [2,4] 2024-05-21T21:54:56.221 INFO:teuthology.orchestra.run.smithi151.stdout:pg_upmap_items 2.10 [2,1] 2024-05-21T21:54:56.221 INFO:teuthology.orchestra.run.smithi151.stdout:pg_upmap_items 2.1d [8,11] 2024-05-21T21:54:56.221 INFO:teuthology.orchestra.run.smithi151.stdout:pg_upmap_items 3.5 [3,9] 2024-05-21T21:54:56.221 INFO:teuthology.orchestra.run.smithi151.stdout:pg_upmap_items 3.13 [4,9] 2024-05-21T21:54:56.221 INFO:teuthology.orchestra.run.smithi151.stdout:pg_upmap_items 3.1d [4,6] 2024-05-21T21:54:56.221 INFO:teuthology.orchestra.run.smithi151.stdout:pg_upmap_items 3.28 [5,8] 2024-05-21T21:54:56.221 INFO:teuthology.orchestra.run.smithi151.stdout:pg_upmap_items 3.36 [4,9] 2024-05-21T21:54:56.221 INFO:teuthology.orchestra.run.smithi151.stdout:blocklist 172.21.15.151:0/31963128 expires 2024-05-22T21:43:09.041455+0000 2024-05-21T21:54:56.221 INFO:teuthology.orchestra.run.smithi151.stdout:blocklist 172.21.15.151:6800/3428827300 expires 2024-05-22T21:43:09.041455+0000 2024-05-21T21:54:56.221 INFO:teuthology.orchestra.run.smithi151.stdout:blocklist 172.21.15.151:6801/3428827300 expires 2024-05-22T21:43:09.041455+0000 2024-05-21T21:54:56.221 INFO:teuthology.orchestra.run.smithi151.stdout:blocklist 172.21.15.151:0/131989141 expires 2024-05-22T21:43:09.041455+0000 2024-05-21T21:54:56.221 INFO:teuthology.orchestra.run.smithi151.stdout:blocklist 172.21.15.151:6801/1128885292 expires 2024-05-22T21:42:35.440147+0000 2024-05-21T21:54:56.221 INFO:teuthology.orchestra.run.smithi151.stdout:blocklist 172.21.15.151:0/2703664519 expires 2024-05-22T21:43:09.041455+0000 2024-05-21T21:54:56.222 INFO:teuthology.orchestra.run.smithi151.stdout:blocklist 172.21.15.151:6800/1128885292 expires 2024-05-22T21:42:35.440147+0000 2024-05-21T21:54:56.222 INFO:teuthology.orchestra.run.smithi151.stdout:blocklist 172.21.15.151:0/3052943699 expires 2024-05-22T21:42:35.440147+0000 2024-05-21T21:54:56.222 INFO:teuthology.orchestra.run.smithi151.stdout:blocklist 172.21.15.151:0/3929452511 expires 2024-05-22T21:42:35.440147+0000 2024-05-21T21:54:56.222 INFO:teuthology.orchestra.run.smithi151.stdout:blocklist 172.21.15.151:0/3762889700 expires 2024-05-22T21:42:35.440147+0000 2024-05-21T21:54:56.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:56.220+0000 7ff868e00700 1 -- 172.21.15.151:0/3715465869 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7ff8480b2900 msgr2=0x7ff8480b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:56.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:56.220+0000 7ff868e00700 1 --2- 172.21.15.151:0/3715465869 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7ff8480b2900 0x7ff8480b4dc0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff84c009c60 tx=0x7ff84c009260 comp rx=0 tx=0).stop 2024-05-21T21:54:56.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:56.220+0000 7ff868e00700 1 -- 172.21.15.151:0/3715465869 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff8641024a0 msgr2=0x7ff864106430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:56.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:56.220+0000 7ff868e00700 1 --2- 172.21.15.151:0/3715465869 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff8641024a0 0x7ff864106430 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff854009040 tx=0x7ff854004d00 comp rx=0 tx=0).stop 2024-05-21T21:54:56.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:56.221+0000 7ff868e00700 1 -- 172.21.15.151:0/3715465869 shutdown_connections 2024-05-21T21:54:56.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:56.221+0000 7ff868e00700 1 --2- 172.21.15.151:0/3715465869 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7ff8480b2900 0x7ff8480b4dc0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:56.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:56.221+0000 7ff868e00700 1 --2- 172.21.15.151:0/3715465869 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff864104040 0x7ff864104fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:56.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:56.221+0000 7ff868e00700 1 --2- 172.21.15.151:0/3715465869 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff8641024a0 0x7ff864106430 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:56.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:56.221+0000 7ff868e00700 1 --2- 172.21.15.151:0/3715465869 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff8641036a0 0x7ff864104a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:56.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:56.221+0000 7ff868e00700 1 -- 172.21.15.151:0/3715465869 >> 172.21.15.151:0/3715465869 conn(0x7ff8640fda40 msgr2=0x7ff86406b220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:54:56.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:56.221+0000 7ff868e00700 1 -- 172.21.15.151:0/3715465869 shutdown_connections 2024-05-21T21:54:56.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:56.221+0000 7ff868e00700 1 -- 172.21.15.151:0/3715465869 wait complete. 2024-05-21T21:54:56.620 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:54:56 smithi151 ceph-mon[31623]: pgmap v425: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-21T21:54:56.620 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:54:56 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/3715465869' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-21T21:54:56.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:54:56 smithi184 ceph-mon[36942]: pgmap v425: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-21T21:54:56.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:54:56 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/3715465869' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-21T21:54:56.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:54:56 smithi164 ceph-mon[36269]: pgmap v425: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-21T21:54:56.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:54:56 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/3715465869' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-21T21:54:57.030 INFO:teuthology.run_tasks:Running task fs.ready... 2024-05-21T21:54:57.041 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-21T21:54:57.489 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:54:58.679 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:54:58 smithi151 ceph-mon[31623]: pgmap v426: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-21T21:54:58.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:54:58 smithi184 ceph-mon[36942]: pgmap v426: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-21T21:54:58.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:54:58 smithi164 ceph-mon[36269]: pgmap v426: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-21T21:54:59.517 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:59.516+0000 7f4dc0600700 1 -- 172.21.15.151:0/823170684 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4db8105760 msgr2=0x7f4db8105be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:59.517 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:59.516+0000 7f4dc0600700 1 --2- 172.21.15.151:0/823170684 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4db8105760 0x7f4db8105be0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4da4009990 tx=0x7f4da4009ca0 comp rx=0 tx=0).stop 2024-05-21T21:54:59.517 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:59.516+0000 7f4dc0600700 1 -- 172.21.15.151:0/823170684 shutdown_connections 2024-05-21T21:54:59.517 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:59.516+0000 7f4dc0600700 1 --2- 172.21.15.151:0/823170684 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4db8106120 0x7f4db8112630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:59.517 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:59.516+0000 7f4dc0600700 1 --2- 172.21.15.151:0/823170684 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4db810b760 0x7f4db810bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:59.517 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:59.516+0000 7f4dc0600700 1 --2- 172.21.15.151:0/823170684 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4db8105760 0x7f4db8105be0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:59.517 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:59.516+0000 7f4dc0600700 1 -- 172.21.15.151:0/823170684 >> 172.21.15.151:0/823170684 conn(0x7f4db8101250 msgr2=0x7f4db8103670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:54:59.517 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:59.516+0000 7f4dc0600700 1 -- 172.21.15.151:0/823170684 shutdown_connections 2024-05-21T21:54:59.518 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:59.516+0000 7f4dc0600700 1 -- 172.21.15.151:0/823170684 wait complete. 2024-05-21T21:54:59.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:59.518+0000 7f4dc0600700 1 Processor -- start 2024-05-21T21:54:59.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:59.518+0000 7f4dc0600700 1 -- start start 2024-05-21T21:54:59.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:59.518+0000 7f4dc0600700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4db8105760 0x7f4db8077a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:59.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:59.518+0000 7f4dc0600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4db8106120 0x7f4db8077f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:59.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:59.518+0000 7f4dc0600700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4db810b760 0x7f4db80784d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:59.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:59.518+0000 7f4dc0600700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4db81159b0 con 0x7f4db8106120 2024-05-21T21:54:59.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:59.518+0000 7f4dc0600700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4db8115830 con 0x7f4db8105760 2024-05-21T21:54:59.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:59.518+0000 7f4dc0600700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4db8115b30 con 0x7f4db810b760 2024-05-21T21:54:59.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:59.519+0000 7f4dbd200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4db8106120 0x7f4db8077f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:59.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:59.519+0000 7f4dbd200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4db8106120 0x7f4db8077f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:43022/0 (socket says 172.21.15.151:43022) 2024-05-21T21:54:59.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:59.519+0000 7f4dbd200700 1 -- 172.21.15.151:0/177074412 learned_addr learned my addr 172.21.15.151:0/177074412 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:54:59.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:59.519+0000 7f4dbdc00700 1 --2- 172.21.15.151:0/177074412 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4db8105760 0x7f4db8077a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:59.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:59.519+0000 7f4dbe600700 1 --2- 172.21.15.151:0/177074412 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4db810b760 0x7f4db80784d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:59.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:59.519+0000 7f4dbdc00700 1 -- 172.21.15.151:0/177074412 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4db810b760 msgr2=0x7f4db80784d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:59.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:59.519+0000 7f4dbdc00700 1 --2- 172.21.15.151:0/177074412 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4db810b760 0x7f4db80784d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:59.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:59.519+0000 7f4dbdc00700 1 -- 172.21.15.151:0/177074412 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4db8106120 msgr2=0x7f4db8077f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:59.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:59.519+0000 7f4dbdc00700 1 --2- 172.21.15.151:0/177074412 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4db8106120 0x7f4db8077f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:59.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:59.519+0000 7f4dbdc00700 1 -- 172.21.15.151:0/177074412 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4db81ac530 con 0x7f4db8105760 2024-05-21T21:54:59.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:59.519+0000 7f4dbd200700 1 --2- 172.21.15.151:0/177074412 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4db8106120 0x7f4db8077f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:54:59.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:59.519+0000 7f4dbdc00700 1 --2- 172.21.15.151:0/177074412 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4db8105760 0x7f4db8077a50 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4dac007ef0 tx=0x7f4dac00d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:54:59.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:59.520+0000 7f4db6000700 1 -- 172.21.15.151:0/177074412 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dac004db0 con 0x7f4db8105760 2024-05-21T21:54:59.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:59.520+0000 7f4dc0600700 1 -- 172.21.15.151:0/177074412 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4db81ac820 con 0x7f4db8105760 2024-05-21T21:54:59.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:59.520+0000 7f4db6000700 1 -- 172.21.15.151:0/177074412 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4dac005020 con 0x7f4db8105760 2024-05-21T21:54:59.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:59.520+0000 7f4db6000700 1 -- 172.21.15.151:0/177074412 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dac015660 con 0x7f4db8105760 2024-05-21T21:54:59.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:59.520+0000 7f4dc0600700 1 -- 172.21.15.151:0/177074412 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4db81acdb0 con 0x7f4db8105760 2024-05-21T21:54:59.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:59.521+0000 7f4dc0600700 1 -- 172.21.15.151:0/177074412 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4db8106ea0 con 0x7f4db8105760 2024-05-21T21:54:59.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:59.523+0000 7f4db6000700 1 -- 172.21.15.151:0/177074412 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f4dac00db20 con 0x7f4db8105760 2024-05-21T21:54:59.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:59.525+0000 7f4db6000700 1 --2- 172.21.15.151:0/177074412 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f4da00b2880 0x7f4da00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:54:59.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:59.525+0000 7f4db6000700 1 -- 172.21.15.151:0/177074412 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f4dac0e1b20 con 0x7f4db8105760 2024-05-21T21:54:59.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:59.525+0000 7f4dbd200700 1 --2- 172.21.15.151:0/177074412 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f4da00b2880 0x7f4da00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:54:59.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:59.526+0000 7f4dbd200700 1 --2- 172.21.15.151:0/177074412 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f4da00b2880 0x7f4da00b4d40 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4da4009750 tx=0x7f4da400b280 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:54:59.529 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:59.529+0000 7f4db6000700 1 -- 172.21.15.151:0/177074412 <== mon.2 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4dac010070 con 0x7f4db8105760 2024-05-21T21:54:59.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:59.814+0000 7f4dc0600700 1 -- 172.21.15.151:0/177074412 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4db8078f80 con 0x7f4db8105760 2024-05-21T21:54:59.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:59.815+0000 7f4db6000700 1 -- 172.21.15.151:0/177074412 <== mon.2 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11958 (secure 0 0 0) 0x7f4dac0a9800 con 0x7f4db8105760 2024-05-21T21:54:59.816 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T21:54:59.816 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14358,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/158010529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":158010529},{"type":"v1","addr":"172.21.15.151:6835","nonce":158010529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6839/2069058115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":2069058115},{"type":"v1","addr":"172.21.15.151:6839","nonce":2069058115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T21:51:19.552069+0000","modified":"2024-05-21T21:51:43.002737+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24530,"mds_1":24493,"mds_2":24491,"mds_3":24478,"mds_4":24473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14403":{"gid":14403,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6841/1749723794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6840","nonce":1749723794},{"type":"v1","addr":"172.21.15.151:6841","nonce":1749723794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/2026266114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":2026266114},{"type":"v1","addr":"172.21.15.184:6833","nonce":2026266114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/2467900844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":2467900844},{"type":"v1","addr":"172.21.15.164:6833","nonce":2467900844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3367640960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3367640960},{"type":"v1","addr":"172.21.15.151:6837","nonce":3367640960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/3264142621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3264142621},{"type":"v1","addr":"172.21.15.184:6835","nonce":3264142621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":17,"addr":"172.21.15.164:6835/2340428470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2340428470},{"type":"v1","addr":"172.21.15.164:6835","nonce":2340428470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":16,"addr":"172.21.15.184:6837/2988194888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":2988194888},{"type":"v1","addr":"172.21.15.184:6837","nonce":2988194888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.164:6837/1554609572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1554609572},{"type":"v1","addr":"172.21.15.164:6837","nonce":1554609572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.184:6839/3059935143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":3059935143},{"type":"v1","addr":"172.21.15.184:6839","nonce":3059935143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.164:6839/2646401247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":2646401247},{"type":"v1","addr":"172.21.15.164:6839","nonce":2646401247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24530,"qdb_cluster":[24473,24478,24491,24493,24530]},"id":1}]} 2024-05-21T21:54:59.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:59.818+0000 7f4dc0600700 1 -- 172.21.15.151:0/177074412 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f4da00b2880 msgr2=0x7f4da00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:59.819 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:59.818+0000 7f4dc0600700 1 --2- 172.21.15.151:0/177074412 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f4da00b2880 0x7f4da00b4d40 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4da4009750 tx=0x7f4da400b280 comp rx=0 tx=0).stop 2024-05-21T21:54:59.819 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:59.818+0000 7f4dc0600700 1 -- 172.21.15.151:0/177074412 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4db8105760 msgr2=0x7f4db8077a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:54:59.819 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:59.818+0000 7f4dc0600700 1 --2- 172.21.15.151:0/177074412 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4db8105760 0x7f4db8077a50 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4dac007ef0 tx=0x7f4dac00d3b0 comp rx=0 tx=0).stop 2024-05-21T21:54:59.819 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:59.818+0000 7f4dc0600700 1 -- 172.21.15.151:0/177074412 shutdown_connections 2024-05-21T21:54:59.819 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:59.818+0000 7f4dc0600700 1 --2- 172.21.15.151:0/177074412 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f4da00b2880 0x7f4da00b4d40 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:59.819 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:59.818+0000 7f4dc0600700 1 --2- 172.21.15.151:0/177074412 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4db8105760 0x7f4db8077a50 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:59.819 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:59.818+0000 7f4dc0600700 1 --2- 172.21.15.151:0/177074412 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4db810b760 0x7f4db80784d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:59.819 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:59.818+0000 7f4dc0600700 1 --2- 172.21.15.151:0/177074412 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4db8106120 0x7f4db8077f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:54:59.819 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:59.818+0000 7f4dc0600700 1 -- 172.21.15.151:0/177074412 >> 172.21.15.151:0/177074412 conn(0x7f4db8101250 msgr2=0x7f4db8101980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:54:59.820 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:59.819+0000 7f4dc0600700 1 -- 172.21.15.151:0/177074412 shutdown_connections 2024-05-21T21:54:59.820 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:54:59.819+0000 7f4dc0600700 1 -- 172.21.15.151:0/177074412 wait complete. 2024-05-21T21:54:59.820 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 28 2024-05-21T21:55:00.543 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:00 smithi151 ceph-mon[31623]: pgmap v427: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-21T21:55:00.543 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:00 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/177074412' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T21:55:00.594 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-21T21:55:00.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:00 smithi184 ceph-mon[36942]: pgmap v427: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-21T21:55:00.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:00 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/177074412' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T21:55:00.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:00 smithi164 ceph-mon[36269]: pgmap v427: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-21T21:55:00.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:00 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/177074412' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T21:55:01.035 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:55:02.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:02 smithi151 ceph-mon[31623]: pgmap v428: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-21T21:55:02.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:02 smithi184 ceph-mon[36942]: pgmap v428: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-21T21:55:02.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:02 smithi164 ceph-mon[36269]: pgmap v428: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-21T21:55:03.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:03.162+0000 7fa45c400700 1 -- 172.21.15.151:0/3209803419 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa4541060f0 msgr2=0x7fa454112600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:03.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:03.162+0000 7fa45c400700 1 --2- 172.21.15.151:0/3209803419 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa4541060f0 0x7fa454112600 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa44c00b0a0 tx=0x7fa44c00b3b0 comp rx=0 tx=0).stop 2024-05-21T21:55:03.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:03.162+0000 7fa45c400700 1 -- 172.21.15.151:0/3209803419 shutdown_connections 2024-05-21T21:55:03.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:03.162+0000 7fa45c400700 1 --2- 172.21.15.151:0/3209803419 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa4541060f0 0x7fa454112600 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:03.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:03.162+0000 7fa45c400700 1 --2- 172.21.15.151:0/3209803419 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa45410b730 0x7fa45410bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:03.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:03.162+0000 7fa45c400700 1 --2- 172.21.15.151:0/3209803419 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa454105730 0x7fa454105bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:03.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:03.162+0000 7fa45c400700 1 -- 172.21.15.151:0/3209803419 >> 172.21.15.151:0/3209803419 conn(0x7fa454101220 msgr2=0x7fa454103640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:55:03.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:03.163+0000 7fa45c400700 1 -- 172.21.15.151:0/3209803419 shutdown_connections 2024-05-21T21:55:03.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:03.163+0000 7fa45c400700 1 -- 172.21.15.151:0/3209803419 wait complete. 2024-05-21T21:55:03.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:03.164+0000 7fa45c400700 1 Processor -- start 2024-05-21T21:55:03.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:03.164+0000 7fa45c400700 1 -- start start 2024-05-21T21:55:03.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:03.164+0000 7fa45c400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa454105730 0x7fa45419bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:03.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:03.164+0000 7fa45c400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa4541060f0 0x7fa45419c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:03.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:03.164+0000 7fa45c400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa45410b730 0x7fa4541a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:03.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:03.164+0000 7fa45c400700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa4541159c0 con 0x7fa454105730 2024-05-21T21:55:03.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:03.164+0000 7fa45c400700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa454115840 con 0x7fa4541060f0 2024-05-21T21:55:03.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:03.164+0000 7fa45c400700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa454115b40 con 0x7fa45410b730 2024-05-21T21:55:03.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:03.165+0000 7fa459a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa454105730 0x7fa45419bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:03.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:03.165+0000 7fa459a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa454105730 0x7fa45419bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:43046/0 (socket says 172.21.15.151:43046) 2024-05-21T21:55:03.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:03.165+0000 7fa459a00700 1 -- 172.21.15.151:0/659228063 learned_addr learned my addr 172.21.15.151:0/659228063 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:55:03.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:03.165+0000 7fa45a400700 1 --2- 172.21.15.151:0/659228063 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa45410b730 0x7fa4541a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:03.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:03.165+0000 7fa459000700 1 --2- 172.21.15.151:0/659228063 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa4541060f0 0x7fa45419c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:03.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:03.165+0000 7fa45a400700 1 -- 172.21.15.151:0/659228063 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa4541060f0 msgr2=0x7fa45419c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:03.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:03.165+0000 7fa45a400700 1 --2- 172.21.15.151:0/659228063 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa4541060f0 0x7fa45419c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:03.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:03.165+0000 7fa45a400700 1 -- 172.21.15.151:0/659228063 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa454105730 msgr2=0x7fa45419bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:03.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:03.165+0000 7fa45a400700 1 --2- 172.21.15.151:0/659228063 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa454105730 0x7fa45419bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:03.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:03.165+0000 7fa45a400700 1 -- 172.21.15.151:0/659228063 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4541a0e00 con 0x7fa45410b730 2024-05-21T21:55:03.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:03.165+0000 7fa45a400700 1 --2- 172.21.15.151:0/659228063 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa45410b730 0x7fa4541a06c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa44c00b070 tx=0x7fa44c004c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:55:03.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:03.166+0000 7fa452000700 1 -- 172.21.15.151:0/659228063 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa44c021070 con 0x7fa45410b730 2024-05-21T21:55:03.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:03.166+0000 7fa452000700 1 -- 172.21.15.151:0/659228063 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa44c007e30 con 0x7fa45410b730 2024-05-21T21:55:03.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:03.166+0000 7fa45c400700 1 -- 172.21.15.151:0/659228063 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4541a1090 con 0x7fa45410b730 2024-05-21T21:55:03.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:03.166+0000 7fa452000700 1 -- 172.21.15.151:0/659228063 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa44c0045f0 con 0x7fa45410b730 2024-05-21T21:55:03.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:03.166+0000 7fa45c400700 1 -- 172.21.15.151:0/659228063 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4541a8970 con 0x7fa45410b730 2024-05-21T21:55:03.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:03.167+0000 7fa45c400700 1 -- 172.21.15.151:0/659228063 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa45406a230 con 0x7fa45410b730 2024-05-21T21:55:03.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:03.170+0000 7fa452000700 1 -- 172.21.15.151:0/659228063 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fa44c007980 con 0x7fa45410b730 2024-05-21T21:55:03.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:03.171+0000 7fa452000700 1 --2- 172.21.15.151:0/659228063 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fa43c0b2900 0x7fa43c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:03.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:03.171+0000 7fa452000700 1 -- 172.21.15.151:0/659228063 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fa44c00f020 con 0x7fa45410b730 2024-05-21T21:55:03.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:03.171+0000 7fa459a00700 1 --2- 172.21.15.151:0/659228063 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fa43c0b2900 0x7fa43c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:03.172 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:03.172+0000 7fa459a00700 1 --2- 172.21.15.151:0/659228063 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fa43c0b2900 0x7fa43c0b4dc0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa448009ca0 tx=0x7fa448009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:55:03.173 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:03.173+0000 7fa452000700 1 -- 172.21.15.151:0/659228063 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa44c02a020 con 0x7fa45410b730 2024-05-21T21:55:03.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:03.452+0000 7fa45c400700 1 -- 172.21.15.151:0/659228063 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa45419d150 con 0x7fa45410b730 2024-05-21T21:55:03.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:03.459+0000 7fa452000700 1 -- 172.21.15.151:0/659228063 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11958 (secure 0 0 0) 0x7fa44c02e3e0 con 0x7fa45410b730 2024-05-21T21:55:03.460 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T21:55:03.460 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14358,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/158010529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":158010529},{"type":"v1","addr":"172.21.15.151:6835","nonce":158010529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6839/2069058115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":2069058115},{"type":"v1","addr":"172.21.15.151:6839","nonce":2069058115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T21:51:19.552069+0000","modified":"2024-05-21T21:51:43.002737+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24530,"mds_1":24493,"mds_2":24491,"mds_3":24478,"mds_4":24473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14403":{"gid":14403,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6841/1749723794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6840","nonce":1749723794},{"type":"v1","addr":"172.21.15.151:6841","nonce":1749723794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/2026266114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":2026266114},{"type":"v1","addr":"172.21.15.184:6833","nonce":2026266114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/2467900844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":2467900844},{"type":"v1","addr":"172.21.15.164:6833","nonce":2467900844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3367640960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3367640960},{"type":"v1","addr":"172.21.15.151:6837","nonce":3367640960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/3264142621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3264142621},{"type":"v1","addr":"172.21.15.184:6835","nonce":3264142621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":17,"addr":"172.21.15.164:6835/2340428470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2340428470},{"type":"v1","addr":"172.21.15.164:6835","nonce":2340428470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":16,"addr":"172.21.15.184:6837/2988194888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":2988194888},{"type":"v1","addr":"172.21.15.184:6837","nonce":2988194888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.164:6837/1554609572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1554609572},{"type":"v1","addr":"172.21.15.164:6837","nonce":1554609572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.184:6839/3059935143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":3059935143},{"type":"v1","addr":"172.21.15.184:6839","nonce":3059935143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.164:6839/2646401247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":2646401247},{"type":"v1","addr":"172.21.15.164:6839","nonce":2646401247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24530,"qdb_cluster":[24473,24478,24491,24493,24530]},"id":1}]} 2024-05-21T21:55:03.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:03.463+0000 7fa45c400700 1 -- 172.21.15.151:0/659228063 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fa43c0b2900 msgr2=0x7fa43c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:03.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:03.463+0000 7fa45c400700 1 --2- 172.21.15.151:0/659228063 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fa43c0b2900 0x7fa43c0b4dc0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa448009ca0 tx=0x7fa448009260 comp rx=0 tx=0).stop 2024-05-21T21:55:03.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:03.463+0000 7fa45c400700 1 -- 172.21.15.151:0/659228063 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa45410b730 msgr2=0x7fa4541a06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:03.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:03.463+0000 7fa45c400700 1 --2- 172.21.15.151:0/659228063 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa45410b730 0x7fa4541a06c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa44c00b070 tx=0x7fa44c004c80 comp rx=0 tx=0).stop 2024-05-21T21:55:03.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:03.464+0000 7fa45c400700 1 -- 172.21.15.151:0/659228063 shutdown_connections 2024-05-21T21:55:03.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:03.464+0000 7fa45c400700 1 --2- 172.21.15.151:0/659228063 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fa43c0b2900 0x7fa43c0b4dc0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:03.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:03.464+0000 7fa45c400700 1 --2- 172.21.15.151:0/659228063 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa45410b730 0x7fa4541a06c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:03.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:03.464+0000 7fa45c400700 1 --2- 172.21.15.151:0/659228063 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa454105730 0x7fa45419bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:03.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:03.464+0000 7fa45c400700 1 --2- 172.21.15.151:0/659228063 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa4541060f0 0x7fa45419c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:03.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:03.464+0000 7fa45c400700 1 -- 172.21.15.151:0/659228063 >> 172.21.15.151:0/659228063 conn(0x7fa454101220 msgr2=0x7fa454101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:55:03.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:03.464+0000 7fa45c400700 1 -- 172.21.15.151:0/659228063 shutdown_connections 2024-05-21T21:55:03.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:03.464+0000 7fa45c400700 1 -- 172.21.15.151:0/659228063 wait complete. 2024-05-21T21:55:03.466 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 28 2024-05-21T21:55:04.252 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph osd dump --format=json 2024-05-21T21:55:04.710 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:55:04.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:04 smithi184 ceph-mon[36942]: pgmap v429: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-21T21:55:04.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:04 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/659228063' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T21:55:04.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:04 smithi164 ceph-mon[36269]: pgmap v429: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-21T21:55:04.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:04 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/659228063' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T21:55:04.859 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:04 smithi151 ceph-mon[31623]: pgmap v429: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-21T21:55:04.859 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:04 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/659228063' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T21:55:06.718 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:06 smithi151 ceph-mon[31623]: pgmap v430: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-21T21:55:06.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:06 smithi184 ceph-mon[36942]: pgmap v430: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-21T21:55:06.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:06 smithi164 ceph-mon[36269]: pgmap v430: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-21T21:55:07.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:07.009+0000 7fb32b200700 1 -- 172.21.15.151:0/27604057 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb32c106030 msgr2=0x7fb32c112540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:07.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:07.009+0000 7fb32b200700 1 --2- 172.21.15.151:0/27604057 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb32c106030 0x7fb32c112540 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb324009f90 tx=0x7fb32400b330 comp rx=0 tx=0).stop 2024-05-21T21:55:07.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:07.009+0000 7fb32b200700 1 -- 172.21.15.151:0/27604057 shutdown_connections 2024-05-21T21:55:07.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:07.009+0000 7fb32b200700 1 --2- 172.21.15.151:0/27604057 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb32c10b670 0x7fb32c10ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:07.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:07.009+0000 7fb32b200700 1 --2- 172.21.15.151:0/27604057 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb32c106030 0x7fb32c112540 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:07.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:07.009+0000 7fb32b200700 1 --2- 172.21.15.151:0/27604057 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb32c105670 0x7fb32c105af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:07.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:07.009+0000 7fb32b200700 1 -- 172.21.15.151:0/27604057 >> 172.21.15.151:0/27604057 conn(0x7fb32c101160 msgr2=0x7fb32c103580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:55:07.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:07.009+0000 7fb32b200700 1 -- 172.21.15.151:0/27604057 shutdown_connections 2024-05-21T21:55:07.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:07.009+0000 7fb32b200700 1 -- 172.21.15.151:0/27604057 wait complete. 2024-05-21T21:55:07.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:07.011+0000 7fb32b200700 1 Processor -- start 2024-05-21T21:55:07.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:07.011+0000 7fb32b200700 1 -- start start 2024-05-21T21:55:07.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:07.011+0000 7fb32b200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb32c105670 0x7fb32c1a18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:07.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:07.011+0000 7fb32b200700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb32c106030 0x7fb32c1a1e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:07.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:07.011+0000 7fb32b200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb32c10b670 0x7fb32c19bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:07.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:07.011+0000 7fb32b200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb32c1158b0 con 0x7fb32c10b670 2024-05-21T21:55:07.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:07.011+0000 7fb32b200700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb32c115730 con 0x7fb32c105670 2024-05-21T21:55:07.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:07.011+0000 7fb32b200700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb32c115a30 con 0x7fb32c106030 2024-05-21T21:55:07.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:07.012+0000 7fb323e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb32c106030 0x7fb32c1a1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:07.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:07.012+0000 7fb329e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb32c105670 0x7fb32c1a18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:07.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:07.012+0000 7fb32a800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb32c10b670 0x7fb32c19bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:07.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:07.012+0000 7fb323e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb32c106030 0x7fb32c1a1e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:39750/0 (socket says 172.21.15.151:39750) 2024-05-21T21:55:07.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:07.012+0000 7fb323e00700 1 -- 172.21.15.151:0/3924143781 learned_addr learned my addr 172.21.15.151:0/3924143781 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:55:07.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:07.012+0000 7fb329e00700 1 -- 172.21.15.151:0/3924143781 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb32c106030 msgr2=0x7fb32c1a1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:07.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:07.012+0000 7fb329e00700 1 --2- 172.21.15.151:0/3924143781 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb32c106030 0x7fb32c1a1e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:07.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:07.012+0000 7fb329e00700 1 -- 172.21.15.151:0/3924143781 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb32c10b670 msgr2=0x7fb32c19bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:07.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:07.012+0000 7fb329e00700 1 --2- 172.21.15.151:0/3924143781 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb32c10b670 0x7fb32c19bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:07.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:07.012+0000 7fb329e00700 1 -- 172.21.15.151:0/3924143781 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb32c19c230 con 0x7fb32c105670 2024-05-21T21:55:07.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:07.012+0000 7fb323e00700 1 --2- 172.21.15.151:0/3924143781 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb32c106030 0x7fb32c1a1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:55:07.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:07.012+0000 7fb32a800700 1 --2- 172.21.15.151:0/3924143781 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb32c10b670 0x7fb32c19bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:55:07.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:07.012+0000 7fb329e00700 1 --2- 172.21.15.151:0/3924143781 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb32c105670 0x7fb32c1a18c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb31c00e9c0 tx=0x7fb31c00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:55:07.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:07.013+0000 7fb322a00700 1 -- 172.21.15.151:0/3924143781 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb31c00c9c0 con 0x7fb32c105670 2024-05-21T21:55:07.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:07.013+0000 7fb322a00700 1 -- 172.21.15.151:0/3924143781 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb31c00cb60 con 0x7fb32c105670 2024-05-21T21:55:07.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:07.013+0000 7fb32b200700 1 -- 172.21.15.151:0/3924143781 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb32c19c520 con 0x7fb32c105670 2024-05-21T21:55:07.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:07.013+0000 7fb322a00700 1 -- 172.21.15.151:0/3924143781 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb31c018420 con 0x7fb32c105670 2024-05-21T21:55:07.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:07.013+0000 7fb32b200700 1 -- 172.21.15.151:0/3924143781 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb32c1a8790 con 0x7fb32c105670 2024-05-21T21:55:07.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:07.014+0000 7fb32b200700 1 -- 172.21.15.151:0/3924143781 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb32c06a230 con 0x7fb32c105670 2024-05-21T21:55:07.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:07.016+0000 7fb322a00700 1 -- 172.21.15.151:0/3924143781 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fb31c010430 con 0x7fb32c105670 2024-05-21T21:55:07.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:07.018+0000 7fb322a00700 1 --2- 172.21.15.151:0/3924143781 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fb3100b2880 0x7fb3100b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:07.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:07.018+0000 7fb322a00700 1 -- 172.21.15.151:0/3924143781 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fb31c0e27f0 con 0x7fb32c105670 2024-05-21T21:55:07.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:07.018+0000 7fb323e00700 1 --2- 172.21.15.151:0/3924143781 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fb3100b2880 0x7fb3100b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:07.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:07.019+0000 7fb323e00700 1 --2- 172.21.15.151:0/3924143781 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fb3100b2880 0x7fb3100b4d40 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb32c10ac70 tx=0x7fb314009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:55:07.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:07.020+0000 7fb322a00700 1 -- 172.21.15.151:0/3924143781 <== mon.2 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb31c0aa4d0 con 0x7fb32c105670 2024-05-21T21:55:07.253 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:07.253+0000 7fb32b200700 1 -- 172.21.15.151:0/3924143781 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb32c02e560 con 0x7fb32c105670 2024-05-21T21:55:07.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:07.254+0000 7fb322a00700 1 -- 172.21.15.151:0/3924143781 <== mon.2 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22744 (secure 0 0 0) 0x7fb31c0a9dd0 con 0x7fb32c105670 2024-05-21T21:55:07.255 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T21:55:07.256 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":82,"fsid":"c8c7a2e6-17ba-11ef-bc9a-c7b262605968","created":"2024-05-21T21:41:52.412833+0000","modified":"2024-05-21T21:52:10.856587+0000","last_up_change":"2024-05-21T21:50:05.401314+0000","last_in_change":"2024-05-21T21:49:45.684136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-21T21:45:55.196046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-21T21:51:11.447428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-21T21:51:15.618605+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"18db8e54-0cfc-4e12-9c84-6f72a99cc989","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":2420400473},{"type":"v1","addr":"172.21.15.151:6803","nonce":2420400473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":2420400473},{"type":"v1","addr":"172.21.15.151:6805","nonce":2420400473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":2420400473},{"type":"v1","addr":"172.21.15.151:6809","nonce":2420400473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":2420400473},{"type":"v1","addr":"172.21.15.151:6807","nonce":2420400473}]},"public_addr":"172.21.15.151:6803/2420400473","cluster_addr":"172.21.15.151:6805/2420400473","heartbeat_back_addr":"172.21.15.151:6809/2420400473","heartbeat_front_addr":"172.21.15.151:6807/2420400473","state":["exists","up"]},{"osd":1,"uuid":"bac384c7-1102-4815-81a8-10cc7661d230","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":3900240447},{"type":"v1","addr":"172.21.15.164:6801","nonce":3900240447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":3900240447},{"type":"v1","addr":"172.21.15.164:6803","nonce":3900240447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":3900240447},{"type":"v1","addr":"172.21.15.164:6807","nonce":3900240447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":3900240447},{"type":"v1","addr":"172.21.15.164:6805","nonce":3900240447}]},"public_addr":"172.21.15.164:6801/3900240447","cluster_addr":"172.21.15.164:6803/3900240447","heartbeat_back_addr":"172.21.15.164:6807/3900240447","heartbeat_front_addr":"172.21.15.164:6805/3900240447","state":["exists","up"]},{"osd":2,"uuid":"b7e4a782-1518-4d43-bee9-7fd768ee5cb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":3971150737},{"type":"v1","addr":"172.21.15.184:6801","nonce":3971150737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3971150737},{"type":"v1","addr":"172.21.15.184:6803","nonce":3971150737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3971150737},{"type":"v1","addr":"172.21.15.184:6807","nonce":3971150737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3971150737},{"type":"v1","addr":"172.21.15.184:6805","nonce":3971150737}]},"public_addr":"172.21.15.184:6801/3971150737","cluster_addr":"172.21.15.184:6803/3971150737","heartbeat_back_addr":"172.21.15.184:6807/3971150737","heartbeat_front_addr":"172.21.15.184:6805/3971150737","state":["exists","up"]},{"osd":3,"uuid":"ab3df826-39e3-4635-a49c-2b5ffdbf12f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":556594400},{"type":"v1","addr":"172.21.15.151:6811","nonce":556594400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":556594400},{"type":"v1","addr":"172.21.15.151:6813","nonce":556594400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":556594400},{"type":"v1","addr":"172.21.15.151:6817","nonce":556594400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":556594400},{"type":"v1","addr":"172.21.15.151:6815","nonce":556594400}]},"public_addr":"172.21.15.151:6811/556594400","cluster_addr":"172.21.15.151:6813/556594400","heartbeat_back_addr":"172.21.15.151:6817/556594400","heartbeat_front_addr":"172.21.15.151:6815/556594400","state":["exists","up"]},{"osd":4,"uuid":"00cbf6ae-0f42-4685-a77d-d803c6467aa0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":988881973},{"type":"v1","addr":"172.21.15.164:6809","nonce":988881973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":988881973},{"type":"v1","addr":"172.21.15.164:6811","nonce":988881973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":988881973},{"type":"v1","addr":"172.21.15.164:6815","nonce":988881973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":988881973},{"type":"v1","addr":"172.21.15.164:6813","nonce":988881973}]},"public_addr":"172.21.15.164:6809/988881973","cluster_addr":"172.21.15.164:6811/988881973","heartbeat_back_addr":"172.21.15.164:6815/988881973","heartbeat_front_addr":"172.21.15.164:6813/988881973","state":["exists","up"]},{"osd":5,"uuid":"fad14221-0ef0-4d40-b414-c6c9f1f9718c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3293311106},{"type":"v1","addr":"172.21.15.184:6809","nonce":3293311106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":3293311106},{"type":"v1","addr":"172.21.15.184:6811","nonce":3293311106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":3293311106},{"type":"v1","addr":"172.21.15.184:6815","nonce":3293311106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":3293311106},{"type":"v1","addr":"172.21.15.184:6813","nonce":3293311106}]},"public_addr":"172.21.15.184:6809/3293311106","cluster_addr":"172.21.15.184:6811/3293311106","heartbeat_back_addr":"172.21.15.184:6815/3293311106","heartbeat_front_addr":"172.21.15.184:6813/3293311106","state":["exists","up"]},{"osd":6,"uuid":"ab8a60f6-0dec-4c1c-89db-f2e2f638f1ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":3377550762},{"type":"v1","addr":"172.21.15.151:6819","nonce":3377550762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":3377550762},{"type":"v1","addr":"172.21.15.151:6821","nonce":3377550762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":3377550762},{"type":"v1","addr":"172.21.15.151:6825","nonce":3377550762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":3377550762},{"type":"v1","addr":"172.21.15.151:6823","nonce":3377550762}]},"public_addr":"172.21.15.151:6819/3377550762","cluster_addr":"172.21.15.151:6821/3377550762","heartbeat_back_addr":"172.21.15.151:6825/3377550762","heartbeat_front_addr":"172.21.15.151:6823/3377550762","state":["exists","up"]},{"osd":7,"uuid":"db332268-9972-4ed7-a307-822b83a0fbb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":3869696936},{"type":"v1","addr":"172.21.15.164:6817","nonce":3869696936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":3869696936},{"type":"v1","addr":"172.21.15.164:6819","nonce":3869696936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":3869696936},{"type":"v1","addr":"172.21.15.164:6823","nonce":3869696936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":3869696936},{"type":"v1","addr":"172.21.15.164:6821","nonce":3869696936}]},"public_addr":"172.21.15.164:6817/3869696936","cluster_addr":"172.21.15.164:6819/3869696936","heartbeat_back_addr":"172.21.15.164:6823/3869696936","heartbeat_front_addr":"172.21.15.164:6821/3869696936","state":["exists","up"]},{"osd":8,"uuid":"fe2fbe1c-549a-4947-a188-426dfca04d10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":575273541},{"type":"v1","addr":"172.21.15.184:6817","nonce":575273541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":575273541},{"type":"v1","addr":"172.21.15.184:6819","nonce":575273541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":575273541},{"type":"v1","addr":"172.21.15.184:6823","nonce":575273541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":575273541},{"type":"v1","addr":"172.21.15.184:6821","nonce":575273541}]},"public_addr":"172.21.15.184:6817/575273541","cluster_addr":"172.21.15.184:6819/575273541","heartbeat_back_addr":"172.21.15.184:6823/575273541","heartbeat_front_addr":"172.21.15.184:6821/575273541","state":["exists","up"]},{"osd":9,"uuid":"3a7b450f-8004-4adf-bae3-970918f98f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":337128533},{"type":"v1","addr":"172.21.15.151:6827","nonce":337128533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":337128533},{"type":"v1","addr":"172.21.15.151:6829","nonce":337128533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":337128533},{"type":"v1","addr":"172.21.15.151:6833","nonce":337128533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":337128533},{"type":"v1","addr":"172.21.15.151:6831","nonce":337128533}]},"public_addr":"172.21.15.151:6827/337128533","cluster_addr":"172.21.15.151:6829/337128533","heartbeat_back_addr":"172.21.15.151:6833/337128533","heartbeat_front_addr":"172.21.15.151:6831/337128533","state":["exists","up"]},{"osd":10,"uuid":"8fcdf271-90c4-4db3-be69-b0ca716dfd80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":1823448782},{"type":"v1","addr":"172.21.15.164:6825","nonce":1823448782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":1823448782},{"type":"v1","addr":"172.21.15.164:6827","nonce":1823448782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":1823448782},{"type":"v1","addr":"172.21.15.164:6831","nonce":1823448782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":1823448782},{"type":"v1","addr":"172.21.15.164:6829","nonce":1823448782}]},"public_addr":"172.21.15.164:6825/1823448782","cluster_addr":"172.21.15.164:6827/1823448782","heartbeat_back_addr":"172.21.15.164:6831/1823448782","heartbeat_front_addr":"172.21.15.164:6829/1823448782","state":["exists","up"]},{"osd":11,"uuid":"e661477b-0c09-409f-a2fe-6ebd34642a8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":1563340128},{"type":"v1","addr":"172.21.15.184:6825","nonce":1563340128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":1563340128},{"type":"v1","addr":"172.21.15.184:6827","nonce":1563340128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":1563340128},{"type":"v1","addr":"172.21.15.184:6831","nonce":1563340128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":1563340128},{"type":"v1","addr":"172.21.15.184:6829","nonce":1563340128}]},"public_addr":"172.21.15.184:6825/1563340128","cluster_addr":"172.21.15.184:6827/1563340128","heartbeat_back_addr":"172.21.15.184:6831/1563340128","heartbeat_front_addr":"172.21.15.184:6829/1563340128","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:45:02.821160+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:45:27.844564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:45:51.904981+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:46:24.376245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:46:50.036621+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:47:14.748422+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:47:47.396383+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:48:13.543174+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:48:38.213285+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:49:11.980042+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:49:38.519761+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:50:03.439898+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.151:0/31963128":"2024-05-22T21:43:09.041455+0000","172.21.15.151:6800/3428827300":"2024-05-22T21:43:09.041455+0000","172.21.15.151:6801/3428827300":"2024-05-22T21:43:09.041455+0000","172.21.15.151:0/131989141":"2024-05-22T21:43:09.041455+0000","172.21.15.151:6801/1128885292":"2024-05-22T21:42:35.440147+0000","172.21.15.151:0/2703664519":"2024-05-22T21:43:09.041455+0000","172.21.15.151:6800/1128885292":"2024-05-22T21:42:35.440147+0000","172.21.15.151:0/3052943699":"2024-05-22T21:42:35.440147+0000","172.21.15.151:0/3929452511":"2024-05-22T21:42:35.440147+0000","172.21.15.151:0/3762889700":"2024-05-22T21:42:35.440147+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-21T21:55:07.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:07.257+0000 7fb32b200700 1 -- 172.21.15.151:0/3924143781 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fb3100b2880 msgr2=0x7fb3100b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:07.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:07.257+0000 7fb32b200700 1 --2- 172.21.15.151:0/3924143781 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fb3100b2880 0x7fb3100b4d40 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb32c10ac70 tx=0x7fb314009260 comp rx=0 tx=0).stop 2024-05-21T21:55:07.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:07.257+0000 7fb32b200700 1 -- 172.21.15.151:0/3924143781 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb32c105670 msgr2=0x7fb32c1a18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:07.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:07.257+0000 7fb32b200700 1 --2- 172.21.15.151:0/3924143781 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb32c105670 0x7fb32c1a18c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb31c00e9c0 tx=0x7fb31c00ecd0 comp rx=0 tx=0).stop 2024-05-21T21:55:07.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:07.258+0000 7fb32b200700 1 -- 172.21.15.151:0/3924143781 shutdown_connections 2024-05-21T21:55:07.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:07.258+0000 7fb32b200700 1 --2- 172.21.15.151:0/3924143781 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fb3100b2880 0x7fb3100b4d40 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:07.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:07.258+0000 7fb32b200700 1 --2- 172.21.15.151:0/3924143781 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb32c105670 0x7fb32c1a18c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:07.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:07.258+0000 7fb32b200700 1 --2- 172.21.15.151:0/3924143781 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb32c10b670 0x7fb32c19bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:07.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:07.258+0000 7fb32b200700 1 --2- 172.21.15.151:0/3924143781 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb32c106030 0x7fb32c1a1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:07.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:07.258+0000 7fb32b200700 1 -- 172.21.15.151:0/3924143781 >> 172.21.15.151:0/3924143781 conn(0x7fb32c101160 msgr2=0x7fb32c101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:55:07.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:07.258+0000 7fb32b200700 1 -- 172.21.15.151:0/3924143781 shutdown_connections 2024-05-21T21:55:07.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:07.258+0000 7fb32b200700 1 -- 172.21.15.151:0/3924143781 wait complete. 2024-05-21T21:55:07.702 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:07 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/3924143781' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T21:55:07.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:07 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/3924143781' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T21:55:07.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:07 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/3924143781' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T21:55:08.172 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-21T21:51:19.552069+0000', 'modified': '2024-05-21T21:51:43.002737+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24530, 'mds_1': 24493, 'mds_2': 24491, 'mds_3': 24478, 'mds_4': 24473}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14403': {'gid': 14403, 'name': 'a', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.151:6841/1749723794', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.151:6840', 'nonce': 1749723794}, {'type': 'v1', 'addr': '172.21.15.151:6841', 'nonce': 1749723794}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24436': {'gid': 24436, 'name': 'l', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.184:6833/2026266114', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6832', 'nonce': 2026266114}, {'type': 'v1', 'addr': '172.21.15.184:6833', 'nonce': 2026266114}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24443': {'gid': 24443, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.164:6833/2467900844', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6832', 'nonce': 2467900844}, {'type': 'v1', 'addr': '172.21.15.164:6833', 'nonce': 2467900844}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24461': {'gid': 24461, 'name': 'd', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.151:6837/3367640960', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.151:6836', 'nonce': 3367640960}, {'type': 'v1', 'addr': '172.21.15.151:6837', 'nonce': 3367640960}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24466': {'gid': 24466, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.184:6835/3264142621', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6834', 'nonce': 3264142621}, {'type': 'v1', 'addr': '172.21.15.184:6835', 'nonce': 3264142621}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24473': {'gid': 24473, 'name': 'e', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.164:6835/2340428470', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6834', 'nonce': 2340428470}, {'type': 'v1', 'addr': '172.21.15.164:6835', 'nonce': 2340428470}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24478': {'gid': 24478, 'name': 'c', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 16, 'addr': '172.21.15.184:6837/2988194888', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6836', 'nonce': 2988194888}, {'type': 'v1', 'addr': '172.21.15.184:6837', 'nonce': 2988194888}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24491': {'gid': 24491, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.164:6837/1554609572', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6836', 'nonce': 1554609572}, {'type': 'v1', 'addr': '172.21.15.164:6837', 'nonce': 1554609572}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24493': {'gid': 24493, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.184:6839/3059935143', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6838', 'nonce': 3059935143}, {'type': 'v1', 'addr': '172.21.15.184:6839', 'nonce': 3059935143}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24530': {'gid': 24530, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.164:6839/2646401247', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6838', 'nonce': 2646401247}, {'type': 'v1', 'addr': '172.21.15.164:6839', 'nonce': 2646401247}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24530, 'qdb_cluster': [24473, 24478, 24491, 24493, 24530]} 2024-05-21T21:55:08.172 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-05-21T21:55:08.172 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.e status 2024-05-21T21:55:08.613 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:55:08.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:08 smithi184 ceph-mon[36942]: pgmap v431: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-21T21:55:08.760 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:08 smithi151 ceph-mon[31623]: pgmap v431: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-21T21:55:08.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:08 smithi164 ceph-mon[36269]: pgmap v431: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-21T21:55:10.587 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:10 smithi151 ceph-mon[31623]: pgmap v432: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-21T21:55:10.735 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:10 smithi184 ceph-mon[36942]: pgmap v432: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-21T21:55:10.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:10 smithi164 ceph-mon[36269]: pgmap v432: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-21T21:55:10.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.851+0000 7eff0ba00700 1 -- 172.21.15.151:0/1905531307 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff0406b220 msgr2=0x7eff0406b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:10.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.851+0000 7eff0ba00700 1 --2- 172.21.15.151:0/1905531307 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff0406b220 0x7eff0406b6a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7efeec009990 tx=0x7efeec009ca0 comp rx=0 tx=0).stop 2024-05-21T21:55:10.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.851+0000 7eff0ba00700 1 -- 172.21.15.151:0/1905531307 shutdown_connections 2024-05-21T21:55:10.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.851+0000 7eff0ba00700 1 --2- 172.21.15.151:0/1905531307 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff0406bbe0 0x7eff041103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:10.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.852+0000 7eff0ba00700 1 --2- 172.21.15.151:0/1905531307 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff041095b0 0x7eff04109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:10.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.852+0000 7eff0ba00700 1 --2- 172.21.15.151:0/1905531307 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff0406b220 0x7eff0406b6a0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:10.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.852+0000 7eff0ba00700 1 -- 172.21.15.151:0/1905531307 >> 172.21.15.151:0/1905531307 conn(0x7eff040793f0 msgr2=0x7eff04079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:55:10.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.852+0000 7eff0ba00700 1 -- 172.21.15.151:0/1905531307 shutdown_connections 2024-05-21T21:55:10.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.852+0000 7eff0ba00700 1 -- 172.21.15.151:0/1905531307 wait complete. 2024-05-21T21:55:10.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.853+0000 7eff0ba00700 1 Processor -- start 2024-05-21T21:55:10.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.853+0000 7eff0ba00700 1 -- start start 2024-05-21T21:55:10.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.854+0000 7eff0ba00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff0406b220 0x7eff04102510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:10.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.854+0000 7eff0ba00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff0406bbe0 0x7eff04102a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:10.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.854+0000 7eff0ba00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff041095b0 0x7eff04102f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:10.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.854+0000 7eff0ba00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7eff04113720 con 0x7eff0406bbe0 2024-05-21T21:55:10.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.854+0000 7eff0ba00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7eff041135a0 con 0x7eff041095b0 2024-05-21T21:55:10.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.854+0000 7eff0ba00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7eff041138a0 con 0x7eff0406b220 2024-05-21T21:55:10.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.854+0000 7eff09000700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff0406b220 0x7eff04102510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:10.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.854+0000 7eff09000700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff0406b220 0x7eff04102510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:39766/0 (socket says 172.21.15.151:39766) 2024-05-21T21:55:10.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.854+0000 7eff09000700 1 -- 172.21.15.151:0/859919380 learned_addr learned my addr 172.21.15.151:0/859919380 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:55:10.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.854+0000 7efefbe00700 1 --2- 172.21.15.151:0/859919380 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff0406bbe0 0x7eff04102a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:10.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.854+0000 7eff09a00700 1 --2- 172.21.15.151:0/859919380 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff041095b0 0x7eff04102f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:10.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.854+0000 7eff09000700 1 -- 172.21.15.151:0/859919380 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff041095b0 msgr2=0x7eff04102f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:10.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.854+0000 7eff09000700 1 --2- 172.21.15.151:0/859919380 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff041095b0 0x7eff04102f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:10.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.854+0000 7eff09000700 1 -- 172.21.15.151:0/859919380 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff0406bbe0 msgr2=0x7eff04102a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:10.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.854+0000 7eff09000700 1 --2- 172.21.15.151:0/859919380 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff0406bbe0 0x7eff04102a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:10.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.854+0000 7eff09000700 1 -- 172.21.15.151:0/859919380 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff041a81a0 con 0x7eff0406b220 2024-05-21T21:55:10.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.855+0000 7eff09000700 1 --2- 172.21.15.151:0/859919380 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff0406b220 0x7eff04102510 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7efef400d980 tx=0x7efef400dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:55:10.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.855+0000 7eff02000700 1 -- 172.21.15.151:0/859919380 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef4016070 con 0x7eff0406b220 2024-05-21T21:55:10.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.855+0000 7eff02000700 1 -- 172.21.15.151:0/859919380 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efef4004830 con 0x7eff0406b220 2024-05-21T21:55:10.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.855+0000 7eff0ba00700 1 -- 172.21.15.151:0/859919380 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff041a8490 con 0x7eff0406b220 2024-05-21T21:55:10.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.855+0000 7eff02000700 1 -- 172.21.15.151:0/859919380 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef4005540 con 0x7eff0406b220 2024-05-21T21:55:10.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.855+0000 7eff0ba00700 1 -- 172.21.15.151:0/859919380 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff041a89d0 con 0x7eff0406b220 2024-05-21T21:55:10.858 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.859+0000 7eff02000700 1 -- 172.21.15.151:0/859919380 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7efef4004220 con 0x7eff0406b220 2024-05-21T21:55:10.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.860+0000 7eff163a3b80 1 Processor -- start 2024-05-21T21:55:10.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.860+0000 7eff163a3b80 1 -- start start 2024-05-21T21:55:10.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.860+0000 7eff163a3b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e498fa8e60 0x55e498fa9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:10.861 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.860+0000 7eff163a3b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e498fa9780 0x55e498fa3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:10.861 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.860+0000 7eff163a3b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e498fa3830 0x55e498fa3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:10.861 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.860+0000 7eff163a3b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e498f01df0 con 0x55e498fa9780 2024-05-21T21:55:10.861 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.860+0000 7eff163a3b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55e498ef1990 con 0x55e498fa8e60 2024-05-21T21:55:10.861 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.860+0000 7eff163a3b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55e498ef59d0 con 0x55e498fa3830 2024-05-21T21:55:10.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.861+0000 7eff09a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e498fa8e60 0x55e498fa9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:10.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.861+0000 7eff09000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e498fa9780 0x55e498fa3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:10.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.861+0000 7eff09a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e498fa8e60 0x55e498fa9240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:50670/0 (socket says 172.21.15.151:50670) 2024-05-21T21:55:10.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.861+0000 7eff09a00700 1 -- 172.21.15.151:0/3563220887 learned_addr learned my addr 172.21.15.151:0/3563220887 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:55:10.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.861+0000 7efefbe00700 1 --2- 172.21.15.151:0/3563220887 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e498fa3830 0x55e498fa3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:10.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.861+0000 7eff02000700 1 --2- 172.21.15.151:0/859919380 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7efee80b2880 0x7efee80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:10.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.861+0000 7eff02000700 1 -- 172.21.15.151:0/859919380 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7efef40e1f80 con 0x7eff0406b220 2024-05-21T21:55:10.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.861+0000 7eff09a00700 1 -- 172.21.15.151:0/3563220887 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e498fa3830 msgr2=0x55e498fa3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:55:10.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.861+0000 7eff09a00700 1 --2- 172.21.15.151:0/3563220887 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e498fa3830 0x55e498fa3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:10.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.861+0000 7eff09a00700 1 -- 172.21.15.151:0/3563220887 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e498fa9780 msgr2=0x55e498fa3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:55:10.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.861+0000 7eff09a00700 1 --2- 172.21.15.151:0/3563220887 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e498fa9780 0x55e498fa3210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:10.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.861+0000 7eff09a00700 1 -- 172.21.15.151:0/3563220887 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e498eba3d0 con 0x55e498fa8e60 2024-05-21T21:55:10.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.861+0000 7eff09a00700 1 --2- 172.21.15.151:0/859919380 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7efee80b2880 0x7efee80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:10.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.862+0000 7eff09a00700 1 --2- 172.21.15.151:0/3563220887 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e498fa8e60 0x55e498fa9240 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7efefc009a90 tx=0x7efefc003f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=383e43f69ec785be server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:55:10.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.862+0000 7efefa000700 1 -- 172.21.15.151:0/3563220887 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efefc018440 con 0x55e498fa8e60 2024-05-21T21:55:10.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.862+0000 7efefa000700 1 -- 172.21.15.151:0/3563220887 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efefc01c050 con 0x55e498fa8e60 2024-05-21T21:55:10.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.862+0000 7efefa000700 1 -- 172.21.15.151:0/3563220887 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efefc022650 con 0x55e498fa8e60 2024-05-21T21:55:10.865 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.863+0000 7eff163a3b80 1 -- 172.21.15.151:0/3563220887 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e498fa8e60 msgr2=0x55e498fa9240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:10.865 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.863+0000 7eff09a00700 1 --2- 172.21.15.151:0/859919380 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7efee80b2880 0x7efee80b4d40 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7efefc021000 tx=0x7efefc01e000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:55:10.865 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.863+0000 7eff163a3b80 1 --2- 172.21.15.151:0/3563220887 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e498fa8e60 0x55e498fa9240 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7efefc009a90 tx=0x7efefc003f90 comp rx=0 tx=0).stop 2024-05-21T21:55:10.865 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.863+0000 7eff163a3b80 1 -- 172.21.15.151:0/3563220887 shutdown_connections 2024-05-21T21:55:10.865 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.863+0000 7eff163a3b80 1 --2- 172.21.15.151:0/3563220887 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e498fa8e60 0x55e498fa9240 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:10.865 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.863+0000 7eff163a3b80 1 --2- 172.21.15.151:0/3563220887 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e498fa3830 0x55e498fa3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:10.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.863+0000 7eff163a3b80 1 --2- 172.21.15.151:0/3563220887 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e498fa9780 0x55e498fa3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:10.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.863+0000 7eff163a3b80 1 -- 172.21.15.151:0/3563220887 >> 172.21.15.151:0/3563220887 conn(0x55e498ed1b90 msgr2=0x55e498f9dfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:55:10.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.863+0000 7eff163a3b80 1 -- 172.21.15.151:0/3563220887 shutdown_connections 2024-05-21T21:55:10.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.863+0000 7eff163a3b80 1 -- 172.21.15.151:0/3563220887 wait complete. 2024-05-21T21:55:10.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.864+0000 7eff163a3b80 1 -- start start 2024-05-21T21:55:10.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.864+0000 7eff163a3b80 1 Processor -- start 2024-05-21T21:55:10.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.864+0000 7eff163a3b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e498fa3830 0x55e498faf440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:10.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.864+0000 7eff163a3b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e498fa8e60 0x55e498fac7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:10.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.864+0000 7eff163a3b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e498fa9780 0x55e498facd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:10.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.864+0000 7eff163a3b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e498df7f50 con 0x55e498fa8e60 2024-05-21T21:55:10.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.864+0000 7eff163a3b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55e498e92710 con 0x55e498fa3830 2024-05-21T21:55:10.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.864+0000 7eff163a3b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55e498fb01f0 con 0x55e498fa9780 2024-05-21T21:55:10.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.864+0000 7eff163a3b80 10 client.0 _ll_register_callbacks cb 0x55e498e9bc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:55:10.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.865+0000 7efefbe00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e498fa8e60 0x55e498fac7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:10.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.865+0000 7efefbe00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e498fa8e60 0x55e498fac7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:37192/0 (socket says 172.21.15.151:37192) 2024-05-21T21:55:10.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.865+0000 7efefbe00700 1 -- 172.21.15.151:0/133248227 learned_addr learned my addr 172.21.15.151:0/133248227 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:55:10.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.865+0000 7eff09000700 1 --2- 172.21.15.151:0/133248227 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e498fa3830 0x55e498faf440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:10.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.865+0000 7eff09a00700 1 --2- 172.21.15.151:0/133248227 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e498fa9780 0x55e498facd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:10.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.865+0000 7efefbe00700 1 -- 172.21.15.151:0/133248227 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e498fa9780 msgr2=0x55e498facd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:55:10.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.865+0000 7efefbe00700 1 --2- 172.21.15.151:0/133248227 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e498fa9780 0x55e498facd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:10.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.865+0000 7efefbe00700 1 -- 172.21.15.151:0/133248227 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e498fa3830 msgr2=0x55e498faf440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:55:10.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.865+0000 7efefbe00700 1 --2- 172.21.15.151:0/133248227 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e498fa3830 0x55e498faf440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:10.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.865+0000 7efefbe00700 1 -- 172.21.15.151:0/133248227 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e498e41880 con 0x55e498fa8e60 2024-05-21T21:55:10.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.865+0000 7eff09000700 1 --2- 172.21.15.151:0/133248227 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e498fa3830 0x55e498faf440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:55:10.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.865+0000 7eff09a00700 1 --2- 172.21.15.151:0/133248227 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e498fa9780 0x55e498facd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T21:55:10.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.865+0000 7efefbe00700 1 --2- 172.21.15.151:0/133248227 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e498fa8e60 0x55e498fac7e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7efeec009990 tx=0x7efeec00bf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd01f3d8c25ba8db server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:55:10.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.865+0000 7efef2a00700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-21T21:55:10.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.866+0000 7efef2a00700 1 -- 172.21.15.151:0/133248227 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeec003a60 con 0x55e498fa8e60 2024-05-21T21:55:10.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.866+0000 7eff163a3b80 1 -- 172.21.15.151:0/133248227 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efefc01c050 con 0x55e498fa8e60 2024-05-21T21:55:10.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.866+0000 7efef2a00700 1 -- 172.21.15.151:0/133248227 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7efeec003c00 con 0x55e498fa8e60 2024-05-21T21:55:10.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.866+0000 7efef2a00700 1 -- 172.21.15.151:0/133248227 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7efeec003680 con 0x55e498fa8e60 2024-05-21T21:55:10.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.866+0000 7eff163a3b80 10 client.14925 fetch_fsmap learned FSMap version 28 2024-05-21T21:55:10.871 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.866+0000 7eff163a3b80 1 -- 172.21.15.151:0/133248227 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e498ec2010 con 0x55e498fa8e60 2024-05-21T21:55:10.871 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.866+0000 7efef2a00700 1 -- 172.21.15.151:0/133248227 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efefc01c050 con 0x55e498fa8e60 2024-05-21T21:55:10.871 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.866+0000 7efef2a00700 1 -- 172.21.15.151:0/133248227 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7efeec00f7a0 con 0x55e498fa8e60 2024-05-21T21:55:10.871 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.867+0000 7eff163a3b80 10 client.14925 fetch_fsmap finished waiting for FSMap version 28 2024-05-21T21:55:10.871 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.867+0000 7eff163a3b80 10 client.14925 resolve_mds: resolved name 'e' to daemon mds.e 2024-05-21T21:55:10.871 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.867+0000 7eff163a3b80 20 client.14925 populate_metadata read hostname 'smithi151' 2024-05-21T21:55:10.871 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.867+0000 7eff163a3b80 1 --2- 172.21.15.151:0/133248227 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x55e498fbcd90 0x55e498fbe200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:10.871 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.867+0000 7eff163a3b80 4 client.14925 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T21:55:10.871 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.867+0000 7eff163a3b80 1 -- 172.21.15.151:0/133248227 --> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e498f03d40 con 0x55e498fbcd90 2024-05-21T21:55:10.871 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.867+0000 7eff09000700 1 --2- 172.21.15.151:0/133248227 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x55e498fbcd90 0x55e498fbe200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:10.871 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.868+0000 7efef2a00700 10 client.14925 ms_handle_connect on v2:172.21.15.164:6834/2340428470 2024-05-21T21:55:10.871 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.868+0000 7eff09000700 1 --2- 172.21.15.151:0/133248227 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x55e498fbcd90 0x55e498fbe200 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=84a322fb5210e126 server_cookie=17aa9422211c91c1 in_seq=0 out_seq=0 2024-05-21T21:55:10.871 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.868+0000 7efef2a00700 1 -- 172.21.15.151:0/133248227 <== mds.4 v2:172.21.15.164:6834/2340428470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55e498f03d40 con 0x55e498fbcd90 2024-05-21T21:55:10.871 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.868+0000 7efef2a00700 10 client.14925 handle_command_reply: tid=0 2024-05-21T21:55:10.871 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.868+0000 7eff163a3b80 1 client.14925 shutdown 2024-05-21T21:55:10.872 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.869+0000 7eff163a3b80 1 -- 172.21.15.151:0/133248227 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e498fa8e60 msgr2=0x55e498fac7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:10.872 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.869+0000 7eff163a3b80 1 --2- 172.21.15.151:0/133248227 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e498fa8e60 0x55e498fac7e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7efeec009990 tx=0x7efeec00bf20 comp rx=0 tx=0).stop 2024-05-21T21:55:10.872 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.869+0000 7eff163a3b80 1 -- 172.21.15.151:0/133248227 shutdown_connections 2024-05-21T21:55:10.872 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.869+0000 7eff163a3b80 1 --2- 172.21.15.151:0/133248227 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x55e498fbcd90 0x55e498fbe200 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:10.872 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.869+0000 7eff163a3b80 1 --2- 172.21.15.151:0/133248227 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e498fa3830 0x55e498faf440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:10.872 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.869+0000 7eff163a3b80 1 --2- 172.21.15.151:0/133248227 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e498fa9780 0x55e498facd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:10.872 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.869+0000 7eff163a3b80 1 --2- 172.21.15.151:0/133248227 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e498fa8e60 0x55e498fac7e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:10.872 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.869+0000 7eff163a3b80 1 -- 172.21.15.151:0/133248227 >> 172.21.15.151:0/133248227 conn(0x55e498ed1b90 msgr2=0x55e498fa2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:55:10.872 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.869+0000 7efef2a00700 0 client.14925 ms_handle_reset on v2:172.21.15.164:6834/2340428470 2024-05-21T21:55:10.872 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.869+0000 7eff163a3b80 1 -- 172.21.15.151:0/133248227 shutdown_connections 2024-05-21T21:55:10.872 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.869+0000 7eff163a3b80 1 -- 172.21.15.151:0/133248227 wait complete. 2024-05-21T21:55:10.872 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.869+0000 7eff163a3b80 20 client.14925 trim_cache size 0 max 16384 2024-05-21T21:55:10.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.891+0000 7eff163a3b80 1 Processor -- start 2024-05-21T21:55:10.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.891+0000 7eff163a3b80 1 -- start start 2024-05-21T21:55:10.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.891+0000 7eff163a3b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e498fa3830 0x55e498fa3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:10.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.891+0000 7eff163a3b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e498fa8e60 0x55e498faccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:10.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.891+0000 7eff163a3b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e498fa9780 0x55e498f9d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:10.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.891+0000 7eff163a3b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e498fbc540 con 0x55e498fa3830 2024-05-21T21:55:10.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.891+0000 7eff163a3b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55e498fbcc10 con 0x55e498fa9780 2024-05-21T21:55:10.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.891+0000 7eff163a3b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55e498fbd2e0 con 0x55e498fa8e60 2024-05-21T21:55:10.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.891+0000 7eff09000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e498fa3830 0x55e498fa3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:10.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.891+0000 7eff09a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e498fa9780 0x55e498f9d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:10.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.891+0000 7eff09a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e498fa9780 0x55e498f9d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:50680/0 (socket says 172.21.15.151:50680) 2024-05-21T21:55:10.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.891+0000 7eff09a00700 1 -- 172.21.15.151:0/1538082166 learned_addr learned my addr 172.21.15.151:0/1538082166 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:55:10.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.891+0000 7efefbe00700 1 --2- 172.21.15.151:0/1538082166 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e498fa8e60 0x55e498faccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:10.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.892+0000 7eff09a00700 1 -- 172.21.15.151:0/1538082166 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e498fa8e60 msgr2=0x55e498faccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:55:10.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.892+0000 7eff09a00700 1 --2- 172.21.15.151:0/1538082166 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e498fa8e60 0x55e498faccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:10.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.892+0000 7eff09a00700 1 -- 172.21.15.151:0/1538082166 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e498fa3830 msgr2=0x55e498fa3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:55:10.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.892+0000 7eff09a00700 1 --2- 172.21.15.151:0/1538082166 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e498fa3830 0x55e498fa3210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:10.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.892+0000 7eff09a00700 1 -- 172.21.15.151:0/1538082166 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e498f96140 con 0x55e498fa9780 2024-05-21T21:55:10.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.892+0000 7eff09a00700 1 --2- 172.21.15.151:0/1538082166 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e498fa9780 0x55e498f9d990 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7efefc0099b0 tx=0x7efefc005af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eca37e4010e1035e server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:55:10.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.892+0000 7efef9600700 1 -- 172.21.15.151:0/1538082166 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efefc02f030 con 0x55e498fa9780 2024-05-21T21:55:10.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.892+0000 7efef9600700 1 -- 172.21.15.151:0/1538082166 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efefc009560 con 0x55e498fa9780 2024-05-21T21:55:10.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.892+0000 7efef9600700 1 -- 172.21.15.151:0/1538082166 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efefc018440 con 0x55e498fa9780 2024-05-21T21:55:10.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.893+0000 7eff163a3b80 1 -- 172.21.15.151:0/1538082166 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e498fa9780 msgr2=0x55e498f9d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:10.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.893+0000 7eff163a3b80 1 --2- 172.21.15.151:0/1538082166 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e498fa9780 0x55e498f9d990 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7efefc0099b0 tx=0x7efefc005af0 comp rx=0 tx=0).stop 2024-05-21T21:55:10.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.893+0000 7eff163a3b80 1 -- 172.21.15.151:0/1538082166 shutdown_connections 2024-05-21T21:55:10.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.893+0000 7eff163a3b80 1 --2- 172.21.15.151:0/1538082166 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e498fa9780 0x55e498f9d990 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:10.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.893+0000 7eff163a3b80 1 --2- 172.21.15.151:0/1538082166 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e498fa3830 0x55e498fa3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:10.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.893+0000 7eff163a3b80 1 --2- 172.21.15.151:0/1538082166 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e498fa8e60 0x55e498faccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:10.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.893+0000 7eff163a3b80 1 -- 172.21.15.151:0/1538082166 >> 172.21.15.151:0/1538082166 conn(0x55e498ed1b90 msgr2=0x55e498fac5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:55:10.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.893+0000 7eff163a3b80 1 -- 172.21.15.151:0/1538082166 shutdown_connections 2024-05-21T21:55:10.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.893+0000 7eff163a3b80 1 -- 172.21.15.151:0/1538082166 wait complete. 2024-05-21T21:55:10.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.894+0000 7eff163a3b80 1 -- start start 2024-05-21T21:55:10.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.894+0000 7eff163a3b80 1 Processor -- start 2024-05-21T21:55:10.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.894+0000 7eff163a3b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e498fa3830 0x55e498fc7680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:10.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.894+0000 7eff163a3b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e498fa8e60 0x55e498fc7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:10.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.894+0000 7eff163a3b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e498fa9780 0x55e498fc2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:10.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.894+0000 7eff163a3b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e498fbe080 con 0x55e498fa9780 2024-05-21T21:55:10.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.894+0000 7eff163a3b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55e498fbd9b0 con 0x55e498fa3830 2024-05-21T21:55:10.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.894+0000 7eff163a3b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55e498fbe750 con 0x55e498fa8e60 2024-05-21T21:55:10.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.894+0000 7eff163a3b80 10 client.0 _ll_register_callbacks cb 0x55e498e9bc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:55:10.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.895+0000 7eff09000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e498fa3830 0x55e498fc7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:10.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.895+0000 7efefbe00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e498fa8e60 0x55e498fc7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:10.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.895+0000 7eff09000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e498fa3830 0x55e498fc7680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:50690/0 (socket says 172.21.15.151:50690) 2024-05-21T21:55:10.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.895+0000 7eff09000700 1 -- 172.21.15.151:0/577210864 learned_addr learned my addr 172.21.15.151:0/577210864 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:55:10.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.895+0000 7eff09a00700 1 --2- 172.21.15.151:0/577210864 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e498fa9780 0x55e498fc2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:10.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.895+0000 7eff09000700 1 -- 172.21.15.151:0/577210864 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e498fa8e60 msgr2=0x55e498fc7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:55:10.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.895+0000 7eff09000700 1 --2- 172.21.15.151:0/577210864 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e498fa8e60 0x55e498fc7bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:10.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.895+0000 7eff09000700 1 -- 172.21.15.151:0/577210864 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e498fa9780 msgr2=0x55e498fc2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:55:10.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.895+0000 7eff09000700 1 --2- 172.21.15.151:0/577210864 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e498fa9780 0x55e498fc2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:10.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.895+0000 7eff09000700 1 -- 172.21.15.151:0/577210864 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e498fc2ca0 con 0x55e498fa3830 2024-05-21T21:55:10.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.895+0000 7eff09000700 1 --2- 172.21.15.151:0/577210864 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e498fa3830 0x55e498fc7680 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7efef400d950 tx=0x7efef40a5ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88e0c0a6938f3b93 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:55:10.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.895+0000 7efef2a00700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-21T21:55:10.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.895+0000 7efef2a00700 1 -- 172.21.15.151:0/577210864 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef40c3060 con 0x55e498fa3830 2024-05-21T21:55:10.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.895+0000 7efef2a00700 1 -- 172.21.15.151:0/577210864 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7efef4004ce0 con 0x55e498fa3830 2024-05-21T21:55:10.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.895+0000 7eff163a3b80 1 -- 172.21.15.151:0/577210864 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efefc009560 con 0x55e498fa3830 2024-05-21T21:55:10.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.896+0000 7efef2a00700 1 -- 172.21.15.151:0/577210864 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7efef4014030 con 0x55e498fa3830 2024-05-21T21:55:10.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.896+0000 7eff163a3b80 10 client.25055 fetch_fsmap learned FSMap version 28 2024-05-21T21:55:10.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.896+0000 7eff163a3b80 1 -- 172.21.15.151:0/577210864 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e499028e80 con 0x55e498fa3830 2024-05-21T21:55:10.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.896+0000 7efef2a00700 1 -- 172.21.15.151:0/577210864 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efefc009560 con 0x55e498fa3830 2024-05-21T21:55:10.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.896+0000 7efef2a00700 1 -- 172.21.15.151:0/577210864 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7efef40a97c0 con 0x55e498fa3830 2024-05-21T21:55:10.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.896+0000 7eff163a3b80 10 client.25055 fetch_fsmap finished waiting for FSMap version 28 2024-05-21T21:55:10.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.896+0000 7eff163a3b80 10 client.25055 resolve_mds: resolved name 'e' to daemon mds.e 2024-05-21T21:55:10.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.896+0000 7eff163a3b80 20 client.25055 populate_metadata read hostname 'smithi151' 2024-05-21T21:55:10.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.896+0000 7eff163a3b80 1 --2- 172.21.15.151:0/577210864 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x55e49902da40 0x55e49902c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:10.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.896+0000 7eff163a3b80 4 client.25055 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-21T21:55:10.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.896+0000 7eff163a3b80 1 -- 172.21.15.151:0/577210864 --> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] -- command(tid 0: {"prefix": "status"}) -- 0x55e498efd520 con 0x55e49902da40 2024-05-21T21:55:10.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.896+0000 7efefbe00700 1 --2- 172.21.15.151:0/577210864 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x55e49902da40 0x55e49902c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:10.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.897+0000 7efefbe00700 1 --2- 172.21.15.151:0/577210864 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x55e49902da40 0x55e49902c750 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8b1c3f96aab6bf3d server_cookie=d1aa24272bb1cfd3 in_seq=0 out_seq=0 2024-05-21T21:55:10.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.897+0000 7efef2a00700 10 client.25055 ms_handle_connect on v2:172.21.15.164:6834/2340428470 2024-05-21T21:55:10.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.897+0000 7efef2a00700 1 -- 172.21.15.151:0/577210864 <== mds.4 v2:172.21.15.164:6834/2340428470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55e498efd520 con 0x55e49902da40 2024-05-21T21:55:10.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.897+0000 7efef2a00700 10 client.25055 handle_command_reply: tid=0 2024-05-21T21:55:10.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.897+0000 7eff163a3b80 1 client.25055 shutdown 2024-05-21T21:55:10.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.897+0000 7eff163a3b80 1 -- 172.21.15.151:0/577210864 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e498fa3830 msgr2=0x55e498fc7680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:10.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.897+0000 7eff163a3b80 1 --2- 172.21.15.151:0/577210864 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e498fa3830 0x55e498fc7680 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7efef400d950 tx=0x7efef40a5ce0 comp rx=0 tx=0).stop 2024-05-21T21:55:10.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.897+0000 7eff163a3b80 1 -- 172.21.15.151:0/577210864 shutdown_connections 2024-05-21T21:55:10.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.897+0000 7eff163a3b80 1 --2- 172.21.15.151:0/577210864 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x55e49902da40 0x55e49902c750 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:10.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.897+0000 7eff163a3b80 1 --2- 172.21.15.151:0/577210864 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e498fa3830 0x55e498fc7680 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:10.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.897+0000 7eff163a3b80 1 --2- 172.21.15.151:0/577210864 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e498fa9780 0x55e498fc2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:10.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.897+0000 7eff163a3b80 1 --2- 172.21.15.151:0/577210864 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e498fa8e60 0x55e498fc7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:10.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.897+0000 7efef2a00700 0 client.25055 ms_handle_reset on v2:172.21.15.164:6834/2340428470 2024-05-21T21:55:10.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.897+0000 7eff163a3b80 1 -- 172.21.15.151:0/577210864 >> 172.21.15.151:0/577210864 conn(0x55e498ed1b90 msgr2=0x55e498f9d990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:55:10.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.898+0000 7eff163a3b80 1 -- 172.21.15.151:0/577210864 shutdown_connections 2024-05-21T21:55:10.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.898+0000 7eff163a3b80 1 -- 172.21.15.151:0/577210864 wait complete. 2024-05-21T21:55:10.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.898+0000 7eff163a3b80 20 client.25055 trim_cache size 0 max 16384 2024-05-21T21:55:10.902 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-05-21T21:55:10.902 INFO:teuthology.orchestra.run.smithi151.stdout: "cluster_fsid": "c8c7a2e6-17ba-11ef-bc9a-c7b262605968", 2024-05-21T21:55:10.902 INFO:teuthology.orchestra.run.smithi151.stdout: "whoami": 4, 2024-05-21T21:55:10.902 INFO:teuthology.orchestra.run.smithi151.stdout: "id": 24473, 2024-05-21T21:55:10.902 INFO:teuthology.orchestra.run.smithi151.stdout: "want_state": "up:active", 2024-05-21T21:55:10.902 INFO:teuthology.orchestra.run.smithi151.stdout: "state": "up:active", 2024-05-21T21:55:10.902 INFO:teuthology.orchestra.run.smithi151.stdout: "fs_name": "cephfs", 2024-05-21T21:55:10.902 INFO:teuthology.orchestra.run.smithi151.stdout: "rank_uptime": 212.56484872499999, 2024-05-21T21:55:10.903 INFO:teuthology.orchestra.run.smithi151.stdout: "mdsmap_epoch": 28, 2024-05-21T21:55:10.903 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_epoch": 82, 2024-05-21T21:55:10.903 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_epoch_barrier": 80, 2024-05-21T21:55:10.903 INFO:teuthology.orchestra.run.smithi151.stdout: "uptime": 275.67666830500002 2024-05-21T21:55:10.903 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-05-21T21:55:10.903 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.898+0000 7eff0ba00700 1 -- 172.21.15.151:0/859919380 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7efee80b2880 msgr2=0x7efee80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:10.903 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.898+0000 7eff0ba00700 1 --2- 172.21.15.151:0/859919380 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7efee80b2880 0x7efee80b4d40 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7efefc021000 tx=0x7efefc01e000 comp rx=0 tx=0).stop 2024-05-21T21:55:10.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.898+0000 7eff0ba00700 1 -- 172.21.15.151:0/859919380 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff0406b220 msgr2=0x7eff04102510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:10.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.898+0000 7eff0ba00700 1 --2- 172.21.15.151:0/859919380 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff0406b220 0x7eff04102510 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7efef400d980 tx=0x7efef400dc90 comp rx=0 tx=0).stop 2024-05-21T21:55:10.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.898+0000 7eff0ba00700 1 -- 172.21.15.151:0/859919380 shutdown_connections 2024-05-21T21:55:10.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.898+0000 7eff0ba00700 1 --2- 172.21.15.151:0/859919380 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7efee80b2880 0x7efee80b4d40 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:10.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.898+0000 7eff0ba00700 1 --2- 172.21.15.151:0/859919380 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff041095b0 0x7eff04102f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:10.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.898+0000 7eff0ba00700 1 --2- 172.21.15.151:0/859919380 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff0406b220 0x7eff04102510 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:10.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.898+0000 7eff0ba00700 1 --2- 172.21.15.151:0/859919380 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff0406bbe0 0x7eff04102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:10.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.898+0000 7eff0ba00700 1 -- 172.21.15.151:0/859919380 >> 172.21.15.151:0/859919380 conn(0x7eff040793f0 msgr2=0x7eff04101140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:55:10.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.899+0000 7eff0ba00700 1 -- 172.21.15.151:0/859919380 shutdown_connections 2024-05-21T21:55:10.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:10.899+0000 7eff0ba00700 1 -- 172.21.15.151:0/859919380 wait complete. 2024-05-21T21:55:11.799 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.c status 2024-05-21T21:55:12.254 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:55:12.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:12 smithi151 ceph-mon[31623]: pgmap v433: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-05-21T21:55:12.735 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:12 smithi184 ceph-mon[36942]: pgmap v433: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-05-21T21:55:12.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:12 smithi164 ceph-mon[36269]: pgmap v433: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-05-21T21:55:14.546 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.545+0000 7f5e03200700 1 -- 172.21.15.151:0/3307495610 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5e0410a440 msgr2=0x7f5e041168e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:14.547 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.545+0000 7f5e03200700 1 --2- 172.21.15.151:0/3307495610 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5e0410a440 0x7f5e041168e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc00b0a0 tx=0x7f5dfc00b3b0 comp rx=0 tx=0).stop 2024-05-21T21:55:14.547 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.546+0000 7f5e03200700 1 -- 172.21.15.151:0/3307495610 shutdown_connections 2024-05-21T21:55:14.547 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.546+0000 7f5e03200700 1 --2- 172.21.15.151:0/3307495610 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5e0410a440 0x7f5e041168e0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:14.547 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.546+0000 7f5e03200700 1 --2- 172.21.15.151:0/3307495610 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5e0410e950 0x7f5e0410ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:14.547 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.546+0000 7f5e03200700 1 --2- 172.21.15.151:0/3307495610 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e04109a80 0x7f5e04109f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:14.547 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.546+0000 7f5e03200700 1 -- 172.21.15.151:0/3307495610 >> 172.21.15.151:0/3307495610 conn(0x7f5e040792d0 msgr2=0x7f5e04108360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:55:14.547 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:14 smithi151 ceph-mon[31623]: pgmap v434: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-05-21T21:55:14.550 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.546+0000 7f5e03200700 1 -- 172.21.15.151:0/3307495610 shutdown_connections 2024-05-21T21:55:14.550 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.546+0000 7f5e03200700 1 -- 172.21.15.151:0/3307495610 wait complete. 2024-05-21T21:55:14.550 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.547+0000 7f5e03200700 1 Processor -- start 2024-05-21T21:55:14.550 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.548+0000 7f5e03200700 1 -- start start 2024-05-21T21:55:14.550 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.548+0000 7f5e03200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5e04109a80 0x7f5e041a0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:14.550 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.548+0000 7f5e03200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e0410a440 0x7f5e041a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:14.550 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.548+0000 7f5e03200700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5e0410e950 0x7f5e041a4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:14.550 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.548+0000 7f5e03200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5e04119c60 con 0x7f5e04109a80 2024-05-21T21:55:14.550 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.548+0000 7f5e03200700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5e04119ae0 con 0x7f5e0410a440 2024-05-21T21:55:14.550 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.548+0000 7f5e03200700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5e04119de0 con 0x7f5e0410e950 2024-05-21T21:55:14.550 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.548+0000 7f5e01e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5e04109a80 0x7f5e041a0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:14.551 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.548+0000 7f5e01e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5e04109a80 0x7f5e041a0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:37224/0 (socket says 172.21.15.151:37224) 2024-05-21T21:55:14.551 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.548+0000 7f5e01e00700 1 -- 172.21.15.151:0/1639540873 learned_addr learned my addr 172.21.15.151:0/1639540873 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:55:14.551 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.548+0000 7f5e01400700 1 --2- 172.21.15.151:0/1639540873 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e0410a440 0x7f5e041a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:14.551 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.548+0000 7f5e02800700 1 --2- 172.21.15.151:0/1639540873 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5e0410e950 0x7f5e041a4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:14.551 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.549+0000 7f5e01400700 1 -- 172.21.15.151:0/1639540873 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5e0410e950 msgr2=0x7f5e041a4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:14.551 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.549+0000 7f5e01400700 1 --2- 172.21.15.151:0/1639540873 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5e0410e950 0x7f5e041a4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:14.551 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.549+0000 7f5e01400700 1 -- 172.21.15.151:0/1639540873 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5e04109a80 msgr2=0x7f5e041a0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:14.551 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.549+0000 7f5e01400700 1 --2- 172.21.15.151:0/1639540873 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5e04109a80 0x7f5e041a0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:14.551 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.549+0000 7f5e01400700 1 -- 172.21.15.151:0/1639540873 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e041a5160 con 0x7f5e0410a440 2024-05-21T21:55:14.551 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.549+0000 7f5e01e00700 1 --2- 172.21.15.151:0/1639540873 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5e04109a80 0x7f5e041a0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:55:14.551 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.549+0000 7f5e02800700 1 --2- 172.21.15.151:0/1639540873 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5e0410e950 0x7f5e041a4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:55:14.552 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.549+0000 7f5e01400700 1 --2- 172.21.15.151:0/1639540873 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e0410a440 0x7f5e041a0690 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5dec00e9f0 tx=0x7f5dec00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:55:14.552 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.549+0000 7f5dfaa00700 1 -- 172.21.15.151:0/1639540873 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dec00c9c0 con 0x7f5e0410a440 2024-05-21T21:55:14.552 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.550+0000 7f5e03200700 1 -- 172.21.15.151:0/1639540873 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e041a5450 con 0x7f5e0410a440 2024-05-21T21:55:14.552 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.550+0000 7f5dfaa00700 1 -- 172.21.15.151:0/1639540873 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5dec00cb60 con 0x7f5e0410a440 2024-05-21T21:55:14.552 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.550+0000 7f5dfaa00700 1 -- 172.21.15.151:0/1639540873 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dec018680 con 0x7f5e0410a440 2024-05-21T21:55:14.552 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.550+0000 7f5e03200700 1 -- 172.21.15.151:0/1639540873 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e041accd0 con 0x7f5e0410a440 2024-05-21T21:55:14.554 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.554+0000 7f5dfaa00700 1 -- 172.21.15.151:0/1639540873 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f5dec010430 con 0x7f5e0410a440 2024-05-21T21:55:14.554 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.554+0000 7f5e13584b80 1 Processor -- start 2024-05-21T21:55:14.554 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.555+0000 7f5e13584b80 1 -- start start 2024-05-21T21:55:14.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.555+0000 7f5e13584b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a8ced2e770 0x55a8ced2eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:14.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.555+0000 7f5e13584b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a8ced2f090 0x55a8ced28d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:14.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.555+0000 7f5e13584b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a8ced29340 0x55a8ced297a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:14.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.555+0000 7f5e13584b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a8cec86df0 con 0x55a8ced2f090 2024-05-21T21:55:14.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.555+0000 7f5e13584b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55a8cec76990 con 0x55a8ced2e770 2024-05-21T21:55:14.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.555+0000 7f5e13584b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55a8cec7a9d0 con 0x55a8ced29340 2024-05-21T21:55:14.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.555+0000 7f5e01e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a8ced2f090 0x55a8ced28d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:14.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.555+0000 7f5e01400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a8ced29340 0x55a8ced297a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:14.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.556+0000 7f5e01400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a8ced29340 0x55a8ced297a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:39818/0 (socket says 172.21.15.151:39818) 2024-05-21T21:55:14.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.556+0000 7f5e01400700 1 -- 172.21.15.151:0/36266450 learned_addr learned my addr 172.21.15.151:0/36266450 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:55:14.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.556+0000 7f5e01e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a8ced2f090 0x55a8ced28d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:37238/0 (socket says 172.21.15.151:37238) 2024-05-21T21:55:14.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.556+0000 7f5e01400700 1 -- 172.21.15.151:0/36266450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a8ced2e770 msgr2=0x55a8ced2eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:55:14.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.556+0000 7f5e02800700 1 --2- 172.21.15.151:0/36266450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a8ced2e770 0x55a8ced2eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:14.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.556+0000 7f5e01400700 1 --2- 172.21.15.151:0/36266450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a8ced2e770 0x55a8ced2eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:14.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.556+0000 7f5e01400700 1 -- 172.21.15.151:0/36266450 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a8ced2f090 msgr2=0x55a8ced28d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:55:14.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.556+0000 7f5e01400700 1 --2- 172.21.15.151:0/36266450 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a8ced2f090 0x55a8ced28d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:14.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.556+0000 7f5e01400700 1 -- 172.21.15.151:0/36266450 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8cec3f3d0 con 0x55a8ced29340 2024-05-21T21:55:14.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.557+0000 7f5e01400700 1 --2- 172.21.15.151:0/36266450 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a8ced29340 0x55a8ced297a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5dec0ea530 tx=0x7f5dec0ea840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8248725ebba4d0f server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:55:14.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.557+0000 7f5dfaa00700 1 --2- 172.21.15.151:0/1639540873 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f5de80b2880 0x7f5de80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:14.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.557+0000 7f5dfaa00700 1 -- 172.21.15.151:0/1639540873 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f5dec0e23a0 con 0x7f5e0410a440 2024-05-21T21:55:14.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.557+0000 7f5e02800700 1 --2- 172.21.15.151:0/1639540873 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f5de80b2880 0x7f5de80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:14.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.557+0000 7f5df3400700 1 -- 172.21.15.151:0/36266450 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dec0a5840 con 0x55a8ced29340 2024-05-21T21:55:14.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.557+0000 7f5df3400700 1 -- 172.21.15.151:0/36266450 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5dec0a9490 con 0x55a8ced29340 2024-05-21T21:55:14.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.557+0000 7f5df3400700 1 -- 172.21.15.151:0/36266450 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dec0a90f0 con 0x55a8ced29340 2024-05-21T21:55:14.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.558+0000 7f5e13584b80 1 -- 172.21.15.151:0/36266450 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a8ced29340 msgr2=0x55a8ced297a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:14.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.558+0000 7f5e13584b80 1 --2- 172.21.15.151:0/36266450 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a8ced29340 0x55a8ced297a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5dec0ea530 tx=0x7f5dec0ea840 comp rx=0 tx=0).stop 2024-05-21T21:55:14.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.558+0000 7f5e02800700 1 --2- 172.21.15.151:0/1639540873 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f5de80b2880 0x7f5de80b4d40 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc00b780 tx=0x7f5dfc006210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:55:14.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.558+0000 7f5e13584b80 1 -- 172.21.15.151:0/36266450 shutdown_connections 2024-05-21T21:55:14.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.558+0000 7f5e13584b80 1 --2- 172.21.15.151:0/36266450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a8ced2e770 0x55a8ced2eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:14.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.558+0000 7f5e13584b80 1 --2- 172.21.15.151:0/36266450 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a8ced29340 0x55a8ced297a0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:14.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.558+0000 7f5e13584b80 1 --2- 172.21.15.151:0/36266450 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a8ced2f090 0x55a8ced28d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:14.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.558+0000 7f5e13584b80 1 -- 172.21.15.151:0/36266450 >> 172.21.15.151:0/36266450 conn(0x55a8cec56b90 msgr2=0x55a8ced237c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:55:14.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.558+0000 7f5e13584b80 1 -- 172.21.15.151:0/36266450 shutdown_connections 2024-05-21T21:55:14.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.558+0000 7f5e13584b80 1 -- 172.21.15.151:0/36266450 wait complete. 2024-05-21T21:55:14.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.559+0000 7f5e13584b80 1 -- start start 2024-05-21T21:55:14.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.559+0000 7f5e13584b80 1 Processor -- start 2024-05-21T21:55:14.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.559+0000 7f5e13584b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a8ced29340 0x55a8ced34da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:14.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.559+0000 7f5e13584b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a8ced2e770 0x55a8ced32140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:14.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.559+0000 7f5e13584b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a8ced2f090 0x55a8ced32680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:14.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.559+0000 7f5e13584b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a8ceb7cf50 con 0x55a8ced29340 2024-05-21T21:55:14.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.559+0000 7f5e13584b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55a8cec17710 con 0x55a8ced2f090 2024-05-21T21:55:14.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.559+0000 7f5e13584b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55a8ced35bb0 con 0x55a8ced2e770 2024-05-21T21:55:14.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.560+0000 7f5e13584b80 10 client.0 _ll_register_callbacks cb 0x55a8cec20c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:55:14.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.560+0000 7f5e01e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a8ced29340 0x55a8ced34da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:14.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.560+0000 7f5e01400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a8ced2e770 0x55a8ced32140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:14.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.560+0000 7f5e01e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a8ced29340 0x55a8ced34da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:37252/0 (socket says 172.21.15.151:37252) 2024-05-21T21:55:14.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.560+0000 7f5e01e00700 1 -- 172.21.15.151:0/2273424025 learned_addr learned my addr 172.21.15.151:0/2273424025 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:55:14.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.560+0000 7f5e02800700 1 --2- 172.21.15.151:0/2273424025 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a8ced2f090 0x55a8ced32680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:14.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.560+0000 7f5e01400700 1 -- 172.21.15.151:0/2273424025 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a8ced2f090 msgr2=0x55a8ced32680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:55:14.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.560+0000 7f5e01400700 1 --2- 172.21.15.151:0/2273424025 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a8ced2f090 0x55a8ced32680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:14.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.560+0000 7f5e01400700 1 -- 172.21.15.151:0/2273424025 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a8ced29340 msgr2=0x55a8ced34da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:55:14.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.560+0000 7f5e01400700 1 --2- 172.21.15.151:0/2273424025 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a8ced29340 0x55a8ced34da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:14.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.560+0000 7f5e01400700 1 -- 172.21.15.151:0/2273424025 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8cebc6880 con 0x55a8ced2e770 2024-05-21T21:55:14.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.560+0000 7f5e01e00700 1 --2- 172.21.15.151:0/2273424025 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a8ced29340 0x55a8ced34da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:55:14.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.561+0000 7f5e01400700 1 --2- 172.21.15.151:0/2273424025 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a8ced2e770 0x55a8ced32140 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5dec0e59e0 tx=0x7f5dec0057e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22f136d488c93e11 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:55:14.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.561+0000 7f5de7e00700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-21T21:55:14.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.561+0000 7f5de7e00700 1 -- 172.21.15.151:0/2273424025 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dec0a92d0 con 0x55a8ced2e770 2024-05-21T21:55:14.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.561+0000 7f5de7e00700 1 -- 172.21.15.151:0/2273424025 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f5dec0e5a70 con 0x55a8ced2e770 2024-05-21T21:55:14.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.561+0000 7f5e13584b80 1 -- 172.21.15.151:0/2273424025 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5dec0a9490 con 0x55a8ced2e770 2024-05-21T21:55:14.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.561+0000 7f5de7e00700 1 -- 172.21.15.151:0/2273424025 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5dec0a9750 con 0x55a8ced2e770 2024-05-21T21:55:14.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.561+0000 7f5e13584b80 10 client.25057 fetch_fsmap learned FSMap version 28 2024-05-21T21:55:14.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.561+0000 7f5e13584b80 1 -- 172.21.15.151:0/2273424025 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a8cec47010 con 0x55a8ced2e770 2024-05-21T21:55:14.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.561+0000 7f5de7e00700 1 -- 172.21.15.151:0/2273424025 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5dec0a9490 con 0x55a8ced2e770 2024-05-21T21:55:14.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.561+0000 7f5de7e00700 1 -- 172.21.15.151:0/2273424025 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f5dec0e43e0 con 0x55a8ced2e770 2024-05-21T21:55:14.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.562+0000 7f5e13584b80 10 client.25057 fetch_fsmap finished waiting for FSMap version 28 2024-05-21T21:55:14.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.562+0000 7f5e13584b80 10 client.25057 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-21T21:55:14.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.562+0000 7f5e13584b80 20 client.25057 populate_metadata read hostname 'smithi151' 2024-05-21T21:55:14.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.562+0000 7f5e13584b80 1 --2- 172.21.15.151:0/2273424025 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x55a8ced413d0 0x55a8ced434e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:14.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.562+0000 7f5e13584b80 4 client.25057 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T21:55:14.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.562+0000 7f5e13584b80 1 -- 172.21.15.151:0/2273424025 --> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a8cec88d40 con 0x55a8ced413d0 2024-05-21T21:55:14.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.562+0000 7f5e01e00700 1 --2- 172.21.15.151:0/2273424025 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x55a8ced413d0 0x55a8ced434e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:14.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.563+0000 7f5e01e00700 1 --2- 172.21.15.151:0/2273424025 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x55a8ced413d0 0x55a8ced434e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=cd8a6cf63fc8c46e server_cookie=c6469254ed4b2771 in_seq=0 out_seq=0 2024-05-21T21:55:14.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.563+0000 7f5de7e00700 10 client.25057 ms_handle_connect on v2:172.21.15.184:6836/2988194888 2024-05-21T21:55:14.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.564+0000 7f5de7e00700 1 -- 172.21.15.151:0/2273424025 <== mds.3 v2:172.21.15.184:6836/2988194888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55a8cec88d40 con 0x55a8ced413d0 2024-05-21T21:55:14.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.564+0000 7f5de7e00700 10 client.25057 handle_command_reply: tid=0 2024-05-21T21:55:14.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.564+0000 7f5e13584b80 1 client.25057 shutdown 2024-05-21T21:55:14.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.564+0000 7f5e13584b80 1 -- 172.21.15.151:0/2273424025 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a8ced2e770 msgr2=0x55a8ced32140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:14.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.564+0000 7f5e13584b80 1 --2- 172.21.15.151:0/2273424025 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a8ced2e770 0x55a8ced32140 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5dec0e59e0 tx=0x7f5dec0057e0 comp rx=0 tx=0).stop 2024-05-21T21:55:14.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.564+0000 7f5e13584b80 1 -- 172.21.15.151:0/2273424025 shutdown_connections 2024-05-21T21:55:14.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.564+0000 7f5e13584b80 1 --2- 172.21.15.151:0/2273424025 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x55a8ced413d0 0x55a8ced434e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:14.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.564+0000 7f5e13584b80 1 --2- 172.21.15.151:0/2273424025 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a8ced2f090 0x55a8ced32680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:14.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.564+0000 7f5e13584b80 1 --2- 172.21.15.151:0/2273424025 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a8ced29340 0x55a8ced34da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:14.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.564+0000 7f5de7e00700 0 client.25057 ms_handle_reset on v2:172.21.15.184:6836/2988194888 2024-05-21T21:55:14.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.564+0000 7f5e13584b80 1 --2- 172.21.15.151:0/2273424025 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a8ced2e770 0x55a8ced32140 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:14.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.564+0000 7f5e13584b80 1 -- 172.21.15.151:0/2273424025 >> 172.21.15.151:0/2273424025 conn(0x55a8cec56b90 msgr2=0x55a8ced22f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:55:14.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.564+0000 7f5e13584b80 1 -- 172.21.15.151:0/2273424025 shutdown_connections 2024-05-21T21:55:14.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.564+0000 7f5e13584b80 1 -- 172.21.15.151:0/2273424025 wait complete. 2024-05-21T21:55:14.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.564+0000 7f5e13584b80 20 client.25057 trim_cache size 0 max 16384 2024-05-21T21:55:14.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.587+0000 7f5e13584b80 1 Processor -- start 2024-05-21T21:55:14.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.588+0000 7f5e13584b80 1 -- start start 2024-05-21T21:55:14.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.588+0000 7f5e13584b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a8ced29340 0x55a8ced28d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:14.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.588+0000 7f5e13584b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a8ced2e770 0x55a8ced2fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:14.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.588+0000 7f5e13584b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a8ced2f090 0x55a8ced236f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:14.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.588+0000 7f5e13584b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a8ced41820 con 0x55a8ced2f090 2024-05-21T21:55:14.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.588+0000 7f5e13584b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55a8ced41ef0 con 0x55a8ced2e770 2024-05-21T21:55:14.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.588+0000 7f5e13584b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55a8ced425c0 con 0x55a8ced29340 2024-05-21T21:55:14.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.588+0000 7f5e01400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a8ced2e770 0x55a8ced2fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:14.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.588+0000 7f5e02800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a8ced2f090 0x55a8ced236f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:14.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.588+0000 7f5e01400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a8ced2e770 0x55a8ced2fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:50732/0 (socket says 172.21.15.151:50732) 2024-05-21T21:55:14.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.588+0000 7f5e01400700 1 -- 172.21.15.151:0/2829481006 learned_addr learned my addr 172.21.15.151:0/2829481006 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:55:14.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.588+0000 7f5e01e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a8ced29340 0x55a8ced28d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:14.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.589+0000 7f5e01400700 1 -- 172.21.15.151:0/2829481006 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a8ced29340 msgr2=0x55a8ced28d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:55:14.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.589+0000 7f5e01400700 1 --2- 172.21.15.151:0/2829481006 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a8ced29340 0x55a8ced28d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:14.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.589+0000 7f5e01400700 1 -- 172.21.15.151:0/2829481006 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a8ced2f090 msgr2=0x55a8ced236f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:55:14.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.589+0000 7f5e01400700 1 --2- 172.21.15.151:0/2829481006 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a8ced2f090 0x55a8ced236f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:14.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.589+0000 7f5e01400700 1 -- 172.21.15.151:0/2829481006 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8ced349e0 con 0x55a8ced2e770 2024-05-21T21:55:14.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.589+0000 7f5e01400700 1 --2- 172.21.15.151:0/2829481006 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a8ced2e770 0x55a8ced2fd00 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5dec0ea450 tx=0x7f5dec00e940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=757db6c94ae3c0c server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:55:14.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.589+0000 7f5df2a00700 1 -- 172.21.15.151:0/2829481006 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dec0a5110 con 0x55a8ced2e770 2024-05-21T21:55:14.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.589+0000 7f5df2a00700 1 -- 172.21.15.151:0/2829481006 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5dec0a9490 con 0x55a8ced2e770 2024-05-21T21:55:14.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.589+0000 7f5df2a00700 1 -- 172.21.15.151:0/2829481006 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dec0b2350 con 0x55a8ced2e770 2024-05-21T21:55:14.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.590+0000 7f5e13584b80 1 -- 172.21.15.151:0/2829481006 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a8ced2e770 msgr2=0x55a8ced2fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:14.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.590+0000 7f5e13584b80 1 --2- 172.21.15.151:0/2829481006 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a8ced2e770 0x55a8ced2fd00 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5dec0ea450 tx=0x7f5dec00e940 comp rx=0 tx=0).stop 2024-05-21T21:55:14.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.590+0000 7f5e13584b80 1 -- 172.21.15.151:0/2829481006 shutdown_connections 2024-05-21T21:55:14.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.590+0000 7f5e13584b80 1 --2- 172.21.15.151:0/2829481006 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a8ced2f090 0x55a8ced236f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:14.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.590+0000 7f5e13584b80 1 --2- 172.21.15.151:0/2829481006 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a8ced29340 0x55a8ced28d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:14.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.590+0000 7f5e13584b80 1 --2- 172.21.15.151:0/2829481006 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a8ced2e770 0x55a8ced2fd00 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:14.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.590+0000 7f5e13584b80 1 -- 172.21.15.151:0/2829481006 >> 172.21.15.151:0/2829481006 conn(0x55a8cec56b90 msgr2=0x55a8ced32590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:55:14.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.590+0000 7f5e13584b80 1 -- 172.21.15.151:0/2829481006 shutdown_connections 2024-05-21T21:55:14.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.590+0000 7f5e13584b80 1 -- 172.21.15.151:0/2829481006 wait complete. 2024-05-21T21:55:14.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.590+0000 7f5e13584b80 1 -- start start 2024-05-21T21:55:14.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.591+0000 7f5e13584b80 1 Processor -- start 2024-05-21T21:55:14.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.591+0000 7f5e13584b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a8ced29340 0x55a8ced4c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:14.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.591+0000 7f5e13584b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a8ced2e770 0x55a8ced4cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:14.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.591+0000 7f5e13584b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a8ced2f090 0x55a8ced47910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:14.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.591+0000 7f5e13584b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a8ced43360 con 0x55a8ced2e770 2024-05-21T21:55:14.595 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.591+0000 7f5e13584b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55a8ced42c90 con 0x55a8ced29340 2024-05-21T21:55:14.595 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.591+0000 7f5e13584b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55a8ced43a30 con 0x55a8ced2f090 2024-05-21T21:55:14.595 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.591+0000 7f5e13584b80 10 client.0 _ll_register_callbacks cb 0x55a8cec20c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:55:14.595 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.591+0000 7f5e01400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a8ced2e770 0x55a8ced4cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:14.595 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.592+0000 7f5e01400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a8ced2e770 0x55a8ced4cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:37262/0 (socket says 172.21.15.151:37262) 2024-05-21T21:55:14.595 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.592+0000 7f5e01400700 1 -- 172.21.15.151:0/3494607714 learned_addr learned my addr 172.21.15.151:0/3494607714 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:55:14.595 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.592+0000 7f5e02800700 1 --2- 172.21.15.151:0/3494607714 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a8ced2f090 0x55a8ced47910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:14.595 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.592+0000 7f5e01e00700 1 --2- 172.21.15.151:0/3494607714 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a8ced29340 0x55a8ced4c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:14.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.592+0000 7f5e01400700 1 -- 172.21.15.151:0/3494607714 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a8ced2f090 msgr2=0x55a8ced47910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:55:14.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.592+0000 7f5e01400700 1 --2- 172.21.15.151:0/3494607714 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a8ced2f090 0x55a8ced47910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:14.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.592+0000 7f5e01400700 1 -- 172.21.15.151:0/3494607714 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a8ced29340 msgr2=0x55a8ced4c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:55:14.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.592+0000 7f5e01400700 1 --2- 172.21.15.151:0/3494607714 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a8ced29340 0x55a8ced4c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:14.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.592+0000 7f5e01400700 1 -- 172.21.15.151:0/3494607714 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8ced47e50 con 0x55a8ced2e770 2024-05-21T21:55:14.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.592+0000 7f5e01400700 1 --2- 172.21.15.151:0/3494607714 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a8ced2e770 0x55a8ced4cea0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5dec0a6bf0 tx=0x7f5dec0ea4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fa7dbd20c02a479 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:55:14.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.592+0000 7f5de7e00700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-21T21:55:14.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.592+0000 7f5de7e00700 1 -- 172.21.15.151:0/3494607714 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dec0a5110 con 0x55a8ced2e770 2024-05-21T21:55:14.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.592+0000 7f5de7e00700 1 -- 172.21.15.151:0/3494607714 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f5dec0e5690 con 0x55a8ced2e770 2024-05-21T21:55:14.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.593+0000 7f5e13584b80 1 -- 172.21.15.151:0/3494607714 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5dec0a9490 con 0x55a8ced2e770 2024-05-21T21:55:14.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.593+0000 7f5de7e00700 1 -- 172.21.15.151:0/3494607714 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5dec0add30 con 0x55a8ced2e770 2024-05-21T21:55:14.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.593+0000 7f5e13584b80 10 client.14949 fetch_fsmap learned FSMap version 28 2024-05-21T21:55:14.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.593+0000 7f5e13584b80 1 -- 172.21.15.151:0/3494607714 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a8cedae040 con 0x55a8ced2e770 2024-05-21T21:55:14.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.593+0000 7f5de7e00700 1 -- 172.21.15.151:0/3494607714 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5dec0a9490 con 0x55a8ced2e770 2024-05-21T21:55:14.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.593+0000 7f5de7e00700 1 -- 172.21.15.151:0/3494607714 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f5dec0a9be0 con 0x55a8ced2e770 2024-05-21T21:55:14.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.593+0000 7f5e13584b80 10 client.14949 fetch_fsmap finished waiting for FSMap version 28 2024-05-21T21:55:14.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.593+0000 7f5e13584b80 10 client.14949 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-21T21:55:14.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.593+0000 7f5e13584b80 20 client.14949 populate_metadata read hostname 'smithi151' 2024-05-21T21:55:14.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.593+0000 7f5e13584b80 1 --2- 172.21.15.151:0/3494607714 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x55a8cedb25d0 0x55a8cedb12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:14.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.593+0000 7f5e13584b80 4 client.14949 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-21T21:55:14.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.593+0000 7f5e13584b80 1 -- 172.21.15.151:0/3494607714 --> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] -- command(tid 0: {"prefix": "status"}) -- 0x55a8cec82520 con 0x55a8cedb25d0 2024-05-21T21:55:14.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.594+0000 7f5e01e00700 1 --2- 172.21.15.151:0/3494607714 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x55a8cedb25d0 0x55a8cedb12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:14.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.594+0000 7f5e01e00700 1 --2- 172.21.15.151:0/3494607714 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x55a8cedb25d0 0x55a8cedb12e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c51d1c93e00d17d0 server_cookie=775128b7854d6e2d in_seq=0 out_seq=0 2024-05-21T21:55:14.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.594+0000 7f5de7e00700 10 client.14949 ms_handle_connect on v2:172.21.15.184:6836/2988194888 2024-05-21T21:55:14.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.595+0000 7f5de7e00700 1 -- 172.21.15.151:0/3494607714 <== mds.3 v2:172.21.15.184:6836/2988194888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55a8cec82520 con 0x55a8cedb25d0 2024-05-21T21:55:14.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.595+0000 7f5de7e00700 10 client.14949 handle_command_reply: tid=0 2024-05-21T21:55:14.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.595+0000 7f5e13584b80 1 client.14949 shutdown 2024-05-21T21:55:14.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.595+0000 7f5e13584b80 1 -- 172.21.15.151:0/3494607714 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a8ced2e770 msgr2=0x55a8ced4cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:14.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.595+0000 7f5e13584b80 1 --2- 172.21.15.151:0/3494607714 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a8ced2e770 0x55a8ced4cea0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5dec0a6bf0 tx=0x7f5dec0ea4f0 comp rx=0 tx=0).stop 2024-05-21T21:55:14.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.595+0000 7f5e13584b80 1 -- 172.21.15.151:0/3494607714 shutdown_connections 2024-05-21T21:55:14.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.595+0000 7f5e13584b80 1 --2- 172.21.15.151:0/3494607714 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x55a8cedb25d0 0x55a8cedb12e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:14.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.595+0000 7f5e13584b80 1 --2- 172.21.15.151:0/3494607714 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a8ced29340 0x55a8ced4c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:14.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.595+0000 7f5e13584b80 1 --2- 172.21.15.151:0/3494607714 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a8ced2f090 0x55a8ced47910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:14.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.595+0000 7f5e13584b80 1 --2- 172.21.15.151:0/3494607714 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a8ced2e770 0x55a8ced4cea0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:14.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.595+0000 7f5e13584b80 1 -- 172.21.15.151:0/3494607714 >> 172.21.15.151:0/3494607714 conn(0x55a8cec56b90 msgr2=0x55a8ced33710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:55:14.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.595+0000 7f5de7e00700 0 client.14949 ms_handle_reset on v2:172.21.15.184:6836/2988194888 2024-05-21T21:55:14.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.595+0000 7f5e13584b80 1 -- 172.21.15.151:0/3494607714 shutdown_connections 2024-05-21T21:55:14.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.595+0000 7f5e13584b80 1 -- 172.21.15.151:0/3494607714 wait complete. 2024-05-21T21:55:14.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.595+0000 7f5e13584b80 20 client.14949 trim_cache size 0 max 16384 2024-05-21T21:55:14.601 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-05-21T21:55:14.601 INFO:teuthology.orchestra.run.smithi151.stdout: "cluster_fsid": "c8c7a2e6-17ba-11ef-bc9a-c7b262605968", 2024-05-21T21:55:14.601 INFO:teuthology.orchestra.run.smithi151.stdout: "whoami": 3, 2024-05-21T21:55:14.601 INFO:teuthology.orchestra.run.smithi151.stdout: "id": 24478, 2024-05-21T21:55:14.601 INFO:teuthology.orchestra.run.smithi151.stdout: "want_state": "up:active", 2024-05-21T21:55:14.601 INFO:teuthology.orchestra.run.smithi151.stdout: "state": "up:active", 2024-05-21T21:55:14.601 INFO:teuthology.orchestra.run.smithi151.stdout: "fs_name": "cephfs", 2024-05-21T21:55:14.601 INFO:teuthology.orchestra.run.smithi151.stdout: "rank_uptime": 217.27203756399999, 2024-05-21T21:55:14.601 INFO:teuthology.orchestra.run.smithi151.stdout: "mdsmap_epoch": 28, 2024-05-21T21:55:14.601 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_epoch": 82, 2024-05-21T21:55:14.601 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_epoch_barrier": 80, 2024-05-21T21:55:14.601 INFO:teuthology.orchestra.run.smithi151.stdout: "uptime": 273.470358589 2024-05-21T21:55:14.602 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-05-21T21:55:14.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.596+0000 7f5e03200700 1 -- 172.21.15.151:0/1639540873 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f5de80b2880 msgr2=0x7f5de80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:14.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.596+0000 7f5e03200700 1 --2- 172.21.15.151:0/1639540873 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f5de80b2880 0x7f5de80b4d40 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc00b780 tx=0x7f5dfc006210 comp rx=0 tx=0).stop 2024-05-21T21:55:14.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.596+0000 7f5e03200700 1 -- 172.21.15.151:0/1639540873 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e0410a440 msgr2=0x7f5e041a0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:14.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.596+0000 7f5e03200700 1 --2- 172.21.15.151:0/1639540873 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e0410a440 0x7f5e041a0690 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5dec00e9f0 tx=0x7f5dec00ed00 comp rx=0 tx=0).stop 2024-05-21T21:55:14.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.596+0000 7f5e03200700 1 -- 172.21.15.151:0/1639540873 shutdown_connections 2024-05-21T21:55:14.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.596+0000 7f5e03200700 1 --2- 172.21.15.151:0/1639540873 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f5de80b2880 0x7f5de80b4d40 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:14.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.596+0000 7f5e03200700 1 --2- 172.21.15.151:0/1639540873 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5e0410e950 0x7f5e041a4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:14.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.596+0000 7f5e03200700 1 --2- 172.21.15.151:0/1639540873 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5e04109a80 0x7f5e041a0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:14.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.596+0000 7f5e03200700 1 --2- 172.21.15.151:0/1639540873 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e0410a440 0x7f5e041a0690 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:14.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.596+0000 7f5e03200700 1 -- 172.21.15.151:0/1639540873 >> 172.21.15.151:0/1639540873 conn(0x7f5e040792d0 msgr2=0x7f5e04079930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:55:14.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.596+0000 7f5e03200700 1 -- 172.21.15.151:0/1639540873 shutdown_connections 2024-05-21T21:55:14.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:14.596+0000 7f5e03200700 1 -- 172.21.15.151:0/1639540873 wait complete. 2024-05-21T21:55:14.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:14 smithi164 ceph-mon[36269]: pgmap v434: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-05-21T21:55:14.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:14 smithi184 ceph-mon[36942]: pgmap v434: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-05-21T21:55:15.384 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.h status 2024-05-21T21:55:15.843 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:55:16.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:16 smithi151 ceph-mon[31623]: pgmap v435: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-05-21T21:55:16.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:16 smithi164 ceph-mon[36269]: pgmap v435: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-05-21T21:55:16.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:16 smithi184 ceph-mon[36942]: pgmap v435: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-05-21T21:55:17.927 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.922+0000 7f1e24e00700 1 -- 172.21.15.151:0/1812371240 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e20106030 msgr2=0x7f1e20112540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:17.927 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.922+0000 7f1e24e00700 1 --2- 172.21.15.151:0/1812371240 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e20106030 0x7f1e20112540 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1e18009f90 tx=0x7f1e1800b330 comp rx=0 tx=0).stop 2024-05-21T21:55:17.928 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.923+0000 7f1e24e00700 1 -- 172.21.15.151:0/1812371240 shutdown_connections 2024-05-21T21:55:17.928 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.923+0000 7f1e24e00700 1 --2- 172.21.15.151:0/1812371240 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e20106030 0x7f1e20112540 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:17.928 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.923+0000 7f1e24e00700 1 --2- 172.21.15.151:0/1812371240 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1e2010b670 0x7f1e2010ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:17.928 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.923+0000 7f1e24e00700 1 --2- 172.21.15.151:0/1812371240 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e20105670 0x7f1e20105af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:17.928 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.923+0000 7f1e24e00700 1 -- 172.21.15.151:0/1812371240 >> 172.21.15.151:0/1812371240 conn(0x7f1e20101160 msgr2=0x7f1e20103580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:55:17.928 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.923+0000 7f1e24e00700 1 -- 172.21.15.151:0/1812371240 shutdown_connections 2024-05-21T21:55:17.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.923+0000 7f1e24e00700 1 -- 172.21.15.151:0/1812371240 wait complete. 2024-05-21T21:55:17.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.924+0000 7f1e24e00700 1 Processor -- start 2024-05-21T21:55:17.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.924+0000 7f1e24e00700 1 -- start start 2024-05-21T21:55:17.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.924+0000 7f1e24e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e20105670 0x7f1e201a18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:17.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.924+0000 7f1e24e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1e20106030 0x7f1e201a1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:17.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.925+0000 7f1e24e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e2010b670 0x7f1e2019ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:17.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.925+0000 7f1e24e00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1e201158b0 con 0x7f1e2010b670 2024-05-21T21:55:17.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.925+0000 7f1e24e00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1e20115730 con 0x7f1e20105670 2024-05-21T21:55:17.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.925+0000 7f1e24e00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1e20115a30 con 0x7f1e20106030 2024-05-21T21:55:17.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.925+0000 7f1e1f400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e20105670 0x7f1e201a18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:17.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.925+0000 7f1e1f400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e20105670 0x7f1e201a18b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:56422/0 (socket says 172.21.15.151:56422) 2024-05-21T21:55:17.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.925+0000 7f1e1f400700 1 -- 172.21.15.151:0/340885932 learned_addr learned my addr 172.21.15.151:0/340885932 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:55:17.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.925+0000 7f1e17e00700 1 --2- 172.21.15.151:0/340885932 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1e20106030 0x7f1e201a1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:17.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.925+0000 7f1e1fe00700 1 --2- 172.21.15.151:0/340885932 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e2010b670 0x7f1e2019ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:17.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.925+0000 7f1e1f400700 1 -- 172.21.15.151:0/340885932 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1e20106030 msgr2=0x7f1e201a1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:17.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.925+0000 7f1e1f400700 1 --2- 172.21.15.151:0/340885932 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1e20106030 0x7f1e201a1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:17.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.925+0000 7f1e1f400700 1 -- 172.21.15.151:0/340885932 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e2010b670 msgr2=0x7f1e2019ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:17.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.925+0000 7f1e1f400700 1 --2- 172.21.15.151:0/340885932 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e2010b670 0x7f1e2019ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:17.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.925+0000 7f1e1f400700 1 -- 172.21.15.151:0/340885932 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e2019c1d0 con 0x7f1e20105670 2024-05-21T21:55:17.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.925+0000 7f1e1fe00700 1 --2- 172.21.15.151:0/340885932 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e2010b670 0x7f1e2019ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T21:55:17.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.926+0000 7f1e1f400700 1 --2- 172.21.15.151:0/340885932 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e20105670 0x7f1e201a18b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1e1000e9a0 tx=0x7f1e1000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:55:17.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.926+0000 7f1e1cc00700 1 -- 172.21.15.151:0/340885932 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e1000cbd0 con 0x7f1e20105670 2024-05-21T21:55:17.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.926+0000 7f1e1cc00700 1 -- 172.21.15.151:0/340885932 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e1000cd70 con 0x7f1e20105670 2024-05-21T21:55:17.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.926+0000 7f1e24e00700 1 -- 172.21.15.151:0/340885932 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e2019c4c0 con 0x7f1e20105670 2024-05-21T21:55:17.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.926+0000 7f1e1cc00700 1 -- 172.21.15.151:0/340885932 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e10018680 con 0x7f1e20105670 2024-05-21T21:55:17.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.926+0000 7f1e24e00700 1 -- 172.21.15.151:0/340885932 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e201a8780 con 0x7f1e20105670 2024-05-21T21:55:17.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.929+0000 7f1e1cc00700 1 -- 172.21.15.151:0/340885932 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f1e10010430 con 0x7f1e20105670 2024-05-21T21:55:17.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.931+0000 7f1e30e7fb80 1 Processor -- start 2024-05-21T21:55:17.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.931+0000 7f1e1cc00700 1 --2- 172.21.15.151:0/340885932 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f1e040b2900 0x7f1e040b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:17.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.931+0000 7f1e30e7fb80 1 -- start start 2024-05-21T21:55:17.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.931+0000 7f1e1cc00700 1 -- 172.21.15.151:0/340885932 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f1e100e23f0 con 0x7f1e20105670 2024-05-21T21:55:17.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.931+0000 7f1e1fe00700 1 --2- 172.21.15.151:0/340885932 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f1e040b2900 0x7f1e040b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:17.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.931+0000 7f1e30e7fb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aa99695770 0x55aa99695b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:17.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.931+0000 7f1e30e7fb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aa99696090 0x55aa9968fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:17.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.931+0000 7f1e30e7fb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aa99690340 0x55aa996907a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:17.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.931+0000 7f1e30e7fb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55aa995eddf0 con 0x55aa99690340 2024-05-21T21:55:17.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.931+0000 7f1e30e7fb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55aa995dd990 con 0x55aa99696090 2024-05-21T21:55:17.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.931+0000 7f1e30e7fb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55aa995e19d0 con 0x55aa99695770 2024-05-21T21:55:17.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.932+0000 7f1e1fe00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aa99690340 0x55aa996907a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:17.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.932+0000 7f1e1fe00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aa99690340 0x55aa996907a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:56036/0 (socket says 172.21.15.151:56036) 2024-05-21T21:55:17.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.932+0000 7f1e1fe00700 1 -- 172.21.15.151:0/2203134423 learned_addr learned my addr 172.21.15.151:0/2203134423 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:55:17.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.932+0000 7f1e17e00700 1 --2- 172.21.15.151:0/2203134423 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aa99696090 0x55aa9968fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:17.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.932+0000 7f1e1f400700 1 --2- 172.21.15.151:0/2203134423 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aa99695770 0x55aa99695b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:17.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.932+0000 7f1e17e00700 1 -- 172.21.15.151:0/2203134423 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aa99695770 msgr2=0x55aa99695b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:55:17.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.932+0000 7f1e17e00700 1 --2- 172.21.15.151:0/2203134423 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aa99695770 0x55aa99695b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:17.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.932+0000 7f1e17e00700 1 -- 172.21.15.151:0/2203134423 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aa99690340 msgr2=0x55aa996907a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:55:17.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.932+0000 7f1e17e00700 1 --2- 172.21.15.151:0/2203134423 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aa99690340 0x55aa996907a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:17.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.932+0000 7f1e17e00700 1 -- 172.21.15.151:0/2203134423 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa995a63d0 con 0x55aa99696090 2024-05-21T21:55:17.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.932+0000 7f1e1fe00700 1 --2- 172.21.15.151:0/340885932 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f1e040b2900 0x7f1e040b4dc0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1e1800ca00 tx=0x7f1e18003c90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:55:17.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.933+0000 7f1e17e00700 1 --2- 172.21.15.151:0/2203134423 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aa99696090 0x55aa9968fd20 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1e08005fd0 tx=0x7f1e08011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35f344e71494da69 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:55:17.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.933+0000 7f1e14c00700 1 -- 172.21.15.151:0/2203134423 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e08004250 con 0x55aa99696090 2024-05-21T21:55:17.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.933+0000 7f1e14c00700 1 -- 172.21.15.151:0/2203134423 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e080043f0 con 0x55aa99696090 2024-05-21T21:55:17.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.933+0000 7f1e14c00700 1 -- 172.21.15.151:0/2203134423 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e08019810 con 0x55aa99696090 2024-05-21T21:55:17.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.934+0000 7f1e30e7fb80 1 -- 172.21.15.151:0/2203134423 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aa99696090 msgr2=0x55aa9968fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:17.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.934+0000 7f1e30e7fb80 1 --2- 172.21.15.151:0/2203134423 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aa99696090 0x55aa9968fd20 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1e08005fd0 tx=0x7f1e08011720 comp rx=0 tx=0).stop 2024-05-21T21:55:17.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.934+0000 7f1e30e7fb80 1 -- 172.21.15.151:0/2203134423 shutdown_connections 2024-05-21T21:55:17.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.934+0000 7f1e30e7fb80 1 --2- 172.21.15.151:0/2203134423 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aa99690340 0x55aa996907a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:17.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.934+0000 7f1e30e7fb80 1 --2- 172.21.15.151:0/2203134423 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aa99695770 0x55aa99695b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:17.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.934+0000 7f1e30e7fb80 1 --2- 172.21.15.151:0/2203134423 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aa99696090 0x55aa9968fd20 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:17.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.934+0000 7f1e30e7fb80 1 -- 172.21.15.151:0/2203134423 >> 172.21.15.151:0/2203134423 conn(0x55aa995bdb90 msgr2=0x55aa9968a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:55:17.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.934+0000 7f1e30e7fb80 1 -- 172.21.15.151:0/2203134423 shutdown_connections 2024-05-21T21:55:17.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.934+0000 7f1e30e7fb80 1 -- 172.21.15.151:0/2203134423 wait complete. 2024-05-21T21:55:17.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.935+0000 7f1e30e7fb80 1 -- start start 2024-05-21T21:55:17.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.935+0000 7f1e30e7fb80 1 Processor -- start 2024-05-21T21:55:17.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.935+0000 7f1e30e7fb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aa99690340 0x55aa9969bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:17.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.935+0000 7f1e30e7fb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aa99695770 0x55aa996991a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:17.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.935+0000 7f1e30e7fb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aa99696090 0x55aa996996e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:17.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.935+0000 7f1e30e7fb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55aa994e3f50 con 0x55aa99696090 2024-05-21T21:55:17.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.935+0000 7f1e30e7fb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55aa9957e710 con 0x55aa99690340 2024-05-21T21:55:17.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.935+0000 7f1e30e7fb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55aa9969cbb0 con 0x55aa99695770 2024-05-21T21:55:17.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.936+0000 7f1e30e7fb80 10 client.0 _ll_register_callbacks cb 0x55aa99587c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:55:17.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.936+0000 7f1e1f400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aa99690340 0x55aa9969bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:17.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.936+0000 7f1e1f400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aa99690340 0x55aa9969bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:56444/0 (socket says 172.21.15.151:56444) 2024-05-21T21:55:17.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.936+0000 7f1e1fe00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aa99696090 0x55aa996996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:17.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.936+0000 7f1e1f400700 1 -- 172.21.15.151:0/2139434221 learned_addr learned my addr 172.21.15.151:0/2139434221 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:55:17.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.936+0000 7f1e17e00700 1 --2- 172.21.15.151:0/2139434221 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aa99695770 0x55aa996991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:17.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.936+0000 7f1e1fe00700 1 -- 172.21.15.151:0/2139434221 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aa99695770 msgr2=0x55aa996991a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:55:17.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.936+0000 7f1e1fe00700 1 --2- 172.21.15.151:0/2139434221 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aa99695770 0x55aa996991a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:17.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.936+0000 7f1e1fe00700 1 -- 172.21.15.151:0/2139434221 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aa99690340 msgr2=0x55aa9969bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:55:17.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.936+0000 7f1e1fe00700 1 --2- 172.21.15.151:0/2139434221 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aa99690340 0x55aa9969bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:17.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.936+0000 7f1e1fe00700 1 -- 172.21.15.151:0/2139434221 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa9952d880 con 0x55aa99696090 2024-05-21T21:55:17.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.936+0000 7f1e1f400700 1 --2- 172.21.15.151:0/2139434221 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aa99690340 0x55aa9969bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:55:17.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.937+0000 7f1e1fe00700 1 --2- 172.21.15.151:0/2139434221 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aa99696090 0x55aa996996e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1e18009f90 tx=0x7f1e1800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c4faf9a95e68981 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:55:17.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.937+0000 7f1e0d600700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-21T21:55:17.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.937+0000 7f1e0d600700 1 -- 172.21.15.151:0/2139434221 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e180076f0 con 0x55aa99696090 2024-05-21T21:55:17.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.937+0000 7f1e0d600700 1 -- 172.21.15.151:0/2139434221 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f1e18026040 con 0x55aa99696090 2024-05-21T21:55:17.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.937+0000 7f1e30e7fb80 1 -- 172.21.15.151:0/2139434221 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e080043f0 con 0x55aa99696090 2024-05-21T21:55:17.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.937+0000 7f1e0d600700 1 -- 172.21.15.151:0/2139434221 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1e18007890 con 0x55aa99696090 2024-05-21T21:55:17.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.937+0000 7f1e30e7fb80 10 client.14961 fetch_fsmap learned FSMap version 28 2024-05-21T21:55:17.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.937+0000 7f1e30e7fb80 1 -- 172.21.15.151:0/2139434221 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55aa995ae010 con 0x55aa99696090 2024-05-21T21:55:17.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.937+0000 7f1e0d600700 1 -- 172.21.15.151:0/2139434221 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e080043f0 con 0x55aa99696090 2024-05-21T21:55:17.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.938+0000 7f1e0d600700 1 -- 172.21.15.151:0/2139434221 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f1e18034020 con 0x55aa99696090 2024-05-21T21:55:17.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.938+0000 7f1e30e7fb80 10 client.14961 fetch_fsmap finished waiting for FSMap version 28 2024-05-21T21:55:17.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.938+0000 7f1e30e7fb80 10 client.14961 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-21T21:55:17.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.938+0000 7f1e30e7fb80 20 client.14961 populate_metadata read hostname 'smithi151' 2024-05-21T21:55:17.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.938+0000 7f1e30e7fb80 1 --2- 172.21.15.151:0/2139434221 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x55aa996b1f60 0x55aa996b33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:17.942 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.938+0000 7f1e30e7fb80 4 client.14961 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T21:55:17.942 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.938+0000 7f1e30e7fb80 1 -- 172.21.15.151:0/2139434221 --> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aa995efd40 con 0x55aa996b1f60 2024-05-21T21:55:17.942 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.938+0000 7f1e1f400700 1 --2- 172.21.15.151:0/2139434221 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x55aa996b1f60 0x55aa996b33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:17.943 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.939+0000 7f1e1f400700 1 --2- 172.21.15.151:0/2139434221 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x55aa996b1f60 0x55aa996b33d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e4428196838d3547 server_cookie=7116728fad1e4839 in_seq=0 out_seq=0 2024-05-21T21:55:17.943 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.939+0000 7f1e0d600700 10 client.14961 ms_handle_connect on v2:172.21.15.164:6836/1554609572 2024-05-21T21:55:17.943 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.940+0000 7f1e0d600700 1 -- 172.21.15.151:0/2139434221 <== mds.2 v2:172.21.15.164:6836/1554609572 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55aa995efd40 con 0x55aa996b1f60 2024-05-21T21:55:17.943 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.940+0000 7f1e0d600700 10 client.14961 handle_command_reply: tid=0 2024-05-21T21:55:17.943 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.940+0000 7f1e30e7fb80 1 client.14961 shutdown 2024-05-21T21:55:17.943 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.940+0000 7f1e30e7fb80 1 -- 172.21.15.151:0/2139434221 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aa99696090 msgr2=0x55aa996996e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:17.943 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.940+0000 7f1e30e7fb80 1 --2- 172.21.15.151:0/2139434221 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aa99696090 0x55aa996996e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1e18009f90 tx=0x7f1e1800b330 comp rx=0 tx=0).stop 2024-05-21T21:55:17.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.940+0000 7f1e30e7fb80 1 -- 172.21.15.151:0/2139434221 shutdown_connections 2024-05-21T21:55:17.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.940+0000 7f1e30e7fb80 1 --2- 172.21.15.151:0/2139434221 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x55aa996b1f60 0x55aa996b33d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:17.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.940+0000 7f1e30e7fb80 1 --2- 172.21.15.151:0/2139434221 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aa99690340 0x55aa9969bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:17.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.940+0000 7f1e30e7fb80 1 --2- 172.21.15.151:0/2139434221 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aa99696090 0x55aa996996e0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:17.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.940+0000 7f1e0d600700 0 client.14961 ms_handle_reset on v2:172.21.15.164:6836/1554609572 2024-05-21T21:55:17.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.940+0000 7f1e30e7fb80 1 --2- 172.21.15.151:0/2139434221 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aa99695770 0x55aa996991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:17.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.940+0000 7f1e30e7fb80 1 -- 172.21.15.151:0/2139434221 >> 172.21.15.151:0/2139434221 conn(0x55aa995bdb90 msgr2=0x55aa99689f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:55:17.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.940+0000 7f1e30e7fb80 1 -- 172.21.15.151:0/2139434221 shutdown_connections 2024-05-21T21:55:17.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.940+0000 7f1e30e7fb80 1 -- 172.21.15.151:0/2139434221 wait complete. 2024-05-21T21:55:17.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.941+0000 7f1e30e7fb80 20 client.14961 trim_cache size 0 max 16384 2024-05-21T21:55:17.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.963+0000 7f1e30e7fb80 1 Processor -- start 2024-05-21T21:55:17.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.964+0000 7f1e30e7fb80 1 -- start start 2024-05-21T21:55:17.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.964+0000 7f1e30e7fb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aa99690340 0x55aa9968fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:17.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.964+0000 7f1e30e7fb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aa99695770 0x55aa9968a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:17.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.964+0000 7f1e30e7fb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aa99696090 0x55aa99698f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:17.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.964+0000 7f1e30e7fb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55aa996b1710 con 0x55aa99696090 2024-05-21T21:55:17.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.964+0000 7f1e30e7fb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55aa996b1de0 con 0x55aa99695770 2024-05-21T21:55:17.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.964+0000 7f1e30e7fb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55aa996b24b0 con 0x55aa99690340 2024-05-21T21:55:17.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.964+0000 7f1e1f400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aa99690340 0x55aa9968fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:17.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.964+0000 7f1e1fe00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aa99696090 0x55aa99698f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:17.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.964+0000 7f1e1f400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aa99690340 0x55aa9968fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:42692/0 (socket says 172.21.15.151:42692) 2024-05-21T21:55:17.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.964+0000 7f1e1f400700 1 -- 172.21.15.151:0/682323645 learned_addr learned my addr 172.21.15.151:0/682323645 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:55:17.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.964+0000 7f1e17e00700 1 --2- 172.21.15.151:0/682323645 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aa99695770 0x55aa9968a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:17.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.965+0000 7f1e1f400700 1 -- 172.21.15.151:0/682323645 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aa99695770 msgr2=0x55aa9968a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:55:17.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.965+0000 7f1e1f400700 1 --2- 172.21.15.151:0/682323645 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aa99695770 0x55aa9968a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:17.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.965+0000 7f1e1f400700 1 -- 172.21.15.151:0/682323645 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aa99696090 msgr2=0x55aa99698f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:55:17.970 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.965+0000 7f1e1f400700 1 --2- 172.21.15.151:0/682323645 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aa99696090 0x55aa99698f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:17.970 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.965+0000 7f1e1f400700 1 -- 172.21.15.151:0/682323645 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa996b4330 con 0x55aa99690340 2024-05-21T21:55:17.970 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.965+0000 7f1e1f400700 1 --2- 172.21.15.151:0/682323645 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aa99690340 0x55aa9968fd20 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1e1000c5b0 tx=0x7f1e100dfd80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1f6df7865ba5822 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:55:17.970 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.965+0000 7f1e0fe00700 1 -- 172.21.15.151:0/682323645 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e100c3070 con 0x55aa99690340 2024-05-21T21:55:17.970 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.965+0000 7f1e0fe00700 1 -- 172.21.15.151:0/682323645 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e100b2170 con 0x55aa99690340 2024-05-21T21:55:17.970 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.965+0000 7f1e0fe00700 1 -- 172.21.15.151:0/682323645 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e1008e030 con 0x55aa99690340 2024-05-21T21:55:17.971 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.966+0000 7f1e30e7fb80 1 -- 172.21.15.151:0/682323645 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aa99690340 msgr2=0x55aa9968fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:17.971 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.966+0000 7f1e30e7fb80 1 --2- 172.21.15.151:0/682323645 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aa99690340 0x55aa9968fd20 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1e1000c5b0 tx=0x7f1e100dfd80 comp rx=0 tx=0).stop 2024-05-21T21:55:17.971 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.966+0000 7f1e30e7fb80 1 -- 172.21.15.151:0/682323645 shutdown_connections 2024-05-21T21:55:17.971 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.966+0000 7f1e30e7fb80 1 --2- 172.21.15.151:0/682323645 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aa99696090 0x55aa99698f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:17.971 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.966+0000 7f1e30e7fb80 1 --2- 172.21.15.151:0/682323645 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aa99690340 0x55aa9968fd20 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:17.971 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.966+0000 7f1e30e7fb80 1 --2- 172.21.15.151:0/682323645 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aa99695770 0x55aa9968a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:17.971 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.966+0000 7f1e30e7fb80 1 -- 172.21.15.151:0/682323645 >> 172.21.15.151:0/682323645 conn(0x55aa995bdb90 msgr2=0x55aa99697a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:55:17.972 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.966+0000 7f1e30e7fb80 1 -- 172.21.15.151:0/682323645 shutdown_connections 2024-05-21T21:55:17.972 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.966+0000 7f1e30e7fb80 1 -- 172.21.15.151:0/682323645 wait complete. 2024-05-21T21:55:17.972 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.966+0000 7f1e30e7fb80 1 -- start start 2024-05-21T21:55:17.972 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.967+0000 7f1e30e7fb80 1 Processor -- start 2024-05-21T21:55:17.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.967+0000 7f1e30e7fb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aa99690340 0x55aa99689ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:17.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.967+0000 7f1e30e7fb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aa99695770 0x55aa9968a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:17.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.967+0000 7f1e30e7fb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aa99696090 0x55aa9968a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:17.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.967+0000 7f1e30e7fb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55aa996b3250 con 0x55aa99690340 2024-05-21T21:55:17.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.967+0000 7f1e30e7fb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55aa996b2b80 con 0x55aa99695770 2024-05-21T21:55:17.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.967+0000 7f1e30e7fb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55aa996b3920 con 0x55aa99696090 2024-05-21T21:55:17.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.967+0000 7f1e30e7fb80 10 client.0 _ll_register_callbacks cb 0x55aa99587c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:55:17.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.968+0000 7f1e17e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aa99695770 0x55aa9968a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:17.979 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.968+0000 7f1e17e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aa99695770 0x55aa9968a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:56462/0 (socket says 172.21.15.151:56462) 2024-05-21T21:55:17.979 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.968+0000 7f1e17e00700 1 -- 172.21.15.151:0/761217784 learned_addr learned my addr 172.21.15.151:0/761217784 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:55:17.979 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.968+0000 7f1e1f400700 1 --2- 172.21.15.151:0/761217784 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aa99690340 0x55aa99689ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:17.979 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.968+0000 7f1e1fe00700 1 --2- 172.21.15.151:0/761217784 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aa99696090 0x55aa9968a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:17.979 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.968+0000 7f1e17e00700 1 -- 172.21.15.151:0/761217784 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aa99696090 msgr2=0x55aa9968a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:55:17.979 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.968+0000 7f1e17e00700 1 --2- 172.21.15.151:0/761217784 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aa99696090 0x55aa9968a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:17.979 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.968+0000 7f1e17e00700 1 -- 172.21.15.151:0/761217784 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aa99690340 msgr2=0x55aa99689ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:55:17.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.968+0000 7f1e17e00700 1 --2- 172.21.15.151:0/761217784 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aa99690340 0x55aa99689ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:17.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.968+0000 7f1e17e00700 1 -- 172.21.15.151:0/761217784 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa996bb1c0 con 0x55aa99695770 2024-05-21T21:55:17.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.968+0000 7f1e1f400700 1 --2- 172.21.15.151:0/761217784 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aa99690340 0x55aa99689ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:55:17.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.968+0000 7f1e17e00700 1 --2- 172.21.15.151:0/761217784 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aa99695770 0x55aa9968a3e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1e08009670 tx=0x7f1e080055c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae70cdbf25e6f655 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:55:17.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.968+0000 7f1e0d600700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-21T21:55:17.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.969+0000 7f1e0d600700 1 -- 172.21.15.151:0/761217784 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e08003890 con 0x55aa99695770 2024-05-21T21:55:17.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.969+0000 7f1e30e7fb80 1 -- 172.21.15.151:0/761217784 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55aa995e9520 con 0x55aa99695770 2024-05-21T21:55:17.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.969+0000 7f1e0d600700 1 -- 172.21.15.151:0/761217784 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f1e08003a30 con 0x55aa99695770 2024-05-21T21:55:17.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.969+0000 7f1e0d600700 1 -- 172.21.15.151:0/761217784 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1e0801a440 con 0x55aa99695770 2024-05-21T21:55:17.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.969+0000 7f1e30e7fb80 10 client.25088 fetch_fsmap learned FSMap version 28 2024-05-21T21:55:17.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.969+0000 7f1e30e7fb80 1 -- 172.21.15.151:0/761217784 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55aa9971dc40 con 0x55aa99695770 2024-05-21T21:55:17.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.969+0000 7f1e0d600700 1 -- 172.21.15.151:0/761217784 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55aa995e9520 con 0x55aa99695770 2024-05-21T21:55:17.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.970+0000 7f1e0d600700 1 -- 172.21.15.151:0/761217784 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f1e08018070 con 0x55aa99695770 2024-05-21T21:55:17.982 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.970+0000 7f1e30e7fb80 10 client.25088 fetch_fsmap finished waiting for FSMap version 28 2024-05-21T21:55:17.982 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.970+0000 7f1e30e7fb80 10 client.25088 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-21T21:55:17.982 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.970+0000 7f1e30e7fb80 20 client.25088 populate_metadata read hostname 'smithi151' 2024-05-21T21:55:17.982 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.970+0000 7f1e30e7fb80 1 --2- 172.21.15.151:0/761217784 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x55aa997225b0 0x55aa997212c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:17.982 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.970+0000 7f1e30e7fb80 4 client.25088 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-21T21:55:17.982 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.970+0000 7f1e30e7fb80 1 -- 172.21.15.151:0/761217784 --> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] -- command(tid 0: {"prefix": "status"}) -- 0x55aa99460200 con 0x55aa997225b0 2024-05-21T21:55:17.982 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.970+0000 7f1e1f400700 1 --2- 172.21.15.151:0/761217784 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x55aa997225b0 0x55aa997212c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:17.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.970+0000 7f1e0d600700 10 client.25088 ms_handle_connect on v2:172.21.15.164:6836/1554609572 2024-05-21T21:55:17.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.970+0000 7f1e1f400700 1 --2- 172.21.15.151:0/761217784 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x55aa997225b0 0x55aa997212c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=96956fa0ff540d8b server_cookie=2cd2527c95023ec3 in_seq=0 out_seq=0 2024-05-21T21:55:17.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.971+0000 7f1e0d600700 1 -- 172.21.15.151:0/761217784 <== mds.2 v2:172.21.15.164:6836/1554609572 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55aa99460200 con 0x55aa997225b0 2024-05-21T21:55:17.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.971+0000 7f1e0d600700 10 client.25088 handle_command_reply: tid=0 2024-05-21T21:55:17.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.971+0000 7f1e30e7fb80 1 client.25088 shutdown 2024-05-21T21:55:17.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.971+0000 7f1e30e7fb80 1 -- 172.21.15.151:0/761217784 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aa99695770 msgr2=0x55aa9968a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:17.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.971+0000 7f1e30e7fb80 1 --2- 172.21.15.151:0/761217784 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aa99695770 0x55aa9968a3e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1e08009670 tx=0x7f1e080055c0 comp rx=0 tx=0).stop 2024-05-21T21:55:17.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.971+0000 7f1e30e7fb80 1 -- 172.21.15.151:0/761217784 shutdown_connections 2024-05-21T21:55:17.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.971+0000 7f1e30e7fb80 1 --2- 172.21.15.151:0/761217784 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x55aa997225b0 0x55aa997212c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:17.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.971+0000 7f1e30e7fb80 1 --2- 172.21.15.151:0/761217784 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aa99696090 0x55aa9968a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:17.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.971+0000 7f1e30e7fb80 1 --2- 172.21.15.151:0/761217784 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55aa99690340 0x55aa99689ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:17.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.971+0000 7f1e30e7fb80 1 --2- 172.21.15.151:0/761217784 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55aa99695770 0x55aa9968a3e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:17.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.971+0000 7f1e0d600700 0 client.25088 ms_handle_reset on v2:172.21.15.164:6836/1554609572 2024-05-21T21:55:17.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.971+0000 7f1e30e7fb80 1 -- 172.21.15.151:0/761217784 >> 172.21.15.151:0/761217784 conn(0x55aa995bdb90 msgr2=0x55aa99696470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:55:17.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.971+0000 7f1e30e7fb80 1 -- 172.21.15.151:0/761217784 shutdown_connections 2024-05-21T21:55:17.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.971+0000 7f1e30e7fb80 1 -- 172.21.15.151:0/761217784 wait complete. 2024-05-21T21:55:17.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.971+0000 7f1e30e7fb80 20 client.25088 trim_cache size 0 max 16384 2024-05-21T21:55:17.985 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-05-21T21:55:17.986 INFO:teuthology.orchestra.run.smithi151.stdout: "cluster_fsid": "c8c7a2e6-17ba-11ef-bc9a-c7b262605968", 2024-05-21T21:55:17.986 INFO:teuthology.orchestra.run.smithi151.stdout: "whoami": 2, 2024-05-21T21:55:17.986 INFO:teuthology.orchestra.run.smithi151.stdout: "id": 24491, 2024-05-21T21:55:17.986 INFO:teuthology.orchestra.run.smithi151.stdout: "want_state": "up:active", 2024-05-21T21:55:17.986 INFO:teuthology.orchestra.run.smithi151.stdout: "state": "up:active", 2024-05-21T21:55:17.986 INFO:teuthology.orchestra.run.smithi151.stdout: "fs_name": "cephfs", 2024-05-21T21:55:17.987 INFO:teuthology.orchestra.run.smithi151.stdout: "rank_uptime": 221.664659296, 2024-05-21T21:55:17.987 INFO:teuthology.orchestra.run.smithi151.stdout: "mdsmap_epoch": 28, 2024-05-21T21:55:17.987 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_epoch": 82, 2024-05-21T21:55:17.987 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_epoch_barrier": 80, 2024-05-21T21:55:17.987 INFO:teuthology.orchestra.run.smithi151.stdout: "uptime": 274.08135602099998 2024-05-21T21:55:17.987 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-05-21T21:55:17.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.972+0000 7f1e24e00700 1 -- 172.21.15.151:0/340885932 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f1e040b2900 msgr2=0x7f1e040b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:17.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.972+0000 7f1e24e00700 1 --2- 172.21.15.151:0/340885932 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f1e040b2900 0x7f1e040b4dc0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1e1800ca00 tx=0x7f1e18003c90 comp rx=0 tx=0).stop 2024-05-21T21:55:17.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.972+0000 7f1e24e00700 1 -- 172.21.15.151:0/340885932 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e20105670 msgr2=0x7f1e201a18b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:17.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.972+0000 7f1e24e00700 1 --2- 172.21.15.151:0/340885932 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e20105670 0x7f1e201a18b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1e1000e9a0 tx=0x7f1e1000ecb0 comp rx=0 tx=0).stop 2024-05-21T21:55:17.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.972+0000 7f1e24e00700 1 -- 172.21.15.151:0/340885932 shutdown_connections 2024-05-21T21:55:17.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.972+0000 7f1e24e00700 1 --2- 172.21.15.151:0/340885932 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f1e040b2900 0x7f1e040b4dc0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:17.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.972+0000 7f1e24e00700 1 --2- 172.21.15.151:0/340885932 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e20105670 0x7f1e201a18b0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:17.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.972+0000 7f1e24e00700 1 --2- 172.21.15.151:0/340885932 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e2010b670 0x7f1e2019ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:17.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.972+0000 7f1e24e00700 1 --2- 172.21.15.151:0/340885932 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1e20106030 0x7f1e201a1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:17.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.972+0000 7f1e24e00700 1 -- 172.21.15.151:0/340885932 >> 172.21.15.151:0/340885932 conn(0x7f1e20101160 msgr2=0x7f1e201017e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:55:17.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.972+0000 7f1e24e00700 1 -- 172.21.15.151:0/340885932 shutdown_connections 2024-05-21T21:55:17.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:17.972+0000 7f1e24e00700 1 -- 172.21.15.151:0/340885932 wait complete. 2024-05-21T21:55:18.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:18 smithi151 ceph-mon[31623]: pgmap v436: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-05-21T21:55:18.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:18 smithi164 ceph-mon[36269]: pgmap v436: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-05-21T21:55:18.886 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.i status 2024-05-21T21:55:18.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:18 smithi184 ceph-mon[36942]: pgmap v436: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-05-21T21:55:19.338 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:55:20.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:20 smithi151 ceph-mon[31623]: pgmap v437: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-05-21T21:55:20.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:20 smithi164 ceph-mon[36269]: pgmap v437: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-05-21T21:55:20.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:20 smithi184 ceph-mon[36942]: pgmap v437: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-05-21T21:55:21.561 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.561+0000 7f2f91e00700 1 -- 172.21.15.151:0/4082148855 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2f8c076090 msgr2=0x7f2f8c10fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:21.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.561+0000 7f2f91e00700 1 --2- 172.21.15.151:0/4082148855 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2f8c076090 0x7f2f8c10fa00 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2f78009990 tx=0x7f2f78009ca0 comp rx=0 tx=0).stop 2024-05-21T21:55:21.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.561+0000 7f2f91e00700 1 -- 172.21.15.151:0/4082148855 shutdown_connections 2024-05-21T21:55:21.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.561+0000 7f2f91e00700 1 --2- 172.21.15.151:0/4082148855 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f8c075770 0x7f2f8c075b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:21.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.561+0000 7f2f91e00700 1 --2- 172.21.15.151:0/4082148855 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2f8c10ff40 0x7f2f8c112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:21.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.561+0000 7f2f91e00700 1 --2- 172.21.15.151:0/4082148855 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2f8c076090 0x7f2f8c10fa00 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:21.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.561+0000 7f2f91e00700 1 -- 172.21.15.151:0/4082148855 >> 172.21.15.151:0/4082148855 conn(0x7f2f8c07a6a0 msgr2=0x7f2f8c07aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:55:21.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.561+0000 7f2f91e00700 1 -- 172.21.15.151:0/4082148855 shutdown_connections 2024-05-21T21:55:21.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.561+0000 7f2f91e00700 1 -- 172.21.15.151:0/4082148855 wait complete. 2024-05-21T21:55:21.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.562+0000 7f2f91e00700 1 Processor -- start 2024-05-21T21:55:21.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.562+0000 7f2f91e00700 1 -- start start 2024-05-21T21:55:21.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.562+0000 7f2f91e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f8c075770 0x7f2f8c19be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:21.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.562+0000 7f2f91e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2f8c076090 0x7f2f8c19c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:21.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.562+0000 7f2f91e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2f8c10ff40 0x7f2f8c1a0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:21.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.563+0000 7f2f91e00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2f8c115920 con 0x7f2f8c076090 2024-05-21T21:55:21.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.563+0000 7f2f91e00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2f8c1157a0 con 0x7f2f8c075770 2024-05-21T21:55:21.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.563+0000 7f2f91e00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2f8c115aa0 con 0x7f2f8c10ff40 2024-05-21T21:55:21.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.563+0000 7f2f8a000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2f8c076090 0x7f2f8c19c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:21.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.563+0000 7f2f8aa00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f8c075770 0x7f2f8c19be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:21.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.563+0000 7f2f8a000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2f8c076090 0x7f2f8c19c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:56086/0 (socket says 172.21.15.151:56086) 2024-05-21T21:55:21.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.563+0000 7f2f8a000700 1 -- 172.21.15.151:0/3786222213 learned_addr learned my addr 172.21.15.151:0/3786222213 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:55:21.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.563+0000 7f2f8b400700 1 --2- 172.21.15.151:0/3786222213 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2f8c10ff40 0x7f2f8c1a0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:21.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.563+0000 7f2f8a000700 1 -- 172.21.15.151:0/3786222213 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2f8c10ff40 msgr2=0x7f2f8c1a0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:21.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.563+0000 7f2f8a000700 1 --2- 172.21.15.151:0/3786222213 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2f8c10ff40 0x7f2f8c1a0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:21.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.563+0000 7f2f8a000700 1 -- 172.21.15.151:0/3786222213 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f8c075770 msgr2=0x7f2f8c19be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:21.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.563+0000 7f2f8a000700 1 --2- 172.21.15.151:0/3786222213 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f8c075770 0x7f2f8c19be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:21.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.563+0000 7f2f8a000700 1 -- 172.21.15.151:0/3786222213 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f8c1a0e50 con 0x7f2f8c076090 2024-05-21T21:55:21.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.563+0000 7f2f8aa00700 1 --2- 172.21.15.151:0/3786222213 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f8c075770 0x7f2f8c19be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:55:21.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.564+0000 7f2f8a000700 1 --2- 172.21.15.151:0/3786222213 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2f8c076090 0x7f2f8c19c3d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2f780057c0 tx=0x7f2f78009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:55:21.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.564+0000 7f2f90a00700 1 -- 172.21.15.151:0/3786222213 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f78026070 con 0x7f2f8c076090 2024-05-21T21:55:21.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.564+0000 7f2f90a00700 1 -- 172.21.15.151:0/3786222213 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f78021410 con 0x7f2f8c076090 2024-05-21T21:55:21.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.564+0000 7f2f91e00700 1 -- 172.21.15.151:0/3786222213 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f8c1a10e0 con 0x7f2f8c076090 2024-05-21T21:55:21.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.564+0000 7f2f90a00700 1 -- 172.21.15.151:0/3786222213 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f78017650 con 0x7f2f8c076090 2024-05-21T21:55:21.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.564+0000 7f2f91e00700 1 -- 172.21.15.151:0/3786222213 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f8c1a89c0 con 0x7f2f8c076090 2024-05-21T21:55:21.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.567+0000 7f2f90a00700 1 -- 172.21.15.151:0/3786222213 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f2f78029070 con 0x7f2f8c076090 2024-05-21T21:55:21.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.569+0000 7f2f9c6e4b80 1 Processor -- start 2024-05-21T21:55:21.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.569+0000 7f2f9c6e4b80 1 -- start start 2024-05-21T21:55:21.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.569+0000 7f2f90a00700 1 --2- 172.21.15.151:0/3786222213 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f2f700b2880 0x7f2f700b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:21.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.569+0000 7f2f9c6e4b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56482ad095c0 0x56482ad099a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:21.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.569+0000 7f2f9c6e4b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56482ad09ee0 0x56482ad03ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:21.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.569+0000 7f2f9c6e4b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56482ad040f0 0x56482ad04570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:21.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.569+0000 7f2f9c6e4b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56482ac61df0 con 0x56482ad095c0 2024-05-21T21:55:21.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.569+0000 7f2f9c6e4b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56482ac51990 con 0x56482ad09ee0 2024-05-21T21:55:21.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.569+0000 7f2f90a00700 1 -- 172.21.15.151:0/3786222213 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f2f780e2f00 con 0x7f2f8c076090 2024-05-21T21:55:21.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.569+0000 7f2f9c6e4b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56482ac559d0 con 0x56482ad040f0 2024-05-21T21:55:21.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.569+0000 7f2f8b400700 1 --2- 172.21.15.151:0/3786222213 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f2f700b2880 0x7f2f700b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:21.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.570+0000 7f2f8aa00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56482ad095c0 0x56482ad099a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:21.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.570+0000 7f2f8b400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56482ad040f0 0x56482ad04570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:21.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.570+0000 7f2f8aa00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56482ad095c0 0x56482ad099a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:56102/0 (socket says 172.21.15.151:56102) 2024-05-21T21:55:21.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.570+0000 7f2f8aa00700 1 -- 172.21.15.151:0/2428198819 learned_addr learned my addr 172.21.15.151:0/2428198819 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:55:21.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.570+0000 7f2f8a000700 1 --2- 172.21.15.151:0/2428198819 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56482ad09ee0 0x56482ad03ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:21.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.570+0000 7f2f8a000700 1 -- 172.21.15.151:0/2428198819 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56482ad040f0 msgr2=0x56482ad04570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:55:21.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.570+0000 7f2f8a000700 1 --2- 172.21.15.151:0/2428198819 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56482ad040f0 0x56482ad04570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:21.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.570+0000 7f2f8a000700 1 -- 172.21.15.151:0/2428198819 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56482ad095c0 msgr2=0x56482ad099a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:55:21.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.570+0000 7f2f8a000700 1 --2- 172.21.15.151:0/2428198819 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56482ad095c0 0x56482ad099a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:21.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.570+0000 7f2f8a000700 1 -- 172.21.15.151:0/2428198819 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56482ac1a3d0 con 0x56482ad09ee0 2024-05-21T21:55:21.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.570+0000 7f2f8aa00700 1 --2- 172.21.15.151:0/2428198819 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56482ad095c0 0x56482ad099a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:55:21.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.571+0000 7f2f8b400700 1 --2- 172.21.15.151:0/3786222213 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f2f700b2880 0x7f2f700b4d40 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2f84009ec0 tx=0x7f2f8400fc40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:55:21.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.571+0000 7f2f8a000700 1 --2- 172.21.15.151:0/2428198819 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56482ad09ee0 0x56482ad03ad0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2f7800c010 tx=0x7f2f78009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3226211a81f6d7e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:55:21.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.571+0000 7f2f80c00700 1 -- 172.21.15.151:0/2428198819 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f780aad00 con 0x56482ad09ee0 2024-05-21T21:55:21.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.571+0000 7f2f80c00700 1 -- 172.21.15.151:0/2428198819 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f780aa600 con 0x56482ad09ee0 2024-05-21T21:55:21.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.571+0000 7f2f80c00700 1 -- 172.21.15.151:0/2428198819 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f780218c0 con 0x56482ad09ee0 2024-05-21T21:55:21.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.572+0000 7f2f9c6e4b80 1 -- 172.21.15.151:0/2428198819 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56482ad09ee0 msgr2=0x56482ad03ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:21.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.572+0000 7f2f9c6e4b80 1 --2- 172.21.15.151:0/2428198819 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56482ad09ee0 0x56482ad03ad0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2f7800c010 tx=0x7f2f78009ca0 comp rx=0 tx=0).stop 2024-05-21T21:55:21.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.572+0000 7f2f9c6e4b80 1 -- 172.21.15.151:0/2428198819 shutdown_connections 2024-05-21T21:55:21.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.572+0000 7f2f9c6e4b80 1 --2- 172.21.15.151:0/2428198819 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56482ad040f0 0x56482ad04570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:21.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.572+0000 7f2f9c6e4b80 1 --2- 172.21.15.151:0/2428198819 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56482ad095c0 0x56482ad099a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:21.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.572+0000 7f2f9c6e4b80 1 --2- 172.21.15.151:0/2428198819 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56482ad09ee0 0x56482ad03ad0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:21.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.572+0000 7f2f9c6e4b80 1 -- 172.21.15.151:0/2428198819 >> 172.21.15.151:0/2428198819 conn(0x56482ac31b90 msgr2=0x56482acfe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:55:21.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.572+0000 7f2f9c6e4b80 1 -- 172.21.15.151:0/2428198819 shutdown_connections 2024-05-21T21:55:21.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.572+0000 7f2f9c6e4b80 1 -- 172.21.15.151:0/2428198819 wait complete. 2024-05-21T21:55:21.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.573+0000 7f2f9c6e4b80 1 -- start start 2024-05-21T21:55:21.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.573+0000 7f2f9c6e4b80 1 Processor -- start 2024-05-21T21:55:21.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.573+0000 7f2f9c6e4b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56482ad040f0 0x56482ad0fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:21.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.573+0000 7f2f9c6e4b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56482ad095c0 0x56482ad0d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:21.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.573+0000 7f2f9c6e4b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56482ad09ee0 0x56482ad0d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:21.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.573+0000 7f2f9c6e4b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56482ab57f50 con 0x56482ad09ee0 2024-05-21T21:55:21.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.573+0000 7f2f9c6e4b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56482abf2710 con 0x56482ad095c0 2024-05-21T21:55:21.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.573+0000 7f2f9c6e4b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56482ad10750 con 0x56482ad040f0 2024-05-21T21:55:21.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.573+0000 7f2f9c6e4b80 10 client.0 _ll_register_callbacks cb 0x56482abfbc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:55:21.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.574+0000 7f2f8b400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56482ad09ee0 0x56482ad0d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:21.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.574+0000 7f2f8aa00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56482ad040f0 0x56482ad0fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:21.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.574+0000 7f2f8b400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56482ad09ee0 0x56482ad0d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:56108/0 (socket says 172.21.15.151:56108) 2024-05-21T21:55:21.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.574+0000 7f2f8b400700 1 -- 172.21.15.151:0/3162993296 learned_addr learned my addr 172.21.15.151:0/3162993296 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:55:21.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.574+0000 7f2f8a000700 1 --2- 172.21.15.151:0/3162993296 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56482ad095c0 0x56482ad0d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:21.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.574+0000 7f2f8aa00700 1 -- 172.21.15.151:0/3162993296 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56482ad095c0 msgr2=0x56482ad0d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:55:21.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.574+0000 7f2f8aa00700 1 --2- 172.21.15.151:0/3162993296 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56482ad095c0 0x56482ad0d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:21.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.574+0000 7f2f8aa00700 1 -- 172.21.15.151:0/3162993296 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56482ad09ee0 msgr2=0x56482ad0d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:55:21.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.574+0000 7f2f8aa00700 1 --2- 172.21.15.151:0/3162993296 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56482ad09ee0 0x56482ad0d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:21.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.574+0000 7f2f8aa00700 1 -- 172.21.15.151:0/3162993296 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56482aba1880 con 0x56482ad040f0 2024-05-21T21:55:21.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.574+0000 7f2f8b400700 1 --2- 172.21.15.151:0/3162993296 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56482ad09ee0 0x56482ad0d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:55:21.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.574+0000 7f2f8aa00700 1 --2- 172.21.15.151:0/3162993296 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56482ad040f0 0x56482ad0fd20 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c0119d0 tx=0x7f2f7c011ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c22e620fdd999ea6 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:55:21.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.574+0000 7f2f75600700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-21T21:55:21.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.575+0000 7f2f75600700 1 -- 172.21.15.151:0/3162993296 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f7c010b70 con 0x56482ad040f0 2024-05-21T21:55:21.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.575+0000 7f2f75600700 1 -- 172.21.15.151:0/3162993296 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f2f7c010d10 con 0x56482ad040f0 2024-05-21T21:55:21.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.575+0000 7f2f9c6e4b80 1 -- 172.21.15.151:0/3162993296 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f780aa600 con 0x56482ad040f0 2024-05-21T21:55:21.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.575+0000 7f2f75600700 1 -- 172.21.15.151:0/3162993296 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2f7c01b420 con 0x56482ad040f0 2024-05-21T21:55:21.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.575+0000 7f2f9c6e4b80 10 client.25093 fetch_fsmap learned FSMap version 28 2024-05-21T21:55:21.578 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.575+0000 7f2f9c6e4b80 1 -- 172.21.15.151:0/3162993296 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56482ac22010 con 0x56482ad040f0 2024-05-21T21:55:21.578 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.575+0000 7f2f75600700 1 -- 172.21.15.151:0/3162993296 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f780aa600 con 0x56482ad040f0 2024-05-21T21:55:21.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.575+0000 7f2f75600700 1 -- 172.21.15.151:0/3162993296 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f2f7c025500 con 0x56482ad040f0 2024-05-21T21:55:21.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.576+0000 7f2f9c6e4b80 10 client.25093 fetch_fsmap finished waiting for FSMap version 28 2024-05-21T21:55:21.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.576+0000 7f2f9c6e4b80 10 client.25093 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-21T21:55:21.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.576+0000 7f2f9c6e4b80 20 client.25093 populate_metadata read hostname 'smithi151' 2024-05-21T21:55:21.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.576+0000 7f2f9c6e4b80 1 --2- 172.21.15.151:0/3162993296 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x56482ad1d2d0 0x56482ad1e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:21.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.576+0000 7f2f9c6e4b80 4 client.25093 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T21:55:21.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.576+0000 7f2f9c6e4b80 1 -- 172.21.15.151:0/3162993296 --> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56482ac63d40 con 0x56482ad1d2d0 2024-05-21T21:55:21.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.576+0000 7f2f8a000700 1 --2- 172.21.15.151:0/3162993296 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x56482ad1d2d0 0x56482ad1e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:21.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.576+0000 7f2f8a000700 1 --2- 172.21.15.151:0/3162993296 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x56482ad1d2d0 0x56482ad1e740 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=15a3ab2881e58b0e server_cookie=8e87a2191b2a03f8 in_seq=0 out_seq=0 2024-05-21T21:55:21.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.576+0000 7f2f75600700 10 client.25093 ms_handle_connect on v2:172.21.15.184:6838/3059935143 2024-05-21T21:55:21.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.577+0000 7f2f75600700 1 -- 172.21.15.151:0/3162993296 <== mds.1 v2:172.21.15.184:6838/3059935143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x56482ac63d40 con 0x56482ad1d2d0 2024-05-21T21:55:21.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.577+0000 7f2f75600700 10 client.25093 handle_command_reply: tid=0 2024-05-21T21:55:21.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.577+0000 7f2f9c6e4b80 1 client.25093 shutdown 2024-05-21T21:55:21.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.578+0000 7f2f9c6e4b80 1 -- 172.21.15.151:0/3162993296 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56482ad040f0 msgr2=0x56482ad0fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:21.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.578+0000 7f2f9c6e4b80 1 --2- 172.21.15.151:0/3162993296 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56482ad040f0 0x56482ad0fd20 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c0119d0 tx=0x7f2f7c011ce0 comp rx=0 tx=0).stop 2024-05-21T21:55:21.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.578+0000 7f2f9c6e4b80 1 -- 172.21.15.151:0/3162993296 shutdown_connections 2024-05-21T21:55:21.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.578+0000 7f2f9c6e4b80 1 --2- 172.21.15.151:0/3162993296 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x56482ad1d2d0 0x56482ad1e740 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:21.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.578+0000 7f2f9c6e4b80 1 --2- 172.21.15.151:0/3162993296 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56482ad09ee0 0x56482ad0d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:21.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.578+0000 7f2f9c6e4b80 1 --2- 172.21.15.151:0/3162993296 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56482ad040f0 0x56482ad0fd20 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:21.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.578+0000 7f2f9c6e4b80 1 --2- 172.21.15.151:0/3162993296 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56482ad095c0 0x56482ad0d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:21.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.578+0000 7f2f75600700 0 client.25093 ms_handle_reset on v2:172.21.15.184:6838/3059935143 2024-05-21T21:55:21.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.578+0000 7f2f9c6e4b80 1 -- 172.21.15.151:0/3162993296 >> 172.21.15.151:0/3162993296 conn(0x56482ac31b90 msgr2=0x56482acfdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:55:21.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.578+0000 7f2f9c6e4b80 1 -- 172.21.15.151:0/3162993296 shutdown_connections 2024-05-21T21:55:21.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.578+0000 7f2f9c6e4b80 1 -- 172.21.15.151:0/3162993296 wait complete. 2024-05-21T21:55:21.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.578+0000 7f2f9c6e4b80 20 client.25093 trim_cache size 0 max 16384 2024-05-21T21:55:21.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.601+0000 7f2f9c6e4b80 1 Processor -- start 2024-05-21T21:55:21.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.601+0000 7f2f9c6e4b80 1 -- start start 2024-05-21T21:55:21.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.601+0000 7f2f9c6e4b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56482ad040f0 0x56482ad03ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:21.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.601+0000 7f2f9c6e4b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56482ad095c0 0x56482ad077e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:21.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.601+0000 7f2f9c6e4b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56482ad09ee0 0x56482acfe4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:21.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.602+0000 7f2f9c6e4b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56482ad1ca80 con 0x56482ad040f0 2024-05-21T21:55:21.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.602+0000 7f2f9c6e4b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56482ad1d150 con 0x56482ad095c0 2024-05-21T21:55:21.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.602+0000 7f2f9c6e4b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56482ad1d820 con 0x56482ad09ee0 2024-05-21T21:55:21.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.602+0000 7f2f8aa00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56482ad040f0 0x56482ad03ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:21.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.602+0000 7f2f8a000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56482ad095c0 0x56482ad077e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:21.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.602+0000 7f2f8a000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56482ad095c0 0x56482ad077e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:56488/0 (socket says 172.21.15.151:56488) 2024-05-21T21:55:21.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.602+0000 7f2f8b400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56482ad09ee0 0x56482acfe4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:21.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.602+0000 7f2f8a000700 1 -- 172.21.15.151:0/457739488 learned_addr learned my addr 172.21.15.151:0/457739488 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:55:21.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.602+0000 7f2f8aa00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56482ad040f0 0x56482ad03ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:56110/0 (socket says 172.21.15.151:56110) 2024-05-21T21:55:21.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.602+0000 7f2f8a000700 1 -- 172.21.15.151:0/457739488 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56482ad09ee0 msgr2=0x56482acfe4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:55:21.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.602+0000 7f2f8a000700 1 --2- 172.21.15.151:0/457739488 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56482ad09ee0 0x56482acfe4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:21.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.602+0000 7f2f8a000700 1 -- 172.21.15.151:0/457739488 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56482ad040f0 msgr2=0x56482ad03ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:55:21.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.602+0000 7f2f8a000700 1 --2- 172.21.15.151:0/457739488 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56482ad040f0 0x56482ad03ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:21.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.602+0000 7f2f8a000700 1 -- 172.21.15.151:0/457739488 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56482ad0b630 con 0x56482ad095c0 2024-05-21T21:55:21.604 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.602+0000 7f2f8a000700 1 --2- 172.21.15.151:0/457739488 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56482ad095c0 0x56482ad077e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2f7800c010 tx=0x7f2f78021c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb51981b066f29d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:55:21.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.603+0000 7f2f77e00700 1 -- 172.21.15.151:0/457739488 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f780c3070 con 0x56482ad095c0 2024-05-21T21:55:21.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.603+0000 7f2f77e00700 1 -- 172.21.15.151:0/457739488 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f780b33d0 con 0x56482ad095c0 2024-05-21T21:55:21.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.603+0000 7f2f77e00700 1 -- 172.21.15.151:0/457739488 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f780218c0 con 0x56482ad095c0 2024-05-21T21:55:21.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.603+0000 7f2f9c6e4b80 1 -- 172.21.15.151:0/457739488 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56482ad095c0 msgr2=0x56482ad077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:21.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.603+0000 7f2f9c6e4b80 1 --2- 172.21.15.151:0/457739488 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56482ad095c0 0x56482ad077e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2f7800c010 tx=0x7f2f78021c60 comp rx=0 tx=0).stop 2024-05-21T21:55:21.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.603+0000 7f2f9c6e4b80 1 -- 172.21.15.151:0/457739488 shutdown_connections 2024-05-21T21:55:21.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.603+0000 7f2f9c6e4b80 1 --2- 172.21.15.151:0/457739488 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56482ad09ee0 0x56482acfe4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:21.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.603+0000 7f2f9c6e4b80 1 --2- 172.21.15.151:0/457739488 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56482ad040f0 0x56482ad03ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:21.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.603+0000 7f2f9c6e4b80 1 --2- 172.21.15.151:0/457739488 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56482ad095c0 0x56482ad077e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:21.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.603+0000 7f2f9c6e4b80 1 -- 172.21.15.151:0/457739488 >> 172.21.15.151:0/457739488 conn(0x56482ac31b90 msgr2=0x56482ad070d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:55:21.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.604+0000 7f2f9c6e4b80 1 -- 172.21.15.151:0/457739488 shutdown_connections 2024-05-21T21:55:21.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.604+0000 7f2f9c6e4b80 1 -- 172.21.15.151:0/457739488 wait complete. 2024-05-21T21:55:21.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.604+0000 7f2f9c6e4b80 1 -- start start 2024-05-21T21:55:21.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.604+0000 7f2f9c6e4b80 1 Processor -- start 2024-05-21T21:55:21.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.605+0000 7f2f9c6e4b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56482ad040f0 0x56482acfdb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:21.606 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.605+0000 7f2f9c6e4b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56482ad095c0 0x56482acfe050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:21.606 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.605+0000 7f2f9c6e4b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56482ad09ee0 0x56482acfe590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:21.606 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.605+0000 7f2f9c6e4b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56482ad1e5c0 con 0x56482ad09ee0 2024-05-21T21:55:21.606 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.605+0000 7f2f9c6e4b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56482ad1def0 con 0x56482ad040f0 2024-05-21T21:55:21.606 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.605+0000 7f2f9c6e4b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56482ad1ec90 con 0x56482ad095c0 2024-05-21T21:55:21.606 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.605+0000 7f2f9c6e4b80 10 client.0 _ll_register_callbacks cb 0x56482abfbc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:55:21.609 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.605+0000 7f2f8b400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56482ad09ee0 0x56482acfe590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:21.609 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.605+0000 7f2f8b400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56482ad09ee0 0x56482acfe590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:56116/0 (socket says 172.21.15.151:56116) 2024-05-21T21:55:21.609 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.605+0000 7f2f8b400700 1 -- 172.21.15.151:0/2243720086 learned_addr learned my addr 172.21.15.151:0/2243720086 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:55:21.609 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.605+0000 7f2f8a000700 1 --2- 172.21.15.151:0/2243720086 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56482ad095c0 0x56482acfe050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:21.609 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.605+0000 7f2f8aa00700 1 --2- 172.21.15.151:0/2243720086 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56482ad040f0 0x56482acfdb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:21.609 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.605+0000 7f2f8a000700 1 -- 172.21.15.151:0/2243720086 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56482ad040f0 msgr2=0x56482acfdb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:55:21.609 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.605+0000 7f2f8a000700 1 --2- 172.21.15.151:0/2243720086 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56482ad040f0 0x56482acfdb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:21.609 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.605+0000 7f2f8a000700 1 -- 172.21.15.151:0/2243720086 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56482ad09ee0 msgr2=0x56482acfe590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:55:21.609 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.605+0000 7f2f8a000700 1 --2- 172.21.15.151:0/2243720086 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56482ad09ee0 0x56482acfe590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:21.610 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.605+0000 7f2f8a000700 1 -- 172.21.15.151:0/2243720086 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56482ad26240 con 0x56482ad095c0 2024-05-21T21:55:21.610 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.605+0000 7f2f8b400700 1 --2- 172.21.15.151:0/2243720086 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56482ad09ee0 0x56482acfe590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:55:21.610 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.606+0000 7f2f8a000700 1 --2- 172.21.15.151:0/2243720086 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56482ad095c0 0x56482acfe050 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x56482ad10640 tx=0x7f2f780a9f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=908ed59a609a2dfc server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:55:21.610 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.606+0000 7f2f75600700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-21T21:55:21.610 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.606+0000 7f2f75600700 1 -- 172.21.15.151:0/2243720086 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f780c9060 con 0x56482ad095c0 2024-05-21T21:55:21.610 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.606+0000 7f2f9c6e4b80 1 -- 172.21.15.151:0/2243720086 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56482ac5d520 con 0x56482ad095c0 2024-05-21T21:55:21.610 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.606+0000 7f2f75600700 1 -- 172.21.15.151:0/2243720086 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f2f780218c0 con 0x56482ad095c0 2024-05-21T21:55:21.610 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.606+0000 7f2f75600700 1 -- 172.21.15.151:0/2243720086 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2f780c4e60 con 0x56482ad095c0 2024-05-21T21:55:21.610 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.606+0000 7f2f9c6e4b80 10 client.25099 fetch_fsmap learned FSMap version 28 2024-05-21T21:55:21.610 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.606+0000 7f2f9c6e4b80 1 -- 172.21.15.151:0/2243720086 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56482ad88e50 con 0x56482ad095c0 2024-05-21T21:55:21.610 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.606+0000 7f2f75600700 1 -- 172.21.15.151:0/2243720086 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56482ac5d520 con 0x56482ad095c0 2024-05-21T21:55:21.610 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.607+0000 7f2f75600700 1 -- 172.21.15.151:0/2243720086 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f2f780aa860 con 0x56482ad095c0 2024-05-21T21:55:21.610 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.607+0000 7f2f9c6e4b80 10 client.25099 fetch_fsmap finished waiting for FSMap version 28 2024-05-21T21:55:21.610 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.607+0000 7f2f9c6e4b80 10 client.25099 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-21T21:55:21.611 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.607+0000 7f2f9c6e4b80 20 client.25099 populate_metadata read hostname 'smithi151' 2024-05-21T21:55:21.611 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.607+0000 7f2f9c6e4b80 1 --2- 172.21.15.151:0/2243720086 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x56482ad8d500 0x56482ad8c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:21.611 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.607+0000 7f2f9c6e4b80 4 client.25099 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-21T21:55:21.611 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.607+0000 7f2f9c6e4b80 1 -- 172.21.15.151:0/2243720086 --> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] -- command(tid 0: {"prefix": "status"}) -- 0x56482aad4200 con 0x56482ad8d500 2024-05-21T21:55:21.611 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.607+0000 7f2f8aa00700 1 --2- 172.21.15.151:0/2243720086 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x56482ad8d500 0x56482ad8c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:21.611 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.608+0000 7f2f8aa00700 1 --2- 172.21.15.151:0/2243720086 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x56482ad8d500 0x56482ad8c210 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d683047e8d2f7497 server_cookie=2ff5501ee4e9ba74 in_seq=0 out_seq=0 2024-05-21T21:55:21.611 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.608+0000 7f2f75600700 10 client.25099 ms_handle_connect on v2:172.21.15.184:6838/3059935143 2024-05-21T21:55:21.611 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.608+0000 7f2f75600700 1 -- 172.21.15.151:0/2243720086 <== mds.1 v2:172.21.15.184:6838/3059935143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x56482aad4200 con 0x56482ad8d500 2024-05-21T21:55:21.611 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.608+0000 7f2f75600700 10 client.25099 handle_command_reply: tid=0 2024-05-21T21:55:21.611 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.608+0000 7f2f9c6e4b80 1 client.25099 shutdown 2024-05-21T21:55:21.611 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.608+0000 7f2f9c6e4b80 1 -- 172.21.15.151:0/2243720086 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56482ad095c0 msgr2=0x56482acfe050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:21.611 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.608+0000 7f2f9c6e4b80 1 --2- 172.21.15.151:0/2243720086 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56482ad095c0 0x56482acfe050 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x56482ad10640 tx=0x7f2f780a9f50 comp rx=0 tx=0).stop 2024-05-21T21:55:21.611 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.608+0000 7f2f9c6e4b80 1 -- 172.21.15.151:0/2243720086 shutdown_connections 2024-05-21T21:55:21.612 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.608+0000 7f2f9c6e4b80 1 --2- 172.21.15.151:0/2243720086 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x56482ad8d500 0x56482ad8c210 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:21.612 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.608+0000 7f2f9c6e4b80 1 --2- 172.21.15.151:0/2243720086 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56482ad040f0 0x56482acfdb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:21.612 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.608+0000 7f2f9c6e4b80 1 --2- 172.21.15.151:0/2243720086 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56482ad09ee0 0x56482acfe590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:21.612 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.608+0000 7f2f75600700 0 client.25099 ms_handle_reset on v2:172.21.15.184:6838/3059935143 2024-05-21T21:55:21.612 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.608+0000 7f2f9c6e4b80 1 --2- 172.21.15.151:0/2243720086 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56482ad095c0 0x56482acfe050 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:21.612 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.608+0000 7f2f9c6e4b80 1 -- 172.21.15.151:0/2243720086 >> 172.21.15.151:0/2243720086 conn(0x56482ac31b90 msgr2=0x56482ad0cea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:55:21.612 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.608+0000 7f2f9c6e4b80 1 -- 172.21.15.151:0/2243720086 shutdown_connections 2024-05-21T21:55:21.612 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.608+0000 7f2f9c6e4b80 1 -- 172.21.15.151:0/2243720086 wait complete. 2024-05-21T21:55:21.612 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.608+0000 7f2f9c6e4b80 20 client.25099 trim_cache size 0 max 16384 2024-05-21T21:55:21.612 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-05-21T21:55:21.612 INFO:teuthology.orchestra.run.smithi151.stdout: "cluster_fsid": "c8c7a2e6-17ba-11ef-bc9a-c7b262605968", 2024-05-21T21:55:21.613 INFO:teuthology.orchestra.run.smithi151.stdout: "whoami": 1, 2024-05-21T21:55:21.613 INFO:teuthology.orchestra.run.smithi151.stdout: "id": 24493, 2024-05-21T21:55:21.613 INFO:teuthology.orchestra.run.smithi151.stdout: "want_state": "up:active", 2024-05-21T21:55:21.613 INFO:teuthology.orchestra.run.smithi151.stdout: "state": "up:active", 2024-05-21T21:55:21.613 INFO:teuthology.orchestra.run.smithi151.stdout: "fs_name": "cephfs", 2024-05-21T21:55:21.613 INFO:teuthology.orchestra.run.smithi151.stdout: "rank_uptime": 226.30975992099999, 2024-05-21T21:55:21.613 INFO:teuthology.orchestra.run.smithi151.stdout: "mdsmap_epoch": 28, 2024-05-21T21:55:21.613 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_epoch": 82, 2024-05-21T21:55:21.613 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_epoch_barrier": 80, 2024-05-21T21:55:21.613 INFO:teuthology.orchestra.run.smithi151.stdout: "uptime": 274.67747026299998 2024-05-21T21:55:21.613 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-05-21T21:55:21.614 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.609+0000 7f2f91e00700 1 -- 172.21.15.151:0/3786222213 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f2f700b2880 msgr2=0x7f2f700b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:21.614 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.609+0000 7f2f91e00700 1 --2- 172.21.15.151:0/3786222213 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f2f700b2880 0x7f2f700b4d40 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2f84009ec0 tx=0x7f2f8400fc40 comp rx=0 tx=0).stop 2024-05-21T21:55:21.614 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.609+0000 7f2f91e00700 1 -- 172.21.15.151:0/3786222213 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2f8c076090 msgr2=0x7f2f8c19c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:21.614 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.609+0000 7f2f91e00700 1 --2- 172.21.15.151:0/3786222213 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2f8c076090 0x7f2f8c19c3d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2f780057c0 tx=0x7f2f78009580 comp rx=0 tx=0).stop 2024-05-21T21:55:21.614 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.609+0000 7f2f91e00700 1 -- 172.21.15.151:0/3786222213 shutdown_connections 2024-05-21T21:55:21.614 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.609+0000 7f2f91e00700 1 --2- 172.21.15.151:0/3786222213 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f2f700b2880 0x7f2f700b4d40 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:21.614 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.609+0000 7f2f91e00700 1 --2- 172.21.15.151:0/3786222213 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f8c075770 0x7f2f8c19be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:21.614 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.609+0000 7f2f91e00700 1 --2- 172.21.15.151:0/3786222213 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2f8c10ff40 0x7f2f8c1a0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:21.614 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.609+0000 7f2f91e00700 1 --2- 172.21.15.151:0/3786222213 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2f8c076090 0x7f2f8c19c3d0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:21.614 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.609+0000 7f2f91e00700 1 -- 172.21.15.151:0/3786222213 >> 172.21.15.151:0/3786222213 conn(0x7f2f8c07a6a0 msgr2=0x7f2f8c1007f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:55:21.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.609+0000 7f2f91e00700 1 -- 172.21.15.151:0/3786222213 shutdown_connections 2024-05-21T21:55:21.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:21.610+0000 7f2f91e00700 1 -- 172.21.15.151:0/3786222213 wait complete. 2024-05-21T21:55:22.391 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.b status 2024-05-21T21:55:22.702 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:22 smithi151 ceph-mon[31623]: pgmap v438: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-05-21T21:55:22.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:22 smithi164 ceph-mon[36269]: pgmap v438: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-05-21T21:55:22.788 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:22 smithi184 ceph-mon[36942]: pgmap v438: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-05-21T21:55:22.830 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:55:24.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:24 smithi151 ceph-mon[31623]: pgmap v439: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-21T21:55:24.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:24 smithi164 ceph-mon[36269]: pgmap v439: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-21T21:55:24.888 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.888+0000 7fd9b1200700 1 -- 172.21.15.151:0/3763319589 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9ac106010 msgr2=0x7fd9ac112520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:24.889 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.888+0000 7fd9b1200700 1 --2- 172.21.15.151:0/3763319589 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9ac106010 0x7fd9ac112520 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd9a400b0a0 tx=0x7fd9a400b3b0 comp rx=0 tx=0).stop 2024-05-21T21:55:24.889 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.888+0000 7fd9b1200700 1 -- 172.21.15.151:0/3763319589 shutdown_connections 2024-05-21T21:55:24.889 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.888+0000 7fd9b1200700 1 --2- 172.21.15.151:0/3763319589 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd9ac10b650 0x7fd9ac10ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:24.889 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.888+0000 7fd9b1200700 1 --2- 172.21.15.151:0/3763319589 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9ac106010 0x7fd9ac112520 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:24.889 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.888+0000 7fd9b1200700 1 --2- 172.21.15.151:0/3763319589 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd9ac105650 0x7fd9ac105ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:24.889 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.888+0000 7fd9b1200700 1 -- 172.21.15.151:0/3763319589 >> 172.21.15.151:0/3763319589 conn(0x7fd9ac101120 msgr2=0x7fd9ac103540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:55:24.889 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.888+0000 7fd9b1200700 1 -- 172.21.15.151:0/3763319589 shutdown_connections 2024-05-21T21:55:24.889 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.888+0000 7fd9b1200700 1 -- 172.21.15.151:0/3763319589 wait complete. 2024-05-21T21:55:24.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.889+0000 7fd9b1200700 1 Processor -- start 2024-05-21T21:55:24.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.889+0000 7fd9b1200700 1 -- start start 2024-05-21T21:55:24.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.890+0000 7fd9b1200700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9ac105650 0x7fd9ac1a1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:24.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.890+0000 7fd9b1200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd9ac106010 0x7fd9ac1a1e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:24.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.890+0000 7fd9b1200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd9ac10b650 0x7fd9ac19bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:24.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.890+0000 7fd9b1200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd9ac115850 con 0x7fd9ac10b650 2024-05-21T21:55:24.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.890+0000 7fd9b1200700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd9ac1156d0 con 0x7fd9ac106010 2024-05-21T21:55:24.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.890+0000 7fd9b1200700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd9ac1159d0 con 0x7fd9ac105650 2024-05-21T21:55:24.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.890+0000 7fd9abe00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd9ac10b650 0x7fd9ac19bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:24.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.890+0000 7fd9ab400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9ac105650 0x7fd9ac1a1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:24.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.890+0000 7fd9aaa00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd9ac106010 0x7fd9ac1a1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:24.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.890+0000 7fd9ab400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9ac105650 0x7fd9ac1a1920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:42762/0 (socket says 172.21.15.151:42762) 2024-05-21T21:55:24.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.890+0000 7fd9ab400700 1 -- 172.21.15.151:0/3273586383 learned_addr learned my addr 172.21.15.151:0/3273586383 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:55:24.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.891+0000 7fd9ab400700 1 -- 172.21.15.151:0/3273586383 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd9ac106010 msgr2=0x7fd9ac1a1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:24.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.891+0000 7fd9ab400700 1 --2- 172.21.15.151:0/3273586383 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd9ac106010 0x7fd9ac1a1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:24.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.891+0000 7fd9ab400700 1 -- 172.21.15.151:0/3273586383 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd9ac10b650 msgr2=0x7fd9ac19bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:24.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.891+0000 7fd9ab400700 1 --2- 172.21.15.151:0/3273586383 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd9ac10b650 0x7fd9ac19bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:24.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.891+0000 7fd9ab400700 1 -- 172.21.15.151:0/3273586383 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9ac19c290 con 0x7fd9ac105650 2024-05-21T21:55:24.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.891+0000 7fd9ab400700 1 --2- 172.21.15.151:0/3273586383 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9ac105650 0x7fd9ac1a1920 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd99c00e9c0 tx=0x7fd99c00ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:55:24.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.891+0000 7fd9a3e00700 1 -- 172.21.15.151:0/3273586383 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd99c00cbd0 con 0x7fd9ac105650 2024-05-21T21:55:24.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.891+0000 7fd9a3e00700 1 -- 172.21.15.151:0/3273586383 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd99c00ee60 con 0x7fd9ac105650 2024-05-21T21:55:24.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.891+0000 7fd9a3e00700 1 -- 172.21.15.151:0/3273586383 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd99c0055c0 con 0x7fd9ac105650 2024-05-21T21:55:24.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.891+0000 7fd9b1200700 1 -- 172.21.15.151:0/3273586383 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9ac19c580 con 0x7fd9ac105650 2024-05-21T21:55:24.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.892+0000 7fd9b1200700 1 -- 172.21.15.151:0/3273586383 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9ac1a87f0 con 0x7fd9ac105650 2024-05-21T21:55:24.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.897+0000 7fd9a3e00700 1 -- 172.21.15.151:0/3273586383 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7fd99c004750 con 0x7fd9ac105650 2024-05-21T21:55:24.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.898+0000 7fd9bd0a1b80 1 Processor -- start 2024-05-21T21:55:24.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.898+0000 7fd9bd0a1b80 1 -- start start 2024-05-21T21:55:24.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.898+0000 7fd9bd0a1b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b821d10fd0 0x55b821d113b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:24.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.898+0000 7fd9bd0a1b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b821d118f0 0x55b821d0b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:24.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.898+0000 7fd9bd0a1b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b821d0ba80 0x55b821d0bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:24.899 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.898+0000 7fd9bd0a1b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55b821c69df0 con 0x55b821d0ba80 2024-05-21T21:55:24.899 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.898+0000 7fd9bd0a1b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55b821c59990 con 0x55b821d10fd0 2024-05-21T21:55:24.899 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.898+0000 7fd9bd0a1b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55b821c5d9d0 con 0x55b821d118f0 2024-05-21T21:55:24.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.899+0000 7fd9a3e00700 1 --2- 172.21.15.151:0/3273586383 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fd9900b2900 0x7fd9900b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:24.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.899+0000 7fd9aaa00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b821d0ba80 0x55b821d0bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:24.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.899+0000 7fd9ab400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b821d118f0 0x55b821d0b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:24.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.899+0000 7fd9abe00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b821d10fd0 0x55b821d113b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:24.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.899+0000 7fd9aaa00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b821d0ba80 0x55b821d0bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:56142/0 (socket says 172.21.15.151:56142) 2024-05-21T21:55:24.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.899+0000 7fd9aaa00700 1 -- 172.21.15.151:0/3869053273 learned_addr learned my addr 172.21.15.151:0/3869053273 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:55:24.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.899+0000 7fd9a3e00700 1 -- 172.21.15.151:0/3273586383 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fd99c0e23d0 con 0x7fd9ac105650 2024-05-21T21:55:24.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.899+0000 7fd9abe00700 1 --2- 172.21.15.151:0/3273586383 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fd9900b2900 0x7fd9900b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:24.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.899+0000 7fd9ab400700 1 -- 172.21.15.151:0/3869053273 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b821d10fd0 msgr2=0x55b821d113b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:55:24.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.899+0000 7fd9ab400700 1 --2- 172.21.15.151:0/3869053273 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b821d10fd0 0x55b821d113b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:24.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.899+0000 7fd9ab400700 1 -- 172.21.15.151:0/3869053273 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b821d0ba80 msgr2=0x55b821d0bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:55:24.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.899+0000 7fd9ab400700 1 --2- 172.21.15.151:0/3869053273 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b821d0ba80 0x55b821d0bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:24.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.899+0000 7fd9ab400700 1 -- 172.21.15.151:0/3869053273 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b821c223d0 con 0x55b821d118f0 2024-05-21T21:55:24.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.899+0000 7fd9aaa00700 1 --2- 172.21.15.151:0/3869053273 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b821d0ba80 0x55b821d0bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:55:24.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.900+0000 7fd9ab400700 1 --2- 172.21.15.151:0/3869053273 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b821d118f0 0x55b821d0b460 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd99c009860 tx=0x7fd99c06f390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e718eea48f7acf1 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:55:24.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.900+0000 7fd9a0c00700 1 -- 172.21.15.151:0/3869053273 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd99c0a9120 con 0x55b821d118f0 2024-05-21T21:55:24.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.900+0000 7fd9abe00700 1 --2- 172.21.15.151:0/3273586383 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fd9900b2900 0x7fd9900b4dc0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd9a4006130 tx=0x7fd9a400b720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:55:24.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.900+0000 7fd9a0c00700 1 -- 172.21.15.151:0/3869053273 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd99c0b2150 con 0x55b821d118f0 2024-05-21T21:55:24.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.900+0000 7fd9a0c00700 1 -- 172.21.15.151:0/3869053273 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd99c0add60 con 0x55b821d118f0 2024-05-21T21:55:24.903 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.901+0000 7fd9bd0a1b80 1 -- 172.21.15.151:0/3869053273 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b821d118f0 msgr2=0x55b821d0b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:24.903 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.901+0000 7fd9bd0a1b80 1 --2- 172.21.15.151:0/3869053273 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b821d118f0 0x55b821d0b460 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd99c009860 tx=0x7fd99c06f390 comp rx=0 tx=0).stop 2024-05-21T21:55:24.903 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.901+0000 7fd9bd0a1b80 1 -- 172.21.15.151:0/3869053273 shutdown_connections 2024-05-21T21:55:24.903 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.901+0000 7fd9bd0a1b80 1 --2- 172.21.15.151:0/3869053273 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b821d10fd0 0x55b821d113b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:24.903 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.901+0000 7fd9bd0a1b80 1 --2- 172.21.15.151:0/3869053273 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b821d0ba80 0x55b821d0bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:24.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.901+0000 7fd9bd0a1b80 1 --2- 172.21.15.151:0/3869053273 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b821d118f0 0x55b821d0b460 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:24.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.901+0000 7fd9bd0a1b80 1 -- 172.21.15.151:0/3869053273 >> 172.21.15.151:0/3869053273 conn(0x55b821c39b90 msgr2=0x55b821d06100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:55:24.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.901+0000 7fd9bd0a1b80 1 -- 172.21.15.151:0/3869053273 shutdown_connections 2024-05-21T21:55:24.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.901+0000 7fd9bd0a1b80 1 -- 172.21.15.151:0/3869053273 wait complete. 2024-05-21T21:55:24.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.902+0000 7fd9bd0a1b80 1 -- start start 2024-05-21T21:55:24.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.902+0000 7fd9bd0a1b80 1 Processor -- start 2024-05-21T21:55:24.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.902+0000 7fd9bd0a1b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b821d0ba80 0x55b821d176a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:24.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.902+0000 7fd9bd0a1b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b821d10fd0 0x55b821d14a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:24.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.902+0000 7fd9bd0a1b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b821d118f0 0x55b821d14f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:24.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.902+0000 7fd9bd0a1b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55b821b5ff50 con 0x55b821d10fd0 2024-05-21T21:55:24.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.902+0000 7fd9bd0a1b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55b821bfa710 con 0x55b821d118f0 2024-05-21T21:55:24.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.902+0000 7fd9bd0a1b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55b821d183b0 con 0x55b821d0ba80 2024-05-21T21:55:24.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.902+0000 7fd9bd0a1b80 10 client.0 _ll_register_callbacks cb 0x55b821c03c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:55:24.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.903+0000 7fd9ab400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b821d0ba80 0x55b821d176a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:24.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.903+0000 7fd9ab400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b821d0ba80 0x55b821d176a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:42784/0 (socket says 172.21.15.151:42784) 2024-05-21T21:55:24.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.903+0000 7fd9ab400700 1 -- 172.21.15.151:0/2713376789 learned_addr learned my addr 172.21.15.151:0/2713376789 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:55:24.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.903+0000 7fd9aaa00700 1 --2- 172.21.15.151:0/2713376789 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b821d10fd0 0x55b821d14a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:24.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.903+0000 7fd9abe00700 1 --2- 172.21.15.151:0/2713376789 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b821d118f0 0x55b821d14f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:24.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.903+0000 7fd9ab400700 1 -- 172.21.15.151:0/2713376789 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b821d118f0 msgr2=0x55b821d14f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:55:24.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.903+0000 7fd9ab400700 1 --2- 172.21.15.151:0/2713376789 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b821d118f0 0x55b821d14f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:24.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.903+0000 7fd9ab400700 1 -- 172.21.15.151:0/2713376789 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b821d10fd0 msgr2=0x55b821d14a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:55:24.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.903+0000 7fd9ab400700 1 --2- 172.21.15.151:0/2713376789 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b821d10fd0 0x55b821d14a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:24.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.903+0000 7fd9ab400700 1 -- 172.21.15.151:0/2713376789 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b821ba9880 con 0x55b821d0ba80 2024-05-21T21:55:24.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.903+0000 7fd9ab400700 1 --2- 172.21.15.151:0/2713376789 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b821d0ba80 0x55b821d176a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd99c0a59e0 tx=0x7fd99c0a5150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd32cdf9301905e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:55:24.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.903+0000 7fd995600700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-21T21:55:24.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.904+0000 7fd995600700 1 -- 172.21.15.151:0/2713376789 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd99c0a94c0 con 0x55b821d0ba80 2024-05-21T21:55:24.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.904+0000 7fd995600700 1 -- 172.21.15.151:0/2713376789 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fd99c0a5fa0 con 0x55b821d0ba80 2024-05-21T21:55:24.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.904+0000 7fd9bd0a1b80 1 -- 172.21.15.151:0/2713376789 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b821c6bd40 con 0x55b821d0ba80 2024-05-21T21:55:24.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.904+0000 7fd995600700 1 -- 172.21.15.151:0/2713376789 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd99c06f400 con 0x55b821d0ba80 2024-05-21T21:55:24.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.904+0000 7fd9bd0a1b80 10 client.25111 fetch_fsmap learned FSMap version 28 2024-05-21T21:55:24.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.904+0000 7fd9bd0a1b80 1 -- 172.21.15.151:0/2713376789 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b821c2a010 con 0x55b821d0ba80 2024-05-21T21:55:24.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.904+0000 7fd995600700 1 -- 172.21.15.151:0/2713376789 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55b821c6bd40 con 0x55b821d0ba80 2024-05-21T21:55:24.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.904+0000 7fd995600700 1 -- 172.21.15.151:0/2713376789 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fd99c0aace0 con 0x55b821d0ba80 2024-05-21T21:55:24.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.904+0000 7fd9bd0a1b80 10 client.25111 fetch_fsmap finished waiting for FSMap version 28 2024-05-21T21:55:24.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.905+0000 7fd9bd0a1b80 10 client.25111 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-21T21:55:24.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.905+0000 7fd9bd0a1b80 20 client.25111 populate_metadata read hostname 'smithi151' 2024-05-21T21:55:24.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.905+0000 7fd9bd0a1b80 1 --2- 172.21.15.151:0/2713376789 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55b821d23c60 0x55b821d25e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:24.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.905+0000 7fd9bd0a1b80 4 client.25111 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T21:55:24.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.905+0000 7fd9bd0a1b80 1 -- 172.21.15.151:0/2713376789 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b821c65520 con 0x55b821d23c60 2024-05-21T21:55:24.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.905+0000 7fd9aaa00700 1 --2- 172.21.15.151:0/2713376789 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55b821d23c60 0x55b821d25e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:24.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.905+0000 7fd9aaa00700 1 --2- 172.21.15.151:0/2713376789 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55b821d23c60 0x55b821d25e30 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee5674c522c2568e server_cookie=f8c67c96fb2519f7 in_seq=0 out_seq=0 2024-05-21T21:55:24.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.905+0000 7fd995600700 10 client.25111 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T21:55:24.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.906+0000 7fd995600700 1 -- 172.21.15.151:0/2713376789 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55b821c65520 con 0x55b821d23c60 2024-05-21T21:55:24.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.906+0000 7fd995600700 10 client.25111 handle_command_reply: tid=0 2024-05-21T21:55:24.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.906+0000 7fd9bd0a1b80 1 client.25111 shutdown 2024-05-21T21:55:24.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.907+0000 7fd9bd0a1b80 1 -- 172.21.15.151:0/2713376789 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b821d0ba80 msgr2=0x55b821d176a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:24.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.907+0000 7fd9bd0a1b80 1 --2- 172.21.15.151:0/2713376789 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b821d0ba80 0x55b821d176a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd99c0a59e0 tx=0x7fd99c0a5150 comp rx=0 tx=0).stop 2024-05-21T21:55:24.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.907+0000 7fd9bd0a1b80 1 -- 172.21.15.151:0/2713376789 shutdown_connections 2024-05-21T21:55:24.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.907+0000 7fd9bd0a1b80 1 --2- 172.21.15.151:0/2713376789 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55b821d23c60 0x55b821d25e30 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:24.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.907+0000 7fd9bd0a1b80 1 --2- 172.21.15.151:0/2713376789 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b821d118f0 0x55b821d14f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:24.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.907+0000 7fd9bd0a1b80 1 --2- 172.21.15.151:0/2713376789 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b821d0ba80 0x55b821d176a0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:24.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.907+0000 7fd995600700 0 client.25111 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T21:55:24.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.907+0000 7fd9bd0a1b80 1 --2- 172.21.15.151:0/2713376789 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b821d10fd0 0x55b821d14a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:24.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.907+0000 7fd9bd0a1b80 1 -- 172.21.15.151:0/2713376789 >> 172.21.15.151:0/2713376789 conn(0x55b821c39b90 msgr2=0x55b821d0a450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:55:24.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.907+0000 7fd9bd0a1b80 1 -- 172.21.15.151:0/2713376789 shutdown_connections 2024-05-21T21:55:24.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.907+0000 7fd9bd0a1b80 1 -- 172.21.15.151:0/2713376789 wait complete. 2024-05-21T21:55:24.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.907+0000 7fd9bd0a1b80 20 client.25111 trim_cache size 0 max 16384 2024-05-21T21:55:24.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.930+0000 7fd9bd0a1b80 1 Processor -- start 2024-05-21T21:55:24.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.930+0000 7fd9bd0a1b80 1 -- start start 2024-05-21T21:55:24.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.930+0000 7fd9bd0a1b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b821d0ba80 0x55b821d0eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:24.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.930+0000 7fd9bd0a1b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b821d10fd0 0x55b821d0f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:24.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.930+0000 7fd9bd0a1b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b821d118f0 0x55b821d05be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:24.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.930+0000 7fd9bd0a1b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55b821d24170 con 0x55b821d0ba80 2024-05-21T21:55:24.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.930+0000 7fd9bd0a1b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55b821d24840 con 0x55b821d118f0 2024-05-21T21:55:24.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.930+0000 7fd9bd0a1b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55b821d24f10 con 0x55b821d10fd0 2024-05-21T21:55:24.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.931+0000 7fd9abe00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b821d118f0 0x55b821d05be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:24.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.931+0000 7fd9ab400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b821d0ba80 0x55b821d0eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:24.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.931+0000 7fd9abe00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b821d118f0 0x55b821d05be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:56546/0 (socket says 172.21.15.151:56546) 2024-05-21T21:55:24.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.931+0000 7fd9abe00700 1 -- 172.21.15.151:0/1239551499 learned_addr learned my addr 172.21.15.151:0/1239551499 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:55:24.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.931+0000 7fd9aaa00700 1 --2- 172.21.15.151:0/1239551499 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b821d10fd0 0x55b821d0f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:24.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.931+0000 7fd9ab400700 1 -- 172.21.15.151:0/1239551499 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b821d10fd0 msgr2=0x55b821d0f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:55:24.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.931+0000 7fd9ab400700 1 --2- 172.21.15.151:0/1239551499 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b821d10fd0 0x55b821d0f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:24.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.931+0000 7fd9ab400700 1 -- 172.21.15.151:0/1239551499 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b821d118f0 msgr2=0x55b821d05be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:55:24.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.931+0000 7fd9ab400700 1 --2- 172.21.15.151:0/1239551499 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b821d118f0 0x55b821d05be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:24.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.931+0000 7fd9ab400700 1 -- 172.21.15.151:0/1239551499 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b821d02570 con 0x55b821d0ba80 2024-05-21T21:55:24.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.931+0000 7fd9abe00700 1 --2- 172.21.15.151:0/1239551499 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b821d118f0 0x55b821d05be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:55:24.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.931+0000 7fd9aaa00700 1 --2- 172.21.15.151:0/1239551499 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b821d10fd0 0x55b821d0f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:55:24.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.932+0000 7fd9ab400700 1 --2- 172.21.15.151:0/1239551499 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b821d0ba80 0x55b821d0eb20 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd99c009230 tx=0x7fd99c005000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a790bdcf810d7df server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:55:24.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.932+0000 7fd997e00700 1 -- 172.21.15.151:0/1239551499 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd99c0a9120 con 0x55b821d0ba80 2024-05-21T21:55:24.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.932+0000 7fd997e00700 1 -- 172.21.15.151:0/1239551499 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd99c0a94c0 con 0x55b821d0ba80 2024-05-21T21:55:24.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.932+0000 7fd997e00700 1 -- 172.21.15.151:0/1239551499 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd99c0a5560 con 0x55b821d0ba80 2024-05-21T21:55:24.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.932+0000 7fd9bd0a1b80 1 -- 172.21.15.151:0/1239551499 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b821d0ba80 msgr2=0x55b821d0eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:24.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.933+0000 7fd9bd0a1b80 1 --2- 172.21.15.151:0/1239551499 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b821d0ba80 0x55b821d0eb20 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd99c009230 tx=0x7fd99c005000 comp rx=0 tx=0).stop 2024-05-21T21:55:24.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.933+0000 7fd9bd0a1b80 1 -- 172.21.15.151:0/1239551499 shutdown_connections 2024-05-21T21:55:24.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.933+0000 7fd9bd0a1b80 1 --2- 172.21.15.151:0/1239551499 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b821d118f0 0x55b821d05be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:24.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.933+0000 7fd9bd0a1b80 1 --2- 172.21.15.151:0/1239551499 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b821d0ba80 0x55b821d0eb20 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:24.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.933+0000 7fd9bd0a1b80 1 --2- 172.21.15.151:0/1239551499 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b821d10fd0 0x55b821d0f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:24.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.933+0000 7fd9bd0a1b80 1 -- 172.21.15.151:0/1239551499 >> 172.21.15.151:0/1239551499 conn(0x55b821c39b90 msgr2=0x55b821cfff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:55:24.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.933+0000 7fd9bd0a1b80 1 -- 172.21.15.151:0/1239551499 shutdown_connections 2024-05-21T21:55:24.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.933+0000 7fd9bd0a1b80 1 -- 172.21.15.151:0/1239551499 wait complete. 2024-05-21T21:55:24.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.933+0000 7fd9bd0a1b80 1 -- start start 2024-05-21T21:55:24.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.933+0000 7fd9bd0a1b80 1 Processor -- start 2024-05-21T21:55:24.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.934+0000 7fd9bd0a1b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b821d0ba80 0x55b821d2f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:24.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.934+0000 7fd9bd0a1b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b821d10fd0 0x55b821d2f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:24.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.934+0000 7fd9bd0a1b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b821d118f0 0x55b821d2a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:24.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.934+0000 7fd9bd0a1b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55b821d25cb0 con 0x55b821d0ba80 2024-05-21T21:55:24.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.934+0000 7fd9bd0a1b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55b821d255e0 con 0x55b821d118f0 2024-05-21T21:55:24.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.934+0000 7fd9bd0a1b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55b821d26380 con 0x55b821d10fd0 2024-05-21T21:55:24.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.934+0000 7fd9bd0a1b80 10 client.0 _ll_register_callbacks cb 0x55b821c03c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:55:24.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.934+0000 7fd9abe00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b821d118f0 0x55b821d2a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:24.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.934+0000 7fd9abe00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b821d118f0 0x55b821d2a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:56552/0 (socket says 172.21.15.151:56552) 2024-05-21T21:55:24.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.934+0000 7fd9abe00700 1 -- 172.21.15.151:0/3247887428 learned_addr learned my addr 172.21.15.151:0/3247887428 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:55:24.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.934+0000 7fd9aaa00700 1 --2- 172.21.15.151:0/3247887428 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b821d10fd0 0x55b821d2f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:24.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.935+0000 7fd9abe00700 1 -- 172.21.15.151:0/3247887428 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b821d10fd0 msgr2=0x55b821d2f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:55:24.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.935+0000 7fd9abe00700 1 --2- 172.21.15.151:0/3247887428 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b821d10fd0 0x55b821d2f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:24.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.935+0000 7fd9abe00700 1 -- 172.21.15.151:0/3247887428 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b821d0ba80 msgr2=0x55b821d2f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:55:24.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.935+0000 7fd9abe00700 1 --2- 172.21.15.151:0/3247887428 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b821d0ba80 0x55b821d2f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:24.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.935+0000 7fd9abe00700 1 -- 172.21.15.151:0/3247887428 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b821d17290 con 0x55b821d118f0 2024-05-21T21:55:24.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.935+0000 7fd9abe00700 1 --2- 172.21.15.151:0/3247887428 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b821d118f0 0x55b821d2a260 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd9a4007780 tx=0x7fd9a40099e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9af77c7d70409f47 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:55:24.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.935+0000 7fd995600700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-21T21:55:24.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.936+0000 7fd995600700 1 -- 172.21.15.151:0/3247887428 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a4032070 con 0x55b821d118f0 2024-05-21T21:55:24.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.936+0000 7fd995600700 1 -- 172.21.15.151:0/3247887428 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fd9a4020650 con 0x55b821d118f0 2024-05-21T21:55:24.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.936+0000 7fd9bd0a1b80 1 -- 172.21.15.151:0/3247887428 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd99c0a94c0 con 0x55b821d118f0 2024-05-21T21:55:24.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.936+0000 7fd995600700 1 -- 172.21.15.151:0/3247887428 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd9a403e050 con 0x55b821d118f0 2024-05-21T21:55:24.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.936+0000 7fd9bd0a1b80 10 client.25124 fetch_fsmap learned FSMap version 28 2024-05-21T21:55:24.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.936+0000 7fd9bd0a1b80 1 -- 172.21.15.151:0/3247887428 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b821d90740 con 0x55b821d118f0 2024-05-21T21:55:24.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.936+0000 7fd995600700 1 -- 172.21.15.151:0/3247887428 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd99c0a94c0 con 0x55b821d118f0 2024-05-21T21:55:24.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.937+0000 7fd995600700 1 -- 172.21.15.151:0/3247887428 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fd9a4011070 con 0x55b821d118f0 2024-05-21T21:55:24.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.937+0000 7fd9bd0a1b80 10 client.25124 fetch_fsmap finished waiting for FSMap version 28 2024-05-21T21:55:24.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.937+0000 7fd9bd0a1b80 10 client.25124 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-21T21:55:24.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.937+0000 7fd9bd0a1b80 20 client.25124 populate_metadata read hostname 'smithi151' 2024-05-21T21:55:24.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.937+0000 7fd9bd0a1b80 1 --2- 172.21.15.151:0/3247887428 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55b821d953d0 0x55b821d940e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:24.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.937+0000 7fd9bd0a1b80 4 client.25124 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-21T21:55:24.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.937+0000 7fd9bd0a1b80 1 -- 172.21.15.151:0/3247887428 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "status"}) -- 0x55b821adc200 con 0x55b821d953d0 2024-05-21T21:55:24.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.937+0000 7fd9ab400700 1 --2- 172.21.15.151:0/3247887428 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55b821d953d0 0x55b821d940e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:24.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.938+0000 7fd9ab400700 1 --2- 172.21.15.151:0/3247887428 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55b821d953d0 0x55b821d940e0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9438a883d6d023f server_cookie=a85db9cc212dd2dc in_seq=0 out_seq=0 2024-05-21T21:55:24.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.938+0000 7fd995600700 10 client.25124 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T21:55:24.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.938+0000 7fd995600700 1 -- 172.21.15.151:0/3247887428 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55b821adc200 con 0x55b821d953d0 2024-05-21T21:55:24.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.938+0000 7fd995600700 10 client.25124 handle_command_reply: tid=0 2024-05-21T21:55:24.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.938+0000 7fd9bd0a1b80 1 client.25124 shutdown 2024-05-21T21:55:24.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.938+0000 7fd9bd0a1b80 1 -- 172.21.15.151:0/3247887428 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b821d118f0 msgr2=0x55b821d2a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:24.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.938+0000 7fd9bd0a1b80 1 --2- 172.21.15.151:0/3247887428 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b821d118f0 0x55b821d2a260 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd9a4007780 tx=0x7fd9a40099e0 comp rx=0 tx=0).stop 2024-05-21T21:55:24.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.938+0000 7fd9bd0a1b80 1 -- 172.21.15.151:0/3247887428 shutdown_connections 2024-05-21T21:55:24.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.938+0000 7fd9bd0a1b80 1 --2- 172.21.15.151:0/3247887428 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55b821d953d0 0x55b821d940e0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:24.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.938+0000 7fd9bd0a1b80 1 --2- 172.21.15.151:0/3247887428 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b821d118f0 0x55b821d2a260 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:24.942 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.938+0000 7fd9bd0a1b80 1 --2- 172.21.15.151:0/3247887428 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b821d0ba80 0x55b821d2f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:24.942 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.938+0000 7fd9bd0a1b80 1 --2- 172.21.15.151:0/3247887428 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b821d10fd0 0x55b821d2f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:24.942 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.938+0000 7fd9bd0a1b80 1 -- 172.21.15.151:0/3247887428 >> 172.21.15.151:0/3247887428 conn(0x55b821c39b90 msgr2=0x55b821d0eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:55:24.942 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.938+0000 7fd995600700 0 client.25124 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T21:55:24.942 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.939+0000 7fd9bd0a1b80 1 -- 172.21.15.151:0/3247887428 shutdown_connections 2024-05-21T21:55:24.942 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.939+0000 7fd9bd0a1b80 1 -- 172.21.15.151:0/3247887428 wait complete. 2024-05-21T21:55:24.942 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.939+0000 7fd9bd0a1b80 20 client.25124 trim_cache size 0 max 16384 2024-05-21T21:55:24.942 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-05-21T21:55:24.942 INFO:teuthology.orchestra.run.smithi151.stdout: "cluster_fsid": "c8c7a2e6-17ba-11ef-bc9a-c7b262605968", 2024-05-21T21:55:24.943 INFO:teuthology.orchestra.run.smithi151.stdout: "whoami": 0, 2024-05-21T21:55:24.943 INFO:teuthology.orchestra.run.smithi151.stdout: "id": 24530, 2024-05-21T21:55:24.943 INFO:teuthology.orchestra.run.smithi151.stdout: "want_state": "up:active", 2024-05-21T21:55:24.943 INFO:teuthology.orchestra.run.smithi151.stdout: "state": "up:active", 2024-05-21T21:55:24.943 INFO:teuthology.orchestra.run.smithi151.stdout: "fs_name": "cephfs", 2024-05-21T21:55:24.943 INFO:teuthology.orchestra.run.smithi151.stdout: "rank_uptime": 245.00212789099999, 2024-05-21T21:55:24.943 INFO:teuthology.orchestra.run.smithi151.stdout: "mdsmap_epoch": 28, 2024-05-21T21:55:24.943 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_epoch": 82, 2024-05-21T21:55:24.943 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_epoch_barrier": 80, 2024-05-21T21:55:24.943 INFO:teuthology.orchestra.run.smithi151.stdout: "uptime": 265.94629271299999 2024-05-21T21:55:24.943 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-05-21T21:55:24.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.940+0000 7fd9b1200700 1 -- 172.21.15.151:0/3273586383 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fd9900b2900 msgr2=0x7fd9900b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:24.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.940+0000 7fd9b1200700 1 --2- 172.21.15.151:0/3273586383 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fd9900b2900 0x7fd9900b4dc0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd9a4006130 tx=0x7fd9a400b720 comp rx=0 tx=0).stop 2024-05-21T21:55:24.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.940+0000 7fd9b1200700 1 -- 172.21.15.151:0/3273586383 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9ac105650 msgr2=0x7fd9ac1a1920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:24.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.940+0000 7fd9b1200700 1 --2- 172.21.15.151:0/3273586383 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9ac105650 0x7fd9ac1a1920 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd99c00e9c0 tx=0x7fd99c00ecd0 comp rx=0 tx=0).stop 2024-05-21T21:55:24.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.940+0000 7fd9b1200700 1 -- 172.21.15.151:0/3273586383 shutdown_connections 2024-05-21T21:55:24.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.940+0000 7fd9b1200700 1 --2- 172.21.15.151:0/3273586383 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fd9900b2900 0x7fd9900b4dc0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:24.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.940+0000 7fd9b1200700 1 --2- 172.21.15.151:0/3273586383 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd9ac10b650 0x7fd9ac19bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:24.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.940+0000 7fd9b1200700 1 --2- 172.21.15.151:0/3273586383 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9ac105650 0x7fd9ac1a1920 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:24.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.940+0000 7fd9b1200700 1 --2- 172.21.15.151:0/3273586383 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd9ac106010 0x7fd9ac1a1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:24.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.940+0000 7fd9b1200700 1 -- 172.21.15.151:0/3273586383 >> 172.21.15.151:0/3273586383 conn(0x7fd9ac101120 msgr2=0x7fd9ac1016e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:55:24.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.940+0000 7fd9b1200700 1 -- 172.21.15.151:0/3273586383 shutdown_connections 2024-05-21T21:55:24.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:24.940+0000 7fd9b1200700 1 -- 172.21.15.151:0/3273586383 wait complete. 2024-05-21T21:55:24.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:24 smithi184 ceph-mon[36942]: pgmap v439: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-21T21:55:25.864 INFO:teuthology.run_tasks:Running task sequential... 2024-05-21T21:55:25.875 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-05-21T21:55:25.875 INFO:teuthology.task.sequential:In sequential, running task print... 2024-05-21T21:55:25.876 INFO:teuthology.task.print:Enabling mgr modules 2024-05-21T21:55:25.876 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-05-21T21:55:25.877 INFO:teuthology.task.exec:Executing custom commands... 2024-05-21T21:55:25.877 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi151.front.sepia.ceph.com 2024-05-21T21:55:25.877 DEBUG:teuthology.orchestra.run.smithi151:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-05-21T21:55:26.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:26.079+0000 7f9b89000640 1 -- 172.21.15.151:0/1927341360 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9b84105ed0 msgr2=0x7f9b84106350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:26.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:26.079+0000 7f9b89000640 1 --2- 172.21.15.151:0/1927341360 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9b84105ed0 0x7f9b84106350 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9b70009900 tx=0x7f9b7002a3f0 comp rx=0 tx=0).stop 2024-05-21T21:55:26.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:26.079+0000 7f9b89000640 1 -- 172.21.15.151:0/1927341360 shutdown_connections 2024-05-21T21:55:26.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:26.079+0000 7f9b89000640 1 --2- 172.21.15.151:0/1927341360 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b84106890 0x7f9b8410d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:26.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:26.079+0000 7f9b89000640 1 --2- 172.21.15.151:0/1927341360 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9b84105ed0 0x7f9b84106350 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:26.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:26.079+0000 7f9b89000640 1 --2- 172.21.15.151:0/1927341360 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b84104ce0 0x7f9b84105100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:26.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:26.079+0000 7f9b89000640 1 -- 172.21.15.151:0/1927341360 >> 172.21.15.151:0/1927341360 conn(0x7f9b84100270 msgr2=0x7f9b84102690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:55:26.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:26.079+0000 7f9b89000640 1 -- 172.21.15.151:0/1927341360 shutdown_connections 2024-05-21T21:55:26.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:26.079+0000 7f9b89000640 1 -- 172.21.15.151:0/1927341360 wait complete. 2024-05-21T21:55:26.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:26.081+0000 7f9b89000640 1 Processor -- start 2024-05-21T21:55:26.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:26.081+0000 7f9b89000640 1 -- start start 2024-05-21T21:55:26.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:26.081+0000 7f9b89000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9b84104ce0 0x7f9b8419ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:26.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:26.081+0000 7f9b89000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b84105ed0 0x7f9b8419b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:26.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:26.081+0000 7f9b89000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b84106890 0x7f9b841a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:26.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:26.081+0000 7f9b89000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9b84110810 con 0x7f9b84104ce0 2024-05-21T21:55:26.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:26.081+0000 7f9b89000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9b84110690 con 0x7f9b84106890 2024-05-21T21:55:26.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:26.081+0000 7f9b89000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9b84110990 con 0x7f9b84105ed0 2024-05-21T21:55:26.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:26.081+0000 7f9b81e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9b84104ce0 0x7f9b8419ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:26.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:26.081+0000 7f9b81400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b84105ed0 0x7f9b8419b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:26.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:26.082+0000 7f9b81400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b84105ed0 0x7f9b8419b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:33072/0 (socket says 172.21.15.151:33072) 2024-05-21T21:55:26.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:26.082+0000 7f9b81400640 1 -- 172.21.15.151:0/2079910281 learned_addr learned my addr 172.21.15.151:0/2079910281 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:55:26.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:26.082+0000 7f9b81e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9b84104ce0 0x7f9b8419ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:40196/0 (socket says 172.21.15.151:40196) 2024-05-21T21:55:26.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:26.082+0000 7f9b82800640 1 --2- 172.21.15.151:0/2079910281 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b84106890 0x7f9b841a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:26.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:26.082+0000 7f9b81e00640 1 -- 172.21.15.151:0/2079910281 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b84105ed0 msgr2=0x7f9b8419b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:26.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:26.082+0000 7f9b81e00640 1 --2- 172.21.15.151:0/2079910281 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b84105ed0 0x7f9b8419b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:26.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:26.082+0000 7f9b81e00640 1 -- 172.21.15.151:0/2079910281 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b84106890 msgr2=0x7f9b841a2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:26.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:26.082+0000 7f9b81e00640 1 --2- 172.21.15.151:0/2079910281 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b84106890 0x7f9b841a2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:26.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:26.082+0000 7f9b81e00640 1 -- 172.21.15.151:0/2079910281 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b841a2a80 con 0x7f9b84104ce0 2024-05-21T21:55:26.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:26.082+0000 7f9b81e00640 1 --2- 172.21.15.151:0/2079910281 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9b84104ce0 0x7f9b8419ae60 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f9b7400e9c0 tx=0x7f9b7400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:55:26.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:26.083+0000 7f9b7aa00640 1 -- 172.21.15.151:0/2079910281 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b7400ccc0 con 0x7f9b84104ce0 2024-05-21T21:55:26.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:26.083+0000 7f9b89000640 1 -- 172.21.15.151:0/2079910281 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b841a2d70 con 0x7f9b84104ce0 2024-05-21T21:55:26.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:26.083+0000 7f9b7aa00640 1 -- 172.21.15.151:0/2079910281 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b740035f0 con 0x7f9b84104ce0 2024-05-21T21:55:26.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:26.083+0000 7f9b7aa00640 1 -- 172.21.15.151:0/2079910281 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b7400ce60 con 0x7f9b84104ce0 2024-05-21T21:55:26.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:26.083+0000 7f9b89000640 1 -- 172.21.15.151:0/2079910281 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b841a31d0 con 0x7f9b84104ce0 2024-05-21T21:55:26.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:26.084+0000 7f9b89000640 1 -- 172.21.15.151:0/2079910281 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b8410c000 con 0x7f9b84104ce0 2024-05-21T21:55:26.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:26.086+0000 7f9b7aa00640 1 -- 172.21.15.151:0/2079910281 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 149653+0+0 (secure 0 0 0) 0x7f9b74004000 con 0x7f9b84104ce0 2024-05-21T21:55:26.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:26.088+0000 7f9b7aa00640 1 --2- 172.21.15.151:0/2079910281 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f9b680b2920 0x7f9b680b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:26.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:26.088+0000 7f9b7aa00640 1 -- 172.21.15.151:0/2079910281 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f9b740e25f0 con 0x7f9b84104ce0 2024-05-21T21:55:26.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:26.088+0000 7f9b81400640 1 --2- 172.21.15.151:0/2079910281 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f9b680b2920 0x7f9b680b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:26.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:26.089+0000 7f9b81400640 1 --2- 172.21.15.151:0/2079910281 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f9b680b2920 0x7f9b680b4de0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f9b70009230 tx=0x7f9b700058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:55:26.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:26.092+0000 7f9b7aa00640 1 -- 172.21.15.151:0/2079910281 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9b740aa2d0 con 0x7f9b84104ce0 2024-05-21T21:55:26.354 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:26.353+0000 7f9b89000640 1 -- 172.21.15.151:0/2079910281 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f9b840636f0 con 0x7f9b84104ce0 2024-05-21T21:55:26.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:26.525+0000 7f9b7aa00640 1 -- 172.21.15.151:0/2079910281 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7f9b740a9bd0 con 0x7f9b84104ce0 2024-05-21T21:55:26.529 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:26.529+0000 7f9b89000640 1 -- 172.21.15.151:0/2079910281 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f9b680b2920 msgr2=0x7f9b680b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:26.529 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:26.529+0000 7f9b89000640 1 --2- 172.21.15.151:0/2079910281 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f9b680b2920 0x7f9b680b4de0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f9b70009230 tx=0x7f9b700058d0 comp rx=0 tx=0).stop 2024-05-21T21:55:26.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:26.530+0000 7f9b89000640 1 -- 172.21.15.151:0/2079910281 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9b84104ce0 msgr2=0x7f9b8419ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:26.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:26.530+0000 7f9b89000640 1 --2- 172.21.15.151:0/2079910281 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9b84104ce0 0x7f9b8419ae60 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f9b7400e9c0 tx=0x7f9b7400ee80 comp rx=0 tx=0).stop 2024-05-21T21:55:26.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:26.530+0000 7f9b89000640 1 -- 172.21.15.151:0/2079910281 shutdown_connections 2024-05-21T21:55:26.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:26.530+0000 7f9b89000640 1 --2- 172.21.15.151:0/2079910281 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f9b680b2920 0x7f9b680b4de0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:26.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:26.530+0000 7f9b89000640 1 --2- 172.21.15.151:0/2079910281 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b84106890 0x7f9b841a2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:26.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:26.530+0000 7f9b89000640 1 --2- 172.21.15.151:0/2079910281 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b84105ed0 0x7f9b8419b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:26.531 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:26.530+0000 7f9b89000640 1 --2- 172.21.15.151:0/2079910281 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9b84104ce0 0x7f9b8419ae60 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:26.531 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:26.530+0000 7f9b89000640 1 -- 172.21.15.151:0/2079910281 >> 172.21.15.151:0/2079910281 conn(0x7f9b84100270 msgr2=0x7f9b841091d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:55:26.531 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:26.531+0000 7f9b89000640 1 -- 172.21.15.151:0/2079910281 shutdown_connections 2024-05-21T21:55:26.531 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:26.531+0000 7f9b89000640 1 -- 172.21.15.151:0/2079910281 wait complete. 2024-05-21T21:55:26.561 INFO:teuthology.run_tasks:Running task kclient... 2024-05-21T21:55:26.591 INFO:tasks.kclient:Mounting kernel clients... 2024-05-21T21:55:26.591 INFO:tasks.kclient:config is {'client.0': {}} 2024-05-21T21:55:26.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-21T21:55:26.592 DEBUG:teuthology.orchestra.run.smithi151:> ip netns list 2024-05-21T21:55:26.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-21T21:55:26.642 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip link delete ceph-brx 2024-05-21T21:55:26.711 INFO:teuthology.orchestra.run.smithi151.stderr:Cannot find device "ceph-brx" 2024-05-21T21:55:26.713 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-21T21:55:26.714 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=legacy', 'wsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-05-21T21:55:26.714 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-21T21:55:26.714 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-21T21:55:26.714 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph fs ls 2024-05-21T21:55:26.853 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:26 smithi151 ceph-mon[31623]: pgmap v440: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-21T21:55:26.853 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:26 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/2079910281' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-05-21T21:55:26.853 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:55:26 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: ignoring --setuser ceph since I am not root 2024-05-21T21:55:26.854 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:55:26 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: ignoring --setgroup ceph since I am not root 2024-05-21T21:55:26.893 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:26 smithi184 ceph-mon[36942]: pgmap v440: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-21T21:55:26.893 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:26 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/2079910281' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-05-21T21:55:26.893 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 21 21:55:26 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-z[38659]: ignoring --setuser ceph since I am not root 2024-05-21T21:55:26.894 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 21 21:55:26 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-z[38659]: ignoring --setgroup ceph since I am not root 2024-05-21T21:55:26.894 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 21 21:55:26 smithi184 ceph-mgr[38701]: -- 172.21.15.184:0/3772030808 <== mon.1 v2:172.21.15.184:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560c142065a0 con 0x560c141dac00 2024-05-21T21:55:26.894 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 21 21:55:26 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-z[38659]: 2024-05-21T21:55:26.709+0000 7fd09350d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-21T21:55:26.900 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:26 smithi164 ceph-mon[36269]: pgmap v440: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-21T21:55:26.900 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:26 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/2079910281' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-05-21T21:55:26.900 INFO:journalctl@ceph.mgr.y.smithi164.stdout:May 21 21:55:26 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-y[37650]: ignoring --setuser ceph since I am not root 2024-05-21T21:55:26.900 INFO:journalctl@ceph.mgr.y.smithi164.stdout:May 21 21:55:26 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-y[37650]: ignoring --setgroup ceph since I am not root 2024-05-21T21:55:26.900 INFO:journalctl@ceph.mgr.y.smithi164.stdout:May 21 21:55:26 smithi164 ceph-mgr[37654]: -- 172.21.15.164:0/1535794399 <== mon.1 v2:172.21.15.184:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cebe1745a0 con 0x55cebe148800 2024-05-21T21:55:26.900 INFO:journalctl@ceph.mgr.y.smithi164.stdout:May 21 21:55:26 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-y[37650]: 2024-05-21T21:55:26.714+0000 7f8c55776200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-21T21:55:26.900 INFO:journalctl@ceph.mgr.y.smithi164.stdout:May 21 21:55:26 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-y[37650]: 2024-05-21T21:55:26.899+0000 7f8c55776200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-21T21:55:27.113 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:55:26 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:55:26.852+0000 7fbf14b61200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-21T21:55:27.122 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:55:27.236 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 21 21:55:26 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-z[38659]: 2024-05-21T21:55:26.892+0000 7fd09350d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-21T21:55:27.236 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 21 21:55:26 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-z[38659]: 2024-05-21T21:55:26.993+0000 7fd09350d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-21T21:55:27.285 INFO:journalctl@ceph.mgr.y.smithi164.stdout:May 21 21:55:27 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-y[37650]: 2024-05-21T21:55:27.002+0000 7f8c55776200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-21T21:55:27.446 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:55:27 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:55:27.112+0000 7fbf14b61200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-21T21:55:27.446 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:55:27 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:55:27.250+0000 7fbf14b61200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-21T21:55:27.550 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 21 21:55:27 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-z[38659]: 2024-05-21T21:55:27.299+0000 7fd09350d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-21T21:55:27.550 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 21 21:55:27 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-z[38659]: 2024-05-21T21:55:27.384+0000 7fd09350d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-21T21:55:27.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:27 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/2079910281' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-05-21T21:55:27.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:27 smithi151 ceph-mon[31623]: mgrmap e19: x(active, since 12m), standbys: z, y 2024-05-21T21:55:27.703 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:55:27 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:55:27.657+0000 7fbf14b61200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-21T21:55:27.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:27 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/2079910281' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-05-21T21:55:27.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:27 smithi164 ceph-mon[36269]: mgrmap e19: x(active, since 12m), standbys: z, y 2024-05-21T21:55:27.786 INFO:journalctl@ceph.mgr.y.smithi164.stdout:May 21 21:55:27 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-y[37650]: 2024-05-21T21:55:27.309+0000 7f8c55776200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-21T21:55:27.787 INFO:journalctl@ceph.mgr.y.smithi164.stdout:May 21 21:55:27 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-y[37650]: 2024-05-21T21:55:27.395+0000 7f8c55776200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-21T21:55:27.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:27 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/2079910281' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-05-21T21:55:27.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:27 smithi184 ceph-mon[36942]: mgrmap e19: x(active, since 12m), standbys: z, y 2024-05-21T21:55:28.058 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:55:27 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:55:27.778+0000 7fbf14b61200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-21T21:55:28.895 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 21 21:55:28 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-z[38659]: 2024-05-21T21:55:28.556+0000 7fd09350d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-21T21:55:28.895 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 21 21:55:28 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-z[38659]: 2024-05-21T21:55:28.723+0000 7fd09350d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-21T21:55:28.920 INFO:journalctl@ceph.mgr.y.smithi164.stdout:May 21 21:55:28 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-y[37650]: 2024-05-21T21:55:28.578+0000 7f8c55776200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-21T21:55:28.921 INFO:journalctl@ceph.mgr.y.smithi164.stdout:May 21 21:55:28 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-y[37650]: 2024-05-21T21:55:28.748+0000 7f8c55776200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-21T21:55:28.921 INFO:journalctl@ceph.mgr.y.smithi164.stdout:May 21 21:55:28 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-y[37650]: 2024-05-21T21:55:28.920+0000 7f8c55776200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-21T21:55:29.182 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 21 21:55:28 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-z[38659]: 2024-05-21T21:55:28.894+0000 7fd09350d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-21T21:55:29.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:29.218+0000 7f74a7400700 1 -- 172.21.15.151:0/1763881008 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74a01037f0 msgr2=0x7f74a0103bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:29.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:29.218+0000 7f74a7400700 1 --2- 172.21.15.151:0/1763881008 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74a01037f0 0x7f74a0103bd0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f74940099c0 tx=0x7f7494009cd0 comp rx=0 tx=0).stop 2024-05-21T21:55:29.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:29.218+0000 7f74a7400700 1 -- 172.21.15.151:0/1763881008 shutdown_connections 2024-05-21T21:55:29.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:29.218+0000 7f74a7400700 1 --2- 172.21.15.151:0/1763881008 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74a01037f0 0x7f74a0103bd0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:29.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:29.218+0000 7f74a7400700 1 --2- 172.21.15.151:0/1763881008 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74a01088f0 0x7f74a0110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:29.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:29.218+0000 7f74a7400700 1 --2- 172.21.15.151:0/1763881008 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74a01041a0 0x7f74a01081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:29.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:29.218+0000 7f74a7400700 1 -- 172.21.15.151:0/1763881008 >> 172.21.15.151:0/1763881008 conn(0x7f74a007a6a0 msgr2=0x7f74a007aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:55:29.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:29.218+0000 7f74a7400700 1 -- 172.21.15.151:0/1763881008 shutdown_connections 2024-05-21T21:55:29.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:29.219+0000 7f74a7400700 1 -- 172.21.15.151:0/1763881008 wait complete. 2024-05-21T21:55:29.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:29.220+0000 7f74a7400700 1 Processor -- start 2024-05-21T21:55:29.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:29.220+0000 7f74a7400700 1 -- start start 2024-05-21T21:55:29.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:29.220+0000 7f74a7400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74a01037f0 0x7f74a019bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:29.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:29.220+0000 7f74a7400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74a01041a0 0x7f74a019c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:29.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:29.220+0000 7f74a7400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74a01088f0 0x7f74a01a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:29.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:29.220+0000 7f74a7400700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f74a0113710 con 0x7f74a01088f0 2024-05-21T21:55:29.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:29.220+0000 7f74a7400700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f74a0113590 con 0x7f74a01041a0 2024-05-21T21:55:29.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:29.220+0000 7f74a7400700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f74a0113890 con 0x7f74a01037f0 2024-05-21T21:55:29.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:29.221+0000 7f749fe00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74a01041a0 0x7f74a019c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:29.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:29.221+0000 7f74a4a00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74a01037f0 0x7f74a019bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:29.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:29.221+0000 7f749fe00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74a01041a0 0x7f74a019c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:60490/0 (socket says 172.21.15.151:60490) 2024-05-21T21:55:29.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:29.221+0000 7f749fe00700 1 -- 172.21.15.151:0/3929697625 learned_addr learned my addr 172.21.15.151:0/3929697625 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:55:29.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:29.221+0000 7f74a5400700 1 --2- 172.21.15.151:0/3929697625 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74a01088f0 0x7f74a01a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:29.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:29.221+0000 7f749fe00700 1 -- 172.21.15.151:0/3929697625 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74a01037f0 msgr2=0x7f74a019bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:29.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:29.221+0000 7f749fe00700 1 --2- 172.21.15.151:0/3929697625 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74a01037f0 0x7f74a019bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:29.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:29.221+0000 7f749fe00700 1 -- 172.21.15.151:0/3929697625 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74a01088f0 msgr2=0x7f74a01a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:29.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:29.221+0000 7f749fe00700 1 --2- 172.21.15.151:0/3929697625 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74a01088f0 0x7f74a01a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:29.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:29.221+0000 7f749fe00700 1 -- 172.21.15.151:0/3929697625 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74a01a0d90 con 0x7f74a01041a0 2024-05-21T21:55:29.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:29.221+0000 7f749fe00700 1 --2- 172.21.15.151:0/3929697625 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74a01041a0 0x7f74a019c2c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f748c00e9a0 tx=0x7f748c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:55:29.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:29.222+0000 7f749d600700 1 -- 172.21.15.151:0/3929697625 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f748c00cbd0 con 0x7f74a01041a0 2024-05-21T21:55:29.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:29.222+0000 7f749d600700 1 -- 172.21.15.151:0/3929697625 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f748c00cd70 con 0x7f74a01041a0 2024-05-21T21:55:29.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:29.222+0000 7f74a7400700 1 -- 172.21.15.151:0/3929697625 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74a01a1080 con 0x7f74a01041a0 2024-05-21T21:55:29.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:29.222+0000 7f749d600700 1 -- 172.21.15.151:0/3929697625 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f748c018680 con 0x7f74a01041a0 2024-05-21T21:55:29.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:29.222+0000 7f74a7400700 1 -- 172.21.15.151:0/3929697625 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74a01a8900 con 0x7f74a01041a0 2024-05-21T21:55:29.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:29.223+0000 7f74a7400700 1 -- 172.21.15.151:0/3929697625 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74a006a230 con 0x7f74a01041a0 2024-05-21T21:55:29.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:29.225+0000 7f749d600700 1 -- 172.21.15.151:0/3929697625 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 19) ==== 149670+0+0 (secure 0 0 0) 0x7f748c010430 con 0x7f74a01041a0 2024-05-21T21:55:29.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:29.226+0000 7f749d600700 1 --2- 172.21.15.151:0/3929697625 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f74880b2950 0x7f74880b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:29.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:29.226+0000 7f74a4a00700 1 -- 172.21.15.151:0/3929697625 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f74880b2950 msgr2=0x7f74880b4e10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.151:6800/2796622483 2024-05-21T21:55:29.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:29.227+0000 7f749d600700 1 -- 172.21.15.151:0/3929697625 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f748c0e24c0 con 0x7f74a01041a0 2024-05-21T21:55:29.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:29.227+0000 7f74a4a00700 1 --2- 172.21.15.151:0/3929697625 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f74880b2950 0x7f74880b4e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-21T21:55:29.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:29.231+0000 7f749d600700 1 -- 172.21.15.151:0/3929697625 <== mon.2 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f748c0aa1a0 con 0x7f74a01041a0 2024-05-21T21:55:29.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:29.428+0000 7f74a4a00700 1 -- 172.21.15.151:0/3929697625 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f74880b2950 msgr2=0x7f74880b4e10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.151:6800/2796622483 2024-05-21T21:55:29.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:29.428+0000 7f74a4a00700 1 --2- 172.21.15.151:0/3929697625 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f74880b2950 0x7f74880b4e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-21T21:55:29.478 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 21 21:55:29 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-z[38659]: 2024-05-21T21:55:29.181+0000 7fd09350d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-21T21:55:29.478 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 21 21:55:29 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-z[38659]: 2024-05-21T21:55:29.275+0000 7fd09350d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-21T21:55:29.508 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:29.507+0000 7f74a7400700 1 -- 172.21.15.151:0/3929697625 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f74a019d0e0 con 0x7f74a01041a0 2024-05-21T21:55:29.508 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:29.507+0000 7f749d600700 1 -- 172.21.15.151:0/3929697625 <== mon.2 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v28) ==== 53+0+73 (secure 0 0 0) 0x7f74a019d0e0 con 0x7f74a01041a0 2024-05-21T21:55:29.509 INFO:teuthology.orchestra.run.smithi151.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-21T21:55:29.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:29.511+0000 7f74a7400700 1 -- 172.21.15.151:0/3929697625 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f74880b2950 msgr2=0x7f74880b4e10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-21T21:55:29.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:29.511+0000 7f74a7400700 1 --2- 172.21.15.151:0/3929697625 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f74880b2950 0x7f74880b4e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:29.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:29.511+0000 7f74a7400700 1 -- 172.21.15.151:0/3929697625 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74a01041a0 msgr2=0x7f74a019c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:29.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:29.511+0000 7f74a7400700 1 --2- 172.21.15.151:0/3929697625 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74a01041a0 0x7f74a019c2c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f748c00e9a0 tx=0x7f748c00ecb0 comp rx=0 tx=0).stop 2024-05-21T21:55:29.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:29.511+0000 7f74a7400700 1 -- 172.21.15.151:0/3929697625 shutdown_connections 2024-05-21T21:55:29.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:29.511+0000 7f74a7400700 1 --2- 172.21.15.151:0/3929697625 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f74880b2950 0x7f74880b4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:29.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:29.511+0000 7f74a7400700 1 --2- 172.21.15.151:0/3929697625 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74a01088f0 0x7f74a01a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:29.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:29.511+0000 7f74a7400700 1 --2- 172.21.15.151:0/3929697625 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74a01037f0 0x7f74a019bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:29.514 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:29.511+0000 7f74a7400700 1 --2- 172.21.15.151:0/3929697625 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74a01041a0 0x7f74a019c2c0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:29.514 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:29.511+0000 7f74a7400700 1 -- 172.21.15.151:0/3929697625 >> 172.21.15.151:0/3929697625 conn(0x7f74a007a6a0 msgr2=0x7f74a0101530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:55:29.514 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:29.512+0000 7f74a7400700 1 -- 172.21.15.151:0/3929697625 shutdown_connections 2024-05-21T21:55:29.514 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:29.512+0000 7f74a7400700 1 -- 172.21.15.151:0/3929697625 wait complete. 2024-05-21T21:55:29.522 INFO:journalctl@ceph.mgr.y.smithi164.stdout:May 21 21:55:29 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-y[37650]: 2024-05-21T21:55:29.209+0000 7f8c55776200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-21T21:55:29.522 INFO:journalctl@ceph.mgr.y.smithi164.stdout:May 21 21:55:29 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-y[37650]: 2024-05-21T21:55:29.303+0000 7f8c55776200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-21T21:55:29.522 INFO:journalctl@ceph.mgr.y.smithi164.stdout:May 21 21:55:29 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-y[37650]: 2024-05-21T21:55:29.521+0000 7f8c55776200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-21T21:55:29.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:29 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/3929697625' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-21T21:55:29.703 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:55:29 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:55:29.370+0000 7fbf14b61200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-21T21:55:29.703 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:55:29 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:55:29.609+0000 7fbf14b61200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-21T21:55:29.735 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:29 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/3929697625' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-21T21:55:29.736 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 21 21:55:29 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-z[38659]: 2024-05-21T21:55:29.477+0000 7fd09350d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-21T21:55:29.736 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 21 21:55:29 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-z[38659]: 2024-05-21T21:55:29.670+0000 7fd09350d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-21T21:55:29.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:29 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/3929697625' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-21T21:55:29.786 INFO:journalctl@ceph.mgr.y.smithi164.stdout:May 21 21:55:29 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-y[37650]: 2024-05-21T21:55:29.717+0000 7f8c55776200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-21T21:55:30.203 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:55:29 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:55:29.831+0000 7fbf14b61200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-21T21:55:30.311 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-21T21:55:30.311 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-21T21:55:30.312 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi151.front.sepia.ceph.com 2024-05-21T21:55:30.312 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-21T21:55:30.312 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-21T21:55:30.312 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-21T21:55:30.312 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-21T21:55:30.312 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-21T21:55:30.312 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-21T21:55:30.312 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-21T21:55:30.312 DEBUG:teuthology.orchestra.run.smithi151:> ip addr 2024-05-21T21:55:30.357 INFO:teuthology.orchestra.run.smithi151.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-21T21:55:30.357 INFO:teuthology.orchestra.run.smithi151.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-21T21:55:30.357 INFO:teuthology.orchestra.run.smithi151.stdout: inet 127.0.0.1/8 scope host lo 2024-05-21T21:55:30.357 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-05-21T21:55:30.357 INFO:teuthology.orchestra.run.smithi151.stdout: inet6 ::1/128 scope host 2024-05-21T21:55:30.357 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-05-21T21:55:30.357 INFO:teuthology.orchestra.run.smithi151.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-21T21:55:30.358 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 0c:c4:7a:da:6e:0c brd ff:ff:ff:ff:ff:ff 2024-05-21T21:55:30.358 INFO:teuthology.orchestra.run.smithi151.stdout: altname enp7s0f0 2024-05-21T21:55:30.358 INFO:teuthology.orchestra.run.smithi151.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-21T21:55:30.358 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 0c:c4:7a:da:6e:0d brd ff:ff:ff:ff:ff:ff 2024-05-21T21:55:30.358 INFO:teuthology.orchestra.run.smithi151.stdout: altname enp7s0f1 2024-05-21T21:55:30.358 INFO:teuthology.orchestra.run.smithi151.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-21T21:55:30.358 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 0c:c4:7a:88:6f:62 brd ff:ff:ff:ff:ff:ff 2024-05-21T21:55:30.358 INFO:teuthology.orchestra.run.smithi151.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-21T21:55:30.358 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 0c:c4:7a:88:6f:63 brd ff:ff:ff:ff:ff:ff 2024-05-21T21:55:30.358 INFO:teuthology.orchestra.run.smithi151.stdout: inet 172.21.15.151/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-21T21:55:30.358 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft 41763sec preferred_lft 41763sec 2024-05-21T21:55:30.358 INFO:teuthology.orchestra.run.smithi151.stdout: inet6 fe80::ec4:7aff:fe88:6f63/64 scope link 2024-05-21T21:55:30.359 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-05-21T21:55:30.360 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-21T21:55:30.360 DEBUG:teuthology.orchestra.run.smithi151:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-21T21:55:30.360 DEBUG:teuthology.orchestra.run.smithi151:> set -e 2024-05-21T21:55:30.360 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip link add name ceph-brx type bridge 2024-05-21T21:55:30.360 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip addr flush dev ceph-brx 2024-05-21T21:55:30.360 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip link set ceph-brx up 2024-05-21T21:55:30.360 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-21T21:55:30.360 DEBUG:teuthology.orchestra.run.smithi151:> ') 2024-05-21T21:55:30.504 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-21T21:55:30.527 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:55:30 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:55:30.222+0000 7fbf14b61200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-21T21:55:30.527 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:55:30 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:55:30.352+0000 7fbf14b61200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-21T21:55:30.736 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 21 21:55:30 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-z[38659]: 2024-05-21T21:55:30.271+0000 7fd09350d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-21T21:55:30.736 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 21 21:55:30 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-z[38659]: 2024-05-21T21:55:30.491+0000 7fd09350d200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-21T21:55:30.786 INFO:journalctl@ceph.mgr.y.smithi164.stdout:May 21 21:55:30 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-y[37650]: 2024-05-21T21:55:30.320+0000 7f8c55776200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-21T21:55:30.786 INFO:journalctl@ceph.mgr.y.smithi164.stdout:May 21 21:55:30 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-y[37650]: 2024-05-21T21:55:30.540+0000 7f8c55776200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-21T21:55:30.794 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:55:30 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:55:30.618+0000 7fbf14b61200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-21T21:55:30.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-21T21:55:30.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-21T21:55:30.893 DEBUG:teuthology.orchestra.run.smithi151:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-21T21:55:30.956 INFO:teuthology.orchestra.run.smithi151.stdout:1 2024-05-21T21:55:30.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-21T21:55:30.959 DEBUG:teuthology.orchestra.run.smithi151:> route 2024-05-21T21:55:31.006 INFO:teuthology.orchestra.run.smithi151.stdout:Kernel IP routing table 2024-05-21T21:55:31.006 INFO:teuthology.orchestra.run.smithi151.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-21T21:55:31.007 INFO:teuthology.orchestra.run.smithi151.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-21T21:55:31.007 INFO:teuthology.orchestra.run.smithi151.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-21T21:55:31.007 INFO:teuthology.orchestra.run.smithi151.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-21T21:55:31.007 DEBUG:teuthology.orchestra.run.smithi151:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-21T21:55:31.007 DEBUG:teuthology.orchestra.run.smithi151:> set -e 2024-05-21T21:55:31.007 DEBUG:teuthology.orchestra.run.smithi151:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-21T21:55:31.007 DEBUG:teuthology.orchestra.run.smithi151:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-21T21:55:31.008 DEBUG:teuthology.orchestra.run.smithi151:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-21T21:55:31.008 DEBUG:teuthology.orchestra.run.smithi151:> ') 2024-05-21T21:55:31.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:31 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-21T21:55:31.119 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:55:30 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:55:30.887+0000 7fbf14b61200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-21T21:55:31.236 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 21 21:55:30 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-z[38659]: 2024-05-21T21:55:30.909+0000 7fd09350d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-21T21:55:31.236 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 21 21:55:31 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-z[38659]: 2024-05-21T21:55:31.005+0000 7fd09350d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-21T21:55:31.285 INFO:journalctl@ceph.mgr.y.smithi164.stdout:May 21 21:55:30 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-y[37650]: 2024-05-21T21:55:30.962+0000 7f8c55776200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-21T21:55:31.286 INFO:journalctl@ceph.mgr.y.smithi164.stdout:May 21 21:55:31 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-y[37650]: 2024-05-21T21:55:31.058+0000 7f8c55776200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-21T21:55:31.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-21T21:55:31.730 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 21 21:55:31 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-z[38659]: 2024-05-21T21:55:31.376+0000 7fd09350d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-21T21:55:31.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-21T21:55:31.738 DEBUG:teuthology.orchestra.run.smithi151:> ip netns list 2024-05-21T21:55:31.780 INFO:journalctl@ceph.mgr.y.smithi164.stdout:May 21 21:55:31 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-y[37650]: 2024-05-21T21:55:31.429+0000 7f8c55776200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-21T21:55:31.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-21T21:55:31.788 DEBUG:teuthology.orchestra.run.smithi151:> ip netns list-id 2024-05-21T21:55:31.879 DEBUG:teuthology.orchestra.run.smithi151:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-21T21:55:31.879 DEBUG:teuthology.orchestra.run.smithi151:> set -e 2024-05-21T21:55:31.879 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-21T21:55:31.879 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-05-21T21:55:31.879 DEBUG:teuthology.orchestra.run.smithi151:> ') 2024-05-21T21:55:31.975 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:31 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-21T21:55:31.985 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 21 21:55:31 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-z[38659]: 2024-05-21T21:55:31.730+0000 7fd09350d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-21T21:55:31.986 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 21 21:55:31 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-z[38659]: 2024-05-21T21:55:31.821+0000 7fd09350d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-21T21:55:31.997 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:55:31 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:55:31.738+0000 7fbf14b61200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-21T21:55:32.036 INFO:journalctl@ceph.mgr.y.smithi164.stdout:May 21 21:55:31 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-y[37650]: 2024-05-21T21:55:31.779+0000 7f8c55776200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-21T21:55:32.036 INFO:journalctl@ceph.mgr.y.smithi164.stdout:May 21 21:55:31 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-y[37650]: 2024-05-21T21:55:31.872+0000 7f8c55776200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-21T21:55:32.040 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-21T21:55:32.050 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-05-21T21:55:32.050 DEBUG:teuthology.orchestra.run.smithi151:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-21T21:55:32.050 DEBUG:teuthology.orchestra.run.smithi151:> set -e 2024-05-21T21:55:32.050 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-05-21T21:55:32.050 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-21T21:55:32.050 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-05-21T21:55:32.050 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-05-21T21:55:32.051 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-05-21T21:55:32.051 DEBUG:teuthology.orchestra.run.smithi151:> ') 2024-05-21T21:55:32.142 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-21T21:55:32.259 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:55:32 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:55:32.037+0000 7fbf14b61200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-21T21:55:32.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-21T21:55:32.336 DEBUG:teuthology.orchestra.run.smithi151:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-21T21:55:32.336 DEBUG:teuthology.orchestra.run.smithi151:> set -e 2024-05-21T21:55:32.336 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip link set brx.0 up 2024-05-21T21:55:32.336 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip link set dev brx.0 master ceph-brx 2024-05-21T21:55:32.336 DEBUG:teuthology.orchestra.run.smithi151:> ') 2024-05-21T21:55:32.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-21T21:55:32.486 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 21 21:55:32 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-z[38659]: 2024-05-21T21:55:32.064+0000 7fd09350d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-21T21:55:32.486 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 21 21:55:32 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-z[38659]: 2024-05-21T21:55:32.196+0000 7fd09350d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-21T21:55:32.486 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 21 21:55:32 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-z[38659]: 2024-05-21T21:55:32.291+0000 7fd09350d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-21T21:55:32.518 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-21T21:55:32.526 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-21T21:55:32.527 DEBUG:teuthology.orchestra.run.smithi151:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-21T21:55:32.536 INFO:journalctl@ceph.mgr.y.smithi164.stdout:May 21 21:55:32 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-y[37650]: 2024-05-21T21:55:32.117+0000 7f8c55776200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-21T21:55:32.536 INFO:journalctl@ceph.mgr.y.smithi164.stdout:May 21 21:55:32 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-y[37650]: 2024-05-21T21:55:32.249+0000 7f8c55776200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-21T21:55:32.536 INFO:journalctl@ceph.mgr.y.smithi164.stdout:May 21 21:55:32 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-y[37650]: 2024-05-21T21:55:32.345+0000 7f8c55776200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-21T21:55:32.565 INFO:teuthology.orchestra.run.smithi151.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-21T21:55:32.566 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-21T21:55:32.566 DEBUG:teuthology.orchestra.run.smithi151:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-21T21:55:32.656 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-05-21T21:55:32.656 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-05-21T21:55:32.656 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/219d07b0-9264-4d1a-9d62-bff6dec196ce 2024-05-21T21:55:32.656 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-21T21:55:32.657 DEBUG:teuthology.orchestra.run.smithi151:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/219d07b0-9264-4d1a-9d62-bff6dec196ce /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,wsync,nofallback 2024-05-21T21:55:32.785 INFO:teuthology.orchestra.run.smithi151.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,wsync,nofallback 2024-05-21T21:55:32.786 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,wsync". 2024-05-21T21:55:32.906 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:55:32 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:55:32.625+0000 7fbf14b61200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-21T21:55:32.907 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:55:32 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:55:32.759+0000 7fbf14b61200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-21T21:55:32.986 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 21 21:55:32 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-z[38659]: 2024-05-21T21:55:32.618+0000 7fd09350d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-21T21:55:32.986 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 21 21:55:32 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-z[38659]: 2024-05-21T21:55:32.709+0000 7fd09350d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-21T21:55:33.036 INFO:journalctl@ceph.mgr.y.smithi164.stdout:May 21 21:55:32 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-y[37650]: 2024-05-21T21:55:32.674+0000 7f8c55776200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-21T21:55:33.036 INFO:journalctl@ceph.mgr.y.smithi164.stdout:May 21 21:55:32 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-y[37650]: 2024-05-21T21:55:32.767+0000 7f8c55776200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-21T21:55:33.527 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:55:33 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:55:33.262+0000 7fbf14b61200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-21T21:55:33.543 INFO:teuthology.orchestra.run.smithi151.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,wsync,nofallback 2024-05-21T21:55:33.543 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,wsync". 2024-05-21T21:55:33.543 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: resolved to: "172.21.15.151:6789,172.21.15.164:6789,172.21.15.184:6789" 2024-05-21T21:55:33.543 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: trying mount with new device syntax: 0@c8c7a2e6-17ba-11ef-bc9a-c7b262605968.cephfs=/volumes/qa/sv_1/219d07b0-9264-4d1a-9d62-bff6dec196ce 2024-05-21T21:55:33.543 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,wsync,name=0,key=0,mon_addr=172.21.15.151:6789/172.21.15.164:6789/172.21.15.184:6789" will pass to kernel 2024-05-21T21:55:33.546 INFO:teuthology.orchestra.run.smithi151.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-21T21:55:33.546 INFO:teuthology.orchestra.run.smithi151.stdout: You just mounted a file system that supports labels which does not 2024-05-21T21:55:33.547 INFO:teuthology.orchestra.run.smithi151.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-21T21:55:33.547 INFO:teuthology.orchestra.run.smithi151.stdout: applications will generate AVC messages and not be allowed access to 2024-05-21T21:55:33.547 INFO:teuthology.orchestra.run.smithi151.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-21T21:55:33.551 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-21T21:55:33.551 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-21T21:55:33.551 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-21T21:55:33.597 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 21 21:55:33 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-z[38659]: 2024-05-21T21:55:33.308+0000 7fd09350d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-21T21:55:33.597 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 21 21:55:33 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-z[38659]: 2024-05-21T21:55:33.526+0000 7fd09350d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-21T21:55:33.623 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-21T21:55:33.712 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:33 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-21T21:55:33.716 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-21T21:55:33.726 INFO:teuthology.orchestra.run.smithi151.stdout:client25147 2024-05-21T21:55:33.726 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-05-21T21:55:33.726 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-21T21:55:33.769 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-21T21:55:33.786 INFO:journalctl@ceph.mgr.y.smithi164.stdout:May 21 21:55:33 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-y[37650]: 2024-05-21T21:55:33.370+0000 7f8c55776200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-21T21:55:33.786 INFO:journalctl@ceph.mgr.y.smithi164.stdout:May 21 21:55:33 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-y[37650]: 2024-05-21T21:55:33.593+0000 7f8c55776200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-21T21:55:33.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:33 smithi164 ceph-mon[36269]: Standby manager daemon z restarted 2024-05-21T21:55:33.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:33 smithi164 ceph-mon[36269]: Standby manager daemon z started 2024-05-21T21:55:33.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:33 smithi164 ceph-mon[36269]: from='mgr.? 172.21.15.184:0/12688979' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-21T21:55:33.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:33 smithi164 ceph-mon[36269]: from='mgr.? 172.21.15.184:0/12688979' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-21T21:55:33.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:33 smithi164 ceph-mon[36269]: from='mgr.? 172.21.15.184:0/12688979' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-21T21:55:33.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:33 smithi164 ceph-mon[36269]: from='mgr.? 172.21.15.184:0/12688979' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-21T21:55:33.888 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:55:33 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:55:33.751+0000 7fbf14b61200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-21T21:55:33.888 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:33 smithi151 ceph-mon[31623]: Standby manager daemon z restarted 2024-05-21T21:55:33.888 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:33 smithi151 ceph-mon[31623]: Standby manager daemon z started 2024-05-21T21:55:33.888 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:33 smithi151 ceph-mon[31623]: from='mgr.? 172.21.15.184:0/12688979' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-21T21:55:33.888 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:33 smithi151 ceph-mon[31623]: from='mgr.? 172.21.15.184:0/12688979' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-21T21:55:33.889 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:33 smithi151 ceph-mon[31623]: from='mgr.? 172.21.15.184:0/12688979' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-21T21:55:33.889 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:33 smithi151 ceph-mon[31623]: from='mgr.? 172.21.15.184:0/12688979' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-21T21:55:33.912 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:33 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-21T21:55:33.917 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-21T21:55:33.927 INFO:teuthology.orchestra.run.smithi151.stdout:client25147 2024-05-21T21:55:33.927 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-21T21:55:33.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:33 smithi184 ceph-mon[36942]: Standby manager daemon z restarted 2024-05-21T21:55:33.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:33 smithi184 ceph-mon[36942]: Standby manager daemon z started 2024-05-21T21:55:33.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:33 smithi184 ceph-mon[36942]: from='mgr.? 172.21.15.184:0/12688979' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-21T21:55:33.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:33 smithi184 ceph-mon[36942]: from='mgr.? 172.21.15.184:0/12688979' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-21T21:55:33.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:33 smithi184 ceph-mon[36942]: from='mgr.? 172.21.15.184:0/12688979' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-21T21:55:33.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:33 smithi184 ceph-mon[36942]: from='mgr.? 172.21.15.184:0/12688979' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-21T21:55:34.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:34 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-21T21:55:34.024 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-21T21:55:34.034 INFO:teuthology.orchestra.run.smithi151.stdout:client25147 2024-05-21T21:55:34.034 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968.client25147/status') 2024-05-21T21:55:34.134 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-21T21:55:34.155 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:55:33 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:55:33.886+0000 7fbf14b61200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-21T21:55:34.162 INFO:teuthology.orchestra.run.smithi151.stdout:instance: client.25147 (3)172.21.15.151:0/443800938 2024-05-21T21:55:34.162 INFO:teuthology.orchestra.run.smithi151.stdout:blocklisted: false 2024-05-21T21:55:34.163 INFO:teuthology.orchestra.run.smithi151.stderr:0+1 records in 2024-05-21T21:55:34.163 INFO:teuthology.orchestra.run.smithi151.stderr:0+1 records out 2024-05-21T21:55:34.163 INFO:teuthology.orchestra.run.smithi151.stderr:71 bytes copied, 7.5888e-05 s, 936 kB/s 2024-05-21T21:55:34.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-21T21:55:34.175 INFO:teuthology.run_tasks:Running task exec... 2024-05-21T21:55:34.186 INFO:teuthology.task.exec:Executing custom commands... 2024-05-21T21:55:34.186 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi151.front.sepia.ceph.com 2024-05-21T21:55:34.186 DEBUG:teuthology.orchestra.run.smithi151:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-05-21T21:55:34.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:34.369+0000 7f9772c00640 1 -- 172.21.15.151:0/1037955750 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f976c103cc0 msgr2=0x7f976c104140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:34.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:34.369+0000 7f9772c00640 1 --2- 172.21.15.151:0/1037955750 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f976c103cc0 0x7f976c104140 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9758009900 tx=0x7f975802a410 comp rx=0 tx=0).stop 2024-05-21T21:55:34.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:34.369+0000 7f9772c00640 1 -- 172.21.15.151:0/1037955750 shutdown_connections 2024-05-21T21:55:34.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:34.369+0000 7f9772c00640 1 --2- 172.21.15.151:0/1037955750 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f976c104680 0x7f976c10af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:34.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:34.369+0000 7f9772c00640 1 --2- 172.21.15.151:0/1037955750 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f976c103cc0 0x7f976c104140 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:34.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:34.369+0000 7f9772c00640 1 --2- 172.21.15.151:0/1037955750 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f976c102ad0 0x7f976c102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:34.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:34.369+0000 7f9772c00640 1 -- 172.21.15.151:0/1037955750 >> 172.21.15.151:0/1037955750 conn(0x7f976c0fe040 msgr2=0x7f976c100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:55:34.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:34.369+0000 7f9772c00640 1 -- 172.21.15.151:0/1037955750 shutdown_connections 2024-05-21T21:55:34.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:34.369+0000 7f9772c00640 1 -- 172.21.15.151:0/1037955750 wait complete. 2024-05-21T21:55:34.373 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:34.370+0000 7f9772c00640 1 Processor -- start 2024-05-21T21:55:34.373 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:34.370+0000 7f9772c00640 1 -- start start 2024-05-21T21:55:34.373 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:34.370+0000 7f9772c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f976c102ad0 0x7f976c19ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:34.373 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:34.370+0000 7f9772c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f976c103cc0 0x7f976c19b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:34.373 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:34.370+0000 7f9772c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f976c104680 0x7f976c1a2400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:34.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:34.370+0000 7f9772c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f976c10e650 con 0x7f976c102ad0 2024-05-21T21:55:34.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:34.370+0000 7f9772c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f976c10e4d0 con 0x7f976c103cc0 2024-05-21T21:55:34.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:34.370+0000 7f9772c00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f976c10e7d0 con 0x7f976c104680 2024-05-21T21:55:34.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:34.371+0000 7f976be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f976c102ad0 0x7f976c19ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:34.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:34.371+0000 7f976be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f976c102ad0 0x7f976c19ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:40242/0 (socket says 172.21.15.151:40242) 2024-05-21T21:55:34.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:34.371+0000 7f976be00640 1 -- 172.21.15.151:0/3417245237 learned_addr learned my addr 172.21.15.151:0/3417245237 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:55:34.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:34.371+0000 7f9770c00640 1 --2- 172.21.15.151:0/3417245237 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f976c104680 0x7f976c1a2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:34.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:34.371+0000 7f976b400640 1 --2- 172.21.15.151:0/3417245237 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f976c103cc0 0x7f976c19b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:34.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:34.371+0000 7f976be00640 1 -- 172.21.15.151:0/3417245237 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f976c104680 msgr2=0x7f976c1a2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:34.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:34.371+0000 7f976be00640 1 --2- 172.21.15.151:0/3417245237 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f976c104680 0x7f976c1a2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:34.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:34.371+0000 7f976be00640 1 -- 172.21.15.151:0/3417245237 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f976c103cc0 msgr2=0x7f976c19b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:34.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:34.371+0000 7f976be00640 1 --2- 172.21.15.151:0/3417245237 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f976c103cc0 0x7f976c19b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:34.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:34.371+0000 7f976be00640 1 -- 172.21.15.151:0/3417245237 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f976c1a2a70 con 0x7f976c102ad0 2024-05-21T21:55:34.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:34.371+0000 7f976be00640 1 --2- 172.21.15.151:0/3417245237 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f976c102ad0 0x7f976c19ae50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f975c00e9c0 tx=0x7f975c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:55:34.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:34.371+0000 7f9768c00640 1 -- 172.21.15.151:0/3417245237 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975c00cde0 con 0x7f976c102ad0 2024-05-21T21:55:34.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:34.372+0000 7f9768c00640 1 -- 172.21.15.151:0/3417245237 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f975c0108e0 con 0x7f976c102ad0 2024-05-21T21:55:34.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:34.372+0000 7f9772c00640 1 -- 172.21.15.151:0/3417245237 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f976c1a2d60 con 0x7f976c102ad0 2024-05-21T21:55:34.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:34.372+0000 7f9768c00640 1 -- 172.21.15.151:0/3417245237 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975c018960 con 0x7f976c102ad0 2024-05-21T21:55:34.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:34.372+0000 7f9772c00640 1 -- 172.21.15.151:0/3417245237 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f976c1a31d0 con 0x7f976c102ad0 2024-05-21T21:55:34.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:34.373+0000 7f9772c00640 1 -- 172.21.15.151:0/3417245237 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f976c1a3400 con 0x7f976c102ad0 2024-05-21T21:55:34.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:34.375+0000 7f9768c00640 1 -- 172.21.15.151:0/3417245237 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 20) ==== 149670+0+0 (secure 0 0 0) 0x7f975c01f020 con 0x7f976c102ad0 2024-05-21T21:55:34.377 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:34.376+0000 7f9768c00640 1 --2- 172.21.15.151:0/3417245237 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f97500b28f0 0x7f97500b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:34.377 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:34.376+0000 7f976b400640 1 -- 172.21.15.151:0/3417245237 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f97500b28f0 msgr2=0x7f97500b4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.151:6800/2796622483 2024-05-21T21:55:34.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:34.376+0000 7f9768c00640 1 -- 172.21.15.151:0/3417245237 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f975c0e24d0 con 0x7f976c102ad0 2024-05-21T21:55:34.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:34.377+0000 7f976b400640 1 --2- 172.21.15.151:0/3417245237 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f97500b28f0 0x7f97500b4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-21T21:55:34.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:34.381+0000 7f9768c00640 1 -- 172.21.15.151:0/3417245237 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f975c054df0 con 0x7f976c102ad0 2024-05-21T21:55:34.431 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:55:34 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:55:34.237+0000 7fbf14b61200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-21T21:55:34.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:34.578+0000 7f976b400640 1 -- 172.21.15.151:0/3417245237 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f97500b28f0 msgr2=0x7f97500b4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.151:6800/2796622483 2024-05-21T21:55:34.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:34.578+0000 7f976b400640 1 --2- 172.21.15.151:0/3417245237 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f97500b28f0 0x7f97500b4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-21T21:55:34.606 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:34.605+0000 7f9768c00640 1 -- 172.21.15.151:0/3417245237 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mgrmap(e 21) ==== 149670+0+0 (secure 0 0 0) 0x7f975c010a90 con 0x7f976c102ad0 2024-05-21T21:55:34.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:34.641+0000 7f9772c00640 1 -- 172.21.15.151:0/3417245237 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7f976c1a35d0 con 0x7f976c102ad0 2024-05-21T21:55:34.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:34 smithi151 ceph-mon[31623]: mgrmap e20: x(active, since 12m), standbys: y, z 2024-05-21T21:55:34.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:34 smithi151 ceph-mon[31623]: Standby manager daemon y restarted 2024-05-21T21:55:34.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:34 smithi151 ceph-mon[31623]: Standby manager daemon y started 2024-05-21T21:55:34.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:34 smithi151 ceph-mon[31623]: from='mgr.? 172.21.15.164:0/2513654895' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-21T21:55:34.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:34 smithi151 ceph-mon[31623]: from='mgr.? 172.21.15.164:0/2513654895' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-21T21:55:34.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:34 smithi151 ceph-mon[31623]: from='mgr.? 172.21.15.164:0/2513654895' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-21T21:55:34.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:34 smithi151 ceph-mon[31623]: from='mgr.? 172.21.15.164:0/2513654895' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-21T21:55:34.704 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:55:34 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:55:34.429+0000 7fbf14b61200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-21T21:55:34.704 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:55:34 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:55:34.563+0000 7fbf14b61200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-21T21:55:34.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:34.979+0000 7f976b400640 1 -- 172.21.15.151:0/3417245237 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f97500b28f0 msgr2=0x7f97500b4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.151:6800/2796622483 2024-05-21T21:55:34.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:34.979+0000 7f976b400640 1 --2- 172.21.15.151:0/3417245237 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f97500b28f0 0x7f97500b4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-21T21:55:34.985 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:34 smithi184 ceph-mon[36942]: mgrmap e20: x(active, since 12m), standbys: y, z 2024-05-21T21:55:34.985 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:34 smithi184 ceph-mon[36942]: Standby manager daemon y restarted 2024-05-21T21:55:34.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:34 smithi184 ceph-mon[36942]: Standby manager daemon y started 2024-05-21T21:55:34.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:34 smithi184 ceph-mon[36942]: from='mgr.? 172.21.15.164:0/2513654895' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-21T21:55:34.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:34 smithi184 ceph-mon[36942]: from='mgr.? 172.21.15.164:0/2513654895' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-21T21:55:34.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:34 smithi184 ceph-mon[36942]: from='mgr.? 172.21.15.164:0/2513654895' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-21T21:55:34.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:34 smithi184 ceph-mon[36942]: from='mgr.? 172.21.15.164:0/2513654895' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-21T21:55:35.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:34 smithi164 ceph-mon[36269]: mgrmap e20: x(active, since 12m), standbys: y, z 2024-05-21T21:55:35.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:34 smithi164 ceph-mon[36269]: Standby manager daemon y restarted 2024-05-21T21:55:35.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:34 smithi164 ceph-mon[36269]: Standby manager daemon y started 2024-05-21T21:55:35.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:34 smithi164 ceph-mon[36269]: from='mgr.? 172.21.15.164:0/2513654895' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-21T21:55:35.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:34 smithi164 ceph-mon[36269]: from='mgr.? 172.21.15.164:0/2513654895' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-21T21:55:35.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:34 smithi164 ceph-mon[36269]: from='mgr.? 172.21.15.164:0/2513654895' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-21T21:55:35.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:34 smithi164 ceph-mon[36269]: from='mgr.? 172.21.15.164:0/2513654895' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-21T21:55:35.453 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:55:34 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:55:34.973+0000 7fbf14b61200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-21T21:55:35.453 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:55:35 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:55:35.100+0000 7fbf14b61200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-21T21:55:35.613 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:35.612+0000 7f9768c00640 1 -- 172.21.15.151:0/3417245237 <== mon.0 v2:172.21.15.151:3300/0 8 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v29) ==== 117+0+0 (secure 0 0 0) 0x7f975c00f3c0 con 0x7f976c102ad0 2024-05-21T21:55:35.617 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:35.616+0000 7f9772c00640 1 -- 172.21.15.151:0/3417245237 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f97500b28f0 msgr2=0x7f97500b4db0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-21T21:55:35.617 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:35.616+0000 7f9772c00640 1 --2- 172.21.15.151:0/3417245237 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f97500b28f0 0x7f97500b4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:35.618 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:35.616+0000 7f9772c00640 1 -- 172.21.15.151:0/3417245237 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f976c102ad0 msgr2=0x7f976c19ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:35.618 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:35.616+0000 7f9772c00640 1 --2- 172.21.15.151:0/3417245237 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f976c102ad0 0x7f976c19ae50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f975c00e9c0 tx=0x7f975c00ee80 comp rx=0 tx=0).stop 2024-05-21T21:55:35.618 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:35.616+0000 7f9772c00640 1 -- 172.21.15.151:0/3417245237 shutdown_connections 2024-05-21T21:55:35.618 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:35.616+0000 7f9772c00640 1 --2- 172.21.15.151:0/3417245237 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7f97500b28f0 0x7f97500b4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:35.618 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:35.616+0000 7f9772c00640 1 --2- 172.21.15.151:0/3417245237 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f976c104680 0x7f976c1a2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:35.618 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:35.616+0000 7f9772c00640 1 --2- 172.21.15.151:0/3417245237 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f976c103cc0 0x7f976c19b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:35.618 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:35.616+0000 7f9772c00640 1 --2- 172.21.15.151:0/3417245237 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f976c102ad0 0x7f976c19ae50 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:35.618 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:35.616+0000 7f9772c00640 1 -- 172.21.15.151:0/3417245237 >> 172.21.15.151:0/3417245237 conn(0x7f976c0fe040 msgr2=0x7f976c074c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:55:35.618 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:35.616+0000 7f9772c00640 1 -- 172.21.15.151:0/3417245237 shutdown_connections 2024-05-21T21:55:35.618 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:35.616+0000 7f9772c00640 1 -- 172.21.15.151:0/3417245237 wait complete. 2024-05-21T21:55:35.637 INFO:teuthology.run_tasks:Running task check-counter... 2024-05-21T21:55:35.647 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-05-21T21:55:35.689 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-21T21:55:35.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:35.877+0000 7fa048a00640 1 -- 172.21.15.151:0/367958636 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa0440754b0 msgr2=0x7fa044075930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:35.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:35.877+0000 7fa048a00640 1 --2- 172.21.15.151:0/367958636 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa0440754b0 0x7fa044075930 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa030009900 tx=0x7fa03002a3f0 comp rx=0 tx=0).stop 2024-05-21T21:55:35.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:35.877+0000 7fa048a00640 1 -- 172.21.15.151:0/367958636 shutdown_connections 2024-05-21T21:55:35.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:35.877+0000 7fa048a00640 1 --2- 172.21.15.151:0/367958636 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa04410a7b0 0x7fa04410cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:35.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:35.877+0000 7fa048a00640 1 --2- 172.21.15.151:0/367958636 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa0440754b0 0x7fa044075930 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:35.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:35.877+0000 7fa048a00640 1 --2- 172.21.15.151:0/367958636 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa044076a90 0x7fa044074ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:35.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:35.877+0000 7fa048a00640 1 -- 172.21.15.151:0/367958636 >> 172.21.15.151:0/367958636 conn(0x7fa0440fde60 msgr2=0x7fa0441002a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:55:35.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:35.878+0000 7fa048a00640 1 -- 172.21.15.151:0/367958636 shutdown_connections 2024-05-21T21:55:35.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:35.878+0000 7fa048a00640 1 -- 172.21.15.151:0/367958636 wait complete. 2024-05-21T21:55:35.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:35.879+0000 7fa048a00640 1 Processor -- start 2024-05-21T21:55:35.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:35.879+0000 7fa048a00640 1 -- start start 2024-05-21T21:55:35.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:35.879+0000 7fa048a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa0440754b0 0x7fa04419ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:35.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:35.879+0000 7fa048a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa044076a90 0x7fa04419b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:35.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:35.879+0000 7fa048a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa04410a7b0 0x7fa0441a22a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:35.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:35.879+0000 7fa048a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa044110650 con 0x7fa0440754b0 2024-05-21T21:55:35.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:35.879+0000 7fa048a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa0441104d0 con 0x7fa04410a7b0 2024-05-21T21:55:35.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:35.879+0000 7fa048a00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa0441107d0 con 0x7fa044076a90 2024-05-21T21:55:35.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:35.879+0000 7fa043400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa0440754b0 0x7fa04419ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:35.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:35.879+0000 7fa043400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa0440754b0 0x7fa04419ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:40546/0 (socket says 172.21.15.151:40546) 2024-05-21T21:55:35.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:35.879+0000 7fa043400640 1 -- 172.21.15.151:0/2047882567 learned_addr learned my addr 172.21.15.151:0/2047882567 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:55:35.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:35.879+0000 7fa042a00640 1 --2- 172.21.15.151:0/2047882567 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa044076a90 0x7fa04419b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:35.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:35.879+0000 7fa043e00640 1 --2- 172.21.15.151:0/2047882567 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa04410a7b0 0x7fa0441a22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:35.884 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:35.880+0000 7fa043400640 1 -- 172.21.15.151:0/2047882567 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa044076a90 msgr2=0x7fa04419b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:35.884 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:35.880+0000 7fa043400640 1 --2- 172.21.15.151:0/2047882567 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa044076a90 0x7fa04419b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:35.884 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:35.880+0000 7fa043400640 1 -- 172.21.15.151:0/2047882567 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa04410a7b0 msgr2=0x7fa0441a22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:35.884 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:35.880+0000 7fa043400640 1 --2- 172.21.15.151:0/2047882567 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa04410a7b0 0x7fa0441a22a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:35.884 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:35.880+0000 7fa043400640 1 -- 172.21.15.151:0/2047882567 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0441a2940 con 0x7fa0440754b0 2024-05-21T21:55:35.884 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:35.880+0000 7fa043400640 1 --2- 172.21.15.151:0/2047882567 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa0440754b0 0x7fa04419ad30 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa03400e9f0 tx=0x7fa03400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:55:35.884 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:35.880+0000 7fa03be00640 1 -- 172.21.15.151:0/2047882567 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa03400cd80 con 0x7fa0440754b0 2024-05-21T21:55:35.884 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:35.880+0000 7fa048a00640 1 -- 172.21.15.151:0/2047882567 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0441a2c30 con 0x7fa0440754b0 2024-05-21T21:55:35.884 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:35.880+0000 7fa03be00640 1 -- 172.21.15.151:0/2047882567 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0340054d0 con 0x7fa0440754b0 2024-05-21T21:55:35.884 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:35.880+0000 7fa03be00640 1 -- 172.21.15.151:0/2047882567 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa034010690 con 0x7fa0440754b0 2024-05-21T21:55:35.884 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:35.881+0000 7fa048a00640 1 -- 172.21.15.151:0/2047882567 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0441a3180 con 0x7fa0440754b0 2024-05-21T21:55:35.885 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:35 smithi151 ceph-mon[31623]: mgrmap e21: x(active, since 12m), standbys: z, y 2024-05-21T21:55:35.885 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:35 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/3417245237' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-21T21:55:35.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:35.882+0000 7fa048a00640 1 -- 172.21.15.151:0/2047882567 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0441094e0 con 0x7fa0440754b0 2024-05-21T21:55:35.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:35.884+0000 7fa03be00640 1 -- 172.21.15.151:0/2047882567 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 21) ==== 149670+0+0 (secure 0 0 0) 0x7fa034004e90 con 0x7fa0440754b0 2024-05-21T21:55:35.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:35.885+0000 7fa03be00640 1 --2- 172.21.15.151:0/2047882567 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fa0280b28c0 0x7fa0280b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:35.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:35.886+0000 7fa042a00640 1 -- 172.21.15.151:0/2047882567 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fa0280b28c0 msgr2=0x7fa0280b4d80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.151:6800/2796622483 2024-05-21T21:55:35.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:35.886+0000 7fa03be00640 1 -- 172.21.15.151:0/2047882567 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fa0340e26b0 con 0x7fa0440754b0 2024-05-21T21:55:35.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:35.886+0000 7fa042a00640 1 --2- 172.21.15.151:0/2047882567 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fa0280b28c0 0x7fa0280b4d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-21T21:55:35.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:35.890+0000 7fa03be00640 1 -- 172.21.15.151:0/2047882567 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa0340aa390 con 0x7fa0440754b0 2024-05-21T21:55:35.985 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:35 smithi184 ceph-mon[36942]: mgrmap e21: x(active, since 12m), standbys: z, y 2024-05-21T21:55:35.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:35 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/3417245237' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-21T21:55:36.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:35 smithi164 ceph-mon[36269]: mgrmap e21: x(active, since 12m), standbys: z, y 2024-05-21T21:55:36.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:35 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/3417245237' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-21T21:55:36.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.087+0000 7fa042a00640 1 -- 172.21.15.151:0/2047882567 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fa0280b28c0 msgr2=0x7fa0280b4d80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.151:6800/2796622483 2024-05-21T21:55:36.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.087+0000 7fa042a00640 1 --2- 172.21.15.151:0/2047882567 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fa0280b28c0 0x7fa0280b4d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-21T21:55:36.105 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.104+0000 7fa048a00640 1 -- 172.21.15.151:0/2047882567 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa044063650 con 0x7fa0440754b0 2024-05-21T21:55:36.106 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.105+0000 7fa03be00640 1 -- 172.21.15.151:0/2047882567 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22744 (secure 0 0 0) 0x7fa0340a9c90 con 0x7fa0440754b0 2024-05-21T21:55:36.107 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T21:55:36.107 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":82,"fsid":"c8c7a2e6-17ba-11ef-bc9a-c7b262605968","created":"2024-05-21T21:41:52.412833+0000","modified":"2024-05-21T21:52:10.856587+0000","last_up_change":"2024-05-21T21:50:05.401314+0000","last_in_change":"2024-05-21T21:49:45.684136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-21T21:45:55.196046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-21T21:51:11.447428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-21T21:51:15.618605+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"18db8e54-0cfc-4e12-9c84-6f72a99cc989","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":2420400473},{"type":"v1","addr":"172.21.15.151:6803","nonce":2420400473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":2420400473},{"type":"v1","addr":"172.21.15.151:6805","nonce":2420400473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":2420400473},{"type":"v1","addr":"172.21.15.151:6809","nonce":2420400473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":2420400473},{"type":"v1","addr":"172.21.15.151:6807","nonce":2420400473}]},"public_addr":"172.21.15.151:6803/2420400473","cluster_addr":"172.21.15.151:6805/2420400473","heartbeat_back_addr":"172.21.15.151:6809/2420400473","heartbeat_front_addr":"172.21.15.151:6807/2420400473","state":["exists","up"]},{"osd":1,"uuid":"bac384c7-1102-4815-81a8-10cc7661d230","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":3900240447},{"type":"v1","addr":"172.21.15.164:6801","nonce":3900240447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":3900240447},{"type":"v1","addr":"172.21.15.164:6803","nonce":3900240447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":3900240447},{"type":"v1","addr":"172.21.15.164:6807","nonce":3900240447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":3900240447},{"type":"v1","addr":"172.21.15.164:6805","nonce":3900240447}]},"public_addr":"172.21.15.164:6801/3900240447","cluster_addr":"172.21.15.164:6803/3900240447","heartbeat_back_addr":"172.21.15.164:6807/3900240447","heartbeat_front_addr":"172.21.15.164:6805/3900240447","state":["exists","up"]},{"osd":2,"uuid":"b7e4a782-1518-4d43-bee9-7fd768ee5cb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":3971150737},{"type":"v1","addr":"172.21.15.184:6801","nonce":3971150737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3971150737},{"type":"v1","addr":"172.21.15.184:6803","nonce":3971150737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3971150737},{"type":"v1","addr":"172.21.15.184:6807","nonce":3971150737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3971150737},{"type":"v1","addr":"172.21.15.184:6805","nonce":3971150737}]},"public_addr":"172.21.15.184:6801/3971150737","cluster_addr":"172.21.15.184:6803/3971150737","heartbeat_back_addr":"172.21.15.184:6807/3971150737","heartbeat_front_addr":"172.21.15.184:6805/3971150737","state":["exists","up"]},{"osd":3,"uuid":"ab3df826-39e3-4635-a49c-2b5ffdbf12f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":556594400},{"type":"v1","addr":"172.21.15.151:6811","nonce":556594400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":556594400},{"type":"v1","addr":"172.21.15.151:6813","nonce":556594400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":556594400},{"type":"v1","addr":"172.21.15.151:6817","nonce":556594400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":556594400},{"type":"v1","addr":"172.21.15.151:6815","nonce":556594400}]},"public_addr":"172.21.15.151:6811/556594400","cluster_addr":"172.21.15.151:6813/556594400","heartbeat_back_addr":"172.21.15.151:6817/556594400","heartbeat_front_addr":"172.21.15.151:6815/556594400","state":["exists","up"]},{"osd":4,"uuid":"00cbf6ae-0f42-4685-a77d-d803c6467aa0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":988881973},{"type":"v1","addr":"172.21.15.164:6809","nonce":988881973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":988881973},{"type":"v1","addr":"172.21.15.164:6811","nonce":988881973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":988881973},{"type":"v1","addr":"172.21.15.164:6815","nonce":988881973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":988881973},{"type":"v1","addr":"172.21.15.164:6813","nonce":988881973}]},"public_addr":"172.21.15.164:6809/988881973","cluster_addr":"172.21.15.164:6811/988881973","heartbeat_back_addr":"172.21.15.164:6815/988881973","heartbeat_front_addr":"172.21.15.164:6813/988881973","state":["exists","up"]},{"osd":5,"uuid":"fad14221-0ef0-4d40-b414-c6c9f1f9718c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3293311106},{"type":"v1","addr":"172.21.15.184:6809","nonce":3293311106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":3293311106},{"type":"v1","addr":"172.21.15.184:6811","nonce":3293311106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":3293311106},{"type":"v1","addr":"172.21.15.184:6815","nonce":3293311106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":3293311106},{"type":"v1","addr":"172.21.15.184:6813","nonce":3293311106}]},"public_addr":"172.21.15.184:6809/3293311106","cluster_addr":"172.21.15.184:6811/3293311106","heartbeat_back_addr":"172.21.15.184:6815/3293311106","heartbeat_front_addr":"172.21.15.184:6813/3293311106","state":["exists","up"]},{"osd":6,"uuid":"ab8a60f6-0dec-4c1c-89db-f2e2f638f1ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":3377550762},{"type":"v1","addr":"172.21.15.151:6819","nonce":3377550762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":3377550762},{"type":"v1","addr":"172.21.15.151:6821","nonce":3377550762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":3377550762},{"type":"v1","addr":"172.21.15.151:6825","nonce":3377550762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":3377550762},{"type":"v1","addr":"172.21.15.151:6823","nonce":3377550762}]},"public_addr":"172.21.15.151:6819/3377550762","cluster_addr":"172.21.15.151:6821/3377550762","heartbeat_back_addr":"172.21.15.151:6825/3377550762","heartbeat_front_addr":"172.21.15.151:6823/3377550762","state":["exists","up"]},{"osd":7,"uuid":"db332268-9972-4ed7-a307-822b83a0fbb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":3869696936},{"type":"v1","addr":"172.21.15.164:6817","nonce":3869696936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":3869696936},{"type":"v1","addr":"172.21.15.164:6819","nonce":3869696936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":3869696936},{"type":"v1","addr":"172.21.15.164:6823","nonce":3869696936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":3869696936},{"type":"v1","addr":"172.21.15.164:6821","nonce":3869696936}]},"public_addr":"172.21.15.164:6817/3869696936","cluster_addr":"172.21.15.164:6819/3869696936","heartbeat_back_addr":"172.21.15.164:6823/3869696936","heartbeat_front_addr":"172.21.15.164:6821/3869696936","state":["exists","up"]},{"osd":8,"uuid":"fe2fbe1c-549a-4947-a188-426dfca04d10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":575273541},{"type":"v1","addr":"172.21.15.184:6817","nonce":575273541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":575273541},{"type":"v1","addr":"172.21.15.184:6819","nonce":575273541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":575273541},{"type":"v1","addr":"172.21.15.184:6823","nonce":575273541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":575273541},{"type":"v1","addr":"172.21.15.184:6821","nonce":575273541}]},"public_addr":"172.21.15.184:6817/575273541","cluster_addr":"172.21.15.184:6819/575273541","heartbeat_back_addr":"172.21.15.184:6823/575273541","heartbeat_front_addr":"172.21.15.184:6821/575273541","state":["exists","up"]},{"osd":9,"uuid":"3a7b450f-8004-4adf-bae3-970918f98f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":337128533},{"type":"v1","addr":"172.21.15.151:6827","nonce":337128533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":337128533},{"type":"v1","addr":"172.21.15.151:6829","nonce":337128533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":337128533},{"type":"v1","addr":"172.21.15.151:6833","nonce":337128533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":337128533},{"type":"v1","addr":"172.21.15.151:6831","nonce":337128533}]},"public_addr":"172.21.15.151:6827/337128533","cluster_addr":"172.21.15.151:6829/337128533","heartbeat_back_addr":"172.21.15.151:6833/337128533","heartbeat_front_addr":"172.21.15.151:6831/337128533","state":["exists","up"]},{"osd":10,"uuid":"8fcdf271-90c4-4db3-be69-b0ca716dfd80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":1823448782},{"type":"v1","addr":"172.21.15.164:6825","nonce":1823448782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":1823448782},{"type":"v1","addr":"172.21.15.164:6827","nonce":1823448782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":1823448782},{"type":"v1","addr":"172.21.15.164:6831","nonce":1823448782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":1823448782},{"type":"v1","addr":"172.21.15.164:6829","nonce":1823448782}]},"public_addr":"172.21.15.164:6825/1823448782","cluster_addr":"172.21.15.164:6827/1823448782","heartbeat_back_addr":"172.21.15.164:6831/1823448782","heartbeat_front_addr":"172.21.15.164:6829/1823448782","state":["exists","up"]},{"osd":11,"uuid":"e661477b-0c09-409f-a2fe-6ebd34642a8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":1563340128},{"type":"v1","addr":"172.21.15.184:6825","nonce":1563340128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":1563340128},{"type":"v1","addr":"172.21.15.184:6827","nonce":1563340128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":1563340128},{"type":"v1","addr":"172.21.15.184:6831","nonce":1563340128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":1563340128},{"type":"v1","addr":"172.21.15.184:6829","nonce":1563340128}]},"public_addr":"172.21.15.184:6825/1563340128","cluster_addr":"172.21.15.184:6827/1563340128","heartbeat_back_addr":"172.21.15.184:6831/1563340128","heartbeat_front_addr":"172.21.15.184:6829/1563340128","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:45:02.821160+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:45:27.844564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:45:51.904981+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:46:24.376245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:46:50.036621+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:47:14.748422+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:47:47.396383+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:48:13.543174+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:48:38.213285+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:49:11.980042+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:49:38.519761+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:50:03.439898+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.151:0/31963128":"2024-05-22T21:43:09.041455+0000","172.21.15.151:6800/3428827300":"2024-05-22T21:43:09.041455+0000","172.21.15.151:6801/3428827300":"2024-05-22T21:43:09.041455+0000","172.21.15.151:0/131989141":"2024-05-22T21:43:09.041455+0000","172.21.15.151:6801/1128885292":"2024-05-22T21:42:35.440147+0000","172.21.15.151:0/2703664519":"2024-05-22T21:43:09.041455+0000","172.21.15.151:6800/1128885292":"2024-05-22T21:42:35.440147+0000","172.21.15.151:0/3052943699":"2024-05-22T21:42:35.440147+0000","172.21.15.151:0/3929452511":"2024-05-22T21:42:35.440147+0000","172.21.15.151:0/3762889700":"2024-05-22T21:42:35.440147+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-21T21:55:36.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.108+0000 7fa048a00640 1 -- 172.21.15.151:0/2047882567 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fa0280b28c0 msgr2=0x7fa0280b4d80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-21T21:55:36.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.108+0000 7fa048a00640 1 --2- 172.21.15.151:0/2047882567 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fa0280b28c0 0x7fa0280b4d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:36.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.108+0000 7fa048a00640 1 -- 172.21.15.151:0/2047882567 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa0440754b0 msgr2=0x7fa04419ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:36.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.108+0000 7fa048a00640 1 --2- 172.21.15.151:0/2047882567 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa0440754b0 0x7fa04419ad30 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa03400e9f0 tx=0x7fa03400edf0 comp rx=0 tx=0).stop 2024-05-21T21:55:36.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.108+0000 7fa048a00640 1 -- 172.21.15.151:0/2047882567 shutdown_connections 2024-05-21T21:55:36.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.108+0000 7fa048a00640 1 --2- 172.21.15.151:0/2047882567 >> [v2:172.21.15.151:6800/2796622483,v1:172.21.15.151:6801/2796622483] conn(0x7fa0280b28c0 0x7fa0280b4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:36.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.108+0000 7fa048a00640 1 --2- 172.21.15.151:0/2047882567 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa04410a7b0 0x7fa0441a22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:36.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.108+0000 7fa048a00640 1 --2- 172.21.15.151:0/2047882567 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa044076a90 0x7fa04419b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:36.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.108+0000 7fa048a00640 1 --2- 172.21.15.151:0/2047882567 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa0440754b0 0x7fa04419ad30 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:36.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.108+0000 7fa048a00640 1 -- 172.21.15.151:0/2047882567 >> 172.21.15.151:0/2047882567 conn(0x7fa0440fde60 msgr2=0x7fa0441066b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:55:36.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.108+0000 7fa048a00640 1 -- 172.21.15.151:0/2047882567 shutdown_connections 2024-05-21T21:55:36.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.108+0000 7fa048a00640 1 -- 172.21.15.151:0/2047882567 wait complete. 2024-05-21T21:55:36.127 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-21T21:45:55.196046+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-21T21:51:11.447428+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-21T21:51:15.618605+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-21T21:55:36.127 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-21T21:55:36.181 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:55:35 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:55:35.880+0000 7fbf14b61200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-21T21:55:36.332 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.331+0000 7fd8d1000640 1 -- 172.21.15.151:0/4153115677 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd8cc105e40 msgr2=0x7fd8cc108230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:36.332 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.331+0000 7fd8d1000640 1 --2- 172.21.15.151:0/4153115677 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd8cc105e40 0x7fd8cc108230 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd8b8009860 tx=0x7fd8b802a340 comp rx=0 tx=0).stop 2024-05-21T21:55:36.332 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.331+0000 7fd8d1000640 1 -- 172.21.15.151:0/4153115677 shutdown_connections 2024-05-21T21:55:36.332 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.331+0000 7fd8d1000640 1 --2- 172.21.15.151:0/4153115677 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8cc108770 0x7fd8cc10abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:36.333 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.331+0000 7fd8d1000640 1 --2- 172.21.15.151:0/4153115677 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd8cc105e40 0x7fd8cc108230 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:36.333 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.331+0000 7fd8d1000640 1 --2- 172.21.15.151:0/4153115677 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd8cc103510 0x7fd8cc105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:36.333 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.331+0000 7fd8d1000640 1 -- 172.21.15.151:0/4153115677 >> 172.21.15.151:0/4153115677 conn(0x7fd8cc0fcf00 msgr2=0x7fd8cc0ff340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:55:36.333 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.331+0000 7fd8d1000640 1 -- 172.21.15.151:0/4153115677 shutdown_connections 2024-05-21T21:55:36.333 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.331+0000 7fd8d1000640 1 -- 172.21.15.151:0/4153115677 wait complete. 2024-05-21T21:55:36.333 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.332+0000 7fd8d1000640 1 Processor -- start 2024-05-21T21:55:36.334 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.333+0000 7fd8d1000640 1 -- start start 2024-05-21T21:55:36.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.333+0000 7fd8d1000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8cc103510 0x7fd8cc19af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:36.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.333+0000 7fd8d1000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd8cc105e40 0x7fd8cc19b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:36.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.333+0000 7fd8d1000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd8cc108770 0x7fd8cc1a2510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:36.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.333+0000 7fd8d1000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd8cc10e5b0 con 0x7fd8cc108770 2024-05-21T21:55:36.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.333+0000 7fd8d1000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd8cc10e430 con 0x7fd8cc105e40 2024-05-21T21:55:36.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.333+0000 7fd8d1000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd8cc10e730 con 0x7fd8cc103510 2024-05-21T21:55:36.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.334+0000 7fd8c9e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8cc103510 0x7fd8cc19af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:36.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.334+0000 7fd8ca800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd8cc108770 0x7fd8cc1a2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:36.336 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.334+0000 7fd8ca800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd8cc108770 0x7fd8cc1a2510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:40618/0 (socket says 172.21.15.151:40618) 2024-05-21T21:55:36.336 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.334+0000 7fd8c9e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8cc103510 0x7fd8cc19af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:43086/0 (socket says 172.21.15.151:43086) 2024-05-21T21:55:36.336 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.334+0000 7fd8ca800640 1 -- 172.21.15.151:0/355547455 learned_addr learned my addr 172.21.15.151:0/355547455 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:55:36.336 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.334+0000 7fd8c9e00640 1 -- 172.21.15.151:0/355547455 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd8cc105e40 msgr2=0x7fd8cc19b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:36.336 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.334+0000 7fd8c9e00640 1 --2- 172.21.15.151:0/355547455 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd8cc105e40 0x7fd8cc19b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:36.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.334+0000 7fd8c9e00640 1 -- 172.21.15.151:0/355547455 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd8cc108770 msgr2=0x7fd8cc1a2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:36.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.334+0000 7fd8c9e00640 1 --2- 172.21.15.151:0/355547455 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd8cc108770 0x7fd8cc1a2510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:36.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.334+0000 7fd8c9e00640 1 -- 172.21.15.151:0/355547455 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8cc1a2b80 con 0x7fd8cc103510 2024-05-21T21:55:36.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.335+0000 7fd8c9e00640 1 --2- 172.21.15.151:0/355547455 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8cc103510 0x7fd8cc19af60 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc00e790 tx=0x7fd8bc00ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:55:36.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.335+0000 7fd8c2a00640 1 -- 172.21.15.151:0/355547455 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8bc00eea0 con 0x7fd8cc103510 2024-05-21T21:55:36.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.335+0000 7fd8c2a00640 1 -- 172.21.15.151:0/355547455 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8bc00ccc0 con 0x7fd8cc103510 2024-05-21T21:55:36.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.335+0000 7fd8c2a00640 1 -- 172.21.15.151:0/355547455 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8bc0187a0 con 0x7fd8cc103510 2024-05-21T21:55:36.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.335+0000 7fd8d1000640 1 -- 172.21.15.151:0/355547455 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8cc1a2e70 con 0x7fd8cc103510 2024-05-21T21:55:36.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.335+0000 7fd8d1000640 1 -- 172.21.15.151:0/355547455 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8cc1a32e0 con 0x7fd8cc103510 2024-05-21T21:55:36.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.337+0000 7fd8d1000640 1 -- 172.21.15.151:0/355547455 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8cc10a2c0 con 0x7fd8cc103510 2024-05-21T21:55:36.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.341+0000 7fd8c2a00640 1 -- 172.21.15.151:0/355547455 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 22) ==== 149337+0+0 (secure 0 0 0) 0x7fd8bc010430 con 0x7fd8cc103510 2024-05-21T21:55:36.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.343+0000 7fd8c2a00640 1 -- 172.21.15.151:0/355547455 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fd8bc0e2020 con 0x7fd8cc103510 2024-05-21T21:55:36.345 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.344+0000 7fd8c2a00640 1 -- 172.21.15.151:0/355547455 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd8bc0aa0b0 con 0x7fd8cc103510 2024-05-21T21:55:36.453 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:55:36 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:55:36.185+0000 7fbf14b61200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-21T21:55:36.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.635+0000 7fd8d1000640 1 -- 172.21.15.151:0/355547455 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd8cc02de50 con 0x7fd8cc103510 2024-05-21T21:55:36.637 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.636+0000 7fd8c2a00640 1 -- 172.21.15.151:0/355547455 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) ==== 93+0+10 (secure 0 0 0) 0x7fd8bc0b2130 con 0x7fd8cc103510 2024-05-21T21:55:36.637 INFO:teuthology.orchestra.run.smithi151.stdout:pg_num: 1 2024-05-21T21:55:36.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.638+0000 7fd8d1000640 1 -- 172.21.15.151:0/355547455 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8cc103510 msgr2=0x7fd8cc19af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:36.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.638+0000 7fd8d1000640 1 --2- 172.21.15.151:0/355547455 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8cc103510 0x7fd8cc19af60 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc00e790 tx=0x7fd8bc00ec50 comp rx=0 tx=0).stop 2024-05-21T21:55:36.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.638+0000 7fd8d1000640 1 -- 172.21.15.151:0/355547455 shutdown_connections 2024-05-21T21:55:36.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.638+0000 7fd8d1000640 1 --2- 172.21.15.151:0/355547455 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd8cc108770 0x7fd8cc1a2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:36.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.638+0000 7fd8d1000640 1 --2- 172.21.15.151:0/355547455 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd8cc105e40 0x7fd8cc19b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:36.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.638+0000 7fd8d1000640 1 --2- 172.21.15.151:0/355547455 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8cc103510 0x7fd8cc19af60 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:36.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.638+0000 7fd8d1000640 1 -- 172.21.15.151:0/355547455 >> 172.21.15.151:0/355547455 conn(0x7fd8cc0fcf00 msgr2=0x7fd8cc102cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:55:36.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.639+0000 7fd8d1000640 1 -- 172.21.15.151:0/355547455 shutdown_connections 2024-05-21T21:55:36.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.639+0000 7fd8d1000640 1 -- 172.21.15.151:0/355547455 wait complete. 2024-05-21T21:55:36.664 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-21T21:55:36.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:36 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/3417245237' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-21T21:55:36.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:36 smithi184 ceph-mon[36942]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T21:55:36.737 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:36 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/2047882567' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T21:55:36.737 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:36 smithi184 ceph-mon[36942]: Active manager daemon x restarted 2024-05-21T21:55:36.737 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:36 smithi184 ceph-mon[36942]: Activating manager daemon x 2024-05-21T21:55:36.737 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:36 smithi184 ceph-mon[36942]: osdmap e83: 12 total, 12 up, 12 in 2024-05-21T21:55:36.737 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:36 smithi184 ceph-mon[36942]: mgrmap e22: x(active, starting, since 0.0144792s), standbys: z, y 2024-05-21T21:55:36.737 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:36 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-21T21:55:36.737 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:36 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T21:55:36.737 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:36 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T21:55:36.737 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:36 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-21T21:55:36.737 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:36 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-21T21:55:36.737 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:36 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-21T21:55:36.737 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:36 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-21T21:55:36.737 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:36 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-21T21:55:36.738 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:36 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-21T21:55:36.738 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:36 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-21T21:55:36.738 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:36 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-21T21:55:36.738 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:36 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-21T21:55:36.738 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:36 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-21T21:55:36.738 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:36 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-21T21:55:36.738 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:36 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-21T21:55:36.738 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:36 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-21T21:55:36.738 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:36 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-21T21:55:36.738 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:36 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-21T21:55:36.738 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:36 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-21T21:55:36.738 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:36 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-21T21:55:36.738 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:36 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-21T21:55:36.739 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:36 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-21T21:55:36.739 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:36 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-21T21:55:36.739 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:36 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-21T21:55:36.739 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:36 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-21T21:55:36.739 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:36 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-21T21:55:36.739 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:36 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-21T21:55:36.739 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:36 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-21T21:55:36.739 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:36 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-21T21:55:36.739 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:36 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-21T21:55:36.739 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:36 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-21T21:55:36.739 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:36 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-21T21:55:36.739 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:36 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-21T21:55:36.739 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:36 smithi184 ceph-mon[36942]: Manager daemon x is now available 2024-05-21T21:55:36.740 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:36 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:55:36.740 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:36 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-21T21:55:36.740 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:36 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-21T21:55:36.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:36 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/3417245237' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-21T21:55:36.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:36 smithi164 ceph-mon[36269]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T21:55:36.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:36 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/2047882567' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T21:55:36.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:36 smithi164 ceph-mon[36269]: Active manager daemon x restarted 2024-05-21T21:55:36.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:36 smithi164 ceph-mon[36269]: Activating manager daemon x 2024-05-21T21:55:36.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:36 smithi164 ceph-mon[36269]: osdmap e83: 12 total, 12 up, 12 in 2024-05-21T21:55:36.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:36 smithi164 ceph-mon[36269]: mgrmap e22: x(active, starting, since 0.0144792s), standbys: z, y 2024-05-21T21:55:36.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:36 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-21T21:55:36.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:36 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T21:55:36.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:36 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T21:55:36.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:36 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-21T21:55:36.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:36 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-21T21:55:36.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:36 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-21T21:55:36.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:36 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-21T21:55:36.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:36 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-21T21:55:36.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:36 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-21T21:55:36.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:36 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-21T21:55:36.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:36 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-21T21:55:36.788 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:36 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-21T21:55:36.788 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:36 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-21T21:55:36.788 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:36 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-21T21:55:36.788 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:36 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-21T21:55:36.788 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:36 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-21T21:55:36.788 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:36 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-21T21:55:36.788 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:36 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-21T21:55:36.788 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:36 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-21T21:55:36.788 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:36 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-21T21:55:36.789 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:36 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-21T21:55:36.789 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:36 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-21T21:55:36.789 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:36 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-21T21:55:36.789 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:36 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-21T21:55:36.789 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:36 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-21T21:55:36.789 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:36 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-21T21:55:36.789 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:36 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-21T21:55:36.789 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:36 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-21T21:55:36.789 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:36 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-21T21:55:36.789 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:36 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-21T21:55:36.789 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:36 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-21T21:55:36.790 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:36 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-21T21:55:36.790 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:36 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-21T21:55:36.790 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:36 smithi164 ceph-mon[36269]: Manager daemon x is now available 2024-05-21T21:55:36.790 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:36 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:55:36.790 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:36 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-21T21:55:36.790 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:36 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-21T21:55:36.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.846+0000 7f58d6a00640 1 -- 172.21.15.151:0/640851852 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f58d0103c90 msgr2=0x7f58d0104110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:36.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.846+0000 7f58d6a00640 1 --2- 172.21.15.151:0/640851852 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f58d0103c90 0x7f58d0104110 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f58bc009900 tx=0x7f58bc02a3f0 comp rx=0 tx=0).stop 2024-05-21T21:55:36.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.847+0000 7f58d6a00640 1 -- 172.21.15.151:0/640851852 shutdown_connections 2024-05-21T21:55:36.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.847+0000 7f58d6a00640 1 --2- 172.21.15.151:0/640851852 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f58d0104650 0x7f58d010aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:36.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.847+0000 7f58d6a00640 1 --2- 172.21.15.151:0/640851852 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f58d0103c90 0x7f58d0104110 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:36.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.847+0000 7f58d6a00640 1 --2- 172.21.15.151:0/640851852 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58d0102aa0 0x7f58d0102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:36.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.847+0000 7f58d6a00640 1 -- 172.21.15.151:0/640851852 >> 172.21.15.151:0/640851852 conn(0x7f58d00fe030 msgr2=0x7f58d0100450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:55:36.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.847+0000 7f58d6a00640 1 -- 172.21.15.151:0/640851852 shutdown_connections 2024-05-21T21:55:36.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.847+0000 7f58d6a00640 1 -- 172.21.15.151:0/640851852 wait complete. 2024-05-21T21:55:36.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.848+0000 7f58d6a00640 1 Processor -- start 2024-05-21T21:55:36.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.848+0000 7f58d6a00640 1 -- start start 2024-05-21T21:55:36.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.849+0000 7f58d6a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f58d0102aa0 0x7f58d0198d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:36.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.849+0000 7f58d6a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f58d0103c90 0x7f58d0199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:36.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.849+0000 7f58d6a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58d0104650 0x7f58d01a0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:36.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.849+0000 7f58d6a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f58d010e600 con 0x7f58d0103c90 2024-05-21T21:55:36.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.849+0000 7f58d6a00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f58d010e480 con 0x7f58d0104650 2024-05-21T21:55:36.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.849+0000 7f58d6a00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f58d010e780 con 0x7f58d0102aa0 2024-05-21T21:55:36.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.849+0000 7f58cfe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f58d0102aa0 0x7f58d0198d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:36.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.849+0000 7f58d4a00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58d0104650 0x7f58d01a0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:36.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.849+0000 7f58cf400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f58d0103c90 0x7f58d0199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:36.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.849+0000 7f58cfe00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f58d0102aa0 0x7f58d0198d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:43182/0 (socket says 172.21.15.151:43182) 2024-05-21T21:55:36.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.849+0000 7f58cfe00640 1 -- 172.21.15.151:0/3957657287 learned_addr learned my addr 172.21.15.151:0/3957657287 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:55:36.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.849+0000 7f58d4a00640 1 -- 172.21.15.151:0/3957657287 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f58d0102aa0 msgr2=0x7f58d0198d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:36.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.849+0000 7f58d4a00640 1 --2- 172.21.15.151:0/3957657287 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f58d0102aa0 0x7f58d0198d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:36.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.849+0000 7f58d4a00640 1 -- 172.21.15.151:0/3957657287 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f58d0103c90 msgr2=0x7f58d0199250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:36.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.849+0000 7f58d4a00640 1 --2- 172.21.15.151:0/3957657287 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f58d0103c90 0x7f58d0199250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:36.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.849+0000 7f58d4a00640 1 -- 172.21.15.151:0/3957657287 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58d01a0920 con 0x7f58d0104650 2024-05-21T21:55:36.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.850+0000 7f58d4a00640 1 --2- 172.21.15.151:0/3957657287 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58d0104650 0x7f58d01a0280 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f58c8009fd0 tx=0x7f58c800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:55:36.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.850+0000 7f58ccc00640 1 -- 172.21.15.151:0/3957657287 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c800ce20 con 0x7f58d0104650 2024-05-21T21:55:36.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.850+0000 7f58ccc00640 1 -- 172.21.15.151:0/3957657287 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58c8005020 con 0x7f58d0104650 2024-05-21T21:55:36.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.850+0000 7f58ccc00640 1 -- 172.21.15.151:0/3957657287 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c8010690 con 0x7f58d0104650 2024-05-21T21:55:36.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.850+0000 7f58d6a00640 1 -- 172.21.15.151:0/3957657287 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58d01a0c10 con 0x7f58d0104650 2024-05-21T21:55:36.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.850+0000 7f58d6a00640 1 -- 172.21.15.151:0/3957657287 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58d01a1110 con 0x7f58d0104650 2024-05-21T21:55:36.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.851+0000 7f58d6a00640 1 -- 172.21.15.151:0/3957657287 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58d0109eb0 con 0x7f58d0104650 2024-05-21T21:55:36.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.854+0000 7f58ccc00640 1 -- 172.21.15.151:0/3957657287 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 22) ==== 149337+0+0 (secure 0 0 0) 0x7f58c80054d0 con 0x7f58d0104650 2024-05-21T21:55:36.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.855+0000 7f58ccc00640 1 -- 172.21.15.151:0/3957657287 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f58c80e24f0 con 0x7f58d0104650 2024-05-21T21:55:36.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:36.859+0000 7f58ccc00640 1 -- 172.21.15.151:0/3957657287 <== mon.2 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f58c80aa500 con 0x7f58d0104650 2024-05-21T21:55:36.957 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:55:36 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:55:36.541+0000 7fbe64200700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-21T21:55:36.958 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:55:36 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:55:36.541+0000 7fbe64200700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-21T21:55:36.958 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:55:36 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:55:36.541+0000 7fbe64200700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-21T21:55:36.958 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:55:36 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:55:36.541+0000 7fbe64200700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-21T21:55:36.958 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:55:36 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:55:36.541+0000 7fbe64200700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-21T21:55:36.958 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:55:36 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:55:36.542+0000 7fbe67e00700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-21T21:55:36.958 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:55:36 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:55:36.542+0000 7fbe67e00700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-21T21:55:36.958 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:55:36 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:55:36.542+0000 7fbe67e00700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-21T21:55:36.958 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:55:36 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:55:36.542+0000 7fbe67e00700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-21T21:55:36.958 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:55:36 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:55:36.542+0000 7fbe67e00700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-21T21:55:36.959 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:36 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/3417245237' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-21T21:55:36.959 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:36 smithi151 ceph-mon[31623]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-21T21:55:36.959 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:36 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/2047882567' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T21:55:36.960 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:36 smithi151 ceph-mon[31623]: Active manager daemon x restarted 2024-05-21T21:55:36.960 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:36 smithi151 ceph-mon[31623]: Activating manager daemon x 2024-05-21T21:55:36.960 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:36 smithi151 ceph-mon[31623]: osdmap e83: 12 total, 12 up, 12 in 2024-05-21T21:55:36.960 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:36 smithi151 ceph-mon[31623]: mgrmap e22: x(active, starting, since 0.0144792s), standbys: z, y 2024-05-21T21:55:36.960 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:36 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-21T21:55:36.960 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:36 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-21T21:55:36.960 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:36 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-21T21:55:36.960 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:36 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-21T21:55:36.960 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:36 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-21T21:55:36.960 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:36 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-21T21:55:36.960 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:36 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-21T21:55:36.960 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:36 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-21T21:55:36.960 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:36 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-21T21:55:36.960 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:36 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-21T21:55:36.961 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:36 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-21T21:55:36.961 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:36 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-21T21:55:36.961 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:36 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-21T21:55:36.961 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:36 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-21T21:55:36.961 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:36 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-21T21:55:36.961 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:36 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-21T21:55:36.961 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:36 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-21T21:55:36.961 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:36 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-21T21:55:36.961 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:36 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-21T21:55:36.961 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:36 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-21T21:55:36.961 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:36 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-21T21:55:36.961 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:36 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-21T21:55:36.961 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:36 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-21T21:55:36.961 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:36 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-21T21:55:36.962 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:36 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-21T21:55:36.962 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:36 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-21T21:55:36.962 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:36 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-21T21:55:36.962 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:36 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-21T21:55:36.962 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:36 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-21T21:55:36.962 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:36 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-21T21:55:36.962 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:36 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-21T21:55:36.962 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:36 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-21T21:55:36.962 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:36 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-21T21:55:36.962 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:36 smithi151 ceph-mon[31623]: Manager daemon x is now available 2024-05-21T21:55:36.962 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:36 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:55:36.962 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:36 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-21T21:55:36.962 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:36 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-21T21:55:37.072 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:37.071+0000 7f58d6a00640 1 -- 172.21.15.151:0/3957657287 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f58d00636f0 con 0x7f58d0104650 2024-05-21T21:55:37.073 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:37.072+0000 7f58ccc00640 1 -- 172.21.15.151:0/3957657287 <== mon.2 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v83) ==== 104+0+11 (secure 0 0 0) 0x7f58c80a9e00 con 0x7f58d0104650 2024-05-21T21:55:37.073 INFO:teuthology.orchestra.run.smithi151.stdout:pg_num: 32 2024-05-21T21:55:37.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:37.074+0000 7f58d6a00640 1 -- 172.21.15.151:0/3957657287 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58d0104650 msgr2=0x7f58d01a0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:37.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:37.074+0000 7f58d6a00640 1 --2- 172.21.15.151:0/3957657287 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58d0104650 0x7f58d01a0280 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f58c8009fd0 tx=0x7f58c800ef10 comp rx=0 tx=0).stop 2024-05-21T21:55:37.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:37.074+0000 7f58d6a00640 1 -- 172.21.15.151:0/3957657287 shutdown_connections 2024-05-21T21:55:37.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:37.074+0000 7f58d6a00640 1 --2- 172.21.15.151:0/3957657287 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58d0104650 0x7f58d01a0280 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:37.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:37.075+0000 7f58d6a00640 1 --2- 172.21.15.151:0/3957657287 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f58d0103c90 0x7f58d0199250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:37.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:37.075+0000 7f58d6a00640 1 --2- 172.21.15.151:0/3957657287 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f58d0102aa0 0x7f58d0198d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:37.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:37.075+0000 7f58d6a00640 1 -- 172.21.15.151:0/3957657287 >> 172.21.15.151:0/3957657287 conn(0x7f58d00fe030 msgr2=0x7f58d0107080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:55:37.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:37.075+0000 7f58d6a00640 1 -- 172.21.15.151:0/3957657287 shutdown_connections 2024-05-21T21:55:37.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:37.075+0000 7f58d6a00640 1 -- 172.21.15.151:0/3957657287 wait complete. 2024-05-21T21:55:37.093 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-21T21:55:37.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:37.297+0000 7f74a6400640 1 -- 172.21.15.151:0/279512953 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74a01029e0 msgr2=0x7f74a0102e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:37.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:37.297+0000 7f74a6400640 1 --2- 172.21.15.151:0/279512953 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74a01029e0 0x7f74a0102e00 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7494009900 tx=0x7f749402a400 comp rx=0 tx=0).stop 2024-05-21T21:55:37.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:37.297+0000 7f74a6400640 1 -- 172.21.15.151:0/279512953 shutdown_connections 2024-05-21T21:55:37.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:37.297+0000 7f74a6400640 1 --2- 172.21.15.151:0/279512953 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74a0104590 0x7f74a010ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:37.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:37.297+0000 7f74a6400640 1 --2- 172.21.15.151:0/279512953 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74a0103bd0 0x7f74a0104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:37.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:37.297+0000 7f74a6400640 1 --2- 172.21.15.151:0/279512953 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74a01029e0 0x7f74a0102e00 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:37.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:37.297+0000 7f74a6400640 1 -- 172.21.15.151:0/279512953 >> 172.21.15.151:0/279512953 conn(0x7f74a00fdf50 msgr2=0x7f74a0100390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:55:37.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:37.297+0000 7f74a6400640 1 -- 172.21.15.151:0/279512953 shutdown_connections 2024-05-21T21:55:37.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:37.297+0000 7f74a6400640 1 -- 172.21.15.151:0/279512953 wait complete. 2024-05-21T21:55:37.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:37.298+0000 7f74a6400640 1 Processor -- start 2024-05-21T21:55:37.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:37.298+0000 7f74a6400640 1 -- start start 2024-05-21T21:55:37.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:37.299+0000 7f74a6400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74a01029e0 0x7f74a019ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:37.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:37.299+0000 7f74a6400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74a0103bd0 0x7f74a019b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:37.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:37.299+0000 7f74a6400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74a0104590 0x7f74a01a23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:37.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:37.299+0000 7f74a6400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f74a010e4f0 con 0x7f74a0103bd0 2024-05-21T21:55:37.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:37.299+0000 7f74a6400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f74a010e370 con 0x7f74a01029e0 2024-05-21T21:55:37.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:37.299+0000 7f74a6400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f74a010e670 con 0x7f74a0104590 2024-05-21T21:55:37.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:37.299+0000 7f749fe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74a0103bd0 0x7f74a019b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:37.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:37.299+0000 7f74a5000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74a01029e0 0x7f74a019ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:37.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:37.299+0000 7f74a5a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74a0104590 0x7f74a01a23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:37.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:37.299+0000 7f749fe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74a0103bd0 0x7f74a019b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:40722/0 (socket says 172.21.15.151:40722) 2024-05-21T21:55:37.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:37.299+0000 7f749fe00640 1 -- 172.21.15.151:0/3002267054 learned_addr learned my addr 172.21.15.151:0/3002267054 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:55:37.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:37.299+0000 7f749fe00640 1 -- 172.21.15.151:0/3002267054 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74a0104590 msgr2=0x7f74a01a23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:37.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:37.299+0000 7f749fe00640 1 --2- 172.21.15.151:0/3002267054 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74a0104590 0x7f74a01a23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:37.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:37.299+0000 7f749fe00640 1 -- 172.21.15.151:0/3002267054 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74a01029e0 msgr2=0x7f74a019ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:37.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:37.299+0000 7f749fe00640 1 --2- 172.21.15.151:0/3002267054 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74a01029e0 0x7f74a019ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:37.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:37.299+0000 7f749fe00640 1 -- 172.21.15.151:0/3002267054 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74a01a2a20 con 0x7f74a0103bd0 2024-05-21T21:55:37.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:37.299+0000 7f74a5a00640 1 --2- 172.21.15.151:0/3002267054 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74a0104590 0x7f74a01a23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:55:37.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:37.300+0000 7f749fe00640 1 --2- 172.21.15.151:0/3002267054 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74a0103bd0 0x7f74a019b340 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f748c00ea60 tx=0x7f748c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:55:37.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:37.300+0000 7f749d600640 1 -- 172.21.15.151:0/3002267054 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f748c00cde0 con 0x7f74a0103bd0 2024-05-21T21:55:37.303 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:37.300+0000 7f749d600640 1 -- 172.21.15.151:0/3002267054 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f748c0035f0 con 0x7f74a0103bd0 2024-05-21T21:55:37.303 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:37.300+0000 7f74a6400640 1 -- 172.21.15.151:0/3002267054 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74a01a2d10 con 0x7f74a0103bd0 2024-05-21T21:55:37.303 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:37.300+0000 7f749d600640 1 -- 172.21.15.151:0/3002267054 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f748c010690 con 0x7f74a0103bd0 2024-05-21T21:55:37.303 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:37.300+0000 7f74a6400640 1 -- 172.21.15.151:0/3002267054 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74a01a3200 con 0x7f74a0103bd0 2024-05-21T21:55:37.303 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:37.301+0000 7f74a6400640 1 -- 172.21.15.151:0/3002267054 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74a01a3490 con 0x7f74a0103bd0 2024-05-21T21:55:37.304 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:37.303+0000 7f749d600640 1 -- 172.21.15.151:0/3002267054 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 23) ==== 149464+0+0 (secure 0 0 0) 0x7f748c0037a0 con 0x7f74a0103bd0 2024-05-21T21:55:37.306 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:37.305+0000 7f749d600640 1 --2- 172.21.15.151:0/3002267054 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f74880b25e0 0x7f74880b4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:37.306 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:37.305+0000 7f749d600640 1 -- 172.21.15.151:0/3002267054 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f748c0e2200 con 0x7f74a0103bd0 2024-05-21T21:55:37.306 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:37.305+0000 7f74a5000640 1 --2- 172.21.15.151:0/3002267054 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f74880b25e0 0x7f74880b4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:37.307 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:37.306+0000 7f74a5000640 1 --2- 172.21.15.151:0/3002267054 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f74880b25e0 0x7f74880b4aa0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7494005ab0 tx=0x7f7494005a20 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:55:37.316 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:37.315+0000 7f749d600640 1 -- 172.21.15.151:0/3002267054 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f748c0aa0c0 con 0x7f74a0103bd0 2024-05-21T21:55:37.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:37.536+0000 7f74a6400640 1 -- 172.21.15.151:0/3002267054 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f74a01a3660 con 0x7f74a0103bd0 2024-05-21T21:55:37.538 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:37.537+0000 7f749d600640 1 -- 172.21.15.151:0/3002267054 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v83) ==== 100+0+11 (secure 0 0 0) 0x7f748c0b2140 con 0x7f74a0103bd0 2024-05-21T21:55:37.538 INFO:teuthology.orchestra.run.smithi151.stdout:pg_num: 64 2024-05-21T21:55:37.540 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:37.539+0000 7f74a6400640 1 -- 172.21.15.151:0/3002267054 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f74880b25e0 msgr2=0x7f74880b4aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:37.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:37.539+0000 7f74a6400640 1 --2- 172.21.15.151:0/3002267054 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f74880b25e0 0x7f74880b4aa0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7494005ab0 tx=0x7f7494005a20 comp rx=0 tx=0).stop 2024-05-21T21:55:37.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:37.539+0000 7f74a6400640 1 -- 172.21.15.151:0/3002267054 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74a0103bd0 msgr2=0x7f74a019b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:37.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:37.539+0000 7f74a6400640 1 --2- 172.21.15.151:0/3002267054 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74a0103bd0 0x7f74a019b340 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f748c00ea60 tx=0x7f748c00ef20 comp rx=0 tx=0).stop 2024-05-21T21:55:37.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:37.539+0000 7f74a6400640 1 -- 172.21.15.151:0/3002267054 shutdown_connections 2024-05-21T21:55:37.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:37.539+0000 7f74a6400640 1 --2- 172.21.15.151:0/3002267054 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f74880b25e0 0x7f74880b4aa0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:37.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:37.539+0000 7f74a6400640 1 --2- 172.21.15.151:0/3002267054 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74a0104590 0x7f74a01a23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:37.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:37.539+0000 7f74a6400640 1 --2- 172.21.15.151:0/3002267054 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74a0103bd0 0x7f74a019b340 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:37.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:37.539+0000 7f74a6400640 1 --2- 172.21.15.151:0/3002267054 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74a01029e0 0x7f74a019ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:37.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:37.539+0000 7f74a6400640 1 -- 172.21.15.151:0/3002267054 >> 172.21.15.151:0/3002267054 conn(0x7f74a00fdf50 msgr2=0x7f74a0074c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:55:37.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:37.540+0000 7f74a6400640 1 -- 172.21.15.151:0/3002267054 shutdown_connections 2024-05-21T21:55:37.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:37.540+0000 7f74a6400640 1 -- 172.21.15.151:0/3002267054 wait complete. 2024-05-21T21:55:37.558 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-05-21T21:55:37.558 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-21T21:55:37.858 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:37 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/355547455' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-21T21:55:37.858 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:37 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/3957657287' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-21T21:55:37.858 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:37 smithi164 ceph-mon[36269]: mgrmap e23: x(active, since 1.02978s), standbys: z, y 2024-05-21T21:55:37.858 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:37 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/3002267054' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-21T21:55:37.938 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:37 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/355547455' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-21T21:55:37.938 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:37 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/3957657287' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-21T21:55:37.938 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:37 smithi151 ceph-mon[31623]: mgrmap e23: x(active, since 1.02978s), standbys: z, y 2024-05-21T21:55:37.938 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:37 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/3002267054' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-21T21:55:37.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:37 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/355547455' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-21T21:55:37.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:37 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/3957657287' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-21T21:55:37.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:37 smithi184 ceph-mon[36942]: mgrmap e23: x(active, since 1.02978s), standbys: z, y 2024-05-21T21:55:37.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:37 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/3002267054' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-21T21:55:38.419 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:55:38.885 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:38 smithi151 ceph-mon[31623]: [21/May/2024:21:55:36] ENGINE Bus STARTING 2024-05-21T21:55:38.885 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:38 smithi151 ceph-mon[31623]: [21/May/2024:21:55:37] ENGINE Serving on https://172.21.15.151:7150 2024-05-21T21:55:38.885 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:38 smithi151 ceph-mon[31623]: [21/May/2024:21:55:37] ENGINE Serving on http://172.21.15.151:8765 2024-05-21T21:55:38.885 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:38 smithi151 ceph-mon[31623]: [21/May/2024:21:55:37] ENGINE Bus STARTED 2024-05-21T21:55:38.885 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:38 smithi151 ceph-mon[31623]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:55:38.885 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:38 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T21:55:38.886 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:38 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T21:55:38.886 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:38 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T21:55:38.886 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:38 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T21:55:38.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:38 smithi184 ceph-mon[36942]: [21/May/2024:21:55:36] ENGINE Bus STARTING 2024-05-21T21:55:38.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:38 smithi184 ceph-mon[36942]: [21/May/2024:21:55:37] ENGINE Serving on https://172.21.15.151:7150 2024-05-21T21:55:38.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:38 smithi184 ceph-mon[36942]: [21/May/2024:21:55:37] ENGINE Serving on http://172.21.15.151:8765 2024-05-21T21:55:38.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:38 smithi184 ceph-mon[36942]: [21/May/2024:21:55:37] ENGINE Bus STARTED 2024-05-21T21:55:38.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:38 smithi184 ceph-mon[36942]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:55:38.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:38 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T21:55:38.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:38 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T21:55:38.987 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:38 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T21:55:38.987 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:38 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T21:55:39.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:38 smithi164 ceph-mon[36269]: [21/May/2024:21:55:36] ENGINE Bus STARTING 2024-05-21T21:55:39.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:38 smithi164 ceph-mon[36269]: [21/May/2024:21:55:37] ENGINE Serving on https://172.21.15.151:7150 2024-05-21T21:55:39.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:38 smithi164 ceph-mon[36269]: [21/May/2024:21:55:37] ENGINE Serving on http://172.21.15.151:8765 2024-05-21T21:55:39.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:38 smithi164 ceph-mon[36269]: [21/May/2024:21:55:37] ENGINE Bus STARTED 2024-05-21T21:55:39.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:38 smithi164 ceph-mon[36269]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:55:39.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:38 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T21:55:39.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:38 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T21:55:39.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:38 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T21:55:39.037 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:38 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T21:55:39.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:39 smithi151 ceph-mon[31623]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:55:39.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:39 smithi151 ceph-mon[31623]: mgrmap e24: x(active, since 2s), standbys: z, y 2024-05-21T21:55:39.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:39 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T21:55:39.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:39 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T21:55:39.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:39 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:55:39.705 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:39 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:55:39.705 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:39 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:55:39.705 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:39 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:55:39.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:39 smithi184 ceph-mon[36942]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:55:39.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:39 smithi184 ceph-mon[36942]: mgrmap e24: x(active, since 2s), standbys: z, y 2024-05-21T21:55:39.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:39 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T21:55:39.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:39 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T21:55:39.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:39 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:55:39.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:39 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:55:39.987 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:39 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:55:39.987 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:39 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:55:40.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:39 smithi164 ceph-mon[36269]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:55:40.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:39 smithi164 ceph-mon[36269]: mgrmap e24: x(active, since 2s), standbys: z, y 2024-05-21T21:55:40.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:39 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T21:55:40.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:39 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T21:55:40.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:39 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:55:40.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:39 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:55:40.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:39 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:55:40.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:39 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:55:40.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:40.578+0000 7efda8c00700 1 -- 172.21.15.151:0/539451531 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efda41088f0 msgr2=0x7efda4110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:40.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:40.578+0000 7efda8c00700 1 --2- 172.21.15.151:0/539451531 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efda41088f0 0x7efda4110470 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7efd9c009fc0 tx=0x7efd9c00b330 comp rx=0 tx=0).stop 2024-05-21T21:55:40.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:40.578+0000 7efda8c00700 1 -- 172.21.15.151:0/539451531 shutdown_connections 2024-05-21T21:55:40.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:40.578+0000 7efda8c00700 1 --2- 172.21.15.151:0/539451531 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efda41088f0 0x7efda4110470 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:40.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:40.578+0000 7efda8c00700 1 --2- 172.21.15.151:0/539451531 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efda41037f0 0x7efda4103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:40.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:40.578+0000 7efda8c00700 1 --2- 172.21.15.151:0/539451531 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efda41041a0 0x7efda41081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:40.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:40.578+0000 7efda8c00700 1 -- 172.21.15.151:0/539451531 >> 172.21.15.151:0/539451531 conn(0x7efda407a6a0 msgr2=0x7efda407aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:55:40.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:40.578+0000 7efda8c00700 1 -- 172.21.15.151:0/539451531 shutdown_connections 2024-05-21T21:55:40.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:40.578+0000 7efda8c00700 1 -- 172.21.15.151:0/539451531 wait complete. 2024-05-21T21:55:40.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:40.579+0000 7efda8c00700 1 Processor -- start 2024-05-21T21:55:40.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:40.579+0000 7efda8c00700 1 -- start start 2024-05-21T21:55:40.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:40.580+0000 7efda8c00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efda41037f0 0x7efda4075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:40.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:40.580+0000 7efda8c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efda41041a0 0x7efda4075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:40.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:40.580+0000 7efda8c00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efda41088f0 0x7efda40762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:40.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:40.580+0000 7efda8c00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7efda41136e0 con 0x7efda41041a0 2024-05-21T21:55:40.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:40.580+0000 7efda8c00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7efda4113560 con 0x7efda41088f0 2024-05-21T21:55:40.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:40.580+0000 7efda8c00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7efda4113860 con 0x7efda41037f0 2024-05-21T21:55:40.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:40.580+0000 7efda1e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efda41037f0 0x7efda4075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:40.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:40.580+0000 7efda1e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efda41037f0 0x7efda4075840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:43244/0 (socket says 172.21.15.151:43244) 2024-05-21T21:55:40.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:40.580+0000 7efda1e00700 1 -- 172.21.15.151:0/2387641785 learned_addr learned my addr 172.21.15.151:0/2387641785 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:55:40.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:40.580+0000 7efda1400700 1 --2- 172.21.15.151:0/2387641785 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efda41041a0 0x7efda4075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:40.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:40.580+0000 7efda2800700 1 --2- 172.21.15.151:0/2387641785 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efda41088f0 0x7efda40762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:40.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:40.581+0000 7efda1e00700 1 -- 172.21.15.151:0/2387641785 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efda41088f0 msgr2=0x7efda40762c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:40.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:40.581+0000 7efda1e00700 1 --2- 172.21.15.151:0/2387641785 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efda41088f0 0x7efda40762c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:40.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:40.581+0000 7efda1e00700 1 -- 172.21.15.151:0/2387641785 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efda41041a0 msgr2=0x7efda4075d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:40.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:40.581+0000 7efda1e00700 1 --2- 172.21.15.151:0/2387641785 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efda41041a0 0x7efda4075d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:40.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:40.581+0000 7efda1e00700 1 -- 172.21.15.151:0/2387641785 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efda41a8030 con 0x7efda41037f0 2024-05-21T21:55:40.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:40.581+0000 7efda1e00700 1 --2- 172.21.15.151:0/2387641785 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efda41037f0 0x7efda4075840 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7efd9400e9c0 tx=0x7efd9400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:55:40.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:40.581+0000 7efd9aa00700 1 -- 172.21.15.151:0/2387641785 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd9400cbd0 con 0x7efda41037f0 2024-05-21T21:55:40.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:40.582+0000 7efda8c00700 1 -- 172.21.15.151:0/2387641785 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efda41a8320 con 0x7efda41037f0 2024-05-21T21:55:40.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:40.582+0000 7efd9aa00700 1 -- 172.21.15.151:0/2387641785 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd9400ee50 con 0x7efda41037f0 2024-05-21T21:55:40.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:40.582+0000 7efd9aa00700 1 -- 172.21.15.151:0/2387641785 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd94005550 con 0x7efda41037f0 2024-05-21T21:55:40.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:40.582+0000 7efda8c00700 1 -- 172.21.15.151:0/2387641785 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efda41a8860 con 0x7efda41037f0 2024-05-21T21:55:40.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:40.583+0000 7efda8c00700 1 -- 172.21.15.151:0/2387641785 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efda410bac0 con 0x7efda41037f0 2024-05-21T21:55:40.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:40.585+0000 7efd9aa00700 1 -- 172.21.15.151:0/2387641785 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 24) ==== 149773+0+0 (secure 0 0 0) 0x7efd94004750 con 0x7efda41037f0 2024-05-21T21:55:40.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:40.587+0000 7efd9aa00700 1 --2- 172.21.15.151:0/2387641785 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7efd880b2a70 0x7efd880b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:40.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:40.588+0000 7efd9aa00700 1 -- 172.21.15.151:0/2387641785 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7efd940e2a80 con 0x7efda41037f0 2024-05-21T21:55:40.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:40.588+0000 7efda1400700 1 --2- 172.21.15.151:0/2387641785 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7efd880b2a70 0x7efd880b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:40.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:40.588+0000 7efda1400700 1 --2- 172.21.15.151:0/2387641785 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7efd880b2a70 0x7efd880b4f30 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7efda41030f0 tx=0x7efd90009310 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:55:40.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:40.589+0000 7efd9aa00700 1 -- 172.21.15.151:0/2387641785 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7efd94054df0 con 0x7efda41037f0 2024-05-21T21:55:40.686 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:40.684+0000 7efd9aa00700 1 -- 172.21.15.151:0/2387641785 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mgrmap(e 25) ==== 149829+0+0 (secure 0 0 0) 0x7efd940a5fa0 con 0x7efda41037f0 2024-05-21T21:55:40.689 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:40 smithi151 ceph-mon[31623]: Adjusting osd_memory_target on smithi184 to 176.2M 2024-05-21T21:55:40.689 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:40 smithi151 ceph-mon[31623]: Unable to set osd_memory_target on smithi184 to 184791040: error parsing value: Value '184791040' is below minimum 939524096 2024-05-21T21:55:40.689 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:40 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T21:55:40.689 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:40 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T21:55:40.689 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:40 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:55:40.689 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:40 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:55:40.689 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:40 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:55:40.689 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:40 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:55:40.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:40.879+0000 7efda8c00700 1 -- 172.21.15.151:0/2387641785 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efda406a230 con 0x7efda41037f0 2024-05-21T21:55:40.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:40.881+0000 7efd9aa00700 1 -- 172.21.15.151:0/2387641785 <== mon.1 v2:172.21.15.184:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11958 (secure 0 0 0) 0x7efd940aa410 con 0x7efda41037f0 2024-05-21T21:55:40.882 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T21:55:40.883 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14358,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/158010529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":158010529},{"type":"v1","addr":"172.21.15.151:6835","nonce":158010529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6839/2069058115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":2069058115},{"type":"v1","addr":"172.21.15.151:6839","nonce":2069058115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T21:51:19.552069+0000","modified":"2024-05-21T21:55:34.643755+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24530,"mds_1":24493,"mds_2":24491,"mds_3":24478,"mds_4":24473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14403":{"gid":14403,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6841/1749723794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6840","nonce":1749723794},{"type":"v1","addr":"172.21.15.151:6841","nonce":1749723794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/2026266114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":2026266114},{"type":"v1","addr":"172.21.15.184:6833","nonce":2026266114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/2467900844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":2467900844},{"type":"v1","addr":"172.21.15.164:6833","nonce":2467900844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3367640960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3367640960},{"type":"v1","addr":"172.21.15.151:6837","nonce":3367640960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/3264142621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3264142621},{"type":"v1","addr":"172.21.15.184:6835","nonce":3264142621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":17,"addr":"172.21.15.164:6835/2340428470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2340428470},{"type":"v1","addr":"172.21.15.164:6835","nonce":2340428470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":16,"addr":"172.21.15.184:6837/2988194888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":2988194888},{"type":"v1","addr":"172.21.15.184:6837","nonce":2988194888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.164:6837/1554609572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1554609572},{"type":"v1","addr":"172.21.15.164:6837","nonce":1554609572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.184:6839/3059935143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":3059935143},{"type":"v1","addr":"172.21.15.184:6839","nonce":3059935143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.164:6839/2646401247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":2646401247},{"type":"v1","addr":"172.21.15.164:6839","nonce":2646401247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24530,"qdb_cluster":[24530,24493,24491,24473,24478]},"id":1}]} 2024-05-21T21:55:40.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:40.884+0000 7efda8c00700 1 -- 172.21.15.151:0/2387641785 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7efd880b2a70 msgr2=0x7efd880b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:40.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:40.884+0000 7efda8c00700 1 --2- 172.21.15.151:0/2387641785 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7efd880b2a70 0x7efd880b4f30 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7efda41030f0 tx=0x7efd90009310 comp rx=0 tx=0).stop 2024-05-21T21:55:40.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:40.884+0000 7efda8c00700 1 -- 172.21.15.151:0/2387641785 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efda41037f0 msgr2=0x7efda4075840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:40.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:40.884+0000 7efda8c00700 1 --2- 172.21.15.151:0/2387641785 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efda41037f0 0x7efda4075840 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7efd9400e9c0 tx=0x7efd9400ecd0 comp rx=0 tx=0).stop 2024-05-21T21:55:40.886 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:40.884+0000 7efda8c00700 1 -- 172.21.15.151:0/2387641785 shutdown_connections 2024-05-21T21:55:40.886 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:40.884+0000 7efda8c00700 1 --2- 172.21.15.151:0/2387641785 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7efd880b2a70 0x7efd880b4f30 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:40.886 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:40.884+0000 7efda8c00700 1 --2- 172.21.15.151:0/2387641785 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efda41088f0 0x7efda40762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:40.886 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:40.884+0000 7efda8c00700 1 --2- 172.21.15.151:0/2387641785 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efda41037f0 0x7efda4075840 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:40.886 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:40.884+0000 7efda8c00700 1 --2- 172.21.15.151:0/2387641785 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efda41041a0 0x7efda4075d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:40.886 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:40.884+0000 7efda8c00700 1 -- 172.21.15.151:0/2387641785 >> 172.21.15.151:0/2387641785 conn(0x7efda407a6a0 msgr2=0x7efda4101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:55:40.886 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:40.884+0000 7efda8c00700 1 -- 172.21.15.151:0/2387641785 shutdown_connections 2024-05-21T21:55:40.886 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:40.885+0000 7efda8c00700 1 -- 172.21.15.151:0/2387641785 wait complete. 2024-05-21T21:55:40.887 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 29 2024-05-21T21:55:40.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:40 smithi184 ceph-mon[36942]: Adjusting osd_memory_target on smithi184 to 176.2M 2024-05-21T21:55:40.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:40 smithi184 ceph-mon[36942]: Unable to set osd_memory_target on smithi184 to 184791040: error parsing value: Value '184791040' is below minimum 939524096 2024-05-21T21:55:40.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:40 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T21:55:40.987 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:40 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T21:55:40.987 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:40 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:55:40.987 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:40 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:55:40.987 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:40 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:55:40.987 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:40 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:55:41.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:40 smithi164 ceph-mon[36269]: Adjusting osd_memory_target on smithi184 to 176.2M 2024-05-21T21:55:41.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:40 smithi164 ceph-mon[36269]: Unable to set osd_memory_target on smithi184 to 184791040: error parsing value: Value '184791040' is below minimum 939524096 2024-05-21T21:55:41.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:40 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T21:55:41.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:40 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T21:55:41.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:40 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:55:41.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:40 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:55:41.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:40 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:55:41.037 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:40 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:55:41.660 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-05-21T21:55:41.660 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-05-21T21:55:41.660 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-21T21:55:41.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:41.847+0000 7eff6f600640 1 -- 172.21.15.151:0/1972703158 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff68077410 msgr2=0x7eff68077830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:41.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:41.847+0000 7eff6f600640 1 --2- 172.21.15.151:0/1972703158 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff68077410 0x7eff68077830 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7eff5c009900 tx=0x7eff5c02a410 comp rx=0 tx=0).stop 2024-05-21T21:55:41.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:41.847+0000 7eff6f600640 1 -- 172.21.15.151:0/1972703158 shutdown_connections 2024-05-21T21:55:41.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:41.847+0000 7eff6f600640 1 --2- 172.21.15.151:0/1972703158 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff68078ff0 0x7eff68079890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:41.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:41.847+0000 7eff6f600640 1 --2- 172.21.15.151:0/1972703158 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff68078650 0x7eff68078ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:41.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:41.847+0000 7eff6f600640 1 --2- 172.21.15.151:0/1972703158 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff68077410 0x7eff68077830 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:41.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:41.847+0000 7eff6f600640 1 -- 172.21.15.151:0/1972703158 >> 172.21.15.151:0/1972703158 conn(0x7eff68100240 msgr2=0x7eff68102660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:55:41.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:41.847+0000 7eff6f600640 1 -- 172.21.15.151:0/1972703158 shutdown_connections 2024-05-21T21:55:41.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:41.847+0000 7eff6f600640 1 -- 172.21.15.151:0/1972703158 wait complete. 2024-05-21T21:55:41.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:41.848+0000 7eff6f600640 1 Processor -- start 2024-05-21T21:55:41.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:41.848+0000 7eff6f600640 1 -- start start 2024-05-21T21:55:41.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:41.849+0000 7eff6f600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff68077410 0x7eff6819f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:41.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:41.849+0000 7eff6f600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff68078650 0x7eff6819f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:41.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:41.849+0000 7eff6f600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff68078ff0 0x7eff681a68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:41.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:41.849+0000 7eff6f600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7eff68114bf0 con 0x7eff68078650 2024-05-21T21:55:41.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:41.849+0000 7eff6f600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7eff68114a70 con 0x7eff68077410 2024-05-21T21:55:41.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:41.849+0000 7eff6f600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7eff68114d70 con 0x7eff68078ff0 2024-05-21T21:55:41.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:41.849+0000 7eff6d600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff68078ff0 0x7eff681a68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:41.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:41.849+0000 7eff6d600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff68078ff0 0x7eff681a68b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:43258/0 (socket says 172.21.15.151:43258) 2024-05-21T21:55:41.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:41.849+0000 7eff67e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff68078650 0x7eff6819f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:41.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:41.849+0000 7eff6d600640 1 -- 172.21.15.151:0/4212653618 learned_addr learned my addr 172.21.15.151:0/4212653618 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:55:41.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:41.849+0000 7eff6cc00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff68077410 0x7eff6819f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:41.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:41.850+0000 7eff6d600640 1 -- 172.21.15.151:0/4212653618 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff68077410 msgr2=0x7eff6819f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:41.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:41.850+0000 7eff6d600640 1 --2- 172.21.15.151:0/4212653618 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff68077410 0x7eff6819f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:41.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:41.850+0000 7eff6d600640 1 -- 172.21.15.151:0/4212653618 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff68078650 msgr2=0x7eff6819f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:41.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:41.850+0000 7eff6d600640 1 --2- 172.21.15.151:0/4212653618 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff68078650 0x7eff6819f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:41.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:41.850+0000 7eff6d600640 1 -- 172.21.15.151:0/4212653618 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff681a6f20 con 0x7eff68078ff0 2024-05-21T21:55:41.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:41.850+0000 7eff6d600640 1 --2- 172.21.15.151:0/4212653618 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff68078ff0 0x7eff681a68b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7eff6000efd0 tx=0x7eff6000c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:55:41.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:41.850+0000 7eff65600640 1 -- 172.21.15.151:0/4212653618 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff60003b70 con 0x7eff68078ff0 2024-05-21T21:55:41.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:41.850+0000 7eff6f600640 1 -- 172.21.15.151:0/4212653618 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff681a7210 con 0x7eff68078ff0 2024-05-21T21:55:41.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:41.850+0000 7eff65600640 1 -- 172.21.15.151:0/4212653618 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff60003d10 con 0x7eff68078ff0 2024-05-21T21:55:41.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:41.850+0000 7eff65600640 1 -- 172.21.15.151:0/4212653618 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff600108a0 con 0x7eff68078ff0 2024-05-21T21:55:41.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:41.850+0000 7eff6f600640 1 -- 172.21.15.151:0/4212653618 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff681a7710 con 0x7eff68078ff0 2024-05-21T21:55:41.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:41.852+0000 7eff6f600640 1 -- 172.21.15.151:0/4212653618 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff68111860 con 0x7eff68078ff0 2024-05-21T21:55:41.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:41.853+0000 7eff65600640 1 -- 172.21.15.151:0/4212653618 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 25) ==== 149829+0+0 (secure 0 0 0) 0x7eff60004020 con 0x7eff68078ff0 2024-05-21T21:55:41.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:41.855+0000 7eff65600640 1 --2- 172.21.15.151:0/4212653618 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7eff500b29f0 0x7eff500b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:41.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:41.855+0000 7eff65600640 1 -- 172.21.15.151:0/4212653618 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7eff600e2c60 con 0x7eff68078ff0 2024-05-21T21:55:41.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:41.855+0000 7eff6cc00640 1 --2- 172.21.15.151:0/4212653618 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7eff500b29f0 0x7eff500b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:41.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:41.856+0000 7eff6cc00640 1 --2- 172.21.15.151:0/4212653618 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7eff500b29f0 0x7eff500b4eb0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7eff5c031500 tx=0x7eff5c005a20 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:55:41.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:41.859+0000 7eff65600640 1 -- 172.21.15.151:0/4212653618 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7eff600aa480 con 0x7eff68078ff0 2024-05-21T21:55:41.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:41 smithi151 ceph-mon[31623]: Adjusting osd_memory_target on smithi164 to 176.2M 2024-05-21T21:55:41.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:41 smithi151 ceph-mon[31623]: Unable to set osd_memory_target on smithi164 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-05-21T21:55:41.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:41 smithi151 ceph-mon[31623]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:55:41.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:41 smithi151 ceph-mon[31623]: mgrmap e25: x(active, since 4s), standbys: z, y 2024-05-21T21:55:41.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:41 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T21:55:41.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:41 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T21:55:41.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:41 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/2387641785' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T21:55:41.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:41 smithi184 ceph-mon[36942]: Adjusting osd_memory_target on smithi164 to 176.2M 2024-05-21T21:55:41.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:41 smithi184 ceph-mon[36942]: Unable to set osd_memory_target on smithi164 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-05-21T21:55:41.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:41 smithi184 ceph-mon[36942]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:55:41.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:41 smithi184 ceph-mon[36942]: mgrmap e25: x(active, since 4s), standbys: z, y 2024-05-21T21:55:41.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:41 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T21:55:41.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:41 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T21:55:41.987 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:41 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/2387641785' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T21:55:42.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:41 smithi164 ceph-mon[36269]: Adjusting osd_memory_target on smithi164 to 176.2M 2024-05-21T21:55:42.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:41 smithi164 ceph-mon[36269]: Unable to set osd_memory_target on smithi164 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-05-21T21:55:42.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:41 smithi164 ceph-mon[36269]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:55:42.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:41 smithi164 ceph-mon[36269]: mgrmap e25: x(active, since 4s), standbys: z, y 2024-05-21T21:55:42.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:41 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T21:55:42.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:41 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T21:55:42.037 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:41 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/2387641785' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T21:55:42.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.059+0000 7eff6f600640 1 -- 172.21.15.151:0/4212653618 --> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7eff68062e90 con 0x7eff500b29f0 2024-05-21T21:55:42.069 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.068+0000 7eff65600640 1 -- 172.21.15.151:0/4212653618 <== mgr.25138 v2:172.21.15.151:6800/958417056 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257822 (secure 0 0 0) 0x7eff68062e90 con 0x7eff500b29f0 2024-05-21T21:55:42.073 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T21:55:42.075 INFO:teuthology.orchestra.run.smithi151.stderr:dumped all 2024-05-21T21:55:42.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.073+0000 7eff6f600640 1 -- 172.21.15.151:0/4212653618 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7eff500b29f0 msgr2=0x7eff500b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:42.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.073+0000 7eff6f600640 1 --2- 172.21.15.151:0/4212653618 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7eff500b29f0 0x7eff500b4eb0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7eff5c031500 tx=0x7eff5c005a20 comp rx=0 tx=0).stop 2024-05-21T21:55:42.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.073+0000 7eff6f600640 1 -- 172.21.15.151:0/4212653618 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff68078ff0 msgr2=0x7eff681a68b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:42.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.073+0000 7eff6f600640 1 --2- 172.21.15.151:0/4212653618 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff68078ff0 0x7eff681a68b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7eff6000efd0 tx=0x7eff6000c6a0 comp rx=0 tx=0).stop 2024-05-21T21:55:42.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.073+0000 7eff6f600640 1 -- 172.21.15.151:0/4212653618 shutdown_connections 2024-05-21T21:55:42.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.073+0000 7eff6f600640 1 --2- 172.21.15.151:0/4212653618 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7eff500b29f0 0x7eff500b4eb0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:42.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.073+0000 7eff6f600640 1 --2- 172.21.15.151:0/4212653618 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff68078ff0 0x7eff681a68b0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:42.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.073+0000 7eff6f600640 1 --2- 172.21.15.151:0/4212653618 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff68078650 0x7eff6819f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:42.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.073+0000 7eff6f600640 1 --2- 172.21.15.151:0/4212653618 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff68077410 0x7eff6819f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:42.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.073+0000 7eff6f600640 1 -- 172.21.15.151:0/4212653618 >> 172.21.15.151:0/4212653618 conn(0x7eff68100240 msgr2=0x7eff681019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:55:42.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.073+0000 7eff6f600640 1 -- 172.21.15.151:0/4212653618 shutdown_connections 2024-05-21T21:55:42.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.074+0000 7eff6f600640 1 -- 172.21.15.151:0/4212653618 wait complete. 2024-05-21T21:55:42.095 INFO:teuthology.orchestra.run.smithi151.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-05-21T21:55:40.219437+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1191254,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":2602,"num_read_kb":1445,"num_write":502,"num_write_kb":3619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":2952069,"num_keys_recovered":17,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":898,"ondisk_log_size":898,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":366336,"kb_used_data":13000,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124575488,"statfs":{"total":1151940427776,"available":1151565299712,"internally_reserved":0,"allocated":13312000,"data_stored":7384855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19140,"internal_metadata":361674044},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.978195"},"pg_stats":[{"pgid":"3.3f","version":"82'8","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.341334+0000","last_change":"2024-05-21T21:51:16.945132+0000","last_active":"2024-05-21T21:55:36.341334+0000","last_peered":"2024-05-21T21:55:36.341334+0000","last_clean":"2024-05-21T21:55:36.341334+0000","last_became_active":"2024-05-21T21:51:16.944766+0000","last_became_peered":"2024-05-21T21:51:16.944766+0000","last_unstale":"2024-05-21T21:55:36.341334+0000","last_undegraded":"2024-05-21T21:55:36.341334+0000","last_fullsized":"2024-05-21T21:55:36.341334+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:54:04.185095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"82'4","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.553624+0000","last_change":"2024-05-21T21:51:16.939291+0000","last_active":"2024-05-21T21:55:36.553624+0000","last_peered":"2024-05-21T21:55:36.553624+0000","last_clean":"2024-05-21T21:55:36.553624+0000","last_became_active":"2024-05-21T21:51:16.938730+0000","last_became_peered":"2024-05-21T21:51:16.938730+0000","last_unstale":"2024-05-21T21:55:36.553624+0000","last_undegraded":"2024-05-21T21:55:36.553624+0000","last_fullsized":"2024-05-21T21:55:36.553624+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:05:30.623744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"82'7","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.553252+0000","last_change":"2024-05-21T21:51:16.941565+0000","last_active":"2024-05-21T21:55:36.553252+0000","last_peered":"2024-05-21T21:55:36.553252+0000","last_clean":"2024-05-21T21:55:36.553252+0000","last_became_active":"2024-05-21T21:51:16.941033+0000","last_became_peered":"2024-05-21T21:51:16.941033+0000","last_unstale":"2024-05-21T21:55:36.553252+0000","last_undegraded":"2024-05-21T21:55:36.553252+0000","last_fullsized":"2024-05-21T21:55:36.553252+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:04:55.789479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"82'8","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.723315+0000","last_change":"2024-05-21T21:51:16.928455+0000","last_active":"2024-05-21T21:55:36.723315+0000","last_peered":"2024-05-21T21:55:36.723315+0000","last_clean":"2024-05-21T21:55:36.723315+0000","last_became_active":"2024-05-21T21:51:16.927874+0000","last_became_peered":"2024-05-21T21:51:16.927874+0000","last_unstale":"2024-05-21T21:55:36.723315+0000","last_undegraded":"2024-05-21T21:55:36.723315+0000","last_fullsized":"2024-05-21T21:55:36.723315+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:25:16.483175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"82'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.556362+0000","last_change":"2024-05-21T21:51:16.937773+0000","last_active":"2024-05-21T21:55:36.556362+0000","last_peered":"2024-05-21T21:55:36.556362+0000","last_clean":"2024-05-21T21:55:36.556362+0000","last_became_active":"2024-05-21T21:51:16.937306+0000","last_became_peered":"2024-05-21T21:51:16.937306+0000","last_unstale":"2024-05-21T21:55:36.556362+0000","last_undegraded":"2024-05-21T21:55:36.556362+0000","last_fullsized":"2024-05-21T21:55:36.556362+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:05:03.762390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"82'6","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.356010+0000","last_change":"2024-05-21T21:51:16.948275+0000","last_active":"2024-05-21T21:55:36.356010+0000","last_peered":"2024-05-21T21:55:36.356010+0000","last_clean":"2024-05-21T21:55:36.356010+0000","last_became_active":"2024-05-21T21:51:16.947890+0000","last_became_peered":"2024-05-21T21:51:16.947890+0000","last_unstale":"2024-05-21T21:55:36.356010+0000","last_undegraded":"2024-05-21T21:55:36.356010+0000","last_fullsized":"2024-05-21T21:55:36.356010+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:08:38.572177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"82'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.550575+0000","last_change":"2024-05-21T21:51:16.946821+0000","last_active":"2024-05-21T21:55:36.550575+0000","last_peered":"2024-05-21T21:55:36.550575+0000","last_clean":"2024-05-21T21:55:36.550575+0000","last_became_active":"2024-05-21T21:51:16.946515+0000","last_became_peered":"2024-05-21T21:51:16.946515+0000","last_unstale":"2024-05-21T21:55:36.550575+0000","last_undegraded":"2024-05-21T21:55:36.550575+0000","last_fullsized":"2024-05-21T21:55:36.550575+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T09:26:25.867895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.342252+0000","last_change":"2024-05-21T21:51:16.929228+0000","last_active":"2024-05-21T21:55:36.342252+0000","last_peered":"2024-05-21T21:55:36.342252+0000","last_clean":"2024-05-21T21:55:36.342252+0000","last_became_active":"2024-05-21T21:51:16.928875+0000","last_became_peered":"2024-05-21T21:51:16.928875+0000","last_unstale":"2024-05-21T21:55:36.342252+0000","last_undegraded":"2024-05-21T21:55:36.342252+0000","last_fullsized":"2024-05-21T21:55:36.342252+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:50:56.431310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"82'11","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.199749+0000","last_change":"2024-05-21T21:51:16.932360+0000","last_active":"2024-05-21T21:55:36.199749+0000","last_peered":"2024-05-21T21:55:36.199749+0000","last_clean":"2024-05-21T21:55:36.199749+0000","last_became_active":"2024-05-21T21:51:16.932036+0000","last_became_peered":"2024-05-21T21:51:16.932036+0000","last_unstale":"2024-05-21T21:55:36.199749+0000","last_undegraded":"2024-05-21T21:55:36.199749+0000","last_fullsized":"2024-05-21T21:55:36.199749+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T09:06:37.257323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"82'16","reported_seq":45,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.356382+0000","last_change":"2024-05-21T21:52:10.876556+0000","last_active":"2024-05-21T21:55:36.356382+0000","last_peered":"2024-05-21T21:55:36.356382+0000","last_clean":"2024-05-21T21:55:36.356382+0000","last_became_active":"2024-05-21T21:52:10.876273+0000","last_became_peered":"2024-05-21T21:52:10.876273+0000","last_unstale":"2024-05-21T21:55:36.356382+0000","last_undegraded":"2024-05-21T21:55:36.356382+0000","last_fullsized":"2024-05-21T21:55:36.356382+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T05:09:40.037731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"82'6","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.550761+0000","last_change":"2024-05-21T21:51:16.933425+0000","last_active":"2024-05-21T21:55:36.550761+0000","last_peered":"2024-05-21T21:55:36.550761+0000","last_clean":"2024-05-21T21:55:36.550761+0000","last_became_active":"2024-05-21T21:51:16.933057+0000","last_became_peered":"2024-05-21T21:51:16.933057+0000","last_unstale":"2024-05-21T21:55:36.550761+0000","last_undegraded":"2024-05-21T21:55:36.550761+0000","last_fullsized":"2024-05-21T21:55:36.550761+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:04:57.237841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"82'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.723313+0000","last_change":"2024-05-21T21:51:16.927014+0000","last_active":"2024-05-21T21:55:36.723313+0000","last_peered":"2024-05-21T21:55:36.723313+0000","last_clean":"2024-05-21T21:55:36.723313+0000","last_became_active":"2024-05-21T21:51:16.925954+0000","last_became_peered":"2024-05-21T21:51:16.925954+0000","last_unstale":"2024-05-21T21:55:36.723313+0000","last_undegraded":"2024-05-21T21:55:36.723313+0000","last_fullsized":"2024-05-21T21:55:36.723313+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:19:20.530727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"82'10","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.555649+0000","last_change":"2024-05-21T21:51:16.930909+0000","last_active":"2024-05-21T21:55:36.555649+0000","last_peered":"2024-05-21T21:55:36.555649+0000","last_clean":"2024-05-21T21:55:36.555649+0000","last_became_active":"2024-05-21T21:51:16.927110+0000","last_became_peered":"2024-05-21T21:51:16.927110+0000","last_unstale":"2024-05-21T21:55:36.555649+0000","last_undegraded":"2024-05-21T21:55:36.555649+0000","last_fullsized":"2024-05-21T21:55:36.555649+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T06:23:55.386047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"82'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.552564+0000","last_change":"2024-05-21T21:51:16.938797+0000","last_active":"2024-05-21T21:55:36.552564+0000","last_peered":"2024-05-21T21:55:36.552564+0000","last_clean":"2024-05-21T21:55:36.552564+0000","last_became_active":"2024-05-21T21:51:16.938263+0000","last_became_peered":"2024-05-21T21:51:16.938263+0000","last_unstale":"2024-05-21T21:55:36.552564+0000","last_undegraded":"2024-05-21T21:55:36.552564+0000","last_fullsized":"2024-05-21T21:55:36.552564+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:37:40.521118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"82'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.552309+0000","last_change":"2024-05-21T21:51:16.944160+0000","last_active":"2024-05-21T21:55:36.552309+0000","last_peered":"2024-05-21T21:55:36.552309+0000","last_clean":"2024-05-21T21:55:36.552309+0000","last_became_active":"2024-05-21T21:51:16.940046+0000","last_became_peered":"2024-05-21T21:51:16.940046+0000","last_unstale":"2024-05-21T21:55:36.552309+0000","last_undegraded":"2024-05-21T21:55:36.552309+0000","last_fullsized":"2024-05-21T21:55:36.552309+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:14:13.278799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.198999+0000","last_change":"2024-05-21T21:51:16.945599+0000","last_active":"2024-05-21T21:55:36.198999+0000","last_peered":"2024-05-21T21:55:36.198999+0000","last_clean":"2024-05-21T21:55:36.198999+0000","last_became_active":"2024-05-21T21:51:16.945196+0000","last_became_peered":"2024-05-21T21:51:16.945196+0000","last_unstale":"2024-05-21T21:55:36.198999+0000","last_undegraded":"2024-05-21T21:55:36.198999+0000","last_fullsized":"2024-05-21T21:55:36.198999+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:59:24.453615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"82'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.341504+0000","last_change":"2024-05-21T21:51:16.944895+0000","last_active":"2024-05-21T21:55:36.341504+0000","last_peered":"2024-05-21T21:55:36.341504+0000","last_clean":"2024-05-21T21:55:36.341504+0000","last_became_active":"2024-05-21T21:51:16.944560+0000","last_became_peered":"2024-05-21T21:51:16.944560+0000","last_unstale":"2024-05-21T21:55:36.341504+0000","last_undegraded":"2024-05-21T21:55:36.341504+0000","last_fullsized":"2024-05-21T21:55:36.341504+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:53:13.264701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"82'10","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.197858+0000","last_change":"2024-05-21T21:51:16.943861+0000","last_active":"2024-05-21T21:55:36.197858+0000","last_peered":"2024-05-21T21:55:36.197858+0000","last_clean":"2024-05-21T21:55:36.197858+0000","last_became_active":"2024-05-21T21:51:16.943123+0000","last_became_peered":"2024-05-21T21:51:16.943123+0000","last_unstale":"2024-05-21T21:55:36.197858+0000","last_undegraded":"2024-05-21T21:55:36.197858+0000","last_fullsized":"2024-05-21T21:55:36.197858+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:35:37.732855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"82'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.201422+0000","last_change":"2024-05-21T21:51:16.940580+0000","last_active":"2024-05-21T21:55:36.201422+0000","last_peered":"2024-05-21T21:55:36.201422+0000","last_clean":"2024-05-21T21:55:36.201422+0000","last_became_active":"2024-05-21T21:51:16.940264+0000","last_became_peered":"2024-05-21T21:51:16.940264+0000","last_unstale":"2024-05-21T21:55:36.201422+0000","last_undegraded":"2024-05-21T21:55:36.201422+0000","last_fullsized":"2024-05-21T21:55:36.201422+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:54:22.915541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'14","reported_seq":283,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.557627+0000","last_change":"2024-05-21T21:51:12.852935+0000","last_active":"2024-05-21T21:55:36.557627+0000","last_peered":"2024-05-21T21:55:36.557627+0000","last_clean":"2024-05-21T21:55:36.557627+0000","last_became_active":"2024-05-21T21:51:12.851802+0000","last_became_peered":"2024-05-21T21:51:12.851802+0000","last_unstale":"2024-05-21T21:55:36.557627+0000","last_undegraded":"2024-05-21T21:55:36.557627+0000","last_fullsized":"2024-05-21T21:55:36.557627+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:13:24.643408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":239,"num_read_kb":239,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"82'5","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.343135+0000","last_change":"2024-05-21T21:51:16.939461+0000","last_active":"2024-05-21T21:55:36.343135+0000","last_peered":"2024-05-21T21:55:36.343135+0000","last_clean":"2024-05-21T21:55:36.343135+0000","last_became_active":"2024-05-21T21:51:16.936322+0000","last_became_peered":"2024-05-21T21:51:16.936322+0000","last_unstale":"2024-05-21T21:55:36.343135+0000","last_undegraded":"2024-05-21T21:55:36.343135+0000","last_fullsized":"2024-05-21T21:55:36.343135+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:37:26.153963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.343791+0000","last_change":"2024-05-21T21:51:12.941838+0000","last_active":"2024-05-21T21:55:36.343791+0000","last_peered":"2024-05-21T21:55:36.343791+0000","last_clean":"2024-05-21T21:55:36.343791+0000","last_became_active":"2024-05-21T21:51:12.940818+0000","last_became_peered":"2024-05-21T21:51:12.940818+0000","last_unstale":"2024-05-21T21:55:36.343791+0000","last_undegraded":"2024-05-21T21:55:36.343791+0000","last_fullsized":"2024-05-21T21:55:36.343791+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:30:42.349241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"82'6","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.554128+0000","last_change":"2024-05-21T21:51:16.950252+0000","last_active":"2024-05-21T21:55:36.554128+0000","last_peered":"2024-05-21T21:55:36.554128+0000","last_clean":"2024-05-21T21:55:36.554128+0000","last_became_active":"2024-05-21T21:51:16.947375+0000","last_became_peered":"2024-05-21T21:51:16.947375+0000","last_unstale":"2024-05-21T21:55:36.554128+0000","last_undegraded":"2024-05-21T21:55:36.554128+0000","last_fullsized":"2024-05-21T21:55:36.554128+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:27:30.462799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"80'12","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.341317+0000","last_change":"2024-05-21T21:52:10.875739+0000","last_active":"2024-05-21T21:55:36.341317+0000","last_peered":"2024-05-21T21:55:36.341317+0000","last_clean":"2024-05-21T21:55:36.341317+0000","last_became_active":"2024-05-21T21:52:10.865846+0000","last_became_peered":"2024-05-21T21:52:10.865846+0000","last_unstale":"2024-05-21T21:55:36.341317+0000","last_undegraded":"2024-05-21T21:55:36.341317+0000","last_fullsized":"2024-05-21T21:55:36.341317+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T06:48:06.465757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.199107+0000","last_change":"2024-05-21T21:51:16.926380+0000","last_active":"2024-05-21T21:55:36.199107+0000","last_peered":"2024-05-21T21:55:36.199107+0000","last_clean":"2024-05-21T21:55:36.199107+0000","last_became_active":"2024-05-21T21:51:16.925821+0000","last_became_peered":"2024-05-21T21:51:16.925821+0000","last_unstale":"2024-05-21T21:55:36.199107+0000","last_undegraded":"2024-05-21T21:55:36.199107+0000","last_fullsized":"2024-05-21T21:55:36.199107+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:42:14.887155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'5","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.556613+0000","last_change":"2024-05-21T21:51:12.854342+0000","last_active":"2024-05-21T21:55:36.556613+0000","last_peered":"2024-05-21T21:55:36.556613+0000","last_clean":"2024-05-21T21:55:36.556613+0000","last_became_active":"2024-05-21T21:51:12.853908+0000","last_became_peered":"2024-05-21T21:51:12.853908+0000","last_unstale":"2024-05-21T21:55:36.556613+0000","last_undegraded":"2024-05-21T21:55:36.556613+0000","last_fullsized":"2024-05-21T21:55:36.556613+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:35:10.374749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"82'15","reported_seq":44,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.352781+0000","last_change":"2024-05-21T21:52:10.973816+0000","last_active":"2024-05-21T21:55:36.352781+0000","last_peered":"2024-05-21T21:55:36.352781+0000","last_clean":"2024-05-21T21:55:36.352781+0000","last_became_active":"2024-05-21T21:52:10.973457+0000","last_became_peered":"2024-05-21T21:52:10.973457+0000","last_unstale":"2024-05-21T21:55:36.352781+0000","last_undegraded":"2024-05-21T21:55:36.352781+0000","last_fullsized":"2024-05-21T21:55:36.352781+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T05:01:41.915158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"83'9","reported_seq":282,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.875854+0000","last_change":"2024-05-21T21:51:12.855658+0000","last_active":"2024-05-21T21:55:36.875854+0000","last_peered":"2024-05-21T21:55:36.875854+0000","last_clean":"2024-05-21T21:55:36.875854+0000","last_became_active":"2024-05-21T21:51:12.855281+0000","last_became_peered":"2024-05-21T21:51:12.855281+0000","last_unstale":"2024-05-21T21:55:36.875854+0000","last_undegraded":"2024-05-21T21:55:36.875854+0000","last_fullsized":"2024-05-21T21:55:36.875854+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:55:08.836045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1855,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":244,"num_read_kb":6,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"82'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.556310+0000","last_change":"2024-05-21T21:51:16.946977+0000","last_active":"2024-05-21T21:55:36.556310+0000","last_peered":"2024-05-21T21:55:36.556310+0000","last_clean":"2024-05-21T21:55:36.556310+0000","last_became_active":"2024-05-21T21:51:16.946562+0000","last_became_peered":"2024-05-21T21:51:16.946562+0000","last_unstale":"2024-05-21T21:55:36.556310+0000","last_undegraded":"2024-05-21T21:55:36.556310+0000","last_fullsized":"2024-05-21T21:55:36.556310+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:43:33.370265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"80'7","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.355936+0000","last_change":"2024-05-21T21:51:12.852574+0000","last_active":"2024-05-21T21:55:36.355936+0000","last_peered":"2024-05-21T21:55:36.355936+0000","last_clean":"2024-05-21T21:55:36.355936+0000","last_became_active":"2024-05-21T21:51:12.852258+0000","last_became_peered":"2024-05-21T21:51:12.852258+0000","last_unstale":"2024-05-21T21:55:36.355936+0000","last_undegraded":"2024-05-21T21:55:36.355936+0000","last_fullsized":"2024-05-21T21:55:36.355936+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:09:40.280821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"82'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.352655+0000","last_change":"2024-05-21T21:51:16.934768+0000","last_active":"2024-05-21T21:55:36.352655+0000","last_peered":"2024-05-21T21:55:36.352655+0000","last_clean":"2024-05-21T21:55:36.352655+0000","last_became_active":"2024-05-21T21:51:16.934275+0000","last_became_peered":"2024-05-21T21:51:16.934275+0000","last_unstale":"2024-05-21T21:55:36.352655+0000","last_undegraded":"2024-05-21T21:55:36.352655+0000","last_fullsized":"2024-05-21T21:55:36.352655+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:31:54.425069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"80'3","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.555219+0000","last_change":"2024-05-21T21:51:12.854166+0000","last_active":"2024-05-21T21:55:36.555219+0000","last_peered":"2024-05-21T21:55:36.555219+0000","last_clean":"2024-05-21T21:55:36.555219+0000","last_became_active":"2024-05-21T21:51:12.853508+0000","last_became_peered":"2024-05-21T21:51:12.853508+0000","last_unstale":"2024-05-21T21:55:36.555219+0000","last_undegraded":"2024-05-21T21:55:36.555219+0000","last_fullsized":"2024-05-21T21:55:36.555219+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:29:41.858655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"82'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.342248+0000","last_change":"2024-05-21T21:51:16.938293+0000","last_active":"2024-05-21T21:55:36.342248+0000","last_peered":"2024-05-21T21:55:36.342248+0000","last_clean":"2024-05-21T21:55:36.342248+0000","last_became_active":"2024-05-21T21:51:16.937970+0000","last_became_peered":"2024-05-21T21:51:16.937970+0000","last_unstale":"2024-05-21T21:55:36.342248+0000","last_undegraded":"2024-05-21T21:55:36.342248+0000","last_fullsized":"2024-05-21T21:55:36.342248+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:18:27.473098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'3","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.352476+0000","last_change":"2024-05-21T21:51:12.853366+0000","last_active":"2024-05-21T21:55:36.352476+0000","last_peered":"2024-05-21T21:55:36.352476+0000","last_clean":"2024-05-21T21:55:36.352476+0000","last_became_active":"2024-05-21T21:51:12.853028+0000","last_became_peered":"2024-05-21T21:51:12.853028+0000","last_unstale":"2024-05-21T21:55:36.352476+0000","last_undegraded":"2024-05-21T21:55:36.352476+0000","last_fullsized":"2024-05-21T21:55:36.352476+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T09:15:52.593028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"82'11","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.197181+0000","last_change":"2024-05-21T21:51:16.947903+0000","last_active":"2024-05-21T21:55:36.197181+0000","last_peered":"2024-05-21T21:55:36.197181+0000","last_clean":"2024-05-21T21:55:36.197181+0000","last_became_active":"2024-05-21T21:51:16.946677+0000","last_became_peered":"2024-05-21T21:51:16.946677+0000","last_unstale":"2024-05-21T21:55:36.197181+0000","last_undegraded":"2024-05-21T21:55:36.197181+0000","last_fullsized":"2024-05-21T21:55:36.197181+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T06:12:56.514284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'7","reported_seq":512,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.746589+0000","last_change":"2024-05-21T21:51:12.854873+0000","last_active":"2024-05-21T21:55:36.746589+0000","last_peered":"2024-05-21T21:55:36.746589+0000","last_clean":"2024-05-21T21:55:36.746589+0000","last_became_active":"2024-05-21T21:51:12.854393+0000","last_became_peered":"2024-05-21T21:51:12.854393+0000","last_unstale":"2024-05-21T21:55:36.746589+0000","last_undegraded":"2024-05-21T21:55:36.746589+0000","last_fullsized":"2024-05-21T21:55:36.746589+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:58:52.411164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":476,"num_read_kb":476,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"82'10","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.199587+0000","last_change":"2024-05-21T21:51:16.951094+0000","last_active":"2024-05-21T21:55:36.199587+0000","last_peered":"2024-05-21T21:55:36.199587+0000","last_clean":"2024-05-21T21:55:36.199587+0000","last_became_active":"2024-05-21T21:51:16.950724+0000","last_became_peered":"2024-05-21T21:51:16.950724+0000","last_unstale":"2024-05-21T21:55:36.199587+0000","last_undegraded":"2024-05-21T21:55:36.199587+0000","last_fullsized":"2024-05-21T21:55:36.199587+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:54:02.835426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'4","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.550904+0000","last_change":"2024-05-21T21:51:12.850427+0000","last_active":"2024-05-21T21:55:36.550904+0000","last_peered":"2024-05-21T21:55:36.550904+0000","last_clean":"2024-05-21T21:55:36.550904+0000","last_became_active":"2024-05-21T21:51:12.850096+0000","last_became_peered":"2024-05-21T21:51:12.850096+0000","last_unstale":"2024-05-21T21:55:36.550904+0000","last_undegraded":"2024-05-21T21:55:36.550904+0000","last_fullsized":"2024-05-21T21:55:36.550904+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:43:22.611043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.554216+0000","last_change":"2024-05-21T21:51:16.942330+0000","last_active":"2024-05-21T21:55:36.554216+0000","last_peered":"2024-05-21T21:55:36.554216+0000","last_clean":"2024-05-21T21:55:36.554216+0000","last_became_active":"2024-05-21T21:51:16.941718+0000","last_became_peered":"2024-05-21T21:51:16.941718+0000","last_unstale":"2024-05-21T21:55:36.554216+0000","last_undegraded":"2024-05-21T21:55:36.554216+0000","last_fullsized":"2024-05-21T21:55:36.554216+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:54:17.176411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'5","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.352907+0000","last_change":"2024-05-21T21:51:12.941100+0000","last_active":"2024-05-21T21:55:36.352907+0000","last_peered":"2024-05-21T21:55:36.352907+0000","last_clean":"2024-05-21T21:55:36.352907+0000","last_became_active":"2024-05-21T21:51:12.940597+0000","last_became_peered":"2024-05-21T21:51:12.940597+0000","last_unstale":"2024-05-21T21:55:36.352907+0000","last_undegraded":"2024-05-21T21:55:36.352907+0000","last_fullsized":"2024-05-21T21:55:36.352907+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:30:58.457108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"82'11","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.199324+0000","last_change":"2024-05-21T21:51:16.921973+0000","last_active":"2024-05-21T21:55:36.199324+0000","last_peered":"2024-05-21T21:55:36.199324+0000","last_clean":"2024-05-21T21:55:36.199324+0000","last_became_active":"2024-05-21T21:51:16.921597+0000","last_became_peered":"2024-05-21T21:51:16.921597+0000","last_unstale":"2024-05-21T21:55:36.199324+0000","last_undegraded":"2024-05-21T21:55:36.199324+0000","last_fullsized":"2024-05-21T21:55:36.199324+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:25:32.395940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'39","reported_seq":557,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.747043+0000","last_change":"2024-05-21T21:51:12.850312+0000","last_active":"2024-05-21T21:55:36.747043+0000","last_peered":"2024-05-21T21:55:36.747043+0000","last_clean":"2024-05-21T21:55:36.747043+0000","last_became_active":"2024-05-21T21:51:12.849837+0000","last_became_peered":"2024-05-21T21:51:12.849837+0000","last_unstale":"2024-05-21T21:55:36.747043+0000","last_undegraded":"2024-05-21T21:55:36.747043+0000","last_fullsized":"2024-05-21T21:55:36.747043+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T07:09:13.339554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66476,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":489,"num_read_kb":309,"num_write":41,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"82'6","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.556542+0000","last_change":"2024-05-21T21:51:16.926625+0000","last_active":"2024-05-21T21:55:36.556542+0000","last_peered":"2024-05-21T21:55:36.556542+0000","last_clean":"2024-05-21T21:55:36.556542+0000","last_became_active":"2024-05-21T21:51:16.925889+0000","last_became_peered":"2024-05-21T21:51:16.925889+0000","last_unstale":"2024-05-21T21:55:36.556542+0000","last_undegraded":"2024-05-21T21:55:36.556542+0000","last_fullsized":"2024-05-21T21:55:36.556542+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:35:23.464747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'9","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.201248+0000","last_change":"2024-05-21T21:51:12.859772+0000","last_active":"2024-05-21T21:55:36.201248+0000","last_peered":"2024-05-21T21:55:36.201248+0000","last_clean":"2024-05-21T21:55:36.201248+0000","last_became_active":"2024-05-21T21:51:12.859290+0000","last_became_peered":"2024-05-21T21:51:12.859290+0000","last_unstale":"2024-05-21T21:55:36.201248+0000","last_undegraded":"2024-05-21T21:55:36.201248+0000","last_fullsized":"2024-05-21T21:55:36.201248+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T05:28:29.953381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.555713+0000","last_change":"2024-05-21T21:51:16.925803+0000","last_active":"2024-05-21T21:55:36.555713+0000","last_peered":"2024-05-21T21:55:36.555713+0000","last_clean":"2024-05-21T21:55:36.555713+0000","last_became_active":"2024-05-21T21:51:16.925111+0000","last_became_peered":"2024-05-21T21:51:16.925111+0000","last_unstale":"2024-05-21T21:55:36.555713+0000","last_undegraded":"2024-05-21T21:55:36.555713+0000","last_fullsized":"2024-05-21T21:55:36.555713+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T06:22:11.353379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'5","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.352681+0000","last_change":"2024-05-21T21:51:12.859593+0000","last_active":"2024-05-21T21:55:36.352681+0000","last_peered":"2024-05-21T21:55:36.352681+0000","last_clean":"2024-05-21T21:55:36.352681+0000","last_became_active":"2024-05-21T21:51:12.859197+0000","last_became_peered":"2024-05-21T21:51:12.859197+0000","last_unstale":"2024-05-21T21:55:36.352681+0000","last_undegraded":"2024-05-21T21:55:36.352681+0000","last_fullsized":"2024-05-21T21:55:36.352681+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:13:12.311496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"82'15","reported_seq":44,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.551024+0000","last_change":"2024-05-21T21:52:10.879028+0000","last_active":"2024-05-21T21:55:36.551024+0000","last_peered":"2024-05-21T21:55:36.551024+0000","last_clean":"2024-05-21T21:55:36.551024+0000","last_became_active":"2024-05-21T21:52:10.878664+0000","last_became_peered":"2024-05-21T21:52:10.878664+0000","last_unstale":"2024-05-21T21:55:36.551024+0000","last_undegraded":"2024-05-21T21:55:36.551024+0000","last_fullsized":"2024-05-21T21:55:36.551024+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:05:53.953546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'7","reported_seq":278,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.723211+0000","last_change":"2024-05-21T21:51:12.852169+0000","last_active":"2024-05-21T21:55:36.723211+0000","last_peered":"2024-05-21T21:55:36.723211+0000","last_clean":"2024-05-21T21:55:36.723211+0000","last_became_active":"2024-05-21T21:51:12.851782+0000","last_became_peered":"2024-05-21T21:51:12.851782+0000","last_unstale":"2024-05-21T21:55:36.723211+0000","last_undegraded":"2024-05-21T21:55:36.723211+0000","last_fullsized":"2024-05-21T21:55:36.723211+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:46:09.692158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":243,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"82'6","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.550518+0000","last_change":"2024-05-21T21:51:16.929264+0000","last_active":"2024-05-21T21:55:36.550518+0000","last_peered":"2024-05-21T21:55:36.550518+0000","last_clean":"2024-05-21T21:55:36.550518+0000","last_became_active":"2024-05-21T21:51:16.928900+0000","last_became_peered":"2024-05-21T21:51:16.928900+0000","last_unstale":"2024-05-21T21:55:36.550518+0000","last_undegraded":"2024-05-21T21:55:36.550518+0000","last_fullsized":"2024-05-21T21:55:36.550518+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:42:28.040939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'17","reported_seq":71,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.556247+0000","last_change":"2024-05-21T21:52:10.984451+0000","last_active":"2024-05-21T21:55:36.556247+0000","last_peered":"2024-05-21T21:55:36.556247+0000","last_clean":"2024-05-21T21:55:36.556247+0000","last_became_active":"2024-05-21T21:52:10.974050+0000","last_became_peered":"2024-05-21T21:52:10.974050+0000","last_unstale":"2024-05-21T21:55:36.556247+0000","last_undegraded":"2024-05-21T21:55:36.556247+0000","last_fullsized":"2024-05-21T21:55:36.556247+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T07:08:18.307022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"82'6","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.352648+0000","last_change":"2024-05-21T21:51:16.935464+0000","last_active":"2024-05-21T21:55:36.352648+0000","last_peered":"2024-05-21T21:55:36.352648+0000","last_clean":"2024-05-21T21:55:36.352648+0000","last_became_active":"2024-05-21T21:51:16.935091+0000","last_became_peered":"2024-05-21T21:51:16.935091+0000","last_unstale":"2024-05-21T21:55:36.352648+0000","last_undegraded":"2024-05-21T21:55:36.352648+0000","last_fullsized":"2024-05-21T21:55:36.352648+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T06:59:16.330649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.556695+0000","last_change":"2024-05-21T21:51:12.853327+0000","last_active":"2024-05-21T21:55:36.556695+0000","last_peered":"2024-05-21T21:55:36.556695+0000","last_clean":"2024-05-21T21:55:36.556695+0000","last_became_active":"2024-05-21T21:51:12.852946+0000","last_became_peered":"2024-05-21T21:51:12.852946+0000","last_unstale":"2024-05-21T21:55:36.556695+0000","last_undegraded":"2024-05-21T21:55:36.556695+0000","last_fullsized":"2024-05-21T21:55:36.556695+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:00:34.256858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"82'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.341305+0000","last_change":"2024-05-21T21:51:16.929341+0000","last_active":"2024-05-21T21:55:36.341305+0000","last_peered":"2024-05-21T21:55:36.341305+0000","last_clean":"2024-05-21T21:55:36.341305+0000","last_became_active":"2024-05-21T21:51:16.929008+0000","last_became_peered":"2024-05-21T21:51:16.929008+0000","last_unstale":"2024-05-21T21:55:36.341305+0000","last_undegraded":"2024-05-21T21:55:36.341305+0000","last_fullsized":"2024-05-21T21:55:36.341305+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:26:26.297136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"80'1","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.197640+0000","last_change":"2024-05-21T21:51:12.945433+0000","last_active":"2024-05-21T21:55:36.197640+0000","last_peered":"2024-05-21T21:55:36.197640+0000","last_clean":"2024-05-21T21:55:36.197640+0000","last_became_active":"2024-05-21T21:51:12.944463+0000","last_became_peered":"2024-05-21T21:51:12.944463+0000","last_unstale":"2024-05-21T21:55:36.197640+0000","last_undegraded":"2024-05-21T21:55:36.197640+0000","last_fullsized":"2024-05-21T21:55:36.197640+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:29:52.014583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'5","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.356192+0000","last_change":"2024-05-21T21:52:10.883554+0000","last_active":"2024-05-21T21:55:36.356192+0000","last_peered":"2024-05-21T21:55:36.356192+0000","last_clean":"2024-05-21T21:55:36.356192+0000","last_became_active":"2024-05-21T21:52:10.877789+0000","last_became_peered":"2024-05-21T21:52:10.877789+0000","last_unstale":"2024-05-21T21:55:36.356192+0000","last_undegraded":"2024-05-21T21:55:36.356192+0000","last_fullsized":"2024-05-21T21:55:36.356192+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:52:37.623551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"83'6","reported_seq":276,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.890796+0000","last_change":"2024-05-21T21:51:12.852883+0000","last_active":"2024-05-21T21:55:36.890796+0000","last_peered":"2024-05-21T21:55:36.890796+0000","last_clean":"2024-05-21T21:55:36.890796+0000","last_became_active":"2024-05-21T21:51:12.852549+0000","last_became_peered":"2024-05-21T21:51:12.852549+0000","last_unstale":"2024-05-21T21:55:36.890796+0000","last_undegraded":"2024-05-21T21:55:36.890796+0000","last_fullsized":"2024-05-21T21:55:36.890796+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:05:53.002286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":240,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"82'7","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.356089+0000","last_change":"2024-05-21T21:51:16.944425+0000","last_active":"2024-05-21T21:55:36.356089+0000","last_peered":"2024-05-21T21:55:36.356089+0000","last_clean":"2024-05-21T21:55:36.356089+0000","last_became_active":"2024-05-21T21:51:16.944033+0000","last_became_peered":"2024-05-21T21:51:16.944033+0000","last_unstale":"2024-05-21T21:55:36.356089+0000","last_undegraded":"2024-05-21T21:55:36.356089+0000","last_fullsized":"2024-05-21T21:55:36.356089+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T07:16:26.802329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"82'8","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.352742+0000","last_change":"2024-05-21T21:51:16.933264+0000","last_active":"2024-05-21T21:55:36.352742+0000","last_peered":"2024-05-21T21:55:36.352742+0000","last_clean":"2024-05-21T21:55:36.352742+0000","last_became_active":"2024-05-21T21:51:16.932911+0000","last_became_peered":"2024-05-21T21:51:16.932911+0000","last_unstale":"2024-05-21T21:55:36.352742+0000","last_undegraded":"2024-05-21T21:55:36.352742+0000","last_fullsized":"2024-05-21T21:55:36.352742+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:25:54.006367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"82'6","reported_seq":275,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.555395+0000","last_change":"2024-05-21T21:51:12.942900+0000","last_active":"2024-05-21T21:55:36.555395+0000","last_peered":"2024-05-21T21:55:36.555395+0000","last_clean":"2024-05-21T21:55:36.555395+0000","last_became_active":"2024-05-21T21:51:12.941038+0000","last_became_peered":"2024-05-21T21:51:12.941038+0000","last_unstale":"2024-05-21T21:55:36.555395+0000","last_undegraded":"2024-05-21T21:55:36.555395+0000","last_fullsized":"2024-05-21T21:55:36.555395+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:38:43.240011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":239,"num_read_kb":239,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"82'8","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.342906+0000","last_change":"2024-05-21T21:51:16.933391+0000","last_active":"2024-05-21T21:55:36.342906+0000","last_peered":"2024-05-21T21:55:36.342906+0000","last_clean":"2024-05-21T21:55:36.342906+0000","last_became_active":"2024-05-21T21:51:16.932320+0000","last_became_peered":"2024-05-21T21:51:16.932320+0000","last_unstale":"2024-05-21T21:55:36.342906+0000","last_undegraded":"2024-05-21T21:55:36.342906+0000","last_fullsized":"2024-05-21T21:55:36.342906+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T09:06:59.126748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'6","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.723215+0000","last_change":"2024-05-21T21:51:12.856342+0000","last_active":"2024-05-21T21:55:36.723215+0000","last_peered":"2024-05-21T21:55:36.723215+0000","last_clean":"2024-05-21T21:55:36.723215+0000","last_became_active":"2024-05-21T21:51:12.855952+0000","last_became_peered":"2024-05-21T21:51:12.855952+0000","last_unstale":"2024-05-21T21:55:36.723215+0000","last_undegraded":"2024-05-21T21:55:36.723215+0000","last_fullsized":"2024-05-21T21:55:36.723215+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:34:40.734034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":103,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.550605+0000","last_change":"2024-05-21T21:49:42.797187+0000","last_active":"2024-05-21T21:55:36.550605+0000","last_peered":"2024-05-21T21:55:36.550605+0000","last_clean":"2024-05-21T21:55:36.550605+0000","last_became_active":"2024-05-21T21:49:42.791291+0000","last_became_peered":"2024-05-21T21:49:42.791291+0000","last_unstale":"2024-05-21T21:55:36.550605+0000","last_undegraded":"2024-05-21T21:55:36.550605+0000","last_fullsized":"2024-05-21T21:55:36.550605+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:45:55.552293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:45:55.552293+0000","last_clean_scrub_stamp":"2024-05-21T21:45:55.552293+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T07:24:18.302523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"82'5","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.555133+0000","last_change":"2024-05-21T21:51:16.926446+0000","last_active":"2024-05-21T21:55:36.555133+0000","last_peered":"2024-05-21T21:55:36.555133+0000","last_clean":"2024-05-21T21:55:36.555133+0000","last_became_active":"2024-05-21T21:51:16.926070+0000","last_became_peered":"2024-05-21T21:51:16.926070+0000","last_unstale":"2024-05-21T21:55:36.555133+0000","last_undegraded":"2024-05-21T21:55:36.555133+0000","last_fullsized":"2024-05-21T21:55:36.555133+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:44:27.366393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'4","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.550735+0000","last_change":"2024-05-21T21:51:12.854371+0000","last_active":"2024-05-21T21:55:36.550735+0000","last_peered":"2024-05-21T21:55:36.550735+0000","last_clean":"2024-05-21T21:55:36.550735+0000","last_became_active":"2024-05-21T21:51:12.853847+0000","last_became_peered":"2024-05-21T21:51:12.853847+0000","last_unstale":"2024-05-21T21:55:36.550735+0000","last_undegraded":"2024-05-21T21:55:36.550735+0000","last_fullsized":"2024-05-21T21:55:36.550735+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:14:10.549901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"80'4","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.556551+0000","last_change":"2024-05-21T21:51:12.941227+0000","last_active":"2024-05-21T21:55:36.556551+0000","last_peered":"2024-05-21T21:55:36.556551+0000","last_clean":"2024-05-21T21:55:36.556551+0000","last_became_active":"2024-05-21T21:51:12.940702+0000","last_became_peered":"2024-05-21T21:51:12.940702+0000","last_unstale":"2024-05-21T21:55:36.556551+0000","last_undegraded":"2024-05-21T21:55:36.556551+0000","last_fullsized":"2024-05-21T21:55:36.556551+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:40:34.124921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"82'9","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.556301+0000","last_change":"2024-05-21T21:51:16.945627+0000","last_active":"2024-05-21T21:55:36.556301+0000","last_peered":"2024-05-21T21:55:36.556301+0000","last_clean":"2024-05-21T21:55:36.556301+0000","last_became_active":"2024-05-21T21:51:16.945297+0000","last_became_peered":"2024-05-21T21:51:16.945297+0000","last_unstale":"2024-05-21T21:55:36.556301+0000","last_undegraded":"2024-05-21T21:55:36.556301+0000","last_fullsized":"2024-05-21T21:55:36.556301+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:54:38.898774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'51","reported_seq":120,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.502619+0000","last_change":"2024-05-21T21:52:10.888076+0000","last_active":"2024-05-21T21:55:36.502619+0000","last_peered":"2024-05-21T21:55:36.502619+0000","last_clean":"2024-05-21T21:55:36.502619+0000","last_became_active":"2024-05-21T21:52:10.879368+0000","last_became_peered":"2024-05-21T21:52:10.879368+0000","last_unstale":"2024-05-21T21:55:36.502619+0000","last_undegraded":"2024-05-21T21:55:36.502619+0000","last_fullsized":"2024-05-21T21:55:36.502619+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:46:43.190411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":91,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"82'5","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.344028+0000","last_change":"2024-05-21T21:51:16.929807+0000","last_active":"2024-05-21T21:55:36.344028+0000","last_peered":"2024-05-21T21:55:36.344028+0000","last_clean":"2024-05-21T21:55:36.344028+0000","last_became_active":"2024-05-21T21:51:16.928859+0000","last_became_peered":"2024-05-21T21:51:16.928859+0000","last_unstale":"2024-05-21T21:55:36.344028+0000","last_undegraded":"2024-05-21T21:55:36.344028+0000","last_fullsized":"2024-05-21T21:55:36.344028+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:41:32.830892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'7","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.723447+0000","last_change":"2024-05-21T21:51:12.850843+0000","last_active":"2024-05-21T21:55:36.723447+0000","last_peered":"2024-05-21T21:55:36.723447+0000","last_clean":"2024-05-21T21:55:36.723447+0000","last_became_active":"2024-05-21T21:51:12.850516+0000","last_became_peered":"2024-05-21T21:51:12.850516+0000","last_unstale":"2024-05-21T21:55:36.723447+0000","last_undegraded":"2024-05-21T21:55:36.723447+0000","last_fullsized":"2024-05-21T21:55:36.723447+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:37:05.191626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"82'7","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.553781+0000","last_change":"2024-05-21T21:51:16.949017+0000","last_active":"2024-05-21T21:55:36.553781+0000","last_peered":"2024-05-21T21:55:36.553781+0000","last_clean":"2024-05-21T21:55:36.553781+0000","last_became_active":"2024-05-21T21:51:16.948691+0000","last_became_peered":"2024-05-21T21:51:16.948691+0000","last_unstale":"2024-05-21T21:55:36.553781+0000","last_undegraded":"2024-05-21T21:55:36.553781+0000","last_fullsized":"2024-05-21T21:55:36.553781+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:32:06.606581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.199191+0000","last_change":"2024-05-21T21:51:12.939203+0000","last_active":"2024-05-21T21:55:36.199191+0000","last_peered":"2024-05-21T21:55:36.199191+0000","last_clean":"2024-05-21T21:55:36.199191+0000","last_became_active":"2024-05-21T21:51:12.938936+0000","last_became_peered":"2024-05-21T21:51:12.938936+0000","last_unstale":"2024-05-21T21:55:36.199191+0000","last_undegraded":"2024-05-21T21:55:36.199191+0000","last_fullsized":"2024-05-21T21:55:36.199191+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:55:50.336362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"82'2","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.556315+0000","last_change":"2024-05-21T21:51:16.926216+0000","last_active":"2024-05-21T21:55:36.556315+0000","last_peered":"2024-05-21T21:55:36.556315+0000","last_clean":"2024-05-21T21:55:36.556315+0000","last_became_active":"2024-05-21T21:51:16.925834+0000","last_became_peered":"2024-05-21T21:51:16.925834+0000","last_unstale":"2024-05-21T21:55:36.556315+0000","last_undegraded":"2024-05-21T21:55:36.556315+0000","last_fullsized":"2024-05-21T21:55:36.556315+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:13:30.192908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'7","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.342741+0000","last_change":"2024-05-21T21:51:12.855306+0000","last_active":"2024-05-21T21:55:36.342741+0000","last_peered":"2024-05-21T21:55:36.342741+0000","last_clean":"2024-05-21T21:55:36.342741+0000","last_became_active":"2024-05-21T21:51:12.854722+0000","last_became_peered":"2024-05-21T21:51:12.854722+0000","last_unstale":"2024-05-21T21:55:36.342741+0000","last_undegraded":"2024-05-21T21:55:36.342741+0000","last_fullsized":"2024-05-21T21:55:36.342741+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:39:41.583219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"82'9","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.341026+0000","last_change":"2024-05-21T21:51:16.944758+0000","last_active":"2024-05-21T21:55:36.341026+0000","last_peered":"2024-05-21T21:55:36.341026+0000","last_clean":"2024-05-21T21:55:36.341026+0000","last_became_active":"2024-05-21T21:51:16.944393+0000","last_became_peered":"2024-05-21T21:51:16.944393+0000","last_unstale":"2024-05-21T21:55:36.341026+0000","last_undegraded":"2024-05-21T21:55:36.341026+0000","last_fullsized":"2024-05-21T21:55:36.341026+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:01:23.048355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'3","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.201175+0000","last_change":"2024-05-21T21:51:12.860378+0000","last_active":"2024-05-21T21:55:36.201175+0000","last_peered":"2024-05-21T21:55:36.201175+0000","last_clean":"2024-05-21T21:55:36.201175+0000","last_became_active":"2024-05-21T21:51:12.859978+0000","last_became_peered":"2024-05-21T21:51:12.859978+0000","last_unstale":"2024-05-21T21:55:36.201175+0000","last_undegraded":"2024-05-21T21:55:36.201175+0000","last_fullsized":"2024-05-21T21:55:36.201175+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T07:52:26.902232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"82'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.343347+0000","last_change":"2024-05-21T21:51:16.943194+0000","last_active":"2024-05-21T21:55:36.343347+0000","last_peered":"2024-05-21T21:55:36.343347+0000","last_clean":"2024-05-21T21:55:36.343347+0000","last_became_active":"2024-05-21T21:51:16.942884+0000","last_became_peered":"2024-05-21T21:51:16.942884+0000","last_unstale":"2024-05-21T21:55:36.343347+0000","last_undegraded":"2024-05-21T21:55:36.343347+0000","last_fullsized":"2024-05-21T21:55:36.343347+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T05:58:17.436995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.556452+0000","last_change":"2024-05-21T21:51:12.853390+0000","last_active":"2024-05-21T21:55:36.556452+0000","last_peered":"2024-05-21T21:55:36.556452+0000","last_clean":"2024-05-21T21:55:36.556452+0000","last_became_active":"2024-05-21T21:51:12.852990+0000","last_became_peered":"2024-05-21T21:51:12.852990+0000","last_unstale":"2024-05-21T21:55:36.556452+0000","last_undegraded":"2024-05-21T21:55:36.556452+0000","last_fullsized":"2024-05-21T21:55:36.556452+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:32:16.392609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"82'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.550802+0000","last_change":"2024-05-21T21:51:16.933430+0000","last_active":"2024-05-21T21:55:36.550802+0000","last_peered":"2024-05-21T21:55:36.550802+0000","last_clean":"2024-05-21T21:55:36.550802+0000","last_became_active":"2024-05-21T21:51:16.933053+0000","last_became_peered":"2024-05-21T21:51:16.933053+0000","last_unstale":"2024-05-21T21:55:36.550802+0000","last_undegraded":"2024-05-21T21:55:36.550802+0000","last_fullsized":"2024-05-21T21:55:36.550802+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T06:14:57.468727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'7","reported_seq":277,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.747135+0000","last_change":"2024-05-21T21:51:12.946113+0000","last_active":"2024-05-21T21:55:36.747135+0000","last_peered":"2024-05-21T21:55:36.747135+0000","last_clean":"2024-05-21T21:55:36.747135+0000","last_became_active":"2024-05-21T21:51:12.945650+0000","last_became_peered":"2024-05-21T21:51:12.945650+0000","last_unstale":"2024-05-21T21:55:36.747135+0000","last_undegraded":"2024-05-21T21:55:36.747135+0000","last_fullsized":"2024-05-21T21:55:36.747135+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T09:40:18.209400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1952,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":241,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"82'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.550873+0000","last_change":"2024-05-21T21:51:16.932632+0000","last_active":"2024-05-21T21:55:36.550873+0000","last_peered":"2024-05-21T21:55:36.550873+0000","last_clean":"2024-05-21T21:55:36.550873+0000","last_became_active":"2024-05-21T21:51:16.932256+0000","last_became_peered":"2024-05-21T21:51:16.932256+0000","last_unstale":"2024-05-21T21:55:36.550873+0000","last_undegraded":"2024-05-21T21:55:36.550873+0000","last_fullsized":"2024-05-21T21:55:36.550873+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:04:48.146485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'41","reported_seq":78,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.496010+0000","last_change":"2024-05-21T21:51:12.860602+0000","last_active":"2024-05-21T21:55:36.496010+0000","last_peered":"2024-05-21T21:55:36.496010+0000","last_clean":"2024-05-21T21:55:36.496010+0000","last_became_active":"2024-05-21T21:51:12.860228+0000","last_became_peered":"2024-05-21T21:51:12.860228+0000","last_unstale":"2024-05-21T21:55:36.496010+0000","last_undegraded":"2024-05-21T21:55:36.496010+0000","last_fullsized":"2024-05-21T21:55:36.496010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:33:39.230872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.551036+0000","last_change":"2024-05-21T21:51:16.929616+0000","last_active":"2024-05-21T21:55:36.551036+0000","last_peered":"2024-05-21T21:55:36.551036+0000","last_clean":"2024-05-21T21:55:36.551036+0000","last_became_active":"2024-05-21T21:51:16.928569+0000","last_became_peered":"2024-05-21T21:51:16.928569+0000","last_unstale":"2024-05-21T21:55:36.551036+0000","last_undegraded":"2024-05-21T21:55:36.551036+0000","last_fullsized":"2024-05-21T21:55:36.551036+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:15:41.790344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'5","reported_seq":46,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.555499+0000","last_change":"2024-05-21T21:52:10.881114+0000","last_active":"2024-05-21T21:55:36.555499+0000","last_peered":"2024-05-21T21:55:36.555499+0000","last_clean":"2024-05-21T21:55:36.555499+0000","last_became_active":"2024-05-21T21:52:10.872022+0000","last_became_peered":"2024-05-21T21:52:10.872022+0000","last_unstale":"2024-05-21T21:55:36.555499+0000","last_undegraded":"2024-05-21T21:55:36.555499+0000","last_fullsized":"2024-05-21T21:55:36.555499+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T06:05:51.419156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.550881+0000","last_change":"2024-05-21T21:51:16.926496+0000","last_active":"2024-05-21T21:55:36.550881+0000","last_peered":"2024-05-21T21:55:36.550881+0000","last_clean":"2024-05-21T21:55:36.550881+0000","last_became_active":"2024-05-21T21:51:16.926112+0000","last_became_peered":"2024-05-21T21:51:16.926112+0000","last_unstale":"2024-05-21T21:55:36.550881+0000","last_undegraded":"2024-05-21T21:55:36.550881+0000","last_fullsized":"2024-05-21T21:55:36.550881+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:26:08.254421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"82'9","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.341047+0000","last_change":"2024-05-21T21:51:16.930522+0000","last_active":"2024-05-21T21:55:36.341047+0000","last_peered":"2024-05-21T21:55:36.341047+0000","last_clean":"2024-05-21T21:55:36.341047+0000","last_became_active":"2024-05-21T21:51:16.929853+0000","last_became_peered":"2024-05-21T21:51:16.929853+0000","last_unstale":"2024-05-21T21:55:36.341047+0000","last_undegraded":"2024-05-21T21:55:36.341047+0000","last_fullsized":"2024-05-21T21:55:36.341047+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:10:57.940009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"82'12","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.201162+0000","last_change":"2024-05-21T21:51:16.931751+0000","last_active":"2024-05-21T21:55:36.201162+0000","last_peered":"2024-05-21T21:55:36.201162+0000","last_clean":"2024-05-21T21:55:36.201162+0000","last_became_active":"2024-05-21T21:51:16.931353+0000","last_became_peered":"2024-05-21T21:51:16.931353+0000","last_unstale":"2024-05-21T21:55:36.201162+0000","last_undegraded":"2024-05-21T21:55:36.201162+0000","last_fullsized":"2024-05-21T21:55:36.201162+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:33:45.297850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"82'8","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.201127+0000","last_change":"2024-05-21T21:51:16.940096+0000","last_active":"2024-05-21T21:55:36.201127+0000","last_peered":"2024-05-21T21:55:36.201127+0000","last_clean":"2024-05-21T21:55:36.201127+0000","last_became_active":"2024-05-21T21:51:16.939740+0000","last_became_peered":"2024-05-21T21:51:16.939740+0000","last_unstale":"2024-05-21T21:55:36.201127+0000","last_undegraded":"2024-05-21T21:55:36.201127+0000","last_fullsized":"2024-05-21T21:55:36.201127+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:59:52.569526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"82'7","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.552155+0000","last_change":"2024-05-21T21:51:16.946488+0000","last_active":"2024-05-21T21:55:36.552155+0000","last_peered":"2024-05-21T21:55:36.552155+0000","last_clean":"2024-05-21T21:55:36.552155+0000","last_became_active":"2024-05-21T21:51:16.945957+0000","last_became_peered":"2024-05-21T21:51:16.945957+0000","last_unstale":"2024-05-21T21:55:36.552155+0000","last_undegraded":"2024-05-21T21:55:36.552155+0000","last_fullsized":"2024-05-21T21:55:36.552155+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:40:36.708344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"82'4","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.556177+0000","last_change":"2024-05-21T21:51:16.949349+0000","last_active":"2024-05-21T21:55:36.556177+0000","last_peered":"2024-05-21T21:55:36.556177+0000","last_clean":"2024-05-21T21:55:36.556177+0000","last_became_active":"2024-05-21T21:51:16.946986+0000","last_became_peered":"2024-05-21T21:51:16.946986+0000","last_unstale":"2024-05-21T21:55:36.556177+0000","last_undegraded":"2024-05-21T21:55:36.556177+0000","last_fullsized":"2024-05-21T21:55:36.556177+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:01:14.877110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"82'6","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.341365+0000","last_change":"2024-05-21T21:51:16.929172+0000","last_active":"2024-05-21T21:55:36.341365+0000","last_peered":"2024-05-21T21:55:36.341365+0000","last_clean":"2024-05-21T21:55:36.341365+0000","last_became_active":"2024-05-21T21:51:16.928823+0000","last_became_peered":"2024-05-21T21:51:16.928823+0000","last_unstale":"2024-05-21T21:55:36.341365+0000","last_undegraded":"2024-05-21T21:55:36.341365+0000","last_fullsized":"2024-05-21T21:55:36.341365+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:09:13.719272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"82'15","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.201467+0000","last_change":"2024-05-21T21:51:16.930332+0000","last_active":"2024-05-21T21:55:36.201467+0000","last_peered":"2024-05-21T21:55:36.201467+0000","last_clean":"2024-05-21T21:55:36.201467+0000","last_became_active":"2024-05-21T21:51:16.930024+0000","last_became_peered":"2024-05-21T21:51:16.930024+0000","last_unstale":"2024-05-21T21:55:36.201467+0000","last_undegraded":"2024-05-21T21:55:36.201467+0000","last_fullsized":"2024-05-21T21:55:36.201467+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:47:41.272725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"82'2","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.343903+0000","last_change":"2024-05-21T21:51:16.930102+0000","last_active":"2024-05-21T21:55:36.343903+0000","last_peered":"2024-05-21T21:55:36.343903+0000","last_clean":"2024-05-21T21:55:36.343903+0000","last_became_active":"2024-05-21T21:51:16.929410+0000","last_became_peered":"2024-05-21T21:51:16.929410+0000","last_unstale":"2024-05-21T21:55:36.343903+0000","last_undegraded":"2024-05-21T21:55:36.343903+0000","last_fullsized":"2024-05-21T21:55:36.343903+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:11:46.296288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"82'6","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.556235+0000","last_change":"2024-05-21T21:52:10.873424+0000","last_active":"2024-05-21T21:55:36.556235+0000","last_peered":"2024-05-21T21:55:36.556235+0000","last_clean":"2024-05-21T21:55:36.556235+0000","last_became_active":"2024-05-21T21:52:10.873118+0000","last_became_peered":"2024-05-21T21:52:10.873118+0000","last_unstale":"2024-05-21T21:55:36.556235+0000","last_undegraded":"2024-05-21T21:55:36.556235+0000","last_fullsized":"2024-05-21T21:55:36.556235+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:05:00.731013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,3],"acting":[8,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.29","version":"82'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.342603+0000","last_change":"2024-05-21T21:51:16.934442+0000","last_active":"2024-05-21T21:55:36.342603+0000","last_peered":"2024-05-21T21:55:36.342603+0000","last_clean":"2024-05-21T21:55:36.342603+0000","last_became_active":"2024-05-21T21:51:16.934086+0000","last_became_peered":"2024-05-21T21:51:16.934086+0000","last_unstale":"2024-05-21T21:55:36.342603+0000","last_undegraded":"2024-05-21T21:55:36.342603+0000","last_fullsized":"2024-05-21T21:55:36.342603+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:36:44.302532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"82'10","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.199405+0000","last_change":"2024-05-21T21:51:16.924479+0000","last_active":"2024-05-21T21:55:36.199405+0000","last_peered":"2024-05-21T21:55:36.199405+0000","last_clean":"2024-05-21T21:55:36.199405+0000","last_became_active":"2024-05-21T21:51:16.923998+0000","last_became_peered":"2024-05-21T21:51:16.923998+0000","last_unstale":"2024-05-21T21:55:36.199405+0000","last_undegraded":"2024-05-21T21:55:36.199405+0000","last_fullsized":"2024-05-21T21:55:36.199405+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:23:07.931731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"82'11","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.555801+0000","last_change":"2024-05-21T21:51:16.947271+0000","last_active":"2024-05-21T21:55:36.555801+0000","last_peered":"2024-05-21T21:55:36.555801+0000","last_clean":"2024-05-21T21:55:36.555801+0000","last_became_active":"2024-05-21T21:51:16.946912+0000","last_became_peered":"2024-05-21T21:51:16.946912+0000","last_unstale":"2024-05-21T21:55:36.555801+0000","last_undegraded":"2024-05-21T21:55:36.555801+0000","last_fullsized":"2024-05-21T21:55:36.555801+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:51:29.646915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"82'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.341021+0000","last_change":"2024-05-21T21:51:16.928852+0000","last_active":"2024-05-21T21:55:36.341021+0000","last_peered":"2024-05-21T21:55:36.341021+0000","last_clean":"2024-05-21T21:55:36.341021+0000","last_became_active":"2024-05-21T21:51:16.928480+0000","last_became_peered":"2024-05-21T21:51:16.928480+0000","last_unstale":"2024-05-21T21:55:36.341021+0000","last_undegraded":"2024-05-21T21:55:36.341021+0000","last_fullsized":"2024-05-21T21:55:36.341021+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T09:12:30.691254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":600652,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":2521,"num_read_kb":1376,"num_write":377,"num_write_kb":1467,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":112,"num_keys_recovered":17,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2174976,"data_stored":1801956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":304,"ondisk_log_size":304,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645381,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":901120,"data_stored":433959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841609,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28736,"kb_used_data":1720,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716416,"statfs":{"total":95995035648,"available":95965609984,"internally_reserved":0,"allocated":1761280,"data_stored":1232552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005134,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":946176,"data_stored":442962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201365,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":921600,"data_stored":434267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397593,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28500,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716652,"statfs":{"total":95995035648,"available":95965851648,"internally_reserved":0,"allocated":1519616,"data_stored":1027768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561119,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":925696,"data_stored":442953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574971,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1003520,"data_stored":510454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738496,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36264,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708888,"statfs":{"total":95995035648,"available":95957901312,"internally_reserved":0,"allocated":946176,"data_stored":444683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411447,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36324,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708828,"statfs":{"total":95995035648,"available":95957839872,"internally_reserved":0,"allocated":1007616,"data_stored":512074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149807,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36236,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708916,"statfs":{"total":95995035648,"available":95957929984,"internally_reserved":0,"allocated":921600,"data_stored":433314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953578,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":925696,"data_stored":442955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757349,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32672,"kb_used_data":1496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712480,"statfs":{"total":95995035648,"available":95961579520,"internally_reserved":0,"allocated":1531904,"data_stored":1026914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":68731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":70351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":557056,"data_stored":525246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":135653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":258048,"data_stored":200344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":395190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-21T21:55:42.099 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-21T21:55:42.320 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.319+0000 7fa49d800640 1 -- 172.21.15.151:0/1322199687 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa498108e80 msgr2=0x7fa4981092a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:42.320 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.319+0000 7fa49d800640 1 --2- 172.21.15.151:0/1322199687 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa498108e80 0x7fa4981092a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa488009900 tx=0x7fa48802a3f0 comp rx=0 tx=0).stop 2024-05-21T21:55:42.320 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.319+0000 7fa49d800640 1 -- 172.21.15.151:0/1322199687 shutdown_connections 2024-05-21T21:55:42.321 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.319+0000 7fa49d800640 1 --2- 172.21.15.151:0/1322199687 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa49810aa20 0x7fa498111340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:42.321 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.319+0000 7fa49d800640 1 --2- 172.21.15.151:0/1322199687 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa49810a060 0x7fa49810a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:42.321 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.319+0000 7fa49d800640 1 --2- 172.21.15.151:0/1322199687 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa498108e80 0x7fa4981092a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:42.321 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.319+0000 7fa49d800640 1 -- 172.21.15.151:0/1322199687 >> 172.21.15.151:0/1322199687 conn(0x7fa498077e90 msgr2=0x7fa49807a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:55:42.321 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.319+0000 7fa49d800640 1 -- 172.21.15.151:0/1322199687 shutdown_connections 2024-05-21T21:55:42.321 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.319+0000 7fa49d800640 1 -- 172.21.15.151:0/1322199687 wait complete. 2024-05-21T21:55:42.324 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.320+0000 7fa49d800640 1 Processor -- start 2024-05-21T21:55:42.324 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.320+0000 7fa49d800640 1 -- start start 2024-05-21T21:55:42.324 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.320+0000 7fa49d800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa498108e80 0x7fa49819f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:42.324 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.320+0000 7fa49d800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa49810a060 0x7fa49819f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:42.324 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.320+0000 7fa49d800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa49810aa20 0x7fa4981a66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:42.324 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.320+0000 7fa49d800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa498114a00 con 0x7fa498108e80 2024-05-21T21:55:42.324 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.320+0000 7fa49d800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa498114880 con 0x7fa49810a060 2024-05-21T21:55:42.324 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.320+0000 7fa49d800640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa498114b80 con 0x7fa49810aa20 2024-05-21T21:55:42.324 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.321+0000 7fa497e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa498108e80 0x7fa49819f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:42.324 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.321+0000 7fa49ce00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa49810aa20 0x7fa4981a66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:42.324 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.321+0000 7fa497e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa498108e80 0x7fa49819f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:40782/0 (socket says 172.21.15.151:40782) 2024-05-21T21:55:42.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.321+0000 7fa49ce00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa49810aa20 0x7fa4981a66f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:43278/0 (socket says 172.21.15.151:43278) 2024-05-21T21:55:42.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.321+0000 7fa497e00640 1 -- 172.21.15.151:0/2603973885 learned_addr learned my addr 172.21.15.151:0/2603973885 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:55:42.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.321+0000 7fa497400640 1 --2- 172.21.15.151:0/2603973885 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa49810a060 0x7fa49819f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:42.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.321+0000 7fa49ce00640 1 -- 172.21.15.151:0/2603973885 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa49810a060 msgr2=0x7fa49819f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:42.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.321+0000 7fa49ce00640 1 --2- 172.21.15.151:0/2603973885 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa49810a060 0x7fa49819f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:42.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.321+0000 7fa49ce00640 1 -- 172.21.15.151:0/2603973885 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa498108e80 msgr2=0x7fa49819f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:42.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.321+0000 7fa49ce00640 1 --2- 172.21.15.151:0/2603973885 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa498108e80 0x7fa49819f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:42.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.321+0000 7fa49ce00640 1 -- 172.21.15.151:0/2603973885 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4981a6d60 con 0x7fa49810aa20 2024-05-21T21:55:42.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.321+0000 7fa49ce00640 1 --2- 172.21.15.151:0/2603973885 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa49810aa20 0x7fa4981a66f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa490009fd0 tx=0x7fa49000ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:55:42.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.322+0000 7fa494c00640 1 -- 172.21.15.151:0/2603973885 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa49000ce20 con 0x7fa49810aa20 2024-05-21T21:55:42.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.322+0000 7fa494c00640 1 -- 172.21.15.151:0/2603973885 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa490005020 con 0x7fa49810aa20 2024-05-21T21:55:42.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.322+0000 7fa49d800640 1 -- 172.21.15.151:0/2603973885 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4981a6ff0 con 0x7fa49810aa20 2024-05-21T21:55:42.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.322+0000 7fa494c00640 1 -- 172.21.15.151:0/2603973885 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa490010690 con 0x7fa49810aa20 2024-05-21T21:55:42.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.322+0000 7fa49d800640 1 -- 172.21.15.151:0/2603973885 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4981a74c0 con 0x7fa49810aa20 2024-05-21T21:55:42.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.323+0000 7fa49d800640 1 -- 172.21.15.151:0/2603973885 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa498110390 con 0x7fa49810aa20 2024-05-21T21:55:42.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.326+0000 7fa494c00640 1 -- 172.21.15.151:0/2603973885 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 25) ==== 149829+0+0 (secure 0 0 0) 0x7fa4900054d0 con 0x7fa49810aa20 2024-05-21T21:55:42.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.327+0000 7fa494c00640 1 --2- 172.21.15.151:0/2603973885 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fa47c0b29f0 0x7fa47c0b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:42.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.327+0000 7fa494c00640 1 -- 172.21.15.151:0/2603973885 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fa4900e2ec0 con 0x7fa49810aa20 2024-05-21T21:55:42.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.327+0000 7fa497e00640 1 --2- 172.21.15.151:0/2603973885 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fa47c0b29f0 0x7fa47c0b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:42.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.328+0000 7fa497e00640 1 --2- 172.21.15.151:0/2603973885 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fa47c0b29f0 0x7fa47c0b4eb0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa488000c00 tx=0x7fa4880314a0 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:55:42.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.329+0000 7fa494c00640 1 -- 172.21.15.151:0/2603973885 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa4900aa6e0 con 0x7fa49810aa20 2024-05-21T21:55:42.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.536+0000 7fa49d800640 1 -- 172.21.15.151:0/2603973885 --> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa498062e90 con 0x7fa47c0b29f0 2024-05-21T21:55:42.554 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.553+0000 7fa494c00640 1 -- 172.21.15.151:0/2603973885 <== mgr.25138 v2:172.21.15.151:6800/958417056 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257822 (secure 0 0 0) 0x7fa498062e90 con 0x7fa47c0b29f0 2024-05-21T21:55:42.557 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T21:55:42.558 INFO:teuthology.orchestra.run.smithi151.stderr:dumped all 2024-05-21T21:55:42.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.557+0000 7fa49d800640 1 -- 172.21.15.151:0/2603973885 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fa47c0b29f0 msgr2=0x7fa47c0b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:42.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.557+0000 7fa49d800640 1 --2- 172.21.15.151:0/2603973885 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fa47c0b29f0 0x7fa47c0b4eb0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa488000c00 tx=0x7fa4880314a0 comp rx=0 tx=0).stop 2024-05-21T21:55:42.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.557+0000 7fa49d800640 1 -- 172.21.15.151:0/2603973885 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa49810aa20 msgr2=0x7fa4981a66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:42.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.557+0000 7fa49d800640 1 --2- 172.21.15.151:0/2603973885 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa49810aa20 0x7fa4981a66f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa490009fd0 tx=0x7fa49000ef10 comp rx=0 tx=0).stop 2024-05-21T21:55:42.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.557+0000 7fa49d800640 1 -- 172.21.15.151:0/2603973885 shutdown_connections 2024-05-21T21:55:42.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.557+0000 7fa49d800640 1 --2- 172.21.15.151:0/2603973885 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fa47c0b29f0 0x7fa47c0b4eb0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:42.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.557+0000 7fa49d800640 1 --2- 172.21.15.151:0/2603973885 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa49810aa20 0x7fa4981a66f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:42.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.557+0000 7fa49d800640 1 --2- 172.21.15.151:0/2603973885 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa49810a060 0x7fa49819f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:42.561 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.557+0000 7fa49d800640 1 --2- 172.21.15.151:0/2603973885 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa498108e80 0x7fa49819f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:42.561 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.557+0000 7fa49d800640 1 -- 172.21.15.151:0/2603973885 >> 172.21.15.151:0/2603973885 conn(0x7fa498077e90 msgr2=0x7fa49810d560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:55:42.561 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.557+0000 7fa49d800640 1 -- 172.21.15.151:0/2603973885 shutdown_connections 2024-05-21T21:55:42.561 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.557+0000 7fa49d800640 1 -- 172.21.15.151:0/2603973885 wait complete. 2024-05-21T21:55:42.577 INFO:teuthology.orchestra.run.smithi151.stdout:{"pg_ready":true,"pg_map":{"version":6,"stamp":"2024-05-21T21:55:42.220097+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1191254,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":2602,"num_read_kb":1445,"num_write":502,"num_write_kb":3619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":2952069,"num_keys_recovered":17,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":898,"ondisk_log_size":898,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":366336,"kb_used_data":13000,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124575488,"statfs":{"total":1151940427776,"available":1151565299712,"internally_reserved":0,"allocated":13312000,"data_stored":7384855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19140,"internal_metadata":361674044},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.978854"},"pg_stats":[{"pgid":"3.3f","version":"82'8","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.341334+0000","last_change":"2024-05-21T21:51:16.945132+0000","last_active":"2024-05-21T21:55:36.341334+0000","last_peered":"2024-05-21T21:55:36.341334+0000","last_clean":"2024-05-21T21:55:36.341334+0000","last_became_active":"2024-05-21T21:51:16.944766+0000","last_became_peered":"2024-05-21T21:51:16.944766+0000","last_unstale":"2024-05-21T21:55:36.341334+0000","last_undegraded":"2024-05-21T21:55:36.341334+0000","last_fullsized":"2024-05-21T21:55:36.341334+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:54:04.185095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"82'4","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.553624+0000","last_change":"2024-05-21T21:51:16.939291+0000","last_active":"2024-05-21T21:55:36.553624+0000","last_peered":"2024-05-21T21:55:36.553624+0000","last_clean":"2024-05-21T21:55:36.553624+0000","last_became_active":"2024-05-21T21:51:16.938730+0000","last_became_peered":"2024-05-21T21:51:16.938730+0000","last_unstale":"2024-05-21T21:55:36.553624+0000","last_undegraded":"2024-05-21T21:55:36.553624+0000","last_fullsized":"2024-05-21T21:55:36.553624+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:05:30.623744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"82'7","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.553252+0000","last_change":"2024-05-21T21:51:16.941565+0000","last_active":"2024-05-21T21:55:36.553252+0000","last_peered":"2024-05-21T21:55:36.553252+0000","last_clean":"2024-05-21T21:55:36.553252+0000","last_became_active":"2024-05-21T21:51:16.941033+0000","last_became_peered":"2024-05-21T21:51:16.941033+0000","last_unstale":"2024-05-21T21:55:36.553252+0000","last_undegraded":"2024-05-21T21:55:36.553252+0000","last_fullsized":"2024-05-21T21:55:36.553252+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:04:55.789479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"82'8","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.723315+0000","last_change":"2024-05-21T21:51:16.928455+0000","last_active":"2024-05-21T21:55:36.723315+0000","last_peered":"2024-05-21T21:55:36.723315+0000","last_clean":"2024-05-21T21:55:36.723315+0000","last_became_active":"2024-05-21T21:51:16.927874+0000","last_became_peered":"2024-05-21T21:51:16.927874+0000","last_unstale":"2024-05-21T21:55:36.723315+0000","last_undegraded":"2024-05-21T21:55:36.723315+0000","last_fullsized":"2024-05-21T21:55:36.723315+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:25:16.483175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"82'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.556362+0000","last_change":"2024-05-21T21:51:16.937773+0000","last_active":"2024-05-21T21:55:36.556362+0000","last_peered":"2024-05-21T21:55:36.556362+0000","last_clean":"2024-05-21T21:55:36.556362+0000","last_became_active":"2024-05-21T21:51:16.937306+0000","last_became_peered":"2024-05-21T21:51:16.937306+0000","last_unstale":"2024-05-21T21:55:36.556362+0000","last_undegraded":"2024-05-21T21:55:36.556362+0000","last_fullsized":"2024-05-21T21:55:36.556362+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:05:03.762390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"82'6","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.356010+0000","last_change":"2024-05-21T21:51:16.948275+0000","last_active":"2024-05-21T21:55:36.356010+0000","last_peered":"2024-05-21T21:55:36.356010+0000","last_clean":"2024-05-21T21:55:36.356010+0000","last_became_active":"2024-05-21T21:51:16.947890+0000","last_became_peered":"2024-05-21T21:51:16.947890+0000","last_unstale":"2024-05-21T21:55:36.356010+0000","last_undegraded":"2024-05-21T21:55:36.356010+0000","last_fullsized":"2024-05-21T21:55:36.356010+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:08:38.572177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"82'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.550575+0000","last_change":"2024-05-21T21:51:16.946821+0000","last_active":"2024-05-21T21:55:36.550575+0000","last_peered":"2024-05-21T21:55:36.550575+0000","last_clean":"2024-05-21T21:55:36.550575+0000","last_became_active":"2024-05-21T21:51:16.946515+0000","last_became_peered":"2024-05-21T21:51:16.946515+0000","last_unstale":"2024-05-21T21:55:36.550575+0000","last_undegraded":"2024-05-21T21:55:36.550575+0000","last_fullsized":"2024-05-21T21:55:36.550575+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T09:26:25.867895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.342252+0000","last_change":"2024-05-21T21:51:16.929228+0000","last_active":"2024-05-21T21:55:36.342252+0000","last_peered":"2024-05-21T21:55:36.342252+0000","last_clean":"2024-05-21T21:55:36.342252+0000","last_became_active":"2024-05-21T21:51:16.928875+0000","last_became_peered":"2024-05-21T21:51:16.928875+0000","last_unstale":"2024-05-21T21:55:36.342252+0000","last_undegraded":"2024-05-21T21:55:36.342252+0000","last_fullsized":"2024-05-21T21:55:36.342252+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:50:56.431310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"82'11","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.199749+0000","last_change":"2024-05-21T21:51:16.932360+0000","last_active":"2024-05-21T21:55:36.199749+0000","last_peered":"2024-05-21T21:55:36.199749+0000","last_clean":"2024-05-21T21:55:36.199749+0000","last_became_active":"2024-05-21T21:51:16.932036+0000","last_became_peered":"2024-05-21T21:51:16.932036+0000","last_unstale":"2024-05-21T21:55:36.199749+0000","last_undegraded":"2024-05-21T21:55:36.199749+0000","last_fullsized":"2024-05-21T21:55:36.199749+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T09:06:37.257323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"82'16","reported_seq":45,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.356382+0000","last_change":"2024-05-21T21:52:10.876556+0000","last_active":"2024-05-21T21:55:36.356382+0000","last_peered":"2024-05-21T21:55:36.356382+0000","last_clean":"2024-05-21T21:55:36.356382+0000","last_became_active":"2024-05-21T21:52:10.876273+0000","last_became_peered":"2024-05-21T21:52:10.876273+0000","last_unstale":"2024-05-21T21:55:36.356382+0000","last_undegraded":"2024-05-21T21:55:36.356382+0000","last_fullsized":"2024-05-21T21:55:36.356382+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T05:09:40.037731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"82'6","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.550761+0000","last_change":"2024-05-21T21:51:16.933425+0000","last_active":"2024-05-21T21:55:36.550761+0000","last_peered":"2024-05-21T21:55:36.550761+0000","last_clean":"2024-05-21T21:55:36.550761+0000","last_became_active":"2024-05-21T21:51:16.933057+0000","last_became_peered":"2024-05-21T21:51:16.933057+0000","last_unstale":"2024-05-21T21:55:36.550761+0000","last_undegraded":"2024-05-21T21:55:36.550761+0000","last_fullsized":"2024-05-21T21:55:36.550761+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:04:57.237841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"82'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.723313+0000","last_change":"2024-05-21T21:51:16.927014+0000","last_active":"2024-05-21T21:55:36.723313+0000","last_peered":"2024-05-21T21:55:36.723313+0000","last_clean":"2024-05-21T21:55:36.723313+0000","last_became_active":"2024-05-21T21:51:16.925954+0000","last_became_peered":"2024-05-21T21:51:16.925954+0000","last_unstale":"2024-05-21T21:55:36.723313+0000","last_undegraded":"2024-05-21T21:55:36.723313+0000","last_fullsized":"2024-05-21T21:55:36.723313+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:19:20.530727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"82'10","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.555649+0000","last_change":"2024-05-21T21:51:16.930909+0000","last_active":"2024-05-21T21:55:36.555649+0000","last_peered":"2024-05-21T21:55:36.555649+0000","last_clean":"2024-05-21T21:55:36.555649+0000","last_became_active":"2024-05-21T21:51:16.927110+0000","last_became_peered":"2024-05-21T21:51:16.927110+0000","last_unstale":"2024-05-21T21:55:36.555649+0000","last_undegraded":"2024-05-21T21:55:36.555649+0000","last_fullsized":"2024-05-21T21:55:36.555649+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T06:23:55.386047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"82'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.552564+0000","last_change":"2024-05-21T21:51:16.938797+0000","last_active":"2024-05-21T21:55:36.552564+0000","last_peered":"2024-05-21T21:55:36.552564+0000","last_clean":"2024-05-21T21:55:36.552564+0000","last_became_active":"2024-05-21T21:51:16.938263+0000","last_became_peered":"2024-05-21T21:51:16.938263+0000","last_unstale":"2024-05-21T21:55:36.552564+0000","last_undegraded":"2024-05-21T21:55:36.552564+0000","last_fullsized":"2024-05-21T21:55:36.552564+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:37:40.521118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"82'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.552309+0000","last_change":"2024-05-21T21:51:16.944160+0000","last_active":"2024-05-21T21:55:36.552309+0000","last_peered":"2024-05-21T21:55:36.552309+0000","last_clean":"2024-05-21T21:55:36.552309+0000","last_became_active":"2024-05-21T21:51:16.940046+0000","last_became_peered":"2024-05-21T21:51:16.940046+0000","last_unstale":"2024-05-21T21:55:36.552309+0000","last_undegraded":"2024-05-21T21:55:36.552309+0000","last_fullsized":"2024-05-21T21:55:36.552309+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:14:13.278799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.198999+0000","last_change":"2024-05-21T21:51:16.945599+0000","last_active":"2024-05-21T21:55:36.198999+0000","last_peered":"2024-05-21T21:55:36.198999+0000","last_clean":"2024-05-21T21:55:36.198999+0000","last_became_active":"2024-05-21T21:51:16.945196+0000","last_became_peered":"2024-05-21T21:51:16.945196+0000","last_unstale":"2024-05-21T21:55:36.198999+0000","last_undegraded":"2024-05-21T21:55:36.198999+0000","last_fullsized":"2024-05-21T21:55:36.198999+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:59:24.453615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"82'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.341504+0000","last_change":"2024-05-21T21:51:16.944895+0000","last_active":"2024-05-21T21:55:36.341504+0000","last_peered":"2024-05-21T21:55:36.341504+0000","last_clean":"2024-05-21T21:55:36.341504+0000","last_became_active":"2024-05-21T21:51:16.944560+0000","last_became_peered":"2024-05-21T21:51:16.944560+0000","last_unstale":"2024-05-21T21:55:36.341504+0000","last_undegraded":"2024-05-21T21:55:36.341504+0000","last_fullsized":"2024-05-21T21:55:36.341504+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:53:13.264701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"82'10","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.197858+0000","last_change":"2024-05-21T21:51:16.943861+0000","last_active":"2024-05-21T21:55:36.197858+0000","last_peered":"2024-05-21T21:55:36.197858+0000","last_clean":"2024-05-21T21:55:36.197858+0000","last_became_active":"2024-05-21T21:51:16.943123+0000","last_became_peered":"2024-05-21T21:51:16.943123+0000","last_unstale":"2024-05-21T21:55:36.197858+0000","last_undegraded":"2024-05-21T21:55:36.197858+0000","last_fullsized":"2024-05-21T21:55:36.197858+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:35:37.732855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"82'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.201422+0000","last_change":"2024-05-21T21:51:16.940580+0000","last_active":"2024-05-21T21:55:36.201422+0000","last_peered":"2024-05-21T21:55:36.201422+0000","last_clean":"2024-05-21T21:55:36.201422+0000","last_became_active":"2024-05-21T21:51:16.940264+0000","last_became_peered":"2024-05-21T21:51:16.940264+0000","last_unstale":"2024-05-21T21:55:36.201422+0000","last_undegraded":"2024-05-21T21:55:36.201422+0000","last_fullsized":"2024-05-21T21:55:36.201422+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:54:22.915541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'14","reported_seq":283,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.557627+0000","last_change":"2024-05-21T21:51:12.852935+0000","last_active":"2024-05-21T21:55:36.557627+0000","last_peered":"2024-05-21T21:55:36.557627+0000","last_clean":"2024-05-21T21:55:36.557627+0000","last_became_active":"2024-05-21T21:51:12.851802+0000","last_became_peered":"2024-05-21T21:51:12.851802+0000","last_unstale":"2024-05-21T21:55:36.557627+0000","last_undegraded":"2024-05-21T21:55:36.557627+0000","last_fullsized":"2024-05-21T21:55:36.557627+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:13:24.643408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":239,"num_read_kb":239,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"82'5","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.343135+0000","last_change":"2024-05-21T21:51:16.939461+0000","last_active":"2024-05-21T21:55:36.343135+0000","last_peered":"2024-05-21T21:55:36.343135+0000","last_clean":"2024-05-21T21:55:36.343135+0000","last_became_active":"2024-05-21T21:51:16.936322+0000","last_became_peered":"2024-05-21T21:51:16.936322+0000","last_unstale":"2024-05-21T21:55:36.343135+0000","last_undegraded":"2024-05-21T21:55:36.343135+0000","last_fullsized":"2024-05-21T21:55:36.343135+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:37:26.153963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.343791+0000","last_change":"2024-05-21T21:51:12.941838+0000","last_active":"2024-05-21T21:55:36.343791+0000","last_peered":"2024-05-21T21:55:36.343791+0000","last_clean":"2024-05-21T21:55:36.343791+0000","last_became_active":"2024-05-21T21:51:12.940818+0000","last_became_peered":"2024-05-21T21:51:12.940818+0000","last_unstale":"2024-05-21T21:55:36.343791+0000","last_undegraded":"2024-05-21T21:55:36.343791+0000","last_fullsized":"2024-05-21T21:55:36.343791+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:30:42.349241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"82'6","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.554128+0000","last_change":"2024-05-21T21:51:16.950252+0000","last_active":"2024-05-21T21:55:36.554128+0000","last_peered":"2024-05-21T21:55:36.554128+0000","last_clean":"2024-05-21T21:55:36.554128+0000","last_became_active":"2024-05-21T21:51:16.947375+0000","last_became_peered":"2024-05-21T21:51:16.947375+0000","last_unstale":"2024-05-21T21:55:36.554128+0000","last_undegraded":"2024-05-21T21:55:36.554128+0000","last_fullsized":"2024-05-21T21:55:36.554128+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:27:30.462799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"80'12","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.341317+0000","last_change":"2024-05-21T21:52:10.875739+0000","last_active":"2024-05-21T21:55:36.341317+0000","last_peered":"2024-05-21T21:55:36.341317+0000","last_clean":"2024-05-21T21:55:36.341317+0000","last_became_active":"2024-05-21T21:52:10.865846+0000","last_became_peered":"2024-05-21T21:52:10.865846+0000","last_unstale":"2024-05-21T21:55:36.341317+0000","last_undegraded":"2024-05-21T21:55:36.341317+0000","last_fullsized":"2024-05-21T21:55:36.341317+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T06:48:06.465757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.199107+0000","last_change":"2024-05-21T21:51:16.926380+0000","last_active":"2024-05-21T21:55:36.199107+0000","last_peered":"2024-05-21T21:55:36.199107+0000","last_clean":"2024-05-21T21:55:36.199107+0000","last_became_active":"2024-05-21T21:51:16.925821+0000","last_became_peered":"2024-05-21T21:51:16.925821+0000","last_unstale":"2024-05-21T21:55:36.199107+0000","last_undegraded":"2024-05-21T21:55:36.199107+0000","last_fullsized":"2024-05-21T21:55:36.199107+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:42:14.887155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'5","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.556613+0000","last_change":"2024-05-21T21:51:12.854342+0000","last_active":"2024-05-21T21:55:36.556613+0000","last_peered":"2024-05-21T21:55:36.556613+0000","last_clean":"2024-05-21T21:55:36.556613+0000","last_became_active":"2024-05-21T21:51:12.853908+0000","last_became_peered":"2024-05-21T21:51:12.853908+0000","last_unstale":"2024-05-21T21:55:36.556613+0000","last_undegraded":"2024-05-21T21:55:36.556613+0000","last_fullsized":"2024-05-21T21:55:36.556613+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:35:10.374749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"82'15","reported_seq":44,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.352781+0000","last_change":"2024-05-21T21:52:10.973816+0000","last_active":"2024-05-21T21:55:36.352781+0000","last_peered":"2024-05-21T21:55:36.352781+0000","last_clean":"2024-05-21T21:55:36.352781+0000","last_became_active":"2024-05-21T21:52:10.973457+0000","last_became_peered":"2024-05-21T21:52:10.973457+0000","last_unstale":"2024-05-21T21:55:36.352781+0000","last_undegraded":"2024-05-21T21:55:36.352781+0000","last_fullsized":"2024-05-21T21:55:36.352781+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T05:01:41.915158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"83'9","reported_seq":282,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.875854+0000","last_change":"2024-05-21T21:51:12.855658+0000","last_active":"2024-05-21T21:55:36.875854+0000","last_peered":"2024-05-21T21:55:36.875854+0000","last_clean":"2024-05-21T21:55:36.875854+0000","last_became_active":"2024-05-21T21:51:12.855281+0000","last_became_peered":"2024-05-21T21:51:12.855281+0000","last_unstale":"2024-05-21T21:55:36.875854+0000","last_undegraded":"2024-05-21T21:55:36.875854+0000","last_fullsized":"2024-05-21T21:55:36.875854+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:55:08.836045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1855,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":244,"num_read_kb":6,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"82'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.556310+0000","last_change":"2024-05-21T21:51:16.946977+0000","last_active":"2024-05-21T21:55:36.556310+0000","last_peered":"2024-05-21T21:55:36.556310+0000","last_clean":"2024-05-21T21:55:36.556310+0000","last_became_active":"2024-05-21T21:51:16.946562+0000","last_became_peered":"2024-05-21T21:51:16.946562+0000","last_unstale":"2024-05-21T21:55:36.556310+0000","last_undegraded":"2024-05-21T21:55:36.556310+0000","last_fullsized":"2024-05-21T21:55:36.556310+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:43:33.370265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"80'7","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.355936+0000","last_change":"2024-05-21T21:51:12.852574+0000","last_active":"2024-05-21T21:55:36.355936+0000","last_peered":"2024-05-21T21:55:36.355936+0000","last_clean":"2024-05-21T21:55:36.355936+0000","last_became_active":"2024-05-21T21:51:12.852258+0000","last_became_peered":"2024-05-21T21:51:12.852258+0000","last_unstale":"2024-05-21T21:55:36.355936+0000","last_undegraded":"2024-05-21T21:55:36.355936+0000","last_fullsized":"2024-05-21T21:55:36.355936+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:09:40.280821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"82'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.352655+0000","last_change":"2024-05-21T21:51:16.934768+0000","last_active":"2024-05-21T21:55:36.352655+0000","last_peered":"2024-05-21T21:55:36.352655+0000","last_clean":"2024-05-21T21:55:36.352655+0000","last_became_active":"2024-05-21T21:51:16.934275+0000","last_became_peered":"2024-05-21T21:51:16.934275+0000","last_unstale":"2024-05-21T21:55:36.352655+0000","last_undegraded":"2024-05-21T21:55:36.352655+0000","last_fullsized":"2024-05-21T21:55:36.352655+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:31:54.425069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"80'3","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.555219+0000","last_change":"2024-05-21T21:51:12.854166+0000","last_active":"2024-05-21T21:55:36.555219+0000","last_peered":"2024-05-21T21:55:36.555219+0000","last_clean":"2024-05-21T21:55:36.555219+0000","last_became_active":"2024-05-21T21:51:12.853508+0000","last_became_peered":"2024-05-21T21:51:12.853508+0000","last_unstale":"2024-05-21T21:55:36.555219+0000","last_undegraded":"2024-05-21T21:55:36.555219+0000","last_fullsized":"2024-05-21T21:55:36.555219+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:29:41.858655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"82'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.342248+0000","last_change":"2024-05-21T21:51:16.938293+0000","last_active":"2024-05-21T21:55:36.342248+0000","last_peered":"2024-05-21T21:55:36.342248+0000","last_clean":"2024-05-21T21:55:36.342248+0000","last_became_active":"2024-05-21T21:51:16.937970+0000","last_became_peered":"2024-05-21T21:51:16.937970+0000","last_unstale":"2024-05-21T21:55:36.342248+0000","last_undegraded":"2024-05-21T21:55:36.342248+0000","last_fullsized":"2024-05-21T21:55:36.342248+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:18:27.473098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'3","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.352476+0000","last_change":"2024-05-21T21:51:12.853366+0000","last_active":"2024-05-21T21:55:36.352476+0000","last_peered":"2024-05-21T21:55:36.352476+0000","last_clean":"2024-05-21T21:55:36.352476+0000","last_became_active":"2024-05-21T21:51:12.853028+0000","last_became_peered":"2024-05-21T21:51:12.853028+0000","last_unstale":"2024-05-21T21:55:36.352476+0000","last_undegraded":"2024-05-21T21:55:36.352476+0000","last_fullsized":"2024-05-21T21:55:36.352476+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T09:15:52.593028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"82'11","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.197181+0000","last_change":"2024-05-21T21:51:16.947903+0000","last_active":"2024-05-21T21:55:36.197181+0000","last_peered":"2024-05-21T21:55:36.197181+0000","last_clean":"2024-05-21T21:55:36.197181+0000","last_became_active":"2024-05-21T21:51:16.946677+0000","last_became_peered":"2024-05-21T21:51:16.946677+0000","last_unstale":"2024-05-21T21:55:36.197181+0000","last_undegraded":"2024-05-21T21:55:36.197181+0000","last_fullsized":"2024-05-21T21:55:36.197181+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T06:12:56.514284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'7","reported_seq":512,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.746589+0000","last_change":"2024-05-21T21:51:12.854873+0000","last_active":"2024-05-21T21:55:36.746589+0000","last_peered":"2024-05-21T21:55:36.746589+0000","last_clean":"2024-05-21T21:55:36.746589+0000","last_became_active":"2024-05-21T21:51:12.854393+0000","last_became_peered":"2024-05-21T21:51:12.854393+0000","last_unstale":"2024-05-21T21:55:36.746589+0000","last_undegraded":"2024-05-21T21:55:36.746589+0000","last_fullsized":"2024-05-21T21:55:36.746589+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:58:52.411164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":476,"num_read_kb":476,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"82'10","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.199587+0000","last_change":"2024-05-21T21:51:16.951094+0000","last_active":"2024-05-21T21:55:36.199587+0000","last_peered":"2024-05-21T21:55:36.199587+0000","last_clean":"2024-05-21T21:55:36.199587+0000","last_became_active":"2024-05-21T21:51:16.950724+0000","last_became_peered":"2024-05-21T21:51:16.950724+0000","last_unstale":"2024-05-21T21:55:36.199587+0000","last_undegraded":"2024-05-21T21:55:36.199587+0000","last_fullsized":"2024-05-21T21:55:36.199587+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:54:02.835426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'4","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.550904+0000","last_change":"2024-05-21T21:51:12.850427+0000","last_active":"2024-05-21T21:55:36.550904+0000","last_peered":"2024-05-21T21:55:36.550904+0000","last_clean":"2024-05-21T21:55:36.550904+0000","last_became_active":"2024-05-21T21:51:12.850096+0000","last_became_peered":"2024-05-21T21:51:12.850096+0000","last_unstale":"2024-05-21T21:55:36.550904+0000","last_undegraded":"2024-05-21T21:55:36.550904+0000","last_fullsized":"2024-05-21T21:55:36.550904+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:43:22.611043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.554216+0000","last_change":"2024-05-21T21:51:16.942330+0000","last_active":"2024-05-21T21:55:36.554216+0000","last_peered":"2024-05-21T21:55:36.554216+0000","last_clean":"2024-05-21T21:55:36.554216+0000","last_became_active":"2024-05-21T21:51:16.941718+0000","last_became_peered":"2024-05-21T21:51:16.941718+0000","last_unstale":"2024-05-21T21:55:36.554216+0000","last_undegraded":"2024-05-21T21:55:36.554216+0000","last_fullsized":"2024-05-21T21:55:36.554216+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:54:17.176411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'5","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.352907+0000","last_change":"2024-05-21T21:51:12.941100+0000","last_active":"2024-05-21T21:55:36.352907+0000","last_peered":"2024-05-21T21:55:36.352907+0000","last_clean":"2024-05-21T21:55:36.352907+0000","last_became_active":"2024-05-21T21:51:12.940597+0000","last_became_peered":"2024-05-21T21:51:12.940597+0000","last_unstale":"2024-05-21T21:55:36.352907+0000","last_undegraded":"2024-05-21T21:55:36.352907+0000","last_fullsized":"2024-05-21T21:55:36.352907+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:30:58.457108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"82'11","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.199324+0000","last_change":"2024-05-21T21:51:16.921973+0000","last_active":"2024-05-21T21:55:36.199324+0000","last_peered":"2024-05-21T21:55:36.199324+0000","last_clean":"2024-05-21T21:55:36.199324+0000","last_became_active":"2024-05-21T21:51:16.921597+0000","last_became_peered":"2024-05-21T21:51:16.921597+0000","last_unstale":"2024-05-21T21:55:36.199324+0000","last_undegraded":"2024-05-21T21:55:36.199324+0000","last_fullsized":"2024-05-21T21:55:36.199324+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:25:32.395940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'39","reported_seq":557,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.747043+0000","last_change":"2024-05-21T21:51:12.850312+0000","last_active":"2024-05-21T21:55:36.747043+0000","last_peered":"2024-05-21T21:55:36.747043+0000","last_clean":"2024-05-21T21:55:36.747043+0000","last_became_active":"2024-05-21T21:51:12.849837+0000","last_became_peered":"2024-05-21T21:51:12.849837+0000","last_unstale":"2024-05-21T21:55:36.747043+0000","last_undegraded":"2024-05-21T21:55:36.747043+0000","last_fullsized":"2024-05-21T21:55:36.747043+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T07:09:13.339554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66476,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":489,"num_read_kb":309,"num_write":41,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"82'6","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.556542+0000","last_change":"2024-05-21T21:51:16.926625+0000","last_active":"2024-05-21T21:55:36.556542+0000","last_peered":"2024-05-21T21:55:36.556542+0000","last_clean":"2024-05-21T21:55:36.556542+0000","last_became_active":"2024-05-21T21:51:16.925889+0000","last_became_peered":"2024-05-21T21:51:16.925889+0000","last_unstale":"2024-05-21T21:55:36.556542+0000","last_undegraded":"2024-05-21T21:55:36.556542+0000","last_fullsized":"2024-05-21T21:55:36.556542+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:35:23.464747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'9","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.201248+0000","last_change":"2024-05-21T21:51:12.859772+0000","last_active":"2024-05-21T21:55:36.201248+0000","last_peered":"2024-05-21T21:55:36.201248+0000","last_clean":"2024-05-21T21:55:36.201248+0000","last_became_active":"2024-05-21T21:51:12.859290+0000","last_became_peered":"2024-05-21T21:51:12.859290+0000","last_unstale":"2024-05-21T21:55:36.201248+0000","last_undegraded":"2024-05-21T21:55:36.201248+0000","last_fullsized":"2024-05-21T21:55:36.201248+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T05:28:29.953381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.555713+0000","last_change":"2024-05-21T21:51:16.925803+0000","last_active":"2024-05-21T21:55:36.555713+0000","last_peered":"2024-05-21T21:55:36.555713+0000","last_clean":"2024-05-21T21:55:36.555713+0000","last_became_active":"2024-05-21T21:51:16.925111+0000","last_became_peered":"2024-05-21T21:51:16.925111+0000","last_unstale":"2024-05-21T21:55:36.555713+0000","last_undegraded":"2024-05-21T21:55:36.555713+0000","last_fullsized":"2024-05-21T21:55:36.555713+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T06:22:11.353379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'5","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.352681+0000","last_change":"2024-05-21T21:51:12.859593+0000","last_active":"2024-05-21T21:55:36.352681+0000","last_peered":"2024-05-21T21:55:36.352681+0000","last_clean":"2024-05-21T21:55:36.352681+0000","last_became_active":"2024-05-21T21:51:12.859197+0000","last_became_peered":"2024-05-21T21:51:12.859197+0000","last_unstale":"2024-05-21T21:55:36.352681+0000","last_undegraded":"2024-05-21T21:55:36.352681+0000","last_fullsized":"2024-05-21T21:55:36.352681+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:13:12.311496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"82'15","reported_seq":44,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.551024+0000","last_change":"2024-05-21T21:52:10.879028+0000","last_active":"2024-05-21T21:55:36.551024+0000","last_peered":"2024-05-21T21:55:36.551024+0000","last_clean":"2024-05-21T21:55:36.551024+0000","last_became_active":"2024-05-21T21:52:10.878664+0000","last_became_peered":"2024-05-21T21:52:10.878664+0000","last_unstale":"2024-05-21T21:55:36.551024+0000","last_undegraded":"2024-05-21T21:55:36.551024+0000","last_fullsized":"2024-05-21T21:55:36.551024+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:05:53.953546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'7","reported_seq":278,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.723211+0000","last_change":"2024-05-21T21:51:12.852169+0000","last_active":"2024-05-21T21:55:36.723211+0000","last_peered":"2024-05-21T21:55:36.723211+0000","last_clean":"2024-05-21T21:55:36.723211+0000","last_became_active":"2024-05-21T21:51:12.851782+0000","last_became_peered":"2024-05-21T21:51:12.851782+0000","last_unstale":"2024-05-21T21:55:36.723211+0000","last_undegraded":"2024-05-21T21:55:36.723211+0000","last_fullsized":"2024-05-21T21:55:36.723211+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:46:09.692158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":243,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"82'6","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.550518+0000","last_change":"2024-05-21T21:51:16.929264+0000","last_active":"2024-05-21T21:55:36.550518+0000","last_peered":"2024-05-21T21:55:36.550518+0000","last_clean":"2024-05-21T21:55:36.550518+0000","last_became_active":"2024-05-21T21:51:16.928900+0000","last_became_peered":"2024-05-21T21:51:16.928900+0000","last_unstale":"2024-05-21T21:55:36.550518+0000","last_undegraded":"2024-05-21T21:55:36.550518+0000","last_fullsized":"2024-05-21T21:55:36.550518+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:42:28.040939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'17","reported_seq":71,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.556247+0000","last_change":"2024-05-21T21:52:10.984451+0000","last_active":"2024-05-21T21:55:36.556247+0000","last_peered":"2024-05-21T21:55:36.556247+0000","last_clean":"2024-05-21T21:55:36.556247+0000","last_became_active":"2024-05-21T21:52:10.974050+0000","last_became_peered":"2024-05-21T21:52:10.974050+0000","last_unstale":"2024-05-21T21:55:36.556247+0000","last_undegraded":"2024-05-21T21:55:36.556247+0000","last_fullsized":"2024-05-21T21:55:36.556247+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T07:08:18.307022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"82'6","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.352648+0000","last_change":"2024-05-21T21:51:16.935464+0000","last_active":"2024-05-21T21:55:36.352648+0000","last_peered":"2024-05-21T21:55:36.352648+0000","last_clean":"2024-05-21T21:55:36.352648+0000","last_became_active":"2024-05-21T21:51:16.935091+0000","last_became_peered":"2024-05-21T21:51:16.935091+0000","last_unstale":"2024-05-21T21:55:36.352648+0000","last_undegraded":"2024-05-21T21:55:36.352648+0000","last_fullsized":"2024-05-21T21:55:36.352648+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T06:59:16.330649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.556695+0000","last_change":"2024-05-21T21:51:12.853327+0000","last_active":"2024-05-21T21:55:36.556695+0000","last_peered":"2024-05-21T21:55:36.556695+0000","last_clean":"2024-05-21T21:55:36.556695+0000","last_became_active":"2024-05-21T21:51:12.852946+0000","last_became_peered":"2024-05-21T21:51:12.852946+0000","last_unstale":"2024-05-21T21:55:36.556695+0000","last_undegraded":"2024-05-21T21:55:36.556695+0000","last_fullsized":"2024-05-21T21:55:36.556695+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:00:34.256858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"82'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.341305+0000","last_change":"2024-05-21T21:51:16.929341+0000","last_active":"2024-05-21T21:55:36.341305+0000","last_peered":"2024-05-21T21:55:36.341305+0000","last_clean":"2024-05-21T21:55:36.341305+0000","last_became_active":"2024-05-21T21:51:16.929008+0000","last_became_peered":"2024-05-21T21:51:16.929008+0000","last_unstale":"2024-05-21T21:55:36.341305+0000","last_undegraded":"2024-05-21T21:55:36.341305+0000","last_fullsized":"2024-05-21T21:55:36.341305+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:26:26.297136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"80'1","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.197640+0000","last_change":"2024-05-21T21:51:12.945433+0000","last_active":"2024-05-21T21:55:36.197640+0000","last_peered":"2024-05-21T21:55:36.197640+0000","last_clean":"2024-05-21T21:55:36.197640+0000","last_became_active":"2024-05-21T21:51:12.944463+0000","last_became_peered":"2024-05-21T21:51:12.944463+0000","last_unstale":"2024-05-21T21:55:36.197640+0000","last_undegraded":"2024-05-21T21:55:36.197640+0000","last_fullsized":"2024-05-21T21:55:36.197640+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:29:52.014583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'5","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.356192+0000","last_change":"2024-05-21T21:52:10.883554+0000","last_active":"2024-05-21T21:55:36.356192+0000","last_peered":"2024-05-21T21:55:36.356192+0000","last_clean":"2024-05-21T21:55:36.356192+0000","last_became_active":"2024-05-21T21:52:10.877789+0000","last_became_peered":"2024-05-21T21:52:10.877789+0000","last_unstale":"2024-05-21T21:55:36.356192+0000","last_undegraded":"2024-05-21T21:55:36.356192+0000","last_fullsized":"2024-05-21T21:55:36.356192+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:52:37.623551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"83'6","reported_seq":276,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.890796+0000","last_change":"2024-05-21T21:51:12.852883+0000","last_active":"2024-05-21T21:55:36.890796+0000","last_peered":"2024-05-21T21:55:36.890796+0000","last_clean":"2024-05-21T21:55:36.890796+0000","last_became_active":"2024-05-21T21:51:12.852549+0000","last_became_peered":"2024-05-21T21:51:12.852549+0000","last_unstale":"2024-05-21T21:55:36.890796+0000","last_undegraded":"2024-05-21T21:55:36.890796+0000","last_fullsized":"2024-05-21T21:55:36.890796+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:05:53.002286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":240,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"82'7","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.356089+0000","last_change":"2024-05-21T21:51:16.944425+0000","last_active":"2024-05-21T21:55:36.356089+0000","last_peered":"2024-05-21T21:55:36.356089+0000","last_clean":"2024-05-21T21:55:36.356089+0000","last_became_active":"2024-05-21T21:51:16.944033+0000","last_became_peered":"2024-05-21T21:51:16.944033+0000","last_unstale":"2024-05-21T21:55:36.356089+0000","last_undegraded":"2024-05-21T21:55:36.356089+0000","last_fullsized":"2024-05-21T21:55:36.356089+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T07:16:26.802329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"82'8","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.352742+0000","last_change":"2024-05-21T21:51:16.933264+0000","last_active":"2024-05-21T21:55:36.352742+0000","last_peered":"2024-05-21T21:55:36.352742+0000","last_clean":"2024-05-21T21:55:36.352742+0000","last_became_active":"2024-05-21T21:51:16.932911+0000","last_became_peered":"2024-05-21T21:51:16.932911+0000","last_unstale":"2024-05-21T21:55:36.352742+0000","last_undegraded":"2024-05-21T21:55:36.352742+0000","last_fullsized":"2024-05-21T21:55:36.352742+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:25:54.006367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"82'6","reported_seq":275,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.555395+0000","last_change":"2024-05-21T21:51:12.942900+0000","last_active":"2024-05-21T21:55:36.555395+0000","last_peered":"2024-05-21T21:55:36.555395+0000","last_clean":"2024-05-21T21:55:36.555395+0000","last_became_active":"2024-05-21T21:51:12.941038+0000","last_became_peered":"2024-05-21T21:51:12.941038+0000","last_unstale":"2024-05-21T21:55:36.555395+0000","last_undegraded":"2024-05-21T21:55:36.555395+0000","last_fullsized":"2024-05-21T21:55:36.555395+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:38:43.240011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":239,"num_read_kb":239,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"82'8","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.342906+0000","last_change":"2024-05-21T21:51:16.933391+0000","last_active":"2024-05-21T21:55:36.342906+0000","last_peered":"2024-05-21T21:55:36.342906+0000","last_clean":"2024-05-21T21:55:36.342906+0000","last_became_active":"2024-05-21T21:51:16.932320+0000","last_became_peered":"2024-05-21T21:51:16.932320+0000","last_unstale":"2024-05-21T21:55:36.342906+0000","last_undegraded":"2024-05-21T21:55:36.342906+0000","last_fullsized":"2024-05-21T21:55:36.342906+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T09:06:59.126748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'6","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.723215+0000","last_change":"2024-05-21T21:51:12.856342+0000","last_active":"2024-05-21T21:55:36.723215+0000","last_peered":"2024-05-21T21:55:36.723215+0000","last_clean":"2024-05-21T21:55:36.723215+0000","last_became_active":"2024-05-21T21:51:12.855952+0000","last_became_peered":"2024-05-21T21:51:12.855952+0000","last_unstale":"2024-05-21T21:55:36.723215+0000","last_undegraded":"2024-05-21T21:55:36.723215+0000","last_fullsized":"2024-05-21T21:55:36.723215+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:34:40.734034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":103,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.550605+0000","last_change":"2024-05-21T21:49:42.797187+0000","last_active":"2024-05-21T21:55:36.550605+0000","last_peered":"2024-05-21T21:55:36.550605+0000","last_clean":"2024-05-21T21:55:36.550605+0000","last_became_active":"2024-05-21T21:49:42.791291+0000","last_became_peered":"2024-05-21T21:49:42.791291+0000","last_unstale":"2024-05-21T21:55:36.550605+0000","last_undegraded":"2024-05-21T21:55:36.550605+0000","last_fullsized":"2024-05-21T21:55:36.550605+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:45:55.552293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:45:55.552293+0000","last_clean_scrub_stamp":"2024-05-21T21:45:55.552293+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T07:24:18.302523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"82'5","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.555133+0000","last_change":"2024-05-21T21:51:16.926446+0000","last_active":"2024-05-21T21:55:36.555133+0000","last_peered":"2024-05-21T21:55:36.555133+0000","last_clean":"2024-05-21T21:55:36.555133+0000","last_became_active":"2024-05-21T21:51:16.926070+0000","last_became_peered":"2024-05-21T21:51:16.926070+0000","last_unstale":"2024-05-21T21:55:36.555133+0000","last_undegraded":"2024-05-21T21:55:36.555133+0000","last_fullsized":"2024-05-21T21:55:36.555133+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:44:27.366393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'4","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.550735+0000","last_change":"2024-05-21T21:51:12.854371+0000","last_active":"2024-05-21T21:55:36.550735+0000","last_peered":"2024-05-21T21:55:36.550735+0000","last_clean":"2024-05-21T21:55:36.550735+0000","last_became_active":"2024-05-21T21:51:12.853847+0000","last_became_peered":"2024-05-21T21:51:12.853847+0000","last_unstale":"2024-05-21T21:55:36.550735+0000","last_undegraded":"2024-05-21T21:55:36.550735+0000","last_fullsized":"2024-05-21T21:55:36.550735+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:14:10.549901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"80'4","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.556551+0000","last_change":"2024-05-21T21:51:12.941227+0000","last_active":"2024-05-21T21:55:36.556551+0000","last_peered":"2024-05-21T21:55:36.556551+0000","last_clean":"2024-05-21T21:55:36.556551+0000","last_became_active":"2024-05-21T21:51:12.940702+0000","last_became_peered":"2024-05-21T21:51:12.940702+0000","last_unstale":"2024-05-21T21:55:36.556551+0000","last_undegraded":"2024-05-21T21:55:36.556551+0000","last_fullsized":"2024-05-21T21:55:36.556551+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:40:34.124921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"82'9","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.556301+0000","last_change":"2024-05-21T21:51:16.945627+0000","last_active":"2024-05-21T21:55:36.556301+0000","last_peered":"2024-05-21T21:55:36.556301+0000","last_clean":"2024-05-21T21:55:36.556301+0000","last_became_active":"2024-05-21T21:51:16.945297+0000","last_became_peered":"2024-05-21T21:51:16.945297+0000","last_unstale":"2024-05-21T21:55:36.556301+0000","last_undegraded":"2024-05-21T21:55:36.556301+0000","last_fullsized":"2024-05-21T21:55:36.556301+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:54:38.898774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'51","reported_seq":120,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.502619+0000","last_change":"2024-05-21T21:52:10.888076+0000","last_active":"2024-05-21T21:55:36.502619+0000","last_peered":"2024-05-21T21:55:36.502619+0000","last_clean":"2024-05-21T21:55:36.502619+0000","last_became_active":"2024-05-21T21:52:10.879368+0000","last_became_peered":"2024-05-21T21:52:10.879368+0000","last_unstale":"2024-05-21T21:55:36.502619+0000","last_undegraded":"2024-05-21T21:55:36.502619+0000","last_fullsized":"2024-05-21T21:55:36.502619+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:46:43.190411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":91,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"82'5","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.344028+0000","last_change":"2024-05-21T21:51:16.929807+0000","last_active":"2024-05-21T21:55:36.344028+0000","last_peered":"2024-05-21T21:55:36.344028+0000","last_clean":"2024-05-21T21:55:36.344028+0000","last_became_active":"2024-05-21T21:51:16.928859+0000","last_became_peered":"2024-05-21T21:51:16.928859+0000","last_unstale":"2024-05-21T21:55:36.344028+0000","last_undegraded":"2024-05-21T21:55:36.344028+0000","last_fullsized":"2024-05-21T21:55:36.344028+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:41:32.830892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'7","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.723447+0000","last_change":"2024-05-21T21:51:12.850843+0000","last_active":"2024-05-21T21:55:36.723447+0000","last_peered":"2024-05-21T21:55:36.723447+0000","last_clean":"2024-05-21T21:55:36.723447+0000","last_became_active":"2024-05-21T21:51:12.850516+0000","last_became_peered":"2024-05-21T21:51:12.850516+0000","last_unstale":"2024-05-21T21:55:36.723447+0000","last_undegraded":"2024-05-21T21:55:36.723447+0000","last_fullsized":"2024-05-21T21:55:36.723447+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:37:05.191626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"82'7","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.553781+0000","last_change":"2024-05-21T21:51:16.949017+0000","last_active":"2024-05-21T21:55:36.553781+0000","last_peered":"2024-05-21T21:55:36.553781+0000","last_clean":"2024-05-21T21:55:36.553781+0000","last_became_active":"2024-05-21T21:51:16.948691+0000","last_became_peered":"2024-05-21T21:51:16.948691+0000","last_unstale":"2024-05-21T21:55:36.553781+0000","last_undegraded":"2024-05-21T21:55:36.553781+0000","last_fullsized":"2024-05-21T21:55:36.553781+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:32:06.606581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.199191+0000","last_change":"2024-05-21T21:51:12.939203+0000","last_active":"2024-05-21T21:55:36.199191+0000","last_peered":"2024-05-21T21:55:36.199191+0000","last_clean":"2024-05-21T21:55:36.199191+0000","last_became_active":"2024-05-21T21:51:12.938936+0000","last_became_peered":"2024-05-21T21:51:12.938936+0000","last_unstale":"2024-05-21T21:55:36.199191+0000","last_undegraded":"2024-05-21T21:55:36.199191+0000","last_fullsized":"2024-05-21T21:55:36.199191+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:55:50.336362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"82'2","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.556315+0000","last_change":"2024-05-21T21:51:16.926216+0000","last_active":"2024-05-21T21:55:36.556315+0000","last_peered":"2024-05-21T21:55:36.556315+0000","last_clean":"2024-05-21T21:55:36.556315+0000","last_became_active":"2024-05-21T21:51:16.925834+0000","last_became_peered":"2024-05-21T21:51:16.925834+0000","last_unstale":"2024-05-21T21:55:36.556315+0000","last_undegraded":"2024-05-21T21:55:36.556315+0000","last_fullsized":"2024-05-21T21:55:36.556315+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:13:30.192908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'7","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.342741+0000","last_change":"2024-05-21T21:51:12.855306+0000","last_active":"2024-05-21T21:55:36.342741+0000","last_peered":"2024-05-21T21:55:36.342741+0000","last_clean":"2024-05-21T21:55:36.342741+0000","last_became_active":"2024-05-21T21:51:12.854722+0000","last_became_peered":"2024-05-21T21:51:12.854722+0000","last_unstale":"2024-05-21T21:55:36.342741+0000","last_undegraded":"2024-05-21T21:55:36.342741+0000","last_fullsized":"2024-05-21T21:55:36.342741+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:39:41.583219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"82'9","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.341026+0000","last_change":"2024-05-21T21:51:16.944758+0000","last_active":"2024-05-21T21:55:36.341026+0000","last_peered":"2024-05-21T21:55:36.341026+0000","last_clean":"2024-05-21T21:55:36.341026+0000","last_became_active":"2024-05-21T21:51:16.944393+0000","last_became_peered":"2024-05-21T21:51:16.944393+0000","last_unstale":"2024-05-21T21:55:36.341026+0000","last_undegraded":"2024-05-21T21:55:36.341026+0000","last_fullsized":"2024-05-21T21:55:36.341026+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:01:23.048355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'3","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.201175+0000","last_change":"2024-05-21T21:51:12.860378+0000","last_active":"2024-05-21T21:55:36.201175+0000","last_peered":"2024-05-21T21:55:36.201175+0000","last_clean":"2024-05-21T21:55:36.201175+0000","last_became_active":"2024-05-21T21:51:12.859978+0000","last_became_peered":"2024-05-21T21:51:12.859978+0000","last_unstale":"2024-05-21T21:55:36.201175+0000","last_undegraded":"2024-05-21T21:55:36.201175+0000","last_fullsized":"2024-05-21T21:55:36.201175+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T07:52:26.902232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"82'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.343347+0000","last_change":"2024-05-21T21:51:16.943194+0000","last_active":"2024-05-21T21:55:36.343347+0000","last_peered":"2024-05-21T21:55:36.343347+0000","last_clean":"2024-05-21T21:55:36.343347+0000","last_became_active":"2024-05-21T21:51:16.942884+0000","last_became_peered":"2024-05-21T21:51:16.942884+0000","last_unstale":"2024-05-21T21:55:36.343347+0000","last_undegraded":"2024-05-21T21:55:36.343347+0000","last_fullsized":"2024-05-21T21:55:36.343347+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T05:58:17.436995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.556452+0000","last_change":"2024-05-21T21:51:12.853390+0000","last_active":"2024-05-21T21:55:36.556452+0000","last_peered":"2024-05-21T21:55:36.556452+0000","last_clean":"2024-05-21T21:55:36.556452+0000","last_became_active":"2024-05-21T21:51:12.852990+0000","last_became_peered":"2024-05-21T21:51:12.852990+0000","last_unstale":"2024-05-21T21:55:36.556452+0000","last_undegraded":"2024-05-21T21:55:36.556452+0000","last_fullsized":"2024-05-21T21:55:36.556452+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:32:16.392609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"82'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.550802+0000","last_change":"2024-05-21T21:51:16.933430+0000","last_active":"2024-05-21T21:55:36.550802+0000","last_peered":"2024-05-21T21:55:36.550802+0000","last_clean":"2024-05-21T21:55:36.550802+0000","last_became_active":"2024-05-21T21:51:16.933053+0000","last_became_peered":"2024-05-21T21:51:16.933053+0000","last_unstale":"2024-05-21T21:55:36.550802+0000","last_undegraded":"2024-05-21T21:55:36.550802+0000","last_fullsized":"2024-05-21T21:55:36.550802+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T06:14:57.468727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'7","reported_seq":277,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.747135+0000","last_change":"2024-05-21T21:51:12.946113+0000","last_active":"2024-05-21T21:55:36.747135+0000","last_peered":"2024-05-21T21:55:36.747135+0000","last_clean":"2024-05-21T21:55:36.747135+0000","last_became_active":"2024-05-21T21:51:12.945650+0000","last_became_peered":"2024-05-21T21:51:12.945650+0000","last_unstale":"2024-05-21T21:55:36.747135+0000","last_undegraded":"2024-05-21T21:55:36.747135+0000","last_fullsized":"2024-05-21T21:55:36.747135+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T09:40:18.209400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1952,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":241,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"82'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.550873+0000","last_change":"2024-05-21T21:51:16.932632+0000","last_active":"2024-05-21T21:55:36.550873+0000","last_peered":"2024-05-21T21:55:36.550873+0000","last_clean":"2024-05-21T21:55:36.550873+0000","last_became_active":"2024-05-21T21:51:16.932256+0000","last_became_peered":"2024-05-21T21:51:16.932256+0000","last_unstale":"2024-05-21T21:55:36.550873+0000","last_undegraded":"2024-05-21T21:55:36.550873+0000","last_fullsized":"2024-05-21T21:55:36.550873+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:04:48.146485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'41","reported_seq":78,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.496010+0000","last_change":"2024-05-21T21:51:12.860602+0000","last_active":"2024-05-21T21:55:36.496010+0000","last_peered":"2024-05-21T21:55:36.496010+0000","last_clean":"2024-05-21T21:55:36.496010+0000","last_became_active":"2024-05-21T21:51:12.860228+0000","last_became_peered":"2024-05-21T21:51:12.860228+0000","last_unstale":"2024-05-21T21:55:36.496010+0000","last_undegraded":"2024-05-21T21:55:36.496010+0000","last_fullsized":"2024-05-21T21:55:36.496010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:33:39.230872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.551036+0000","last_change":"2024-05-21T21:51:16.929616+0000","last_active":"2024-05-21T21:55:36.551036+0000","last_peered":"2024-05-21T21:55:36.551036+0000","last_clean":"2024-05-21T21:55:36.551036+0000","last_became_active":"2024-05-21T21:51:16.928569+0000","last_became_peered":"2024-05-21T21:51:16.928569+0000","last_unstale":"2024-05-21T21:55:36.551036+0000","last_undegraded":"2024-05-21T21:55:36.551036+0000","last_fullsized":"2024-05-21T21:55:36.551036+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:15:41.790344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'5","reported_seq":46,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.555499+0000","last_change":"2024-05-21T21:52:10.881114+0000","last_active":"2024-05-21T21:55:36.555499+0000","last_peered":"2024-05-21T21:55:36.555499+0000","last_clean":"2024-05-21T21:55:36.555499+0000","last_became_active":"2024-05-21T21:52:10.872022+0000","last_became_peered":"2024-05-21T21:52:10.872022+0000","last_unstale":"2024-05-21T21:55:36.555499+0000","last_undegraded":"2024-05-21T21:55:36.555499+0000","last_fullsized":"2024-05-21T21:55:36.555499+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T06:05:51.419156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.550881+0000","last_change":"2024-05-21T21:51:16.926496+0000","last_active":"2024-05-21T21:55:36.550881+0000","last_peered":"2024-05-21T21:55:36.550881+0000","last_clean":"2024-05-21T21:55:36.550881+0000","last_became_active":"2024-05-21T21:51:16.926112+0000","last_became_peered":"2024-05-21T21:51:16.926112+0000","last_unstale":"2024-05-21T21:55:36.550881+0000","last_undegraded":"2024-05-21T21:55:36.550881+0000","last_fullsized":"2024-05-21T21:55:36.550881+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:26:08.254421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"82'9","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.341047+0000","last_change":"2024-05-21T21:51:16.930522+0000","last_active":"2024-05-21T21:55:36.341047+0000","last_peered":"2024-05-21T21:55:36.341047+0000","last_clean":"2024-05-21T21:55:36.341047+0000","last_became_active":"2024-05-21T21:51:16.929853+0000","last_became_peered":"2024-05-21T21:51:16.929853+0000","last_unstale":"2024-05-21T21:55:36.341047+0000","last_undegraded":"2024-05-21T21:55:36.341047+0000","last_fullsized":"2024-05-21T21:55:36.341047+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:10:57.940009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"82'12","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.201162+0000","last_change":"2024-05-21T21:51:16.931751+0000","last_active":"2024-05-21T21:55:36.201162+0000","last_peered":"2024-05-21T21:55:36.201162+0000","last_clean":"2024-05-21T21:55:36.201162+0000","last_became_active":"2024-05-21T21:51:16.931353+0000","last_became_peered":"2024-05-21T21:51:16.931353+0000","last_unstale":"2024-05-21T21:55:36.201162+0000","last_undegraded":"2024-05-21T21:55:36.201162+0000","last_fullsized":"2024-05-21T21:55:36.201162+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:33:45.297850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"82'8","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.201127+0000","last_change":"2024-05-21T21:51:16.940096+0000","last_active":"2024-05-21T21:55:36.201127+0000","last_peered":"2024-05-21T21:55:36.201127+0000","last_clean":"2024-05-21T21:55:36.201127+0000","last_became_active":"2024-05-21T21:51:16.939740+0000","last_became_peered":"2024-05-21T21:51:16.939740+0000","last_unstale":"2024-05-21T21:55:36.201127+0000","last_undegraded":"2024-05-21T21:55:36.201127+0000","last_fullsized":"2024-05-21T21:55:36.201127+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:59:52.569526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"82'7","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.552155+0000","last_change":"2024-05-21T21:51:16.946488+0000","last_active":"2024-05-21T21:55:36.552155+0000","last_peered":"2024-05-21T21:55:36.552155+0000","last_clean":"2024-05-21T21:55:36.552155+0000","last_became_active":"2024-05-21T21:51:16.945957+0000","last_became_peered":"2024-05-21T21:51:16.945957+0000","last_unstale":"2024-05-21T21:55:36.552155+0000","last_undegraded":"2024-05-21T21:55:36.552155+0000","last_fullsized":"2024-05-21T21:55:36.552155+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:40:36.708344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"82'4","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.556177+0000","last_change":"2024-05-21T21:51:16.949349+0000","last_active":"2024-05-21T21:55:36.556177+0000","last_peered":"2024-05-21T21:55:36.556177+0000","last_clean":"2024-05-21T21:55:36.556177+0000","last_became_active":"2024-05-21T21:51:16.946986+0000","last_became_peered":"2024-05-21T21:51:16.946986+0000","last_unstale":"2024-05-21T21:55:36.556177+0000","last_undegraded":"2024-05-21T21:55:36.556177+0000","last_fullsized":"2024-05-21T21:55:36.556177+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:01:14.877110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"82'6","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.341365+0000","last_change":"2024-05-21T21:51:16.929172+0000","last_active":"2024-05-21T21:55:36.341365+0000","last_peered":"2024-05-21T21:55:36.341365+0000","last_clean":"2024-05-21T21:55:36.341365+0000","last_became_active":"2024-05-21T21:51:16.928823+0000","last_became_peered":"2024-05-21T21:51:16.928823+0000","last_unstale":"2024-05-21T21:55:36.341365+0000","last_undegraded":"2024-05-21T21:55:36.341365+0000","last_fullsized":"2024-05-21T21:55:36.341365+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:09:13.719272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"82'15","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.201467+0000","last_change":"2024-05-21T21:51:16.930332+0000","last_active":"2024-05-21T21:55:36.201467+0000","last_peered":"2024-05-21T21:55:36.201467+0000","last_clean":"2024-05-21T21:55:36.201467+0000","last_became_active":"2024-05-21T21:51:16.930024+0000","last_became_peered":"2024-05-21T21:51:16.930024+0000","last_unstale":"2024-05-21T21:55:36.201467+0000","last_undegraded":"2024-05-21T21:55:36.201467+0000","last_fullsized":"2024-05-21T21:55:36.201467+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:47:41.272725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"82'2","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.343903+0000","last_change":"2024-05-21T21:51:16.930102+0000","last_active":"2024-05-21T21:55:36.343903+0000","last_peered":"2024-05-21T21:55:36.343903+0000","last_clean":"2024-05-21T21:55:36.343903+0000","last_became_active":"2024-05-21T21:51:16.929410+0000","last_became_peered":"2024-05-21T21:51:16.929410+0000","last_unstale":"2024-05-21T21:55:36.343903+0000","last_undegraded":"2024-05-21T21:55:36.343903+0000","last_fullsized":"2024-05-21T21:55:36.343903+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:11:46.296288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"82'6","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.556235+0000","last_change":"2024-05-21T21:52:10.873424+0000","last_active":"2024-05-21T21:55:36.556235+0000","last_peered":"2024-05-21T21:55:36.556235+0000","last_clean":"2024-05-21T21:55:36.556235+0000","last_became_active":"2024-05-21T21:52:10.873118+0000","last_became_peered":"2024-05-21T21:52:10.873118+0000","last_unstale":"2024-05-21T21:55:36.556235+0000","last_undegraded":"2024-05-21T21:55:36.556235+0000","last_fullsized":"2024-05-21T21:55:36.556235+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:05:00.731013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,3],"acting":[8,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.29","version":"82'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.342603+0000","last_change":"2024-05-21T21:51:16.934442+0000","last_active":"2024-05-21T21:55:36.342603+0000","last_peered":"2024-05-21T21:55:36.342603+0000","last_clean":"2024-05-21T21:55:36.342603+0000","last_became_active":"2024-05-21T21:51:16.934086+0000","last_became_peered":"2024-05-21T21:51:16.934086+0000","last_unstale":"2024-05-21T21:55:36.342603+0000","last_undegraded":"2024-05-21T21:55:36.342603+0000","last_fullsized":"2024-05-21T21:55:36.342603+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:36:44.302532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"82'10","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.199405+0000","last_change":"2024-05-21T21:51:16.924479+0000","last_active":"2024-05-21T21:55:36.199405+0000","last_peered":"2024-05-21T21:55:36.199405+0000","last_clean":"2024-05-21T21:55:36.199405+0000","last_became_active":"2024-05-21T21:51:16.923998+0000","last_became_peered":"2024-05-21T21:51:16.923998+0000","last_unstale":"2024-05-21T21:55:36.199405+0000","last_undegraded":"2024-05-21T21:55:36.199405+0000","last_fullsized":"2024-05-21T21:55:36.199405+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:23:07.931731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"82'11","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.555801+0000","last_change":"2024-05-21T21:51:16.947271+0000","last_active":"2024-05-21T21:55:36.555801+0000","last_peered":"2024-05-21T21:55:36.555801+0000","last_clean":"2024-05-21T21:55:36.555801+0000","last_became_active":"2024-05-21T21:51:16.946912+0000","last_became_peered":"2024-05-21T21:51:16.946912+0000","last_unstale":"2024-05-21T21:55:36.555801+0000","last_undegraded":"2024-05-21T21:55:36.555801+0000","last_fullsized":"2024-05-21T21:55:36.555801+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:51:29.646915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"82'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.341021+0000","last_change":"2024-05-21T21:51:16.928852+0000","last_active":"2024-05-21T21:55:36.341021+0000","last_peered":"2024-05-21T21:55:36.341021+0000","last_clean":"2024-05-21T21:55:36.341021+0000","last_became_active":"2024-05-21T21:51:16.928480+0000","last_became_peered":"2024-05-21T21:51:16.928480+0000","last_unstale":"2024-05-21T21:55:36.341021+0000","last_undegraded":"2024-05-21T21:55:36.341021+0000","last_fullsized":"2024-05-21T21:55:36.341021+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T09:12:30.691254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":600652,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":2521,"num_read_kb":1376,"num_write":377,"num_write_kb":1467,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":112,"num_keys_recovered":17,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2174976,"data_stored":1801956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":304,"ondisk_log_size":304,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645381,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":901120,"data_stored":433959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841609,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28736,"kb_used_data":1720,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716416,"statfs":{"total":95995035648,"available":95965609984,"internally_reserved":0,"allocated":1761280,"data_stored":1232552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005134,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":946176,"data_stored":442962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201365,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":921600,"data_stored":434267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397593,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28500,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716652,"statfs":{"total":95995035648,"available":95965851648,"internally_reserved":0,"allocated":1519616,"data_stored":1027768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561119,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":925696,"data_stored":442953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574971,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1003520,"data_stored":510454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738496,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36264,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708888,"statfs":{"total":95995035648,"available":95957901312,"internally_reserved":0,"allocated":946176,"data_stored":444683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411447,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36324,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708828,"statfs":{"total":95995035648,"available":95957839872,"internally_reserved":0,"allocated":1007616,"data_stored":512074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149807,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36236,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708916,"statfs":{"total":95995035648,"available":95957929984,"internally_reserved":0,"allocated":921600,"data_stored":433314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953578,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":925696,"data_stored":442955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757349,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32672,"kb_used_data":1496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712480,"statfs":{"total":95995035648,"available":95961579520,"internally_reserved":0,"allocated":1531904,"data_stored":1026914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":68731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":70351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":557056,"data_stored":525246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":135653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":258048,"data_stored":200344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":395190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-21T21:55:42.580 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-05-21T21:55:42.581 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-21T21:55:42.805 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.804+0000 7fcd63e00640 1 -- 172.21.15.151:0/1735666331 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcd64075430 msgr2=0x7fcd64075890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:42.805 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.804+0000 7fcd63e00640 1 --2- 172.21.15.151:0/1735666331 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcd64075430 0x7fcd64075890 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fcd50009900 tx=0x7fcd5002a480 comp rx=0 tx=0).stop 2024-05-21T21:55:42.805 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.804+0000 7fcd63e00640 1 -- 172.21.15.151:0/1735666331 shutdown_connections 2024-05-21T21:55:42.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.804+0000 7fcd63e00640 1 --2- 172.21.15.151:0/1735666331 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd6410a7a0 0x7fcd6410cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:42.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.804+0000 7fcd63e00640 1 --2- 172.21.15.151:0/1735666331 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcd64075430 0x7fcd64075890 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:42.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.804+0000 7fcd63e00640 1 --2- 172.21.15.151:0/1735666331 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd64076a10 0x7fcd64074e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:42.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.804+0000 7fcd63e00640 1 -- 172.21.15.151:0/1735666331 >> 172.21.15.151:0/1735666331 conn(0x7fcd640fde60 msgr2=0x7fcd641002a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:55:42.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.804+0000 7fcd63e00640 1 -- 172.21.15.151:0/1735666331 shutdown_connections 2024-05-21T21:55:42.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.804+0000 7fcd63e00640 1 -- 172.21.15.151:0/1735666331 wait complete. 2024-05-21T21:55:42.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.805+0000 7fcd63e00640 1 Processor -- start 2024-05-21T21:55:42.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.805+0000 7fcd63e00640 1 -- start start 2024-05-21T21:55:42.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.805+0000 7fcd63e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd64075430 0x7fcd6419ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:42.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.805+0000 7fcd63e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcd64076a10 0x7fcd6419b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:42.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.805+0000 7fcd63e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd6410a7a0 0x7fcd641a22f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:42.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.805+0000 7fcd63e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcd64110680 con 0x7fcd64076a10 2024-05-21T21:55:42.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.805+0000 7fcd63e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcd64110500 con 0x7fcd6410a7a0 2024-05-21T21:55:42.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.805+0000 7fcd63e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fcd64110800 con 0x7fcd64075430 2024-05-21T21:55:42.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.806+0000 7fcd62000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcd64076a10 0x7fcd6419b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:42.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.806+0000 7fcd63400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd6410a7a0 0x7fcd641a22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:42.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.806+0000 7fcd62a00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd64075430 0x7fcd6419ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:42.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.806+0000 7fcd63400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd6410a7a0 0x7fcd641a22f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:33378/0 (socket says 172.21.15.151:33378) 2024-05-21T21:55:42.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.806+0000 7fcd62000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcd64076a10 0x7fcd6419b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:40796/0 (socket says 172.21.15.151:40796) 2024-05-21T21:55:42.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.806+0000 7fcd63400640 1 -- 172.21.15.151:0/1556269076 learned_addr learned my addr 172.21.15.151:0/1556269076 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:55:42.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.806+0000 7fcd63400640 1 -- 172.21.15.151:0/1556269076 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd64075430 msgr2=0x7fcd6419ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:42.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.806+0000 7fcd63400640 1 --2- 172.21.15.151:0/1556269076 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd64075430 0x7fcd6419ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:42.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.806+0000 7fcd63400640 1 -- 172.21.15.151:0/1556269076 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcd64076a10 msgr2=0x7fcd6419b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:42.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.806+0000 7fcd63400640 1 --2- 172.21.15.151:0/1556269076 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcd64076a10 0x7fcd6419b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:42.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.806+0000 7fcd63400640 1 -- 172.21.15.151:0/1556269076 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd641a2960 con 0x7fcd6410a7a0 2024-05-21T21:55:42.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.806+0000 7fcd62a00640 1 --2- 172.21.15.151:0/1556269076 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd64075430 0x7fcd6419ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:55:42.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.807+0000 7fcd63400640 1 --2- 172.21.15.151:0/1556269076 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd6410a7a0 0x7fcd641a22f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c00eb10 tx=0x7fcd5c00c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:55:42.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.807+0000 7fcd5b400640 1 -- 172.21.15.151:0/1556269076 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd5c004b20 con 0x7fcd6410a7a0 2024-05-21T21:55:42.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.807+0000 7fcd5b400640 1 -- 172.21.15.151:0/1556269076 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd5c004cc0 con 0x7fcd6410a7a0 2024-05-21T21:55:42.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.807+0000 7fcd5b400640 1 -- 172.21.15.151:0/1556269076 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd5c010670 con 0x7fcd6410a7a0 2024-05-21T21:55:42.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.807+0000 7fcd63e00640 1 -- 172.21.15.151:0/1556269076 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd641a2c50 con 0x7fcd6410a7a0 2024-05-21T21:55:42.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.807+0000 7fcd63e00640 1 -- 172.21.15.151:0/1556269076 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd641a31c0 con 0x7fcd6410a7a0 2024-05-21T21:55:42.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.809+0000 7fcd63e00640 1 -- 172.21.15.151:0/1556269076 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd64069790 con 0x7fcd6410a7a0 2024-05-21T21:55:42.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.810+0000 7fcd5b400640 1 -- 172.21.15.151:0/1556269076 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 149829+0+0 (secure 0 0 0) 0x7fcd5c005020 con 0x7fcd6410a7a0 2024-05-21T21:55:42.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.812+0000 7fcd5b400640 1 --2- 172.21.15.151:0/1556269076 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fcd480b2a70 0x7fcd480b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:42.813 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.812+0000 7fcd5b400640 1 -- 172.21.15.151:0/1556269076 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fcd5c0e2c80 con 0x7fcd6410a7a0 2024-05-21T21:55:42.813 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.812+0000 7fcd62a00640 1 --2- 172.21.15.151:0/1556269076 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fcd480b2a70 0x7fcd480b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:42.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.813+0000 7fcd62a00640 1 --2- 172.21.15.151:0/1556269076 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fcd480b2a70 0x7fcd480b4f30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fcd54009fc0 tx=0x7fcd54005f90 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:55:42.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:42.816+0000 7fcd5b400640 1 -- 172.21.15.151:0/1556269076 <== mon.2 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcd5c0aa500 con 0x7fcd6410a7a0 2024-05-21T21:55:43.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:43.020+0000 7fcd63e00640 1 -- 172.21.15.151:0/1556269076 --> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcd64062e90 con 0x7fcd480b2a70 2024-05-21T21:55:43.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:43.029+0000 7fcd5b400640 1 -- 172.21.15.151:0/1556269076 <== mgr.25138 v2:172.21.15.151:6800/958417056 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257822 (secure 0 0 0) 0x7fcd64062e90 con 0x7fcd480b2a70 2024-05-21T21:55:43.033 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T21:55:43.034 INFO:teuthology.orchestra.run.smithi151.stderr:dumped all 2024-05-21T21:55:43.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:43.033+0000 7fcd63e00640 1 -- 172.21.15.151:0/1556269076 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fcd480b2a70 msgr2=0x7fcd480b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:43.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:43.033+0000 7fcd63e00640 1 --2- 172.21.15.151:0/1556269076 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fcd480b2a70 0x7fcd480b4f30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fcd54009fc0 tx=0x7fcd54005f90 comp rx=0 tx=0).stop 2024-05-21T21:55:43.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:43.033+0000 7fcd63e00640 1 -- 172.21.15.151:0/1556269076 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd6410a7a0 msgr2=0x7fcd641a22f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:43.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:43.033+0000 7fcd63e00640 1 --2- 172.21.15.151:0/1556269076 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd6410a7a0 0x7fcd641a22f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c00eb10 tx=0x7fcd5c00c2d0 comp rx=0 tx=0).stop 2024-05-21T21:55:43.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:43.033+0000 7fcd63e00640 1 -- 172.21.15.151:0/1556269076 shutdown_connections 2024-05-21T21:55:43.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:43.033+0000 7fcd63e00640 1 --2- 172.21.15.151:0/1556269076 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fcd480b2a70 0x7fcd480b4f30 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:43.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:43.033+0000 7fcd63e00640 1 --2- 172.21.15.151:0/1556269076 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd6410a7a0 0x7fcd641a22f0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:43.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:43.033+0000 7fcd63e00640 1 --2- 172.21.15.151:0/1556269076 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcd64076a10 0x7fcd6419b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:43.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:43.033+0000 7fcd63e00640 1 --2- 172.21.15.151:0/1556269076 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd64075430 0x7fcd6419ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:43.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:43.033+0000 7fcd63e00640 1 -- 172.21.15.151:0/1556269076 >> 172.21.15.151:0/1556269076 conn(0x7fcd640fde60 msgr2=0x7fcd64106730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:55:43.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:43.033+0000 7fcd63e00640 1 -- 172.21.15.151:0/1556269076 shutdown_connections 2024-05-21T21:55:43.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:43.033+0000 7fcd63e00640 1 -- 172.21.15.151:0/1556269076 wait complete. 2024-05-21T21:55:43.052 INFO:teuthology.orchestra.run.smithi151.stdout:{"pg_ready":true,"pg_map":{"version":6,"stamp":"2024-05-21T21:55:42.220097+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1191254,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":2602,"num_read_kb":1445,"num_write":502,"num_write_kb":3619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":2952069,"num_keys_recovered":17,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":898,"ondisk_log_size":898,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":366336,"kb_used_data":13000,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124575488,"statfs":{"total":1151940427776,"available":1151565299712,"internally_reserved":0,"allocated":13312000,"data_stored":7384855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19140,"internal_metadata":361674044},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.978854"},"pg_stats":[{"pgid":"3.3f","version":"82'8","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.341334+0000","last_change":"2024-05-21T21:51:16.945132+0000","last_active":"2024-05-21T21:55:36.341334+0000","last_peered":"2024-05-21T21:55:36.341334+0000","last_clean":"2024-05-21T21:55:36.341334+0000","last_became_active":"2024-05-21T21:51:16.944766+0000","last_became_peered":"2024-05-21T21:51:16.944766+0000","last_unstale":"2024-05-21T21:55:36.341334+0000","last_undegraded":"2024-05-21T21:55:36.341334+0000","last_fullsized":"2024-05-21T21:55:36.341334+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:54:04.185095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"82'4","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.553624+0000","last_change":"2024-05-21T21:51:16.939291+0000","last_active":"2024-05-21T21:55:36.553624+0000","last_peered":"2024-05-21T21:55:36.553624+0000","last_clean":"2024-05-21T21:55:36.553624+0000","last_became_active":"2024-05-21T21:51:16.938730+0000","last_became_peered":"2024-05-21T21:51:16.938730+0000","last_unstale":"2024-05-21T21:55:36.553624+0000","last_undegraded":"2024-05-21T21:55:36.553624+0000","last_fullsized":"2024-05-21T21:55:36.553624+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:05:30.623744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"82'7","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.553252+0000","last_change":"2024-05-21T21:51:16.941565+0000","last_active":"2024-05-21T21:55:36.553252+0000","last_peered":"2024-05-21T21:55:36.553252+0000","last_clean":"2024-05-21T21:55:36.553252+0000","last_became_active":"2024-05-21T21:51:16.941033+0000","last_became_peered":"2024-05-21T21:51:16.941033+0000","last_unstale":"2024-05-21T21:55:36.553252+0000","last_undegraded":"2024-05-21T21:55:36.553252+0000","last_fullsized":"2024-05-21T21:55:36.553252+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:04:55.789479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"82'8","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.723315+0000","last_change":"2024-05-21T21:51:16.928455+0000","last_active":"2024-05-21T21:55:36.723315+0000","last_peered":"2024-05-21T21:55:36.723315+0000","last_clean":"2024-05-21T21:55:36.723315+0000","last_became_active":"2024-05-21T21:51:16.927874+0000","last_became_peered":"2024-05-21T21:51:16.927874+0000","last_unstale":"2024-05-21T21:55:36.723315+0000","last_undegraded":"2024-05-21T21:55:36.723315+0000","last_fullsized":"2024-05-21T21:55:36.723315+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:25:16.483175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"82'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.556362+0000","last_change":"2024-05-21T21:51:16.937773+0000","last_active":"2024-05-21T21:55:36.556362+0000","last_peered":"2024-05-21T21:55:36.556362+0000","last_clean":"2024-05-21T21:55:36.556362+0000","last_became_active":"2024-05-21T21:51:16.937306+0000","last_became_peered":"2024-05-21T21:51:16.937306+0000","last_unstale":"2024-05-21T21:55:36.556362+0000","last_undegraded":"2024-05-21T21:55:36.556362+0000","last_fullsized":"2024-05-21T21:55:36.556362+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:05:03.762390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"82'6","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.356010+0000","last_change":"2024-05-21T21:51:16.948275+0000","last_active":"2024-05-21T21:55:36.356010+0000","last_peered":"2024-05-21T21:55:36.356010+0000","last_clean":"2024-05-21T21:55:36.356010+0000","last_became_active":"2024-05-21T21:51:16.947890+0000","last_became_peered":"2024-05-21T21:51:16.947890+0000","last_unstale":"2024-05-21T21:55:36.356010+0000","last_undegraded":"2024-05-21T21:55:36.356010+0000","last_fullsized":"2024-05-21T21:55:36.356010+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:08:38.572177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"82'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.550575+0000","last_change":"2024-05-21T21:51:16.946821+0000","last_active":"2024-05-21T21:55:36.550575+0000","last_peered":"2024-05-21T21:55:36.550575+0000","last_clean":"2024-05-21T21:55:36.550575+0000","last_became_active":"2024-05-21T21:51:16.946515+0000","last_became_peered":"2024-05-21T21:51:16.946515+0000","last_unstale":"2024-05-21T21:55:36.550575+0000","last_undegraded":"2024-05-21T21:55:36.550575+0000","last_fullsized":"2024-05-21T21:55:36.550575+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T09:26:25.867895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.342252+0000","last_change":"2024-05-21T21:51:16.929228+0000","last_active":"2024-05-21T21:55:36.342252+0000","last_peered":"2024-05-21T21:55:36.342252+0000","last_clean":"2024-05-21T21:55:36.342252+0000","last_became_active":"2024-05-21T21:51:16.928875+0000","last_became_peered":"2024-05-21T21:51:16.928875+0000","last_unstale":"2024-05-21T21:55:36.342252+0000","last_undegraded":"2024-05-21T21:55:36.342252+0000","last_fullsized":"2024-05-21T21:55:36.342252+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:50:56.431310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"82'11","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.199749+0000","last_change":"2024-05-21T21:51:16.932360+0000","last_active":"2024-05-21T21:55:36.199749+0000","last_peered":"2024-05-21T21:55:36.199749+0000","last_clean":"2024-05-21T21:55:36.199749+0000","last_became_active":"2024-05-21T21:51:16.932036+0000","last_became_peered":"2024-05-21T21:51:16.932036+0000","last_unstale":"2024-05-21T21:55:36.199749+0000","last_undegraded":"2024-05-21T21:55:36.199749+0000","last_fullsized":"2024-05-21T21:55:36.199749+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T09:06:37.257323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"82'16","reported_seq":45,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.356382+0000","last_change":"2024-05-21T21:52:10.876556+0000","last_active":"2024-05-21T21:55:36.356382+0000","last_peered":"2024-05-21T21:55:36.356382+0000","last_clean":"2024-05-21T21:55:36.356382+0000","last_became_active":"2024-05-21T21:52:10.876273+0000","last_became_peered":"2024-05-21T21:52:10.876273+0000","last_unstale":"2024-05-21T21:55:36.356382+0000","last_undegraded":"2024-05-21T21:55:36.356382+0000","last_fullsized":"2024-05-21T21:55:36.356382+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T05:09:40.037731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"82'6","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.550761+0000","last_change":"2024-05-21T21:51:16.933425+0000","last_active":"2024-05-21T21:55:36.550761+0000","last_peered":"2024-05-21T21:55:36.550761+0000","last_clean":"2024-05-21T21:55:36.550761+0000","last_became_active":"2024-05-21T21:51:16.933057+0000","last_became_peered":"2024-05-21T21:51:16.933057+0000","last_unstale":"2024-05-21T21:55:36.550761+0000","last_undegraded":"2024-05-21T21:55:36.550761+0000","last_fullsized":"2024-05-21T21:55:36.550761+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:04:57.237841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"82'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.723313+0000","last_change":"2024-05-21T21:51:16.927014+0000","last_active":"2024-05-21T21:55:36.723313+0000","last_peered":"2024-05-21T21:55:36.723313+0000","last_clean":"2024-05-21T21:55:36.723313+0000","last_became_active":"2024-05-21T21:51:16.925954+0000","last_became_peered":"2024-05-21T21:51:16.925954+0000","last_unstale":"2024-05-21T21:55:36.723313+0000","last_undegraded":"2024-05-21T21:55:36.723313+0000","last_fullsized":"2024-05-21T21:55:36.723313+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:19:20.530727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"82'10","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.555649+0000","last_change":"2024-05-21T21:51:16.930909+0000","last_active":"2024-05-21T21:55:36.555649+0000","last_peered":"2024-05-21T21:55:36.555649+0000","last_clean":"2024-05-21T21:55:36.555649+0000","last_became_active":"2024-05-21T21:51:16.927110+0000","last_became_peered":"2024-05-21T21:51:16.927110+0000","last_unstale":"2024-05-21T21:55:36.555649+0000","last_undegraded":"2024-05-21T21:55:36.555649+0000","last_fullsized":"2024-05-21T21:55:36.555649+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T06:23:55.386047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"82'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.552564+0000","last_change":"2024-05-21T21:51:16.938797+0000","last_active":"2024-05-21T21:55:36.552564+0000","last_peered":"2024-05-21T21:55:36.552564+0000","last_clean":"2024-05-21T21:55:36.552564+0000","last_became_active":"2024-05-21T21:51:16.938263+0000","last_became_peered":"2024-05-21T21:51:16.938263+0000","last_unstale":"2024-05-21T21:55:36.552564+0000","last_undegraded":"2024-05-21T21:55:36.552564+0000","last_fullsized":"2024-05-21T21:55:36.552564+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:37:40.521118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"82'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.552309+0000","last_change":"2024-05-21T21:51:16.944160+0000","last_active":"2024-05-21T21:55:36.552309+0000","last_peered":"2024-05-21T21:55:36.552309+0000","last_clean":"2024-05-21T21:55:36.552309+0000","last_became_active":"2024-05-21T21:51:16.940046+0000","last_became_peered":"2024-05-21T21:51:16.940046+0000","last_unstale":"2024-05-21T21:55:36.552309+0000","last_undegraded":"2024-05-21T21:55:36.552309+0000","last_fullsized":"2024-05-21T21:55:36.552309+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:14:13.278799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.198999+0000","last_change":"2024-05-21T21:51:16.945599+0000","last_active":"2024-05-21T21:55:36.198999+0000","last_peered":"2024-05-21T21:55:36.198999+0000","last_clean":"2024-05-21T21:55:36.198999+0000","last_became_active":"2024-05-21T21:51:16.945196+0000","last_became_peered":"2024-05-21T21:51:16.945196+0000","last_unstale":"2024-05-21T21:55:36.198999+0000","last_undegraded":"2024-05-21T21:55:36.198999+0000","last_fullsized":"2024-05-21T21:55:36.198999+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:59:24.453615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"82'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.341504+0000","last_change":"2024-05-21T21:51:16.944895+0000","last_active":"2024-05-21T21:55:36.341504+0000","last_peered":"2024-05-21T21:55:36.341504+0000","last_clean":"2024-05-21T21:55:36.341504+0000","last_became_active":"2024-05-21T21:51:16.944560+0000","last_became_peered":"2024-05-21T21:51:16.944560+0000","last_unstale":"2024-05-21T21:55:36.341504+0000","last_undegraded":"2024-05-21T21:55:36.341504+0000","last_fullsized":"2024-05-21T21:55:36.341504+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:53:13.264701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"82'10","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.197858+0000","last_change":"2024-05-21T21:51:16.943861+0000","last_active":"2024-05-21T21:55:36.197858+0000","last_peered":"2024-05-21T21:55:36.197858+0000","last_clean":"2024-05-21T21:55:36.197858+0000","last_became_active":"2024-05-21T21:51:16.943123+0000","last_became_peered":"2024-05-21T21:51:16.943123+0000","last_unstale":"2024-05-21T21:55:36.197858+0000","last_undegraded":"2024-05-21T21:55:36.197858+0000","last_fullsized":"2024-05-21T21:55:36.197858+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:35:37.732855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"82'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.201422+0000","last_change":"2024-05-21T21:51:16.940580+0000","last_active":"2024-05-21T21:55:36.201422+0000","last_peered":"2024-05-21T21:55:36.201422+0000","last_clean":"2024-05-21T21:55:36.201422+0000","last_became_active":"2024-05-21T21:51:16.940264+0000","last_became_peered":"2024-05-21T21:51:16.940264+0000","last_unstale":"2024-05-21T21:55:36.201422+0000","last_undegraded":"2024-05-21T21:55:36.201422+0000","last_fullsized":"2024-05-21T21:55:36.201422+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:54:22.915541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'14","reported_seq":283,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.557627+0000","last_change":"2024-05-21T21:51:12.852935+0000","last_active":"2024-05-21T21:55:36.557627+0000","last_peered":"2024-05-21T21:55:36.557627+0000","last_clean":"2024-05-21T21:55:36.557627+0000","last_became_active":"2024-05-21T21:51:12.851802+0000","last_became_peered":"2024-05-21T21:51:12.851802+0000","last_unstale":"2024-05-21T21:55:36.557627+0000","last_undegraded":"2024-05-21T21:55:36.557627+0000","last_fullsized":"2024-05-21T21:55:36.557627+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:13:24.643408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":239,"num_read_kb":239,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"82'5","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.343135+0000","last_change":"2024-05-21T21:51:16.939461+0000","last_active":"2024-05-21T21:55:36.343135+0000","last_peered":"2024-05-21T21:55:36.343135+0000","last_clean":"2024-05-21T21:55:36.343135+0000","last_became_active":"2024-05-21T21:51:16.936322+0000","last_became_peered":"2024-05-21T21:51:16.936322+0000","last_unstale":"2024-05-21T21:55:36.343135+0000","last_undegraded":"2024-05-21T21:55:36.343135+0000","last_fullsized":"2024-05-21T21:55:36.343135+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:37:26.153963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.343791+0000","last_change":"2024-05-21T21:51:12.941838+0000","last_active":"2024-05-21T21:55:36.343791+0000","last_peered":"2024-05-21T21:55:36.343791+0000","last_clean":"2024-05-21T21:55:36.343791+0000","last_became_active":"2024-05-21T21:51:12.940818+0000","last_became_peered":"2024-05-21T21:51:12.940818+0000","last_unstale":"2024-05-21T21:55:36.343791+0000","last_undegraded":"2024-05-21T21:55:36.343791+0000","last_fullsized":"2024-05-21T21:55:36.343791+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:30:42.349241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"82'6","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.554128+0000","last_change":"2024-05-21T21:51:16.950252+0000","last_active":"2024-05-21T21:55:36.554128+0000","last_peered":"2024-05-21T21:55:36.554128+0000","last_clean":"2024-05-21T21:55:36.554128+0000","last_became_active":"2024-05-21T21:51:16.947375+0000","last_became_peered":"2024-05-21T21:51:16.947375+0000","last_unstale":"2024-05-21T21:55:36.554128+0000","last_undegraded":"2024-05-21T21:55:36.554128+0000","last_fullsized":"2024-05-21T21:55:36.554128+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:27:30.462799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"80'12","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.341317+0000","last_change":"2024-05-21T21:52:10.875739+0000","last_active":"2024-05-21T21:55:36.341317+0000","last_peered":"2024-05-21T21:55:36.341317+0000","last_clean":"2024-05-21T21:55:36.341317+0000","last_became_active":"2024-05-21T21:52:10.865846+0000","last_became_peered":"2024-05-21T21:52:10.865846+0000","last_unstale":"2024-05-21T21:55:36.341317+0000","last_undegraded":"2024-05-21T21:55:36.341317+0000","last_fullsized":"2024-05-21T21:55:36.341317+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T06:48:06.465757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.199107+0000","last_change":"2024-05-21T21:51:16.926380+0000","last_active":"2024-05-21T21:55:36.199107+0000","last_peered":"2024-05-21T21:55:36.199107+0000","last_clean":"2024-05-21T21:55:36.199107+0000","last_became_active":"2024-05-21T21:51:16.925821+0000","last_became_peered":"2024-05-21T21:51:16.925821+0000","last_unstale":"2024-05-21T21:55:36.199107+0000","last_undegraded":"2024-05-21T21:55:36.199107+0000","last_fullsized":"2024-05-21T21:55:36.199107+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:42:14.887155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'5","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.556613+0000","last_change":"2024-05-21T21:51:12.854342+0000","last_active":"2024-05-21T21:55:36.556613+0000","last_peered":"2024-05-21T21:55:36.556613+0000","last_clean":"2024-05-21T21:55:36.556613+0000","last_became_active":"2024-05-21T21:51:12.853908+0000","last_became_peered":"2024-05-21T21:51:12.853908+0000","last_unstale":"2024-05-21T21:55:36.556613+0000","last_undegraded":"2024-05-21T21:55:36.556613+0000","last_fullsized":"2024-05-21T21:55:36.556613+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:35:10.374749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"82'15","reported_seq":44,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.352781+0000","last_change":"2024-05-21T21:52:10.973816+0000","last_active":"2024-05-21T21:55:36.352781+0000","last_peered":"2024-05-21T21:55:36.352781+0000","last_clean":"2024-05-21T21:55:36.352781+0000","last_became_active":"2024-05-21T21:52:10.973457+0000","last_became_peered":"2024-05-21T21:52:10.973457+0000","last_unstale":"2024-05-21T21:55:36.352781+0000","last_undegraded":"2024-05-21T21:55:36.352781+0000","last_fullsized":"2024-05-21T21:55:36.352781+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T05:01:41.915158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"83'9","reported_seq":282,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.875854+0000","last_change":"2024-05-21T21:51:12.855658+0000","last_active":"2024-05-21T21:55:36.875854+0000","last_peered":"2024-05-21T21:55:36.875854+0000","last_clean":"2024-05-21T21:55:36.875854+0000","last_became_active":"2024-05-21T21:51:12.855281+0000","last_became_peered":"2024-05-21T21:51:12.855281+0000","last_unstale":"2024-05-21T21:55:36.875854+0000","last_undegraded":"2024-05-21T21:55:36.875854+0000","last_fullsized":"2024-05-21T21:55:36.875854+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:55:08.836045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1855,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":244,"num_read_kb":6,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"82'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.556310+0000","last_change":"2024-05-21T21:51:16.946977+0000","last_active":"2024-05-21T21:55:36.556310+0000","last_peered":"2024-05-21T21:55:36.556310+0000","last_clean":"2024-05-21T21:55:36.556310+0000","last_became_active":"2024-05-21T21:51:16.946562+0000","last_became_peered":"2024-05-21T21:51:16.946562+0000","last_unstale":"2024-05-21T21:55:36.556310+0000","last_undegraded":"2024-05-21T21:55:36.556310+0000","last_fullsized":"2024-05-21T21:55:36.556310+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:43:33.370265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"80'7","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.355936+0000","last_change":"2024-05-21T21:51:12.852574+0000","last_active":"2024-05-21T21:55:36.355936+0000","last_peered":"2024-05-21T21:55:36.355936+0000","last_clean":"2024-05-21T21:55:36.355936+0000","last_became_active":"2024-05-21T21:51:12.852258+0000","last_became_peered":"2024-05-21T21:51:12.852258+0000","last_unstale":"2024-05-21T21:55:36.355936+0000","last_undegraded":"2024-05-21T21:55:36.355936+0000","last_fullsized":"2024-05-21T21:55:36.355936+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:09:40.280821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"82'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.352655+0000","last_change":"2024-05-21T21:51:16.934768+0000","last_active":"2024-05-21T21:55:36.352655+0000","last_peered":"2024-05-21T21:55:36.352655+0000","last_clean":"2024-05-21T21:55:36.352655+0000","last_became_active":"2024-05-21T21:51:16.934275+0000","last_became_peered":"2024-05-21T21:51:16.934275+0000","last_unstale":"2024-05-21T21:55:36.352655+0000","last_undegraded":"2024-05-21T21:55:36.352655+0000","last_fullsized":"2024-05-21T21:55:36.352655+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:31:54.425069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"80'3","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.555219+0000","last_change":"2024-05-21T21:51:12.854166+0000","last_active":"2024-05-21T21:55:36.555219+0000","last_peered":"2024-05-21T21:55:36.555219+0000","last_clean":"2024-05-21T21:55:36.555219+0000","last_became_active":"2024-05-21T21:51:12.853508+0000","last_became_peered":"2024-05-21T21:51:12.853508+0000","last_unstale":"2024-05-21T21:55:36.555219+0000","last_undegraded":"2024-05-21T21:55:36.555219+0000","last_fullsized":"2024-05-21T21:55:36.555219+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:29:41.858655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"82'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.342248+0000","last_change":"2024-05-21T21:51:16.938293+0000","last_active":"2024-05-21T21:55:36.342248+0000","last_peered":"2024-05-21T21:55:36.342248+0000","last_clean":"2024-05-21T21:55:36.342248+0000","last_became_active":"2024-05-21T21:51:16.937970+0000","last_became_peered":"2024-05-21T21:51:16.937970+0000","last_unstale":"2024-05-21T21:55:36.342248+0000","last_undegraded":"2024-05-21T21:55:36.342248+0000","last_fullsized":"2024-05-21T21:55:36.342248+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:18:27.473098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'3","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.352476+0000","last_change":"2024-05-21T21:51:12.853366+0000","last_active":"2024-05-21T21:55:36.352476+0000","last_peered":"2024-05-21T21:55:36.352476+0000","last_clean":"2024-05-21T21:55:36.352476+0000","last_became_active":"2024-05-21T21:51:12.853028+0000","last_became_peered":"2024-05-21T21:51:12.853028+0000","last_unstale":"2024-05-21T21:55:36.352476+0000","last_undegraded":"2024-05-21T21:55:36.352476+0000","last_fullsized":"2024-05-21T21:55:36.352476+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T09:15:52.593028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"82'11","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.197181+0000","last_change":"2024-05-21T21:51:16.947903+0000","last_active":"2024-05-21T21:55:36.197181+0000","last_peered":"2024-05-21T21:55:36.197181+0000","last_clean":"2024-05-21T21:55:36.197181+0000","last_became_active":"2024-05-21T21:51:16.946677+0000","last_became_peered":"2024-05-21T21:51:16.946677+0000","last_unstale":"2024-05-21T21:55:36.197181+0000","last_undegraded":"2024-05-21T21:55:36.197181+0000","last_fullsized":"2024-05-21T21:55:36.197181+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T06:12:56.514284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'7","reported_seq":512,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.746589+0000","last_change":"2024-05-21T21:51:12.854873+0000","last_active":"2024-05-21T21:55:36.746589+0000","last_peered":"2024-05-21T21:55:36.746589+0000","last_clean":"2024-05-21T21:55:36.746589+0000","last_became_active":"2024-05-21T21:51:12.854393+0000","last_became_peered":"2024-05-21T21:51:12.854393+0000","last_unstale":"2024-05-21T21:55:36.746589+0000","last_undegraded":"2024-05-21T21:55:36.746589+0000","last_fullsized":"2024-05-21T21:55:36.746589+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:58:52.411164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":476,"num_read_kb":476,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"82'10","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.199587+0000","last_change":"2024-05-21T21:51:16.951094+0000","last_active":"2024-05-21T21:55:36.199587+0000","last_peered":"2024-05-21T21:55:36.199587+0000","last_clean":"2024-05-21T21:55:36.199587+0000","last_became_active":"2024-05-21T21:51:16.950724+0000","last_became_peered":"2024-05-21T21:51:16.950724+0000","last_unstale":"2024-05-21T21:55:36.199587+0000","last_undegraded":"2024-05-21T21:55:36.199587+0000","last_fullsized":"2024-05-21T21:55:36.199587+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:54:02.835426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'4","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.550904+0000","last_change":"2024-05-21T21:51:12.850427+0000","last_active":"2024-05-21T21:55:36.550904+0000","last_peered":"2024-05-21T21:55:36.550904+0000","last_clean":"2024-05-21T21:55:36.550904+0000","last_became_active":"2024-05-21T21:51:12.850096+0000","last_became_peered":"2024-05-21T21:51:12.850096+0000","last_unstale":"2024-05-21T21:55:36.550904+0000","last_undegraded":"2024-05-21T21:55:36.550904+0000","last_fullsized":"2024-05-21T21:55:36.550904+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:43:22.611043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.554216+0000","last_change":"2024-05-21T21:51:16.942330+0000","last_active":"2024-05-21T21:55:36.554216+0000","last_peered":"2024-05-21T21:55:36.554216+0000","last_clean":"2024-05-21T21:55:36.554216+0000","last_became_active":"2024-05-21T21:51:16.941718+0000","last_became_peered":"2024-05-21T21:51:16.941718+0000","last_unstale":"2024-05-21T21:55:36.554216+0000","last_undegraded":"2024-05-21T21:55:36.554216+0000","last_fullsized":"2024-05-21T21:55:36.554216+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:54:17.176411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'5","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.352907+0000","last_change":"2024-05-21T21:51:12.941100+0000","last_active":"2024-05-21T21:55:36.352907+0000","last_peered":"2024-05-21T21:55:36.352907+0000","last_clean":"2024-05-21T21:55:36.352907+0000","last_became_active":"2024-05-21T21:51:12.940597+0000","last_became_peered":"2024-05-21T21:51:12.940597+0000","last_unstale":"2024-05-21T21:55:36.352907+0000","last_undegraded":"2024-05-21T21:55:36.352907+0000","last_fullsized":"2024-05-21T21:55:36.352907+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:30:58.457108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"82'11","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.199324+0000","last_change":"2024-05-21T21:51:16.921973+0000","last_active":"2024-05-21T21:55:36.199324+0000","last_peered":"2024-05-21T21:55:36.199324+0000","last_clean":"2024-05-21T21:55:36.199324+0000","last_became_active":"2024-05-21T21:51:16.921597+0000","last_became_peered":"2024-05-21T21:51:16.921597+0000","last_unstale":"2024-05-21T21:55:36.199324+0000","last_undegraded":"2024-05-21T21:55:36.199324+0000","last_fullsized":"2024-05-21T21:55:36.199324+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:25:32.395940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'39","reported_seq":557,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.747043+0000","last_change":"2024-05-21T21:51:12.850312+0000","last_active":"2024-05-21T21:55:36.747043+0000","last_peered":"2024-05-21T21:55:36.747043+0000","last_clean":"2024-05-21T21:55:36.747043+0000","last_became_active":"2024-05-21T21:51:12.849837+0000","last_became_peered":"2024-05-21T21:51:12.849837+0000","last_unstale":"2024-05-21T21:55:36.747043+0000","last_undegraded":"2024-05-21T21:55:36.747043+0000","last_fullsized":"2024-05-21T21:55:36.747043+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T07:09:13.339554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66476,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":489,"num_read_kb":309,"num_write":41,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"82'6","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.556542+0000","last_change":"2024-05-21T21:51:16.926625+0000","last_active":"2024-05-21T21:55:36.556542+0000","last_peered":"2024-05-21T21:55:36.556542+0000","last_clean":"2024-05-21T21:55:36.556542+0000","last_became_active":"2024-05-21T21:51:16.925889+0000","last_became_peered":"2024-05-21T21:51:16.925889+0000","last_unstale":"2024-05-21T21:55:36.556542+0000","last_undegraded":"2024-05-21T21:55:36.556542+0000","last_fullsized":"2024-05-21T21:55:36.556542+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:35:23.464747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'9","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.201248+0000","last_change":"2024-05-21T21:51:12.859772+0000","last_active":"2024-05-21T21:55:36.201248+0000","last_peered":"2024-05-21T21:55:36.201248+0000","last_clean":"2024-05-21T21:55:36.201248+0000","last_became_active":"2024-05-21T21:51:12.859290+0000","last_became_peered":"2024-05-21T21:51:12.859290+0000","last_unstale":"2024-05-21T21:55:36.201248+0000","last_undegraded":"2024-05-21T21:55:36.201248+0000","last_fullsized":"2024-05-21T21:55:36.201248+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T05:28:29.953381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.555713+0000","last_change":"2024-05-21T21:51:16.925803+0000","last_active":"2024-05-21T21:55:36.555713+0000","last_peered":"2024-05-21T21:55:36.555713+0000","last_clean":"2024-05-21T21:55:36.555713+0000","last_became_active":"2024-05-21T21:51:16.925111+0000","last_became_peered":"2024-05-21T21:51:16.925111+0000","last_unstale":"2024-05-21T21:55:36.555713+0000","last_undegraded":"2024-05-21T21:55:36.555713+0000","last_fullsized":"2024-05-21T21:55:36.555713+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T06:22:11.353379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'5","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.352681+0000","last_change":"2024-05-21T21:51:12.859593+0000","last_active":"2024-05-21T21:55:36.352681+0000","last_peered":"2024-05-21T21:55:36.352681+0000","last_clean":"2024-05-21T21:55:36.352681+0000","last_became_active":"2024-05-21T21:51:12.859197+0000","last_became_peered":"2024-05-21T21:51:12.859197+0000","last_unstale":"2024-05-21T21:55:36.352681+0000","last_undegraded":"2024-05-21T21:55:36.352681+0000","last_fullsized":"2024-05-21T21:55:36.352681+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:13:12.311496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"82'15","reported_seq":44,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.551024+0000","last_change":"2024-05-21T21:52:10.879028+0000","last_active":"2024-05-21T21:55:36.551024+0000","last_peered":"2024-05-21T21:55:36.551024+0000","last_clean":"2024-05-21T21:55:36.551024+0000","last_became_active":"2024-05-21T21:52:10.878664+0000","last_became_peered":"2024-05-21T21:52:10.878664+0000","last_unstale":"2024-05-21T21:55:36.551024+0000","last_undegraded":"2024-05-21T21:55:36.551024+0000","last_fullsized":"2024-05-21T21:55:36.551024+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:05:53.953546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'7","reported_seq":278,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.723211+0000","last_change":"2024-05-21T21:51:12.852169+0000","last_active":"2024-05-21T21:55:36.723211+0000","last_peered":"2024-05-21T21:55:36.723211+0000","last_clean":"2024-05-21T21:55:36.723211+0000","last_became_active":"2024-05-21T21:51:12.851782+0000","last_became_peered":"2024-05-21T21:51:12.851782+0000","last_unstale":"2024-05-21T21:55:36.723211+0000","last_undegraded":"2024-05-21T21:55:36.723211+0000","last_fullsized":"2024-05-21T21:55:36.723211+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:46:09.692158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":243,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"82'6","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.550518+0000","last_change":"2024-05-21T21:51:16.929264+0000","last_active":"2024-05-21T21:55:36.550518+0000","last_peered":"2024-05-21T21:55:36.550518+0000","last_clean":"2024-05-21T21:55:36.550518+0000","last_became_active":"2024-05-21T21:51:16.928900+0000","last_became_peered":"2024-05-21T21:51:16.928900+0000","last_unstale":"2024-05-21T21:55:36.550518+0000","last_undegraded":"2024-05-21T21:55:36.550518+0000","last_fullsized":"2024-05-21T21:55:36.550518+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:42:28.040939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'17","reported_seq":71,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.556247+0000","last_change":"2024-05-21T21:52:10.984451+0000","last_active":"2024-05-21T21:55:36.556247+0000","last_peered":"2024-05-21T21:55:36.556247+0000","last_clean":"2024-05-21T21:55:36.556247+0000","last_became_active":"2024-05-21T21:52:10.974050+0000","last_became_peered":"2024-05-21T21:52:10.974050+0000","last_unstale":"2024-05-21T21:55:36.556247+0000","last_undegraded":"2024-05-21T21:55:36.556247+0000","last_fullsized":"2024-05-21T21:55:36.556247+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T07:08:18.307022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"82'6","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.352648+0000","last_change":"2024-05-21T21:51:16.935464+0000","last_active":"2024-05-21T21:55:36.352648+0000","last_peered":"2024-05-21T21:55:36.352648+0000","last_clean":"2024-05-21T21:55:36.352648+0000","last_became_active":"2024-05-21T21:51:16.935091+0000","last_became_peered":"2024-05-21T21:51:16.935091+0000","last_unstale":"2024-05-21T21:55:36.352648+0000","last_undegraded":"2024-05-21T21:55:36.352648+0000","last_fullsized":"2024-05-21T21:55:36.352648+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T06:59:16.330649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.556695+0000","last_change":"2024-05-21T21:51:12.853327+0000","last_active":"2024-05-21T21:55:36.556695+0000","last_peered":"2024-05-21T21:55:36.556695+0000","last_clean":"2024-05-21T21:55:36.556695+0000","last_became_active":"2024-05-21T21:51:12.852946+0000","last_became_peered":"2024-05-21T21:51:12.852946+0000","last_unstale":"2024-05-21T21:55:36.556695+0000","last_undegraded":"2024-05-21T21:55:36.556695+0000","last_fullsized":"2024-05-21T21:55:36.556695+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:00:34.256858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"82'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.341305+0000","last_change":"2024-05-21T21:51:16.929341+0000","last_active":"2024-05-21T21:55:36.341305+0000","last_peered":"2024-05-21T21:55:36.341305+0000","last_clean":"2024-05-21T21:55:36.341305+0000","last_became_active":"2024-05-21T21:51:16.929008+0000","last_became_peered":"2024-05-21T21:51:16.929008+0000","last_unstale":"2024-05-21T21:55:36.341305+0000","last_undegraded":"2024-05-21T21:55:36.341305+0000","last_fullsized":"2024-05-21T21:55:36.341305+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:26:26.297136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"80'1","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.197640+0000","last_change":"2024-05-21T21:51:12.945433+0000","last_active":"2024-05-21T21:55:36.197640+0000","last_peered":"2024-05-21T21:55:36.197640+0000","last_clean":"2024-05-21T21:55:36.197640+0000","last_became_active":"2024-05-21T21:51:12.944463+0000","last_became_peered":"2024-05-21T21:51:12.944463+0000","last_unstale":"2024-05-21T21:55:36.197640+0000","last_undegraded":"2024-05-21T21:55:36.197640+0000","last_fullsized":"2024-05-21T21:55:36.197640+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:29:52.014583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'5","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.356192+0000","last_change":"2024-05-21T21:52:10.883554+0000","last_active":"2024-05-21T21:55:36.356192+0000","last_peered":"2024-05-21T21:55:36.356192+0000","last_clean":"2024-05-21T21:55:36.356192+0000","last_became_active":"2024-05-21T21:52:10.877789+0000","last_became_peered":"2024-05-21T21:52:10.877789+0000","last_unstale":"2024-05-21T21:55:36.356192+0000","last_undegraded":"2024-05-21T21:55:36.356192+0000","last_fullsized":"2024-05-21T21:55:36.356192+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:52:37.623551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"83'6","reported_seq":276,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.890796+0000","last_change":"2024-05-21T21:51:12.852883+0000","last_active":"2024-05-21T21:55:36.890796+0000","last_peered":"2024-05-21T21:55:36.890796+0000","last_clean":"2024-05-21T21:55:36.890796+0000","last_became_active":"2024-05-21T21:51:12.852549+0000","last_became_peered":"2024-05-21T21:51:12.852549+0000","last_unstale":"2024-05-21T21:55:36.890796+0000","last_undegraded":"2024-05-21T21:55:36.890796+0000","last_fullsized":"2024-05-21T21:55:36.890796+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:05:53.002286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":240,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"82'7","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.356089+0000","last_change":"2024-05-21T21:51:16.944425+0000","last_active":"2024-05-21T21:55:36.356089+0000","last_peered":"2024-05-21T21:55:36.356089+0000","last_clean":"2024-05-21T21:55:36.356089+0000","last_became_active":"2024-05-21T21:51:16.944033+0000","last_became_peered":"2024-05-21T21:51:16.944033+0000","last_unstale":"2024-05-21T21:55:36.356089+0000","last_undegraded":"2024-05-21T21:55:36.356089+0000","last_fullsized":"2024-05-21T21:55:36.356089+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T07:16:26.802329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"82'8","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.352742+0000","last_change":"2024-05-21T21:51:16.933264+0000","last_active":"2024-05-21T21:55:36.352742+0000","last_peered":"2024-05-21T21:55:36.352742+0000","last_clean":"2024-05-21T21:55:36.352742+0000","last_became_active":"2024-05-21T21:51:16.932911+0000","last_became_peered":"2024-05-21T21:51:16.932911+0000","last_unstale":"2024-05-21T21:55:36.352742+0000","last_undegraded":"2024-05-21T21:55:36.352742+0000","last_fullsized":"2024-05-21T21:55:36.352742+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:25:54.006367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"82'6","reported_seq":275,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.555395+0000","last_change":"2024-05-21T21:51:12.942900+0000","last_active":"2024-05-21T21:55:36.555395+0000","last_peered":"2024-05-21T21:55:36.555395+0000","last_clean":"2024-05-21T21:55:36.555395+0000","last_became_active":"2024-05-21T21:51:12.941038+0000","last_became_peered":"2024-05-21T21:51:12.941038+0000","last_unstale":"2024-05-21T21:55:36.555395+0000","last_undegraded":"2024-05-21T21:55:36.555395+0000","last_fullsized":"2024-05-21T21:55:36.555395+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:38:43.240011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":239,"num_read_kb":239,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"82'8","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.342906+0000","last_change":"2024-05-21T21:51:16.933391+0000","last_active":"2024-05-21T21:55:36.342906+0000","last_peered":"2024-05-21T21:55:36.342906+0000","last_clean":"2024-05-21T21:55:36.342906+0000","last_became_active":"2024-05-21T21:51:16.932320+0000","last_became_peered":"2024-05-21T21:51:16.932320+0000","last_unstale":"2024-05-21T21:55:36.342906+0000","last_undegraded":"2024-05-21T21:55:36.342906+0000","last_fullsized":"2024-05-21T21:55:36.342906+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T09:06:59.126748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'6","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.723215+0000","last_change":"2024-05-21T21:51:12.856342+0000","last_active":"2024-05-21T21:55:36.723215+0000","last_peered":"2024-05-21T21:55:36.723215+0000","last_clean":"2024-05-21T21:55:36.723215+0000","last_became_active":"2024-05-21T21:51:12.855952+0000","last_became_peered":"2024-05-21T21:51:12.855952+0000","last_unstale":"2024-05-21T21:55:36.723215+0000","last_undegraded":"2024-05-21T21:55:36.723215+0000","last_fullsized":"2024-05-21T21:55:36.723215+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:34:40.734034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":103,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.550605+0000","last_change":"2024-05-21T21:49:42.797187+0000","last_active":"2024-05-21T21:55:36.550605+0000","last_peered":"2024-05-21T21:55:36.550605+0000","last_clean":"2024-05-21T21:55:36.550605+0000","last_became_active":"2024-05-21T21:49:42.791291+0000","last_became_peered":"2024-05-21T21:49:42.791291+0000","last_unstale":"2024-05-21T21:55:36.550605+0000","last_undegraded":"2024-05-21T21:55:36.550605+0000","last_fullsized":"2024-05-21T21:55:36.550605+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:45:55.552293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:45:55.552293+0000","last_clean_scrub_stamp":"2024-05-21T21:45:55.552293+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T07:24:18.302523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"82'5","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.555133+0000","last_change":"2024-05-21T21:51:16.926446+0000","last_active":"2024-05-21T21:55:36.555133+0000","last_peered":"2024-05-21T21:55:36.555133+0000","last_clean":"2024-05-21T21:55:36.555133+0000","last_became_active":"2024-05-21T21:51:16.926070+0000","last_became_peered":"2024-05-21T21:51:16.926070+0000","last_unstale":"2024-05-21T21:55:36.555133+0000","last_undegraded":"2024-05-21T21:55:36.555133+0000","last_fullsized":"2024-05-21T21:55:36.555133+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:44:27.366393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'4","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.550735+0000","last_change":"2024-05-21T21:51:12.854371+0000","last_active":"2024-05-21T21:55:36.550735+0000","last_peered":"2024-05-21T21:55:36.550735+0000","last_clean":"2024-05-21T21:55:36.550735+0000","last_became_active":"2024-05-21T21:51:12.853847+0000","last_became_peered":"2024-05-21T21:51:12.853847+0000","last_unstale":"2024-05-21T21:55:36.550735+0000","last_undegraded":"2024-05-21T21:55:36.550735+0000","last_fullsized":"2024-05-21T21:55:36.550735+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:14:10.549901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"80'4","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.556551+0000","last_change":"2024-05-21T21:51:12.941227+0000","last_active":"2024-05-21T21:55:36.556551+0000","last_peered":"2024-05-21T21:55:36.556551+0000","last_clean":"2024-05-21T21:55:36.556551+0000","last_became_active":"2024-05-21T21:51:12.940702+0000","last_became_peered":"2024-05-21T21:51:12.940702+0000","last_unstale":"2024-05-21T21:55:36.556551+0000","last_undegraded":"2024-05-21T21:55:36.556551+0000","last_fullsized":"2024-05-21T21:55:36.556551+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:40:34.124921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"82'9","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.556301+0000","last_change":"2024-05-21T21:51:16.945627+0000","last_active":"2024-05-21T21:55:36.556301+0000","last_peered":"2024-05-21T21:55:36.556301+0000","last_clean":"2024-05-21T21:55:36.556301+0000","last_became_active":"2024-05-21T21:51:16.945297+0000","last_became_peered":"2024-05-21T21:51:16.945297+0000","last_unstale":"2024-05-21T21:55:36.556301+0000","last_undegraded":"2024-05-21T21:55:36.556301+0000","last_fullsized":"2024-05-21T21:55:36.556301+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:54:38.898774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'51","reported_seq":120,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.502619+0000","last_change":"2024-05-21T21:52:10.888076+0000","last_active":"2024-05-21T21:55:36.502619+0000","last_peered":"2024-05-21T21:55:36.502619+0000","last_clean":"2024-05-21T21:55:36.502619+0000","last_became_active":"2024-05-21T21:52:10.879368+0000","last_became_peered":"2024-05-21T21:52:10.879368+0000","last_unstale":"2024-05-21T21:55:36.502619+0000","last_undegraded":"2024-05-21T21:55:36.502619+0000","last_fullsized":"2024-05-21T21:55:36.502619+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:46:43.190411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":91,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"82'5","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.344028+0000","last_change":"2024-05-21T21:51:16.929807+0000","last_active":"2024-05-21T21:55:36.344028+0000","last_peered":"2024-05-21T21:55:36.344028+0000","last_clean":"2024-05-21T21:55:36.344028+0000","last_became_active":"2024-05-21T21:51:16.928859+0000","last_became_peered":"2024-05-21T21:51:16.928859+0000","last_unstale":"2024-05-21T21:55:36.344028+0000","last_undegraded":"2024-05-21T21:55:36.344028+0000","last_fullsized":"2024-05-21T21:55:36.344028+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:41:32.830892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'7","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.723447+0000","last_change":"2024-05-21T21:51:12.850843+0000","last_active":"2024-05-21T21:55:36.723447+0000","last_peered":"2024-05-21T21:55:36.723447+0000","last_clean":"2024-05-21T21:55:36.723447+0000","last_became_active":"2024-05-21T21:51:12.850516+0000","last_became_peered":"2024-05-21T21:51:12.850516+0000","last_unstale":"2024-05-21T21:55:36.723447+0000","last_undegraded":"2024-05-21T21:55:36.723447+0000","last_fullsized":"2024-05-21T21:55:36.723447+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:37:05.191626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"82'7","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.553781+0000","last_change":"2024-05-21T21:51:16.949017+0000","last_active":"2024-05-21T21:55:36.553781+0000","last_peered":"2024-05-21T21:55:36.553781+0000","last_clean":"2024-05-21T21:55:36.553781+0000","last_became_active":"2024-05-21T21:51:16.948691+0000","last_became_peered":"2024-05-21T21:51:16.948691+0000","last_unstale":"2024-05-21T21:55:36.553781+0000","last_undegraded":"2024-05-21T21:55:36.553781+0000","last_fullsized":"2024-05-21T21:55:36.553781+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:32:06.606581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.199191+0000","last_change":"2024-05-21T21:51:12.939203+0000","last_active":"2024-05-21T21:55:36.199191+0000","last_peered":"2024-05-21T21:55:36.199191+0000","last_clean":"2024-05-21T21:55:36.199191+0000","last_became_active":"2024-05-21T21:51:12.938936+0000","last_became_peered":"2024-05-21T21:51:12.938936+0000","last_unstale":"2024-05-21T21:55:36.199191+0000","last_undegraded":"2024-05-21T21:55:36.199191+0000","last_fullsized":"2024-05-21T21:55:36.199191+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:55:50.336362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"82'2","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.556315+0000","last_change":"2024-05-21T21:51:16.926216+0000","last_active":"2024-05-21T21:55:36.556315+0000","last_peered":"2024-05-21T21:55:36.556315+0000","last_clean":"2024-05-21T21:55:36.556315+0000","last_became_active":"2024-05-21T21:51:16.925834+0000","last_became_peered":"2024-05-21T21:51:16.925834+0000","last_unstale":"2024-05-21T21:55:36.556315+0000","last_undegraded":"2024-05-21T21:55:36.556315+0000","last_fullsized":"2024-05-21T21:55:36.556315+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:13:30.192908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'7","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.342741+0000","last_change":"2024-05-21T21:51:12.855306+0000","last_active":"2024-05-21T21:55:36.342741+0000","last_peered":"2024-05-21T21:55:36.342741+0000","last_clean":"2024-05-21T21:55:36.342741+0000","last_became_active":"2024-05-21T21:51:12.854722+0000","last_became_peered":"2024-05-21T21:51:12.854722+0000","last_unstale":"2024-05-21T21:55:36.342741+0000","last_undegraded":"2024-05-21T21:55:36.342741+0000","last_fullsized":"2024-05-21T21:55:36.342741+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:39:41.583219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"82'9","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.341026+0000","last_change":"2024-05-21T21:51:16.944758+0000","last_active":"2024-05-21T21:55:36.341026+0000","last_peered":"2024-05-21T21:55:36.341026+0000","last_clean":"2024-05-21T21:55:36.341026+0000","last_became_active":"2024-05-21T21:51:16.944393+0000","last_became_peered":"2024-05-21T21:51:16.944393+0000","last_unstale":"2024-05-21T21:55:36.341026+0000","last_undegraded":"2024-05-21T21:55:36.341026+0000","last_fullsized":"2024-05-21T21:55:36.341026+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:01:23.048355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'3","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.201175+0000","last_change":"2024-05-21T21:51:12.860378+0000","last_active":"2024-05-21T21:55:36.201175+0000","last_peered":"2024-05-21T21:55:36.201175+0000","last_clean":"2024-05-21T21:55:36.201175+0000","last_became_active":"2024-05-21T21:51:12.859978+0000","last_became_peered":"2024-05-21T21:51:12.859978+0000","last_unstale":"2024-05-21T21:55:36.201175+0000","last_undegraded":"2024-05-21T21:55:36.201175+0000","last_fullsized":"2024-05-21T21:55:36.201175+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T07:52:26.902232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"82'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.343347+0000","last_change":"2024-05-21T21:51:16.943194+0000","last_active":"2024-05-21T21:55:36.343347+0000","last_peered":"2024-05-21T21:55:36.343347+0000","last_clean":"2024-05-21T21:55:36.343347+0000","last_became_active":"2024-05-21T21:51:16.942884+0000","last_became_peered":"2024-05-21T21:51:16.942884+0000","last_unstale":"2024-05-21T21:55:36.343347+0000","last_undegraded":"2024-05-21T21:55:36.343347+0000","last_fullsized":"2024-05-21T21:55:36.343347+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T05:58:17.436995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.556452+0000","last_change":"2024-05-21T21:51:12.853390+0000","last_active":"2024-05-21T21:55:36.556452+0000","last_peered":"2024-05-21T21:55:36.556452+0000","last_clean":"2024-05-21T21:55:36.556452+0000","last_became_active":"2024-05-21T21:51:12.852990+0000","last_became_peered":"2024-05-21T21:51:12.852990+0000","last_unstale":"2024-05-21T21:55:36.556452+0000","last_undegraded":"2024-05-21T21:55:36.556452+0000","last_fullsized":"2024-05-21T21:55:36.556452+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:32:16.392609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"82'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.550802+0000","last_change":"2024-05-21T21:51:16.933430+0000","last_active":"2024-05-21T21:55:36.550802+0000","last_peered":"2024-05-21T21:55:36.550802+0000","last_clean":"2024-05-21T21:55:36.550802+0000","last_became_active":"2024-05-21T21:51:16.933053+0000","last_became_peered":"2024-05-21T21:51:16.933053+0000","last_unstale":"2024-05-21T21:55:36.550802+0000","last_undegraded":"2024-05-21T21:55:36.550802+0000","last_fullsized":"2024-05-21T21:55:36.550802+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T06:14:57.468727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'7","reported_seq":277,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.747135+0000","last_change":"2024-05-21T21:51:12.946113+0000","last_active":"2024-05-21T21:55:36.747135+0000","last_peered":"2024-05-21T21:55:36.747135+0000","last_clean":"2024-05-21T21:55:36.747135+0000","last_became_active":"2024-05-21T21:51:12.945650+0000","last_became_peered":"2024-05-21T21:51:12.945650+0000","last_unstale":"2024-05-21T21:55:36.747135+0000","last_undegraded":"2024-05-21T21:55:36.747135+0000","last_fullsized":"2024-05-21T21:55:36.747135+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T09:40:18.209400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1952,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":241,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"82'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.550873+0000","last_change":"2024-05-21T21:51:16.932632+0000","last_active":"2024-05-21T21:55:36.550873+0000","last_peered":"2024-05-21T21:55:36.550873+0000","last_clean":"2024-05-21T21:55:36.550873+0000","last_became_active":"2024-05-21T21:51:16.932256+0000","last_became_peered":"2024-05-21T21:51:16.932256+0000","last_unstale":"2024-05-21T21:55:36.550873+0000","last_undegraded":"2024-05-21T21:55:36.550873+0000","last_fullsized":"2024-05-21T21:55:36.550873+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:04:48.146485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'41","reported_seq":78,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.496010+0000","last_change":"2024-05-21T21:51:12.860602+0000","last_active":"2024-05-21T21:55:36.496010+0000","last_peered":"2024-05-21T21:55:36.496010+0000","last_clean":"2024-05-21T21:55:36.496010+0000","last_became_active":"2024-05-21T21:51:12.860228+0000","last_became_peered":"2024-05-21T21:51:12.860228+0000","last_unstale":"2024-05-21T21:55:36.496010+0000","last_undegraded":"2024-05-21T21:55:36.496010+0000","last_fullsized":"2024-05-21T21:55:36.496010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:33:39.230872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.551036+0000","last_change":"2024-05-21T21:51:16.929616+0000","last_active":"2024-05-21T21:55:36.551036+0000","last_peered":"2024-05-21T21:55:36.551036+0000","last_clean":"2024-05-21T21:55:36.551036+0000","last_became_active":"2024-05-21T21:51:16.928569+0000","last_became_peered":"2024-05-21T21:51:16.928569+0000","last_unstale":"2024-05-21T21:55:36.551036+0000","last_undegraded":"2024-05-21T21:55:36.551036+0000","last_fullsized":"2024-05-21T21:55:36.551036+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:15:41.790344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'5","reported_seq":46,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.555499+0000","last_change":"2024-05-21T21:52:10.881114+0000","last_active":"2024-05-21T21:55:36.555499+0000","last_peered":"2024-05-21T21:55:36.555499+0000","last_clean":"2024-05-21T21:55:36.555499+0000","last_became_active":"2024-05-21T21:52:10.872022+0000","last_became_peered":"2024-05-21T21:52:10.872022+0000","last_unstale":"2024-05-21T21:55:36.555499+0000","last_undegraded":"2024-05-21T21:55:36.555499+0000","last_fullsized":"2024-05-21T21:55:36.555499+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T06:05:51.419156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.550881+0000","last_change":"2024-05-21T21:51:16.926496+0000","last_active":"2024-05-21T21:55:36.550881+0000","last_peered":"2024-05-21T21:55:36.550881+0000","last_clean":"2024-05-21T21:55:36.550881+0000","last_became_active":"2024-05-21T21:51:16.926112+0000","last_became_peered":"2024-05-21T21:51:16.926112+0000","last_unstale":"2024-05-21T21:55:36.550881+0000","last_undegraded":"2024-05-21T21:55:36.550881+0000","last_fullsized":"2024-05-21T21:55:36.550881+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:26:08.254421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"82'9","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.341047+0000","last_change":"2024-05-21T21:51:16.930522+0000","last_active":"2024-05-21T21:55:36.341047+0000","last_peered":"2024-05-21T21:55:36.341047+0000","last_clean":"2024-05-21T21:55:36.341047+0000","last_became_active":"2024-05-21T21:51:16.929853+0000","last_became_peered":"2024-05-21T21:51:16.929853+0000","last_unstale":"2024-05-21T21:55:36.341047+0000","last_undegraded":"2024-05-21T21:55:36.341047+0000","last_fullsized":"2024-05-21T21:55:36.341047+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:10:57.940009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"82'12","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.201162+0000","last_change":"2024-05-21T21:51:16.931751+0000","last_active":"2024-05-21T21:55:36.201162+0000","last_peered":"2024-05-21T21:55:36.201162+0000","last_clean":"2024-05-21T21:55:36.201162+0000","last_became_active":"2024-05-21T21:51:16.931353+0000","last_became_peered":"2024-05-21T21:51:16.931353+0000","last_unstale":"2024-05-21T21:55:36.201162+0000","last_undegraded":"2024-05-21T21:55:36.201162+0000","last_fullsized":"2024-05-21T21:55:36.201162+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:33:45.297850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"82'8","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.201127+0000","last_change":"2024-05-21T21:51:16.940096+0000","last_active":"2024-05-21T21:55:36.201127+0000","last_peered":"2024-05-21T21:55:36.201127+0000","last_clean":"2024-05-21T21:55:36.201127+0000","last_became_active":"2024-05-21T21:51:16.939740+0000","last_became_peered":"2024-05-21T21:51:16.939740+0000","last_unstale":"2024-05-21T21:55:36.201127+0000","last_undegraded":"2024-05-21T21:55:36.201127+0000","last_fullsized":"2024-05-21T21:55:36.201127+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:59:52.569526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"82'7","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.552155+0000","last_change":"2024-05-21T21:51:16.946488+0000","last_active":"2024-05-21T21:55:36.552155+0000","last_peered":"2024-05-21T21:55:36.552155+0000","last_clean":"2024-05-21T21:55:36.552155+0000","last_became_active":"2024-05-21T21:51:16.945957+0000","last_became_peered":"2024-05-21T21:51:16.945957+0000","last_unstale":"2024-05-21T21:55:36.552155+0000","last_undegraded":"2024-05-21T21:55:36.552155+0000","last_fullsized":"2024-05-21T21:55:36.552155+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:40:36.708344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"82'4","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.556177+0000","last_change":"2024-05-21T21:51:16.949349+0000","last_active":"2024-05-21T21:55:36.556177+0000","last_peered":"2024-05-21T21:55:36.556177+0000","last_clean":"2024-05-21T21:55:36.556177+0000","last_became_active":"2024-05-21T21:51:16.946986+0000","last_became_peered":"2024-05-21T21:51:16.946986+0000","last_unstale":"2024-05-21T21:55:36.556177+0000","last_undegraded":"2024-05-21T21:55:36.556177+0000","last_fullsized":"2024-05-21T21:55:36.556177+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:01:14.877110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"82'6","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.341365+0000","last_change":"2024-05-21T21:51:16.929172+0000","last_active":"2024-05-21T21:55:36.341365+0000","last_peered":"2024-05-21T21:55:36.341365+0000","last_clean":"2024-05-21T21:55:36.341365+0000","last_became_active":"2024-05-21T21:51:16.928823+0000","last_became_peered":"2024-05-21T21:51:16.928823+0000","last_unstale":"2024-05-21T21:55:36.341365+0000","last_undegraded":"2024-05-21T21:55:36.341365+0000","last_fullsized":"2024-05-21T21:55:36.341365+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:09:13.719272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"82'15","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.201467+0000","last_change":"2024-05-21T21:51:16.930332+0000","last_active":"2024-05-21T21:55:36.201467+0000","last_peered":"2024-05-21T21:55:36.201467+0000","last_clean":"2024-05-21T21:55:36.201467+0000","last_became_active":"2024-05-21T21:51:16.930024+0000","last_became_peered":"2024-05-21T21:51:16.930024+0000","last_unstale":"2024-05-21T21:55:36.201467+0000","last_undegraded":"2024-05-21T21:55:36.201467+0000","last_fullsized":"2024-05-21T21:55:36.201467+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:47:41.272725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"82'2","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.343903+0000","last_change":"2024-05-21T21:51:16.930102+0000","last_active":"2024-05-21T21:55:36.343903+0000","last_peered":"2024-05-21T21:55:36.343903+0000","last_clean":"2024-05-21T21:55:36.343903+0000","last_became_active":"2024-05-21T21:51:16.929410+0000","last_became_peered":"2024-05-21T21:51:16.929410+0000","last_unstale":"2024-05-21T21:55:36.343903+0000","last_undegraded":"2024-05-21T21:55:36.343903+0000","last_fullsized":"2024-05-21T21:55:36.343903+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:11:46.296288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"82'6","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.556235+0000","last_change":"2024-05-21T21:52:10.873424+0000","last_active":"2024-05-21T21:55:36.556235+0000","last_peered":"2024-05-21T21:55:36.556235+0000","last_clean":"2024-05-21T21:55:36.556235+0000","last_became_active":"2024-05-21T21:52:10.873118+0000","last_became_peered":"2024-05-21T21:52:10.873118+0000","last_unstale":"2024-05-21T21:55:36.556235+0000","last_undegraded":"2024-05-21T21:55:36.556235+0000","last_fullsized":"2024-05-21T21:55:36.556235+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:05:00.731013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,3],"acting":[8,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.29","version":"82'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.342603+0000","last_change":"2024-05-21T21:51:16.934442+0000","last_active":"2024-05-21T21:55:36.342603+0000","last_peered":"2024-05-21T21:55:36.342603+0000","last_clean":"2024-05-21T21:55:36.342603+0000","last_became_active":"2024-05-21T21:51:16.934086+0000","last_became_peered":"2024-05-21T21:51:16.934086+0000","last_unstale":"2024-05-21T21:55:36.342603+0000","last_undegraded":"2024-05-21T21:55:36.342603+0000","last_fullsized":"2024-05-21T21:55:36.342603+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:36:44.302532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"82'10","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.199405+0000","last_change":"2024-05-21T21:51:16.924479+0000","last_active":"2024-05-21T21:55:36.199405+0000","last_peered":"2024-05-21T21:55:36.199405+0000","last_clean":"2024-05-21T21:55:36.199405+0000","last_became_active":"2024-05-21T21:51:16.923998+0000","last_became_peered":"2024-05-21T21:51:16.923998+0000","last_unstale":"2024-05-21T21:55:36.199405+0000","last_undegraded":"2024-05-21T21:55:36.199405+0000","last_fullsized":"2024-05-21T21:55:36.199405+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:23:07.931731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"82'11","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.555801+0000","last_change":"2024-05-21T21:51:16.947271+0000","last_active":"2024-05-21T21:55:36.555801+0000","last_peered":"2024-05-21T21:55:36.555801+0000","last_clean":"2024-05-21T21:55:36.555801+0000","last_became_active":"2024-05-21T21:51:16.946912+0000","last_became_peered":"2024-05-21T21:51:16.946912+0000","last_unstale":"2024-05-21T21:55:36.555801+0000","last_undegraded":"2024-05-21T21:55:36.555801+0000","last_fullsized":"2024-05-21T21:55:36.555801+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:51:29.646915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"82'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.341021+0000","last_change":"2024-05-21T21:51:16.928852+0000","last_active":"2024-05-21T21:55:36.341021+0000","last_peered":"2024-05-21T21:55:36.341021+0000","last_clean":"2024-05-21T21:55:36.341021+0000","last_became_active":"2024-05-21T21:51:16.928480+0000","last_became_peered":"2024-05-21T21:51:16.928480+0000","last_unstale":"2024-05-21T21:55:36.341021+0000","last_undegraded":"2024-05-21T21:55:36.341021+0000","last_fullsized":"2024-05-21T21:55:36.341021+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T09:12:30.691254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":600652,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":2521,"num_read_kb":1376,"num_write":377,"num_write_kb":1467,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":112,"num_keys_recovered":17,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2174976,"data_stored":1801956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":304,"ondisk_log_size":304,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645381,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":901120,"data_stored":433959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841609,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28736,"kb_used_data":1720,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716416,"statfs":{"total":95995035648,"available":95965609984,"internally_reserved":0,"allocated":1761280,"data_stored":1232552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005134,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":946176,"data_stored":442962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201365,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":921600,"data_stored":434267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397593,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28500,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716652,"statfs":{"total":95995035648,"available":95965851648,"internally_reserved":0,"allocated":1519616,"data_stored":1027768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561119,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":925696,"data_stored":442953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574971,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1003520,"data_stored":510454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738496,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36264,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708888,"statfs":{"total":95995035648,"available":95957901312,"internally_reserved":0,"allocated":946176,"data_stored":444683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411447,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36324,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708828,"statfs":{"total":95995035648,"available":95957839872,"internally_reserved":0,"allocated":1007616,"data_stored":512074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149807,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36236,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708916,"statfs":{"total":95995035648,"available":95957929984,"internally_reserved":0,"allocated":921600,"data_stored":433314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953578,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":925696,"data_stored":442955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757349,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32672,"kb_used_data":1496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712480,"statfs":{"total":95995035648,"available":95961579520,"internally_reserved":0,"allocated":1531904,"data_stored":1026914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":68731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":70351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":557056,"data_stored":525246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":135653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":258048,"data_stored":200344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":395190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-21T21:55:43.056 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-21T21:55:43.491 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:55:43.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:43 smithi151 ceph-mon[31623]: from='client.25231 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T21:55:43.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:43 smithi151 ceph-mon[31623]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:55:43.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:43 smithi151 ceph-mon[31623]: from='client.25237 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T21:55:43.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:43 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T21:55:43.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:43 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T21:55:43.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:43 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:55:43.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:43 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:55:43.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:43 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:55:43.954 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:43 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:55:43.954 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:43 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:55:43.954 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:43 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:55:43.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:43 smithi184 ceph-mon[36942]: from='client.25231 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T21:55:43.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:43 smithi184 ceph-mon[36942]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:55:43.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:43 smithi184 ceph-mon[36942]: from='client.25237 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T21:55:43.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:43 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T21:55:43.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:43 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T21:55:43.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:43 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:55:43.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:43 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:55:43.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:43 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:55:43.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:43 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:55:43.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:43 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:55:43.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:43 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:55:44.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:43 smithi164 ceph-mon[36269]: from='client.25231 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T21:55:44.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:43 smithi164 ceph-mon[36269]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T21:55:44.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:43 smithi164 ceph-mon[36269]: from='client.25237 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T21:55:44.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:43 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T21:55:44.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:43 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T21:55:44.037 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:43 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:55:44.037 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:43 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:55:44.037 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:43 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:55:44.037 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:43 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-21T21:55:44.037 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:43 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:55:44.037 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:43 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:55:44.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:44 smithi151 ceph-mon[31623]: Adjusting osd_memory_target on smithi151 to 176.4M 2024-05-21T21:55:44.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:44 smithi151 ceph-mon[31623]: Unable to set osd_memory_target on smithi151 to 185064857: error parsing value: Value '185064857' is below minimum 939524096 2024-05-21T21:55:44.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:44 smithi151 ceph-mon[31623]: Updating smithi151:/etc/ceph/ceph.conf 2024-05-21T21:55:44.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:44 smithi151 ceph-mon[31623]: Updating smithi164:/etc/ceph/ceph.conf 2024-05-21T21:55:44.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:44 smithi151 ceph-mon[31623]: Updating smithi184:/etc/ceph/ceph.conf 2024-05-21T21:55:44.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:44 smithi151 ceph-mon[31623]: from='client.25238 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T21:55:44.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:44 smithi151 ceph-mon[31623]: Updating smithi164:/var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T21:55:44.954 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:44 smithi151 ceph-mon[31623]: Updating smithi184:/var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T21:55:44.954 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:44 smithi151 ceph-mon[31623]: Updating smithi151:/var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T21:55:44.954 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:44 smithi151 ceph-mon[31623]: Updating smithi164:/etc/ceph/ceph.client.admin.keyring 2024-05-21T21:55:44.954 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:44 smithi151 ceph-mon[31623]: Updating smithi184:/etc/ceph/ceph.client.admin.keyring 2024-05-21T21:55:44.954 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:44 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T21:55:44.954 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:44 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T21:55:44.954 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:44 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T21:55:44.954 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:44 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T21:55:44.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:44 smithi184 ceph-mon[36942]: Adjusting osd_memory_target on smithi151 to 176.4M 2024-05-21T21:55:44.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:44 smithi184 ceph-mon[36942]: Unable to set osd_memory_target on smithi151 to 185064857: error parsing value: Value '185064857' is below minimum 939524096 2024-05-21T21:55:44.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:44 smithi184 ceph-mon[36942]: Updating smithi151:/etc/ceph/ceph.conf 2024-05-21T21:55:44.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:44 smithi184 ceph-mon[36942]: Updating smithi164:/etc/ceph/ceph.conf 2024-05-21T21:55:44.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:44 smithi184 ceph-mon[36942]: Updating smithi184:/etc/ceph/ceph.conf 2024-05-21T21:55:44.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:44 smithi184 ceph-mon[36942]: from='client.25238 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T21:55:44.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:44 smithi184 ceph-mon[36942]: Updating smithi164:/var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T21:55:44.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:44 smithi184 ceph-mon[36942]: Updating smithi184:/var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T21:55:44.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:44 smithi184 ceph-mon[36942]: Updating smithi151:/var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T21:55:44.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:44 smithi184 ceph-mon[36942]: Updating smithi164:/etc/ceph/ceph.client.admin.keyring 2024-05-21T21:55:44.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:44 smithi184 ceph-mon[36942]: Updating smithi184:/etc/ceph/ceph.client.admin.keyring 2024-05-21T21:55:44.987 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:44 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T21:55:44.987 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:44 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T21:55:44.987 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:44 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T21:55:44.987 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:44 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T21:55:45.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:44 smithi164 ceph-mon[36269]: Adjusting osd_memory_target on smithi151 to 176.4M 2024-05-21T21:55:45.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:44 smithi164 ceph-mon[36269]: Unable to set osd_memory_target on smithi151 to 185064857: error parsing value: Value '185064857' is below minimum 939524096 2024-05-21T21:55:45.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:44 smithi164 ceph-mon[36269]: Updating smithi151:/etc/ceph/ceph.conf 2024-05-21T21:55:45.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:44 smithi164 ceph-mon[36269]: Updating smithi164:/etc/ceph/ceph.conf 2024-05-21T21:55:45.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:44 smithi164 ceph-mon[36269]: Updating smithi184:/etc/ceph/ceph.conf 2024-05-21T21:55:45.037 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:44 smithi164 ceph-mon[36269]: from='client.25238 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T21:55:45.037 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:44 smithi164 ceph-mon[36269]: Updating smithi164:/var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T21:55:45.037 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:44 smithi164 ceph-mon[36269]: Updating smithi184:/var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T21:55:45.037 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:44 smithi164 ceph-mon[36269]: Updating smithi151:/var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/config/ceph.conf 2024-05-21T21:55:45.037 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:44 smithi164 ceph-mon[36269]: Updating smithi164:/etc/ceph/ceph.client.admin.keyring 2024-05-21T21:55:45.037 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:44 smithi164 ceph-mon[36269]: Updating smithi184:/etc/ceph/ceph.client.admin.keyring 2024-05-21T21:55:45.037 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:44 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T21:55:45.037 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:44 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T21:55:45.037 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:44 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T21:55:45.037 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:44 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T21:55:45.820 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:45.819+0000 7f5ed5600700 1 -- 172.21.15.151:0/3232289101 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5ed01037f0 msgr2=0x7f5ed0103bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:45.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:45.819+0000 7f5ed5600700 1 --2- 172.21.15.151:0/3232289101 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5ed01037f0 0x7f5ed0103bd0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5ec00099c0 tx=0x7f5ec0009cd0 comp rx=0 tx=0).stop 2024-05-21T21:55:45.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:45.819+0000 7f5ed5600700 1 -- 172.21.15.151:0/3232289101 shutdown_connections 2024-05-21T21:55:45.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:45.819+0000 7f5ed5600700 1 --2- 172.21.15.151:0/3232289101 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ed01088f0 0x7f5ed0110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:45.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:45.819+0000 7f5ed5600700 1 --2- 172.21.15.151:0/3232289101 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5ed01037f0 0x7f5ed0103bd0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:45.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:45.819+0000 7f5ed5600700 1 --2- 172.21.15.151:0/3232289101 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ed01041a0 0x7f5ed01081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:45.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:45.819+0000 7f5ed5600700 1 -- 172.21.15.151:0/3232289101 >> 172.21.15.151:0/3232289101 conn(0x7f5ed007a6a0 msgr2=0x7f5ed007aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:55:45.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:45.819+0000 7f5ed5600700 1 -- 172.21.15.151:0/3232289101 shutdown_connections 2024-05-21T21:55:45.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:45.819+0000 7f5ed5600700 1 -- 172.21.15.151:0/3232289101 wait complete. 2024-05-21T21:55:45.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:45.821+0000 7f5ed5600700 1 Processor -- start 2024-05-21T21:55:45.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:45.821+0000 7f5ed5600700 1 -- start start 2024-05-21T21:55:45.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:45.821+0000 7f5ed5600700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ed01037f0 0x7f5ed019f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:45.823 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:45.821+0000 7f5ed5600700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ed01041a0 0x7f5ed019fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:45.823 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:45.821+0000 7f5ed5600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5ed01088f0 0x7f5ed01999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:45.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:45.821+0000 7f5ed5600700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed0113730 con 0x7f5ed01088f0 2024-05-21T21:55:45.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:45.821+0000 7f5ed5600700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed01135b0 con 0x7f5ed01041a0 2024-05-21T21:55:45.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:45.821+0000 7f5ed5600700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed01138b0 con 0x7f5ed01037f0 2024-05-21T21:55:45.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:45.822+0000 7f5ecea00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ed01037f0 0x7f5ed019f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:45.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:45.822+0000 7f5ecea00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ed01037f0 0x7f5ed019f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:39082/0 (socket says 172.21.15.151:39082) 2024-05-21T21:55:45.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:45.822+0000 7f5ecea00700 1 -- 172.21.15.151:0/2265525168 learned_addr learned my addr 172.21.15.151:0/2265525168 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:55:45.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:45.822+0000 7f5ece000700 1 --2- 172.21.15.151:0/2265525168 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ed01041a0 0x7f5ed019fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:45.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:45.822+0000 7f5ecf400700 1 --2- 172.21.15.151:0/2265525168 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5ed01088f0 0x7f5ed01999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:45.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:45.822+0000 7f5ecea00700 1 -- 172.21.15.151:0/2265525168 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ed01041a0 msgr2=0x7f5ed019fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:45.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:45.822+0000 7f5ecea00700 1 --2- 172.21.15.151:0/2265525168 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ed01041a0 0x7f5ed019fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:45.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:45.822+0000 7f5ecea00700 1 -- 172.21.15.151:0/2265525168 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5ed01088f0 msgr2=0x7f5ed01999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:45.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:45.822+0000 7f5ecea00700 1 --2- 172.21.15.151:0/2265525168 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5ed01088f0 0x7f5ed01999d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:45.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:45.822+0000 7f5ecea00700 1 -- 172.21.15.151:0/2265525168 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ed019a130 con 0x7f5ed01037f0 2024-05-21T21:55:45.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:45.822+0000 7f5ece000700 1 --2- 172.21.15.151:0/2265525168 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ed01041a0 0x7f5ed019fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:55:45.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:45.823+0000 7f5ecea00700 1 --2- 172.21.15.151:0/2265525168 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ed01037f0 0x7f5ed019f810 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5ec0006010 tx=0x7f5ec0005280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:55:45.826 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:45.823+0000 7f5ec7400700 1 -- 172.21.15.151:0/2265525168 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec0026070 con 0x7f5ed01037f0 2024-05-21T21:55:45.826 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:45.823+0000 7f5ed5600700 1 -- 172.21.15.151:0/2265525168 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ed019a3c0 con 0x7f5ed01037f0 2024-05-21T21:55:45.826 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:45.823+0000 7f5ec7400700 1 -- 172.21.15.151:0/2265525168 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ec0005680 con 0x7f5ed01037f0 2024-05-21T21:55:45.826 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:45.823+0000 7f5ec7400700 1 -- 172.21.15.151:0/2265525168 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec000fb20 con 0x7f5ed01037f0 2024-05-21T21:55:45.826 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:45.823+0000 7f5ed5600700 1 -- 172.21.15.151:0/2265525168 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ed01a66e0 con 0x7f5ed01037f0 2024-05-21T21:55:45.826 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:45.824+0000 7f5ed5600700 1 -- 172.21.15.151:0/2265525168 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ed006a230 con 0x7f5ed01037f0 2024-05-21T21:55:45.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:45.827+0000 7f5ec7400700 1 -- 172.21.15.151:0/2265525168 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 25) ==== 149829+0+0 (secure 0 0 0) 0x7f5ec0029070 con 0x7f5ed01037f0 2024-05-21T21:55:45.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:45.828+0000 7f5ec7400700 1 --2- 172.21.15.151:0/2265525168 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f5eb40b2a80 0x7f5eb40b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:45.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:45.828+0000 7f5ec7400700 1 -- 172.21.15.151:0/2265525168 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f5ec00e36c0 con 0x7f5ed01037f0 2024-05-21T21:55:45.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:45.829+0000 7f5ece000700 1 --2- 172.21.15.151:0/2265525168 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f5eb40b2a80 0x7f5eb40b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:45.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:45.829+0000 7f5ece000700 1 --2- 172.21.15.151:0/2265525168 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f5eb40b2a80 0x7f5eb40b4f40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc009680 tx=0x7f5ebc009260 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:55:45.831 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:45.830+0000 7f5ec7400700 1 -- 172.21.15.151:0/2265525168 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5ec0030020 con 0x7f5ed01037f0 2024-05-21T21:55:45.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:45 smithi151 ceph-mon[31623]: Updating smithi164:/var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-21T21:55:45.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:45 smithi151 ceph-mon[31623]: Updating smithi184:/var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-21T21:55:45.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:45 smithi151 ceph-mon[31623]: Updating smithi151:/etc/ceph/ceph.client.admin.keyring 2024-05-21T21:55:45.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:45 smithi151 ceph-mon[31623]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 293 B/s wr, 18 op/s 2024-05-21T21:55:45.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:45 smithi151 ceph-mon[31623]: Updating smithi151:/var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-21T21:55:45.954 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:45 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T21:55:45.954 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:45 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T21:55:45.954 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:45 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T21:55:45.954 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:45 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:55:45.954 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:45 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:55:45.954 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:45 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:55:45.954 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:45 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T21:55:45.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:45 smithi184 ceph-mon[36942]: Updating smithi164:/var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-21T21:55:45.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:45 smithi184 ceph-mon[36942]: Updating smithi184:/var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-21T21:55:45.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:45 smithi184 ceph-mon[36942]: Updating smithi151:/etc/ceph/ceph.client.admin.keyring 2024-05-21T21:55:45.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:45 smithi184 ceph-mon[36942]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 293 B/s wr, 18 op/s 2024-05-21T21:55:45.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:45 smithi184 ceph-mon[36942]: Updating smithi151:/var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-21T21:55:45.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:45 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T21:55:45.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:45 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T21:55:45.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:45 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T21:55:45.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:45 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:55:45.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:45 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:55:45.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:45 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:55:45.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:45 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T21:55:46.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:45 smithi164 ceph-mon[36269]: Updating smithi164:/var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-21T21:55:46.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:45 smithi164 ceph-mon[36269]: Updating smithi184:/var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-21T21:55:46.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:45 smithi164 ceph-mon[36269]: Updating smithi151:/etc/ceph/ceph.client.admin.keyring 2024-05-21T21:55:46.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:45 smithi164 ceph-mon[36269]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 293 B/s wr, 18 op/s 2024-05-21T21:55:46.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:45 smithi164 ceph-mon[36269]: Updating smithi151:/var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/config/ceph.client.admin.keyring 2024-05-21T21:55:46.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:45 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T21:55:46.037 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:45 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T21:55:46.037 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:45 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T21:55:46.037 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:45 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:55:46.037 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:45 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:55:46.037 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:45 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:55:46.037 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:45 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T21:55:46.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:46.116+0000 7f5ed5600700 1 -- 172.21.15.151:0/2265525168 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5ed019af00 con 0x7f5ed01037f0 2024-05-21T21:55:46.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:46.118+0000 7f5ec7400700 1 -- 172.21.15.151:0/2265525168 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11958 (secure 0 0 0) 0x7f5ec0013070 con 0x7f5ed01037f0 2024-05-21T21:55:46.119 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T21:55:46.120 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14358,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/158010529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":158010529},{"type":"v1","addr":"172.21.15.151:6835","nonce":158010529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6839/2069058115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":2069058115},{"type":"v1","addr":"172.21.15.151:6839","nonce":2069058115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T21:51:19.552069+0000","modified":"2024-05-21T21:55:34.643755+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24530,"mds_1":24493,"mds_2":24491,"mds_3":24478,"mds_4":24473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14403":{"gid":14403,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6841/1749723794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6840","nonce":1749723794},{"type":"v1","addr":"172.21.15.151:6841","nonce":1749723794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/2026266114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":2026266114},{"type":"v1","addr":"172.21.15.184:6833","nonce":2026266114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/2467900844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":2467900844},{"type":"v1","addr":"172.21.15.164:6833","nonce":2467900844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3367640960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3367640960},{"type":"v1","addr":"172.21.15.151:6837","nonce":3367640960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/3264142621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3264142621},{"type":"v1","addr":"172.21.15.184:6835","nonce":3264142621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":17,"addr":"172.21.15.164:6835/2340428470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2340428470},{"type":"v1","addr":"172.21.15.164:6835","nonce":2340428470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":16,"addr":"172.21.15.184:6837/2988194888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":2988194888},{"type":"v1","addr":"172.21.15.184:6837","nonce":2988194888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.164:6837/1554609572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1554609572},{"type":"v1","addr":"172.21.15.164:6837","nonce":1554609572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.184:6839/3059935143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":3059935143},{"type":"v1","addr":"172.21.15.184:6839","nonce":3059935143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.164:6839/2646401247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":2646401247},{"type":"v1","addr":"172.21.15.164:6839","nonce":2646401247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24530,"qdb_cluster":[24530,24493,24491,24473,24478]},"id":1}]} 2024-05-21T21:55:46.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:46.121+0000 7f5ed5600700 1 -- 172.21.15.151:0/2265525168 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f5eb40b2a80 msgr2=0x7f5eb40b4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:46.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:46.121+0000 7f5ed5600700 1 --2- 172.21.15.151:0/2265525168 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f5eb40b2a80 0x7f5eb40b4f40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc009680 tx=0x7f5ebc009260 comp rx=0 tx=0).stop 2024-05-21T21:55:46.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:46.121+0000 7f5ed5600700 1 -- 172.21.15.151:0/2265525168 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ed01037f0 msgr2=0x7f5ed019f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:46.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:46.121+0000 7f5ed5600700 1 --2- 172.21.15.151:0/2265525168 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ed01037f0 0x7f5ed019f810 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5ec0006010 tx=0x7f5ec0005280 comp rx=0 tx=0).stop 2024-05-21T21:55:46.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:46.121+0000 7f5ed5600700 1 -- 172.21.15.151:0/2265525168 shutdown_connections 2024-05-21T21:55:46.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:46.121+0000 7f5ed5600700 1 --2- 172.21.15.151:0/2265525168 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f5eb40b2a80 0x7f5eb40b4f40 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:46.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:46.121+0000 7f5ed5600700 1 --2- 172.21.15.151:0/2265525168 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5ed01088f0 0x7f5ed01999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:46.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:46.121+0000 7f5ed5600700 1 --2- 172.21.15.151:0/2265525168 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ed01037f0 0x7f5ed019f810 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:46.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:46.121+0000 7f5ed5600700 1 --2- 172.21.15.151:0/2265525168 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ed01041a0 0x7f5ed019fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:46.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:46.121+0000 7f5ed5600700 1 -- 172.21.15.151:0/2265525168 >> 172.21.15.151:0/2265525168 conn(0x7f5ed007a6a0 msgr2=0x7f5ed0101750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:55:46.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:46.121+0000 7f5ed5600700 1 -- 172.21.15.151:0/2265525168 shutdown_connections 2024-05-21T21:55:46.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:46.121+0000 7f5ed5600700 1 -- 172.21.15.151:0/2265525168 wait complete. 2024-05-21T21:55:46.124 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 29 2024-05-21T21:55:46.870 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:46 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/2265525168' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T21:55:46.870 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:46 smithi151 ceph-mon[31623]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 228 B/s wr, 14 op/s 2024-05-21T21:55:46.928 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph osd dump --format=json 2024-05-21T21:55:46.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:46 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/2265525168' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T21:55:46.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:46 smithi184 ceph-mon[36942]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 228 B/s wr, 14 op/s 2024-05-21T21:55:47.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:46 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/2265525168' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T21:55:47.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:46 smithi164 ceph-mon[36269]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 228 B/s wr, 14 op/s 2024-05-21T21:55:47.371 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:55:49.531 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:49.529+0000 7f00e9800700 1 -- 172.21.15.151:0/542647921 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f00e4105670 msgr2=0x7f00e4105af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:49.531 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:49.529+0000 7f00e9800700 1 --2- 172.21.15.151:0/542647921 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f00e4105670 0x7f00e4105af0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f00d0009990 tx=0x7f00d0009ca0 comp rx=0 tx=0).stop 2024-05-21T21:55:49.531 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:49.529+0000 7f00e9800700 1 -- 172.21.15.151:0/542647921 shutdown_connections 2024-05-21T21:55:49.531 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:49.529+0000 7f00e9800700 1 --2- 172.21.15.151:0/542647921 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00e4106030 0x7f00e4112540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:49.531 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:49.529+0000 7f00e9800700 1 --2- 172.21.15.151:0/542647921 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f00e410b670 0x7f00e410ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:49.531 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:49.529+0000 7f00e9800700 1 --2- 172.21.15.151:0/542647921 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f00e4105670 0x7f00e4105af0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:49.531 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:49.529+0000 7f00e9800700 1 -- 172.21.15.151:0/542647921 >> 172.21.15.151:0/542647921 conn(0x7f00e4101160 msgr2=0x7f00e4103580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:55:49.531 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:49.529+0000 7f00e9800700 1 -- 172.21.15.151:0/542647921 shutdown_connections 2024-05-21T21:55:49.532 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:49.529+0000 7f00e9800700 1 -- 172.21.15.151:0/542647921 wait complete. 2024-05-21T21:55:49.532 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:49 smithi151 ceph-mon[31623]: pgmap v9: 97 pgs: 97 active+clean; 1.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 186 B/s wr, 16 op/s 2024-05-21T21:55:49.533 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:49.530+0000 7f00e9800700 1 Processor -- start 2024-05-21T21:55:49.533 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:49.530+0000 7f00e9800700 1 -- start start 2024-05-21T21:55:49.533 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:49.531+0000 7f00e9800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f00e4105670 0x7f00e419bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:49.533 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:49.531+0000 7f00e9800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00e4106030 0x7f00e419c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:49.533 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:49.531+0000 7f00e9800700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f00e410b670 0x7f00e41a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:49.534 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:49.531+0000 7f00e9800700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f00e41158c0 con 0x7f00e4105670 2024-05-21T21:55:49.534 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:49.531+0000 7f00e9800700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f00e4115740 con 0x7f00e4106030 2024-05-21T21:55:49.534 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:49.531+0000 7f00e9800700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f00e4115a40 con 0x7f00e410b670 2024-05-21T21:55:49.534 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:49.531+0000 7f00e8e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f00e410b670 0x7f00e41a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:49.534 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:49.531+0000 7f00e3e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f00e4105670 0x7f00e419bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:49.534 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:49.531+0000 7f00e8e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f00e410b670 0x7f00e41a0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:39106/0 (socket says 172.21.15.151:39106) 2024-05-21T21:55:49.534 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:49.531+0000 7f00e8e00700 1 -- 172.21.15.151:0/923556729 learned_addr learned my addr 172.21.15.151:0/923556729 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:55:49.534 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:49.531+0000 7f00e3400700 1 --2- 172.21.15.151:0/923556729 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00e4106030 0x7f00e419c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:49.534 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:49.532+0000 7f00e8e00700 1 -- 172.21.15.151:0/923556729 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00e4106030 msgr2=0x7f00e419c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:49.534 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:49.532+0000 7f00e8e00700 1 --2- 172.21.15.151:0/923556729 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00e4106030 0x7f00e419c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:49.534 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:49.532+0000 7f00e8e00700 1 -- 172.21.15.151:0/923556729 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f00e4105670 msgr2=0x7f00e419bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:49.535 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:49.532+0000 7f00e8e00700 1 --2- 172.21.15.151:0/923556729 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f00e4105670 0x7f00e419bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:49.535 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:49.532+0000 7f00e8e00700 1 -- 172.21.15.151:0/923556729 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00e41a0da0 con 0x7f00e410b670 2024-05-21T21:55:49.535 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:49.532+0000 7f00e3e00700 1 --2- 172.21.15.151:0/923556729 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f00e4105670 0x7f00e419bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T21:55:49.535 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:49.533+0000 7f00e8e00700 1 --2- 172.21.15.151:0/923556729 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f00e410b670 0x7f00e41a0660 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f00dc007ee0 tx=0x7f00dc00d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:55:49.536 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:49.533+0000 7f00e0c00700 1 -- 172.21.15.151:0/923556729 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00dc00ddd0 con 0x7f00e410b670 2024-05-21T21:55:49.536 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:49.533+0000 7f00e9800700 1 -- 172.21.15.151:0/923556729 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00e41a1090 con 0x7f00e410b670 2024-05-21T21:55:49.536 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:49.533+0000 7f00e9800700 1 -- 172.21.15.151:0/923556729 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00e41a8910 con 0x7f00e410b670 2024-05-21T21:55:49.536 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:49.533+0000 7f00e0c00700 1 -- 172.21.15.151:0/923556729 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00dc004020 con 0x7f00e410b670 2024-05-21T21:55:49.536 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:49.533+0000 7f00e0c00700 1 -- 172.21.15.151:0/923556729 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00dc015660 con 0x7f00e410b670 2024-05-21T21:55:49.536 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:49.534+0000 7f00e9800700 1 -- 172.21.15.151:0/923556729 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00e406a230 con 0x7f00e410b670 2024-05-21T21:55:49.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:49 smithi164 ceph-mon[36269]: pgmap v9: 97 pgs: 97 active+clean; 1.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 186 B/s wr, 16 op/s 2024-05-21T21:55:49.538 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:49.537+0000 7f00e0c00700 1 -- 172.21.15.151:0/923556729 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 25) ==== 149829+0+0 (secure 0 0 0) 0x7f00dc0044b0 con 0x7f00e410b670 2024-05-21T21:55:49.539 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:49.538+0000 7f00e0c00700 1 --2- 172.21.15.151:0/923556729 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f00c80b2b00 0x7f00c80b4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:49.539 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:49.538+0000 7f00e0c00700 1 -- 172.21.15.151:0/923556729 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f00dc0e29e0 con 0x7f00e410b670 2024-05-21T21:55:49.540 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:49.539+0000 7f00e3e00700 1 --2- 172.21.15.151:0/923556729 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f00c80b2b00 0x7f00c80b4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:49.540 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:49.539+0000 7f00e3e00700 1 --2- 172.21.15.151:0/923556729 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f00c80b2b00 0x7f00c80b4fc0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f00d4009c90 tx=0x7f00d4009260 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:55:49.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:49.540+0000 7f00e0c00700 1 -- 172.21.15.151:0/923556729 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f00dc023070 con 0x7f00e410b670 2024-05-21T21:55:49.735 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:49 smithi184 ceph-mon[36942]: pgmap v9: 97 pgs: 97 active+clean; 1.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 186 B/s wr, 16 op/s 2024-05-21T21:55:49.776 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:49.775+0000 7f00e9800700 1 -- 172.21.15.151:0/923556729 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f00e402e560 con 0x7f00e410b670 2024-05-21T21:55:49.778 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:49.777+0000 7f00e0c00700 1 -- 172.21.15.151:0/923556729 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+23128 (secure 0 0 0) 0x7f00dc010070 con 0x7f00e410b670 2024-05-21T21:55:49.779 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T21:55:49.779 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":83,"fsid":"c8c7a2e6-17ba-11ef-bc9a-c7b262605968","created":"2024-05-21T21:41:52.412833+0000","modified":"2024-05-21T21:55:36.189583+0000","last_up_change":"2024-05-21T21:50:05.401314+0000","last_in_change":"2024-05-21T21:49:45.684136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-21T21:45:55.196046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-21T21:51:11.447428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-21T21:51:15.618605+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"18db8e54-0cfc-4e12-9c84-6f72a99cc989","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":2420400473},{"type":"v1","addr":"172.21.15.151:6803","nonce":2420400473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":2420400473},{"type":"v1","addr":"172.21.15.151:6805","nonce":2420400473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":2420400473},{"type":"v1","addr":"172.21.15.151:6809","nonce":2420400473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":2420400473},{"type":"v1","addr":"172.21.15.151:6807","nonce":2420400473}]},"public_addr":"172.21.15.151:6803/2420400473","cluster_addr":"172.21.15.151:6805/2420400473","heartbeat_back_addr":"172.21.15.151:6809/2420400473","heartbeat_front_addr":"172.21.15.151:6807/2420400473","state":["exists","up"]},{"osd":1,"uuid":"bac384c7-1102-4815-81a8-10cc7661d230","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":3900240447},{"type":"v1","addr":"172.21.15.164:6801","nonce":3900240447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":3900240447},{"type":"v1","addr":"172.21.15.164:6803","nonce":3900240447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":3900240447},{"type":"v1","addr":"172.21.15.164:6807","nonce":3900240447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":3900240447},{"type":"v1","addr":"172.21.15.164:6805","nonce":3900240447}]},"public_addr":"172.21.15.164:6801/3900240447","cluster_addr":"172.21.15.164:6803/3900240447","heartbeat_back_addr":"172.21.15.164:6807/3900240447","heartbeat_front_addr":"172.21.15.164:6805/3900240447","state":["exists","up"]},{"osd":2,"uuid":"b7e4a782-1518-4d43-bee9-7fd768ee5cb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":3971150737},{"type":"v1","addr":"172.21.15.184:6801","nonce":3971150737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3971150737},{"type":"v1","addr":"172.21.15.184:6803","nonce":3971150737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3971150737},{"type":"v1","addr":"172.21.15.184:6807","nonce":3971150737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3971150737},{"type":"v1","addr":"172.21.15.184:6805","nonce":3971150737}]},"public_addr":"172.21.15.184:6801/3971150737","cluster_addr":"172.21.15.184:6803/3971150737","heartbeat_back_addr":"172.21.15.184:6807/3971150737","heartbeat_front_addr":"172.21.15.184:6805/3971150737","state":["exists","up"]},{"osd":3,"uuid":"ab3df826-39e3-4635-a49c-2b5ffdbf12f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":556594400},{"type":"v1","addr":"172.21.15.151:6811","nonce":556594400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":556594400},{"type":"v1","addr":"172.21.15.151:6813","nonce":556594400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":556594400},{"type":"v1","addr":"172.21.15.151:6817","nonce":556594400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":556594400},{"type":"v1","addr":"172.21.15.151:6815","nonce":556594400}]},"public_addr":"172.21.15.151:6811/556594400","cluster_addr":"172.21.15.151:6813/556594400","heartbeat_back_addr":"172.21.15.151:6817/556594400","heartbeat_front_addr":"172.21.15.151:6815/556594400","state":["exists","up"]},{"osd":4,"uuid":"00cbf6ae-0f42-4685-a77d-d803c6467aa0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":988881973},{"type":"v1","addr":"172.21.15.164:6809","nonce":988881973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":988881973},{"type":"v1","addr":"172.21.15.164:6811","nonce":988881973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":988881973},{"type":"v1","addr":"172.21.15.164:6815","nonce":988881973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":988881973},{"type":"v1","addr":"172.21.15.164:6813","nonce":988881973}]},"public_addr":"172.21.15.164:6809/988881973","cluster_addr":"172.21.15.164:6811/988881973","heartbeat_back_addr":"172.21.15.164:6815/988881973","heartbeat_front_addr":"172.21.15.164:6813/988881973","state":["exists","up"]},{"osd":5,"uuid":"fad14221-0ef0-4d40-b414-c6c9f1f9718c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3293311106},{"type":"v1","addr":"172.21.15.184:6809","nonce":3293311106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":3293311106},{"type":"v1","addr":"172.21.15.184:6811","nonce":3293311106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":3293311106},{"type":"v1","addr":"172.21.15.184:6815","nonce":3293311106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":3293311106},{"type":"v1","addr":"172.21.15.184:6813","nonce":3293311106}]},"public_addr":"172.21.15.184:6809/3293311106","cluster_addr":"172.21.15.184:6811/3293311106","heartbeat_back_addr":"172.21.15.184:6815/3293311106","heartbeat_front_addr":"172.21.15.184:6813/3293311106","state":["exists","up"]},{"osd":6,"uuid":"ab8a60f6-0dec-4c1c-89db-f2e2f638f1ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":3377550762},{"type":"v1","addr":"172.21.15.151:6819","nonce":3377550762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":3377550762},{"type":"v1","addr":"172.21.15.151:6821","nonce":3377550762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":3377550762},{"type":"v1","addr":"172.21.15.151:6825","nonce":3377550762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":3377550762},{"type":"v1","addr":"172.21.15.151:6823","nonce":3377550762}]},"public_addr":"172.21.15.151:6819/3377550762","cluster_addr":"172.21.15.151:6821/3377550762","heartbeat_back_addr":"172.21.15.151:6825/3377550762","heartbeat_front_addr":"172.21.15.151:6823/3377550762","state":["exists","up"]},{"osd":7,"uuid":"db332268-9972-4ed7-a307-822b83a0fbb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":3869696936},{"type":"v1","addr":"172.21.15.164:6817","nonce":3869696936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":3869696936},{"type":"v1","addr":"172.21.15.164:6819","nonce":3869696936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":3869696936},{"type":"v1","addr":"172.21.15.164:6823","nonce":3869696936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":3869696936},{"type":"v1","addr":"172.21.15.164:6821","nonce":3869696936}]},"public_addr":"172.21.15.164:6817/3869696936","cluster_addr":"172.21.15.164:6819/3869696936","heartbeat_back_addr":"172.21.15.164:6823/3869696936","heartbeat_front_addr":"172.21.15.164:6821/3869696936","state":["exists","up"]},{"osd":8,"uuid":"fe2fbe1c-549a-4947-a188-426dfca04d10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":575273541},{"type":"v1","addr":"172.21.15.184:6817","nonce":575273541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":575273541},{"type":"v1","addr":"172.21.15.184:6819","nonce":575273541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":575273541},{"type":"v1","addr":"172.21.15.184:6823","nonce":575273541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":575273541},{"type":"v1","addr":"172.21.15.184:6821","nonce":575273541}]},"public_addr":"172.21.15.184:6817/575273541","cluster_addr":"172.21.15.184:6819/575273541","heartbeat_back_addr":"172.21.15.184:6823/575273541","heartbeat_front_addr":"172.21.15.184:6821/575273541","state":["exists","up"]},{"osd":9,"uuid":"3a7b450f-8004-4adf-bae3-970918f98f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":337128533},{"type":"v1","addr":"172.21.15.151:6827","nonce":337128533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":337128533},{"type":"v1","addr":"172.21.15.151:6829","nonce":337128533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":337128533},{"type":"v1","addr":"172.21.15.151:6833","nonce":337128533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":337128533},{"type":"v1","addr":"172.21.15.151:6831","nonce":337128533}]},"public_addr":"172.21.15.151:6827/337128533","cluster_addr":"172.21.15.151:6829/337128533","heartbeat_back_addr":"172.21.15.151:6833/337128533","heartbeat_front_addr":"172.21.15.151:6831/337128533","state":["exists","up"]},{"osd":10,"uuid":"8fcdf271-90c4-4db3-be69-b0ca716dfd80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":1823448782},{"type":"v1","addr":"172.21.15.164:6825","nonce":1823448782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":1823448782},{"type":"v1","addr":"172.21.15.164:6827","nonce":1823448782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":1823448782},{"type":"v1","addr":"172.21.15.164:6831","nonce":1823448782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":1823448782},{"type":"v1","addr":"172.21.15.164:6829","nonce":1823448782}]},"public_addr":"172.21.15.164:6825/1823448782","cluster_addr":"172.21.15.164:6827/1823448782","heartbeat_back_addr":"172.21.15.164:6831/1823448782","heartbeat_front_addr":"172.21.15.164:6829/1823448782","state":["exists","up"]},{"osd":11,"uuid":"e661477b-0c09-409f-a2fe-6ebd34642a8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":1563340128},{"type":"v1","addr":"172.21.15.184:6825","nonce":1563340128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":1563340128},{"type":"v1","addr":"172.21.15.184:6827","nonce":1563340128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":1563340128},{"type":"v1","addr":"172.21.15.184:6831","nonce":1563340128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":1563340128},{"type":"v1","addr":"172.21.15.184:6829","nonce":1563340128}]},"public_addr":"172.21.15.184:6825/1563340128","cluster_addr":"172.21.15.184:6827/1563340128","heartbeat_back_addr":"172.21.15.184:6831/1563340128","heartbeat_front_addr":"172.21.15.184:6829/1563340128","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:45:02.821160+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:45:27.844564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:45:51.904981+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:46:24.376245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:46:50.036621+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:47:14.748422+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:47:47.396383+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:48:13.543174+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:48:38.213285+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:49:11.980042+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:49:38.519761+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:50:03.439898+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.151:0/3481112063":"2024-05-22T21:55:36.189531+0000","172.21.15.151:0/2415279581":"2024-05-22T21:55:36.189531+0000","172.21.15.151:0/1946920088":"2024-05-22T21:55:36.189531+0000","172.21.15.151:6800/2796622483":"2024-05-22T21:55:36.189531+0000","172.21.15.151:0/31963128":"2024-05-22T21:43:09.041455+0000","172.21.15.151:6801/2796622483":"2024-05-22T21:55:36.189531+0000","172.21.15.151:6800/3428827300":"2024-05-22T21:43:09.041455+0000","172.21.15.151:0/3232339048":"2024-05-22T21:55:36.189531+0000","172.21.15.151:6801/3428827300":"2024-05-22T21:43:09.041455+0000","172.21.15.151:0/131989141":"2024-05-22T21:43:09.041455+0000","172.21.15.151:6801/1128885292":"2024-05-22T21:42:35.440147+0000","172.21.15.151:0/2703664519":"2024-05-22T21:43:09.041455+0000","172.21.15.151:6800/1128885292":"2024-05-22T21:42:35.440147+0000","172.21.15.151:0/3052943699":"2024-05-22T21:42:35.440147+0000","172.21.15.151:0/3929452511":"2024-05-22T21:42:35.440147+0000","172.21.15.151:0/3762889700":"2024-05-22T21:42:35.440147+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-21T21:55:49.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:49.780+0000 7f00e9800700 1 -- 172.21.15.151:0/923556729 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f00c80b2b00 msgr2=0x7f00c80b4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:49.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:49.780+0000 7f00e9800700 1 --2- 172.21.15.151:0/923556729 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f00c80b2b00 0x7f00c80b4fc0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f00d4009c90 tx=0x7f00d4009260 comp rx=0 tx=0).stop 2024-05-21T21:55:49.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:49.780+0000 7f00e9800700 1 -- 172.21.15.151:0/923556729 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f00e410b670 msgr2=0x7f00e41a0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:49.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:49.780+0000 7f00e9800700 1 --2- 172.21.15.151:0/923556729 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f00e410b670 0x7f00e41a0660 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f00dc007ee0 tx=0x7f00dc00d3b0 comp rx=0 tx=0).stop 2024-05-21T21:55:49.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:49.780+0000 7f00e9800700 1 -- 172.21.15.151:0/923556729 shutdown_connections 2024-05-21T21:55:49.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:49.780+0000 7f00e9800700 1 --2- 172.21.15.151:0/923556729 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f00c80b2b00 0x7f00c80b4fc0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:49.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:49.780+0000 7f00e9800700 1 --2- 172.21.15.151:0/923556729 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f00e410b670 0x7f00e41a0660 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:49.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:49.780+0000 7f00e9800700 1 --2- 172.21.15.151:0/923556729 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f00e4105670 0x7f00e419bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:49.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:49.780+0000 7f00e9800700 1 --2- 172.21.15.151:0/923556729 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00e4106030 0x7f00e419c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:49.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:49.780+0000 7f00e9800700 1 -- 172.21.15.151:0/923556729 >> 172.21.15.151:0/923556729 conn(0x7f00e4101160 msgr2=0x7f00e4101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:55:49.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:49.781+0000 7f00e9800700 1 -- 172.21.15.151:0/923556729 shutdown_connections 2024-05-21T21:55:49.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:49.781+0000 7f00e9800700 1 -- 172.21.15.151:0/923556729 wait complete. 2024-05-21T21:55:50.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:50 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/923556729' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T21:55:50.586 DEBUG:tasks.fwd_scrub:Yielding 2024-05-21T21:55:50.586 INFO:teuthology.run_tasks:Running task exec... 2024-05-21T21:55:50.595 INFO:teuthology.task.exec:Executing custom commands... 2024-05-21T21:55:50.595 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi151.front.sepia.ceph.com 2024-05-21T21:55:50.595 DEBUG:teuthology.orchestra.run.smithi151:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/log_level debug' 2024-05-21T21:55:50.596 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-05-21T21:55:50.596 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-21T21:55:50.596 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-21T21:55:50.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:50 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/923556729' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T21:55:50.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:50 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/923556729' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T21:55:50.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:50.786+0000 7f2fd8400640 1 -- 172.21.15.151:0/784309073 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2fd0104ea0 msgr2=0x7f2fd0105280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:50.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:50.786+0000 7f2fd8400640 1 --2- 172.21.15.151:0/784309073 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2fd0104ea0 0x7f2fd0105280 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2fc4009860 tx=0x7f2fc402a340 comp rx=0 tx=0).stop 2024-05-21T21:55:50.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:50.787+0000 7f2fd8400640 1 -- 172.21.15.151:0/784309073 shutdown_connections 2024-05-21T21:55:50.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:50.787+0000 7f2fd8400640 1 --2- 172.21.15.151:0/784309073 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2fd0109f70 0x7f2fd0111b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:50.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:50.787+0000 7f2fd8400640 1 --2- 172.21.15.151:0/784309073 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2fd0105850 0x7f2fd0109890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:50.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:50.787+0000 7f2fd8400640 1 --2- 172.21.15.151:0/784309073 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2fd0104ea0 0x7f2fd0105280 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:50.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:50.787+0000 7f2fd8400640 1 -- 172.21.15.151:0/784309073 >> 172.21.15.151:0/784309073 conn(0x7f2fd0100740 msgr2=0x7f2fd0102b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:55:50.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:50.787+0000 7f2fd8400640 1 -- 172.21.15.151:0/784309073 shutdown_connections 2024-05-21T21:55:50.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:50.787+0000 7f2fd8400640 1 -- 172.21.15.151:0/784309073 wait complete. 2024-05-21T21:55:50.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:50.788+0000 7f2fd8400640 1 Processor -- start 2024-05-21T21:55:50.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:50.788+0000 7f2fd8400640 1 -- start start 2024-05-21T21:55:50.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:50.789+0000 7f2fd8400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2fd0104ea0 0x7f2fd019b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:50.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:50.789+0000 7f2fd8400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2fd0109f70 0x7f2fd019b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:50.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:50.789+0000 7f2fd8400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2fd019fc20 0x7f2fd01a00d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:50.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:50.789+0000 7f2fd8400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd0114ed0 con 0x7f2fd0109f70 2024-05-21T21:55:50.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:50.789+0000 7f2fd8400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd0114d50 con 0x7f2fd0104ea0 2024-05-21T21:55:50.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:50.789+0000 7f2fd8400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd0115050 con 0x7f2fd019fc20 2024-05-21T21:55:50.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:50.789+0000 7f2fd5000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2fd0109f70 0x7f2fd019b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:50.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:50.789+0000 7f2fd5000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2fd0109f70 0x7f2fd019b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41990/0 (socket says 172.21.15.151:41990) 2024-05-21T21:55:50.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:50.789+0000 7f2fd5000640 1 -- 172.21.15.151:0/2942350460 learned_addr learned my addr 172.21.15.151:0/2942350460 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:55:50.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:50.789+0000 7f2fd5a00640 1 --2- 172.21.15.151:0/2942350460 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2fd0104ea0 0x7f2fd019b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:50.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:50.789+0000 7f2fd6400640 1 --2- 172.21.15.151:0/2942350460 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2fd019fc20 0x7f2fd01a00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:50.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:50.790+0000 7f2fd5000640 1 -- 172.21.15.151:0/2942350460 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2fd019fc20 msgr2=0x7f2fd01a00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:50.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:50.790+0000 7f2fd5000640 1 --2- 172.21.15.151:0/2942350460 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2fd019fc20 0x7f2fd01a00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:50.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:50.790+0000 7f2fd5000640 1 -- 172.21.15.151:0/2942350460 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2fd0104ea0 msgr2=0x7f2fd019b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:50.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:50.790+0000 7f2fd5000640 1 --2- 172.21.15.151:0/2942350460 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2fd0104ea0 0x7f2fd019b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:50.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:50.790+0000 7f2fd5000640 1 -- 172.21.15.151:0/2942350460 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fd01a0710 con 0x7f2fd0109f70 2024-05-21T21:55:50.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:50.790+0000 7f2fd5a00640 1 --2- 172.21.15.151:0/2942350460 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2fd0104ea0 0x7f2fd019b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:55:50.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:50.790+0000 7f2fd5000640 1 --2- 172.21.15.151:0/2942350460 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2fd0109f70 0x7f2fd019b8f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc00e9a0 tx=0x7f2fbc00ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:55:50.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:50.790+0000 7f2fce000640 1 -- 172.21.15.151:0/2942350460 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fbc00cd80 con 0x7f2fd0109f70 2024-05-21T21:55:50.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:50.791+0000 7f2fce000640 1 -- 172.21.15.151:0/2942350460 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2fbc0035f0 con 0x7f2fd0109f70 2024-05-21T21:55:50.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:50.791+0000 7f2fd8400640 1 -- 172.21.15.151:0/2942350460 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fd01a0a00 con 0x7f2fd0109f70 2024-05-21T21:55:50.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:50.791+0000 7f2fce000640 1 -- 172.21.15.151:0/2942350460 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fbc010690 con 0x7f2fd0109f70 2024-05-21T21:55:50.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:50.791+0000 7f2fd8400640 1 -- 172.21.15.151:0/2942350460 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fd01a81e0 con 0x7f2fd0109f70 2024-05-21T21:55:50.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:50.792+0000 7f2fd8400640 1 -- 172.21.15.151:0/2942350460 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fd00507e0 con 0x7f2fd0109f70 2024-05-21T21:55:50.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:50.794+0000 7f2fce000640 1 -- 172.21.15.151:0/2942350460 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 25) ==== 149829+0+0 (secure 0 0 0) 0x7f2fbc004000 con 0x7f2fd0109f70 2024-05-21T21:55:50.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:50.797+0000 7f2fce000640 1 --2- 172.21.15.151:0/2942350460 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f2fb80b2b10 0x7f2fb80b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:50.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:50.797+0000 7f2fce000640 1 -- 172.21.15.151:0/2942350460 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f2fbc0e29e0 con 0x7f2fd0109f70 2024-05-21T21:55:50.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:50.797+0000 7f2fd5a00640 1 --2- 172.21.15.151:0/2942350460 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f2fb80b2b10 0x7f2fb80b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:50.799 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:50.798+0000 7f2fd5a00640 1 --2- 172.21.15.151:0/2942350460 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f2fb80b2b10 0x7f2fb80b4fd0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2fc4009830 tx=0x7f2fc4030340 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:55:50.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:50.801+0000 7f2fce000640 1 -- 172.21.15.151:0/2942350460 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2fbc0042f0 con 0x7f2fd0109f70 2024-05-21T21:55:51.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.017+0000 7f2fd8400640 1 -- 172.21.15.151:0/2942350460 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/log_level}] v 0) -- 0x7f2fd010d1a0 con 0x7f2fd0109f70 2024-05-21T21:55:51.023 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.022+0000 7f2fce000640 1 -- 172.21.15.151:0/2942350460 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/log_level}]=0 v27)=0 v27) ==== 129+0+0 (secure 0 0 0) 0x7f2fbc0aa200 con 0x7f2fd0109f70 2024-05-21T21:55:51.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.025+0000 7f2fd8400640 1 -- 172.21.15.151:0/2942350460 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f2fb80b2b10 msgr2=0x7f2fb80b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:51.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.026+0000 7f2fd8400640 1 --2- 172.21.15.151:0/2942350460 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f2fb80b2b10 0x7f2fb80b4fd0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2fc4009830 tx=0x7f2fc4030340 comp rx=0 tx=0).stop 2024-05-21T21:55:51.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.026+0000 7f2fd8400640 1 -- 172.21.15.151:0/2942350460 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2fd0109f70 msgr2=0x7f2fd019b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:51.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.026+0000 7f2fd8400640 1 --2- 172.21.15.151:0/2942350460 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2fd0109f70 0x7f2fd019b8f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc00e9a0 tx=0x7f2fbc00ee60 comp rx=0 tx=0).stop 2024-05-21T21:55:51.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.026+0000 7f2fd8400640 1 -- 172.21.15.151:0/2942350460 shutdown_connections 2024-05-21T21:55:51.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.026+0000 7f2fd8400640 1 --2- 172.21.15.151:0/2942350460 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f2fb80b2b10 0x7f2fb80b4fd0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:51.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.026+0000 7f2fd8400640 1 --2- 172.21.15.151:0/2942350460 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2fd019fc20 0x7f2fd01a00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:51.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.026+0000 7f2fd8400640 1 --2- 172.21.15.151:0/2942350460 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2fd0109f70 0x7f2fd019b8f0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:51.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.026+0000 7f2fd8400640 1 --2- 172.21.15.151:0/2942350460 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2fd0104ea0 0x7f2fd019b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:51.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.026+0000 7f2fd8400640 1 -- 172.21.15.151:0/2942350460 >> 172.21.15.151:0/2942350460 conn(0x7f2fd0100740 msgr2=0x7f2fd0102a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:55:51.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.026+0000 7f2fd8400640 1 -- 172.21.15.151:0/2942350460 shutdown_connections 2024-05-21T21:55:51.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.026+0000 7f2fd8400640 1 -- 172.21.15.151:0/2942350460 wait complete. 2024-05-21T21:55:51.039 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:55:51.047 DEBUG:teuthology.orchestra.run.smithi151:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-05-21T21:55:51.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.230+0000 7fcf52400640 1 -- 172.21.15.151:0/2190025290 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf4c1079b0 msgr2=0x7fcf4c107d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:51.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.230+0000 7fcf52400640 1 --2- 172.21.15.151:0/2190025290 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf4c1079b0 0x7fcf4c107d90 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fcf40009930 tx=0x7fcf4002a420 comp rx=0 tx=0).stop 2024-05-21T21:55:51.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.230+0000 7fcf52400640 1 -- 172.21.15.151:0/2190025290 shutdown_connections 2024-05-21T21:55:51.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.230+0000 7fcf52400640 1 --2- 172.21.15.151:0/2190025290 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcf4c101320 0x7fcf4c10f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:51.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.230+0000 7fcf52400640 1 --2- 172.21.15.151:0/2190025290 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf4c100960 0x7fcf4c100de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:51.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.230+0000 7fcf52400640 1 --2- 172.21.15.151:0/2190025290 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf4c1079b0 0x7fcf4c107d90 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:51.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.230+0000 7fcf52400640 1 -- 172.21.15.151:0/2190025290 >> 172.21.15.151:0/2190025290 conn(0x7fcf4c0778b0 msgr2=0x7fcf4c0fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:55:51.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.230+0000 7fcf52400640 1 -- 172.21.15.151:0/2190025290 shutdown_connections 2024-05-21T21:55:51.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.230+0000 7fcf52400640 1 -- 172.21.15.151:0/2190025290 wait complete. 2024-05-21T21:55:51.233 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.231+0000 7fcf52400640 1 Processor -- start 2024-05-21T21:55:51.233 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.231+0000 7fcf52400640 1 -- start start 2024-05-21T21:55:51.233 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.232+0000 7fcf52400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf4c100960 0x7fcf4c1023d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:51.233 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.232+0000 7fcf52400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf4c101320 0x7fcf4c102910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:51.233 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.232+0000 7fcf52400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcf4c1079b0 0x7fcf4c102e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:51.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.232+0000 7fcf52400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcf4c112b80 con 0x7fcf4c100960 2024-05-21T21:55:51.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.232+0000 7fcf52400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcf4c112a00 con 0x7fcf4c1079b0 2024-05-21T21:55:51.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.232+0000 7fcf52400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fcf4c112d00 con 0x7fcf4c101320 2024-05-21T21:55:51.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.232+0000 7fcf51000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf4c100960 0x7fcf4c1023d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:51.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.232+0000 7fcf51000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf4c100960 0x7fcf4c1023d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:42006/0 (socket says 172.21.15.151:42006) 2024-05-21T21:55:51.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.232+0000 7fcf51000640 1 -- 172.21.15.151:0/3289174534 learned_addr learned my addr 172.21.15.151:0/3289174534 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:55:51.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.232+0000 7fcf51a00640 1 --2- 172.21.15.151:0/3289174534 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcf4c1079b0 0x7fcf4c102e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:51.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.232+0000 7fcf4be00640 1 --2- 172.21.15.151:0/3289174534 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf4c101320 0x7fcf4c102910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:51.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.233+0000 7fcf51000640 1 -- 172.21.15.151:0/3289174534 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf4c101320 msgr2=0x7fcf4c102910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:51.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.233+0000 7fcf51000640 1 --2- 172.21.15.151:0/3289174534 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf4c101320 0x7fcf4c102910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:51.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.233+0000 7fcf51000640 1 -- 172.21.15.151:0/3289174534 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcf4c1079b0 msgr2=0x7fcf4c102e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:51.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.233+0000 7fcf51000640 1 --2- 172.21.15.151:0/3289174534 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcf4c1079b0 0x7fcf4c102e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:51.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.233+0000 7fcf51000640 1 -- 172.21.15.151:0/3289174534 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf4c1a7580 con 0x7fcf4c100960 2024-05-21T21:55:51.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.233+0000 7fcf51a00640 1 --2- 172.21.15.151:0/3289174534 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcf4c1079b0 0x7fcf4c102e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:55:51.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.233+0000 7fcf4be00640 1 --2- 172.21.15.151:0/3289174534 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf4c101320 0x7fcf4c102910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:55:51.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.233+0000 7fcf51000640 1 --2- 172.21.15.151:0/3289174534 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf4c100960 0x7fcf4c1023d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fcf40005ab0 tx=0x7fcf40003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:55:51.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.233+0000 7fcf49600640 1 -- 172.21.15.151:0/3289174534 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf4003f070 con 0x7fcf4c100960 2024-05-21T21:55:51.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.233+0000 7fcf49600640 1 -- 172.21.15.151:0/3289174534 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf40031b50 con 0x7fcf4c100960 2024-05-21T21:55:51.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.233+0000 7fcf49600640 1 -- 172.21.15.151:0/3289174534 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf40035030 con 0x7fcf4c100960 2024-05-21T21:55:51.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.233+0000 7fcf52400640 1 -- 172.21.15.151:0/3289174534 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf4c1a7810 con 0x7fcf4c100960 2024-05-21T21:55:51.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.233+0000 7fcf52400640 1 -- 172.21.15.151:0/3289174534 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf4c1a7cb0 con 0x7fcf4c100960 2024-05-21T21:55:51.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.235+0000 7fcf52400640 1 -- 172.21.15.151:0/3289174534 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf4c051030 con 0x7fcf4c100960 2024-05-21T21:55:51.238 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.237+0000 7fcf49600640 1 -- 172.21.15.151:0/3289174534 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 25) ==== 149829+0+0 (secure 0 0 0) 0x7fcf40042070 con 0x7fcf4c100960 2024-05-21T21:55:51.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.238+0000 7fcf49600640 1 --2- 172.21.15.151:0/3289174534 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fcf340b2a90 0x7fcf340b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:51.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.239+0000 7fcf49600640 1 -- 172.21.15.151:0/3289174534 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fcf40102c50 con 0x7fcf4c100960 2024-05-21T21:55:51.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.239+0000 7fcf4be00640 1 --2- 172.21.15.151:0/3289174534 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fcf340b2a90 0x7fcf340b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:51.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.239+0000 7fcf4be00640 1 --2- 172.21.15.151:0/3289174534 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fcf340b2a90 0x7fcf340b4f50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c103e10 tx=0x7fcf38005f90 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:55:51.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.243+0000 7fcf49600640 1 -- 172.21.15.151:0/3289174534 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcf400c9e20 con 0x7fcf4c100960 2024-05-21T21:55:51.383 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:51 smithi151 ceph-mon[31623]: pgmap v10: 97 pgs: 97 active+clean; 1.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 170 B/s wr, 14 op/s 2024-05-21T21:55:51.383 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:51 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/2942350460' entity='client.admin' 2024-05-21T21:55:51.383 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:51 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:55:51.383 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:51 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:55:51.383 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:51 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:55:51.383 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:51 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T21:55:51.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.459+0000 7fcf52400640 1 -- 172.21.15.151:0/3289174534 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7fcf4c02dba0 con 0x7fcf4c100960 2024-05-21T21:55:51.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.467+0000 7fcf49600640 1 -- 172.21.15.151:0/3289174534 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v28)=0 v28) ==== 138+0+0 (secure 0 0 0) 0x7fcf400ce120 con 0x7fcf4c100960 2024-05-21T21:55:51.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.470+0000 7fcf52400640 1 -- 172.21.15.151:0/3289174534 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fcf340b2a90 msgr2=0x7fcf340b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:51.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.470+0000 7fcf52400640 1 --2- 172.21.15.151:0/3289174534 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fcf340b2a90 0x7fcf340b4f50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c103e10 tx=0x7fcf38005f90 comp rx=0 tx=0).stop 2024-05-21T21:55:51.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.470+0000 7fcf52400640 1 -- 172.21.15.151:0/3289174534 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf4c100960 msgr2=0x7fcf4c1023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:51.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.470+0000 7fcf52400640 1 --2- 172.21.15.151:0/3289174534 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf4c100960 0x7fcf4c1023d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fcf40005ab0 tx=0x7fcf40003b90 comp rx=0 tx=0).stop 2024-05-21T21:55:51.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.470+0000 7fcf52400640 1 -- 172.21.15.151:0/3289174534 shutdown_connections 2024-05-21T21:55:51.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.470+0000 7fcf52400640 1 --2- 172.21.15.151:0/3289174534 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fcf340b2a90 0x7fcf340b4f50 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:51.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.470+0000 7fcf52400640 1 --2- 172.21.15.151:0/3289174534 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcf4c1079b0 0x7fcf4c102e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:51.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.470+0000 7fcf52400640 1 --2- 172.21.15.151:0/3289174534 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf4c101320 0x7fcf4c102910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:51.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.470+0000 7fcf52400640 1 --2- 172.21.15.151:0/3289174534 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf4c100960 0x7fcf4c1023d0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:51.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.470+0000 7fcf52400640 1 -- 172.21.15.151:0/3289174534 >> 172.21.15.151:0/3289174534 conn(0x7fcf4c0778b0 msgr2=0x7fcf4c0fcee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:55:51.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.471+0000 7fcf52400640 1 -- 172.21.15.151:0/3289174534 shutdown_connections 2024-05-21T21:55:51.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.471+0000 7fcf52400640 1 -- 172.21.15.151:0/3289174534 wait complete. 2024-05-21T21:55:51.493 DEBUG:teuthology.orchestra.run.smithi151:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-05-21T21:55:51.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.671+0000 7f4188c00640 1 -- 172.21.15.151:0/3929472971 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4184108860 msgr2=0x7f4184108c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:51.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.671+0000 7f4188c00640 1 --2- 172.21.15.151:0/3929472971 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4184108860 0x7f4184108c40 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4174009cd0 tx=0x7f417402a8f0 comp rx=0 tx=0).stop 2024-05-21T21:55:51.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.672+0000 7f4188c00640 1 -- 172.21.15.151:0/3929472971 shutdown_connections 2024-05-21T21:55:51.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.672+0000 7f4188c00640 1 --2- 172.21.15.151:0/3929472971 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4184103210 0x7f418410f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:51.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.672+0000 7f4188c00640 1 --2- 172.21.15.151:0/3929472971 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4184102850 0x7f4184102cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:51.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.672+0000 7f4188c00640 1 --2- 172.21.15.151:0/3929472971 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4184108860 0x7f4184108c40 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:51.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.672+0000 7f4188c00640 1 -- 172.21.15.151:0/3929472971 >> 172.21.15.151:0/3929472971 conn(0x7f41840fe3a0 msgr2=0x7f41841007c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:55:51.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.672+0000 7f4188c00640 1 -- 172.21.15.151:0/3929472971 shutdown_connections 2024-05-21T21:55:51.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.672+0000 7f4188c00640 1 -- 172.21.15.151:0/3929472971 wait complete. 2024-05-21T21:55:51.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.673+0000 7f4188c00640 1 Processor -- start 2024-05-21T21:55:51.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.673+0000 7f4188c00640 1 -- start start 2024-05-21T21:55:51.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.673+0000 7f4188c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4184102850 0x7f418419b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:51.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.673+0000 7f4188c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4184103210 0x7f418419b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:51.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.673+0000 7f4188c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4184108860 0x7f418419fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:51.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.673+0000 7f4188c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4184112b20 con 0x7f4184108860 2024-05-21T21:55:51.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.673+0000 7f4188c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f41841129a0 con 0x7f4184102850 2024-05-21T21:55:51.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.673+0000 7f4188c00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4184112ca0 con 0x7f4184103210 2024-05-21T21:55:51.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.673+0000 7f4183400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4184102850 0x7f418419b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:51.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.674+0000 7f4183400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4184102850 0x7f418419b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:43142/0 (socket says 172.21.15.151:43142) 2024-05-21T21:55:51.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.674+0000 7f4183400640 1 -- 172.21.15.151:0/2238351186 learned_addr learned my addr 172.21.15.151:0/2238351186 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:55:51.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.674+0000 7f417aa00640 1 --2- 172.21.15.151:0/2238351186 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4184103210 0x7f418419b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:51.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.674+0000 7f4183e00640 1 --2- 172.21.15.151:0/2238351186 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4184108860 0x7f418419fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:51.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.674+0000 7f4183400640 1 -- 172.21.15.151:0/2238351186 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4184103210 msgr2=0x7f418419b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:51.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.674+0000 7f4183400640 1 --2- 172.21.15.151:0/2238351186 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4184103210 0x7f418419b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:51.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.674+0000 7f4183400640 1 -- 172.21.15.151:0/2238351186 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4184108860 msgr2=0x7f418419fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:51.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.674+0000 7f4183400640 1 --2- 172.21.15.151:0/2238351186 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4184108860 0x7f418419fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:51.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.674+0000 7f4183400640 1 -- 172.21.15.151:0/2238351186 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41841a02d0 con 0x7f4184102850 2024-05-21T21:55:51.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.674+0000 7f4183400640 1 --2- 172.21.15.151:0/2238351186 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4184102850 0x7f418419b2e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4174006010 tx=0x7f4174030cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:55:51.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.675+0000 7f4180c00640 1 -- 172.21.15.151:0/2238351186 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4174004040 con 0x7f4184102850 2024-05-21T21:55:51.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.675+0000 7f4180c00640 1 -- 172.21.15.151:0/2238351186 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4174040070 con 0x7f4184102850 2024-05-21T21:55:51.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.675+0000 7f4188c00640 1 -- 172.21.15.151:0/2238351186 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41841a0560 con 0x7f4184102850 2024-05-21T21:55:51.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.675+0000 7f4180c00640 1 -- 172.21.15.151:0/2238351186 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4174045b10 con 0x7f4184102850 2024-05-21T21:55:51.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.675+0000 7f4188c00640 1 -- 172.21.15.151:0/2238351186 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41841a7e50 con 0x7f4184102850 2024-05-21T21:55:51.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.676+0000 7f4188c00640 1 -- 172.21.15.151:0/2238351186 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4184103f90 con 0x7f4184102850 2024-05-21T21:55:51.679 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.678+0000 7f4180c00640 1 -- 172.21.15.151:0/2238351186 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 149829+0+0 (secure 0 0 0) 0x7f4174045cb0 con 0x7f4184102850 2024-05-21T21:55:51.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.680+0000 7f4180c00640 1 --2- 172.21.15.151:0/2238351186 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f41680b29e0 0x7f41680b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:51.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.680+0000 7f4180c00640 1 -- 172.21.15.151:0/2238351186 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f41741038a0 con 0x7f4184102850 2024-05-21T21:55:51.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.680+0000 7f417aa00640 1 --2- 172.21.15.151:0/2238351186 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f41680b29e0 0x7f41680b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:51.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.680+0000 7f417aa00640 1 --2- 172.21.15.151:0/2238351186 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f41680b29e0 0x7f41680b4ea0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f418410faa0 tx=0x7f416c005f90 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:55:51.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.682+0000 7f4180c00640 1 -- 172.21.15.151:0/2238351186 <== mon.2 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f41740098b0 con 0x7f4184102850 2024-05-21T21:55:51.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:51 smithi184 ceph-mon[36942]: pgmap v10: 97 pgs: 97 active+clean; 1.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 170 B/s wr, 14 op/s 2024-05-21T21:55:51.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:51 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/2942350460' entity='client.admin' 2024-05-21T21:55:51.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:51 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:55:51.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:51 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:55:51.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:51 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:55:51.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:51 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T21:55:51.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:51 smithi164 ceph-mon[36269]: pgmap v10: 97 pgs: 97 active+clean; 1.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 170 B/s wr, 14 op/s 2024-05-21T21:55:51.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:51 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/2942350460' entity='client.admin' 2024-05-21T21:55:51.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:51 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:55:51.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:51 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:55:51.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:51 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:55:51.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:51 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T21:55:51.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.894+0000 7f4188c00640 1 -- 172.21.15.151:0/2238351186 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f41841a80f0 con 0x7f4184102850 2024-05-21T21:55:51.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.903+0000 7f4180c00640 1 -- 172.21.15.151:0/2238351186 <== mon.2 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v29)=0 v29) ==== 133+0+0 (secure 0 0 0) 0x7f41740cb230 con 0x7f4184102850 2024-05-21T21:55:51.907 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.906+0000 7f4188c00640 1 -- 172.21.15.151:0/2238351186 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f41680b29e0 msgr2=0x7f41680b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:51.907 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.906+0000 7f4188c00640 1 --2- 172.21.15.151:0/2238351186 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f41680b29e0 0x7f41680b4ea0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f418410faa0 tx=0x7f416c005f90 comp rx=0 tx=0).stop 2024-05-21T21:55:51.907 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.906+0000 7f4188c00640 1 -- 172.21.15.151:0/2238351186 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4184102850 msgr2=0x7f418419b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:51.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.906+0000 7f4188c00640 1 --2- 172.21.15.151:0/2238351186 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4184102850 0x7f418419b2e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4174006010 tx=0x7f4174030cf0 comp rx=0 tx=0).stop 2024-05-21T21:55:51.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.906+0000 7f4188c00640 1 -- 172.21.15.151:0/2238351186 shutdown_connections 2024-05-21T21:55:51.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.906+0000 7f4188c00640 1 --2- 172.21.15.151:0/2238351186 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f41680b29e0 0x7f41680b4ea0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:51.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.906+0000 7f4188c00640 1 --2- 172.21.15.151:0/2238351186 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4184108860 0x7f418419fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:51.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.906+0000 7f4188c00640 1 --2- 172.21.15.151:0/2238351186 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4184103210 0x7f418419b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:51.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.906+0000 7f4188c00640 1 --2- 172.21.15.151:0/2238351186 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4184102850 0x7f418419b2e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:51.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.906+0000 7f4188c00640 1 -- 172.21.15.151:0/2238351186 >> 172.21.15.151:0/2238351186 conn(0x7f41840fe3a0 msgr2=0x7f418410e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:55:51.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.907+0000 7f4188c00640 1 -- 172.21.15.151:0/2238351186 shutdown_connections 2024-05-21T21:55:51.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:51.907+0000 7f4188c00640 1 -- 172.21.15.151:0/2238351186 wait complete. 2024-05-21T21:55:51.929 DEBUG:teuthology.orchestra.run.smithi151:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-05-21T21:55:52.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.111+0000 7f71d9200640 1 -- 172.21.15.151:0/1880194227 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f71d41092b0 msgr2=0x7f71d4109690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:52.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.111+0000 7f71d9200640 1 --2- 172.21.15.151:0/1880194227 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f71d41092b0 0x7f71d4109690 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f71c4009900 tx=0x7f71c402a3f0 comp rx=0 tx=0).stop 2024-05-21T21:55:52.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.111+0000 7f71d9200640 1 -- 172.21.15.151:0/1880194227 shutdown_connections 2024-05-21T21:55:52.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.111+0000 7f71d9200640 1 --2- 172.21.15.151:0/1880194227 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f71d410e380 0x7f71d4115f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:52.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.111+0000 7f71d9200640 1 --2- 172.21.15.151:0/1880194227 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71d4109c60 0x7f71d410dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:52.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.111+0000 7f71d9200640 1 --2- 172.21.15.151:0/1880194227 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f71d41092b0 0x7f71d4109690 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:52.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.111+0000 7f71d9200640 1 -- 172.21.15.151:0/1880194227 >> 172.21.15.151:0/1880194227 conn(0x7f71d4076fe0 msgr2=0x7f71d4079400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:55:52.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.111+0000 7f71d9200640 1 -- 172.21.15.151:0/1880194227 shutdown_connections 2024-05-21T21:55:52.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.111+0000 7f71d9200640 1 -- 172.21.15.151:0/1880194227 wait complete. 2024-05-21T21:55:52.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.112+0000 7f71d9200640 1 Processor -- start 2024-05-21T21:55:52.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.112+0000 7f71d9200640 1 -- start start 2024-05-21T21:55:52.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.112+0000 7f71d9200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f71d41092b0 0x7f71d41a55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:52.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.112+0000 7f71d9200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f71d410e380 0x7f71d41a5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:52.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.112+0000 7f71d9200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71d419f6c0 0x7f71d419fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:52.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.112+0000 7f71d9200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f71d41192c0 con 0x7f71d410e380 2024-05-21T21:55:52.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.112+0000 7f71d9200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f71d4119140 con 0x7f71d41092b0 2024-05-21T21:55:52.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.112+0000 7f71d9200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f71d4119440 con 0x7f71d419f6c0 2024-05-21T21:55:52.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.112+0000 7f71d1400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f71d410e380 0x7f71d41a5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:52.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.112+0000 7f71d1e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f71d41092b0 0x7f71d41a55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:52.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.112+0000 7f71d1400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f71d410e380 0x7f71d41a5af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:42048/0 (socket says 172.21.15.151:42048) 2024-05-21T21:55:52.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.112+0000 7f71d1400640 1 -- 172.21.15.151:0/2941876298 learned_addr learned my addr 172.21.15.151:0/2941876298 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:55:52.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.113+0000 7f71d2800640 1 --2- 172.21.15.151:0/2941876298 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71d419f6c0 0x7f71d419fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:52.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.113+0000 7f71d1e00640 1 -- 172.21.15.151:0/2941876298 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71d419f6c0 msgr2=0x7f71d419fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:52.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.113+0000 7f71d1e00640 1 --2- 172.21.15.151:0/2941876298 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71d419f6c0 0x7f71d419fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:52.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.113+0000 7f71d1e00640 1 -- 172.21.15.151:0/2941876298 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f71d410e380 msgr2=0x7f71d41a5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:52.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.113+0000 7f71d1e00640 1 --2- 172.21.15.151:0/2941876298 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f71d410e380 0x7f71d41a5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:52.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.113+0000 7f71d1e00640 1 -- 172.21.15.151:0/2941876298 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71d41a02c0 con 0x7f71d41092b0 2024-05-21T21:55:52.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.113+0000 7f71d1e00640 1 --2- 172.21.15.151:0/2941876298 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f71d41092b0 0x7f71d41a55b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f71c402a910 tx=0x7f71c4031a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:55:52.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.113+0000 7f71caa00640 1 -- 172.21.15.151:0/2941876298 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c403f030 con 0x7f71d41092b0 2024-05-21T21:55:52.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.113+0000 7f71caa00640 1 -- 172.21.15.151:0/2941876298 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71c4004490 con 0x7f71d41092b0 2024-05-21T21:55:52.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.113+0000 7f71d9200640 1 -- 172.21.15.151:0/2941876298 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71d41a0550 con 0x7f71d41092b0 2024-05-21T21:55:52.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.113+0000 7f71caa00640 1 -- 172.21.15.151:0/2941876298 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c4035070 con 0x7f71d41092b0 2024-05-21T21:55:52.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.114+0000 7f71d9200640 1 -- 172.21.15.151:0/2941876298 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71d41ac640 con 0x7f71d41092b0 2024-05-21T21:55:52.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.115+0000 7f71d9200640 1 -- 172.21.15.151:0/2941876298 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71d410a780 con 0x7f71d41092b0 2024-05-21T21:55:52.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.117+0000 7f71caa00640 1 -- 172.21.15.151:0/2941876298 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 25) ==== 149829+0+0 (secure 0 0 0) 0x7f71c4042030 con 0x7f71d41092b0 2024-05-21T21:55:52.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.118+0000 7f71caa00640 1 --2- 172.21.15.151:0/2941876298 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f71b80b2a90 0x7f71b80b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:52.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.118+0000 7f71caa00640 1 -- 172.21.15.151:0/2941876298 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f71c4102cf0 con 0x7f71d41092b0 2024-05-21T21:55:52.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.118+0000 7f71d1400640 1 --2- 172.21.15.151:0/2941876298 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f71b80b2a90 0x7f71b80b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:52.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.119+0000 7f71d1400640 1 --2- 172.21.15.151:0/2941876298 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f71b80b2a90 0x7f71b80b4f50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f71d41a0ec0 tx=0x7f71c0005f90 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:55:52.121 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.120+0000 7f71caa00640 1 -- 172.21.15.151:0/2941876298 <== mon.2 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f71c40ca510 con 0x7f71d41092b0 2024-05-21T21:55:52.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.367+0000 7f71d9200640 1 -- 172.21.15.151:0/2941876298 --> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f71d4111620 con 0x7f71b80b2a90 2024-05-21T21:55:52.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.479+0000 7f71caa00640 1 -- 172.21.15.151:0/2941876298 <== mgr.25138 v2:172.21.15.151:6800/958417056 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f71c00099c0 con 0x7f71b80b2a90 2024-05-21T21:55:52.481 INFO:teuthology.orchestra.run.smithi151.stdout:Schedule set for path / 2024-05-21T21:55:52.485 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.484+0000 7f71d9200640 1 -- 172.21.15.151:0/2941876298 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f71b80b2a90 msgr2=0x7f71b80b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:52.485 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.484+0000 7f71d9200640 1 --2- 172.21.15.151:0/2941876298 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f71b80b2a90 0x7f71b80b4f50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f71d41a0ec0 tx=0x7f71c0005f90 comp rx=0 tx=0).stop 2024-05-21T21:55:52.485 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.484+0000 7f71d9200640 1 -- 172.21.15.151:0/2941876298 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f71d41092b0 msgr2=0x7f71d41a55b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:52.485 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.484+0000 7f71d9200640 1 --2- 172.21.15.151:0/2941876298 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f71d41092b0 0x7f71d41a55b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f71c402a910 tx=0x7f71c4031a20 comp rx=0 tx=0).stop 2024-05-21T21:55:52.485 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.484+0000 7f71d9200640 1 -- 172.21.15.151:0/2941876298 shutdown_connections 2024-05-21T21:55:52.486 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.484+0000 7f71d9200640 1 --2- 172.21.15.151:0/2941876298 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f71b80b2a90 0x7f71b80b4f50 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:52.486 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.484+0000 7f71d9200640 1 --2- 172.21.15.151:0/2941876298 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71d419f6c0 0x7f71d419fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:52.486 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.484+0000 7f71d9200640 1 --2- 172.21.15.151:0/2941876298 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f71d410e380 0x7f71d41a5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:52.486 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.484+0000 7f71d9200640 1 --2- 172.21.15.151:0/2941876298 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f71d41092b0 0x7f71d41a55b0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:52.486 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.484+0000 7f71d9200640 1 -- 172.21.15.151:0/2941876298 >> 172.21.15.151:0/2941876298 conn(0x7f71d4076fe0 msgr2=0x7f71d4079370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:55:52.486 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.484+0000 7f71d9200640 1 -- 172.21.15.151:0/2941876298 shutdown_connections 2024-05-21T21:55:52.486 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.484+0000 7f71d9200640 1 -- 172.21.15.151:0/2941876298 wait complete. 2024-05-21T21:55:52.508 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:52 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/3289174534' entity='client.admin' 2024-05-21T21:55:52.508 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:52 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:55:52.508 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:52 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:55:52.508 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:52 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:55:52.508 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:52 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T21:55:52.508 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:52 smithi151 ceph-mon[31623]: from='client.? ' entity='client.admin' 2024-05-21T21:55:52.508 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:52 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:55:52.508 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:52 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:55:52.509 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:52 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:55:52.509 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:52 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T21:55:52.511 DEBUG:teuthology.orchestra.run.smithi151:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-05-21T21:55:52.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.693+0000 7f774a400640 1 -- 172.21.15.151:0/913059999 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7744109910 msgr2=0x7f7744115d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:52.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.693+0000 7f774a400640 1 --2- 172.21.15.151:0/913059999 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7744109910 0x7f7744115d50 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f773c009c10 tx=0x7f773c02a6f0 comp rx=0 tx=0).stop 2024-05-21T21:55:52.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.694+0000 7f774a400640 1 -- 172.21.15.151:0/913059999 shutdown_connections 2024-05-21T21:55:52.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.694+0000 7f774a400640 1 --2- 172.21.15.151:0/913059999 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7744109910 0x7f7744115d50 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:52.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.694+0000 7f774a400640 1 --2- 172.21.15.151:0/913059999 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7744108f90 0x7f77441093d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:52.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.694+0000 7f774a400640 1 --2- 172.21.15.151:0/913059999 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f774410ef20 0x7f774410f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:52.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.694+0000 7f774a400640 1 -- 172.21.15.151:0/913059999 >> 172.21.15.151:0/913059999 conn(0x7f7744078830 msgr2=0x7f7744107870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:55:52.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.694+0000 7f774a400640 1 -- 172.21.15.151:0/913059999 shutdown_connections 2024-05-21T21:55:52.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.694+0000 7f774a400640 1 -- 172.21.15.151:0/913059999 wait complete. 2024-05-21T21:55:52.698 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.695+0000 7f774a400640 1 Processor -- start 2024-05-21T21:55:52.698 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.695+0000 7f774a400640 1 -- start start 2024-05-21T21:55:52.698 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.695+0000 7f774a400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7744108f90 0x7f774419f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:52.698 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.695+0000 7f774a400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7744109910 0x7f774419fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:52.698 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.695+0000 7f774a400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f774410ef20 0x7f77441a3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:52.698 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.695+0000 7f774a400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f77441190e0 con 0x7f774410ef20 2024-05-21T21:55:52.698 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.695+0000 7f774a400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7744118f60 con 0x7f7744108f90 2024-05-21T21:55:52.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.695+0000 7f774a400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7744119260 con 0x7f7744109910 2024-05-21T21:55:52.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.696+0000 7f7749a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f774410ef20 0x7f77441a3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:52.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.696+0000 7f773be00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7744109910 0x7f774419fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:52.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.696+0000 7f7749a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f774410ef20 0x7f77441a3ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:42062/0 (socket says 172.21.15.151:42062) 2024-05-21T21:55:52.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.696+0000 7f7749a00640 1 -- 172.21.15.151:0/1991455687 learned_addr learned my addr 172.21.15.151:0/1991455687 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:55:52.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.696+0000 7f7749000640 1 --2- 172.21.15.151:0/1991455687 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7744108f90 0x7f774419f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:52.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.696+0000 7f7749a00640 1 -- 172.21.15.151:0/1991455687 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7744109910 msgr2=0x7f774419fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:52.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.696+0000 7f7749a00640 1 --2- 172.21.15.151:0/1991455687 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7744109910 0x7f774419fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:52.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.696+0000 7f7749a00640 1 -- 172.21.15.151:0/1991455687 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7744108f90 msgr2=0x7f774419f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:52.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.696+0000 7f7749a00640 1 --2- 172.21.15.151:0/1991455687 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7744108f90 0x7f774419f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:52.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.696+0000 7f7749a00640 1 -- 172.21.15.151:0/1991455687 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77441a4610 con 0x7f774410ef20 2024-05-21T21:55:52.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.696+0000 7f773be00640 1 --2- 172.21.15.151:0/1991455687 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7744109910 0x7f774419fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:55:52.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.696+0000 7f7749a00640 1 --2- 172.21.15.151:0/1991455687 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f774410ef20 0x7f77441a3ee0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f773c02ac10 tx=0x7f773c003b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:55:52.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.697+0000 7f7742000640 1 -- 172.21.15.151:0/1991455687 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f773c041020 con 0x7f774410ef20 2024-05-21T21:55:52.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.697+0000 7f7742000640 1 -- 172.21.15.151:0/1991455687 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f773c0318f0 con 0x7f774410ef20 2024-05-21T21:55:52.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.697+0000 7f774a400640 1 -- 172.21.15.151:0/1991455687 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77441a48a0 con 0x7f774410ef20 2024-05-21T21:55:52.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.697+0000 7f7742000640 1 -- 172.21.15.151:0/1991455687 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f773c036040 con 0x7f774410ef20 2024-05-21T21:55:52.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.697+0000 7f774a400640 1 -- 172.21.15.151:0/1991455687 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77441ac190 con 0x7f774410ef20 2024-05-21T21:55:52.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.698+0000 7f774a400640 1 -- 172.21.15.151:0/1991455687 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7744051030 con 0x7f774410ef20 2024-05-21T21:55:52.701 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.700+0000 7f7742000640 1 -- 172.21.15.151:0/1991455687 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 25) ==== 149829+0+0 (secure 0 0 0) 0x7f773c044020 con 0x7f774410ef20 2024-05-21T21:55:52.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.703+0000 7f7742000640 1 --2- 172.21.15.151:0/1991455687 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f77240b29e0 0x7f77240b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:52.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.703+0000 7f7742000640 1 -- 172.21.15.151:0/1991455687 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f773c103050 con 0x7f774410ef20 2024-05-21T21:55:52.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.703+0000 7f7749000640 1 --2- 172.21.15.151:0/1991455687 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f77240b29e0 0x7f77240b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:52.705 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.704+0000 7f7749000640 1 --2- 172.21.15.151:0/1991455687 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f77240b29e0 0x7f77240b4ea0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f772c009b50 tx=0x7f772c005f90 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:55:52.708 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.708+0000 7f7742000640 1 -- 172.21.15.151:0/1991455687 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f773c04b030 con 0x7f774410ef20 2024-05-21T21:55:52.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:52 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/3289174534' entity='client.admin' 2024-05-21T21:55:52.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:52 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:55:52.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:52 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:55:52.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:52 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:55:52.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:52 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T21:55:52.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:52 smithi184 ceph-mon[36942]: from='client.? ' entity='client.admin' 2024-05-21T21:55:52.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:52 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:55:52.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:52 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:55:52.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:52 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:55:52.737 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:52 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T21:55:52.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:52 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/3289174534' entity='client.admin' 2024-05-21T21:55:52.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:52 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:55:52.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:52 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:55:52.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:52 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:55:52.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:52 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T21:55:52.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:52 smithi164 ceph-mon[36269]: from='client.? ' entity='client.admin' 2024-05-21T21:55:52.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:52 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:55:52.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:52 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:55:52.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:52 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:55:52.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:52 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T21:55:52.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:52.949+0000 7f774a400640 1 -- 172.21.15.151:0/1991455687 --> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f7744062e90 con 0x7f77240b29e0 2024-05-21T21:55:53.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.065+0000 7f7742000640 1 -- 172.21.15.151:0/1991455687 <== mgr.25138 v2:172.21.15.151:6800/958417056 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f7744062e90 con 0x7f77240b29e0 2024-05-21T21:55:53.067 INFO:teuthology.orchestra.run.smithi151.stdout:Retention added to path / 2024-05-21T21:55:53.069 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.068+0000 7f774a400640 1 -- 172.21.15.151:0/1991455687 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f77240b29e0 msgr2=0x7f77240b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:53.069 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.068+0000 7f774a400640 1 --2- 172.21.15.151:0/1991455687 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f77240b29e0 0x7f77240b4ea0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f772c009b50 tx=0x7f772c005f90 comp rx=0 tx=0).stop 2024-05-21T21:55:53.069 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.068+0000 7f774a400640 1 -- 172.21.15.151:0/1991455687 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f774410ef20 msgr2=0x7f77441a3ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:53.069 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.068+0000 7f774a400640 1 --2- 172.21.15.151:0/1991455687 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f774410ef20 0x7f77441a3ee0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f773c02ac10 tx=0x7f773c003b80 comp rx=0 tx=0).stop 2024-05-21T21:55:53.069 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.068+0000 7f774a400640 1 -- 172.21.15.151:0/1991455687 shutdown_connections 2024-05-21T21:55:53.070 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.068+0000 7f774a400640 1 --2- 172.21.15.151:0/1991455687 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f77240b29e0 0x7f77240b4ea0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:53.070 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.068+0000 7f774a400640 1 --2- 172.21.15.151:0/1991455687 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f774410ef20 0x7f77441a3ee0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:53.070 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.068+0000 7f774a400640 1 --2- 172.21.15.151:0/1991455687 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7744109910 0x7f774419fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:53.070 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.068+0000 7f774a400640 1 --2- 172.21.15.151:0/1991455687 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7744108f90 0x7f774419f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:53.070 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.068+0000 7f774a400640 1 -- 172.21.15.151:0/1991455687 >> 172.21.15.151:0/1991455687 conn(0x7f7744078830 msgr2=0x7f77440799e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:55:53.070 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.068+0000 7f774a400640 1 -- 172.21.15.151:0/1991455687 shutdown_connections 2024-05-21T21:55:53.070 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.068+0000 7f774a400640 1 -- 172.21.15.151:0/1991455687 wait complete. 2024-05-21T21:55:53.088 DEBUG:teuthology.orchestra.run.smithi151:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-05-21T21:55:53.137 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.136+0000 7ffbbfe00700 1 -- 172.21.15.151:0/2549242877 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffbb81041a0 msgr2=0x7ffbb81081c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:53.138 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.136+0000 7ffbbfe00700 1 --2- 172.21.15.151:0/2549242877 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffbb81041a0 0x7ffbb81081c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ffba4009990 tx=0x7ffba4009ca0 comp rx=0 tx=0).stop 2024-05-21T21:55:53.138 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.136+0000 7ffbbfe00700 1 -- 172.21.15.151:0/2549242877 shutdown_connections 2024-05-21T21:55:53.138 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.136+0000 7ffbbfe00700 1 --2- 172.21.15.151:0/2549242877 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffbb81088f0 0x7ffbb8110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:53.138 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.136+0000 7ffbbfe00700 1 --2- 172.21.15.151:0/2549242877 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffbb81037f0 0x7ffbb8103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:53.138 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.136+0000 7ffbbfe00700 1 --2- 172.21.15.151:0/2549242877 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffbb81041a0 0x7ffbb81081c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:53.138 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.136+0000 7ffbbfe00700 1 -- 172.21.15.151:0/2549242877 >> 172.21.15.151:0/2549242877 conn(0x7ffbb807a6a0 msgr2=0x7ffbb807aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:55:53.138 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.136+0000 7ffbbfe00700 1 -- 172.21.15.151:0/2549242877 shutdown_connections 2024-05-21T21:55:53.138 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.136+0000 7ffbbfe00700 1 -- 172.21.15.151:0/2549242877 wait complete. 2024-05-21T21:55:53.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.137+0000 7ffbbfe00700 1 Processor -- start 2024-05-21T21:55:53.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.137+0000 7ffbbfe00700 1 -- start start 2024-05-21T21:55:53.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.138+0000 7ffbbfe00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffbb81037f0 0x7ffbb819be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:53.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.138+0000 7ffbbfe00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffbb81041a0 0x7ffbb819c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:53.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.138+0000 7ffbbfe00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffbb81088f0 0x7ffbb81a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:53.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.138+0000 7ffbbfe00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb8113700 con 0x7ffbb81041a0 2024-05-21T21:55:53.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.138+0000 7ffbbfe00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb8113580 con 0x7ffbb81088f0 2024-05-21T21:55:53.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.138+0000 7ffbbfe00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb8113880 con 0x7ffbb81037f0 2024-05-21T21:55:53.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.138+0000 7ffbbd400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffbb81037f0 0x7ffbb819be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:53.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.138+0000 7ffbbd400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffbb81037f0 0x7ffbb819be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:39212/0 (socket says 172.21.15.151:39212) 2024-05-21T21:55:53.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.138+0000 7ffbbca00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffbb81041a0 0x7ffbb819c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:53.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.138+0000 7ffbbd400700 1 -- 172.21.15.151:0/172725855 learned_addr learned my addr 172.21.15.151:0/172725855 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:55:53.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.138+0000 7ffbbde00700 1 --2- 172.21.15.151:0/172725855 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffbb81088f0 0x7ffbb81a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:53.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.139+0000 7ffbbd400700 1 -- 172.21.15.151:0/172725855 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffbb81088f0 msgr2=0x7ffbb81a06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:53.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.139+0000 7ffbbd400700 1 --2- 172.21.15.151:0/172725855 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffbb81088f0 0x7ffbb81a06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:53.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.139+0000 7ffbbd400700 1 -- 172.21.15.151:0/172725855 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffbb81041a0 msgr2=0x7ffbb819c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:53.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.139+0000 7ffbbd400700 1 --2- 172.21.15.151:0/172725855 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffbb81041a0 0x7ffbb819c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:53.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.139+0000 7ffbbd400700 1 -- 172.21.15.151:0/172725855 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbb81a0de0 con 0x7ffbb81037f0 2024-05-21T21:55:53.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.139+0000 7ffbbca00700 1 --2- 172.21.15.151:0/172725855 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffbb81041a0 0x7ffbb819c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T21:55:53.142 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.139+0000 7ffbbd400700 1 --2- 172.21.15.151:0/172725855 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffbb81037f0 0x7ffbb819be20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ffbac00e9a0 tx=0x7ffbac00ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:55:53.142 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.139+0000 7ffbb6000700 1 -- 172.21.15.151:0/172725855 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbac00cbd0 con 0x7ffbb81037f0 2024-05-21T21:55:53.142 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.139+0000 7ffbb6000700 1 -- 172.21.15.151:0/172725855 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffbac00cd70 con 0x7ffbb81037f0 2024-05-21T21:55:53.142 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.139+0000 7ffbb6000700 1 -- 172.21.15.151:0/172725855 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbac018680 con 0x7ffbb81037f0 2024-05-21T21:55:53.142 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.139+0000 7ffbbfe00700 1 -- 172.21.15.151:0/172725855 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbb81a10d0 con 0x7ffbb81037f0 2024-05-21T21:55:53.142 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.140+0000 7ffbbfe00700 1 -- 172.21.15.151:0/172725855 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbb81a8950 con 0x7ffbb81037f0 2024-05-21T21:55:53.146 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.145+0000 7ffbb6000700 1 -- 172.21.15.151:0/172725855 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 25) ==== 149829+0+0 (secure 0 0 0) 0x7ffbac010430 con 0x7ffbb81037f0 2024-05-21T21:55:53.146 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.145+0000 7ffbca79db80 1 Processor -- start 2024-05-21T21:55:53.147 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.146+0000 7ffbca79db80 1 -- start start 2024-05-21T21:55:53.147 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.146+0000 7ffbca79db80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555746d33f30 0x555746d34310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:53.147 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.146+0000 7ffbca79db80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555746d34850 0x555746d2e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:53.147 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.146+0000 7ffbca79db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555746d2e900 0x555746d2ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:53.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.146+0000 7ffbca79db80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x555746c8cec0 con 0x555746d33f30 2024-05-21T21:55:53.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.146+0000 7ffbca79db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x555746c7ca60 con 0x555746d2e900 2024-05-21T21:55:53.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.146+0000 7ffbca79db80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x555746c80aa0 con 0x555746d34850 2024-05-21T21:55:53.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.147+0000 7ffbb6000700 1 --2- 172.21.15.151:0/172725855 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7ffba00b2b00 0x7ffba00b4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:53.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.147+0000 7ffbbde00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555746d33f30 0x555746d34310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:53.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.147+0000 7ffbbd400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555746d34850 0x555746d2e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:53.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.147+0000 7ffbbca00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555746d2e900 0x555746d2ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:53.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.147+0000 7ffbbde00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555746d33f30 0x555746d34310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:42078/0 (socket says 172.21.15.151:42078) 2024-05-21T21:55:53.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.147+0000 7ffbbde00700 1 -- 172.21.15.151:0/2077076124 learned_addr learned my addr 172.21.15.151:0/2077076124 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:55:53.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.147+0000 7ffbb6000700 1 -- 172.21.15.151:0/172725855 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7ffbac0e29d0 con 0x7ffbb81037f0 2024-05-21T21:55:53.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.147+0000 7ffbbde00700 1 --2- 172.21.15.151:0/172725855 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7ffba00b2b00 0x7ffba00b4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:53.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.147+0000 7ffbbd400700 1 -- 172.21.15.151:0/2077076124 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555746d2e900 msgr2=0x555746d2ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:55:53.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.147+0000 7ffbbd400700 1 --2- 172.21.15.151:0/2077076124 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555746d2e900 0x555746d2ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:53.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.147+0000 7ffbbd400700 1 -- 172.21.15.151:0/2077076124 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555746d33f30 msgr2=0x555746d34310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:55:53.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.147+0000 7ffbbd400700 1 --2- 172.21.15.151:0/2077076124 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555746d33f30 0x555746d34310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:53.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.147+0000 7ffbbd400700 1 -- 172.21.15.151:0/2077076124 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555746c454a0 con 0x555746d34850 2024-05-21T21:55:53.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.148+0000 7ffbbd400700 1 --2- 172.21.15.151:0/2077076124 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555746d34850 0x555746d2e2e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ffbac0e0010 tx=0x7ffbac01c9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c630bd213ae1ea52 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:55:53.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.148+0000 7ffbaaa00700 1 -- 172.21.15.151:0/2077076124 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbac026040 con 0x555746d34850 2024-05-21T21:55:53.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.148+0000 7ffbaaa00700 1 -- 172.21.15.151:0/2077076124 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffbac0b2290 con 0x555746d34850 2024-05-21T21:55:53.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.148+0000 7ffbaaa00700 1 -- 172.21.15.151:0/2077076124 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbac0b2950 con 0x555746d34850 2024-05-21T21:55:53.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.149+0000 7ffbca79db80 1 -- 172.21.15.151:0/2077076124 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555746d34850 msgr2=0x555746d2e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:53.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.149+0000 7ffbbde00700 1 --2- 172.21.15.151:0/172725855 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7ffba00b2b00 0x7ffba00b4fc0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ffbb0009da0 tx=0x7ffbb0004b40 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:55:53.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.149+0000 7ffbca79db80 1 --2- 172.21.15.151:0/2077076124 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555746d34850 0x555746d2e2e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ffbac0e0010 tx=0x7ffbac01c9c0 comp rx=0 tx=0).stop 2024-05-21T21:55:53.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.149+0000 7ffbca79db80 1 -- 172.21.15.151:0/2077076124 shutdown_connections 2024-05-21T21:55:53.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.149+0000 7ffbca79db80 1 --2- 172.21.15.151:0/2077076124 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555746d2e900 0x555746d2ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:53.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.149+0000 7ffbca79db80 1 --2- 172.21.15.151:0/2077076124 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555746d33f30 0x555746d34310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:53.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.149+0000 7ffbca79db80 1 --2- 172.21.15.151:0/2077076124 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555746d34850 0x555746d2e2e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:53.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.149+0000 7ffbca79db80 1 -- 172.21.15.151:0/2077076124 >> 172.21.15.151:0/2077076124 conn(0x555746c5cc60 msgr2=0x555746d29090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:55:53.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.149+0000 7ffbca79db80 1 -- 172.21.15.151:0/2077076124 shutdown_connections 2024-05-21T21:55:53.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.149+0000 7ffbca79db80 1 -- 172.21.15.151:0/2077076124 wait complete. 2024-05-21T21:55:53.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.150+0000 7ffbca79db80 1 -- start start 2024-05-21T21:55:53.152 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.150+0000 7ffbca79db80 1 Processor -- start 2024-05-21T21:55:53.155 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.151+0000 7ffbca79db80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555746d2e900 0x555746d3a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:53.155 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.151+0000 7ffbca79db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555746d33f30 0x555746d37940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:53.155 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.151+0000 7ffbca79db80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555746d34850 0x555746d37e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:53.155 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.151+0000 7ffbca79db80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x555746b83020 con 0x555746d2e900 2024-05-21T21:55:53.155 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.151+0000 7ffbca79db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x555746c1d7e0 con 0x555746d33f30 2024-05-21T21:55:53.155 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.151+0000 7ffbca79db80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x555746d3b200 con 0x555746d34850 2024-05-21T21:55:53.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.151+0000 7ffbca79db80 10 client.0 _ll_register_callbacks cb 0x555746c26cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:55:53.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.151+0000 7ffbbd400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555746d2e900 0x555746d3a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:53.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.151+0000 7ffbbca00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555746d33f30 0x555746d37940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:53.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.151+0000 7ffbbd400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555746d2e900 0x555746d3a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:42088/0 (socket says 172.21.15.151:42088) 2024-05-21T21:55:53.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.151+0000 7ffbbd400700 1 -- 172.21.15.151:0/3359847605 learned_addr learned my addr 172.21.15.151:0/3359847605 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:55:53.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.151+0000 7ffbbde00700 1 --2- 172.21.15.151:0/3359847605 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555746d34850 0x555746d37e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:53.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.151+0000 7ffbbd400700 1 -- 172.21.15.151:0/3359847605 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555746d34850 msgr2=0x555746d37e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:55:53.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.151+0000 7ffbbd400700 1 --2- 172.21.15.151:0/3359847605 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555746d34850 0x555746d37e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:53.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.151+0000 7ffbbd400700 1 -- 172.21.15.151:0/3359847605 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555746d33f30 msgr2=0x555746d37940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:55:53.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.151+0000 7ffbbd400700 1 --2- 172.21.15.151:0/3359847605 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555746d33f30 0x555746d37940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:53.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.152+0000 7ffbbd400700 1 -- 172.21.15.151:0/3359847605 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555746bcc950 con 0x555746d2e900 2024-05-21T21:55:53.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.152+0000 7ffbbd400700 1 --2- 172.21.15.151:0/3359847605 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555746d2e900 0x555746d3a5a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ffbac00c5b0 tx=0x7ffbac0a9600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef34c15ca43601bd server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:55:53.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.152+0000 7ffb9f400700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-21T21:55:53.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.152+0000 7ffb9f400700 1 -- 172.21.15.151:0/3359847605 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbac0a9af0 con 0x555746d2e900 2024-05-21T21:55:53.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.152+0000 7ffb9f400700 1 -- 172.21.15.151:0/3359847605 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7ffbac0e04d0 con 0x555746d2e900 2024-05-21T21:55:53.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.152+0000 7ffbca79db80 1 -- 172.21.15.151:0/3359847605 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555746c8ee10 con 0x555746d2e900 2024-05-21T21:55:53.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.152+0000 7ffb9f400700 1 -- 172.21.15.151:0/3359847605 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ffbac0aa1f0 con 0x555746d2e900 2024-05-21T21:55:53.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.152+0000 7ffbca79db80 10 client.15168 fetch_fsmap learned FSMap version 29 2024-05-21T21:55:53.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.153+0000 7ffbca79db80 1 -- 172.21.15.151:0/3359847605 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555746c4d0e0 con 0x555746d2e900 2024-05-21T21:55:53.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.153+0000 7ffb9f400700 1 -- 172.21.15.151:0/3359847605 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x555746c8ee10 con 0x555746d2e900 2024-05-21T21:55:53.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.153+0000 7ffb9f400700 1 -- 172.21.15.151:0/3359847605 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7ffbac0a5460 con 0x555746d2e900 2024-05-21T21:55:53.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.153+0000 7ffbca79db80 10 client.15168 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T21:55:53.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.153+0000 7ffbca79db80 10 client.15168 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T21:55:53.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.153+0000 7ffbca79db80 20 client.15168 populate_metadata read hostname 'smithi151' 2024-05-21T21:55:53.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.153+0000 7ffbca79db80 1 --2- 172.21.15.151:0/3359847605 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x555746d47bf0 0x555746d49fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:53.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.153+0000 7ffbca79db80 4 client.15168 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T21:55:53.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.153+0000 7ffbca79db80 1 -- 172.21.15.151:0/3359847605 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555746c885f0 con 0x555746d47bf0 2024-05-21T21:55:53.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.154+0000 7ffbbca00700 1 --2- 172.21.15.151:0/3359847605 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x555746d47bf0 0x555746d49fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:53.159 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.154+0000 7ffbbca00700 1 --2- 172.21.15.151:0/3359847605 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x555746d47bf0 0x555746d49fe0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0796b1e64ecf8a6 server_cookie=9028434746136895 in_seq=0 out_seq=0 2024-05-21T21:55:53.159 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.154+0000 7ffb9f400700 10 client.15168 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T21:55:53.159 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.155+0000 7ffb9f400700 1 -- 172.21.15.151:0/3359847605 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x555746c885f0 con 0x555746d47bf0 2024-05-21T21:55:53.159 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.155+0000 7ffb9f400700 10 client.15168 handle_command_reply: tid=0 2024-05-21T21:55:53.159 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.155+0000 7ffbca79db80 1 client.15168 shutdown 2024-05-21T21:55:53.159 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.155+0000 7ffbca79db80 1 -- 172.21.15.151:0/3359847605 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555746d2e900 msgr2=0x555746d3a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:53.159 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.155+0000 7ffbca79db80 1 --2- 172.21.15.151:0/3359847605 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555746d2e900 0x555746d3a5a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ffbac00c5b0 tx=0x7ffbac0a9600 comp rx=0 tx=0).stop 2024-05-21T21:55:53.159 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.156+0000 7ffbca79db80 1 -- 172.21.15.151:0/3359847605 shutdown_connections 2024-05-21T21:55:53.159 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.156+0000 7ffbca79db80 1 --2- 172.21.15.151:0/3359847605 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555746d34850 0x555746d37e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:53.160 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.156+0000 7ffbca79db80 1 --2- 172.21.15.151:0/3359847605 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555746d2e900 0x555746d3a5a0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:53.160 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.156+0000 7ffbca79db80 1 --2- 172.21.15.151:0/3359847605 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x555746d47bf0 0x555746d49fe0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:53.160 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.156+0000 7ffbca79db80 1 --2- 172.21.15.151:0/3359847605 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555746d33f30 0x555746d37940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:53.160 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.156+0000 7ffb9f400700 0 client.15168 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T21:55:53.160 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.156+0000 7ffbca79db80 1 -- 172.21.15.151:0/3359847605 >> 172.21.15.151:0/3359847605 conn(0x555746c5cc60 msgr2=0x555746d2d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:55:53.160 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.156+0000 7ffbca79db80 1 -- 172.21.15.151:0/3359847605 shutdown_connections 2024-05-21T21:55:53.160 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.156+0000 7ffbca79db80 1 -- 172.21.15.151:0/3359847605 wait complete. 2024-05-21T21:55:53.160 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.156+0000 7ffbca79db80 20 client.15168 trim_cache size 0 max 16384 2024-05-21T21:55:53.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.201+0000 7ffbca79db80 1 Processor -- start 2024-05-21T21:55:53.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.201+0000 7ffbca79db80 1 -- start start 2024-05-21T21:55:53.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.201+0000 7ffbca79db80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555746d2e900 0x555746d2e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:53.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.201+0000 7ffbca79db80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555746d33f30 0x555746d35580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:53.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.201+0000 7ffbca79db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555746d34850 0x555746d290d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:53.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.201+0000 7ffbca79db80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb81b0130 con 0x555746d33f30 2024-05-21T21:55:53.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.201+0000 7ffbca79db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb81b1c70 con 0x555746d34850 2024-05-21T21:55:53.203 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.201+0000 7ffbca79db80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb81b0800 con 0x555746d2e900 2024-05-21T21:55:53.203 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.201+0000 7ffbbde00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555746d34850 0x555746d290d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:53.203 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.201+0000 7ffbbca00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555746d33f30 0x555746d35580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:53.203 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.202+0000 7ffbbde00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555746d34850 0x555746d290d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:43222/0 (socket says 172.21.15.151:43222) 2024-05-21T21:55:53.204 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.202+0000 7ffbbde00700 1 -- 172.21.15.151:0/3797085888 learned_addr learned my addr 172.21.15.151:0/3797085888 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:55:53.204 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.202+0000 7ffbbd400700 1 --2- 172.21.15.151:0/3797085888 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555746d2e900 0x555746d2e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:53.204 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.202+0000 7ffbbde00700 1 -- 172.21.15.151:0/3797085888 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555746d2e900 msgr2=0x555746d2e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:55:53.204 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.202+0000 7ffbbde00700 1 --2- 172.21.15.151:0/3797085888 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555746d2e900 0x555746d2e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:53.204 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.202+0000 7ffbbde00700 1 -- 172.21.15.151:0/3797085888 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555746d33f30 msgr2=0x555746d35580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:55:53.204 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.202+0000 7ffbbde00700 1 --2- 172.21.15.151:0/3797085888 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555746d33f30 0x555746d35580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:53.204 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.202+0000 7ffbbde00700 1 -- 172.21.15.151:0/3797085888 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555746d34650 con 0x555746d34850 2024-05-21T21:55:53.204 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.202+0000 7ffbbde00700 1 --2- 172.21.15.151:0/3797085888 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555746d34850 0x555746d290d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ffbb0009420 tx=0x7ffbb0009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0b808736188a069 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:55:53.205 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.203+0000 7ffbaa000700 1 -- 172.21.15.151:0/3797085888 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb0016880 con 0x555746d34850 2024-05-21T21:55:53.205 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.203+0000 7ffbaa000700 1 -- 172.21.15.151:0/3797085888 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffbb001d030 con 0x555746d34850 2024-05-21T21:55:53.205 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.203+0000 7ffbaa000700 1 -- 172.21.15.151:0/3797085888 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb0004830 con 0x555746d34850 2024-05-21T21:55:53.206 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.203+0000 7ffbca79db80 1 -- 172.21.15.151:0/3797085888 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555746d34850 msgr2=0x555746d290d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:53.206 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.203+0000 7ffbca79db80 1 --2- 172.21.15.151:0/3797085888 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555746d34850 0x555746d290d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ffbb0009420 tx=0x7ffbb0009920 comp rx=0 tx=0).stop 2024-05-21T21:55:53.206 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.204+0000 7ffbca79db80 1 -- 172.21.15.151:0/3797085888 shutdown_connections 2024-05-21T21:55:53.206 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.204+0000 7ffbca79db80 1 --2- 172.21.15.151:0/3797085888 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555746d34850 0x555746d290d0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:53.206 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.204+0000 7ffbca79db80 1 --2- 172.21.15.151:0/3797085888 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555746d2e900 0x555746d2e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:53.206 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.204+0000 7ffbca79db80 1 --2- 172.21.15.151:0/3797085888 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555746d33f30 0x555746d35580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:53.206 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.204+0000 7ffbca79db80 1 -- 172.21.15.151:0/3797085888 >> 172.21.15.151:0/3797085888 conn(0x555746c5cc60 msgr2=0x555746d36100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:55:53.206 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.204+0000 7ffbca79db80 1 -- 172.21.15.151:0/3797085888 shutdown_connections 2024-05-21T21:55:53.206 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.204+0000 7ffbca79db80 1 -- 172.21.15.151:0/3797085888 wait complete. 2024-05-21T21:55:53.206 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.204+0000 7ffbca79db80 1 -- start start 2024-05-21T21:55:53.206 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.205+0000 7ffbca79db80 1 Processor -- start 2024-05-21T21:55:53.207 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.205+0000 7ffbca79db80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555746d2e900 0x555746d4bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:53.207 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.205+0000 7ffbca79db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555746d33f30 0x555746d4c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:53.208 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.205+0000 7ffbca79db80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555746d34850 0x555746d4c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:53.208 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.205+0000 7ffbca79db80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb81b15a0 con 0x555746d34850 2024-05-21T21:55:53.208 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.205+0000 7ffbca79db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb81b0ed0 con 0x555746d33f30 2024-05-21T21:55:53.208 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.205+0000 7ffbca79db80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb81ae0d0 con 0x555746d2e900 2024-05-21T21:55:53.208 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.205+0000 7ffbca79db80 10 client.0 _ll_register_callbacks cb 0x555746c26cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:55:53.208 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.206+0000 7ffbbd400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555746d2e900 0x555746d4bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:53.208 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.206+0000 7ffbbde00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555746d34850 0x555746d4c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:53.208 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.206+0000 7ffbbd400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555746d2e900 0x555746d4bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:39234/0 (socket says 172.21.15.151:39234) 2024-05-21T21:55:53.208 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.206+0000 7ffbbde00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555746d34850 0x555746d4c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:42094/0 (socket says 172.21.15.151:42094) 2024-05-21T21:55:53.208 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.206+0000 7ffbbd400700 1 -- 172.21.15.151:0/3205753520 learned_addr learned my addr 172.21.15.151:0/3205753520 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:55:53.208 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.206+0000 7ffbbca00700 1 --2- 172.21.15.151:0/3205753520 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555746d33f30 0x555746d4c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:53.208 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.206+0000 7ffbbca00700 1 -- 172.21.15.151:0/3205753520 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555746d2e900 msgr2=0x555746d4bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:55:53.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.206+0000 7ffbbca00700 1 --2- 172.21.15.151:0/3205753520 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555746d2e900 0x555746d4bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:53.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.206+0000 7ffbbca00700 1 -- 172.21.15.151:0/3205753520 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555746d34850 msgr2=0x555746d4c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:55:53.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.206+0000 7ffbbca00700 1 --2- 172.21.15.151:0/3205753520 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555746d34850 0x555746d4c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:53.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.206+0000 7ffbbca00700 1 -- 172.21.15.151:0/3205753520 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555746db25d0 con 0x555746d33f30 2024-05-21T21:55:53.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.206+0000 7ffbbde00700 1 --2- 172.21.15.151:0/3205753520 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555746d34850 0x555746d4c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:55:53.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.206+0000 7ffbbca00700 1 --2- 172.21.15.151:0/3205753520 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555746d33f30 0x555746d4c490 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ffbb8103320 tx=0x7ffba40094b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7c2b63be18cbebd server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:55:53.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.206+0000 7ffb9f400700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-21T21:55:53.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.207+0000 7ffb9f400700 1 -- 172.21.15.151:0/3205753520 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffba400b9f0 con 0x555746d33f30 2024-05-21T21:55:53.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.207+0000 7ffb9f400700 1 -- 172.21.15.151:0/3205753520 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7ffba400bb90 con 0x555746d33f30 2024-05-21T21:55:53.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.207+0000 7ffbca79db80 1 -- 172.21.15.151:0/3205753520 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffbb001d030 con 0x555746d33f30 2024-05-21T21:55:53.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.207+0000 7ffb9f400700 1 -- 172.21.15.151:0/3205753520 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ffba400bd90 con 0x555746d33f30 2024-05-21T21:55:53.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.207+0000 7ffbca79db80 10 client.25298 fetch_fsmap learned FSMap version 29 2024-05-21T21:55:53.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.207+0000 7ffbca79db80 1 -- 172.21.15.151:0/3205753520 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555746db3970 con 0x555746d33f30 2024-05-21T21:55:53.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.208+0000 7ffb9f400700 1 -- 172.21.15.151:0/3205753520 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffbb001d030 con 0x555746d33f30 2024-05-21T21:55:53.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.208+0000 7ffb9f400700 1 -- 172.21.15.151:0/3205753520 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7ffba4038500 con 0x555746d33f30 2024-05-21T21:55:53.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.208+0000 7ffbca79db80 10 client.25298 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T21:55:53.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.208+0000 7ffbca79db80 10 client.25298 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T21:55:53.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.208+0000 7ffbca79db80 20 client.25298 populate_metadata read hostname 'smithi151' 2024-05-21T21:55:53.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.208+0000 7ffbca79db80 1 --2- 172.21.15.151:0/3205753520 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x555746db4010 0x555746db6430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:53.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.208+0000 7ffbca79db80 4 client.25298 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-21T21:55:53.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.208+0000 7ffbca79db80 1 -- 172.21.15.151:0/3205753520 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x555746aff2d0 con 0x555746db4010 2024-05-21T21:55:53.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.208+0000 7ffbbd400700 1 --2- 172.21.15.151:0/3205753520 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x555746db4010 0x555746db6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:53.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.209+0000 7ffb9f400700 10 client.25298 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T21:55:53.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.209+0000 7ffbbd400700 1 --2- 172.21.15.151:0/3205753520 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x555746db4010 0x555746db6430 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e82734822b3773d8 server_cookie=66e17d43a04971c9 in_seq=0 out_seq=0 2024-05-21T21:55:53.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.210+0000 7ffb9f400700 1 -- 172.21.15.151:0/3205753520 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x555746aff2d0 con 0x555746db4010 2024-05-21T21:55:53.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.210+0000 7ffb9f400700 10 client.25298 handle_command_reply: tid=0 2024-05-21T21:55:53.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.210+0000 7ffbca79db80 1 client.25298 shutdown 2024-05-21T21:55:53.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.210+0000 7ffbca79db80 1 -- 172.21.15.151:0/3205753520 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555746d33f30 msgr2=0x555746d4c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:53.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.210+0000 7ffbca79db80 1 --2- 172.21.15.151:0/3205753520 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555746d33f30 0x555746d4c490 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ffbb8103320 tx=0x7ffba40094b0 comp rx=0 tx=0).stop 2024-05-21T21:55:53.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.211+0000 7ffbca79db80 1 -- 172.21.15.151:0/3205753520 shutdown_connections 2024-05-21T21:55:53.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.211+0000 7ffbca79db80 1 --2- 172.21.15.151:0/3205753520 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555746d34850 0x555746d4c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:53.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.211+0000 7ffbca79db80 1 --2- 172.21.15.151:0/3205753520 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555746d2e900 0x555746d4bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:53.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.211+0000 7ffbca79db80 1 --2- 172.21.15.151:0/3205753520 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x555746db4010 0x555746db6430 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:53.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.211+0000 7ffbca79db80 1 --2- 172.21.15.151:0/3205753520 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555746d33f30 0x555746d4c490 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:53.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.211+0000 7ffb9f400700 0 client.25298 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T21:55:53.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.211+0000 7ffbca79db80 1 -- 172.21.15.151:0/3205753520 >> 172.21.15.151:0/3205753520 conn(0x555746c5cc60 msgr2=0x555746d36990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:55:53.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.211+0000 7ffbca79db80 1 -- 172.21.15.151:0/3205753520 shutdown_connections 2024-05-21T21:55:53.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.211+0000 7ffbca79db80 1 -- 172.21.15.151:0/3205753520 wait complete. 2024-05-21T21:55:53.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.211+0000 7ffbca79db80 20 client.25298 trim_cache size 0 max 16384 2024-05-21T21:55:53.214 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-05-21T21:55:53.214 INFO:teuthology.orchestra.run.smithi151.stdout: "return_code": 0, 2024-05-21T21:55:53.214 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_tag": "02ccb105-4b0a-4d9a-a4bb-6b2e269f31aa", 2024-05-21T21:55:53.214 INFO:teuthology.orchestra.run.smithi151.stdout: "mode": "asynchronous" 2024-05-21T21:55:53.214 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-05-21T21:55:53.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.212+0000 7ffbbfe00700 1 -- 172.21.15.151:0/172725855 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7ffba00b2b00 msgr2=0x7ffba00b4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:53.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.212+0000 7ffbbfe00700 1 --2- 172.21.15.151:0/172725855 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7ffba00b2b00 0x7ffba00b4fc0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ffbb0009da0 tx=0x7ffbb0004b40 comp rx=0 tx=0).stop 2024-05-21T21:55:53.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.212+0000 7ffbbfe00700 1 -- 172.21.15.151:0/172725855 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffbb81037f0 msgr2=0x7ffbb819be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:53.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.212+0000 7ffbbfe00700 1 --2- 172.21.15.151:0/172725855 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffbb81037f0 0x7ffbb819be20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ffbac00e9a0 tx=0x7ffbac00ed60 comp rx=0 tx=0).stop 2024-05-21T21:55:53.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.212+0000 7ffbbfe00700 1 -- 172.21.15.151:0/172725855 shutdown_connections 2024-05-21T21:55:53.216 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.212+0000 7ffbbfe00700 1 --2- 172.21.15.151:0/172725855 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7ffba00b2b00 0x7ffba00b4fc0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:53.216 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.212+0000 7ffbbfe00700 1 --2- 172.21.15.151:0/172725855 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffbb81088f0 0x7ffbb81a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:53.216 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.212+0000 7ffbbfe00700 1 --2- 172.21.15.151:0/172725855 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffbb81037f0 0x7ffbb819be20 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:53.216 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.212+0000 7ffbbfe00700 1 --2- 172.21.15.151:0/172725855 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffbb81041a0 0x7ffbb819c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:53.216 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.212+0000 7ffbbfe00700 1 -- 172.21.15.151:0/172725855 >> 172.21.15.151:0/172725855 conn(0x7ffbb807a6a0 msgr2=0x7ffbb81019e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:55:53.216 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.212+0000 7ffbbfe00700 1 -- 172.21.15.151:0/172725855 shutdown_connections 2024-05-21T21:55:53.216 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.212+0000 7ffbbfe00700 1 -- 172.21.15.151:0/172725855 wait complete. 2024-05-21T21:55:53.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.299+0000 7f95ae400640 1 -- 172.21.15.151:0/3418235369 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f95a8108860 msgr2=0x7f95a8108c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:53.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.299+0000 7f95ae400640 1 --2- 172.21.15.151:0/3418235369 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f95a8108860 0x7f95a8108c40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f9598009930 tx=0x7f959802a420 comp rx=0 tx=0).stop 2024-05-21T21:55:53.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.299+0000 7f95ae400640 1 -- 172.21.15.151:0/3418235369 shutdown_connections 2024-05-21T21:55:53.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.299+0000 7f95ae400640 1 --2- 172.21.15.151:0/3418235369 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95a8103210 0x7f95a810f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:53.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.299+0000 7f95ae400640 1 --2- 172.21.15.151:0/3418235369 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f95a8102850 0x7f95a8102cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:53.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.299+0000 7f95ae400640 1 --2- 172.21.15.151:0/3418235369 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f95a8108860 0x7f95a8108c40 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:53.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.299+0000 7f95ae400640 1 -- 172.21.15.151:0/3418235369 >> 172.21.15.151:0/3418235369 conn(0x7f95a80fe3a0 msgr2=0x7f95a81007c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:55:53.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.299+0000 7f95ae400640 1 -- 172.21.15.151:0/3418235369 shutdown_connections 2024-05-21T21:55:53.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.299+0000 7f95ae400640 1 -- 172.21.15.151:0/3418235369 wait complete. 2024-05-21T21:55:53.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.300+0000 7f95ae400640 1 Processor -- start 2024-05-21T21:55:53.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.300+0000 7f95ae400640 1 -- start start 2024-05-21T21:55:53.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.300+0000 7f95ae400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f95a8102850 0x7f95a819b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:53.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.300+0000 7f95ae400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f95a8103210 0x7f95a819b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:53.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.301+0000 7f95ae400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95a8108860 0x7f95a819fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:53.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.301+0000 7f95ae400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f95a8112ae0 con 0x7f95a8102850 2024-05-21T21:55:53.303 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.301+0000 7f95ae400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f95a8112960 con 0x7f95a8103210 2024-05-21T21:55:53.303 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.301+0000 7f95ae400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f95a8112c60 con 0x7f95a8108860 2024-05-21T21:55:53.303 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.301+0000 7f95ad000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f95a8102850 0x7f95a819b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:53.303 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.301+0000 7f95ad000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f95a8102850 0x7f95a819b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:42126/0 (socket says 172.21.15.151:42126) 2024-05-21T21:55:53.303 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.301+0000 7f95ad000640 1 -- 172.21.15.151:0/2512756535 learned_addr learned my addr 172.21.15.151:0/2512756535 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:55:53.303 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.301+0000 7f959fe00640 1 --2- 172.21.15.151:0/2512756535 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f95a8103210 0x7f95a819b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:53.303 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.301+0000 7f95ada00640 1 --2- 172.21.15.151:0/2512756535 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95a8108860 0x7f95a819fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:53.303 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.301+0000 7f95ad000640 1 -- 172.21.15.151:0/2512756535 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95a8108860 msgr2=0x7f95a819fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:53.303 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.301+0000 7f95ad000640 1 --2- 172.21.15.151:0/2512756535 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95a8108860 0x7f95a819fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:53.303 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.301+0000 7f95ad000640 1 -- 172.21.15.151:0/2512756535 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f95a8103210 msgr2=0x7f95a819b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:53.303 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.301+0000 7f95ad000640 1 --2- 172.21.15.151:0/2512756535 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f95a8103210 0x7f95a819b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:53.303 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.301+0000 7f95ad000640 1 -- 172.21.15.151:0/2512756535 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95a81a0330 con 0x7f95a8102850 2024-05-21T21:55:53.304 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.302+0000 7f95ad000640 1 --2- 172.21.15.151:0/2512756535 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f95a8102850 0x7f95a819b340 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9598005940 tx=0x7f95980040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:55:53.304 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.302+0000 7f95a6000640 1 -- 172.21.15.151:0/2512756535 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f959803f030 con 0x7f95a8102850 2024-05-21T21:55:53.304 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.302+0000 7f95ae400640 1 -- 172.21.15.151:0/2512756535 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95a81a05c0 con 0x7f95a8102850 2024-05-21T21:55:53.305 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.302+0000 7f95a6000640 1 -- 172.21.15.151:0/2512756535 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95980044c0 con 0x7f95a8102850 2024-05-21T21:55:53.305 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.302+0000 7f95a6000640 1 -- 172.21.15.151:0/2512756535 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9598035070 con 0x7f95a8102850 2024-05-21T21:55:53.305 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.302+0000 7f95ae400640 1 -- 172.21.15.151:0/2512756535 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95a81a7eb0 con 0x7f95a8102850 2024-05-21T21:55:53.305 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.303+0000 7f95ae400640 1 -- 172.21.15.151:0/2512756535 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95a8103f90 con 0x7f95a8102850 2024-05-21T21:55:53.306 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.305+0000 7f95a6000640 1 -- 172.21.15.151:0/2512756535 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 25) ==== 149829+0+0 (secure 0 0 0) 0x7f9598042030 con 0x7f95a8102850 2024-05-21T21:55:53.308 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.307+0000 7f95a6000640 1 --2- 172.21.15.151:0/2512756535 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f958c0b29e0 0x7f958c0b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:53.308 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.307+0000 7f95a6000640 1 -- 172.21.15.151:0/2512756535 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f9598102b10 con 0x7f95a8102850 2024-05-21T21:55:53.308 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.307+0000 7f959fe00640 1 --2- 172.21.15.151:0/2512756535 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f958c0b29e0 0x7f958c0b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:53.309 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.308+0000 7f959fe00640 1 --2- 172.21.15.151:0/2512756535 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f958c0b29e0 0x7f958c0b4ea0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f95a810fa40 tx=0x7f9590005f90 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:55:53.312 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.311+0000 7f95a6000640 1 -- 172.21.15.151:0/2512756535 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f95980ca330 con 0x7f95a8102850 2024-05-21T21:55:53.549 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.548+0000 7f95ae400640 1 -- 172.21.15.151:0/2512756535 --> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f95a80ffa10 con 0x7f958c0b29e0 2024-05-21T21:55:53.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.565+0000 7f95a6000640 1 -- 172.21.15.151:0/2512756535 <== mgr.25138 v2:172.21.15.151:6800/958417056 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f95a80ffa10 con 0x7f958c0b29e0 2024-05-21T21:55:53.566 INFO:teuthology.orchestra.run.smithi151.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-05-21T00:00:00", "created": "2024-05-21T21:55:52", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-05-21T21:55:53.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.568+0000 7f95ae400640 1 -- 172.21.15.151:0/2512756535 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f958c0b29e0 msgr2=0x7f958c0b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:53.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.568+0000 7f95ae400640 1 --2- 172.21.15.151:0/2512756535 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f958c0b29e0 0x7f958c0b4ea0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f95a810fa40 tx=0x7f9590005f90 comp rx=0 tx=0).stop 2024-05-21T21:55:53.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.568+0000 7f95ae400640 1 -- 172.21.15.151:0/2512756535 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f95a8102850 msgr2=0x7f95a819b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:53.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.568+0000 7f95ae400640 1 --2- 172.21.15.151:0/2512756535 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f95a8102850 0x7f95a819b340 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9598005940 tx=0x7f95980040c0 comp rx=0 tx=0).stop 2024-05-21T21:55:53.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.568+0000 7f95ae400640 1 -- 172.21.15.151:0/2512756535 shutdown_connections 2024-05-21T21:55:53.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.568+0000 7f95ae400640 1 --2- 172.21.15.151:0/2512756535 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f958c0b29e0 0x7f958c0b4ea0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:53.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.568+0000 7f95ae400640 1 --2- 172.21.15.151:0/2512756535 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95a8108860 0x7f95a819fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:53.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.568+0000 7f95ae400640 1 --2- 172.21.15.151:0/2512756535 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f95a8103210 0x7f95a819b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:53.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.568+0000 7f95ae400640 1 --2- 172.21.15.151:0/2512756535 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f95a8102850 0x7f95a819b340 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:53.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.568+0000 7f95ae400640 1 -- 172.21.15.151:0/2512756535 >> 172.21.15.151:0/2512756535 conn(0x7f95a80fe3a0 msgr2=0x7f95a810dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:55:53.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.568+0000 7f95ae400640 1 -- 172.21.15.151:0/2512756535 shutdown_connections 2024-05-21T21:55:53.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.568+0000 7f95ae400640 1 -- 172.21.15.151:0/2512756535 wait complete. 2024-05-21T21:55:53.587 DEBUG:teuthology.orchestra.run.smithi151:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-05-21T21:55:53.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:53 smithi184 ceph-mon[36942]: pgmap v11: 97 pgs: 97 active+clean; 1.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 170 B/s wr, 14 op/s 2024-05-21T21:55:53.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:53 smithi184 ceph-mon[36942]: from='client.25274 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:55:53.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.772+0000 7fc2b3e00640 1 -- 172.21.15.151:0/3938664089 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc2b4075710 msgr2=0x7fc2b4113320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:53.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.772+0000 7fc2b3e00640 1 --2- 172.21.15.151:0/3938664089 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc2b4075710 0x7fc2b4113320 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc29c009900 tx=0x7fc29c02a480 comp rx=0 tx=0).stop 2024-05-21T21:55:53.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.772+0000 7fc2b3e00640 1 -- 172.21.15.151:0/3938664089 shutdown_connections 2024-05-21T21:55:53.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.772+0000 7fc2b3e00640 1 --2- 172.21.15.151:0/3938664089 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc2b4113860 0x7fc2b4115c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:53.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.772+0000 7fc2b3e00640 1 --2- 172.21.15.151:0/3938664089 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc2b4075710 0x7fc2b4113320 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:53.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.772+0000 7fc2b3e00640 1 --2- 172.21.15.151:0/3938664089 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2b4074df0 0x7fc2b40751d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:53.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.772+0000 7fc2b3e00640 1 -- 172.21.15.151:0/3938664089 >> 172.21.15.151:0/3938664089 conn(0x7fc2b40fe3a0 msgr2=0x7fc2b41007c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:55:53.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.772+0000 7fc2b3e00640 1 -- 172.21.15.151:0/3938664089 shutdown_connections 2024-05-21T21:55:53.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.772+0000 7fc2b3e00640 1 -- 172.21.15.151:0/3938664089 wait complete. 2024-05-21T21:55:53.776 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.773+0000 7fc2b3e00640 1 Processor -- start 2024-05-21T21:55:53.776 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.773+0000 7fc2b3e00640 1 -- start start 2024-05-21T21:55:53.776 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.774+0000 7fc2b3e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc2b4074df0 0x7fc2b419f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:53.776 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.774+0000 7fc2b3e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc2b4075710 0x7fc2b419fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:53.777 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.774+0000 7fc2b3e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2b4113860 0x7fc2b41a3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:53.777 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.774+0000 7fc2b3e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b4119130 con 0x7fc2b4075710 2024-05-21T21:55:53.777 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.774+0000 7fc2b3e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b4118fb0 con 0x7fc2b4074df0 2024-05-21T21:55:53.777 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.774+0000 7fc2b3e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b41192b0 con 0x7fc2b4113860 2024-05-21T21:55:53.777 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.774+0000 7fc2aa000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc2b4075710 0x7fc2b419fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:53.777 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.774+0000 7fc2b3400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2b4113860 0x7fc2b41a3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:53.777 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.774+0000 7fc2b3400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2b4113860 0x7fc2b41a3fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:39276/0 (socket says 172.21.15.151:39276) 2024-05-21T21:55:53.777 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.774+0000 7fc2aa000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc2b4075710 0x7fc2b419fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:42152/0 (socket says 172.21.15.151:42152) 2024-05-21T21:55:53.777 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.774+0000 7fc2b3400640 1 -- 172.21.15.151:0/1498116325 learned_addr learned my addr 172.21.15.151:0/1498116325 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:55:53.777 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.774+0000 7fc2b2a00640 1 --2- 172.21.15.151:0/1498116325 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc2b4074df0 0x7fc2b419f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:53.778 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.774+0000 7fc2b3400640 1 -- 172.21.15.151:0/1498116325 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc2b4074df0 msgr2=0x7fc2b419f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:53.778 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.774+0000 7fc2b3400640 1 --2- 172.21.15.151:0/1498116325 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc2b4074df0 0x7fc2b419f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:53.778 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.774+0000 7fc2b3400640 1 -- 172.21.15.151:0/1498116325 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc2b4075710 msgr2=0x7fc2b419fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:53.778 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.774+0000 7fc2b3400640 1 --2- 172.21.15.151:0/1498116325 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc2b4075710 0x7fc2b419fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:53.778 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.774+0000 7fc2b3400640 1 -- 172.21.15.151:0/1498116325 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2b41a46e0 con 0x7fc2b4113860 2024-05-21T21:55:53.778 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.775+0000 7fc2b3400640 1 --2- 172.21.15.151:0/1498116325 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2b4113860 0x7fc2b41a3fb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac00eb40 tx=0x7fc2ac00c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:55:53.778 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.775+0000 7fc2abe00640 1 -- 172.21.15.151:0/1498116325 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac004b20 con 0x7fc2b4113860 2024-05-21T21:55:53.778 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.775+0000 7fc2abe00640 1 -- 172.21.15.151:0/1498116325 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2ac004cc0 con 0x7fc2b4113860 2024-05-21T21:55:53.778 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.775+0000 7fc2b3e00640 1 -- 172.21.15.151:0/1498116325 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2b41a49d0 con 0x7fc2b4113860 2024-05-21T21:55:53.778 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.775+0000 7fc2abe00640 1 -- 172.21.15.151:0/1498116325 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac010720 con 0x7fc2b4113860 2024-05-21T21:55:53.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.775+0000 7fc2b3e00640 1 -- 172.21.15.151:0/1498116325 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2b41ac260 con 0x7fc2b4113860 2024-05-21T21:55:53.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.776+0000 7fc2b3e00640 1 -- 172.21.15.151:0/1498116325 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2b4051030 con 0x7fc2b4113860 2024-05-21T21:55:53.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.778+0000 7fc2abe00640 1 -- 172.21.15.151:0/1498116325 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 25) ==== 149829+0+0 (secure 0 0 0) 0x7fc2ac005560 con 0x7fc2b4113860 2024-05-21T21:55:53.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.780+0000 7fc2abe00640 1 --2- 172.21.15.151:0/1498116325 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fc2980b29e0 0x7fc2980b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:53.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.780+0000 7fc2abe00640 1 -- 172.21.15.151:0/1498116325 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fc2ac0e2ee0 con 0x7fc2b4113860 2024-05-21T21:55:53.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.780+0000 7fc2b2a00640 1 --2- 172.21.15.151:0/1498116325 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fc2980b29e0 0x7fc2980b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:53.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.781+0000 7fc2b2a00640 1 --2- 172.21.15.151:0/1498116325 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fc2980b29e0 0x7fc2980b4ea0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc2a4009bd0 tx=0x7fc2a4005f90 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:55:53.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:53.784+0000 7fc2abe00640 1 -- 172.21.15.151:0/1498116325 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc2ac0aa700 con 0x7fc2b4113860 2024-05-21T21:55:53.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:53 smithi164 ceph-mon[36269]: pgmap v11: 97 pgs: 97 active+clean; 1.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 170 B/s wr, 14 op/s 2024-05-21T21:55:53.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:53 smithi164 ceph-mon[36269]: from='client.25274 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:55:53.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:53 smithi151 ceph-mon[31623]: pgmap v11: 97 pgs: 97 active+clean; 1.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 170 B/s wr, 14 op/s 2024-05-21T21:55:53.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:53 smithi151 ceph-mon[31623]: from='client.25274 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:55:54.022 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:54.021+0000 7fc2b3e00640 1 -- 172.21.15.151:0/1498116325 --> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7fc2b4062e90 con 0x7fc2980b29e0 2024-05-21T21:55:54.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:54.030+0000 7fc2abe00640 1 -- 172.21.15.151:0/1498116325 <== mgr.25138 v2:172.21.15.151:6800/958417056 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7fc2b4062e90 con 0x7fc2980b29e0 2024-05-21T21:55:54.032 INFO:teuthology.orchestra.run.smithi151.stdout:/ 1m 6m3h 2024-05-21T21:55:54.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:54.033+0000 7fc2b3e00640 1 -- 172.21.15.151:0/1498116325 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fc2980b29e0 msgr2=0x7fc2980b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:54.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:54.033+0000 7fc2b3e00640 1 --2- 172.21.15.151:0/1498116325 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fc2980b29e0 0x7fc2980b4ea0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc2a4009bd0 tx=0x7fc2a4005f90 comp rx=0 tx=0).stop 2024-05-21T21:55:54.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:54.033+0000 7fc2b3e00640 1 -- 172.21.15.151:0/1498116325 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2b4113860 msgr2=0x7fc2b41a3fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:54.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:54.033+0000 7fc2b3e00640 1 --2- 172.21.15.151:0/1498116325 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2b4113860 0x7fc2b41a3fb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac00eb40 tx=0x7fc2ac00c2d0 comp rx=0 tx=0).stop 2024-05-21T21:55:54.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:54.033+0000 7fc2b3e00640 1 -- 172.21.15.151:0/1498116325 shutdown_connections 2024-05-21T21:55:54.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:54.033+0000 7fc2b3e00640 1 --2- 172.21.15.151:0/1498116325 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fc2980b29e0 0x7fc2980b4ea0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:54.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:54.033+0000 7fc2b3e00640 1 --2- 172.21.15.151:0/1498116325 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2b4113860 0x7fc2b41a3fb0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:54.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:54.033+0000 7fc2b3e00640 1 --2- 172.21.15.151:0/1498116325 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc2b4075710 0x7fc2b419fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:54.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:54.033+0000 7fc2b3e00640 1 --2- 172.21.15.151:0/1498116325 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc2b4074df0 0x7fc2b419f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:54.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:54.033+0000 7fc2b3e00640 1 -- 172.21.15.151:0/1498116325 >> 172.21.15.151:0/1498116325 conn(0x7fc2b40fe3a0 msgr2=0x7fc2b40ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:55:54.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:54.033+0000 7fc2b3e00640 1 -- 172.21.15.151:0/1498116325 shutdown_connections 2024-05-21T21:55:54.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:54.033+0000 7fc2b3e00640 1 -- 172.21.15.151:0/1498116325 wait complete. 2024-05-21T21:55:54.052 DEBUG:teuthology.orchestra.run.smithi151:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-05-21T21:55:54.074 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-21T21:55:54.129 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-05-21T21:55:54.140 INFO:teuthology.run_tasks:Running task quiescer... 2024-05-21T21:55:54.171 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-21T21:55:54.171 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph pg dump --format=json 2024-05-21T21:55:54.543 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:55:54.595 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:55:54.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:54 smithi184 ceph-mon[36942]: from='client.15156 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:55:54.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:54 smithi184 ceph-mon[36942]: scrub queued for path: / 2024-05-21T21:55:54.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:54 smithi184 ceph-mon[36942]: scrub summary: idle+waiting paths [/] 2024-05-21T21:55:54.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:54 smithi184 ceph-mon[36942]: scrub summary: active paths [/] 2024-05-21T21:55:54.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:54 smithi184 ceph-mon[36942]: scrub summary: idle+waiting paths [/] 2024-05-21T21:55:54.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:54 smithi184 ceph-mon[36942]: from='client.15180 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:55:54.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:54 smithi164 ceph-mon[36269]: from='client.15156 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:55:54.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:54 smithi164 ceph-mon[36269]: scrub queued for path: / 2024-05-21T21:55:54.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:54 smithi164 ceph-mon[36269]: scrub summary: idle+waiting paths [/] 2024-05-21T21:55:54.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:54 smithi164 ceph-mon[36269]: scrub summary: active paths [/] 2024-05-21T21:55:54.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:54 smithi164 ceph-mon[36269]: scrub summary: idle+waiting paths [/] 2024-05-21T21:55:54.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:54 smithi164 ceph-mon[36269]: from='client.15180 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:55:54.843 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:54 smithi151 ceph-mon[31623]: from='client.15156 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:55:54.843 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:54 smithi151 ceph-mon[31623]: scrub queued for path: / 2024-05-21T21:55:54.843 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:54 smithi151 ceph-mon[31623]: scrub summary: idle+waiting paths [/] 2024-05-21T21:55:54.844 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:54 smithi151 ceph-mon[31623]: scrub summary: active paths [/] 2024-05-21T21:55:54.844 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:54 smithi151 ceph-mon[31623]: scrub summary: idle+waiting paths [/] 2024-05-21T21:55:54.844 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:54 smithi151 ceph-mon[31623]: from='client.15180 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:55:55.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:55 smithi151 ceph-mon[31623]: from='client.25315 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:55:55.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:55 smithi151 ceph-mon[31623]: pgmap v12: 97 pgs: 97 active+clean; 1.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 170 B/s wr, 19 op/s 2024-05-21T21:55:55.735 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:55 smithi184 ceph-mon[36942]: from='client.25315 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:55:55.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:55 smithi184 ceph-mon[36942]: pgmap v12: 97 pgs: 97 active+clean; 1.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 170 B/s wr, 19 op/s 2024-05-21T21:55:55.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:55 smithi164 ceph-mon[36269]: from='client.25315 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T21:55:55.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:55 smithi164 ceph-mon[36269]: pgmap v12: 97 pgs: 97 active+clean; 1.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 170 B/s wr, 19 op/s 2024-05-21T21:55:57.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:57 smithi184 ceph-mon[36942]: pgmap v13: 97 pgs: 97 active+clean; 1.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T21:55:57.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:57 smithi164 ceph-mon[36269]: pgmap v13: 97 pgs: 97 active+clean; 1.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T21:55:57.799 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:57 smithi151 ceph-mon[31623]: pgmap v13: 97 pgs: 97 active+clean; 1.1 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T21:55:58.044 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.043+0000 7fe2b6e00700 1 -- 172.21.15.151:0/3491999009 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe2b0076290 msgr2=0x7fe2b010f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:58.045 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.043+0000 7fe2b6e00700 1 --2- 172.21.15.151:0/3491999009 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe2b0076290 0x7fe2b010f770 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe29c0098f0 tx=0x7fe29c009c00 comp rx=0 tx=0).stop 2024-05-21T21:55:58.045 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.043+0000 7fe2b6e00700 1 -- 172.21.15.151:0/3491999009 shutdown_connections 2024-05-21T21:55:58.045 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.043+0000 7fe2b6e00700 1 --2- 172.21.15.151:0/3491999009 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe2b0075970 0x7fe2b0075d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:58.045 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.043+0000 7fe2b6e00700 1 --2- 172.21.15.151:0/3491999009 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe2b010fcb0 0x7fe2b0112120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:58.045 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.043+0000 7fe2b6e00700 1 --2- 172.21.15.151:0/3491999009 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe2b0076290 0x7fe2b010f770 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:58.045 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.043+0000 7fe2b6e00700 1 -- 172.21.15.151:0/3491999009 >> 172.21.15.151:0/3491999009 conn(0x7fe2b00fefc0 msgr2=0x7fe2b01013e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:55:58.045 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.043+0000 7fe2b6e00700 1 -- 172.21.15.151:0/3491999009 shutdown_connections 2024-05-21T21:55:58.045 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.043+0000 7fe2b6e00700 1 -- 172.21.15.151:0/3491999009 wait complete. 2024-05-21T21:55:58.046 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.045+0000 7fe2b6e00700 1 Processor -- start 2024-05-21T21:55:58.046 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.045+0000 7fe2b6e00700 1 -- start start 2024-05-21T21:55:58.047 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.046+0000 7fe2b6e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe2b0075970 0x7fe2b019bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:58.047 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.046+0000 7fe2b6e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe2b0076290 0x7fe2b019c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:58.047 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.046+0000 7fe2b6e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe2b010fcb0 0x7fe2b01a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:58.047 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.046+0000 7fe2b6e00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b01158a0 con 0x7fe2b010fcb0 2024-05-21T21:55:58.047 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.046+0000 7fe2b6e00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b0115720 con 0x7fe2b0076290 2024-05-21T21:55:58.047 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.046+0000 7fe2b6e00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b0115a20 con 0x7fe2b0075970 2024-05-21T21:55:58.047 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.046+0000 7fe2b5a00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe2b0075970 0x7fe2b019bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:58.047 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.046+0000 7fe2b5a00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe2b0075970 0x7fe2b019bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:39952/0 (socket says 172.21.15.151:39952) 2024-05-21T21:55:58.047 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.046+0000 7fe2b5a00700 1 -- 172.21.15.151:0/3697338114 learned_addr learned my addr 172.21.15.151:0/3697338114 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:55:58.047 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.046+0000 7fe2b6400700 1 --2- 172.21.15.151:0/3697338114 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe2b010fcb0 0x7fe2b01a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:58.047 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.046+0000 7fe2b5000700 1 --2- 172.21.15.151:0/3697338114 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe2b0076290 0x7fe2b019c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:58.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.047+0000 7fe2b5a00700 1 -- 172.21.15.151:0/3697338114 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe2b0076290 msgr2=0x7fe2b019c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:58.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.047+0000 7fe2b5a00700 1 --2- 172.21.15.151:0/3697338114 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe2b0076290 0x7fe2b019c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:58.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.047+0000 7fe2b5a00700 1 -- 172.21.15.151:0/3697338114 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe2b010fcb0 msgr2=0x7fe2b01a06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:58.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.047+0000 7fe2b5a00700 1 --2- 172.21.15.151:0/3697338114 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe2b010fcb0 0x7fe2b01a06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:58.049 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.047+0000 7fe2b5a00700 1 -- 172.21.15.151:0/3697338114 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2b01a0e00 con 0x7fe2b0075970 2024-05-21T21:55:58.049 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.047+0000 7fe2b5a00700 1 --2- 172.21.15.151:0/3697338114 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe2b0075970 0x7fe2b019bdf0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe2a400e9c0 tx=0x7fe2a400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:55:58.049 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.047+0000 7fe2ae000700 1 -- 172.21.15.151:0/3697338114 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a400cbd0 con 0x7fe2b0075970 2024-05-21T21:55:58.049 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.047+0000 7fe2ae000700 1 -- 172.21.15.151:0/3697338114 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2a400ee60 con 0x7fe2b0075970 2024-05-21T21:55:58.049 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.047+0000 7fe2b6e00700 1 -- 172.21.15.151:0/3697338114 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2b01a10f0 con 0x7fe2b0075970 2024-05-21T21:55:58.049 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.048+0000 7fe2ae000700 1 -- 172.21.15.151:0/3697338114 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a40055c0 con 0x7fe2b0075970 2024-05-21T21:55:58.049 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.048+0000 7fe2b6e00700 1 -- 172.21.15.151:0/3697338114 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2b01a8970 con 0x7fe2b0075970 2024-05-21T21:55:58.051 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.051+0000 7fe2ae000700 1 -- 172.21.15.151:0/3697338114 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 25) ==== 149829+0+0 (secure 0 0 0) 0x7fe2a4004750 con 0x7fe2b0075970 2024-05-21T21:55:58.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.052+0000 7fe2c2cb9b80 1 Processor -- start 2024-05-21T21:55:58.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.052+0000 7fe2c2cb9b80 1 -- start start 2024-05-21T21:55:58.053 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.053+0000 7fe2c2cb9b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5634993b9020 0x5634993b9400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:58.053 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.053+0000 7fe2c2cb9b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634993b9940 0x5634993b34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:58.053 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.053+0000 7fe2c2cb9b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5634993b3ad0 0x5634993b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:58.053 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.053+0000 7fe2c2cb9b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x563499311e40 con 0x5634993b3ad0 2024-05-21T21:55:58.053 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.053+0000 7fe2c2cb9b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5634993019e0 con 0x5634993b9020 2024-05-21T21:55:58.053 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.053+0000 7fe2c2cb9b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x563499305a20 con 0x5634993b9940 2024-05-21T21:55:58.054 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.053+0000 7fe2b5a00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634993b9940 0x5634993b34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:58.054 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.053+0000 7fe2b5000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5634993b3ad0 0x5634993b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:58.054 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.053+0000 7fe2b5000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5634993b3ad0 0x5634993b3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:60062/0 (socket says 172.21.15.151:60062) 2024-05-21T21:55:58.054 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.053+0000 7fe2b5000700 1 -- 172.21.15.151:0/366408066 learned_addr learned my addr 172.21.15.151:0/366408066 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:55:58.054 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.053+0000 7fe2ae000700 1 --2- 172.21.15.151:0/3697338114 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fe2980b2b00 0x7fe2980b4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:58.055 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.054+0000 7fe2b5000700 1 -- 172.21.15.151:0/366408066 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634993b9940 msgr2=0x5634993b34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:55:58.055 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.054+0000 7fe2b5000700 1 --2- 172.21.15.151:0/366408066 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634993b9940 0x5634993b34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:58.055 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.054+0000 7fe2b5000700 1 -- 172.21.15.151:0/366408066 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5634993b9020 msgr2=0x5634993b9400 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-21T21:55:58.055 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.054+0000 7fe2b5000700 1 --2- 172.21.15.151:0/366408066 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5634993b9020 0x5634993b9400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:58.055 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.054+0000 7fe2b5000700 1 -- 172.21.15.151:0/366408066 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634992ca420 con 0x5634993b3ad0 2024-05-21T21:55:58.055 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.054+0000 7fe2ae000700 1 -- 172.21.15.151:0/3697338114 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fe2a40e2a50 con 0x7fe2b0075970 2024-05-21T21:55:58.055 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.054+0000 7fe2b5000700 1 --2- 172.21.15.151:0/366408066 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5634993b3ad0 0x5634993b3f30 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe29c0098f0 tx=0x7fe29c00bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1b7611695b44dab server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:55:58.055 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.055+0000 7fe2a2a00700 1 -- 172.21.15.151:0/366408066 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe29c026070 con 0x5634993b3ad0 2024-05-21T21:55:58.055 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.055+0000 7fe2a2a00700 1 -- 172.21.15.151:0/366408066 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe29c00f910 con 0x5634993b3ad0 2024-05-21T21:55:58.055 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.055+0000 7fe2a2a00700 1 -- 172.21.15.151:0/366408066 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe29c0038e0 con 0x5634993b3ad0 2024-05-21T21:55:58.056 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.056+0000 7fe2c2cb9b80 1 -- 172.21.15.151:0/366408066 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5634993b3ad0 msgr2=0x5634993b3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:58.056 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.056+0000 7fe2c2cb9b80 1 --2- 172.21.15.151:0/366408066 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5634993b3ad0 0x5634993b3f30 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe29c0098f0 tx=0x7fe29c00bf60 comp rx=0 tx=0).stop 2024-05-21T21:55:58.057 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.057+0000 7fe2b6400700 1 --2- 172.21.15.151:0/3697338114 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fe2980b2b00 0x7fe2980b4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:58.057 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.057+0000 7fe2c2cb9b80 1 -- 172.21.15.151:0/366408066 shutdown_connections 2024-05-21T21:55:58.057 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.057+0000 7fe2c2cb9b80 1 --2- 172.21.15.151:0/366408066 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5634993b9020 0x5634993b9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:58.057 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.057+0000 7fe2c2cb9b80 1 --2- 172.21.15.151:0/366408066 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5634993b3ad0 0x5634993b3f30 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:58.057 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.057+0000 7fe2c2cb9b80 1 --2- 172.21.15.151:0/366408066 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634993b9940 0x5634993b34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:58.057 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.057+0000 7fe2c2cb9b80 1 -- 172.21.15.151:0/366408066 >> 172.21.15.151:0/366408066 conn(0x5634992e1be0 msgr2=0x5634993ae150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:55:58.057 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.057+0000 7fe2c2cb9b80 1 -- 172.21.15.151:0/366408066 shutdown_connections 2024-05-21T21:55:58.057 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.057+0000 7fe2c2cb9b80 1 -- 172.21.15.151:0/366408066 wait complete. 2024-05-21T21:55:58.058 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.058+0000 7fe2b6400700 1 --2- 172.21.15.151:0/3697338114 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fe2980b2b00 0x7fe2980b4fc0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe2a800f4d0 tx=0x7fe2a800b040 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:55:58.058 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.058+0000 7fe2c2cb9b80 1 -- start start 2024-05-21T21:55:58.058 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.058+0000 7fe2c2cb9b80 1 Processor -- start 2024-05-21T21:55:58.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.058+0000 7fe2c2cb9b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634993b3ad0 0x5634993bf6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:58.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.059+0000 7fe2c2cb9b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5634993b9020 0x5634993bcae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:58.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.059+0000 7fe2c2cb9b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5634993b9940 0x5634993bd020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:58.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.059+0000 7fe2c2cb9b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x563499207fa0 con 0x5634993b9940 2024-05-21T21:55:58.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.059+0000 7fe2c2cb9b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5634992a2760 con 0x5634993b9020 2024-05-21T21:55:58.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.059+0000 7fe2c2cb9b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5634993c0400 con 0x5634993b3ad0 2024-05-21T21:55:58.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.059+0000 7fe2c2cb9b80 10 client.0 _ll_register_callbacks cb 0x5634992abc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:55:58.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.059+0000 7fe2b5000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5634993b9020 0x5634993bcae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:58.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.059+0000 7fe2b5000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5634993b9020 0x5634993bcae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:53732/0 (socket says 172.21.15.151:53732) 2024-05-21T21:55:58.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.059+0000 7fe2b5000700 1 -- 172.21.15.151:0/2167205450 learned_addr learned my addr 172.21.15.151:0/2167205450 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:55:58.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.059+0000 7fe2b5a00700 1 --2- 172.21.15.151:0/2167205450 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634993b3ad0 0x5634993bf6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:58.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.059+0000 7fe2b6400700 1 --2- 172.21.15.151:0/2167205450 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5634993b9940 0x5634993bd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:58.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.059+0000 7fe2b5000700 1 -- 172.21.15.151:0/2167205450 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634993b3ad0 msgr2=0x5634993bf6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:55:58.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.059+0000 7fe2b5000700 1 --2- 172.21.15.151:0/2167205450 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634993b3ad0 0x5634993bf6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:58.061 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.059+0000 7fe2b5000700 1 -- 172.21.15.151:0/2167205450 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5634993b9940 msgr2=0x5634993bd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:55:58.061 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.059+0000 7fe2b5000700 1 --2- 172.21.15.151:0/2167205450 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5634993b9940 0x5634993bd020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:58.061 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.059+0000 7fe2b5000700 1 -- 172.21.15.151:0/2167205450 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634992518d0 con 0x5634993b9020 2024-05-21T21:55:58.061 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.059+0000 7fe2b5a00700 1 --2- 172.21.15.151:0/2167205450 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634993b3ad0 0x5634993bf6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:55:58.062 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.060+0000 7fe2b5000700 1 --2- 172.21.15.151:0/2167205450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5634993b9020 0x5634993bcae0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe29c024040 tx=0x7fe29c005050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a36eba42e77733 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:55:58.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.060+0000 7fe297400700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-21T21:55:58.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.060+0000 7fe297400700 1 -- 172.21.15.151:0/2167205450 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe29c026040 con 0x5634993b9020 2024-05-21T21:55:58.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.060+0000 7fe297400700 1 -- 172.21.15.151:0/2167205450 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fe29c00f660 con 0x5634993b9020 2024-05-21T21:55:58.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.060+0000 7fe2c2cb9b80 1 -- 172.21.15.151:0/2167205450 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe29c00f910 con 0x5634993b9020 2024-05-21T21:55:58.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.061+0000 7fe297400700 1 -- 172.21.15.151:0/2167205450 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe29c039050 con 0x5634993b9020 2024-05-21T21:55:58.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.061+0000 7fe2c2cb9b80 10 client.25319 fetch_fsmap learned FSMap version 29 2024-05-21T21:55:58.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.061+0000 7fe2c2cb9b80 1 -- 172.21.15.151:0/2167205450 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5634992d2060 con 0x5634993b9020 2024-05-21T21:55:58.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.061+0000 7fe297400700 1 -- 172.21.15.151:0/2167205450 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe29c00f910 con 0x5634993b9020 2024-05-21T21:55:58.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.061+0000 7fe297400700 1 -- 172.21.15.151:0/2167205450 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fe29c034030 con 0x5634993b9020 2024-05-21T21:55:58.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.061+0000 7fe2c2cb9b80 10 client.25319 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T21:55:58.064 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.061+0000 7fe2c2cb9b80 10 client.25319 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T21:55:58.064 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.061+0000 7fe2c2cb9b80 20 client.25319 populate_metadata read hostname 'smithi151' 2024-05-21T21:55:58.064 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.061+0000 7fe2c2cb9b80 1 --2- 172.21.15.151:0/2167205450 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5634993cce30 0x5634993cf220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:58.064 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.062+0000 7fe2c2cb9b80 4 client.25319 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T21:55:58.064 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.062+0000 7fe2c2cb9b80 1 -- 172.21.15.151:0/2167205450 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563499313d90 con 0x5634993cce30 2024-05-21T21:55:58.064 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.062+0000 7fe2b5a00700 1 --2- 172.21.15.151:0/2167205450 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5634993cce30 0x5634993cf220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:58.064 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.062+0000 7fe2b5a00700 1 --2- 172.21.15.151:0/2167205450 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5634993cce30 0x5634993cf220 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b981573dcf0abd65 server_cookie=5014f796e0c32f64 in_seq=0 out_seq=0 2024-05-21T21:55:58.064 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.062+0000 7fe297400700 10 client.25319 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T21:55:58.065 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.063+0000 7fe297400700 1 -- 172.21.15.151:0/2167205450 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x563499313d90 con 0x5634993cce30 2024-05-21T21:55:58.065 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.063+0000 7fe297400700 10 client.25319 handle_command_reply: tid=0 2024-05-21T21:55:58.065 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.063+0000 7fe2c2cb9b80 1 client.25319 shutdown 2024-05-21T21:55:58.065 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.063+0000 7fe2c2cb9b80 1 -- 172.21.15.151:0/2167205450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5634993b9020 msgr2=0x5634993bcae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:58.065 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.063+0000 7fe2c2cb9b80 1 --2- 172.21.15.151:0/2167205450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5634993b9020 0x5634993bcae0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe29c024040 tx=0x7fe29c005050 comp rx=0 tx=0).stop 2024-05-21T21:55:58.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.063+0000 7fe2c2cb9b80 1 -- 172.21.15.151:0/2167205450 shutdown_connections 2024-05-21T21:55:58.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.063+0000 7fe2c2cb9b80 1 --2- 172.21.15.151:0/2167205450 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5634993b9940 0x5634993bd020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:58.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.063+0000 7fe2c2cb9b80 1 --2- 172.21.15.151:0/2167205450 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634993b3ad0 0x5634993bf6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:58.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.063+0000 7fe2c2cb9b80 1 --2- 172.21.15.151:0/2167205450 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5634993cce30 0x5634993cf220 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:58.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.064+0000 7fe2c2cb9b80 1 --2- 172.21.15.151:0/2167205450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5634993b9020 0x5634993bcae0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:58.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.064+0000 7fe2c2cb9b80 1 -- 172.21.15.151:0/2167205450 >> 172.21.15.151:0/2167205450 conn(0x5634992e1be0 msgr2=0x5634993b24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:55:58.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.064+0000 7fe297400700 0 client.25319 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T21:55:58.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.064+0000 7fe2c2cb9b80 1 -- 172.21.15.151:0/2167205450 shutdown_connections 2024-05-21T21:55:58.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.064+0000 7fe2c2cb9b80 1 -- 172.21.15.151:0/2167205450 wait complete. 2024-05-21T21:55:58.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.064+0000 7fe2c2cb9b80 20 client.25319 trim_cache size 0 max 16384 2024-05-21T21:55:58.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.087+0000 7fe2c2cb9b80 1 Processor -- start 2024-05-21T21:55:58.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.087+0000 7fe2c2cb9b80 1 -- start start 2024-05-21T21:55:58.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.087+0000 7fe2c2cb9b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5634993b3ad0 0x5634993bcaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:58.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.087+0000 7fe2c2cb9b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5634993b9020 0x5634993bcfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:58.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.087+0000 7fe2c2cb9b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634993b9940 0x5634993d2250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:58.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.087+0000 7fe2c2cb9b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b01afd80 con 0x5634993b3ad0 2024-05-21T21:55:58.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.087+0000 7fe2c2cb9b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b01b18c0 con 0x5634993b9020 2024-05-21T21:55:58.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.087+0000 7fe2c2cb9b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b01b0450 con 0x5634993b9940 2024-05-21T21:55:58.089 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.088+0000 7fe2b5a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5634993b3ad0 0x5634993bcaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:58.089 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.088+0000 7fe2b5000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5634993b9020 0x5634993bcfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:58.089 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.088+0000 7fe2b6400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634993b9940 0x5634993d2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:58.089 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.088+0000 7fe2b5000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5634993b9020 0x5634993bcfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:53738/0 (socket says 172.21.15.151:53738) 2024-05-21T21:55:58.089 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.088+0000 7fe2b5000700 1 -- 172.21.15.151:0/653895042 learned_addr learned my addr 172.21.15.151:0/653895042 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:55:58.089 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.088+0000 7fe2b5000700 1 -- 172.21.15.151:0/653895042 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634993b9940 msgr2=0x5634993d2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:55:58.090 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.088+0000 7fe2b5000700 1 --2- 172.21.15.151:0/653895042 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634993b9940 0x5634993d2250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:58.090 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.088+0000 7fe2b5000700 1 -- 172.21.15.151:0/653895042 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5634993b3ad0 msgr2=0x5634993bcaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:55:58.090 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.088+0000 7fe2b5000700 1 --2- 172.21.15.151:0/653895042 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5634993b3ad0 0x5634993bcaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:58.090 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.088+0000 7fe2b5000700 1 -- 172.21.15.151:0/653895042 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634993bf330 con 0x5634993b9020 2024-05-21T21:55:58.090 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.089+0000 7fe2b5000700 1 --2- 172.21.15.151:0/653895042 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5634993b9020 0x5634993bcfe0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe29c011040 tx=0x7fe29c00b560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d394603296210f9d server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:55:58.090 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.089+0000 7fe2a2000700 1 -- 172.21.15.151:0/653895042 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe29c042070 con 0x5634993b9020 2024-05-21T21:55:58.090 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.089+0000 7fe2a2000700 1 -- 172.21.15.151:0/653895042 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe29c00bc20 con 0x5634993b9020 2024-05-21T21:55:58.090 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.089+0000 7fe2a2000700 1 -- 172.21.15.151:0/653895042 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe29c026040 con 0x5634993b9020 2024-05-21T21:55:58.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.090+0000 7fe2c2cb9b80 1 -- 172.21.15.151:0/653895042 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5634993b9020 msgr2=0x5634993bcfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:58.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.090+0000 7fe2c2cb9b80 1 --2- 172.21.15.151:0/653895042 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5634993b9020 0x5634993bcfe0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe29c011040 tx=0x7fe29c00b560 comp rx=0 tx=0).stop 2024-05-21T21:55:58.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.090+0000 7fe2c2cb9b80 1 -- 172.21.15.151:0/653895042 shutdown_connections 2024-05-21T21:55:58.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.090+0000 7fe2c2cb9b80 1 --2- 172.21.15.151:0/653895042 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634993b9940 0x5634993d2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:58.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.090+0000 7fe2c2cb9b80 1 --2- 172.21.15.151:0/653895042 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5634993b3ad0 0x5634993bcaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:58.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.090+0000 7fe2c2cb9b80 1 --2- 172.21.15.151:0/653895042 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5634993b9020 0x5634993bcfe0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:58.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.090+0000 7fe2c2cb9b80 1 -- 172.21.15.151:0/653895042 >> 172.21.15.151:0/653895042 conn(0x5634992e1be0 msgr2=0x5634993adc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:55:58.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.090+0000 7fe2c2cb9b80 1 -- 172.21.15.151:0/653895042 shutdown_connections 2024-05-21T21:55:58.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.090+0000 7fe2c2cb9b80 1 -- 172.21.15.151:0/653895042 wait complete. 2024-05-21T21:55:58.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.091+0000 7fe2c2cb9b80 1 -- start start 2024-05-21T21:55:58.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.091+0000 7fe2c2cb9b80 1 Processor -- start 2024-05-21T21:55:58.093 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.091+0000 7fe2c2cb9b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5634993b3ad0 0x5634993d01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:58.093 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.091+0000 7fe2c2cb9b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634993b9020 0x5634993d0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:58.093 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.091+0000 7fe2c2cb9b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5634993b9940 0x5634993d0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:58.093 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.091+0000 7fe2c2cb9b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b01b11f0 con 0x5634993b3ad0 2024-05-21T21:55:58.094 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.091+0000 7fe2c2cb9b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b01b0b20 con 0x5634993b9940 2024-05-21T21:55:58.094 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.091+0000 7fe2c2cb9b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b01add40 con 0x5634993b9020 2024-05-21T21:55:58.094 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.091+0000 7fe2c2cb9b80 10 client.0 _ll_register_callbacks cb 0x5634992abc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:55:58.094 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.092+0000 7fe2b5a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5634993b3ad0 0x5634993d01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:58.094 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.092+0000 7fe2b5a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5634993b3ad0 0x5634993d01c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:60088/0 (socket says 172.21.15.151:60088) 2024-05-21T21:55:58.094 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.092+0000 7fe2b5a00700 1 -- 172.21.15.151:0/293216268 learned_addr learned my addr 172.21.15.151:0/293216268 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:55:58.094 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.092+0000 7fe2b6400700 1 --2- 172.21.15.151:0/293216268 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5634993b9940 0x5634993d0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:58.094 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.092+0000 7fe2b5000700 1 --2- 172.21.15.151:0/293216268 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634993b9020 0x5634993d0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:58.094 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.092+0000 7fe2b5a00700 1 -- 172.21.15.151:0/293216268 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634993b9020 msgr2=0x5634993d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:55:58.094 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.092+0000 7fe2b5a00700 1 --2- 172.21.15.151:0/293216268 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634993b9020 0x5634993d0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:58.094 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.092+0000 7fe2b5a00700 1 -- 172.21.15.151:0/293216268 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5634993b9940 msgr2=0x5634993d0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:55:58.094 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.092+0000 7fe2b5a00700 1 --2- 172.21.15.151:0/293216268 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5634993b9940 0x5634993d0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:58.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.092+0000 7fe2b5a00700 1 -- 172.21.15.151:0/293216268 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563499436880 con 0x5634993b3ad0 2024-05-21T21:55:58.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.092+0000 7fe2b5a00700 1 --2- 172.21.15.151:0/293216268 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5634993b3ad0 0x5634993d01c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe2a4009fd0 tx=0x7fe2a40cc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49884aeeffbf481c server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:55:58.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.092+0000 7fe297400700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-21T21:55:58.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.093+0000 7fe297400700 1 -- 172.21.15.151:0/293216268 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a40c3060 con 0x5634993b3ad0 2024-05-21T21:55:58.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.093+0000 7fe297400700 1 -- 172.21.15.151:0/293216268 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fe2a40e0550 con 0x5634993b3ad0 2024-05-21T21:55:58.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.093+0000 7fe2c2cb9b80 1 -- 172.21.15.151:0/293216268 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe29c00bc20 con 0x5634993b3ad0 2024-05-21T21:55:58.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.093+0000 7fe297400700 1 -- 172.21.15.151:0/293216268 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe2a40a94c0 con 0x5634993b3ad0 2024-05-21T21:55:58.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.093+0000 7fe2c2cb9b80 10 client.15204 fetch_fsmap learned FSMap version 29 2024-05-21T21:55:58.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.093+0000 7fe2c2cb9b80 1 -- 172.21.15.151:0/293216268 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563499437b50 con 0x5634993b3ad0 2024-05-21T21:55:58.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.093+0000 7fe297400700 1 -- 172.21.15.151:0/293216268 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe29c00bc20 con 0x5634993b3ad0 2024-05-21T21:55:58.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.094+0000 7fe297400700 1 -- 172.21.15.151:0/293216268 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fe2a40a54e0 con 0x5634993b3ad0 2024-05-21T21:55:58.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.094+0000 7fe2c2cb9b80 10 client.15204 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T21:55:58.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.094+0000 7fe2c2cb9b80 10 client.15204 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T21:55:58.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.094+0000 7fe2c2cb9b80 20 client.15204 populate_metadata read hostname 'smithi151' 2024-05-21T21:55:58.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.094+0000 7fe2c2cb9b80 1 --2- 172.21.15.151:0/293216268 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x563499438160 0x56349943a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:58.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.094+0000 7fe2c2cb9b80 4 client.15204 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-21T21:55:58.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.094+0000 7fe2c2cb9b80 1 -- 172.21.15.151:0/293216268 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56349930d570 con 0x563499438160 2024-05-21T21:55:58.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.094+0000 7fe2b5000700 1 --2- 172.21.15.151:0/293216268 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x563499438160 0x56349943a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:58.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.095+0000 7fe2b5000700 1 --2- 172.21.15.151:0/293216268 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x563499438160 0x56349943a580 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a77b37faf746588a server_cookie=22c93ce20853e64f in_seq=0 out_seq=0 2024-05-21T21:55:58.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.095+0000 7fe297400700 10 client.15204 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T21:55:58.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.095+0000 7fe297400700 1 -- 172.21.15.151:0/293216268 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56349930d570 con 0x563499438160 2024-05-21T21:55:58.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.095+0000 7fe297400700 10 client.15204 handle_command_reply: tid=0 2024-05-21T21:55:58.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.095+0000 7fe2c2cb9b80 1 client.15204 shutdown 2024-05-21T21:55:58.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.095+0000 7fe2c2cb9b80 1 -- 172.21.15.151:0/293216268 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5634993b3ad0 msgr2=0x5634993d01c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:58.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.095+0000 7fe2c2cb9b80 1 --2- 172.21.15.151:0/293216268 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5634993b3ad0 0x5634993d01c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe2a4009fd0 tx=0x7fe2a40cc040 comp rx=0 tx=0).stop 2024-05-21T21:55:58.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.095+0000 7fe2c2cb9b80 1 -- 172.21.15.151:0/293216268 shutdown_connections 2024-05-21T21:55:58.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.095+0000 7fe2c2cb9b80 1 --2- 172.21.15.151:0/293216268 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x563499438160 0x56349943a580 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:58.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.095+0000 7fe2c2cb9b80 1 --2- 172.21.15.151:0/293216268 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5634993b9940 0x5634993d0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:58.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.095+0000 7fe2c2cb9b80 1 --2- 172.21.15.151:0/293216268 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5634993b3ad0 0x5634993d01c0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:58.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.095+0000 7fe2c2cb9b80 1 --2- 172.21.15.151:0/293216268 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5634993b9020 0x5634993d0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:58.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.095+0000 7fe297400700 0 client.15204 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T21:55:58.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.095+0000 7fe2c2cb9b80 1 -- 172.21.15.151:0/293216268 >> 172.21.15.151:0/293216268 conn(0x5634992e1be0 msgr2=0x5634993bc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:55:58.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.095+0000 7fe2c2cb9b80 1 -- 172.21.15.151:0/293216268 shutdown_connections 2024-05-21T21:55:58.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.095+0000 7fe2c2cb9b80 1 -- 172.21.15.151:0/293216268 wait complete. 2024-05-21T21:55:58.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.096+0000 7fe2c2cb9b80 20 client.15204 trim_cache size 0 max 16384 2024-05-21T21:55:58.099 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-05-21T21:55:58.099 INFO:teuthology.orchestra.run.smithi151.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-21T21:55:58.099 INFO:teuthology.orchestra.run.smithi151.stdout: "scrubs": { 2024-05-21T21:55:58.099 INFO:teuthology.orchestra.run.smithi151.stdout: "02ccb105-4b0a-4d9a-a4bb-6b2e269f31aa": { 2024-05-21T21:55:58.099 INFO:teuthology.orchestra.run.smithi151.stdout: "path": "/", 2024-05-21T21:55:58.099 INFO:teuthology.orchestra.run.smithi151.stdout: "tag": "02ccb105-4b0a-4d9a-a4bb-6b2e269f31aa", 2024-05-21T21:55:58.099 INFO:teuthology.orchestra.run.smithi151.stdout: "options": "recursive,force" 2024-05-21T21:55:58.099 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T21:55:58.099 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T21:55:58.099 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-05-21T21:55:58.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.096+0000 7fe2b6e00700 1 -- 172.21.15.151:0/3697338114 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fe2980b2b00 msgr2=0x7fe2980b4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:58.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.096+0000 7fe2b6e00700 1 --2- 172.21.15.151:0/3697338114 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fe2980b2b00 0x7fe2980b4fc0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe2a800f4d0 tx=0x7fe2a800b040 comp rx=0 tx=0).stop 2024-05-21T21:55:58.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.096+0000 7fe2b6e00700 1 -- 172.21.15.151:0/3697338114 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe2b0075970 msgr2=0x7fe2b019bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:58.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.096+0000 7fe2b6e00700 1 --2- 172.21.15.151:0/3697338114 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe2b0075970 0x7fe2b019bdf0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe2a400e9c0 tx=0x7fe2a400ecd0 comp rx=0 tx=0).stop 2024-05-21T21:55:58.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.096+0000 7fe2b6e00700 1 -- 172.21.15.151:0/3697338114 shutdown_connections 2024-05-21T21:55:58.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.096+0000 7fe2b6e00700 1 --2- 172.21.15.151:0/3697338114 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fe2980b2b00 0x7fe2980b4fc0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:58.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.096+0000 7fe2b6e00700 1 --2- 172.21.15.151:0/3697338114 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe2b010fcb0 0x7fe2b01a06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:58.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.096+0000 7fe2b6e00700 1 --2- 172.21.15.151:0/3697338114 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe2b0075970 0x7fe2b019bdf0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:58.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.096+0000 7fe2b6e00700 1 --2- 172.21.15.151:0/3697338114 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe2b0076290 0x7fe2b019c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:58.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.096+0000 7fe2b6e00700 1 -- 172.21.15.151:0/3697338114 >> 172.21.15.151:0/3697338114 conn(0x7fe2b00fefc0 msgr2=0x7fe2b00ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:55:58.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.097+0000 7fe2b6e00700 1 -- 172.21.15.151:0/3697338114 shutdown_connections 2024-05-21T21:55:58.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.097+0000 7fe2b6e00700 1 -- 172.21.15.151:0/3697338114 wait complete. 2024-05-21T21:55:58.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.138+0000 7f83c5800700 1 -- 172.21.15.151:0/1004383318 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f83c010fff0 msgr2=0x7f83c01123e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:58.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.138+0000 7f83c5800700 1 --2- 172.21.15.151:0/1004383318 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f83c010fff0 0x7f83c01123e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f83b8009f90 tx=0x7f83b800b330 comp rx=0 tx=0).stop 2024-05-21T21:55:58.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.138+0000 7f83c5800700 1 -- 172.21.15.151:0/1004383318 shutdown_connections 2024-05-21T21:55:58.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.138+0000 7f83c5800700 1 --2- 172.21.15.151:0/1004383318 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f83c006b250 0x7f83c006b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:58.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.138+0000 7f83c5800700 1 --2- 172.21.15.151:0/1004383318 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f83c010fff0 0x7f83c01123e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:58.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.138+0000 7f83c5800700 1 --2- 172.21.15.151:0/1004383318 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f83c006bb70 0x7f83c010fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:58.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.138+0000 7f83c5800700 1 -- 172.21.15.151:0/1004383318 >> 172.21.15.151:0/1004383318 conn(0x7f83c00793f0 msgr2=0x7f83c0079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:55:58.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.138+0000 7f83c5800700 1 -- 172.21.15.151:0/1004383318 shutdown_connections 2024-05-21T21:55:58.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.138+0000 7f83c5800700 1 -- 172.21.15.151:0/1004383318 wait complete. 2024-05-21T21:55:58.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.140+0000 7f83c5800700 1 Processor -- start 2024-05-21T21:55:58.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.140+0000 7f83c5800700 1 -- start start 2024-05-21T21:55:58.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.140+0000 7f83c5800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f83c006b250 0x7f83c019bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:58.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.140+0000 7f83c5800700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f83c006bb70 0x7f83c019c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:58.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.140+0000 7f83c5800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f83c010fff0 0x7f83c01a07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:58.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.140+0000 7f83c5800700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f83c0115960 con 0x7f83c010fff0 2024-05-21T21:55:58.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.140+0000 7f83c5800700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f83c01157e0 con 0x7f83c006b250 2024-05-21T21:55:58.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.140+0000 7f83c5800700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f83c0115ae0 con 0x7f83c006bb70 2024-05-21T21:55:58.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.141+0000 7f83bf400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f83c010fff0 0x7f83c01a07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:58.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.141+0000 7f83bf400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f83c010fff0 0x7f83c01a07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:60102/0 (socket says 172.21.15.151:60102) 2024-05-21T21:55:58.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.141+0000 7f83bf400700 1 -- 172.21.15.151:0/3826466525 learned_addr learned my addr 172.21.15.151:0/3826466525 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:55:58.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.141+0000 7f83bea00700 1 --2- 172.21.15.151:0/3826466525 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f83c006b250 0x7f83c019bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:58.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.141+0000 7f83b7e00700 1 --2- 172.21.15.151:0/3826466525 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f83c006bb70 0x7f83c019c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:58.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.141+0000 7f83bf400700 1 -- 172.21.15.151:0/3826466525 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f83c006bb70 msgr2=0x7f83c019c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:58.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.141+0000 7f83bf400700 1 --2- 172.21.15.151:0/3826466525 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f83c006bb70 0x7f83c019c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:58.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.141+0000 7f83bf400700 1 -- 172.21.15.151:0/3826466525 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f83c006b250 msgr2=0x7f83c019bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:58.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.141+0000 7f83bf400700 1 --2- 172.21.15.151:0/3826466525 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f83c006b250 0x7f83c019bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:58.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.141+0000 7f83bf400700 1 -- 172.21.15.151:0/3826466525 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83c01a0ee0 con 0x7f83c010fff0 2024-05-21T21:55:58.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.141+0000 7f83bea00700 1 --2- 172.21.15.151:0/3826466525 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f83c006b250 0x7f83c019bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:55:58.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.141+0000 7f83bf400700 1 --2- 172.21.15.151:0/3826466525 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f83c010fff0 0x7f83c01a07a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f83b8009f60 tx=0x7f83b8009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:55:58.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.142+0000 7f83b7400700 1 -- 172.21.15.151:0/3826466525 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b8021070 con 0x7f83c010fff0 2024-05-21T21:55:58.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.142+0000 7f83b7400700 1 -- 172.21.15.151:0/3826466525 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83b8007ac0 con 0x7f83c010fff0 2024-05-21T21:55:58.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.142+0000 7f83c5800700 1 -- 172.21.15.151:0/3826466525 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83c01a1170 con 0x7f83c010fff0 2024-05-21T21:55:58.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.142+0000 7f83b7400700 1 -- 172.21.15.151:0/3826466525 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b8005040 con 0x7f83c010fff0 2024-05-21T21:55:58.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.142+0000 7f83c5800700 1 -- 172.21.15.151:0/3826466525 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83c01a8a50 con 0x7f83c010fff0 2024-05-21T21:55:58.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.143+0000 7f83c5800700 1 -- 172.21.15.151:0/3826466525 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83c0102a60 con 0x7f83c010fff0 2024-05-21T21:55:58.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.145+0000 7f83b7400700 1 -- 172.21.15.151:0/3826466525 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 25) ==== 149829+0+0 (secure 0 0 0) 0x7f83b8024070 con 0x7f83c010fff0 2024-05-21T21:55:58.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.147+0000 7f83b7400700 1 --2- 172.21.15.151:0/3826466525 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f83a40b2a80 0x7f83a40b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:55:58.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.147+0000 7f83b7400700 1 -- 172.21.15.151:0/3826466525 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f83b80e3860 con 0x7f83c010fff0 2024-05-21T21:55:58.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.147+0000 7f83bea00700 1 --2- 172.21.15.151:0/3826466525 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f83a40b2a80 0x7f83a40b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:55:58.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.148+0000 7f83bea00700 1 --2- 172.21.15.151:0/3826466525 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f83a40b2a80 0x7f83a40b4f40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f83b0011490 tx=0x7f83b0009260 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:55:58.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.153+0000 7f83b7400700 1 -- 172.21.15.151:0/3826466525 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f83b80ab110 con 0x7f83c010fff0 2024-05-21T21:55:58.392 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.392+0000 7f83c5800700 1 -- 172.21.15.151:0/3826466525 --> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f83c019d230 con 0x7f83a40b2a80 2024-05-21T21:55:58.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.405+0000 7f83b7400700 1 -- 172.21.15.151:0/3826466525 <== mgr.25138 v2:172.21.15.151:6800/958417056 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257820 (secure 0 0 0) 0x7f83c019d230 con 0x7f83a40b2a80 2024-05-21T21:55:58.410 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T21:55:58.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.410+0000 7f83c5800700 1 -- 172.21.15.151:0/3826466525 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f83a40b2a80 msgr2=0x7f83a40b4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:58.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.410+0000 7f83c5800700 1 --2- 172.21.15.151:0/3826466525 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f83a40b2a80 0x7f83a40b4f40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f83b0011490 tx=0x7f83b0009260 comp rx=0 tx=0).stop 2024-05-21T21:55:58.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.410+0000 7f83c5800700 1 -- 172.21.15.151:0/3826466525 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f83c010fff0 msgr2=0x7f83c01a07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:55:58.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.410+0000 7f83c5800700 1 --2- 172.21.15.151:0/3826466525 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f83c010fff0 0x7f83c01a07a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f83b8009f60 tx=0x7f83b8009510 comp rx=0 tx=0).stop 2024-05-21T21:55:58.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.411+0000 7f83c5800700 1 -- 172.21.15.151:0/3826466525 shutdown_connections 2024-05-21T21:55:58.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.411+0000 7f83c5800700 1 --2- 172.21.15.151:0/3826466525 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f83a40b2a80 0x7f83a40b4f40 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:58.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.411+0000 7f83c5800700 1 --2- 172.21.15.151:0/3826466525 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f83c006b250 0x7f83c019bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:58.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.411+0000 7f83c5800700 1 --2- 172.21.15.151:0/3826466525 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f83c010fff0 0x7f83c01a07a0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:58.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.411+0000 7f83c5800700 1 --2- 172.21.15.151:0/3826466525 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f83c006bb70 0x7f83c019c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:55:58.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.411+0000 7f83c5800700 1 -- 172.21.15.151:0/3826466525 >> 172.21.15.151:0/3826466525 conn(0x7f83c00793f0 msgr2=0x7f83c0111500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:55:58.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.411+0000 7f83c5800700 1 -- 172.21.15.151:0/3826466525 shutdown_connections 2024-05-21T21:55:58.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:55:58.411+0000 7f83c5800700 1 -- 172.21.15.151:0/3826466525 wait complete. 2024-05-21T21:55:58.413 INFO:teuthology.orchestra.run.smithi151.stderr:dumped all 2024-05-21T21:55:59.265 INFO:teuthology.orchestra.run.smithi151.stdout:{"pg_ready":true,"pg_map":{"version":14,"stamp":"2024-05-21T21:55:58.225638+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1389024,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":3121,"num_read_kb":1958,"num_write":646,"num_write_kb":4538,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":2952069,"num_keys_recovered":17,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":988,"ondisk_log_size":988,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":368760,"kb_used_data":15072,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124573064,"statfs":{"total":1151940427776,"available":1151562817536,"internally_reserved":0,"allocated":15433728,"data_stored":9467748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19140,"internal_metadata":361674044},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":197770,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":406,"num_read_kb":309,"num_write":128,"num_write_kb":917,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004480"},"pg_stats":[{"pgid":"3.3f","version":"82'8","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.341334+0000","last_change":"2024-05-21T21:51:16.945132+0000","last_active":"2024-05-21T21:55:36.341334+0000","last_peered":"2024-05-21T21:55:36.341334+0000","last_clean":"2024-05-21T21:55:36.341334+0000","last_became_active":"2024-05-21T21:51:16.944766+0000","last_became_peered":"2024-05-21T21:51:16.944766+0000","last_unstale":"2024-05-21T21:55:36.341334+0000","last_undegraded":"2024-05-21T21:55:36.341334+0000","last_fullsized":"2024-05-21T21:55:36.341334+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:54:04.185095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"82'4","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.553624+0000","last_change":"2024-05-21T21:51:16.939291+0000","last_active":"2024-05-21T21:55:36.553624+0000","last_peered":"2024-05-21T21:55:36.553624+0000","last_clean":"2024-05-21T21:55:36.553624+0000","last_became_active":"2024-05-21T21:51:16.938730+0000","last_became_peered":"2024-05-21T21:51:16.938730+0000","last_unstale":"2024-05-21T21:55:36.553624+0000","last_undegraded":"2024-05-21T21:55:36.553624+0000","last_fullsized":"2024-05-21T21:55:36.553624+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:05:30.623744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"82'7","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.553252+0000","last_change":"2024-05-21T21:51:16.941565+0000","last_active":"2024-05-21T21:55:36.553252+0000","last_peered":"2024-05-21T21:55:36.553252+0000","last_clean":"2024-05-21T21:55:36.553252+0000","last_became_active":"2024-05-21T21:51:16.941033+0000","last_became_peered":"2024-05-21T21:51:16.941033+0000","last_unstale":"2024-05-21T21:55:36.553252+0000","last_undegraded":"2024-05-21T21:55:36.553252+0000","last_fullsized":"2024-05-21T21:55:36.553252+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:04:55.789479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"82'8","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.723315+0000","last_change":"2024-05-21T21:51:16.928455+0000","last_active":"2024-05-21T21:55:36.723315+0000","last_peered":"2024-05-21T21:55:36.723315+0000","last_clean":"2024-05-21T21:55:36.723315+0000","last_became_active":"2024-05-21T21:51:16.927874+0000","last_became_peered":"2024-05-21T21:51:16.927874+0000","last_unstale":"2024-05-21T21:55:36.723315+0000","last_undegraded":"2024-05-21T21:55:36.723315+0000","last_fullsized":"2024-05-21T21:55:36.723315+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:25:16.483175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"82'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.556362+0000","last_change":"2024-05-21T21:51:16.937773+0000","last_active":"2024-05-21T21:55:36.556362+0000","last_peered":"2024-05-21T21:55:36.556362+0000","last_clean":"2024-05-21T21:55:36.556362+0000","last_became_active":"2024-05-21T21:51:16.937306+0000","last_became_peered":"2024-05-21T21:51:16.937306+0000","last_unstale":"2024-05-21T21:55:36.556362+0000","last_undegraded":"2024-05-21T21:55:36.556362+0000","last_fullsized":"2024-05-21T21:55:36.556362+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:05:03.762390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"82'6","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.356010+0000","last_change":"2024-05-21T21:51:16.948275+0000","last_active":"2024-05-21T21:55:36.356010+0000","last_peered":"2024-05-21T21:55:36.356010+0000","last_clean":"2024-05-21T21:55:36.356010+0000","last_became_active":"2024-05-21T21:51:16.947890+0000","last_became_peered":"2024-05-21T21:51:16.947890+0000","last_unstale":"2024-05-21T21:55:36.356010+0000","last_undegraded":"2024-05-21T21:55:36.356010+0000","last_fullsized":"2024-05-21T21:55:36.356010+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:08:38.572177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"82'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.550575+0000","last_change":"2024-05-21T21:51:16.946821+0000","last_active":"2024-05-21T21:55:36.550575+0000","last_peered":"2024-05-21T21:55:36.550575+0000","last_clean":"2024-05-21T21:55:36.550575+0000","last_became_active":"2024-05-21T21:51:16.946515+0000","last_became_peered":"2024-05-21T21:51:16.946515+0000","last_unstale":"2024-05-21T21:55:36.550575+0000","last_undegraded":"2024-05-21T21:55:36.550575+0000","last_fullsized":"2024-05-21T21:55:36.550575+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T09:26:25.867895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.342252+0000","last_change":"2024-05-21T21:51:16.929228+0000","last_active":"2024-05-21T21:55:36.342252+0000","last_peered":"2024-05-21T21:55:36.342252+0000","last_clean":"2024-05-21T21:55:36.342252+0000","last_became_active":"2024-05-21T21:51:16.928875+0000","last_became_peered":"2024-05-21T21:51:16.928875+0000","last_unstale":"2024-05-21T21:55:36.342252+0000","last_undegraded":"2024-05-21T21:55:36.342252+0000","last_fullsized":"2024-05-21T21:55:36.342252+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:50:56.431310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"82'11","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.199749+0000","last_change":"2024-05-21T21:51:16.932360+0000","last_active":"2024-05-21T21:55:36.199749+0000","last_peered":"2024-05-21T21:55:36.199749+0000","last_clean":"2024-05-21T21:55:36.199749+0000","last_became_active":"2024-05-21T21:51:16.932036+0000","last_became_peered":"2024-05-21T21:51:16.932036+0000","last_unstale":"2024-05-21T21:55:36.199749+0000","last_undegraded":"2024-05-21T21:55:36.199749+0000","last_fullsized":"2024-05-21T21:55:36.199749+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T09:06:37.257323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"82'16","reported_seq":45,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.356382+0000","last_change":"2024-05-21T21:52:10.876556+0000","last_active":"2024-05-21T21:55:36.356382+0000","last_peered":"2024-05-21T21:55:36.356382+0000","last_clean":"2024-05-21T21:55:36.356382+0000","last_became_active":"2024-05-21T21:52:10.876273+0000","last_became_peered":"2024-05-21T21:52:10.876273+0000","last_unstale":"2024-05-21T21:55:36.356382+0000","last_undegraded":"2024-05-21T21:55:36.356382+0000","last_fullsized":"2024-05-21T21:55:36.356382+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T05:09:40.037731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"82'6","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.550761+0000","last_change":"2024-05-21T21:51:16.933425+0000","last_active":"2024-05-21T21:55:36.550761+0000","last_peered":"2024-05-21T21:55:36.550761+0000","last_clean":"2024-05-21T21:55:36.550761+0000","last_became_active":"2024-05-21T21:51:16.933057+0000","last_became_peered":"2024-05-21T21:51:16.933057+0000","last_unstale":"2024-05-21T21:55:36.550761+0000","last_undegraded":"2024-05-21T21:55:36.550761+0000","last_fullsized":"2024-05-21T21:55:36.550761+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:04:57.237841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"82'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.723313+0000","last_change":"2024-05-21T21:51:16.927014+0000","last_active":"2024-05-21T21:55:36.723313+0000","last_peered":"2024-05-21T21:55:36.723313+0000","last_clean":"2024-05-21T21:55:36.723313+0000","last_became_active":"2024-05-21T21:51:16.925954+0000","last_became_peered":"2024-05-21T21:51:16.925954+0000","last_unstale":"2024-05-21T21:55:36.723313+0000","last_undegraded":"2024-05-21T21:55:36.723313+0000","last_fullsized":"2024-05-21T21:55:36.723313+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:19:20.530727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"82'10","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.555649+0000","last_change":"2024-05-21T21:51:16.930909+0000","last_active":"2024-05-21T21:55:36.555649+0000","last_peered":"2024-05-21T21:55:36.555649+0000","last_clean":"2024-05-21T21:55:36.555649+0000","last_became_active":"2024-05-21T21:51:16.927110+0000","last_became_peered":"2024-05-21T21:51:16.927110+0000","last_unstale":"2024-05-21T21:55:36.555649+0000","last_undegraded":"2024-05-21T21:55:36.555649+0000","last_fullsized":"2024-05-21T21:55:36.555649+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T06:23:55.386047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"82'7","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.552611+0000","last_change":"2024-05-21T21:51:16.938797+0000","last_active":"2024-05-21T21:55:36.552611+0000","last_peered":"2024-05-21T21:55:36.552611+0000","last_clean":"2024-05-21T21:55:36.552611+0000","last_became_active":"2024-05-21T21:51:16.938263+0000","last_became_peered":"2024-05-21T21:51:16.938263+0000","last_unstale":"2024-05-21T21:55:36.552611+0000","last_undegraded":"2024-05-21T21:55:36.552611+0000","last_fullsized":"2024-05-21T21:55:36.552611+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:37:40.521118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"82'7","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.552363+0000","last_change":"2024-05-21T21:51:16.944160+0000","last_active":"2024-05-21T21:55:36.552363+0000","last_peered":"2024-05-21T21:55:36.552363+0000","last_clean":"2024-05-21T21:55:36.552363+0000","last_became_active":"2024-05-21T21:51:16.940046+0000","last_became_peered":"2024-05-21T21:51:16.940046+0000","last_unstale":"2024-05-21T21:55:36.552363+0000","last_undegraded":"2024-05-21T21:55:36.552363+0000","last_fullsized":"2024-05-21T21:55:36.552363+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:14:13.278799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.198999+0000","last_change":"2024-05-21T21:51:16.945599+0000","last_active":"2024-05-21T21:55:36.198999+0000","last_peered":"2024-05-21T21:55:36.198999+0000","last_clean":"2024-05-21T21:55:36.198999+0000","last_became_active":"2024-05-21T21:51:16.945196+0000","last_became_peered":"2024-05-21T21:51:16.945196+0000","last_unstale":"2024-05-21T21:55:36.198999+0000","last_undegraded":"2024-05-21T21:55:36.198999+0000","last_fullsized":"2024-05-21T21:55:36.198999+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:59:24.453615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"82'8","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.341533+0000","last_change":"2024-05-21T21:51:16.944895+0000","last_active":"2024-05-21T21:55:36.341533+0000","last_peered":"2024-05-21T21:55:36.341533+0000","last_clean":"2024-05-21T21:55:36.341533+0000","last_became_active":"2024-05-21T21:51:16.944560+0000","last_became_peered":"2024-05-21T21:51:16.944560+0000","last_unstale":"2024-05-21T21:55:36.341533+0000","last_undegraded":"2024-05-21T21:55:36.341533+0000","last_fullsized":"2024-05-21T21:55:36.341533+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:53:13.264701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"82'10","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.197923+0000","last_change":"2024-05-21T21:51:16.943861+0000","last_active":"2024-05-21T21:55:36.197923+0000","last_peered":"2024-05-21T21:55:36.197923+0000","last_clean":"2024-05-21T21:55:36.197923+0000","last_became_active":"2024-05-21T21:51:16.943123+0000","last_became_peered":"2024-05-21T21:51:16.943123+0000","last_unstale":"2024-05-21T21:55:36.197923+0000","last_undegraded":"2024-05-21T21:55:36.197923+0000","last_fullsized":"2024-05-21T21:55:36.197923+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:35:37.732855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"82'7","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.201474+0000","last_change":"2024-05-21T21:51:16.940580+0000","last_active":"2024-05-21T21:55:36.201474+0000","last_peered":"2024-05-21T21:55:36.201474+0000","last_clean":"2024-05-21T21:55:36.201474+0000","last_became_active":"2024-05-21T21:51:16.940264+0000","last_became_peered":"2024-05-21T21:51:16.940264+0000","last_unstale":"2024-05-21T21:55:36.201474+0000","last_undegraded":"2024-05-21T21:55:36.201474+0000","last_fullsized":"2024-05-21T21:55:36.201474+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:54:22.915541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'14","reported_seq":304,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:55.928643+0000","last_change":"2024-05-21T21:51:12.852935+0000","last_active":"2024-05-21T21:55:55.928643+0000","last_peered":"2024-05-21T21:55:55.928643+0000","last_clean":"2024-05-21T21:55:55.928643+0000","last_became_active":"2024-05-21T21:51:12.851802+0000","last_became_peered":"2024-05-21T21:51:12.851802+0000","last_unstale":"2024-05-21T21:55:55.928643+0000","last_undegraded":"2024-05-21T21:55:55.928643+0000","last_fullsized":"2024-05-21T21:55:55.928643+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:13:24.643408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":260,"num_read_kb":260,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"82'5","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.343135+0000","last_change":"2024-05-21T21:51:16.939461+0000","last_active":"2024-05-21T21:55:36.343135+0000","last_peered":"2024-05-21T21:55:36.343135+0000","last_clean":"2024-05-21T21:55:36.343135+0000","last_became_active":"2024-05-21T21:51:16.936322+0000","last_became_peered":"2024-05-21T21:51:16.936322+0000","last_unstale":"2024-05-21T21:55:36.343135+0000","last_undegraded":"2024-05-21T21:55:36.343135+0000","last_fullsized":"2024-05-21T21:55:36.343135+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:37:26.153963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.343791+0000","last_change":"2024-05-21T21:51:12.941838+0000","last_active":"2024-05-21T21:55:36.343791+0000","last_peered":"2024-05-21T21:55:36.343791+0000","last_clean":"2024-05-21T21:55:36.343791+0000","last_became_active":"2024-05-21T21:51:12.940818+0000","last_became_peered":"2024-05-21T21:51:12.940818+0000","last_unstale":"2024-05-21T21:55:36.343791+0000","last_undegraded":"2024-05-21T21:55:36.343791+0000","last_fullsized":"2024-05-21T21:55:36.343791+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:30:42.349241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"82'6","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.554128+0000","last_change":"2024-05-21T21:51:16.950252+0000","last_active":"2024-05-21T21:55:36.554128+0000","last_peered":"2024-05-21T21:55:36.554128+0000","last_clean":"2024-05-21T21:55:36.554128+0000","last_became_active":"2024-05-21T21:51:16.947375+0000","last_became_peered":"2024-05-21T21:51:16.947375+0000","last_unstale":"2024-05-21T21:55:36.554128+0000","last_undegraded":"2024-05-21T21:55:36.554128+0000","last_fullsized":"2024-05-21T21:55:36.554128+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:27:30.462799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"83'13","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.341387+0000","last_change":"2024-05-21T21:52:10.875739+0000","last_active":"2024-05-21T21:55:36.341387+0000","last_peered":"2024-05-21T21:55:36.341387+0000","last_clean":"2024-05-21T21:55:36.341387+0000","last_became_active":"2024-05-21T21:52:10.865846+0000","last_became_peered":"2024-05-21T21:52:10.865846+0000","last_unstale":"2024-05-21T21:55:36.341387+0000","last_undegraded":"2024-05-21T21:55:36.341387+0000","last_fullsized":"2024-05-21T21:55:36.341387+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T06:48:06.465757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.199107+0000","last_change":"2024-05-21T21:51:16.926380+0000","last_active":"2024-05-21T21:55:36.199107+0000","last_peered":"2024-05-21T21:55:36.199107+0000","last_clean":"2024-05-21T21:55:36.199107+0000","last_became_active":"2024-05-21T21:51:16.925821+0000","last_became_peered":"2024-05-21T21:51:16.925821+0000","last_unstale":"2024-05-21T21:55:36.199107+0000","last_undegraded":"2024-05-21T21:55:36.199107+0000","last_fullsized":"2024-05-21T21:55:36.199107+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:42:14.887155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'5","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.556613+0000","last_change":"2024-05-21T21:51:12.854342+0000","last_active":"2024-05-21T21:55:36.556613+0000","last_peered":"2024-05-21T21:55:36.556613+0000","last_clean":"2024-05-21T21:55:36.556613+0000","last_became_active":"2024-05-21T21:51:12.853908+0000","last_became_peered":"2024-05-21T21:51:12.853908+0000","last_unstale":"2024-05-21T21:55:36.556613+0000","last_undegraded":"2024-05-21T21:55:36.556613+0000","last_fullsized":"2024-05-21T21:55:36.556613+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:35:10.374749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"82'15","reported_seq":44,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.352781+0000","last_change":"2024-05-21T21:52:10.973816+0000","last_active":"2024-05-21T21:55:36.352781+0000","last_peered":"2024-05-21T21:55:36.352781+0000","last_clean":"2024-05-21T21:55:36.352781+0000","last_became_active":"2024-05-21T21:52:10.973457+0000","last_became_peered":"2024-05-21T21:52:10.973457+0000","last_unstale":"2024-05-21T21:55:36.352781+0000","last_undegraded":"2024-05-21T21:55:36.352781+0000","last_fullsized":"2024-05-21T21:55:36.352781+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T05:01:41.915158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"83'9","reported_seq":302,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:55.912999+0000","last_change":"2024-05-21T21:51:12.855658+0000","last_active":"2024-05-21T21:55:55.912999+0000","last_peered":"2024-05-21T21:55:55.912999+0000","last_clean":"2024-05-21T21:55:55.912999+0000","last_became_active":"2024-05-21T21:51:12.855281+0000","last_became_peered":"2024-05-21T21:51:12.855281+0000","last_unstale":"2024-05-21T21:55:55.912999+0000","last_undegraded":"2024-05-21T21:55:55.912999+0000","last_fullsized":"2024-05-21T21:55:55.912999+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:55:08.836045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1855,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":264,"num_read_kb":6,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"82'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.556310+0000","last_change":"2024-05-21T21:51:16.946977+0000","last_active":"2024-05-21T21:55:36.556310+0000","last_peered":"2024-05-21T21:55:36.556310+0000","last_clean":"2024-05-21T21:55:36.556310+0000","last_became_active":"2024-05-21T21:51:16.946562+0000","last_became_peered":"2024-05-21T21:51:16.946562+0000","last_unstale":"2024-05-21T21:55:36.556310+0000","last_undegraded":"2024-05-21T21:55:36.556310+0000","last_fullsized":"2024-05-21T21:55:36.556310+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:43:33.370265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"80'7","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.355936+0000","last_change":"2024-05-21T21:51:12.852574+0000","last_active":"2024-05-21T21:55:36.355936+0000","last_peered":"2024-05-21T21:55:36.355936+0000","last_clean":"2024-05-21T21:55:36.355936+0000","last_became_active":"2024-05-21T21:51:12.852258+0000","last_became_peered":"2024-05-21T21:51:12.852258+0000","last_unstale":"2024-05-21T21:55:36.355936+0000","last_undegraded":"2024-05-21T21:55:36.355936+0000","last_fullsized":"2024-05-21T21:55:36.355936+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:09:40.280821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"82'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.352655+0000","last_change":"2024-05-21T21:51:16.934768+0000","last_active":"2024-05-21T21:55:36.352655+0000","last_peered":"2024-05-21T21:55:36.352655+0000","last_clean":"2024-05-21T21:55:36.352655+0000","last_became_active":"2024-05-21T21:51:16.934275+0000","last_became_peered":"2024-05-21T21:51:16.934275+0000","last_unstale":"2024-05-21T21:55:36.352655+0000","last_undegraded":"2024-05-21T21:55:36.352655+0000","last_fullsized":"2024-05-21T21:55:36.352655+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:31:54.425069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"80'3","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.555219+0000","last_change":"2024-05-21T21:51:12.854166+0000","last_active":"2024-05-21T21:55:36.555219+0000","last_peered":"2024-05-21T21:55:36.555219+0000","last_clean":"2024-05-21T21:55:36.555219+0000","last_became_active":"2024-05-21T21:51:12.853508+0000","last_became_peered":"2024-05-21T21:51:12.853508+0000","last_unstale":"2024-05-21T21:55:36.555219+0000","last_undegraded":"2024-05-21T21:55:36.555219+0000","last_fullsized":"2024-05-21T21:55:36.555219+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:29:41.858655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"82'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.342248+0000","last_change":"2024-05-21T21:51:16.938293+0000","last_active":"2024-05-21T21:55:36.342248+0000","last_peered":"2024-05-21T21:55:36.342248+0000","last_clean":"2024-05-21T21:55:36.342248+0000","last_became_active":"2024-05-21T21:51:16.937970+0000","last_became_peered":"2024-05-21T21:51:16.937970+0000","last_unstale":"2024-05-21T21:55:36.342248+0000","last_undegraded":"2024-05-21T21:55:36.342248+0000","last_fullsized":"2024-05-21T21:55:36.342248+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:18:27.473098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'3","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.352476+0000","last_change":"2024-05-21T21:51:12.853366+0000","last_active":"2024-05-21T21:55:36.352476+0000","last_peered":"2024-05-21T21:55:36.352476+0000","last_clean":"2024-05-21T21:55:36.352476+0000","last_became_active":"2024-05-21T21:51:12.853028+0000","last_became_peered":"2024-05-21T21:51:12.853028+0000","last_unstale":"2024-05-21T21:55:36.352476+0000","last_undegraded":"2024-05-21T21:55:36.352476+0000","last_fullsized":"2024-05-21T21:55:36.352476+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T09:15:52.593028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"82'11","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.197181+0000","last_change":"2024-05-21T21:51:16.947903+0000","last_active":"2024-05-21T21:55:36.197181+0000","last_peered":"2024-05-21T21:55:36.197181+0000","last_clean":"2024-05-21T21:55:36.197181+0000","last_became_active":"2024-05-21T21:51:16.946677+0000","last_became_peered":"2024-05-21T21:51:16.946677+0000","last_unstale":"2024-05-21T21:55:36.197181+0000","last_undegraded":"2024-05-21T21:55:36.197181+0000","last_fullsized":"2024-05-21T21:55:36.197181+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T06:12:56.514284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'7","reported_seq":552,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:56.777139+0000","last_change":"2024-05-21T21:51:12.854873+0000","last_active":"2024-05-21T21:55:56.777139+0000","last_peered":"2024-05-21T21:55:56.777139+0000","last_clean":"2024-05-21T21:55:56.777139+0000","last_became_active":"2024-05-21T21:51:12.854393+0000","last_became_peered":"2024-05-21T21:51:12.854393+0000","last_unstale":"2024-05-21T21:55:56.777139+0000","last_undegraded":"2024-05-21T21:55:56.777139+0000","last_fullsized":"2024-05-21T21:55:56.777139+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:58:52.411164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":516,"num_read_kb":516,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"82'10","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.199587+0000","last_change":"2024-05-21T21:51:16.951094+0000","last_active":"2024-05-21T21:55:36.199587+0000","last_peered":"2024-05-21T21:55:36.199587+0000","last_clean":"2024-05-21T21:55:36.199587+0000","last_became_active":"2024-05-21T21:51:16.950724+0000","last_became_peered":"2024-05-21T21:51:16.950724+0000","last_unstale":"2024-05-21T21:55:36.199587+0000","last_undegraded":"2024-05-21T21:55:36.199587+0000","last_fullsized":"2024-05-21T21:55:36.199587+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:54:02.835426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'4","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.550904+0000","last_change":"2024-05-21T21:51:12.850427+0000","last_active":"2024-05-21T21:55:36.550904+0000","last_peered":"2024-05-21T21:55:36.550904+0000","last_clean":"2024-05-21T21:55:36.550904+0000","last_became_active":"2024-05-21T21:51:12.850096+0000","last_became_peered":"2024-05-21T21:51:12.850096+0000","last_unstale":"2024-05-21T21:55:36.550904+0000","last_undegraded":"2024-05-21T21:55:36.550904+0000","last_fullsized":"2024-05-21T21:55:36.550904+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:43:22.611043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.554216+0000","last_change":"2024-05-21T21:51:16.942330+0000","last_active":"2024-05-21T21:55:36.554216+0000","last_peered":"2024-05-21T21:55:36.554216+0000","last_clean":"2024-05-21T21:55:36.554216+0000","last_became_active":"2024-05-21T21:51:16.941718+0000","last_became_peered":"2024-05-21T21:51:16.941718+0000","last_unstale":"2024-05-21T21:55:36.554216+0000","last_undegraded":"2024-05-21T21:55:36.554216+0000","last_fullsized":"2024-05-21T21:55:36.554216+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:54:17.176411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'5","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.352907+0000","last_change":"2024-05-21T21:51:12.941100+0000","last_active":"2024-05-21T21:55:36.352907+0000","last_peered":"2024-05-21T21:55:36.352907+0000","last_clean":"2024-05-21T21:55:36.352907+0000","last_became_active":"2024-05-21T21:51:12.940597+0000","last_became_peered":"2024-05-21T21:51:12.940597+0000","last_unstale":"2024-05-21T21:55:36.352907+0000","last_undegraded":"2024-05-21T21:55:36.352907+0000","last_fullsized":"2024-05-21T21:55:36.352907+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:30:58.457108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"82'11","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.199324+0000","last_change":"2024-05-21T21:51:16.921973+0000","last_active":"2024-05-21T21:55:36.199324+0000","last_peered":"2024-05-21T21:55:36.199324+0000","last_clean":"2024-05-21T21:55:36.199324+0000","last_became_active":"2024-05-21T21:51:16.921597+0000","last_became_peered":"2024-05-21T21:51:16.921597+0000","last_unstale":"2024-05-21T21:55:36.199324+0000","last_undegraded":"2024-05-21T21:55:36.199324+0000","last_fullsized":"2024-05-21T21:55:36.199324+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:25:32.395940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'41","reported_seq":602,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:56.777926+0000","last_change":"2024-05-21T21:51:12.850312+0000","last_active":"2024-05-21T21:55:56.777926+0000","last_peered":"2024-05-21T21:55:56.777926+0000","last_clean":"2024-05-21T21:55:56.777926+0000","last_became_active":"2024-05-21T21:51:12.849837+0000","last_became_peered":"2024-05-21T21:51:12.849837+0000","last_unstale":"2024-05-21T21:55:56.777926+0000","last_undegraded":"2024-05-21T21:55:56.777926+0000","last_fullsized":"2024-05-21T21:55:56.777926+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T07:09:13.339554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67614,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":531,"num_read_kb":332,"num_write":43,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"82'6","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.556542+0000","last_change":"2024-05-21T21:51:16.926625+0000","last_active":"2024-05-21T21:55:36.556542+0000","last_peered":"2024-05-21T21:55:36.556542+0000","last_clean":"2024-05-21T21:55:36.556542+0000","last_became_active":"2024-05-21T21:51:16.925889+0000","last_became_peered":"2024-05-21T21:51:16.925889+0000","last_unstale":"2024-05-21T21:55:36.556542+0000","last_undegraded":"2024-05-21T21:55:36.556542+0000","last_fullsized":"2024-05-21T21:55:36.556542+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:35:23.464747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'9","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.201314+0000","last_change":"2024-05-21T21:51:12.859772+0000","last_active":"2024-05-21T21:55:36.201314+0000","last_peered":"2024-05-21T21:55:36.201314+0000","last_clean":"2024-05-21T21:55:36.201314+0000","last_became_active":"2024-05-21T21:51:12.859290+0000","last_became_peered":"2024-05-21T21:51:12.859290+0000","last_unstale":"2024-05-21T21:55:36.201314+0000","last_undegraded":"2024-05-21T21:55:36.201314+0000","last_fullsized":"2024-05-21T21:55:36.201314+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T05:28:29.953381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.555713+0000","last_change":"2024-05-21T21:51:16.925803+0000","last_active":"2024-05-21T21:55:36.555713+0000","last_peered":"2024-05-21T21:55:36.555713+0000","last_clean":"2024-05-21T21:55:36.555713+0000","last_became_active":"2024-05-21T21:51:16.925111+0000","last_became_peered":"2024-05-21T21:51:16.925111+0000","last_unstale":"2024-05-21T21:55:36.555713+0000","last_undegraded":"2024-05-21T21:55:36.555713+0000","last_fullsized":"2024-05-21T21:55:36.555713+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T06:22:11.353379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'5","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.352741+0000","last_change":"2024-05-21T21:51:12.859593+0000","last_active":"2024-05-21T21:55:36.352741+0000","last_peered":"2024-05-21T21:55:36.352741+0000","last_clean":"2024-05-21T21:55:36.352741+0000","last_became_active":"2024-05-21T21:51:12.859197+0000","last_became_peered":"2024-05-21T21:51:12.859197+0000","last_unstale":"2024-05-21T21:55:36.352741+0000","last_undegraded":"2024-05-21T21:55:36.352741+0000","last_fullsized":"2024-05-21T21:55:36.352741+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:13:12.311496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"82'15","reported_seq":44,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.551024+0000","last_change":"2024-05-21T21:52:10.879028+0000","last_active":"2024-05-21T21:55:36.551024+0000","last_peered":"2024-05-21T21:55:36.551024+0000","last_clean":"2024-05-21T21:55:36.551024+0000","last_became_active":"2024-05-21T21:52:10.878664+0000","last_became_peered":"2024-05-21T21:52:10.878664+0000","last_unstale":"2024-05-21T21:55:36.551024+0000","last_undegraded":"2024-05-21T21:55:36.551024+0000","last_fullsized":"2024-05-21T21:55:36.551024+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:05:53.953546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'7","reported_seq":299,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:55.906932+0000","last_change":"2024-05-21T21:51:12.852169+0000","last_active":"2024-05-21T21:55:55.906932+0000","last_peered":"2024-05-21T21:55:55.906932+0000","last_clean":"2024-05-21T21:55:55.906932+0000","last_became_active":"2024-05-21T21:51:12.851782+0000","last_became_peered":"2024-05-21T21:51:12.851782+0000","last_unstale":"2024-05-21T21:55:55.906932+0000","last_undegraded":"2024-05-21T21:55:55.906932+0000","last_fullsized":"2024-05-21T21:55:55.906932+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:46:09.692158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":263,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"82'6","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.550587+0000","last_change":"2024-05-21T21:51:16.929264+0000","last_active":"2024-05-21T21:55:36.550587+0000","last_peered":"2024-05-21T21:55:36.550587+0000","last_clean":"2024-05-21T21:55:36.550587+0000","last_became_active":"2024-05-21T21:51:16.928900+0000","last_became_peered":"2024-05-21T21:51:16.928900+0000","last_unstale":"2024-05-21T21:55:36.550587+0000","last_undegraded":"2024-05-21T21:55:36.550587+0000","last_fullsized":"2024-05-21T21:55:36.550587+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:42:28.040939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'17","reported_seq":72,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.556306+0000","last_change":"2024-05-21T21:52:10.984451+0000","last_active":"2024-05-21T21:55:36.556306+0000","last_peered":"2024-05-21T21:55:36.556306+0000","last_clean":"2024-05-21T21:55:36.556306+0000","last_became_active":"2024-05-21T21:52:10.974050+0000","last_became_peered":"2024-05-21T21:52:10.974050+0000","last_unstale":"2024-05-21T21:55:36.556306+0000","last_undegraded":"2024-05-21T21:55:36.556306+0000","last_fullsized":"2024-05-21T21:55:36.556306+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T07:08:18.307022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"82'6","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.352888+0000","last_change":"2024-05-21T21:51:16.935464+0000","last_active":"2024-05-21T21:55:36.352888+0000","last_peered":"2024-05-21T21:55:36.352888+0000","last_clean":"2024-05-21T21:55:36.352888+0000","last_became_active":"2024-05-21T21:51:16.935091+0000","last_became_peered":"2024-05-21T21:51:16.935091+0000","last_unstale":"2024-05-21T21:55:36.352888+0000","last_undegraded":"2024-05-21T21:55:36.352888+0000","last_fullsized":"2024-05-21T21:55:36.352888+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T06:59:16.330649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.556757+0000","last_change":"2024-05-21T21:51:12.853327+0000","last_active":"2024-05-21T21:55:36.556757+0000","last_peered":"2024-05-21T21:55:36.556757+0000","last_clean":"2024-05-21T21:55:36.556757+0000","last_became_active":"2024-05-21T21:51:12.852946+0000","last_became_peered":"2024-05-21T21:51:12.852946+0000","last_unstale":"2024-05-21T21:55:36.556757+0000","last_undegraded":"2024-05-21T21:55:36.556757+0000","last_fullsized":"2024-05-21T21:55:36.556757+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:00:34.256858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"82'5","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.341370+0000","last_change":"2024-05-21T21:51:16.929341+0000","last_active":"2024-05-21T21:55:36.341370+0000","last_peered":"2024-05-21T21:55:36.341370+0000","last_clean":"2024-05-21T21:55:36.341370+0000","last_became_active":"2024-05-21T21:51:16.929008+0000","last_became_peered":"2024-05-21T21:51:16.929008+0000","last_unstale":"2024-05-21T21:55:36.341370+0000","last_undegraded":"2024-05-21T21:55:36.341370+0000","last_fullsized":"2024-05-21T21:55:36.341370+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:26:26.297136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"80'1","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.197640+0000","last_change":"2024-05-21T21:51:12.945433+0000","last_active":"2024-05-21T21:55:36.197640+0000","last_peered":"2024-05-21T21:55:36.197640+0000","last_clean":"2024-05-21T21:55:36.197640+0000","last_became_active":"2024-05-21T21:51:12.944463+0000","last_became_peered":"2024-05-21T21:51:12.944463+0000","last_unstale":"2024-05-21T21:55:36.197640+0000","last_undegraded":"2024-05-21T21:55:36.197640+0000","last_fullsized":"2024-05-21T21:55:36.197640+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:29:52.014583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'5","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.356192+0000","last_change":"2024-05-21T21:52:10.883554+0000","last_active":"2024-05-21T21:55:36.356192+0000","last_peered":"2024-05-21T21:55:36.356192+0000","last_clean":"2024-05-21T21:55:36.356192+0000","last_became_active":"2024-05-21T21:52:10.877789+0000","last_became_peered":"2024-05-21T21:52:10.877789+0000","last_unstale":"2024-05-21T21:55:36.356192+0000","last_undegraded":"2024-05-21T21:55:36.356192+0000","last_fullsized":"2024-05-21T21:55:36.356192+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:52:37.623551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"83'6","reported_seq":296,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:55.929924+0000","last_change":"2024-05-21T21:51:12.852883+0000","last_active":"2024-05-21T21:55:55.929924+0000","last_peered":"2024-05-21T21:55:55.929924+0000","last_clean":"2024-05-21T21:55:55.929924+0000","last_became_active":"2024-05-21T21:51:12.852549+0000","last_became_peered":"2024-05-21T21:51:12.852549+0000","last_unstale":"2024-05-21T21:55:55.929924+0000","last_undegraded":"2024-05-21T21:55:55.929924+0000","last_fullsized":"2024-05-21T21:55:55.929924+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:05:53.002286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":260,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"82'7","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.356089+0000","last_change":"2024-05-21T21:51:16.944425+0000","last_active":"2024-05-21T21:55:36.356089+0000","last_peered":"2024-05-21T21:55:36.356089+0000","last_clean":"2024-05-21T21:55:36.356089+0000","last_became_active":"2024-05-21T21:51:16.944033+0000","last_became_peered":"2024-05-21T21:51:16.944033+0000","last_unstale":"2024-05-21T21:55:36.356089+0000","last_undegraded":"2024-05-21T21:55:36.356089+0000","last_fullsized":"2024-05-21T21:55:36.356089+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T07:16:26.802329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"82'8","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.352742+0000","last_change":"2024-05-21T21:51:16.933264+0000","last_active":"2024-05-21T21:55:36.352742+0000","last_peered":"2024-05-21T21:55:36.352742+0000","last_clean":"2024-05-21T21:55:36.352742+0000","last_became_active":"2024-05-21T21:51:16.932911+0000","last_became_peered":"2024-05-21T21:51:16.932911+0000","last_unstale":"2024-05-21T21:55:36.352742+0000","last_undegraded":"2024-05-21T21:55:36.352742+0000","last_fullsized":"2024-05-21T21:55:36.352742+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:25:54.006367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"82'6","reported_seq":295,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:55.906271+0000","last_change":"2024-05-21T21:51:12.942900+0000","last_active":"2024-05-21T21:55:55.906271+0000","last_peered":"2024-05-21T21:55:55.906271+0000","last_clean":"2024-05-21T21:55:55.906271+0000","last_became_active":"2024-05-21T21:51:12.941038+0000","last_became_peered":"2024-05-21T21:51:12.941038+0000","last_unstale":"2024-05-21T21:55:55.906271+0000","last_undegraded":"2024-05-21T21:55:55.906271+0000","last_fullsized":"2024-05-21T21:55:55.906271+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:38:43.240011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":259,"num_read_kb":259,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"82'8","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.342906+0000","last_change":"2024-05-21T21:51:16.933391+0000","last_active":"2024-05-21T21:55:36.342906+0000","last_peered":"2024-05-21T21:55:36.342906+0000","last_clean":"2024-05-21T21:55:36.342906+0000","last_became_active":"2024-05-21T21:51:16.932320+0000","last_became_peered":"2024-05-21T21:51:16.932320+0000","last_unstale":"2024-05-21T21:55:36.342906+0000","last_undegraded":"2024-05-21T21:55:36.342906+0000","last_fullsized":"2024-05-21T21:55:36.342906+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T09:06:59.126748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'6","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.723215+0000","last_change":"2024-05-21T21:51:12.856342+0000","last_active":"2024-05-21T21:55:36.723215+0000","last_peered":"2024-05-21T21:55:36.723215+0000","last_clean":"2024-05-21T21:55:36.723215+0000","last_became_active":"2024-05-21T21:51:12.855952+0000","last_became_peered":"2024-05-21T21:51:12.855952+0000","last_unstale":"2024-05-21T21:55:36.723215+0000","last_undegraded":"2024-05-21T21:55:36.723215+0000","last_fullsized":"2024-05-21T21:55:36.723215+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:34:40.734034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"83'99","reported_seq":131,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:38.358271+0000","last_change":"2024-05-21T21:49:42.797187+0000","last_active":"2024-05-21T21:55:38.358271+0000","last_peered":"2024-05-21T21:55:38.358271+0000","last_clean":"2024-05-21T21:55:38.358271+0000","last_became_active":"2024-05-21T21:49:42.791291+0000","last_became_peered":"2024-05-21T21:49:42.791291+0000","last_unstale":"2024-05-21T21:55:38.358271+0000","last_undegraded":"2024-05-21T21:55:38.358271+0000","last_fullsized":"2024-05-21T21:55:38.358271+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:45:55.552293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:45:55.552293+0000","last_clean_scrub_stamp":"2024-05-21T21:45:55.552293+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T07:24:18.302523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":144,"num_read_kb":248,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"82'5","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.555133+0000","last_change":"2024-05-21T21:51:16.926446+0000","last_active":"2024-05-21T21:55:36.555133+0000","last_peered":"2024-05-21T21:55:36.555133+0000","last_clean":"2024-05-21T21:55:36.555133+0000","last_became_active":"2024-05-21T21:51:16.926070+0000","last_became_peered":"2024-05-21T21:51:16.926070+0000","last_unstale":"2024-05-21T21:55:36.555133+0000","last_undegraded":"2024-05-21T21:55:36.555133+0000","last_fullsized":"2024-05-21T21:55:36.555133+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:44:27.366393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'4","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.550735+0000","last_change":"2024-05-21T21:51:12.854371+0000","last_active":"2024-05-21T21:55:36.550735+0000","last_peered":"2024-05-21T21:55:36.550735+0000","last_clean":"2024-05-21T21:55:36.550735+0000","last_became_active":"2024-05-21T21:51:12.853847+0000","last_became_peered":"2024-05-21T21:51:12.853847+0000","last_unstale":"2024-05-21T21:55:36.550735+0000","last_undegraded":"2024-05-21T21:55:36.550735+0000","last_fullsized":"2024-05-21T21:55:36.550735+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:14:10.549901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"80'4","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.556551+0000","last_change":"2024-05-21T21:51:12.941227+0000","last_active":"2024-05-21T21:55:36.556551+0000","last_peered":"2024-05-21T21:55:36.556551+0000","last_clean":"2024-05-21T21:55:36.556551+0000","last_became_active":"2024-05-21T21:51:12.940702+0000","last_became_peered":"2024-05-21T21:51:12.940702+0000","last_unstale":"2024-05-21T21:55:36.556551+0000","last_undegraded":"2024-05-21T21:55:36.556551+0000","last_fullsized":"2024-05-21T21:55:36.556551+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:40:34.124921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"82'9","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.556301+0000","last_change":"2024-05-21T21:51:16.945627+0000","last_active":"2024-05-21T21:55:36.556301+0000","last_peered":"2024-05-21T21:55:36.556301+0000","last_clean":"2024-05-21T21:55:36.556301+0000","last_became_active":"2024-05-21T21:51:16.945297+0000","last_became_peered":"2024-05-21T21:51:16.945297+0000","last_unstale":"2024-05-21T21:55:36.556301+0000","last_undegraded":"2024-05-21T21:55:36.556301+0000","last_fullsized":"2024-05-21T21:55:36.556301+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:54:38.898774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'104","reported_seq":242,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:54.029254+0000","last_change":"2024-05-21T21:52:10.888076+0000","last_active":"2024-05-21T21:55:54.029254+0000","last_peered":"2024-05-21T21:55:54.029254+0000","last_clean":"2024-05-21T21:55:54.029254+0000","last_became_active":"2024-05-21T21:52:10.879368+0000","last_became_peered":"2024-05-21T21:52:10.879368+0000","last_unstale":"2024-05-21T21:55:54.029254+0000","last_undegraded":"2024-05-21T21:55:54.029254+0000","last_fullsized":"2024-05-21T21:55:54.029254+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:46:43.190411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":312,"num_read_kb":277,"num_write":190,"num_write_kb":1286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"82'5","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.344028+0000","last_change":"2024-05-21T21:51:16.929807+0000","last_active":"2024-05-21T21:55:36.344028+0000","last_peered":"2024-05-21T21:55:36.344028+0000","last_clean":"2024-05-21T21:55:36.344028+0000","last_became_active":"2024-05-21T21:51:16.928859+0000","last_became_peered":"2024-05-21T21:51:16.928859+0000","last_unstale":"2024-05-21T21:55:36.344028+0000","last_undegraded":"2024-05-21T21:55:36.344028+0000","last_fullsized":"2024-05-21T21:55:36.344028+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:41:32.830892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'7","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.723447+0000","last_change":"2024-05-21T21:51:12.850843+0000","last_active":"2024-05-21T21:55:36.723447+0000","last_peered":"2024-05-21T21:55:36.723447+0000","last_clean":"2024-05-21T21:55:36.723447+0000","last_became_active":"2024-05-21T21:51:12.850516+0000","last_became_peered":"2024-05-21T21:51:12.850516+0000","last_unstale":"2024-05-21T21:55:36.723447+0000","last_undegraded":"2024-05-21T21:55:36.723447+0000","last_fullsized":"2024-05-21T21:55:36.723447+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:37:05.191626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"82'7","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.553781+0000","last_change":"2024-05-21T21:51:16.949017+0000","last_active":"2024-05-21T21:55:36.553781+0000","last_peered":"2024-05-21T21:55:36.553781+0000","last_clean":"2024-05-21T21:55:36.553781+0000","last_became_active":"2024-05-21T21:51:16.948691+0000","last_became_peered":"2024-05-21T21:51:16.948691+0000","last_unstale":"2024-05-21T21:55:36.553781+0000","last_undegraded":"2024-05-21T21:55:36.553781+0000","last_fullsized":"2024-05-21T21:55:36.553781+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:32:06.606581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.199191+0000","last_change":"2024-05-21T21:51:12.939203+0000","last_active":"2024-05-21T21:55:36.199191+0000","last_peered":"2024-05-21T21:55:36.199191+0000","last_clean":"2024-05-21T21:55:36.199191+0000","last_became_active":"2024-05-21T21:51:12.938936+0000","last_became_peered":"2024-05-21T21:51:12.938936+0000","last_unstale":"2024-05-21T21:55:36.199191+0000","last_undegraded":"2024-05-21T21:55:36.199191+0000","last_fullsized":"2024-05-21T21:55:36.199191+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:55:50.336362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"82'2","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.556315+0000","last_change":"2024-05-21T21:51:16.926216+0000","last_active":"2024-05-21T21:55:36.556315+0000","last_peered":"2024-05-21T21:55:36.556315+0000","last_clean":"2024-05-21T21:55:36.556315+0000","last_became_active":"2024-05-21T21:51:16.925834+0000","last_became_peered":"2024-05-21T21:51:16.925834+0000","last_unstale":"2024-05-21T21:55:36.556315+0000","last_undegraded":"2024-05-21T21:55:36.556315+0000","last_fullsized":"2024-05-21T21:55:36.556315+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:13:30.192908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'7","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.342741+0000","last_change":"2024-05-21T21:51:12.855306+0000","last_active":"2024-05-21T21:55:36.342741+0000","last_peered":"2024-05-21T21:55:36.342741+0000","last_clean":"2024-05-21T21:55:36.342741+0000","last_became_active":"2024-05-21T21:51:12.854722+0000","last_became_peered":"2024-05-21T21:51:12.854722+0000","last_unstale":"2024-05-21T21:55:36.342741+0000","last_undegraded":"2024-05-21T21:55:36.342741+0000","last_fullsized":"2024-05-21T21:55:36.342741+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:39:41.583219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"82'9","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.341026+0000","last_change":"2024-05-21T21:51:16.944758+0000","last_active":"2024-05-21T21:55:36.341026+0000","last_peered":"2024-05-21T21:55:36.341026+0000","last_clean":"2024-05-21T21:55:36.341026+0000","last_became_active":"2024-05-21T21:51:16.944393+0000","last_became_peered":"2024-05-21T21:51:16.944393+0000","last_unstale":"2024-05-21T21:55:36.341026+0000","last_undegraded":"2024-05-21T21:55:36.341026+0000","last_fullsized":"2024-05-21T21:55:36.341026+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:01:23.048355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'4","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:37.227713+0000","last_change":"2024-05-21T21:51:12.860378+0000","last_active":"2024-05-21T21:55:37.227713+0000","last_peered":"2024-05-21T21:55:37.227713+0000","last_clean":"2024-05-21T21:55:37.227713+0000","last_became_active":"2024-05-21T21:51:12.859978+0000","last_became_peered":"2024-05-21T21:51:12.859978+0000","last_unstale":"2024-05-21T21:55:37.227713+0000","last_undegraded":"2024-05-21T21:55:37.227713+0000","last_fullsized":"2024-05-21T21:55:37.227713+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T07:52:26.902232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"82'8","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.343395+0000","last_change":"2024-05-21T21:51:16.943194+0000","last_active":"2024-05-21T21:55:36.343395+0000","last_peered":"2024-05-21T21:55:36.343395+0000","last_clean":"2024-05-21T21:55:36.343395+0000","last_became_active":"2024-05-21T21:51:16.942884+0000","last_became_peered":"2024-05-21T21:51:16.942884+0000","last_unstale":"2024-05-21T21:55:36.343395+0000","last_undegraded":"2024-05-21T21:55:36.343395+0000","last_fullsized":"2024-05-21T21:55:36.343395+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T05:58:17.436995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.556514+0000","last_change":"2024-05-21T21:51:12.853390+0000","last_active":"2024-05-21T21:55:36.556514+0000","last_peered":"2024-05-21T21:55:36.556514+0000","last_clean":"2024-05-21T21:55:36.556514+0000","last_became_active":"2024-05-21T21:51:12.852990+0000","last_became_peered":"2024-05-21T21:51:12.852990+0000","last_unstale":"2024-05-21T21:55:36.556514+0000","last_undegraded":"2024-05-21T21:55:36.556514+0000","last_fullsized":"2024-05-21T21:55:36.556514+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:32:16.392609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"82'5","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.550848+0000","last_change":"2024-05-21T21:51:16.933430+0000","last_active":"2024-05-21T21:55:36.550848+0000","last_peered":"2024-05-21T21:55:36.550848+0000","last_clean":"2024-05-21T21:55:36.550848+0000","last_became_active":"2024-05-21T21:51:16.933053+0000","last_became_peered":"2024-05-21T21:51:16.933053+0000","last_unstale":"2024-05-21T21:55:36.550848+0000","last_undegraded":"2024-05-21T21:55:36.550848+0000","last_fullsized":"2024-05-21T21:55:36.550848+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T06:14:57.468727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'7","reported_seq":298,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:55.776030+0000","last_change":"2024-05-21T21:51:12.946113+0000","last_active":"2024-05-21T21:55:55.776030+0000","last_peered":"2024-05-21T21:55:55.776030+0000","last_clean":"2024-05-21T21:55:55.776030+0000","last_became_active":"2024-05-21T21:51:12.945650+0000","last_became_peered":"2024-05-21T21:51:12.945650+0000","last_unstale":"2024-05-21T21:55:55.776030+0000","last_undegraded":"2024-05-21T21:55:55.776030+0000","last_fullsized":"2024-05-21T21:55:55.776030+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T09:40:18.209400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1952,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":261,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"82'7","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.550939+0000","last_change":"2024-05-21T21:51:16.932632+0000","last_active":"2024-05-21T21:55:36.550939+0000","last_peered":"2024-05-21T21:55:36.550939+0000","last_clean":"2024-05-21T21:55:36.550939+0000","last_became_active":"2024-05-21T21:51:16.932256+0000","last_became_peered":"2024-05-21T21:51:16.932256+0000","last_unstale":"2024-05-21T21:55:36.550939+0000","last_undegraded":"2024-05-21T21:55:36.550939+0000","last_fullsized":"2024-05-21T21:55:36.550939+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:04:48.146485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'67","reported_seq":128,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:53.558384+0000","last_change":"2024-05-21T21:51:12.860602+0000","last_active":"2024-05-21T21:55:53.558384+0000","last_peered":"2024-05-21T21:55:53.558384+0000","last_clean":"2024-05-21T21:55:53.558384+0000","last_became_active":"2024-05-21T21:51:12.860228+0000","last_became_peered":"2024-05-21T21:51:12.860228+0000","last_unstale":"2024-05-21T21:55:53.558384+0000","last_undegraded":"2024-05-21T21:55:53.558384+0000","last_fullsized":"2024-05-21T21:55:53.558384+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:33:39.230872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":33,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'7","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.551083+0000","last_change":"2024-05-21T21:51:16.929616+0000","last_active":"2024-05-21T21:55:36.551083+0000","last_peered":"2024-05-21T21:55:36.551083+0000","last_clean":"2024-05-21T21:55:36.551083+0000","last_became_active":"2024-05-21T21:51:16.928569+0000","last_became_peered":"2024-05-21T21:51:16.928569+0000","last_unstale":"2024-05-21T21:55:36.551083+0000","last_undegraded":"2024-05-21T21:55:36.551083+0000","last_fullsized":"2024-05-21T21:55:36.551083+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:15:41.790344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'5","reported_seq":47,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.555560+0000","last_change":"2024-05-21T21:52:10.881114+0000","last_active":"2024-05-21T21:55:36.555560+0000","last_peered":"2024-05-21T21:55:36.555560+0000","last_clean":"2024-05-21T21:55:36.555560+0000","last_became_active":"2024-05-21T21:52:10.872022+0000","last_became_peered":"2024-05-21T21:52:10.872022+0000","last_unstale":"2024-05-21T21:55:36.555560+0000","last_undegraded":"2024-05-21T21:55:36.555560+0000","last_fullsized":"2024-05-21T21:55:36.555560+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T06:05:51.419156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"82'9","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.550977+0000","last_change":"2024-05-21T21:51:16.926496+0000","last_active":"2024-05-21T21:55:36.550977+0000","last_peered":"2024-05-21T21:55:36.550977+0000","last_clean":"2024-05-21T21:55:36.550977+0000","last_became_active":"2024-05-21T21:51:16.926112+0000","last_became_peered":"2024-05-21T21:51:16.926112+0000","last_unstale":"2024-05-21T21:55:36.550977+0000","last_undegraded":"2024-05-21T21:55:36.550977+0000","last_fullsized":"2024-05-21T21:55:36.550977+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:26:08.254421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"82'9","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.341047+0000","last_change":"2024-05-21T21:51:16.930522+0000","last_active":"2024-05-21T21:55:36.341047+0000","last_peered":"2024-05-21T21:55:36.341047+0000","last_clean":"2024-05-21T21:55:36.341047+0000","last_became_active":"2024-05-21T21:51:16.929853+0000","last_became_peered":"2024-05-21T21:51:16.929853+0000","last_unstale":"2024-05-21T21:55:36.341047+0000","last_undegraded":"2024-05-21T21:55:36.341047+0000","last_fullsized":"2024-05-21T21:55:36.341047+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:10:57.940009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"82'12","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.201162+0000","last_change":"2024-05-21T21:51:16.931751+0000","last_active":"2024-05-21T21:55:36.201162+0000","last_peered":"2024-05-21T21:55:36.201162+0000","last_clean":"2024-05-21T21:55:36.201162+0000","last_became_active":"2024-05-21T21:51:16.931353+0000","last_became_peered":"2024-05-21T21:51:16.931353+0000","last_unstale":"2024-05-21T21:55:36.201162+0000","last_undegraded":"2024-05-21T21:55:36.201162+0000","last_fullsized":"2024-05-21T21:55:36.201162+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:33:45.297850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"82'8","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.201127+0000","last_change":"2024-05-21T21:51:16.940096+0000","last_active":"2024-05-21T21:55:36.201127+0000","last_peered":"2024-05-21T21:55:36.201127+0000","last_clean":"2024-05-21T21:55:36.201127+0000","last_became_active":"2024-05-21T21:51:16.939740+0000","last_became_peered":"2024-05-21T21:51:16.939740+0000","last_unstale":"2024-05-21T21:55:36.201127+0000","last_undegraded":"2024-05-21T21:55:36.201127+0000","last_fullsized":"2024-05-21T21:55:36.201127+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:59:52.569526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"82'7","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.552155+0000","last_change":"2024-05-21T21:51:16.946488+0000","last_active":"2024-05-21T21:55:36.552155+0000","last_peered":"2024-05-21T21:55:36.552155+0000","last_clean":"2024-05-21T21:55:36.552155+0000","last_became_active":"2024-05-21T21:51:16.945957+0000","last_became_peered":"2024-05-21T21:51:16.945957+0000","last_unstale":"2024-05-21T21:55:36.552155+0000","last_undegraded":"2024-05-21T21:55:36.552155+0000","last_fullsized":"2024-05-21T21:55:36.552155+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:40:36.708344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"82'4","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.556177+0000","last_change":"2024-05-21T21:51:16.949349+0000","last_active":"2024-05-21T21:55:36.556177+0000","last_peered":"2024-05-21T21:55:36.556177+0000","last_clean":"2024-05-21T21:55:36.556177+0000","last_became_active":"2024-05-21T21:51:16.946986+0000","last_became_peered":"2024-05-21T21:51:16.946986+0000","last_unstale":"2024-05-21T21:55:36.556177+0000","last_undegraded":"2024-05-21T21:55:36.556177+0000","last_fullsized":"2024-05-21T21:55:36.556177+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:01:14.877110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"82'6","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.341365+0000","last_change":"2024-05-21T21:51:16.929172+0000","last_active":"2024-05-21T21:55:36.341365+0000","last_peered":"2024-05-21T21:55:36.341365+0000","last_clean":"2024-05-21T21:55:36.341365+0000","last_became_active":"2024-05-21T21:51:16.928823+0000","last_became_peered":"2024-05-21T21:51:16.928823+0000","last_unstale":"2024-05-21T21:55:36.341365+0000","last_undegraded":"2024-05-21T21:55:36.341365+0000","last_fullsized":"2024-05-21T21:55:36.341365+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:09:13.719272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"82'15","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.201520+0000","last_change":"2024-05-21T21:51:16.930332+0000","last_active":"2024-05-21T21:55:36.201520+0000","last_peered":"2024-05-21T21:55:36.201520+0000","last_clean":"2024-05-21T21:55:36.201520+0000","last_became_active":"2024-05-21T21:51:16.930024+0000","last_became_peered":"2024-05-21T21:51:16.930024+0000","last_unstale":"2024-05-21T21:55:36.201520+0000","last_undegraded":"2024-05-21T21:55:36.201520+0000","last_fullsized":"2024-05-21T21:55:36.201520+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:47:41.272725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"82'2","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.343903+0000","last_change":"2024-05-21T21:51:16.930102+0000","last_active":"2024-05-21T21:55:36.343903+0000","last_peered":"2024-05-21T21:55:36.343903+0000","last_clean":"2024-05-21T21:55:36.343903+0000","last_became_active":"2024-05-21T21:51:16.929410+0000","last_became_peered":"2024-05-21T21:51:16.929410+0000","last_unstale":"2024-05-21T21:55:36.343903+0000","last_undegraded":"2024-05-21T21:55:36.343903+0000","last_fullsized":"2024-05-21T21:55:36.343903+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:11:46.296288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"82'6","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.556235+0000","last_change":"2024-05-21T21:52:10.873424+0000","last_active":"2024-05-21T21:55:36.556235+0000","last_peered":"2024-05-21T21:55:36.556235+0000","last_clean":"2024-05-21T21:55:36.556235+0000","last_became_active":"2024-05-21T21:52:10.873118+0000","last_became_peered":"2024-05-21T21:52:10.873118+0000","last_unstale":"2024-05-21T21:55:36.556235+0000","last_undegraded":"2024-05-21T21:55:36.556235+0000","last_fullsized":"2024-05-21T21:55:36.556235+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:05:00.731013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,3],"acting":[8,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.29","version":"82'8","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.342675+0000","last_change":"2024-05-21T21:51:16.934442+0000","last_active":"2024-05-21T21:55:36.342675+0000","last_peered":"2024-05-21T21:55:36.342675+0000","last_clean":"2024-05-21T21:55:36.342675+0000","last_became_active":"2024-05-21T21:51:16.934086+0000","last_became_peered":"2024-05-21T21:51:16.934086+0000","last_unstale":"2024-05-21T21:55:36.342675+0000","last_undegraded":"2024-05-21T21:55:36.342675+0000","last_fullsized":"2024-05-21T21:55:36.342675+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:36:44.302532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"82'10","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.199462+0000","last_change":"2024-05-21T21:51:16.924479+0000","last_active":"2024-05-21T21:55:36.199462+0000","last_peered":"2024-05-21T21:55:36.199462+0000","last_clean":"2024-05-21T21:55:36.199462+0000","last_became_active":"2024-05-21T21:51:16.923998+0000","last_became_peered":"2024-05-21T21:51:16.923998+0000","last_unstale":"2024-05-21T21:55:36.199462+0000","last_undegraded":"2024-05-21T21:55:36.199462+0000","last_fullsized":"2024-05-21T21:55:36.199462+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:23:07.931731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"82'11","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.555830+0000","last_change":"2024-05-21T21:51:16.947271+0000","last_active":"2024-05-21T21:55:36.555830+0000","last_peered":"2024-05-21T21:55:36.555830+0000","last_clean":"2024-05-21T21:55:36.555830+0000","last_became_active":"2024-05-21T21:51:16.946912+0000","last_became_peered":"2024-05-21T21:51:16.946912+0000","last_unstale":"2024-05-21T21:55:36.555830+0000","last_undegraded":"2024-05-21T21:55:36.555830+0000","last_fullsized":"2024-05-21T21:55:36.555830+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:51:29.646915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"82'8","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.341091+0000","last_change":"2024-05-21T21:51:16.928852+0000","last_active":"2024-05-21T21:55:36.341091+0000","last_peered":"2024-05-21T21:55:36.341091+0000","last_clean":"2024-05-21T21:55:36.341091+0000","last_became_active":"2024-05-21T21:51:16.928480+0000","last_became_peered":"2024-05-21T21:51:16.928480+0000","last_unstale":"2024-05-21T21:55:36.341091+0000","last_undegraded":"2024-05-21T21:55:36.341091+0000","last_fullsized":"2024-05-21T21:55:36.341091+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T09:12:30.691254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":144,"num_read_kb":248,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":798422,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":2977,"num_read_kb":1710,"num_write":509,"num_write_kb":2386,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":112,"num_keys_recovered":17,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2764800,"data_stored":2395266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":387,"ondisk_log_size":387,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645385,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28296,"kb_used_data":1280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716856,"statfs":{"total":95995035648,"available":95966060544,"internally_reserved":0,"allocated":1310720,"data_stored":836913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841613,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":29048,"kb_used_data":1912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716104,"statfs":{"total":95995035648,"available":95965290496,"internally_reserved":0,"allocated":1957888,"data_stored":1430322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005138,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28324,"kb_used_data":1308,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716828,"statfs":{"total":95995035648,"available":95966031872,"internally_reserved":0,"allocated":1339392,"data_stored":836178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201369,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":937984,"data_stored":443223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397597,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28516,"kb_used_data":1500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716636,"statfs":{"total":95995035648,"available":95965835264,"internally_reserved":0,"allocated":1536000,"data_stored":1037506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561123,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":925696,"data_stored":442953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574975,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":511592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738500,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36264,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708888,"statfs":{"total":95995035648,"available":95957901312,"internally_reserved":0,"allocated":946176,"data_stored":444683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411451,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36320,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708832,"statfs":{"total":95995035648,"available":95957843968,"internally_reserved":0,"allocated":1007616,"data_stored":513212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149811,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37080,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708072,"statfs":{"total":95995035648,"available":95957065728,"internally_reserved":0,"allocated":1662976,"data_stored":1163718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953582,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":925696,"data_stored":442955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757353,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":33132,"kb_used_data":1836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712020,"statfs":{"total":95995035648,"available":95961108480,"internally_reserved":0,"allocated":1880064,"data_stored":1364493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":69869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":71489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":753664,"data_stored":721878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":332285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":398114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":395190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-21T21:55:59.270 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph pg dump --format=json 2024-05-21T21:55:59.635 INFO:tasks.cephfs.filesystem:scrub status for tag:02ccb105-4b0a-4d9a-a4bb-6b2e269f31aa - {'path': '/', 'tag': '02ccb105-4b0a-4d9a-a4bb-6b2e269f31aa', 'options': 'recursive,force'} 2024-05-21T21:55:59.778 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:55:59.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:59 smithi164 ceph-mon[36269]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 76 KiB/s wr, 44 op/s 2024-05-21T21:55:59.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:55:59 smithi164 ceph-mon[36269]: from='client.15210 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T21:55:59.926 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:59 smithi151 ceph-mon[31623]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 76 KiB/s wr, 44 op/s 2024-05-21T21:55:59.926 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:55:59 smithi151 ceph-mon[31623]: from='client.15210 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T21:55:59.985 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:59 smithi184 ceph-mon[36942]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 76 KiB/s wr, 44 op/s 2024-05-21T21:55:59.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:55:59 smithi184 ceph-mon[36942]: from='client.15210 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T21:56:00.203 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:56:00 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:56:00.088+0000 7fbe3a400700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-21T21:56:00.203 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:56:00 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:56:00.088+0000 7fbe3a400700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-21T21:56:00.203 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:56:00 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:56:00.088+0000 7fbe3a400700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-21T21:56:00.204 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:56:00 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:56:00.088+0000 7fbe3a400700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-21T21:56:00.204 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 21:56:00 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T21:56:00.088+0000 7fbe3a400700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-21T21:56:01.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:56:01 smithi151 ceph-mon[31623]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 76 KiB/s wr, 40 op/s 2024-05-21T21:56:01.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:56:01 smithi151 ceph-mon[31623]: mgrmap e26: x(active, since 24s), standbys: z, y 2024-05-21T21:56:01.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:56:01 smithi164 ceph-mon[36269]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 76 KiB/s wr, 40 op/s 2024-05-21T21:56:01.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:56:01 smithi164 ceph-mon[36269]: mgrmap e26: x(active, since 24s), standbys: z, y 2024-05-21T21:56:01.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:01.910+0000 7f94b1000700 1 -- 172.21.15.151:0/2504562826 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f94ac10b760 msgr2=0x7f94ac10bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:01.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:01.910+0000 7f94b1000700 1 --2- 172.21.15.151:0/2504562826 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f94ac10b760 0x7f94ac10bb40 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f949c0098f0 tx=0x7f949c009c00 comp rx=0 tx=0).stop 2024-05-21T21:56:01.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:01.910+0000 7f94b1000700 1 -- 172.21.15.151:0/2504562826 shutdown_connections 2024-05-21T21:56:01.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:01.910+0000 7f94b1000700 1 --2- 172.21.15.151:0/2504562826 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f94ac10b760 0x7f94ac10bb40 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:01.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:01.910+0000 7f94b1000700 1 --2- 172.21.15.151:0/2504562826 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f94ac106120 0x7f94ac112630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:01.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:01.910+0000 7f94b1000700 1 --2- 172.21.15.151:0/2504562826 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f94ac105760 0x7f94ac105be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:01.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:01.910+0000 7f94b1000700 1 -- 172.21.15.151:0/2504562826 >> 172.21.15.151:0/2504562826 conn(0x7f94ac101250 msgr2=0x7f94ac103670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:56:01.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:01.910+0000 7f94b1000700 1 -- 172.21.15.151:0/2504562826 shutdown_connections 2024-05-21T21:56:01.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:01.910+0000 7f94b1000700 1 -- 172.21.15.151:0/2504562826 wait complete. 2024-05-21T21:56:01.913 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:01.911+0000 7f94b1000700 1 Processor -- start 2024-05-21T21:56:01.913 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:01.911+0000 7f94b1000700 1 -- start start 2024-05-21T21:56:01.913 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:01.912+0000 7f94b1000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f94ac105760 0x7f94ac19bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:01.913 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:01.912+0000 7f94b1000700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f94ac106120 0x7f94ac19c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:01.913 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:01.912+0000 7f94b1000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f94ac10b760 0x7f94ac1a0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:01.913 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:01.912+0000 7f94b1000700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f94ac1159b0 con 0x7f94ac105760 2024-05-21T21:56:01.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:01.912+0000 7f94b1000700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f94ac115830 con 0x7f94ac10b760 2024-05-21T21:56:01.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:01.912+0000 7f94b1000700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f94ac115b30 con 0x7f94ac106120 2024-05-21T21:56:01.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:01.912+0000 7f94a9e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f94ac105760 0x7f94ac19bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:01.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:01.912+0000 7f94a9e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f94ac105760 0x7f94ac19bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:60148/0 (socket says 172.21.15.151:60148) 2024-05-21T21:56:01.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:01.912+0000 7f94a9e00700 1 -- 172.21.15.151:0/3261938397 learned_addr learned my addr 172.21.15.151:0/3261938397 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:56:01.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:01.912+0000 7f94a3e00700 1 --2- 172.21.15.151:0/3261938397 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f94ac106120 0x7f94ac19c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:01.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:01.912+0000 7f94aa800700 1 --2- 172.21.15.151:0/3261938397 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f94ac10b760 0x7f94ac1a0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:01.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:01.912+0000 7f94a9e00700 1 -- 172.21.15.151:0/3261938397 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f94ac106120 msgr2=0x7f94ac19c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:01.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:01.912+0000 7f94a9e00700 1 --2- 172.21.15.151:0/3261938397 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f94ac106120 0x7f94ac19c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:01.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:01.912+0000 7f94a9e00700 1 -- 172.21.15.151:0/3261938397 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f94ac10b760 msgr2=0x7f94ac1a0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:01.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:01.912+0000 7f94a9e00700 1 --2- 172.21.15.151:0/3261938397 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f94ac10b760 0x7f94ac1a0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:01.915 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:01.912+0000 7f94a9e00700 1 -- 172.21.15.151:0/3261938397 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94ac1a0ec0 con 0x7f94ac105760 2024-05-21T21:56:01.915 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:01.912+0000 7f94a3e00700 1 --2- 172.21.15.151:0/3261938397 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f94ac106120 0x7f94ac19c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:56:01.915 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:01.912+0000 7f94aa800700 1 --2- 172.21.15.151:0/3261938397 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f94ac10b760 0x7f94ac1a0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:56:01.915 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:01.913+0000 7f94a9e00700 1 --2- 172.21.15.151:0/3261938397 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f94ac105760 0x7f94ac19bf00 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f949c009230 tx=0x7f949c00fa70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:56:01.915 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:01.913+0000 7f94a2a00700 1 -- 172.21.15.151:0/3261938397 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f949c005140 con 0x7f94ac105760 2024-05-21T21:56:01.915 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:01.913+0000 7f94a2a00700 1 -- 172.21.15.151:0/3261938397 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f949c00b560 con 0x7f94ac105760 2024-05-21T21:56:01.915 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:01.913+0000 7f94b1000700 1 -- 172.21.15.151:0/3261938397 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94ac1a1150 con 0x7f94ac105760 2024-05-21T21:56:01.915 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:01.913+0000 7f94a2a00700 1 -- 172.21.15.151:0/3261938397 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f949c003970 con 0x7f94ac105760 2024-05-21T21:56:01.916 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:01.913+0000 7f94b1000700 1 -- 172.21.15.151:0/3261938397 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94ac1a8a30 con 0x7f94ac105760 2024-05-21T21:56:01.916 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:01.914+0000 7f94b1000700 1 -- 172.21.15.151:0/3261938397 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94ac06a230 con 0x7f94ac105760 2024-05-21T21:56:01.916 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:01.917+0000 7f94a2a00700 1 -- 172.21.15.151:0/3261938397 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 26) ==== 149886+0+0 (secure 0 0 0) 0x7f949c00ba50 con 0x7f94ac105760 2024-05-21T21:56:01.918 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:01.918+0000 7f94a2a00700 1 --2- 172.21.15.151:0/3261938397 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f948c0b2b10 0x7f948c0b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:01.918 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:01.918+0000 7f94a2a00700 1 -- 172.21.15.151:0/3261938397 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f949c0e3810 con 0x7f94ac105760 2024-05-21T21:56:01.918 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:01.918+0000 7f94a3e00700 1 --2- 172.21.15.151:0/3261938397 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f948c0b2b10 0x7f948c0b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:01.919 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:01.919+0000 7f94a3e00700 1 --2- 172.21.15.151:0/3261938397 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f948c0b2b10 0x7f948c0b4fd0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f94ac103930 tx=0x7f9494009310 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:56:01.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:01.923+0000 7f94a2a00700 1 -- 172.21.15.151:0/3261938397 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f949c013030 con 0x7f94ac105760 2024-05-21T21:56:01.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:56:01 smithi184 ceph-mon[36942]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 76 KiB/s wr, 40 op/s 2024-05-21T21:56:01.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:56:01 smithi184 ceph-mon[36942]: mgrmap e26: x(active, since 24s), standbys: z, y 2024-05-21T21:56:02.152 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:02.151+0000 7f94b1000700 1 -- 172.21.15.151:0/3261938397 --> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f94ac19d210 con 0x7f948c0b2b10 2024-05-21T21:56:02.161 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:02.161+0000 7f94a2a00700 1 -- 172.21.15.151:0/3261938397 <== mgr.25138 v2:172.21.15.151:6800/958417056 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257820 (secure 0 0 0) 0x7f94ac19d210 con 0x7f948c0b2b10 2024-05-21T21:56:02.167 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T21:56:02.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:02.167+0000 7f94b1000700 1 -- 172.21.15.151:0/3261938397 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f948c0b2b10 msgr2=0x7f948c0b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:02.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:02.167+0000 7f94b1000700 1 --2- 172.21.15.151:0/3261938397 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f948c0b2b10 0x7f948c0b4fd0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f94ac103930 tx=0x7f9494009310 comp rx=0 tx=0).stop 2024-05-21T21:56:02.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:02.168+0000 7f94b1000700 1 -- 172.21.15.151:0/3261938397 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f94ac105760 msgr2=0x7f94ac19bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:02.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:02.168+0000 7f94b1000700 1 --2- 172.21.15.151:0/3261938397 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f94ac105760 0x7f94ac19bf00 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f949c009230 tx=0x7f949c00fa70 comp rx=0 tx=0).stop 2024-05-21T21:56:02.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:02.168+0000 7f94b1000700 1 -- 172.21.15.151:0/3261938397 shutdown_connections 2024-05-21T21:56:02.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:02.168+0000 7f94b1000700 1 --2- 172.21.15.151:0/3261938397 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f948c0b2b10 0x7f948c0b4fd0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:02.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:02.168+0000 7f94b1000700 1 --2- 172.21.15.151:0/3261938397 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f94ac10b760 0x7f94ac1a0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:02.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:02.168+0000 7f94b1000700 1 --2- 172.21.15.151:0/3261938397 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f94ac105760 0x7f94ac19bf00 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:02.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:02.168+0000 7f94b1000700 1 --2- 172.21.15.151:0/3261938397 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f94ac106120 0x7f94ac19c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:02.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:02.168+0000 7f94b1000700 1 -- 172.21.15.151:0/3261938397 >> 172.21.15.151:0/3261938397 conn(0x7f94ac101250 msgr2=0x7f94ac101aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:56:02.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:02.168+0000 7f94b1000700 1 -- 172.21.15.151:0/3261938397 shutdown_connections 2024-05-21T21:56:02.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:02.168+0000 7f94b1000700 1 -- 172.21.15.151:0/3261938397 wait complete. 2024-05-21T21:56:02.170 INFO:teuthology.orchestra.run.smithi151.stderr:dumped all 2024-05-21T21:56:03.021 INFO:teuthology.orchestra.run.smithi151.stdout:{"pg_ready":true,"pg_map":{"version":15,"stamp":"2024-05-21T21:56:00.226466+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1389024,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":3121,"num_read_kb":1958,"num_write":646,"num_write_kb":4538,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":2952069,"num_keys_recovered":17,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":988,"ondisk_log_size":988,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":368760,"kb_used_data":15072,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124573064,"statfs":{"total":1151940427776,"available":1151562817536,"internally_reserved":0,"allocated":15433728,"data_stored":9467748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19140,"internal_metadata":361674044},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":197770,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":356,"num_read_kb":284,"num_write":128,"num_write_kb":917,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004734"},"pg_stats":[{"pgid":"3.3f","version":"82'8","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.341334+0000","last_change":"2024-05-21T21:51:16.945132+0000","last_active":"2024-05-21T21:55:36.341334+0000","last_peered":"2024-05-21T21:55:36.341334+0000","last_clean":"2024-05-21T21:55:36.341334+0000","last_became_active":"2024-05-21T21:51:16.944766+0000","last_became_peered":"2024-05-21T21:51:16.944766+0000","last_unstale":"2024-05-21T21:55:36.341334+0000","last_undegraded":"2024-05-21T21:55:36.341334+0000","last_fullsized":"2024-05-21T21:55:36.341334+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:54:04.185095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"82'4","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.553624+0000","last_change":"2024-05-21T21:51:16.939291+0000","last_active":"2024-05-21T21:55:36.553624+0000","last_peered":"2024-05-21T21:55:36.553624+0000","last_clean":"2024-05-21T21:55:36.553624+0000","last_became_active":"2024-05-21T21:51:16.938730+0000","last_became_peered":"2024-05-21T21:51:16.938730+0000","last_unstale":"2024-05-21T21:55:36.553624+0000","last_undegraded":"2024-05-21T21:55:36.553624+0000","last_fullsized":"2024-05-21T21:55:36.553624+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:05:30.623744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"82'7","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.553252+0000","last_change":"2024-05-21T21:51:16.941565+0000","last_active":"2024-05-21T21:55:36.553252+0000","last_peered":"2024-05-21T21:55:36.553252+0000","last_clean":"2024-05-21T21:55:36.553252+0000","last_became_active":"2024-05-21T21:51:16.941033+0000","last_became_peered":"2024-05-21T21:51:16.941033+0000","last_unstale":"2024-05-21T21:55:36.553252+0000","last_undegraded":"2024-05-21T21:55:36.553252+0000","last_fullsized":"2024-05-21T21:55:36.553252+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:04:55.789479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"82'8","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.723315+0000","last_change":"2024-05-21T21:51:16.928455+0000","last_active":"2024-05-21T21:55:36.723315+0000","last_peered":"2024-05-21T21:55:36.723315+0000","last_clean":"2024-05-21T21:55:36.723315+0000","last_became_active":"2024-05-21T21:51:16.927874+0000","last_became_peered":"2024-05-21T21:51:16.927874+0000","last_unstale":"2024-05-21T21:55:36.723315+0000","last_undegraded":"2024-05-21T21:55:36.723315+0000","last_fullsized":"2024-05-21T21:55:36.723315+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:25:16.483175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"82'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.556362+0000","last_change":"2024-05-21T21:51:16.937773+0000","last_active":"2024-05-21T21:55:36.556362+0000","last_peered":"2024-05-21T21:55:36.556362+0000","last_clean":"2024-05-21T21:55:36.556362+0000","last_became_active":"2024-05-21T21:51:16.937306+0000","last_became_peered":"2024-05-21T21:51:16.937306+0000","last_unstale":"2024-05-21T21:55:36.556362+0000","last_undegraded":"2024-05-21T21:55:36.556362+0000","last_fullsized":"2024-05-21T21:55:36.556362+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:05:03.762390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"82'6","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.356010+0000","last_change":"2024-05-21T21:51:16.948275+0000","last_active":"2024-05-21T21:55:36.356010+0000","last_peered":"2024-05-21T21:55:36.356010+0000","last_clean":"2024-05-21T21:55:36.356010+0000","last_became_active":"2024-05-21T21:51:16.947890+0000","last_became_peered":"2024-05-21T21:51:16.947890+0000","last_unstale":"2024-05-21T21:55:36.356010+0000","last_undegraded":"2024-05-21T21:55:36.356010+0000","last_fullsized":"2024-05-21T21:55:36.356010+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:08:38.572177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"82'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.550575+0000","last_change":"2024-05-21T21:51:16.946821+0000","last_active":"2024-05-21T21:55:36.550575+0000","last_peered":"2024-05-21T21:55:36.550575+0000","last_clean":"2024-05-21T21:55:36.550575+0000","last_became_active":"2024-05-21T21:51:16.946515+0000","last_became_peered":"2024-05-21T21:51:16.946515+0000","last_unstale":"2024-05-21T21:55:36.550575+0000","last_undegraded":"2024-05-21T21:55:36.550575+0000","last_fullsized":"2024-05-21T21:55:36.550575+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T09:26:25.867895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.342252+0000","last_change":"2024-05-21T21:51:16.929228+0000","last_active":"2024-05-21T21:55:36.342252+0000","last_peered":"2024-05-21T21:55:36.342252+0000","last_clean":"2024-05-21T21:55:36.342252+0000","last_became_active":"2024-05-21T21:51:16.928875+0000","last_became_peered":"2024-05-21T21:51:16.928875+0000","last_unstale":"2024-05-21T21:55:36.342252+0000","last_undegraded":"2024-05-21T21:55:36.342252+0000","last_fullsized":"2024-05-21T21:55:36.342252+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:50:56.431310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"82'11","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.199749+0000","last_change":"2024-05-21T21:51:16.932360+0000","last_active":"2024-05-21T21:55:36.199749+0000","last_peered":"2024-05-21T21:55:36.199749+0000","last_clean":"2024-05-21T21:55:36.199749+0000","last_became_active":"2024-05-21T21:51:16.932036+0000","last_became_peered":"2024-05-21T21:51:16.932036+0000","last_unstale":"2024-05-21T21:55:36.199749+0000","last_undegraded":"2024-05-21T21:55:36.199749+0000","last_fullsized":"2024-05-21T21:55:36.199749+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T09:06:37.257323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"82'16","reported_seq":45,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.356382+0000","last_change":"2024-05-21T21:52:10.876556+0000","last_active":"2024-05-21T21:55:36.356382+0000","last_peered":"2024-05-21T21:55:36.356382+0000","last_clean":"2024-05-21T21:55:36.356382+0000","last_became_active":"2024-05-21T21:52:10.876273+0000","last_became_peered":"2024-05-21T21:52:10.876273+0000","last_unstale":"2024-05-21T21:55:36.356382+0000","last_undegraded":"2024-05-21T21:55:36.356382+0000","last_fullsized":"2024-05-21T21:55:36.356382+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T05:09:40.037731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"82'6","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.550761+0000","last_change":"2024-05-21T21:51:16.933425+0000","last_active":"2024-05-21T21:55:36.550761+0000","last_peered":"2024-05-21T21:55:36.550761+0000","last_clean":"2024-05-21T21:55:36.550761+0000","last_became_active":"2024-05-21T21:51:16.933057+0000","last_became_peered":"2024-05-21T21:51:16.933057+0000","last_unstale":"2024-05-21T21:55:36.550761+0000","last_undegraded":"2024-05-21T21:55:36.550761+0000","last_fullsized":"2024-05-21T21:55:36.550761+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:04:57.237841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"82'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.723313+0000","last_change":"2024-05-21T21:51:16.927014+0000","last_active":"2024-05-21T21:55:36.723313+0000","last_peered":"2024-05-21T21:55:36.723313+0000","last_clean":"2024-05-21T21:55:36.723313+0000","last_became_active":"2024-05-21T21:51:16.925954+0000","last_became_peered":"2024-05-21T21:51:16.925954+0000","last_unstale":"2024-05-21T21:55:36.723313+0000","last_undegraded":"2024-05-21T21:55:36.723313+0000","last_fullsized":"2024-05-21T21:55:36.723313+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:19:20.530727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"82'10","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.555649+0000","last_change":"2024-05-21T21:51:16.930909+0000","last_active":"2024-05-21T21:55:36.555649+0000","last_peered":"2024-05-21T21:55:36.555649+0000","last_clean":"2024-05-21T21:55:36.555649+0000","last_became_active":"2024-05-21T21:51:16.927110+0000","last_became_peered":"2024-05-21T21:51:16.927110+0000","last_unstale":"2024-05-21T21:55:36.555649+0000","last_undegraded":"2024-05-21T21:55:36.555649+0000","last_fullsized":"2024-05-21T21:55:36.555649+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T06:23:55.386047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"82'7","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.552611+0000","last_change":"2024-05-21T21:51:16.938797+0000","last_active":"2024-05-21T21:55:36.552611+0000","last_peered":"2024-05-21T21:55:36.552611+0000","last_clean":"2024-05-21T21:55:36.552611+0000","last_became_active":"2024-05-21T21:51:16.938263+0000","last_became_peered":"2024-05-21T21:51:16.938263+0000","last_unstale":"2024-05-21T21:55:36.552611+0000","last_undegraded":"2024-05-21T21:55:36.552611+0000","last_fullsized":"2024-05-21T21:55:36.552611+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:37:40.521118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"82'7","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.552363+0000","last_change":"2024-05-21T21:51:16.944160+0000","last_active":"2024-05-21T21:55:36.552363+0000","last_peered":"2024-05-21T21:55:36.552363+0000","last_clean":"2024-05-21T21:55:36.552363+0000","last_became_active":"2024-05-21T21:51:16.940046+0000","last_became_peered":"2024-05-21T21:51:16.940046+0000","last_unstale":"2024-05-21T21:55:36.552363+0000","last_undegraded":"2024-05-21T21:55:36.552363+0000","last_fullsized":"2024-05-21T21:55:36.552363+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:14:13.278799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.198999+0000","last_change":"2024-05-21T21:51:16.945599+0000","last_active":"2024-05-21T21:55:36.198999+0000","last_peered":"2024-05-21T21:55:36.198999+0000","last_clean":"2024-05-21T21:55:36.198999+0000","last_became_active":"2024-05-21T21:51:16.945196+0000","last_became_peered":"2024-05-21T21:51:16.945196+0000","last_unstale":"2024-05-21T21:55:36.198999+0000","last_undegraded":"2024-05-21T21:55:36.198999+0000","last_fullsized":"2024-05-21T21:55:36.198999+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:59:24.453615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"82'8","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.341533+0000","last_change":"2024-05-21T21:51:16.944895+0000","last_active":"2024-05-21T21:55:36.341533+0000","last_peered":"2024-05-21T21:55:36.341533+0000","last_clean":"2024-05-21T21:55:36.341533+0000","last_became_active":"2024-05-21T21:51:16.944560+0000","last_became_peered":"2024-05-21T21:51:16.944560+0000","last_unstale":"2024-05-21T21:55:36.341533+0000","last_undegraded":"2024-05-21T21:55:36.341533+0000","last_fullsized":"2024-05-21T21:55:36.341533+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:53:13.264701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"82'10","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.197923+0000","last_change":"2024-05-21T21:51:16.943861+0000","last_active":"2024-05-21T21:55:36.197923+0000","last_peered":"2024-05-21T21:55:36.197923+0000","last_clean":"2024-05-21T21:55:36.197923+0000","last_became_active":"2024-05-21T21:51:16.943123+0000","last_became_peered":"2024-05-21T21:51:16.943123+0000","last_unstale":"2024-05-21T21:55:36.197923+0000","last_undegraded":"2024-05-21T21:55:36.197923+0000","last_fullsized":"2024-05-21T21:55:36.197923+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:35:37.732855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"82'7","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.201474+0000","last_change":"2024-05-21T21:51:16.940580+0000","last_active":"2024-05-21T21:55:36.201474+0000","last_peered":"2024-05-21T21:55:36.201474+0000","last_clean":"2024-05-21T21:55:36.201474+0000","last_became_active":"2024-05-21T21:51:16.940264+0000","last_became_peered":"2024-05-21T21:51:16.940264+0000","last_unstale":"2024-05-21T21:55:36.201474+0000","last_undegraded":"2024-05-21T21:55:36.201474+0000","last_fullsized":"2024-05-21T21:55:36.201474+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:54:22.915541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'14","reported_seq":304,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:55.928643+0000","last_change":"2024-05-21T21:51:12.852935+0000","last_active":"2024-05-21T21:55:55.928643+0000","last_peered":"2024-05-21T21:55:55.928643+0000","last_clean":"2024-05-21T21:55:55.928643+0000","last_became_active":"2024-05-21T21:51:12.851802+0000","last_became_peered":"2024-05-21T21:51:12.851802+0000","last_unstale":"2024-05-21T21:55:55.928643+0000","last_undegraded":"2024-05-21T21:55:55.928643+0000","last_fullsized":"2024-05-21T21:55:55.928643+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:13:24.643408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":260,"num_read_kb":260,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"82'5","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.343135+0000","last_change":"2024-05-21T21:51:16.939461+0000","last_active":"2024-05-21T21:55:36.343135+0000","last_peered":"2024-05-21T21:55:36.343135+0000","last_clean":"2024-05-21T21:55:36.343135+0000","last_became_active":"2024-05-21T21:51:16.936322+0000","last_became_peered":"2024-05-21T21:51:16.936322+0000","last_unstale":"2024-05-21T21:55:36.343135+0000","last_undegraded":"2024-05-21T21:55:36.343135+0000","last_fullsized":"2024-05-21T21:55:36.343135+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:37:26.153963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.343791+0000","last_change":"2024-05-21T21:51:12.941838+0000","last_active":"2024-05-21T21:55:36.343791+0000","last_peered":"2024-05-21T21:55:36.343791+0000","last_clean":"2024-05-21T21:55:36.343791+0000","last_became_active":"2024-05-21T21:51:12.940818+0000","last_became_peered":"2024-05-21T21:51:12.940818+0000","last_unstale":"2024-05-21T21:55:36.343791+0000","last_undegraded":"2024-05-21T21:55:36.343791+0000","last_fullsized":"2024-05-21T21:55:36.343791+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:30:42.349241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"82'6","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.554128+0000","last_change":"2024-05-21T21:51:16.950252+0000","last_active":"2024-05-21T21:55:36.554128+0000","last_peered":"2024-05-21T21:55:36.554128+0000","last_clean":"2024-05-21T21:55:36.554128+0000","last_became_active":"2024-05-21T21:51:16.947375+0000","last_became_peered":"2024-05-21T21:51:16.947375+0000","last_unstale":"2024-05-21T21:55:36.554128+0000","last_undegraded":"2024-05-21T21:55:36.554128+0000","last_fullsized":"2024-05-21T21:55:36.554128+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:27:30.462799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"83'13","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.341387+0000","last_change":"2024-05-21T21:52:10.875739+0000","last_active":"2024-05-21T21:55:36.341387+0000","last_peered":"2024-05-21T21:55:36.341387+0000","last_clean":"2024-05-21T21:55:36.341387+0000","last_became_active":"2024-05-21T21:52:10.865846+0000","last_became_peered":"2024-05-21T21:52:10.865846+0000","last_unstale":"2024-05-21T21:55:36.341387+0000","last_undegraded":"2024-05-21T21:55:36.341387+0000","last_fullsized":"2024-05-21T21:55:36.341387+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T06:48:06.465757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.199107+0000","last_change":"2024-05-21T21:51:16.926380+0000","last_active":"2024-05-21T21:55:36.199107+0000","last_peered":"2024-05-21T21:55:36.199107+0000","last_clean":"2024-05-21T21:55:36.199107+0000","last_became_active":"2024-05-21T21:51:16.925821+0000","last_became_peered":"2024-05-21T21:51:16.925821+0000","last_unstale":"2024-05-21T21:55:36.199107+0000","last_undegraded":"2024-05-21T21:55:36.199107+0000","last_fullsized":"2024-05-21T21:55:36.199107+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:42:14.887155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'5","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.556613+0000","last_change":"2024-05-21T21:51:12.854342+0000","last_active":"2024-05-21T21:55:36.556613+0000","last_peered":"2024-05-21T21:55:36.556613+0000","last_clean":"2024-05-21T21:55:36.556613+0000","last_became_active":"2024-05-21T21:51:12.853908+0000","last_became_peered":"2024-05-21T21:51:12.853908+0000","last_unstale":"2024-05-21T21:55:36.556613+0000","last_undegraded":"2024-05-21T21:55:36.556613+0000","last_fullsized":"2024-05-21T21:55:36.556613+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:35:10.374749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"82'15","reported_seq":44,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.352781+0000","last_change":"2024-05-21T21:52:10.973816+0000","last_active":"2024-05-21T21:55:36.352781+0000","last_peered":"2024-05-21T21:55:36.352781+0000","last_clean":"2024-05-21T21:55:36.352781+0000","last_became_active":"2024-05-21T21:52:10.973457+0000","last_became_peered":"2024-05-21T21:52:10.973457+0000","last_unstale":"2024-05-21T21:55:36.352781+0000","last_undegraded":"2024-05-21T21:55:36.352781+0000","last_fullsized":"2024-05-21T21:55:36.352781+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T05:01:41.915158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"83'9","reported_seq":302,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:55.912999+0000","last_change":"2024-05-21T21:51:12.855658+0000","last_active":"2024-05-21T21:55:55.912999+0000","last_peered":"2024-05-21T21:55:55.912999+0000","last_clean":"2024-05-21T21:55:55.912999+0000","last_became_active":"2024-05-21T21:51:12.855281+0000","last_became_peered":"2024-05-21T21:51:12.855281+0000","last_unstale":"2024-05-21T21:55:55.912999+0000","last_undegraded":"2024-05-21T21:55:55.912999+0000","last_fullsized":"2024-05-21T21:55:55.912999+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:55:08.836045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1855,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":264,"num_read_kb":6,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"82'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.556310+0000","last_change":"2024-05-21T21:51:16.946977+0000","last_active":"2024-05-21T21:55:36.556310+0000","last_peered":"2024-05-21T21:55:36.556310+0000","last_clean":"2024-05-21T21:55:36.556310+0000","last_became_active":"2024-05-21T21:51:16.946562+0000","last_became_peered":"2024-05-21T21:51:16.946562+0000","last_unstale":"2024-05-21T21:55:36.556310+0000","last_undegraded":"2024-05-21T21:55:36.556310+0000","last_fullsized":"2024-05-21T21:55:36.556310+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:43:33.370265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"80'7","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.355936+0000","last_change":"2024-05-21T21:51:12.852574+0000","last_active":"2024-05-21T21:55:36.355936+0000","last_peered":"2024-05-21T21:55:36.355936+0000","last_clean":"2024-05-21T21:55:36.355936+0000","last_became_active":"2024-05-21T21:51:12.852258+0000","last_became_peered":"2024-05-21T21:51:12.852258+0000","last_unstale":"2024-05-21T21:55:36.355936+0000","last_undegraded":"2024-05-21T21:55:36.355936+0000","last_fullsized":"2024-05-21T21:55:36.355936+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:09:40.280821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"82'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.352655+0000","last_change":"2024-05-21T21:51:16.934768+0000","last_active":"2024-05-21T21:55:36.352655+0000","last_peered":"2024-05-21T21:55:36.352655+0000","last_clean":"2024-05-21T21:55:36.352655+0000","last_became_active":"2024-05-21T21:51:16.934275+0000","last_became_peered":"2024-05-21T21:51:16.934275+0000","last_unstale":"2024-05-21T21:55:36.352655+0000","last_undegraded":"2024-05-21T21:55:36.352655+0000","last_fullsized":"2024-05-21T21:55:36.352655+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:31:54.425069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"80'3","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.555219+0000","last_change":"2024-05-21T21:51:12.854166+0000","last_active":"2024-05-21T21:55:36.555219+0000","last_peered":"2024-05-21T21:55:36.555219+0000","last_clean":"2024-05-21T21:55:36.555219+0000","last_became_active":"2024-05-21T21:51:12.853508+0000","last_became_peered":"2024-05-21T21:51:12.853508+0000","last_unstale":"2024-05-21T21:55:36.555219+0000","last_undegraded":"2024-05-21T21:55:36.555219+0000","last_fullsized":"2024-05-21T21:55:36.555219+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:29:41.858655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"82'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.342248+0000","last_change":"2024-05-21T21:51:16.938293+0000","last_active":"2024-05-21T21:55:36.342248+0000","last_peered":"2024-05-21T21:55:36.342248+0000","last_clean":"2024-05-21T21:55:36.342248+0000","last_became_active":"2024-05-21T21:51:16.937970+0000","last_became_peered":"2024-05-21T21:51:16.937970+0000","last_unstale":"2024-05-21T21:55:36.342248+0000","last_undegraded":"2024-05-21T21:55:36.342248+0000","last_fullsized":"2024-05-21T21:55:36.342248+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:18:27.473098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'3","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.352476+0000","last_change":"2024-05-21T21:51:12.853366+0000","last_active":"2024-05-21T21:55:36.352476+0000","last_peered":"2024-05-21T21:55:36.352476+0000","last_clean":"2024-05-21T21:55:36.352476+0000","last_became_active":"2024-05-21T21:51:12.853028+0000","last_became_peered":"2024-05-21T21:51:12.853028+0000","last_unstale":"2024-05-21T21:55:36.352476+0000","last_undegraded":"2024-05-21T21:55:36.352476+0000","last_fullsized":"2024-05-21T21:55:36.352476+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T09:15:52.593028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"82'11","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.197181+0000","last_change":"2024-05-21T21:51:16.947903+0000","last_active":"2024-05-21T21:55:36.197181+0000","last_peered":"2024-05-21T21:55:36.197181+0000","last_clean":"2024-05-21T21:55:36.197181+0000","last_became_active":"2024-05-21T21:51:16.946677+0000","last_became_peered":"2024-05-21T21:51:16.946677+0000","last_unstale":"2024-05-21T21:55:36.197181+0000","last_undegraded":"2024-05-21T21:55:36.197181+0000","last_fullsized":"2024-05-21T21:55:36.197181+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T06:12:56.514284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'7","reported_seq":552,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:56.777139+0000","last_change":"2024-05-21T21:51:12.854873+0000","last_active":"2024-05-21T21:55:56.777139+0000","last_peered":"2024-05-21T21:55:56.777139+0000","last_clean":"2024-05-21T21:55:56.777139+0000","last_became_active":"2024-05-21T21:51:12.854393+0000","last_became_peered":"2024-05-21T21:51:12.854393+0000","last_unstale":"2024-05-21T21:55:56.777139+0000","last_undegraded":"2024-05-21T21:55:56.777139+0000","last_fullsized":"2024-05-21T21:55:56.777139+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:58:52.411164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":516,"num_read_kb":516,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"82'10","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.199587+0000","last_change":"2024-05-21T21:51:16.951094+0000","last_active":"2024-05-21T21:55:36.199587+0000","last_peered":"2024-05-21T21:55:36.199587+0000","last_clean":"2024-05-21T21:55:36.199587+0000","last_became_active":"2024-05-21T21:51:16.950724+0000","last_became_peered":"2024-05-21T21:51:16.950724+0000","last_unstale":"2024-05-21T21:55:36.199587+0000","last_undegraded":"2024-05-21T21:55:36.199587+0000","last_fullsized":"2024-05-21T21:55:36.199587+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:54:02.835426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'4","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.550904+0000","last_change":"2024-05-21T21:51:12.850427+0000","last_active":"2024-05-21T21:55:36.550904+0000","last_peered":"2024-05-21T21:55:36.550904+0000","last_clean":"2024-05-21T21:55:36.550904+0000","last_became_active":"2024-05-21T21:51:12.850096+0000","last_became_peered":"2024-05-21T21:51:12.850096+0000","last_unstale":"2024-05-21T21:55:36.550904+0000","last_undegraded":"2024-05-21T21:55:36.550904+0000","last_fullsized":"2024-05-21T21:55:36.550904+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:43:22.611043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.554216+0000","last_change":"2024-05-21T21:51:16.942330+0000","last_active":"2024-05-21T21:55:36.554216+0000","last_peered":"2024-05-21T21:55:36.554216+0000","last_clean":"2024-05-21T21:55:36.554216+0000","last_became_active":"2024-05-21T21:51:16.941718+0000","last_became_peered":"2024-05-21T21:51:16.941718+0000","last_unstale":"2024-05-21T21:55:36.554216+0000","last_undegraded":"2024-05-21T21:55:36.554216+0000","last_fullsized":"2024-05-21T21:55:36.554216+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:54:17.176411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'5","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.352907+0000","last_change":"2024-05-21T21:51:12.941100+0000","last_active":"2024-05-21T21:55:36.352907+0000","last_peered":"2024-05-21T21:55:36.352907+0000","last_clean":"2024-05-21T21:55:36.352907+0000","last_became_active":"2024-05-21T21:51:12.940597+0000","last_became_peered":"2024-05-21T21:51:12.940597+0000","last_unstale":"2024-05-21T21:55:36.352907+0000","last_undegraded":"2024-05-21T21:55:36.352907+0000","last_fullsized":"2024-05-21T21:55:36.352907+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:30:58.457108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"82'11","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.199324+0000","last_change":"2024-05-21T21:51:16.921973+0000","last_active":"2024-05-21T21:55:36.199324+0000","last_peered":"2024-05-21T21:55:36.199324+0000","last_clean":"2024-05-21T21:55:36.199324+0000","last_became_active":"2024-05-21T21:51:16.921597+0000","last_became_peered":"2024-05-21T21:51:16.921597+0000","last_unstale":"2024-05-21T21:55:36.199324+0000","last_undegraded":"2024-05-21T21:55:36.199324+0000","last_fullsized":"2024-05-21T21:55:36.199324+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:25:32.395940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'41","reported_seq":602,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:56.777926+0000","last_change":"2024-05-21T21:51:12.850312+0000","last_active":"2024-05-21T21:55:56.777926+0000","last_peered":"2024-05-21T21:55:56.777926+0000","last_clean":"2024-05-21T21:55:56.777926+0000","last_became_active":"2024-05-21T21:51:12.849837+0000","last_became_peered":"2024-05-21T21:51:12.849837+0000","last_unstale":"2024-05-21T21:55:56.777926+0000","last_undegraded":"2024-05-21T21:55:56.777926+0000","last_fullsized":"2024-05-21T21:55:56.777926+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T07:09:13.339554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67614,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":531,"num_read_kb":332,"num_write":43,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"82'6","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.556542+0000","last_change":"2024-05-21T21:51:16.926625+0000","last_active":"2024-05-21T21:55:36.556542+0000","last_peered":"2024-05-21T21:55:36.556542+0000","last_clean":"2024-05-21T21:55:36.556542+0000","last_became_active":"2024-05-21T21:51:16.925889+0000","last_became_peered":"2024-05-21T21:51:16.925889+0000","last_unstale":"2024-05-21T21:55:36.556542+0000","last_undegraded":"2024-05-21T21:55:36.556542+0000","last_fullsized":"2024-05-21T21:55:36.556542+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:35:23.464747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'9","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.201314+0000","last_change":"2024-05-21T21:51:12.859772+0000","last_active":"2024-05-21T21:55:36.201314+0000","last_peered":"2024-05-21T21:55:36.201314+0000","last_clean":"2024-05-21T21:55:36.201314+0000","last_became_active":"2024-05-21T21:51:12.859290+0000","last_became_peered":"2024-05-21T21:51:12.859290+0000","last_unstale":"2024-05-21T21:55:36.201314+0000","last_undegraded":"2024-05-21T21:55:36.201314+0000","last_fullsized":"2024-05-21T21:55:36.201314+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T05:28:29.953381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.555713+0000","last_change":"2024-05-21T21:51:16.925803+0000","last_active":"2024-05-21T21:55:36.555713+0000","last_peered":"2024-05-21T21:55:36.555713+0000","last_clean":"2024-05-21T21:55:36.555713+0000","last_became_active":"2024-05-21T21:51:16.925111+0000","last_became_peered":"2024-05-21T21:51:16.925111+0000","last_unstale":"2024-05-21T21:55:36.555713+0000","last_undegraded":"2024-05-21T21:55:36.555713+0000","last_fullsized":"2024-05-21T21:55:36.555713+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T06:22:11.353379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'5","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.352741+0000","last_change":"2024-05-21T21:51:12.859593+0000","last_active":"2024-05-21T21:55:36.352741+0000","last_peered":"2024-05-21T21:55:36.352741+0000","last_clean":"2024-05-21T21:55:36.352741+0000","last_became_active":"2024-05-21T21:51:12.859197+0000","last_became_peered":"2024-05-21T21:51:12.859197+0000","last_unstale":"2024-05-21T21:55:36.352741+0000","last_undegraded":"2024-05-21T21:55:36.352741+0000","last_fullsized":"2024-05-21T21:55:36.352741+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:13:12.311496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"82'15","reported_seq":44,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.551024+0000","last_change":"2024-05-21T21:52:10.879028+0000","last_active":"2024-05-21T21:55:36.551024+0000","last_peered":"2024-05-21T21:55:36.551024+0000","last_clean":"2024-05-21T21:55:36.551024+0000","last_became_active":"2024-05-21T21:52:10.878664+0000","last_became_peered":"2024-05-21T21:52:10.878664+0000","last_unstale":"2024-05-21T21:55:36.551024+0000","last_undegraded":"2024-05-21T21:55:36.551024+0000","last_fullsized":"2024-05-21T21:55:36.551024+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:05:53.953546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'7","reported_seq":299,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:55.906932+0000","last_change":"2024-05-21T21:51:12.852169+0000","last_active":"2024-05-21T21:55:55.906932+0000","last_peered":"2024-05-21T21:55:55.906932+0000","last_clean":"2024-05-21T21:55:55.906932+0000","last_became_active":"2024-05-21T21:51:12.851782+0000","last_became_peered":"2024-05-21T21:51:12.851782+0000","last_unstale":"2024-05-21T21:55:55.906932+0000","last_undegraded":"2024-05-21T21:55:55.906932+0000","last_fullsized":"2024-05-21T21:55:55.906932+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:46:09.692158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":263,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"82'6","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.550587+0000","last_change":"2024-05-21T21:51:16.929264+0000","last_active":"2024-05-21T21:55:36.550587+0000","last_peered":"2024-05-21T21:55:36.550587+0000","last_clean":"2024-05-21T21:55:36.550587+0000","last_became_active":"2024-05-21T21:51:16.928900+0000","last_became_peered":"2024-05-21T21:51:16.928900+0000","last_unstale":"2024-05-21T21:55:36.550587+0000","last_undegraded":"2024-05-21T21:55:36.550587+0000","last_fullsized":"2024-05-21T21:55:36.550587+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:42:28.040939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'17","reported_seq":72,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.556306+0000","last_change":"2024-05-21T21:52:10.984451+0000","last_active":"2024-05-21T21:55:36.556306+0000","last_peered":"2024-05-21T21:55:36.556306+0000","last_clean":"2024-05-21T21:55:36.556306+0000","last_became_active":"2024-05-21T21:52:10.974050+0000","last_became_peered":"2024-05-21T21:52:10.974050+0000","last_unstale":"2024-05-21T21:55:36.556306+0000","last_undegraded":"2024-05-21T21:55:36.556306+0000","last_fullsized":"2024-05-21T21:55:36.556306+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T07:08:18.307022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"82'6","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.352888+0000","last_change":"2024-05-21T21:51:16.935464+0000","last_active":"2024-05-21T21:55:36.352888+0000","last_peered":"2024-05-21T21:55:36.352888+0000","last_clean":"2024-05-21T21:55:36.352888+0000","last_became_active":"2024-05-21T21:51:16.935091+0000","last_became_peered":"2024-05-21T21:51:16.935091+0000","last_unstale":"2024-05-21T21:55:36.352888+0000","last_undegraded":"2024-05-21T21:55:36.352888+0000","last_fullsized":"2024-05-21T21:55:36.352888+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T06:59:16.330649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.556757+0000","last_change":"2024-05-21T21:51:12.853327+0000","last_active":"2024-05-21T21:55:36.556757+0000","last_peered":"2024-05-21T21:55:36.556757+0000","last_clean":"2024-05-21T21:55:36.556757+0000","last_became_active":"2024-05-21T21:51:12.852946+0000","last_became_peered":"2024-05-21T21:51:12.852946+0000","last_unstale":"2024-05-21T21:55:36.556757+0000","last_undegraded":"2024-05-21T21:55:36.556757+0000","last_fullsized":"2024-05-21T21:55:36.556757+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:00:34.256858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"82'5","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.341370+0000","last_change":"2024-05-21T21:51:16.929341+0000","last_active":"2024-05-21T21:55:36.341370+0000","last_peered":"2024-05-21T21:55:36.341370+0000","last_clean":"2024-05-21T21:55:36.341370+0000","last_became_active":"2024-05-21T21:51:16.929008+0000","last_became_peered":"2024-05-21T21:51:16.929008+0000","last_unstale":"2024-05-21T21:55:36.341370+0000","last_undegraded":"2024-05-21T21:55:36.341370+0000","last_fullsized":"2024-05-21T21:55:36.341370+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:26:26.297136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"80'1","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.197640+0000","last_change":"2024-05-21T21:51:12.945433+0000","last_active":"2024-05-21T21:55:36.197640+0000","last_peered":"2024-05-21T21:55:36.197640+0000","last_clean":"2024-05-21T21:55:36.197640+0000","last_became_active":"2024-05-21T21:51:12.944463+0000","last_became_peered":"2024-05-21T21:51:12.944463+0000","last_unstale":"2024-05-21T21:55:36.197640+0000","last_undegraded":"2024-05-21T21:55:36.197640+0000","last_fullsized":"2024-05-21T21:55:36.197640+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:29:52.014583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'5","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.356192+0000","last_change":"2024-05-21T21:52:10.883554+0000","last_active":"2024-05-21T21:55:36.356192+0000","last_peered":"2024-05-21T21:55:36.356192+0000","last_clean":"2024-05-21T21:55:36.356192+0000","last_became_active":"2024-05-21T21:52:10.877789+0000","last_became_peered":"2024-05-21T21:52:10.877789+0000","last_unstale":"2024-05-21T21:55:36.356192+0000","last_undegraded":"2024-05-21T21:55:36.356192+0000","last_fullsized":"2024-05-21T21:55:36.356192+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:52:37.623551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"83'6","reported_seq":296,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:55.929924+0000","last_change":"2024-05-21T21:51:12.852883+0000","last_active":"2024-05-21T21:55:55.929924+0000","last_peered":"2024-05-21T21:55:55.929924+0000","last_clean":"2024-05-21T21:55:55.929924+0000","last_became_active":"2024-05-21T21:51:12.852549+0000","last_became_peered":"2024-05-21T21:51:12.852549+0000","last_unstale":"2024-05-21T21:55:55.929924+0000","last_undegraded":"2024-05-21T21:55:55.929924+0000","last_fullsized":"2024-05-21T21:55:55.929924+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:05:53.002286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":260,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"82'7","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.356089+0000","last_change":"2024-05-21T21:51:16.944425+0000","last_active":"2024-05-21T21:55:36.356089+0000","last_peered":"2024-05-21T21:55:36.356089+0000","last_clean":"2024-05-21T21:55:36.356089+0000","last_became_active":"2024-05-21T21:51:16.944033+0000","last_became_peered":"2024-05-21T21:51:16.944033+0000","last_unstale":"2024-05-21T21:55:36.356089+0000","last_undegraded":"2024-05-21T21:55:36.356089+0000","last_fullsized":"2024-05-21T21:55:36.356089+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T07:16:26.802329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"82'8","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.352742+0000","last_change":"2024-05-21T21:51:16.933264+0000","last_active":"2024-05-21T21:55:36.352742+0000","last_peered":"2024-05-21T21:55:36.352742+0000","last_clean":"2024-05-21T21:55:36.352742+0000","last_became_active":"2024-05-21T21:51:16.932911+0000","last_became_peered":"2024-05-21T21:51:16.932911+0000","last_unstale":"2024-05-21T21:55:36.352742+0000","last_undegraded":"2024-05-21T21:55:36.352742+0000","last_fullsized":"2024-05-21T21:55:36.352742+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:25:54.006367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"82'6","reported_seq":295,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:55.906271+0000","last_change":"2024-05-21T21:51:12.942900+0000","last_active":"2024-05-21T21:55:55.906271+0000","last_peered":"2024-05-21T21:55:55.906271+0000","last_clean":"2024-05-21T21:55:55.906271+0000","last_became_active":"2024-05-21T21:51:12.941038+0000","last_became_peered":"2024-05-21T21:51:12.941038+0000","last_unstale":"2024-05-21T21:55:55.906271+0000","last_undegraded":"2024-05-21T21:55:55.906271+0000","last_fullsized":"2024-05-21T21:55:55.906271+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:38:43.240011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":259,"num_read_kb":259,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"82'8","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.342906+0000","last_change":"2024-05-21T21:51:16.933391+0000","last_active":"2024-05-21T21:55:36.342906+0000","last_peered":"2024-05-21T21:55:36.342906+0000","last_clean":"2024-05-21T21:55:36.342906+0000","last_became_active":"2024-05-21T21:51:16.932320+0000","last_became_peered":"2024-05-21T21:51:16.932320+0000","last_unstale":"2024-05-21T21:55:36.342906+0000","last_undegraded":"2024-05-21T21:55:36.342906+0000","last_fullsized":"2024-05-21T21:55:36.342906+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T09:06:59.126748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'6","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.723215+0000","last_change":"2024-05-21T21:51:12.856342+0000","last_active":"2024-05-21T21:55:36.723215+0000","last_peered":"2024-05-21T21:55:36.723215+0000","last_clean":"2024-05-21T21:55:36.723215+0000","last_became_active":"2024-05-21T21:51:12.855952+0000","last_became_peered":"2024-05-21T21:51:12.855952+0000","last_unstale":"2024-05-21T21:55:36.723215+0000","last_undegraded":"2024-05-21T21:55:36.723215+0000","last_fullsized":"2024-05-21T21:55:36.723215+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:34:40.734034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"83'99","reported_seq":131,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:38.358271+0000","last_change":"2024-05-21T21:49:42.797187+0000","last_active":"2024-05-21T21:55:38.358271+0000","last_peered":"2024-05-21T21:55:38.358271+0000","last_clean":"2024-05-21T21:55:38.358271+0000","last_became_active":"2024-05-21T21:49:42.791291+0000","last_became_peered":"2024-05-21T21:49:42.791291+0000","last_unstale":"2024-05-21T21:55:38.358271+0000","last_undegraded":"2024-05-21T21:55:38.358271+0000","last_fullsized":"2024-05-21T21:55:38.358271+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:45:55.552293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:45:55.552293+0000","last_clean_scrub_stamp":"2024-05-21T21:45:55.552293+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T07:24:18.302523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":144,"num_read_kb":248,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"82'5","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.555133+0000","last_change":"2024-05-21T21:51:16.926446+0000","last_active":"2024-05-21T21:55:36.555133+0000","last_peered":"2024-05-21T21:55:36.555133+0000","last_clean":"2024-05-21T21:55:36.555133+0000","last_became_active":"2024-05-21T21:51:16.926070+0000","last_became_peered":"2024-05-21T21:51:16.926070+0000","last_unstale":"2024-05-21T21:55:36.555133+0000","last_undegraded":"2024-05-21T21:55:36.555133+0000","last_fullsized":"2024-05-21T21:55:36.555133+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:44:27.366393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'4","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.550735+0000","last_change":"2024-05-21T21:51:12.854371+0000","last_active":"2024-05-21T21:55:36.550735+0000","last_peered":"2024-05-21T21:55:36.550735+0000","last_clean":"2024-05-21T21:55:36.550735+0000","last_became_active":"2024-05-21T21:51:12.853847+0000","last_became_peered":"2024-05-21T21:51:12.853847+0000","last_unstale":"2024-05-21T21:55:36.550735+0000","last_undegraded":"2024-05-21T21:55:36.550735+0000","last_fullsized":"2024-05-21T21:55:36.550735+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:14:10.549901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"80'4","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.556551+0000","last_change":"2024-05-21T21:51:12.941227+0000","last_active":"2024-05-21T21:55:36.556551+0000","last_peered":"2024-05-21T21:55:36.556551+0000","last_clean":"2024-05-21T21:55:36.556551+0000","last_became_active":"2024-05-21T21:51:12.940702+0000","last_became_peered":"2024-05-21T21:51:12.940702+0000","last_unstale":"2024-05-21T21:55:36.556551+0000","last_undegraded":"2024-05-21T21:55:36.556551+0000","last_fullsized":"2024-05-21T21:55:36.556551+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:40:34.124921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"82'9","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.556301+0000","last_change":"2024-05-21T21:51:16.945627+0000","last_active":"2024-05-21T21:55:36.556301+0000","last_peered":"2024-05-21T21:55:36.556301+0000","last_clean":"2024-05-21T21:55:36.556301+0000","last_became_active":"2024-05-21T21:51:16.945297+0000","last_became_peered":"2024-05-21T21:51:16.945297+0000","last_unstale":"2024-05-21T21:55:36.556301+0000","last_undegraded":"2024-05-21T21:55:36.556301+0000","last_fullsized":"2024-05-21T21:55:36.556301+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:54:38.898774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'104","reported_seq":242,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:54.029254+0000","last_change":"2024-05-21T21:52:10.888076+0000","last_active":"2024-05-21T21:55:54.029254+0000","last_peered":"2024-05-21T21:55:54.029254+0000","last_clean":"2024-05-21T21:55:54.029254+0000","last_became_active":"2024-05-21T21:52:10.879368+0000","last_became_peered":"2024-05-21T21:52:10.879368+0000","last_unstale":"2024-05-21T21:55:54.029254+0000","last_undegraded":"2024-05-21T21:55:54.029254+0000","last_fullsized":"2024-05-21T21:55:54.029254+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:46:43.190411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":312,"num_read_kb":277,"num_write":190,"num_write_kb":1286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"82'5","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.344028+0000","last_change":"2024-05-21T21:51:16.929807+0000","last_active":"2024-05-21T21:55:36.344028+0000","last_peered":"2024-05-21T21:55:36.344028+0000","last_clean":"2024-05-21T21:55:36.344028+0000","last_became_active":"2024-05-21T21:51:16.928859+0000","last_became_peered":"2024-05-21T21:51:16.928859+0000","last_unstale":"2024-05-21T21:55:36.344028+0000","last_undegraded":"2024-05-21T21:55:36.344028+0000","last_fullsized":"2024-05-21T21:55:36.344028+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:41:32.830892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'7","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.723447+0000","last_change":"2024-05-21T21:51:12.850843+0000","last_active":"2024-05-21T21:55:36.723447+0000","last_peered":"2024-05-21T21:55:36.723447+0000","last_clean":"2024-05-21T21:55:36.723447+0000","last_became_active":"2024-05-21T21:51:12.850516+0000","last_became_peered":"2024-05-21T21:51:12.850516+0000","last_unstale":"2024-05-21T21:55:36.723447+0000","last_undegraded":"2024-05-21T21:55:36.723447+0000","last_fullsized":"2024-05-21T21:55:36.723447+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:37:05.191626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"82'7","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.553781+0000","last_change":"2024-05-21T21:51:16.949017+0000","last_active":"2024-05-21T21:55:36.553781+0000","last_peered":"2024-05-21T21:55:36.553781+0000","last_clean":"2024-05-21T21:55:36.553781+0000","last_became_active":"2024-05-21T21:51:16.948691+0000","last_became_peered":"2024-05-21T21:51:16.948691+0000","last_unstale":"2024-05-21T21:55:36.553781+0000","last_undegraded":"2024-05-21T21:55:36.553781+0000","last_fullsized":"2024-05-21T21:55:36.553781+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:32:06.606581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.199191+0000","last_change":"2024-05-21T21:51:12.939203+0000","last_active":"2024-05-21T21:55:36.199191+0000","last_peered":"2024-05-21T21:55:36.199191+0000","last_clean":"2024-05-21T21:55:36.199191+0000","last_became_active":"2024-05-21T21:51:12.938936+0000","last_became_peered":"2024-05-21T21:51:12.938936+0000","last_unstale":"2024-05-21T21:55:36.199191+0000","last_undegraded":"2024-05-21T21:55:36.199191+0000","last_fullsized":"2024-05-21T21:55:36.199191+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:55:50.336362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"82'2","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.556315+0000","last_change":"2024-05-21T21:51:16.926216+0000","last_active":"2024-05-21T21:55:36.556315+0000","last_peered":"2024-05-21T21:55:36.556315+0000","last_clean":"2024-05-21T21:55:36.556315+0000","last_became_active":"2024-05-21T21:51:16.925834+0000","last_became_peered":"2024-05-21T21:51:16.925834+0000","last_unstale":"2024-05-21T21:55:36.556315+0000","last_undegraded":"2024-05-21T21:55:36.556315+0000","last_fullsized":"2024-05-21T21:55:36.556315+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:13:30.192908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'7","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.342741+0000","last_change":"2024-05-21T21:51:12.855306+0000","last_active":"2024-05-21T21:55:36.342741+0000","last_peered":"2024-05-21T21:55:36.342741+0000","last_clean":"2024-05-21T21:55:36.342741+0000","last_became_active":"2024-05-21T21:51:12.854722+0000","last_became_peered":"2024-05-21T21:51:12.854722+0000","last_unstale":"2024-05-21T21:55:36.342741+0000","last_undegraded":"2024-05-21T21:55:36.342741+0000","last_fullsized":"2024-05-21T21:55:36.342741+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:39:41.583219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"82'9","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.341026+0000","last_change":"2024-05-21T21:51:16.944758+0000","last_active":"2024-05-21T21:55:36.341026+0000","last_peered":"2024-05-21T21:55:36.341026+0000","last_clean":"2024-05-21T21:55:36.341026+0000","last_became_active":"2024-05-21T21:51:16.944393+0000","last_became_peered":"2024-05-21T21:51:16.944393+0000","last_unstale":"2024-05-21T21:55:36.341026+0000","last_undegraded":"2024-05-21T21:55:36.341026+0000","last_fullsized":"2024-05-21T21:55:36.341026+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:01:23.048355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'4","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:37.227713+0000","last_change":"2024-05-21T21:51:12.860378+0000","last_active":"2024-05-21T21:55:37.227713+0000","last_peered":"2024-05-21T21:55:37.227713+0000","last_clean":"2024-05-21T21:55:37.227713+0000","last_became_active":"2024-05-21T21:51:12.859978+0000","last_became_peered":"2024-05-21T21:51:12.859978+0000","last_unstale":"2024-05-21T21:55:37.227713+0000","last_undegraded":"2024-05-21T21:55:37.227713+0000","last_fullsized":"2024-05-21T21:55:37.227713+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T07:52:26.902232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"82'8","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.343395+0000","last_change":"2024-05-21T21:51:16.943194+0000","last_active":"2024-05-21T21:55:36.343395+0000","last_peered":"2024-05-21T21:55:36.343395+0000","last_clean":"2024-05-21T21:55:36.343395+0000","last_became_active":"2024-05-21T21:51:16.942884+0000","last_became_peered":"2024-05-21T21:51:16.942884+0000","last_unstale":"2024-05-21T21:55:36.343395+0000","last_undegraded":"2024-05-21T21:55:36.343395+0000","last_fullsized":"2024-05-21T21:55:36.343395+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T05:58:17.436995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.556514+0000","last_change":"2024-05-21T21:51:12.853390+0000","last_active":"2024-05-21T21:55:36.556514+0000","last_peered":"2024-05-21T21:55:36.556514+0000","last_clean":"2024-05-21T21:55:36.556514+0000","last_became_active":"2024-05-21T21:51:12.852990+0000","last_became_peered":"2024-05-21T21:51:12.852990+0000","last_unstale":"2024-05-21T21:55:36.556514+0000","last_undegraded":"2024-05-21T21:55:36.556514+0000","last_fullsized":"2024-05-21T21:55:36.556514+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:32:16.392609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"82'5","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.550848+0000","last_change":"2024-05-21T21:51:16.933430+0000","last_active":"2024-05-21T21:55:36.550848+0000","last_peered":"2024-05-21T21:55:36.550848+0000","last_clean":"2024-05-21T21:55:36.550848+0000","last_became_active":"2024-05-21T21:51:16.933053+0000","last_became_peered":"2024-05-21T21:51:16.933053+0000","last_unstale":"2024-05-21T21:55:36.550848+0000","last_undegraded":"2024-05-21T21:55:36.550848+0000","last_fullsized":"2024-05-21T21:55:36.550848+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T06:14:57.468727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'7","reported_seq":298,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:55.776030+0000","last_change":"2024-05-21T21:51:12.946113+0000","last_active":"2024-05-21T21:55:55.776030+0000","last_peered":"2024-05-21T21:55:55.776030+0000","last_clean":"2024-05-21T21:55:55.776030+0000","last_became_active":"2024-05-21T21:51:12.945650+0000","last_became_peered":"2024-05-21T21:51:12.945650+0000","last_unstale":"2024-05-21T21:55:55.776030+0000","last_undegraded":"2024-05-21T21:55:55.776030+0000","last_fullsized":"2024-05-21T21:55:55.776030+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T09:40:18.209400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1952,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":261,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"82'7","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.550939+0000","last_change":"2024-05-21T21:51:16.932632+0000","last_active":"2024-05-21T21:55:36.550939+0000","last_peered":"2024-05-21T21:55:36.550939+0000","last_clean":"2024-05-21T21:55:36.550939+0000","last_became_active":"2024-05-21T21:51:16.932256+0000","last_became_peered":"2024-05-21T21:51:16.932256+0000","last_unstale":"2024-05-21T21:55:36.550939+0000","last_undegraded":"2024-05-21T21:55:36.550939+0000","last_fullsized":"2024-05-21T21:55:36.550939+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:04:48.146485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'67","reported_seq":128,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:53.558384+0000","last_change":"2024-05-21T21:51:12.860602+0000","last_active":"2024-05-21T21:55:53.558384+0000","last_peered":"2024-05-21T21:55:53.558384+0000","last_clean":"2024-05-21T21:55:53.558384+0000","last_became_active":"2024-05-21T21:51:12.860228+0000","last_became_peered":"2024-05-21T21:51:12.860228+0000","last_unstale":"2024-05-21T21:55:53.558384+0000","last_undegraded":"2024-05-21T21:55:53.558384+0000","last_fullsized":"2024-05-21T21:55:53.558384+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:33:39.230872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":33,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'7","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.551083+0000","last_change":"2024-05-21T21:51:16.929616+0000","last_active":"2024-05-21T21:55:36.551083+0000","last_peered":"2024-05-21T21:55:36.551083+0000","last_clean":"2024-05-21T21:55:36.551083+0000","last_became_active":"2024-05-21T21:51:16.928569+0000","last_became_peered":"2024-05-21T21:51:16.928569+0000","last_unstale":"2024-05-21T21:55:36.551083+0000","last_undegraded":"2024-05-21T21:55:36.551083+0000","last_fullsized":"2024-05-21T21:55:36.551083+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:15:41.790344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'5","reported_seq":47,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.555560+0000","last_change":"2024-05-21T21:52:10.881114+0000","last_active":"2024-05-21T21:55:36.555560+0000","last_peered":"2024-05-21T21:55:36.555560+0000","last_clean":"2024-05-21T21:55:36.555560+0000","last_became_active":"2024-05-21T21:52:10.872022+0000","last_became_peered":"2024-05-21T21:52:10.872022+0000","last_unstale":"2024-05-21T21:55:36.555560+0000","last_undegraded":"2024-05-21T21:55:36.555560+0000","last_fullsized":"2024-05-21T21:55:36.555560+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T06:05:51.419156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"82'9","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.550977+0000","last_change":"2024-05-21T21:51:16.926496+0000","last_active":"2024-05-21T21:55:36.550977+0000","last_peered":"2024-05-21T21:55:36.550977+0000","last_clean":"2024-05-21T21:55:36.550977+0000","last_became_active":"2024-05-21T21:51:16.926112+0000","last_became_peered":"2024-05-21T21:51:16.926112+0000","last_unstale":"2024-05-21T21:55:36.550977+0000","last_undegraded":"2024-05-21T21:55:36.550977+0000","last_fullsized":"2024-05-21T21:55:36.550977+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:26:08.254421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"82'9","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.341047+0000","last_change":"2024-05-21T21:51:16.930522+0000","last_active":"2024-05-21T21:55:36.341047+0000","last_peered":"2024-05-21T21:55:36.341047+0000","last_clean":"2024-05-21T21:55:36.341047+0000","last_became_active":"2024-05-21T21:51:16.929853+0000","last_became_peered":"2024-05-21T21:51:16.929853+0000","last_unstale":"2024-05-21T21:55:36.341047+0000","last_undegraded":"2024-05-21T21:55:36.341047+0000","last_fullsized":"2024-05-21T21:55:36.341047+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:10:57.940009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"82'12","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.201162+0000","last_change":"2024-05-21T21:51:16.931751+0000","last_active":"2024-05-21T21:55:36.201162+0000","last_peered":"2024-05-21T21:55:36.201162+0000","last_clean":"2024-05-21T21:55:36.201162+0000","last_became_active":"2024-05-21T21:51:16.931353+0000","last_became_peered":"2024-05-21T21:51:16.931353+0000","last_unstale":"2024-05-21T21:55:36.201162+0000","last_undegraded":"2024-05-21T21:55:36.201162+0000","last_fullsized":"2024-05-21T21:55:36.201162+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:33:45.297850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"82'8","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.201127+0000","last_change":"2024-05-21T21:51:16.940096+0000","last_active":"2024-05-21T21:55:36.201127+0000","last_peered":"2024-05-21T21:55:36.201127+0000","last_clean":"2024-05-21T21:55:36.201127+0000","last_became_active":"2024-05-21T21:51:16.939740+0000","last_became_peered":"2024-05-21T21:51:16.939740+0000","last_unstale":"2024-05-21T21:55:36.201127+0000","last_undegraded":"2024-05-21T21:55:36.201127+0000","last_fullsized":"2024-05-21T21:55:36.201127+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:59:52.569526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"82'7","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.552155+0000","last_change":"2024-05-21T21:51:16.946488+0000","last_active":"2024-05-21T21:55:36.552155+0000","last_peered":"2024-05-21T21:55:36.552155+0000","last_clean":"2024-05-21T21:55:36.552155+0000","last_became_active":"2024-05-21T21:51:16.945957+0000","last_became_peered":"2024-05-21T21:51:16.945957+0000","last_unstale":"2024-05-21T21:55:36.552155+0000","last_undegraded":"2024-05-21T21:55:36.552155+0000","last_fullsized":"2024-05-21T21:55:36.552155+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:40:36.708344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"82'4","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.556177+0000","last_change":"2024-05-21T21:51:16.949349+0000","last_active":"2024-05-21T21:55:36.556177+0000","last_peered":"2024-05-21T21:55:36.556177+0000","last_clean":"2024-05-21T21:55:36.556177+0000","last_became_active":"2024-05-21T21:51:16.946986+0000","last_became_peered":"2024-05-21T21:51:16.946986+0000","last_unstale":"2024-05-21T21:55:36.556177+0000","last_undegraded":"2024-05-21T21:55:36.556177+0000","last_fullsized":"2024-05-21T21:55:36.556177+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:01:14.877110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"82'6","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.341365+0000","last_change":"2024-05-21T21:51:16.929172+0000","last_active":"2024-05-21T21:55:36.341365+0000","last_peered":"2024-05-21T21:55:36.341365+0000","last_clean":"2024-05-21T21:55:36.341365+0000","last_became_active":"2024-05-21T21:51:16.928823+0000","last_became_peered":"2024-05-21T21:51:16.928823+0000","last_unstale":"2024-05-21T21:55:36.341365+0000","last_undegraded":"2024-05-21T21:55:36.341365+0000","last_fullsized":"2024-05-21T21:55:36.341365+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:09:13.719272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"82'15","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.201520+0000","last_change":"2024-05-21T21:51:16.930332+0000","last_active":"2024-05-21T21:55:36.201520+0000","last_peered":"2024-05-21T21:55:36.201520+0000","last_clean":"2024-05-21T21:55:36.201520+0000","last_became_active":"2024-05-21T21:51:16.930024+0000","last_became_peered":"2024-05-21T21:51:16.930024+0000","last_unstale":"2024-05-21T21:55:36.201520+0000","last_undegraded":"2024-05-21T21:55:36.201520+0000","last_fullsized":"2024-05-21T21:55:36.201520+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:47:41.272725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"82'2","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.343903+0000","last_change":"2024-05-21T21:51:16.930102+0000","last_active":"2024-05-21T21:55:36.343903+0000","last_peered":"2024-05-21T21:55:36.343903+0000","last_clean":"2024-05-21T21:55:36.343903+0000","last_became_active":"2024-05-21T21:51:16.929410+0000","last_became_peered":"2024-05-21T21:51:16.929410+0000","last_unstale":"2024-05-21T21:55:36.343903+0000","last_undegraded":"2024-05-21T21:55:36.343903+0000","last_fullsized":"2024-05-21T21:55:36.343903+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:11:46.296288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"82'6","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.556235+0000","last_change":"2024-05-21T21:52:10.873424+0000","last_active":"2024-05-21T21:55:36.556235+0000","last_peered":"2024-05-21T21:55:36.556235+0000","last_clean":"2024-05-21T21:55:36.556235+0000","last_became_active":"2024-05-21T21:52:10.873118+0000","last_became_peered":"2024-05-21T21:52:10.873118+0000","last_unstale":"2024-05-21T21:55:36.556235+0000","last_undegraded":"2024-05-21T21:55:36.556235+0000","last_fullsized":"2024-05-21T21:55:36.556235+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:05:00.731013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,3],"acting":[8,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.29","version":"82'8","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.342675+0000","last_change":"2024-05-21T21:51:16.934442+0000","last_active":"2024-05-21T21:55:36.342675+0000","last_peered":"2024-05-21T21:55:36.342675+0000","last_clean":"2024-05-21T21:55:36.342675+0000","last_became_active":"2024-05-21T21:51:16.934086+0000","last_became_peered":"2024-05-21T21:51:16.934086+0000","last_unstale":"2024-05-21T21:55:36.342675+0000","last_undegraded":"2024-05-21T21:55:36.342675+0000","last_fullsized":"2024-05-21T21:55:36.342675+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:36:44.302532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"82'10","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.199462+0000","last_change":"2024-05-21T21:51:16.924479+0000","last_active":"2024-05-21T21:55:36.199462+0000","last_peered":"2024-05-21T21:55:36.199462+0000","last_clean":"2024-05-21T21:55:36.199462+0000","last_became_active":"2024-05-21T21:51:16.923998+0000","last_became_peered":"2024-05-21T21:51:16.923998+0000","last_unstale":"2024-05-21T21:55:36.199462+0000","last_undegraded":"2024-05-21T21:55:36.199462+0000","last_fullsized":"2024-05-21T21:55:36.199462+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:23:07.931731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"82'11","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.555830+0000","last_change":"2024-05-21T21:51:16.947271+0000","last_active":"2024-05-21T21:55:36.555830+0000","last_peered":"2024-05-21T21:55:36.555830+0000","last_clean":"2024-05-21T21:55:36.555830+0000","last_became_active":"2024-05-21T21:51:16.946912+0000","last_became_peered":"2024-05-21T21:51:16.946912+0000","last_unstale":"2024-05-21T21:55:36.555830+0000","last_undegraded":"2024-05-21T21:55:36.555830+0000","last_fullsized":"2024-05-21T21:55:36.555830+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:51:29.646915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"82'8","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.341091+0000","last_change":"2024-05-21T21:51:16.928852+0000","last_active":"2024-05-21T21:55:36.341091+0000","last_peered":"2024-05-21T21:55:36.341091+0000","last_clean":"2024-05-21T21:55:36.341091+0000","last_became_active":"2024-05-21T21:51:16.928480+0000","last_became_peered":"2024-05-21T21:51:16.928480+0000","last_unstale":"2024-05-21T21:55:36.341091+0000","last_undegraded":"2024-05-21T21:55:36.341091+0000","last_fullsized":"2024-05-21T21:55:36.341091+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T09:12:30.691254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":144,"num_read_kb":248,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":798422,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":2977,"num_read_kb":1710,"num_write":509,"num_write_kb":2386,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":112,"num_keys_recovered":17,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2764800,"data_stored":2395266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":387,"ondisk_log_size":387,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645385,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28296,"kb_used_data":1280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716856,"statfs":{"total":95995035648,"available":95966060544,"internally_reserved":0,"allocated":1310720,"data_stored":836913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841613,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":29048,"kb_used_data":1912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716104,"statfs":{"total":95995035648,"available":95965290496,"internally_reserved":0,"allocated":1957888,"data_stored":1430322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005138,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28324,"kb_used_data":1308,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716828,"statfs":{"total":95995035648,"available":95966031872,"internally_reserved":0,"allocated":1339392,"data_stored":836178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201369,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":937984,"data_stored":443223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397597,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28516,"kb_used_data":1500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716636,"statfs":{"total":95995035648,"available":95965835264,"internally_reserved":0,"allocated":1536000,"data_stored":1037506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561123,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":925696,"data_stored":442953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574975,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":511592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738500,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36264,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708888,"statfs":{"total":95995035648,"available":95957901312,"internally_reserved":0,"allocated":946176,"data_stored":444683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411451,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36320,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708832,"statfs":{"total":95995035648,"available":95957843968,"internally_reserved":0,"allocated":1007616,"data_stored":513212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149811,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37080,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708072,"statfs":{"total":95995035648,"available":95957065728,"internally_reserved":0,"allocated":1662976,"data_stored":1163718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953582,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":925696,"data_stored":442955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757353,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":33132,"kb_used_data":1836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712020,"statfs":{"total":95995035648,"available":95961108480,"internally_reserved":0,"allocated":1880064,"data_stored":1364493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":69869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":71489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":753664,"data_stored":721878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":332285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":398114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":395190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-21T21:56:03.025 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-21T21:56:03.025 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph pg dump --format=json 2024-05-21T21:56:03.522 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:56:03.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:56:03 smithi164 ceph-mon[36269]: from='client.15222 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T21:56:03.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:56:03 smithi164 ceph-mon[36269]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 76 KiB/s wr, 40 op/s 2024-05-21T21:56:03.831 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:56:03 smithi151 ceph-mon[31623]: from='client.15222 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T21:56:03.831 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:56:03 smithi151 ceph-mon[31623]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 76 KiB/s wr, 40 op/s 2024-05-21T21:56:03.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:56:03 smithi184 ceph-mon[36942]: from='client.15222 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T21:56:03.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:56:03 smithi184 ceph-mon[36942]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 76 KiB/s wr, 40 op/s 2024-05-21T21:56:04.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:56:04 smithi184 ceph-mon[36942]: scrub summary: idle 2024-05-21T21:56:04.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:56:04 smithi164 ceph-mon[36269]: scrub summary: idle 2024-05-21T21:56:04.872 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:56:04 smithi151 ceph-mon[31623]: scrub summary: idle 2024-05-21T21:56:05.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:05.670+0000 7f4ba1e00700 1 -- 172.21.15.151:0/4057166720 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4b9c06b250 msgr2=0x7f4b9c06b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:05.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:05.670+0000 7f4ba1e00700 1 --2- 172.21.15.151:0/4057166720 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4b9c06b250 0x7f4b9c06b630 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c00b0a0 tx=0x7f4b8c00b3b0 comp rx=0 tx=0).stop 2024-05-21T21:56:05.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:05.670+0000 7f4ba1e00700 1 -- 172.21.15.151:0/4057166720 shutdown_connections 2024-05-21T21:56:05.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:05.670+0000 7f4ba1e00700 1 --2- 172.21.15.151:0/4057166720 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4b9c10fff0 0x7f4b9c1123e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:05.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:05.670+0000 7f4ba1e00700 1 --2- 172.21.15.151:0/4057166720 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4b9c06b250 0x7f4b9c06b630 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:05.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:05.670+0000 7f4ba1e00700 1 --2- 172.21.15.151:0/4057166720 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4b9c06bb70 0x7f4b9c10fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:05.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:05.670+0000 7f4ba1e00700 1 -- 172.21.15.151:0/4057166720 >> 172.21.15.151:0/4057166720 conn(0x7f4b9c0793f0 msgr2=0x7f4b9c079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:56:05.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:05.670+0000 7f4ba1e00700 1 -- 172.21.15.151:0/4057166720 shutdown_connections 2024-05-21T21:56:05.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:05.670+0000 7f4ba1e00700 1 -- 172.21.15.151:0/4057166720 wait complete. 2024-05-21T21:56:05.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:05.671+0000 7f4ba1e00700 1 Processor -- start 2024-05-21T21:56:05.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:05.671+0000 7f4ba1e00700 1 -- start start 2024-05-21T21:56:05.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:05.671+0000 7f4ba1e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4b9c06b250 0x7f4b9c19be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:05.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:05.671+0000 7f4ba1e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4b9c06bb70 0x7f4b9c19c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:05.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:05.672+0000 7f4ba1e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4b9c10fff0 0x7f4b9c1a0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:05.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:05.672+0000 7f4ba1e00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4b9c115960 con 0x7f4b9c10fff0 2024-05-21T21:56:05.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:05.672+0000 7f4ba1e00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4b9c1157e0 con 0x7f4b9c06bb70 2024-05-21T21:56:05.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:05.672+0000 7f4ba1e00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4b9c115ae0 con 0x7f4b9c06b250 2024-05-21T21:56:05.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:05.672+0000 7f4b9aa00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4b9c06b250 0x7f4b9c19be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:05.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:05.672+0000 7f4b9b400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4b9c10fff0 0x7f4b9c1a0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:05.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:05.672+0000 7f4b92000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4b9c06bb70 0x7f4b9c19c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:05.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:05.672+0000 7f4b9b400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4b9c10fff0 0x7f4b9c1a0740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:60174/0 (socket says 172.21.15.151:60174) 2024-05-21T21:56:05.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:05.672+0000 7f4b9aa00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4b9c06b250 0x7f4b9c19be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:40040/0 (socket says 172.21.15.151:40040) 2024-05-21T21:56:05.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:05.672+0000 7f4b9b400700 1 -- 172.21.15.151:0/806012687 learned_addr learned my addr 172.21.15.151:0/806012687 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:56:05.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:05.672+0000 7f4b92000700 1 -- 172.21.15.151:0/806012687 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4b9c06b250 msgr2=0x7f4b9c19be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:05.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:05.672+0000 7f4b92000700 1 --2- 172.21.15.151:0/806012687 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4b9c06b250 0x7f4b9c19be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:05.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:05.672+0000 7f4b92000700 1 -- 172.21.15.151:0/806012687 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4b9c10fff0 msgr2=0x7f4b9c1a0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:05.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:05.672+0000 7f4b92000700 1 --2- 172.21.15.151:0/806012687 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4b9c10fff0 0x7f4b9c1a0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:05.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:05.672+0000 7f4b92000700 1 -- 172.21.15.151:0/806012687 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b9c1a0e80 con 0x7f4b9c06bb70 2024-05-21T21:56:05.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:05.672+0000 7f4b9b400700 1 --2- 172.21.15.151:0/806012687 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4b9c10fff0 0x7f4b9c1a0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:56:05.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:05.673+0000 7f4b92000700 1 --2- 172.21.15.151:0/806012687 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4b9c06bb70 0x7f4b9c19c3b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4b8400e9f0 tx=0x7f4b8400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:56:05.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:05.673+0000 7f4ba0a00700 1 -- 172.21.15.151:0/806012687 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b84004220 con 0x7f4b9c06bb70 2024-05-21T21:56:05.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:05.673+0000 7f4ba1e00700 1 -- 172.21.15.151:0/806012687 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b9c1a1170 con 0x7f4b9c06bb70 2024-05-21T21:56:05.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:05.673+0000 7f4ba0a00700 1 -- 172.21.15.151:0/806012687 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b840043c0 con 0x7f4b9c06bb70 2024-05-21T21:56:05.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:05.673+0000 7f4ba0a00700 1 -- 172.21.15.151:0/806012687 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b84005020 con 0x7f4b9c06bb70 2024-05-21T21:56:05.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:05.673+0000 7f4ba1e00700 1 -- 172.21.15.151:0/806012687 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b9c1a89f0 con 0x7f4b9c06bb70 2024-05-21T21:56:05.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:05.674+0000 7f4ba1e00700 1 -- 172.21.15.151:0/806012687 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b9c06a230 con 0x7f4b9c06bb70 2024-05-21T21:56:05.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:05.677+0000 7f4ba0a00700 1 -- 172.21.15.151:0/806012687 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 149886+0+0 (secure 0 0 0) 0x7f4b84004750 con 0x7f4b9c06bb70 2024-05-21T21:56:05.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:05.678+0000 7f4ba0a00700 1 --2- 172.21.15.151:0/806012687 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f4b800b2b10 0x7f4b800b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:05.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:05.678+0000 7f4ba0a00700 1 -- 172.21.15.151:0/806012687 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f4b840e2760 con 0x7f4b9c06bb70 2024-05-21T21:56:05.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:05.678+0000 7f4b9aa00700 1 --2- 172.21.15.151:0/806012687 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f4b800b2b10 0x7f4b800b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:05.679 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:05.679+0000 7f4b9aa00700 1 --2- 172.21.15.151:0/806012687 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f4b800b2b10 0x7f4b800b4fd0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c0062a0 tx=0x7f4b8c006210 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:56:05.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:05.681+0000 7f4ba0a00700 1 -- 172.21.15.151:0/806012687 <== mon.2 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4b840a9f70 con 0x7f4b9c06bb70 2024-05-21T21:56:05.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:05.909+0000 7f4ba1e00700 1 -- 172.21.15.151:0/806012687 --> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4b9c063980 con 0x7f4b800b2b10 2024-05-21T21:56:05.910 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:56:05 smithi151 ceph-mon[31623]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 99 KiB/s wr, 59 op/s 2024-05-21T21:56:05.932 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:05.931+0000 7f4ba0a00700 1 -- 172.21.15.151:0/806012687 <== mgr.25138 v2:172.21.15.151:6800/958417056 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257828 (secure 0 0 0) 0x7f4b9c063980 con 0x7f4b800b2b10 2024-05-21T21:56:05.934 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T21:56:05.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:05.936+0000 7f4ba1e00700 1 -- 172.21.15.151:0/806012687 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f4b800b2b10 msgr2=0x7f4b800b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:05.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:05.936+0000 7f4ba1e00700 1 --2- 172.21.15.151:0/806012687 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f4b800b2b10 0x7f4b800b4fd0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c0062a0 tx=0x7f4b8c006210 comp rx=0 tx=0).stop 2024-05-21T21:56:05.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:05.936+0000 7f4ba1e00700 1 -- 172.21.15.151:0/806012687 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4b9c06bb70 msgr2=0x7f4b9c19c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:05.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:05.936+0000 7f4ba1e00700 1 --2- 172.21.15.151:0/806012687 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4b9c06bb70 0x7f4b9c19c3b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4b8400e9f0 tx=0x7f4b8400ed00 comp rx=0 tx=0).stop 2024-05-21T21:56:05.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:05.937+0000 7f4ba1e00700 1 -- 172.21.15.151:0/806012687 shutdown_connections 2024-05-21T21:56:05.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:05.937+0000 7f4ba1e00700 1 --2- 172.21.15.151:0/806012687 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f4b800b2b10 0x7f4b800b4fd0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:05.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:05.937+0000 7f4ba1e00700 1 --2- 172.21.15.151:0/806012687 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4b9c10fff0 0x7f4b9c1a0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:05.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:05.937+0000 7f4ba1e00700 1 --2- 172.21.15.151:0/806012687 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4b9c06b250 0x7f4b9c19be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:05.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:05.937+0000 7f4ba1e00700 1 --2- 172.21.15.151:0/806012687 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4b9c06bb70 0x7f4b9c19c3b0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:05.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:05.937+0000 7f4ba1e00700 1 -- 172.21.15.151:0/806012687 >> 172.21.15.151:0/806012687 conn(0x7f4b9c0793f0 msgr2=0x7f4b9c1114a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:56:05.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:05.937+0000 7f4ba1e00700 1 -- 172.21.15.151:0/806012687 shutdown_connections 2024-05-21T21:56:05.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:05.937+0000 7f4ba1e00700 1 -- 172.21.15.151:0/806012687 wait complete. 2024-05-21T21:56:05.941 INFO:teuthology.orchestra.run.smithi151.stderr:dumped all 2024-05-21T21:56:05.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:56:05 smithi184 ceph-mon[36942]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 99 KiB/s wr, 59 op/s 2024-05-21T21:56:06.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:56:05 smithi164 ceph-mon[36269]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 99 KiB/s wr, 59 op/s 2024-05-21T21:56:06.892 INFO:teuthology.orchestra.run.smithi151.stdout:{"pg_ready":true,"pg_map":{"version":17,"stamp":"2024-05-21T21:56:04.227506+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1391244,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":3295,"num_read_kb":2097,"num_write":707,"num_write_kb":4807,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":2952069,"num_keys_recovered":17,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1027,"ondisk_log_size":1027,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":368760,"kb_used_data":15072,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124573064,"statfs":{"total":1151940427776,"available":1151562817536,"internally_reserved":0,"allocated":15433728,"data_stored":9474408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19140,"internal_metadata":361674044},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":199990,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":530,"num_read_kb":423,"num_write":189,"num_write_kb":1186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003724"},"pg_stats":[{"pgid":"3.3f","version":"82'8","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.341334+0000","last_change":"2024-05-21T21:51:16.945132+0000","last_active":"2024-05-21T21:55:36.341334+0000","last_peered":"2024-05-21T21:55:36.341334+0000","last_clean":"2024-05-21T21:55:36.341334+0000","last_became_active":"2024-05-21T21:51:16.944766+0000","last_became_peered":"2024-05-21T21:51:16.944766+0000","last_unstale":"2024-05-21T21:55:36.341334+0000","last_undegraded":"2024-05-21T21:55:36.341334+0000","last_fullsized":"2024-05-21T21:55:36.341334+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:54:04.185095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"82'4","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.553624+0000","last_change":"2024-05-21T21:51:16.939291+0000","last_active":"2024-05-21T21:55:36.553624+0000","last_peered":"2024-05-21T21:55:36.553624+0000","last_clean":"2024-05-21T21:55:36.553624+0000","last_became_active":"2024-05-21T21:51:16.938730+0000","last_became_peered":"2024-05-21T21:51:16.938730+0000","last_unstale":"2024-05-21T21:55:36.553624+0000","last_undegraded":"2024-05-21T21:55:36.553624+0000","last_fullsized":"2024-05-21T21:55:36.553624+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:05:30.623744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"82'7","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.553252+0000","last_change":"2024-05-21T21:51:16.941565+0000","last_active":"2024-05-21T21:55:36.553252+0000","last_peered":"2024-05-21T21:55:36.553252+0000","last_clean":"2024-05-21T21:55:36.553252+0000","last_became_active":"2024-05-21T21:51:16.941033+0000","last_became_peered":"2024-05-21T21:51:16.941033+0000","last_unstale":"2024-05-21T21:55:36.553252+0000","last_undegraded":"2024-05-21T21:55:36.553252+0000","last_fullsized":"2024-05-21T21:55:36.553252+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:04:55.789479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"82'8","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.723315+0000","last_change":"2024-05-21T21:51:16.928455+0000","last_active":"2024-05-21T21:55:36.723315+0000","last_peered":"2024-05-21T21:55:36.723315+0000","last_clean":"2024-05-21T21:55:36.723315+0000","last_became_active":"2024-05-21T21:51:16.927874+0000","last_became_peered":"2024-05-21T21:51:16.927874+0000","last_unstale":"2024-05-21T21:55:36.723315+0000","last_undegraded":"2024-05-21T21:55:36.723315+0000","last_fullsized":"2024-05-21T21:55:36.723315+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:25:16.483175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"82'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.556362+0000","last_change":"2024-05-21T21:51:16.937773+0000","last_active":"2024-05-21T21:55:36.556362+0000","last_peered":"2024-05-21T21:55:36.556362+0000","last_clean":"2024-05-21T21:55:36.556362+0000","last_became_active":"2024-05-21T21:51:16.937306+0000","last_became_peered":"2024-05-21T21:51:16.937306+0000","last_unstale":"2024-05-21T21:55:36.556362+0000","last_undegraded":"2024-05-21T21:55:36.556362+0000","last_fullsized":"2024-05-21T21:55:36.556362+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:05:03.762390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"82'6","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.356010+0000","last_change":"2024-05-21T21:51:16.948275+0000","last_active":"2024-05-21T21:55:36.356010+0000","last_peered":"2024-05-21T21:55:36.356010+0000","last_clean":"2024-05-21T21:55:36.356010+0000","last_became_active":"2024-05-21T21:51:16.947890+0000","last_became_peered":"2024-05-21T21:51:16.947890+0000","last_unstale":"2024-05-21T21:55:36.356010+0000","last_undegraded":"2024-05-21T21:55:36.356010+0000","last_fullsized":"2024-05-21T21:55:36.356010+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:08:38.572177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"82'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.550575+0000","last_change":"2024-05-21T21:51:16.946821+0000","last_active":"2024-05-21T21:55:36.550575+0000","last_peered":"2024-05-21T21:55:36.550575+0000","last_clean":"2024-05-21T21:55:36.550575+0000","last_became_active":"2024-05-21T21:51:16.946515+0000","last_became_peered":"2024-05-21T21:51:16.946515+0000","last_unstale":"2024-05-21T21:55:36.550575+0000","last_undegraded":"2024-05-21T21:55:36.550575+0000","last_fullsized":"2024-05-21T21:55:36.550575+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T09:26:25.867895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.342252+0000","last_change":"2024-05-21T21:51:16.929228+0000","last_active":"2024-05-21T21:55:36.342252+0000","last_peered":"2024-05-21T21:55:36.342252+0000","last_clean":"2024-05-21T21:55:36.342252+0000","last_became_active":"2024-05-21T21:51:16.928875+0000","last_became_peered":"2024-05-21T21:51:16.928875+0000","last_unstale":"2024-05-21T21:55:36.342252+0000","last_undegraded":"2024-05-21T21:55:36.342252+0000","last_fullsized":"2024-05-21T21:55:36.342252+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:50:56.431310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"82'11","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.199749+0000","last_change":"2024-05-21T21:51:16.932360+0000","last_active":"2024-05-21T21:55:36.199749+0000","last_peered":"2024-05-21T21:55:36.199749+0000","last_clean":"2024-05-21T21:55:36.199749+0000","last_became_active":"2024-05-21T21:51:16.932036+0000","last_became_peered":"2024-05-21T21:51:16.932036+0000","last_unstale":"2024-05-21T21:55:36.199749+0000","last_undegraded":"2024-05-21T21:55:36.199749+0000","last_fullsized":"2024-05-21T21:55:36.199749+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T09:06:37.257323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"82'16","reported_seq":45,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.356382+0000","last_change":"2024-05-21T21:52:10.876556+0000","last_active":"2024-05-21T21:55:36.356382+0000","last_peered":"2024-05-21T21:55:36.356382+0000","last_clean":"2024-05-21T21:55:36.356382+0000","last_became_active":"2024-05-21T21:52:10.876273+0000","last_became_peered":"2024-05-21T21:52:10.876273+0000","last_unstale":"2024-05-21T21:55:36.356382+0000","last_undegraded":"2024-05-21T21:55:36.356382+0000","last_fullsized":"2024-05-21T21:55:36.356382+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T05:09:40.037731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"82'6","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.550761+0000","last_change":"2024-05-21T21:51:16.933425+0000","last_active":"2024-05-21T21:55:36.550761+0000","last_peered":"2024-05-21T21:55:36.550761+0000","last_clean":"2024-05-21T21:55:36.550761+0000","last_became_active":"2024-05-21T21:51:16.933057+0000","last_became_peered":"2024-05-21T21:51:16.933057+0000","last_unstale":"2024-05-21T21:55:36.550761+0000","last_undegraded":"2024-05-21T21:55:36.550761+0000","last_fullsized":"2024-05-21T21:55:36.550761+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:04:57.237841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"82'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.723313+0000","last_change":"2024-05-21T21:51:16.927014+0000","last_active":"2024-05-21T21:55:36.723313+0000","last_peered":"2024-05-21T21:55:36.723313+0000","last_clean":"2024-05-21T21:55:36.723313+0000","last_became_active":"2024-05-21T21:51:16.925954+0000","last_became_peered":"2024-05-21T21:51:16.925954+0000","last_unstale":"2024-05-21T21:55:36.723313+0000","last_undegraded":"2024-05-21T21:55:36.723313+0000","last_fullsized":"2024-05-21T21:55:36.723313+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:19:20.530727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"82'10","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.555649+0000","last_change":"2024-05-21T21:51:16.930909+0000","last_active":"2024-05-21T21:55:36.555649+0000","last_peered":"2024-05-21T21:55:36.555649+0000","last_clean":"2024-05-21T21:55:36.555649+0000","last_became_active":"2024-05-21T21:51:16.927110+0000","last_became_peered":"2024-05-21T21:51:16.927110+0000","last_unstale":"2024-05-21T21:55:36.555649+0000","last_undegraded":"2024-05-21T21:55:36.555649+0000","last_fullsized":"2024-05-21T21:55:36.555649+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T06:23:55.386047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"82'7","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.552611+0000","last_change":"2024-05-21T21:51:16.938797+0000","last_active":"2024-05-21T21:55:36.552611+0000","last_peered":"2024-05-21T21:55:36.552611+0000","last_clean":"2024-05-21T21:55:36.552611+0000","last_became_active":"2024-05-21T21:51:16.938263+0000","last_became_peered":"2024-05-21T21:51:16.938263+0000","last_unstale":"2024-05-21T21:55:36.552611+0000","last_undegraded":"2024-05-21T21:55:36.552611+0000","last_fullsized":"2024-05-21T21:55:36.552611+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:37:40.521118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"82'7","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.552363+0000","last_change":"2024-05-21T21:51:16.944160+0000","last_active":"2024-05-21T21:55:36.552363+0000","last_peered":"2024-05-21T21:55:36.552363+0000","last_clean":"2024-05-21T21:55:36.552363+0000","last_became_active":"2024-05-21T21:51:16.940046+0000","last_became_peered":"2024-05-21T21:51:16.940046+0000","last_unstale":"2024-05-21T21:55:36.552363+0000","last_undegraded":"2024-05-21T21:55:36.552363+0000","last_fullsized":"2024-05-21T21:55:36.552363+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:14:13.278799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"82'9","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.199075+0000","last_change":"2024-05-21T21:51:16.945599+0000","last_active":"2024-05-21T21:55:36.199075+0000","last_peered":"2024-05-21T21:55:36.199075+0000","last_clean":"2024-05-21T21:55:36.199075+0000","last_became_active":"2024-05-21T21:51:16.945196+0000","last_became_peered":"2024-05-21T21:51:16.945196+0000","last_unstale":"2024-05-21T21:55:36.199075+0000","last_undegraded":"2024-05-21T21:55:36.199075+0000","last_fullsized":"2024-05-21T21:55:36.199075+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:59:24.453615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"82'8","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.341533+0000","last_change":"2024-05-21T21:51:16.944895+0000","last_active":"2024-05-21T21:55:36.341533+0000","last_peered":"2024-05-21T21:55:36.341533+0000","last_clean":"2024-05-21T21:55:36.341533+0000","last_became_active":"2024-05-21T21:51:16.944560+0000","last_became_peered":"2024-05-21T21:51:16.944560+0000","last_unstale":"2024-05-21T21:55:36.341533+0000","last_undegraded":"2024-05-21T21:55:36.341533+0000","last_fullsized":"2024-05-21T21:55:36.341533+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:53:13.264701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"82'10","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.197923+0000","last_change":"2024-05-21T21:51:16.943861+0000","last_active":"2024-05-21T21:55:36.197923+0000","last_peered":"2024-05-21T21:55:36.197923+0000","last_clean":"2024-05-21T21:55:36.197923+0000","last_became_active":"2024-05-21T21:51:16.943123+0000","last_became_peered":"2024-05-21T21:51:16.943123+0000","last_unstale":"2024-05-21T21:55:36.197923+0000","last_undegraded":"2024-05-21T21:55:36.197923+0000","last_fullsized":"2024-05-21T21:55:36.197923+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:35:37.732855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"82'7","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.201474+0000","last_change":"2024-05-21T21:51:16.940580+0000","last_active":"2024-05-21T21:55:36.201474+0000","last_peered":"2024-05-21T21:55:36.201474+0000","last_clean":"2024-05-21T21:55:36.201474+0000","last_became_active":"2024-05-21T21:51:16.940264+0000","last_became_peered":"2024-05-21T21:51:16.940264+0000","last_unstale":"2024-05-21T21:55:36.201474+0000","last_undegraded":"2024-05-21T21:55:36.201474+0000","last_fullsized":"2024-05-21T21:55:36.201474+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:54:22.915541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'14","reported_seq":309,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:56:00.941791+0000","last_change":"2024-05-21T21:51:12.852935+0000","last_active":"2024-05-21T21:56:00.941791+0000","last_peered":"2024-05-21T21:56:00.941791+0000","last_clean":"2024-05-21T21:56:00.941791+0000","last_became_active":"2024-05-21T21:51:12.851802+0000","last_became_peered":"2024-05-21T21:51:12.851802+0000","last_unstale":"2024-05-21T21:56:00.941791+0000","last_undegraded":"2024-05-21T21:56:00.941791+0000","last_fullsized":"2024-05-21T21:56:00.941791+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:13:24.643408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":265,"num_read_kb":265,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"82'5","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.343135+0000","last_change":"2024-05-21T21:51:16.939461+0000","last_active":"2024-05-21T21:55:36.343135+0000","last_peered":"2024-05-21T21:55:36.343135+0000","last_clean":"2024-05-21T21:55:36.343135+0000","last_became_active":"2024-05-21T21:51:16.936322+0000","last_became_peered":"2024-05-21T21:51:16.936322+0000","last_unstale":"2024-05-21T21:55:36.343135+0000","last_undegraded":"2024-05-21T21:55:36.343135+0000","last_fullsized":"2024-05-21T21:55:36.343135+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:37:26.153963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.343791+0000","last_change":"2024-05-21T21:51:12.941838+0000","last_active":"2024-05-21T21:55:36.343791+0000","last_peered":"2024-05-21T21:55:36.343791+0000","last_clean":"2024-05-21T21:55:36.343791+0000","last_became_active":"2024-05-21T21:51:12.940818+0000","last_became_peered":"2024-05-21T21:51:12.940818+0000","last_unstale":"2024-05-21T21:55:36.343791+0000","last_undegraded":"2024-05-21T21:55:36.343791+0000","last_fullsized":"2024-05-21T21:55:36.343791+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:30:42.349241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"82'6","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.554128+0000","last_change":"2024-05-21T21:51:16.950252+0000","last_active":"2024-05-21T21:55:36.554128+0000","last_peered":"2024-05-21T21:55:36.554128+0000","last_clean":"2024-05-21T21:55:36.554128+0000","last_became_active":"2024-05-21T21:51:16.947375+0000","last_became_peered":"2024-05-21T21:51:16.947375+0000","last_unstale":"2024-05-21T21:55:36.554128+0000","last_undegraded":"2024-05-21T21:55:36.554128+0000","last_fullsized":"2024-05-21T21:55:36.554128+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:27:30.462799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"83'13","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.341387+0000","last_change":"2024-05-21T21:52:10.875739+0000","last_active":"2024-05-21T21:55:36.341387+0000","last_peered":"2024-05-21T21:55:36.341387+0000","last_clean":"2024-05-21T21:55:36.341387+0000","last_became_active":"2024-05-21T21:52:10.865846+0000","last_became_peered":"2024-05-21T21:52:10.865846+0000","last_unstale":"2024-05-21T21:55:36.341387+0000","last_undegraded":"2024-05-21T21:55:36.341387+0000","last_fullsized":"2024-05-21T21:55:36.341387+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T06:48:06.465757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.199107+0000","last_change":"2024-05-21T21:51:16.926380+0000","last_active":"2024-05-21T21:55:36.199107+0000","last_peered":"2024-05-21T21:55:36.199107+0000","last_clean":"2024-05-21T21:55:36.199107+0000","last_became_active":"2024-05-21T21:51:16.925821+0000","last_became_peered":"2024-05-21T21:51:16.925821+0000","last_unstale":"2024-05-21T21:55:36.199107+0000","last_undegraded":"2024-05-21T21:55:36.199107+0000","last_fullsized":"2024-05-21T21:55:36.199107+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:42:14.887155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'5","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.556613+0000","last_change":"2024-05-21T21:51:12.854342+0000","last_active":"2024-05-21T21:55:36.556613+0000","last_peered":"2024-05-21T21:55:36.556613+0000","last_clean":"2024-05-21T21:55:36.556613+0000","last_became_active":"2024-05-21T21:51:12.853908+0000","last_became_peered":"2024-05-21T21:51:12.853908+0000","last_unstale":"2024-05-21T21:55:36.556613+0000","last_undegraded":"2024-05-21T21:55:36.556613+0000","last_fullsized":"2024-05-21T21:55:36.556613+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:35:10.374749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"82'15","reported_seq":44,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.352781+0000","last_change":"2024-05-21T21:52:10.973816+0000","last_active":"2024-05-21T21:55:36.352781+0000","last_peered":"2024-05-21T21:55:36.352781+0000","last_clean":"2024-05-21T21:55:36.352781+0000","last_became_active":"2024-05-21T21:52:10.973457+0000","last_became_peered":"2024-05-21T21:52:10.973457+0000","last_unstale":"2024-05-21T21:55:36.352781+0000","last_undegraded":"2024-05-21T21:55:36.352781+0000","last_fullsized":"2024-05-21T21:55:36.352781+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T05:01:41.915158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"83'10","reported_seq":309,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:56:00.925487+0000","last_change":"2024-05-21T21:51:12.855658+0000","last_active":"2024-05-21T21:56:00.925487+0000","last_peered":"2024-05-21T21:56:00.925487+0000","last_clean":"2024-05-21T21:56:00.925487+0000","last_became_active":"2024-05-21T21:51:12.855281+0000","last_became_peered":"2024-05-21T21:51:12.855281+0000","last_unstale":"2024-05-21T21:56:00.925487+0000","last_undegraded":"2024-05-21T21:56:00.925487+0000","last_fullsized":"2024-05-21T21:56:00.925487+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:55:08.836045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":270,"num_read_kb":7,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"82'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.556310+0000","last_change":"2024-05-21T21:51:16.946977+0000","last_active":"2024-05-21T21:55:36.556310+0000","last_peered":"2024-05-21T21:55:36.556310+0000","last_clean":"2024-05-21T21:55:36.556310+0000","last_became_active":"2024-05-21T21:51:16.946562+0000","last_became_peered":"2024-05-21T21:51:16.946562+0000","last_unstale":"2024-05-21T21:55:36.556310+0000","last_undegraded":"2024-05-21T21:55:36.556310+0000","last_fullsized":"2024-05-21T21:55:36.556310+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:43:33.370265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"80'7","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.355936+0000","last_change":"2024-05-21T21:51:12.852574+0000","last_active":"2024-05-21T21:55:36.355936+0000","last_peered":"2024-05-21T21:55:36.355936+0000","last_clean":"2024-05-21T21:55:36.355936+0000","last_became_active":"2024-05-21T21:51:12.852258+0000","last_became_peered":"2024-05-21T21:51:12.852258+0000","last_unstale":"2024-05-21T21:55:36.355936+0000","last_undegraded":"2024-05-21T21:55:36.355936+0000","last_fullsized":"2024-05-21T21:55:36.355936+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:09:40.280821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"82'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.352655+0000","last_change":"2024-05-21T21:51:16.934768+0000","last_active":"2024-05-21T21:55:36.352655+0000","last_peered":"2024-05-21T21:55:36.352655+0000","last_clean":"2024-05-21T21:55:36.352655+0000","last_became_active":"2024-05-21T21:51:16.934275+0000","last_became_peered":"2024-05-21T21:51:16.934275+0000","last_unstale":"2024-05-21T21:55:36.352655+0000","last_undegraded":"2024-05-21T21:55:36.352655+0000","last_fullsized":"2024-05-21T21:55:36.352655+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:31:54.425069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"80'3","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.555219+0000","last_change":"2024-05-21T21:51:12.854166+0000","last_active":"2024-05-21T21:55:36.555219+0000","last_peered":"2024-05-21T21:55:36.555219+0000","last_clean":"2024-05-21T21:55:36.555219+0000","last_became_active":"2024-05-21T21:51:12.853508+0000","last_became_peered":"2024-05-21T21:51:12.853508+0000","last_unstale":"2024-05-21T21:55:36.555219+0000","last_undegraded":"2024-05-21T21:55:36.555219+0000","last_fullsized":"2024-05-21T21:55:36.555219+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:29:41.858655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"82'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.342248+0000","last_change":"2024-05-21T21:51:16.938293+0000","last_active":"2024-05-21T21:55:36.342248+0000","last_peered":"2024-05-21T21:55:36.342248+0000","last_clean":"2024-05-21T21:55:36.342248+0000","last_became_active":"2024-05-21T21:51:16.937970+0000","last_became_peered":"2024-05-21T21:51:16.937970+0000","last_unstale":"2024-05-21T21:55:36.342248+0000","last_undegraded":"2024-05-21T21:55:36.342248+0000","last_fullsized":"2024-05-21T21:55:36.342248+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:18:27.473098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'3","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.352559+0000","last_change":"2024-05-21T21:51:12.853366+0000","last_active":"2024-05-21T21:55:36.352559+0000","last_peered":"2024-05-21T21:55:36.352559+0000","last_clean":"2024-05-21T21:55:36.352559+0000","last_became_active":"2024-05-21T21:51:12.853028+0000","last_became_peered":"2024-05-21T21:51:12.853028+0000","last_unstale":"2024-05-21T21:55:36.352559+0000","last_undegraded":"2024-05-21T21:55:36.352559+0000","last_fullsized":"2024-05-21T21:55:36.352559+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T09:15:52.593028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"82'11","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.197181+0000","last_change":"2024-05-21T21:51:16.947903+0000","last_active":"2024-05-21T21:55:36.197181+0000","last_peered":"2024-05-21T21:55:36.197181+0000","last_clean":"2024-05-21T21:55:36.197181+0000","last_became_active":"2024-05-21T21:51:16.946677+0000","last_became_peered":"2024-05-21T21:51:16.946677+0000","last_unstale":"2024-05-21T21:55:36.197181+0000","last_undegraded":"2024-05-21T21:55:36.197181+0000","last_fullsized":"2024-05-21T21:55:36.197181+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T06:12:56.514284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'9","reported_seq":564,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:56:01.786079+0000","last_change":"2024-05-21T21:51:12.854873+0000","last_active":"2024-05-21T21:56:01.786079+0000","last_peered":"2024-05-21T21:56:01.786079+0000","last_clean":"2024-05-21T21:56:01.786079+0000","last_became_active":"2024-05-21T21:51:12.854393+0000","last_became_peered":"2024-05-21T21:51:12.854393+0000","last_unstale":"2024-05-21T21:56:01.786079+0000","last_undegraded":"2024-05-21T21:56:01.786079+0000","last_fullsized":"2024-05-21T21:56:01.786079+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:58:52.411164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":526,"num_read_kb":526,"num_write":10,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"82'10","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.199587+0000","last_change":"2024-05-21T21:51:16.951094+0000","last_active":"2024-05-21T21:55:36.199587+0000","last_peered":"2024-05-21T21:55:36.199587+0000","last_clean":"2024-05-21T21:55:36.199587+0000","last_became_active":"2024-05-21T21:51:16.950724+0000","last_became_peered":"2024-05-21T21:51:16.950724+0000","last_unstale":"2024-05-21T21:55:36.199587+0000","last_undegraded":"2024-05-21T21:55:36.199587+0000","last_fullsized":"2024-05-21T21:55:36.199587+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:54:02.835426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'4","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.550904+0000","last_change":"2024-05-21T21:51:12.850427+0000","last_active":"2024-05-21T21:55:36.550904+0000","last_peered":"2024-05-21T21:55:36.550904+0000","last_clean":"2024-05-21T21:55:36.550904+0000","last_became_active":"2024-05-21T21:51:12.850096+0000","last_became_peered":"2024-05-21T21:51:12.850096+0000","last_unstale":"2024-05-21T21:55:36.550904+0000","last_undegraded":"2024-05-21T21:55:36.550904+0000","last_fullsized":"2024-05-21T21:55:36.550904+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:43:22.611043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.554216+0000","last_change":"2024-05-21T21:51:16.942330+0000","last_active":"2024-05-21T21:55:36.554216+0000","last_peered":"2024-05-21T21:55:36.554216+0000","last_clean":"2024-05-21T21:55:36.554216+0000","last_became_active":"2024-05-21T21:51:16.941718+0000","last_became_peered":"2024-05-21T21:51:16.941718+0000","last_unstale":"2024-05-21T21:55:36.554216+0000","last_undegraded":"2024-05-21T21:55:36.554216+0000","last_fullsized":"2024-05-21T21:55:36.554216+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:54:17.176411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'5","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.352994+0000","last_change":"2024-05-21T21:51:12.941100+0000","last_active":"2024-05-21T21:55:36.352994+0000","last_peered":"2024-05-21T21:55:36.352994+0000","last_clean":"2024-05-21T21:55:36.352994+0000","last_became_active":"2024-05-21T21:51:12.940597+0000","last_became_peered":"2024-05-21T21:51:12.940597+0000","last_unstale":"2024-05-21T21:55:36.352994+0000","last_undegraded":"2024-05-21T21:55:36.352994+0000","last_fullsized":"2024-05-21T21:55:36.352994+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:30:58.457108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"82'11","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.199382+0000","last_change":"2024-05-21T21:51:16.921973+0000","last_active":"2024-05-21T21:55:36.199382+0000","last_peered":"2024-05-21T21:55:36.199382+0000","last_clean":"2024-05-21T21:55:36.199382+0000","last_became_active":"2024-05-21T21:51:16.921597+0000","last_became_peered":"2024-05-21T21:51:16.921597+0000","last_unstale":"2024-05-21T21:55:36.199382+0000","last_undegraded":"2024-05-21T21:55:36.199382+0000","last_fullsized":"2024-05-21T21:55:36.199382+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:25:32.395940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'44","reported_seq":616,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:56:01.786154+0000","last_change":"2024-05-21T21:51:12.850312+0000","last_active":"2024-05-21T21:56:01.786154+0000","last_peered":"2024-05-21T21:56:01.786154+0000","last_clean":"2024-05-21T21:56:01.786154+0000","last_became_active":"2024-05-21T21:51:12.849837+0000","last_became_peered":"2024-05-21T21:51:12.849837+0000","last_unstale":"2024-05-21T21:56:01.786154+0000","last_undegraded":"2024-05-21T21:56:01.786154+0000","last_fullsized":"2024-05-21T21:56:01.786154+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T07:09:13.339554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69052,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":542,"num_read_kb":339,"num_write":46,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"82'6","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.556609+0000","last_change":"2024-05-21T21:51:16.926625+0000","last_active":"2024-05-21T21:55:36.556609+0000","last_peered":"2024-05-21T21:55:36.556609+0000","last_clean":"2024-05-21T21:55:36.556609+0000","last_became_active":"2024-05-21T21:51:16.925889+0000","last_became_peered":"2024-05-21T21:51:16.925889+0000","last_unstale":"2024-05-21T21:55:36.556609+0000","last_undegraded":"2024-05-21T21:55:36.556609+0000","last_fullsized":"2024-05-21T21:55:36.556609+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:35:23.464747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'9","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.201314+0000","last_change":"2024-05-21T21:51:12.859772+0000","last_active":"2024-05-21T21:55:36.201314+0000","last_peered":"2024-05-21T21:55:36.201314+0000","last_clean":"2024-05-21T21:55:36.201314+0000","last_became_active":"2024-05-21T21:51:12.859290+0000","last_became_peered":"2024-05-21T21:51:12.859290+0000","last_unstale":"2024-05-21T21:55:36.201314+0000","last_undegraded":"2024-05-21T21:55:36.201314+0000","last_fullsized":"2024-05-21T21:55:36.201314+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T05:28:29.953381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"82'9","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.555763+0000","last_change":"2024-05-21T21:51:16.925803+0000","last_active":"2024-05-21T21:55:36.555763+0000","last_peered":"2024-05-21T21:55:36.555763+0000","last_clean":"2024-05-21T21:55:36.555763+0000","last_became_active":"2024-05-21T21:51:16.925111+0000","last_became_peered":"2024-05-21T21:51:16.925111+0000","last_unstale":"2024-05-21T21:55:36.555763+0000","last_undegraded":"2024-05-21T21:55:36.555763+0000","last_fullsized":"2024-05-21T21:55:36.555763+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T06:22:11.353379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'5","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.352741+0000","last_change":"2024-05-21T21:51:12.859593+0000","last_active":"2024-05-21T21:55:36.352741+0000","last_peered":"2024-05-21T21:55:36.352741+0000","last_clean":"2024-05-21T21:55:36.352741+0000","last_became_active":"2024-05-21T21:51:12.859197+0000","last_became_peered":"2024-05-21T21:51:12.859197+0000","last_unstale":"2024-05-21T21:55:36.352741+0000","last_undegraded":"2024-05-21T21:55:36.352741+0000","last_fullsized":"2024-05-21T21:55:36.352741+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:13:12.311496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"82'15","reported_seq":45,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.551077+0000","last_change":"2024-05-21T21:52:10.879028+0000","last_active":"2024-05-21T21:55:36.551077+0000","last_peered":"2024-05-21T21:55:36.551077+0000","last_clean":"2024-05-21T21:55:36.551077+0000","last_became_active":"2024-05-21T21:52:10.878664+0000","last_became_peered":"2024-05-21T21:52:10.878664+0000","last_unstale":"2024-05-21T21:55:36.551077+0000","last_undegraded":"2024-05-21T21:55:36.551077+0000","last_fullsized":"2024-05-21T21:55:36.551077+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:05:53.953546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'7","reported_seq":304,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:56:00.919342+0000","last_change":"2024-05-21T21:51:12.852169+0000","last_active":"2024-05-21T21:56:00.919342+0000","last_peered":"2024-05-21T21:56:00.919342+0000","last_clean":"2024-05-21T21:56:00.919342+0000","last_became_active":"2024-05-21T21:51:12.851782+0000","last_became_peered":"2024-05-21T21:51:12.851782+0000","last_unstale":"2024-05-21T21:56:00.919342+0000","last_undegraded":"2024-05-21T21:56:00.919342+0000","last_fullsized":"2024-05-21T21:56:00.919342+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:46:09.692158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":268,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"82'6","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.550587+0000","last_change":"2024-05-21T21:51:16.929264+0000","last_active":"2024-05-21T21:55:36.550587+0000","last_peered":"2024-05-21T21:55:36.550587+0000","last_clean":"2024-05-21T21:55:36.550587+0000","last_became_active":"2024-05-21T21:51:16.928900+0000","last_became_peered":"2024-05-21T21:51:16.928900+0000","last_unstale":"2024-05-21T21:55:36.550587+0000","last_undegraded":"2024-05-21T21:55:36.550587+0000","last_fullsized":"2024-05-21T21:55:36.550587+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:42:28.040939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'17","reported_seq":72,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.556306+0000","last_change":"2024-05-21T21:52:10.984451+0000","last_active":"2024-05-21T21:55:36.556306+0000","last_peered":"2024-05-21T21:55:36.556306+0000","last_clean":"2024-05-21T21:55:36.556306+0000","last_became_active":"2024-05-21T21:52:10.974050+0000","last_became_peered":"2024-05-21T21:52:10.974050+0000","last_unstale":"2024-05-21T21:55:36.556306+0000","last_undegraded":"2024-05-21T21:55:36.556306+0000","last_fullsized":"2024-05-21T21:55:36.556306+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T07:08:18.307022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"82'6","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.352888+0000","last_change":"2024-05-21T21:51:16.935464+0000","last_active":"2024-05-21T21:55:36.352888+0000","last_peered":"2024-05-21T21:55:36.352888+0000","last_clean":"2024-05-21T21:55:36.352888+0000","last_became_active":"2024-05-21T21:51:16.935091+0000","last_became_peered":"2024-05-21T21:51:16.935091+0000","last_unstale":"2024-05-21T21:55:36.352888+0000","last_undegraded":"2024-05-21T21:55:36.352888+0000","last_fullsized":"2024-05-21T21:55:36.352888+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T06:59:16.330649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.556757+0000","last_change":"2024-05-21T21:51:12.853327+0000","last_active":"2024-05-21T21:55:36.556757+0000","last_peered":"2024-05-21T21:55:36.556757+0000","last_clean":"2024-05-21T21:55:36.556757+0000","last_became_active":"2024-05-21T21:51:12.852946+0000","last_became_peered":"2024-05-21T21:51:12.852946+0000","last_unstale":"2024-05-21T21:55:36.556757+0000","last_undegraded":"2024-05-21T21:55:36.556757+0000","last_fullsized":"2024-05-21T21:55:36.556757+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:00:34.256858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"82'5","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.341370+0000","last_change":"2024-05-21T21:51:16.929341+0000","last_active":"2024-05-21T21:55:36.341370+0000","last_peered":"2024-05-21T21:55:36.341370+0000","last_clean":"2024-05-21T21:55:36.341370+0000","last_became_active":"2024-05-21T21:51:16.929008+0000","last_became_peered":"2024-05-21T21:51:16.929008+0000","last_unstale":"2024-05-21T21:55:36.341370+0000","last_undegraded":"2024-05-21T21:55:36.341370+0000","last_fullsized":"2024-05-21T21:55:36.341370+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:26:26.297136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"80'1","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.197640+0000","last_change":"2024-05-21T21:51:12.945433+0000","last_active":"2024-05-21T21:55:36.197640+0000","last_peered":"2024-05-21T21:55:36.197640+0000","last_clean":"2024-05-21T21:55:36.197640+0000","last_became_active":"2024-05-21T21:51:12.944463+0000","last_became_peered":"2024-05-21T21:51:12.944463+0000","last_unstale":"2024-05-21T21:55:36.197640+0000","last_undegraded":"2024-05-21T21:55:36.197640+0000","last_fullsized":"2024-05-21T21:55:36.197640+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:29:52.014583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'5","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.356192+0000","last_change":"2024-05-21T21:52:10.883554+0000","last_active":"2024-05-21T21:55:36.356192+0000","last_peered":"2024-05-21T21:55:36.356192+0000","last_clean":"2024-05-21T21:55:36.356192+0000","last_became_active":"2024-05-21T21:52:10.877789+0000","last_became_peered":"2024-05-21T21:52:10.877789+0000","last_unstale":"2024-05-21T21:55:36.356192+0000","last_undegraded":"2024-05-21T21:55:36.356192+0000","last_fullsized":"2024-05-21T21:55:36.356192+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:52:37.623551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"83'6","reported_seq":301,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:56:00.942799+0000","last_change":"2024-05-21T21:51:12.852883+0000","last_active":"2024-05-21T21:56:00.942799+0000","last_peered":"2024-05-21T21:56:00.942799+0000","last_clean":"2024-05-21T21:56:00.942799+0000","last_became_active":"2024-05-21T21:51:12.852549+0000","last_became_peered":"2024-05-21T21:51:12.852549+0000","last_unstale":"2024-05-21T21:56:00.942799+0000","last_undegraded":"2024-05-21T21:56:00.942799+0000","last_fullsized":"2024-05-21T21:56:00.942799+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:05:53.002286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":265,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"82'7","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.356089+0000","last_change":"2024-05-21T21:51:16.944425+0000","last_active":"2024-05-21T21:55:36.356089+0000","last_peered":"2024-05-21T21:55:36.356089+0000","last_clean":"2024-05-21T21:55:36.356089+0000","last_became_active":"2024-05-21T21:51:16.944033+0000","last_became_peered":"2024-05-21T21:51:16.944033+0000","last_unstale":"2024-05-21T21:55:36.356089+0000","last_undegraded":"2024-05-21T21:55:36.356089+0000","last_fullsized":"2024-05-21T21:55:36.356089+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T07:16:26.802329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"82'8","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.352742+0000","last_change":"2024-05-21T21:51:16.933264+0000","last_active":"2024-05-21T21:55:36.352742+0000","last_peered":"2024-05-21T21:55:36.352742+0000","last_clean":"2024-05-21T21:55:36.352742+0000","last_became_active":"2024-05-21T21:51:16.932911+0000","last_became_peered":"2024-05-21T21:51:16.932911+0000","last_unstale":"2024-05-21T21:55:36.352742+0000","last_undegraded":"2024-05-21T21:55:36.352742+0000","last_fullsized":"2024-05-21T21:55:36.352742+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:25:54.006367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"82'6","reported_seq":300,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:56:00.918745+0000","last_change":"2024-05-21T21:51:12.942900+0000","last_active":"2024-05-21T21:56:00.918745+0000","last_peered":"2024-05-21T21:56:00.918745+0000","last_clean":"2024-05-21T21:56:00.918745+0000","last_became_active":"2024-05-21T21:51:12.941038+0000","last_became_peered":"2024-05-21T21:51:12.941038+0000","last_unstale":"2024-05-21T21:56:00.918745+0000","last_undegraded":"2024-05-21T21:56:00.918745+0000","last_fullsized":"2024-05-21T21:56:00.918745+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:38:43.240011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":264,"num_read_kb":264,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"82'8","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.342906+0000","last_change":"2024-05-21T21:51:16.933391+0000","last_active":"2024-05-21T21:55:36.342906+0000","last_peered":"2024-05-21T21:55:36.342906+0000","last_clean":"2024-05-21T21:55:36.342906+0000","last_became_active":"2024-05-21T21:51:16.932320+0000","last_became_peered":"2024-05-21T21:51:16.932320+0000","last_unstale":"2024-05-21T21:55:36.342906+0000","last_undegraded":"2024-05-21T21:55:36.342906+0000","last_fullsized":"2024-05-21T21:55:36.342906+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T09:06:59.126748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'6","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.723215+0000","last_change":"2024-05-21T21:51:12.856342+0000","last_active":"2024-05-21T21:55:36.723215+0000","last_peered":"2024-05-21T21:55:36.723215+0000","last_clean":"2024-05-21T21:55:36.723215+0000","last_became_active":"2024-05-21T21:51:12.855952+0000","last_became_peered":"2024-05-21T21:51:12.855952+0000","last_unstale":"2024-05-21T21:55:36.723215+0000","last_undegraded":"2024-05-21T21:55:36.723215+0000","last_fullsized":"2024-05-21T21:55:36.723215+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:34:40.734034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"83'99","reported_seq":131,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:38.358271+0000","last_change":"2024-05-21T21:49:42.797187+0000","last_active":"2024-05-21T21:55:38.358271+0000","last_peered":"2024-05-21T21:55:38.358271+0000","last_clean":"2024-05-21T21:55:38.358271+0000","last_became_active":"2024-05-21T21:49:42.791291+0000","last_became_peered":"2024-05-21T21:49:42.791291+0000","last_unstale":"2024-05-21T21:55:38.358271+0000","last_undegraded":"2024-05-21T21:55:38.358271+0000","last_fullsized":"2024-05-21T21:55:38.358271+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:45:55.552293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:45:55.552293+0000","last_clean_scrub_stamp":"2024-05-21T21:45:55.552293+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T07:24:18.302523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":144,"num_read_kb":248,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"82'5","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.555133+0000","last_change":"2024-05-21T21:51:16.926446+0000","last_active":"2024-05-21T21:55:36.555133+0000","last_peered":"2024-05-21T21:55:36.555133+0000","last_clean":"2024-05-21T21:55:36.555133+0000","last_became_active":"2024-05-21T21:51:16.926070+0000","last_became_peered":"2024-05-21T21:51:16.926070+0000","last_unstale":"2024-05-21T21:55:36.555133+0000","last_undegraded":"2024-05-21T21:55:36.555133+0000","last_fullsized":"2024-05-21T21:55:36.555133+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:44:27.366393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'4","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.550735+0000","last_change":"2024-05-21T21:51:12.854371+0000","last_active":"2024-05-21T21:55:36.550735+0000","last_peered":"2024-05-21T21:55:36.550735+0000","last_clean":"2024-05-21T21:55:36.550735+0000","last_became_active":"2024-05-21T21:51:12.853847+0000","last_became_peered":"2024-05-21T21:51:12.853847+0000","last_unstale":"2024-05-21T21:55:36.550735+0000","last_undegraded":"2024-05-21T21:55:36.550735+0000","last_fullsized":"2024-05-21T21:55:36.550735+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:14:10.549901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"80'4","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.556551+0000","last_change":"2024-05-21T21:51:12.941227+0000","last_active":"2024-05-21T21:55:36.556551+0000","last_peered":"2024-05-21T21:55:36.556551+0000","last_clean":"2024-05-21T21:55:36.556551+0000","last_became_active":"2024-05-21T21:51:12.940702+0000","last_became_peered":"2024-05-21T21:51:12.940702+0000","last_unstale":"2024-05-21T21:55:36.556551+0000","last_undegraded":"2024-05-21T21:55:36.556551+0000","last_fullsized":"2024-05-21T21:55:36.556551+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:40:34.124921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"82'9","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.556301+0000","last_change":"2024-05-21T21:51:16.945627+0000","last_active":"2024-05-21T21:55:36.556301+0000","last_peered":"2024-05-21T21:55:36.556301+0000","last_clean":"2024-05-21T21:55:36.556301+0000","last_became_active":"2024-05-21T21:51:16.945297+0000","last_became_peered":"2024-05-21T21:51:16.945297+0000","last_unstale":"2024-05-21T21:55:36.556301+0000","last_undegraded":"2024-05-21T21:55:36.556301+0000","last_fullsized":"2024-05-21T21:55:36.556301+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:54:38.898774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'128","reported_seq":299,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:56:00.209196+0000","last_change":"2024-05-21T21:52:10.888076+0000","last_active":"2024-05-21T21:56:00.209196+0000","last_peered":"2024-05-21T21:56:00.209196+0000","last_clean":"2024-05-21T21:56:00.209196+0000","last_became_active":"2024-05-21T21:52:10.879368+0000","last_became_peered":"2024-05-21T21:52:10.879368+0000","last_unstale":"2024-05-21T21:56:00.209196+0000","last_undegraded":"2024-05-21T21:56:00.209196+0000","last_fullsized":"2024-05-21T21:56:00.209196+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:46:43.190411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":422,"num_read_kb":376,"num_write":236,"num_write_kb":1414,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"82'5","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.344028+0000","last_change":"2024-05-21T21:51:16.929807+0000","last_active":"2024-05-21T21:55:36.344028+0000","last_peered":"2024-05-21T21:55:36.344028+0000","last_clean":"2024-05-21T21:55:36.344028+0000","last_became_active":"2024-05-21T21:51:16.928859+0000","last_became_peered":"2024-05-21T21:51:16.928859+0000","last_unstale":"2024-05-21T21:55:36.344028+0000","last_undegraded":"2024-05-21T21:55:36.344028+0000","last_fullsized":"2024-05-21T21:55:36.344028+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:41:32.830892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'7","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.723447+0000","last_change":"2024-05-21T21:51:12.850843+0000","last_active":"2024-05-21T21:55:36.723447+0000","last_peered":"2024-05-21T21:55:36.723447+0000","last_clean":"2024-05-21T21:55:36.723447+0000","last_became_active":"2024-05-21T21:51:12.850516+0000","last_became_peered":"2024-05-21T21:51:12.850516+0000","last_unstale":"2024-05-21T21:55:36.723447+0000","last_undegraded":"2024-05-21T21:55:36.723447+0000","last_fullsized":"2024-05-21T21:55:36.723447+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:37:05.191626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"82'7","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.553781+0000","last_change":"2024-05-21T21:51:16.949017+0000","last_active":"2024-05-21T21:55:36.553781+0000","last_peered":"2024-05-21T21:55:36.553781+0000","last_clean":"2024-05-21T21:55:36.553781+0000","last_became_active":"2024-05-21T21:51:16.948691+0000","last_became_peered":"2024-05-21T21:51:16.948691+0000","last_unstale":"2024-05-21T21:55:36.553781+0000","last_undegraded":"2024-05-21T21:55:36.553781+0000","last_fullsized":"2024-05-21T21:55:36.553781+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:32:06.606581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.199191+0000","last_change":"2024-05-21T21:51:12.939203+0000","last_active":"2024-05-21T21:55:36.199191+0000","last_peered":"2024-05-21T21:55:36.199191+0000","last_clean":"2024-05-21T21:55:36.199191+0000","last_became_active":"2024-05-21T21:51:12.938936+0000","last_became_peered":"2024-05-21T21:51:12.938936+0000","last_unstale":"2024-05-21T21:55:36.199191+0000","last_undegraded":"2024-05-21T21:55:36.199191+0000","last_fullsized":"2024-05-21T21:55:36.199191+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:55:50.336362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"82'2","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.556315+0000","last_change":"2024-05-21T21:51:16.926216+0000","last_active":"2024-05-21T21:55:36.556315+0000","last_peered":"2024-05-21T21:55:36.556315+0000","last_clean":"2024-05-21T21:55:36.556315+0000","last_became_active":"2024-05-21T21:51:16.925834+0000","last_became_peered":"2024-05-21T21:51:16.925834+0000","last_unstale":"2024-05-21T21:55:36.556315+0000","last_undegraded":"2024-05-21T21:55:36.556315+0000","last_fullsized":"2024-05-21T21:55:36.556315+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:13:30.192908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'7","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.342741+0000","last_change":"2024-05-21T21:51:12.855306+0000","last_active":"2024-05-21T21:55:36.342741+0000","last_peered":"2024-05-21T21:55:36.342741+0000","last_clean":"2024-05-21T21:55:36.342741+0000","last_became_active":"2024-05-21T21:51:12.854722+0000","last_became_peered":"2024-05-21T21:51:12.854722+0000","last_unstale":"2024-05-21T21:55:36.342741+0000","last_undegraded":"2024-05-21T21:55:36.342741+0000","last_fullsized":"2024-05-21T21:55:36.342741+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:39:41.583219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"82'9","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.341026+0000","last_change":"2024-05-21T21:51:16.944758+0000","last_active":"2024-05-21T21:55:36.341026+0000","last_peered":"2024-05-21T21:55:36.341026+0000","last_clean":"2024-05-21T21:55:36.341026+0000","last_became_active":"2024-05-21T21:51:16.944393+0000","last_became_peered":"2024-05-21T21:51:16.944393+0000","last_unstale":"2024-05-21T21:55:36.341026+0000","last_undegraded":"2024-05-21T21:55:36.341026+0000","last_fullsized":"2024-05-21T21:55:36.341026+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:01:23.048355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'4","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:37.227713+0000","last_change":"2024-05-21T21:51:12.860378+0000","last_active":"2024-05-21T21:55:37.227713+0000","last_peered":"2024-05-21T21:55:37.227713+0000","last_clean":"2024-05-21T21:55:37.227713+0000","last_became_active":"2024-05-21T21:51:12.859978+0000","last_became_peered":"2024-05-21T21:51:12.859978+0000","last_unstale":"2024-05-21T21:55:37.227713+0000","last_undegraded":"2024-05-21T21:55:37.227713+0000","last_fullsized":"2024-05-21T21:55:37.227713+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T07:52:26.902232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"82'8","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.343395+0000","last_change":"2024-05-21T21:51:16.943194+0000","last_active":"2024-05-21T21:55:36.343395+0000","last_peered":"2024-05-21T21:55:36.343395+0000","last_clean":"2024-05-21T21:55:36.343395+0000","last_became_active":"2024-05-21T21:51:16.942884+0000","last_became_peered":"2024-05-21T21:51:16.942884+0000","last_unstale":"2024-05-21T21:55:36.343395+0000","last_undegraded":"2024-05-21T21:55:36.343395+0000","last_fullsized":"2024-05-21T21:55:36.343395+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T05:58:17.436995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.556514+0000","last_change":"2024-05-21T21:51:12.853390+0000","last_active":"2024-05-21T21:55:36.556514+0000","last_peered":"2024-05-21T21:55:36.556514+0000","last_clean":"2024-05-21T21:55:36.556514+0000","last_became_active":"2024-05-21T21:51:12.852990+0000","last_became_peered":"2024-05-21T21:51:12.852990+0000","last_unstale":"2024-05-21T21:55:36.556514+0000","last_undegraded":"2024-05-21T21:55:36.556514+0000","last_fullsized":"2024-05-21T21:55:36.556514+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:32:16.392609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"82'5","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.550848+0000","last_change":"2024-05-21T21:51:16.933430+0000","last_active":"2024-05-21T21:55:36.550848+0000","last_peered":"2024-05-21T21:55:36.550848+0000","last_clean":"2024-05-21T21:55:36.550848+0000","last_became_active":"2024-05-21T21:51:16.933053+0000","last_became_peered":"2024-05-21T21:51:16.933053+0000","last_unstale":"2024-05-21T21:55:36.550848+0000","last_undegraded":"2024-05-21T21:55:36.550848+0000","last_fullsized":"2024-05-21T21:55:36.550848+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T06:14:57.468727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'8","reported_seq":305,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:56:00.785171+0000","last_change":"2024-05-21T21:51:12.946113+0000","last_active":"2024-05-21T21:56:00.785171+0000","last_peered":"2024-05-21T21:56:00.785171+0000","last_clean":"2024-05-21T21:56:00.785171+0000","last_became_active":"2024-05-21T21:51:12.945650+0000","last_became_peered":"2024-05-21T21:51:12.945650+0000","last_unstale":"2024-05-21T21:56:00.785171+0000","last_undegraded":"2024-05-21T21:56:00.785171+0000","last_fullsized":"2024-05-21T21:56:00.785171+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T09:40:18.209400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2343,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":267,"num_read_kb":4,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"82'7","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.550939+0000","last_change":"2024-05-21T21:51:16.932632+0000","last_active":"2024-05-21T21:55:36.550939+0000","last_peered":"2024-05-21T21:55:36.550939+0000","last_clean":"2024-05-21T21:55:36.550939+0000","last_became_active":"2024-05-21T21:51:16.932256+0000","last_became_peered":"2024-05-21T21:51:16.932256+0000","last_unstale":"2024-05-21T21:55:36.550939+0000","last_undegraded":"2024-05-21T21:55:36.550939+0000","last_fullsized":"2024-05-21T21:55:36.550939+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:04:48.146485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'75","reported_seq":147,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:56:00.196496+0000","last_change":"2024-05-21T21:51:12.860602+0000","last_active":"2024-05-21T21:56:00.196496+0000","last_peered":"2024-05-21T21:56:00.196496+0000","last_clean":"2024-05-21T21:56:00.196496+0000","last_became_active":"2024-05-21T21:51:12.860228+0000","last_became_peered":"2024-05-21T21:51:12.860228+0000","last_unstale":"2024-05-21T21:56:00.196496+0000","last_undegraded":"2024-05-21T21:56:00.196496+0000","last_fullsized":"2024-05-21T21:56:00.196496+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:33:39.230872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":45,"num_read_kb":44,"num_write":84,"num_write_kb":1075,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'7","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.551083+0000","last_change":"2024-05-21T21:51:16.929616+0000","last_active":"2024-05-21T21:55:36.551083+0000","last_peered":"2024-05-21T21:55:36.551083+0000","last_clean":"2024-05-21T21:55:36.551083+0000","last_became_active":"2024-05-21T21:51:16.928569+0000","last_became_peered":"2024-05-21T21:51:16.928569+0000","last_unstale":"2024-05-21T21:55:36.551083+0000","last_undegraded":"2024-05-21T21:55:36.551083+0000","last_fullsized":"2024-05-21T21:55:36.551083+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:15:41.790344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'5","reported_seq":47,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.555560+0000","last_change":"2024-05-21T21:52:10.881114+0000","last_active":"2024-05-21T21:55:36.555560+0000","last_peered":"2024-05-21T21:55:36.555560+0000","last_clean":"2024-05-21T21:55:36.555560+0000","last_became_active":"2024-05-21T21:52:10.872022+0000","last_became_peered":"2024-05-21T21:52:10.872022+0000","last_unstale":"2024-05-21T21:55:36.555560+0000","last_undegraded":"2024-05-21T21:55:36.555560+0000","last_fullsized":"2024-05-21T21:55:36.555560+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:11.816899+0000","last_clean_scrub_stamp":"2024-05-21T21:51:11.816899+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T06:05:51.419156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"82'9","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.550977+0000","last_change":"2024-05-21T21:51:16.926496+0000","last_active":"2024-05-21T21:55:36.550977+0000","last_peered":"2024-05-21T21:55:36.550977+0000","last_clean":"2024-05-21T21:55:36.550977+0000","last_became_active":"2024-05-21T21:51:16.926112+0000","last_became_peered":"2024-05-21T21:51:16.926112+0000","last_unstale":"2024-05-21T21:55:36.550977+0000","last_undegraded":"2024-05-21T21:55:36.550977+0000","last_fullsized":"2024-05-21T21:55:36.550977+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:26:08.254421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"82'9","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.341047+0000","last_change":"2024-05-21T21:51:16.930522+0000","last_active":"2024-05-21T21:55:36.341047+0000","last_peered":"2024-05-21T21:55:36.341047+0000","last_clean":"2024-05-21T21:55:36.341047+0000","last_became_active":"2024-05-21T21:51:16.929853+0000","last_became_peered":"2024-05-21T21:51:16.929853+0000","last_unstale":"2024-05-21T21:55:36.341047+0000","last_undegraded":"2024-05-21T21:55:36.341047+0000","last_fullsized":"2024-05-21T21:55:36.341047+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T08:10:57.940009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"82'12","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.201162+0000","last_change":"2024-05-21T21:51:16.931751+0000","last_active":"2024-05-21T21:55:36.201162+0000","last_peered":"2024-05-21T21:55:36.201162+0000","last_clean":"2024-05-21T21:55:36.201162+0000","last_became_active":"2024-05-21T21:51:16.931353+0000","last_became_peered":"2024-05-21T21:51:16.931353+0000","last_unstale":"2024-05-21T21:55:36.201162+0000","last_undegraded":"2024-05-21T21:55:36.201162+0000","last_fullsized":"2024-05-21T21:55:36.201162+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:33:45.297850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"82'8","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.201127+0000","last_change":"2024-05-21T21:51:16.940096+0000","last_active":"2024-05-21T21:55:36.201127+0000","last_peered":"2024-05-21T21:55:36.201127+0000","last_clean":"2024-05-21T21:55:36.201127+0000","last_became_active":"2024-05-21T21:51:16.939740+0000","last_became_peered":"2024-05-21T21:51:16.939740+0000","last_unstale":"2024-05-21T21:55:36.201127+0000","last_undegraded":"2024-05-21T21:55:36.201127+0000","last_fullsized":"2024-05-21T21:55:36.201127+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:59:52.569526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"82'7","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.552155+0000","last_change":"2024-05-21T21:51:16.946488+0000","last_active":"2024-05-21T21:55:36.552155+0000","last_peered":"2024-05-21T21:55:36.552155+0000","last_clean":"2024-05-21T21:55:36.552155+0000","last_became_active":"2024-05-21T21:51:16.945957+0000","last_became_peered":"2024-05-21T21:51:16.945957+0000","last_unstale":"2024-05-21T21:55:36.552155+0000","last_undegraded":"2024-05-21T21:55:36.552155+0000","last_fullsized":"2024-05-21T21:55:36.552155+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:40:36.708344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"82'4","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.556177+0000","last_change":"2024-05-21T21:51:16.949349+0000","last_active":"2024-05-21T21:55:36.556177+0000","last_peered":"2024-05-21T21:55:36.556177+0000","last_clean":"2024-05-21T21:55:36.556177+0000","last_became_active":"2024-05-21T21:51:16.946986+0000","last_became_peered":"2024-05-21T21:51:16.946986+0000","last_unstale":"2024-05-21T21:55:36.556177+0000","last_undegraded":"2024-05-21T21:55:36.556177+0000","last_fullsized":"2024-05-21T21:55:36.556177+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:01:14.877110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"82'6","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.341365+0000","last_change":"2024-05-21T21:51:16.929172+0000","last_active":"2024-05-21T21:55:36.341365+0000","last_peered":"2024-05-21T21:55:36.341365+0000","last_clean":"2024-05-21T21:55:36.341365+0000","last_became_active":"2024-05-21T21:51:16.928823+0000","last_became_peered":"2024-05-21T21:51:16.928823+0000","last_unstale":"2024-05-21T21:55:36.341365+0000","last_undegraded":"2024-05-21T21:55:36.341365+0000","last_fullsized":"2024-05-21T21:55:36.341365+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:09:13.719272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"82'15","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.201520+0000","last_change":"2024-05-21T21:51:16.930332+0000","last_active":"2024-05-21T21:55:36.201520+0000","last_peered":"2024-05-21T21:55:36.201520+0000","last_clean":"2024-05-21T21:55:36.201520+0000","last_became_active":"2024-05-21T21:51:16.930024+0000","last_became_peered":"2024-05-21T21:51:16.930024+0000","last_unstale":"2024-05-21T21:55:36.201520+0000","last_undegraded":"2024-05-21T21:55:36.201520+0000","last_fullsized":"2024-05-21T21:55:36.201520+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:47:41.272725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"82'2","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.343903+0000","last_change":"2024-05-21T21:51:16.930102+0000","last_active":"2024-05-21T21:55:36.343903+0000","last_peered":"2024-05-21T21:55:36.343903+0000","last_clean":"2024-05-21T21:55:36.343903+0000","last_became_active":"2024-05-21T21:51:16.929410+0000","last_became_peered":"2024-05-21T21:51:16.929410+0000","last_unstale":"2024-05-21T21:55:36.343903+0000","last_undegraded":"2024-05-21T21:55:36.343903+0000","last_fullsized":"2024-05-21T21:55:36.343903+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:11:46.296288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"82'6","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.556235+0000","last_change":"2024-05-21T21:52:10.873424+0000","last_active":"2024-05-21T21:55:36.556235+0000","last_peered":"2024-05-21T21:55:36.556235+0000","last_clean":"2024-05-21T21:55:36.556235+0000","last_became_active":"2024-05-21T21:52:10.873118+0000","last_became_peered":"2024-05-21T21:52:10.873118+0000","last_unstale":"2024-05-21T21:55:36.556235+0000","last_undegraded":"2024-05-21T21:55:36.556235+0000","last_fullsized":"2024-05-21T21:55:36.556235+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:05:00.731013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,3],"acting":[8,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.29","version":"82'8","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.342675+0000","last_change":"2024-05-21T21:51:16.934442+0000","last_active":"2024-05-21T21:55:36.342675+0000","last_peered":"2024-05-21T21:55:36.342675+0000","last_clean":"2024-05-21T21:55:36.342675+0000","last_became_active":"2024-05-21T21:51:16.934086+0000","last_became_peered":"2024-05-21T21:51:16.934086+0000","last_unstale":"2024-05-21T21:55:36.342675+0000","last_undegraded":"2024-05-21T21:55:36.342675+0000","last_fullsized":"2024-05-21T21:55:36.342675+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:36:44.302532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"82'10","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.199462+0000","last_change":"2024-05-21T21:51:16.924479+0000","last_active":"2024-05-21T21:55:36.199462+0000","last_peered":"2024-05-21T21:55:36.199462+0000","last_clean":"2024-05-21T21:55:36.199462+0000","last_became_active":"2024-05-21T21:51:16.923998+0000","last_became_peered":"2024-05-21T21:51:16.923998+0000","last_unstale":"2024-05-21T21:55:36.199462+0000","last_undegraded":"2024-05-21T21:55:36.199462+0000","last_fullsized":"2024-05-21T21:55:36.199462+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:23:07.931731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"82'11","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.555830+0000","last_change":"2024-05-21T21:51:16.947271+0000","last_active":"2024-05-21T21:55:36.555830+0000","last_peered":"2024-05-21T21:55:36.555830+0000","last_clean":"2024-05-21T21:55:36.555830+0000","last_became_active":"2024-05-21T21:51:16.946912+0000","last_became_peered":"2024-05-21T21:51:16.946912+0000","last_unstale":"2024-05-21T21:55:36.555830+0000","last_undegraded":"2024-05-21T21:55:36.555830+0000","last_fullsized":"2024-05-21T21:55:36.555830+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:51:29.646915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"82'8","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-21T21:55:36.341091+0000","last_change":"2024-05-21T21:51:16.928852+0000","last_active":"2024-05-21T21:55:36.341091+0000","last_peered":"2024-05-21T21:55:36.341091+0000","last_clean":"2024-05-21T21:55:36.341091+0000","last_became_active":"2024-05-21T21:51:16.928480+0000","last_became_peered":"2024-05-21T21:51:16.928480+0000","last_unstale":"2024-05-21T21:55:36.341091+0000","last_undegraded":"2024-05-21T21:55:36.341091+0000","last_fullsized":"2024-05-21T21:55:36.341091+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T21:51:15.874318+0000","last_clean_scrub_stamp":"2024-05-21T21:51:15.874318+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T09:12:30.691254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":144,"num_read_kb":248,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":800642,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":3151,"num_read_kb":1849,"num_write":570,"num_write_kb":2655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":112,"num_keys_recovered":17,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2764800,"data_stored":2401926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":426,"ondisk_log_size":426,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645386,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28296,"kb_used_data":1280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716856,"statfs":{"total":95995035648,"available":95966060544,"internally_reserved":0,"allocated":1310720,"data_stored":837304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841614,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":29048,"kb_used_data":1912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716104,"statfs":{"total":95995035648,"available":95965290496,"internally_reserved":0,"allocated":1957888,"data_stored":1431760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005139,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28324,"kb_used_data":1308,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716828,"statfs":{"total":95995035648,"available":95966031872,"internally_reserved":0,"allocated":1339392,"data_stored":836178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201370,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":937984,"data_stored":443223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397598,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28516,"kb_used_data":1500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716636,"statfs":{"total":95995035648,"available":95965835264,"internally_reserved":0,"allocated":1536000,"data_stored":1037897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561124,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":925696,"data_stored":442953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574976,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":513030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738501,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36264,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708888,"statfs":{"total":95995035648,"available":95957901312,"internally_reserved":0,"allocated":946176,"data_stored":445074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411452,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36320,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708832,"statfs":{"total":95995035648,"available":95957843968,"internally_reserved":0,"allocated":1007616,"data_stored":515432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149812,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37080,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708072,"statfs":{"total":95995035648,"available":95957065728,"internally_reserved":0,"allocated":1662976,"data_stored":1163718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953583,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":925696,"data_stored":442955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757354,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":33132,"kb_used_data":1836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712020,"statfs":{"total":95995035648,"available":95961108480,"internally_reserved":0,"allocated":1880064,"data_stored":1364884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":71307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":73709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":753664,"data_stored":721878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":332676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":399552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":395581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-21T21:56:06.897 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-21T21:56:07.382 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:56:07.863 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:56:07 smithi151 ceph-mon[31623]: from='client.25349 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T21:56:07.863 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:56:07 smithi151 ceph-mon[31623]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 99 KiB/s wr, 55 op/s 2024-05-21T21:56:07.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:56:07 smithi184 ceph-mon[36942]: from='client.25349 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T21:56:07.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:56:07 smithi184 ceph-mon[36942]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 99 KiB/s wr, 55 op/s 2024-05-21T21:56:08.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:56:07 smithi164 ceph-mon[36269]: from='client.25349 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-21T21:56:08.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:56:07 smithi164 ceph-mon[36269]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 99 KiB/s wr, 55 op/s 2024-05-21T21:56:09.655 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:09.654+0000 7f83ae600700 1 -- 172.21.15.151:0/596872186 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f83a8076090 msgr2=0x7f83a810fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:09.655 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:09.654+0000 7f83ae600700 1 --2- 172.21.15.151:0/596872186 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f83a8076090 0x7f83a810fa30 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f8394009990 tx=0x7f8394009ca0 comp rx=0 tx=0).stop 2024-05-21T21:56:09.655 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:09.654+0000 7f83ae600700 1 -- 172.21.15.151:0/596872186 shutdown_connections 2024-05-21T21:56:09.655 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:09.654+0000 7f83ae600700 1 --2- 172.21.15.151:0/596872186 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f83a810ff70 0x7f83a8112360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:09.655 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:09.654+0000 7f83ae600700 1 --2- 172.21.15.151:0/596872186 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f83a8075770 0x7f83a8075b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:09.655 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:09.654+0000 7f83ae600700 1 --2- 172.21.15.151:0/596872186 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f83a8076090 0x7f83a810fa30 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:09.655 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:09.654+0000 7f83ae600700 1 -- 172.21.15.151:0/596872186 >> 172.21.15.151:0/596872186 conn(0x7f83a807a6f0 msgr2=0x7f83a807ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:56:09.655 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:09.654+0000 7f83ae600700 1 -- 172.21.15.151:0/596872186 shutdown_connections 2024-05-21T21:56:09.655 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:09.654+0000 7f83ae600700 1 -- 172.21.15.151:0/596872186 wait complete. 2024-05-21T21:56:09.656 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:09.655+0000 7f83ae600700 1 Processor -- start 2024-05-21T21:56:09.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:09.655+0000 7f83ae600700 1 -- start start 2024-05-21T21:56:09.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:09.656+0000 7f83ae600700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f83a8075770 0x7f83a819bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:09.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:09.656+0000 7f83ae600700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f83a8076090 0x7f83a819c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:09.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:09.656+0000 7f83ae600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f83a810ff70 0x7f83a81a07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:09.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:09.656+0000 7f83ae600700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f83a81159b0 con 0x7f83a810ff70 2024-05-21T21:56:09.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:09.656+0000 7f83ae600700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f83a8115830 con 0x7f83a8076090 2024-05-21T21:56:09.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:09.656+0000 7f83ae600700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f83a8115b30 con 0x7f83a8075770 2024-05-21T21:56:09.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:09.656+0000 7f83a7e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f83a810ff70 0x7f83a81a07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:09.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:09.656+0000 7f83a6a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f83a8076090 0x7f83a819c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:09.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:09.656+0000 7f83a6a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f83a8076090 0x7f83a819c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:43946/0 (socket says 172.21.15.151:43946) 2024-05-21T21:56:09.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:09.656+0000 7f83a6a00700 1 -- 172.21.15.151:0/3985151083 learned_addr learned my addr 172.21.15.151:0/3985151083 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:56:09.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:09.656+0000 7f83a7400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f83a8075770 0x7f83a819bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:09.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:09.657+0000 7f83a6a00700 1 -- 172.21.15.151:0/3985151083 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f83a8075770 msgr2=0x7f83a819bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:09.658 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:09.657+0000 7f83a6a00700 1 --2- 172.21.15.151:0/3985151083 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f83a8075770 0x7f83a819bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:09.658 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:09.657+0000 7f83a6a00700 1 -- 172.21.15.151:0/3985151083 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f83a810ff70 msgr2=0x7f83a81a07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:09.658 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:09.657+0000 7f83a6a00700 1 --2- 172.21.15.151:0/3985151083 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f83a810ff70 0x7f83a81a07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:09.658 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:09.657+0000 7f83a6a00700 1 -- 172.21.15.151:0/3985151083 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83a81a0f00 con 0x7f83a8076090 2024-05-21T21:56:09.658 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:09.657+0000 7f83a7e00700 1 --2- 172.21.15.151:0/3985151083 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f83a810ff70 0x7f83a81a07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:56:09.659 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:09.657+0000 7f83a6a00700 1 --2- 172.21.15.151:0/3985151083 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f83a8076090 0x7f83a819c480 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8394005910 tx=0x7f839400b670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:56:09.659 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:09.657+0000 7f839fe00700 1 -- 172.21.15.151:0/3985151083 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8394027070 con 0x7f83a8076090 2024-05-21T21:56:09.659 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:09.657+0000 7f839fe00700 1 -- 172.21.15.151:0/3985151083 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8394022470 con 0x7f83a8076090 2024-05-21T21:56:09.659 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:09.657+0000 7f83ae600700 1 -- 172.21.15.151:0/3985151083 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83a81a1190 con 0x7f83a8076090 2024-05-21T21:56:09.659 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:09.657+0000 7f839fe00700 1 -- 172.21.15.151:0/3985151083 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8394017630 con 0x7f83a8076090 2024-05-21T21:56:09.659 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:09.657+0000 7f83ae600700 1 -- 172.21.15.151:0/3985151083 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83a81a8a70 con 0x7f83a8076090 2024-05-21T21:56:09.659 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:09.658+0000 7f83ae600700 1 -- 172.21.15.151:0/3985151083 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83a806a230 con 0x7f83a8076090 2024-05-21T21:56:09.660 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:09.660+0000 7f839fe00700 1 -- 172.21.15.151:0/3985151083 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 149886+0+0 (secure 0 0 0) 0x7f839402a070 con 0x7f83a8076090 2024-05-21T21:56:09.662 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:09.662+0000 7f839fe00700 1 --2- 172.21.15.151:0/3985151083 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f838c0b2b90 0x7f838c0b5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:09.662 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:09.662+0000 7f839fe00700 1 -- 172.21.15.151:0/3985151083 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f83940e34c0 con 0x7f83a8076090 2024-05-21T21:56:09.662 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:09.662+0000 7f83a7400700 1 --2- 172.21.15.151:0/3985151083 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f838c0b2b90 0x7f838c0b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:09.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:09.663+0000 7f83a7400700 1 --2- 172.21.15.151:0/3985151083 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f838c0b2b90 0x7f838c0b5050 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8398009c60 tx=0x7f8398009260 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:56:09.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:09.667+0000 7f839fe00700 1 -- 172.21.15.151:0/3985151083 <== mon.2 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8394013070 con 0x7f83a8076090 2024-05-21T21:56:09.838 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:56:09 smithi164 ceph-mon[36269]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 99 KiB/s wr, 60 op/s 2024-05-21T21:56:09.946 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:09.945+0000 7f83ae600700 1 -- 172.21.15.151:0/3985151083 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f83a802e560 con 0x7f83a8076090 2024-05-21T21:56:09.946 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:56:09 smithi151 ceph-mon[31623]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 99 KiB/s wr, 60 op/s 2024-05-21T21:56:09.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:09.947+0000 7f839fe00700 1 -- 172.21.15.151:0/3985151083 <== mon.2 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11958 (secure 0 0 0) 0x7f83940aac90 con 0x7f83a8076090 2024-05-21T21:56:09.948 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T21:56:09.948 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14358,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/158010529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":158010529},{"type":"v1","addr":"172.21.15.151:6835","nonce":158010529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6839/2069058115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":2069058115},{"type":"v1","addr":"172.21.15.151:6839","nonce":2069058115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T21:51:19.552069+0000","modified":"2024-05-21T21:55:34.643755+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24530,"mds_1":24493,"mds_2":24491,"mds_3":24478,"mds_4":24473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14403":{"gid":14403,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6841/1749723794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6840","nonce":1749723794},{"type":"v1","addr":"172.21.15.151:6841","nonce":1749723794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/2026266114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":2026266114},{"type":"v1","addr":"172.21.15.184:6833","nonce":2026266114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/2467900844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":2467900844},{"type":"v1","addr":"172.21.15.164:6833","nonce":2467900844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3367640960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3367640960},{"type":"v1","addr":"172.21.15.151:6837","nonce":3367640960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/3264142621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3264142621},{"type":"v1","addr":"172.21.15.184:6835","nonce":3264142621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":17,"addr":"172.21.15.164:6835/2340428470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2340428470},{"type":"v1","addr":"172.21.15.164:6835","nonce":2340428470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":16,"addr":"172.21.15.184:6837/2988194888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":2988194888},{"type":"v1","addr":"172.21.15.184:6837","nonce":2988194888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.164:6837/1554609572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1554609572},{"type":"v1","addr":"172.21.15.164:6837","nonce":1554609572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.184:6839/3059935143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":3059935143},{"type":"v1","addr":"172.21.15.184:6839","nonce":3059935143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.164:6839/2646401247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":2646401247},{"type":"v1","addr":"172.21.15.164:6839","nonce":2646401247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24530,"qdb_cluster":[24530,24493,24491,24473,24478]},"id":1}]} 2024-05-21T21:56:09.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:09.950+0000 7f83ae600700 1 -- 172.21.15.151:0/3985151083 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f838c0b2b90 msgr2=0x7f838c0b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:09.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:09.950+0000 7f83ae600700 1 --2- 172.21.15.151:0/3985151083 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f838c0b2b90 0x7f838c0b5050 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8398009c60 tx=0x7f8398009260 comp rx=0 tx=0).stop 2024-05-21T21:56:09.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:09.950+0000 7f83ae600700 1 -- 172.21.15.151:0/3985151083 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f83a8076090 msgr2=0x7f83a819c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:09.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:09.950+0000 7f83ae600700 1 --2- 172.21.15.151:0/3985151083 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f83a8076090 0x7f83a819c480 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8394005910 tx=0x7f839400b670 comp rx=0 tx=0).stop 2024-05-21T21:56:09.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:09.950+0000 7f83ae600700 1 -- 172.21.15.151:0/3985151083 shutdown_connections 2024-05-21T21:56:09.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:09.950+0000 7f83ae600700 1 --2- 172.21.15.151:0/3985151083 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f838c0b2b90 0x7f838c0b5050 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:09.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:09.950+0000 7f83ae600700 1 --2- 172.21.15.151:0/3985151083 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f83a810ff70 0x7f83a81a07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:09.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:09.950+0000 7f83ae600700 1 --2- 172.21.15.151:0/3985151083 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f83a8075770 0x7f83a819bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:09.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:09.950+0000 7f83ae600700 1 --2- 172.21.15.151:0/3985151083 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f83a8076090 0x7f83a819c480 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:09.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:09.950+0000 7f83ae600700 1 -- 172.21.15.151:0/3985151083 >> 172.21.15.151:0/3985151083 conn(0x7f83a807a6f0 msgr2=0x7f83a8100820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:56:09.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:09.950+0000 7f83ae600700 1 -- 172.21.15.151:0/3985151083 shutdown_connections 2024-05-21T21:56:09.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:09.950+0000 7f83ae600700 1 -- 172.21.15.151:0/3985151083 wait complete. 2024-05-21T21:56:09.952 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 29 2024-05-21T21:56:09.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:56:09 smithi184 ceph-mon[36942]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 99 KiB/s wr, 60 op/s 2024-05-21T21:56:10.687 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:56:10 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/3985151083' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T21:56:10.753 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-21T21:56:10.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:56:10 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/3985151083' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T21:56:11.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:56:10 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/3985151083' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T21:56:11.199 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:56:11.815 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:56:11 smithi151 ceph-mon[31623]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-05-21T21:56:11.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:56:11 smithi184 ceph-mon[36942]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-05-21T21:56:12.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:56:11 smithi164 ceph-mon[36269]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-05-21T21:56:13.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:13.459+0000 7f2c4f200700 1 -- 172.21.15.151:0/2132378570 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c50106030 msgr2=0x7f2c50112540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:13.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:13.459+0000 7f2c4f200700 1 --2- 172.21.15.151:0/2132378570 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c50106030 0x7f2c50112540 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2c4800b0a0 tx=0x7f2c4800b3b0 comp rx=0 tx=0).stop 2024-05-21T21:56:13.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:13.459+0000 7f2c4f200700 1 -- 172.21.15.151:0/2132378570 shutdown_connections 2024-05-21T21:56:13.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:13.459+0000 7f2c4f200700 1 --2- 172.21.15.151:0/2132378570 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2c5010b670 0x7f2c5010ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:13.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:13.459+0000 7f2c4f200700 1 --2- 172.21.15.151:0/2132378570 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c50106030 0x7f2c50112540 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:13.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:13.459+0000 7f2c4f200700 1 --2- 172.21.15.151:0/2132378570 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2c50105670 0x7f2c50105af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:13.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:13.459+0000 7f2c4f200700 1 -- 172.21.15.151:0/2132378570 >> 172.21.15.151:0/2132378570 conn(0x7f2c50101160 msgr2=0x7f2c50103580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:56:13.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:13.459+0000 7f2c4f200700 1 -- 172.21.15.151:0/2132378570 shutdown_connections 2024-05-21T21:56:13.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:13.459+0000 7f2c4f200700 1 -- 172.21.15.151:0/2132378570 wait complete. 2024-05-21T21:56:13.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:13.460+0000 7f2c4f200700 1 Processor -- start 2024-05-21T21:56:13.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:13.460+0000 7f2c4f200700 1 -- start start 2024-05-21T21:56:13.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:13.461+0000 7f2c4f200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c50105670 0x7f2c5019bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:13.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:13.461+0000 7f2c4f200700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2c50106030 0x7f2c5019c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:13.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:13.461+0000 7f2c4f200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2c5010b670 0x7f2c501a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:13.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:13.461+0000 7f2c4f200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2c501158c0 con 0x7f2c50105670 2024-05-21T21:56:13.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:13.461+0000 7f2c4f200700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2c50115740 con 0x7f2c5010b670 2024-05-21T21:56:13.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:13.461+0000 7f2c4f200700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2c50115a40 con 0x7f2c50106030 2024-05-21T21:56:13.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:13.461+0000 7f2c4de00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c50105670 0x7f2c5019bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:13.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:13.461+0000 7f2c4d400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2c50106030 0x7f2c5019c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:13.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:13.461+0000 7f2c4d400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2c50106030 0x7f2c5019c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:52274/0 (socket says 172.21.15.151:52274) 2024-05-21T21:56:13.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:13.461+0000 7f2c4d400700 1 -- 172.21.15.151:0/2490690730 learned_addr learned my addr 172.21.15.151:0/2490690730 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:56:13.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:13.461+0000 7f2c4e800700 1 --2- 172.21.15.151:0/2490690730 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2c5010b670 0x7f2c501a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:13.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:13.462+0000 7f2c4e800700 1 -- 172.21.15.151:0/2490690730 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2c50106030 msgr2=0x7f2c5019c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:13.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:13.462+0000 7f2c4e800700 1 --2- 172.21.15.151:0/2490690730 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2c50106030 0x7f2c5019c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:13.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:13.462+0000 7f2c4e800700 1 -- 172.21.15.151:0/2490690730 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c50105670 msgr2=0x7f2c5019bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:13.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:13.462+0000 7f2c4e800700 1 --2- 172.21.15.151:0/2490690730 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c50105670 0x7f2c5019bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:13.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:13.462+0000 7f2c4e800700 1 -- 172.21.15.151:0/2490690730 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c501a0d20 con 0x7f2c5010b670 2024-05-21T21:56:13.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:13.462+0000 7f2c4de00700 1 --2- 172.21.15.151:0/2490690730 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c50105670 0x7f2c5019bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:56:13.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:13.462+0000 7f2c4e800700 1 --2- 172.21.15.151:0/2490690730 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2c5010b670 0x7f2c501a05e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2c4800b070 tx=0x7f2c48004c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:56:13.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:13.463+0000 7f2c46a00700 1 -- 172.21.15.151:0/2490690730 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c48021070 con 0x7f2c5010b670 2024-05-21T21:56:13.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:13.463+0000 7f2c4f200700 1 -- 172.21.15.151:0/2490690730 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c501a0fb0 con 0x7f2c5010b670 2024-05-21T21:56:13.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:13.463+0000 7f2c46a00700 1 -- 172.21.15.151:0/2490690730 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c48007e30 con 0x7f2c5010b670 2024-05-21T21:56:13.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:13.463+0000 7f2c46a00700 1 -- 172.21.15.151:0/2490690730 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c480045f0 con 0x7f2c5010b670 2024-05-21T21:56:13.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:13.463+0000 7f2c4f200700 1 -- 172.21.15.151:0/2490690730 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c501a8890 con 0x7f2c5010b670 2024-05-21T21:56:13.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:13.464+0000 7f2c4f200700 1 -- 172.21.15.151:0/2490690730 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c5006a230 con 0x7f2c5010b670 2024-05-21T21:56:13.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:13.466+0000 7f2c46a00700 1 -- 172.21.15.151:0/2490690730 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 149886+0+0 (secure 0 0 0) 0x7f2c48007980 con 0x7f2c5010b670 2024-05-21T21:56:13.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:13.468+0000 7f2c46a00700 1 --2- 172.21.15.151:0/2490690730 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f2c340b2b10 0x7f2c340b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:13.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:13.468+0000 7f2c46a00700 1 -- 172.21.15.151:0/2490690730 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f2c4800f020 con 0x7f2c5010b670 2024-05-21T21:56:13.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:13.468+0000 7f2c4de00700 1 --2- 172.21.15.151:0/2490690730 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f2c340b2b10 0x7f2c340b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:13.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:13.469+0000 7f2c4de00700 1 --2- 172.21.15.151:0/2490690730 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f2c340b2b10 0x7f2c340b4fd0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2c40009fd0 tx=0x7f2c40009260 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:56:13.470 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:13.470+0000 7f2c46a00700 1 -- 172.21.15.151:0/2490690730 <== mon.2 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2c4802a020 con 0x7f2c5010b670 2024-05-21T21:56:13.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:13.752+0000 7f2c4f200700 1 -- 172.21.15.151:0/2490690730 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2c5002e560 con 0x7f2c5010b670 2024-05-21T21:56:13.754 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:13.754+0000 7f2c46a00700 1 -- 172.21.15.151:0/2490690730 <== mon.2 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11958 (secure 0 0 0) 0x7f2c4802e3e0 con 0x7f2c5010b670 2024-05-21T21:56:13.754 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T21:56:13.754 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14358,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/158010529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":158010529},{"type":"v1","addr":"172.21.15.151:6835","nonce":158010529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6839/2069058115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":2069058115},{"type":"v1","addr":"172.21.15.151:6839","nonce":2069058115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T21:51:19.552069+0000","modified":"2024-05-21T21:55:34.643755+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24530,"mds_1":24493,"mds_2":24491,"mds_3":24478,"mds_4":24473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14403":{"gid":14403,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6841/1749723794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6840","nonce":1749723794},{"type":"v1","addr":"172.21.15.151:6841","nonce":1749723794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/2026266114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":2026266114},{"type":"v1","addr":"172.21.15.184:6833","nonce":2026266114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/2467900844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":2467900844},{"type":"v1","addr":"172.21.15.164:6833","nonce":2467900844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3367640960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3367640960},{"type":"v1","addr":"172.21.15.151:6837","nonce":3367640960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/3264142621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3264142621},{"type":"v1","addr":"172.21.15.184:6835","nonce":3264142621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":17,"addr":"172.21.15.164:6835/2340428470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2340428470},{"type":"v1","addr":"172.21.15.164:6835","nonce":2340428470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":16,"addr":"172.21.15.184:6837/2988194888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":2988194888},{"type":"v1","addr":"172.21.15.184:6837","nonce":2988194888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.164:6837/1554609572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1554609572},{"type":"v1","addr":"172.21.15.164:6837","nonce":1554609572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.184:6839/3059935143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":3059935143},{"type":"v1","addr":"172.21.15.184:6839","nonce":3059935143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.164:6839/2646401247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":2646401247},{"type":"v1","addr":"172.21.15.164:6839","nonce":2646401247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24530,"qdb_cluster":[24530,24493,24491,24473,24478]},"id":1}]} 2024-05-21T21:56:13.757 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:13.757+0000 7f2c4f200700 1 -- 172.21.15.151:0/2490690730 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f2c340b2b10 msgr2=0x7f2c340b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:13.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:13.757+0000 7f2c4f200700 1 --2- 172.21.15.151:0/2490690730 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f2c340b2b10 0x7f2c340b4fd0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2c40009fd0 tx=0x7f2c40009260 comp rx=0 tx=0).stop 2024-05-21T21:56:13.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:13.757+0000 7f2c4f200700 1 -- 172.21.15.151:0/2490690730 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2c5010b670 msgr2=0x7f2c501a05e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:13.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:13.757+0000 7f2c4f200700 1 --2- 172.21.15.151:0/2490690730 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2c5010b670 0x7f2c501a05e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2c4800b070 tx=0x7f2c48004c80 comp rx=0 tx=0).stop 2024-05-21T21:56:13.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:13.757+0000 7f2c4f200700 1 -- 172.21.15.151:0/2490690730 shutdown_connections 2024-05-21T21:56:13.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:13.757+0000 7f2c4f200700 1 --2- 172.21.15.151:0/2490690730 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f2c340b2b10 0x7f2c340b4fd0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:13.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:13.757+0000 7f2c4f200700 1 --2- 172.21.15.151:0/2490690730 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2c5010b670 0x7f2c501a05e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:13.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:13.757+0000 7f2c4f200700 1 --2- 172.21.15.151:0/2490690730 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c50105670 0x7f2c5019bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:13.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:13.757+0000 7f2c4f200700 1 --2- 172.21.15.151:0/2490690730 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2c50106030 0x7f2c5019c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:13.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:13.757+0000 7f2c4f200700 1 -- 172.21.15.151:0/2490690730 >> 172.21.15.151:0/2490690730 conn(0x7f2c50101160 msgr2=0x7f2c501016a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:56:13.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:13.757+0000 7f2c4f200700 1 -- 172.21.15.151:0/2490690730 shutdown_connections 2024-05-21T21:56:13.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:13.758+0000 7f2c4f200700 1 -- 172.21.15.151:0/2490690730 wait complete. 2024-05-21T21:56:13.759 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 29 2024-05-21T21:56:13.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:56:13 smithi151 ceph-mon[31623]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-05-21T21:56:13.985 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:56:13 smithi184 ceph-mon[36942]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-05-21T21:56:14.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:56:13 smithi164 ceph-mon[36269]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-05-21T21:56:14.666 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph osd dump --format=json 2024-05-21T21:56:14.739 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:56:14 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/2490690730' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T21:56:14.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:56:14 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/2490690730' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T21:56:15.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:56:14 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/2490690730' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T21:56:15.115 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:56:15.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:56:15 smithi151 ceph-mon[31623]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-05-21T21:56:15.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:56:15 smithi184 ceph-mon[36942]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-05-21T21:56:16.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:56:15 smithi164 ceph-mon[36269]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-05-21T21:56:17.286 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:17.285+0000 7f1025200700 1 -- 172.21.15.151:0/210577385 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10201095a0 msgr2=0x7f1020109980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:17.286 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:17.285+0000 7f1025200700 1 --2- 172.21.15.151:0/210577385 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10201095a0 0x7f1020109980 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1010009990 tx=0x7f1010009ca0 comp rx=0 tx=0).stop 2024-05-21T21:56:17.286 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:17.285+0000 7f1025200700 1 -- 172.21.15.151:0/210577385 shutdown_connections 2024-05-21T21:56:17.286 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:17.285+0000 7f1025200700 1 --2- 172.21.15.151:0/210577385 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f102006c470 0x7f10201103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:17.286 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:17.285+0000 7f1025200700 1 --2- 172.21.15.151:0/210577385 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10201095a0 0x7f1020109980 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:17.286 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:17.285+0000 7f1025200700 1 --2- 172.21.15.151:0/210577385 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f102006bab0 0x7f102006bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:17.286 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:17.285+0000 7f1025200700 1 -- 172.21.15.151:0/210577385 >> 172.21.15.151:0/210577385 conn(0x7f10200793b0 msgr2=0x7f10200797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:56:17.287 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:17.285+0000 7f1025200700 1 -- 172.21.15.151:0/210577385 shutdown_connections 2024-05-21T21:56:17.287 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:17.285+0000 7f1025200700 1 -- 172.21.15.151:0/210577385 wait complete. 2024-05-21T21:56:17.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:17.286+0000 7f1025200700 1 Processor -- start 2024-05-21T21:56:17.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:17.287+0000 7f1025200700 1 -- start start 2024-05-21T21:56:17.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:17.287+0000 7f1025200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f102006bab0 0x7f102019bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:17.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:17.287+0000 7f1025200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f102006c470 0x7f102019c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:17.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:17.287+0000 7f1025200700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10201095a0 0x7f10201a07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:17.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:17.287+0000 7f1025200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1020113730 con 0x7f102006bab0 2024-05-21T21:56:17.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:17.287+0000 7f1025200700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f10201135b0 con 0x7f102006c470 2024-05-21T21:56:17.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:17.287+0000 7f1025200700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f10201138b0 con 0x7f10201095a0 2024-05-21T21:56:17.289 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:17.287+0000 7f101de00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f102006bab0 0x7f102019bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:17.289 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:17.287+0000 7f101de00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f102006bab0 0x7f102019bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:53840/0 (socket says 172.21.15.151:53840) 2024-05-21T21:56:17.289 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:17.287+0000 7f101de00700 1 -- 172.21.15.151:0/2202356165 learned_addr learned my addr 172.21.15.151:0/2202356165 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:56:17.289 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:17.288+0000 7f101d400700 1 --2- 172.21.15.151:0/2202356165 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f102006c470 0x7f102019c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:17.289 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:17.288+0000 7f101e800700 1 --2- 172.21.15.151:0/2202356165 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10201095a0 0x7f10201a07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:17.289 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:17.288+0000 7f101de00700 1 -- 172.21.15.151:0/2202356165 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10201095a0 msgr2=0x7f10201a07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:17.289 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:17.288+0000 7f101de00700 1 --2- 172.21.15.151:0/2202356165 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10201095a0 0x7f10201a07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:17.289 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:17.288+0000 7f101de00700 1 -- 172.21.15.151:0/2202356165 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f102006c470 msgr2=0x7f102019c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:17.289 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:17.288+0000 7f101de00700 1 --2- 172.21.15.151:0/2202356165 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f102006c470 0x7f102019c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:17.289 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:17.288+0000 7f101de00700 1 -- 172.21.15.151:0/2202356165 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10201a0f00 con 0x7f102006bab0 2024-05-21T21:56:17.289 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:17.288+0000 7f101d400700 1 --2- 172.21.15.151:0/2202356165 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f102006c470 0x7f102019c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:56:17.290 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:17.288+0000 7f101de00700 1 --2- 172.21.15.151:0/2202356165 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f102006bab0 0x7f102019bf40 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1010000c00 tx=0x7f10100057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:56:17.290 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:17.289+0000 7f1016a00700 1 -- 172.21.15.151:0/2202356165 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1010027070 con 0x7f102006bab0 2024-05-21T21:56:17.290 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:17.289+0000 7f1016a00700 1 -- 172.21.15.151:0/2202356165 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1010022470 con 0x7f102006bab0 2024-05-21T21:56:17.290 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:17.289+0000 7f1025200700 1 -- 172.21.15.151:0/2202356165 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10201a1190 con 0x7f102006bab0 2024-05-21T21:56:17.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:17.289+0000 7f1016a00700 1 -- 172.21.15.151:0/2202356165 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1010017650 con 0x7f102006bab0 2024-05-21T21:56:17.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:17.289+0000 7f1025200700 1 -- 172.21.15.151:0/2202356165 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10201a8a70 con 0x7f102006bab0 2024-05-21T21:56:17.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:17.290+0000 7f1025200700 1 -- 172.21.15.151:0/2202356165 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f102006a230 con 0x7f102006bab0 2024-05-21T21:56:17.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:17.292+0000 7f1016a00700 1 -- 172.21.15.151:0/2202356165 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 26) ==== 149886+0+0 (secure 0 0 0) 0x7f101002a070 con 0x7f102006bab0 2024-05-21T21:56:17.294 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:17.294+0000 7f1016a00700 1 --2- 172.21.15.151:0/2202356165 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f10040b2b90 0x7f10040b5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:17.294 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:17.294+0000 7f1016a00700 1 -- 172.21.15.151:0/2202356165 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f10100e3600 con 0x7f102006bab0 2024-05-21T21:56:17.294 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:17.294+0000 7f101d400700 1 --2- 172.21.15.151:0/2202356165 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f10040b2b90 0x7f10040b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:17.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:17.295+0000 7f101d400700 1 --2- 172.21.15.151:0/2202356165 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f10040b2b90 0x7f10040b5050 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1020110790 tx=0x7f100c009260 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:56:17.296 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:17.296+0000 7f1016a00700 1 -- 172.21.15.151:0/2202356165 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1010013070 con 0x7f102006bab0 2024-05-21T21:56:17.531 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:17.530+0000 7f1025200700 1 -- 172.21.15.151:0/2202356165 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f10201a8c40 con 0x7f102006bab0 2024-05-21T21:56:17.535 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:17.535+0000 7f1016a00700 1 -- 172.21.15.151:0/2202356165 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+23128 (secure 0 0 0) 0x7f10100aadd0 con 0x7f102006bab0 2024-05-21T21:56:17.536 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T21:56:17.536 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":83,"fsid":"c8c7a2e6-17ba-11ef-bc9a-c7b262605968","created":"2024-05-21T21:41:52.412833+0000","modified":"2024-05-21T21:55:36.189583+0000","last_up_change":"2024-05-21T21:50:05.401314+0000","last_in_change":"2024-05-21T21:49:45.684136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-21T21:45:55.196046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-21T21:51:11.447428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-21T21:51:15.618605+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"18db8e54-0cfc-4e12-9c84-6f72a99cc989","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":2420400473},{"type":"v1","addr":"172.21.15.151:6803","nonce":2420400473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":2420400473},{"type":"v1","addr":"172.21.15.151:6805","nonce":2420400473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":2420400473},{"type":"v1","addr":"172.21.15.151:6809","nonce":2420400473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":2420400473},{"type":"v1","addr":"172.21.15.151:6807","nonce":2420400473}]},"public_addr":"172.21.15.151:6803/2420400473","cluster_addr":"172.21.15.151:6805/2420400473","heartbeat_back_addr":"172.21.15.151:6809/2420400473","heartbeat_front_addr":"172.21.15.151:6807/2420400473","state":["exists","up"]},{"osd":1,"uuid":"bac384c7-1102-4815-81a8-10cc7661d230","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":3900240447},{"type":"v1","addr":"172.21.15.164:6801","nonce":3900240447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":3900240447},{"type":"v1","addr":"172.21.15.164:6803","nonce":3900240447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":3900240447},{"type":"v1","addr":"172.21.15.164:6807","nonce":3900240447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":3900240447},{"type":"v1","addr":"172.21.15.164:6805","nonce":3900240447}]},"public_addr":"172.21.15.164:6801/3900240447","cluster_addr":"172.21.15.164:6803/3900240447","heartbeat_back_addr":"172.21.15.164:6807/3900240447","heartbeat_front_addr":"172.21.15.164:6805/3900240447","state":["exists","up"]},{"osd":2,"uuid":"b7e4a782-1518-4d43-bee9-7fd768ee5cb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":3971150737},{"type":"v1","addr":"172.21.15.184:6801","nonce":3971150737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3971150737},{"type":"v1","addr":"172.21.15.184:6803","nonce":3971150737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3971150737},{"type":"v1","addr":"172.21.15.184:6807","nonce":3971150737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3971150737},{"type":"v1","addr":"172.21.15.184:6805","nonce":3971150737}]},"public_addr":"172.21.15.184:6801/3971150737","cluster_addr":"172.21.15.184:6803/3971150737","heartbeat_back_addr":"172.21.15.184:6807/3971150737","heartbeat_front_addr":"172.21.15.184:6805/3971150737","state":["exists","up"]},{"osd":3,"uuid":"ab3df826-39e3-4635-a49c-2b5ffdbf12f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":556594400},{"type":"v1","addr":"172.21.15.151:6811","nonce":556594400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":556594400},{"type":"v1","addr":"172.21.15.151:6813","nonce":556594400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":556594400},{"type":"v1","addr":"172.21.15.151:6817","nonce":556594400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":556594400},{"type":"v1","addr":"172.21.15.151:6815","nonce":556594400}]},"public_addr":"172.21.15.151:6811/556594400","cluster_addr":"172.21.15.151:6813/556594400","heartbeat_back_addr":"172.21.15.151:6817/556594400","heartbeat_front_addr":"172.21.15.151:6815/556594400","state":["exists","up"]},{"osd":4,"uuid":"00cbf6ae-0f42-4685-a77d-d803c6467aa0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":988881973},{"type":"v1","addr":"172.21.15.164:6809","nonce":988881973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":988881973},{"type":"v1","addr":"172.21.15.164:6811","nonce":988881973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":988881973},{"type":"v1","addr":"172.21.15.164:6815","nonce":988881973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":988881973},{"type":"v1","addr":"172.21.15.164:6813","nonce":988881973}]},"public_addr":"172.21.15.164:6809/988881973","cluster_addr":"172.21.15.164:6811/988881973","heartbeat_back_addr":"172.21.15.164:6815/988881973","heartbeat_front_addr":"172.21.15.164:6813/988881973","state":["exists","up"]},{"osd":5,"uuid":"fad14221-0ef0-4d40-b414-c6c9f1f9718c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3293311106},{"type":"v1","addr":"172.21.15.184:6809","nonce":3293311106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":3293311106},{"type":"v1","addr":"172.21.15.184:6811","nonce":3293311106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":3293311106},{"type":"v1","addr":"172.21.15.184:6815","nonce":3293311106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":3293311106},{"type":"v1","addr":"172.21.15.184:6813","nonce":3293311106}]},"public_addr":"172.21.15.184:6809/3293311106","cluster_addr":"172.21.15.184:6811/3293311106","heartbeat_back_addr":"172.21.15.184:6815/3293311106","heartbeat_front_addr":"172.21.15.184:6813/3293311106","state":["exists","up"]},{"osd":6,"uuid":"ab8a60f6-0dec-4c1c-89db-f2e2f638f1ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":3377550762},{"type":"v1","addr":"172.21.15.151:6819","nonce":3377550762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":3377550762},{"type":"v1","addr":"172.21.15.151:6821","nonce":3377550762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":3377550762},{"type":"v1","addr":"172.21.15.151:6825","nonce":3377550762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":3377550762},{"type":"v1","addr":"172.21.15.151:6823","nonce":3377550762}]},"public_addr":"172.21.15.151:6819/3377550762","cluster_addr":"172.21.15.151:6821/3377550762","heartbeat_back_addr":"172.21.15.151:6825/3377550762","heartbeat_front_addr":"172.21.15.151:6823/3377550762","state":["exists","up"]},{"osd":7,"uuid":"db332268-9972-4ed7-a307-822b83a0fbb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":3869696936},{"type":"v1","addr":"172.21.15.164:6817","nonce":3869696936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":3869696936},{"type":"v1","addr":"172.21.15.164:6819","nonce":3869696936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":3869696936},{"type":"v1","addr":"172.21.15.164:6823","nonce":3869696936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":3869696936},{"type":"v1","addr":"172.21.15.164:6821","nonce":3869696936}]},"public_addr":"172.21.15.164:6817/3869696936","cluster_addr":"172.21.15.164:6819/3869696936","heartbeat_back_addr":"172.21.15.164:6823/3869696936","heartbeat_front_addr":"172.21.15.164:6821/3869696936","state":["exists","up"]},{"osd":8,"uuid":"fe2fbe1c-549a-4947-a188-426dfca04d10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":575273541},{"type":"v1","addr":"172.21.15.184:6817","nonce":575273541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":575273541},{"type":"v1","addr":"172.21.15.184:6819","nonce":575273541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":575273541},{"type":"v1","addr":"172.21.15.184:6823","nonce":575273541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":575273541},{"type":"v1","addr":"172.21.15.184:6821","nonce":575273541}]},"public_addr":"172.21.15.184:6817/575273541","cluster_addr":"172.21.15.184:6819/575273541","heartbeat_back_addr":"172.21.15.184:6823/575273541","heartbeat_front_addr":"172.21.15.184:6821/575273541","state":["exists","up"]},{"osd":9,"uuid":"3a7b450f-8004-4adf-bae3-970918f98f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":337128533},{"type":"v1","addr":"172.21.15.151:6827","nonce":337128533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":337128533},{"type":"v1","addr":"172.21.15.151:6829","nonce":337128533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":337128533},{"type":"v1","addr":"172.21.15.151:6833","nonce":337128533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":337128533},{"type":"v1","addr":"172.21.15.151:6831","nonce":337128533}]},"public_addr":"172.21.15.151:6827/337128533","cluster_addr":"172.21.15.151:6829/337128533","heartbeat_back_addr":"172.21.15.151:6833/337128533","heartbeat_front_addr":"172.21.15.151:6831/337128533","state":["exists","up"]},{"osd":10,"uuid":"8fcdf271-90c4-4db3-be69-b0ca716dfd80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":1823448782},{"type":"v1","addr":"172.21.15.164:6825","nonce":1823448782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":1823448782},{"type":"v1","addr":"172.21.15.164:6827","nonce":1823448782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":1823448782},{"type":"v1","addr":"172.21.15.164:6831","nonce":1823448782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":1823448782},{"type":"v1","addr":"172.21.15.164:6829","nonce":1823448782}]},"public_addr":"172.21.15.164:6825/1823448782","cluster_addr":"172.21.15.164:6827/1823448782","heartbeat_back_addr":"172.21.15.164:6831/1823448782","heartbeat_front_addr":"172.21.15.164:6829/1823448782","state":["exists","up"]},{"osd":11,"uuid":"e661477b-0c09-409f-a2fe-6ebd34642a8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":1563340128},{"type":"v1","addr":"172.21.15.184:6825","nonce":1563340128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":1563340128},{"type":"v1","addr":"172.21.15.184:6827","nonce":1563340128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":1563340128},{"type":"v1","addr":"172.21.15.184:6831","nonce":1563340128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":1563340128},{"type":"v1","addr":"172.21.15.184:6829","nonce":1563340128}]},"public_addr":"172.21.15.184:6825/1563340128","cluster_addr":"172.21.15.184:6827/1563340128","heartbeat_back_addr":"172.21.15.184:6831/1563340128","heartbeat_front_addr":"172.21.15.184:6829/1563340128","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:45:02.821160+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:45:27.844564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:45:51.904981+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:46:24.376245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:46:50.036621+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:47:14.748422+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:47:47.396383+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:48:13.543174+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:48:38.213285+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:49:11.980042+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:49:38.519761+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:50:03.439898+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.151:0/3481112063":"2024-05-22T21:55:36.189531+0000","172.21.15.151:0/2415279581":"2024-05-22T21:55:36.189531+0000","172.21.15.151:0/1946920088":"2024-05-22T21:55:36.189531+0000","172.21.15.151:6800/2796622483":"2024-05-22T21:55:36.189531+0000","172.21.15.151:0/31963128":"2024-05-22T21:43:09.041455+0000","172.21.15.151:6801/2796622483":"2024-05-22T21:55:36.189531+0000","172.21.15.151:6800/3428827300":"2024-05-22T21:43:09.041455+0000","172.21.15.151:0/3232339048":"2024-05-22T21:55:36.189531+0000","172.21.15.151:6801/3428827300":"2024-05-22T21:43:09.041455+0000","172.21.15.151:0/131989141":"2024-05-22T21:43:09.041455+0000","172.21.15.151:6801/1128885292":"2024-05-22T21:42:35.440147+0000","172.21.15.151:0/2703664519":"2024-05-22T21:43:09.041455+0000","172.21.15.151:6800/1128885292":"2024-05-22T21:42:35.440147+0000","172.21.15.151:0/3052943699":"2024-05-22T21:42:35.440147+0000","172.21.15.151:0/3929452511":"2024-05-22T21:42:35.440147+0000","172.21.15.151:0/3762889700":"2024-05-22T21:42:35.440147+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-21T21:56:17.538 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:17.538+0000 7f1025200700 1 -- 172.21.15.151:0/2202356165 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f10040b2b90 msgr2=0x7f10040b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:17.538 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:17.538+0000 7f1025200700 1 --2- 172.21.15.151:0/2202356165 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f10040b2b90 0x7f10040b5050 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1020110790 tx=0x7f100c009260 comp rx=0 tx=0).stop 2024-05-21T21:56:17.538 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:17.538+0000 7f1025200700 1 -- 172.21.15.151:0/2202356165 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f102006bab0 msgr2=0x7f102019bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:17.538 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:17.538+0000 7f1025200700 1 --2- 172.21.15.151:0/2202356165 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f102006bab0 0x7f102019bf40 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1010000c00 tx=0x7f10100057c0 comp rx=0 tx=0).stop 2024-05-21T21:56:17.539 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:17.538+0000 7f1025200700 1 -- 172.21.15.151:0/2202356165 shutdown_connections 2024-05-21T21:56:17.539 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:17.538+0000 7f1025200700 1 --2- 172.21.15.151:0/2202356165 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f10040b2b90 0x7f10040b5050 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:17.539 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:17.538+0000 7f1025200700 1 --2- 172.21.15.151:0/2202356165 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f10201095a0 0x7f10201a07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:17.539 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:17.538+0000 7f1025200700 1 --2- 172.21.15.151:0/2202356165 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f102006bab0 0x7f102019bf40 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:17.539 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:17.538+0000 7f1025200700 1 --2- 172.21.15.151:0/2202356165 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f102006c470 0x7f102019c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:17.539 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:17.538+0000 7f1025200700 1 -- 172.21.15.151:0/2202356165 >> 172.21.15.151:0/2202356165 conn(0x7f10200793b0 msgr2=0x7f1020100ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:56:17.540 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:17.538+0000 7f1025200700 1 -- 172.21.15.151:0/2202356165 shutdown_connections 2024-05-21T21:56:17.540 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:17.539+0000 7f1025200700 1 -- 172.21.15.151:0/2202356165 wait complete. 2024-05-21T21:56:17.855 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:56:17 smithi151 ceph-mon[31623]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-21T21:56:17.855 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:56:17 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/2202356165' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T21:56:17.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:56:17 smithi184 ceph-mon[36942]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-21T21:56:17.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:56:17 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/2202356165' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T21:56:18.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:56:17 smithi164 ceph-mon[36269]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-21T21:56:18.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:56:17 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/2202356165' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T21:56:18.311 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-05-21T21:56:18.769 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:56:19.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:56:19 smithi151 ceph-mon[31623]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 12 op/s 2024-05-21T21:56:19.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:56:19 smithi184 ceph-mon[36942]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 12 op/s 2024-05-21T21:56:20.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:56:19 smithi164 ceph-mon[36269]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 12 op/s 2024-05-21T21:56:21.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:56:21 smithi151 ceph-mon[31623]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2024-05-21T21:56:21.860 INFO:teuthology.orchestra.run.smithi151.stdout:{"run_dir":"/var/run/ceph"} 2024-05-21T21:56:21.860 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-05-21T21:56:21.861 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 3849 2024-05-21T21:56:21.861 DEBUG:tasks.quiescer:Yielding 2024-05-21T21:56:21.861 INFO:teuthology.run_tasks:Running task workunit... 2024-05-21T21:56:21.876 INFO:tasks.workunit:Pulling workunits from ref 7956b5e965f6ee5669688b5643c11d881b72442c 2024-05-21T21:56:21.876 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-05-21T21:56:21.876 INFO:tasks.workunit:timeout=3h 2024-05-21T21:56:21.876 INFO:tasks.workunit:cleanup=True 2024-05-21T21:56:21.877 DEBUG:teuthology.orchestra.run.smithi151:> stat -- /home/ubuntu/cephtest/mnt.0 2024-05-21T21:56:21.877 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-21T21:56:21.927 INFO:teuthology.orchestra.run.smithi151.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-05-21T21:56:21.928 INFO:teuthology.orchestra.run.smithi151.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-05-21T21:56:21.928 INFO:teuthology.orchestra.run.smithi151.stdout:Device: 77h/119d Inode: 1099511628283 Links: 2 2024-05-21T21:56:21.928 INFO:teuthology.orchestra.run.smithi151.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-21T21:56:21.928 INFO:teuthology.orchestra.run.smithi151.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-05-21T21:56:21.928 INFO:teuthology.orchestra.run.smithi151.stdout:Access: 2024-05-21 21:51:57.698963207 +0000 2024-05-21T21:56:21.928 INFO:teuthology.orchestra.run.smithi151.stdout:Modify: 2024-05-21 21:51:57.698963207 +0000 2024-05-21T21:56:21.928 INFO:teuthology.orchestra.run.smithi151.stdout:Change: 2024-05-21 21:55:33.616925130 +0000 2024-05-21T21:56:21.928 INFO:teuthology.orchestra.run.smithi151.stdout: Birth: 2024-05-21 21:51:57.698963207 +0000 2024-05-21T21:56:21.928 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-05-21T21:56:21.928 DEBUG:teuthology.orchestra.run.smithi151:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-05-21T21:56:21.985 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:56:21 smithi184 ceph-mon[36942]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2024-05-21T21:56:22.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:56:21 smithi164 ceph-mon[36269]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2024-05-21T21:56:22.054 DEBUG:teuthology.orchestra.run.smithi151:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 7956b5e965f6ee5669688b5643c11d881b72442c 2024-05-21T21:56:22.106 INFO:tasks.workunit.client.0.smithi151.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-05-21T21:56:22.314 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:56:23.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:56:23 smithi151 ceph-mon[31623]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-21T21:56:23.985 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:56:23 smithi184 ceph-mon[36942]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-21T21:56:24.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:56:23 smithi164 ceph-mon[36269]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-21T21:56:24.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:24.587+0000 7f8efc000700 1 -- 172.21.15.151:0/4192604706 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ef410b730 msgr2=0x7f8ef410bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:24.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:24.587+0000 7f8efc000700 1 --2- 172.21.15.151:0/4192604706 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ef410b730 0x7f8ef410bb10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8ee8009990 tx=0x7f8ee8009ca0 comp rx=0 tx=0).stop 2024-05-21T21:56:24.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:24.587+0000 7f8efc000700 1 -- 172.21.15.151:0/4192604706 shutdown_connections 2024-05-21T21:56:24.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:24.587+0000 7f8efc000700 1 --2- 172.21.15.151:0/4192604706 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8ef41060f0 0x7f8ef4112600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:24.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:24.587+0000 7f8efc000700 1 --2- 172.21.15.151:0/4192604706 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ef410b730 0x7f8ef410bb10 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:24.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:24.587+0000 7f8efc000700 1 --2- 172.21.15.151:0/4192604706 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8ef4105730 0x7f8ef4105bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:24.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:24.587+0000 7f8efc000700 1 -- 172.21.15.151:0/4192604706 >> 172.21.15.151:0/4192604706 conn(0x7f8ef4101220 msgr2=0x7f8ef4103640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:56:24.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:24.587+0000 7f8efc000700 1 -- 172.21.15.151:0/4192604706 shutdown_connections 2024-05-21T21:56:24.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:24.587+0000 7f8efc000700 1 -- 172.21.15.151:0/4192604706 wait complete. 2024-05-21T21:56:24.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:24.589+0000 7f8efc000700 1 Processor -- start 2024-05-21T21:56:24.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:24.589+0000 7f8efc000700 1 -- start start 2024-05-21T21:56:24.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:24.589+0000 7f8efc000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ef4105730 0x7f8ef419beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:24.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:24.589+0000 7f8efc000700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8ef41060f0 0x7f8ef419c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:24.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:24.589+0000 7f8efc000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8ef410b730 0x7f8ef41a0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:24.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:24.589+0000 7f8efc000700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef4115980 con 0x7f8ef4105730 2024-05-21T21:56:24.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:24.589+0000 7f8efc000700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef4115800 con 0x7f8ef410b730 2024-05-21T21:56:24.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:24.589+0000 7f8efc000700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef4115b00 con 0x7f8ef41060f0 2024-05-21T21:56:24.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:24.590+0000 7f8ef9600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ef4105730 0x7f8ef419beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:24.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:24.590+0000 7f8ef9600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ef4105730 0x7f8ef419beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:53862/0 (socket says 172.21.15.151:53862) 2024-05-21T21:56:24.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:24.590+0000 7f8ef9600700 1 -- 172.21.15.151:0/3010523002 learned_addr learned my addr 172.21.15.151:0/3010523002 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:56:24.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:24.590+0000 7f8efa000700 1 --2- 172.21.15.151:0/3010523002 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8ef410b730 0x7f8ef41a0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:24.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:24.590+0000 7f8ef8c00700 1 --2- 172.21.15.151:0/3010523002 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8ef41060f0 0x7f8ef419c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:24.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:24.590+0000 7f8ef9600700 1 -- 172.21.15.151:0/3010523002 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8ef41060f0 msgr2=0x7f8ef419c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:24.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:24.590+0000 7f8ef9600700 1 --2- 172.21.15.151:0/3010523002 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8ef41060f0 0x7f8ef419c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:24.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:24.590+0000 7f8ef9600700 1 -- 172.21.15.151:0/3010523002 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8ef410b730 msgr2=0x7f8ef41a0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:24.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:24.590+0000 7f8ef9600700 1 --2- 172.21.15.151:0/3010523002 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8ef410b730 0x7f8ef41a0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:24.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:24.590+0000 7f8ef9600700 1 -- 172.21.15.151:0/3010523002 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ef41a0e70 con 0x7f8ef4105730 2024-05-21T21:56:24.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:24.590+0000 7f8efa000700 1 --2- 172.21.15.151:0/3010523002 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8ef410b730 0x7f8ef41a0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:56:24.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:24.591+0000 7f8ef9600700 1 --2- 172.21.15.151:0/3010523002 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ef4105730 0x7f8ef419beb0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8ee8000c00 tx=0x7f8ee80057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:56:24.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:24.591+0000 7f8ef2000700 1 -- 172.21.15.151:0/3010523002 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee8027070 con 0x7f8ef4105730 2024-05-21T21:56:24.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:24.591+0000 7f8ef2000700 1 -- 172.21.15.151:0/3010523002 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ee8022470 con 0x7f8ef4105730 2024-05-21T21:56:24.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:24.591+0000 7f8ef2000700 1 -- 172.21.15.151:0/3010523002 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee8017650 con 0x7f8ef4105730 2024-05-21T21:56:24.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:24.591+0000 7f8efc000700 1 -- 172.21.15.151:0/3010523002 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ef41a1100 con 0x7f8ef4105730 2024-05-21T21:56:24.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:24.591+0000 7f8efc000700 1 -- 172.21.15.151:0/3010523002 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ef41a89e0 con 0x7f8ef4105730 2024-05-21T21:56:24.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:24.592+0000 7f8efc000700 1 -- 172.21.15.151:0/3010523002 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ef406a230 con 0x7f8ef4105730 2024-05-21T21:56:24.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:24.594+0000 7f8ef2000700 1 -- 172.21.15.151:0/3010523002 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 26) ==== 149886+0+0 (secure 0 0 0) 0x7f8ee802a070 con 0x7f8ef4105730 2024-05-21T21:56:24.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:24.596+0000 7f8ef2000700 1 --2- 172.21.15.151:0/3010523002 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f8edc0b2b90 0x7f8edc0b5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:24.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:24.596+0000 7f8ef2000700 1 -- 172.21.15.151:0/3010523002 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f8ee80e38c0 con 0x7f8ef4105730 2024-05-21T21:56:24.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:24.596+0000 7f8ef8c00700 1 --2- 172.21.15.151:0/3010523002 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f8edc0b2b90 0x7f8edc0b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:24.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:24.597+0000 7f8ef8c00700 1 --2- 172.21.15.151:0/3010523002 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f8edc0b2b90 0x7f8edc0b5050 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8ef41038b0 tx=0x7f8ee0009260 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:56:24.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:24.599+0000 7f8ef2000700 1 -- 172.21.15.151:0/3010523002 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8ee8013070 con 0x7f8ef4105730 2024-05-21T21:56:24.884 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:24.884+0000 7f8efc000700 1 -- 172.21.15.151:0/3010523002 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8ef419d1c0 con 0x7f8ef4105730 2024-05-21T21:56:24.886 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:24.885+0000 7f8ef2000700 1 -- 172.21.15.151:0/3010523002 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11958 (secure 0 0 0) 0x7f8ee80ab090 con 0x7f8ef4105730 2024-05-21T21:56:24.886 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T21:56:24.887 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14358,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/158010529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":158010529},{"type":"v1","addr":"172.21.15.151:6835","nonce":158010529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6839/2069058115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":2069058115},{"type":"v1","addr":"172.21.15.151:6839","nonce":2069058115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T21:51:19.552069+0000","modified":"2024-05-21T21:55:34.643755+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24530,"mds_1":24493,"mds_2":24491,"mds_3":24478,"mds_4":24473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14403":{"gid":14403,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6841/1749723794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6840","nonce":1749723794},{"type":"v1","addr":"172.21.15.151:6841","nonce":1749723794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/2026266114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":2026266114},{"type":"v1","addr":"172.21.15.184:6833","nonce":2026266114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/2467900844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":2467900844},{"type":"v1","addr":"172.21.15.164:6833","nonce":2467900844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3367640960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3367640960},{"type":"v1","addr":"172.21.15.151:6837","nonce":3367640960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/3264142621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3264142621},{"type":"v1","addr":"172.21.15.184:6835","nonce":3264142621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":17,"addr":"172.21.15.164:6835/2340428470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2340428470},{"type":"v1","addr":"172.21.15.164:6835","nonce":2340428470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":16,"addr":"172.21.15.184:6837/2988194888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":2988194888},{"type":"v1","addr":"172.21.15.184:6837","nonce":2988194888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.164:6837/1554609572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1554609572},{"type":"v1","addr":"172.21.15.164:6837","nonce":1554609572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.184:6839/3059935143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":3059935143},{"type":"v1","addr":"172.21.15.184:6839","nonce":3059935143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.164:6839/2646401247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":2646401247},{"type":"v1","addr":"172.21.15.164:6839","nonce":2646401247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24530,"qdb_cluster":[24530,24493,24491,24473,24478]},"id":1}]} 2024-05-21T21:56:24.890 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:24.890+0000 7f8efc000700 1 -- 172.21.15.151:0/3010523002 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f8edc0b2b90 msgr2=0x7f8edc0b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:24.890 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:24.890+0000 7f8efc000700 1 --2- 172.21.15.151:0/3010523002 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f8edc0b2b90 0x7f8edc0b5050 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8ef41038b0 tx=0x7f8ee0009260 comp rx=0 tx=0).stop 2024-05-21T21:56:24.890 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:24.890+0000 7f8efc000700 1 -- 172.21.15.151:0/3010523002 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ef4105730 msgr2=0x7f8ef419beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:24.890 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:24.890+0000 7f8efc000700 1 --2- 172.21.15.151:0/3010523002 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ef4105730 0x7f8ef419beb0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8ee8000c00 tx=0x7f8ee80057c0 comp rx=0 tx=0).stop 2024-05-21T21:56:24.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:24.890+0000 7f8efc000700 1 -- 172.21.15.151:0/3010523002 shutdown_connections 2024-05-21T21:56:24.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:24.890+0000 7f8efc000700 1 --2- 172.21.15.151:0/3010523002 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f8edc0b2b90 0x7f8edc0b5050 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:24.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:24.890+0000 7f8efc000700 1 --2- 172.21.15.151:0/3010523002 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8ef410b730 0x7f8ef41a0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:24.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:24.890+0000 7f8efc000700 1 --2- 172.21.15.151:0/3010523002 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ef4105730 0x7f8ef419beb0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:24.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:24.890+0000 7f8efc000700 1 --2- 172.21.15.151:0/3010523002 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8ef41060f0 0x7f8ef419c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:24.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:24.890+0000 7f8efc000700 1 -- 172.21.15.151:0/3010523002 >> 172.21.15.151:0/3010523002 conn(0x7f8ef4101220 msgr2=0x7f8ef4101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:56:24.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:24.890+0000 7f8efc000700 1 -- 172.21.15.151:0/3010523002 shutdown_connections 2024-05-21T21:56:24.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:24.890+0000 7f8efc000700 1 -- 172.21.15.151:0/3010523002 wait complete. 2024-05-21T21:56:24.892 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 29 2024-05-21T21:56:25.751 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 29, 'flags': 306, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': True}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-21T21:51:19.552069+0000', 'modified': '2024-05-21T21:55:34.643755+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24530, 'mds_1': 24493, 'mds_2': 24491, 'mds_3': 24478, 'mds_4': 24473}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14403': {'gid': 14403, 'name': 'a', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.151:6841/1749723794', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.151:6840', 'nonce': 1749723794}, {'type': 'v1', 'addr': '172.21.15.151:6841', 'nonce': 1749723794}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24436': {'gid': 24436, 'name': 'l', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.184:6833/2026266114', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6832', 'nonce': 2026266114}, {'type': 'v1', 'addr': '172.21.15.184:6833', 'nonce': 2026266114}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24443': {'gid': 24443, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.164:6833/2467900844', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6832', 'nonce': 2467900844}, {'type': 'v1', 'addr': '172.21.15.164:6833', 'nonce': 2467900844}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24461': {'gid': 24461, 'name': 'd', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.151:6837/3367640960', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.151:6836', 'nonce': 3367640960}, {'type': 'v1', 'addr': '172.21.15.151:6837', 'nonce': 3367640960}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24466': {'gid': 24466, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.184:6835/3264142621', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6834', 'nonce': 3264142621}, {'type': 'v1', 'addr': '172.21.15.184:6835', 'nonce': 3264142621}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24473': {'gid': 24473, 'name': 'e', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.164:6835/2340428470', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6834', 'nonce': 2340428470}, {'type': 'v1', 'addr': '172.21.15.164:6835', 'nonce': 2340428470}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24478': {'gid': 24478, 'name': 'c', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 16, 'addr': '172.21.15.184:6837/2988194888', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6836', 'nonce': 2988194888}, {'type': 'v1', 'addr': '172.21.15.184:6837', 'nonce': 2988194888}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24491': {'gid': 24491, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.164:6837/1554609572', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6836', 'nonce': 1554609572}, {'type': 'v1', 'addr': '172.21.15.164:6837', 'nonce': 1554609572}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24493': {'gid': 24493, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.184:6839/3059935143', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6838', 'nonce': 3059935143}, {'type': 'v1', 'addr': '172.21.15.184:6839', 'nonce': 3059935143}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24530': {'gid': 24530, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.164:6839/2646401247', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6838', 'nonce': 2646401247}, {'type': 'v1', 'addr': '172.21.15.164:6839', 'nonce': 2646401247}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24530, 'qdb_cluster': [24530, 24493, 24491, 24473, 24478]} 2024-05-21T21:56:25.751 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-05-21T21:56:25.752 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.e status 2024-05-21T21:56:25.834 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:56:25 smithi151 ceph-mon[31623]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1023 B/s wr, 12 op/s 2024-05-21T21:56:25.834 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:56:25 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/3010523002' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T21:56:25.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:56:25 smithi184 ceph-mon[36942]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1023 B/s wr, 12 op/s 2024-05-21T21:56:25.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:56:25 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/3010523002' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T21:56:26.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:56:25 smithi164 ceph-mon[36269]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1023 B/s wr, 12 op/s 2024-05-21T21:56:26.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:56:25 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/3010523002' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T21:56:26.212 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:56:27.881 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:56:27 smithi151 ceph-mon[31623]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 852 B/s wr, 8 op/s 2024-05-21T21:56:27.985 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:56:27 smithi184 ceph-mon[36942]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 852 B/s wr, 8 op/s 2024-05-21T21:56:28.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:56:27 smithi164 ceph-mon[36269]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 852 B/s wr, 8 op/s 2024-05-21T21:56:28.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.360+0000 7ff9a6a00700 1 -- 172.21.15.151:0/90246045 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff9a01142f0 msgr2=0x7ff9a01166e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:28.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.360+0000 7ff9a6a00700 1 --2- 172.21.15.151:0/90246045 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff9a01142f0 0x7ff9a01166e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff99800b0a0 tx=0x7ff99800b3b0 comp rx=0 tx=0).stop 2024-05-21T21:56:28.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.360+0000 7ff9a6a00700 1 -- 172.21.15.151:0/90246045 shutdown_connections 2024-05-21T21:56:28.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.360+0000 7ff9a6a00700 1 --2- 172.21.15.151:0/90246045 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff9a01142f0 0x7ff9a01166e0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:28.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.360+0000 7ff9a6a00700 1 --2- 172.21.15.151:0/90246045 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9a0077a70 0x7ff9a0077e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:28.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.360+0000 7ff9a6a00700 1 --2- 172.21.15.151:0/90246045 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff9a0078390 0x7ff9a0113db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:28.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.360+0000 7ff9a6a00700 1 -- 172.21.15.151:0/90246045 >> 172.21.15.151:0/90246045 conn(0x7ff9a0101180 msgr2=0x7ff9a01035a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:56:28.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.361+0000 7ff9a6a00700 1 -- 172.21.15.151:0/90246045 shutdown_connections 2024-05-21T21:56:28.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.361+0000 7ff9a6a00700 1 -- 172.21.15.151:0/90246045 wait complete. 2024-05-21T21:56:28.363 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.362+0000 7ff9a6a00700 1 Processor -- start 2024-05-21T21:56:28.363 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.362+0000 7ff9a6a00700 1 -- start start 2024-05-21T21:56:28.363 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.363+0000 7ff9a6a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff9a0077a70 0x7ff9a01a0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:28.363 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.363+0000 7ff9a6a00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9a0078390 0x7ff9a01a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:28.363 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.363+0000 7ff9a6a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff9a01142f0 0x7ff9a01a49f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:28.364 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.363+0000 7ff9a6a00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a0119c90 con 0x7ff9a0077a70 2024-05-21T21:56:28.364 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.363+0000 7ff9a6a00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a0119b10 con 0x7ff9a01142f0 2024-05-21T21:56:28.364 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.363+0000 7ff9a6a00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a0119e10 con 0x7ff9a0078390 2024-05-21T21:56:28.364 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.363+0000 7ff9a5600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff9a0077a70 0x7ff9a01a0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:28.364 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.363+0000 7ff9a5600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff9a0077a70 0x7ff9a01a0120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:58452/0 (socket says 172.21.15.151:58452) 2024-05-21T21:56:28.364 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.363+0000 7ff9a5600700 1 -- 172.21.15.151:0/2905543758 learned_addr learned my addr 172.21.15.151:0/2905543758 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:56:28.364 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.363+0000 7ff9a4c00700 1 --2- 172.21.15.151:0/2905543758 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9a0078390 0x7ff9a01a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:28.364 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.363+0000 7ff9a6000700 1 --2- 172.21.15.151:0/2905543758 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff9a01142f0 0x7ff9a01a49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:28.364 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.364+0000 7ff9a5600700 1 -- 172.21.15.151:0/2905543758 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9a0078390 msgr2=0x7ff9a01a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:28.364 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.364+0000 7ff9a5600700 1 --2- 172.21.15.151:0/2905543758 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9a0078390 0x7ff9a01a0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:28.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.364+0000 7ff9a5600700 1 -- 172.21.15.151:0/2905543758 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff9a01142f0 msgr2=0x7ff9a01a49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:28.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.364+0000 7ff9a5600700 1 --2- 172.21.15.151:0/2905543758 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff9a01142f0 0x7ff9a01a49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:28.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.364+0000 7ff9a5600700 1 -- 172.21.15.151:0/2905543758 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9a01a5130 con 0x7ff9a0077a70 2024-05-21T21:56:28.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.364+0000 7ff9a4c00700 1 --2- 172.21.15.151:0/2905543758 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9a0078390 0x7ff9a01a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:56:28.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.364+0000 7ff9a5600700 1 --2- 172.21.15.151:0/2905543758 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff9a0077a70 0x7ff9a01a0120 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff99400e9c0 tx=0x7ff99400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:56:28.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.364+0000 7ff99e000700 1 -- 172.21.15.151:0/2905543758 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff99400cbd0 con 0x7ff9a0077a70 2024-05-21T21:56:28.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.364+0000 7ff99e000700 1 -- 172.21.15.151:0/2905543758 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff99400ee60 con 0x7ff9a0077a70 2024-05-21T21:56:28.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.364+0000 7ff9a6a00700 1 -- 172.21.15.151:0/2905543758 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9a01a5420 con 0x7ff9a0077a70 2024-05-21T21:56:28.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.364+0000 7ff99e000700 1 -- 172.21.15.151:0/2905543758 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9940055c0 con 0x7ff9a0077a70 2024-05-21T21:56:28.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.364+0000 7ff9a6a00700 1 -- 172.21.15.151:0/2905543758 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9a01acca0 con 0x7ff9a0077a70 2024-05-21T21:56:28.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.370+0000 7ff99e000700 1 -- 172.21.15.151:0/2905543758 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 26) ==== 149886+0+0 (secure 0 0 0) 0x7ff994004750 con 0x7ff9a0077a70 2024-05-21T21:56:28.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.371+0000 7ff9b2990b80 1 Processor -- start 2024-05-21T21:56:28.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.371+0000 7ff9b2990b80 1 -- start start 2024-05-21T21:56:28.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.371+0000 7ff9b2990b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55acdd23f770 0x55acdd23fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:28.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.371+0000 7ff9b2990b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55acdd240090 0x55acdd239d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:28.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.371+0000 7ff99e000700 1 --2- 172.21.15.151:0/2905543758 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7ff9880b2b90 0x7ff9880b5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:28.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.371+0000 7ff9b2990b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55acdd23a340 0x55acdd23a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:28.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.371+0000 7ff9b2990b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55acdd197df0 con 0x55acdd23f770 2024-05-21T21:56:28.372 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.371+0000 7ff9b2990b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55acdd187990 con 0x55acdd23a340 2024-05-21T21:56:28.372 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.371+0000 7ff9b2990b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55acdd18b9d0 con 0x55acdd240090 2024-05-21T21:56:28.373 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.371+0000 7ff99e000700 1 -- 172.21.15.151:0/2905543758 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7ff9940e2ac0 con 0x7ff9a0077a70 2024-05-21T21:56:28.373 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.372+0000 7ff9a6000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55acdd23f770 0x55acdd23fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:28.373 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.372+0000 7ff9a6000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55acdd23f770 0x55acdd23fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:58460/0 (socket says 172.21.15.151:58460) 2024-05-21T21:56:28.373 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.372+0000 7ff9a6000700 1 -- 172.21.15.151:0/421285638 learned_addr learned my addr 172.21.15.151:0/421285638 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:56:28.373 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.372+0000 7ff9a4c00700 1 --2- 172.21.15.151:0/421285638 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55acdd240090 0x55acdd239d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:28.373 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.372+0000 7ff9a5600700 1 --2- 172.21.15.151:0/2905543758 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7ff9880b2b90 0x7ff9880b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:28.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.372+0000 7ff9a6000700 1 --2- 172.21.15.151:0/421285638 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55acdd23a340 0x55acdd23a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:28.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.372+0000 7ff9a4c00700 1 -- 172.21.15.151:0/421285638 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55acdd23a340 msgr2=0x55acdd23a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:56:28.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.372+0000 7ff9a4c00700 1 --2- 172.21.15.151:0/421285638 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55acdd23a340 0x55acdd23a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:28.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.372+0000 7ff9a4c00700 1 -- 172.21.15.151:0/421285638 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55acdd23f770 msgr2=0x55acdd23fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:56:28.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.372+0000 7ff9a4c00700 1 --2- 172.21.15.151:0/421285638 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55acdd23f770 0x55acdd23fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:28.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.372+0000 7ff9a4c00700 1 -- 172.21.15.151:0/421285638 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55acdd1503d0 con 0x55acdd240090 2024-05-21T21:56:28.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.373+0000 7ff9a5600700 1 --2- 172.21.15.151:0/2905543758 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7ff9880b2b90 0x7ff9880b5050 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff9940e0100 tx=0x7ff9940defc0 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:56:28.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.373+0000 7ff9a4c00700 1 --2- 172.21.15.151:0/421285638 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55acdd240090 0x55acdd239d20 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff98c009fb0 tx=0x7ff98c011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5cd5bc19b50b1357 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:56:28.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.373+0000 7ff992a00700 1 -- 172.21.15.151:0/421285638 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff98c004230 con 0x55acdd240090 2024-05-21T21:56:28.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.373+0000 7ff992a00700 1 -- 172.21.15.151:0/421285638 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff98c004ce0 con 0x55acdd240090 2024-05-21T21:56:28.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.373+0000 7ff992a00700 1 -- 172.21.15.151:0/421285638 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff98c005690 con 0x55acdd240090 2024-05-21T21:56:28.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.374+0000 7ff9b2990b80 1 -- 172.21.15.151:0/421285638 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55acdd240090 msgr2=0x55acdd239d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:28.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.374+0000 7ff9b2990b80 1 --2- 172.21.15.151:0/421285638 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55acdd240090 0x55acdd239d20 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff98c009fb0 tx=0x7ff98c011720 comp rx=0 tx=0).stop 2024-05-21T21:56:28.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.374+0000 7ff9b2990b80 1 -- 172.21.15.151:0/421285638 shutdown_connections 2024-05-21T21:56:28.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.374+0000 7ff9b2990b80 1 --2- 172.21.15.151:0/421285638 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55acdd23a340 0x55acdd23a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:28.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.374+0000 7ff9b2990b80 1 --2- 172.21.15.151:0/421285638 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55acdd23f770 0x55acdd23fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:28.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.374+0000 7ff9b2990b80 1 --2- 172.21.15.151:0/421285638 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55acdd240090 0x55acdd239d20 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:28.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.374+0000 7ff9b2990b80 1 -- 172.21.15.151:0/421285638 >> 172.21.15.151:0/421285638 conn(0x55acdd167b90 msgr2=0x55acdd2347c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:56:28.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.374+0000 7ff9b2990b80 1 -- 172.21.15.151:0/421285638 shutdown_connections 2024-05-21T21:56:28.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.374+0000 7ff9b2990b80 1 -- 172.21.15.151:0/421285638 wait complete. 2024-05-21T21:56:28.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.375+0000 7ff9b2990b80 1 -- start start 2024-05-21T21:56:28.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.375+0000 7ff9b2990b80 1 Processor -- start 2024-05-21T21:56:28.377 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.375+0000 7ff9b2990b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55acdd23a340 0x55acdd245da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:28.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.375+0000 7ff9b2990b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55acdd23f770 0x55acdd243140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:28.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.376+0000 7ff9b2990b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55acdd240090 0x55acdd243680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:28.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.376+0000 7ff9b2990b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55acdd08df50 con 0x55acdd23f770 2024-05-21T21:56:28.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.376+0000 7ff9b2990b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55acdd128710 con 0x55acdd240090 2024-05-21T21:56:28.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.376+0000 7ff9b2990b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55acdd246bb0 con 0x55acdd23a340 2024-05-21T21:56:28.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.376+0000 7ff9b2990b80 10 client.0 _ll_register_callbacks cb 0x55acdd131c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:56:28.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.376+0000 7ff9a4c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55acdd23f770 0x55acdd243140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:28.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.376+0000 7ff9a6000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55acdd240090 0x55acdd243680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:28.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.376+0000 7ff9a4c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55acdd23f770 0x55acdd243140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:58472/0 (socket says 172.21.15.151:58472) 2024-05-21T21:56:28.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.376+0000 7ff9a4c00700 1 -- 172.21.15.151:0/3610085193 learned_addr learned my addr 172.21.15.151:0/3610085193 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:56:28.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.376+0000 7ff9a6000700 1 --2- 172.21.15.151:0/3610085193 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55acdd23a340 0x55acdd245da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:28.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.376+0000 7ff9a4c00700 1 -- 172.21.15.151:0/3610085193 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55acdd23a340 msgr2=0x55acdd245da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:56:28.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.376+0000 7ff9a4c00700 1 --2- 172.21.15.151:0/3610085193 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55acdd23a340 0x55acdd245da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:28.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.376+0000 7ff9a4c00700 1 -- 172.21.15.151:0/3610085193 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55acdd240090 msgr2=0x55acdd243680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:56:28.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.376+0000 7ff9a4c00700 1 --2- 172.21.15.151:0/3610085193 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55acdd240090 0x55acdd243680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:28.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.376+0000 7ff9a4c00700 1 -- 172.21.15.151:0/3610085193 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55acdd0d7880 con 0x55acdd23f770 2024-05-21T21:56:28.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.376+0000 7ff9a6000700 1 --2- 172.21.15.151:0/3610085193 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55acdd240090 0x55acdd243680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:56:28.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.377+0000 7ff9a4c00700 1 --2- 172.21.15.151:0/3610085193 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55acdd23f770 0x55acdd243140 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff98c009230 tx=0x7ff98c004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=562687b59e9679ee server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:56:28.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.377+0000 7ff987400700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-21T21:56:28.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.377+0000 7ff987400700 1 -- 172.21.15.151:0/3610085193 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff98c026070 con 0x55acdd23f770 2024-05-21T21:56:28.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.377+0000 7ff987400700 1 -- 172.21.15.151:0/3610085193 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7ff98c0054e0 con 0x55acdd23f770 2024-05-21T21:56:28.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.377+0000 7ff9b2990b80 1 -- 172.21.15.151:0/3610085193 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff98c004ce0 con 0x55acdd23f770 2024-05-21T21:56:28.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.378+0000 7ff987400700 1 -- 172.21.15.151:0/3610085193 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff98c032050 con 0x55acdd23f770 2024-05-21T21:56:28.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.378+0000 7ff9b2990b80 10 client.15264 fetch_fsmap learned FSMap version 29 2024-05-21T21:56:28.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.378+0000 7ff9b2990b80 1 -- 172.21.15.151:0/3610085193 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55acdd158010 con 0x55acdd23f770 2024-05-21T21:56:28.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.378+0000 7ff987400700 1 -- 172.21.15.151:0/3610085193 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff98c004ce0 con 0x55acdd23f770 2024-05-21T21:56:28.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.378+0000 7ff987400700 1 -- 172.21.15.151:0/3610085193 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7ff98c00e590 con 0x55acdd23f770 2024-05-21T21:56:28.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.378+0000 7ff9b2990b80 10 client.15264 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T21:56:28.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.378+0000 7ff9b2990b80 10 client.15264 resolve_mds: resolved name 'e' to daemon mds.e 2024-05-21T21:56:28.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.378+0000 7ff9b2990b80 20 client.15264 populate_metadata read hostname 'smithi151' 2024-05-21T21:56:28.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.378+0000 7ff9b2990b80 1 --2- 172.21.15.151:0/3610085193 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x55acdd2533c0 0x55acdd254830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:28.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.378+0000 7ff9b2990b80 4 client.15264 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T21:56:28.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.378+0000 7ff9b2990b80 1 -- 172.21.15.151:0/3610085193 --> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55acdd199d40 con 0x55acdd2533c0 2024-05-21T21:56:28.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.379+0000 7ff9a6000700 1 --2- 172.21.15.151:0/3610085193 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x55acdd2533c0 0x55acdd254830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:28.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.379+0000 7ff9a6000700 1 --2- 172.21.15.151:0/3610085193 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x55acdd2533c0 0x55acdd254830 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d7c6e567a6e5f5e3 server_cookie=4e2e6d785b74775 in_seq=0 out_seq=0 2024-05-21T21:56:28.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.379+0000 7ff987400700 10 client.15264 ms_handle_connect on v2:172.21.15.164:6834/2340428470 2024-05-21T21:56:28.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.380+0000 7ff987400700 1 -- 172.21.15.151:0/3610085193 <== mds.4 v2:172.21.15.164:6834/2340428470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55acdd199d40 con 0x55acdd2533c0 2024-05-21T21:56:28.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.380+0000 7ff987400700 10 client.15264 handle_command_reply: tid=0 2024-05-21T21:56:28.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.380+0000 7ff9b2990b80 1 client.15264 shutdown 2024-05-21T21:56:28.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.380+0000 7ff9b2990b80 1 -- 172.21.15.151:0/3610085193 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55acdd23f770 msgr2=0x55acdd243140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:28.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.380+0000 7ff9b2990b80 1 --2- 172.21.15.151:0/3610085193 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55acdd23f770 0x55acdd243140 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff98c009230 tx=0x7ff98c004a50 comp rx=0 tx=0).stop 2024-05-21T21:56:28.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.380+0000 7ff9b2990b80 1 -- 172.21.15.151:0/3610085193 shutdown_connections 2024-05-21T21:56:28.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.380+0000 7ff9b2990b80 1 --2- 172.21.15.151:0/3610085193 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x55acdd2533c0 0x55acdd254830 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:28.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.380+0000 7ff9b2990b80 1 --2- 172.21.15.151:0/3610085193 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55acdd240090 0x55acdd243680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:28.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.380+0000 7ff9b2990b80 1 --2- 172.21.15.151:0/3610085193 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55acdd23a340 0x55acdd245da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:28.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.380+0000 7ff9b2990b80 1 --2- 172.21.15.151:0/3610085193 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55acdd23f770 0x55acdd243140 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:28.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.380+0000 7ff987400700 0 client.15264 ms_handle_reset on v2:172.21.15.164:6834/2340428470 2024-05-21T21:56:28.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.380+0000 7ff9b2990b80 1 -- 172.21.15.151:0/3610085193 >> 172.21.15.151:0/3610085193 conn(0x55acdd167b90 msgr2=0x55acdd233f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:56:28.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.381+0000 7ff9b2990b80 1 -- 172.21.15.151:0/3610085193 shutdown_connections 2024-05-21T21:56:28.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.381+0000 7ff9b2990b80 1 -- 172.21.15.151:0/3610085193 wait complete. 2024-05-21T21:56:28.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.381+0000 7ff9b2990b80 20 client.15264 trim_cache size 0 max 16384 2024-05-21T21:56:28.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.404+0000 7ff9b2990b80 1 Processor -- start 2024-05-21T21:56:28.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.404+0000 7ff9b2990b80 1 -- start start 2024-05-21T21:56:28.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.404+0000 7ff9b2990b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55acdd23a340 0x55acdd239d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:28.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.404+0000 7ff9b2990b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55acdd23f770 0x55acdd2346f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:28.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.404+0000 7ff9b2990b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55acdd240090 0x55acdd242f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:28.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.404+0000 7ff9b2990b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55acdd252b70 con 0x55acdd23a340 2024-05-21T21:56:28.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.404+0000 7ff9b2990b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55acdd253240 con 0x55acdd23f770 2024-05-21T21:56:28.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.404+0000 7ff9b2990b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55acdd253910 con 0x55acdd240090 2024-05-21T21:56:28.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.405+0000 7ff9a6000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55acdd23a340 0x55acdd239d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:28.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.405+0000 7ff9a6000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55acdd23a340 0x55acdd239d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:58486/0 (socket says 172.21.15.151:58486) 2024-05-21T21:56:28.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.405+0000 7ff9a6000700 1 -- 172.21.15.151:0/1230319039 learned_addr learned my addr 172.21.15.151:0/1230319039 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:56:28.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.405+0000 7ff9a4c00700 1 --2- 172.21.15.151:0/1230319039 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55acdd23f770 0x55acdd2346f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:28.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.405+0000 7ff9a6000700 1 --2- 172.21.15.151:0/1230319039 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55acdd240090 0x55acdd242f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:28.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.405+0000 7ff9a6000700 1 -- 172.21.15.151:0/1230319039 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55acdd240090 msgr2=0x55acdd242f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:56:28.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.405+0000 7ff9a6000700 1 --2- 172.21.15.151:0/1230319039 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55acdd240090 0x55acdd242f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:28.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.405+0000 7ff9a6000700 1 -- 172.21.15.151:0/1230319039 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55acdd23f770 msgr2=0x55acdd2346f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:56:28.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.405+0000 7ff9a6000700 1 --2- 172.21.15.151:0/1230319039 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55acdd23f770 0x55acdd2346f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:28.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.405+0000 7ff9a6000700 1 -- 172.21.15.151:0/1230319039 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55acdd2459e0 con 0x55acdd23a340 2024-05-21T21:56:28.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.405+0000 7ff9a4c00700 1 --2- 172.21.15.151:0/1230319039 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55acdd23f770 0x55acdd2346f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:56:28.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.406+0000 7ff9a6000700 1 --2- 172.21.15.151:0/1230319039 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55acdd23a340 0x55acdd239d20 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff99800d010 tx=0x7ff998009300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e8f1f77b27af8b1 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:56:28.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.406+0000 7ff992000700 1 -- 172.21.15.151:0/1230319039 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff998032070 con 0x55acdd23a340 2024-05-21T21:56:28.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.406+0000 7ff992000700 1 -- 172.21.15.151:0/1230319039 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff99802d330 con 0x55acdd23a340 2024-05-21T21:56:28.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.406+0000 7ff992000700 1 -- 172.21.15.151:0/1230319039 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff998020690 con 0x55acdd23a340 2024-05-21T21:56:28.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.407+0000 7ff9b2990b80 1 -- 172.21.15.151:0/1230319039 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55acdd23a340 msgr2=0x55acdd239d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:28.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.407+0000 7ff9b2990b80 1 --2- 172.21.15.151:0/1230319039 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55acdd23a340 0x55acdd239d20 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff99800d010 tx=0x7ff998009300 comp rx=0 tx=0).stop 2024-05-21T21:56:28.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.407+0000 7ff9b2990b80 1 -- 172.21.15.151:0/1230319039 shutdown_connections 2024-05-21T21:56:28.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.407+0000 7ff9b2990b80 1 --2- 172.21.15.151:0/1230319039 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55acdd240090 0x55acdd242f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:28.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.407+0000 7ff9b2990b80 1 --2- 172.21.15.151:0/1230319039 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55acdd23a340 0x55acdd239d20 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:28.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.407+0000 7ff9b2990b80 1 --2- 172.21.15.151:0/1230319039 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55acdd23f770 0x55acdd2346f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:28.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.407+0000 7ff9b2990b80 1 -- 172.21.15.151:0/1230319039 >> 172.21.15.151:0/1230319039 conn(0x55acdd167b90 msgr2=0x55acdd236180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:56:28.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.407+0000 7ff9b2990b80 1 -- 172.21.15.151:0/1230319039 shutdown_connections 2024-05-21T21:56:28.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.407+0000 7ff9b2990b80 1 -- 172.21.15.151:0/1230319039 wait complete. 2024-05-21T21:56:28.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.407+0000 7ff9b2990b80 1 -- start start 2024-05-21T21:56:28.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.408+0000 7ff9b2990b80 1 Processor -- start 2024-05-21T21:56:28.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.408+0000 7ff9b2990b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55acdd23a340 0x55acdd233d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:28.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.408+0000 7ff9b2990b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55acdd23f770 0x55acdd234240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:28.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.408+0000 7ff9b2990b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55acdd240090 0x55acdd234780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:28.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.408+0000 7ff9b2990b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55acdd2546b0 con 0x55acdd240090 2024-05-21T21:56:28.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.408+0000 7ff9b2990b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55acdd253fe0 con 0x55acdd23f770 2024-05-21T21:56:28.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.408+0000 7ff9b2990b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55acdd254d80 con 0x55acdd23a340 2024-05-21T21:56:28.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.408+0000 7ff9b2990b80 10 client.0 _ll_register_callbacks cb 0x55acdd131c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:56:28.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.409+0000 7ff9a6000700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55acdd23a340 0x55acdd233d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:28.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.409+0000 7ff9a4c00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55acdd23f770 0x55acdd234240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:28.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.409+0000 7ff9a6000700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55acdd23a340 0x55acdd233d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:48182/0 (socket says 172.21.15.151:48182) 2024-05-21T21:56:28.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.409+0000 7ff9a6000700 1 -- 172.21.15.151:0/1191037612 learned_addr learned my addr 172.21.15.151:0/1191037612 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:56:28.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.409+0000 7ff9a6000700 1 --2- 172.21.15.151:0/1191037612 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55acdd240090 0x55acdd234780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:28.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.409+0000 7ff9a4c00700 1 -- 172.21.15.151:0/1191037612 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55acdd23a340 msgr2=0x55acdd233d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:56:28.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.409+0000 7ff9a4c00700 1 --2- 172.21.15.151:0/1191037612 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55acdd23a340 0x55acdd233d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:28.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.409+0000 7ff9a4c00700 1 -- 172.21.15.151:0/1191037612 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55acdd240090 msgr2=0x55acdd234780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:56:28.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.409+0000 7ff9a4c00700 1 --2- 172.21.15.151:0/1191037612 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55acdd240090 0x55acdd234780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:28.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.409+0000 7ff9a4c00700 1 -- 172.21.15.151:0/1191037612 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55acdd25c310 con 0x55acdd23f770 2024-05-21T21:56:28.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.409+0000 7ff9a4c00700 1 --2- 172.21.15.151:0/1191037612 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55acdd23f770 0x55acdd234240 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff98c000c00 tx=0x7ff98c005710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91b445a65dc2b6a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:56:28.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.409+0000 7ff987400700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-21T21:56:28.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.410+0000 7ff987400700 1 -- 172.21.15.151:0/1191037612 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff98c026040 con 0x55acdd23f770 2024-05-21T21:56:28.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.410+0000 7ff9b2990b80 1 -- 172.21.15.151:0/1191037612 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff99802d330 con 0x55acdd23f770 2024-05-21T21:56:28.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.410+0000 7ff987400700 1 -- 172.21.15.151:0/1191037612 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7ff98c00edb0 con 0x55acdd23f770 2024-05-21T21:56:28.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.410+0000 7ff987400700 1 -- 172.21.15.151:0/1191037612 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff98c004ac0 con 0x55acdd23f770 2024-05-21T21:56:28.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.410+0000 7ff9b2990b80 10 client.25391 fetch_fsmap learned FSMap version 29 2024-05-21T21:56:28.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.410+0000 7ff9b2990b80 1 -- 172.21.15.151:0/1191037612 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55acdd2beef0 con 0x55acdd23f770 2024-05-21T21:56:28.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.410+0000 7ff987400700 1 -- 172.21.15.151:0/1191037612 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff99802d330 con 0x55acdd23f770 2024-05-21T21:56:28.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.411+0000 7ff987400700 1 -- 172.21.15.151:0/1191037612 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7ff98c022b20 con 0x55acdd23f770 2024-05-21T21:56:28.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.411+0000 7ff9b2990b80 10 client.25391 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T21:56:28.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.411+0000 7ff9b2990b80 10 client.25391 resolve_mds: resolved name 'e' to daemon mds.e 2024-05-21T21:56:28.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.411+0000 7ff9b2990b80 20 client.25391 populate_metadata read hostname 'smithi151' 2024-05-21T21:56:28.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.411+0000 7ff9b2990b80 1 --2- 172.21.15.151:0/1191037612 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x55acdd2c3910 0x55acdd2c2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:28.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.411+0000 7ff9b2990b80 4 client.25391 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-21T21:56:28.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.411+0000 7ff9b2990b80 1 -- 172.21.15.151:0/1191037612 --> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] -- command(tid 0: {"prefix": "status"}) -- 0x55acdd193520 con 0x55acdd2c3910 2024-05-21T21:56:28.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.411+0000 7ff9a6000700 1 --2- 172.21.15.151:0/1191037612 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x55acdd2c3910 0x55acdd2c2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:28.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.412+0000 7ff9a6000700 1 --2- 172.21.15.151:0/1191037612 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x55acdd2c3910 0x55acdd2c2620 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=de24d895917ed1b7 server_cookie=4030122e273413bc in_seq=0 out_seq=0 2024-05-21T21:56:28.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.412+0000 7ff987400700 10 client.25391 ms_handle_connect on v2:172.21.15.164:6834/2340428470 2024-05-21T21:56:28.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.412+0000 7ff987400700 1 -- 172.21.15.151:0/1191037612 <== mds.4 v2:172.21.15.164:6834/2340428470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55acdd193520 con 0x55acdd2c3910 2024-05-21T21:56:28.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.412+0000 7ff987400700 10 client.25391 handle_command_reply: tid=0 2024-05-21T21:56:28.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.412+0000 7ff9b2990b80 1 client.25391 shutdown 2024-05-21T21:56:28.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.412+0000 7ff9b2990b80 1 -- 172.21.15.151:0/1191037612 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55acdd23f770 msgr2=0x55acdd234240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:28.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.412+0000 7ff9b2990b80 1 --2- 172.21.15.151:0/1191037612 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55acdd23f770 0x55acdd234240 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff98c000c00 tx=0x7ff98c005710 comp rx=0 tx=0).stop 2024-05-21T21:56:28.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.412+0000 7ff9b2990b80 1 -- 172.21.15.151:0/1191037612 shutdown_connections 2024-05-21T21:56:28.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.412+0000 7ff9b2990b80 1 --2- 172.21.15.151:0/1191037612 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x55acdd2c3910 0x55acdd2c2620 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:28.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.412+0000 7ff9b2990b80 1 --2- 172.21.15.151:0/1191037612 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55acdd240090 0x55acdd234780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:28.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.412+0000 7ff9b2990b80 1 --2- 172.21.15.151:0/1191037612 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55acdd23a340 0x55acdd233d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:28.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.412+0000 7ff987400700 0 client.25391 ms_handle_reset on v2:172.21.15.164:6834/2340428470 2024-05-21T21:56:28.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.412+0000 7ff9b2990b80 1 --2- 172.21.15.151:0/1191037612 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55acdd23f770 0x55acdd234240 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:28.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.412+0000 7ff9b2990b80 1 -- 172.21.15.151:0/1191037612 >> 172.21.15.151:0/1191037612 conn(0x55acdd167b90 msgr2=0x55acdd242f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:56:28.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.412+0000 7ff9b2990b80 1 -- 172.21.15.151:0/1191037612 shutdown_connections 2024-05-21T21:56:28.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.412+0000 7ff9b2990b80 1 -- 172.21.15.151:0/1191037612 wait complete. 2024-05-21T21:56:28.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.412+0000 7ff9b2990b80 20 client.25391 trim_cache size 0 max 16384 2024-05-21T21:56:28.416 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-05-21T21:56:28.416 INFO:teuthology.orchestra.run.smithi151.stdout: "cluster_fsid": "c8c7a2e6-17ba-11ef-bc9a-c7b262605968", 2024-05-21T21:56:28.416 INFO:teuthology.orchestra.run.smithi151.stdout: "whoami": 4, 2024-05-21T21:56:28.416 INFO:teuthology.orchestra.run.smithi151.stdout: "id": 24473, 2024-05-21T21:56:28.416 INFO:teuthology.orchestra.run.smithi151.stdout: "want_state": "up:active", 2024-05-21T21:56:28.416 INFO:teuthology.orchestra.run.smithi151.stdout: "state": "up:active", 2024-05-21T21:56:28.417 INFO:teuthology.orchestra.run.smithi151.stdout: "fs_name": "cephfs", 2024-05-21T21:56:28.417 INFO:teuthology.orchestra.run.smithi151.stdout: "rank_uptime": 290.07947178799998, 2024-05-21T21:56:28.417 INFO:teuthology.orchestra.run.smithi151.stdout: "mdsmap_epoch": 29, 2024-05-21T21:56:28.417 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_epoch": 83, 2024-05-21T21:56:28.417 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_epoch_barrier": 82, 2024-05-21T21:56:28.417 INFO:teuthology.orchestra.run.smithi151.stdout: "uptime": 353.19129125799998 2024-05-21T21:56:28.417 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-05-21T21:56:28.417 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.413+0000 7ff9a6a00700 1 -- 172.21.15.151:0/2905543758 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7ff9880b2b90 msgr2=0x7ff9880b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:28.417 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.413+0000 7ff9a6a00700 1 --2- 172.21.15.151:0/2905543758 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7ff9880b2b90 0x7ff9880b5050 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff9940e0100 tx=0x7ff9940defc0 comp rx=0 tx=0).stop 2024-05-21T21:56:28.417 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.413+0000 7ff9a6a00700 1 -- 172.21.15.151:0/2905543758 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff9a0077a70 msgr2=0x7ff9a01a0120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:28.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.413+0000 7ff9a6a00700 1 --2- 172.21.15.151:0/2905543758 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff9a0077a70 0x7ff9a01a0120 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff99400e9c0 tx=0x7ff99400ecd0 comp rx=0 tx=0).stop 2024-05-21T21:56:28.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.413+0000 7ff9a6a00700 1 -- 172.21.15.151:0/2905543758 shutdown_connections 2024-05-21T21:56:28.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.413+0000 7ff9a6a00700 1 --2- 172.21.15.151:0/2905543758 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7ff9880b2b90 0x7ff9880b5050 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:28.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.413+0000 7ff9a6a00700 1 --2- 172.21.15.151:0/2905543758 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff9a01142f0 0x7ff9a01a49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:28.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.413+0000 7ff9a6a00700 1 --2- 172.21.15.151:0/2905543758 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff9a0077a70 0x7ff9a01a0120 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:28.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.413+0000 7ff9a6a00700 1 --2- 172.21.15.151:0/2905543758 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9a0078390 0x7ff9a01a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:28.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.413+0000 7ff9a6a00700 1 -- 172.21.15.151:0/2905543758 >> 172.21.15.151:0/2905543758 conn(0x7ff9a0101180 msgr2=0x7ff9a0103570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:56:28.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.414+0000 7ff9a6a00700 1 -- 172.21.15.151:0/2905543758 shutdown_connections 2024-05-21T21:56:28.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:28.414+0000 7ff9a6a00700 1 -- 172.21.15.151:0/2905543758 wait complete. 2024-05-21T21:56:29.446 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.c status 2024-05-21T21:56:29.637 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-21T21:56:29.702 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:56:29 smithi151 ceph-mon[31623]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 938 B/s wr, 13 op/s 2024-05-21T21:56:29.898 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:56:29.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:56:29 smithi184 ceph-mon[36942]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 938 B/s wr, 13 op/s 2024-05-21T21:56:30.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:56:29 smithi164 ceph-mon[36269]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 938 B/s wr, 13 op/s 2024-05-21T21:56:31.132 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:56:31.680 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:56:31 smithi151 ceph-mon[31623]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 938 B/s wr, 8 op/s 2024-05-21T21:56:31.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:56:31 smithi184 ceph-mon[36942]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 938 B/s wr, 8 op/s 2024-05-21T21:56:32.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:56:31 smithi164 ceph-mon[36269]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 938 B/s wr, 8 op/s 2024-05-21T21:56:32.056 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.056+0000 7f242d800700 1 -- 172.21.15.151:0/3797600377 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f242810b670 msgr2=0x7f242810ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:32.057 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.056+0000 7f242d800700 1 --2- 172.21.15.151:0/3797600377 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f242810b670 0x7f242810ba50 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f24180098f0 tx=0x7f2418009c00 comp rx=0 tx=0).stop 2024-05-21T21:56:32.057 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.056+0000 7f242d800700 1 -- 172.21.15.151:0/3797600377 shutdown_connections 2024-05-21T21:56:32.057 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.056+0000 7f242d800700 1 --2- 172.21.15.151:0/3797600377 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f242810b670 0x7f242810ba50 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:32.057 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.056+0000 7f242d800700 1 --2- 172.21.15.151:0/3797600377 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2428106030 0x7f2428112540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:32.057 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.056+0000 7f242d800700 1 --2- 172.21.15.151:0/3797600377 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2428105670 0x7f2428105af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:32.057 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.056+0000 7f242d800700 1 -- 172.21.15.151:0/3797600377 >> 172.21.15.151:0/3797600377 conn(0x7f2428101160 msgr2=0x7f2428103580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:56:32.057 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.056+0000 7f242d800700 1 -- 172.21.15.151:0/3797600377 shutdown_connections 2024-05-21T21:56:32.057 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.056+0000 7f242d800700 1 -- 172.21.15.151:0/3797600377 wait complete. 2024-05-21T21:56:32.058 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.057+0000 7f242d800700 1 Processor -- start 2024-05-21T21:56:32.058 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.057+0000 7f242d800700 1 -- start start 2024-05-21T21:56:32.058 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.058+0000 7f242d800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2428105670 0x7f242819bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:32.058 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.058+0000 7f242d800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2428106030 0x7f242819c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:32.058 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.058+0000 7f242d800700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f242810b670 0x7f24281a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:32.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.058+0000 7f242d800700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f24281158c0 con 0x7f2428106030 2024-05-21T21:56:32.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.058+0000 7f242d800700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2428115740 con 0x7f2428105670 2024-05-21T21:56:32.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.058+0000 7f242d800700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2428115a40 con 0x7f242810b670 2024-05-21T21:56:32.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.058+0000 7f242ce00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f242810b670 0x7f24281a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:32.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.058+0000 7f242ce00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f242810b670 0x7f24281a06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:48200/0 (socket says 172.21.15.151:48200) 2024-05-21T21:56:32.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.058+0000 7f242ce00700 1 -- 172.21.15.151:0/2452055674 learned_addr learned my addr 172.21.15.151:0/2452055674 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:56:32.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.058+0000 7f2427e00700 1 --2- 172.21.15.151:0/2452055674 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2428105670 0x7f242819bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:32.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.058+0000 7f2427400700 1 --2- 172.21.15.151:0/2452055674 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2428106030 0x7f242819c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:32.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.059+0000 7f242ce00700 1 -- 172.21.15.151:0/2452055674 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2428105670 msgr2=0x7f242819bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:32.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.059+0000 7f242ce00700 1 --2- 172.21.15.151:0/2452055674 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2428105670 0x7f242819bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:32.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.059+0000 7f242ce00700 1 -- 172.21.15.151:0/2452055674 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2428106030 msgr2=0x7f242819c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:32.061 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.059+0000 7f242ce00700 1 --2- 172.21.15.151:0/2452055674 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2428106030 0x7f242819c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:32.061 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.059+0000 7f242ce00700 1 -- 172.21.15.151:0/2452055674 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24281a0de0 con 0x7f242810b670 2024-05-21T21:56:32.061 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.059+0000 7f2427400700 1 --2- 172.21.15.151:0/2452055674 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2428106030 0x7f242819c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T21:56:32.061 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.059+0000 7f2427e00700 1 --2- 172.21.15.151:0/2452055674 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2428105670 0x7f242819bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:56:32.061 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.059+0000 7f242ce00700 1 --2- 172.21.15.151:0/2452055674 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f242810b670 0x7f24281a06a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f242000e3c0 tx=0x7f242000e6d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:56:32.061 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.059+0000 7f2424c00700 1 -- 172.21.15.151:0/2452055674 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24200092d0 con 0x7f242810b670 2024-05-21T21:56:32.061 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.059+0000 7f242d800700 1 -- 172.21.15.151:0/2452055674 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24281a10d0 con 0x7f242810b670 2024-05-21T21:56:32.061 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.059+0000 7f2424c00700 1 -- 172.21.15.151:0/2452055674 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f242000ee70 con 0x7f242810b670 2024-05-21T21:56:32.061 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.059+0000 7f2424c00700 1 -- 172.21.15.151:0/2452055674 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2420004030 con 0x7f242810b670 2024-05-21T21:56:32.061 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.059+0000 7f242d800700 1 -- 172.21.15.151:0/2452055674 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24281a8950 con 0x7f242810b670 2024-05-21T21:56:32.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.063+0000 7f2424c00700 1 -- 172.21.15.151:0/2452055674 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 26) ==== 149886+0+0 (secure 0 0 0) 0x7f2420012050 con 0x7f242810b670 2024-05-21T21:56:32.064 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.064+0000 7f243969db80 1 Processor -- start 2024-05-21T21:56:32.064 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.064+0000 7f243969db80 1 -- start start 2024-05-21T21:56:32.065 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.065+0000 7f243969db80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a4a17fffd0 0x55a4a18003b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:32.065 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.065+0000 7f243969db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a4a18008f0 0x55a4a17fa460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:32.065 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.065+0000 7f243969db80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4a17faa80 0x55a4a17faee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:32.065 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.065+0000 7f243969db80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a4a1758df0 con 0x55a4a17fffd0 2024-05-21T21:56:32.065 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.065+0000 7f243969db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55a4a1748990 con 0x55a4a18008f0 2024-05-21T21:56:32.065 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.065+0000 7f243969db80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55a4a174c9d0 con 0x55a4a17faa80 2024-05-21T21:56:32.068 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.065+0000 7f2427400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a4a17fffd0 0x55a4a18003b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:32.068 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.065+0000 7f2427e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a4a18008f0 0x55a4a17fa460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:32.068 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.065+0000 7f2427400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a4a17fffd0 0x55a4a18003b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:58512/0 (socket says 172.21.15.151:58512) 2024-05-21T21:56:32.068 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.065+0000 7f2427e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a4a18008f0 0x55a4a17fa460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:35290/0 (socket says 172.21.15.151:35290) 2024-05-21T21:56:32.069 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.065+0000 7f2424c00700 1 --2- 172.21.15.151:0/2452055674 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f240c0b2b10 0x7f240c0b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:32.069 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.065+0000 7f2427400700 1 -- 172.21.15.151:0/1798203956 learned_addr learned my addr 172.21.15.151:0/1798203956 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:56:32.069 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.065+0000 7f2427400700 1 --2- 172.21.15.151:0/1798203956 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4a17faa80 0x55a4a17faee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:32.069 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.065+0000 7f242ce00700 1 --2- 172.21.15.151:0/2452055674 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f240c0b2b10 0x7f240c0b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:32.069 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.066+0000 7f2424c00700 1 -- 172.21.15.151:0/2452055674 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f24200e2550 con 0x7f242810b670 2024-05-21T21:56:32.069 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.066+0000 7f2427400700 1 -- 172.21.15.151:0/1798203956 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4a17faa80 msgr2=0x55a4a17faee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:56:32.069 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.066+0000 7f2427400700 1 --2- 172.21.15.151:0/1798203956 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4a17faa80 0x55a4a17faee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:32.069 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.066+0000 7f2427400700 1 -- 172.21.15.151:0/1798203956 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a4a18008f0 msgr2=0x55a4a17fa460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:56:32.070 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.066+0000 7f2427400700 1 --2- 172.21.15.151:0/1798203956 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a4a18008f0 0x55a4a17fa460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:32.070 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.066+0000 7f2427400700 1 -- 172.21.15.151:0/1798203956 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4a17113d0 con 0x55a4a17fffd0 2024-05-21T21:56:32.070 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.066+0000 7f2427400700 1 --2- 172.21.15.151:0/1798203956 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a4a17fffd0 0x55a4a18003b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f24140138a0 tx=0x7f2414009770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e3997533515d94b server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:56:32.070 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.066+0000 7f242ce00700 1 --2- 172.21.15.151:0/2452055674 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f240c0b2b10 0x7f240c0b4fd0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f24200dfb30 tx=0x7f2420009c00 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:56:32.070 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.066+0000 7f241d600700 1 -- 172.21.15.151:0/1798203956 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24140130d0 con 0x55a4a17fffd0 2024-05-21T21:56:32.070 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.067+0000 7f241d600700 1 -- 172.21.15.151:0/1798203956 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2414011920 con 0x55a4a17fffd0 2024-05-21T21:56:32.070 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.067+0000 7f241d600700 1 -- 172.21.15.151:0/1798203956 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f241400f640 con 0x55a4a17fffd0 2024-05-21T21:56:32.070 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.067+0000 7f243969db80 1 -- 172.21.15.151:0/1798203956 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a4a17fffd0 msgr2=0x55a4a18003b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:32.070 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.067+0000 7f243969db80 1 --2- 172.21.15.151:0/1798203956 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a4a17fffd0 0x55a4a18003b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f24140138a0 tx=0x7f2414009770 comp rx=0 tx=0).stop 2024-05-21T21:56:32.070 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.067+0000 7f243969db80 1 -- 172.21.15.151:0/1798203956 shutdown_connections 2024-05-21T21:56:32.070 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.067+0000 7f243969db80 1 --2- 172.21.15.151:0/1798203956 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4a17faa80 0x55a4a17faee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:32.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.067+0000 7f243969db80 1 --2- 172.21.15.151:0/1798203956 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a4a17fffd0 0x55a4a18003b0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:32.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.067+0000 7f243969db80 1 --2- 172.21.15.151:0/1798203956 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a4a18008f0 0x55a4a17fa460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:32.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.067+0000 7f243969db80 1 -- 172.21.15.151:0/1798203956 >> 172.21.15.151:0/1798203956 conn(0x55a4a1728b90 msgr2=0x55a4a17f5100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:56:32.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.068+0000 7f243969db80 1 -- 172.21.15.151:0/1798203956 shutdown_connections 2024-05-21T21:56:32.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.068+0000 7f243969db80 1 -- 172.21.15.151:0/1798203956 wait complete. 2024-05-21T21:56:32.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.068+0000 7f243969db80 1 -- start start 2024-05-21T21:56:32.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.068+0000 7f243969db80 1 Processor -- start 2024-05-21T21:56:32.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.069+0000 7f243969db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a4a17faa80 0x55a4a18065b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:32.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.069+0000 7f243969db80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4a17fffd0 0x55a4a1803950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:32.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.069+0000 7f243969db80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a4a18008f0 0x55a4a1803e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:32.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.069+0000 7f243969db80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a4a164ef50 con 0x55a4a18008f0 2024-05-21T21:56:32.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.069+0000 7f243969db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55a4a16e9710 con 0x55a4a17faa80 2024-05-21T21:56:32.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.069+0000 7f243969db80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55a4a18073b0 con 0x55a4a17fffd0 2024-05-21T21:56:32.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.069+0000 7f243969db80 10 client.0 _ll_register_callbacks cb 0x55a4a16f2c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:56:32.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.069+0000 7f2427e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4a17fffd0 0x55a4a1803950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:32.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.069+0000 7f2427400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a4a18008f0 0x55a4a1803e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:32.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.069+0000 7f2427e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4a17fffd0 0x55a4a1803950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:48220/0 (socket says 172.21.15.151:48220) 2024-05-21T21:56:32.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.069+0000 7f2427e00700 1 -- 172.21.15.151:0/1648702101 learned_addr learned my addr 172.21.15.151:0/1648702101 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:56:32.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.069+0000 7f2427400700 1 --2- 172.21.15.151:0/1648702101 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a4a17faa80 0x55a4a18065b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:32.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.070+0000 7f2427e00700 1 -- 172.21.15.151:0/1648702101 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a4a17faa80 msgr2=0x55a4a18065b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:56:32.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.070+0000 7f2427e00700 1 --2- 172.21.15.151:0/1648702101 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a4a17faa80 0x55a4a18065b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:32.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.070+0000 7f2427e00700 1 -- 172.21.15.151:0/1648702101 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a4a18008f0 msgr2=0x55a4a1803e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:56:32.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.070+0000 7f2427e00700 1 --2- 172.21.15.151:0/1648702101 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a4a18008f0 0x55a4a1803e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:32.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.070+0000 7f2427e00700 1 -- 172.21.15.151:0/1648702101 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4a1698880 con 0x55a4a17fffd0 2024-05-21T21:56:32.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.070+0000 7f2427400700 1 --2- 172.21.15.151:0/1648702101 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a4a18008f0 0x55a4a1803e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T21:56:32.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.070+0000 7f2427e00700 1 --2- 172.21.15.151:0/1648702101 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4a17fffd0 0x55a4a1803950 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2418009230 tx=0x7f24180094f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b2315b1ac407412 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:56:32.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.070+0000 7f2412000700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-21T21:56:32.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.070+0000 7f2412000700 1 -- 172.21.15.151:0/1648702101 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2418005510 con 0x55a4a17fffd0 2024-05-21T21:56:32.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.070+0000 7f2412000700 1 -- 172.21.15.151:0/1648702101 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f2418003680 con 0x55a4a17fffd0 2024-05-21T21:56:32.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.070+0000 7f243969db80 1 -- 172.21.15.151:0/1648702101 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2414011920 con 0x55a4a17fffd0 2024-05-21T21:56:32.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.071+0000 7f2412000700 1 -- 172.21.15.151:0/1648702101 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2418031050 con 0x55a4a17fffd0 2024-05-21T21:56:32.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.071+0000 7f243969db80 10 client.25411 fetch_fsmap learned FSMap version 29 2024-05-21T21:56:32.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.071+0000 7f243969db80 1 -- 172.21.15.151:0/1648702101 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a4a1719010 con 0x55a4a17fffd0 2024-05-21T21:56:32.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.071+0000 7f2412000700 1 -- 172.21.15.151:0/1648702101 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2414011920 con 0x55a4a17fffd0 2024-05-21T21:56:32.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.071+0000 7f2412000700 1 -- 172.21.15.151:0/1648702101 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f241800b900 con 0x55a4a17fffd0 2024-05-21T21:56:32.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.071+0000 7f243969db80 10 client.25411 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T21:56:32.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.071+0000 7f243969db80 10 client.25411 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-21T21:56:32.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.071+0000 7f243969db80 20 client.25411 populate_metadata read hostname 'smithi151' 2024-05-21T21:56:32.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.071+0000 7f243969db80 1 --2- 172.21.15.151:0/1648702101 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x55a4a181c720 0x55a4a181db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:32.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.072+0000 7f243969db80 4 client.25411 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T21:56:32.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.072+0000 7f243969db80 1 -- 172.21.15.151:0/1648702101 --> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a4a175ad40 con 0x55a4a181c720 2024-05-21T21:56:32.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.072+0000 7f2427400700 1 --2- 172.21.15.151:0/1648702101 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x55a4a181c720 0x55a4a181db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:32.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.072+0000 7f2427400700 1 --2- 172.21.15.151:0/1648702101 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x55a4a181c720 0x55a4a181db90 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=146046d9a8161dec server_cookie=b173898eeaccb38c in_seq=0 out_seq=0 2024-05-21T21:56:32.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.072+0000 7f2412000700 10 client.25411 ms_handle_connect on v2:172.21.15.184:6836/2988194888 2024-05-21T21:56:32.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.073+0000 7f2412000700 1 -- 172.21.15.151:0/1648702101 <== mds.3 v2:172.21.15.184:6836/2988194888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55a4a175ad40 con 0x55a4a181c720 2024-05-21T21:56:32.078 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.073+0000 7f2412000700 10 client.25411 handle_command_reply: tid=0 2024-05-21T21:56:32.078 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.073+0000 7f243969db80 1 client.25411 shutdown 2024-05-21T21:56:32.078 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.074+0000 7f243969db80 1 -- 172.21.15.151:0/1648702101 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4a17fffd0 msgr2=0x55a4a1803950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:32.078 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.074+0000 7f243969db80 1 --2- 172.21.15.151:0/1648702101 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4a17fffd0 0x55a4a1803950 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2418009230 tx=0x7f24180094f0 comp rx=0 tx=0).stop 2024-05-21T21:56:32.078 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.074+0000 7f243969db80 1 -- 172.21.15.151:0/1648702101 shutdown_connections 2024-05-21T21:56:32.078 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.074+0000 7f243969db80 1 --2- 172.21.15.151:0/1648702101 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x55a4a181c720 0x55a4a181db90 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:32.078 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.074+0000 7f243969db80 1 --2- 172.21.15.151:0/1648702101 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a4a17faa80 0x55a4a18065b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:32.078 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.074+0000 7f243969db80 1 --2- 172.21.15.151:0/1648702101 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a4a18008f0 0x55a4a1803e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:32.078 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.074+0000 7f243969db80 1 --2- 172.21.15.151:0/1648702101 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4a17fffd0 0x55a4a1803950 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:32.078 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.074+0000 7f2412000700 0 client.25411 ms_handle_reset on v2:172.21.15.184:6836/2988194888 2024-05-21T21:56:32.078 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.074+0000 7f243969db80 1 -- 172.21.15.151:0/1648702101 >> 172.21.15.151:0/1648702101 conn(0x55a4a1728b90 msgr2=0x55a4a17f9450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:56:32.078 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.074+0000 7f243969db80 1 -- 172.21.15.151:0/1648702101 shutdown_connections 2024-05-21T21:56:32.078 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.074+0000 7f243969db80 1 -- 172.21.15.151:0/1648702101 wait complete. 2024-05-21T21:56:32.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.074+0000 7f243969db80 20 client.25411 trim_cache size 0 max 16384 2024-05-21T21:56:32.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.099+0000 7f243969db80 1 Processor -- start 2024-05-21T21:56:32.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.099+0000 7f243969db80 1 -- start start 2024-05-21T21:56:32.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.099+0000 7f243969db80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4a17faa80 0x55a4a17fa460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:32.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.099+0000 7f243969db80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a4a17fffd0 0x55a4a1802240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:32.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.099+0000 7f243969db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a4a18008f0 0x55a4a1802780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:32.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.099+0000 7f243969db80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a4a181bed0 con 0x55a4a17fffd0 2024-05-21T21:56:32.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.099+0000 7f243969db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55a4a181c5a0 con 0x55a4a18008f0 2024-05-21T21:56:32.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.099+0000 7f243969db80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55a4a181cc70 con 0x55a4a17faa80 2024-05-21T21:56:32.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.100+0000 7f2427400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4a17faa80 0x55a4a17fa460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:32.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.100+0000 7f2427400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4a17faa80 0x55a4a17fa460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:48224/0 (socket says 172.21.15.151:48224) 2024-05-21T21:56:32.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.100+0000 7f2427e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a4a17fffd0 0x55a4a1802240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:32.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.100+0000 7f2427400700 1 -- 172.21.15.151:0/971581939 learned_addr learned my addr 172.21.15.151:0/971581939 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:56:32.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.100+0000 7f2427400700 1 --2- 172.21.15.151:0/971581939 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a4a18008f0 0x55a4a1802780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:32.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.100+0000 7f2427400700 1 -- 172.21.15.151:0/971581939 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a4a18008f0 msgr2=0x55a4a1802780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:56:32.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.100+0000 7f2427400700 1 --2- 172.21.15.151:0/971581939 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a4a18008f0 0x55a4a1802780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:32.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.100+0000 7f2427400700 1 -- 172.21.15.151:0/971581939 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a4a17fffd0 msgr2=0x55a4a1802240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:56:32.102 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.100+0000 7f2427400700 1 --2- 172.21.15.151:0/971581939 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a4a17fffd0 0x55a4a1802240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:32.102 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.100+0000 7f2427400700 1 -- 172.21.15.151:0/971581939 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4a17f6860 con 0x55a4a17faa80 2024-05-21T21:56:32.102 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.100+0000 7f2427400700 1 --2- 172.21.15.151:0/971581939 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4a17faa80 0x55a4a17fa460 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2414009850 tx=0x7f2414013310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf9f6f4a9a0cf23 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:56:32.102 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.101+0000 7f241cc00700 1 -- 172.21.15.151:0/971581939 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2414005030 con 0x55a4a17faa80 2024-05-21T21:56:32.102 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.101+0000 7f241cc00700 1 -- 172.21.15.151:0/971581939 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24140046f0 con 0x55a4a17faa80 2024-05-21T21:56:32.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.101+0000 7f241cc00700 1 -- 172.21.15.151:0/971581939 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24140115f0 con 0x55a4a17faa80 2024-05-21T21:56:32.106 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.101+0000 7f243969db80 1 -- 172.21.15.151:0/971581939 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4a17faa80 msgr2=0x55a4a17fa460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:32.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.102+0000 7f243969db80 1 --2- 172.21.15.151:0/971581939 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4a17faa80 0x55a4a17fa460 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2414009850 tx=0x7f2414013310 comp rx=0 tx=0).stop 2024-05-21T21:56:32.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.102+0000 7f243969db80 1 -- 172.21.15.151:0/971581939 shutdown_connections 2024-05-21T21:56:32.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.102+0000 7f243969db80 1 --2- 172.21.15.151:0/971581939 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a4a18008f0 0x55a4a1802780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:32.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.102+0000 7f243969db80 1 --2- 172.21.15.151:0/971581939 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4a17faa80 0x55a4a17fa460 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:32.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.102+0000 7f243969db80 1 --2- 172.21.15.151:0/971581939 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a4a17fffd0 0x55a4a1802240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:32.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.102+0000 7f243969db80 1 -- 172.21.15.151:0/971581939 >> 172.21.15.151:0/971581939 conn(0x55a4a1728b90 msgr2=0x55a4a17fdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:56:32.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.102+0000 7f243969db80 1 -- 172.21.15.151:0/971581939 shutdown_connections 2024-05-21T21:56:32.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.102+0000 7f243969db80 1 -- 172.21.15.151:0/971581939 wait complete. 2024-05-21T21:56:32.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.102+0000 7f243969db80 1 -- start start 2024-05-21T21:56:32.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.102+0000 7f243969db80 1 Processor -- start 2024-05-21T21:56:32.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.103+0000 7f243969db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a4a17faa80 0x55a4a1827010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:32.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.103+0000 7f243969db80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4a17fffd0 0x55a4a1827550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:32.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.103+0000 7f243969db80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a4a18008f0 0x55a4a1821fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:32.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.103+0000 7f243969db80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a4a181da10 con 0x55a4a18008f0 2024-05-21T21:56:32.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.103+0000 7f243969db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55a4a181d340 con 0x55a4a17faa80 2024-05-21T21:56:32.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.103+0000 7f243969db80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55a4a181e0e0 con 0x55a4a17fffd0 2024-05-21T21:56:32.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.103+0000 7f243969db80 10 client.0 _ll_register_callbacks cb 0x55a4a16f2c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:56:32.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.103+0000 7f2427e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4a17fffd0 0x55a4a1827550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:32.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.103+0000 7f2427400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a4a18008f0 0x55a4a1821fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:32.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.103+0000 7f2427e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4a17fffd0 0x55a4a1827550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:48228/0 (socket says 172.21.15.151:48228) 2024-05-21T21:56:32.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.103+0000 7f2427e00700 1 -- 172.21.15.151:0/3271343846 learned_addr learned my addr 172.21.15.151:0/3271343846 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:56:32.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.103+0000 7f2427400700 1 --2- 172.21.15.151:0/3271343846 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a4a17faa80 0x55a4a1827010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:32.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.104+0000 7f2427e00700 1 -- 172.21.15.151:0/3271343846 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a4a17faa80 msgr2=0x55a4a1827010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:56:32.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.104+0000 7f2427e00700 1 --2- 172.21.15.151:0/3271343846 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a4a17faa80 0x55a4a1827010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:32.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.104+0000 7f2427e00700 1 -- 172.21.15.151:0/3271343846 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a4a18008f0 msgr2=0x55a4a1821fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:56:32.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.104+0000 7f2427e00700 1 --2- 172.21.15.151:0/3271343846 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a4a18008f0 0x55a4a1821fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:32.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.104+0000 7f2427e00700 1 -- 172.21.15.151:0/3271343846 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4a1822600 con 0x55a4a17fffd0 2024-05-21T21:56:32.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.104+0000 7f2427e00700 1 --2- 172.21.15.151:0/3271343846 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4a17fffd0 0x55a4a1827550 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f24180092a0 tx=0x7f2418005540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9a259a71cd2210f server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:56:32.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.104+0000 7f2412000700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-21T21:56:32.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.105+0000 7f2412000700 1 -- 172.21.15.151:0/3271343846 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f241800b970 con 0x55a4a17fffd0 2024-05-21T21:56:32.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.105+0000 7f243969db80 1 -- 172.21.15.151:0/3271343846 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24140046f0 con 0x55a4a17fffd0 2024-05-21T21:56:32.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.105+0000 7f2412000700 1 -- 172.21.15.151:0/3271343846 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f2418023850 con 0x55a4a17fffd0 2024-05-21T21:56:32.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.105+0000 7f2412000700 1 -- 172.21.15.151:0/3271343846 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2418023640 con 0x55a4a17fffd0 2024-05-21T21:56:32.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.105+0000 7f243969db80 10 client.25417 fetch_fsmap learned FSMap version 29 2024-05-21T21:56:32.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.105+0000 7f243969db80 1 -- 172.21.15.151:0/3271343846 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a4a18887f0 con 0x55a4a17fffd0 2024-05-21T21:56:32.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.105+0000 7f2412000700 1 -- 172.21.15.151:0/3271343846 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24140046f0 con 0x55a4a17fffd0 2024-05-21T21:56:32.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.105+0000 7f2412000700 1 -- 172.21.15.151:0/3271343846 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f2418003680 con 0x55a4a17fffd0 2024-05-21T21:56:32.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.105+0000 7f243969db80 10 client.25417 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T21:56:32.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.106+0000 7f243969db80 10 client.25417 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-21T21:56:32.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.106+0000 7f243969db80 20 client.25417 populate_metadata read hostname 'smithi151' 2024-05-21T21:56:32.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.106+0000 7f243969db80 1 --2- 172.21.15.151:0/3271343846 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x55a4a188d310 0x55a4a188c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:32.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.106+0000 7f243969db80 4 client.25417 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-21T21:56:32.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.106+0000 7f243969db80 1 -- 172.21.15.151:0/3271343846 --> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] -- command(tid 0: {"prefix": "status"}) -- 0x55a4a1754520 con 0x55a4a188d310 2024-05-21T21:56:32.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.106+0000 7f2427400700 1 --2- 172.21.15.151:0/3271343846 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x55a4a188d310 0x55a4a188c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:32.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.106+0000 7f2427400700 1 --2- 172.21.15.151:0/3271343846 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x55a4a188d310 0x55a4a188c020 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=cb22365ab9b76683 server_cookie=988ab98b150ef03 in_seq=0 out_seq=0 2024-05-21T21:56:32.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.106+0000 7f2412000700 10 client.25417 ms_handle_connect on v2:172.21.15.184:6836/2988194888 2024-05-21T21:56:32.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.107+0000 7f2412000700 1 -- 172.21.15.151:0/3271343846 <== mds.3 v2:172.21.15.184:6836/2988194888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55a4a1754520 con 0x55a4a188d310 2024-05-21T21:56:32.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.107+0000 7f2412000700 10 client.25417 handle_command_reply: tid=0 2024-05-21T21:56:32.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.107+0000 7f243969db80 1 client.25417 shutdown 2024-05-21T21:56:32.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.107+0000 7f243969db80 1 -- 172.21.15.151:0/3271343846 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4a17fffd0 msgr2=0x55a4a1827550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:32.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.107+0000 7f243969db80 1 --2- 172.21.15.151:0/3271343846 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4a17fffd0 0x55a4a1827550 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f24180092a0 tx=0x7f2418005540 comp rx=0 tx=0).stop 2024-05-21T21:56:32.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.107+0000 7f243969db80 1 -- 172.21.15.151:0/3271343846 shutdown_connections 2024-05-21T21:56:32.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.107+0000 7f243969db80 1 --2- 172.21.15.151:0/3271343846 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x55a4a188d310 0x55a4a188c020 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:32.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.107+0000 7f243969db80 1 --2- 172.21.15.151:0/3271343846 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a4a17faa80 0x55a4a1827010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:32.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.107+0000 7f243969db80 1 --2- 172.21.15.151:0/3271343846 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a4a18008f0 0x55a4a1821fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:32.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.107+0000 7f2412000700 0 client.25417 ms_handle_reset on v2:172.21.15.184:6836/2988194888 2024-05-21T21:56:32.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.107+0000 7f243969db80 1 --2- 172.21.15.151:0/3271343846 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4a17fffd0 0x55a4a1827550 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:32.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.107+0000 7f243969db80 1 -- 172.21.15.151:0/3271343846 >> 172.21.15.151:0/3271343846 conn(0x55a4a1728b90 msgr2=0x55a4a17f4be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:56:32.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.107+0000 7f243969db80 1 -- 172.21.15.151:0/3271343846 shutdown_connections 2024-05-21T21:56:32.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.107+0000 7f243969db80 1 -- 172.21.15.151:0/3271343846 wait complete. 2024-05-21T21:56:32.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.107+0000 7f243969db80 20 client.25417 trim_cache size 0 max 16384 2024-05-21T21:56:32.112 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-05-21T21:56:32.112 INFO:teuthology.orchestra.run.smithi151.stdout: "cluster_fsid": "c8c7a2e6-17ba-11ef-bc9a-c7b262605968", 2024-05-21T21:56:32.112 INFO:teuthology.orchestra.run.smithi151.stdout: "whoami": 3, 2024-05-21T21:56:32.113 INFO:teuthology.orchestra.run.smithi151.stdout: "id": 24478, 2024-05-21T21:56:32.113 INFO:teuthology.orchestra.run.smithi151.stdout: "want_state": "up:active", 2024-05-21T21:56:32.113 INFO:teuthology.orchestra.run.smithi151.stdout: "state": "up:active", 2024-05-21T21:56:32.113 INFO:teuthology.orchestra.run.smithi151.stdout: "fs_name": "cephfs", 2024-05-21T21:56:32.113 INFO:teuthology.orchestra.run.smithi151.stdout: "rank_uptime": 294.78414636100001, 2024-05-21T21:56:32.113 INFO:teuthology.orchestra.run.smithi151.stdout: "mdsmap_epoch": 29, 2024-05-21T21:56:32.113 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_epoch": 83, 2024-05-21T21:56:32.113 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_epoch_barrier": 82, 2024-05-21T21:56:32.113 INFO:teuthology.orchestra.run.smithi151.stdout: "uptime": 350.98246636300001 2024-05-21T21:56:32.113 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-05-21T21:56:32.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.108+0000 7f242d800700 1 -- 172.21.15.151:0/2452055674 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f240c0b2b10 msgr2=0x7f240c0b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:32.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.108+0000 7f242d800700 1 --2- 172.21.15.151:0/2452055674 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f240c0b2b10 0x7f240c0b4fd0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f24200dfb30 tx=0x7f2420009c00 comp rx=0 tx=0).stop 2024-05-21T21:56:32.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.108+0000 7f242d800700 1 -- 172.21.15.151:0/2452055674 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f242810b670 msgr2=0x7f24281a06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:32.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.108+0000 7f242d800700 1 --2- 172.21.15.151:0/2452055674 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f242810b670 0x7f24281a06a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f242000e3c0 tx=0x7f242000e6d0 comp rx=0 tx=0).stop 2024-05-21T21:56:32.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.108+0000 7f242d800700 1 -- 172.21.15.151:0/2452055674 shutdown_connections 2024-05-21T21:56:32.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.108+0000 7f242d800700 1 --2- 172.21.15.151:0/2452055674 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f240c0b2b10 0x7f240c0b4fd0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:32.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.108+0000 7f242d800700 1 --2- 172.21.15.151:0/2452055674 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2428105670 0x7f242819bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:32.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.108+0000 7f242d800700 1 --2- 172.21.15.151:0/2452055674 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f242810b670 0x7f24281a06a0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:32.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.108+0000 7f242d800700 1 --2- 172.21.15.151:0/2452055674 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2428106030 0x7f242819c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:32.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.108+0000 7f242d800700 1 -- 172.21.15.151:0/2452055674 >> 172.21.15.151:0/2452055674 conn(0x7f2428101160 msgr2=0x7f2428101980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:56:32.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.108+0000 7f242d800700 1 -- 172.21.15.151:0/2452055674 shutdown_connections 2024-05-21T21:56:32.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:32.109+0000 7f242d800700 1 -- 172.21.15.151:0/2452055674 wait complete. 2024-05-21T21:56:33.145 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.h status 2024-05-21T21:56:33.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:56:33 smithi151 ceph-mon[31623]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 938 B/s wr, 10 op/s 2024-05-21T21:56:33.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:56:33 smithi184 ceph-mon[36942]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 938 B/s wr, 10 op/s 2024-05-21T21:56:34.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:56:33 smithi164 ceph-mon[36269]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 938 B/s wr, 10 op/s 2024-05-21T21:56:34.171 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:56:34.510 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.509+0000 7f12c2800700 1 -- 172.21.15.151:0/1799090198 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12bc105780 msgr2=0x7f12bc105c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:34.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.509+0000 7f12c2800700 1 --2- 172.21.15.151:0/1799090198 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12bc105780 0x7f12bc105c00 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f12a4009990 tx=0x7f12a4009ca0 comp rx=0 tx=0).stop 2024-05-21T21:56:34.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.510+0000 7f12c2800700 1 -- 172.21.15.151:0/1799090198 shutdown_connections 2024-05-21T21:56:34.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.510+0000 7f12c2800700 1 --2- 172.21.15.151:0/1799090198 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f12bc10b780 0x7f12bc10bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:34.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.510+0000 7f12c2800700 1 --2- 172.21.15.151:0/1799090198 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f12bc106140 0x7f12bc112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:34.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.510+0000 7f12c2800700 1 --2- 172.21.15.151:0/1799090198 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12bc105780 0x7f12bc105c00 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:34.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.510+0000 7f12c2800700 1 -- 172.21.15.151:0/1799090198 >> 172.21.15.151:0/1799090198 conn(0x7f12bc101270 msgr2=0x7f12bc103690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:56:34.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.510+0000 7f12c2800700 1 -- 172.21.15.151:0/1799090198 shutdown_connections 2024-05-21T21:56:34.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.510+0000 7f12c2800700 1 -- 172.21.15.151:0/1799090198 wait complete. 2024-05-21T21:56:34.512 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.511+0000 7f12c2800700 1 Processor -- start 2024-05-21T21:56:34.512 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.511+0000 7f12c2800700 1 -- start start 2024-05-21T21:56:34.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.512+0000 7f12c2800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f12bc105780 0x7f12bc077a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:34.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.512+0000 7f12c2800700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f12bc106140 0x7f12bc077f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:34.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.512+0000 7f12c2800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12bc10b780 0x7f12bc0784d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:34.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.512+0000 7f12c2800700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f12bc115980 con 0x7f12bc10b780 2024-05-21T21:56:34.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.512+0000 7f12c2800700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f12bc115800 con 0x7f12bc105780 2024-05-21T21:56:34.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.512+0000 7f12c2800700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f12bc115b00 con 0x7f12bc106140 2024-05-21T21:56:34.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.512+0000 7f12bbe00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12bc10b780 0x7f12bc0784d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:34.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.512+0000 7f12bbe00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12bc10b780 0x7f12bc0784d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:58546/0 (socket says 172.21.15.151:58546) 2024-05-21T21:56:34.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.512+0000 7f12bbe00700 1 -- 172.21.15.151:0/2590950793 learned_addr learned my addr 172.21.15.151:0/2590950793 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:56:34.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.512+0000 7f12b2a00700 1 --2- 172.21.15.151:0/2590950793 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f12bc106140 0x7f12bc077f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:34.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.512+0000 7f12bb400700 1 --2- 172.21.15.151:0/2590950793 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f12bc105780 0x7f12bc077a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:34.514 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.513+0000 7f12b2a00700 1 -- 172.21.15.151:0/2590950793 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f12bc105780 msgr2=0x7f12bc077a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:34.514 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.513+0000 7f12b2a00700 1 --2- 172.21.15.151:0/2590950793 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f12bc105780 0x7f12bc077a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:34.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.513+0000 7f12b2a00700 1 -- 172.21.15.151:0/2590950793 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12bc10b780 msgr2=0x7f12bc0784d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:34.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.513+0000 7f12b2a00700 1 --2- 172.21.15.151:0/2590950793 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12bc10b780 0x7f12bc0784d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:34.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.513+0000 7f12b2a00700 1 -- 172.21.15.151:0/2590950793 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12bc1ac4e0 con 0x7f12bc106140 2024-05-21T21:56:34.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.513+0000 7f12bbe00700 1 --2- 172.21.15.151:0/2590950793 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12bc10b780 0x7f12bc0784d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:56:34.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.513+0000 7f12b2a00700 1 --2- 172.21.15.151:0/2590950793 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f12bc106140 0x7f12bc077f90 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f12a40059d0 tx=0x7f12a40047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:56:34.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.513+0000 7f12b8c00700 1 -- 172.21.15.151:0/2590950793 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a4027070 con 0x7f12bc106140 2024-05-21T21:56:34.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.514+0000 7f12b8c00700 1 -- 172.21.15.151:0/2590950793 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12a4022470 con 0x7f12bc106140 2024-05-21T21:56:34.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.514+0000 7f12c2800700 1 -- 172.21.15.151:0/2590950793 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12bc1ac770 con 0x7f12bc106140 2024-05-21T21:56:34.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.514+0000 7f12b8c00700 1 -- 172.21.15.151:0/2590950793 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a4017630 con 0x7f12bc106140 2024-05-21T21:56:34.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.514+0000 7f12c2800700 1 -- 172.21.15.151:0/2590950793 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12bc1acc20 con 0x7f12bc106140 2024-05-21T21:56:34.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.519+0000 7f12b8c00700 1 -- 172.21.15.151:0/2590950793 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 26) ==== 149886+0+0 (secure 0 0 0) 0x7f12a402a070 con 0x7f12bc106140 2024-05-21T21:56:34.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.519+0000 7f12cd0d6b80 1 Processor -- start 2024-05-21T21:56:34.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.520+0000 7f12cd0d6b80 1 -- start start 2024-05-21T21:56:34.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.520+0000 7f12cd0d6b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e691ed3610 0x55e691ed39f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:34.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.520+0000 7f12cd0d6b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e691ed3f30 0x55e691ecdb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:34.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.520+0000 7f12cd0d6b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e691ece140 0x55e691ece5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:34.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.520+0000 7f12cd0d6b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e691e2be40 con 0x55e691ece140 2024-05-21T21:56:34.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.520+0000 7f12cd0d6b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55e691e1b9e0 con 0x55e691ed3f30 2024-05-21T21:56:34.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.520+0000 7f12cd0d6b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55e691e1fa20 con 0x55e691ed3610 2024-05-21T21:56:34.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.520+0000 7f12bbe00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e691ed3610 0x55e691ed39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:34.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.520+0000 7f12bbe00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e691ed3610 0x55e691ed39f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:48248/0 (socket says 172.21.15.151:48248) 2024-05-21T21:56:34.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.520+0000 7f12bbe00700 1 -- 172.21.15.151:0/586081601 learned_addr learned my addr 172.21.15.151:0/586081601 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:56:34.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.521+0000 7f12b2a00700 1 --2- 172.21.15.151:0/586081601 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e691ece140 0x55e691ece5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:34.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.521+0000 7f12bb400700 1 --2- 172.21.15.151:0/586081601 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e691ed3f30 0x55e691ecdb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:34.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.521+0000 7f12bbe00700 1 -- 172.21.15.151:0/586081601 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e691ed3f30 msgr2=0x55e691ecdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:56:34.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.521+0000 7f12bbe00700 1 --2- 172.21.15.151:0/586081601 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e691ed3f30 0x55e691ecdb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:34.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.521+0000 7f12bbe00700 1 -- 172.21.15.151:0/586081601 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e691ece140 msgr2=0x55e691ece5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:56:34.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.521+0000 7f12bbe00700 1 --2- 172.21.15.151:0/586081601 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e691ece140 0x55e691ece5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:34.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.521+0000 7f12b8c00700 1 --2- 172.21.15.151:0/2590950793 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f12a00b2bc0 0x7f12a00b5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:34.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.521+0000 7f12bbe00700 1 -- 172.21.15.151:0/586081601 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e691de4420 con 0x55e691ed3610 2024-05-21T21:56:34.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.521+0000 7f12bb400700 1 --2- 172.21.15.151:0/586081601 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e691ed3f30 0x55e691ecdb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T21:56:34.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.521+0000 7f12b8c00700 1 -- 172.21.15.151:0/2590950793 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f12a40e3450 con 0x7f12bc106140 2024-05-21T21:56:34.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.521+0000 7f12bbe00700 1 --2- 172.21.15.151:0/2590950793 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f12a00b2bc0 0x7f12a00b5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:34.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.522+0000 7f12bbe00700 1 --2- 172.21.15.151:0/586081601 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e691ed3610 0x55e691ed39f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f12b4009500 tx=0x7f12b4009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5636adbde6b7d1f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:56:34.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.522+0000 7f12b0c00700 1 -- 172.21.15.151:0/586081601 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b4017a40 con 0x55e691ed3610 2024-05-21T21:56:34.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.522+0000 7f12b0c00700 1 -- 172.21.15.151:0/586081601 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12b400e940 con 0x55e691ed3610 2024-05-21T21:56:34.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.522+0000 7f12b0c00700 1 -- 172.21.15.151:0/586081601 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b4015410 con 0x55e691ed3610 2024-05-21T21:56:34.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.523+0000 7f12cd0d6b80 1 -- 172.21.15.151:0/586081601 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e691ed3610 msgr2=0x55e691ed39f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:34.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.523+0000 7f12bbe00700 1 --2- 172.21.15.151:0/2590950793 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f12a00b2bc0 0x7f12a00b5080 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f12b4020000 tx=0x7f12b4023000 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:56:34.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.523+0000 7f12cd0d6b80 1 --2- 172.21.15.151:0/586081601 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e691ed3610 0x55e691ed39f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f12b4009500 tx=0x7f12b4009920 comp rx=0 tx=0).stop 2024-05-21T21:56:34.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.523+0000 7f12cd0d6b80 1 -- 172.21.15.151:0/586081601 shutdown_connections 2024-05-21T21:56:34.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.523+0000 7f12cd0d6b80 1 --2- 172.21.15.151:0/586081601 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e691ece140 0x55e691ece5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:34.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.523+0000 7f12cd0d6b80 1 --2- 172.21.15.151:0/586081601 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e691ed3610 0x55e691ed39f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:34.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.523+0000 7f12cd0d6b80 1 --2- 172.21.15.151:0/586081601 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e691ed3f30 0x55e691ecdb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:34.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.523+0000 7f12cd0d6b80 1 -- 172.21.15.151:0/586081601 >> 172.21.15.151:0/586081601 conn(0x55e691dfbbe0 msgr2=0x55e691ec8690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:56:34.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.523+0000 7f12cd0d6b80 1 -- 172.21.15.151:0/586081601 shutdown_connections 2024-05-21T21:56:34.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.523+0000 7f12cd0d6b80 1 -- 172.21.15.151:0/586081601 wait complete. 2024-05-21T21:56:34.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.524+0000 7f12cd0d6b80 1 -- start start 2024-05-21T21:56:34.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.524+0000 7f12cd0d6b80 1 Processor -- start 2024-05-21T21:56:34.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.525+0000 7f12cd0d6b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e691ece140 0x55e691ed9c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:34.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.525+0000 7f12cd0d6b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e691ed3610 0x55e691ed7030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:34.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.525+0000 7f12cd0d6b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e691ed3f30 0x55e691ed7570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:34.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.525+0000 7f12cd0d6b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e691d21fa0 con 0x55e691ed3f30 2024-05-21T21:56:34.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.525+0000 7f12cd0d6b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55e691dbc760 con 0x55e691ed3610 2024-05-21T21:56:34.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.525+0000 7f12cd0d6b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55e691eda9e0 con 0x55e691ece140 2024-05-21T21:56:34.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.525+0000 7f12cd0d6b80 10 client.0 _ll_register_callbacks cb 0x55e691dc5c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:56:34.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.525+0000 7f12b2a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e691ed3610 0x55e691ed7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:34.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.525+0000 7f12bb400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e691ece140 0x55e691ed9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:34.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.525+0000 7f12b2a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e691ed3610 0x55e691ed7030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:35350/0 (socket says 172.21.15.151:35350) 2024-05-21T21:56:34.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.525+0000 7f12b2a00700 1 -- 172.21.15.151:0/286295481 learned_addr learned my addr 172.21.15.151:0/286295481 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:56:34.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.525+0000 7f12bbe00700 1 --2- 172.21.15.151:0/286295481 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e691ed3f30 0x55e691ed7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:34.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.525+0000 7f12bb400700 1 -- 172.21.15.151:0/286295481 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e691ed3610 msgr2=0x55e691ed7030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:56:34.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.525+0000 7f12bb400700 1 --2- 172.21.15.151:0/286295481 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e691ed3610 0x55e691ed7030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:34.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.525+0000 7f12bb400700 1 -- 172.21.15.151:0/286295481 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e691ed3f30 msgr2=0x55e691ed7570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:56:34.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.525+0000 7f12bb400700 1 --2- 172.21.15.151:0/286295481 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e691ed3f30 0x55e691ed7570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:34.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.525+0000 7f12bb400700 1 -- 172.21.15.151:0/286295481 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e691d6b8d0 con 0x55e691ece140 2024-05-21T21:56:34.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.525+0000 7f12b2a00700 1 --2- 172.21.15.151:0/286295481 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e691ed3610 0x55e691ed7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:56:34.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.526+0000 7f12bb400700 1 --2- 172.21.15.151:0/286295481 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e691ece140 0x55e691ed9c90 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f12ac0116f0 tx=0x7f12ac011a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=207b051f572efc93 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:56:34.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.526+0000 7f12a9600700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-21T21:56:34.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.526+0000 7f12a9600700 1 -- 172.21.15.151:0/286295481 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12ac01e020 con 0x55e691ece140 2024-05-21T21:56:34.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.526+0000 7f12a9600700 1 -- 172.21.15.151:0/286295481 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f12ac004830 con 0x55e691ece140 2024-05-21T21:56:34.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.526+0000 7f12cd0d6b80 1 -- 172.21.15.151:0/286295481 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e691e2dd90 con 0x55e691ece140 2024-05-21T21:56:34.529 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.526+0000 7f12a9600700 1 -- 172.21.15.151:0/286295481 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f12ac004a30 con 0x55e691ece140 2024-05-21T21:56:34.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.527+0000 7f12cd0d6b80 10 client.25429 fetch_fsmap learned FSMap version 29 2024-05-21T21:56:34.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.527+0000 7f12cd0d6b80 1 -- 172.21.15.151:0/286295481 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e691dec060 con 0x55e691ece140 2024-05-21T21:56:34.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.527+0000 7f12a9600700 1 -- 172.21.15.151:0/286295481 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55e691e2dd90 con 0x55e691ece140 2024-05-21T21:56:34.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.527+0000 7f12a9600700 1 -- 172.21.15.151:0/286295481 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f12ac019070 con 0x55e691ece140 2024-05-21T21:56:34.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.527+0000 7f12cd0d6b80 10 client.25429 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T21:56:34.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.527+0000 7f12cd0d6b80 10 client.25429 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T21:56:34.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.527+0000 7f12cd0d6b80 20 client.25429 populate_metadata read hostname 'smithi151' 2024-05-21T21:56:34.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.527+0000 7f12cd0d6b80 1 --2- 172.21.15.151:0/286295481 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55e691ee75a0 0x55e691ee99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:34.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.527+0000 7f12cd0d6b80 4 client.25429 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T21:56:34.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.527+0000 7f12cd0d6b80 1 -- 172.21.15.151:0/286295481 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e691e27570 con 0x55e691ee75a0 2024-05-21T21:56:34.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.528+0000 7f12b2a00700 1 --2- 172.21.15.151:0/286295481 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55e691ee75a0 0x55e691ee99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:34.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.528+0000 7f12b2a00700 1 --2- 172.21.15.151:0/286295481 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55e691ee75a0 0x55e691ee99b0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2646dde3543c478 server_cookie=b873aa5c819f5ea1 in_seq=0 out_seq=0 2024-05-21T21:56:34.531 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.528+0000 7f12a9600700 10 client.25429 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T21:56:34.531 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.529+0000 7f12a9600700 1 -- 172.21.15.151:0/286295481 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55e691e27570 con 0x55e691ee75a0 2024-05-21T21:56:34.531 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.529+0000 7f12a9600700 10 client.25429 handle_command_reply: tid=0 2024-05-21T21:56:34.531 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.529+0000 7f12cd0d6b80 1 client.25429 shutdown 2024-05-21T21:56:34.531 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.529+0000 7f12cd0d6b80 1 -- 172.21.15.151:0/286295481 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e691ece140 msgr2=0x55e691ed9c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:34.531 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.529+0000 7f12cd0d6b80 1 --2- 172.21.15.151:0/286295481 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e691ece140 0x55e691ed9c90 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f12ac0116f0 tx=0x7f12ac011a00 comp rx=0 tx=0).stop 2024-05-21T21:56:34.531 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.529+0000 7f12cd0d6b80 1 -- 172.21.15.151:0/286295481 shutdown_connections 2024-05-21T21:56:34.531 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.529+0000 7f12cd0d6b80 1 --2- 172.21.15.151:0/286295481 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e691ed3f30 0x55e691ed7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:34.531 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.529+0000 7f12cd0d6b80 1 --2- 172.21.15.151:0/286295481 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e691ece140 0x55e691ed9c90 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:34.531 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.529+0000 7f12cd0d6b80 1 --2- 172.21.15.151:0/286295481 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55e691ee75a0 0x55e691ee99b0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:34.531 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.529+0000 7f12cd0d6b80 1 --2- 172.21.15.151:0/286295481 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e691ed3610 0x55e691ed7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:34.531 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.529+0000 7f12a9600700 0 client.25429 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T21:56:34.531 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.529+0000 7f12cd0d6b80 1 -- 172.21.15.151:0/286295481 >> 172.21.15.151:0/286295481 conn(0x55e691dfbbe0 msgr2=0x55e691ec7d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:56:34.531 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.530+0000 7f12cd0d6b80 1 -- 172.21.15.151:0/286295481 shutdown_connections 2024-05-21T21:56:34.532 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.530+0000 7f12cd0d6b80 1 -- 172.21.15.151:0/286295481 wait complete. 2024-05-21T21:56:34.532 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.530+0000 7f12cd0d6b80 20 client.25429 trim_cache size 0 max 16384 2024-05-21T21:56:34.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.555+0000 7f12cd0d6b80 1 Processor -- start 2024-05-21T21:56:34.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.556+0000 7f12cd0d6b80 1 -- start start 2024-05-21T21:56:34.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.556+0000 7f12cd0d6b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e691ece140 0x55e691ecdb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:34.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.556+0000 7f12cd0d6b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e691ed3610 0x55e691ed76a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:34.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.556+0000 7f12cd0d6b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e691ed3f30 0x55e691ee9990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:34.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.556+0000 7f12cd0d6b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f12bc1b43e0 con 0x55e691ece140 2024-05-21T21:56:34.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.556+0000 7f12cd0d6b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f12bc1b5f20 con 0x55e691ed3610 2024-05-21T21:56:34.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.556+0000 7f12cd0d6b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f12bc1b4ab0 con 0x55e691ed3f30 2024-05-21T21:56:34.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.556+0000 7f12bb400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e691ece140 0x55e691ecdb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:34.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.556+0000 7f12bb400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e691ece140 0x55e691ecdb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:58562/0 (socket says 172.21.15.151:58562) 2024-05-21T21:56:34.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.556+0000 7f12bb400700 1 -- 172.21.15.151:0/2790343100 learned_addr learned my addr 172.21.15.151:0/2790343100 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:56:34.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.556+0000 7f12b2a00700 1 --2- 172.21.15.151:0/2790343100 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e691ed3610 0x55e691ed76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:34.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.557+0000 7f12bbe00700 1 --2- 172.21.15.151:0/2790343100 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e691ed3f30 0x55e691ee9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:34.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.557+0000 7f12bb400700 1 -- 172.21.15.151:0/2790343100 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e691ed3f30 msgr2=0x55e691ee9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:56:34.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.557+0000 7f12bb400700 1 --2- 172.21.15.151:0/2790343100 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e691ed3f30 0x55e691ee9990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:34.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.557+0000 7f12bb400700 1 -- 172.21.15.151:0/2790343100 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e691ed3610 msgr2=0x55e691ed76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:56:34.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.557+0000 7f12bb400700 1 --2- 172.21.15.151:0/2790343100 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e691ed3610 0x55e691ed76a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:34.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.557+0000 7f12bb400700 1 -- 172.21.15.151:0/2790343100 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e691ed98d0 con 0x55e691ece140 2024-05-21T21:56:34.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.558+0000 7f12b2a00700 1 --2- 172.21.15.151:0/2790343100 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e691ed3610 0x55e691ed76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T21:56:34.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.558+0000 7f12bb400700 1 --2- 172.21.15.151:0/2790343100 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e691ece140 0x55e691ecdb20 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f12ac022ab0 tx=0x7f12ac022ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37f1081882d19d43 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:56:34.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.558+0000 7f12abe00700 1 -- 172.21.15.151:0/2790343100 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12ac01e040 con 0x55e691ece140 2024-05-21T21:56:34.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.558+0000 7f12abe00700 1 -- 172.21.15.151:0/2790343100 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12ac030070 con 0x55e691ece140 2024-05-21T21:56:34.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.558+0000 7f12abe00700 1 -- 172.21.15.151:0/2790343100 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12ac003680 con 0x55e691ece140 2024-05-21T21:56:34.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.559+0000 7f12cd0d6b80 1 -- 172.21.15.151:0/2790343100 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e691ece140 msgr2=0x55e691ecdb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:34.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.559+0000 7f12cd0d6b80 1 --2- 172.21.15.151:0/2790343100 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e691ece140 0x55e691ecdb20 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f12ac022ab0 tx=0x7f12ac022ae0 comp rx=0 tx=0).stop 2024-05-21T21:56:34.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.559+0000 7f12cd0d6b80 1 -- 172.21.15.151:0/2790343100 shutdown_connections 2024-05-21T21:56:34.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.559+0000 7f12cd0d6b80 1 --2- 172.21.15.151:0/2790343100 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e691ed3f30 0x55e691ee9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:34.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.559+0000 7f12cd0d6b80 1 --2- 172.21.15.151:0/2790343100 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e691ece140 0x55e691ecdb20 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:34.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.559+0000 7f12cd0d6b80 1 --2- 172.21.15.151:0/2790343100 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e691ed3610 0x55e691ed76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:34.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.559+0000 7f12cd0d6b80 1 -- 172.21.15.151:0/2790343100 >> 172.21.15.151:0/2790343100 conn(0x55e691dfbbe0 msgr2=0x55e691ed43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:56:34.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.559+0000 7f12cd0d6b80 1 -- 172.21.15.151:0/2790343100 shutdown_connections 2024-05-21T21:56:34.561 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.559+0000 7f12cd0d6b80 1 -- 172.21.15.151:0/2790343100 wait complete. 2024-05-21T21:56:34.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.560+0000 7f12cd0d6b80 1 -- start start 2024-05-21T21:56:34.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.560+0000 7f12cd0d6b80 1 Processor -- start 2024-05-21T21:56:34.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.561+0000 7f12cd0d6b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e691ece140 0x55e691ee0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:34.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.561+0000 7f12cd0d6b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e691ed3610 0x55e691eeeb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:34.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.561+0000 7f12cd0d6b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e691ed3f30 0x55e691eef0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:34.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.561+0000 7f12cd0d6b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f12bc1b5850 con 0x55e691ece140 2024-05-21T21:56:34.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.561+0000 7f12cd0d6b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f12bc1b5180 con 0x55e691ed3610 2024-05-21T21:56:34.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.561+0000 7f12cd0d6b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f12bc1b2380 con 0x55e691ed3f30 2024-05-21T21:56:34.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.561+0000 7f12cd0d6b80 10 client.0 _ll_register_callbacks cb 0x55e691dc5c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:56:34.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.561+0000 7f12bb400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e691ece140 0x55e691ee0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:34.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.561+0000 7f12bb400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e691ece140 0x55e691ee0380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:58576/0 (socket says 172.21.15.151:58576) 2024-05-21T21:56:34.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.561+0000 7f12bb400700 1 -- 172.21.15.151:0/3385627989 learned_addr learned my addr 172.21.15.151:0/3385627989 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:56:34.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.561+0000 7f12b2a00700 1 --2- 172.21.15.151:0/3385627989 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e691ed3610 0x55e691eeeb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:34.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.561+0000 7f12bbe00700 1 --2- 172.21.15.151:0/3385627989 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e691ed3f30 0x55e691eef0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:34.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.562+0000 7f12bbe00700 1 -- 172.21.15.151:0/3385627989 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e691ed3610 msgr2=0x55e691eeeb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:56:34.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.562+0000 7f12bbe00700 1 --2- 172.21.15.151:0/3385627989 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e691ed3610 0x55e691eeeb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:34.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.562+0000 7f12bbe00700 1 -- 172.21.15.151:0/3385627989 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e691ece140 msgr2=0x55e691ee0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:56:34.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.562+0000 7f12bbe00700 1 --2- 172.21.15.151:0/3385627989 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e691ece140 0x55e691ee0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:34.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.562+0000 7f12bbe00700 1 -- 172.21.15.151:0/3385627989 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e691eef920 con 0x55e691ed3f30 2024-05-21T21:56:34.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.562+0000 7f12b2a00700 1 --2- 172.21.15.151:0/3385627989 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e691ed3610 0x55e691eeeb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:56:34.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.562+0000 7f12bbe00700 1 --2- 172.21.15.151:0/3385627989 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e691ed3f30 0x55e691eef0b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f12b4022b20 tx=0x7f12b4022e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2e3ab24b6662c62 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:56:34.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.562+0000 7f12a9600700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-21T21:56:34.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.562+0000 7f12a9600700 1 -- 172.21.15.151:0/3385627989 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b4032070 con 0x55e691ed3f30 2024-05-21T21:56:34.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.562+0000 7f12cd0d6b80 1 -- 172.21.15.151:0/3385627989 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12ac030070 con 0x55e691ed3f30 2024-05-21T21:56:34.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.562+0000 7f12a9600700 1 -- 172.21.15.151:0/3385627989 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f12b4022820 con 0x55e691ed3f30 2024-05-21T21:56:34.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.563+0000 7f12a9600700 1 -- 172.21.15.151:0/3385627989 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f12b4026070 con 0x55e691ed3f30 2024-05-21T21:56:34.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.563+0000 7f12cd0d6b80 10 client.25435 fetch_fsmap learned FSMap version 29 2024-05-21T21:56:34.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.563+0000 7f12cd0d6b80 1 -- 172.21.15.151:0/3385627989 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e691f52180 con 0x55e691ed3f30 2024-05-21T21:56:34.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.563+0000 7f12a9600700 1 -- 172.21.15.151:0/3385627989 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12ac030070 con 0x55e691ed3f30 2024-05-21T21:56:34.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.563+0000 7f12a9600700 1 -- 172.21.15.151:0/3385627989 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f12b402b040 con 0x55e691ed3f30 2024-05-21T21:56:34.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.563+0000 7f12cd0d6b80 10 client.25435 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T21:56:34.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.563+0000 7f12cd0d6b80 10 client.25435 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T21:56:34.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.563+0000 7f12cd0d6b80 20 client.25435 populate_metadata read hostname 'smithi151' 2024-05-21T21:56:34.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.563+0000 7f12cd0d6b80 1 --2- 172.21.15.151:0/3385627989 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55e691f52590 0x55e691f54980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:34.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.563+0000 7f12cd0d6b80 4 client.25435 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-21T21:56:34.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.563+0000 7f12cd0d6b80 1 -- 172.21.15.151:0/3385627989 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e691c9e250 con 0x55e691f52590 2024-05-21T21:56:34.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.564+0000 7f12bb400700 1 --2- 172.21.15.151:0/3385627989 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55e691f52590 0x55e691f54980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:34.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.564+0000 7f12bb400700 1 --2- 172.21.15.151:0/3385627989 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55e691f52590 0x55e691f54980 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc0cc39663c95b16 server_cookie=d6df3000d217d02f in_seq=0 out_seq=0 2024-05-21T21:56:34.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.564+0000 7f12a9600700 10 client.25435 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T21:56:34.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.565+0000 7f12a9600700 1 -- 172.21.15.151:0/3385627989 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55e691c9e250 con 0x55e691f52590 2024-05-21T21:56:34.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.565+0000 7f12a9600700 10 client.25435 handle_command_reply: tid=0 2024-05-21T21:56:34.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.565+0000 7f12cd0d6b80 1 client.25435 shutdown 2024-05-21T21:56:34.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.565+0000 7f12cd0d6b80 1 -- 172.21.15.151:0/3385627989 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e691ed3f30 msgr2=0x55e691eef0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:34.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.565+0000 7f12cd0d6b80 1 --2- 172.21.15.151:0/3385627989 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e691ed3f30 0x55e691eef0b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f12b4022b20 tx=0x7f12b4022e20 comp rx=0 tx=0).stop 2024-05-21T21:56:34.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.565+0000 7f12cd0d6b80 1 -- 172.21.15.151:0/3385627989 shutdown_connections 2024-05-21T21:56:34.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.565+0000 7f12cd0d6b80 1 --2- 172.21.15.151:0/3385627989 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e691ed3f30 0x55e691eef0b0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:34.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.565+0000 7f12cd0d6b80 1 --2- 172.21.15.151:0/3385627989 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e691ece140 0x55e691ee0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:34.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.565+0000 7f12cd0d6b80 1 --2- 172.21.15.151:0/3385627989 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55e691f52590 0x55e691f54980 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:34.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.565+0000 7f12cd0d6b80 1 --2- 172.21.15.151:0/3385627989 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e691ed3610 0x55e691eeeb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:34.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.565+0000 7f12a9600700 0 client.25435 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T21:56:34.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.565+0000 7f12cd0d6b80 1 -- 172.21.15.151:0/3385627989 >> 172.21.15.151:0/3385627989 conn(0x55e691dfbbe0 msgr2=0x55e691ed20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:56:34.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.565+0000 7f12cd0d6b80 1 -- 172.21.15.151:0/3385627989 shutdown_connections 2024-05-21T21:56:34.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.565+0000 7f12cd0d6b80 1 -- 172.21.15.151:0/3385627989 wait complete. 2024-05-21T21:56:34.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.565+0000 7f12cd0d6b80 20 client.25435 trim_cache size 0 max 16384 2024-05-21T21:56:34.568 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-05-21T21:56:34.568 INFO:teuthology.orchestra.run.smithi151.stdout: "status": "no active scrubs running", 2024-05-21T21:56:34.568 INFO:teuthology.orchestra.run.smithi151.stdout: "scrubs": {} 2024-05-21T21:56:34.568 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-05-21T21:56:34.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.566+0000 7f12c2800700 1 -- 172.21.15.151:0/2590950793 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f12a00b2bc0 msgr2=0x7f12a00b5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:34.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.566+0000 7f12c2800700 1 --2- 172.21.15.151:0/2590950793 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f12a00b2bc0 0x7f12a00b5080 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f12b4020000 tx=0x7f12b4023000 comp rx=0 tx=0).stop 2024-05-21T21:56:34.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.566+0000 7f12c2800700 1 -- 172.21.15.151:0/2590950793 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f12bc106140 msgr2=0x7f12bc077f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:34.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.566+0000 7f12c2800700 1 --2- 172.21.15.151:0/2590950793 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f12bc106140 0x7f12bc077f90 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f12a40059d0 tx=0x7f12a40047b0 comp rx=0 tx=0).stop 2024-05-21T21:56:34.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.566+0000 7f12c2800700 1 -- 172.21.15.151:0/2590950793 shutdown_connections 2024-05-21T21:56:34.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.566+0000 7f12c2800700 1 --2- 172.21.15.151:0/2590950793 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f12a00b2bc0 0x7f12a00b5080 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:34.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.566+0000 7f12c2800700 1 --2- 172.21.15.151:0/2590950793 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f12bc105780 0x7f12bc077a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:34.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.566+0000 7f12c2800700 1 --2- 172.21.15.151:0/2590950793 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12bc10b780 0x7f12bc0784d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:34.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.566+0000 7f12c2800700 1 --2- 172.21.15.151:0/2590950793 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f12bc106140 0x7f12bc077f90 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:34.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.566+0000 7f12c2800700 1 -- 172.21.15.151:0/2590950793 >> 172.21.15.151:0/2590950793 conn(0x7f12bc101270 msgr2=0x7f12bc101870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:56:34.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.566+0000 7f12c2800700 1 -- 172.21.15.151:0/2590950793 shutdown_connections 2024-05-21T21:56:34.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:34.566+0000 7f12c2800700 1 -- 172.21.15.151:0/2590950793 wait complete. 2024-05-21T21:56:35.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:56:35 smithi151 ceph-mon[31623]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 853 B/s wr, 12 op/s 2024-05-21T21:56:35.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:56:35 smithi184 ceph-mon[36942]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 853 B/s wr, 12 op/s 2024-05-21T21:56:36.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:56:35 smithi164 ceph-mon[36269]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 853 B/s wr, 12 op/s 2024-05-21T21:56:36.578 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-21T21:56:36.579 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-21T21:56:37.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.096+0000 7fb4cec00700 1 -- 172.21.15.151:0/3553322780 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb4c8075770 msgr2=0x7fb4c8075b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:37.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.096+0000 7fb4cec00700 1 --2- 172.21.15.151:0/3553322780 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb4c8075770 0x7fb4c8075b50 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb4b8009990 tx=0x7fb4b8009ca0 comp rx=0 tx=0).stop 2024-05-21T21:56:37.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.096+0000 7fb4cec00700 1 -- 172.21.15.151:0/3553322780 shutdown_connections 2024-05-21T21:56:37.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.096+0000 7fb4cec00700 1 --2- 172.21.15.151:0/3553322780 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4c810ff40 0x7fb4c8112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:37.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.096+0000 7fb4cec00700 1 --2- 172.21.15.151:0/3553322780 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb4c8075770 0x7fb4c8075b50 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:37.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.096+0000 7fb4cec00700 1 --2- 172.21.15.151:0/3553322780 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb4c8076090 0x7fb4c810fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:37.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.096+0000 7fb4cec00700 1 -- 172.21.15.151:0/3553322780 >> 172.21.15.151:0/3553322780 conn(0x7fb4c807a6a0 msgr2=0x7fb4c807aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:56:37.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.096+0000 7fb4cec00700 1 -- 172.21.15.151:0/3553322780 shutdown_connections 2024-05-21T21:56:37.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.096+0000 7fb4cec00700 1 -- 172.21.15.151:0/3553322780 wait complete. 2024-05-21T21:56:37.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.097+0000 7fb4cec00700 1 Processor -- start 2024-05-21T21:56:37.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.097+0000 7fb4cec00700 1 -- start start 2024-05-21T21:56:37.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.098+0000 7fb4cec00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb4c8075770 0x7fb4c819be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:37.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.098+0000 7fb4cec00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4c8076090 0x7fb4c819c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:37.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.098+0000 7fb4cec00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb4c810ff40 0x7fb4c81a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:37.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.098+0000 7fb4cec00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c8115940 con 0x7fb4c810ff40 2024-05-21T21:56:37.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.098+0000 7fb4cec00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c81157c0 con 0x7fb4c8075770 2024-05-21T21:56:37.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.098+0000 7fb4cec00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c8115ac0 con 0x7fb4c8076090 2024-05-21T21:56:37.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.098+0000 7fb4c7e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb4c8075770 0x7fb4c819be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:37.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.098+0000 7fb4c7e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb4c8075770 0x7fb4c819be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:60886/0 (socket says 172.21.15.151:60886) 2024-05-21T21:56:37.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.098+0000 7fb4c7e00700 1 -- 172.21.15.151:0/3722002823 learned_addr learned my addr 172.21.15.151:0/3722002823 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:56:37.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.099+0000 7fb4c7400700 1 --2- 172.21.15.151:0/3722002823 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4c8076090 0x7fb4c819c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:37.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.099+0000 7fb4ccc00700 1 --2- 172.21.15.151:0/3722002823 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb4c810ff40 0x7fb4c81a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:37.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.099+0000 7fb4c7e00700 1 -- 172.21.15.151:0/3722002823 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4c8076090 msgr2=0x7fb4c819c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:37.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.099+0000 7fb4c7e00700 1 --2- 172.21.15.151:0/3722002823 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4c8076090 0x7fb4c819c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:37.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.099+0000 7fb4c7e00700 1 -- 172.21.15.151:0/3722002823 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb4c810ff40 msgr2=0x7fb4c81a06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:37.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.099+0000 7fb4c7e00700 1 --2- 172.21.15.151:0/3722002823 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb4c810ff40 0x7fb4c81a06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:37.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.099+0000 7fb4c7e00700 1 -- 172.21.15.151:0/3722002823 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4c81a0e20 con 0x7fb4c8075770 2024-05-21T21:56:37.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.099+0000 7fb4ccc00700 1 --2- 172.21.15.151:0/3722002823 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb4c810ff40 0x7fb4c81a06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T21:56:37.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.099+0000 7fb4c7e00700 1 --2- 172.21.15.151:0/3722002823 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb4c8075770 0x7fb4c819be10 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb4b8000c00 tx=0x7fb4b80057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:56:37.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.100+0000 7fb4c4c00700 1 -- 172.21.15.151:0/3722002823 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b8027070 con 0x7fb4c8075770 2024-05-21T21:56:37.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.100+0000 7fb4c4c00700 1 -- 172.21.15.151:0/3722002823 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4b8022470 con 0x7fb4c8075770 2024-05-21T21:56:37.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.100+0000 7fb4c4c00700 1 -- 172.21.15.151:0/3722002823 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b8017650 con 0x7fb4c8075770 2024-05-21T21:56:37.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.100+0000 7fb4cec00700 1 -- 172.21.15.151:0/3722002823 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4c81a10b0 con 0x7fb4c8075770 2024-05-21T21:56:37.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.100+0000 7fb4cec00700 1 -- 172.21.15.151:0/3722002823 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4c81a8990 con 0x7fb4c8075770 2024-05-21T21:56:37.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.104+0000 7fb4c4c00700 1 -- 172.21.15.151:0/3722002823 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 149886+0+0 (secure 0 0 0) 0x7fb4b8009d10 con 0x7fb4c8075770 2024-05-21T21:56:37.105 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.105+0000 7fb4d9635b80 1 Processor -- start 2024-05-21T21:56:37.105 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.105+0000 7fb4d9635b80 1 -- start start 2024-05-21T21:56:37.105 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.105+0000 7fb4d9635b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562eaac905c0 0x562eaac909a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:37.106 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.105+0000 7fb4d9635b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562eaac90ee0 0x562eaac8aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:37.106 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.105+0000 7fb4d9635b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562eaac8b0f0 0x562eaac8b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:37.106 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.105+0000 7fb4d9635b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x562eaabe8df0 con 0x562eaac905c0 2024-05-21T21:56:37.106 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.105+0000 7fb4d9635b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x562eaabd8990 con 0x562eaac8b0f0 2024-05-21T21:56:37.106 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.105+0000 7fb4d9635b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x562eaabdc9d0 con 0x562eaac90ee0 2024-05-21T21:56:37.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.106+0000 7fb4ccc00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562eaac905c0 0x562eaac909a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:37.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.106+0000 7fb4c7400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562eaac8b0f0 0x562eaac8b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:37.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.106+0000 7fb4ccc00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562eaac905c0 0x562eaac909a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:57852/0 (socket says 172.21.15.151:57852) 2024-05-21T21:56:37.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.106+0000 7fb4c4c00700 1 --2- 172.21.15.151:0/3722002823 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fb4ac0b2b10 0x7fb4ac0b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:37.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.106+0000 7fb4ccc00700 1 -- 172.21.15.151:0/3472996401 learned_addr learned my addr 172.21.15.151:0/3472996401 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:56:37.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.106+0000 7fb4c4c00700 1 -- 172.21.15.151:0/3722002823 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fb4b80e3830 con 0x7fb4c8075770 2024-05-21T21:56:37.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.106+0000 7fb4ccc00700 1 --2- 172.21.15.151:0/3722002823 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fb4ac0b2b10 0x7fb4ac0b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:37.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.106+0000 7fb4c7400700 1 -- 172.21.15.151:0/3472996401 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562eaac90ee0 msgr2=0x562eaac8aad0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-21T21:56:37.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.106+0000 7fb4c7400700 1 --2- 172.21.15.151:0/3472996401 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562eaac90ee0 0x562eaac8aad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:37.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.106+0000 7fb4c7400700 1 -- 172.21.15.151:0/3472996401 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562eaac905c0 msgr2=0x562eaac909a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:56:37.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.106+0000 7fb4c7400700 1 --2- 172.21.15.151:0/3472996401 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562eaac905c0 0x562eaac909a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:37.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.106+0000 7fb4c7400700 1 -- 172.21.15.151:0/3472996401 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562eaaba13d0 con 0x562eaac8b0f0 2024-05-21T21:56:37.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.107+0000 7fb4c7400700 1 --2- 172.21.15.151:0/3472996401 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562eaac8b0f0 0x562eaac8b570 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb4c8076a70 tx=0x7fb4b4007f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23a29f57848c7da5 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:56:37.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.107+0000 7fb4bd600700 1 -- 172.21.15.151:0/3472996401 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b4004310 con 0x562eaac8b0f0 2024-05-21T21:56:37.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.107+0000 7fb4bd600700 1 -- 172.21.15.151:0/3472996401 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4b4015070 con 0x562eaac8b0f0 2024-05-21T21:56:37.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.107+0000 7fb4bd600700 1 -- 172.21.15.151:0/3472996401 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b4005020 con 0x562eaac8b0f0 2024-05-21T21:56:37.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.108+0000 7fb4ccc00700 1 --2- 172.21.15.151:0/3722002823 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fb4ac0b2b10 0x7fb4ac0b4fd0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb4c00094f0 tx=0x7fb4c0004b40 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:56:37.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.108+0000 7fb4d9635b80 1 -- 172.21.15.151:0/3472996401 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562eaac8b0f0 msgr2=0x562eaac8b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:37.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.108+0000 7fb4d9635b80 1 --2- 172.21.15.151:0/3472996401 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562eaac8b0f0 0x562eaac8b570 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb4c8076a70 tx=0x7fb4b4007f10 comp rx=0 tx=0).stop 2024-05-21T21:56:37.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.108+0000 7fb4d9635b80 1 -- 172.21.15.151:0/3472996401 shutdown_connections 2024-05-21T21:56:37.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.108+0000 7fb4d9635b80 1 --2- 172.21.15.151:0/3472996401 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562eaac8b0f0 0x562eaac8b570 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:37.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.108+0000 7fb4d9635b80 1 --2- 172.21.15.151:0/3472996401 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562eaac905c0 0x562eaac909a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:37.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.108+0000 7fb4d9635b80 1 --2- 172.21.15.151:0/3472996401 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562eaac90ee0 0x562eaac8aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:37.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.108+0000 7fb4d9635b80 1 -- 172.21.15.151:0/3472996401 >> 172.21.15.151:0/3472996401 conn(0x562eaabb8b90 msgr2=0x562eaac85640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:56:37.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.109+0000 7fb4d9635b80 1 -- 172.21.15.151:0/3472996401 shutdown_connections 2024-05-21T21:56:37.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.109+0000 7fb4d9635b80 1 -- 172.21.15.151:0/3472996401 wait complete. 2024-05-21T21:56:37.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.109+0000 7fb4d9635b80 1 -- start start 2024-05-21T21:56:37.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.110+0000 7fb4d9635b80 1 Processor -- start 2024-05-21T21:56:37.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.110+0000 7fb4d9635b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562eaac8b0f0 0x562eaac96c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:37.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.110+0000 7fb4d9635b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562eaac905c0 0x562eaac93fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:37.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.110+0000 7fb4d9635b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562eaac90ee0 0x562eaac94510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:37.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.111+0000 7fb4d9635b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x562eaaadef50 con 0x562eaac90ee0 2024-05-21T21:56:37.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.111+0000 7fb4d9635b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x562eaab79710 con 0x562eaac8b0f0 2024-05-21T21:56:37.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.111+0000 7fb4d9635b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x562eaac97990 con 0x562eaac905c0 2024-05-21T21:56:37.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.111+0000 7fb4d9635b80 10 client.0 _ll_register_callbacks cb 0x562eaab82c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:56:37.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.111+0000 7fb4c7e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562eaac8b0f0 0x562eaac96c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:37.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.111+0000 7fb4c7400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562eaac905c0 0x562eaac93fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:37.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.111+0000 7fb4c7e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562eaac8b0f0 0x562eaac96c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:60890/0 (socket says 172.21.15.151:60890) 2024-05-21T21:56:37.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.111+0000 7fb4c7e00700 1 -- 172.21.15.151:0/3301486664 learned_addr learned my addr 172.21.15.151:0/3301486664 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:56:37.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.111+0000 7fb4ccc00700 1 --2- 172.21.15.151:0/3301486664 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562eaac90ee0 0x562eaac94510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:37.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.111+0000 7fb4c7e00700 1 -- 172.21.15.151:0/3301486664 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562eaac905c0 msgr2=0x562eaac93fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:56:37.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.111+0000 7fb4c7e00700 1 --2- 172.21.15.151:0/3301486664 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562eaac905c0 0x562eaac93fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:37.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.111+0000 7fb4c7e00700 1 -- 172.21.15.151:0/3301486664 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562eaac90ee0 msgr2=0x562eaac94510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:56:37.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.111+0000 7fb4c7e00700 1 --2- 172.21.15.151:0/3301486664 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562eaac90ee0 0x562eaac94510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:37.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.111+0000 7fb4c7e00700 1 -- 172.21.15.151:0/3301486664 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562eaab28880 con 0x562eaac8b0f0 2024-05-21T21:56:37.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.112+0000 7fb4c7e00700 1 --2- 172.21.15.151:0/3301486664 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562eaac8b0f0 0x562eaac96c30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb4b80aa890 tx=0x7fb4b80aa250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c80b243e06c7919b server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:56:37.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.112+0000 7fb4b2000700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-21T21:56:37.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.112+0000 7fb4b2000700 1 -- 172.21.15.151:0/3301486664 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b80aa410 con 0x562eaac8b0f0 2024-05-21T21:56:37.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.112+0000 7fb4b2000700 1 -- 172.21.15.151:0/3301486664 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fb4b8022920 con 0x562eaac8b0f0 2024-05-21T21:56:37.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.112+0000 7fb4d9635b80 1 -- 172.21.15.151:0/3301486664 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4b4015070 con 0x562eaac8b0f0 2024-05-21T21:56:37.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.112+0000 7fb4b2000700 1 -- 172.21.15.151:0/3301486664 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb4b80ab000 con 0x562eaac8b0f0 2024-05-21T21:56:37.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.112+0000 7fb4d9635b80 10 client.25436 fetch_fsmap learned FSMap version 29 2024-05-21T21:56:37.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.112+0000 7fb4d9635b80 1 -- 172.21.15.151:0/3301486664 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562eaaba9010 con 0x562eaac8b0f0 2024-05-21T21:56:37.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.113+0000 7fb4b2000700 1 -- 172.21.15.151:0/3301486664 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4b4015070 con 0x562eaac8b0f0 2024-05-21T21:56:37.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.113+0000 7fb4b2000700 1 -- 172.21.15.151:0/3301486664 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fb4b80aabf0 con 0x562eaac8b0f0 2024-05-21T21:56:37.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.113+0000 7fb4d9635b80 10 client.25436 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T21:56:37.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.113+0000 7fb4d9635b80 10 client.25436 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-21T21:56:37.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.113+0000 7fb4d9635b80 20 client.25436 populate_metadata read hostname 'smithi151' 2024-05-21T21:56:37.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.113+0000 7fb4d9635b80 1 --2- 172.21.15.151:0/3301486664 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x562eaaca4480 0x562eaaca58f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:37.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.113+0000 7fb4d9635b80 4 client.25436 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T21:56:37.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.113+0000 7fb4d9635b80 1 -- 172.21.15.151:0/3301486664 --> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562eaabead40 con 0x562eaaca4480 2024-05-21T21:56:37.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.114+0000 7fb4c7400700 1 --2- 172.21.15.151:0/3301486664 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x562eaaca4480 0x562eaaca58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:37.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.114+0000 7fb4c7400700 1 --2- 172.21.15.151:0/3301486664 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x562eaaca4480 0x562eaaca58f0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c49e55815b891475 server_cookie=f87983c8019340d2 in_seq=0 out_seq=0 2024-05-21T21:56:37.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.114+0000 7fb4b2000700 10 client.25436 ms_handle_connect on v2:172.21.15.164:6836/1554609572 2024-05-21T21:56:37.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.115+0000 7fb4b2000700 1 -- 172.21.15.151:0/3301486664 <== mds.2 v2:172.21.15.164:6836/1554609572 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x562eaabead40 con 0x562eaaca4480 2024-05-21T21:56:37.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.115+0000 7fb4b2000700 10 client.25436 handle_command_reply: tid=0 2024-05-21T21:56:37.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.115+0000 7fb4d9635b80 1 client.25436 shutdown 2024-05-21T21:56:37.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.115+0000 7fb4d9635b80 1 -- 172.21.15.151:0/3301486664 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562eaac8b0f0 msgr2=0x562eaac96c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:37.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.115+0000 7fb4d9635b80 1 --2- 172.21.15.151:0/3301486664 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562eaac8b0f0 0x562eaac96c30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb4b80aa890 tx=0x7fb4b80aa250 comp rx=0 tx=0).stop 2024-05-21T21:56:37.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.115+0000 7fb4d9635b80 1 -- 172.21.15.151:0/3301486664 shutdown_connections 2024-05-21T21:56:37.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.116+0000 7fb4d9635b80 1 --2- 172.21.15.151:0/3301486664 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x562eaaca4480 0x562eaaca58f0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:37.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.116+0000 7fb4d9635b80 1 --2- 172.21.15.151:0/3301486664 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562eaac8b0f0 0x562eaac96c30 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:37.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.116+0000 7fb4d9635b80 1 --2- 172.21.15.151:0/3301486664 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562eaac90ee0 0x562eaac94510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:37.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.116+0000 7fb4d9635b80 1 --2- 172.21.15.151:0/3301486664 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562eaac905c0 0x562eaac93fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:37.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.116+0000 7fb4d9635b80 1 -- 172.21.15.151:0/3301486664 >> 172.21.15.151:0/3301486664 conn(0x562eaabb8b90 msgr2=0x562eaac84d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:56:37.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.116+0000 7fb4b2000700 0 client.25436 ms_handle_reset on v2:172.21.15.164:6836/1554609572 2024-05-21T21:56:37.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.116+0000 7fb4d9635b80 1 -- 172.21.15.151:0/3301486664 shutdown_connections 2024-05-21T21:56:37.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.116+0000 7fb4d9635b80 1 -- 172.21.15.151:0/3301486664 wait complete. 2024-05-21T21:56:37.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.116+0000 7fb4d9635b80 20 client.25436 trim_cache size 0 max 16384 2024-05-21T21:56:37.154 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:56:37.155 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.155+0000 7fb4d9635b80 1 Processor -- start 2024-05-21T21:56:37.155 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.155+0000 7fb4d9635b80 1 -- start start 2024-05-21T21:56:37.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.155+0000 7fb4d9635b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562eaac8b0f0 0x562eaac8aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:37.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.155+0000 7fb4d9635b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562eaac905c0 0x562eaac944c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:37.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.155+0000 7fb4d9635b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562eaac90ee0 0x562eaac854a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:37.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.155+0000 7fb4d9635b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x562eaaca3c30 con 0x562eaac90ee0 2024-05-21T21:56:37.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.155+0000 7fb4d9635b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x562eaaca4300 con 0x562eaac905c0 2024-05-21T21:56:37.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.155+0000 7fb4d9635b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x562eaaca49d0 con 0x562eaac8b0f0 2024-05-21T21:56:37.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.156+0000 7fb4c7e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562eaac8b0f0 0x562eaac8aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:37.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.156+0000 7fb4c7e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562eaac8b0f0 0x562eaac8aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:52238/0 (socket says 172.21.15.151:52238) 2024-05-21T21:56:37.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.156+0000 7fb4c7e00700 1 -- 172.21.15.151:0/445623135 learned_addr learned my addr 172.21.15.151:0/445623135 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:56:37.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.156+0000 7fb4ccc00700 1 --2- 172.21.15.151:0/445623135 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562eaac90ee0 0x562eaac854a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:37.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.156+0000 7fb4c7400700 1 --2- 172.21.15.151:0/445623135 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562eaac905c0 0x562eaac944c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:37.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.156+0000 7fb4c7e00700 1 -- 172.21.15.151:0/445623135 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562eaac905c0 msgr2=0x562eaac944c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:56:37.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.156+0000 7fb4c7e00700 1 --2- 172.21.15.151:0/445623135 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562eaac905c0 0x562eaac944c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:37.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.156+0000 7fb4c7e00700 1 -- 172.21.15.151:0/445623135 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562eaac90ee0 msgr2=0x562eaac854a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:56:37.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.156+0000 7fb4c7e00700 1 --2- 172.21.15.151:0/445623135 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562eaac90ee0 0x562eaac854a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:37.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.156+0000 7fb4c7e00700 1 -- 172.21.15.151:0/445623135 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562eaac86840 con 0x562eaac8b0f0 2024-05-21T21:56:37.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.156+0000 7fb4ccc00700 1 --2- 172.21.15.151:0/445623135 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562eaac90ee0 0x562eaac854a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T21:56:37.159 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.157+0000 7fb4c7e00700 1 --2- 172.21.15.151:0/445623135 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562eaac8b0f0 0x562eaac8aad0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb4b802f030 tx=0x7fb4b80a6170 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1910c563d3db9645 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:56:37.159 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.157+0000 7fb4bcc00700 1 -- 172.21.15.151:0/445623135 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b8070040 con 0x562eaac8b0f0 2024-05-21T21:56:37.159 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.157+0000 7fb4bcc00700 1 -- 172.21.15.151:0/445623135 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4b80aa900 con 0x562eaac8b0f0 2024-05-21T21:56:37.159 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.157+0000 7fb4bcc00700 1 -- 172.21.15.151:0/445623135 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b8022920 con 0x562eaac8b0f0 2024-05-21T21:56:37.159 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.158+0000 7fb4d9635b80 1 -- 172.21.15.151:0/445623135 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562eaac8b0f0 msgr2=0x562eaac8aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:37.159 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.158+0000 7fb4d9635b80 1 --2- 172.21.15.151:0/445623135 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562eaac8b0f0 0x562eaac8aad0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb4b802f030 tx=0x7fb4b80a6170 comp rx=0 tx=0).stop 2024-05-21T21:56:37.159 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.158+0000 7fb4d9635b80 1 -- 172.21.15.151:0/445623135 shutdown_connections 2024-05-21T21:56:37.159 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.158+0000 7fb4d9635b80 1 --2- 172.21.15.151:0/445623135 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562eaac90ee0 0x562eaac854a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:37.159 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.158+0000 7fb4d9635b80 1 --2- 172.21.15.151:0/445623135 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562eaac8b0f0 0x562eaac8aad0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:37.159 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.158+0000 7fb4d9635b80 1 --2- 172.21.15.151:0/445623135 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562eaac905c0 0x562eaac944c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:37.159 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.158+0000 7fb4d9635b80 1 -- 172.21.15.151:0/445623135 >> 172.21.15.151:0/445623135 conn(0x562eaabb8b90 msgr2=0x562eaac93db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:56:37.159 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.158+0000 7fb4d9635b80 1 -- 172.21.15.151:0/445623135 shutdown_connections 2024-05-21T21:56:37.159 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.158+0000 7fb4d9635b80 1 -- 172.21.15.151:0/445623135 wait complete. 2024-05-21T21:56:37.161 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.159+0000 7fb4d9635b80 1 -- start start 2024-05-21T21:56:37.161 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.159+0000 7fb4d9635b80 1 Processor -- start 2024-05-21T21:56:37.161 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.159+0000 7fb4d9635b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562eaac8b0f0 0x562eaacaed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:37.161 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.159+0000 7fb4d9635b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562eaac905c0 0x562eaacaf2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:37.161 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.159+0000 7fb4d9635b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562eaac90ee0 0x562eaaca9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:37.161 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.159+0000 7fb4d9635b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x562eaaca5770 con 0x562eaac8b0f0 2024-05-21T21:56:37.161 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.159+0000 7fb4d9635b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x562eaaca50a0 con 0x562eaac90ee0 2024-05-21T21:56:37.161 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.159+0000 7fb4d9635b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x562eaaca5e40 con 0x562eaac905c0 2024-05-21T21:56:37.161 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.160+0000 7fb4d9635b80 10 client.0 _ll_register_callbacks cb 0x562eaab82c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:56:37.161 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.160+0000 7fb4c7e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562eaac8b0f0 0x562eaacaed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:37.161 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.160+0000 7fb4c7400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562eaac905c0 0x562eaacaf2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:37.162 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.160+0000 7fb4c7400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562eaac905c0 0x562eaacaf2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:52248/0 (socket says 172.21.15.151:52248) 2024-05-21T21:56:37.162 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.160+0000 7fb4c7e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562eaac8b0f0 0x562eaacaed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:57884/0 (socket says 172.21.15.151:57884) 2024-05-21T21:56:37.162 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.160+0000 7fb4c7400700 1 -- 172.21.15.151:0/4221102150 learned_addr learned my addr 172.21.15.151:0/4221102150 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:56:37.162 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.160+0000 7fb4ccc00700 1 --2- 172.21.15.151:0/4221102150 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562eaac90ee0 0x562eaaca9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:37.162 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.160+0000 7fb4c7400700 1 -- 172.21.15.151:0/4221102150 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562eaac90ee0 msgr2=0x562eaaca9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:56:37.162 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.160+0000 7fb4c7400700 1 --2- 172.21.15.151:0/4221102150 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562eaac90ee0 0x562eaaca9d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:37.162 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.160+0000 7fb4c7400700 1 -- 172.21.15.151:0/4221102150 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562eaac8b0f0 msgr2=0x562eaacaed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:56:37.162 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.160+0000 7fb4c7400700 1 --2- 172.21.15.151:0/4221102150 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562eaac8b0f0 0x562eaacaed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:37.162 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.160+0000 7fb4c7400700 1 -- 172.21.15.151:0/4221102150 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562eaacaa260 con 0x562eaac905c0 2024-05-21T21:56:37.162 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.161+0000 7fb4c7400700 1 --2- 172.21.15.151:0/4221102150 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562eaac905c0 0x562eaacaf2b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb4c8076a70 tx=0x7fb4b4004580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fee5eab0d59241c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:56:37.162 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.161+0000 7fb4b2000700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-21T21:56:37.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.161+0000 7fb4b2000700 1 -- 172.21.15.151:0/4221102150 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b4005c20 con 0x562eaac905c0 2024-05-21T21:56:37.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.161+0000 7fb4d9635b80 1 -- 172.21.15.151:0/4221102150 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4b80aa900 con 0x562eaac905c0 2024-05-21T21:56:37.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.161+0000 7fb4b2000700 1 -- 172.21.15.151:0/4221102150 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fb4b4010d70 con 0x562eaac905c0 2024-05-21T21:56:37.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.162+0000 7fb4b2000700 1 -- 172.21.15.151:0/4221102150 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb4b4003680 con 0x562eaac905c0 2024-05-21T21:56:37.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.162+0000 7fb4d9635b80 10 client.25450 fetch_fsmap learned FSMap version 29 2024-05-21T21:56:37.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.162+0000 7fb4d9635b80 1 -- 172.21.15.151:0/4221102150 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562eaad102d0 con 0x562eaac905c0 2024-05-21T21:56:37.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.162+0000 7fb4b2000700 1 -- 172.21.15.151:0/4221102150 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4b80aa900 con 0x562eaac905c0 2024-05-21T21:56:37.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.162+0000 7fb4b2000700 1 -- 172.21.15.151:0/4221102150 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fb4b4011970 con 0x562eaac905c0 2024-05-21T21:56:37.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.162+0000 7fb4d9635b80 10 client.25450 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T21:56:37.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.162+0000 7fb4d9635b80 10 client.25450 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-21T21:56:37.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.162+0000 7fb4d9635b80 20 client.25450 populate_metadata read hostname 'smithi151' 2024-05-21T21:56:37.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.162+0000 7fb4d9635b80 1 --2- 172.21.15.151:0/4221102150 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x562eaad14d80 0x562eaad13a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:37.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.162+0000 7fb4d9635b80 4 client.25450 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-21T21:56:37.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.162+0000 7fb4d9635b80 1 -- 172.21.15.151:0/4221102150 --> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] -- command(tid 0: {"prefix": "status"}) -- 0x562eaabe4520 con 0x562eaad14d80 2024-05-21T21:56:37.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.163+0000 7fb4c7e00700 1 --2- 172.21.15.151:0/4221102150 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x562eaad14d80 0x562eaad13a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:37.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.163+0000 7fb4c7e00700 1 --2- 172.21.15.151:0/4221102150 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x562eaad14d80 0x562eaad13a90 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dc7f00c5ba1efde6 server_cookie=4734fb4db1ea9c8a in_seq=0 out_seq=0 2024-05-21T21:56:37.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.163+0000 7fb4b2000700 10 client.25450 ms_handle_connect on v2:172.21.15.164:6836/1554609572 2024-05-21T21:56:37.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.163+0000 7fb4b2000700 1 -- 172.21.15.151:0/4221102150 <== mds.2 v2:172.21.15.164:6836/1554609572 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x562eaabe4520 con 0x562eaad14d80 2024-05-21T21:56:37.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.163+0000 7fb4b2000700 10 client.25450 handle_command_reply: tid=0 2024-05-21T21:56:37.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.163+0000 7fb4d9635b80 1 client.25450 shutdown 2024-05-21T21:56:37.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.164+0000 7fb4d9635b80 1 -- 172.21.15.151:0/4221102150 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562eaac905c0 msgr2=0x562eaacaf2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:37.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.164+0000 7fb4d9635b80 1 --2- 172.21.15.151:0/4221102150 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562eaac905c0 0x562eaacaf2b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb4c8076a70 tx=0x7fb4b4004580 comp rx=0 tx=0).stop 2024-05-21T21:56:37.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.164+0000 7fb4d9635b80 1 -- 172.21.15.151:0/4221102150 shutdown_connections 2024-05-21T21:56:37.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.164+0000 7fb4d9635b80 1 --2- 172.21.15.151:0/4221102150 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x562eaad14d80 0x562eaad13a90 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:37.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.164+0000 7fb4d9635b80 1 --2- 172.21.15.151:0/4221102150 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562eaac90ee0 0x562eaaca9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:37.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.164+0000 7fb4d9635b80 1 --2- 172.21.15.151:0/4221102150 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562eaac8b0f0 0x562eaacaed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:37.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.164+0000 7fb4d9635b80 1 --2- 172.21.15.151:0/4221102150 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562eaac905c0 0x562eaacaf2b0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:37.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.164+0000 7fb4d9635b80 1 -- 172.21.15.151:0/4221102150 >> 172.21.15.151:0/4221102150 conn(0x562eaabb8b90 msgr2=0x562eaac91320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:56:37.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.164+0000 7fb4b2000700 0 client.25450 ms_handle_reset on v2:172.21.15.164:6836/1554609572 2024-05-21T21:56:37.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.164+0000 7fb4d9635b80 1 -- 172.21.15.151:0/4221102150 shutdown_connections 2024-05-21T21:56:37.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.164+0000 7fb4d9635b80 1 -- 172.21.15.151:0/4221102150 wait complete. 2024-05-21T21:56:37.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.164+0000 7fb4d9635b80 20 client.25450 trim_cache size 0 max 16384 2024-05-21T21:56:37.167 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-05-21T21:56:37.167 INFO:teuthology.orchestra.run.smithi151.stdout: "cluster_fsid": "c8c7a2e6-17ba-11ef-bc9a-c7b262605968", 2024-05-21T21:56:37.167 INFO:teuthology.orchestra.run.smithi151.stdout: "whoami": 2, 2024-05-21T21:56:37.167 INFO:teuthology.orchestra.run.smithi151.stdout: "id": 24491, 2024-05-21T21:56:37.168 INFO:teuthology.orchestra.run.smithi151.stdout: "want_state": "up:active", 2024-05-21T21:56:37.168 INFO:teuthology.orchestra.run.smithi151.stdout: "state": "up:active", 2024-05-21T21:56:37.168 INFO:teuthology.orchestra.run.smithi151.stdout: "fs_name": "cephfs", 2024-05-21T21:56:37.168 INFO:teuthology.orchestra.run.smithi151.stdout: "rank_uptime": 300.85743764699998, 2024-05-21T21:56:37.168 INFO:teuthology.orchestra.run.smithi151.stdout: "mdsmap_epoch": 29, 2024-05-21T21:56:37.168 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_epoch": 83, 2024-05-21T21:56:37.168 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_epoch_barrier": 82, 2024-05-21T21:56:37.168 INFO:teuthology.orchestra.run.smithi151.stdout: "uptime": 353.27413384699997 2024-05-21T21:56:37.168 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-05-21T21:56:37.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.165+0000 7fb4cec00700 1 -- 172.21.15.151:0/3722002823 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fb4ac0b2b10 msgr2=0x7fb4ac0b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:37.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.165+0000 7fb4cec00700 1 --2- 172.21.15.151:0/3722002823 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fb4ac0b2b10 0x7fb4ac0b4fd0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb4c00094f0 tx=0x7fb4c0004b40 comp rx=0 tx=0).stop 2024-05-21T21:56:37.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.165+0000 7fb4cec00700 1 -- 172.21.15.151:0/3722002823 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb4c8075770 msgr2=0x7fb4c819be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:37.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.165+0000 7fb4cec00700 1 --2- 172.21.15.151:0/3722002823 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb4c8075770 0x7fb4c819be10 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb4b8000c00 tx=0x7fb4b80057c0 comp rx=0 tx=0).stop 2024-05-21T21:56:37.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.165+0000 7fb4cec00700 1 -- 172.21.15.151:0/3722002823 shutdown_connections 2024-05-21T21:56:37.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.165+0000 7fb4cec00700 1 --2- 172.21.15.151:0/3722002823 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fb4ac0b2b10 0x7fb4ac0b4fd0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:37.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.165+0000 7fb4cec00700 1 --2- 172.21.15.151:0/3722002823 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb4c8075770 0x7fb4c819be10 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:37.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.165+0000 7fb4cec00700 1 --2- 172.21.15.151:0/3722002823 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb4c810ff40 0x7fb4c81a06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:37.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.165+0000 7fb4cec00700 1 --2- 172.21.15.151:0/3722002823 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4c8076090 0x7fb4c819c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:37.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.165+0000 7fb4cec00700 1 -- 172.21.15.151:0/3722002823 >> 172.21.15.151:0/3722002823 conn(0x7fb4c807a6a0 msgr2=0x7fb4c8100750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:56:37.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.165+0000 7fb4cec00700 1 -- 172.21.15.151:0/3722002823 shutdown_connections 2024-05-21T21:56:37.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:37.165+0000 7fb4cec00700 1 -- 172.21.15.151:0/3722002823 wait complete. 2024-05-21T21:56:37.889 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:56:37 smithi151 ceph-mon[31623]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 85 B/s wr, 8 op/s 2024-05-21T21:56:37.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:56:37 smithi184 ceph-mon[36942]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 85 B/s wr, 8 op/s 2024-05-21T21:56:38.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:56:37 smithi164 ceph-mon[36269]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 85 B/s wr, 8 op/s 2024-05-21T21:56:39.351 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.i status 2024-05-21T21:56:39.672 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:56:39 smithi151 ceph-mon[31623]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 85 B/s wr, 12 op/s 2024-05-21T21:56:39.974 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:39.973+0000 7fc1f7a00700 1 -- 172.21.15.151:0/1662805645 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc1f0076090 msgr2=0x7fc1f010fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:39.975 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:39.973+0000 7fc1f7a00700 1 --2- 172.21.15.151:0/1662805645 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc1f0076090 0x7fc1f010fa00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc1dc0098f0 tx=0x7fc1dc009c00 comp rx=0 tx=0).stop 2024-05-21T21:56:39.975 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:39.973+0000 7fc1f7a00700 1 -- 172.21.15.151:0/1662805645 shutdown_connections 2024-05-21T21:56:39.975 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:39.973+0000 7fc1f7a00700 1 --2- 172.21.15.151:0/1662805645 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc1f010ff40 0x7fc1f0112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:39.975 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:39.973+0000 7fc1f7a00700 1 --2- 172.21.15.151:0/1662805645 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc1f0075770 0x7fc1f0075b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:39.975 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:39.973+0000 7fc1f7a00700 1 --2- 172.21.15.151:0/1662805645 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc1f0076090 0x7fc1f010fa00 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:39.975 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:39.973+0000 7fc1f7a00700 1 -- 172.21.15.151:0/1662805645 >> 172.21.15.151:0/1662805645 conn(0x7fc1f007a6a0 msgr2=0x7fc1f007aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:56:39.975 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:39.973+0000 7fc1f7a00700 1 -- 172.21.15.151:0/1662805645 shutdown_connections 2024-05-21T21:56:39.975 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:39.973+0000 7fc1f7a00700 1 -- 172.21.15.151:0/1662805645 wait complete. 2024-05-21T21:56:39.975 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:39.974+0000 7fc1f7a00700 1 Processor -- start 2024-05-21T21:56:39.975 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:39.974+0000 7fc1f7a00700 1 -- start start 2024-05-21T21:56:39.976 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:39.975+0000 7fc1f7a00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc1f0075770 0x7fc1f019bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:39.976 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:39.975+0000 7fc1f7a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc1f0076090 0x7fc1f019c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:39.976 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:39.975+0000 7fc1f7a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc1f010ff40 0x7fc1f01a07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:39.976 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:39.975+0000 7fc1f7a00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f0115960 con 0x7fc1f010ff40 2024-05-21T21:56:39.976 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:39.975+0000 7fc1f7a00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f01157e0 con 0x7fc1f0076090 2024-05-21T21:56:39.976 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:39.975+0000 7fc1f7a00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f0115ae0 con 0x7fc1f0075770 2024-05-21T21:56:39.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:39.976+0000 7fc1efe00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc1f0076090 0x7fc1f019c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:39.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:39.976+0000 7fc1f5000700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc1f0075770 0x7fc1f019bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:39.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:39.976+0000 7fc1f5a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc1f010ff40 0x7fc1f01a07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:39.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:39.976+0000 7fc1efe00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc1f0076090 0x7fc1f019c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:60926/0 (socket says 172.21.15.151:60926) 2024-05-21T21:56:39.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:39.976+0000 7fc1efe00700 1 -- 172.21.15.151:0/3921920390 learned_addr learned my addr 172.21.15.151:0/3921920390 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:56:39.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:39.976+0000 7fc1efe00700 1 -- 172.21.15.151:0/3921920390 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc1f0075770 msgr2=0x7fc1f019bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:39.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:39.976+0000 7fc1efe00700 1 --2- 172.21.15.151:0/3921920390 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc1f0075770 0x7fc1f019bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:39.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:39.976+0000 7fc1efe00700 1 -- 172.21.15.151:0/3921920390 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc1f010ff40 msgr2=0x7fc1f01a07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:39.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:39.977+0000 7fc1efe00700 1 --2- 172.21.15.151:0/3921920390 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc1f010ff40 0x7fc1f01a07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:39.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:39.977+0000 7fc1efe00700 1 -- 172.21.15.151:0/3921920390 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1f01a0ee0 con 0x7fc1f0076090 2024-05-21T21:56:39.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:39.977+0000 7fc1f5a00700 1 --2- 172.21.15.151:0/3921920390 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc1f010ff40 0x7fc1f01a07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:56:39.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:39.977+0000 7fc1efe00700 1 --2- 172.21.15.151:0/3921920390 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc1f0076090 0x7fc1f019c410 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc1dc009230 tx=0x7fc1dc00f990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:56:39.979 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:39.977+0000 7fc1ed600700 1 -- 172.21.15.151:0/3921920390 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1dc027070 con 0x7fc1f0076090 2024-05-21T21:56:39.979 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:39.977+0000 7fc1ed600700 1 -- 172.21.15.151:0/3921920390 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1dc005570 con 0x7fc1f0076090 2024-05-21T21:56:39.979 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:39.977+0000 7fc1ed600700 1 -- 172.21.15.151:0/3921920390 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1dc004930 con 0x7fc1f0076090 2024-05-21T21:56:39.979 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:39.977+0000 7fc1f7a00700 1 -- 172.21.15.151:0/3921920390 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1f01a1170 con 0x7fc1f0076090 2024-05-21T21:56:39.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:39.978+0000 7fc1f7a00700 1 -- 172.21.15.151:0/3921920390 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1f01a8a50 con 0x7fc1f0076090 2024-05-21T21:56:39.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:39.979+0000 7fc1f7a00700 1 -- 172.21.15.151:0/3921920390 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1f0076e60 con 0x7fc1f0076090 2024-05-21T21:56:39.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:39.983+0000 7fc1ed600700 1 -- 172.21.15.151:0/3921920390 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 149886+0+0 (secure 0 0 0) 0x7fc1dc02a070 con 0x7fc1f0076090 2024-05-21T21:56:39.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:39.985+0000 7fc1ed600700 1 --2- 172.21.15.151:0/3921920390 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fc1d80b2b10 0x7fc1d80b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:39.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:56:39 smithi184 ceph-mon[36942]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 85 B/s wr, 12 op/s 2024-05-21T21:56:39.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:39.985+0000 7fc1ed600700 1 -- 172.21.15.151:0/3921920390 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fc1dc0e3700 con 0x7fc1f0076090 2024-05-21T21:56:39.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:39.985+0000 7fc1f5000700 1 --2- 172.21.15.151:0/3921920390 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fc1d80b2b10 0x7fc1d80b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:39.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:39.986+0000 7fc1ed600700 1 -- 172.21.15.151:0/3921920390 <== mon.2 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc1dc031020 con 0x7fc1f0076090 2024-05-21T21:56:39.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:39.986+0000 7fc1f5000700 1 --2- 172.21.15.151:0/3921920390 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fc1d80b2b10 0x7fc1d80b4fd0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc1e4009c60 tx=0x7fc1e4009260 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:56:39.989 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:56:40.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:56:39 smithi164 ceph-mon[36269]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 85 B/s wr, 12 op/s 2024-05-21T21:56:40.290 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:40.289+0000 7fc1f7a00700 1 -- 172.21.15.151:0/3921920390 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc1f002e560 con 0x7fc1f0076090 2024-05-21T21:56:40.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:40.291+0000 7fc1ed600700 1 -- 172.21.15.151:0/3921920390 <== mon.2 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11958 (secure 0 0 0) 0x7fc1dc013030 con 0x7fc1f0076090 2024-05-21T21:56:40.291 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T21:56:40.291 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14358,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/158010529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":158010529},{"type":"v1","addr":"172.21.15.151:6835","nonce":158010529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6839/2069058115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":2069058115},{"type":"v1","addr":"172.21.15.151:6839","nonce":2069058115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T21:51:19.552069+0000","modified":"2024-05-21T21:55:34.643755+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24530,"mds_1":24493,"mds_2":24491,"mds_3":24478,"mds_4":24473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14403":{"gid":14403,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6841/1749723794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6840","nonce":1749723794},{"type":"v1","addr":"172.21.15.151:6841","nonce":1749723794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/2026266114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":2026266114},{"type":"v1","addr":"172.21.15.184:6833","nonce":2026266114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/2467900844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":2467900844},{"type":"v1","addr":"172.21.15.164:6833","nonce":2467900844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3367640960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3367640960},{"type":"v1","addr":"172.21.15.151:6837","nonce":3367640960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/3264142621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3264142621},{"type":"v1","addr":"172.21.15.184:6835","nonce":3264142621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":17,"addr":"172.21.15.164:6835/2340428470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2340428470},{"type":"v1","addr":"172.21.15.164:6835","nonce":2340428470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":16,"addr":"172.21.15.184:6837/2988194888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":2988194888},{"type":"v1","addr":"172.21.15.184:6837","nonce":2988194888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.164:6837/1554609572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1554609572},{"type":"v1","addr":"172.21.15.164:6837","nonce":1554609572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.184:6839/3059935143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":3059935143},{"type":"v1","addr":"172.21.15.184:6839","nonce":3059935143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.164:6839/2646401247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":2646401247},{"type":"v1","addr":"172.21.15.164:6839","nonce":2646401247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24530,"qdb_cluster":[24530,24493,24491,24473,24478]},"id":1}]} 2024-05-21T21:56:40.294 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:40.294+0000 7fc1f7a00700 1 -- 172.21.15.151:0/3921920390 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fc1d80b2b10 msgr2=0x7fc1d80b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:40.294 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:40.294+0000 7fc1f7a00700 1 --2- 172.21.15.151:0/3921920390 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fc1d80b2b10 0x7fc1d80b4fd0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc1e4009c60 tx=0x7fc1e4009260 comp rx=0 tx=0).stop 2024-05-21T21:56:40.294 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:40.294+0000 7fc1f7a00700 1 -- 172.21.15.151:0/3921920390 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc1f0076090 msgr2=0x7fc1f019c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:40.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:40.294+0000 7fc1f7a00700 1 --2- 172.21.15.151:0/3921920390 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc1f0076090 0x7fc1f019c410 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc1dc009230 tx=0x7fc1dc00f990 comp rx=0 tx=0).stop 2024-05-21T21:56:40.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:40.294+0000 7fc1f7a00700 1 -- 172.21.15.151:0/3921920390 shutdown_connections 2024-05-21T21:56:40.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:40.294+0000 7fc1f7a00700 1 --2- 172.21.15.151:0/3921920390 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fc1d80b2b10 0x7fc1d80b4fd0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:40.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:40.294+0000 7fc1f7a00700 1 --2- 172.21.15.151:0/3921920390 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc1f010ff40 0x7fc1f01a07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:40.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:40.294+0000 7fc1f7a00700 1 --2- 172.21.15.151:0/3921920390 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc1f0075770 0x7fc1f019bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:40.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:40.294+0000 7fc1f7a00700 1 --2- 172.21.15.151:0/3921920390 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc1f0076090 0x7fc1f019c410 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:40.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:40.294+0000 7fc1f7a00700 1 -- 172.21.15.151:0/3921920390 >> 172.21.15.151:0/3921920390 conn(0x7fc1f007a6a0 msgr2=0x7fc1f0100850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:56:40.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:40.295+0000 7fc1f7a00700 1 -- 172.21.15.151:0/3921920390 shutdown_connections 2024-05-21T21:56:40.296 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:40.295+0000 7fc1f7a00700 1 -- 172.21.15.151:0/3921920390 wait complete. 2024-05-21T21:56:40.296 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 29 2024-05-21T21:56:40.938 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:56:40 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/3921920390' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T21:56:40.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:56:40 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/3921920390' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T21:56:41.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:56:40 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/3921920390' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T21:56:41.558 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-21T21:56:41.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:56:41 smithi151 ceph-mon[31623]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T21:56:41.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:56:41 smithi184 ceph-mon[36942]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T21:56:42.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:56:41 smithi164 ceph-mon[36269]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T21:56:42.117 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:56:42.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.920+0000 7fe072e00700 1 -- 172.21.15.151:0/3266393493 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe06c1095b0 msgr2=0x7fe06c109990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:42.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.920+0000 7fe072e00700 1 --2- 172.21.15.151:0/3266393493 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe06c1095b0 0x7fe06c109990 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe05c009990 tx=0x7fe05c009ca0 comp rx=0 tx=0).stop 2024-05-21T21:56:42.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.921+0000 7fe072e00700 1 -- 172.21.15.151:0/3266393493 shutdown_connections 2024-05-21T21:56:42.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.921+0000 7fe072e00700 1 --2- 172.21.15.151:0/3266393493 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe06c1095b0 0x7fe06c109990 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:42.922 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.921+0000 7fe072e00700 1 --2- 172.21.15.151:0/3266393493 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe06c06bbe0 0x7fe06c1103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:42.922 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.921+0000 7fe072e00700 1 --2- 172.21.15.151:0/3266393493 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe06c06b220 0x7fe06c06b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:42.922 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.921+0000 7fe072e00700 1 -- 172.21.15.151:0/3266393493 >> 172.21.15.151:0/3266393493 conn(0x7fe06c0793f0 msgr2=0x7fe06c079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:56:42.922 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.921+0000 7fe072e00700 1 -- 172.21.15.151:0/3266393493 shutdown_connections 2024-05-21T21:56:42.922 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.921+0000 7fe072e00700 1 -- 172.21.15.151:0/3266393493 wait complete. 2024-05-21T21:56:42.922 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.922+0000 7fe072e00700 1 Processor -- start 2024-05-21T21:56:42.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.922+0000 7fe072e00700 1 -- start start 2024-05-21T21:56:42.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.922+0000 7fe072e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe06c06b220 0x7fe06c19f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:42.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.922+0000 7fe072e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe06c06bbe0 0x7fe06c19fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:42.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.922+0000 7fe072e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe06c1095b0 0x7fe06c199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:42.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.922+0000 7fe072e00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe06c113710 con 0x7fe06c06b220 2024-05-21T21:56:42.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.922+0000 7fe072e00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe06c113590 con 0x7fe06c06bbe0 2024-05-21T21:56:42.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.922+0000 7fe072e00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe06c113890 con 0x7fe06c1095b0 2024-05-21T21:56:42.924 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.923+0000 7fe06be00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe06c06b220 0x7fe06c19f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:42.924 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.923+0000 7fe06be00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe06c06b220 0x7fe06c19f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:57924/0 (socket says 172.21.15.151:57924) 2024-05-21T21:56:42.924 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.923+0000 7fe06be00700 1 -- 172.21.15.151:0/344441524 learned_addr learned my addr 172.21.15.151:0/344441524 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:56:42.924 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.923+0000 7fe063400700 1 --2- 172.21.15.151:0/344441524 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe06c06bbe0 0x7fe06c19fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:42.924 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.923+0000 7fe070e00700 1 --2- 172.21.15.151:0/344441524 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe06c1095b0 0x7fe06c199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:42.924 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.923+0000 7fe063400700 1 -- 172.21.15.151:0/344441524 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe06c1095b0 msgr2=0x7fe06c199a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:42.924 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.923+0000 7fe063400700 1 --2- 172.21.15.151:0/344441524 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe06c1095b0 0x7fe06c199a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:42.924 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.923+0000 7fe063400700 1 -- 172.21.15.151:0/344441524 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe06c06b220 msgr2=0x7fe06c19f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:42.924 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.923+0000 7fe063400700 1 --2- 172.21.15.151:0/344441524 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe06c06b220 0x7fe06c19f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:42.925 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.923+0000 7fe063400700 1 -- 172.21.15.151:0/344441524 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe06c19a1b0 con 0x7fe06c06bbe0 2024-05-21T21:56:42.925 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.924+0000 7fe063400700 1 --2- 172.21.15.151:0/344441524 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe06c06bbe0 0x7fe06c19fdd0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe05400e9f0 tx=0x7fe05400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:56:42.925 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.924+0000 7fe069600700 1 -- 172.21.15.151:0/344441524 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe05400cb70 con 0x7fe06c06bbe0 2024-05-21T21:56:42.925 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.924+0000 7fe069600700 1 -- 172.21.15.151:0/344441524 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe05400ee90 con 0x7fe06c06bbe0 2024-05-21T21:56:42.925 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.924+0000 7fe072e00700 1 -- 172.21.15.151:0/344441524 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe06c19a4a0 con 0x7fe06c06bbe0 2024-05-21T21:56:42.925 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.924+0000 7fe069600700 1 -- 172.21.15.151:0/344441524 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe054003d40 con 0x7fe06c06bbe0 2024-05-21T21:56:42.925 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.924+0000 7fe072e00700 1 -- 172.21.15.151:0/344441524 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe06c1a6760 con 0x7fe06c06bbe0 2024-05-21T21:56:42.927 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.927+0000 7fe069600700 1 -- 172.21.15.151:0/344441524 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 149886+0+0 (secure 0 0 0) 0x7fe054003ee0 con 0x7fe06c06bbe0 2024-05-21T21:56:42.928 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.929+0000 7fe07d6b6b80 1 Processor -- start 2024-05-21T21:56:42.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.929+0000 7fe07d6b6b80 1 -- start start 2024-05-21T21:56:42.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.929+0000 7fe069600700 1 --2- 172.21.15.151:0/344441524 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fe0500b2b90 0x7fe0500b5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:42.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.929+0000 7fe069600700 1 -- 172.21.15.151:0/344441524 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fe0540e2990 con 0x7fe06c06bbe0 2024-05-21T21:56:42.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.929+0000 7fe07d6b6b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56532c4d6e60 0x56532c4d7240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:42.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.929+0000 7fe070e00700 1 --2- 172.21.15.151:0/344441524 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fe0500b2b90 0x7fe0500b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:42.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.929+0000 7fe07d6b6b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56532c4d7780 0x56532c4d1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:42.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.929+0000 7fe07d6b6b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56532c4d1830 0x56532c4d1c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:42.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.929+0000 7fe07d6b6b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56532c42fdf0 con 0x56532c4d6e60 2024-05-21T21:56:42.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.929+0000 7fe07d6b6b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56532c41f990 con 0x56532c4d1830 2024-05-21T21:56:42.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.929+0000 7fe07d6b6b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56532c4239d0 con 0x56532c4d7780 2024-05-21T21:56:42.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.930+0000 7fe06be00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56532c4d6e60 0x56532c4d7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:42.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.930+0000 7fe06be00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56532c4d6e60 0x56532c4d7240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:57938/0 (socket says 172.21.15.151:57938) 2024-05-21T21:56:42.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.930+0000 7fe06be00700 1 -- 172.21.15.151:0/3679803988 learned_addr learned my addr 172.21.15.151:0/3679803988 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:56:42.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.930+0000 7fe063400700 1 --2- 172.21.15.151:0/3679803988 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56532c4d7780 0x56532c4d1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:42.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.930+0000 7fe070e00700 1 --2- 172.21.15.151:0/3679803988 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56532c4d1830 0x56532c4d1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:42.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.930+0000 7fe06be00700 1 -- 172.21.15.151:0/3679803988 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56532c4d7780 msgr2=0x56532c4d1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:56:42.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.930+0000 7fe06be00700 1 --2- 172.21.15.151:0/3679803988 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56532c4d7780 0x56532c4d1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:42.932 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.930+0000 7fe06be00700 1 -- 172.21.15.151:0/3679803988 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56532c4d1830 msgr2=0x56532c4d1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:56:42.932 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.930+0000 7fe06be00700 1 --2- 172.21.15.151:0/3679803988 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56532c4d1830 0x56532c4d1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:42.932 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.930+0000 7fe06be00700 1 -- 172.21.15.151:0/3679803988 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56532c3e83d0 con 0x56532c4d6e60 2024-05-21T21:56:42.932 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.930+0000 7fe063400700 1 --2- 172.21.15.151:0/3679803988 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56532c4d7780 0x56532c4d1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:56:42.932 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.930+0000 7fe070e00700 1 --2- 172.21.15.151:0/3679803988 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56532c4d1830 0x56532c4d1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T21:56:42.932 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.930+0000 7fe070e00700 1 --2- 172.21.15.151:0/344441524 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fe0500b2b90 0x7fe0500b5050 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe06400f870 tx=0x7fe06400fc30 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:56:42.932 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.931+0000 7fe06be00700 1 --2- 172.21.15.151:0/3679803988 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56532c4d6e60 0x56532c4d7240 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe05c0096d0 tx=0x7fe05c009700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=623b46b3ba51f764 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:56:42.932 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.931+0000 7fe061600700 1 -- 172.21.15.151:0/3679803988 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe05c027070 con 0x56532c4d6e60 2024-05-21T21:56:42.932 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.931+0000 7fe061600700 1 -- 172.21.15.151:0/3679803988 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe05c0056d0 con 0x56532c4d6e60 2024-05-21T21:56:42.932 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.931+0000 7fe061600700 1 -- 172.21.15.151:0/3679803988 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe05c003be0 con 0x56532c4d6e60 2024-05-21T21:56:42.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.932+0000 7fe07d6b6b80 1 -- 172.21.15.151:0/3679803988 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56532c4d6e60 msgr2=0x56532c4d7240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:42.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.932+0000 7fe07d6b6b80 1 --2- 172.21.15.151:0/3679803988 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56532c4d6e60 0x56532c4d7240 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe05c0096d0 tx=0x7fe05c009700 comp rx=0 tx=0).stop 2024-05-21T21:56:42.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.932+0000 7fe07d6b6b80 1 -- 172.21.15.151:0/3679803988 shutdown_connections 2024-05-21T21:56:42.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.932+0000 7fe07d6b6b80 1 --2- 172.21.15.151:0/3679803988 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56532c4d1830 0x56532c4d1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:42.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.932+0000 7fe07d6b6b80 1 --2- 172.21.15.151:0/3679803988 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56532c4d6e60 0x56532c4d7240 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:42.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.932+0000 7fe07d6b6b80 1 --2- 172.21.15.151:0/3679803988 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56532c4d7780 0x56532c4d1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:42.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.932+0000 7fe07d6b6b80 1 -- 172.21.15.151:0/3679803988 >> 172.21.15.151:0/3679803988 conn(0x56532c3ffb90 msgr2=0x56532c4cbfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:56:42.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.932+0000 7fe07d6b6b80 1 -- 172.21.15.151:0/3679803988 shutdown_connections 2024-05-21T21:56:42.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.932+0000 7fe07d6b6b80 1 -- 172.21.15.151:0/3679803988 wait complete. 2024-05-21T21:56:42.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.933+0000 7fe07d6b6b80 1 -- start start 2024-05-21T21:56:42.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.933+0000 7fe07d6b6b80 1 Processor -- start 2024-05-21T21:56:42.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.933+0000 7fe07d6b6b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56532c4d1830 0x56532c4dd460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:42.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.933+0000 7fe07d6b6b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56532c4d6e60 0x56532c4da830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:42.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.933+0000 7fe07d6b6b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56532c4d7780 0x56532c4dad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:42.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.933+0000 7fe07d6b6b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56532c325f50 con 0x56532c4d1830 2024-05-21T21:56:42.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.933+0000 7fe07d6b6b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56532c3c0710 con 0x56532c4d7780 2024-05-21T21:56:42.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.933+0000 7fe07d6b6b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56532c4de190 con 0x56532c4d6e60 2024-05-21T21:56:42.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.933+0000 7fe07d6b6b80 10 client.0 _ll_register_callbacks cb 0x56532c3c9c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:56:42.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.934+0000 7fe06be00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56532c4d1830 0x56532c4dd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:42.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.934+0000 7fe06be00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56532c4d1830 0x56532c4dd460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:57954/0 (socket says 172.21.15.151:57954) 2024-05-21T21:56:42.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.934+0000 7fe06be00700 1 -- 172.21.15.151:0/2982497606 learned_addr learned my addr 172.21.15.151:0/2982497606 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:56:42.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.934+0000 7fe063400700 1 --2- 172.21.15.151:0/2982497606 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56532c4d6e60 0x56532c4da830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:42.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.934+0000 7fe070e00700 1 --2- 172.21.15.151:0/2982497606 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56532c4d7780 0x56532c4dad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:42.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.934+0000 7fe06be00700 1 -- 172.21.15.151:0/2982497606 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56532c4d6e60 msgr2=0x56532c4da830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:56:42.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.934+0000 7fe06be00700 1 --2- 172.21.15.151:0/2982497606 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56532c4d6e60 0x56532c4da830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:42.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.934+0000 7fe06be00700 1 -- 172.21.15.151:0/2982497606 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56532c4d7780 msgr2=0x56532c4dad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:56:42.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.934+0000 7fe06be00700 1 --2- 172.21.15.151:0/2982497606 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56532c4d7780 0x56532c4dad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:42.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.934+0000 7fe06be00700 1 -- 172.21.15.151:0/2982497606 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56532c36f880 con 0x56532c4d1830 2024-05-21T21:56:42.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.934+0000 7fe063400700 1 --2- 172.21.15.151:0/2982497606 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56532c4d6e60 0x56532c4da830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:56:42.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.934+0000 7fe06be00700 1 --2- 172.21.15.151:0/2982497606 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56532c4d1830 0x56532c4dd460 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe05c003bd0 tx=0x7fe05c003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e4c7452e1a18051 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:56:42.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.935+0000 7fe05a000700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-21T21:56:42.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.935+0000 7fe05a000700 1 -- 172.21.15.151:0/2982497606 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe05c031060 con 0x56532c4d1830 2024-05-21T21:56:42.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.935+0000 7fe05a000700 1 -- 172.21.15.151:0/2982497606 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fe05c00b720 con 0x56532c4d1830 2024-05-21T21:56:42.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.935+0000 7fe07d6b6b80 1 -- 172.21.15.151:0/2982497606 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe05c0056d0 con 0x56532c4d1830 2024-05-21T21:56:42.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.935+0000 7fe05a000700 1 -- 172.21.15.151:0/2982497606 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe05c027070 con 0x56532c4d1830 2024-05-21T21:56:42.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.935+0000 7fe07d6b6b80 10 client.15342 fetch_fsmap learned FSMap version 29 2024-05-21T21:56:42.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.935+0000 7fe07d6b6b80 1 -- 172.21.15.151:0/2982497606 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56532c3f0010 con 0x56532c4d1830 2024-05-21T21:56:42.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.935+0000 7fe05a000700 1 -- 172.21.15.151:0/2982497606 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe05c0056d0 con 0x56532c4d1830 2024-05-21T21:56:42.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.936+0000 7fe05a000700 1 -- 172.21.15.151:0/2982497606 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fe05c021070 con 0x56532c4d1830 2024-05-21T21:56:42.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.936+0000 7fe07d6b6b80 10 client.15342 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T21:56:42.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.936+0000 7fe07d6b6b80 10 client.15342 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-21T21:56:42.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.936+0000 7fe07d6b6b80 20 client.15342 populate_metadata read hostname 'smithi151' 2024-05-21T21:56:42.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.936+0000 7fe07d6b6b80 1 --2- 172.21.15.151:0/2982497606 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x56532c4edda0 0x56532c4eff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:42.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.936+0000 7fe07d6b6b80 4 client.15342 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T21:56:42.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.936+0000 7fe07d6b6b80 1 -- 172.21.15.151:0/2982497606 --> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56532c431d40 con 0x56532c4edda0 2024-05-21T21:56:42.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.936+0000 7fe063400700 1 --2- 172.21.15.151:0/2982497606 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x56532c4edda0 0x56532c4eff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:42.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.937+0000 7fe05a000700 10 client.15342 ms_handle_connect on v2:172.21.15.184:6838/3059935143 2024-05-21T21:56:42.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.937+0000 7fe063400700 1 --2- 172.21.15.151:0/2982497606 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x56532c4edda0 0x56532c4eff00 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=35f75a63acc20790 server_cookie=b65ef2fccdcc7c64 in_seq=0 out_seq=0 2024-05-21T21:56:42.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.937+0000 7fe05a000700 1 -- 172.21.15.151:0/2982497606 <== mds.1 v2:172.21.15.184:6838/3059935143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x56532c431d40 con 0x56532c4edda0 2024-05-21T21:56:42.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.937+0000 7fe05a000700 10 client.15342 handle_command_reply: tid=0 2024-05-21T21:56:42.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.938+0000 7fe07d6b6b80 1 client.15342 shutdown 2024-05-21T21:56:42.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.938+0000 7fe07d6b6b80 1 -- 172.21.15.151:0/2982497606 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56532c4d1830 msgr2=0x56532c4dd460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:42.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.938+0000 7fe07d6b6b80 1 --2- 172.21.15.151:0/2982497606 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56532c4d1830 0x56532c4dd460 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe05c003bd0 tx=0x7fe05c003680 comp rx=0 tx=0).stop 2024-05-21T21:56:42.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.938+0000 7fe07d6b6b80 1 -- 172.21.15.151:0/2982497606 shutdown_connections 2024-05-21T21:56:42.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.938+0000 7fe07d6b6b80 1 --2- 172.21.15.151:0/2982497606 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x56532c4edda0 0x56532c4eff00 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:42.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.938+0000 7fe07d6b6b80 1 --2- 172.21.15.151:0/2982497606 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56532c4d7780 0x56532c4dad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:42.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.938+0000 7fe07d6b6b80 1 --2- 172.21.15.151:0/2982497606 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56532c4d1830 0x56532c4dd460 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:42.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.938+0000 7fe07d6b6b80 1 --2- 172.21.15.151:0/2982497606 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56532c4d6e60 0x56532c4da830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:42.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.938+0000 7fe05a000700 0 client.15342 ms_handle_reset on v2:172.21.15.184:6838/3059935143 2024-05-21T21:56:42.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.938+0000 7fe07d6b6b80 1 -- 172.21.15.151:0/2982497606 >> 172.21.15.151:0/2982497606 conn(0x56532c3ffb90 msgr2=0x56532c4d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:56:42.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.938+0000 7fe07d6b6b80 1 -- 172.21.15.151:0/2982497606 shutdown_connections 2024-05-21T21:56:42.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.938+0000 7fe07d6b6b80 1 -- 172.21.15.151:0/2982497606 wait complete. 2024-05-21T21:56:42.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.938+0000 7fe07d6b6b80 20 client.15342 trim_cache size 0 max 16384 2024-05-21T21:56:42.962 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.962+0000 7fe07d6b6b80 1 Processor -- start 2024-05-21T21:56:42.962 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.962+0000 7fe07d6b6b80 1 -- start start 2024-05-21T21:56:42.962 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.962+0000 7fe07d6b6b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56532c4d1830 0x56532c4d1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:42.963 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.962+0000 7fe07d6b6b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56532c4d6e60 0x56532c4d1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:42.963 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.962+0000 7fe07d6b6b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56532c4d7780 0x56532c4dcb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:42.963 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.962+0000 7fe07d6b6b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56532c4ee240 con 0x56532c4d7780 2024-05-21T21:56:42.963 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.962+0000 7fe07d6b6b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56532c4ee910 con 0x56532c4d1830 2024-05-21T21:56:42.963 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.962+0000 7fe07d6b6b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56532c4eefe0 con 0x56532c4d6e60 2024-05-21T21:56:42.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.963+0000 7fe06be00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56532c4d1830 0x56532c4d1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:42.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.963+0000 7fe070e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56532c4d7780 0x56532c4dcb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:42.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.963+0000 7fe06be00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56532c4d1830 0x56532c4d1210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:60972/0 (socket says 172.21.15.151:60972) 2024-05-21T21:56:42.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.963+0000 7fe06be00700 1 -- 172.21.15.151:0/2134412637 learned_addr learned my addr 172.21.15.151:0/2134412637 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:56:42.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.963+0000 7fe063400700 1 --2- 172.21.15.151:0/2134412637 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56532c4d6e60 0x56532c4d1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:42.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.963+0000 7fe06be00700 1 -- 172.21.15.151:0/2134412637 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56532c4d6e60 msgr2=0x56532c4d1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:56:42.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.963+0000 7fe06be00700 1 --2- 172.21.15.151:0/2134412637 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56532c4d6e60 0x56532c4d1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:42.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.963+0000 7fe06be00700 1 -- 172.21.15.151:0/2134412637 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56532c4d7780 msgr2=0x56532c4dcb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:56:42.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.963+0000 7fe06be00700 1 --2- 172.21.15.151:0/2134412637 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56532c4d7780 0x56532c4dcb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:42.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.963+0000 7fe06be00700 1 -- 172.21.15.151:0/2134412637 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56532c4dd0a0 con 0x56532c4d1830 2024-05-21T21:56:42.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.963+0000 7fe063400700 1 --2- 172.21.15.151:0/2134412637 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56532c4d6e60 0x56532c4d1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T21:56:42.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.964+0000 7fe06be00700 1 --2- 172.21.15.151:0/2134412637 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56532c4d1830 0x56532c4d1210 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe05c003830 tx=0x7fe05c009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0683ad1543dfdcb server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:56:42.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.964+0000 7fe060c00700 1 -- 172.21.15.151:0/2134412637 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe05c031040 con 0x56532c4d1830 2024-05-21T21:56:42.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.964+0000 7fe060c00700 1 -- 172.21.15.151:0/2134412637 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe05c028040 con 0x56532c4d1830 2024-05-21T21:56:42.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.964+0000 7fe060c00700 1 -- 172.21.15.151:0/2134412637 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe05c022b70 con 0x56532c4d1830 2024-05-21T21:56:42.966 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.965+0000 7fe07d6b6b80 1 -- 172.21.15.151:0/2134412637 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56532c4d1830 msgr2=0x56532c4d1210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:42.966 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.965+0000 7fe07d6b6b80 1 --2- 172.21.15.151:0/2134412637 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56532c4d1830 0x56532c4d1210 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe05c003830 tx=0x7fe05c009ca0 comp rx=0 tx=0).stop 2024-05-21T21:56:42.966 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.965+0000 7fe07d6b6b80 1 -- 172.21.15.151:0/2134412637 shutdown_connections 2024-05-21T21:56:42.966 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.965+0000 7fe07d6b6b80 1 --2- 172.21.15.151:0/2134412637 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56532c4d1830 0x56532c4d1210 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:42.966 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.965+0000 7fe07d6b6b80 1 --2- 172.21.15.151:0/2134412637 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56532c4d7780 0x56532c4dcb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:42.966 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.965+0000 7fe07d6b6b80 1 --2- 172.21.15.151:0/2134412637 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56532c4d6e60 0x56532c4d1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:42.966 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.965+0000 7fe07d6b6b80 1 -- 172.21.15.151:0/2134412637 >> 172.21.15.151:0/2134412637 conn(0x56532c3ffb90 msgr2=0x56532c4d4970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:56:42.966 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.966+0000 7fe07d6b6b80 1 -- 172.21.15.151:0/2134412637 shutdown_connections 2024-05-21T21:56:42.966 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.966+0000 7fe07d6b6b80 1 -- 172.21.15.151:0/2134412637 wait complete. 2024-05-21T21:56:42.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.966+0000 7fe07d6b6b80 1 -- start start 2024-05-21T21:56:42.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.966+0000 7fe07d6b6b80 1 Processor -- start 2024-05-21T21:56:42.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.967+0000 7fe07d6b6b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56532c4d1830 0x56532c4f9560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:42.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.967+0000 7fe07d6b6b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56532c4d6e60 0x56532c4f9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:42.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.967+0000 7fe07d6b6b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56532c4d7780 0x56532c4f4390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:42.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.967+0000 7fe07d6b6b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56532c4efd80 con 0x56532c4d6e60 2024-05-21T21:56:42.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.967+0000 7fe07d6b6b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56532c4ef6b0 con 0x56532c4d1830 2024-05-21T21:56:42.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.967+0000 7fe07d6b6b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56532c4f0450 con 0x56532c4d7780 2024-05-21T21:56:42.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.967+0000 7fe07d6b6b80 10 client.0 _ll_register_callbacks cb 0x56532c3c9c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:56:42.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.967+0000 7fe06be00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56532c4d1830 0x56532c4f9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:42.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.967+0000 7fe063400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56532c4d6e60 0x56532c4f9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:42.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.967+0000 7fe06be00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56532c4d1830 0x56532c4f9560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:60974/0 (socket says 172.21.15.151:60974) 2024-05-21T21:56:42.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.967+0000 7fe06be00700 1 -- 172.21.15.151:0/1549371392 learned_addr learned my addr 172.21.15.151:0/1549371392 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:56:42.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.967+0000 7fe070e00700 1 --2- 172.21.15.151:0/1549371392 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56532c4d7780 0x56532c4f4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:42.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.968+0000 7fe06be00700 1 -- 172.21.15.151:0/1549371392 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56532c4d7780 msgr2=0x56532c4f4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:56:42.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.968+0000 7fe06be00700 1 --2- 172.21.15.151:0/1549371392 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56532c4d7780 0x56532c4f4390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:42.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.968+0000 7fe06be00700 1 -- 172.21.15.151:0/1549371392 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56532c4d6e60 msgr2=0x56532c4f9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:56:42.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.968+0000 7fe06be00700 1 --2- 172.21.15.151:0/1549371392 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56532c4d6e60 0x56532c4f9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:42.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.968+0000 7fe06be00700 1 -- 172.21.15.151:0/1549371392 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56532c4f4bd0 con 0x56532c4d1830 2024-05-21T21:56:42.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.968+0000 7fe063400700 1 --2- 172.21.15.151:0/1549371392 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56532c4d6e60 0x56532c4f9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:56:42.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.968+0000 7fe06be00700 1 --2- 172.21.15.151:0/1549371392 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56532c4d1830 0x56532c4f9560 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe05c0036c0 tx=0x7fe05c022750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1032513c6e4b68b server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:56:42.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.968+0000 7fe05a000700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-21T21:56:42.970 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.968+0000 7fe05a000700 1 -- 172.21.15.151:0/1549371392 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe05c043060 con 0x56532c4d1830 2024-05-21T21:56:42.970 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.968+0000 7fe07d6b6b80 1 -- 172.21.15.151:0/1549371392 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe05c028040 con 0x56532c4d1830 2024-05-21T21:56:42.970 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.968+0000 7fe05a000700 1 -- 172.21.15.151:0/1549371392 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fe05c00f870 con 0x56532c4d1830 2024-05-21T21:56:42.970 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.968+0000 7fe05a000700 1 -- 172.21.15.151:0/1549371392 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe05c031040 con 0x56532c4d1830 2024-05-21T21:56:42.971 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.969+0000 7fe07d6b6b80 10 client.25466 fetch_fsmap learned FSMap version 29 2024-05-21T21:56:42.971 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.969+0000 7fe07d6b6b80 1 -- 172.21.15.151:0/1549371392 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56532c55adc0 con 0x56532c4d1830 2024-05-21T21:56:42.971 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.970+0000 7fe05a000700 1 -- 172.21.15.151:0/1549371392 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe05c028040 con 0x56532c4d1830 2024-05-21T21:56:42.971 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.970+0000 7fe05a000700 1 -- 172.21.15.151:0/1549371392 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fe05c021070 con 0x56532c4d1830 2024-05-21T21:56:42.971 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.970+0000 7fe07d6b6b80 10 client.25466 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T21:56:42.971 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.970+0000 7fe07d6b6b80 10 client.25466 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-21T21:56:42.971 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.970+0000 7fe07d6b6b80 20 client.25466 populate_metadata read hostname 'smithi151' 2024-05-21T21:56:42.971 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.971+0000 7fe07d6b6b80 1 --2- 172.21.15.151:0/1549371392 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x56532c55f5f0 0x56532c55e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:42.971 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.971+0000 7fe07d6b6b80 4 client.25466 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-21T21:56:42.971 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.971+0000 7fe07d6b6b80 1 -- 172.21.15.151:0/1549371392 --> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] -- command(tid 0: {"prefix": "status"}) -- 0x56532c42b520 con 0x56532c55f5f0 2024-05-21T21:56:42.972 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.971+0000 7fe063400700 1 --2- 172.21.15.151:0/1549371392 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x56532c55f5f0 0x56532c55e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:42.972 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.971+0000 7fe063400700 1 --2- 172.21.15.151:0/1549371392 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x56532c55f5f0 0x56532c55e300 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ac94feab2165dca5 server_cookie=1880f57612e2b0e5 in_seq=0 out_seq=0 2024-05-21T21:56:42.972 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.971+0000 7fe05a000700 10 client.25466 ms_handle_connect on v2:172.21.15.184:6838/3059935143 2024-05-21T21:56:42.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.972+0000 7fe05a000700 1 -- 172.21.15.151:0/1549371392 <== mds.1 v2:172.21.15.184:6838/3059935143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x56532c42b520 con 0x56532c55f5f0 2024-05-21T21:56:42.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.972+0000 7fe05a000700 10 client.25466 handle_command_reply: tid=0 2024-05-21T21:56:42.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.972+0000 7fe07d6b6b80 1 client.25466 shutdown 2024-05-21T21:56:42.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.972+0000 7fe07d6b6b80 1 -- 172.21.15.151:0/1549371392 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56532c4d1830 msgr2=0x56532c4f9560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:42.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.972+0000 7fe07d6b6b80 1 --2- 172.21.15.151:0/1549371392 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56532c4d1830 0x56532c4f9560 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe05c0036c0 tx=0x7fe05c022750 comp rx=0 tx=0).stop 2024-05-21T21:56:42.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.972+0000 7fe07d6b6b80 1 -- 172.21.15.151:0/1549371392 shutdown_connections 2024-05-21T21:56:42.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.972+0000 7fe07d6b6b80 1 --2- 172.21.15.151:0/1549371392 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x56532c55f5f0 0x56532c55e300 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:42.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.972+0000 7fe07d6b6b80 1 --2- 172.21.15.151:0/1549371392 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56532c4d1830 0x56532c4f9560 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:42.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.972+0000 7fe07d6b6b80 1 --2- 172.21.15.151:0/1549371392 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56532c4d7780 0x56532c4f4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:42.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.972+0000 7fe07d6b6b80 1 --2- 172.21.15.151:0/1549371392 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56532c4d6e60 0x56532c4f9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:42.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.972+0000 7fe07d6b6b80 1 -- 172.21.15.151:0/1549371392 >> 172.21.15.151:0/1549371392 conn(0x56532c3ffb90 msgr2=0x56532c4d4970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:56:42.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.972+0000 7fe05a000700 0 client.25466 ms_handle_reset on v2:172.21.15.184:6838/3059935143 2024-05-21T21:56:42.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.972+0000 7fe07d6b6b80 1 -- 172.21.15.151:0/1549371392 shutdown_connections 2024-05-21T21:56:42.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.972+0000 7fe07d6b6b80 1 -- 172.21.15.151:0/1549371392 wait complete. 2024-05-21T21:56:42.974 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.972+0000 7fe07d6b6b80 20 client.25466 trim_cache size 0 max 16384 2024-05-21T21:56:42.974 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-05-21T21:56:42.974 INFO:teuthology.orchestra.run.smithi151.stdout: "cluster_fsid": "c8c7a2e6-17ba-11ef-bc9a-c7b262605968", 2024-05-21T21:56:42.974 INFO:teuthology.orchestra.run.smithi151.stdout: "whoami": 1, 2024-05-21T21:56:42.974 INFO:teuthology.orchestra.run.smithi151.stdout: "id": 24493, 2024-05-21T21:56:42.974 INFO:teuthology.orchestra.run.smithi151.stdout: "want_state": "up:active", 2024-05-21T21:56:42.974 INFO:teuthology.orchestra.run.smithi151.stdout: "state": "up:active", 2024-05-21T21:56:42.975 INFO:teuthology.orchestra.run.smithi151.stdout: "fs_name": "cephfs", 2024-05-21T21:56:42.975 INFO:teuthology.orchestra.run.smithi151.stdout: "rank_uptime": 307.67364975800001, 2024-05-21T21:56:42.975 INFO:teuthology.orchestra.run.smithi151.stdout: "mdsmap_epoch": 29, 2024-05-21T21:56:42.975 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_epoch": 83, 2024-05-21T21:56:42.975 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_epoch_barrier": 82, 2024-05-21T21:56:42.975 INFO:teuthology.orchestra.run.smithi151.stdout: "uptime": 356.04135886300003 2024-05-21T21:56:42.975 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-05-21T21:56:42.975 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.973+0000 7fe072e00700 1 -- 172.21.15.151:0/344441524 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fe0500b2b90 msgr2=0x7fe0500b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:42.975 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.973+0000 7fe072e00700 1 --2- 172.21.15.151:0/344441524 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fe0500b2b90 0x7fe0500b5050 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe06400f870 tx=0x7fe06400fc30 comp rx=0 tx=0).stop 2024-05-21T21:56:42.976 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.973+0000 7fe072e00700 1 -- 172.21.15.151:0/344441524 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe06c06bbe0 msgr2=0x7fe06c19fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:42.976 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.973+0000 7fe072e00700 1 --2- 172.21.15.151:0/344441524 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe06c06bbe0 0x7fe06c19fdd0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe05400e9f0 tx=0x7fe05400ed00 comp rx=0 tx=0).stop 2024-05-21T21:56:42.976 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.973+0000 7fe072e00700 1 -- 172.21.15.151:0/344441524 shutdown_connections 2024-05-21T21:56:42.976 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.973+0000 7fe072e00700 1 --2- 172.21.15.151:0/344441524 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fe0500b2b90 0x7fe0500b5050 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:42.976 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.973+0000 7fe072e00700 1 --2- 172.21.15.151:0/344441524 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe06c1095b0 0x7fe06c199a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:42.976 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.973+0000 7fe072e00700 1 --2- 172.21.15.151:0/344441524 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe06c06b220 0x7fe06c19f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:42.976 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.973+0000 7fe072e00700 1 --2- 172.21.15.151:0/344441524 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe06c06bbe0 0x7fe06c19fdd0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:42.976 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.973+0000 7fe072e00700 1 -- 172.21.15.151:0/344441524 >> 172.21.15.151:0/344441524 conn(0x7fe06c0793f0 msgr2=0x7fe06c1010e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:56:42.976 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.974+0000 7fe072e00700 1 -- 172.21.15.151:0/344441524 shutdown_connections 2024-05-21T21:56:42.976 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:42.974+0000 7fe072e00700 1 -- 172.21.15.151:0/344441524 wait complete. 2024-05-21T21:56:43.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:56:43 smithi151 ceph-mon[31623]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-21T21:56:43.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:56:43 smithi184 ceph-mon[36942]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-21T21:56:44.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:56:43 smithi164 ceph-mon[36269]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-21T21:56:44.645 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.b status 2024-05-21T21:56:45.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.201+0000 7fcf83000700 1 -- 172.21.15.151:0/4155428132 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf7c06bbe0 msgr2=0x7fcf7c1103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:45.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.201+0000 7fcf83000700 1 --2- 172.21.15.151:0/4155428132 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf7c06bbe0 0x7fcf7c1103d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fcf7400b0a0 tx=0x7fcf7400b3b0 comp rx=0 tx=0).stop 2024-05-21T21:56:45.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.201+0000 7fcf83000700 1 -- 172.21.15.151:0/4155428132 shutdown_connections 2024-05-21T21:56:45.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.201+0000 7fcf83000700 1 --2- 172.21.15.151:0/4155428132 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf7c06bbe0 0x7fcf7c1103d0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:45.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.201+0000 7fcf83000700 1 --2- 172.21.15.151:0/4155428132 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf7c1095b0 0x7fcf7c109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:45.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.201+0000 7fcf83000700 1 --2- 172.21.15.151:0/4155428132 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcf7c06b220 0x7fcf7c06b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:45.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.201+0000 7fcf83000700 1 -- 172.21.15.151:0/4155428132 >> 172.21.15.151:0/4155428132 conn(0x7fcf7c0793f0 msgr2=0x7fcf7c079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:56:45.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.201+0000 7fcf83000700 1 -- 172.21.15.151:0/4155428132 shutdown_connections 2024-05-21T21:56:45.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.201+0000 7fcf83000700 1 -- 172.21.15.151:0/4155428132 wait complete. 2024-05-21T21:56:45.203 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.203+0000 7fcf83000700 1 Processor -- start 2024-05-21T21:56:45.203 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.203+0000 7fcf83000700 1 -- start start 2024-05-21T21:56:45.205 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.204+0000 7fcf83000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcf7c06b220 0x7fcf7c102510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:45.205 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.204+0000 7fcf83000700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf7c06bbe0 0x7fcf7c102a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:45.205 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.204+0000 7fcf83000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf7c1095b0 0x7fcf7c102f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:45.205 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.204+0000 7fcf83000700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7c1136e0 con 0x7fcf7c1095b0 2024-05-21T21:56:45.205 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.204+0000 7fcf83000700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7c113560 con 0x7fcf7c06b220 2024-05-21T21:56:45.205 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.204+0000 7fcf83000700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7c113860 con 0x7fcf7c06bbe0 2024-05-21T21:56:45.206 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.204+0000 7fcf7be00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcf7c06b220 0x7fcf7c102510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:45.206 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.204+0000 7fcf7be00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcf7c06b220 0x7fcf7c102510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:60986/0 (socket says 172.21.15.151:60986) 2024-05-21T21:56:45.206 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.204+0000 7fcf7be00700 1 -- 172.21.15.151:0/3158736282 learned_addr learned my addr 172.21.15.151:0/3158736282 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:56:45.206 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.204+0000 7fcf7b400700 1 --2- 172.21.15.151:0/3158736282 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf7c06bbe0 0x7fcf7c102a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:45.206 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.204+0000 7fcf81000700 1 --2- 172.21.15.151:0/3158736282 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf7c1095b0 0x7fcf7c102f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:45.206 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.205+0000 7fcf7b400700 1 -- 172.21.15.151:0/3158736282 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcf7c06b220 msgr2=0x7fcf7c102510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:45.206 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.205+0000 7fcf7b400700 1 --2- 172.21.15.151:0/3158736282 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcf7c06b220 0x7fcf7c102510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:45.206 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.205+0000 7fcf7b400700 1 -- 172.21.15.151:0/3158736282 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf7c1095b0 msgr2=0x7fcf7c102f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:45.206 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.205+0000 7fcf7b400700 1 --2- 172.21.15.151:0/3158736282 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf7c1095b0 0x7fcf7c102f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:45.206 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.205+0000 7fcf7b400700 1 -- 172.21.15.151:0/3158736282 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf7c1a8140 con 0x7fcf7c06bbe0 2024-05-21T21:56:45.206 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.205+0000 7fcf81000700 1 --2- 172.21.15.151:0/3158736282 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf7c1095b0 0x7fcf7c102f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T21:56:45.206 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.205+0000 7fcf7be00700 1 --2- 172.21.15.151:0/3158736282 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcf7c06b220 0x7fcf7c102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:56:45.206 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.205+0000 7fcf7b400700 1 --2- 172.21.15.151:0/3158736282 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf7c06bbe0 0x7fcf7c102a50 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fcf6800e9f0 tx=0x7fcf6800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:56:45.207 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.205+0000 7fcf78c00700 1 -- 172.21.15.151:0/3158736282 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf6800cb70 con 0x7fcf7c06bbe0 2024-05-21T21:56:45.207 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.205+0000 7fcf78c00700 1 -- 172.21.15.151:0/3158736282 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf680044d0 con 0x7fcf7c06bbe0 2024-05-21T21:56:45.207 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.205+0000 7fcf83000700 1 -- 172.21.15.151:0/3158736282 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf7c1a8430 con 0x7fcf7c06bbe0 2024-05-21T21:56:45.207 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.205+0000 7fcf78c00700 1 -- 172.21.15.151:0/3158736282 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf68003e00 con 0x7fcf7c06bbe0 2024-05-21T21:56:45.207 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.205+0000 7fcf83000700 1 -- 172.21.15.151:0/3158736282 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf7c1a8970 con 0x7fcf7c06bbe0 2024-05-21T21:56:45.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.210+0000 7fcf78c00700 1 -- 172.21.15.151:0/3158736282 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 26) ==== 149886+0+0 (secure 0 0 0) 0x7fcf68010430 con 0x7fcf7c06bbe0 2024-05-21T21:56:45.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.210+0000 7fcf8d963b80 1 Processor -- start 2024-05-21T21:56:45.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.210+0000 7fcf8d963b80 1 -- start start 2024-05-21T21:56:45.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.211+0000 7fcf8d963b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564e0ce4be20 0x564e0ce4c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:45.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.211+0000 7fcf8d963b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564e0ce4c740 0x564e0ce46330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:45.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.211+0000 7fcf8d963b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564e0ce46950 0x564e0ce46dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:45.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.211+0000 7fcf8d963b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x564e0cd9be40 con 0x564e0ce4c740 2024-05-21T21:56:45.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.211+0000 7fcf8d963b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x564e0cd8b9e0 con 0x564e0ce46950 2024-05-21T21:56:45.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.211+0000 7fcf8d963b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x564e0cd8fa20 con 0x564e0ce4be20 2024-05-21T21:56:45.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.211+0000 7fcf81000700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564e0ce4be20 0x564e0ce4c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:45.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.211+0000 7fcf7be00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564e0ce4c740 0x564e0ce46330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:45.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.211+0000 7fcf7b400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564e0ce46950 0x564e0ce46dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:45.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.211+0000 7fcf7be00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564e0ce4c740 0x564e0ce46330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:58000/0 (socket says 172.21.15.151:58000) 2024-05-21T21:56:45.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.211+0000 7fcf7be00700 1 -- 172.21.15.151:0/4150814071 learned_addr learned my addr 172.21.15.151:0/4150814071 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:56:45.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.212+0000 7fcf81000700 1 -- 172.21.15.151:0/4150814071 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564e0ce46950 msgr2=0x564e0ce46dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:56:45.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.212+0000 7fcf81000700 1 --2- 172.21.15.151:0/4150814071 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564e0ce46950 0x564e0ce46dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:45.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.212+0000 7fcf81000700 1 -- 172.21.15.151:0/4150814071 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564e0ce4c740 msgr2=0x564e0ce46330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:56:45.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.212+0000 7fcf81000700 1 --2- 172.21.15.151:0/4150814071 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564e0ce4c740 0x564e0ce46330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:45.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.212+0000 7fcf81000700 1 -- 172.21.15.151:0/4150814071 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e0cd54420 con 0x564e0ce4be20 2024-05-21T21:56:45.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.212+0000 7fcf78c00700 1 --2- 172.21.15.151:0/3158736282 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fcf600b2b10 0x7fcf600b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:45.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.212+0000 7fcf7b400700 1 --2- 172.21.15.151:0/4150814071 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564e0ce46950 0x564e0ce46dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:56:45.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.212+0000 7fcf78c00700 1 -- 172.21.15.151:0/3158736282 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fcf680e2b30 con 0x7fcf7c06bbe0 2024-05-21T21:56:45.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.212+0000 7fcf81000700 1 --2- 172.21.15.151:0/4150814071 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564e0ce4be20 0x564e0ce4c200 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fcf7400d010 tx=0x7fcf74009360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=853a9bb700a1fc1a server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:56:45.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.212+0000 7fcf7be00700 1 --2- 172.21.15.151:0/3158736282 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fcf600b2b10 0x7fcf600b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:45.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.213+0000 7fcf71600700 1 -- 172.21.15.151:0/4150814071 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf7400bcb0 con 0x564e0ce4be20 2024-05-21T21:56:45.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.213+0000 7fcf71600700 1 -- 172.21.15.151:0/4150814071 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf740074e0 con 0x564e0ce4be20 2024-05-21T21:56:45.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.213+0000 7fcf71600700 1 -- 172.21.15.151:0/4150814071 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf74025680 con 0x564e0ce4be20 2024-05-21T21:56:45.219 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.214+0000 7fcf8d963b80 1 -- 172.21.15.151:0/4150814071 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564e0ce4be20 msgr2=0x564e0ce4c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:45.219 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.214+0000 7fcf7be00700 1 --2- 172.21.15.151:0/3158736282 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fcf600b2b10 0x7fcf600b4fd0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c011870 tx=0x7fcf6c009260 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:56:45.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.214+0000 7fcf8d963b80 1 --2- 172.21.15.151:0/4150814071 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564e0ce4be20 0x564e0ce4c200 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fcf7400d010 tx=0x7fcf74009360 comp rx=0 tx=0).stop 2024-05-21T21:56:45.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.214+0000 7fcf8d963b80 1 -- 172.21.15.151:0/4150814071 shutdown_connections 2024-05-21T21:56:45.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.214+0000 7fcf8d963b80 1 --2- 172.21.15.151:0/4150814071 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564e0ce46950 0x564e0ce46dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:45.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.214+0000 7fcf8d963b80 1 --2- 172.21.15.151:0/4150814071 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564e0ce4be20 0x564e0ce4c200 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:45.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.214+0000 7fcf8d963b80 1 --2- 172.21.15.151:0/4150814071 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564e0ce4c740 0x564e0ce46330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:45.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.214+0000 7fcf8d963b80 1 -- 172.21.15.151:0/4150814071 >> 172.21.15.151:0/4150814071 conn(0x564e0cd6bbe0 msgr2=0x564e0ce38690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:56:45.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.214+0000 7fcf8d963b80 1 -- 172.21.15.151:0/4150814071 shutdown_connections 2024-05-21T21:56:45.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.214+0000 7fcf8d963b80 1 -- 172.21.15.151:0/4150814071 wait complete. 2024-05-21T21:56:45.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.215+0000 7fcf8d963b80 1 -- start start 2024-05-21T21:56:45.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.215+0000 7fcf8d963b80 1 Processor -- start 2024-05-21T21:56:45.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.216+0000 7fcf8d963b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564e0ce46950 0x564e0ce524a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:45.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.216+0000 7fcf8d963b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564e0ce4be20 0x564e0ce4f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:45.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.216+0000 7fcf8d963b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564e0ce4c740 0x564e0ce4fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:45.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.216+0000 7fcf8d963b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x564e0cc91fa0 con 0x564e0ce46950 2024-05-21T21:56:45.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.216+0000 7fcf8d963b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x564e0cd2c760 con 0x564e0ce4be20 2024-05-21T21:56:45.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.216+0000 7fcf8d963b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x564e0ce53190 con 0x564e0ce4c740 2024-05-21T21:56:45.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.216+0000 7fcf8d963b80 10 client.0 _ll_register_callbacks cb 0x564e0cd35c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:56:45.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.216+0000 7fcf7be00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564e0ce46950 0x564e0ce524a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:45.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.216+0000 7fcf81000700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564e0ce4c740 0x564e0ce4fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:45.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.216+0000 7fcf7be00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564e0ce46950 0x564e0ce524a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:58012/0 (socket says 172.21.15.151:58012) 2024-05-21T21:56:45.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.216+0000 7fcf81000700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564e0ce4c740 0x564e0ce4fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:52332/0 (socket says 172.21.15.151:52332) 2024-05-21T21:56:45.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.216+0000 7fcf7be00700 1 -- 172.21.15.151:0/382196220 learned_addr learned my addr 172.21.15.151:0/382196220 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:56:45.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.217+0000 7fcf7b400700 1 --2- 172.21.15.151:0/382196220 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564e0ce4be20 0x564e0ce4f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:45.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.217+0000 7fcf7be00700 1 -- 172.21.15.151:0/382196220 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564e0ce4c740 msgr2=0x564e0ce4fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:56:45.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.217+0000 7fcf7be00700 1 --2- 172.21.15.151:0/382196220 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564e0ce4c740 0x564e0ce4fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:45.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.217+0000 7fcf7be00700 1 -- 172.21.15.151:0/382196220 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564e0ce4be20 msgr2=0x564e0ce4f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:56:45.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.217+0000 7fcf7be00700 1 --2- 172.21.15.151:0/382196220 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564e0ce4be20 0x564e0ce4f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:45.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.217+0000 7fcf7be00700 1 -- 172.21.15.151:0/382196220 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e0ccdb8d0 con 0x564e0ce46950 2024-05-21T21:56:45.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.217+0000 7fcf7be00700 1 --2- 172.21.15.151:0/382196220 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564e0ce46950 0x564e0ce524a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c00fad0 tx=0x7fcf6c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e28c12ea2a2ef028 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:56:45.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.217+0000 7fcf66000700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-21T21:56:45.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.218+0000 7fcf66000700 1 -- 172.21.15.151:0/382196220 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf6c02d060 con 0x564e0ce46950 2024-05-21T21:56:45.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.218+0000 7fcf66000700 1 -- 172.21.15.151:0/382196220 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fcf6c017030 con 0x564e0ce46950 2024-05-21T21:56:45.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.218+0000 7fcf8d963b80 1 -- 172.21.15.151:0/382196220 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf740074e0 con 0x564e0ce46950 2024-05-21T21:56:45.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.218+0000 7fcf66000700 1 -- 172.21.15.151:0/382196220 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcf6c01d7b0 con 0x564e0ce46950 2024-05-21T21:56:45.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.218+0000 7fcf8d963b80 10 client.15360 fetch_fsmap learned FSMap version 29 2024-05-21T21:56:45.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.218+0000 7fcf8d963b80 1 -- 172.21.15.151:0/382196220 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564e0cd5c060 con 0x564e0ce46950 2024-05-21T21:56:45.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.219+0000 7fcf66000700 1 -- 172.21.15.151:0/382196220 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf740074e0 con 0x564e0ce46950 2024-05-21T21:56:45.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.219+0000 7fcf66000700 1 -- 172.21.15.151:0/382196220 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fcf6c030a40 con 0x564e0ce46950 2024-05-21T21:56:45.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.219+0000 7fcf8d963b80 10 client.15360 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T21:56:45.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.219+0000 7fcf8d963b80 10 client.15360 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-05-21T21:56:45.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.219+0000 7fcf8d963b80 20 client.15360 populate_metadata read hostname 'smithi151' 2024-05-21T21:56:45.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.219+0000 7fcf8d963b80 1 --2- 172.21.15.151:0/382196220 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x564e0ce5fd50 0x564e0ce62140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:45.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.219+0000 7fcf8d963b80 4 client.15360 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T21:56:45.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.219+0000 7fcf8d963b80 1 -- 172.21.15.151:0/382196220 --> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564e0cd9dd90 con 0x564e0ce5fd50 2024-05-21T21:56:45.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.220+0000 7fcf7b400700 1 --2- 172.21.15.151:0/382196220 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x564e0ce5fd50 0x564e0ce62140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:45.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.220+0000 7fcf7b400700 1 --2- 172.21.15.151:0/382196220 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x564e0ce5fd50 0x564e0ce62140 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c52dbb4be6132111 server_cookie=65c7d3b8ad1c4d7a in_seq=0 out_seq=0 2024-05-21T21:56:45.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.220+0000 7fcf66000700 10 client.15360 ms_handle_connect on v2:172.21.15.164:6834/2340428470 2024-05-21T21:56:45.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.221+0000 7fcf66000700 1 -- 172.21.15.151:0/382196220 <== mds.4 v2:172.21.15.164:6834/2340428470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x564e0cd9dd90 con 0x564e0ce5fd50 2024-05-21T21:56:45.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.221+0000 7fcf66000700 10 client.15360 handle_command_reply: tid=0 2024-05-21T21:56:45.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.221+0000 7fcf8d963b80 1 client.15360 shutdown 2024-05-21T21:56:45.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.221+0000 7fcf8d963b80 1 -- 172.21.15.151:0/382196220 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564e0ce46950 msgr2=0x564e0ce524a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:45.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.221+0000 7fcf8d963b80 1 --2- 172.21.15.151:0/382196220 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564e0ce46950 0x564e0ce524a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c00fad0 tx=0x7fcf6c0047c0 comp rx=0 tx=0).stop 2024-05-21T21:56:45.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.222+0000 7fcf8d963b80 1 -- 172.21.15.151:0/382196220 shutdown_connections 2024-05-21T21:56:45.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.222+0000 7fcf8d963b80 1 --2- 172.21.15.151:0/382196220 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x564e0ce5fd50 0x564e0ce62140 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:45.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.222+0000 7fcf8d963b80 1 --2- 172.21.15.151:0/382196220 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564e0ce4c740 0x564e0ce4fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:45.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.222+0000 7fcf8d963b80 1 --2- 172.21.15.151:0/382196220 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564e0ce46950 0x564e0ce524a0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:45.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.222+0000 7fcf66000700 0 client.15360 ms_handle_reset on v2:172.21.15.164:6834/2340428470 2024-05-21T21:56:45.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.222+0000 7fcf8d963b80 1 --2- 172.21.15.151:0/382196220 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564e0ce4be20 0x564e0ce4f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:45.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.222+0000 7fcf8d963b80 1 -- 172.21.15.151:0/382196220 >> 172.21.15.151:0/382196220 conn(0x564e0cd6bbe0 msgr2=0x564e0ce37d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:56:45.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.222+0000 7fcf8d963b80 1 -- 172.21.15.151:0/382196220 shutdown_connections 2024-05-21T21:56:45.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.222+0000 7fcf8d963b80 1 -- 172.21.15.151:0/382196220 wait complete. 2024-05-21T21:56:45.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.222+0000 7fcf8d963b80 20 client.15360 trim_cache size 0 max 16384 2024-05-21T21:56:45.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.239+0000 7fcf8d963b80 1 Processor -- start 2024-05-21T21:56:45.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.239+0000 7fcf8d963b80 1 -- start start 2024-05-21T21:56:45.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.239+0000 7fcf8d963b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564e0ce46950 0x564e0ce46330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:45.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.240+0000 7fcf8d963b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564e0ce4be20 0x564e0ce49f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:45.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.240+0000 7fcf8d963b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564e0ce4c740 0x564e0ce46d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:45.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.240+0000 7fcf8d963b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7c1abd00 con 0x564e0ce4c740 2024-05-21T21:56:45.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.240+0000 7fcf8d963b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7c1ad840 con 0x564e0ce46950 2024-05-21T21:56:45.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.240+0000 7fcf8d963b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7c1ac3d0 con 0x564e0ce4be20 2024-05-21T21:56:45.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.240+0000 7fcf81000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564e0ce4c740 0x564e0ce46d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:45.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.240+0000 7fcf81000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564e0ce4c740 0x564e0ce46d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:58014/0 (socket says 172.21.15.151:58014) 2024-05-21T21:56:45.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.240+0000 7fcf81000700 1 -- 172.21.15.151:0/3468051821 learned_addr learned my addr 172.21.15.151:0/3468051821 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:56:45.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.240+0000 7fcf7be00700 1 --2- 172.21.15.151:0/3468051821 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564e0ce46950 0x564e0ce46330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:45.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.240+0000 7fcf7b400700 1 --2- 172.21.15.151:0/3468051821 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564e0ce4be20 0x564e0ce49f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:45.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.240+0000 7fcf7be00700 1 -- 172.21.15.151:0/3468051821 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564e0ce4be20 msgr2=0x564e0ce49f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:56:45.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.240+0000 7fcf7be00700 1 --2- 172.21.15.151:0/3468051821 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564e0ce4be20 0x564e0ce49f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:45.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.240+0000 7fcf7be00700 1 -- 172.21.15.151:0/3468051821 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564e0ce4c740 msgr2=0x564e0ce46d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:56:45.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.240+0000 7fcf7be00700 1 --2- 172.21.15.151:0/3468051821 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564e0ce4c740 0x564e0ce46d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:45.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.240+0000 7fcf7be00700 1 -- 172.21.15.151:0/3468051821 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e0ce3a0e0 con 0x564e0ce46950 2024-05-21T21:56:45.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.240+0000 7fcf7b400700 1 --2- 172.21.15.151:0/3468051821 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564e0ce4be20 0x564e0ce49f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T21:56:45.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.241+0000 7fcf7be00700 1 --2- 172.21.15.151:0/3468051821 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564e0ce46950 0x564e0ce46330 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c030e10 tx=0x7fcf6c009ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ac6840552f36465 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:56:45.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.241+0000 7fcf70c00700 1 -- 172.21.15.151:0/3468051821 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf6c040070 con 0x564e0ce46950 2024-05-21T21:56:45.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.241+0000 7fcf70c00700 1 -- 172.21.15.151:0/3468051821 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf6c01de00 con 0x564e0ce46950 2024-05-21T21:56:45.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.241+0000 7fcf70c00700 1 -- 172.21.15.151:0/3468051821 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf6c0303a0 con 0x564e0ce46950 2024-05-21T21:56:45.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.242+0000 7fcf8d963b80 1 -- 172.21.15.151:0/3468051821 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564e0ce46950 msgr2=0x564e0ce46330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:45.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.242+0000 7fcf8d963b80 1 --2- 172.21.15.151:0/3468051821 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564e0ce46950 0x564e0ce46330 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c030e10 tx=0x7fcf6c009ea0 comp rx=0 tx=0).stop 2024-05-21T21:56:45.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.242+0000 7fcf8d963b80 1 -- 172.21.15.151:0/3468051821 shutdown_connections 2024-05-21T21:56:45.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.242+0000 7fcf8d963b80 1 --2- 172.21.15.151:0/3468051821 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564e0ce46950 0x564e0ce46330 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:45.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.242+0000 7fcf8d963b80 1 --2- 172.21.15.151:0/3468051821 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564e0ce4c740 0x564e0ce46d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:45.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.242+0000 7fcf8d963b80 1 --2- 172.21.15.151:0/3468051821 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564e0ce4be20 0x564e0ce49f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:45.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.242+0000 7fcf8d963b80 1 -- 172.21.15.151:0/3468051821 >> 172.21.15.151:0/3468051821 conn(0x564e0cd6bbe0 msgr2=0x564e0ce4cb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:56:45.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.242+0000 7fcf8d963b80 1 -- 172.21.15.151:0/3468051821 shutdown_connections 2024-05-21T21:56:45.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.243+0000 7fcf8d963b80 1 -- 172.21.15.151:0/3468051821 wait complete. 2024-05-21T21:56:45.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.243+0000 7fcf8d963b80 1 -- start start 2024-05-21T21:56:45.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.243+0000 7fcf8d963b80 1 Processor -- start 2024-05-21T21:56:45.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.244+0000 7fcf8d963b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564e0ce46950 0x564e0ce58600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:45.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.244+0000 7fcf8d963b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564e0ce4be20 0x564e0ce58b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:45.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.244+0000 7fcf8d963b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564e0ce4c740 0x564e0ce66e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:45.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.244+0000 7fcf8d963b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7c1ad170 con 0x564e0ce4be20 2024-05-21T21:56:45.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.244+0000 7fcf8d963b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7c1acaa0 con 0x564e0ce4c740 2024-05-21T21:56:45.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.244+0000 7fcf8d963b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7c19dbc0 con 0x564e0ce46950 2024-05-21T21:56:45.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.244+0000 7fcf8d963b80 10 client.0 _ll_register_callbacks cb 0x564e0cd35c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:56:45.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.245+0000 7fcf7be00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564e0ce46950 0x564e0ce58600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:45.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.245+0000 7fcf81000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564e0ce4c740 0x564e0ce66e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:45.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.245+0000 7fcf7b400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564e0ce4be20 0x564e0ce58b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:45.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.245+0000 7fcf81000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564e0ce4c740 0x564e0ce66e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:32788/0 (socket says 172.21.15.151:32788) 2024-05-21T21:56:45.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.245+0000 7fcf81000700 1 -- 172.21.15.151:0/610139097 learned_addr learned my addr 172.21.15.151:0/610139097 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:56:45.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.245+0000 7fcf7be00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564e0ce46950 0x564e0ce58600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:52354/0 (socket says 172.21.15.151:52354) 2024-05-21T21:56:45.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.245+0000 7fcf81000700 1 -- 172.21.15.151:0/610139097 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564e0ce46950 msgr2=0x564e0ce58600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:56:45.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.245+0000 7fcf81000700 1 --2- 172.21.15.151:0/610139097 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564e0ce46950 0x564e0ce58600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:45.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.245+0000 7fcf81000700 1 -- 172.21.15.151:0/610139097 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564e0ce4be20 msgr2=0x564e0ce58b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:56:45.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.245+0000 7fcf81000700 1 --2- 172.21.15.151:0/610139097 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564e0ce4be20 0x564e0ce58b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:45.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.245+0000 7fcf81000700 1 -- 172.21.15.151:0/610139097 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e0ce67640 con 0x564e0ce4c740 2024-05-21T21:56:45.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.245+0000 7fcf7b400700 1 --2- 172.21.15.151:0/610139097 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564e0ce4be20 0x564e0ce58b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T21:56:45.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.245+0000 7fcf81000700 1 --2- 172.21.15.151:0/610139097 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564e0ce4c740 0x564e0ce66e00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fcf74009610 tx=0x7fcf74003c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c17ef2c5e4f0948e server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:56:45.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.245+0000 7fcf66000700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-21T21:56:45.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.246+0000 7fcf66000700 1 -- 172.21.15.151:0/610139097 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf740076f0 con 0x564e0ce4c740 2024-05-21T21:56:45.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.246+0000 7fcf66000700 1 -- 172.21.15.151:0/610139097 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fcf74007890 con 0x564e0ce4c740 2024-05-21T21:56:45.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.246+0000 7fcf8d963b80 1 -- 172.21.15.151:0/610139097 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf6c01de00 con 0x564e0ce4c740 2024-05-21T21:56:45.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.246+0000 7fcf66000700 1 -- 172.21.15.151:0/610139097 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcf74007a90 con 0x564e0ce4c740 2024-05-21T21:56:45.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.246+0000 7fcf8d963b80 10 client.25481 fetch_fsmap learned FSMap version 29 2024-05-21T21:56:45.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.246+0000 7fcf8d963b80 1 -- 172.21.15.151:0/610139097 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564e0ceca420 con 0x564e0ce4c740 2024-05-21T21:56:45.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.246+0000 7fcf66000700 1 -- 172.21.15.151:0/610139097 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf6c01de00 con 0x564e0ce4c740 2024-05-21T21:56:45.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.247+0000 7fcf66000700 1 -- 172.21.15.151:0/610139097 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fcf74011070 con 0x564e0ce4c740 2024-05-21T21:56:45.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.247+0000 7fcf8d963b80 10 client.25481 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T21:56:45.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.247+0000 7fcf8d963b80 10 client.25481 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-05-21T21:56:45.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.247+0000 7fcf8d963b80 20 client.25481 populate_metadata read hostname 'smithi151' 2024-05-21T21:56:45.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.247+0000 7fcf8d963b80 1 --2- 172.21.15.151:0/610139097 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x564e0ceca6a0 0x564e0cecca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:45.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.247+0000 7fcf8d963b80 4 client.25481 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T21:56:45.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.247+0000 7fcf8d963b80 1 -- 172.21.15.151:0/610139097 --> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564e0cd97570 con 0x564e0ceca6a0 2024-05-21T21:56:45.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.247+0000 7fcf7be00700 1 --2- 172.21.15.151:0/610139097 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x564e0ceca6a0 0x564e0cecca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:45.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.248+0000 7fcf7be00700 1 --2- 172.21.15.151:0/610139097 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x564e0ceca6a0 0x564e0cecca90 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=16baa5e194d6b0a6 server_cookie=5039096ee051dff9 in_seq=0 out_seq=0 2024-05-21T21:56:45.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.248+0000 7fcf66000700 10 client.25481 ms_handle_connect on v2:172.21.15.164:6834/2340428470 2024-05-21T21:56:45.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.248+0000 7fcf66000700 1 -- 172.21.15.151:0/610139097 <== mds.4 v2:172.21.15.164:6834/2340428470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564e0cd97570 con 0x564e0ceca6a0 2024-05-21T21:56:45.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.248+0000 7fcf66000700 10 client.25481 handle_command_reply: tid=0 2024-05-21T21:56:45.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.248+0000 7fcf8d963b80 1 client.25481 shutdown 2024-05-21T21:56:45.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.248+0000 7fcf8d963b80 1 -- 172.21.15.151:0/610139097 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564e0ce4c740 msgr2=0x564e0ce66e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:45.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.248+0000 7fcf8d963b80 1 --2- 172.21.15.151:0/610139097 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564e0ce4c740 0x564e0ce66e00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fcf74009610 tx=0x7fcf74003c30 comp rx=0 tx=0).stop 2024-05-21T21:56:45.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.248+0000 7fcf8d963b80 1 -- 172.21.15.151:0/610139097 shutdown_connections 2024-05-21T21:56:45.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.248+0000 7fcf8d963b80 1 --2- 172.21.15.151:0/610139097 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x564e0ceca6a0 0x564e0cecca90 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:45.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.248+0000 7fcf8d963b80 1 --2- 172.21.15.151:0/610139097 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564e0ce4c740 0x564e0ce66e00 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:45.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.248+0000 7fcf8d963b80 1 --2- 172.21.15.151:0/610139097 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564e0ce46950 0x564e0ce58600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:45.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.248+0000 7fcf8d963b80 1 --2- 172.21.15.151:0/610139097 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564e0ce4be20 0x564e0ce58b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:45.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.248+0000 7fcf66000700 0 client.25481 ms_handle_reset on v2:172.21.15.164:6834/2340428470 2024-05-21T21:56:45.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.248+0000 7fcf8d963b80 1 -- 172.21.15.151:0/610139097 >> 172.21.15.151:0/610139097 conn(0x564e0cd6bbe0 msgr2=0x564e0ce4d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:56:45.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.249+0000 7fcf8d963b80 1 -- 172.21.15.151:0/610139097 shutdown_connections 2024-05-21T21:56:45.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.249+0000 7fcf8d963b80 1 -- 172.21.15.151:0/610139097 wait complete. 2024-05-21T21:56:45.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.249+0000 7fcf8d963b80 20 client.25481 trim_cache size 0 max 16384 2024-05-21T21:56:45.252 INFO:teuthology.orchestra.run.smithi151.stdout:[] 2024-05-21T21:56:45.253 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.250+0000 7fcf83000700 1 -- 172.21.15.151:0/3158736282 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fcf600b2b10 msgr2=0x7fcf600b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:45.253 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.250+0000 7fcf83000700 1 --2- 172.21.15.151:0/3158736282 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fcf600b2b10 0x7fcf600b4fd0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c011870 tx=0x7fcf6c009260 comp rx=0 tx=0).stop 2024-05-21T21:56:45.253 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.250+0000 7fcf83000700 1 -- 172.21.15.151:0/3158736282 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf7c06bbe0 msgr2=0x7fcf7c102a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:45.253 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.250+0000 7fcf83000700 1 --2- 172.21.15.151:0/3158736282 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf7c06bbe0 0x7fcf7c102a50 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fcf6800e9f0 tx=0x7fcf6800ed00 comp rx=0 tx=0).stop 2024-05-21T21:56:45.253 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.250+0000 7fcf83000700 1 -- 172.21.15.151:0/3158736282 shutdown_connections 2024-05-21T21:56:45.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.250+0000 7fcf83000700 1 --2- 172.21.15.151:0/3158736282 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fcf600b2b10 0x7fcf600b4fd0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:45.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.250+0000 7fcf83000700 1 --2- 172.21.15.151:0/3158736282 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcf7c06b220 0x7fcf7c102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:45.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.250+0000 7fcf83000700 1 --2- 172.21.15.151:0/3158736282 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf7c1095b0 0x7fcf7c102f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:45.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.250+0000 7fcf83000700 1 --2- 172.21.15.151:0/3158736282 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcf7c06bbe0 0x7fcf7c102a50 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:45.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.250+0000 7fcf83000700 1 -- 172.21.15.151:0/3158736282 >> 172.21.15.151:0/3158736282 conn(0x7fcf7c0793f0 msgr2=0x7fcf7c101180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:56:45.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.250+0000 7fcf83000700 1 -- 172.21.15.151:0/3158736282 shutdown_connections 2024-05-21T21:56:45.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:45.251+0000 7fcf83000700 1 -- 172.21.15.151:0/3158736282 wait complete. 2024-05-21T21:56:45.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:56:45 smithi151 ceph-mon[31623]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-21T21:56:45.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:56:45 smithi184 ceph-mon[36942]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-21T21:56:46.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:56:45 smithi164 ceph-mon[36269]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-21T21:56:46.077 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:56:46.119 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-21T21:56:47.469 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:56:47.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:56:47 smithi151 ceph-mon[31623]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T21:56:47.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:56:47 smithi184 ceph-mon[36942]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T21:56:48.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:56:47 smithi164 ceph-mon[36269]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T21:56:48.705 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.703+0000 7f7442200700 1 -- 172.21.15.151:0/3752208727 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f743c10eeb0 msgr2=0x7f743c116a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:48.705 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.703+0000 7f7442200700 1 --2- 172.21.15.151:0/3752208727 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f743c10eeb0 0x7f743c116a80 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f743400b0d0 tx=0x7f743400b3e0 comp rx=0 tx=0).stop 2024-05-21T21:56:48.705 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.703+0000 7f7442200700 1 -- 172.21.15.151:0/3752208727 shutdown_connections 2024-05-21T21:56:48.705 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.703+0000 7f7442200700 1 --2- 172.21.15.151:0/3752208727 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f743c10eeb0 0x7f743c116a80 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:48.705 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.703+0000 7f7442200700 1 --2- 172.21.15.151:0/3752208727 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f743c109dd0 0x7f743c10a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:48.705 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.703+0000 7f7442200700 1 --2- 172.21.15.151:0/3752208727 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f743c10a780 0x7f743c10e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:48.705 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.703+0000 7f7442200700 1 -- 172.21.15.151:0/3752208727 >> 172.21.15.151:0/3752208727 conn(0x7f743c076d10 msgr2=0x7f743c108360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:56:48.705 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.703+0000 7f7442200700 1 -- 172.21.15.151:0/3752208727 shutdown_connections 2024-05-21T21:56:48.705 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.703+0000 7f7442200700 1 -- 172.21.15.151:0/3752208727 wait complete. 2024-05-21T21:56:48.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.704+0000 7f7442200700 1 Processor -- start 2024-05-21T21:56:48.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.705+0000 7f7442200700 1 -- start start 2024-05-21T21:56:48.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.705+0000 7f7442200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f743c109dd0 0x7f743c1a00c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:48.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.705+0000 7f7442200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f743c10a780 0x7f743c1a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:48.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.705+0000 7f7442200700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f743c10eeb0 0x7f743c1a4990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:48.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.705+0000 7f7442200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f743c119d90 con 0x7f743c10a780 2024-05-21T21:56:48.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.705+0000 7f7442200700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f743c119c10 con 0x7f743c109dd0 2024-05-21T21:56:48.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.705+0000 7f7442200700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f743c119f10 con 0x7f743c10eeb0 2024-05-21T21:56:48.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.705+0000 7f7433e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f743c10a780 0x7f743c1a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:48.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.705+0000 7f743be00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f743c10eeb0 0x7f743c1a4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:48.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.705+0000 7f7433e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f743c10a780 0x7f743c1a0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49156/0 (socket says 172.21.15.151:49156) 2024-05-21T21:56:48.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.705+0000 7f7433e00700 1 -- 172.21.15.151:0/3809143450 learned_addr learned my addr 172.21.15.151:0/3809143450 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:56:48.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.705+0000 7f743b400700 1 --2- 172.21.15.151:0/3809143450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f743c109dd0 0x7f743c1a00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:48.708 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.706+0000 7f743be00700 1 -- 172.21.15.151:0/3809143450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f743c109dd0 msgr2=0x7f743c1a00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:48.708 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.706+0000 7f743be00700 1 --2- 172.21.15.151:0/3809143450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f743c109dd0 0x7f743c1a00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:48.708 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.706+0000 7f743be00700 1 -- 172.21.15.151:0/3809143450 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f743c10a780 msgr2=0x7f743c1a0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:48.708 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.706+0000 7f743be00700 1 --2- 172.21.15.151:0/3809143450 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f743c10a780 0x7f743c1a0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:48.708 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.706+0000 7f743be00700 1 -- 172.21.15.151:0/3809143450 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f743c1a50d0 con 0x7f743c10eeb0 2024-05-21T21:56:48.708 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.706+0000 7f743be00700 1 --2- 172.21.15.151:0/3809143450 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f743c10eeb0 0x7f743c1a4990 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f743400b820 tx=0x7f74340079a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:56:48.708 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.706+0000 7f7438c00700 1 -- 172.21.15.151:0/3809143450 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7434021070 con 0x7f743c10eeb0 2024-05-21T21:56:48.708 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.707+0000 7f7442200700 1 -- 172.21.15.151:0/3809143450 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f743c1a5360 con 0x7f743c10eeb0 2024-05-21T21:56:48.708 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.707+0000 7f7438c00700 1 -- 172.21.15.151:0/3809143450 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f743401d900 con 0x7f743c10eeb0 2024-05-21T21:56:48.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.707+0000 7f7438c00700 1 -- 172.21.15.151:0/3809143450 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f743401b910 con 0x7f743c10eeb0 2024-05-21T21:56:48.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.707+0000 7f7442200700 1 -- 172.21.15.151:0/3809143450 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f743c1acc40 con 0x7f743c10eeb0 2024-05-21T21:56:48.712 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.712+0000 7f7438c00700 1 -- 172.21.15.151:0/3809143450 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 26) ==== 149886+0+0 (secure 0 0 0) 0x7f7434024070 con 0x7f743c10eeb0 2024-05-21T21:56:48.713 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.713+0000 7f744cb04b80 1 Processor -- start 2024-05-21T21:56:48.714 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.713+0000 7f744cb04b80 1 -- start start 2024-05-21T21:56:48.714 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.713+0000 7f744cb04b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5650b81e9e60 0x5650b81ea240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:48.714 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.713+0000 7f744cb04b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650b81ea780 0x5650b81e4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:48.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.713+0000 7f744cb04b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650b81e4830 0x5650b81e4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:48.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.713+0000 7f744cb04b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5650b8142df0 con 0x5650b81ea780 2024-05-21T21:56:48.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.713+0000 7f744cb04b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5650b8132990 con 0x5650b81e4830 2024-05-21T21:56:48.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.713+0000 7f744cb04b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5650b81369d0 con 0x5650b81e9e60 2024-05-21T21:56:48.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.713+0000 7f7438c00700 1 --2- 172.21.15.151:0/3809143450 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f74200b2b90 0x7f74200b5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:48.716 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.714+0000 7f743b400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650b81ea780 0x5650b81e4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:48.716 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.714+0000 7f7433e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5650b81e9e60 0x5650b81ea240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:48.716 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.714+0000 7f743b400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650b81ea780 0x5650b81e4210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49158/0 (socket says 172.21.15.151:49158) 2024-05-21T21:56:48.716 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.714+0000 7f743b400700 1 -- 172.21.15.151:0/1744521474 learned_addr learned my addr 172.21.15.151:0/1744521474 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:56:48.716 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.714+0000 7f7433e00700 1 --2- 172.21.15.151:0/1744521474 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650b81e4830 0x5650b81e4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:48.716 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.714+0000 7f743be00700 1 --2- 172.21.15.151:0/3809143450 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f74200b2b90 0x7f74200b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:48.716 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.714+0000 7f7438c00700 1 -- 172.21.15.151:0/3809143450 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f74340e39c0 con 0x7f743c10eeb0 2024-05-21T21:56:48.716 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.714+0000 7f7433e00700 1 -- 172.21.15.151:0/1744521474 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650b81e4830 msgr2=0x5650b81e4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:56:48.716 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.714+0000 7f7433e00700 1 --2- 172.21.15.151:0/1744521474 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650b81e4830 0x5650b81e4c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:48.716 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.714+0000 7f7433e00700 1 -- 172.21.15.151:0/1744521474 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650b81ea780 msgr2=0x5650b81e4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:56:48.716 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.714+0000 7f7433e00700 1 --2- 172.21.15.151:0/1744521474 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650b81ea780 0x5650b81e4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:48.716 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.714+0000 7f7433e00700 1 -- 172.21.15.151:0/1744521474 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650b80fb3d0 con 0x5650b81e9e60 2024-05-21T21:56:48.717 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.714+0000 7f7433e00700 1 --2- 172.21.15.151:0/1744521474 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5650b81e9e60 0x5650b81ea240 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f742400a6d0 tx=0x7f742400a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a449bbc9386c62c server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:56:48.717 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.714+0000 7f743be00700 1 --2- 172.21.15.151:0/3809143450 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f74200b2b90 0x7f74200b5050 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f74340e0fa0 tx=0x7f74340099e0 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:56:48.717 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.715+0000 7f7430c00700 1 -- 172.21.15.151:0/1744521474 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74240142d0 con 0x5650b81e9e60 2024-05-21T21:56:48.717 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.715+0000 7f7430c00700 1 -- 172.21.15.151:0/1744521474 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7424011b60 con 0x5650b81e9e60 2024-05-21T21:56:48.717 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.715+0000 7f7430c00700 1 -- 172.21.15.151:0/1744521474 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7424012940 con 0x5650b81e9e60 2024-05-21T21:56:48.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.715+0000 7f744cb04b80 1 -- 172.21.15.151:0/1744521474 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5650b81e9e60 msgr2=0x5650b81ea240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:48.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.715+0000 7f744cb04b80 1 --2- 172.21.15.151:0/1744521474 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5650b81e9e60 0x5650b81ea240 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f742400a6d0 tx=0x7f742400a660 comp rx=0 tx=0).stop 2024-05-21T21:56:48.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.716+0000 7f744cb04b80 1 -- 172.21.15.151:0/1744521474 shutdown_connections 2024-05-21T21:56:48.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.716+0000 7f744cb04b80 1 --2- 172.21.15.151:0/1744521474 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650b81e4830 0x5650b81e4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:48.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.716+0000 7f744cb04b80 1 --2- 172.21.15.151:0/1744521474 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5650b81e9e60 0x5650b81ea240 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:48.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.716+0000 7f744cb04b80 1 --2- 172.21.15.151:0/1744521474 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650b81ea780 0x5650b81e4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:48.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.716+0000 7f744cb04b80 1 -- 172.21.15.151:0/1744521474 >> 172.21.15.151:0/1744521474 conn(0x5650b8112b90 msgr2=0x5650b81defc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:56:48.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.716+0000 7f744cb04b80 1 -- 172.21.15.151:0/1744521474 shutdown_connections 2024-05-21T21:56:48.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.716+0000 7f744cb04b80 1 -- 172.21.15.151:0/1744521474 wait complete. 2024-05-21T21:56:48.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.716+0000 7f744cb04b80 1 -- start start 2024-05-21T21:56:48.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.717+0000 7f744cb04b80 1 Processor -- start 2024-05-21T21:56:48.720 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.717+0000 7f744cb04b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5650b81e4830 0x5650b81f04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:48.720 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.717+0000 7f744cb04b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650b81e9e60 0x5650b81ed870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:48.720 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.717+0000 7f744cb04b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650b81ea780 0x5650b81eddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:48.720 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.717+0000 7f744cb04b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5650b8038f50 con 0x5650b81e9e60 2024-05-21T21:56:48.720 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.717+0000 7f744cb04b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5650b80d3710 con 0x5650b81ea780 2024-05-21T21:56:48.720 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.717+0000 7f744cb04b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5650b81f11f0 con 0x5650b81e4830 2024-05-21T21:56:48.720 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.717+0000 7f744cb04b80 10 client.0 _ll_register_callbacks cb 0x5650b80dcc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:56:48.720 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.718+0000 7f743b400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650b81e9e60 0x5650b81ed870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:48.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.718+0000 7f7433e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5650b81e4830 0x5650b81f04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:48.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.718+0000 7f7433e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5650b81e4830 0x5650b81f04d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:57920/0 (socket says 172.21.15.151:57920) 2024-05-21T21:56:48.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.718+0000 7f7433e00700 1 -- 172.21.15.151:0/1845654342 learned_addr learned my addr 172.21.15.151:0/1845654342 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:56:48.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.718+0000 7f7433e00700 1 --2- 172.21.15.151:0/1845654342 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650b81ea780 0x5650b81eddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:48.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.718+0000 7f743b400700 1 -- 172.21.15.151:0/1845654342 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5650b81e4830 msgr2=0x5650b81f04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:56:48.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.718+0000 7f743b400700 1 --2- 172.21.15.151:0/1845654342 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5650b81e4830 0x5650b81f04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:48.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.718+0000 7f743b400700 1 -- 172.21.15.151:0/1845654342 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650b81ea780 msgr2=0x5650b81eddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:56:48.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.718+0000 7f743b400700 1 --2- 172.21.15.151:0/1845654342 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650b81ea780 0x5650b81eddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:48.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.718+0000 7f743b400700 1 -- 172.21.15.151:0/1845654342 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650b8082880 con 0x5650b81e9e60 2024-05-21T21:56:48.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.718+0000 7f7433e00700 1 --2- 172.21.15.151:0/1845654342 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650b81ea780 0x5650b81eddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T21:56:48.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.718+0000 7f743b400700 1 --2- 172.21.15.151:0/1845654342 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650b81e9e60 0x5650b81ed870 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f742c010a20 tx=0x7f742c010d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36aeee85529ffe7b server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:56:48.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.718+0000 7f7429600700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-21T21:56:48.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.719+0000 7f7429600700 1 -- 172.21.15.151:0/1845654342 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f742c004300 con 0x5650b81e9e60 2024-05-21T21:56:48.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.719+0000 7f7429600700 1 -- 172.21.15.151:0/1845654342 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f742c0044a0 con 0x5650b81e9e60 2024-05-21T21:56:48.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.719+0000 7f744cb04b80 1 -- 172.21.15.151:0/1845654342 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7424011b60 con 0x5650b81e9e60 2024-05-21T21:56:48.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.719+0000 7f7429600700 1 -- 172.21.15.151:0/1845654342 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f742c00f890 con 0x5650b81e9e60 2024-05-21T21:56:48.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.719+0000 7f744cb04b80 10 client.15378 fetch_fsmap learned FSMap version 29 2024-05-21T21:56:48.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.719+0000 7f744cb04b80 1 -- 172.21.15.151:0/1845654342 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5650b8103010 con 0x5650b81e9e60 2024-05-21T21:56:48.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.719+0000 7f7429600700 1 -- 172.21.15.151:0/1845654342 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7424011b60 con 0x5650b81e9e60 2024-05-21T21:56:48.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.720+0000 7f7429600700 1 -- 172.21.15.151:0/1845654342 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f742c01a420 con 0x5650b81e9e60 2024-05-21T21:56:48.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.720+0000 7f744cb04b80 10 client.15378 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T21:56:48.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.720+0000 7f744cb04b80 10 client.15378 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-21T21:56:48.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.720+0000 7f744cb04b80 20 client.15378 populate_metadata read hostname 'smithi151' 2024-05-21T21:56:48.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.720+0000 7f744cb04b80 1 --2- 172.21.15.151:0/1845654342 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5650b81fde10 0x5650b81ff280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:48.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.720+0000 7f744cb04b80 4 client.15378 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T21:56:48.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.720+0000 7f744cb04b80 1 -- 172.21.15.151:0/1845654342 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5650b8144d40 con 0x5650b81fde10 2024-05-21T21:56:48.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.720+0000 7f7433e00700 1 --2- 172.21.15.151:0/1845654342 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5650b81fde10 0x5650b81ff280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:48.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.721+0000 7f7433e00700 1 --2- 172.21.15.151:0/1845654342 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5650b81fde10 0x5650b81ff280 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee95a2ad89839584 server_cookie=e44ab406b7bb19af in_seq=0 out_seq=0 2024-05-21T21:56:48.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.721+0000 7f7429600700 10 client.15378 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T21:56:48.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.722+0000 7f7429600700 1 -- 172.21.15.151:0/1845654342 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5650b8144d40 con 0x5650b81fde10 2024-05-21T21:56:48.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.722+0000 7f7429600700 10 client.15378 handle_command_reply: tid=0 2024-05-21T21:56:48.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.722+0000 7f744cb04b80 1 client.15378 shutdown 2024-05-21T21:56:48.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.722+0000 7f744cb04b80 1 -- 172.21.15.151:0/1845654342 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650b81e9e60 msgr2=0x5650b81ed870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:48.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.722+0000 7f744cb04b80 1 --2- 172.21.15.151:0/1845654342 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650b81e9e60 0x5650b81ed870 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f742c010a20 tx=0x7f742c010d30 comp rx=0 tx=0).stop 2024-05-21T21:56:48.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.722+0000 7f744cb04b80 1 -- 172.21.15.151:0/1845654342 shutdown_connections 2024-05-21T21:56:48.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.722+0000 7f744cb04b80 1 --2- 172.21.15.151:0/1845654342 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5650b81fde10 0x5650b81ff280 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:48.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.722+0000 7f744cb04b80 1 --2- 172.21.15.151:0/1845654342 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650b81ea780 0x5650b81eddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:48.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.722+0000 7f744cb04b80 1 --2- 172.21.15.151:0/1845654342 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5650b81e4830 0x5650b81f04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:48.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.722+0000 7f744cb04b80 1 --2- 172.21.15.151:0/1845654342 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650b81e9e60 0x5650b81ed870 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:48.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.722+0000 7f744cb04b80 1 -- 172.21.15.151:0/1845654342 >> 172.21.15.151:0/1845654342 conn(0x5650b8112b90 msgr2=0x5650b81e3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:56:48.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.722+0000 7f7429600700 0 client.15378 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T21:56:48.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.722+0000 7f744cb04b80 1 -- 172.21.15.151:0/1845654342 shutdown_connections 2024-05-21T21:56:48.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.722+0000 7f744cb04b80 1 -- 172.21.15.151:0/1845654342 wait complete. 2024-05-21T21:56:48.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.722+0000 7f744cb04b80 20 client.15378 trim_cache size 0 max 16384 2024-05-21T21:56:48.748 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.746+0000 7f744cb04b80 1 Processor -- start 2024-05-21T21:56:48.748 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.746+0000 7f744cb04b80 1 -- start start 2024-05-21T21:56:48.748 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.747+0000 7f744cb04b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5650b81e4830 0x5650b81e4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:48.748 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.747+0000 7f744cb04b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650b81e9e60 0x5650b81d8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:48.748 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.747+0000 7f744cb04b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650b81ea780 0x5650b81e7970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:48.748 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.747+0000 7f744cb04b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5650b81fd5c0 con 0x5650b81ea780 2024-05-21T21:56:48.748 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.747+0000 7f744cb04b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5650b81fdc90 con 0x5650b81e9e60 2024-05-21T21:56:48.748 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.747+0000 7f744cb04b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5650b81fe360 con 0x5650b81e4830 2024-05-21T21:56:48.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.747+0000 7f7433e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5650b81e4830 0x5650b81e4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:48.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.747+0000 7f7433e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650b81ea780 0x5650b81e7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:48.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.747+0000 7f7433e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650b81ea780 0x5650b81e7970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49182/0 (socket says 172.21.15.151:49182) 2024-05-21T21:56:48.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.747+0000 7f743b400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650b81e9e60 0x5650b81d8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:48.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.747+0000 7f7433e00700 1 -- 172.21.15.151:0/3035451421 learned_addr learned my addr 172.21.15.151:0/3035451421 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:56:48.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.748+0000 7f7433e00700 1 -- 172.21.15.151:0/3035451421 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5650b81e4830 msgr2=0x5650b81e4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:56:48.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.748+0000 7f7433e00700 1 --2- 172.21.15.151:0/3035451421 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5650b81e4830 0x5650b81e4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:48.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.748+0000 7f7433e00700 1 -- 172.21.15.151:0/3035451421 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650b81e9e60 msgr2=0x5650b81d8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:56:48.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.748+0000 7f7433e00700 1 --2- 172.21.15.151:0/3035451421 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650b81e9e60 0x5650b81d8e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:48.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.748+0000 7f7433e00700 1 -- 172.21.15.151:0/3035451421 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650b81fb820 con 0x5650b81ea780 2024-05-21T21:56:48.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.748+0000 7f7433e00700 1 --2- 172.21.15.151:0/3035451421 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650b81ea780 0x5650b81e7970 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f742400a6d0 tx=0x7f7424012560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e0343545866145d server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:56:48.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.748+0000 7f742be00700 1 -- 172.21.15.151:0/3035451421 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7424014110 con 0x5650b81ea780 2024-05-21T21:56:48.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.748+0000 7f742be00700 1 -- 172.21.15.151:0/3035451421 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74240142b0 con 0x5650b81ea780 2024-05-21T21:56:48.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.748+0000 7f742be00700 1 -- 172.21.15.151:0/3035451421 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7424004a20 con 0x5650b81ea780 2024-05-21T21:56:48.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.749+0000 7f744cb04b80 1 -- 172.21.15.151:0/3035451421 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650b81ea780 msgr2=0x5650b81e7970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:48.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.749+0000 7f744cb04b80 1 --2- 172.21.15.151:0/3035451421 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650b81ea780 0x5650b81e7970 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f742400a6d0 tx=0x7f7424012560 comp rx=0 tx=0).stop 2024-05-21T21:56:48.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.749+0000 7f744cb04b80 1 -- 172.21.15.151:0/3035451421 shutdown_connections 2024-05-21T21:56:48.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.749+0000 7f744cb04b80 1 --2- 172.21.15.151:0/3035451421 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650b81ea780 0x5650b81e7970 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:48.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.749+0000 7f744cb04b80 1 --2- 172.21.15.151:0/3035451421 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5650b81e4830 0x5650b81e4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:48.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.749+0000 7f744cb04b80 1 --2- 172.21.15.151:0/3035451421 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650b81e9e60 0x5650b81d8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:48.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.749+0000 7f744cb04b80 1 -- 172.21.15.151:0/3035451421 >> 172.21.15.151:0/3035451421 conn(0x5650b8112b90 msgr2=0x5650b81e06b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:56:48.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.749+0000 7f744cb04b80 1 -- 172.21.15.151:0/3035451421 shutdown_connections 2024-05-21T21:56:48.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.750+0000 7f744cb04b80 1 -- 172.21.15.151:0/3035451421 wait complete. 2024-05-21T21:56:48.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.750+0000 7f744cb04b80 1 -- start start 2024-05-21T21:56:48.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.750+0000 7f744cb04b80 1 Processor -- start 2024-05-21T21:56:48.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.751+0000 7f744cb04b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650b81e4830 0x5650b81e3620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:48.754 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.751+0000 7f744cb04b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650b81e9e60 0x5650b81e3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:48.754 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.751+0000 7f744cb04b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5650b81ea780 0x5650b81e40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:48.754 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.751+0000 7f744cb04b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5650b81ff100 con 0x5650b81e9e60 2024-05-21T21:56:48.754 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.751+0000 7f744cb04b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5650b81fea30 con 0x5650b81e4830 2024-05-21T21:56:48.754 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.751+0000 7f744cb04b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5650b81ff7d0 con 0x5650b81ea780 2024-05-21T21:56:48.754 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.751+0000 7f744cb04b80 10 client.0 _ll_register_callbacks cb 0x5650b80dcc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:56:48.754 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.751+0000 7f7433e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650b81e4830 0x5650b81e3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:48.754 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.751+0000 7f743b400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650b81e9e60 0x5650b81e3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:48.754 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.751+0000 7f7433e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5650b81ea780 0x5650b81e40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:48.754 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.751+0000 7f743b400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650b81e9e60 0x5650b81e3b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49184/0 (socket says 172.21.15.151:49184) 2024-05-21T21:56:48.754 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.751+0000 7f743b400700 1 -- 172.21.15.151:0/291290622 learned_addr learned my addr 172.21.15.151:0/291290622 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:56:48.754 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.751+0000 7f743b400700 1 -- 172.21.15.151:0/291290622 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5650b81ea780 msgr2=0x5650b81e40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:56:48.754 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.751+0000 7f743b400700 1 --2- 172.21.15.151:0/291290622 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5650b81ea780 0x5650b81e40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:48.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.751+0000 7f743b400700 1 -- 172.21.15.151:0/291290622 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650b81e4830 msgr2=0x5650b81e3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:56:48.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.751+0000 7f743b400700 1 --2- 172.21.15.151:0/291290622 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650b81e4830 0x5650b81e3620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:48.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.751+0000 7f743b400700 1 -- 172.21.15.151:0/291290622 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650b8206db0 con 0x5650b81e9e60 2024-05-21T21:56:48.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.752+0000 7f743b400700 1 --2- 172.21.15.151:0/291290622 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650b81e9e60 0x5650b81e3b60 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f742c000c00 tx=0x7f742c01bb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab640b3abcaceb67 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:56:48.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.752+0000 7f7429600700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-21T21:56:48.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.752+0000 7f7429600700 1 -- 172.21.15.151:0/291290622 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f742c032060 con 0x5650b81e9e60 2024-05-21T21:56:48.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.752+0000 7f7429600700 1 -- 172.21.15.151:0/291290622 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f742c005380 con 0x5650b81e9e60 2024-05-21T21:56:48.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.752+0000 7f744cb04b80 1 -- 172.21.15.151:0/291290622 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5650b813e520 con 0x5650b81e9e60 2024-05-21T21:56:48.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.752+0000 7f7429600700 1 -- 172.21.15.151:0/291290622 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f742c005580 con 0x5650b81e9e60 2024-05-21T21:56:48.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.752+0000 7f744cb04b80 10 client.15384 fetch_fsmap learned FSMap version 29 2024-05-21T21:56:48.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.752+0000 7f744cb04b80 1 -- 172.21.15.151:0/291290622 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5650b82699a0 con 0x5650b81e9e60 2024-05-21T21:56:48.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.753+0000 7f7429600700 1 -- 172.21.15.151:0/291290622 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5650b813e520 con 0x5650b81e9e60 2024-05-21T21:56:48.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.753+0000 7f7429600700 1 -- 172.21.15.151:0/291290622 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f742c003a60 con 0x5650b81e9e60 2024-05-21T21:56:48.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.753+0000 7f744cb04b80 10 client.15384 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T21:56:48.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.753+0000 7f744cb04b80 10 client.15384 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-21T21:56:48.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.753+0000 7f744cb04b80 20 client.15384 populate_metadata read hostname 'smithi151' 2024-05-21T21:56:48.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.753+0000 7f744cb04b80 1 --2- 172.21.15.151:0/291290622 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5650b826e5b0 0x5650b826d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:48.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.753+0000 7f744cb04b80 4 client.15384 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-21T21:56:48.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.753+0000 7f744cb04b80 1 -- 172.21.15.151:0/291290622 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "status"}) -- 0x5650b7fb5200 con 0x5650b826e5b0 2024-05-21T21:56:48.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.753+0000 7f7433e00700 1 --2- 172.21.15.151:0/291290622 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5650b826e5b0 0x5650b826d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:48.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.754+0000 7f7429600700 10 client.15384 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T21:56:48.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.754+0000 7f7433e00700 1 --2- 172.21.15.151:0/291290622 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5650b826e5b0 0x5650b826d2c0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4970c2a9ee61ab server_cookie=b4905d9d91a56e6e in_seq=0 out_seq=0 2024-05-21T21:56:48.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.754+0000 7f7429600700 1 -- 172.21.15.151:0/291290622 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5650b7fb5200 con 0x5650b826e5b0 2024-05-21T21:56:48.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.754+0000 7f7429600700 10 client.15384 handle_command_reply: tid=0 2024-05-21T21:56:48.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.754+0000 7f744cb04b80 1 client.15384 shutdown 2024-05-21T21:56:48.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.754+0000 7f744cb04b80 1 -- 172.21.15.151:0/291290622 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650b81e9e60 msgr2=0x5650b81e3b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:48.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.754+0000 7f744cb04b80 1 --2- 172.21.15.151:0/291290622 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650b81e9e60 0x5650b81e3b60 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f742c000c00 tx=0x7f742c01bb10 comp rx=0 tx=0).stop 2024-05-21T21:56:48.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.754+0000 7f744cb04b80 1 -- 172.21.15.151:0/291290622 shutdown_connections 2024-05-21T21:56:48.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.754+0000 7f744cb04b80 1 --2- 172.21.15.151:0/291290622 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5650b826e5b0 0x5650b826d2c0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:48.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.754+0000 7f744cb04b80 1 --2- 172.21.15.151:0/291290622 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650b81e4830 0x5650b81e3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:48.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.754+0000 7f744cb04b80 1 --2- 172.21.15.151:0/291290622 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5650b81ea780 0x5650b81e40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:48.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.754+0000 7f744cb04b80 1 --2- 172.21.15.151:0/291290622 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650b81e9e60 0x5650b81e3b60 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:48.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.754+0000 7f744cb04b80 1 -- 172.21.15.151:0/291290622 >> 172.21.15.151:0/291290622 conn(0x5650b8112b90 msgr2=0x5650b81d8e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:56:48.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.755+0000 7f7429600700 0 client.15384 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T21:56:48.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.755+0000 7f744cb04b80 1 -- 172.21.15.151:0/291290622 shutdown_connections 2024-05-21T21:56:48.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.755+0000 7f744cb04b80 1 -- 172.21.15.151:0/291290622 wait complete. 2024-05-21T21:56:48.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.755+0000 7f744cb04b80 20 client.15384 trim_cache size 0 max 16384 2024-05-21T21:56:48.761 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-05-21T21:56:48.761 INFO:teuthology.orchestra.run.smithi151.stdout: "cluster_fsid": "c8c7a2e6-17ba-11ef-bc9a-c7b262605968", 2024-05-21T21:56:48.761 INFO:teuthology.orchestra.run.smithi151.stdout: "whoami": 0, 2024-05-21T21:56:48.761 INFO:teuthology.orchestra.run.smithi151.stdout: "id": 24530, 2024-05-21T21:56:48.761 INFO:teuthology.orchestra.run.smithi151.stdout: "want_state": "up:active", 2024-05-21T21:56:48.761 INFO:teuthology.orchestra.run.smithi151.stdout: "state": "up:active", 2024-05-21T21:56:48.761 INFO:teuthology.orchestra.run.smithi151.stdout: "fs_name": "cephfs", 2024-05-21T21:56:48.761 INFO:teuthology.orchestra.run.smithi151.stdout: "rank_uptime": 328.81936848100003, 2024-05-21T21:56:48.761 INFO:teuthology.orchestra.run.smithi151.stdout: "mdsmap_epoch": 29, 2024-05-21T21:56:48.761 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_epoch": 83, 2024-05-21T21:56:48.761 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_epoch_barrier": 82, 2024-05-21T21:56:48.761 INFO:teuthology.orchestra.run.smithi151.stdout: "uptime": 349.76352869800002 2024-05-21T21:56:48.761 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-05-21T21:56:48.762 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.756+0000 7f7442200700 1 -- 172.21.15.151:0/3809143450 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f74200b2b90 msgr2=0x7f74200b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:48.762 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.756+0000 7f7442200700 1 --2- 172.21.15.151:0/3809143450 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f74200b2b90 0x7f74200b5050 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f74340e0fa0 tx=0x7f74340099e0 comp rx=0 tx=0).stop 2024-05-21T21:56:48.762 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.756+0000 7f7442200700 1 -- 172.21.15.151:0/3809143450 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f743c10eeb0 msgr2=0x7f743c1a4990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:48.762 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.756+0000 7f7442200700 1 --2- 172.21.15.151:0/3809143450 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f743c10eeb0 0x7f743c1a4990 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f743400b820 tx=0x7f74340079a0 comp rx=0 tx=0).stop 2024-05-21T21:56:48.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.756+0000 7f7442200700 1 -- 172.21.15.151:0/3809143450 shutdown_connections 2024-05-21T21:56:48.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.756+0000 7f7442200700 1 --2- 172.21.15.151:0/3809143450 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f74200b2b90 0x7f74200b5050 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:48.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.756+0000 7f7442200700 1 --2- 172.21.15.151:0/3809143450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f743c109dd0 0x7f743c1a00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:48.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.756+0000 7f7442200700 1 --2- 172.21.15.151:0/3809143450 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f743c10eeb0 0x7f743c1a4990 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:48.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.756+0000 7f7442200700 1 --2- 172.21.15.151:0/3809143450 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f743c10a780 0x7f743c1a0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:48.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.756+0000 7f7442200700 1 -- 172.21.15.151:0/3809143450 >> 172.21.15.151:0/3809143450 conn(0x7f743c076d10 msgr2=0x7f743c077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:56:48.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.756+0000 7f7442200700 1 -- 172.21.15.151:0/3809143450 shutdown_connections 2024-05-21T21:56:48.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:48.756+0000 7f7442200700 1 -- 172.21.15.151:0/3809143450 wait complete. 2024-05-21T21:56:49.921 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:56:49 smithi151 ceph-mon[31623]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T21:56:49.985 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:56:49 smithi184 ceph-mon[36942]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T21:56:50.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:56:49 smithi164 ceph-mon[36269]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T21:56:50.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.293+0000 7fee2d400700 1 -- 172.21.15.151:0/3239540511 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fee28114710 msgr2=0x7fee28116b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:50.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.293+0000 7fee2d400700 1 --2- 172.21.15.151:0/3239540511 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fee28114710 0x7fee28116b00 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fee2000b0a0 tx=0x7fee2000b3b0 comp rx=0 tx=0).stop 2024-05-21T21:56:50.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.293+0000 7fee2d400700 1 -- 172.21.15.151:0/3239540511 shutdown_connections 2024-05-21T21:56:50.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.293+0000 7fee2d400700 1 --2- 172.21.15.151:0/3239540511 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fee28114710 0x7fee28116b00 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:50.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.293+0000 7fee2d400700 1 --2- 172.21.15.151:0/3239540511 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee28075770 0x7fee28075b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:50.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.293+0000 7fee2d400700 1 --2- 172.21.15.151:0/3239540511 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fee28076090 0x7fee281141d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:50.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.293+0000 7fee2d400700 1 -- 172.21.15.151:0/3239540511 >> 172.21.15.151:0/3239540511 conn(0x7fee2807a6a0 msgr2=0x7fee2807aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:56:50.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.294+0000 7fee2d400700 1 -- 172.21.15.151:0/3239540511 shutdown_connections 2024-05-21T21:56:50.296 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.294+0000 7fee2d400700 1 -- 172.21.15.151:0/3239540511 wait complete. 2024-05-21T21:56:50.296 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.295+0000 7fee2d400700 1 Processor -- start 2024-05-21T21:56:50.297 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.296+0000 7fee2d400700 1 -- start start 2024-05-21T21:56:50.297 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.296+0000 7fee2d400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fee28075770 0x7fee281a0270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:50.297 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.296+0000 7fee2d400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fee28076090 0x7fee281a07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:50.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.296+0000 7fee2d400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee28114710 0x7fee281a4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:50.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.296+0000 7fee2d400700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fee28119ce0 con 0x7fee28075770 2024-05-21T21:56:50.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.296+0000 7fee2d400700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fee28119b60 con 0x7fee28076090 2024-05-21T21:56:50.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.296+0000 7fee2d400700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fee28119e60 con 0x7fee28114710 2024-05-21T21:56:50.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.297+0000 7fee26800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fee28075770 0x7fee281a0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:50.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.297+0000 7fee26800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fee28075770 0x7fee281a0270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49212/0 (socket says 172.21.15.151:49212) 2024-05-21T21:56:50.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.297+0000 7fee26800700 1 -- 172.21.15.151:0/2767634688 learned_addr learned my addr 172.21.15.151:0/2767634688 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:56:50.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.297+0000 7fee25e00700 1 --2- 172.21.15.151:0/2767634688 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fee28076090 0x7fee281a07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:50.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.297+0000 7fee27200700 1 --2- 172.21.15.151:0/2767634688 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee28114710 0x7fee281a4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:50.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.297+0000 7fee26800700 1 -- 172.21.15.151:0/2767634688 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee28114710 msgr2=0x7fee281a4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:50.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.297+0000 7fee26800700 1 --2- 172.21.15.151:0/2767634688 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee28114710 0x7fee281a4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:50.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.297+0000 7fee26800700 1 -- 172.21.15.151:0/2767634688 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fee28076090 msgr2=0x7fee281a07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:50.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.297+0000 7fee26800700 1 --2- 172.21.15.151:0/2767634688 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fee28076090 0x7fee281a07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:50.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.297+0000 7fee26800700 1 -- 172.21.15.151:0/2767634688 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee281a5280 con 0x7fee28075770 2024-05-21T21:56:50.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.297+0000 7fee25e00700 1 --2- 172.21.15.151:0/2767634688 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fee28076090 0x7fee281a07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:56:50.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.298+0000 7fee26800700 1 --2- 172.21.15.151:0/2767634688 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fee28075770 0x7fee281a0270 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fee1800e9a0 tx=0x7fee1800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:56:50.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.298+0000 7fee1f400700 1 -- 172.21.15.151:0/2767634688 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee1800ca10 con 0x7fee28075770 2024-05-21T21:56:50.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.298+0000 7fee1f400700 1 -- 172.21.15.151:0/2767634688 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee1800cbb0 con 0x7fee28075770 2024-05-21T21:56:50.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.298+0000 7fee2d400700 1 -- 172.21.15.151:0/2767634688 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee281a5570 con 0x7fee28075770 2024-05-21T21:56:50.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.298+0000 7fee1f400700 1 -- 172.21.15.151:0/2767634688 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee18018680 con 0x7fee28075770 2024-05-21T21:56:50.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.298+0000 7fee2d400700 1 -- 172.21.15.151:0/2767634688 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee281acdf0 con 0x7fee28075770 2024-05-21T21:56:50.303 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.302+0000 7fee1f400700 1 -- 172.21.15.151:0/2767634688 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 26) ==== 149886+0+0 (secure 0 0 0) 0x7fee18010430 con 0x7fee28075770 2024-05-21T21:56:50.304 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.303+0000 7fee37ddbb80 1 Processor -- start 2024-05-21T21:56:50.304 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.303+0000 7fee37ddbb80 1 -- start start 2024-05-21T21:56:50.304 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.303+0000 7fee1f400700 1 --2- 172.21.15.151:0/2767634688 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fee0c0b2b10 0x7fee0c0b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:50.305 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.303+0000 7fee37ddbb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55924bf98610 0x55924bf989f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:50.305 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.303+0000 7fee37ddbb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55924bf98f30 0x55924bf92b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:50.305 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.303+0000 7fee37ddbb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55924bf93140 0x55924bf935c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:50.305 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.303+0000 7fee37ddbb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55924bef0e40 con 0x55924bf98610 2024-05-21T21:56:50.305 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.303+0000 7fee37ddbb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55924bee09e0 con 0x55924bf93140 2024-05-21T21:56:50.305 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.303+0000 7fee37ddbb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55924bee4a20 con 0x55924bf98f30 2024-05-21T21:56:50.306 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.304+0000 7fee27200700 1 --2- 172.21.15.151:0/2767634688 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fee0c0b2b10 0x7fee0c0b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:50.306 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.304+0000 7fee1f400700 1 -- 172.21.15.151:0/2767634688 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fee180e2a10 con 0x7fee28075770 2024-05-21T21:56:50.306 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.304+0000 7fee26800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55924bf98610 0x55924bf989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:50.306 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.304+0000 7fee25e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55924bf98f30 0x55924bf92b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:50.306 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.304+0000 7fee25e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55924bf98f30 0x55924bf92b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:57968/0 (socket says 172.21.15.151:57968) 2024-05-21T21:56:50.306 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.304+0000 7fee26800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55924bf98610 0x55924bf989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49218/0 (socket says 172.21.15.151:49218) 2024-05-21T21:56:50.306 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.304+0000 7fee25e00700 1 -- 172.21.15.151:0/1134795298 learned_addr learned my addr 172.21.15.151:0/1134795298 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:56:50.306 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.304+0000 7fee27200700 1 --2- 172.21.15.151:0/1134795298 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55924bf93140 0x55924bf935c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:50.307 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.304+0000 7fee25e00700 1 -- 172.21.15.151:0/1134795298 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55924bf93140 msgr2=0x55924bf935c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:56:50.307 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.305+0000 7fee25e00700 1 --2- 172.21.15.151:0/1134795298 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55924bf93140 0x55924bf935c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:50.307 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.305+0000 7fee25e00700 1 -- 172.21.15.151:0/1134795298 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55924bf98610 msgr2=0x55924bf989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:56:50.307 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.305+0000 7fee25e00700 1 --2- 172.21.15.151:0/1134795298 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55924bf98610 0x55924bf989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:50.307 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.305+0000 7fee25e00700 1 -- 172.21.15.151:0/1134795298 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55924bea9420 con 0x55924bf98f30 2024-05-21T21:56:50.307 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.305+0000 7fee26800700 1 --2- 172.21.15.151:0/1134795298 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55924bf98610 0x55924bf989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:56:50.307 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.305+0000 7fee25e00700 1 --2- 172.21.15.151:0/1134795298 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55924bf98f30 0x55924bf92b20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fee140114f0 tx=0x7fee14011800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40322ebf3c6e88a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:56:50.307 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.305+0000 7fee27200700 1 --2- 172.21.15.151:0/2767634688 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fee0c0b2b10 0x7fee0c0b4fd0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fee20009fd0 tx=0x7fee2000bbe0 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:56:50.308 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.305+0000 7fee13e00700 1 -- 172.21.15.151:0/1134795298 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee14004230 con 0x55924bf98f30 2024-05-21T21:56:50.308 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.305+0000 7fee13e00700 1 -- 172.21.15.151:0/1134795298 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee140043d0 con 0x55924bf98f30 2024-05-21T21:56:50.308 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.305+0000 7fee13e00700 1 -- 172.21.15.151:0/1134795298 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee1400e970 con 0x55924bf98f30 2024-05-21T21:56:50.308 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.306+0000 7fee37ddbb80 1 -- 172.21.15.151:0/1134795298 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55924bf98f30 msgr2=0x55924bf92b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:50.308 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.306+0000 7fee37ddbb80 1 --2- 172.21.15.151:0/1134795298 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55924bf98f30 0x55924bf92b20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fee140114f0 tx=0x7fee14011800 comp rx=0 tx=0).stop 2024-05-21T21:56:50.308 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.306+0000 7fee37ddbb80 1 -- 172.21.15.151:0/1134795298 shutdown_connections 2024-05-21T21:56:50.309 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.306+0000 7fee37ddbb80 1 --2- 172.21.15.151:0/1134795298 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55924bf93140 0x55924bf935c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:50.309 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.306+0000 7fee37ddbb80 1 --2- 172.21.15.151:0/1134795298 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55924bf98610 0x55924bf989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:50.309 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.306+0000 7fee37ddbb80 1 --2- 172.21.15.151:0/1134795298 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55924bf98f30 0x55924bf92b20 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:50.309 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.306+0000 7fee37ddbb80 1 -- 172.21.15.151:0/1134795298 >> 172.21.15.151:0/1134795298 conn(0x55924bec0be0 msgr2=0x55924bf8d690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:56:50.309 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.306+0000 7fee37ddbb80 1 -- 172.21.15.151:0/1134795298 shutdown_connections 2024-05-21T21:56:50.309 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.306+0000 7fee37ddbb80 1 -- 172.21.15.151:0/1134795298 wait complete. 2024-05-21T21:56:50.309 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.307+0000 7fee37ddbb80 1 -- start start 2024-05-21T21:56:50.309 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.307+0000 7fee37ddbb80 1 Processor -- start 2024-05-21T21:56:50.310 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.308+0000 7fee37ddbb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55924bf93140 0x55924bf9eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:50.310 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.308+0000 7fee37ddbb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55924bf98610 0x55924bf9bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:50.311 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.308+0000 7fee37ddbb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55924bf98f30 0x55924bf9c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:50.311 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.308+0000 7fee37ddbb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55924bde6fa0 con 0x55924bf98610 2024-05-21T21:56:50.311 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.308+0000 7fee37ddbb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55924be81760 con 0x55924bf98f30 2024-05-21T21:56:50.311 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.308+0000 7fee37ddbb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55924bf9f9e0 con 0x55924bf93140 2024-05-21T21:56:50.311 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.308+0000 7fee25e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55924bf98610 0x55924bf9bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:50.311 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.308+0000 7fee26800700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55924bf93140 0x55924bf9eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:50.311 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.308+0000 7fee26800700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55924bf93140 0x55924bf9eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:57970/0 (socket says 172.21.15.151:57970) 2024-05-21T21:56:50.311 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.308+0000 7fee25e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55924bf98610 0x55924bf9bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49234/0 (socket says 172.21.15.151:49234) 2024-05-21T21:56:50.311 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.308+0000 7fee26800700 1 -- 172.21.15.151:0/3085115632 learned_addr learned my addr 172.21.15.151:0/3085115632 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:56:50.311 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.308+0000 7fee27200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55924bf98f30 0x55924bf9c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:50.311 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.308+0000 7fee37ddbb80 10 client.0 _ll_register_callbacks cb 0x55924be8ac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:56:50.311 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.309+0000 7fee27200700 1 -- 172.21.15.151:0/3085115632 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55924bf93140 msgr2=0x55924bf9eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:56:50.311 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.309+0000 7fee27200700 1 --2- 172.21.15.151:0/3085115632 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55924bf93140 0x55924bf9eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:50.312 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.309+0000 7fee27200700 1 -- 172.21.15.151:0/3085115632 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55924bf98610 msgr2=0x55924bf9bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:56:50.312 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.309+0000 7fee27200700 1 --2- 172.21.15.151:0/3085115632 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55924bf98610 0x55924bf9bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:50.312 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.309+0000 7fee27200700 1 -- 172.21.15.151:0/3085115632 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55924be308d0 con 0x55924bf98f30 2024-05-21T21:56:50.312 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.309+0000 7fee26800700 1 --2- 172.21.15.151:0/3085115632 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55924bf93140 0x55924bf9eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:56:50.312 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.309+0000 7fee27200700 1 --2- 172.21.15.151:0/3085115632 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55924bf98f30 0x55924bf9c4b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fee2000d010 tx=0x7fee2000b3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf6975374b7a1a13 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:56:50.312 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.309+0000 7fee10c00700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-21T21:56:50.312 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.310+0000 7fee10c00700 1 -- 172.21.15.151:0/3085115632 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee20007940 con 0x55924bf98f30 2024-05-21T21:56:50.312 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.310+0000 7fee10c00700 1 -- 172.21.15.151:0/3085115632 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fee20007ae0 con 0x55924bf98f30 2024-05-21T21:56:50.312 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.310+0000 7fee37ddbb80 1 -- 172.21.15.151:0/3085115632 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee140043d0 con 0x55924bf98f30 2024-05-21T21:56:50.314 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.310+0000 7fee10c00700 1 -- 172.21.15.151:0/3085115632 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fee20038050 con 0x55924bf98f30 2024-05-21T21:56:50.314 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.310+0000 7fee37ddbb80 10 client.25508 fetch_fsmap learned FSMap version 29 2024-05-21T21:56:50.314 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.310+0000 7fee37ddbb80 1 -- 172.21.15.151:0/3085115632 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55924beb1060 con 0x55924bf98f30 2024-05-21T21:56:50.314 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.310+0000 7fee10c00700 1 -- 172.21.15.151:0/3085115632 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee140043d0 con 0x55924bf98f30 2024-05-21T21:56:50.314 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.310+0000 7fee10c00700 1 -- 172.21.15.151:0/3085115632 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fee20011070 con 0x55924bf98f30 2024-05-21T21:56:50.315 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.311+0000 7fee37ddbb80 10 client.25508 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T21:56:50.315 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.311+0000 7fee37ddbb80 10 client.25508 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-21T21:56:50.315 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.311+0000 7fee37ddbb80 20 client.25508 populate_metadata read hostname 'smithi151' 2024-05-21T21:56:50.315 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.311+0000 7fee37ddbb80 1 --2- 172.21.15.151:0/3085115632 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x55924bfac490 0x55924bfae8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:50.315 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.311+0000 7fee37ddbb80 4 client.25508 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T21:56:50.315 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.311+0000 7fee37ddbb80 1 -- 172.21.15.151:0/3085115632 --> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55924bef2d90 con 0x55924bfac490 2024-05-21T21:56:50.315 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.311+0000 7fee26800700 1 --2- 172.21.15.151:0/3085115632 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x55924bfac490 0x55924bfae8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:50.315 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.312+0000 7fee26800700 1 --2- 172.21.15.151:0/3085115632 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x55924bfac490 0x55924bfae8c0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=64e4f0fad17f87a5 server_cookie=4aef791b34332644 in_seq=0 out_seq=0 2024-05-21T21:56:50.315 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.312+0000 7fee10c00700 10 client.25508 ms_handle_connect on v2:172.21.15.184:6836/2988194888 2024-05-21T21:56:50.315 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.312+0000 7fee10c00700 1 -- 172.21.15.151:0/3085115632 <== mds.3 v2:172.21.15.184:6836/2988194888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55924bef2d90 con 0x55924bfac490 2024-05-21T21:56:50.315 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.312+0000 7fee10c00700 10 client.25508 handle_command_reply: tid=0 2024-05-21T21:56:50.315 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.312+0000 7fee37ddbb80 1 client.25508 shutdown 2024-05-21T21:56:50.315 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.313+0000 7fee37ddbb80 1 -- 172.21.15.151:0/3085115632 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55924bf98f30 msgr2=0x55924bf9c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:50.315 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.313+0000 7fee37ddbb80 1 --2- 172.21.15.151:0/3085115632 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55924bf98f30 0x55924bf9c4b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fee2000d010 tx=0x7fee2000b3b0 comp rx=0 tx=0).stop 2024-05-21T21:56:50.315 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.313+0000 7fee37ddbb80 1 -- 172.21.15.151:0/3085115632 shutdown_connections 2024-05-21T21:56:50.316 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.313+0000 7fee37ddbb80 1 --2- 172.21.15.151:0/3085115632 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x55924bfac490 0x55924bfae8c0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:50.316 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.313+0000 7fee37ddbb80 1 --2- 172.21.15.151:0/3085115632 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55924bf98f30 0x55924bf9c4b0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:50.316 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.313+0000 7fee37ddbb80 1 --2- 172.21.15.151:0/3085115632 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55924bf93140 0x55924bf9eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:50.316 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.313+0000 7fee37ddbb80 1 --2- 172.21.15.151:0/3085115632 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55924bf98610 0x55924bf9bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:50.316 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.313+0000 7fee10c00700 0 client.25508 ms_handle_reset on v2:172.21.15.184:6836/2988194888 2024-05-21T21:56:50.316 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.313+0000 7fee37ddbb80 1 -- 172.21.15.151:0/3085115632 >> 172.21.15.151:0/3085115632 conn(0x55924bec0be0 msgr2=0x55924bf8cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:56:50.316 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.313+0000 7fee37ddbb80 1 -- 172.21.15.151:0/3085115632 shutdown_connections 2024-05-21T21:56:50.316 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.313+0000 7fee37ddbb80 1 -- 172.21.15.151:0/3085115632 wait complete. 2024-05-21T21:56:50.316 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.313+0000 7fee37ddbb80 20 client.25508 trim_cache size 0 max 16384 2024-05-21T21:56:50.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.330+0000 7fee37ddbb80 1 Processor -- start 2024-05-21T21:56:50.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.330+0000 7fee37ddbb80 1 -- start start 2024-05-21T21:56:50.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.330+0000 7fee37ddbb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55924bf93140 0x55924bf92b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:50.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.330+0000 7fee37ddbb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55924bf98610 0x55924bf9af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:50.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.330+0000 7fee37ddbb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55924bf98f30 0x55924bf8d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:50.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.330+0000 7fee37ddbb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fee281b44a0 con 0x55924bf93140 2024-05-21T21:56:50.332 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.330+0000 7fee37ddbb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fee281b5fe0 con 0x55924bf98610 2024-05-21T21:56:50.332 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.330+0000 7fee37ddbb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fee281b4b70 con 0x55924bf98f30 2024-05-21T21:56:50.333 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.331+0000 7fee26800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55924bf93140 0x55924bf92b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:50.333 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.331+0000 7fee25e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55924bf98610 0x55924bf9af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:50.333 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.331+0000 7fee26800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55924bf93140 0x55924bf92b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49248/0 (socket says 172.21.15.151:49248) 2024-05-21T21:56:50.333 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.331+0000 7fee26800700 1 -- 172.21.15.151:0/1749415385 learned_addr learned my addr 172.21.15.151:0/1749415385 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:56:50.333 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.331+0000 7fee27200700 1 --2- 172.21.15.151:0/1749415385 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55924bf98f30 0x55924bf8d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:50.333 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.331+0000 7fee27200700 1 -- 172.21.15.151:0/1749415385 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55924bf98610 msgr2=0x55924bf9af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:56:50.333 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.331+0000 7fee27200700 1 --2- 172.21.15.151:0/1749415385 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55924bf98610 0x55924bf9af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:50.333 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.331+0000 7fee27200700 1 -- 172.21.15.151:0/1749415385 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55924bf93140 msgr2=0x55924bf92b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:56:50.333 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.331+0000 7fee27200700 1 --2- 172.21.15.151:0/1749415385 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55924bf93140 0x55924bf92b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:50.333 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.331+0000 7fee27200700 1 -- 172.21.15.151:0/1749415385 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55924bf9a4e0 con 0x55924bf98f30 2024-05-21T21:56:50.333 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.331+0000 7fee25e00700 1 --2- 172.21.15.151:0/1749415385 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55924bf98610 0x55924bf9af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:56:50.333 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.331+0000 7fee26800700 1 --2- 172.21.15.151:0/1749415385 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55924bf93140 0x55924bf92b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:56:50.333 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.331+0000 7fee27200700 1 --2- 172.21.15.151:0/1749415385 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55924bf98f30 0x55924bf8d4f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fee2000d010 tx=0x7fee2002cf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8bc0796be3c7f9d server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:56:50.333 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.332+0000 7fee13400700 1 -- 172.21.15.151:0/1749415385 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee20005c60 con 0x55924bf98f30 2024-05-21T21:56:50.334 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.332+0000 7fee13400700 1 -- 172.21.15.151:0/1749415385 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee20005e00 con 0x55924bf98f30 2024-05-21T21:56:50.334 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.332+0000 7fee13400700 1 -- 172.21.15.151:0/1749415385 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee20007cf0 con 0x55924bf98f30 2024-05-21T21:56:50.334 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.332+0000 7fee37ddbb80 1 -- 172.21.15.151:0/1749415385 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55924bf98f30 msgr2=0x55924bf8d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:50.334 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.332+0000 7fee37ddbb80 1 --2- 172.21.15.151:0/1749415385 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55924bf98f30 0x55924bf8d4f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fee2000d010 tx=0x7fee2002cf60 comp rx=0 tx=0).stop 2024-05-21T21:56:50.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.332+0000 7fee37ddbb80 1 -- 172.21.15.151:0/1749415385 shutdown_connections 2024-05-21T21:56:50.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.332+0000 7fee37ddbb80 1 --2- 172.21.15.151:0/1749415385 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55924bf98f30 0x55924bf8d4f0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:50.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.332+0000 7fee37ddbb80 1 --2- 172.21.15.151:0/1749415385 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55924bf93140 0x55924bf92b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:50.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.332+0000 7fee37ddbb80 1 --2- 172.21.15.151:0/1749415385 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55924bf98610 0x55924bf9af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:50.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.332+0000 7fee37ddbb80 1 -- 172.21.15.151:0/1749415385 >> 172.21.15.151:0/1749415385 conn(0x55924bec0be0 msgr2=0x55924bf9a860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:56:50.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.333+0000 7fee37ddbb80 1 -- 172.21.15.151:0/1749415385 shutdown_connections 2024-05-21T21:56:50.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.333+0000 7fee37ddbb80 1 -- 172.21.15.151:0/1749415385 wait complete. 2024-05-21T21:56:50.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.333+0000 7fee37ddbb80 1 -- start start 2024-05-21T21:56:50.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.333+0000 7fee37ddbb80 1 Processor -- start 2024-05-21T21:56:50.336 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.334+0000 7fee37ddbb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55924bf93140 0x55924bfa4c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:50.336 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.334+0000 7fee37ddbb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55924bf98610 0x55924bfa51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:50.336 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.334+0000 7fee37ddbb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55924bf98f30 0x55924bfb31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:50.336 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.334+0000 7fee37ddbb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fee281b5910 con 0x55924bf98f30 2024-05-21T21:56:50.336 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.334+0000 7fee37ddbb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fee281b5240 con 0x55924bf93140 2024-05-21T21:56:50.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.334+0000 7fee37ddbb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fee281b2440 con 0x55924bf98610 2024-05-21T21:56:50.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.334+0000 7fee37ddbb80 10 client.0 _ll_register_callbacks cb 0x55924be8ac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:56:50.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.334+0000 7fee27200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55924bf98f30 0x55924bfb31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:50.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.334+0000 7fee26800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55924bf93140 0x55924bfa4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:50.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.335+0000 7fee26800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55924bf93140 0x55924bfa4c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:56300/0 (socket says 172.21.15.151:56300) 2024-05-21T21:56:50.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.334+0000 7fee25e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55924bf98610 0x55924bfa51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:50.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.335+0000 7fee26800700 1 -- 172.21.15.151:0/2367044888 learned_addr learned my addr 172.21.15.151:0/2367044888 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:56:50.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.335+0000 7fee26800700 1 -- 172.21.15.151:0/2367044888 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55924bf98610 msgr2=0x55924bfa51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:56:50.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.335+0000 7fee26800700 1 --2- 172.21.15.151:0/2367044888 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55924bf98610 0x55924bfa51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:50.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.335+0000 7fee26800700 1 -- 172.21.15.151:0/2367044888 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55924bf98f30 msgr2=0x55924bfb31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:56:50.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.335+0000 7fee26800700 1 --2- 172.21.15.151:0/2367044888 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55924bf98f30 0x55924bfb31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:50.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.335+0000 7fee26800700 1 -- 172.21.15.151:0/2367044888 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55924bfb3750 con 0x55924bf93140 2024-05-21T21:56:50.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.335+0000 7fee27200700 1 --2- 172.21.15.151:0/2367044888 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55924bf98f30 0x55924bfb31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:56:50.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.335+0000 7fee26800700 1 --2- 172.21.15.151:0/2367044888 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55924bf93140 0x55924bfa4c70 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fee1808ffd0 tx=0x7fee180a59f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2074c21921138ad1 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:56:50.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.335+0000 7fee10c00700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-21T21:56:50.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.336+0000 7fee10c00700 1 -- 172.21.15.151:0/2367044888 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee1808ae00 con 0x55924bf93140 2024-05-21T21:56:50.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.336+0000 7fee10c00700 1 -- 172.21.15.151:0/2367044888 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fee180a60a0 con 0x55924bf93140 2024-05-21T21:56:50.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.336+0000 7fee37ddbb80 1 -- 172.21.15.151:0/2367044888 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee20005e00 con 0x55924bf93140 2024-05-21T21:56:50.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.336+0000 7fee10c00700 1 -- 172.21.15.151:0/2367044888 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fee180a9ab0 con 0x55924bf93140 2024-05-21T21:56:50.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.336+0000 7fee37ddbb80 10 client.25514 fetch_fsmap learned FSMap version 29 2024-05-21T21:56:50.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.336+0000 7fee37ddbb80 1 -- 172.21.15.151:0/2367044888 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55924c0168e0 con 0x55924bf93140 2024-05-21T21:56:50.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.336+0000 7fee10c00700 1 -- 172.21.15.151:0/2367044888 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee20005e00 con 0x55924bf93140 2024-05-21T21:56:50.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.336+0000 7fee10c00700 1 -- 172.21.15.151:0/2367044888 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fee180b2510 con 0x55924bf93140 2024-05-21T21:56:50.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.336+0000 7fee37ddbb80 10 client.25514 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T21:56:50.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.337+0000 7fee37ddbb80 10 client.25514 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-21T21:56:50.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.337+0000 7fee37ddbb80 20 client.25514 populate_metadata read hostname 'smithi151' 2024-05-21T21:56:50.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.337+0000 7fee37ddbb80 1 --2- 172.21.15.151:0/2367044888 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x55924c016f40 0x55924c019360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:50.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.337+0000 7fee37ddbb80 4 client.25514 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T21:56:50.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.337+0000 7fee37ddbb80 1 -- 172.21.15.151:0/2367044888 --> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55924beec570 con 0x55924c016f40 2024-05-21T21:56:50.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.337+0000 7fee25e00700 1 --2- 172.21.15.151:0/2367044888 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x55924c016f40 0x55924c019360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:50.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.338+0000 7fee25e00700 1 --2- 172.21.15.151:0/2367044888 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x55924c016f40 0x55924c019360 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=47ae9f3c93f74d1d server_cookie=96b747f65ad056e5 in_seq=0 out_seq=0 2024-05-21T21:56:50.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.338+0000 7fee10c00700 10 client.25514 ms_handle_connect on v2:172.21.15.184:6836/2988194888 2024-05-21T21:56:50.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.338+0000 7fee10c00700 1 -- 172.21.15.151:0/2367044888 <== mds.3 v2:172.21.15.184:6836/2988194888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55924beec570 con 0x55924c016f40 2024-05-21T21:56:50.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.338+0000 7fee10c00700 10 client.25514 handle_command_reply: tid=0 2024-05-21T21:56:50.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.338+0000 7fee37ddbb80 1 client.25514 shutdown 2024-05-21T21:56:50.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.338+0000 7fee37ddbb80 1 -- 172.21.15.151:0/2367044888 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55924bf93140 msgr2=0x55924bfa4c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:50.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.338+0000 7fee37ddbb80 1 --2- 172.21.15.151:0/2367044888 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55924bf93140 0x55924bfa4c70 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fee1808ffd0 tx=0x7fee180a59f0 comp rx=0 tx=0).stop 2024-05-21T21:56:50.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.338+0000 7fee37ddbb80 1 -- 172.21.15.151:0/2367044888 shutdown_connections 2024-05-21T21:56:50.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.338+0000 7fee37ddbb80 1 --2- 172.21.15.151:0/2367044888 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x55924c016f40 0x55924c019360 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:50.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.338+0000 7fee37ddbb80 1 --2- 172.21.15.151:0/2367044888 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55924bf93140 0x55924bfa4c70 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:50.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.338+0000 7fee37ddbb80 1 --2- 172.21.15.151:0/2367044888 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55924bf98f30 0x55924bfb31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:50.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.338+0000 7fee37ddbb80 1 --2- 172.21.15.151:0/2367044888 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55924bf98610 0x55924bfa51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:50.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.338+0000 7fee10c00700 0 client.25514 ms_handle_reset on v2:172.21.15.184:6836/2988194888 2024-05-21T21:56:50.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.338+0000 7fee37ddbb80 1 -- 172.21.15.151:0/2367044888 >> 172.21.15.151:0/2367044888 conn(0x55924bec0be0 msgr2=0x55924bf9b0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:56:50.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.338+0000 7fee37ddbb80 1 -- 172.21.15.151:0/2367044888 shutdown_connections 2024-05-21T21:56:50.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.338+0000 7fee37ddbb80 1 -- 172.21.15.151:0/2367044888 wait complete. 2024-05-21T21:56:50.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.338+0000 7fee37ddbb80 20 client.25514 trim_cache size 0 max 16384 2024-05-21T21:56:50.342 INFO:teuthology.orchestra.run.smithi151.stdout:[] 2024-05-21T21:56:50.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.339+0000 7fee2d400700 1 -- 172.21.15.151:0/2767634688 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fee0c0b2b10 msgr2=0x7fee0c0b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:50.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.339+0000 7fee2d400700 1 --2- 172.21.15.151:0/2767634688 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fee0c0b2b10 0x7fee0c0b4fd0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fee20009fd0 tx=0x7fee2000bbe0 comp rx=0 tx=0).stop 2024-05-21T21:56:50.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.339+0000 7fee2d400700 1 -- 172.21.15.151:0/2767634688 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fee28075770 msgr2=0x7fee281a0270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:50.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.339+0000 7fee2d400700 1 --2- 172.21.15.151:0/2767634688 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fee28075770 0x7fee281a0270 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fee1800e9a0 tx=0x7fee1800ecb0 comp rx=0 tx=0).stop 2024-05-21T21:56:50.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.339+0000 7fee2d400700 1 -- 172.21.15.151:0/2767634688 shutdown_connections 2024-05-21T21:56:50.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.339+0000 7fee2d400700 1 --2- 172.21.15.151:0/2767634688 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fee0c0b2b10 0x7fee0c0b4fd0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:50.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.339+0000 7fee2d400700 1 --2- 172.21.15.151:0/2767634688 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee28114710 0x7fee281a4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:50.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.339+0000 7fee2d400700 1 --2- 172.21.15.151:0/2767634688 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fee28075770 0x7fee281a0270 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:50.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.339+0000 7fee2d400700 1 --2- 172.21.15.151:0/2767634688 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fee28076090 0x7fee281a07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:50.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.339+0000 7fee2d400700 1 -- 172.21.15.151:0/2767634688 >> 172.21.15.151:0/2767634688 conn(0x7fee2807a6a0 msgr2=0x7fee28113540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:56:50.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.339+0000 7fee2d400700 1 -- 172.21.15.151:0/2767634688 shutdown_connections 2024-05-21T21:56:50.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:50.339+0000 7fee2d400700 1 -- 172.21.15.151:0/2767634688 wait complete. 2024-05-21T21:56:50.675 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-05-21T21:56:51.415 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-21T21:56:51.868 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:56:51.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:56:51 smithi151 ceph-mon[31623]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T21:56:51.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:56:51 smithi184 ceph-mon[36942]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T21:56:52.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:56:51 smithi164 ceph-mon[36269]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T21:56:52.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:56:52 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:56:52.985 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:56:52 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:56:53.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:56:52 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:56:53.918 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:56:53 smithi151 ceph-mon[31623]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-21T21:56:53.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:56:53 smithi184 ceph-mon[36942]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-21T21:56:54.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:56:53 smithi164 ceph-mon[36269]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-05-21T21:56:54.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.089+0000 7f1341e00700 1 -- 172.21.15.151:0/1851746772 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f133c103460 msgr2=0x7f133c1038e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:54.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.089+0000 7f1341e00700 1 --2- 172.21.15.151:0/1851746772 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f133c103460 0x7f133c1038e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f13280098f0 tx=0x7f1328009c00 comp rx=0 tx=0).stop 2024-05-21T21:56:54.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.089+0000 7f1341e00700 1 -- 172.21.15.151:0/1851746772 shutdown_connections 2024-05-21T21:56:54.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.089+0000 7f1341e00700 1 --2- 172.21.15.151:0/1851746772 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f133c103e20 0x7f133c110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:54.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.089+0000 7f1341e00700 1 --2- 172.21.15.151:0/1851746772 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f133c1094f0 0x7f133c1098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:54.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.089+0000 7f1341e00700 1 --2- 172.21.15.151:0/1851746772 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f133c103460 0x7f133c1038e0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:54.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.089+0000 7f1341e00700 1 -- 172.21.15.151:0/1851746772 >> 172.21.15.151:0/1851746772 conn(0x7f133c0fef50 msgr2=0x7f133c101370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:56:54.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.089+0000 7f1341e00700 1 -- 172.21.15.151:0/1851746772 shutdown_connections 2024-05-21T21:56:54.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.089+0000 7f1341e00700 1 -- 172.21.15.151:0/1851746772 wait complete. 2024-05-21T21:56:54.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.091+0000 7f1341e00700 1 Processor -- start 2024-05-21T21:56:54.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.091+0000 7f1341e00700 1 -- start start 2024-05-21T21:56:54.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.092+0000 7f1341e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f133c103460 0x7f133c19be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:54.093 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.092+0000 7f1341e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f133c103e20 0x7f133c19c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:54.093 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.092+0000 7f1341e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f133c1094f0 0x7f133c1a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:54.093 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.092+0000 7f1341e00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f133c113650 con 0x7f133c1094f0 2024-05-21T21:56:54.093 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.092+0000 7f1341e00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f133c1134d0 con 0x7f133c103e20 2024-05-21T21:56:54.093 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.092+0000 7f1341e00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f133c1137d0 con 0x7f133c103460 2024-05-21T21:56:54.094 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.092+0000 7f133be00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f133c103e20 0x7f133c19c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:54.094 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.092+0000 7f133be00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f133c103e20 0x7f133c19c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:56320/0 (socket says 172.21.15.151:56320) 2024-05-21T21:56:54.094 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.092+0000 7f133be00700 1 -- 172.21.15.151:0/3703592213 learned_addr learned my addr 172.21.15.151:0/3703592213 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:56:54.094 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.092+0000 7f1340a00700 1 --2- 172.21.15.151:0/3703592213 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f133c103460 0x7f133c19be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:54.094 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.092+0000 7f1341400700 1 --2- 172.21.15.151:0/3703592213 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f133c1094f0 0x7f133c1a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:54.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.092+0000 7f1340a00700 1 -- 172.21.15.151:0/3703592213 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f133c103e20 msgr2=0x7f133c19c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:54.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.092+0000 7f1340a00700 1 --2- 172.21.15.151:0/3703592213 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f133c103e20 0x7f133c19c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:54.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.093+0000 7f1340a00700 1 -- 172.21.15.151:0/3703592213 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f133c1094f0 msgr2=0x7f133c1a06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:54.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.093+0000 7f1340a00700 1 --2- 172.21.15.151:0/3703592213 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f133c1094f0 0x7f133c1a06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:54.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.093+0000 7f1340a00700 1 -- 172.21.15.151:0/3703592213 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f133c1a0e10 con 0x7f133c103460 2024-05-21T21:56:54.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.093+0000 7f133be00700 1 --2- 172.21.15.151:0/3703592213 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f133c103e20 0x7f133c19c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:56:54.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.093+0000 7f1340a00700 1 --2- 172.21.15.151:0/3703592213 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f133c103460 0x7f133c19be00 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f133000e9c0 tx=0x7f133000ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:56:54.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.093+0000 7f1339600700 1 -- 172.21.15.151:0/3703592213 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f133000c9c0 con 0x7f133c103460 2024-05-21T21:56:54.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.093+0000 7f1341e00700 1 -- 172.21.15.151:0/3703592213 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f133c1a1100 con 0x7f133c103460 2024-05-21T21:56:54.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.093+0000 7f1339600700 1 -- 172.21.15.151:0/3703592213 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f133000cb60 con 0x7f133c103460 2024-05-21T21:56:54.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.093+0000 7f1339600700 1 -- 172.21.15.151:0/3703592213 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1330018420 con 0x7f133c103460 2024-05-21T21:56:54.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.094+0000 7f1341e00700 1 -- 172.21.15.151:0/3703592213 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f133c1a8980 con 0x7f133c103460 2024-05-21T21:56:54.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.097+0000 7f1339600700 1 -- 172.21.15.151:0/3703592213 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 26) ==== 149886+0+0 (secure 0 0 0) 0x7f1330010430 con 0x7f133c103460 2024-05-21T21:56:54.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.098+0000 7f1339600700 1 --2- 172.21.15.151:0/3703592213 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f13240b2b10 0x7f13240b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:54.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.099+0000 7f1339600700 1 -- 172.21.15.151:0/3703592213 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f13300e2930 con 0x7f133c103460 2024-05-21T21:56:54.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.099+0000 7f133be00700 1 --2- 172.21.15.151:0/3703592213 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f13240b2b10 0x7f13240b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:54.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.099+0000 7f134dcf3b80 1 Processor -- start 2024-05-21T21:56:54.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.100+0000 7f134dcf3b80 1 -- start start 2024-05-21T21:56:54.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.100+0000 7f133be00700 1 --2- 172.21.15.151:0/3703592213 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f13240b2b10 0x7f13240b4fd0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f133c1107e0 tx=0x7f1328019040 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:56:54.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.100+0000 7f134dcf3b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a512e5e7c0 0x55a512e5eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:54.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.100+0000 7f134dcf3b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a512e5f0e0 0x55a512e58d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:54.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.100+0000 7f134dcf3b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a512e59390 0x55a512e597f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:54.102 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.100+0000 7f134dcf3b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a512db6e40 con 0x55a512e59390 2024-05-21T21:56:54.102 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.100+0000 7f134dcf3b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55a512da69e0 con 0x55a512e5e7c0 2024-05-21T21:56:54.102 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.100+0000 7f134dcf3b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55a512daaa20 con 0x55a512e5f0e0 2024-05-21T21:56:54.102 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.101+0000 7f1340a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a512e5e7c0 0x55a512e5eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:54.102 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.101+0000 7f1341400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a512e59390 0x55a512e597f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:54.102 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.101+0000 7f1340a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a512e5e7c0 0x55a512e5eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:56328/0 (socket says 172.21.15.151:56328) 2024-05-21T21:56:54.102 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.101+0000 7f133be00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a512e5f0e0 0x55a512e58d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:54.102 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.101+0000 7f1340a00700 1 -- 172.21.15.151:0/1316881881 learned_addr learned my addr 172.21.15.151:0/1316881881 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:56:54.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.101+0000 7f1341400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a512e59390 0x55a512e597f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49296/0 (socket says 172.21.15.151:49296) 2024-05-21T21:56:54.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.101+0000 7f1340a00700 1 -- 172.21.15.151:0/1316881881 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a512e5f0e0 msgr2=0x55a512e58d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:56:54.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.101+0000 7f1340a00700 1 --2- 172.21.15.151:0/1316881881 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a512e5f0e0 0x55a512e58d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:54.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.101+0000 7f1340a00700 1 -- 172.21.15.151:0/1316881881 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a512e59390 msgr2=0x55a512e597f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:56:54.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.101+0000 7f1340a00700 1 --2- 172.21.15.151:0/1316881881 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a512e59390 0x55a512e597f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:54.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.101+0000 7f1340a00700 1 -- 172.21.15.151:0/1316881881 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a512d6f420 con 0x55a512e5e7c0 2024-05-21T21:56:54.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.101+0000 7f1340a00700 1 --2- 172.21.15.151:0/1316881881 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a512e5e7c0 0x55a512e5eba0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f13300a59f0 tx=0x7f13300a9480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=240ef69994847d28 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:56:54.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.102+0000 7f132e000700 1 -- 172.21.15.151:0/1316881881 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13300a9970 con 0x55a512e5e7c0 2024-05-21T21:56:54.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.102+0000 7f132e000700 1 -- 172.21.15.151:0/1316881881 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13300add20 con 0x55a512e5e7c0 2024-05-21T21:56:54.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.102+0000 7f132e000700 1 -- 172.21.15.151:0/1316881881 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13300a9dd0 con 0x55a512e5e7c0 2024-05-21T21:56:54.105 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.103+0000 7f134dcf3b80 1 -- 172.21.15.151:0/1316881881 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a512e5e7c0 msgr2=0x55a512e5eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:54.105 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.103+0000 7f134dcf3b80 1 --2- 172.21.15.151:0/1316881881 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a512e5e7c0 0x55a512e5eba0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f13300a59f0 tx=0x7f13300a9480 comp rx=0 tx=0).stop 2024-05-21T21:56:54.105 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.103+0000 7f134dcf3b80 1 -- 172.21.15.151:0/1316881881 shutdown_connections 2024-05-21T21:56:54.105 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.103+0000 7f134dcf3b80 1 --2- 172.21.15.151:0/1316881881 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a512e5e7c0 0x55a512e5eba0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:54.105 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.103+0000 7f134dcf3b80 1 --2- 172.21.15.151:0/1316881881 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a512e59390 0x55a512e597f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:54.105 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.103+0000 7f134dcf3b80 1 --2- 172.21.15.151:0/1316881881 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a512e5f0e0 0x55a512e58d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:54.105 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.103+0000 7f134dcf3b80 1 -- 172.21.15.151:0/1316881881 >> 172.21.15.151:0/1316881881 conn(0x55a512d86be0 msgr2=0x55a512e53810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:56:54.105 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.103+0000 7f134dcf3b80 1 -- 172.21.15.151:0/1316881881 shutdown_connections 2024-05-21T21:56:54.105 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.103+0000 7f134dcf3b80 1 -- 172.21.15.151:0/1316881881 wait complete. 2024-05-21T21:56:54.106 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.104+0000 7f134dcf3b80 1 -- start start 2024-05-21T21:56:54.106 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.104+0000 7f134dcf3b80 1 Processor -- start 2024-05-21T21:56:54.106 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.104+0000 7f134dcf3b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a512e59390 0x55a512e64e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:54.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.105+0000 7f134dcf3b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a512e5e7c0 0x55a512e621b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:54.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.105+0000 7f134dcf3b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a512e5f0e0 0x55a512e626f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:54.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.105+0000 7f134dcf3b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a512cacfa0 con 0x55a512e5e7c0 2024-05-21T21:56:54.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.105+0000 7f134dcf3b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55a512d47760 con 0x55a512e59390 2024-05-21T21:56:54.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.105+0000 7f134dcf3b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55a512e65b40 con 0x55a512e5f0e0 2024-05-21T21:56:54.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.105+0000 7f134dcf3b80 10 client.0 _ll_register_callbacks cb 0x55a512d50c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:56:54.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.105+0000 7f1340a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a512e59390 0x55a512e64e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:54.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.105+0000 7f1340a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a512e59390 0x55a512e64e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:56332/0 (socket says 172.21.15.151:56332) 2024-05-21T21:56:54.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.105+0000 7f1340a00700 1 -- 172.21.15.151:0/3029752827 learned_addr learned my addr 172.21.15.151:0/3029752827 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:56:54.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.105+0000 7f1341400700 1 --2- 172.21.15.151:0/3029752827 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a512e5f0e0 0x55a512e626f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:54.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.105+0000 7f133be00700 1 --2- 172.21.15.151:0/3029752827 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a512e5e7c0 0x55a512e621b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:54.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.105+0000 7f1340a00700 1 -- 172.21.15.151:0/3029752827 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a512e5f0e0 msgr2=0x55a512e626f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:56:54.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.105+0000 7f1340a00700 1 --2- 172.21.15.151:0/3029752827 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a512e5f0e0 0x55a512e626f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:54.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.105+0000 7f1340a00700 1 -- 172.21.15.151:0/3029752827 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a512e5e7c0 msgr2=0x55a512e621b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:56:54.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.105+0000 7f1340a00700 1 --2- 172.21.15.151:0/3029752827 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a512e5e7c0 0x55a512e621b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:54.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.105+0000 7f1340a00700 1 -- 172.21.15.151:0/3029752827 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a512cf68d0 con 0x55a512e59390 2024-05-21T21:56:54.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.105+0000 7f1341400700 1 --2- 172.21.15.151:0/3029752827 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a512e5f0e0 0x55a512e626f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:56:54.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.106+0000 7f1340a00700 1 --2- 172.21.15.151:0/3029752827 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a512e59390 0x55a512e64e10 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1330005fd0 tx=0x7f133000c750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f346df8596f5b96 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:56:54.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.106+0000 7f1322a00700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-21T21:56:54.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.106+0000 7f1322a00700 1 -- 172.21.15.151:0/3029752827 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13300c2030 con 0x55a512e59390 2024-05-21T21:56:54.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.106+0000 7f1322a00700 1 -- 172.21.15.151:0/3029752827 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f13300a5f60 con 0x55a512e59390 2024-05-21T21:56:54.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.106+0000 7f134dcf3b80 1 -- 172.21.15.151:0/3029752827 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13300add20 con 0x55a512e59390 2024-05-21T21:56:54.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.106+0000 7f1322a00700 1 -- 172.21.15.151:0/3029752827 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f13300b27d0 con 0x55a512e59390 2024-05-21T21:56:54.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.106+0000 7f134dcf3b80 10 client.25526 fetch_fsmap learned FSMap version 29 2024-05-21T21:56:54.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.106+0000 7f134dcf3b80 1 -- 172.21.15.151:0/3029752827 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a512d77060 con 0x55a512e59390 2024-05-21T21:56:54.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.106+0000 7f1322a00700 1 -- 172.21.15.151:0/3029752827 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13300add20 con 0x55a512e59390 2024-05-21T21:56:54.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.107+0000 7f1322a00700 1 -- 172.21.15.151:0/3029752827 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f13300e02b0 con 0x55a512e59390 2024-05-21T21:56:54.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.107+0000 7f134dcf3b80 10 client.25526 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T21:56:54.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.107+0000 7f134dcf3b80 10 client.25526 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-05-21T21:56:54.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.107+0000 7f134dcf3b80 20 client.25526 populate_metadata read hostname 'smithi151' 2024-05-21T21:56:54.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.107+0000 7f134dcf3b80 1 --2- 172.21.15.151:0/3029752827 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x55a512e72500 0x55a512e748f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:54.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.107+0000 7f134dcf3b80 4 client.25526 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T21:56:54.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.107+0000 7f134dcf3b80 1 -- 172.21.15.151:0/3029752827 --> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a512db8d90 con 0x55a512e72500 2024-05-21T21:56:54.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.107+0000 7f133be00700 1 --2- 172.21.15.151:0/3029752827 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x55a512e72500 0x55a512e748f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:54.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.108+0000 7f1322a00700 10 client.25526 ms_handle_connect on v2:172.21.15.164:6836/1554609572 2024-05-21T21:56:54.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.108+0000 7f133be00700 1 --2- 172.21.15.151:0/3029752827 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x55a512e72500 0x55a512e748f0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=85707e852344f5dd server_cookie=a064dac1f91af225 in_seq=0 out_seq=0 2024-05-21T21:56:54.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.108+0000 7f1322a00700 1 -- 172.21.15.151:0/3029752827 <== mds.2 v2:172.21.15.164:6836/1554609572 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55a512db8d90 con 0x55a512e72500 2024-05-21T21:56:54.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.108+0000 7f1322a00700 10 client.25526 handle_command_reply: tid=0 2024-05-21T21:56:54.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.109+0000 7f134dcf3b80 1 client.25526 shutdown 2024-05-21T21:56:54.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.109+0000 7f134dcf3b80 1 -- 172.21.15.151:0/3029752827 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a512e59390 msgr2=0x55a512e64e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:54.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.109+0000 7f134dcf3b80 1 --2- 172.21.15.151:0/3029752827 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a512e59390 0x55a512e64e10 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1330005fd0 tx=0x7f133000c750 comp rx=0 tx=0).stop 2024-05-21T21:56:54.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.109+0000 7f134dcf3b80 1 -- 172.21.15.151:0/3029752827 shutdown_connections 2024-05-21T21:56:54.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.109+0000 7f134dcf3b80 1 --2- 172.21.15.151:0/3029752827 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x55a512e72500 0x55a512e748f0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:54.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.109+0000 7f134dcf3b80 1 --2- 172.21.15.151:0/3029752827 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a512e59390 0x55a512e64e10 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:54.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.109+0000 7f134dcf3b80 1 --2- 172.21.15.151:0/3029752827 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a512e5f0e0 0x55a512e626f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:54.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.109+0000 7f134dcf3b80 1 --2- 172.21.15.151:0/3029752827 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a512e5e7c0 0x55a512e621b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:54.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.109+0000 7f1322a00700 0 client.25526 ms_handle_reset on v2:172.21.15.164:6836/1554609572 2024-05-21T21:56:54.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.109+0000 7f134dcf3b80 1 -- 172.21.15.151:0/3029752827 >> 172.21.15.151:0/3029752827 conn(0x55a512d86be0 msgr2=0x55a512e52fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:56:54.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.109+0000 7f134dcf3b80 1 -- 172.21.15.151:0/3029752827 shutdown_connections 2024-05-21T21:56:54.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.109+0000 7f134dcf3b80 1 -- 172.21.15.151:0/3029752827 wait complete. 2024-05-21T21:56:54.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.109+0000 7f134dcf3b80 20 client.25526 trim_cache size 0 max 16384 2024-05-21T21:56:54.127 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.126+0000 7f134dcf3b80 1 Processor -- start 2024-05-21T21:56:54.127 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.126+0000 7f134dcf3b80 1 -- start start 2024-05-21T21:56:54.127 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.126+0000 7f134dcf3b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a512e59390 0x55a512e58d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:54.127 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.126+0000 7f134dcf3b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a512e5e7c0 0x55a512e5fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:54.127 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.126+0000 7f134dcf3b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a512e5f0e0 0x55a512e53740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:54.127 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.126+0000 7f134dcf3b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f133c1affa0 con 0x55a512e59390 2024-05-21T21:56:54.128 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.126+0000 7f134dcf3b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f133c1b1ae0 con 0x55a512e5e7c0 2024-05-21T21:56:54.128 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.126+0000 7f134dcf3b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f133c1b0670 con 0x55a512e5f0e0 2024-05-21T21:56:54.129 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.126+0000 7f1340a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a512e59390 0x55a512e58d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:54.129 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.127+0000 7f1340a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a512e59390 0x55a512e58d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49314/0 (socket says 172.21.15.151:49314) 2024-05-21T21:56:54.129 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.127+0000 7f1340a00700 1 -- 172.21.15.151:0/1085417958 learned_addr learned my addr 172.21.15.151:0/1085417958 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:56:54.129 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.127+0000 7f1341400700 1 --2- 172.21.15.151:0/1085417958 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a512e5f0e0 0x55a512e53740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:54.129 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.127+0000 7f133be00700 1 --2- 172.21.15.151:0/1085417958 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a512e5e7c0 0x55a512e5fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:54.129 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.127+0000 7f1340a00700 1 -- 172.21.15.151:0/1085417958 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a512e5f0e0 msgr2=0x55a512e53740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:56:54.129 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.127+0000 7f1340a00700 1 --2- 172.21.15.151:0/1085417958 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a512e5f0e0 0x55a512e53740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:54.129 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.127+0000 7f1340a00700 1 -- 172.21.15.151:0/1085417958 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a512e5e7c0 msgr2=0x55a512e5fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:56:54.129 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.127+0000 7f1340a00700 1 --2- 172.21.15.151:0/1085417958 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a512e5e7c0 0x55a512e5fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:54.129 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.127+0000 7f1340a00700 1 -- 172.21.15.151:0/1085417958 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a512e542e0 con 0x55a512e59390 2024-05-21T21:56:54.129 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.127+0000 7f133be00700 1 --2- 172.21.15.151:0/1085417958 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a512e5e7c0 0x55a512e5fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:56:54.129 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.127+0000 7f1340a00700 1 --2- 172.21.15.151:0/1085417958 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a512e59390 0x55a512e58d70 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f13300a59f0 tx=0x7f13300a9480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9554cded8fe673df server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:56:54.130 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.128+0000 7f132d600700 1 -- 172.21.15.151:0/1085417958 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13300a5830 con 0x55a512e59390 2024-05-21T21:56:54.130 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.128+0000 7f132d600700 1 -- 172.21.15.151:0/1085417958 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13300add20 con 0x55a512e59390 2024-05-21T21:56:54.130 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.128+0000 7f132d600700 1 -- 172.21.15.151:0/1085417958 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13300a8e30 con 0x55a512e59390 2024-05-21T21:56:54.131 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.128+0000 7f134dcf3b80 1 -- 172.21.15.151:0/1085417958 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a512e59390 msgr2=0x55a512e58d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:54.131 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.128+0000 7f134dcf3b80 1 --2- 172.21.15.151:0/1085417958 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a512e59390 0x55a512e58d70 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f13300a59f0 tx=0x7f13300a9480 comp rx=0 tx=0).stop 2024-05-21T21:56:54.131 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.128+0000 7f134dcf3b80 1 -- 172.21.15.151:0/1085417958 shutdown_connections 2024-05-21T21:56:54.131 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.128+0000 7f134dcf3b80 1 --2- 172.21.15.151:0/1085417958 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a512e5f0e0 0x55a512e53740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:54.131 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.128+0000 7f134dcf3b80 1 --2- 172.21.15.151:0/1085417958 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a512e59390 0x55a512e58d70 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:54.131 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.128+0000 7f134dcf3b80 1 --2- 172.21.15.151:0/1085417958 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a512e5e7c0 0x55a512e5fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:54.131 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.128+0000 7f134dcf3b80 1 -- 172.21.15.151:0/1085417958 >> 172.21.15.151:0/1085417958 conn(0x55a512d86be0 msgr2=0x55a512e60ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:56:54.131 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.129+0000 7f134dcf3b80 1 -- 172.21.15.151:0/1085417958 shutdown_connections 2024-05-21T21:56:54.131 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.129+0000 7f134dcf3b80 1 -- 172.21.15.151:0/1085417958 wait complete. 2024-05-21T21:56:54.131 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.129+0000 7f134dcf3b80 1 -- start start 2024-05-21T21:56:54.132 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.129+0000 7f134dcf3b80 1 Processor -- start 2024-05-21T21:56:54.132 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.130+0000 7f134dcf3b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a512e59390 0x55a512ed5150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:54.132 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.130+0000 7f134dcf3b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a512e5e7c0 0x55a512ed5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:54.132 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.130+0000 7f134dcf3b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a512e5f0e0 0x55a512e74f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:54.132 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.130+0000 7f134dcf3b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f133c1b1410 con 0x55a512e59390 2024-05-21T21:56:54.132 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.130+0000 7f134dcf3b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f133c1b0d40 con 0x55a512e5e7c0 2024-05-21T21:56:54.132 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.130+0000 7f134dcf3b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f133c1adf40 con 0x55a512e5f0e0 2024-05-21T21:56:54.132 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.130+0000 7f134dcf3b80 10 client.0 _ll_register_callbacks cb 0x55a512d50c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:56:54.134 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.130+0000 7f1341400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a512e5f0e0 0x55a512e74f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:54.134 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.130+0000 7f1340a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a512e59390 0x55a512ed5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:54.134 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.130+0000 7f1341400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a512e5f0e0 0x55a512e74f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:58026/0 (socket says 172.21.15.151:58026) 2024-05-21T21:56:54.134 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.130+0000 7f1341400700 1 -- 172.21.15.151:0/2113453601 learned_addr learned my addr 172.21.15.151:0/2113453601 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:56:54.134 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.130+0000 7f133be00700 1 --2- 172.21.15.151:0/2113453601 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a512e5e7c0 0x55a512ed5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:54.134 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.131+0000 7f1341400700 1 -- 172.21.15.151:0/2113453601 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a512e5e7c0 msgr2=0x55a512ed5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:56:54.135 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.131+0000 7f1341400700 1 --2- 172.21.15.151:0/2113453601 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a512e5e7c0 0x55a512ed5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:54.135 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.131+0000 7f1341400700 1 -- 172.21.15.151:0/2113453601 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a512e59390 msgr2=0x55a512ed5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:56:54.135 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.131+0000 7f1341400700 1 --2- 172.21.15.151:0/2113453601 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a512e59390 0x55a512ed5150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:54.135 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.131+0000 7f1341400700 1 -- 172.21.15.151:0/2113453601 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a512e75610 con 0x55a512e5f0e0 2024-05-21T21:56:54.135 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.131+0000 7f1340a00700 1 --2- 172.21.15.151:0/2113453601 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a512e59390 0x55a512ed5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:56:54.135 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.131+0000 7f1341400700 1 --2- 172.21.15.151:0/2113453601 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a512e5f0e0 0x55a512e74f10 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1334014fc0 tx=0x7f13340077e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5894fdddd8fb5db server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:56:54.135 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.131+0000 7f1322a00700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-21T21:56:54.135 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.131+0000 7f1322a00700 1 -- 172.21.15.151:0/2113453601 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1334025060 con 0x55a512e5f0e0 2024-05-21T21:56:54.135 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.131+0000 7f1322a00700 1 -- 172.21.15.151:0/2113453601 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f1334020410 con 0x55a512e5f0e0 2024-05-21T21:56:54.135 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.131+0000 7f134dcf3b80 1 -- 172.21.15.151:0/2113453601 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13300add20 con 0x55a512e5f0e0 2024-05-21T21:56:54.135 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.132+0000 7f1322a00700 1 -- 172.21.15.151:0/2113453601 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1334020610 con 0x55a512e5f0e0 2024-05-21T21:56:54.135 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.132+0000 7f134dcf3b80 10 client.25543 fetch_fsmap learned FSMap version 29 2024-05-21T21:56:54.136 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.132+0000 7f134dcf3b80 1 -- 172.21.15.151:0/2113453601 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a512edc910 con 0x55a512e5f0e0 2024-05-21T21:56:54.136 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.132+0000 7f1322a00700 1 -- 172.21.15.151:0/2113453601 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13300add20 con 0x55a512e5f0e0 2024-05-21T21:56:54.136 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.132+0000 7f1322a00700 1 -- 172.21.15.151:0/2113453601 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f1334015520 con 0x55a512e5f0e0 2024-05-21T21:56:54.136 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.132+0000 7f134dcf3b80 10 client.25543 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T21:56:54.136 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.132+0000 7f134dcf3b80 10 client.25543 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-05-21T21:56:54.136 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.132+0000 7f134dcf3b80 20 client.25543 populate_metadata read hostname 'smithi151' 2024-05-21T21:56:54.136 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.132+0000 7f134dcf3b80 1 --2- 172.21.15.151:0/2113453601 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x55a512edd020 0x55a512edf440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:54.136 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.132+0000 7f134dcf3b80 4 client.25543 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T21:56:54.136 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.132+0000 7f134dcf3b80 1 -- 172.21.15.151:0/2113453601 --> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a512db2570 con 0x55a512edd020 2024-05-21T21:56:54.136 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.133+0000 7f1340a00700 1 --2- 172.21.15.151:0/2113453601 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x55a512edd020 0x55a512edf440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:54.136 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.133+0000 7f1340a00700 1 --2- 172.21.15.151:0/2113453601 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x55a512edd020 0x55a512edf440 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f1f8aa85312723c8 server_cookie=e4da703f6e98adbd in_seq=0 out_seq=0 2024-05-21T21:56:54.136 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.133+0000 7f1322a00700 10 client.25543 ms_handle_connect on v2:172.21.15.164:6836/1554609572 2024-05-21T21:56:54.136 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.133+0000 7f1322a00700 1 -- 172.21.15.151:0/2113453601 <== mds.2 v2:172.21.15.164:6836/1554609572 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a512db2570 con 0x55a512edd020 2024-05-21T21:56:54.137 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.133+0000 7f1322a00700 10 client.25543 handle_command_reply: tid=0 2024-05-21T21:56:54.137 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.133+0000 7f134dcf3b80 1 client.25543 shutdown 2024-05-21T21:56:54.137 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.134+0000 7f134dcf3b80 1 -- 172.21.15.151:0/2113453601 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a512e5f0e0 msgr2=0x55a512e74f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:54.138 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.134+0000 7f134dcf3b80 1 --2- 172.21.15.151:0/2113453601 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a512e5f0e0 0x55a512e74f10 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1334014fc0 tx=0x7f13340077e0 comp rx=0 tx=0).stop 2024-05-21T21:56:54.138 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.134+0000 7f134dcf3b80 1 -- 172.21.15.151:0/2113453601 shutdown_connections 2024-05-21T21:56:54.138 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.134+0000 7f134dcf3b80 1 --2- 172.21.15.151:0/2113453601 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x55a512edd020 0x55a512edf440 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:54.138 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.134+0000 7f134dcf3b80 1 --2- 172.21.15.151:0/2113453601 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a512e5f0e0 0x55a512e74f10 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:54.138 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.134+0000 7f134dcf3b80 1 --2- 172.21.15.151:0/2113453601 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a512e59390 0x55a512ed5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:54.138 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.134+0000 7f1322a00700 0 client.25543 ms_handle_reset on v2:172.21.15.164:6836/1554609572 2024-05-21T21:56:54.138 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.134+0000 7f134dcf3b80 1 --2- 172.21.15.151:0/2113453601 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a512e5e7c0 0x55a512ed5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:54.138 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.134+0000 7f134dcf3b80 1 -- 172.21.15.151:0/2113453601 >> 172.21.15.151:0/2113453601 conn(0x55a512d86be0 msgr2=0x55a512e5fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:56:54.138 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.134+0000 7f134dcf3b80 1 -- 172.21.15.151:0/2113453601 shutdown_connections 2024-05-21T21:56:54.138 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.134+0000 7f134dcf3b80 1 -- 172.21.15.151:0/2113453601 wait complete. 2024-05-21T21:56:54.138 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.134+0000 7f134dcf3b80 20 client.25543 trim_cache size 0 max 16384 2024-05-21T21:56:54.139 INFO:teuthology.orchestra.run.smithi151.stdout:[] 2024-05-21T21:56:54.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.135+0000 7f1341e00700 1 -- 172.21.15.151:0/3703592213 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f13240b2b10 msgr2=0x7f13240b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:54.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.135+0000 7f1341e00700 1 --2- 172.21.15.151:0/3703592213 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f13240b2b10 0x7f13240b4fd0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f133c1107e0 tx=0x7f1328019040 comp rx=0 tx=0).stop 2024-05-21T21:56:54.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.135+0000 7f1341e00700 1 -- 172.21.15.151:0/3703592213 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f133c103460 msgr2=0x7f133c19be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:54.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.135+0000 7f1341e00700 1 --2- 172.21.15.151:0/3703592213 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f133c103460 0x7f133c19be00 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f133000e9c0 tx=0x7f133000ecd0 comp rx=0 tx=0).stop 2024-05-21T21:56:54.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.135+0000 7f1341e00700 1 -- 172.21.15.151:0/3703592213 shutdown_connections 2024-05-21T21:56:54.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.135+0000 7f1341e00700 1 --2- 172.21.15.151:0/3703592213 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f13240b2b10 0x7f13240b4fd0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:54.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.135+0000 7f1341e00700 1 --2- 172.21.15.151:0/3703592213 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f133c1094f0 0x7f133c1a06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:54.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.135+0000 7f1341e00700 1 --2- 172.21.15.151:0/3703592213 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f133c103460 0x7f133c19be00 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:54.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.135+0000 7f1341e00700 1 --2- 172.21.15.151:0/3703592213 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f133c103e20 0x7f133c19c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:54.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.135+0000 7f1341e00700 1 -- 172.21.15.151:0/3703592213 >> 172.21.15.151:0/3703592213 conn(0x7f133c0fef50 msgr2=0x7f133c0ff570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:56:54.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.135+0000 7f1341e00700 1 -- 172.21.15.151:0/3703592213 shutdown_connections 2024-05-21T21:56:54.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:54.135+0000 7f1341e00700 1 -- 172.21.15.151:0/3703592213 wait complete. 2024-05-21T21:56:54.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:56:54 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:56:54.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:56:54 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:56:54.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:56:54 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T21:56:54.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:56:54 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:56:54.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:56:54 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:56:54.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:56:54 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T21:56:55.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:56:54 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:56:55.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:56:54 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:56:55.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:56:54 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T21:56:55.086 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-21T21:56:55.561 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:56:55.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:56:55 smithi151 ceph-mon[31623]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-21T21:56:55.985 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:56:55 smithi184 ceph-mon[36942]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-21T21:56:56.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:56:55 smithi164 ceph-mon[36269]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-21T21:56:56.932 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:56:56 smithi151 ceph-mon[31623]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T21:56:56.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:56:56 smithi184 ceph-mon[36942]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T21:56:57.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:56:56 smithi164 ceph-mon[36269]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T21:56:57.756 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.754+0000 7f37d2000700 1 -- 172.21.15.151:0/3283138039 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f37cc106010 msgr2=0x7f37cc112520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:57.756 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.754+0000 7f37d2000700 1 --2- 172.21.15.151:0/3283138039 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f37cc106010 0x7f37cc112520 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f37c400b0a0 tx=0x7f37c400b3b0 comp rx=0 tx=0).stop 2024-05-21T21:56:57.757 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.755+0000 7f37d2000700 1 -- 172.21.15.151:0/3283138039 shutdown_connections 2024-05-21T21:56:57.757 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.755+0000 7f37d2000700 1 --2- 172.21.15.151:0/3283138039 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f37cc106010 0x7f37cc112520 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:57.757 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.755+0000 7f37d2000700 1 --2- 172.21.15.151:0/3283138039 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37cc10b650 0x7f37cc10ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:57.757 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.755+0000 7f37d2000700 1 --2- 172.21.15.151:0/3283138039 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37cc105650 0x7f37cc105ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:57.757 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.755+0000 7f37d2000700 1 -- 172.21.15.151:0/3283138039 >> 172.21.15.151:0/3283138039 conn(0x7f37cc101120 msgr2=0x7f37cc103540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:56:57.757 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.755+0000 7f37d2000700 1 -- 172.21.15.151:0/3283138039 shutdown_connections 2024-05-21T21:56:57.757 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.755+0000 7f37d2000700 1 -- 172.21.15.151:0/3283138039 wait complete. 2024-05-21T21:56:57.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.756+0000 7f37d2000700 1 Processor -- start 2024-05-21T21:56:57.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.756+0000 7f37d2000700 1 -- start start 2024-05-21T21:56:57.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.757+0000 7f37d2000700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37cc105650 0x7f37cc1a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:57.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.757+0000 7f37d2000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37cc106010 0x7f37cc1a1e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:57.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.757+0000 7f37d2000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f37cc10b650 0x7f37cc19ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:57.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.757+0000 7f37d2000700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f37cc115850 con 0x7f37cc10b650 2024-05-21T21:56:57.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.757+0000 7f37d2000700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f37cc1156d0 con 0x7f37cc106010 2024-05-21T21:56:57.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.757+0000 7f37d2000700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f37cc1159d0 con 0x7f37cc105650 2024-05-21T21:56:57.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.757+0000 7f37d0c00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37cc105650 0x7f37cc1a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:57.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.757+0000 7f37d1600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f37cc10b650 0x7f37cc19ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:57.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.757+0000 7f37cbe00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37cc106010 0x7f37cc1a1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:57.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.757+0000 7f37d0c00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37cc105650 0x7f37cc1a18d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:60876/0 (socket says 172.21.15.151:60876) 2024-05-21T21:56:57.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.757+0000 7f37d0c00700 1 -- 172.21.15.151:0/1604048473 learned_addr learned my addr 172.21.15.151:0/1604048473 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:56:57.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.757+0000 7f37d0c00700 1 -- 172.21.15.151:0/1604048473 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37cc106010 msgr2=0x7f37cc1a1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:57.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.757+0000 7f37d0c00700 1 --2- 172.21.15.151:0/1604048473 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37cc106010 0x7f37cc1a1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:57.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.757+0000 7f37d0c00700 1 -- 172.21.15.151:0/1604048473 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f37cc10b650 msgr2=0x7f37cc19ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:57.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.757+0000 7f37d0c00700 1 --2- 172.21.15.151:0/1604048473 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f37cc10b650 0x7f37cc19ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:57.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.757+0000 7f37d0c00700 1 -- 172.21.15.151:0/1604048473 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37cc19c1f0 con 0x7f37cc105650 2024-05-21T21:56:57.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.758+0000 7f37d1600700 1 --2- 172.21.15.151:0/1604048473 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f37cc10b650 0x7f37cc19ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:56:57.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.758+0000 7f37d0c00700 1 --2- 172.21.15.151:0/1604048473 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37cc105650 0x7f37cc1a18d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f37c000e9c0 tx=0x7f37c000ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:56:57.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.758+0000 7f37c9600700 1 -- 172.21.15.151:0/1604048473 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c000cbd0 con 0x7f37cc105650 2024-05-21T21:56:57.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.758+0000 7f37d2000700 1 -- 172.21.15.151:0/1604048473 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37cc19c4e0 con 0x7f37cc105650 2024-05-21T21:56:57.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.758+0000 7f37d2000700 1 -- 172.21.15.151:0/1604048473 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37cc1a87a0 con 0x7f37cc105650 2024-05-21T21:56:57.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.758+0000 7f37c9600700 1 -- 172.21.15.151:0/1604048473 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37c000ee60 con 0x7f37cc105650 2024-05-21T21:56:57.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.758+0000 7f37c9600700 1 -- 172.21.15.151:0/1604048473 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c0005550 con 0x7f37cc105650 2024-05-21T21:56:57.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.762+0000 7f37c9600700 1 -- 172.21.15.151:0/1604048473 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 26) ==== 149886+0+0 (secure 0 0 0) 0x7f37c0004750 con 0x7f37cc105650 2024-05-21T21:56:57.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.763+0000 7f37ddfdfb80 1 Processor -- start 2024-05-21T21:56:57.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.764+0000 7f37ddfdfb80 1 -- start start 2024-05-21T21:56:57.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.764+0000 7f37c9600700 1 --2- 172.21.15.151:0/1604048473 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f37b40b2b10 0x7f37b40b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:57.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.764+0000 7f37c9600700 1 -- 172.21.15.151:0/1604048473 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f37c00e2b40 con 0x7f37cc105650 2024-05-21T21:56:57.766 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.764+0000 7f37ddfdfb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563fd1c50020 0x563fd1c50400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:57.766 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.764+0000 7f37ddfdfb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563fd1c50940 0x563fd1c4a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:57.766 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.764+0000 7f37ddfdfb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563fd1c4aad0 0x563fd1c4af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:57.766 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.764+0000 7f37ddfdfb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x563fd1ba8e40 con 0x563fd1c4aad0 2024-05-21T21:56:57.766 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.764+0000 7f37ddfdfb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x563fd1b989e0 con 0x563fd1c50940 2024-05-21T21:56:57.766 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.764+0000 7f37ddfdfb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x563fd1b9ca20 con 0x563fd1c50020 2024-05-21T21:56:57.766 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.764+0000 7f37d1600700 1 --2- 172.21.15.151:0/1604048473 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f37b40b2b10 0x7f37b40b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:57.766 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.764+0000 7f37d0c00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563fd1c50020 0x563fd1c50400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:57.766 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.764+0000 7f37d0c00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563fd1c50020 0x563fd1c50400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:60890/0 (socket says 172.21.15.151:60890) 2024-05-21T21:56:57.766 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.764+0000 7f37d0c00700 1 -- 172.21.15.151:0/554913190 learned_addr learned my addr 172.21.15.151:0/554913190 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:56:57.767 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.765+0000 7f37d1600700 1 --2- 172.21.15.151:0/554913190 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563fd1c4aad0 0x563fd1c4af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:57.767 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.765+0000 7f37d0c00700 1 -- 172.21.15.151:0/554913190 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563fd1c50940 msgr2=0x563fd1c4a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:56:57.767 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.765+0000 7f37d0c00700 1 --2- 172.21.15.151:0/554913190 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563fd1c50940 0x563fd1c4a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:57.767 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.765+0000 7f37d0c00700 1 -- 172.21.15.151:0/554913190 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563fd1c4aad0 msgr2=0x563fd1c4af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:56:57.767 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.765+0000 7f37d0c00700 1 --2- 172.21.15.151:0/554913190 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563fd1c4aad0 0x563fd1c4af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:57.767 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.765+0000 7f37d0c00700 1 -- 172.21.15.151:0/554913190 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fd1b61420 con 0x563fd1c50020 2024-05-21T21:56:57.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.765+0000 7f37d1600700 1 --2- 172.21.15.151:0/554913190 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563fd1c4aad0 0x563fd1c4af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T21:56:57.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.765+0000 7f37d0c00700 1 --2- 172.21.15.151:0/554913190 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563fd1c50020 0x563fd1c50400 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f37c00a9b10 tx=0x7f37c00a93f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8a1e417568711ef server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:56:57.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.766+0000 7f37d1600700 1 --2- 172.21.15.151:0/1604048473 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f37b40b2b10 0x7f37b40b4fd0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f37c4009e60 tx=0x7f37c400bf60 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:56:57.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.766+0000 7f37be000700 1 -- 172.21.15.151:0/554913190 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c00aa280 con 0x563fd1c50020 2024-05-21T21:56:57.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.766+0000 7f37be000700 1 -- 172.21.15.151:0/554913190 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37c00a9b80 con 0x563fd1c50020 2024-05-21T21:56:57.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.766+0000 7f37be000700 1 -- 172.21.15.151:0/554913190 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c00b2790 con 0x563fd1c50020 2024-05-21T21:56:57.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.767+0000 7f37ddfdfb80 1 -- 172.21.15.151:0/554913190 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563fd1c50020 msgr2=0x563fd1c50400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:57.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.767+0000 7f37ddfdfb80 1 --2- 172.21.15.151:0/554913190 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563fd1c50020 0x563fd1c50400 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f37c00a9b10 tx=0x7f37c00a93f0 comp rx=0 tx=0).stop 2024-05-21T21:56:57.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.767+0000 7f37ddfdfb80 1 -- 172.21.15.151:0/554913190 shutdown_connections 2024-05-21T21:56:57.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.767+0000 7f37ddfdfb80 1 --2- 172.21.15.151:0/554913190 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563fd1c4aad0 0x563fd1c4af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:57.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.767+0000 7f37ddfdfb80 1 --2- 172.21.15.151:0/554913190 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563fd1c50020 0x563fd1c50400 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:57.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.767+0000 7f37ddfdfb80 1 --2- 172.21.15.151:0/554913190 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563fd1c50940 0x563fd1c4a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:57.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.767+0000 7f37ddfdfb80 1 -- 172.21.15.151:0/554913190 >> 172.21.15.151:0/554913190 conn(0x563fd1b78be0 msgr2=0x563fd1c45150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:56:57.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.767+0000 7f37ddfdfb80 1 -- 172.21.15.151:0/554913190 shutdown_connections 2024-05-21T21:56:57.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.767+0000 7f37ddfdfb80 1 -- 172.21.15.151:0/554913190 wait complete. 2024-05-21T21:56:57.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.768+0000 7f37ddfdfb80 1 -- start start 2024-05-21T21:56:57.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.768+0000 7f37ddfdfb80 1 Processor -- start 2024-05-21T21:56:57.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.768+0000 7f37ddfdfb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563fd1c4aad0 0x563fd1c56730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:57.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.768+0000 7f37ddfdfb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563fd1c50020 0x563fd1c53ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:57.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.768+0000 7f37ddfdfb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563fd1c50940 0x563fd1c54010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:57.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.768+0000 7f37ddfdfb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x563fd1a9efa0 con 0x563fd1c4aad0 2024-05-21T21:56:57.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.768+0000 7f37ddfdfb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x563fd1b39760 con 0x563fd1c50940 2024-05-21T21:56:57.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.769+0000 7f37ddfdfb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x563fd1c57460 con 0x563fd1c50020 2024-05-21T21:56:57.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.769+0000 7f37ddfdfb80 10 client.0 _ll_register_callbacks cb 0x563fd1b42c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:56:57.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.769+0000 7f37cbe00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563fd1c50020 0x563fd1c53ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:57.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.769+0000 7f37d0c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563fd1c4aad0 0x563fd1c56730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:57.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.769+0000 7f37cbe00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563fd1c50020 0x563fd1c53ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:60898/0 (socket says 172.21.15.151:60898) 2024-05-21T21:56:57.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.769+0000 7f37d0c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563fd1c4aad0 0x563fd1c56730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:33896/0 (socket says 172.21.15.151:33896) 2024-05-21T21:56:57.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.769+0000 7f37cbe00700 1 -- 172.21.15.151:0/1975203479 learned_addr learned my addr 172.21.15.151:0/1975203479 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:56:57.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.769+0000 7f37d1600700 1 --2- 172.21.15.151:0/1975203479 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563fd1c50940 0x563fd1c54010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:57.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.769+0000 7f37cbe00700 1 -- 172.21.15.151:0/1975203479 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563fd1c50940 msgr2=0x563fd1c54010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:56:57.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.769+0000 7f37cbe00700 1 --2- 172.21.15.151:0/1975203479 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563fd1c50940 0x563fd1c54010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:57.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.769+0000 7f37cbe00700 1 -- 172.21.15.151:0/1975203479 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563fd1c4aad0 msgr2=0x563fd1c56730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:56:57.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.769+0000 7f37cbe00700 1 --2- 172.21.15.151:0/1975203479 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563fd1c4aad0 0x563fd1c56730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:57.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.769+0000 7f37cbe00700 1 -- 172.21.15.151:0/1975203479 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fd1ae88d0 con 0x563fd1c50020 2024-05-21T21:56:57.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.770+0000 7f37cbe00700 1 --2- 172.21.15.151:0/1975203479 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563fd1c50020 0x563fd1c53ad0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f37b80116c0 tx=0x7f37b80119d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d913976dc63e7f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:56:57.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.770+0000 7f37b2a00700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-21T21:56:57.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.770+0000 7f37b2a00700 1 -- 172.21.15.151:0/1975203479 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b801e060 con 0x563fd1c50020 2024-05-21T21:56:57.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.770+0000 7f37ddfdfb80 1 -- 172.21.15.151:0/1975203479 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37c00a9b80 con 0x563fd1c50020 2024-05-21T21:56:57.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.770+0000 7f37b2a00700 1 -- 172.21.15.151:0/1975203479 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f37b800e880 con 0x563fd1c50020 2024-05-21T21:56:57.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.770+0000 7f37b2a00700 1 -- 172.21.15.151:0/1975203479 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f37b800ea80 con 0x563fd1c50020 2024-05-21T21:56:57.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.770+0000 7f37ddfdfb80 10 client.25555 fetch_fsmap learned FSMap version 29 2024-05-21T21:56:57.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.770+0000 7f37ddfdfb80 1 -- 172.21.15.151:0/1975203479 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563fd1b69060 con 0x563fd1c50020 2024-05-21T21:56:57.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.771+0000 7f37b2a00700 1 -- 172.21.15.151:0/1975203479 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37c00a9b80 con 0x563fd1c50020 2024-05-21T21:56:57.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.771+0000 7f37b2a00700 1 -- 172.21.15.151:0/1975203479 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f37b8021c30 con 0x563fd1c50020 2024-05-21T21:56:57.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.771+0000 7f37ddfdfb80 10 client.25555 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T21:56:57.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.771+0000 7f37ddfdfb80 10 client.25555 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-21T21:56:57.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.771+0000 7f37ddfdfb80 20 client.25555 populate_metadata read hostname 'smithi151' 2024-05-21T21:56:57.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.771+0000 7f37ddfdfb80 1 --2- 172.21.15.151:0/1975203479 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x563fd1c63ff0 0x563fd1c663e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:57.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.771+0000 7f37ddfdfb80 4 client.25555 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T21:56:57.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.771+0000 7f37ddfdfb80 1 -- 172.21.15.151:0/1975203479 --> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563fd1baad90 con 0x563fd1c63ff0 2024-05-21T21:56:57.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.772+0000 7f37d0c00700 1 --2- 172.21.15.151:0/1975203479 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x563fd1c63ff0 0x563fd1c663e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:57.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.772+0000 7f37d0c00700 1 --2- 172.21.15.151:0/1975203479 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x563fd1c63ff0 0x563fd1c663e0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=af50364538f7bff6 server_cookie=a47510fbf18a92ba in_seq=0 out_seq=0 2024-05-21T21:56:57.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.772+0000 7f37b2a00700 10 client.25555 ms_handle_connect on v2:172.21.15.184:6838/3059935143 2024-05-21T21:56:57.776 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.773+0000 7f37b2a00700 1 -- 172.21.15.151:0/1975203479 <== mds.1 v2:172.21.15.184:6838/3059935143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x563fd1baad90 con 0x563fd1c63ff0 2024-05-21T21:56:57.776 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.773+0000 7f37b2a00700 10 client.25555 handle_command_reply: tid=0 2024-05-21T21:56:57.776 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.773+0000 7f37ddfdfb80 1 client.25555 shutdown 2024-05-21T21:56:57.776 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.773+0000 7f37ddfdfb80 1 -- 172.21.15.151:0/1975203479 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563fd1c50020 msgr2=0x563fd1c53ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:57.776 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.773+0000 7f37ddfdfb80 1 --2- 172.21.15.151:0/1975203479 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563fd1c50020 0x563fd1c53ad0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f37b80116c0 tx=0x7f37b80119d0 comp rx=0 tx=0).stop 2024-05-21T21:56:57.776 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.773+0000 7f37ddfdfb80 1 -- 172.21.15.151:0/1975203479 shutdown_connections 2024-05-21T21:56:57.776 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.773+0000 7f37ddfdfb80 1 --2- 172.21.15.151:0/1975203479 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x563fd1c63ff0 0x563fd1c663e0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:57.776 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.773+0000 7f37ddfdfb80 1 --2- 172.21.15.151:0/1975203479 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563fd1c50940 0x563fd1c54010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:57.776 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.773+0000 7f37ddfdfb80 1 --2- 172.21.15.151:0/1975203479 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563fd1c4aad0 0x563fd1c56730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:57.776 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.773+0000 7f37ddfdfb80 1 --2- 172.21.15.151:0/1975203479 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563fd1c50020 0x563fd1c53ad0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:57.776 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.773+0000 7f37ddfdfb80 1 -- 172.21.15.151:0/1975203479 >> 172.21.15.151:0/1975203479 conn(0x563fd1b78be0 msgr2=0x563fd1c494a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:56:57.776 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.773+0000 7f37b2a00700 0 client.25555 ms_handle_reset on v2:172.21.15.184:6838/3059935143 2024-05-21T21:56:57.776 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.774+0000 7f37ddfdfb80 1 -- 172.21.15.151:0/1975203479 shutdown_connections 2024-05-21T21:56:57.776 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.774+0000 7f37ddfdfb80 1 -- 172.21.15.151:0/1975203479 wait complete. 2024-05-21T21:56:57.776 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.774+0000 7f37ddfdfb80 20 client.25555 trim_cache size 0 max 16384 2024-05-21T21:56:57.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.790+0000 7f37ddfdfb80 1 Processor -- start 2024-05-21T21:56:57.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.790+0000 7f37ddfdfb80 1 -- start start 2024-05-21T21:56:57.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.791+0000 7f37ddfdfb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563fd1c4aad0 0x563fd1c4a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:57.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.791+0000 7f37ddfdfb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563fd1c50020 0x563fd1c523c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:57.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.791+0000 7f37ddfdfb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563fd1c50940 0x563fd1c52900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:57.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.791+0000 7f37ddfdfb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f37cc1afe90 con 0x563fd1c4aad0 2024-05-21T21:56:57.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.791+0000 7f37ddfdfb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f37cc1b19d0 con 0x563fd1c50020 2024-05-21T21:56:57.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.791+0000 7f37ddfdfb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f37cc1b0560 con 0x563fd1c50940 2024-05-21T21:56:57.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.791+0000 7f37d0c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563fd1c4aad0 0x563fd1c4a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:57.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.791+0000 7f37d0c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563fd1c4aad0 0x563fd1c4a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:33900/0 (socket says 172.21.15.151:33900) 2024-05-21T21:56:57.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.791+0000 7f37d0c00700 1 -- 172.21.15.151:0/3424146201 learned_addr learned my addr 172.21.15.151:0/3424146201 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:56:57.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.791+0000 7f37cbe00700 1 --2- 172.21.15.151:0/3424146201 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563fd1c50020 0x563fd1c523c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:57.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.791+0000 7f37d1600700 1 --2- 172.21.15.151:0/3424146201 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563fd1c50940 0x563fd1c52900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:57.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.791+0000 7f37d0c00700 1 -- 172.21.15.151:0/3424146201 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563fd1c50940 msgr2=0x563fd1c52900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:56:57.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.791+0000 7f37d0c00700 1 --2- 172.21.15.151:0/3424146201 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563fd1c50940 0x563fd1c52900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:57.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.791+0000 7f37d0c00700 1 -- 172.21.15.151:0/3424146201 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563fd1c50020 msgr2=0x563fd1c523c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:56:57.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.791+0000 7f37d0c00700 1 --2- 172.21.15.151:0/3424146201 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563fd1c50020 0x563fd1c523c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:57.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.791+0000 7f37d0c00700 1 -- 172.21.15.151:0/3424146201 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fd1c61ac0 con 0x563fd1c4aad0 2024-05-21T21:56:57.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.792+0000 7f37d0c00700 1 --2- 172.21.15.151:0/3424146201 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563fd1c4aad0 0x563fd1c4a4b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f37c0009230 tx=0x7f37c00a94d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=959aaf12e64fb97e server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:56:57.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.792+0000 7f37bd600700 1 -- 172.21.15.151:0/3424146201 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c008e070 con 0x563fd1c4aad0 2024-05-21T21:56:57.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.792+0000 7f37bd600700 1 -- 172.21.15.151:0/3424146201 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37c00a9b80 con 0x563fd1c4aad0 2024-05-21T21:56:57.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.792+0000 7f37bd600700 1 -- 172.21.15.151:0/3424146201 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c00a6170 con 0x563fd1c4aad0 2024-05-21T21:56:57.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.793+0000 7f37ddfdfb80 1 -- 172.21.15.151:0/3424146201 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563fd1c4aad0 msgr2=0x563fd1c4a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:57.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.793+0000 7f37ddfdfb80 1 --2- 172.21.15.151:0/3424146201 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563fd1c4aad0 0x563fd1c4a4b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f37c0009230 tx=0x7f37c00a94d0 comp rx=0 tx=0).stop 2024-05-21T21:56:57.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.793+0000 7f37ddfdfb80 1 -- 172.21.15.151:0/3424146201 shutdown_connections 2024-05-21T21:56:57.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.793+0000 7f37ddfdfb80 1 --2- 172.21.15.151:0/3424146201 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563fd1c50940 0x563fd1c52900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:57.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.793+0000 7f37ddfdfb80 1 --2- 172.21.15.151:0/3424146201 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563fd1c4aad0 0x563fd1c4a4b0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:57.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.793+0000 7f37ddfdfb80 1 --2- 172.21.15.151:0/3424146201 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563fd1c50020 0x563fd1c523c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:57.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.793+0000 7f37ddfdfb80 1 -- 172.21.15.151:0/3424146201 >> 172.21.15.151:0/3424146201 conn(0x563fd1b78be0 msgr2=0x563fd1c4db70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:56:57.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.793+0000 7f37ddfdfb80 1 -- 172.21.15.151:0/3424146201 shutdown_connections 2024-05-21T21:56:57.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.793+0000 7f37ddfdfb80 1 -- 172.21.15.151:0/3424146201 wait complete. 2024-05-21T21:56:57.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.793+0000 7f37ddfdfb80 1 -- start start 2024-05-21T21:56:57.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.794+0000 7f37ddfdfb80 1 Processor -- start 2024-05-21T21:56:57.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.794+0000 7f37ddfdfb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563fd1c4aad0 0x563fd1c5cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:57.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.794+0000 7f37ddfdfb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563fd1c50020 0x563fd1c6b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:57.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.794+0000 7f37ddfdfb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563fd1c50940 0x563fd1c6ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:57.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.794+0000 7f37ddfdfb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f37cc1b1300 con 0x563fd1c4aad0 2024-05-21T21:56:57.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.794+0000 7f37ddfdfb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f37cc1b0c30 con 0x563fd1c50020 2024-05-21T21:56:57.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.794+0000 7f37ddfdfb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f37cc1ade30 con 0x563fd1c50940 2024-05-21T21:56:57.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.794+0000 7f37ddfdfb80 10 client.0 _ll_register_callbacks cb 0x563fd1b42c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:56:57.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.794+0000 7f37d0c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563fd1c4aad0 0x563fd1c5cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:57.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.795+0000 7f37d0c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563fd1c4aad0 0x563fd1c5cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:33906/0 (socket says 172.21.15.151:33906) 2024-05-21T21:56:57.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.795+0000 7f37d0c00700 1 -- 172.21.15.151:0/869003810 learned_addr learned my addr 172.21.15.151:0/869003810 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:56:57.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.795+0000 7f37d1600700 1 --2- 172.21.15.151:0/869003810 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563fd1c50940 0x563fd1c6ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:57.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.795+0000 7f37cbe00700 1 --2- 172.21.15.151:0/869003810 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563fd1c50020 0x563fd1c6b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:57.799 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.795+0000 7f37d0c00700 1 -- 172.21.15.151:0/869003810 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563fd1c50940 msgr2=0x563fd1c6ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:56:57.799 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.795+0000 7f37d0c00700 1 --2- 172.21.15.151:0/869003810 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563fd1c50940 0x563fd1c6ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:57.799 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.795+0000 7f37d0c00700 1 -- 172.21.15.151:0/869003810 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563fd1c50020 msgr2=0x563fd1c6b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:56:57.799 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.795+0000 7f37d0c00700 1 --2- 172.21.15.151:0/869003810 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563fd1c50020 0x563fd1c6b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:57.799 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.795+0000 7f37d0c00700 1 -- 172.21.15.151:0/869003810 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fd1c6c230 con 0x563fd1c4aad0 2024-05-21T21:56:57.799 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.795+0000 7f37d0c00700 1 --2- 172.21.15.151:0/869003810 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563fd1c4aad0 0x563fd1c5cd60 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f37c00a5c00 tx=0x7f37c00c42b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6300e2551214995e server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:56:57.799 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.795+0000 7f37b2a00700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-21T21:56:57.799 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.795+0000 7f37b2a00700 1 -- 172.21.15.151:0/869003810 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c008e040 con 0x563fd1c4aad0 2024-05-21T21:56:57.799 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.795+0000 7f37b2a00700 1 -- 172.21.15.151:0/869003810 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f37c00a6170 con 0x563fd1c4aad0 2024-05-21T21:56:57.799 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.795+0000 7f37ddfdfb80 1 -- 172.21.15.151:0/869003810 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37c00a9b80 con 0x563fd1c4aad0 2024-05-21T21:56:57.799 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.796+0000 7f37b2a00700 1 -- 172.21.15.151:0/869003810 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f37c00aa280 con 0x563fd1c4aad0 2024-05-21T21:56:57.799 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.796+0000 7f37ddfdfb80 10 client.15435 fetch_fsmap learned FSMap version 29 2024-05-21T21:56:57.799 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.796+0000 7f37ddfdfb80 1 -- 172.21.15.151:0/869003810 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563fd1cd7390 con 0x563fd1c4aad0 2024-05-21T21:56:57.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.796+0000 7f37b2a00700 1 -- 172.21.15.151:0/869003810 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37c00a9b80 con 0x563fd1c4aad0 2024-05-21T21:56:57.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.796+0000 7f37b2a00700 1 -- 172.21.15.151:0/869003810 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f37c00a8de0 con 0x563fd1c4aad0 2024-05-21T21:56:57.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.796+0000 7f37ddfdfb80 10 client.15435 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T21:56:57.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.796+0000 7f37ddfdfb80 10 client.15435 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-21T21:56:57.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.796+0000 7f37ddfdfb80 20 client.15435 populate_metadata read hostname 'smithi151' 2024-05-21T21:56:57.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.796+0000 7f37ddfdfb80 1 --2- 172.21.15.151:0/869003810 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x563fd1cd7610 0x563fd1cd9a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:56:57.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.796+0000 7f37ddfdfb80 4 client.15435 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T21:56:57.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.796+0000 7f37ddfdfb80 1 -- 172.21.15.151:0/869003810 --> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563fd1ba4570 con 0x563fd1cd7610 2024-05-21T21:56:57.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.797+0000 7f37cbe00700 1 --2- 172.21.15.151:0/869003810 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x563fd1cd7610 0x563fd1cd9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:56:57.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.797+0000 7f37cbe00700 1 --2- 172.21.15.151:0/869003810 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x563fd1cd7610 0x563fd1cd9a00 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=45a214579143f5d1 server_cookie=b1eb332dc93018fd in_seq=0 out_seq=0 2024-05-21T21:56:57.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.797+0000 7f37b2a00700 10 client.15435 ms_handle_connect on v2:172.21.15.184:6838/3059935143 2024-05-21T21:56:57.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.797+0000 7f37b2a00700 1 -- 172.21.15.151:0/869003810 <== mds.1 v2:172.21.15.184:6838/3059935143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563fd1ba4570 con 0x563fd1cd7610 2024-05-21T21:56:57.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.797+0000 7f37b2a00700 10 client.15435 handle_command_reply: tid=0 2024-05-21T21:56:57.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.797+0000 7f37ddfdfb80 1 client.15435 shutdown 2024-05-21T21:56:57.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.798+0000 7f37ddfdfb80 1 -- 172.21.15.151:0/869003810 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563fd1c4aad0 msgr2=0x563fd1c5cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:57.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.798+0000 7f37ddfdfb80 1 --2- 172.21.15.151:0/869003810 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563fd1c4aad0 0x563fd1c5cd60 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f37c00a5c00 tx=0x7f37c00c42b0 comp rx=0 tx=0).stop 2024-05-21T21:56:57.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.798+0000 7f37ddfdfb80 1 -- 172.21.15.151:0/869003810 shutdown_connections 2024-05-21T21:56:57.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.798+0000 7f37ddfdfb80 1 --2- 172.21.15.151:0/869003810 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x563fd1cd7610 0x563fd1cd9a00 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:57.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.798+0000 7f37ddfdfb80 1 --2- 172.21.15.151:0/869003810 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563fd1c50940 0x563fd1c6ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:57.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.798+0000 7f37ddfdfb80 1 --2- 172.21.15.151:0/869003810 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563fd1c4aad0 0x563fd1c5cd60 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:57.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.798+0000 7f37ddfdfb80 1 --2- 172.21.15.151:0/869003810 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563fd1c50020 0x563fd1c6b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:57.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.798+0000 7f37b2a00700 0 client.15435 ms_handle_reset on v2:172.21.15.184:6838/3059935143 2024-05-21T21:56:57.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.798+0000 7f37ddfdfb80 1 -- 172.21.15.151:0/869003810 >> 172.21.15.151:0/869003810 conn(0x563fd1b78be0 msgr2=0x563fd1c4db70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:56:57.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.798+0000 7f37ddfdfb80 1 -- 172.21.15.151:0/869003810 shutdown_connections 2024-05-21T21:56:57.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.798+0000 7f37ddfdfb80 1 -- 172.21.15.151:0/869003810 wait complete. 2024-05-21T21:56:57.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.798+0000 7f37ddfdfb80 20 client.15435 trim_cache size 0 max 16384 2024-05-21T21:56:57.803 INFO:teuthology.orchestra.run.smithi151.stdout:[] 2024-05-21T21:56:57.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.799+0000 7f37d2000700 1 -- 172.21.15.151:0/1604048473 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f37b40b2b10 msgr2=0x7f37b40b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:57.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.799+0000 7f37d2000700 1 --2- 172.21.15.151:0/1604048473 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f37b40b2b10 0x7f37b40b4fd0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f37c4009e60 tx=0x7f37c400bf60 comp rx=0 tx=0).stop 2024-05-21T21:56:57.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.799+0000 7f37d2000700 1 -- 172.21.15.151:0/1604048473 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37cc105650 msgr2=0x7f37cc1a18d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:56:57.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.799+0000 7f37d2000700 1 --2- 172.21.15.151:0/1604048473 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37cc105650 0x7f37cc1a18d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f37c000e9c0 tx=0x7f37c000ecd0 comp rx=0 tx=0).stop 2024-05-21T21:56:57.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.799+0000 7f37d2000700 1 -- 172.21.15.151:0/1604048473 shutdown_connections 2024-05-21T21:56:57.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.799+0000 7f37d2000700 1 --2- 172.21.15.151:0/1604048473 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f37b40b2b10 0x7f37b40b4fd0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:57.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.799+0000 7f37d2000700 1 --2- 172.21.15.151:0/1604048473 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f37cc10b650 0x7f37cc19ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:57.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.799+0000 7f37d2000700 1 --2- 172.21.15.151:0/1604048473 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37cc105650 0x7f37cc1a18d0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:57.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.799+0000 7f37d2000700 1 --2- 172.21.15.151:0/1604048473 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37cc106010 0x7f37cc1a1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:56:57.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.799+0000 7f37d2000700 1 -- 172.21.15.151:0/1604048473 >> 172.21.15.151:0/1604048473 conn(0x7f37cc101120 msgr2=0x7f37cc101660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:56:57.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.799+0000 7f37d2000700 1 -- 172.21.15.151:0/1604048473 shutdown_connections 2024-05-21T21:56:57.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:56:57.799+0000 7f37d2000700 1 -- 172.21.15.151:0/1604048473 wait complete. 2024-05-21T21:56:58.670 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-21T21:56:59.108 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:56:59.611 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:56:59 smithi151 ceph-mon[31623]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 853 B/s wr, 12 op/s 2024-05-21T21:56:59.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:56:59 smithi184 ceph-mon[36942]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 853 B/s wr, 12 op/s 2024-05-21T21:56:59.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:56:59 smithi164 ceph-mon[36269]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 853 B/s wr, 12 op/s 2024-05-21T21:57:01.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.324+0000 7ffa9ac00700 1 -- 172.21.15.151:0/1559903940 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffa9410fff0 msgr2=0x7ffa941123e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:01.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.324+0000 7ffa9ac00700 1 --2- 172.21.15.151:0/1559903940 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffa9410fff0 0x7ffa941123e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c009f90 tx=0x7ffa8c00b330 comp rx=0 tx=0).stop 2024-05-21T21:57:01.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.325+0000 7ffa9ac00700 1 -- 172.21.15.151:0/1559903940 shutdown_connections 2024-05-21T21:57:01.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.325+0000 7ffa9ac00700 1 --2- 172.21.15.151:0/1559903940 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffa9410fff0 0x7ffa941123e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:01.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.325+0000 7ffa9ac00700 1 --2- 172.21.15.151:0/1559903940 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffa9406b250 0x7ffa9406b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:01.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.325+0000 7ffa9ac00700 1 --2- 172.21.15.151:0/1559903940 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa9406bb70 0x7ffa9410fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:01.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.325+0000 7ffa9ac00700 1 -- 172.21.15.151:0/1559903940 >> 172.21.15.151:0/1559903940 conn(0x7ffa940793f0 msgr2=0x7ffa94079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:57:01.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.325+0000 7ffa9ac00700 1 -- 172.21.15.151:0/1559903940 shutdown_connections 2024-05-21T21:57:01.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.325+0000 7ffa9ac00700 1 -- 172.21.15.151:0/1559903940 wait complete. 2024-05-21T21:57:01.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.326+0000 7ffa9ac00700 1 Processor -- start 2024-05-21T21:57:01.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.326+0000 7ffa9ac00700 1 -- start start 2024-05-21T21:57:01.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.326+0000 7ffa9ac00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffa9406b250 0x7ffa9419bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:01.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.326+0000 7ffa9ac00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa9406bb70 0x7ffa9419c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:01.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.326+0000 7ffa9ac00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffa9410fff0 0x7ffa941a0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:01.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.326+0000 7ffa9ac00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ffa94115960 con 0x7ffa9406bb70 2024-05-21T21:57:01.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.326+0000 7ffa9ac00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ffa941157e0 con 0x7ffa9406b250 2024-05-21T21:57:01.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.326+0000 7ffa9ac00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ffa94115ae0 con 0x7ffa9410fff0 2024-05-21T21:57:01.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.327+0000 7ffa93400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa9406bb70 0x7ffa9419c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:01.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.327+0000 7ffa93e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffa9406b250 0x7ffa9419bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:01.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.327+0000 7ffa93400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa9406bb70 0x7ffa9419c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:33924/0 (socket says 172.21.15.151:33924) 2024-05-21T21:57:01.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.327+0000 7ffa93400700 1 -- 172.21.15.151:0/3480960524 learned_addr learned my addr 172.21.15.151:0/3480960524 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:57:01.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.327+0000 7ffa93e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffa9406b250 0x7ffa9419bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:38462/0 (socket says 172.21.15.151:38462) 2024-05-21T21:57:01.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.327+0000 7ffa98c00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffa9410fff0 0x7ffa941a0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:01.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.327+0000 7ffa93e00700 1 -- 172.21.15.151:0/3480960524 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffa9410fff0 msgr2=0x7ffa941a0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:01.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.327+0000 7ffa93e00700 1 --2- 172.21.15.151:0/3480960524 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffa9410fff0 0x7ffa941a0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:01.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.327+0000 7ffa93e00700 1 -- 172.21.15.151:0/3480960524 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa9406bb70 msgr2=0x7ffa9419c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:01.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.327+0000 7ffa93e00700 1 --2- 172.21.15.151:0/3480960524 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa9406bb70 0x7ffa9419c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:01.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.327+0000 7ffa93e00700 1 -- 172.21.15.151:0/3480960524 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa941a0e90 con 0x7ffa9406b250 2024-05-21T21:57:01.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.328+0000 7ffa93e00700 1 --2- 172.21.15.151:0/3480960524 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffa9406b250 0x7ffa9419bed0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ffa8400e9f0 tx=0x7ffa8400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:57:01.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.328+0000 7ffa90c00700 1 -- 172.21.15.151:0/3480960524 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa84004220 con 0x7ffa9406b250 2024-05-21T21:57:01.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.328+0000 7ffa90c00700 1 -- 172.21.15.151:0/3480960524 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa840043c0 con 0x7ffa9406b250 2024-05-21T21:57:01.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.328+0000 7ffa9ac00700 1 -- 172.21.15.151:0/3480960524 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa941a1180 con 0x7ffa9406b250 2024-05-21T21:57:01.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.328+0000 7ffa90c00700 1 -- 172.21.15.151:0/3480960524 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa84005020 con 0x7ffa9406b250 2024-05-21T21:57:01.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.328+0000 7ffa9ac00700 1 -- 172.21.15.151:0/3480960524 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa941a8a00 con 0x7ffa9406b250 2024-05-21T21:57:01.332 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.331+0000 7ffa90c00700 1 -- 172.21.15.151:0/3480960524 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 149886+0+0 (secure 0 0 0) 0x7ffa84004750 con 0x7ffa9406b250 2024-05-21T21:57:01.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.334+0000 7ffaa54a4b80 1 Processor -- start 2024-05-21T21:57:01.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.334+0000 7ffa90c00700 1 --2- 172.21.15.151:0/3480960524 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7ffa780b2b90 0x7ffa780b5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:01.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.334+0000 7ffaa54a4b80 1 -- start start 2024-05-21T21:57:01.336 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.334+0000 7ffa90c00700 1 -- 172.21.15.151:0/3480960524 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7ffa840e2760 con 0x7ffa9406b250 2024-05-21T21:57:01.336 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.334+0000 7ffa98c00700 1 --2- 172.21.15.151:0/3480960524 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7ffa780b2b90 0x7ffa780b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:01.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.335+0000 7ffaa54a4b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5572c8b97eb0 0x5572c8b98290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:01.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.335+0000 7ffaa54a4b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572c8b987d0 0x5572c8b92260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:01.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.335+0000 7ffaa54a4b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5572c8b92880 0x5572c8b92ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:01.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.335+0000 7ffaa54a4b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5572c8af0e40 con 0x5572c8b987d0 2024-05-21T21:57:01.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.335+0000 7ffaa54a4b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5572c8ae09e0 con 0x5572c8b97eb0 2024-05-21T21:57:01.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.335+0000 7ffaa54a4b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5572c8ae4a20 con 0x5572c8b92880 2024-05-21T21:57:01.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.335+0000 7ffa93e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5572c8b97eb0 0x5572c8b98290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:01.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.335+0000 7ffa93e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5572c8b97eb0 0x5572c8b98290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:38470/0 (socket says 172.21.15.151:38470) 2024-05-21T21:57:01.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.335+0000 7ffa93e00700 1 -- 172.21.15.151:0/975145695 learned_addr learned my addr 172.21.15.151:0/975145695 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:57:01.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.335+0000 7ffa93400700 1 --2- 172.21.15.151:0/975145695 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572c8b987d0 0x5572c8b92260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:01.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.335+0000 7ffa98c00700 1 --2- 172.21.15.151:0/975145695 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5572c8b92880 0x5572c8b92ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:01.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.335+0000 7ffa93e00700 1 -- 172.21.15.151:0/975145695 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5572c8b92880 msgr2=0x5572c8b92ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:01.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.335+0000 7ffa93e00700 1 --2- 172.21.15.151:0/975145695 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5572c8b92880 0x5572c8b92ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:01.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.335+0000 7ffa93e00700 1 -- 172.21.15.151:0/975145695 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572c8b987d0 msgr2=0x5572c8b92260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:01.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.335+0000 7ffa93e00700 1 --2- 172.21.15.151:0/975145695 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572c8b987d0 0x5572c8b92260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:01.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.335+0000 7ffa93e00700 1 -- 172.21.15.151:0/975145695 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572c8aa9420 con 0x5572c8b97eb0 2024-05-21T21:57:01.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.335+0000 7ffa98c00700 1 --2- 172.21.15.151:0/975145695 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5572c8b92880 0x5572c8b92ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T21:57:01.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.335+0000 7ffa93400700 1 --2- 172.21.15.151:0/975145695 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572c8b987d0 0x5572c8b92260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T21:57:01.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.336+0000 7ffa98c00700 1 --2- 172.21.15.151:0/3480960524 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7ffa780b2b90 0x7ffa780b5050 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c009da0 tx=0x7ffa8c009420 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:57:01.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.336+0000 7ffa93e00700 1 --2- 172.21.15.151:0/975145695 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5572c8b97eb0 0x5572c8b98290 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ffa840a9800 tx=0x7ffa840a8fe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f164d7be6190a250 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:57:01.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.336+0000 7ffa89600700 1 -- 172.21.15.151:0/975145695 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa8401c070 con 0x5572c8b97eb0 2024-05-21T21:57:01.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.336+0000 7ffa89600700 1 -- 172.21.15.151:0/975145695 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa840a9870 con 0x5572c8b97eb0 2024-05-21T21:57:01.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.336+0000 7ffa89600700 1 -- 172.21.15.151:0/975145695 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa840b2240 con 0x5572c8b97eb0 2024-05-21T21:57:01.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.337+0000 7ffaa54a4b80 1 -- 172.21.15.151:0/975145695 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5572c8b97eb0 msgr2=0x5572c8b98290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:01.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.337+0000 7ffaa54a4b80 1 --2- 172.21.15.151:0/975145695 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5572c8b97eb0 0x5572c8b98290 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ffa840a9800 tx=0x7ffa840a8fe0 comp rx=0 tx=0).stop 2024-05-21T21:57:01.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.337+0000 7ffaa54a4b80 1 -- 172.21.15.151:0/975145695 shutdown_connections 2024-05-21T21:57:01.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.337+0000 7ffaa54a4b80 1 --2- 172.21.15.151:0/975145695 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5572c8b97eb0 0x5572c8b98290 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:01.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.337+0000 7ffaa54a4b80 1 --2- 172.21.15.151:0/975145695 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5572c8b92880 0x5572c8b92ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:01.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.337+0000 7ffaa54a4b80 1 --2- 172.21.15.151:0/975145695 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572c8b987d0 0x5572c8b92260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:01.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.337+0000 7ffaa54a4b80 1 -- 172.21.15.151:0/975145695 >> 172.21.15.151:0/975145695 conn(0x5572c8ac0be0 msgr2=0x5572c8b8d010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:57:01.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.337+0000 7ffaa54a4b80 1 -- 172.21.15.151:0/975145695 shutdown_connections 2024-05-21T21:57:01.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.337+0000 7ffaa54a4b80 1 -- 172.21.15.151:0/975145695 wait complete. 2024-05-21T21:57:01.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.338+0000 7ffaa54a4b80 1 -- start start 2024-05-21T21:57:01.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.338+0000 7ffaa54a4b80 1 Processor -- start 2024-05-21T21:57:01.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.338+0000 7ffaa54a4b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5572c8b92880 0x5572c8b9e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:01.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.338+0000 7ffaa54a4b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5572c8b97eb0 0x5572c8b9b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:01.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.338+0000 7ffaa54a4b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572c8b987d0 0x5572c8b9be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:01.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.338+0000 7ffaa54a4b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5572c89e6fa0 con 0x5572c8b987d0 2024-05-21T21:57:01.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.338+0000 7ffaa54a4b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5572c8a81760 con 0x5572c8b97eb0 2024-05-21T21:57:01.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.338+0000 7ffaa54a4b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5572c8b9f1e0 con 0x5572c8b92880 2024-05-21T21:57:01.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.339+0000 7ffaa54a4b80 10 client.0 _ll_register_callbacks cb 0x5572c8a8ac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:57:01.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.339+0000 7ffa93400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5572c8b97eb0 0x5572c8b9b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:01.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.339+0000 7ffa93400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5572c8b97eb0 0x5572c8b9b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:38476/0 (socket says 172.21.15.151:38476) 2024-05-21T21:57:01.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.339+0000 7ffa93400700 1 -- 172.21.15.151:0/504026046 learned_addr learned my addr 172.21.15.151:0/504026046 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:57:01.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.339+0000 7ffa93e00700 1 --2- 172.21.15.151:0/504026046 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5572c8b92880 0x5572c8b9e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:01.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.339+0000 7ffa98c00700 1 --2- 172.21.15.151:0/504026046 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572c8b987d0 0x5572c8b9be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:01.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.339+0000 7ffa93400700 1 -- 172.21.15.151:0/504026046 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5572c8b92880 msgr2=0x5572c8b9e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:01.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.339+0000 7ffa93400700 1 --2- 172.21.15.151:0/504026046 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5572c8b92880 0x5572c8b9e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:01.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.339+0000 7ffa93400700 1 -- 172.21.15.151:0/504026046 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572c8b987d0 msgr2=0x5572c8b9be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:01.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.339+0000 7ffa93400700 1 --2- 172.21.15.151:0/504026046 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572c8b987d0 0x5572c8b9be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:01.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.339+0000 7ffa93400700 1 -- 172.21.15.151:0/504026046 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572c8a308d0 con 0x5572c8b97eb0 2024-05-21T21:57:01.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.340+0000 7ffa93400700 1 --2- 172.21.15.151:0/504026046 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5572c8b97eb0 0x5572c8b9b950 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ffa80012790 tx=0x7ffa80012aa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e600e3aaf0d1156 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:57:01.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.340+0000 7ffa7e000700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-21T21:57:01.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.340+0000 7ffa7e000700 1 -- 172.21.15.151:0/504026046 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa8000f4e0 con 0x5572c8b97eb0 2024-05-21T21:57:01.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.340+0000 7ffaa54a4b80 1 -- 172.21.15.151:0/504026046 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa840a9870 con 0x5572c8b97eb0 2024-05-21T21:57:01.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.340+0000 7ffa7e000700 1 -- 172.21.15.151:0/504026046 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7ffa8000f680 con 0x5572c8b97eb0 2024-05-21T21:57:01.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.340+0000 7ffa7e000700 1 -- 172.21.15.151:0/504026046 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ffa8000f880 con 0x5572c8b97eb0 2024-05-21T21:57:01.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.340+0000 7ffaa54a4b80 10 client.25559 fetch_fsmap learned FSMap version 29 2024-05-21T21:57:01.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.340+0000 7ffaa54a4b80 1 -- 172.21.15.151:0/504026046 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5572c8ab1060 con 0x5572c8b97eb0 2024-05-21T21:57:01.345 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.341+0000 7ffa7e000700 1 -- 172.21.15.151:0/504026046 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa840a9870 con 0x5572c8b97eb0 2024-05-21T21:57:01.345 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.341+0000 7ffa7e000700 1 -- 172.21.15.151:0/504026046 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7ffa80010af0 con 0x5572c8b97eb0 2024-05-21T21:57:01.346 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.341+0000 7ffaa54a4b80 10 client.25559 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T21:57:01.346 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.341+0000 7ffaa54a4b80 10 client.25559 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T21:57:01.346 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.341+0000 7ffaa54a4b80 20 client.25559 populate_metadata read hostname 'smithi151' 2024-05-21T21:57:01.346 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.341+0000 7ffaa54a4b80 1 --2- 172.21.15.151:0/504026046 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5572c8babdd0 0x5572c8bae1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:01.346 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.341+0000 7ffaa54a4b80 4 client.25559 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T21:57:01.346 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.341+0000 7ffaa54a4b80 1 -- 172.21.15.151:0/504026046 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5572c8af2d90 con 0x5572c8babdd0 2024-05-21T21:57:01.346 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.341+0000 7ffa93e00700 1 --2- 172.21.15.151:0/504026046 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5572c8babdd0 0x5572c8bae1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:01.346 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.342+0000 7ffa93e00700 1 --2- 172.21.15.151:0/504026046 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5572c8babdd0 0x5572c8bae1f0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29f875a546061f0b server_cookie=9429c054b17f7b18 in_seq=0 out_seq=0 2024-05-21T21:57:01.346 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.342+0000 7ffa7e000700 10 client.25559 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T21:57:01.346 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.343+0000 7ffa7e000700 1 -- 172.21.15.151:0/504026046 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5572c8af2d90 con 0x5572c8babdd0 2024-05-21T21:57:01.346 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.343+0000 7ffa7e000700 10 client.25559 handle_command_reply: tid=0 2024-05-21T21:57:01.346 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.343+0000 7ffaa54a4b80 1 client.25559 shutdown 2024-05-21T21:57:01.346 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.343+0000 7ffaa54a4b80 1 -- 172.21.15.151:0/504026046 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5572c8b97eb0 msgr2=0x5572c8b9b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:01.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.343+0000 7ffaa54a4b80 1 --2- 172.21.15.151:0/504026046 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5572c8b97eb0 0x5572c8b9b950 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ffa80012790 tx=0x7ffa80012aa0 comp rx=0 tx=0).stop 2024-05-21T21:57:01.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.343+0000 7ffaa54a4b80 1 -- 172.21.15.151:0/504026046 shutdown_connections 2024-05-21T21:57:01.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.343+0000 7ffaa54a4b80 1 --2- 172.21.15.151:0/504026046 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572c8b987d0 0x5572c8b9be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:01.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.343+0000 7ffaa54a4b80 1 --2- 172.21.15.151:0/504026046 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5572c8b92880 0x5572c8b9e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:01.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.343+0000 7ffaa54a4b80 1 --2- 172.21.15.151:0/504026046 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5572c8babdd0 0x5572c8bae1f0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:01.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.343+0000 7ffaa54a4b80 1 --2- 172.21.15.151:0/504026046 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5572c8b97eb0 0x5572c8b9b950 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:01.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.343+0000 7ffa7e000700 0 client.25559 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T21:57:01.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.343+0000 7ffaa54a4b80 1 -- 172.21.15.151:0/504026046 >> 172.21.15.151:0/504026046 conn(0x5572c8ac0be0 msgr2=0x5572c8b91250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:57:01.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.343+0000 7ffaa54a4b80 1 -- 172.21.15.151:0/504026046 shutdown_connections 2024-05-21T21:57:01.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.343+0000 7ffaa54a4b80 1 -- 172.21.15.151:0/504026046 wait complete. 2024-05-21T21:57:01.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.343+0000 7ffaa54a4b80 20 client.25559 trim_cache size 0 max 16384 2024-05-21T21:57:01.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.360+0000 7ffaa54a4b80 1 Processor -- start 2024-05-21T21:57:01.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.360+0000 7ffaa54a4b80 1 -- start start 2024-05-21T21:57:01.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.360+0000 7ffaa54a4b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572c8b92880 0x5572c8b92260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:01.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.360+0000 7ffaa54a4b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5572c8b97eb0 0x5572c8b92c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:01.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.360+0000 7ffaa54a4b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5572c8b987d0 0x5572c8b9a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:01.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.360+0000 7ffaa54a4b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ffa941b0090 con 0x5572c8b92880 2024-05-21T21:57:01.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.360+0000 7ffaa54a4b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ffa941b1bd0 con 0x5572c8b97eb0 2024-05-21T21:57:01.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.360+0000 7ffaa54a4b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ffa941b0760 con 0x5572c8b987d0 2024-05-21T21:57:01.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.360+0000 7ffa93e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572c8b92880 0x5572c8b92260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:01.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.360+0000 7ffa93e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572c8b92880 0x5572c8b92260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:33942/0 (socket says 172.21.15.151:33942) 2024-05-21T21:57:01.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.360+0000 7ffa93e00700 1 -- 172.21.15.151:0/176245570 learned_addr learned my addr 172.21.15.151:0/176245570 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:57:01.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.361+0000 7ffa93400700 1 --2- 172.21.15.151:0/176245570 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5572c8b97eb0 0x5572c8b92c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:01.363 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.361+0000 7ffa98c00700 1 --2- 172.21.15.151:0/176245570 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5572c8b987d0 0x5572c8b9a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:01.364 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.361+0000 7ffa93e00700 1 -- 172.21.15.151:0/176245570 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5572c8b987d0 msgr2=0x5572c8b9a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:01.364 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.361+0000 7ffa93e00700 1 --2- 172.21.15.151:0/176245570 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5572c8b987d0 0x5572c8b9a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:01.364 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.361+0000 7ffa93e00700 1 -- 172.21.15.151:0/176245570 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5572c8b97eb0 msgr2=0x5572c8b92c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:01.364 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.361+0000 7ffa93e00700 1 --2- 172.21.15.151:0/176245570 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5572c8b97eb0 0x5572c8b92c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:01.364 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.361+0000 7ffa93e00700 1 -- 172.21.15.151:0/176245570 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572c8b99ec0 con 0x5572c8b92880 2024-05-21T21:57:01.364 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.361+0000 7ffa93400700 1 --2- 172.21.15.151:0/176245570 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5572c8b97eb0 0x5572c8b92c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:57:01.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.361+0000 7ffa93e00700 1 --2- 172.21.15.151:0/176245570 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572c8b92880 0x5572c8b92260 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ffa8400c800 tx=0x7ffa84091040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=915fdeca14ba250d server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:57:01.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.361+0000 7ffa88c00700 1 -- 172.21.15.151:0/176245570 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa8401c040 con 0x5572c8b92880 2024-05-21T21:57:01.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.361+0000 7ffa88c00700 1 -- 172.21.15.151:0/176245570 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa840b26d0 con 0x5572c8b92880 2024-05-21T21:57:01.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.362+0000 7ffa88c00700 1 -- 172.21.15.151:0/176245570 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa840b24a0 con 0x5572c8b92880 2024-05-21T21:57:01.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.362+0000 7ffaa54a4b80 1 -- 172.21.15.151:0/176245570 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572c8b92880 msgr2=0x5572c8b92260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:01.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.362+0000 7ffaa54a4b80 1 --2- 172.21.15.151:0/176245570 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572c8b92880 0x5572c8b92260 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ffa8400c800 tx=0x7ffa84091040 comp rx=0 tx=0).stop 2024-05-21T21:57:01.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.362+0000 7ffaa54a4b80 1 -- 172.21.15.151:0/176245570 shutdown_connections 2024-05-21T21:57:01.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.362+0000 7ffaa54a4b80 1 --2- 172.21.15.151:0/176245570 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5572c8b987d0 0x5572c8b9a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:01.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.362+0000 7ffaa54a4b80 1 --2- 172.21.15.151:0/176245570 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572c8b92880 0x5572c8b92260 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:01.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.362+0000 7ffaa54a4b80 1 --2- 172.21.15.151:0/176245570 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5572c8b97eb0 0x5572c8b92c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:01.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.362+0000 7ffaa54a4b80 1 -- 172.21.15.151:0/176245570 >> 172.21.15.151:0/176245570 conn(0x5572c8ac0be0 msgr2=0x5572c8b86e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:57:01.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.362+0000 7ffaa54a4b80 1 -- 172.21.15.151:0/176245570 shutdown_connections 2024-05-21T21:57:01.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.362+0000 7ffaa54a4b80 1 -- 172.21.15.151:0/176245570 wait complete. 2024-05-21T21:57:01.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.363+0000 7ffaa54a4b80 1 -- start start 2024-05-21T21:57:01.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.363+0000 7ffaa54a4b80 1 Processor -- start 2024-05-21T21:57:01.368 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.364+0000 7ffaa54a4b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5572c8b92880 0x5572c8ba4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:01.368 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.364+0000 7ffaa54a4b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572c8b97eb0 0x5572c8bb3250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:01.368 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.364+0000 7ffaa54a4b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5572c8b987d0 0x5572c8bb3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:01.368 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.364+0000 7ffaa54a4b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ffa941b1500 con 0x5572c8b97eb0 2024-05-21T21:57:01.368 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.364+0000 7ffaa54a4b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ffa941b0e30 con 0x5572c8b92880 2024-05-21T21:57:01.368 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.364+0000 7ffaa54a4b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ffa941ae030 con 0x5572c8b987d0 2024-05-21T21:57:01.368 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.364+0000 7ffaa54a4b80 10 client.0 _ll_register_callbacks cb 0x5572c8a8ac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:57:01.368 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.364+0000 7ffa93400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572c8b97eb0 0x5572c8bb3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:01.368 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.364+0000 7ffa93e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5572c8b92880 0x5572c8ba4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:01.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.364+0000 7ffa93e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5572c8b92880 0x5572c8ba4a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:38486/0 (socket says 172.21.15.151:38486) 2024-05-21T21:57:01.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.364+0000 7ffa93e00700 1 -- 172.21.15.151:0/2039447957 learned_addr learned my addr 172.21.15.151:0/2039447957 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:57:01.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.364+0000 7ffa93400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572c8b97eb0 0x5572c8bb3250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:33958/0 (socket says 172.21.15.151:33958) 2024-05-21T21:57:01.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.364+0000 7ffa98c00700 1 --2- 172.21.15.151:0/2039447957 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5572c8b987d0 0x5572c8bb3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:01.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.364+0000 7ffa93400700 1 -- 172.21.15.151:0/2039447957 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5572c8b987d0 msgr2=0x5572c8bb3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:01.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.364+0000 7ffa93400700 1 --2- 172.21.15.151:0/2039447957 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5572c8b987d0 0x5572c8bb3790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:01.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.364+0000 7ffa93400700 1 -- 172.21.15.151:0/2039447957 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5572c8b92880 msgr2=0x5572c8ba4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:01.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.364+0000 7ffa93400700 1 --2- 172.21.15.151:0/2039447957 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5572c8b92880 0x5572c8ba4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:01.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.364+0000 7ffa93400700 1 -- 172.21.15.151:0/2039447957 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572c8bb3ed0 con 0x5572c8b97eb0 2024-05-21T21:57:01.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.364+0000 7ffa93e00700 1 --2- 172.21.15.151:0/2039447957 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5572c8b92880 0x5572c8ba4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:57:01.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.364+0000 7ffa98c00700 1 --2- 172.21.15.151:0/2039447957 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5572c8b987d0 0x5572c8bb3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T21:57:01.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.365+0000 7ffa93400700 1 --2- 172.21.15.151:0/2039447957 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572c8b97eb0 0x5572c8bb3250 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ffa80010f30 tx=0x7ffa8000f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b13c8a55365c24d server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:57:01.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.365+0000 7ffa7e000700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-21T21:57:01.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.365+0000 7ffa7e000700 1 -- 172.21.15.151:0/2039447957 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa80031060 con 0x5572c8b97eb0 2024-05-21T21:57:01.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.365+0000 7ffa7e000700 1 -- 172.21.15.151:0/2039447957 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7ffa8000fc30 con 0x5572c8b97eb0 2024-05-21T21:57:01.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.365+0000 7ffaa54a4b80 1 -- 172.21.15.151:0/2039447957 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa840b26d0 con 0x5572c8b97eb0 2024-05-21T21:57:01.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.365+0000 7ffa7e000700 1 -- 172.21.15.151:0/2039447957 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ffa8000fe50 con 0x5572c8b97eb0 2024-05-21T21:57:01.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.365+0000 7ffaa54a4b80 10 client.15453 fetch_fsmap learned FSMap version 29 2024-05-21T21:57:01.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.365+0000 7ffaa54a4b80 1 -- 172.21.15.151:0/2039447957 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5572c8c168e0 con 0x5572c8b97eb0 2024-05-21T21:57:01.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.366+0000 7ffa7e000700 1 -- 172.21.15.151:0/2039447957 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa840b26d0 con 0x5572c8b97eb0 2024-05-21T21:57:01.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.366+0000 7ffa7e000700 1 -- 172.21.15.151:0/2039447957 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7ffa80005420 con 0x5572c8b97eb0 2024-05-21T21:57:01.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.366+0000 7ffaa54a4b80 10 client.15453 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T21:57:01.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.366+0000 7ffaa54a4b80 10 client.15453 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T21:57:01.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.366+0000 7ffaa54a4b80 20 client.15453 populate_metadata read hostname 'smithi151' 2024-05-21T21:57:01.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.366+0000 7ffaa54a4b80 1 --2- 172.21.15.151:0/2039447957 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5572c8c16ff0 0x5572c8c19410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:01.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.366+0000 7ffaa54a4b80 4 client.15453 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T21:57:01.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.366+0000 7ffaa54a4b80 1 -- 172.21.15.151:0/2039447957 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5572c8aec570 con 0x5572c8c16ff0 2024-05-21T21:57:01.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.366+0000 7ffa93e00700 1 --2- 172.21.15.151:0/2039447957 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5572c8c16ff0 0x5572c8c19410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:01.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.367+0000 7ffa93e00700 1 --2- 172.21.15.151:0/2039447957 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5572c8c16ff0 0x5572c8c19410 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=157853f4c7befb7f server_cookie=43d6ed09976dad6e in_seq=0 out_seq=0 2024-05-21T21:57:01.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.367+0000 7ffa7e000700 10 client.15453 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T21:57:01.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.367+0000 7ffa7e000700 1 -- 172.21.15.151:0/2039447957 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5572c8aec570 con 0x5572c8c16ff0 2024-05-21T21:57:01.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.367+0000 7ffa7e000700 10 client.15453 handle_command_reply: tid=0 2024-05-21T21:57:01.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.367+0000 7ffaa54a4b80 1 client.15453 shutdown 2024-05-21T21:57:01.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.367+0000 7ffaa54a4b80 1 -- 172.21.15.151:0/2039447957 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572c8b97eb0 msgr2=0x5572c8bb3250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:01.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.367+0000 7ffaa54a4b80 1 --2- 172.21.15.151:0/2039447957 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572c8b97eb0 0x5572c8bb3250 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ffa80010f30 tx=0x7ffa8000f710 comp rx=0 tx=0).stop 2024-05-21T21:57:01.372 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.367+0000 7ffaa54a4b80 1 -- 172.21.15.151:0/2039447957 shutdown_connections 2024-05-21T21:57:01.372 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.367+0000 7ffaa54a4b80 1 --2- 172.21.15.151:0/2039447957 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5572c8c16ff0 0x5572c8c19410 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:01.372 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.367+0000 7ffa7e000700 0 client.15453 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T21:57:01.372 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.367+0000 7ffaa54a4b80 1 --2- 172.21.15.151:0/2039447957 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5572c8b92880 0x5572c8ba4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:01.372 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.367+0000 7ffaa54a4b80 1 --2- 172.21.15.151:0/2039447957 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5572c8b987d0 0x5572c8bb3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:01.372 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.367+0000 7ffaa54a4b80 1 --2- 172.21.15.151:0/2039447957 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572c8b97eb0 0x5572c8bb3250 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:01.372 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.367+0000 7ffaa54a4b80 1 -- 172.21.15.151:0/2039447957 >> 172.21.15.151:0/2039447957 conn(0x5572c8ac0be0 msgr2=0x5572c8b9a200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:57:01.372 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.367+0000 7ffaa54a4b80 1 -- 172.21.15.151:0/2039447957 shutdown_connections 2024-05-21T21:57:01.373 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.368+0000 7ffaa54a4b80 1 -- 172.21.15.151:0/2039447957 wait complete. 2024-05-21T21:57:01.373 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.368+0000 7ffaa54a4b80 20 client.15453 trim_cache size 0 max 16384 2024-05-21T21:57:01.373 INFO:teuthology.orchestra.run.smithi151.stdout:[] 2024-05-21T21:57:01.373 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.368+0000 7ffa9ac00700 1 -- 172.21.15.151:0/3480960524 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7ffa780b2b90 msgr2=0x7ffa780b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:01.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.368+0000 7ffa9ac00700 1 --2- 172.21.15.151:0/3480960524 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7ffa780b2b90 0x7ffa780b5050 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c009da0 tx=0x7ffa8c009420 comp rx=0 tx=0).stop 2024-05-21T21:57:01.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.368+0000 7ffa9ac00700 1 -- 172.21.15.151:0/3480960524 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffa9406b250 msgr2=0x7ffa9419bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:01.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.368+0000 7ffa9ac00700 1 --2- 172.21.15.151:0/3480960524 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffa9406b250 0x7ffa9419bed0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ffa8400e9f0 tx=0x7ffa8400ed00 comp rx=0 tx=0).stop 2024-05-21T21:57:01.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.368+0000 7ffa9ac00700 1 -- 172.21.15.151:0/3480960524 shutdown_connections 2024-05-21T21:57:01.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.368+0000 7ffa9ac00700 1 --2- 172.21.15.151:0/3480960524 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7ffa780b2b90 0x7ffa780b5050 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:01.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.368+0000 7ffa9ac00700 1 --2- 172.21.15.151:0/3480960524 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffa9406b250 0x7ffa9419bed0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:01.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.368+0000 7ffa9ac00700 1 --2- 172.21.15.151:0/3480960524 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffa9410fff0 0x7ffa941a0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:01.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.368+0000 7ffa9ac00700 1 --2- 172.21.15.151:0/3480960524 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa9406bb70 0x7ffa9419c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:01.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.368+0000 7ffa9ac00700 1 -- 172.21.15.151:0/3480960524 >> 172.21.15.151:0/3480960524 conn(0x7ffa940793f0 msgr2=0x7ffa941114e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:57:01.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.369+0000 7ffa9ac00700 1 -- 172.21.15.151:0/3480960524 shutdown_connections 2024-05-21T21:57:01.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:01.369+0000 7ffa9ac00700 1 -- 172.21.15.151:0/3480960524 wait complete. 2024-05-21T21:57:01.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:57:01 smithi151 ceph-mon[31623]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 853 B/s wr, 8 op/s 2024-05-21T21:57:01.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:57:01 smithi184 ceph-mon[36942]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 853 B/s wr, 8 op/s 2024-05-21T21:57:01.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:57:01 smithi164 ceph-mon[36269]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 853 B/s wr, 8 op/s 2024-05-21T21:57:03.222 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-21T21:57:03.223 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-21T21:57:03.676 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:57:03.702 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:57:03 smithi151 ceph-mon[31623]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 27 op/s 2024-05-21T21:57:03.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:57:03 smithi184 ceph-mon[36942]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 27 op/s 2024-05-21T21:57:03.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:57:03 smithi164 ceph-mon[36269]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 27 op/s 2024-05-21T21:57:05.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:57:05 smithi151 ceph-mon[31623]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-05-21T21:57:05.675 INFO:tasks.quiescer.fs.[cephfs]:Total duration (52.3) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-05-21T21:57:05.675 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 26.15 2024-05-21T21:57:05.676 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-21T21:57:05.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:57:05 smithi184 ceph-mon[36942]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-05-21T21:57:05.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:57:05 smithi164 ceph-mon[36269]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-05-21T21:57:06.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.007+0000 7ff35d000700 1 -- 172.21.15.151:0/2013603598 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff358103490 msgr2=0x7ff358103910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:06.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.007+0000 7ff35d000700 1 --2- 172.21.15.151:0/2013603598 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff358103490 0x7ff358103910 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff344009990 tx=0x7ff344009ca0 comp rx=0 tx=0).stop 2024-05-21T21:57:06.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.007+0000 7ff35d000700 1 -- 172.21.15.151:0/2013603598 shutdown_connections 2024-05-21T21:57:06.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.007+0000 7ff35d000700 1 --2- 172.21.15.151:0/2013603598 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff358109540 0x7ff358109920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:06.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.007+0000 7ff35d000700 1 --2- 172.21.15.151:0/2013603598 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff358103e50 0x7ff358110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:06.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.007+0000 7ff35d000700 1 --2- 172.21.15.151:0/2013603598 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff358103490 0x7ff358103910 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:06.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.007+0000 7ff35d000700 1 -- 172.21.15.151:0/2013603598 >> 172.21.15.151:0/2013603598 conn(0x7ff3580fefc0 msgr2=0x7ff3581013e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:57:06.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.007+0000 7ff35d000700 1 -- 172.21.15.151:0/2013603598 shutdown_connections 2024-05-21T21:57:06.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.007+0000 7ff35d000700 1 -- 172.21.15.151:0/2013603598 wait complete. 2024-05-21T21:57:06.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.009+0000 7ff35d000700 1 Processor -- start 2024-05-21T21:57:06.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.009+0000 7ff35d000700 1 -- start start 2024-05-21T21:57:06.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.009+0000 7ff35d000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff358103490 0x7ff3581a1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:06.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.009+0000 7ff35d000700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff358103e50 0x7ff3581a1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:06.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.010+0000 7ff35d000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff358109540 0x7ff35819bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:06.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.010+0000 7ff35d000700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff358113640 con 0x7ff358109540 2024-05-21T21:57:06.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.010+0000 7ff35d000700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff3581134c0 con 0x7ff358103490 2024-05-21T21:57:06.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.010+0000 7ff35d000700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff3581137c0 con 0x7ff358103e50 2024-05-21T21:57:06.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.010+0000 7ff357400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff358103490 0x7ff3581a1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:06.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.010+0000 7ff356a00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff358103e50 0x7ff3581a1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:06.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.010+0000 7ff357400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff358103490 0x7ff3581a1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:32962/0 (socket says 172.21.15.151:32962) 2024-05-21T21:57:06.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.010+0000 7ff357400700 1 -- 172.21.15.151:0/1849828787 learned_addr learned my addr 172.21.15.151:0/1849828787 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:57:06.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.010+0000 7ff357e00700 1 --2- 172.21.15.151:0/1849828787 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff358109540 0x7ff35819bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:06.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.010+0000 7ff357400700 1 -- 172.21.15.151:0/1849828787 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff358103e50 msgr2=0x7ff3581a1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:06.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.010+0000 7ff357400700 1 --2- 172.21.15.151:0/1849828787 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff358103e50 0x7ff3581a1f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:06.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.010+0000 7ff357400700 1 -- 172.21.15.151:0/1849828787 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff358109540 msgr2=0x7ff35819bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:06.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.010+0000 7ff357400700 1 --2- 172.21.15.151:0/1849828787 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff358109540 0x7ff35819bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:06.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.010+0000 7ff357400700 1 -- 172.21.15.151:0/1849828787 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff35819c400 con 0x7ff358103490 2024-05-21T21:57:06.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.010+0000 7ff356a00700 1 --2- 172.21.15.151:0/1849828787 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff358103e50 0x7ff3581a1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T21:57:06.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.011+0000 7ff357400700 1 --2- 172.21.15.151:0/1849828787 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff358103490 0x7ff3581a1a50 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff34800e9c0 tx=0x7ff34800ed80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:57:06.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.011+0000 7ff34fe00700 1 -- 172.21.15.151:0/1849828787 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff34800cbd0 con 0x7ff358103490 2024-05-21T21:57:06.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.011+0000 7ff34fe00700 1 -- 172.21.15.151:0/1849828787 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff34800cd70 con 0x7ff358103490 2024-05-21T21:57:06.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.011+0000 7ff35d000700 1 -- 172.21.15.151:0/1849828787 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff35819c6f0 con 0x7ff358103490 2024-05-21T21:57:06.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.011+0000 7ff34fe00700 1 -- 172.21.15.151:0/1849828787 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff348010430 con 0x7ff358103490 2024-05-21T21:57:06.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.011+0000 7ff35d000700 1 -- 172.21.15.151:0/1849828787 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3581a89d0 con 0x7ff358103490 2024-05-21T21:57:06.017 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.016+0000 7ff36905db80 1 Processor -- start 2024-05-21T21:57:06.017 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.016+0000 7ff36905db80 1 -- start start 2024-05-21T21:57:06.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.017+0000 7ff36905db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ad6fee00a0 0x55ad6fee0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:06.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.017+0000 7ff36905db80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ad6fee09c0 0x55ad6feda530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:06.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.017+0000 7ff36905db80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ad6fedab50 0x55ad6fedafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:06.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.017+0000 7ff36905db80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ad6fe38ec0 con 0x55ad6fee09c0 2024-05-21T21:57:06.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.017+0000 7ff36905db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55ad6fe28a60 con 0x55ad6fee00a0 2024-05-21T21:57:06.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.017+0000 7ff36905db80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55ad6fe2caa0 con 0x55ad6fedab50 2024-05-21T21:57:06.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.017+0000 7ff34fe00700 1 -- 172.21.15.151:0/1849828787 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 26) ==== 149886+0+0 (secure 0 0 0) 0x7ff3480105d0 con 0x7ff358103490 2024-05-21T21:57:06.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.017+0000 7ff357e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ad6fee00a0 0x55ad6fee0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:06.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.017+0000 7ff356a00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ad6fedab50 0x55ad6fedafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:06.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.017+0000 7ff357e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ad6fee00a0 0x55ad6fee0480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:32968/0 (socket says 172.21.15.151:32968) 2024-05-21T21:57:06.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.017+0000 7ff357e00700 1 -- 172.21.15.151:0/591467065 learned_addr learned my addr 172.21.15.151:0/591467065 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:57:06.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.018+0000 7ff357400700 1 --2- 172.21.15.151:0/591467065 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ad6fee09c0 0x55ad6feda530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:06.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.018+0000 7ff357e00700 1 -- 172.21.15.151:0/591467065 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ad6fedab50 msgr2=0x55ad6fedafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:06.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.018+0000 7ff357e00700 1 --2- 172.21.15.151:0/591467065 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ad6fedab50 0x55ad6fedafb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:06.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.018+0000 7ff357e00700 1 -- 172.21.15.151:0/591467065 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ad6fee09c0 msgr2=0x55ad6feda530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:06.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.018+0000 7ff357e00700 1 --2- 172.21.15.151:0/591467065 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ad6fee09c0 0x55ad6feda530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:06.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.018+0000 7ff357e00700 1 -- 172.21.15.151:0/591467065 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad6fdf14a0 con 0x55ad6fee00a0 2024-05-21T21:57:06.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.018+0000 7ff357400700 1 --2- 172.21.15.151:0/591467065 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ad6fee09c0 0x55ad6feda530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T21:57:06.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.018+0000 7ff356a00700 1 --2- 172.21.15.151:0/591467065 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ad6fedab50 0x55ad6fedafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:57:06.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.018+0000 7ff357e00700 1 --2- 172.21.15.151:0/591467065 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ad6fee00a0 0x55ad6fee0480 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff350009440 tx=0x7ff350009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c73d52ce791cf80d server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:57:06.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.019+0000 7ff34cc00700 1 -- 172.21.15.151:0/591467065 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff350013810 con 0x55ad6fee00a0 2024-05-21T21:57:06.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.019+0000 7ff34cc00700 1 -- 172.21.15.151:0/591467065 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff350015440 con 0x55ad6fee00a0 2024-05-21T21:57:06.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.019+0000 7ff34cc00700 1 -- 172.21.15.151:0/591467065 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff350013e60 con 0x55ad6fee00a0 2024-05-21T21:57:06.023 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.020+0000 7ff34fe00700 1 --2- 172.21.15.151:0/1849828787 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7ff33c0b2b10 0x7ff33c0b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:06.023 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.020+0000 7ff34fe00700 1 -- 172.21.15.151:0/1849828787 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7ff3480e2b00 con 0x7ff358103490 2024-05-21T21:57:06.023 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.020+0000 7ff357400700 1 --2- 172.21.15.151:0/1849828787 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7ff33c0b2b10 0x7ff33c0b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:06.023 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.020+0000 7ff36905db80 1 -- 172.21.15.151:0/591467065 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ad6fee00a0 msgr2=0x55ad6fee0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:06.023 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.020+0000 7ff36905db80 1 --2- 172.21.15.151:0/591467065 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ad6fee00a0 0x55ad6fee0480 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff350009440 tx=0x7ff350009920 comp rx=0 tx=0).stop 2024-05-21T21:57:06.023 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.020+0000 7ff36905db80 1 -- 172.21.15.151:0/591467065 shutdown_connections 2024-05-21T21:57:06.023 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.020+0000 7ff36905db80 1 --2- 172.21.15.151:0/591467065 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ad6fee00a0 0x55ad6fee0480 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:06.023 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.020+0000 7ff36905db80 1 --2- 172.21.15.151:0/591467065 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ad6fedab50 0x55ad6fedafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:06.023 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.020+0000 7ff36905db80 1 --2- 172.21.15.151:0/591467065 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ad6fee09c0 0x55ad6feda530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:06.023 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.020+0000 7ff36905db80 1 -- 172.21.15.151:0/591467065 >> 172.21.15.151:0/591467065 conn(0x55ad6fe08c60 msgr2=0x55ad6fed51d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:57:06.023 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.020+0000 7ff36905db80 1 -- 172.21.15.151:0/591467065 shutdown_connections 2024-05-21T21:57:06.024 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.020+0000 7ff36905db80 1 -- 172.21.15.151:0/591467065 wait complete. 2024-05-21T21:57:06.024 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.021+0000 7ff36905db80 1 -- start start 2024-05-21T21:57:06.024 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.021+0000 7ff357400700 1 --2- 172.21.15.151:0/1849828787 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7ff33c0b2b10 0x7ff33c0b4fd0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff3480e0140 tx=0x7ff348017f80 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:57:06.025 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.022+0000 7ff36905db80 1 Processor -- start 2024-05-21T21:57:06.026 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.022+0000 7ff36905db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ad6fedab50 0x55ad6fee6600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:06.026 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.022+0000 7ff36905db80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ad6fee00a0 0x55ad6fee39a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:06.026 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.022+0000 7ff36905db80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ad6fee09c0 0x55ad6fee3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:06.026 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.022+0000 7ff36905db80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ad6fd2f020 con 0x55ad6fee09c0 2024-05-21T21:57:06.026 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.022+0000 7ff36905db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55ad6fdc97e0 con 0x55ad6fedab50 2024-05-21T21:57:06.026 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.022+0000 7ff36905db80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55ad6fee74e0 con 0x55ad6fee00a0 2024-05-21T21:57:06.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.022+0000 7ff36905db80 10 client.0 _ll_register_callbacks cb 0x55ad6fdd2cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:57:06.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.023+0000 7ff357e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ad6fee09c0 0x55ad6fee3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:06.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.023+0000 7ff356a00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ad6fee00a0 0x55ad6fee39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:06.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.023+0000 7ff357e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ad6fee09c0 0x55ad6fee3ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:47056/0 (socket says 172.21.15.151:47056) 2024-05-21T21:57:06.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.023+0000 7ff357e00700 1 -- 172.21.15.151:0/942850655 learned_addr learned my addr 172.21.15.151:0/942850655 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:57:06.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.023+0000 7ff357e00700 1 --2- 172.21.15.151:0/942850655 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ad6fedab50 0x55ad6fee6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:06.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.023+0000 7ff356a00700 1 -- 172.21.15.151:0/942850655 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ad6fedab50 msgr2=0x55ad6fee6600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:06.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.023+0000 7ff356a00700 1 --2- 172.21.15.151:0/942850655 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ad6fedab50 0x55ad6fee6600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:06.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.023+0000 7ff356a00700 1 -- 172.21.15.151:0/942850655 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ad6fee09c0 msgr2=0x55ad6fee3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:06.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.024+0000 7ff356a00700 1 --2- 172.21.15.151:0/942850655 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ad6fee09c0 0x55ad6fee3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:06.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.024+0000 7ff356a00700 1 -- 172.21.15.151:0/942850655 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad6fd78950 con 0x55ad6fee00a0 2024-05-21T21:57:06.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.024+0000 7ff356a00700 1 --2- 172.21.15.151:0/942850655 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ad6fee00a0 0x55ad6fee39a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff35819d300 tx=0x7ff34400bf00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b5bfec91bfa3778 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:57:06.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.024+0000 7ff341600700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-21T21:57:06.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.024+0000 7ff341600700 1 -- 172.21.15.151:0/942850655 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff344027030 con 0x55ad6fee00a0 2024-05-21T21:57:06.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.024+0000 7ff341600700 1 -- 172.21.15.151:0/942850655 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7ff34400f8c0 con 0x55ad6fee00a0 2024-05-21T21:57:06.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.025+0000 7ff36905db80 1 -- 172.21.15.151:0/942850655 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff350015440 con 0x55ad6fee00a0 2024-05-21T21:57:06.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.025+0000 7ff341600700 1 -- 172.21.15.151:0/942850655 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff34400b610 con 0x55ad6fee00a0 2024-05-21T21:57:06.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.025+0000 7ff36905db80 10 client.25591 fetch_fsmap learned FSMap version 29 2024-05-21T21:57:06.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.025+0000 7ff36905db80 1 -- 172.21.15.151:0/942850655 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ad6fdf90e0 con 0x55ad6fee00a0 2024-05-21T21:57:06.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.025+0000 7ff341600700 1 -- 172.21.15.151:0/942850655 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff350015440 con 0x55ad6fee00a0 2024-05-21T21:57:06.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.026+0000 7ff341600700 1 -- 172.21.15.151:0/942850655 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7ff344023cb0 con 0x55ad6fee00a0 2024-05-21T21:57:06.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.026+0000 7ff36905db80 10 client.25591 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T21:57:06.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.026+0000 7ff36905db80 10 client.25591 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T21:57:06.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.026+0000 7ff36905db80 20 client.25591 populate_metadata read hostname 'smithi151' 2024-05-21T21:57:06.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.026+0000 7ff36905db80 1 --2- 172.21.15.151:0/942850655 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55ad6fef3fe0 0x55ad6fef6400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:06.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.026+0000 7ff36905db80 4 client.25591 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T21:57:06.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.026+0000 7ff36905db80 1 -- 172.21.15.151:0/942850655 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ad6fe3ae10 con 0x55ad6fef3fe0 2024-05-21T21:57:06.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.026+0000 7ff357e00700 1 --2- 172.21.15.151:0/942850655 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55ad6fef3fe0 0x55ad6fef6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:06.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.027+0000 7ff341600700 10 client.25591 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T21:57:06.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.027+0000 7ff357e00700 1 --2- 172.21.15.151:0/942850655 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55ad6fef3fe0 0x55ad6fef6400 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bcf587ddc382af30 server_cookie=27ef9fcd2a0ac485 in_seq=0 out_seq=0 2024-05-21T21:57:06.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.028+0000 7ff341600700 1 -- 172.21.15.151:0/942850655 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55ad6fe3ae10 con 0x55ad6fef3fe0 2024-05-21T21:57:06.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.028+0000 7ff341600700 10 client.25591 handle_command_reply: tid=0 2024-05-21T21:57:06.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.028+0000 7ff36905db80 1 client.25591 shutdown 2024-05-21T21:57:06.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.028+0000 7ff36905db80 1 -- 172.21.15.151:0/942850655 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ad6fee00a0 msgr2=0x55ad6fee39a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:06.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.028+0000 7ff36905db80 1 --2- 172.21.15.151:0/942850655 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ad6fee00a0 0x55ad6fee39a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff35819d300 tx=0x7ff34400bf00 comp rx=0 tx=0).stop 2024-05-21T21:57:06.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.028+0000 7ff36905db80 1 -- 172.21.15.151:0/942850655 shutdown_connections 2024-05-21T21:57:06.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.028+0000 7ff36905db80 1 --2- 172.21.15.151:0/942850655 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55ad6fef3fe0 0x55ad6fef6400 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:06.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.028+0000 7ff36905db80 1 --2- 172.21.15.151:0/942850655 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ad6fedab50 0x55ad6fee6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:06.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.029+0000 7ff36905db80 1 --2- 172.21.15.151:0/942850655 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ad6fee09c0 0x55ad6fee3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:06.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.029+0000 7ff36905db80 1 --2- 172.21.15.151:0/942850655 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ad6fee00a0 0x55ad6fee39a0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:06.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.029+0000 7ff36905db80 1 -- 172.21.15.151:0/942850655 >> 172.21.15.151:0/942850655 conn(0x55ad6fe08c60 msgr2=0x55ad6fed9520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:57:06.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.029+0000 7ff341600700 0 client.25591 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T21:57:06.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.029+0000 7ff36905db80 1 -- 172.21.15.151:0/942850655 shutdown_connections 2024-05-21T21:57:06.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.029+0000 7ff36905db80 1 -- 172.21.15.151:0/942850655 wait complete. 2024-05-21T21:57:06.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.029+0000 7ff36905db80 20 client.25591 trim_cache size 0 max 16384 2024-05-21T21:57:06.070 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.069+0000 7ff36905db80 1 Processor -- start 2024-05-21T21:57:06.070 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.069+0000 7ff36905db80 1 -- start start 2024-05-21T21:57:06.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.070+0000 7ff36905db80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ad6fedab50 0x55ad6fed4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:06.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.070+0000 7ff36905db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ad6fee09c0 0x55ad6fed51f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:06.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.070+0000 7ff36905db80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ad6fee3fd0 0x55ad6fed9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:06.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.070+0000 7ff36905db80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff33800d580 con 0x55ad6fee3fd0 2024-05-21T21:57:06.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.070+0000 7ff36905db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff33800f0c0 con 0x55ad6fee09c0 2024-05-21T21:57:06.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.070+0000 7ff36905db80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff33800dc50 con 0x55ad6fedab50 2024-05-21T21:57:06.072 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.070+0000 7ff357e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ad6fedab50 0x55ad6fed4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:06.072 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.070+0000 7ff357e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ad6fedab50 0x55ad6fed4cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:37026/0 (socket says 172.21.15.151:37026) 2024-05-21T21:57:06.072 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.070+0000 7ff357e00700 1 -- 172.21.15.151:0/424058868 learned_addr learned my addr 172.21.15.151:0/424058868 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:57:06.072 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.070+0000 7ff357e00700 1 --2- 172.21.15.151:0/424058868 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ad6fee3fd0 0x55ad6fed9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:06.072 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.070+0000 7ff356a00700 1 --2- 172.21.15.151:0/424058868 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ad6fee09c0 0x55ad6fed51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:06.073 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.071+0000 7ff356a00700 1 -- 172.21.15.151:0/424058868 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ad6fedab50 msgr2=0x55ad6fed4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:06.073 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.071+0000 7ff356a00700 1 --2- 172.21.15.151:0/424058868 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ad6fedab50 0x55ad6fed4cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:06.073 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.071+0000 7ff356a00700 1 -- 172.21.15.151:0/424058868 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ad6fee3fd0 msgr2=0x55ad6fed9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:06.073 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.071+0000 7ff356a00700 1 --2- 172.21.15.151:0/424058868 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ad6fee3fd0 0x55ad6fed9dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:06.073 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.071+0000 7ff356a00700 1 -- 172.21.15.151:0/424058868 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad6fecfec0 con 0x55ad6fee09c0 2024-05-21T21:57:06.073 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.072+0000 7ff357e00700 1 --2- 172.21.15.151:0/424058868 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ad6fedab50 0x55ad6fed4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:57:06.074 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.072+0000 7ff356a00700 1 --2- 172.21.15.151:0/424058868 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ad6fee09c0 0x55ad6fed51f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x55ad6fee4a30 tx=0x7ff344009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=491f39869d77c5fb server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:57:06.074 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.072+0000 7ff343e00700 1 -- 172.21.15.151:0/424058868 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff344017730 con 0x55ad6fee09c0 2024-05-21T21:57:06.074 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.072+0000 7ff343e00700 1 -- 172.21.15.151:0/424058868 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3440178d0 con 0x55ad6fee09c0 2024-05-21T21:57:06.074 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.072+0000 7ff343e00700 1 -- 172.21.15.151:0/424058868 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff344005280 con 0x55ad6fee09c0 2024-05-21T21:57:06.074 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.074+0000 7ff36905db80 1 -- 172.21.15.151:0/424058868 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ad6fee09c0 msgr2=0x55ad6fed51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:06.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.074+0000 7ff36905db80 1 --2- 172.21.15.151:0/424058868 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ad6fee09c0 0x55ad6fed51f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x55ad6fee4a30 tx=0x7ff344009310 comp rx=0 tx=0).stop 2024-05-21T21:57:06.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.074+0000 7ff36905db80 1 -- 172.21.15.151:0/424058868 shutdown_connections 2024-05-21T21:57:06.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.074+0000 7ff36905db80 1 --2- 172.21.15.151:0/424058868 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ad6fee3fd0 0x55ad6fed9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:06.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.074+0000 7ff36905db80 1 --2- 172.21.15.151:0/424058868 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ad6fedab50 0x55ad6fed4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:06.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.074+0000 7ff36905db80 1 --2- 172.21.15.151:0/424058868 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ad6fee09c0 0x55ad6fed51f0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:06.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.074+0000 7ff36905db80 1 -- 172.21.15.151:0/424058868 >> 172.21.15.151:0/424058868 conn(0x55ad6fe08c60 msgr2=0x55ad6fee2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:57:06.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.074+0000 7ff36905db80 1 -- 172.21.15.151:0/424058868 shutdown_connections 2024-05-21T21:57:06.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.074+0000 7ff36905db80 1 -- 172.21.15.151:0/424058868 wait complete. 2024-05-21T21:57:06.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.075+0000 7ff36905db80 1 -- start start 2024-05-21T21:57:06.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.075+0000 7ff36905db80 1 Processor -- start 2024-05-21T21:57:06.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.075+0000 7ff36905db80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ad6fedab50 0x55ad6fee9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:06.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.075+0000 7ff36905db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ad6fee09c0 0x55ad6feea0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:06.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.075+0000 7ff36905db80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ad6fee3fd0 0x55ad6ff5ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:06.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.075+0000 7ff36905db80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff33800e9f0 con 0x55ad6fedab50 2024-05-21T21:57:06.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.075+0000 7ff36905db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff33800e320 con 0x55ad6fee09c0 2024-05-21T21:57:06.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.076+0000 7ff36905db80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff33800b520 con 0x55ad6fee3fd0 2024-05-21T21:57:06.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.076+0000 7ff36905db80 10 client.0 _ll_register_callbacks cb 0x55ad6fdd2cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:57:06.078 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.076+0000 7ff357e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ad6fedab50 0x55ad6fee9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:06.078 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.076+0000 7ff357e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ad6fedab50 0x55ad6fee9bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:47076/0 (socket says 172.21.15.151:47076) 2024-05-21T21:57:06.078 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.076+0000 7ff357e00700 1 -- 172.21.15.151:0/2209219173 learned_addr learned my addr 172.21.15.151:0/2209219173 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:57:06.078 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.076+0000 7ff356a00700 1 --2- 172.21.15.151:0/2209219173 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ad6fee09c0 0x55ad6feea0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:06.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.076+0000 7ff357e00700 1 --2- 172.21.15.151:0/2209219173 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ad6fee3fd0 0x55ad6ff5ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:06.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.076+0000 7ff356a00700 1 -- 172.21.15.151:0/2209219173 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ad6fee3fd0 msgr2=0x55ad6ff5ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:06.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.076+0000 7ff356a00700 1 --2- 172.21.15.151:0/2209219173 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ad6fee3fd0 0x55ad6ff5ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:06.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.076+0000 7ff356a00700 1 -- 172.21.15.151:0/2209219173 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ad6fedab50 msgr2=0x55ad6fee9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:06.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.076+0000 7ff356a00700 1 --2- 172.21.15.151:0/2209219173 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ad6fedab50 0x55ad6fee9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:06.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.076+0000 7ff356a00700 1 -- 172.21.15.151:0/2209219173 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad6ff70100 con 0x55ad6fee09c0 2024-05-21T21:57:06.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.077+0000 7ff357e00700 1 --2- 172.21.15.151:0/2209219173 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ad6fedab50 0x55ad6fee9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T21:57:06.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.077+0000 7ff356a00700 1 --2- 172.21.15.151:0/2209219173 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ad6fee09c0 0x55ad6feea0f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x55ad6fed1d60 tx=0x7ff344004b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e98eea77a5710bb server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:57:06.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.077+0000 7ff341600700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-21T21:57:06.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.077+0000 7ff341600700 1 -- 172.21.15.151:0/2209219173 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff344042020 con 0x55ad6fee09c0 2024-05-21T21:57:06.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.077+0000 7ff341600700 1 -- 172.21.15.151:0/2209219173 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7ff344005230 con 0x55ad6fee09c0 2024-05-21T21:57:06.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.078+0000 7ff36905db80 1 -- 172.21.15.151:0/2209219173 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3440178d0 con 0x55ad6fee09c0 2024-05-21T21:57:06.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.078+0000 7ff341600700 1 -- 172.21.15.151:0/2209219173 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff344017730 con 0x55ad6fee09c0 2024-05-21T21:57:06.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.078+0000 7ff36905db80 10 client.25580 fetch_fsmap learned FSMap version 29 2024-05-21T21:57:06.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.078+0000 7ff36905db80 1 -- 172.21.15.151:0/2209219173 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ad6ff713b0 con 0x55ad6fee09c0 2024-05-21T21:57:06.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.078+0000 7ff341600700 1 -- 172.21.15.151:0/2209219173 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3440178d0 con 0x55ad6fee09c0 2024-05-21T21:57:06.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.079+0000 7ff341600700 1 -- 172.21.15.151:0/2209219173 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7ff344005430 con 0x55ad6fee09c0 2024-05-21T21:57:06.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.079+0000 7ff36905db80 10 client.25580 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T21:57:06.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.079+0000 7ff36905db80 10 client.25580 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T21:57:06.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.079+0000 7ff36905db80 20 client.25580 populate_metadata read hostname 'smithi151' 2024-05-21T21:57:06.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.079+0000 7ff36905db80 1 --2- 172.21.15.151:0/2209219173 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55ad6ff71720 0x55ad6ff73b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:06.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.079+0000 7ff36905db80 4 client.25580 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-21T21:57:06.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.079+0000 7ff36905db80 1 -- 172.21.15.151:0/2209219173 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ad6fe345f0 con 0x55ad6ff71720 2024-05-21T21:57:06.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.079+0000 7ff357e00700 1 --2- 172.21.15.151:0/2209219173 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55ad6ff71720 0x55ad6ff73b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:06.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.080+0000 7ff357e00700 1 --2- 172.21.15.151:0/2209219173 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55ad6ff71720 0x55ad6ff73b40 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70170554ae638325 server_cookie=b3ebe3f737da51ca in_seq=0 out_seq=0 2024-05-21T21:57:06.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.080+0000 7ff341600700 10 client.25580 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T21:57:06.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.081+0000 7ff341600700 1 -- 172.21.15.151:0/2209219173 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ad6fe345f0 con 0x55ad6ff71720 2024-05-21T21:57:06.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.081+0000 7ff341600700 10 client.25580 handle_command_reply: tid=0 2024-05-21T21:57:06.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.081+0000 7ff36905db80 1 client.25580 shutdown 2024-05-21T21:57:06.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.081+0000 7ff36905db80 1 -- 172.21.15.151:0/2209219173 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ad6fee09c0 msgr2=0x55ad6feea0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:06.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.081+0000 7ff36905db80 1 --2- 172.21.15.151:0/2209219173 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ad6fee09c0 0x55ad6feea0f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x55ad6fed1d60 tx=0x7ff344004b10 comp rx=0 tx=0).stop 2024-05-21T21:57:06.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.081+0000 7ff36905db80 1 -- 172.21.15.151:0/2209219173 shutdown_connections 2024-05-21T21:57:06.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.081+0000 7ff36905db80 1 --2- 172.21.15.151:0/2209219173 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ad6fee3fd0 0x55ad6ff5ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:06.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.081+0000 7ff36905db80 1 --2- 172.21.15.151:0/2209219173 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ad6fedab50 0x55ad6fee9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:06.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.081+0000 7ff36905db80 1 --2- 172.21.15.151:0/2209219173 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55ad6ff71720 0x55ad6ff73b40 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:06.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.081+0000 7ff36905db80 1 --2- 172.21.15.151:0/2209219173 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ad6fee09c0 0x55ad6feea0f0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:06.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.081+0000 7ff36905db80 1 -- 172.21.15.151:0/2209219173 >> 172.21.15.151:0/2209219173 conn(0x55ad6fe08c60 msgr2=0x55ad6fed9d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:57:06.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.081+0000 7ff341600700 0 client.25580 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T21:57:06.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.082+0000 7ff36905db80 1 -- 172.21.15.151:0/2209219173 shutdown_connections 2024-05-21T21:57:06.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.082+0000 7ff36905db80 1 -- 172.21.15.151:0/2209219173 wait complete. 2024-05-21T21:57:06.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.082+0000 7ff36905db80 20 client.25580 trim_cache size 0 max 16384 2024-05-21T21:57:06.084 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-05-21T21:57:06.084 INFO:teuthology.orchestra.run.smithi151.stdout: "return_code": 0, 2024-05-21T21:57:06.084 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_tag": "49b0b652-88de-4191-844f-fd7f97bd8116", 2024-05-21T21:57:06.084 INFO:teuthology.orchestra.run.smithi151.stdout: "mode": "asynchronous" 2024-05-21T21:57:06.084 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-05-21T21:57:06.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.083+0000 7ff35d000700 1 -- 172.21.15.151:0/1849828787 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7ff33c0b2b10 msgr2=0x7ff33c0b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:06.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.083+0000 7ff35d000700 1 --2- 172.21.15.151:0/1849828787 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7ff33c0b2b10 0x7ff33c0b4fd0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff3480e0140 tx=0x7ff348017f80 comp rx=0 tx=0).stop 2024-05-21T21:57:06.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.083+0000 7ff35d000700 1 -- 172.21.15.151:0/1849828787 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff358103490 msgr2=0x7ff3581a1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:06.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.083+0000 7ff35d000700 1 --2- 172.21.15.151:0/1849828787 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff358103490 0x7ff3581a1a50 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff34800e9c0 tx=0x7ff34800ed80 comp rx=0 tx=0).stop 2024-05-21T21:57:06.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.083+0000 7ff35d000700 1 -- 172.21.15.151:0/1849828787 shutdown_connections 2024-05-21T21:57:06.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.083+0000 7ff35d000700 1 --2- 172.21.15.151:0/1849828787 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7ff33c0b2b10 0x7ff33c0b4fd0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:06.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.083+0000 7ff35d000700 1 --2- 172.21.15.151:0/1849828787 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff358103490 0x7ff3581a1a50 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:06.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.083+0000 7ff35d000700 1 --2- 172.21.15.151:0/1849828787 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff358109540 0x7ff35819bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:06.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.083+0000 7ff35d000700 1 --2- 172.21.15.151:0/1849828787 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff358103e50 0x7ff3581a1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:06.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.083+0000 7ff35d000700 1 -- 172.21.15.151:0/1849828787 >> 172.21.15.151:0/1849828787 conn(0x7ff3580fefc0 msgr2=0x7ff35810ef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:57:06.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.083+0000 7ff35d000700 1 -- 172.21.15.151:0/1849828787 shutdown_connections 2024-05-21T21:57:06.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:06.083+0000 7ff35d000700 1 -- 172.21.15.151:0/1849828787 wait complete. 2024-05-21T21:57:06.343 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:57:06.986 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-21T21:57:07.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:57:07 smithi151 ceph-mon[31623]: scrub queued for path: / 2024-05-21T21:57:07.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:57:07 smithi151 ceph-mon[31623]: scrub summary: idle+waiting paths [/] 2024-05-21T21:57:07.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:57:07 smithi151 ceph-mon[31623]: scrub summary: active paths [/] 2024-05-21T21:57:07.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:57:07 smithi151 ceph-mon[31623]: scrub summary: idle+waiting paths [/] 2024-05-21T21:57:07.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:57:07 smithi151 ceph-mon[31623]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-05-21T21:57:07.735 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:57:07 smithi184 ceph-mon[36942]: scrub queued for path: / 2024-05-21T21:57:07.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:57:07 smithi184 ceph-mon[36942]: scrub summary: idle+waiting paths [/] 2024-05-21T21:57:07.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:57:07 smithi184 ceph-mon[36942]: scrub summary: active paths [/] 2024-05-21T21:57:07.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:57:07 smithi184 ceph-mon[36942]: scrub summary: idle+waiting paths [/] 2024-05-21T21:57:07.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:57:07 smithi184 ceph-mon[36942]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-05-21T21:57:07.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:57:07 smithi164 ceph-mon[36269]: scrub queued for path: / 2024-05-21T21:57:07.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:57:07 smithi164 ceph-mon[36269]: scrub summary: idle+waiting paths [/] 2024-05-21T21:57:07.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:57:07 smithi164 ceph-mon[36269]: scrub summary: active paths [/] 2024-05-21T21:57:07.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:57:07 smithi164 ceph-mon[36269]: scrub summary: idle+waiting paths [/] 2024-05-21T21:57:07.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:57:07 smithi164 ceph-mon[36269]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-05-21T21:57:08.277 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:57:09.155 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:09.099+0000 7f88ec800700 1 -- 172.21.15.151:0/2039546353 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88e41037f0 msgr2=0x7f88e4103bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:09.155 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:09.099+0000 7f88ec800700 1 --2- 172.21.15.151:0/2039546353 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88e41037f0 0x7f88e4103bd0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f88d80099c0 tx=0x7f88d8009cd0 comp rx=0 tx=0).stop 2024-05-21T21:57:09.155 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:09.099+0000 7f88ec800700 1 -- 172.21.15.151:0/2039546353 shutdown_connections 2024-05-21T21:57:09.155 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:09.099+0000 7f88ec800700 1 --2- 172.21.15.151:0/2039546353 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88e41037f0 0x7f88e4103bd0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:09.155 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:09.099+0000 7f88ec800700 1 --2- 172.21.15.151:0/2039546353 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f88e41088f0 0x7f88e4110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:09.155 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:09.099+0000 7f88ec800700 1 --2- 172.21.15.151:0/2039546353 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f88e41041a0 0x7f88e41081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:09.155 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:09.099+0000 7f88ec800700 1 -- 172.21.15.151:0/2039546353 >> 172.21.15.151:0/2039546353 conn(0x7f88e407a6a0 msgr2=0x7f88e407aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:57:09.155 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:09.099+0000 7f88ec800700 1 -- 172.21.15.151:0/2039546353 shutdown_connections 2024-05-21T21:57:09.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:09.099+0000 7f88ec800700 1 -- 172.21.15.151:0/2039546353 wait complete. 2024-05-21T21:57:09.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:09.101+0000 7f88ec800700 1 Processor -- start 2024-05-21T21:57:09.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:09.101+0000 7f88ec800700 1 -- start start 2024-05-21T21:57:09.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:09.101+0000 7f88ec800700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f88e41037f0 0x7f88e419f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:09.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:09.101+0000 7f88ec800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f88e41041a0 0x7f88e419fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:09.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:09.101+0000 7f88ec800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88e41088f0 0x7f88e41999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:09.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:09.101+0000 7f88ec800700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f88e4113730 con 0x7f88e41041a0 2024-05-21T21:57:09.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:09.101+0000 7f88ec800700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f88e41135b0 con 0x7f88e41088f0 2024-05-21T21:57:09.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:09.101+0000 7f88ec800700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f88e41138b0 con 0x7f88e41037f0 2024-05-21T21:57:09.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:09.101+0000 7f88e9400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f88e41041a0 0x7f88e419fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:09.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:09.102+0000 7f88e9400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f88e41041a0 0x7f88e419fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:47084/0 (socket says 172.21.15.151:47084) 2024-05-21T21:57:09.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:09.102+0000 7f88e9400700 1 -- 172.21.15.151:0/1313820761 learned_addr learned my addr 172.21.15.151:0/1313820761 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:57:09.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:09.102+0000 7f88e9e00700 1 --2- 172.21.15.151:0/1313820761 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f88e41037f0 0x7f88e419f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:09.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:09.102+0000 7f88ea800700 1 --2- 172.21.15.151:0/1313820761 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88e41088f0 0x7f88e41999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:09.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:09.102+0000 7f88e9400700 1 -- 172.21.15.151:0/1313820761 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f88e41037f0 msgr2=0x7f88e419f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:09.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:09.102+0000 7f88e9400700 1 --2- 172.21.15.151:0/1313820761 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f88e41037f0 0x7f88e419f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:09.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:09.102+0000 7f88e9400700 1 -- 172.21.15.151:0/1313820761 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88e41088f0 msgr2=0x7f88e41999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:09.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:09.102+0000 7f88e9400700 1 --2- 172.21.15.151:0/1313820761 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88e41088f0 0x7f88e41999d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:09.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:09.102+0000 7f88e9400700 1 -- 172.21.15.151:0/1313820761 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88e419a130 con 0x7f88e41041a0 2024-05-21T21:57:09.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:09.102+0000 7f88e9400700 1 --2- 172.21.15.151:0/1313820761 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f88e41041a0 0x7f88e419fd50 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f88d400e9c0 tx=0x7f88d400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:57:09.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:09.103+0000 7f88dea00700 1 -- 172.21.15.151:0/1313820761 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d400cbd0 con 0x7f88e41041a0 2024-05-21T21:57:09.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:09.103+0000 7f88dea00700 1 -- 172.21.15.151:0/1313820761 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88d400ee60 con 0x7f88e41041a0 2024-05-21T21:57:09.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:09.103+0000 7f88dea00700 1 -- 172.21.15.151:0/1313820761 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d40055c0 con 0x7f88e41041a0 2024-05-21T21:57:09.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:09.103+0000 7f88ec800700 1 -- 172.21.15.151:0/1313820761 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88e419a420 con 0x7f88e41041a0 2024-05-21T21:57:09.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:09.103+0000 7f88ec800700 1 -- 172.21.15.151:0/1313820761 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88e4075770 con 0x7f88e41041a0 2024-05-21T21:57:09.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:09.104+0000 7f88ec800700 1 -- 172.21.15.151:0/1313820761 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88e410bb30 con 0x7f88e41041a0 2024-05-21T21:57:09.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:09.108+0000 7f88dea00700 1 -- 172.21.15.151:0/1313820761 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7f88d4004750 con 0x7f88e41041a0 2024-05-21T21:57:09.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:09.110+0000 7f88dea00700 1 --2- 172.21.15.151:0/1313820761 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f88cc0b29f0 0x7f88cc0b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:09.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:09.110+0000 7f88dea00700 1 -- 172.21.15.151:0/1313820761 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f88d40e28f0 con 0x7f88e41041a0 2024-05-21T21:57:09.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:09.110+0000 7f88e9e00700 1 --2- 172.21.15.151:0/1313820761 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f88cc0b29f0 0x7f88cc0b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:09.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:09.111+0000 7f88e9e00700 1 --2- 172.21.15.151:0/1313820761 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f88cc0b29f0 0x7f88cc0b4eb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f88d8000c00 tx=0x7f88d8020040 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:57:09.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:09.115+0000 7f88dea00700 1 -- 172.21.15.151:0/1313820761 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f88d40aa280 con 0x7f88e41041a0 2024-05-21T21:57:09.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:09.430+0000 7f88ec800700 1 -- 172.21.15.151:0/1313820761 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f88e406a230 con 0x7f88e41041a0 2024-05-21T21:57:09.433 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:09.431+0000 7f88dea00700 1 -- 172.21.15.151:0/1313820761 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11958 (secure 0 0 0) 0x7f88d40a9b80 con 0x7f88e41041a0 2024-05-21T21:57:09.433 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T21:57:09.433 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14358,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/158010529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":158010529},{"type":"v1","addr":"172.21.15.151:6835","nonce":158010529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6839/2069058115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":2069058115},{"type":"v1","addr":"172.21.15.151:6839","nonce":2069058115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T21:51:19.552069+0000","modified":"2024-05-21T21:55:34.643755+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24530,"mds_1":24493,"mds_2":24491,"mds_3":24478,"mds_4":24473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14403":{"gid":14403,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6841/1749723794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6840","nonce":1749723794},{"type":"v1","addr":"172.21.15.151:6841","nonce":1749723794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/2026266114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":2026266114},{"type":"v1","addr":"172.21.15.184:6833","nonce":2026266114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/2467900844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":2467900844},{"type":"v1","addr":"172.21.15.164:6833","nonce":2467900844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3367640960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3367640960},{"type":"v1","addr":"172.21.15.151:6837","nonce":3367640960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/3264142621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3264142621},{"type":"v1","addr":"172.21.15.184:6835","nonce":3264142621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":17,"addr":"172.21.15.164:6835/2340428470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2340428470},{"type":"v1","addr":"172.21.15.164:6835","nonce":2340428470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":16,"addr":"172.21.15.184:6837/2988194888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":2988194888},{"type":"v1","addr":"172.21.15.184:6837","nonce":2988194888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.164:6837/1554609572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1554609572},{"type":"v1","addr":"172.21.15.164:6837","nonce":1554609572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.184:6839/3059935143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":3059935143},{"type":"v1","addr":"172.21.15.184:6839","nonce":3059935143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.164:6839/2646401247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":2646401247},{"type":"v1","addr":"172.21.15.164:6839","nonce":2646401247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24530,"qdb_cluster":[24530,24493,24491,24473,24478]},"id":1}]} 2024-05-21T21:57:09.435 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:09.434+0000 7f88ec800700 1 -- 172.21.15.151:0/1313820761 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f88cc0b29f0 msgr2=0x7f88cc0b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:09.435 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:09.434+0000 7f88ec800700 1 --2- 172.21.15.151:0/1313820761 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f88cc0b29f0 0x7f88cc0b4eb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f88d8000c00 tx=0x7f88d8020040 comp rx=0 tx=0).stop 2024-05-21T21:57:09.436 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:09.434+0000 7f88ec800700 1 -- 172.21.15.151:0/1313820761 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f88e41041a0 msgr2=0x7f88e419fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:09.436 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:09.434+0000 7f88ec800700 1 --2- 172.21.15.151:0/1313820761 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f88e41041a0 0x7f88e419fd50 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f88d400e9c0 tx=0x7f88d400ecd0 comp rx=0 tx=0).stop 2024-05-21T21:57:09.436 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:09.434+0000 7f88ec800700 1 -- 172.21.15.151:0/1313820761 shutdown_connections 2024-05-21T21:57:09.436 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:09.434+0000 7f88ec800700 1 --2- 172.21.15.151:0/1313820761 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f88cc0b29f0 0x7f88cc0b4eb0 secure :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f88d8000c00 tx=0x7f88d8020040 comp rx=0 tx=0).stop 2024-05-21T21:57:09.436 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:09.434+0000 7f88ec800700 1 --2- 172.21.15.151:0/1313820761 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f88e41088f0 0x7f88e41999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:09.436 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:09.434+0000 7f88ec800700 1 --2- 172.21.15.151:0/1313820761 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f88e41037f0 0x7f88e419f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:09.436 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:09.434+0000 7f88ec800700 1 --2- 172.21.15.151:0/1313820761 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f88e41041a0 0x7f88e419fd50 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:09.436 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:09.434+0000 7f88ec800700 1 -- 172.21.15.151:0/1313820761 >> 172.21.15.151:0/1313820761 conn(0x7f88e407a6a0 msgr2=0x7f88e41017c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:57:09.436 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:09.435+0000 7f88ec800700 1 -- 172.21.15.151:0/1313820761 shutdown_connections 2024-05-21T21:57:09.437 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:09.435+0000 7f88ec800700 1 -- 172.21.15.151:0/1313820761 wait complete. 2024-05-21T21:57:09.437 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 29 2024-05-21T21:57:09.588 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:57:09 smithi164 ceph-mon[36269]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 KiB/s wr, 29 op/s 2024-05-21T21:57:09.588 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:57:09 smithi164 ceph-mon[36269]: mgrmap e27: x(active, since 92s), standbys: z, y 2024-05-21T21:57:09.588 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:57:09 smithi164 ceph-mon[36269]: scrub summary: idle 2024-05-21T21:57:09.632 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:57:09 smithi151 ceph-mon[31623]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 KiB/s wr, 29 op/s 2024-05-21T21:57:09.632 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:57:09 smithi151 ceph-mon[31623]: mgrmap e27: x(active, since 92s), standbys: z, y 2024-05-21T21:57:09.632 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:57:09 smithi151 ceph-mon[31623]: scrub summary: idle 2024-05-21T21:57:09.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:57:09 smithi184 ceph-mon[36942]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 KiB/s wr, 29 op/s 2024-05-21T21:57:09.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:57:09 smithi184 ceph-mon[36942]: mgrmap e27: x(active, since 92s), standbys: z, y 2024-05-21T21:57:09.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:57:09 smithi184 ceph-mon[36942]: scrub summary: idle 2024-05-21T21:57:10.654 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:57:10 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/1313820761' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T21:57:10.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:57:10 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/1313820761' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T21:57:10.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:57:10 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/1313820761' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T21:57:11.062 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24530 quiesce db / --timeout 90 --expiration 146.15 --await-for 45.0' 2024-05-21T21:57:11.062 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24530 quiesce db / --timeout 90 --expiration 146.15 --await-for 45.0 2024-05-21T21:57:11.352 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:57:11 smithi151 ceph-mon[31623]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-05-21T21:57:11.694 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.692+0000 7fece1e00700 1 -- 172.21.15.151:0/3411876572 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fecdc1037f0 msgr2=0x7fecdc103bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:11.694 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.692+0000 7fece1e00700 1 --2- 172.21.15.151:0/3411876572 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fecdc1037f0 0x7fecdc103bd0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7feccc0099c0 tx=0x7feccc009cd0 comp rx=0 tx=0).stop 2024-05-21T21:57:11.694 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.692+0000 7fece1e00700 1 -- 172.21.15.151:0/3411876572 shutdown_connections 2024-05-21T21:57:11.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.692+0000 7fece1e00700 1 --2- 172.21.15.151:0/3411876572 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fecdc1037f0 0x7fecdc103bd0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:11.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.692+0000 7fece1e00700 1 --2- 172.21.15.151:0/3411876572 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fecdc1088f0 0x7fecdc110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:11.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.692+0000 7fece1e00700 1 --2- 172.21.15.151:0/3411876572 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fecdc1041a0 0x7fecdc1081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:11.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.692+0000 7fece1e00700 1 -- 172.21.15.151:0/3411876572 >> 172.21.15.151:0/3411876572 conn(0x7fecdc07a6a0 msgr2=0x7fecdc07aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:57:11.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.692+0000 7fece1e00700 1 -- 172.21.15.151:0/3411876572 shutdown_connections 2024-05-21T21:57:11.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.692+0000 7fece1e00700 1 -- 172.21.15.151:0/3411876572 wait complete. 2024-05-21T21:57:11.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.694+0000 7fece1e00700 1 Processor -- start 2024-05-21T21:57:11.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.694+0000 7fece1e00700 1 -- start start 2024-05-21T21:57:11.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.694+0000 7fece1e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fecdc1037f0 0x7fecdc19bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:11.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.694+0000 7fece1e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fecdc1041a0 0x7fecdc19c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:11.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.695+0000 7fece1e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fecdc1088f0 0x7fecdc1a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:11.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.695+0000 7fece1e00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fecdc113730 con 0x7fecdc1037f0 2024-05-21T21:57:11.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.695+0000 7fece1e00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fecdc1135b0 con 0x7fecdc1088f0 2024-05-21T21:57:11.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.695+0000 7fece1e00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fecdc1138b0 con 0x7fecdc1041a0 2024-05-21T21:57:11.698 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.695+0000 7fecdaa00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fecdc1037f0 0x7fecdc19bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:11.698 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.695+0000 7fecdaa00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fecdc1037f0 0x7fecdc19bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:47090/0 (socket says 172.21.15.151:47090) 2024-05-21T21:57:11.698 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.695+0000 7fecdaa00700 1 -- 172.21.15.151:0/2548007017 learned_addr learned my addr 172.21.15.151:0/2548007017 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:57:11.698 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.695+0000 7fecda000700 1 --2- 172.21.15.151:0/2548007017 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fecdc1041a0 0x7fecdc19c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:11.698 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.695+0000 7fecdaa00700 1 -- 172.21.15.151:0/2548007017 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fecdc1041a0 msgr2=0x7fecdc19c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:11.698 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.695+0000 7fecdaa00700 1 --2- 172.21.15.151:0/2548007017 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fecdc1041a0 0x7fecdc19c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:11.698 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.695+0000 7fecdaa00700 1 -- 172.21.15.151:0/2548007017 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fecdc1088f0 msgr2=0x7fecdc1a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:11.698 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.695+0000 7fecdaa00700 1 --2- 172.21.15.151:0/2548007017 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fecdc1088f0 0x7fecdc1a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:11.698 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.695+0000 7fecdaa00700 1 -- 172.21.15.151:0/2548007017 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecdc1a0df0 con 0x7fecdc1037f0 2024-05-21T21:57:11.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.696+0000 7fecdaa00700 1 --2- 172.21.15.151:0/2548007017 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fecdc1037f0 0x7fecdc19bde0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7feccc005910 tx=0x7feccc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:57:11.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.696+0000 7fece0a00700 1 -- 172.21.15.151:0/2548007017 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feccc026070 con 0x7fecdc1037f0 2024-05-21T21:57:11.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.696+0000 7fece0a00700 1 -- 172.21.15.151:0/2548007017 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feccc003c70 con 0x7fecdc1037f0 2024-05-21T21:57:11.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.696+0000 7fece0a00700 1 -- 172.21.15.151:0/2548007017 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feccc00fae0 con 0x7fecdc1037f0 2024-05-21T21:57:11.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.696+0000 7fece1e00700 1 -- 172.21.15.151:0/2548007017 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecdc1a1080 con 0x7fecdc1037f0 2024-05-21T21:57:11.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.697+0000 7fece1e00700 1 -- 172.21.15.151:0/2548007017 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecdc1a8960 con 0x7fecdc1037f0 2024-05-21T21:57:11.703 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.702+0000 7fecec72cb80 1 Processor -- start 2024-05-21T21:57:11.703 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.702+0000 7fecec72cb80 1 -- start start 2024-05-21T21:57:11.703 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.702+0000 7fece0a00700 1 -- 172.21.15.151:0/2548007017 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7feccc029070 con 0x7fecdc1037f0 2024-05-21T21:57:11.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.703+0000 7fecec72cb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561f6cebe00 0x5561f6cec1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:11.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.703+0000 7fecec72cb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5561f6cec720 0x5561f6ce6330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:11.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.703+0000 7fecec72cb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561f6ce6950 0x5561f6ce6db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:11.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.703+0000 7fecec72cb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5561f6c3be40 con 0x5561f6ce6950 2024-05-21T21:57:11.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.703+0000 7fecec72cb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5561f6c2b9e0 con 0x5561f6cec720 2024-05-21T21:57:11.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.703+0000 7fecec72cb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5561f6c2fa20 con 0x5561f6cebe00 2024-05-21T21:57:11.705 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.703+0000 7fecdaa00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5561f6cec720 0x5561f6ce6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:11.705 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.703+0000 7fecdb400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561f6cebe00 0x5561f6cec1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:11.705 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.703+0000 7fecdaa00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5561f6cec720 0x5561f6ce6330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:33016/0 (socket says 172.21.15.151:33016) 2024-05-21T21:57:11.705 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.703+0000 7fecdaa00700 1 -- 172.21.15.151:0/633313633 learned_addr learned my addr 172.21.15.151:0/633313633 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:57:11.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.704+0000 7fecdaa00700 1 -- 172.21.15.151:0/633313633 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561f6cebe00 msgr2=0x5561f6cec1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:11.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.704+0000 7fecdaa00700 1 --2- 172.21.15.151:0/633313633 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561f6cebe00 0x5561f6cec1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:11.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.704+0000 7fecdaa00700 1 -- 172.21.15.151:0/633313633 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561f6ce6950 msgr2=0x5561f6ce6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:11.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.704+0000 7fecdaa00700 1 --2- 172.21.15.151:0/633313633 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561f6ce6950 0x5561f6ce6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:11.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.704+0000 7fecdaa00700 1 -- 172.21.15.151:0/633313633 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561f6bf4420 con 0x5561f6cec720 2024-05-21T21:57:11.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.704+0000 7fece0a00700 1 --2- 172.21.15.151:0/2548007017 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fecc00b29f0 0x7fecc00b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:11.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.704+0000 7fece0a00700 1 -- 172.21.15.151:0/2548007017 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7feccc0e2ea0 con 0x7fecdc1037f0 2024-05-21T21:57:11.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.705+0000 7fecdb400700 1 --2- 172.21.15.151:0/2548007017 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fecc00b29f0 0x7fecc00b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:11.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.705+0000 7fecdaa00700 1 --2- 172.21.15.151:0/633313633 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5561f6cec720 0x5561f6ce6330 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7feccc0e04e0 tx=0x7feccc0eb3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7427b391e49681 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:57:11.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.705+0000 7fecdb400700 1 --2- 172.21.15.151:0/2548007017 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fecc00b29f0 0x7fecc00b4eb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fecd400f560 tx=0x7fecd400b040 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:57:11.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.706+0000 7fecd0c00700 1 -- 172.21.15.151:0/633313633 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feccc0a6080 con 0x5561f6cec720 2024-05-21T21:57:11.708 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.706+0000 7fecd0c00700 1 -- 172.21.15.151:0/633313633 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feccc003e20 con 0x5561f6cec720 2024-05-21T21:57:11.708 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.706+0000 7fecd0c00700 1 -- 172.21.15.151:0/633313633 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feccc0a9930 con 0x5561f6cec720 2024-05-21T21:57:11.708 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.707+0000 7fecec72cb80 1 -- 172.21.15.151:0/633313633 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5561f6cec720 msgr2=0x5561f6ce6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:11.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.707+0000 7fecec72cb80 1 --2- 172.21.15.151:0/633313633 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5561f6cec720 0x5561f6ce6330 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7feccc0e04e0 tx=0x7feccc0eb3c0 comp rx=0 tx=0).stop 2024-05-21T21:57:11.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.707+0000 7fecec72cb80 1 -- 172.21.15.151:0/633313633 shutdown_connections 2024-05-21T21:57:11.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.707+0000 7fecec72cb80 1 --2- 172.21.15.151:0/633313633 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561f6ce6950 0x5561f6ce6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:11.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.707+0000 7fecec72cb80 1 --2- 172.21.15.151:0/633313633 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561f6cebe00 0x5561f6cec1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:11.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.707+0000 7fecec72cb80 1 --2- 172.21.15.151:0/633313633 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5561f6cec720 0x5561f6ce6330 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:11.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.707+0000 7fecec72cb80 1 -- 172.21.15.151:0/633313633 >> 172.21.15.151:0/633313633 conn(0x5561f6c0bbe0 msgr2=0x5561f6cd8690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:57:11.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.707+0000 7fecec72cb80 1 -- 172.21.15.151:0/633313633 shutdown_connections 2024-05-21T21:57:11.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.707+0000 7fecec72cb80 1 -- 172.21.15.151:0/633313633 wait complete. 2024-05-21T21:57:11.710 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.708+0000 7fecec72cb80 1 -- start start 2024-05-21T21:57:11.710 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.708+0000 7fecec72cb80 1 Processor -- start 2024-05-21T21:57:11.711 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.709+0000 7fecec72cb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5561f6ce6950 0x5561f6cf2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:11.711 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.709+0000 7fecec72cb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561f6cebe00 0x5561f6cef7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:11.711 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.709+0000 7fecec72cb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561f6cec720 0x5561f6cefcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:11.711 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.709+0000 7fecec72cb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5561f6b31fa0 con 0x5561f6cebe00 2024-05-21T21:57:11.711 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.709+0000 7fecec72cb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5561f6bcc760 con 0x5561f6ce6950 2024-05-21T21:57:11.711 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.709+0000 7fecec72cb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5561f6cf2fe0 con 0x5561f6cec720 2024-05-21T21:57:11.711 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.709+0000 7fecec72cb80 10 client.0 _ll_register_callbacks cb 0x5561f6bd5c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:57:11.711 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.710+0000 7fecdb400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561f6cec720 0x5561f6cefcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:11.712 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.710+0000 7fecdb400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561f6cec720 0x5561f6cefcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:37080/0 (socket says 172.21.15.151:37080) 2024-05-21T21:57:11.712 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.710+0000 7fecdb400700 1 -- 172.21.15.151:0/2008853737 learned_addr learned my addr 172.21.15.151:0/2008853737 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:57:11.712 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.710+0000 7fecdaa00700 1 --2- 172.21.15.151:0/2008853737 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5561f6ce6950 0x5561f6cf2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:11.712 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.710+0000 7fecda000700 1 --2- 172.21.15.151:0/2008853737 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561f6cebe00 0x5561f6cef7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:11.713 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.710+0000 7fecdb400700 1 -- 172.21.15.151:0/2008853737 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5561f6ce6950 msgr2=0x5561f6cf2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:11.713 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.710+0000 7fecdb400700 1 --2- 172.21.15.151:0/2008853737 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5561f6ce6950 0x5561f6cf2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:11.713 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.710+0000 7fecdb400700 1 -- 172.21.15.151:0/2008853737 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561f6cebe00 msgr2=0x5561f6cef7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:11.713 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.710+0000 7fecdb400700 1 --2- 172.21.15.151:0/2008853737 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561f6cebe00 0x5561f6cef7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:11.713 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.710+0000 7fecdb400700 1 -- 172.21.15.151:0/2008853737 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561f6b7b8d0 con 0x5561f6cec720 2024-05-21T21:57:11.713 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.710+0000 7fecdaa00700 1 --2- 172.21.15.151:0/2008853737 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5561f6ce6950 0x5561f6cf2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:57:11.714 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.710+0000 7fecda000700 1 --2- 172.21.15.151:0/2008853737 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561f6cebe00 0x5561f6cef7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T21:57:11.714 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.711+0000 7fecdb400700 1 --2- 172.21.15.151:0/2008853737 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561f6cec720 0x5561f6cefcf0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fecd40060b0 tx=0x7fecd4009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b405552ec424726f server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:57:11.714 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.711+0000 7fecc5600700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-21T21:57:11.714 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.711+0000 7fecc5600700 1 -- 172.21.15.151:0/2008853737 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd4025070 con 0x5561f6cec720 2024-05-21T21:57:11.714 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.711+0000 7fecc5600700 1 -- 172.21.15.151:0/2008853737 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fecd4004db0 con 0x5561f6cec720 2024-05-21T21:57:11.714 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.711+0000 7fecec72cb80 1 -- 172.21.15.151:0/2008853737 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5561f6c3dd90 con 0x5561f6cec720 2024-05-21T21:57:11.714 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.712+0000 7fecc5600700 1 -- 172.21.15.151:0/2008853737 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fecd4015e40 con 0x5561f6cec720 2024-05-21T21:57:11.714 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.712+0000 7fecec72cb80 10 client.25615 fetch_fsmap learned FSMap version 29 2024-05-21T21:57:11.714 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.712+0000 7fecec72cb80 1 -- 172.21.15.151:0/2008853737 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5561f6bfc060 con 0x5561f6cec720 2024-05-21T21:57:11.714 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.712+0000 7fecc5600700 1 -- 172.21.15.151:0/2008853737 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5561f6c3dd90 con 0x5561f6cec720 2024-05-21T21:57:11.716 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.712+0000 7fecc5600700 1 -- 172.21.15.151:0/2008853737 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fecd4020070 con 0x5561f6cec720 2024-05-21T21:57:11.716 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.712+0000 7fecec72cb80 10 client.25615 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T21:57:11.717 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.712+0000 7fecec72cb80 10 client.25615 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T21:57:11.717 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.713+0000 7fecec72cb80 20 client.25615 populate_metadata read hostname 'smithi151' 2024-05-21T21:57:11.717 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.713+0000 7fecec72cb80 1 --2- 172.21.15.151:0/2008853737 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5561f6cffbb0 0x5561f6d01fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:11.717 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.713+0000 7fecec72cb80 4 client.25615 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T21:57:11.717 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.713+0000 7fecec72cb80 1 -- 172.21.15.151:0/2008853737 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5561f6c37570 con 0x5561f6cffbb0 2024-05-21T21:57:11.717 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.713+0000 7fecdaa00700 1 --2- 172.21.15.151:0/2008853737 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5561f6cffbb0 0x5561f6d01fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:11.717 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.714+0000 7fecdaa00700 1 --2- 172.21.15.151:0/2008853737 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5561f6cffbb0 0x5561f6d01fd0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2dfbb8091a2af7f server_cookie=4ec14192286c0195 in_seq=0 out_seq=0 2024-05-21T21:57:11.717 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.714+0000 7fecc5600700 10 client.25615 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T21:57:11.717 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.714+0000 7fecc5600700 1 -- 172.21.15.151:0/2008853737 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5561f6c37570 con 0x5561f6cffbb0 2024-05-21T21:57:11.717 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.714+0000 7fecc5600700 10 client.25615 handle_command_reply: tid=0 2024-05-21T21:57:11.717 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.714+0000 7fecec72cb80 1 client.25615 shutdown 2024-05-21T21:57:11.717 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.715+0000 7fecec72cb80 1 -- 172.21.15.151:0/2008853737 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561f6cec720 msgr2=0x5561f6cefcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:11.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.715+0000 7fecec72cb80 1 --2- 172.21.15.151:0/2008853737 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561f6cec720 0x5561f6cefcf0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fecd40060b0 tx=0x7fecd4009920 comp rx=0 tx=0).stop 2024-05-21T21:57:11.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.715+0000 7fecec72cb80 1 -- 172.21.15.151:0/2008853737 shutdown_connections 2024-05-21T21:57:11.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.715+0000 7fecec72cb80 1 --2- 172.21.15.151:0/2008853737 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5561f6cffbb0 0x5561f6d01fd0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:11.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.715+0000 7fecec72cb80 1 --2- 172.21.15.151:0/2008853737 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5561f6ce6950 0x5561f6cf2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:11.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.715+0000 7fecec72cb80 1 --2- 172.21.15.151:0/2008853737 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561f6cec720 0x5561f6cefcf0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:11.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.715+0000 7fecec72cb80 1 --2- 172.21.15.151:0/2008853737 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561f6cebe00 0x5561f6cef7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:11.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.715+0000 7fecc5600700 0 client.25615 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T21:57:11.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.715+0000 7fecec72cb80 1 -- 172.21.15.151:0/2008853737 >> 172.21.15.151:0/2008853737 conn(0x5561f6c0bbe0 msgr2=0x5561f6cd7d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:57:11.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.715+0000 7fecec72cb80 1 -- 172.21.15.151:0/2008853737 shutdown_connections 2024-05-21T21:57:11.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.715+0000 7fecec72cb80 1 -- 172.21.15.151:0/2008853737 wait complete. 2024-05-21T21:57:11.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.715+0000 7fecec72cb80 20 client.25615 trim_cache size 0 max 16384 2024-05-21T21:57:11.735 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:57:11 smithi184 ceph-mon[36942]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-05-21T21:57:11.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.748+0000 7fecec72cb80 1 Processor -- start 2024-05-21T21:57:11.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.748+0000 7fecec72cb80 1 -- start start 2024-05-21T21:57:11.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.749+0000 7fecec72cb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5561f6ce6950 0x5561f6ced250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:11.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.749+0000 7fecec72cb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561f6cebe00 0x5561f6cd84f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:11.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.749+0000 7fecec72cb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561f6cec720 0x5561f6d07880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:11.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.749+0000 7fecec72cb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fecdc1abb00 con 0x5561f6cec720 2024-05-21T21:57:11.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.749+0000 7fecec72cb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fecdc1ad640 con 0x5561f6ce6950 2024-05-21T21:57:11.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.749+0000 7fecec72cb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fecdc1ac1d0 con 0x5561f6cebe00 2024-05-21T21:57:11.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.749+0000 7fecdaa00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5561f6ce6950 0x5561f6ced250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:11.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.749+0000 7fecdaa00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5561f6ce6950 0x5561f6ced250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:33030/0 (socket says 172.21.15.151:33030) 2024-05-21T21:57:11.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.749+0000 7fecdaa00700 1 -- 172.21.15.151:0/3602126586 learned_addr learned my addr 172.21.15.151:0/3602126586 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:57:11.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.749+0000 7fecda000700 1 --2- 172.21.15.151:0/3602126586 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561f6cebe00 0x5561f6cd84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:11.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.750+0000 7fecdaa00700 1 -- 172.21.15.151:0/3602126586 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561f6cebe00 msgr2=0x5561f6cd84f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:11.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.750+0000 7fecdaa00700 1 --2- 172.21.15.151:0/3602126586 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561f6cebe00 0x5561f6cd84f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:11.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.750+0000 7fecdaa00700 1 -- 172.21.15.151:0/3602126586 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561f6cec720 msgr2=0x5561f6d07880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:11.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.750+0000 7fecdaa00700 1 --2- 172.21.15.151:0/3602126586 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561f6cec720 0x5561f6d07880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:11.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.750+0000 7fecdaa00700 1 -- 172.21.15.151:0/3602126586 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561f6cedd20 con 0x5561f6ce6950 2024-05-21T21:57:11.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.750+0000 7fecdaa00700 1 --2- 172.21.15.151:0/3602126586 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5561f6ce6950 0x5561f6ced250 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7feccc0a6760 tx=0x7feccc00f560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=834bb3f5b0faf7af server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:57:11.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.750+0000 7fecc7e00700 1 -- 172.21.15.151:0/3602126586 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feccc0a9b10 con 0x5561f6ce6950 2024-05-21T21:57:11.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.750+0000 7fecc7e00700 1 -- 172.21.15.151:0/3602126586 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feccc0a9cd0 con 0x5561f6ce6950 2024-05-21T21:57:11.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.750+0000 7fecc7e00700 1 -- 172.21.15.151:0/3602126586 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feccc0a9f90 con 0x5561f6ce6950 2024-05-21T21:57:11.754 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.751+0000 7fecec72cb80 1 -- 172.21.15.151:0/3602126586 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5561f6ce6950 msgr2=0x5561f6ced250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:11.754 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.751+0000 7fecec72cb80 1 --2- 172.21.15.151:0/3602126586 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5561f6ce6950 0x5561f6ced250 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7feccc0a6760 tx=0x7feccc00f560 comp rx=0 tx=0).stop 2024-05-21T21:57:11.754 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.751+0000 7fecec72cb80 1 -- 172.21.15.151:0/3602126586 shutdown_connections 2024-05-21T21:57:11.754 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.751+0000 7fecec72cb80 1 --2- 172.21.15.151:0/3602126586 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5561f6ce6950 0x5561f6ced250 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:11.754 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.751+0000 7fecec72cb80 1 --2- 172.21.15.151:0/3602126586 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561f6cec720 0x5561f6d07880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:11.754 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.751+0000 7fecec72cb80 1 --2- 172.21.15.151:0/3602126586 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561f6cebe00 0x5561f6cd84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:11.754 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.751+0000 7fecec72cb80 1 -- 172.21.15.151:0/3602126586 >> 172.21.15.151:0/3602126586 conn(0x5561f6c0bbe0 msgr2=0x5561f6cecb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:57:11.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.752+0000 7fecec72cb80 1 -- 172.21.15.151:0/3602126586 shutdown_connections 2024-05-21T21:57:11.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.752+0000 7fecec72cb80 1 -- 172.21.15.151:0/3602126586 wait complete. 2024-05-21T21:57:11.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.752+0000 7fecec72cb80 1 -- start start 2024-05-21T21:57:11.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.752+0000 7fecec72cb80 1 Processor -- start 2024-05-21T21:57:11.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.753+0000 7fecec72cb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5561f6ce6950 0x5561f6cf8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:11.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.753+0000 7fecec72cb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561f6cebe00 0x5561f6cf8770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:11.757 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.753+0000 7fecec72cb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561f6cec720 0x5561f6d07590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:11.757 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.753+0000 7fecec72cb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fecdc1acf70 con 0x5561f6cebe00 2024-05-21T21:57:11.757 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.753+0000 7fecec72cb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fecdc1ac8a0 con 0x5561f6ce6950 2024-05-21T21:57:11.757 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.753+0000 7fecec72cb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fecdc10e510 con 0x5561f6cec720 2024-05-21T21:57:11.757 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.753+0000 7fecec72cb80 10 client.0 _ll_register_callbacks cb 0x5561f6bd5c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:57:11.757 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.753+0000 7fecda000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561f6cebe00 0x5561f6cf8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:11.757 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.753+0000 7fecda000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561f6cebe00 0x5561f6cf8770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:47134/0 (socket says 172.21.15.151:47134) 2024-05-21T21:57:11.757 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.753+0000 7fecdaa00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5561f6ce6950 0x5561f6cf8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:11.757 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.753+0000 7fecda000700 1 -- 172.21.15.151:0/2234757450 learned_addr learned my addr 172.21.15.151:0/2234757450 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:57:11.757 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.754+0000 7fecdb400700 1 --2- 172.21.15.151:0/2234757450 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561f6cec720 0x5561f6d07590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:11.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.754+0000 7fecdaa00700 1 -- 172.21.15.151:0/2234757450 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561f6cec720 msgr2=0x5561f6d07590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:11.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.754+0000 7fecdaa00700 1 --2- 172.21.15.151:0/2234757450 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561f6cec720 0x5561f6d07590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:11.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.754+0000 7fecdaa00700 1 -- 172.21.15.151:0/2234757450 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561f6cebe00 msgr2=0x5561f6cf8770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:11.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.754+0000 7fecdaa00700 1 --2- 172.21.15.151:0/2234757450 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561f6cebe00 0x5561f6cf8770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:11.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.754+0000 7fecdaa00700 1 -- 172.21.15.151:0/2234757450 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561f6d66400 con 0x5561f6ce6950 2024-05-21T21:57:11.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.754+0000 7fecda000700 1 --2- 172.21.15.151:0/2234757450 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561f6cebe00 0x5561f6cf8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:57:11.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.754+0000 7fecdaa00700 1 --2- 172.21.15.151:0/2234757450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5561f6ce6950 0x5561f6cf8230 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7feccc009990 tx=0x7feccc0aa8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dafc3bd65f8b809e server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:57:11.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.754+0000 7fecc5600700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-21T21:57:11.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.755+0000 7fecc5600700 1 -- 172.21.15.151:0/2234757450 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feccc0a9f90 con 0x5561f6ce6950 2024-05-21T21:57:11.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.755+0000 7fecc5600700 1 -- 172.21.15.151:0/2234757450 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7feccc0e0820 con 0x5561f6ce6950 2024-05-21T21:57:11.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.755+0000 7fecec72cb80 1 -- 172.21.15.151:0/2234757450 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feccc0a9cd0 con 0x5561f6ce6950 2024-05-21T21:57:11.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.755+0000 7fecc5600700 1 -- 172.21.15.151:0/2234757450 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7feccc0ae370 con 0x5561f6ce6950 2024-05-21T21:57:11.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.755+0000 7fecec72cb80 10 client.25601 fetch_fsmap learned FSMap version 29 2024-05-21T21:57:11.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.755+0000 7fecec72cb80 1 -- 172.21.15.151:0/2234757450 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5561f6d7bc50 con 0x5561f6ce6950 2024-05-21T21:57:11.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.756+0000 7fecc5600700 1 -- 172.21.15.151:0/2234757450 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feccc0a9cd0 con 0x5561f6ce6950 2024-05-21T21:57:11.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.756+0000 7fecc5600700 1 -- 172.21.15.151:0/2234757450 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7feccc0e6040 con 0x5561f6ce6950 2024-05-21T21:57:11.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.756+0000 7fecec72cb80 10 client.25601 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T21:57:11.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.756+0000 7fecec72cb80 10 client.25601 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T21:57:11.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.756+0000 7fecec72cb80 20 client.25601 populate_metadata read hostname 'smithi151' 2024-05-21T21:57:11.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.756+0000 7fecec72cb80 1 --2- 172.21.15.151:0/2234757450 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5561f6d7be10 0x5561f6d7e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:11.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.756+0000 7fecec72cb80 4 client.25601 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-21T21:57:11.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.756+0000 7fecec72cb80 1 -- 172.21.15.151:0/2234757450 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5561f6aae250 con 0x5561f6d7be10 2024-05-21T21:57:11.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.757+0000 7fecda000700 1 --2- 172.21.15.151:0/2234757450 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5561f6d7be10 0x5561f6d7e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:11.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.757+0000 7fecc5600700 10 client.25601 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T21:57:11.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.757+0000 7fecda000700 1 --2- 172.21.15.151:0/2234757450 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5561f6d7be10 0x5561f6d7e230 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf493112bf47b9e5 server_cookie=362dfa46b05d6866 in_seq=0 out_seq=0 2024-05-21T21:57:11.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.757+0000 7fecc5600700 1 -- 172.21.15.151:0/2234757450 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5561f6aae250 con 0x5561f6d7be10 2024-05-21T21:57:11.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.757+0000 7fecc5600700 10 client.25601 handle_command_reply: tid=0 2024-05-21T21:57:11.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.757+0000 7fecec72cb80 1 client.25601 shutdown 2024-05-21T21:57:11.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.758+0000 7fecec72cb80 1 -- 172.21.15.151:0/2234757450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5561f6ce6950 msgr2=0x5561f6cf8230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:11.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.758+0000 7fecec72cb80 1 --2- 172.21.15.151:0/2234757450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5561f6ce6950 0x5561f6cf8230 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7feccc009990 tx=0x7feccc0aa8c0 comp rx=0 tx=0).stop 2024-05-21T21:57:11.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.758+0000 7fecec72cb80 1 -- 172.21.15.151:0/2234757450 shutdown_connections 2024-05-21T21:57:11.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.758+0000 7fecec72cb80 1 --2- 172.21.15.151:0/2234757450 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5561f6d7be10 0x5561f6d7e230 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:11.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.758+0000 7fecec72cb80 1 --2- 172.21.15.151:0/2234757450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5561f6ce6950 0x5561f6cf8230 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:11.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.758+0000 7fecec72cb80 1 --2- 172.21.15.151:0/2234757450 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561f6cec720 0x5561f6d07590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:11.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.758+0000 7fecc5600700 0 client.25601 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T21:57:11.762 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.758+0000 7fecec72cb80 1 --2- 172.21.15.151:0/2234757450 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561f6cebe00 0x5561f6cf8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:11.762 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.758+0000 7fecec72cb80 1 -- 172.21.15.151:0/2234757450 >> 172.21.15.151:0/2234757450 conn(0x5561f6c0bbe0 msgr2=0x5561f6cea910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:57:11.762 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.758+0000 7fecec72cb80 1 -- 172.21.15.151:0/2234757450 shutdown_connections 2024-05-21T21:57:11.762 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.758+0000 7fecec72cb80 1 -- 172.21.15.151:0/2234757450 wait complete. 2024-05-21T21:57:11.762 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.758+0000 7fecec72cb80 20 client.25601 trim_cache size 0 max 16384 2024-05-21T21:57:11.762 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-05-21T21:57:11.762 INFO:teuthology.orchestra.run.smithi151.stdout: "status": "no active scrubs running", 2024-05-21T21:57:11.762 INFO:teuthology.orchestra.run.smithi151.stdout: "scrubs": {} 2024-05-21T21:57:11.763 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-05-21T21:57:11.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.759+0000 7fece1e00700 1 -- 172.21.15.151:0/2548007017 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fecc00b29f0 msgr2=0x7fecc00b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:11.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.759+0000 7fece1e00700 1 --2- 172.21.15.151:0/2548007017 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fecc00b29f0 0x7fecc00b4eb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fecd400f560 tx=0x7fecd400b040 comp rx=0 tx=0).stop 2024-05-21T21:57:11.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.759+0000 7fece1e00700 1 -- 172.21.15.151:0/2548007017 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fecdc1037f0 msgr2=0x7fecdc19bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:11.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.759+0000 7fece1e00700 1 --2- 172.21.15.151:0/2548007017 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fecdc1037f0 0x7fecdc19bde0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7feccc005910 tx=0x7feccc004270 comp rx=0 tx=0).stop 2024-05-21T21:57:11.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.759+0000 7fece1e00700 1 -- 172.21.15.151:0/2548007017 shutdown_connections 2024-05-21T21:57:11.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.759+0000 7fece1e00700 1 --2- 172.21.15.151:0/2548007017 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fecc00b29f0 0x7fecc00b4eb0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:11.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.759+0000 7fece1e00700 1 --2- 172.21.15.151:0/2548007017 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fecdc1088f0 0x7fecdc1a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:11.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.759+0000 7fece1e00700 1 --2- 172.21.15.151:0/2548007017 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fecdc1037f0 0x7fecdc19bde0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:11.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.759+0000 7fece1e00700 1 --2- 172.21.15.151:0/2548007017 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fecdc1041a0 0x7fecdc19c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:11.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.759+0000 7fece1e00700 1 -- 172.21.15.151:0/2548007017 >> 172.21.15.151:0/2548007017 conn(0x7fecdc07a6a0 msgr2=0x7fecdc1015e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:57:11.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.760+0000 7fece1e00700 1 -- 172.21.15.151:0/2548007017 shutdown_connections 2024-05-21T21:57:11.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:11.760+0000 7fece1e00700 1 -- 172.21.15.151:0/2548007017 wait complete. 2024-05-21T21:57:11.778 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:57:11.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:57:11 smithi164 ceph-mon[36269]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-05-21T21:57:12.942 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-21T21:57:12.943 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-21T21:57:13.702 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:57:13 smithi151 ceph-mon[31623]: pgmap v51: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 29 op/s 2024-05-21T21:57:13.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:57:13 smithi184 ceph-mon[36942]: pgmap v51: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 29 op/s 2024-05-21T21:57:13.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:57:13 smithi164 ceph-mon[36269]: pgmap v51: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 29 op/s 2024-05-21T21:57:13.822 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:57:15.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:57:15 smithi151 ceph-mon[31623]: pgmap v52: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 853 B/s wr, 10 op/s 2024-05-21T21:57:15.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:57:15 smithi184 ceph-mon[36942]: pgmap v52: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 853 B/s wr, 10 op/s 2024-05-21T21:57:15.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.371+0000 7f74d1e00700 1 -- 172.21.15.151:0/3448488094 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74cc103850 msgr2=0x7f74cc103c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:15.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.371+0000 7f74d1e00700 1 --2- 172.21.15.151:0/3448488094 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74cc103850 0x7f74cc103c30 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f74bc0099c0 tx=0x7f74bc009cd0 comp rx=0 tx=0).stop 2024-05-21T21:57:15.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.371+0000 7f74d1e00700 1 -- 172.21.15.151:0/3448488094 shutdown_connections 2024-05-21T21:57:15.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.371+0000 7f74d1e00700 1 --2- 172.21.15.151:0/3448488094 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74cc108840 0x7f74cc110450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:15.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.371+0000 7f74d1e00700 1 --2- 172.21.15.151:0/3448488094 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74cc103850 0x7f74cc103c30 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:15.742 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.371+0000 7f74d1e00700 1 --2- 172.21.15.151:0/3448488094 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74cc104200 0x7f74cc108160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:15.742 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.371+0000 7f74d1e00700 1 -- 172.21.15.151:0/3448488094 >> 172.21.15.151:0/3448488094 conn(0x7f74cc07a610 msgr2=0x7f74cc07aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:57:15.742 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.371+0000 7f74d1e00700 1 -- 172.21.15.151:0/3448488094 shutdown_connections 2024-05-21T21:57:15.742 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.371+0000 7f74d1e00700 1 -- 172.21.15.151:0/3448488094 wait complete. 2024-05-21T21:57:15.742 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.372+0000 7f74d1e00700 1 Processor -- start 2024-05-21T21:57:15.742 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.372+0000 7f74d1e00700 1 -- start start 2024-05-21T21:57:15.742 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.372+0000 7f74d1e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74cc103850 0x7f74cc19bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:15.742 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.372+0000 7f74d1e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74cc104200 0x7f74cc19c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:15.742 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.372+0000 7f74d1e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74cc108840 0x7f74cc1a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:15.742 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.373+0000 7f74d1e00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f74cc113770 con 0x7f74cc104200 2024-05-21T21:57:15.742 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.373+0000 7f74d1e00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f74cc1135f0 con 0x7f74cc103850 2024-05-21T21:57:15.742 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.373+0000 7f74d1e00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f74cc1138f0 con 0x7f74cc108840 2024-05-21T21:57:15.742 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.373+0000 7f74caa00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74cc103850 0x7f74cc19bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:15.743 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.373+0000 7f74ca000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74cc104200 0x7f74cc19c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:15.743 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.373+0000 7f74caa00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74cc103850 0x7f74cc19bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:33060/0 (socket says 172.21.15.151:33060) 2024-05-21T21:57:15.743 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.373+0000 7f74caa00700 1 -- 172.21.15.151:0/1054060605 learned_addr learned my addr 172.21.15.151:0/1054060605 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:57:15.743 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.373+0000 7f74cb400700 1 --2- 172.21.15.151:0/1054060605 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74cc108840 0x7f74cc1a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:15.743 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.373+0000 7f74cb400700 1 -- 172.21.15.151:0/1054060605 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74cc103850 msgr2=0x7f74cc19bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:15.743 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.373+0000 7f74cb400700 1 --2- 172.21.15.151:0/1054060605 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74cc103850 0x7f74cc19bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:15.743 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.373+0000 7f74cb400700 1 -- 172.21.15.151:0/1054060605 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74cc104200 msgr2=0x7f74cc19c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:15.743 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.373+0000 7f74cb400700 1 --2- 172.21.15.151:0/1054060605 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74cc104200 0x7f74cc19c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:15.743 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.373+0000 7f74cb400700 1 -- 172.21.15.151:0/1054060605 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74cc1a0de0 con 0x7f74cc108840 2024-05-21T21:57:15.743 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.373+0000 7f74ca000700 1 --2- 172.21.15.151:0/1054060605 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74cc104200 0x7f74cc19c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:57:15.743 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.374+0000 7f74cb400700 1 --2- 172.21.15.151:0/1054060605 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74cc108840 0x7f74cc1a06a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f74c400efc0 tx=0x7f74c400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:57:15.743 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.374+0000 7f74d0a00700 1 -- 172.21.15.151:0/1054060605 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c40092e0 con 0x7f74cc108840 2024-05-21T21:57:15.743 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.374+0000 7f74d1e00700 1 -- 172.21.15.151:0/1054060605 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74cc1a1070 con 0x7f74cc108840 2024-05-21T21:57:15.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.374+0000 7f74d0a00700 1 -- 172.21.15.151:0/1054060605 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74c4004bb0 con 0x7f74cc108840 2024-05-21T21:57:15.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.374+0000 7f74d0a00700 1 -- 172.21.15.151:0/1054060605 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c4004020 con 0x7f74cc108840 2024-05-21T21:57:15.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.374+0000 7f74d1e00700 1 -- 172.21.15.151:0/1054060605 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74cc1a8950 con 0x7f74cc108840 2024-05-21T21:57:15.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.377+0000 7f74d0a00700 1 -- 172.21.15.151:0/1054060605 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7f74c401b030 con 0x7f74cc108840 2024-05-21T21:57:15.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.379+0000 7f74dc6c9b80 1 Processor -- start 2024-05-21T21:57:15.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.379+0000 7f74d0a00700 1 --2- 172.21.15.151:0/1054060605 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f74b00b2b20 0x7f74b00b4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:15.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.379+0000 7f74dc6c9b80 1 -- start start 2024-05-21T21:57:15.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.379+0000 7f74d0a00700 1 -- 172.21.15.151:0/1054060605 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f74c40e2080 con 0x7f74cc108840 2024-05-21T21:57:15.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.380+0000 7f74caa00700 1 --2- 172.21.15.151:0/1054060605 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f74b00b2b20 0x7f74b00b4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:15.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.380+0000 7f74dc6c9b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558cdb7e4480 0x558cdb7e4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:15.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.380+0000 7f74dc6c9b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558cdb7e4da0 0x558cdb7dea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:15.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.380+0000 7f74dc6c9b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558cdb7df020 0x558cdb7df460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:15.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.380+0000 7f74dc6c9b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x558cdb73cfe0 con 0x558cdb7e4da0 2024-05-21T21:57:15.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.380+0000 7f74dc6c9b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x558cdb72cb80 con 0x558cdb7df020 2024-05-21T21:57:15.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.380+0000 7f74dc6c9b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x558cdb730bc0 con 0x558cdb7e4480 2024-05-21T21:57:15.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.380+0000 7f74caa00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558cdb7e4480 0x558cdb7e4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:15.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.380+0000 7f74caa00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558cdb7e4480 0x558cdb7e4860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:37106/0 (socket says 172.21.15.151:37106) 2024-05-21T21:57:15.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.380+0000 7f74caa00700 1 -- 172.21.15.151:0/545320538 learned_addr learned my addr 172.21.15.151:0/545320538 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:57:15.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.380+0000 7f74ca000700 1 --2- 172.21.15.151:0/545320538 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558cdb7e4da0 0x558cdb7dea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:15.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.380+0000 7f74cb400700 1 --2- 172.21.15.151:0/545320538 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558cdb7df020 0x558cdb7df460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:15.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.381+0000 7f74cb400700 1 -- 172.21.15.151:0/545320538 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558cdb7e4480 msgr2=0x558cdb7e4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:15.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.381+0000 7f74cb400700 1 --2- 172.21.15.151:0/545320538 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558cdb7e4480 0x558cdb7e4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:15.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.381+0000 7f74cb400700 1 -- 172.21.15.151:0/545320538 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558cdb7e4da0 msgr2=0x558cdb7dea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:15.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.381+0000 7f74cb400700 1 --2- 172.21.15.151:0/545320538 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558cdb7e4da0 0x558cdb7dea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:15.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.381+0000 7f74cb400700 1 -- 172.21.15.151:0/545320538 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558cdb6f55c0 con 0x558cdb7df020 2024-05-21T21:57:15.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.381+0000 7f74caa00700 1 --2- 172.21.15.151:0/545320538 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558cdb7e4480 0x558cdb7e4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:57:15.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.381+0000 7f74caa00700 1 --2- 172.21.15.151:0/1054060605 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f74b00b2b20 0x7f74b00b4fe0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f74bc000c00 tx=0x7f74bc005130 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:57:15.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.381+0000 7f74cb400700 1 --2- 172.21.15.151:0/545320538 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558cdb7df020 0x558cdb7df460 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f74c40df780 tx=0x7f74c4004d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95ef18a930be42d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:57:15.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.381+0000 7f74c0c00700 1 -- 172.21.15.151:0/545320538 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c40a93a0 con 0x558cdb7df020 2024-05-21T21:57:15.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.381+0000 7f74c0c00700 1 -- 172.21.15.151:0/545320538 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74c40ad750 con 0x558cdb7df020 2024-05-21T21:57:15.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.381+0000 7f74c0c00700 1 -- 172.21.15.151:0/545320538 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c40a4f70 con 0x558cdb7df020 2024-05-21T21:57:15.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.382+0000 7f74dc6c9b80 1 -- 172.21.15.151:0/545320538 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558cdb7df020 msgr2=0x558cdb7df460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:15.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.382+0000 7f74dc6c9b80 1 --2- 172.21.15.151:0/545320538 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558cdb7df020 0x558cdb7df460 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f74c40df780 tx=0x7f74c4004d60 comp rx=0 tx=0).stop 2024-05-21T21:57:15.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.382+0000 7f74dc6c9b80 1 -- 172.21.15.151:0/545320538 shutdown_connections 2024-05-21T21:57:15.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.382+0000 7f74dc6c9b80 1 --2- 172.21.15.151:0/545320538 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558cdb7df020 0x558cdb7df460 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:15.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.382+0000 7f74dc6c9b80 1 --2- 172.21.15.151:0/545320538 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558cdb7e4480 0x558cdb7e4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:15.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.382+0000 7f74dc6c9b80 1 --2- 172.21.15.151:0/545320538 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558cdb7e4da0 0x558cdb7dea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:15.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.382+0000 7f74dc6c9b80 1 -- 172.21.15.151:0/545320538 >> 172.21.15.151:0/545320538 conn(0x558cdb70cd80 msgr2=0x558cdb7d94c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:57:15.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.382+0000 7f74dc6c9b80 1 -- 172.21.15.151:0/545320538 shutdown_connections 2024-05-21T21:57:15.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.382+0000 7f74dc6c9b80 1 -- 172.21.15.151:0/545320538 wait complete. 2024-05-21T21:57:15.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.383+0000 7f74dc6c9b80 1 -- start start 2024-05-21T21:57:15.747 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.383+0000 7f74dc6c9b80 1 Processor -- start 2024-05-21T21:57:15.747 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.384+0000 7f74dc6c9b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558cdb7df020 0x558cdb7eab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:15.747 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.384+0000 7f74dc6c9b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558cdb7e4480 0x558cdb7e7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:15.747 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.384+0000 7f74dc6c9b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558cdb7e4da0 0x558cdb7e8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:15.747 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.384+0000 7f74dc6c9b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x558cdb633140 con 0x558cdb7e4480 2024-05-21T21:57:15.747 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.384+0000 7f74dc6c9b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x558cdb6cd900 con 0x558cdb7df020 2024-05-21T21:57:15.747 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.384+0000 7f74dc6c9b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x558cdb7eb880 con 0x558cdb7e4da0 2024-05-21T21:57:15.747 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.384+0000 7f74dc6c9b80 10 client.0 _ll_register_callbacks cb 0x558cdb6d6e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:57:15.747 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.384+0000 7f74ca000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558cdb7e4480 0x558cdb7e7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:15.747 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.384+0000 7f74caa00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558cdb7df020 0x558cdb7eab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:15.747 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.384+0000 7f74caa00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558cdb7df020 0x558cdb7eab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:33092/0 (socket says 172.21.15.151:33092) 2024-05-21T21:57:15.747 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.384+0000 7f74caa00700 1 -- 172.21.15.151:0/652509086 learned_addr learned my addr 172.21.15.151:0/652509086 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:57:15.747 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.385+0000 7f74cb400700 1 --2- 172.21.15.151:0/652509086 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558cdb7e4da0 0x558cdb7e8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:15.748 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.385+0000 7f74caa00700 1 -- 172.21.15.151:0/652509086 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558cdb7e4da0 msgr2=0x558cdb7e8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:15.748 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.385+0000 7f74caa00700 1 --2- 172.21.15.151:0/652509086 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558cdb7e4da0 0x558cdb7e8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:15.748 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.385+0000 7f74caa00700 1 -- 172.21.15.151:0/652509086 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558cdb7e4480 msgr2=0x558cdb7e7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:15.748 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.385+0000 7f74caa00700 1 --2- 172.21.15.151:0/652509086 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558cdb7e4480 0x558cdb7e7ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:15.748 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.385+0000 7f74caa00700 1 -- 172.21.15.151:0/652509086 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558cdb67ca70 con 0x558cdb7df020 2024-05-21T21:57:15.748 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.385+0000 7f74ca000700 1 --2- 172.21.15.151:0/652509086 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558cdb7e4480 0x558cdb7e7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:57:15.748 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.385+0000 7f74caa00700 1 --2- 172.21.15.151:0/652509086 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558cdb7df020 0x558cdb7eab50 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f74bc00b580 tx=0x7f74bc009cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41dad6fa05455c6a server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:57:15.748 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.385+0000 7f74b5600700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-21T21:57:15.748 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.385+0000 7f74b5600700 1 -- 172.21.15.151:0/652509086 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74bc035060 con 0x558cdb7df020 2024-05-21T21:57:15.748 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.385+0000 7f74b5600700 1 -- 172.21.15.151:0/652509086 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f74bc00bba0 con 0x558cdb7df020 2024-05-21T21:57:15.748 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.385+0000 7f74dc6c9b80 1 -- 172.21.15.151:0/652509086 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74c40ad750 con 0x558cdb7df020 2024-05-21T21:57:15.748 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.386+0000 7f74b5600700 1 -- 172.21.15.151:0/652509086 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f74bc0056b0 con 0x558cdb7df020 2024-05-21T21:57:15.748 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.386+0000 7f74dc6c9b80 10 client.25613 fetch_fsmap learned FSMap version 29 2024-05-21T21:57:15.748 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.386+0000 7f74dc6c9b80 1 -- 172.21.15.151:0/652509086 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558cdb6fd200 con 0x558cdb7df020 2024-05-21T21:57:15.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.386+0000 7f74b5600700 1 -- 172.21.15.151:0/652509086 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74c40ad750 con 0x558cdb7df020 2024-05-21T21:57:15.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.386+0000 7f74b5600700 1 -- 172.21.15.151:0/652509086 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f74bc02f070 con 0x558cdb7df020 2024-05-21T21:57:15.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.386+0000 7f74dc6c9b80 10 client.25613 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T21:57:15.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.386+0000 7f74dc6c9b80 10 client.25613 resolve_mds: validated gid 24530 aka daemon mds.b 2024-05-21T21:57:15.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.386+0000 7f74dc6c9b80 20 client.25613 populate_metadata read hostname 'smithi151' 2024-05-21T21:57:15.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.387+0000 7f74dc6c9b80 1 --2- 172.21.15.151:0/652509086 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x558cdb7f8510 0x558cdb7fa940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:15.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.387+0000 7f74dc6c9b80 4 client.25613 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T21:57:15.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.387+0000 7f74dc6c9b80 1 -- 172.21.15.151:0/652509086 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558cdb73ef30 con 0x558cdb7f8510 2024-05-21T21:57:15.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.387+0000 7f74ca000700 1 --2- 172.21.15.151:0/652509086 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x558cdb7f8510 0x558cdb7fa940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:15.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.387+0000 7f74ca000700 1 --2- 172.21.15.151:0/652509086 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x558cdb7f8510 0x558cdb7fa940 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13f5169a2f69aca3 server_cookie=e389c19cc3e8c845 in_seq=0 out_seq=0 2024-05-21T21:57:15.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.387+0000 7f74b5600700 10 client.25613 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T21:57:15.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.388+0000 7f74b5600700 1 -- 172.21.15.151:0/652509086 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x558cdb73ef30 con 0x558cdb7f8510 2024-05-21T21:57:15.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.388+0000 7f74b5600700 10 client.25613 handle_command_reply: tid=0 2024-05-21T21:57:15.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.388+0000 7f74dc6c9b80 1 client.25613 shutdown 2024-05-21T21:57:15.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.388+0000 7f74dc6c9b80 1 -- 172.21.15.151:0/652509086 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558cdb7df020 msgr2=0x558cdb7eab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:15.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.388+0000 7f74dc6c9b80 1 --2- 172.21.15.151:0/652509086 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558cdb7df020 0x558cdb7eab50 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f74bc00b580 tx=0x7f74bc009cd0 comp rx=0 tx=0).stop 2024-05-21T21:57:15.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.389+0000 7f74dc6c9b80 1 -- 172.21.15.151:0/652509086 shutdown_connections 2024-05-21T21:57:15.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.389+0000 7f74dc6c9b80 1 --2- 172.21.15.151:0/652509086 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x558cdb7f8510 0x558cdb7fa940 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:15.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.389+0000 7f74dc6c9b80 1 --2- 172.21.15.151:0/652509086 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558cdb7df020 0x558cdb7eab50 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:15.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.389+0000 7f74dc6c9b80 1 --2- 172.21.15.151:0/652509086 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558cdb7e4da0 0x558cdb7e8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:15.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.389+0000 7f74b5600700 0 client.25613 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T21:57:15.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.389+0000 7f74dc6c9b80 1 --2- 172.21.15.151:0/652509086 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558cdb7e4480 0x558cdb7e7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:15.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.389+0000 7f74dc6c9b80 1 -- 172.21.15.151:0/652509086 >> 172.21.15.151:0/652509086 conn(0x558cdb70cd80 msgr2=0x558cdb7dd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:57:15.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.389+0000 7f74dc6c9b80 1 -- 172.21.15.151:0/652509086 shutdown_connections 2024-05-21T21:57:15.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.389+0000 7f74dc6c9b80 1 -- 172.21.15.151:0/652509086 wait complete. 2024-05-21T21:57:15.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.389+0000 7f74dc6c9b80 20 client.25613 trim_cache size 0 max 16384 2024-05-21T21:57:15.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.411+0000 7f74dc6c9b80 1 Processor -- start 2024-05-21T21:57:15.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.411+0000 7f74dc6c9b80 1 -- start start 2024-05-21T21:57:15.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.411+0000 7f74dc6c9b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558cdb7df020 0x558cdb7e7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:15.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.411+0000 7f74dc6c9b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558cdb7e4480 0x558cdb7e83f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:15.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.411+0000 7f74dc6c9b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558cdb7e4da0 0x558cdb800250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:15.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.411+0000 7f74dc6c9b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f74cc1aff50 con 0x558cdb7e4da0 2024-05-21T21:57:15.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.411+0000 7f74dc6c9b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f74cc1b1a90 con 0x558cdb7df020 2024-05-21T21:57:15.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.411+0000 7f74dc6c9b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f74cc1b0620 con 0x558cdb7e4480 2024-05-21T21:57:15.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.412+0000 7f74cb400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558cdb7e4da0 0x558cdb800250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:15.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.412+0000 7f74cb400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558cdb7e4da0 0x558cdb800250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:47172/0 (socket says 172.21.15.151:47172) 2024-05-21T21:57:15.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.412+0000 7f74cb400700 1 -- 172.21.15.151:0/3974043265 learned_addr learned my addr 172.21.15.151:0/3974043265 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:57:15.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.412+0000 7f74ca000700 1 --2- 172.21.15.151:0/3974043265 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558cdb7e4480 0x558cdb7e83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:15.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.412+0000 7f74caa00700 1 --2- 172.21.15.151:0/3974043265 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558cdb7df020 0x558cdb7e7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:15.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.412+0000 7f74caa00700 1 -- 172.21.15.151:0/3974043265 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558cdb7e4480 msgr2=0x558cdb7e83f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:15.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.412+0000 7f74caa00700 1 --2- 172.21.15.151:0/3974043265 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558cdb7e4480 0x558cdb7e83f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:15.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.412+0000 7f74caa00700 1 -- 172.21.15.151:0/3974043265 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558cdb7e4da0 msgr2=0x558cdb800250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:15.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.412+0000 7f74caa00700 1 --2- 172.21.15.151:0/3974043265 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558cdb7e4da0 0x558cdb800250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:15.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.412+0000 7f74caa00700 1 -- 172.21.15.151:0/3974043265 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558cdb7fafc0 con 0x558cdb7df020 2024-05-21T21:57:15.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.412+0000 7f74ca000700 1 --2- 172.21.15.151:0/3974043265 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558cdb7e4480 0x558cdb7e83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:57:15.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.412+0000 7f74cb400700 1 --2- 172.21.15.151:0/3974043265 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558cdb7e4da0 0x558cdb800250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:57:15.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.413+0000 7f74caa00700 1 --2- 172.21.15.151:0/3974043265 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558cdb7df020 0x558cdb7e7eb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f74bc010e50 tx=0x7f74bc004e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b65df2b34a5ccaa4 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:57:15.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.413+0000 7f74b7e00700 1 -- 172.21.15.151:0/3974043265 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74bc035040 con 0x558cdb7df020 2024-05-21T21:57:15.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.413+0000 7f74b7e00700 1 -- 172.21.15.151:0/3974043265 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74bc00bba0 con 0x558cdb7df020 2024-05-21T21:57:15.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.413+0000 7f74b7e00700 1 -- 172.21.15.151:0/3974043265 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74bc027830 con 0x558cdb7df020 2024-05-21T21:57:15.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.414+0000 7f74dc6c9b80 1 -- 172.21.15.151:0/3974043265 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558cdb7df020 msgr2=0x558cdb7e7eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:15.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.414+0000 7f74dc6c9b80 1 --2- 172.21.15.151:0/3974043265 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558cdb7df020 0x558cdb7e7eb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f74bc010e50 tx=0x7f74bc004e10 comp rx=0 tx=0).stop 2024-05-21T21:57:15.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.414+0000 7f74dc6c9b80 1 -- 172.21.15.151:0/3974043265 shutdown_connections 2024-05-21T21:57:15.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.414+0000 7f74dc6c9b80 1 --2- 172.21.15.151:0/3974043265 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558cdb7df020 0x558cdb7e7eb0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:15.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.414+0000 7f74dc6c9b80 1 --2- 172.21.15.151:0/3974043265 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558cdb7e4da0 0x558cdb800250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:15.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.414+0000 7f74dc6c9b80 1 --2- 172.21.15.151:0/3974043265 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558cdb7e4480 0x558cdb7e83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:15.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.414+0000 7f74dc6c9b80 1 -- 172.21.15.151:0/3974043265 >> 172.21.15.151:0/3974043265 conn(0x558cdb70cd80 msgr2=0x558cdb7e67e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:57:15.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.414+0000 7f74dc6c9b80 1 -- 172.21.15.151:0/3974043265 shutdown_connections 2024-05-21T21:57:15.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.414+0000 7f74dc6c9b80 1 -- 172.21.15.151:0/3974043265 wait complete. 2024-05-21T21:57:15.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.415+0000 7f74dc6c9b80 1 -- start start 2024-05-21T21:57:15.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.415+0000 7f74dc6c9b80 1 Processor -- start 2024-05-21T21:57:15.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.415+0000 7f74dc6c9b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558cdb7df020 0x558cdb7f11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:15.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.415+0000 7f74dc6c9b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558cdb7e4480 0x558cdb7edae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:15.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.415+0000 7f74dc6c9b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558cdb7e4da0 0x558cdb7ee020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:15.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.415+0000 7f74dc6c9b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f74cc1b13c0 con 0x558cdb7e4da0 2024-05-21T21:57:15.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.415+0000 7f74dc6c9b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f74cc1b0cf0 con 0x558cdb7e4480 2024-05-21T21:57:15.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.415+0000 7f74dc6c9b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f74cc1adef0 con 0x558cdb7df020 2024-05-21T21:57:15.754 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.415+0000 7f74dc6c9b80 10 client.0 _ll_register_callbacks cb 0x558cdb6d6e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:57:15.754 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.416+0000 7f74cb400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558cdb7e4da0 0x558cdb7ee020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:15.754 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.416+0000 7f74cb400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558cdb7e4da0 0x558cdb7ee020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:47188/0 (socket says 172.21.15.151:47188) 2024-05-21T21:57:15.754 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.416+0000 7f74cb400700 1 -- 172.21.15.151:0/2434591218 learned_addr learned my addr 172.21.15.151:0/2434591218 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:57:15.754 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.416+0000 7f74ca000700 1 --2- 172.21.15.151:0/2434591218 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558cdb7e4480 0x558cdb7edae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:15.754 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.416+0000 7f74caa00700 1 --2- 172.21.15.151:0/2434591218 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558cdb7df020 0x558cdb7f11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:15.754 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.416+0000 7f74cb400700 1 -- 172.21.15.151:0/2434591218 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558cdb7df020 msgr2=0x558cdb7f11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:15.754 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.416+0000 7f74cb400700 1 --2- 172.21.15.151:0/2434591218 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558cdb7df020 0x558cdb7f11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:15.754 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.416+0000 7f74cb400700 1 -- 172.21.15.151:0/2434591218 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558cdb7e4480 msgr2=0x558cdb7edae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:15.754 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.416+0000 7f74cb400700 1 --2- 172.21.15.151:0/2434591218 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558cdb7e4480 0x558cdb7edae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:15.754 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.416+0000 7f74cb400700 1 -- 172.21.15.151:0/2434591218 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558cdb7ee860 con 0x558cdb7e4da0 2024-05-21T21:57:15.754 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.416+0000 7f74ca000700 1 --2- 172.21.15.151:0/2434591218 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558cdb7e4480 0x558cdb7edae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:57:15.754 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.417+0000 7f74cb400700 1 --2- 172.21.15.151:0/2434591218 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558cdb7e4da0 0x558cdb7ee020 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f74c40a90a0 tx=0x7f74c40dfce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edd88c6f10792b31 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:57:15.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.417+0000 7f74b5600700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-21T21:57:15.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.417+0000 7f74b5600700 1 -- 172.21.15.151:0/2434591218 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c40c3060 con 0x558cdb7e4da0 2024-05-21T21:57:15.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.417+0000 7f74b5600700 1 -- 172.21.15.151:0/2434591218 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f74c40a8810 con 0x558cdb7e4da0 2024-05-21T21:57:15.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.417+0000 7f74dc6c9b80 1 -- 172.21.15.151:0/2434591218 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74bc00bba0 con 0x558cdb7e4da0 2024-05-21T21:57:15.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.417+0000 7f74b5600700 1 -- 172.21.15.151:0/2434591218 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f74c40ca050 con 0x558cdb7e4da0 2024-05-21T21:57:15.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.417+0000 7f74dc6c9b80 10 client.15504 fetch_fsmap learned FSMap version 29 2024-05-21T21:57:15.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.417+0000 7f74dc6c9b80 1 -- 172.21.15.151:0/2434591218 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558cdb8640c0 con 0x558cdb7e4da0 2024-05-21T21:57:15.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.417+0000 7f74b5600700 1 -- 172.21.15.151:0/2434591218 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74bc00bba0 con 0x558cdb7e4da0 2024-05-21T21:57:15.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.419+0000 7f74b5600700 1 -- 172.21.15.151:0/2434591218 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f74c40a3ac0 con 0x558cdb7e4da0 2024-05-21T21:57:15.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.419+0000 7f74dc6c9b80 10 client.15504 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T21:57:15.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.419+0000 7f74dc6c9b80 10 client.15504 resolve_mds: validated gid 24530 aka daemon mds.b 2024-05-21T21:57:15.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.419+0000 7f74dc6c9b80 20 client.15504 populate_metadata read hostname 'smithi151' 2024-05-21T21:57:15.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.419+0000 7f74dc6c9b80 1 --2- 172.21.15.151:0/2434591218 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x558cdb8643a0 0x558cdb866790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:15.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.419+0000 7f74dc6c9b80 4 client.15504 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 146.15, "await_for": 45.0}] 2024-05-21T21:57:15.756 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.419+0000 7f74dc6c9b80 1 -- 172.21.15.151:0/2434591218 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 146.15, "await_for": 45.0}) -- 0x558cdb738710 con 0x558cdb8643a0 2024-05-21T21:57:15.756 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.419+0000 7f74caa00700 1 --2- 172.21.15.151:0/2434591218 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x558cdb8643a0 0x558cdb866790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:15.756 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.419+0000 7f74b5600700 10 client.15504 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T21:57:15.756 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.419+0000 7f74caa00700 1 --2- 172.21.15.151:0/2434591218 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x558cdb8643a0 0x558cdb866790 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6451338d8cdbfe8c server_cookie=499cdc208c00673 in_seq=0 out_seq=0 2024-05-21T21:57:15.757 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.430+0000 7f74b5600700 1 -- 172.21.15.151:0/2434591218 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x558cdb738710 con 0x558cdb8643a0 2024-05-21T21:57:15.757 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.430+0000 7f74b5600700 10 client.15504 handle_command_reply: tid=0 2024-05-21T21:57:15.757 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.430+0000 7f74dc6c9b80 1 client.15504 shutdown 2024-05-21T21:57:15.757 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.430+0000 7f74dc6c9b80 1 -- 172.21.15.151:0/2434591218 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558cdb7e4da0 msgr2=0x558cdb7ee020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:15.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.430+0000 7f74dc6c9b80 1 --2- 172.21.15.151:0/2434591218 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558cdb7e4da0 0x558cdb7ee020 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f74c40a90a0 tx=0x7f74c40dfce0 comp rx=0 tx=0).stop 2024-05-21T21:57:15.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.430+0000 7f74dc6c9b80 1 -- 172.21.15.151:0/2434591218 shutdown_connections 2024-05-21T21:57:15.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.430+0000 7f74dc6c9b80 1 --2- 172.21.15.151:0/2434591218 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558cdb7e4da0 0x558cdb7ee020 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:15.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.430+0000 7f74dc6c9b80 1 --2- 172.21.15.151:0/2434591218 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558cdb7df020 0x558cdb7f11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:15.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.430+0000 7f74dc6c9b80 1 --2- 172.21.15.151:0/2434591218 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x558cdb8643a0 0x558cdb866790 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:15.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.430+0000 7f74dc6c9b80 1 --2- 172.21.15.151:0/2434591218 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558cdb7e4480 0x558cdb7edae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:15.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.430+0000 7f74dc6c9b80 1 -- 172.21.15.151:0/2434591218 >> 172.21.15.151:0/2434591218 conn(0x558cdb70cd80 msgr2=0x558cdb7d2e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:57:15.758 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-05-21T21:57:15.759 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 29, 2024-05-21T21:57:15.759 INFO:teuthology.orchestra.run.smithi151.stdout: "leader": 24530, 2024-05-21T21:57:15.759 INFO:teuthology.orchestra.run.smithi151.stdout: "set_version": 2, 2024-05-21T21:57:15.759 INFO:teuthology.orchestra.run.smithi151.stdout: "sets": { 2024-05-21T21:57:15.759 INFO:teuthology.orchestra.run.smithi151.stdout: "f1b4324": { 2024-05-21T21:57:15.759 INFO:teuthology.orchestra.run.smithi151.stdout: "version": 2, 2024-05-21T21:57:15.759 INFO:teuthology.orchestra.run.smithi151.stdout: "age_ref": 0.0, 2024-05-21T21:57:15.759 INFO:teuthology.orchestra.run.smithi151.stdout: "state": { 2024-05-21T21:57:15.759 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "QUIESCED", 2024-05-21T21:57:15.759 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 0.0 2024-05-21T21:57:15.759 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T21:57:15.759 INFO:teuthology.orchestra.run.smithi151.stdout: "timeout": 90.0, 2024-05-21T21:57:15.759 INFO:teuthology.orchestra.run.smithi151.stdout: "expiration": 146.2, 2024-05-21T21:57:15.760 INFO:teuthology.orchestra.run.smithi151.stdout: "members": { 2024-05-21T21:57:15.760 INFO:teuthology.orchestra.run.smithi151.stdout: "file:/": { 2024-05-21T21:57:15.760 INFO:teuthology.orchestra.run.smithi151.stdout: "excluded": false, 2024-05-21T21:57:15.760 INFO:teuthology.orchestra.run.smithi151.stdout: "state": { 2024-05-21T21:57:15.760 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "QUIESCED", 2024-05-21T21:57:15.760 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 0.0 2024-05-21T21:57:15.760 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T21:57:15.760 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T21:57:15.760 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T21:57:15.760 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T21:57:15.760 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T21:57:15.760 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-05-21T21:57:15.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.430+0000 7f74b5600700 0 client.15504 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T21:57:15.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.430+0000 7f74dc6c9b80 1 -- 172.21.15.151:0/2434591218 shutdown_connections 2024-05-21T21:57:15.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.430+0000 7f74dc6c9b80 1 -- 172.21.15.151:0/2434591218 wait complete. 2024-05-21T21:57:15.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.430+0000 7f74dc6c9b80 20 client.15504 trim_cache size 0 max 16384 2024-05-21T21:57:15.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.431+0000 7f74d1e00700 1 -- 172.21.15.151:0/1054060605 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f74b00b2b20 msgr2=0x7f74b00b4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:15.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.431+0000 7f74d1e00700 1 --2- 172.21.15.151:0/1054060605 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f74b00b2b20 0x7f74b00b4fe0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f74bc000c00 tx=0x7f74bc005130 comp rx=0 tx=0).stop 2024-05-21T21:57:15.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.431+0000 7f74d1e00700 1 -- 172.21.15.151:0/1054060605 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74cc108840 msgr2=0x7f74cc1a06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:15.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.431+0000 7f74d1e00700 1 --2- 172.21.15.151:0/1054060605 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74cc108840 0x7f74cc1a06a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f74c400efc0 tx=0x7f74c400c3a0 comp rx=0 tx=0).stop 2024-05-21T21:57:15.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.431+0000 7f74d1e00700 1 -- 172.21.15.151:0/1054060605 shutdown_connections 2024-05-21T21:57:15.762 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.431+0000 7f74d1e00700 1 --2- 172.21.15.151:0/1054060605 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f74b00b2b20 0x7f74b00b4fe0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:15.762 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.431+0000 7f74d1e00700 1 --2- 172.21.15.151:0/1054060605 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74cc103850 0x7f74cc19bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:15.762 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.431+0000 7f74d1e00700 1 --2- 172.21.15.151:0/1054060605 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74cc108840 0x7f74cc1a06a0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:15.762 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.431+0000 7f74d1e00700 1 --2- 172.21.15.151:0/1054060605 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74cc104200 0x7f74cc19c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:15.762 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.431+0000 7f74d1e00700 1 -- 172.21.15.151:0/1054060605 >> 172.21.15.151:0/1054060605 conn(0x7f74cc07a610 msgr2=0x7f74cc102590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:57:15.762 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.432+0000 7f74d1e00700 1 -- 172.21.15.151:0/1054060605 shutdown_connections 2024-05-21T21:57:15.762 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:15.432+0000 7f74d1e00700 1 -- 172.21.15.151:0/1054060605 wait complete. 2024-05-21T21:57:15.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:57:15 smithi164 ceph-mon[36269]: pgmap v52: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 853 B/s wr, 10 op/s 2024-05-21T21:57:16.605 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'f1b4324', quiesce took 10.9 seconds. Will release after: 15.249999999999998 2024-05-21T21:57:17.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:17.191+0000 7f8d00a00700 1 -- 172.21.15.151:0/1147285136 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8cfc109540 msgr2=0x7f8cfc109920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:17.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:17.191+0000 7f8d00a00700 1 --2- 172.21.15.151:0/1147285136 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8cfc109540 0x7f8cfc109920 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f8cec0099c0 tx=0x7f8cec009cd0 comp rx=0 tx=0).stop 2024-05-21T21:57:17.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:17.191+0000 7f8d00a00700 1 -- 172.21.15.151:0/1147285136 shutdown_connections 2024-05-21T21:57:17.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:17.191+0000 7f8d00a00700 1 --2- 172.21.15.151:0/1147285136 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8cfc103e50 0x7f8cfc110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:17.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:17.191+0000 7f8d00a00700 1 --2- 172.21.15.151:0/1147285136 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8cfc109540 0x7f8cfc109920 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:17.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:17.191+0000 7f8d00a00700 1 --2- 172.21.15.151:0/1147285136 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8cfc103490 0x7f8cfc103910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:17.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:17.191+0000 7f8d00a00700 1 -- 172.21.15.151:0/1147285136 >> 172.21.15.151:0/1147285136 conn(0x7f8cfc0fefc0 msgr2=0x7f8cfc1013e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:57:17.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:17.191+0000 7f8d00a00700 1 -- 172.21.15.151:0/1147285136 shutdown_connections 2024-05-21T21:57:17.193 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:17.191+0000 7f8d00a00700 1 -- 172.21.15.151:0/1147285136 wait complete. 2024-05-21T21:57:17.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:17.192+0000 7f8d00a00700 1 Processor -- start 2024-05-21T21:57:17.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:17.192+0000 7f8d00a00700 1 -- start start 2024-05-21T21:57:17.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:17.193+0000 7f8d00a00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8cfc103490 0x7f8cfc19f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:17.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:17.193+0000 7f8d00a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8cfc103e50 0x7f8cfc19fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:17.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:17.193+0000 7f8d00a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8cfc109540 0x7f8cfc1998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:17.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:17.193+0000 7f8d00a00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8cfc1136a0 con 0x7f8cfc109540 2024-05-21T21:57:17.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:17.193+0000 7f8d00a00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8cfc113520 con 0x7f8cfc103e50 2024-05-21T21:57:17.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:17.193+0000 7f8d00a00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8cfc113820 con 0x7f8cfc103490 2024-05-21T21:57:17.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:17.193+0000 7f8cf9e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8cfc103e50 0x7f8cfc19fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:17.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:17.193+0000 7f8cfb200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8cfc109540 0x7f8cfc1998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:17.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:17.193+0000 7f8cf9e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8cfc103e50 0x7f8cfc19fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:44340/0 (socket says 172.21.15.151:44340) 2024-05-21T21:57:17.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:17.193+0000 7f8cf9e00700 1 -- 172.21.15.151:0/3323478464 learned_addr learned my addr 172.21.15.151:0/3323478464 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:57:17.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:17.193+0000 7f8cfa800700 1 --2- 172.21.15.151:0/3323478464 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8cfc103490 0x7f8cfc19f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:17.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:17.193+0000 7f8cf9e00700 1 -- 172.21.15.151:0/3323478464 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8cfc103490 msgr2=0x7f8cfc19f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:17.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:17.193+0000 7f8cf9e00700 1 --2- 172.21.15.151:0/3323478464 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8cfc103490 0x7f8cfc19f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:17.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:17.193+0000 7f8cf9e00700 1 -- 172.21.15.151:0/3323478464 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8cfc109540 msgr2=0x7f8cfc1998c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:17.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:17.193+0000 7f8cf9e00700 1 --2- 172.21.15.151:0/3323478464 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8cfc109540 0x7f8cfc1998c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:17.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:17.193+0000 7f8cf9e00700 1 -- 172.21.15.151:0/3323478464 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cfc19a020 con 0x7f8cfc103e50 2024-05-21T21:57:17.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:17.194+0000 7f8cf9e00700 1 --2- 172.21.15.151:0/3323478464 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8cfc103e50 0x7f8cfc19fc40 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8ce800d980 tx=0x7f8ce800dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:57:17.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:17.194+0000 7f8cf3400700 1 -- 172.21.15.151:0/3323478464 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce8016070 con 0x7f8cfc103e50 2024-05-21T21:57:17.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:17.194+0000 7f8cf3400700 1 -- 172.21.15.151:0/3323478464 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ce8004830 con 0x7f8cfc103e50 2024-05-21T21:57:17.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:17.194+0000 7f8d00a00700 1 -- 172.21.15.151:0/3323478464 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cfc19a310 con 0x7f8cfc103e50 2024-05-21T21:57:17.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:17.194+0000 7f8d00a00700 1 -- 172.21.15.151:0/3323478464 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cfc075770 con 0x7f8cfc103e50 2024-05-21T21:57:17.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:17.194+0000 7f8cf3400700 1 -- 172.21.15.151:0/3323478464 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce8005540 con 0x7f8cfc103e50 2024-05-21T21:57:17.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:17.195+0000 7f8d00a00700 1 -- 172.21.15.151:0/3323478464 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cdc005330 con 0x7f8cfc103e50 2024-05-21T21:57:17.198 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:17.198+0000 7f8cf3400700 1 -- 172.21.15.151:0/3323478464 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7f8ce8004bc0 con 0x7f8cfc103e50 2024-05-21T21:57:17.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:17.200+0000 7f8cf3400700 1 --2- 172.21.15.151:0/3323478464 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f8ce00b29f0 0x7f8ce00b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:17.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:17.200+0000 7f8cf3400700 1 -- 172.21.15.151:0/3323478464 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f8ce80e2850 con 0x7f8cfc103e50 2024-05-21T21:57:17.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:17.200+0000 7f8cfa800700 1 --2- 172.21.15.151:0/3323478464 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f8ce00b29f0 0x7f8ce00b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:17.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:17.201+0000 7f8cfa800700 1 --2- 172.21.15.151:0/3323478464 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f8ce00b29f0 0x7f8ce00b4eb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8cec000c00 tx=0x7f8cec005fb0 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:57:17.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:17.202+0000 7f8cf3400700 1 -- 172.21.15.151:0/3323478464 <== mon.2 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8ce801f020 con 0x7f8cfc103e50 2024-05-21T21:57:17.452 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:57:17 smithi151 ceph-mon[31623]: pgmap v53: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 853 B/s wr, 10 op/s 2024-05-21T21:57:17.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:17.506+0000 7f8d00a00700 1 -- 172.21.15.151:0/3323478464 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8cdc005620 con 0x7f8cfc103e50 2024-05-21T21:57:17.507 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:17.507+0000 7f8cf3400700 1 -- 172.21.15.151:0/3323478464 <== mon.2 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11958 (secure 0 0 0) 0x7f8ce80aa1e0 con 0x7f8cfc103e50 2024-05-21T21:57:17.508 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T21:57:17.508 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14358,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/158010529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":158010529},{"type":"v1","addr":"172.21.15.151:6835","nonce":158010529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6839/2069058115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":2069058115},{"type":"v1","addr":"172.21.15.151:6839","nonce":2069058115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T21:51:19.552069+0000","modified":"2024-05-21T21:55:34.643755+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24530,"mds_1":24493,"mds_2":24491,"mds_3":24478,"mds_4":24473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14403":{"gid":14403,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6841/1749723794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6840","nonce":1749723794},{"type":"v1","addr":"172.21.15.151:6841","nonce":1749723794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/2026266114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":2026266114},{"type":"v1","addr":"172.21.15.184:6833","nonce":2026266114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/2467900844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":2467900844},{"type":"v1","addr":"172.21.15.164:6833","nonce":2467900844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3367640960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3367640960},{"type":"v1","addr":"172.21.15.151:6837","nonce":3367640960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/3264142621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3264142621},{"type":"v1","addr":"172.21.15.184:6835","nonce":3264142621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":17,"addr":"172.21.15.164:6835/2340428470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2340428470},{"type":"v1","addr":"172.21.15.164:6835","nonce":2340428470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":16,"addr":"172.21.15.184:6837/2988194888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":2988194888},{"type":"v1","addr":"172.21.15.184:6837","nonce":2988194888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.164:6837/1554609572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1554609572},{"type":"v1","addr":"172.21.15.164:6837","nonce":1554609572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.184:6839/3059935143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":3059935143},{"type":"v1","addr":"172.21.15.184:6839","nonce":3059935143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.164:6839/2646401247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":2646401247},{"type":"v1","addr":"172.21.15.164:6839","nonce":2646401247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24530,"qdb_cluster":[24530,24493,24491,24473,24478]},"id":1}]} 2024-05-21T21:57:17.510 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:17.510+0000 7f8d00a00700 1 -- 172.21.15.151:0/3323478464 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f8ce00b29f0 msgr2=0x7f8ce00b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:17.510 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:17.510+0000 7f8d00a00700 1 --2- 172.21.15.151:0/3323478464 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f8ce00b29f0 0x7f8ce00b4eb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8cec000c00 tx=0x7f8cec005fb0 comp rx=0 tx=0).stop 2024-05-21T21:57:17.510 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:17.510+0000 7f8d00a00700 1 -- 172.21.15.151:0/3323478464 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8cfc103e50 msgr2=0x7f8cfc19fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:17.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:17.510+0000 7f8d00a00700 1 --2- 172.21.15.151:0/3323478464 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8cfc103e50 0x7f8cfc19fc40 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8ce800d980 tx=0x7f8ce800dc90 comp rx=0 tx=0).stop 2024-05-21T21:57:17.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:17.510+0000 7f8d00a00700 1 -- 172.21.15.151:0/3323478464 shutdown_connections 2024-05-21T21:57:17.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:17.510+0000 7f8d00a00700 1 --2- 172.21.15.151:0/3323478464 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f8ce00b29f0 0x7f8ce00b4eb0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:17.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:17.510+0000 7f8d00a00700 1 --2- 172.21.15.151:0/3323478464 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8cfc109540 0x7f8cfc1998c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:17.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:17.510+0000 7f8d00a00700 1 --2- 172.21.15.151:0/3323478464 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8cfc103490 0x7f8cfc19f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:17.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:17.510+0000 7f8d00a00700 1 --2- 172.21.15.151:0/3323478464 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8cfc103e50 0x7f8cfc19fc40 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:17.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:17.510+0000 7f8d00a00700 1 -- 172.21.15.151:0/3323478464 >> 172.21.15.151:0/3323478464 conn(0x7f8cfc0fefc0 msgr2=0x7f8cfc1005c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:57:17.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:17.510+0000 7f8d00a00700 1 -- 172.21.15.151:0/3323478464 shutdown_connections 2024-05-21T21:57:17.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:17.510+0000 7f8d00a00700 1 -- 172.21.15.151:0/3323478464 wait complete. 2024-05-21T21:57:17.512 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 29 2024-05-21T21:57:17.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:57:17 smithi184 ceph-mon[36942]: pgmap v53: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 853 B/s wr, 10 op/s 2024-05-21T21:57:17.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:57:17 smithi164 ceph-mon[36269]: pgmap v53: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 853 B/s wr, 10 op/s 2024-05-21T21:57:18.380 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-21T21:57:18.702 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:57:18 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/3323478464' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T21:57:18.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:57:18 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/3323478464' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T21:57:18.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:57:18 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/3323478464' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T21:57:18.881 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:57:19.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:57:19 smithi151 ceph-mon[31623]: pgmap v54: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 853 B/s wr, 14 op/s 2024-05-21T21:57:19.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:57:19 smithi184 ceph-mon[36942]: pgmap v54: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 853 B/s wr, 14 op/s 2024-05-21T21:57:19.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:57:19 smithi164 ceph-mon[36269]: pgmap v54: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 853 B/s wr, 14 op/s 2024-05-21T21:57:20.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.860+0000 7f29f5c00700 1 -- 172.21.15.151:0/4161558874 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f29f01076b0 msgr2=0x7f29f010f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:20.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.860+0000 7f29f5c00700 1 --2- 172.21.15.151:0/4161558874 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f29f01076b0 0x7f29f010f280 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f29e8009f90 tx=0x7f29e800b330 comp rx=0 tx=0).stop 2024-05-21T21:57:20.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.860+0000 7f29f5c00700 1 -- 172.21.15.151:0/4161558874 shutdown_connections 2024-05-21T21:57:20.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.860+0000 7f29f5c00700 1 --2- 172.21.15.151:0/4161558874 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f29f01076b0 0x7f29f010f280 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:20.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.860+0000 7f29f5c00700 1 --2- 172.21.15.151:0/4161558874 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f29f01025d0 0x7f29f01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:20.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.860+0000 7f29f5c00700 1 --2- 172.21.15.151:0/4161558874 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f29f0102f80 0x7f29f0106fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:20.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.860+0000 7f29f5c00700 1 -- 172.21.15.151:0/4161558874 >> 172.21.15.151:0/4161558874 conn(0x7f29f00fde60 msgr2=0x7f29f0100280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:57:20.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.861+0000 7f29f5c00700 1 -- 172.21.15.151:0/4161558874 shutdown_connections 2024-05-21T21:57:20.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.861+0000 7f29f5c00700 1 -- 172.21.15.151:0/4161558874 wait complete. 2024-05-21T21:57:20.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.862+0000 7f29f5c00700 1 Processor -- start 2024-05-21T21:57:20.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.862+0000 7f29f5c00700 1 -- start start 2024-05-21T21:57:20.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.863+0000 7f29f5c00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f29f01025d0 0x7f29f01a4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:20.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.863+0000 7f29f5c00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f29f0102f80 0x7f29f01a5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:20.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.863+0000 7f29f5c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f29f01076b0 0x7f29f01a02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:20.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.863+0000 7f29f5c00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f29f01125c0 con 0x7f29f01076b0 2024-05-21T21:57:20.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.863+0000 7f29f5c00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f29f0112440 con 0x7f29f0102f80 2024-05-21T21:57:20.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.863+0000 7f29f5c00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f29f0112740 con 0x7f29f01025d0 2024-05-21T21:57:20.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.863+0000 7f29ef400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f29f0102f80 0x7f29f01a5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:20.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.863+0000 7f29efe00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f29f01025d0 0x7f29f01a4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:20.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.863+0000 7f29ef400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f29f0102f80 0x7f29f01a5270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:44348/0 (socket says 172.21.15.151:44348) 2024-05-21T21:57:20.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.864+0000 7f29ef400700 1 -- 172.21.15.151:0/1096743825 learned_addr learned my addr 172.21.15.151:0/1096743825 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:57:20.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.864+0000 7f29f5200700 1 --2- 172.21.15.151:0/1096743825 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f29f01076b0 0x7f29f01a02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:20.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.864+0000 7f29f5200700 1 -- 172.21.15.151:0/1096743825 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f29f01025d0 msgr2=0x7f29f01a4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:20.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.864+0000 7f29f5200700 1 --2- 172.21.15.151:0/1096743825 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f29f01025d0 0x7f29f01a4d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:20.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.864+0000 7f29f5200700 1 -- 172.21.15.151:0/1096743825 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f29f0102f80 msgr2=0x7f29f01a5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:20.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.864+0000 7f29f5200700 1 --2- 172.21.15.151:0/1096743825 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f29f0102f80 0x7f29f01a5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:20.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.864+0000 7f29f5200700 1 -- 172.21.15.151:0/1096743825 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29f01a0a80 con 0x7f29f01076b0 2024-05-21T21:57:20.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.864+0000 7f29efe00700 1 --2- 172.21.15.151:0/1096743825 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f29f01025d0 0x7f29f01a4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:57:20.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.864+0000 7f29f5200700 1 --2- 172.21.15.151:0/1096743825 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f29f01076b0 0x7f29f01a02a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f29e800b6c0 tx=0x7f29e800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:57:20.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.865+0000 7f29ecc00700 1 -- 172.21.15.151:0/1096743825 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e8021070 con 0x7f29f01076b0 2024-05-21T21:57:20.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.865+0000 7f29f5c00700 1 -- 172.21.15.151:0/1096743825 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29f01a0d10 con 0x7f29f01076b0 2024-05-21T21:57:20.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.865+0000 7f29ecc00700 1 -- 172.21.15.151:0/1096743825 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29e8007d60 con 0x7f29f01076b0 2024-05-21T21:57:20.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.865+0000 7f29ecc00700 1 -- 172.21.15.151:0/1096743825 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e8005040 con 0x7f29f01076b0 2024-05-21T21:57:20.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.865+0000 7f29f5c00700 1 -- 172.21.15.151:0/1096743825 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29f01acc60 con 0x7f29f01076b0 2024-05-21T21:57:20.871 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.871+0000 7f2a01c0fb80 1 Processor -- start 2024-05-21T21:57:20.872 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.871+0000 7f29ecc00700 1 -- 172.21.15.151:0/1096743825 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7f29e8024070 con 0x7f29f01076b0 2024-05-21T21:57:20.872 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.872+0000 7f2a01c0fb80 1 -- start start 2024-05-21T21:57:20.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.872+0000 7f2a01c0fb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5603edf43020 0x5603edf43400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:20.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.872+0000 7f2a01c0fb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5603edf43940 0x5603edf3d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:20.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.872+0000 7f2a01c0fb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603edf3dad0 0x5603edf3df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:20.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.872+0000 7f2a01c0fb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5603ede9be40 con 0x5603edf3dad0 2024-05-21T21:57:20.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.872+0000 7f2a01c0fb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5603ede8b9e0 con 0x5603edf43020 2024-05-21T21:57:20.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.872+0000 7f2a01c0fb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5603ede8fa20 con 0x5603edf43940 2024-05-21T21:57:20.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.873+0000 7f29ef400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5603edf43020 0x5603edf43400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:20.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.873+0000 7f29efe00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5603edf43940 0x5603edf3d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:20.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.873+0000 7f29ef400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603edf3dad0 0x5603edf3df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:20.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.873+0000 7f29ef400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603edf3dad0 0x5603edf3df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:37464/0 (socket says 172.21.15.151:37464) 2024-05-21T21:57:20.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.873+0000 7f29ef400700 1 -- 172.21.15.151:0/1170258800 learned_addr learned my addr 172.21.15.151:0/1170258800 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:57:20.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.873+0000 7f29efe00700 1 -- 172.21.15.151:0/1170258800 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5603edf43020 msgr2=0x5603edf43400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:20.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.873+0000 7f29efe00700 1 --2- 172.21.15.151:0/1170258800 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5603edf43020 0x5603edf43400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:20.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.873+0000 7f29efe00700 1 -- 172.21.15.151:0/1170258800 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603edf3dad0 msgr2=0x5603edf3df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:20.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.873+0000 7f29efe00700 1 --2- 172.21.15.151:0/1170258800 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603edf3dad0 0x5603edf3df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:20.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.873+0000 7f29efe00700 1 -- 172.21.15.151:0/1170258800 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603ede54420 con 0x5603edf43940 2024-05-21T21:57:20.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.873+0000 7f29ef400700 1 --2- 172.21.15.151:0/1170258800 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603edf3dad0 0x5603edf3df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:57:20.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.874+0000 7f29efe00700 1 --2- 172.21.15.151:0/1170258800 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5603edf43940 0x5603edf3d4b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f29e00114c0 tx=0x7f29e00117d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f04e74d69c7b5222 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:57:20.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.875+0000 7f29e5600700 1 -- 172.21.15.151:0/1170258800 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e00042f0 con 0x5603edf43940 2024-05-21T21:57:20.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.875+0000 7f29e5600700 1 -- 172.21.15.151:0/1170258800 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29e0011e50 con 0x5603edf43940 2024-05-21T21:57:20.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.875+0000 7f29e5600700 1 -- 172.21.15.151:0/1170258800 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e0005560 con 0x5603edf43940 2024-05-21T21:57:20.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.876+0000 7f2a01c0fb80 1 -- 172.21.15.151:0/1170258800 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5603edf43940 msgr2=0x5603edf3d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:20.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.876+0000 7f2a01c0fb80 1 --2- 172.21.15.151:0/1170258800 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5603edf43940 0x5603edf3d4b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f29e00114c0 tx=0x7f29e00117d0 comp rx=0 tx=0).stop 2024-05-21T21:57:20.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.876+0000 7f29ecc00700 1 --2- 172.21.15.151:0/1096743825 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f29d40b2a70 0x7f29d40b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:20.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.876+0000 7f2a01c0fb80 1 -- 172.21.15.151:0/1170258800 shutdown_connections 2024-05-21T21:57:20.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.876+0000 7f2a01c0fb80 1 --2- 172.21.15.151:0/1170258800 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5603edf43020 0x5603edf43400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:20.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.876+0000 7f2a01c0fb80 1 --2- 172.21.15.151:0/1170258800 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603edf3dad0 0x5603edf3df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:20.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.876+0000 7f2a01c0fb80 1 --2- 172.21.15.151:0/1170258800 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5603edf43940 0x5603edf3d4b0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:20.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.876+0000 7f2a01c0fb80 1 -- 172.21.15.151:0/1170258800 >> 172.21.15.151:0/1170258800 conn(0x5603ede6bbe0 msgr2=0x5603edf38150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:57:20.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.876+0000 7f29ef400700 1 --2- 172.21.15.151:0/1096743825 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f29d40b2a70 0x7f29d40b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:20.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.876+0000 7f2a01c0fb80 1 -- 172.21.15.151:0/1170258800 shutdown_connections 2024-05-21T21:57:20.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.876+0000 7f2a01c0fb80 1 -- 172.21.15.151:0/1170258800 wait complete. 2024-05-21T21:57:20.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.877+0000 7f29ecc00700 1 -- 172.21.15.151:0/1096743825 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f29e800f020 con 0x7f29f01076b0 2024-05-21T21:57:20.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.877+0000 7f29ef400700 1 --2- 172.21.15.151:0/1096743825 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f29d40b2a70 0x7f29d40b4f30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f29dc009ac0 tx=0x7f29dc0047c0 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:57:20.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.877+0000 7f2a01c0fb80 1 -- start start 2024-05-21T21:57:20.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.877+0000 7f2a01c0fb80 1 Processor -- start 2024-05-21T21:57:20.888 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.878+0000 7f2a01c0fb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603edf3dad0 0x5603edf49580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:20.889 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.878+0000 7f2a01c0fb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5603edf43020 0x5603edf46920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:20.889 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.878+0000 7f2a01c0fb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5603edf43940 0x5603edf46e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:20.889 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.878+0000 7f2a01c0fb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5603edd91fa0 con 0x5603edf3dad0 2024-05-21T21:57:20.889 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.878+0000 7f2a01c0fb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5603ede2c760 con 0x5603edf43940 2024-05-21T21:57:20.889 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.878+0000 7f2a01c0fb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5603edf4a460 con 0x5603edf43020 2024-05-21T21:57:20.889 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.878+0000 7f2a01c0fb80 10 client.0 _ll_register_callbacks cb 0x5603ede35c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:57:20.890 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.878+0000 7f29efe00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603edf3dad0 0x5603edf49580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:20.890 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.879+0000 7f29efe00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603edf3dad0 0x5603edf49580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:37466/0 (socket says 172.21.15.151:37466) 2024-05-21T21:57:20.890 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.879+0000 7f29efe00700 1 -- 172.21.15.151:0/1019064605 learned_addr learned my addr 172.21.15.151:0/1019064605 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:57:20.890 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.879+0000 7f29f5200700 1 --2- 172.21.15.151:0/1019064605 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5603edf43940 0x5603edf46e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:20.890 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.879+0000 7f29ef400700 1 --2- 172.21.15.151:0/1019064605 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5603edf43020 0x5603edf46920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:20.890 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.879+0000 7f29f5200700 1 -- 172.21.15.151:0/1019064605 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5603edf43020 msgr2=0x5603edf46920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:20.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.879+0000 7f29f5200700 1 --2- 172.21.15.151:0/1019064605 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5603edf43020 0x5603edf46920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:20.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.879+0000 7f29f5200700 1 -- 172.21.15.151:0/1019064605 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603edf3dad0 msgr2=0x5603edf49580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:20.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.879+0000 7f29f5200700 1 --2- 172.21.15.151:0/1019064605 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603edf3dad0 0x5603edf49580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:20.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.879+0000 7f29f5200700 1 -- 172.21.15.151:0/1019064605 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603edddb8d0 con 0x5603edf43940 2024-05-21T21:57:20.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.880+0000 7f29efe00700 1 --2- 172.21.15.151:0/1019064605 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603edf3dad0 0x5603edf49580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:57:20.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.880+0000 7f29f5200700 1 --2- 172.21.15.151:0/1019064605 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5603edf43940 0x5603edf46e60 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f29e80e0f50 tx=0x7f29e80aa5b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81677c1d5c6ec61 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:57:20.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.880+0000 7f29da000700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-21T21:57:20.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.880+0000 7f29da000700 1 -- 172.21.15.151:0/1019064605 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e80ab270 con 0x5603edf43940 2024-05-21T21:57:20.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.880+0000 7f29da000700 1 -- 172.21.15.151:0/1019064605 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f29e801d690 con 0x5603edf43940 2024-05-21T21:57:20.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.880+0000 7f2a01c0fb80 1 -- 172.21.15.151:0/1019064605 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29e0011e50 con 0x5603edf43940 2024-05-21T21:57:20.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.881+0000 7f29da000700 1 -- 172.21.15.151:0/1019064605 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f29e80aab70 con 0x5603edf43940 2024-05-21T21:57:20.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.881+0000 7f2a01c0fb80 10 client.25637 fetch_fsmap learned FSMap version 29 2024-05-21T21:57:20.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.881+0000 7f2a01c0fb80 1 -- 172.21.15.151:0/1019064605 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5603ede5c060 con 0x5603edf43940 2024-05-21T21:57:20.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.881+0000 7f29da000700 1 -- 172.21.15.151:0/1019064605 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29e0011e50 con 0x5603edf43940 2024-05-21T21:57:20.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.881+0000 7f29da000700 1 -- 172.21.15.151:0/1019064605 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f29e8011070 con 0x5603edf43940 2024-05-21T21:57:20.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.882+0000 7f2a01c0fb80 10 client.25637 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T21:57:20.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.882+0000 7f2a01c0fb80 10 client.25637 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-05-21T21:57:20.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.882+0000 7f2a01c0fb80 20 client.25637 populate_metadata read hostname 'smithi151' 2024-05-21T21:57:20.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.882+0000 7f2a01c0fb80 1 --2- 172.21.15.151:0/1019064605 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x5603edf56f40 0x5603edf59380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:20.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.882+0000 7f2a01c0fb80 4 client.25637 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T21:57:20.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.882+0000 7f2a01c0fb80 1 -- 172.21.15.151:0/1019064605 --> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5603ede9dd90 con 0x5603edf56f40 2024-05-21T21:57:20.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.883+0000 7f29efe00700 1 --2- 172.21.15.151:0/1019064605 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x5603edf56f40 0x5603edf59380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:20.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.883+0000 7f29da000700 10 client.25637 ms_handle_connect on v2:172.21.15.164:6834/2340428470 2024-05-21T21:57:20.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.883+0000 7f29efe00700 1 --2- 172.21.15.151:0/1019064605 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x5603edf56f40 0x5603edf59380 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ce94abd44aeacd51 server_cookie=1f782e0d184e9a56 in_seq=0 out_seq=0 2024-05-21T21:57:20.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.884+0000 7f29da000700 1 -- 172.21.15.151:0/1019064605 <== mds.4 v2:172.21.15.164:6834/2340428470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5603ede9dd90 con 0x5603edf56f40 2024-05-21T21:57:20.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.884+0000 7f29da000700 10 client.25637 handle_command_reply: tid=0 2024-05-21T21:57:20.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.884+0000 7f2a01c0fb80 1 client.25637 shutdown 2024-05-21T21:57:20.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.884+0000 7f2a01c0fb80 1 -- 172.21.15.151:0/1019064605 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5603edf43940 msgr2=0x5603edf46e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:20.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.884+0000 7f2a01c0fb80 1 --2- 172.21.15.151:0/1019064605 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5603edf43940 0x5603edf46e60 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f29e80e0f50 tx=0x7f29e80aa5b0 comp rx=0 tx=0).stop 2024-05-21T21:57:20.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.884+0000 7f2a01c0fb80 1 -- 172.21.15.151:0/1019064605 shutdown_connections 2024-05-21T21:57:20.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.884+0000 7f2a01c0fb80 1 --2- 172.21.15.151:0/1019064605 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x5603edf56f40 0x5603edf59380 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:20.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.884+0000 7f2a01c0fb80 1 --2- 172.21.15.151:0/1019064605 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5603edf43940 0x5603edf46e60 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:20.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.884+0000 7f29da000700 0 client.25637 ms_handle_reset on v2:172.21.15.164:6834/2340428470 2024-05-21T21:57:20.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.884+0000 7f2a01c0fb80 1 --2- 172.21.15.151:0/1019064605 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603edf3dad0 0x5603edf49580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:20.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.884+0000 7f2a01c0fb80 1 --2- 172.21.15.151:0/1019064605 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5603edf43020 0x5603edf46920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:20.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.884+0000 7f2a01c0fb80 1 -- 172.21.15.151:0/1019064605 >> 172.21.15.151:0/1019064605 conn(0x5603ede6bbe0 msgr2=0x5603edf3c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:57:20.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.885+0000 7f2a01c0fb80 1 -- 172.21.15.151:0/1019064605 shutdown_connections 2024-05-21T21:57:20.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.885+0000 7f2a01c0fb80 1 -- 172.21.15.151:0/1019064605 wait complete. 2024-05-21T21:57:20.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.885+0000 7f2a01c0fb80 20 client.25637 trim_cache size 0 max 16384 2024-05-21T21:57:20.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.901+0000 7f2a01c0fb80 1 Processor -- start 2024-05-21T21:57:20.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.901+0000 7f2a01c0fb80 1 -- start start 2024-05-21T21:57:20.903 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.902+0000 7f2a01c0fb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5603edf3dad0 0x5603edf3d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:20.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.902+0000 7f2a01c0fb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5603edf43020 0x5603edf45210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:20.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.902+0000 7f2a01c0fb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603edf43940 0x5603edf45750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:20.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.902+0000 7f2a01c0fb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f29f01b4180 con 0x5603edf43940 2024-05-21T21:57:20.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.902+0000 7f2a01c0fb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f29f01b5cc0 con 0x5603edf43020 2024-05-21T21:57:20.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.902+0000 7f2a01c0fb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f29f01b4850 con 0x5603edf3dad0 2024-05-21T21:57:20.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.902+0000 7f29efe00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5603edf3dad0 0x5603edf3d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:20.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.902+0000 7f29efe00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5603edf3dad0 0x5603edf3d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:33546/0 (socket says 172.21.15.151:33546) 2024-05-21T21:57:20.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.902+0000 7f29efe00700 1 -- 172.21.15.151:0/903129968 learned_addr learned my addr 172.21.15.151:0/903129968 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:57:20.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.902+0000 7f29f5200700 1 --2- 172.21.15.151:0/903129968 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603edf43940 0x5603edf45750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:20.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.903+0000 7f29ef400700 1 --2- 172.21.15.151:0/903129968 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5603edf43020 0x5603edf45210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:20.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.903+0000 7f29efe00700 1 -- 172.21.15.151:0/903129968 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5603edf43020 msgr2=0x5603edf45210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:20.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.903+0000 7f29efe00700 1 --2- 172.21.15.151:0/903129968 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5603edf43020 0x5603edf45210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:20.906 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.903+0000 7f29efe00700 1 -- 172.21.15.151:0/903129968 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603edf43940 msgr2=0x5603edf45750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:20.906 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.903+0000 7f29efe00700 1 --2- 172.21.15.151:0/903129968 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603edf43940 0x5603edf45750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:20.906 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.903+0000 7f29efe00700 1 -- 172.21.15.151:0/903129968 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603edf44e90 con 0x5603edf3dad0 2024-05-21T21:57:20.906 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.903+0000 7f29efe00700 1 --2- 172.21.15.151:0/903129968 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5603edf3dad0 0x5603edf3d4b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f29e00097a0 tx=0x7f29e0005ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3cd20e92122def89 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:57:20.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.904+0000 7f29e4c00700 1 -- 172.21.15.151:0/903129968 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e001e040 con 0x5603edf3dad0 2024-05-21T21:57:20.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.904+0000 7f29e4c00700 1 -- 172.21.15.151:0/903129968 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29e001f040 con 0x5603edf3dad0 2024-05-21T21:57:20.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.904+0000 7f29e4c00700 1 -- 172.21.15.151:0/903129968 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e00047c0 con 0x5603edf3dad0 2024-05-21T21:57:20.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.905+0000 7f2a01c0fb80 1 -- 172.21.15.151:0/903129968 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5603edf3dad0 msgr2=0x5603edf3d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:20.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.905+0000 7f2a01c0fb80 1 --2- 172.21.15.151:0/903129968 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5603edf3dad0 0x5603edf3d4b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f29e00097a0 tx=0x7f29e0005ce0 comp rx=0 tx=0).stop 2024-05-21T21:57:20.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.905+0000 7f2a01c0fb80 1 -- 172.21.15.151:0/903129968 shutdown_connections 2024-05-21T21:57:20.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.905+0000 7f2a01c0fb80 1 --2- 172.21.15.151:0/903129968 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603edf43940 0x5603edf45750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:20.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.905+0000 7f2a01c0fb80 1 --2- 172.21.15.151:0/903129968 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5603edf3dad0 0x5603edf3d4b0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:20.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.905+0000 7f2a01c0fb80 1 --2- 172.21.15.151:0/903129968 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5603edf43020 0x5603edf45210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:20.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.905+0000 7f2a01c0fb80 1 -- 172.21.15.151:0/903129968 >> 172.21.15.151:0/903129968 conn(0x5603ede6bbe0 msgr2=0x5603edf31fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:57:20.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.905+0000 7f2a01c0fb80 1 -- 172.21.15.151:0/903129968 shutdown_connections 2024-05-21T21:57:20.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.905+0000 7f2a01c0fb80 1 -- 172.21.15.151:0/903129968 wait complete. 2024-05-21T21:57:20.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.906+0000 7f2a01c0fb80 1 -- start start 2024-05-21T21:57:20.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.906+0000 7f2a01c0fb80 1 Processor -- start 2024-05-21T21:57:20.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.906+0000 7f2a01c0fb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5603edf3dad0 0x5603edf4fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:20.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.907+0000 7f2a01c0fb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5603edf43020 0x5603edf5e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:20.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.907+0000 7f2a01c0fb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603edf43940 0x5603edf5ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:20.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.907+0000 7f2a01c0fb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f29f01b55f0 con 0x5603edf43940 2024-05-21T21:57:20.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.907+0000 7f2a01c0fb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f29f01b4f20 con 0x5603edf3dad0 2024-05-21T21:57:20.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.907+0000 7f2a01c0fb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f29f01b2140 con 0x5603edf43020 2024-05-21T21:57:20.915 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.907+0000 7f2a01c0fb80 10 client.0 _ll_register_callbacks cb 0x5603ede35c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:57:20.915 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.907+0000 7f29efe00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5603edf3dad0 0x5603edf4fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:20.915 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.907+0000 7f29efe00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5603edf3dad0 0x5603edf4fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:44372/0 (socket says 172.21.15.151:44372) 2024-05-21T21:57:20.915 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.907+0000 7f29efe00700 1 -- 172.21.15.151:0/1356696118 learned_addr learned my addr 172.21.15.151:0/1356696118 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:57:20.915 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.907+0000 7f29ef400700 1 --2- 172.21.15.151:0/1356696118 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5603edf43020 0x5603edf5e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:20.915 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.907+0000 7f29efe00700 1 -- 172.21.15.151:0/1356696118 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5603edf43020 msgr2=0x5603edf5e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:20.915 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.907+0000 7f29efe00700 1 --2- 172.21.15.151:0/1356696118 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5603edf43020 0x5603edf5e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:20.915 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.907+0000 7f29efe00700 1 -- 172.21.15.151:0/1356696118 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603edf43940 msgr2=0x5603edf5ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:20.915 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.907+0000 7f29f5200700 1 --2- 172.21.15.151:0/1356696118 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603edf43940 0x5603edf5ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:20.915 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.907+0000 7f29efe00700 1 --2- 172.21.15.151:0/1356696118 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603edf43940 0x5603edf5ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:20.915 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.907+0000 7f29efe00700 1 -- 172.21.15.151:0/1356696118 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603edf5f340 con 0x5603edf3dad0 2024-05-21T21:57:20.915 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.907+0000 7f29f5200700 1 --2- 172.21.15.151:0/1356696118 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603edf43940 0x5603edf5ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T21:57:20.916 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.908+0000 7f29efe00700 1 --2- 172.21.15.151:0/1356696118 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5603edf3dad0 0x5603edf4fe90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f29e0011b20 tx=0x7f29e0005570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab4b33347ef36792 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:57:20.916 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.908+0000 7f29da000700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-21T21:57:20.916 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.908+0000 7f29da000700 1 -- 172.21.15.151:0/1356696118 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e0003a90 con 0x5603edf3dad0 2024-05-21T21:57:20.916 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.908+0000 7f29da000700 1 -- 172.21.15.151:0/1356696118 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f29e0003c30 con 0x5603edf3dad0 2024-05-21T21:57:20.916 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.908+0000 7f2a01c0fb80 1 -- 172.21.15.151:0/1356696118 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29e001f040 con 0x5603edf3dad0 2024-05-21T21:57:20.916 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.908+0000 7f29da000700 1 -- 172.21.15.151:0/1356696118 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f29e0004560 con 0x5603edf3dad0 2024-05-21T21:57:20.916 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.909+0000 7f2a01c0fb80 10 client.25643 fetch_fsmap learned FSMap version 29 2024-05-21T21:57:20.916 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.909+0000 7f2a01c0fb80 1 -- 172.21.15.151:0/1356696118 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5603edfc1bd0 con 0x5603edf3dad0 2024-05-21T21:57:20.916 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.909+0000 7f29da000700 1 -- 172.21.15.151:0/1356696118 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29e001f040 con 0x5603edf3dad0 2024-05-21T21:57:20.916 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.909+0000 7f29da000700 1 -- 172.21.15.151:0/1356696118 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f29e0024040 con 0x5603edf3dad0 2024-05-21T21:57:20.916 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.909+0000 7f2a01c0fb80 10 client.25643 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T21:57:20.916 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.909+0000 7f2a01c0fb80 10 client.25643 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-05-21T21:57:20.916 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.909+0000 7f2a01c0fb80 20 client.25643 populate_metadata read hostname 'smithi151' 2024-05-21T21:57:20.916 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.909+0000 7f2a01c0fb80 1 --2- 172.21.15.151:0/1356696118 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x5603edfc1fe0 0x5603edfc43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:20.917 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.909+0000 7f2a01c0fb80 4 client.25643 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T21:57:20.917 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.909+0000 7f2a01c0fb80 1 -- 172.21.15.151:0/1356696118 --> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5603ede97570 con 0x5603edfc1fe0 2024-05-21T21:57:20.917 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.910+0000 7f29ef400700 1 --2- 172.21.15.151:0/1356696118 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x5603edfc1fe0 0x5603edfc43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:20.917 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.911+0000 7f29ef400700 1 --2- 172.21.15.151:0/1356696118 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x5603edfc1fe0 0x5603edfc43d0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=635c6f1485914797 server_cookie=7d6576af2b11e61a in_seq=0 out_seq=0 2024-05-21T21:57:20.917 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.911+0000 7f29da000700 10 client.25643 ms_handle_connect on v2:172.21.15.164:6834/2340428470 2024-05-21T21:57:20.917 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.911+0000 7f29da000700 1 -- 172.21.15.151:0/1356696118 <== mds.4 v2:172.21.15.164:6834/2340428470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5603ede97570 con 0x5603edfc1fe0 2024-05-21T21:57:20.917 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.911+0000 7f29da000700 10 client.25643 handle_command_reply: tid=0 2024-05-21T21:57:20.917 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.911+0000 7f2a01c0fb80 1 client.25643 shutdown 2024-05-21T21:57:20.917 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.912+0000 7f2a01c0fb80 1 -- 172.21.15.151:0/1356696118 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5603edf3dad0 msgr2=0x5603edf4fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:20.917 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.912+0000 7f2a01c0fb80 1 --2- 172.21.15.151:0/1356696118 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5603edf3dad0 0x5603edf4fe90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f29e0011b20 tx=0x7f29e0005570 comp rx=0 tx=0).stop 2024-05-21T21:57:20.917 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.912+0000 7f2a01c0fb80 1 -- 172.21.15.151:0/1356696118 shutdown_connections 2024-05-21T21:57:20.917 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.912+0000 7f2a01c0fb80 1 --2- 172.21.15.151:0/1356696118 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x5603edfc1fe0 0x5603edfc43d0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:20.917 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.912+0000 7f2a01c0fb80 1 --2- 172.21.15.151:0/1356696118 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5603edf3dad0 0x5603edf4fe90 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:20.917 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.912+0000 7f2a01c0fb80 1 --2- 172.21.15.151:0/1356696118 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603edf43940 0x5603edf5ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:20.918 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.912+0000 7f2a01c0fb80 1 --2- 172.21.15.151:0/1356696118 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5603edf43020 0x5603edf5e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:20.918 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.912+0000 7f29da000700 0 client.25643 ms_handle_reset on v2:172.21.15.164:6834/2340428470 2024-05-21T21:57:20.918 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.912+0000 7f2a01c0fb80 1 -- 172.21.15.151:0/1356696118 >> 172.21.15.151:0/1356696118 conn(0x5603ede6bbe0 msgr2=0x5603edf37c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:57:20.918 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.912+0000 7f2a01c0fb80 1 -- 172.21.15.151:0/1356696118 shutdown_connections 2024-05-21T21:57:20.918 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.913+0000 7f2a01c0fb80 1 -- 172.21.15.151:0/1356696118 wait complete. 2024-05-21T21:57:20.918 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.913+0000 7f2a01c0fb80 20 client.25643 trim_cache size 0 max 16384 2024-05-21T21:57:20.918 INFO:teuthology.orchestra.run.smithi151.stdout:[] 2024-05-21T21:57:20.919 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.914+0000 7f29f5c00700 1 -- 172.21.15.151:0/1096743825 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f29d40b2a70 msgr2=0x7f29d40b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:20.919 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.914+0000 7f29f5c00700 1 --2- 172.21.15.151:0/1096743825 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f29d40b2a70 0x7f29d40b4f30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f29dc009ac0 tx=0x7f29dc0047c0 comp rx=0 tx=0).stop 2024-05-21T21:57:20.919 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.914+0000 7f29f5c00700 1 -- 172.21.15.151:0/1096743825 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f29f01076b0 msgr2=0x7f29f01a02a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:20.919 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.914+0000 7f29f5c00700 1 --2- 172.21.15.151:0/1096743825 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f29f01076b0 0x7f29f01a02a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f29e800b6c0 tx=0x7f29e800cc30 comp rx=0 tx=0).stop 2024-05-21T21:57:20.919 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.914+0000 7f29f5c00700 1 -- 172.21.15.151:0/1096743825 shutdown_connections 2024-05-21T21:57:20.919 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.914+0000 7f29f5c00700 1 --2- 172.21.15.151:0/1096743825 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f29d40b2a70 0x7f29d40b4f30 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:20.919 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.914+0000 7f29f5c00700 1 --2- 172.21.15.151:0/1096743825 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f29f01076b0 0x7f29f01a02a0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:20.919 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.914+0000 7f29f5c00700 1 --2- 172.21.15.151:0/1096743825 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f29f01025d0 0x7f29f01a4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:20.919 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.914+0000 7f29f5c00700 1 --2- 172.21.15.151:0/1096743825 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f29f0102f80 0x7f29f01a5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:20.919 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.914+0000 7f29f5c00700 1 -- 172.21.15.151:0/1096743825 >> 172.21.15.151:0/1096743825 conn(0x7f29f00fde60 msgr2=0x7f29f00ff170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:57:20.919 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.914+0000 7f29f5c00700 1 -- 172.21.15.151:0/1096743825 shutdown_connections 2024-05-21T21:57:20.920 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:20.914+0000 7f29f5c00700 1 -- 172.21.15.151:0/1096743825 wait complete. 2024-05-21T21:57:21.702 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:57:21 smithi151 ceph-mon[31623]: pgmap v55: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-05-21T21:57:21.735 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:57:21 smithi184 ceph-mon[36942]: pgmap v55: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-05-21T21:57:21.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:57:21 smithi164 ceph-mon[36269]: pgmap v55: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-05-21T21:57:21.852 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-21T21:57:22.284 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:57:23.677 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:57:23 smithi151 ceph-mon[31623]: pgmap v56: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 0 B/s wr, 12 op/s 2024-05-21T21:57:23.737 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:57:23 smithi184 ceph-mon[36942]: pgmap v56: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 0 B/s wr, 12 op/s 2024-05-21T21:57:23.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:57:23 smithi164 ceph-mon[36269]: pgmap v56: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 0 B/s wr, 12 op/s 2024-05-21T21:57:24.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.450+0000 7f0cfc200700 1 -- 172.21.15.151:0/3157819620 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0cf4103590 msgr2=0x7f0cf4103a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:24.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.450+0000 7f0cfc200700 1 --2- 172.21.15.151:0/3157819620 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0cf4103590 0x7f0cf4103a10 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0ce0009990 tx=0x7f0ce0009ca0 comp rx=0 tx=0).stop 2024-05-21T21:57:24.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.450+0000 7f0cfc200700 1 -- 172.21.15.151:0/3157819620 shutdown_connections 2024-05-21T21:57:24.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.450+0000 7f0cfc200700 1 --2- 172.21.15.151:0/3157819620 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0cf4103f50 0x7f0cf4110430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:24.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.450+0000 7f0cfc200700 1 --2- 172.21.15.151:0/3157819620 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0cf41096a0 0x7f0cf4109a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:24.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.450+0000 7f0cfc200700 1 --2- 172.21.15.151:0/3157819620 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0cf4103590 0x7f0cf4103a10 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:24.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.450+0000 7f0cfc200700 1 -- 172.21.15.151:0/3157819620 >> 172.21.15.151:0/3157819620 conn(0x7f0cf407a6f0 msgr2=0x7f0cf407ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:57:24.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.450+0000 7f0cfc200700 1 -- 172.21.15.151:0/3157819620 shutdown_connections 2024-05-21T21:57:24.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.450+0000 7f0cfc200700 1 -- 172.21.15.151:0/3157819620 wait complete. 2024-05-21T21:57:24.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.451+0000 7f0cfc200700 1 Processor -- start 2024-05-21T21:57:24.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.452+0000 7f0cfc200700 1 -- start start 2024-05-21T21:57:24.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.452+0000 7f0cfc200700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0cf4103590 0x7f0cf4075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:24.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.452+0000 7f0cfc200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0cf4103f50 0x7f0cf4075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:24.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.452+0000 7f0cfc200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0cf41096a0 0x7f0cf40762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:24.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.452+0000 7f0cfc200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf4113770 con 0x7f0cf41096a0 2024-05-21T21:57:24.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.452+0000 7f0cfc200700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf41135f0 con 0x7f0cf4103f50 2024-05-21T21:57:24.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.452+0000 7f0cfc200700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf41138f0 con 0x7f0cf4103590 2024-05-21T21:57:24.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.453+0000 7f0cf8e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0cf4103f50 0x7f0cf4075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:24.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.453+0000 7f0cf8e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0cf4103f50 0x7f0cf4075d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:44394/0 (socket says 172.21.15.151:44394) 2024-05-21T21:57:24.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.453+0000 7f0cf8e00700 1 -- 172.21.15.151:0/198081429 learned_addr learned my addr 172.21.15.151:0/198081429 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:57:24.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.453+0000 7f0cf9800700 1 --2- 172.21.15.151:0/198081429 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0cf4103590 0x7f0cf4075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:24.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.453+0000 7f0cfa200700 1 --2- 172.21.15.151:0/198081429 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0cf41096a0 0x7f0cf40762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:24.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.453+0000 7f0cf8e00700 1 -- 172.21.15.151:0/198081429 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0cf4103590 msgr2=0x7f0cf4075840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:24.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.453+0000 7f0cf8e00700 1 --2- 172.21.15.151:0/198081429 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0cf4103590 0x7f0cf4075840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:24.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.453+0000 7f0cf8e00700 1 -- 172.21.15.151:0/198081429 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0cf41096a0 msgr2=0x7f0cf40762c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:24.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.453+0000 7f0cf8e00700 1 --2- 172.21.15.151:0/198081429 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0cf41096a0 0x7f0cf40762c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:24.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.453+0000 7f0cf8e00700 1 -- 172.21.15.151:0/198081429 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cf41a1e90 con 0x7f0cf4103f50 2024-05-21T21:57:24.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.454+0000 7f0cf8e00700 1 --2- 172.21.15.151:0/198081429 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0cf4103f50 0x7f0cf4075d80 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0ce000b2e0 tx=0x7f0ce00057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:57:24.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.454+0000 7f0cf2000700 1 -- 172.21.15.151:0/198081429 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce0026070 con 0x7f0cf4103f50 2024-05-21T21:57:24.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.454+0000 7f0cfc200700 1 -- 172.21.15.151:0/198081429 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cf41a2120 con 0x7f0cf4103f50 2024-05-21T21:57:24.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.454+0000 7f0cf2000700 1 -- 172.21.15.151:0/198081429 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ce0021410 con 0x7f0cf4103f50 2024-05-21T21:57:24.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.454+0000 7f0cf2000700 1 -- 172.21.15.151:0/198081429 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce0017650 con 0x7f0cf4103f50 2024-05-21T21:57:24.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.454+0000 7f0cfc200700 1 -- 172.21.15.151:0/198081429 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cf41a2650 con 0x7f0cf4103f50 2024-05-21T21:57:24.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.459+0000 7f0cf2000700 1 -- 172.21.15.151:0/198081429 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7f0ce0029070 con 0x7f0cf4103f50 2024-05-21T21:57:24.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.460+0000 7f0d06c54b80 1 Processor -- start 2024-05-21T21:57:24.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.461+0000 7f0d06c54b80 1 -- start start 2024-05-21T21:57:24.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.461+0000 7f0d06c54b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560fe5db4eb0 0x560fe5db5290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:24.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.461+0000 7f0d06c54b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560fe5db57d0 0x560fe5daf260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:24.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.461+0000 7f0d06c54b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560fe5daf880 0x560fe5dafce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:24.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.462+0000 7f0d06c54b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x560fe5d0de40 con 0x560fe5db57d0 2024-05-21T21:57:24.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.462+0000 7f0d06c54b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x560fe5cfd9e0 con 0x560fe5daf880 2024-05-21T21:57:24.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.462+0000 7f0d06c54b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x560fe5d01a20 con 0x560fe5db4eb0 2024-05-21T21:57:24.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.462+0000 7f0cf2000700 1 --2- 172.21.15.151:0/198081429 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f0cdc0b2a70 0x7f0cdc0b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:24.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.462+0000 7f0cf9800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560fe5db57d0 0x560fe5daf260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:24.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.462+0000 7f0cf9800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560fe5db57d0 0x560fe5daf260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:37514/0 (socket says 172.21.15.151:37514) 2024-05-21T21:57:24.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.462+0000 7f0cf9800700 1 -- 172.21.15.151:0/1678788114 learned_addr learned my addr 172.21.15.151:0/1678788114 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:57:24.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.462+0000 7f0cfa200700 1 --2- 172.21.15.151:0/1678788114 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560fe5db4eb0 0x560fe5db5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:24.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.462+0000 7f0cf8e00700 1 --2- 172.21.15.151:0/1678788114 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560fe5daf880 0x560fe5dafce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:24.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.463+0000 7f0cf9800700 1 -- 172.21.15.151:0/1678788114 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560fe5db4eb0 msgr2=0x560fe5db5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:24.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.463+0000 7f0cf9800700 1 --2- 172.21.15.151:0/1678788114 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560fe5db4eb0 0x560fe5db5290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:24.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.463+0000 7f0cf9800700 1 -- 172.21.15.151:0/1678788114 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560fe5daf880 msgr2=0x560fe5dafce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:24.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.463+0000 7f0cf9800700 1 --2- 172.21.15.151:0/1678788114 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560fe5daf880 0x560fe5dafce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:24.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.462+0000 7f0cf2000700 1 -- 172.21.15.151:0/198081429 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f0ce00e3350 con 0x7f0cf4103f50 2024-05-21T21:57:24.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.463+0000 7f0cf9800700 1 -- 172.21.15.151:0/1678788114 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fe5cc6420 con 0x560fe5db57d0 2024-05-21T21:57:24.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.463+0000 7f0cf8e00700 1 --2- 172.21.15.151:0/1678788114 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560fe5daf880 0x560fe5dafce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:57:24.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.463+0000 7f0cfa200700 1 --2- 172.21.15.151:0/198081429 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f0cdc0b2a70 0x7f0cdc0b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:24.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.463+0000 7f0cf9800700 1 --2- 172.21.15.151:0/1678788114 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560fe5db57d0 0x560fe5daf260 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0ce8009d00 tx=0x7f0ce8011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eae3e82685b091b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:57:24.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.464+0000 7f0ce6a00700 1 -- 172.21.15.151:0/1678788114 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce801e070 con 0x560fe5db57d0 2024-05-21T21:57:24.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.464+0000 7f0ce6a00700 1 -- 172.21.15.151:0/1678788114 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ce8009e50 con 0x560fe5db57d0 2024-05-21T21:57:24.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.464+0000 7f0ce6a00700 1 -- 172.21.15.151:0/1678788114 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce800e9e0 con 0x560fe5db57d0 2024-05-21T21:57:24.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.465+0000 7f0d06c54b80 1 -- 172.21.15.151:0/1678788114 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560fe5db57d0 msgr2=0x560fe5daf260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:24.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.465+0000 7f0cfa200700 1 --2- 172.21.15.151:0/198081429 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f0cdc0b2a70 0x7f0cdc0b4f30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0cec009550 tx=0x7f0cec004b40 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:57:24.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.465+0000 7f0d06c54b80 1 --2- 172.21.15.151:0/1678788114 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560fe5db57d0 0x560fe5daf260 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0ce8009d00 tx=0x7f0ce8011720 comp rx=0 tx=0).stop 2024-05-21T21:57:24.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.465+0000 7f0d06c54b80 1 -- 172.21.15.151:0/1678788114 shutdown_connections 2024-05-21T21:57:24.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.465+0000 7f0d06c54b80 1 --2- 172.21.15.151:0/1678788114 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560fe5daf880 0x560fe5dafce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:24.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.465+0000 7f0d06c54b80 1 --2- 172.21.15.151:0/1678788114 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560fe5db4eb0 0x560fe5db5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:24.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.465+0000 7f0d06c54b80 1 --2- 172.21.15.151:0/1678788114 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560fe5db57d0 0x560fe5daf260 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:24.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.465+0000 7f0d06c54b80 1 -- 172.21.15.151:0/1678788114 >> 172.21.15.151:0/1678788114 conn(0x560fe5cddbe0 msgr2=0x560fe5daa010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:57:24.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.466+0000 7f0d06c54b80 1 -- 172.21.15.151:0/1678788114 shutdown_connections 2024-05-21T21:57:24.467 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.466+0000 7f0d06c54b80 1 -- 172.21.15.151:0/1678788114 wait complete. 2024-05-21T21:57:24.467 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.467+0000 7f0d06c54b80 1 -- start start 2024-05-21T21:57:24.467 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.467+0000 7f0d06c54b80 1 Processor -- start 2024-05-21T21:57:24.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.468+0000 7f0d06c54b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560fe5daf880 0x560fe5dbb460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:24.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.468+0000 7f0d06c54b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560fe5db4eb0 0x560fe5db8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:24.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.468+0000 7f0d06c54b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560fe5db57d0 0x560fe5db8d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:24.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.468+0000 7f0d06c54b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x560fe5c03fa0 con 0x560fe5daf880 2024-05-21T21:57:24.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.468+0000 7f0d06c54b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x560fe5c9e760 con 0x560fe5db4eb0 2024-05-21T21:57:24.469 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.468+0000 7f0d06c54b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x560fe5dcd250 con 0x560fe5db57d0 2024-05-21T21:57:24.469 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.468+0000 7f0d06c54b80 10 client.0 _ll_register_callbacks cb 0x560fe5ca7c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:57:24.469 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.468+0000 7f0cfa200700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560fe5db57d0 0x560fe5db8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:24.469 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.468+0000 7f0cfa200700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560fe5db57d0 0x560fe5db8d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:33582/0 (socket says 172.21.15.151:33582) 2024-05-21T21:57:24.469 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.468+0000 7f0cfa200700 1 -- 172.21.15.151:0/2668313376 learned_addr learned my addr 172.21.15.151:0/2668313376 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:57:24.469 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.468+0000 7f0cf9800700 1 --2- 172.21.15.151:0/2668313376 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560fe5daf880 0x560fe5dbb460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:24.470 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.469+0000 7f0cf8e00700 1 --2- 172.21.15.151:0/2668313376 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560fe5db4eb0 0x560fe5db8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:24.470 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.469+0000 7f0cfa200700 1 -- 172.21.15.151:0/2668313376 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560fe5db4eb0 msgr2=0x560fe5db8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:24.470 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.469+0000 7f0cfa200700 1 --2- 172.21.15.151:0/2668313376 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560fe5db4eb0 0x560fe5db8800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:24.470 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.469+0000 7f0cfa200700 1 -- 172.21.15.151:0/2668313376 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560fe5daf880 msgr2=0x560fe5dbb460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:24.470 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.469+0000 7f0cfa200700 1 --2- 172.21.15.151:0/2668313376 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560fe5daf880 0x560fe5dbb460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:24.470 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.469+0000 7f0cfa200700 1 -- 172.21.15.151:0/2668313376 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fe5c4d8d0 con 0x560fe5db57d0 2024-05-21T21:57:24.470 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.470+0000 7f0cf9800700 1 --2- 172.21.15.151:0/2668313376 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560fe5daf880 0x560fe5dbb460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:57:24.470 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.470+0000 7f0cfa200700 1 --2- 172.21.15.151:0/2668313376 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560fe5db57d0 0x560fe5db8d40 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0cec018840 tx=0x7f0cec004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d62ff327f34d2dc1 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:57:24.471 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.470+0000 7f0cdb400700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-21T21:57:24.471 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.470+0000 7f0cdb400700 1 -- 172.21.15.151:0/2668313376 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cec028060 con 0x560fe5db57d0 2024-05-21T21:57:24.471 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.470+0000 7f0cdb400700 1 -- 172.21.15.151:0/2668313376 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f0cec017860 con 0x560fe5db57d0 2024-05-21T21:57:24.471 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.471+0000 7f0d06c54b80 1 -- 172.21.15.151:0/2668313376 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ce8009e50 con 0x560fe5db57d0 2024-05-21T21:57:24.471 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.471+0000 7f0cdb400700 1 -- 172.21.15.151:0/2668313376 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0cec017aa0 con 0x560fe5db57d0 2024-05-21T21:57:24.471 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.471+0000 7f0d06c54b80 10 client.25672 fetch_fsmap learned FSMap version 29 2024-05-21T21:57:24.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.471+0000 7f0d06c54b80 1 -- 172.21.15.151:0/2668313376 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560fe5cce060 con 0x560fe5db57d0 2024-05-21T21:57:24.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.471+0000 7f0cdb400700 1 -- 172.21.15.151:0/2668313376 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ce8009e50 con 0x560fe5db57d0 2024-05-21T21:57:24.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.472+0000 7f0cdb400700 1 -- 172.21.15.151:0/2668313376 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f0cec02c3a0 con 0x560fe5db57d0 2024-05-21T21:57:24.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.472+0000 7f0d06c54b80 10 client.25672 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T21:57:24.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.472+0000 7f0d06c54b80 10 client.25672 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-21T21:57:24.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.472+0000 7f0d06c54b80 20 client.25672 populate_metadata read hostname 'smithi151' 2024-05-21T21:57:24.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.472+0000 7f0d06c54b80 1 --2- 172.21.15.151:0/2668313376 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x560fe5dd9d30 0x560fe5ddc120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:24.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.472+0000 7f0d06c54b80 4 client.25672 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T21:57:24.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.472+0000 7f0d06c54b80 1 -- 172.21.15.151:0/2668313376 --> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560fe5d0fd90 con 0x560fe5dd9d30 2024-05-21T21:57:24.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.473+0000 7f0cf9800700 1 --2- 172.21.15.151:0/2668313376 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x560fe5dd9d30 0x560fe5ddc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:24.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.473+0000 7f0cf9800700 1 --2- 172.21.15.151:0/2668313376 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x560fe5dd9d30 0x560fe5ddc120 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=81ea62f7fe67d47e server_cookie=550831a5c5c7a9ce in_seq=0 out_seq=0 2024-05-21T21:57:24.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.473+0000 7f0cdb400700 10 client.25672 ms_handle_connect on v2:172.21.15.184:6836/2988194888 2024-05-21T21:57:24.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.474+0000 7f0cdb400700 1 -- 172.21.15.151:0/2668313376 <== mds.3 v2:172.21.15.184:6836/2988194888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x560fe5d0fd90 con 0x560fe5dd9d30 2024-05-21T21:57:24.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.474+0000 7f0cdb400700 10 client.25672 handle_command_reply: tid=0 2024-05-21T21:57:24.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.474+0000 7f0d06c54b80 1 client.25672 shutdown 2024-05-21T21:57:24.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.475+0000 7f0d06c54b80 1 -- 172.21.15.151:0/2668313376 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560fe5db57d0 msgr2=0x560fe5db8d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:24.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.475+0000 7f0d06c54b80 1 --2- 172.21.15.151:0/2668313376 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560fe5db57d0 0x560fe5db8d40 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0cec018840 tx=0x7f0cec004ad0 comp rx=0 tx=0).stop 2024-05-21T21:57:24.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.475+0000 7f0d06c54b80 1 -- 172.21.15.151:0/2668313376 shutdown_connections 2024-05-21T21:57:24.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.475+0000 7f0d06c54b80 1 --2- 172.21.15.151:0/2668313376 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x560fe5dd9d30 0x560fe5ddc120 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:24.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.475+0000 7f0d06c54b80 1 --2- 172.21.15.151:0/2668313376 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560fe5db57d0 0x560fe5db8d40 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:24.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.475+0000 7f0d06c54b80 1 --2- 172.21.15.151:0/2668313376 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560fe5daf880 0x560fe5dbb460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:24.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.475+0000 7f0cdb400700 0 client.25672 ms_handle_reset on v2:172.21.15.184:6836/2988194888 2024-05-21T21:57:24.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.475+0000 7f0d06c54b80 1 --2- 172.21.15.151:0/2668313376 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560fe5db4eb0 0x560fe5db8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:24.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.475+0000 7f0d06c54b80 1 -- 172.21.15.151:0/2668313376 >> 172.21.15.151:0/2668313376 conn(0x560fe5cddbe0 msgr2=0x560fe5dae250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:57:24.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.475+0000 7f0d06c54b80 1 -- 172.21.15.151:0/2668313376 shutdown_connections 2024-05-21T21:57:24.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.476+0000 7f0d06c54b80 1 -- 172.21.15.151:0/2668313376 wait complete. 2024-05-21T21:57:24.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.476+0000 7f0d06c54b80 20 client.25672 trim_cache size 0 max 16384 2024-05-21T21:57:24.499 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.498+0000 7f0d06c54b80 1 Processor -- start 2024-05-21T21:57:24.499 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.499+0000 7f0d06c54b80 1 -- start start 2024-05-21T21:57:24.499 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.499+0000 7f0d06c54b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560fe5daf880 0x560fe5daf260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:24.499 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.499+0000 7f0d06c54b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560fe5db4eb0 0x560fe5da99e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:24.499 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.499+0000 7f0d06c54b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560fe5db57d0 0x560fe5da9f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:24.500 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.499+0000 7f0d06c54b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf41a9c00 con 0x560fe5db4eb0 2024-05-21T21:57:24.500 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.499+0000 7f0d06c54b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf406e000 con 0x560fe5daf880 2024-05-21T21:57:24.500 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.499+0000 7f0d06c54b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf41aa2d0 con 0x560fe5db57d0 2024-05-21T21:57:24.500 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.499+0000 7f0cf8e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560fe5db4eb0 0x560fe5da99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:24.500 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.499+0000 7f0cfa200700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560fe5db57d0 0x560fe5da9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:24.501 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.500+0000 7f0cfa200700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560fe5db57d0 0x560fe5da9f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:33590/0 (socket says 172.21.15.151:33590) 2024-05-21T21:57:24.501 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.500+0000 7f0cf9800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560fe5daf880 0x560fe5daf260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:24.501 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.500+0000 7f0cfa200700 1 -- 172.21.15.151:0/2458770572 learned_addr learned my addr 172.21.15.151:0/2458770572 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:57:24.501 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.500+0000 7f0cfa200700 1 -- 172.21.15.151:0/2458770572 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560fe5daf880 msgr2=0x560fe5daf260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:24.501 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.500+0000 7f0cfa200700 1 --2- 172.21.15.151:0/2458770572 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560fe5daf880 0x560fe5daf260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:24.501 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.500+0000 7f0cfa200700 1 -- 172.21.15.151:0/2458770572 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560fe5db4eb0 msgr2=0x560fe5da99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:24.501 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.500+0000 7f0cfa200700 1 --2- 172.21.15.151:0/2458770572 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560fe5db4eb0 0x560fe5da99e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:24.501 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.500+0000 7f0cfa200700 1 -- 172.21.15.151:0/2458770572 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fe5ddc750 con 0x560fe5db57d0 2024-05-21T21:57:24.501 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.500+0000 7f0cf8e00700 1 --2- 172.21.15.151:0/2458770572 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560fe5db4eb0 0x560fe5da99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T21:57:24.501 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.500+0000 7f0cfa200700 1 --2- 172.21.15.151:0/2458770572 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560fe5db57d0 0x560fe5da9f20 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0cec009fb0 tx=0x7f0cec018f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9c8558397aec473 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:57:24.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.501+0000 7f0ce6000700 1 -- 172.21.15.151:0/2458770572 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cec032080 con 0x560fe5db57d0 2024-05-21T21:57:24.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.501+0000 7f0ce6000700 1 -- 172.21.15.151:0/2458770572 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cec02c5a0 con 0x560fe5db57d0 2024-05-21T21:57:24.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.501+0000 7f0ce6000700 1 -- 172.21.15.151:0/2458770572 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cec0178c0 con 0x560fe5db57d0 2024-05-21T21:57:24.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.502+0000 7f0d06c54b80 1 -- 172.21.15.151:0/2458770572 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560fe5db57d0 msgr2=0x560fe5da9f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:24.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.502+0000 7f0d06c54b80 1 --2- 172.21.15.151:0/2458770572 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560fe5db57d0 0x560fe5da9f20 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0cec009fb0 tx=0x7f0cec018f90 comp rx=0 tx=0).stop 2024-05-21T21:57:24.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.502+0000 7f0d06c54b80 1 -- 172.21.15.151:0/2458770572 shutdown_connections 2024-05-21T21:57:24.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.502+0000 7f0d06c54b80 1 --2- 172.21.15.151:0/2458770572 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560fe5daf880 0x560fe5daf260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:24.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.502+0000 7f0d06c54b80 1 --2- 172.21.15.151:0/2458770572 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560fe5db57d0 0x560fe5da9f20 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:24.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.502+0000 7f0d06c54b80 1 --2- 172.21.15.151:0/2458770572 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560fe5db4eb0 0x560fe5da99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:24.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.502+0000 7f0d06c54b80 1 -- 172.21.15.151:0/2458770572 >> 172.21.15.151:0/2458770572 conn(0x560fe5cddbe0 msgr2=0x560fe5db70f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:57:24.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.502+0000 7f0d06c54b80 1 -- 172.21.15.151:0/2458770572 shutdown_connections 2024-05-21T21:57:24.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.502+0000 7f0d06c54b80 1 -- 172.21.15.151:0/2458770572 wait complete. 2024-05-21T21:57:24.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.503+0000 7f0d06c54b80 1 -- start start 2024-05-21T21:57:24.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.503+0000 7f0d06c54b80 1 Processor -- start 2024-05-21T21:57:24.504 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.504+0000 7f0d06c54b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560fe5daf880 0x560fe5dd2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:24.504 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.504+0000 7f0d06c54b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560fe5db4eb0 0x560fe5de13b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:24.504 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.504+0000 7f0d06c54b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560fe5db57d0 0x560fe5de1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:24.504 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.504+0000 7f0d06c54b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf406d930 con 0x560fe5db4eb0 2024-05-21T21:57:24.504 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.504+0000 7f0d06c54b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf406d260 con 0x560fe5db57d0 2024-05-21T21:57:24.504 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.504+0000 7f0d06c54b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf41a7ba0 con 0x560fe5daf880 2024-05-21T21:57:24.504 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.504+0000 7f0d06c54b80 10 client.0 _ll_register_callbacks cb 0x560fe5ca7c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:57:24.505 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.504+0000 7f0cf8e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560fe5db4eb0 0x560fe5de13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:24.505 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.504+0000 7f0cf9800700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560fe5daf880 0x560fe5dd2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:24.505 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.504+0000 7f0cf8e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560fe5db4eb0 0x560fe5de13b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:37554/0 (socket says 172.21.15.151:37554) 2024-05-21T21:57:24.505 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.504+0000 7f0cf8e00700 1 -- 172.21.15.151:0/4135293977 learned_addr learned my addr 172.21.15.151:0/4135293977 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:57:24.505 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.505+0000 7f0cfa200700 1 --2- 172.21.15.151:0/4135293977 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560fe5db57d0 0x560fe5de1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:24.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.505+0000 7f0cf8e00700 1 -- 172.21.15.151:0/4135293977 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560fe5daf880 msgr2=0x560fe5dd2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:24.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.505+0000 7f0cf8e00700 1 --2- 172.21.15.151:0/4135293977 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560fe5daf880 0x560fe5dd2d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:24.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.505+0000 7f0cf8e00700 1 -- 172.21.15.151:0/4135293977 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560fe5db57d0 msgr2=0x560fe5de1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:24.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.505+0000 7f0cf8e00700 1 --2- 172.21.15.151:0/4135293977 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560fe5db57d0 0x560fe5de1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:24.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.505+0000 7f0cf8e00700 1 -- 172.21.15.151:0/4135293977 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fe5e43850 con 0x560fe5db4eb0 2024-05-21T21:57:24.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.506+0000 7f0cf8e00700 1 --2- 172.21.15.151:0/4135293977 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560fe5db4eb0 0x560fe5de13b0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f0ce00058b0 tx=0x7f0ce00e0fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55b437f0feb9dd2c server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:57:24.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.506+0000 7f0cdb400700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-21T21:57:24.507 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.506+0000 7f0cdb400700 1 -- 172.21.15.151:0/4135293977 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce00aa670 con 0x560fe5db4eb0 2024-05-21T21:57:24.507 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.506+0000 7f0cdb400700 1 -- 172.21.15.151:0/4135293977 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f0ce00aa8d0 con 0x560fe5db4eb0 2024-05-21T21:57:24.507 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.506+0000 7f0d06c54b80 1 -- 172.21.15.151:0/4135293977 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0cec02c5a0 con 0x560fe5db4eb0 2024-05-21T21:57:24.507 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.507+0000 7f0cdb400700 1 -- 172.21.15.151:0/4135293977 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0ce00aea20 con 0x560fe5db4eb0 2024-05-21T21:57:24.507 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.507+0000 7f0d06c54b80 10 client.15543 fetch_fsmap learned FSMap version 29 2024-05-21T21:57:24.507 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.507+0000 7f0d06c54b80 1 -- 172.21.15.151:0/4135293977 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560fe5e44a70 con 0x560fe5db4eb0 2024-05-21T21:57:24.508 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.507+0000 7f0cdb400700 1 -- 172.21.15.151:0/4135293977 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cec02c5a0 con 0x560fe5db4eb0 2024-05-21T21:57:24.508 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.508+0000 7f0cdb400700 1 -- 172.21.15.151:0/4135293977 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f0ce0075ec0 con 0x560fe5db4eb0 2024-05-21T21:57:24.508 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.508+0000 7f0d06c54b80 10 client.15543 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T21:57:24.508 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.508+0000 7f0d06c54b80 10 client.15543 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-21T21:57:24.509 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.508+0000 7f0d06c54b80 20 client.15543 populate_metadata read hostname 'smithi151' 2024-05-21T21:57:24.509 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.508+0000 7f0d06c54b80 1 --2- 172.21.15.151:0/4135293977 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x560fe5e451d0 0x560fe5e475f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:24.509 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.508+0000 7f0d06c54b80 4 client.15543 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T21:57:24.509 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.508+0000 7f0d06c54b80 1 -- 172.21.15.151:0/4135293977 --> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560fe5d09570 con 0x560fe5e451d0 2024-05-21T21:57:24.509 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.508+0000 7f0cf9800700 1 --2- 172.21.15.151:0/4135293977 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x560fe5e451d0 0x560fe5e475f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:24.510 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.509+0000 7f0cf9800700 1 --2- 172.21.15.151:0/4135293977 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x560fe5e451d0 0x560fe5e475f0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7beba2dd667633f server_cookie=d81db0b5aa05a4c8 in_seq=0 out_seq=0 2024-05-21T21:57:24.510 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.509+0000 7f0cdb400700 10 client.15543 ms_handle_connect on v2:172.21.15.184:6836/2988194888 2024-05-21T21:57:24.510 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.509+0000 7f0cdb400700 1 -- 172.21.15.151:0/4135293977 <== mds.3 v2:172.21.15.184:6836/2988194888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560fe5d09570 con 0x560fe5e451d0 2024-05-21T21:57:24.510 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.509+0000 7f0cdb400700 10 client.15543 handle_command_reply: tid=0 2024-05-21T21:57:24.510 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.509+0000 7f0d06c54b80 1 client.15543 shutdown 2024-05-21T21:57:24.510 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.509+0000 7f0d06c54b80 1 -- 172.21.15.151:0/4135293977 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560fe5db4eb0 msgr2=0x560fe5de13b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:24.510 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.509+0000 7f0d06c54b80 1 --2- 172.21.15.151:0/4135293977 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560fe5db4eb0 0x560fe5de13b0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f0ce00058b0 tx=0x7f0ce00e0fb0 comp rx=0 tx=0).stop 2024-05-21T21:57:24.510 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.509+0000 7f0d06c54b80 1 -- 172.21.15.151:0/4135293977 shutdown_connections 2024-05-21T21:57:24.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.509+0000 7f0d06c54b80 1 --2- 172.21.15.151:0/4135293977 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x560fe5e451d0 0x560fe5e475f0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:24.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.509+0000 7f0d06c54b80 1 --2- 172.21.15.151:0/4135293977 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560fe5db57d0 0x560fe5de1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:24.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.509+0000 7f0d06c54b80 1 --2- 172.21.15.151:0/4135293977 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560fe5daf880 0x560fe5dd2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:24.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.509+0000 7f0d06c54b80 1 --2- 172.21.15.151:0/4135293977 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560fe5db4eb0 0x560fe5de13b0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:24.512 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.509+0000 7f0cdb400700 0 client.15543 ms_handle_reset on v2:172.21.15.184:6836/2988194888 2024-05-21T21:57:24.512 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.509+0000 7f0d06c54b80 1 -- 172.21.15.151:0/4135293977 >> 172.21.15.151:0/4135293977 conn(0x560fe5cddbe0 msgr2=0x560fe5da99e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:57:24.512 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.510+0000 7f0d06c54b80 1 -- 172.21.15.151:0/4135293977 shutdown_connections 2024-05-21T21:57:24.512 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.510+0000 7f0d06c54b80 1 -- 172.21.15.151:0/4135293977 wait complete. 2024-05-21T21:57:24.512 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.510+0000 7f0d06c54b80 20 client.15543 trim_cache size 0 max 16384 2024-05-21T21:57:24.512 INFO:teuthology.orchestra.run.smithi151.stdout:[] 2024-05-21T21:57:24.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.511+0000 7f0cfc200700 1 -- 172.21.15.151:0/198081429 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f0cdc0b2a70 msgr2=0x7f0cdc0b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:24.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.511+0000 7f0cfc200700 1 --2- 172.21.15.151:0/198081429 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f0cdc0b2a70 0x7f0cdc0b4f30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0cec009550 tx=0x7f0cec004b40 comp rx=0 tx=0).stop 2024-05-21T21:57:24.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.511+0000 7f0cfc200700 1 -- 172.21.15.151:0/198081429 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0cf4103f50 msgr2=0x7f0cf4075d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:24.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.511+0000 7f0cfc200700 1 --2- 172.21.15.151:0/198081429 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0cf4103f50 0x7f0cf4075d80 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0ce000b2e0 tx=0x7f0ce00057c0 comp rx=0 tx=0).stop 2024-05-21T21:57:24.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.511+0000 7f0cfc200700 1 -- 172.21.15.151:0/198081429 shutdown_connections 2024-05-21T21:57:24.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.511+0000 7f0cfc200700 1 --2- 172.21.15.151:0/198081429 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f0cdc0b2a70 0x7f0cdc0b4f30 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:24.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.511+0000 7f0cfc200700 1 --2- 172.21.15.151:0/198081429 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0cf41096a0 0x7f0cf40762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:24.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.511+0000 7f0cfc200700 1 --2- 172.21.15.151:0/198081429 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0cf4103590 0x7f0cf4075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:24.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.511+0000 7f0cfc200700 1 --2- 172.21.15.151:0/198081429 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0cf4103f50 0x7f0cf4075d80 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:24.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.511+0000 7f0cfc200700 1 -- 172.21.15.151:0/198081429 >> 172.21.15.151:0/198081429 conn(0x7f0cf407a6f0 msgr2=0x7f0cf4102120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:57:24.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.512+0000 7f0cfc200700 1 -- 172.21.15.151:0/198081429 shutdown_connections 2024-05-21T21:57:24.514 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:24.512+0000 7f0cfc200700 1 -- 172.21.15.151:0/198081429 wait complete. 2024-05-21T21:57:25.292 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-21T21:57:25.702 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:57:25 smithi151 ceph-mon[31623]: pgmap v57: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-05-21T21:57:25.727 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:57:25.735 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:57:25 smithi184 ceph-mon[36942]: pgmap v57: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-05-21T21:57:25.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:57:25 smithi164 ceph-mon[36269]: pgmap v57: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-05-21T21:57:27.513 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:57:27 smithi151 ceph-mon[31623]: pgmap v58: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-05-21T21:57:27.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.723+0000 7fd537800700 1 -- 172.21.15.151:0/2247499887 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd530075770 msgr2=0x7fd530075b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:27.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.723+0000 7fd537800700 1 --2- 172.21.15.151:0/2247499887 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd530075770 0x7fd530075b50 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fd5240098f0 tx=0x7fd524009c00 comp rx=0 tx=0).stop 2024-05-21T21:57:27.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.723+0000 7fd537800700 1 -- 172.21.15.151:0/2247499887 shutdown_connections 2024-05-21T21:57:27.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.724+0000 7fd537800700 1 --2- 172.21.15.151:0/2247499887 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd53010ffa0 0x7fd530112390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:27.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.724+0000 7fd537800700 1 --2- 172.21.15.151:0/2247499887 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd530075770 0x7fd530075b50 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:27.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.724+0000 7fd537800700 1 --2- 172.21.15.151:0/2247499887 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd530076090 0x7fd53010fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:27.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.724+0000 7fd537800700 1 -- 172.21.15.151:0/2247499887 >> 172.21.15.151:0/2247499887 conn(0x7fd53007a6a0 msgr2=0x7fd53007aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:57:27.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.724+0000 7fd537800700 1 -- 172.21.15.151:0/2247499887 shutdown_connections 2024-05-21T21:57:27.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.724+0000 7fd537800700 1 -- 172.21.15.151:0/2247499887 wait complete. 2024-05-21T21:57:27.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.725+0000 7fd537800700 1 Processor -- start 2024-05-21T21:57:27.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.725+0000 7fd537800700 1 -- start start 2024-05-21T21:57:27.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.725+0000 7fd537800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd530075770 0x7fd53019bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:27.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.725+0000 7fd537800700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd530076090 0x7fd53019c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:27.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.725+0000 7fd537800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd53010ffa0 0x7fd5301a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:27.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.726+0000 7fd537800700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd530115980 con 0x7fd53010ffa0 2024-05-21T21:57:27.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.726+0000 7fd537800700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd530115800 con 0x7fd530075770 2024-05-21T21:57:27.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.726+0000 7fd537800700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd530115b00 con 0x7fd530076090 2024-05-21T21:57:27.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.726+0000 7fd534e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd530075770 0x7fd53019bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:27.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.726+0000 7fd534e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd530075770 0x7fd53019bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:38000/0 (socket says 172.21.15.151:38000) 2024-05-21T21:57:27.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.726+0000 7fd534e00700 1 -- 172.21.15.151:0/3637585074 learned_addr learned my addr 172.21.15.151:0/3637585074 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:57:27.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.726+0000 7fd52fe00700 1 --2- 172.21.15.151:0/3637585074 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd530076090 0x7fd53019c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:27.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.726+0000 7fd535800700 1 --2- 172.21.15.151:0/3637585074 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd53010ffa0 0x7fd5301a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:27.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.726+0000 7fd534e00700 1 -- 172.21.15.151:0/3637585074 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd530076090 msgr2=0x7fd53019c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:27.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.726+0000 7fd534e00700 1 --2- 172.21.15.151:0/3637585074 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd530076090 0x7fd53019c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:27.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.726+0000 7fd534e00700 1 -- 172.21.15.151:0/3637585074 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd53010ffa0 msgr2=0x7fd5301a06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:27.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.726+0000 7fd534e00700 1 --2- 172.21.15.151:0/3637585074 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd53010ffa0 0x7fd5301a06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:27.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.726+0000 7fd534e00700 1 -- 172.21.15.151:0/3637585074 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5301a0e00 con 0x7fd530075770 2024-05-21T21:57:27.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.727+0000 7fd535800700 1 --2- 172.21.15.151:0/3637585074 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd53010ffa0 0x7fd5301a06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T21:57:27.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.727+0000 7fd534e00700 1 --2- 172.21.15.151:0/3637585074 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd530075770 0x7fd53019bdf0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd524009230 tx=0x7fd52400fa70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:57:27.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.727+0000 7fd52d600700 1 -- 172.21.15.151:0/3637585074 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd524005140 con 0x7fd530075770 2024-05-21T21:57:27.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.727+0000 7fd52d600700 1 -- 172.21.15.151:0/3637585074 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd52400b560 con 0x7fd530075770 2024-05-21T21:57:27.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.727+0000 7fd537800700 1 -- 172.21.15.151:0/3637585074 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5301a1090 con 0x7fd530075770 2024-05-21T21:57:27.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.727+0000 7fd52d600700 1 -- 172.21.15.151:0/3637585074 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd524003ae0 con 0x7fd530075770 2024-05-21T21:57:27.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.727+0000 7fd537800700 1 -- 172.21.15.151:0/3637585074 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5301a8970 con 0x7fd530075770 2024-05-21T21:57:27.731 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.731+0000 7fd52d600700 1 -- 172.21.15.151:0/3637585074 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7fd524003c80 con 0x7fd530075770 2024-05-21T21:57:27.732 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.732+0000 7fd5420c7b80 1 Processor -- start 2024-05-21T21:57:27.732 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.732+0000 7fd5420c7b80 1 -- start start 2024-05-21T21:57:27.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.732+0000 7fd5420c7b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563632841eb0 0x563632842290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:27.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.732+0000 7fd5420c7b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5636328427d0 0x56363283c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:27.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.733+0000 7fd5420c7b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56363283c880 0x56363283cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:27.735 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.733+0000 7fd5420c7b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56363279ae40 con 0x563632841eb0 2024-05-21T21:57:27.735 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.733+0000 7fd5420c7b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56363278a9e0 con 0x56363283c880 2024-05-21T21:57:27.735 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.733+0000 7fd5420c7b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56363278ea20 con 0x5636328427d0 2024-05-21T21:57:27.735 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.733+0000 7fd534e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5636328427d0 0x56363283c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:27.735 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.733+0000 7fd534e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5636328427d0 0x56363283c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:40678/0 (socket says 172.21.15.151:40678) 2024-05-21T21:57:27.735 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.733+0000 7fd534e00700 1 -- 172.21.15.151:0/3249040312 learned_addr learned my addr 172.21.15.151:0/3249040312 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:57:27.735 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.733+0000 7fd535800700 1 --2- 172.21.15.151:0/3249040312 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563632841eb0 0x563632842290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:27.735 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.733+0000 7fd52d600700 1 --2- 172.21.15.151:0/3637585074 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fd5180b2a70 0x7fd5180b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:27.735 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.733+0000 7fd52fe00700 1 --2- 172.21.15.151:0/3249040312 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56363283c880 0x56363283cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:27.735 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.733+0000 7fd52d600700 1 -- 172.21.15.151:0/3637585074 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fd5240e35a0 con 0x7fd530075770 2024-05-21T21:57:27.735 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.734+0000 7fd52fe00700 1 -- 172.21.15.151:0/3249040312 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5636328427d0 msgr2=0x56363283c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:27.735 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.734+0000 7fd52fe00700 1 --2- 172.21.15.151:0/3249040312 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5636328427d0 0x56363283c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:27.735 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.734+0000 7fd52fe00700 1 -- 172.21.15.151:0/3249040312 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563632841eb0 msgr2=0x563632842290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:27.736 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.734+0000 7fd52fe00700 1 --2- 172.21.15.151:0/3249040312 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563632841eb0 0x563632842290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:27.736 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.734+0000 7fd52fe00700 1 -- 172.21.15.151:0/3249040312 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563632753420 con 0x56363283c880 2024-05-21T21:57:27.736 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.734+0000 7fd535800700 1 --2- 172.21.15.151:0/3249040312 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563632841eb0 0x563632842290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:57:27.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:57:27 smithi184 ceph-mon[36942]: pgmap v58: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-05-21T21:57:27.737 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.735+0000 7fd52fe00700 1 --2- 172.21.15.151:0/3249040312 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56363283c880 0x56363283cce0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd51c005fd0 tx=0x7fd51c011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5acdfbbfa75f693e server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:57:27.737 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.735+0000 7fd535800700 1 --2- 172.21.15.151:0/3637585074 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fd5180b2a70 0x7fd5180b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:27.737 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.735+0000 7fd522000700 1 -- 172.21.15.151:0/3249040312 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd51c004250 con 0x56363283c880 2024-05-21T21:57:27.738 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.735+0000 7fd522000700 1 -- 172.21.15.151:0/3249040312 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd51c0043f0 con 0x56363283c880 2024-05-21T21:57:27.738 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.735+0000 7fd522000700 1 -- 172.21.15.151:0/3249040312 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd51c019800 con 0x56363283c880 2024-05-21T21:57:27.738 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.736+0000 7fd5420c7b80 1 -- 172.21.15.151:0/3249040312 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56363283c880 msgr2=0x56363283cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:27.738 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.736+0000 7fd5420c7b80 1 --2- 172.21.15.151:0/3249040312 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56363283c880 0x56363283cce0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd51c005fd0 tx=0x7fd51c011720 comp rx=0 tx=0).stop 2024-05-21T21:57:27.738 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.736+0000 7fd535800700 1 --2- 172.21.15.151:0/3637585074 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fd5180b2a70 0x7fd5180b4f30 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd5280094f0 tx=0x7fd528004ad0 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:57:27.738 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.736+0000 7fd5420c7b80 1 -- 172.21.15.151:0/3249040312 shutdown_connections 2024-05-21T21:57:27.738 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.736+0000 7fd5420c7b80 1 --2- 172.21.15.151:0/3249040312 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56363283c880 0x56363283cce0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:27.738 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.736+0000 7fd5420c7b80 1 --2- 172.21.15.151:0/3249040312 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563632841eb0 0x563632842290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:27.738 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.736+0000 7fd5420c7b80 1 --2- 172.21.15.151:0/3249040312 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5636328427d0 0x56363283c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:27.738 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.736+0000 7fd5420c7b80 1 -- 172.21.15.151:0/3249040312 >> 172.21.15.151:0/3249040312 conn(0x56363276abe0 msgr2=0x563632837010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:57:27.738 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.736+0000 7fd5420c7b80 1 -- 172.21.15.151:0/3249040312 shutdown_connections 2024-05-21T21:57:27.738 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.736+0000 7fd5420c7b80 1 -- 172.21.15.151:0/3249040312 wait complete. 2024-05-21T21:57:27.739 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.737+0000 7fd5420c7b80 1 -- start start 2024-05-21T21:57:27.739 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.737+0000 7fd5420c7b80 1 Processor -- start 2024-05-21T21:57:27.740 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.738+0000 7fd5420c7b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56363283c880 0x5636328484a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:27.740 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.738+0000 7fd5420c7b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563632841eb0 0x563632845840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:27.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.738+0000 7fd5420c7b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5636328427d0 0x563632845d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:27.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.738+0000 7fd5420c7b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x563632690fa0 con 0x5636328427d0 2024-05-21T21:57:27.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.738+0000 7fd5420c7b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56363272b760 con 0x563632841eb0 2024-05-21T21:57:27.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.738+0000 7fd5420c7b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x563632849240 con 0x56363283c880 2024-05-21T21:57:27.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.738+0000 7fd5420c7b80 10 client.0 _ll_register_callbacks cb 0x563632734c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:57:27.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.738+0000 7fd534e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56363283c880 0x5636328484a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:27.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.738+0000 7fd534e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56363283c880 0x5636328484a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:40680/0 (socket says 172.21.15.151:40680) 2024-05-21T21:57:27.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.738+0000 7fd534e00700 1 -- 172.21.15.151:0/3090940758 learned_addr learned my addr 172.21.15.151:0/3090940758 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:57:27.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.738+0000 7fd52fe00700 1 --2- 172.21.15.151:0/3090940758 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563632841eb0 0x563632845840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:27.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.738+0000 7fd535800700 1 --2- 172.21.15.151:0/3090940758 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5636328427d0 0x563632845d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:27.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.739+0000 7fd534e00700 1 -- 172.21.15.151:0/3090940758 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563632841eb0 msgr2=0x563632845840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:27.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.739+0000 7fd534e00700 1 --2- 172.21.15.151:0/3090940758 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563632841eb0 0x563632845840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:27.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.739+0000 7fd534e00700 1 -- 172.21.15.151:0/3090940758 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5636328427d0 msgr2=0x563632845d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:27.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.739+0000 7fd534e00700 1 --2- 172.21.15.151:0/3090940758 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5636328427d0 0x563632845d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:27.742 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.739+0000 7fd534e00700 1 -- 172.21.15.151:0/3090940758 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636326da8d0 con 0x56363283c880 2024-05-21T21:57:27.742 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.739+0000 7fd535800700 1 --2- 172.21.15.151:0/3090940758 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5636328427d0 0x563632845d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T21:57:27.742 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.739+0000 7fd534e00700 1 --2- 172.21.15.151:0/3090940758 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56363283c880 0x5636328484a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd5240e3840 tx=0x7fd524009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea600e6e672a79fd server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:57:27.742 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.739+0000 7fd516a00700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-21T21:57:27.742 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.740+0000 7fd516a00700 1 -- 172.21.15.151:0/3090940758 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5240aa690 con 0x56363283c880 2024-05-21T21:57:27.742 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.740+0000 7fd516a00700 1 -- 172.21.15.151:0/3090940758 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fd5240a6eb0 con 0x56363283c880 2024-05-21T21:57:27.742 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.740+0000 7fd5420c7b80 1 -- 172.21.15.151:0/3090940758 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd51c0043f0 con 0x56363283c880 2024-05-21T21:57:27.742 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.740+0000 7fd516a00700 1 -- 172.21.15.151:0/3090940758 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd5240aa030 con 0x56363283c880 2024-05-21T21:57:27.742 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.740+0000 7fd5420c7b80 10 client.25690 fetch_fsmap learned FSMap version 29 2024-05-21T21:57:27.742 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.740+0000 7fd5420c7b80 1 -- 172.21.15.151:0/3090940758 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56363275b060 con 0x56363283c880 2024-05-21T21:57:27.742 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.740+0000 7fd516a00700 1 -- 172.21.15.151:0/3090940758 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd51c0043f0 con 0x56363283c880 2024-05-21T21:57:27.743 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.741+0000 7fd516a00700 1 -- 172.21.15.151:0/3090940758 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fd5240a6b50 con 0x56363283c880 2024-05-21T21:57:27.743 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.741+0000 7fd5420c7b80 10 client.25690 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T21:57:27.743 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.741+0000 7fd5420c7b80 10 client.25690 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-05-21T21:57:27.743 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.741+0000 7fd5420c7b80 20 client.25690 populate_metadata read hostname 'smithi151' 2024-05-21T21:57:27.743 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.741+0000 7fd5420c7b80 1 --2- 172.21.15.151:0/3090940758 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x563632855cb0 0x5636328580a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:27.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.741+0000 7fd5420c7b80 4 client.25690 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T21:57:27.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.741+0000 7fd5420c7b80 1 -- 172.21.15.151:0/3090940758 --> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56363279cd90 con 0x563632855cb0 2024-05-21T21:57:27.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.742+0000 7fd52fe00700 1 --2- 172.21.15.151:0/3090940758 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x563632855cb0 0x5636328580a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:27.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.743+0000 7fd52fe00700 1 --2- 172.21.15.151:0/3090940758 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x563632855cb0 0x5636328580a0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bbdcf7e860c03c3 server_cookie=dcb22d6178bd758d in_seq=0 out_seq=0 2024-05-21T21:57:27.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.743+0000 7fd516a00700 10 client.25690 ms_handle_connect on v2:172.21.15.164:6836/1554609572 2024-05-21T21:57:27.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.743+0000 7fd516a00700 1 -- 172.21.15.151:0/3090940758 <== mds.2 v2:172.21.15.164:6836/1554609572 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x56363279cd90 con 0x563632855cb0 2024-05-21T21:57:27.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.743+0000 7fd516a00700 10 client.25690 handle_command_reply: tid=0 2024-05-21T21:57:27.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.744+0000 7fd5420c7b80 1 client.25690 shutdown 2024-05-21T21:57:27.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.744+0000 7fd5420c7b80 1 -- 172.21.15.151:0/3090940758 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56363283c880 msgr2=0x5636328484a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:27.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.744+0000 7fd5420c7b80 1 --2- 172.21.15.151:0/3090940758 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56363283c880 0x5636328484a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd5240e3840 tx=0x7fd524009260 comp rx=0 tx=0).stop 2024-05-21T21:57:27.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.744+0000 7fd5420c7b80 1 -- 172.21.15.151:0/3090940758 shutdown_connections 2024-05-21T21:57:27.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.744+0000 7fd5420c7b80 1 --2- 172.21.15.151:0/3090940758 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x563632855cb0 0x5636328580a0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:27.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.744+0000 7fd5420c7b80 1 --2- 172.21.15.151:0/3090940758 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5636328427d0 0x563632845d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:27.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.744+0000 7fd516a00700 0 client.25690 ms_handle_reset on v2:172.21.15.164:6836/1554609572 2024-05-21T21:57:27.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.744+0000 7fd5420c7b80 1 --2- 172.21.15.151:0/3090940758 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56363283c880 0x5636328484a0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:27.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.744+0000 7fd5420c7b80 1 --2- 172.21.15.151:0/3090940758 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563632841eb0 0x563632845840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:27.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.744+0000 7fd5420c7b80 1 -- 172.21.15.151:0/3090940758 >> 172.21.15.151:0/3090940758 conn(0x56363276abe0 msgr2=0x56363283b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:57:27.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.744+0000 7fd5420c7b80 1 -- 172.21.15.151:0/3090940758 shutdown_connections 2024-05-21T21:57:27.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.745+0000 7fd5420c7b80 1 -- 172.21.15.151:0/3090940758 wait complete. 2024-05-21T21:57:27.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.745+0000 7fd5420c7b80 20 client.25690 trim_cache size 0 max 16384 2024-05-21T21:57:27.762 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.762+0000 7fd5420c7b80 1 Processor -- start 2024-05-21T21:57:27.762 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.762+0000 7fd5420c7b80 1 -- start start 2024-05-21T21:57:27.762 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.762+0000 7fd5420c7b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56363283c880 0x56363283c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:27.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.762+0000 7fd5420c7b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563632841eb0 0x563632845620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:27.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.762+0000 7fd5420c7b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5636328427d0 0x563632845b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:27.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.762+0000 7fd5420c7b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd5301aff90 con 0x563632841eb0 2024-05-21T21:57:27.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.762+0000 7fd5420c7b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd5301b1ad0 con 0x56363283c880 2024-05-21T21:57:27.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.762+0000 7fd5420c7b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd5301b0660 con 0x5636328427d0 2024-05-21T21:57:27.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.763+0000 7fd534e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56363283c880 0x56363283c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:27.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.763+0000 7fd534e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56363283c880 0x56363283c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:38032/0 (socket says 172.21.15.151:38032) 2024-05-21T21:57:27.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.763+0000 7fd534e00700 1 -- 172.21.15.151:0/3818863741 learned_addr learned my addr 172.21.15.151:0/3818863741 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:57:27.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.763+0000 7fd52fe00700 1 --2- 172.21.15.151:0/3818863741 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563632841eb0 0x563632845620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:27.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.763+0000 7fd535800700 1 --2- 172.21.15.151:0/3818863741 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5636328427d0 0x563632845b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:27.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.763+0000 7fd535800700 1 -- 172.21.15.151:0/3818863741 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56363283c880 msgr2=0x56363283c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:27.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.763+0000 7fd535800700 1 --2- 172.21.15.151:0/3818863741 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56363283c880 0x56363283c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:27.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.763+0000 7fd535800700 1 -- 172.21.15.151:0/3818863741 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563632841eb0 msgr2=0x563632845620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:27.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.763+0000 7fd535800700 1 --2- 172.21.15.151:0/3818863741 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563632841eb0 0x563632845620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:27.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.763+0000 7fd535800700 1 -- 172.21.15.151:0/3818863741 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563632837f40 con 0x5636328427d0 2024-05-21T21:57:27.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.764+0000 7fd535800700 1 --2- 172.21.15.151:0/3818863741 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5636328427d0 0x563632845b60 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd528007550 tx=0x7fd528016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f79ab200bbd1c768 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:57:27.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.764+0000 7fd521600700 1 -- 172.21.15.151:0/3818863741 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd528017e60 con 0x5636328427d0 2024-05-21T21:57:27.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.764+0000 7fd521600700 1 -- 172.21.15.151:0/3818863741 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd528029070 con 0x5636328427d0 2024-05-21T21:57:27.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.764+0000 7fd521600700 1 -- 172.21.15.151:0/3818863741 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd528016960 con 0x5636328427d0 2024-05-21T21:57:27.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.765+0000 7fd5420c7b80 1 -- 172.21.15.151:0/3818863741 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5636328427d0 msgr2=0x563632845b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:27.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.765+0000 7fd5420c7b80 1 --2- 172.21.15.151:0/3818863741 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5636328427d0 0x563632845b60 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd528007550 tx=0x7fd528016000 comp rx=0 tx=0).stop 2024-05-21T21:57:27.766 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.765+0000 7fd5420c7b80 1 -- 172.21.15.151:0/3818863741 shutdown_connections 2024-05-21T21:57:27.766 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.765+0000 7fd5420c7b80 1 --2- 172.21.15.151:0/3818863741 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56363283c880 0x56363283c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:27.767 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.765+0000 7fd5420c7b80 1 --2- 172.21.15.151:0/3818863741 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5636328427d0 0x563632845b60 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:27.767 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.765+0000 7fd5420c7b80 1 --2- 172.21.15.151:0/3818863741 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563632841eb0 0x563632845620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:27.767 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.765+0000 7fd5420c7b80 1 -- 172.21.15.151:0/3818863741 >> 172.21.15.151:0/3818863741 conn(0x56363276abe0 msgr2=0x563632844150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:57:27.767 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.766+0000 7fd5420c7b80 1 -- 172.21.15.151:0/3818863741 shutdown_connections 2024-05-21T21:57:27.767 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.766+0000 7fd5420c7b80 1 -- 172.21.15.151:0/3818863741 wait complete. 2024-05-21T21:57:27.767 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.767+0000 7fd5420c7b80 1 -- start start 2024-05-21T21:57:27.767 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.767+0000 7fd5420c7b80 1 Processor -- start 2024-05-21T21:57:27.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.768+0000 7fd5420c7b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56363283c880 0x56363285d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:27.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.768+0000 7fd5420c7b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563632841eb0 0x56363285dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:27.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.768+0000 7fd5420c7b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636328427d0 0x56363285e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:27.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.768+0000 7fd5420c7b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd5301b1400 con 0x56363283c880 2024-05-21T21:57:27.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.768+0000 7fd5420c7b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd5301b0d30 con 0x5636328427d0 2024-05-21T21:57:27.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.768+0000 7fd5420c7b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd5301adf30 con 0x563632841eb0 2024-05-21T21:57:27.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.768+0000 7fd5420c7b80 10 client.0 _ll_register_callbacks cb 0x563632734c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:57:27.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.769+0000 7fd534e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56363283c880 0x56363285d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:27.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.769+0000 7fd535800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636328427d0 0x56363285e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:27.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.769+0000 7fd52fe00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563632841eb0 0x56363285dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:27.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.769+0000 7fd535800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636328427d0 0x56363285e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:38046/0 (socket says 172.21.15.151:38046) 2024-05-21T21:57:27.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.769+0000 7fd535800700 1 -- 172.21.15.151:0/3753307496 learned_addr learned my addr 172.21.15.151:0/3753307496 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:57:27.770 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.769+0000 7fd535800700 1 -- 172.21.15.151:0/3753307496 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563632841eb0 msgr2=0x56363285dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:27.770 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.769+0000 7fd535800700 1 --2- 172.21.15.151:0/3753307496 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563632841eb0 0x56363285dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:27.770 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.769+0000 7fd535800700 1 -- 172.21.15.151:0/3753307496 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56363283c880 msgr2=0x56363285d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:27.770 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.769+0000 7fd535800700 1 --2- 172.21.15.151:0/3753307496 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56363283c880 0x56363285d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:27.770 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.769+0000 7fd535800700 1 -- 172.21.15.151:0/3753307496 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636328c82a0 con 0x5636328427d0 2024-05-21T21:57:27.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.770+0000 7fd534e00700 1 --2- 172.21.15.151:0/3753307496 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56363283c880 0x56363285d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T21:57:27.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.770+0000 7fd52fe00700 1 --2- 172.21.15.151:0/3753307496 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563632841eb0 0x56363285dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:57:27.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.770+0000 7fd535800700 1 --2- 172.21.15.151:0/3753307496 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636328427d0 0x56363285e140 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd528016db0 tx=0x7fd528016de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2a65aa1c479dd45 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:57:27.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.770+0000 7fd516a00700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-21T21:57:27.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.771+0000 7fd516a00700 1 -- 172.21.15.151:0/3753307496 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd528017e60 con 0x5636328427d0 2024-05-21T21:57:27.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.771+0000 7fd516a00700 1 -- 172.21.15.151:0/3753307496 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fd528024640 con 0x5636328427d0 2024-05-21T21:57:27.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.771+0000 7fd5420c7b80 1 -- 172.21.15.151:0/3753307496 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd528029070 con 0x5636328427d0 2024-05-21T21:57:27.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.771+0000 7fd516a00700 1 -- 172.21.15.151:0/3753307496 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd528017e60 con 0x5636328427d0 2024-05-21T21:57:27.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.772+0000 7fd5420c7b80 10 client.25679 fetch_fsmap learned FSMap version 29 2024-05-21T21:57:27.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.772+0000 7fd5420c7b80 1 -- 172.21.15.151:0/3753307496 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5636328c9560 con 0x5636328427d0 2024-05-21T21:57:27.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.772+0000 7fd516a00700 1 -- 172.21.15.151:0/3753307496 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd528029070 con 0x5636328427d0 2024-05-21T21:57:27.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.772+0000 7fd516a00700 1 -- 172.21.15.151:0/3753307496 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fd528042720 con 0x5636328427d0 2024-05-21T21:57:27.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.772+0000 7fd5420c7b80 10 client.25679 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T21:57:27.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.772+0000 7fd5420c7b80 10 client.25679 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-05-21T21:57:27.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.772+0000 7fd5420c7b80 20 client.25679 populate_metadata read hostname 'smithi151' 2024-05-21T21:57:27.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.772+0000 7fd5420c7b80 1 --2- 172.21.15.151:0/3753307496 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x5636328c9b90 0x5636328cbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:27.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.772+0000 7fd5420c7b80 4 client.25679 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T21:57:27.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.772+0000 7fd5420c7b80 1 -- 172.21.15.151:0/3753307496 --> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563632796570 con 0x5636328c9b90 2024-05-21T21:57:27.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.773+0000 7fd534e00700 1 --2- 172.21.15.151:0/3753307496 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x5636328c9b90 0x5636328cbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:27.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.774+0000 7fd534e00700 1 --2- 172.21.15.151:0/3753307496 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x5636328c9b90 0x5636328cbfb0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bb268cf9d34b7b6d server_cookie=94912cae3bd699e1 in_seq=0 out_seq=0 2024-05-21T21:57:27.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.774+0000 7fd516a00700 10 client.25679 ms_handle_connect on v2:172.21.15.164:6836/1554609572 2024-05-21T21:57:27.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.775+0000 7fd516a00700 1 -- 172.21.15.151:0/3753307496 <== mds.2 v2:172.21.15.164:6836/1554609572 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563632796570 con 0x5636328c9b90 2024-05-21T21:57:27.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.775+0000 7fd516a00700 10 client.25679 handle_command_reply: tid=0 2024-05-21T21:57:27.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.775+0000 7fd5420c7b80 1 client.25679 shutdown 2024-05-21T21:57:27.776 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.775+0000 7fd5420c7b80 1 -- 172.21.15.151:0/3753307496 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636328427d0 msgr2=0x56363285e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:27.776 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.775+0000 7fd5420c7b80 1 --2- 172.21.15.151:0/3753307496 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636328427d0 0x56363285e140 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd528016db0 tx=0x7fd528016de0 comp rx=0 tx=0).stop 2024-05-21T21:57:27.776 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.776+0000 7fd5420c7b80 1 -- 172.21.15.151:0/3753307496 shutdown_connections 2024-05-21T21:57:27.777 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.776+0000 7fd5420c7b80 1 --2- 172.21.15.151:0/3753307496 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x5636328c9b90 0x5636328cbfb0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:27.777 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.776+0000 7fd516a00700 0 client.25679 ms_handle_reset on v2:172.21.15.164:6836/1554609572 2024-05-21T21:57:27.777 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.776+0000 7fd5420c7b80 1 --2- 172.21.15.151:0/3753307496 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5636328427d0 0x56363285e140 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:27.777 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.776+0000 7fd5420c7b80 1 --2- 172.21.15.151:0/3753307496 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56363283c880 0x56363285d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:27.777 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.776+0000 7fd5420c7b80 1 --2- 172.21.15.151:0/3753307496 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563632841eb0 0x56363285dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:27.777 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.776+0000 7fd5420c7b80 1 -- 172.21.15.151:0/3753307496 >> 172.21.15.151:0/3753307496 conn(0x56363276abe0 msgr2=0x563632844090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:57:27.777 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.776+0000 7fd5420c7b80 1 -- 172.21.15.151:0/3753307496 shutdown_connections 2024-05-21T21:57:27.777 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.777+0000 7fd5420c7b80 1 -- 172.21.15.151:0/3753307496 wait complete. 2024-05-21T21:57:27.778 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.777+0000 7fd5420c7b80 20 client.25679 trim_cache size 0 max 16384 2024-05-21T21:57:27.778 INFO:teuthology.orchestra.run.smithi151.stdout:[] 2024-05-21T21:57:27.778 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.778+0000 7fd537800700 1 -- 172.21.15.151:0/3637585074 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fd5180b2a70 msgr2=0x7fd5180b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:27.778 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.778+0000 7fd537800700 1 --2- 172.21.15.151:0/3637585074 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fd5180b2a70 0x7fd5180b4f30 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd5280094f0 tx=0x7fd528004ad0 comp rx=0 tx=0).stop 2024-05-21T21:57:27.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.778+0000 7fd537800700 1 -- 172.21.15.151:0/3637585074 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd530075770 msgr2=0x7fd53019bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:27.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.778+0000 7fd537800700 1 --2- 172.21.15.151:0/3637585074 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd530075770 0x7fd53019bdf0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd524009230 tx=0x7fd52400fa70 comp rx=0 tx=0).stop 2024-05-21T21:57:27.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.778+0000 7fd537800700 1 -- 172.21.15.151:0/3637585074 shutdown_connections 2024-05-21T21:57:27.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.778+0000 7fd537800700 1 --2- 172.21.15.151:0/3637585074 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fd5180b2a70 0x7fd5180b4f30 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:27.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.778+0000 7fd537800700 1 --2- 172.21.15.151:0/3637585074 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd530075770 0x7fd53019bdf0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:27.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.778+0000 7fd537800700 1 --2- 172.21.15.151:0/3637585074 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd53010ffa0 0x7fd5301a06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:27.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.778+0000 7fd537800700 1 --2- 172.21.15.151:0/3637585074 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd530076090 0x7fd53019c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:27.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.778+0000 7fd537800700 1 -- 172.21.15.151:0/3637585074 >> 172.21.15.151:0/3637585074 conn(0x7fd53007a6a0 msgr2=0x7fd530100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:57:27.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.779+0000 7fd537800700 1 -- 172.21.15.151:0/3637585074 shutdown_connections 2024-05-21T21:57:27.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:27.779+0000 7fd537800700 1 -- 172.21.15.151:0/3637585074 wait complete. 2024-05-21T21:57:27.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:57:27 smithi164 ceph-mon[36269]: pgmap v58: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-05-21T21:57:28.663 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-21T21:57:29.155 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:57:29.571 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:57:29 smithi151 ceph-mon[31623]: pgmap v59: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2024-05-21T21:57:29.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:57:29 smithi184 ceph-mon[36942]: pgmap v59: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2024-05-21T21:57:29.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:57:29 smithi164 ceph-mon[36269]: pgmap v59: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2024-05-21T21:57:31.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.030+0000 7fcdc9a00700 1 -- 172.21.15.151:0/3665922794 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcdc4109540 msgr2=0x7fcdc4109920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:31.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.030+0000 7fcdc9a00700 1 --2- 172.21.15.151:0/3665922794 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcdc4109540 0x7fcdc4109920 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fcdb4009920 tx=0x7fcdb4009c30 comp rx=0 tx=0).stop 2024-05-21T21:57:31.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.030+0000 7fcdc9a00700 1 -- 172.21.15.151:0/3665922794 shutdown_connections 2024-05-21T21:57:31.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.030+0000 7fcdc9a00700 1 --2- 172.21.15.151:0/3665922794 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcdc4109540 0x7fcdc4109920 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:31.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.030+0000 7fcdc9a00700 1 --2- 172.21.15.151:0/3665922794 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcdc4103e50 0x7fcdc4110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:31.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.030+0000 7fcdc9a00700 1 --2- 172.21.15.151:0/3665922794 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcdc4103490 0x7fcdc4103910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:31.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.030+0000 7fcdc9a00700 1 -- 172.21.15.151:0/3665922794 >> 172.21.15.151:0/3665922794 conn(0x7fcdc40fefc0 msgr2=0x7fcdc41013e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:57:31.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.030+0000 7fcdc9a00700 1 -- 172.21.15.151:0/3665922794 shutdown_connections 2024-05-21T21:57:31.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.031+0000 7fcdc9a00700 1 -- 172.21.15.151:0/3665922794 wait complete. 2024-05-21T21:57:31.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.032+0000 7fcdc9a00700 1 Processor -- start 2024-05-21T21:57:31.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.033+0000 7fcdc9a00700 1 -- start start 2024-05-21T21:57:31.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.033+0000 7fcdc9a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcdc4103490 0x7fcdc419f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:31.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.033+0000 7fcdc9a00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcdc4103e50 0x7fcdc419fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:31.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.033+0000 7fcdc9a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcdc4109540 0x7fcdc4199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:31.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.033+0000 7fcdc9a00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc4113650 con 0x7fcdc4109540 2024-05-21T21:57:31.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.033+0000 7fcdc9a00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc41134d0 con 0x7fcdc4103490 2024-05-21T21:57:31.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.033+0000 7fcdc9a00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc41137d0 con 0x7fcdc4103e50 2024-05-21T21:57:31.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.034+0000 7fcdc3400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcdc4103e50 0x7fcdc419fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:31.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.034+0000 7fcdc3e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcdc4103490 0x7fcdc419f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:31.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.034+0000 7fcdc3e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcdc4103490 0x7fcdc419f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:38058/0 (socket says 172.21.15.151:38058) 2024-05-21T21:57:31.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.034+0000 7fcdc3e00700 1 -- 172.21.15.151:0/3368127504 learned_addr learned my addr 172.21.15.151:0/3368127504 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:57:31.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.034+0000 7fcdc9000700 1 --2- 172.21.15.151:0/3368127504 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcdc4109540 0x7fcdc4199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:31.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.035+0000 7fcdc9000700 1 -- 172.21.15.151:0/3368127504 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcdc4103e50 msgr2=0x7fcdc419fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:31.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.035+0000 7fcdc9000700 1 --2- 172.21.15.151:0/3368127504 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcdc4103e50 0x7fcdc419fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:31.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.035+0000 7fcdc9000700 1 -- 172.21.15.151:0/3368127504 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcdc4103490 msgr2=0x7fcdc419f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:31.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.035+0000 7fcdc9000700 1 --2- 172.21.15.151:0/3368127504 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcdc4103490 0x7fcdc419f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:31.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.035+0000 7fcdc9000700 1 -- 172.21.15.151:0/3368127504 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdc419a0d0 con 0x7fcdc4109540 2024-05-21T21:57:31.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.035+0000 7fcdc3e00700 1 --2- 172.21.15.151:0/3368127504 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcdc4103490 0x7fcdc419f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:57:31.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.035+0000 7fcdc9000700 1 --2- 172.21.15.151:0/3368127504 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcdc4109540 0x7fcdc4199970 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc00efc0 tx=0x7fcdbc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:57:31.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.036+0000 7fcdc0c00700 1 -- 172.21.15.151:0/3368127504 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdbc0092e0 con 0x7fcdc4109540 2024-05-21T21:57:31.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.036+0000 7fcdc0c00700 1 -- 172.21.15.151:0/3368127504 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdbc004bb0 con 0x7fcdc4109540 2024-05-21T21:57:31.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.036+0000 7fcdc9a00700 1 -- 172.21.15.151:0/3368127504 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdc419a360 con 0x7fcdc4109540 2024-05-21T21:57:31.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.036+0000 7fcdc9a00700 1 -- 172.21.15.151:0/3368127504 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdc4075770 con 0x7fcdc4109540 2024-05-21T21:57:31.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.036+0000 7fcdc0c00700 1 -- 172.21.15.151:0/3368127504 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdbc004060 con 0x7fcdc4109540 2024-05-21T21:57:31.043 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.042+0000 7fcdc0c00700 1 -- 172.21.15.151:0/3368127504 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7fcdbc01b030 con 0x7fcdc4109540 2024-05-21T21:57:31.043 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.042+0000 7fcdd597db80 1 Processor -- start 2024-05-21T21:57:31.043 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.043+0000 7fcdd597db80 1 -- start start 2024-05-21T21:57:31.045 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.043+0000 7fcdd597db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c02125f020 0x55c02125f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:31.046 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.043+0000 7fcdd597db80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c02125f940 0x55c0212594b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:31.046 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.043+0000 7fcdd597db80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c021259ad0 0x55c021259f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:31.046 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.043+0000 7fcdd597db80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c0211b7e40 con 0x55c02125f940 2024-05-21T21:57:31.046 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.043+0000 7fcdd597db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55c0211a79e0 con 0x55c02125f020 2024-05-21T21:57:31.046 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.043+0000 7fcdd597db80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55c0211aba20 con 0x55c021259ad0 2024-05-21T21:57:31.046 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.044+0000 7fcdc3400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c02125f020 0x55c02125f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:31.046 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.044+0000 7fcdc3400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c02125f020 0x55c02125f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:38062/0 (socket says 172.21.15.151:38062) 2024-05-21T21:57:31.046 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.044+0000 7fcdc3400700 1 -- 172.21.15.151:0/1521149735 learned_addr learned my addr 172.21.15.151:0/1521149735 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:57:31.046 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.044+0000 7fcdc3400700 1 --2- 172.21.15.151:0/1521149735 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c021259ad0 0x55c021259f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:31.046 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.044+0000 7fcdc3e00700 1 --2- 172.21.15.151:0/1521149735 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c02125f940 0x55c0212594b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:31.046 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.044+0000 7fcdc3400700 1 -- 172.21.15.151:0/1521149735 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c021259ad0 msgr2=0x55c021259f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:31.046 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.044+0000 7fcdc3400700 1 --2- 172.21.15.151:0/1521149735 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c021259ad0 0x55c021259f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:31.046 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.044+0000 7fcdc3400700 1 -- 172.21.15.151:0/1521149735 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c02125f940 msgr2=0x55c0212594b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:31.047 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.044+0000 7fcdc0c00700 1 --2- 172.21.15.151:0/3368127504 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fcda80b2a70 0x7fcda80b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:31.047 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.045+0000 7fcdc0c00700 1 -- 172.21.15.151:0/3368127504 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fcdbc0e1ff0 con 0x7fcdc4109540 2024-05-21T21:57:31.047 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.045+0000 7fcdc3400700 1 --2- 172.21.15.151:0/1521149735 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c02125f940 0x55c0212594b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:31.047 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.045+0000 7fcdc3400700 1 -- 172.21.15.151:0/1521149735 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c021170420 con 0x55c02125f020 2024-05-21T21:57:31.047 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.045+0000 7fcdc3e00700 1 --2- 172.21.15.151:0/1521149735 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c02125f940 0x55c0212594b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T21:57:31.047 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.045+0000 7fcdc3400700 1 --2- 172.21.15.151:0/3368127504 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fcda80b2a70 0x7fcda80b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:31.047 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.047+0000 7fcdc3400700 1 --2- 172.21.15.151:0/3368127504 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fcda80b2a70 0x7fcda80b4f30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fcdb0008000 tx=0x7fcdb001a000 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:57:31.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.047+0000 7fcdc3400700 1 --2- 172.21.15.151:0/1521149735 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c02125f020 0x55c02125f400 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fcdb0012650 tx=0x7fcdb0009770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b44486a8acb9ec4e server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:57:31.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.047+0000 7fcdb9600700 1 -- 172.21.15.151:0/1521149735 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb0023070 con 0x55c02125f020 2024-05-21T21:57:31.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.047+0000 7fcdb9600700 1 -- 172.21.15.151:0/1521149735 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdb001fab0 con 0x55c02125f020 2024-05-21T21:57:31.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.047+0000 7fcdb9600700 1 -- 172.21.15.151:0/1521149735 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb000ea50 con 0x55c02125f020 2024-05-21T21:57:31.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.048+0000 7fcdd597db80 1 -- 172.21.15.151:0/1521149735 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c02125f020 msgr2=0x55c02125f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:31.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.048+0000 7fcdd597db80 1 --2- 172.21.15.151:0/1521149735 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c02125f020 0x55c02125f400 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fcdb0012650 tx=0x7fcdb0009770 comp rx=0 tx=0).stop 2024-05-21T21:57:31.049 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.049+0000 7fcdd597db80 1 -- 172.21.15.151:0/1521149735 shutdown_connections 2024-05-21T21:57:31.049 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.049+0000 7fcdd597db80 1 --2- 172.21.15.151:0/1521149735 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c02125f020 0x55c02125f400 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:31.049 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.049+0000 7fcdd597db80 1 --2- 172.21.15.151:0/1521149735 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c021259ad0 0x55c021259f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:31.049 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.049+0000 7fcdd597db80 1 --2- 172.21.15.151:0/1521149735 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c02125f940 0x55c0212594b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:31.049 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.049+0000 7fcdd597db80 1 -- 172.21.15.151:0/1521149735 >> 172.21.15.151:0/1521149735 conn(0x55c021187be0 msgr2=0x55c021254150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:57:31.050 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.049+0000 7fcdd597db80 1 -- 172.21.15.151:0/1521149735 shutdown_connections 2024-05-21T21:57:31.050 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.049+0000 7fcdd597db80 1 -- 172.21.15.151:0/1521149735 wait complete. 2024-05-21T21:57:31.050 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.050+0000 7fcdd597db80 1 -- start start 2024-05-21T21:57:31.050 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.050+0000 7fcdd597db80 1 Processor -- start 2024-05-21T21:57:31.051 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.051+0000 7fcdd597db80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c021259ad0 0x55c021267ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:31.051 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.051+0000 7fcdd597db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c02125f020 0x55c021268220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:31.051 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.051+0000 7fcdd597db80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c02125f940 0x55c02126da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:31.051 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.051+0000 7fcdd597db80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c0210adfa0 con 0x55c02125f940 2024-05-21T21:57:31.051 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.051+0000 7fcdd597db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55c021148760 con 0x55c02125f020 2024-05-21T21:57:31.051 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.051+0000 7fcdd597db80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55c021260250 con 0x55c021259ad0 2024-05-21T21:57:31.053 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.051+0000 7fcdd597db80 10 client.0 _ll_register_callbacks cb 0x55c021151c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:57:31.053 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.051+0000 7fcdc3e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c021259ad0 0x55c021267ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:31.053 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.051+0000 7fcdc3400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c02125f020 0x55c021268220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:31.053 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.052+0000 7fcdc9000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c02125f940 0x55c02126da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:31.053 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.052+0000 7fcdc9000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c02125f940 0x55c02126da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:58804/0 (socket says 172.21.15.151:58804) 2024-05-21T21:57:31.053 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.052+0000 7fcdc9000700 1 -- 172.21.15.151:0/2285869058 learned_addr learned my addr 172.21.15.151:0/2285869058 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:57:31.053 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.052+0000 7fcdc9000700 1 -- 172.21.15.151:0/2285869058 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c021259ad0 msgr2=0x55c021267ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:31.053 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.052+0000 7fcdc9000700 1 --2- 172.21.15.151:0/2285869058 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c021259ad0 0x55c021267ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:31.053 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.052+0000 7fcdc9000700 1 -- 172.21.15.151:0/2285869058 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c02125f020 msgr2=0x55c021268220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:31.053 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.052+0000 7fcdc9000700 1 --2- 172.21.15.151:0/2285869058 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c02125f020 0x55c021268220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:31.053 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.052+0000 7fcdc9000700 1 -- 172.21.15.151:0/2285869058 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0210f78d0 con 0x55c02125f940 2024-05-21T21:57:31.053 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.053+0000 7fcdc9000700 1 --2- 172.21.15.151:0/2285869058 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c02125f940 0x55c02126da30 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc0a92e0 tx=0x7fcdbc004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fd26ab25f6c9164 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:57:31.053 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.053+0000 7fcdae000700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-21T21:57:31.054 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.054+0000 7fcdae000700 1 -- 172.21.15.151:0/2285869058 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdbc0a9350 con 0x55c02125f940 2024-05-21T21:57:31.054 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.054+0000 7fcdae000700 1 -- 172.21.15.151:0/2285869058 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fcdbc0b1ea0 con 0x55c02125f940 2024-05-21T21:57:31.054 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.054+0000 7fcdd597db80 1 -- 172.21.15.151:0/2285869058 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcdb001fab0 con 0x55c02125f940 2024-05-21T21:57:31.055 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.054+0000 7fcdae000700 1 -- 172.21.15.151:0/2285869058 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcdbc0ad700 con 0x55c02125f940 2024-05-21T21:57:31.055 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.054+0000 7fcdd597db80 10 client.15567 fetch_fsmap learned FSMap version 29 2024-05-21T21:57:31.055 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.054+0000 7fcdd597db80 1 -- 172.21.15.151:0/2285869058 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c021178060 con 0x55c02125f940 2024-05-21T21:57:31.055 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.055+0000 7fcdae000700 1 -- 172.21.15.151:0/2285869058 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdb001fab0 con 0x55c02125f940 2024-05-21T21:57:31.055 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.055+0000 7fcdae000700 1 -- 172.21.15.151:0/2285869058 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fcdbc0a4cc0 con 0x55c02125f940 2024-05-21T21:57:31.056 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.055+0000 7fcdd597db80 10 client.15567 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T21:57:31.056 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.055+0000 7fcdd597db80 10 client.15567 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-21T21:57:31.056 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.055+0000 7fcdd597db80 20 client.15567 populate_metadata read hostname 'smithi151' 2024-05-21T21:57:31.056 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.055+0000 7fcdd597db80 1 --2- 172.21.15.151:0/2285869058 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x55c0212840e0 0x55c0212864d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:31.057 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.055+0000 7fcdd597db80 4 client.15567 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T21:57:31.057 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.055+0000 7fcdd597db80 1 -- 172.21.15.151:0/2285869058 --> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c0211b9d90 con 0x55c0212840e0 2024-05-21T21:57:31.057 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.056+0000 7fcdc3e00700 1 --2- 172.21.15.151:0/2285869058 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x55c0212840e0 0x55c0212864d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:31.057 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.057+0000 7fcdc3e00700 1 --2- 172.21.15.151:0/2285869058 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x55c0212840e0 0x55c0212864d0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=efc82f52c018ad70 server_cookie=35f1b24bc61b289a in_seq=0 out_seq=0 2024-05-21T21:57:31.057 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.057+0000 7fcdae000700 10 client.15567 ms_handle_connect on v2:172.21.15.184:6838/3059935143 2024-05-21T21:57:31.058 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.058+0000 7fcdae000700 1 -- 172.21.15.151:0/2285869058 <== mds.1 v2:172.21.15.184:6838/3059935143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55c0211b9d90 con 0x55c0212840e0 2024-05-21T21:57:31.058 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.058+0000 7fcdae000700 10 client.15567 handle_command_reply: tid=0 2024-05-21T21:57:31.058 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.058+0000 7fcdd597db80 1 client.15567 shutdown 2024-05-21T21:57:31.058 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.058+0000 7fcdd597db80 1 -- 172.21.15.151:0/2285869058 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c02125f940 msgr2=0x55c02126da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:31.058 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.058+0000 7fcdd597db80 1 --2- 172.21.15.151:0/2285869058 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c02125f940 0x55c02126da30 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc0a92e0 tx=0x7fcdbc004f90 comp rx=0 tx=0).stop 2024-05-21T21:57:31.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.058+0000 7fcdd597db80 1 -- 172.21.15.151:0/2285869058 shutdown_connections 2024-05-21T21:57:31.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.058+0000 7fcdd597db80 1 --2- 172.21.15.151:0/2285869058 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x55c0212840e0 0x55c0212864d0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:31.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.058+0000 7fcdae000700 0 client.15567 ms_handle_reset on v2:172.21.15.184:6838/3059935143 2024-05-21T21:57:31.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.059+0000 7fcdd597db80 1 --2- 172.21.15.151:0/2285869058 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c02125f940 0x55c02126da30 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:31.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.059+0000 7fcdd597db80 1 --2- 172.21.15.151:0/2285869058 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c021259ad0 0x55c021267ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:31.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.059+0000 7fcdd597db80 1 --2- 172.21.15.151:0/2285869058 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c02125f020 0x55c021268220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:31.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.059+0000 7fcdd597db80 1 -- 172.21.15.151:0/2285869058 >> 172.21.15.151:0/2285869058 conn(0x55c021187be0 msgr2=0x55c0212584a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:57:31.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.059+0000 7fcdd597db80 1 -- 172.21.15.151:0/2285869058 shutdown_connections 2024-05-21T21:57:31.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.059+0000 7fcdd597db80 1 -- 172.21.15.151:0/2285869058 wait complete. 2024-05-21T21:57:31.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.059+0000 7fcdd597db80 20 client.15567 trim_cache size 0 max 16384 2024-05-21T21:57:31.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.075+0000 7fcdd597db80 1 Processor -- start 2024-05-21T21:57:31.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.076+0000 7fcdd597db80 1 -- start start 2024-05-21T21:57:31.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.076+0000 7fcdd597db80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c021259ad0 0x55c02126d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:31.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.076+0000 7fcdd597db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c02125f020 0x55c021253c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:31.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.076+0000 7fcdd597db80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c02125f940 0x55c021254170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:31.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.076+0000 7fcdd597db80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcda400d600 con 0x55c021259ad0 2024-05-21T21:57:31.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.076+0000 7fcdd597db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcda400f140 con 0x55c02125f020 2024-05-21T21:57:31.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.076+0000 7fcdd597db80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fcda400dcd0 con 0x55c02125f940 2024-05-21T21:57:31.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.076+0000 7fcdc3400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c02125f020 0x55c021253c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:31.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.076+0000 7fcdc3e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c021259ad0 0x55c02126d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:31.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.076+0000 7fcdc9000700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c02125f940 0x55c021254170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:31.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.076+0000 7fcdc3400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c02125f020 0x55c021253c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:38082/0 (socket says 172.21.15.151:38082) 2024-05-21T21:57:31.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.076+0000 7fcdc3400700 1 -- 172.21.15.151:0/2257183855 learned_addr learned my addr 172.21.15.151:0/2257183855 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:57:31.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.077+0000 7fcdc3400700 1 -- 172.21.15.151:0/2257183855 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c02125f940 msgr2=0x55c021254170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:31.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.077+0000 7fcdc3400700 1 --2- 172.21.15.151:0/2257183855 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c02125f940 0x55c021254170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:31.078 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.077+0000 7fcdc3400700 1 -- 172.21.15.151:0/2257183855 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c021259ad0 msgr2=0x55c02126d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:31.078 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.077+0000 7fcdc3400700 1 --2- 172.21.15.151:0/2257183855 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c021259ad0 0x55c02126d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:31.078 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.077+0000 7fcdc3400700 1 -- 172.21.15.151:0/2257183855 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0212640e0 con 0x55c02125f020 2024-05-21T21:57:31.078 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.077+0000 7fcdc9000700 1 --2- 172.21.15.151:0/2257183855 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c02125f940 0x55c021254170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:57:31.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.077+0000 7fcdc3e00700 1 --2- 172.21.15.151:0/2257183855 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c021259ad0 0x55c02126d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T21:57:31.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.078+0000 7fcdc3400700 1 --2- 172.21.15.151:0/2257183855 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c02125f020 0x55c021253c30 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fcdb0011010 tx=0x7fcdb0004240 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1dbbc08dabceb333 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:57:31.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.078+0000 7fcdb8c00700 1 -- 172.21.15.151:0/2257183855 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb0023050 con 0x55c02125f020 2024-05-21T21:57:31.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.078+0000 7fcdb8c00700 1 -- 172.21.15.151:0/2257183855 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdb0004630 con 0x55c02125f020 2024-05-21T21:57:31.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.078+0000 7fcdb8c00700 1 -- 172.21.15.151:0/2257183855 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb0028690 con 0x55c02125f020 2024-05-21T21:57:31.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.079+0000 7fcdd597db80 1 -- 172.21.15.151:0/2257183855 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c02125f020 msgr2=0x55c021253c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:31.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.079+0000 7fcdd597db80 1 --2- 172.21.15.151:0/2257183855 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c02125f020 0x55c021253c30 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fcdb0011010 tx=0x7fcdb0004240 comp rx=0 tx=0).stop 2024-05-21T21:57:31.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.079+0000 7fcdd597db80 1 -- 172.21.15.151:0/2257183855 shutdown_connections 2024-05-21T21:57:31.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.079+0000 7fcdd597db80 1 --2- 172.21.15.151:0/2257183855 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c02125f940 0x55c021254170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:31.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.079+0000 7fcdd597db80 1 --2- 172.21.15.151:0/2257183855 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c021259ad0 0x55c02126d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:31.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.079+0000 7fcdd597db80 1 --2- 172.21.15.151:0/2257183855 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c02125f020 0x55c021253c30 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:31.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.079+0000 7fcdd597db80 1 -- 172.21.15.151:0/2257183855 >> 172.21.15.151:0/2257183855 conn(0x55c021187be0 msgr2=0x55c0212619e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:57:31.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.080+0000 7fcdd597db80 1 -- 172.21.15.151:0/2257183855 shutdown_connections 2024-05-21T21:57:31.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.080+0000 7fcdd597db80 1 -- 172.21.15.151:0/2257183855 wait complete. 2024-05-21T21:57:31.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.080+0000 7fcdd597db80 1 -- start start 2024-05-21T21:57:31.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.080+0000 7fcdd597db80 1 Processor -- start 2024-05-21T21:57:31.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.081+0000 7fcdd597db80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c021259ad0 0x55c02128b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:31.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.081+0000 7fcdd597db80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c02125f020 0x55c02128be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:31.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.081+0000 7fcdd597db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c02125f940 0x55c02128c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:31.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.081+0000 7fcdd597db80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcda400ea70 con 0x55c021259ad0 2024-05-21T21:57:31.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.081+0000 7fcdd597db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcda400e3a0 con 0x55c02125f940 2024-05-21T21:57:31.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.081+0000 7fcdd597db80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fcda400b5a0 con 0x55c02125f020 2024-05-21T21:57:31.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.081+0000 7fcdd597db80 10 client.0 _ll_register_callbacks cb 0x55c021151c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:57:31.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.081+0000 7fcdc3400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c02125f020 0x55c02128be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:31.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.081+0000 7fcdc3400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c02125f020 0x55c02128be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:40740/0 (socket says 172.21.15.151:40740) 2024-05-21T21:57:31.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.081+0000 7fcdc3400700 1 -- 172.21.15.151:0/3220133523 learned_addr learned my addr 172.21.15.151:0/3220133523 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:57:31.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.081+0000 7fcdc9000700 1 --2- 172.21.15.151:0/3220133523 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c02125f940 0x55c02128c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:31.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.082+0000 7fcdc3400700 1 -- 172.21.15.151:0/3220133523 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c02125f940 msgr2=0x55c02128c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:31.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.082+0000 7fcdc3400700 1 --2- 172.21.15.151:0/3220133523 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c02125f940 0x55c02128c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:31.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.082+0000 7fcdc3400700 1 -- 172.21.15.151:0/3220133523 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c021259ad0 msgr2=0x55c02128b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:31.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.082+0000 7fcdc3400700 1 --2- 172.21.15.151:0/3220133523 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c021259ad0 0x55c02128b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:31.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.082+0000 7fcdc3400700 1 -- 172.21.15.151:0/3220133523 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c021268590 con 0x55c02125f020 2024-05-21T21:57:31.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.082+0000 7fcdc9000700 1 --2- 172.21.15.151:0/3220133523 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c02125f940 0x55c02128c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T21:57:31.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.082+0000 7fcdc3400700 1 --2- 172.21.15.151:0/3220133523 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c02125f020 0x55c02128be70 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fcdb0003e60 tx=0x7fcdb00038b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c81720ebaca6d4b server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:57:31.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.082+0000 7fcdae000700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-21T21:57:31.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.083+0000 7fcdae000700 1 -- 172.21.15.151:0/3220133523 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb0004430 con 0x55c02125f020 2024-05-21T21:57:31.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.083+0000 7fcdae000700 1 -- 172.21.15.151:0/3220133523 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fcdb0037070 con 0x55c02125f020 2024-05-21T21:57:31.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.083+0000 7fcdd597db80 1 -- 172.21.15.151:0/3220133523 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcdb0004630 con 0x55c02125f020 2024-05-21T21:57:31.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.083+0000 7fcdae000700 1 -- 172.21.15.151:0/3220133523 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcdb0023020 con 0x55c02125f020 2024-05-21T21:57:31.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.083+0000 7fcdd597db80 10 client.25714 fetch_fsmap learned FSMap version 29 2024-05-21T21:57:31.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.083+0000 7fcdd597db80 1 -- 172.21.15.151:0/3220133523 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c0212eebf0 con 0x55c02125f020 2024-05-21T21:57:31.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.084+0000 7fcdae000700 1 -- 172.21.15.151:0/3220133523 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdb0004630 con 0x55c02125f020 2024-05-21T21:57:31.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.084+0000 7fcdae000700 1 -- 172.21.15.151:0/3220133523 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fcdb0030070 con 0x55c02125f020 2024-05-21T21:57:31.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.084+0000 7fcdd597db80 10 client.25714 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T21:57:31.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.084+0000 7fcdd597db80 10 client.25714 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-21T21:57:31.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.084+0000 7fcdd597db80 20 client.25714 populate_metadata read hostname 'smithi151' 2024-05-21T21:57:31.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.084+0000 7fcdd597db80 1 --2- 172.21.15.151:0/3220133523 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x55c0212eefa0 0x55c0212f13c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:31.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.084+0000 7fcdd597db80 4 client.25714 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T21:57:31.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.084+0000 7fcdd597db80 1 -- 172.21.15.151:0/3220133523 --> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c0211b3570 con 0x55c0212eefa0 2024-05-21T21:57:31.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.085+0000 7fcdc3e00700 1 --2- 172.21.15.151:0/3220133523 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x55c0212eefa0 0x55c0212f13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:31.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.085+0000 7fcdae000700 10 client.25714 ms_handle_connect on v2:172.21.15.184:6838/3059935143 2024-05-21T21:57:31.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.085+0000 7fcdc3e00700 1 --2- 172.21.15.151:0/3220133523 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x55c0212eefa0 0x55c0212f13c0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d957919ea2d9f1a server_cookie=db798d5d247d2725 in_seq=0 out_seq=0 2024-05-21T21:57:31.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.086+0000 7fcdae000700 1 -- 172.21.15.151:0/3220133523 <== mds.1 v2:172.21.15.184:6838/3059935143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c0211b3570 con 0x55c0212eefa0 2024-05-21T21:57:31.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.086+0000 7fcdae000700 10 client.25714 handle_command_reply: tid=0 2024-05-21T21:57:31.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.086+0000 7fcdd597db80 1 client.25714 shutdown 2024-05-21T21:57:31.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.086+0000 7fcdd597db80 1 -- 172.21.15.151:0/3220133523 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c02125f020 msgr2=0x55c02128be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:31.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.086+0000 7fcdd597db80 1 --2- 172.21.15.151:0/3220133523 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c02125f020 0x55c02128be70 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fcdb0003e60 tx=0x7fcdb00038b0 comp rx=0 tx=0).stop 2024-05-21T21:57:31.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.086+0000 7fcdd597db80 1 -- 172.21.15.151:0/3220133523 shutdown_connections 2024-05-21T21:57:31.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.086+0000 7fcdd597db80 1 --2- 172.21.15.151:0/3220133523 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x55c0212eefa0 0x55c0212f13c0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:31.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.086+0000 7fcdd597db80 1 --2- 172.21.15.151:0/3220133523 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c02125f940 0x55c02128c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:31.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.086+0000 7fcdd597db80 1 --2- 172.21.15.151:0/3220133523 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c021259ad0 0x55c02128b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:31.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.086+0000 7fcdd597db80 1 --2- 172.21.15.151:0/3220133523 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c02125f020 0x55c02128be70 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:31.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.086+0000 7fcdd597db80 1 -- 172.21.15.151:0/3220133523 >> 172.21.15.151:0/3220133523 conn(0x55c021187be0 msgr2=0x55c0212619a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:57:31.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.086+0000 7fcdae000700 0 client.25714 ms_handle_reset on v2:172.21.15.184:6838/3059935143 2024-05-21T21:57:31.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.086+0000 7fcdd597db80 1 -- 172.21.15.151:0/3220133523 shutdown_connections 2024-05-21T21:57:31.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.086+0000 7fcdd597db80 1 -- 172.21.15.151:0/3220133523 wait complete. 2024-05-21T21:57:31.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.087+0000 7fcdd597db80 20 client.25714 trim_cache size 0 max 16384 2024-05-21T21:57:31.088 INFO:teuthology.orchestra.run.smithi151.stdout:[] 2024-05-21T21:57:31.089 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.088+0000 7fcdc9a00700 1 -- 172.21.15.151:0/3368127504 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fcda80b2a70 msgr2=0x7fcda80b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:31.089 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.088+0000 7fcdc9a00700 1 --2- 172.21.15.151:0/3368127504 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fcda80b2a70 0x7fcda80b4f30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fcdb0008000 tx=0x7fcdb001a000 comp rx=0 tx=0).stop 2024-05-21T21:57:31.089 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.088+0000 7fcdc9a00700 1 -- 172.21.15.151:0/3368127504 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcdc4109540 msgr2=0x7fcdc4199970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:31.089 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.088+0000 7fcdc9a00700 1 --2- 172.21.15.151:0/3368127504 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcdc4109540 0x7fcdc4199970 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc00efc0 tx=0x7fcdbc00c3a0 comp rx=0 tx=0).stop 2024-05-21T21:57:31.089 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.088+0000 7fcdc9a00700 1 -- 172.21.15.151:0/3368127504 shutdown_connections 2024-05-21T21:57:31.089 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.088+0000 7fcdc9a00700 1 --2- 172.21.15.151:0/3368127504 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fcda80b2a70 0x7fcda80b4f30 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:31.089 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.088+0000 7fcdc9a00700 1 --2- 172.21.15.151:0/3368127504 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcdc4103490 0x7fcdc419f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:31.090 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.088+0000 7fcdc9a00700 1 --2- 172.21.15.151:0/3368127504 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcdc4109540 0x7fcdc4199970 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:31.090 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.088+0000 7fcdc9a00700 1 --2- 172.21.15.151:0/3368127504 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcdc4103e50 0x7fcdc419fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:31.090 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.088+0000 7fcdc9a00700 1 -- 172.21.15.151:0/3368127504 >> 172.21.15.151:0/3368127504 conn(0x7fcdc40fefc0 msgr2=0x7fcdc4100820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:57:31.090 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.088+0000 7fcdc9a00700 1 -- 172.21.15.151:0/3368127504 shutdown_connections 2024-05-21T21:57:31.090 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:31.088+0000 7fcdc9a00700 1 -- 172.21.15.151:0/3368127504 wait complete. 2024-05-21T21:57:31.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:57:31 smithi151 ceph-mon[31623]: pgmap v60: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-05-21T21:57:31.735 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:57:31 smithi184 ceph-mon[36942]: pgmap v60: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-05-21T21:57:31.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:57:31 smithi164 ceph-mon[36269]: pgmap v60: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-05-21T21:57:31.843 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-21T21:57:31.857 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-21T21:57:32.424 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:57:32.502 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:57:33.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:57:33 smithi151 ceph-mon[31623]: pgmap v61: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2024-05-21T21:57:33.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:57:33 smithi184 ceph-mon[36942]: pgmap v61: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2024-05-21T21:57:33.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:57:33 smithi164 ceph-mon[36269]: pgmap v61: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2024-05-21T21:57:35.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:57:35 smithi151 ceph-mon[31623]: pgmap v62: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-05-21T21:57:35.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:57:35 smithi184 ceph-mon[36942]: pgmap v62: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-05-21T21:57:35.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:57:35 smithi164 ceph-mon[36269]: pgmap v62: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-05-21T21:57:37.690 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:57:37 smithi164 ceph-mon[36269]: pgmap v63: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-05-21T21:57:37.702 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:57:37 smithi151 ceph-mon[31623]: pgmap v63: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-05-21T21:57:37.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:57:37 smithi184 ceph-mon[36942]: pgmap v63: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-05-21T21:57:38.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.062+0000 7fd739200700 1 -- 172.21.15.151:0/2002011189 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd734076200 msgr2=0x7fd734113f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:38.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.062+0000 7fd739200700 1 --2- 172.21.15.151:0/2002011189 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd734076200 0x7fd734113f40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd720009990 tx=0x7fd720009ca0 comp rx=0 tx=0).stop 2024-05-21T21:57:38.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.062+0000 7fd739200700 1 -- 172.21.15.151:0/2002011189 shutdown_connections 2024-05-21T21:57:38.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.062+0000 7fd739200700 1 --2- 172.21.15.151:0/2002011189 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd734114480 0x7fd734116870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:38.064 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.062+0000 7fd739200700 1 --2- 172.21.15.151:0/2002011189 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd7340758e0 0x7fd734075cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:38.064 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.062+0000 7fd739200700 1 --2- 172.21.15.151:0/2002011189 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd734076200 0x7fd734113f40 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:38.064 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.062+0000 7fd739200700 1 -- 172.21.15.151:0/2002011189 >> 172.21.15.151:0/2002011189 conn(0x7fd7340fefc0 msgr2=0x7fd7341013e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:57:38.064 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.062+0000 7fd739200700 1 -- 172.21.15.151:0/2002011189 shutdown_connections 2024-05-21T21:57:38.064 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.063+0000 7fd739200700 1 -- 172.21.15.151:0/2002011189 wait complete. 2024-05-21T21:57:38.065 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.064+0000 7fd739200700 1 Processor -- start 2024-05-21T21:57:38.065 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.064+0000 7fd739200700 1 -- start start 2024-05-21T21:57:38.065 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.065+0000 7fd739200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd7340758e0 0x7fd734075060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:38.065 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.065+0000 7fd739200700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd734076200 0x7fd73406d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:38.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.065+0000 7fd739200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd734114480 0x7fd73406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:38.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.066+0000 7fd739200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd734119cc0 con 0x7fd734114480 2024-05-21T21:57:38.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.066+0000 7fd739200700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd734119b40 con 0x7fd7340758e0 2024-05-21T21:57:38.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.066+0000 7fd739200700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd734119e40 con 0x7fd734076200 2024-05-21T21:57:38.067 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.066+0000 7fd733400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd7340758e0 0x7fd734075060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:38.067 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.066+0000 7fd732a00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd734076200 0x7fd73406d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:38.067 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.066+0000 7fd733e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd734114480 0x7fd73406d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:38.067 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.066+0000 7fd732a00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd734076200 0x7fd73406d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:39038/0 (socket says 172.21.15.151:39038) 2024-05-21T21:57:38.067 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.066+0000 7fd732a00700 1 -- 172.21.15.151:0/2508962559 learned_addr learned my addr 172.21.15.151:0/2508962559 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:57:38.067 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.066+0000 7fd733e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd734114480 0x7fd73406d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:55746/0 (socket says 172.21.15.151:55746) 2024-05-21T21:57:38.067 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.067+0000 7fd732a00700 1 -- 172.21.15.151:0/2508962559 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd7340758e0 msgr2=0x7fd734075060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:38.068 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.067+0000 7fd732a00700 1 --2- 172.21.15.151:0/2508962559 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd7340758e0 0x7fd734075060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:38.068 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.067+0000 7fd732a00700 1 -- 172.21.15.151:0/2508962559 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd734114480 msgr2=0x7fd73406d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:38.068 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.067+0000 7fd732a00700 1 --2- 172.21.15.151:0/2508962559 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd734114480 0x7fd73406d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:38.068 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.067+0000 7fd732a00700 1 -- 172.21.15.151:0/2508962559 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd73406dde0 con 0x7fd734076200 2024-05-21T21:57:38.068 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.067+0000 7fd733400700 1 --2- 172.21.15.151:0/2508962559 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd7340758e0 0x7fd734075060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:57:38.068 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.068+0000 7fd732a00700 1 --2- 172.21.15.151:0/2508962559 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd734076200 0x7fd73406d030 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd720006010 tx=0x7fd7200057c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:57:38.068 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.068+0000 7fd72be00700 1 -- 172.21.15.151:0/2508962559 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd720027070 con 0x7fd734076200 2024-05-21T21:57:38.069 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.068+0000 7fd739200700 1 -- 172.21.15.151:0/2508962559 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7341b4f60 con 0x7fd734076200 2024-05-21T21:57:38.070 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.068+0000 7fd739200700 1 -- 172.21.15.151:0/2508962559 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7341b53c0 con 0x7fd734076200 2024-05-21T21:57:38.070 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.069+0000 7fd72be00700 1 -- 172.21.15.151:0/2508962559 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd720005da0 con 0x7fd734076200 2024-05-21T21:57:38.070 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.069+0000 7fd72be00700 1 -- 172.21.15.151:0/2508962559 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd720027070 con 0x7fd734076200 2024-05-21T21:57:38.073 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.073+0000 7fd745246b80 1 Processor -- start 2024-05-21T21:57:38.074 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.074+0000 7fd745246b80 1 -- start start 2024-05-21T21:57:38.074 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.074+0000 7fd745246b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560a64e25020 0x560a64e25400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:38.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.074+0000 7fd745246b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560a64e25940 0x560a64e1f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:38.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.074+0000 7fd745246b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560a64e1fad0 0x560a64e1ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:38.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.074+0000 7fd745246b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x560a64d7de40 con 0x560a64e1fad0 2024-05-21T21:57:38.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.074+0000 7fd745246b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x560a64d6d9e0 con 0x560a64e25020 2024-05-21T21:57:38.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.074+0000 7fd745246b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x560a64d71a20 con 0x560a64e25940 2024-05-21T21:57:38.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.074+0000 7fd72be00700 1 -- 172.21.15.151:0/2508962559 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7fd720009d10 con 0x7fd734076200 2024-05-21T21:57:38.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.075+0000 7fd733400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560a64e25940 0x560a64e1f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:38.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.075+0000 7fd733400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560a64e25940 0x560a64e1f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:39052/0 (socket says 172.21.15.151:39052) 2024-05-21T21:57:38.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.075+0000 7fd733400700 1 -- 172.21.15.151:0/296094977 learned_addr learned my addr 172.21.15.151:0/296094977 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:57:38.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.075+0000 7fd733e00700 1 --2- 172.21.15.151:0/296094977 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560a64e25020 0x560a64e25400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:38.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.075+0000 7fd733400700 1 -- 172.21.15.151:0/296094977 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560a64e25020 msgr2=0x560a64e25400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:38.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.075+0000 7fd733400700 1 --2- 172.21.15.151:0/296094977 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560a64e25020 0x560a64e25400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:38.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.075+0000 7fd733400700 1 -- 172.21.15.151:0/296094977 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560a64e1fad0 msgr2=0x560a64e1ff30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-21T21:57:38.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.075+0000 7fd733400700 1 --2- 172.21.15.151:0/296094977 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560a64e1fad0 0x560a64e1ff30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:38.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.075+0000 7fd733400700 1 -- 172.21.15.151:0/296094977 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a64d36420 con 0x560a64e25940 2024-05-21T21:57:38.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.075+0000 7fd733400700 1 --2- 172.21.15.151:0/296094977 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560a64e25940 0x560a64e1f4b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd7240115c0 tx=0x7fd7240118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8af4f72f854f1c4f server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:57:38.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.076+0000 7fd728c00700 1 -- 172.21.15.151:0/296094977 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd72401e070 con 0x560a64e25940 2024-05-21T21:57:38.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.076+0000 7fd728c00700 1 -- 172.21.15.151:0/296094977 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7240198c0 con 0x560a64e25940 2024-05-21T21:57:38.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.076+0000 7fd728c00700 1 -- 172.21.15.151:0/296094977 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd724018970 con 0x560a64e25940 2024-05-21T21:57:38.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.077+0000 7fd745246b80 1 -- 172.21.15.151:0/296094977 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560a64e25940 msgr2=0x560a64e1f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:38.078 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.077+0000 7fd745246b80 1 --2- 172.21.15.151:0/296094977 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560a64e25940 0x560a64e1f4b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd7240115c0 tx=0x7fd7240118d0 comp rx=0 tx=0).stop 2024-05-21T21:57:38.078 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.077+0000 7fd745246b80 1 -- 172.21.15.151:0/296094977 shutdown_connections 2024-05-21T21:57:38.078 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.077+0000 7fd745246b80 1 --2- 172.21.15.151:0/296094977 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560a64e25020 0x560a64e25400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:38.078 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.077+0000 7fd745246b80 1 --2- 172.21.15.151:0/296094977 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560a64e1fad0 0x560a64e1ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:38.078 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.077+0000 7fd745246b80 1 --2- 172.21.15.151:0/296094977 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560a64e25940 0x560a64e1f4b0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:38.078 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.077+0000 7fd745246b80 1 -- 172.21.15.151:0/296094977 >> 172.21.15.151:0/296094977 conn(0x560a64d4dbe0 msgr2=0x560a64e1a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:57:38.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.077+0000 7fd745246b80 1 -- 172.21.15.151:0/296094977 shutdown_connections 2024-05-21T21:57:38.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.077+0000 7fd745246b80 1 -- 172.21.15.151:0/296094977 wait complete. 2024-05-21T21:57:38.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.078+0000 7fd72be00700 1 --2- 172.21.15.151:0/2508962559 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fd7180b2d00 0x7fd7180b51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:38.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.078+0000 7fd745246b80 1 -- start start 2024-05-21T21:57:38.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.078+0000 7fd745246b80 1 Processor -- start 2024-05-21T21:57:38.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.079+0000 7fd72be00700 1 -- 172.21.15.151:0/2508962559 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fd72001e030 con 0x7fd734076200 2024-05-21T21:57:38.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.079+0000 7fd733e00700 1 --2- 172.21.15.151:0/2508962559 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fd7180b2d00 0x7fd7180b51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:38.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.080+0000 7fd733e00700 1 --2- 172.21.15.151:0/2508962559 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fd7180b2d00 0x7fd7180b51c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd73406be50 tx=0x7fd72c00b040 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:57:38.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.081+0000 7fd745246b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560a64e1fad0 0x560a64e2b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:38.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.081+0000 7fd745246b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560a64e25020 0x560a64e28a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:38.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.081+0000 7fd745246b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560a64e25940 0x560a64e28f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:38.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.081+0000 7fd745246b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x560a64c73fa0 con 0x560a64e1fad0 2024-05-21T21:57:38.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.081+0000 7fd745246b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x560a64d0e760 con 0x560a64e25020 2024-05-21T21:57:38.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.081+0000 7fd745246b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x560a64e2c460 con 0x560a64e25940 2024-05-21T21:57:38.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.081+0000 7fd745246b80 10 client.0 _ll_register_callbacks cb 0x560a64d17c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:57:38.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.082+0000 7fd733400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560a64e1fad0 0x560a64e2b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:38.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.082+0000 7fd732a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560a64e25020 0x560a64e28a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:38.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.082+0000 7fd733400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560a64e1fad0 0x560a64e2b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:55752/0 (socket says 172.21.15.151:55752) 2024-05-21T21:57:38.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.082+0000 7fd733400700 1 -- 172.21.15.151:0/3593914323 learned_addr learned my addr 172.21.15.151:0/3593914323 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:57:38.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.082+0000 7fd733e00700 1 --2- 172.21.15.151:0/3593914323 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560a64e25940 0x560a64e28f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:38.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.082+0000 7fd733400700 1 -- 172.21.15.151:0/3593914323 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560a64e25940 msgr2=0x560a64e28f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:38.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.082+0000 7fd733400700 1 --2- 172.21.15.151:0/3593914323 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560a64e25940 0x560a64e28f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:38.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.082+0000 7fd733400700 1 -- 172.21.15.151:0/3593914323 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560a64e25020 msgr2=0x560a64e28a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:38.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.083+0000 7fd733400700 1 --2- 172.21.15.151:0/3593914323 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560a64e25020 0x560a64e28a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:38.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.083+0000 7fd733400700 1 -- 172.21.15.151:0/3593914323 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a64cbd8d0 con 0x560a64e1fad0 2024-05-21T21:57:38.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.083+0000 7fd733e00700 1 --2- 172.21.15.151:0/3593914323 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560a64e25940 0x560a64e28f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:57:38.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.083+0000 7fd732a00700 1 --2- 172.21.15.151:0/3593914323 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560a64e25020 0x560a64e28a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:57:38.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.083+0000 7fd733400700 1 --2- 172.21.15.151:0/3593914323 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560a64e1fad0 0x560a64e2b690 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fd724005920 tx=0x7fd72401acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcb24ac587d02075 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:57:38.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.083+0000 7fd71d600700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-21T21:57:38.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.083+0000 7fd71d600700 1 -- 172.21.15.151:0/3593914323 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd724027060 con 0x560a64e1fad0 2024-05-21T21:57:38.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.083+0000 7fd71d600700 1 -- 172.21.15.151:0/3593914323 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fd724005030 con 0x560a64e1fad0 2024-05-21T21:57:38.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.084+0000 7fd745246b80 1 -- 172.21.15.151:0/3593914323 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7240198c0 con 0x560a64e1fad0 2024-05-21T21:57:38.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.084+0000 7fd71d600700 1 -- 172.21.15.151:0/3593914323 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd72401e040 con 0x560a64e1fad0 2024-05-21T21:57:38.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.084+0000 7fd745246b80 10 client.15579 fetch_fsmap learned FSMap version 29 2024-05-21T21:57:38.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.084+0000 7fd745246b80 1 -- 172.21.15.151:0/3593914323 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560a64d3e060 con 0x560a64e1fad0 2024-05-21T21:57:38.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.084+0000 7fd71d600700 1 -- 172.21.15.151:0/3593914323 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7240198c0 con 0x560a64e1fad0 2024-05-21T21:57:38.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.085+0000 7fd71d600700 1 -- 172.21.15.151:0/3593914323 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fd724021030 con 0x560a64e1fad0 2024-05-21T21:57:38.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.085+0000 7fd745246b80 10 client.15579 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T21:57:38.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.085+0000 7fd745246b80 10 client.15579 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T21:57:38.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.085+0000 7fd745246b80 20 client.15579 populate_metadata read hostname 'smithi151' 2024-05-21T21:57:38.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.085+0000 7fd745246b80 1 --2- 172.21.15.151:0/3593914323 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x560a64e38fe0 0x560a64e3b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:38.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.085+0000 7fd745246b80 4 client.15579 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T21:57:38.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.085+0000 7fd745246b80 1 -- 172.21.15.151:0/3593914323 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560a64d7fd90 con 0x560a64e38fe0 2024-05-21T21:57:38.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.086+0000 7fd732a00700 1 --2- 172.21.15.151:0/3593914323 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x560a64e38fe0 0x560a64e3b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:38.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.086+0000 7fd732a00700 1 --2- 172.21.15.151:0/3593914323 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x560a64e38fe0 0x560a64e3b3d0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1694269cd695d932 server_cookie=46218c48707cf4fe in_seq=0 out_seq=0 2024-05-21T21:57:38.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.086+0000 7fd71d600700 10 client.15579 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T21:57:38.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.087+0000 7fd71d600700 1 -- 172.21.15.151:0/3593914323 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x560a64d7fd90 con 0x560a64e38fe0 2024-05-21T21:57:38.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.087+0000 7fd71d600700 10 client.15579 handle_command_reply: tid=0 2024-05-21T21:57:38.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.087+0000 7fd745246b80 1 client.15579 shutdown 2024-05-21T21:57:38.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.088+0000 7fd745246b80 1 -- 172.21.15.151:0/3593914323 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560a64e1fad0 msgr2=0x560a64e2b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:38.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.088+0000 7fd745246b80 1 --2- 172.21.15.151:0/3593914323 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560a64e1fad0 0x560a64e2b690 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fd724005920 tx=0x7fd72401acb0 comp rx=0 tx=0).stop 2024-05-21T21:57:38.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.088+0000 7fd745246b80 1 -- 172.21.15.151:0/3593914323 shutdown_connections 2024-05-21T21:57:38.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.088+0000 7fd745246b80 1 --2- 172.21.15.151:0/3593914323 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560a64e25940 0x560a64e28f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:38.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.088+0000 7fd745246b80 1 --2- 172.21.15.151:0/3593914323 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560a64e1fad0 0x560a64e2b690 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:38.089 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.088+0000 7fd745246b80 1 --2- 172.21.15.151:0/3593914323 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x560a64e38fe0 0x560a64e3b3d0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:38.089 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.088+0000 7fd745246b80 1 --2- 172.21.15.151:0/3593914323 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560a64e25020 0x560a64e28a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:38.089 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.088+0000 7fd745246b80 1 -- 172.21.15.151:0/3593914323 >> 172.21.15.151:0/3593914323 conn(0x560a64d4dbe0 msgr2=0x560a64e1e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:57:38.089 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.088+0000 7fd71d600700 0 client.15579 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T21:57:38.089 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.088+0000 7fd745246b80 1 -- 172.21.15.151:0/3593914323 shutdown_connections 2024-05-21T21:57:38.089 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.089+0000 7fd745246b80 1 -- 172.21.15.151:0/3593914323 wait complete. 2024-05-21T21:57:38.089 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.089+0000 7fd745246b80 20 client.15579 trim_cache size 0 max 16384 2024-05-21T21:57:38.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.107+0000 7fd745246b80 1 Processor -- start 2024-05-21T21:57:38.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.107+0000 7fd745246b80 1 -- start start 2024-05-21T21:57:38.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.107+0000 7fd745246b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560a64e1fad0 0x560a64e2aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:38.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.107+0000 7fd745246b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560a64e25020 0x560a64e1feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:38.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.107+0000 7fd745246b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560a64e25940 0x560a64e289f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:38.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.107+0000 7fd745246b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd71400d580 con 0x560a64e1fad0 2024-05-21T21:57:38.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.107+0000 7fd745246b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd71400f0c0 con 0x560a64e25940 2024-05-21T21:57:38.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.107+0000 7fd745246b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd71400dc50 con 0x560a64e25020 2024-05-21T21:57:38.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.108+0000 7fd733400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560a64e1fad0 0x560a64e2aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:38.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.108+0000 7fd733400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560a64e1fad0 0x560a64e2aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:55754/0 (socket says 172.21.15.151:55754) 2024-05-21T21:57:38.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.108+0000 7fd732a00700 1 --2- 172.21.15.151:0/844573550 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560a64e25020 0x560a64e1feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:38.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.108+0000 7fd733400700 1 -- 172.21.15.151:0/844573550 learned_addr learned my addr 172.21.15.151:0/844573550 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:57:38.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.108+0000 7fd733e00700 1 --2- 172.21.15.151:0/844573550 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560a64e25940 0x560a64e289f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:38.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.108+0000 7fd732a00700 1 -- 172.21.15.151:0/844573550 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560a64e25940 msgr2=0x560a64e289f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:38.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.108+0000 7fd732a00700 1 --2- 172.21.15.151:0/844573550 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560a64e25940 0x560a64e289f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:38.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.108+0000 7fd732a00700 1 -- 172.21.15.151:0/844573550 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560a64e1fad0 msgr2=0x560a64e2aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:38.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.108+0000 7fd732a00700 1 --2- 172.21.15.151:0/844573550 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560a64e1fad0 0x560a64e2aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:38.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.108+0000 7fd732a00700 1 -- 172.21.15.151:0/844573550 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a64e2be90 con 0x560a64e25020 2024-05-21T21:57:38.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.109+0000 7fd733e00700 1 --2- 172.21.15.151:0/844573550 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560a64e25940 0x560a64e289f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T21:57:38.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.109+0000 7fd732a00700 1 --2- 172.21.15.151:0/844573550 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560a64e25020 0x560a64e1feb0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd7200dfe50 tx=0x7fd72000b610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df914a807922c075 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:57:38.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.109+0000 7fd71fe00700 1 -- 172.21.15.151:0/844573550 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd720027040 con 0x560a64e25020 2024-05-21T21:57:38.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.109+0000 7fd71fe00700 1 -- 172.21.15.151:0/844573550 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7200b2180 con 0x560a64e25020 2024-05-21T21:57:38.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.109+0000 7fd71fe00700 1 -- 172.21.15.151:0/844573550 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7200cc0c0 con 0x560a64e25020 2024-05-21T21:57:38.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.110+0000 7fd745246b80 1 -- 172.21.15.151:0/844573550 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560a64e25020 msgr2=0x560a64e1feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:38.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.110+0000 7fd745246b80 1 --2- 172.21.15.151:0/844573550 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560a64e25020 0x560a64e1feb0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd7200dfe50 tx=0x7fd72000b610 comp rx=0 tx=0).stop 2024-05-21T21:57:38.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.110+0000 7fd745246b80 1 -- 172.21.15.151:0/844573550 shutdown_connections 2024-05-21T21:57:38.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.110+0000 7fd745246b80 1 --2- 172.21.15.151:0/844573550 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560a64e25940 0x560a64e289f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:38.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.110+0000 7fd745246b80 1 --2- 172.21.15.151:0/844573550 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560a64e1fad0 0x560a64e2aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:38.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.110+0000 7fd745246b80 1 --2- 172.21.15.151:0/844573550 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560a64e25020 0x560a64e1feb0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:38.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.110+0000 7fd745246b80 1 -- 172.21.15.151:0/844573550 >> 172.21.15.151:0/844573550 conn(0x560a64d4dbe0 msgr2=0x560a64e22b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:57:38.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.111+0000 7fd745246b80 1 -- 172.21.15.151:0/844573550 shutdown_connections 2024-05-21T21:57:38.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.111+0000 7fd745246b80 1 -- 172.21.15.151:0/844573550 wait complete. 2024-05-21T21:57:38.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.111+0000 7fd745246b80 1 -- start start 2024-05-21T21:57:38.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.111+0000 7fd745246b80 1 Processor -- start 2024-05-21T21:57:38.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.112+0000 7fd745246b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560a64e1fad0 0x560a64e31780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:38.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.112+0000 7fd745246b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560a64e25020 0x560a64e31cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:38.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.112+0000 7fd745246b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560a64e25940 0x560a64e3fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:38.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.112+0000 7fd745246b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd71400e9f0 con 0x560a64e25020 2024-05-21T21:57:38.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.112+0000 7fd745246b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd71400e320 con 0x560a64e25940 2024-05-21T21:57:38.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.112+0000 7fd745246b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd71400b520 con 0x560a64e1fad0 2024-05-21T21:57:38.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.112+0000 7fd745246b80 10 client.0 _ll_register_callbacks cb 0x560a64d17c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:57:38.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.112+0000 7fd733400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560a64e1fad0 0x560a64e31780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:38.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.112+0000 7fd733400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560a64e1fad0 0x560a64e31780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:39070/0 (socket says 172.21.15.151:39070) 2024-05-21T21:57:38.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.113+0000 7fd732a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560a64e25020 0x560a64e31cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:38.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.113+0000 7fd733e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560a64e25940 0x560a64e3fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:38.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.113+0000 7fd733e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560a64e25940 0x560a64e3fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:47064/0 (socket says 172.21.15.151:47064) 2024-05-21T21:57:38.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.113+0000 7fd733e00700 1 -- 172.21.15.151:0/3926212459 learned_addr learned my addr 172.21.15.151:0/3926212459 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:57:38.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.113+0000 7fd733400700 1 -- 172.21.15.151:0/3926212459 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560a64e25940 msgr2=0x560a64e3fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:38.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.113+0000 7fd733400700 1 --2- 172.21.15.151:0/3926212459 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560a64e25940 0x560a64e3fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:38.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.113+0000 7fd733400700 1 -- 172.21.15.151:0/3926212459 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560a64e25020 msgr2=0x560a64e31cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:38.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.113+0000 7fd733400700 1 --2- 172.21.15.151:0/3926212459 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560a64e25020 0x560a64e31cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:38.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.113+0000 7fd733400700 1 -- 172.21.15.151:0/3926212459 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a64e3ffc0 con 0x560a64e1fad0 2024-05-21T21:57:38.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.114+0000 7fd733400700 1 --2- 172.21.15.151:0/3926212459 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560a64e1fad0 0x560a64e31780 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd724011d50 tx=0x7fd724004d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=198756b6a27f11bd server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:57:38.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.114+0000 7fd71d600700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-21T21:57:38.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.114+0000 7fd71d600700 1 -- 172.21.15.151:0/3926212459 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd724027040 con 0x560a64e1fad0 2024-05-21T21:57:38.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.114+0000 7fd71d600700 1 -- 172.21.15.151:0/3926212459 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fd724005410 con 0x560a64e1fad0 2024-05-21T21:57:38.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.114+0000 7fd745246b80 1 -- 172.21.15.151:0/3926212459 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560a64d79570 con 0x560a64e1fad0 2024-05-21T21:57:38.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.115+0000 7fd71d600700 1 -- 172.21.15.151:0/3926212459 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd724005ce0 con 0x560a64e1fad0 2024-05-21T21:57:38.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.115+0000 7fd745246b80 10 client.25732 fetch_fsmap learned FSMap version 29 2024-05-21T21:57:38.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.115+0000 7fd745246b80 1 -- 172.21.15.151:0/3926212459 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560a64ea30a0 con 0x560a64e1fad0 2024-05-21T21:57:38.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.115+0000 7fd71d600700 1 -- 172.21.15.151:0/3926212459 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x560a64d79570 con 0x560a64e1fad0 2024-05-21T21:57:38.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.116+0000 7fd71d600700 1 -- 172.21.15.151:0/3926212459 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fd724021040 con 0x560a64e1fad0 2024-05-21T21:57:38.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.116+0000 7fd745246b80 10 client.25732 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T21:57:38.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.116+0000 7fd745246b80 10 client.25732 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T21:57:38.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.116+0000 7fd745246b80 20 client.25732 populate_metadata read hostname 'smithi151' 2024-05-21T21:57:38.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.116+0000 7fd745246b80 1 --2- 172.21.15.151:0/3926212459 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x560a64ea3760 0x560a64ea5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:38.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.116+0000 7fd745246b80 4 client.25732 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T21:57:38.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.116+0000 7fd745246b80 1 -- 172.21.15.151:0/3926212459 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560a64bf0250 con 0x560a64ea3760 2024-05-21T21:57:38.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.116+0000 7fd732a00700 1 --2- 172.21.15.151:0/3926212459 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x560a64ea3760 0x560a64ea5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:38.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.117+0000 7fd732a00700 1 --2- 172.21.15.151:0/3926212459 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x560a64ea3760 0x560a64ea5b80 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d0ca8867fe7427b server_cookie=6af14505e011bbad in_seq=0 out_seq=0 2024-05-21T21:57:38.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.117+0000 7fd71d600700 10 client.25732 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T21:57:38.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.117+0000 7fd71d600700 1 -- 172.21.15.151:0/3926212459 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560a64bf0250 con 0x560a64ea3760 2024-05-21T21:57:38.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.117+0000 7fd71d600700 10 client.25732 handle_command_reply: tid=0 2024-05-21T21:57:38.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.117+0000 7fd745246b80 1 client.25732 shutdown 2024-05-21T21:57:38.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.118+0000 7fd745246b80 1 -- 172.21.15.151:0/3926212459 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560a64e1fad0 msgr2=0x560a64e31780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:38.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.118+0000 7fd745246b80 1 --2- 172.21.15.151:0/3926212459 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560a64e1fad0 0x560a64e31780 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd724011d50 tx=0x7fd724004d10 comp rx=0 tx=0).stop 2024-05-21T21:57:38.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.118+0000 7fd745246b80 1 -- 172.21.15.151:0/3926212459 shutdown_connections 2024-05-21T21:57:38.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.118+0000 7fd745246b80 1 --2- 172.21.15.151:0/3926212459 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x560a64ea3760 0x560a64ea5b80 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:38.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.118+0000 7fd745246b80 1 --2- 172.21.15.151:0/3926212459 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560a64e25940 0x560a64e3fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:38.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.118+0000 7fd745246b80 1 --2- 172.21.15.151:0/3926212459 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560a64e1fad0 0x560a64e31780 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:38.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.118+0000 7fd745246b80 1 --2- 172.21.15.151:0/3926212459 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560a64e25020 0x560a64e31cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:38.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.118+0000 7fd745246b80 1 -- 172.21.15.151:0/3926212459 >> 172.21.15.151:0/3926212459 conn(0x560a64d4dbe0 msgr2=0x560a64e289d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:57:38.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.118+0000 7fd71d600700 0 client.25732 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T21:57:38.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.118+0000 7fd745246b80 1 -- 172.21.15.151:0/3926212459 shutdown_connections 2024-05-21T21:57:38.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.118+0000 7fd745246b80 1 -- 172.21.15.151:0/3926212459 wait complete. 2024-05-21T21:57:38.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.119+0000 7fd745246b80 20 client.25732 trim_cache size 0 max 16384 2024-05-21T21:57:38.120 INFO:teuthology.orchestra.run.smithi151.stdout:[] 2024-05-21T21:57:38.121 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.119+0000 7fd739200700 1 -- 172.21.15.151:0/2508962559 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fd7180b2d00 msgr2=0x7fd7180b51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:38.121 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.119+0000 7fd739200700 1 --2- 172.21.15.151:0/2508962559 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fd7180b2d00 0x7fd7180b51c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd73406be50 tx=0x7fd72c00b040 comp rx=0 tx=0).stop 2024-05-21T21:57:38.121 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.119+0000 7fd739200700 1 -- 172.21.15.151:0/2508962559 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd734076200 msgr2=0x7fd73406d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:38.121 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.119+0000 7fd739200700 1 --2- 172.21.15.151:0/2508962559 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd734076200 0x7fd73406d030 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd720006010 tx=0x7fd7200057c0 comp rx=0 tx=0).stop 2024-05-21T21:57:38.121 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.120+0000 7fd739200700 1 -- 172.21.15.151:0/2508962559 shutdown_connections 2024-05-21T21:57:38.121 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.120+0000 7fd739200700 1 --2- 172.21.15.151:0/2508962559 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fd7180b2d00 0x7fd7180b51c0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:38.121 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.120+0000 7fd739200700 1 --2- 172.21.15.151:0/2508962559 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd7340758e0 0x7fd734075060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:38.121 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.120+0000 7fd739200700 1 --2- 172.21.15.151:0/2508962559 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd734114480 0x7fd73406d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:38.122 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.120+0000 7fd739200700 1 --2- 172.21.15.151:0/2508962559 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd734076200 0x7fd73406d030 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:38.122 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.120+0000 7fd739200700 1 -- 172.21.15.151:0/2508962559 >> 172.21.15.151:0/2508962559 conn(0x7fd7340fefc0 msgr2=0x7fd734100820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:57:38.122 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.120+0000 7fd739200700 1 -- 172.21.15.151:0/2508962559 shutdown_connections 2024-05-21T21:57:38.122 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:38.120+0000 7fd739200700 1 -- 172.21.15.151:0/2508962559 wait complete. 2024-05-21T21:57:39.415 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:57:39 smithi151 ceph-mon[31623]: pgmap v64: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2024-05-21T21:57:39.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:57:39 smithi184 ceph-mon[36942]: pgmap v64: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2024-05-21T21:57:39.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:57:39 smithi164 ceph-mon[36269]: pgmap v64: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2024-05-21T21:57:40.132 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-21T21:57:40.133 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-21T21:57:40.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:40.218+0000 7f57b1800700 1 -- 172.21.15.151:0/2176777462 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57ac10ff40 msgr2=0x7f57ac112330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:40.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:40.218+0000 7f57b1800700 1 --2- 172.21.15.151:0/2176777462 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57ac10ff40 0x7f57ac112330 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f57a4009f90 tx=0x7f57a400b330 comp rx=0 tx=0).stop 2024-05-21T21:57:40.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:40.218+0000 7f57b1800700 1 -- 172.21.15.151:0/2176777462 shutdown_connections 2024-05-21T21:57:40.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:40.218+0000 7f57b1800700 1 --2- 172.21.15.151:0/2176777462 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57ac10ff40 0x7f57ac112330 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:40.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:40.218+0000 7f57b1800700 1 --2- 172.21.15.151:0/2176777462 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f57ac075770 0x7f57ac075b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:40.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:40.218+0000 7f57b1800700 1 --2- 172.21.15.151:0/2176777462 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f57ac076090 0x7f57ac10fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:40.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:40.218+0000 7f57b1800700 1 -- 172.21.15.151:0/2176777462 >> 172.21.15.151:0/2176777462 conn(0x7f57ac07a6a0 msgr2=0x7f57ac07aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:57:40.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:40.218+0000 7f57b1800700 1 -- 172.21.15.151:0/2176777462 shutdown_connections 2024-05-21T21:57:40.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:40.218+0000 7f57b1800700 1 -- 172.21.15.151:0/2176777462 wait complete. 2024-05-21T21:57:40.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:40.221+0000 7f57b1800700 1 Processor -- start 2024-05-21T21:57:40.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:40.221+0000 7f57b1800700 1 -- start start 2024-05-21T21:57:40.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:40.221+0000 7f57b1800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57ac075770 0x7f57ac19bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:40.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:40.221+0000 7f57b1800700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f57ac076090 0x7f57ac19c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:40.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:40.222+0000 7f57b1800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f57ac10ff40 0x7f57ac1a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:40.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:40.222+0000 7f57b1800700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f57ac115940 con 0x7f57ac10ff40 2024-05-21T21:57:40.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:40.222+0000 7f57b1800700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f57ac1157c0 con 0x7f57ac075770 2024-05-21T21:57:40.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:40.222+0000 7f57b1800700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f57ac115ac0 con 0x7f57ac076090 2024-05-21T21:57:40.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:40.222+0000 7f57aaa00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57ac075770 0x7f57ac19bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:40.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:40.222+0000 7f57aaa00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57ac075770 0x7f57ac19bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:47074/0 (socket says 172.21.15.151:47074) 2024-05-21T21:57:40.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:40.222+0000 7f57aaa00700 1 -- 172.21.15.151:0/21874388 learned_addr learned my addr 172.21.15.151:0/21874388 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:57:40.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:40.222+0000 7f57aa000700 1 --2- 172.21.15.151:0/21874388 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f57ac076090 0x7f57ac19c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:40.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:40.222+0000 7f57ab400700 1 --2- 172.21.15.151:0/21874388 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f57ac10ff40 0x7f57ac1a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:40.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:40.223+0000 7f57aaa00700 1 -- 172.21.15.151:0/21874388 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f57ac076090 msgr2=0x7f57ac19c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:40.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:40.223+0000 7f57aaa00700 1 --2- 172.21.15.151:0/21874388 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f57ac076090 0x7f57ac19c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:40.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:40.223+0000 7f57aaa00700 1 -- 172.21.15.151:0/21874388 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f57ac10ff40 msgr2=0x7f57ac1a06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:40.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:40.223+0000 7f57aaa00700 1 --2- 172.21.15.151:0/21874388 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f57ac10ff40 0x7f57ac1a06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:40.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:40.223+0000 7f57aaa00700 1 -- 172.21.15.151:0/21874388 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57ac1a0de0 con 0x7f57ac075770 2024-05-21T21:57:40.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:40.223+0000 7f57aa000700 1 --2- 172.21.15.151:0/21874388 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f57ac076090 0x7f57ac19c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:57:40.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:40.223+0000 7f57aaa00700 1 --2- 172.21.15.151:0/21874388 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57ac075770 0x7f57ac19bdd0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f579c00e9c0 tx=0x7f579c00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:57:40.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:40.224+0000 7f57a3400700 1 -- 172.21.15.151:0/21874388 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f579c004a30 con 0x7f57ac075770 2024-05-21T21:57:40.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:40.224+0000 7f57a3400700 1 -- 172.21.15.151:0/21874388 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f579c00cc70 con 0x7f57ac075770 2024-05-21T21:57:40.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:40.224+0000 7f57b1800700 1 -- 172.21.15.151:0/21874388 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57ac1a10d0 con 0x7f57ac075770 2024-05-21T21:57:40.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:40.224+0000 7f57a3400700 1 -- 172.21.15.151:0/21874388 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f579c005490 con 0x7f57ac075770 2024-05-21T21:57:40.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:40.224+0000 7f57b1800700 1 -- 172.21.15.151:0/21874388 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57ac103590 con 0x7f57ac075770 2024-05-21T21:57:40.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:40.226+0000 7f57b1800700 1 -- 172.21.15.151:0/21874388 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57ac06a230 con 0x7f57ac075770 2024-05-21T21:57:40.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:40.229+0000 7f57a3400700 1 -- 172.21.15.151:0/21874388 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7f579c004bd0 con 0x7f57ac075770 2024-05-21T21:57:40.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:40.232+0000 7f57a3400700 1 --2- 172.21.15.151:0/21874388 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f57900b2af0 0x7f57900b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:40.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:40.233+0000 7f57a3400700 1 -- 172.21.15.151:0/21874388 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f579c0e26f0 con 0x7f57ac075770 2024-05-21T21:57:40.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:40.234+0000 7f57aa000700 1 --2- 172.21.15.151:0/21874388 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f57900b2af0 0x7f57900b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:40.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:40.236+0000 7f57a3400700 1 -- 172.21.15.151:0/21874388 <== mon.2 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f579c0aa080 con 0x7f57ac075770 2024-05-21T21:57:40.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:40.237+0000 7f57aa000700 1 --2- 172.21.15.151:0/21874388 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f57900b2af0 0x7f57900b4fb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f57ac111b40 tx=0x7f5798009260 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:57:40.710 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:40.710+0000 7f57b1800700 1 -- 172.21.15.151:0/21874388 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f57ac19d130 con 0x7f57ac075770 2024-05-21T21:57:40.713 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:40.711+0000 7f57a3400700 1 -- 172.21.15.151:0/21874388 <== mon.2 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11958 (secure 0 0 0) 0x7f579c0a9980 con 0x7f57ac075770 2024-05-21T21:57:40.713 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T21:57:40.714 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14358,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/158010529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":158010529},{"type":"v1","addr":"172.21.15.151:6835","nonce":158010529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6839/2069058115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":2069058115},{"type":"v1","addr":"172.21.15.151:6839","nonce":2069058115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T21:51:19.552069+0000","modified":"2024-05-21T21:55:34.643755+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24530,"mds_1":24493,"mds_2":24491,"mds_3":24478,"mds_4":24473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14403":{"gid":14403,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6841/1749723794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6840","nonce":1749723794},{"type":"v1","addr":"172.21.15.151:6841","nonce":1749723794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/2026266114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":2026266114},{"type":"v1","addr":"172.21.15.184:6833","nonce":2026266114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/2467900844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":2467900844},{"type":"v1","addr":"172.21.15.164:6833","nonce":2467900844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3367640960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3367640960},{"type":"v1","addr":"172.21.15.151:6837","nonce":3367640960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/3264142621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3264142621},{"type":"v1","addr":"172.21.15.184:6835","nonce":3264142621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":17,"addr":"172.21.15.164:6835/2340428470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2340428470},{"type":"v1","addr":"172.21.15.164:6835","nonce":2340428470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":16,"addr":"172.21.15.184:6837/2988194888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":2988194888},{"type":"v1","addr":"172.21.15.184:6837","nonce":2988194888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.164:6837/1554609572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1554609572},{"type":"v1","addr":"172.21.15.164:6837","nonce":1554609572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.184:6839/3059935143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":3059935143},{"type":"v1","addr":"172.21.15.184:6839","nonce":3059935143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.164:6839/2646401247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":2646401247},{"type":"v1","addr":"172.21.15.164:6839","nonce":2646401247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24530,"qdb_cluster":[24530,24493,24491,24473,24478]},"id":1}]} 2024-05-21T21:57:40.714 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:40.714+0000 7f57b1800700 1 -- 172.21.15.151:0/21874388 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f57900b2af0 msgr2=0x7f57900b4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:40.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:40.714+0000 7f57b1800700 1 --2- 172.21.15.151:0/21874388 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f57900b2af0 0x7f57900b4fb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f57ac111b40 tx=0x7f5798009260 comp rx=0 tx=0).stop 2024-05-21T21:57:40.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:40.714+0000 7f57b1800700 1 -- 172.21.15.151:0/21874388 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57ac075770 msgr2=0x7f57ac19bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:40.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:40.714+0000 7f57b1800700 1 --2- 172.21.15.151:0/21874388 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57ac075770 0x7f57ac19bdd0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f579c00e9c0 tx=0x7f579c00ecd0 comp rx=0 tx=0).stop 2024-05-21T21:57:40.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:40.715+0000 7f57b1800700 1 -- 172.21.15.151:0/21874388 shutdown_connections 2024-05-21T21:57:40.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:40.715+0000 7f57b1800700 1 --2- 172.21.15.151:0/21874388 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f57900b2af0 0x7f57900b4fb0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:40.716 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:40.715+0000 7f57b1800700 1 --2- 172.21.15.151:0/21874388 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f57ac075770 0x7f57ac19bdd0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:40.716 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:40.715+0000 7f57b1800700 1 --2- 172.21.15.151:0/21874388 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f57ac10ff40 0x7f57ac1a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:40.716 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:40.715+0000 7f57b1800700 1 --2- 172.21.15.151:0/21874388 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f57ac076090 0x7f57ac19c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:40.716 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:40.715+0000 7f57b1800700 1 -- 172.21.15.151:0/21874388 >> 172.21.15.151:0/21874388 conn(0x7f57ac07a6a0 msgr2=0x7f57ac1006d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:57:40.716 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:40.716+0000 7f57b1800700 1 -- 172.21.15.151:0/21874388 shutdown_connections 2024-05-21T21:57:40.716 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:40.716+0000 7f57b1800700 1 -- 172.21.15.151:0/21874388 wait complete. 2024-05-21T21:57:40.718 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 29 2024-05-21T21:57:41.447 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:57:41.488 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24530 quiesce db --set-id f1b4324 --release --await' 2024-05-21T21:57:41.488 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24530 quiesce db --set-id f1b4324 --release --await 2024-05-21T21:57:41.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:57:41 smithi151 ceph-mon[31623]: pgmap v65: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T21:57:41.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:57:41 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/21874388' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T21:57:41.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:57:41 smithi184 ceph-mon[36942]: pgmap v65: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T21:57:41.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:57:41 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/21874388' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T21:57:41.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:57:41 smithi164 ceph-mon[36269]: pgmap v65: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T21:57:41.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:57:41 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/21874388' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T21:57:42.766 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:57:43.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:57:43 smithi151 ceph-mon[31623]: pgmap v66: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T21:57:43.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:57:43 smithi184 ceph-mon[36942]: pgmap v66: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T21:57:43.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:57:43 smithi164 ceph-mon[36269]: pgmap v66: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T21:57:44.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.858+0000 7f074e200700 1 -- 172.21.15.151:0/8238280 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0748075770 msgr2=0x7f0748075b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:44.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.858+0000 7f074e200700 1 --2- 172.21.15.151:0/8238280 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0748075770 0x7f0748075b50 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0738009990 tx=0x7f0738009ca0 comp rx=0 tx=0).stop 2024-05-21T21:57:44.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.858+0000 7f074e200700 1 -- 172.21.15.151:0/8238280 shutdown_connections 2024-05-21T21:57:44.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.858+0000 7f074e200700 1 --2- 172.21.15.151:0/8238280 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f074810ffa0 0x7f0748112390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:44.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.858+0000 7f074e200700 1 --2- 172.21.15.151:0/8238280 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0748075770 0x7f0748075b50 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:44.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.858+0000 7f074e200700 1 --2- 172.21.15.151:0/8238280 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0748076090 0x7f074810fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:44.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.858+0000 7f074e200700 1 -- 172.21.15.151:0/8238280 >> 172.21.15.151:0/8238280 conn(0x7f074807a6a0 msgr2=0x7f074807aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:57:44.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.859+0000 7f074e200700 1 -- 172.21.15.151:0/8238280 shutdown_connections 2024-05-21T21:57:44.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.859+0000 7f074e200700 1 -- 172.21.15.151:0/8238280 wait complete. 2024-05-21T21:57:44.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.861+0000 7f074e200700 1 Processor -- start 2024-05-21T21:57:44.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.861+0000 7f074e200700 1 -- start start 2024-05-21T21:57:44.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.861+0000 7f074e200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0748075770 0x7f074819be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:44.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.861+0000 7f074e200700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0748076090 0x7f074819c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:44.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.862+0000 7f074e200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f074810ffa0 0x7f07481a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:44.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.862+0000 7f074e200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0748115980 con 0x7f074810ffa0 2024-05-21T21:57:44.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.862+0000 7f074e200700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0748115800 con 0x7f0748075770 2024-05-21T21:57:44.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.862+0000 7f074e200700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0748115b00 con 0x7f0748076090 2024-05-21T21:57:44.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.862+0000 7f0746a00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0748076090 0x7f074819c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:44.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.862+0000 7f0746a00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0748076090 0x7f074819c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:39102/0 (socket says 172.21.15.151:39102) 2024-05-21T21:57:44.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.862+0000 7f0746a00700 1 -- 172.21.15.151:0/2986857080 learned_addr learned my addr 172.21.15.151:0/2986857080 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:57:44.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.862+0000 7f0747e00700 1 --2- 172.21.15.151:0/2986857080 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f074810ffa0 0x7f07481a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:44.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.862+0000 7f0747400700 1 --2- 172.21.15.151:0/2986857080 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0748075770 0x7f074819be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:44.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.863+0000 7f0746a00700 1 -- 172.21.15.151:0/2986857080 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0748075770 msgr2=0x7f074819be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:44.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.863+0000 7f0746a00700 1 --2- 172.21.15.151:0/2986857080 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0748075770 0x7f074819be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:44.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.863+0000 7f0746a00700 1 -- 172.21.15.151:0/2986857080 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f074810ffa0 msgr2=0x7f07481a06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:44.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.863+0000 7f0746a00700 1 --2- 172.21.15.151:0/2986857080 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f074810ffa0 0x7f07481a06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:44.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.863+0000 7f0746a00700 1 -- 172.21.15.151:0/2986857080 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07481a0e10 con 0x7f0748076090 2024-05-21T21:57:44.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.863+0000 7f0747e00700 1 --2- 172.21.15.151:0/2986857080 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f074810ffa0 0x7f07481a06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:57:44.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.864+0000 7f0746a00700 1 --2- 172.21.15.151:0/2986857080 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0748076090 0x7f074819c390 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f073400e9a0 tx=0x7f073400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:57:44.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.864+0000 7f073fe00700 1 -- 172.21.15.151:0/2986857080 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f073400cbd0 con 0x7f0748076090 2024-05-21T21:57:44.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.864+0000 7f073fe00700 1 -- 172.21.15.151:0/2986857080 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f073400cd70 con 0x7f0748076090 2024-05-21T21:57:44.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.864+0000 7f074e200700 1 -- 172.21.15.151:0/2986857080 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07481a1100 con 0x7f0748076090 2024-05-21T21:57:44.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.864+0000 7f073fe00700 1 -- 172.21.15.151:0/2986857080 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0734018680 con 0x7f0748076090 2024-05-21T21:57:44.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.864+0000 7f074e200700 1 -- 172.21.15.151:0/2986857080 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07481a8980 con 0x7f0748076090 2024-05-21T21:57:44.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.871+0000 7f0758ac0b80 1 Processor -- start 2024-05-21T21:57:44.178 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.871+0000 7f0758ac0b80 1 -- start start 2024-05-21T21:57:44.178 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.872+0000 7f0758ac0b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56245c603690 0x56245c603a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:44.178 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.872+0000 7f0758ac0b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56245c603fb0 0x56245c5fdba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:44.178 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.872+0000 7f0758ac0b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56245c5fe1c0 0x56245c5fe640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:44.178 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.872+0000 7f0758ac0b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56245c55bec0 con 0x56245c603fb0 2024-05-21T21:57:44.178 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.872+0000 7f0758ac0b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56245c54ba60 con 0x56245c603690 2024-05-21T21:57:44.178 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.872+0000 7f0758ac0b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56245c54faa0 con 0x56245c5fe1c0 2024-05-21T21:57:44.178 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.872+0000 7f0747e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56245c603690 0x56245c603a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:44.178 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.872+0000 7f0747e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56245c603690 0x56245c603a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:47104/0 (socket says 172.21.15.151:47104) 2024-05-21T21:57:44.178 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.872+0000 7f0747e00700 1 -- 172.21.15.151:0/1532812487 learned_addr learned my addr 172.21.15.151:0/1532812487 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:57:44.178 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.872+0000 7f0747400700 1 --2- 172.21.15.151:0/1532812487 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56245c603fb0 0x56245c5fdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:44.178 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.873+0000 7f0747e00700 1 -- 172.21.15.151:0/1532812487 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56245c5fe1c0 msgr2=0x56245c5fe640 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-21T21:57:44.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.873+0000 7f0747e00700 1 --2- 172.21.15.151:0/1532812487 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56245c5fe1c0 0x56245c5fe640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:44.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.873+0000 7f0747e00700 1 -- 172.21.15.151:0/1532812487 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56245c603fb0 msgr2=0x56245c5fdba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:44.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.873+0000 7f0747e00700 1 --2- 172.21.15.151:0/1532812487 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56245c603fb0 0x56245c5fdba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:44.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.873+0000 7f0747e00700 1 -- 172.21.15.151:0/1532812487 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56245c5144a0 con 0x56245c603690 2024-05-21T21:57:44.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.873+0000 7f073fe00700 1 -- 172.21.15.151:0/2986857080 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7f0734010430 con 0x7f0748076090 2024-05-21T21:57:44.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.873+0000 7f0747e00700 1 --2- 172.21.15.151:0/1532812487 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56245c603690 0x56245c603a70 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0740003da0 tx=0x7f0740009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d26b1f83a46155 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:57:44.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.873+0000 7f073cc00700 1 -- 172.21.15.151:0/1532812487 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0740019020 con 0x56245c603690 2024-05-21T21:57:44.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.873+0000 7f073cc00700 1 -- 172.21.15.151:0/1532812487 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07400092d0 con 0x56245c603690 2024-05-21T21:57:44.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.873+0000 7f073cc00700 1 -- 172.21.15.151:0/1532812487 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0740007940 con 0x56245c603690 2024-05-21T21:57:44.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.874+0000 7f0758ac0b80 1 -- 172.21.15.151:0/1532812487 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56245c603690 msgr2=0x56245c603a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:44.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.874+0000 7f0758ac0b80 1 --2- 172.21.15.151:0/1532812487 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56245c603690 0x56245c603a70 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0740003da0 tx=0x7f0740009920 comp rx=0 tx=0).stop 2024-05-21T21:57:44.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.875+0000 7f0758ac0b80 1 -- 172.21.15.151:0/1532812487 shutdown_connections 2024-05-21T21:57:44.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.875+0000 7f0758ac0b80 1 --2- 172.21.15.151:0/1532812487 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56245c603690 0x56245c603a70 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:44.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.875+0000 7f0758ac0b80 1 --2- 172.21.15.151:0/1532812487 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56245c5fe1c0 0x56245c5fe640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:44.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.875+0000 7f0758ac0b80 1 --2- 172.21.15.151:0/1532812487 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56245c603fb0 0x56245c5fdba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:44.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.875+0000 7f0758ac0b80 1 -- 172.21.15.151:0/1532812487 >> 172.21.15.151:0/1532812487 conn(0x56245c52bc60 msgr2=0x56245c5f8710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:57:44.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.875+0000 7f0758ac0b80 1 -- 172.21.15.151:0/1532812487 shutdown_connections 2024-05-21T21:57:44.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.876+0000 7f0758ac0b80 1 -- 172.21.15.151:0/1532812487 wait complete. 2024-05-21T21:57:44.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.877+0000 7f073fe00700 1 --2- 172.21.15.151:0/2986857080 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f072c0b2a00 0x7f072c0b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:44.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.877+0000 7f0758ac0b80 1 -- start start 2024-05-21T21:57:44.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.877+0000 7f073fe00700 1 -- 172.21.15.151:0/2986857080 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f07340e2840 con 0x7f0748076090 2024-05-21T21:57:44.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.877+0000 7f0758ac0b80 1 Processor -- start 2024-05-21T21:57:44.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.878+0000 7f0747e00700 1 --2- 172.21.15.151:0/2986857080 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f072c0b2a00 0x7f072c0b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:44.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.879+0000 7f0758ac0b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56245c5fe1c0 0x56245c609cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:44.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.879+0000 7f0758ac0b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56245c603690 0x56245c6070d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:44.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.879+0000 7f0758ac0b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56245c603fb0 0x56245c607610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:44.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.879+0000 7f0758ac0b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56245c452020 con 0x56245c5fe1c0 2024-05-21T21:57:44.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.879+0000 7f0758ac0b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56245c4ec7e0 con 0x56245c603690 2024-05-21T21:57:44.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.879+0000 7f0758ac0b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56245c60aa60 con 0x56245c603fb0 2024-05-21T21:57:44.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.879+0000 7f0758ac0b80 10 client.0 _ll_register_callbacks cb 0x56245c4f5cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:57:44.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.879+0000 7f0747e00700 1 --2- 172.21.15.151:0/2986857080 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f072c0b2a00 0x7f072c0b4ec0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0740009dc0 tx=0x7f0740009ac0 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:57:44.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.879+0000 7f0746a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56245c603690 0x56245c6070d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:44.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.879+0000 7f0746a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56245c603690 0x56245c6070d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:47118/0 (socket says 172.21.15.151:47118) 2024-05-21T21:57:44.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.879+0000 7f0746a00700 1 -- 172.21.15.151:0/1808598228 learned_addr learned my addr 172.21.15.151:0/1808598228 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:57:44.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.880+0000 7f0747400700 1 --2- 172.21.15.151:0/1808598228 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56245c5fe1c0 0x56245c609cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:44.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.880+0000 7f0747e00700 1 --2- 172.21.15.151:0/1808598228 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56245c603fb0 0x56245c607610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:44.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.881+0000 7f0747400700 1 -- 172.21.15.151:0/1808598228 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56245c603fb0 msgr2=0x56245c607610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:44.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.881+0000 7f0747400700 1 --2- 172.21.15.151:0/1808598228 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56245c603fb0 0x56245c607610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:44.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.881+0000 7f0747400700 1 -- 172.21.15.151:0/1808598228 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56245c603690 msgr2=0x56245c6070d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:44.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.881+0000 7f0747400700 1 --2- 172.21.15.151:0/1808598228 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56245c603690 0x56245c6070d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:44.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.881+0000 7f0747400700 1 -- 172.21.15.151:0/1808598228 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56245c49b950 con 0x56245c5fe1c0 2024-05-21T21:57:44.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.881+0000 7f0747400700 1 --2- 172.21.15.151:0/1808598228 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56245c5fe1c0 0x56245c609cd0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f0738000c00 tx=0x7f0738005730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1f5e21e63cc0e1d server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:57:44.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.881+0000 7f0731600700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-21T21:57:44.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.881+0000 7f0731600700 1 -- 172.21.15.151:0/1808598228 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0738027060 con 0x56245c5fe1c0 2024-05-21T21:57:44.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.882+0000 7f0731600700 1 -- 172.21.15.151:0/1808598228 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f0738003680 con 0x56245c5fe1c0 2024-05-21T21:57:44.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.882+0000 7f0758ac0b80 1 -- 172.21.15.151:0/1808598228 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07400092d0 con 0x56245c5fe1c0 2024-05-21T21:57:44.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.882+0000 7f0731600700 1 -- 172.21.15.151:0/1808598228 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f07380095d0 con 0x56245c5fe1c0 2024-05-21T21:57:44.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.882+0000 7f0758ac0b80 10 client.15603 fetch_fsmap learned FSMap version 29 2024-05-21T21:57:44.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.882+0000 7f0758ac0b80 1 -- 172.21.15.151:0/1808598228 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56245c51c0e0 con 0x56245c5fe1c0 2024-05-21T21:57:44.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.883+0000 7f0731600700 1 -- 172.21.15.151:0/1808598228 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07400092d0 con 0x56245c5fe1c0 2024-05-21T21:57:44.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.883+0000 7f0731600700 1 -- 172.21.15.151:0/1808598228 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f0738021070 con 0x56245c5fe1c0 2024-05-21T21:57:44.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.883+0000 7f0758ac0b80 10 client.15603 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T21:57:44.183 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.883+0000 7f0758ac0b80 10 client.15603 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T21:57:44.183 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.883+0000 7f0758ac0b80 20 client.15603 populate_metadata read hostname 'smithi151' 2024-05-21T21:57:44.183 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.883+0000 7f0758ac0b80 1 --2- 172.21.15.151:0/1808598228 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x56245c617510 0x56245c619930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:44.183 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.883+0000 7f0758ac0b80 4 client.15603 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T21:57:44.183 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.883+0000 7f0758ac0b80 1 -- 172.21.15.151:0/1808598228 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56245c55de10 con 0x56245c617510 2024-05-21T21:57:44.183 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.884+0000 7f0746a00700 1 --2- 172.21.15.151:0/1808598228 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x56245c617510 0x56245c619930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:44.183 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.884+0000 7f0746a00700 1 --2- 172.21.15.151:0/1808598228 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x56245c617510 0x56245c619930 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4833e0aceaf8c9ab server_cookie=9ac72350a4a512b9 in_seq=0 out_seq=0 2024-05-21T21:57:44.183 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.884+0000 7f0731600700 10 client.15603 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T21:57:44.183 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.885+0000 7f0731600700 1 -- 172.21.15.151:0/1808598228 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x56245c55de10 con 0x56245c617510 2024-05-21T21:57:44.183 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.885+0000 7f0731600700 10 client.15603 handle_command_reply: tid=0 2024-05-21T21:57:44.183 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.885+0000 7f0758ac0b80 1 client.15603 shutdown 2024-05-21T21:57:44.183 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.885+0000 7f0758ac0b80 1 -- 172.21.15.151:0/1808598228 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56245c5fe1c0 msgr2=0x56245c609cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:44.183 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.885+0000 7f0758ac0b80 1 --2- 172.21.15.151:0/1808598228 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56245c5fe1c0 0x56245c609cd0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f0738000c00 tx=0x7f0738005730 comp rx=0 tx=0).stop 2024-05-21T21:57:44.183 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.886+0000 7f0758ac0b80 1 -- 172.21.15.151:0/1808598228 shutdown_connections 2024-05-21T21:57:44.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.886+0000 7f0758ac0b80 1 --2- 172.21.15.151:0/1808598228 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56245c603fb0 0x56245c607610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:44.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.886+0000 7f0758ac0b80 1 --2- 172.21.15.151:0/1808598228 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56245c5fe1c0 0x56245c609cd0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:44.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.886+0000 7f0758ac0b80 1 --2- 172.21.15.151:0/1808598228 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x56245c617510 0x56245c619930 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:44.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.886+0000 7f0758ac0b80 1 --2- 172.21.15.151:0/1808598228 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56245c603690 0x56245c6070d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:44.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.886+0000 7f0758ac0b80 1 -- 172.21.15.151:0/1808598228 >> 172.21.15.151:0/1808598228 conn(0x56245c52bc60 msgr2=0x56245c5f7df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:57:44.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.886+0000 7f0731600700 0 client.15603 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T21:57:44.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.886+0000 7f0758ac0b80 1 -- 172.21.15.151:0/1808598228 shutdown_connections 2024-05-21T21:57:44.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.886+0000 7f0758ac0b80 1 -- 172.21.15.151:0/1808598228 wait complete. 2024-05-21T21:57:44.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.886+0000 7f0758ac0b80 20 client.15603 trim_cache size 0 max 16384 2024-05-21T21:57:44.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.928+0000 7f0758ac0b80 1 Processor -- start 2024-05-21T21:57:44.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.928+0000 7f0758ac0b80 1 -- start start 2024-05-21T21:57:44.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.929+0000 7f0758ac0b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56245c603690 0x56245c5fdba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:44.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.929+0000 7f0758ac0b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56245c603fb0 0x56245c604c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:44.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.929+0000 7f0758ac0b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56245c613b00 0x56245c5f8570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:44.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.929+0000 7f0758ac0b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f07481b0030 con 0x56245c603fb0 2024-05-21T21:57:44.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.929+0000 7f0758ac0b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f07481b1b70 con 0x56245c613b00 2024-05-21T21:57:44.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.929+0000 7f0758ac0b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f07481b0700 con 0x56245c603690 2024-05-21T21:57:44.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.930+0000 7f0747400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56245c603690 0x56245c5fdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:44.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.930+0000 7f0747400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56245c603690 0x56245c5fdba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:39112/0 (socket says 172.21.15.151:39112) 2024-05-21T21:57:44.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.930+0000 7f0747400700 1 -- 172.21.15.151:0/2279260196 learned_addr learned my addr 172.21.15.151:0/2279260196 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:57:44.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.930+0000 7f0747e00700 1 --2- 172.21.15.151:0/2279260196 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56245c613b00 0x56245c5f8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:44.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.930+0000 7f0746a00700 1 --2- 172.21.15.151:0/2279260196 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56245c603fb0 0x56245c604c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:44.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.931+0000 7f0747400700 1 -- 172.21.15.151:0/2279260196 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56245c613b00 msgr2=0x56245c5f8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:44.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.931+0000 7f0747400700 1 --2- 172.21.15.151:0/2279260196 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56245c613b00 0x56245c5f8570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:44.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.931+0000 7f0747400700 1 -- 172.21.15.151:0/2279260196 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56245c603fb0 msgr2=0x56245c604c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:44.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.931+0000 7f0747400700 1 --2- 172.21.15.151:0/2279260196 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56245c603fb0 0x56245c604c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:44.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.931+0000 7f0747400700 1 -- 172.21.15.151:0/2279260196 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56245c605640 con 0x56245c603690 2024-05-21T21:57:44.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.931+0000 7f0747400700 1 --2- 172.21.15.151:0/2279260196 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56245c603690 0x56245c5fdba0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0738009990 tx=0x7f073800bab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65e4deb2121f2637 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:57:44.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.932+0000 7f0733e00700 1 -- 172.21.15.151:0/2279260196 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0738031080 con 0x56245c603690 2024-05-21T21:57:44.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.932+0000 7f0733e00700 1 -- 172.21.15.151:0/2279260196 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f073800f460 con 0x56245c603690 2024-05-21T21:57:44.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.932+0000 7f0733e00700 1 -- 172.21.15.151:0/2279260196 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0738003d90 con 0x56245c603690 2024-05-21T21:57:44.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.933+0000 7f0758ac0b80 1 -- 172.21.15.151:0/2279260196 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56245c603690 msgr2=0x56245c5fdba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:44.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.933+0000 7f0758ac0b80 1 --2- 172.21.15.151:0/2279260196 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56245c603690 0x56245c5fdba0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0738009990 tx=0x7f073800bab0 comp rx=0 tx=0).stop 2024-05-21T21:57:44.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.933+0000 7f0758ac0b80 1 -- 172.21.15.151:0/2279260196 shutdown_connections 2024-05-21T21:57:44.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.933+0000 7f0758ac0b80 1 --2- 172.21.15.151:0/2279260196 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56245c613b00 0x56245c5f8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:44.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.933+0000 7f0758ac0b80 1 --2- 172.21.15.151:0/2279260196 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56245c603690 0x56245c5fdba0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:44.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.933+0000 7f0758ac0b80 1 --2- 172.21.15.151:0/2279260196 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56245c603fb0 0x56245c604c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:44.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.933+0000 7f0758ac0b80 1 -- 172.21.15.151:0/2279260196 >> 172.21.15.151:0/2279260196 conn(0x56245c52bc60 msgr2=0x56245c607520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:57:44.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.934+0000 7f0758ac0b80 1 -- 172.21.15.151:0/2279260196 shutdown_connections 2024-05-21T21:57:44.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.934+0000 7f0758ac0b80 1 -- 172.21.15.151:0/2279260196 wait complete. 2024-05-21T21:57:44.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.934+0000 7f0758ac0b80 1 -- start start 2024-05-21T21:57:44.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.934+0000 7f0758ac0b80 1 Processor -- start 2024-05-21T21:57:44.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.935+0000 7f0758ac0b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56245c603690 0x56245c60ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:44.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.935+0000 7f0758ac0b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56245c603fb0 0x56245c60d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:44.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.935+0000 7f0758ac0b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56245c613b00 0x56245c60d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:44.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.935+0000 7f0758ac0b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f07481b14a0 con 0x56245c603690 2024-05-21T21:57:44.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.935+0000 7f0758ac0b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f07481b0dd0 con 0x56245c603fb0 2024-05-21T21:57:44.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.935+0000 7f0758ac0b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f07481adfd0 con 0x56245c613b00 2024-05-21T21:57:44.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.935+0000 7f0758ac0b80 10 client.0 _ll_register_callbacks cb 0x56245c4f5cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:57:44.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.936+0000 7f0746a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56245c603fb0 0x56245c60d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:44.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.936+0000 7f0747400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56245c603690 0x56245c60ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:44.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.936+0000 7f0746a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56245c603fb0 0x56245c60d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:47132/0 (socket says 172.21.15.151:47132) 2024-05-21T21:57:44.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.936+0000 7f0746a00700 1 -- 172.21.15.151:0/3168949988 learned_addr learned my addr 172.21.15.151:0/3168949988 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:57:44.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.936+0000 7f0747e00700 1 --2- 172.21.15.151:0/3168949988 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56245c613b00 0x56245c60d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:44.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.936+0000 7f0746a00700 1 -- 172.21.15.151:0/3168949988 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56245c613b00 msgr2=0x56245c60d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:44.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.936+0000 7f0746a00700 1 --2- 172.21.15.151:0/3168949988 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56245c613b00 0x56245c60d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:44.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.936+0000 7f0746a00700 1 -- 172.21.15.151:0/3168949988 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56245c603690 msgr2=0x56245c60ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:44.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.937+0000 7f0746a00700 1 --2- 172.21.15.151:0/3168949988 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56245c603690 0x56245c60ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:44.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.937+0000 7f0746a00700 1 -- 172.21.15.151:0/3168949988 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56245c6821c0 con 0x56245c603fb0 2024-05-21T21:57:44.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.937+0000 7f0747400700 1 --2- 172.21.15.151:0/3168949988 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56245c603690 0x56245c60ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:57:44.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.937+0000 7f0746a00700 1 --2- 172.21.15.151:0/3168949988 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56245c603fb0 0x56245c60d200 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f07340a97d0 tx=0x7f0734010bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ecfcf6e0f3784544 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:57:44.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.937+0000 7f0731600700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-21T21:57:44.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.937+0000 7f0731600700 1 -- 172.21.15.151:0/3168949988 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07340b2270 con 0x56245c603fb0 2024-05-21T21:57:44.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.938+0000 7f0731600700 1 -- 172.21.15.151:0/3168949988 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f07340aae00 con 0x56245c603fb0 2024-05-21T21:57:44.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.938+0000 7f0758ac0b80 1 -- 172.21.15.151:0/3168949988 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f073800f460 con 0x56245c603fb0 2024-05-21T21:57:44.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.939+0000 7f0731600700 1 -- 172.21.15.151:0/3168949988 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f07340a95e0 con 0x56245c603fb0 2024-05-21T21:57:44.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.939+0000 7f0758ac0b80 10 client.25733 fetch_fsmap learned FSMap version 29 2024-05-21T21:57:44.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.940+0000 7f0758ac0b80 1 -- 172.21.15.151:0/3168949988 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56245c683540 con 0x56245c603fb0 2024-05-21T21:57:44.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.940+0000 7f0731600700 1 -- 172.21.15.151:0/3168949988 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f073800f460 con 0x56245c603fb0 2024-05-21T21:57:44.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.940+0000 7f0731600700 1 -- 172.21.15.151:0/3168949988 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f0734010720 con 0x56245c603fb0 2024-05-21T21:57:44.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.941+0000 7f0758ac0b80 10 client.25733 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T21:57:44.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.941+0000 7f0758ac0b80 10 client.25733 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T21:57:44.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.941+0000 7f0758ac0b80 20 client.25733 populate_metadata read hostname 'smithi151' 2024-05-21T21:57:44.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.941+0000 7f0758ac0b80 1 --2- 172.21.15.151:0/3168949988 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x56245c683bb0 0x56245c685fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:44.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.941+0000 7f0758ac0b80 4 client.25733 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-21T21:57:44.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.941+0000 7f0758ac0b80 1 -- 172.21.15.151:0/3168949988 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56245c5575f0 con 0x56245c683bb0 2024-05-21T21:57:44.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.941+0000 7f0747400700 1 --2- 172.21.15.151:0/3168949988 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x56245c683bb0 0x56245c685fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:44.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.942+0000 7f0747400700 1 --2- 172.21.15.151:0/3168949988 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x56245c683bb0 0x56245c685fd0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8cfe379a57f27e54 server_cookie=bb03c14a7b01cc0e in_seq=0 out_seq=0 2024-05-21T21:57:44.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.942+0000 7f0731600700 10 client.25733 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T21:57:44.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.943+0000 7f0731600700 1 -- 172.21.15.151:0/3168949988 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56245c5575f0 con 0x56245c683bb0 2024-05-21T21:57:44.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.943+0000 7f0731600700 10 client.25733 handle_command_reply: tid=0 2024-05-21T21:57:44.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.943+0000 7f0758ac0b80 1 client.25733 shutdown 2024-05-21T21:57:44.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.944+0000 7f0758ac0b80 1 -- 172.21.15.151:0/3168949988 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56245c603fb0 msgr2=0x56245c60d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:44.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.944+0000 7f0758ac0b80 1 --2- 172.21.15.151:0/3168949988 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56245c603fb0 0x56245c60d200 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f07340a97d0 tx=0x7f0734010bb0 comp rx=0 tx=0).stop 2024-05-21T21:57:44.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.944+0000 7f0758ac0b80 1 -- 172.21.15.151:0/3168949988 shutdown_connections 2024-05-21T21:57:44.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.944+0000 7f0758ac0b80 1 --2- 172.21.15.151:0/3168949988 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56245c613b00 0x56245c60d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:44.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.944+0000 7f0758ac0b80 1 --2- 172.21.15.151:0/3168949988 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56245c603690 0x56245c60ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:44.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.944+0000 7f0758ac0b80 1 --2- 172.21.15.151:0/3168949988 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x56245c683bb0 0x56245c685fd0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:44.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.944+0000 7f0758ac0b80 1 --2- 172.21.15.151:0/3168949988 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56245c603fb0 0x56245c60d200 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:44.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.944+0000 7f0758ac0b80 1 -- 172.21.15.151:0/3168949988 >> 172.21.15.151:0/3168949988 conn(0x56245c52bc60 msgr2=0x56245c6021e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:57:44.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.944+0000 7f0731600700 0 client.25733 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T21:57:44.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.944+0000 7f0758ac0b80 1 -- 172.21.15.151:0/3168949988 shutdown_connections 2024-05-21T21:57:44.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.944+0000 7f0758ac0b80 1 -- 172.21.15.151:0/3168949988 wait complete. 2024-05-21T21:57:44.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.945+0000 7f0758ac0b80 20 client.25733 trim_cache size 0 max 16384 2024-05-21T21:57:44.191 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-05-21T21:57:44.191 INFO:teuthology.orchestra.run.smithi151.stdout: "return_code": 0, 2024-05-21T21:57:44.191 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_tag": "e4872893-1415-4c15-b059-da679779e2bb", 2024-05-21T21:57:44.191 INFO:teuthology.orchestra.run.smithi151.stdout: "mode": "asynchronous" 2024-05-21T21:57:44.191 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-05-21T21:57:44.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.946+0000 7f074e200700 1 -- 172.21.15.151:0/2986857080 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f072c0b2a00 msgr2=0x7f072c0b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:44.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.946+0000 7f074e200700 1 --2- 172.21.15.151:0/2986857080 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f072c0b2a00 0x7f072c0b4ec0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0740009dc0 tx=0x7f0740009ac0 comp rx=0 tx=0).stop 2024-05-21T21:57:44.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.946+0000 7f074e200700 1 -- 172.21.15.151:0/2986857080 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0748076090 msgr2=0x7f074819c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:44.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.946+0000 7f074e200700 1 --2- 172.21.15.151:0/2986857080 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0748076090 0x7f074819c390 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f073400e9a0 tx=0x7f073400ecb0 comp rx=0 tx=0).stop 2024-05-21T21:57:44.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.947+0000 7f074e200700 1 -- 172.21.15.151:0/2986857080 shutdown_connections 2024-05-21T21:57:44.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.947+0000 7f074e200700 1 --2- 172.21.15.151:0/2986857080 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f072c0b2a00 0x7f072c0b4ec0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:44.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.947+0000 7f074e200700 1 --2- 172.21.15.151:0/2986857080 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0748075770 0x7f074819be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:44.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.947+0000 7f074e200700 1 --2- 172.21.15.151:0/2986857080 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f074810ffa0 0x7f07481a06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:44.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.947+0000 7f074e200700 1 --2- 172.21.15.151:0/2986857080 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0748076090 0x7f074819c390 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:44.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.947+0000 7f074e200700 1 -- 172.21.15.151:0/2986857080 >> 172.21.15.151:0/2986857080 conn(0x7f074807a6a0 msgr2=0x7f07481007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:57:44.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.947+0000 7f074e200700 1 -- 172.21.15.151:0/2986857080 shutdown_connections 2024-05-21T21:57:44.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:43.947+0000 7f074e200700 1 -- 172.21.15.151:0/2986857080 wait complete. 2024-05-21T21:57:44.427 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:57:44 smithi151 ceph-mon[31623]: scrub queued for path: / 2024-05-21T21:57:44.427 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:57:44 smithi151 ceph-mon[31623]: scrub summary: idle+waiting paths [/] 2024-05-21T21:57:44.427 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:57:44 smithi151 ceph-mon[31623]: scrub summary: active paths [/] 2024-05-21T21:57:44.427 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:57:44 smithi151 ceph-mon[31623]: scrub summary: idle+waiting paths [/] 2024-05-21T21:57:44.427 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:57:44 smithi151 ceph-mon[31623]: scrub summary: idle 2024-05-21T21:57:44.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:57:44 smithi184 ceph-mon[36942]: scrub queued for path: / 2024-05-21T21:57:44.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:57:44 smithi184 ceph-mon[36942]: scrub summary: idle+waiting paths [/] 2024-05-21T21:57:44.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:57:44 smithi184 ceph-mon[36942]: scrub summary: active paths [/] 2024-05-21T21:57:44.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:57:44 smithi184 ceph-mon[36942]: scrub summary: idle+waiting paths [/] 2024-05-21T21:57:44.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:57:44 smithi184 ceph-mon[36942]: scrub summary: idle 2024-05-21T21:57:44.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:57:44 smithi164 ceph-mon[36269]: scrub queued for path: / 2024-05-21T21:57:44.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:57:44 smithi164 ceph-mon[36269]: scrub summary: idle+waiting paths [/] 2024-05-21T21:57:44.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:57:44 smithi164 ceph-mon[36269]: scrub summary: active paths [/] 2024-05-21T21:57:44.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:57:44 smithi164 ceph-mon[36269]: scrub summary: idle+waiting paths [/] 2024-05-21T21:57:44.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:57:44 smithi164 ceph-mon[36269]: scrub summary: idle 2024-05-21T21:57:45.381 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-21T21:57:45.590 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:57:45 smithi151 ceph-mon[31623]: pgmap v67: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T21:57:45.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:57:45 smithi184 ceph-mon[36942]: pgmap v67: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T21:57:45.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:57:45 smithi164 ceph-mon[36269]: pgmap v67: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T21:57:45.940 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:57:45.963 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.963+0000 7f5189e00700 1 -- 172.21.15.151:0/116252816 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f518406b220 msgr2=0x7f518406b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:45.963 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.963+0000 7f5189e00700 1 --2- 172.21.15.151:0/116252816 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f518406b220 0x7f518406b6a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5170009990 tx=0x7f5170009ca0 comp rx=0 tx=0).stop 2024-05-21T21:57:45.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.963+0000 7f5189e00700 1 -- 172.21.15.151:0/116252816 shutdown_connections 2024-05-21T21:57:45.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.963+0000 7f5189e00700 1 --2- 172.21.15.151:0/116252816 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f518406bbe0 0x7f51841103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:45.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.963+0000 7f5189e00700 1 --2- 172.21.15.151:0/116252816 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f51841095b0 0x7f5184109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:45.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.963+0000 7f5189e00700 1 --2- 172.21.15.151:0/116252816 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f518406b220 0x7f518406b6a0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:45.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.963+0000 7f5189e00700 1 -- 172.21.15.151:0/116252816 >> 172.21.15.151:0/116252816 conn(0x7f51840793f0 msgr2=0x7f5184079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:57:45.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.963+0000 7f5189e00700 1 -- 172.21.15.151:0/116252816 shutdown_connections 2024-05-21T21:57:45.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.963+0000 7f5189e00700 1 -- 172.21.15.151:0/116252816 wait complete. 2024-05-21T21:57:45.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.965+0000 7f5189e00700 1 Processor -- start 2024-05-21T21:57:45.966 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.965+0000 7f5189e00700 1 -- start start 2024-05-21T21:57:45.966 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.966+0000 7f5189e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f518406b220 0x7f518419f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:45.966 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.966+0000 7f5189e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f518406bbe0 0x7f518419fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:45.966 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.966+0000 7f5189e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f51841095b0 0x7f5184199a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:45.966 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.966+0000 7f5189e00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5184113740 con 0x7f518406b220 2024-05-21T21:57:45.966 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.966+0000 7f5189e00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f51841135c0 con 0x7f51841095b0 2024-05-21T21:57:45.966 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.966+0000 7f5189e00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f51841138c0 con 0x7f518406bbe0 2024-05-21T21:57:45.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.967+0000 7f5182000700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f518406bbe0 0x7f518419fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:45.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.967+0000 7f5182000700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f518406bbe0 0x7f518419fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:37222/0 (socket says 172.21.15.151:37222) 2024-05-21T21:57:45.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.967+0000 7f5182000700 1 -- 172.21.15.151:0/1494071252 learned_addr learned my addr 172.21.15.151:0/1494071252 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:57:45.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.967+0000 7f5182a00700 1 --2- 172.21.15.151:0/1494071252 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f518406b220 0x7f518419f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:45.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.968+0000 7f5183400700 1 --2- 172.21.15.151:0/1494071252 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f51841095b0 0x7f5184199a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:45.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.968+0000 7f5182000700 1 -- 172.21.15.151:0/1494071252 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f51841095b0 msgr2=0x7f5184199a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:45.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.968+0000 7f5182000700 1 --2- 172.21.15.151:0/1494071252 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f51841095b0 0x7f5184199a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:45.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.968+0000 7f5182000700 1 -- 172.21.15.151:0/1494071252 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f518406b220 msgr2=0x7f518419f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:45.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.968+0000 7f5182000700 1 --2- 172.21.15.151:0/1494071252 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f518406b220 0x7f518419f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:45.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.968+0000 7f5182000700 1 -- 172.21.15.151:0/1494071252 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f518419a1c0 con 0x7f518406bbe0 2024-05-21T21:57:45.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.968+0000 7f5182a00700 1 --2- 172.21.15.151:0/1494071252 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f518406b220 0x7f518419f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T21:57:45.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.969+0000 7f5182000700 1 --2- 172.21.15.151:0/1494071252 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f518406bbe0 0x7f518419fde0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5170005910 tx=0x7f517000b670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:57:45.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.969+0000 7f5188a00700 1 -- 172.21.15.151:0/1494071252 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f517000fa80 con 0x7f518406bbe0 2024-05-21T21:57:45.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.969+0000 7f5189e00700 1 -- 172.21.15.151:0/1494071252 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f518419a450 con 0x7f518406bbe0 2024-05-21T21:57:45.970 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.969+0000 7f5189e00700 1 -- 172.21.15.151:0/1494071252 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5184102440 con 0x7f518406bbe0 2024-05-21T21:57:45.970 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.970+0000 7f5188a00700 1 -- 172.21.15.151:0/1494071252 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f517000fc20 con 0x7f518406bbe0 2024-05-21T21:57:45.970 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.970+0000 7f5188a00700 1 -- 172.21.15.151:0/1494071252 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5170017650 con 0x7f518406bbe0 2024-05-21T21:57:45.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.975+0000 7f5188a00700 1 -- 172.21.15.151:0/1494071252 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7f5170009d10 con 0x7f518406bbe0 2024-05-21T21:57:45.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.976+0000 7f5194692b80 1 Processor -- start 2024-05-21T21:57:45.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.977+0000 7f5194692b80 1 -- start start 2024-05-21T21:57:45.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.977+0000 7f5194692b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c8c5253e0 0x560c8c5257c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:45.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.977+0000 7f5194692b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560c8c525d00 0x560c8c51f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:45.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.977+0000 7f5194692b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560c8c51ff80 0x560c8c5203c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:45.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.977+0000 7f5194692b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x560c8c47df40 con 0x560c8c51ff80 2024-05-21T21:57:45.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.977+0000 7f5194692b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x560c8c46dae0 con 0x560c8c5253e0 2024-05-21T21:57:45.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.977+0000 7f5194692b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x560c8c471b20 con 0x560c8c525d00 2024-05-21T21:57:45.979 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.978+0000 7f5183400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c8c5253e0 0x560c8c5257c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:45.979 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.978+0000 7f5182a00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560c8c525d00 0x560c8c51f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:45.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.978+0000 7f5182a00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560c8c525d00 0x560c8c51f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:37228/0 (socket says 172.21.15.151:37228) 2024-05-21T21:57:45.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.978+0000 7f5182a00700 1 -- 172.21.15.151:0/714442733 learned_addr learned my addr 172.21.15.151:0/714442733 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:57:45.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.978+0000 7f5182000700 1 --2- 172.21.15.151:0/714442733 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560c8c51ff80 0x560c8c5203c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:45.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.978+0000 7f5182a00700 1 -- 172.21.15.151:0/714442733 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c8c5253e0 msgr2=0x560c8c5257c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:45.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.978+0000 7f5182a00700 1 --2- 172.21.15.151:0/714442733 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c8c5253e0 0x560c8c5257c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:45.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.978+0000 7f5182a00700 1 -- 172.21.15.151:0/714442733 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560c8c51ff80 msgr2=0x560c8c5203c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:45.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.978+0000 7f5188a00700 1 --2- 172.21.15.151:0/1494071252 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f51680b2a70 0x7f51680b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:45.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.979+0000 7f5183400700 1 --2- 172.21.15.151:0/1494071252 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f51680b2a70 0x7f51680b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:45.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.979+0000 7f5188a00700 1 -- 172.21.15.151:0/1494071252 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f51700e76e0 con 0x7f518406bbe0 2024-05-21T21:57:45.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.980+0000 7f5183400700 1 --2- 172.21.15.151:0/1494071252 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f51680b2a70 0x7f51680b4f30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f517c009fd0 tx=0x7f517c009040 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:57:45.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.980+0000 7f5182a00700 1 --2- 172.21.15.151:0/714442733 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560c8c51ff80 0x560c8c5203c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:45.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.980+0000 7f5182a00700 1 -- 172.21.15.151:0/714442733 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c8c436520 con 0x560c8c525d00 2024-05-21T21:57:45.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.980+0000 7f5182000700 1 --2- 172.21.15.151:0/714442733 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560c8c51ff80 0x560c8c5203c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T21:57:45.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.980+0000 7f5182a00700 1 --2- 172.21.15.151:0/714442733 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560c8c525d00 0x560c8c51f960 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5174009c40 tx=0x7f5174009f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=486cb3078f09e78e server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:57:45.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.980+0000 7f5178c00700 1 -- 172.21.15.151:0/714442733 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5174016070 con 0x560c8c525d00 2024-05-21T21:57:45.982 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.980+0000 7f5178c00700 1 -- 172.21.15.151:0/714442733 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5174004830 con 0x560c8c525d00 2024-05-21T21:57:45.982 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.981+0000 7f5178c00700 1 -- 172.21.15.151:0/714442733 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51740054e0 con 0x560c8c525d00 2024-05-21T21:57:45.982 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.982+0000 7f5194692b80 1 -- 172.21.15.151:0/714442733 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560c8c525d00 msgr2=0x560c8c51f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:45.982 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.982+0000 7f5194692b80 1 --2- 172.21.15.151:0/714442733 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560c8c525d00 0x560c8c51f960 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5174009c40 tx=0x7f5174009f50 comp rx=0 tx=0).stop 2024-05-21T21:57:45.982 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.982+0000 7f5194692b80 1 -- 172.21.15.151:0/714442733 shutdown_connections 2024-05-21T21:57:45.982 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.982+0000 7f5194692b80 1 --2- 172.21.15.151:0/714442733 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c8c5253e0 0x560c8c5257c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:45.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.982+0000 7f5194692b80 1 --2- 172.21.15.151:0/714442733 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560c8c51ff80 0x560c8c5203c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:45.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.982+0000 7f5194692b80 1 --2- 172.21.15.151:0/714442733 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560c8c525d00 0x560c8c51f960 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:45.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.982+0000 7f5194692b80 1 -- 172.21.15.151:0/714442733 >> 172.21.15.151:0/714442733 conn(0x560c8c44dce0 msgr2=0x560c8c51a420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:57:45.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.982+0000 7f5194692b80 1 -- 172.21.15.151:0/714442733 shutdown_connections 2024-05-21T21:57:45.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.983+0000 7f5194692b80 1 -- 172.21.15.151:0/714442733 wait complete. 2024-05-21T21:57:45.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.983+0000 7f5194692b80 1 -- start start 2024-05-21T21:57:45.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.983+0000 7f5194692b80 1 Processor -- start 2024-05-21T21:57:45.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.984+0000 7f5194692b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c8c51ff80 0x560c8c52b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:45.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.984+0000 7f5194692b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560c8c5253e0 0x560c8c528d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:45.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.984+0000 7f5194692b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560c8c525d00 0x560c8c5292a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:45.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.984+0000 7f5194692b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x560c8c3740a0 con 0x560c8c5253e0 2024-05-21T21:57:45.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.984+0000 7f5194692b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x560c8c40e860 con 0x560c8c51ff80 2024-05-21T21:57:45.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.984+0000 7f5194692b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x560c8c52c7e0 con 0x560c8c525d00 2024-05-21T21:57:45.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.984+0000 7f5194692b80 10 client.0 _ll_register_callbacks cb 0x560c8c417d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:57:45.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.985+0000 7f5182a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c8c51ff80 0x560c8c52b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:45.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.985+0000 7f5182a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c8c51ff80 0x560c8c52b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:59548/0 (socket says 172.21.15.151:59548) 2024-05-21T21:57:45.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.985+0000 7f5182a00700 1 -- 172.21.15.151:0/4037034306 learned_addr learned my addr 172.21.15.151:0/4037034306 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:57:45.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.985+0000 7f5183400700 1 --2- 172.21.15.151:0/4037034306 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560c8c525d00 0x560c8c5292a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:45.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.986+0000 7f5182000700 1 --2- 172.21.15.151:0/4037034306 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560c8c5253e0 0x560c8c528d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:45.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.986+0000 7f5182a00700 1 -- 172.21.15.151:0/4037034306 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560c8c525d00 msgr2=0x560c8c5292a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:45.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.986+0000 7f5182a00700 1 --2- 172.21.15.151:0/4037034306 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560c8c525d00 0x560c8c5292a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:45.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.986+0000 7f5182a00700 1 -- 172.21.15.151:0/4037034306 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560c8c5253e0 msgr2=0x560c8c528d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:45.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.986+0000 7f5182a00700 1 --2- 172.21.15.151:0/4037034306 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560c8c5253e0 0x560c8c528d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:45.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.986+0000 7f5182a00700 1 -- 172.21.15.151:0/4037034306 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c8c3bd9d0 con 0x560c8c51ff80 2024-05-21T21:57:45.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.986+0000 7f5183400700 1 --2- 172.21.15.151:0/4037034306 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560c8c525d00 0x560c8c5292a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:57:45.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.987+0000 7f5182a00700 1 --2- 172.21.15.151:0/4037034306 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c8c51ff80 0x560c8c52b9c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f517400eb30 tx=0x7f517400eb60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1769e717f61ab20 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:57:45.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.987+0000 7f516d600700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-21T21:57:45.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.987+0000 7f516d600700 1 -- 172.21.15.151:0/4037034306 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5174016040 con 0x560c8c51ff80 2024-05-21T21:57:45.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.987+0000 7f5194692b80 1 -- 172.21.15.151:0/4037034306 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5174004830 con 0x560c8c51ff80 2024-05-21T21:57:45.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.987+0000 7f516d600700 1 -- 172.21.15.151:0/4037034306 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f5174004020 con 0x560c8c51ff80 2024-05-21T21:57:45.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.987+0000 7f516d600700 1 -- 172.21.15.151:0/4037034306 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5174004220 con 0x560c8c51ff80 2024-05-21T21:57:45.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.988+0000 7f5194692b80 10 client.25745 fetch_fsmap learned FSMap version 29 2024-05-21T21:57:45.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.988+0000 7f5194692b80 1 -- 172.21.15.151:0/4037034306 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560c8c43e160 con 0x560c8c51ff80 2024-05-21T21:57:45.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.988+0000 7f516d600700 1 -- 172.21.15.151:0/4037034306 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5174004830 con 0x560c8c51ff80 2024-05-21T21:57:45.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.988+0000 7f516d600700 1 -- 172.21.15.151:0/4037034306 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f517400f040 con 0x560c8c51ff80 2024-05-21T21:57:45.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.988+0000 7f5194692b80 10 client.25745 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T21:57:45.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.988+0000 7f5194692b80 10 client.25745 resolve_mds: validated gid 24530 aka daemon mds.b 2024-05-21T21:57:45.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.988+0000 7f5194692b80 20 client.25745 populate_metadata read hostname 'smithi151' 2024-05-21T21:57:45.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.988+0000 7f5194692b80 1 --2- 172.21.15.151:0/4037034306 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x560c8c541a20 0x560c8c543e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:45.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.988+0000 7f5194692b80 4 client.25745 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T21:57:45.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.988+0000 7f5194692b80 1 -- 172.21.15.151:0/4037034306 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560c8c47fe90 con 0x560c8c541a20 2024-05-21T21:57:45.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.989+0000 7f5182000700 1 --2- 172.21.15.151:0/4037034306 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x560c8c541a20 0x560c8c543e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:45.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.989+0000 7f5182000700 1 --2- 172.21.15.151:0/4037034306 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x560c8c541a20 0x560c8c543e10 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ea83f4035010db7 server_cookie=387c466d6bf72164 in_seq=0 out_seq=0 2024-05-21T21:57:45.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.989+0000 7f516d600700 10 client.25745 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T21:57:45.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.990+0000 7f516d600700 1 -- 172.21.15.151:0/4037034306 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x560c8c47fe90 con 0x560c8c541a20 2024-05-21T21:57:45.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.990+0000 7f516d600700 10 client.25745 handle_command_reply: tid=0 2024-05-21T21:57:45.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.990+0000 7f5194692b80 1 client.25745 shutdown 2024-05-21T21:57:45.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.991+0000 7f5194692b80 1 -- 172.21.15.151:0/4037034306 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c8c51ff80 msgr2=0x560c8c52b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:45.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.991+0000 7f5194692b80 1 --2- 172.21.15.151:0/4037034306 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c8c51ff80 0x560c8c52b9c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f517400eb30 tx=0x7f517400eb60 comp rx=0 tx=0).stop 2024-05-21T21:57:45.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.991+0000 7f5194692b80 1 -- 172.21.15.151:0/4037034306 shutdown_connections 2024-05-21T21:57:45.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.991+0000 7f5194692b80 1 --2- 172.21.15.151:0/4037034306 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x560c8c541a20 0x560c8c543e10 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:45.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.991+0000 7f5194692b80 1 --2- 172.21.15.151:0/4037034306 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c8c51ff80 0x560c8c52b9c0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:45.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.991+0000 7f5194692b80 1 --2- 172.21.15.151:0/4037034306 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560c8c525d00 0x560c8c5292a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:45.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.991+0000 7f5194692b80 1 --2- 172.21.15.151:0/4037034306 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560c8c5253e0 0x560c8c528d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:45.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.991+0000 7f5194692b80 1 -- 172.21.15.151:0/4037034306 >> 172.21.15.151:0/4037034306 conn(0x560c8c44dce0 msgr2=0x560c8c51e950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:57:45.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.991+0000 7f516d600700 0 client.25745 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T21:57:45.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.991+0000 7f5194692b80 1 -- 172.21.15.151:0/4037034306 shutdown_connections 2024-05-21T21:57:45.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.991+0000 7f5194692b80 1 -- 172.21.15.151:0/4037034306 wait complete. 2024-05-21T21:57:45.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:45.991+0000 7f5194692b80 20 client.25745 trim_cache size 0 max 16384 2024-05-21T21:57:46.026 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.025+0000 7f5194692b80 1 Processor -- start 2024-05-21T21:57:46.026 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.026+0000 7f5194692b80 1 -- start start 2024-05-21T21:57:46.026 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.026+0000 7f5194692b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560c8c51ff80 0x560c8c529250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:46.026 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.026+0000 7f5194692b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560c8c5253e0 0x560c8c5493e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:46.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.026+0000 7f5194692b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c8c525d00 0x560c8c549920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:46.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.026+0000 7f5194692b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f516400d670 con 0x560c8c51ff80 2024-05-21T21:57:46.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.026+0000 7f5194692b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f516400f1b0 con 0x560c8c525d00 2024-05-21T21:57:46.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.026+0000 7f5194692b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f516400dd40 con 0x560c8c5253e0 2024-05-21T21:57:46.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.027+0000 7f5183400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c8c525d00 0x560c8c549920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:46.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.027+0000 7f5183400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c8c525d00 0x560c8c549920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:59554/0 (socket says 172.21.15.151:59554) 2024-05-21T21:57:46.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.027+0000 7f5183400700 1 -- 172.21.15.151:0/923994569 learned_addr learned my addr 172.21.15.151:0/923994569 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:57:46.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.027+0000 7f5182a00700 1 --2- 172.21.15.151:0/923994569 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560c8c51ff80 0x560c8c529250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:46.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.028+0000 7f5183400700 1 -- 172.21.15.151:0/923994569 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560c8c5253e0 msgr2=0x560c8c5493e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-21T21:57:46.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.028+0000 7f5183400700 1 --2- 172.21.15.151:0/923994569 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560c8c5253e0 0x560c8c5493e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:46.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.028+0000 7f5183400700 1 -- 172.21.15.151:0/923994569 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560c8c51ff80 msgr2=0x560c8c529250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:46.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.028+0000 7f5183400700 1 --2- 172.21.15.151:0/923994569 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560c8c51ff80 0x560c8c529250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:46.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.028+0000 7f5183400700 1 -- 172.21.15.151:0/923994569 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c8c51bde0 con 0x560c8c525d00 2024-05-21T21:57:46.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.029+0000 7f5183400700 1 --2- 172.21.15.151:0/923994569 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c8c525d00 0x560c8c549920 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f517c016840 tx=0x7f517c004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45e68c382e667e7c server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:57:46.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.030+0000 7f516fe00700 1 -- 172.21.15.151:0/923994569 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f517c028070 con 0x560c8c525d00 2024-05-21T21:57:46.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.030+0000 7f516fe00700 1 -- 172.21.15.151:0/923994569 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f517c024ce0 con 0x560c8c525d00 2024-05-21T21:57:46.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.030+0000 7f516fe00700 1 -- 172.21.15.151:0/923994569 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f517c01c070 con 0x560c8c525d00 2024-05-21T21:57:46.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.031+0000 7f5194692b80 1 -- 172.21.15.151:0/923994569 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c8c525d00 msgr2=0x560c8c549920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:46.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.031+0000 7f5194692b80 1 --2- 172.21.15.151:0/923994569 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c8c525d00 0x560c8c549920 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f517c016840 tx=0x7f517c004ad0 comp rx=0 tx=0).stop 2024-05-21T21:57:46.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.031+0000 7f5194692b80 1 -- 172.21.15.151:0/923994569 shutdown_connections 2024-05-21T21:57:46.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.031+0000 7f5194692b80 1 --2- 172.21.15.151:0/923994569 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c8c525d00 0x560c8c549920 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:46.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.031+0000 7f5194692b80 1 --2- 172.21.15.151:0/923994569 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560c8c51ff80 0x560c8c529250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:46.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.031+0000 7f5194692b80 1 --2- 172.21.15.151:0/923994569 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560c8c5253e0 0x560c8c5493e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:46.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.031+0000 7f5194692b80 1 -- 172.21.15.151:0/923994569 >> 172.21.15.151:0/923994569 conn(0x560c8c44dce0 msgr2=0x560c8c528b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:57:46.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.032+0000 7f5194692b80 1 -- 172.21.15.151:0/923994569 shutdown_connections 2024-05-21T21:57:46.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.032+0000 7f5194692b80 1 -- 172.21.15.151:0/923994569 wait complete. 2024-05-21T21:57:46.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.032+0000 7f5194692b80 1 -- start start 2024-05-21T21:57:46.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.033+0000 7f5194692b80 1 Processor -- start 2024-05-21T21:57:46.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.033+0000 7f5194692b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c8c51ff80 0x560c8c53a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:46.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.033+0000 7f5194692b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560c8c5253e0 0x560c8c537360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:46.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.033+0000 7f5194692b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560c8c525d00 0x560c8c5378a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:46.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.033+0000 7f5194692b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f516400eae0 con 0x560c8c5253e0 2024-05-21T21:57:46.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.033+0000 7f5194692b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f516400e410 con 0x560c8c51ff80 2024-05-21T21:57:46.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.033+0000 7f5194692b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f516400b5b0 con 0x560c8c525d00 2024-05-21T21:57:46.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.034+0000 7f5194692b80 10 client.0 _ll_register_callbacks cb 0x560c8c51aa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:57:46.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.034+0000 7f5182000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560c8c5253e0 0x560c8c537360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:46.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.034+0000 7f5182000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560c8c5253e0 0x560c8c537360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:59164/0 (socket says 172.21.15.151:59164) 2024-05-21T21:57:46.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.034+0000 7f5182000700 1 -- 172.21.15.151:0/414113609 learned_addr learned my addr 172.21.15.151:0/414113609 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:57:46.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.034+0000 7f5182a00700 1 --2- 172.21.15.151:0/414113609 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c8c51ff80 0x560c8c53a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:46.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.034+0000 7f5183400700 1 --2- 172.21.15.151:0/414113609 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560c8c525d00 0x560c8c5378a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:46.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.035+0000 7f5182000700 1 -- 172.21.15.151:0/414113609 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560c8c525d00 msgr2=0x560c8c5378a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:46.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.035+0000 7f5182000700 1 --2- 172.21.15.151:0/414113609 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560c8c525d00 0x560c8c5378a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:46.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.035+0000 7f5182000700 1 -- 172.21.15.151:0/414113609 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c8c51ff80 msgr2=0x560c8c53a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:46.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.035+0000 7f5182000700 1 --2- 172.21.15.151:0/414113609 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c8c51ff80 0x560c8c53a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:46.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.035+0000 7f5182000700 1 -- 172.21.15.151:0/414113609 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c8c537fb0 con 0x560c8c5253e0 2024-05-21T21:57:46.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.035+0000 7f5182a00700 1 --2- 172.21.15.151:0/414113609 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c8c51ff80 0x560c8c53a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:57:46.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.035+0000 7f5182000700 1 --2- 172.21.15.151:0/414113609 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560c8c5253e0 0x560c8c537360 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f51700abd80 tx=0x7f5170027fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c9ec12da2bed0b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:57:46.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.035+0000 7f516d600700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-21T21:57:46.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.036+0000 7f516d600700 1 -- 172.21.15.151:0/414113609 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51700e5270 con 0x560c8c5253e0 2024-05-21T21:57:46.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.036+0000 7f5194692b80 1 -- 172.21.15.151:0/414113609 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f517c024ce0 con 0x560c8c5253e0 2024-05-21T21:57:46.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.036+0000 7f516d600700 1 -- 172.21.15.151:0/414113609 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f51700ab100 con 0x560c8c5253e0 2024-05-21T21:57:46.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.036+0000 7f516d600700 1 -- 172.21.15.151:0/414113609 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f51700b7220 con 0x560c8c5253e0 2024-05-21T21:57:46.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.036+0000 7f5194692b80 10 client.15621 fetch_fsmap learned FSMap version 29 2024-05-21T21:57:46.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.036+0000 7f5194692b80 1 -- 172.21.15.151:0/414113609 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560c8c5ad2d0 con 0x560c8c5253e0 2024-05-21T21:57:46.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.037+0000 7f516d600700 1 -- 172.21.15.151:0/414113609 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f517c024ce0 con 0x560c8c5253e0 2024-05-21T21:57:46.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.037+0000 7f516d600700 1 -- 172.21.15.151:0/414113609 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f51700aed70 con 0x560c8c5253e0 2024-05-21T21:57:46.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.037+0000 7f5194692b80 10 client.15621 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T21:57:46.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.037+0000 7f5194692b80 10 client.15621 resolve_mds: validated gid 24530 aka daemon mds.b 2024-05-21T21:57:46.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.037+0000 7f5194692b80 20 client.15621 populate_metadata read hostname 'smithi151' 2024-05-21T21:57:46.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.037+0000 7f5194692b80 1 --2- 172.21.15.151:0/414113609 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x560c8c5ad9e0 0x560c8c5afe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:46.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.037+0000 7f5194692b80 4 client.15621 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "f1b4324", "release": true, "await": true}] 2024-05-21T21:57:46.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.037+0000 7f5194692b80 1 -- 172.21.15.151:0/414113609 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "quiesce db", "set_id": "f1b4324", "release": true, "await": true}) -- 0x560c8c479670 con 0x560c8c5ad9e0 2024-05-21T21:57:46.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.038+0000 7f5182a00700 1 --2- 172.21.15.151:0/414113609 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x560c8c5ad9e0 0x560c8c5afe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:46.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.039+0000 7f5182a00700 1 --2- 172.21.15.151:0/414113609 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x560c8c5ad9e0 0x560c8c5afe00 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3e3d575b61a2de3 server_cookie=9455b46c3c1c1090 in_seq=0 out_seq=0 2024-05-21T21:57:46.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.039+0000 7f516d600700 10 client.15621 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T21:57:46.042 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.042+0000 7f516d600700 1 -- 172.21.15.151:0/414113609 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x560c8c479670 con 0x560c8c5ad9e0 2024-05-21T21:57:46.042 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.042+0000 7f516d600700 10 client.15621 handle_command_reply: tid=0 2024-05-21T21:57:46.042 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.042+0000 7f5194692b80 1 client.15621 shutdown 2024-05-21T21:57:46.042 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.042+0000 7f5194692b80 1 -- 172.21.15.151:0/414113609 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560c8c5253e0 msgr2=0x560c8c537360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:46.042 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.042+0000 7f5194692b80 1 --2- 172.21.15.151:0/414113609 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560c8c5253e0 0x560c8c537360 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f51700abd80 tx=0x7f5170027fc0 comp rx=0 tx=0).stop 2024-05-21T21:57:46.043 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.042+0000 7f5194692b80 1 -- 172.21.15.151:0/414113609 shutdown_connections 2024-05-21T21:57:46.043 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.042+0000 7f5194692b80 1 --2- 172.21.15.151:0/414113609 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x560c8c5ad9e0 0x560c8c5afe00 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:46.043 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.042+0000 7f5194692b80 1 --2- 172.21.15.151:0/414113609 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560c8c51ff80 0x560c8c53a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:46.043 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.042+0000 7f5194692b80 1 --2- 172.21.15.151:0/414113609 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560c8c525d00 0x560c8c5378a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:46.043 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.042+0000 7f5194692b80 1 --2- 172.21.15.151:0/414113609 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560c8c5253e0 0x560c8c537360 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:46.043 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.042+0000 7f516d600700 0 client.15621 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T21:57:46.043 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.042+0000 7f5194692b80 1 -- 172.21.15.151:0/414113609 >> 172.21.15.151:0/414113609 conn(0x560c8c44dce0 msgr2=0x560c8c513d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:57:46.043 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.043+0000 7f5194692b80 1 -- 172.21.15.151:0/414113609 shutdown_connections 2024-05-21T21:57:46.043 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.043+0000 7f5194692b80 1 -- 172.21.15.151:0/414113609 wait complete. 2024-05-21T21:57:46.044 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.043+0000 7f5194692b80 20 client.15621 trim_cache size 0 max 16384 2024-05-21T21:57:46.044 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-05-21T21:57:46.044 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 29, 2024-05-21T21:57:46.044 INFO:teuthology.orchestra.run.smithi151.stdout: "leader": 24530, 2024-05-21T21:57:46.044 INFO:teuthology.orchestra.run.smithi151.stdout: "set_version": 4, 2024-05-21T21:57:46.044 INFO:teuthology.orchestra.run.smithi151.stdout: "sets": { 2024-05-21T21:57:46.044 INFO:teuthology.orchestra.run.smithi151.stdout: "f1b4324": { 2024-05-21T21:57:46.044 INFO:teuthology.orchestra.run.smithi151.stdout: "version": 4, 2024-05-21T21:57:46.044 INFO:teuthology.orchestra.run.smithi151.stdout: "age_ref": 0.0, 2024-05-21T21:57:46.045 INFO:teuthology.orchestra.run.smithi151.stdout: "state": { 2024-05-21T21:57:46.045 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "RELEASED", 2024-05-21T21:57:46.045 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 0.0 2024-05-21T21:57:46.045 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T21:57:46.045 INFO:teuthology.orchestra.run.smithi151.stdout: "timeout": 90.0, 2024-05-21T21:57:46.045 INFO:teuthology.orchestra.run.smithi151.stdout: "expiration": 146.2, 2024-05-21T21:57:46.045 INFO:teuthology.orchestra.run.smithi151.stdout: "members": { 2024-05-21T21:57:46.045 INFO:teuthology.orchestra.run.smithi151.stdout: "file:/": { 2024-05-21T21:57:46.045 INFO:teuthology.orchestra.run.smithi151.stdout: "excluded": false, 2024-05-21T21:57:46.045 INFO:teuthology.orchestra.run.smithi151.stdout: "state": { 2024-05-21T21:57:46.045 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "RELEASED", 2024-05-21T21:57:46.045 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 0.0 2024-05-21T21:57:46.045 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T21:57:46.045 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T21:57:46.045 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T21:57:46.045 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T21:57:46.046 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T21:57:46.046 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-05-21T21:57:46.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.048+0000 7f5189e00700 1 -- 172.21.15.151:0/1494071252 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f51680b2a70 msgr2=0x7f51680b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:46.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.048+0000 7f5189e00700 1 --2- 172.21.15.151:0/1494071252 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f51680b2a70 0x7f51680b4f30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f517c009fd0 tx=0x7f517c009040 comp rx=0 tx=0).stop 2024-05-21T21:57:46.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.048+0000 7f5189e00700 1 -- 172.21.15.151:0/1494071252 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f518406bbe0 msgr2=0x7f518419fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:46.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.048+0000 7f5189e00700 1 --2- 172.21.15.151:0/1494071252 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f518406bbe0 0x7f518419fde0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5170005910 tx=0x7f517000b670 comp rx=0 tx=0).stop 2024-05-21T21:57:46.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.048+0000 7f5189e00700 1 -- 172.21.15.151:0/1494071252 shutdown_connections 2024-05-21T21:57:46.049 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.048+0000 7f5189e00700 1 --2- 172.21.15.151:0/1494071252 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f51680b2a70 0x7f51680b4f30 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:46.049 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.048+0000 7f5189e00700 1 --2- 172.21.15.151:0/1494071252 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f51841095b0 0x7f5184199a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:46.049 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.048+0000 7f5189e00700 1 --2- 172.21.15.151:0/1494071252 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f518406b220 0x7f518419f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:46.049 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.048+0000 7f5189e00700 1 --2- 172.21.15.151:0/1494071252 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f518406bbe0 0x7f518419fde0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:46.049 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.048+0000 7f5189e00700 1 -- 172.21.15.151:0/1494071252 >> 172.21.15.151:0/1494071252 conn(0x7f51840793f0 msgr2=0x7f5184101240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:57:46.049 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.048+0000 7f5189e00700 1 -- 172.21.15.151:0/1494071252 shutdown_connections 2024-05-21T21:57:46.049 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:46.049+0000 7f5189e00700 1 -- 172.21.15.151:0/1494071252 wait complete. 2024-05-21T21:57:47.432 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:57:47 smithi151 ceph-mon[31623]: pgmap v68: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T21:57:47.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:57:47 smithi184 ceph-mon[36942]: pgmap v68: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T21:57:47.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:57:47 smithi164 ceph-mon[36269]: pgmap v68: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T21:57:47.969 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'f1b4324', total seconds elapsed: 42.3 2024-05-21T21:57:47.970 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 26.15 2024-05-21T21:57:47.970 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-21T21:57:48.658 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.655+0000 7f4d9ea00700 1 -- 172.21.15.151:0/3738051725 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4d98104630 msgr2=0x7f4d98104a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:48.658 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.655+0000 7f4d9ea00700 1 --2- 172.21.15.151:0/3738051725 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4d98104630 0x7f4d98104a10 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4d840098f0 tx=0x7f4d84009c00 comp rx=0 tx=0).stop 2024-05-21T21:57:48.659 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.655+0000 7f4d9ea00700 1 -- 172.21.15.151:0/3738051725 shutdown_connections 2024-05-21T21:57:48.659 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.655+0000 7f4d9ea00700 1 --2- 172.21.15.151:0/3738051725 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4d98104630 0x7f4d98104a10 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:48.659 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.655+0000 7f4d9ea00700 1 --2- 172.21.15.151:0/3738051725 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4d9810da60 0x7f4d9810ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:48.659 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.655+0000 7f4d9ea00700 1 --2- 172.21.15.151:0/3738051725 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4d98104f50 0x7f4d9810d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:48.659 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.655+0000 7f4d9ea00700 1 -- 172.21.15.151:0/3738051725 >> 172.21.15.151:0/3738051725 conn(0x7f4d980fde60 msgr2=0x7f4d98100280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:57:48.659 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.656+0000 7f4d9ea00700 1 -- 172.21.15.151:0/3738051725 shutdown_connections 2024-05-21T21:57:48.659 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.656+0000 7f4d9ea00700 1 -- 172.21.15.151:0/3738051725 wait complete. 2024-05-21T21:57:48.659 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.657+0000 7f4d9ea00700 1 Processor -- start 2024-05-21T21:57:48.659 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.657+0000 7f4d9ea00700 1 -- start start 2024-05-21T21:57:48.659 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.658+0000 7f4d9ea00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4d98104630 0x7f4d98074d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:48.660 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.658+0000 7f4d9ea00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4d98104f50 0x7f4d9806de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:48.660 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.658+0000 7f4d9ea00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4d9810da60 0x7f4d9806e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:48.660 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.659+0000 7f4d9ea00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4d981136a0 con 0x7f4d9810da60 2024-05-21T21:57:48.660 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.659+0000 7f4d9e000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4d9810da60 0x7f4d9806e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:48.660 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.659+0000 7f4d9e000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4d9810da60 0x7f4d9806e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:59172/0 (socket says 172.21.15.151:59172) 2024-05-21T21:57:48.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.659+0000 7f4d9e000700 1 -- 172.21.15.151:0/11772867 learned_addr learned my addr 172.21.15.151:0/11772867 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:57:48.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.659+0000 7f4d9ea00700 1 -- 172.21.15.151:0/11772867 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4d98113520 con 0x7f4d98104630 2024-05-21T21:57:48.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.659+0000 7f4d9ea00700 1 -- 172.21.15.151:0/11772867 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4d98113820 con 0x7f4d98104f50 2024-05-21T21:57:48.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.660+0000 7f4d9d600700 1 --2- 172.21.15.151:0/11772867 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4d98104630 0x7f4d98074d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:48.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.660+0000 7f4d9e000700 1 -- 172.21.15.151:0/11772867 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4d98104f50 msgr2=0x7f4d9806de90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-21T21:57:48.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.660+0000 7f4d9e000700 1 --2- 172.21.15.151:0/11772867 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4d98104f50 0x7f4d9806de90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:48.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.660+0000 7f4d9e000700 1 -- 172.21.15.151:0/11772867 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4d98104630 msgr2=0x7f4d98074d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:48.662 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.661+0000 7f4d9e000700 1 --2- 172.21.15.151:0/11772867 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4d98104630 0x7f4d98074d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:48.662 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.661+0000 7f4d9e000700 1 -- 172.21.15.151:0/11772867 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d9806ea10 con 0x7f4d9810da60 2024-05-21T21:57:48.662 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.661+0000 7f4d9d600700 1 --2- 172.21.15.151:0/11772867 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4d98104630 0x7f4d98074d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:57:48.662 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.661+0000 7f4d9e000700 1 --2- 172.21.15.151:0/11772867 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4d9810da60 0x7f4d9806e3d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f4d9000e3c0 tx=0x7f4d9000e6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:57:48.662 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.662+0000 7f4d96000700 1 -- 172.21.15.151:0/11772867 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d900092d0 con 0x7f4d9810da60 2024-05-21T21:57:48.662 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.662+0000 7f4d96000700 1 -- 172.21.15.151:0/11772867 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d9000ee70 con 0x7f4d9810da60 2024-05-21T21:57:48.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.662+0000 7f4d96000700 1 -- 172.21.15.151:0/11772867 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d90004030 con 0x7f4d9810da60 2024-05-21T21:57:48.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.662+0000 7f4d9ea00700 1 -- 172.21.15.151:0/11772867 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d98072890 con 0x7f4d9810da60 2024-05-21T21:57:48.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.663+0000 7f4d9ea00700 1 -- 172.21.15.151:0/11772867 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d98072d80 con 0x7f4d9810da60 2024-05-21T21:57:48.666 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:57:48.668 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.667+0000 7f4d96000700 1 -- 172.21.15.151:0/11772867 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7f4d90012050 con 0x7f4d9810da60 2024-05-21T21:57:48.669 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.669+0000 7f4d96000700 1 --2- 172.21.15.151:0/11772867 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f4d800b2a70 0x7f4d800b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:48.669 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.669+0000 7f4d96000700 1 -- 172.21.15.151:0/11772867 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f4d900e1df0 con 0x7f4d9810da60 2024-05-21T21:57:48.669 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.669+0000 7f4daa934b80 1 Processor -- start 2024-05-21T21:57:48.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.670+0000 7f4daa934b80 1 -- start start 2024-05-21T21:57:48.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.670+0000 7f4d9d600700 1 --2- 172.21.15.151:0/11772867 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f4d800b2a70 0x7f4d800b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:48.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.670+0000 7f4daa934b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e09ae0e7c0 0x55e09ae0eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:48.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.670+0000 7f4daa934b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e09ae0f0e0 0x55e09ae08d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:48.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.671+0000 7f4daa934b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e09ae09390 0x55e09ae097f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:48.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.671+0000 7f4daa934b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e09ad66e40 con 0x55e09ae0f0e0 2024-05-21T21:57:48.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.671+0000 7f4daa934b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55e09ad569e0 con 0x55e09ae0e7c0 2024-05-21T21:57:48.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.671+0000 7f4daa934b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55e09ad5aa20 con 0x55e09ae09390 2024-05-21T21:57:48.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.671+0000 7f4d9cc00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e09ae0f0e0 0x55e09ae08d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:48.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.672+0000 7f4d9e000700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e09ae09390 0x55e09ae097f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:48.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.672+0000 7f4d9e000700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e09ae09390 0x55e09ae097f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:37264/0 (socket says 172.21.15.151:37264) 2024-05-21T21:57:48.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.672+0000 7f4d9e000700 1 -- 172.21.15.151:0/398330233 learned_addr learned my addr 172.21.15.151:0/398330233 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:57:48.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.672+0000 7f4d9d600700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e09ae0e7c0 0x55e09ae0eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:48.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.672+0000 7f4d9cc00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e09ae0f0e0 0x55e09ae08d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:59182/0 (socket says 172.21.15.151:59182) 2024-05-21T21:57:48.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.672+0000 7f4d9d600700 1 --2- 172.21.15.151:0/11772867 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f4d800b2a70 0x7f4d800b4f30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4d840098c0 tx=0x7f4d8401d040 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:57:48.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.673+0000 7f4d9cc00700 1 -- 172.21.15.151:0/398330233 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e09ae09390 msgr2=0x55e09ae097f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:48.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.673+0000 7f4d9cc00700 1 --2- 172.21.15.151:0/398330233 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e09ae09390 0x55e09ae097f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:48.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.673+0000 7f4d9cc00700 1 -- 172.21.15.151:0/398330233 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e09ae0e7c0 msgr2=0x55e09ae0eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:48.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.673+0000 7f4d9cc00700 1 --2- 172.21.15.151:0/398330233 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e09ae0e7c0 0x55e09ae0eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:48.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.673+0000 7f4d9cc00700 1 -- 172.21.15.151:0/398330233 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e09ad1f420 con 0x55e09ae0f0e0 2024-05-21T21:57:48.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.673+0000 7f4d9d600700 1 --2- 172.21.15.151:0/398330233 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e09ae0e7c0 0x55e09ae0eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:57:48.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.674+0000 7f4d9cc00700 1 --2- 172.21.15.151:0/398330233 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e09ae0f0e0 0x55e09ae08d70 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f4d9806f640 tx=0x7f4d8c009e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=668fdfb0c50fabf7 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:57:48.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.674+0000 7f4d8aa00700 1 -- 172.21.15.151:0/398330233 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d8c004230 con 0x55e09ae0f0e0 2024-05-21T21:57:48.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.674+0000 7f4d8aa00700 1 -- 172.21.15.151:0/398330233 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d8c0043d0 con 0x55e09ae0f0e0 2024-05-21T21:57:48.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.674+0000 7f4d8aa00700 1 -- 172.21.15.151:0/398330233 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d8c00ee80 con 0x55e09ae0f0e0 2024-05-21T21:57:48.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.675+0000 7f4daa934b80 1 -- 172.21.15.151:0/398330233 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e09ae0f0e0 msgr2=0x55e09ae08d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:48.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.675+0000 7f4daa934b80 1 --2- 172.21.15.151:0/398330233 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e09ae0f0e0 0x55e09ae08d70 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f4d9806f640 tx=0x7f4d8c009e00 comp rx=0 tx=0).stop 2024-05-21T21:57:48.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.676+0000 7f4daa934b80 1 -- 172.21.15.151:0/398330233 shutdown_connections 2024-05-21T21:57:48.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.676+0000 7f4daa934b80 1 --2- 172.21.15.151:0/398330233 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e09ae0e7c0 0x55e09ae0eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:48.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.676+0000 7f4daa934b80 1 --2- 172.21.15.151:0/398330233 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e09ae09390 0x55e09ae097f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:48.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.677+0000 7f4daa934b80 1 --2- 172.21.15.151:0/398330233 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e09ae0f0e0 0x55e09ae08d70 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:48.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.677+0000 7f4daa934b80 1 -- 172.21.15.151:0/398330233 >> 172.21.15.151:0/398330233 conn(0x55e09ad36be0 msgr2=0x55e09ae03810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:57:48.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.677+0000 7f4daa934b80 1 -- 172.21.15.151:0/398330233 shutdown_connections 2024-05-21T21:57:48.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.677+0000 7f4daa934b80 1 -- 172.21.15.151:0/398330233 wait complete. 2024-05-21T21:57:48.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.678+0000 7f4daa934b80 1 -- start start 2024-05-21T21:57:48.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.678+0000 7f4daa934b80 1 Processor -- start 2024-05-21T21:57:48.679 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.679+0000 7f4daa934b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e09ae09390 0x55e09ae14dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:48.679 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.679+0000 7f4daa934b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e09ae0e7c0 0x55e09ae12170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:48.679 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.679+0000 7f4daa934b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e09ae0f0e0 0x55e09ae126b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:48.679 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.679+0000 7f4daa934b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e09ac5cfa0 con 0x55e09ae0e7c0 2024-05-21T21:57:48.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.679+0000 7f4daa934b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55e09acf7760 con 0x55e09ae0f0e0 2024-05-21T21:57:48.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.679+0000 7f4daa934b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55e09ae15c00 con 0x55e09ae09390 2024-05-21T21:57:48.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.679+0000 7f4daa934b80 10 client.0 _ll_register_callbacks cb 0x55e09ad00c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:57:48.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.679+0000 7f4d9cc00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e09ae0e7c0 0x55e09ae12170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:48.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.680+0000 7f4d9e000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e09ae0f0e0 0x55e09ae126b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:48.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.680+0000 7f4d9e000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e09ae0f0e0 0x55e09ae126b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:59592/0 (socket says 172.21.15.151:59592) 2024-05-21T21:57:48.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.680+0000 7f4d9e000700 1 -- 172.21.15.151:0/2626626911 learned_addr learned my addr 172.21.15.151:0/2626626911 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:57:48.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.680+0000 7f4d9d600700 1 --2- 172.21.15.151:0/2626626911 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e09ae09390 0x55e09ae14dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:48.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.680+0000 7f4d9cc00700 1 -- 172.21.15.151:0/2626626911 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e09ae09390 msgr2=0x55e09ae14dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:48.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.680+0000 7f4d9cc00700 1 --2- 172.21.15.151:0/2626626911 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e09ae09390 0x55e09ae14dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:48.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.680+0000 7f4d9cc00700 1 -- 172.21.15.151:0/2626626911 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e09ae0f0e0 msgr2=0x55e09ae126b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:48.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.680+0000 7f4d9cc00700 1 --2- 172.21.15.151:0/2626626911 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e09ae0f0e0 0x55e09ae126b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:48.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.680+0000 7f4d9cc00700 1 -- 172.21.15.151:0/2626626911 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e09aca68d0 con 0x55e09ae0e7c0 2024-05-21T21:57:48.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.680+0000 7f4d9e000700 1 --2- 172.21.15.151:0/2626626911 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e09ae0f0e0 0x55e09ae126b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T21:57:48.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.681+0000 7f4d9cc00700 1 --2- 172.21.15.151:0/2626626911 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e09ae0e7c0 0x55e09ae12170 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f4d9806f640 tx=0x7f4d8c004a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8c28d2a367d3e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:57:48.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.681+0000 7f4d7f400700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-21T21:57:48.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.681+0000 7f4d7f400700 1 -- 172.21.15.151:0/2626626911 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d8c004d30 con 0x55e09ae0e7c0 2024-05-21T21:57:48.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.682+0000 7f4d7f400700 1 -- 172.21.15.151:0/2626626911 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f4d8c020040 con 0x55e09ae0e7c0 2024-05-21T21:57:48.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.682+0000 7f4daa934b80 1 -- 172.21.15.151:0/2626626911 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d8c0043d0 con 0x55e09ae0e7c0 2024-05-21T21:57:48.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.682+0000 7f4d7f400700 1 -- 172.21.15.151:0/2626626911 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4d8c005020 con 0x55e09ae0e7c0 2024-05-21T21:57:48.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.682+0000 7f4daa934b80 10 client.15633 fetch_fsmap learned FSMap version 29 2024-05-21T21:57:48.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.682+0000 7f4daa934b80 1 -- 172.21.15.151:0/2626626911 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e09ad27060 con 0x55e09ae0e7c0 2024-05-21T21:57:48.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.683+0000 7f4d7f400700 1 -- 172.21.15.151:0/2626626911 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d8c0043d0 con 0x55e09ae0e7c0 2024-05-21T21:57:48.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.683+0000 7f4d7f400700 1 -- 172.21.15.151:0/2626626911 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f4d8c018700 con 0x55e09ae0e7c0 2024-05-21T21:57:48.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.683+0000 7f4daa934b80 10 client.15633 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T21:57:48.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.683+0000 7f4daa934b80 10 client.15633 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T21:57:48.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.683+0000 7f4daa934b80 20 client.15633 populate_metadata read hostname 'smithi151' 2024-05-21T21:57:48.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.683+0000 7f4daa934b80 1 --2- 172.21.15.151:0/2626626911 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55e09ae2ae40 0x55e09ae2d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:48.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.683+0000 7f4daa934b80 4 client.15633 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T21:57:48.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.683+0000 7f4daa934b80 1 -- 172.21.15.151:0/2626626911 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e09ad68d90 con 0x55e09ae2ae40 2024-05-21T21:57:48.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.684+0000 7f4d9d600700 1 --2- 172.21.15.151:0/2626626911 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55e09ae2ae40 0x55e09ae2d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:48.685 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.685+0000 7f4d9d600700 1 --2- 172.21.15.151:0/2626626911 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55e09ae2ae40 0x55e09ae2d230 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e31ce9ef722e0a6 server_cookie=6ff8d631338ab843 in_seq=0 out_seq=0 2024-05-21T21:57:48.685 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.685+0000 7f4d7f400700 10 client.15633 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T21:57:48.685 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.686+0000 7f4d7f400700 1 -- 172.21.15.151:0/2626626911 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55e09ad68d90 con 0x55e09ae2ae40 2024-05-21T21:57:48.686 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.686+0000 7f4d7f400700 10 client.15633 handle_command_reply: tid=0 2024-05-21T21:57:48.686 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.686+0000 7f4daa934b80 1 client.15633 shutdown 2024-05-21T21:57:48.687 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.686+0000 7f4daa934b80 1 -- 172.21.15.151:0/2626626911 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e09ae0e7c0 msgr2=0x55e09ae12170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:48.687 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.686+0000 7f4daa934b80 1 --2- 172.21.15.151:0/2626626911 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e09ae0e7c0 0x55e09ae12170 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f4d9806f640 tx=0x7f4d8c004a10 comp rx=0 tx=0).stop 2024-05-21T21:57:48.687 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.687+0000 7f4daa934b80 1 -- 172.21.15.151:0/2626626911 shutdown_connections 2024-05-21T21:57:48.687 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.687+0000 7f4daa934b80 1 --2- 172.21.15.151:0/2626626911 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55e09ae2ae40 0x55e09ae2d230 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:48.687 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.687+0000 7f4daa934b80 1 --2- 172.21.15.151:0/2626626911 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e09ae0f0e0 0x55e09ae126b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:48.687 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.687+0000 7f4daa934b80 1 --2- 172.21.15.151:0/2626626911 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e09ae09390 0x55e09ae14dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:48.687 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.687+0000 7f4d7f400700 0 client.15633 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T21:57:48.687 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.687+0000 7f4daa934b80 1 --2- 172.21.15.151:0/2626626911 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e09ae0e7c0 0x55e09ae12170 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:48.687 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.687+0000 7f4daa934b80 1 -- 172.21.15.151:0/2626626911 >> 172.21.15.151:0/2626626911 conn(0x55e09ad36be0 msgr2=0x55e09ae02fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:57:48.687 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.687+0000 7f4daa934b80 1 -- 172.21.15.151:0/2626626911 shutdown_connections 2024-05-21T21:57:48.688 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.688+0000 7f4daa934b80 1 -- 172.21.15.151:0/2626626911 wait complete. 2024-05-21T21:57:48.688 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.688+0000 7f4daa934b80 20 client.15633 trim_cache size 0 max 16384 2024-05-21T21:57:48.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.727+0000 7f4daa934b80 1 Processor -- start 2024-05-21T21:57:48.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.727+0000 7f4daa934b80 1 -- start start 2024-05-21T21:57:48.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.727+0000 7f4daa934b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e09ae09390 0x55e09ae08d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:48.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.727+0000 7f4daa934b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e09ae0e7c0 0x55e09ae112f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:48.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.727+0000 7f4daa934b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e09ae0f0e0 0x55e09ae03740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:48.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.727+0000 7f4daa934b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4d981b4520 con 0x55e09ae0e7c0 2024-05-21T21:57:48.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.727+0000 7f4daa934b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4d981b6060 con 0x55e09ae0f0e0 2024-05-21T21:57:48.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.727+0000 7f4daa934b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4d981b4bf0 con 0x55e09ae09390 2024-05-21T21:57:48.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.728+0000 7f4d9cc00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e09ae0e7c0 0x55e09ae112f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:48.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.728+0000 7f4d9cc00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e09ae0e7c0 0x55e09ae112f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:59206/0 (socket says 172.21.15.151:59206) 2024-05-21T21:57:48.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.728+0000 7f4d9cc00700 1 -- 172.21.15.151:0/1849175031 learned_addr learned my addr 172.21.15.151:0/1849175031 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:57:48.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.728+0000 7f4d9d600700 1 --2- 172.21.15.151:0/1849175031 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e09ae09390 0x55e09ae08d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:48.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.728+0000 7f4d9e000700 1 --2- 172.21.15.151:0/1849175031 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e09ae0f0e0 0x55e09ae03740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:48.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.730+0000 7f4d9cc00700 1 -- 172.21.15.151:0/1849175031 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e09ae09390 msgr2=0x55e09ae08d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:48.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.730+0000 7f4d9cc00700 1 --2- 172.21.15.151:0/1849175031 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e09ae09390 0x55e09ae08d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:48.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.730+0000 7f4d9cc00700 1 -- 172.21.15.151:0/1849175031 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e09ae0f0e0 msgr2=0x55e09ae03740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:48.731 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.730+0000 7f4d9cc00700 1 --2- 172.21.15.151:0/1849175031 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e09ae0f0e0 0x55e09ae03740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:48.731 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.730+0000 7f4d9cc00700 1 -- 172.21.15.151:0/1849175031 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e09ae14a10 con 0x55e09ae0e7c0 2024-05-21T21:57:48.731 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.730+0000 7f4d9cc00700 1 --2- 172.21.15.151:0/1849175031 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e09ae0e7c0 0x55e09ae112f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c018b00 tx=0x7f4d8c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce7155912b8f5329 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:57:48.731 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.731+0000 7f4d8a000700 1 -- 172.21.15.151:0/1849175031 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d8c00e7b0 con 0x55e09ae0e7c0 2024-05-21T21:57:48.732 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.731+0000 7f4d8a000700 1 -- 172.21.15.151:0/1849175031 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d8c009350 con 0x55e09ae0e7c0 2024-05-21T21:57:48.732 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.731+0000 7f4d8a000700 1 -- 172.21.15.151:0/1849175031 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d8c005450 con 0x55e09ae0e7c0 2024-05-21T21:57:48.732 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.732+0000 7f4daa934b80 1 -- 172.21.15.151:0/1849175031 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e09ae0e7c0 msgr2=0x55e09ae112f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:48.732 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.732+0000 7f4daa934b80 1 --2- 172.21.15.151:0/1849175031 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e09ae0e7c0 0x55e09ae112f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c018b00 tx=0x7f4d8c0046b0 comp rx=0 tx=0).stop 2024-05-21T21:57:48.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.732+0000 7f4daa934b80 1 -- 172.21.15.151:0/1849175031 shutdown_connections 2024-05-21T21:57:48.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.732+0000 7f4daa934b80 1 --2- 172.21.15.151:0/1849175031 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e09ae0f0e0 0x55e09ae03740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:48.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.732+0000 7f4daa934b80 1 --2- 172.21.15.151:0/1849175031 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e09ae09390 0x55e09ae08d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:48.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.732+0000 7f4daa934b80 1 --2- 172.21.15.151:0/1849175031 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e09ae0e7c0 0x55e09ae112f0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:48.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.732+0000 7f4daa934b80 1 -- 172.21.15.151:0/1849175031 >> 172.21.15.151:0/1849175031 conn(0x55e09ad36be0 msgr2=0x55e09ae0fbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:57:48.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.732+0000 7f4daa934b80 1 -- 172.21.15.151:0/1849175031 shutdown_connections 2024-05-21T21:57:48.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.732+0000 7f4daa934b80 1 -- 172.21.15.151:0/1849175031 wait complete. 2024-05-21T21:57:48.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.733+0000 7f4daa934b80 1 -- start start 2024-05-21T21:57:48.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.733+0000 7f4daa934b80 1 Processor -- start 2024-05-21T21:57:48.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.733+0000 7f4daa934b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e09ae09390 0x55e09ae1ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:48.735 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.733+0000 7f4daa934b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e09ae0e7c0 0x55e09ae1b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:48.735 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.734+0000 7f4daa934b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e09ae0f0e0 0x55e09ae17e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:48.735 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.734+0000 7f4daa934b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4d981b5990 con 0x55e09ae0e7c0 2024-05-21T21:57:48.735 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.734+0000 7f4daa934b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4d981b52c0 con 0x55e09ae0f0e0 2024-05-21T21:57:48.735 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.734+0000 7f4daa934b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4d981b24c0 con 0x55e09ae09390 2024-05-21T21:57:48.735 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.734+0000 7f4daa934b80 10 client.0 _ll_register_callbacks cb 0x55e09ad00c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:57:48.735 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.734+0000 7f4d9cc00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e09ae0e7c0 0x55e09ae1b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:48.735 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.734+0000 7f4d9e000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e09ae0f0e0 0x55e09ae17e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:48.735 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.734+0000 7f4d9d600700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e09ae09390 0x55e09ae1ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:48.736 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.734+0000 7f4d9e000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e09ae0f0e0 0x55e09ae17e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:59606/0 (socket says 172.21.15.151:59606) 2024-05-21T21:57:48.736 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.734+0000 7f4d9e000700 1 -- 172.21.15.151:0/1926277309 learned_addr learned my addr 172.21.15.151:0/1926277309 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:57:48.736 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.734+0000 7f4d9e000700 1 -- 172.21.15.151:0/1926277309 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e09ae09390 msgr2=0x55e09ae1ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:48.736 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.734+0000 7f4d9e000700 1 --2- 172.21.15.151:0/1926277309 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e09ae09390 0x55e09ae1ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:48.736 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.734+0000 7f4d9e000700 1 -- 172.21.15.151:0/1926277309 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e09ae0e7c0 msgr2=0x55e09ae1b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:48.736 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.734+0000 7f4d9e000700 1 --2- 172.21.15.151:0/1926277309 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e09ae0e7c0 0x55e09ae1b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:48.736 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.734+0000 7f4d9e000700 1 -- 172.21.15.151:0/1926277309 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e09ae18530 con 0x55e09ae0f0e0 2024-05-21T21:57:48.736 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.735+0000 7f4d9e000700 1 --2- 172.21.15.151:0/1926277309 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e09ae0f0e0 0x55e09ae17e60 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4d900a51a0 tx=0x7f4d90007870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f503edf0d5e8e820 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:57:48.737 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.735+0000 7f4d7f400700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-21T21:57:48.737 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.735+0000 7f4d7f400700 1 -- 172.21.15.151:0/1926277309 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d900c3060 con 0x55e09ae0f0e0 2024-05-21T21:57:48.737 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.735+0000 7f4d7f400700 1 -- 172.21.15.151:0/1926277309 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f4d900b1ab0 con 0x55e09ae0f0e0 2024-05-21T21:57:48.737 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.735+0000 7f4daa934b80 1 -- 172.21.15.151:0/1926277309 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d8c009350 con 0x55e09ae0f0e0 2024-05-21T21:57:48.737 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.736+0000 7f4d7f400700 1 -- 172.21.15.151:0/1926277309 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4d900c8050 con 0x55e09ae0f0e0 2024-05-21T21:57:48.737 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.736+0000 7f4daa934b80 10 client.25769 fetch_fsmap learned FSMap version 29 2024-05-21T21:57:48.737 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.736+0000 7f4daa934b80 1 -- 172.21.15.151:0/1926277309 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e09ae961a0 con 0x55e09ae0f0e0 2024-05-21T21:57:48.737 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.736+0000 7f4d7f400700 1 -- 172.21.15.151:0/1926277309 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d8c009350 con 0x55e09ae0f0e0 2024-05-21T21:57:48.737 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.737+0000 7f4d7f400700 1 -- 172.21.15.151:0/1926277309 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f4d900a4ae0 con 0x55e09ae0f0e0 2024-05-21T21:57:48.738 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.737+0000 7f4daa934b80 10 client.25769 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T21:57:48.738 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.737+0000 7f4daa934b80 10 client.25769 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T21:57:48.738 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.737+0000 7f4daa934b80 20 client.25769 populate_metadata read hostname 'smithi151' 2024-05-21T21:57:48.738 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.737+0000 7f4daa934b80 1 --2- 172.21.15.151:0/1926277309 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55e09ae96850 0x55e09ae98c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:48.738 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.737+0000 7f4daa934b80 4 client.25769 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-21T21:57:48.738 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.737+0000 7f4daa934b80 1 -- 172.21.15.151:0/1926277309 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e09ad62570 con 0x55e09ae96850 2024-05-21T21:57:48.738 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.738+0000 7f4d9d600700 1 --2- 172.21.15.151:0/1926277309 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55e09ae96850 0x55e09ae98c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:48.739 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.738+0000 7f4d9d600700 1 --2- 172.21.15.151:0/1926277309 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55e09ae96850 0x55e09ae98c70 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2d17780c0b70770 server_cookie=9d43f9403e347b01 in_seq=0 out_seq=0 2024-05-21T21:57:48.739 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.738+0000 7f4d7f400700 10 client.25769 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T21:57:48.739 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.738+0000 7f4d7f400700 1 -- 172.21.15.151:0/1926277309 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55e09ad62570 con 0x55e09ae96850 2024-05-21T21:57:48.739 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.738+0000 7f4d7f400700 10 client.25769 handle_command_reply: tid=0 2024-05-21T21:57:48.739 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.739+0000 7f4daa934b80 1 client.25769 shutdown 2024-05-21T21:57:48.740 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.739+0000 7f4daa934b80 1 -- 172.21.15.151:0/1926277309 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e09ae0f0e0 msgr2=0x55e09ae17e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:48.740 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.739+0000 7f4daa934b80 1 --2- 172.21.15.151:0/1926277309 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e09ae0f0e0 0x55e09ae17e60 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4d900a51a0 tx=0x7f4d90007870 comp rx=0 tx=0).stop 2024-05-21T21:57:48.740 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.739+0000 7f4daa934b80 1 -- 172.21.15.151:0/1926277309 shutdown_connections 2024-05-21T21:57:48.740 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.739+0000 7f4daa934b80 1 --2- 172.21.15.151:0/1926277309 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55e09ae96850 0x55e09ae98c70 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:48.740 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.739+0000 7f4d7f400700 0 client.25769 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T21:57:48.740 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.739+0000 7f4daa934b80 1 --2- 172.21.15.151:0/1926277309 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e09ae0f0e0 0x55e09ae17e60 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:48.740 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.739+0000 7f4daa934b80 1 --2- 172.21.15.151:0/1926277309 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e09ae09390 0x55e09ae1ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:48.740 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.739+0000 7f4daa934b80 1 --2- 172.21.15.151:0/1926277309 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e09ae0e7c0 0x55e09ae1b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:48.740 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.739+0000 7f4daa934b80 1 -- 172.21.15.151:0/1926277309 >> 172.21.15.151:0/1926277309 conn(0x55e09ad36be0 msgr2=0x55e09ae13740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:57:48.740 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.739+0000 7f4daa934b80 1 -- 172.21.15.151:0/1926277309 shutdown_connections 2024-05-21T21:57:48.740 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.739+0000 7f4daa934b80 1 -- 172.21.15.151:0/1926277309 wait complete. 2024-05-21T21:57:48.740 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.740+0000 7f4daa934b80 20 client.25769 trim_cache size 0 max 16384 2024-05-21T21:57:48.741 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-05-21T21:57:48.741 INFO:teuthology.orchestra.run.smithi151.stdout: "status": "no active scrubs running", 2024-05-21T21:57:48.741 INFO:teuthology.orchestra.run.smithi151.stdout: "scrubs": {} 2024-05-21T21:57:48.741 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-05-21T21:57:48.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.740+0000 7f4d9ea00700 1 -- 172.21.15.151:0/11772867 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f4d800b2a70 msgr2=0x7f4d800b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:48.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.740+0000 7f4d9ea00700 1 --2- 172.21.15.151:0/11772867 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f4d800b2a70 0x7f4d800b4f30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4d840098c0 tx=0x7f4d8401d040 comp rx=0 tx=0).stop 2024-05-21T21:57:48.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.741+0000 7f4d9ea00700 1 -- 172.21.15.151:0/11772867 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4d9810da60 msgr2=0x7f4d9806e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:48.742 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.741+0000 7f4d9ea00700 1 --2- 172.21.15.151:0/11772867 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4d9810da60 0x7f4d9806e3d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f4d9000e3c0 tx=0x7f4d9000e6d0 comp rx=0 tx=0).stop 2024-05-21T21:57:48.742 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.741+0000 7f4d9ea00700 1 -- 172.21.15.151:0/11772867 shutdown_connections 2024-05-21T21:57:48.742 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.741+0000 7f4d9ea00700 1 --2- 172.21.15.151:0/11772867 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f4d800b2a70 0x7f4d800b4f30 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:48.742 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.741+0000 7f4d9ea00700 1 --2- 172.21.15.151:0/11772867 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4d98104630 0x7f4d98074d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:48.742 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.741+0000 7f4d9ea00700 1 --2- 172.21.15.151:0/11772867 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4d9810da60 0x7f4d9806e3d0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:48.742 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.741+0000 7f4d9ea00700 1 --2- 172.21.15.151:0/11772867 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4d98104f50 0x7f4d9806de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:48.742 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.741+0000 7f4d9ea00700 1 -- 172.21.15.151:0/11772867 >> 172.21.15.151:0/11772867 conn(0x7f4d980fde60 msgr2=0x7f4d980fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:57:48.742 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.742+0000 7f4d9ea00700 1 -- 172.21.15.151:0/11772867 shutdown_connections 2024-05-21T21:57:48.742 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:48.742+0000 7f4d9ea00700 1 -- 172.21.15.151:0/11772867 wait complete. 2024-05-21T21:57:49.702 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:57:49 smithi151 ceph-mon[31623]: pgmap v69: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-05-21T21:57:49.735 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:57:49 smithi184 ceph-mon[36942]: pgmap v69: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-05-21T21:57:49.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:57:49 smithi164 ceph-mon[36269]: pgmap v69: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-05-21T21:57:50.757 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-21T21:57:50.757 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-21T21:57:51.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:51.344+0000 7f4a55200700 1 -- 172.21.15.151:0/1756768552 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4a50076200 msgr2=0x7f4a50113f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:51.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:51.344+0000 7f4a55200700 1 --2- 172.21.15.151:0/1756768552 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4a50076200 0x7f4a50113f40 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c009990 tx=0x7f4a3c009ca0 comp rx=0 tx=0).stop 2024-05-21T21:57:51.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:51.344+0000 7f4a55200700 1 -- 172.21.15.151:0/1756768552 shutdown_connections 2024-05-21T21:57:51.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:51.344+0000 7f4a55200700 1 --2- 172.21.15.151:0/1756768552 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a50114480 0x7f4a50116870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:51.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:51.344+0000 7f4a55200700 1 --2- 172.21.15.151:0/1756768552 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4a500758e0 0x7f4a50075cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:51.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:51.344+0000 7f4a55200700 1 --2- 172.21.15.151:0/1756768552 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4a50076200 0x7f4a50113f40 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:51.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:51.344+0000 7f4a55200700 1 -- 172.21.15.151:0/1756768552 >> 172.21.15.151:0/1756768552 conn(0x7f4a500fefc0 msgr2=0x7f4a501013e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:57:51.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:51.344+0000 7f4a55200700 1 -- 172.21.15.151:0/1756768552 shutdown_connections 2024-05-21T21:57:51.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:51.344+0000 7f4a55200700 1 -- 172.21.15.151:0/1756768552 wait complete. 2024-05-21T21:57:51.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:51.345+0000 7f4a55200700 1 Processor -- start 2024-05-21T21:57:51.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:51.345+0000 7f4a55200700 1 -- start start 2024-05-21T21:57:51.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:51.346+0000 7f4a55200700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a500758e0 0x7f4a501a01c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:51.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:51.346+0000 7f4a55200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4a50076200 0x7f4a501a0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:51.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:51.346+0000 7f4a55200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4a50114480 0x7f4a501a4a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:51.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:51.346+0000 7f4a55200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4a50119cd0 con 0x7f4a50114480 2024-05-21T21:57:51.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:51.346+0000 7f4a55200700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4a50119b50 con 0x7f4a50076200 2024-05-21T21:57:51.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:51.346+0000 7f4a55200700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4a50119e50 con 0x7f4a500758e0 2024-05-21T21:57:51.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:51.346+0000 7f4a4ea00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4a50076200 0x7f4a501a0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:51.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:51.346+0000 7f4a4ea00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4a50076200 0x7f4a501a0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:59614/0 (socket says 172.21.15.151:59614) 2024-05-21T21:57:51.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:51.346+0000 7f4a4ea00700 1 -- 172.21.15.151:0/1002498484 learned_addr learned my addr 172.21.15.151:0/1002498484 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:57:51.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:51.347+0000 7f4a4fe00700 1 --2- 172.21.15.151:0/1002498484 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4a50114480 0x7f4a501a4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:51.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:51.347+0000 7f4a4ea00700 1 -- 172.21.15.151:0/1002498484 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a500758e0 msgr2=0x7f4a501a01c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-21T21:57:51.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:51.347+0000 7f4a4ea00700 1 --2- 172.21.15.151:0/1002498484 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a500758e0 0x7f4a501a01c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:51.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:51.347+0000 7f4a4ea00700 1 -- 172.21.15.151:0/1002498484 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4a50114480 msgr2=0x7f4a501a4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:51.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:51.347+0000 7f4a4ea00700 1 --2- 172.21.15.151:0/1002498484 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4a50114480 0x7f4a501a4a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:51.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:51.347+0000 7f4a4ea00700 1 -- 172.21.15.151:0/1002498484 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a501a51d0 con 0x7f4a50076200 2024-05-21T21:57:51.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:51.347+0000 7f4a4fe00700 1 --2- 172.21.15.151:0/1002498484 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4a50114480 0x7f4a501a4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T21:57:51.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:51.347+0000 7f4a4ea00700 1 --2- 172.21.15.151:0/1002498484 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4a50076200 0x7f4a501a0700 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c005b40 tx=0x7f4a3c005da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:57:51.350 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:51.348+0000 7f4a47e00700 1 -- 172.21.15.151:0/1002498484 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a3c026070 con 0x7f4a50076200 2024-05-21T21:57:51.350 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:51.348+0000 7f4a47e00700 1 -- 172.21.15.151:0/1002498484 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a3c021410 con 0x7f4a50076200 2024-05-21T21:57:51.350 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:51.348+0000 7f4a47e00700 1 -- 172.21.15.151:0/1002498484 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a3c017650 con 0x7f4a50076200 2024-05-21T21:57:51.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:51.348+0000 7f4a55200700 1 -- 172.21.15.151:0/1002498484 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a501a5460 con 0x7f4a50076200 2024-05-21T21:57:51.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:51.348+0000 7f4a55200700 1 -- 172.21.15.151:0/1002498484 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a501acd40 con 0x7f4a50076200 2024-05-21T21:57:51.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:51.349+0000 7f4a55200700 1 -- 172.21.15.151:0/1002498484 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a5006a230 con 0x7f4a50076200 2024-05-21T21:57:51.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:51.352+0000 7f4a47e00700 1 -- 172.21.15.151:0/1002498484 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7f4a3c029070 con 0x7f4a50076200 2024-05-21T21:57:51.354 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:51.354+0000 7f4a47e00700 1 --2- 172.21.15.151:0/1002498484 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f4a340b2af0 0x7f4a340b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:51.354 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:51.354+0000 7f4a47e00700 1 -- 172.21.15.151:0/1002498484 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f4a3c0e3480 con 0x7f4a50076200 2024-05-21T21:57:51.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:51.355+0000 7f4a4f400700 1 --2- 172.21.15.151:0/1002498484 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f4a340b2af0 0x7f4a340b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:51.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:51.356+0000 7f4a4f400700 1 --2- 172.21.15.151:0/1002498484 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f4a340b2af0 0x7f4a340b4fb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4a40009730 tx=0x7f4a40006cb0 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:57:51.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:51.358+0000 7f4a47e00700 1 -- 172.21.15.151:0/1002498484 <== mon.2 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4a3c013070 con 0x7f4a50076200 2024-05-21T21:57:51.395 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:57:51.567 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:57:51 smithi151 ceph-mon[31623]: pgmap v70: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-05-21T21:57:51.652 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:51.652+0000 7f4a55200700 1 -- 172.21.15.151:0/1002498484 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4a5002e560 con 0x7f4a50076200 2024-05-21T21:57:51.653 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:51.653+0000 7f4a47e00700 1 -- 172.21.15.151:0/1002498484 <== mon.2 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11958 (secure 0 0 0) 0x7f4a3c0aaf30 con 0x7f4a50076200 2024-05-21T21:57:51.654 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T21:57:51.654 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14358,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/158010529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":158010529},{"type":"v1","addr":"172.21.15.151:6835","nonce":158010529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6839/2069058115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":2069058115},{"type":"v1","addr":"172.21.15.151:6839","nonce":2069058115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T21:51:19.552069+0000","modified":"2024-05-21T21:55:34.643755+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24530,"mds_1":24493,"mds_2":24491,"mds_3":24478,"mds_4":24473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14403":{"gid":14403,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6841/1749723794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6840","nonce":1749723794},{"type":"v1","addr":"172.21.15.151:6841","nonce":1749723794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/2026266114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":2026266114},{"type":"v1","addr":"172.21.15.184:6833","nonce":2026266114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/2467900844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":2467900844},{"type":"v1","addr":"172.21.15.164:6833","nonce":2467900844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3367640960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3367640960},{"type":"v1","addr":"172.21.15.151:6837","nonce":3367640960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/3264142621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3264142621},{"type":"v1","addr":"172.21.15.184:6835","nonce":3264142621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":17,"addr":"172.21.15.164:6835/2340428470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2340428470},{"type":"v1","addr":"172.21.15.164:6835","nonce":2340428470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":16,"addr":"172.21.15.184:6837/2988194888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":2988194888},{"type":"v1","addr":"172.21.15.184:6837","nonce":2988194888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.164:6837/1554609572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1554609572},{"type":"v1","addr":"172.21.15.164:6837","nonce":1554609572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.184:6839/3059935143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":3059935143},{"type":"v1","addr":"172.21.15.184:6839","nonce":3059935143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.164:6839/2646401247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":2646401247},{"type":"v1","addr":"172.21.15.164:6839","nonce":2646401247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24530,"qdb_cluster":[24530,24493,24491,24473,24478]},"id":1}]} 2024-05-21T21:57:51.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:51.657+0000 7f4a55200700 1 -- 172.21.15.151:0/1002498484 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f4a340b2af0 msgr2=0x7f4a340b4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:51.658 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:51.657+0000 7f4a55200700 1 --2- 172.21.15.151:0/1002498484 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f4a340b2af0 0x7f4a340b4fb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4a40009730 tx=0x7f4a40006cb0 comp rx=0 tx=0).stop 2024-05-21T21:57:51.658 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:51.657+0000 7f4a55200700 1 -- 172.21.15.151:0/1002498484 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4a50076200 msgr2=0x7f4a501a0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:51.658 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:51.657+0000 7f4a55200700 1 --2- 172.21.15.151:0/1002498484 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4a50076200 0x7f4a501a0700 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c005b40 tx=0x7f4a3c005da0 comp rx=0 tx=0).stop 2024-05-21T21:57:51.658 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:51.658+0000 7f4a55200700 1 -- 172.21.15.151:0/1002498484 shutdown_connections 2024-05-21T21:57:51.658 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:51.658+0000 7f4a55200700 1 --2- 172.21.15.151:0/1002498484 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f4a340b2af0 0x7f4a340b4fb0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:51.659 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:51.658+0000 7f4a55200700 1 --2- 172.21.15.151:0/1002498484 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4a50114480 0x7f4a501a4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:51.659 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:51.658+0000 7f4a55200700 1 --2- 172.21.15.151:0/1002498484 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a500758e0 0x7f4a501a01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:51.659 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:51.658+0000 7f4a55200700 1 --2- 172.21.15.151:0/1002498484 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4a50076200 0x7f4a501a0700 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:51.659 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:51.658+0000 7f4a55200700 1 -- 172.21.15.151:0/1002498484 >> 172.21.15.151:0/1002498484 conn(0x7f4a500fefc0 msgr2=0x7f4a50100820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:57:51.659 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:51.658+0000 7f4a55200700 1 -- 172.21.15.151:0/1002498484 shutdown_connections 2024-05-21T21:57:51.659 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:51.658+0000 7f4a55200700 1 -- 172.21.15.151:0/1002498484 wait complete. 2024-05-21T21:57:51.660 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 29 2024-05-21T21:57:51.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:57:51 smithi184 ceph-mon[36942]: pgmap v70: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-05-21T21:57:51.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:57:51 smithi164 ceph-mon[36269]: pgmap v70: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-05-21T21:57:52.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:57:52 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/1002498484' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T21:57:52.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:57:52 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/1002498484' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T21:57:52.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:57:52 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/1002498484' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T21:57:52.861 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24530 quiesce db / --timeout 90 --expiration 146.15 --await-for 45.0' 2024-05-21T21:57:52.862 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24530 quiesce db / --timeout 90 --expiration 146.15 --await-for 45.0 2024-05-21T21:57:53.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:57:53 smithi151 ceph-mon[31623]: pgmap v71: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-05-21T21:57:53.735 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:57:53 smithi184 ceph-mon[36942]: pgmap v71: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-05-21T21:57:53.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:57:53 smithi164 ceph-mon[36269]: pgmap v71: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-05-21T21:57:54.276 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:57:54.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:54.655+0000 7f634a200700 1 -- 172.21.15.151:0/1188894612 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6344075770 msgr2=0x7f6344075b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:54.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:54.655+0000 7f634a200700 1 --2- 172.21.15.151:0/1188894612 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6344075770 0x7f6344075b50 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f6334009990 tx=0x7f6334009ca0 comp rx=0 tx=0).stop 2024-05-21T21:57:54.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:54.656+0000 7f634a200700 1 -- 172.21.15.151:0/1188894612 shutdown_connections 2024-05-21T21:57:54.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:54.656+0000 7f634a200700 1 --2- 172.21.15.151:0/1188894612 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f634410ff40 0x7f6344112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:54.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:54.656+0000 7f634a200700 1 --2- 172.21.15.151:0/1188894612 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6344075770 0x7f6344075b50 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:54.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:54.656+0000 7f634a200700 1 --2- 172.21.15.151:0/1188894612 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6344076090 0x7f634410fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:54.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:54.656+0000 7f634a200700 1 -- 172.21.15.151:0/1188894612 >> 172.21.15.151:0/1188894612 conn(0x7f634407a6a0 msgr2=0x7f634407aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:57:54.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:54.656+0000 7f634a200700 1 -- 172.21.15.151:0/1188894612 shutdown_connections 2024-05-21T21:57:54.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:54.656+0000 7f634a200700 1 -- 172.21.15.151:0/1188894612 wait complete. 2024-05-21T21:57:54.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:54.658+0000 7f634a200700 1 Processor -- start 2024-05-21T21:57:54.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:54.658+0000 7f634a200700 1 -- start start 2024-05-21T21:57:54.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:54.658+0000 7f634a200700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6344075770 0x7f634419beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:54.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:54.658+0000 7f634a200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6344076090 0x7f634419c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:54.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:54.658+0000 7f634a200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f634410ff40 0x7f63441a0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:54.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:54.658+0000 7f634a200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6344115940 con 0x7f634410ff40 2024-05-21T21:57:54.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:54.658+0000 7f634a200700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f63441157c0 con 0x7f6344076090 2024-05-21T21:57:54.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:54.658+0000 7f634a200700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6344115ac0 con 0x7f6344075770 2024-05-21T21:57:54.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:54.659+0000 7f6342a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6344076090 0x7f634419c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:54.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:54.659+0000 7f6343400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6344075770 0x7f634419beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:54.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:54.659+0000 7f6342a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6344076090 0x7f634419c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:59628/0 (socket says 172.21.15.151:59628) 2024-05-21T21:57:54.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:54.659+0000 7f6342a00700 1 -- 172.21.15.151:0/2748955485 learned_addr learned my addr 172.21.15.151:0/2748955485 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:57:54.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:54.659+0000 7f6343e00700 1 --2- 172.21.15.151:0/2748955485 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f634410ff40 0x7f63441a0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:54.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:54.659+0000 7f6343400700 1 -- 172.21.15.151:0/2748955485 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6344076090 msgr2=0x7f634419c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:54.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:54.659+0000 7f6343400700 1 --2- 172.21.15.151:0/2748955485 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6344076090 0x7f634419c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:54.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:54.659+0000 7f6343400700 1 -- 172.21.15.151:0/2748955485 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f634410ff40 msgr2=0x7f63441a0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:54.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:54.659+0000 7f6343400700 1 --2- 172.21.15.151:0/2748955485 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f634410ff40 0x7f63441a0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:54.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:54.659+0000 7f6343400700 1 -- 172.21.15.151:0/2748955485 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63441a0e70 con 0x7f6344075770 2024-05-21T21:57:54.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:54.659+0000 7f6343400700 1 --2- 172.21.15.151:0/2748955485 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6344075770 0x7f634419beb0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6334000c00 tx=0x7f63340047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:57:54.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:54.660+0000 7f633be00700 1 -- 172.21.15.151:0/2748955485 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6334026070 con 0x7f6344075770 2024-05-21T21:57:54.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:54.660+0000 7f633be00700 1 -- 172.21.15.151:0/2748955485 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6334021410 con 0x7f6344075770 2024-05-21T21:57:54.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:54.660+0000 7f634a200700 1 -- 172.21.15.151:0/2748955485 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63441a1100 con 0x7f6344075770 2024-05-21T21:57:54.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:54.660+0000 7f633be00700 1 -- 172.21.15.151:0/2748955485 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6334017650 con 0x7f6344075770 2024-05-21T21:57:54.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:54.660+0000 7f634a200700 1 -- 172.21.15.151:0/2748955485 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63441a89e0 con 0x7f6344075770 2024-05-21T21:57:54.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:54.661+0000 7f634a200700 1 -- 172.21.15.151:0/2748955485 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6344076e60 con 0x7f6344075770 2024-05-21T21:57:54.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:54.663+0000 7f633be00700 1 -- 172.21.15.151:0/2748955485 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7f6334029070 con 0x7f6344075770 2024-05-21T21:57:54.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:54.666+0000 7f633be00700 1 --2- 172.21.15.151:0/2748955485 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f63280b29f0 0x7f63280b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:54.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:54.666+0000 7f633be00700 1 -- 172.21.15.151:0/2748955485 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f63340e3400 con 0x7f6344075770 2024-05-21T21:57:54.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:54.666+0000 7f6342a00700 1 --2- 172.21.15.151:0/2748955485 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f63280b29f0 0x7f63280b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:54.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:54.667+0000 7f6342a00700 1 --2- 172.21.15.151:0/2748955485 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f63280b29f0 0x7f63280b4eb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6330009680 tx=0x7f6330009260 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:57:54.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:54.669+0000 7f633be00700 1 -- 172.21.15.151:0/2748955485 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6334013070 con 0x7f6344075770 2024-05-21T21:57:54.962 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:54.962+0000 7f634a200700 1 -- 172.21.15.151:0/2748955485 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f634402e560 con 0x7f6344075770 2024-05-21T21:57:54.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:54.963+0000 7f633be00700 1 -- 172.21.15.151:0/2748955485 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11958 (secure 0 0 0) 0x7f63340aae20 con 0x7f6344075770 2024-05-21T21:57:54.964 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T21:57:54.964 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14358,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/158010529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":158010529},{"type":"v1","addr":"172.21.15.151:6835","nonce":158010529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6839/2069058115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":2069058115},{"type":"v1","addr":"172.21.15.151:6839","nonce":2069058115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T21:51:19.552069+0000","modified":"2024-05-21T21:55:34.643755+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24530,"mds_1":24493,"mds_2":24491,"mds_3":24478,"mds_4":24473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14403":{"gid":14403,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6841/1749723794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6840","nonce":1749723794},{"type":"v1","addr":"172.21.15.151:6841","nonce":1749723794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/2026266114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":2026266114},{"type":"v1","addr":"172.21.15.184:6833","nonce":2026266114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/2467900844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":2467900844},{"type":"v1","addr":"172.21.15.164:6833","nonce":2467900844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3367640960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3367640960},{"type":"v1","addr":"172.21.15.151:6837","nonce":3367640960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/3264142621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3264142621},{"type":"v1","addr":"172.21.15.184:6835","nonce":3264142621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":17,"addr":"172.21.15.164:6835/2340428470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2340428470},{"type":"v1","addr":"172.21.15.164:6835","nonce":2340428470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":16,"addr":"172.21.15.184:6837/2988194888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":2988194888},{"type":"v1","addr":"172.21.15.184:6837","nonce":2988194888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.164:6837/1554609572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1554609572},{"type":"v1","addr":"172.21.15.164:6837","nonce":1554609572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.184:6839/3059935143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":3059935143},{"type":"v1","addr":"172.21.15.184:6839","nonce":3059935143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.164:6839/2646401247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":2646401247},{"type":"v1","addr":"172.21.15.164:6839","nonce":2646401247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24530,"qdb_cluster":[24530,24493,24491,24473,24478]},"id":1}]} 2024-05-21T21:57:54.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:54.966+0000 7f634a200700 1 -- 172.21.15.151:0/2748955485 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f63280b29f0 msgr2=0x7f63280b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:54.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:54.966+0000 7f634a200700 1 --2- 172.21.15.151:0/2748955485 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f63280b29f0 0x7f63280b4eb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6330009680 tx=0x7f6330009260 comp rx=0 tx=0).stop 2024-05-21T21:57:54.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:54.966+0000 7f634a200700 1 -- 172.21.15.151:0/2748955485 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6344075770 msgr2=0x7f634419beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:54.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:54.966+0000 7f634a200700 1 --2- 172.21.15.151:0/2748955485 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6344075770 0x7f634419beb0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6334000c00 tx=0x7f63340047b0 comp rx=0 tx=0).stop 2024-05-21T21:57:54.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:54.967+0000 7f634a200700 1 -- 172.21.15.151:0/2748955485 shutdown_connections 2024-05-21T21:57:54.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:54.967+0000 7f634a200700 1 --2- 172.21.15.151:0/2748955485 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f63280b29f0 0x7f63280b4eb0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:54.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:54.967+0000 7f634a200700 1 --2- 172.21.15.151:0/2748955485 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f634410ff40 0x7f63441a0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:54.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:54.967+0000 7f634a200700 1 --2- 172.21.15.151:0/2748955485 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6344075770 0x7f634419beb0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:54.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:54.967+0000 7f634a200700 1 --2- 172.21.15.151:0/2748955485 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6344076090 0x7f634419c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:54.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:54.967+0000 7f634a200700 1 -- 172.21.15.151:0/2748955485 >> 172.21.15.151:0/2748955485 conn(0x7f634407a6a0 msgr2=0x7f6344101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:57:54.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:54.967+0000 7f634a200700 1 -- 172.21.15.151:0/2748955485 shutdown_connections 2024-05-21T21:57:54.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:54.967+0000 7f634a200700 1 -- 172.21.15.151:0/2748955485 wait complete. 2024-05-21T21:57:54.968 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 29 2024-05-21T21:57:55.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:57:54 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:57:55.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:57:54 smithi151 ceph-mon[31623]: pgmap v72: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-05-21T21:57:55.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:57:54 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/2748955485' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T21:57:55.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:57:54 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:57:55.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:57:54 smithi184 ceph-mon[36942]: pgmap v72: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-05-21T21:57:55.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:57:54 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/2748955485' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T21:57:55.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:57:54 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:57:55.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:57:54 smithi164 ceph-mon[36269]: pgmap v72: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-05-21T21:57:55.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:57:54 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/2748955485' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T21:57:56.179 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:57:56 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:57:56.179 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:57:56 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:57:56.179 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:57:56 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T21:57:56.227 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-21T21:57:56.485 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:57:56 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:57:56.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:57:56 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:57:56.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:57:56 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T21:57:56.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:57:56 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:57:56.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:57:56 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:57:56.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:57:56 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T21:57:56.677 INFO:tasks.workunit.client.0.smithi151.stderr:Updating files: 57% (7051/12271) Updating files: 58% (7118/12271) Updating files: 59% (7240/12271) Updating files: 60% (7363/12271) Updating files: 61% (7486/12271) Updating files: 62% (7609/12271) Updating files: 63% (7731/12271) Updating files: 64% (7854/12271) Updating files: 65% (7977/12271) Updating files: 66% (8099/12271) Updating files: 67% (8222/12271) Updating files: 68% (8345/12271) Updating files: 69% (8467/12271) Updating files: 70% (8590/12271) Updating files: 71% (8713/12271) Updating files: 72% (8836/12271) Updating files: 73% (8958/12271) Updating files: 74% (9081/12271) Updating files: 75% (9204/12271) Updating files: 76% (9326/12271) Updating files: 77% (9449/12271) Updating files: 78% (9572/12271) Updating files: 79% (9695/12271) Updating files: 80% (9817/12271) Updating files: 81% (9940/12271) Updating files: 82% (10063/12271) Updating files: 83% (10185/12271) Updating files: 84% (10308/12271) Updating files: 85% (10431/12271) Updating files: 86% (10554/12271) Updating files: 87% (10676/12271) Updating files: 88% (10799/12271) Updating files: 89% (10922/12271) Updating files: 90% (11044/12271) Updating files: 91% (11167/12271) Updating files: 92% (11290/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 94% (11544/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-05-21T21:57:57.381 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:57:57 smithi151 ceph-mon[31623]: pgmap v73: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-05-21T21:57:57.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:57:57 smithi184 ceph-mon[36942]: pgmap v73: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-05-21T21:57:57.505 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:57:57.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:57:57 smithi164 ceph-mon[36269]: pgmap v73: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-05-21T21:57:57.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.555+0000 7fcfe8c00700 1 -- 172.21.15.151:0/4150117305 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcfe41084f0 msgr2=0x7fcfe4105fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:57.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.555+0000 7fcfe8c00700 1 --2- 172.21.15.151:0/4150117305 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcfe41084f0 0x7fcfe4105fa0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fcfd4009990 tx=0x7fcfd4009ca0 comp rx=0 tx=0).stop 2024-05-21T21:57:57.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.555+0000 7fcfe8c00700 1 -- 172.21.15.151:0/4150117305 shutdown_connections 2024-05-21T21:57:57.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.555+0000 7fcfe8c00700 1 --2- 172.21.15.151:0/4150117305 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcfe406b240 0x7fcfe406b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:57.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.555+0000 7fcfe8c00700 1 --2- 172.21.15.151:0/4150117305 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcfe41084f0 0x7fcfe4105fa0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:57.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.555+0000 7fcfe8c00700 1 --2- 172.21.15.151:0/4150117305 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcfe41064e0 0x7fcfe4106960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:57.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.555+0000 7fcfe8c00700 1 -- 172.21.15.151:0/4150117305 >> 172.21.15.151:0/4150117305 conn(0x7fcfe40793e0 msgr2=0x7fcfe40797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:57:57.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.556+0000 7fcfe8c00700 1 -- 172.21.15.151:0/4150117305 shutdown_connections 2024-05-21T21:57:57.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.556+0000 7fcfe8c00700 1 -- 172.21.15.151:0/4150117305 wait complete. 2024-05-21T21:57:57.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.557+0000 7fcfe8c00700 1 Processor -- start 2024-05-21T21:57:57.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.557+0000 7fcfe8c00700 1 -- start start 2024-05-21T21:57:57.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.558+0000 7fcfe8c00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcfe406b240 0x7fcfe41a01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:57.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.558+0000 7fcfe8c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcfe41064e0 0x7fcfe41a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:57.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.558+0000 7fcfe8c00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcfe41084f0 0x7fcfe41a4a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:57.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.558+0000 7fcfe8c00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcfe4119d30 con 0x7fcfe41064e0 2024-05-21T21:57:57.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.558+0000 7fcfe8c00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcfe4119bb0 con 0x7fcfe41084f0 2024-05-21T21:57:57.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.558+0000 7fcfe8c00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fcfe4119eb0 con 0x7fcfe406b240 2024-05-21T21:57:57.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.558+0000 7fcfe1400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcfe41064e0 0x7fcfe41a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:57.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.558+0000 7fcfe1e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcfe406b240 0x7fcfe41a01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:57.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.558+0000 7fcfe1400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcfe41064e0 0x7fcfe41a06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:53612/0 (socket says 172.21.15.151:53612) 2024-05-21T21:57:57.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.558+0000 7fcfe1400700 1 -- 172.21.15.151:0/2940520783 learned_addr learned my addr 172.21.15.151:0/2940520783 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:57:57.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.558+0000 7fcfe2800700 1 --2- 172.21.15.151:0/2940520783 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcfe41084f0 0x7fcfe41a4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:57.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.559+0000 7fcfe1400700 1 -- 172.21.15.151:0/2940520783 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcfe406b240 msgr2=0x7fcfe41a01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:57.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.559+0000 7fcfe1400700 1 --2- 172.21.15.151:0/2940520783 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcfe406b240 0x7fcfe41a01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:57.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.559+0000 7fcfe1400700 1 -- 172.21.15.151:0/2940520783 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcfe41084f0 msgr2=0x7fcfe41a4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:57.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.559+0000 7fcfe1400700 1 --2- 172.21.15.151:0/2940520783 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcfe41084f0 0x7fcfe41a4a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:57.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.559+0000 7fcfe1400700 1 -- 172.21.15.151:0/2940520783 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfe41a51b0 con 0x7fcfe41064e0 2024-05-21T21:57:57.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.559+0000 7fcfe2800700 1 --2- 172.21.15.151:0/2940520783 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcfe41084f0 0x7fcfe41a4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T21:57:57.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.559+0000 7fcfe1e00700 1 --2- 172.21.15.151:0/2940520783 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcfe406b240 0x7fcfe41a01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T21:57:57.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.559+0000 7fcfe1400700 1 --2- 172.21.15.151:0/2940520783 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcfe41064e0 0x7fcfe41a06e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fcfd000e9a0 tx=0x7fcfd000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:57:57.561 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.559+0000 7fcfdaa00700 1 -- 172.21.15.151:0/2940520783 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd000cbd0 con 0x7fcfe41064e0 2024-05-21T21:57:57.561 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.559+0000 7fcfe8c00700 1 -- 172.21.15.151:0/2940520783 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfe41a54a0 con 0x7fcfe41064e0 2024-05-21T21:57:57.561 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.560+0000 7fcfdaa00700 1 -- 172.21.15.151:0/2940520783 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfd000cd70 con 0x7fcfe41064e0 2024-05-21T21:57:57.561 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.560+0000 7fcfdaa00700 1 -- 172.21.15.151:0/2940520783 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd0018680 con 0x7fcfe41064e0 2024-05-21T21:57:57.561 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.560+0000 7fcfe8c00700 1 -- 172.21.15.151:0/2940520783 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfe41acd20 con 0x7fcfe41064e0 2024-05-21T21:57:57.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.566+0000 7fcff3610b80 1 Processor -- start 2024-05-21T21:57:57.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.566+0000 7fcfdaa00700 1 -- 172.21.15.151:0/2940520783 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7fcfd0018820 con 0x7fcfe41064e0 2024-05-21T21:57:57.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.566+0000 7fcff3610b80 1 -- start start 2024-05-21T21:57:57.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.566+0000 7fcff3610b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e7994b3480 0x55e7994b3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:57.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.566+0000 7fcff3610b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7994b3da0 0x55e7994ada00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:57.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.566+0000 7fcff3610b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e7994ae020 0x55e7994ae460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:57.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.566+0000 7fcff3610b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e79940bfe0 con 0x55e7994b3480 2024-05-21T21:57:57.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.566+0000 7fcff3610b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55e7993fbb80 con 0x55e7994b3da0 2024-05-21T21:57:57.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.566+0000 7fcff3610b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55e7993ffbc0 con 0x55e7994ae020 2024-05-21T21:57:57.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.567+0000 7fcfe2800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e7994b3480 0x55e7994b3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:57.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.567+0000 7fcfe1e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7994b3da0 0x55e7994ada00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:57.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.567+0000 7fcfe2800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e7994b3480 0x55e7994b3860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:53622/0 (socket says 172.21.15.151:53622) 2024-05-21T21:57:57.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.567+0000 7fcfe2800700 1 -- 172.21.15.151:0/510180145 learned_addr learned my addr 172.21.15.151:0/510180145 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:57:57.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.567+0000 7fcfe1400700 1 --2- 172.21.15.151:0/510180145 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e7994ae020 0x55e7994ae460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:57.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.567+0000 7fcfe2800700 1 -- 172.21.15.151:0/510180145 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e7994ae020 msgr2=0x55e7994ae460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:57.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.567+0000 7fcfe2800700 1 --2- 172.21.15.151:0/510180145 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e7994ae020 0x55e7994ae460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:57.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.567+0000 7fcfe2800700 1 -- 172.21.15.151:0/510180145 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7994b3da0 msgr2=0x55e7994ada00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:57.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.567+0000 7fcfe2800700 1 --2- 172.21.15.151:0/510180145 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7994b3da0 0x55e7994ada00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:57.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.567+0000 7fcfe2800700 1 -- 172.21.15.151:0/510180145 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7993c45c0 con 0x55e7994b3480 2024-05-21T21:57:57.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.568+0000 7fcfe2800700 1 --2- 172.21.15.151:0/510180145 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e7994b3480 0x55e7994b3860 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc0087c0 tx=0x7fcfdc008ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15f3e55c26deb8c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:57:57.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.568+0000 7fcfdaa00700 1 --2- 172.21.15.151:0/2940520783 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fcfc80b2aa0 0x7fcfc80b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:57.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.568+0000 7fcfdaa00700 1 -- 172.21.15.151:0/2940520783 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fcfd00e28a0 con 0x7fcfe41064e0 2024-05-21T21:57:57.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.568+0000 7fcfcf400700 1 -- 172.21.15.151:0/510180145 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfdc003c00 con 0x55e7994b3480 2024-05-21T21:57:57.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.568+0000 7fcfcf400700 1 -- 172.21.15.151:0/510180145 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfdc003da0 con 0x55e7994b3480 2024-05-21T21:57:57.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.568+0000 7fcfcf400700 1 -- 172.21.15.151:0/510180145 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfdc0129c0 con 0x55e7994b3480 2024-05-21T21:57:57.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.568+0000 7fcfe1e00700 1 --2- 172.21.15.151:0/2940520783 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fcfc80b2aa0 0x7fcfc80b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:57.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.569+0000 7fcfe1e00700 1 --2- 172.21.15.151:0/2940520783 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fcfc80b2aa0 0x7fcfc80b4f60 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fcfd400b2e0 tx=0x7fcfd401a040 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:57:57.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.569+0000 7fcff3610b80 1 -- 172.21.15.151:0/510180145 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e7994b3480 msgr2=0x55e7994b3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:57.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.569+0000 7fcff3610b80 1 --2- 172.21.15.151:0/510180145 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e7994b3480 0x55e7994b3860 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc0087c0 tx=0x7fcfdc008ad0 comp rx=0 tx=0).stop 2024-05-21T21:57:57.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.569+0000 7fcff3610b80 1 -- 172.21.15.151:0/510180145 shutdown_connections 2024-05-21T21:57:57.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.569+0000 7fcff3610b80 1 --2- 172.21.15.151:0/510180145 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e7994ae020 0x55e7994ae460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:57.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.569+0000 7fcff3610b80 1 --2- 172.21.15.151:0/510180145 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e7994b3480 0x55e7994b3860 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:57.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.569+0000 7fcff3610b80 1 --2- 172.21.15.151:0/510180145 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7994b3da0 0x55e7994ada00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:57.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.569+0000 7fcff3610b80 1 -- 172.21.15.151:0/510180145 >> 172.21.15.151:0/510180145 conn(0x55e7993dbd80 msgr2=0x55e7994a84c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:57:57.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.570+0000 7fcff3610b80 1 -- 172.21.15.151:0/510180145 shutdown_connections 2024-05-21T21:57:57.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.570+0000 7fcff3610b80 1 -- 172.21.15.151:0/510180145 wait complete. 2024-05-21T21:57:57.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.571+0000 7fcff3610b80 1 -- start start 2024-05-21T21:57:57.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.571+0000 7fcff3610b80 1 Processor -- start 2024-05-21T21:57:57.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.571+0000 7fcff3610b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e7994ae020 0x55e7994b9a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:57.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.571+0000 7fcff3610b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7994b3480 0x55e7994b6e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:57.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.571+0000 7fcff3610b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e7994b3da0 0x55e7994b7360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:57.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.571+0000 7fcff3610b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e799302140 con 0x55e7994b3da0 2024-05-21T21:57:57.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.571+0000 7fcff3610b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55e79939c900 con 0x55e7994b3480 2024-05-21T21:57:57.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.571+0000 7fcff3610b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55e7994ba880 con 0x55e7994ae020 2024-05-21T21:57:57.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.572+0000 7fcff3610b80 10 client.0 _ll_register_callbacks cb 0x55e7993a5e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:57:57.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.572+0000 7fcfe2800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e7994b3da0 0x55e7994b7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:57.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.572+0000 7fcfe1e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e7994ae020 0x55e7994b9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:57.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.572+0000 7fcfe2800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e7994b3da0 0x55e7994b7360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:53638/0 (socket says 172.21.15.151:53638) 2024-05-21T21:57:57.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.572+0000 7fcfe1e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e7994ae020 0x55e7994b9a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:36900/0 (socket says 172.21.15.151:36900) 2024-05-21T21:57:57.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.572+0000 7fcfe1e00700 1 -- 172.21.15.151:0/3142093406 learned_addr learned my addr 172.21.15.151:0/3142093406 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:57:57.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.572+0000 7fcfe1400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7994b3480 0x55e7994b6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:57.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.572+0000 7fcfe1400700 1 -- 172.21.15.151:0/3142093406 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e7994ae020 msgr2=0x55e7994b9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:57.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.572+0000 7fcfe1400700 1 --2- 172.21.15.151:0/3142093406 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e7994ae020 0x55e7994b9a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:57.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.572+0000 7fcfe1400700 1 -- 172.21.15.151:0/3142093406 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e7994b3da0 msgr2=0x55e7994b7360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:57.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.572+0000 7fcfe1400700 1 --2- 172.21.15.151:0/3142093406 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e7994b3da0 0x55e7994b7360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:57.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.572+0000 7fcfe1400700 1 -- 172.21.15.151:0/3142093406 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e79934ba70 con 0x55e7994b3480 2024-05-21T21:57:57.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.572+0000 7fcfe1e00700 1 --2- 172.21.15.151:0/3142093406 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e7994ae020 0x55e7994b9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:57:57.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.572+0000 7fcfe2800700 1 --2- 172.21.15.151:0/3142093406 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e7994b3da0 0x55e7994b7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:57:57.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.573+0000 7fcfe1400700 1 --2- 172.21.15.151:0/3142093406 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7994b3480 0x55e7994b6e20 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fcfd00dffa0 tx=0x7fcfd00b27e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8242bf264f1e0424 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:57:57.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.573+0000 7fcfc3e00700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-21T21:57:57.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.573+0000 7fcfc3e00700 1 -- 172.21.15.151:0/3142093406 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd00a9480 con 0x55e7994b3480 2024-05-21T21:57:57.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.573+0000 7fcff3610b80 1 -- 172.21.15.151:0/3142093406 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfdc003da0 con 0x55e7994b3480 2024-05-21T21:57:57.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.573+0000 7fcfc3e00700 1 -- 172.21.15.151:0/3142093406 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fcfd00a56b0 con 0x55e7994b3480 2024-05-21T21:57:57.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.573+0000 7fcfc3e00700 1 -- 172.21.15.151:0/3142093406 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcfd00a52c0 con 0x55e7994b3480 2024-05-21T21:57:57.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.573+0000 7fcff3610b80 10 client.25790 fetch_fsmap learned FSMap version 29 2024-05-21T21:57:57.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.573+0000 7fcff3610b80 1 -- 172.21.15.151:0/3142093406 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e7993cc200 con 0x55e7994b3480 2024-05-21T21:57:57.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.574+0000 7fcfc3e00700 1 -- 172.21.15.151:0/3142093406 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfdc003da0 con 0x55e7994b3480 2024-05-21T21:57:57.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.574+0000 7fcfc3e00700 1 -- 172.21.15.151:0/3142093406 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fcfd00cc0e0 con 0x55e7994b3480 2024-05-21T21:57:57.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.574+0000 7fcff3610b80 10 client.25790 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T21:57:57.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.574+0000 7fcff3610b80 10 client.25790 resolve_mds: validated gid 24530 aka daemon mds.b 2024-05-21T21:57:57.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.574+0000 7fcff3610b80 20 client.25790 populate_metadata read hostname 'smithi151' 2024-05-21T21:57:57.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.574+0000 7fcff3610b80 1 --2- 172.21.15.151:0/3142093406 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55e7994c73c0 0x55e7994c97b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:57.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.574+0000 7fcff3610b80 4 client.25790 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T21:57:57.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.574+0000 7fcff3610b80 1 -- 172.21.15.151:0/3142093406 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e79940df30 con 0x55e7994c73c0 2024-05-21T21:57:57.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.575+0000 7fcfe1e00700 1 --2- 172.21.15.151:0/3142093406 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55e7994c73c0 0x55e7994c97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:57.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.576+0000 7fcfe1e00700 1 --2- 172.21.15.151:0/3142093406 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55e7994c73c0 0x55e7994c97b0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9c1ddef669f880e server_cookie=e088a11d51f421d5 in_seq=0 out_seq=0 2024-05-21T21:57:57.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.576+0000 7fcfc3e00700 10 client.25790 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T21:57:57.578 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.576+0000 7fcfc3e00700 1 -- 172.21.15.151:0/3142093406 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55e79940df30 con 0x55e7994c73c0 2024-05-21T21:57:57.578 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.576+0000 7fcfc3e00700 10 client.25790 handle_command_reply: tid=0 2024-05-21T21:57:57.578 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.577+0000 7fcff3610b80 1 client.25790 shutdown 2024-05-21T21:57:57.578 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.577+0000 7fcff3610b80 1 -- 172.21.15.151:0/3142093406 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7994b3480 msgr2=0x55e7994b6e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:57.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.577+0000 7fcff3610b80 1 --2- 172.21.15.151:0/3142093406 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7994b3480 0x55e7994b6e20 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fcfd00dffa0 tx=0x7fcfd00b27e0 comp rx=0 tx=0).stop 2024-05-21T21:57:57.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.577+0000 7fcff3610b80 1 -- 172.21.15.151:0/3142093406 shutdown_connections 2024-05-21T21:57:57.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.577+0000 7fcff3610b80 1 --2- 172.21.15.151:0/3142093406 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e7994b3da0 0x55e7994b7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:57.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.577+0000 7fcff3610b80 1 --2- 172.21.15.151:0/3142093406 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e7994ae020 0x55e7994b9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:57.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.577+0000 7fcff3610b80 1 --2- 172.21.15.151:0/3142093406 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55e7994c73c0 0x55e7994c97b0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:57.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.577+0000 7fcff3610b80 1 --2- 172.21.15.151:0/3142093406 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7994b3480 0x55e7994b6e20 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:57.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.577+0000 7fcff3610b80 1 -- 172.21.15.151:0/3142093406 >> 172.21.15.151:0/3142093406 conn(0x55e7993dbd80 msgr2=0x55e7994ac9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:57:57.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.577+0000 7fcfc3e00700 0 client.25790 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T21:57:57.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.577+0000 7fcff3610b80 1 -- 172.21.15.151:0/3142093406 shutdown_connections 2024-05-21T21:57:57.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.578+0000 7fcff3610b80 1 -- 172.21.15.151:0/3142093406 wait complete. 2024-05-21T21:57:57.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.578+0000 7fcff3610b80 20 client.25790 trim_cache size 0 max 16384 2024-05-21T21:57:57.604 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.604+0000 7fcff3610b80 1 Processor -- start 2024-05-21T21:57:57.604 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.604+0000 7fcff3610b80 1 -- start start 2024-05-21T21:57:57.604 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.604+0000 7fcff3610b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7994ae020 0x55e7994b7150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:57.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.604+0000 7fcff3610b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e7994b3480 0x55e7994b1570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:57.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.604+0000 7fcff3610b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e7994b3da0 0x55e7994b4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:57.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.604+0000 7fcff3610b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc400d590 con 0x55e7994b3480 2024-05-21T21:57:57.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.604+0000 7fcff3610b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc400f0d0 con 0x55e7994ae020 2024-05-21T21:57:57.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.604+0000 7fcff3610b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc400dc60 con 0x55e7994b3da0 2024-05-21T21:57:57.606 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.605+0000 7fcfe2800700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e7994b3da0 0x55e7994b4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:57.606 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.605+0000 7fcfe1400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e7994b3480 0x55e7994b1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:57.606 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.605+0000 7fcfe1e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7994ae020 0x55e7994b7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:57.606 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.605+0000 7fcfe2800700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e7994b3da0 0x55e7994b4890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:36902/0 (socket says 172.21.15.151:36902) 2024-05-21T21:57:57.606 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.605+0000 7fcfe2800700 1 -- 172.21.15.151:0/205859513 learned_addr learned my addr 172.21.15.151:0/205859513 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:57:57.606 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.605+0000 7fcfe1e00700 1 -- 172.21.15.151:0/205859513 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e7994b3da0 msgr2=0x55e7994b4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:57.607 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.605+0000 7fcfe1e00700 1 --2- 172.21.15.151:0/205859513 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e7994b3da0 0x55e7994b4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:57.607 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.605+0000 7fcfe1e00700 1 -- 172.21.15.151:0/205859513 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e7994b3480 msgr2=0x55e7994b1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:57.607 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.605+0000 7fcfe1e00700 1 --2- 172.21.15.151:0/205859513 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e7994b3480 0x55e7994b1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:57.607 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.605+0000 7fcfe1e00700 1 -- 172.21.15.151:0/205859513 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7994b4dd0 con 0x55e7994ae020 2024-05-21T21:57:57.607 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.605+0000 7fcfe2800700 1 --2- 172.21.15.151:0/205859513 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e7994b3da0 0x55e7994b4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:57:57.607 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.605+0000 7fcfe1400700 1 --2- 172.21.15.151:0/205859513 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e7994b3480 0x55e7994b1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:57:57.607 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.606+0000 7fcfe1e00700 1 --2- 172.21.15.151:0/205859513 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7994ae020 0x55e7994b7150 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x55e7994ca030 tx=0x7fcfd4021000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2ddff892df1acda server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:57:57.607 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.606+0000 7fcfcea00700 1 -- 172.21.15.151:0/205859513 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd4023c70 con 0x55e7994ae020 2024-05-21T21:57:57.607 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.606+0000 7fcfcea00700 1 -- 172.21.15.151:0/205859513 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfd4023e10 con 0x55e7994ae020 2024-05-21T21:57:57.607 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.606+0000 7fcfcea00700 1 -- 172.21.15.151:0/205859513 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd4025750 con 0x55e7994ae020 2024-05-21T21:57:57.611 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.607+0000 7fcff3610b80 1 -- 172.21.15.151:0/205859513 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7994ae020 msgr2=0x55e7994b7150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:57.611 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.607+0000 7fcff3610b80 1 --2- 172.21.15.151:0/205859513 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7994ae020 0x55e7994b7150 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x55e7994ca030 tx=0x7fcfd4021000 comp rx=0 tx=0).stop 2024-05-21T21:57:57.611 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.607+0000 7fcff3610b80 1 -- 172.21.15.151:0/205859513 shutdown_connections 2024-05-21T21:57:57.612 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.607+0000 7fcff3610b80 1 --2- 172.21.15.151:0/205859513 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7994ae020 0x55e7994b7150 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:57.612 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.607+0000 7fcff3610b80 1 --2- 172.21.15.151:0/205859513 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e7994b3da0 0x55e7994b4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:57.612 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.607+0000 7fcff3610b80 1 --2- 172.21.15.151:0/205859513 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e7994b3480 0x55e7994b1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:57.612 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.607+0000 7fcff3610b80 1 -- 172.21.15.151:0/205859513 >> 172.21.15.151:0/205859513 conn(0x55e7993dbd80 msgr2=0x55e7994b4180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:57:57.612 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.607+0000 7fcff3610b80 1 -- 172.21.15.151:0/205859513 shutdown_connections 2024-05-21T21:57:57.612 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.607+0000 7fcff3610b80 1 -- 172.21.15.151:0/205859513 wait complete. 2024-05-21T21:57:57.612 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.607+0000 7fcff3610b80 1 -- start start 2024-05-21T21:57:57.612 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.608+0000 7fcff3610b80 1 Processor -- start 2024-05-21T21:57:57.612 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.608+0000 7fcff3610b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e7994ae020 0x55e7994c0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:57.612 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.608+0000 7fcff3610b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e7994b3480 0x55e7994bcb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:57.612 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.608+0000 7fcff3610b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7994b3da0 0x55e7994bd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:57.612 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.608+0000 7fcff3610b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc400ea00 con 0x55e7994b3480 2024-05-21T21:57:57.613 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.608+0000 7fcff3610b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc400e330 con 0x55e7994b3da0 2024-05-21T21:57:57.613 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.608+0000 7fcff3610b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc400b530 con 0x55e7994ae020 2024-05-21T21:57:57.613 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.608+0000 7fcff3610b80 10 client.0 _ll_register_callbacks cb 0x55e7993a5e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:57:57.613 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.609+0000 7fcfe1400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e7994b3480 0x55e7994bcb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:57.613 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.609+0000 7fcfe1400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e7994b3480 0x55e7994bcb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:53662/0 (socket says 172.21.15.151:53662) 2024-05-21T21:57:57.613 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.609+0000 7fcfe1400700 1 -- 172.21.15.151:0/2439685282 learned_addr learned my addr 172.21.15.151:0/2439685282 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:57:57.613 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.609+0000 7fcfe1e00700 1 --2- 172.21.15.151:0/2439685282 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e7994ae020 0x55e7994c0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:57.613 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.609+0000 7fcfe1e00700 1 -- 172.21.15.151:0/2439685282 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7994b3da0 msgr2=0x55e7994bd0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:57.613 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.609+0000 7fcfe1e00700 1 --2- 172.21.15.151:0/2439685282 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7994b3da0 0x55e7994bd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:57.613 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.609+0000 7fcfe1e00700 1 -- 172.21.15.151:0/2439685282 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e7994b3480 msgr2=0x55e7994bcb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:57:57.613 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.609+0000 7fcfe1e00700 1 --2- 172.21.15.151:0/2439685282 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e7994b3480 0x55e7994bcb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:57.614 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.609+0000 7fcfe1e00700 1 -- 172.21.15.151:0/2439685282 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7994bd790 con 0x55e7994ae020 2024-05-21T21:57:57.614 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.609+0000 7fcfe1400700 1 --2- 172.21.15.151:0/2439685282 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e7994b3480 0x55e7994bcb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:57:57.614 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.609+0000 7fcfe1e00700 1 --2- 172.21.15.151:0/2439685282 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e7994ae020 0x55e7994c0100 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fcfd40098e0 tx=0x7fcfd4009290 comp rx=0 tx=0).ready entity=mon.1 client_cookie=960500d2ece09b6b server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:57:57.614 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.609+0000 7fcfc3e00700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-21T21:57:57.614 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.610+0000 7fcfc3e00700 1 -- 172.21.15.151:0/2439685282 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd400f870 con 0x55e7994ae020 2024-05-21T21:57:57.614 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.610+0000 7fcff3610b80 1 -- 172.21.15.151:0/2439685282 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfd4023e10 con 0x55e7994ae020 2024-05-21T21:57:57.614 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.610+0000 7fcfc3e00700 1 -- 172.21.15.151:0/2439685282 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fcfd400fa10 con 0x55e7994ae020 2024-05-21T21:57:57.614 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.610+0000 7fcfc3e00700 1 -- 172.21.15.151:0/2439685282 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcfd404c050 con 0x55e7994ae020 2024-05-21T21:57:57.614 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.610+0000 7fcff3610b80 10 client.25810 fetch_fsmap learned FSMap version 29 2024-05-21T21:57:57.614 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.610+0000 7fcff3610b80 1 -- 172.21.15.151:0/2439685282 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e799532e70 con 0x55e7994ae020 2024-05-21T21:57:57.614 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.610+0000 7fcfc3e00700 1 -- 172.21.15.151:0/2439685282 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfd4023e10 con 0x55e7994ae020 2024-05-21T21:57:57.614 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.610+0000 7fcfc3e00700 1 -- 172.21.15.151:0/2439685282 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fcfd4025460 con 0x55e7994ae020 2024-05-21T21:57:57.614 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.610+0000 7fcff3610b80 10 client.25810 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T21:57:57.614 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.610+0000 7fcff3610b80 10 client.25810 resolve_mds: validated gid 24530 aka daemon mds.b 2024-05-21T21:57:57.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.610+0000 7fcff3610b80 20 client.25810 populate_metadata read hostname 'smithi151' 2024-05-21T21:57:57.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.611+0000 7fcff3610b80 1 --2- 172.21.15.151:0/2439685282 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55e799533510 0x55e799535930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:57:57.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.611+0000 7fcff3610b80 4 client.25810 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 146.15, "await_for": 45.0}] 2024-05-21T21:57:57.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.611+0000 7fcff3610b80 1 -- 172.21.15.151:0/2439685282 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 146.15, "await_for": 45.0}) -- 0x55e799407710 con 0x55e799533510 2024-05-21T21:57:57.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.611+0000 7fcfe1400700 1 --2- 172.21.15.151:0/2439685282 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55e799533510 0x55e799535930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:57:57.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.611+0000 7fcfe1400700 1 --2- 172.21.15.151:0/2439685282 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55e799533510 0x55e799535930 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6cb4d853f733651 server_cookie=6965efa121185be5 in_seq=0 out_seq=0 2024-05-21T21:57:57.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.611+0000 7fcfc3e00700 10 client.25810 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T21:57:57.622 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.621+0000 7fcfc3e00700 1 -- 172.21.15.151:0/2439685282 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55e799407710 con 0x55e799533510 2024-05-21T21:57:57.622 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.622+0000 7fcfc3e00700 10 client.25810 handle_command_reply: tid=0 2024-05-21T21:57:57.623 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.622+0000 7fcff3610b80 1 client.25810 shutdown 2024-05-21T21:57:57.623 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.622+0000 7fcff3610b80 1 -- 172.21.15.151:0/2439685282 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e7994ae020 msgr2=0x55e7994c0100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:57.623 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.622+0000 7fcff3610b80 1 --2- 172.21.15.151:0/2439685282 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e7994ae020 0x55e7994c0100 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fcfd40098e0 tx=0x7fcfd4009290 comp rx=0 tx=0).stop 2024-05-21T21:57:57.623 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.622+0000 7fcff3610b80 1 -- 172.21.15.151:0/2439685282 shutdown_connections 2024-05-21T21:57:57.623 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.622+0000 7fcff3610b80 1 --2- 172.21.15.151:0/2439685282 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55e799533510 0x55e799535930 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:57.623 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.622+0000 7fcff3610b80 1 --2- 172.21.15.151:0/2439685282 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7994b3da0 0x55e7994bd0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:57.623 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.622+0000 7fcfc3e00700 0 client.25810 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T21:57:57.623 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.622+0000 7fcff3610b80 1 --2- 172.21.15.151:0/2439685282 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e7994ae020 0x55e7994c0100 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:57.623 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.622+0000 7fcff3610b80 1 --2- 172.21.15.151:0/2439685282 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e7994b3480 0x55e7994bcb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:57.623 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.622+0000 7fcff3610b80 1 -- 172.21.15.151:0/2439685282 >> 172.21.15.151:0/2439685282 conn(0x55e7993dbd80 msgr2=0x55e7994a1e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:57:57.623 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.622+0000 7fcff3610b80 1 -- 172.21.15.151:0/2439685282 shutdown_connections 2024-05-21T21:57:57.623 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.622+0000 7fcff3610b80 1 -- 172.21.15.151:0/2439685282 wait complete. 2024-05-21T21:57:57.624 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.622+0000 7fcff3610b80 20 client.25810 trim_cache size 0 max 16384 2024-05-21T21:57:57.624 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-05-21T21:57:57.624 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 29, 2024-05-21T21:57:57.624 INFO:teuthology.orchestra.run.smithi151.stdout: "leader": 24530, 2024-05-21T21:57:57.624 INFO:teuthology.orchestra.run.smithi151.stdout: "set_version": 6, 2024-05-21T21:57:57.624 INFO:teuthology.orchestra.run.smithi151.stdout: "sets": { 2024-05-21T21:57:57.624 INFO:teuthology.orchestra.run.smithi151.stdout: "acaf991b": { 2024-05-21T21:57:57.624 INFO:teuthology.orchestra.run.smithi151.stdout: "version": 6, 2024-05-21T21:57:57.625 INFO:teuthology.orchestra.run.smithi151.stdout: "age_ref": 0.0, 2024-05-21T21:57:57.625 INFO:teuthology.orchestra.run.smithi151.stdout: "state": { 2024-05-21T21:57:57.625 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "QUIESCED", 2024-05-21T21:57:57.625 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 0.0 2024-05-21T21:57:57.625 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T21:57:57.625 INFO:teuthology.orchestra.run.smithi151.stdout: "timeout": 90.0, 2024-05-21T21:57:57.625 INFO:teuthology.orchestra.run.smithi151.stdout: "expiration": 146.2, 2024-05-21T21:57:57.625 INFO:teuthology.orchestra.run.smithi151.stdout: "members": { 2024-05-21T21:57:57.625 INFO:teuthology.orchestra.run.smithi151.stdout: "file:/": { 2024-05-21T21:57:57.625 INFO:teuthology.orchestra.run.smithi151.stdout: "excluded": false, 2024-05-21T21:57:57.625 INFO:teuthology.orchestra.run.smithi151.stdout: "state": { 2024-05-21T21:57:57.625 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "QUIESCED", 2024-05-21T21:57:57.625 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 0.0 2024-05-21T21:57:57.625 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T21:57:57.625 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T21:57:57.626 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T21:57:57.626 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T21:57:57.626 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T21:57:57.626 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-05-21T21:57:57.626 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.623+0000 7fcfe8c00700 1 -- 172.21.15.151:0/2940520783 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fcfc80b2aa0 msgr2=0x7fcfc80b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:57.626 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.623+0000 7fcfe8c00700 1 --2- 172.21.15.151:0/2940520783 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fcfc80b2aa0 0x7fcfc80b4f60 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fcfd400b2e0 tx=0x7fcfd401a040 comp rx=0 tx=0).stop 2024-05-21T21:57:57.626 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.623+0000 7fcfe8c00700 1 -- 172.21.15.151:0/2940520783 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcfe41064e0 msgr2=0x7fcfe41a06e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:57:57.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.623+0000 7fcfe8c00700 1 --2- 172.21.15.151:0/2940520783 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcfe41064e0 0x7fcfe41a06e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fcfd000e9a0 tx=0x7fcfd000ed60 comp rx=0 tx=0).stop 2024-05-21T21:57:57.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.623+0000 7fcfe8c00700 1 -- 172.21.15.151:0/2940520783 shutdown_connections 2024-05-21T21:57:57.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.623+0000 7fcfe8c00700 1 --2- 172.21.15.151:0/2940520783 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fcfc80b2aa0 0x7fcfc80b4f60 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:57.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.623+0000 7fcfe8c00700 1 --2- 172.21.15.151:0/2940520783 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcfe41084f0 0x7fcfe41a4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:57.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.623+0000 7fcfe8c00700 1 --2- 172.21.15.151:0/2940520783 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcfe406b240 0x7fcfe41a01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:57.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.623+0000 7fcfe8c00700 1 --2- 172.21.15.151:0/2940520783 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcfe41064e0 0x7fcfe41a06e0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:57:57.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.623+0000 7fcfe8c00700 1 -- 172.21.15.151:0/2940520783 >> 172.21.15.151:0/2940520783 conn(0x7fcfe40793e0 msgr2=0x7fcfe41002b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:57:57.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.624+0000 7fcfe8c00700 1 -- 172.21.15.151:0/2940520783 shutdown_connections 2024-05-21T21:57:57.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:57:57.624+0000 7fcfe8c00700 1 -- 172.21.15.151:0/2940520783 wait complete. 2024-05-21T21:57:57.769 INFO:tasks.workunit.client.0.smithi151.stderr:Note: switching to '7956b5e965f6ee5669688b5643c11d881b72442c'. 2024-05-21T21:57:57.769 INFO:tasks.workunit.client.0.smithi151.stderr: 2024-05-21T21:57:57.769 INFO:tasks.workunit.client.0.smithi151.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-05-21T21:57:57.769 INFO:tasks.workunit.client.0.smithi151.stderr:changes and commit them, and you can discard any commits you make in this 2024-05-21T21:57:57.770 INFO:tasks.workunit.client.0.smithi151.stderr:state without impacting any branches by switching back to a branch. 2024-05-21T21:57:57.770 INFO:tasks.workunit.client.0.smithi151.stderr: 2024-05-21T21:57:57.770 INFO:tasks.workunit.client.0.smithi151.stderr:If you want to create a new branch to retain commits you create, you may 2024-05-21T21:57:57.770 INFO:tasks.workunit.client.0.smithi151.stderr:do so (now or later) by using -c with the switch command. Example: 2024-05-21T21:57:57.770 INFO:tasks.workunit.client.0.smithi151.stderr: 2024-05-21T21:57:57.770 INFO:tasks.workunit.client.0.smithi151.stderr: git switch -c 2024-05-21T21:57:57.770 INFO:tasks.workunit.client.0.smithi151.stderr: 2024-05-21T21:57:57.770 INFO:tasks.workunit.client.0.smithi151.stderr:Or undo this operation with: 2024-05-21T21:57:57.770 INFO:tasks.workunit.client.0.smithi151.stderr: 2024-05-21T21:57:57.770 INFO:tasks.workunit.client.0.smithi151.stderr: git switch - 2024-05-21T21:57:57.770 INFO:tasks.workunit.client.0.smithi151.stderr: 2024-05-21T21:57:57.770 INFO:tasks.workunit.client.0.smithi151.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-05-21T21:57:57.770 INFO:tasks.workunit.client.0.smithi151.stderr: 2024-05-21T21:57:57.771 INFO:tasks.workunit.client.0.smithi151.stderr:HEAD is now at 7956b5e965f mds/quiesce: overdrive an export if it hasn't frozen the tree yet 2024-05-21T21:57:57.784 DEBUG:teuthology.orchestra.run.smithi151:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-05-21T21:57:57.905 INFO:tasks.workunit.client.0.smithi151.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-05-21T21:57:57.915 INFO:tasks.workunit.client.0.smithi151.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-21T21:57:57.915 INFO:tasks.workunit.client.0.smithi151.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-05-21T21:57:59.698 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:57:59 smithi151 ceph-mon[31623]: pgmap v74: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-05-21T21:57:59.735 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:57:59 smithi184 ceph-mon[36942]: pgmap v74: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-05-21T21:57:59.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:57:59 smithi164 ceph-mon[36269]: pgmap v74: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-05-21T21:58:00.109 INFO:tasks.workunit.client.0.smithi151.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-05-21T21:58:00.209 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'acaf991b', quiesce took 12.2 seconds. Will release after: 13.95 2024-05-21T21:58:00.227 INFO:tasks.workunit.client.0.smithi151.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-05-21T21:58:00.306 INFO:tasks.workunit.client.0.smithi151.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-21T21:58:00.312 INFO:tasks.workunit.client.0.smithi151.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-21T21:58:00.312 INFO:tasks.workunit.client.0.smithi151.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-05-21T21:58:00.401 INFO:tasks.workunit.client.0.smithi151.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-21T21:58:00.413 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-05-21T21:58:00.414 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-05-21T21:58:00.513 INFO:tasks.workunit:Running workunits matching suites/fsync-tester.sh on client.0... 2024-05-21T21:58:00.514 INFO:tasks.workunit:Running workunit suites/fsync-tester.sh... 2024-05-21T21:58:00.515 DEBUG:teuthology.orchestra.run.smithi151:workunit test suites/fsync-tester.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=7956b5e965f6ee5669688b5643c11d881b72442c TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsync-tester.sh 2024-05-21T21:58:00.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.827+0000 7f5a37e00700 1 -- 172.21.15.151:0/2593584880 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a30105760 msgr2=0x7f5a30105be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:00.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.827+0000 7f5a37e00700 1 --2- 172.21.15.151:0/2593584880 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a30105760 0x7f5a30105be0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c009990 tx=0x7f5a1c009ca0 comp rx=0 tx=0).stop 2024-05-21T21:58:00.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.828+0000 7f5a37e00700 1 -- 172.21.15.151:0/2593584880 shutdown_connections 2024-05-21T21:58:00.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.828+0000 7f5a37e00700 1 --2- 172.21.15.151:0/2593584880 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5a3010b760 0x7f5a3010bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:00.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.828+0000 7f5a37e00700 1 --2- 172.21.15.151:0/2593584880 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a30106120 0x7f5a30112630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:00.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.828+0000 7f5a37e00700 1 --2- 172.21.15.151:0/2593584880 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a30105760 0x7f5a30105be0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:00.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.828+0000 7f5a37e00700 1 -- 172.21.15.151:0/2593584880 >> 172.21.15.151:0/2593584880 conn(0x7f5a30101250 msgr2=0x7f5a30103670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:58:00.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.828+0000 7f5a37e00700 1 -- 172.21.15.151:0/2593584880 shutdown_connections 2024-05-21T21:58:00.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.828+0000 7f5a37e00700 1 -- 172.21.15.151:0/2593584880 wait complete. 2024-05-21T21:58:00.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.829+0000 7f5a37e00700 1 Processor -- start 2024-05-21T21:58:00.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.829+0000 7f5a37e00700 1 -- start start 2024-05-21T21:58:00.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.829+0000 7f5a37e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5a30105760 0x7f5a3019be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:00.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.829+0000 7f5a37e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a30106120 0x7f5a3019c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:00.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.829+0000 7f5a37e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a3010b760 0x7f5a301a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:00.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.829+0000 7f5a37e00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5a30115960 con 0x7f5a3010b760 2024-05-21T21:58:00.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.829+0000 7f5a37e00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5a301157e0 con 0x7f5a30105760 2024-05-21T21:58:00.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.829+0000 7f5a37e00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5a30115ae0 con 0x7f5a30106120 2024-05-21T21:58:00.831 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.830+0000 7f5a34a00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a30106120 0x7f5a3019c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:00.831 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.830+0000 7f5a35400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5a30105760 0x7f5a3019be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:00.831 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.830+0000 7f5a34a00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a30106120 0x7f5a3019c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:36922/0 (socket says 172.21.15.151:36922) 2024-05-21T21:58:00.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.830+0000 7f5a34a00700 1 -- 172.21.15.151:0/3443123533 learned_addr learned my addr 172.21.15.151:0/3443123533 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:58:00.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.830+0000 7f5a35e00700 1 --2- 172.21.15.151:0/3443123533 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a3010b760 0x7f5a301a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:00.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.830+0000 7f5a35400700 1 -- 172.21.15.151:0/3443123533 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a30106120 msgr2=0x7f5a3019c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:00.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.830+0000 7f5a35400700 1 --2- 172.21.15.151:0/3443123533 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a30106120 0x7f5a3019c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:00.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.830+0000 7f5a35400700 1 -- 172.21.15.151:0/3443123533 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a3010b760 msgr2=0x7f5a301a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:00.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.830+0000 7f5a35400700 1 --2- 172.21.15.151:0/3443123533 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a3010b760 0x7f5a301a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:00.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.830+0000 7f5a35400700 1 -- 172.21.15.151:0/3443123533 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a301a0df0 con 0x7f5a30105760 2024-05-21T21:58:00.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.831+0000 7f5a35400700 1 --2- 172.21.15.151:0/3443123533 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5a30105760 0x7f5a3019be30 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5a2400e9f0 tx=0x7f5a2400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:58:00.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.831+0000 7f5a2e000700 1 -- 172.21.15.151:0/3443123533 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a2400cb70 con 0x7f5a30105760 2024-05-21T21:58:00.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.831+0000 7f5a37e00700 1 -- 172.21.15.151:0/3443123533 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a301a10e0 con 0x7f5a30105760 2024-05-21T21:58:00.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.831+0000 7f5a2e000700 1 -- 172.21.15.151:0/3443123533 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a2400ee90 con 0x7f5a30105760 2024-05-21T21:58:00.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.831+0000 7f5a2e000700 1 -- 172.21.15.151:0/3443123533 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a24003d40 con 0x7f5a30105760 2024-05-21T21:58:00.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.831+0000 7f5a37e00700 1 -- 172.21.15.151:0/3443123533 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a301a8960 con 0x7f5a30105760 2024-05-21T21:58:00.837 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.836+0000 7f5a2e000700 1 -- 172.21.15.151:0/3443123533 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7f5a24003f00 con 0x7f5a30105760 2024-05-21T21:58:00.837 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.837+0000 7f5a42732b80 1 Processor -- start 2024-05-21T21:58:00.837 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.837+0000 7f5a42732b80 1 -- start start 2024-05-21T21:58:00.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.837+0000 7f5a42732b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556431901610 0x5564319019f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:00.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.838+0000 7f5a42732b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556431901f30 0x5564318fbb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:00.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.838+0000 7f5a42732b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5564318fc140 0x5564318fc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:00.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.838+0000 7f5a42732b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x556431859e40 con 0x556431901f30 2024-05-21T21:58:00.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.838+0000 7f5a42732b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5564318499e0 con 0x5564318fc140 2024-05-21T21:58:00.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.838+0000 7f5a42732b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55643184da20 con 0x556431901610 2024-05-21T21:58:00.839 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.838+0000 7f5a35400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556431901f30 0x5564318fbb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:00.839 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.838+0000 7f5a34a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5564318fc140 0x5564318fc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:00.839 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.838+0000 7f5a34a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5564318fc140 0x5564318fc5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:40788/0 (socket says 172.21.15.151:40788) 2024-05-21T21:58:00.839 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.838+0000 7f5a35e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556431901610 0x5564319019f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:00.839 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.838+0000 7f5a35400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556431901f30 0x5564318fbb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:53690/0 (socket says 172.21.15.151:53690) 2024-05-21T21:58:00.839 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.838+0000 7f5a34a00700 1 -- 172.21.15.151:0/506789820 learned_addr learned my addr 172.21.15.151:0/506789820 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:58:00.839 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.838+0000 7f5a34a00700 1 -- 172.21.15.151:0/506789820 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556431901610 msgr2=0x5564319019f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:00.840 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.838+0000 7f5a34a00700 1 --2- 172.21.15.151:0/506789820 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556431901610 0x5564319019f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:00.840 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.838+0000 7f5a34a00700 1 -- 172.21.15.151:0/506789820 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556431901f30 msgr2=0x5564318fbb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:00.840 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.838+0000 7f5a34a00700 1 --2- 172.21.15.151:0/506789820 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556431901f30 0x5564318fbb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:00.840 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.838+0000 7f5a34a00700 1 -- 172.21.15.151:0/506789820 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556431812420 con 0x5564318fc140 2024-05-21T21:58:00.840 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.838+0000 7f5a2e000700 1 --2- 172.21.15.151:0/3443123533 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f5a180b2a70 0x7f5a180b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:00.840 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.839+0000 7f5a2e000700 1 -- 172.21.15.151:0/3443123533 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f5a240e26b0 con 0x7f5a30105760 2024-05-21T21:58:00.840 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.839+0000 7f5a35e00700 1 --2- 172.21.15.151:0/3443123533 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f5a180b2a70 0x7f5a180b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:00.840 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.839+0000 7f5a34a00700 1 --2- 172.21.15.151:0/506789820 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5564318fc140 0x5564318fc5c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c009740 tx=0x7f5a1c005c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9bc00488c7d5d5e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:58:00.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.839+0000 7f5a22a00700 1 -- 172.21.15.151:0/506789820 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a1c027070 con 0x5564318fc140 2024-05-21T21:58:00.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.840+0000 7f5a22a00700 1 -- 172.21.15.151:0/506789820 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a1c005650 con 0x5564318fc140 2024-05-21T21:58:00.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.840+0000 7f5a22a00700 1 -- 172.21.15.151:0/506789820 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a1c003be0 con 0x5564318fc140 2024-05-21T21:58:00.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.840+0000 7f5a42732b80 1 -- 172.21.15.151:0/506789820 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5564318fc140 msgr2=0x5564318fc5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:00.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.840+0000 7f5a35e00700 1 --2- 172.21.15.151:0/3443123533 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f5a180b2a70 0x7f5a180b4f30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5a280127f0 tx=0x7f5a2800b040 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:58:00.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.840+0000 7f5a42732b80 1 --2- 172.21.15.151:0/506789820 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5564318fc140 0x5564318fc5c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c009740 tx=0x7f5a1c005c20 comp rx=0 tx=0).stop 2024-05-21T21:58:00.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.840+0000 7f5a42732b80 1 -- 172.21.15.151:0/506789820 shutdown_connections 2024-05-21T21:58:00.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.840+0000 7f5a42732b80 1 --2- 172.21.15.151:0/506789820 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5564318fc140 0x5564318fc5c0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:00.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.840+0000 7f5a42732b80 1 --2- 172.21.15.151:0/506789820 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556431901610 0x5564319019f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:00.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.840+0000 7f5a42732b80 1 --2- 172.21.15.151:0/506789820 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556431901f30 0x5564318fbb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:00.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.840+0000 7f5a42732b80 1 -- 172.21.15.151:0/506789820 >> 172.21.15.151:0/506789820 conn(0x556431829be0 msgr2=0x5564318f6690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:58:00.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.841+0000 7f5a42732b80 1 -- 172.21.15.151:0/506789820 shutdown_connections 2024-05-21T21:58:00.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.841+0000 7f5a42732b80 1 -- 172.21.15.151:0/506789820 wait complete. 2024-05-21T21:58:00.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.841+0000 7f5a42732b80 1 -- start start 2024-05-21T21:58:00.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.841+0000 7f5a42732b80 1 Processor -- start 2024-05-21T21:58:00.843 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.842+0000 7f5a42732b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5564318fc140 0x556431907bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:00.844 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.842+0000 7f5a42732b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556431901610 0x556431904f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:00.844 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.842+0000 7f5a42732b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556431901f30 0x5564319054b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:00.844 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.842+0000 7f5a42732b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55643174ffa0 con 0x556431901610 2024-05-21T21:58:00.844 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.842+0000 7f5a42732b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5564317ea760 con 0x556431901f30 2024-05-21T21:58:00.844 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.842+0000 7f5a42732b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x556431908980 con 0x5564318fc140 2024-05-21T21:58:00.844 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.842+0000 7f5a42732b80 10 client.0 _ll_register_callbacks cb 0x5564317f3c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:58:00.844 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.842+0000 7f5a35400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5564318fc140 0x556431907bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:00.844 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.842+0000 7f5a34a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556431901610 0x556431904f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:00.844 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.842+0000 7f5a35400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5564318fc140 0x556431907bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:36928/0 (socket says 172.21.15.151:36928) 2024-05-21T21:58:00.844 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.842+0000 7f5a35400700 1 -- 172.21.15.151:0/1487413669 learned_addr learned my addr 172.21.15.151:0/1487413669 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:58:00.844 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.842+0000 7f5a35e00700 1 --2- 172.21.15.151:0/1487413669 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556431901f30 0x5564319054b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:00.844 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.843+0000 7f5a35400700 1 -- 172.21.15.151:0/1487413669 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556431901f30 msgr2=0x5564319054b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:00.844 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.843+0000 7f5a35400700 1 --2- 172.21.15.151:0/1487413669 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556431901f30 0x5564319054b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:00.844 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.843+0000 7f5a35400700 1 -- 172.21.15.151:0/1487413669 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556431901610 msgr2=0x556431904f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:00.845 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.843+0000 7f5a35400700 1 --2- 172.21.15.151:0/1487413669 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556431901610 0x556431904f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:00.845 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.843+0000 7f5a35400700 1 -- 172.21.15.151:0/1487413669 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564317998d0 con 0x5564318fc140 2024-05-21T21:58:00.845 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.843+0000 7f5a35e00700 1 --2- 172.21.15.151:0/1487413669 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556431901f30 0x5564319054b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T21:58:00.845 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.843+0000 7f5a35400700 1 --2- 172.21.15.151:0/1487413669 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5564318fc140 0x556431907bd0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5a24009230 tx=0x7f5a240a97c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=900e5d6dfdf8ab1b server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:58:00.845 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.843+0000 7f5a17400700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-21T21:58:00.845 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.843+0000 7f5a17400700 1 -- 172.21.15.151:0/1487413669 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a2408fe60 con 0x5564318fc140 2024-05-21T21:58:00.845 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.843+0000 7f5a17400700 1 -- 172.21.15.151:0/1487413669 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f5a240e4be0 con 0x5564318fc140 2024-05-21T21:58:00.845 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.843+0000 7f5a42732b80 1 -- 172.21.15.151:0/1487413669 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a1c005650 con 0x5564318fc140 2024-05-21T21:58:00.845 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.844+0000 7f5a17400700 1 -- 172.21.15.151:0/1487413669 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5a240b20e0 con 0x5564318fc140 2024-05-21T21:58:00.845 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.844+0000 7f5a42732b80 10 client.25822 fetch_fsmap learned FSMap version 29 2024-05-21T21:58:00.845 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.844+0000 7f5a42732b80 1 -- 172.21.15.151:0/1487413669 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55643181a060 con 0x5564318fc140 2024-05-21T21:58:00.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.844+0000 7f5a17400700 1 -- 172.21.15.151:0/1487413669 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a1c005650 con 0x5564318fc140 2024-05-21T21:58:00.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.844+0000 7f5a17400700 1 -- 172.21.15.151:0/1487413669 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f5a240a9ba0 con 0x5564318fc140 2024-05-21T21:58:00.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.844+0000 7f5a42732b80 10 client.25822 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T21:58:00.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.844+0000 7f5a42732b80 10 client.25822 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-05-21T21:58:00.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.844+0000 7f5a42732b80 20 client.25822 populate_metadata read hostname 'smithi151' 2024-05-21T21:58:00.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.844+0000 7f5a42732b80 1 --2- 172.21.15.151:0/1487413669 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x5564319154f0 0x5564319178e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:00.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.844+0000 7f5a42732b80 4 client.25822 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T21:58:00.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.844+0000 7f5a42732b80 1 -- 172.21.15.151:0/1487413669 --> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55643185bd90 con 0x5564319154f0 2024-05-21T21:58:00.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.845+0000 7f5a34a00700 1 --2- 172.21.15.151:0/1487413669 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x5564319154f0 0x5564319178e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:00.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.845+0000 7f5a34a00700 1 --2- 172.21.15.151:0/1487413669 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x5564319154f0 0x5564319178e0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8d488af4f615e649 server_cookie=eebf3e456029579f in_seq=0 out_seq=0 2024-05-21T21:58:00.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.845+0000 7f5a17400700 10 client.25822 ms_handle_connect on v2:172.21.15.164:6834/2340428470 2024-05-21T21:58:00.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.846+0000 7f5a17400700 1 -- 172.21.15.151:0/1487413669 <== mds.4 v2:172.21.15.164:6834/2340428470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55643185bd90 con 0x5564319154f0 2024-05-21T21:58:00.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.846+0000 7f5a17400700 10 client.25822 handle_command_reply: tid=0 2024-05-21T21:58:00.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.846+0000 7f5a42732b80 1 client.25822 shutdown 2024-05-21T21:58:00.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.847+0000 7f5a42732b80 1 -- 172.21.15.151:0/1487413669 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5564318fc140 msgr2=0x556431907bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:00.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.847+0000 7f5a42732b80 1 --2- 172.21.15.151:0/1487413669 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5564318fc140 0x556431907bd0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5a24009230 tx=0x7f5a240a97c0 comp rx=0 tx=0).stop 2024-05-21T21:58:00.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.847+0000 7f5a42732b80 1 -- 172.21.15.151:0/1487413669 shutdown_connections 2024-05-21T21:58:00.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.847+0000 7f5a42732b80 1 --2- 172.21.15.151:0/1487413669 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x5564319154f0 0x5564319178e0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:00.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.847+0000 7f5a17400700 0 client.25822 ms_handle_reset on v2:172.21.15.164:6834/2340428470 2024-05-21T21:58:00.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.847+0000 7f5a42732b80 1 --2- 172.21.15.151:0/1487413669 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556431901f30 0x5564319054b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:00.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.847+0000 7f5a42732b80 1 --2- 172.21.15.151:0/1487413669 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5564318fc140 0x556431907bd0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:00.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.847+0000 7f5a42732b80 1 --2- 172.21.15.151:0/1487413669 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556431901610 0x556431904f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:00.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.847+0000 7f5a42732b80 1 -- 172.21.15.151:0/1487413669 >> 172.21.15.151:0/1487413669 conn(0x556431829be0 msgr2=0x5564318f5d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:58:00.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.847+0000 7f5a42732b80 1 -- 172.21.15.151:0/1487413669 shutdown_connections 2024-05-21T21:58:00.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.847+0000 7f5a42732b80 1 -- 172.21.15.151:0/1487413669 wait complete. 2024-05-21T21:58:00.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.847+0000 7f5a42732b80 20 client.25822 trim_cache size 0 max 16384 2024-05-21T21:58:00.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.863+0000 7f5a42732b80 1 Processor -- start 2024-05-21T21:58:00.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.864+0000 7f5a42732b80 1 -- start start 2024-05-21T21:58:00.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.864+0000 7f5a42732b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5564318fc140 0x5564318fbb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:00.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.864+0000 7f5a42732b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556431901610 0x556431903ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:00.865 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.864+0000 7f5a42732b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556431901f30 0x5564318fc520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:00.865 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.864+0000 7f5a42732b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5a301affc0 con 0x556431901610 2024-05-21T21:58:00.865 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.864+0000 7f5a42732b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5a301b1b00 con 0x5564318fc140 2024-05-21T21:58:00.865 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.864+0000 7f5a42732b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5a301b0690 con 0x556431901f30 2024-05-21T21:58:00.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.864+0000 7f5a35400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5564318fc140 0x5564318fbb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:00.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.864+0000 7f5a35400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5564318fc140 0x5564318fbb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:40814/0 (socket says 172.21.15.151:40814) 2024-05-21T21:58:00.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.864+0000 7f5a35400700 1 -- 172.21.15.151:0/2819225667 learned_addr learned my addr 172.21.15.151:0/2819225667 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:58:00.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.864+0000 7f5a35e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556431901f30 0x5564318fc520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:00.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.864+0000 7f5a34a00700 1 --2- 172.21.15.151:0/2819225667 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556431901610 0x556431903ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:00.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.865+0000 7f5a34a00700 1 -- 172.21.15.151:0/2819225667 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556431901f30 msgr2=0x5564318fc520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:00.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.865+0000 7f5a34a00700 1 --2- 172.21.15.151:0/2819225667 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556431901f30 0x5564318fc520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:00.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.865+0000 7f5a34a00700 1 -- 172.21.15.151:0/2819225667 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5564318fc140 msgr2=0x5564318fbb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:00.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.865+0000 7f5a34a00700 1 --2- 172.21.15.151:0/2819225667 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5564318fc140 0x5564318fbb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:00.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.865+0000 7f5a34a00700 1 -- 172.21.15.151:0/2819225667 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556431901d30 con 0x556431901610 2024-05-21T21:58:00.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.865+0000 7f5a35400700 1 --2- 172.21.15.151:0/2819225667 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5564318fc140 0x5564318fbb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:58:00.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.865+0000 7f5a34a00700 1 --2- 172.21.15.151:0/2819225667 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556431901610 0x556431903ed0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c009710 tx=0x7f5a1c0037c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2610ab3d8872a29f server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:58:00.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.865+0000 7f5a22000700 1 -- 172.21.15.151:0/2819225667 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a1c027040 con 0x556431901610 2024-05-21T21:58:00.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.866+0000 7f5a22000700 1 -- 172.21.15.151:0/2819225667 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a1c034040 con 0x556431901610 2024-05-21T21:58:00.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.866+0000 7f5a22000700 1 -- 172.21.15.151:0/2819225667 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a1c00f460 con 0x556431901610 2024-05-21T21:58:00.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.866+0000 7f5a42732b80 1 -- 172.21.15.151:0/2819225667 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556431901610 msgr2=0x556431903ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:00.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.866+0000 7f5a42732b80 1 --2- 172.21.15.151:0/2819225667 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556431901610 0x556431903ed0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c009710 tx=0x7f5a1c0037c0 comp rx=0 tx=0).stop 2024-05-21T21:58:00.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.866+0000 7f5a42732b80 1 -- 172.21.15.151:0/2819225667 shutdown_connections 2024-05-21T21:58:00.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.866+0000 7f5a42732b80 1 --2- 172.21.15.151:0/2819225667 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5564318fc140 0x5564318fbb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:00.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.866+0000 7f5a42732b80 1 --2- 172.21.15.151:0/2819225667 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556431901f30 0x5564318fc520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:00.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.866+0000 7f5a42732b80 1 --2- 172.21.15.151:0/2819225667 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556431901610 0x556431903ed0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:00.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.866+0000 7f5a42732b80 1 -- 172.21.15.151:0/2819225667 >> 172.21.15.151:0/2819225667 conn(0x556431829be0 msgr2=0x556431902310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:58:00.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.866+0000 7f5a42732b80 1 -- 172.21.15.151:0/2819225667 shutdown_connections 2024-05-21T21:58:00.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.866+0000 7f5a42732b80 1 -- 172.21.15.151:0/2819225667 wait complete. 2024-05-21T21:58:00.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.867+0000 7f5a42732b80 1 -- start start 2024-05-21T21:58:00.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.867+0000 7f5a42732b80 1 Processor -- start 2024-05-21T21:58:00.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.868+0000 7f5a42732b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5564318fc140 0x55643190e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:00.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.868+0000 7f5a42732b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556431901610 0x55643191c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:00.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.868+0000 7f5a42732b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556431901f30 0x55643191cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:00.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.868+0000 7f5a42732b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5a301b1430 con 0x556431901610 2024-05-21T21:58:00.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.868+0000 7f5a42732b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5a301b0d60 con 0x556431901f30 2024-05-21T21:58:00.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.868+0000 7f5a42732b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5a301adf60 con 0x5564318fc140 2024-05-21T21:58:00.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.868+0000 7f5a42732b80 10 client.0 _ll_register_callbacks cb 0x5564317f3c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:58:00.871 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.868+0000 7f5a34a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556431901610 0x55643191c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:00.871 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.868+0000 7f5a35e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556431901f30 0x55643191cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:00.871 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.868+0000 7f5a34a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556431901610 0x55643191c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:53720/0 (socket says 172.21.15.151:53720) 2024-05-21T21:58:00.871 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.868+0000 7f5a35e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556431901f30 0x55643191cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:40820/0 (socket says 172.21.15.151:40820) 2024-05-21T21:58:00.871 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.868+0000 7f5a34a00700 1 -- 172.21.15.151:0/2501400119 learned_addr learned my addr 172.21.15.151:0/2501400119 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:58:00.871 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.868+0000 7f5a35400700 1 --2- 172.21.15.151:0/2501400119 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5564318fc140 0x55643190e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:00.871 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.869+0000 7f5a35e00700 1 -- 172.21.15.151:0/2501400119 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5564318fc140 msgr2=0x55643190e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:00.872 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.869+0000 7f5a35e00700 1 --2- 172.21.15.151:0/2501400119 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5564318fc140 0x55643190e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:00.872 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.869+0000 7f5a35e00700 1 -- 172.21.15.151:0/2501400119 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556431901610 msgr2=0x55643191c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:00.872 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.869+0000 7f5a35e00700 1 --2- 172.21.15.151:0/2501400119 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556431901610 0x55643191c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:00.872 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.869+0000 7f5a35e00700 1 -- 172.21.15.151:0/2501400119 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55643191d3a0 con 0x556431901f30 2024-05-21T21:58:00.872 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.869+0000 7f5a35400700 1 --2- 172.21.15.151:0/2501400119 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5564318fc140 0x55643190e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:58:00.872 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.869+0000 7f5a34a00700 1 --2- 172.21.15.151:0/2501400119 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556431901610 0x55643191c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:58:00.872 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.869+0000 7f5a35e00700 1 --2- 172.21.15.151:0/2501400119 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556431901f30 0x55643191cd30 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5a280125d0 tx=0x7f5a28004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a10e1e0bc7774437 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:58:00.872 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.869+0000 7f5a17400700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-21T21:58:00.872 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.870+0000 7f5a17400700 1 -- 172.21.15.151:0/2501400119 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a280258a0 con 0x556431901f30 2024-05-21T21:58:00.872 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.870+0000 7f5a42732b80 1 -- 172.21.15.151:0/2501400119 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a1c034040 con 0x556431901f30 2024-05-21T21:58:00.872 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.870+0000 7f5a17400700 1 -- 172.21.15.151:0/2501400119 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f5a28025a40 con 0x556431901f30 2024-05-21T21:58:00.872 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.870+0000 7f5a17400700 1 -- 172.21.15.151:0/2501400119 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5a28004930 con 0x556431901f30 2024-05-21T21:58:00.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.870+0000 7f5a42732b80 10 client.25811 fetch_fsmap learned FSMap version 29 2024-05-21T21:58:00.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.870+0000 7f5a42732b80 1 -- 172.21.15.151:0/2501400119 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55643197ff00 con 0x556431901f30 2024-05-21T21:58:00.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.870+0000 7f5a17400700 1 -- 172.21.15.151:0/2501400119 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a1c034040 con 0x556431901f30 2024-05-21T21:58:00.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.870+0000 7f5a17400700 1 -- 172.21.15.151:0/2501400119 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f5a2802c020 con 0x556431901f30 2024-05-21T21:58:00.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.871+0000 7f5a42732b80 10 client.25811 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T21:58:00.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.871+0000 7f5a42732b80 10 client.25811 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-05-21T21:58:00.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.871+0000 7f5a42732b80 20 client.25811 populate_metadata read hostname 'smithi151' 2024-05-21T21:58:00.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.871+0000 7f5a42732b80 1 --2- 172.21.15.151:0/2501400119 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x5564319154f0 0x556431982680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:00.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.871+0000 7f5a42732b80 4 client.25811 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T21:58:00.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.871+0000 7f5a42732b80 1 -- 172.21.15.151:0/2501400119 --> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556431855570 con 0x5564319154f0 2024-05-21T21:58:00.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.871+0000 7f5a35400700 1 --2- 172.21.15.151:0/2501400119 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x5564319154f0 0x556431982680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:00.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.871+0000 7f5a35400700 1 --2- 172.21.15.151:0/2501400119 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x5564319154f0 0x556431982680 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c822273353f1cb9 server_cookie=4a3293aa542fffd8 in_seq=0 out_seq=0 2024-05-21T21:58:00.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.871+0000 7f5a17400700 10 client.25811 ms_handle_connect on v2:172.21.15.164:6834/2340428470 2024-05-21T21:58:00.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.872+0000 7f5a17400700 1 -- 172.21.15.151:0/2501400119 <== mds.4 v2:172.21.15.164:6834/2340428470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556431855570 con 0x5564319154f0 2024-05-21T21:58:00.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.872+0000 7f5a17400700 10 client.25811 handle_command_reply: tid=0 2024-05-21T21:58:00.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.872+0000 7f5a42732b80 1 client.25811 shutdown 2024-05-21T21:58:00.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.872+0000 7f5a42732b80 1 -- 172.21.15.151:0/2501400119 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556431901f30 msgr2=0x55643191cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:00.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.872+0000 7f5a42732b80 1 --2- 172.21.15.151:0/2501400119 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556431901f30 0x55643191cd30 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5a280125d0 tx=0x7f5a28004b40 comp rx=0 tx=0).stop 2024-05-21T21:58:00.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.872+0000 7f5a42732b80 1 -- 172.21.15.151:0/2501400119 shutdown_connections 2024-05-21T21:58:00.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.872+0000 7f5a42732b80 1 --2- 172.21.15.151:0/2501400119 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x5564319154f0 0x556431982680 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:00.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.872+0000 7f5a42732b80 1 --2- 172.21.15.151:0/2501400119 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556431901f30 0x55643191cd30 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:00.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.872+0000 7f5a42732b80 1 --2- 172.21.15.151:0/2501400119 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5564318fc140 0x55643190e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:00.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.872+0000 7f5a42732b80 1 --2- 172.21.15.151:0/2501400119 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556431901610 0x55643191c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:00.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.872+0000 7f5a42732b80 1 -- 172.21.15.151:0/2501400119 >> 172.21.15.151:0/2501400119 conn(0x556431829be0 msgr2=0x5564319065b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:58:00.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.872+0000 7f5a17400700 0 client.25811 ms_handle_reset on v2:172.21.15.164:6834/2340428470 2024-05-21T21:58:00.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.872+0000 7f5a42732b80 1 -- 172.21.15.151:0/2501400119 shutdown_connections 2024-05-21T21:58:00.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.872+0000 7f5a42732b80 1 -- 172.21.15.151:0/2501400119 wait complete. 2024-05-21T21:58:00.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.873+0000 7f5a42732b80 20 client.25811 trim_cache size 0 max 16384 2024-05-21T21:58:00.877 INFO:teuthology.orchestra.run.smithi151.stdout:[] 2024-05-21T21:58:00.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.873+0000 7f5a37e00700 1 -- 172.21.15.151:0/3443123533 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f5a180b2a70 msgr2=0x7f5a180b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:00.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.873+0000 7f5a37e00700 1 --2- 172.21.15.151:0/3443123533 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f5a180b2a70 0x7f5a180b4f30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5a280127f0 tx=0x7f5a2800b040 comp rx=0 tx=0).stop 2024-05-21T21:58:00.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.873+0000 7f5a37e00700 1 -- 172.21.15.151:0/3443123533 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5a30105760 msgr2=0x7f5a3019be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:00.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.873+0000 7f5a37e00700 1 --2- 172.21.15.151:0/3443123533 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5a30105760 0x7f5a3019be30 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5a2400e9f0 tx=0x7f5a2400ed00 comp rx=0 tx=0).stop 2024-05-21T21:58:00.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.873+0000 7f5a37e00700 1 -- 172.21.15.151:0/3443123533 shutdown_connections 2024-05-21T21:58:00.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.873+0000 7f5a37e00700 1 --2- 172.21.15.151:0/3443123533 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f5a180b2a70 0x7f5a180b4f30 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:00.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.873+0000 7f5a37e00700 1 --2- 172.21.15.151:0/3443123533 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5a30105760 0x7f5a3019be30 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:00.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.873+0000 7f5a37e00700 1 --2- 172.21.15.151:0/3443123533 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a3010b760 0x7f5a301a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:00.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.873+0000 7f5a37e00700 1 --2- 172.21.15.151:0/3443123533 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a30106120 0x7f5a3019c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:00.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.873+0000 7f5a37e00700 1 -- 172.21.15.151:0/3443123533 >> 172.21.15.151:0/3443123533 conn(0x7f5a30101250 msgr2=0x7f5a30101930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:58:00.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.874+0000 7f5a37e00700 1 -- 172.21.15.151:0/3443123533 shutdown_connections 2024-05-21T21:58:00.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:00.874+0000 7f5a37e00700 1 -- 172.21.15.151:0/3443123533 wait complete. 2024-05-21T21:58:01.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:58:01 smithi151 ceph-mon[31623]: pgmap v75: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T21:58:01.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:58:01 smithi184 ceph-mon[36942]: pgmap v75: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T21:58:01.749 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-21T21:58:01.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:58:01 smithi164 ceph-mon[36269]: pgmap v75: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T21:58:02.208 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:58:03.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:58:03 smithi151 ceph-mon[31623]: pgmap v76: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-05-21T21:58:03.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:58:03 smithi184 ceph-mon[36942]: pgmap v76: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-05-21T21:58:03.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:58:03 smithi164 ceph-mon[36269]: pgmap v76: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-05-21T21:58:04.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.411+0000 7fce27200700 1 -- 172.21.15.151:0/3597092733 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce20109a80 msgr2=0x7fce20109f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:04.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.411+0000 7fce27200700 1 --2- 172.21.15.151:0/3597092733 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce20109a80 0x7fce20109f00 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fce10009990 tx=0x7fce10009ca0 comp rx=0 tx=0).stop 2024-05-21T21:58:04.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.412+0000 7fce27200700 1 -- 172.21.15.151:0/3597092733 shutdown_connections 2024-05-21T21:58:04.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.412+0000 7fce27200700 1 --2- 172.21.15.151:0/3597092733 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce2010a440 0x7fce201168e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:04.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.412+0000 7fce27200700 1 --2- 172.21.15.151:0/3597092733 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce2010e950 0x7fce2010ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:04.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.412+0000 7fce27200700 1 --2- 172.21.15.151:0/3597092733 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce20109a80 0x7fce20109f00 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:04.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.412+0000 7fce27200700 1 -- 172.21.15.151:0/3597092733 >> 172.21.15.151:0/3597092733 conn(0x7fce200792d0 msgr2=0x7fce20108360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:58:04.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.412+0000 7fce27200700 1 -- 172.21.15.151:0/3597092733 shutdown_connections 2024-05-21T21:58:04.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.412+0000 7fce27200700 1 -- 172.21.15.151:0/3597092733 wait complete. 2024-05-21T21:58:04.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.413+0000 7fce27200700 1 Processor -- start 2024-05-21T21:58:04.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.413+0000 7fce27200700 1 -- start start 2024-05-21T21:58:04.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.414+0000 7fce27200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce20109a80 0x7fce201a0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:04.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.414+0000 7fce27200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce2010a440 0x7fce201a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:04.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.414+0000 7fce27200700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce2010e950 0x7fce201a4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:04.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.414+0000 7fce27200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fce20119c80 con 0x7fce20109a80 2024-05-21T21:58:04.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.414+0000 7fce27200700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fce20119b00 con 0x7fce2010a440 2024-05-21T21:58:04.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.414+0000 7fce27200700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fce20119e00 con 0x7fce2010e950 2024-05-21T21:58:04.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.414+0000 7fce25e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce20109a80 0x7fce201a0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:04.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.414+0000 7fce26800700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce2010e950 0x7fce201a4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:04.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.414+0000 7fce26800700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce2010e950 0x7fce201a4a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:36980/0 (socket says 172.21.15.151:36980) 2024-05-21T21:58:04.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.414+0000 7fce25400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce2010a440 0x7fce201a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:04.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.414+0000 7fce26800700 1 -- 172.21.15.151:0/369837677 learned_addr learned my addr 172.21.15.151:0/369837677 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:58:04.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.415+0000 7fce25400700 1 -- 172.21.15.151:0/369837677 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce2010e950 msgr2=0x7fce201a4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:04.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.415+0000 7fce25400700 1 --2- 172.21.15.151:0/369837677 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce2010e950 0x7fce201a4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:04.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.415+0000 7fce25400700 1 -- 172.21.15.151:0/369837677 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce20109a80 msgr2=0x7fce201a0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:04.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.415+0000 7fce25400700 1 --2- 172.21.15.151:0/369837677 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce20109a80 0x7fce201a0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:04.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.415+0000 7fce25400700 1 -- 172.21.15.151:0/369837677 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce201a5160 con 0x7fce2010a440 2024-05-21T21:58:04.417 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.415+0000 7fce25400700 1 --2- 172.21.15.151:0/369837677 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce2010a440 0x7fce201a0690 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fce100098e0 tx=0x7fce100057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:58:04.417 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.415+0000 7fce1aa00700 1 -- 172.21.15.151:0/369837677 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce10027070 con 0x7fce2010a440 2024-05-21T21:58:04.417 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.415+0000 7fce1aa00700 1 -- 172.21.15.151:0/369837677 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce10022470 con 0x7fce2010a440 2024-05-21T21:58:04.417 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.415+0000 7fce27200700 1 -- 172.21.15.151:0/369837677 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce201a53f0 con 0x7fce2010a440 2024-05-21T21:58:04.417 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.415+0000 7fce1aa00700 1 -- 172.21.15.151:0/369837677 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce10017650 con 0x7fce2010a440 2024-05-21T21:58:04.417 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.416+0000 7fce27200700 1 -- 172.21.15.151:0/369837677 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce201accd0 con 0x7fce2010a440 2024-05-21T21:58:04.419 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.419+0000 7fce1aa00700 1 -- 172.21.15.151:0/369837677 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7fce1002a070 con 0x7fce2010a440 2024-05-21T21:58:04.420 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.420+0000 7fce33166b80 1 Processor -- start 2024-05-21T21:58:04.420 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.420+0000 7fce33166b80 1 -- start start 2024-05-21T21:58:04.422 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.421+0000 7fce33166b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559f6614c020 0x559f6614c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:04.422 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.421+0000 7fce33166b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559f6614c940 0x559f661464b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:04.422 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.421+0000 7fce33166b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559f66146ad0 0x559f66146f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:04.422 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.421+0000 7fce1aa00700 1 --2- 172.21.15.151:0/369837677 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fce080b29f0 0x7fce080b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:04.422 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.421+0000 7fce33166b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x559f660a4e40 con 0x559f6614c940 2024-05-21T21:58:04.422 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.421+0000 7fce33166b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x559f660949e0 con 0x559f6614c020 2024-05-21T21:58:04.422 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.421+0000 7fce33166b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x559f66098a20 con 0x559f66146ad0 2024-05-21T21:58:04.422 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.421+0000 7fce1aa00700 1 -- 172.21.15.151:0/369837677 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fce100e3340 con 0x7fce2010a440 2024-05-21T21:58:04.422 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.421+0000 7fce25e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559f6614c940 0x559f661464b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:04.422 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.421+0000 7fce25e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559f6614c940 0x559f661464b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:53754/0 (socket says 172.21.15.151:53754) 2024-05-21T21:58:04.422 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.421+0000 7fce25e00700 1 -- 172.21.15.151:0/219172757 learned_addr learned my addr 172.21.15.151:0/219172757 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:58:04.423 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.421+0000 7fce25400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559f66146ad0 0x559f66146f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:04.423 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.421+0000 7fce26800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559f6614c020 0x559f6614c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:04.423 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.422+0000 7fce25400700 1 -- 172.21.15.151:0/219172757 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559f6614c020 msgr2=0x559f6614c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:04.423 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.422+0000 7fce26800700 1 --2- 172.21.15.151:0/369837677 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fce080b29f0 0x7fce080b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:04.423 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.422+0000 7fce25400700 1 --2- 172.21.15.151:0/219172757 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559f6614c020 0x559f6614c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:04.423 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.422+0000 7fce25400700 1 -- 172.21.15.151:0/219172757 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559f6614c940 msgr2=0x559f661464b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:04.423 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.422+0000 7fce25400700 1 --2- 172.21.15.151:0/219172757 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559f6614c940 0x559f661464b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:04.423 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.422+0000 7fce25400700 1 -- 172.21.15.151:0/219172757 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f6605d420 con 0x559f66146ad0 2024-05-21T21:58:04.424 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.422+0000 7fce25400700 1 --2- 172.21.15.151:0/219172757 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559f66146ad0 0x559f66146f30 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fce100e0a40 tx=0x7fce10022cc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=637f6446f328555 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:58:04.424 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.422+0000 7fce0f400700 1 -- 172.21.15.151:0/219172757 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce100aad60 con 0x559f66146ad0 2024-05-21T21:58:04.424 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.422+0000 7fce0f400700 1 -- 172.21.15.151:0/219172757 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce100aa660 con 0x559f66146ad0 2024-05-21T21:58:04.424 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.422+0000 7fce0f400700 1 -- 172.21.15.151:0/219172757 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce100e0cc0 con 0x559f66146ad0 2024-05-21T21:58:04.424 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.423+0000 7fce33166b80 1 -- 172.21.15.151:0/219172757 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559f66146ad0 msgr2=0x559f66146f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:04.424 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.423+0000 7fce33166b80 1 --2- 172.21.15.151:0/219172757 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559f66146ad0 0x559f66146f30 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fce100e0a40 tx=0x7fce10022cc0 comp rx=0 tx=0).stop 2024-05-21T21:58:04.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.423+0000 7fce26800700 1 --2- 172.21.15.151:0/369837677 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fce080b29f0 0x7fce080b4eb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fce1c009550 tx=0x7fce1c00b040 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:58:04.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.423+0000 7fce33166b80 1 -- 172.21.15.151:0/219172757 shutdown_connections 2024-05-21T21:58:04.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.423+0000 7fce33166b80 1 --2- 172.21.15.151:0/219172757 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559f6614c020 0x559f6614c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:04.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.423+0000 7fce33166b80 1 --2- 172.21.15.151:0/219172757 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559f66146ad0 0x559f66146f30 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:04.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.423+0000 7fce33166b80 1 --2- 172.21.15.151:0/219172757 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559f6614c940 0x559f661464b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:04.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.423+0000 7fce33166b80 1 -- 172.21.15.151:0/219172757 >> 172.21.15.151:0/219172757 conn(0x559f66074be0 msgr2=0x559f66141150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:58:04.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.423+0000 7fce33166b80 1 -- 172.21.15.151:0/219172757 shutdown_connections 2024-05-21T21:58:04.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.423+0000 7fce33166b80 1 -- 172.21.15.151:0/219172757 wait complete. 2024-05-21T21:58:04.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.424+0000 7fce33166b80 1 -- start start 2024-05-21T21:58:04.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.424+0000 7fce33166b80 1 Processor -- start 2024-05-21T21:58:04.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.425+0000 7fce33166b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559f66146ad0 0x559f661526a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:04.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.425+0000 7fce33166b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559f6614c020 0x559f6614fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:04.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.425+0000 7fce33166b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559f6614c940 0x559f6614ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:04.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.425+0000 7fce33166b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x559f65f9afa0 con 0x559f6614c020 2024-05-21T21:58:04.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.425+0000 7fce33166b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x559f66035760 con 0x559f66146ad0 2024-05-21T21:58:04.428 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.425+0000 7fce33166b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x559f66153220 con 0x559f6614c940 2024-05-21T21:58:04.428 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.425+0000 7fce33166b80 10 client.0 _ll_register_callbacks cb 0x559f6603ec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:58:04.428 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.425+0000 7fce25e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559f66146ad0 0x559f661526a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:04.428 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.425+0000 7fce25e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559f66146ad0 0x559f661526a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:40866/0 (socket says 172.21.15.151:40866) 2024-05-21T21:58:04.428 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.425+0000 7fce25e00700 1 -- 172.21.15.151:0/2289851617 learned_addr learned my addr 172.21.15.151:0/2289851617 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:58:04.428 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.425+0000 7fce26800700 1 --2- 172.21.15.151:0/2289851617 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559f6614c940 0x559f6614ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:04.428 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.425+0000 7fce25400700 1 --2- 172.21.15.151:0/2289851617 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559f6614c020 0x559f6614fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:04.428 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.425+0000 7fce25e00700 1 -- 172.21.15.151:0/2289851617 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559f6614c940 msgr2=0x559f6614ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:04.428 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.425+0000 7fce25e00700 1 --2- 172.21.15.151:0/2289851617 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559f6614c940 0x559f6614ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:04.428 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.425+0000 7fce25e00700 1 -- 172.21.15.151:0/2289851617 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559f6614c020 msgr2=0x559f6614fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:04.428 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.425+0000 7fce25e00700 1 --2- 172.21.15.151:0/2289851617 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559f6614c020 0x559f6614fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:04.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.425+0000 7fce25e00700 1 -- 172.21.15.151:0/2289851617 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f65fe48d0 con 0x559f66146ad0 2024-05-21T21:58:04.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.426+0000 7fce25e00700 1 --2- 172.21.15.151:0/2289851617 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559f66146ad0 0x559f661526a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fce14011730 tx=0x7fce14011a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=623b8300d93ecca6 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:58:04.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.426+0000 7fce03e00700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-21T21:58:04.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.426+0000 7fce03e00700 1 -- 172.21.15.151:0/2289851617 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce1400f690 con 0x559f66146ad0 2024-05-21T21:58:04.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.426+0000 7fce03e00700 1 -- 172.21.15.151:0/2289851617 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fce1400f830 con 0x559f66146ad0 2024-05-21T21:58:04.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.426+0000 7fce33166b80 1 -- 172.21.15.151:0/2289851617 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce100aa660 con 0x559f66146ad0 2024-05-21T21:58:04.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.426+0000 7fce03e00700 1 -- 172.21.15.151:0/2289851617 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fce14027050 con 0x559f66146ad0 2024-05-21T21:58:04.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.426+0000 7fce33166b80 10 client.25823 fetch_fsmap learned FSMap version 29 2024-05-21T21:58:04.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.426+0000 7fce33166b80 1 -- 172.21.15.151:0/2289851617 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559f66065060 con 0x559f66146ad0 2024-05-21T21:58:04.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.427+0000 7fce03e00700 1 -- 172.21.15.151:0/2289851617 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce100aa660 con 0x559f66146ad0 2024-05-21T21:58:04.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.427+0000 7fce03e00700 1 -- 172.21.15.151:0/2289851617 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fce14022030 con 0x559f66146ad0 2024-05-21T21:58:04.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.427+0000 7fce33166b80 10 client.25823 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T21:58:04.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.427+0000 7fce33166b80 10 client.25823 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-21T21:58:04.430 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.427+0000 7fce33166b80 20 client.25823 populate_metadata read hostname 'smithi151' 2024-05-21T21:58:04.430 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.427+0000 7fce33166b80 1 --2- 172.21.15.151:0/2289851617 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x559f6615fd10 0x559f66162180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:04.430 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.427+0000 7fce33166b80 4 client.25823 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T21:58:04.430 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.427+0000 7fce33166b80 1 -- 172.21.15.151:0/2289851617 --> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f660a6d90 con 0x559f6615fd10 2024-05-21T21:58:04.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.428+0000 7fce25400700 1 --2- 172.21.15.151:0/2289851617 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x559f6615fd10 0x559f66162180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:04.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.428+0000 7fce03e00700 10 client.25823 ms_handle_connect on v2:172.21.15.184:6836/2988194888 2024-05-21T21:58:04.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.428+0000 7fce25400700 1 --2- 172.21.15.151:0/2289851617 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x559f6615fd10 0x559f66162180 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9bb225992abed61f server_cookie=6b58230aa16a36e3 in_seq=0 out_seq=0 2024-05-21T21:58:04.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.429+0000 7fce03e00700 1 -- 172.21.15.151:0/2289851617 <== mds.3 v2:172.21.15.184:6836/2988194888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x559f660a6d90 con 0x559f6615fd10 2024-05-21T21:58:04.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.429+0000 7fce03e00700 10 client.25823 handle_command_reply: tid=0 2024-05-21T21:58:04.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.429+0000 7fce33166b80 1 client.25823 shutdown 2024-05-21T21:58:04.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.429+0000 7fce33166b80 1 -- 172.21.15.151:0/2289851617 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559f66146ad0 msgr2=0x559f661526a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:04.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.429+0000 7fce33166b80 1 --2- 172.21.15.151:0/2289851617 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559f66146ad0 0x559f661526a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fce14011730 tx=0x7fce14011a40 comp rx=0 tx=0).stop 2024-05-21T21:58:04.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.429+0000 7fce33166b80 1 -- 172.21.15.151:0/2289851617 shutdown_connections 2024-05-21T21:58:04.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.429+0000 7fce33166b80 1 --2- 172.21.15.151:0/2289851617 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x559f6615fd10 0x559f66162180 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:04.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.429+0000 7fce33166b80 1 --2- 172.21.15.151:0/2289851617 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559f66146ad0 0x559f661526a0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:04.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.429+0000 7fce33166b80 1 --2- 172.21.15.151:0/2289851617 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559f6614c940 0x559f6614ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:04.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.429+0000 7fce33166b80 1 --2- 172.21.15.151:0/2289851617 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559f6614c020 0x559f6614fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:04.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.429+0000 7fce33166b80 1 -- 172.21.15.151:0/2289851617 >> 172.21.15.151:0/2289851617 conn(0x559f66074be0 msgr2=0x559f661454a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:58:04.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.430+0000 7fce03e00700 0 client.25823 ms_handle_reset on v2:172.21.15.184:6836/2988194888 2024-05-21T21:58:04.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.430+0000 7fce33166b80 1 -- 172.21.15.151:0/2289851617 shutdown_connections 2024-05-21T21:58:04.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.430+0000 7fce33166b80 1 -- 172.21.15.151:0/2289851617 wait complete. 2024-05-21T21:58:04.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.430+0000 7fce33166b80 20 client.25823 trim_cache size 0 max 16384 2024-05-21T21:58:04.447 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.446+0000 7fce33166b80 1 Processor -- start 2024-05-21T21:58:04.447 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.446+0000 7fce33166b80 1 -- start start 2024-05-21T21:58:04.447 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.447+0000 7fce33166b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559f66146ad0 0x559f6614c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:04.447 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.447+0000 7fce33166b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559f6614c020 0x559f66145ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:04.447 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.447+0000 7fce33166b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559f6614c940 0x559f66146430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:04.447 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.447+0000 7fce33166b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fce201b4250 con 0x559f6614c020 2024-05-21T21:58:04.447 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.447+0000 7fce33166b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fce201b5d90 con 0x559f6614c940 2024-05-21T21:58:04.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.447+0000 7fce33166b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fce201b4920 con 0x559f66146ad0 2024-05-21T21:58:04.449 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.447+0000 7fce25400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559f6614c020 0x559f66145ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:04.449 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.447+0000 7fce26800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559f6614c940 0x559f66146430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:04.449 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.447+0000 7fce25400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559f6614c020 0x559f66145ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:53776/0 (socket says 172.21.15.151:53776) 2024-05-21T21:58:04.449 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.447+0000 7fce25400700 1 -- 172.21.15.151:0/3334753431 learned_addr learned my addr 172.21.15.151:0/3334753431 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:58:04.449 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.447+0000 7fce26800700 1 --2- 172.21.15.151:0/3334753431 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559f6614c940 0x559f66146430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:40882/0 (socket says 172.21.15.151:40882) 2024-05-21T21:58:04.449 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.447+0000 7fce25e00700 1 --2- 172.21.15.151:0/3334753431 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559f66146ad0 0x559f6614c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:04.449 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.448+0000 7fce25400700 1 -- 172.21.15.151:0/3334753431 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559f66146ad0 msgr2=0x559f6614c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:04.449 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.448+0000 7fce25400700 1 --2- 172.21.15.151:0/3334753431 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559f66146ad0 0x559f6614c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:04.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.448+0000 7fce25400700 1 -- 172.21.15.151:0/3334753431 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559f6614c940 msgr2=0x559f66146430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:04.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.448+0000 7fce25400700 1 --2- 172.21.15.151:0/3334753431 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559f6614c940 0x559f66146430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:04.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.448+0000 7fce25400700 1 -- 172.21.15.151:0/3334753431 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f6613d5c0 con 0x559f6614c020 2024-05-21T21:58:04.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.448+0000 7fce25e00700 1 --2- 172.21.15.151:0/3334753431 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559f66146ad0 0x559f6614c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:58:04.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.448+0000 7fce25400700 1 --2- 172.21.15.151:0/3334753431 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559f6614c020 0x559f66145ef0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fce100e0aa0 tx=0x7fce100cd040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86fa6356a5385080 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:58:04.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.448+0000 7fce0ea00700 1 -- 172.21.15.151:0/3334753431 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce100aea10 con 0x559f6614c020 2024-05-21T21:58:04.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.448+0000 7fce0ea00700 1 -- 172.21.15.151:0/3334753431 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce100b3430 con 0x559f6614c020 2024-05-21T21:58:04.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.448+0000 7fce0ea00700 1 -- 172.21.15.151:0/3334753431 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce100e0cc0 con 0x559f6614c020 2024-05-21T21:58:04.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.449+0000 7fce33166b80 1 -- 172.21.15.151:0/3334753431 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559f6614c020 msgr2=0x559f66145ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:04.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.449+0000 7fce33166b80 1 --2- 172.21.15.151:0/3334753431 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559f6614c020 0x559f66145ef0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fce100e0aa0 tx=0x7fce100cd040 comp rx=0 tx=0).stop 2024-05-21T21:58:04.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.449+0000 7fce33166b80 1 -- 172.21.15.151:0/3334753431 shutdown_connections 2024-05-21T21:58:04.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.449+0000 7fce33166b80 1 --2- 172.21.15.151:0/3334753431 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559f6614c940 0x559f66146430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:04.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.449+0000 7fce33166b80 1 --2- 172.21.15.151:0/3334753431 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559f66146ad0 0x559f6614c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:04.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.449+0000 7fce33166b80 1 --2- 172.21.15.151:0/3334753431 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559f6614c020 0x559f66145ef0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:04.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.449+0000 7fce33166b80 1 -- 172.21.15.151:0/3334753431 >> 172.21.15.151:0/3334753431 conn(0x559f66074be0 msgr2=0x559f661454a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:58:04.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.449+0000 7fce33166b80 1 -- 172.21.15.151:0/3334753431 shutdown_connections 2024-05-21T21:58:04.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.450+0000 7fce33166b80 1 -- 172.21.15.151:0/3334753431 wait complete. 2024-05-21T21:58:04.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.450+0000 7fce33166b80 1 -- start start 2024-05-21T21:58:04.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.450+0000 7fce33166b80 1 Processor -- start 2024-05-21T21:58:04.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.451+0000 7fce33166b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559f66146ad0 0x559f66158470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:04.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.451+0000 7fce33166b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559f6614c020 0x559f661589b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:04.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.451+0000 7fce33166b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559f6614c940 0x559f66166a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:04.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.451+0000 7fce33166b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fce201b56c0 con 0x559f6614c020 2024-05-21T21:58:04.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.451+0000 7fce33166b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fce201b4ff0 con 0x559f6614c940 2024-05-21T21:58:04.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.451+0000 7fce33166b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fce201b21f0 con 0x559f66146ad0 2024-05-21T21:58:04.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.451+0000 7fce33166b80 10 client.0 _ll_register_callbacks cb 0x559f6603ec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:58:04.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.451+0000 7fce25400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559f6614c020 0x559f661589b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:04.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.451+0000 7fce26800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559f6614c940 0x559f66166a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:04.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.451+0000 7fce26800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559f6614c940 0x559f66166a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:40892/0 (socket says 172.21.15.151:40892) 2024-05-21T21:58:04.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.451+0000 7fce25400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559f6614c020 0x559f661589b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:53790/0 (socket says 172.21.15.151:53790) 2024-05-21T21:58:04.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.451+0000 7fce26800700 1 -- 172.21.15.151:0/2872619460 learned_addr learned my addr 172.21.15.151:0/2872619460 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:58:04.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.451+0000 7fce25e00700 1 --2- 172.21.15.151:0/2872619460 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559f66146ad0 0x559f66158470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:04.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.451+0000 7fce26800700 1 -- 172.21.15.151:0/2872619460 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559f66146ad0 msgr2=0x559f66158470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:04.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.451+0000 7fce26800700 1 --2- 172.21.15.151:0/2872619460 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559f66146ad0 0x559f66158470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:04.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.451+0000 7fce26800700 1 -- 172.21.15.151:0/2872619460 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559f6614c020 msgr2=0x559f661589b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:04.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.451+0000 7fce26800700 1 --2- 172.21.15.151:0/2872619460 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559f6614c020 0x559f661589b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:04.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.452+0000 7fce26800700 1 -- 172.21.15.151:0/2872619460 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f66166f80 con 0x559f6614c940 2024-05-21T21:58:04.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.452+0000 7fce25400700 1 --2- 172.21.15.151:0/2872619460 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559f6614c020 0x559f661589b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:58:04.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.452+0000 7fce26800700 1 --2- 172.21.15.151:0/2872619460 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559f6614c940 0x559f66166a40 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fce1c01ad00 tx=0x7fce1c004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48eb4d077c706c10 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:58:04.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.452+0000 7fce03e00700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-21T21:58:04.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.452+0000 7fce03e00700 1 -- 172.21.15.151:0/2872619460 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce1c029030 con 0x559f6614c940 2024-05-21T21:58:04.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.452+0000 7fce03e00700 1 -- 172.21.15.151:0/2872619460 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fce1c012040 con 0x559f6614c940 2024-05-21T21:58:04.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.452+0000 7fce33166b80 1 -- 172.21.15.151:0/2872619460 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce100b3430 con 0x559f6614c940 2024-05-21T21:58:04.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.453+0000 7fce03e00700 1 -- 172.21.15.151:0/2872619460 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fce1c033050 con 0x559f6614c940 2024-05-21T21:58:04.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.453+0000 7fce33166b80 10 client.25829 fetch_fsmap learned FSMap version 29 2024-05-21T21:58:04.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.453+0000 7fce33166b80 1 -- 172.21.15.151:0/2872619460 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559f661ca0a0 con 0x559f6614c940 2024-05-21T21:58:04.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.453+0000 7fce03e00700 1 -- 172.21.15.151:0/2872619460 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce100b3430 con 0x559f6614c940 2024-05-21T21:58:04.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.453+0000 7fce03e00700 1 -- 172.21.15.151:0/2872619460 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fce1c004020 con 0x559f6614c940 2024-05-21T21:58:04.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.453+0000 7fce33166b80 10 client.25829 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T21:58:04.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.453+0000 7fce33166b80 10 client.25829 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-21T21:58:04.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.453+0000 7fce33166b80 20 client.25829 populate_metadata read hostname 'smithi151' 2024-05-21T21:58:04.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.453+0000 7fce33166b80 1 --2- 172.21.15.151:0/2872619460 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x559f661ca830 0x559f661ccc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:04.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.453+0000 7fce33166b80 4 client.25829 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T21:58:04.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.453+0000 7fce33166b80 1 -- 172.21.15.151:0/2872619460 --> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559f660a0570 con 0x559f661ca830 2024-05-21T21:58:04.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.454+0000 7fce25e00700 1 --2- 172.21.15.151:0/2872619460 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x559f661ca830 0x559f661ccc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:04.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.454+0000 7fce25e00700 1 --2- 172.21.15.151:0/2872619460 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x559f661ca830 0x559f661ccc50 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e40db13e6dabb1fb server_cookie=f4a9953e8067ecaa in_seq=0 out_seq=0 2024-05-21T21:58:04.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.454+0000 7fce03e00700 10 client.25829 ms_handle_connect on v2:172.21.15.184:6836/2988194888 2024-05-21T21:58:04.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.455+0000 7fce03e00700 1 -- 172.21.15.151:0/2872619460 <== mds.3 v2:172.21.15.184:6836/2988194888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559f660a0570 con 0x559f661ca830 2024-05-21T21:58:04.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.455+0000 7fce03e00700 10 client.25829 handle_command_reply: tid=0 2024-05-21T21:58:04.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.455+0000 7fce33166b80 1 client.25829 shutdown 2024-05-21T21:58:04.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.455+0000 7fce33166b80 1 -- 172.21.15.151:0/2872619460 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559f6614c940 msgr2=0x559f66166a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:04.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.455+0000 7fce33166b80 1 --2- 172.21.15.151:0/2872619460 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559f6614c940 0x559f66166a40 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fce1c01ad00 tx=0x7fce1c004b40 comp rx=0 tx=0).stop 2024-05-21T21:58:04.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.455+0000 7fce33166b80 1 -- 172.21.15.151:0/2872619460 shutdown_connections 2024-05-21T21:58:04.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.455+0000 7fce33166b80 1 --2- 172.21.15.151:0/2872619460 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x559f661ca830 0x559f661ccc50 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:04.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.455+0000 7fce33166b80 1 --2- 172.21.15.151:0/2872619460 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559f6614c940 0x559f66166a40 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:04.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.455+0000 7fce33166b80 1 --2- 172.21.15.151:0/2872619460 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559f66146ad0 0x559f66158470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:04.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.455+0000 7fce03e00700 0 client.25829 ms_handle_reset on v2:172.21.15.184:6836/2988194888 2024-05-21T21:58:04.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.455+0000 7fce33166b80 1 --2- 172.21.15.151:0/2872619460 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559f6614c020 0x559f661589b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:04.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.455+0000 7fce33166b80 1 -- 172.21.15.151:0/2872619460 >> 172.21.15.151:0/2872619460 conn(0x559f66074be0 msgr2=0x559f66140c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:58:04.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.455+0000 7fce33166b80 1 -- 172.21.15.151:0/2872619460 shutdown_connections 2024-05-21T21:58:04.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.455+0000 7fce33166b80 1 -- 172.21.15.151:0/2872619460 wait complete. 2024-05-21T21:58:04.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.455+0000 7fce33166b80 20 client.25829 trim_cache size 0 max 16384 2024-05-21T21:58:04.459 INFO:teuthology.orchestra.run.smithi151.stdout:[] 2024-05-21T21:58:04.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.456+0000 7fce27200700 1 -- 172.21.15.151:0/369837677 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fce080b29f0 msgr2=0x7fce080b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:04.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.456+0000 7fce27200700 1 --2- 172.21.15.151:0/369837677 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fce080b29f0 0x7fce080b4eb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fce1c009550 tx=0x7fce1c00b040 comp rx=0 tx=0).stop 2024-05-21T21:58:04.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.456+0000 7fce27200700 1 -- 172.21.15.151:0/369837677 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce2010a440 msgr2=0x7fce201a0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:04.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.456+0000 7fce27200700 1 --2- 172.21.15.151:0/369837677 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce2010a440 0x7fce201a0690 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fce100098e0 tx=0x7fce100057c0 comp rx=0 tx=0).stop 2024-05-21T21:58:04.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.456+0000 7fce27200700 1 -- 172.21.15.151:0/369837677 shutdown_connections 2024-05-21T21:58:04.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.456+0000 7fce27200700 1 --2- 172.21.15.151:0/369837677 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fce080b29f0 0x7fce080b4eb0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:04.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.456+0000 7fce27200700 1 --2- 172.21.15.151:0/369837677 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce2010e950 0x7fce201a4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:04.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.456+0000 7fce27200700 1 --2- 172.21.15.151:0/369837677 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce20109a80 0x7fce201a0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:04.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.456+0000 7fce27200700 1 --2- 172.21.15.151:0/369837677 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce2010a440 0x7fce201a0690 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:04.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.456+0000 7fce27200700 1 -- 172.21.15.151:0/369837677 >> 172.21.15.151:0/369837677 conn(0x7fce200792d0 msgr2=0x7fce20079990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:58:04.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.456+0000 7fce27200700 1 -- 172.21.15.151:0/369837677 shutdown_connections 2024-05-21T21:58:04.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:04.456+0000 7fce27200700 1 -- 172.21.15.151:0/369837677 wait complete. 2024-05-21T21:58:05.217 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-21T21:58:05.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:58:05 smithi151 ceph-mon[31623]: pgmap v77: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-05-21T21:58:05.669 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:58:05.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:58:05 smithi184 ceph-mon[36942]: pgmap v77: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-05-21T21:58:05.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:58:05 smithi164 ceph-mon[36269]: pgmap v77: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-05-21T21:58:07.553 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:58:07 smithi151 ceph-mon[31623]: pgmap v78: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-05-21T21:58:07.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:58:07 smithi184 ceph-mon[36942]: pgmap v78: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-05-21T21:58:07.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:58:07 smithi164 ceph-mon[36269]: pgmap v78: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-05-21T21:58:07.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.851+0000 7fa92f600700 1 -- 172.21.15.151:0/2384401663 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa928108490 msgr2=0x7fa928108870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:07.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.851+0000 7fa92f600700 1 --2- 172.21.15.151:0/2384401663 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa928108490 0x7fa928108870 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa91c009990 tx=0x7fa91c009ca0 comp rx=0 tx=0).stop 2024-05-21T21:58:07.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.852+0000 7fa92f600700 1 -- 172.21.15.151:0/2384401663 shutdown_connections 2024-05-21T21:58:07.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.852+0000 7fa92f600700 1 --2- 172.21.15.151:0/2384401663 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa92806c7a0 0x7fa928105fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:07.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.852+0000 7fa92f600700 1 --2- 172.21.15.151:0/2384401663 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa928108490 0x7fa928108870 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:07.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.852+0000 7fa92f600700 1 --2- 172.21.15.151:0/2384401663 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa92806bc10 0x7fa92806c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:07.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.852+0000 7fa92f600700 1 -- 172.21.15.151:0/2384401663 >> 172.21.15.151:0/2384401663 conn(0x7fa9280793f0 msgr2=0x7fa928079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:58:07.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.852+0000 7fa92f600700 1 -- 172.21.15.151:0/2384401663 shutdown_connections 2024-05-21T21:58:07.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.852+0000 7fa92f600700 1 -- 172.21.15.151:0/2384401663 wait complete. 2024-05-21T21:58:07.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.853+0000 7fa92f600700 1 Processor -- start 2024-05-21T21:58:07.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.853+0000 7fa92f600700 1 -- start start 2024-05-21T21:58:07.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.854+0000 7fa92f600700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa92806bc10 0x7fa9281a0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:07.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.854+0000 7fa92f600700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa92806c7a0 0x7fa9281a0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:07.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.854+0000 7fa92f600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa928108490 0x7fa9281a4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:07.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.854+0000 7fa92f600700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa928119d40 con 0x7fa928108490 2024-05-21T21:58:07.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.854+0000 7fa92f600700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa928119bc0 con 0x7fa92806bc10 2024-05-21T21:58:07.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.854+0000 7fa92f600700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa928119ec0 con 0x7fa92806c7a0 2024-05-21T21:58:07.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.854+0000 7fa92cc00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa92806bc10 0x7fa9281a0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:07.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.854+0000 7fa92d600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa928108490 0x7fa9281a4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:07.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.854+0000 7fa92cc00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa92806bc10 0x7fa9281a0250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:34018/0 (socket says 172.21.15.151:34018) 2024-05-21T21:58:07.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.854+0000 7fa92cc00700 1 -- 172.21.15.151:0/1125819987 learned_addr learned my addr 172.21.15.151:0/1125819987 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:58:07.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.854+0000 7fa927e00700 1 --2- 172.21.15.151:0/1125819987 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa92806c7a0 0x7fa9281a0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:07.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.855+0000 7fa92cc00700 1 -- 172.21.15.151:0/1125819987 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa92806c7a0 msgr2=0x7fa9281a0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:07.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.855+0000 7fa92cc00700 1 --2- 172.21.15.151:0/1125819987 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa92806c7a0 0x7fa9281a0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:07.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.855+0000 7fa92cc00700 1 -- 172.21.15.151:0/1125819987 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa928108490 msgr2=0x7fa9281a4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:07.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.855+0000 7fa92cc00700 1 --2- 172.21.15.151:0/1125819987 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa928108490 0x7fa9281a4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:07.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.855+0000 7fa92cc00700 1 -- 172.21.15.151:0/1125819987 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9281a5260 con 0x7fa92806bc10 2024-05-21T21:58:07.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.855+0000 7fa92d600700 1 --2- 172.21.15.151:0/1125819987 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa928108490 0x7fa9281a4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:58:07.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.855+0000 7fa92cc00700 1 --2- 172.21.15.151:0/1125819987 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa92806bc10 0x7fa9281a0250 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa91c000c00 tx=0x7fa91c00bfd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:58:07.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.855+0000 7fa925600700 1 -- 172.21.15.151:0/1125819987 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa91c027070 con 0x7fa92806bc10 2024-05-21T21:58:07.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.856+0000 7fa925600700 1 -- 172.21.15.151:0/1125819987 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa91c022470 con 0x7fa92806bc10 2024-05-21T21:58:07.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.856+0000 7fa92f600700 1 -- 172.21.15.151:0/1125819987 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9281a54f0 con 0x7fa92806bc10 2024-05-21T21:58:07.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.856+0000 7fa925600700 1 -- 172.21.15.151:0/1125819987 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa91c017650 con 0x7fa92806bc10 2024-05-21T21:58:07.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.856+0000 7fa92f600700 1 -- 172.21.15.151:0/1125819987 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9281acdd0 con 0x7fa92806bc10 2024-05-21T21:58:07.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.859+0000 7fa925600700 1 -- 172.21.15.151:0/1125819987 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7fa91c02a070 con 0x7fa92806bc10 2024-05-21T21:58:07.861 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.861+0000 7fa939fcdb80 1 Processor -- start 2024-05-21T21:58:07.861 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.861+0000 7fa939fcdb80 1 -- start start 2024-05-21T21:58:07.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.861+0000 7fa939fcdb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a384f24eb0 0x55a384f25290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:07.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.862+0000 7fa939fcdb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a384f257d0 0x55a384f1f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:07.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.862+0000 7fa939fcdb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a384f1f880 0x55a384f1fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:07.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.862+0000 7fa939fcdb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a384e7de40 con 0x55a384f1f880 2024-05-21T21:58:07.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.862+0000 7fa939fcdb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55a384e6d9e0 con 0x55a384f257d0 2024-05-21T21:58:07.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.862+0000 7fa939fcdb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55a384e71a20 con 0x55a384f24eb0 2024-05-21T21:58:07.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.862+0000 7fa92cc00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a384f257d0 0x55a384f1f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:07.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.862+0000 7fa925600700 1 --2- 172.21.15.151:0/1125819987 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fa9100b29f0 0x7fa9100b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:07.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.862+0000 7fa92cc00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a384f257d0 0x55a384f1f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:34030/0 (socket says 172.21.15.151:34030) 2024-05-21T21:58:07.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.862+0000 7fa92cc00700 1 -- 172.21.15.151:0/2531588777 learned_addr learned my addr 172.21.15.151:0/2531588777 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:58:07.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.862+0000 7fa927e00700 1 --2- 172.21.15.151:0/2531588777 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a384f1f880 0x55a384f1fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:07.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.862+0000 7fa92d600700 1 --2- 172.21.15.151:0/2531588777 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a384f24eb0 0x55a384f25290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:07.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.862+0000 7fa92cc00700 1 -- 172.21.15.151:0/2531588777 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a384f24eb0 msgr2=0x55a384f25290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:07.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.863+0000 7fa92cc00700 1 --2- 172.21.15.151:0/2531588777 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a384f24eb0 0x55a384f25290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:07.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.863+0000 7fa92cc00700 1 -- 172.21.15.151:0/2531588777 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a384f1f880 msgr2=0x55a384f1fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:07.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.863+0000 7fa92cc00700 1 --2- 172.21.15.151:0/2531588777 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a384f1f880 0x55a384f1fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:07.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.863+0000 7fa92d600700 1 --2- 172.21.15.151:0/1125819987 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fa9100b29f0 0x7fa9100b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:07.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.863+0000 7fa925600700 1 -- 172.21.15.151:0/1125819987 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fa91c0e3390 con 0x7fa92806bc10 2024-05-21T21:58:07.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.863+0000 7fa92cc00700 1 -- 172.21.15.151:0/2531588777 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a384e36420 con 0x55a384f257d0 2024-05-21T21:58:07.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.863+0000 7fa927e00700 1 --2- 172.21.15.151:0/2531588777 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a384f1f880 0x55a384f1fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:58:07.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.863+0000 7fa92cc00700 1 --2- 172.21.15.151:0/2531588777 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a384f257d0 0x55a384f1f260 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa91c0a7920 tx=0x7fa91c009710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a4d6ce8391cd157 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:58:07.865 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.863+0000 7fa91a000700 1 -- 172.21.15.151:0/2531588777 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa91c0aa000 con 0x55a384f257d0 2024-05-21T21:58:07.865 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.863+0000 7fa91a000700 1 -- 172.21.15.151:0/2531588777 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa91c0aa1c0 con 0x55a384f257d0 2024-05-21T21:58:07.865 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.863+0000 7fa91a000700 1 -- 172.21.15.151:0/2531588777 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa91c0a9e20 con 0x55a384f257d0 2024-05-21T21:58:07.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.864+0000 7fa939fcdb80 1 -- 172.21.15.151:0/2531588777 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a384f257d0 msgr2=0x55a384f1f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:07.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.864+0000 7fa92d600700 1 --2- 172.21.15.151:0/1125819987 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fa9100b29f0 0x7fa9100b4eb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa920009fd0 tx=0x7fa92000b040 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:58:07.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.864+0000 7fa939fcdb80 1 --2- 172.21.15.151:0/2531588777 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a384f257d0 0x55a384f1f260 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa91c0a7920 tx=0x7fa91c009710 comp rx=0 tx=0).stop 2024-05-21T21:58:07.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.864+0000 7fa939fcdb80 1 -- 172.21.15.151:0/2531588777 shutdown_connections 2024-05-21T21:58:07.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.864+0000 7fa939fcdb80 1 --2- 172.21.15.151:0/2531588777 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a384f1f880 0x55a384f1fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:07.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.864+0000 7fa939fcdb80 1 --2- 172.21.15.151:0/2531588777 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a384f24eb0 0x55a384f25290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:07.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.864+0000 7fa939fcdb80 1 --2- 172.21.15.151:0/2531588777 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a384f257d0 0x55a384f1f260 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:07.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.864+0000 7fa939fcdb80 1 -- 172.21.15.151:0/2531588777 >> 172.21.15.151:0/2531588777 conn(0x55a384e4dbe0 msgr2=0x55a384f1a010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:58:07.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.865+0000 7fa939fcdb80 1 -- 172.21.15.151:0/2531588777 shutdown_connections 2024-05-21T21:58:07.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.865+0000 7fa939fcdb80 1 -- 172.21.15.151:0/2531588777 wait complete. 2024-05-21T21:58:07.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.865+0000 7fa939fcdb80 1 -- start start 2024-05-21T21:58:07.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.865+0000 7fa939fcdb80 1 Processor -- start 2024-05-21T21:58:07.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.866+0000 7fa939fcdb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a384f1f880 0x55a384f2b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:07.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.866+0000 7fa939fcdb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a384f24eb0 0x55a384f287d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:07.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.866+0000 7fa939fcdb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a384f257d0 0x55a384f28d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:07.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.866+0000 7fa939fcdb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a384d73fa0 con 0x55a384f1f880 2024-05-21T21:58:07.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.866+0000 7fa939fcdb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55a384e0e760 con 0x55a384f257d0 2024-05-21T21:58:07.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.866+0000 7fa939fcdb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55a384f2c000 con 0x55a384f24eb0 2024-05-21T21:58:07.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.866+0000 7fa939fcdb80 10 client.0 _ll_register_callbacks cb 0x55a384e17c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:58:07.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.866+0000 7fa92cc00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a384f1f880 0x55a384f2b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:07.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.866+0000 7fa92cc00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a384f1f880 0x55a384f2b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:57196/0 (socket says 172.21.15.151:57196) 2024-05-21T21:58:07.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.866+0000 7fa92cc00700 1 -- 172.21.15.151:0/3561107168 learned_addr learned my addr 172.21.15.151:0/3561107168 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:58:07.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.866+0000 7fa927e00700 1 --2- 172.21.15.151:0/3561107168 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a384f24eb0 0x55a384f287d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:07.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.866+0000 7fa92d600700 1 --2- 172.21.15.151:0/3561107168 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a384f257d0 0x55a384f28d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:07.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.867+0000 7fa92cc00700 1 -- 172.21.15.151:0/3561107168 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a384f24eb0 msgr2=0x55a384f287d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:07.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.867+0000 7fa92cc00700 1 --2- 172.21.15.151:0/3561107168 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a384f24eb0 0x55a384f287d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:07.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.867+0000 7fa92cc00700 1 -- 172.21.15.151:0/3561107168 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a384f257d0 msgr2=0x55a384f28d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:07.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.867+0000 7fa92cc00700 1 --2- 172.21.15.151:0/3561107168 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a384f257d0 0x55a384f28d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:07.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.867+0000 7fa92cc00700 1 -- 172.21.15.151:0/3561107168 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a384dbd8d0 con 0x55a384f1f880 2024-05-21T21:58:07.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.867+0000 7fa92cc00700 1 --2- 172.21.15.151:0/3561107168 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a384f1f880 0x55a384f2b430 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa91c0b1040 tx=0x7fa91c0aac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bd71568fc223fbe server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:58:07.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.867+0000 7fa90ea00700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-21T21:58:07.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.867+0000 7fa90ea00700 1 -- 172.21.15.151:0/3561107168 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa91c0a9e20 con 0x55a384f1f880 2024-05-21T21:58:07.871 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.867+0000 7fa90ea00700 1 -- 172.21.15.151:0/3561107168 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fa91c0e6ad0 con 0x55a384f1f880 2024-05-21T21:58:07.871 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.867+0000 7fa939fcdb80 1 -- 172.21.15.151:0/3561107168 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa91c0aa1c0 con 0x55a384f1f880 2024-05-21T21:58:07.871 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.868+0000 7fa90ea00700 1 -- 172.21.15.151:0/3561107168 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa91c0aa000 con 0x55a384f1f880 2024-05-21T21:58:07.871 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.868+0000 7fa939fcdb80 10 client.15714 fetch_fsmap learned FSMap version 29 2024-05-21T21:58:07.871 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.868+0000 7fa939fcdb80 1 -- 172.21.15.151:0/3561107168 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a384e3e060 con 0x55a384f1f880 2024-05-21T21:58:07.871 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.868+0000 7fa90ea00700 1 -- 172.21.15.151:0/3561107168 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa91c0aa1c0 con 0x55a384f1f880 2024-05-21T21:58:07.871 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.868+0000 7fa90ea00700 1 -- 172.21.15.151:0/3561107168 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fa91c092070 con 0x55a384f1f880 2024-05-21T21:58:07.871 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.868+0000 7fa939fcdb80 10 client.15714 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T21:58:07.871 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.868+0000 7fa939fcdb80 10 client.15714 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-05-21T21:58:07.871 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.868+0000 7fa939fcdb80 20 client.15714 populate_metadata read hostname 'smithi151' 2024-05-21T21:58:07.871 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.868+0000 7fa939fcdb80 1 --2- 172.21.15.151:0/3561107168 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x55a384f38830 0x55a384f3ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:07.871 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.868+0000 7fa939fcdb80 4 client.15714 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T21:58:07.871 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.868+0000 7fa939fcdb80 1 -- 172.21.15.151:0/3561107168 --> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a384e7fd90 con 0x55a384f38830 2024-05-21T21:58:07.872 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.869+0000 7fa927e00700 1 --2- 172.21.15.151:0/3561107168 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x55a384f38830 0x55a384f3ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:07.872 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.869+0000 7fa927e00700 1 --2- 172.21.15.151:0/3561107168 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x55a384f38830 0x55a384f3ac20 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5d83d0b6fbe9606b server_cookie=5c968c924d74ff6b in_seq=0 out_seq=0 2024-05-21T21:58:07.872 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.869+0000 7fa90ea00700 10 client.15714 ms_handle_connect on v2:172.21.15.164:6836/1554609572 2024-05-21T21:58:07.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.870+0000 7fa90ea00700 1 -- 172.21.15.151:0/3561107168 <== mds.2 v2:172.21.15.164:6836/1554609572 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55a384e7fd90 con 0x55a384f38830 2024-05-21T21:58:07.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.870+0000 7fa90ea00700 10 client.15714 handle_command_reply: tid=0 2024-05-21T21:58:07.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.870+0000 7fa939fcdb80 1 client.15714 shutdown 2024-05-21T21:58:07.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.871+0000 7fa939fcdb80 1 -- 172.21.15.151:0/3561107168 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a384f1f880 msgr2=0x55a384f2b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:07.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.871+0000 7fa939fcdb80 1 --2- 172.21.15.151:0/3561107168 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a384f1f880 0x55a384f2b430 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa91c0b1040 tx=0x7fa91c0aac20 comp rx=0 tx=0).stop 2024-05-21T21:58:07.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.871+0000 7fa939fcdb80 1 -- 172.21.15.151:0/3561107168 shutdown_connections 2024-05-21T21:58:07.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.871+0000 7fa939fcdb80 1 --2- 172.21.15.151:0/3561107168 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x55a384f38830 0x55a384f3ac20 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:07.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.871+0000 7fa939fcdb80 1 --2- 172.21.15.151:0/3561107168 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a384f257d0 0x55a384f28d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:07.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.871+0000 7fa939fcdb80 1 --2- 172.21.15.151:0/3561107168 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a384f1f880 0x55a384f2b430 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:07.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.871+0000 7fa939fcdb80 1 --2- 172.21.15.151:0/3561107168 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a384f24eb0 0x55a384f287d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:07.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.871+0000 7fa939fcdb80 1 -- 172.21.15.151:0/3561107168 >> 172.21.15.151:0/3561107168 conn(0x55a384e4dbe0 msgr2=0x55a384f1e250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:58:07.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.871+0000 7fa90ea00700 0 client.15714 ms_handle_reset on v2:172.21.15.164:6836/1554609572 2024-05-21T21:58:07.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.871+0000 7fa939fcdb80 1 -- 172.21.15.151:0/3561107168 shutdown_connections 2024-05-21T21:58:07.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.871+0000 7fa939fcdb80 1 -- 172.21.15.151:0/3561107168 wait complete. 2024-05-21T21:58:07.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.871+0000 7fa939fcdb80 20 client.15714 trim_cache size 0 max 16384 2024-05-21T21:58:07.888 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.888+0000 7fa939fcdb80 1 Processor -- start 2024-05-21T21:58:07.888 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.888+0000 7fa939fcdb80 1 -- start start 2024-05-21T21:58:07.889 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.888+0000 7fa939fcdb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a384f1f880 0x55a384f1f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:07.889 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.888+0000 7fa939fcdb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a384f24eb0 0x55a384f25c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:07.889 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.888+0000 7fa939fcdb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a384f257d0 0x55a384f261b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:07.889 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.888+0000 7fa939fcdb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa9281b4300 con 0x55a384f257d0 2024-05-21T21:58:07.889 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.888+0000 7fa939fcdb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa9281b5e40 con 0x55a384f24eb0 2024-05-21T21:58:07.889 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.888+0000 7fa939fcdb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa9281b49d0 con 0x55a384f1f880 2024-05-21T21:58:07.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.888+0000 7fa92cc00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a384f1f880 0x55a384f1f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:07.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.889+0000 7fa92cc00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a384f1f880 0x55a384f1f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:37662/0 (socket says 172.21.15.151:37662) 2024-05-21T21:58:07.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.889+0000 7fa92cc00700 1 -- 172.21.15.151:0/2058361164 learned_addr learned my addr 172.21.15.151:0/2058361164 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:58:07.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.889+0000 7fa92d600700 1 --2- 172.21.15.151:0/2058361164 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a384f257d0 0x55a384f261b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:07.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.889+0000 7fa927e00700 1 --2- 172.21.15.151:0/2058361164 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a384f24eb0 0x55a384f25c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:07.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.889+0000 7fa92cc00700 1 -- 172.21.15.151:0/2058361164 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a384f24eb0 msgr2=0x55a384f25c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:07.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.889+0000 7fa92cc00700 1 --2- 172.21.15.151:0/2058361164 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a384f24eb0 0x55a384f25c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:07.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.889+0000 7fa92cc00700 1 -- 172.21.15.151:0/2058361164 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a384f257d0 msgr2=0x55a384f261b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:07.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.889+0000 7fa92cc00700 1 --2- 172.21.15.151:0/2058361164 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a384f257d0 0x55a384f261b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:07.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.889+0000 7fa92cc00700 1 -- 172.21.15.151:0/2058361164 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a384f26d70 con 0x55a384f1f880 2024-05-21T21:58:07.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.889+0000 7fa927e00700 1 --2- 172.21.15.151:0/2058361164 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a384f24eb0 0x55a384f25c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:58:07.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.889+0000 7fa92cc00700 1 --2- 172.21.15.151:0/2058361164 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a384f1f880 0x55a384f1f260 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa91c0b1130 tx=0x7fa91c0a6c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed62f5f6427bc0be server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:58:07.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.890+0000 7fa919600700 1 -- 172.21.15.151:0/2058361164 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa91c0aadb0 con 0x55a384f1f880 2024-05-21T21:58:07.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.890+0000 7fa919600700 1 -- 172.21.15.151:0/2058361164 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa91c0aa6b0 con 0x55a384f1f880 2024-05-21T21:58:07.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.890+0000 7fa919600700 1 -- 172.21.15.151:0/2058361164 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa91c0b3250 con 0x55a384f1f880 2024-05-21T21:58:07.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.890+0000 7fa939fcdb80 1 -- 172.21.15.151:0/2058361164 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a384f1f880 msgr2=0x55a384f1f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:07.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.890+0000 7fa939fcdb80 1 --2- 172.21.15.151:0/2058361164 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a384f1f880 0x55a384f1f260 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa91c0b1130 tx=0x7fa91c0a6c10 comp rx=0 tx=0).stop 2024-05-21T21:58:07.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.890+0000 7fa939fcdb80 1 -- 172.21.15.151:0/2058361164 shutdown_connections 2024-05-21T21:58:07.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.890+0000 7fa939fcdb80 1 --2- 172.21.15.151:0/2058361164 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a384f257d0 0x55a384f261b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:07.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.890+0000 7fa939fcdb80 1 --2- 172.21.15.151:0/2058361164 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a384f1f880 0x55a384f1f260 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:07.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.890+0000 7fa939fcdb80 1 --2- 172.21.15.151:0/2058361164 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a384f24eb0 0x55a384f25c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:07.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.890+0000 7fa939fcdb80 1 -- 172.21.15.151:0/2058361164 >> 172.21.15.151:0/2058361164 conn(0x55a384e4dbe0 msgr2=0x55a384f270c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:58:07.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.890+0000 7fa939fcdb80 1 -- 172.21.15.151:0/2058361164 shutdown_connections 2024-05-21T21:58:07.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.891+0000 7fa939fcdb80 1 -- 172.21.15.151:0/2058361164 wait complete. 2024-05-21T21:58:07.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.891+0000 7fa939fcdb80 1 -- start start 2024-05-21T21:58:07.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.891+0000 7fa939fcdb80 1 Processor -- start 2024-05-21T21:58:07.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.892+0000 7fa939fcdb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a384f1f880 0x55a384f31620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:07.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.892+0000 7fa939fcdb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a384f24eb0 0x55a384f3fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:07.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.892+0000 7fa939fcdb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a384f257d0 0x55a384f400d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:07.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.892+0000 7fa939fcdb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa9281b5770 con 0x55a384f24eb0 2024-05-21T21:58:07.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.892+0000 7fa939fcdb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa9281b50a0 con 0x55a384f1f880 2024-05-21T21:58:07.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.892+0000 7fa939fcdb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa9281b22a0 con 0x55a384f257d0 2024-05-21T21:58:07.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.892+0000 7fa939fcdb80 10 client.0 _ll_register_callbacks cb 0x55a384e17c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:58:07.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.892+0000 7fa927e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a384f24eb0 0x55a384f3fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:07.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.892+0000 7fa92cc00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a384f1f880 0x55a384f31620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:07.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.892+0000 7fa92cc00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a384f1f880 0x55a384f31620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:34064/0 (socket says 172.21.15.151:34064) 2024-05-21T21:58:07.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.892+0000 7fa927e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a384f24eb0 0x55a384f3fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:57216/0 (socket says 172.21.15.151:57216) 2024-05-21T21:58:07.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.892+0000 7fa92cc00700 1 -- 172.21.15.151:0/2113836221 learned_addr learned my addr 172.21.15.151:0/2113836221 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:58:07.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.892+0000 7fa92d600700 1 --2- 172.21.15.151:0/2113836221 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a384f257d0 0x55a384f400d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:07.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.893+0000 7fa92cc00700 1 -- 172.21.15.151:0/2113836221 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a384f257d0 msgr2=0x55a384f400d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:07.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.893+0000 7fa92cc00700 1 --2- 172.21.15.151:0/2113836221 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a384f257d0 0x55a384f400d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:07.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.893+0000 7fa92cc00700 1 -- 172.21.15.151:0/2113836221 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a384f24eb0 msgr2=0x55a384f3fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:07.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.893+0000 7fa92cc00700 1 --2- 172.21.15.151:0/2113836221 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a384f24eb0 0x55a384f3fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:07.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.893+0000 7fa92cc00700 1 -- 172.21.15.151:0/2113836221 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a384f40610 con 0x55a384f1f880 2024-05-21T21:58:07.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.893+0000 7fa92d600700 1 --2- 172.21.15.151:0/2113836221 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a384f257d0 0x55a384f400d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T21:58:07.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.893+0000 7fa927e00700 1 --2- 172.21.15.151:0/2113836221 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a384f24eb0 0x55a384f3fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:58:07.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.893+0000 7fa92cc00700 1 --2- 172.21.15.151:0/2113836221 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a384f1f880 0x55a384f31620 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa91c0e6df0 tx=0x7fa91c00b280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4a7b3a7096c305d server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:58:07.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.893+0000 7fa90ea00700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-21T21:58:07.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.893+0000 7fa90ea00700 1 -- 172.21.15.151:0/2113836221 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa91c0aadb0 con 0x55a384f1f880 2024-05-21T21:58:07.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.893+0000 7fa90ea00700 1 -- 172.21.15.151:0/2113836221 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fa91c0b3200 con 0x55a384f1f880 2024-05-21T21:58:07.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.893+0000 7fa939fcdb80 1 -- 172.21.15.151:0/2113836221 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa91c0aa6b0 con 0x55a384f1f880 2024-05-21T21:58:07.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.894+0000 7fa90ea00700 1 -- 172.21.15.151:0/2113836221 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa91c0b3480 con 0x55a384f1f880 2024-05-21T21:58:07.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.894+0000 7fa939fcdb80 10 client.25847 fetch_fsmap learned FSMap version 29 2024-05-21T21:58:07.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.894+0000 7fa939fcdb80 1 -- 172.21.15.151:0/2113836221 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a384f40610 con 0x55a384f1f880 2024-05-21T21:58:07.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.894+0000 7fa90ea00700 1 -- 172.21.15.151:0/2113836221 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa91c0aa6b0 con 0x55a384f1f880 2024-05-21T21:58:07.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.894+0000 7fa90ea00700 1 -- 172.21.15.151:0/2113836221 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fa91c05d3b0 con 0x55a384f1f880 2024-05-21T21:58:07.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.894+0000 7fa939fcdb80 10 client.25847 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T21:58:07.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.894+0000 7fa939fcdb80 10 client.25847 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-05-21T21:58:07.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.894+0000 7fa939fcdb80 20 client.25847 populate_metadata read hostname 'smithi151' 2024-05-21T21:58:07.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.894+0000 7fa939fcdb80 1 --2- 172.21.15.151:0/2113836221 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x55a384fa33c0 0x55a384fa57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:07.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.894+0000 7fa939fcdb80 4 client.25847 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T21:58:07.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.894+0000 7fa939fcdb80 1 -- 172.21.15.151:0/2113836221 --> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a384e79570 con 0x55a384fa33c0 2024-05-21T21:58:07.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.894+0000 7fa927e00700 1 --2- 172.21.15.151:0/2113836221 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x55a384fa33c0 0x55a384fa57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:07.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.895+0000 7fa927e00700 1 --2- 172.21.15.151:0/2113836221 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x55a384fa33c0 0x55a384fa57e0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e3ad4f50ffb31c17 server_cookie=962e35900ec69137 in_seq=0 out_seq=0 2024-05-21T21:58:07.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.895+0000 7fa90ea00700 10 client.25847 ms_handle_connect on v2:172.21.15.164:6836/1554609572 2024-05-21T21:58:07.899 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.895+0000 7fa90ea00700 1 -- 172.21.15.151:0/2113836221 <== mds.2 v2:172.21.15.164:6836/1554609572 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a384e79570 con 0x55a384fa33c0 2024-05-21T21:58:07.899 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.895+0000 7fa90ea00700 10 client.25847 handle_command_reply: tid=0 2024-05-21T21:58:07.899 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.895+0000 7fa939fcdb80 1 client.25847 shutdown 2024-05-21T21:58:07.899 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.895+0000 7fa939fcdb80 1 -- 172.21.15.151:0/2113836221 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a384f1f880 msgr2=0x55a384f31620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:07.899 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.895+0000 7fa939fcdb80 1 --2- 172.21.15.151:0/2113836221 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a384f1f880 0x55a384f31620 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa91c0e6df0 tx=0x7fa91c00b280 comp rx=0 tx=0).stop 2024-05-21T21:58:07.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.895+0000 7fa939fcdb80 1 -- 172.21.15.151:0/2113836221 shutdown_connections 2024-05-21T21:58:07.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.895+0000 7fa939fcdb80 1 --2- 172.21.15.151:0/2113836221 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x55a384fa33c0 0x55a384fa57e0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:07.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.895+0000 7fa939fcdb80 1 --2- 172.21.15.151:0/2113836221 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a384f1f880 0x55a384f31620 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:07.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.895+0000 7fa939fcdb80 1 --2- 172.21.15.151:0/2113836221 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a384f257d0 0x55a384f400d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:07.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.895+0000 7fa939fcdb80 1 --2- 172.21.15.151:0/2113836221 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a384f24eb0 0x55a384f3fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:07.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.895+0000 7fa90ea00700 0 client.25847 ms_handle_reset on v2:172.21.15.164:6836/1554609572 2024-05-21T21:58:07.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.895+0000 7fa939fcdb80 1 -- 172.21.15.151:0/2113836221 >> 172.21.15.151:0/2113836221 conn(0x55a384e4dbe0 msgr2=0x55a384f199e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:58:07.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.896+0000 7fa939fcdb80 1 -- 172.21.15.151:0/2113836221 shutdown_connections 2024-05-21T21:58:07.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.896+0000 7fa939fcdb80 1 -- 172.21.15.151:0/2113836221 wait complete. 2024-05-21T21:58:07.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.896+0000 7fa939fcdb80 20 client.25847 trim_cache size 0 max 16384 2024-05-21T21:58:07.901 INFO:teuthology.orchestra.run.smithi151.stdout:[] 2024-05-21T21:58:07.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.896+0000 7fa92f600700 1 -- 172.21.15.151:0/1125819987 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fa9100b29f0 msgr2=0x7fa9100b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:07.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.896+0000 7fa92f600700 1 --2- 172.21.15.151:0/1125819987 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fa9100b29f0 0x7fa9100b4eb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa920009fd0 tx=0x7fa92000b040 comp rx=0 tx=0).stop 2024-05-21T21:58:07.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.896+0000 7fa92f600700 1 -- 172.21.15.151:0/1125819987 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa92806bc10 msgr2=0x7fa9281a0250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:07.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.896+0000 7fa92f600700 1 --2- 172.21.15.151:0/1125819987 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa92806bc10 0x7fa9281a0250 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa91c000c00 tx=0x7fa91c00bfd0 comp rx=0 tx=0).stop 2024-05-21T21:58:07.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.897+0000 7fa92f600700 1 -- 172.21.15.151:0/1125819987 shutdown_connections 2024-05-21T21:58:07.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.897+0000 7fa92f600700 1 --2- 172.21.15.151:0/1125819987 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fa9100b29f0 0x7fa9100b4eb0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:07.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.897+0000 7fa92f600700 1 --2- 172.21.15.151:0/1125819987 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa92806bc10 0x7fa9281a0250 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:07.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.897+0000 7fa92f600700 1 --2- 172.21.15.151:0/1125819987 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa928108490 0x7fa9281a4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:07.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.897+0000 7fa92f600700 1 --2- 172.21.15.151:0/1125819987 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa92806c7a0 0x7fa9281a0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:07.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.897+0000 7fa92f600700 1 -- 172.21.15.151:0/1125819987 >> 172.21.15.151:0/1125819987 conn(0x7fa9280793f0 msgr2=0x7fa928101810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:58:07.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.897+0000 7fa92f600700 1 -- 172.21.15.151:0/1125819987 shutdown_connections 2024-05-21T21:58:07.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:07.897+0000 7fa92f600700 1 -- 172.21.15.151:0/1125819987 wait complete. 2024-05-21T21:58:08.876 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-21T21:58:09.320 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:58:09.603 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:58:09 smithi151 ceph-mon[31623]: pgmap v79: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-05-21T21:58:09.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:58:09 smithi184 ceph-mon[36942]: pgmap v79: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-05-21T21:58:09.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:58:09 smithi164 ceph-mon[36269]: pgmap v79: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-05-21T21:58:11.507 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.505+0000 7f70c9600700 1 -- 172.21.15.151:0/1822686136 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f70c406c020 msgr2=0x7f70c406c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:11.508 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.505+0000 7f70c9600700 1 --2- 172.21.15.151:0/1822686136 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f70c406c020 0x7f70c406c480 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f70b0009990 tx=0x7f70b0009ca0 comp rx=0 tx=0).stop 2024-05-21T21:58:11.508 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.505+0000 7f70c9600700 1 -- 172.21.15.151:0/1822686136 shutdown_connections 2024-05-21T21:58:11.508 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.505+0000 7f70c9600700 1 --2- 172.21.15.151:0/1822686136 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f70c4105f70 0x7f70c4106400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:11.508 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.505+0000 7f70c9600700 1 --2- 172.21.15.151:0/1822686136 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f70c4108460 0x7f70c406bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:11.508 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.505+0000 7f70c9600700 1 --2- 172.21.15.151:0/1822686136 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f70c406c020 0x7f70c406c480 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:11.508 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.505+0000 7f70c9600700 1 -- 172.21.15.151:0/1822686136 >> 172.21.15.151:0/1822686136 conn(0x7f70c40793b0 msgr2=0x7f70c40797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:58:11.508 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.505+0000 7f70c9600700 1 -- 172.21.15.151:0/1822686136 shutdown_connections 2024-05-21T21:58:11.508 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.505+0000 7f70c9600700 1 -- 172.21.15.151:0/1822686136 wait complete. 2024-05-21T21:58:11.508 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.506+0000 7f70c9600700 1 Processor -- start 2024-05-21T21:58:11.510 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.507+0000 7f70c9600700 1 -- start start 2024-05-21T21:58:11.510 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.507+0000 7f70c9600700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f70c406c020 0x7f70c41a01c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:11.510 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.507+0000 7f70c9600700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f70c4105f70 0x7f70c41a0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:11.510 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.507+0000 7f70c9600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f70c4108460 0x7f70c41a4a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:11.510 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.507+0000 7f70c9600700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f70c4119d10 con 0x7f70c4108460 2024-05-21T21:58:11.510 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.507+0000 7f70c9600700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f70c4119b90 con 0x7f70c406c020 2024-05-21T21:58:11.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.507+0000 7f70c9600700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f70c4119e90 con 0x7f70c4105f70 2024-05-21T21:58:11.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.507+0000 7f70c3400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f70c4108460 0x7f70c41a4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:11.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.507+0000 7f70c2a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f70c406c020 0x7f70c41a01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:11.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.507+0000 7f70c2a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f70c406c020 0x7f70c41a01c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:34078/0 (socket says 172.21.15.151:34078) 2024-05-21T21:58:11.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.507+0000 7f70c2a00700 1 -- 172.21.15.151:0/602684594 learned_addr learned my addr 172.21.15.151:0/602684594 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:58:11.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.507+0000 7f70c2000700 1 --2- 172.21.15.151:0/602684594 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f70c4105f70 0x7f70c41a0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:11.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.508+0000 7f70c3400700 1 -- 172.21.15.151:0/602684594 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f70c4105f70 msgr2=0x7f70c41a0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:11.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.508+0000 7f70c3400700 1 --2- 172.21.15.151:0/602684594 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f70c4105f70 0x7f70c41a0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:11.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.508+0000 7f70c3400700 1 -- 172.21.15.151:0/602684594 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f70c406c020 msgr2=0x7f70c41a01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:11.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.508+0000 7f70c3400700 1 --2- 172.21.15.151:0/602684594 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f70c406c020 0x7f70c41a01c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:11.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.508+0000 7f70c3400700 1 -- 172.21.15.151:0/602684594 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70c41a51d0 con 0x7f70c4108460 2024-05-21T21:58:11.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.508+0000 7f70c2a00700 1 --2- 172.21.15.151:0/602684594 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f70c406c020 0x7f70c41a01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:58:11.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.508+0000 7f70c3400700 1 --2- 172.21.15.151:0/602684594 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f70c4108460 0x7f70c41a4a90 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f70bc00efc0 tx=0x7f70bc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:58:11.512 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.508+0000 7f70bb400700 1 -- 172.21.15.151:0/602684594 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70bc0092e0 con 0x7f70c4108460 2024-05-21T21:58:11.512 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.508+0000 7f70c9600700 1 -- 172.21.15.151:0/602684594 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70c41a5460 con 0x7f70c4108460 2024-05-21T21:58:11.512 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.508+0000 7f70c9600700 1 -- 172.21.15.151:0/602684594 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70c41acd40 con 0x7f70c4108460 2024-05-21T21:58:11.512 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.509+0000 7f70bb400700 1 -- 172.21.15.151:0/602684594 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70bc004bb0 con 0x7f70c4108460 2024-05-21T21:58:11.512 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.509+0000 7f70bb400700 1 -- 172.21.15.151:0/602684594 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70bc004020 con 0x7f70c4108460 2024-05-21T21:58:11.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.512+0000 7f70bb400700 1 -- 172.21.15.151:0/602684594 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7f70bc01b030 con 0x7f70c4108460 2024-05-21T21:58:11.514 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.513+0000 7f70d4005b80 1 Processor -- start 2024-05-21T21:58:11.514 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.513+0000 7f70d4005b80 1 -- start start 2024-05-21T21:58:11.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.513+0000 7f70d4005b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5589a9eedeb0 0x5589a9eee290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:11.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.513+0000 7f70d4005b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5589a9eee7d0 0x5589a9ee8260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:11.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.513+0000 7f70d4005b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589a9ee8880 0x5589a9ee8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:11.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.513+0000 7f70d4005b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5589a9e46e40 con 0x5589a9eedeb0 2024-05-21T21:58:11.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.513+0000 7f70d4005b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5589a9e369e0 con 0x5589a9eee7d0 2024-05-21T21:58:11.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.513+0000 7f70d4005b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5589a9e3aa20 con 0x5589a9ee8880 2024-05-21T21:58:11.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.514+0000 7f70c2000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5589a9eedeb0 0x5589a9eee290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:11.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.514+0000 7f70c2000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5589a9eedeb0 0x5589a9eee290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:57228/0 (socket says 172.21.15.151:57228) 2024-05-21T21:58:11.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.514+0000 7f70c2000700 1 -- 172.21.15.151:0/2827393207 learned_addr learned my addr 172.21.15.151:0/2827393207 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:58:11.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.514+0000 7f70bb400700 1 --2- 172.21.15.151:0/602684594 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f70a80b29f0 0x7f70a80b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:11.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.514+0000 7f70c2a00700 1 --2- 172.21.15.151:0/2827393207 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5589a9eee7d0 0x5589a9ee8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:11.517 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.514+0000 7f70bb400700 1 -- 172.21.15.151:0/602684594 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f70bc0e2000 con 0x7f70c4108460 2024-05-21T21:58:11.517 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.514+0000 7f70c2000700 1 --2- 172.21.15.151:0/2827393207 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589a9ee8880 0x5589a9ee8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:11.517 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.514+0000 7f70c3400700 1 --2- 172.21.15.151:0/602684594 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f70a80b29f0 0x7f70a80b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:11.517 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.514+0000 7f70c2a00700 1 -- 172.21.15.151:0/2827393207 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589a9ee8880 msgr2=0x5589a9ee8ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:11.517 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.514+0000 7f70c2a00700 1 --2- 172.21.15.151:0/2827393207 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589a9ee8880 0x5589a9ee8ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:11.517 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.514+0000 7f70c2a00700 1 -- 172.21.15.151:0/2827393207 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5589a9eedeb0 msgr2=0x5589a9eee290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:11.517 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.514+0000 7f70c2a00700 1 --2- 172.21.15.151:0/2827393207 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5589a9eedeb0 0x5589a9eee290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:11.518 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.514+0000 7f70c2a00700 1 -- 172.21.15.151:0/2827393207 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589a9dff420 con 0x5589a9eee7d0 2024-05-21T21:58:11.518 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.514+0000 7f70c2000700 1 --2- 172.21.15.151:0/2827393207 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589a9ee8880 0x5589a9ee8ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T21:58:11.518 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.515+0000 7f70c2a00700 1 --2- 172.21.15.151:0/2827393207 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5589a9eee7d0 0x5589a9ee8260 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f70b40114c0 tx=0x7f70b40117d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=337ea5e83182652e server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:58:11.518 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.515+0000 7f70afe00700 1 -- 172.21.15.151:0/2827393207 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b4004230 con 0x5589a9eee7d0 2024-05-21T21:58:11.518 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.515+0000 7f70afe00700 1 -- 172.21.15.151:0/2827393207 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70b4011e50 con 0x5589a9eee7d0 2024-05-21T21:58:11.518 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.515+0000 7f70c3400700 1 --2- 172.21.15.151:0/602684594 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f70a80b29f0 0x7f70a80b4eb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f70bc0a9080 tx=0x7f70bc009480 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:58:11.518 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.515+0000 7f70afe00700 1 -- 172.21.15.151:0/2827393207 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b400e890 con 0x5589a9eee7d0 2024-05-21T21:58:11.518 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.516+0000 7f70d4005b80 1 -- 172.21.15.151:0/2827393207 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5589a9eee7d0 msgr2=0x5589a9ee8260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:11.518 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.516+0000 7f70d4005b80 1 --2- 172.21.15.151:0/2827393207 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5589a9eee7d0 0x5589a9ee8260 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f70b40114c0 tx=0x7f70b40117d0 comp rx=0 tx=0).stop 2024-05-21T21:58:11.518 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.516+0000 7f70d4005b80 1 -- 172.21.15.151:0/2827393207 shutdown_connections 2024-05-21T21:58:11.518 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.516+0000 7f70d4005b80 1 --2- 172.21.15.151:0/2827393207 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589a9ee8880 0x5589a9ee8ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:11.518 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.516+0000 7f70d4005b80 1 --2- 172.21.15.151:0/2827393207 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5589a9eedeb0 0x5589a9eee290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:11.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.516+0000 7f70d4005b80 1 --2- 172.21.15.151:0/2827393207 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5589a9eee7d0 0x5589a9ee8260 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:11.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.516+0000 7f70d4005b80 1 -- 172.21.15.151:0/2827393207 >> 172.21.15.151:0/2827393207 conn(0x5589a9e16be0 msgr2=0x5589a9ee3010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:58:11.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.516+0000 7f70d4005b80 1 -- 172.21.15.151:0/2827393207 shutdown_connections 2024-05-21T21:58:11.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.516+0000 7f70d4005b80 1 -- 172.21.15.151:0/2827393207 wait complete. 2024-05-21T21:58:11.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.516+0000 7f70d4005b80 1 -- start start 2024-05-21T21:58:11.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.517+0000 7f70d4005b80 1 Processor -- start 2024-05-21T21:58:11.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.517+0000 7f70d4005b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5589a9ee8880 0x5589a9ef4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:11.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.517+0000 7f70d4005b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589a9eedeb0 0x5589a9ef1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:11.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.517+0000 7f70d4005b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5589a9eee7d0 0x5589a9ef1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:11.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.517+0000 7f70d4005b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5589a9d3cfa0 con 0x5589a9ee8880 2024-05-21T21:58:11.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.517+0000 7f70d4005b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5589a9dd7760 con 0x5589a9eee7d0 2024-05-21T21:58:11.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.517+0000 7f70d4005b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5589a9ef5240 con 0x5589a9eedeb0 2024-05-21T21:58:11.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.517+0000 7f70d4005b80 10 client.0 _ll_register_callbacks cb 0x5589a9de0c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:58:11.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.518+0000 7f70c2000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5589a9ee8880 0x5589a9ef4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:11.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.518+0000 7f70c2000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5589a9ee8880 0x5589a9ef4440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:57236/0 (socket says 172.21.15.151:57236) 2024-05-21T21:58:11.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.518+0000 7f70c2000700 1 -- 172.21.15.151:0/1794490460 learned_addr learned my addr 172.21.15.151:0/1794490460 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:58:11.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.518+0000 7f70c2a00700 1 --2- 172.21.15.151:0/1794490460 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589a9eedeb0 0x5589a9ef1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:11.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.518+0000 7f70c2000700 1 --2- 172.21.15.151:0/1794490460 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5589a9eee7d0 0x5589a9ef1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:11.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.518+0000 7f70c2000700 1 -- 172.21.15.151:0/1794490460 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589a9eedeb0 msgr2=0x5589a9ef1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:11.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.518+0000 7f70c2000700 1 --2- 172.21.15.151:0/1794490460 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589a9eedeb0 0x5589a9ef1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:11.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.518+0000 7f70c2000700 1 -- 172.21.15.151:0/1794490460 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5589a9eee7d0 msgr2=0x5589a9ef1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:11.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.518+0000 7f70c2000700 1 --2- 172.21.15.151:0/1794490460 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5589a9eee7d0 0x5589a9ef1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:11.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.518+0000 7f70c2000700 1 -- 172.21.15.151:0/1794490460 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589a9d868d0 con 0x5589a9ee8880 2024-05-21T21:58:11.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.518+0000 7f70c2000700 1 --2- 172.21.15.151:0/1794490460 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5589a9ee8880 0x5589a9ef4440 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f70b000b2e0 tx=0x7f70b0005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b6f5101c4a38980 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:58:11.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.518+0000 7f70acc00700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-21T21:58:11.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.519+0000 7f70acc00700 1 -- 172.21.15.151:0/1794490460 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b0021040 con 0x5589a9ee8880 2024-05-21T21:58:11.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.519+0000 7f70acc00700 1 -- 172.21.15.151:0/1794490460 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f70b0003680 con 0x5589a9ee8880 2024-05-21T21:58:11.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.519+0000 7f70d4005b80 1 -- 172.21.15.151:0/1794490460 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70b4011e50 con 0x5589a9ee8880 2024-05-21T21:58:11.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.519+0000 7f70acc00700 1 -- 172.21.15.151:0/1794490460 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f70b0003880 con 0x5589a9ee8880 2024-05-21T21:58:11.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.519+0000 7f70d4005b80 10 client.15732 fetch_fsmap learned FSMap version 29 2024-05-21T21:58:11.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.519+0000 7f70d4005b80 1 -- 172.21.15.151:0/1794490460 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5589a9e07060 con 0x5589a9ee8880 2024-05-21T21:58:11.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.519+0000 7f70acc00700 1 -- 172.21.15.151:0/1794490460 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70b4011e50 con 0x5589a9ee8880 2024-05-21T21:58:11.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.520+0000 7f70acc00700 1 -- 172.21.15.151:0/1794490460 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f70b0024070 con 0x5589a9ee8880 2024-05-21T21:58:11.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.520+0000 7f70d4005b80 10 client.15732 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T21:58:11.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.520+0000 7f70d4005b80 10 client.15732 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-21T21:58:11.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.520+0000 7f70d4005b80 20 client.15732 populate_metadata read hostname 'smithi151' 2024-05-21T21:58:11.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.520+0000 7f70d4005b80 1 --2- 172.21.15.151:0/1794490460 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x5589a9f01d70 0x5589a9f041b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:11.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.520+0000 7f70d4005b80 4 client.15732 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T21:58:11.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.520+0000 7f70d4005b80 1 -- 172.21.15.151:0/1794490460 --> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5589a9e48d90 con 0x5589a9f01d70 2024-05-21T21:58:11.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.520+0000 7f70c2a00700 1 --2- 172.21.15.151:0/1794490460 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x5589a9f01d70 0x5589a9f041b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:11.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.521+0000 7f70c2a00700 1 --2- 172.21.15.151:0/1794490460 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x5589a9f01d70 0x5589a9f041b0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dde00a39c80216d8 server_cookie=9c46a0ac50a52335 in_seq=0 out_seq=0 2024-05-21T21:58:11.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.521+0000 7f70acc00700 10 client.15732 ms_handle_connect on v2:172.21.15.184:6838/3059935143 2024-05-21T21:58:11.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.521+0000 7f70acc00700 1 -- 172.21.15.151:0/1794490460 <== mds.1 v2:172.21.15.184:6838/3059935143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5589a9e48d90 con 0x5589a9f01d70 2024-05-21T21:58:11.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.521+0000 7f70acc00700 10 client.15732 handle_command_reply: tid=0 2024-05-21T21:58:11.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.521+0000 7f70d4005b80 1 client.15732 shutdown 2024-05-21T21:58:11.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.522+0000 7f70d4005b80 1 -- 172.21.15.151:0/1794490460 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5589a9ee8880 msgr2=0x5589a9ef4440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:11.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.522+0000 7f70d4005b80 1 --2- 172.21.15.151:0/1794490460 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5589a9ee8880 0x5589a9ef4440 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f70b000b2e0 tx=0x7f70b0005fd0 comp rx=0 tx=0).stop 2024-05-21T21:58:11.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.522+0000 7f70d4005b80 1 -- 172.21.15.151:0/1794490460 shutdown_connections 2024-05-21T21:58:11.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.522+0000 7f70d4005b80 1 --2- 172.21.15.151:0/1794490460 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x5589a9f01d70 0x5589a9f041b0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:11.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.522+0000 7f70d4005b80 1 --2- 172.21.15.151:0/1794490460 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5589a9eee7d0 0x5589a9ef1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:11.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.522+0000 7f70d4005b80 1 --2- 172.21.15.151:0/1794490460 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5589a9ee8880 0x5589a9ef4440 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:11.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.522+0000 7f70d4005b80 1 --2- 172.21.15.151:0/1794490460 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589a9eedeb0 0x5589a9ef1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:11.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.522+0000 7f70acc00700 0 client.15732 ms_handle_reset on v2:172.21.15.184:6838/3059935143 2024-05-21T21:58:11.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.522+0000 7f70d4005b80 1 -- 172.21.15.151:0/1794490460 >> 172.21.15.151:0/1794490460 conn(0x5589a9e16be0 msgr2=0x5589a9ee7250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:58:11.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.522+0000 7f70d4005b80 1 -- 172.21.15.151:0/1794490460 shutdown_connections 2024-05-21T21:58:11.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.522+0000 7f70d4005b80 1 -- 172.21.15.151:0/1794490460 wait complete. 2024-05-21T21:58:11.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.522+0000 7f70d4005b80 20 client.15732 trim_cache size 0 max 16384 2024-05-21T21:58:11.540 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.539+0000 7f70d4005b80 1 Processor -- start 2024-05-21T21:58:11.540 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.539+0000 7f70d4005b80 1 -- start start 2024-05-21T21:58:11.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.539+0000 7f70d4005b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589a9ee8880 0x5589a9ee8260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:11.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.539+0000 7f70d4005b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5589a9eedeb0 0x5589a9ef3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:11.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.539+0000 7f70d4005b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5589a9eee7d0 0x5589a9ef0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:11.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.539+0000 7f70d4005b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f70a400d580 con 0x5589a9eee7d0 2024-05-21T21:58:11.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.539+0000 7f70d4005b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f70a400f0c0 con 0x5589a9eedeb0 2024-05-21T21:58:11.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.539+0000 7f70d4005b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f70a400dc50 con 0x5589a9ee8880 2024-05-21T21:58:11.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.539+0000 7f70c2000700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589a9ee8880 0x5589a9ee8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:11.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.540+0000 7f70c2000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5589a9eee7d0 0x5589a9ef0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:11.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.540+0000 7f70c2000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5589a9eee7d0 0x5589a9ef0120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:57250/0 (socket says 172.21.15.151:57250) 2024-05-21T21:58:11.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.540+0000 7f70c2000700 1 -- 172.21.15.151:0/3491247663 learned_addr learned my addr 172.21.15.151:0/3491247663 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:58:11.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.540+0000 7f70c2a00700 1 --2- 172.21.15.151:0/3491247663 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5589a9eedeb0 0x5589a9ef3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:11.543 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.540+0000 7f70c2000700 1 -- 172.21.15.151:0/3491247663 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589a9ee8880 msgr2=0x5589a9ee8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:11.543 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.540+0000 7f70c2000700 1 --2- 172.21.15.151:0/3491247663 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589a9ee8880 0x5589a9ee8260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:11.543 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.540+0000 7f70c2000700 1 -- 172.21.15.151:0/3491247663 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5589a9eedeb0 msgr2=0x5589a9ef3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:11.543 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.540+0000 7f70c2000700 1 --2- 172.21.15.151:0/3491247663 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5589a9eedeb0 0x5589a9ef3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:11.543 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.540+0000 7f70c2000700 1 -- 172.21.15.151:0/3491247663 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589a9eefda0 con 0x5589a9eee7d0 2024-05-21T21:58:11.543 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.540+0000 7f70c2000700 1 --2- 172.21.15.151:0/3491247663 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5589a9eee7d0 0x5589a9ef0120 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f70b000b6c0 tx=0x7f70b0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2278d28dcaabec0f server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:58:11.543 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.540+0000 7f70af400700 1 -- 172.21.15.151:0/3491247663 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b0036070 con 0x5589a9eee7d0 2024-05-21T21:58:11.543 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.540+0000 7f70af400700 1 -- 172.21.15.151:0/3491247663 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70b00058e0 con 0x5589a9eee7d0 2024-05-21T21:58:11.543 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.541+0000 7f70af400700 1 -- 172.21.15.151:0/3491247663 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b000f580 con 0x5589a9eee7d0 2024-05-21T21:58:11.544 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.541+0000 7f70d4005b80 1 -- 172.21.15.151:0/3491247663 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5589a9eee7d0 msgr2=0x5589a9ef0120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:11.544 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.541+0000 7f70d4005b80 1 --2- 172.21.15.151:0/3491247663 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5589a9eee7d0 0x5589a9ef0120 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f70b000b6c0 tx=0x7f70b0000bc0 comp rx=0 tx=0).stop 2024-05-21T21:58:11.544 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.541+0000 7f70d4005b80 1 -- 172.21.15.151:0/3491247663 shutdown_connections 2024-05-21T21:58:11.544 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.541+0000 7f70d4005b80 1 --2- 172.21.15.151:0/3491247663 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5589a9eee7d0 0x5589a9ef0120 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:11.544 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.541+0000 7f70d4005b80 1 --2- 172.21.15.151:0/3491247663 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589a9ee8880 0x5589a9ee8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:11.544 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.541+0000 7f70d4005b80 1 --2- 172.21.15.151:0/3491247663 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5589a9eedeb0 0x5589a9ef3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:11.544 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.541+0000 7f70d4005b80 1 -- 172.21.15.151:0/3491247663 >> 172.21.15.151:0/3491247663 conn(0x5589a9e16be0 msgr2=0x5589a9eeb9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:58:11.544 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.541+0000 7f70d4005b80 1 -- 172.21.15.151:0/3491247663 shutdown_connections 2024-05-21T21:58:11.544 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.541+0000 7f70d4005b80 1 -- 172.21.15.151:0/3491247663 wait complete. 2024-05-21T21:58:11.544 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.542+0000 7f70d4005b80 1 -- start start 2024-05-21T21:58:11.544 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.542+0000 7f70d4005b80 1 Processor -- start 2024-05-21T21:58:11.547 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.542+0000 7f70d4005b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5589a9ee8880 0x5589a9f097e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:11.547 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.542+0000 7f70d4005b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5589a9eedeb0 0x5589a9f09d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:11.547 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.542+0000 7f70d4005b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589a9eee7d0 0x5589a9f0a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:11.547 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.542+0000 7f70d4005b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f70a400e9f0 con 0x5589a9eedeb0 2024-05-21T21:58:11.547 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.542+0000 7f70d4005b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f70a400e320 con 0x5589a9ee8880 2024-05-21T21:58:11.547 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.542+0000 7f70d4005b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f70a400b520 con 0x5589a9eee7d0 2024-05-21T21:58:11.547 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.543+0000 7f70d4005b80 10 client.0 _ll_register_callbacks cb 0x5589a9de0c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:58:11.547 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.543+0000 7f70c2a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5589a9eedeb0 0x5589a9f09d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:11.548 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.543+0000 7f70c2000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5589a9ee8880 0x5589a9f097e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:11.548 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.543+0000 7f70c2a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5589a9eedeb0 0x5589a9f09d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:57254/0 (socket says 172.21.15.151:57254) 2024-05-21T21:58:11.548 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.543+0000 7f70c2a00700 1 -- 172.21.15.151:0/1174015137 learned_addr learned my addr 172.21.15.151:0/1174015137 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:58:11.548 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.543+0000 7f70c2000700 1 --2- 172.21.15.151:0/1174015137 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589a9eee7d0 0x5589a9f0a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:11.548 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.543+0000 7f70c2a00700 1 -- 172.21.15.151:0/1174015137 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589a9eee7d0 msgr2=0x5589a9f0a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:11.548 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.543+0000 7f70c2a00700 1 --2- 172.21.15.151:0/1174015137 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589a9eee7d0 0x5589a9f0a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:11.548 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.543+0000 7f70c2a00700 1 -- 172.21.15.151:0/1174015137 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5589a9ee8880 msgr2=0x5589a9f097e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:11.548 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.543+0000 7f70c2a00700 1 --2- 172.21.15.151:0/1174015137 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5589a9ee8880 0x5589a9f097e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:11.548 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.543+0000 7f70c2a00700 1 -- 172.21.15.151:0/1174015137 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589a9f6bb40 con 0x5589a9eedeb0 2024-05-21T21:58:11.548 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.544+0000 7f70c2a00700 1 --2- 172.21.15.151:0/1174015137 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5589a9eedeb0 0x5589a9f09d20 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f70b4011b20 tx=0x7f70b4004210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=889c6c5388a8624f server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:58:11.548 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.544+0000 7f70acc00700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-21T21:58:11.549 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.544+0000 7f70acc00700 1 -- 172.21.15.151:0/1174015137 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b4012e60 con 0x5589a9eedeb0 2024-05-21T21:58:11.549 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.544+0000 7f70acc00700 1 -- 172.21.15.151:0/1174015137 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f70b4005240 con 0x5589a9eedeb0 2024-05-21T21:58:11.549 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.544+0000 7f70d4005b80 1 -- 172.21.15.151:0/1174015137 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70b00058e0 con 0x5589a9eedeb0 2024-05-21T21:58:11.549 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.544+0000 7f70acc00700 1 -- 172.21.15.151:0/1174015137 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f70b403c050 con 0x5589a9eedeb0 2024-05-21T21:58:11.549 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.544+0000 7f70d4005b80 10 client.15738 fetch_fsmap learned FSMap version 29 2024-05-21T21:58:11.549 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.544+0000 7f70d4005b80 1 -- 172.21.15.151:0/1174015137 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5589a9f6ce20 con 0x5589a9eedeb0 2024-05-21T21:58:11.549 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.544+0000 7f70acc00700 1 -- 172.21.15.151:0/1174015137 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70b00058e0 con 0x5589a9eedeb0 2024-05-21T21:58:11.549 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.545+0000 7f70acc00700 1 -- 172.21.15.151:0/1174015137 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f70b402b2b0 con 0x5589a9eedeb0 2024-05-21T21:58:11.549 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.545+0000 7f70d4005b80 10 client.15738 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T21:58:11.549 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.545+0000 7f70d4005b80 10 client.15738 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-21T21:58:11.549 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.545+0000 7f70d4005b80 20 client.15738 populate_metadata read hostname 'smithi151' 2024-05-21T21:58:11.549 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.545+0000 7f70d4005b80 1 --2- 172.21.15.151:0/1174015137 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x5589a9f6d3c0 0x5589a9f6f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:11.549 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.545+0000 7f70d4005b80 4 client.15738 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T21:58:11.550 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.545+0000 7f70d4005b80 1 -- 172.21.15.151:0/1174015137 --> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5589a9e42570 con 0x5589a9f6d3c0 2024-05-21T21:58:11.550 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.545+0000 7f70c2000700 1 --2- 172.21.15.151:0/1174015137 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x5589a9f6d3c0 0x5589a9f6f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:11.550 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.546+0000 7f70c2000700 1 --2- 172.21.15.151:0/1174015137 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x5589a9f6d3c0 0x5589a9f6f7e0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7f5705ce5725bb3 server_cookie=3fb9adb4ef9099fc in_seq=0 out_seq=0 2024-05-21T21:58:11.550 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.546+0000 7f70acc00700 10 client.15738 ms_handle_connect on v2:172.21.15.184:6838/3059935143 2024-05-21T21:58:11.550 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.546+0000 7f70acc00700 1 -- 172.21.15.151:0/1174015137 <== mds.1 v2:172.21.15.184:6838/3059935143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5589a9e42570 con 0x5589a9f6d3c0 2024-05-21T21:58:11.550 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.546+0000 7f70acc00700 10 client.15738 handle_command_reply: tid=0 2024-05-21T21:58:11.550 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.546+0000 7f70d4005b80 1 client.15738 shutdown 2024-05-21T21:58:11.550 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.546+0000 7f70d4005b80 1 -- 172.21.15.151:0/1174015137 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5589a9eedeb0 msgr2=0x5589a9f09d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:11.550 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.546+0000 7f70d4005b80 1 --2- 172.21.15.151:0/1174015137 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5589a9eedeb0 0x5589a9f09d20 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f70b4011b20 tx=0x7f70b4004210 comp rx=0 tx=0).stop 2024-05-21T21:58:11.551 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.546+0000 7f70d4005b80 1 -- 172.21.15.151:0/1174015137 shutdown_connections 2024-05-21T21:58:11.551 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.546+0000 7f70d4005b80 1 --2- 172.21.15.151:0/1174015137 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x5589a9f6d3c0 0x5589a9f6f7e0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:11.551 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.546+0000 7f70d4005b80 1 --2- 172.21.15.151:0/1174015137 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5589a9ee8880 0x5589a9f097e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:11.551 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.546+0000 7f70d4005b80 1 --2- 172.21.15.151:0/1174015137 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589a9eee7d0 0x5589a9f0a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:11.552 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.546+0000 7f70acc00700 0 client.15738 ms_handle_reset on v2:172.21.15.184:6838/3059935143 2024-05-21T21:58:11.552 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.546+0000 7f70d4005b80 1 --2- 172.21.15.151:0/1174015137 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5589a9eedeb0 0x5589a9f09d20 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:11.552 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.546+0000 7f70d4005b80 1 -- 172.21.15.151:0/1174015137 >> 172.21.15.151:0/1174015137 conn(0x5589a9e16be0 msgr2=0x5589a9ef0050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:58:11.552 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.546+0000 7f70d4005b80 1 -- 172.21.15.151:0/1174015137 shutdown_connections 2024-05-21T21:58:11.552 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.546+0000 7f70d4005b80 1 -- 172.21.15.151:0/1174015137 wait complete. 2024-05-21T21:58:11.552 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.546+0000 7f70d4005b80 20 client.15738 trim_cache size 0 max 16384 2024-05-21T21:58:11.552 INFO:teuthology.orchestra.run.smithi151.stdout:[] 2024-05-21T21:58:11.553 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.547+0000 7f70c9600700 1 -- 172.21.15.151:0/602684594 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f70a80b29f0 msgr2=0x7f70a80b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:11.553 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.547+0000 7f70c9600700 1 --2- 172.21.15.151:0/602684594 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f70a80b29f0 0x7f70a80b4eb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f70bc0a9080 tx=0x7f70bc009480 comp rx=0 tx=0).stop 2024-05-21T21:58:11.553 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.547+0000 7f70c9600700 1 -- 172.21.15.151:0/602684594 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f70c4108460 msgr2=0x7f70c41a4a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:11.553 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.547+0000 7f70c9600700 1 --2- 172.21.15.151:0/602684594 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f70c4108460 0x7f70c41a4a90 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f70bc00efc0 tx=0x7f70bc00c3a0 comp rx=0 tx=0).stop 2024-05-21T21:58:11.553 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.547+0000 7f70c9600700 1 -- 172.21.15.151:0/602684594 shutdown_connections 2024-05-21T21:58:11.553 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.547+0000 7f70c9600700 1 --2- 172.21.15.151:0/602684594 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f70a80b29f0 0x7f70a80b4eb0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:11.553 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.547+0000 7f70c9600700 1 --2- 172.21.15.151:0/602684594 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f70c406c020 0x7f70c41a01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:11.554 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.547+0000 7f70c9600700 1 --2- 172.21.15.151:0/602684594 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f70c4108460 0x7f70c41a4a90 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:11.554 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.547+0000 7f70c9600700 1 --2- 172.21.15.151:0/602684594 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f70c4105f70 0x7f70c41a0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:11.554 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.547+0000 7f70c9600700 1 -- 172.21.15.151:0/602684594 >> 172.21.15.151:0/602684594 conn(0x7f70c40793b0 msgr2=0x7f70c4104f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:58:11.554 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.547+0000 7f70c9600700 1 -- 172.21.15.151:0/602684594 shutdown_connections 2024-05-21T21:58:11.554 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:11.547+0000 7f70c9600700 1 -- 172.21.15.151:0/602684594 wait complete. 2024-05-21T21:58:11.564 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:58:11 smithi151 ceph-mon[31623]: pgmap v80: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-05-21T21:58:11.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:58:11 smithi184 ceph-mon[36942]: pgmap v80: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-05-21T21:58:11.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:58:11 smithi164 ceph-mon[36269]: pgmap v80: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-05-21T21:58:12.612 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-21T21:58:13.063 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:58:13.530 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:58:13 smithi151 ceph-mon[31623]: pgmap v81: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-05-21T21:58:13.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:58:13 smithi184 ceph-mon[36942]: pgmap v81: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-05-21T21:58:13.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:58:13 smithi164 ceph-mon[36269]: pgmap v81: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-05-21T21:58:14.160 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-21T21:58:15.406 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:58:15.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.459+0000 7fcacbc00700 1 -- 172.21.15.151:0/1965724300 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcac4075770 msgr2=0x7fcac4075b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:15.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.459+0000 7fcacbc00700 1 --2- 172.21.15.151:0/1965724300 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcac4075770 0x7fcac4075b50 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fcab8009990 tx=0x7fcab8009ca0 comp rx=0 tx=0).stop 2024-05-21T21:58:15.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.459+0000 7fcacbc00700 1 -- 172.21.15.151:0/1965724300 shutdown_connections 2024-05-21T21:58:15.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.459+0000 7fcacbc00700 1 --2- 172.21.15.151:0/1965724300 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcac410ff40 0x7fcac4112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:15.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.459+0000 7fcacbc00700 1 --2- 172.21.15.151:0/1965724300 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcac4075770 0x7fcac4075b50 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:15.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.459+0000 7fcacbc00700 1 --2- 172.21.15.151:0/1965724300 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcac4076090 0x7fcac410fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:15.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.459+0000 7fcacbc00700 1 -- 172.21.15.151:0/1965724300 >> 172.21.15.151:0/1965724300 conn(0x7fcac407a6a0 msgr2=0x7fcac407aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:58:15.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.459+0000 7fcacbc00700 1 -- 172.21.15.151:0/1965724300 shutdown_connections 2024-05-21T21:58:15.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.459+0000 7fcacbc00700 1 -- 172.21.15.151:0/1965724300 wait complete. 2024-05-21T21:58:15.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.460+0000 7fcacbc00700 1 Processor -- start 2024-05-21T21:58:15.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.461+0000 7fcacbc00700 1 -- start start 2024-05-21T21:58:15.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.461+0000 7fcacbc00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcac4075770 0x7fcac419be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:15.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.461+0000 7fcacbc00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcac4076090 0x7fcac419c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:15.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.461+0000 7fcacbc00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcac410ff40 0x7fcac41a0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:15.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.461+0000 7fcacbc00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcac41158f0 con 0x7fcac410ff40 2024-05-21T21:58:15.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.461+0000 7fcacbc00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcac4115770 con 0x7fcac4076090 2024-05-21T21:58:15.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.461+0000 7fcacbc00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fcac4115a70 con 0x7fcac4075770 2024-05-21T21:58:15.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.461+0000 7fcac9c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcac410ff40 0x7fcac41a0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:15.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.461+0000 7fcac9c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcac410ff40 0x7fcac41a0710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:57264/0 (socket says 172.21.15.151:57264) 2024-05-21T21:58:15.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.461+0000 7fcac9200700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcac4075770 0x7fcac419be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:15.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.461+0000 7fcac9c00700 1 -- 172.21.15.151:0/3204159282 learned_addr learned my addr 172.21.15.151:0/3204159282 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:58:15.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.461+0000 7fcac3e00700 1 --2- 172.21.15.151:0/3204159282 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcac4076090 0x7fcac419c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:15.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.462+0000 7fcac9200700 1 -- 172.21.15.151:0/3204159282 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcac4076090 msgr2=0x7fcac419c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:15.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.462+0000 7fcac9200700 1 --2- 172.21.15.151:0/3204159282 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcac4076090 0x7fcac419c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:15.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.462+0000 7fcac9200700 1 -- 172.21.15.151:0/3204159282 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcac410ff40 msgr2=0x7fcac41a0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:15.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.462+0000 7fcac9200700 1 --2- 172.21.15.151:0/3204159282 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcac410ff40 0x7fcac41a0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:15.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.462+0000 7fcac9200700 1 -- 172.21.15.151:0/3204159282 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcac41a0e50 con 0x7fcac4075770 2024-05-21T21:58:15.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.462+0000 7fcac9200700 1 --2- 172.21.15.151:0/3204159282 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcac4075770 0x7fcac419be40 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fcab8006010 tx=0x7fcab80057c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:58:15.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.462+0000 7fcac1600700 1 -- 172.21.15.151:0/3204159282 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab8027070 con 0x7fcac4075770 2024-05-21T21:58:15.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.462+0000 7fcac1600700 1 -- 172.21.15.151:0/3204159282 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcab8022470 con 0x7fcac4075770 2024-05-21T21:58:15.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.462+0000 7fcacbc00700 1 -- 172.21.15.151:0/3204159282 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcac41a10e0 con 0x7fcac4075770 2024-05-21T21:58:15.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.462+0000 7fcac1600700 1 -- 172.21.15.151:0/3204159282 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab8017650 con 0x7fcac4075770 2024-05-21T21:58:15.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.463+0000 7fcacbc00700 1 -- 172.21.15.151:0/3204159282 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcac41a89c0 con 0x7fcac4075770 2024-05-21T21:58:15.467 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.466+0000 7fcac1600700 1 -- 172.21.15.151:0/3204159282 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7fcab8009d10 con 0x7fcac4075770 2024-05-21T21:58:15.469 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.468+0000 7fcac1600700 1 --2- 172.21.15.151:0/3204159282 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fcaac0b29f0 0x7fcaac0b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:15.469 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.468+0000 7fcac1600700 1 -- 172.21.15.151:0/3204159282 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fcab80e2f30 con 0x7fcac4075770 2024-05-21T21:58:15.470 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.468+0000 7fcac3e00700 1 --2- 172.21.15.151:0/3204159282 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fcaac0b29f0 0x7fcaac0b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:15.471 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.469+0000 7fcac3e00700 1 --2- 172.21.15.151:0/3204159282 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fcaac0b29f0 0x7fcaac0b4eb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fcab0009680 tx=0x7fcab0009260 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:58:15.471 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.470+0000 7fcad6639b80 1 Processor -- start 2024-05-21T21:58:15.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.470+0000 7fcad6639b80 1 -- start start 2024-05-21T21:58:15.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.470+0000 7fcad6639b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556e84ff4e20 0x556e84ff5200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:15.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.470+0000 7fcad6639b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556e84ff5740 0x556e84fef330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:15.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.470+0000 7fcad6639b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556e84fef950 0x556e84fefdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:15.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.470+0000 7fcad6639b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x556e84f44e40 con 0x556e84fef950 2024-05-21T21:58:15.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.470+0000 7fcad6639b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x556e84f349e0 con 0x556e84ff5740 2024-05-21T21:58:15.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.471+0000 7fcad6639b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x556e84f38a20 con 0x556e84ff4e20 2024-05-21T21:58:15.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.471+0000 7fcac3e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556e84ff5740 0x556e84fef330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:15.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.471+0000 7fcac9200700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556e84ff4e20 0x556e84ff5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:15.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.471+0000 7fcac9200700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556e84ff4e20 0x556e84ff5200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:37750/0 (socket says 172.21.15.151:37750) 2024-05-21T21:58:15.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.471+0000 7fcac9200700 1 -- 172.21.15.151:0/2818375288 learned_addr learned my addr 172.21.15.151:0/2818375288 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:58:15.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.471+0000 7fcac9c00700 1 --2- 172.21.15.151:0/2818375288 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556e84fef950 0x556e84fefdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:15.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.471+0000 7fcac9200700 1 -- 172.21.15.151:0/2818375288 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556e84ff5740 msgr2=0x556e84fef330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:15.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.471+0000 7fcac9200700 1 --2- 172.21.15.151:0/2818375288 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556e84ff5740 0x556e84fef330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:15.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.471+0000 7fcac9200700 1 -- 172.21.15.151:0/2818375288 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556e84fef950 msgr2=0x556e84fefdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:15.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.471+0000 7fcac9200700 1 --2- 172.21.15.151:0/2818375288 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556e84fef950 0x556e84fefdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:15.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.471+0000 7fcac9200700 1 -- 172.21.15.151:0/2818375288 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e84efd420 con 0x556e84ff4e20 2024-05-21T21:58:15.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.471+0000 7fcac9c00700 1 --2- 172.21.15.151:0/2818375288 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556e84fef950 0x556e84fefdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T21:58:15.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.472+0000 7fcac9200700 1 --2- 172.21.15.151:0/2818375288 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556e84ff4e20 0x556e84ff5200 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fcab80a62d0 tx=0x7fcab80a9ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e4db760274edb23 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:58:15.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.472+0000 7fcab6000700 1 -- 172.21.15.151:0/2818375288 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab80aa250 con 0x556e84ff4e20 2024-05-21T21:58:15.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.472+0000 7fcab6000700 1 -- 172.21.15.151:0/2818375288 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcab80ae600 con 0x556e84ff4e20 2024-05-21T21:58:15.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.472+0000 7fcab6000700 1 -- 172.21.15.151:0/2818375288 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab80aa4b0 con 0x556e84ff4e20 2024-05-21T21:58:15.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.473+0000 7fcad6639b80 1 -- 172.21.15.151:0/2818375288 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556e84ff4e20 msgr2=0x556e84ff5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:15.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.473+0000 7fcad6639b80 1 --2- 172.21.15.151:0/2818375288 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556e84ff4e20 0x556e84ff5200 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fcab80a62d0 tx=0x7fcab80a9ca0 comp rx=0 tx=0).stop 2024-05-21T21:58:15.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.473+0000 7fcad6639b80 1 -- 172.21.15.151:0/2818375288 shutdown_connections 2024-05-21T21:58:15.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.473+0000 7fcad6639b80 1 --2- 172.21.15.151:0/2818375288 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556e84fef950 0x556e84fefdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:15.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.473+0000 7fcad6639b80 1 --2- 172.21.15.151:0/2818375288 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556e84ff4e20 0x556e84ff5200 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:15.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.473+0000 7fcad6639b80 1 --2- 172.21.15.151:0/2818375288 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556e84ff5740 0x556e84fef330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:15.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.473+0000 7fcad6639b80 1 -- 172.21.15.151:0/2818375288 >> 172.21.15.151:0/2818375288 conn(0x556e84f14be0 msgr2=0x556e84fe1690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:58:15.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.473+0000 7fcad6639b80 1 -- 172.21.15.151:0/2818375288 shutdown_connections 2024-05-21T21:58:15.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.473+0000 7fcad6639b80 1 -- 172.21.15.151:0/2818375288 wait complete. 2024-05-21T21:58:15.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.474+0000 7fcad6639b80 1 -- start start 2024-05-21T21:58:15.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.474+0000 7fcad6639b80 1 Processor -- start 2024-05-21T21:58:15.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.475+0000 7fcad6639b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556e84fef950 0x556e84ffb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:15.477 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.475+0000 7fcad6639b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556e84ff4e20 0x556e84ff89c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:15.477 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.475+0000 7fcad6639b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556e84ff5740 0x556e84ff8f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:15.477 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.475+0000 7fcad6639b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x556e84e3afa0 con 0x556e84ff4e20 2024-05-21T21:58:15.477 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.475+0000 7fcad6639b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x556e84ed5760 con 0x556e84ff5740 2024-05-21T21:58:15.477 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.475+0000 7fcad6639b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x556e8500cfc0 con 0x556e84fef950 2024-05-21T21:58:15.477 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.475+0000 7fcad6639b80 10 client.0 _ll_register_callbacks cb 0x556e84edec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:58:15.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.476+0000 7fcac9200700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556e84fef950 0x556e84ffb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:15.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.476+0000 7fcac3e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556e84ff4e20 0x556e84ff89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:15.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.476+0000 7fcac3e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556e84ff4e20 0x556e84ff89c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:57274/0 (socket says 172.21.15.151:57274) 2024-05-21T21:58:15.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.476+0000 7fcac3e00700 1 -- 172.21.15.151:0/572352305 learned_addr learned my addr 172.21.15.151:0/572352305 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:58:15.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.476+0000 7fcac9c00700 1 --2- 172.21.15.151:0/572352305 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556e84ff5740 0x556e84ff8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:15.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.476+0000 7fcac9200700 1 -- 172.21.15.151:0/572352305 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556e84ff5740 msgr2=0x556e84ff8f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:15.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.476+0000 7fcac9200700 1 --2- 172.21.15.151:0/572352305 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556e84ff5740 0x556e84ff8f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:15.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.476+0000 7fcac9200700 1 -- 172.21.15.151:0/572352305 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556e84ff4e20 msgr2=0x556e84ff89c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:15.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.476+0000 7fcac9200700 1 --2- 172.21.15.151:0/572352305 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556e84ff4e20 0x556e84ff89c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:15.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.476+0000 7fcac9200700 1 -- 172.21.15.151:0/572352305 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e84e848d0 con 0x556e84fef950 2024-05-21T21:58:15.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.476+0000 7fcac9c00700 1 --2- 172.21.15.151:0/572352305 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556e84ff5740 0x556e84ff8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T21:58:15.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.476+0000 7fcac9200700 1 --2- 172.21.15.151:0/572352305 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556e84fef950 0x556e84ffb620 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fcab80a62d0 tx=0x7fcab8022bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=598c811548acb32b server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:58:15.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.476+0000 7fcaaaa00700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-21T21:58:15.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.477+0000 7fcaaaa00700 1 -- 172.21.15.151:0/572352305 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab80c4060 con 0x556e84fef950 2024-05-21T21:58:15.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.477+0000 7fcaaaa00700 1 -- 172.21.15.151:0/572352305 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fcab80aa6c0 con 0x556e84fef950 2024-05-21T21:58:15.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.477+0000 7fcad6639b80 1 -- 172.21.15.151:0/572352305 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcab80ae600 con 0x556e84fef950 2024-05-21T21:58:15.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.477+0000 7fcaaaa00700 1 -- 172.21.15.151:0/572352305 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcab80aa250 con 0x556e84fef950 2024-05-21T21:58:15.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.477+0000 7fcad6639b80 10 client.25891 fetch_fsmap learned FSMap version 29 2024-05-21T21:58:15.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.477+0000 7fcad6639b80 1 -- 172.21.15.151:0/572352305 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556e84f05060 con 0x556e84fef950 2024-05-21T21:58:15.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.477+0000 7fcaaaa00700 1 -- 172.21.15.151:0/572352305 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcab80ae600 con 0x556e84fef950 2024-05-21T21:58:15.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.478+0000 7fcaaaa00700 1 -- 172.21.15.151:0/572352305 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fcab80a94b0 con 0x556e84fef950 2024-05-21T21:58:15.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.478+0000 7fcad6639b80 10 client.25891 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T21:58:15.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.478+0000 7fcad6639b80 10 client.25891 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T21:58:15.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.478+0000 7fcad6639b80 20 client.25891 populate_metadata read hostname 'smithi151' 2024-05-21T21:58:15.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.478+0000 7fcad6639b80 1 --2- 172.21.15.151:0/572352305 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x556e85019a40 0x556e8501be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:15.481 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.478+0000 7fcad6639b80 4 client.25891 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T21:58:15.481 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.478+0000 7fcad6639b80 1 -- 172.21.15.151:0/572352305 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e84f46d90 con 0x556e85019a40 2024-05-21T21:58:15.481 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.478+0000 7fcac3e00700 1 --2- 172.21.15.151:0/572352305 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x556e85019a40 0x556e8501be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:15.481 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.479+0000 7fcac3e00700 1 --2- 172.21.15.151:0/572352305 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x556e85019a40 0x556e8501be30 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a36750941a1bbb95 server_cookie=c1ee2d5cf012c9a6 in_seq=0 out_seq=0 2024-05-21T21:58:15.481 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.479+0000 7fcaaaa00700 10 client.25891 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T21:58:15.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.480+0000 7fcaaaa00700 1 -- 172.21.15.151:0/572352305 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x556e84f46d90 con 0x556e85019a40 2024-05-21T21:58:15.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.480+0000 7fcaaaa00700 10 client.25891 handle_command_reply: tid=0 2024-05-21T21:58:15.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.480+0000 7fcad6639b80 1 client.25891 shutdown 2024-05-21T21:58:15.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.480+0000 7fcad6639b80 1 -- 172.21.15.151:0/572352305 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556e84fef950 msgr2=0x556e84ffb620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:15.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.480+0000 7fcad6639b80 1 --2- 172.21.15.151:0/572352305 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556e84fef950 0x556e84ffb620 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fcab80a62d0 tx=0x7fcab8022bf0 comp rx=0 tx=0).stop 2024-05-21T21:58:15.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.481+0000 7fcad6639b80 1 -- 172.21.15.151:0/572352305 shutdown_connections 2024-05-21T21:58:15.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.481+0000 7fcad6639b80 1 --2- 172.21.15.151:0/572352305 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x556e85019a40 0x556e8501be30 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:15.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.481+0000 7fcad6639b80 1 --2- 172.21.15.151:0/572352305 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556e84ff5740 0x556e84ff8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:15.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.481+0000 7fcad6639b80 1 --2- 172.21.15.151:0/572352305 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556e84fef950 0x556e84ffb620 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:15.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.481+0000 7fcad6639b80 1 --2- 172.21.15.151:0/572352305 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556e84ff4e20 0x556e84ff89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:15.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.481+0000 7fcad6639b80 1 -- 172.21.15.151:0/572352305 >> 172.21.15.151:0/572352305 conn(0x556e84f14be0 msgr2=0x556e84fe0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:58:15.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.481+0000 7fcaaaa00700 0 client.25891 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T21:58:15.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.481+0000 7fcad6639b80 1 -- 172.21.15.151:0/572352305 shutdown_connections 2024-05-21T21:58:15.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.481+0000 7fcad6639b80 1 -- 172.21.15.151:0/572352305 wait complete. 2024-05-21T21:58:15.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.481+0000 7fcad6639b80 20 client.25891 trim_cache size 0 max 16384 2024-05-21T21:58:15.501 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.500+0000 7fcad6639b80 1 Processor -- start 2024-05-21T21:58:15.501 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.500+0000 7fcad6639b80 1 -- start start 2024-05-21T21:58:15.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.500+0000 7fcad6639b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556e84fef950 0x556e85015f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:15.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.500+0000 7fcad6639b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556e84ff4e20 0x556e84ff4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:15.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.500+0000 7fcad6639b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556e84ff5740 0x556e84ff2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:15.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.500+0000 7fcad6639b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcac406ff10 con 0x556e84ff4e20 2024-05-21T21:58:15.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.500+0000 7fcad6639b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcac4071a50 con 0x556e84fef950 2024-05-21T21:58:15.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.500+0000 7fcad6639b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fcac40705e0 con 0x556e84ff5740 2024-05-21T21:58:15.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.501+0000 7fcac3e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556e84ff4e20 0x556e84ff4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:15.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.501+0000 7fcac9200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556e84fef950 0x556e85015f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:15.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.501+0000 7fcac9c00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556e84ff5740 0x556e84ff2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:15.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.501+0000 7fcac3e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556e84ff4e20 0x556e84ff4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:57282/0 (socket says 172.21.15.151:57282) 2024-05-21T21:58:15.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.501+0000 7fcac9200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556e84fef950 0x556e85015f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:34160/0 (socket says 172.21.15.151:34160) 2024-05-21T21:58:15.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.501+0000 7fcac3e00700 1 -- 172.21.15.151:0/2780911864 learned_addr learned my addr 172.21.15.151:0/2780911864 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:58:15.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.501+0000 7fcac9c00700 1 -- 172.21.15.151:0/2780911864 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556e84fef950 msgr2=0x556e85015f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:15.504 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.501+0000 7fcac9c00700 1 --2- 172.21.15.151:0/2780911864 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556e84fef950 0x556e85015f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:15.504 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.501+0000 7fcac9c00700 1 -- 172.21.15.151:0/2780911864 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556e84ff4e20 msgr2=0x556e84ff4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:15.504 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.501+0000 7fcac9c00700 1 --2- 172.21.15.151:0/2780911864 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556e84ff4e20 0x556e84ff4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:15.504 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.501+0000 7fcac9c00700 1 -- 172.21.15.151:0/2780911864 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e84ff6f30 con 0x556e84ff5740 2024-05-21T21:58:15.504 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.501+0000 7fcac3e00700 1 --2- 172.21.15.151:0/2780911864 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556e84ff4e20 0x556e84ff4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:58:15.504 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.501+0000 7fcac9200700 1 --2- 172.21.15.151:0/2780911864 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556e84fef950 0x556e85015f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:58:15.504 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.502+0000 7fcac9c00700 1 --2- 172.21.15.151:0/2780911864 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556e84ff5740 0x556e84ff2fa0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fcabc009fd0 tx=0x7fcabc00b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32b06b66edf5782 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:58:15.504 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.502+0000 7fcab5600700 1 -- 172.21.15.151:0/2780911864 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcabc003c00 con 0x556e84ff5740 2024-05-21T21:58:15.504 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.502+0000 7fcab5600700 1 -- 172.21.15.151:0/2780911864 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcabc003da0 con 0x556e84ff5740 2024-05-21T21:58:15.504 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.502+0000 7fcab5600700 1 -- 172.21.15.151:0/2780911864 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcabc017910 con 0x556e84ff5740 2024-05-21T21:58:15.505 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.503+0000 7fcad6639b80 1 -- 172.21.15.151:0/2780911864 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556e84ff5740 msgr2=0x556e84ff2fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:15.505 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.503+0000 7fcad6639b80 1 --2- 172.21.15.151:0/2780911864 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556e84ff5740 0x556e84ff2fa0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fcabc009fd0 tx=0x7fcabc00b040 comp rx=0 tx=0).stop 2024-05-21T21:58:15.505 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.503+0000 7fcad6639b80 1 -- 172.21.15.151:0/2780911864 shutdown_connections 2024-05-21T21:58:15.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.503+0000 7fcad6639b80 1 --2- 172.21.15.151:0/2780911864 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556e84fef950 0x556e85015f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:15.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.503+0000 7fcad6639b80 1 --2- 172.21.15.151:0/2780911864 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556e84ff5740 0x556e84ff2fa0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:15.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.503+0000 7fcad6639b80 1 --2- 172.21.15.151:0/2780911864 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556e84ff4e20 0x556e84ff4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:15.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.503+0000 7fcad6639b80 1 -- 172.21.15.151:0/2780911864 >> 172.21.15.151:0/2780911864 conn(0x556e84f14be0 msgr2=0x556e84ff5c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:58:15.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.503+0000 7fcad6639b80 1 -- 172.21.15.151:0/2780911864 shutdown_connections 2024-05-21T21:58:15.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.503+0000 7fcad6639b80 1 -- 172.21.15.151:0/2780911864 wait complete. 2024-05-21T21:58:15.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.503+0000 7fcad6639b80 1 -- start start 2024-05-21T21:58:15.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.504+0000 7fcad6639b80 1 Processor -- start 2024-05-21T21:58:15.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.504+0000 7fcad6639b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556e84fef950 0x556e85020f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:15.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.504+0000 7fcad6639b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556e84ff4e20 0x556e85021440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:15.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.504+0000 7fcad6639b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556e84ff5740 0x556e85021980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:15.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.504+0000 7fcad6639b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcac4071380 con 0x556e84fef950 2024-05-21T21:58:15.507 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.504+0000 7fcad6639b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcac4070cb0 con 0x556e84ff4e20 2024-05-21T21:58:15.507 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.504+0000 7fcad6639b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fcac41076e0 con 0x556e84ff5740 2024-05-21T21:58:15.507 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.504+0000 7fcad6639b80 10 client.0 _ll_register_callbacks cb 0x556e84edec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:58:15.509 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.505+0000 7fcac9200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556e84fef950 0x556e85020f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:15.509 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.505+0000 7fcac9c00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556e84ff5740 0x556e85021980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:15.510 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.505+0000 7fcac9200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556e84fef950 0x556e85020f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:57286/0 (socket says 172.21.15.151:57286) 2024-05-21T21:58:15.510 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.505+0000 7fcac9200700 1 -- 172.21.15.151:0/1464695271 learned_addr learned my addr 172.21.15.151:0/1464695271 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:58:15.510 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.505+0000 7fcac3e00700 1 --2- 172.21.15.151:0/1464695271 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556e84ff4e20 0x556e85021440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:15.510 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.505+0000 7fcac9200700 1 -- 172.21.15.151:0/1464695271 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556e84ff5740 msgr2=0x556e85021980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:15.510 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.505+0000 7fcac9200700 1 --2- 172.21.15.151:0/1464695271 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556e84ff5740 0x556e85021980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:15.510 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.505+0000 7fcac9200700 1 -- 172.21.15.151:0/1464695271 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556e84ff4e20 msgr2=0x556e85021440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:15.510 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.505+0000 7fcac9200700 1 --2- 172.21.15.151:0/1464695271 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556e84ff4e20 0x556e85021440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:15.510 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.505+0000 7fcac9200700 1 -- 172.21.15.151:0/1464695271 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e85083360 con 0x556e84fef950 2024-05-21T21:58:15.510 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.505+0000 7fcac9c00700 1 --2- 172.21.15.151:0/1464695271 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556e84ff5740 0x556e85021980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:58:15.510 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.505+0000 7fcac9200700 1 --2- 172.21.15.151:0/1464695271 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556e84fef950 0x556e85020f00 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fcab80b2f90 tx=0x7fcab80e0730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a69c222b0ea09985 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:58:15.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.505+0000 7fcaaaa00700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-21T21:58:15.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.506+0000 7fcaaaa00700 1 -- 172.21.15.151:0/1464695271 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab80c4070 con 0x556e84fef950 2024-05-21T21:58:15.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.506+0000 7fcad6639b80 1 -- 172.21.15.151:0/1464695271 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcabc003da0 con 0x556e84fef950 2024-05-21T21:58:15.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.506+0000 7fcaaaa00700 1 -- 172.21.15.151:0/1464695271 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fcab80ae170 con 0x556e84fef950 2024-05-21T21:58:15.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.506+0000 7fcaaaa00700 1 -- 172.21.15.151:0/1464695271 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcab80b2da0 con 0x556e84fef950 2024-05-21T21:58:15.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.506+0000 7fcad6639b80 10 client.15756 fetch_fsmap learned FSMap version 29 2024-05-21T21:58:15.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.506+0000 7fcad6639b80 1 -- 172.21.15.151:0/1464695271 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556e85083360 con 0x556e84fef950 2024-05-21T21:58:15.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.506+0000 7fcaaaa00700 1 -- 172.21.15.151:0/1464695271 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcabc003da0 con 0x556e84fef950 2024-05-21T21:58:15.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.506+0000 7fcaaaa00700 1 -- 172.21.15.151:0/1464695271 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fcab80ca030 con 0x556e84fef950 2024-05-21T21:58:15.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.506+0000 7fcad6639b80 10 client.15756 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T21:58:15.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.506+0000 7fcad6639b80 10 client.15756 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T21:58:15.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.506+0000 7fcad6639b80 20 client.15756 populate_metadata read hostname 'smithi151' 2024-05-21T21:58:15.512 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.506+0000 7fcad6639b80 1 --2- 172.21.15.151:0/1464695271 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x556e850847e0 0x556e85086c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:15.512 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.507+0000 7fcad6639b80 4 client.15756 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T21:58:15.512 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.507+0000 7fcad6639b80 1 -- 172.21.15.151:0/1464695271 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556e84f40570 con 0x556e850847e0 2024-05-21T21:58:15.512 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.507+0000 7fcac3e00700 1 --2- 172.21.15.151:0/1464695271 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x556e850847e0 0x556e85086c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:15.512 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.507+0000 7fcac3e00700 1 --2- 172.21.15.151:0/1464695271 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x556e850847e0 0x556e85086c00 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0ec7fa409b083aa server_cookie=fc9d640d8fb5eb4e in_seq=0 out_seq=0 2024-05-21T21:58:15.512 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.507+0000 7fcaaaa00700 10 client.15756 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T21:58:15.512 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.508+0000 7fcaaaa00700 1 -- 172.21.15.151:0/1464695271 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556e84f40570 con 0x556e850847e0 2024-05-21T21:58:15.512 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.508+0000 7fcaaaa00700 10 client.15756 handle_command_reply: tid=0 2024-05-21T21:58:15.512 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.508+0000 7fcad6639b80 1 client.15756 shutdown 2024-05-21T21:58:15.512 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.508+0000 7fcad6639b80 1 -- 172.21.15.151:0/1464695271 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556e84fef950 msgr2=0x556e85020f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:15.512 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.508+0000 7fcad6639b80 1 --2- 172.21.15.151:0/1464695271 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556e84fef950 0x556e85020f00 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fcab80b2f90 tx=0x7fcab80e0730 comp rx=0 tx=0).stop 2024-05-21T21:58:15.512 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.508+0000 7fcad6639b80 1 -- 172.21.15.151:0/1464695271 shutdown_connections 2024-05-21T21:58:15.512 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.508+0000 7fcad6639b80 1 --2- 172.21.15.151:0/1464695271 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556e84ff5740 0x556e85021980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:15.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.508+0000 7fcad6639b80 1 --2- 172.21.15.151:0/1464695271 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556e84fef950 0x556e85020f00 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:15.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.508+0000 7fcad6639b80 1 --2- 172.21.15.151:0/1464695271 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x556e850847e0 0x556e85086c00 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:15.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.508+0000 7fcad6639b80 1 --2- 172.21.15.151:0/1464695271 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556e84ff4e20 0x556e85021440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:15.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.508+0000 7fcaaaa00700 0 client.15756 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T21:58:15.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.508+0000 7fcad6639b80 1 -- 172.21.15.151:0/1464695271 >> 172.21.15.151:0/1464695271 conn(0x556e84f14be0 msgr2=0x556e84ff6490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:58:15.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.508+0000 7fcad6639b80 1 -- 172.21.15.151:0/1464695271 shutdown_connections 2024-05-21T21:58:15.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.508+0000 7fcad6639b80 1 -- 172.21.15.151:0/1464695271 wait complete. 2024-05-21T21:58:15.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.508+0000 7fcad6639b80 20 client.15756 trim_cache size 0 max 16384 2024-05-21T21:58:15.513 INFO:teuthology.orchestra.run.smithi151.stdout:[] 2024-05-21T21:58:15.514 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.509+0000 7fcacbc00700 1 -- 172.21.15.151:0/3204159282 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fcaac0b29f0 msgr2=0x7fcaac0b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:15.514 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.509+0000 7fcacbc00700 1 --2- 172.21.15.151:0/3204159282 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fcaac0b29f0 0x7fcaac0b4eb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fcab0009680 tx=0x7fcab0009260 comp rx=0 tx=0).stop 2024-05-21T21:58:15.514 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.509+0000 7fcacbc00700 1 -- 172.21.15.151:0/3204159282 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcac4075770 msgr2=0x7fcac419be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:15.514 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.509+0000 7fcacbc00700 1 --2- 172.21.15.151:0/3204159282 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcac4075770 0x7fcac419be40 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fcab8006010 tx=0x7fcab80057c0 comp rx=0 tx=0).stop 2024-05-21T21:58:15.514 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.509+0000 7fcacbc00700 1 -- 172.21.15.151:0/3204159282 shutdown_connections 2024-05-21T21:58:15.514 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.509+0000 7fcacbc00700 1 --2- 172.21.15.151:0/3204159282 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fcaac0b29f0 0x7fcaac0b4eb0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:15.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.509+0000 7fcacbc00700 1 --2- 172.21.15.151:0/3204159282 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcac410ff40 0x7fcac41a0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:15.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.509+0000 7fcacbc00700 1 --2- 172.21.15.151:0/3204159282 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcac4075770 0x7fcac419be40 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:15.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.509+0000 7fcacbc00700 1 --2- 172.21.15.151:0/3204159282 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcac4076090 0x7fcac419c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:15.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.509+0000 7fcacbc00700 1 -- 172.21.15.151:0/3204159282 >> 172.21.15.151:0/3204159282 conn(0x7fcac407a6a0 msgr2=0x7fcac4101ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:58:15.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.509+0000 7fcacbc00700 1 -- 172.21.15.151:0/3204159282 shutdown_connections 2024-05-21T21:58:15.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:15.509+0000 7fcacbc00700 1 -- 172.21.15.151:0/3204159282 wait complete. 2024-05-21T21:58:15.559 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:58:15 smithi151 ceph-mon[31623]: pgmap v82: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-05-21T21:58:15.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:58:15 smithi184 ceph-mon[36942]: pgmap v82: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-05-21T21:58:15.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:58:15 smithi164 ceph-mon[36269]: pgmap v82: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-05-21T21:58:17.702 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:58:17 smithi151 ceph-mon[31623]: pgmap v83: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-05-21T21:58:17.727 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-21T21:58:17.727 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-21T21:58:17.735 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:58:17 smithi184 ceph-mon[36942]: pgmap v83: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-05-21T21:58:17.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:58:17 smithi164 ceph-mon[36269]: pgmap v83: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-05-21T21:58:18.319 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:18.317+0000 7f9599600700 1 -- 172.21.15.151:0/630968217 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9594103f50 msgr2=0x7f9594110430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:18.319 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:18.317+0000 7f9599600700 1 --2- 172.21.15.151:0/630968217 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9594103f50 0x7f9594110430 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f958c00b0a0 tx=0x7f958c00b3b0 comp rx=0 tx=0).stop 2024-05-21T21:58:18.319 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:18.317+0000 7f9599600700 1 -- 172.21.15.151:0/630968217 shutdown_connections 2024-05-21T21:58:18.319 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:18.317+0000 7f9599600700 1 --2- 172.21.15.151:0/630968217 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f95941096a0 0x7f9594109a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:18.319 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:18.317+0000 7f9599600700 1 --2- 172.21.15.151:0/630968217 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9594103f50 0x7f9594110430 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:18.319 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:18.317+0000 7f9599600700 1 --2- 172.21.15.151:0/630968217 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9594103590 0x7f9594103a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:18.319 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:18.317+0000 7f9599600700 1 -- 172.21.15.151:0/630968217 >> 172.21.15.151:0/630968217 conn(0x7f959407a6f0 msgr2=0x7f959407ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:58:18.320 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:18.317+0000 7f9599600700 1 -- 172.21.15.151:0/630968217 shutdown_connections 2024-05-21T21:58:18.320 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:18.317+0000 7f9599600700 1 -- 172.21.15.151:0/630968217 wait complete. 2024-05-21T21:58:18.320 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:18.319+0000 7f9599600700 1 Processor -- start 2024-05-21T21:58:18.320 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:18.319+0000 7f9599600700 1 -- start start 2024-05-21T21:58:18.321 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:18.320+0000 7f9599600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9594103590 0x7f9594075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:18.321 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:18.320+0000 7f9599600700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9594103f50 0x7f9594075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:18.321 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:18.320+0000 7f9599600700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95941096a0 0x7f95940762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:18.321 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:18.320+0000 7f9599600700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9594113770 con 0x7f9594103590 2024-05-21T21:58:18.321 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:18.320+0000 7f9599600700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f95941135f0 con 0x7f9594103f50 2024-05-21T21:58:18.321 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:18.320+0000 7f9599600700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f95941138f0 con 0x7f95941096a0 2024-05-21T21:58:18.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:18.320+0000 7f9592a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9594103590 0x7f9594075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:18.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:18.320+0000 7f9593400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95941096a0 0x7f95940762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:18.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:18.320+0000 7f9593400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95941096a0 0x7f95940762c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:33372/0 (socket says 172.21.15.151:33372) 2024-05-21T21:58:18.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:18.320+0000 7f9593400700 1 -- 172.21.15.151:0/3584941957 learned_addr learned my addr 172.21.15.151:0/3584941957 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:58:18.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:18.320+0000 7f9592000700 1 --2- 172.21.15.151:0/3584941957 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9594103f50 0x7f9594075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:18.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:18.321+0000 7f9593400700 1 -- 172.21.15.151:0/3584941957 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9594103f50 msgr2=0x7f9594075d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:18.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:18.321+0000 7f9593400700 1 --2- 172.21.15.151:0/3584941957 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9594103f50 0x7f9594075d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:18.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:18.321+0000 7f9593400700 1 -- 172.21.15.151:0/3584941957 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9594103590 msgr2=0x7f9594075840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:18.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:18.321+0000 7f9593400700 1 --2- 172.21.15.151:0/3584941957 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9594103590 0x7f9594075840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:18.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:18.321+0000 7f9593400700 1 -- 172.21.15.151:0/3584941957 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95941a8100 con 0x7f95941096a0 2024-05-21T21:58:18.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:18.321+0000 7f9593400700 1 --2- 172.21.15.151:0/3584941957 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95941096a0 0x7f95940762c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f958c00b070 tx=0x7f958c004c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:58:18.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:18.321+0000 7f958b400700 1 -- 172.21.15.151:0/3584941957 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f958c021070 con 0x7f95941096a0 2024-05-21T21:58:18.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:18.321+0000 7f9599600700 1 -- 172.21.15.151:0/3584941957 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95941a8390 con 0x7f95941096a0 2024-05-21T21:58:18.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:18.321+0000 7f958b400700 1 -- 172.21.15.151:0/3584941957 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f958c007e30 con 0x7f95941096a0 2024-05-21T21:58:18.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:18.321+0000 7f958b400700 1 -- 172.21.15.151:0/3584941957 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f958c004570 con 0x7f95941096a0 2024-05-21T21:58:18.324 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:18.322+0000 7f9599600700 1 -- 172.21.15.151:0/3584941957 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95941a8890 con 0x7f95941096a0 2024-05-21T21:58:18.324 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:18.323+0000 7f9599600700 1 -- 172.21.15.151:0/3584941957 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f959406a230 con 0x7f95941096a0 2024-05-21T21:58:18.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:18.326+0000 7f958b400700 1 -- 172.21.15.151:0/3584941957 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7f958c004710 con 0x7f95941096a0 2024-05-21T21:58:18.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:18.328+0000 7f958b400700 1 --2- 172.21.15.151:0/3584941957 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f95780b29f0 0x7f95780b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:18.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:18.328+0000 7f958b400700 1 -- 172.21.15.151:0/3584941957 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f958c00f020 con 0x7f95941096a0 2024-05-21T21:58:18.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:18.328+0000 7f9592a00700 1 --2- 172.21.15.151:0/3584941957 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f95780b29f0 0x7f95780b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:18.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:18.329+0000 7f9592a00700 1 --2- 172.21.15.151:0/3584941957 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f95780b29f0 0x7f95780b4eb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9584009be0 tx=0x7f9584009260 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:58:18.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:18.329+0000 7f958b400700 1 -- 172.21.15.151:0/3584941957 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f958c02a020 con 0x7f95941096a0 2024-05-21T21:58:18.342 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:58:18.623 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:18.621+0000 7f9599600700 1 -- 172.21.15.151:0/3584941957 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f959402e560 con 0x7f95941096a0 2024-05-21T21:58:18.623 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:18.622+0000 7f958b400700 1 -- 172.21.15.151:0/3584941957 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11958 (secure 0 0 0) 0x7f958c0ab700 con 0x7f95941096a0 2024-05-21T21:58:18.624 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T21:58:18.624 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14358,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/158010529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":158010529},{"type":"v1","addr":"172.21.15.151:6835","nonce":158010529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6839/2069058115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":2069058115},{"type":"v1","addr":"172.21.15.151:6839","nonce":2069058115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T21:51:19.552069+0000","modified":"2024-05-21T21:55:34.643755+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24530,"mds_1":24493,"mds_2":24491,"mds_3":24478,"mds_4":24473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14403":{"gid":14403,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6841/1749723794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6840","nonce":1749723794},{"type":"v1","addr":"172.21.15.151:6841","nonce":1749723794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/2026266114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":2026266114},{"type":"v1","addr":"172.21.15.184:6833","nonce":2026266114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/2467900844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":2467900844},{"type":"v1","addr":"172.21.15.164:6833","nonce":2467900844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3367640960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3367640960},{"type":"v1","addr":"172.21.15.151:6837","nonce":3367640960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/3264142621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3264142621},{"type":"v1","addr":"172.21.15.184:6835","nonce":3264142621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":17,"addr":"172.21.15.164:6835/2340428470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2340428470},{"type":"v1","addr":"172.21.15.164:6835","nonce":2340428470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":16,"addr":"172.21.15.184:6837/2988194888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":2988194888},{"type":"v1","addr":"172.21.15.184:6837","nonce":2988194888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.164:6837/1554609572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1554609572},{"type":"v1","addr":"172.21.15.164:6837","nonce":1554609572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.184:6839/3059935143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":3059935143},{"type":"v1","addr":"172.21.15.184:6839","nonce":3059935143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.164:6839/2646401247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":2646401247},{"type":"v1","addr":"172.21.15.164:6839","nonce":2646401247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24530,"qdb_cluster":[24530,24493,24491,24473,24478]},"id":1}]} 2024-05-21T21:58:18.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:18.626+0000 7f9599600700 1 -- 172.21.15.151:0/3584941957 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f95780b29f0 msgr2=0x7f95780b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:18.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:18.626+0000 7f9599600700 1 --2- 172.21.15.151:0/3584941957 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f95780b29f0 0x7f95780b4eb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9584009be0 tx=0x7f9584009260 comp rx=0 tx=0).stop 2024-05-21T21:58:18.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:18.626+0000 7f9599600700 1 -- 172.21.15.151:0/3584941957 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95941096a0 msgr2=0x7f95940762c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:18.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:18.626+0000 7f9599600700 1 --2- 172.21.15.151:0/3584941957 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95941096a0 0x7f95940762c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f958c00b070 tx=0x7f958c004c80 comp rx=0 tx=0).stop 2024-05-21T21:58:18.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:18.626+0000 7f9599600700 1 -- 172.21.15.151:0/3584941957 shutdown_connections 2024-05-21T21:58:18.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:18.626+0000 7f9599600700 1 --2- 172.21.15.151:0/3584941957 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f95780b29f0 0x7f95780b4eb0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:18.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:18.626+0000 7f9599600700 1 --2- 172.21.15.151:0/3584941957 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95941096a0 0x7f95940762c0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:18.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:18.626+0000 7f9599600700 1 --2- 172.21.15.151:0/3584941957 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9594103590 0x7f9594075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:18.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:18.626+0000 7f9599600700 1 --2- 172.21.15.151:0/3584941957 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9594103f50 0x7f9594075d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:18.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:18.626+0000 7f9599600700 1 -- 172.21.15.151:0/3584941957 >> 172.21.15.151:0/3584941957 conn(0x7f959407a6f0 msgr2=0x7f9594102080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:58:18.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:18.626+0000 7f9599600700 1 -- 172.21.15.151:0/3584941957 shutdown_connections 2024-05-21T21:58:18.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:18.626+0000 7f9599600700 1 -- 172.21.15.151:0/3584941957 wait complete. 2024-05-21T21:58:18.629 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 29 2024-05-21T21:58:19.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:58:19 smithi151 ceph-mon[31623]: pgmap v84: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 12 op/s 2024-05-21T21:58:19.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:58:19 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/3584941957' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T21:58:19.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:58:19 smithi184 ceph-mon[36942]: pgmap v84: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 12 op/s 2024-05-21T21:58:19.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:58:19 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/3584941957' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T21:58:19.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:58:19 smithi164 ceph-mon[36269]: pgmap v84: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 12 op/s 2024-05-21T21:58:19.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:58:19 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/3584941957' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T21:58:20.433 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24530 quiesce db --set-id acaf991b --release --await' 2024-05-21T21:58:20.433 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24530 quiesce db --set-id acaf991b --release --await 2024-05-21T21:58:20.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.979+0000 7f67a7200700 1 -- 172.21.15.151:0/4161897027 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f67a0104fb0 msgr2=0x7f67a010d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:20.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.979+0000 7f67a7200700 1 --2- 172.21.15.151:0/4161897027 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f67a0104fb0 0x7f67a010d4a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f6790009990 tx=0x7f6790009ca0 comp rx=0 tx=0).stop 2024-05-21T21:58:20.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.979+0000 7f67a7200700 1 -- 172.21.15.151:0/4161897027 shutdown_connections 2024-05-21T21:58:20.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.979+0000 7f67a7200700 1 --2- 172.21.15.151:0/4161897027 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f67a0104690 0x7f67a0104a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:20.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.979+0000 7f67a7200700 1 --2- 172.21.15.151:0/4161897027 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f67a010da70 0x7f67a010ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:20.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.979+0000 7f67a7200700 1 --2- 172.21.15.151:0/4161897027 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f67a0104fb0 0x7f67a010d4a0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:20.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.979+0000 7f67a7200700 1 -- 172.21.15.151:0/4161897027 >> 172.21.15.151:0/4161897027 conn(0x7f67a00fde60 msgr2=0x7f67a0100280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:58:20.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.979+0000 7f67a7200700 1 -- 172.21.15.151:0/4161897027 shutdown_connections 2024-05-21T21:58:20.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.979+0000 7f67a7200700 1 -- 172.21.15.151:0/4161897027 wait complete. 2024-05-21T21:58:20.982 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.980+0000 7f67a7200700 1 Processor -- start 2024-05-21T21:58:20.982 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.980+0000 7f67a7200700 1 -- start start 2024-05-21T21:58:20.982 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.981+0000 7f67a7200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f67a0104690 0x7f67a019be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:20.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.981+0000 7f67a7200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f67a0104fb0 0x7f67a019c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:20.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.981+0000 7f67a7200700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f67a010da70 0x7f67a01a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:20.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.981+0000 7f67a7200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f67a0103e10 con 0x7f67a0104690 2024-05-21T21:58:20.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.981+0000 7f67a7200700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f67a0103c90 con 0x7f67a0104fb0 2024-05-21T21:58:20.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.981+0000 7f67a7200700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f67a0103f90 con 0x7f67a010da70 2024-05-21T21:58:20.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.981+0000 7f67a5e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f67a0104690 0x7f67a019be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:20.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.981+0000 7f67a5e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f67a0104690 0x7f67a019be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52414/0 (socket says 172.21.15.151:52414) 2024-05-21T21:58:20.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.981+0000 7f67a5e00700 1 -- 172.21.15.151:0/2161366302 learned_addr learned my addr 172.21.15.151:0/2161366302 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:58:20.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.981+0000 7f67a5400700 1 --2- 172.21.15.151:0/2161366302 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f67a0104fb0 0x7f67a019c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:20.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.982+0000 7f67a6800700 1 --2- 172.21.15.151:0/2161366302 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f67a010da70 0x7f67a01a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:20.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.982+0000 7f67a5e00700 1 -- 172.21.15.151:0/2161366302 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f67a010da70 msgr2=0x7f67a01a06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:20.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.982+0000 7f67a5e00700 1 --2- 172.21.15.151:0/2161366302 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f67a010da70 0x7f67a01a06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:20.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.982+0000 7f67a5e00700 1 -- 172.21.15.151:0/2161366302 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f67a0104fb0 msgr2=0x7f67a019c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:20.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.982+0000 7f67a5e00700 1 --2- 172.21.15.151:0/2161366302 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f67a0104fb0 0x7f67a019c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:20.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.982+0000 7f67a5e00700 1 -- 172.21.15.151:0/2161366302 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67a01a0de0 con 0x7f67a0104690 2024-05-21T21:58:20.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.982+0000 7f67a5400700 1 --2- 172.21.15.151:0/2161366302 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f67a0104fb0 0x7f67a019c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:58:20.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.982+0000 7f67a5e00700 1 --2- 172.21.15.151:0/2161366302 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f67a0104690 0x7f67a019be20 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f679400e9a0 tx=0x7f679400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:58:20.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.982+0000 7f679aa00700 1 -- 172.21.15.151:0/2161366302 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f679400cbd0 con 0x7f67a0104690 2024-05-21T21:58:20.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.982+0000 7f679aa00700 1 -- 172.21.15.151:0/2161366302 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f679400cd70 con 0x7f67a0104690 2024-05-21T21:58:20.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.982+0000 7f67a7200700 1 -- 172.21.15.151:0/2161366302 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67a01a10d0 con 0x7f67a0104690 2024-05-21T21:58:20.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.983+0000 7f679aa00700 1 -- 172.21.15.151:0/2161366302 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6794018680 con 0x7f67a0104690 2024-05-21T21:58:20.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.983+0000 7f67a7200700 1 -- 172.21.15.151:0/2161366302 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67a01a8950 con 0x7f67a0104690 2024-05-21T21:58:20.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.988+0000 7f679aa00700 1 -- 172.21.15.151:0/2161366302 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7f6794018820 con 0x7f67a0104690 2024-05-21T21:58:20.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.989+0000 7f67b30c7b80 1 Processor -- start 2024-05-21T21:58:20.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.989+0000 7f67b30c7b80 1 -- start start 2024-05-21T21:58:20.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.989+0000 7f67b30c7b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c967260a0 0x563c96726480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:20.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.989+0000 7f67b30c7b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563c967269c0 0x563c96720530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:20.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.989+0000 7f67b30c7b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c96720b50 0x563c96720fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:20.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.989+0000 7f67b30c7b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x563c9667eec0 con 0x563c967269c0 2024-05-21T21:58:20.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.989+0000 7f67b30c7b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x563c9666ea60 con 0x563c967260a0 2024-05-21T21:58:20.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.989+0000 7f67b30c7b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x563c96672aa0 con 0x563c96720b50 2024-05-21T21:58:20.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.990+0000 7f67a6800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c967260a0 0x563c96726480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:20.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.990+0000 7f67a6800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c967260a0 0x563c96726480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:37794/0 (socket says 172.21.15.151:37794) 2024-05-21T21:58:20.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.990+0000 7f67a6800700 1 -- 172.21.15.151:0/1777764020 learned_addr learned my addr 172.21.15.151:0/1777764020 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:58:20.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.990+0000 7f67a5e00700 1 --2- 172.21.15.151:0/1777764020 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563c967269c0 0x563c96720530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:20.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.990+0000 7f67a6800700 1 -- 172.21.15.151:0/1777764020 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c96720b50 msgr2=0x563c96720fb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-21T21:58:20.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.990+0000 7f67a6800700 1 --2- 172.21.15.151:0/1777764020 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c96720b50 0x563c96720fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:20.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.990+0000 7f67a6800700 1 -- 172.21.15.151:0/1777764020 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563c967269c0 msgr2=0x563c96720530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:20.994 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.990+0000 7f67a6800700 1 --2- 172.21.15.151:0/1777764020 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563c967269c0 0x563c96720530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:20.994 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.990+0000 7f67a6800700 1 -- 172.21.15.151:0/1777764020 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c966374a0 con 0x563c967260a0 2024-05-21T21:58:20.994 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.990+0000 7f67a6800700 1 --2- 172.21.15.151:0/1777764020 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c967260a0 0x563c96726480 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f679c009460 tx=0x7f679c009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50b3994a3b5c86c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:58:20.994 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.991+0000 7f678f400700 1 -- 172.21.15.151:0/1777764020 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f679c013810 con 0x563c967260a0 2024-05-21T21:58:20.994 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.991+0000 7f678f400700 1 -- 172.21.15.151:0/1777764020 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f679c015440 con 0x563c967260a0 2024-05-21T21:58:20.994 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.991+0000 7f678f400700 1 -- 172.21.15.151:0/1777764020 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f679c013e60 con 0x563c967260a0 2024-05-21T21:58:20.994 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.991+0000 7f679aa00700 1 --2- 172.21.15.151:0/2161366302 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f67880b2a70 0x7f67880b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:20.994 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.991+0000 7f679aa00700 1 -- 172.21.15.151:0/2161366302 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f67940e27c0 con 0x7f67a0104690 2024-05-21T21:58:20.994 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.992+0000 7f67b30c7b80 1 -- 172.21.15.151:0/1777764020 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c967260a0 msgr2=0x563c96726480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:20.994 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.992+0000 7f67b30c7b80 1 --2- 172.21.15.151:0/1777764020 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c967260a0 0x563c96726480 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f679c009460 tx=0x7f679c009920 comp rx=0 tx=0).stop 2024-05-21T21:58:20.994 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.992+0000 7f67a5e00700 1 --2- 172.21.15.151:0/2161366302 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f67880b2a70 0x7f67880b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:20.994 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.992+0000 7f67b30c7b80 1 -- 172.21.15.151:0/1777764020 shutdown_connections 2024-05-21T21:58:20.995 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.992+0000 7f67b30c7b80 1 --2- 172.21.15.151:0/1777764020 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c967260a0 0x563c96726480 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:20.995 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.992+0000 7f67b30c7b80 1 --2- 172.21.15.151:0/1777764020 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c96720b50 0x563c96720fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:20.995 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.992+0000 7f67b30c7b80 1 --2- 172.21.15.151:0/1777764020 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563c967269c0 0x563c96720530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:20.995 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.992+0000 7f67b30c7b80 1 -- 172.21.15.151:0/1777764020 >> 172.21.15.151:0/1777764020 conn(0x563c9664ec60 msgr2=0x563c9671b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:58:20.995 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.992+0000 7f67b30c7b80 1 -- 172.21.15.151:0/1777764020 shutdown_connections 2024-05-21T21:58:20.995 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.992+0000 7f67b30c7b80 1 -- 172.21.15.151:0/1777764020 wait complete. 2024-05-21T21:58:20.995 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.992+0000 7f67a5e00700 1 --2- 172.21.15.151:0/2161366302 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f67880b2a70 0x7f67880b4f30 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f67940dffd0 tx=0x7f67940e8550 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:58:20.998 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.993+0000 7f67b30c7b80 1 -- start start 2024-05-21T21:58:20.998 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.993+0000 7f67b30c7b80 1 Processor -- start 2024-05-21T21:58:20.998 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.994+0000 7f67b30c7b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c96720b50 0x563c9672c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:20.998 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.994+0000 7f67b30c7b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563c967260a0 0x563c96729a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:20.998 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.994+0000 7f67b30c7b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c967269c0 0x563c96729f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:20.998 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.994+0000 7f67b30c7b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x563c96575020 con 0x563c967260a0 2024-05-21T21:58:20.998 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.994+0000 7f67b30c7b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x563c9660f7e0 con 0x563c96720b50 2024-05-21T21:58:20.998 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.994+0000 7f67b30c7b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x563c9672d4e0 con 0x563c967269c0 2024-05-21T21:58:20.998 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.994+0000 7f67b30c7b80 10 client.0 _ll_register_callbacks cb 0x563c96618cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:58:20.999 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.994+0000 7f67a6800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c96720b50 0x563c9672c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:20.999 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.994+0000 7f67a6800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c96720b50 0x563c9672c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:37804/0 (socket says 172.21.15.151:37804) 2024-05-21T21:58:20.999 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.994+0000 7f67a6800700 1 -- 172.21.15.151:0/1256719374 learned_addr learned my addr 172.21.15.151:0/1256719374 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:58:20.999 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.994+0000 7f67a6800700 1 --2- 172.21.15.151:0/1256719374 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c967269c0 0x563c96729f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:20.999 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.994+0000 7f67a5400700 1 --2- 172.21.15.151:0/1256719374 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563c967260a0 0x563c96729a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:20.999 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.995+0000 7f67a6800700 1 -- 172.21.15.151:0/1256719374 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c967269c0 msgr2=0x563c96729f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:20.999 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.995+0000 7f67a6800700 1 --2- 172.21.15.151:0/1256719374 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c967269c0 0x563c96729f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:20.999 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.995+0000 7f67a6800700 1 -- 172.21.15.151:0/1256719374 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563c967260a0 msgr2=0x563c96729a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:20.999 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.995+0000 7f67a6800700 1 --2- 172.21.15.151:0/1256719374 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563c967260a0 0x563c96729a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:20.999 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.995+0000 7f67a6800700 1 -- 172.21.15.151:0/1256719374 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c965be950 con 0x563c96720b50 2024-05-21T21:58:20.999 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.995+0000 7f67a6800700 1 --2- 172.21.15.151:0/1256719374 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c96720b50 0x563c9672c670 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f679c000f80 tx=0x7f679c009e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a1c10a3e8a117d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:58:20.999 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.995+0000 7f6783e00700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-21T21:58:20.999 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.995+0000 7f6783e00700 1 -- 172.21.15.151:0/1256719374 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f679c013cc0 con 0x563c96720b50 2024-05-21T21:58:20.999 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.995+0000 7f6783e00700 1 -- 172.21.15.151:0/1256719374 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f679c013e60 con 0x563c96720b50 2024-05-21T21:58:21.000 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.995+0000 7f67b30c7b80 1 -- 172.21.15.151:0/1256719374 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f679c015440 con 0x563c96720b50 2024-05-21T21:58:21.000 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.996+0000 7f6783e00700 1 -- 172.21.15.151:0/1256719374 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f679c022470 con 0x563c96720b50 2024-05-21T21:58:21.000 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.996+0000 7f67b30c7b80 10 client.25901 fetch_fsmap learned FSMap version 29 2024-05-21T21:58:21.000 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.996+0000 7f67b30c7b80 1 -- 172.21.15.151:0/1256719374 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563c9663f0e0 con 0x563c96720b50 2024-05-21T21:58:21.000 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.996+0000 7f6783e00700 1 -- 172.21.15.151:0/1256719374 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f679c015440 con 0x563c96720b50 2024-05-21T21:58:21.000 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.996+0000 7f6783e00700 1 -- 172.21.15.151:0/1256719374 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f679c01f7d0 con 0x563c96720b50 2024-05-21T21:58:21.000 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.996+0000 7f67b30c7b80 10 client.25901 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T21:58:21.000 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.996+0000 7f67b30c7b80 10 client.25901 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T21:58:21.000 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.996+0000 7f67b30c7b80 20 client.25901 populate_metadata read hostname 'smithi151' 2024-05-21T21:58:21.000 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.996+0000 7f67b30c7b80 1 --2- 172.21.15.151:0/1256719374 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x563c96739f60 0x563c9673c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:21.000 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.996+0000 7f67b30c7b80 4 client.25901 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T21:58:21.000 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.996+0000 7f67b30c7b80 1 -- 172.21.15.151:0/1256719374 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563c96680e10 con 0x563c96739f60 2024-05-21T21:58:21.000 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.997+0000 7f67a5400700 1 --2- 172.21.15.151:0/1256719374 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x563c96739f60 0x563c9673c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:21.002 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.997+0000 7f67a5400700 1 --2- 172.21.15.151:0/1256719374 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x563c96739f60 0x563c9673c350 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad4fc80aec703325 server_cookie=e6c7fdf50cc786ae in_seq=0 out_seq=0 2024-05-21T21:58:21.002 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.997+0000 7f6783e00700 10 client.25901 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T21:58:21.002 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.998+0000 7f6783e00700 1 -- 172.21.15.151:0/1256719374 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x563c96680e10 con 0x563c96739f60 2024-05-21T21:58:21.002 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.998+0000 7f6783e00700 10 client.25901 handle_command_reply: tid=0 2024-05-21T21:58:21.002 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.998+0000 7f67b30c7b80 1 client.25901 shutdown 2024-05-21T21:58:21.002 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.998+0000 7f67b30c7b80 1 -- 172.21.15.151:0/1256719374 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c96720b50 msgr2=0x563c9672c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:21.002 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.998+0000 7f67b30c7b80 1 --2- 172.21.15.151:0/1256719374 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c96720b50 0x563c9672c670 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f679c000f80 tx=0x7f679c009e70 comp rx=0 tx=0).stop 2024-05-21T21:58:21.002 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.998+0000 7f67b30c7b80 1 -- 172.21.15.151:0/1256719374 shutdown_connections 2024-05-21T21:58:21.002 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.998+0000 7f67b30c7b80 1 --2- 172.21.15.151:0/1256719374 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x563c96739f60 0x563c9673c350 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:21.002 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.999+0000 7f67b30c7b80 1 --2- 172.21.15.151:0/1256719374 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c96720b50 0x563c9672c670 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:21.002 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.999+0000 7f6783e00700 0 client.25901 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T21:58:21.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.999+0000 7f67b30c7b80 1 --2- 172.21.15.151:0/1256719374 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c967269c0 0x563c96729f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:21.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.999+0000 7f67b30c7b80 1 --2- 172.21.15.151:0/1256719374 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563c967260a0 0x563c96729a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:21.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.999+0000 7f67b30c7b80 1 -- 172.21.15.151:0/1256719374 >> 172.21.15.151:0/1256719374 conn(0x563c9664ec60 msgr2=0x563c9671f520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:58:21.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.999+0000 7f67b30c7b80 1 -- 172.21.15.151:0/1256719374 shutdown_connections 2024-05-21T21:58:21.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.999+0000 7f67b30c7b80 1 -- 172.21.15.151:0/1256719374 wait complete. 2024-05-21T21:58:21.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:20.999+0000 7f67b30c7b80 20 client.25901 trim_cache size 0 max 16384 2024-05-21T21:58:21.041 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.039+0000 7f67b30c7b80 1 Processor -- start 2024-05-21T21:58:21.041 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.039+0000 7f67b30c7b80 1 -- start start 2024-05-21T21:58:21.042 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.040+0000 7f67b30c7b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c96720b50 0x563c96728300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:21.042 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.040+0000 7f67b30c7b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c967260a0 0x563c96728840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:21.042 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.040+0000 7f67b30c7b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563c967269c0 0x563c9671acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:21.042 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.040+0000 7f67b30c7b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f678400d580 con 0x563c967269c0 2024-05-21T21:58:21.042 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.040+0000 7f67b30c7b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f678400f0c0 con 0x563c96720b50 2024-05-21T21:58:21.042 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.040+0000 7f67b30c7b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f678400dc50 con 0x563c967260a0 2024-05-21T21:58:21.042 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.040+0000 7f67a6800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c96720b50 0x563c96728300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:21.042 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.040+0000 7f67a6800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c96720b50 0x563c96728300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:37810/0 (socket says 172.21.15.151:37810) 2024-05-21T21:58:21.042 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.040+0000 7f67a6800700 1 -- 172.21.15.151:0/48954497 learned_addr learned my addr 172.21.15.151:0/48954497 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:58:21.042 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.041+0000 7f67a6800700 1 --2- 172.21.15.151:0/48954497 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563c967269c0 0x563c9671acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:21.042 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.041+0000 7f67a5400700 1 --2- 172.21.15.151:0/48954497 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c967260a0 0x563c96728840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:21.043 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.041+0000 7f67a6800700 1 -- 172.21.15.151:0/48954497 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c967260a0 msgr2=0x563c96728840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:21.043 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.041+0000 7f67a6800700 1 --2- 172.21.15.151:0/48954497 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c967260a0 0x563c96728840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:21.043 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.041+0000 7f67a6800700 1 -- 172.21.15.151:0/48954497 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563c967269c0 msgr2=0x563c9671acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:21.043 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.041+0000 7f67a6800700 1 --2- 172.21.15.151:0/48954497 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563c967269c0 0x563c9671acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:21.044 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.041+0000 7f67a6800700 1 -- 172.21.15.151:0/48954497 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c96727f80 con 0x563c96720b50 2024-05-21T21:58:21.044 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.041+0000 7f67a5400700 1 --2- 172.21.15.151:0/48954497 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c967260a0 0x563c96728840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:58:21.044 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.041+0000 7f67a6800700 1 --2- 172.21.15.151:0/48954497 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c96720b50 0x563c96728300 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f679c0095b0 tx=0x7f679c013610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33ab9f17c205fd3d server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:58:21.044 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.041+0000 7f678ea00700 1 -- 172.21.15.151:0/48954497 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f679c01fe60 con 0x563c96720b50 2024-05-21T21:58:21.044 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.042+0000 7f678ea00700 1 -- 172.21.15.151:0/48954497 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f679c015440 con 0x563c96720b50 2024-05-21T21:58:21.044 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.042+0000 7f678ea00700 1 -- 172.21.15.151:0/48954497 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f679c01f850 con 0x563c96720b50 2024-05-21T21:58:21.045 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.043+0000 7f67b30c7b80 1 -- 172.21.15.151:0/48954497 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c96720b50 msgr2=0x563c96728300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:21.045 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.043+0000 7f67b30c7b80 1 --2- 172.21.15.151:0/48954497 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c96720b50 0x563c96728300 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f679c0095b0 tx=0x7f679c013610 comp rx=0 tx=0).stop 2024-05-21T21:58:21.045 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.043+0000 7f67b30c7b80 1 -- 172.21.15.151:0/48954497 shutdown_connections 2024-05-21T21:58:21.045 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.043+0000 7f67b30c7b80 1 --2- 172.21.15.151:0/48954497 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c96720b50 0x563c96728300 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:21.045 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.043+0000 7f67b30c7b80 1 --2- 172.21.15.151:0/48954497 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563c967269c0 0x563c9671acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:21.045 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.043+0000 7f67b30c7b80 1 --2- 172.21.15.151:0/48954497 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c967260a0 0x563c96728840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:21.046 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.043+0000 7f67b30c7b80 1 -- 172.21.15.151:0/48954497 >> 172.21.15.151:0/48954497 conn(0x563c9664ec60 msgr2=0x563c96723bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:58:21.046 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.043+0000 7f67b30c7b80 1 -- 172.21.15.151:0/48954497 shutdown_connections 2024-05-21T21:58:21.046 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.043+0000 7f67b30c7b80 1 -- 172.21.15.151:0/48954497 wait complete. 2024-05-21T21:58:21.046 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.044+0000 7f67b30c7b80 1 -- start start 2024-05-21T21:58:21.046 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.044+0000 7f67b30c7b80 1 Processor -- start 2024-05-21T21:58:21.047 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.045+0000 7f67b30c7b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563c96720b50 0x563c96732d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:21.047 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.045+0000 7f67b30c7b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c967260a0 0x563c9672f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:21.047 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.045+0000 7f67b30c7b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c967269c0 0x563c9672fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:21.047 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.045+0000 7f67b30c7b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f678400e9f0 con 0x563c96720b50 2024-05-21T21:58:21.047 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.045+0000 7f67b30c7b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f678400e320 con 0x563c967260a0 2024-05-21T21:58:21.047 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.045+0000 7f67b30c7b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f678400b520 con 0x563c967269c0 2024-05-21T21:58:21.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.045+0000 7f67b30c7b80 10 client.0 _ll_register_callbacks cb 0x563c96618cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:58:21.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.045+0000 7f67a5400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c967260a0 0x563c9672f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:21.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.045+0000 7f67a5400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c967260a0 0x563c9672f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:37822/0 (socket says 172.21.15.151:37822) 2024-05-21T21:58:21.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.045+0000 7f67a5400700 1 -- 172.21.15.151:0/1408129947 learned_addr learned my addr 172.21.15.151:0/1408129947 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:58:21.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.045+0000 7f67a6800700 1 --2- 172.21.15.151:0/1408129947 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563c96720b50 0x563c96732d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:21.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.046+0000 7f67a5400700 1 -- 172.21.15.151:0/1408129947 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c967269c0 msgr2=0x563c9672fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:21.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.046+0000 7f67a6800700 1 --2- 172.21.15.151:0/1408129947 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c967269c0 0x563c9672fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:21.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.046+0000 7f67a5400700 1 --2- 172.21.15.151:0/1408129947 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c967269c0 0x563c9672fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:21.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.046+0000 7f67a5400700 1 -- 172.21.15.151:0/1408129947 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563c96720b50 msgr2=0x563c96732d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:21.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.046+0000 7f67a5400700 1 --2- 172.21.15.151:0/1408129947 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563c96720b50 0x563c96732d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:21.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.046+0000 7f67a5400700 1 -- 172.21.15.151:0/1408129947 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c967303b0 con 0x563c967260a0 2024-05-21T21:58:21.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.046+0000 7f67a6800700 1 --2- 172.21.15.151:0/1408129947 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c967269c0 0x563c9672fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T21:58:21.049 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.046+0000 7f67a5400700 1 --2- 172.21.15.151:0/1408129947 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c967260a0 0x563c9672f7d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f67a0105410 tx=0x7f6790009600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98e8d3cc57b14701 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:58:21.049 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.046+0000 7f6783e00700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-21T21:58:21.050 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.046+0000 7f6783e00700 1 -- 172.21.15.151:0/1408129947 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6790005600 con 0x563c967260a0 2024-05-21T21:58:21.050 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.046+0000 7f6783e00700 1 -- 172.21.15.151:0/1408129947 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f6790003680 con 0x563c967260a0 2024-05-21T21:58:21.050 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.046+0000 7f67b30c7b80 1 -- 172.21.15.151:0/1408129947 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f679c015440 con 0x563c967260a0 2024-05-21T21:58:21.050 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.047+0000 7f6783e00700 1 -- 172.21.15.151:0/1408129947 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6790003880 con 0x563c967260a0 2024-05-21T21:58:21.050 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.047+0000 7f67b30c7b80 10 client.25907 fetch_fsmap learned FSMap version 29 2024-05-21T21:58:21.050 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.047+0000 7f67b30c7b80 1 -- 172.21.15.151:0/1408129947 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563c967adf40 con 0x563c967260a0 2024-05-21T21:58:21.050 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.047+0000 7f6783e00700 1 -- 172.21.15.151:0/1408129947 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f679c015440 con 0x563c967260a0 2024-05-21T21:58:21.050 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.047+0000 7f6783e00700 1 -- 172.21.15.151:0/1408129947 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f679002d020 con 0x563c967260a0 2024-05-21T21:58:21.050 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.047+0000 7f67b30c7b80 10 client.25907 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T21:58:21.051 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.047+0000 7f67b30c7b80 10 client.25907 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T21:58:21.051 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.047+0000 7f67b30c7b80 20 client.25907 populate_metadata read hostname 'smithi151' 2024-05-21T21:58:21.051 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.047+0000 7f67b30c7b80 1 --2- 172.21.15.151:0/1408129947 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x563c96739f60 0x563c967b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:21.051 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.048+0000 7f67b30c7b80 4 client.25907 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-21T21:58:21.051 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.048+0000 7f67b30c7b80 1 -- 172.21.15.151:0/1408129947 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563c9667a5f0 con 0x563c96739f60 2024-05-21T21:58:21.051 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.048+0000 7f67a6800700 1 --2- 172.21.15.151:0/1408129947 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x563c96739f60 0x563c967b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:21.051 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.048+0000 7f67a6800700 1 --2- 172.21.15.151:0/1408129947 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x563c96739f60 0x563c967b06c0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8575e4ade81b3717 server_cookie=ea9d3f5dc8e4ec51 in_seq=0 out_seq=0 2024-05-21T21:58:21.051 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.048+0000 7f6783e00700 10 client.25907 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T21:58:21.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.049+0000 7f6783e00700 1 -- 172.21.15.151:0/1408129947 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563c9667a5f0 con 0x563c96739f60 2024-05-21T21:58:21.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.049+0000 7f6783e00700 10 client.25907 handle_command_reply: tid=0 2024-05-21T21:58:21.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.050+0000 7f67b30c7b80 1 client.25907 shutdown 2024-05-21T21:58:21.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.050+0000 7f67b30c7b80 1 -- 172.21.15.151:0/1408129947 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c967260a0 msgr2=0x563c9672f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:21.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.050+0000 7f67b30c7b80 1 --2- 172.21.15.151:0/1408129947 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c967260a0 0x563c9672f7d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f67a0105410 tx=0x7f6790009600 comp rx=0 tx=0).stop 2024-05-21T21:58:21.053 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.050+0000 7f67b30c7b80 1 -- 172.21.15.151:0/1408129947 shutdown_connections 2024-05-21T21:58:21.053 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.050+0000 7f67b30c7b80 1 --2- 172.21.15.151:0/1408129947 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563c967269c0 0x563c9672fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:21.053 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.050+0000 7f67b30c7b80 1 --2- 172.21.15.151:0/1408129947 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563c96720b50 0x563c96732d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:21.053 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.050+0000 7f67b30c7b80 1 --2- 172.21.15.151:0/1408129947 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x563c96739f60 0x563c967b06c0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:21.053 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.050+0000 7f67b30c7b80 1 --2- 172.21.15.151:0/1408129947 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563c967260a0 0x563c9672f7d0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:21.053 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.050+0000 7f67b30c7b80 1 -- 172.21.15.151:0/1408129947 >> 172.21.15.151:0/1408129947 conn(0x563c9664ec60 msgr2=0x563c9671acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:58:21.053 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.050+0000 7f6783e00700 0 client.25907 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T21:58:21.053 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.050+0000 7f67b30c7b80 1 -- 172.21.15.151:0/1408129947 shutdown_connections 2024-05-21T21:58:21.053 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.050+0000 7f67b30c7b80 1 -- 172.21.15.151:0/1408129947 wait complete. 2024-05-21T21:58:21.053 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.050+0000 7f67b30c7b80 20 client.25907 trim_cache size 0 max 16384 2024-05-21T21:58:21.054 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-05-21T21:58:21.054 INFO:teuthology.orchestra.run.smithi151.stdout: "return_code": 0, 2024-05-21T21:58:21.054 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_tag": "56b09f29-1b13-4c54-a1ed-af125566b2e0", 2024-05-21T21:58:21.054 INFO:teuthology.orchestra.run.smithi151.stdout: "mode": "asynchronous" 2024-05-21T21:58:21.054 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-05-21T21:58:21.055 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.051+0000 7f67a7200700 1 -- 172.21.15.151:0/2161366302 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f67880b2a70 msgr2=0x7f67880b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:21.055 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.051+0000 7f67a7200700 1 --2- 172.21.15.151:0/2161366302 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f67880b2a70 0x7f67880b4f30 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f67940dffd0 tx=0x7f67940e8550 comp rx=0 tx=0).stop 2024-05-21T21:58:21.055 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.051+0000 7f67a7200700 1 -- 172.21.15.151:0/2161366302 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f67a0104690 msgr2=0x7f67a019be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:21.055 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.051+0000 7f67a7200700 1 --2- 172.21.15.151:0/2161366302 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f67a0104690 0x7f67a019be20 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f679400e9a0 tx=0x7f679400ecb0 comp rx=0 tx=0).stop 2024-05-21T21:58:21.055 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.051+0000 7f67a7200700 1 -- 172.21.15.151:0/2161366302 shutdown_connections 2024-05-21T21:58:21.055 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.051+0000 7f67a7200700 1 --2- 172.21.15.151:0/2161366302 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f67880b2a70 0x7f67880b4f30 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:21.055 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.051+0000 7f67a7200700 1 --2- 172.21.15.151:0/2161366302 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f67a010da70 0x7f67a01a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:21.055 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.051+0000 7f67a7200700 1 --2- 172.21.15.151:0/2161366302 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f67a0104690 0x7f67a019be20 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:21.055 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.051+0000 7f67a7200700 1 --2- 172.21.15.151:0/2161366302 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f67a0104fb0 0x7f67a019c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:21.055 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.051+0000 7f67a7200700 1 -- 172.21.15.151:0/2161366302 >> 172.21.15.151:0/2161366302 conn(0x7f67a00fde60 msgr2=0x7f67a00fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:58:21.055 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.052+0000 7f67a7200700 1 -- 172.21.15.151:0/2161366302 shutdown_connections 2024-05-21T21:58:21.055 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:21.052+0000 7f67a7200700 1 -- 172.21.15.151:0/2161366302 wait complete. 2024-05-21T21:58:21.333 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:58:21.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:58:21 smithi151 ceph-mon[31623]: pgmap v85: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-05-21T21:58:21.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:58:21 smithi151 ceph-mon[31623]: scrub queued for path: / 2024-05-21T21:58:21.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:58:21 smithi151 ceph-mon[31623]: scrub summary: idle+waiting paths [/] 2024-05-21T21:58:21.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:58:21 smithi151 ceph-mon[31623]: scrub summary: active paths [/] 2024-05-21T21:58:21.735 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:58:21 smithi184 ceph-mon[36942]: pgmap v85: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-05-21T21:58:21.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:58:21 smithi184 ceph-mon[36942]: scrub queued for path: / 2024-05-21T21:58:21.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:58:21 smithi184 ceph-mon[36942]: scrub summary: idle+waiting paths [/] 2024-05-21T21:58:21.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:58:21 smithi184 ceph-mon[36942]: scrub summary: active paths [/] 2024-05-21T21:58:21.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:58:21 smithi164 ceph-mon[36269]: pgmap v85: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-05-21T21:58:21.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:58:21 smithi164 ceph-mon[36269]: scrub queued for path: / 2024-05-21T21:58:21.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:58:21 smithi164 ceph-mon[36269]: scrub summary: idle+waiting paths [/] 2024-05-21T21:58:21.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:58:21 smithi164 ceph-mon[36269]: scrub summary: active paths [/] 2024-05-21T21:58:21.945 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-21T21:58:22.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:58:22 smithi151 ceph-mon[31623]: scrub summary: idle+waiting paths [/] 2024-05-21T21:58:22.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:58:22 smithi184 ceph-mon[36942]: scrub summary: idle+waiting paths [/] 2024-05-21T21:58:22.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:58:22 smithi164 ceph-mon[36269]: scrub summary: idle+waiting paths [/] 2024-05-21T21:58:23.247 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:58:23.637 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:58:23 smithi184 ceph-mon[36942]: pgmap v86: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 85 B/s wr, 12 op/s 2024-05-21T21:58:23.669 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:58:23 smithi151 ceph-mon[31623]: pgmap v86: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 85 B/s wr, 12 op/s 2024-05-21T21:58:23.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:58:23 smithi164 ceph-mon[36269]: pgmap v86: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 85 B/s wr, 12 op/s 2024-05-21T21:58:24.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.218+0000 7f3183200700 1 -- 172.21.15.151:0/2588883235 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3184104740 msgr2=0x7f3184104b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:24.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.218+0000 7f3183200700 1 --2- 172.21.15.151:0/2588883235 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3184104740 0x7f3184104b20 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f317c009f90 tx=0x7f317c00b330 comp rx=0 tx=0).stop 2024-05-21T21:58:24.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.218+0000 7f3183200700 1 -- 172.21.15.151:0/2588883235 shutdown_connections 2024-05-21T21:58:24.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.218+0000 7f3183200700 1 --2- 172.21.15.151:0/2588883235 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f318410db20 0x7f318410ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:24.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.218+0000 7f3183200700 1 --2- 172.21.15.151:0/2588883235 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3184104740 0x7f3184104b20 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:24.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.218+0000 7f3183200700 1 --2- 172.21.15.151:0/2588883235 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3184105060 0x7f318410d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:24.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.218+0000 7f3183200700 1 -- 172.21.15.151:0/2588883235 >> 172.21.15.151:0/2588883235 conn(0x7f31840fded0 msgr2=0x7f31841002f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:58:24.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.218+0000 7f3183200700 1 -- 172.21.15.151:0/2588883235 shutdown_connections 2024-05-21T21:58:24.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.218+0000 7f3183200700 1 -- 172.21.15.151:0/2588883235 wait complete. 2024-05-21T21:58:24.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.220+0000 7f3183200700 1 Processor -- start 2024-05-21T21:58:24.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.220+0000 7f3183200700 1 -- start start 2024-05-21T21:58:24.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.220+0000 7f3183200700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3184104740 0x7f318419be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:24.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.220+0000 7f3183200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3184105060 0x7f318419c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:24.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.220+0000 7f3183200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f318410db20 0x7f31841a0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:24.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.220+0000 7f3183200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f31841136d0 con 0x7f318410db20 2024-05-21T21:58:24.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.220+0000 7f3183200700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3184113550 con 0x7f3184105060 2024-05-21T21:58:24.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.220+0000 7f3183200700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f3184113850 con 0x7f3184104740 2024-05-21T21:58:24.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.220+0000 7f3181400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3184105060 0x7f318419c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:24.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.221+0000 7f3181400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3184105060 0x7f318419c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:37834/0 (socket says 172.21.15.151:37834) 2024-05-21T21:58:24.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.221+0000 7f3181400700 1 -- 172.21.15.151:0/3378883178 learned_addr learned my addr 172.21.15.151:0/3378883178 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:58:24.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.221+0000 7f3181e00700 1 --2- 172.21.15.151:0/3378883178 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3184104740 0x7f318419be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:24.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.221+0000 7f3182800700 1 --2- 172.21.15.151:0/3378883178 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f318410db20 0x7f31841a0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:24.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.221+0000 7f3181400700 1 -- 172.21.15.151:0/3378883178 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3184104740 msgr2=0x7f318419be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:24.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.221+0000 7f3181400700 1 --2- 172.21.15.151:0/3378883178 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3184104740 0x7f318419be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:24.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.221+0000 7f3181400700 1 -- 172.21.15.151:0/3378883178 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f318410db20 msgr2=0x7f31841a0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:24.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.221+0000 7f3181400700 1 --2- 172.21.15.151:0/3378883178 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f318410db20 0x7f31841a0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:24.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.221+0000 7f3181400700 1 -- 172.21.15.151:0/3378883178 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31841a0e50 con 0x7f3184105060 2024-05-21T21:58:24.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.221+0000 7f3181400700 1 --2- 172.21.15.151:0/3378883178 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3184105060 0x7f318419c380 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f317000e9c0 tx=0x7f317000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:58:24.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.222+0000 7f317aa00700 1 -- 172.21.15.151:0/3378883178 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3170004a30 con 0x7f3184105060 2024-05-21T21:58:24.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.222+0000 7f3183200700 1 -- 172.21.15.151:0/3378883178 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31841a1140 con 0x7f3184105060 2024-05-21T21:58:24.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.222+0000 7f3183200700 1 -- 172.21.15.151:0/3378883178 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31841a89c0 con 0x7f3184105060 2024-05-21T21:58:24.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.222+0000 7f317aa00700 1 -- 172.21.15.151:0/3378883178 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f317000cc70 con 0x7f3184105060 2024-05-21T21:58:24.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.222+0000 7f317aa00700 1 -- 172.21.15.151:0/3378883178 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3170005540 con 0x7f3184105060 2024-05-21T21:58:24.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.226+0000 7f317aa00700 1 -- 172.21.15.151:0/3378883178 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7f3170004bd0 con 0x7f3184105060 2024-05-21T21:58:24.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.226+0000 7f31938ebb80 1 Processor -- start 2024-05-21T21:58:24.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.227+0000 7f31938ebb80 1 -- start start 2024-05-21T21:58:24.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.227+0000 7f31938ebb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56544594bf40 0x56544594c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:24.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.227+0000 7f31938ebb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565445946220 0x565445946680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:24.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.227+0000 7f31938ebb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565445946bc0 0x565445952c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:24.229 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.227+0000 7f31938ebb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5654458a3f40 con 0x56544594bf40 2024-05-21T21:58:24.229 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.227+0000 7f31938ebb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x565445893ae0 con 0x565445946220 2024-05-21T21:58:24.229 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.227+0000 7f31938ebb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x565445897b20 con 0x565445946bc0 2024-05-21T21:58:24.229 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.227+0000 7f3182800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56544594bf40 0x56544594c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:24.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.228+0000 7f3182800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56544594bf40 0x56544594c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52468/0 (socket says 172.21.15.151:52468) 2024-05-21T21:58:24.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.228+0000 7f3182800700 1 -- 172.21.15.151:0/664109467 learned_addr learned my addr 172.21.15.151:0/664109467 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:58:24.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.228+0000 7f3181e00700 1 --2- 172.21.15.151:0/664109467 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565445946220 0x565445946680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:24.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.228+0000 7f3181400700 1 --2- 172.21.15.151:0/664109467 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565445946bc0 0x565445952c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:24.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.228+0000 7f3181e00700 1 -- 172.21.15.151:0/664109467 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565445946bc0 msgr2=0x565445952c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:24.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.228+0000 7f3181e00700 1 --2- 172.21.15.151:0/664109467 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565445946bc0 0x565445952c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:24.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.228+0000 7f3181e00700 1 -- 172.21.15.151:0/664109467 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56544594bf40 msgr2=0x56544594c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:24.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.228+0000 7f3181e00700 1 --2- 172.21.15.151:0/664109467 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56544594bf40 0x56544594c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:24.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.228+0000 7f3181e00700 1 -- 172.21.15.151:0/664109467 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56544585c520 con 0x565445946220 2024-05-21T21:58:24.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.228+0000 7f317aa00700 1 --2- 172.21.15.151:0/3378883178 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f31680b2a70 0x7f31680b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:24.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.228+0000 7f317aa00700 1 -- 172.21.15.151:0/3378883178 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f31700e2650 con 0x7f3184105060 2024-05-21T21:58:24.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.228+0000 7f3181e00700 1 --2- 172.21.15.151:0/664109467 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565445946220 0x565445946680 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f317c00b6c0 tx=0x7f317c0092e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c8ad3aa0a72f522 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:58:24.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.229+0000 7f3182800700 1 --2- 172.21.15.151:0/3378883178 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f31680b2a70 0x7f31680b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:24.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.229+0000 7f316f400700 1 -- 172.21.15.151:0/664109467 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f317c020070 con 0x565445946220 2024-05-21T21:58:24.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.229+0000 7f316f400700 1 -- 172.21.15.151:0/664109467 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f317c003cc0 con 0x565445946220 2024-05-21T21:58:24.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.229+0000 7f316f400700 1 -- 172.21.15.151:0/664109467 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f317c007b30 con 0x565445946220 2024-05-21T21:58:24.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.230+0000 7f31938ebb80 1 -- 172.21.15.151:0/664109467 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565445946220 msgr2=0x565445946680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:24.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.230+0000 7f3182800700 1 --2- 172.21.15.151:0/3378883178 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f31680b2a70 0x7f31680b4f30 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3174009f30 tx=0x7f3174009260 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:58:24.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.230+0000 7f31938ebb80 1 --2- 172.21.15.151:0/664109467 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565445946220 0x565445946680 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f317c00b6c0 tx=0x7f317c0092e0 comp rx=0 tx=0).stop 2024-05-21T21:58:24.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.230+0000 7f31938ebb80 1 -- 172.21.15.151:0/664109467 shutdown_connections 2024-05-21T21:58:24.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.230+0000 7f31938ebb80 1 --2- 172.21.15.151:0/664109467 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565445946bc0 0x565445952c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:24.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.230+0000 7f31938ebb80 1 --2- 172.21.15.151:0/664109467 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56544594bf40 0x56544594c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:24.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.230+0000 7f31938ebb80 1 --2- 172.21.15.151:0/664109467 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565445946220 0x565445946680 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:24.233 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.230+0000 7f31938ebb80 1 -- 172.21.15.151:0/664109467 >> 172.21.15.151:0/664109467 conn(0x565445873ce0 msgr2=0x565445940e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:58:24.233 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.230+0000 7f31938ebb80 1 -- 172.21.15.151:0/664109467 shutdown_connections 2024-05-21T21:58:24.233 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.230+0000 7f31938ebb80 1 -- 172.21.15.151:0/664109467 wait complete. 2024-05-21T21:58:24.233 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.231+0000 7f31938ebb80 1 -- start start 2024-05-21T21:58:24.233 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.231+0000 7f31938ebb80 1 Processor -- start 2024-05-21T21:58:24.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.231+0000 7f31938ebb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565445946220 0x565445952820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:24.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.231+0000 7f31938ebb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x565445946bc0 0x56544594fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:24.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.231+0000 7f31938ebb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56544594bf40 0x565445950100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:24.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.231+0000 7f31938ebb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56544579a0a0 con 0x565445946bc0 2024-05-21T21:58:24.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.231+0000 7f31938ebb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x565445834860 con 0x56544594bf40 2024-05-21T21:58:24.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.231+0000 7f31938ebb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5654459531d0 con 0x565445946220 2024-05-21T21:58:24.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.231+0000 7f31938ebb80 10 client.0 _ll_register_callbacks cb 0x56544583dd70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:58:24.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.232+0000 7f3181400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x565445946bc0 0x56544594fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:24.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.232+0000 7f3181e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565445946220 0x565445952820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:24.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.232+0000 7f3182800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56544594bf40 0x565445950100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:24.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.232+0000 7f3181400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x565445946bc0 0x56544594fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52474/0 (socket says 172.21.15.151:52474) 2024-05-21T21:58:24.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.232+0000 7f3181400700 1 -- 172.21.15.151:0/444168943 learned_addr learned my addr 172.21.15.151:0/444168943 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:58:24.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.232+0000 7f3181400700 1 -- 172.21.15.151:0/444168943 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565445946220 msgr2=0x565445952820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:24.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.232+0000 7f3181400700 1 --2- 172.21.15.151:0/444168943 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565445946220 0x565445952820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:24.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.232+0000 7f3181400700 1 -- 172.21.15.151:0/444168943 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56544594bf40 msgr2=0x565445950100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:24.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.232+0000 7f3181400700 1 --2- 172.21.15.151:0/444168943 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56544594bf40 0x565445950100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:24.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.232+0000 7f3181400700 1 -- 172.21.15.151:0/444168943 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654457e39d0 con 0x565445946bc0 2024-05-21T21:58:24.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.232+0000 7f3182800700 1 --2- 172.21.15.151:0/444168943 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56544594bf40 0x565445950100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:58:24.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.232+0000 7f3181400700 1 --2- 172.21.15.151:0/444168943 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x565445946bc0 0x56544594fbc0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f31700095c0 tx=0x7f317000cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6480dba57da6ddad server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:58:24.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.232+0000 7f3163e00700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-21T21:58:24.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.233+0000 7f3163e00700 1 -- 172.21.15.151:0/444168943 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31700a9050 con 0x565445946bc0 2024-05-21T21:58:24.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.233+0000 7f3163e00700 1 -- 172.21.15.151:0/444168943 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f31700056e0 con 0x565445946bc0 2024-05-21T21:58:24.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.233+0000 7f31938ebb80 1 -- 172.21.15.151:0/444168943 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f317c003cc0 con 0x565445946bc0 2024-05-21T21:58:24.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.233+0000 7f3163e00700 1 -- 172.21.15.151:0/444168943 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f31700b2080 con 0x565445946bc0 2024-05-21T21:58:24.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.233+0000 7f31938ebb80 10 client.15792 fetch_fsmap learned FSMap version 29 2024-05-21T21:58:24.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.233+0000 7f31938ebb80 1 -- 172.21.15.151:0/444168943 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x565445864160 con 0x565445946bc0 2024-05-21T21:58:24.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.233+0000 7f3163e00700 1 -- 172.21.15.151:0/444168943 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f317c003cc0 con 0x565445946bc0 2024-05-21T21:58:24.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.234+0000 7f3163e00700 1 -- 172.21.15.151:0/444168943 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f31700b22b0 con 0x565445946bc0 2024-05-21T21:58:24.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.234+0000 7f31938ebb80 10 client.15792 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T21:58:24.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.234+0000 7f31938ebb80 10 client.15792 resolve_mds: validated gid 24530 aka daemon mds.b 2024-05-21T21:58:24.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.234+0000 7f31938ebb80 20 client.15792 populate_metadata read hostname 'smithi151' 2024-05-21T21:58:24.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.234+0000 7f31938ebb80 1 --2- 172.21.15.151:0/444168943 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x56544595fb90 0x565445961fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:24.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.234+0000 7f31938ebb80 4 client.15792 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T21:58:24.238 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.234+0000 7f31938ebb80 1 -- 172.21.15.151:0/444168943 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5654458a5e90 con 0x56544595fb90 2024-05-21T21:58:24.238 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.234+0000 7f3181e00700 1 --2- 172.21.15.151:0/444168943 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x56544595fb90 0x565445961fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:24.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.235+0000 7f3181e00700 1 --2- 172.21.15.151:0/444168943 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x56544595fb90 0x565445961fe0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1409b5f642ec12f6 server_cookie=26ecd816f30ae6e0 in_seq=0 out_seq=0 2024-05-21T21:58:24.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.235+0000 7f3163e00700 10 client.15792 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T21:58:24.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.235+0000 7f3163e00700 1 -- 172.21.15.151:0/444168943 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5654458a5e90 con 0x56544595fb90 2024-05-21T21:58:24.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.236+0000 7f3163e00700 10 client.15792 handle_command_reply: tid=0 2024-05-21T21:58:24.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.236+0000 7f31938ebb80 1 client.15792 shutdown 2024-05-21T21:58:24.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.236+0000 7f31938ebb80 1 -- 172.21.15.151:0/444168943 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x565445946bc0 msgr2=0x56544594fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:24.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.236+0000 7f31938ebb80 1 --2- 172.21.15.151:0/444168943 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x565445946bc0 0x56544594fbc0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f31700095c0 tx=0x7f317000cb70 comp rx=0 tx=0).stop 2024-05-21T21:58:24.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.236+0000 7f31938ebb80 1 -- 172.21.15.151:0/444168943 shutdown_connections 2024-05-21T21:58:24.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.236+0000 7f31938ebb80 1 --2- 172.21.15.151:0/444168943 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x56544595fb90 0x565445961fe0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:24.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.236+0000 7f31938ebb80 1 --2- 172.21.15.151:0/444168943 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56544594bf40 0x565445950100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:24.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.236+0000 7f31938ebb80 1 --2- 172.21.15.151:0/444168943 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565445946220 0x565445952820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:24.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.236+0000 7f31938ebb80 1 --2- 172.21.15.151:0/444168943 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x565445946bc0 0x56544594fbc0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:24.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.236+0000 7f31938ebb80 1 -- 172.21.15.151:0/444168943 >> 172.21.15.151:0/444168943 conn(0x565445873ce0 msgr2=0x565445940470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:58:24.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.236+0000 7f3163e00700 0 client.15792 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T21:58:24.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.236+0000 7f31938ebb80 1 -- 172.21.15.151:0/444168943 shutdown_connections 2024-05-21T21:58:24.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.236+0000 7f31938ebb80 1 -- 172.21.15.151:0/444168943 wait complete. 2024-05-21T21:58:24.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.237+0000 7f31938ebb80 20 client.15792 trim_cache size 0 max 16384 2024-05-21T21:58:24.260 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.259+0000 7f31938ebb80 1 Processor -- start 2024-05-21T21:58:24.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.259+0000 7f31938ebb80 1 -- start start 2024-05-21T21:58:24.262 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.260+0000 7f31938ebb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565445946220 0x565445940bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:24.262 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.260+0000 7f31938ebb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565445946bc0 0x5654459511c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:24.262 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.260+0000 7f31938ebb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56544594bf40 0x565445951700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:24.262 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.260+0000 7f31938ebb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f316400d580 con 0x56544594bf40 2024-05-21T21:58:24.262 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.260+0000 7f31938ebb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f316400f0c0 con 0x565445946bc0 2024-05-21T21:58:24.262 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.260+0000 7f31938ebb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f316400dc50 con 0x565445946220 2024-05-21T21:58:24.262 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.260+0000 7f3182800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56544594bf40 0x565445951700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:24.262 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.260+0000 7f3181e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565445946220 0x565445940bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:24.262 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.260+0000 7f3181e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565445946220 0x565445940bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:33474/0 (socket says 172.21.15.151:33474) 2024-05-21T21:58:24.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.260+0000 7f3181e00700 1 -- 172.21.15.151:0/3138919212 learned_addr learned my addr 172.21.15.151:0/3138919212 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:58:24.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.260+0000 7f3182800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56544594bf40 0x565445951700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52480/0 (socket says 172.21.15.151:52480) 2024-05-21T21:58:24.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.260+0000 7f3181400700 1 --2- 172.21.15.151:0/3138919212 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565445946bc0 0x5654459511c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:24.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.261+0000 7f3181e00700 1 -- 172.21.15.151:0/3138919212 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565445946bc0 msgr2=0x5654459511c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:24.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.261+0000 7f3181e00700 1 --2- 172.21.15.151:0/3138919212 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565445946bc0 0x5654459511c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:24.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.261+0000 7f3181e00700 1 -- 172.21.15.151:0/3138919212 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56544594bf40 msgr2=0x565445951700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:24.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.261+0000 7f3181e00700 1 --2- 172.21.15.151:0/3138919212 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56544594bf40 0x565445951700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:24.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.261+0000 7f3181e00700 1 -- 172.21.15.151:0/3138919212 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56544594e130 con 0x565445946220 2024-05-21T21:58:24.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.261+0000 7f3181e00700 1 --2- 172.21.15.151:0/3138919212 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565445946220 0x565445940bf0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f317c00b6c0 tx=0x7f317c0096e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78b7b176e4eaf578 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:58:24.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.261+0000 7f316ea00700 1 -- 172.21.15.151:0/3138919212 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f317c020040 con 0x565445946220 2024-05-21T21:58:24.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.261+0000 7f316ea00700 1 -- 172.21.15.151:0/3138919212 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f317c003b80 con 0x565445946220 2024-05-21T21:58:24.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.261+0000 7f316ea00700 1 -- 172.21.15.151:0/3138919212 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f317c007d80 con 0x565445946220 2024-05-21T21:58:24.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.262+0000 7f31938ebb80 1 -- 172.21.15.151:0/3138919212 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565445946220 msgr2=0x565445940bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:24.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.262+0000 7f31938ebb80 1 --2- 172.21.15.151:0/3138919212 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565445946220 0x565445940bf0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f317c00b6c0 tx=0x7f317c0096e0 comp rx=0 tx=0).stop 2024-05-21T21:58:24.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.262+0000 7f31938ebb80 1 -- 172.21.15.151:0/3138919212 shutdown_connections 2024-05-21T21:58:24.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.262+0000 7f31938ebb80 1 --2- 172.21.15.151:0/3138919212 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56544594bf40 0x565445951700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:24.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.262+0000 7f31938ebb80 1 --2- 172.21.15.151:0/3138919212 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565445946220 0x565445940bf0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:24.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.262+0000 7f31938ebb80 1 --2- 172.21.15.151:0/3138919212 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x565445946bc0 0x5654459511c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:24.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.262+0000 7f31938ebb80 1 -- 172.21.15.151:0/3138919212 >> 172.21.15.151:0/3138919212 conn(0x565445873ce0 msgr2=0x565445945aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:58:24.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.262+0000 7f31938ebb80 1 -- 172.21.15.151:0/3138919212 shutdown_connections 2024-05-21T21:58:24.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.262+0000 7f31938ebb80 1 -- 172.21.15.151:0/3138919212 wait complete. 2024-05-21T21:58:24.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.263+0000 7f31938ebb80 1 -- start start 2024-05-21T21:58:24.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.263+0000 7f31938ebb80 1 Processor -- start 2024-05-21T21:58:24.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.263+0000 7f31938ebb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x565445946220 0x565445956bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:24.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.263+0000 7f31938ebb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565445946bc0 0x5654459570f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:24.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.263+0000 7f31938ebb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56544594bf40 0x565445967680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:24.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.263+0000 7f31938ebb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f316400e9f0 con 0x565445946220 2024-05-21T21:58:24.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.263+0000 7f31938ebb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f316400e320 con 0x56544594bf40 2024-05-21T21:58:24.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.263+0000 7f31938ebb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f316400b520 con 0x565445946bc0 2024-05-21T21:58:24.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.264+0000 7f31938ebb80 10 client.0 _ll_register_callbacks cb 0x56544583dd70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:58:24.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.264+0000 7f3181e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x565445946220 0x565445956bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:24.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.264+0000 7f3182800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56544594bf40 0x565445967680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:24.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.264+0000 7f3181400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565445946bc0 0x5654459570f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:24.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.264+0000 7f3182800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56544594bf40 0x565445967680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:37874/0 (socket says 172.21.15.151:37874) 2024-05-21T21:58:24.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.264+0000 7f3182800700 1 -- 172.21.15.151:0/1084519024 learned_addr learned my addr 172.21.15.151:0/1084519024 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:58:24.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.264+0000 7f3181400700 1 -- 172.21.15.151:0/1084519024 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56544594bf40 msgr2=0x565445967680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:24.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.264+0000 7f3181400700 1 --2- 172.21.15.151:0/1084519024 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56544594bf40 0x565445967680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:24.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.264+0000 7f3181400700 1 -- 172.21.15.151:0/1084519024 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x565445946220 msgr2=0x565445956bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:24.269 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.264+0000 7f3181400700 1 --2- 172.21.15.151:0/1084519024 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x565445946220 0x565445956bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:24.269 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.264+0000 7f3181400700 1 -- 172.21.15.151:0/1084519024 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654459579c0 con 0x565445946bc0 2024-05-21T21:58:24.269 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.264+0000 7f3181400700 1 --2- 172.21.15.151:0/1084519024 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565445946bc0 0x5654459570f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f31700a6b50 tx=0x7f31700cadc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dde93dc6000ba159 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:58:24.269 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.264+0000 7f3163e00700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-21T21:58:24.269 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.265+0000 7f3163e00700 1 -- 172.21.15.151:0/1084519024 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31700a9050 con 0x565445946bc0 2024-05-21T21:58:24.269 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.265+0000 7f3163e00700 1 -- 172.21.15.151:0/1084519024 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f31700a5ed0 con 0x565445946bc0 2024-05-21T21:58:24.269 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.265+0000 7f31938ebb80 1 -- 172.21.15.151:0/1084519024 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f317c003b80 con 0x565445946bc0 2024-05-21T21:58:24.269 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.265+0000 7f3163e00700 1 -- 172.21.15.151:0/1084519024 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f31700a93f0 con 0x565445946bc0 2024-05-21T21:58:24.269 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.265+0000 7f31938ebb80 10 client.25933 fetch_fsmap learned FSMap version 29 2024-05-21T21:58:24.269 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.265+0000 7f31938ebb80 1 -- 172.21.15.151:0/1084519024 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5654459cadf0 con 0x565445946bc0 2024-05-21T21:58:24.269 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.265+0000 7f3163e00700 1 -- 172.21.15.151:0/1084519024 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f317c003b80 con 0x565445946bc0 2024-05-21T21:58:24.269 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.265+0000 7f3163e00700 1 -- 172.21.15.151:0/1084519024 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f31700e20d0 con 0x565445946bc0 2024-05-21T21:58:24.269 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.266+0000 7f31938ebb80 10 client.25933 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T21:58:24.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.266+0000 7f31938ebb80 10 client.25933 resolve_mds: validated gid 24530 aka daemon mds.b 2024-05-21T21:58:24.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.266+0000 7f31938ebb80 20 client.25933 populate_metadata read hostname 'smithi151' 2024-05-21T21:58:24.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.266+0000 7f31938ebb80 1 --2- 172.21.15.151:0/1084519024 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5654459cb4c0 0x5654459cd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:24.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.266+0000 7f31938ebb80 4 client.25933 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "acaf991b", "release": true, "await": true}] 2024-05-21T21:58:24.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.266+0000 7f31938ebb80 1 -- 172.21.15.151:0/1084519024 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "quiesce db", "set_id": "acaf991b", "release": true, "await": true}) -- 0x56544589f670 con 0x5654459cb4c0 2024-05-21T21:58:24.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.266+0000 7f3181e00700 1 --2- 172.21.15.151:0/1084519024 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5654459cb4c0 0x5654459cd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:24.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.266+0000 7f3181e00700 1 --2- 172.21.15.151:0/1084519024 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5654459cb4c0 0x5654459cd8e0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89ebc3ac0349ecbc server_cookie=2413d57493f6f97b in_seq=0 out_seq=0 2024-05-21T21:58:24.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.266+0000 7f3163e00700 10 client.25933 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T21:58:24.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.271+0000 7f3163e00700 1 -- 172.21.15.151:0/1084519024 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x56544589f670 con 0x5654459cb4c0 2024-05-21T21:58:24.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.271+0000 7f3163e00700 10 client.25933 handle_command_reply: tid=0 2024-05-21T21:58:24.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.271+0000 7f31938ebb80 1 client.25933 shutdown 2024-05-21T21:58:24.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.271+0000 7f31938ebb80 1 -- 172.21.15.151:0/1084519024 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565445946bc0 msgr2=0x5654459570f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:24.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.271+0000 7f31938ebb80 1 --2- 172.21.15.151:0/1084519024 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565445946bc0 0x5654459570f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f31700a6b50 tx=0x7f31700cadc0 comp rx=0 tx=0).stop 2024-05-21T21:58:24.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.271+0000 7f31938ebb80 1 -- 172.21.15.151:0/1084519024 shutdown_connections 2024-05-21T21:58:24.273 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.271+0000 7f31938ebb80 1 --2- 172.21.15.151:0/1084519024 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5654459cb4c0 0x5654459cd8e0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:24.273 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.271+0000 7f31938ebb80 1 --2- 172.21.15.151:0/1084519024 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56544594bf40 0x565445967680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:24.273 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.271+0000 7f31938ebb80 1 --2- 172.21.15.151:0/1084519024 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x565445946220 0x565445956bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:24.273 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.271+0000 7f31938ebb80 1 --2- 172.21.15.151:0/1084519024 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x565445946bc0 0x5654459570f0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:24.273 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.271+0000 7f3163e00700 0 client.25933 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T21:58:24.273 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.271+0000 7f31938ebb80 1 -- 172.21.15.151:0/1084519024 >> 172.21.15.151:0/1084519024 conn(0x565445873ce0 msgr2=0x565445949810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:58:24.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.271+0000 7f31938ebb80 1 -- 172.21.15.151:0/1084519024 shutdown_connections 2024-05-21T21:58:24.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.271+0000 7f31938ebb80 1 -- 172.21.15.151:0/1084519024 wait complete. 2024-05-21T21:58:24.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.272+0000 7f31938ebb80 20 client.25933 trim_cache size 0 max 16384 2024-05-21T21:58:24.274 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-05-21T21:58:24.274 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 29, 2024-05-21T21:58:24.274 INFO:teuthology.orchestra.run.smithi151.stdout: "leader": 24530, 2024-05-21T21:58:24.274 INFO:teuthology.orchestra.run.smithi151.stdout: "set_version": 8, 2024-05-21T21:58:24.274 INFO:teuthology.orchestra.run.smithi151.stdout: "sets": { 2024-05-21T21:58:24.275 INFO:teuthology.orchestra.run.smithi151.stdout: "acaf991b": { 2024-05-21T21:58:24.275 INFO:teuthology.orchestra.run.smithi151.stdout: "version": 8, 2024-05-21T21:58:24.275 INFO:teuthology.orchestra.run.smithi151.stdout: "age_ref": 0.0, 2024-05-21T21:58:24.275 INFO:teuthology.orchestra.run.smithi151.stdout: "state": { 2024-05-21T21:58:24.275 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "RELEASED", 2024-05-21T21:58:24.275 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 0.0 2024-05-21T21:58:24.275 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T21:58:24.275 INFO:teuthology.orchestra.run.smithi151.stdout: "timeout": 90.0, 2024-05-21T21:58:24.275 INFO:teuthology.orchestra.run.smithi151.stdout: "expiration": 146.2, 2024-05-21T21:58:24.275 INFO:teuthology.orchestra.run.smithi151.stdout: "members": { 2024-05-21T21:58:24.275 INFO:teuthology.orchestra.run.smithi151.stdout: "file:/": { 2024-05-21T21:58:24.275 INFO:teuthology.orchestra.run.smithi151.stdout: "excluded": false, 2024-05-21T21:58:24.275 INFO:teuthology.orchestra.run.smithi151.stdout: "state": { 2024-05-21T21:58:24.275 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "RELEASED", 2024-05-21T21:58:24.276 INFO:teuthology.orchestra.run.smithi151.stdout: "age": 0.0 2024-05-21T21:58:24.276 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T21:58:24.276 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T21:58:24.276 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T21:58:24.276 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T21:58:24.276 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T21:58:24.276 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-05-21T21:58:24.277 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.272+0000 7f3183200700 1 -- 172.21.15.151:0/3378883178 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f31680b2a70 msgr2=0x7f31680b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:24.277 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.272+0000 7f3183200700 1 --2- 172.21.15.151:0/3378883178 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f31680b2a70 0x7f31680b4f30 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3174009f30 tx=0x7f3174009260 comp rx=0 tx=0).stop 2024-05-21T21:58:24.277 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.272+0000 7f3183200700 1 -- 172.21.15.151:0/3378883178 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3184105060 msgr2=0x7f318419c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:24.277 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.272+0000 7f3183200700 1 --2- 172.21.15.151:0/3378883178 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3184105060 0x7f318419c380 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f317000e9c0 tx=0x7f317000ecd0 comp rx=0 tx=0).stop 2024-05-21T21:58:24.277 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.273+0000 7f3183200700 1 -- 172.21.15.151:0/3378883178 shutdown_connections 2024-05-21T21:58:24.277 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.273+0000 7f3183200700 1 --2- 172.21.15.151:0/3378883178 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f31680b2a70 0x7f31680b4f30 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:24.277 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.273+0000 7f3183200700 1 --2- 172.21.15.151:0/3378883178 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f318410db20 0x7f31841a0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:24.277 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.273+0000 7f3183200700 1 --2- 172.21.15.151:0/3378883178 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3184104740 0x7f318419be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:24.277 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.273+0000 7f3183200700 1 --2- 172.21.15.151:0/3378883178 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3184105060 0x7f318419c380 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:24.278 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.273+0000 7f3183200700 1 -- 172.21.15.151:0/3378883178 >> 172.21.15.151:0/3378883178 conn(0x7f31840fded0 msgr2=0x7f31841002c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:58:24.278 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.273+0000 7f3183200700 1 -- 172.21.15.151:0/3378883178 shutdown_connections 2024-05-21T21:58:24.278 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:24.273+0000 7f3183200700 1 -- 172.21.15.151:0/3378883178 wait complete. 2024-05-21T21:58:24.285 INFO:tasks.workunit.client.0.smithi151.stderr:+ wget http://download.ceph.com/qa/fsync-tester.c 2024-05-21T21:58:24.400 INFO:tasks.workunit.client.0.smithi151.stderr:--2024-05-21 21:58:24-- http://download.ceph.com/qa/fsync-tester.c 2024-05-21T21:58:24.402 INFO:tasks.workunit.client.0.smithi151.stderr:Resolving download.ceph.com (download.ceph.com)... 158.69.68.124, 2607:5300:201:2000::3:58a1 2024-05-21T21:58:24.445 INFO:tasks.workunit.client.0.smithi151.stderr:Connecting to download.ceph.com (download.ceph.com)|158.69.68.124|:80... connected. 2024-05-21T21:58:24.481 INFO:tasks.workunit.client.0.smithi151.stderr:HTTP request sent, awaiting response... 200 OK 2024-05-21T21:58:24.482 INFO:tasks.workunit.client.0.smithi151.stderr:Length: 3024 (3.0K) [application/octet-stream] 2024-05-21T21:58:24.485 INFO:tasks.workunit.client.0.smithi151.stderr:Saving to: ‘fsync-tester.c’ 2024-05-21T21:58:24.494 INFO:tasks.workunit.client.0.smithi151.stderr: 2024-05-21T21:58:24.494 INFO:tasks.workunit.client.0.smithi151.stderr: 0K .. 100% 6.21M=0s 2024-05-21T21:58:24.494 INFO:tasks.workunit.client.0.smithi151.stderr: 2024-05-21T21:58:24.494 INFO:tasks.workunit.client.0.smithi151.stderr:2024-05-21 21:58:24 (6.21 MB/s) - ‘fsync-tester.c’ saved [3024/3024] 2024-05-21T21:58:24.495 INFO:tasks.workunit.client.0.smithi151.stderr: 2024-05-21T21:58:24.496 INFO:tasks.workunit.client.0.smithi151.stderr:+ gcc -D_GNU_SOURCE fsync-tester.c -o fsync-tester 2024-05-21T21:58:24.659 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:58:24 smithi151 ceph-mon[31623]: scrub summary: idle 2024-05-21T21:58:24.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:58:24 smithi184 ceph-mon[36942]: scrub summary: idle 2024-05-21T21:58:24.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:58:24 smithi164 ceph-mon[36269]: scrub summary: idle 2024-05-21T21:58:24.824 INFO:tasks.workunit.client.0.smithi151.stderr:+ ./fsync-tester 2024-05-21T21:58:25.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:58:25 smithi151 ceph-mon[31623]: pgmap v87: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-05-21T21:58:25.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:58:25 smithi184 ceph-mon[36942]: pgmap v87: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-05-21T21:58:25.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:58:25 smithi164 ceph-mon[36269]: pgmap v87: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-05-21T21:58:25.989 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'acaf991b', total seconds elapsed: 38.0 2024-05-21T21:58:26.518 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.517+0000 7f37ef200700 1 -- 172.21.15.151:0/3842199400 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f37e81095b0 msgr2=0x7f37e8109990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:26.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.517+0000 7f37ef200700 1 --2- 172.21.15.151:0/3842199400 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f37e81095b0 0x7f37e8109990 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f37d8009990 tx=0x7f37d8009ca0 comp rx=0 tx=0).stop 2024-05-21T21:58:26.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.517+0000 7f37ef200700 1 -- 172.21.15.151:0/3842199400 shutdown_connections 2024-05-21T21:58:26.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.517+0000 7f37ef200700 1 --2- 172.21.15.151:0/3842199400 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37e806bbe0 0x7f37e81103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:26.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.517+0000 7f37ef200700 1 --2- 172.21.15.151:0/3842199400 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f37e81095b0 0x7f37e8109990 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:26.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.517+0000 7f37ef200700 1 --2- 172.21.15.151:0/3842199400 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37e806b220 0x7f37e806b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:26.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.517+0000 7f37ef200700 1 -- 172.21.15.151:0/3842199400 >> 172.21.15.151:0/3842199400 conn(0x7f37e80793f0 msgr2=0x7f37e8079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:58:26.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.517+0000 7f37ef200700 1 -- 172.21.15.151:0/3842199400 shutdown_connections 2024-05-21T21:58:26.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.517+0000 7f37ef200700 1 -- 172.21.15.151:0/3842199400 wait complete. 2024-05-21T21:58:26.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.518+0000 7f37ef200700 1 Processor -- start 2024-05-21T21:58:26.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.518+0000 7f37ef200700 1 -- start start 2024-05-21T21:58:26.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.519+0000 7f37ef200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f37e806b220 0x7f37e819bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:26.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.519+0000 7f37ef200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37e806bbe0 0x7f37e819c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:26.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.519+0000 7f37ef200700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37e81095b0 0x7f37e81a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:26.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.519+0000 7f37ef200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f37e8113740 con 0x7f37e806b220 2024-05-21T21:58:26.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.519+0000 7f37ef200700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f37e81135c0 con 0x7f37e806bbe0 2024-05-21T21:58:26.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.519+0000 7f37ef200700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f37e81138c0 con 0x7f37e81095b0 2024-05-21T21:58:26.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.519+0000 7f37e7e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f37e806b220 0x7f37e819bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:26.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.519+0000 7f37e7e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f37e806b220 0x7f37e819bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49680/0 (socket says 172.21.15.151:49680) 2024-05-21T21:58:26.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.519+0000 7f37e7e00700 1 -- 172.21.15.151:0/1089417356 learned_addr learned my addr 172.21.15.151:0/1089417356 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:58:26.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.519+0000 7f37e7400700 1 --2- 172.21.15.151:0/1089417356 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37e806bbe0 0x7f37e819c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:26.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.519+0000 7f37ed200700 1 --2- 172.21.15.151:0/1089417356 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37e81095b0 0x7f37e81a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:26.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.520+0000 7f37e7e00700 1 -- 172.21.15.151:0/1089417356 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37e81095b0 msgr2=0x7f37e81a0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:26.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.520+0000 7f37e7e00700 1 --2- 172.21.15.151:0/1089417356 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37e81095b0 0x7f37e81a0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:26.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.520+0000 7f37e7e00700 1 -- 172.21.15.151:0/1089417356 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37e806bbe0 msgr2=0x7f37e819c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:26.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.520+0000 7f37e7e00700 1 --2- 172.21.15.151:0/1089417356 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37e806bbe0 0x7f37e819c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:26.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.520+0000 7f37e7e00700 1 -- 172.21.15.151:0/1089417356 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37e81a0f60 con 0x7f37e806b220 2024-05-21T21:58:26.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.520+0000 7f37e7e00700 1 --2- 172.21.15.151:0/1089417356 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f37e806b220 0x7f37e819bf50 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f37d8000c00 tx=0x7f37d80057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:58:26.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.520+0000 7f37e4c00700 1 -- 172.21.15.151:0/1089417356 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d8027070 con 0x7f37e806b220 2024-05-21T21:58:26.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.520+0000 7f37ef200700 1 -- 172.21.15.151:0/1089417356 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37e81a11f0 con 0x7f37e806b220 2024-05-21T21:58:26.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.520+0000 7f37e4c00700 1 -- 172.21.15.151:0/1089417356 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37d8022470 con 0x7f37e806b220 2024-05-21T21:58:26.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.521+0000 7f37e4c00700 1 -- 172.21.15.151:0/1089417356 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d8017650 con 0x7f37e806b220 2024-05-21T21:58:26.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.521+0000 7f37ef200700 1 -- 172.21.15.151:0/1089417356 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37e81a8ad0 con 0x7f37e806b220 2024-05-21T21:58:26.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.524+0000 7f37e4c00700 1 -- 172.21.15.151:0/1089417356 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7f37d802a070 con 0x7f37e806b220 2024-05-21T21:58:26.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.525+0000 7f37f9c47b80 1 Processor -- start 2024-05-21T21:58:26.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.525+0000 7f37f9c47b80 1 -- start start 2024-05-21T21:58:26.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.526+0000 7f37e4c00700 1 --2- 172.21.15.151:0/1089417356 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f37cc0b2a70 0x7f37cc0b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:26.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.526+0000 7f37f9c47b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5626543eb610 0x5626543eb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:26.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.526+0000 7f37f9c47b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5626543ebf30 0x5626543e5b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:26.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.526+0000 7f37f9c47b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5626543e6140 0x5626543e65c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:26.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.526+0000 7f37f9c47b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x562654343e40 con 0x5626543ebf30 2024-05-21T21:58:26.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.526+0000 7f37e4c00700 1 -- 172.21.15.151:0/1089417356 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f37d80e3250 con 0x7f37e806b220 2024-05-21T21:58:26.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.526+0000 7f37f9c47b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5626543339e0 con 0x5626543eb610 2024-05-21T21:58:26.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.526+0000 7f37f9c47b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x562654337a20 con 0x5626543e6140 2024-05-21T21:58:26.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.526+0000 7f37ed200700 1 --2- 172.21.15.151:0/1089417356 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f37cc0b2a70 0x7f37cc0b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:26.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.526+0000 7f37e7e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5626543eb610 0x5626543eb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:26.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.526+0000 7f37ed200700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5626543e6140 0x5626543e65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:26.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.526+0000 7f37e7e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5626543eb610 0x5626543eb9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:54082/0 (socket says 172.21.15.151:54082) 2024-05-21T21:58:26.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.526+0000 7f37e7e00700 1 -- 172.21.15.151:0/4253239217 learned_addr learned my addr 172.21.15.151:0/4253239217 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:58:26.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.526+0000 7f37e7400700 1 --2- 172.21.15.151:0/4253239217 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5626543ebf30 0x5626543e5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:26.529 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.527+0000 7f37e7e00700 1 -- 172.21.15.151:0/4253239217 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5626543e6140 msgr2=0x5626543e65c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:26.529 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.527+0000 7f37e7e00700 1 --2- 172.21.15.151:0/4253239217 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5626543e6140 0x5626543e65c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:26.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.527+0000 7f37e7e00700 1 -- 172.21.15.151:0/4253239217 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5626543ebf30 msgr2=0x5626543e5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:26.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.527+0000 7f37e7e00700 1 --2- 172.21.15.151:0/4253239217 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5626543ebf30 0x5626543e5b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:26.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.527+0000 7f37e7e00700 1 -- 172.21.15.151:0/4253239217 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626542fc420 con 0x5626543eb610 2024-05-21T21:58:26.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.527+0000 7f37e7400700 1 --2- 172.21.15.151:0/4253239217 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5626543ebf30 0x5626543e5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T21:58:26.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.527+0000 7f37ed200700 1 --2- 172.21.15.151:0/1089417356 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f37cc0b2a70 0x7f37cc0b4f30 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f37e0009ec0 tx=0x7f37e000fc40 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:58:26.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.527+0000 7f37e7e00700 1 --2- 172.21.15.151:0/4253239217 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5626543eb610 0x5626543eb9f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f37d8006010 tx=0x7f37d80a9e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=834bf9cbd4cba98b server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:58:26.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.527+0000 7f37dd600700 1 -- 172.21.15.151:0/4253239217 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d80ae980 con 0x5626543eb610 2024-05-21T21:58:26.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.527+0000 7f37dd600700 1 -- 172.21.15.151:0/4253239217 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37d80b33a0 con 0x5626543eb610 2024-05-21T21:58:26.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.527+0000 7f37dd600700 1 -- 172.21.15.151:0/4253239217 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d80aa830 con 0x5626543eb610 2024-05-21T21:58:26.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.528+0000 7f37f9c47b80 1 -- 172.21.15.151:0/4253239217 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5626543eb610 msgr2=0x5626543eb9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:26.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.528+0000 7f37f9c47b80 1 --2- 172.21.15.151:0/4253239217 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5626543eb610 0x5626543eb9f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f37d8006010 tx=0x7f37d80a9e30 comp rx=0 tx=0).stop 2024-05-21T21:58:26.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.528+0000 7f37f9c47b80 1 -- 172.21.15.151:0/4253239217 shutdown_connections 2024-05-21T21:58:26.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.528+0000 7f37f9c47b80 1 --2- 172.21.15.151:0/4253239217 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5626543eb610 0x5626543eb9f0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:26.531 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.528+0000 7f37f9c47b80 1 --2- 172.21.15.151:0/4253239217 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5626543e6140 0x5626543e65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:26.531 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.528+0000 7f37f9c47b80 1 --2- 172.21.15.151:0/4253239217 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5626543ebf30 0x5626543e5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:26.531 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.528+0000 7f37f9c47b80 1 -- 172.21.15.151:0/4253239217 >> 172.21.15.151:0/4253239217 conn(0x562654313be0 msgr2=0x5626543e0690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:58:26.531 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.528+0000 7f37f9c47b80 1 -- 172.21.15.151:0/4253239217 shutdown_connections 2024-05-21T21:58:26.531 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.528+0000 7f37f9c47b80 1 -- 172.21.15.151:0/4253239217 wait complete. 2024-05-21T21:58:26.533 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.529+0000 7f37f9c47b80 1 -- start start 2024-05-21T21:58:26.533 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.529+0000 7f37f9c47b80 1 Processor -- start 2024-05-21T21:58:26.533 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.530+0000 7f37f9c47b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5626543e6140 0x5626543f1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:26.533 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.530+0000 7f37f9c47b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5626543eb610 0x5626543ef100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:26.533 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.530+0000 7f37f9c47b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5626543ebf30 0x5626543ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:26.533 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.530+0000 7f37f9c47b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x562654239fa0 con 0x5626543e6140 2024-05-21T21:58:26.533 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.530+0000 7f37f9c47b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5626542d4760 con 0x5626543ebf30 2024-05-21T21:58:26.534 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.530+0000 7f37f9c47b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5626543f27a0 con 0x5626543eb610 2024-05-21T21:58:26.534 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.530+0000 7f37f9c47b80 10 client.0 _ll_register_callbacks cb 0x5626542ddc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:58:26.534 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.530+0000 7f37e7e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5626543e6140 0x5626543f1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:26.534 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.530+0000 7f37e7e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5626543e6140 0x5626543f1d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49690/0 (socket says 172.21.15.151:49690) 2024-05-21T21:58:26.534 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.530+0000 7f37e7e00700 1 -- 172.21.15.151:0/2353397296 learned_addr learned my addr 172.21.15.151:0/2353397296 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:58:26.534 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.530+0000 7f37ed200700 1 --2- 172.21.15.151:0/2353397296 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5626543ebf30 0x5626543ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:26.534 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.530+0000 7f37e7400700 1 --2- 172.21.15.151:0/2353397296 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5626543eb610 0x5626543ef100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:26.534 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.530+0000 7f37e7e00700 1 -- 172.21.15.151:0/2353397296 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5626543eb610 msgr2=0x5626543ef100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:26.534 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.530+0000 7f37e7e00700 1 --2- 172.21.15.151:0/2353397296 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5626543eb610 0x5626543ef100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:26.534 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.530+0000 7f37e7e00700 1 -- 172.21.15.151:0/2353397296 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5626543ebf30 msgr2=0x5626543ef640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:26.534 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.530+0000 7f37e7e00700 1 --2- 172.21.15.151:0/2353397296 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5626543ebf30 0x5626543ef640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:26.534 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.530+0000 7f37e7e00700 1 -- 172.21.15.151:0/2353397296 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626542838d0 con 0x5626543e6140 2024-05-21T21:58:26.534 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.531+0000 7f37e7e00700 1 --2- 172.21.15.151:0/2353397296 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5626543e6140 0x5626543f1d60 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f37d80aa560 tx=0x7f37d80a65c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61af1654d50b5e5e server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:58:26.535 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.531+0000 7f37d2000700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-21T21:58:26.535 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.531+0000 7f37d2000700 1 -- 172.21.15.151:0/2353397296 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d8022920 con 0x5626543e6140 2024-05-21T21:58:26.535 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.531+0000 7f37f9c47b80 1 -- 172.21.15.151:0/2353397296 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37d80b33a0 con 0x5626543e6140 2024-05-21T21:58:26.535 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.531+0000 7f37d2000700 1 -- 172.21.15.151:0/2353397296 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f37d80aa830 con 0x5626543e6140 2024-05-21T21:58:26.535 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.531+0000 7f37d2000700 1 -- 172.21.15.151:0/2353397296 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f37d8022ac0 con 0x5626543e6140 2024-05-21T21:58:26.535 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.531+0000 7f37f9c47b80 10 client.15810 fetch_fsmap learned FSMap version 29 2024-05-21T21:58:26.535 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.531+0000 7f37f9c47b80 1 -- 172.21.15.151:0/2353397296 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562654304060 con 0x5626543e6140 2024-05-21T21:58:26.535 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.532+0000 7f37d2000700 1 -- 172.21.15.151:0/2353397296 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37d80b33a0 con 0x5626543e6140 2024-05-21T21:58:26.535 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.532+0000 7f37d2000700 1 -- 172.21.15.151:0/2353397296 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f37d8005210 con 0x5626543e6140 2024-05-21T21:58:26.535 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.532+0000 7f37f9c47b80 10 client.15810 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T21:58:26.535 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.532+0000 7f37f9c47b80 10 client.15810 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T21:58:26.535 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.532+0000 7f37f9c47b80 20 client.15810 populate_metadata read hostname 'smithi151' 2024-05-21T21:58:26.535 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.532+0000 7f37f9c47b80 1 --2- 172.21.15.151:0/2353397296 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5626543ff1b0 0x5626544015a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:26.535 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.532+0000 7f37f9c47b80 4 client.15810 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T21:58:26.536 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.532+0000 7f37f9c47b80 1 -- 172.21.15.151:0/2353397296 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562654345d90 con 0x5626543ff1b0 2024-05-21T21:58:26.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.532+0000 7f37e7400700 1 --2- 172.21.15.151:0/2353397296 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5626543ff1b0 0x5626544015a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:26.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.533+0000 7f37e7400700 1 --2- 172.21.15.151:0/2353397296 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5626543ff1b0 0x5626544015a0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7fd12ef4fdb23b6 server_cookie=2a7c07c3bfd3f7e3 in_seq=0 out_seq=0 2024-05-21T21:58:26.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.533+0000 7f37d2000700 10 client.15810 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T21:58:26.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.534+0000 7f37d2000700 1 -- 172.21.15.151:0/2353397296 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x562654345d90 con 0x5626543ff1b0 2024-05-21T21:58:26.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.534+0000 7f37d2000700 10 client.15810 handle_command_reply: tid=0 2024-05-21T21:58:26.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.534+0000 7f37f9c47b80 1 client.15810 shutdown 2024-05-21T21:58:26.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.534+0000 7f37f9c47b80 1 -- 172.21.15.151:0/2353397296 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5626543e6140 msgr2=0x5626543f1d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:26.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.534+0000 7f37f9c47b80 1 --2- 172.21.15.151:0/2353397296 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5626543e6140 0x5626543f1d60 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f37d80aa560 tx=0x7f37d80a65c0 comp rx=0 tx=0).stop 2024-05-21T21:58:26.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.534+0000 7f37f9c47b80 1 -- 172.21.15.151:0/2353397296 shutdown_connections 2024-05-21T21:58:26.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.534+0000 7f37f9c47b80 1 --2- 172.21.15.151:0/2353397296 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5626543ff1b0 0x5626544015a0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:26.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.534+0000 7f37d2000700 0 client.15810 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T21:58:26.538 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.534+0000 7f37f9c47b80 1 --2- 172.21.15.151:0/2353397296 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5626543ebf30 0x5626543ef640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:26.538 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.534+0000 7f37f9c47b80 1 --2- 172.21.15.151:0/2353397296 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5626543e6140 0x5626543f1d60 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:26.538 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.534+0000 7f37f9c47b80 1 --2- 172.21.15.151:0/2353397296 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5626543eb610 0x5626543ef100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:26.538 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.534+0000 7f37f9c47b80 1 -- 172.21.15.151:0/2353397296 >> 172.21.15.151:0/2353397296 conn(0x562654313be0 msgr2=0x5626543dfd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:58:26.538 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.534+0000 7f37f9c47b80 1 -- 172.21.15.151:0/2353397296 shutdown_connections 2024-05-21T21:58:26.538 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.534+0000 7f37f9c47b80 1 -- 172.21.15.151:0/2353397296 wait complete. 2024-05-21T21:58:26.538 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.535+0000 7f37f9c47b80 20 client.15810 trim_cache size 0 max 16384 2024-05-21T21:58:26.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.557+0000 7f37f9c47b80 1 Processor -- start 2024-05-21T21:58:26.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.557+0000 7f37f9c47b80 1 -- start start 2024-05-21T21:58:26.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.557+0000 7f37f9c47b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5626543e6140 0x5626543e5b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:26.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.558+0000 7f37f9c47b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5626543eb610 0x5626543e04f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:26.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.558+0000 7f37f9c47b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5626543ebf30 0x5626543eca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:26.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.558+0000 7f37f9c47b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f37e81aff80 con 0x5626543e6140 2024-05-21T21:58:26.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.558+0000 7f37f9c47b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f37e81b1ac0 con 0x5626543ebf30 2024-05-21T21:58:26.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.558+0000 7f37f9c47b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f37e81b0650 con 0x5626543eb610 2024-05-21T21:58:26.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.558+0000 7f37e7400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5626543eb610 0x5626543e04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:26.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.558+0000 7f37e7e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5626543e6140 0x5626543e5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:26.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.558+0000 7f37e7400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5626543eb610 0x5626543e04f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:58296/0 (socket says 172.21.15.151:58296) 2024-05-21T21:58:26.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.558+0000 7f37e7400700 1 -- 172.21.15.151:0/2282237114 learned_addr learned my addr 172.21.15.151:0/2282237114 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:58:26.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.558+0000 7f37ed200700 1 --2- 172.21.15.151:0/2282237114 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5626543ebf30 0x5626543eca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:26.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.558+0000 7f37e7e00700 1 -- 172.21.15.151:0/2282237114 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5626543eb610 msgr2=0x5626543e04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:26.561 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.558+0000 7f37e7e00700 1 --2- 172.21.15.151:0/2282237114 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5626543eb610 0x5626543e04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:26.561 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.558+0000 7f37e7e00700 1 -- 172.21.15.151:0/2282237114 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5626543ebf30 msgr2=0x5626543eca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:26.561 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.558+0000 7f37e7e00700 1 --2- 172.21.15.151:0/2282237114 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5626543ebf30 0x5626543eca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:26.561 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.558+0000 7f37e7e00700 1 -- 172.21.15.151:0/2282237114 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626543e14a0 con 0x5626543e6140 2024-05-21T21:58:26.561 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.559+0000 7f37e7e00700 1 --2- 172.21.15.151:0/2282237114 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5626543e6140 0x5626543e5b20 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f37d80aa2d0 tx=0x7f37d80e09d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=893d12026be0d48a server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:58:26.561 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.559+0000 7f37dcc00700 1 -- 172.21.15.151:0/2282237114 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d8022920 con 0x5626543e6140 2024-05-21T21:58:26.561 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.559+0000 7f37dcc00700 1 -- 172.21.15.151:0/2282237114 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37d80aa0e0 con 0x5626543e6140 2024-05-21T21:58:26.561 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.559+0000 7f37dcc00700 1 -- 172.21.15.151:0/2282237114 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d80b2f10 con 0x5626543e6140 2024-05-21T21:58:26.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.560+0000 7f37f9c47b80 1 -- 172.21.15.151:0/2282237114 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5626543e6140 msgr2=0x5626543e5b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:26.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.560+0000 7f37f9c47b80 1 --2- 172.21.15.151:0/2282237114 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5626543e6140 0x5626543e5b20 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f37d80aa2d0 tx=0x7f37d80e09d0 comp rx=0 tx=0).stop 2024-05-21T21:58:26.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.560+0000 7f37f9c47b80 1 -- 172.21.15.151:0/2282237114 shutdown_connections 2024-05-21T21:58:26.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.560+0000 7f37f9c47b80 1 --2- 172.21.15.151:0/2282237114 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5626543ebf30 0x5626543eca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:26.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.560+0000 7f37f9c47b80 1 --2- 172.21.15.151:0/2282237114 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5626543e6140 0x5626543e5b20 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:26.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.560+0000 7f37f9c47b80 1 --2- 172.21.15.151:0/2282237114 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5626543eb610 0x5626543e04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:26.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.560+0000 7f37f9c47b80 1 -- 172.21.15.151:0/2282237114 >> 172.21.15.151:0/2282237114 conn(0x562654313be0 msgr2=0x5626543ec310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:58:26.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.560+0000 7f37f9c47b80 1 -- 172.21.15.151:0/2282237114 shutdown_connections 2024-05-21T21:58:26.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.560+0000 7f37f9c47b80 1 -- 172.21.15.151:0/2282237114 wait complete. 2024-05-21T21:58:26.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.561+0000 7f37f9c47b80 1 -- start start 2024-05-21T21:58:26.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.561+0000 7f37f9c47b80 1 Processor -- start 2024-05-21T21:58:26.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.561+0000 7f37f9c47b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5626543e6140 0x5626544025e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:26.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.561+0000 7f37f9c47b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5626543eb610 0x562654402b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:26.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.561+0000 7f37f9c47b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5626543ebf30 0x562654403060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:26.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.561+0000 7f37f9c47b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f37e81b13f0 con 0x5626543ebf30 2024-05-21T21:58:26.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.561+0000 7f37f9c47b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f37e81b0d20 con 0x5626543eb610 2024-05-21T21:58:26.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.561+0000 7f37f9c47b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f37e81adf40 con 0x5626543e6140 2024-05-21T21:58:26.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.561+0000 7f37f9c47b80 10 client.0 _ll_register_callbacks cb 0x5626542ddc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:58:26.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.562+0000 7f37e7400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5626543eb610 0x562654402b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:26.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.562+0000 7f37e7400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5626543eb610 0x562654402b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:54114/0 (socket says 172.21.15.151:54114) 2024-05-21T21:58:26.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.562+0000 7f37ed200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5626543ebf30 0x562654403060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:26.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.562+0000 7f37e7400700 1 -- 172.21.15.151:0/1322702286 learned_addr learned my addr 172.21.15.151:0/1322702286 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:58:26.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.562+0000 7f37e7e00700 1 --2- 172.21.15.151:0/1322702286 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5626543e6140 0x5626544025e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:26.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.562+0000 7f37e7400700 1 -- 172.21.15.151:0/1322702286 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5626543e6140 msgr2=0x5626544025e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:26.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.562+0000 7f37e7400700 1 --2- 172.21.15.151:0/1322702286 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5626543e6140 0x5626544025e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:26.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.562+0000 7f37e7400700 1 -- 172.21.15.151:0/1322702286 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5626543ebf30 msgr2=0x562654403060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:26.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.562+0000 7f37e7400700 1 --2- 172.21.15.151:0/1322702286 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5626543ebf30 0x562654403060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:26.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.562+0000 7f37e7400700 1 -- 172.21.15.151:0/1322702286 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562654468ca0 con 0x5626543eb610 2024-05-21T21:58:26.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.562+0000 7f37e7e00700 1 --2- 172.21.15.151:0/1322702286 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5626543e6140 0x5626544025e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T21:58:26.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.563+0000 7f37e7400700 1 --2- 172.21.15.151:0/1322702286 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5626543eb610 0x562654402b20 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x5626543f06d0 tx=0x7f37d4004320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2cbee2209d2071 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:58:26.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.563+0000 7f37d2000700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-21T21:58:26.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.563+0000 7f37d2000700 1 -- 172.21.15.151:0/1322702286 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d4004830 con 0x5626543eb610 2024-05-21T21:58:26.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.563+0000 7f37f9c47b80 1 -- 172.21.15.151:0/1322702286 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37d80aa0e0 con 0x5626543eb610 2024-05-21T21:58:26.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.563+0000 7f37d2000700 1 -- 172.21.15.151:0/1322702286 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f37d40049d0 con 0x5626543eb610 2024-05-21T21:58:26.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.563+0000 7f37d2000700 1 -- 172.21.15.151:0/1322702286 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f37d402f050 con 0x5626543eb610 2024-05-21T21:58:26.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.563+0000 7f37f9c47b80 10 client.25943 fetch_fsmap learned FSMap version 29 2024-05-21T21:58:26.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.563+0000 7f37f9c47b80 1 -- 172.21.15.151:0/1322702286 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56265446a040 con 0x5626543eb610 2024-05-21T21:58:26.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.564+0000 7f37d2000700 1 -- 172.21.15.151:0/1322702286 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37d80aa0e0 con 0x5626543eb610 2024-05-21T21:58:26.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.564+0000 7f37d2000700 1 -- 172.21.15.151:0/1322702286 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f37d4010530 con 0x5626543eb610 2024-05-21T21:58:26.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.564+0000 7f37f9c47b80 10 client.25943 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T21:58:26.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.564+0000 7f37f9c47b80 10 client.25943 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T21:58:26.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.564+0000 7f37f9c47b80 20 client.25943 populate_metadata read hostname 'smithi151' 2024-05-21T21:58:26.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.564+0000 7f37f9c47b80 1 --2- 172.21.15.151:0/1322702286 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x56265446a730 0x56265446cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:26.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.564+0000 7f37f9c47b80 4 client.25943 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-21T21:58:26.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.564+0000 7f37f9c47b80 1 -- 172.21.15.151:0/1322702286 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56265433f570 con 0x56265446a730 2024-05-21T21:58:26.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.564+0000 7f37e7e00700 1 --2- 172.21.15.151:0/1322702286 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x56265446a730 0x56265446cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:26.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.565+0000 7f37e7e00700 1 --2- 172.21.15.151:0/1322702286 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x56265446a730 0x56265446cb50 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26dd0b3cd8a5e026 server_cookie=b5ec181066922e11 in_seq=0 out_seq=0 2024-05-21T21:58:26.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.565+0000 7f37d2000700 10 client.25943 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T21:58:26.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.565+0000 7f37d2000700 1 -- 172.21.15.151:0/1322702286 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56265433f570 con 0x56265446a730 2024-05-21T21:58:26.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.565+0000 7f37d2000700 10 client.25943 handle_command_reply: tid=0 2024-05-21T21:58:26.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.565+0000 7f37f9c47b80 1 client.25943 shutdown 2024-05-21T21:58:26.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.565+0000 7f37f9c47b80 1 -- 172.21.15.151:0/1322702286 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5626543eb610 msgr2=0x562654402b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:26.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.565+0000 7f37f9c47b80 1 --2- 172.21.15.151:0/1322702286 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5626543eb610 0x562654402b20 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x5626543f06d0 tx=0x7f37d4004320 comp rx=0 tx=0).stop 2024-05-21T21:58:26.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.565+0000 7f37f9c47b80 1 -- 172.21.15.151:0/1322702286 shutdown_connections 2024-05-21T21:58:26.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.565+0000 7f37f9c47b80 1 --2- 172.21.15.151:0/1322702286 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5626543ebf30 0x562654403060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:26.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.565+0000 7f37f9c47b80 1 --2- 172.21.15.151:0/1322702286 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5626543e6140 0x5626544025e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:26.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.565+0000 7f37f9c47b80 1 --2- 172.21.15.151:0/1322702286 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x56265446a730 0x56265446cb50 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:26.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.565+0000 7f37f9c47b80 1 --2- 172.21.15.151:0/1322702286 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5626543eb610 0x562654402b20 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:26.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.565+0000 7f37f9c47b80 1 -- 172.21.15.151:0/1322702286 >> 172.21.15.151:0/1322702286 conn(0x562654313be0 msgr2=0x5626543ef5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:58:26.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.565+0000 7f37d2000700 0 client.25943 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T21:58:26.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.565+0000 7f37f9c47b80 1 -- 172.21.15.151:0/1322702286 shutdown_connections 2024-05-21T21:58:26.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.566+0000 7f37f9c47b80 1 -- 172.21.15.151:0/1322702286 wait complete. 2024-05-21T21:58:26.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.566+0000 7f37f9c47b80 20 client.25943 trim_cache size 0 max 16384 2024-05-21T21:58:26.570 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-05-21T21:58:26.570 INFO:teuthology.orchestra.run.smithi151.stdout: "status": "no active scrubs running", 2024-05-21T21:58:26.570 INFO:teuthology.orchestra.run.smithi151.stdout: "scrubs": {} 2024-05-21T21:58:26.570 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-05-21T21:58:26.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.566+0000 7f37ef200700 1 -- 172.21.15.151:0/1089417356 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f37cc0b2a70 msgr2=0x7f37cc0b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:26.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.566+0000 7f37ef200700 1 --2- 172.21.15.151:0/1089417356 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f37cc0b2a70 0x7f37cc0b4f30 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f37e0009ec0 tx=0x7f37e000fc40 comp rx=0 tx=0).stop 2024-05-21T21:58:26.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.566+0000 7f37ef200700 1 -- 172.21.15.151:0/1089417356 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f37e806b220 msgr2=0x7f37e819bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:26.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.566+0000 7f37ef200700 1 --2- 172.21.15.151:0/1089417356 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f37e806b220 0x7f37e819bf50 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f37d8000c00 tx=0x7f37d80057c0 comp rx=0 tx=0).stop 2024-05-21T21:58:26.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.567+0000 7f37ef200700 1 -- 172.21.15.151:0/1089417356 shutdown_connections 2024-05-21T21:58:26.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.567+0000 7f37ef200700 1 --2- 172.21.15.151:0/1089417356 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f37cc0b2a70 0x7f37cc0b4f30 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:26.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.567+0000 7f37ef200700 1 --2- 172.21.15.151:0/1089417356 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37e81095b0 0x7f37e81a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:26.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.567+0000 7f37ef200700 1 --2- 172.21.15.151:0/1089417356 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f37e806b220 0x7f37e819bf50 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:26.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.567+0000 7f37ef200700 1 --2- 172.21.15.151:0/1089417356 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37e806bbe0 0x7f37e819c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:26.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.567+0000 7f37ef200700 1 -- 172.21.15.151:0/1089417356 >> 172.21.15.151:0/1089417356 conn(0x7f37e80793f0 msgr2=0x7f37e8100e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:58:26.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.567+0000 7f37ef200700 1 -- 172.21.15.151:0/1089417356 shutdown_connections 2024-05-21T21:58:26.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:26.567+0000 7f37ef200700 1 -- 172.21.15.151:0/1089417356 wait complete. 2024-05-21T21:58:27.448 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-21T21:58:27.448 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-21T21:58:27.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:58:27 smithi151 ceph-mon[31623]: pgmap v88: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-05-21T21:58:27.735 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:58:27 smithi184 ceph-mon[36942]: pgmap v88: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-05-21T21:58:27.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:58:27 smithi164 ceph-mon[36269]: pgmap v88: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-05-21T21:58:27.888 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:58:29.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:58:29 smithi151 ceph-mon[31623]: pgmap v89: 97 pgs: 97 active+clean; 2.4 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 178 KiB/s wr, 14 op/s 2024-05-21T21:58:29.735 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:58:29 smithi184 ceph-mon[36942]: pgmap v89: 97 pgs: 97 active+clean; 2.4 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 178 KiB/s wr, 14 op/s 2024-05-21T21:58:29.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:58:29 smithi164 ceph-mon[36269]: pgmap v89: 97 pgs: 97 active+clean; 2.4 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 178 KiB/s wr, 14 op/s 2024-05-21T21:58:31.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:58:31 smithi151 ceph-mon[31623]: pgmap v90: 97 pgs: 97 active+clean; 2.4 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 178 KiB/s wr, 10 op/s 2024-05-21T21:58:31.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:58:31 smithi184 ceph-mon[36942]: pgmap v90: 97 pgs: 97 active+clean; 2.4 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 178 KiB/s wr, 10 op/s 2024-05-21T21:58:31.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:58:31 smithi164 ceph-mon[36269]: pgmap v90: 97 pgs: 97 active+clean; 2.4 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 178 KiB/s wr, 10 op/s 2024-05-21T21:58:32.042 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:32.040+0000 7fe212e00700 1 -- 172.21.15.151:0/70046015 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe20c1088f0 msgr2=0x7fe20c110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:32.042 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:32.040+0000 7fe212e00700 1 --2- 172.21.15.151:0/70046015 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe20c1088f0 0x7fe20c110470 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe20400b0d0 tx=0x7fe20400b3e0 comp rx=0 tx=0).stop 2024-05-21T21:58:32.042 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:32.040+0000 7fe212e00700 1 -- 172.21.15.151:0/70046015 shutdown_connections 2024-05-21T21:58:32.042 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:32.040+0000 7fe212e00700 1 --2- 172.21.15.151:0/70046015 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe20c1088f0 0x7fe20c110470 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:32.042 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:32.040+0000 7fe212e00700 1 --2- 172.21.15.151:0/70046015 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe20c1037f0 0x7fe20c103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:32.042 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:32.040+0000 7fe212e00700 1 --2- 172.21.15.151:0/70046015 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe20c1041a0 0x7fe20c1081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:32.042 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:32.040+0000 7fe212e00700 1 -- 172.21.15.151:0/70046015 >> 172.21.15.151:0/70046015 conn(0x7fe20c07a6a0 msgr2=0x7fe20c07aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:58:32.042 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:32.040+0000 7fe212e00700 1 -- 172.21.15.151:0/70046015 shutdown_connections 2024-05-21T21:58:32.043 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:32.041+0000 7fe212e00700 1 -- 172.21.15.151:0/70046015 wait complete. 2024-05-21T21:58:32.043 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:32.042+0000 7fe212e00700 1 Processor -- start 2024-05-21T21:58:32.043 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:32.042+0000 7fe212e00700 1 -- start start 2024-05-21T21:58:32.044 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:32.043+0000 7fe212e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe20c1037f0 0x7fe20c19f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:32.044 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:32.043+0000 7fe212e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe20c1041a0 0x7fe20c19fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:32.044 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:32.043+0000 7fe212e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe20c1088f0 0x7fe20c199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:32.044 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:32.043+0000 7fe212e00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe20c113710 con 0x7fe20c1037f0 2024-05-21T21:58:32.044 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:32.043+0000 7fe212e00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe20c113590 con 0x7fe20c1088f0 2024-05-21T21:58:32.044 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:32.043+0000 7fe212e00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe20c113890 con 0x7fe20c1041a0 2024-05-21T21:58:32.045 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:32.043+0000 7fe20b400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe20c1041a0 0x7fe20c19fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:32.046 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:32.043+0000 7fe20be00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe20c1037f0 0x7fe20c19f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:32.046 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:32.043+0000 7fe20b400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe20c1041a0 0x7fe20c19fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:58332/0 (socket says 172.21.15.151:58332) 2024-05-21T21:58:32.046 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:32.043+0000 7fe20b400700 1 -- 172.21.15.151:0/2478364973 learned_addr learned my addr 172.21.15.151:0/2478364973 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:58:32.046 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:32.043+0000 7fe210e00700 1 --2- 172.21.15.151:0/2478364973 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe20c1088f0 0x7fe20c199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:32.046 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:32.044+0000 7fe210e00700 1 -- 172.21.15.151:0/2478364973 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe20c1041a0 msgr2=0x7fe20c19fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:32.046 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:32.044+0000 7fe210e00700 1 --2- 172.21.15.151:0/2478364973 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe20c1041a0 0x7fe20c19fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:32.046 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:32.044+0000 7fe210e00700 1 -- 172.21.15.151:0/2478364973 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe20c1037f0 msgr2=0x7fe20c19f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:32.046 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:32.044+0000 7fe210e00700 1 --2- 172.21.15.151:0/2478364973 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe20c1037f0 0x7fe20c19f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:32.046 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:32.044+0000 7fe210e00700 1 -- 172.21.15.151:0/2478364973 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe20c19a0b0 con 0x7fe20c1088f0 2024-05-21T21:58:32.046 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:32.044+0000 7fe20be00700 1 --2- 172.21.15.151:0/2478364973 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe20c1037f0 0x7fe20c19f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:58:32.046 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:32.044+0000 7fe210e00700 1 --2- 172.21.15.151:0/2478364973 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe20c1088f0 0x7fe20c199950 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe20400b770 tx=0x7fe204007980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:58:32.046 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:32.044+0000 7fe208c00700 1 -- 172.21.15.151:0/2478364973 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe20401f070 con 0x7fe20c1088f0 2024-05-21T21:58:32.046 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:32.044+0000 7fe208c00700 1 -- 172.21.15.151:0/2478364973 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe204007dc0 con 0x7fe20c1088f0 2024-05-21T21:58:32.047 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:32.044+0000 7fe212e00700 1 -- 172.21.15.151:0/2478364973 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe20c19a340 con 0x7fe20c1088f0 2024-05-21T21:58:32.047 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:32.044+0000 7fe208c00700 1 -- 172.21.15.151:0/2478364973 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe20401b430 con 0x7fe20c1088f0 2024-05-21T21:58:32.047 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:32.045+0000 7fe212e00700 1 -- 172.21.15.151:0/2478364973 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe20c1a6660 con 0x7fe20c1088f0 2024-05-21T21:58:32.047 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:32.046+0000 7fe212e00700 1 -- 172.21.15.151:0/2478364973 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe20c104c80 con 0x7fe20c1088f0 2024-05-21T21:58:32.049 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:32.048+0000 7fe208c00700 1 -- 172.21.15.151:0/2478364973 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7fe204051d10 con 0x7fe20c1088f0 2024-05-21T21:58:32.051 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:32.050+0000 7fe208c00700 1 --2- 172.21.15.151:0/2478364973 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fe1f00b29f0 0x7fe1f00b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:32.051 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:32.050+0000 7fe208c00700 1 -- 172.21.15.151:0/2478364973 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fe2040e3390 con 0x7fe20c1088f0 2024-05-21T21:58:32.051 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:32.050+0000 7fe20be00700 1 --2- 172.21.15.151:0/2478364973 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fe1f00b29f0 0x7fe1f00b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:32.051 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:32.051+0000 7fe20be00700 1 --2- 172.21.15.151:0/2478364973 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fe1f00b29f0 0x7fe1f00b4eb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe20c19aa90 tx=0x7fe1fc009260 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:58:32.053 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:32.052+0000 7fe208c00700 1 -- 172.21.15.151:0/2478364973 <== mon.2 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe2040aadb0 con 0x7fe20c1088f0 2024-05-21T21:58:32.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:32.335+0000 7fe212e00700 1 -- 172.21.15.151:0/2478364973 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe20c19ae80 con 0x7fe20c1088f0 2024-05-21T21:58:32.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:32.336+0000 7fe208c00700 1 -- 172.21.15.151:0/2478364973 <== mon.2 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11958 (secure 0 0 0) 0x7fe2040aa6b0 con 0x7fe20c1088f0 2024-05-21T21:58:32.338 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T21:58:32.338 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14358,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/158010529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":158010529},{"type":"v1","addr":"172.21.15.151:6835","nonce":158010529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6839/2069058115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":2069058115},{"type":"v1","addr":"172.21.15.151:6839","nonce":2069058115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T21:51:19.552069+0000","modified":"2024-05-21T21:55:34.643755+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24530,"mds_1":24493,"mds_2":24491,"mds_3":24478,"mds_4":24473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14403":{"gid":14403,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6841/1749723794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6840","nonce":1749723794},{"type":"v1","addr":"172.21.15.151:6841","nonce":1749723794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/2026266114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":2026266114},{"type":"v1","addr":"172.21.15.184:6833","nonce":2026266114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/2467900844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":2467900844},{"type":"v1","addr":"172.21.15.164:6833","nonce":2467900844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3367640960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3367640960},{"type":"v1","addr":"172.21.15.151:6837","nonce":3367640960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/3264142621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3264142621},{"type":"v1","addr":"172.21.15.184:6835","nonce":3264142621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":17,"addr":"172.21.15.164:6835/2340428470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2340428470},{"type":"v1","addr":"172.21.15.164:6835","nonce":2340428470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":16,"addr":"172.21.15.184:6837/2988194888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":2988194888},{"type":"v1","addr":"172.21.15.184:6837","nonce":2988194888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.164:6837/1554609572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1554609572},{"type":"v1","addr":"172.21.15.164:6837","nonce":1554609572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.184:6839/3059935143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":3059935143},{"type":"v1","addr":"172.21.15.184:6839","nonce":3059935143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.164:6839/2646401247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":2646401247},{"type":"v1","addr":"172.21.15.164:6839","nonce":2646401247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24530,"qdb_cluster":[24530,24493,24491,24473,24478]},"id":1}]} 2024-05-21T21:58:32.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:32.339+0000 7fe212e00700 1 -- 172.21.15.151:0/2478364973 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fe1f00b29f0 msgr2=0x7fe1f00b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:32.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:32.339+0000 7fe212e00700 1 --2- 172.21.15.151:0/2478364973 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fe1f00b29f0 0x7fe1f00b4eb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe20c19aa90 tx=0x7fe1fc009260 comp rx=0 tx=0).stop 2024-05-21T21:58:32.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:32.339+0000 7fe212e00700 1 -- 172.21.15.151:0/2478364973 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe20c1088f0 msgr2=0x7fe20c199950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:32.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:32.339+0000 7fe212e00700 1 --2- 172.21.15.151:0/2478364973 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe20c1088f0 0x7fe20c199950 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe20400b770 tx=0x7fe204007980 comp rx=0 tx=0).stop 2024-05-21T21:58:32.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:32.340+0000 7fe212e00700 1 -- 172.21.15.151:0/2478364973 shutdown_connections 2024-05-21T21:58:32.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:32.340+0000 7fe212e00700 1 --2- 172.21.15.151:0/2478364973 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fe1f00b29f0 0x7fe1f00b4eb0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:32.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:32.340+0000 7fe212e00700 1 --2- 172.21.15.151:0/2478364973 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe20c1088f0 0x7fe20c199950 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:32.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:32.340+0000 7fe212e00700 1 --2- 172.21.15.151:0/2478364973 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe20c1037f0 0x7fe20c19f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:32.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:32.340+0000 7fe212e00700 1 --2- 172.21.15.151:0/2478364973 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe20c1041a0 0x7fe20c19fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:32.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:32.340+0000 7fe212e00700 1 -- 172.21.15.151:0/2478364973 >> 172.21.15.151:0/2478364973 conn(0x7fe20c07a6a0 msgr2=0x7fe20c101560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:58:32.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:32.340+0000 7fe212e00700 1 -- 172.21.15.151:0/2478364973 shutdown_connections 2024-05-21T21:58:32.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:32.340+0000 7fe212e00700 1 -- 172.21.15.151:0/2478364973 wait complete. 2024-05-21T21:58:32.342 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 29 2024-05-21T21:58:32.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:58:32 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/2478364973' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T21:58:32.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:58:32 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/2478364973' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T21:58:32.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:58:32 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/2478364973' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T21:58:33.211 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-21T21:58:33.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:58:33 smithi151 ceph-mon[31623]: pgmap v91: 97 pgs: 97 active+clean; 75 MiB data, 579 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.6 MiB/s wr, 17 op/s 2024-05-21T21:58:33.651 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:58:33.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:58:33 smithi184 ceph-mon[36942]: pgmap v91: 97 pgs: 97 active+clean; 75 MiB data, 579 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.6 MiB/s wr, 17 op/s 2024-05-21T21:58:33.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:58:33 smithi164 ceph-mon[36269]: pgmap v91: 97 pgs: 97 active+clean; 75 MiB data, 579 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.6 MiB/s wr, 17 op/s 2024-05-21T21:58:35.573 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:58:35 smithi151 ceph-mon[31623]: pgmap v92: 97 pgs: 97 active+clean; 75 MiB data, 579 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.6 MiB/s wr, 13 op/s 2024-05-21T21:58:35.735 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:58:35 smithi184 ceph-mon[36942]: pgmap v92: 97 pgs: 97 active+clean; 75 MiB data, 579 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.6 MiB/s wr, 13 op/s 2024-05-21T21:58:35.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:58:35 smithi164 ceph-mon[36269]: pgmap v92: 97 pgs: 97 active+clean; 75 MiB data, 579 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.6 MiB/s wr, 13 op/s 2024-05-21T21:58:35.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.912+0000 7f62dfe00700 1 -- 172.21.15.151:0/3418360318 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62e0109540 msgr2=0x7f62e0109920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:35.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.912+0000 7f62dfe00700 1 --2- 172.21.15.151:0/3418360318 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62e0109540 0x7f62e0109920 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f62d00099c0 tx=0x7f62d0009cd0 comp rx=0 tx=0).stop 2024-05-21T21:58:35.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.912+0000 7f62dfe00700 1 -- 172.21.15.151:0/3418360318 shutdown_connections 2024-05-21T21:58:35.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.912+0000 7f62dfe00700 1 --2- 172.21.15.151:0/3418360318 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62e0109540 0x7f62e0109920 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:35.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.912+0000 7f62dfe00700 1 --2- 172.21.15.151:0/3418360318 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f62e0103e50 0x7f62e0110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:35.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.912+0000 7f62dfe00700 1 --2- 172.21.15.151:0/3418360318 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62e0103490 0x7f62e0103910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:35.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.912+0000 7f62dfe00700 1 -- 172.21.15.151:0/3418360318 >> 172.21.15.151:0/3418360318 conn(0x7f62e00fefc0 msgr2=0x7f62e01013e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:58:35.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.912+0000 7f62dfe00700 1 -- 172.21.15.151:0/3418360318 shutdown_connections 2024-05-21T21:58:35.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.912+0000 7f62dfe00700 1 -- 172.21.15.151:0/3418360318 wait complete. 2024-05-21T21:58:35.915 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.913+0000 7f62dfe00700 1 Processor -- start 2024-05-21T21:58:35.915 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.913+0000 7f62dfe00700 1 -- start start 2024-05-21T21:58:35.915 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.914+0000 7f62dfe00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62e0103490 0x7f62e01a1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:35.915 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.914+0000 7f62dfe00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f62e0103e50 0x7f62e01a1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:35.915 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.914+0000 7f62dfe00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62e0109540 0x7f62e019bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:35.915 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.914+0000 7f62dfe00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f62e0077390 con 0x7f62e0103e50 2024-05-21T21:58:35.915 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.914+0000 7f62dfe00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f62e0077210 con 0x7f62e0103490 2024-05-21T21:58:35.916 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.914+0000 7f62dfe00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f62e0077510 con 0x7f62e0109540 2024-05-21T21:58:35.916 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.914+0000 7f62df400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62e0109540 0x7f62e019bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:35.916 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.915+0000 7f62df400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62e0109540 0x7f62e019bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:37094/0 (socket says 172.21.15.151:37094) 2024-05-21T21:58:35.916 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.915+0000 7f62df400700 1 -- 172.21.15.151:0/3041403058 learned_addr learned my addr 172.21.15.151:0/3041403058 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:58:35.916 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.915+0000 7f62de000700 1 --2- 172.21.15.151:0/3041403058 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f62e0103e50 0x7f62e01a1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:35.917 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.915+0000 7f62dea00700 1 --2- 172.21.15.151:0/3041403058 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62e0103490 0x7f62e01a1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:35.917 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.915+0000 7f62df400700 1 -- 172.21.15.151:0/3041403058 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62e0103490 msgr2=0x7f62e01a1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:35.917 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.915+0000 7f62df400700 1 --2- 172.21.15.151:0/3041403058 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62e0103490 0x7f62e01a1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:35.917 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.915+0000 7f62df400700 1 -- 172.21.15.151:0/3041403058 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f62e0103e50 msgr2=0x7f62e01a1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:35.917 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.915+0000 7f62df400700 1 --2- 172.21.15.151:0/3041403058 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f62e0103e50 0x7f62e01a1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:35.917 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.915+0000 7f62df400700 1 -- 172.21.15.151:0/3041403058 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62e019c440 con 0x7f62e0109540 2024-05-21T21:58:35.917 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.915+0000 7f62dea00700 1 --2- 172.21.15.151:0/3041403058 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62e0103490 0x7f62e01a1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T21:58:35.917 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.915+0000 7f62de000700 1 --2- 172.21.15.151:0/3041403058 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f62e0103e50 0x7f62e01a1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T21:58:35.918 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.916+0000 7f62df400700 1 --2- 172.21.15.151:0/3041403058 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62e0109540 0x7f62e019bb60 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f62d800ef30 tx=0x7f62d800c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:58:35.918 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.916+0000 7f62d7400700 1 -- 172.21.15.151:0/3041403058 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d80092e0 con 0x7f62e0109540 2024-05-21T21:58:35.918 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.916+0000 7f62dfe00700 1 -- 172.21.15.151:0/3041403058 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62e019c6d0 con 0x7f62e0109540 2024-05-21T21:58:35.918 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.916+0000 7f62d7400700 1 -- 172.21.15.151:0/3041403058 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62d8004bb0 con 0x7f62e0109540 2024-05-21T21:58:35.918 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.916+0000 7f62dfe00700 1 -- 172.21.15.151:0/3041403058 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62e01a87f0 con 0x7f62e0109540 2024-05-21T21:58:35.918 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.916+0000 7f62d7400700 1 -- 172.21.15.151:0/3041403058 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d8004280 con 0x7f62e0109540 2024-05-21T21:58:35.920 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.920+0000 7f62d7400700 1 -- 172.21.15.151:0/3041403058 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7f62d801b030 con 0x7f62e0109540 2024-05-21T21:58:35.922 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.921+0000 7f62f0478b80 1 Processor -- start 2024-05-21T21:58:35.922 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.921+0000 7f62f0478b80 1 -- start start 2024-05-21T21:58:35.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.922+0000 7f62f0478b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56377063e7c0 0x56377063eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:35.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.922+0000 7f62f0478b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56377063f0e0 0x563770638d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:35.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.922+0000 7f62f0478b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563770639390 0x5637706397f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:35.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.922+0000 7f62f0478b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x563770596e40 con 0x56377063f0e0 2024-05-21T21:58:35.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.922+0000 7f62f0478b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5637705869e0 con 0x56377063e7c0 2024-05-21T21:58:35.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.922+0000 7f62f0478b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56377058aa20 con 0x563770639390 2024-05-21T21:58:35.924 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.922+0000 7f62d7400700 1 --2- 172.21.15.151:0/3041403058 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f62c40b29f0 0x7f62c40b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:35.924 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.922+0000 7f62de000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56377063e7c0 0x56377063eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:35.924 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.922+0000 7f62d7400700 1 -- 172.21.15.151:0/3041403058 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f62d80e2320 con 0x7f62e0109540 2024-05-21T21:58:35.924 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.922+0000 7f62de000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56377063e7c0 0x56377063eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:52794/0 (socket says 172.21.15.151:52794) 2024-05-21T21:58:35.924 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.922+0000 7f62de000700 1 -- 172.21.15.151:0/2713487348 learned_addr learned my addr 172.21.15.151:0/2713487348 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:58:35.925 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.922+0000 7f62dea00700 1 --2- 172.21.15.151:0/2713487348 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56377063f0e0 0x563770638d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:35.925 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.923+0000 7f62de000700 1 --2- 172.21.15.151:0/2713487348 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563770639390 0x5637706397f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:35.925 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.923+0000 7f62dea00700 1 -- 172.21.15.151:0/2713487348 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563770639390 msgr2=0x5637706397f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:35.925 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.923+0000 7f62dea00700 1 --2- 172.21.15.151:0/2713487348 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563770639390 0x5637706397f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:35.925 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.923+0000 7f62dea00700 1 -- 172.21.15.151:0/2713487348 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56377063e7c0 msgr2=0x56377063eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:35.925 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.923+0000 7f62dea00700 1 --2- 172.21.15.151:0/2713487348 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56377063e7c0 0x56377063eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:35.925 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.923+0000 7f62dea00700 1 -- 172.21.15.151:0/2713487348 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56377054f420 con 0x56377063f0e0 2024-05-21T21:58:35.925 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.923+0000 7f62df400700 1 --2- 172.21.15.151:0/3041403058 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f62c40b29f0 0x7f62c40b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:35.926 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.924+0000 7f62dea00700 1 --2- 172.21.15.151:0/2713487348 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56377063f0e0 0x563770638d70 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f62d00099c0 tx=0x7f62d0005f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee44d30497e41357 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:58:35.926 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.924+0000 7f62df400700 1 --2- 172.21.15.151:0/3041403058 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f62c40b29f0 0x7f62c40b4eb0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f62d8009c10 tx=0x7f62d8009040 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:58:35.926 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.924+0000 7f62cbe00700 1 -- 172.21.15.151:0/2713487348 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d0026070 con 0x56377063f0e0 2024-05-21T21:58:35.926 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.924+0000 7f62cbe00700 1 -- 172.21.15.151:0/2713487348 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62d0009510 con 0x56377063f0e0 2024-05-21T21:58:35.926 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.924+0000 7f62cbe00700 1 -- 172.21.15.151:0/2713487348 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d00056a0 con 0x56377063f0e0 2024-05-21T21:58:35.927 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.925+0000 7f62f0478b80 1 -- 172.21.15.151:0/2713487348 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56377063f0e0 msgr2=0x563770638d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:35.927 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.925+0000 7f62f0478b80 1 --2- 172.21.15.151:0/2713487348 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56377063f0e0 0x563770638d70 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f62d00099c0 tx=0x7f62d0005f00 comp rx=0 tx=0).stop 2024-05-21T21:58:35.927 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.925+0000 7f62f0478b80 1 -- 172.21.15.151:0/2713487348 shutdown_connections 2024-05-21T21:58:35.927 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.925+0000 7f62f0478b80 1 --2- 172.21.15.151:0/2713487348 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56377063e7c0 0x56377063eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:35.927 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.925+0000 7f62f0478b80 1 --2- 172.21.15.151:0/2713487348 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563770639390 0x5637706397f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:35.927 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.925+0000 7f62f0478b80 1 --2- 172.21.15.151:0/2713487348 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56377063f0e0 0x563770638d70 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:35.927 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.925+0000 7f62f0478b80 1 -- 172.21.15.151:0/2713487348 >> 172.21.15.151:0/2713487348 conn(0x563770566be0 msgr2=0x563770633810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:58:35.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.926+0000 7f62f0478b80 1 -- 172.21.15.151:0/2713487348 shutdown_connections 2024-05-21T21:58:35.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.926+0000 7f62f0478b80 1 -- 172.21.15.151:0/2713487348 wait complete. 2024-05-21T21:58:35.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.927+0000 7f62f0478b80 1 -- start start 2024-05-21T21:58:35.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.927+0000 7f62f0478b80 1 Processor -- start 2024-05-21T21:58:35.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.928+0000 7f62f0478b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563770639390 0x563770645090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:35.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.928+0000 7f62f0478b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56377063e7c0 0x5637706423c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:35.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.928+0000 7f62f0478b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563770642900 0x563770642d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:35.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.928+0000 7f62f0478b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56377048cfa0 con 0x563770642900 2024-05-21T21:58:35.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.928+0000 7f62f0478b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x563770527760 con 0x563770639390 2024-05-21T21:58:35.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.928+0000 7f62f0478b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x563770645ba0 con 0x56377063e7c0 2024-05-21T21:58:35.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.928+0000 7f62f0478b80 10 client.0 _ll_register_callbacks cb 0x563770530c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:58:35.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.929+0000 7f62de000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563770639390 0x563770645090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:35.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.929+0000 7f62de000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563770639390 0x563770645090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:52810/0 (socket says 172.21.15.151:52810) 2024-05-21T21:58:35.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.929+0000 7f62de000700 1 -- 172.21.15.151:0/2891534361 learned_addr learned my addr 172.21.15.151:0/2891534361 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:58:35.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.929+0000 7f62de000700 1 -- 172.21.15.151:0/2891534361 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56377063e7c0 msgr2=0x5637706423c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-21T21:58:35.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.929+0000 7f62de000700 1 --2- 172.21.15.151:0/2891534361 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56377063e7c0 0x5637706423c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:35.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.929+0000 7f62de000700 1 -- 172.21.15.151:0/2891534361 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563770642900 msgr2=0x563770642d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:35.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.929+0000 7f62de000700 1 --2- 172.21.15.151:0/2891534361 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563770642900 0x563770642d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:35.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.929+0000 7f62de000700 1 -- 172.21.15.151:0/2891534361 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637704d68d0 con 0x563770639390 2024-05-21T21:58:35.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.930+0000 7f62de000700 1 --2- 172.21.15.151:0/2891534361 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563770639390 0x563770645090 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f62cc009ab0 tx=0x7f62cc0116c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16574a62204bfee5 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:58:35.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.930+0000 7f62c8c00700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-21T21:58:35.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.930+0000 7f62c8c00700 1 -- 172.21.15.151:0/2891534361 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62cc011d00 con 0x563770639390 2024-05-21T21:58:35.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.930+0000 7f62c8c00700 1 -- 172.21.15.151:0/2891534361 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f62cc020430 con 0x563770639390 2024-05-21T21:58:35.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.931+0000 7f62f0478b80 1 -- 172.21.15.151:0/2891534361 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62d0009510 con 0x563770639390 2024-05-21T21:58:35.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.932+0000 7f62c8c00700 1 -- 172.21.15.151:0/2891534361 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f62cc020630 con 0x563770639390 2024-05-21T21:58:35.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.933+0000 7f62f0478b80 10 client.25958 fetch_fsmap learned FSMap version 29 2024-05-21T21:58:35.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.933+0000 7f62f0478b80 1 -- 172.21.15.151:0/2891534361 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563770557060 con 0x563770639390 2024-05-21T21:58:35.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.935+0000 7f62c8c00700 1 -- 172.21.15.151:0/2891534361 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62d0009510 con 0x563770639390 2024-05-21T21:58:35.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.935+0000 7f62c8c00700 1 -- 172.21.15.151:0/2891534361 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f62cc00f7e0 con 0x563770639390 2024-05-21T21:58:35.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.935+0000 7f62f0478b80 10 client.25958 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T21:58:35.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.935+0000 7f62f0478b80 10 client.25958 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-05-21T21:58:35.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.935+0000 7f62f0478b80 20 client.25958 populate_metadata read hostname 'smithi151' 2024-05-21T21:58:35.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.935+0000 7f62f0478b80 1 --2- 172.21.15.151:0/2891534361 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x56377065b560 0x56377065d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:35.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.935+0000 7f62f0478b80 4 client.25958 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T21:58:35.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.935+0000 7f62f0478b80 1 -- 172.21.15.151:0/2891534361 --> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563770598d90 con 0x56377065b560 2024-05-21T21:58:35.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.943+0000 7f62dea00700 1 --2- 172.21.15.151:0/2891534361 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x56377065b560 0x56377065d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:35.946 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.944+0000 7f62dea00700 1 --2- 172.21.15.151:0/2891534361 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x56377065b560 0x56377065d9d0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=465121e6d7ce0eda server_cookie=281b240cfb4bbf32 in_seq=0 out_seq=0 2024-05-21T21:58:35.946 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.944+0000 7f62c8c00700 10 client.25958 ms_handle_connect on v2:172.21.15.164:6834/2340428470 2024-05-21T21:58:35.954 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.945+0000 7f62c8c00700 1 -- 172.21.15.151:0/2891534361 <== mds.4 v2:172.21.15.164:6834/2340428470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x563770598d90 con 0x56377065b560 2024-05-21T21:58:35.954 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.945+0000 7f62c8c00700 10 client.25958 handle_command_reply: tid=0 2024-05-21T21:58:35.954 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.945+0000 7f62f0478b80 1 client.25958 shutdown 2024-05-21T21:58:35.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.946+0000 7f62f0478b80 1 -- 172.21.15.151:0/2891534361 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563770639390 msgr2=0x563770645090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:35.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.946+0000 7f62f0478b80 1 --2- 172.21.15.151:0/2891534361 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563770639390 0x563770645090 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f62cc009ab0 tx=0x7f62cc0116c0 comp rx=0 tx=0).stop 2024-05-21T21:58:35.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.948+0000 7f62f0478b80 1 -- 172.21.15.151:0/2891534361 shutdown_connections 2024-05-21T21:58:35.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.948+0000 7f62f0478b80 1 --2- 172.21.15.151:0/2891534361 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x56377065b560 0x56377065d9d0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:35.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.948+0000 7f62f0478b80 1 --2- 172.21.15.151:0/2891534361 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563770639390 0x563770645090 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:35.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.948+0000 7f62f0478b80 1 --2- 172.21.15.151:0/2891534361 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563770642900 0x563770642d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:35.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.948+0000 7f62f0478b80 1 --2- 172.21.15.151:0/2891534361 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56377063e7c0 0x5637706423c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:35.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.948+0000 7f62c8c00700 0 client.25958 ms_handle_reset on v2:172.21.15.164:6834/2340428470 2024-05-21T21:58:35.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.948+0000 7f62f0478b80 1 -- 172.21.15.151:0/2891534361 >> 172.21.15.151:0/2891534361 conn(0x563770566be0 msgr2=0x563770632fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:58:35.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.949+0000 7f62f0478b80 1 -- 172.21.15.151:0/2891534361 shutdown_connections 2024-05-21T21:58:35.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.949+0000 7f62f0478b80 1 -- 172.21.15.151:0/2891534361 wait complete. 2024-05-21T21:58:35.956 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:35.949+0000 7f62f0478b80 20 client.25958 trim_cache size 0 max 16384 2024-05-21T21:58:36.002 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.001+0000 7f62f0478b80 1 Processor -- start 2024-05-21T21:58:36.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.001+0000 7f62f0478b80 1 -- start start 2024-05-21T21:58:36.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.001+0000 7f62f0478b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563770639390 0x563770638d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:36.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.001+0000 7f62f0478b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56377063e7c0 0x56377063dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:36.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.001+0000 7f62f0478b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563770642900 0x563770633740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:36.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.001+0000 7f62f0478b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f62e01aba40 con 0x56377063e7c0 2024-05-21T21:58:36.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.001+0000 7f62f0478b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f62e01ad580 con 0x563770642900 2024-05-21T21:58:36.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.001+0000 7f62f0478b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f62e01ac110 con 0x563770639390 2024-05-21T21:58:36.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.002+0000 7f62de000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563770642900 0x563770633740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:36.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.002+0000 7f62de000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563770642900 0x563770633740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:52822/0 (socket says 172.21.15.151:52822) 2024-05-21T21:58:36.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.002+0000 7f62de000700 1 -- 172.21.15.151:0/841232412 learned_addr learned my addr 172.21.15.151:0/841232412 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:58:36.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.002+0000 7f62dea00700 1 --2- 172.21.15.151:0/841232412 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56377063e7c0 0x56377063dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:36.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.003+0000 7f62de000700 1 --2- 172.21.15.151:0/841232412 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563770639390 0x563770638d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:36.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.003+0000 7f62dea00700 1 -- 172.21.15.151:0/841232412 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563770639390 msgr2=0x563770638d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:36.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.003+0000 7f62dea00700 1 --2- 172.21.15.151:0/841232412 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563770639390 0x563770638d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:36.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.003+0000 7f62dea00700 1 -- 172.21.15.151:0/841232412 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563770642900 msgr2=0x563770633740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:36.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.003+0000 7f62dea00700 1 --2- 172.21.15.151:0/841232412 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563770642900 0x563770633740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:36.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.003+0000 7f62dea00700 1 -- 172.21.15.151:0/841232412 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637706354c0 con 0x56377063e7c0 2024-05-21T21:58:36.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.004+0000 7f62dea00700 1 --2- 172.21.15.151:0/841232412 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56377063e7c0 0x56377063dd50 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f62d001e000 tx=0x7f62d0005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=391a5cc3ff17da87 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:58:36.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.005+0000 7f62cb400700 1 -- 172.21.15.151:0/841232412 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d0026040 con 0x56377063e7c0 2024-05-21T21:58:36.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.005+0000 7f62cb400700 1 -- 172.21.15.151:0/841232412 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62d0003c70 con 0x56377063e7c0 2024-05-21T21:58:36.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.005+0000 7f62cb400700 1 -- 172.21.15.151:0/841232412 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d000bca0 con 0x56377063e7c0 2024-05-21T21:58:36.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.006+0000 7f62f0478b80 1 -- 172.21.15.151:0/841232412 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56377063e7c0 msgr2=0x56377063dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:36.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.006+0000 7f62f0478b80 1 --2- 172.21.15.151:0/841232412 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56377063e7c0 0x56377063dd50 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f62d001e000 tx=0x7f62d0005040 comp rx=0 tx=0).stop 2024-05-21T21:58:36.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.007+0000 7f62f0478b80 1 -- 172.21.15.151:0/841232412 shutdown_connections 2024-05-21T21:58:36.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.007+0000 7f62f0478b80 1 --2- 172.21.15.151:0/841232412 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563770642900 0x563770633740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:36.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.007+0000 7f62f0478b80 1 --2- 172.21.15.151:0/841232412 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563770639390 0x563770638d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:36.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.007+0000 7f62f0478b80 1 --2- 172.21.15.151:0/841232412 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56377063e7c0 0x56377063dd50 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:36.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.007+0000 7f62f0478b80 1 -- 172.21.15.151:0/841232412 >> 172.21.15.151:0/841232412 conn(0x563770566be0 msgr2=0x563770640cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:58:36.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.009+0000 7f62f0478b80 1 -- 172.21.15.151:0/841232412 shutdown_connections 2024-05-21T21:58:36.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.009+0000 7f62f0478b80 1 -- 172.21.15.151:0/841232412 wait complete. 2024-05-21T21:58:36.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.010+0000 7f62f0478b80 1 -- start start 2024-05-21T21:58:36.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.010+0000 7f62f0478b80 1 Processor -- start 2024-05-21T21:58:36.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.012+0000 7f62f0478b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56377063e7c0 0x5637706c1610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:36.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.012+0000 7f62f0478b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56377065b560 0x5637706c1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:36.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.012+0000 7f62f0478b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5637706628c0 0x563770662d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:36.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.012+0000 7f62f0478b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f62e01aceb0 con 0x5637706628c0 2024-05-21T21:58:36.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.012+0000 7f62f0478b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f62e01ac7e0 con 0x56377063e7c0 2024-05-21T21:58:36.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.012+0000 7f62f0478b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f62e0115c30 con 0x56377065b560 2024-05-21T21:58:36.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.012+0000 7f62f0478b80 10 client.0 _ll_register_callbacks cb 0x563770530c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:58:36.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.013+0000 7f62de000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56377063e7c0 0x5637706c1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:36.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.014+0000 7f62dea00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56377065b560 0x5637706c1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:36.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.014+0000 7f62dea00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56377065b560 0x5637706c1b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:37122/0 (socket says 172.21.15.151:37122) 2024-05-21T21:58:36.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.014+0000 7f62dea00700 1 -- 172.21.15.151:0/407569417 learned_addr learned my addr 172.21.15.151:0/407569417 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:58:36.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.013+0000 7f62de000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56377063e7c0 0x5637706c1610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:52834/0 (socket says 172.21.15.151:52834) 2024-05-21T21:58:36.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.017+0000 7f62dea00700 1 -- 172.21.15.151:0/407569417 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56377063e7c0 msgr2=0x5637706c1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:36.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.017+0000 7f62dea00700 1 --2- 172.21.15.151:0/407569417 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56377063e7c0 0x5637706c1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:36.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.017+0000 7f62dea00700 1 -- 172.21.15.151:0/407569417 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5637706628c0 msgr2=0x563770662d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:36.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.017+0000 7f62de000700 1 --2- 172.21.15.151:0/407569417 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5637706628c0 0x563770662d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:36.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.017+0000 7f62dea00700 1 --2- 172.21.15.151:0/407569417 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5637706628c0 0x563770662d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:36.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.017+0000 7f62dea00700 1 -- 172.21.15.151:0/407569417 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563770663560 con 0x56377065b560 2024-05-21T21:58:36.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.018+0000 7f62de000700 1 --2- 172.21.15.151:0/407569417 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5637706628c0 0x563770662d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T21:58:36.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.019+0000 7f62dea00700 1 --2- 172.21.15.151:0/407569417 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56377065b560 0x5637706c1b50 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f62d002c040 tx=0x7f62d000b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14b70f09299c4a80 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:58:36.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.019+0000 7f62c8c00700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-21T21:58:36.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.019+0000 7f62c8c00700 1 -- 172.21.15.151:0/407569417 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d0026050 con 0x56377065b560 2024-05-21T21:58:36.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.019+0000 7f62c8c00700 1 -- 172.21.15.151:0/407569417 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f62d00037c0 con 0x56377065b560 2024-05-21T21:58:36.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.020+0000 7f62f0478b80 1 -- 172.21.15.151:0/407569417 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62d0003c70 con 0x56377065b560 2024-05-21T21:58:36.023 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.023+0000 7f62c8c00700 1 -- 172.21.15.151:0/407569417 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f62d00056e0 con 0x56377065b560 2024-05-21T21:58:36.023 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.023+0000 7f62f0478b80 10 client.25969 fetch_fsmap learned FSMap version 29 2024-05-21T21:58:36.023 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.023+0000 7f62f0478b80 1 -- 172.21.15.151:0/407569417 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56377064feb0 con 0x56377065b560 2024-05-21T21:58:36.024 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.023+0000 7f62c8c00700 1 -- 172.21.15.151:0/407569417 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62d0003c70 con 0x56377065b560 2024-05-21T21:58:36.025 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.024+0000 7f62c8c00700 1 -- 172.21.15.151:0/407569417 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f62d002c530 con 0x56377065b560 2024-05-21T21:58:36.026 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.024+0000 7f62f0478b80 10 client.25969 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T21:58:36.026 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.024+0000 7f62f0478b80 10 client.25969 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-05-21T21:58:36.026 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.024+0000 7f62f0478b80 20 client.25969 populate_metadata read hostname 'smithi151' 2024-05-21T21:58:36.026 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.024+0000 7f62f0478b80 1 --2- 172.21.15.151:0/407569417 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x563770650140 0x563770652530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:36.026 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.024+0000 7f62f0478b80 4 client.25969 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T21:58:36.026 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.024+0000 7f62f0478b80 1 -- 172.21.15.151:0/407569417 --> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563770592570 con 0x563770650140 2024-05-21T21:58:36.026 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.025+0000 7f62de000700 1 --2- 172.21.15.151:0/407569417 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x563770650140 0x563770652530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:36.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.025+0000 7f62de000700 1 --2- 172.21.15.151:0/407569417 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x563770650140 0x563770652530 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d2bd4879c6d0c9ec server_cookie=df8adfd7aeb3b23a in_seq=0 out_seq=0 2024-05-21T21:58:36.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.025+0000 7f62c8c00700 10 client.25969 ms_handle_connect on v2:172.21.15.164:6834/2340428470 2024-05-21T21:58:36.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.026+0000 7f62c8c00700 1 -- 172.21.15.151:0/407569417 <== mds.4 v2:172.21.15.164:6834/2340428470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563770592570 con 0x563770650140 2024-05-21T21:58:36.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.026+0000 7f62c8c00700 10 client.25969 handle_command_reply: tid=0 2024-05-21T21:58:36.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.026+0000 7f62f0478b80 1 client.25969 shutdown 2024-05-21T21:58:36.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.026+0000 7f62f0478b80 1 -- 172.21.15.151:0/407569417 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56377065b560 msgr2=0x5637706c1b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:36.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.026+0000 7f62f0478b80 1 --2- 172.21.15.151:0/407569417 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56377065b560 0x5637706c1b50 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f62d002c040 tx=0x7f62d000b790 comp rx=0 tx=0).stop 2024-05-21T21:58:36.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.027+0000 7f62f0478b80 1 -- 172.21.15.151:0/407569417 shutdown_connections 2024-05-21T21:58:36.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.027+0000 7f62f0478b80 1 --2- 172.21.15.151:0/407569417 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x563770650140 0x563770652530 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:36.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.027+0000 7f62f0478b80 1 --2- 172.21.15.151:0/407569417 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56377063e7c0 0x5637706c1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:36.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.027+0000 7f62f0478b80 1 --2- 172.21.15.151:0/407569417 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5637706628c0 0x563770662d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:36.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.027+0000 7f62f0478b80 1 --2- 172.21.15.151:0/407569417 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56377065b560 0x5637706c1b50 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:36.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.027+0000 7f62f0478b80 1 -- 172.21.15.151:0/407569417 >> 172.21.15.151:0/407569417 conn(0x563770566be0 msgr2=0x56377063d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:58:36.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.027+0000 7f62c8c00700 0 client.25969 ms_handle_reset on v2:172.21.15.164:6834/2340428470 2024-05-21T21:58:36.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.028+0000 7f62f0478b80 1 -- 172.21.15.151:0/407569417 shutdown_connections 2024-05-21T21:58:36.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.028+0000 7f62f0478b80 1 -- 172.21.15.151:0/407569417 wait complete. 2024-05-21T21:58:36.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.028+0000 7f62f0478b80 20 client.25969 trim_cache size 0 max 16384 2024-05-21T21:58:36.031 INFO:teuthology.orchestra.run.smithi151.stdout:[] 2024-05-21T21:58:36.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.030+0000 7f62dfe00700 1 -- 172.21.15.151:0/3041403058 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f62c40b29f0 msgr2=0x7f62c40b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:36.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.030+0000 7f62dfe00700 1 --2- 172.21.15.151:0/3041403058 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f62c40b29f0 0x7f62c40b4eb0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f62d8009c10 tx=0x7f62d8009040 comp rx=0 tx=0).stop 2024-05-21T21:58:36.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.030+0000 7f62dfe00700 1 -- 172.21.15.151:0/3041403058 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62e0109540 msgr2=0x7f62e019bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:36.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.030+0000 7f62dfe00700 1 --2- 172.21.15.151:0/3041403058 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62e0109540 0x7f62e019bb60 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f62d800ef30 tx=0x7f62d800c450 comp rx=0 tx=0).stop 2024-05-21T21:58:36.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.030+0000 7f62dfe00700 1 -- 172.21.15.151:0/3041403058 shutdown_connections 2024-05-21T21:58:36.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.030+0000 7f62dfe00700 1 --2- 172.21.15.151:0/3041403058 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f62c40b29f0 0x7f62c40b4eb0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:36.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.030+0000 7f62dfe00700 1 --2- 172.21.15.151:0/3041403058 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62e0103490 0x7f62e01a1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:36.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.030+0000 7f62dfe00700 1 --2- 172.21.15.151:0/3041403058 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62e0109540 0x7f62e019bb60 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:36.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.030+0000 7f62dfe00700 1 --2- 172.21.15.151:0/3041403058 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f62e0103e50 0x7f62e01a1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:36.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.030+0000 7f62dfe00700 1 -- 172.21.15.151:0/3041403058 >> 172.21.15.151:0/3041403058 conn(0x7f62e00fefc0 msgr2=0x7f62e01006d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:58:36.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.031+0000 7f62dfe00700 1 -- 172.21.15.151:0/3041403058 shutdown_connections 2024-05-21T21:58:36.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:36.031+0000 7f62dfe00700 1 -- 172.21.15.151:0/3041403058 wait complete. 2024-05-21T21:58:36.872 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-21T21:58:37.307 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:58:37.684 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:58:37 smithi151 ceph-mon[31623]: pgmap v93: 97 pgs: 97 active+clean; 75 MiB data, 579 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.6 MiB/s wr, 13 op/s 2024-05-21T21:58:37.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:58:37 smithi184 ceph-mon[36942]: pgmap v93: 97 pgs: 97 active+clean; 75 MiB data, 579 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.6 MiB/s wr, 13 op/s 2024-05-21T21:58:37.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:58:37 smithi164 ceph-mon[36269]: pgmap v93: 97 pgs: 97 active+clean; 75 MiB data, 579 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.6 MiB/s wr, 13 op/s 2024-05-21T21:58:39.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.460+0000 7f7b76000700 1 -- 172.21.15.151:0/2352834043 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b70103f50 msgr2=0x7f7b70110430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:39.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.460+0000 7f7b76000700 1 --2- 172.21.15.151:0/2352834043 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b70103f50 0x7f7b70110430 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7b6800b0a0 tx=0x7f7b6800b3b0 comp rx=0 tx=0).stop 2024-05-21T21:58:39.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.460+0000 7f7b76000700 1 -- 172.21.15.151:0/2352834043 shutdown_connections 2024-05-21T21:58:39.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.460+0000 7f7b76000700 1 --2- 172.21.15.151:0/2352834043 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b701096a0 0x7f7b70109a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:39.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.460+0000 7f7b76000700 1 --2- 172.21.15.151:0/2352834043 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b70103f50 0x7f7b70110430 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:39.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.460+0000 7f7b76000700 1 --2- 172.21.15.151:0/2352834043 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7b70103590 0x7f7b70103a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:39.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.460+0000 7f7b76000700 1 -- 172.21.15.151:0/2352834043 >> 172.21.15.151:0/2352834043 conn(0x7f7b7007a6f0 msgr2=0x7f7b7007ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:58:39.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.460+0000 7f7b76000700 1 -- 172.21.15.151:0/2352834043 shutdown_connections 2024-05-21T21:58:39.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.460+0000 7f7b76000700 1 -- 172.21.15.151:0/2352834043 wait complete. 2024-05-21T21:58:39.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.461+0000 7f7b76000700 1 Processor -- start 2024-05-21T21:58:39.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.462+0000 7f7b76000700 1 -- start start 2024-05-21T21:58:39.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.462+0000 7f7b76000700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b70103590 0x7f7b7019bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:39.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.462+0000 7f7b76000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7b70103f50 0x7f7b7019c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:39.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.462+0000 7f7b76000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b701096a0 0x7f7b701a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:39.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.462+0000 7f7b76000700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7b70113770 con 0x7f7b70103f50 2024-05-21T21:58:39.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.462+0000 7f7b76000700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7b701135f0 con 0x7f7b701096a0 2024-05-21T21:58:39.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.462+0000 7f7b76000700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7b701138f0 con 0x7f7b70103590 2024-05-21T21:58:39.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.462+0000 7f7b6ea00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7b70103f50 0x7f7b7019c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:39.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.463+0000 7f7b6fe00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b701096a0 0x7f7b701a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:39.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.463+0000 7f7b6f400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b70103590 0x7f7b7019bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:39.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.463+0000 7f7b6ea00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7b70103f50 0x7f7b7019c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:34122/0 (socket says 172.21.15.151:34122) 2024-05-21T21:58:39.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.463+0000 7f7b6ea00700 1 -- 172.21.15.151:0/3481601463 learned_addr learned my addr 172.21.15.151:0/3481601463 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:58:39.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.463+0000 7f7b6fe00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b701096a0 0x7f7b701a05e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:52856/0 (socket says 172.21.15.151:52856) 2024-05-21T21:58:39.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.463+0000 7f7b6fe00700 1 -- 172.21.15.151:0/3481601463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b70103590 msgr2=0x7f7b7019bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:39.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.463+0000 7f7b6fe00700 1 --2- 172.21.15.151:0/3481601463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b70103590 0x7f7b7019bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:39.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.463+0000 7f7b6fe00700 1 -- 172.21.15.151:0/3481601463 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7b70103f50 msgr2=0x7f7b7019c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:39.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.463+0000 7f7b6fe00700 1 --2- 172.21.15.151:0/3481601463 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7b70103f50 0x7f7b7019c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:39.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.463+0000 7f7b6fe00700 1 -- 172.21.15.151:0/3481601463 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b701a0d20 con 0x7f7b701096a0 2024-05-21T21:58:39.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.463+0000 7f7b6f400700 1 --2- 172.21.15.151:0/3481601463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b70103590 0x7f7b7019bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:58:39.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.463+0000 7f7b6ea00700 1 --2- 172.21.15.151:0/3481601463 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7b70103f50 0x7f7b7019c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:58:39.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.463+0000 7f7b6fe00700 1 --2- 172.21.15.151:0/3481601463 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b701096a0 0x7f7b701a05e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7b6800b070 tx=0x7f7b68004d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:58:39.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.464+0000 7f7b67e00700 1 -- 172.21.15.151:0/3481601463 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b68021070 con 0x7f7b701096a0 2024-05-21T21:58:39.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.464+0000 7f7b76000700 1 -- 172.21.15.151:0/3481601463 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b701a0fb0 con 0x7f7b701096a0 2024-05-21T21:58:39.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.464+0000 7f7b67e00700 1 -- 172.21.15.151:0/3481601463 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b68007e30 con 0x7f7b701096a0 2024-05-21T21:58:39.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.464+0000 7f7b67e00700 1 -- 172.21.15.151:0/3481601463 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b680044c0 con 0x7f7b701096a0 2024-05-21T21:58:39.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.464+0000 7f7b76000700 1 -- 172.21.15.151:0/3481601463 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b701a8890 con 0x7f7b701096a0 2024-05-21T21:58:39.467 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.467+0000 7f7b67e00700 1 -- 172.21.15.151:0/3481601463 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7f7b68004660 con 0x7f7b701096a0 2024-05-21T21:58:39.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.468+0000 7f7b808adb80 1 Processor -- start 2024-05-21T21:58:39.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.468+0000 7f7b808adb80 1 -- start start 2024-05-21T21:58:39.470 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.469+0000 7f7b808adb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560e0daa8eb0 0x560e0daa9290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:39.470 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.469+0000 7f7b808adb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560e0daa97d0 0x560e0daa3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:39.470 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.469+0000 7f7b808adb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e0daa3880 0x560e0daa3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:39.470 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.469+0000 7f7b808adb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x560e0da01e40 con 0x560e0daa3880 2024-05-21T21:58:39.470 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.469+0000 7f7b67e00700 1 --2- 172.21.15.151:0/3481601463 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f7b540b29f0 0x7f7b540b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:39.470 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.469+0000 7f7b808adb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x560e0d9f19e0 con 0x560e0daa97d0 2024-05-21T21:58:39.470 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.469+0000 7f7b808adb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x560e0d9f5a20 con 0x560e0daa8eb0 2024-05-21T21:58:39.470 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.469+0000 7f7b67e00700 1 -- 172.21.15.151:0/3481601463 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f7b6800f020 con 0x7f7b701096a0 2024-05-21T21:58:39.470 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.469+0000 7f7b6ea00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560e0daa8eb0 0x560e0daa9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:39.470 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.469+0000 7f7b6ea00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560e0daa8eb0 0x560e0daa9290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:37156/0 (socket says 172.21.15.151:37156) 2024-05-21T21:58:39.470 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.469+0000 7f7b6ea00700 1 -- 172.21.15.151:0/1354246888 learned_addr learned my addr 172.21.15.151:0/1354246888 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:58:39.470 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.469+0000 7f7b6ea00700 1 --2- 172.21.15.151:0/3481601463 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f7b540b29f0 0x7f7b540b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:39.470 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.469+0000 7f7b6f400700 1 --2- 172.21.15.151:0/1354246888 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560e0daa97d0 0x560e0daa3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:39.471 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.470+0000 7f7b6fe00700 1 --2- 172.21.15.151:0/1354246888 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e0daa3880 0x560e0daa3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:39.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.470+0000 7f7b6ea00700 1 -- 172.21.15.151:0/1354246888 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560e0daa97d0 msgr2=0x560e0daa3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:39.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.470+0000 7f7b6ea00700 1 --2- 172.21.15.151:0/1354246888 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560e0daa97d0 0x560e0daa3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:39.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.470+0000 7f7b6ea00700 1 -- 172.21.15.151:0/1354246888 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e0daa3880 msgr2=0x560e0daa3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:39.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.470+0000 7f7b6ea00700 1 --2- 172.21.15.151:0/1354246888 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e0daa3880 0x560e0daa3ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:39.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.470+0000 7f7b6ea00700 1 -- 172.21.15.151:0/1354246888 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e0d9ba420 con 0x560e0daa8eb0 2024-05-21T21:58:39.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.470+0000 7f7b6ea00700 1 --2- 172.21.15.151:0/1354246888 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560e0daa8eb0 0x560e0daa9290 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c00f3c0 tx=0x7f7b5c00f6d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d4c6c169367635b server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:58:39.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.471+0000 7f7b6ea00700 1 --2- 172.21.15.151:0/3481601463 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f7b540b29f0 0x7f7b540b4eb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c00fa20 tx=0x7f7b5c018000 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:58:39.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.471+0000 7f7b64c00700 1 -- 172.21.15.151:0/1354246888 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b5c022070 con 0x560e0daa8eb0 2024-05-21T21:58:39.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.471+0000 7f7b64c00700 1 -- 172.21.15.151:0/1354246888 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b5c00ec00 con 0x560e0daa8eb0 2024-05-21T21:58:39.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.471+0000 7f7b64c00700 1 -- 172.21.15.151:0/1354246888 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b5c026be0 con 0x560e0daa8eb0 2024-05-21T21:58:39.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.472+0000 7f7b808adb80 1 -- 172.21.15.151:0/1354246888 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560e0daa8eb0 msgr2=0x560e0daa9290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:39.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.472+0000 7f7b808adb80 1 --2- 172.21.15.151:0/1354246888 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560e0daa8eb0 0x560e0daa9290 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c00f3c0 tx=0x7f7b5c00f6d0 comp rx=0 tx=0).stop 2024-05-21T21:58:39.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.472+0000 7f7b808adb80 1 -- 172.21.15.151:0/1354246888 shutdown_connections 2024-05-21T21:58:39.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.472+0000 7f7b808adb80 1 --2- 172.21.15.151:0/1354246888 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e0daa3880 0x560e0daa3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:39.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.472+0000 7f7b808adb80 1 --2- 172.21.15.151:0/1354246888 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560e0daa8eb0 0x560e0daa9290 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:39.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.472+0000 7f7b808adb80 1 --2- 172.21.15.151:0/1354246888 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560e0daa97d0 0x560e0daa3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:39.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.472+0000 7f7b808adb80 1 -- 172.21.15.151:0/1354246888 >> 172.21.15.151:0/1354246888 conn(0x560e0d9d1be0 msgr2=0x560e0da9e010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:58:39.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.472+0000 7f7b808adb80 1 -- 172.21.15.151:0/1354246888 shutdown_connections 2024-05-21T21:58:39.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.472+0000 7f7b808adb80 1 -- 172.21.15.151:0/1354246888 wait complete. 2024-05-21T21:58:39.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.473+0000 7f7b808adb80 1 -- start start 2024-05-21T21:58:39.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.473+0000 7f7b808adb80 1 Processor -- start 2024-05-21T21:58:39.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.473+0000 7f7b808adb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e0daa3880 0x560e0daaf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:39.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.473+0000 7f7b808adb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560e0daa8eb0 0x560e0daac8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:39.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.473+0000 7f7b808adb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560e0daa97d0 0x560e0daace30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:39.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.473+0000 7f7b808adb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x560e0d8f7fa0 con 0x560e0daa3880 2024-05-21T21:58:39.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.473+0000 7f7b808adb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x560e0d992760 con 0x560e0daa97d0 2024-05-21T21:58:39.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.473+0000 7f7b808adb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x560e0dab0180 con 0x560e0daa8eb0 2024-05-21T21:58:39.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.473+0000 7f7b808adb80 10 client.0 _ll_register_callbacks cb 0x560e0d99bc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:58:39.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.474+0000 7f7b6ea00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560e0daa8eb0 0x560e0daac8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:39.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.474+0000 7f7b6f400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e0daa3880 0x560e0daaf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:39.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.474+0000 7f7b6ea00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560e0daa8eb0 0x560e0daac8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:37158/0 (socket says 172.21.15.151:37158) 2024-05-21T21:58:39.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.474+0000 7f7b6f400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e0daa3880 0x560e0daaf550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:34134/0 (socket says 172.21.15.151:34134) 2024-05-21T21:58:39.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.474+0000 7f7b6ea00700 1 -- 172.21.15.151:0/3447203342 learned_addr learned my addr 172.21.15.151:0/3447203342 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:58:39.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.474+0000 7f7b6fe00700 1 --2- 172.21.15.151:0/3447203342 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560e0daa97d0 0x560e0daace30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:39.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.474+0000 7f7b6ea00700 1 -- 172.21.15.151:0/3447203342 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560e0daa97d0 msgr2=0x560e0daace30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:39.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.474+0000 7f7b6ea00700 1 --2- 172.21.15.151:0/3447203342 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560e0daa97d0 0x560e0daace30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:39.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.474+0000 7f7b6ea00700 1 -- 172.21.15.151:0/3447203342 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e0daa3880 msgr2=0x560e0daaf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:39.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.474+0000 7f7b6ea00700 1 --2- 172.21.15.151:0/3447203342 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e0daa3880 0x560e0daaf550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:39.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.474+0000 7f7b6ea00700 1 -- 172.21.15.151:0/3447203342 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e0d9418d0 con 0x560e0daa8eb0 2024-05-21T21:58:39.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.474+0000 7f7b6ea00700 1 --2- 172.21.15.151:0/3447203342 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560e0daa8eb0 0x560e0daac8f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c009d30 tx=0x7f7b5c026580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b8958725df6d88e server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:58:39.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.474+0000 7f7b59600700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-21T21:58:39.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.475+0000 7f7b59600700 1 -- 172.21.15.151:0/3447203342 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b5c034060 con 0x560e0daa8eb0 2024-05-21T21:58:39.477 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.475+0000 7f7b808adb80 1 -- 172.21.15.151:0/3447203342 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b5c00ec00 con 0x560e0daa8eb0 2024-05-21T21:58:39.477 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.475+0000 7f7b59600700 1 -- 172.21.15.151:0/3447203342 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f7b5c005110 con 0x560e0daa8eb0 2024-05-21T21:58:39.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.475+0000 7f7b59600700 1 -- 172.21.15.151:0/3447203342 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7b5c022070 con 0x560e0daa8eb0 2024-05-21T21:58:39.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.475+0000 7f7b808adb80 10 client.25981 fetch_fsmap learned FSMap version 29 2024-05-21T21:58:39.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.475+0000 7f7b808adb80 1 -- 172.21.15.151:0/3447203342 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560e0d9c2060 con 0x560e0daa8eb0 2024-05-21T21:58:39.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.475+0000 7f7b59600700 1 -- 172.21.15.151:0/3447203342 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b5c00ec00 con 0x560e0daa8eb0 2024-05-21T21:58:39.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.476+0000 7f7b59600700 1 -- 172.21.15.151:0/3447203342 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f7b5c003680 con 0x560e0daa8eb0 2024-05-21T21:58:39.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.476+0000 7f7b808adb80 10 client.25981 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T21:58:39.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.476+0000 7f7b808adb80 10 client.25981 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-21T21:58:39.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.476+0000 7f7b808adb80 20 client.25981 populate_metadata read hostname 'smithi151' 2024-05-21T21:58:39.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.476+0000 7f7b808adb80 1 --2- 172.21.15.151:0/3447203342 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x560e0dabcd70 0x560e0dabf160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:39.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.476+0000 7f7b808adb80 4 client.25981 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T21:58:39.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.476+0000 7f7b808adb80 1 -- 172.21.15.151:0/3447203342 --> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e0da03d90 con 0x560e0dabcd70 2024-05-21T21:58:39.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.476+0000 7f7b6f400700 1 --2- 172.21.15.151:0/3447203342 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x560e0dabcd70 0x560e0dabf160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:39.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.477+0000 7f7b59600700 10 client.25981 ms_handle_connect on v2:172.21.15.184:6836/2988194888 2024-05-21T21:58:39.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.477+0000 7f7b6f400700 1 --2- 172.21.15.151:0/3447203342 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x560e0dabcd70 0x560e0dabf160 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4222d3a77060edb5 server_cookie=c0837b686e2b3a05 in_seq=0 out_seq=0 2024-05-21T21:58:39.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.478+0000 7f7b59600700 1 -- 172.21.15.151:0/3447203342 <== mds.3 v2:172.21.15.184:6836/2988194888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x560e0da03d90 con 0x560e0dabcd70 2024-05-21T21:58:39.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.478+0000 7f7b59600700 10 client.25981 handle_command_reply: tid=0 2024-05-21T21:58:39.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.478+0000 7f7b808adb80 1 client.25981 shutdown 2024-05-21T21:58:39.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.478+0000 7f7b808adb80 1 -- 172.21.15.151:0/3447203342 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560e0daa8eb0 msgr2=0x560e0daac8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:39.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.478+0000 7f7b808adb80 1 --2- 172.21.15.151:0/3447203342 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560e0daa8eb0 0x560e0daac8f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c009d30 tx=0x7f7b5c026580 comp rx=0 tx=0).stop 2024-05-21T21:58:39.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.478+0000 7f7b808adb80 1 -- 172.21.15.151:0/3447203342 shutdown_connections 2024-05-21T21:58:39.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.478+0000 7f7b808adb80 1 --2- 172.21.15.151:0/3447203342 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x560e0dabcd70 0x560e0dabf160 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:39.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.478+0000 7f7b808adb80 1 --2- 172.21.15.151:0/3447203342 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560e0daa97d0 0x560e0daace30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:39.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.478+0000 7f7b808adb80 1 --2- 172.21.15.151:0/3447203342 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e0daa3880 0x560e0daaf550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:39.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.478+0000 7f7b59600700 0 client.25981 ms_handle_reset on v2:172.21.15.184:6836/2988194888 2024-05-21T21:58:39.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.478+0000 7f7b808adb80 1 --2- 172.21.15.151:0/3447203342 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560e0daa8eb0 0x560e0daac8f0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:39.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.478+0000 7f7b808adb80 1 -- 172.21.15.151:0/3447203342 >> 172.21.15.151:0/3447203342 conn(0x560e0d9d1be0 msgr2=0x560e0daa2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:58:39.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.478+0000 7f7b808adb80 1 -- 172.21.15.151:0/3447203342 shutdown_connections 2024-05-21T21:58:39.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.478+0000 7f7b808adb80 1 -- 172.21.15.151:0/3447203342 wait complete. 2024-05-21T21:58:39.481 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.478+0000 7f7b808adb80 20 client.25981 trim_cache size 0 max 16384 2024-05-21T21:58:39.495 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.495+0000 7f7b808adb80 1 Processor -- start 2024-05-21T21:58:39.495 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.495+0000 7f7b808adb80 1 -- start start 2024-05-21T21:58:39.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.495+0000 7f7b808adb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560e0daa3880 0x560e0daa3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:39.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.495+0000 7f7b808adb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e0daa8eb0 0x560e0daa9290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:39.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.495+0000 7f7b808adb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560e0daa97d0 0x560e0daa69c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:39.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.495+0000 7f7b808adb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7b701afe40 con 0x560e0daa8eb0 2024-05-21T21:58:39.497 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.495+0000 7f7b808adb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7b701b1980 con 0x560e0daa3880 2024-05-21T21:58:39.497 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.495+0000 7f7b808adb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7b701b0510 con 0x560e0daa97d0 2024-05-21T21:58:39.497 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.496+0000 7f7b6ea00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e0daa8eb0 0x560e0daa9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:39.497 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.496+0000 7f7b6ea00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e0daa8eb0 0x560e0daa9290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:34146/0 (socket says 172.21.15.151:34146) 2024-05-21T21:58:39.497 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.496+0000 7f7b6ea00700 1 -- 172.21.15.151:0/794726548 learned_addr learned my addr 172.21.15.151:0/794726548 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:58:39.497 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.496+0000 7f7b6f400700 1 --2- 172.21.15.151:0/794726548 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560e0daa3880 0x560e0daa3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:39.497 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.496+0000 7f7b6fe00700 1 --2- 172.21.15.151:0/794726548 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560e0daa97d0 0x560e0daa69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:39.497 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.496+0000 7f7b6ea00700 1 -- 172.21.15.151:0/794726548 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560e0daa97d0 msgr2=0x560e0daa69c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:39.497 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.496+0000 7f7b6ea00700 1 --2- 172.21.15.151:0/794726548 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560e0daa97d0 0x560e0daa69c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:39.497 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.496+0000 7f7b6ea00700 1 -- 172.21.15.151:0/794726548 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560e0daa3880 msgr2=0x560e0daa3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:39.497 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.496+0000 7f7b6ea00700 1 --2- 172.21.15.151:0/794726548 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560e0daa3880 0x560e0daa3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:39.497 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.496+0000 7f7b6ea00700 1 -- 172.21.15.151:0/794726548 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e0daaae60 con 0x560e0daa8eb0 2024-05-21T21:58:39.498 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.497+0000 7f7b6ea00700 1 --2- 172.21.15.151:0/794726548 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e0daa8eb0 0x560e0daa9290 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c005e10 tx=0x7f7b5c026670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=904180a0b7c3af68 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:58:39.498 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.497+0000 7f7b5be00700 1 -- 172.21.15.151:0/794726548 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b5c034040 con 0x560e0daa8eb0 2024-05-21T21:58:39.498 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.497+0000 7f7b5be00700 1 -- 172.21.15.151:0/794726548 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b5c011520 con 0x560e0daa8eb0 2024-05-21T21:58:39.499 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.497+0000 7f7b5be00700 1 -- 172.21.15.151:0/794726548 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b5c005620 con 0x560e0daa8eb0 2024-05-21T21:58:39.499 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.498+0000 7f7b808adb80 1 -- 172.21.15.151:0/794726548 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e0daa8eb0 msgr2=0x560e0daa9290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:39.499 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.498+0000 7f7b808adb80 1 --2- 172.21.15.151:0/794726548 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e0daa8eb0 0x560e0daa9290 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c005e10 tx=0x7f7b5c026670 comp rx=0 tx=0).stop 2024-05-21T21:58:39.499 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.498+0000 7f7b808adb80 1 -- 172.21.15.151:0/794726548 shutdown_connections 2024-05-21T21:58:39.499 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.498+0000 7f7b808adb80 1 --2- 172.21.15.151:0/794726548 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560e0daa3880 0x560e0daa3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:39.499 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.498+0000 7f7b808adb80 1 --2- 172.21.15.151:0/794726548 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560e0daa97d0 0x560e0daa69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:39.499 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.498+0000 7f7b808adb80 1 --2- 172.21.15.151:0/794726548 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e0daa8eb0 0x560e0daa9290 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:39.499 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.498+0000 7f7b808adb80 1 -- 172.21.15.151:0/794726548 >> 172.21.15.151:0/794726548 conn(0x560e0d9d1be0 msgr2=0x560e0daab1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:58:39.499 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.498+0000 7f7b808adb80 1 -- 172.21.15.151:0/794726548 shutdown_connections 2024-05-21T21:58:39.499 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.498+0000 7f7b808adb80 1 -- 172.21.15.151:0/794726548 wait complete. 2024-05-21T21:58:39.499 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.498+0000 7f7b808adb80 1 -- start start 2024-05-21T21:58:39.500 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.499+0000 7f7b808adb80 1 Processor -- start 2024-05-21T21:58:39.500 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.499+0000 7f7b808adb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560e0daa3880 0x560e0dab59f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:39.501 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.499+0000 7f7b808adb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560e0daa97d0 0x560e0dac3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:39.501 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.499+0000 7f7b808adb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e0dac4420 0x560e0dac48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:39.501 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.499+0000 7f7b808adb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7b701b12b0 con 0x560e0dac4420 2024-05-21T21:58:39.501 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.499+0000 7f7b808adb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7b701b0be0 con 0x560e0daa97d0 2024-05-21T21:58:39.501 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.499+0000 7f7b808adb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7b701adde0 con 0x560e0daa3880 2024-05-21T21:58:39.501 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.499+0000 7f7b808adb80 10 client.0 _ll_register_callbacks cb 0x560e0d99bc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:58:39.501 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.500+0000 7f7b6f400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560e0daa3880 0x560e0dab59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:39.501 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.500+0000 7f7b6ea00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560e0daa97d0 0x560e0dac3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:39.501 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.500+0000 7f7b6f400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560e0daa3880 0x560e0dab59f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:37180/0 (socket says 172.21.15.151:37180) 2024-05-21T21:58:39.501 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.500+0000 7f7b6f400700 1 -- 172.21.15.151:0/3345862515 learned_addr learned my addr 172.21.15.151:0/3345862515 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:58:39.501 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.500+0000 7f7b6fe00700 1 --2- 172.21.15.151:0/3345862515 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e0dac4420 0x560e0dac48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:39.501 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.500+0000 7f7b6ea00700 1 -- 172.21.15.151:0/3345862515 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560e0daa3880 msgr2=0x560e0dab59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:39.501 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.500+0000 7f7b6ea00700 1 --2- 172.21.15.151:0/3345862515 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560e0daa3880 0x560e0dab59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:39.501 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.500+0000 7f7b6ea00700 1 -- 172.21.15.151:0/3345862515 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e0dac4420 msgr2=0x560e0dac48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:39.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.500+0000 7f7b6ea00700 1 --2- 172.21.15.151:0/3345862515 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e0dac4420 0x560e0dac48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:39.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.500+0000 7f7b6ea00700 1 -- 172.21.15.151:0/3345862515 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e0db26750 con 0x560e0daa97d0 2024-05-21T21:58:39.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.500+0000 7f7b6f400700 1 --2- 172.21.15.151:0/3345862515 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560e0daa3880 0x560e0dab59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:58:39.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.500+0000 7f7b6fe00700 1 --2- 172.21.15.151:0/3345862515 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e0dac4420 0x560e0dac48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T21:58:39.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.500+0000 7f7b6ea00700 1 --2- 172.21.15.151:0/3345862515 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560e0daa97d0 0x560e0dac3ee0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c004000 tx=0x7f7b5c005ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e2635e4a4a0d84f server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:58:39.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.500+0000 7f7b59600700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-21T21:58:39.504 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.501+0000 7f7b59600700 1 -- 172.21.15.151:0/3345862515 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b5c011020 con 0x560e0daa97d0 2024-05-21T21:58:39.504 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.501+0000 7f7b59600700 1 -- 172.21.15.151:0/3345862515 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f7b5c0111c0 con 0x560e0daa97d0 2024-05-21T21:58:39.504 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.501+0000 7f7b808adb80 1 -- 172.21.15.151:0/3345862515 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b5c011520 con 0x560e0daa97d0 2024-05-21T21:58:39.504 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.501+0000 7f7b59600700 1 -- 172.21.15.151:0/3345862515 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7b5c005110 con 0x560e0daa97d0 2024-05-21T21:58:39.504 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.501+0000 7f7b808adb80 10 client.25982 fetch_fsmap learned FSMap version 29 2024-05-21T21:58:39.504 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.501+0000 7f7b808adb80 1 -- 172.21.15.151:0/3345862515 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560e0db27a10 con 0x560e0daa97d0 2024-05-21T21:58:39.504 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.501+0000 7f7b59600700 1 -- 172.21.15.151:0/3345862515 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b5c011520 con 0x560e0daa97d0 2024-05-21T21:58:39.504 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.502+0000 7f7b59600700 1 -- 172.21.15.151:0/3345862515 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f7b5c041450 con 0x560e0daa97d0 2024-05-21T21:58:39.504 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.502+0000 7f7b808adb80 10 client.25982 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T21:58:39.504 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.502+0000 7f7b808adb80 10 client.25982 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-21T21:58:39.504 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.502+0000 7f7b808adb80 20 client.25982 populate_metadata read hostname 'smithi151' 2024-05-21T21:58:39.505 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.502+0000 7f7b808adb80 1 --2- 172.21.15.151:0/3345862515 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x560e0db27fb0 0x560e0db2a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:39.505 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.502+0000 7f7b808adb80 4 client.25982 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T21:58:39.505 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.502+0000 7f7b808adb80 1 -- 172.21.15.151:0/3345862515 --> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560e0d9fd570 con 0x560e0db27fb0 2024-05-21T21:58:39.505 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.502+0000 7f7b6f400700 1 --2- 172.21.15.151:0/3345862515 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x560e0db27fb0 0x560e0db2a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:39.505 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.503+0000 7f7b6f400700 1 --2- 172.21.15.151:0/3345862515 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x560e0db27fb0 0x560e0db2a3a0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=905178f56874f70b server_cookie=b2c2b7e3d0659876 in_seq=0 out_seq=0 2024-05-21T21:58:39.505 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.503+0000 7f7b59600700 10 client.25982 ms_handle_connect on v2:172.21.15.184:6836/2988194888 2024-05-21T21:58:39.505 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.503+0000 7f7b59600700 1 -- 172.21.15.151:0/3345862515 <== mds.3 v2:172.21.15.184:6836/2988194888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560e0d9fd570 con 0x560e0db27fb0 2024-05-21T21:58:39.505 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.503+0000 7f7b59600700 10 client.25982 handle_command_reply: tid=0 2024-05-21T21:58:39.505 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.503+0000 7f7b808adb80 1 client.25982 shutdown 2024-05-21T21:58:39.505 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.503+0000 7f7b808adb80 1 -- 172.21.15.151:0/3345862515 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560e0daa97d0 msgr2=0x560e0dac3ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:39.505 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.503+0000 7f7b808adb80 1 --2- 172.21.15.151:0/3345862515 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560e0daa97d0 0x560e0dac3ee0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c004000 tx=0x7f7b5c005ce0 comp rx=0 tx=0).stop 2024-05-21T21:58:39.505 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.503+0000 7f7b808adb80 1 -- 172.21.15.151:0/3345862515 shutdown_connections 2024-05-21T21:58:39.505 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.503+0000 7f7b808adb80 1 --2- 172.21.15.151:0/3345862515 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x560e0db27fb0 0x560e0db2a3a0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:39.505 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.503+0000 7f7b808adb80 1 --2- 172.21.15.151:0/3345862515 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e0dac4420 0x560e0dac48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:39.505 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.503+0000 7f7b808adb80 1 --2- 172.21.15.151:0/3345862515 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560e0daa3880 0x560e0dab59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:39.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.503+0000 7f7b808adb80 1 --2- 172.21.15.151:0/3345862515 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560e0daa97d0 0x560e0dac3ee0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:39.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.503+0000 7f7b808adb80 1 -- 172.21.15.151:0/3345862515 >> 172.21.15.151:0/3345862515 conn(0x560e0d9d1be0 msgr2=0x560e0daac6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:58:39.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.503+0000 7f7b59600700 0 client.25982 ms_handle_reset on v2:172.21.15.184:6836/2988194888 2024-05-21T21:58:39.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.503+0000 7f7b808adb80 1 -- 172.21.15.151:0/3345862515 shutdown_connections 2024-05-21T21:58:39.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.503+0000 7f7b808adb80 1 -- 172.21.15.151:0/3345862515 wait complete. 2024-05-21T21:58:39.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.504+0000 7f7b808adb80 20 client.25982 trim_cache size 0 max 16384 2024-05-21T21:58:39.506 INFO:teuthology.orchestra.run.smithi151.stdout:[] 2024-05-21T21:58:39.507 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.504+0000 7f7b76000700 1 -- 172.21.15.151:0/3481601463 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f7b540b29f0 msgr2=0x7f7b540b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:39.507 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.504+0000 7f7b76000700 1 --2- 172.21.15.151:0/3481601463 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f7b540b29f0 0x7f7b540b4eb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c00fa20 tx=0x7f7b5c018000 comp rx=0 tx=0).stop 2024-05-21T21:58:39.507 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.504+0000 7f7b76000700 1 -- 172.21.15.151:0/3481601463 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b701096a0 msgr2=0x7f7b701a05e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:39.507 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.504+0000 7f7b76000700 1 --2- 172.21.15.151:0/3481601463 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b701096a0 0x7f7b701a05e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7b6800b070 tx=0x7f7b68004d00 comp rx=0 tx=0).stop 2024-05-21T21:58:39.507 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.504+0000 7f7b76000700 1 -- 172.21.15.151:0/3481601463 shutdown_connections 2024-05-21T21:58:39.507 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.504+0000 7f7b76000700 1 --2- 172.21.15.151:0/3481601463 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f7b540b29f0 0x7f7b540b4eb0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:39.507 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.504+0000 7f7b76000700 1 --2- 172.21.15.151:0/3481601463 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b701096a0 0x7f7b701a05e0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:39.508 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.504+0000 7f7b76000700 1 --2- 172.21.15.151:0/3481601463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b70103590 0x7f7b7019bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:39.508 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.504+0000 7f7b76000700 1 --2- 172.21.15.151:0/3481601463 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7b70103f50 0x7f7b7019c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:39.508 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.504+0000 7f7b76000700 1 -- 172.21.15.151:0/3481601463 >> 172.21.15.151:0/3481601463 conn(0x7f7b7007a6f0 msgr2=0x7f7b70101ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:58:39.508 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.504+0000 7f7b76000700 1 -- 172.21.15.151:0/3481601463 shutdown_connections 2024-05-21T21:58:39.508 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:39.505+0000 7f7b76000700 1 -- 172.21.15.151:0/3481601463 wait complete. 2024-05-21T21:58:39.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:58:39 smithi151 ceph-mon[31623]: pgmap v94: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 28 MiB/s wr, 25 op/s 2024-05-21T21:58:39.735 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:58:39 smithi184 ceph-mon[36942]: pgmap v94: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 28 MiB/s wr, 25 op/s 2024-05-21T21:58:39.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:58:39 smithi164 ceph-mon[36269]: pgmap v94: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 28 MiB/s wr, 25 op/s 2024-05-21T21:58:40.447 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-21T21:58:40.865 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:58:41.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:58:41 smithi151 ceph-mon[31623]: pgmap v95: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 28 MiB/s wr, 19 op/s 2024-05-21T21:58:41.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:58:41 smithi184 ceph-mon[36942]: pgmap v95: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 28 MiB/s wr, 19 op/s 2024-05-21T21:58:41.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:58:41 smithi164 ceph-mon[36269]: pgmap v95: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 28 MiB/s wr, 19 op/s 2024-05-21T21:58:43.067 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.066+0000 7f079ea00700 1 -- 172.21.15.151:0/341666874 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0798103f80 msgr2=0x7f0798110410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:43.068 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.066+0000 7f079ea00700 1 --2- 172.21.15.151:0/341666874 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0798103f80 0x7f0798110410 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f079000b0a0 tx=0x7f079000b3b0 comp rx=0 tx=0).stop 2024-05-21T21:58:43.068 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.066+0000 7f079ea00700 1 -- 172.21.15.151:0/341666874 shutdown_connections 2024-05-21T21:58:43.068 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.066+0000 7f079ea00700 1 --2- 172.21.15.151:0/341666874 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0798103f80 0x7f0798110410 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:43.068 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.066+0000 7f079ea00700 1 --2- 172.21.15.151:0/341666874 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07981095d0 0x7f07981099b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:43.068 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.066+0000 7f079ea00700 1 --2- 172.21.15.151:0/341666874 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07981035c0 0x7f0798103a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:43.068 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.066+0000 7f079ea00700 1 -- 172.21.15.151:0/341666874 >> 172.21.15.151:0/341666874 conn(0x7f079807a610 msgr2=0x7f079807aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:58:43.068 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.067+0000 7f079ea00700 1 -- 172.21.15.151:0/341666874 shutdown_connections 2024-05-21T21:58:43.068 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.067+0000 7f079ea00700 1 -- 172.21.15.151:0/341666874 wait complete. 2024-05-21T21:58:43.070 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.068+0000 7f079ea00700 1 Processor -- start 2024-05-21T21:58:43.070 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.068+0000 7f079ea00700 1 -- start start 2024-05-21T21:58:43.070 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.068+0000 7f079ea00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f07981035c0 0x7f07981a1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:43.070 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.068+0000 7f079ea00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0798103f80 0x7f07981a1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:43.070 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.069+0000 7f079ea00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07981095d0 0x7f079819bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:43.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.069+0000 7f079ea00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0798113720 con 0x7f07981035c0 2024-05-21T21:58:43.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.069+0000 7f079ea00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f07981135a0 con 0x7f07981095d0 2024-05-21T21:58:43.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.069+0000 7f079ea00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f07981138a0 con 0x7f0798103f80 2024-05-21T21:58:43.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.069+0000 7f0797400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0798103f80 0x7f07981a1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:43.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.069+0000 7f0797e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f07981035c0 0x7f07981a1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:43.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.069+0000 7f0797400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0798103f80 0x7f07981a1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:37192/0 (socket says 172.21.15.151:37192) 2024-05-21T21:58:43.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.069+0000 7f0797400700 1 -- 172.21.15.151:0/1886581332 learned_addr learned my addr 172.21.15.151:0/1886581332 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:58:43.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.069+0000 7f079ca00700 1 --2- 172.21.15.151:0/1886581332 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07981095d0 0x7f079819bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:43.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.069+0000 7f0797400700 1 -- 172.21.15.151:0/1886581332 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07981095d0 msgr2=0x7f079819bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:43.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.069+0000 7f0797400700 1 --2- 172.21.15.151:0/1886581332 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07981095d0 0x7f079819bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:43.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.069+0000 7f0797400700 1 -- 172.21.15.151:0/1886581332 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f07981035c0 msgr2=0x7f07981a1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:43.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.069+0000 7f0797400700 1 --2- 172.21.15.151:0/1886581332 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f07981035c0 0x7f07981a1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:43.072 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.069+0000 7f0797400700 1 -- 172.21.15.151:0/1886581332 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f079819c260 con 0x7f0798103f80 2024-05-21T21:58:43.072 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.069+0000 7f0797e00700 1 --2- 172.21.15.151:0/1886581332 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f07981035c0 0x7f07981a1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:58:43.072 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.070+0000 7f0797400700 1 --2- 172.21.15.151:0/1886581332 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0798103f80 0x7f07981a1e80 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f078400e9a0 tx=0x7f078400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:58:43.072 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.070+0000 7f0794c00700 1 -- 172.21.15.151:0/1886581332 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f078400ca10 con 0x7f0798103f80 2024-05-21T21:58:43.072 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.070+0000 7f0794c00700 1 -- 172.21.15.151:0/1886581332 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f078400cbb0 con 0x7f0798103f80 2024-05-21T21:58:43.072 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.070+0000 7f0794c00700 1 -- 172.21.15.151:0/1886581332 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0784018680 con 0x7f0798103f80 2024-05-21T21:58:43.072 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.070+0000 7f079ea00700 1 -- 172.21.15.151:0/1886581332 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f079819c550 con 0x7f0798103f80 2024-05-21T21:58:43.072 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.070+0000 7f079ea00700 1 -- 172.21.15.151:0/1886581332 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07981a8810 con 0x7f0798103f80 2024-05-21T21:58:43.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.076+0000 7f07a92cbb80 1 Processor -- start 2024-05-21T21:58:43.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.076+0000 7f0794c00700 1 -- 172.21.15.151:0/1886581332 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7f0784010430 con 0x7f0798103f80 2024-05-21T21:58:43.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.076+0000 7f07a92cbb80 1 -- start start 2024-05-21T21:58:43.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.077+0000 7f07a92cbb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e409aa2610 0x55e409aa29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:43.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.077+0000 7f07a92cbb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e409aa2f30 0x55e409a9cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:43.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.077+0000 7f07a92cbb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e409a9d140 0x55e409a9d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:43.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.077+0000 7f07a92cbb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e4099fae40 con 0x55e409aa2f30 2024-05-21T21:58:43.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.077+0000 7f07a92cbb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55e4099ea9e0 con 0x55e409a9d140 2024-05-21T21:58:43.078 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.077+0000 7f07a92cbb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55e4099eea20 con 0x55e409aa2610 2024-05-21T21:58:43.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.077+0000 7f079ca00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e409aa2610 0x55e409aa29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:43.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.077+0000 7f0797e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e409aa2f30 0x55e409a9cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:43.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.077+0000 7f079ca00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e409aa2610 0x55e409aa29f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:37208/0 (socket says 172.21.15.151:37208) 2024-05-21T21:58:43.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.077+0000 7f079ca00700 1 -- 172.21.15.151:0/3897236636 learned_addr learned my addr 172.21.15.151:0/3897236636 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:58:43.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.077+0000 7f0797400700 1 --2- 172.21.15.151:0/3897236636 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e409a9d140 0x55e409a9d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:43.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.078+0000 7f0797e00700 1 -- 172.21.15.151:0/3897236636 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e409aa2610 msgr2=0x55e409aa29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:43.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.078+0000 7f0797e00700 1 --2- 172.21.15.151:0/3897236636 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e409aa2610 0x55e409aa29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:43.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.078+0000 7f0797e00700 1 -- 172.21.15.151:0/3897236636 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e409a9d140 msgr2=0x55e409a9d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:43.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.078+0000 7f0797e00700 1 --2- 172.21.15.151:0/3897236636 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e409a9d140 0x55e409a9d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:43.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.078+0000 7f0797e00700 1 -- 172.21.15.151:0/3897236636 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4099b3420 con 0x55e409aa2f30 2024-05-21T21:58:43.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.078+0000 7f079ca00700 1 --2- 172.21.15.151:0/3897236636 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e409aa2610 0x55e409aa29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:58:43.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.078+0000 7f0797e00700 1 --2- 172.21.15.151:0/3897236636 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e409aa2f30 0x55e409a9cb20 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f07880114f0 tx=0x7f0788011800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa000e4c2b57c189 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:58:43.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.078+0000 7f078d600700 1 -- 172.21.15.151:0/3897236636 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0788004230 con 0x55e409aa2f30 2024-05-21T21:58:43.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.079+0000 7f078d600700 1 -- 172.21.15.151:0/3897236636 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07880043d0 con 0x55e409aa2f30 2024-05-21T21:58:43.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.079+0000 7f078d600700 1 -- 172.21.15.151:0/3897236636 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0788005650 con 0x55e409aa2f30 2024-05-21T21:58:43.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.079+0000 7f0794c00700 1 --2- 172.21.15.151:0/1886581332 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f077c0b2a70 0x7f077c0b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:43.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.079+0000 7f0794c00700 1 -- 172.21.15.151:0/1886581332 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f07840e2700 con 0x7f0798103f80 2024-05-21T21:58:43.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.079+0000 7f079ca00700 1 --2- 172.21.15.151:0/1886581332 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f077c0b2a70 0x7f077c0b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:43.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.080+0000 7f07a92cbb80 1 -- 172.21.15.151:0/3897236636 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e409aa2f30 msgr2=0x55e409a9cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:43.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.080+0000 7f07a92cbb80 1 --2- 172.21.15.151:0/3897236636 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e409aa2f30 0x55e409a9cb20 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f07880114f0 tx=0x7f0788011800 comp rx=0 tx=0).stop 2024-05-21T21:58:43.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.080+0000 7f079ca00700 1 --2- 172.21.15.151:0/1886581332 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f077c0b2a70 0x7f077c0b4f30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0790015000 tx=0x7f079000b720 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:58:43.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.080+0000 7f07a92cbb80 1 -- 172.21.15.151:0/3897236636 shutdown_connections 2024-05-21T21:58:43.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.080+0000 7f07a92cbb80 1 --2- 172.21.15.151:0/3897236636 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e409a9d140 0x55e409a9d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:43.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.080+0000 7f07a92cbb80 1 --2- 172.21.15.151:0/3897236636 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e409aa2610 0x55e409aa29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:43.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.080+0000 7f07a92cbb80 1 --2- 172.21.15.151:0/3897236636 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e409aa2f30 0x55e409a9cb20 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:43.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.080+0000 7f07a92cbb80 1 -- 172.21.15.151:0/3897236636 >> 172.21.15.151:0/3897236636 conn(0x55e4099cabe0 msgr2=0x55e409a97690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:58:43.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.080+0000 7f07a92cbb80 1 -- 172.21.15.151:0/3897236636 shutdown_connections 2024-05-21T21:58:43.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.080+0000 7f07a92cbb80 1 -- 172.21.15.151:0/3897236636 wait complete. 2024-05-21T21:58:43.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.081+0000 7f07a92cbb80 1 -- start start 2024-05-21T21:58:43.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.081+0000 7f07a92cbb80 1 Processor -- start 2024-05-21T21:58:43.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.081+0000 7f07a92cbb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e409a9d140 0x55e409aa8b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:43.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.081+0000 7f07a92cbb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e409aa2610 0x55e409aa5f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:43.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.081+0000 7f07a92cbb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e409aa2f30 0x55e409aa6440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:43.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.081+0000 7f07a92cbb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e4098f0fa0 con 0x55e409a9d140 2024-05-21T21:58:43.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.081+0000 7f07a92cbb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55e40998b760 con 0x55e409aa2f30 2024-05-21T21:58:43.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.081+0000 7f07a92cbb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55e409aa99e0 con 0x55e409aa2610 2024-05-21T21:58:43.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.081+0000 7f07a92cbb80 10 client.0 _ll_register_callbacks cb 0x55e409994c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:58:43.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.082+0000 7f0797e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e409a9d140 0x55e409aa8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:43.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.082+0000 7f0797e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e409a9d140 0x55e409aa8b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:34190/0 (socket says 172.21.15.151:34190) 2024-05-21T21:58:43.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.082+0000 7f0797e00700 1 -- 172.21.15.151:0/1665466665 learned_addr learned my addr 172.21.15.151:0/1665466665 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:58:43.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.082+0000 7f0797400700 1 --2- 172.21.15.151:0/1665466665 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e409aa2610 0x55e409aa5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:43.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.082+0000 7f079ca00700 1 --2- 172.21.15.151:0/1665466665 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e409aa2f30 0x55e409aa6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:43.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.082+0000 7f0797e00700 1 -- 172.21.15.151:0/1665466665 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e409aa2610 msgr2=0x55e409aa5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:43.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.082+0000 7f0797e00700 1 --2- 172.21.15.151:0/1665466665 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e409aa2610 0x55e409aa5f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:43.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.082+0000 7f0797e00700 1 -- 172.21.15.151:0/1665466665 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e409aa2f30 msgr2=0x55e409aa6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:43.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.082+0000 7f0797e00700 1 --2- 172.21.15.151:0/1665466665 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e409aa2f30 0x55e409aa6440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:43.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.082+0000 7f0797e00700 1 -- 172.21.15.151:0/1665466665 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e40993a8d0 con 0x55e409a9d140 2024-05-21T21:58:43.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.082+0000 7f079ca00700 1 --2- 172.21.15.151:0/1665466665 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e409aa2f30 0x55e409aa6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T21:58:43.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.082+0000 7f0797400700 1 --2- 172.21.15.151:0/1665466665 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e409aa2610 0x55e409aa5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:58:43.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.083+0000 7f0797e00700 1 --2- 172.21.15.151:0/1665466665 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e409a9d140 0x55e409aa8b60 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0788009790 tx=0x7f0788004980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd873e571e01ed75 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:58:43.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.083+0000 7f0782000700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-21T21:58:43.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.083+0000 7f0782000700 1 -- 172.21.15.151:0/1665466665 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f078800e670 con 0x55e409a9d140 2024-05-21T21:58:43.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.083+0000 7f0782000700 1 -- 172.21.15.151:0/1665466665 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f078800e810 con 0x55e409a9d140 2024-05-21T21:58:43.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.083+0000 7f07a92cbb80 1 -- 172.21.15.151:0/1665466665 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07880043d0 con 0x55e409a9d140 2024-05-21T21:58:43.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.083+0000 7f0782000700 1 -- 172.21.15.151:0/1665466665 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0788032050 con 0x55e409a9d140 2024-05-21T21:58:43.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.083+0000 7f07a92cbb80 10 client.15864 fetch_fsmap learned FSMap version 29 2024-05-21T21:58:43.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.083+0000 7f07a92cbb80 1 -- 172.21.15.151:0/1665466665 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e4099bb060 con 0x55e409a9d140 2024-05-21T21:58:43.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.084+0000 7f0782000700 1 -- 172.21.15.151:0/1665466665 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07880043d0 con 0x55e409a9d140 2024-05-21T21:58:43.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.084+0000 7f0782000700 1 -- 172.21.15.151:0/1665466665 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f0788019550 con 0x55e409a9d140 2024-05-21T21:58:43.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.084+0000 7f07a92cbb80 10 client.15864 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T21:58:43.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.084+0000 7f07a92cbb80 10 client.15864 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-05-21T21:58:43.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.084+0000 7f07a92cbb80 20 client.15864 populate_metadata read hostname 'smithi151' 2024-05-21T21:58:43.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.084+0000 7f07a92cbb80 1 --2- 172.21.15.151:0/1665466665 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x55e409ab62d0 0x55e409ab86c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:43.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.084+0000 7f07a92cbb80 4 client.15864 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T21:58:43.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.084+0000 7f07a92cbb80 1 -- 172.21.15.151:0/1665466665 --> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e4099fcd90 con 0x55e409ab62d0 2024-05-21T21:58:43.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.085+0000 7f0797400700 1 --2- 172.21.15.151:0/1665466665 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x55e409ab62d0 0x55e409ab86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:43.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.085+0000 7f0797400700 1 --2- 172.21.15.151:0/1665466665 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x55e409ab62d0 0x55e409ab86c0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=df1a1786579815b7 server_cookie=e7152d0d22306bed in_seq=0 out_seq=0 2024-05-21T21:58:43.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.085+0000 7f0782000700 10 client.15864 ms_handle_connect on v2:172.21.15.164:6836/1554609572 2024-05-21T21:58:43.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.086+0000 7f0782000700 1 -- 172.21.15.151:0/1665466665 <== mds.2 v2:172.21.15.164:6836/1554609572 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55e4099fcd90 con 0x55e409ab62d0 2024-05-21T21:58:43.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.086+0000 7f0782000700 10 client.15864 handle_command_reply: tid=0 2024-05-21T21:58:43.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.086+0000 7f07a92cbb80 1 client.15864 shutdown 2024-05-21T21:58:43.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.086+0000 7f07a92cbb80 1 -- 172.21.15.151:0/1665466665 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e409a9d140 msgr2=0x55e409aa8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:43.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.086+0000 7f07a92cbb80 1 --2- 172.21.15.151:0/1665466665 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e409a9d140 0x55e409aa8b60 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0788009790 tx=0x7f0788004980 comp rx=0 tx=0).stop 2024-05-21T21:58:43.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.087+0000 7f07a92cbb80 1 -- 172.21.15.151:0/1665466665 shutdown_connections 2024-05-21T21:58:43.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.087+0000 7f07a92cbb80 1 --2- 172.21.15.151:0/1665466665 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x55e409ab62d0 0x55e409ab86c0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:43.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.087+0000 7f07a92cbb80 1 --2- 172.21.15.151:0/1665466665 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e409aa2f30 0x55e409aa6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:43.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.087+0000 7f07a92cbb80 1 --2- 172.21.15.151:0/1665466665 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e409a9d140 0x55e409aa8b60 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:43.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.087+0000 7f07a92cbb80 1 --2- 172.21.15.151:0/1665466665 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e409aa2610 0x55e409aa5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:43.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.087+0000 7f07a92cbb80 1 -- 172.21.15.151:0/1665466665 >> 172.21.15.151:0/1665466665 conn(0x55e4099cabe0 msgr2=0x55e409a96d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:58:43.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.087+0000 7f0782000700 0 client.15864 ms_handle_reset on v2:172.21.15.164:6836/1554609572 2024-05-21T21:58:43.089 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.087+0000 7f07a92cbb80 1 -- 172.21.15.151:0/1665466665 shutdown_connections 2024-05-21T21:58:43.089 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.087+0000 7f07a92cbb80 1 -- 172.21.15.151:0/1665466665 wait complete. 2024-05-21T21:58:43.089 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.087+0000 7f07a92cbb80 20 client.15864 trim_cache size 0 max 16384 2024-05-21T21:58:43.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.104+0000 7f07a92cbb80 1 Processor -- start 2024-05-21T21:58:43.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.104+0000 7f07a92cbb80 1 -- start start 2024-05-21T21:58:43.105 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.104+0000 7f07a92cbb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e409a9d140 0x55e409a9cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:43.105 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.104+0000 7f07a92cbb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e409aa2610 0x55e409aa3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:43.105 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.104+0000 7f07a92cbb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e409aa2f30 0x55e409a974f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:43.105 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.104+0000 7f07a92cbb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f07981affe0 con 0x55e409aa2610 2024-05-21T21:58:43.105 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.104+0000 7f07a92cbb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f07981b1b20 con 0x55e409a9d140 2024-05-21T21:58:43.105 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.104+0000 7f07a92cbb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f07981b06b0 con 0x55e409aa2f30 2024-05-21T21:58:43.106 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.105+0000 7f0797400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e409aa2610 0x55e409aa3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:43.106 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.105+0000 7f0797400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e409aa2610 0x55e409aa3ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:34196/0 (socket says 172.21.15.151:34196) 2024-05-21T21:58:43.106 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.105+0000 7f0797400700 1 -- 172.21.15.151:0/1456708252 learned_addr learned my addr 172.21.15.151:0/1456708252 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:58:43.106 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.105+0000 7f0797e00700 1 --2- 172.21.15.151:0/1456708252 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e409a9d140 0x55e409a9cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:43.106 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.105+0000 7f079ca00700 1 --2- 172.21.15.151:0/1456708252 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e409aa2f30 0x55e409a974f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:43.106 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.105+0000 7f0797400700 1 -- 172.21.15.151:0/1456708252 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e409aa2f30 msgr2=0x55e409a974f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:43.106 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.105+0000 7f0797400700 1 --2- 172.21.15.151:0/1456708252 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e409aa2f30 0x55e409a974f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:43.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.105+0000 7f0797400700 1 -- 172.21.15.151:0/1456708252 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e409a9d140 msgr2=0x55e409a9cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:43.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.105+0000 7f0797400700 1 --2- 172.21.15.151:0/1456708252 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e409a9d140 0x55e409a9cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:43.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.105+0000 7f0797400700 1 -- 172.21.15.151:0/1456708252 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e409aa09d0 con 0x55e409aa2610 2024-05-21T21:58:43.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.105+0000 7f0797e00700 1 --2- 172.21.15.151:0/1456708252 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e409a9d140 0x55e409a9cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:58:43.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.106+0000 7f0797400700 1 --2- 172.21.15.151:0/1456708252 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e409aa2610 0x55e409aa3ae0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f07840e0460 tx=0x7f07840cc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a281e535720093 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:58:43.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.106+0000 7f078cc00700 1 -- 172.21.15.151:0/1456708252 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07840a5180 con 0x55e409aa2610 2024-05-21T21:58:43.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.106+0000 7f078cc00700 1 -- 172.21.15.151:0/1456708252 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07840b2190 con 0x55e409aa2610 2024-05-21T21:58:43.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.106+0000 7f078cc00700 1 -- 172.21.15.151:0/1456708252 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07840b2850 con 0x55e409aa2610 2024-05-21T21:58:43.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.107+0000 7f07a92cbb80 1 -- 172.21.15.151:0/1456708252 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e409aa2610 msgr2=0x55e409aa3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:43.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.107+0000 7f07a92cbb80 1 --2- 172.21.15.151:0/1456708252 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e409aa2610 0x55e409aa3ae0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f07840e0460 tx=0x7f07840cc040 comp rx=0 tx=0).stop 2024-05-21T21:58:43.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.107+0000 7f07a92cbb80 1 -- 172.21.15.151:0/1456708252 shutdown_connections 2024-05-21T21:58:43.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.107+0000 7f07a92cbb80 1 --2- 172.21.15.151:0/1456708252 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e409a9d140 0x55e409a9cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:43.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.107+0000 7f07a92cbb80 1 --2- 172.21.15.151:0/1456708252 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e409aa2f30 0x55e409a974f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:43.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.107+0000 7f07a92cbb80 1 --2- 172.21.15.151:0/1456708252 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e409aa2610 0x55e409aa3ae0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:43.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.107+0000 7f07a92cbb80 1 -- 172.21.15.151:0/1456708252 >> 172.21.15.151:0/1456708252 conn(0x55e4099cabe0 msgr2=0x55e409aa33d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:58:43.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.107+0000 7f07a92cbb80 1 -- 172.21.15.151:0/1456708252 shutdown_connections 2024-05-21T21:58:43.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.107+0000 7f07a92cbb80 1 -- 172.21.15.151:0/1456708252 wait complete. 2024-05-21T21:58:43.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.108+0000 7f07a92cbb80 1 -- start start 2024-05-21T21:58:43.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.108+0000 7f07a92cbb80 1 Processor -- start 2024-05-21T21:58:43.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.108+0000 7f07a92cbb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e409a9d140 0x55e409aaeca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:43.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.108+0000 7f07a92cbb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e409aa2610 0x55e409aaf1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:43.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.108+0000 7f07a92cbb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e409aa2f30 0x55e409abcee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:43.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.108+0000 7f07a92cbb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f07981b1450 con 0x55e409a9d140 2024-05-21T21:58:43.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.108+0000 7f07a92cbb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f07981b0d80 con 0x55e409aa2610 2024-05-21T21:58:43.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.108+0000 7f07a92cbb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f07981adf80 con 0x55e409aa2f30 2024-05-21T21:58:43.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.108+0000 7f07a92cbb80 10 client.0 _ll_register_callbacks cb 0x55e409994c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:58:43.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.109+0000 7f0797e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e409a9d140 0x55e409aaeca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:43.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.109+0000 7f0797e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e409a9d140 0x55e409aaeca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:34206/0 (socket says 172.21.15.151:34206) 2024-05-21T21:58:43.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.109+0000 7f0797e00700 1 -- 172.21.15.151:0/1160866172 learned_addr learned my addr 172.21.15.151:0/1160866172 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:58:43.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.109+0000 7f0797400700 1 --2- 172.21.15.151:0/1160866172 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e409aa2610 0x55e409aaf1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:43.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.109+0000 7f079ca00700 1 --2- 172.21.15.151:0/1160866172 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e409aa2f30 0x55e409abcee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:43.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.109+0000 7f0797400700 1 -- 172.21.15.151:0/1160866172 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e409aa2f30 msgr2=0x55e409abcee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:43.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.109+0000 7f0797400700 1 --2- 172.21.15.151:0/1160866172 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e409aa2f30 0x55e409abcee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:43.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.109+0000 7f0797400700 1 -- 172.21.15.151:0/1160866172 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e409a9d140 msgr2=0x55e409aaeca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:43.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.109+0000 7f0797400700 1 --2- 172.21.15.151:0/1160866172 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e409a9d140 0x55e409aaeca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:43.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.109+0000 7f0797400700 1 -- 172.21.15.151:0/1160866172 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e409abd420 con 0x55e409aa2610 2024-05-21T21:58:43.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.109+0000 7f0797e00700 1 --2- 172.21.15.151:0/1160866172 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e409a9d140 0x55e409aaeca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:58:43.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.109+0000 7f079ca00700 1 --2- 172.21.15.151:0/1160866172 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e409aa2f30 0x55e409abcee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T21:58:43.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.109+0000 7f0797400700 1 --2- 172.21.15.151:0/1160866172 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e409aa2610 0x55e409aaf1e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x55e409aa9aa0 tx=0x7f07840e85b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5fbfa887c0a5b820 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:58:43.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.109+0000 7f0782000700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-21T21:58:43.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.109+0000 7f0782000700 1 -- 172.21.15.151:0/1160866172 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07840a9500 con 0x55e409aa2610 2024-05-21T21:58:43.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.110+0000 7f0782000700 1 -- 172.21.15.151:0/1160866172 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f07840a9e60 con 0x55e409aa2610 2024-05-21T21:58:43.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.110+0000 7f07a92cbb80 1 -- 172.21.15.151:0/1160866172 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e4099f6570 con 0x55e409aa2610 2024-05-21T21:58:43.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.110+0000 7f0782000700 1 -- 172.21.15.151:0/1160866172 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f07840aa0f0 con 0x55e409aa2610 2024-05-21T21:58:43.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.110+0000 7f07a92cbb80 10 client.26000 fetch_fsmap learned FSMap version 29 2024-05-21T21:58:43.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.110+0000 7f07a92cbb80 1 -- 172.21.15.151:0/1160866172 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e409b28c50 con 0x55e409aa2610 2024-05-21T21:58:43.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.110+0000 7f0782000700 1 -- 172.21.15.151:0/1160866172 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55e4099f6570 con 0x55e409aa2610 2024-05-21T21:58:43.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.110+0000 7f0782000700 1 -- 172.21.15.151:0/1160866172 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f07840cc0b0 con 0x55e409aa2610 2024-05-21T21:58:43.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.110+0000 7f07a92cbb80 10 client.26000 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T21:58:43.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.110+0000 7f07a92cbb80 10 client.26000 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-05-21T21:58:43.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.110+0000 7f07a92cbb80 20 client.26000 populate_metadata read hostname 'smithi151' 2024-05-21T21:58:43.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.110+0000 7f07a92cbb80 1 --2- 172.21.15.151:0/1160866172 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x55e409b29220 0x55e409b2b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:43.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.110+0000 7f07a92cbb80 4 client.26000 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T21:58:43.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.110+0000 7f07a92cbb80 1 -- 172.21.15.151:0/1160866172 --> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e40986d250 con 0x55e409b29220 2024-05-21T21:58:43.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.111+0000 7f0797e00700 1 --2- 172.21.15.151:0/1160866172 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x55e409b29220 0x55e409b2b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:43.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.111+0000 7f0797e00700 1 --2- 172.21.15.151:0/1160866172 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x55e409b29220 0x55e409b2b640 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c3c98e983c3e5941 server_cookie=32109ccef90bf77d in_seq=0 out_seq=0 2024-05-21T21:58:43.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.111+0000 7f0782000700 10 client.26000 ms_handle_connect on v2:172.21.15.164:6836/1554609572 2024-05-21T21:58:43.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.111+0000 7f0782000700 1 -- 172.21.15.151:0/1160866172 <== mds.2 v2:172.21.15.164:6836/1554609572 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e40986d250 con 0x55e409b29220 2024-05-21T21:58:43.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.111+0000 7f0782000700 10 client.26000 handle_command_reply: tid=0 2024-05-21T21:58:43.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.111+0000 7f07a92cbb80 1 client.26000 shutdown 2024-05-21T21:58:43.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.112+0000 7f07a92cbb80 1 -- 172.21.15.151:0/1160866172 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e409aa2610 msgr2=0x55e409aaf1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:43.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.112+0000 7f07a92cbb80 1 --2- 172.21.15.151:0/1160866172 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e409aa2610 0x55e409aaf1e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x55e409aa9aa0 tx=0x7f07840e85b0 comp rx=0 tx=0).stop 2024-05-21T21:58:43.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.112+0000 7f07a92cbb80 1 -- 172.21.15.151:0/1160866172 shutdown_connections 2024-05-21T21:58:43.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.112+0000 7f07a92cbb80 1 --2- 172.21.15.151:0/1160866172 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x55e409b29220 0x55e409b2b640 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:43.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.112+0000 7f07a92cbb80 1 --2- 172.21.15.151:0/1160866172 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e409aa2f30 0x55e409abcee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:43.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.112+0000 7f07a92cbb80 1 --2- 172.21.15.151:0/1160866172 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e409a9d140 0x55e409aaeca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:43.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.112+0000 7f0782000700 0 client.26000 ms_handle_reset on v2:172.21.15.164:6836/1554609572 2024-05-21T21:58:43.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.112+0000 7f07a92cbb80 1 --2- 172.21.15.151:0/1160866172 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e409aa2610 0x55e409aaf1e0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:43.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.112+0000 7f07a92cbb80 1 -- 172.21.15.151:0/1160866172 >> 172.21.15.151:0/1160866172 conn(0x55e4099cabe0 msgr2=0x55e409aa47f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:58:43.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.112+0000 7f07a92cbb80 1 -- 172.21.15.151:0/1160866172 shutdown_connections 2024-05-21T21:58:43.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.112+0000 7f07a92cbb80 1 -- 172.21.15.151:0/1160866172 wait complete. 2024-05-21T21:58:43.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.112+0000 7f07a92cbb80 20 client.26000 trim_cache size 0 max 16384 2024-05-21T21:58:43.115 INFO:teuthology.orchestra.run.smithi151.stdout:[] 2024-05-21T21:58:43.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.113+0000 7f079ea00700 1 -- 172.21.15.151:0/1886581332 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f077c0b2a70 msgr2=0x7f077c0b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:43.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.113+0000 7f079ea00700 1 --2- 172.21.15.151:0/1886581332 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f077c0b2a70 0x7f077c0b4f30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0790015000 tx=0x7f079000b720 comp rx=0 tx=0).stop 2024-05-21T21:58:43.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.113+0000 7f079ea00700 1 -- 172.21.15.151:0/1886581332 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0798103f80 msgr2=0x7f07981a1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:43.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.113+0000 7f079ea00700 1 --2- 172.21.15.151:0/1886581332 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0798103f80 0x7f07981a1e80 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f078400e9a0 tx=0x7f078400ecb0 comp rx=0 tx=0).stop 2024-05-21T21:58:43.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.113+0000 7f079ea00700 1 -- 172.21.15.151:0/1886581332 shutdown_connections 2024-05-21T21:58:43.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.113+0000 7f079ea00700 1 --2- 172.21.15.151:0/1886581332 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f077c0b2a70 0x7f077c0b4f30 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:43.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.113+0000 7f079ea00700 1 --2- 172.21.15.151:0/1886581332 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07981095d0 0x7f079819bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:43.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.113+0000 7f079ea00700 1 --2- 172.21.15.151:0/1886581332 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f07981035c0 0x7f07981a1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:43.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.113+0000 7f079ea00700 1 --2- 172.21.15.151:0/1886581332 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0798103f80 0x7f07981a1e80 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:43.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.113+0000 7f079ea00700 1 -- 172.21.15.151:0/1886581332 >> 172.21.15.151:0/1886581332 conn(0x7f079807a610 msgr2=0x7f07980759a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:58:43.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.113+0000 7f079ea00700 1 -- 172.21.15.151:0/1886581332 shutdown_connections 2024-05-21T21:58:43.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:43.113+0000 7f079ea00700 1 -- 172.21.15.151:0/1886581332 wait complete. 2024-05-21T21:58:43.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:58:43 smithi151 ceph-mon[31623]: pgmap v96: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 29 MiB/s wr, 25 op/s 2024-05-21T21:58:43.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:58:43 smithi184 ceph-mon[36942]: pgmap v96: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 29 MiB/s wr, 25 op/s 2024-05-21T21:58:43.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:58:43 smithi164 ceph-mon[36269]: pgmap v96: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 29 MiB/s wr, 25 op/s 2024-05-21T21:58:44.061 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-21T21:58:44.487 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:58:45.666 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:58:45 smithi151 ceph-mon[31623]: pgmap v97: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 MiB/s wr, 17 op/s 2024-05-21T21:58:45.735 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:58:45 smithi184 ceph-mon[36942]: pgmap v97: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 MiB/s wr, 17 op/s 2024-05-21T21:58:45.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:58:45 smithi164 ceph-mon[36269]: pgmap v97: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 MiB/s wr, 17 op/s 2024-05-21T21:58:46.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.728+0000 7f7fb7600700 1 -- 172.21.15.151:0/1135827888 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7fb0076090 msgr2=0x7f7fb01141d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:46.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.728+0000 7f7fb7600700 1 --2- 172.21.15.151:0/1135827888 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7fb0076090 0x7f7fb01141d0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c009990 tx=0x7f7f9c009ca0 comp rx=0 tx=0).stop 2024-05-21T21:58:46.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.728+0000 7f7fb7600700 1 -- 172.21.15.151:0/1135827888 shutdown_connections 2024-05-21T21:58:46.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.728+0000 7f7fb7600700 1 --2- 172.21.15.151:0/1135827888 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7fb0075770 0x7f7fb0075b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:46.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.728+0000 7f7fb7600700 1 --2- 172.21.15.151:0/1135827888 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7fb0114710 0x7f7fb0116b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:46.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.728+0000 7f7fb7600700 1 --2- 172.21.15.151:0/1135827888 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7fb0076090 0x7f7fb01141d0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:46.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.728+0000 7f7fb7600700 1 -- 172.21.15.151:0/1135827888 >> 172.21.15.151:0/1135827888 conn(0x7f7fb007a6a0 msgr2=0x7f7fb007aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:58:46.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.728+0000 7f7fb7600700 1 -- 172.21.15.151:0/1135827888 shutdown_connections 2024-05-21T21:58:46.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.729+0000 7f7fb7600700 1 -- 172.21.15.151:0/1135827888 wait complete. 2024-05-21T21:58:46.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.729+0000 7f7fb7600700 1 Processor -- start 2024-05-21T21:58:46.731 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.730+0000 7f7fb7600700 1 -- start start 2024-05-21T21:58:46.732 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.730+0000 7f7fb7600700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7fb0075770 0x7f7fb01a0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:46.732 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.730+0000 7f7fb7600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7fb0076090 0x7f7fb01a0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:46.732 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.730+0000 7f7fb7600700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7fb0114710 0x7f7fb01a4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:46.732 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.730+0000 7f7fb7600700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb0119d20 con 0x7f7fb0076090 2024-05-21T21:58:46.732 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.730+0000 7f7fb7600700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb0119ba0 con 0x7f7fb0075770 2024-05-21T21:58:46.732 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.730+0000 7f7fb7600700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb0119ea0 con 0x7f7fb0114710 2024-05-21T21:58:46.732 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.730+0000 7f7fafe00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7fb0076090 0x7f7fb01a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:46.732 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.730+0000 7f7fb4c00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7fb0075770 0x7f7fb01a0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:46.732 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.731+0000 7f7fb5600700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7fb0114710 0x7f7fb01a4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:46.732 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.731+0000 7f7fb5600700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7fb0114710 0x7f7fb01a4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:46700/0 (socket says 172.21.15.151:46700) 2024-05-21T21:58:46.733 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.731+0000 7f7fb4c00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7fb0075770 0x7f7fb01a0230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:57938/0 (socket says 172.21.15.151:57938) 2024-05-21T21:58:46.733 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.731+0000 7f7fb5600700 1 -- 172.21.15.151:0/1057243793 learned_addr learned my addr 172.21.15.151:0/1057243793 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:58:46.733 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.731+0000 7f7fb5600700 1 -- 172.21.15.151:0/1057243793 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7fb0075770 msgr2=0x7f7fb01a0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:46.733 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.731+0000 7f7fb5600700 1 --2- 172.21.15.151:0/1057243793 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7fb0075770 0x7f7fb01a0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:46.733 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.731+0000 7f7fb5600700 1 -- 172.21.15.151:0/1057243793 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7fb0076090 msgr2=0x7f7fb01a0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:46.733 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.731+0000 7f7fb5600700 1 --2- 172.21.15.151:0/1057243793 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7fb0076090 0x7f7fb01a0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:46.733 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.731+0000 7f7fb5600700 1 -- 172.21.15.151:0/1057243793 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fb01a5240 con 0x7f7fb0114710 2024-05-21T21:58:46.733 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.731+0000 7f7fb5600700 1 --2- 172.21.15.151:0/1057243793 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7fb0114710 0x7f7fb01a4b00 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7fa800ef90 tx=0x7f7fa800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:58:46.733 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.732+0000 7f7fad600700 1 -- 172.21.15.151:0/1057243793 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa80092e0 con 0x7f7fb0114710 2024-05-21T21:58:46.733 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.732+0000 7f7fad600700 1 -- 172.21.15.151:0/1057243793 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fa8004bb0 con 0x7f7fb0114710 2024-05-21T21:58:46.733 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.732+0000 7f7fb7600700 1 -- 172.21.15.151:0/1057243793 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fb01a54d0 con 0x7f7fb0114710 2024-05-21T21:58:46.733 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.732+0000 7f7fad600700 1 -- 172.21.15.151:0/1057243793 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa8004280 con 0x7f7fb0114710 2024-05-21T21:58:46.733 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.732+0000 7f7fb7600700 1 -- 172.21.15.151:0/1057243793 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fb01acdb0 con 0x7f7fb0114710 2024-05-21T21:58:46.735 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.735+0000 7f7fad600700 1 -- 172.21.15.151:0/1057243793 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7f7fa801b030 con 0x7f7fb0114710 2024-05-21T21:58:46.736 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.736+0000 7f7fc1f60b80 1 Processor -- start 2024-05-21T21:58:46.736 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.736+0000 7f7fc1f60b80 1 -- start start 2024-05-21T21:58:46.737 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.737+0000 7f7fad600700 1 --2- 172.21.15.151:0/1057243793 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f7f980b29f0 0x7f7f980b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:46.738 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.737+0000 7f7fc1f60b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5597115cceb0 0x5597115cd290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:46.738 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.737+0000 7f7fc1f60b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5597115cd7d0 0x5597115c7260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:46.739 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.737+0000 7f7fad600700 1 -- 172.21.15.151:0/1057243793 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f7fa80e2130 con 0x7f7fb0114710 2024-05-21T21:58:46.739 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.737+0000 7f7fc1f60b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5597115c7880 0x5597115c7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:46.739 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.737+0000 7f7fc1f60b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x559711525e40 con 0x5597115cceb0 2024-05-21T21:58:46.739 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.737+0000 7f7fc1f60b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5597115159e0 con 0x5597115c7880 2024-05-21T21:58:46.739 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.737+0000 7f7fc1f60b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x559711519a20 con 0x5597115cd7d0 2024-05-21T21:58:46.739 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.737+0000 7f7fafe00700 1 --2- 172.21.15.151:0/1057243793 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f7f980b29f0 0x7f7f980b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:46.739 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.737+0000 7f7fb4c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5597115cceb0 0x5597115cd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:46.739 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.737+0000 7f7fb4c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5597115cceb0 0x5597115cd290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:46950/0 (socket says 172.21.15.151:46950) 2024-05-21T21:58:46.739 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.737+0000 7f7fb4c00700 1 -- 172.21.15.151:0/4212258766 learned_addr learned my addr 172.21.15.151:0/4212258766 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:58:46.739 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.737+0000 7f7fafe00700 1 --2- 172.21.15.151:0/4212258766 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5597115cd7d0 0x5597115c7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:46.739 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.737+0000 7f7fb5600700 1 --2- 172.21.15.151:0/4212258766 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5597115c7880 0x5597115c7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:46.739 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.738+0000 7f7fb4c00700 1 -- 172.21.15.151:0/4212258766 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5597115cd7d0 msgr2=0x5597115c7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:46.739 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.738+0000 7f7fb4c00700 1 --2- 172.21.15.151:0/4212258766 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5597115cd7d0 0x5597115c7260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:46.740 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.738+0000 7f7fb4c00700 1 -- 172.21.15.151:0/4212258766 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5597115c7880 msgr2=0x5597115c7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:46.740 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.738+0000 7f7fb4c00700 1 --2- 172.21.15.151:0/4212258766 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5597115c7880 0x5597115c7ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:46.740 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.738+0000 7f7fb4c00700 1 -- 172.21.15.151:0/4212258766 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597114de420 con 0x5597115cceb0 2024-05-21T21:58:46.740 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.738+0000 7f7fafe00700 1 --2- 172.21.15.151:0/4212258766 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5597115cd7d0 0x5597115c7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T21:58:46.740 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.738+0000 7f7fb4c00700 1 --2- 172.21.15.151:0/4212258766 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5597115cceb0 0x5597115cd290 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f7fa4009d80 tx=0x7f7fa4011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a64cc5dbdb733bb8 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:58:46.740 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.738+0000 7f7fafe00700 1 --2- 172.21.15.151:0/1057243793 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f7f980b29f0 0x7f7f980b4eb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c0098e0 tx=0x7f7f9c005290 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:58:46.740 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.739+0000 7f7fa2000700 1 -- 172.21.15.151:0/4212258766 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa4004230 con 0x5597115cceb0 2024-05-21T21:58:46.740 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.739+0000 7f7fa2000700 1 -- 172.21.15.151:0/4212258766 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fa40043d0 con 0x5597115cceb0 2024-05-21T21:58:46.740 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.739+0000 7f7fa2000700 1 -- 172.21.15.151:0/4212258766 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa40197a0 con 0x5597115cceb0 2024-05-21T21:58:46.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.740+0000 7f7fc1f60b80 1 -- 172.21.15.151:0/4212258766 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5597115cceb0 msgr2=0x5597115cd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:46.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.740+0000 7f7fc1f60b80 1 --2- 172.21.15.151:0/4212258766 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5597115cceb0 0x5597115cd290 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f7fa4009d80 tx=0x7f7fa4011720 comp rx=0 tx=0).stop 2024-05-21T21:58:46.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.740+0000 7f7fc1f60b80 1 -- 172.21.15.151:0/4212258766 shutdown_connections 2024-05-21T21:58:46.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.740+0000 7f7fc1f60b80 1 --2- 172.21.15.151:0/4212258766 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5597115c7880 0x5597115c7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:46.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.740+0000 7f7fc1f60b80 1 --2- 172.21.15.151:0/4212258766 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5597115cceb0 0x5597115cd290 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:46.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.740+0000 7f7fc1f60b80 1 --2- 172.21.15.151:0/4212258766 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5597115cd7d0 0x5597115c7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:46.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.740+0000 7f7fc1f60b80 1 -- 172.21.15.151:0/4212258766 >> 172.21.15.151:0/4212258766 conn(0x5597114f5be0 msgr2=0x5597115c2010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:58:46.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.740+0000 7f7fc1f60b80 1 -- 172.21.15.151:0/4212258766 shutdown_connections 2024-05-21T21:58:46.742 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.740+0000 7f7fc1f60b80 1 -- 172.21.15.151:0/4212258766 wait complete. 2024-05-21T21:58:46.742 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.741+0000 7f7fc1f60b80 1 -- start start 2024-05-21T21:58:46.742 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.741+0000 7f7fc1f60b80 1 Processor -- start 2024-05-21T21:58:46.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.742+0000 7f7fc1f60b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5597115c7880 0x5597115d3400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:46.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.742+0000 7f7fc1f60b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5597115cceb0 0x5597115d07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:46.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.742+0000 7f7fc1f60b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5597115cd7d0 0x5597115d0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:46.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.742+0000 7f7fc1f60b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55971141bfa0 con 0x5597115c7880 2024-05-21T21:58:46.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.742+0000 7f7fc1f60b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5597114b6760 con 0x5597115cceb0 2024-05-21T21:58:46.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.742+0000 7f7fc1f60b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5597115d4240 con 0x5597115cd7d0 2024-05-21T21:58:46.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.742+0000 7f7fc1f60b80 10 client.0 _ll_register_callbacks cb 0x5597114bfc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:58:46.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.742+0000 7f7fafe00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5597115cceb0 0x5597115d07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:46.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.742+0000 7f7fb4c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5597115c7880 0x5597115d3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:46.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.742+0000 7f7fafe00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5597115cceb0 0x5597115d07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:57964/0 (socket says 172.21.15.151:57964) 2024-05-21T21:58:46.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.742+0000 7f7fafe00700 1 -- 172.21.15.151:0/1590479075 learned_addr learned my addr 172.21.15.151:0/1590479075 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:58:46.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.742+0000 7f7fb5600700 1 --2- 172.21.15.151:0/1590479075 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5597115cd7d0 0x5597115d0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:46.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.742+0000 7f7fafe00700 1 -- 172.21.15.151:0/1590479075 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5597115cd7d0 msgr2=0x5597115d0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:46.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.742+0000 7f7fafe00700 1 --2- 172.21.15.151:0/1590479075 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5597115cd7d0 0x5597115d0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:46.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.742+0000 7f7fafe00700 1 -- 172.21.15.151:0/1590479075 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5597115c7880 msgr2=0x5597115d3400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:46.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.742+0000 7f7fafe00700 1 --2- 172.21.15.151:0/1590479075 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5597115c7880 0x5597115d3400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:46.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.742+0000 7f7fafe00700 1 -- 172.21.15.151:0/1590479075 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597114658d0 con 0x5597115cceb0 2024-05-21T21:58:46.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.742+0000 7f7fb5600700 1 --2- 172.21.15.151:0/1590479075 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5597115cd7d0 0x5597115d0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T21:58:46.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.743+0000 7f7fafe00700 1 --2- 172.21.15.151:0/1590479075 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5597115cceb0 0x5597115d07a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c015000 tx=0x7f7f9c009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e8a0e073917aeb6 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:58:46.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.743+0000 7f7f96a00700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-21T21:58:46.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.743+0000 7f7f96a00700 1 -- 172.21.15.151:0/1590479075 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f9c037060 con 0x5597115cceb0 2024-05-21T21:58:46.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.743+0000 7f7f96a00700 1 -- 172.21.15.151:0/1590479075 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f7f9c003a10 con 0x5597115cceb0 2024-05-21T21:58:46.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.743+0000 7f7fc1f60b80 1 -- 172.21.15.151:0/1590479075 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fa40043d0 con 0x5597115cceb0 2024-05-21T21:58:46.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.744+0000 7f7f96a00700 1 -- 172.21.15.151:0/1590479075 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7f9c021040 con 0x5597115cceb0 2024-05-21T21:58:46.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.744+0000 7f7fc1f60b80 10 client.26012 fetch_fsmap learned FSMap version 29 2024-05-21T21:58:46.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.744+0000 7f7fc1f60b80 1 -- 172.21.15.151:0/1590479075 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5597114e6060 con 0x5597115cceb0 2024-05-21T21:58:46.747 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.744+0000 7f7f96a00700 1 -- 172.21.15.151:0/1590479075 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fa40043d0 con 0x5597115cceb0 2024-05-21T21:58:46.747 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.744+0000 7f7f96a00700 1 -- 172.21.15.151:0/1590479075 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f7f9c00faf0 con 0x5597115cceb0 2024-05-21T21:58:46.747 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.744+0000 7f7fc1f60b80 10 client.26012 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T21:58:46.747 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.744+0000 7f7fc1f60b80 10 client.26012 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-21T21:58:46.748 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.744+0000 7f7fc1f60b80 20 client.26012 populate_metadata read hostname 'smithi151' 2024-05-21T21:58:46.748 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.744+0000 7f7fc1f60b80 1 --2- 172.21.15.151:0/1590479075 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x5597115e0d10 0x5597115e3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:46.748 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.744+0000 7f7fc1f60b80 4 client.26012 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T21:58:46.748 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.744+0000 7f7fc1f60b80 1 -- 172.21.15.151:0/1590479075 --> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559711527d90 con 0x5597115e0d10 2024-05-21T21:58:46.748 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.745+0000 7f7fb4c00700 1 --2- 172.21.15.151:0/1590479075 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x5597115e0d10 0x5597115e3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:46.748 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.746+0000 7f7fb4c00700 1 --2- 172.21.15.151:0/1590479075 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x5597115e0d10 0x5597115e3120 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c6219cb678ec0bfa server_cookie=e25b3b0b9d99ae25 in_seq=0 out_seq=0 2024-05-21T21:58:46.748 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.746+0000 7f7f96a00700 10 client.26012 ms_handle_connect on v2:172.21.15.184:6838/3059935143 2024-05-21T21:58:46.748 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.746+0000 7f7f96a00700 1 -- 172.21.15.151:0/1590479075 <== mds.1 v2:172.21.15.184:6838/3059935143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x559711527d90 con 0x5597115e0d10 2024-05-21T21:58:46.748 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.746+0000 7f7f96a00700 10 client.26012 handle_command_reply: tid=0 2024-05-21T21:58:46.748 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.747+0000 7f7fc1f60b80 1 client.26012 shutdown 2024-05-21T21:58:46.748 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.747+0000 7f7fc1f60b80 1 -- 172.21.15.151:0/1590479075 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5597115cceb0 msgr2=0x5597115d07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:46.748 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.747+0000 7f7fc1f60b80 1 --2- 172.21.15.151:0/1590479075 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5597115cceb0 0x5597115d07a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c015000 tx=0x7f7f9c009ca0 comp rx=0 tx=0).stop 2024-05-21T21:58:46.748 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.747+0000 7f7fc1f60b80 1 -- 172.21.15.151:0/1590479075 shutdown_connections 2024-05-21T21:58:46.748 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.747+0000 7f7fc1f60b80 1 --2- 172.21.15.151:0/1590479075 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x5597115e0d10 0x5597115e3120 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:46.748 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.747+0000 7f7fc1f60b80 1 --2- 172.21.15.151:0/1590479075 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5597115cd7d0 0x5597115d0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:46.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.747+0000 7f7fc1f60b80 1 --2- 172.21.15.151:0/1590479075 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5597115c7880 0x5597115d3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:46.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.747+0000 7f7fc1f60b80 1 --2- 172.21.15.151:0/1590479075 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5597115cceb0 0x5597115d07a0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:46.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.747+0000 7f7f96a00700 0 client.26012 ms_handle_reset on v2:172.21.15.184:6838/3059935143 2024-05-21T21:58:46.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.747+0000 7f7fc1f60b80 1 -- 172.21.15.151:0/1590479075 >> 172.21.15.151:0/1590479075 conn(0x5597114f5be0 msgr2=0x5597115c6250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:58:46.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.747+0000 7f7fc1f60b80 1 -- 172.21.15.151:0/1590479075 shutdown_connections 2024-05-21T21:58:46.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.747+0000 7f7fc1f60b80 1 -- 172.21.15.151:0/1590479075 wait complete. 2024-05-21T21:58:46.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.747+0000 7f7fc1f60b80 20 client.26012 trim_cache size 0 max 16384 2024-05-21T21:58:46.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.764+0000 7f7fc1f60b80 1 Processor -- start 2024-05-21T21:58:46.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.764+0000 7f7fc1f60b80 1 -- start start 2024-05-21T21:58:46.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.764+0000 7f7fc1f60b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5597115c7880 0x5597115c7260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:46.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.764+0000 7f7fc1f60b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5597115cceb0 0x5597115c19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:46.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.764+0000 7f7fc1f60b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5597115cd7d0 0x5597115c1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:46.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.764+0000 7f7fc1f60b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb01b4410 con 0x5597115cceb0 2024-05-21T21:58:46.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.764+0000 7f7fc1f60b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb01b5f50 con 0x5597115cd7d0 2024-05-21T21:58:46.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.764+0000 7f7fc1f60b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb01b4ae0 con 0x5597115c7880 2024-05-21T21:58:46.766 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.765+0000 7f7fafe00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5597115cceb0 0x5597115c19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:46.766 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.765+0000 7f7fafe00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5597115cceb0 0x5597115c19e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:46968/0 (socket says 172.21.15.151:46968) 2024-05-21T21:58:46.766 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.765+0000 7f7fb4c00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5597115c7880 0x5597115c7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:46.766 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.765+0000 7f7fb5600700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5597115cd7d0 0x5597115c1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:46.766 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.765+0000 7f7fafe00700 1 -- 172.21.15.151:0/1284570763 learned_addr learned my addr 172.21.15.151:0/1284570763 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:58:46.766 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.765+0000 7f7fb4c00700 1 -- 172.21.15.151:0/1284570763 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5597115cd7d0 msgr2=0x5597115c1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:46.766 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.765+0000 7f7fb4c00700 1 --2- 172.21.15.151:0/1284570763 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5597115cd7d0 0x5597115c1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:46.767 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.765+0000 7f7fb4c00700 1 -- 172.21.15.151:0/1284570763 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5597115cceb0 msgr2=0x5597115c19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:46.767 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.765+0000 7f7fb4c00700 1 --2- 172.21.15.151:0/1284570763 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5597115cceb0 0x5597115c19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:46.767 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.765+0000 7f7fb4c00700 1 -- 172.21.15.151:0/1284570763 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597115cd290 con 0x5597115c7880 2024-05-21T21:58:46.767 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.765+0000 7f7fb5600700 1 --2- 172.21.15.151:0/1284570763 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5597115cd7d0 0x5597115c1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:58:46.767 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.765+0000 7f7fb4c00700 1 --2- 172.21.15.151:0/1284570763 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5597115c7880 0x5597115c7260 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f7fa4005950 tx=0x7f7fa4009710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=685e33d2dfc1a74b server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:58:46.767 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.766+0000 7f7fa1600700 1 -- 172.21.15.151:0/1284570763 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa40055b0 con 0x5597115c7880 2024-05-21T21:58:46.767 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.766+0000 7f7fa1600700 1 -- 172.21.15.151:0/1284570763 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fa4003b30 con 0x5597115c7880 2024-05-21T21:58:46.767 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.766+0000 7f7fa1600700 1 -- 172.21.15.151:0/1284570763 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa402a640 con 0x5597115c7880 2024-05-21T21:58:46.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.767+0000 7f7fc1f60b80 1 -- 172.21.15.151:0/1284570763 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5597115c7880 msgr2=0x5597115c7260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:46.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.767+0000 7f7fc1f60b80 1 --2- 172.21.15.151:0/1284570763 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5597115c7880 0x5597115c7260 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f7fa4005950 tx=0x7f7fa4009710 comp rx=0 tx=0).stop 2024-05-21T21:58:46.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.767+0000 7f7fc1f60b80 1 -- 172.21.15.151:0/1284570763 shutdown_connections 2024-05-21T21:58:46.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.767+0000 7f7fc1f60b80 1 --2- 172.21.15.151:0/1284570763 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5597115cd7d0 0x5597115c1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:46.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.767+0000 7f7fc1f60b80 1 --2- 172.21.15.151:0/1284570763 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5597115c7880 0x5597115c7260 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:46.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.767+0000 7f7fc1f60b80 1 --2- 172.21.15.151:0/1284570763 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5597115cceb0 0x5597115c19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:46.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.767+0000 7f7fc1f60b80 1 -- 172.21.15.151:0/1284570763 >> 172.21.15.151:0/1284570763 conn(0x5597114f5be0 msgr2=0x5597115cf0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:58:46.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.767+0000 7f7fc1f60b80 1 -- 172.21.15.151:0/1284570763 shutdown_connections 2024-05-21T21:58:46.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.767+0000 7f7fc1f60b80 1 -- 172.21.15.151:0/1284570763 wait complete. 2024-05-21T21:58:46.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.767+0000 7f7fc1f60b80 1 -- start start 2024-05-21T21:58:46.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.767+0000 7f7fc1f60b80 1 Processor -- start 2024-05-21T21:58:46.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.768+0000 7f7fc1f60b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5597115c7880 0x5597115e84e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:46.770 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.768+0000 7f7fc1f60b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5597115cceb0 0x5597115e8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:46.770 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.768+0000 7f7fc1f60b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5597115cd7d0 0x5597115e8f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:46.770 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.768+0000 7f7fc1f60b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb01b5880 con 0x5597115c7880 2024-05-21T21:58:46.770 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.768+0000 7f7fc1f60b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb01b51b0 con 0x5597115cceb0 2024-05-21T21:58:46.770 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.768+0000 7f7fc1f60b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb01b23b0 con 0x5597115cd7d0 2024-05-21T21:58:46.770 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.768+0000 7f7fc1f60b80 10 client.0 _ll_register_callbacks cb 0x5597115bc610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:58:46.770 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.768+0000 7f7fb4c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5597115c7880 0x5597115e84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:46.770 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.768+0000 7f7fb4c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5597115c7880 0x5597115e84e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:46978/0 (socket says 172.21.15.151:46978) 2024-05-21T21:58:46.770 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.768+0000 7f7fb4c00700 1 -- 172.21.15.151:0/3705059411 learned_addr learned my addr 172.21.15.151:0/3705059411 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:58:46.770 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.769+0000 7f7fb5600700 1 --2- 172.21.15.151:0/3705059411 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5597115cd7d0 0x5597115e8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:46.770 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.769+0000 7f7fafe00700 1 --2- 172.21.15.151:0/3705059411 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5597115cceb0 0x5597115e8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:46.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.769+0000 7f7fb4c00700 1 -- 172.21.15.151:0/3705059411 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5597115cd7d0 msgr2=0x5597115e8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:46.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.769+0000 7f7fb4c00700 1 --2- 172.21.15.151:0/3705059411 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5597115cd7d0 0x5597115e8f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:46.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.769+0000 7f7fb4c00700 1 -- 172.21.15.151:0/3705059411 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5597115cceb0 msgr2=0x5597115e8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:46.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.769+0000 7f7fb4c00700 1 --2- 172.21.15.151:0/3705059411 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5597115cceb0 0x5597115e8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:46.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.769+0000 7f7fb4c00700 1 -- 172.21.15.151:0/3705059411 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55971164a8c0 con 0x5597115c7880 2024-05-21T21:58:46.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.769+0000 7f7fb4c00700 1 --2- 172.21.15.151:0/3705059411 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5597115c7880 0x5597115e84e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f7fa4011760 tx=0x7f7fa40052c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9977fe98c4337bd server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:58:46.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.769+0000 7f7f96a00700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-21T21:58:46.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.770+0000 7f7f96a00700 1 -- 172.21.15.151:0/3705059411 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa4003680 con 0x5597115c7880 2024-05-21T21:58:46.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.770+0000 7f7fc1f60b80 1 -- 172.21.15.151:0/3705059411 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fa4003b30 con 0x5597115c7880 2024-05-21T21:58:46.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.770+0000 7f7f96a00700 1 -- 172.21.15.151:0/3705059411 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f7fa4003820 con 0x5597115c7880 2024-05-21T21:58:46.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.770+0000 7f7f96a00700 1 -- 172.21.15.151:0/3705059411 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7fa4043050 con 0x5597115c7880 2024-05-21T21:58:46.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.770+0000 7f7fc1f60b80 10 client.15888 fetch_fsmap learned FSMap version 29 2024-05-21T21:58:46.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.770+0000 7f7fc1f60b80 1 -- 172.21.15.151:0/3705059411 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55971164bba0 con 0x5597115c7880 2024-05-21T21:58:46.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.770+0000 7f7f96a00700 1 -- 172.21.15.151:0/3705059411 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fa4003b30 con 0x5597115c7880 2024-05-21T21:58:46.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.770+0000 7f7f96a00700 1 -- 172.21.15.151:0/3705059411 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f7fa4024410 con 0x5597115c7880 2024-05-21T21:58:46.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.770+0000 7f7fc1f60b80 10 client.15888 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T21:58:46.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.771+0000 7f7fc1f60b80 10 client.15888 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-21T21:58:46.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.771+0000 7f7fc1f60b80 20 client.15888 populate_metadata read hostname 'smithi151' 2024-05-21T21:58:46.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.771+0000 7f7fc1f60b80 1 --2- 172.21.15.151:0/3705059411 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x55971164bfa0 0x55971164e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:46.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.771+0000 7f7fc1f60b80 4 client.15888 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T21:58:46.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.771+0000 7f7fc1f60b80 1 -- 172.21.15.151:0/3705059411 --> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559711521570 con 0x55971164bfa0 2024-05-21T21:58:46.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.771+0000 7f7fafe00700 1 --2- 172.21.15.151:0/3705059411 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x55971164bfa0 0x55971164e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:46.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.771+0000 7f7fafe00700 1 --2- 172.21.15.151:0/3705059411 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x55971164bfa0 0x55971164e390 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fc28acd534370b01 server_cookie=6f34b9cca05314ed in_seq=0 out_seq=0 2024-05-21T21:58:46.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.771+0000 7f7f96a00700 10 client.15888 ms_handle_connect on v2:172.21.15.184:6838/3059935143 2024-05-21T21:58:46.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.772+0000 7f7f96a00700 1 -- 172.21.15.151:0/3705059411 <== mds.1 v2:172.21.15.184:6838/3059935143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559711521570 con 0x55971164bfa0 2024-05-21T21:58:46.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.772+0000 7f7f96a00700 10 client.15888 handle_command_reply: tid=0 2024-05-21T21:58:46.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.772+0000 7f7fc1f60b80 1 client.15888 shutdown 2024-05-21T21:58:46.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.772+0000 7f7fc1f60b80 1 -- 172.21.15.151:0/3705059411 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5597115c7880 msgr2=0x5597115e84e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:46.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.772+0000 7f7fc1f60b80 1 --2- 172.21.15.151:0/3705059411 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5597115c7880 0x5597115e84e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f7fa4011760 tx=0x7f7fa40052c0 comp rx=0 tx=0).stop 2024-05-21T21:58:46.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.772+0000 7f7fc1f60b80 1 -- 172.21.15.151:0/3705059411 shutdown_connections 2024-05-21T21:58:46.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.772+0000 7f7fc1f60b80 1 --2- 172.21.15.151:0/3705059411 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x55971164bfa0 0x55971164e390 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:46.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.772+0000 7f7fc1f60b80 1 --2- 172.21.15.151:0/3705059411 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5597115cd7d0 0x5597115e8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:46.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.772+0000 7f7fc1f60b80 1 --2- 172.21.15.151:0/3705059411 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5597115c7880 0x5597115e84e0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:46.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.772+0000 7f7fc1f60b80 1 --2- 172.21.15.151:0/3705059411 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5597115cceb0 0x5597115e8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:46.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.772+0000 7f7fc1f60b80 1 -- 172.21.15.151:0/3705059411 >> 172.21.15.151:0/3705059411 conn(0x5597114f5be0 msgr2=0x5597115c19e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:58:46.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.772+0000 7f7f96a00700 0 client.15888 ms_handle_reset on v2:172.21.15.184:6838/3059935143 2024-05-21T21:58:46.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.772+0000 7f7fc1f60b80 1 -- 172.21.15.151:0/3705059411 shutdown_connections 2024-05-21T21:58:46.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.772+0000 7f7fc1f60b80 1 -- 172.21.15.151:0/3705059411 wait complete. 2024-05-21T21:58:46.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.772+0000 7f7fc1f60b80 20 client.15888 trim_cache size 0 max 16384 2024-05-21T21:58:46.776 INFO:teuthology.orchestra.run.smithi151.stdout:[] 2024-05-21T21:58:46.776 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.773+0000 7f7fb7600700 1 -- 172.21.15.151:0/1057243793 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f7f980b29f0 msgr2=0x7f7f980b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:46.776 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.773+0000 7f7fb7600700 1 --2- 172.21.15.151:0/1057243793 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f7f980b29f0 0x7f7f980b4eb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c0098e0 tx=0x7f7f9c005290 comp rx=0 tx=0).stop 2024-05-21T21:58:46.777 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.773+0000 7f7fb7600700 1 -- 172.21.15.151:0/1057243793 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7fb0114710 msgr2=0x7f7fb01a4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:46.777 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.773+0000 7f7fb7600700 1 --2- 172.21.15.151:0/1057243793 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7fb0114710 0x7f7fb01a4b00 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7fa800ef90 tx=0x7f7fa800c3a0 comp rx=0 tx=0).stop 2024-05-21T21:58:46.777 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.773+0000 7f7fb7600700 1 -- 172.21.15.151:0/1057243793 shutdown_connections 2024-05-21T21:58:46.777 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.773+0000 7f7fb7600700 1 --2- 172.21.15.151:0/1057243793 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f7f980b29f0 0x7f7f980b4eb0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:46.777 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.773+0000 7f7fb7600700 1 --2- 172.21.15.151:0/1057243793 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7fb0075770 0x7f7fb01a0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:46.777 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.773+0000 7f7fb7600700 1 --2- 172.21.15.151:0/1057243793 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7fb0114710 0x7f7fb01a4b00 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:46.777 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.773+0000 7f7fb7600700 1 --2- 172.21.15.151:0/1057243793 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7fb0076090 0x7f7fb01a0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:46.777 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.773+0000 7f7fb7600700 1 -- 172.21.15.151:0/1057243793 >> 172.21.15.151:0/1057243793 conn(0x7f7fb007a6a0 msgr2=0x7f7fb01135b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:58:46.777 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.774+0000 7f7fb7600700 1 -- 172.21.15.151:0/1057243793 shutdown_connections 2024-05-21T21:58:46.777 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:46.774+0000 7f7fb7600700 1 -- 172.21.15.151:0/1057243793 wait complete. 2024-05-21T21:58:47.675 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-21T21:58:47.702 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:58:47 smithi151 ceph-mon[31623]: pgmap v98: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 MiB/s wr, 17 op/s 2024-05-21T21:58:47.735 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:58:47 smithi184 ceph-mon[36942]: pgmap v98: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 MiB/s wr, 17 op/s 2024-05-21T21:58:47.785 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:58:47 smithi164 ceph-mon[36269]: pgmap v98: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 MiB/s wr, 17 op/s 2024-05-21T21:58:48.122 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:58:49.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:58:49 smithi151 ceph-mon[31623]: pgmap v99: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 23 MiB/s wr, 23 op/s 2024-05-21T21:58:49.735 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:58:49 smithi184 ceph-mon[36942]: pgmap v99: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 23 MiB/s wr, 23 op/s 2024-05-21T21:58:49.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:58:49 smithi164 ceph-mon[36269]: pgmap v99: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 23 MiB/s wr, 23 op/s 2024-05-21T21:58:50.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.342+0000 7f5868c00700 1 -- 172.21.15.151:0/4171229616 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f58641094f0 msgr2=0x7f58641098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:50.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.342+0000 7f5868c00700 1 --2- 172.21.15.151:0/4171229616 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f58641094f0 0x7f58641098d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f5854009990 tx=0x7f5854009ca0 comp rx=0 tx=0).stop 2024-05-21T21:58:50.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.343+0000 7f5868c00700 1 -- 172.21.15.151:0/4171229616 shutdown_connections 2024-05-21T21:58:50.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.343+0000 7f5868c00700 1 --2- 172.21.15.151:0/4171229616 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5864103e20 0x7f5864110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:50.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.343+0000 7f5868c00700 1 --2- 172.21.15.151:0/4171229616 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f58641094f0 0x7f58641098d0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:50.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.343+0000 7f5868c00700 1 --2- 172.21.15.151:0/4171229616 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5864103460 0x7f58641038e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:50.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.343+0000 7f5868c00700 1 -- 172.21.15.151:0/4171229616 >> 172.21.15.151:0/4171229616 conn(0x7f58640fef90 msgr2=0x7f58641013b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:58:50.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.343+0000 7f5868c00700 1 -- 172.21.15.151:0/4171229616 shutdown_connections 2024-05-21T21:58:50.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.343+0000 7f5868c00700 1 -- 172.21.15.151:0/4171229616 wait complete. 2024-05-21T21:58:50.345 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.344+0000 7f5868c00700 1 Processor -- start 2024-05-21T21:58:50.345 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.344+0000 7f5868c00700 1 -- start start 2024-05-21T21:58:50.345 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.344+0000 7f5868c00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5864103460 0x7f586419bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:50.345 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.344+0000 7f5868c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5864103e20 0x7f586419c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:50.345 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.344+0000 7f5868c00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f58641094f0 0x7f58641a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:50.345 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.344+0000 7f5868c00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5864113600 con 0x7f5864103e20 2024-05-21T21:58:50.346 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.344+0000 7f5868c00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5864113480 con 0x7f5864103460 2024-05-21T21:58:50.346 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.344+0000 7f5868c00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5864113780 con 0x7f58641094f0 2024-05-21T21:58:50.346 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.345+0000 7f585be00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5864103e20 0x7f586419c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:50.346 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.345+0000 7f585be00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5864103e20 0x7f586419c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:46984/0 (socket says 172.21.15.151:46984) 2024-05-21T21:58:50.346 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.345+0000 7f585be00700 1 -- 172.21.15.151:0/1412845536 learned_addr learned my addr 172.21.15.151:0/1412845536 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:58:50.346 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.345+0000 7f5863400700 1 --2- 172.21.15.151:0/1412845536 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5864103460 0x7f586419bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:50.346 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.345+0000 7f5863e00700 1 --2- 172.21.15.151:0/1412845536 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f58641094f0 0x7f58641a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:50.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.345+0000 7f585be00700 1 -- 172.21.15.151:0/1412845536 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f58641094f0 msgr2=0x7f58641a05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:50.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.345+0000 7f585be00700 1 --2- 172.21.15.151:0/1412845536 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f58641094f0 0x7f58641a05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:50.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.345+0000 7f585be00700 1 -- 172.21.15.151:0/1412845536 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5864103460 msgr2=0x7f586419bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:50.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.345+0000 7f585be00700 1 --2- 172.21.15.151:0/1412845536 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5864103460 0x7f586419bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:50.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.345+0000 7f585be00700 1 -- 172.21.15.151:0/1412845536 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58641a0d20 con 0x7f5864103e20 2024-05-21T21:58:50.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.345+0000 7f5863400700 1 --2- 172.21.15.151:0/1412845536 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5864103460 0x7f586419bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:58:50.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.346+0000 7f585be00700 1 --2- 172.21.15.151:0/1412845536 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5864103e20 0x7f586419c250 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f584c00e9a0 tx=0x7f584c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:58:50.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.346+0000 7f5860c00700 1 -- 172.21.15.151:0/1412845536 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f584c00ca10 con 0x7f5864103e20 2024-05-21T21:58:50.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.346+0000 7f5860c00700 1 -- 172.21.15.151:0/1412845536 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f584c00cbb0 con 0x7f5864103e20 2024-05-21T21:58:50.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.346+0000 7f5868c00700 1 -- 172.21.15.151:0/1412845536 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58641a1010 con 0x7f5864103e20 2024-05-21T21:58:50.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.346+0000 7f5860c00700 1 -- 172.21.15.151:0/1412845536 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f584c018680 con 0x7f5864103e20 2024-05-21T21:58:50.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.346+0000 7f5868c00700 1 -- 172.21.15.151:0/1412845536 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58641a8890 con 0x7f5864103e20 2024-05-21T21:58:50.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.351+0000 7f5860c00700 1 -- 172.21.15.151:0/1412845536 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7f584c018820 con 0x7f5864103e20 2024-05-21T21:58:50.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.352+0000 7f5874afbb80 1 Processor -- start 2024-05-21T21:58:50.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.352+0000 7f5874afbb80 1 -- start start 2024-05-21T21:58:50.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.352+0000 7f5874afbb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55faa54b4020 0x55faa54b4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:50.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.352+0000 7f5874afbb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55faa54b4940 0x55faa54ae4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:50.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.352+0000 7f5874afbb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55faa54aead0 0x55faa54aef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:50.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.352+0000 7f5874afbb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55faa540ce40 con 0x55faa54b4940 2024-05-21T21:58:50.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.352+0000 7f5874afbb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55faa53fc9e0 con 0x55faa54b4020 2024-05-21T21:58:50.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.352+0000 7f5874afbb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55faa5400a20 con 0x55faa54aead0 2024-05-21T21:58:50.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.353+0000 7f5863400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55faa54b4940 0x55faa54ae4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:50.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.353+0000 7f5863400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55faa54b4940 0x55faa54ae4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:46996/0 (socket says 172.21.15.151:46996) 2024-05-21T21:58:50.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.353+0000 7f5863400700 1 -- 172.21.15.151:0/3456312273 learned_addr learned my addr 172.21.15.151:0/3456312273 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:58:50.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.353+0000 7f5863e00700 1 --2- 172.21.15.151:0/3456312273 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55faa54b4020 0x55faa54b4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:50.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.353+0000 7f585be00700 1 --2- 172.21.15.151:0/3456312273 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55faa54aead0 0x55faa54aef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:50.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.353+0000 7f5860c00700 1 --2- 172.21.15.151:0/1412845536 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f58480b29f0 0x7f58480b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:50.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.353+0000 7f5863400700 1 -- 172.21.15.151:0/3456312273 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55faa54aead0 msgr2=0x55faa54aef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:50.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.353+0000 7f5863400700 1 --2- 172.21.15.151:0/3456312273 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55faa54aead0 0x55faa54aef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:50.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.353+0000 7f5863400700 1 -- 172.21.15.151:0/3456312273 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55faa54b4020 msgr2=0x55faa54b4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:50.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.353+0000 7f5863400700 1 --2- 172.21.15.151:0/3456312273 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55faa54b4020 0x55faa54b4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:50.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.353+0000 7f5863400700 1 -- 172.21.15.151:0/3456312273 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55faa53c5420 con 0x55faa54b4940 2024-05-21T21:58:50.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.354+0000 7f5860c00700 1 -- 172.21.15.151:0/1412845536 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f584c0e2890 con 0x7f5864103e20 2024-05-21T21:58:50.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.354+0000 7f5863e00700 1 --2- 172.21.15.151:0/1412845536 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f58480b29f0 0x7f58480b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:50.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.354+0000 7f5863400700 1 --2- 172.21.15.151:0/3456312273 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55faa54b4940 0x55faa54ae4b0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f5854009990 tx=0x7f5854005bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4e4b1ccd11a1300 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:58:50.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.354+0000 7f5858c00700 1 -- 172.21.15.151:0/3456312273 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5854027070 con 0x55faa54b4940 2024-05-21T21:58:50.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.354+0000 7f5858c00700 1 -- 172.21.15.151:0/3456312273 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5854003680 con 0x55faa54b4940 2024-05-21T21:58:50.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.354+0000 7f5858c00700 1 -- 172.21.15.151:0/3456312273 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f585400f6c0 con 0x55faa54b4940 2024-05-21T21:58:50.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.355+0000 7f5874afbb80 1 -- 172.21.15.151:0/3456312273 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55faa54b4940 msgr2=0x55faa54ae4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:50.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.355+0000 7f5863e00700 1 --2- 172.21.15.151:0/1412845536 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f58480b29f0 0x7f58480b4eb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f585c009790 tx=0x7f585c004ad0 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:58:50.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.355+0000 7f5874afbb80 1 --2- 172.21.15.151:0/3456312273 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55faa54b4940 0x55faa54ae4b0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f5854009990 tx=0x7f5854005bc0 comp rx=0 tx=0).stop 2024-05-21T21:58:50.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.355+0000 7f5874afbb80 1 -- 172.21.15.151:0/3456312273 shutdown_connections 2024-05-21T21:58:50.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.355+0000 7f5874afbb80 1 --2- 172.21.15.151:0/3456312273 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55faa54b4020 0x55faa54b4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:50.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.355+0000 7f5874afbb80 1 --2- 172.21.15.151:0/3456312273 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55faa54aead0 0x55faa54aef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:50.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.355+0000 7f5874afbb80 1 --2- 172.21.15.151:0/3456312273 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55faa54b4940 0x55faa54ae4b0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:50.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.355+0000 7f5874afbb80 1 -- 172.21.15.151:0/3456312273 >> 172.21.15.151:0/3456312273 conn(0x55faa53dcbe0 msgr2=0x55faa54a9150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:58:50.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.355+0000 7f5874afbb80 1 -- 172.21.15.151:0/3456312273 shutdown_connections 2024-05-21T21:58:50.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.356+0000 7f5874afbb80 1 -- 172.21.15.151:0/3456312273 wait complete. 2024-05-21T21:58:50.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.356+0000 7f5874afbb80 1 -- start start 2024-05-21T21:58:50.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.356+0000 7f5874afbb80 1 Processor -- start 2024-05-21T21:58:50.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.357+0000 7f5874afbb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55faa54aead0 0x55faa54ba6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:50.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.357+0000 7f5874afbb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55faa54b4020 0x55faa54b7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:50.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.357+0000 7f5874afbb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55faa54b4940 0x55faa54b7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:50.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.357+0000 7f5874afbb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55faa5302fa0 con 0x55faa54b4940 2024-05-21T21:58:50.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.357+0000 7f5874afbb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55faa539d760 con 0x55faa54aead0 2024-05-21T21:58:50.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.357+0000 7f5874afbb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55faa54bb400 con 0x55faa54b4020 2024-05-21T21:58:50.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.357+0000 7f5874afbb80 10 client.0 _ll_register_callbacks cb 0x55faa53a6c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:58:50.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.357+0000 7f5863e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55faa54b4940 0x55faa54b7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:50.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.357+0000 7f5863400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55faa54aead0 0x55faa54ba6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:50.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.357+0000 7f5863400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55faa54aead0 0x55faa54ba6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:58020/0 (socket says 172.21.15.151:58020) 2024-05-21T21:58:50.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.357+0000 7f5863e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55faa54b4940 0x55faa54b7fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:47006/0 (socket says 172.21.15.151:47006) 2024-05-21T21:58:50.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.357+0000 7f5863400700 1 -- 172.21.15.151:0/2867221595 learned_addr learned my addr 172.21.15.151:0/2867221595 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:58:50.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.357+0000 7f585be00700 1 --2- 172.21.15.151:0/2867221595 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55faa54b4020 0x55faa54b7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:50.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.358+0000 7f5863400700 1 -- 172.21.15.151:0/2867221595 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55faa54b4020 msgr2=0x55faa54b7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:50.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.358+0000 7f5863400700 1 --2- 172.21.15.151:0/2867221595 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55faa54b4020 0x55faa54b7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:50.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.358+0000 7f5863400700 1 -- 172.21.15.151:0/2867221595 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55faa54b4940 msgr2=0x55faa54b7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:50.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.358+0000 7f5863400700 1 --2- 172.21.15.151:0/2867221595 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55faa54b4940 0x55faa54b7fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:50.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.358+0000 7f5863400700 1 -- 172.21.15.151:0/2867221595 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55faa534c8d0 con 0x55faa54aead0 2024-05-21T21:58:50.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.358+0000 7f5863400700 1 --2- 172.21.15.151:0/2867221595 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55faa54aead0 0x55faa54ba6d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5854009350 tx=0x7f5854003eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=989c1643fca29f server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:58:50.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.358+0000 7f5851600700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-21T21:58:50.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.358+0000 7f5851600700 1 -- 172.21.15.151:0/2867221595 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5854027040 con 0x55faa54aead0 2024-05-21T21:58:50.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.358+0000 7f5874afbb80 1 -- 172.21.15.151:0/2867221595 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5854003680 con 0x55faa54aead0 2024-05-21T21:58:50.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.358+0000 7f5851600700 1 -- 172.21.15.151:0/2867221595 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f5854005290 con 0x55faa54aead0 2024-05-21T21:58:50.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.359+0000 7f5851600700 1 -- 172.21.15.151:0/2867221595 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5854038050 con 0x55faa54aead0 2024-05-21T21:58:50.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.359+0000 7f5874afbb80 10 client.26030 fetch_fsmap learned FSMap version 29 2024-05-21T21:58:50.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.359+0000 7f5874afbb80 1 -- 172.21.15.151:0/2867221595 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55faa53cd060 con 0x55faa54aead0 2024-05-21T21:58:50.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.359+0000 7f5851600700 1 -- 172.21.15.151:0/2867221595 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5854003680 con 0x55faa54aead0 2024-05-21T21:58:50.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.359+0000 7f5851600700 1 -- 172.21.15.151:0/2867221595 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f5854033030 con 0x55faa54aead0 2024-05-21T21:58:50.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.359+0000 7f5874afbb80 10 client.26030 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T21:58:50.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.359+0000 7f5874afbb80 10 client.26030 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T21:58:50.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.359+0000 7f5874afbb80 20 client.26030 populate_metadata read hostname 'smithi151' 2024-05-21T21:58:50.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.359+0000 7f5874afbb80 1 --2- 172.21.15.151:0/2867221595 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55faa54c7e10 0x55faa54ca200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:50.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.359+0000 7f5874afbb80 4 client.26030 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T21:58:50.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.359+0000 7f5874afbb80 1 -- 172.21.15.151:0/2867221595 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55faa540ed90 con 0x55faa54c7e10 2024-05-21T21:58:50.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.360+0000 7f585be00700 1 --2- 172.21.15.151:0/2867221595 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55faa54c7e10 0x55faa54ca200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:50.363 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.360+0000 7f585be00700 1 --2- 172.21.15.151:0/2867221595 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55faa54c7e10 0x55faa54ca200 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=209a4585de2c0dd4 server_cookie=56bcea4bbe83043 in_seq=0 out_seq=0 2024-05-21T21:58:50.363 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.360+0000 7f5851600700 10 client.26030 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T21:58:50.364 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.361+0000 7f5851600700 1 -- 172.21.15.151:0/2867221595 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55faa540ed90 con 0x55faa54c7e10 2024-05-21T21:58:50.364 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.361+0000 7f5851600700 10 client.26030 handle_command_reply: tid=0 2024-05-21T21:58:50.364 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.361+0000 7f5874afbb80 1 client.26030 shutdown 2024-05-21T21:58:50.364 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.361+0000 7f5874afbb80 1 -- 172.21.15.151:0/2867221595 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55faa54aead0 msgr2=0x55faa54ba6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:50.364 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.361+0000 7f5874afbb80 1 --2- 172.21.15.151:0/2867221595 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55faa54aead0 0x55faa54ba6d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5854009350 tx=0x7f5854003eb0 comp rx=0 tx=0).stop 2024-05-21T21:58:50.364 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.361+0000 7f5874afbb80 1 -- 172.21.15.151:0/2867221595 shutdown_connections 2024-05-21T21:58:50.364 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.361+0000 7f5874afbb80 1 --2- 172.21.15.151:0/2867221595 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55faa54c7e10 0x55faa54ca200 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:50.364 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.361+0000 7f5874afbb80 1 --2- 172.21.15.151:0/2867221595 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55faa54aead0 0x55faa54ba6d0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:50.364 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.361+0000 7f5874afbb80 1 --2- 172.21.15.151:0/2867221595 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55faa54b4940 0x55faa54b7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:50.364 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.361+0000 7f5874afbb80 1 --2- 172.21.15.151:0/2867221595 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55faa54b4020 0x55faa54b7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:50.364 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.361+0000 7f5851600700 0 client.26030 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T21:58:50.364 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.361+0000 7f5874afbb80 1 -- 172.21.15.151:0/2867221595 >> 172.21.15.151:0/2867221595 conn(0x55faa53dcbe0 msgr2=0x55faa54ad4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:58:50.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.361+0000 7f5874afbb80 1 -- 172.21.15.151:0/2867221595 shutdown_connections 2024-05-21T21:58:50.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.362+0000 7f5874afbb80 1 -- 172.21.15.151:0/2867221595 wait complete. 2024-05-21T21:58:50.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.362+0000 7f5874afbb80 20 client.26030 trim_cache size 0 max 16384 2024-05-21T21:58:50.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.378+0000 7f5874afbb80 1 Processor -- start 2024-05-21T21:58:50.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.378+0000 7f5874afbb80 1 -- start start 2024-05-21T21:58:50.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.378+0000 7f5874afbb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55faa54aead0 0x55faa54ae4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:50.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.378+0000 7f5874afbb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55faa54b4020 0x55faa54cd220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:50.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.378+0000 7f5874afbb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55faa54b4940 0x55faa54cd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:50.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.379+0000 7f5874afbb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f58641afe30 con 0x55faa54aead0 2024-05-21T21:58:50.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.379+0000 7f5874afbb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f58641b1970 con 0x55faa54b4020 2024-05-21T21:58:50.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.379+0000 7f5874afbb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f58641b0500 con 0x55faa54b4940 2024-05-21T21:58:50.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.379+0000 7f5863400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55faa54aead0 0x55faa54ae4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:50.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.379+0000 7f585be00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55faa54b4020 0x55faa54cd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:50.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.379+0000 7f585be00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55faa54b4020 0x55faa54cd220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:58032/0 (socket says 172.21.15.151:58032) 2024-05-21T21:58:50.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.379+0000 7f585be00700 1 -- 172.21.15.151:0/3400156887 learned_addr learned my addr 172.21.15.151:0/3400156887 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:58:50.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.379+0000 7f5863e00700 1 --2- 172.21.15.151:0/3400156887 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55faa54b4940 0x55faa54cd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:50.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.379+0000 7f585be00700 1 -- 172.21.15.151:0/3400156887 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55faa54b4940 msgr2=0x55faa54cd760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:50.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.379+0000 7f585be00700 1 --2- 172.21.15.151:0/3400156887 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55faa54b4940 0x55faa54cd760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:50.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.379+0000 7f585be00700 1 -- 172.21.15.151:0/3400156887 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55faa54aead0 msgr2=0x55faa54ae4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:50.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.379+0000 7f585be00700 1 --2- 172.21.15.151:0/3400156887 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55faa54aead0 0x55faa54ae4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:50.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.379+0000 7f585be00700 1 -- 172.21.15.151:0/3400156887 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55faa54b5fe0 con 0x55faa54b4020 2024-05-21T21:58:50.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.380+0000 7f585be00700 1 --2- 172.21.15.151:0/3400156887 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55faa54b4020 0x55faa54cd220 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f584c0a5ba0 tx=0x7f584c091040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6055836d9b37364 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:58:50.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.380+0000 7f5853e00700 1 -- 172.21.15.151:0/3400156887 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f584c0a52b0 con 0x55faa54b4020 2024-05-21T21:58:50.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.380+0000 7f5853e00700 1 -- 172.21.15.151:0/3400156887 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f584c0aae50 con 0x55faa54b4020 2024-05-21T21:58:50.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.380+0000 7f5853e00700 1 -- 172.21.15.151:0/3400156887 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f584c0a98f0 con 0x55faa54b4020 2024-05-21T21:58:50.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.381+0000 7f5874afbb80 1 -- 172.21.15.151:0/3400156887 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55faa54b4020 msgr2=0x55faa54cd220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:50.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.381+0000 7f5874afbb80 1 --2- 172.21.15.151:0/3400156887 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55faa54b4020 0x55faa54cd220 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f584c0a5ba0 tx=0x7f584c091040 comp rx=0 tx=0).stop 2024-05-21T21:58:50.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.381+0000 7f5874afbb80 1 -- 172.21.15.151:0/3400156887 shutdown_connections 2024-05-21T21:58:50.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.381+0000 7f5874afbb80 1 --2- 172.21.15.151:0/3400156887 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55faa54b4940 0x55faa54cd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:50.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.381+0000 7f5874afbb80 1 --2- 172.21.15.151:0/3400156887 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55faa54aead0 0x55faa54ae4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:50.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.381+0000 7f5874afbb80 1 --2- 172.21.15.151:0/3400156887 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55faa54b4020 0x55faa54cd220 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:50.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.381+0000 7f5874afbb80 1 -- 172.21.15.151:0/3400156887 >> 172.21.15.151:0/3400156887 conn(0x55faa53dcbe0 msgr2=0x55faa54a8c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:58:50.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.381+0000 7f5874afbb80 1 -- 172.21.15.151:0/3400156887 shutdown_connections 2024-05-21T21:58:50.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.381+0000 7f5874afbb80 1 -- 172.21.15.151:0/3400156887 wait complete. 2024-05-21T21:58:50.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.381+0000 7f5874afbb80 1 -- start start 2024-05-21T21:58:50.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.382+0000 7f5874afbb80 1 Processor -- start 2024-05-21T21:58:50.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.382+0000 7f5874afbb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55faa54aead0 0x55faa54cda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:50.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.382+0000 7f5874afbb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55faa54b4020 0x55faa54cae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:50.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.382+0000 7f5874afbb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55faa54b4940 0x55faa54cb410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:50.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.382+0000 7f5874afbb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f58641b12a0 con 0x55faa54b4020 2024-05-21T21:58:50.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.382+0000 7f5874afbb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f58641b0bd0 con 0x55faa54b4940 2024-05-21T21:58:50.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.382+0000 7f5874afbb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f58641addd0 con 0x55faa54aead0 2024-05-21T21:58:50.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.382+0000 7f5874afbb80 10 client.0 _ll_register_callbacks cb 0x55faa53a6c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:58:50.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.382+0000 7f585be00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55faa54b4020 0x55faa54cae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:50.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.383+0000 7f585be00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55faa54b4020 0x55faa54cae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:47024/0 (socket says 172.21.15.151:47024) 2024-05-21T21:58:50.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.383+0000 7f585be00700 1 -- 172.21.15.151:0/54999281 learned_addr learned my addr 172.21.15.151:0/54999281 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:58:50.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.383+0000 7f5863400700 1 --2- 172.21.15.151:0/54999281 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55faa54aead0 0x55faa54cda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:50.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.383+0000 7f5863e00700 1 --2- 172.21.15.151:0/54999281 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55faa54b4940 0x55faa54cb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:50.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.383+0000 7f585be00700 1 -- 172.21.15.151:0/54999281 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55faa54aead0 msgr2=0x55faa54cda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:50.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.383+0000 7f585be00700 1 --2- 172.21.15.151:0/54999281 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55faa54aead0 0x55faa54cda10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:50.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.383+0000 7f585be00700 1 -- 172.21.15.151:0/54999281 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55faa54b4940 msgr2=0x55faa54cb410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:50.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.383+0000 7f585be00700 1 --2- 172.21.15.151:0/54999281 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55faa54b4940 0x55faa54cb410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:50.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.383+0000 7f585be00700 1 -- 172.21.15.151:0/54999281 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55faa5531650 con 0x55faa54b4020 2024-05-21T21:58:50.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.383+0000 7f5863400700 1 --2- 172.21.15.151:0/54999281 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55faa54aead0 0x55faa54cda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:58:50.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.383+0000 7f585be00700 1 --2- 172.21.15.151:0/54999281 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55faa54b4020 0x55faa54cae00 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f584c000c00 tx=0x7f584c0dff60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a132b3548a6ec677 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:58:50.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.383+0000 7f5851600700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-21T21:58:50.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.383+0000 7f5851600700 1 -- 172.21.15.151:0/54999281 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f584c0c3060 con 0x55faa54b4020 2024-05-21T21:58:50.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.384+0000 7f5851600700 1 -- 172.21.15.151:0/54999281 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f584c0a9d80 con 0x55faa54b4020 2024-05-21T21:58:50.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.384+0000 7f5874afbb80 1 -- 172.21.15.151:0/54999281 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f584c0aae50 con 0x55faa54b4020 2024-05-21T21:58:50.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.384+0000 7f5851600700 1 -- 172.21.15.151:0/54999281 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f584c0a98f0 con 0x55faa54b4020 2024-05-21T21:58:50.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.384+0000 7f5874afbb80 10 client.15906 fetch_fsmap learned FSMap version 29 2024-05-21T21:58:50.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.384+0000 7f5874afbb80 1 -- 172.21.15.151:0/54999281 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55faa5532790 con 0x55faa54b4020 2024-05-21T21:58:50.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.384+0000 7f5851600700 1 -- 172.21.15.151:0/54999281 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f584c0aae50 con 0x55faa54b4020 2024-05-21T21:58:50.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.384+0000 7f5851600700 1 -- 172.21.15.151:0/54999281 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f584c0b24f0 con 0x55faa54b4020 2024-05-21T21:58:50.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.384+0000 7f5874afbb80 10 client.15906 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T21:58:50.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.384+0000 7f5874afbb80 10 client.15906 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T21:58:50.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.384+0000 7f5874afbb80 20 client.15906 populate_metadata read hostname 'smithi151' 2024-05-21T21:58:50.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.384+0000 7f5874afbb80 1 --2- 172.21.15.151:0/54999281 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55faa5532e40 0x55faa5535260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:50.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.384+0000 7f5874afbb80 4 client.15906 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T21:58:50.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.384+0000 7f5874afbb80 1 -- 172.21.15.151:0/54999281 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55faa5408570 con 0x55faa5532e40 2024-05-21T21:58:50.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.385+0000 7f5863400700 1 --2- 172.21.15.151:0/54999281 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55faa5532e40 0x55faa5535260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:50.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.385+0000 7f5863400700 1 --2- 172.21.15.151:0/54999281 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55faa5532e40 0x55faa5535260 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=941a41833f1f9df9 server_cookie=6344a84953f7ae2b in_seq=0 out_seq=0 2024-05-21T21:58:50.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.385+0000 7f5851600700 10 client.15906 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T21:58:50.389 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.385+0000 7f5851600700 1 -- 172.21.15.151:0/54999281 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55faa5408570 con 0x55faa5532e40 2024-05-21T21:58:50.389 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.385+0000 7f5851600700 10 client.15906 handle_command_reply: tid=0 2024-05-21T21:58:50.389 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.385+0000 7f5874afbb80 1 client.15906 shutdown 2024-05-21T21:58:50.389 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.386+0000 7f5874afbb80 1 -- 172.21.15.151:0/54999281 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55faa54b4020 msgr2=0x55faa54cae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:50.389 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.386+0000 7f5874afbb80 1 --2- 172.21.15.151:0/54999281 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55faa54b4020 0x55faa54cae00 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f584c000c00 tx=0x7f584c0dff60 comp rx=0 tx=0).stop 2024-05-21T21:58:50.389 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.386+0000 7f5874afbb80 1 -- 172.21.15.151:0/54999281 shutdown_connections 2024-05-21T21:58:50.389 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.386+0000 7f5874afbb80 1 --2- 172.21.15.151:0/54999281 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55faa5532e40 0x55faa5535260 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:50.389 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.386+0000 7f5874afbb80 1 --2- 172.21.15.151:0/54999281 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55faa54b4940 0x55faa54cb410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:50.389 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.386+0000 7f5874afbb80 1 --2- 172.21.15.151:0/54999281 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55faa54aead0 0x55faa54cda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:50.389 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.386+0000 7f5874afbb80 1 --2- 172.21.15.151:0/54999281 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55faa54b4020 0x55faa54cae00 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:50.389 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.386+0000 7f5851600700 0 client.15906 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T21:58:50.389 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.386+0000 7f5874afbb80 1 -- 172.21.15.151:0/54999281 >> 172.21.15.151:0/54999281 conn(0x55faa53dcbe0 msgr2=0x55faa54b4d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:58:50.389 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.386+0000 7f5874afbb80 1 -- 172.21.15.151:0/54999281 shutdown_connections 2024-05-21T21:58:50.389 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.386+0000 7f5874afbb80 1 -- 172.21.15.151:0/54999281 wait complete. 2024-05-21T21:58:50.389 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.386+0000 7f5874afbb80 20 client.15906 trim_cache size 0 max 16384 2024-05-21T21:58:50.390 INFO:teuthology.orchestra.run.smithi151.stdout:[] 2024-05-21T21:58:50.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.387+0000 7f5868c00700 1 -- 172.21.15.151:0/1412845536 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f58480b29f0 msgr2=0x7f58480b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:50.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.387+0000 7f5868c00700 1 --2- 172.21.15.151:0/1412845536 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f58480b29f0 0x7f58480b4eb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f585c009790 tx=0x7f585c004ad0 comp rx=0 tx=0).stop 2024-05-21T21:58:50.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.387+0000 7f5868c00700 1 -- 172.21.15.151:0/1412845536 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5864103e20 msgr2=0x7f586419c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:50.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.387+0000 7f5868c00700 1 --2- 172.21.15.151:0/1412845536 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5864103e20 0x7f586419c250 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f584c00e9a0 tx=0x7f584c00ecb0 comp rx=0 tx=0).stop 2024-05-21T21:58:50.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.387+0000 7f5868c00700 1 -- 172.21.15.151:0/1412845536 shutdown_connections 2024-05-21T21:58:50.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.387+0000 7f5868c00700 1 --2- 172.21.15.151:0/1412845536 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f58480b29f0 0x7f58480b4eb0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:50.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.387+0000 7f5868c00700 1 --2- 172.21.15.151:0/1412845536 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5864103460 0x7f586419bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:50.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.387+0000 7f5868c00700 1 --2- 172.21.15.151:0/1412845536 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f58641094f0 0x7f58641a05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:50.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.387+0000 7f5868c00700 1 --2- 172.21.15.151:0/1412845536 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5864103e20 0x7f586419c250 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:50.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.387+0000 7f5868c00700 1 -- 172.21.15.151:0/1412845536 >> 172.21.15.151:0/1412845536 conn(0x7f58640fef90 msgr2=0x7f58640ff5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:58:50.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.387+0000 7f5868c00700 1 -- 172.21.15.151:0/1412845536 shutdown_connections 2024-05-21T21:58:50.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:50.387+0000 7f5868c00700 1 -- 172.21.15.151:0/1412845536 wait complete. 2024-05-21T21:58:51.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:58:51 smithi151 ceph-mon[31623]: pgmap v100: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 864 KiB/s wr, 10 op/s 2024-05-21T21:58:51.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:58:51 smithi184 ceph-mon[36942]: pgmap v100: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 864 KiB/s wr, 10 op/s 2024-05-21T21:58:51.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:58:51 smithi164 ceph-mon[36269]: pgmap v100: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 864 KiB/s wr, 10 op/s 2024-05-21T21:58:52.310 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-21T21:58:52.310 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-21T21:58:52.756 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:58:53.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:58:53 smithi184 ceph-mon[36942]: pgmap v101: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 1.3 MiB/s wr, 16 op/s 2024-05-21T21:58:53.738 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:58:53 smithi151 ceph-mon[31623]: pgmap v101: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 1.3 MiB/s wr, 16 op/s 2024-05-21T21:58:53.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:58:53 smithi164 ceph-mon[36269]: pgmap v101: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 1.3 MiB/s wr, 16 op/s 2024-05-21T21:58:54.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.937+0000 7f0a6fc00700 1 -- 172.21.15.151:0/2926190129 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0a681096a0 msgr2=0x7f0a68109a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:54.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.937+0000 7f0a6fc00700 1 --2- 172.21.15.151:0/2926190129 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0a681096a0 0x7f0a68109a80 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c009990 tx=0x7f0a5c009ca0 comp rx=0 tx=0).stop 2024-05-21T21:58:54.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.937+0000 7f0a6fc00700 1 -- 172.21.15.151:0/2926190129 shutdown_connections 2024-05-21T21:58:54.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.937+0000 7f0a6fc00700 1 --2- 172.21.15.151:0/2926190129 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0a68103f50 0x7f0a68110430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:54.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.937+0000 7f0a6fc00700 1 --2- 172.21.15.151:0/2926190129 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0a681096a0 0x7f0a68109a80 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:54.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.937+0000 7f0a6fc00700 1 --2- 172.21.15.151:0/2926190129 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a68103590 0x7f0a68103a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:54.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.937+0000 7f0a6fc00700 1 -- 172.21.15.151:0/2926190129 >> 172.21.15.151:0/2926190129 conn(0x7f0a6807a6f0 msgr2=0x7f0a6807ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:58:54.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.937+0000 7f0a6fc00700 1 -- 172.21.15.151:0/2926190129 shutdown_connections 2024-05-21T21:58:54.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.937+0000 7f0a6fc00700 1 -- 172.21.15.151:0/2926190129 wait complete. 2024-05-21T21:58:54.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.938+0000 7f0a6fc00700 1 Processor -- start 2024-05-21T21:58:54.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.938+0000 7f0a6fc00700 1 -- start start 2024-05-21T21:58:54.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.938+0000 7f0a6fc00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0a68103590 0x7f0a6810a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:54.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.938+0000 7f0a6fc00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0a68103f50 0x7f0a6810a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:54.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.939+0000 7f0a6fc00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a681096a0 0x7f0a6810af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:54.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.939+0000 7f0a6fc00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0a68113780 con 0x7f0a68103590 2024-05-21T21:58:54.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.939+0000 7f0a6fc00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0a68113600 con 0x7f0a681096a0 2024-05-21T21:58:54.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.939+0000 7f0a6fc00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0a68113900 con 0x7f0a68103f50 2024-05-21T21:58:54.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.939+0000 7f0a6d200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0a68103590 0x7f0a6810a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:54.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.939+0000 7f0a6d200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0a68103590 0x7f0a6810a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:47038/0 (socket says 172.21.15.151:47038) 2024-05-21T21:58:54.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.939+0000 7f0a6d200700 1 -- 172.21.15.151:0/1527607671 learned_addr learned my addr 172.21.15.151:0/1527607671 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:58:54.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.939+0000 7f0a67e00700 1 --2- 172.21.15.151:0/1527607671 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0a68103f50 0x7f0a6810a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:54.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.939+0000 7f0a6dc00700 1 --2- 172.21.15.151:0/1527607671 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a681096a0 0x7f0a6810af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:54.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.939+0000 7f0a67e00700 1 -- 172.21.15.151:0/1527607671 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a681096a0 msgr2=0x7f0a6810af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:54.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.939+0000 7f0a67e00700 1 --2- 172.21.15.151:0/1527607671 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a681096a0 0x7f0a6810af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:54.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.939+0000 7f0a67e00700 1 -- 172.21.15.151:0/1527607671 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0a68103590 msgr2=0x7f0a6810a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:54.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.939+0000 7f0a67e00700 1 --2- 172.21.15.151:0/1527607671 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0a68103590 0x7f0a6810a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:54.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.939+0000 7f0a67e00700 1 -- 172.21.15.151:0/1527607671 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a681ac440 con 0x7f0a68103f50 2024-05-21T21:58:54.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.939+0000 7f0a6d200700 1 --2- 172.21.15.151:0/1527607671 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0a68103590 0x7f0a6810a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:58:54.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.940+0000 7f0a67e00700 1 --2- 172.21.15.151:0/1527607671 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0a68103f50 0x7f0a6810a9d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0a5400ea20 tx=0x7f0a5400ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:58:54.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.940+0000 7f0a65600700 1 -- 172.21.15.151:0/1527607671 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a54004220 con 0x7f0a68103f50 2024-05-21T21:58:54.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.940+0000 7f0a6fc00700 1 -- 172.21.15.151:0/1527607671 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a681ac730 con 0x7f0a68103f50 2024-05-21T21:58:54.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.940+0000 7f0a65600700 1 -- 172.21.15.151:0/1527607671 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a5400ce20 con 0x7f0a68103f50 2024-05-21T21:58:54.942 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.940+0000 7f0a65600700 1 -- 172.21.15.151:0/1527607671 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a54003cf0 con 0x7f0a68103f50 2024-05-21T21:58:54.942 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.940+0000 7f0a6fc00700 1 -- 172.21.15.151:0/1527607671 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a681acc70 con 0x7f0a68103f50 2024-05-21T21:58:54.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.945+0000 7f0a7a5deb80 1 Processor -- start 2024-05-21T21:58:54.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.945+0000 7f0a7a5deb80 1 -- start start 2024-05-21T21:58:54.946 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.945+0000 7f0a7a5deb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560091ee6f30 0x560091ee7310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:54.946 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.945+0000 7f0a7a5deb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560091ee7850 0x560091ee12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:54.946 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.945+0000 7f0a7a5deb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560091ee1900 0x560091ee1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:54.946 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.945+0000 7f0a7a5deb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x560091e3fec0 con 0x560091ee7850 2024-05-21T21:58:54.946 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.945+0000 7f0a7a5deb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x560091e2fa60 con 0x560091ee6f30 2024-05-21T21:58:54.946 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.945+0000 7f0a7a5deb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x560091e33aa0 con 0x560091ee1900 2024-05-21T21:58:54.946 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.945+0000 7f0a65600700 1 -- 172.21.15.151:0/1527607671 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7f0a5400c970 con 0x7f0a68103f50 2024-05-21T21:58:54.946 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.945+0000 7f0a6d200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560091ee7850 0x560091ee12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:54.946 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.946+0000 7f0a6dc00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560091ee6f30 0x560091ee7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:54.946 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.946+0000 7f0a6d200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560091ee7850 0x560091ee12e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:47046/0 (socket says 172.21.15.151:47046) 2024-05-21T21:58:54.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.946+0000 7f0a6d200700 1 -- 172.21.15.151:0/1396884260 learned_addr learned my addr 172.21.15.151:0/1396884260 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:58:54.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.946+0000 7f0a67e00700 1 --2- 172.21.15.151:0/1396884260 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560091ee1900 0x560091ee1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:54.948 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.946+0000 7f0a6dc00700 1 -- 172.21.15.151:0/1396884260 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560091ee1900 msgr2=0x560091ee1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:54.948 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.946+0000 7f0a6dc00700 1 --2- 172.21.15.151:0/1396884260 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560091ee1900 0x560091ee1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:54.948 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.946+0000 7f0a6dc00700 1 -- 172.21.15.151:0/1396884260 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560091ee7850 msgr2=0x560091ee12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:54.948 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.946+0000 7f0a6dc00700 1 --2- 172.21.15.151:0/1396884260 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560091ee7850 0x560091ee12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:54.949 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.946+0000 7f0a6dc00700 1 -- 172.21.15.151:0/1396884260 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560091df84a0 con 0x560091ee6f30 2024-05-21T21:58:54.949 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.946+0000 7f0a6d200700 1 --2- 172.21.15.151:0/1396884260 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560091ee7850 0x560091ee12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:58:54.949 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.946+0000 7f0a67e00700 1 --2- 172.21.15.151:0/1396884260 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560091ee1900 0x560091ee1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T21:58:54.949 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.946+0000 7f0a6dc00700 1 --2- 172.21.15.151:0/1396884260 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560091ee6f30 0x560091ee7310 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0a60011550 tx=0x7f0a60009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=360a95ae89a84aac server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:58:54.949 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.947+0000 7f0a5a000700 1 -- 172.21.15.151:0/1396884260 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a60004bb0 con 0x560091ee6f30 2024-05-21T21:58:54.949 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.947+0000 7f0a5a000700 1 -- 172.21.15.151:0/1396884260 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a60004d50 con 0x560091ee6f30 2024-05-21T21:58:54.949 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.947+0000 7f0a5a000700 1 -- 172.21.15.151:0/1396884260 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a600046a0 con 0x560091ee6f30 2024-05-21T21:58:54.949 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.948+0000 7f0a7a5deb80 1 -- 172.21.15.151:0/1396884260 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560091ee6f30 msgr2=0x560091ee7310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:54.949 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.948+0000 7f0a7a5deb80 1 --2- 172.21.15.151:0/1396884260 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560091ee6f30 0x560091ee7310 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0a60011550 tx=0x7f0a60009920 comp rx=0 tx=0).stop 2024-05-21T21:58:54.949 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.948+0000 7f0a7a5deb80 1 -- 172.21.15.151:0/1396884260 shutdown_connections 2024-05-21T21:58:54.949 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.948+0000 7f0a7a5deb80 1 --2- 172.21.15.151:0/1396884260 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560091ee6f30 0x560091ee7310 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:54.949 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.948+0000 7f0a7a5deb80 1 --2- 172.21.15.151:0/1396884260 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560091ee1900 0x560091ee1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:54.949 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.948+0000 7f0a7a5deb80 1 --2- 172.21.15.151:0/1396884260 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560091ee7850 0x560091ee12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:54.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.948+0000 7f0a7a5deb80 1 -- 172.21.15.151:0/1396884260 >> 172.21.15.151:0/1396884260 conn(0x560091e0fc60 msgr2=0x560091edc090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:58:54.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.948+0000 7f0a65600700 1 --2- 172.21.15.151:0/1527607671 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f0a500b2a70 0x7f0a500b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:54.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.948+0000 7f0a7a5deb80 1 -- 172.21.15.151:0/1396884260 shutdown_connections 2024-05-21T21:58:54.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.948+0000 7f0a65600700 1 -- 172.21.15.151:0/1527607671 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f0a540e2850 con 0x7f0a68103f50 2024-05-21T21:58:54.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.948+0000 7f0a7a5deb80 1 -- 172.21.15.151:0/1396884260 wait complete. 2024-05-21T21:58:54.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.949+0000 7f0a6d200700 1 --2- 172.21.15.151:0/1527607671 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f0a500b2a70 0x7f0a500b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:54.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.949+0000 7f0a7a5deb80 1 -- start start 2024-05-21T21:58:54.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.949+0000 7f0a7a5deb80 1 Processor -- start 2024-05-21T21:58:54.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.949+0000 7f0a6d200700 1 --2- 172.21.15.151:0/1527607671 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f0a500b2a70 0x7f0a500b4f30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c000c00 tx=0x7f0a5c00b280 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:58:54.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.950+0000 7f0a7a5deb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560091ee1900 0x560091eed4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:54.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.950+0000 7f0a7a5deb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560091ee6f30 0x560091eea880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:54.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.950+0000 7f0a7a5deb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560091ee7850 0x560091eeadc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:54.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.950+0000 7f0a7a5deb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x560091d36020 con 0x560091ee6f30 2024-05-21T21:58:54.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.950+0000 7f0a7a5deb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x560091dd07e0 con 0x560091ee1900 2024-05-21T21:58:54.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.950+0000 7f0a7a5deb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x560091eee2c0 con 0x560091ee7850 2024-05-21T21:58:54.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.950+0000 7f0a7a5deb80 10 client.0 _ll_register_callbacks cb 0x560091dd9cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:58:54.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.950+0000 7f0a67e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560091ee6f30 0x560091eea880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:54.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.950+0000 7f0a67e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560091ee6f30 0x560091eea880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:47058/0 (socket says 172.21.15.151:47058) 2024-05-21T21:58:54.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.950+0000 7f0a67e00700 1 -- 172.21.15.151:0/1559075266 learned_addr learned my addr 172.21.15.151:0/1559075266 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:58:54.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.950+0000 7f0a6d200700 1 --2- 172.21.15.151:0/1559075266 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560091ee1900 0x560091eed4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:54.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.950+0000 7f0a6dc00700 1 --2- 172.21.15.151:0/1559075266 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560091ee7850 0x560091eeadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:54.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.951+0000 7f0a67e00700 1 -- 172.21.15.151:0/1559075266 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560091ee7850 msgr2=0x560091eeadc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:54.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.951+0000 7f0a67e00700 1 --2- 172.21.15.151:0/1559075266 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560091ee7850 0x560091eeadc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:54.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.951+0000 7f0a67e00700 1 -- 172.21.15.151:0/1559075266 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560091ee1900 msgr2=0x560091eed4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:54.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.951+0000 7f0a67e00700 1 --2- 172.21.15.151:0/1559075266 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560091ee1900 0x560091eed4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:54.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.951+0000 7f0a67e00700 1 -- 172.21.15.151:0/1559075266 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560091d7f950 con 0x560091ee6f30 2024-05-21T21:58:54.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.951+0000 7f0a6dc00700 1 --2- 172.21.15.151:0/1559075266 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560091ee7850 0x560091eeadc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:58:54.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.951+0000 7f0a67e00700 1 --2- 172.21.15.151:0/1559075266 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560091ee6f30 0x560091eea880 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f0a5400e9f0 tx=0x7f0a540045c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e927a89030bc2aeb server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:58:54.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.951+0000 7f0a4ea00700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-21T21:58:54.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.951+0000 7f0a4ea00700 1 -- 172.21.15.151:0/1559075266 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a540a98b0 con 0x560091ee6f30 2024-05-21T21:58:54.954 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.951+0000 7f0a4ea00700 1 -- 172.21.15.151:0/1559075266 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f0a540b26c0 con 0x560091ee6f30 2024-05-21T21:58:54.954 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.951+0000 7f0a7a5deb80 1 -- 172.21.15.151:0/1559075266 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a60004d50 con 0x560091ee6f30 2024-05-21T21:58:54.954 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.952+0000 7f0a4ea00700 1 -- 172.21.15.151:0/1559075266 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0a540a9250 con 0x560091ee6f30 2024-05-21T21:58:54.954 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.952+0000 7f0a7a5deb80 10 client.15918 fetch_fsmap learned FSMap version 29 2024-05-21T21:58:54.954 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.952+0000 7f0a7a5deb80 1 -- 172.21.15.151:0/1559075266 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560091e000e0 con 0x560091ee6f30 2024-05-21T21:58:54.956 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.952+0000 7f0a4ea00700 1 -- 172.21.15.151:0/1559075266 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a60004d50 con 0x560091ee6f30 2024-05-21T21:58:54.956 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.952+0000 7f0a4ea00700 1 -- 172.21.15.151:0/1559075266 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f0a540cc0d0 con 0x560091ee6f30 2024-05-21T21:58:54.956 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.952+0000 7f0a7a5deb80 10 client.15918 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T21:58:54.956 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.952+0000 7f0a7a5deb80 10 client.15918 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T21:58:54.956 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.952+0000 7f0a7a5deb80 20 client.15918 populate_metadata read hostname 'smithi151' 2024-05-21T21:58:54.956 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.952+0000 7f0a7a5deb80 1 --2- 172.21.15.151:0/1559075266 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x560091efadd0 0x560091efd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:54.956 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.952+0000 7f0a7a5deb80 4 client.15918 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T21:58:54.956 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.952+0000 7f0a7a5deb80 1 -- 172.21.15.151:0/1559075266 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560091e41e10 con 0x560091efadd0 2024-05-21T21:58:54.956 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.953+0000 7f0a6d200700 1 --2- 172.21.15.151:0/1559075266 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x560091efadd0 0x560091efd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:54.956 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.953+0000 7f0a6d200700 1 --2- 172.21.15.151:0/1559075266 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x560091efadd0 0x560091efd1c0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ceab91738914914 server_cookie=3681f6d742e25c01 in_seq=0 out_seq=0 2024-05-21T21:58:54.956 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.953+0000 7f0a4ea00700 10 client.15918 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T21:58:54.957 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.954+0000 7f0a4ea00700 1 -- 172.21.15.151:0/1559075266 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x560091e41e10 con 0x560091efadd0 2024-05-21T21:58:54.957 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.954+0000 7f0a4ea00700 10 client.15918 handle_command_reply: tid=0 2024-05-21T21:58:54.957 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.954+0000 7f0a7a5deb80 1 client.15918 shutdown 2024-05-21T21:58:54.957 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.954+0000 7f0a7a5deb80 1 -- 172.21.15.151:0/1559075266 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560091ee6f30 msgr2=0x560091eea880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:54.957 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.954+0000 7f0a7a5deb80 1 --2- 172.21.15.151:0/1559075266 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560091ee6f30 0x560091eea880 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f0a5400e9f0 tx=0x7f0a540045c0 comp rx=0 tx=0).stop 2024-05-21T21:58:54.957 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.955+0000 7f0a7a5deb80 1 -- 172.21.15.151:0/1559075266 shutdown_connections 2024-05-21T21:58:54.957 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.955+0000 7f0a7a5deb80 1 --2- 172.21.15.151:0/1559075266 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x560091efadd0 0x560091efd1c0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:54.957 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.955+0000 7f0a7a5deb80 1 --2- 172.21.15.151:0/1559075266 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560091ee1900 0x560091eed4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:54.957 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.955+0000 7f0a7a5deb80 1 --2- 172.21.15.151:0/1559075266 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560091ee7850 0x560091eeadc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:54.957 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.955+0000 7f0a7a5deb80 1 --2- 172.21.15.151:0/1559075266 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560091ee6f30 0x560091eea880 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:54.957 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.955+0000 7f0a4ea00700 0 client.15918 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T21:58:54.958 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.955+0000 7f0a7a5deb80 1 -- 172.21.15.151:0/1559075266 >> 172.21.15.151:0/1559075266 conn(0x560091e0fc60 msgr2=0x560091ee02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:58:54.958 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.955+0000 7f0a7a5deb80 1 -- 172.21.15.151:0/1559075266 shutdown_connections 2024-05-21T21:58:54.958 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.955+0000 7f0a7a5deb80 1 -- 172.21.15.151:0/1559075266 wait complete. 2024-05-21T21:58:54.958 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.955+0000 7f0a7a5deb80 20 client.15918 trim_cache size 0 max 16384 2024-05-21T21:58:54.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.977+0000 7f0a7a5deb80 1 Processor -- start 2024-05-21T21:58:54.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.977+0000 7f0a7a5deb80 1 -- start start 2024-05-21T21:58:54.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.977+0000 7f0a7a5deb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560091ee1900 0x560091ee12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:54.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.977+0000 7f0a7a5deb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560091ee6f30 0x560091ee84c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:54.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.977+0000 7f0a7a5deb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560091ee7850 0x560091ee9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:54.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.977+0000 7f0a7a5deb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0a681b4360 con 0x560091ee7850 2024-05-21T21:58:54.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.977+0000 7f0a7a5deb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0a681b5ea0 con 0x560091ee6f30 2024-05-21T21:58:54.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.977+0000 7f0a7a5deb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0a681b4a30 con 0x560091ee1900 2024-05-21T21:58:54.979 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.978+0000 7f0a67e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560091ee6f30 0x560091ee84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:54.979 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.978+0000 7f0a6dc00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560091ee7850 0x560091ee9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:54.979 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.978+0000 7f0a67e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560091ee6f30 0x560091ee84c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:58092/0 (socket says 172.21.15.151:58092) 2024-05-21T21:58:54.979 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.978+0000 7f0a67e00700 1 -- 172.21.15.151:0/2488085729 learned_addr learned my addr 172.21.15.151:0/2488085729 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:58:54.979 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.978+0000 7f0a6d200700 1 --2- 172.21.15.151:0/2488085729 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560091ee1900 0x560091ee12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:54.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.978+0000 7f0a6dc00700 1 -- 172.21.15.151:0/2488085729 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560091ee1900 msgr2=0x560091ee12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:54.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.978+0000 7f0a6dc00700 1 --2- 172.21.15.151:0/2488085729 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560091ee1900 0x560091ee12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:54.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.978+0000 7f0a6dc00700 1 -- 172.21.15.151:0/2488085729 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560091ee6f30 msgr2=0x560091ee84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:54.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.978+0000 7f0a6dc00700 1 --2- 172.21.15.151:0/2488085729 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560091ee6f30 0x560091ee84c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:54.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.978+0000 7f0a6dc00700 1 -- 172.21.15.151:0/2488085729 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560091edcb60 con 0x560091ee7850 2024-05-21T21:58:54.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.978+0000 7f0a6d200700 1 --2- 172.21.15.151:0/2488085729 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560091ee1900 0x560091ee12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:58:54.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.979+0000 7f0a6dc00700 1 --2- 172.21.15.151:0/2488085729 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560091ee7850 0x560091ee9880 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f0a60009e10 tx=0x7f0a60009d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6877abbc6f7acd64 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:58:54.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.979+0000 7f0a59600700 1 -- 172.21.15.151:0/2488085729 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a6002b070 con 0x560091ee7850 2024-05-21T21:58:54.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.979+0000 7f0a59600700 1 -- 172.21.15.151:0/2488085729 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a60003f10 con 0x560091ee7850 2024-05-21T21:58:54.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.979+0000 7f0a59600700 1 -- 172.21.15.151:0/2488085729 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a60007a10 con 0x560091ee7850 2024-05-21T21:58:54.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.980+0000 7f0a7a5deb80 1 -- 172.21.15.151:0/2488085729 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560091ee7850 msgr2=0x560091ee9880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:54.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.980+0000 7f0a7a5deb80 1 --2- 172.21.15.151:0/2488085729 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560091ee7850 0x560091ee9880 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f0a60009e10 tx=0x7f0a60009d80 comp rx=0 tx=0).stop 2024-05-21T21:58:54.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.980+0000 7f0a7a5deb80 1 -- 172.21.15.151:0/2488085729 shutdown_connections 2024-05-21T21:58:54.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.980+0000 7f0a7a5deb80 1 --2- 172.21.15.151:0/2488085729 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560091ee7850 0x560091ee9880 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:54.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.980+0000 7f0a7a5deb80 1 --2- 172.21.15.151:0/2488085729 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560091ee1900 0x560091ee12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:54.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.980+0000 7f0a7a5deb80 1 --2- 172.21.15.151:0/2488085729 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560091ee6f30 0x560091ee84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:54.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.980+0000 7f0a7a5deb80 1 -- 172.21.15.151:0/2488085729 >> 172.21.15.151:0/2488085729 conn(0x560091e0fc60 msgr2=0x560091ee9170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:58:54.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.980+0000 7f0a7a5deb80 1 -- 172.21.15.151:0/2488085729 shutdown_connections 2024-05-21T21:58:54.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.980+0000 7f0a7a5deb80 1 -- 172.21.15.151:0/2488085729 wait complete. 2024-05-21T21:58:54.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.980+0000 7f0a7a5deb80 1 -- start start 2024-05-21T21:58:54.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.981+0000 7f0a7a5deb80 1 Processor -- start 2024-05-21T21:58:54.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.981+0000 7f0a7a5deb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560091ee1900 0x560091ef0520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:54.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.981+0000 7f0a7a5deb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560091ee6f30 0x560091ef0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:54.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.981+0000 7f0a7a5deb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560091ee7850 0x560091ef0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:54.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.981+0000 7f0a7a5deb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0a681b57d0 con 0x560091ee1900 2024-05-21T21:58:54.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.981+0000 7f0a7a5deb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0a681b5100 con 0x560091ee7850 2024-05-21T21:58:54.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.981+0000 7f0a7a5deb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0a681b2300 con 0x560091ee6f30 2024-05-21T21:58:54.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.981+0000 7f0a7a5deb80 10 client.0 _ll_register_callbacks cb 0x560091dd9cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:58:54.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.981+0000 7f0a6d200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560091ee1900 0x560091ef0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:54.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.982+0000 7f0a6d200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560091ee1900 0x560091ef0520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:47064/0 (socket says 172.21.15.151:47064) 2024-05-21T21:58:54.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.982+0000 7f0a67e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560091ee6f30 0x560091ef0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:54.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.982+0000 7f0a6d200700 1 -- 172.21.15.151:0/2907383424 learned_addr learned my addr 172.21.15.151:0/2907383424 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:58:54.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.982+0000 7f0a6dc00700 1 --2- 172.21.15.151:0/2907383424 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560091ee7850 0x560091ef0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:54.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.982+0000 7f0a67e00700 1 -- 172.21.15.151:0/2907383424 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560091ee7850 msgr2=0x560091ef0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:54.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.982+0000 7f0a67e00700 1 --2- 172.21.15.151:0/2907383424 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560091ee7850 0x560091ef0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:54.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.982+0000 7f0a67e00700 1 -- 172.21.15.151:0/2907383424 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560091ee1900 msgr2=0x560091ef0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:54.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.982+0000 7f0a67e00700 1 --2- 172.21.15.151:0/2907383424 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560091ee1900 0x560091ef0520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:54.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.982+0000 7f0a67e00700 1 -- 172.21.15.151:0/2907383424 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560091f65b90 con 0x560091ee6f30 2024-05-21T21:58:54.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.982+0000 7f0a6d200700 1 --2- 172.21.15.151:0/2907383424 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560091ee1900 0x560091ef0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:58:54.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.982+0000 7f0a67e00700 1 --2- 172.21.15.151:0/2907383424 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560091ee6f30 0x560091ef0a60 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0a540a5b60 tx=0x7f0a540a5ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f68340f86c4b580 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:58:54.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.982+0000 7f0a4ea00700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-21T21:58:54.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.982+0000 7f0a4ea00700 1 -- 172.21.15.151:0/2907383424 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a5406f070 con 0x560091ee6f30 2024-05-21T21:58:54.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.983+0000 7f0a4ea00700 1 -- 172.21.15.151:0/2907383424 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f0a540a8f50 con 0x560091ee6f30 2024-05-21T21:58:54.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.983+0000 7f0a7a5deb80 1 -- 172.21.15.151:0/2907383424 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a60003f10 con 0x560091ee6f30 2024-05-21T21:58:54.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.983+0000 7f0a4ea00700 1 -- 172.21.15.151:0/2907383424 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0a540a9430 con 0x560091ee6f30 2024-05-21T21:58:54.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.983+0000 7f0a7a5deb80 10 client.26059 fetch_fsmap learned FSMap version 29 2024-05-21T21:58:54.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.983+0000 7f0a7a5deb80 1 -- 172.21.15.151:0/2907383424 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560091f66ed0 con 0x560091ee6f30 2024-05-21T21:58:54.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.983+0000 7f0a4ea00700 1 -- 172.21.15.151:0/2907383424 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a60003f10 con 0x560091ee6f30 2024-05-21T21:58:54.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.983+0000 7f0a4ea00700 1 -- 172.21.15.151:0/2907383424 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f0a540a5450 con 0x560091ee6f30 2024-05-21T21:58:54.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.984+0000 7f0a7a5deb80 10 client.26059 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T21:58:54.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.984+0000 7f0a7a5deb80 10 client.26059 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T21:58:54.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.984+0000 7f0a7a5deb80 20 client.26059 populate_metadata read hostname 'smithi151' 2024-05-21T21:58:54.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.984+0000 7f0a7a5deb80 1 --2- 172.21.15.151:0/2907383424 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x560091f67480 0x560091f698a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:54.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.984+0000 7f0a7a5deb80 4 client.26059 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-21T21:58:54.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.984+0000 7f0a7a5deb80 1 -- 172.21.15.151:0/2907383424 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560091e3b5f0 con 0x560091f67480 2024-05-21T21:58:54.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.984+0000 7f0a6d200700 1 --2- 172.21.15.151:0/2907383424 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x560091f67480 0x560091f698a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:54.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.984+0000 7f0a6d200700 1 --2- 172.21.15.151:0/2907383424 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x560091f67480 0x560091f698a0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e00ce158db0d2509 server_cookie=d3f6e40a9c12b472 in_seq=0 out_seq=0 2024-05-21T21:58:54.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.984+0000 7f0a4ea00700 10 client.26059 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T21:58:54.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.986+0000 7f0a4ea00700 1 -- 172.21.15.151:0/2907383424 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560091e3b5f0 con 0x560091f67480 2024-05-21T21:58:54.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.986+0000 7f0a4ea00700 10 client.26059 handle_command_reply: tid=0 2024-05-21T21:58:54.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.986+0000 7f0a7a5deb80 1 client.26059 shutdown 2024-05-21T21:58:54.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.986+0000 7f0a7a5deb80 1 -- 172.21.15.151:0/2907383424 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560091ee6f30 msgr2=0x560091ef0a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:54.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.986+0000 7f0a7a5deb80 1 --2- 172.21.15.151:0/2907383424 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560091ee6f30 0x560091ef0a60 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0a540a5b60 tx=0x7f0a540a5ad0 comp rx=0 tx=0).stop 2024-05-21T21:58:54.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.986+0000 7f0a7a5deb80 1 -- 172.21.15.151:0/2907383424 shutdown_connections 2024-05-21T21:58:54.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.986+0000 7f0a7a5deb80 1 --2- 172.21.15.151:0/2907383424 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x560091f67480 0x560091f698a0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:54.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.986+0000 7f0a7a5deb80 1 --2- 172.21.15.151:0/2907383424 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560091ee7850 0x560091ef0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:54.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.986+0000 7f0a7a5deb80 1 --2- 172.21.15.151:0/2907383424 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560091ee1900 0x560091ef0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:54.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.986+0000 7f0a7a5deb80 1 --2- 172.21.15.151:0/2907383424 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560091ee6f30 0x560091ef0a60 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:54.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.986+0000 7f0a4ea00700 0 client.26059 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T21:58:54.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.986+0000 7f0a7a5deb80 1 -- 172.21.15.151:0/2907383424 >> 172.21.15.151:0/2907383424 conn(0x560091e0fc60 msgr2=0x560091ee5a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:58:54.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.986+0000 7f0a7a5deb80 1 -- 172.21.15.151:0/2907383424 shutdown_connections 2024-05-21T21:58:54.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.986+0000 7f0a7a5deb80 1 -- 172.21.15.151:0/2907383424 wait complete. 2024-05-21T21:58:54.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.986+0000 7f0a7a5deb80 20 client.26059 trim_cache size 0 max 16384 2024-05-21T21:58:54.989 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-05-21T21:58:54.989 INFO:teuthology.orchestra.run.smithi151.stdout: "return_code": 0, 2024-05-21T21:58:54.989 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_tag": "c643db33-a2fa-48c5-b4a8-a7d82b566da9", 2024-05-21T21:58:54.989 INFO:teuthology.orchestra.run.smithi151.stdout: "mode": "asynchronous" 2024-05-21T21:58:54.990 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-05-21T21:58:54.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.987+0000 7f0a6fc00700 1 -- 172.21.15.151:0/1527607671 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f0a500b2a70 msgr2=0x7f0a500b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:54.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.987+0000 7f0a6fc00700 1 --2- 172.21.15.151:0/1527607671 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f0a500b2a70 0x7f0a500b4f30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c000c00 tx=0x7f0a5c00b280 comp rx=0 tx=0).stop 2024-05-21T21:58:54.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.987+0000 7f0a6fc00700 1 -- 172.21.15.151:0/1527607671 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0a68103f50 msgr2=0x7f0a6810a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:54.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.987+0000 7f0a6fc00700 1 --2- 172.21.15.151:0/1527607671 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0a68103f50 0x7f0a6810a9d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0a5400ea20 tx=0x7f0a5400ed30 comp rx=0 tx=0).stop 2024-05-21T21:58:54.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.987+0000 7f0a6fc00700 1 -- 172.21.15.151:0/1527607671 shutdown_connections 2024-05-21T21:58:54.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.987+0000 7f0a6fc00700 1 --2- 172.21.15.151:0/1527607671 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f0a500b2a70 0x7f0a500b4f30 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:54.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.987+0000 7f0a6fc00700 1 --2- 172.21.15.151:0/1527607671 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a681096a0 0x7f0a6810af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:54.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.987+0000 7f0a6fc00700 1 --2- 172.21.15.151:0/1527607671 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0a68103590 0x7f0a6810a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:54.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.987+0000 7f0a6fc00700 1 --2- 172.21.15.151:0/1527607671 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0a68103f50 0x7f0a6810a9d0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:54.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.987+0000 7f0a6fc00700 1 -- 172.21.15.151:0/1527607671 >> 172.21.15.151:0/1527607671 conn(0x7f0a6807a6f0 msgr2=0x7f0a6810ea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:58:54.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.987+0000 7f0a6fc00700 1 -- 172.21.15.151:0/1527607671 shutdown_connections 2024-05-21T21:58:54.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:54.988+0000 7f0a6fc00700 1 -- 172.21.15.151:0/1527607671 wait complete. 2024-05-21T21:58:55.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:58:55 smithi151 ceph-mon[31623]: pgmap v102: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 864 KiB/s wr, 10 op/s 2024-05-21T21:58:55.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:58:55 smithi151 ceph-mon[31623]: scrub queued for path: / 2024-05-21T21:58:55.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:58:55 smithi151 ceph-mon[31623]: scrub summary: idle+waiting paths [/] 2024-05-21T21:58:55.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:58:55 smithi151 ceph-mon[31623]: scrub summary: active paths [/] 2024-05-21T21:58:55.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:58:55 smithi151 ceph-mon[31623]: scrub summary: idle+waiting paths [/] 2024-05-21T21:58:55.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:58:55 smithi184 ceph-mon[36942]: pgmap v102: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 864 KiB/s wr, 10 op/s 2024-05-21T21:58:55.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:58:55 smithi184 ceph-mon[36942]: scrub queued for path: / 2024-05-21T21:58:55.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:58:55 smithi184 ceph-mon[36942]: scrub summary: idle+waiting paths [/] 2024-05-21T21:58:55.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:58:55 smithi184 ceph-mon[36942]: scrub summary: active paths [/] 2024-05-21T21:58:55.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:58:55 smithi184 ceph-mon[36942]: scrub summary: idle+waiting paths [/] 2024-05-21T21:58:55.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:58:55 smithi164 ceph-mon[36269]: pgmap v102: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 864 KiB/s wr, 10 op/s 2024-05-21T21:58:55.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:58:55 smithi164 ceph-mon[36269]: scrub queued for path: / 2024-05-21T21:58:55.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:58:55 smithi164 ceph-mon[36269]: scrub summary: idle+waiting paths [/] 2024-05-21T21:58:55.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:58:55 smithi164 ceph-mon[36269]: scrub summary: active paths [/] 2024-05-21T21:58:55.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:58:55 smithi164 ceph-mon[36269]: scrub summary: idle+waiting paths [/] 2024-05-21T21:58:55.859 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-21T21:58:56.304 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:58:56.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:58:56 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:58:56.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:58:56 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:58:56.791 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:58:56 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:58:57.696 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:58:57 smithi151 ceph-mon[31623]: pgmap v103: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 864 KiB/s wr, 10 op/s 2024-05-21T21:58:57.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:58:57 smithi184 ceph-mon[36942]: pgmap v103: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 864 KiB/s wr, 10 op/s 2024-05-21T21:58:57.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:58:57 smithi164 ceph-mon[36269]: pgmap v103: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 864 KiB/s wr, 10 op/s 2024-05-21T21:58:58.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.477+0000 7effcec00700 1 -- 172.21.15.151:0/3950271501 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7effc810fd10 msgr2=0x7effc8112120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:58.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.477+0000 7effcec00700 1 --2- 172.21.15.151:0/3950271501 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7effc810fd10 0x7effc8112120 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7effc0009f90 tx=0x7effc000b330 comp rx=0 tx=0).stop 2024-05-21T21:58:58.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.477+0000 7effcec00700 1 -- 172.21.15.151:0/3950271501 shutdown_connections 2024-05-21T21:58:58.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.477+0000 7effcec00700 1 --2- 172.21.15.151:0/3950271501 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7effc810fd10 0x7effc8112120 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:58.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.477+0000 7effcec00700 1 --2- 172.21.15.151:0/3950271501 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7effc8075910 0x7effc8075cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:58.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.477+0000 7effcec00700 1 --2- 172.21.15.151:0/3950271501 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7effc8076230 0x7effc810f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:58.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.477+0000 7effcec00700 1 -- 172.21.15.151:0/3950271501 >> 172.21.15.151:0/3950271501 conn(0x7effc80fefc0 msgr2=0x7effc81013e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:58:58.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.477+0000 7effcec00700 1 -- 172.21.15.151:0/3950271501 shutdown_connections 2024-05-21T21:58:58.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.477+0000 7effcec00700 1 -- 172.21.15.151:0/3950271501 wait complete. 2024-05-21T21:58:58.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.478+0000 7effcec00700 1 Processor -- start 2024-05-21T21:58:58.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.479+0000 7effcec00700 1 -- start start 2024-05-21T21:58:58.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.479+0000 7effcec00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7effc8075910 0x7effc81a1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:58.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.479+0000 7effcec00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7effc8076230 0x7effc81a1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:58.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.479+0000 7effcec00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7effc810fd10 0x7effc819bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:58.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.479+0000 7effcec00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7effc8115890 con 0x7effc810fd10 2024-05-21T21:58:58.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.479+0000 7effcec00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7effc8115710 con 0x7effc8076230 2024-05-21T21:58:58.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.479+0000 7effcec00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7effc8115a10 con 0x7effc8075910 2024-05-21T21:58:58.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.479+0000 7effce200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7effc810fd10 0x7effc819bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:58.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.479+0000 7effce200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7effc810fd10 0x7effc819bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:35500/0 (socket says 172.21.15.151:35500) 2024-05-21T21:58:58.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.479+0000 7effce200700 1 -- 172.21.15.151:0/2536773837 learned_addr learned my addr 172.21.15.151:0/2536773837 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:58:58.481 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.479+0000 7effcd800700 1 --2- 172.21.15.151:0/2536773837 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7effc8075910 0x7effc81a1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:58.481 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.479+0000 7effcce00700 1 --2- 172.21.15.151:0/2536773837 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7effc8076230 0x7effc81a1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:58.481 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.480+0000 7effce200700 1 -- 172.21.15.151:0/2536773837 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7effc8075910 msgr2=0x7effc81a1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:58.481 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.480+0000 7effce200700 1 --2- 172.21.15.151:0/2536773837 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7effc8075910 0x7effc81a1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:58.481 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.480+0000 7effce200700 1 -- 172.21.15.151:0/2536773837 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7effc8076230 msgr2=0x7effc81a1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:58.481 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.480+0000 7effce200700 1 --2- 172.21.15.151:0/2536773837 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7effc8076230 0x7effc81a1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:58.481 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.480+0000 7effce200700 1 -- 172.21.15.151:0/2536773837 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effc819c3f0 con 0x7effc810fd10 2024-05-21T21:58:58.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.480+0000 7effce200700 1 --2- 172.21.15.151:0/2536773837 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7effc810fd10 0x7effc819bb10 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7effc0009f60 tx=0x7effc0009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:58:58.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.480+0000 7effc6000700 1 -- 172.21.15.151:0/2536773837 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc0021070 con 0x7effc810fd10 2024-05-21T21:58:58.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.480+0000 7effc6000700 1 -- 172.21.15.151:0/2536773837 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7effc0007ac0 con 0x7effc810fd10 2024-05-21T21:58:58.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.480+0000 7effc6000700 1 -- 172.21.15.151:0/2536773837 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc0005040 con 0x7effc810fd10 2024-05-21T21:58:58.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.480+0000 7effcec00700 1 -- 172.21.15.151:0/2536773837 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effc819c680 con 0x7effc810fd10 2024-05-21T21:58:58.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.481+0000 7effcec00700 1 -- 172.21.15.151:0/2536773837 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effc81a89c0 con 0x7effc810fd10 2024-05-21T21:58:58.484 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.484+0000 7effc6000700 1 -- 172.21.15.151:0/2536773837 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7effc0024070 con 0x7effc810fd10 2024-05-21T21:58:58.486 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.486+0000 7effdab0cb80 1 Processor -- start 2024-05-21T21:58:58.486 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.486+0000 7effdab0cb80 1 -- start start 2024-05-21T21:58:58.486 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.486+0000 7effc6000700 1 --2- 172.21.15.151:0/2536773837 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7effb00b29f0 0x7effb00b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:58.487 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.486+0000 7effdab0cb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56385e6d97c0 0x56385e6d9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:58.487 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.486+0000 7effc6000700 1 -- 172.21.15.151:0/2536773837 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7effc000f020 con 0x7effc810fd10 2024-05-21T21:58:58.487 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.486+0000 7effdab0cb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56385e6da0e0 0x56385e6d3d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:58.487 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.486+0000 7effdab0cb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56385e6d4390 0x56385e6d47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:58.487 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.486+0000 7effdab0cb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56385e631e40 con 0x56385e6d97c0 2024-05-21T21:58:58.487 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.486+0000 7effdab0cb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56385e6219e0 con 0x56385e6da0e0 2024-05-21T21:58:58.487 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.486+0000 7effdab0cb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56385e625a20 con 0x56385e6d4390 2024-05-21T21:58:58.487 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.486+0000 7effcd800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56385e6d97c0 0x56385e6d9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:58.487 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.487+0000 7effcd800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56385e6d97c0 0x56385e6d9ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:35514/0 (socket says 172.21.15.151:35514) 2024-05-21T21:58:58.487 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.487+0000 7effcd800700 1 -- 172.21.15.151:0/2372200727 learned_addr learned my addr 172.21.15.151:0/2372200727 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:58:58.487 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.487+0000 7effce200700 1 --2- 172.21.15.151:0/2372200727 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56385e6d4390 0x56385e6d47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:58.488 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.487+0000 7effcce00700 1 --2- 172.21.15.151:0/2536773837 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7effb00b29f0 0x7effb00b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:58.488 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.487+0000 7effcd800700 1 -- 172.21.15.151:0/2372200727 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56385e6d4390 msgr2=0x56385e6d47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:58.488 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.487+0000 7effcd800700 1 --2- 172.21.15.151:0/2372200727 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56385e6d4390 0x56385e6d47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:58.488 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.487+0000 7effcd800700 1 -- 172.21.15.151:0/2372200727 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56385e6da0e0 msgr2=0x56385e6d3d70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-21T21:58:58.488 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.487+0000 7effcd800700 1 --2- 172.21.15.151:0/2372200727 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56385e6da0e0 0x56385e6d3d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:58.488 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.487+0000 7effcd800700 1 -- 172.21.15.151:0/2372200727 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56385e5ea420 con 0x56385e6d97c0 2024-05-21T21:58:58.488 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.487+0000 7effcd800700 1 --2- 172.21.15.151:0/2372200727 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56385e6d97c0 0x56385e6d9ba0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7effbc009d80 tx=0x7effbc011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33688af95b5690a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:58:58.489 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.488+0000 7effbaa00700 1 -- 172.21.15.151:0/2372200727 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effbc004230 con 0x56385e6d97c0 2024-05-21T21:58:58.489 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.488+0000 7effbaa00700 1 -- 172.21.15.151:0/2372200727 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7effbc0043d0 con 0x56385e6d97c0 2024-05-21T21:58:58.489 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.488+0000 7effcce00700 1 --2- 172.21.15.151:0/2536773837 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7effb00b29f0 0x7effb00b4eb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7effc819d260 tx=0x7effb4009310 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:58:58.489 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.488+0000 7effbaa00700 1 -- 172.21.15.151:0/2372200727 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effbc019830 con 0x56385e6d97c0 2024-05-21T21:58:58.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.488+0000 7effdab0cb80 1 -- 172.21.15.151:0/2372200727 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56385e6d97c0 msgr2=0x56385e6d9ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:58.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.488+0000 7effdab0cb80 1 --2- 172.21.15.151:0/2372200727 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56385e6d97c0 0x56385e6d9ba0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7effbc009d80 tx=0x7effbc011720 comp rx=0 tx=0).stop 2024-05-21T21:58:58.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.489+0000 7effdab0cb80 1 -- 172.21.15.151:0/2372200727 shutdown_connections 2024-05-21T21:58:58.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.489+0000 7effdab0cb80 1 --2- 172.21.15.151:0/2372200727 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56385e6d4390 0x56385e6d47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:58.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.489+0000 7effdab0cb80 1 --2- 172.21.15.151:0/2372200727 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56385e6d97c0 0x56385e6d9ba0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:58.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.489+0000 7effdab0cb80 1 --2- 172.21.15.151:0/2372200727 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56385e6da0e0 0x56385e6d3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:58.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.489+0000 7effdab0cb80 1 -- 172.21.15.151:0/2372200727 >> 172.21.15.151:0/2372200727 conn(0x56385e601be0 msgr2=0x56385e6ce810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:58:58.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.489+0000 7effdab0cb80 1 -- 172.21.15.151:0/2372200727 shutdown_connections 2024-05-21T21:58:58.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.489+0000 7effdab0cb80 1 -- 172.21.15.151:0/2372200727 wait complete. 2024-05-21T21:58:58.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.489+0000 7effdab0cb80 1 -- start start 2024-05-21T21:58:58.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.490+0000 7effdab0cb80 1 Processor -- start 2024-05-21T21:58:58.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.490+0000 7effdab0cb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56385e6d4390 0x56385e6dfe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:58.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.490+0000 7effdab0cb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56385e6d97c0 0x56385e6dd1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:58.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.490+0000 7effdab0cb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56385e6da0e0 0x56385e6dd710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:58.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.490+0000 7effdab0cb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56385e527fa0 con 0x56385e6d4390 2024-05-21T21:58:58.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.490+0000 7effdab0cb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56385e5c2760 con 0x56385e6d97c0 2024-05-21T21:58:58.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.490+0000 7effdab0cb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56385e6e0c00 con 0x56385e6da0e0 2024-05-21T21:58:58.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.490+0000 7effdab0cb80 10 client.0 _ll_register_callbacks cb 0x56385e5cbc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:58:58.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.491+0000 7effcd800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56385e6d4390 0x56385e6dfe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:58.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.491+0000 7effcce00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56385e6d97c0 0x56385e6dd1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:58.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.491+0000 7effcd800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56385e6d4390 0x56385e6dfe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:35530/0 (socket says 172.21.15.151:35530) 2024-05-21T21:58:58.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.491+0000 7effcd800700 1 -- 172.21.15.151:0/2367590264 learned_addr learned my addr 172.21.15.151:0/2367590264 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:58:58.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.491+0000 7effce200700 1 --2- 172.21.15.151:0/2367590264 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56385e6da0e0 0x56385e6dd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:58.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.491+0000 7effcd800700 1 -- 172.21.15.151:0/2367590264 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56385e6da0e0 msgr2=0x56385e6dd710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:58.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.491+0000 7effcd800700 1 --2- 172.21.15.151:0/2367590264 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56385e6da0e0 0x56385e6dd710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:58.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.491+0000 7effcd800700 1 -- 172.21.15.151:0/2367590264 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56385e6d97c0 msgr2=0x56385e6dd1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:58.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.491+0000 7effcd800700 1 --2- 172.21.15.151:0/2367590264 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56385e6d97c0 0x56385e6dd1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:58.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.491+0000 7effcd800700 1 -- 172.21.15.151:0/2367590264 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56385e5718d0 con 0x56385e6d4390 2024-05-21T21:58:58.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.491+0000 7effcce00700 1 --2- 172.21.15.151:0/2367590264 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56385e6d97c0 0x56385e6dd1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:58:58.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.491+0000 7effcd800700 1 --2- 172.21.15.151:0/2367590264 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56385e6d4390 0x56385e6dfe30 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7effbc000c00 tx=0x7effbc005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=632683dcde9da12f server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:58:58.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.491+0000 7effaf400700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-21T21:58:58.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.492+0000 7effaf400700 1 -- 172.21.15.151:0/2367590264 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effbc027070 con 0x56385e6d4390 2024-05-21T21:58:58.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.492+0000 7effaf400700 1 -- 172.21.15.151:0/2367590264 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7effbc004950 con 0x56385e6d4390 2024-05-21T21:58:58.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.492+0000 7effdab0cb80 1 -- 172.21.15.151:0/2367590264 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7effbc0043d0 con 0x56385e6d4390 2024-05-21T21:58:58.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.492+0000 7effaf400700 1 -- 172.21.15.151:0/2367590264 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7effbc033050 con 0x56385e6d4390 2024-05-21T21:58:58.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.492+0000 7effdab0cb80 10 client.15936 fetch_fsmap learned FSMap version 29 2024-05-21T21:58:58.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.492+0000 7effdab0cb80 1 -- 172.21.15.151:0/2367590264 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56385e5f2060 con 0x56385e6d4390 2024-05-21T21:58:58.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.492+0000 7effaf400700 1 -- 172.21.15.151:0/2367590264 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7effbc0043d0 con 0x56385e6d4390 2024-05-21T21:58:58.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.493+0000 7effaf400700 1 -- 172.21.15.151:0/2367590264 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7effbc023460 con 0x56385e6d4390 2024-05-21T21:58:58.495 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.493+0000 7effdab0cb80 10 client.15936 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T21:58:58.495 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.493+0000 7effdab0cb80 10 client.15936 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T21:58:58.495 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.493+0000 7effdab0cb80 20 client.15936 populate_metadata read hostname 'smithi151' 2024-05-21T21:58:58.495 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.493+0000 7effdab0cb80 1 --2- 172.21.15.151:0/2367590264 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x56385e6ed670 0x56385e6efa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:58.495 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.493+0000 7effdab0cb80 4 client.15936 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T21:58:58.495 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.493+0000 7effdab0cb80 1 -- 172.21.15.151:0/2367590264 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56385e633d90 con 0x56385e6ed670 2024-05-21T21:58:58.495 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.493+0000 7effcce00700 1 --2- 172.21.15.151:0/2367590264 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x56385e6ed670 0x56385e6efa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:58.495 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.494+0000 7effcce00700 1 --2- 172.21.15.151:0/2367590264 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x56385e6ed670 0x56385e6efa60 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b0fb3a3bb77eb5a server_cookie=66174cf1ef1dc4eb in_seq=0 out_seq=0 2024-05-21T21:58:58.495 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.494+0000 7effaf400700 10 client.15936 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T21:58:58.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.494+0000 7effaf400700 1 -- 172.21.15.151:0/2367590264 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x56385e633d90 con 0x56385e6ed670 2024-05-21T21:58:58.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.494+0000 7effaf400700 10 client.15936 handle_command_reply: tid=0 2024-05-21T21:58:58.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.495+0000 7effdab0cb80 1 client.15936 shutdown 2024-05-21T21:58:58.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.495+0000 7effdab0cb80 1 -- 172.21.15.151:0/2367590264 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56385e6d4390 msgr2=0x56385e6dfe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:58.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.495+0000 7effdab0cb80 1 --2- 172.21.15.151:0/2367590264 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56385e6d4390 0x56385e6dfe30 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7effbc000c00 tx=0x7effbc005270 comp rx=0 tx=0).stop 2024-05-21T21:58:58.497 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.495+0000 7effdab0cb80 1 -- 172.21.15.151:0/2367590264 shutdown_connections 2024-05-21T21:58:58.497 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.495+0000 7effdab0cb80 1 --2- 172.21.15.151:0/2367590264 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56385e6da0e0 0x56385e6dd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:58.497 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.495+0000 7effdab0cb80 1 --2- 172.21.15.151:0/2367590264 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56385e6d4390 0x56385e6dfe30 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:58.497 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.495+0000 7effdab0cb80 1 --2- 172.21.15.151:0/2367590264 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x56385e6ed670 0x56385e6efa60 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:58.497 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.495+0000 7effdab0cb80 1 --2- 172.21.15.151:0/2367590264 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56385e6d97c0 0x56385e6dd1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:58.497 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.495+0000 7effaf400700 0 client.15936 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T21:58:58.497 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.495+0000 7effdab0cb80 1 -- 172.21.15.151:0/2367590264 >> 172.21.15.151:0/2367590264 conn(0x56385e601be0 msgr2=0x56385e6cdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:58:58.497 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.495+0000 7effdab0cb80 1 -- 172.21.15.151:0/2367590264 shutdown_connections 2024-05-21T21:58:58.497 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.495+0000 7effdab0cb80 1 -- 172.21.15.151:0/2367590264 wait complete. 2024-05-21T21:58:58.497 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.495+0000 7effdab0cb80 20 client.15936 trim_cache size 0 max 16384 2024-05-21T21:58:58.518 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.518+0000 7effdab0cb80 1 Processor -- start 2024-05-21T21:58:58.518 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.518+0000 7effdab0cb80 1 -- start start 2024-05-21T21:58:58.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.518+0000 7effdab0cb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56385e6d4390 0x56385e6d3d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:58.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.518+0000 7effdab0cb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56385e6d97c0 0x56385e6dc350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:58.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.518+0000 7effdab0cb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56385e6da0e0 0x56385e6ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:58.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.518+0000 7effdab0cb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7effc81afe40 con 0x56385e6d4390 2024-05-21T21:58:58.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.518+0000 7effdab0cb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7effc81b1980 con 0x56385e6da0e0 2024-05-21T21:58:58.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.518+0000 7effdab0cb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7effc81b0510 con 0x56385e6d97c0 2024-05-21T21:58:58.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.518+0000 7effcd800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56385e6d4390 0x56385e6d3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:58.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.519+0000 7effcd800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56385e6d4390 0x56385e6d3d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:35546/0 (socket says 172.21.15.151:35546) 2024-05-21T21:58:58.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.519+0000 7effcd800700 1 -- 172.21.15.151:0/1453995190 learned_addr learned my addr 172.21.15.151:0/1453995190 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:58:58.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.519+0000 7effcce00700 1 --2- 172.21.15.151:0/1453995190 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56385e6d97c0 0x56385e6dc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:58.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.519+0000 7effce200700 1 --2- 172.21.15.151:0/1453995190 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56385e6da0e0 0x56385e6ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:58.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.519+0000 7effcce00700 1 -- 172.21.15.151:0/1453995190 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56385e6da0e0 msgr2=0x56385e6ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:58.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.519+0000 7effcce00700 1 --2- 172.21.15.151:0/1453995190 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56385e6da0e0 0x56385e6ce740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:58.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.519+0000 7effcce00700 1 -- 172.21.15.151:0/1453995190 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56385e6d4390 msgr2=0x56385e6d3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:58.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.519+0000 7effcce00700 1 --2- 172.21.15.151:0/1453995190 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56385e6d4390 0x56385e6d3d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:58.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.519+0000 7effcce00700 1 -- 172.21.15.151:0/1453995190 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56385e6db740 con 0x56385e6d97c0 2024-05-21T21:58:58.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.519+0000 7effcce00700 1 --2- 172.21.15.151:0/1453995190 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56385e6d97c0 0x56385e6dc350 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x56385e6f0170 tx=0x7effb4014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=638fbc9dd512cc65 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:58:58.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.520+0000 7effba000700 1 -- 172.21.15.151:0/1453995190 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb4004830 con 0x56385e6d97c0 2024-05-21T21:58:58.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.520+0000 7effba000700 1 -- 172.21.15.151:0/1453995190 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7effb4005020 con 0x56385e6d97c0 2024-05-21T21:58:58.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.520+0000 7effba000700 1 -- 172.21.15.151:0/1453995190 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb40038e0 con 0x56385e6d97c0 2024-05-21T21:58:58.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.520+0000 7effdab0cb80 1 -- 172.21.15.151:0/1453995190 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56385e6d97c0 msgr2=0x56385e6dc350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:58.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.520+0000 7effdab0cb80 1 --2- 172.21.15.151:0/1453995190 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56385e6d97c0 0x56385e6dc350 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x56385e6f0170 tx=0x7effb4014000 comp rx=0 tx=0).stop 2024-05-21T21:58:58.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.521+0000 7effdab0cb80 1 -- 172.21.15.151:0/1453995190 shutdown_connections 2024-05-21T21:58:58.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.521+0000 7effdab0cb80 1 --2- 172.21.15.151:0/1453995190 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56385e6da0e0 0x56385e6ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:58.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.521+0000 7effdab0cb80 1 --2- 172.21.15.151:0/1453995190 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56385e6d4390 0x56385e6d3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:58.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.521+0000 7effdab0cb80 1 --2- 172.21.15.151:0/1453995190 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56385e6d97c0 0x56385e6dc350 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:58.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.521+0000 7effdab0cb80 1 -- 172.21.15.151:0/1453995190 >> 172.21.15.151:0/1453995190 conn(0x56385e601be0 msgr2=0x56385e6dab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:58:58.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.521+0000 7effdab0cb80 1 -- 172.21.15.151:0/1453995190 shutdown_connections 2024-05-21T21:58:58.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.521+0000 7effdab0cb80 1 -- 172.21.15.151:0/1453995190 wait complete. 2024-05-21T21:58:58.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.521+0000 7effdab0cb80 1 -- start start 2024-05-21T21:58:58.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.521+0000 7effdab0cb80 1 Processor -- start 2024-05-21T21:58:58.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.522+0000 7effdab0cb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56385e6d4390 0x56385e6e6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:58.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.522+0000 7effdab0cb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56385e6d97c0 0x56385e6f4b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:58.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.522+0000 7effdab0cb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56385e6da0e0 0x56385e6f5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:58.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.522+0000 7effdab0cb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7effc81b12b0 con 0x56385e6d97c0 2024-05-21T21:58:58.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.522+0000 7effdab0cb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7effc81b0be0 con 0x56385e6da0e0 2024-05-21T21:58:58.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.522+0000 7effdab0cb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7effc81adde0 con 0x56385e6d4390 2024-05-21T21:58:58.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.522+0000 7effdab0cb80 10 client.0 _ll_register_callbacks cb 0x56385e5cbc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:58:58.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.522+0000 7effcd800700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56385e6d4390 0x56385e6e6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:58.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.522+0000 7effcd800700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56385e6d4390 0x56385e6e6680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:36292/0 (socket says 172.21.15.151:36292) 2024-05-21T21:58:58.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.522+0000 7effcd800700 1 -- 172.21.15.151:0/3647114456 learned_addr learned my addr 172.21.15.151:0/3647114456 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:58:58.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.522+0000 7effcce00700 1 --2- 172.21.15.151:0/3647114456 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56385e6d97c0 0x56385e6f4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:58.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.522+0000 7effce200700 1 --2- 172.21.15.151:0/3647114456 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56385e6da0e0 0x56385e6f5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:58.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.523+0000 7effcd800700 1 -- 172.21.15.151:0/3647114456 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56385e6da0e0 msgr2=0x56385e6f5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:58.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.523+0000 7effcd800700 1 --2- 172.21.15.151:0/3647114456 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56385e6da0e0 0x56385e6f5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:58.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.523+0000 7effcd800700 1 -- 172.21.15.151:0/3647114456 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56385e6d97c0 msgr2=0x56385e6f4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:58:58.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.523+0000 7effcd800700 1 --2- 172.21.15.151:0/3647114456 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56385e6d97c0 0x56385e6f4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:58.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.523+0000 7effcd800700 1 -- 172.21.15.151:0/3647114456 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56385e6f5930 con 0x56385e6d4390 2024-05-21T21:58:58.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.523+0000 7effce200700 1 --2- 172.21.15.151:0/3647114456 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56385e6da0e0 0x56385e6f5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T21:58:58.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.523+0000 7effcd800700 1 --2- 172.21.15.151:0/3647114456 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56385e6d4390 0x56385e6e6680 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7effbc009cd0 tx=0x7effbc0239d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f51b4acdd152ff9 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:58:58.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.523+0000 7effaf400700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-21T21:58:58.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.523+0000 7effaf400700 1 -- 172.21.15.151:0/3647114456 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effbc03c060 con 0x56385e6d4390 2024-05-21T21:58:58.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.523+0000 7effaf400700 1 -- 172.21.15.151:0/3647114456 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7effbc003680 con 0x56385e6d4390 2024-05-21T21:58:58.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.523+0000 7effdab0cb80 1 -- 172.21.15.151:0/3647114456 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7effb4005020 con 0x56385e6d4390 2024-05-21T21:58:58.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.524+0000 7effaf400700 1 -- 172.21.15.151:0/3647114456 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7effbc027050 con 0x56385e6d4390 2024-05-21T21:58:58.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.524+0000 7effdab0cb80 10 client.26077 fetch_fsmap learned FSMap version 29 2024-05-21T21:58:58.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.524+0000 7effdab0cb80 1 -- 172.21.15.151:0/3647114456 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56385e7581c0 con 0x56385e6d4390 2024-05-21T21:58:58.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.524+0000 7effaf400700 1 -- 172.21.15.151:0/3647114456 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7effb4005020 con 0x56385e6d4390 2024-05-21T21:58:58.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.524+0000 7effaf400700 1 -- 172.21.15.151:0/3647114456 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7effbc02c5b0 con 0x56385e6d4390 2024-05-21T21:58:58.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.524+0000 7effdab0cb80 10 client.26077 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T21:58:58.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.524+0000 7effdab0cb80 10 client.26077 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T21:58:58.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.524+0000 7effdab0cb80 20 client.26077 populate_metadata read hostname 'smithi151' 2024-05-21T21:58:58.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.524+0000 7effdab0cb80 1 --2- 172.21.15.151:0/3647114456 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x56385e758810 0x56385e75ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:58:58.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.524+0000 7effdab0cb80 4 client.26077 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-21T21:58:58.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.524+0000 7effdab0cb80 1 -- 172.21.15.151:0/3647114456 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56385e62d570 con 0x56385e758810 2024-05-21T21:58:58.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.525+0000 7effcce00700 1 --2- 172.21.15.151:0/3647114456 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x56385e758810 0x56385e75ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:58:58.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.525+0000 7effcce00700 1 --2- 172.21.15.151:0/3647114456 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x56385e758810 0x56385e75ac30 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=192d0f0955b890b9 server_cookie=715ffa2162bf2eb9 in_seq=0 out_seq=0 2024-05-21T21:58:58.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.525+0000 7effaf400700 10 client.26077 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T21:58:58.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.525+0000 7effaf400700 1 -- 172.21.15.151:0/3647114456 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56385e62d570 con 0x56385e758810 2024-05-21T21:58:58.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.525+0000 7effaf400700 10 client.26077 handle_command_reply: tid=0 2024-05-21T21:58:58.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.526+0000 7effdab0cb80 1 client.26077 shutdown 2024-05-21T21:58:58.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.526+0000 7effdab0cb80 1 -- 172.21.15.151:0/3647114456 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56385e6d4390 msgr2=0x56385e6e6680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:58.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.526+0000 7effdab0cb80 1 --2- 172.21.15.151:0/3647114456 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56385e6d4390 0x56385e6e6680 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7effbc009cd0 tx=0x7effbc0239d0 comp rx=0 tx=0).stop 2024-05-21T21:58:58.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.526+0000 7effdab0cb80 1 -- 172.21.15.151:0/3647114456 shutdown_connections 2024-05-21T21:58:58.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.526+0000 7effdab0cb80 1 --2- 172.21.15.151:0/3647114456 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x56385e758810 0x56385e75ac30 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:58.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.526+0000 7effdab0cb80 1 --2- 172.21.15.151:0/3647114456 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56385e6da0e0 0x56385e6f5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:58.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.526+0000 7effdab0cb80 1 --2- 172.21.15.151:0/3647114456 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56385e6d4390 0x56385e6e6680 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:58.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.526+0000 7effdab0cb80 1 --2- 172.21.15.151:0/3647114456 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56385e6d97c0 0x56385e6f4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:58.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.526+0000 7effdab0cb80 1 -- 172.21.15.151:0/3647114456 >> 172.21.15.151:0/3647114456 conn(0x56385e601be0 msgr2=0x56385e6dc350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:58:58.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.526+0000 7effaf400700 0 client.26077 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T21:58:58.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.526+0000 7effdab0cb80 1 -- 172.21.15.151:0/3647114456 shutdown_connections 2024-05-21T21:58:58.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.526+0000 7effdab0cb80 1 -- 172.21.15.151:0/3647114456 wait complete. 2024-05-21T21:58:58.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.526+0000 7effdab0cb80 20 client.26077 trim_cache size 0 max 16384 2024-05-21T21:58:58.529 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-05-21T21:58:58.529 INFO:teuthology.orchestra.run.smithi151.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-21T21:58:58.529 INFO:teuthology.orchestra.run.smithi151.stdout: "scrubs": { 2024-05-21T21:58:58.529 INFO:teuthology.orchestra.run.smithi151.stdout: "c643db33-a2fa-48c5-b4a8-a7d82b566da9": { 2024-05-21T21:58:58.529 INFO:teuthology.orchestra.run.smithi151.stdout: "path": "/", 2024-05-21T21:58:58.529 INFO:teuthology.orchestra.run.smithi151.stdout: "tag": "c643db33-a2fa-48c5-b4a8-a7d82b566da9", 2024-05-21T21:58:58.529 INFO:teuthology.orchestra.run.smithi151.stdout: "options": "recursive,force" 2024-05-21T21:58:58.529 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T21:58:58.529 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T21:58:58.529 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-05-21T21:58:58.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.527+0000 7effcec00700 1 -- 172.21.15.151:0/2536773837 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7effb00b29f0 msgr2=0x7effb00b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:58.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.527+0000 7effcec00700 1 --2- 172.21.15.151:0/2536773837 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7effb00b29f0 0x7effb00b4eb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7effc819d260 tx=0x7effb4009310 comp rx=0 tx=0).stop 2024-05-21T21:58:58.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.527+0000 7effcec00700 1 -- 172.21.15.151:0/2536773837 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7effc810fd10 msgr2=0x7effc819bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:58:58.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.527+0000 7effcec00700 1 --2- 172.21.15.151:0/2536773837 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7effc810fd10 0x7effc819bb10 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7effc0009f60 tx=0x7effc0009510 comp rx=0 tx=0).stop 2024-05-21T21:58:58.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.527+0000 7effcec00700 1 -- 172.21.15.151:0/2536773837 shutdown_connections 2024-05-21T21:58:58.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.527+0000 7effcec00700 1 --2- 172.21.15.151:0/2536773837 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7effb00b29f0 0x7effb00b4eb0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:58.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.527+0000 7effcec00700 1 --2- 172.21.15.151:0/2536773837 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7effc810fd10 0x7effc819bb10 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:58.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.527+0000 7effcec00700 1 --2- 172.21.15.151:0/2536773837 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7effc8075910 0x7effc81a1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:58.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.527+0000 7effcec00700 1 --2- 172.21.15.151:0/2536773837 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7effc8076230 0x7effc81a1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:58:58.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.527+0000 7effcec00700 1 -- 172.21.15.151:0/2536773837 >> 172.21.15.151:0/2536773837 conn(0x7effc80fefc0 msgr2=0x7effc80ffc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:58:58.531 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.527+0000 7effcec00700 1 -- 172.21.15.151:0/2536773837 shutdown_connections 2024-05-21T21:58:58.531 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:58:58.527+0000 7effcec00700 1 -- 172.21.15.151:0/2536773837 wait complete. 2024-05-21T21:58:58.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:58:58 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:58:58.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:58:58 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:58:58.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:58:58 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T21:58:58.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:58:58 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:58:58.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:58:58 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:58:58.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:58:58 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T21:58:58.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:58:58 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:58:58.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:58:58 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:58:58.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:58:58 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T21:58:59.374 INFO:tasks.cephfs.filesystem:scrub status for tag:c643db33-a2fa-48c5-b4a8-a7d82b566da9 - {'path': '/', 'tag': 'c643db33-a2fa-48c5-b4a8-a7d82b566da9', 'options': 'recursive,force'} 2024-05-21T21:58:59.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:58:59 smithi151 ceph-mon[31623]: pgmap v104: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 1.3 MiB/s wr, 16 op/s 2024-05-21T21:58:59.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:58:59 smithi151 ceph-mon[31623]: scrub summary: idle 2024-05-21T21:58:59.985 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:58:59 smithi184 ceph-mon[36942]: pgmap v104: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 1.3 MiB/s wr, 16 op/s 2024-05-21T21:58:59.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:58:59 smithi184 ceph-mon[36942]: scrub summary: idle 2024-05-21T21:59:00.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:58:59 smithi164 ceph-mon[36269]: pgmap v104: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 1.3 MiB/s wr, 16 op/s 2024-05-21T21:59:00.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:58:59 smithi164 ceph-mon[36269]: scrub summary: idle 2024-05-21T21:59:01.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:59:01 smithi151 ceph-mon[31623]: pgmap v105: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 865 KiB/s wr, 11 op/s 2024-05-21T21:59:01.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:59:01 smithi184 ceph-mon[36942]: pgmap v105: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 865 KiB/s wr, 11 op/s 2024-05-21T21:59:02.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:59:01 smithi164 ceph-mon[36269]: pgmap v105: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 865 KiB/s wr, 11 op/s 2024-05-21T21:59:03.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:59:03 smithi151 ceph-mon[31623]: pgmap v106: 97 pgs: 97 active+clean; 260 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 1.4 MiB/s wr, 82 op/s 2024-05-21T21:59:03.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:59:03 smithi184 ceph-mon[36942]: pgmap v106: 97 pgs: 97 active+clean; 260 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 1.4 MiB/s wr, 82 op/s 2024-05-21T21:59:04.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:59:03 smithi164 ceph-mon[36269]: pgmap v106: 97 pgs: 97 active+clean; 260 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 1.4 MiB/s wr, 82 op/s 2024-05-21T21:59:05.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:59:05 smithi151 ceph-mon[31623]: pgmap v107: 97 pgs: 97 active+clean; 260 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 1006 KiB/s wr, 77 op/s 2024-05-21T21:59:05.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:59:05 smithi184 ceph-mon[36942]: pgmap v107: 97 pgs: 97 active+clean; 260 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 1006 KiB/s wr, 77 op/s 2024-05-21T21:59:06.035 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:59:05 smithi164 ceph-mon[36269]: pgmap v107: 97 pgs: 97 active+clean; 260 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 1006 KiB/s wr, 77 op/s 2024-05-21T21:59:07.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:59:07 smithi151 ceph-mon[31623]: pgmap v108: 97 pgs: 97 active+clean; 260 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 1006 KiB/s wr, 77 op/s 2024-05-21T21:59:07.985 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:59:07 smithi184 ceph-mon[36942]: pgmap v108: 97 pgs: 97 active+clean; 260 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 1006 KiB/s wr, 77 op/s 2024-05-21T21:59:08.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:59:07 smithi164 ceph-mon[36269]: pgmap v108: 97 pgs: 97 active+clean; 260 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 1006 KiB/s wr, 77 op/s 2024-05-21T21:59:09.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:59:09 smithi151 ceph-mon[31623]: pgmap v109: 97 pgs: 97 active+clean; 260 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 2.0 MiB/s wr, 227 op/s 2024-05-21T21:59:09.985 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:59:09 smithi184 ceph-mon[36942]: pgmap v109: 97 pgs: 97 active+clean; 260 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 2.0 MiB/s wr, 227 op/s 2024-05-21T21:59:10.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:59:09 smithi164 ceph-mon[36269]: pgmap v109: 97 pgs: 97 active+clean; 260 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 2.0 MiB/s wr, 227 op/s 2024-05-21T21:59:11.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:59:11 smithi151 ceph-mon[31623]: pgmap v110: 97 pgs: 97 active+clean; 260 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 1.5 MiB/s wr, 221 op/s 2024-05-21T21:59:11.985 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:59:11 smithi184 ceph-mon[36942]: pgmap v110: 97 pgs: 97 active+clean; 260 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 1.5 MiB/s wr, 221 op/s 2024-05-21T21:59:12.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:59:11 smithi164 ceph-mon[36269]: pgmap v110: 97 pgs: 97 active+clean; 260 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 1.5 MiB/s wr, 221 op/s 2024-05-21T21:59:13.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:59:13 smithi151 ceph-mon[31623]: pgmap v111: 97 pgs: 97 active+clean; 260 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 3.0 MiB/s wr, 371 op/s 2024-05-21T21:59:13.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:59:13 smithi184 ceph-mon[36942]: pgmap v111: 97 pgs: 97 active+clean; 260 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 3.0 MiB/s wr, 371 op/s 2024-05-21T21:59:14.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:59:13 smithi164 ceph-mon[36269]: pgmap v111: 97 pgs: 97 active+clean; 260 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 3.0 MiB/s wr, 371 op/s 2024-05-21T21:59:15.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:59:15 smithi184 ceph-mon[36942]: pgmap v112: 97 pgs: 97 active+clean; 260 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 2.5 MiB/s wr, 300 op/s 2024-05-21T21:59:16.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:59:15 smithi164 ceph-mon[36269]: pgmap v112: 97 pgs: 97 active+clean; 260 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 2.5 MiB/s wr, 300 op/s 2024-05-21T21:59:16.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:59:15 smithi151 ceph-mon[31623]: pgmap v112: 97 pgs: 97 active+clean; 260 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 2.5 MiB/s wr, 300 op/s 2024-05-21T21:59:17.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:59:17 smithi184 ceph-mon[36942]: pgmap v113: 97 pgs: 97 active+clean; 260 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 2.5 MiB/s wr, 300 op/s 2024-05-21T21:59:18.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:59:17 smithi164 ceph-mon[36269]: pgmap v113: 97 pgs: 97 active+clean; 260 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 2.5 MiB/s wr, 300 op/s 2024-05-21T21:59:18.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:59:17 smithi151 ceph-mon[31623]: pgmap v113: 97 pgs: 97 active+clean; 260 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 2.5 MiB/s wr, 300 op/s 2024-05-21T21:59:19.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:59:19 smithi184 ceph-mon[36942]: pgmap v114: 97 pgs: 97 active+clean; 260 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 3.4 MiB/s wr, 442 op/s 2024-05-21T21:59:20.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:59:19 smithi164 ceph-mon[36269]: pgmap v114: 97 pgs: 97 active+clean; 260 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 3.4 MiB/s wr, 442 op/s 2024-05-21T21:59:20.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:59:19 smithi151 ceph-mon[31623]: pgmap v114: 97 pgs: 97 active+clean; 260 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 3.4 MiB/s wr, 442 op/s 2024-05-21T21:59:20.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:59:20 smithi184 ceph-mon[36942]: pgmap v115: 97 pgs: 97 active+clean; 260 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 2.4 MiB/s wr, 292 op/s 2024-05-21T21:59:21.037 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:59:20 smithi164 ceph-mon[36269]: pgmap v115: 97 pgs: 97 active+clean; 260 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 2.4 MiB/s wr, 292 op/s 2024-05-21T21:59:21.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:59:20 smithi151 ceph-mon[31623]: pgmap v115: 97 pgs: 97 active+clean; 260 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 2.4 MiB/s wr, 292 op/s 2024-05-21T21:59:23.689 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:59:23 smithi184 ceph-mon[36942]: pgmap v116: 97 pgs: 97 active+clean; 285 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 24 MiB/s wr, 1.29k op/s 2024-05-21T21:59:23.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:59:23 smithi151 ceph-mon[31623]: pgmap v116: 97 pgs: 97 active+clean; 285 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 24 MiB/s wr, 1.29k op/s 2024-05-21T21:59:23.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:59:23 smithi164 ceph-mon[36269]: pgmap v116: 97 pgs: 97 active+clean; 285 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 24 MiB/s wr, 1.29k op/s 2024-05-21T21:59:25.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:59:25 smithi151 ceph-mon[31623]: pgmap v117: 97 pgs: 97 active+clean; 285 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 MiB/s wr, 1.14k op/s 2024-05-21T21:59:25.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:59:25 smithi184 ceph-mon[36942]: pgmap v117: 97 pgs: 97 active+clean; 285 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 MiB/s wr, 1.14k op/s 2024-05-21T21:59:25.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:59:25 smithi164 ceph-mon[36269]: pgmap v117: 97 pgs: 97 active+clean; 285 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 MiB/s wr, 1.14k op/s 2024-05-21T21:59:26.187 INFO:tasks.workunit.client.0.smithi151.stdout:setting up random write file 2024-05-21T21:59:26.187 INFO:tasks.workunit.client.0.smithi151.stdout:done setting up random write file 2024-05-21T21:59:26.187 INFO:tasks.workunit.client.0.smithi151.stdout:starting fsync run 2024-05-21T21:59:26.187 INFO:tasks.workunit.client.0.smithi151.stdout:write time: 0.0010s fsync time: 0.0353s 2024-05-21T21:59:26.187 INFO:tasks.workunit.client.0.smithi151.stdout:write time: 0.0006s fsync time: 0.0234s 2024-05-21T21:59:26.188 INFO:tasks.workunit.client.0.smithi151.stdout:write time: 0.0010s fsync time: 0.0191s 2024-05-21T21:59:26.188 INFO:tasks.workunit.client.0.smithi151.stdout:write time: 0.0013s fsync time: 0.0301s 2024-05-21T21:59:26.188 INFO:tasks.workunit.client.0.smithi151.stdout:write time: 0.0006s fsync time: 0.0141s 2024-05-21T21:59:26.188 INFO:tasks.workunit.client.0.smithi151.stdout:write time: 0.0014s fsync time: 0.0182s 2024-05-21T21:59:26.188 INFO:tasks.workunit.client.0.smithi151.stdout:write time: 0.0006s fsync time: 0.0153s 2024-05-21T21:59:26.188 INFO:tasks.workunit.client.0.smithi151.stdout:write time: 0.0014s fsync time: 0.0231s 2024-05-21T21:59:26.188 INFO:tasks.workunit.client.0.smithi151.stdout:write time: 0.0006s fsync time: 0.0094s 2024-05-21T21:59:26.188 INFO:tasks.workunit.client.0.smithi151.stdout:write time: 0.0013s fsync time: 0.0219s 2024-05-21T21:59:26.188 INFO:tasks.workunit.client.0.smithi151.stdout:write time: 0.0018s fsync time: 0.0159s 2024-05-21T21:59:26.188 INFO:tasks.workunit.client.0.smithi151.stdout:write time: 0.0014s fsync time: 0.0191s 2024-05-21T21:59:26.188 INFO:tasks.workunit.client.0.smithi151.stdout:write time: 0.0005s fsync time: 0.0134s 2024-05-21T21:59:26.188 INFO:tasks.workunit.client.0.smithi151.stdout:write time: 0.0009s fsync time: 0.0185s 2024-05-21T21:59:26.188 INFO:tasks.workunit.client.0.smithi151.stdout:write time: 0.0014s fsync time: 0.0201s 2024-05-21T21:59:26.189 INFO:tasks.workunit.client.0.smithi151.stdout:write time: 0.0005s fsync time: 0.0111s 2024-05-21T21:59:26.189 INFO:tasks.workunit.client.0.smithi151.stdout:write time: 0.0014s fsync time: 0.0344s 2024-05-21T21:59:26.189 INFO:tasks.workunit.client.0.smithi151.stdout:write time: 0.0012s fsync time: 0.0238s 2024-05-21T21:59:26.189 INFO:tasks.workunit.client.0.smithi151.stdout:write time: 0.0014s fsync time: 0.0214s 2024-05-21T21:59:26.189 INFO:tasks.workunit.client.0.smithi151.stdout:write time: 0.0014s fsync time: 0.0267s 2024-05-21T21:59:26.189 INFO:tasks.workunit.client.0.smithi151.stdout:write time: 0.0013s fsync time: 0.0348s 2024-05-21T21:59:26.189 INFO:tasks.workunit.client.0.smithi151.stdout:write time: 0.0014s fsync time: 0.0236s 2024-05-21T21:59:26.189 INFO:tasks.workunit.client.0.smithi151.stdout:write time: 0.0008s fsync time: 0.0099s 2024-05-21T21:59:26.189 INFO:tasks.workunit.client.0.smithi151.stdout:write time: 0.0013s fsync time: 0.0224s 2024-05-21T21:59:26.189 INFO:tasks.workunit.client.0.smithi151.stdout:write time: 0.0014s fsync time: 0.0245s 2024-05-21T21:59:26.189 INFO:tasks.workunit.client.0.smithi151.stdout:write time: 0.0014s fsync time: 0.0167s 2024-05-21T21:59:26.189 INFO:tasks.workunit.client.0.smithi151.stdout:write time: 0.0014s fsync time: 0.0245s 2024-05-21T21:59:26.189 INFO:tasks.workunit.client.0.smithi151.stdout:write time: 0.0006s fsync time: 0.0126s 2024-05-21T21:59:26.189 INFO:tasks.workunit.client.0.smithi151.stdout:write time: 0.0014s fsync time: 0.0277s 2024-05-21T21:59:26.190 INFO:tasks.workunit.client.0.smithi151.stdout:write time: 0.0008s fsync time: 0.0118s 2024-05-21T21:59:26.190 INFO:tasks.workunit.client.0.smithi151.stdout:write time: 0.0013s fsync time: 0.0320s 2024-05-21T21:59:26.190 INFO:tasks.workunit.client.0.smithi151.stdout:write time: 0.0010s fsync time: 0.0197s 2024-05-21T21:59:26.190 INFO:tasks.workunit.client.0.smithi151.stdout:write time: 0.0011s fsync time: 0.0137s 2024-05-21T21:59:26.190 INFO:tasks.workunit.client.0.smithi151.stdout:write time: 0.0014s fsync time: 0.0242s 2024-05-21T21:59:26.190 INFO:tasks.workunit.client.0.smithi151.stdout:write time: 0.0013s fsync time: 0.0226s 2024-05-21T21:59:26.190 INFO:tasks.workunit.client.0.smithi151.stdout:write time: 0.0017s fsync time: 0.0286s 2024-05-21T21:59:26.190 INFO:tasks.workunit.client.0.smithi151.stdout:write time: 0.0019s fsync time: 0.0278s 2024-05-21T21:59:26.190 INFO:tasks.workunit.client.0.smithi151.stdout:write time: 0.0020s fsync time: 0.0269s 2024-05-21T21:59:26.190 INFO:tasks.workunit.client.0.smithi151.stdout:write time: 0.0017s fsync time: 0.0268s 2024-05-21T21:59:26.190 INFO:tasks.workunit.client.0.smithi151.stdout:write time: 0.0015s fsync time: 0.0248s 2024-05-21T21:59:26.190 INFO:tasks.workunit.client.0.smithi151.stdout:write time: 0.0018s fsync time: 0.0275s 2024-05-21T21:59:26.190 INFO:tasks.workunit.client.0.smithi151.stdout:write time: 0.0012s fsync time: 0.0256s 2024-05-21T21:59:26.190 INFO:tasks.workunit.client.0.smithi151.stdout:write time: 0.0017s fsync time: 0.0273s 2024-05-21T21:59:26.191 INFO:tasks.workunit.client.0.smithi151.stdout:write time: 0.0015s fsync time: 0.0288s 2024-05-21T21:59:26.191 INFO:tasks.workunit.client.0.smithi151.stdout:write time: 0.0017s fsync time: 0.0274s 2024-05-21T21:59:26.191 INFO:tasks.workunit.client.0.smithi151.stdout:write time: 0.0014s fsync time: 0.0275s 2024-05-21T21:59:26.191 INFO:tasks.workunit.client.0.smithi151.stdout:write time: 0.0015s fsync time: 0.0300s 2024-05-21T21:59:26.191 INFO:tasks.workunit.client.0.smithi151.stdout:write time: 0.0015s fsync time: 0.0262s 2024-05-21T21:59:26.191 INFO:tasks.workunit.client.0.smithi151.stdout:write time: 0.0016s fsync time: 0.0268s 2024-05-21T21:59:26.191 INFO:tasks.workunit.client.0.smithi151.stdout:write time: 0.0017s fsync time: 0.0334s 2024-05-21T21:59:26.191 INFO:tasks.workunit.client.0.smithi151.stdout:write time: 0.0016s fsync time: 0.0315s 2024-05-21T21:59:26.191 INFO:tasks.workunit.client.0.smithi151.stdout:write time: 0.0014s fsync time: 0.0232s 2024-05-21T21:59:26.191 INFO:tasks.workunit.client.0.smithi151.stdout:write time: 0.0024s fsync time: 0.0273s 2024-05-21T21:59:26.191 INFO:tasks.workunit.client.0.smithi151.stdout:write time: 0.0009s fsync time: 0.3097s 2024-05-21T21:59:26.191 INFO:tasks.workunit.client.0.smithi151.stdout:write time: 0.0014s fsync time: 0.0226s 2024-05-21T21:59:26.191 INFO:tasks.workunit.client.0.smithi151.stdout:write time: 0.0013s fsync time: 0.0273s 2024-05-21T21:59:26.191 INFO:tasks.workunit.client.0.smithi151.stdout:write time: 0.0008s fsync time: 0.0167s 2024-05-21T21:59:26.192 INFO:tasks.workunit.client.0.smithi151.stdout:write time: 0.0009s fsync time: 0.0171s 2024-05-21T21:59:26.192 INFO:tasks.workunit.client.0.smithi151.stdout:write time: 0.0013s fsync time: 0.0212s 2024-05-21T21:59:26.192 INFO:tasks.workunit.client.0.smithi151.stdout:write time: 0.0015s fsync time: 0.0200s 2024-05-21T21:59:26.192 INFO:tasks.workunit.client.0.smithi151.stdout:run done 60 fsyncs total, killing random writer 2024-05-21T21:59:26.192 INFO:tasks.workunit.client.0.smithi151.stderr:+ echo /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2024-05-21T21:59:26.193 INFO:tasks.workunit.client.0.smithi151.stdout:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2024-05-21T21:59:26.193 INFO:tasks.workunit.client.0.smithi151.stderr:+ whereis lsof 2024-05-21T21:59:26.242 INFO:tasks.workunit.client.0.smithi151.stdout:lsof: /usr/bin/lsof /usr/share/man/man1/lsof.1.gz 2024-05-21T21:59:26.242 INFO:tasks.workunit.client.0.smithi151.stderr:+ lsof 2024-05-21T21:59:26.267 INFO:tasks.workunit.client.0.smithi151.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/eab2034bc8bc3c828a99c886df4527cd1c79b1853767fa1d5b093e24c7836493/merged 2024-05-21T21:59:26.267 INFO:tasks.workunit.client.0.smithi151.stderr: Output information may be incomplete. 2024-05-21T21:59:26.267 INFO:tasks.workunit.client.0.smithi151.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/fe30556184b0a0ef56cdcb291f93ed92202a2fd3a4ca797fda5bd755d669596c/merged 2024-05-21T21:59:26.267 INFO:tasks.workunit.client.0.smithi151.stderr: Output information may be incomplete. 2024-05-21T21:59:26.269 INFO:tasks.workunit.client.0.smithi151.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/a13a2879c755c6bd4da2740a11ee32a4380aff372ab8615175681c8f9ff7324a/merged 2024-05-21T21:59:26.270 INFO:tasks.workunit.client.0.smithi151.stderr: Output information may be incomplete. 2024-05-21T21:59:26.270 INFO:tasks.workunit.client.0.smithi151.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/7bcec15d90b2c50641340cbad3f57fa519e3828323e8cbd69143c0e8d5494d01/merged 2024-05-21T21:59:26.270 INFO:tasks.workunit.client.0.smithi151.stderr: Output information may be incomplete. 2024-05-21T21:59:26.270 INFO:tasks.workunit.client.0.smithi151.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/f0a11b5789bc8f569f7e8a3299914500bb5163daffc274c0cc81adb49adaa086/merged 2024-05-21T21:59:26.270 INFO:tasks.workunit.client.0.smithi151.stderr: Output information may be incomplete. 2024-05-21T21:59:26.270 INFO:tasks.workunit.client.0.smithi151.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/64768dfff34c0097516505fc6c56b3544d9acce0a23925a1a6071731f1348111/merged 2024-05-21T21:59:26.270 INFO:tasks.workunit.client.0.smithi151.stderr: Output information may be incomplete. 2024-05-21T21:59:26.272 INFO:tasks.workunit.client.0.smithi151.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/0089121b777e0e728cc566d00ae1176216ac83e6bf7c5b4f9614f7e25dd673e0/merged 2024-05-21T21:59:26.272 INFO:tasks.workunit.client.0.smithi151.stderr: Output information may be incomplete. 2024-05-21T21:59:26.273 INFO:tasks.workunit.client.0.smithi151.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/c1a24ea2467a8ed8365a81ad3f2fdebdbc970575be7866d356c98b9949881d5c/merged 2024-05-21T21:59:26.273 INFO:tasks.workunit.client.0.smithi151.stderr: Output information may be incomplete. 2024-05-21T21:59:26.273 INFO:tasks.workunit.client.0.smithi151.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/fcadf03ad7845d5f5c8ecb85eeba03dba5005d465ca7e20e72d0362c2887572e/merged 2024-05-21T21:59:26.273 INFO:tasks.workunit.client.0.smithi151.stderr: Output information may be incomplete. 2024-05-21T21:59:26.273 INFO:tasks.workunit.client.0.smithi151.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/d3db1bbba036e3863df78895ede9f93848c28312ad317c8645d6bfcc779ebc17/merged 2024-05-21T21:59:26.273 INFO:tasks.workunit.client.0.smithi151.stderr: Output information may be incomplete. 2024-05-21T21:59:26.423 INFO:tasks.workunit.client.0.smithi151.stdout:COMMAND PID TID TASKCMD USER FD TYPE DEVICE SIZE/OFF NODE NAME 2024-05-21T21:59:26.423 INFO:tasks.workunit.client.0.smithi151.stdout:systemd 1 root cwd unknown /proc/1/cwd (readlink: Permission denied) 2024-05-21T21:59:26.423 INFO:tasks.workunit.client.0.smithi151.stdout:systemd 1 root rtd unknown /proc/1/root (readlink: Permission denied) 2024-05-21T21:59:26.423 INFO:tasks.workunit.client.0.smithi151.stdout:systemd 1 root txt unknown /proc/1/exe (readlink: Permission denied) 2024-05-21T21:59:26.423 INFO:tasks.workunit.client.0.smithi151.stdout:systemd 1 root NOFD /proc/1/fd (opendir: Permission denied) 2024-05-21T21:59:26.423 INFO:tasks.workunit.client.0.smithi151.stdout:kthreadd 2 root cwd unknown /proc/2/cwd (readlink: Permission denied) 2024-05-21T21:59:26.423 INFO:tasks.workunit.client.0.smithi151.stdout:kthreadd 2 root rtd unknown /proc/2/root (readlink: Permission denied) 2024-05-21T21:59:26.423 INFO:tasks.workunit.client.0.smithi151.stdout:kthreadd 2 root txt unknown /proc/2/exe (readlink: Permission denied) 2024-05-21T21:59:26.423 INFO:tasks.workunit.client.0.smithi151.stdout:kthreadd 2 root NOFD /proc/2/fd (opendir: Permission denied) 2024-05-21T21:59:26.424 INFO:tasks.workunit.client.0.smithi151.stdout:pool_work 3 root cwd unknown /proc/3/cwd (readlink: Permission denied) 2024-05-21T21:59:26.424 INFO:tasks.workunit.client.0.smithi151.stdout:pool_work 3 root rtd unknown /proc/3/root (readlink: Permission denied) 2024-05-21T21:59:26.424 INFO:tasks.workunit.client.0.smithi151.stdout:pool_work 3 root txt unknown /proc/3/exe (readlink: Permission denied) 2024-05-21T21:59:26.424 INFO:tasks.workunit.client.0.smithi151.stdout:pool_work 3 root NOFD /proc/3/fd (opendir: Permission denied) 2024-05-21T21:59:26.424 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 4 root cwd unknown /proc/4/cwd (readlink: Permission denied) 2024-05-21T21:59:26.424 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 4 root rtd unknown /proc/4/root (readlink: Permission denied) 2024-05-21T21:59:26.424 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 4 root txt unknown /proc/4/exe (readlink: Permission denied) 2024-05-21T21:59:26.424 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 4 root NOFD /proc/4/fd (opendir: Permission denied) 2024-05-21T21:59:26.424 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 5 root cwd unknown /proc/5/cwd (readlink: Permission denied) 2024-05-21T21:59:26.424 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 5 root rtd unknown /proc/5/root (readlink: Permission denied) 2024-05-21T21:59:26.424 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 5 root txt unknown /proc/5/exe (readlink: Permission denied) 2024-05-21T21:59:26.424 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 5 root NOFD /proc/5/fd (opendir: Permission denied) 2024-05-21T21:59:26.424 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 6 root cwd unknown /proc/6/cwd (readlink: Permission denied) 2024-05-21T21:59:26.424 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 6 root rtd unknown /proc/6/root (readlink: Permission denied) 2024-05-21T21:59:26.424 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 6 root txt unknown /proc/6/exe (readlink: Permission denied) 2024-05-21T21:59:26.425 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 6 root NOFD /proc/6/fd (opendir: Permission denied) 2024-05-21T21:59:26.425 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/0 9 root cwd unknown /proc/9/cwd (readlink: Permission denied) 2024-05-21T21:59:26.425 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/0 9 root rtd unknown /proc/9/root (readlink: Permission denied) 2024-05-21T21:59:26.425 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/0 9 root txt unknown /proc/9/exe (readlink: Permission denied) 2024-05-21T21:59:26.425 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/0 9 root NOFD /proc/9/fd (opendir: Permission denied) 2024-05-21T21:59:26.425 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/u 10 root cwd unknown /proc/10/cwd (readlink: Permission denied) 2024-05-21T21:59:26.425 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/u 10 root rtd unknown /proc/10/root (readlink: Permission denied) 2024-05-21T21:59:26.425 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/u 10 root txt unknown /proc/10/exe (readlink: Permission denied) 2024-05-21T21:59:26.425 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/u 10 root NOFD /proc/10/fd (opendir: Permission denied) 2024-05-21T21:59:26.425 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 11 root cwd unknown /proc/11/cwd (readlink: Permission denied) 2024-05-21T21:59:26.425 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 11 root rtd unknown /proc/11/root (readlink: Permission denied) 2024-05-21T21:59:26.425 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 11 root txt unknown /proc/11/exe (readlink: Permission denied) 2024-05-21T21:59:26.425 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 11 root NOFD /proc/11/fd (opendir: Permission denied) 2024-05-21T21:59:26.426 INFO:tasks.workunit.client.0.smithi151.stdout:rcu_tasks 12 root cwd unknown /proc/12/cwd (readlink: Permission denied) 2024-05-21T21:59:26.426 INFO:tasks.workunit.client.0.smithi151.stdout:rcu_tasks 12 root rtd unknown /proc/12/root (readlink: Permission denied) 2024-05-21T21:59:26.426 INFO:tasks.workunit.client.0.smithi151.stdout:rcu_tasks 12 root txt unknown /proc/12/exe (readlink: Permission denied) 2024-05-21T21:59:26.426 INFO:tasks.workunit.client.0.smithi151.stdout:rcu_tasks 12 root NOFD /proc/12/fd (opendir: Permission denied) 2024-05-21T21:59:26.426 INFO:tasks.workunit.client.0.smithi151.stdout:rcu_tasks 13 root cwd unknown /proc/13/cwd (readlink: Permission denied) 2024-05-21T21:59:26.426 INFO:tasks.workunit.client.0.smithi151.stdout:rcu_tasks 13 root rtd unknown /proc/13/root (readlink: Permission denied) 2024-05-21T21:59:26.426 INFO:tasks.workunit.client.0.smithi151.stdout:rcu_tasks 13 root txt unknown /proc/13/exe (readlink: Permission denied) 2024-05-21T21:59:26.426 INFO:tasks.workunit.client.0.smithi151.stdout:rcu_tasks 13 root NOFD /proc/13/fd (opendir: Permission denied) 2024-05-21T21:59:26.426 INFO:tasks.workunit.client.0.smithi151.stdout:rcu_tasks 14 root cwd unknown /proc/14/cwd (readlink: Permission denied) 2024-05-21T21:59:26.426 INFO:tasks.workunit.client.0.smithi151.stdout:rcu_tasks 14 root rtd unknown /proc/14/root (readlink: Permission denied) 2024-05-21T21:59:26.426 INFO:tasks.workunit.client.0.smithi151.stdout:rcu_tasks 14 root txt unknown /proc/14/exe (readlink: Permission denied) 2024-05-21T21:59:26.426 INFO:tasks.workunit.client.0.smithi151.stdout:rcu_tasks 14 root NOFD /proc/14/fd (opendir: Permission denied) 2024-05-21T21:59:26.426 INFO:tasks.workunit.client.0.smithi151.stdout:ksoftirqd 15 root cwd unknown /proc/15/cwd (readlink: Permission denied) 2024-05-21T21:59:26.426 INFO:tasks.workunit.client.0.smithi151.stdout:ksoftirqd 15 root rtd unknown /proc/15/root (readlink: Permission denied) 2024-05-21T21:59:26.427 INFO:tasks.workunit.client.0.smithi151.stdout:ksoftirqd 15 root txt unknown /proc/15/exe (readlink: Permission denied) 2024-05-21T21:59:26.427 INFO:tasks.workunit.client.0.smithi151.stdout:ksoftirqd 15 root NOFD /proc/15/fd (opendir: Permission denied) 2024-05-21T21:59:26.427 INFO:tasks.workunit.client.0.smithi151.stdout:rcu_preem 16 root cwd unknown /proc/16/cwd (readlink: Permission denied) 2024-05-21T21:59:26.427 INFO:tasks.workunit.client.0.smithi151.stdout:rcu_preem 16 root rtd unknown /proc/16/root (readlink: Permission denied) 2024-05-21T21:59:26.427 INFO:tasks.workunit.client.0.smithi151.stdout:rcu_preem 16 root txt unknown /proc/16/exe (readlink: Permission denied) 2024-05-21T21:59:26.427 INFO:tasks.workunit.client.0.smithi151.stdout:rcu_preem 16 root NOFD /proc/16/fd (opendir: Permission denied) 2024-05-21T21:59:26.427 INFO:tasks.workunit.client.0.smithi151.stdout:rcu_exp_p 17 root cwd unknown /proc/17/cwd (readlink: Permission denied) 2024-05-21T21:59:26.427 INFO:tasks.workunit.client.0.smithi151.stdout:rcu_exp_p 17 root rtd unknown /proc/17/root (readlink: Permission denied) 2024-05-21T21:59:26.427 INFO:tasks.workunit.client.0.smithi151.stdout:rcu_exp_p 17 root txt unknown /proc/17/exe (readlink: Permission denied) 2024-05-21T21:59:26.427 INFO:tasks.workunit.client.0.smithi151.stdout:rcu_exp_p 17 root NOFD /proc/17/fd (opendir: Permission denied) 2024-05-21T21:59:26.427 INFO:tasks.workunit.client.0.smithi151.stdout:rcu_exp_g 18 root cwd unknown /proc/18/cwd (readlink: Permission denied) 2024-05-21T21:59:26.427 INFO:tasks.workunit.client.0.smithi151.stdout:rcu_exp_g 18 root rtd unknown /proc/18/root (readlink: Permission denied) 2024-05-21T21:59:26.427 INFO:tasks.workunit.client.0.smithi151.stdout:rcu_exp_g 18 root txt unknown /proc/18/exe (readlink: Permission denied) 2024-05-21T21:59:26.427 INFO:tasks.workunit.client.0.smithi151.stdout:rcu_exp_g 18 root NOFD /proc/18/fd (opendir: Permission denied) 2024-05-21T21:59:26.428 INFO:tasks.workunit.client.0.smithi151.stdout:migration 19 root cwd unknown /proc/19/cwd (readlink: Permission denied) 2024-05-21T21:59:26.428 INFO:tasks.workunit.client.0.smithi151.stdout:migration 19 root rtd unknown /proc/19/root (readlink: Permission denied) 2024-05-21T21:59:26.428 INFO:tasks.workunit.client.0.smithi151.stdout:migration 19 root txt unknown /proc/19/exe (readlink: Permission denied) 2024-05-21T21:59:26.428 INFO:tasks.workunit.client.0.smithi151.stdout:migration 19 root NOFD /proc/19/fd (opendir: Permission denied) 2024-05-21T21:59:26.428 INFO:tasks.workunit.client.0.smithi151.stdout:cpuhp/0 20 root cwd unknown /proc/20/cwd (readlink: Permission denied) 2024-05-21T21:59:26.428 INFO:tasks.workunit.client.0.smithi151.stdout:cpuhp/0 20 root rtd unknown /proc/20/root (readlink: Permission denied) 2024-05-21T21:59:26.428 INFO:tasks.workunit.client.0.smithi151.stdout:cpuhp/0 20 root txt unknown /proc/20/exe (readlink: Permission denied) 2024-05-21T21:59:26.428 INFO:tasks.workunit.client.0.smithi151.stdout:cpuhp/0 20 root NOFD /proc/20/fd (opendir: Permission denied) 2024-05-21T21:59:26.428 INFO:tasks.workunit.client.0.smithi151.stdout:cpuhp/1 21 root cwd unknown /proc/21/cwd (readlink: Permission denied) 2024-05-21T21:59:26.428 INFO:tasks.workunit.client.0.smithi151.stdout:cpuhp/1 21 root rtd unknown /proc/21/root (readlink: Permission denied) 2024-05-21T21:59:26.428 INFO:tasks.workunit.client.0.smithi151.stdout:cpuhp/1 21 root txt unknown /proc/21/exe (readlink: Permission denied) 2024-05-21T21:59:26.428 INFO:tasks.workunit.client.0.smithi151.stdout:cpuhp/1 21 root NOFD /proc/21/fd (opendir: Permission denied) 2024-05-21T21:59:26.428 INFO:tasks.workunit.client.0.smithi151.stdout:migration 22 root cwd unknown /proc/22/cwd (readlink: Permission denied) 2024-05-21T21:59:26.428 INFO:tasks.workunit.client.0.smithi151.stdout:migration 22 root rtd unknown /proc/22/root (readlink: Permission denied) 2024-05-21T21:59:26.429 INFO:tasks.workunit.client.0.smithi151.stdout:migration 22 root txt unknown /proc/22/exe (readlink: Permission denied) 2024-05-21T21:59:26.429 INFO:tasks.workunit.client.0.smithi151.stdout:migration 22 root NOFD /proc/22/fd (opendir: Permission denied) 2024-05-21T21:59:26.429 INFO:tasks.workunit.client.0.smithi151.stdout:ksoftirqd 23 root cwd unknown /proc/23/cwd (readlink: Permission denied) 2024-05-21T21:59:26.429 INFO:tasks.workunit.client.0.smithi151.stdout:ksoftirqd 23 root rtd unknown /proc/23/root (readlink: Permission denied) 2024-05-21T21:59:26.429 INFO:tasks.workunit.client.0.smithi151.stdout:ksoftirqd 23 root txt unknown /proc/23/exe (readlink: Permission denied) 2024-05-21T21:59:26.429 INFO:tasks.workunit.client.0.smithi151.stdout:ksoftirqd 23 root NOFD /proc/23/fd (opendir: Permission denied) 2024-05-21T21:59:26.429 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/1 25 root cwd unknown /proc/25/cwd (readlink: Permission denied) 2024-05-21T21:59:26.429 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/1 25 root rtd unknown /proc/25/root (readlink: Permission denied) 2024-05-21T21:59:26.429 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/1 25 root txt unknown /proc/25/exe (readlink: Permission denied) 2024-05-21T21:59:26.429 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/1 25 root NOFD /proc/25/fd (opendir: Permission denied) 2024-05-21T21:59:26.429 INFO:tasks.workunit.client.0.smithi151.stdout:cpuhp/2 26 root cwd unknown /proc/26/cwd (readlink: Permission denied) 2024-05-21T21:59:26.430 INFO:tasks.workunit.client.0.smithi151.stdout:cpuhp/2 26 root rtd unknown /proc/26/root (readlink: Permission denied) 2024-05-21T21:59:26.430 INFO:tasks.workunit.client.0.smithi151.stdout:cpuhp/2 26 root txt unknown /proc/26/exe (readlink: Permission denied) 2024-05-21T21:59:26.430 INFO:tasks.workunit.client.0.smithi151.stdout:cpuhp/2 26 root NOFD /proc/26/fd (opendir: Permission denied) 2024-05-21T21:59:26.430 INFO:tasks.workunit.client.0.smithi151.stdout:migration 27 root cwd unknown /proc/27/cwd (readlink: Permission denied) 2024-05-21T21:59:26.430 INFO:tasks.workunit.client.0.smithi151.stdout:migration 27 root rtd unknown /proc/27/root (readlink: Permission denied) 2024-05-21T21:59:26.430 INFO:tasks.workunit.client.0.smithi151.stdout:migration 27 root txt unknown /proc/27/exe (readlink: Permission denied) 2024-05-21T21:59:26.430 INFO:tasks.workunit.client.0.smithi151.stdout:migration 27 root NOFD /proc/27/fd (opendir: Permission denied) 2024-05-21T21:59:26.430 INFO:tasks.workunit.client.0.smithi151.stdout:ksoftirqd 28 root cwd unknown /proc/28/cwd (readlink: Permission denied) 2024-05-21T21:59:26.430 INFO:tasks.workunit.client.0.smithi151.stdout:ksoftirqd 28 root rtd unknown /proc/28/root (readlink: Permission denied) 2024-05-21T21:59:26.430 INFO:tasks.workunit.client.0.smithi151.stdout:ksoftirqd 28 root txt unknown /proc/28/exe (readlink: Permission denied) 2024-05-21T21:59:26.431 INFO:tasks.workunit.client.0.smithi151.stdout:ksoftirqd 28 root NOFD /proc/28/fd (opendir: Permission denied) 2024-05-21T21:59:26.431 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/2 29 root cwd unknown /proc/29/cwd (readlink: Permission denied) 2024-05-21T21:59:26.431 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/2 29 root rtd unknown /proc/29/root (readlink: Permission denied) 2024-05-21T21:59:26.431 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/2 29 root txt unknown /proc/29/exe (readlink: Permission denied) 2024-05-21T21:59:26.431 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/2 29 root NOFD /proc/29/fd (opendir: Permission denied) 2024-05-21T21:59:26.431 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/2 30 root cwd unknown /proc/30/cwd (readlink: Permission denied) 2024-05-21T21:59:26.431 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/2 30 root rtd unknown /proc/30/root (readlink: Permission denied) 2024-05-21T21:59:26.431 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/2 30 root txt unknown /proc/30/exe (readlink: Permission denied) 2024-05-21T21:59:26.431 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/2 30 root NOFD /proc/30/fd (opendir: Permission denied) 2024-05-21T21:59:26.431 INFO:tasks.workunit.client.0.smithi151.stdout:cpuhp/3 31 root cwd unknown /proc/31/cwd (readlink: Permission denied) 2024-05-21T21:59:26.431 INFO:tasks.workunit.client.0.smithi151.stdout:cpuhp/3 31 root rtd unknown /proc/31/root (readlink: Permission denied) 2024-05-21T21:59:26.431 INFO:tasks.workunit.client.0.smithi151.stdout:cpuhp/3 31 root txt unknown /proc/31/exe (readlink: Permission denied) 2024-05-21T21:59:26.432 INFO:tasks.workunit.client.0.smithi151.stdout:cpuhp/3 31 root NOFD /proc/31/fd (opendir: Permission denied) 2024-05-21T21:59:26.432 INFO:tasks.workunit.client.0.smithi151.stdout:migration 32 root cwd unknown /proc/32/cwd (readlink: Permission denied) 2024-05-21T21:59:26.432 INFO:tasks.workunit.client.0.smithi151.stdout:migration 32 root rtd unknown /proc/32/root (readlink: Permission denied) 2024-05-21T21:59:26.432 INFO:tasks.workunit.client.0.smithi151.stdout:migration 32 root txt unknown /proc/32/exe (readlink: Permission denied) 2024-05-21T21:59:26.432 INFO:tasks.workunit.client.0.smithi151.stdout:migration 32 root NOFD /proc/32/fd (opendir: Permission denied) 2024-05-21T21:59:26.432 INFO:tasks.workunit.client.0.smithi151.stdout:ksoftirqd 33 root cwd unknown /proc/33/cwd (readlink: Permission denied) 2024-05-21T21:59:26.432 INFO:tasks.workunit.client.0.smithi151.stdout:ksoftirqd 33 root rtd unknown /proc/33/root (readlink: Permission denied) 2024-05-21T21:59:26.432 INFO:tasks.workunit.client.0.smithi151.stdout:ksoftirqd 33 root txt unknown /proc/33/exe (readlink: Permission denied) 2024-05-21T21:59:26.432 INFO:tasks.workunit.client.0.smithi151.stdout:ksoftirqd 33 root NOFD /proc/33/fd (opendir: Permission denied) 2024-05-21T21:59:26.432 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/3 35 root cwd unknown /proc/35/cwd (readlink: Permission denied) 2024-05-21T21:59:26.432 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/3 35 root rtd unknown /proc/35/root (readlink: Permission denied) 2024-05-21T21:59:26.432 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/3 35 root txt unknown /proc/35/exe (readlink: Permission denied) 2024-05-21T21:59:26.432 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/3 35 root NOFD /proc/35/fd (opendir: Permission denied) 2024-05-21T21:59:26.433 INFO:tasks.workunit.client.0.smithi151.stdout:cpuhp/4 36 root cwd unknown /proc/36/cwd (readlink: Permission denied) 2024-05-21T21:59:26.433 INFO:tasks.workunit.client.0.smithi151.stdout:cpuhp/4 36 root rtd unknown /proc/36/root (readlink: Permission denied) 2024-05-21T21:59:26.433 INFO:tasks.workunit.client.0.smithi151.stdout:cpuhp/4 36 root txt unknown /proc/36/exe (readlink: Permission denied) 2024-05-21T21:59:26.433 INFO:tasks.workunit.client.0.smithi151.stdout:cpuhp/4 36 root NOFD /proc/36/fd (opendir: Permission denied) 2024-05-21T21:59:26.433 INFO:tasks.workunit.client.0.smithi151.stdout:migration 37 root cwd unknown /proc/37/cwd (readlink: Permission denied) 2024-05-21T21:59:26.433 INFO:tasks.workunit.client.0.smithi151.stdout:migration 37 root rtd unknown /proc/37/root (readlink: Permission denied) 2024-05-21T21:59:26.433 INFO:tasks.workunit.client.0.smithi151.stdout:migration 37 root txt unknown /proc/37/exe (readlink: Permission denied) 2024-05-21T21:59:26.433 INFO:tasks.workunit.client.0.smithi151.stdout:migration 37 root NOFD /proc/37/fd (opendir: Permission denied) 2024-05-21T21:59:26.433 INFO:tasks.workunit.client.0.smithi151.stdout:ksoftirqd 38 root cwd unknown /proc/38/cwd (readlink: Permission denied) 2024-05-21T21:59:26.433 INFO:tasks.workunit.client.0.smithi151.stdout:ksoftirqd 38 root rtd unknown /proc/38/root (readlink: Permission denied) 2024-05-21T21:59:26.433 INFO:tasks.workunit.client.0.smithi151.stdout:ksoftirqd 38 root txt unknown /proc/38/exe (readlink: Permission denied) 2024-05-21T21:59:26.433 INFO:tasks.workunit.client.0.smithi151.stdout:ksoftirqd 38 root NOFD /proc/38/fd (opendir: Permission denied) 2024-05-21T21:59:26.433 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/4 40 root cwd unknown /proc/40/cwd (readlink: Permission denied) 2024-05-21T21:59:26.434 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/4 40 root rtd unknown /proc/40/root (readlink: Permission denied) 2024-05-21T21:59:26.434 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/4 40 root txt unknown /proc/40/exe (readlink: Permission denied) 2024-05-21T21:59:26.434 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/4 40 root NOFD /proc/40/fd (opendir: Permission denied) 2024-05-21T21:59:26.434 INFO:tasks.workunit.client.0.smithi151.stdout:cpuhp/5 41 root cwd unknown /proc/41/cwd (readlink: Permission denied) 2024-05-21T21:59:26.434 INFO:tasks.workunit.client.0.smithi151.stdout:cpuhp/5 41 root rtd unknown /proc/41/root (readlink: Permission denied) 2024-05-21T21:59:26.434 INFO:tasks.workunit.client.0.smithi151.stdout:cpuhp/5 41 root txt unknown /proc/41/exe (readlink: Permission denied) 2024-05-21T21:59:26.434 INFO:tasks.workunit.client.0.smithi151.stdout:cpuhp/5 41 root NOFD /proc/41/fd (opendir: Permission denied) 2024-05-21T21:59:26.434 INFO:tasks.workunit.client.0.smithi151.stdout:migration 42 root cwd unknown /proc/42/cwd (readlink: Permission denied) 2024-05-21T21:59:26.434 INFO:tasks.workunit.client.0.smithi151.stdout:migration 42 root rtd unknown /proc/42/root (readlink: Permission denied) 2024-05-21T21:59:26.434 INFO:tasks.workunit.client.0.smithi151.stdout:migration 42 root txt unknown /proc/42/exe (readlink: Permission denied) 2024-05-21T21:59:26.434 INFO:tasks.workunit.client.0.smithi151.stdout:migration 42 root NOFD /proc/42/fd (opendir: Permission denied) 2024-05-21T21:59:26.434 INFO:tasks.workunit.client.0.smithi151.stdout:ksoftirqd 43 root cwd unknown /proc/43/cwd (readlink: Permission denied) 2024-05-21T21:59:26.435 INFO:tasks.workunit.client.0.smithi151.stdout:ksoftirqd 43 root rtd unknown /proc/43/root (readlink: Permission denied) 2024-05-21T21:59:26.435 INFO:tasks.workunit.client.0.smithi151.stdout:ksoftirqd 43 root txt unknown /proc/43/exe (readlink: Permission denied) 2024-05-21T21:59:26.435 INFO:tasks.workunit.client.0.smithi151.stdout:ksoftirqd 43 root NOFD /proc/43/fd (opendir: Permission denied) 2024-05-21T21:59:26.435 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/5 45 root cwd unknown /proc/45/cwd (readlink: Permission denied) 2024-05-21T21:59:26.435 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/5 45 root rtd unknown /proc/45/root (readlink: Permission denied) 2024-05-21T21:59:26.435 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/5 45 root txt unknown /proc/45/exe (readlink: Permission denied) 2024-05-21T21:59:26.435 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/5 45 root NOFD /proc/45/fd (opendir: Permission denied) 2024-05-21T21:59:26.435 INFO:tasks.workunit.client.0.smithi151.stdout:cpuhp/6 46 root cwd unknown /proc/46/cwd (readlink: Permission denied) 2024-05-21T21:59:26.435 INFO:tasks.workunit.client.0.smithi151.stdout:cpuhp/6 46 root rtd unknown /proc/46/root (readlink: Permission denied) 2024-05-21T21:59:26.435 INFO:tasks.workunit.client.0.smithi151.stdout:cpuhp/6 46 root txt unknown /proc/46/exe (readlink: Permission denied) 2024-05-21T21:59:26.436 INFO:tasks.workunit.client.0.smithi151.stdout:cpuhp/6 46 root NOFD /proc/46/fd (opendir: Permission denied) 2024-05-21T21:59:26.436 INFO:tasks.workunit.client.0.smithi151.stdout:migration 47 root cwd unknown /proc/47/cwd (readlink: Permission denied) 2024-05-21T21:59:26.436 INFO:tasks.workunit.client.0.smithi151.stdout:migration 47 root rtd unknown /proc/47/root (readlink: Permission denied) 2024-05-21T21:59:26.436 INFO:tasks.workunit.client.0.smithi151.stdout:migration 47 root txt unknown /proc/47/exe (readlink: Permission denied) 2024-05-21T21:59:26.436 INFO:tasks.workunit.client.0.smithi151.stdout:migration 47 root NOFD /proc/47/fd (opendir: Permission denied) 2024-05-21T21:59:26.436 INFO:tasks.workunit.client.0.smithi151.stdout:ksoftirqd 48 root cwd unknown /proc/48/cwd (readlink: Permission denied) 2024-05-21T21:59:26.436 INFO:tasks.workunit.client.0.smithi151.stdout:ksoftirqd 48 root rtd unknown /proc/48/root (readlink: Permission denied) 2024-05-21T21:59:26.436 INFO:tasks.workunit.client.0.smithi151.stdout:ksoftirqd 48 root txt unknown /proc/48/exe (readlink: Permission denied) 2024-05-21T21:59:26.436 INFO:tasks.workunit.client.0.smithi151.stdout:ksoftirqd 48 root NOFD /proc/48/fd (opendir: Permission denied) 2024-05-21T21:59:26.436 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/6 50 root cwd unknown /proc/50/cwd (readlink: Permission denied) 2024-05-21T21:59:26.436 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/6 50 root rtd unknown /proc/50/root (readlink: Permission denied) 2024-05-21T21:59:26.437 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/6 50 root txt unknown /proc/50/exe (readlink: Permission denied) 2024-05-21T21:59:26.437 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/6 50 root NOFD /proc/50/fd (opendir: Permission denied) 2024-05-21T21:59:26.437 INFO:tasks.workunit.client.0.smithi151.stdout:cpuhp/7 51 root cwd unknown /proc/51/cwd (readlink: Permission denied) 2024-05-21T21:59:26.437 INFO:tasks.workunit.client.0.smithi151.stdout:cpuhp/7 51 root rtd unknown /proc/51/root (readlink: Permission denied) 2024-05-21T21:59:26.437 INFO:tasks.workunit.client.0.smithi151.stdout:cpuhp/7 51 root txt unknown /proc/51/exe (readlink: Permission denied) 2024-05-21T21:59:26.437 INFO:tasks.workunit.client.0.smithi151.stdout:cpuhp/7 51 root NOFD /proc/51/fd (opendir: Permission denied) 2024-05-21T21:59:26.437 INFO:tasks.workunit.client.0.smithi151.stdout:migration 52 root cwd unknown /proc/52/cwd (readlink: Permission denied) 2024-05-21T21:59:26.437 INFO:tasks.workunit.client.0.smithi151.stdout:migration 52 root rtd unknown /proc/52/root (readlink: Permission denied) 2024-05-21T21:59:26.437 INFO:tasks.workunit.client.0.smithi151.stdout:migration 52 root txt unknown /proc/52/exe (readlink: Permission denied) 2024-05-21T21:59:26.437 INFO:tasks.workunit.client.0.smithi151.stdout:migration 52 root NOFD /proc/52/fd (opendir: Permission denied) 2024-05-21T21:59:26.437 INFO:tasks.workunit.client.0.smithi151.stdout:ksoftirqd 53 root cwd unknown /proc/53/cwd (readlink: Permission denied) 2024-05-21T21:59:26.438 INFO:tasks.workunit.client.0.smithi151.stdout:ksoftirqd 53 root rtd unknown /proc/53/root (readlink: Permission denied) 2024-05-21T21:59:26.438 INFO:tasks.workunit.client.0.smithi151.stdout:ksoftirqd 53 root txt unknown /proc/53/exe (readlink: Permission denied) 2024-05-21T21:59:26.438 INFO:tasks.workunit.client.0.smithi151.stdout:ksoftirqd 53 root NOFD /proc/53/fd (opendir: Permission denied) 2024-05-21T21:59:26.438 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/7 55 root cwd unknown /proc/55/cwd (readlink: Permission denied) 2024-05-21T21:59:26.438 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/7 55 root rtd unknown /proc/55/root (readlink: Permission denied) 2024-05-21T21:59:26.438 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/7 55 root txt unknown /proc/55/exe (readlink: Permission denied) 2024-05-21T21:59:26.438 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/7 55 root NOFD /proc/55/fd (opendir: Permission denied) 2024-05-21T21:59:26.438 INFO:tasks.workunit.client.0.smithi151.stdout:kdevtmpfs 56 root cwd unknown /proc/56/cwd (readlink: Permission denied) 2024-05-21T21:59:26.438 INFO:tasks.workunit.client.0.smithi151.stdout:kdevtmpfs 56 root rtd unknown /proc/56/root (readlink: Permission denied) 2024-05-21T21:59:26.438 INFO:tasks.workunit.client.0.smithi151.stdout:kdevtmpfs 56 root txt unknown /proc/56/exe (readlink: Permission denied) 2024-05-21T21:59:26.439 INFO:tasks.workunit.client.0.smithi151.stdout:kdevtmpfs 56 root NOFD /proc/56/fd (opendir: Permission denied) 2024-05-21T21:59:26.439 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 57 root cwd unknown /proc/57/cwd (readlink: Permission denied) 2024-05-21T21:59:26.439 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 57 root rtd unknown /proc/57/root (readlink: Permission denied) 2024-05-21T21:59:26.439 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 57 root txt unknown /proc/57/exe (readlink: Permission denied) 2024-05-21T21:59:26.439 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 57 root NOFD /proc/57/fd (opendir: Permission denied) 2024-05-21T21:59:26.439 INFO:tasks.workunit.client.0.smithi151.stdout:kauditd 58 root cwd unknown /proc/58/cwd (readlink: Permission denied) 2024-05-21T21:59:26.439 INFO:tasks.workunit.client.0.smithi151.stdout:kauditd 58 root rtd unknown /proc/58/root (readlink: Permission denied) 2024-05-21T21:59:26.439 INFO:tasks.workunit.client.0.smithi151.stdout:kauditd 58 root txt unknown /proc/58/exe (readlink: Permission denied) 2024-05-21T21:59:26.439 INFO:tasks.workunit.client.0.smithi151.stdout:kauditd 58 root NOFD /proc/58/fd (opendir: Permission denied) 2024-05-21T21:59:26.439 INFO:tasks.workunit.client.0.smithi151.stdout:khungtask 59 root cwd unknown /proc/59/cwd (readlink: Permission denied) 2024-05-21T21:59:26.439 INFO:tasks.workunit.client.0.smithi151.stdout:khungtask 59 root rtd unknown /proc/59/root (readlink: Permission denied) 2024-05-21T21:59:26.439 INFO:tasks.workunit.client.0.smithi151.stdout:khungtask 59 root txt unknown /proc/59/exe (readlink: Permission denied) 2024-05-21T21:59:26.440 INFO:tasks.workunit.client.0.smithi151.stdout:khungtask 59 root NOFD /proc/59/fd (opendir: Permission denied) 2024-05-21T21:59:26.440 INFO:tasks.workunit.client.0.smithi151.stdout:oom_reape 61 root cwd unknown /proc/61/cwd (readlink: Permission denied) 2024-05-21T21:59:26.440 INFO:tasks.workunit.client.0.smithi151.stdout:oom_reape 61 root rtd unknown /proc/61/root (readlink: Permission denied) 2024-05-21T21:59:26.440 INFO:tasks.workunit.client.0.smithi151.stdout:oom_reape 61 root txt unknown /proc/61/exe (readlink: Permission denied) 2024-05-21T21:59:26.440 INFO:tasks.workunit.client.0.smithi151.stdout:oom_reape 61 root NOFD /proc/61/fd (opendir: Permission denied) 2024-05-21T21:59:26.440 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 63 root cwd unknown /proc/63/cwd (readlink: Permission denied) 2024-05-21T21:59:26.440 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 63 root rtd unknown /proc/63/root (readlink: Permission denied) 2024-05-21T21:59:26.440 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 63 root txt unknown /proc/63/exe (readlink: Permission denied) 2024-05-21T21:59:26.440 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 63 root NOFD /proc/63/fd (opendir: Permission denied) 2024-05-21T21:59:26.440 INFO:tasks.workunit.client.0.smithi151.stdout:kcompactd 64 root cwd unknown /proc/64/cwd (readlink: Permission denied) 2024-05-21T21:59:26.440 INFO:tasks.workunit.client.0.smithi151.stdout:kcompactd 64 root rtd unknown /proc/64/root (readlink: Permission denied) 2024-05-21T21:59:26.440 INFO:tasks.workunit.client.0.smithi151.stdout:kcompactd 64 root txt unknown /proc/64/exe (readlink: Permission denied) 2024-05-21T21:59:26.440 INFO:tasks.workunit.client.0.smithi151.stdout:kcompactd 64 root NOFD /proc/64/fd (opendir: Permission denied) 2024-05-21T21:59:26.441 INFO:tasks.workunit.client.0.smithi151.stdout:ksmd 65 root cwd unknown /proc/65/cwd (readlink: Permission denied) 2024-05-21T21:59:26.441 INFO:tasks.workunit.client.0.smithi151.stdout:ksmd 65 root rtd unknown /proc/65/root (readlink: Permission denied) 2024-05-21T21:59:26.441 INFO:tasks.workunit.client.0.smithi151.stdout:ksmd 65 root txt unknown /proc/65/exe (readlink: Permission denied) 2024-05-21T21:59:26.441 INFO:tasks.workunit.client.0.smithi151.stdout:ksmd 65 root NOFD /proc/65/fd (opendir: Permission denied) 2024-05-21T21:59:26.441 INFO:tasks.workunit.client.0.smithi151.stdout:khugepage 66 root cwd unknown /proc/66/cwd (readlink: Permission denied) 2024-05-21T21:59:26.441 INFO:tasks.workunit.client.0.smithi151.stdout:khugepage 66 root rtd unknown /proc/66/root (readlink: Permission denied) 2024-05-21T21:59:26.441 INFO:tasks.workunit.client.0.smithi151.stdout:khugepage 66 root txt unknown /proc/66/exe (readlink: Permission denied) 2024-05-21T21:59:26.441 INFO:tasks.workunit.client.0.smithi151.stdout:khugepage 66 root NOFD /proc/66/fd (opendir: Permission denied) 2024-05-21T21:59:26.441 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 67 root cwd unknown /proc/67/cwd (readlink: Permission denied) 2024-05-21T21:59:26.441 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 67 root rtd unknown /proc/67/root (readlink: Permission denied) 2024-05-21T21:59:26.441 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 67 root txt unknown /proc/67/exe (readlink: Permission denied) 2024-05-21T21:59:26.441 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 67 root NOFD /proc/67/fd (opendir: Permission denied) 2024-05-21T21:59:26.442 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 68 root cwd unknown /proc/68/cwd (readlink: Permission denied) 2024-05-21T21:59:26.442 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 68 root rtd unknown /proc/68/root (readlink: Permission denied) 2024-05-21T21:59:26.442 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 68 root txt unknown /proc/68/exe (readlink: Permission denied) 2024-05-21T21:59:26.442 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 68 root NOFD /proc/68/fd (opendir: Permission denied) 2024-05-21T21:59:26.442 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 69 root cwd unknown /proc/69/cwd (readlink: Permission denied) 2024-05-21T21:59:26.442 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 69 root rtd unknown /proc/69/root (readlink: Permission denied) 2024-05-21T21:59:26.442 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 69 root txt unknown /proc/69/exe (readlink: Permission denied) 2024-05-21T21:59:26.442 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 69 root NOFD /proc/69/fd (opendir: Permission denied) 2024-05-21T21:59:26.442 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 70 root cwd unknown /proc/70/cwd (readlink: Permission denied) 2024-05-21T21:59:26.442 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 70 root rtd unknown /proc/70/root (readlink: Permission denied) 2024-05-21T21:59:26.442 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 70 root txt unknown /proc/70/exe (readlink: Permission denied) 2024-05-21T21:59:26.442 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 70 root NOFD /proc/70/fd (opendir: Permission denied) 2024-05-21T21:59:26.443 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/2 71 root cwd unknown /proc/71/cwd (readlink: Permission denied) 2024-05-21T21:59:26.443 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/2 71 root rtd unknown /proc/71/root (readlink: Permission denied) 2024-05-21T21:59:26.443 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/2 71 root txt unknown /proc/71/exe (readlink: Permission denied) 2024-05-21T21:59:26.443 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/2 71 root NOFD /proc/71/fd (opendir: Permission denied) 2024-05-21T21:59:26.443 INFO:tasks.workunit.client.0.smithi151.stdout:irq/9-acp 78 root cwd unknown /proc/78/cwd (readlink: Permission denied) 2024-05-21T21:59:26.443 INFO:tasks.workunit.client.0.smithi151.stdout:irq/9-acp 78 root rtd unknown /proc/78/root (readlink: Permission denied) 2024-05-21T21:59:26.443 INFO:tasks.workunit.client.0.smithi151.stdout:irq/9-acp 78 root txt unknown /proc/78/exe (readlink: Permission denied) 2024-05-21T21:59:26.443 INFO:tasks.workunit.client.0.smithi151.stdout:irq/9-acp 78 root NOFD /proc/78/fd (opendir: Permission denied) 2024-05-21T21:59:26.443 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 79 root cwd unknown /proc/79/cwd (readlink: Permission denied) 2024-05-21T21:59:26.443 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 79 root rtd unknown /proc/79/root (readlink: Permission denied) 2024-05-21T21:59:26.443 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 79 root txt unknown /proc/79/exe (readlink: Permission denied) 2024-05-21T21:59:26.443 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 79 root NOFD /proc/79/fd (opendir: Permission denied) 2024-05-21T21:59:26.443 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 80 root cwd unknown /proc/80/cwd (readlink: Permission denied) 2024-05-21T21:59:26.444 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 80 root rtd unknown /proc/80/root (readlink: Permission denied) 2024-05-21T21:59:26.444 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 80 root txt unknown /proc/80/exe (readlink: Permission denied) 2024-05-21T21:59:26.444 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 80 root NOFD /proc/80/fd (opendir: Permission denied) 2024-05-21T21:59:26.444 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 81 root cwd unknown /proc/81/cwd (readlink: Permission denied) 2024-05-21T21:59:26.444 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 81 root rtd unknown /proc/81/root (readlink: Permission denied) 2024-05-21T21:59:26.444 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 81 root txt unknown /proc/81/exe (readlink: Permission denied) 2024-05-21T21:59:26.444 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 81 root NOFD /proc/81/fd (opendir: Permission denied) 2024-05-21T21:59:26.444 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 82 root cwd unknown /proc/82/cwd (readlink: Permission denied) 2024-05-21T21:59:26.444 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 82 root rtd unknown /proc/82/root (readlink: Permission denied) 2024-05-21T21:59:26.444 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 82 root txt unknown /proc/82/exe (readlink: Permission denied) 2024-05-21T21:59:26.444 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 82 root NOFD /proc/82/fd (opendir: Permission denied) 2024-05-21T21:59:26.444 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 83 root cwd unknown /proc/83/cwd (readlink: Permission denied) 2024-05-21T21:59:26.444 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 83 root rtd unknown /proc/83/root (readlink: Permission denied) 2024-05-21T21:59:26.445 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 83 root txt unknown /proc/83/exe (readlink: Permission denied) 2024-05-21T21:59:26.445 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 83 root NOFD /proc/83/fd (opendir: Permission denied) 2024-05-21T21:59:26.445 INFO:tasks.workunit.client.0.smithi151.stdout:watchdogd 84 root cwd unknown /proc/84/cwd (readlink: Permission denied) 2024-05-21T21:59:26.445 INFO:tasks.workunit.client.0.smithi151.stdout:watchdogd 84 root rtd unknown /proc/84/root (readlink: Permission denied) 2024-05-21T21:59:26.445 INFO:tasks.workunit.client.0.smithi151.stdout:watchdogd 84 root txt unknown /proc/84/exe (readlink: Permission denied) 2024-05-21T21:59:26.445 INFO:tasks.workunit.client.0.smithi151.stdout:watchdogd 84 root NOFD /proc/84/fd (opendir: Permission denied) 2024-05-21T21:59:26.445 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/0 85 root cwd unknown /proc/85/cwd (readlink: Permission denied) 2024-05-21T21:59:26.445 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/0 85 root rtd unknown /proc/85/root (readlink: Permission denied) 2024-05-21T21:59:26.445 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/0 85 root txt unknown /proc/85/exe (readlink: Permission denied) 2024-05-21T21:59:26.445 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/0 85 root NOFD /proc/85/fd (opendir: Permission denied) 2024-05-21T21:59:26.445 INFO:tasks.workunit.client.0.smithi151.stdout:kswapd0 86 root cwd unknown /proc/86/cwd (readlink: Permission denied) 2024-05-21T21:59:26.445 INFO:tasks.workunit.client.0.smithi151.stdout:kswapd0 86 root rtd unknown /proc/86/root (readlink: Permission denied) 2024-05-21T21:59:26.446 INFO:tasks.workunit.client.0.smithi151.stdout:kswapd0 86 root txt unknown /proc/86/exe (readlink: Permission denied) 2024-05-21T21:59:26.446 INFO:tasks.workunit.client.0.smithi151.stdout:kswapd0 86 root NOFD /proc/86/fd (opendir: Permission denied) 2024-05-21T21:59:26.446 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 88 root cwd unknown /proc/88/cwd (readlink: Permission denied) 2024-05-21T21:59:26.446 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 88 root rtd unknown /proc/88/root (readlink: Permission denied) 2024-05-21T21:59:26.446 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 88 root txt unknown /proc/88/exe (readlink: Permission denied) 2024-05-21T21:59:26.446 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 88 root NOFD /proc/88/fd (opendir: Permission denied) 2024-05-21T21:59:26.446 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 91 root cwd unknown /proc/91/cwd (readlink: Permission denied) 2024-05-21T21:59:26.446 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 91 root rtd unknown /proc/91/root (readlink: Permission denied) 2024-05-21T21:59:26.446 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 91 root txt unknown /proc/91/exe (readlink: Permission denied) 2024-05-21T21:59:26.446 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 91 root NOFD /proc/91/fd (opendir: Permission denied) 2024-05-21T21:59:26.446 INFO:tasks.workunit.client.0.smithi151.stdout:scsi_eh_0 93 root cwd unknown /proc/93/cwd (readlink: Permission denied) 2024-05-21T21:59:26.446 INFO:tasks.workunit.client.0.smithi151.stdout:scsi_eh_0 93 root rtd unknown /proc/93/root (readlink: Permission denied) 2024-05-21T21:59:26.446 INFO:tasks.workunit.client.0.smithi151.stdout:scsi_eh_0 93 root txt unknown /proc/93/exe (readlink: Permission denied) 2024-05-21T21:59:26.447 INFO:tasks.workunit.client.0.smithi151.stdout:scsi_eh_0 93 root NOFD /proc/93/fd (opendir: Permission denied) 2024-05-21T21:59:26.447 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 94 root cwd unknown /proc/94/cwd (readlink: Permission denied) 2024-05-21T21:59:26.447 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 94 root rtd unknown /proc/94/root (readlink: Permission denied) 2024-05-21T21:59:26.447 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 94 root txt unknown /proc/94/exe (readlink: Permission denied) 2024-05-21T21:59:26.447 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 94 root NOFD /proc/94/fd (opendir: Permission denied) 2024-05-21T21:59:26.447 INFO:tasks.workunit.client.0.smithi151.stdout:scsi_eh_1 95 root cwd unknown /proc/95/cwd (readlink: Permission denied) 2024-05-21T21:59:26.447 INFO:tasks.workunit.client.0.smithi151.stdout:scsi_eh_1 95 root rtd unknown /proc/95/root (readlink: Permission denied) 2024-05-21T21:59:26.447 INFO:tasks.workunit.client.0.smithi151.stdout:scsi_eh_1 95 root txt unknown /proc/95/exe (readlink: Permission denied) 2024-05-21T21:59:26.447 INFO:tasks.workunit.client.0.smithi151.stdout:scsi_eh_1 95 root NOFD /proc/95/fd (opendir: Permission denied) 2024-05-21T21:59:26.447 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 96 root cwd unknown /proc/96/cwd (readlink: Permission denied) 2024-05-21T21:59:26.447 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 96 root rtd unknown /proc/96/root (readlink: Permission denied) 2024-05-21T21:59:26.447 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 96 root txt unknown /proc/96/exe (readlink: Permission denied) 2024-05-21T21:59:26.447 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 96 root NOFD /proc/96/fd (opendir: Permission denied) 2024-05-21T21:59:26.448 INFO:tasks.workunit.client.0.smithi151.stdout:scsi_eh_2 97 root cwd unknown /proc/97/cwd (readlink: Permission denied) 2024-05-21T21:59:26.448 INFO:tasks.workunit.client.0.smithi151.stdout:scsi_eh_2 97 root rtd unknown /proc/97/root (readlink: Permission denied) 2024-05-21T21:59:26.448 INFO:tasks.workunit.client.0.smithi151.stdout:scsi_eh_2 97 root txt unknown /proc/97/exe (readlink: Permission denied) 2024-05-21T21:59:26.448 INFO:tasks.workunit.client.0.smithi151.stdout:scsi_eh_2 97 root NOFD /proc/97/fd (opendir: Permission denied) 2024-05-21T21:59:26.448 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 98 root cwd unknown /proc/98/cwd (readlink: Permission denied) 2024-05-21T21:59:26.448 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 98 root rtd unknown /proc/98/root (readlink: Permission denied) 2024-05-21T21:59:26.448 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 98 root txt unknown /proc/98/exe (readlink: Permission denied) 2024-05-21T21:59:26.448 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 98 root NOFD /proc/98/fd (opendir: Permission denied) 2024-05-21T21:59:26.448 INFO:tasks.workunit.client.0.smithi151.stdout:scsi_eh_3 99 root cwd unknown /proc/99/cwd (readlink: Permission denied) 2024-05-21T21:59:26.448 INFO:tasks.workunit.client.0.smithi151.stdout:scsi_eh_3 99 root rtd unknown /proc/99/root (readlink: Permission denied) 2024-05-21T21:59:26.448 INFO:tasks.workunit.client.0.smithi151.stdout:scsi_eh_3 99 root txt unknown /proc/99/exe (readlink: Permission denied) 2024-05-21T21:59:26.448 INFO:tasks.workunit.client.0.smithi151.stdout:scsi_eh_3 99 root NOFD /proc/99/fd (opendir: Permission denied) 2024-05-21T21:59:26.448 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 100 root cwd unknown /proc/100/cwd (readlink: Permission denied) 2024-05-21T21:59:26.449 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 100 root rtd unknown /proc/100/root (readlink: Permission denied) 2024-05-21T21:59:26.449 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 100 root txt unknown /proc/100/exe (readlink: Permission denied) 2024-05-21T21:59:26.449 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 100 root NOFD /proc/100/fd (opendir: Permission denied) 2024-05-21T21:59:26.449 INFO:tasks.workunit.client.0.smithi151.stdout:scsi_eh_4 104 root cwd unknown /proc/104/cwd (readlink: Permission denied) 2024-05-21T21:59:26.449 INFO:tasks.workunit.client.0.smithi151.stdout:scsi_eh_4 104 root rtd unknown /proc/104/root (readlink: Permission denied) 2024-05-21T21:59:26.449 INFO:tasks.workunit.client.0.smithi151.stdout:scsi_eh_4 104 root txt unknown /proc/104/exe (readlink: Permission denied) 2024-05-21T21:59:26.449 INFO:tasks.workunit.client.0.smithi151.stdout:scsi_eh_4 104 root NOFD /proc/104/fd (opendir: Permission denied) 2024-05-21T21:59:26.449 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 105 root cwd unknown /proc/105/cwd (readlink: Permission denied) 2024-05-21T21:59:26.449 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 105 root rtd unknown /proc/105/root (readlink: Permission denied) 2024-05-21T21:59:26.449 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 105 root txt unknown /proc/105/exe (readlink: Permission denied) 2024-05-21T21:59:26.449 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 105 root NOFD /proc/105/fd (opendir: Permission denied) 2024-05-21T21:59:26.449 INFO:tasks.workunit.client.0.smithi151.stdout:scsi_eh_5 106 root cwd unknown /proc/106/cwd (readlink: Permission denied) 2024-05-21T21:59:26.449 INFO:tasks.workunit.client.0.smithi151.stdout:scsi_eh_5 106 root rtd unknown /proc/106/root (readlink: Permission denied) 2024-05-21T21:59:26.449 INFO:tasks.workunit.client.0.smithi151.stdout:scsi_eh_5 106 root txt unknown /proc/106/exe (readlink: Permission denied) 2024-05-21T21:59:26.449 INFO:tasks.workunit.client.0.smithi151.stdout:scsi_eh_5 106 root NOFD /proc/106/fd (opendir: Permission denied) 2024-05-21T21:59:26.450 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 107 root cwd unknown /proc/107/cwd (readlink: Permission denied) 2024-05-21T21:59:26.450 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 107 root rtd unknown /proc/107/root (readlink: Permission denied) 2024-05-21T21:59:26.450 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 107 root txt unknown /proc/107/exe (readlink: Permission denied) 2024-05-21T21:59:26.450 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 107 root NOFD /proc/107/fd (opendir: Permission denied) 2024-05-21T21:59:26.450 INFO:tasks.workunit.client.0.smithi151.stdout:scsi_eh_6 108 root cwd unknown /proc/108/cwd (readlink: Permission denied) 2024-05-21T21:59:26.450 INFO:tasks.workunit.client.0.smithi151.stdout:scsi_eh_6 108 root rtd unknown /proc/108/root (readlink: Permission denied) 2024-05-21T21:59:26.450 INFO:tasks.workunit.client.0.smithi151.stdout:scsi_eh_6 108 root txt unknown /proc/108/exe (readlink: Permission denied) 2024-05-21T21:59:26.450 INFO:tasks.workunit.client.0.smithi151.stdout:scsi_eh_6 108 root NOFD /proc/108/fd (opendir: Permission denied) 2024-05-21T21:59:26.450 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 109 root cwd unknown /proc/109/cwd (readlink: Permission denied) 2024-05-21T21:59:26.450 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 109 root rtd unknown /proc/109/root (readlink: Permission denied) 2024-05-21T21:59:26.450 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 109 root txt unknown /proc/109/exe (readlink: Permission denied) 2024-05-21T21:59:26.450 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 109 root NOFD /proc/109/fd (opendir: Permission denied) 2024-05-21T21:59:26.451 INFO:tasks.workunit.client.0.smithi151.stdout:scsi_eh_7 110 root cwd unknown /proc/110/cwd (readlink: Permission denied) 2024-05-21T21:59:26.451 INFO:tasks.workunit.client.0.smithi151.stdout:scsi_eh_7 110 root rtd unknown /proc/110/root (readlink: Permission denied) 2024-05-21T21:59:26.451 INFO:tasks.workunit.client.0.smithi151.stdout:scsi_eh_7 110 root txt unknown /proc/110/exe (readlink: Permission denied) 2024-05-21T21:59:26.451 INFO:tasks.workunit.client.0.smithi151.stdout:scsi_eh_7 110 root NOFD /proc/110/fd (opendir: Permission denied) 2024-05-21T21:59:26.451 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 111 root cwd unknown /proc/111/cwd (readlink: Permission denied) 2024-05-21T21:59:26.451 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 111 root rtd unknown /proc/111/root (readlink: Permission denied) 2024-05-21T21:59:26.451 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 111 root txt unknown /proc/111/exe (readlink: Permission denied) 2024-05-21T21:59:26.451 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 111 root NOFD /proc/111/fd (opendir: Permission denied) 2024-05-21T21:59:26.451 INFO:tasks.workunit.client.0.smithi151.stdout:scsi_eh_8 112 root cwd unknown /proc/112/cwd (readlink: Permission denied) 2024-05-21T21:59:26.451 INFO:tasks.workunit.client.0.smithi151.stdout:scsi_eh_8 112 root rtd unknown /proc/112/root (readlink: Permission denied) 2024-05-21T21:59:26.451 INFO:tasks.workunit.client.0.smithi151.stdout:scsi_eh_8 112 root txt unknown /proc/112/exe (readlink: Permission denied) 2024-05-21T21:59:26.451 INFO:tasks.workunit.client.0.smithi151.stdout:scsi_eh_8 112 root NOFD /proc/112/fd (opendir: Permission denied) 2024-05-21T21:59:26.451 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 113 root cwd unknown /proc/113/cwd (readlink: Permission denied) 2024-05-21T21:59:26.452 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 113 root rtd unknown /proc/113/root (readlink: Permission denied) 2024-05-21T21:59:26.452 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 113 root txt unknown /proc/113/exe (readlink: Permission denied) 2024-05-21T21:59:26.452 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 113 root NOFD /proc/113/fd (opendir: Permission denied) 2024-05-21T21:59:26.452 INFO:tasks.workunit.client.0.smithi151.stdout:scsi_eh_9 114 root cwd unknown /proc/114/cwd (readlink: Permission denied) 2024-05-21T21:59:26.452 INFO:tasks.workunit.client.0.smithi151.stdout:scsi_eh_9 114 root rtd unknown /proc/114/root (readlink: Permission denied) 2024-05-21T21:59:26.452 INFO:tasks.workunit.client.0.smithi151.stdout:scsi_eh_9 114 root txt unknown /proc/114/exe (readlink: Permission denied) 2024-05-21T21:59:26.452 INFO:tasks.workunit.client.0.smithi151.stdout:scsi_eh_9 114 root NOFD /proc/114/fd (opendir: Permission denied) 2024-05-21T21:59:26.452 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 115 root cwd unknown /proc/115/cwd (readlink: Permission denied) 2024-05-21T21:59:26.452 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 115 root rtd unknown /proc/115/root (readlink: Permission denied) 2024-05-21T21:59:26.452 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 115 root txt unknown /proc/115/exe (readlink: Permission denied) 2024-05-21T21:59:26.452 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 115 root NOFD /proc/115/fd (opendir: Permission denied) 2024-05-21T21:59:26.452 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/u 119 root cwd unknown /proc/119/cwd (readlink: Permission denied) 2024-05-21T21:59:26.452 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/u 119 root rtd unknown /proc/119/root (readlink: Permission denied) 2024-05-21T21:59:26.453 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/u 119 root txt unknown /proc/119/exe (readlink: Permission denied) 2024-05-21T21:59:26.453 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/u 119 root NOFD /proc/119/fd (opendir: Permission denied) 2024-05-21T21:59:26.453 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/u 120 root cwd unknown /proc/120/cwd (readlink: Permission denied) 2024-05-21T21:59:26.453 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/u 120 root rtd unknown /proc/120/root (readlink: Permission denied) 2024-05-21T21:59:26.453 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/u 120 root txt unknown /proc/120/exe (readlink: Permission denied) 2024-05-21T21:59:26.453 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/u 120 root NOFD /proc/120/fd (opendir: Permission denied) 2024-05-21T21:59:26.453 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/4 127 root cwd unknown /proc/127/cwd (readlink: Permission denied) 2024-05-21T21:59:26.453 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/4 127 root rtd unknown /proc/127/root (readlink: Permission denied) 2024-05-21T21:59:26.453 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/4 127 root txt unknown /proc/127/exe (readlink: Permission denied) 2024-05-21T21:59:26.453 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/4 127 root NOFD /proc/127/fd (opendir: Permission denied) 2024-05-21T21:59:26.453 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 128 root cwd unknown /proc/128/cwd (readlink: Permission denied) 2024-05-21T21:59:26.453 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 128 root rtd unknown /proc/128/root (readlink: Permission denied) 2024-05-21T21:59:26.453 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 128 root txt unknown /proc/128/exe (readlink: Permission denied) 2024-05-21T21:59:26.453 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 128 root NOFD /proc/128/fd (opendir: Permission denied) 2024-05-21T21:59:26.454 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 129 root cwd unknown /proc/129/cwd (readlink: Permission denied) 2024-05-21T21:59:26.454 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 129 root rtd unknown /proc/129/root (readlink: Permission denied) 2024-05-21T21:59:26.454 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 129 root txt unknown /proc/129/exe (readlink: Permission denied) 2024-05-21T21:59:26.454 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 129 root NOFD /proc/129/fd (opendir: Permission denied) 2024-05-21T21:59:26.454 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/5 130 root cwd unknown /proc/130/cwd (readlink: Permission denied) 2024-05-21T21:59:26.454 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/5 130 root rtd unknown /proc/130/root (readlink: Permission denied) 2024-05-21T21:59:26.454 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/5 130 root txt unknown /proc/130/exe (readlink: Permission denied) 2024-05-21T21:59:26.454 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/5 130 root NOFD /proc/130/fd (opendir: Permission denied) 2024-05-21T21:59:26.454 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 131 root cwd unknown /proc/131/cwd (readlink: Permission denied) 2024-05-21T21:59:26.454 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 131 root rtd unknown /proc/131/root (readlink: Permission denied) 2024-05-21T21:59:26.454 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 131 root txt unknown /proc/131/exe (readlink: Permission denied) 2024-05-21T21:59:26.454 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 131 root NOFD /proc/131/fd (opendir: Permission denied) 2024-05-21T21:59:26.455 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/u 139 root cwd unknown /proc/139/cwd (readlink: Permission denied) 2024-05-21T21:59:26.455 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/u 139 root rtd unknown /proc/139/root (readlink: Permission denied) 2024-05-21T21:59:26.455 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/u 139 root txt unknown /proc/139/exe (readlink: Permission denied) 2024-05-21T21:59:26.455 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/u 139 root NOFD /proc/139/fd (opendir: Permission denied) 2024-05-21T21:59:26.455 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/6 248 root cwd unknown /proc/248/cwd (readlink: Permission denied) 2024-05-21T21:59:26.455 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/6 248 root rtd unknown /proc/248/root (readlink: Permission denied) 2024-05-21T21:59:26.455 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/6 248 root txt unknown /proc/248/exe (readlink: Permission denied) 2024-05-21T21:59:26.455 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/6 248 root NOFD /proc/248/fd (opendir: Permission denied) 2024-05-21T21:59:26.455 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/7 250 root cwd unknown /proc/250/cwd (readlink: Permission denied) 2024-05-21T21:59:26.455 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/7 250 root rtd unknown /proc/250/root (readlink: Permission denied) 2024-05-21T21:59:26.455 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/7 250 root txt unknown /proc/250/exe (readlink: Permission denied) 2024-05-21T21:59:26.455 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/7 250 root NOFD /proc/250/fd (opendir: Permission denied) 2024-05-21T21:59:26.455 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/1 253 root cwd unknown /proc/253/cwd (readlink: Permission denied) 2024-05-21T21:59:26.456 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/1 253 root rtd unknown /proc/253/root (readlink: Permission denied) 2024-05-21T21:59:26.456 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/1 253 root txt unknown /proc/253/exe (readlink: Permission denied) 2024-05-21T21:59:26.456 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/1 253 root NOFD /proc/253/fd (opendir: Permission denied) 2024-05-21T21:59:26.456 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/2 257 root cwd unknown /proc/257/cwd (readlink: Permission denied) 2024-05-21T21:59:26.456 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/2 257 root rtd unknown /proc/257/root (readlink: Permission denied) 2024-05-21T21:59:26.456 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/2 257 root txt unknown /proc/257/exe (readlink: Permission denied) 2024-05-21T21:59:26.456 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/2 257 root NOFD /proc/257/fd (opendir: Permission denied) 2024-05-21T21:59:26.456 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/1 259 root cwd unknown /proc/259/cwd (readlink: Permission denied) 2024-05-21T21:59:26.456 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/1 259 root rtd unknown /proc/259/root (readlink: Permission denied) 2024-05-21T21:59:26.456 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/1 259 root txt unknown /proc/259/exe (readlink: Permission denied) 2024-05-21T21:59:26.456 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/1 259 root NOFD /proc/259/fd (opendir: Permission denied) 2024-05-21T21:59:26.456 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/3 268 root cwd unknown /proc/268/cwd (readlink: Permission denied) 2024-05-21T21:59:26.457 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/3 268 root rtd unknown /proc/268/root (readlink: Permission denied) 2024-05-21T21:59:26.457 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/3 268 root txt unknown /proc/268/exe (readlink: Permission denied) 2024-05-21T21:59:26.457 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/3 268 root NOFD /proc/268/fd (opendir: Permission denied) 2024-05-21T21:59:26.457 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 398 root cwd unknown /proc/398/cwd (readlink: Permission denied) 2024-05-21T21:59:26.457 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 398 root rtd unknown /proc/398/root (readlink: Permission denied) 2024-05-21T21:59:26.457 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 398 root txt unknown /proc/398/exe (readlink: Permission denied) 2024-05-21T21:59:26.457 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 398 root NOFD /proc/398/fd (opendir: Permission denied) 2024-05-21T21:59:26.457 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 399 root cwd unknown /proc/399/cwd (readlink: Permission denied) 2024-05-21T21:59:26.457 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 399 root rtd unknown /proc/399/root (readlink: Permission denied) 2024-05-21T21:59:26.457 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 399 root txt unknown /proc/399/exe (readlink: Permission denied) 2024-05-21T21:59:26.457 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 399 root NOFD /proc/399/fd (opendir: Permission denied) 2024-05-21T21:59:26.457 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 400 root cwd unknown /proc/400/cwd (readlink: Permission denied) 2024-05-21T21:59:26.457 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 400 root rtd unknown /proc/400/root (readlink: Permission denied) 2024-05-21T21:59:26.458 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 400 root txt unknown /proc/400/exe (readlink: Permission denied) 2024-05-21T21:59:26.458 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 400 root NOFD /proc/400/fd (opendir: Permission denied) 2024-05-21T21:59:26.458 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 429 root cwd unknown /proc/429/cwd (readlink: Permission denied) 2024-05-21T21:59:26.458 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 429 root rtd unknown /proc/429/root (readlink: Permission denied) 2024-05-21T21:59:26.458 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 429 root txt unknown /proc/429/exe (readlink: Permission denied) 2024-05-21T21:59:26.458 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 429 root NOFD /proc/429/fd (opendir: Permission denied) 2024-05-21T21:59:26.458 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/1 443 root cwd unknown /proc/443/cwd (readlink: Permission denied) 2024-05-21T21:59:26.458 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/1 443 root rtd unknown /proc/443/root (readlink: Permission denied) 2024-05-21T21:59:26.458 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/1 443 root txt unknown /proc/443/exe (readlink: Permission denied) 2024-05-21T21:59:26.458 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/1 443 root NOFD /proc/443/fd (opendir: Permission denied) 2024-05-21T21:59:26.458 INFO:tasks.workunit.client.0.smithi151.stdout:jbd2/sda1 444 root cwd unknown /proc/444/cwd (readlink: Permission denied) 2024-05-21T21:59:26.458 INFO:tasks.workunit.client.0.smithi151.stdout:jbd2/sda1 444 root rtd unknown /proc/444/root (readlink: Permission denied) 2024-05-21T21:59:26.459 INFO:tasks.workunit.client.0.smithi151.stdout:jbd2/sda1 444 root txt unknown /proc/444/exe (readlink: Permission denied) 2024-05-21T21:59:26.459 INFO:tasks.workunit.client.0.smithi151.stdout:jbd2/sda1 444 root NOFD /proc/444/fd (opendir: Permission denied) 2024-05-21T21:59:26.459 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 445 root cwd unknown /proc/445/cwd (readlink: Permission denied) 2024-05-21T21:59:26.459 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 445 root rtd unknown /proc/445/root (readlink: Permission denied) 2024-05-21T21:59:26.459 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 445 root txt unknown /proc/445/exe (readlink: Permission denied) 2024-05-21T21:59:26.459 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 445 root NOFD /proc/445/fd (opendir: Permission denied) 2024-05-21T21:59:26.459 INFO:tasks.workunit.client.0.smithi151.stdout:systemd-j 521 root cwd unknown /proc/521/cwd (readlink: Permission denied) 2024-05-21T21:59:26.459 INFO:tasks.workunit.client.0.smithi151.stdout:systemd-j 521 root rtd unknown /proc/521/root (readlink: Permission denied) 2024-05-21T21:59:26.459 INFO:tasks.workunit.client.0.smithi151.stdout:systemd-j 521 root txt unknown /proc/521/exe (readlink: Permission denied) 2024-05-21T21:59:26.459 INFO:tasks.workunit.client.0.smithi151.stdout:systemd-j 521 root NOFD /proc/521/fd (opendir: Permission denied) 2024-05-21T21:59:26.459 INFO:tasks.workunit.client.0.smithi151.stdout:ext4lazyi 534 root cwd unknown /proc/534/cwd (readlink: Permission denied) 2024-05-21T21:59:26.459 INFO:tasks.workunit.client.0.smithi151.stdout:ext4lazyi 534 root rtd unknown /proc/534/root (readlink: Permission denied) 2024-05-21T21:59:26.459 INFO:tasks.workunit.client.0.smithi151.stdout:ext4lazyi 534 root txt unknown /proc/534/exe (readlink: Permission denied) 2024-05-21T21:59:26.460 INFO:tasks.workunit.client.0.smithi151.stdout:ext4lazyi 534 root NOFD /proc/534/fd (opendir: Permission denied) 2024-05-21T21:59:26.460 INFO:tasks.workunit.client.0.smithi151.stdout:systemd-u 541 root cwd unknown /proc/541/cwd (readlink: Permission denied) 2024-05-21T21:59:26.460 INFO:tasks.workunit.client.0.smithi151.stdout:systemd-u 541 root rtd unknown /proc/541/root (readlink: Permission denied) 2024-05-21T21:59:26.460 INFO:tasks.workunit.client.0.smithi151.stdout:systemd-u 541 root txt unknown /proc/541/exe (readlink: Permission denied) 2024-05-21T21:59:26.460 INFO:tasks.workunit.client.0.smithi151.stdout:systemd-u 541 root NOFD /proc/541/fd (opendir: Permission denied) 2024-05-21T21:59:26.460 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 590 root cwd unknown /proc/590/cwd (readlink: Permission denied) 2024-05-21T21:59:26.460 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 590 root rtd unknown /proc/590/root (readlink: Permission denied) 2024-05-21T21:59:26.460 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 590 root txt unknown /proc/590/exe (readlink: Permission denied) 2024-05-21T21:59:26.460 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 590 root NOFD /proc/590/fd (opendir: Permission denied) 2024-05-21T21:59:26.460 INFO:tasks.workunit.client.0.smithi151.stdout:rpcbind 604 rpc cwd unknown /proc/604/cwd (readlink: Permission denied) 2024-05-21T21:59:26.460 INFO:tasks.workunit.client.0.smithi151.stdout:rpcbind 604 rpc rtd unknown /proc/604/root (readlink: Permission denied) 2024-05-21T21:59:26.460 INFO:tasks.workunit.client.0.smithi151.stdout:rpcbind 604 rpc txt unknown /proc/604/exe (readlink: Permission denied) 2024-05-21T21:59:26.461 INFO:tasks.workunit.client.0.smithi151.stdout:rpcbind 604 rpc NOFD /proc/604/fd (opendir: Permission denied) 2024-05-21T21:59:26.461 INFO:tasks.workunit.client.0.smithi151.stdout:kipmi0 607 root cwd unknown /proc/607/cwd (readlink: Permission denied) 2024-05-21T21:59:26.461 INFO:tasks.workunit.client.0.smithi151.stdout:kipmi0 607 root rtd unknown /proc/607/root (readlink: Permission denied) 2024-05-21T21:59:26.461 INFO:tasks.workunit.client.0.smithi151.stdout:kipmi0 607 root txt unknown /proc/607/exe (readlink: Permission denied) 2024-05-21T21:59:26.461 INFO:tasks.workunit.client.0.smithi151.stdout:kipmi0 607 root NOFD /proc/607/fd (opendir: Permission denied) 2024-05-21T21:59:26.461 INFO:tasks.workunit.client.0.smithi151.stdout:auditd 613 root cwd unknown /proc/613/cwd (readlink: Permission denied) 2024-05-21T21:59:26.461 INFO:tasks.workunit.client.0.smithi151.stdout:auditd 613 root rtd unknown /proc/613/root (readlink: Permission denied) 2024-05-21T21:59:26.461 INFO:tasks.workunit.client.0.smithi151.stdout:auditd 613 root txt unknown /proc/613/exe (readlink: Permission denied) 2024-05-21T21:59:26.461 INFO:tasks.workunit.client.0.smithi151.stdout:auditd 613 root NOFD /proc/613/fd (opendir: Permission denied) 2024-05-21T21:59:26.461 INFO:tasks.workunit.client.0.smithi151.stdout:auditd 613 614 auditd root cwd unknown /proc/613/task/614/cwd (readlink: Permission denied) 2024-05-21T21:59:26.461 INFO:tasks.workunit.client.0.smithi151.stdout:auditd 613 614 auditd root rtd unknown /proc/613/task/614/root (readlink: Permission denied) 2024-05-21T21:59:26.461 INFO:tasks.workunit.client.0.smithi151.stdout:auditd 613 614 auditd root txt unknown /proc/613/task/614/exe (readlink: Permission denied) 2024-05-21T21:59:26.461 INFO:tasks.workunit.client.0.smithi151.stdout:auditd 613 614 auditd root NOFD /proc/613/task/614/fd (opendir: Permission denied) 2024-05-21T21:59:26.462 INFO:tasks.workunit.client.0.smithi151.stdout:auditd 613 616 auditd root cwd unknown /proc/613/task/616/cwd (readlink: Permission denied) 2024-05-21T21:59:26.462 INFO:tasks.workunit.client.0.smithi151.stdout:auditd 613 616 auditd root rtd unknown /proc/613/task/616/root (readlink: Permission denied) 2024-05-21T21:59:26.462 INFO:tasks.workunit.client.0.smithi151.stdout:auditd 613 616 auditd root txt unknown /proc/613/task/616/exe (readlink: Permission denied) 2024-05-21T21:59:26.462 INFO:tasks.workunit.client.0.smithi151.stdout:auditd 613 616 auditd root NOFD /proc/613/task/616/fd (opendir: Permission denied) 2024-05-21T21:59:26.462 INFO:tasks.workunit.client.0.smithi151.stdout:sedispatc 615 root cwd unknown /proc/615/cwd (readlink: Permission denied) 2024-05-21T21:59:26.462 INFO:tasks.workunit.client.0.smithi151.stdout:sedispatc 615 root rtd unknown /proc/615/root (readlink: Permission denied) 2024-05-21T21:59:26.462 INFO:tasks.workunit.client.0.smithi151.stdout:sedispatc 615 root txt unknown /proc/615/exe (readlink: Permission denied) 2024-05-21T21:59:26.462 INFO:tasks.workunit.client.0.smithi151.stdout:sedispatc 615 root NOFD /proc/615/fd (opendir: Permission denied) 2024-05-21T21:59:26.462 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 620 root cwd unknown /proc/620/cwd (readlink: Permission denied) 2024-05-21T21:59:26.462 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 620 root rtd unknown /proc/620/root (readlink: Permission denied) 2024-05-21T21:59:26.462 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 620 root txt unknown /proc/620/exe (readlink: Permission denied) 2024-05-21T21:59:26.462 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 620 root NOFD /proc/620/fd (opendir: Permission denied) 2024-05-21T21:59:26.462 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 621 root cwd unknown /proc/621/cwd (readlink: Permission denied) 2024-05-21T21:59:26.463 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 621 root rtd unknown /proc/621/root (readlink: Permission denied) 2024-05-21T21:59:26.463 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 621 root txt unknown /proc/621/exe (readlink: Permission denied) 2024-05-21T21:59:26.463 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 621 root NOFD /proc/621/fd (opendir: Permission denied) 2024-05-21T21:59:26.463 INFO:tasks.workunit.client.0.smithi151.stdout:dbus-brok 657 dbus cwd unknown /proc/657/cwd (readlink: Permission denied) 2024-05-21T21:59:26.463 INFO:tasks.workunit.client.0.smithi151.stdout:dbus-brok 657 dbus rtd unknown /proc/657/root (readlink: Permission denied) 2024-05-21T21:59:26.463 INFO:tasks.workunit.client.0.smithi151.stdout:dbus-brok 657 dbus txt unknown /proc/657/exe (readlink: Permission denied) 2024-05-21T21:59:26.463 INFO:tasks.workunit.client.0.smithi151.stdout:dbus-brok 657 dbus NOFD /proc/657/fd (opendir: Permission denied) 2024-05-21T21:59:26.463 INFO:tasks.workunit.client.0.smithi151.stdout:dbus-brok 658 dbus cwd unknown /proc/658/cwd (readlink: Permission denied) 2024-05-21T21:59:26.463 INFO:tasks.workunit.client.0.smithi151.stdout:dbus-brok 658 dbus rtd unknown /proc/658/root (readlink: Permission denied) 2024-05-21T21:59:26.463 INFO:tasks.workunit.client.0.smithi151.stdout:dbus-brok 658 dbus txt unknown /proc/658/exe (readlink: Permission denied) 2024-05-21T21:59:26.464 INFO:tasks.workunit.client.0.smithi151.stdout:dbus-brok 658 dbus NOFD /proc/658/fd (opendir: Permission denied) 2024-05-21T21:59:26.464 INFO:tasks.workunit.client.0.smithi151.stdout:NetworkMa 659 root cwd unknown /proc/659/cwd (readlink: Permission denied) 2024-05-21T21:59:26.464 INFO:tasks.workunit.client.0.smithi151.stdout:NetworkMa 659 root rtd unknown /proc/659/root (readlink: Permission denied) 2024-05-21T21:59:26.464 INFO:tasks.workunit.client.0.smithi151.stdout:NetworkMa 659 root txt unknown /proc/659/exe (readlink: Permission denied) 2024-05-21T21:59:26.464 INFO:tasks.workunit.client.0.smithi151.stdout:NetworkMa 659 root NOFD /proc/659/fd (opendir: Permission denied) 2024-05-21T21:59:26.464 INFO:tasks.workunit.client.0.smithi151.stdout:NetworkMa 659 674 gmain root cwd unknown /proc/659/task/674/cwd (readlink: Permission denied) 2024-05-21T21:59:26.464 INFO:tasks.workunit.client.0.smithi151.stdout:NetworkMa 659 674 gmain root rtd unknown /proc/659/task/674/root (readlink: Permission denied) 2024-05-21T21:59:26.464 INFO:tasks.workunit.client.0.smithi151.stdout:NetworkMa 659 674 gmain root txt unknown /proc/659/task/674/exe (readlink: Permission denied) 2024-05-21T21:59:26.464 INFO:tasks.workunit.client.0.smithi151.stdout:NetworkMa 659 674 gmain root NOFD /proc/659/task/674/fd (opendir: Permission denied) 2024-05-21T21:59:26.464 INFO:tasks.workunit.client.0.smithi151.stdout:NetworkMa 659 675 gdbus root cwd unknown /proc/659/task/675/cwd (readlink: Permission denied) 2024-05-21T21:59:26.464 INFO:tasks.workunit.client.0.smithi151.stdout:NetworkMa 659 675 gdbus root rtd unknown /proc/659/task/675/root (readlink: Permission denied) 2024-05-21T21:59:26.465 INFO:tasks.workunit.client.0.smithi151.stdout:NetworkMa 659 675 gdbus root txt unknown /proc/659/task/675/exe (readlink: Permission denied) 2024-05-21T21:59:26.465 INFO:tasks.workunit.client.0.smithi151.stdout:NetworkMa 659 675 gdbus root NOFD /proc/659/task/675/fd (opendir: Permission denied) 2024-05-21T21:59:26.465 INFO:tasks.workunit.client.0.smithi151.stdout:irqbalanc 662 root cwd unknown /proc/662/cwd (readlink: Permission denied) 2024-05-21T21:59:26.465 INFO:tasks.workunit.client.0.smithi151.stdout:irqbalanc 662 root rtd unknown /proc/662/root (readlink: Permission denied) 2024-05-21T21:59:26.465 INFO:tasks.workunit.client.0.smithi151.stdout:irqbalanc 662 root txt unknown /proc/662/exe (readlink: Permission denied) 2024-05-21T21:59:26.465 INFO:tasks.workunit.client.0.smithi151.stdout:irqbalanc 662 root NOFD /proc/662/fd (opendir: Permission denied) 2024-05-21T21:59:26.465 INFO:tasks.workunit.client.0.smithi151.stdout:irqbalanc 662 667 gmain root cwd unknown /proc/662/task/667/cwd (readlink: Permission denied) 2024-05-21T21:59:26.465 INFO:tasks.workunit.client.0.smithi151.stdout:irqbalanc 662 667 gmain root rtd unknown /proc/662/task/667/root (readlink: Permission denied) 2024-05-21T21:59:26.465 INFO:tasks.workunit.client.0.smithi151.stdout:irqbalanc 662 667 gmain root txt unknown /proc/662/task/667/exe (readlink: Permission denied) 2024-05-21T21:59:26.465 INFO:tasks.workunit.client.0.smithi151.stdout:irqbalanc 662 667 gmain root NOFD /proc/662/task/667/fd (opendir: Permission denied) 2024-05-21T21:59:26.465 INFO:tasks.workunit.client.0.smithi151.stdout:lsmd 663 libstoragemgmt cwd unknown /proc/663/cwd (readlink: Permission denied) 2024-05-21T21:59:26.466 INFO:tasks.workunit.client.0.smithi151.stdout:lsmd 663 libstoragemgmt rtd unknown /proc/663/root (readlink: Permission denied) 2024-05-21T21:59:26.466 INFO:tasks.workunit.client.0.smithi151.stdout:lsmd 663 libstoragemgmt txt unknown /proc/663/exe (readlink: Permission denied) 2024-05-21T21:59:26.466 INFO:tasks.workunit.client.0.smithi151.stdout:lsmd 663 libstoragemgmt NOFD /proc/663/fd (opendir: Permission denied) 2024-05-21T21:59:26.466 INFO:tasks.workunit.client.0.smithi151.stdout:smartd 664 root cwd unknown /proc/664/cwd (readlink: Permission denied) 2024-05-21T21:59:26.466 INFO:tasks.workunit.client.0.smithi151.stdout:smartd 664 root rtd unknown /proc/664/root (readlink: Permission denied) 2024-05-21T21:59:26.466 INFO:tasks.workunit.client.0.smithi151.stdout:smartd 664 root txt unknown /proc/664/exe (readlink: Permission denied) 2024-05-21T21:59:26.466 INFO:tasks.workunit.client.0.smithi151.stdout:smartd 664 root NOFD /proc/664/fd (opendir: Permission denied) 2024-05-21T21:59:26.466 INFO:tasks.workunit.client.0.smithi151.stdout:systemd-l 666 root cwd unknown /proc/666/cwd (readlink: Permission denied) 2024-05-21T21:59:26.466 INFO:tasks.workunit.client.0.smithi151.stdout:systemd-l 666 root rtd unknown /proc/666/root (readlink: Permission denied) 2024-05-21T21:59:26.466 INFO:tasks.workunit.client.0.smithi151.stdout:systemd-l 666 root txt unknown /proc/666/exe (readlink: Permission denied) 2024-05-21T21:59:26.466 INFO:tasks.workunit.client.0.smithi151.stdout:systemd-l 666 root NOFD /proc/666/fd (opendir: Permission denied) 2024-05-21T21:59:26.467 INFO:tasks.workunit.client.0.smithi151.stdout:sshd 705 root cwd unknown /proc/705/cwd (readlink: Permission denied) 2024-05-21T21:59:26.467 INFO:tasks.workunit.client.0.smithi151.stdout:sshd 705 root rtd unknown /proc/705/root (readlink: Permission denied) 2024-05-21T21:59:26.467 INFO:tasks.workunit.client.0.smithi151.stdout:sshd 705 root txt unknown /proc/705/exe (readlink: Permission denied) 2024-05-21T21:59:26.467 INFO:tasks.workunit.client.0.smithi151.stdout:sshd 705 root NOFD /proc/705/fd (opendir: Permission denied) 2024-05-21T21:59:26.467 INFO:tasks.workunit.client.0.smithi151.stdout:gssproxy 713 root cwd unknown /proc/713/cwd (readlink: Permission denied) 2024-05-21T21:59:26.467 INFO:tasks.workunit.client.0.smithi151.stdout:gssproxy 713 root rtd unknown /proc/713/root (readlink: Permission denied) 2024-05-21T21:59:26.467 INFO:tasks.workunit.client.0.smithi151.stdout:gssproxy 713 root txt unknown /proc/713/exe (readlink: Permission denied) 2024-05-21T21:59:26.467 INFO:tasks.workunit.client.0.smithi151.stdout:gssproxy 713 root NOFD /proc/713/fd (opendir: Permission denied) 2024-05-21T21:59:26.467 INFO:tasks.workunit.client.0.smithi151.stdout:gssproxy 713 714 gssproxy root cwd unknown /proc/713/task/714/cwd (readlink: Permission denied) 2024-05-21T21:59:26.467 INFO:tasks.workunit.client.0.smithi151.stdout:gssproxy 713 714 gssproxy root rtd unknown /proc/713/task/714/root (readlink: Permission denied) 2024-05-21T21:59:26.467 INFO:tasks.workunit.client.0.smithi151.stdout:gssproxy 713 714 gssproxy root txt unknown /proc/713/task/714/exe (readlink: Permission denied) 2024-05-21T21:59:26.467 INFO:tasks.workunit.client.0.smithi151.stdout:gssproxy 713 714 gssproxy root NOFD /proc/713/task/714/fd (opendir: Permission denied) 2024-05-21T21:59:26.468 INFO:tasks.workunit.client.0.smithi151.stdout:gssproxy 713 715 gssproxy root cwd unknown /proc/713/task/715/cwd (readlink: Permission denied) 2024-05-21T21:59:26.468 INFO:tasks.workunit.client.0.smithi151.stdout:gssproxy 713 715 gssproxy root rtd unknown /proc/713/task/715/root (readlink: Permission denied) 2024-05-21T21:59:26.468 INFO:tasks.workunit.client.0.smithi151.stdout:gssproxy 713 715 gssproxy root txt unknown /proc/713/task/715/exe (readlink: Permission denied) 2024-05-21T21:59:26.468 INFO:tasks.workunit.client.0.smithi151.stdout:gssproxy 713 715 gssproxy root NOFD /proc/713/task/715/fd (opendir: Permission denied) 2024-05-21T21:59:26.468 INFO:tasks.workunit.client.0.smithi151.stdout:gssproxy 713 716 gssproxy root cwd unknown /proc/713/task/716/cwd (readlink: Permission denied) 2024-05-21T21:59:26.468 INFO:tasks.workunit.client.0.smithi151.stdout:gssproxy 713 716 gssproxy root rtd unknown /proc/713/task/716/root (readlink: Permission denied) 2024-05-21T21:59:26.468 INFO:tasks.workunit.client.0.smithi151.stdout:gssproxy 713 716 gssproxy root txt unknown /proc/713/task/716/exe (readlink: Permission denied) 2024-05-21T21:59:26.468 INFO:tasks.workunit.client.0.smithi151.stdout:gssproxy 713 716 gssproxy root NOFD /proc/713/task/716/fd (opendir: Permission denied) 2024-05-21T21:59:26.468 INFO:tasks.workunit.client.0.smithi151.stdout:gssproxy 713 717 gssproxy root cwd unknown /proc/713/task/717/cwd (readlink: Permission denied) 2024-05-21T21:59:26.468 INFO:tasks.workunit.client.0.smithi151.stdout:gssproxy 713 717 gssproxy root rtd unknown /proc/713/task/717/root (readlink: Permission denied) 2024-05-21T21:59:26.468 INFO:tasks.workunit.client.0.smithi151.stdout:gssproxy 713 717 gssproxy root txt unknown /proc/713/task/717/exe (readlink: Permission denied) 2024-05-21T21:59:26.469 INFO:tasks.workunit.client.0.smithi151.stdout:gssproxy 713 717 gssproxy root NOFD /proc/713/task/717/fd (opendir: Permission denied) 2024-05-21T21:59:26.469 INFO:tasks.workunit.client.0.smithi151.stdout:gssproxy 713 718 gssproxy root cwd unknown /proc/713/task/718/cwd (readlink: Permission denied) 2024-05-21T21:59:26.469 INFO:tasks.workunit.client.0.smithi151.stdout:gssproxy 713 718 gssproxy root rtd unknown /proc/713/task/718/root (readlink: Permission denied) 2024-05-21T21:59:26.469 INFO:tasks.workunit.client.0.smithi151.stdout:gssproxy 713 718 gssproxy root txt unknown /proc/713/task/718/exe (readlink: Permission denied) 2024-05-21T21:59:26.469 INFO:tasks.workunit.client.0.smithi151.stdout:gssproxy 713 718 gssproxy root NOFD /proc/713/task/718/fd (opendir: Permission denied) 2024-05-21T21:59:26.469 INFO:tasks.workunit.client.0.smithi151.stdout:atd 720 root cwd unknown /proc/720/cwd (readlink: Permission denied) 2024-05-21T21:59:26.469 INFO:tasks.workunit.client.0.smithi151.stdout:atd 720 root rtd unknown /proc/720/root (readlink: Permission denied) 2024-05-21T21:59:26.469 INFO:tasks.workunit.client.0.smithi151.stdout:atd 720 root txt unknown /proc/720/exe (readlink: Permission denied) 2024-05-21T21:59:26.469 INFO:tasks.workunit.client.0.smithi151.stdout:atd 720 root NOFD /proc/720/fd (opendir: Permission denied) 2024-05-21T21:59:26.469 INFO:tasks.workunit.client.0.smithi151.stdout:crond 721 root cwd unknown /proc/721/cwd (readlink: Permission denied) 2024-05-21T21:59:26.469 INFO:tasks.workunit.client.0.smithi151.stdout:crond 721 root rtd unknown /proc/721/root (readlink: Permission denied) 2024-05-21T21:59:26.469 INFO:tasks.workunit.client.0.smithi151.stdout:crond 721 root txt unknown /proc/721/exe (readlink: Permission denied) 2024-05-21T21:59:26.469 INFO:tasks.workunit.client.0.smithi151.stdout:crond 721 root NOFD /proc/721/fd (opendir: Permission denied) 2024-05-21T21:59:26.469 INFO:tasks.workunit.client.0.smithi151.stdout:nrpe 739 nrpe cwd unknown /proc/739/cwd (readlink: Permission denied) 2024-05-21T21:59:26.470 INFO:tasks.workunit.client.0.smithi151.stdout:nrpe 739 nrpe rtd unknown /proc/739/root (readlink: Permission denied) 2024-05-21T21:59:26.470 INFO:tasks.workunit.client.0.smithi151.stdout:nrpe 739 nrpe txt unknown /proc/739/exe (readlink: Permission denied) 2024-05-21T21:59:26.470 INFO:tasks.workunit.client.0.smithi151.stdout:nrpe 739 nrpe NOFD /proc/739/fd (opendir: Permission denied) 2024-05-21T21:59:26.470 INFO:tasks.workunit.client.0.smithi151.stdout:agetty 1065 root cwd unknown /proc/1065/cwd (readlink: Permission denied) 2024-05-21T21:59:26.470 INFO:tasks.workunit.client.0.smithi151.stdout:agetty 1065 root rtd unknown /proc/1065/root (readlink: Permission denied) 2024-05-21T21:59:26.470 INFO:tasks.workunit.client.0.smithi151.stdout:agetty 1065 root txt unknown /proc/1065/exe (readlink: Permission denied) 2024-05-21T21:59:26.470 INFO:tasks.workunit.client.0.smithi151.stdout:agetty 1065 root NOFD /proc/1065/fd (opendir: Permission denied) 2024-05-21T21:59:26.470 INFO:tasks.workunit.client.0.smithi151.stdout:agetty 1068 root cwd unknown /proc/1068/cwd (readlink: Permission denied) 2024-05-21T21:59:26.470 INFO:tasks.workunit.client.0.smithi151.stdout:agetty 1068 root rtd unknown /proc/1068/root (readlink: Permission denied) 2024-05-21T21:59:26.470 INFO:tasks.workunit.client.0.smithi151.stdout:agetty 1068 root txt unknown /proc/1068/exe (readlink: Permission denied) 2024-05-21T21:59:26.470 INFO:tasks.workunit.client.0.smithi151.stdout:agetty 1068 root NOFD /proc/1068/fd (opendir: Permission denied) 2024-05-21T21:59:26.470 INFO:tasks.workunit.client.0.smithi151.stdout:systemd 4213 ubuntu cwd DIR 8,1 4096 2 / 2024-05-21T21:59:26.470 INFO:tasks.workunit.client.0.smithi151.stdout:systemd 4213 ubuntu rtd DIR 8,1 4096 2 / 2024-05-21T21:59:26.471 INFO:tasks.workunit.client.0.smithi151.stdout:systemd 4213 ubuntu txt REG 8,1 98216 18583 /usr/lib/systemd/systemd 2024-05-21T21:59:26.471 INFO:tasks.workunit.client.0.smithi151.stdout:systemd 4213 ubuntu DEL REG 8,1 849 /etc/selinux/targeted/contexts/files/file_contexts.homedirs.bin 2024-05-21T21:59:26.471 INFO:tasks.workunit.client.0.smithi151.stdout:systemd 4213 ubuntu DEL REG 8,1 25 /etc/selinux/targeted/contexts/files/file_contexts.bin 2024-05-21T21:59:26.471 INFO:tasks.workunit.client.0.smithi151.stdout:systemd 4213 ubuntu mem REG 8,1 44784 7268 /usr/lib64/libffi.so.8.1.0 2024-05-21T21:59:26.471 INFO:tasks.workunit.client.0.smithi151.stdout:systemd 4213 ubuntu mem REG 8,1 153600 7292 /usr/lib64/libgpg-error.so.0.32.0 2024-05-21T21:59:26.471 INFO:tasks.workunit.client.0.smithi151.stdout:systemd 4213 ubuntu mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-05-21T21:59:26.471 INFO:tasks.workunit.client.0.smithi151.stdout:systemd 4213 ubuntu mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-05-21T21:59:26.471 INFO:tasks.workunit.client.0.smithi151.stdout:systemd 4213 ubuntu mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-05-21T21:59:26.471 INFO:tasks.workunit.client.0.smithi151.stdout:systemd 4213 ubuntu mem REG 8,1 178728 7177 /usr/lib64/liblzma.so.5.2.5 2024-05-21T21:59:26.471 INFO:tasks.workunit.client.0.smithi151.stdout:systemd 4213 ubuntu mem REG 8,1 882376 7182 /usr/lib64/libzstd.so.1.5.1 2024-05-21T21:59:26.471 INFO:tasks.workunit.client.0.smithi151.stdout:systemd 4213 ubuntu DEL REG 8,1 7490 /usr/lib64/libcrypto.so.3.0.7 2024-05-21T21:59:26.471 INFO:tasks.workunit.client.0.smithi151.stdout:systemd 4213 ubuntu mem REG 8,1 28552 7287 /usr/lib64/libattr.so.1.1.2501 2024-05-21T21:59:26.471 INFO:tasks.workunit.client.0.smithi151.stdout:systemd 4213 ubuntu mem REG 8,1 32528 7206 /usr/lib64/libcap-ng.so.0.0.0 2024-05-21T21:59:26.471 INFO:tasks.workunit.client.0.smithi151.stdout:systemd 4213 ubuntu mem REG 8,1 1714208 7271 /usr/lib64/libp11-kit.so.0.3.1 2024-05-21T21:59:26.471 INFO:tasks.workunit.client.0.smithi151.stdout:systemd 4213 ubuntu mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-05-21T21:59:26.472 INFO:tasks.workunit.client.0.smithi151.stdout:systemd 4213 ubuntu mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-05-21T21:59:26.472 INFO:tasks.workunit.client.0.smithi151.stdout:systemd 4213 ubuntu mem REG 8,1 1288504 7366 /usr/lib64/libgcrypt.so.20.4.0 2024-05-21T21:59:26.472 INFO:tasks.workunit.client.0.smithi151.stdout:systemd 4213 ubuntu mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-05-21T21:59:26.472 INFO:tasks.workunit.client.0.smithi151.stdout:systemd 4213 ubuntu mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-05-21T21:59:26.472 INFO:tasks.workunit.client.0.smithi151.stdout:systemd 4213 ubuntu mem REG 8,1 291512 7639 /usr/lib64/libmount.so.1.1.0 2024-05-21T21:59:26.472 INFO:tasks.workunit.client.0.smithi151.stdout:systemd 4213 ubuntu mem REG 8,1 3785856 46432 /usr/lib64/systemd/libsystemd-shared-252.so 2024-05-21T21:59:26.472 INFO:tasks.workunit.client.0.smithi151.stdout:systemd 4213 ubuntu mem REG 8,1 40424 8439 /usr/lib64/libeconf.so.0.4.1 2024-05-21T21:59:26.472 INFO:tasks.workunit.client.0.smithi151.stdout:systemd 4213 ubuntu mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-05-21T21:59:26.472 INFO:tasks.workunit.client.0.smithi151.stdout:systemd 4213 ubuntu mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-05-21T21:59:26.472 INFO:tasks.workunit.client.0.smithi151.stdout:systemd 4213 ubuntu mem REG 8,1 138848 7210 /usr/lib64/libaudit.so.1.0.0 2024-05-21T21:59:26.473 INFO:tasks.workunit.client.0.smithi151.stdout:systemd 4213 ubuntu mem REG 8,1 2157904 46429 /usr/lib64/systemd/libsystemd-core-252.so 2024-05-21T21:59:26.473 INFO:tasks.workunit.client.0.smithi151.stdout:systemd 4213 ubuntu mem REG 8,1 40496 7288 /usr/lib64/libacl.so.1.1.2301 2024-05-21T21:59:26.473 INFO:tasks.workunit.client.0.smithi151.stdout:systemd 4213 ubuntu mem REG 8,1 107424 7663 /usr/lib64/libkmod.so.2.3.6 2024-05-21T21:59:26.473 INFO:tasks.workunit.client.0.smithi151.stdout:systemd 4213 ubuntu mem REG 8,1 69592 8448 /usr/lib64/libpam.so.0.85.1 2024-05-21T21:59:26.473 INFO:tasks.workunit.client.0.smithi151.stdout:systemd 4213 ubuntu mem REG 8,1 127680 7455 /usr/lib64/libseccomp.so.2.5.2 2024-05-21T21:59:26.473 INFO:tasks.workunit.client.0.smithi151.stdout:systemd 4213 ubuntu mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-05-21T21:59:26.473 INFO:tasks.workunit.client.0.smithi151.stdout:systemd 4213 ubuntu mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-05-21T21:59:26.473 INFO:tasks.workunit.client.0.smithi151.stdout:systemd 4213 ubuntu mem REG 0,20 0 19 /sys/fs/selinux/status 2024-05-21T21:59:26.473 INFO:tasks.workunit.client.0.smithi151.stdout:systemd 4213 ubuntu 0r CHR 1,3 0t0 4 /dev/null 2024-05-21T21:59:26.473 INFO:tasks.workunit.client.0.smithi151.stdout:systemd 4213 ubuntu 1u unix 0x0000000000000000 0t0 9912 type=STREAM (CONNECTED) 2024-05-21T21:59:26.473 INFO:tasks.workunit.client.0.smithi151.stdout:systemd 4213 ubuntu 2u unix 0x0000000000000000 0t0 9912 type=STREAM (CONNECTED) 2024-05-21T21:59:26.473 INFO:tasks.workunit.client.0.smithi151.stdout:systemd 4213 ubuntu 3u unix 0x0000000000000000 0t0 9940 type=DGRAM (CONNECTED) 2024-05-21T21:59:26.473 INFO:tasks.workunit.client.0.smithi151.stdout:systemd 4213 ubuntu 4u a_inode 0,15 0 50 [eventpoll:5,6,8,9,11,12,14,15,16,17,19,20,21] 2024-05-21T21:59:26.474 INFO:tasks.workunit.client.0.smithi151.stdout:systemd 4213 ubuntu 5u a_inode 0,15 0 50 [signalfd] 2024-05-21T21:59:26.474 INFO:tasks.workunit.client.0.smithi151.stdout:systemd 4213 ubuntu 6r a_inode 0,15 0 50 inotify 2024-05-21T21:59:26.474 INFO:tasks.workunit.client.0.smithi151.stdout:systemd 4213 ubuntu 7r DIR 0,26 0 3581 /sys/fs/cgroup/user.slice/user-1000.slice/user@1000.service 2024-05-21T21:59:26.474 INFO:tasks.workunit.client.0.smithi151.stdout:systemd 4213 ubuntu 8u a_inode 0,15 0 50 [timerfd] 2024-05-21T21:59:26.474 INFO:tasks.workunit.client.0.smithi151.stdout:systemd 4213 ubuntu 9u a_inode 0,15 0 50 [eventpoll:10,13] 2024-05-21T21:59:26.474 INFO:tasks.workunit.client.0.smithi151.stdout:systemd 4213 ubuntu 10r REG 0,21 0 9929 /proc/4213/mountinfo 2024-05-21T21:59:26.474 INFO:tasks.workunit.client.0.smithi151.stdout:systemd 4213 ubuntu 11r a_inode 0,15 0 50 inotify 2024-05-21T21:59:26.474 INFO:tasks.workunit.client.0.smithi151.stdout:systemd 4213 ubuntu 12u unix 0x0000000000000000 0t0 9962 /run/user/1000/bus type=STREAM (LISTEN) 2024-05-21T21:59:26.474 INFO:tasks.workunit.client.0.smithi151.stdout:systemd 4213 ubuntu 13r a_inode 0,15 0 50 inotify 2024-05-21T21:59:26.474 INFO:tasks.workunit.client.0.smithi151.stdout:systemd 4213 ubuntu 14r REG 0,21 0 4026532078 /proc/swaps 2024-05-21T21:59:26.474 INFO:tasks.workunit.client.0.smithi151.stdout:systemd 4213 ubuntu 15u netlink 0t0 9944 KOBJECT_UEVENT 2024-05-21T21:59:26.474 INFO:tasks.workunit.client.0.smithi151.stdout:systemd 4213 ubuntu 16u unix 0x0000000000000000 0t0 9949 /run/user/1000/systemd/notify type=DGRAM (UNCONNECTED) 2024-05-21T21:59:26.474 INFO:tasks.workunit.client.0.smithi151.stdout:systemd 4213 ubuntu 17u unix 0x0000000000000000 0t0 9950 type=DGRAM (CONNECTED) 2024-05-21T21:59:26.474 INFO:tasks.workunit.client.0.smithi151.stdout:systemd 4213 ubuntu 18u unix 0x0000000000000000 0t0 9951 type=DGRAM (CONNECTED) 2024-05-21T21:59:26.475 INFO:tasks.workunit.client.0.smithi151.stdout:systemd 4213 ubuntu 19u unix 0x0000000000000000 0t0 9952 /run/user/1000/systemd/private type=STREAM (LISTEN) 2024-05-21T21:59:26.475 INFO:tasks.workunit.client.0.smithi151.stdout:systemd 4213 ubuntu 20u unix 0x0000000000000000 0t0 9954 type=STREAM (CONNECTED) 2024-05-21T21:59:26.475 INFO:tasks.workunit.client.0.smithi151.stdout:systemd 4213 ubuntu 21u a_inode 0,15 0 50 [timerfd] 2024-05-21T21:59:26.475 INFO:tasks.workunit.client.0.smithi151.stdout:(sd-pam) 4216 ubuntu cwd unknown /proc/4216/cwd (readlink: Permission denied) 2024-05-21T21:59:26.475 INFO:tasks.workunit.client.0.smithi151.stdout:(sd-pam) 4216 ubuntu rtd unknown /proc/4216/root (readlink: Permission denied) 2024-05-21T21:59:26.475 INFO:tasks.workunit.client.0.smithi151.stdout:(sd-pam) 4216 ubuntu txt unknown /proc/4216/exe (readlink: Permission denied) 2024-05-21T21:59:26.475 INFO:tasks.workunit.client.0.smithi151.stdout:(sd-pam) 4216 ubuntu NOFD /proc/4216/fd (opendir: Permission denied) 2024-05-21T21:59:26.475 INFO:tasks.workunit.client.0.smithi151.stdout:rsyslogd 4459 root cwd unknown /proc/4459/cwd (readlink: Permission denied) 2024-05-21T21:59:26.475 INFO:tasks.workunit.client.0.smithi151.stdout:rsyslogd 4459 root rtd unknown /proc/4459/root (readlink: Permission denied) 2024-05-21T21:59:26.475 INFO:tasks.workunit.client.0.smithi151.stdout:rsyslogd 4459 root txt unknown /proc/4459/exe (readlink: Permission denied) 2024-05-21T21:59:26.475 INFO:tasks.workunit.client.0.smithi151.stdout:rsyslogd 4459 root NOFD /proc/4459/fd (opendir: Permission denied) 2024-05-21T21:59:26.475 INFO:tasks.workunit.client.0.smithi151.stdout:rsyslogd 4459 4461 in:imjour root cwd unknown /proc/4459/task/4461/cwd (readlink: Permission denied) 2024-05-21T21:59:26.475 INFO:tasks.workunit.client.0.smithi151.stdout:rsyslogd 4459 4461 in:imjour root rtd unknown /proc/4459/task/4461/root (readlink: Permission denied) 2024-05-21T21:59:26.475 INFO:tasks.workunit.client.0.smithi151.stdout:rsyslogd 4459 4461 in:imjour root txt unknown /proc/4459/task/4461/exe (readlink: Permission denied) 2024-05-21T21:59:26.476 INFO:tasks.workunit.client.0.smithi151.stdout:rsyslogd 4459 4461 in:imjour root NOFD /proc/4459/task/4461/fd (opendir: Permission denied) 2024-05-21T21:59:26.476 INFO:tasks.workunit.client.0.smithi151.stdout:rsyslogd 4459 4462 rs:main root cwd unknown /proc/4459/task/4462/cwd (readlink: Permission denied) 2024-05-21T21:59:26.476 INFO:tasks.workunit.client.0.smithi151.stdout:rsyslogd 4459 4462 rs:main root rtd unknown /proc/4459/task/4462/root (readlink: Permission denied) 2024-05-21T21:59:26.476 INFO:tasks.workunit.client.0.smithi151.stdout:rsyslogd 4459 4462 rs:main root txt unknown /proc/4459/task/4462/exe (readlink: Permission denied) 2024-05-21T21:59:26.476 INFO:tasks.workunit.client.0.smithi151.stdout:rsyslogd 4459 4462 rs:main root NOFD /proc/4459/task/4462/fd (opendir: Permission denied) 2024-05-21T21:59:26.476 INFO:tasks.workunit.client.0.smithi151.stdout:polkitd 16316 polkitd cwd unknown /proc/16316/cwd (readlink: Permission denied) 2024-05-21T21:59:26.476 INFO:tasks.workunit.client.0.smithi151.stdout:polkitd 16316 polkitd rtd unknown /proc/16316/root (readlink: Permission denied) 2024-05-21T21:59:26.476 INFO:tasks.workunit.client.0.smithi151.stdout:polkitd 16316 polkitd txt unknown /proc/16316/exe (readlink: Permission denied) 2024-05-21T21:59:26.476 INFO:tasks.workunit.client.0.smithi151.stdout:polkitd 16316 polkitd NOFD /proc/16316/fd (opendir: Permission denied) 2024-05-21T21:59:26.476 INFO:tasks.workunit.client.0.smithi151.stdout:polkitd 16316 16318 gmain polkitd cwd unknown /proc/16316/task/16318/cwd (readlink: Permission denied) 2024-05-21T21:59:26.476 INFO:tasks.workunit.client.0.smithi151.stdout:polkitd 16316 16318 gmain polkitd rtd unknown /proc/16316/task/16318/root (readlink: Permission denied) 2024-05-21T21:59:26.476 INFO:tasks.workunit.client.0.smithi151.stdout:polkitd 16316 16318 gmain polkitd txt unknown /proc/16316/task/16318/exe (readlink: Permission denied) 2024-05-21T21:59:26.476 INFO:tasks.workunit.client.0.smithi151.stdout:polkitd 16316 16318 gmain polkitd NOFD /proc/16316/task/16318/fd (opendir: Permission denied) 2024-05-21T21:59:26.477 INFO:tasks.workunit.client.0.smithi151.stdout:polkitd 16316 16319 gdbus polkitd cwd unknown /proc/16316/task/16319/cwd (readlink: Permission denied) 2024-05-21T21:59:26.477 INFO:tasks.workunit.client.0.smithi151.stdout:polkitd 16316 16319 gdbus polkitd rtd unknown /proc/16316/task/16319/root (readlink: Permission denied) 2024-05-21T21:59:26.477 INFO:tasks.workunit.client.0.smithi151.stdout:polkitd 16316 16319 gdbus polkitd txt unknown /proc/16316/task/16319/exe (readlink: Permission denied) 2024-05-21T21:59:26.477 INFO:tasks.workunit.client.0.smithi151.stdout:polkitd 16316 16319 gdbus polkitd NOFD /proc/16316/task/16319/fd (opendir: Permission denied) 2024-05-21T21:59:26.477 INFO:tasks.workunit.client.0.smithi151.stdout:polkitd 16316 16320 JS\x20Hel polkitd cwd unknown /proc/16316/task/16320/cwd (readlink: Permission denied) 2024-05-21T21:59:26.477 INFO:tasks.workunit.client.0.smithi151.stdout:polkitd 16316 16320 JS\x20Hel polkitd rtd unknown /proc/16316/task/16320/root (readlink: Permission denied) 2024-05-21T21:59:26.477 INFO:tasks.workunit.client.0.smithi151.stdout:polkitd 16316 16320 JS\x20Hel polkitd txt unknown /proc/16316/task/16320/exe (readlink: Permission denied) 2024-05-21T21:59:26.477 INFO:tasks.workunit.client.0.smithi151.stdout:polkitd 16316 16320 JS\x20Hel polkitd NOFD /proc/16316/task/16320/fd (opendir: Permission denied) 2024-05-21T21:59:26.477 INFO:tasks.workunit.client.0.smithi151.stdout:polkitd 16316 16321 JS\x20Hel polkitd cwd unknown /proc/16316/task/16321/cwd (readlink: Permission denied) 2024-05-21T21:59:26.477 INFO:tasks.workunit.client.0.smithi151.stdout:polkitd 16316 16321 JS\x20Hel polkitd rtd unknown /proc/16316/task/16321/root (readlink: Permission denied) 2024-05-21T21:59:26.477 INFO:tasks.workunit.client.0.smithi151.stdout:polkitd 16316 16321 JS\x20Hel polkitd txt unknown /proc/16316/task/16321/exe (readlink: Permission denied) 2024-05-21T21:59:26.477 INFO:tasks.workunit.client.0.smithi151.stdout:polkitd 16316 16321 JS\x20Hel polkitd NOFD /proc/16316/task/16321/fd (opendir: Permission denied) 2024-05-21T21:59:26.477 INFO:tasks.workunit.client.0.smithi151.stdout:polkitd 16316 16322 JS\x20Hel polkitd cwd unknown /proc/16316/task/16322/cwd (readlink: Permission denied) 2024-05-21T21:59:26.477 INFO:tasks.workunit.client.0.smithi151.stdout:polkitd 16316 16322 JS\x20Hel polkitd rtd unknown /proc/16316/task/16322/root (readlink: Permission denied) 2024-05-21T21:59:26.478 INFO:tasks.workunit.client.0.smithi151.stdout:polkitd 16316 16322 JS\x20Hel polkitd txt unknown /proc/16316/task/16322/exe (readlink: Permission denied) 2024-05-21T21:59:26.478 INFO:tasks.workunit.client.0.smithi151.stdout:polkitd 16316 16322 JS\x20Hel polkitd NOFD /proc/16316/task/16322/fd (opendir: Permission denied) 2024-05-21T21:59:26.478 INFO:tasks.workunit.client.0.smithi151.stdout:polkitd 16316 16323 JS\x20Hel polkitd cwd unknown /proc/16316/task/16323/cwd (readlink: Permission denied) 2024-05-21T21:59:26.478 INFO:tasks.workunit.client.0.smithi151.stdout:polkitd 16316 16323 JS\x20Hel polkitd rtd unknown /proc/16316/task/16323/root (readlink: Permission denied) 2024-05-21T21:59:26.478 INFO:tasks.workunit.client.0.smithi151.stdout:polkitd 16316 16323 JS\x20Hel polkitd txt unknown /proc/16316/task/16323/exe (readlink: Permission denied) 2024-05-21T21:59:26.478 INFO:tasks.workunit.client.0.smithi151.stdout:polkitd 16316 16323 JS\x20Hel polkitd NOFD /proc/16316/task/16323/fd (opendir: Permission denied) 2024-05-21T21:59:26.478 INFO:tasks.workunit.client.0.smithi151.stdout:polkitd 16316 16324 JS\x20Hel polkitd cwd unknown /proc/16316/task/16324/cwd (readlink: Permission denied) 2024-05-21T21:59:26.478 INFO:tasks.workunit.client.0.smithi151.stdout:polkitd 16316 16324 JS\x20Hel polkitd rtd unknown /proc/16316/task/16324/root (readlink: Permission denied) 2024-05-21T21:59:26.478 INFO:tasks.workunit.client.0.smithi151.stdout:polkitd 16316 16324 JS\x20Hel polkitd txt unknown /proc/16316/task/16324/exe (readlink: Permission denied) 2024-05-21T21:59:26.478 INFO:tasks.workunit.client.0.smithi151.stdout:polkitd 16316 16324 JS\x20Hel polkitd NOFD /proc/16316/task/16324/fd (opendir: Permission denied) 2024-05-21T21:59:26.478 INFO:tasks.workunit.client.0.smithi151.stdout:polkitd 16316 16325 JS\x20Hel polkitd cwd unknown /proc/16316/task/16325/cwd (readlink: Permission denied) 2024-05-21T21:59:26.478 INFO:tasks.workunit.client.0.smithi151.stdout:polkitd 16316 16325 JS\x20Hel polkitd rtd unknown /proc/16316/task/16325/root (readlink: Permission denied) 2024-05-21T21:59:26.479 INFO:tasks.workunit.client.0.smithi151.stdout:polkitd 16316 16325 JS\x20Hel polkitd txt unknown /proc/16316/task/16325/exe (readlink: Permission denied) 2024-05-21T21:59:26.479 INFO:tasks.workunit.client.0.smithi151.stdout:polkitd 16316 16325 JS\x20Hel polkitd NOFD /proc/16316/task/16325/fd (opendir: Permission denied) 2024-05-21T21:59:26.479 INFO:tasks.workunit.client.0.smithi151.stdout:polkitd 16316 16326 JS\x20Hel polkitd cwd unknown /proc/16316/task/16326/cwd (readlink: Permission denied) 2024-05-21T21:59:26.479 INFO:tasks.workunit.client.0.smithi151.stdout:polkitd 16316 16326 JS\x20Hel polkitd rtd unknown /proc/16316/task/16326/root (readlink: Permission denied) 2024-05-21T21:59:26.479 INFO:tasks.workunit.client.0.smithi151.stdout:polkitd 16316 16326 JS\x20Hel polkitd txt unknown /proc/16316/task/16326/exe (readlink: Permission denied) 2024-05-21T21:59:26.479 INFO:tasks.workunit.client.0.smithi151.stdout:polkitd 16316 16326 JS\x20Hel polkitd NOFD /proc/16316/task/16326/fd (opendir: Permission denied) 2024-05-21T21:59:26.479 INFO:tasks.workunit.client.0.smithi151.stdout:polkitd 16316 16327 JS\x20Hel polkitd cwd unknown /proc/16316/task/16327/cwd (readlink: Permission denied) 2024-05-21T21:59:26.479 INFO:tasks.workunit.client.0.smithi151.stdout:polkitd 16316 16327 JS\x20Hel polkitd rtd unknown /proc/16316/task/16327/root (readlink: Permission denied) 2024-05-21T21:59:26.479 INFO:tasks.workunit.client.0.smithi151.stdout:polkitd 16316 16327 JS\x20Hel polkitd txt unknown /proc/16316/task/16327/exe (readlink: Permission denied) 2024-05-21T21:59:26.479 INFO:tasks.workunit.client.0.smithi151.stdout:polkitd 16316 16327 JS\x20Hel polkitd NOFD /proc/16316/task/16327/fd (opendir: Permission denied) 2024-05-21T21:59:26.479 INFO:tasks.workunit.client.0.smithi151.stdout:polkitd 16316 16328 polkitd polkitd cwd unknown /proc/16316/task/16328/cwd (readlink: Permission denied) 2024-05-21T21:59:26.479 INFO:tasks.workunit.client.0.smithi151.stdout:polkitd 16316 16328 polkitd polkitd rtd unknown /proc/16316/task/16328/root (readlink: Permission denied) 2024-05-21T21:59:26.479 INFO:tasks.workunit.client.0.smithi151.stdout:polkitd 16316 16328 polkitd polkitd txt unknown /proc/16316/task/16328/exe (readlink: Permission denied) 2024-05-21T21:59:26.479 INFO:tasks.workunit.client.0.smithi151.stdout:polkitd 16316 16328 polkitd polkitd NOFD /proc/16316/task/16328/fd (opendir: Permission denied) 2024-05-21T21:59:26.480 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/7 19778 root cwd unknown /proc/19778/cwd (readlink: Permission denied) 2024-05-21T21:59:26.480 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/7 19778 root rtd unknown /proc/19778/root (readlink: Permission denied) 2024-05-21T21:59:26.480 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/7 19778 root txt unknown /proc/19778/exe (readlink: Permission denied) 2024-05-21T21:59:26.480 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/7 19778 root NOFD /proc/19778/fd (opendir: Permission denied) 2024-05-21T21:59:26.480 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 20185 root cwd unknown /proc/20185/cwd (readlink: Permission denied) 2024-05-21T21:59:26.480 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 20185 root rtd unknown /proc/20185/root (readlink: Permission denied) 2024-05-21T21:59:26.480 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 20185 root txt unknown /proc/20185/exe (readlink: Permission denied) 2024-05-21T21:59:26.480 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 20185 root NOFD /proc/20185/fd (opendir: Permission denied) 2024-05-21T21:59:26.480 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 20215 root cwd unknown /proc/20215/cwd (readlink: Permission denied) 2024-05-21T21:59:26.480 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 20215 root rtd unknown /proc/20215/root (readlink: Permission denied) 2024-05-21T21:59:26.480 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 20215 root txt unknown /proc/20215/exe (readlink: Permission denied) 2024-05-21T21:59:26.480 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 20215 root NOFD /proc/20215/fd (opendir: Permission denied) 2024-05-21T21:59:26.480 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 20244 root cwd unknown /proc/20244/cwd (readlink: Permission denied) 2024-05-21T21:59:26.480 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 20244 root rtd unknown /proc/20244/root (readlink: Permission denied) 2024-05-21T21:59:26.481 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 20244 root txt unknown /proc/20244/exe (readlink: Permission denied) 2024-05-21T21:59:26.481 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 20244 root NOFD /proc/20244/fd (opendir: Permission denied) 2024-05-21T21:59:26.481 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 20273 root cwd unknown /proc/20273/cwd (readlink: Permission denied) 2024-05-21T21:59:26.481 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 20273 root rtd unknown /proc/20273/root (readlink: Permission denied) 2024-05-21T21:59:26.481 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 20273 root txt unknown /proc/20273/exe (readlink: Permission denied) 2024-05-21T21:59:26.481 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 20273 root NOFD /proc/20273/fd (opendir: Permission denied) 2024-05-21T21:59:26.481 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/0 20279 root cwd unknown /proc/20279/cwd (readlink: Permission denied) 2024-05-21T21:59:26.481 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/0 20279 root rtd unknown /proc/20279/root (readlink: Permission denied) 2024-05-21T21:59:26.481 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/0 20279 root txt unknown /proc/20279/exe (readlink: Permission denied) 2024-05-21T21:59:26.481 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/0 20279 root NOFD /proc/20279/fd (opendir: Permission denied) 2024-05-21T21:59:26.481 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 20303 root cwd unknown /proc/20303/cwd (readlink: Permission denied) 2024-05-21T21:59:26.481 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 20303 root rtd unknown /proc/20303/root (readlink: Permission denied) 2024-05-21T21:59:26.481 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 20303 root txt unknown /proc/20303/exe (readlink: Permission denied) 2024-05-21T21:59:26.481 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 20303 root NOFD /proc/20303/fd (opendir: Permission denied) 2024-05-21T21:59:26.481 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 20467 root cwd unknown /proc/20467/cwd (readlink: Permission denied) 2024-05-21T21:59:26.482 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 20467 root rtd unknown /proc/20467/root (readlink: Permission denied) 2024-05-21T21:59:26.482 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 20467 root txt unknown /proc/20467/exe (readlink: Permission denied) 2024-05-21T21:59:26.482 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 20467 root NOFD /proc/20467/fd (opendir: Permission denied) 2024-05-21T21:59:26.482 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 20468 root cwd unknown /proc/20468/cwd (readlink: Permission denied) 2024-05-21T21:59:26.482 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 20468 root rtd unknown /proc/20468/root (readlink: Permission denied) 2024-05-21T21:59:26.482 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 20468 root txt unknown /proc/20468/exe (readlink: Permission denied) 2024-05-21T21:59:26.482 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 20468 root NOFD /proc/20468/fd (opendir: Permission denied) 2024-05-21T21:59:26.482 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 20469 root cwd unknown /proc/20469/cwd (readlink: Permission denied) 2024-05-21T21:59:26.482 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 20469 root rtd unknown /proc/20469/root (readlink: Permission denied) 2024-05-21T21:59:26.482 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 20469 root txt unknown /proc/20469/exe (readlink: Permission denied) 2024-05-21T21:59:26.482 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 20469 root NOFD /proc/20469/fd (opendir: Permission denied) 2024-05-21T21:59:26.482 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 20470 root cwd unknown /proc/20470/cwd (readlink: Permission denied) 2024-05-21T21:59:26.483 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 20470 root rtd unknown /proc/20470/root (readlink: Permission denied) 2024-05-21T21:59:26.483 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 20470 root txt unknown /proc/20470/exe (readlink: Permission denied) 2024-05-21T21:59:26.483 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 20470 root NOFD /proc/20470/fd (opendir: Permission denied) 2024-05-21T21:59:26.483 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 20471 root cwd unknown /proc/20471/cwd (readlink: Permission denied) 2024-05-21T21:59:26.483 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 20471 root rtd unknown /proc/20471/root (readlink: Permission denied) 2024-05-21T21:59:26.483 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 20471 root txt unknown /proc/20471/exe (readlink: Permission denied) 2024-05-21T21:59:26.483 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 20471 root NOFD /proc/20471/fd (opendir: Permission denied) 2024-05-21T21:59:26.483 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 20472 root cwd unknown /proc/20472/cwd (readlink: Permission denied) 2024-05-21T21:59:26.483 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 20472 root rtd unknown /proc/20472/root (readlink: Permission denied) 2024-05-21T21:59:26.483 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 20472 root txt unknown /proc/20472/exe (readlink: Permission denied) 2024-05-21T21:59:26.483 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 20472 root NOFD /proc/20472/fd (opendir: Permission denied) 2024-05-21T21:59:26.483 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 20473 root cwd unknown /proc/20473/cwd (readlink: Permission denied) 2024-05-21T21:59:26.483 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 20473 root rtd unknown /proc/20473/root (readlink: Permission denied) 2024-05-21T21:59:26.484 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 20473 root txt unknown /proc/20473/exe (readlink: Permission denied) 2024-05-21T21:59:26.484 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 20473 root NOFD /proc/20473/fd (opendir: Permission denied) 2024-05-21T21:59:26.484 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 20474 root cwd unknown /proc/20474/cwd (readlink: Permission denied) 2024-05-21T21:59:26.484 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 20474 root rtd unknown /proc/20474/root (readlink: Permission denied) 2024-05-21T21:59:26.484 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 20474 root txt unknown /proc/20474/exe (readlink: Permission denied) 2024-05-21T21:59:26.484 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 20474 root NOFD /proc/20474/fd (opendir: Permission denied) 2024-05-21T21:59:26.484 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 20475 root cwd unknown /proc/20475/cwd (readlink: Permission denied) 2024-05-21T21:59:26.484 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 20475 root rtd unknown /proc/20475/root (readlink: Permission denied) 2024-05-21T21:59:26.484 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 20475 root txt unknown /proc/20475/exe (readlink: Permission denied) 2024-05-21T21:59:26.484 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 20475 root NOFD /proc/20475/fd (opendir: Permission denied) 2024-05-21T21:59:26.484 INFO:tasks.workunit.client.0.smithi151.stdout:xfsaild/d 20476 root cwd unknown /proc/20476/cwd (readlink: Permission denied) 2024-05-21T21:59:26.484 INFO:tasks.workunit.client.0.smithi151.stdout:xfsaild/d 20476 root rtd unknown /proc/20476/root (readlink: Permission denied) 2024-05-21T21:59:26.484 INFO:tasks.workunit.client.0.smithi151.stdout:xfsaild/d 20476 root txt unknown /proc/20476/exe (readlink: Permission denied) 2024-05-21T21:59:26.484 INFO:tasks.workunit.client.0.smithi151.stdout:xfsaild/d 20476 root NOFD /proc/20476/fd (opendir: Permission denied) 2024-05-21T21:59:26.485 INFO:tasks.workunit.client.0.smithi151.stdout:sshd 22841 root cwd unknown /proc/22841/cwd (readlink: Permission denied) 2024-05-21T21:59:26.485 INFO:tasks.workunit.client.0.smithi151.stdout:sshd 22841 root rtd unknown /proc/22841/root (readlink: Permission denied) 2024-05-21T21:59:26.485 INFO:tasks.workunit.client.0.smithi151.stdout:sshd 22841 root txt unknown /proc/22841/exe (readlink: Permission denied) 2024-05-21T21:59:26.485 INFO:tasks.workunit.client.0.smithi151.stdout:sshd 22841 root NOFD /proc/22841/fd (opendir: Permission denied) 2024-05-21T21:59:26.485 INFO:tasks.workunit.client.0.smithi151.stdout:sshd 22844 ubuntu cwd unknown /proc/22844/cwd (readlink: Permission denied) 2024-05-21T21:59:26.485 INFO:tasks.workunit.client.0.smithi151.stdout:sshd 22844 ubuntu rtd unknown /proc/22844/root (readlink: Permission denied) 2024-05-21T21:59:26.485 INFO:tasks.workunit.client.0.smithi151.stdout:sshd 22844 ubuntu txt unknown /proc/22844/exe (readlink: Permission denied) 2024-05-21T21:59:26.485 INFO:tasks.workunit.client.0.smithi151.stdout:sshd 22844 ubuntu NOFD /proc/22844/fd (opendir: Permission denied) 2024-05-21T21:59:26.485 INFO:tasks.workunit.client.0.smithi151.stdout:chronyd 22905 chrony cwd unknown /proc/22905/cwd (readlink: Permission denied) 2024-05-21T21:59:26.485 INFO:tasks.workunit.client.0.smithi151.stdout:chronyd 22905 chrony rtd unknown /proc/22905/root (readlink: Permission denied) 2024-05-21T21:59:26.485 INFO:tasks.workunit.client.0.smithi151.stdout:chronyd 22905 chrony txt unknown /proc/22905/exe (readlink: Permission denied) 2024-05-21T21:59:26.485 INFO:tasks.workunit.client.0.smithi151.stdout:chronyd 22905 chrony NOFD /proc/22905/fd (opendir: Permission denied) 2024-05-21T21:59:26.485 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-cras 23242 ceph cwd unknown /proc/23242/cwd (readlink: Permission denied) 2024-05-21T21:59:26.486 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-cras 23242 ceph rtd unknown /proc/23242/root (readlink: Permission denied) 2024-05-21T21:59:26.486 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-cras 23242 ceph txt unknown /proc/23242/exe (readlink: Permission denied) 2024-05-21T21:59:26.486 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-cras 23242 ceph NOFD /proc/23242/fd (opendir: Permission denied) 2024-05-21T21:59:26.486 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/3 24481 root cwd unknown /proc/24481/cwd (readlink: Permission denied) 2024-05-21T21:59:26.486 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/3 24481 root rtd unknown /proc/24481/root (readlink: Permission denied) 2024-05-21T21:59:26.486 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/3 24481 root txt unknown /proc/24481/exe (readlink: Permission denied) 2024-05-21T21:59:26.486 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/3 24481 root NOFD /proc/24481/fd (opendir: Permission denied) 2024-05-21T21:59:26.486 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/3 24482 root cwd unknown /proc/24482/cwd (readlink: Permission denied) 2024-05-21T21:59:26.486 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/3 24482 root rtd unknown /proc/24482/root (readlink: Permission denied) 2024-05-21T21:59:26.486 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/3 24482 root txt unknown /proc/24482/exe (readlink: Permission denied) 2024-05-21T21:59:26.486 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/3 24482 root NOFD /proc/24482/fd (opendir: Permission denied) 2024-05-21T21:59:26.486 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/5 24651 root cwd unknown /proc/24651/cwd (readlink: Permission denied) 2024-05-21T21:59:26.486 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/5 24651 root rtd unknown /proc/24651/root (readlink: Permission denied) 2024-05-21T21:59:26.486 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/5 24651 root txt unknown /proc/24651/exe (readlink: Permission denied) 2024-05-21T21:59:26.487 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/5 24651 root NOFD /proc/24651/fd (opendir: Permission denied) 2024-05-21T21:59:26.487 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/6 30469 root cwd unknown /proc/30469/cwd (readlink: Permission denied) 2024-05-21T21:59:26.487 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/6 30469 root rtd unknown /proc/30469/root (readlink: Permission denied) 2024-05-21T21:59:26.487 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/6 30469 root txt unknown /proc/30469/exe (readlink: Permission denied) 2024-05-21T21:59:26.487 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/6 30469 root NOFD /proc/30469/fd (opendir: Permission denied) 2024-05-21T21:59:26.487 INFO:tasks.workunit.client.0.smithi151.stdout:sudo 30620 root cwd unknown /proc/30620/cwd (readlink: Permission denied) 2024-05-21T21:59:26.487 INFO:tasks.workunit.client.0.smithi151.stdout:sudo 30620 root rtd unknown /proc/30620/root (readlink: Permission denied) 2024-05-21T21:59:26.487 INFO:tasks.workunit.client.0.smithi151.stdout:sudo 30620 root txt unknown /proc/30620/exe (readlink: Permission denied) 2024-05-21T21:59:26.487 INFO:tasks.workunit.client.0.smithi151.stdout:sudo 30620 root NOFD /proc/30620/fd (opendir: Permission denied) 2024-05-21T21:59:26.487 INFO:tasks.workunit.client.0.smithi151.stdout:sudo 30636 root cwd unknown /proc/30636/cwd (readlink: Permission denied) 2024-05-21T21:59:26.487 INFO:tasks.workunit.client.0.smithi151.stdout:sudo 30636 root rtd unknown /proc/30636/root (readlink: Permission denied) 2024-05-21T21:59:26.487 INFO:tasks.workunit.client.0.smithi151.stdout:sudo 30636 root txt unknown /proc/30636/exe (readlink: Permission denied) 2024-05-21T21:59:26.487 INFO:tasks.workunit.client.0.smithi151.stdout:sudo 30636 root NOFD /proc/30636/fd (opendir: Permission denied) 2024-05-21T21:59:26.487 INFO:tasks.workunit.client.0.smithi151.stdout:journalct 30648 root cwd unknown /proc/30648/cwd (readlink: Permission denied) 2024-05-21T21:59:26.488 INFO:tasks.workunit.client.0.smithi151.stdout:journalct 30648 root rtd unknown /proc/30648/root (readlink: Permission denied) 2024-05-21T21:59:26.488 INFO:tasks.workunit.client.0.smithi151.stdout:journalct 30648 root txt unknown /proc/30648/exe (readlink: Permission denied) 2024-05-21T21:59:26.488 INFO:tasks.workunit.client.0.smithi151.stdout:journalct 30648 root NOFD /proc/30648/fd (opendir: Permission denied) 2024-05-21T21:59:26.488 INFO:tasks.workunit.client.0.smithi151.stdout:journalct 30661 root cwd unknown /proc/30661/cwd (readlink: Permission denied) 2024-05-21T21:59:26.488 INFO:tasks.workunit.client.0.smithi151.stdout:journalct 30661 root rtd unknown /proc/30661/root (readlink: Permission denied) 2024-05-21T21:59:26.488 INFO:tasks.workunit.client.0.smithi151.stdout:journalct 30661 root txt unknown /proc/30661/exe (readlink: Permission denied) 2024-05-21T21:59:26.488 INFO:tasks.workunit.client.0.smithi151.stdout:journalct 30661 root NOFD /proc/30661/fd (opendir: Permission denied) 2024-05-21T21:59:26.488 INFO:tasks.workunit.client.0.smithi151.stdout:conmon 31619 root cwd unknown /proc/31619/cwd (readlink: Permission denied) 2024-05-21T21:59:26.488 INFO:tasks.workunit.client.0.smithi151.stdout:conmon 31619 root rtd unknown /proc/31619/root (readlink: Permission denied) 2024-05-21T21:59:26.488 INFO:tasks.workunit.client.0.smithi151.stdout:conmon 31619 root txt unknown /proc/31619/exe (readlink: Permission denied) 2024-05-21T21:59:26.488 INFO:tasks.workunit.client.0.smithi151.stdout:conmon 31619 root NOFD /proc/31619/fd (opendir: Permission denied) 2024-05-21T21:59:26.488 INFO:tasks.workunit.client.0.smithi151.stdout:podman-in 31621 root cwd unknown /proc/31621/cwd (readlink: Permission denied) 2024-05-21T21:59:26.488 INFO:tasks.workunit.client.0.smithi151.stdout:podman-in 31621 root rtd unknown /proc/31621/root (readlink: Permission denied) 2024-05-21T21:59:26.489 INFO:tasks.workunit.client.0.smithi151.stdout:podman-in 31621 root txt unknown /proc/31621/exe (readlink: Permission denied) 2024-05-21T21:59:26.489 INFO:tasks.workunit.client.0.smithi151.stdout:podman-in 31621 root NOFD /proc/31621/fd (opendir: Permission denied) 2024-05-21T21:59:26.489 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 ceph cwd unknown /proc/31623/cwd (readlink: Permission denied) 2024-05-21T21:59:26.489 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 ceph rtd unknown /proc/31623/root (readlink: Permission denied) 2024-05-21T21:59:26.489 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 ceph txt unknown /proc/31623/exe (readlink: Permission denied) 2024-05-21T21:59:26.489 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 ceph NOFD /proc/31623/fd (opendir: Permission denied) 2024-05-21T21:59:26.489 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 31624 log ceph cwd unknown /proc/31623/task/31624/cwd (readlink: Permission denied) 2024-05-21T21:59:26.489 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 31624 log ceph rtd unknown /proc/31623/task/31624/root (readlink: Permission denied) 2024-05-21T21:59:26.489 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 31624 log ceph txt unknown /proc/31623/task/31624/exe (readlink: Permission denied) 2024-05-21T21:59:26.489 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 31624 log ceph NOFD /proc/31623/task/31624/fd (opendir: Permission denied) 2024-05-21T21:59:26.489 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 31625 service ceph cwd unknown /proc/31623/task/31625/cwd (readlink: Permission denied) 2024-05-21T21:59:26.489 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 31625 service ceph rtd unknown /proc/31623/task/31625/root (readlink: Permission denied) 2024-05-21T21:59:26.489 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 31625 service ceph txt unknown /proc/31623/task/31625/exe (readlink: Permission denied) 2024-05-21T21:59:26.490 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 31625 service ceph NOFD /proc/31623/task/31625/fd (opendir: Permission denied) 2024-05-21T21:59:26.490 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 31626 admin_soc ceph cwd unknown /proc/31623/task/31626/cwd (readlink: Permission denied) 2024-05-21T21:59:26.490 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 31626 admin_soc ceph rtd unknown /proc/31623/task/31626/root (readlink: Permission denied) 2024-05-21T21:59:26.490 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 31626 admin_soc ceph txt unknown /proc/31623/task/31626/exe (readlink: Permission denied) 2024-05-21T21:59:26.490 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 31626 admin_soc ceph NOFD /proc/31623/task/31626/fd (opendir: Permission denied) 2024-05-21T21:59:26.490 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 31627 signal_ha ceph cwd unknown /proc/31623/task/31627/cwd (readlink: Permission denied) 2024-05-21T21:59:26.490 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 31627 signal_ha ceph rtd unknown /proc/31623/task/31627/root (readlink: Permission denied) 2024-05-21T21:59:26.490 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 31627 signal_ha ceph txt unknown /proc/31623/task/31627/exe (readlink: Permission denied) 2024-05-21T21:59:26.490 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 31627 signal_ha ceph NOFD /proc/31623/task/31627/fd (opendir: Permission denied) 2024-05-21T21:59:26.490 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 31628 rocksdb:l ceph cwd unknown /proc/31623/task/31628/cwd (readlink: Permission denied) 2024-05-21T21:59:26.490 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 31628 rocksdb:l ceph rtd unknown /proc/31623/task/31628/root (readlink: Permission denied) 2024-05-21T21:59:26.490 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 31628 rocksdb:l ceph txt unknown /proc/31623/task/31628/exe (readlink: Permission denied) 2024-05-21T21:59:26.490 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 31628 rocksdb:l ceph NOFD /proc/31623/task/31628/fd (opendir: Permission denied) 2024-05-21T21:59:26.490 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 31629 rocksdb:h ceph cwd unknown /proc/31623/task/31629/cwd (readlink: Permission denied) 2024-05-21T21:59:26.491 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 31629 rocksdb:h ceph rtd unknown /proc/31623/task/31629/root (readlink: Permission denied) 2024-05-21T21:59:26.491 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 31629 rocksdb:h ceph txt unknown /proc/31623/task/31629/exe (readlink: Permission denied) 2024-05-21T21:59:26.491 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 31629 rocksdb:h ceph NOFD /proc/31623/task/31629/fd (opendir: Permission denied) 2024-05-21T21:59:26.491 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 31645 ceph-mon ceph cwd unknown /proc/31623/task/31645/cwd (readlink: Permission denied) 2024-05-21T21:59:26.491 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 31645 ceph-mon ceph rtd unknown /proc/31623/task/31645/root (readlink: Permission denied) 2024-05-21T21:59:26.491 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 31645 ceph-mon ceph txt unknown /proc/31623/task/31645/exe (readlink: Permission denied) 2024-05-21T21:59:26.491 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 31645 ceph-mon ceph NOFD /proc/31623/task/31645/fd (opendir: Permission denied) 2024-05-21T21:59:26.491 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 31646 fn_monsto ceph cwd unknown /proc/31623/task/31646/cwd (readlink: Permission denied) 2024-05-21T21:59:26.491 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 31646 fn_monsto ceph rtd unknown /proc/31623/task/31646/root (readlink: Permission denied) 2024-05-21T21:59:26.491 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 31646 fn_monsto ceph txt unknown /proc/31623/task/31646/exe (readlink: Permission denied) 2024-05-21T21:59:26.491 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 31646 fn_monsto ceph NOFD /proc/31623/task/31646/fd (opendir: Permission denied) 2024-05-21T21:59:26.491 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 31647 msgr-work ceph cwd unknown /proc/31623/task/31647/cwd (readlink: Permission denied) 2024-05-21T21:59:26.491 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 31647 msgr-work ceph rtd unknown /proc/31623/task/31647/root (readlink: Permission denied) 2024-05-21T21:59:26.492 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 31647 msgr-work ceph txt unknown /proc/31623/task/31647/exe (readlink: Permission denied) 2024-05-21T21:59:26.492 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 31647 msgr-work ceph NOFD /proc/31623/task/31647/fd (opendir: Permission denied) 2024-05-21T21:59:26.492 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 31648 msgr-work ceph cwd unknown /proc/31623/task/31648/cwd (readlink: Permission denied) 2024-05-21T21:59:26.492 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 31648 msgr-work ceph rtd unknown /proc/31623/task/31648/root (readlink: Permission denied) 2024-05-21T21:59:26.492 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 31648 msgr-work ceph txt unknown /proc/31623/task/31648/exe (readlink: Permission denied) 2024-05-21T21:59:26.492 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 31648 msgr-work ceph NOFD /proc/31623/task/31648/fd (opendir: Permission denied) 2024-05-21T21:59:26.492 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 31649 msgr-work ceph cwd unknown /proc/31623/task/31649/cwd (readlink: Permission denied) 2024-05-21T21:59:26.492 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 31649 msgr-work ceph rtd unknown /proc/31623/task/31649/root (readlink: Permission denied) 2024-05-21T21:59:26.492 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 31649 msgr-work ceph txt unknown /proc/31623/task/31649/exe (readlink: Permission denied) 2024-05-21T21:59:26.492 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 31649 msgr-work ceph NOFD /proc/31623/task/31649/fd (opendir: Permission denied) 2024-05-21T21:59:26.492 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 31650 OpHistory ceph cwd unknown /proc/31623/task/31650/cwd (readlink: Permission denied) 2024-05-21T21:59:26.492 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 31650 OpHistory ceph rtd unknown /proc/31623/task/31650/root (readlink: Permission denied) 2024-05-21T21:59:26.492 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 31650 OpHistory ceph txt unknown /proc/31623/task/31650/exe (readlink: Permission denied) 2024-05-21T21:59:26.493 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 31650 OpHistory ceph NOFD /proc/31623/task/31650/fd (opendir: Permission denied) 2024-05-21T21:59:26.493 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 31651 fin ceph cwd unknown /proc/31623/task/31651/cwd (readlink: Permission denied) 2024-05-21T21:59:26.493 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 31651 fin ceph rtd unknown /proc/31623/task/31651/root (readlink: Permission denied) 2024-05-21T21:59:26.493 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 31651 fin ceph txt unknown /proc/31623/task/31651/exe (readlink: Permission denied) 2024-05-21T21:59:26.493 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 31651 fin ceph NOFD /proc/31623/task/31651/fd (opendir: Permission denied) 2024-05-21T21:59:26.493 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 31652 safe_time ceph cwd unknown /proc/31623/task/31652/cwd (readlink: Permission denied) 2024-05-21T21:59:26.493 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 31652 safe_time ceph rtd unknown /proc/31623/task/31652/root (readlink: Permission denied) 2024-05-21T21:59:26.493 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 31652 safe_time ceph txt unknown /proc/31623/task/31652/exe (readlink: Permission denied) 2024-05-21T21:59:26.493 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 31652 safe_time ceph NOFD /proc/31623/task/31652/fd (opendir: Permission denied) 2024-05-21T21:59:26.493 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 31653 cpu_tp ceph cwd unknown /proc/31623/task/31653/cwd (readlink: Permission denied) 2024-05-21T21:59:26.493 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 31653 cpu_tp ceph rtd unknown /proc/31623/task/31653/root (readlink: Permission denied) 2024-05-21T21:59:26.493 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 31653 cpu_tp ceph txt unknown /proc/31623/task/31653/exe (readlink: Permission denied) 2024-05-21T21:59:26.493 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 31653 cpu_tp ceph NOFD /proc/31623/task/31653/fd (opendir: Permission denied) 2024-05-21T21:59:26.493 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 31654 cpu_tp ceph cwd unknown /proc/31623/task/31654/cwd (readlink: Permission denied) 2024-05-21T21:59:26.494 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 31654 cpu_tp ceph rtd unknown /proc/31623/task/31654/root (readlink: Permission denied) 2024-05-21T21:59:26.494 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 31654 cpu_tp ceph txt unknown /proc/31623/task/31654/exe (readlink: Permission denied) 2024-05-21T21:59:26.494 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 31654 cpu_tp ceph NOFD /proc/31623/task/31654/fd (opendir: Permission denied) 2024-05-21T21:59:26.494 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 31655 cpu_tp ceph cwd unknown /proc/31623/task/31655/cwd (readlink: Permission denied) 2024-05-21T21:59:26.494 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 31655 cpu_tp ceph rtd unknown /proc/31623/task/31655/root (readlink: Permission denied) 2024-05-21T21:59:26.494 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 31655 cpu_tp ceph txt unknown /proc/31623/task/31655/exe (readlink: Permission denied) 2024-05-21T21:59:26.494 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 31655 cpu_tp ceph NOFD /proc/31623/task/31655/fd (opendir: Permission denied) 2024-05-21T21:59:26.494 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 31656 cpu_tp ceph cwd unknown /proc/31623/task/31656/cwd (readlink: Permission denied) 2024-05-21T21:59:26.494 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 31656 cpu_tp ceph rtd unknown /proc/31623/task/31656/root (readlink: Permission denied) 2024-05-21T21:59:26.494 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 31656 cpu_tp ceph txt unknown /proc/31623/task/31656/exe (readlink: Permission denied) 2024-05-21T21:59:26.494 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 31656 cpu_tp ceph NOFD /proc/31623/task/31656/fd (opendir: Permission denied) 2024-05-21T21:59:26.494 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 31657 ms_dispat ceph cwd unknown /proc/31623/task/31657/cwd (readlink: Permission denied) 2024-05-21T21:59:26.495 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 31657 ms_dispat ceph rtd unknown /proc/31623/task/31657/root (readlink: Permission denied) 2024-05-21T21:59:26.495 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 31657 ms_dispat ceph txt unknown /proc/31623/task/31657/exe (readlink: Permission denied) 2024-05-21T21:59:26.495 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 31657 ms_dispat ceph NOFD /proc/31623/task/31657/fd (opendir: Permission denied) 2024-05-21T21:59:26.495 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 31658 ms_local ceph cwd unknown /proc/31623/task/31658/cwd (readlink: Permission denied) 2024-05-21T21:59:26.495 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 31658 ms_local ceph rtd unknown /proc/31623/task/31658/root (readlink: Permission denied) 2024-05-21T21:59:26.495 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 31658 ms_local ceph txt unknown /proc/31623/task/31658/exe (readlink: Permission denied) 2024-05-21T21:59:26.495 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 31658 ms_local ceph NOFD /proc/31623/task/31658/fd (opendir: Permission denied) 2024-05-21T21:59:26.495 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 31659 safe_time ceph cwd unknown /proc/31623/task/31659/cwd (readlink: Permission denied) 2024-05-21T21:59:26.495 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 31659 safe_time ceph rtd unknown /proc/31623/task/31659/root (readlink: Permission denied) 2024-05-21T21:59:26.495 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 31659 safe_time ceph txt unknown /proc/31623/task/31659/exe (readlink: Permission denied) 2024-05-21T21:59:26.495 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 31659 safe_time ceph NOFD /proc/31623/task/31659/fd (opendir: Permission denied) 2024-05-21T21:59:26.495 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 31660 ms_dispat ceph cwd unknown /proc/31623/task/31660/cwd (readlink: Permission denied) 2024-05-21T21:59:26.495 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 31660 ms_dispat ceph rtd unknown /proc/31623/task/31660/root (readlink: Permission denied) 2024-05-21T21:59:26.495 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 31660 ms_dispat ceph txt unknown /proc/31623/task/31660/exe (readlink: Permission denied) 2024-05-21T21:59:26.496 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 31660 ms_dispat ceph NOFD /proc/31623/task/31660/fd (opendir: Permission denied) 2024-05-21T21:59:26.496 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 31661 ms_local ceph cwd unknown /proc/31623/task/31661/cwd (readlink: Permission denied) 2024-05-21T21:59:26.496 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 31661 ms_local ceph rtd unknown /proc/31623/task/31661/root (readlink: Permission denied) 2024-05-21T21:59:26.496 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 31661 ms_local ceph txt unknown /proc/31623/task/31661/exe (readlink: Permission denied) 2024-05-21T21:59:26.496 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 31661 ms_local ceph NOFD /proc/31623/task/31661/fd (opendir: Permission denied) 2024-05-21T21:59:26.496 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 68352 rstore_co ceph cwd unknown /proc/31623/task/68352/cwd (readlink: Permission denied) 2024-05-21T21:59:26.496 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 68352 rstore_co ceph rtd unknown /proc/31623/task/68352/root (readlink: Permission denied) 2024-05-21T21:59:26.496 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 68352 rstore_co ceph txt unknown /proc/31623/task/68352/exe (readlink: Permission denied) 2024-05-21T21:59:26.496 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mon 31623 68352 rstore_co ceph NOFD /proc/31623/task/68352/fd (opendir: Permission denied) 2024-05-21T21:59:26.496 INFO:tasks.workunit.client.0.smithi151.stdout:conmon 31852 root cwd unknown /proc/31852/cwd (readlink: Permission denied) 2024-05-21T21:59:26.496 INFO:tasks.workunit.client.0.smithi151.stdout:conmon 31852 root rtd unknown /proc/31852/root (readlink: Permission denied) 2024-05-21T21:59:26.496 INFO:tasks.workunit.client.0.smithi151.stdout:conmon 31852 root txt unknown /proc/31852/exe (readlink: Permission denied) 2024-05-21T21:59:26.496 INFO:tasks.workunit.client.0.smithi151.stdout:conmon 31852 root NOFD /proc/31852/fd (opendir: Permission denied) 2024-05-21T21:59:26.497 INFO:tasks.workunit.client.0.smithi151.stdout:podman-in 31854 root cwd unknown /proc/31854/cwd (readlink: Permission denied) 2024-05-21T21:59:26.497 INFO:tasks.workunit.client.0.smithi151.stdout:podman-in 31854 root rtd unknown /proc/31854/root (readlink: Permission denied) 2024-05-21T21:59:26.497 INFO:tasks.workunit.client.0.smithi151.stdout:podman-in 31854 root txt unknown /proc/31854/exe (readlink: Permission denied) 2024-05-21T21:59:26.497 INFO:tasks.workunit.client.0.smithi151.stdout:podman-in 31854 root NOFD /proc/31854/fd (opendir: Permission denied) 2024-05-21T21:59:26.497 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 ceph cwd unknown /proc/31856/cwd (readlink: Permission denied) 2024-05-21T21:59:26.497 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 ceph rtd unknown /proc/31856/root (readlink: Permission denied) 2024-05-21T21:59:26.497 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 ceph txt unknown /proc/31856/exe (readlink: Permission denied) 2024-05-21T21:59:26.497 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 ceph NOFD /proc/31856/fd (opendir: Permission denied) 2024-05-21T21:59:26.497 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86020 log ceph cwd unknown /proc/31856/task/86020/cwd (readlink: Permission denied) 2024-05-21T21:59:26.497 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86020 log ceph rtd unknown /proc/31856/task/86020/root (readlink: Permission denied) 2024-05-21T21:59:26.497 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86020 log ceph txt unknown /proc/31856/task/86020/exe (readlink: Permission denied) 2024-05-21T21:59:26.497 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86020 log ceph NOFD /proc/31856/task/86020/fd (opendir: Permission denied) 2024-05-21T21:59:26.497 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86024 msgr-work ceph cwd unknown /proc/31856/task/86024/cwd (readlink: Permission denied) 2024-05-21T21:59:26.497 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86024 msgr-work ceph rtd unknown /proc/31856/task/86024/root (readlink: Permission denied) 2024-05-21T21:59:26.498 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86024 msgr-work ceph txt unknown /proc/31856/task/86024/exe (readlink: Permission denied) 2024-05-21T21:59:26.498 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86024 msgr-work ceph NOFD /proc/31856/task/86024/fd (opendir: Permission denied) 2024-05-21T21:59:26.498 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86026 msgr-work ceph cwd unknown /proc/31856/task/86026/cwd (readlink: Permission denied) 2024-05-21T21:59:26.498 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86026 msgr-work ceph rtd unknown /proc/31856/task/86026/root (readlink: Permission denied) 2024-05-21T21:59:26.498 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86026 msgr-work ceph txt unknown /proc/31856/task/86026/exe (readlink: Permission denied) 2024-05-21T21:59:26.498 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86026 msgr-work ceph NOFD /proc/31856/task/86026/fd (opendir: Permission denied) 2024-05-21T21:59:26.498 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86027 msgr-work ceph cwd unknown /proc/31856/task/86027/cwd (readlink: Permission denied) 2024-05-21T21:59:26.498 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86027 msgr-work ceph rtd unknown /proc/31856/task/86027/root (readlink: Permission denied) 2024-05-21T21:59:26.498 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86027 msgr-work ceph txt unknown /proc/31856/task/86027/exe (readlink: Permission denied) 2024-05-21T21:59:26.498 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86027 msgr-work ceph NOFD /proc/31856/task/86027/fd (opendir: Permission denied) 2024-05-21T21:59:26.498 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86033 service ceph cwd unknown /proc/31856/task/86033/cwd (readlink: Permission denied) 2024-05-21T21:59:26.498 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86033 service ceph rtd unknown /proc/31856/task/86033/root (readlink: Permission denied) 2024-05-21T21:59:26.499 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86033 service ceph txt unknown /proc/31856/task/86033/exe (readlink: Permission denied) 2024-05-21T21:59:26.499 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86033 service ceph NOFD /proc/31856/task/86033/fd (opendir: Permission denied) 2024-05-21T21:59:26.499 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86034 admin_soc ceph cwd unknown /proc/31856/task/86034/cwd (readlink: Permission denied) 2024-05-21T21:59:26.499 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86034 admin_soc ceph rtd unknown /proc/31856/task/86034/root (readlink: Permission denied) 2024-05-21T21:59:26.499 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86034 admin_soc ceph txt unknown /proc/31856/task/86034/exe (readlink: Permission denied) 2024-05-21T21:59:26.499 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86034 admin_soc ceph NOFD /proc/31856/task/86034/fd (opendir: Permission denied) 2024-05-21T21:59:26.499 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86035 ceph_time ceph cwd unknown /proc/31856/task/86035/cwd (readlink: Permission denied) 2024-05-21T21:59:26.499 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86035 ceph_time ceph rtd unknown /proc/31856/task/86035/root (readlink: Permission denied) 2024-05-21T21:59:26.499 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86035 ceph_time ceph txt unknown /proc/31856/task/86035/exe (readlink: Permission denied) 2024-05-21T21:59:26.499 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86035 ceph_time ceph NOFD /proc/31856/task/86035/fd (opendir: Permission denied) 2024-05-21T21:59:26.499 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86036 fn_anonym ceph cwd unknown /proc/31856/task/86036/cwd (readlink: Permission denied) 2024-05-21T21:59:26.499 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86036 fn_anonym ceph rtd unknown /proc/31856/task/86036/root (readlink: Permission denied) 2024-05-21T21:59:26.499 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86036 fn_anonym ceph txt unknown /proc/31856/task/86036/exe (readlink: Permission denied) 2024-05-21T21:59:26.499 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86036 fn_anonym ceph NOFD /proc/31856/task/86036/fd (opendir: Permission denied) 2024-05-21T21:59:26.500 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86037 signal_ha ceph cwd unknown /proc/31856/task/86037/cwd (readlink: Permission denied) 2024-05-21T21:59:26.500 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86037 signal_ha ceph rtd unknown /proc/31856/task/86037/root (readlink: Permission denied) 2024-05-21T21:59:26.500 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86037 signal_ha ceph txt unknown /proc/31856/task/86037/exe (readlink: Permission denied) 2024-05-21T21:59:26.500 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86037 signal_ha ceph NOFD /proc/31856/task/86037/fd (opendir: Permission denied) 2024-05-21T21:59:26.500 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86038 mgrsb-fin ceph cwd unknown /proc/31856/task/86038/cwd (readlink: Permission denied) 2024-05-21T21:59:26.500 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86038 mgrsb-fin ceph rtd unknown /proc/31856/task/86038/root (readlink: Permission denied) 2024-05-21T21:59:26.500 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86038 mgrsb-fin ceph txt unknown /proc/31856/task/86038/exe (readlink: Permission denied) 2024-05-21T21:59:26.500 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86038 mgrsb-fin ceph NOFD /proc/31856/task/86038/fd (opendir: Permission denied) 2024-05-21T21:59:26.500 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86039 ms_dispat ceph cwd unknown /proc/31856/task/86039/cwd (readlink: Permission denied) 2024-05-21T21:59:26.500 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86039 ms_dispat ceph rtd unknown /proc/31856/task/86039/root (readlink: Permission denied) 2024-05-21T21:59:26.500 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86039 ms_dispat ceph txt unknown /proc/31856/task/86039/exe (readlink: Permission denied) 2024-05-21T21:59:26.500 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86039 ms_dispat ceph NOFD /proc/31856/task/86039/fd (opendir: Permission denied) 2024-05-21T21:59:26.500 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86040 ms_local ceph cwd unknown /proc/31856/task/86040/cwd (readlink: Permission denied) 2024-05-21T21:59:26.500 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86040 ms_local ceph rtd unknown /proc/31856/task/86040/root (readlink: Permission denied) 2024-05-21T21:59:26.501 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86040 ms_local ceph txt unknown /proc/31856/task/86040/exe (readlink: Permission denied) 2024-05-21T21:59:26.501 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86040 ms_local ceph NOFD /proc/31856/task/86040/fd (opendir: Permission denied) 2024-05-21T21:59:26.501 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86041 io_contex ceph cwd unknown /proc/31856/task/86041/cwd (readlink: Permission denied) 2024-05-21T21:59:26.501 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86041 io_contex ceph rtd unknown /proc/31856/task/86041/root (readlink: Permission denied) 2024-05-21T21:59:26.501 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86041 io_contex ceph txt unknown /proc/31856/task/86041/exe (readlink: Permission denied) 2024-05-21T21:59:26.501 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86041 io_contex ceph NOFD /proc/31856/task/86041/fd (opendir: Permission denied) 2024-05-21T21:59:26.501 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86042 io_contex ceph cwd unknown /proc/31856/task/86042/cwd (readlink: Permission denied) 2024-05-21T21:59:26.501 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86042 io_contex ceph rtd unknown /proc/31856/task/86042/root (readlink: Permission denied) 2024-05-21T21:59:26.501 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86042 io_contex ceph txt unknown /proc/31856/task/86042/exe (readlink: Permission denied) 2024-05-21T21:59:26.501 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86042 io_contex ceph NOFD /proc/31856/task/86042/fd (opendir: Permission denied) 2024-05-21T21:59:26.501 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86044 safe_time ceph cwd unknown /proc/31856/task/86044/cwd (readlink: Permission denied) 2024-05-21T21:59:26.501 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86044 safe_time ceph rtd unknown /proc/31856/task/86044/root (readlink: Permission denied) 2024-05-21T21:59:26.501 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86044 safe_time ceph txt unknown /proc/31856/task/86044/exe (readlink: Permission denied) 2024-05-21T21:59:26.501 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86044 safe_time ceph NOFD /proc/31856/task/86044/fd (opendir: Permission denied) 2024-05-21T21:59:26.501 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86045 safe_time ceph cwd unknown /proc/31856/task/86045/cwd (readlink: Permission denied) 2024-05-21T21:59:26.502 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86045 safe_time ceph rtd unknown /proc/31856/task/86045/root (readlink: Permission denied) 2024-05-21T21:59:26.502 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86045 safe_time ceph txt unknown /proc/31856/task/86045/exe (readlink: Permission denied) 2024-05-21T21:59:26.502 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86045 safe_time ceph NOFD /proc/31856/task/86045/fd (opendir: Permission denied) 2024-05-21T21:59:26.502 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86046 safe_time ceph cwd unknown /proc/31856/task/86046/cwd (readlink: Permission denied) 2024-05-21T21:59:26.502 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86046 safe_time ceph rtd unknown /proc/31856/task/86046/root (readlink: Permission denied) 2024-05-21T21:59:26.502 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86046 safe_time ceph txt unknown /proc/31856/task/86046/exe (readlink: Permission denied) 2024-05-21T21:59:26.502 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86046 safe_time ceph NOFD /proc/31856/task/86046/fd (opendir: Permission denied) 2024-05-21T21:59:26.502 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86047 fn_anonym ceph cwd unknown /proc/31856/task/86047/cwd (readlink: Permission denied) 2024-05-21T21:59:26.502 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86047 fn_anonym ceph rtd unknown /proc/31856/task/86047/root (readlink: Permission denied) 2024-05-21T21:59:26.502 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86047 fn_anonym ceph txt unknown /proc/31856/task/86047/exe (readlink: Permission denied) 2024-05-21T21:59:26.502 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86047 fn_anonym ceph NOFD /proc/31856/task/86047/fd (opendir: Permission denied) 2024-05-21T21:59:26.503 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86048 flusher ceph cwd unknown /proc/31856/task/86048/cwd (readlink: Permission denied) 2024-05-21T21:59:26.503 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86048 flusher ceph rtd unknown /proc/31856/task/86048/root (readlink: Permission denied) 2024-05-21T21:59:26.503 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86048 flusher ceph txt unknown /proc/31856/task/86048/exe (readlink: Permission denied) 2024-05-21T21:59:26.503 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86048 flusher ceph NOFD /proc/31856/task/86048/fd (opendir: Permission denied) 2024-05-21T21:59:26.503 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86049 safe_time ceph cwd unknown /proc/31856/task/86049/cwd (readlink: Permission denied) 2024-05-21T21:59:26.503 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86049 safe_time ceph rtd unknown /proc/31856/task/86049/root (readlink: Permission denied) 2024-05-21T21:59:26.503 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86049 safe_time ceph txt unknown /proc/31856/task/86049/exe (readlink: Permission denied) 2024-05-21T21:59:26.503 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86049 safe_time ceph NOFD /proc/31856/task/86049/fd (opendir: Permission denied) 2024-05-21T21:59:26.503 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86824 OpHistory ceph cwd unknown /proc/31856/task/86824/cwd (readlink: Permission denied) 2024-05-21T21:59:26.503 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86824 OpHistory ceph rtd unknown /proc/31856/task/86824/root (readlink: Permission denied) 2024-05-21T21:59:26.503 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86824 OpHistory ceph txt unknown /proc/31856/task/86824/exe (readlink: Permission denied) 2024-05-21T21:59:26.503 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86824 OpHistory ceph NOFD /proc/31856/task/86824/fd (opendir: Permission denied) 2024-05-21T21:59:26.504 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86825 mgr-fin ceph cwd unknown /proc/31856/task/86825/cwd (readlink: Permission denied) 2024-05-21T21:59:26.504 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86825 mgr-fin ceph rtd unknown /proc/31856/task/86825/root (readlink: Permission denied) 2024-05-21T21:59:26.504 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86825 mgr-fin ceph txt unknown /proc/31856/task/86825/exe (readlink: Permission denied) 2024-05-21T21:59:26.504 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86825 mgr-fin ceph NOFD /proc/31856/task/86825/fd (opendir: Permission denied) 2024-05-21T21:59:26.504 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86826 ms_dispat ceph cwd unknown /proc/31856/task/86826/cwd (readlink: Permission denied) 2024-05-21T21:59:26.504 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86826 ms_dispat ceph rtd unknown /proc/31856/task/86826/root (readlink: Permission denied) 2024-05-21T21:59:26.504 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86826 ms_dispat ceph txt unknown /proc/31856/task/86826/exe (readlink: Permission denied) 2024-05-21T21:59:26.504 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86826 ms_dispat ceph NOFD /proc/31856/task/86826/fd (opendir: Permission denied) 2024-05-21T21:59:26.504 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86827 ms_local ceph cwd unknown /proc/31856/task/86827/cwd (readlink: Permission denied) 2024-05-21T21:59:26.504 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86827 ms_local ceph rtd unknown /proc/31856/task/86827/root (readlink: Permission denied) 2024-05-21T21:59:26.504 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86827 ms_local ceph txt unknown /proc/31856/task/86827/exe (readlink: Permission denied) 2024-05-21T21:59:26.504 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86827 ms_local ceph NOFD /proc/31856/task/86827/fd (opendir: Permission denied) 2024-05-21T21:59:26.504 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86828 safe_time ceph cwd unknown /proc/31856/task/86828/cwd (readlink: Permission denied) 2024-05-21T21:59:26.505 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86828 safe_time ceph rtd unknown /proc/31856/task/86828/root (readlink: Permission denied) 2024-05-21T21:59:26.505 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86828 safe_time ceph txt unknown /proc/31856/task/86828/exe (readlink: Permission denied) 2024-05-21T21:59:26.505 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86828 safe_time ceph NOFD /proc/31856/task/86828/fd (opendir: Permission denied) 2024-05-21T21:59:26.505 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86829 cmdfin ceph cwd unknown /proc/31856/task/86829/cwd (readlink: Permission denied) 2024-05-21T21:59:26.505 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86829 cmdfin ceph rtd unknown /proc/31856/task/86829/root (readlink: Permission denied) 2024-05-21T21:59:26.505 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86829 cmdfin ceph txt unknown /proc/31856/task/86829/exe (readlink: Permission denied) 2024-05-21T21:59:26.505 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86829 cmdfin ceph NOFD /proc/31856/task/86829/fd (opendir: Permission denied) 2024-05-21T21:59:26.505 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86833 io_contex ceph cwd unknown /proc/31856/task/86833/cwd (readlink: Permission denied) 2024-05-21T21:59:26.505 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86833 io_contex ceph rtd unknown /proc/31856/task/86833/root (readlink: Permission denied) 2024-05-21T21:59:26.505 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86833 io_contex ceph txt unknown /proc/31856/task/86833/exe (readlink: Permission denied) 2024-05-21T21:59:26.505 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86833 io_contex ceph NOFD /proc/31856/task/86833/fd (opendir: Permission denied) 2024-05-21T21:59:26.505 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86834 io_contex ceph cwd unknown /proc/31856/task/86834/cwd (readlink: Permission denied) 2024-05-21T21:59:26.505 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86834 io_contex ceph rtd unknown /proc/31856/task/86834/root (readlink: Permission denied) 2024-05-21T21:59:26.505 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86834 io_contex ceph txt unknown /proc/31856/task/86834/exe (readlink: Permission denied) 2024-05-21T21:59:26.506 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86834 io_contex ceph NOFD /proc/31856/task/86834/fd (opendir: Permission denied) 2024-05-21T21:59:26.506 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86835 ceph_time ceph cwd unknown /proc/31856/task/86835/cwd (readlink: Permission denied) 2024-05-21T21:59:26.506 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86835 ceph_time ceph rtd unknown /proc/31856/task/86835/root (readlink: Permission denied) 2024-05-21T21:59:26.506 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86835 ceph_time ceph txt unknown /proc/31856/task/86835/exe (readlink: Permission denied) 2024-05-21T21:59:26.506 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86835 ceph_time ceph NOFD /proc/31856/task/86835/fd (opendir: Permission denied) 2024-05-21T21:59:26.506 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86836 ms_dispat ceph cwd unknown /proc/31856/task/86836/cwd (readlink: Permission denied) 2024-05-21T21:59:26.506 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86836 ms_dispat ceph rtd unknown /proc/31856/task/86836/root (readlink: Permission denied) 2024-05-21T21:59:26.506 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86836 ms_dispat ceph txt unknown /proc/31856/task/86836/exe (readlink: Permission denied) 2024-05-21T21:59:26.506 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86836 ms_dispat ceph NOFD /proc/31856/task/86836/fd (opendir: Permission denied) 2024-05-21T21:59:26.506 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86837 ms_local ceph cwd unknown /proc/31856/task/86837/cwd (readlink: Permission denied) 2024-05-21T21:59:26.506 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86837 ms_local ceph rtd unknown /proc/31856/task/86837/root (readlink: Permission denied) 2024-05-21T21:59:26.506 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86837 ms_local ceph txt unknown /proc/31856/task/86837/exe (readlink: Permission denied) 2024-05-21T21:59:26.506 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86837 ms_local ceph NOFD /proc/31856/task/86837/fd (opendir: Permission denied) 2024-05-21T21:59:26.506 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86838 safe_time ceph cwd unknown /proc/31856/task/86838/cwd (readlink: Permission denied) 2024-05-21T21:59:26.507 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86838 safe_time ceph rtd unknown /proc/31856/task/86838/root (readlink: Permission denied) 2024-05-21T21:59:26.507 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86838 safe_time ceph txt unknown /proc/31856/task/86838/exe (readlink: Permission denied) 2024-05-21T21:59:26.507 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86838 safe_time ceph NOFD /proc/31856/task/86838/fd (opendir: Permission denied) 2024-05-21T21:59:26.507 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86839 safe_time ceph cwd unknown /proc/31856/task/86839/cwd (readlink: Permission denied) 2024-05-21T21:59:26.507 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86839 safe_time ceph rtd unknown /proc/31856/task/86839/root (readlink: Permission denied) 2024-05-21T21:59:26.507 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86839 safe_time ceph txt unknown /proc/31856/task/86839/exe (readlink: Permission denied) 2024-05-21T21:59:26.507 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86839 safe_time ceph NOFD /proc/31856/task/86839/fd (opendir: Permission denied) 2024-05-21T21:59:26.507 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86840 balancer ceph cwd unknown /proc/31856/task/86840/cwd (readlink: Permission denied) 2024-05-21T21:59:26.507 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86840 balancer ceph rtd unknown /proc/31856/task/86840/root (readlink: Permission denied) 2024-05-21T21:59:26.507 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86840 balancer ceph txt unknown /proc/31856/task/86840/exe (readlink: Permission denied) 2024-05-21T21:59:26.507 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86840 balancer ceph NOFD /proc/31856/task/86840/fd (opendir: Permission denied) 2024-05-21T21:59:26.507 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86841 m-fin-bal ceph cwd unknown /proc/31856/task/86841/cwd (readlink: Permission denied) 2024-05-21T21:59:26.507 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86841 m-fin-bal ceph rtd unknown /proc/31856/task/86841/root (readlink: Permission denied) 2024-05-21T21:59:26.507 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86841 m-fin-bal ceph txt unknown /proc/31856/task/86841/exe (readlink: Permission denied) 2024-05-21T21:59:26.507 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86841 m-fin-bal ceph NOFD /proc/31856/task/86841/fd (opendir: Permission denied) 2024-05-21T21:59:26.508 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86842 mgr-fin ceph cwd unknown /proc/31856/task/86842/cwd (readlink: Permission denied) 2024-05-21T21:59:26.508 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86842 mgr-fin ceph rtd unknown /proc/31856/task/86842/root (readlink: Permission denied) 2024-05-21T21:59:26.508 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86842 mgr-fin ceph txt unknown /proc/31856/task/86842/exe (readlink: Permission denied) 2024-05-21T21:59:26.508 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86842 mgr-fin ceph NOFD /proc/31856/task/86842/fd (opendir: Permission denied) 2024-05-21T21:59:26.508 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86843 mgr-fin ceph cwd unknown /proc/31856/task/86843/cwd (readlink: Permission denied) 2024-05-21T21:59:26.508 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86843 mgr-fin ceph rtd unknown /proc/31856/task/86843/root (readlink: Permission denied) 2024-05-21T21:59:26.508 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86843 mgr-fin ceph txt unknown /proc/31856/task/86843/exe (readlink: Permission denied) 2024-05-21T21:59:26.508 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86843 mgr-fin ceph NOFD /proc/31856/task/86843/fd (opendir: Permission denied) 2024-05-21T21:59:26.508 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86844 mgr-fin ceph cwd unknown /proc/31856/task/86844/cwd (readlink: Permission denied) 2024-05-21T21:59:26.508 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86844 mgr-fin ceph rtd unknown /proc/31856/task/86844/root (readlink: Permission denied) 2024-05-21T21:59:26.508 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86844 mgr-fin ceph txt unknown /proc/31856/task/86844/exe (readlink: Permission denied) 2024-05-21T21:59:26.508 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86844 mgr-fin ceph NOFD /proc/31856/task/86844/fd (opendir: Permission denied) 2024-05-21T21:59:26.508 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86845 mgr-fin ceph cwd unknown /proc/31856/task/86845/cwd (readlink: Permission denied) 2024-05-21T21:59:26.509 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86845 mgr-fin ceph rtd unknown /proc/31856/task/86845/root (readlink: Permission denied) 2024-05-21T21:59:26.509 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86845 mgr-fin ceph txt unknown /proc/31856/task/86845/exe (readlink: Permission denied) 2024-05-21T21:59:26.509 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86845 mgr-fin ceph NOFD /proc/31856/task/86845/fd (opendir: Permission denied) 2024-05-21T21:59:26.509 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86846 mgr-fin ceph cwd unknown /proc/31856/task/86846/cwd (readlink: Permission denied) 2024-05-21T21:59:26.509 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86846 mgr-fin ceph rtd unknown /proc/31856/task/86846/root (readlink: Permission denied) 2024-05-21T21:59:26.509 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86846 mgr-fin ceph txt unknown /proc/31856/task/86846/exe (readlink: Permission denied) 2024-05-21T21:59:26.509 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86846 mgr-fin ceph NOFD /proc/31856/task/86846/fd (opendir: Permission denied) 2024-05-21T21:59:26.509 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86847 mgr-fin ceph cwd unknown /proc/31856/task/86847/cwd (readlink: Permission denied) 2024-05-21T21:59:26.509 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86847 mgr-fin ceph rtd unknown /proc/31856/task/86847/root (readlink: Permission denied) 2024-05-21T21:59:26.509 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86847 mgr-fin ceph txt unknown /proc/31856/task/86847/exe (readlink: Permission denied) 2024-05-21T21:59:26.509 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86847 mgr-fin ceph NOFD /proc/31856/task/86847/fd (opendir: Permission denied) 2024-05-21T21:59:26.509 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86848 mgr-fin ceph cwd unknown /proc/31856/task/86848/cwd (readlink: Permission denied) 2024-05-21T21:59:26.509 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86848 mgr-fin ceph rtd unknown /proc/31856/task/86848/root (readlink: Permission denied) 2024-05-21T21:59:26.509 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86848 mgr-fin ceph txt unknown /proc/31856/task/86848/exe (readlink: Permission denied) 2024-05-21T21:59:26.510 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86848 mgr-fin ceph NOFD /proc/31856/task/86848/fd (opendir: Permission denied) 2024-05-21T21:59:26.510 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86849 mgr-fin ceph cwd unknown /proc/31856/task/86849/cwd (readlink: Permission denied) 2024-05-21T21:59:26.510 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86849 mgr-fin ceph rtd unknown /proc/31856/task/86849/root (readlink: Permission denied) 2024-05-21T21:59:26.510 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86849 mgr-fin ceph txt unknown /proc/31856/task/86849/exe (readlink: Permission denied) 2024-05-21T21:59:26.510 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86849 mgr-fin ceph NOFD /proc/31856/task/86849/fd (opendir: Permission denied) 2024-05-21T21:59:26.510 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86850 mgr-fin ceph cwd unknown /proc/31856/task/86850/cwd (readlink: Permission denied) 2024-05-21T21:59:26.510 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86850 mgr-fin ceph rtd unknown /proc/31856/task/86850/root (readlink: Permission denied) 2024-05-21T21:59:26.510 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86850 mgr-fin ceph txt unknown /proc/31856/task/86850/exe (readlink: Permission denied) 2024-05-21T21:59:26.510 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86850 mgr-fin ceph NOFD /proc/31856/task/86850/fd (opendir: Permission denied) 2024-05-21T21:59:26.510 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86851 mgr-fin ceph cwd unknown /proc/31856/task/86851/cwd (readlink: Permission denied) 2024-05-21T21:59:26.510 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86851 mgr-fin ceph rtd unknown /proc/31856/task/86851/root (readlink: Permission denied) 2024-05-21T21:59:26.510 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86851 mgr-fin ceph txt unknown /proc/31856/task/86851/exe (readlink: Permission denied) 2024-05-21T21:59:26.510 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86851 mgr-fin ceph NOFD /proc/31856/task/86851/fd (opendir: Permission denied) 2024-05-21T21:59:26.510 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86852 mgr-fin ceph cwd unknown /proc/31856/task/86852/cwd (readlink: Permission denied) 2024-05-21T21:59:26.510 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86852 mgr-fin ceph rtd unknown /proc/31856/task/86852/root (readlink: Permission denied) 2024-05-21T21:59:26.511 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86852 mgr-fin ceph txt unknown /proc/31856/task/86852/exe (readlink: Permission denied) 2024-05-21T21:59:26.511 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86852 mgr-fin ceph NOFD /proc/31856/task/86852/fd (opendir: Permission denied) 2024-05-21T21:59:26.511 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86853 mgr-fin ceph cwd unknown /proc/31856/task/86853/cwd (readlink: Permission denied) 2024-05-21T21:59:26.511 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86853 mgr-fin ceph rtd unknown /proc/31856/task/86853/root (readlink: Permission denied) 2024-05-21T21:59:26.511 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86853 mgr-fin ceph txt unknown /proc/31856/task/86853/exe (readlink: Permission denied) 2024-05-21T21:59:26.511 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86853 mgr-fin ceph NOFD /proc/31856/task/86853/fd (opendir: Permission denied) 2024-05-21T21:59:26.511 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86854 mgr-fin ceph cwd unknown /proc/31856/task/86854/cwd (readlink: Permission denied) 2024-05-21T21:59:26.511 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86854 mgr-fin ceph rtd unknown /proc/31856/task/86854/root (readlink: Permission denied) 2024-05-21T21:59:26.511 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86854 mgr-fin ceph txt unknown /proc/31856/task/86854/exe (readlink: Permission denied) 2024-05-21T21:59:26.511 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86854 mgr-fin ceph NOFD /proc/31856/task/86854/fd (opendir: Permission denied) 2024-05-21T21:59:26.511 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86855 mgr-fin ceph cwd unknown /proc/31856/task/86855/cwd (readlink: Permission denied) 2024-05-21T21:59:26.511 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86855 mgr-fin ceph rtd unknown /proc/31856/task/86855/root (readlink: Permission denied) 2024-05-21T21:59:26.511 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86855 mgr-fin ceph txt unknown /proc/31856/task/86855/exe (readlink: Permission denied) 2024-05-21T21:59:26.511 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86855 mgr-fin ceph NOFD /proc/31856/task/86855/fd (opendir: Permission denied) 2024-05-21T21:59:26.511 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86856 mgr-fin ceph cwd unknown /proc/31856/task/86856/cwd (readlink: Permission denied) 2024-05-21T21:59:26.512 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86856 mgr-fin ceph rtd unknown /proc/31856/task/86856/root (readlink: Permission denied) 2024-05-21T21:59:26.512 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86856 mgr-fin ceph txt unknown /proc/31856/task/86856/exe (readlink: Permission denied) 2024-05-21T21:59:26.512 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86856 mgr-fin ceph NOFD /proc/31856/task/86856/fd (opendir: Permission denied) 2024-05-21T21:59:26.512 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86857 mgr-fin ceph cwd unknown /proc/31856/task/86857/cwd (readlink: Permission denied) 2024-05-21T21:59:26.512 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86857 mgr-fin ceph rtd unknown /proc/31856/task/86857/root (readlink: Permission denied) 2024-05-21T21:59:26.512 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86857 mgr-fin ceph txt unknown /proc/31856/task/86857/exe (readlink: Permission denied) 2024-05-21T21:59:26.512 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86857 mgr-fin ceph NOFD /proc/31856/task/86857/fd (opendir: Permission denied) 2024-05-21T21:59:26.512 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86858 cephadm ceph cwd unknown /proc/31856/task/86858/cwd (readlink: Permission denied) 2024-05-21T21:59:26.512 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86858 cephadm ceph rtd unknown /proc/31856/task/86858/root (readlink: Permission denied) 2024-05-21T21:59:26.512 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86858 cephadm ceph txt unknown /proc/31856/task/86858/exe (readlink: Permission denied) 2024-05-21T21:59:26.512 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86858 cephadm ceph NOFD /proc/31856/task/86858/fd (opendir: Permission denied) 2024-05-21T21:59:26.512 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86859 m-fin-cep ceph cwd unknown /proc/31856/task/86859/cwd (readlink: Permission denied) 2024-05-21T21:59:26.512 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86859 m-fin-cep ceph rtd unknown /proc/31856/task/86859/root (readlink: Permission denied) 2024-05-21T21:59:26.512 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86859 m-fin-cep ceph txt unknown /proc/31856/task/86859/exe (readlink: Permission denied) 2024-05-21T21:59:26.512 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86859 m-fin-cep ceph NOFD /proc/31856/task/86859/fd (opendir: Permission denied) 2024-05-21T21:59:26.513 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86860 mgr-fin ceph cwd unknown /proc/31856/task/86860/cwd (readlink: Permission denied) 2024-05-21T21:59:26.513 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86860 mgr-fin ceph rtd unknown /proc/31856/task/86860/root (readlink: Permission denied) 2024-05-21T21:59:26.513 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86860 mgr-fin ceph txt unknown /proc/31856/task/86860/exe (readlink: Permission denied) 2024-05-21T21:59:26.513 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86860 mgr-fin ceph NOFD /proc/31856/task/86860/fd (opendir: Permission denied) 2024-05-21T21:59:26.513 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86861 crash ceph cwd unknown /proc/31856/task/86861/cwd (readlink: Permission denied) 2024-05-21T21:59:26.513 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86861 crash ceph rtd unknown /proc/31856/task/86861/root (readlink: Permission denied) 2024-05-21T21:59:26.513 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86861 crash ceph txt unknown /proc/31856/task/86861/exe (readlink: Permission denied) 2024-05-21T21:59:26.513 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86861 crash ceph NOFD /proc/31856/task/86861/fd (opendir: Permission denied) 2024-05-21T21:59:26.513 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86862 m-fin-cra ceph cwd unknown /proc/31856/task/86862/cwd (readlink: Permission denied) 2024-05-21T21:59:26.513 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86862 m-fin-cra ceph rtd unknown /proc/31856/task/86862/root (readlink: Permission denied) 2024-05-21T21:59:26.513 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86862 m-fin-cra ceph txt unknown /proc/31856/task/86862/exe (readlink: Permission denied) 2024-05-21T21:59:26.513 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86862 m-fin-cra ceph NOFD /proc/31856/task/86862/fd (opendir: Permission denied) 2024-05-21T21:59:26.513 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86863 cephadm ceph cwd unknown /proc/31856/task/86863/cwd (readlink: Permission denied) 2024-05-21T21:59:26.513 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86863 cephadm ceph rtd unknown /proc/31856/task/86863/root (readlink: Permission denied) 2024-05-21T21:59:26.514 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86863 cephadm ceph txt unknown /proc/31856/task/86863/exe (readlink: Permission denied) 2024-05-21T21:59:26.514 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86863 cephadm ceph NOFD /proc/31856/task/86863/fd (opendir: Permission denied) 2024-05-21T21:59:26.514 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86864 mgr-fin ceph cwd unknown /proc/31856/task/86864/cwd (readlink: Permission denied) 2024-05-21T21:59:26.514 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86864 mgr-fin ceph rtd unknown /proc/31856/task/86864/root (readlink: Permission denied) 2024-05-21T21:59:26.514 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86864 mgr-fin ceph txt unknown /proc/31856/task/86864/exe (readlink: Permission denied) 2024-05-21T21:59:26.514 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86864 mgr-fin ceph NOFD /proc/31856/task/86864/fd (opendir: Permission denied) 2024-05-21T21:59:26.514 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86865 mgr-fin ceph cwd unknown /proc/31856/task/86865/cwd (readlink: Permission denied) 2024-05-21T21:59:26.514 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86865 mgr-fin ceph rtd unknown /proc/31856/task/86865/root (readlink: Permission denied) 2024-05-21T21:59:26.514 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86865 mgr-fin ceph txt unknown /proc/31856/task/86865/exe (readlink: Permission denied) 2024-05-21T21:59:26.514 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86865 mgr-fin ceph NOFD /proc/31856/task/86865/fd (opendir: Permission denied) 2024-05-21T21:59:26.514 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86866 mgr-fin ceph cwd unknown /proc/31856/task/86866/cwd (readlink: Permission denied) 2024-05-21T21:59:26.514 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86866 mgr-fin ceph rtd unknown /proc/31856/task/86866/root (readlink: Permission denied) 2024-05-21T21:59:26.514 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86866 mgr-fin ceph txt unknown /proc/31856/task/86866/exe (readlink: Permission denied) 2024-05-21T21:59:26.514 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86866 mgr-fin ceph NOFD /proc/31856/task/86866/fd (opendir: Permission denied) 2024-05-21T21:59:26.514 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86867 dashboard ceph cwd unknown /proc/31856/task/86867/cwd (readlink: Permission denied) 2024-05-21T21:59:26.515 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86867 dashboard ceph rtd unknown /proc/31856/task/86867/root (readlink: Permission denied) 2024-05-21T21:59:26.515 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86867 dashboard ceph txt unknown /proc/31856/task/86867/exe (readlink: Permission denied) 2024-05-21T21:59:26.515 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86867 dashboard ceph NOFD /proc/31856/task/86867/fd (opendir: Permission denied) 2024-05-21T21:59:26.515 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86868 mgr-fin ceph cwd unknown /proc/31856/task/86868/cwd (readlink: Permission denied) 2024-05-21T21:59:26.515 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86868 mgr-fin ceph rtd unknown /proc/31856/task/86868/root (readlink: Permission denied) 2024-05-21T21:59:26.515 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86868 mgr-fin ceph txt unknown /proc/31856/task/86868/exe (readlink: Permission denied) 2024-05-21T21:59:26.515 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86868 mgr-fin ceph NOFD /proc/31856/task/86868/fd (opendir: Permission denied) 2024-05-21T21:59:26.515 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86869 m-fin-das ceph cwd unknown /proc/31856/task/86869/cwd (readlink: Permission denied) 2024-05-21T21:59:26.515 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86869 m-fin-das ceph rtd unknown /proc/31856/task/86869/root (readlink: Permission denied) 2024-05-21T21:59:26.515 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86869 m-fin-das ceph txt unknown /proc/31856/task/86869/exe (readlink: Permission denied) 2024-05-21T21:59:26.515 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86869 m-fin-das ceph NOFD /proc/31856/task/86869/fd (opendir: Permission denied) 2024-05-21T21:59:26.515 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86870 mgr-fin ceph cwd unknown /proc/31856/task/86870/cwd (readlink: Permission denied) 2024-05-21T21:59:26.515 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86870 mgr-fin ceph rtd unknown /proc/31856/task/86870/root (readlink: Permission denied) 2024-05-21T21:59:26.515 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86870 mgr-fin ceph txt unknown /proc/31856/task/86870/exe (readlink: Permission denied) 2024-05-21T21:59:26.515 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86870 mgr-fin ceph NOFD /proc/31856/task/86870/fd (opendir: Permission denied) 2024-05-21T21:59:26.516 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86871 devicehea ceph cwd unknown /proc/31856/task/86871/cwd (readlink: Permission denied) 2024-05-21T21:59:26.516 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86871 devicehea ceph rtd unknown /proc/31856/task/86871/root (readlink: Permission denied) 2024-05-21T21:59:26.516 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86871 devicehea ceph txt unknown /proc/31856/task/86871/exe (readlink: Permission denied) 2024-05-21T21:59:26.516 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86871 devicehea ceph NOFD /proc/31856/task/86871/fd (opendir: Permission denied) 2024-05-21T21:59:26.516 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86872 m-fin-dev ceph cwd unknown /proc/31856/task/86872/cwd (readlink: Permission denied) 2024-05-21T21:59:26.516 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86872 m-fin-dev ceph rtd unknown /proc/31856/task/86872/root (readlink: Permission denied) 2024-05-21T21:59:26.516 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86872 m-fin-dev ceph txt unknown /proc/31856/task/86872/exe (readlink: Permission denied) 2024-05-21T21:59:26.516 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86872 m-fin-dev ceph NOFD /proc/31856/task/86872/fd (opendir: Permission denied) 2024-05-21T21:59:26.516 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86873 mgr-fin ceph cwd unknown /proc/31856/task/86873/cwd (readlink: Permission denied) 2024-05-21T21:59:26.516 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86873 mgr-fin ceph rtd unknown /proc/31856/task/86873/root (readlink: Permission denied) 2024-05-21T21:59:26.516 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86873 mgr-fin ceph txt unknown /proc/31856/task/86873/exe (readlink: Permission denied) 2024-05-21T21:59:26.516 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86873 mgr-fin ceph NOFD /proc/31856/task/86873/fd (opendir: Permission denied) 2024-05-21T21:59:26.516 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86875 m-fin-ios ceph cwd unknown /proc/31856/task/86875/cwd (readlink: Permission denied) 2024-05-21T21:59:26.516 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86875 m-fin-ios ceph rtd unknown /proc/31856/task/86875/root (readlink: Permission denied) 2024-05-21T21:59:26.516 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86875 m-fin-ios ceph txt unknown /proc/31856/task/86875/exe (readlink: Permission denied) 2024-05-21T21:59:26.517 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86875 m-fin-ios ceph NOFD /proc/31856/task/86875/fd (opendir: Permission denied) 2024-05-21T21:59:26.517 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86876 mgr-fin ceph cwd unknown /proc/31856/task/86876/cwd (readlink: Permission denied) 2024-05-21T21:59:26.517 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86876 mgr-fin ceph rtd unknown /proc/31856/task/86876/root (readlink: Permission denied) 2024-05-21T21:59:26.517 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86876 mgr-fin ceph txt unknown /proc/31856/task/86876/exe (readlink: Permission denied) 2024-05-21T21:59:26.517 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86876 mgr-fin ceph NOFD /proc/31856/task/86876/fd (opendir: Permission denied) 2024-05-21T21:59:26.517 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86878 m-fin-nfs ceph cwd unknown /proc/31856/task/86878/cwd (readlink: Permission denied) 2024-05-21T21:59:26.517 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86878 m-fin-nfs ceph rtd unknown /proc/31856/task/86878/root (readlink: Permission denied) 2024-05-21T21:59:26.517 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86878 m-fin-nfs ceph txt unknown /proc/31856/task/86878/exe (readlink: Permission denied) 2024-05-21T21:59:26.517 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86878 m-fin-nfs ceph NOFD /proc/31856/task/86878/fd (opendir: Permission denied) 2024-05-21T21:59:26.517 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86879 mgr-fin ceph cwd unknown /proc/31856/task/86879/cwd (readlink: Permission denied) 2024-05-21T21:59:26.517 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86879 mgr-fin ceph rtd unknown /proc/31856/task/86879/root (readlink: Permission denied) 2024-05-21T21:59:26.517 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86879 mgr-fin ceph txt unknown /proc/31856/task/86879/exe (readlink: Permission denied) 2024-05-21T21:59:26.517 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86879 mgr-fin ceph NOFD /proc/31856/task/86879/fd (opendir: Permission denied) 2024-05-21T21:59:26.517 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86881 m-fin-orc ceph cwd unknown /proc/31856/task/86881/cwd (readlink: Permission denied) 2024-05-21T21:59:26.517 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86881 m-fin-orc ceph rtd unknown /proc/31856/task/86881/root (readlink: Permission denied) 2024-05-21T21:59:26.518 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86881 m-fin-orc ceph txt unknown /proc/31856/task/86881/exe (readlink: Permission denied) 2024-05-21T21:59:26.518 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86881 m-fin-orc ceph NOFD /proc/31856/task/86881/fd (opendir: Permission denied) 2024-05-21T21:59:26.518 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86882 mgr-fin ceph cwd unknown /proc/31856/task/86882/cwd (readlink: Permission denied) 2024-05-21T21:59:26.518 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86882 mgr-fin ceph rtd unknown /proc/31856/task/86882/root (readlink: Permission denied) 2024-05-21T21:59:26.518 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86882 mgr-fin ceph txt unknown /proc/31856/task/86882/exe (readlink: Permission denied) 2024-05-21T21:59:26.518 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86882 mgr-fin ceph NOFD /proc/31856/task/86882/fd (opendir: Permission denied) 2024-05-21T21:59:26.518 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86883 mgr-fin ceph cwd unknown /proc/31856/task/86883/cwd (readlink: Permission denied) 2024-05-21T21:59:26.518 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86883 mgr-fin ceph rtd unknown /proc/31856/task/86883/root (readlink: Permission denied) 2024-05-21T21:59:26.518 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86883 mgr-fin ceph txt unknown /proc/31856/task/86883/exe (readlink: Permission denied) 2024-05-21T21:59:26.518 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86883 mgr-fin ceph NOFD /proc/31856/task/86883/fd (opendir: Permission denied) 2024-05-21T21:59:26.518 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86884 mgr-fin ceph cwd unknown /proc/31856/task/86884/cwd (readlink: Permission denied) 2024-05-21T21:59:26.518 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86884 mgr-fin ceph rtd unknown /proc/31856/task/86884/root (readlink: Permission denied) 2024-05-21T21:59:26.518 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86884 mgr-fin ceph txt unknown /proc/31856/task/86884/exe (readlink: Permission denied) 2024-05-21T21:59:26.518 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86884 mgr-fin ceph NOFD /proc/31856/task/86884/fd (opendir: Permission denied) 2024-05-21T21:59:26.519 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86885 pg_autosc ceph cwd unknown /proc/31856/task/86885/cwd (readlink: Permission denied) 2024-05-21T21:59:26.519 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86885 pg_autosc ceph rtd unknown /proc/31856/task/86885/root (readlink: Permission denied) 2024-05-21T21:59:26.519 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86885 pg_autosc ceph txt unknown /proc/31856/task/86885/exe (readlink: Permission denied) 2024-05-21T21:59:26.519 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86885 pg_autosc ceph NOFD /proc/31856/task/86885/fd (opendir: Permission denied) 2024-05-21T21:59:26.519 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86886 m-fin-pg_ ceph cwd unknown /proc/31856/task/86886/cwd (readlink: Permission denied) 2024-05-21T21:59:26.519 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86886 m-fin-pg_ ceph rtd unknown /proc/31856/task/86886/root (readlink: Permission denied) 2024-05-21T21:59:26.519 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86886 m-fin-pg_ ceph txt unknown /proc/31856/task/86886/exe (readlink: Permission denied) 2024-05-21T21:59:26.519 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86886 m-fin-pg_ ceph NOFD /proc/31856/task/86886/fd (opendir: Permission denied) 2024-05-21T21:59:26.519 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86887 progress ceph cwd unknown /proc/31856/task/86887/cwd (readlink: Permission denied) 2024-05-21T21:59:26.519 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86887 progress ceph rtd unknown /proc/31856/task/86887/root (readlink: Permission denied) 2024-05-21T21:59:26.519 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86887 progress ceph txt unknown /proc/31856/task/86887/exe (readlink: Permission denied) 2024-05-21T21:59:26.519 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86887 progress ceph NOFD /proc/31856/task/86887/fd (opendir: Permission denied) 2024-05-21T21:59:26.519 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86888 m-fin-pro ceph cwd unknown /proc/31856/task/86888/cwd (readlink: Permission denied) 2024-05-21T21:59:26.520 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86888 m-fin-pro ceph rtd unknown /proc/31856/task/86888/root (readlink: Permission denied) 2024-05-21T21:59:26.520 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86888 m-fin-pro ceph txt unknown /proc/31856/task/86888/exe (readlink: Permission denied) 2024-05-21T21:59:26.520 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86888 m-fin-pro ceph NOFD /proc/31856/task/86888/fd (opendir: Permission denied) 2024-05-21T21:59:26.520 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86893 io_contex ceph cwd unknown /proc/31856/task/86893/cwd (readlink: Permission denied) 2024-05-21T21:59:26.520 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86893 io_contex ceph rtd unknown /proc/31856/task/86893/root (readlink: Permission denied) 2024-05-21T21:59:26.520 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86893 io_contex ceph txt unknown /proc/31856/task/86893/exe (readlink: Permission denied) 2024-05-21T21:59:26.520 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86893 io_contex ceph NOFD /proc/31856/task/86893/fd (opendir: Permission denied) 2024-05-21T21:59:26.520 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86894 io_contex ceph cwd unknown /proc/31856/task/86894/cwd (readlink: Permission denied) 2024-05-21T21:59:26.520 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86894 io_contex ceph rtd unknown /proc/31856/task/86894/root (readlink: Permission denied) 2024-05-21T21:59:26.520 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86894 io_contex ceph txt unknown /proc/31856/task/86894/exe (readlink: Permission denied) 2024-05-21T21:59:26.520 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86894 io_contex ceph NOFD /proc/31856/task/86894/fd (opendir: Permission denied) 2024-05-21T21:59:26.520 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86895 ceph_time ceph cwd unknown /proc/31856/task/86895/cwd (readlink: Permission denied) 2024-05-21T21:59:26.520 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86895 ceph_time ceph rtd unknown /proc/31856/task/86895/root (readlink: Permission denied) 2024-05-21T21:59:26.520 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86895 ceph_time ceph txt unknown /proc/31856/task/86895/exe (readlink: Permission denied) 2024-05-21T21:59:26.520 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86895 ceph_time ceph NOFD /proc/31856/task/86895/fd (opendir: Permission denied) 2024-05-21T21:59:26.521 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86896 ms_dispat ceph cwd unknown /proc/31856/task/86896/cwd (readlink: Permission denied) 2024-05-21T21:59:26.521 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86896 ms_dispat ceph rtd unknown /proc/31856/task/86896/root (readlink: Permission denied) 2024-05-21T21:59:26.521 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86896 ms_dispat ceph txt unknown /proc/31856/task/86896/exe (readlink: Permission denied) 2024-05-21T21:59:26.521 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86896 ms_dispat ceph NOFD /proc/31856/task/86896/fd (opendir: Permission denied) 2024-05-21T21:59:26.521 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86897 ms_local ceph cwd unknown /proc/31856/task/86897/cwd (readlink: Permission denied) 2024-05-21T21:59:26.521 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86897 ms_local ceph rtd unknown /proc/31856/task/86897/root (readlink: Permission denied) 2024-05-21T21:59:26.521 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86897 ms_local ceph txt unknown /proc/31856/task/86897/exe (readlink: Permission denied) 2024-05-21T21:59:26.521 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86897 ms_local ceph NOFD /proc/31856/task/86897/fd (opendir: Permission denied) 2024-05-21T21:59:26.521 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86898 safe_time ceph cwd unknown /proc/31856/task/86898/cwd (readlink: Permission denied) 2024-05-21T21:59:26.521 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86898 safe_time ceph rtd unknown /proc/31856/task/86898/root (readlink: Permission denied) 2024-05-21T21:59:26.521 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86898 safe_time ceph txt unknown /proc/31856/task/86898/exe (readlink: Permission denied) 2024-05-21T21:59:26.521 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86898 safe_time ceph NOFD /proc/31856/task/86898/fd (opendir: Permission denied) 2024-05-21T21:59:26.521 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86900 safe_time ceph cwd unknown /proc/31856/task/86900/cwd (readlink: Permission denied) 2024-05-21T21:59:26.521 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86900 safe_time ceph rtd unknown /proc/31856/task/86900/root (readlink: Permission denied) 2024-05-21T21:59:26.521 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86900 safe_time ceph txt unknown /proc/31856/task/86900/exe (readlink: Permission denied) 2024-05-21T21:59:26.522 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86900 safe_time ceph NOFD /proc/31856/task/86900/fd (opendir: Permission denied) 2024-05-21T21:59:26.522 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86901 mgr-fin ceph cwd unknown /proc/31856/task/86901/cwd (readlink: Permission denied) 2024-05-21T21:59:26.522 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86901 mgr-fin ceph rtd unknown /proc/31856/task/86901/root (readlink: Permission denied) 2024-05-21T21:59:26.522 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86901 mgr-fin ceph txt unknown /proc/31856/task/86901/exe (readlink: Permission denied) 2024-05-21T21:59:26.522 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86901 mgr-fin ceph NOFD /proc/31856/task/86901/fd (opendir: Permission denied) 2024-05-21T21:59:26.522 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86911 m-fin-rbd ceph cwd unknown /proc/31856/task/86911/cwd (readlink: Permission denied) 2024-05-21T21:59:26.522 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86911 m-fin-rbd ceph rtd unknown /proc/31856/task/86911/root (readlink: Permission denied) 2024-05-21T21:59:26.522 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86911 m-fin-rbd ceph txt unknown /proc/31856/task/86911/exe (readlink: Permission denied) 2024-05-21T21:59:26.522 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86911 m-fin-rbd ceph NOFD /proc/31856/task/86911/fd (opendir: Permission denied) 2024-05-21T21:59:26.522 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86912 restful ceph cwd unknown /proc/31856/task/86912/cwd (readlink: Permission denied) 2024-05-21T21:59:26.522 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86912 restful ceph rtd unknown /proc/31856/task/86912/root (readlink: Permission denied) 2024-05-21T21:59:26.523 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86912 restful ceph txt unknown /proc/31856/task/86912/exe (readlink: Permission denied) 2024-05-21T21:59:26.523 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86912 restful ceph NOFD /proc/31856/task/86912/fd (opendir: Permission denied) 2024-05-21T21:59:26.523 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86913 m-fin-res ceph cwd unknown /proc/31856/task/86913/cwd (readlink: Permission denied) 2024-05-21T21:59:26.523 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86913 m-fin-res ceph rtd unknown /proc/31856/task/86913/root (readlink: Permission denied) 2024-05-21T21:59:26.523 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86913 m-fin-res ceph txt unknown /proc/31856/task/86913/exe (readlink: Permission denied) 2024-05-21T21:59:26.523 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86913 m-fin-res ceph NOFD /proc/31856/task/86913/fd (opendir: Permission denied) 2024-05-21T21:59:26.523 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86914 mgr-fin ceph cwd unknown /proc/31856/task/86914/cwd (readlink: Permission denied) 2024-05-21T21:59:26.523 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86914 mgr-fin ceph rtd unknown /proc/31856/task/86914/root (readlink: Permission denied) 2024-05-21T21:59:26.523 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86914 mgr-fin ceph txt unknown /proc/31856/task/86914/exe (readlink: Permission denied) 2024-05-21T21:59:26.523 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86914 mgr-fin ceph NOFD /proc/31856/task/86914/fd (opendir: Permission denied) 2024-05-21T21:59:26.523 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86919 mgr-fin ceph cwd unknown /proc/31856/task/86919/cwd (readlink: Permission denied) 2024-05-21T21:59:26.523 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86919 mgr-fin ceph rtd unknown /proc/31856/task/86919/root (readlink: Permission denied) 2024-05-21T21:59:26.523 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86919 mgr-fin ceph txt unknown /proc/31856/task/86919/exe (readlink: Permission denied) 2024-05-21T21:59:26.524 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86919 mgr-fin ceph NOFD /proc/31856/task/86919/fd (opendir: Permission denied) 2024-05-21T21:59:26.524 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86924 mgr-fin ceph cwd unknown /proc/31856/task/86924/cwd (readlink: Permission denied) 2024-05-21T21:59:26.524 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86924 mgr-fin ceph rtd unknown /proc/31856/task/86924/root (readlink: Permission denied) 2024-05-21T21:59:26.524 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86924 mgr-fin ceph txt unknown /proc/31856/task/86924/exe (readlink: Permission denied) 2024-05-21T21:59:26.524 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86924 mgr-fin ceph NOFD /proc/31856/task/86924/fd (opendir: Permission denied) 2024-05-21T21:59:26.524 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86926 mgr-fin ceph cwd unknown /proc/31856/task/86926/cwd (readlink: Permission denied) 2024-05-21T21:59:26.524 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86926 mgr-fin ceph rtd unknown /proc/31856/task/86926/root (readlink: Permission denied) 2024-05-21T21:59:26.524 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86926 mgr-fin ceph txt unknown /proc/31856/task/86926/exe (readlink: Permission denied) 2024-05-21T21:59:26.524 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86926 mgr-fin ceph NOFD /proc/31856/task/86926/fd (opendir: Permission denied) 2024-05-21T21:59:26.524 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86927 mgr-fin ceph cwd unknown /proc/31856/task/86927/cwd (readlink: Permission denied) 2024-05-21T21:59:26.524 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86927 mgr-fin ceph rtd unknown /proc/31856/task/86927/root (readlink: Permission denied) 2024-05-21T21:59:26.524 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86927 mgr-fin ceph txt unknown /proc/31856/task/86927/exe (readlink: Permission denied) 2024-05-21T21:59:26.524 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86927 mgr-fin ceph NOFD /proc/31856/task/86927/fd (opendir: Permission denied) 2024-05-21T21:59:26.524 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86928 mgr-fin ceph cwd unknown /proc/31856/task/86928/cwd (readlink: Permission denied) 2024-05-21T21:59:26.524 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86928 mgr-fin ceph rtd unknown /proc/31856/task/86928/root (readlink: Permission denied) 2024-05-21T21:59:26.525 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86928 mgr-fin ceph txt unknown /proc/31856/task/86928/exe (readlink: Permission denied) 2024-05-21T21:59:26.525 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86928 mgr-fin ceph NOFD /proc/31856/task/86928/fd (opendir: Permission denied) 2024-05-21T21:59:26.525 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86932 io_contex ceph cwd unknown /proc/31856/task/86932/cwd (readlink: Permission denied) 2024-05-21T21:59:26.525 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86932 io_contex ceph rtd unknown /proc/31856/task/86932/root (readlink: Permission denied) 2024-05-21T21:59:26.525 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86932 io_contex ceph txt unknown /proc/31856/task/86932/exe (readlink: Permission denied) 2024-05-21T21:59:26.525 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86932 io_contex ceph NOFD /proc/31856/task/86932/fd (opendir: Permission denied) 2024-05-21T21:59:26.525 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86933 io_contex ceph cwd unknown /proc/31856/task/86933/cwd (readlink: Permission denied) 2024-05-21T21:59:26.525 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86933 io_contex ceph rtd unknown /proc/31856/task/86933/root (readlink: Permission denied) 2024-05-21T21:59:26.525 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86933 io_contex ceph txt unknown /proc/31856/task/86933/exe (readlink: Permission denied) 2024-05-21T21:59:26.525 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86933 io_contex ceph NOFD /proc/31856/task/86933/fd (opendir: Permission denied) 2024-05-21T21:59:26.525 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86934 ceph_time ceph cwd unknown /proc/31856/task/86934/cwd (readlink: Permission denied) 2024-05-21T21:59:26.525 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86934 ceph_time ceph rtd unknown /proc/31856/task/86934/root (readlink: Permission denied) 2024-05-21T21:59:26.525 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86934 ceph_time ceph txt unknown /proc/31856/task/86934/exe (readlink: Permission denied) 2024-05-21T21:59:26.525 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86934 ceph_time ceph NOFD /proc/31856/task/86934/fd (opendir: Permission denied) 2024-05-21T21:59:26.525 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86935 ms_dispat ceph cwd unknown /proc/31856/task/86935/cwd (readlink: Permission denied) 2024-05-21T21:59:26.526 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86935 ms_dispat ceph rtd unknown /proc/31856/task/86935/root (readlink: Permission denied) 2024-05-21T21:59:26.526 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86935 ms_dispat ceph txt unknown /proc/31856/task/86935/exe (readlink: Permission denied) 2024-05-21T21:59:26.526 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86935 ms_dispat ceph NOFD /proc/31856/task/86935/fd (opendir: Permission denied) 2024-05-21T21:59:26.526 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86936 ms_local ceph cwd unknown /proc/31856/task/86936/cwd (readlink: Permission denied) 2024-05-21T21:59:26.526 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86936 ms_local ceph rtd unknown /proc/31856/task/86936/root (readlink: Permission denied) 2024-05-21T21:59:26.526 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86936 ms_local ceph txt unknown /proc/31856/task/86936/exe (readlink: Permission denied) 2024-05-21T21:59:26.526 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86936 ms_local ceph NOFD /proc/31856/task/86936/fd (opendir: Permission denied) 2024-05-21T21:59:26.526 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86937 safe_time ceph cwd unknown /proc/31856/task/86937/cwd (readlink: Permission denied) 2024-05-21T21:59:26.526 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86937 safe_time ceph rtd unknown /proc/31856/task/86937/root (readlink: Permission denied) 2024-05-21T21:59:26.526 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86937 safe_time ceph txt unknown /proc/31856/task/86937/exe (readlink: Permission denied) 2024-05-21T21:59:26.526 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86937 safe_time ceph NOFD /proc/31856/task/86937/fd (opendir: Permission denied) 2024-05-21T21:59:26.526 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86938 safe_time ceph cwd unknown /proc/31856/task/86938/cwd (readlink: Permission denied) 2024-05-21T21:59:26.526 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86938 safe_time ceph rtd unknown /proc/31856/task/86938/root (readlink: Permission denied) 2024-05-21T21:59:26.526 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86938 safe_time ceph txt unknown /proc/31856/task/86938/exe (readlink: Permission denied) 2024-05-21T21:59:26.526 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86938 safe_time ceph NOFD /proc/31856/task/86938/fd (opendir: Permission denied) 2024-05-21T21:59:26.527 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86943 m-fin-sna ceph cwd unknown /proc/31856/task/86943/cwd (readlink: Permission denied) 2024-05-21T21:59:26.527 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86943 m-fin-sna ceph rtd unknown /proc/31856/task/86943/root (readlink: Permission denied) 2024-05-21T21:59:26.527 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86943 m-fin-sna ceph txt unknown /proc/31856/task/86943/exe (readlink: Permission denied) 2024-05-21T21:59:26.527 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86943 m-fin-sna ceph NOFD /proc/31856/task/86943/fd (opendir: Permission denied) 2024-05-21T21:59:26.527 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86945 m-fin-sta ceph cwd unknown /proc/31856/task/86945/cwd (readlink: Permission denied) 2024-05-21T21:59:26.527 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86945 m-fin-sta ceph rtd unknown /proc/31856/task/86945/root (readlink: Permission denied) 2024-05-21T21:59:26.527 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86945 m-fin-sta ceph txt unknown /proc/31856/task/86945/exe (readlink: Permission denied) 2024-05-21T21:59:26.527 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86945 m-fin-sta ceph NOFD /proc/31856/task/86945/fd (opendir: Permission denied) 2024-05-21T21:59:26.527 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86946 telemetry ceph cwd unknown /proc/31856/task/86946/cwd (readlink: Permission denied) 2024-05-21T21:59:26.527 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86946 telemetry ceph rtd unknown /proc/31856/task/86946/root (readlink: Permission denied) 2024-05-21T21:59:26.527 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86946 telemetry ceph txt unknown /proc/31856/task/86946/exe (readlink: Permission denied) 2024-05-21T21:59:26.527 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86946 telemetry ceph NOFD /proc/31856/task/86946/fd (opendir: Permission denied) 2024-05-21T21:59:26.527 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86947 m-fin-tel ceph cwd unknown /proc/31856/task/86947/cwd (readlink: Permission denied) 2024-05-21T21:59:26.528 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86947 m-fin-tel ceph rtd unknown /proc/31856/task/86947/root (readlink: Permission denied) 2024-05-21T21:59:26.528 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86947 m-fin-tel ceph txt unknown /proc/31856/task/86947/exe (readlink: Permission denied) 2024-05-21T21:59:26.528 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86947 m-fin-tel ceph NOFD /proc/31856/task/86947/fd (opendir: Permission denied) 2024-05-21T21:59:26.528 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86948 mgr-fin ceph cwd unknown /proc/31856/task/86948/cwd (readlink: Permission denied) 2024-05-21T21:59:26.528 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86948 mgr-fin ceph rtd unknown /proc/31856/task/86948/root (readlink: Permission denied) 2024-05-21T21:59:26.528 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86948 mgr-fin ceph txt unknown /proc/31856/task/86948/exe (readlink: Permission denied) 2024-05-21T21:59:26.528 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86948 mgr-fin ceph NOFD /proc/31856/task/86948/fd (opendir: Permission denied) 2024-05-21T21:59:26.528 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86952 io_contex ceph cwd unknown /proc/31856/task/86952/cwd (readlink: Permission denied) 2024-05-21T21:59:26.528 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86952 io_contex ceph rtd unknown /proc/31856/task/86952/root (readlink: Permission denied) 2024-05-21T21:59:26.528 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86952 io_contex ceph txt unknown /proc/31856/task/86952/exe (readlink: Permission denied) 2024-05-21T21:59:26.528 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86952 io_contex ceph NOFD /proc/31856/task/86952/fd (opendir: Permission denied) 2024-05-21T21:59:26.528 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86953 io_contex ceph cwd unknown /proc/31856/task/86953/cwd (readlink: Permission denied) 2024-05-21T21:59:26.528 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86953 io_contex ceph rtd unknown /proc/31856/task/86953/root (readlink: Permission denied) 2024-05-21T21:59:26.529 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86953 io_contex ceph txt unknown /proc/31856/task/86953/exe (readlink: Permission denied) 2024-05-21T21:59:26.529 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86953 io_contex ceph NOFD /proc/31856/task/86953/fd (opendir: Permission denied) 2024-05-21T21:59:26.529 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86954 ceph_time ceph cwd unknown /proc/31856/task/86954/cwd (readlink: Permission denied) 2024-05-21T21:59:26.529 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86954 ceph_time ceph rtd unknown /proc/31856/task/86954/root (readlink: Permission denied) 2024-05-21T21:59:26.529 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86954 ceph_time ceph txt unknown /proc/31856/task/86954/exe (readlink: Permission denied) 2024-05-21T21:59:26.529 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86954 ceph_time ceph NOFD /proc/31856/task/86954/fd (opendir: Permission denied) 2024-05-21T21:59:26.529 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86955 ms_dispat ceph cwd unknown /proc/31856/task/86955/cwd (readlink: Permission denied) 2024-05-21T21:59:26.529 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86955 ms_dispat ceph rtd unknown /proc/31856/task/86955/root (readlink: Permission denied) 2024-05-21T21:59:26.529 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86955 ms_dispat ceph txt unknown /proc/31856/task/86955/exe (readlink: Permission denied) 2024-05-21T21:59:26.529 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86955 ms_dispat ceph NOFD /proc/31856/task/86955/fd (opendir: Permission denied) 2024-05-21T21:59:26.529 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86956 ms_local ceph cwd unknown /proc/31856/task/86956/cwd (readlink: Permission denied) 2024-05-21T21:59:26.529 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86956 ms_local ceph rtd unknown /proc/31856/task/86956/root (readlink: Permission denied) 2024-05-21T21:59:26.529 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86956 ms_local ceph txt unknown /proc/31856/task/86956/exe (readlink: Permission denied) 2024-05-21T21:59:26.529 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86956 ms_local ceph NOFD /proc/31856/task/86956/fd (opendir: Permission denied) 2024-05-21T21:59:26.529 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86957 safe_time ceph cwd unknown /proc/31856/task/86957/cwd (readlink: Permission denied) 2024-05-21T21:59:26.530 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86957 safe_time ceph rtd unknown /proc/31856/task/86957/root (readlink: Permission denied) 2024-05-21T21:59:26.530 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86957 safe_time ceph txt unknown /proc/31856/task/86957/exe (readlink: Permission denied) 2024-05-21T21:59:26.530 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86957 safe_time ceph NOFD /proc/31856/task/86957/fd (opendir: Permission denied) 2024-05-21T21:59:26.530 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86958 safe_time ceph cwd unknown /proc/31856/task/86958/cwd (readlink: Permission denied) 2024-05-21T21:59:26.530 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86958 safe_time ceph rtd unknown /proc/31856/task/86958/root (readlink: Permission denied) 2024-05-21T21:59:26.530 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86958 safe_time ceph txt unknown /proc/31856/task/86958/exe (readlink: Permission denied) 2024-05-21T21:59:26.530 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86958 safe_time ceph NOFD /proc/31856/task/86958/fd (opendir: Permission denied) 2024-05-21T21:59:26.530 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86959 mgr-fin ceph cwd unknown /proc/31856/task/86959/cwd (readlink: Permission denied) 2024-05-21T21:59:26.530 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86959 mgr-fin ceph rtd unknown /proc/31856/task/86959/root (readlink: Permission denied) 2024-05-21T21:59:26.530 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86959 mgr-fin ceph txt unknown /proc/31856/task/86959/exe (readlink: Permission denied) 2024-05-21T21:59:26.530 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86959 mgr-fin ceph NOFD /proc/31856/task/86959/fd (opendir: Permission denied) 2024-05-21T21:59:26.530 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86960 mgr-fin ceph cwd unknown /proc/31856/task/86960/cwd (readlink: Permission denied) 2024-05-21T21:59:26.530 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86960 mgr-fin ceph rtd unknown /proc/31856/task/86960/root (readlink: Permission denied) 2024-05-21T21:59:26.530 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86960 mgr-fin ceph txt unknown /proc/31856/task/86960/exe (readlink: Permission denied) 2024-05-21T21:59:26.531 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86960 mgr-fin ceph NOFD /proc/31856/task/86960/fd (opendir: Permission denied) 2024-05-21T21:59:26.531 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86961 mgr-fin ceph cwd unknown /proc/31856/task/86961/cwd (readlink: Permission denied) 2024-05-21T21:59:26.531 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86961 mgr-fin ceph rtd unknown /proc/31856/task/86961/root (readlink: Permission denied) 2024-05-21T21:59:26.531 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86961 mgr-fin ceph txt unknown /proc/31856/task/86961/exe (readlink: Permission denied) 2024-05-21T21:59:26.531 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86961 mgr-fin ceph NOFD /proc/31856/task/86961/fd (opendir: Permission denied) 2024-05-21T21:59:26.531 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86962 mgr-fin ceph cwd unknown /proc/31856/task/86962/cwd (readlink: Permission denied) 2024-05-21T21:59:26.531 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86962 mgr-fin ceph rtd unknown /proc/31856/task/86962/root (readlink: Permission denied) 2024-05-21T21:59:26.531 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86962 mgr-fin ceph txt unknown /proc/31856/task/86962/exe (readlink: Permission denied) 2024-05-21T21:59:26.531 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86962 mgr-fin ceph NOFD /proc/31856/task/86962/fd (opendir: Permission denied) 2024-05-21T21:59:26.531 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86963 mgr-fin ceph cwd unknown /proc/31856/task/86963/cwd (readlink: Permission denied) 2024-05-21T21:59:26.531 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86963 mgr-fin ceph rtd unknown /proc/31856/task/86963/root (readlink: Permission denied) 2024-05-21T21:59:26.531 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86963 mgr-fin ceph txt unknown /proc/31856/task/86963/exe (readlink: Permission denied) 2024-05-21T21:59:26.531 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86963 mgr-fin ceph NOFD /proc/31856/task/86963/fd (opendir: Permission denied) 2024-05-21T21:59:26.531 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86964 mgr-fin ceph cwd unknown /proc/31856/task/86964/cwd (readlink: Permission denied) 2024-05-21T21:59:26.532 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86964 mgr-fin ceph rtd unknown /proc/31856/task/86964/root (readlink: Permission denied) 2024-05-21T21:59:26.532 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86964 mgr-fin ceph txt unknown /proc/31856/task/86964/exe (readlink: Permission denied) 2024-05-21T21:59:26.532 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86964 mgr-fin ceph NOFD /proc/31856/task/86964/fd (opendir: Permission denied) 2024-05-21T21:59:26.532 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86965 mgr-fin ceph cwd unknown /proc/31856/task/86965/cwd (readlink: Permission denied) 2024-05-21T21:59:26.532 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86965 mgr-fin ceph rtd unknown /proc/31856/task/86965/root (readlink: Permission denied) 2024-05-21T21:59:26.532 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86965 mgr-fin ceph txt unknown /proc/31856/task/86965/exe (readlink: Permission denied) 2024-05-21T21:59:26.532 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86965 mgr-fin ceph NOFD /proc/31856/task/86965/fd (opendir: Permission denied) 2024-05-21T21:59:26.532 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86966 mgr-fin ceph cwd unknown /proc/31856/task/86966/cwd (readlink: Permission denied) 2024-05-21T21:59:26.532 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86966 mgr-fin ceph rtd unknown /proc/31856/task/86966/root (readlink: Permission denied) 2024-05-21T21:59:26.532 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86966 mgr-fin ceph txt unknown /proc/31856/task/86966/exe (readlink: Permission denied) 2024-05-21T21:59:26.532 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86966 mgr-fin ceph NOFD /proc/31856/task/86966/fd (opendir: Permission denied) 2024-05-21T21:59:26.532 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86967 mgr-fin ceph cwd unknown /proc/31856/task/86967/cwd (readlink: Permission denied) 2024-05-21T21:59:26.532 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86967 mgr-fin ceph rtd unknown /proc/31856/task/86967/root (readlink: Permission denied) 2024-05-21T21:59:26.532 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86967 mgr-fin ceph txt unknown /proc/31856/task/86967/exe (readlink: Permission denied) 2024-05-21T21:59:26.532 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86967 mgr-fin ceph NOFD /proc/31856/task/86967/fd (opendir: Permission denied) 2024-05-21T21:59:26.533 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86968 mgr-fin ceph cwd unknown /proc/31856/task/86968/cwd (readlink: Permission denied) 2024-05-21T21:59:26.533 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86968 mgr-fin ceph rtd unknown /proc/31856/task/86968/root (readlink: Permission denied) 2024-05-21T21:59:26.533 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86968 mgr-fin ceph txt unknown /proc/31856/task/86968/exe (readlink: Permission denied) 2024-05-21T21:59:26.533 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86968 mgr-fin ceph NOFD /proc/31856/task/86968/fd (opendir: Permission denied) 2024-05-21T21:59:26.533 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86969 mgr-fin ceph cwd unknown /proc/31856/task/86969/cwd (readlink: Permission denied) 2024-05-21T21:59:26.533 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86969 mgr-fin ceph rtd unknown /proc/31856/task/86969/root (readlink: Permission denied) 2024-05-21T21:59:26.533 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86969 mgr-fin ceph txt unknown /proc/31856/task/86969/exe (readlink: Permission denied) 2024-05-21T21:59:26.533 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86969 mgr-fin ceph NOFD /proc/31856/task/86969/fd (opendir: Permission denied) 2024-05-21T21:59:26.533 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86970 mgr-fin ceph cwd unknown /proc/31856/task/86970/cwd (readlink: Permission denied) 2024-05-21T21:59:26.533 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86970 mgr-fin ceph rtd unknown /proc/31856/task/86970/root (readlink: Permission denied) 2024-05-21T21:59:26.533 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86970 mgr-fin ceph txt unknown /proc/31856/task/86970/exe (readlink: Permission denied) 2024-05-21T21:59:26.533 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86970 mgr-fin ceph NOFD /proc/31856/task/86970/fd (opendir: Permission denied) 2024-05-21T21:59:26.533 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86972 m-fin-vol ceph cwd unknown /proc/31856/task/86972/cwd (readlink: Permission denied) 2024-05-21T21:59:26.533 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86972 m-fin-vol ceph rtd unknown /proc/31856/task/86972/root (readlink: Permission denied) 2024-05-21T21:59:26.533 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86972 m-fin-vol ceph txt unknown /proc/31856/task/86972/exe (readlink: Permission denied) 2024-05-21T21:59:26.534 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86972 m-fin-vol ceph NOFD /proc/31856/task/86972/fd (opendir: Permission denied) 2024-05-21T21:59:26.534 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86973 io_contex ceph cwd unknown /proc/31856/task/86973/cwd (readlink: Permission denied) 2024-05-21T21:59:26.534 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86973 io_contex ceph rtd unknown /proc/31856/task/86973/root (readlink: Permission denied) 2024-05-21T21:59:26.534 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86973 io_contex ceph txt unknown /proc/31856/task/86973/exe (readlink: Permission denied) 2024-05-21T21:59:26.534 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86973 io_contex ceph NOFD /proc/31856/task/86973/fd (opendir: Permission denied) 2024-05-21T21:59:26.534 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86974 io_contex ceph cwd unknown /proc/31856/task/86974/cwd (readlink: Permission denied) 2024-05-21T21:59:26.534 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86974 io_contex ceph rtd unknown /proc/31856/task/86974/root (readlink: Permission denied) 2024-05-21T21:59:26.534 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86974 io_contex ceph txt unknown /proc/31856/task/86974/exe (readlink: Permission denied) 2024-05-21T21:59:26.534 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 86974 io_contex ceph NOFD /proc/31856/task/86974/fd (opendir: Permission denied) 2024-05-21T21:59:26.534 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87039 dashboard ceph cwd unknown /proc/31856/task/87039/cwd (readlink: Permission denied) 2024-05-21T21:59:26.534 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87039 dashboard ceph rtd unknown /proc/31856/task/87039/root (readlink: Permission denied) 2024-05-21T21:59:26.534 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87039 dashboard ceph txt unknown /proc/31856/task/87039/exe (readlink: Permission denied) 2024-05-21T21:59:26.534 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87039 dashboard ceph NOFD /proc/31856/task/87039/fd (opendir: Permission denied) 2024-05-21T21:59:26.534 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87040 dashboard ceph cwd unknown /proc/31856/task/87040/cwd (readlink: Permission denied) 2024-05-21T21:59:26.534 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87040 dashboard ceph rtd unknown /proc/31856/task/87040/root (readlink: Permission denied) 2024-05-21T21:59:26.535 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87040 dashboard ceph txt unknown /proc/31856/task/87040/exe (readlink: Permission denied) 2024-05-21T21:59:26.535 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87040 dashboard ceph NOFD /proc/31856/task/87040/fd (opendir: Permission denied) 2024-05-21T21:59:26.535 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87041 dashboard ceph cwd unknown /proc/31856/task/87041/cwd (readlink: Permission denied) 2024-05-21T21:59:26.535 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87041 dashboard ceph rtd unknown /proc/31856/task/87041/root (readlink: Permission denied) 2024-05-21T21:59:26.535 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87041 dashboard ceph txt unknown /proc/31856/task/87041/exe (readlink: Permission denied) 2024-05-21T21:59:26.535 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87041 dashboard ceph NOFD /proc/31856/task/87041/fd (opendir: Permission denied) 2024-05-21T21:59:26.535 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87042 dashboard ceph cwd unknown /proc/31856/task/87042/cwd (readlink: Permission denied) 2024-05-21T21:59:26.535 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87042 dashboard ceph rtd unknown /proc/31856/task/87042/root (readlink: Permission denied) 2024-05-21T21:59:26.535 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87042 dashboard ceph txt unknown /proc/31856/task/87042/exe (readlink: Permission denied) 2024-05-21T21:59:26.535 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87042 dashboard ceph NOFD /proc/31856/task/87042/fd (opendir: Permission denied) 2024-05-21T21:59:26.535 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87043 dashboard ceph cwd unknown /proc/31856/task/87043/cwd (readlink: Permission denied) 2024-05-21T21:59:26.535 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87043 dashboard ceph rtd unknown /proc/31856/task/87043/root (readlink: Permission denied) 2024-05-21T21:59:26.535 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87043 dashboard ceph txt unknown /proc/31856/task/87043/exe (readlink: Permission denied) 2024-05-21T21:59:26.535 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87043 dashboard ceph NOFD /proc/31856/task/87043/fd (opendir: Permission denied) 2024-05-21T21:59:26.535 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87044 dashboard ceph cwd unknown /proc/31856/task/87044/cwd (readlink: Permission denied) 2024-05-21T21:59:26.536 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87044 dashboard ceph rtd unknown /proc/31856/task/87044/root (readlink: Permission denied) 2024-05-21T21:59:26.536 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87044 dashboard ceph txt unknown /proc/31856/task/87044/exe (readlink: Permission denied) 2024-05-21T21:59:26.536 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87044 dashboard ceph NOFD /proc/31856/task/87044/fd (opendir: Permission denied) 2024-05-21T21:59:26.536 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87045 dashboard ceph cwd unknown /proc/31856/task/87045/cwd (readlink: Permission denied) 2024-05-21T21:59:26.536 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87045 dashboard ceph rtd unknown /proc/31856/task/87045/root (readlink: Permission denied) 2024-05-21T21:59:26.536 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87045 dashboard ceph txt unknown /proc/31856/task/87045/exe (readlink: Permission denied) 2024-05-21T21:59:26.536 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87045 dashboard ceph NOFD /proc/31856/task/87045/fd (opendir: Permission denied) 2024-05-21T21:59:26.536 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87046 dashboard ceph cwd unknown /proc/31856/task/87046/cwd (readlink: Permission denied) 2024-05-21T21:59:26.536 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87046 dashboard ceph rtd unknown /proc/31856/task/87046/root (readlink: Permission denied) 2024-05-21T21:59:26.536 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87046 dashboard ceph txt unknown /proc/31856/task/87046/exe (readlink: Permission denied) 2024-05-21T21:59:26.536 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87046 dashboard ceph NOFD /proc/31856/task/87046/fd (opendir: Permission denied) 2024-05-21T21:59:26.536 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87047 dashboard ceph cwd unknown /proc/31856/task/87047/cwd (readlink: Permission denied) 2024-05-21T21:59:26.537 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87047 dashboard ceph rtd unknown /proc/31856/task/87047/root (readlink: Permission denied) 2024-05-21T21:59:26.537 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87047 dashboard ceph txt unknown /proc/31856/task/87047/exe (readlink: Permission denied) 2024-05-21T21:59:26.537 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87047 dashboard ceph NOFD /proc/31856/task/87047/fd (opendir: Permission denied) 2024-05-21T21:59:26.537 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87048 dashboard ceph cwd unknown /proc/31856/task/87048/cwd (readlink: Permission denied) 2024-05-21T21:59:26.537 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87048 dashboard ceph rtd unknown /proc/31856/task/87048/root (readlink: Permission denied) 2024-05-21T21:59:26.537 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87048 dashboard ceph txt unknown /proc/31856/task/87048/exe (readlink: Permission denied) 2024-05-21T21:59:26.537 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87048 dashboard ceph NOFD /proc/31856/task/87048/fd (opendir: Permission denied) 2024-05-21T21:59:26.537 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87049 dashboard ceph cwd unknown /proc/31856/task/87049/cwd (readlink: Permission denied) 2024-05-21T21:59:26.537 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87049 dashboard ceph rtd unknown /proc/31856/task/87049/root (readlink: Permission denied) 2024-05-21T21:59:26.537 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87049 dashboard ceph txt unknown /proc/31856/task/87049/exe (readlink: Permission denied) 2024-05-21T21:59:26.537 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87049 dashboard ceph NOFD /proc/31856/task/87049/fd (opendir: Permission denied) 2024-05-21T21:59:26.537 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87106 dashboard ceph cwd unknown /proc/31856/task/87106/cwd (readlink: Permission denied) 2024-05-21T21:59:26.537 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87106 dashboard ceph rtd unknown /proc/31856/task/87106/root (readlink: Permission denied) 2024-05-21T21:59:26.538 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87106 dashboard ceph txt unknown /proc/31856/task/87106/exe (readlink: Permission denied) 2024-05-21T21:59:26.538 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87106 dashboard ceph NOFD /proc/31856/task/87106/fd (opendir: Permission denied) 2024-05-21T21:59:26.538 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87118 mgr-fin ceph cwd unknown /proc/31856/task/87118/cwd (readlink: Permission denied) 2024-05-21T21:59:26.538 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87118 mgr-fin ceph rtd unknown /proc/31856/task/87118/root (readlink: Permission denied) 2024-05-21T21:59:26.538 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87118 mgr-fin ceph txt unknown /proc/31856/task/87118/exe (readlink: Permission denied) 2024-05-21T21:59:26.538 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87118 mgr-fin ceph NOFD /proc/31856/task/87118/fd (opendir: Permission denied) 2024-05-21T21:59:26.538 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87120 mgr-fin ceph cwd unknown /proc/31856/task/87120/cwd (readlink: Permission denied) 2024-05-21T21:59:26.538 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87120 mgr-fin ceph rtd unknown /proc/31856/task/87120/root (readlink: Permission denied) 2024-05-21T21:59:26.538 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87120 mgr-fin ceph txt unknown /proc/31856/task/87120/exe (readlink: Permission denied) 2024-05-21T21:59:26.538 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87120 mgr-fin ceph NOFD /proc/31856/task/87120/fd (opendir: Permission denied) 2024-05-21T21:59:26.538 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87121 mgr-fin ceph cwd unknown /proc/31856/task/87121/cwd (readlink: Permission denied) 2024-05-21T21:59:26.538 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87121 mgr-fin ceph rtd unknown /proc/31856/task/87121/root (readlink: Permission denied) 2024-05-21T21:59:26.538 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87121 mgr-fin ceph txt unknown /proc/31856/task/87121/exe (readlink: Permission denied) 2024-05-21T21:59:26.538 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87121 mgr-fin ceph NOFD /proc/31856/task/87121/fd (opendir: Permission denied) 2024-05-21T21:59:26.539 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87122 mgr-fin ceph cwd unknown /proc/31856/task/87122/cwd (readlink: Permission denied) 2024-05-21T21:59:26.539 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87122 mgr-fin ceph rtd unknown /proc/31856/task/87122/root (readlink: Permission denied) 2024-05-21T21:59:26.539 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87122 mgr-fin ceph txt unknown /proc/31856/task/87122/exe (readlink: Permission denied) 2024-05-21T21:59:26.539 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87122 mgr-fin ceph NOFD /proc/31856/task/87122/fd (opendir: Permission denied) 2024-05-21T21:59:26.539 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87124 mgr-fin ceph cwd unknown /proc/31856/task/87124/cwd (readlink: Permission denied) 2024-05-21T21:59:26.539 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87124 mgr-fin ceph rtd unknown /proc/31856/task/87124/root (readlink: Permission denied) 2024-05-21T21:59:26.539 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87124 mgr-fin ceph txt unknown /proc/31856/task/87124/exe (readlink: Permission denied) 2024-05-21T21:59:26.539 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87124 mgr-fin ceph NOFD /proc/31856/task/87124/fd (opendir: Permission denied) 2024-05-21T21:59:26.539 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87125 mgr-fin ceph cwd unknown /proc/31856/task/87125/cwd (readlink: Permission denied) 2024-05-21T21:59:26.539 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87125 mgr-fin ceph rtd unknown /proc/31856/task/87125/root (readlink: Permission denied) 2024-05-21T21:59:26.539 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87125 mgr-fin ceph txt unknown /proc/31856/task/87125/exe (readlink: Permission denied) 2024-05-21T21:59:26.539 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87125 mgr-fin ceph NOFD /proc/31856/task/87125/fd (opendir: Permission denied) 2024-05-21T21:59:26.539 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87126 mgr-fin ceph cwd unknown /proc/31856/task/87126/cwd (readlink: Permission denied) 2024-05-21T21:59:26.540 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87126 mgr-fin ceph rtd unknown /proc/31856/task/87126/root (readlink: Permission denied) 2024-05-21T21:59:26.540 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87126 mgr-fin ceph txt unknown /proc/31856/task/87126/exe (readlink: Permission denied) 2024-05-21T21:59:26.540 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87126 mgr-fin ceph NOFD /proc/31856/task/87126/fd (opendir: Permission denied) 2024-05-21T21:59:26.540 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87127 mgr-fin ceph cwd unknown /proc/31856/task/87127/cwd (readlink: Permission denied) 2024-05-21T21:59:26.540 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87127 mgr-fin ceph rtd unknown /proc/31856/task/87127/root (readlink: Permission denied) 2024-05-21T21:59:26.540 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87127 mgr-fin ceph txt unknown /proc/31856/task/87127/exe (readlink: Permission denied) 2024-05-21T21:59:26.540 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87127 mgr-fin ceph NOFD /proc/31856/task/87127/fd (opendir: Permission denied) 2024-05-21T21:59:26.540 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87128 mgr-fin ceph cwd unknown /proc/31856/task/87128/cwd (readlink: Permission denied) 2024-05-21T21:59:26.540 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87128 mgr-fin ceph rtd unknown /proc/31856/task/87128/root (readlink: Permission denied) 2024-05-21T21:59:26.540 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87128 mgr-fin ceph txt unknown /proc/31856/task/87128/exe (readlink: Permission denied) 2024-05-21T21:59:26.540 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87128 mgr-fin ceph NOFD /proc/31856/task/87128/fd (opendir: Permission denied) 2024-05-21T21:59:26.540 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87129 mgr-fin ceph cwd unknown /proc/31856/task/87129/cwd (readlink: Permission denied) 2024-05-21T21:59:26.540 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87129 mgr-fin ceph rtd unknown /proc/31856/task/87129/root (readlink: Permission denied) 2024-05-21T21:59:26.540 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87129 mgr-fin ceph txt unknown /proc/31856/task/87129/exe (readlink: Permission denied) 2024-05-21T21:59:26.540 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87129 mgr-fin ceph NOFD /proc/31856/task/87129/fd (opendir: Permission denied) 2024-05-21T21:59:26.541 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87130 mgr-fin ceph cwd unknown /proc/31856/task/87130/cwd (readlink: Permission denied) 2024-05-21T21:59:26.541 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87130 mgr-fin ceph rtd unknown /proc/31856/task/87130/root (readlink: Permission denied) 2024-05-21T21:59:26.541 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87130 mgr-fin ceph txt unknown /proc/31856/task/87130/exe (readlink: Permission denied) 2024-05-21T21:59:26.541 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87130 mgr-fin ceph NOFD /proc/31856/task/87130/fd (opendir: Permission denied) 2024-05-21T21:59:26.541 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87134 mgr-fin ceph cwd unknown /proc/31856/task/87134/cwd (readlink: Permission denied) 2024-05-21T21:59:26.541 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87134 mgr-fin ceph rtd unknown /proc/31856/task/87134/root (readlink: Permission denied) 2024-05-21T21:59:26.541 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87134 mgr-fin ceph txt unknown /proc/31856/task/87134/exe (readlink: Permission denied) 2024-05-21T21:59:26.541 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87134 mgr-fin ceph NOFD /proc/31856/task/87134/fd (opendir: Permission denied) 2024-05-21T21:59:26.541 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87135 mgr-fin ceph cwd unknown /proc/31856/task/87135/cwd (readlink: Permission denied) 2024-05-21T21:59:26.541 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87135 mgr-fin ceph rtd unknown /proc/31856/task/87135/root (readlink: Permission denied) 2024-05-21T21:59:26.541 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87135 mgr-fin ceph txt unknown /proc/31856/task/87135/exe (readlink: Permission denied) 2024-05-21T21:59:26.541 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87135 mgr-fin ceph NOFD /proc/31856/task/87135/fd (opendir: Permission denied) 2024-05-21T21:59:26.541 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87136 mgr-fin ceph cwd unknown /proc/31856/task/87136/cwd (readlink: Permission denied) 2024-05-21T21:59:26.541 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87136 mgr-fin ceph rtd unknown /proc/31856/task/87136/root (readlink: Permission denied) 2024-05-21T21:59:26.542 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87136 mgr-fin ceph txt unknown /proc/31856/task/87136/exe (readlink: Permission denied) 2024-05-21T21:59:26.542 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87136 mgr-fin ceph NOFD /proc/31856/task/87136/fd (opendir: Permission denied) 2024-05-21T21:59:26.542 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87137 mgr-fin ceph cwd unknown /proc/31856/task/87137/cwd (readlink: Permission denied) 2024-05-21T21:59:26.542 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87137 mgr-fin ceph rtd unknown /proc/31856/task/87137/root (readlink: Permission denied) 2024-05-21T21:59:26.542 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87137 mgr-fin ceph txt unknown /proc/31856/task/87137/exe (readlink: Permission denied) 2024-05-21T21:59:26.542 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87137 mgr-fin ceph NOFD /proc/31856/task/87137/fd (opendir: Permission denied) 2024-05-21T21:59:26.542 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87138 mgr-fin ceph cwd unknown /proc/31856/task/87138/cwd (readlink: Permission denied) 2024-05-21T21:59:26.542 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87138 mgr-fin ceph rtd unknown /proc/31856/task/87138/root (readlink: Permission denied) 2024-05-21T21:59:26.542 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87138 mgr-fin ceph txt unknown /proc/31856/task/87138/exe (readlink: Permission denied) 2024-05-21T21:59:26.542 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87138 mgr-fin ceph NOFD /proc/31856/task/87138/fd (opendir: Permission denied) 2024-05-21T21:59:26.542 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87139 mgr-fin ceph cwd unknown /proc/31856/task/87139/cwd (readlink: Permission denied) 2024-05-21T21:59:26.542 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87139 mgr-fin ceph rtd unknown /proc/31856/task/87139/root (readlink: Permission denied) 2024-05-21T21:59:26.542 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87139 mgr-fin ceph txt unknown /proc/31856/task/87139/exe (readlink: Permission denied) 2024-05-21T21:59:26.542 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87139 mgr-fin ceph NOFD /proc/31856/task/87139/fd (opendir: Permission denied) 2024-05-21T21:59:26.543 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87140 mgr-fin ceph cwd unknown /proc/31856/task/87140/cwd (readlink: Permission denied) 2024-05-21T21:59:26.543 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87140 mgr-fin ceph rtd unknown /proc/31856/task/87140/root (readlink: Permission denied) 2024-05-21T21:59:26.543 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87140 mgr-fin ceph txt unknown /proc/31856/task/87140/exe (readlink: Permission denied) 2024-05-21T21:59:26.543 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87140 mgr-fin ceph NOFD /proc/31856/task/87140/fd (opendir: Permission denied) 2024-05-21T21:59:26.543 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87141 mgr-fin ceph cwd unknown /proc/31856/task/87141/cwd (readlink: Permission denied) 2024-05-21T21:59:26.543 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87141 mgr-fin ceph rtd unknown /proc/31856/task/87141/root (readlink: Permission denied) 2024-05-21T21:59:26.543 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87141 mgr-fin ceph txt unknown /proc/31856/task/87141/exe (readlink: Permission denied) 2024-05-21T21:59:26.543 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87141 mgr-fin ceph NOFD /proc/31856/task/87141/fd (opendir: Permission denied) 2024-05-21T21:59:26.543 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87142 mgr-fin ceph cwd unknown /proc/31856/task/87142/cwd (readlink: Permission denied) 2024-05-21T21:59:26.543 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87142 mgr-fin ceph rtd unknown /proc/31856/task/87142/root (readlink: Permission denied) 2024-05-21T21:59:26.543 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87142 mgr-fin ceph txt unknown /proc/31856/task/87142/exe (readlink: Permission denied) 2024-05-21T21:59:26.543 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87142 mgr-fin ceph NOFD /proc/31856/task/87142/fd (opendir: Permission denied) 2024-05-21T21:59:26.543 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87143 mgr-fin ceph cwd unknown /proc/31856/task/87143/cwd (readlink: Permission denied) 2024-05-21T21:59:26.543 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87143 mgr-fin ceph rtd unknown /proc/31856/task/87143/root (readlink: Permission denied) 2024-05-21T21:59:26.543 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87143 mgr-fin ceph txt unknown /proc/31856/task/87143/exe (readlink: Permission denied) 2024-05-21T21:59:26.544 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87143 mgr-fin ceph NOFD /proc/31856/task/87143/fd (opendir: Permission denied) 2024-05-21T21:59:26.544 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87144 mgr-fin ceph cwd unknown /proc/31856/task/87144/cwd (readlink: Permission denied) 2024-05-21T21:59:26.544 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87144 mgr-fin ceph rtd unknown /proc/31856/task/87144/root (readlink: Permission denied) 2024-05-21T21:59:26.544 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87144 mgr-fin ceph txt unknown /proc/31856/task/87144/exe (readlink: Permission denied) 2024-05-21T21:59:26.544 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87144 mgr-fin ceph NOFD /proc/31856/task/87144/fd (opendir: Permission denied) 2024-05-21T21:59:26.544 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87345 devicehea ceph cwd unknown /proc/31856/task/87345/cwd (readlink: Permission denied) 2024-05-21T21:59:26.544 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87345 devicehea ceph rtd unknown /proc/31856/task/87345/root (readlink: Permission denied) 2024-05-21T21:59:26.544 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87345 devicehea ceph txt unknown /proc/31856/task/87345/exe (readlink: Permission denied) 2024-05-21T21:59:26.544 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87345 devicehea ceph NOFD /proc/31856/task/87345/fd (opendir: Permission denied) 2024-05-21T21:59:26.544 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87349 io_contex ceph cwd unknown /proc/31856/task/87349/cwd (readlink: Permission denied) 2024-05-21T21:59:26.544 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87349 io_contex ceph rtd unknown /proc/31856/task/87349/root (readlink: Permission denied) 2024-05-21T21:59:26.544 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87349 io_contex ceph txt unknown /proc/31856/task/87349/exe (readlink: Permission denied) 2024-05-21T21:59:26.544 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87349 io_contex ceph NOFD /proc/31856/task/87349/fd (opendir: Permission denied) 2024-05-21T21:59:26.544 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87350 io_contex ceph cwd unknown /proc/31856/task/87350/cwd (readlink: Permission denied) 2024-05-21T21:59:26.545 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87350 io_contex ceph rtd unknown /proc/31856/task/87350/root (readlink: Permission denied) 2024-05-21T21:59:26.545 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87350 io_contex ceph txt unknown /proc/31856/task/87350/exe (readlink: Permission denied) 2024-05-21T21:59:26.545 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87350 io_contex ceph NOFD /proc/31856/task/87350/fd (opendir: Permission denied) 2024-05-21T21:59:26.545 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87351 ceph_time ceph cwd unknown /proc/31856/task/87351/cwd (readlink: Permission denied) 2024-05-21T21:59:26.545 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87351 ceph_time ceph rtd unknown /proc/31856/task/87351/root (readlink: Permission denied) 2024-05-21T21:59:26.545 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87351 ceph_time ceph txt unknown /proc/31856/task/87351/exe (readlink: Permission denied) 2024-05-21T21:59:26.545 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87351 ceph_time ceph NOFD /proc/31856/task/87351/fd (opendir: Permission denied) 2024-05-21T21:59:26.545 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87352 ms_dispat ceph cwd unknown /proc/31856/task/87352/cwd (readlink: Permission denied) 2024-05-21T21:59:26.545 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87352 ms_dispat ceph rtd unknown /proc/31856/task/87352/root (readlink: Permission denied) 2024-05-21T21:59:26.545 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87352 ms_dispat ceph txt unknown /proc/31856/task/87352/exe (readlink: Permission denied) 2024-05-21T21:59:26.545 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87352 ms_dispat ceph NOFD /proc/31856/task/87352/fd (opendir: Permission denied) 2024-05-21T21:59:26.545 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87353 ms_local ceph cwd unknown /proc/31856/task/87353/cwd (readlink: Permission denied) 2024-05-21T21:59:26.546 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87353 ms_local ceph rtd unknown /proc/31856/task/87353/root (readlink: Permission denied) 2024-05-21T21:59:26.546 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87353 ms_local ceph txt unknown /proc/31856/task/87353/exe (readlink: Permission denied) 2024-05-21T21:59:26.546 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87353 ms_local ceph NOFD /proc/31856/task/87353/fd (opendir: Permission denied) 2024-05-21T21:59:26.546 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87354 safe_time ceph cwd unknown /proc/31856/task/87354/cwd (readlink: Permission denied) 2024-05-21T21:59:26.546 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87354 safe_time ceph rtd unknown /proc/31856/task/87354/root (readlink: Permission denied) 2024-05-21T21:59:26.546 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87354 safe_time ceph txt unknown /proc/31856/task/87354/exe (readlink: Permission denied) 2024-05-21T21:59:26.546 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87354 safe_time ceph NOFD /proc/31856/task/87354/fd (opendir: Permission denied) 2024-05-21T21:59:26.546 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87355 safe_time ceph cwd unknown /proc/31856/task/87355/cwd (readlink: Permission denied) 2024-05-21T21:59:26.546 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87355 safe_time ceph rtd unknown /proc/31856/task/87355/root (readlink: Permission denied) 2024-05-21T21:59:26.546 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87355 safe_time ceph txt unknown /proc/31856/task/87355/exe (readlink: Permission denied) 2024-05-21T21:59:26.546 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 87355 safe_time ceph NOFD /proc/31856/task/87355/fd (opendir: Permission denied) 2024-05-21T21:59:26.546 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 90112 ceph_time ceph cwd unknown /proc/31856/task/90112/cwd (readlink: Permission denied) 2024-05-21T21:59:26.546 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 90112 ceph_time ceph rtd unknown /proc/31856/task/90112/root (readlink: Permission denied) 2024-05-21T21:59:26.546 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 90112 ceph_time ceph txt unknown /proc/31856/task/90112/exe (readlink: Permission denied) 2024-05-21T21:59:26.547 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 90112 ceph_time ceph NOFD /proc/31856/task/90112/fd (opendir: Permission denied) 2024-05-21T21:59:26.547 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 90113 fn_anonym ceph cwd unknown /proc/31856/task/90113/cwd (readlink: Permission denied) 2024-05-21T21:59:26.547 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 90113 fn_anonym ceph rtd unknown /proc/31856/task/90113/root (readlink: Permission denied) 2024-05-21T21:59:26.547 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 90113 fn_anonym ceph txt unknown /proc/31856/task/90113/exe (readlink: Permission denied) 2024-05-21T21:59:26.547 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 90113 fn_anonym ceph NOFD /proc/31856/task/90113/fd (opendir: Permission denied) 2024-05-21T21:59:26.547 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 90114 safe_time ceph cwd unknown /proc/31856/task/90114/cwd (readlink: Permission denied) 2024-05-21T21:59:26.547 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 90114 safe_time ceph rtd unknown /proc/31856/task/90114/root (readlink: Permission denied) 2024-05-21T21:59:26.547 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 90114 safe_time ceph txt unknown /proc/31856/task/90114/exe (readlink: Permission denied) 2024-05-21T21:59:26.547 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 90114 safe_time ceph NOFD /proc/31856/task/90114/fd (opendir: Permission denied) 2024-05-21T21:59:26.547 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 90115 fn_anonym ceph cwd unknown /proc/31856/task/90115/cwd (readlink: Permission denied) 2024-05-21T21:59:26.547 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 90115 fn_anonym ceph rtd unknown /proc/31856/task/90115/root (readlink: Permission denied) 2024-05-21T21:59:26.547 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 90115 fn_anonym ceph txt unknown /proc/31856/task/90115/exe (readlink: Permission denied) 2024-05-21T21:59:26.547 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 90115 fn_anonym ceph NOFD /proc/31856/task/90115/fd (opendir: Permission denied) 2024-05-21T21:59:26.547 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 90116 flusher ceph cwd unknown /proc/31856/task/90116/cwd (readlink: Permission denied) 2024-05-21T21:59:26.548 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 90116 flusher ceph rtd unknown /proc/31856/task/90116/root (readlink: Permission denied) 2024-05-21T21:59:26.548 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 90116 flusher ceph txt unknown /proc/31856/task/90116/exe (readlink: Permission denied) 2024-05-21T21:59:26.548 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 90116 flusher ceph NOFD /proc/31856/task/90116/fd (opendir: Permission denied) 2024-05-21T21:59:26.548 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 90117 ms_dispat ceph cwd unknown /proc/31856/task/90117/cwd (readlink: Permission denied) 2024-05-21T21:59:26.548 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 90117 ms_dispat ceph rtd unknown /proc/31856/task/90117/root (readlink: Permission denied) 2024-05-21T21:59:26.548 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 90117 ms_dispat ceph txt unknown /proc/31856/task/90117/exe (readlink: Permission denied) 2024-05-21T21:59:26.548 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 90117 ms_dispat ceph NOFD /proc/31856/task/90117/fd (opendir: Permission denied) 2024-05-21T21:59:26.548 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 90118 ms_local ceph cwd unknown /proc/31856/task/90118/cwd (readlink: Permission denied) 2024-05-21T21:59:26.548 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 90118 ms_local ceph rtd unknown /proc/31856/task/90118/root (readlink: Permission denied) 2024-05-21T21:59:26.548 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 90118 ms_local ceph txt unknown /proc/31856/task/90118/exe (readlink: Permission denied) 2024-05-21T21:59:26.548 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 90118 ms_local ceph NOFD /proc/31856/task/90118/fd (opendir: Permission denied) 2024-05-21T21:59:26.548 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 90119 safe_time ceph cwd unknown /proc/31856/task/90119/cwd (readlink: Permission denied) 2024-05-21T21:59:26.548 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 90119 safe_time ceph rtd unknown /proc/31856/task/90119/root (readlink: Permission denied) 2024-05-21T21:59:26.548 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 90119 safe_time ceph txt unknown /proc/31856/task/90119/exe (readlink: Permission denied) 2024-05-21T21:59:26.548 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 90119 safe_time ceph NOFD /proc/31856/task/90119/fd (opendir: Permission denied) 2024-05-21T21:59:26.549 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 90121 m-fin-sna ceph cwd unknown /proc/31856/task/90121/cwd (readlink: Permission denied) 2024-05-21T21:59:26.549 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 90121 m-fin-sna ceph rtd unknown /proc/31856/task/90121/root (readlink: Permission denied) 2024-05-21T21:59:26.549 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 90121 m-fin-sna ceph txt unknown /proc/31856/task/90121/exe (readlink: Permission denied) 2024-05-21T21:59:26.549 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 90121 m-fin-sna ceph NOFD /proc/31856/task/90121/fd (opendir: Permission denied) 2024-05-21T21:59:26.549 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 99658 m-fin-sna ceph cwd unknown /proc/31856/task/99658/cwd (readlink: Permission denied) 2024-05-21T21:59:26.549 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 99658 m-fin-sna ceph rtd unknown /proc/31856/task/99658/root (readlink: Permission denied) 2024-05-21T21:59:26.549 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 99658 m-fin-sna ceph txt unknown /proc/31856/task/99658/exe (readlink: Permission denied) 2024-05-21T21:59:26.549 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mgr 31856 99658 m-fin-sna ceph NOFD /proc/31856/task/99658/fd (opendir: Permission denied) 2024-05-21T21:59:26.549 INFO:tasks.workunit.client.0.smithi151.stdout:systemd 33002 root cwd unknown /proc/33002/cwd (readlink: Permission denied) 2024-05-21T21:59:26.549 INFO:tasks.workunit.client.0.smithi151.stdout:systemd 33002 root rtd unknown /proc/33002/root (readlink: Permission denied) 2024-05-21T21:59:26.549 INFO:tasks.workunit.client.0.smithi151.stdout:systemd 33002 root txt unknown /proc/33002/exe (readlink: Permission denied) 2024-05-21T21:59:26.549 INFO:tasks.workunit.client.0.smithi151.stdout:systemd 33002 root NOFD /proc/33002/fd (opendir: Permission denied) 2024-05-21T21:59:26.549 INFO:tasks.workunit.client.0.smithi151.stdout:(sd-pam) 33004 root cwd unknown /proc/33004/cwd (readlink: Permission denied) 2024-05-21T21:59:26.549 INFO:tasks.workunit.client.0.smithi151.stdout:(sd-pam) 33004 root rtd unknown /proc/33004/root (readlink: Permission denied) 2024-05-21T21:59:26.549 INFO:tasks.workunit.client.0.smithi151.stdout:(sd-pam) 33004 root txt unknown /proc/33004/exe (readlink: Permission denied) 2024-05-21T21:59:26.550 INFO:tasks.workunit.client.0.smithi151.stdout:(sd-pam) 33004 root NOFD /proc/33004/fd (opendir: Permission denied) 2024-05-21T21:59:26.550 INFO:tasks.workunit.client.0.smithi151.stdout:conmon 44060 root cwd unknown /proc/44060/cwd (readlink: Permission denied) 2024-05-21T21:59:26.550 INFO:tasks.workunit.client.0.smithi151.stdout:conmon 44060 root rtd unknown /proc/44060/root (readlink: Permission denied) 2024-05-21T21:59:26.550 INFO:tasks.workunit.client.0.smithi151.stdout:conmon 44060 root txt unknown /proc/44060/exe (readlink: Permission denied) 2024-05-21T21:59:26.550 INFO:tasks.workunit.client.0.smithi151.stdout:conmon 44060 root NOFD /proc/44060/fd (opendir: Permission denied) 2024-05-21T21:59:26.550 INFO:tasks.workunit.client.0.smithi151.stdout:podman-in 44062 root cwd unknown /proc/44062/cwd (readlink: Permission denied) 2024-05-21T21:59:26.550 INFO:tasks.workunit.client.0.smithi151.stdout:podman-in 44062 root rtd unknown /proc/44062/root (readlink: Permission denied) 2024-05-21T21:59:26.550 INFO:tasks.workunit.client.0.smithi151.stdout:podman-in 44062 root txt unknown /proc/44062/exe (readlink: Permission denied) 2024-05-21T21:59:26.550 INFO:tasks.workunit.client.0.smithi151.stdout:podman-in 44062 root NOFD /proc/44062/fd (opendir: Permission denied) 2024-05-21T21:59:26.550 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 ceph cwd unknown /proc/44064/cwd (readlink: Permission denied) 2024-05-21T21:59:26.550 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 ceph rtd unknown /proc/44064/root (readlink: Permission denied) 2024-05-21T21:59:26.550 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 ceph txt unknown /proc/44064/exe (readlink: Permission denied) 2024-05-21T21:59:26.550 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 ceph NOFD /proc/44064/fd (opendir: Permission denied) 2024-05-21T21:59:26.551 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44065 log ceph cwd unknown /proc/44064/task/44065/cwd (readlink: Permission denied) 2024-05-21T21:59:26.551 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44065 log ceph rtd unknown /proc/44064/task/44065/root (readlink: Permission denied) 2024-05-21T21:59:26.551 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44065 log ceph txt unknown /proc/44064/task/44065/exe (readlink: Permission denied) 2024-05-21T21:59:26.551 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44065 log ceph NOFD /proc/44064/task/44065/fd (opendir: Permission denied) 2024-05-21T21:59:26.551 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44067 msgr-work ceph cwd unknown /proc/44064/task/44067/cwd (readlink: Permission denied) 2024-05-21T21:59:26.551 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44067 msgr-work ceph rtd unknown /proc/44064/task/44067/root (readlink: Permission denied) 2024-05-21T21:59:26.551 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44067 msgr-work ceph txt unknown /proc/44064/task/44067/exe (readlink: Permission denied) 2024-05-21T21:59:26.551 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44067 msgr-work ceph NOFD /proc/44064/task/44067/fd (opendir: Permission denied) 2024-05-21T21:59:26.551 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44068 msgr-work ceph cwd unknown /proc/44064/task/44068/cwd (readlink: Permission denied) 2024-05-21T21:59:26.551 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44068 msgr-work ceph rtd unknown /proc/44064/task/44068/root (readlink: Permission denied) 2024-05-21T21:59:26.551 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44068 msgr-work ceph txt unknown /proc/44064/task/44068/exe (readlink: Permission denied) 2024-05-21T21:59:26.551 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44068 msgr-work ceph NOFD /proc/44064/task/44068/fd (opendir: Permission denied) 2024-05-21T21:59:26.551 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44069 msgr-work ceph cwd unknown /proc/44064/task/44069/cwd (readlink: Permission denied) 2024-05-21T21:59:26.552 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44069 msgr-work ceph rtd unknown /proc/44064/task/44069/root (readlink: Permission denied) 2024-05-21T21:59:26.552 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44069 msgr-work ceph txt unknown /proc/44064/task/44069/exe (readlink: Permission denied) 2024-05-21T21:59:26.552 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44069 msgr-work ceph NOFD /proc/44064/task/44069/fd (opendir: Permission denied) 2024-05-21T21:59:26.552 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44073 service ceph cwd unknown /proc/44064/task/44073/cwd (readlink: Permission denied) 2024-05-21T21:59:26.552 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44073 service ceph rtd unknown /proc/44064/task/44073/root (readlink: Permission denied) 2024-05-21T21:59:26.552 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44073 service ceph txt unknown /proc/44064/task/44073/exe (readlink: Permission denied) 2024-05-21T21:59:26.552 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44073 service ceph NOFD /proc/44064/task/44073/fd (opendir: Permission denied) 2024-05-21T21:59:26.552 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44074 admin_soc ceph cwd unknown /proc/44064/task/44074/cwd (readlink: Permission denied) 2024-05-21T21:59:26.552 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44074 admin_soc ceph rtd unknown /proc/44064/task/44074/root (readlink: Permission denied) 2024-05-21T21:59:26.552 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44074 admin_soc ceph txt unknown /proc/44064/task/44074/exe (readlink: Permission denied) 2024-05-21T21:59:26.552 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44074 admin_soc ceph NOFD /proc/44064/task/44074/fd (opendir: Permission denied) 2024-05-21T21:59:26.552 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44217 signal_ha ceph cwd unknown /proc/44064/task/44217/cwd (readlink: Permission denied) 2024-05-21T21:59:26.552 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44217 signal_ha ceph rtd unknown /proc/44064/task/44217/root (readlink: Permission denied) 2024-05-21T21:59:26.553 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44217 signal_ha ceph txt unknown /proc/44064/task/44217/exe (readlink: Permission denied) 2024-05-21T21:59:26.553 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44217 signal_ha ceph NOFD /proc/44064/task/44217/fd (opendir: Permission denied) 2024-05-21T21:59:26.553 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44218 io_contex ceph cwd unknown /proc/44064/task/44218/cwd (readlink: Permission denied) 2024-05-21T21:59:26.553 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44218 io_contex ceph rtd unknown /proc/44064/task/44218/root (readlink: Permission denied) 2024-05-21T21:59:26.553 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44218 io_contex ceph txt unknown /proc/44064/task/44218/exe (readlink: Permission denied) 2024-05-21T21:59:26.553 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44218 io_contex ceph NOFD /proc/44064/task/44218/fd (opendir: Permission denied) 2024-05-21T21:59:26.553 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44219 io_contex ceph cwd unknown /proc/44064/task/44219/cwd (readlink: Permission denied) 2024-05-21T21:59:26.553 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44219 io_contex ceph rtd unknown /proc/44064/task/44219/root (readlink: Permission denied) 2024-05-21T21:59:26.553 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44219 io_contex ceph txt unknown /proc/44064/task/44219/exe (readlink: Permission denied) 2024-05-21T21:59:26.553 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44219 io_contex ceph NOFD /proc/44064/task/44219/fd (opendir: Permission denied) 2024-05-21T21:59:26.553 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44243 OpHistory ceph cwd unknown /proc/44064/task/44243/cwd (readlink: Permission denied) 2024-05-21T21:59:26.553 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44243 OpHistory ceph rtd unknown /proc/44064/task/44243/root (readlink: Permission denied) 2024-05-21T21:59:26.553 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44243 OpHistory ceph txt unknown /proc/44064/task/44243/exe (readlink: Permission denied) 2024-05-21T21:59:26.553 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44243 OpHistory ceph NOFD /proc/44064/task/44243/fd (opendir: Permission denied) 2024-05-21T21:59:26.553 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44244 ceph_time ceph cwd unknown /proc/44064/task/44244/cwd (readlink: Permission denied) 2024-05-21T21:59:26.554 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44244 ceph_time ceph rtd unknown /proc/44064/task/44244/root (readlink: Permission denied) 2024-05-21T21:59:26.554 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44244 ceph_time ceph txt unknown /proc/44064/task/44244/exe (readlink: Permission denied) 2024-05-21T21:59:26.554 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44244 ceph_time ceph NOFD /proc/44064/task/44244/fd (opendir: Permission denied) 2024-05-21T21:59:26.554 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44329 ceph-osd ceph cwd unknown /proc/44064/task/44329/cwd (readlink: Permission denied) 2024-05-21T21:59:26.554 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44329 ceph-osd ceph rtd unknown /proc/44064/task/44329/root (readlink: Permission denied) 2024-05-21T21:59:26.554 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44329 ceph-osd ceph txt unknown /proc/44064/task/44329/exe (readlink: Permission denied) 2024-05-21T21:59:26.554 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44329 ceph-osd ceph NOFD /proc/44064/task/44329/fd (opendir: Permission denied) 2024-05-21T21:59:26.554 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44351 ceph-osd ceph cwd unknown /proc/44064/task/44351/cwd (readlink: Permission denied) 2024-05-21T21:59:26.554 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44351 ceph-osd ceph rtd unknown /proc/44064/task/44351/root (readlink: Permission denied) 2024-05-21T21:59:26.554 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44351 ceph-osd ceph txt unknown /proc/44064/task/44351/exe (readlink: Permission denied) 2024-05-21T21:59:26.554 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44351 ceph-osd ceph NOFD /proc/44064/task/44351/fd (opendir: Permission denied) 2024-05-21T21:59:26.554 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44382 ceph-osd ceph cwd unknown /proc/44064/task/44382/cwd (readlink: Permission denied) 2024-05-21T21:59:26.554 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44382 ceph-osd ceph rtd unknown /proc/44064/task/44382/root (readlink: Permission denied) 2024-05-21T21:59:26.554 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44382 ceph-osd ceph txt unknown /proc/44064/task/44382/exe (readlink: Permission denied) 2024-05-21T21:59:26.555 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44382 ceph-osd ceph NOFD /proc/44064/task/44382/fd (opendir: Permission denied) 2024-05-21T21:59:26.555 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44390 ceph-osd ceph cwd unknown /proc/44064/task/44390/cwd (readlink: Permission denied) 2024-05-21T21:59:26.555 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44390 ceph-osd ceph rtd unknown /proc/44064/task/44390/root (readlink: Permission denied) 2024-05-21T21:59:26.555 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44390 ceph-osd ceph txt unknown /proc/44064/task/44390/exe (readlink: Permission denied) 2024-05-21T21:59:26.555 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44390 ceph-osd ceph NOFD /proc/44064/task/44390/fd (opendir: Permission denied) 2024-05-21T21:59:26.555 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44413 ceph-osd ceph cwd unknown /proc/44064/task/44413/cwd (readlink: Permission denied) 2024-05-21T21:59:26.555 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44413 ceph-osd ceph rtd unknown /proc/44064/task/44413/root (readlink: Permission denied) 2024-05-21T21:59:26.555 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44413 ceph-osd ceph txt unknown /proc/44064/task/44413/exe (readlink: Permission denied) 2024-05-21T21:59:26.555 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44413 ceph-osd ceph NOFD /proc/44064/task/44413/fd (opendir: Permission denied) 2024-05-21T21:59:26.555 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44419 ceph-osd ceph cwd unknown /proc/44064/task/44419/cwd (readlink: Permission denied) 2024-05-21T21:59:26.555 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44419 ceph-osd ceph rtd unknown /proc/44064/task/44419/root (readlink: Permission denied) 2024-05-21T21:59:26.555 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44419 ceph-osd ceph txt unknown /proc/44064/task/44419/exe (readlink: Permission denied) 2024-05-21T21:59:26.555 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44419 ceph-osd ceph NOFD /proc/44064/task/44419/fd (opendir: Permission denied) 2024-05-21T21:59:26.555 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44430 ceph-osd ceph cwd unknown /proc/44064/task/44430/cwd (readlink: Permission denied) 2024-05-21T21:59:26.555 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44430 ceph-osd ceph rtd unknown /proc/44064/task/44430/root (readlink: Permission denied) 2024-05-21T21:59:26.556 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44430 ceph-osd ceph txt unknown /proc/44064/task/44430/exe (readlink: Permission denied) 2024-05-21T21:59:26.556 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44430 ceph-osd ceph NOFD /proc/44064/task/44430/fd (opendir: Permission denied) 2024-05-21T21:59:26.556 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44478 ceph-osd ceph cwd unknown /proc/44064/task/44478/cwd (readlink: Permission denied) 2024-05-21T21:59:26.556 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44478 ceph-osd ceph rtd unknown /proc/44064/task/44478/root (readlink: Permission denied) 2024-05-21T21:59:26.556 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44478 ceph-osd ceph txt unknown /proc/44064/task/44478/exe (readlink: Permission denied) 2024-05-21T21:59:26.556 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44478 ceph-osd ceph NOFD /proc/44064/task/44478/fd (opendir: Permission denied) 2024-05-21T21:59:26.556 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44479 ceph-osd ceph cwd unknown /proc/44064/task/44479/cwd (readlink: Permission denied) 2024-05-21T21:59:26.556 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44479 ceph-osd ceph rtd unknown /proc/44064/task/44479/root (readlink: Permission denied) 2024-05-21T21:59:26.556 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44479 ceph-osd ceph txt unknown /proc/44064/task/44479/exe (readlink: Permission denied) 2024-05-21T21:59:26.556 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44479 ceph-osd ceph NOFD /proc/44064/task/44479/fd (opendir: Permission denied) 2024-05-21T21:59:26.556 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44480 safe_time ceph cwd unknown /proc/44064/task/44480/cwd (readlink: Permission denied) 2024-05-21T21:59:26.556 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44480 safe_time ceph rtd unknown /proc/44064/task/44480/root (readlink: Permission denied) 2024-05-21T21:59:26.556 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44480 safe_time ceph txt unknown /proc/44064/task/44480/exe (readlink: Permission denied) 2024-05-21T21:59:26.556 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44480 safe_time ceph NOFD /proc/44064/task/44480/fd (opendir: Permission denied) 2024-05-21T21:59:26.557 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44481 safe_time ceph cwd unknown /proc/44064/task/44481/cwd (readlink: Permission denied) 2024-05-21T21:59:26.557 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44481 safe_time ceph rtd unknown /proc/44064/task/44481/root (readlink: Permission denied) 2024-05-21T21:59:26.557 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44481 safe_time ceph txt unknown /proc/44064/task/44481/exe (readlink: Permission denied) 2024-05-21T21:59:26.557 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44481 safe_time ceph NOFD /proc/44064/task/44481/fd (opendir: Permission denied) 2024-05-21T21:59:26.557 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44482 safe_time ceph cwd unknown /proc/44064/task/44482/cwd (readlink: Permission denied) 2024-05-21T21:59:26.557 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44482 safe_time ceph rtd unknown /proc/44064/task/44482/root (readlink: Permission denied) 2024-05-21T21:59:26.557 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44482 safe_time ceph txt unknown /proc/44064/task/44482/exe (readlink: Permission denied) 2024-05-21T21:59:26.557 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44482 safe_time ceph NOFD /proc/44064/task/44482/fd (opendir: Permission denied) 2024-05-21T21:59:26.557 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44483 safe_time ceph cwd unknown /proc/44064/task/44483/cwd (readlink: Permission denied) 2024-05-21T21:59:26.557 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44483 safe_time ceph rtd unknown /proc/44064/task/44483/root (readlink: Permission denied) 2024-05-21T21:59:26.557 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44483 safe_time ceph txt unknown /proc/44064/task/44483/exe (readlink: Permission denied) 2024-05-21T21:59:26.557 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44483 safe_time ceph NOFD /proc/44064/task/44483/fd (opendir: Permission denied) 2024-05-21T21:59:26.557 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44484 fn_anonym ceph cwd unknown /proc/44064/task/44484/cwd (readlink: Permission denied) 2024-05-21T21:59:26.557 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44484 fn_anonym ceph rtd unknown /proc/44064/task/44484/root (readlink: Permission denied) 2024-05-21T21:59:26.557 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44484 fn_anonym ceph txt unknown /proc/44064/task/44484/exe (readlink: Permission denied) 2024-05-21T21:59:26.558 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44484 fn_anonym ceph NOFD /proc/44064/task/44484/fd (opendir: Permission denied) 2024-05-21T21:59:26.558 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44492 rocksdb:l ceph cwd unknown /proc/44064/task/44492/cwd (readlink: Permission denied) 2024-05-21T21:59:26.558 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44492 rocksdb:l ceph rtd unknown /proc/44064/task/44492/root (readlink: Permission denied) 2024-05-21T21:59:26.558 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44492 rocksdb:l ceph txt unknown /proc/44064/task/44492/exe (readlink: Permission denied) 2024-05-21T21:59:26.558 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44492 rocksdb:l ceph NOFD /proc/44064/task/44492/fd (opendir: Permission denied) 2024-05-21T21:59:26.558 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44493 rocksdb:l ceph cwd unknown /proc/44064/task/44493/cwd (readlink: Permission denied) 2024-05-21T21:59:26.558 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44493 rocksdb:l ceph rtd unknown /proc/44064/task/44493/root (readlink: Permission denied) 2024-05-21T21:59:26.558 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44493 rocksdb:l ceph txt unknown /proc/44064/task/44493/exe (readlink: Permission denied) 2024-05-21T21:59:26.558 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44493 rocksdb:l ceph NOFD /proc/44064/task/44493/fd (opendir: Permission denied) 2024-05-21T21:59:26.558 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44494 rocksdb:l ceph cwd unknown /proc/44064/task/44494/cwd (readlink: Permission denied) 2024-05-21T21:59:26.558 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44494 rocksdb:l ceph rtd unknown /proc/44064/task/44494/root (readlink: Permission denied) 2024-05-21T21:59:26.558 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44494 rocksdb:l ceph txt unknown /proc/44064/task/44494/exe (readlink: Permission denied) 2024-05-21T21:59:26.558 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44494 rocksdb:l ceph NOFD /proc/44064/task/44494/fd (opendir: Permission denied) 2024-05-21T21:59:26.559 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44495 rocksdb:h ceph cwd unknown /proc/44064/task/44495/cwd (readlink: Permission denied) 2024-05-21T21:59:26.559 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44495 rocksdb:h ceph rtd unknown /proc/44064/task/44495/root (readlink: Permission denied) 2024-05-21T21:59:26.559 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44495 rocksdb:h ceph txt unknown /proc/44064/task/44495/exe (readlink: Permission denied) 2024-05-21T21:59:26.559 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44495 rocksdb:h ceph NOFD /proc/44064/task/44495/fd (opendir: Permission denied) 2024-05-21T21:59:26.559 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44910 bstore_ai ceph cwd unknown /proc/44064/task/44910/cwd (readlink: Permission denied) 2024-05-21T21:59:26.559 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44910 bstore_ai ceph rtd unknown /proc/44064/task/44910/root (readlink: Permission denied) 2024-05-21T21:59:26.559 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44910 bstore_ai ceph txt unknown /proc/44064/task/44910/exe (readlink: Permission denied) 2024-05-21T21:59:26.559 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 44910 bstore_ai ceph NOFD /proc/44064/task/44910/fd (opendir: Permission denied) 2024-05-21T21:59:26.559 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45110 bstore_ai ceph cwd unknown /proc/44064/task/45110/cwd (readlink: Permission denied) 2024-05-21T21:59:26.559 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45110 bstore_ai ceph rtd unknown /proc/44064/task/45110/root (readlink: Permission denied) 2024-05-21T21:59:26.559 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45110 bstore_ai ceph txt unknown /proc/44064/task/45110/exe (readlink: Permission denied) 2024-05-21T21:59:26.559 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45110 bstore_ai ceph NOFD /proc/44064/task/45110/fd (opendir: Permission denied) 2024-05-21T21:59:26.559 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45295 ceph-osd ceph cwd unknown /proc/44064/task/45295/cwd (readlink: Permission denied) 2024-05-21T21:59:26.559 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45295 ceph-osd ceph rtd unknown /proc/44064/task/45295/root (readlink: Permission denied) 2024-05-21T21:59:26.560 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45295 ceph-osd ceph txt unknown /proc/44064/task/45295/exe (readlink: Permission denied) 2024-05-21T21:59:26.560 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45295 ceph-osd ceph NOFD /proc/44064/task/45295/fd (opendir: Permission denied) 2024-05-21T21:59:26.560 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45296 cfin ceph cwd unknown /proc/44064/task/45296/cwd (readlink: Permission denied) 2024-05-21T21:59:26.560 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45296 cfin ceph rtd unknown /proc/44064/task/45296/root (readlink: Permission denied) 2024-05-21T21:59:26.560 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45296 cfin ceph txt unknown /proc/44064/task/45296/exe (readlink: Permission denied) 2024-05-21T21:59:26.560 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45296 cfin ceph NOFD /proc/44064/task/45296/fd (opendir: Permission denied) 2024-05-21T21:59:26.560 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45297 bstore_kv ceph cwd unknown /proc/44064/task/45297/cwd (readlink: Permission denied) 2024-05-21T21:59:26.560 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45297 bstore_kv ceph rtd unknown /proc/44064/task/45297/root (readlink: Permission denied) 2024-05-21T21:59:26.560 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45297 bstore_kv ceph txt unknown /proc/44064/task/45297/exe (readlink: Permission denied) 2024-05-21T21:59:26.560 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45297 bstore_kv ceph NOFD /proc/44064/task/45297/fd (opendir: Permission denied) 2024-05-21T21:59:26.560 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45299 bstore_kv ceph cwd unknown /proc/44064/task/45299/cwd (readlink: Permission denied) 2024-05-21T21:59:26.560 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45299 bstore_kv ceph rtd unknown /proc/44064/task/45299/root (readlink: Permission denied) 2024-05-21T21:59:26.560 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45299 bstore_kv ceph txt unknown /proc/44064/task/45299/exe (readlink: Permission denied) 2024-05-21T21:59:26.560 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45299 bstore_kv ceph NOFD /proc/44064/task/45299/fd (opendir: Permission denied) 2024-05-21T21:59:26.561 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45300 bstore_me ceph cwd unknown /proc/44064/task/45300/cwd (readlink: Permission denied) 2024-05-21T21:59:26.561 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45300 bstore_me ceph rtd unknown /proc/44064/task/45300/root (readlink: Permission denied) 2024-05-21T21:59:26.561 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45300 bstore_me ceph txt unknown /proc/44064/task/45300/exe (readlink: Permission denied) 2024-05-21T21:59:26.561 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45300 bstore_me ceph NOFD /proc/44064/task/45300/fd (opendir: Permission denied) 2024-05-21T21:59:26.561 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45301 ms_dispat ceph cwd unknown /proc/44064/task/45301/cwd (readlink: Permission denied) 2024-05-21T21:59:26.561 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45301 ms_dispat ceph rtd unknown /proc/44064/task/45301/root (readlink: Permission denied) 2024-05-21T21:59:26.561 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45301 ms_dispat ceph txt unknown /proc/44064/task/45301/exe (readlink: Permission denied) 2024-05-21T21:59:26.561 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45301 ms_dispat ceph NOFD /proc/44064/task/45301/fd (opendir: Permission denied) 2024-05-21T21:59:26.561 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45302 ms_local ceph cwd unknown /proc/44064/task/45302/cwd (readlink: Permission denied) 2024-05-21T21:59:26.561 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45302 ms_local ceph rtd unknown /proc/44064/task/45302/root (readlink: Permission denied) 2024-05-21T21:59:26.561 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45302 ms_local ceph txt unknown /proc/44064/task/45302/exe (readlink: Permission denied) 2024-05-21T21:59:26.561 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45302 ms_local ceph NOFD /proc/44064/task/45302/fd (opendir: Permission denied) 2024-05-21T21:59:26.561 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45303 safe_time ceph cwd unknown /proc/44064/task/45303/cwd (readlink: Permission denied) 2024-05-21T21:59:26.562 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45303 safe_time ceph rtd unknown /proc/44064/task/45303/root (readlink: Permission denied) 2024-05-21T21:59:26.562 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45303 safe_time ceph txt unknown /proc/44064/task/45303/exe (readlink: Permission denied) 2024-05-21T21:59:26.562 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45303 safe_time ceph NOFD /proc/44064/task/45303/fd (opendir: Permission denied) 2024-05-21T21:59:26.562 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45304 safe_time ceph cwd unknown /proc/44064/task/45304/cwd (readlink: Permission denied) 2024-05-21T21:59:26.562 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45304 safe_time ceph rtd unknown /proc/44064/task/45304/root (readlink: Permission denied) 2024-05-21T21:59:26.562 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45304 safe_time ceph txt unknown /proc/44064/task/45304/exe (readlink: Permission denied) 2024-05-21T21:59:26.562 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45304 safe_time ceph NOFD /proc/44064/task/45304/fd (opendir: Permission denied) 2024-05-21T21:59:26.562 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45305 ms_dispat ceph cwd unknown /proc/44064/task/45305/cwd (readlink: Permission denied) 2024-05-21T21:59:26.562 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45305 ms_dispat ceph rtd unknown /proc/44064/task/45305/root (readlink: Permission denied) 2024-05-21T21:59:26.562 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45305 ms_dispat ceph txt unknown /proc/44064/task/45305/exe (readlink: Permission denied) 2024-05-21T21:59:26.562 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45305 ms_dispat ceph NOFD /proc/44064/task/45305/fd (opendir: Permission denied) 2024-05-21T21:59:26.562 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45306 ms_local ceph cwd unknown /proc/44064/task/45306/cwd (readlink: Permission denied) 2024-05-21T21:59:26.562 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45306 ms_local ceph rtd unknown /proc/44064/task/45306/root (readlink: Permission denied) 2024-05-21T21:59:26.563 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45306 ms_local ceph txt unknown /proc/44064/task/45306/exe (readlink: Permission denied) 2024-05-21T21:59:26.563 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45306 ms_local ceph NOFD /proc/44064/task/45306/fd (opendir: Permission denied) 2024-05-21T21:59:26.563 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45307 ms_dispat ceph cwd unknown /proc/44064/task/45307/cwd (readlink: Permission denied) 2024-05-21T21:59:26.563 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45307 ms_dispat ceph rtd unknown /proc/44064/task/45307/root (readlink: Permission denied) 2024-05-21T21:59:26.563 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45307 ms_dispat ceph txt unknown /proc/44064/task/45307/exe (readlink: Permission denied) 2024-05-21T21:59:26.563 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45307 ms_dispat ceph NOFD /proc/44064/task/45307/fd (opendir: Permission denied) 2024-05-21T21:59:26.563 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45308 ms_local ceph cwd unknown /proc/44064/task/45308/cwd (readlink: Permission denied) 2024-05-21T21:59:26.563 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45308 ms_local ceph rtd unknown /proc/44064/task/45308/root (readlink: Permission denied) 2024-05-21T21:59:26.563 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45308 ms_local ceph txt unknown /proc/44064/task/45308/exe (readlink: Permission denied) 2024-05-21T21:59:26.563 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45308 ms_local ceph NOFD /proc/44064/task/45308/fd (opendir: Permission denied) 2024-05-21T21:59:26.563 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45309 ms_dispat ceph cwd unknown /proc/44064/task/45309/cwd (readlink: Permission denied) 2024-05-21T21:59:26.563 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45309 ms_dispat ceph rtd unknown /proc/44064/task/45309/root (readlink: Permission denied) 2024-05-21T21:59:26.563 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45309 ms_dispat ceph txt unknown /proc/44064/task/45309/exe (readlink: Permission denied) 2024-05-21T21:59:26.563 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45309 ms_dispat ceph NOFD /proc/44064/task/45309/fd (opendir: Permission denied) 2024-05-21T21:59:26.564 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45310 ms_local ceph cwd unknown /proc/44064/task/45310/cwd (readlink: Permission denied) 2024-05-21T21:59:26.564 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45310 ms_local ceph rtd unknown /proc/44064/task/45310/root (readlink: Permission denied) 2024-05-21T21:59:26.564 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45310 ms_local ceph txt unknown /proc/44064/task/45310/exe (readlink: Permission denied) 2024-05-21T21:59:26.564 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45310 ms_local ceph NOFD /proc/44064/task/45310/fd (opendir: Permission denied) 2024-05-21T21:59:26.564 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45311 ms_dispat ceph cwd unknown /proc/44064/task/45311/cwd (readlink: Permission denied) 2024-05-21T21:59:26.564 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45311 ms_dispat ceph rtd unknown /proc/44064/task/45311/root (readlink: Permission denied) 2024-05-21T21:59:26.564 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45311 ms_dispat ceph txt unknown /proc/44064/task/45311/exe (readlink: Permission denied) 2024-05-21T21:59:26.564 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45311 ms_dispat ceph NOFD /proc/44064/task/45311/fd (opendir: Permission denied) 2024-05-21T21:59:26.564 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45313 ms_local ceph cwd unknown /proc/44064/task/45313/cwd (readlink: Permission denied) 2024-05-21T21:59:26.564 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45313 ms_local ceph rtd unknown /proc/44064/task/45313/root (readlink: Permission denied) 2024-05-21T21:59:26.564 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45313 ms_local ceph txt unknown /proc/44064/task/45313/exe (readlink: Permission denied) 2024-05-21T21:59:26.564 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45313 ms_local ceph NOFD /proc/44064/task/45313/fd (opendir: Permission denied) 2024-05-21T21:59:26.564 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45314 ms_dispat ceph cwd unknown /proc/44064/task/45314/cwd (readlink: Permission denied) 2024-05-21T21:59:26.565 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45314 ms_dispat ceph rtd unknown /proc/44064/task/45314/root (readlink: Permission denied) 2024-05-21T21:59:26.565 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45314 ms_dispat ceph txt unknown /proc/44064/task/45314/exe (readlink: Permission denied) 2024-05-21T21:59:26.565 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45314 ms_dispat ceph NOFD /proc/44064/task/45314/fd (opendir: Permission denied) 2024-05-21T21:59:26.565 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45316 ms_local ceph cwd unknown /proc/44064/task/45316/cwd (readlink: Permission denied) 2024-05-21T21:59:26.565 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45316 ms_local ceph rtd unknown /proc/44064/task/45316/root (readlink: Permission denied) 2024-05-21T21:59:26.565 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45316 ms_local ceph txt unknown /proc/44064/task/45316/exe (readlink: Permission denied) 2024-05-21T21:59:26.565 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45316 ms_local ceph NOFD /proc/44064/task/45316/fd (opendir: Permission denied) 2024-05-21T21:59:26.565 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45317 ms_dispat ceph cwd unknown /proc/44064/task/45317/cwd (readlink: Permission denied) 2024-05-21T21:59:26.565 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45317 ms_dispat ceph rtd unknown /proc/44064/task/45317/root (readlink: Permission denied) 2024-05-21T21:59:26.565 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45317 ms_dispat ceph txt unknown /proc/44064/task/45317/exe (readlink: Permission denied) 2024-05-21T21:59:26.565 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45317 ms_dispat ceph NOFD /proc/44064/task/45317/fd (opendir: Permission denied) 2024-05-21T21:59:26.565 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45318 ms_local ceph cwd unknown /proc/44064/task/45318/cwd (readlink: Permission denied) 2024-05-21T21:59:26.565 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45318 ms_local ceph rtd unknown /proc/44064/task/45318/root (readlink: Permission denied) 2024-05-21T21:59:26.565 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45318 ms_local ceph txt unknown /proc/44064/task/45318/exe (readlink: Permission denied) 2024-05-21T21:59:26.565 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45318 ms_local ceph NOFD /proc/44064/task/45318/fd (opendir: Permission denied) 2024-05-21T21:59:26.566 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45319 fn_anonym ceph cwd unknown /proc/44064/task/45319/cwd (readlink: Permission denied) 2024-05-21T21:59:26.566 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45319 fn_anonym ceph rtd unknown /proc/44064/task/45319/root (readlink: Permission denied) 2024-05-21T21:59:26.566 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45319 fn_anonym ceph txt unknown /proc/44064/task/45319/exe (readlink: Permission denied) 2024-05-21T21:59:26.566 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45319 fn_anonym ceph NOFD /proc/44064/task/45319/fd (opendir: Permission denied) 2024-05-21T21:59:26.566 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45320 finisher ceph cwd unknown /proc/44064/task/45320/cwd (readlink: Permission denied) 2024-05-21T21:59:26.566 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45320 finisher ceph rtd unknown /proc/44064/task/45320/root (readlink: Permission denied) 2024-05-21T21:59:26.566 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45320 finisher ceph txt unknown /proc/44064/task/45320/exe (readlink: Permission denied) 2024-05-21T21:59:26.566 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45320 finisher ceph NOFD /proc/44064/task/45320/fd (opendir: Permission denied) 2024-05-21T21:59:26.566 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45321 safe_time ceph cwd unknown /proc/44064/task/45321/cwd (readlink: Permission denied) 2024-05-21T21:59:26.566 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45321 safe_time ceph rtd unknown /proc/44064/task/45321/root (readlink: Permission denied) 2024-05-21T21:59:26.566 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45321 safe_time ceph txt unknown /proc/44064/task/45321/exe (readlink: Permission denied) 2024-05-21T21:59:26.566 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45321 safe_time ceph NOFD /proc/44064/task/45321/fd (opendir: Permission denied) 2024-05-21T21:59:26.566 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45322 safe_time ceph cwd unknown /proc/44064/task/45322/cwd (readlink: Permission denied) 2024-05-21T21:59:26.566 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45322 safe_time ceph rtd unknown /proc/44064/task/45322/root (readlink: Permission denied) 2024-05-21T21:59:26.566 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45322 safe_time ceph txt unknown /proc/44064/task/45322/exe (readlink: Permission denied) 2024-05-21T21:59:26.567 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45322 safe_time ceph NOFD /proc/44064/task/45322/fd (opendir: Permission denied) 2024-05-21T21:59:26.567 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45323 ceph-osd ceph cwd unknown /proc/44064/task/45323/cwd (readlink: Permission denied) 2024-05-21T21:59:26.567 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45323 ceph-osd ceph rtd unknown /proc/44064/task/45323/root (readlink: Permission denied) 2024-05-21T21:59:26.567 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45323 ceph-osd ceph txt unknown /proc/44064/task/45323/exe (readlink: Permission denied) 2024-05-21T21:59:26.567 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45323 ceph-osd ceph NOFD /proc/44064/task/45323/fd (opendir: Permission denied) 2024-05-21T21:59:26.567 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45324 osd_srv_a ceph cwd unknown /proc/44064/task/45324/cwd (readlink: Permission denied) 2024-05-21T21:59:26.567 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45324 osd_srv_a ceph rtd unknown /proc/44064/task/45324/root (readlink: Permission denied) 2024-05-21T21:59:26.567 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45324 osd_srv_a ceph txt unknown /proc/44064/task/45324/exe (readlink: Permission denied) 2024-05-21T21:59:26.567 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45324 osd_srv_a ceph NOFD /proc/44064/task/45324/fd (opendir: Permission denied) 2024-05-21T21:59:26.567 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45325 tp_osd_tp ceph cwd unknown /proc/44064/task/45325/cwd (readlink: Permission denied) 2024-05-21T21:59:26.567 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45325 tp_osd_tp ceph rtd unknown /proc/44064/task/45325/root (readlink: Permission denied) 2024-05-21T21:59:26.567 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45325 tp_osd_tp ceph txt unknown /proc/44064/task/45325/exe (readlink: Permission denied) 2024-05-21T21:59:26.568 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45325 tp_osd_tp ceph NOFD /proc/44064/task/45325/fd (opendir: Permission denied) 2024-05-21T21:59:26.568 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45326 tp_osd_tp ceph cwd unknown /proc/44064/task/45326/cwd (readlink: Permission denied) 2024-05-21T21:59:26.568 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45326 tp_osd_tp ceph rtd unknown /proc/44064/task/45326/root (readlink: Permission denied) 2024-05-21T21:59:26.568 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45326 tp_osd_tp ceph txt unknown /proc/44064/task/45326/exe (readlink: Permission denied) 2024-05-21T21:59:26.568 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45326 tp_osd_tp ceph NOFD /proc/44064/task/45326/fd (opendir: Permission denied) 2024-05-21T21:59:26.568 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45327 tp_osd_tp ceph cwd unknown /proc/44064/task/45327/cwd (readlink: Permission denied) 2024-05-21T21:59:26.568 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45327 tp_osd_tp ceph rtd unknown /proc/44064/task/45327/root (readlink: Permission denied) 2024-05-21T21:59:26.568 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45327 tp_osd_tp ceph txt unknown /proc/44064/task/45327/exe (readlink: Permission denied) 2024-05-21T21:59:26.568 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45327 tp_osd_tp ceph NOFD /proc/44064/task/45327/fd (opendir: Permission denied) 2024-05-21T21:59:26.568 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45328 tp_osd_tp ceph cwd unknown /proc/44064/task/45328/cwd (readlink: Permission denied) 2024-05-21T21:59:26.568 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45328 tp_osd_tp ceph rtd unknown /proc/44064/task/45328/root (readlink: Permission denied) 2024-05-21T21:59:26.568 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45328 tp_osd_tp ceph txt unknown /proc/44064/task/45328/exe (readlink: Permission denied) 2024-05-21T21:59:26.568 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45328 tp_osd_tp ceph NOFD /proc/44064/task/45328/fd (opendir: Permission denied) 2024-05-21T21:59:26.569 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45329 tp_osd_tp ceph cwd unknown /proc/44064/task/45329/cwd (readlink: Permission denied) 2024-05-21T21:59:26.569 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45329 tp_osd_tp ceph rtd unknown /proc/44064/task/45329/root (readlink: Permission denied) 2024-05-21T21:59:26.569 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45329 tp_osd_tp ceph txt unknown /proc/44064/task/45329/exe (readlink: Permission denied) 2024-05-21T21:59:26.569 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45329 tp_osd_tp ceph NOFD /proc/44064/task/45329/fd (opendir: Permission denied) 2024-05-21T21:59:26.569 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45330 tp_osd_tp ceph cwd unknown /proc/44064/task/45330/cwd (readlink: Permission denied) 2024-05-21T21:59:26.569 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45330 tp_osd_tp ceph rtd unknown /proc/44064/task/45330/root (readlink: Permission denied) 2024-05-21T21:59:26.569 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45330 tp_osd_tp ceph txt unknown /proc/44064/task/45330/exe (readlink: Permission denied) 2024-05-21T21:59:26.569 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45330 tp_osd_tp ceph NOFD /proc/44064/task/45330/fd (opendir: Permission denied) 2024-05-21T21:59:26.569 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45331 tp_osd_tp ceph cwd unknown /proc/44064/task/45331/cwd (readlink: Permission denied) 2024-05-21T21:59:26.569 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45331 tp_osd_tp ceph rtd unknown /proc/44064/task/45331/root (readlink: Permission denied) 2024-05-21T21:59:26.569 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45331 tp_osd_tp ceph txt unknown /proc/44064/task/45331/exe (readlink: Permission denied) 2024-05-21T21:59:26.570 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45331 tp_osd_tp ceph NOFD /proc/44064/task/45331/fd (opendir: Permission denied) 2024-05-21T21:59:26.570 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45332 tp_osd_tp ceph cwd unknown /proc/44064/task/45332/cwd (readlink: Permission denied) 2024-05-21T21:59:26.570 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45332 tp_osd_tp ceph rtd unknown /proc/44064/task/45332/root (readlink: Permission denied) 2024-05-21T21:59:26.570 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45332 tp_osd_tp ceph txt unknown /proc/44064/task/45332/exe (readlink: Permission denied) 2024-05-21T21:59:26.570 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45332 tp_osd_tp ceph NOFD /proc/44064/task/45332/fd (opendir: Permission denied) 2024-05-21T21:59:26.570 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45333 tp_osd_tp ceph cwd unknown /proc/44064/task/45333/cwd (readlink: Permission denied) 2024-05-21T21:59:26.570 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45333 tp_osd_tp ceph rtd unknown /proc/44064/task/45333/root (readlink: Permission denied) 2024-05-21T21:59:26.570 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45333 tp_osd_tp ceph txt unknown /proc/44064/task/45333/exe (readlink: Permission denied) 2024-05-21T21:59:26.570 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45333 tp_osd_tp ceph NOFD /proc/44064/task/45333/fd (opendir: Permission denied) 2024-05-21T21:59:26.570 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45334 tp_osd_tp ceph cwd unknown /proc/44064/task/45334/cwd (readlink: Permission denied) 2024-05-21T21:59:26.570 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45334 tp_osd_tp ceph rtd unknown /proc/44064/task/45334/root (readlink: Permission denied) 2024-05-21T21:59:26.571 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45334 tp_osd_tp ceph txt unknown /proc/44064/task/45334/exe (readlink: Permission denied) 2024-05-21T21:59:26.571 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45334 tp_osd_tp ceph NOFD /proc/44064/task/45334/fd (opendir: Permission denied) 2024-05-21T21:59:26.571 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45335 tp_osd_tp ceph cwd unknown /proc/44064/task/45335/cwd (readlink: Permission denied) 2024-05-21T21:59:26.571 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45335 tp_osd_tp ceph rtd unknown /proc/44064/task/45335/root (readlink: Permission denied) 2024-05-21T21:59:26.571 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45335 tp_osd_tp ceph txt unknown /proc/44064/task/45335/exe (readlink: Permission denied) 2024-05-21T21:59:26.571 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45335 tp_osd_tp ceph NOFD /proc/44064/task/45335/fd (opendir: Permission denied) 2024-05-21T21:59:26.571 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45336 tp_osd_tp ceph cwd unknown /proc/44064/task/45336/cwd (readlink: Permission denied) 2024-05-21T21:59:26.571 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45336 tp_osd_tp ceph rtd unknown /proc/44064/task/45336/root (readlink: Permission denied) 2024-05-21T21:59:26.571 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45336 tp_osd_tp ceph txt unknown /proc/44064/task/45336/exe (readlink: Permission denied) 2024-05-21T21:59:26.571 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45336 tp_osd_tp ceph NOFD /proc/44064/task/45336/fd (opendir: Permission denied) 2024-05-21T21:59:26.571 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45337 tp_osd_tp ceph cwd unknown /proc/44064/task/45337/cwd (readlink: Permission denied) 2024-05-21T21:59:26.571 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45337 tp_osd_tp ceph rtd unknown /proc/44064/task/45337/root (readlink: Permission denied) 2024-05-21T21:59:26.572 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45337 tp_osd_tp ceph txt unknown /proc/44064/task/45337/exe (readlink: Permission denied) 2024-05-21T21:59:26.572 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45337 tp_osd_tp ceph NOFD /proc/44064/task/45337/fd (opendir: Permission denied) 2024-05-21T21:59:26.572 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45338 tp_osd_tp ceph cwd unknown /proc/44064/task/45338/cwd (readlink: Permission denied) 2024-05-21T21:59:26.572 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45338 tp_osd_tp ceph rtd unknown /proc/44064/task/45338/root (readlink: Permission denied) 2024-05-21T21:59:26.572 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45338 tp_osd_tp ceph txt unknown /proc/44064/task/45338/exe (readlink: Permission denied) 2024-05-21T21:59:26.572 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45338 tp_osd_tp ceph NOFD /proc/44064/task/45338/fd (opendir: Permission denied) 2024-05-21T21:59:26.572 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45339 tp_osd_tp ceph cwd unknown /proc/44064/task/45339/cwd (readlink: Permission denied) 2024-05-21T21:59:26.572 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45339 tp_osd_tp ceph rtd unknown /proc/44064/task/45339/root (readlink: Permission denied) 2024-05-21T21:59:26.572 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45339 tp_osd_tp ceph txt unknown /proc/44064/task/45339/exe (readlink: Permission denied) 2024-05-21T21:59:26.572 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45339 tp_osd_tp ceph NOFD /proc/44064/task/45339/fd (opendir: Permission denied) 2024-05-21T21:59:26.572 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45340 tp_osd_tp ceph cwd unknown /proc/44064/task/45340/cwd (readlink: Permission denied) 2024-05-21T21:59:26.572 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45340 tp_osd_tp ceph rtd unknown /proc/44064/task/45340/root (readlink: Permission denied) 2024-05-21T21:59:26.573 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45340 tp_osd_tp ceph txt unknown /proc/44064/task/45340/exe (readlink: Permission denied) 2024-05-21T21:59:26.573 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45340 tp_osd_tp ceph NOFD /proc/44064/task/45340/fd (opendir: Permission denied) 2024-05-21T21:59:26.573 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45341 osd_srv_h ceph cwd unknown /proc/44064/task/45341/cwd (readlink: Permission denied) 2024-05-21T21:59:26.573 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45341 osd_srv_h ceph rtd unknown /proc/44064/task/45341/root (readlink: Permission denied) 2024-05-21T21:59:26.573 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45341 osd_srv_h ceph txt unknown /proc/44064/task/45341/exe (readlink: Permission denied) 2024-05-21T21:59:26.573 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 44064 45341 osd_srv_h ceph NOFD /proc/44064/task/45341/fd (opendir: Permission denied) 2024-05-21T21:59:26.573 INFO:tasks.workunit.client.0.smithi151.stdout:sudo 45466 root cwd unknown /proc/45466/cwd (readlink: Permission denied) 2024-05-21T21:59:26.573 INFO:tasks.workunit.client.0.smithi151.stdout:sudo 45466 root rtd unknown /proc/45466/root (readlink: Permission denied) 2024-05-21T21:59:26.573 INFO:tasks.workunit.client.0.smithi151.stdout:sudo 45466 root txt unknown /proc/45466/exe (readlink: Permission denied) 2024-05-21T21:59:26.573 INFO:tasks.workunit.client.0.smithi151.stdout:sudo 45466 root NOFD /proc/45466/fd (opendir: Permission denied) 2024-05-21T21:59:26.573 INFO:tasks.workunit.client.0.smithi151.stdout:journalct 45495 root cwd unknown /proc/45495/cwd (readlink: Permission denied) 2024-05-21T21:59:26.574 INFO:tasks.workunit.client.0.smithi151.stdout:journalct 45495 root rtd unknown /proc/45495/root (readlink: Permission denied) 2024-05-21T21:59:26.574 INFO:tasks.workunit.client.0.smithi151.stdout:journalct 45495 root txt unknown /proc/45495/exe (readlink: Permission denied) 2024-05-21T21:59:26.574 INFO:tasks.workunit.client.0.smithi151.stdout:journalct 45495 root NOFD /proc/45495/fd (opendir: Permission denied) 2024-05-21T21:59:26.574 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/4 49325 root cwd unknown /proc/49325/cwd (readlink: Permission denied) 2024-05-21T21:59:26.574 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/4 49325 root rtd unknown /proc/49325/root (readlink: Permission denied) 2024-05-21T21:59:26.574 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/4 49325 root txt unknown /proc/49325/exe (readlink: Permission denied) 2024-05-21T21:59:26.574 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/4 49325 root NOFD /proc/49325/fd (opendir: Permission denied) 2024-05-21T21:59:26.574 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/7 50141 root cwd unknown /proc/50141/cwd (readlink: Permission denied) 2024-05-21T21:59:26.574 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/7 50141 root rtd unknown /proc/50141/root (readlink: Permission denied) 2024-05-21T21:59:26.575 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/7 50141 root txt unknown /proc/50141/exe (readlink: Permission denied) 2024-05-21T21:59:26.575 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/7 50141 root NOFD /proc/50141/fd (opendir: Permission denied) 2024-05-21T21:59:26.575 INFO:tasks.workunit.client.0.smithi151.stdout:conmon 50544 root cwd unknown /proc/50544/cwd (readlink: Permission denied) 2024-05-21T21:59:26.575 INFO:tasks.workunit.client.0.smithi151.stdout:conmon 50544 root rtd unknown /proc/50544/root (readlink: Permission denied) 2024-05-21T21:59:26.575 INFO:tasks.workunit.client.0.smithi151.stdout:conmon 50544 root txt unknown /proc/50544/exe (readlink: Permission denied) 2024-05-21T21:59:26.575 INFO:tasks.workunit.client.0.smithi151.stdout:conmon 50544 root NOFD /proc/50544/fd (opendir: Permission denied) 2024-05-21T21:59:26.575 INFO:tasks.workunit.client.0.smithi151.stdout:podman-in 50546 root cwd unknown /proc/50546/cwd (readlink: Permission denied) 2024-05-21T21:59:26.575 INFO:tasks.workunit.client.0.smithi151.stdout:podman-in 50546 root rtd unknown /proc/50546/root (readlink: Permission denied) 2024-05-21T21:59:26.575 INFO:tasks.workunit.client.0.smithi151.stdout:podman-in 50546 root txt unknown /proc/50546/exe (readlink: Permission denied) 2024-05-21T21:59:26.575 INFO:tasks.workunit.client.0.smithi151.stdout:podman-in 50546 root NOFD /proc/50546/fd (opendir: Permission denied) 2024-05-21T21:59:26.575 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 ceph cwd unknown /proc/50548/cwd (readlink: Permission denied) 2024-05-21T21:59:26.575 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 ceph rtd unknown /proc/50548/root (readlink: Permission denied) 2024-05-21T21:59:26.576 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 ceph txt unknown /proc/50548/exe (readlink: Permission denied) 2024-05-21T21:59:26.576 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 ceph NOFD /proc/50548/fd (opendir: Permission denied) 2024-05-21T21:59:26.576 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50549 log ceph cwd unknown /proc/50548/task/50549/cwd (readlink: Permission denied) 2024-05-21T21:59:26.576 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50549 log ceph rtd unknown /proc/50548/task/50549/root (readlink: Permission denied) 2024-05-21T21:59:26.576 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50549 log ceph txt unknown /proc/50548/task/50549/exe (readlink: Permission denied) 2024-05-21T21:59:26.576 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50549 log ceph NOFD /proc/50548/task/50549/fd (opendir: Permission denied) 2024-05-21T21:59:26.576 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50551 msgr-work ceph cwd unknown /proc/50548/task/50551/cwd (readlink: Permission denied) 2024-05-21T21:59:26.576 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50551 msgr-work ceph rtd unknown /proc/50548/task/50551/root (readlink: Permission denied) 2024-05-21T21:59:26.576 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50551 msgr-work ceph txt unknown /proc/50548/task/50551/exe (readlink: Permission denied) 2024-05-21T21:59:26.576 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50551 msgr-work ceph NOFD /proc/50548/task/50551/fd (opendir: Permission denied) 2024-05-21T21:59:26.576 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50552 msgr-work ceph cwd unknown /proc/50548/task/50552/cwd (readlink: Permission denied) 2024-05-21T21:59:26.576 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50552 msgr-work ceph rtd unknown /proc/50548/task/50552/root (readlink: Permission denied) 2024-05-21T21:59:26.576 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50552 msgr-work ceph txt unknown /proc/50548/task/50552/exe (readlink: Permission denied) 2024-05-21T21:59:26.577 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50552 msgr-work ceph NOFD /proc/50548/task/50552/fd (opendir: Permission denied) 2024-05-21T21:59:26.577 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50553 msgr-work ceph cwd unknown /proc/50548/task/50553/cwd (readlink: Permission denied) 2024-05-21T21:59:26.577 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50553 msgr-work ceph rtd unknown /proc/50548/task/50553/root (readlink: Permission denied) 2024-05-21T21:59:26.577 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50553 msgr-work ceph txt unknown /proc/50548/task/50553/exe (readlink: Permission denied) 2024-05-21T21:59:26.577 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50553 msgr-work ceph NOFD /proc/50548/task/50553/fd (opendir: Permission denied) 2024-05-21T21:59:26.577 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50557 service ceph cwd unknown /proc/50548/task/50557/cwd (readlink: Permission denied) 2024-05-21T21:59:26.577 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50557 service ceph rtd unknown /proc/50548/task/50557/root (readlink: Permission denied) 2024-05-21T21:59:26.577 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50557 service ceph txt unknown /proc/50548/task/50557/exe (readlink: Permission denied) 2024-05-21T21:59:26.577 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50557 service ceph NOFD /proc/50548/task/50557/fd (opendir: Permission denied) 2024-05-21T21:59:26.577 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50558 admin_soc ceph cwd unknown /proc/50548/task/50558/cwd (readlink: Permission denied) 2024-05-21T21:59:26.577 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50558 admin_soc ceph rtd unknown /proc/50548/task/50558/root (readlink: Permission denied) 2024-05-21T21:59:26.577 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50558 admin_soc ceph txt unknown /proc/50548/task/50558/exe (readlink: Permission denied) 2024-05-21T21:59:26.577 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50558 admin_soc ceph NOFD /proc/50548/task/50558/fd (opendir: Permission denied) 2024-05-21T21:59:26.577 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50702 signal_ha ceph cwd unknown /proc/50548/task/50702/cwd (readlink: Permission denied) 2024-05-21T21:59:26.578 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50702 signal_ha ceph rtd unknown /proc/50548/task/50702/root (readlink: Permission denied) 2024-05-21T21:59:26.578 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50702 signal_ha ceph txt unknown /proc/50548/task/50702/exe (readlink: Permission denied) 2024-05-21T21:59:26.578 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50702 signal_ha ceph NOFD /proc/50548/task/50702/fd (opendir: Permission denied) 2024-05-21T21:59:26.578 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50703 io_contex ceph cwd unknown /proc/50548/task/50703/cwd (readlink: Permission denied) 2024-05-21T21:59:26.578 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50703 io_contex ceph rtd unknown /proc/50548/task/50703/root (readlink: Permission denied) 2024-05-21T21:59:26.578 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50703 io_contex ceph txt unknown /proc/50548/task/50703/exe (readlink: Permission denied) 2024-05-21T21:59:26.578 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50703 io_contex ceph NOFD /proc/50548/task/50703/fd (opendir: Permission denied) 2024-05-21T21:59:26.578 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50704 io_contex ceph cwd unknown /proc/50548/task/50704/cwd (readlink: Permission denied) 2024-05-21T21:59:26.578 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50704 io_contex ceph rtd unknown /proc/50548/task/50704/root (readlink: Permission denied) 2024-05-21T21:59:26.578 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50704 io_contex ceph txt unknown /proc/50548/task/50704/exe (readlink: Permission denied) 2024-05-21T21:59:26.578 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50704 io_contex ceph NOFD /proc/50548/task/50704/fd (opendir: Permission denied) 2024-05-21T21:59:26.578 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50719 OpHistory ceph cwd unknown /proc/50548/task/50719/cwd (readlink: Permission denied) 2024-05-21T21:59:26.578 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50719 OpHistory ceph rtd unknown /proc/50548/task/50719/root (readlink: Permission denied) 2024-05-21T21:59:26.578 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50719 OpHistory ceph txt unknown /proc/50548/task/50719/exe (readlink: Permission denied) 2024-05-21T21:59:26.579 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50719 OpHistory ceph NOFD /proc/50548/task/50719/fd (opendir: Permission denied) 2024-05-21T21:59:26.579 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50720 ceph_time ceph cwd unknown /proc/50548/task/50720/cwd (readlink: Permission denied) 2024-05-21T21:59:26.579 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50720 ceph_time ceph rtd unknown /proc/50548/task/50720/root (readlink: Permission denied) 2024-05-21T21:59:26.579 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50720 ceph_time ceph txt unknown /proc/50548/task/50720/exe (readlink: Permission denied) 2024-05-21T21:59:26.579 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50720 ceph_time ceph NOFD /proc/50548/task/50720/fd (opendir: Permission denied) 2024-05-21T21:59:26.579 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50816 ceph-osd ceph cwd unknown /proc/50548/task/50816/cwd (readlink: Permission denied) 2024-05-21T21:59:26.579 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50816 ceph-osd ceph rtd unknown /proc/50548/task/50816/root (readlink: Permission denied) 2024-05-21T21:59:26.579 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50816 ceph-osd ceph txt unknown /proc/50548/task/50816/exe (readlink: Permission denied) 2024-05-21T21:59:26.579 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50816 ceph-osd ceph NOFD /proc/50548/task/50816/fd (opendir: Permission denied) 2024-05-21T21:59:26.579 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50828 ceph-osd ceph cwd unknown /proc/50548/task/50828/cwd (readlink: Permission denied) 2024-05-21T21:59:26.579 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50828 ceph-osd ceph rtd unknown /proc/50548/task/50828/root (readlink: Permission denied) 2024-05-21T21:59:26.579 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50828 ceph-osd ceph txt unknown /proc/50548/task/50828/exe (readlink: Permission denied) 2024-05-21T21:59:26.579 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50828 ceph-osd ceph NOFD /proc/50548/task/50828/fd (opendir: Permission denied) 2024-05-21T21:59:26.579 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50857 ceph-osd ceph cwd unknown /proc/50548/task/50857/cwd (readlink: Permission denied) 2024-05-21T21:59:26.579 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50857 ceph-osd ceph rtd unknown /proc/50548/task/50857/root (readlink: Permission denied) 2024-05-21T21:59:26.580 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50857 ceph-osd ceph txt unknown /proc/50548/task/50857/exe (readlink: Permission denied) 2024-05-21T21:59:26.580 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50857 ceph-osd ceph NOFD /proc/50548/task/50857/fd (opendir: Permission denied) 2024-05-21T21:59:26.580 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50877 ceph-osd ceph cwd unknown /proc/50548/task/50877/cwd (readlink: Permission denied) 2024-05-21T21:59:26.580 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50877 ceph-osd ceph rtd unknown /proc/50548/task/50877/root (readlink: Permission denied) 2024-05-21T21:59:26.580 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50877 ceph-osd ceph txt unknown /proc/50548/task/50877/exe (readlink: Permission denied) 2024-05-21T21:59:26.580 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50877 ceph-osd ceph NOFD /proc/50548/task/50877/fd (opendir: Permission denied) 2024-05-21T21:59:26.580 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50884 ceph-osd ceph cwd unknown /proc/50548/task/50884/cwd (readlink: Permission denied) 2024-05-21T21:59:26.580 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50884 ceph-osd ceph rtd unknown /proc/50548/task/50884/root (readlink: Permission denied) 2024-05-21T21:59:26.580 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50884 ceph-osd ceph txt unknown /proc/50548/task/50884/exe (readlink: Permission denied) 2024-05-21T21:59:26.580 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50884 ceph-osd ceph NOFD /proc/50548/task/50884/fd (opendir: Permission denied) 2024-05-21T21:59:26.580 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50905 ceph-osd ceph cwd unknown /proc/50548/task/50905/cwd (readlink: Permission denied) 2024-05-21T21:59:26.580 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50905 ceph-osd ceph rtd unknown /proc/50548/task/50905/root (readlink: Permission denied) 2024-05-21T21:59:26.580 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50905 ceph-osd ceph txt unknown /proc/50548/task/50905/exe (readlink: Permission denied) 2024-05-21T21:59:26.580 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50905 ceph-osd ceph NOFD /proc/50548/task/50905/fd (opendir: Permission denied) 2024-05-21T21:59:26.580 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50912 ceph-osd ceph cwd unknown /proc/50548/task/50912/cwd (readlink: Permission denied) 2024-05-21T21:59:26.581 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50912 ceph-osd ceph rtd unknown /proc/50548/task/50912/root (readlink: Permission denied) 2024-05-21T21:59:26.581 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50912 ceph-osd ceph txt unknown /proc/50548/task/50912/exe (readlink: Permission denied) 2024-05-21T21:59:26.581 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50912 ceph-osd ceph NOFD /proc/50548/task/50912/fd (opendir: Permission denied) 2024-05-21T21:59:26.581 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50936 ceph-osd ceph cwd unknown /proc/50548/task/50936/cwd (readlink: Permission denied) 2024-05-21T21:59:26.581 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50936 ceph-osd ceph rtd unknown /proc/50548/task/50936/root (readlink: Permission denied) 2024-05-21T21:59:26.581 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50936 ceph-osd ceph txt unknown /proc/50548/task/50936/exe (readlink: Permission denied) 2024-05-21T21:59:26.581 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50936 ceph-osd ceph NOFD /proc/50548/task/50936/fd (opendir: Permission denied) 2024-05-21T21:59:26.581 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50937 ceph-osd ceph cwd unknown /proc/50548/task/50937/cwd (readlink: Permission denied) 2024-05-21T21:59:26.581 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50937 ceph-osd ceph rtd unknown /proc/50548/task/50937/root (readlink: Permission denied) 2024-05-21T21:59:26.581 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50937 ceph-osd ceph txt unknown /proc/50548/task/50937/exe (readlink: Permission denied) 2024-05-21T21:59:26.581 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50937 ceph-osd ceph NOFD /proc/50548/task/50937/fd (opendir: Permission denied) 2024-05-21T21:59:26.581 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50938 safe_time ceph cwd unknown /proc/50548/task/50938/cwd (readlink: Permission denied) 2024-05-21T21:59:26.581 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50938 safe_time ceph rtd unknown /proc/50548/task/50938/root (readlink: Permission denied) 2024-05-21T21:59:26.581 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50938 safe_time ceph txt unknown /proc/50548/task/50938/exe (readlink: Permission denied) 2024-05-21T21:59:26.582 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50938 safe_time ceph NOFD /proc/50548/task/50938/fd (opendir: Permission denied) 2024-05-21T21:59:26.582 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50939 safe_time ceph cwd unknown /proc/50548/task/50939/cwd (readlink: Permission denied) 2024-05-21T21:59:26.582 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50939 safe_time ceph rtd unknown /proc/50548/task/50939/root (readlink: Permission denied) 2024-05-21T21:59:26.582 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50939 safe_time ceph txt unknown /proc/50548/task/50939/exe (readlink: Permission denied) 2024-05-21T21:59:26.582 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50939 safe_time ceph NOFD /proc/50548/task/50939/fd (opendir: Permission denied) 2024-05-21T21:59:26.582 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50940 safe_time ceph cwd unknown /proc/50548/task/50940/cwd (readlink: Permission denied) 2024-05-21T21:59:26.582 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50940 safe_time ceph rtd unknown /proc/50548/task/50940/root (readlink: Permission denied) 2024-05-21T21:59:26.582 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50940 safe_time ceph txt unknown /proc/50548/task/50940/exe (readlink: Permission denied) 2024-05-21T21:59:26.582 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50940 safe_time ceph NOFD /proc/50548/task/50940/fd (opendir: Permission denied) 2024-05-21T21:59:26.582 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50941 safe_time ceph cwd unknown /proc/50548/task/50941/cwd (readlink: Permission denied) 2024-05-21T21:59:26.582 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50941 safe_time ceph rtd unknown /proc/50548/task/50941/root (readlink: Permission denied) 2024-05-21T21:59:26.582 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50941 safe_time ceph txt unknown /proc/50548/task/50941/exe (readlink: Permission denied) 2024-05-21T21:59:26.583 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50941 safe_time ceph NOFD /proc/50548/task/50941/fd (opendir: Permission denied) 2024-05-21T21:59:26.583 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50942 fn_anonym ceph cwd unknown /proc/50548/task/50942/cwd (readlink: Permission denied) 2024-05-21T21:59:26.583 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50942 fn_anonym ceph rtd unknown /proc/50548/task/50942/root (readlink: Permission denied) 2024-05-21T21:59:26.583 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50942 fn_anonym ceph txt unknown /proc/50548/task/50942/exe (readlink: Permission denied) 2024-05-21T21:59:26.583 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50942 fn_anonym ceph NOFD /proc/50548/task/50942/fd (opendir: Permission denied) 2024-05-21T21:59:26.583 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50948 rocksdb:l ceph cwd unknown /proc/50548/task/50948/cwd (readlink: Permission denied) 2024-05-21T21:59:26.583 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50948 rocksdb:l ceph rtd unknown /proc/50548/task/50948/root (readlink: Permission denied) 2024-05-21T21:59:26.583 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50948 rocksdb:l ceph txt unknown /proc/50548/task/50948/exe (readlink: Permission denied) 2024-05-21T21:59:26.583 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50948 rocksdb:l ceph NOFD /proc/50548/task/50948/fd (opendir: Permission denied) 2024-05-21T21:59:26.583 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50949 rocksdb:l ceph cwd unknown /proc/50548/task/50949/cwd (readlink: Permission denied) 2024-05-21T21:59:26.583 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50949 rocksdb:l ceph rtd unknown /proc/50548/task/50949/root (readlink: Permission denied) 2024-05-21T21:59:26.583 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50949 rocksdb:l ceph txt unknown /proc/50548/task/50949/exe (readlink: Permission denied) 2024-05-21T21:59:26.583 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50949 rocksdb:l ceph NOFD /proc/50548/task/50949/fd (opendir: Permission denied) 2024-05-21T21:59:26.583 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50950 rocksdb:l ceph cwd unknown /proc/50548/task/50950/cwd (readlink: Permission denied) 2024-05-21T21:59:26.584 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50950 rocksdb:l ceph rtd unknown /proc/50548/task/50950/root (readlink: Permission denied) 2024-05-21T21:59:26.584 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50950 rocksdb:l ceph txt unknown /proc/50548/task/50950/exe (readlink: Permission denied) 2024-05-21T21:59:26.584 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50950 rocksdb:l ceph NOFD /proc/50548/task/50950/fd (opendir: Permission denied) 2024-05-21T21:59:26.584 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50951 rocksdb:h ceph cwd unknown /proc/50548/task/50951/cwd (readlink: Permission denied) 2024-05-21T21:59:26.584 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50951 rocksdb:h ceph rtd unknown /proc/50548/task/50951/root (readlink: Permission denied) 2024-05-21T21:59:26.584 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50951 rocksdb:h ceph txt unknown /proc/50548/task/50951/exe (readlink: Permission denied) 2024-05-21T21:59:26.584 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 50951 rocksdb:h ceph NOFD /proc/50548/task/50951/fd (opendir: Permission denied) 2024-05-21T21:59:26.584 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51381 bstore_ai ceph cwd unknown /proc/50548/task/51381/cwd (readlink: Permission denied) 2024-05-21T21:59:26.584 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51381 bstore_ai ceph rtd unknown /proc/50548/task/51381/root (readlink: Permission denied) 2024-05-21T21:59:26.584 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51381 bstore_ai ceph txt unknown /proc/50548/task/51381/exe (readlink: Permission denied) 2024-05-21T21:59:26.584 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51381 bstore_ai ceph NOFD /proc/50548/task/51381/fd (opendir: Permission denied) 2024-05-21T21:59:26.584 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51615 bstore_ai ceph cwd unknown /proc/50548/task/51615/cwd (readlink: Permission denied) 2024-05-21T21:59:26.584 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51615 bstore_ai ceph rtd unknown /proc/50548/task/51615/root (readlink: Permission denied) 2024-05-21T21:59:26.584 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51615 bstore_ai ceph txt unknown /proc/50548/task/51615/exe (readlink: Permission denied) 2024-05-21T21:59:26.584 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51615 bstore_ai ceph NOFD /proc/50548/task/51615/fd (opendir: Permission denied) 2024-05-21T21:59:26.584 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51801 ceph-osd ceph cwd unknown /proc/50548/task/51801/cwd (readlink: Permission denied) 2024-05-21T21:59:26.585 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51801 ceph-osd ceph rtd unknown /proc/50548/task/51801/root (readlink: Permission denied) 2024-05-21T21:59:26.585 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51801 ceph-osd ceph txt unknown /proc/50548/task/51801/exe (readlink: Permission denied) 2024-05-21T21:59:26.585 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51801 ceph-osd ceph NOFD /proc/50548/task/51801/fd (opendir: Permission denied) 2024-05-21T21:59:26.585 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51802 cfin ceph cwd unknown /proc/50548/task/51802/cwd (readlink: Permission denied) 2024-05-21T21:59:26.585 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51802 cfin ceph rtd unknown /proc/50548/task/51802/root (readlink: Permission denied) 2024-05-21T21:59:26.585 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51802 cfin ceph txt unknown /proc/50548/task/51802/exe (readlink: Permission denied) 2024-05-21T21:59:26.585 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51802 cfin ceph NOFD /proc/50548/task/51802/fd (opendir: Permission denied) 2024-05-21T21:59:26.585 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51803 bstore_kv ceph cwd unknown /proc/50548/task/51803/cwd (readlink: Permission denied) 2024-05-21T21:59:26.585 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51803 bstore_kv ceph rtd unknown /proc/50548/task/51803/root (readlink: Permission denied) 2024-05-21T21:59:26.585 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51803 bstore_kv ceph txt unknown /proc/50548/task/51803/exe (readlink: Permission denied) 2024-05-21T21:59:26.585 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51803 bstore_kv ceph NOFD /proc/50548/task/51803/fd (opendir: Permission denied) 2024-05-21T21:59:26.585 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51804 bstore_kv ceph cwd unknown /proc/50548/task/51804/cwd (readlink: Permission denied) 2024-05-21T21:59:26.585 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51804 bstore_kv ceph rtd unknown /proc/50548/task/51804/root (readlink: Permission denied) 2024-05-21T21:59:26.585 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51804 bstore_kv ceph txt unknown /proc/50548/task/51804/exe (readlink: Permission denied) 2024-05-21T21:59:26.585 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51804 bstore_kv ceph NOFD /proc/50548/task/51804/fd (opendir: Permission denied) 2024-05-21T21:59:26.586 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51805 bstore_me ceph cwd unknown /proc/50548/task/51805/cwd (readlink: Permission denied) 2024-05-21T21:59:26.586 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51805 bstore_me ceph rtd unknown /proc/50548/task/51805/root (readlink: Permission denied) 2024-05-21T21:59:26.586 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51805 bstore_me ceph txt unknown /proc/50548/task/51805/exe (readlink: Permission denied) 2024-05-21T21:59:26.586 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51805 bstore_me ceph NOFD /proc/50548/task/51805/fd (opendir: Permission denied) 2024-05-21T21:59:26.586 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51806 ms_dispat ceph cwd unknown /proc/50548/task/51806/cwd (readlink: Permission denied) 2024-05-21T21:59:26.586 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51806 ms_dispat ceph rtd unknown /proc/50548/task/51806/root (readlink: Permission denied) 2024-05-21T21:59:26.586 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51806 ms_dispat ceph txt unknown /proc/50548/task/51806/exe (readlink: Permission denied) 2024-05-21T21:59:26.586 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51806 ms_dispat ceph NOFD /proc/50548/task/51806/fd (opendir: Permission denied) 2024-05-21T21:59:26.586 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51807 ms_local ceph cwd unknown /proc/50548/task/51807/cwd (readlink: Permission denied) 2024-05-21T21:59:26.586 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51807 ms_local ceph rtd unknown /proc/50548/task/51807/root (readlink: Permission denied) 2024-05-21T21:59:26.586 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51807 ms_local ceph txt unknown /proc/50548/task/51807/exe (readlink: Permission denied) 2024-05-21T21:59:26.586 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51807 ms_local ceph NOFD /proc/50548/task/51807/fd (opendir: Permission denied) 2024-05-21T21:59:26.586 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51808 safe_time ceph cwd unknown /proc/50548/task/51808/cwd (readlink: Permission denied) 2024-05-21T21:59:26.586 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51808 safe_time ceph rtd unknown /proc/50548/task/51808/root (readlink: Permission denied) 2024-05-21T21:59:26.587 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51808 safe_time ceph txt unknown /proc/50548/task/51808/exe (readlink: Permission denied) 2024-05-21T21:59:26.587 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51808 safe_time ceph NOFD /proc/50548/task/51808/fd (opendir: Permission denied) 2024-05-21T21:59:26.587 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51809 safe_time ceph cwd unknown /proc/50548/task/51809/cwd (readlink: Permission denied) 2024-05-21T21:59:26.587 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51809 safe_time ceph rtd unknown /proc/50548/task/51809/root (readlink: Permission denied) 2024-05-21T21:59:26.587 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51809 safe_time ceph txt unknown /proc/50548/task/51809/exe (readlink: Permission denied) 2024-05-21T21:59:26.587 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51809 safe_time ceph NOFD /proc/50548/task/51809/fd (opendir: Permission denied) 2024-05-21T21:59:26.587 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51810 ms_dispat ceph cwd unknown /proc/50548/task/51810/cwd (readlink: Permission denied) 2024-05-21T21:59:26.587 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51810 ms_dispat ceph rtd unknown /proc/50548/task/51810/root (readlink: Permission denied) 2024-05-21T21:59:26.587 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51810 ms_dispat ceph txt unknown /proc/50548/task/51810/exe (readlink: Permission denied) 2024-05-21T21:59:26.587 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51810 ms_dispat ceph NOFD /proc/50548/task/51810/fd (opendir: Permission denied) 2024-05-21T21:59:26.587 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51811 ms_local ceph cwd unknown /proc/50548/task/51811/cwd (readlink: Permission denied) 2024-05-21T21:59:26.588 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51811 ms_local ceph rtd unknown /proc/50548/task/51811/root (readlink: Permission denied) 2024-05-21T21:59:26.588 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51811 ms_local ceph txt unknown /proc/50548/task/51811/exe (readlink: Permission denied) 2024-05-21T21:59:26.588 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51811 ms_local ceph NOFD /proc/50548/task/51811/fd (opendir: Permission denied) 2024-05-21T21:59:26.588 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51812 ms_dispat ceph cwd unknown /proc/50548/task/51812/cwd (readlink: Permission denied) 2024-05-21T21:59:26.588 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51812 ms_dispat ceph rtd unknown /proc/50548/task/51812/root (readlink: Permission denied) 2024-05-21T21:59:26.588 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51812 ms_dispat ceph txt unknown /proc/50548/task/51812/exe (readlink: Permission denied) 2024-05-21T21:59:26.588 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51812 ms_dispat ceph NOFD /proc/50548/task/51812/fd (opendir: Permission denied) 2024-05-21T21:59:26.588 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51813 ms_local ceph cwd unknown /proc/50548/task/51813/cwd (readlink: Permission denied) 2024-05-21T21:59:26.588 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51813 ms_local ceph rtd unknown /proc/50548/task/51813/root (readlink: Permission denied) 2024-05-21T21:59:26.588 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51813 ms_local ceph txt unknown /proc/50548/task/51813/exe (readlink: Permission denied) 2024-05-21T21:59:26.588 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51813 ms_local ceph NOFD /proc/50548/task/51813/fd (opendir: Permission denied) 2024-05-21T21:59:26.588 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51814 ms_dispat ceph cwd unknown /proc/50548/task/51814/cwd (readlink: Permission denied) 2024-05-21T21:59:26.588 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51814 ms_dispat ceph rtd unknown /proc/50548/task/51814/root (readlink: Permission denied) 2024-05-21T21:59:26.588 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51814 ms_dispat ceph txt unknown /proc/50548/task/51814/exe (readlink: Permission denied) 2024-05-21T21:59:26.589 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51814 ms_dispat ceph NOFD /proc/50548/task/51814/fd (opendir: Permission denied) 2024-05-21T21:59:26.589 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51815 ms_local ceph cwd unknown /proc/50548/task/51815/cwd (readlink: Permission denied) 2024-05-21T21:59:26.589 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51815 ms_local ceph rtd unknown /proc/50548/task/51815/root (readlink: Permission denied) 2024-05-21T21:59:26.589 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51815 ms_local ceph txt unknown /proc/50548/task/51815/exe (readlink: Permission denied) 2024-05-21T21:59:26.589 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51815 ms_local ceph NOFD /proc/50548/task/51815/fd (opendir: Permission denied) 2024-05-21T21:59:26.589 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51816 ms_dispat ceph cwd unknown /proc/50548/task/51816/cwd (readlink: Permission denied) 2024-05-21T21:59:26.589 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51816 ms_dispat ceph rtd unknown /proc/50548/task/51816/root (readlink: Permission denied) 2024-05-21T21:59:26.589 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51816 ms_dispat ceph txt unknown /proc/50548/task/51816/exe (readlink: Permission denied) 2024-05-21T21:59:26.589 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51816 ms_dispat ceph NOFD /proc/50548/task/51816/fd (opendir: Permission denied) 2024-05-21T21:59:26.589 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51817 ms_local ceph cwd unknown /proc/50548/task/51817/cwd (readlink: Permission denied) 2024-05-21T21:59:26.589 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51817 ms_local ceph rtd unknown /proc/50548/task/51817/root (readlink: Permission denied) 2024-05-21T21:59:26.589 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51817 ms_local ceph txt unknown /proc/50548/task/51817/exe (readlink: Permission denied) 2024-05-21T21:59:26.589 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51817 ms_local ceph NOFD /proc/50548/task/51817/fd (opendir: Permission denied) 2024-05-21T21:59:26.589 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51818 ms_dispat ceph cwd unknown /proc/50548/task/51818/cwd (readlink: Permission denied) 2024-05-21T21:59:26.590 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51818 ms_dispat ceph rtd unknown /proc/50548/task/51818/root (readlink: Permission denied) 2024-05-21T21:59:26.590 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51818 ms_dispat ceph txt unknown /proc/50548/task/51818/exe (readlink: Permission denied) 2024-05-21T21:59:26.590 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51818 ms_dispat ceph NOFD /proc/50548/task/51818/fd (opendir: Permission denied) 2024-05-21T21:59:26.590 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51819 ms_local ceph cwd unknown /proc/50548/task/51819/cwd (readlink: Permission denied) 2024-05-21T21:59:26.590 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51819 ms_local ceph rtd unknown /proc/50548/task/51819/root (readlink: Permission denied) 2024-05-21T21:59:26.590 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51819 ms_local ceph txt unknown /proc/50548/task/51819/exe (readlink: Permission denied) 2024-05-21T21:59:26.590 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51819 ms_local ceph NOFD /proc/50548/task/51819/fd (opendir: Permission denied) 2024-05-21T21:59:26.590 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51820 ms_dispat ceph cwd unknown /proc/50548/task/51820/cwd (readlink: Permission denied) 2024-05-21T21:59:26.590 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51820 ms_dispat ceph rtd unknown /proc/50548/task/51820/root (readlink: Permission denied) 2024-05-21T21:59:26.590 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51820 ms_dispat ceph txt unknown /proc/50548/task/51820/exe (readlink: Permission denied) 2024-05-21T21:59:26.590 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51820 ms_dispat ceph NOFD /proc/50548/task/51820/fd (opendir: Permission denied) 2024-05-21T21:59:26.590 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51821 ms_local ceph cwd unknown /proc/50548/task/51821/cwd (readlink: Permission denied) 2024-05-21T21:59:26.590 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51821 ms_local ceph rtd unknown /proc/50548/task/51821/root (readlink: Permission denied) 2024-05-21T21:59:26.591 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51821 ms_local ceph txt unknown /proc/50548/task/51821/exe (readlink: Permission denied) 2024-05-21T21:59:26.591 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51821 ms_local ceph NOFD /proc/50548/task/51821/fd (opendir: Permission denied) 2024-05-21T21:59:26.591 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51822 fn_anonym ceph cwd unknown /proc/50548/task/51822/cwd (readlink: Permission denied) 2024-05-21T21:59:26.591 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51822 fn_anonym ceph rtd unknown /proc/50548/task/51822/root (readlink: Permission denied) 2024-05-21T21:59:26.591 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51822 fn_anonym ceph txt unknown /proc/50548/task/51822/exe (readlink: Permission denied) 2024-05-21T21:59:26.591 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51822 fn_anonym ceph NOFD /proc/50548/task/51822/fd (opendir: Permission denied) 2024-05-21T21:59:26.591 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51823 finisher ceph cwd unknown /proc/50548/task/51823/cwd (readlink: Permission denied) 2024-05-21T21:59:26.591 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51823 finisher ceph rtd unknown /proc/50548/task/51823/root (readlink: Permission denied) 2024-05-21T21:59:26.591 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51823 finisher ceph txt unknown /proc/50548/task/51823/exe (readlink: Permission denied) 2024-05-21T21:59:26.591 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51823 finisher ceph NOFD /proc/50548/task/51823/fd (opendir: Permission denied) 2024-05-21T21:59:26.591 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51824 safe_time ceph cwd unknown /proc/50548/task/51824/cwd (readlink: Permission denied) 2024-05-21T21:59:26.591 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51824 safe_time ceph rtd unknown /proc/50548/task/51824/root (readlink: Permission denied) 2024-05-21T21:59:26.591 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51824 safe_time ceph txt unknown /proc/50548/task/51824/exe (readlink: Permission denied) 2024-05-21T21:59:26.591 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51824 safe_time ceph NOFD /proc/50548/task/51824/fd (opendir: Permission denied) 2024-05-21T21:59:26.592 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51825 safe_time ceph cwd unknown /proc/50548/task/51825/cwd (readlink: Permission denied) 2024-05-21T21:59:26.592 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51825 safe_time ceph rtd unknown /proc/50548/task/51825/root (readlink: Permission denied) 2024-05-21T21:59:26.592 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51825 safe_time ceph txt unknown /proc/50548/task/51825/exe (readlink: Permission denied) 2024-05-21T21:59:26.592 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51825 safe_time ceph NOFD /proc/50548/task/51825/fd (opendir: Permission denied) 2024-05-21T21:59:26.592 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51826 ceph-osd ceph cwd unknown /proc/50548/task/51826/cwd (readlink: Permission denied) 2024-05-21T21:59:26.592 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51826 ceph-osd ceph rtd unknown /proc/50548/task/51826/root (readlink: Permission denied) 2024-05-21T21:59:26.592 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51826 ceph-osd ceph txt unknown /proc/50548/task/51826/exe (readlink: Permission denied) 2024-05-21T21:59:26.592 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51826 ceph-osd ceph NOFD /proc/50548/task/51826/fd (opendir: Permission denied) 2024-05-21T21:59:26.592 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51827 osd_srv_a ceph cwd unknown /proc/50548/task/51827/cwd (readlink: Permission denied) 2024-05-21T21:59:26.592 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51827 osd_srv_a ceph rtd unknown /proc/50548/task/51827/root (readlink: Permission denied) 2024-05-21T21:59:26.592 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51827 osd_srv_a ceph txt unknown /proc/50548/task/51827/exe (readlink: Permission denied) 2024-05-21T21:59:26.592 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51827 osd_srv_a ceph NOFD /proc/50548/task/51827/fd (opendir: Permission denied) 2024-05-21T21:59:26.592 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51828 tp_osd_tp ceph cwd unknown /proc/50548/task/51828/cwd (readlink: Permission denied) 2024-05-21T21:59:26.592 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51828 tp_osd_tp ceph rtd unknown /proc/50548/task/51828/root (readlink: Permission denied) 2024-05-21T21:59:26.592 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51828 tp_osd_tp ceph txt unknown /proc/50548/task/51828/exe (readlink: Permission denied) 2024-05-21T21:59:26.592 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51828 tp_osd_tp ceph NOFD /proc/50548/task/51828/fd (opendir: Permission denied) 2024-05-21T21:59:26.593 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51829 tp_osd_tp ceph cwd unknown /proc/50548/task/51829/cwd (readlink: Permission denied) 2024-05-21T21:59:26.593 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51829 tp_osd_tp ceph rtd unknown /proc/50548/task/51829/root (readlink: Permission denied) 2024-05-21T21:59:26.593 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51829 tp_osd_tp ceph txt unknown /proc/50548/task/51829/exe (readlink: Permission denied) 2024-05-21T21:59:26.593 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51829 tp_osd_tp ceph NOFD /proc/50548/task/51829/fd (opendir: Permission denied) 2024-05-21T21:59:26.593 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51830 tp_osd_tp ceph cwd unknown /proc/50548/task/51830/cwd (readlink: Permission denied) 2024-05-21T21:59:26.593 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51830 tp_osd_tp ceph rtd unknown /proc/50548/task/51830/root (readlink: Permission denied) 2024-05-21T21:59:26.593 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51830 tp_osd_tp ceph txt unknown /proc/50548/task/51830/exe (readlink: Permission denied) 2024-05-21T21:59:26.593 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51830 tp_osd_tp ceph NOFD /proc/50548/task/51830/fd (opendir: Permission denied) 2024-05-21T21:59:26.593 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51831 tp_osd_tp ceph cwd unknown /proc/50548/task/51831/cwd (readlink: Permission denied) 2024-05-21T21:59:26.593 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51831 tp_osd_tp ceph rtd unknown /proc/50548/task/51831/root (readlink: Permission denied) 2024-05-21T21:59:26.593 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51831 tp_osd_tp ceph txt unknown /proc/50548/task/51831/exe (readlink: Permission denied) 2024-05-21T21:59:26.593 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51831 tp_osd_tp ceph NOFD /proc/50548/task/51831/fd (opendir: Permission denied) 2024-05-21T21:59:26.593 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51833 tp_osd_tp ceph cwd unknown /proc/50548/task/51833/cwd (readlink: Permission denied) 2024-05-21T21:59:26.593 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51833 tp_osd_tp ceph rtd unknown /proc/50548/task/51833/root (readlink: Permission denied) 2024-05-21T21:59:26.594 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51833 tp_osd_tp ceph txt unknown /proc/50548/task/51833/exe (readlink: Permission denied) 2024-05-21T21:59:26.594 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51833 tp_osd_tp ceph NOFD /proc/50548/task/51833/fd (opendir: Permission denied) 2024-05-21T21:59:26.594 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51835 tp_osd_tp ceph cwd unknown /proc/50548/task/51835/cwd (readlink: Permission denied) 2024-05-21T21:59:26.594 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51835 tp_osd_tp ceph rtd unknown /proc/50548/task/51835/root (readlink: Permission denied) 2024-05-21T21:59:26.594 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51835 tp_osd_tp ceph txt unknown /proc/50548/task/51835/exe (readlink: Permission denied) 2024-05-21T21:59:26.594 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51835 tp_osd_tp ceph NOFD /proc/50548/task/51835/fd (opendir: Permission denied) 2024-05-21T21:59:26.594 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51836 tp_osd_tp ceph cwd unknown /proc/50548/task/51836/cwd (readlink: Permission denied) 2024-05-21T21:59:26.594 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51836 tp_osd_tp ceph rtd unknown /proc/50548/task/51836/root (readlink: Permission denied) 2024-05-21T21:59:26.594 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51836 tp_osd_tp ceph txt unknown /proc/50548/task/51836/exe (readlink: Permission denied) 2024-05-21T21:59:26.594 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51836 tp_osd_tp ceph NOFD /proc/50548/task/51836/fd (opendir: Permission denied) 2024-05-21T21:59:26.594 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51837 tp_osd_tp ceph cwd unknown /proc/50548/task/51837/cwd (readlink: Permission denied) 2024-05-21T21:59:26.594 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51837 tp_osd_tp ceph rtd unknown /proc/50548/task/51837/root (readlink: Permission denied) 2024-05-21T21:59:26.594 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51837 tp_osd_tp ceph txt unknown /proc/50548/task/51837/exe (readlink: Permission denied) 2024-05-21T21:59:26.594 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51837 tp_osd_tp ceph NOFD /proc/50548/task/51837/fd (opendir: Permission denied) 2024-05-21T21:59:26.595 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51838 tp_osd_tp ceph cwd unknown /proc/50548/task/51838/cwd (readlink: Permission denied) 2024-05-21T21:59:26.595 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51838 tp_osd_tp ceph rtd unknown /proc/50548/task/51838/root (readlink: Permission denied) 2024-05-21T21:59:26.595 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51838 tp_osd_tp ceph txt unknown /proc/50548/task/51838/exe (readlink: Permission denied) 2024-05-21T21:59:26.595 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51838 tp_osd_tp ceph NOFD /proc/50548/task/51838/fd (opendir: Permission denied) 2024-05-21T21:59:26.595 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51839 tp_osd_tp ceph cwd unknown /proc/50548/task/51839/cwd (readlink: Permission denied) 2024-05-21T21:59:26.595 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51839 tp_osd_tp ceph rtd unknown /proc/50548/task/51839/root (readlink: Permission denied) 2024-05-21T21:59:26.595 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51839 tp_osd_tp ceph txt unknown /proc/50548/task/51839/exe (readlink: Permission denied) 2024-05-21T21:59:26.595 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51839 tp_osd_tp ceph NOFD /proc/50548/task/51839/fd (opendir: Permission denied) 2024-05-21T21:59:26.595 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51840 tp_osd_tp ceph cwd unknown /proc/50548/task/51840/cwd (readlink: Permission denied) 2024-05-21T21:59:26.595 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51840 tp_osd_tp ceph rtd unknown /proc/50548/task/51840/root (readlink: Permission denied) 2024-05-21T21:59:26.595 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51840 tp_osd_tp ceph txt unknown /proc/50548/task/51840/exe (readlink: Permission denied) 2024-05-21T21:59:26.595 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51840 tp_osd_tp ceph NOFD /proc/50548/task/51840/fd (opendir: Permission denied) 2024-05-21T21:59:26.595 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51841 tp_osd_tp ceph cwd unknown /proc/50548/task/51841/cwd (readlink: Permission denied) 2024-05-21T21:59:26.595 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51841 tp_osd_tp ceph rtd unknown /proc/50548/task/51841/root (readlink: Permission denied) 2024-05-21T21:59:26.595 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51841 tp_osd_tp ceph txt unknown /proc/50548/task/51841/exe (readlink: Permission denied) 2024-05-21T21:59:26.596 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51841 tp_osd_tp ceph NOFD /proc/50548/task/51841/fd (opendir: Permission denied) 2024-05-21T21:59:26.596 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51842 tp_osd_tp ceph cwd unknown /proc/50548/task/51842/cwd (readlink: Permission denied) 2024-05-21T21:59:26.596 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51842 tp_osd_tp ceph rtd unknown /proc/50548/task/51842/root (readlink: Permission denied) 2024-05-21T21:59:26.596 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51842 tp_osd_tp ceph txt unknown /proc/50548/task/51842/exe (readlink: Permission denied) 2024-05-21T21:59:26.596 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51842 tp_osd_tp ceph NOFD /proc/50548/task/51842/fd (opendir: Permission denied) 2024-05-21T21:59:26.596 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51843 tp_osd_tp ceph cwd unknown /proc/50548/task/51843/cwd (readlink: Permission denied) 2024-05-21T21:59:26.596 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51843 tp_osd_tp ceph rtd unknown /proc/50548/task/51843/root (readlink: Permission denied) 2024-05-21T21:59:26.596 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51843 tp_osd_tp ceph txt unknown /proc/50548/task/51843/exe (readlink: Permission denied) 2024-05-21T21:59:26.596 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51843 tp_osd_tp ceph NOFD /proc/50548/task/51843/fd (opendir: Permission denied) 2024-05-21T21:59:26.596 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51844 tp_osd_tp ceph cwd unknown /proc/50548/task/51844/cwd (readlink: Permission denied) 2024-05-21T21:59:26.596 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51844 tp_osd_tp ceph rtd unknown /proc/50548/task/51844/root (readlink: Permission denied) 2024-05-21T21:59:26.596 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51844 tp_osd_tp ceph txt unknown /proc/50548/task/51844/exe (readlink: Permission denied) 2024-05-21T21:59:26.596 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51844 tp_osd_tp ceph NOFD /proc/50548/task/51844/fd (opendir: Permission denied) 2024-05-21T21:59:26.596 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51845 tp_osd_tp ceph cwd unknown /proc/50548/task/51845/cwd (readlink: Permission denied) 2024-05-21T21:59:26.597 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51845 tp_osd_tp ceph rtd unknown /proc/50548/task/51845/root (readlink: Permission denied) 2024-05-21T21:59:26.597 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51845 tp_osd_tp ceph txt unknown /proc/50548/task/51845/exe (readlink: Permission denied) 2024-05-21T21:59:26.597 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51845 tp_osd_tp ceph NOFD /proc/50548/task/51845/fd (opendir: Permission denied) 2024-05-21T21:59:26.597 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51846 osd_srv_h ceph cwd unknown /proc/50548/task/51846/cwd (readlink: Permission denied) 2024-05-21T21:59:26.597 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51846 osd_srv_h ceph rtd unknown /proc/50548/task/51846/root (readlink: Permission denied) 2024-05-21T21:59:26.597 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51846 osd_srv_h ceph txt unknown /proc/50548/task/51846/exe (readlink: Permission denied) 2024-05-21T21:59:26.597 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 50548 51846 osd_srv_h ceph NOFD /proc/50548/task/51846/fd (opendir: Permission denied) 2024-05-21T21:59:26.597 INFO:tasks.workunit.client.0.smithi151.stdout:sudo 51957 root cwd unknown /proc/51957/cwd (readlink: Permission denied) 2024-05-21T21:59:26.597 INFO:tasks.workunit.client.0.smithi151.stdout:sudo 51957 root rtd unknown /proc/51957/root (readlink: Permission denied) 2024-05-21T21:59:26.597 INFO:tasks.workunit.client.0.smithi151.stdout:sudo 51957 root txt unknown /proc/51957/exe (readlink: Permission denied) 2024-05-21T21:59:26.597 INFO:tasks.workunit.client.0.smithi151.stdout:sudo 51957 root NOFD /proc/51957/fd (opendir: Permission denied) 2024-05-21T21:59:26.597 INFO:tasks.workunit.client.0.smithi151.stdout:journalct 51975 root cwd unknown /proc/51975/cwd (readlink: Permission denied) 2024-05-21T21:59:26.598 INFO:tasks.workunit.client.0.smithi151.stdout:journalct 51975 root rtd unknown /proc/51975/root (readlink: Permission denied) 2024-05-21T21:59:26.598 INFO:tasks.workunit.client.0.smithi151.stdout:journalct 51975 root txt unknown /proc/51975/exe (readlink: Permission denied) 2024-05-21T21:59:26.598 INFO:tasks.workunit.client.0.smithi151.stdout:journalct 51975 root NOFD /proc/51975/fd (opendir: Permission denied) 2024-05-21T21:59:26.598 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/u 56780 root cwd unknown /proc/56780/cwd (readlink: Permission denied) 2024-05-21T21:59:26.598 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/u 56780 root rtd unknown /proc/56780/root (readlink: Permission denied) 2024-05-21T21:59:26.598 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/u 56780 root txt unknown /proc/56780/exe (readlink: Permission denied) 2024-05-21T21:59:26.598 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/u 56780 root NOFD /proc/56780/fd (opendir: Permission denied) 2024-05-21T21:59:26.598 INFO:tasks.workunit.client.0.smithi151.stdout:conmon 57087 root cwd unknown /proc/57087/cwd (readlink: Permission denied) 2024-05-21T21:59:26.598 INFO:tasks.workunit.client.0.smithi151.stdout:conmon 57087 root rtd unknown /proc/57087/root (readlink: Permission denied) 2024-05-21T21:59:26.598 INFO:tasks.workunit.client.0.smithi151.stdout:conmon 57087 root txt unknown /proc/57087/exe (readlink: Permission denied) 2024-05-21T21:59:26.598 INFO:tasks.workunit.client.0.smithi151.stdout:conmon 57087 root NOFD /proc/57087/fd (opendir: Permission denied) 2024-05-21T21:59:26.598 INFO:tasks.workunit.client.0.smithi151.stdout:podman-in 57089 root cwd unknown /proc/57089/cwd (readlink: Permission denied) 2024-05-21T21:59:26.599 INFO:tasks.workunit.client.0.smithi151.stdout:podman-in 57089 root rtd unknown /proc/57089/root (readlink: Permission denied) 2024-05-21T21:59:26.599 INFO:tasks.workunit.client.0.smithi151.stdout:podman-in 57089 root txt unknown /proc/57089/exe (readlink: Permission denied) 2024-05-21T21:59:26.599 INFO:tasks.workunit.client.0.smithi151.stdout:podman-in 57089 root NOFD /proc/57089/fd (opendir: Permission denied) 2024-05-21T21:59:26.599 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 ceph cwd unknown /proc/57091/cwd (readlink: Permission denied) 2024-05-21T21:59:26.599 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 ceph rtd unknown /proc/57091/root (readlink: Permission denied) 2024-05-21T21:59:26.599 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 ceph txt unknown /proc/57091/exe (readlink: Permission denied) 2024-05-21T21:59:26.599 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 ceph NOFD /proc/57091/fd (opendir: Permission denied) 2024-05-21T21:59:26.599 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 57092 log ceph cwd unknown /proc/57091/task/57092/cwd (readlink: Permission denied) 2024-05-21T21:59:26.599 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 57092 log ceph rtd unknown /proc/57091/task/57092/root (readlink: Permission denied) 2024-05-21T21:59:26.599 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 57092 log ceph txt unknown /proc/57091/task/57092/exe (readlink: Permission denied) 2024-05-21T21:59:26.599 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 57092 log ceph NOFD /proc/57091/task/57092/fd (opendir: Permission denied) 2024-05-21T21:59:26.599 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 57094 msgr-work ceph cwd unknown /proc/57091/task/57094/cwd (readlink: Permission denied) 2024-05-21T21:59:26.600 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 57094 msgr-work ceph rtd unknown /proc/57091/task/57094/root (readlink: Permission denied) 2024-05-21T21:59:26.600 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 57094 msgr-work ceph txt unknown /proc/57091/task/57094/exe (readlink: Permission denied) 2024-05-21T21:59:26.600 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 57094 msgr-work ceph NOFD /proc/57091/task/57094/fd (opendir: Permission denied) 2024-05-21T21:59:26.600 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 57095 msgr-work ceph cwd unknown /proc/57091/task/57095/cwd (readlink: Permission denied) 2024-05-21T21:59:26.600 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 57095 msgr-work ceph rtd unknown /proc/57091/task/57095/root (readlink: Permission denied) 2024-05-21T21:59:26.600 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 57095 msgr-work ceph txt unknown /proc/57091/task/57095/exe (readlink: Permission denied) 2024-05-21T21:59:26.600 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 57095 msgr-work ceph NOFD /proc/57091/task/57095/fd (opendir: Permission denied) 2024-05-21T21:59:26.600 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 57096 msgr-work ceph cwd unknown /proc/57091/task/57096/cwd (readlink: Permission denied) 2024-05-21T21:59:26.600 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 57096 msgr-work ceph rtd unknown /proc/57091/task/57096/root (readlink: Permission denied) 2024-05-21T21:59:26.600 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 57096 msgr-work ceph txt unknown /proc/57091/task/57096/exe (readlink: Permission denied) 2024-05-21T21:59:26.600 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 57096 msgr-work ceph NOFD /proc/57091/task/57096/fd (opendir: Permission denied) 2024-05-21T21:59:26.600 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 57100 service ceph cwd unknown /proc/57091/task/57100/cwd (readlink: Permission denied) 2024-05-21T21:59:26.600 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 57100 service ceph rtd unknown /proc/57091/task/57100/root (readlink: Permission denied) 2024-05-21T21:59:26.600 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 57100 service ceph txt unknown /proc/57091/task/57100/exe (readlink: Permission denied) 2024-05-21T21:59:26.601 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 57100 service ceph NOFD /proc/57091/task/57100/fd (opendir: Permission denied) 2024-05-21T21:59:26.601 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 57101 admin_soc ceph cwd unknown /proc/57091/task/57101/cwd (readlink: Permission denied) 2024-05-21T21:59:26.601 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 57101 admin_soc ceph rtd unknown /proc/57091/task/57101/root (readlink: Permission denied) 2024-05-21T21:59:26.601 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 57101 admin_soc ceph txt unknown /proc/57091/task/57101/exe (readlink: Permission denied) 2024-05-21T21:59:26.601 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 57101 admin_soc ceph NOFD /proc/57091/task/57101/fd (opendir: Permission denied) 2024-05-21T21:59:26.601 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 57244 signal_ha ceph cwd unknown /proc/57091/task/57244/cwd (readlink: Permission denied) 2024-05-21T21:59:26.601 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 57244 signal_ha ceph rtd unknown /proc/57091/task/57244/root (readlink: Permission denied) 2024-05-21T21:59:26.601 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 57244 signal_ha ceph txt unknown /proc/57091/task/57244/exe (readlink: Permission denied) 2024-05-21T21:59:26.601 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 57244 signal_ha ceph NOFD /proc/57091/task/57244/fd (opendir: Permission denied) 2024-05-21T21:59:26.601 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 57245 io_contex ceph cwd unknown /proc/57091/task/57245/cwd (readlink: Permission denied) 2024-05-21T21:59:26.601 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 57245 io_contex ceph rtd unknown /proc/57091/task/57245/root (readlink: Permission denied) 2024-05-21T21:59:26.601 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 57245 io_contex ceph txt unknown /proc/57091/task/57245/exe (readlink: Permission denied) 2024-05-21T21:59:26.601 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 57245 io_contex ceph NOFD /proc/57091/task/57245/fd (opendir: Permission denied) 2024-05-21T21:59:26.601 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 57246 io_contex ceph cwd unknown /proc/57091/task/57246/cwd (readlink: Permission denied) 2024-05-21T21:59:26.602 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 57246 io_contex ceph rtd unknown /proc/57091/task/57246/root (readlink: Permission denied) 2024-05-21T21:59:26.602 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 57246 io_contex ceph txt unknown /proc/57091/task/57246/exe (readlink: Permission denied) 2024-05-21T21:59:26.602 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 57246 io_contex ceph NOFD /proc/57091/task/57246/fd (opendir: Permission denied) 2024-05-21T21:59:26.602 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 57264 OpHistory ceph cwd unknown /proc/57091/task/57264/cwd (readlink: Permission denied) 2024-05-21T21:59:26.602 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 57264 OpHistory ceph rtd unknown /proc/57091/task/57264/root (readlink: Permission denied) 2024-05-21T21:59:26.602 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 57264 OpHistory ceph txt unknown /proc/57091/task/57264/exe (readlink: Permission denied) 2024-05-21T21:59:26.602 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 57264 OpHistory ceph NOFD /proc/57091/task/57264/fd (opendir: Permission denied) 2024-05-21T21:59:26.602 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 57265 ceph_time ceph cwd unknown /proc/57091/task/57265/cwd (readlink: Permission denied) 2024-05-21T21:59:26.602 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 57265 ceph_time ceph rtd unknown /proc/57091/task/57265/root (readlink: Permission denied) 2024-05-21T21:59:26.602 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 57265 ceph_time ceph txt unknown /proc/57091/task/57265/exe (readlink: Permission denied) 2024-05-21T21:59:26.602 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 57265 ceph_time ceph NOFD /proc/57091/task/57265/fd (opendir: Permission denied) 2024-05-21T21:59:26.602 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 57469 ceph-osd ceph cwd unknown /proc/57091/task/57469/cwd (readlink: Permission denied) 2024-05-21T21:59:26.602 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 57469 ceph-osd ceph rtd unknown /proc/57091/task/57469/root (readlink: Permission denied) 2024-05-21T21:59:26.602 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 57469 ceph-osd ceph txt unknown /proc/57091/task/57469/exe (readlink: Permission denied) 2024-05-21T21:59:26.603 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 57469 ceph-osd ceph NOFD /proc/57091/task/57469/fd (opendir: Permission denied) 2024-05-21T21:59:26.603 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 57470 safe_time ceph cwd unknown /proc/57091/task/57470/cwd (readlink: Permission denied) 2024-05-21T21:59:26.603 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 57470 safe_time ceph rtd unknown /proc/57091/task/57470/root (readlink: Permission denied) 2024-05-21T21:59:26.603 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 57470 safe_time ceph txt unknown /proc/57091/task/57470/exe (readlink: Permission denied) 2024-05-21T21:59:26.603 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 57470 safe_time ceph NOFD /proc/57091/task/57470/fd (opendir: Permission denied) 2024-05-21T21:59:26.603 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 57471 safe_time ceph cwd unknown /proc/57091/task/57471/cwd (readlink: Permission denied) 2024-05-21T21:59:26.603 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 57471 safe_time ceph rtd unknown /proc/57091/task/57471/root (readlink: Permission denied) 2024-05-21T21:59:26.603 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 57471 safe_time ceph txt unknown /proc/57091/task/57471/exe (readlink: Permission denied) 2024-05-21T21:59:26.603 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 57471 safe_time ceph NOFD /proc/57091/task/57471/fd (opendir: Permission denied) 2024-05-21T21:59:26.603 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 57472 safe_time ceph cwd unknown /proc/57091/task/57472/cwd (readlink: Permission denied) 2024-05-21T21:59:26.603 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 57472 safe_time ceph rtd unknown /proc/57091/task/57472/root (readlink: Permission denied) 2024-05-21T21:59:26.603 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 57472 safe_time ceph txt unknown /proc/57091/task/57472/exe (readlink: Permission denied) 2024-05-21T21:59:26.604 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 57472 safe_time ceph NOFD /proc/57091/task/57472/fd (opendir: Permission denied) 2024-05-21T21:59:26.604 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 57473 safe_time ceph cwd unknown /proc/57091/task/57473/cwd (readlink: Permission denied) 2024-05-21T21:59:26.604 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 57473 safe_time ceph rtd unknown /proc/57091/task/57473/root (readlink: Permission denied) 2024-05-21T21:59:26.604 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 57473 safe_time ceph txt unknown /proc/57091/task/57473/exe (readlink: Permission denied) 2024-05-21T21:59:26.604 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 57473 safe_time ceph NOFD /proc/57091/task/57473/fd (opendir: Permission denied) 2024-05-21T21:59:26.604 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 57474 fn_anonym ceph cwd unknown /proc/57091/task/57474/cwd (readlink: Permission denied) 2024-05-21T21:59:26.604 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 57474 fn_anonym ceph rtd unknown /proc/57091/task/57474/root (readlink: Permission denied) 2024-05-21T21:59:26.604 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 57474 fn_anonym ceph txt unknown /proc/57091/task/57474/exe (readlink: Permission denied) 2024-05-21T21:59:26.604 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 57474 fn_anonym ceph NOFD /proc/57091/task/57474/fd (opendir: Permission denied) 2024-05-21T21:59:26.604 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 57479 rocksdb:l ceph cwd unknown /proc/57091/task/57479/cwd (readlink: Permission denied) 2024-05-21T21:59:26.604 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 57479 rocksdb:l ceph rtd unknown /proc/57091/task/57479/root (readlink: Permission denied) 2024-05-21T21:59:26.604 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 57479 rocksdb:l ceph txt unknown /proc/57091/task/57479/exe (readlink: Permission denied) 2024-05-21T21:59:26.604 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 57479 rocksdb:l ceph NOFD /proc/57091/task/57479/fd (opendir: Permission denied) 2024-05-21T21:59:26.604 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 57480 rocksdb:l ceph cwd unknown /proc/57091/task/57480/cwd (readlink: Permission denied) 2024-05-21T21:59:26.605 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 57480 rocksdb:l ceph rtd unknown /proc/57091/task/57480/root (readlink: Permission denied) 2024-05-21T21:59:26.605 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 57480 rocksdb:l ceph txt unknown /proc/57091/task/57480/exe (readlink: Permission denied) 2024-05-21T21:59:26.605 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 57480 rocksdb:l ceph NOFD /proc/57091/task/57480/fd (opendir: Permission denied) 2024-05-21T21:59:26.605 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 57481 rocksdb:l ceph cwd unknown /proc/57091/task/57481/cwd (readlink: Permission denied) 2024-05-21T21:59:26.605 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 57481 rocksdb:l ceph rtd unknown /proc/57091/task/57481/root (readlink: Permission denied) 2024-05-21T21:59:26.605 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 57481 rocksdb:l ceph txt unknown /proc/57091/task/57481/exe (readlink: Permission denied) 2024-05-21T21:59:26.605 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 57481 rocksdb:l ceph NOFD /proc/57091/task/57481/fd (opendir: Permission denied) 2024-05-21T21:59:26.605 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 57482 rocksdb:h ceph cwd unknown /proc/57091/task/57482/cwd (readlink: Permission denied) 2024-05-21T21:59:26.605 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 57482 rocksdb:h ceph rtd unknown /proc/57091/task/57482/root (readlink: Permission denied) 2024-05-21T21:59:26.605 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 57482 rocksdb:h ceph txt unknown /proc/57091/task/57482/exe (readlink: Permission denied) 2024-05-21T21:59:26.605 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 57482 rocksdb:h ceph NOFD /proc/57091/task/57482/fd (opendir: Permission denied) 2024-05-21T21:59:26.605 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 57936 bstore_ai ceph cwd unknown /proc/57091/task/57936/cwd (readlink: Permission denied) 2024-05-21T21:59:26.605 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 57936 bstore_ai ceph rtd unknown /proc/57091/task/57936/root (readlink: Permission denied) 2024-05-21T21:59:26.606 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 57936 bstore_ai ceph txt unknown /proc/57091/task/57936/exe (readlink: Permission denied) 2024-05-21T21:59:26.606 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 57936 bstore_ai ceph NOFD /proc/57091/task/57936/fd (opendir: Permission denied) 2024-05-21T21:59:26.606 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58191 bstore_ai ceph cwd unknown /proc/57091/task/58191/cwd (readlink: Permission denied) 2024-05-21T21:59:26.606 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58191 bstore_ai ceph rtd unknown /proc/57091/task/58191/root (readlink: Permission denied) 2024-05-21T21:59:26.606 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58191 bstore_ai ceph txt unknown /proc/57091/task/58191/exe (readlink: Permission denied) 2024-05-21T21:59:26.606 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58191 bstore_ai ceph NOFD /proc/57091/task/58191/fd (opendir: Permission denied) 2024-05-21T21:59:26.606 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58374 ceph-osd ceph cwd unknown /proc/57091/task/58374/cwd (readlink: Permission denied) 2024-05-21T21:59:26.606 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58374 ceph-osd ceph rtd unknown /proc/57091/task/58374/root (readlink: Permission denied) 2024-05-21T21:59:26.606 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58374 ceph-osd ceph txt unknown /proc/57091/task/58374/exe (readlink: Permission denied) 2024-05-21T21:59:26.606 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58374 ceph-osd ceph NOFD /proc/57091/task/58374/fd (opendir: Permission denied) 2024-05-21T21:59:26.606 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58375 cfin ceph cwd unknown /proc/57091/task/58375/cwd (readlink: Permission denied) 2024-05-21T21:59:26.606 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58375 cfin ceph rtd unknown /proc/57091/task/58375/root (readlink: Permission denied) 2024-05-21T21:59:26.606 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58375 cfin ceph txt unknown /proc/57091/task/58375/exe (readlink: Permission denied) 2024-05-21T21:59:26.607 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58375 cfin ceph NOFD /proc/57091/task/58375/fd (opendir: Permission denied) 2024-05-21T21:59:26.607 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58376 bstore_kv ceph cwd unknown /proc/57091/task/58376/cwd (readlink: Permission denied) 2024-05-21T21:59:26.607 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58376 bstore_kv ceph rtd unknown /proc/57091/task/58376/root (readlink: Permission denied) 2024-05-21T21:59:26.607 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58376 bstore_kv ceph txt unknown /proc/57091/task/58376/exe (readlink: Permission denied) 2024-05-21T21:59:26.607 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58376 bstore_kv ceph NOFD /proc/57091/task/58376/fd (opendir: Permission denied) 2024-05-21T21:59:26.607 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58377 bstore_kv ceph cwd unknown /proc/57091/task/58377/cwd (readlink: Permission denied) 2024-05-21T21:59:26.607 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58377 bstore_kv ceph rtd unknown /proc/57091/task/58377/root (readlink: Permission denied) 2024-05-21T21:59:26.607 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58377 bstore_kv ceph txt unknown /proc/57091/task/58377/exe (readlink: Permission denied) 2024-05-21T21:59:26.607 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58377 bstore_kv ceph NOFD /proc/57091/task/58377/fd (opendir: Permission denied) 2024-05-21T21:59:26.607 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58378 bstore_me ceph cwd unknown /proc/57091/task/58378/cwd (readlink: Permission denied) 2024-05-21T21:59:26.607 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58378 bstore_me ceph rtd unknown /proc/57091/task/58378/root (readlink: Permission denied) 2024-05-21T21:59:26.607 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58378 bstore_me ceph txt unknown /proc/57091/task/58378/exe (readlink: Permission denied) 2024-05-21T21:59:26.607 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58378 bstore_me ceph NOFD /proc/57091/task/58378/fd (opendir: Permission denied) 2024-05-21T21:59:26.607 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58382 ms_dispat ceph cwd unknown /proc/57091/task/58382/cwd (readlink: Permission denied) 2024-05-21T21:59:26.608 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58382 ms_dispat ceph rtd unknown /proc/57091/task/58382/root (readlink: Permission denied) 2024-05-21T21:59:26.608 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58382 ms_dispat ceph txt unknown /proc/57091/task/58382/exe (readlink: Permission denied) 2024-05-21T21:59:26.608 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58382 ms_dispat ceph NOFD /proc/57091/task/58382/fd (opendir: Permission denied) 2024-05-21T21:59:26.608 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58383 ms_local ceph cwd unknown /proc/57091/task/58383/cwd (readlink: Permission denied) 2024-05-21T21:59:26.608 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58383 ms_local ceph rtd unknown /proc/57091/task/58383/root (readlink: Permission denied) 2024-05-21T21:59:26.608 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58383 ms_local ceph txt unknown /proc/57091/task/58383/exe (readlink: Permission denied) 2024-05-21T21:59:26.608 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58383 ms_local ceph NOFD /proc/57091/task/58383/fd (opendir: Permission denied) 2024-05-21T21:59:26.608 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58384 safe_time ceph cwd unknown /proc/57091/task/58384/cwd (readlink: Permission denied) 2024-05-21T21:59:26.608 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58384 safe_time ceph rtd unknown /proc/57091/task/58384/root (readlink: Permission denied) 2024-05-21T21:59:26.608 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58384 safe_time ceph txt unknown /proc/57091/task/58384/exe (readlink: Permission denied) 2024-05-21T21:59:26.608 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58384 safe_time ceph NOFD /proc/57091/task/58384/fd (opendir: Permission denied) 2024-05-21T21:59:26.608 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58385 safe_time ceph cwd unknown /proc/57091/task/58385/cwd (readlink: Permission denied) 2024-05-21T21:59:26.608 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58385 safe_time ceph rtd unknown /proc/57091/task/58385/root (readlink: Permission denied) 2024-05-21T21:59:26.608 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58385 safe_time ceph txt unknown /proc/57091/task/58385/exe (readlink: Permission denied) 2024-05-21T21:59:26.609 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58385 safe_time ceph NOFD /proc/57091/task/58385/fd (opendir: Permission denied) 2024-05-21T21:59:26.609 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58386 ms_dispat ceph cwd unknown /proc/57091/task/58386/cwd (readlink: Permission denied) 2024-05-21T21:59:26.609 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58386 ms_dispat ceph rtd unknown /proc/57091/task/58386/root (readlink: Permission denied) 2024-05-21T21:59:26.609 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58386 ms_dispat ceph txt unknown /proc/57091/task/58386/exe (readlink: Permission denied) 2024-05-21T21:59:26.609 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58386 ms_dispat ceph NOFD /proc/57091/task/58386/fd (opendir: Permission denied) 2024-05-21T21:59:26.609 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58387 ms_local ceph cwd unknown /proc/57091/task/58387/cwd (readlink: Permission denied) 2024-05-21T21:59:26.609 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58387 ms_local ceph rtd unknown /proc/57091/task/58387/root (readlink: Permission denied) 2024-05-21T21:59:26.609 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58387 ms_local ceph txt unknown /proc/57091/task/58387/exe (readlink: Permission denied) 2024-05-21T21:59:26.609 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58387 ms_local ceph NOFD /proc/57091/task/58387/fd (opendir: Permission denied) 2024-05-21T21:59:26.609 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58388 ms_dispat ceph cwd unknown /proc/57091/task/58388/cwd (readlink: Permission denied) 2024-05-21T21:59:26.609 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58388 ms_dispat ceph rtd unknown /proc/57091/task/58388/root (readlink: Permission denied) 2024-05-21T21:59:26.609 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58388 ms_dispat ceph txt unknown /proc/57091/task/58388/exe (readlink: Permission denied) 2024-05-21T21:59:26.609 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58388 ms_dispat ceph NOFD /proc/57091/task/58388/fd (opendir: Permission denied) 2024-05-21T21:59:26.609 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58389 ms_local ceph cwd unknown /proc/57091/task/58389/cwd (readlink: Permission denied) 2024-05-21T21:59:26.609 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58389 ms_local ceph rtd unknown /proc/57091/task/58389/root (readlink: Permission denied) 2024-05-21T21:59:26.610 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58389 ms_local ceph txt unknown /proc/57091/task/58389/exe (readlink: Permission denied) 2024-05-21T21:59:26.610 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58389 ms_local ceph NOFD /proc/57091/task/58389/fd (opendir: Permission denied) 2024-05-21T21:59:26.610 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58390 ms_dispat ceph cwd unknown /proc/57091/task/58390/cwd (readlink: Permission denied) 2024-05-21T21:59:26.610 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58390 ms_dispat ceph rtd unknown /proc/57091/task/58390/root (readlink: Permission denied) 2024-05-21T21:59:26.610 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58390 ms_dispat ceph txt unknown /proc/57091/task/58390/exe (readlink: Permission denied) 2024-05-21T21:59:26.610 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58390 ms_dispat ceph NOFD /proc/57091/task/58390/fd (opendir: Permission denied) 2024-05-21T21:59:26.610 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58391 ms_local ceph cwd unknown /proc/57091/task/58391/cwd (readlink: Permission denied) 2024-05-21T21:59:26.610 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58391 ms_local ceph rtd unknown /proc/57091/task/58391/root (readlink: Permission denied) 2024-05-21T21:59:26.610 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58391 ms_local ceph txt unknown /proc/57091/task/58391/exe (readlink: Permission denied) 2024-05-21T21:59:26.610 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58391 ms_local ceph NOFD /proc/57091/task/58391/fd (opendir: Permission denied) 2024-05-21T21:59:26.610 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58392 ms_dispat ceph cwd unknown /proc/57091/task/58392/cwd (readlink: Permission denied) 2024-05-21T21:59:26.610 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58392 ms_dispat ceph rtd unknown /proc/57091/task/58392/root (readlink: Permission denied) 2024-05-21T21:59:26.610 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58392 ms_dispat ceph txt unknown /proc/57091/task/58392/exe (readlink: Permission denied) 2024-05-21T21:59:26.610 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58392 ms_dispat ceph NOFD /proc/57091/task/58392/fd (opendir: Permission denied) 2024-05-21T21:59:26.611 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58393 ms_local ceph cwd unknown /proc/57091/task/58393/cwd (readlink: Permission denied) 2024-05-21T21:59:26.611 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58393 ms_local ceph rtd unknown /proc/57091/task/58393/root (readlink: Permission denied) 2024-05-21T21:59:26.611 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58393 ms_local ceph txt unknown /proc/57091/task/58393/exe (readlink: Permission denied) 2024-05-21T21:59:26.611 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58393 ms_local ceph NOFD /proc/57091/task/58393/fd (opendir: Permission denied) 2024-05-21T21:59:26.611 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58394 ms_dispat ceph cwd unknown /proc/57091/task/58394/cwd (readlink: Permission denied) 2024-05-21T21:59:26.611 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58394 ms_dispat ceph rtd unknown /proc/57091/task/58394/root (readlink: Permission denied) 2024-05-21T21:59:26.611 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58394 ms_dispat ceph txt unknown /proc/57091/task/58394/exe (readlink: Permission denied) 2024-05-21T21:59:26.611 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58394 ms_dispat ceph NOFD /proc/57091/task/58394/fd (opendir: Permission denied) 2024-05-21T21:59:26.611 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58395 ms_local ceph cwd unknown /proc/57091/task/58395/cwd (readlink: Permission denied) 2024-05-21T21:59:26.611 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58395 ms_local ceph rtd unknown /proc/57091/task/58395/root (readlink: Permission denied) 2024-05-21T21:59:26.611 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58395 ms_local ceph txt unknown /proc/57091/task/58395/exe (readlink: Permission denied) 2024-05-21T21:59:26.611 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58395 ms_local ceph NOFD /proc/57091/task/58395/fd (opendir: Permission denied) 2024-05-21T21:59:26.611 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58396 ms_dispat ceph cwd unknown /proc/57091/task/58396/cwd (readlink: Permission denied) 2024-05-21T21:59:26.611 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58396 ms_dispat ceph rtd unknown /proc/57091/task/58396/root (readlink: Permission denied) 2024-05-21T21:59:26.611 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58396 ms_dispat ceph txt unknown /proc/57091/task/58396/exe (readlink: Permission denied) 2024-05-21T21:59:26.612 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58396 ms_dispat ceph NOFD /proc/57091/task/58396/fd (opendir: Permission denied) 2024-05-21T21:59:26.612 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58397 ms_local ceph cwd unknown /proc/57091/task/58397/cwd (readlink: Permission denied) 2024-05-21T21:59:26.612 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58397 ms_local ceph rtd unknown /proc/57091/task/58397/root (readlink: Permission denied) 2024-05-21T21:59:26.612 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58397 ms_local ceph txt unknown /proc/57091/task/58397/exe (readlink: Permission denied) 2024-05-21T21:59:26.612 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58397 ms_local ceph NOFD /proc/57091/task/58397/fd (opendir: Permission denied) 2024-05-21T21:59:26.612 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58398 fn_anonym ceph cwd unknown /proc/57091/task/58398/cwd (readlink: Permission denied) 2024-05-21T21:59:26.612 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58398 fn_anonym ceph rtd unknown /proc/57091/task/58398/root (readlink: Permission denied) 2024-05-21T21:59:26.612 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58398 fn_anonym ceph txt unknown /proc/57091/task/58398/exe (readlink: Permission denied) 2024-05-21T21:59:26.612 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58398 fn_anonym ceph NOFD /proc/57091/task/58398/fd (opendir: Permission denied) 2024-05-21T21:59:26.612 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58399 finisher ceph cwd unknown /proc/57091/task/58399/cwd (readlink: Permission denied) 2024-05-21T21:59:26.612 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58399 finisher ceph rtd unknown /proc/57091/task/58399/root (readlink: Permission denied) 2024-05-21T21:59:26.612 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58399 finisher ceph txt unknown /proc/57091/task/58399/exe (readlink: Permission denied) 2024-05-21T21:59:26.612 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58399 finisher ceph NOFD /proc/57091/task/58399/fd (opendir: Permission denied) 2024-05-21T21:59:26.612 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58400 safe_time ceph cwd unknown /proc/57091/task/58400/cwd (readlink: Permission denied) 2024-05-21T21:59:26.613 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58400 safe_time ceph rtd unknown /proc/57091/task/58400/root (readlink: Permission denied) 2024-05-21T21:59:26.613 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58400 safe_time ceph txt unknown /proc/57091/task/58400/exe (readlink: Permission denied) 2024-05-21T21:59:26.613 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58400 safe_time ceph NOFD /proc/57091/task/58400/fd (opendir: Permission denied) 2024-05-21T21:59:26.613 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58401 safe_time ceph cwd unknown /proc/57091/task/58401/cwd (readlink: Permission denied) 2024-05-21T21:59:26.613 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58401 safe_time ceph rtd unknown /proc/57091/task/58401/root (readlink: Permission denied) 2024-05-21T21:59:26.613 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58401 safe_time ceph txt unknown /proc/57091/task/58401/exe (readlink: Permission denied) 2024-05-21T21:59:26.613 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58401 safe_time ceph NOFD /proc/57091/task/58401/fd (opendir: Permission denied) 2024-05-21T21:59:26.613 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58402 ceph-osd ceph cwd unknown /proc/57091/task/58402/cwd (readlink: Permission denied) 2024-05-21T21:59:26.613 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58402 ceph-osd ceph rtd unknown /proc/57091/task/58402/root (readlink: Permission denied) 2024-05-21T21:59:26.613 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58402 ceph-osd ceph txt unknown /proc/57091/task/58402/exe (readlink: Permission denied) 2024-05-21T21:59:26.613 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58402 ceph-osd ceph NOFD /proc/57091/task/58402/fd (opendir: Permission denied) 2024-05-21T21:59:26.613 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58403 osd_srv_a ceph cwd unknown /proc/57091/task/58403/cwd (readlink: Permission denied) 2024-05-21T21:59:26.613 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58403 osd_srv_a ceph rtd unknown /proc/57091/task/58403/root (readlink: Permission denied) 2024-05-21T21:59:26.613 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58403 osd_srv_a ceph txt unknown /proc/57091/task/58403/exe (readlink: Permission denied) 2024-05-21T21:59:26.613 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58403 osd_srv_a ceph NOFD /proc/57091/task/58403/fd (opendir: Permission denied) 2024-05-21T21:59:26.614 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58404 tp_osd_tp ceph cwd unknown /proc/57091/task/58404/cwd (readlink: Permission denied) 2024-05-21T21:59:26.614 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58404 tp_osd_tp ceph rtd unknown /proc/57091/task/58404/root (readlink: Permission denied) 2024-05-21T21:59:26.614 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58404 tp_osd_tp ceph txt unknown /proc/57091/task/58404/exe (readlink: Permission denied) 2024-05-21T21:59:26.614 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58404 tp_osd_tp ceph NOFD /proc/57091/task/58404/fd (opendir: Permission denied) 2024-05-21T21:59:26.614 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58405 tp_osd_tp ceph cwd unknown /proc/57091/task/58405/cwd (readlink: Permission denied) 2024-05-21T21:59:26.614 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58405 tp_osd_tp ceph rtd unknown /proc/57091/task/58405/root (readlink: Permission denied) 2024-05-21T21:59:26.614 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58405 tp_osd_tp ceph txt unknown /proc/57091/task/58405/exe (readlink: Permission denied) 2024-05-21T21:59:26.614 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58405 tp_osd_tp ceph NOFD /proc/57091/task/58405/fd (opendir: Permission denied) 2024-05-21T21:59:26.614 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58406 tp_osd_tp ceph cwd unknown /proc/57091/task/58406/cwd (readlink: Permission denied) 2024-05-21T21:59:26.614 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58406 tp_osd_tp ceph rtd unknown /proc/57091/task/58406/root (readlink: Permission denied) 2024-05-21T21:59:26.614 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58406 tp_osd_tp ceph txt unknown /proc/57091/task/58406/exe (readlink: Permission denied) 2024-05-21T21:59:26.614 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58406 tp_osd_tp ceph NOFD /proc/57091/task/58406/fd (opendir: Permission denied) 2024-05-21T21:59:26.614 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58407 tp_osd_tp ceph cwd unknown /proc/57091/task/58407/cwd (readlink: Permission denied) 2024-05-21T21:59:26.614 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58407 tp_osd_tp ceph rtd unknown /proc/57091/task/58407/root (readlink: Permission denied) 2024-05-21T21:59:26.615 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58407 tp_osd_tp ceph txt unknown /proc/57091/task/58407/exe (readlink: Permission denied) 2024-05-21T21:59:26.615 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58407 tp_osd_tp ceph NOFD /proc/57091/task/58407/fd (opendir: Permission denied) 2024-05-21T21:59:26.615 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58408 tp_osd_tp ceph cwd unknown /proc/57091/task/58408/cwd (readlink: Permission denied) 2024-05-21T21:59:26.615 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58408 tp_osd_tp ceph rtd unknown /proc/57091/task/58408/root (readlink: Permission denied) 2024-05-21T21:59:26.615 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58408 tp_osd_tp ceph txt unknown /proc/57091/task/58408/exe (readlink: Permission denied) 2024-05-21T21:59:26.615 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58408 tp_osd_tp ceph NOFD /proc/57091/task/58408/fd (opendir: Permission denied) 2024-05-21T21:59:26.615 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58409 tp_osd_tp ceph cwd unknown /proc/57091/task/58409/cwd (readlink: Permission denied) 2024-05-21T21:59:26.615 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58409 tp_osd_tp ceph rtd unknown /proc/57091/task/58409/root (readlink: Permission denied) 2024-05-21T21:59:26.615 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58409 tp_osd_tp ceph txt unknown /proc/57091/task/58409/exe (readlink: Permission denied) 2024-05-21T21:59:26.615 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58409 tp_osd_tp ceph NOFD /proc/57091/task/58409/fd (opendir: Permission denied) 2024-05-21T21:59:26.615 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58410 tp_osd_tp ceph cwd unknown /proc/57091/task/58410/cwd (readlink: Permission denied) 2024-05-21T21:59:26.615 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58410 tp_osd_tp ceph rtd unknown /proc/57091/task/58410/root (readlink: Permission denied) 2024-05-21T21:59:26.615 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58410 tp_osd_tp ceph txt unknown /proc/57091/task/58410/exe (readlink: Permission denied) 2024-05-21T21:59:26.615 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58410 tp_osd_tp ceph NOFD /proc/57091/task/58410/fd (opendir: Permission denied) 2024-05-21T21:59:26.615 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58411 tp_osd_tp ceph cwd unknown /proc/57091/task/58411/cwd (readlink: Permission denied) 2024-05-21T21:59:26.616 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58411 tp_osd_tp ceph rtd unknown /proc/57091/task/58411/root (readlink: Permission denied) 2024-05-21T21:59:26.616 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58411 tp_osd_tp ceph txt unknown /proc/57091/task/58411/exe (readlink: Permission denied) 2024-05-21T21:59:26.616 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58411 tp_osd_tp ceph NOFD /proc/57091/task/58411/fd (opendir: Permission denied) 2024-05-21T21:59:26.616 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58412 tp_osd_tp ceph cwd unknown /proc/57091/task/58412/cwd (readlink: Permission denied) 2024-05-21T21:59:26.616 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58412 tp_osd_tp ceph rtd unknown /proc/57091/task/58412/root (readlink: Permission denied) 2024-05-21T21:59:26.616 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58412 tp_osd_tp ceph txt unknown /proc/57091/task/58412/exe (readlink: Permission denied) 2024-05-21T21:59:26.616 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58412 tp_osd_tp ceph NOFD /proc/57091/task/58412/fd (opendir: Permission denied) 2024-05-21T21:59:26.616 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58413 tp_osd_tp ceph cwd unknown /proc/57091/task/58413/cwd (readlink: Permission denied) 2024-05-21T21:59:26.616 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58413 tp_osd_tp ceph rtd unknown /proc/57091/task/58413/root (readlink: Permission denied) 2024-05-21T21:59:26.616 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58413 tp_osd_tp ceph txt unknown /proc/57091/task/58413/exe (readlink: Permission denied) 2024-05-21T21:59:26.616 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58413 tp_osd_tp ceph NOFD /proc/57091/task/58413/fd (opendir: Permission denied) 2024-05-21T21:59:26.616 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58414 tp_osd_tp ceph cwd unknown /proc/57091/task/58414/cwd (readlink: Permission denied) 2024-05-21T21:59:26.616 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58414 tp_osd_tp ceph rtd unknown /proc/57091/task/58414/root (readlink: Permission denied) 2024-05-21T21:59:26.616 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58414 tp_osd_tp ceph txt unknown /proc/57091/task/58414/exe (readlink: Permission denied) 2024-05-21T21:59:26.616 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58414 tp_osd_tp ceph NOFD /proc/57091/task/58414/fd (opendir: Permission denied) 2024-05-21T21:59:26.616 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58415 tp_osd_tp ceph cwd unknown /proc/57091/task/58415/cwd (readlink: Permission denied) 2024-05-21T21:59:26.617 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58415 tp_osd_tp ceph rtd unknown /proc/57091/task/58415/root (readlink: Permission denied) 2024-05-21T21:59:26.617 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58415 tp_osd_tp ceph txt unknown /proc/57091/task/58415/exe (readlink: Permission denied) 2024-05-21T21:59:26.617 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58415 tp_osd_tp ceph NOFD /proc/57091/task/58415/fd (opendir: Permission denied) 2024-05-21T21:59:26.617 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58416 tp_osd_tp ceph cwd unknown /proc/57091/task/58416/cwd (readlink: Permission denied) 2024-05-21T21:59:26.617 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58416 tp_osd_tp ceph rtd unknown /proc/57091/task/58416/root (readlink: Permission denied) 2024-05-21T21:59:26.617 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58416 tp_osd_tp ceph txt unknown /proc/57091/task/58416/exe (readlink: Permission denied) 2024-05-21T21:59:26.617 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58416 tp_osd_tp ceph NOFD /proc/57091/task/58416/fd (opendir: Permission denied) 2024-05-21T21:59:26.617 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58417 tp_osd_tp ceph cwd unknown /proc/57091/task/58417/cwd (readlink: Permission denied) 2024-05-21T21:59:26.617 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58417 tp_osd_tp ceph rtd unknown /proc/57091/task/58417/root (readlink: Permission denied) 2024-05-21T21:59:26.617 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58417 tp_osd_tp ceph txt unknown /proc/57091/task/58417/exe (readlink: Permission denied) 2024-05-21T21:59:26.617 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58417 tp_osd_tp ceph NOFD /proc/57091/task/58417/fd (opendir: Permission denied) 2024-05-21T21:59:26.617 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58418 tp_osd_tp ceph cwd unknown /proc/57091/task/58418/cwd (readlink: Permission denied) 2024-05-21T21:59:26.617 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58418 tp_osd_tp ceph rtd unknown /proc/57091/task/58418/root (readlink: Permission denied) 2024-05-21T21:59:26.618 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58418 tp_osd_tp ceph txt unknown /proc/57091/task/58418/exe (readlink: Permission denied) 2024-05-21T21:59:26.618 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58418 tp_osd_tp ceph NOFD /proc/57091/task/58418/fd (opendir: Permission denied) 2024-05-21T21:59:26.618 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58419 tp_osd_tp ceph cwd unknown /proc/57091/task/58419/cwd (readlink: Permission denied) 2024-05-21T21:59:26.618 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58419 tp_osd_tp ceph rtd unknown /proc/57091/task/58419/root (readlink: Permission denied) 2024-05-21T21:59:26.618 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58419 tp_osd_tp ceph txt unknown /proc/57091/task/58419/exe (readlink: Permission denied) 2024-05-21T21:59:26.618 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58419 tp_osd_tp ceph NOFD /proc/57091/task/58419/fd (opendir: Permission denied) 2024-05-21T21:59:26.618 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58420 osd_srv_h ceph cwd unknown /proc/57091/task/58420/cwd (readlink: Permission denied) 2024-05-21T21:59:26.618 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58420 osd_srv_h ceph rtd unknown /proc/57091/task/58420/root (readlink: Permission denied) 2024-05-21T21:59:26.618 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58420 osd_srv_h ceph txt unknown /proc/57091/task/58420/exe (readlink: Permission denied) 2024-05-21T21:59:26.618 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 57091 58420 osd_srv_h ceph NOFD /proc/57091/task/58420/fd (opendir: Permission denied) 2024-05-21T21:59:26.618 INFO:tasks.workunit.client.0.smithi151.stdout:sudo 58590 root cwd unknown /proc/58590/cwd (readlink: Permission denied) 2024-05-21T21:59:26.618 INFO:tasks.workunit.client.0.smithi151.stdout:sudo 58590 root rtd unknown /proc/58590/root (readlink: Permission denied) 2024-05-21T21:59:26.618 INFO:tasks.workunit.client.0.smithi151.stdout:sudo 58590 root txt unknown /proc/58590/exe (readlink: Permission denied) 2024-05-21T21:59:26.619 INFO:tasks.workunit.client.0.smithi151.stdout:sudo 58590 root NOFD /proc/58590/fd (opendir: Permission denied) 2024-05-21T21:59:26.619 INFO:tasks.workunit.client.0.smithi151.stdout:journalct 58616 root cwd unknown /proc/58616/cwd (readlink: Permission denied) 2024-05-21T21:59:26.619 INFO:tasks.workunit.client.0.smithi151.stdout:journalct 58616 root rtd unknown /proc/58616/root (readlink: Permission denied) 2024-05-21T21:59:26.619 INFO:tasks.workunit.client.0.smithi151.stdout:journalct 58616 root txt unknown /proc/58616/exe (readlink: Permission denied) 2024-05-21T21:59:26.619 INFO:tasks.workunit.client.0.smithi151.stdout:journalct 58616 root NOFD /proc/58616/fd (opendir: Permission denied) 2024-05-21T21:59:26.619 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/6 62696 root cwd unknown /proc/62696/cwd (readlink: Permission denied) 2024-05-21T21:59:26.619 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/6 62696 root rtd unknown /proc/62696/root (readlink: Permission denied) 2024-05-21T21:59:26.619 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/6 62696 root txt unknown /proc/62696/exe (readlink: Permission denied) 2024-05-21T21:59:26.619 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/6 62696 root NOFD /proc/62696/fd (opendir: Permission denied) 2024-05-21T21:59:26.619 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/0 63257 root cwd unknown /proc/63257/cwd (readlink: Permission denied) 2024-05-21T21:59:26.619 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/0 63257 root rtd unknown /proc/63257/root (readlink: Permission denied) 2024-05-21T21:59:26.619 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/0 63257 root txt unknown /proc/63257/exe (readlink: Permission denied) 2024-05-21T21:59:26.619 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/0 63257 root NOFD /proc/63257/fd (opendir: Permission denied) 2024-05-21T21:59:26.619 INFO:tasks.workunit.client.0.smithi151.stdout:conmon 64086 root cwd unknown /proc/64086/cwd (readlink: Permission denied) 2024-05-21T21:59:26.620 INFO:tasks.workunit.client.0.smithi151.stdout:conmon 64086 root rtd unknown /proc/64086/root (readlink: Permission denied) 2024-05-21T21:59:26.620 INFO:tasks.workunit.client.0.smithi151.stdout:conmon 64086 root txt unknown /proc/64086/exe (readlink: Permission denied) 2024-05-21T21:59:26.620 INFO:tasks.workunit.client.0.smithi151.stdout:conmon 64086 root NOFD /proc/64086/fd (opendir: Permission denied) 2024-05-21T21:59:26.620 INFO:tasks.workunit.client.0.smithi151.stdout:podman-in 64088 root cwd unknown /proc/64088/cwd (readlink: Permission denied) 2024-05-21T21:59:26.620 INFO:tasks.workunit.client.0.smithi151.stdout:podman-in 64088 root rtd unknown /proc/64088/root (readlink: Permission denied) 2024-05-21T21:59:26.620 INFO:tasks.workunit.client.0.smithi151.stdout:podman-in 64088 root txt unknown /proc/64088/exe (readlink: Permission denied) 2024-05-21T21:59:26.620 INFO:tasks.workunit.client.0.smithi151.stdout:podman-in 64088 root NOFD /proc/64088/fd (opendir: Permission denied) 2024-05-21T21:59:26.620 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 ceph cwd unknown /proc/64090/cwd (readlink: Permission denied) 2024-05-21T21:59:26.620 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 ceph rtd unknown /proc/64090/root (readlink: Permission denied) 2024-05-21T21:59:26.620 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 ceph txt unknown /proc/64090/exe (readlink: Permission denied) 2024-05-21T21:59:26.620 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 ceph NOFD /proc/64090/fd (opendir: Permission denied) 2024-05-21T21:59:26.620 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 64091 log ceph cwd unknown /proc/64090/task/64091/cwd (readlink: Permission denied) 2024-05-21T21:59:26.620 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 64091 log ceph rtd unknown /proc/64090/task/64091/root (readlink: Permission denied) 2024-05-21T21:59:26.620 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 64091 log ceph txt unknown /proc/64090/task/64091/exe (readlink: Permission denied) 2024-05-21T21:59:26.620 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 64091 log ceph NOFD /proc/64090/task/64091/fd (opendir: Permission denied) 2024-05-21T21:59:26.620 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 64093 msgr-work ceph cwd unknown /proc/64090/task/64093/cwd (readlink: Permission denied) 2024-05-21T21:59:26.621 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 64093 msgr-work ceph rtd unknown /proc/64090/task/64093/root (readlink: Permission denied) 2024-05-21T21:59:26.621 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 64093 msgr-work ceph txt unknown /proc/64090/task/64093/exe (readlink: Permission denied) 2024-05-21T21:59:26.621 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 64093 msgr-work ceph NOFD /proc/64090/task/64093/fd (opendir: Permission denied) 2024-05-21T21:59:26.621 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 64094 msgr-work ceph cwd unknown /proc/64090/task/64094/cwd (readlink: Permission denied) 2024-05-21T21:59:26.621 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 64094 msgr-work ceph rtd unknown /proc/64090/task/64094/root (readlink: Permission denied) 2024-05-21T21:59:26.621 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 64094 msgr-work ceph txt unknown /proc/64090/task/64094/exe (readlink: Permission denied) 2024-05-21T21:59:26.621 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 64094 msgr-work ceph NOFD /proc/64090/task/64094/fd (opendir: Permission denied) 2024-05-21T21:59:26.621 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 64095 msgr-work ceph cwd unknown /proc/64090/task/64095/cwd (readlink: Permission denied) 2024-05-21T21:59:26.621 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 64095 msgr-work ceph rtd unknown /proc/64090/task/64095/root (readlink: Permission denied) 2024-05-21T21:59:26.621 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 64095 msgr-work ceph txt unknown /proc/64090/task/64095/exe (readlink: Permission denied) 2024-05-21T21:59:26.621 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 64095 msgr-work ceph NOFD /proc/64090/task/64095/fd (opendir: Permission denied) 2024-05-21T21:59:26.621 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 64099 service ceph cwd unknown /proc/64090/task/64099/cwd (readlink: Permission denied) 2024-05-21T21:59:26.621 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 64099 service ceph rtd unknown /proc/64090/task/64099/root (readlink: Permission denied) 2024-05-21T21:59:26.621 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 64099 service ceph txt unknown /proc/64090/task/64099/exe (readlink: Permission denied) 2024-05-21T21:59:26.621 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 64099 service ceph NOFD /proc/64090/task/64099/fd (opendir: Permission denied) 2024-05-21T21:59:26.622 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 64100 admin_soc ceph cwd unknown /proc/64090/task/64100/cwd (readlink: Permission denied) 2024-05-21T21:59:26.622 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 64100 admin_soc ceph rtd unknown /proc/64090/task/64100/root (readlink: Permission denied) 2024-05-21T21:59:26.622 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 64100 admin_soc ceph txt unknown /proc/64090/task/64100/exe (readlink: Permission denied) 2024-05-21T21:59:26.622 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 64100 admin_soc ceph NOFD /proc/64090/task/64100/fd (opendir: Permission denied) 2024-05-21T21:59:26.622 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 64245 signal_ha ceph cwd unknown /proc/64090/task/64245/cwd (readlink: Permission denied) 2024-05-21T21:59:26.622 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 64245 signal_ha ceph rtd unknown /proc/64090/task/64245/root (readlink: Permission denied) 2024-05-21T21:59:26.622 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 64245 signal_ha ceph txt unknown /proc/64090/task/64245/exe (readlink: Permission denied) 2024-05-21T21:59:26.622 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 64245 signal_ha ceph NOFD /proc/64090/task/64245/fd (opendir: Permission denied) 2024-05-21T21:59:26.622 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 64246 io_contex ceph cwd unknown /proc/64090/task/64246/cwd (readlink: Permission denied) 2024-05-21T21:59:26.622 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 64246 io_contex ceph rtd unknown /proc/64090/task/64246/root (readlink: Permission denied) 2024-05-21T21:59:26.622 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 64246 io_contex ceph txt unknown /proc/64090/task/64246/exe (readlink: Permission denied) 2024-05-21T21:59:26.622 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 64246 io_contex ceph NOFD /proc/64090/task/64246/fd (opendir: Permission denied) 2024-05-21T21:59:26.622 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 64247 io_contex ceph cwd unknown /proc/64090/task/64247/cwd (readlink: Permission denied) 2024-05-21T21:59:26.623 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 64247 io_contex ceph rtd unknown /proc/64090/task/64247/root (readlink: Permission denied) 2024-05-21T21:59:26.623 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 64247 io_contex ceph txt unknown /proc/64090/task/64247/exe (readlink: Permission denied) 2024-05-21T21:59:26.623 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 64247 io_contex ceph NOFD /proc/64090/task/64247/fd (opendir: Permission denied) 2024-05-21T21:59:26.623 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 64280 OpHistory ceph cwd unknown /proc/64090/task/64280/cwd (readlink: Permission denied) 2024-05-21T21:59:26.623 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 64280 OpHistory ceph rtd unknown /proc/64090/task/64280/root (readlink: Permission denied) 2024-05-21T21:59:26.623 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 64280 OpHistory ceph txt unknown /proc/64090/task/64280/exe (readlink: Permission denied) 2024-05-21T21:59:26.623 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 64280 OpHistory ceph NOFD /proc/64090/task/64280/fd (opendir: Permission denied) 2024-05-21T21:59:26.623 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 64281 ceph_time ceph cwd unknown /proc/64090/task/64281/cwd (readlink: Permission denied) 2024-05-21T21:59:26.623 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 64281 ceph_time ceph rtd unknown /proc/64090/task/64281/root (readlink: Permission denied) 2024-05-21T21:59:26.623 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 64281 ceph_time ceph txt unknown /proc/64090/task/64281/exe (readlink: Permission denied) 2024-05-21T21:59:26.623 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 64281 ceph_time ceph NOFD /proc/64090/task/64281/fd (opendir: Permission denied) 2024-05-21T21:59:26.623 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 64495 ceph-osd ceph cwd unknown /proc/64090/task/64495/cwd (readlink: Permission denied) 2024-05-21T21:59:26.623 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 64495 ceph-osd ceph rtd unknown /proc/64090/task/64495/root (readlink: Permission denied) 2024-05-21T21:59:26.624 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 64495 ceph-osd ceph txt unknown /proc/64090/task/64495/exe (readlink: Permission denied) 2024-05-21T21:59:26.624 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 64495 ceph-osd ceph NOFD /proc/64090/task/64495/fd (opendir: Permission denied) 2024-05-21T21:59:26.624 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 64496 safe_time ceph cwd unknown /proc/64090/task/64496/cwd (readlink: Permission denied) 2024-05-21T21:59:26.624 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 64496 safe_time ceph rtd unknown /proc/64090/task/64496/root (readlink: Permission denied) 2024-05-21T21:59:26.624 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 64496 safe_time ceph txt unknown /proc/64090/task/64496/exe (readlink: Permission denied) 2024-05-21T21:59:26.624 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 64496 safe_time ceph NOFD /proc/64090/task/64496/fd (opendir: Permission denied) 2024-05-21T21:59:26.624 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 64497 safe_time ceph cwd unknown /proc/64090/task/64497/cwd (readlink: Permission denied) 2024-05-21T21:59:26.624 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 64497 safe_time ceph rtd unknown /proc/64090/task/64497/root (readlink: Permission denied) 2024-05-21T21:59:26.624 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 64497 safe_time ceph txt unknown /proc/64090/task/64497/exe (readlink: Permission denied) 2024-05-21T21:59:26.624 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 64497 safe_time ceph NOFD /proc/64090/task/64497/fd (opendir: Permission denied) 2024-05-21T21:59:26.624 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 64498 safe_time ceph cwd unknown /proc/64090/task/64498/cwd (readlink: Permission denied) 2024-05-21T21:59:26.624 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 64498 safe_time ceph rtd unknown /proc/64090/task/64498/root (readlink: Permission denied) 2024-05-21T21:59:26.624 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 64498 safe_time ceph txt unknown /proc/64090/task/64498/exe (readlink: Permission denied) 2024-05-21T21:59:26.624 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 64498 safe_time ceph NOFD /proc/64090/task/64498/fd (opendir: Permission denied) 2024-05-21T21:59:26.624 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 64499 safe_time ceph cwd unknown /proc/64090/task/64499/cwd (readlink: Permission denied) 2024-05-21T21:59:26.625 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 64499 safe_time ceph rtd unknown /proc/64090/task/64499/root (readlink: Permission denied) 2024-05-21T21:59:26.625 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 64499 safe_time ceph txt unknown /proc/64090/task/64499/exe (readlink: Permission denied) 2024-05-21T21:59:26.625 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 64499 safe_time ceph NOFD /proc/64090/task/64499/fd (opendir: Permission denied) 2024-05-21T21:59:26.625 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 64500 fn_anonym ceph cwd unknown /proc/64090/task/64500/cwd (readlink: Permission denied) 2024-05-21T21:59:26.625 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 64500 fn_anonym ceph rtd unknown /proc/64090/task/64500/root (readlink: Permission denied) 2024-05-21T21:59:26.625 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 64500 fn_anonym ceph txt unknown /proc/64090/task/64500/exe (readlink: Permission denied) 2024-05-21T21:59:26.625 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 64500 fn_anonym ceph NOFD /proc/64090/task/64500/fd (opendir: Permission denied) 2024-05-21T21:59:26.625 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 64505 rocksdb:l ceph cwd unknown /proc/64090/task/64505/cwd (readlink: Permission denied) 2024-05-21T21:59:26.625 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 64505 rocksdb:l ceph rtd unknown /proc/64090/task/64505/root (readlink: Permission denied) 2024-05-21T21:59:26.625 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 64505 rocksdb:l ceph txt unknown /proc/64090/task/64505/exe (readlink: Permission denied) 2024-05-21T21:59:26.625 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 64505 rocksdb:l ceph NOFD /proc/64090/task/64505/fd (opendir: Permission denied) 2024-05-21T21:59:26.625 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 64506 rocksdb:l ceph cwd unknown /proc/64090/task/64506/cwd (readlink: Permission denied) 2024-05-21T21:59:26.625 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 64506 rocksdb:l ceph rtd unknown /proc/64090/task/64506/root (readlink: Permission denied) 2024-05-21T21:59:26.625 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 64506 rocksdb:l ceph txt unknown /proc/64090/task/64506/exe (readlink: Permission denied) 2024-05-21T21:59:26.625 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 64506 rocksdb:l ceph NOFD /proc/64090/task/64506/fd (opendir: Permission denied) 2024-05-21T21:59:26.626 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 64507 rocksdb:l ceph cwd unknown /proc/64090/task/64507/cwd (readlink: Permission denied) 2024-05-21T21:59:26.626 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 64507 rocksdb:l ceph rtd unknown /proc/64090/task/64507/root (readlink: Permission denied) 2024-05-21T21:59:26.626 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 64507 rocksdb:l ceph txt unknown /proc/64090/task/64507/exe (readlink: Permission denied) 2024-05-21T21:59:26.626 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 64507 rocksdb:l ceph NOFD /proc/64090/task/64507/fd (opendir: Permission denied) 2024-05-21T21:59:26.626 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 64508 rocksdb:h ceph cwd unknown /proc/64090/task/64508/cwd (readlink: Permission denied) 2024-05-21T21:59:26.626 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 64508 rocksdb:h ceph rtd unknown /proc/64090/task/64508/root (readlink: Permission denied) 2024-05-21T21:59:26.626 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 64508 rocksdb:h ceph txt unknown /proc/64090/task/64508/exe (readlink: Permission denied) 2024-05-21T21:59:26.626 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 64508 rocksdb:h ceph NOFD /proc/64090/task/64508/fd (opendir: Permission denied) 2024-05-21T21:59:26.626 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 64963 bstore_ai ceph cwd unknown /proc/64090/task/64963/cwd (readlink: Permission denied) 2024-05-21T21:59:26.626 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 64963 bstore_ai ceph rtd unknown /proc/64090/task/64963/root (readlink: Permission denied) 2024-05-21T21:59:26.626 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 64963 bstore_ai ceph txt unknown /proc/64090/task/64963/exe (readlink: Permission denied) 2024-05-21T21:59:26.626 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 64963 bstore_ai ceph NOFD /proc/64090/task/64963/fd (opendir: Permission denied) 2024-05-21T21:59:26.626 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65235 bstore_ai ceph cwd unknown /proc/64090/task/65235/cwd (readlink: Permission denied) 2024-05-21T21:59:26.626 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65235 bstore_ai ceph rtd unknown /proc/64090/task/65235/root (readlink: Permission denied) 2024-05-21T21:59:26.626 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65235 bstore_ai ceph txt unknown /proc/64090/task/65235/exe (readlink: Permission denied) 2024-05-21T21:59:26.627 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65235 bstore_ai ceph NOFD /proc/64090/task/65235/fd (opendir: Permission denied) 2024-05-21T21:59:26.627 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65419 ceph-osd ceph cwd unknown /proc/64090/task/65419/cwd (readlink: Permission denied) 2024-05-21T21:59:26.627 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65419 ceph-osd ceph rtd unknown /proc/64090/task/65419/root (readlink: Permission denied) 2024-05-21T21:59:26.627 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65419 ceph-osd ceph txt unknown /proc/64090/task/65419/exe (readlink: Permission denied) 2024-05-21T21:59:26.627 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65419 ceph-osd ceph NOFD /proc/64090/task/65419/fd (opendir: Permission denied) 2024-05-21T21:59:26.627 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65420 cfin ceph cwd unknown /proc/64090/task/65420/cwd (readlink: Permission denied) 2024-05-21T21:59:26.627 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65420 cfin ceph rtd unknown /proc/64090/task/65420/root (readlink: Permission denied) 2024-05-21T21:59:26.627 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65420 cfin ceph txt unknown /proc/64090/task/65420/exe (readlink: Permission denied) 2024-05-21T21:59:26.627 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65420 cfin ceph NOFD /proc/64090/task/65420/fd (opendir: Permission denied) 2024-05-21T21:59:26.627 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65421 bstore_kv ceph cwd unknown /proc/64090/task/65421/cwd (readlink: Permission denied) 2024-05-21T21:59:26.627 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65421 bstore_kv ceph rtd unknown /proc/64090/task/65421/root (readlink: Permission denied) 2024-05-21T21:59:26.627 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65421 bstore_kv ceph txt unknown /proc/64090/task/65421/exe (readlink: Permission denied) 2024-05-21T21:59:26.627 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65421 bstore_kv ceph NOFD /proc/64090/task/65421/fd (opendir: Permission denied) 2024-05-21T21:59:26.627 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65422 bstore_kv ceph cwd unknown /proc/64090/task/65422/cwd (readlink: Permission denied) 2024-05-21T21:59:26.628 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65422 bstore_kv ceph rtd unknown /proc/64090/task/65422/root (readlink: Permission denied) 2024-05-21T21:59:26.628 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65422 bstore_kv ceph txt unknown /proc/64090/task/65422/exe (readlink: Permission denied) 2024-05-21T21:59:26.628 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65422 bstore_kv ceph NOFD /proc/64090/task/65422/fd (opendir: Permission denied) 2024-05-21T21:59:26.628 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65423 bstore_me ceph cwd unknown /proc/64090/task/65423/cwd (readlink: Permission denied) 2024-05-21T21:59:26.628 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65423 bstore_me ceph rtd unknown /proc/64090/task/65423/root (readlink: Permission denied) 2024-05-21T21:59:26.628 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65423 bstore_me ceph txt unknown /proc/64090/task/65423/exe (readlink: Permission denied) 2024-05-21T21:59:26.628 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65423 bstore_me ceph NOFD /proc/64090/task/65423/fd (opendir: Permission denied) 2024-05-21T21:59:26.628 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65432 ms_dispat ceph cwd unknown /proc/64090/task/65432/cwd (readlink: Permission denied) 2024-05-21T21:59:26.628 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65432 ms_dispat ceph rtd unknown /proc/64090/task/65432/root (readlink: Permission denied) 2024-05-21T21:59:26.628 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65432 ms_dispat ceph txt unknown /proc/64090/task/65432/exe (readlink: Permission denied) 2024-05-21T21:59:26.628 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65432 ms_dispat ceph NOFD /proc/64090/task/65432/fd (opendir: Permission denied) 2024-05-21T21:59:26.628 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65433 ms_local ceph cwd unknown /proc/64090/task/65433/cwd (readlink: Permission denied) 2024-05-21T21:59:26.628 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65433 ms_local ceph rtd unknown /proc/64090/task/65433/root (readlink: Permission denied) 2024-05-21T21:59:26.628 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65433 ms_local ceph txt unknown /proc/64090/task/65433/exe (readlink: Permission denied) 2024-05-21T21:59:26.629 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65433 ms_local ceph NOFD /proc/64090/task/65433/fd (opendir: Permission denied) 2024-05-21T21:59:26.629 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65434 safe_time ceph cwd unknown /proc/64090/task/65434/cwd (readlink: Permission denied) 2024-05-21T21:59:26.629 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65434 safe_time ceph rtd unknown /proc/64090/task/65434/root (readlink: Permission denied) 2024-05-21T21:59:26.629 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65434 safe_time ceph txt unknown /proc/64090/task/65434/exe (readlink: Permission denied) 2024-05-21T21:59:26.629 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65434 safe_time ceph NOFD /proc/64090/task/65434/fd (opendir: Permission denied) 2024-05-21T21:59:26.629 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65435 safe_time ceph cwd unknown /proc/64090/task/65435/cwd (readlink: Permission denied) 2024-05-21T21:59:26.629 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65435 safe_time ceph rtd unknown /proc/64090/task/65435/root (readlink: Permission denied) 2024-05-21T21:59:26.629 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65435 safe_time ceph txt unknown /proc/64090/task/65435/exe (readlink: Permission denied) 2024-05-21T21:59:26.629 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65435 safe_time ceph NOFD /proc/64090/task/65435/fd (opendir: Permission denied) 2024-05-21T21:59:26.629 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65436 ms_dispat ceph cwd unknown /proc/64090/task/65436/cwd (readlink: Permission denied) 2024-05-21T21:59:26.629 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65436 ms_dispat ceph rtd unknown /proc/64090/task/65436/root (readlink: Permission denied) 2024-05-21T21:59:26.629 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65436 ms_dispat ceph txt unknown /proc/64090/task/65436/exe (readlink: Permission denied) 2024-05-21T21:59:26.629 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65436 ms_dispat ceph NOFD /proc/64090/task/65436/fd (opendir: Permission denied) 2024-05-21T21:59:26.629 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65437 ms_local ceph cwd unknown /proc/64090/task/65437/cwd (readlink: Permission denied) 2024-05-21T21:59:26.629 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65437 ms_local ceph rtd unknown /proc/64090/task/65437/root (readlink: Permission denied) 2024-05-21T21:59:26.630 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65437 ms_local ceph txt unknown /proc/64090/task/65437/exe (readlink: Permission denied) 2024-05-21T21:59:26.630 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65437 ms_local ceph NOFD /proc/64090/task/65437/fd (opendir: Permission denied) 2024-05-21T21:59:26.630 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65438 ms_dispat ceph cwd unknown /proc/64090/task/65438/cwd (readlink: Permission denied) 2024-05-21T21:59:26.630 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65438 ms_dispat ceph rtd unknown /proc/64090/task/65438/root (readlink: Permission denied) 2024-05-21T21:59:26.630 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65438 ms_dispat ceph txt unknown /proc/64090/task/65438/exe (readlink: Permission denied) 2024-05-21T21:59:26.630 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65438 ms_dispat ceph NOFD /proc/64090/task/65438/fd (opendir: Permission denied) 2024-05-21T21:59:26.630 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65439 ms_local ceph cwd unknown /proc/64090/task/65439/cwd (readlink: Permission denied) 2024-05-21T21:59:26.630 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65439 ms_local ceph rtd unknown /proc/64090/task/65439/root (readlink: Permission denied) 2024-05-21T21:59:26.630 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65439 ms_local ceph txt unknown /proc/64090/task/65439/exe (readlink: Permission denied) 2024-05-21T21:59:26.630 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65439 ms_local ceph NOFD /proc/64090/task/65439/fd (opendir: Permission denied) 2024-05-21T21:59:26.630 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65440 ms_dispat ceph cwd unknown /proc/64090/task/65440/cwd (readlink: Permission denied) 2024-05-21T21:59:26.630 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65440 ms_dispat ceph rtd unknown /proc/64090/task/65440/root (readlink: Permission denied) 2024-05-21T21:59:26.630 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65440 ms_dispat ceph txt unknown /proc/64090/task/65440/exe (readlink: Permission denied) 2024-05-21T21:59:26.630 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65440 ms_dispat ceph NOFD /proc/64090/task/65440/fd (opendir: Permission denied) 2024-05-21T21:59:26.631 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65441 ms_local ceph cwd unknown /proc/64090/task/65441/cwd (readlink: Permission denied) 2024-05-21T21:59:26.631 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65441 ms_local ceph rtd unknown /proc/64090/task/65441/root (readlink: Permission denied) 2024-05-21T21:59:26.631 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65441 ms_local ceph txt unknown /proc/64090/task/65441/exe (readlink: Permission denied) 2024-05-21T21:59:26.631 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65441 ms_local ceph NOFD /proc/64090/task/65441/fd (opendir: Permission denied) 2024-05-21T21:59:26.631 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65442 ms_dispat ceph cwd unknown /proc/64090/task/65442/cwd (readlink: Permission denied) 2024-05-21T21:59:26.631 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65442 ms_dispat ceph rtd unknown /proc/64090/task/65442/root (readlink: Permission denied) 2024-05-21T21:59:26.631 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65442 ms_dispat ceph txt unknown /proc/64090/task/65442/exe (readlink: Permission denied) 2024-05-21T21:59:26.631 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65442 ms_dispat ceph NOFD /proc/64090/task/65442/fd (opendir: Permission denied) 2024-05-21T21:59:26.631 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65443 ms_local ceph cwd unknown /proc/64090/task/65443/cwd (readlink: Permission denied) 2024-05-21T21:59:26.631 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65443 ms_local ceph rtd unknown /proc/64090/task/65443/root (readlink: Permission denied) 2024-05-21T21:59:26.631 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65443 ms_local ceph txt unknown /proc/64090/task/65443/exe (readlink: Permission denied) 2024-05-21T21:59:26.631 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65443 ms_local ceph NOFD /proc/64090/task/65443/fd (opendir: Permission denied) 2024-05-21T21:59:26.631 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65444 ms_dispat ceph cwd unknown /proc/64090/task/65444/cwd (readlink: Permission denied) 2024-05-21T21:59:26.631 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65444 ms_dispat ceph rtd unknown /proc/64090/task/65444/root (readlink: Permission denied) 2024-05-21T21:59:26.632 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65444 ms_dispat ceph txt unknown /proc/64090/task/65444/exe (readlink: Permission denied) 2024-05-21T21:59:26.632 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65444 ms_dispat ceph NOFD /proc/64090/task/65444/fd (opendir: Permission denied) 2024-05-21T21:59:26.632 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65445 ms_local ceph cwd unknown /proc/64090/task/65445/cwd (readlink: Permission denied) 2024-05-21T21:59:26.632 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65445 ms_local ceph rtd unknown /proc/64090/task/65445/root (readlink: Permission denied) 2024-05-21T21:59:26.632 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65445 ms_local ceph txt unknown /proc/64090/task/65445/exe (readlink: Permission denied) 2024-05-21T21:59:26.632 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65445 ms_local ceph NOFD /proc/64090/task/65445/fd (opendir: Permission denied) 2024-05-21T21:59:26.632 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65446 ms_dispat ceph cwd unknown /proc/64090/task/65446/cwd (readlink: Permission denied) 2024-05-21T21:59:26.632 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65446 ms_dispat ceph rtd unknown /proc/64090/task/65446/root (readlink: Permission denied) 2024-05-21T21:59:26.632 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65446 ms_dispat ceph txt unknown /proc/64090/task/65446/exe (readlink: Permission denied) 2024-05-21T21:59:26.632 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65446 ms_dispat ceph NOFD /proc/64090/task/65446/fd (opendir: Permission denied) 2024-05-21T21:59:26.632 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65447 ms_local ceph cwd unknown /proc/64090/task/65447/cwd (readlink: Permission denied) 2024-05-21T21:59:26.632 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65447 ms_local ceph rtd unknown /proc/64090/task/65447/root (readlink: Permission denied) 2024-05-21T21:59:26.632 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65447 ms_local ceph txt unknown /proc/64090/task/65447/exe (readlink: Permission denied) 2024-05-21T21:59:26.632 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65447 ms_local ceph NOFD /proc/64090/task/65447/fd (opendir: Permission denied) 2024-05-21T21:59:26.632 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65448 fn_anonym ceph cwd unknown /proc/64090/task/65448/cwd (readlink: Permission denied) 2024-05-21T21:59:26.632 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65448 fn_anonym ceph rtd unknown /proc/64090/task/65448/root (readlink: Permission denied) 2024-05-21T21:59:26.633 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65448 fn_anonym ceph txt unknown /proc/64090/task/65448/exe (readlink: Permission denied) 2024-05-21T21:59:26.633 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65448 fn_anonym ceph NOFD /proc/64090/task/65448/fd (opendir: Permission denied) 2024-05-21T21:59:26.633 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65449 finisher ceph cwd unknown /proc/64090/task/65449/cwd (readlink: Permission denied) 2024-05-21T21:59:26.633 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65449 finisher ceph rtd unknown /proc/64090/task/65449/root (readlink: Permission denied) 2024-05-21T21:59:26.633 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65449 finisher ceph txt unknown /proc/64090/task/65449/exe (readlink: Permission denied) 2024-05-21T21:59:26.633 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65449 finisher ceph NOFD /proc/64090/task/65449/fd (opendir: Permission denied) 2024-05-21T21:59:26.633 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65450 safe_time ceph cwd unknown /proc/64090/task/65450/cwd (readlink: Permission denied) 2024-05-21T21:59:26.633 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65450 safe_time ceph rtd unknown /proc/64090/task/65450/root (readlink: Permission denied) 2024-05-21T21:59:26.633 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65450 safe_time ceph txt unknown /proc/64090/task/65450/exe (readlink: Permission denied) 2024-05-21T21:59:26.633 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65450 safe_time ceph NOFD /proc/64090/task/65450/fd (opendir: Permission denied) 2024-05-21T21:59:26.633 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65451 safe_time ceph cwd unknown /proc/64090/task/65451/cwd (readlink: Permission denied) 2024-05-21T21:59:26.633 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65451 safe_time ceph rtd unknown /proc/64090/task/65451/root (readlink: Permission denied) 2024-05-21T21:59:26.633 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65451 safe_time ceph txt unknown /proc/64090/task/65451/exe (readlink: Permission denied) 2024-05-21T21:59:26.633 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65451 safe_time ceph NOFD /proc/64090/task/65451/fd (opendir: Permission denied) 2024-05-21T21:59:26.633 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65452 ceph-osd ceph cwd unknown /proc/64090/task/65452/cwd (readlink: Permission denied) 2024-05-21T21:59:26.634 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65452 ceph-osd ceph rtd unknown /proc/64090/task/65452/root (readlink: Permission denied) 2024-05-21T21:59:26.634 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65452 ceph-osd ceph txt unknown /proc/64090/task/65452/exe (readlink: Permission denied) 2024-05-21T21:59:26.634 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65452 ceph-osd ceph NOFD /proc/64090/task/65452/fd (opendir: Permission denied) 2024-05-21T21:59:26.634 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65453 osd_srv_a ceph cwd unknown /proc/64090/task/65453/cwd (readlink: Permission denied) 2024-05-21T21:59:26.634 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65453 osd_srv_a ceph rtd unknown /proc/64090/task/65453/root (readlink: Permission denied) 2024-05-21T21:59:26.634 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65453 osd_srv_a ceph txt unknown /proc/64090/task/65453/exe (readlink: Permission denied) 2024-05-21T21:59:26.634 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65453 osd_srv_a ceph NOFD /proc/64090/task/65453/fd (opendir: Permission denied) 2024-05-21T21:59:26.634 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65454 tp_osd_tp ceph cwd unknown /proc/64090/task/65454/cwd (readlink: Permission denied) 2024-05-21T21:59:26.634 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65454 tp_osd_tp ceph rtd unknown /proc/64090/task/65454/root (readlink: Permission denied) 2024-05-21T21:59:26.634 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65454 tp_osd_tp ceph txt unknown /proc/64090/task/65454/exe (readlink: Permission denied) 2024-05-21T21:59:26.634 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65454 tp_osd_tp ceph NOFD /proc/64090/task/65454/fd (opendir: Permission denied) 2024-05-21T21:59:26.634 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65455 tp_osd_tp ceph cwd unknown /proc/64090/task/65455/cwd (readlink: Permission denied) 2024-05-21T21:59:26.634 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65455 tp_osd_tp ceph rtd unknown /proc/64090/task/65455/root (readlink: Permission denied) 2024-05-21T21:59:26.634 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65455 tp_osd_tp ceph txt unknown /proc/64090/task/65455/exe (readlink: Permission denied) 2024-05-21T21:59:26.634 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65455 tp_osd_tp ceph NOFD /proc/64090/task/65455/fd (opendir: Permission denied) 2024-05-21T21:59:26.635 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65456 tp_osd_tp ceph cwd unknown /proc/64090/task/65456/cwd (readlink: Permission denied) 2024-05-21T21:59:26.635 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65456 tp_osd_tp ceph rtd unknown /proc/64090/task/65456/root (readlink: Permission denied) 2024-05-21T21:59:26.635 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65456 tp_osd_tp ceph txt unknown /proc/64090/task/65456/exe (readlink: Permission denied) 2024-05-21T21:59:26.635 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65456 tp_osd_tp ceph NOFD /proc/64090/task/65456/fd (opendir: Permission denied) 2024-05-21T21:59:26.635 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65457 tp_osd_tp ceph cwd unknown /proc/64090/task/65457/cwd (readlink: Permission denied) 2024-05-21T21:59:26.635 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65457 tp_osd_tp ceph rtd unknown /proc/64090/task/65457/root (readlink: Permission denied) 2024-05-21T21:59:26.635 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65457 tp_osd_tp ceph txt unknown /proc/64090/task/65457/exe (readlink: Permission denied) 2024-05-21T21:59:26.635 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65457 tp_osd_tp ceph NOFD /proc/64090/task/65457/fd (opendir: Permission denied) 2024-05-21T21:59:26.635 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65458 tp_osd_tp ceph cwd unknown /proc/64090/task/65458/cwd (readlink: Permission denied) 2024-05-21T21:59:26.635 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65458 tp_osd_tp ceph rtd unknown /proc/64090/task/65458/root (readlink: Permission denied) 2024-05-21T21:59:26.635 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65458 tp_osd_tp ceph txt unknown /proc/64090/task/65458/exe (readlink: Permission denied) 2024-05-21T21:59:26.635 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65458 tp_osd_tp ceph NOFD /proc/64090/task/65458/fd (opendir: Permission denied) 2024-05-21T21:59:26.635 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65459 tp_osd_tp ceph cwd unknown /proc/64090/task/65459/cwd (readlink: Permission denied) 2024-05-21T21:59:26.636 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65459 tp_osd_tp ceph rtd unknown /proc/64090/task/65459/root (readlink: Permission denied) 2024-05-21T21:59:26.636 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65459 tp_osd_tp ceph txt unknown /proc/64090/task/65459/exe (readlink: Permission denied) 2024-05-21T21:59:26.636 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65459 tp_osd_tp ceph NOFD /proc/64090/task/65459/fd (opendir: Permission denied) 2024-05-21T21:59:26.636 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65460 tp_osd_tp ceph cwd unknown /proc/64090/task/65460/cwd (readlink: Permission denied) 2024-05-21T21:59:26.636 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65460 tp_osd_tp ceph rtd unknown /proc/64090/task/65460/root (readlink: Permission denied) 2024-05-21T21:59:26.636 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65460 tp_osd_tp ceph txt unknown /proc/64090/task/65460/exe (readlink: Permission denied) 2024-05-21T21:59:26.636 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65460 tp_osd_tp ceph NOFD /proc/64090/task/65460/fd (opendir: Permission denied) 2024-05-21T21:59:26.636 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65461 tp_osd_tp ceph cwd unknown /proc/64090/task/65461/cwd (readlink: Permission denied) 2024-05-21T21:59:26.636 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65461 tp_osd_tp ceph rtd unknown /proc/64090/task/65461/root (readlink: Permission denied) 2024-05-21T21:59:26.636 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65461 tp_osd_tp ceph txt unknown /proc/64090/task/65461/exe (readlink: Permission denied) 2024-05-21T21:59:26.636 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65461 tp_osd_tp ceph NOFD /proc/64090/task/65461/fd (opendir: Permission denied) 2024-05-21T21:59:26.636 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65462 tp_osd_tp ceph cwd unknown /proc/64090/task/65462/cwd (readlink: Permission denied) 2024-05-21T21:59:26.636 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65462 tp_osd_tp ceph rtd unknown /proc/64090/task/65462/root (readlink: Permission denied) 2024-05-21T21:59:26.636 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65462 tp_osd_tp ceph txt unknown /proc/64090/task/65462/exe (readlink: Permission denied) 2024-05-21T21:59:26.636 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65462 tp_osd_tp ceph NOFD /proc/64090/task/65462/fd (opendir: Permission denied) 2024-05-21T21:59:26.637 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65463 tp_osd_tp ceph cwd unknown /proc/64090/task/65463/cwd (readlink: Permission denied) 2024-05-21T21:59:26.637 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65463 tp_osd_tp ceph rtd unknown /proc/64090/task/65463/root (readlink: Permission denied) 2024-05-21T21:59:26.637 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65463 tp_osd_tp ceph txt unknown /proc/64090/task/65463/exe (readlink: Permission denied) 2024-05-21T21:59:26.637 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65463 tp_osd_tp ceph NOFD /proc/64090/task/65463/fd (opendir: Permission denied) 2024-05-21T21:59:26.637 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65464 tp_osd_tp ceph cwd unknown /proc/64090/task/65464/cwd (readlink: Permission denied) 2024-05-21T21:59:26.637 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65464 tp_osd_tp ceph rtd unknown /proc/64090/task/65464/root (readlink: Permission denied) 2024-05-21T21:59:26.637 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65464 tp_osd_tp ceph txt unknown /proc/64090/task/65464/exe (readlink: Permission denied) 2024-05-21T21:59:26.637 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65464 tp_osd_tp ceph NOFD /proc/64090/task/65464/fd (opendir: Permission denied) 2024-05-21T21:59:26.637 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65465 tp_osd_tp ceph cwd unknown /proc/64090/task/65465/cwd (readlink: Permission denied) 2024-05-21T21:59:26.637 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65465 tp_osd_tp ceph rtd unknown /proc/64090/task/65465/root (readlink: Permission denied) 2024-05-21T21:59:26.637 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65465 tp_osd_tp ceph txt unknown /proc/64090/task/65465/exe (readlink: Permission denied) 2024-05-21T21:59:26.637 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65465 tp_osd_tp ceph NOFD /proc/64090/task/65465/fd (opendir: Permission denied) 2024-05-21T21:59:26.637 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65466 tp_osd_tp ceph cwd unknown /proc/64090/task/65466/cwd (readlink: Permission denied) 2024-05-21T21:59:26.637 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65466 tp_osd_tp ceph rtd unknown /proc/64090/task/65466/root (readlink: Permission denied) 2024-05-21T21:59:26.637 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65466 tp_osd_tp ceph txt unknown /proc/64090/task/65466/exe (readlink: Permission denied) 2024-05-21T21:59:26.638 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65466 tp_osd_tp ceph NOFD /proc/64090/task/65466/fd (opendir: Permission denied) 2024-05-21T21:59:26.638 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65467 tp_osd_tp ceph cwd unknown /proc/64090/task/65467/cwd (readlink: Permission denied) 2024-05-21T21:59:26.638 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65467 tp_osd_tp ceph rtd unknown /proc/64090/task/65467/root (readlink: Permission denied) 2024-05-21T21:59:26.638 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65467 tp_osd_tp ceph txt unknown /proc/64090/task/65467/exe (readlink: Permission denied) 2024-05-21T21:59:26.638 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65467 tp_osd_tp ceph NOFD /proc/64090/task/65467/fd (opendir: Permission denied) 2024-05-21T21:59:26.638 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65468 tp_osd_tp ceph cwd unknown /proc/64090/task/65468/cwd (readlink: Permission denied) 2024-05-21T21:59:26.638 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65468 tp_osd_tp ceph rtd unknown /proc/64090/task/65468/root (readlink: Permission denied) 2024-05-21T21:59:26.638 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65468 tp_osd_tp ceph txt unknown /proc/64090/task/65468/exe (readlink: Permission denied) 2024-05-21T21:59:26.638 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65468 tp_osd_tp ceph NOFD /proc/64090/task/65468/fd (opendir: Permission denied) 2024-05-21T21:59:26.638 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65469 tp_osd_tp ceph cwd unknown /proc/64090/task/65469/cwd (readlink: Permission denied) 2024-05-21T21:59:26.638 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65469 tp_osd_tp ceph rtd unknown /proc/64090/task/65469/root (readlink: Permission denied) 2024-05-21T21:59:26.638 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65469 tp_osd_tp ceph txt unknown /proc/64090/task/65469/exe (readlink: Permission denied) 2024-05-21T21:59:26.638 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65469 tp_osd_tp ceph NOFD /proc/64090/task/65469/fd (opendir: Permission denied) 2024-05-21T21:59:26.638 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65470 osd_srv_h ceph cwd unknown /proc/64090/task/65470/cwd (readlink: Permission denied) 2024-05-21T21:59:26.639 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65470 osd_srv_h ceph rtd unknown /proc/64090/task/65470/root (readlink: Permission denied) 2024-05-21T21:59:26.639 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65470 osd_srv_h ceph txt unknown /proc/64090/task/65470/exe (readlink: Permission denied) 2024-05-21T21:59:26.639 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-osd 64090 65470 osd_srv_h ceph NOFD /proc/64090/task/65470/fd (opendir: Permission denied) 2024-05-21T21:59:26.639 INFO:tasks.workunit.client.0.smithi151.stdout:sudo 65586 root cwd unknown /proc/65586/cwd (readlink: Permission denied) 2024-05-21T21:59:26.639 INFO:tasks.workunit.client.0.smithi151.stdout:sudo 65586 root rtd unknown /proc/65586/root (readlink: Permission denied) 2024-05-21T21:59:26.639 INFO:tasks.workunit.client.0.smithi151.stdout:sudo 65586 root txt unknown /proc/65586/exe (readlink: Permission denied) 2024-05-21T21:59:26.639 INFO:tasks.workunit.client.0.smithi151.stdout:sudo 65586 root NOFD /proc/65586/fd (opendir: Permission denied) 2024-05-21T21:59:26.639 INFO:tasks.workunit.client.0.smithi151.stdout:journalct 65610 root cwd unknown /proc/65610/cwd (readlink: Permission denied) 2024-05-21T21:59:26.639 INFO:tasks.workunit.client.0.smithi151.stdout:journalct 65610 root rtd unknown /proc/65610/root (readlink: Permission denied) 2024-05-21T21:59:26.639 INFO:tasks.workunit.client.0.smithi151.stdout:journalct 65610 root txt unknown /proc/65610/exe (readlink: Permission denied) 2024-05-21T21:59:26.639 INFO:tasks.workunit.client.0.smithi151.stdout:journalct 65610 root NOFD /proc/65610/fd (opendir: Permission denied) 2024-05-21T21:59:26.639 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/5 68429 root cwd unknown /proc/68429/cwd (readlink: Permission denied) 2024-05-21T21:59:26.639 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/5 68429 root rtd unknown /proc/68429/root (readlink: Permission denied) 2024-05-21T21:59:26.639 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/5 68429 root txt unknown /proc/68429/exe (readlink: Permission denied) 2024-05-21T21:59:26.640 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/5 68429 root NOFD /proc/68429/fd (opendir: Permission denied) 2024-05-21T21:59:26.640 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/1 68482 root cwd unknown /proc/68482/cwd (readlink: Permission denied) 2024-05-21T21:59:26.640 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/1 68482 root rtd unknown /proc/68482/root (readlink: Permission denied) 2024-05-21T21:59:26.640 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/1 68482 root txt unknown /proc/68482/exe (readlink: Permission denied) 2024-05-21T21:59:26.640 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/1 68482 root NOFD /proc/68482/fd (opendir: Permission denied) 2024-05-21T21:59:26.640 INFO:tasks.workunit.client.0.smithi151.stdout:sudo 69043 root cwd unknown /proc/69043/cwd (readlink: Permission denied) 2024-05-21T21:59:26.640 INFO:tasks.workunit.client.0.smithi151.stdout:sudo 69043 root rtd unknown /proc/69043/root (readlink: Permission denied) 2024-05-21T21:59:26.640 INFO:tasks.workunit.client.0.smithi151.stdout:sudo 69043 root txt unknown /proc/69043/exe (readlink: Permission denied) 2024-05-21T21:59:26.640 INFO:tasks.workunit.client.0.smithi151.stdout:sudo 69043 root NOFD /proc/69043/fd (opendir: Permission denied) 2024-05-21T21:59:26.640 INFO:tasks.workunit.client.0.smithi151.stdout:sudo 69059 root cwd unknown /proc/69059/cwd (readlink: Permission denied) 2024-05-21T21:59:26.640 INFO:tasks.workunit.client.0.smithi151.stdout:sudo 69059 root rtd unknown /proc/69059/root (readlink: Permission denied) 2024-05-21T21:59:26.640 INFO:tasks.workunit.client.0.smithi151.stdout:sudo 69059 root txt unknown /proc/69059/exe (readlink: Permission denied) 2024-05-21T21:59:26.640 INFO:tasks.workunit.client.0.smithi151.stdout:sudo 69059 root NOFD /proc/69059/fd (opendir: Permission denied) 2024-05-21T21:59:26.640 INFO:tasks.workunit.client.0.smithi151.stdout:journalct 69064 root cwd unknown /proc/69064/cwd (readlink: Permission denied) 2024-05-21T21:59:26.640 INFO:tasks.workunit.client.0.smithi151.stdout:journalct 69064 root rtd unknown /proc/69064/root (readlink: Permission denied) 2024-05-21T21:59:26.641 INFO:tasks.workunit.client.0.smithi151.stdout:journalct 69064 root txt unknown /proc/69064/exe (readlink: Permission denied) 2024-05-21T21:59:26.641 INFO:tasks.workunit.client.0.smithi151.stdout:journalct 69064 root NOFD /proc/69064/fd (opendir: Permission denied) 2024-05-21T21:59:26.641 INFO:tasks.workunit.client.0.smithi151.stdout:sudo 69086 root cwd unknown /proc/69086/cwd (readlink: Permission denied) 2024-05-21T21:59:26.641 INFO:tasks.workunit.client.0.smithi151.stdout:sudo 69086 root rtd unknown /proc/69086/root (readlink: Permission denied) 2024-05-21T21:59:26.641 INFO:tasks.workunit.client.0.smithi151.stdout:sudo 69086 root txt unknown /proc/69086/exe (readlink: Permission denied) 2024-05-21T21:59:26.641 INFO:tasks.workunit.client.0.smithi151.stdout:sudo 69086 root NOFD /proc/69086/fd (opendir: Permission denied) 2024-05-21T21:59:26.641 INFO:tasks.workunit.client.0.smithi151.stdout:journalct 69091 root cwd unknown /proc/69091/cwd (readlink: Permission denied) 2024-05-21T21:59:26.641 INFO:tasks.workunit.client.0.smithi151.stdout:journalct 69091 root rtd unknown /proc/69091/root (readlink: Permission denied) 2024-05-21T21:59:26.641 INFO:tasks.workunit.client.0.smithi151.stdout:journalct 69091 root txt unknown /proc/69091/exe (readlink: Permission denied) 2024-05-21T21:59:26.641 INFO:tasks.workunit.client.0.smithi151.stdout:journalct 69091 root NOFD /proc/69091/fd (opendir: Permission denied) 2024-05-21T21:59:26.641 INFO:tasks.workunit.client.0.smithi151.stdout:sudo 69108 root cwd unknown /proc/69108/cwd (readlink: Permission denied) 2024-05-21T21:59:26.641 INFO:tasks.workunit.client.0.smithi151.stdout:sudo 69108 root rtd unknown /proc/69108/root (readlink: Permission denied) 2024-05-21T21:59:26.641 INFO:tasks.workunit.client.0.smithi151.stdout:sudo 69108 root txt unknown /proc/69108/exe (readlink: Permission denied) 2024-05-21T21:59:26.641 INFO:tasks.workunit.client.0.smithi151.stdout:sudo 69108 root NOFD /proc/69108/fd (opendir: Permission denied) 2024-05-21T21:59:26.641 INFO:tasks.workunit.client.0.smithi151.stdout:journalct 69113 root cwd unknown /proc/69113/cwd (readlink: Permission denied) 2024-05-21T21:59:26.642 INFO:tasks.workunit.client.0.smithi151.stdout:journalct 69113 root rtd unknown /proc/69113/root (readlink: Permission denied) 2024-05-21T21:59:26.642 INFO:tasks.workunit.client.0.smithi151.stdout:journalct 69113 root txt unknown /proc/69113/exe (readlink: Permission denied) 2024-05-21T21:59:26.642 INFO:tasks.workunit.client.0.smithi151.stdout:journalct 69113 root NOFD /proc/69113/fd (opendir: Permission denied) 2024-05-21T21:59:26.642 INFO:tasks.workunit.client.0.smithi151.stdout:journalct 69127 root cwd unknown /proc/69127/cwd (readlink: Permission denied) 2024-05-21T21:59:26.642 INFO:tasks.workunit.client.0.smithi151.stdout:journalct 69127 root rtd unknown /proc/69127/root (readlink: Permission denied) 2024-05-21T21:59:26.642 INFO:tasks.workunit.client.0.smithi151.stdout:journalct 69127 root txt unknown /proc/69127/exe (readlink: Permission denied) 2024-05-21T21:59:26.642 INFO:tasks.workunit.client.0.smithi151.stdout:journalct 69127 root NOFD /proc/69127/fd (opendir: Permission denied) 2024-05-21T21:59:26.642 INFO:tasks.workunit.client.0.smithi151.stdout:conmon 69722 root cwd unknown /proc/69722/cwd (readlink: Permission denied) 2024-05-21T21:59:26.642 INFO:tasks.workunit.client.0.smithi151.stdout:conmon 69722 root rtd unknown /proc/69722/root (readlink: Permission denied) 2024-05-21T21:59:26.642 INFO:tasks.workunit.client.0.smithi151.stdout:conmon 69722 root txt unknown /proc/69722/exe (readlink: Permission denied) 2024-05-21T21:59:26.642 INFO:tasks.workunit.client.0.smithi151.stdout:conmon 69722 root NOFD /proc/69722/fd (opendir: Permission denied) 2024-05-21T21:59:26.642 INFO:tasks.workunit.client.0.smithi151.stdout:podman-in 69724 root cwd unknown /proc/69724/cwd (readlink: Permission denied) 2024-05-21T21:59:26.642 INFO:tasks.workunit.client.0.smithi151.stdout:podman-in 69724 root rtd unknown /proc/69724/root (readlink: Permission denied) 2024-05-21T21:59:26.642 INFO:tasks.workunit.client.0.smithi151.stdout:podman-in 69724 root txt unknown /proc/69724/exe (readlink: Permission denied) 2024-05-21T21:59:26.642 INFO:tasks.workunit.client.0.smithi151.stdout:podman-in 69724 root NOFD /proc/69724/fd (opendir: Permission denied) 2024-05-21T21:59:26.643 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 69729 ceph cwd unknown /proc/69729/cwd (readlink: Permission denied) 2024-05-21T21:59:26.643 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 69729 ceph rtd unknown /proc/69729/root (readlink: Permission denied) 2024-05-21T21:59:26.643 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 69729 ceph txt unknown /proc/69729/exe (readlink: Permission denied) 2024-05-21T21:59:26.643 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 69729 ceph NOFD /proc/69729/fd (opendir: Permission denied) 2024-05-21T21:59:26.643 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 69729 69730 log ceph cwd unknown /proc/69729/task/69730/cwd (readlink: Permission denied) 2024-05-21T21:59:26.643 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 69729 69730 log ceph rtd unknown /proc/69729/task/69730/root (readlink: Permission denied) 2024-05-21T21:59:26.643 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 69729 69730 log ceph txt unknown /proc/69729/task/69730/exe (readlink: Permission denied) 2024-05-21T21:59:26.643 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 69729 69730 log ceph NOFD /proc/69729/task/69730/fd (opendir: Permission denied) 2024-05-21T21:59:26.643 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 69729 69732 msgr-work ceph cwd unknown /proc/69729/task/69732/cwd (readlink: Permission denied) 2024-05-21T21:59:26.643 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 69729 69732 msgr-work ceph rtd unknown /proc/69729/task/69732/root (readlink: Permission denied) 2024-05-21T21:59:26.643 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 69729 69732 msgr-work ceph txt unknown /proc/69729/task/69732/exe (readlink: Permission denied) 2024-05-21T21:59:26.643 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 69729 69732 msgr-work ceph NOFD /proc/69729/task/69732/fd (opendir: Permission denied) 2024-05-21T21:59:26.643 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 69729 69733 msgr-work ceph cwd unknown /proc/69729/task/69733/cwd (readlink: Permission denied) 2024-05-21T21:59:26.643 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 69729 69733 msgr-work ceph rtd unknown /proc/69729/task/69733/root (readlink: Permission denied) 2024-05-21T21:59:26.643 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 69729 69733 msgr-work ceph txt unknown /proc/69729/task/69733/exe (readlink: Permission denied) 2024-05-21T21:59:26.644 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 69729 69733 msgr-work ceph NOFD /proc/69729/task/69733/fd (opendir: Permission denied) 2024-05-21T21:59:26.644 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 69729 69734 msgr-work ceph cwd unknown /proc/69729/task/69734/cwd (readlink: Permission denied) 2024-05-21T21:59:26.644 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 69729 69734 msgr-work ceph rtd unknown /proc/69729/task/69734/root (readlink: Permission denied) 2024-05-21T21:59:26.644 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 69729 69734 msgr-work ceph txt unknown /proc/69729/task/69734/exe (readlink: Permission denied) 2024-05-21T21:59:26.644 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 69729 69734 msgr-work ceph NOFD /proc/69729/task/69734/fd (opendir: Permission denied) 2024-05-21T21:59:26.644 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 69729 69738 service ceph cwd unknown /proc/69729/task/69738/cwd (readlink: Permission denied) 2024-05-21T21:59:26.644 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 69729 69738 service ceph rtd unknown /proc/69729/task/69738/root (readlink: Permission denied) 2024-05-21T21:59:26.644 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 69729 69738 service ceph txt unknown /proc/69729/task/69738/exe (readlink: Permission denied) 2024-05-21T21:59:26.644 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 69729 69738 service ceph NOFD /proc/69729/task/69738/fd (opendir: Permission denied) 2024-05-21T21:59:26.644 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 69729 69739 admin_soc ceph cwd unknown /proc/69729/task/69739/cwd (readlink: Permission denied) 2024-05-21T21:59:26.644 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 69729 69739 admin_soc ceph rtd unknown /proc/69729/task/69739/root (readlink: Permission denied) 2024-05-21T21:59:26.644 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 69729 69739 admin_soc ceph txt unknown /proc/69729/task/69739/exe (readlink: Permission denied) 2024-05-21T21:59:26.644 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 69729 69739 admin_soc ceph NOFD /proc/69729/task/69739/fd (opendir: Permission denied) 2024-05-21T21:59:26.644 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 69729 69740 signal_ha ceph cwd unknown /proc/69729/task/69740/cwd (readlink: Permission denied) 2024-05-21T21:59:26.644 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 69729 69740 signal_ha ceph rtd unknown /proc/69729/task/69740/root (readlink: Permission denied) 2024-05-21T21:59:26.644 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 69729 69740 signal_ha ceph txt unknown /proc/69729/task/69740/exe (readlink: Permission denied) 2024-05-21T21:59:26.645 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 69729 69740 signal_ha ceph NOFD /proc/69729/task/69740/fd (opendir: Permission denied) 2024-05-21T21:59:26.645 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 69729 69741 io_contex ceph cwd unknown /proc/69729/task/69741/cwd (readlink: Permission denied) 2024-05-21T21:59:26.645 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 69729 69741 io_contex ceph rtd unknown /proc/69729/task/69741/root (readlink: Permission denied) 2024-05-21T21:59:26.645 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 69729 69741 io_contex ceph txt unknown /proc/69729/task/69741/exe (readlink: Permission denied) 2024-05-21T21:59:26.645 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 69729 69741 io_contex ceph NOFD /proc/69729/task/69741/fd (opendir: Permission denied) 2024-05-21T21:59:26.645 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 69729 69742 io_contex ceph cwd unknown /proc/69729/task/69742/cwd (readlink: Permission denied) 2024-05-21T21:59:26.645 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 69729 69742 io_contex ceph rtd unknown /proc/69729/task/69742/root (readlink: Permission denied) 2024-05-21T21:59:26.645 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 69729 69742 io_contex ceph txt unknown /proc/69729/task/69742/exe (readlink: Permission denied) 2024-05-21T21:59:26.645 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 69729 69742 io_contex ceph NOFD /proc/69729/task/69742/fd (opendir: Permission denied) 2024-05-21T21:59:26.645 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 69729 69743 ms_dispat ceph cwd unknown /proc/69729/task/69743/cwd (readlink: Permission denied) 2024-05-21T21:59:26.645 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 69729 69743 ms_dispat ceph rtd unknown /proc/69729/task/69743/root (readlink: Permission denied) 2024-05-21T21:59:26.645 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 69729 69743 ms_dispat ceph txt unknown /proc/69729/task/69743/exe (readlink: Permission denied) 2024-05-21T21:59:26.645 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 69729 69743 ms_dispat ceph NOFD /proc/69729/task/69743/fd (opendir: Permission denied) 2024-05-21T21:59:26.645 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 69729 69744 ms_local ceph cwd unknown /proc/69729/task/69744/cwd (readlink: Permission denied) 2024-05-21T21:59:26.646 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 69729 69744 ms_local ceph rtd unknown /proc/69729/task/69744/root (readlink: Permission denied) 2024-05-21T21:59:26.646 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 69729 69744 ms_local ceph txt unknown /proc/69729/task/69744/exe (readlink: Permission denied) 2024-05-21T21:59:26.646 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 69729 69744 ms_local ceph NOFD /proc/69729/task/69744/fd (opendir: Permission denied) 2024-05-21T21:59:26.646 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 69729 69745 safe_time ceph cwd unknown /proc/69729/task/69745/cwd (readlink: Permission denied) 2024-05-21T21:59:26.646 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 69729 69745 safe_time ceph rtd unknown /proc/69729/task/69745/root (readlink: Permission denied) 2024-05-21T21:59:26.646 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 69729 69745 safe_time ceph txt unknown /proc/69729/task/69745/exe (readlink: Permission denied) 2024-05-21T21:59:26.646 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 69729 69745 safe_time ceph NOFD /proc/69729/task/69745/fd (opendir: Permission denied) 2024-05-21T21:59:26.646 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 69729 69746 safe_time ceph cwd unknown /proc/69729/task/69746/cwd (readlink: Permission denied) 2024-05-21T21:59:26.646 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 69729 69746 safe_time ceph rtd unknown /proc/69729/task/69746/root (readlink: Permission denied) 2024-05-21T21:59:26.646 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 69729 69746 safe_time ceph txt unknown /proc/69729/task/69746/exe (readlink: Permission denied) 2024-05-21T21:59:26.646 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 69729 69746 safe_time ceph NOFD /proc/69729/task/69746/fd (opendir: Permission denied) 2024-05-21T21:59:26.646 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 69729 69747 ceph-mds ceph cwd unknown /proc/69729/task/69747/cwd (readlink: Permission denied) 2024-05-21T21:59:26.646 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 69729 69747 ceph-mds ceph rtd unknown /proc/69729/task/69747/root (readlink: Permission denied) 2024-05-21T21:59:26.647 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 69729 69747 ceph-mds ceph txt unknown /proc/69729/task/69747/exe (readlink: Permission denied) 2024-05-21T21:59:26.647 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 69729 69747 ceph-mds ceph NOFD /proc/69729/task/69747/fd (opendir: Permission denied) 2024-05-21T21:59:26.647 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 69729 69752 safe_time ceph cwd unknown /proc/69729/task/69752/cwd (readlink: Permission denied) 2024-05-21T21:59:26.647 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 69729 69752 safe_time ceph rtd unknown /proc/69729/task/69752/root (readlink: Permission denied) 2024-05-21T21:59:26.647 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 69729 69752 safe_time ceph txt unknown /proc/69729/task/69752/exe (readlink: Permission denied) 2024-05-21T21:59:26.647 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 69729 69752 safe_time ceph NOFD /proc/69729/task/69752/fd (opendir: Permission denied) 2024-05-21T21:59:26.647 INFO:tasks.workunit.client.0.smithi151.stdout:conmon 70133 root cwd unknown /proc/70133/cwd (readlink: Permission denied) 2024-05-21T21:59:26.647 INFO:tasks.workunit.client.0.smithi151.stdout:conmon 70133 root rtd unknown /proc/70133/root (readlink: Permission denied) 2024-05-21T21:59:26.647 INFO:tasks.workunit.client.0.smithi151.stdout:conmon 70133 root txt unknown /proc/70133/exe (readlink: Permission denied) 2024-05-21T21:59:26.647 INFO:tasks.workunit.client.0.smithi151.stdout:conmon 70133 root NOFD /proc/70133/fd (opendir: Permission denied) 2024-05-21T21:59:26.647 INFO:tasks.workunit.client.0.smithi151.stdout:podman-in 70135 root cwd unknown /proc/70135/cwd (readlink: Permission denied) 2024-05-21T21:59:26.647 INFO:tasks.workunit.client.0.smithi151.stdout:podman-in 70135 root rtd unknown /proc/70135/root (readlink: Permission denied) 2024-05-21T21:59:26.647 INFO:tasks.workunit.client.0.smithi151.stdout:podman-in 70135 root txt unknown /proc/70135/exe (readlink: Permission denied) 2024-05-21T21:59:26.647 INFO:tasks.workunit.client.0.smithi151.stdout:podman-in 70135 root NOFD /proc/70135/fd (opendir: Permission denied) 2024-05-21T21:59:26.647 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 ceph cwd unknown /proc/70143/cwd (readlink: Permission denied) 2024-05-21T21:59:26.648 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 ceph rtd unknown /proc/70143/root (readlink: Permission denied) 2024-05-21T21:59:26.648 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 ceph txt unknown /proc/70143/exe (readlink: Permission denied) 2024-05-21T21:59:26.648 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 ceph NOFD /proc/70143/fd (opendir: Permission denied) 2024-05-21T21:59:26.648 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 70144 log ceph cwd unknown /proc/70143/task/70144/cwd (readlink: Permission denied) 2024-05-21T21:59:26.648 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 70144 log ceph rtd unknown /proc/70143/task/70144/root (readlink: Permission denied) 2024-05-21T21:59:26.648 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 70144 log ceph txt unknown /proc/70143/task/70144/exe (readlink: Permission denied) 2024-05-21T21:59:26.648 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 70144 log ceph NOFD /proc/70143/task/70144/fd (opendir: Permission denied) 2024-05-21T21:59:26.648 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 70146 msgr-work ceph cwd unknown /proc/70143/task/70146/cwd (readlink: Permission denied) 2024-05-21T21:59:26.648 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 70146 msgr-work ceph rtd unknown /proc/70143/task/70146/root (readlink: Permission denied) 2024-05-21T21:59:26.648 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 70146 msgr-work ceph txt unknown /proc/70143/task/70146/exe (readlink: Permission denied) 2024-05-21T21:59:26.648 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 70146 msgr-work ceph NOFD /proc/70143/task/70146/fd (opendir: Permission denied) 2024-05-21T21:59:26.648 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 70147 msgr-work ceph cwd unknown /proc/70143/task/70147/cwd (readlink: Permission denied) 2024-05-21T21:59:26.648 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 70147 msgr-work ceph rtd unknown /proc/70143/task/70147/root (readlink: Permission denied) 2024-05-21T21:59:26.648 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 70147 msgr-work ceph txt unknown /proc/70143/task/70147/exe (readlink: Permission denied) 2024-05-21T21:59:26.648 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 70147 msgr-work ceph NOFD /proc/70143/task/70147/fd (opendir: Permission denied) 2024-05-21T21:59:26.649 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 70148 msgr-work ceph cwd unknown /proc/70143/task/70148/cwd (readlink: Permission denied) 2024-05-21T21:59:26.649 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 70148 msgr-work ceph rtd unknown /proc/70143/task/70148/root (readlink: Permission denied) 2024-05-21T21:59:26.649 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 70148 msgr-work ceph txt unknown /proc/70143/task/70148/exe (readlink: Permission denied) 2024-05-21T21:59:26.649 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 70148 msgr-work ceph NOFD /proc/70143/task/70148/fd (opendir: Permission denied) 2024-05-21T21:59:26.649 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 70152 service ceph cwd unknown /proc/70143/task/70152/cwd (readlink: Permission denied) 2024-05-21T21:59:26.649 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 70152 service ceph rtd unknown /proc/70143/task/70152/root (readlink: Permission denied) 2024-05-21T21:59:26.649 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 70152 service ceph txt unknown /proc/70143/task/70152/exe (readlink: Permission denied) 2024-05-21T21:59:26.649 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 70152 service ceph NOFD /proc/70143/task/70152/fd (opendir: Permission denied) 2024-05-21T21:59:26.649 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 70153 admin_soc ceph cwd unknown /proc/70143/task/70153/cwd (readlink: Permission denied) 2024-05-21T21:59:26.649 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 70153 admin_soc ceph rtd unknown /proc/70143/task/70153/root (readlink: Permission denied) 2024-05-21T21:59:26.649 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 70153 admin_soc ceph txt unknown /proc/70143/task/70153/exe (readlink: Permission denied) 2024-05-21T21:59:26.649 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 70153 admin_soc ceph NOFD /proc/70143/task/70153/fd (opendir: Permission denied) 2024-05-21T21:59:26.649 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 70154 signal_ha ceph cwd unknown /proc/70143/task/70154/cwd (readlink: Permission denied) 2024-05-21T21:59:26.649 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 70154 signal_ha ceph rtd unknown /proc/70143/task/70154/root (readlink: Permission denied) 2024-05-21T21:59:26.650 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 70154 signal_ha ceph txt unknown /proc/70143/task/70154/exe (readlink: Permission denied) 2024-05-21T21:59:26.650 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 70154 signal_ha ceph NOFD /proc/70143/task/70154/fd (opendir: Permission denied) 2024-05-21T21:59:26.650 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 70155 io_contex ceph cwd unknown /proc/70143/task/70155/cwd (readlink: Permission denied) 2024-05-21T21:59:26.650 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 70155 io_contex ceph rtd unknown /proc/70143/task/70155/root (readlink: Permission denied) 2024-05-21T21:59:26.650 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 70155 io_contex ceph txt unknown /proc/70143/task/70155/exe (readlink: Permission denied) 2024-05-21T21:59:26.650 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 70155 io_contex ceph NOFD /proc/70143/task/70155/fd (opendir: Permission denied) 2024-05-21T21:59:26.650 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 70156 io_contex ceph cwd unknown /proc/70143/task/70156/cwd (readlink: Permission denied) 2024-05-21T21:59:26.650 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 70156 io_contex ceph rtd unknown /proc/70143/task/70156/root (readlink: Permission denied) 2024-05-21T21:59:26.650 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 70156 io_contex ceph txt unknown /proc/70143/task/70156/exe (readlink: Permission denied) 2024-05-21T21:59:26.650 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 70156 io_contex ceph NOFD /proc/70143/task/70156/fd (opendir: Permission denied) 2024-05-21T21:59:26.650 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 70157 ms_dispat ceph cwd unknown /proc/70143/task/70157/cwd (readlink: Permission denied) 2024-05-21T21:59:26.650 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 70157 ms_dispat ceph rtd unknown /proc/70143/task/70157/root (readlink: Permission denied) 2024-05-21T21:59:26.650 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 70157 ms_dispat ceph txt unknown /proc/70143/task/70157/exe (readlink: Permission denied) 2024-05-21T21:59:26.650 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 70157 ms_dispat ceph NOFD /proc/70143/task/70157/fd (opendir: Permission denied) 2024-05-21T21:59:26.651 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 70158 ms_local ceph cwd unknown /proc/70143/task/70158/cwd (readlink: Permission denied) 2024-05-21T21:59:26.651 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 70158 ms_local ceph rtd unknown /proc/70143/task/70158/root (readlink: Permission denied) 2024-05-21T21:59:26.651 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 70158 ms_local ceph txt unknown /proc/70143/task/70158/exe (readlink: Permission denied) 2024-05-21T21:59:26.651 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 70158 ms_local ceph NOFD /proc/70143/task/70158/fd (opendir: Permission denied) 2024-05-21T21:59:26.651 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 70159 safe_time ceph cwd unknown /proc/70143/task/70159/cwd (readlink: Permission denied) 2024-05-21T21:59:26.651 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 70159 safe_time ceph rtd unknown /proc/70143/task/70159/root (readlink: Permission denied) 2024-05-21T21:59:26.651 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 70159 safe_time ceph txt unknown /proc/70143/task/70159/exe (readlink: Permission denied) 2024-05-21T21:59:26.651 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 70159 safe_time ceph NOFD /proc/70143/task/70159/fd (opendir: Permission denied) 2024-05-21T21:59:26.651 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 70160 safe_time ceph cwd unknown /proc/70143/task/70160/cwd (readlink: Permission denied) 2024-05-21T21:59:26.651 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 70160 safe_time ceph rtd unknown /proc/70143/task/70160/root (readlink: Permission denied) 2024-05-21T21:59:26.651 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 70160 safe_time ceph txt unknown /proc/70143/task/70160/exe (readlink: Permission denied) 2024-05-21T21:59:26.651 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 70160 safe_time ceph NOFD /proc/70143/task/70160/fd (opendir: Permission denied) 2024-05-21T21:59:26.651 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 70161 ceph-mds ceph cwd unknown /proc/70143/task/70161/cwd (readlink: Permission denied) 2024-05-21T21:59:26.651 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 70161 ceph-mds ceph rtd unknown /proc/70143/task/70161/root (readlink: Permission denied) 2024-05-21T21:59:26.652 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 70161 ceph-mds ceph txt unknown /proc/70143/task/70161/exe (readlink: Permission denied) 2024-05-21T21:59:26.652 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 70161 ceph-mds ceph NOFD /proc/70143/task/70161/fd (opendir: Permission denied) 2024-05-21T21:59:26.652 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 70168 safe_time ceph cwd unknown /proc/70143/task/70168/cwd (readlink: Permission denied) 2024-05-21T21:59:26.652 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 70168 safe_time ceph rtd unknown /proc/70143/task/70168/root (readlink: Permission denied) 2024-05-21T21:59:26.652 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 70168 safe_time ceph txt unknown /proc/70143/task/70168/exe (readlink: Permission denied) 2024-05-21T21:59:26.652 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 70168 safe_time ceph NOFD /proc/70143/task/70168/fd (opendir: Permission denied) 2024-05-21T21:59:26.652 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 73760 ceph_time ceph cwd unknown /proc/70143/task/73760/cwd (readlink: Permission denied) 2024-05-21T21:59:26.652 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 73760 ceph_time ceph rtd unknown /proc/70143/task/73760/root (readlink: Permission denied) 2024-05-21T21:59:26.652 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 73760 ceph_time ceph txt unknown /proc/70143/task/73760/exe (readlink: Permission denied) 2024-05-21T21:59:26.652 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 73760 ceph_time ceph NOFD /proc/70143/task/73760/fd (opendir: Permission denied) 2024-05-21T21:59:26.652 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 73761 OpHistory ceph cwd unknown /proc/70143/task/73761/cwd (readlink: Permission denied) 2024-05-21T21:59:26.652 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 73761 OpHistory ceph rtd unknown /proc/70143/task/73761/root (readlink: Permission denied) 2024-05-21T21:59:26.652 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 73761 OpHistory ceph txt unknown /proc/70143/task/73761/exe (readlink: Permission denied) 2024-05-21T21:59:26.653 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 73761 OpHistory ceph NOFD /proc/70143/task/73761/fd (opendir: Permission denied) 2024-05-21T21:59:26.653 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 73764 ms_dispat ceph cwd unknown /proc/70143/task/73764/cwd (readlink: Permission denied) 2024-05-21T21:59:26.653 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 73764 ms_dispat ceph rtd unknown /proc/70143/task/73764/root (readlink: Permission denied) 2024-05-21T21:59:26.653 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 73764 ms_dispat ceph txt unknown /proc/70143/task/73764/exe (readlink: Permission denied) 2024-05-21T21:59:26.653 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 73764 ms_dispat ceph NOFD /proc/70143/task/73764/fd (opendir: Permission denied) 2024-05-21T21:59:26.653 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 73765 ms_dispat ceph cwd unknown /proc/70143/task/73765/cwd (readlink: Permission denied) 2024-05-21T21:59:26.653 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 73765 ms_dispat ceph rtd unknown /proc/70143/task/73765/root (readlink: Permission denied) 2024-05-21T21:59:26.653 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 73765 ms_dispat ceph txt unknown /proc/70143/task/73765/exe (readlink: Permission denied) 2024-05-21T21:59:26.653 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 73765 ms_dispat ceph NOFD /proc/70143/task/73765/fd (opendir: Permission denied) 2024-05-21T21:59:26.653 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 73769 mds_rank_ ceph cwd unknown /proc/70143/task/73769/cwd (readlink: Permission denied) 2024-05-21T21:59:26.653 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 73769 mds_rank_ ceph rtd unknown /proc/70143/task/73769/root (readlink: Permission denied) 2024-05-21T21:59:26.653 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 73769 mds_rank_ ceph txt unknown /proc/70143/task/73769/exe (readlink: Permission denied) 2024-05-21T21:59:26.653 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 73769 mds_rank_ ceph NOFD /proc/70143/task/73769/fd (opendir: Permission denied) 2024-05-21T21:59:26.653 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 73771 PQ_Finish ceph cwd unknown /proc/70143/task/73771/cwd (readlink: Permission denied) 2024-05-21T21:59:26.654 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 73771 PQ_Finish ceph rtd unknown /proc/70143/task/73771/root (readlink: Permission denied) 2024-05-21T21:59:26.654 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 73771 PQ_Finish ceph txt unknown /proc/70143/task/73771/exe (readlink: Permission denied) 2024-05-21T21:59:26.654 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 73771 PQ_Finish ceph NOFD /proc/70143/task/73771/fd (opendir: Permission denied) 2024-05-21T21:59:26.654 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 73773 safe_time ceph cwd unknown /proc/70143/task/73773/cwd (readlink: Permission denied) 2024-05-21T21:59:26.654 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 73773 safe_time ceph rtd unknown /proc/70143/task/73773/root (readlink: Permission denied) 2024-05-21T21:59:26.654 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 73773 safe_time ceph txt unknown /proc/70143/task/73773/exe (readlink: Permission denied) 2024-05-21T21:59:26.654 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 73773 safe_time ceph NOFD /proc/70143/task/73773/fd (opendir: Permission denied) 2024-05-21T21:59:26.654 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 73774 MR_Finish ceph cwd unknown /proc/70143/task/73774/cwd (readlink: Permission denied) 2024-05-21T21:59:26.654 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 73774 MR_Finish ceph rtd unknown /proc/70143/task/73774/root (readlink: Permission denied) 2024-05-21T21:59:26.654 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 73774 MR_Finish ceph txt unknown /proc/70143/task/73774/exe (readlink: Permission denied) 2024-05-21T21:59:26.654 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 73774 MR_Finish ceph NOFD /proc/70143/task/73774/fd (opendir: Permission denied) 2024-05-21T21:59:26.654 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 73778 md_submit ceph cwd unknown /proc/70143/task/73778/cwd (readlink: Permission denied) 2024-05-21T21:59:26.654 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 73778 md_submit ceph rtd unknown /proc/70143/task/73778/root (readlink: Permission denied) 2024-05-21T21:59:26.655 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 73778 md_submit ceph txt unknown /proc/70143/task/73778/exe (readlink: Permission denied) 2024-05-21T21:59:26.655 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70143 73778 md_submit ceph NOFD /proc/70143/task/73778/fd (opendir: Permission denied) 2024-05-21T21:59:26.655 INFO:tasks.workunit.client.0.smithi151.stdout:conmon 70955 root cwd unknown /proc/70955/cwd (readlink: Permission denied) 2024-05-21T21:59:26.655 INFO:tasks.workunit.client.0.smithi151.stdout:conmon 70955 root rtd unknown /proc/70955/root (readlink: Permission denied) 2024-05-21T21:59:26.655 INFO:tasks.workunit.client.0.smithi151.stdout:conmon 70955 root txt unknown /proc/70955/exe (readlink: Permission denied) 2024-05-21T21:59:26.655 INFO:tasks.workunit.client.0.smithi151.stdout:conmon 70955 root NOFD /proc/70955/fd (opendir: Permission denied) 2024-05-21T21:59:26.655 INFO:tasks.workunit.client.0.smithi151.stdout:podman-in 70957 root cwd unknown /proc/70957/cwd (readlink: Permission denied) 2024-05-21T21:59:26.655 INFO:tasks.workunit.client.0.smithi151.stdout:podman-in 70957 root rtd unknown /proc/70957/root (readlink: Permission denied) 2024-05-21T21:59:26.655 INFO:tasks.workunit.client.0.smithi151.stdout:podman-in 70957 root txt unknown /proc/70957/exe (readlink: Permission denied) 2024-05-21T21:59:26.655 INFO:tasks.workunit.client.0.smithi151.stdout:podman-in 70957 root NOFD /proc/70957/fd (opendir: Permission denied) 2024-05-21T21:59:26.655 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70959 ceph cwd unknown /proc/70959/cwd (readlink: Permission denied) 2024-05-21T21:59:26.655 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70959 ceph rtd unknown /proc/70959/root (readlink: Permission denied) 2024-05-21T21:59:26.655 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70959 ceph txt unknown /proc/70959/exe (readlink: Permission denied) 2024-05-21T21:59:26.656 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70959 ceph NOFD /proc/70959/fd (opendir: Permission denied) 2024-05-21T21:59:26.656 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70959 70960 log ceph cwd unknown /proc/70959/task/70960/cwd (readlink: Permission denied) 2024-05-21T21:59:26.656 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70959 70960 log ceph rtd unknown /proc/70959/task/70960/root (readlink: Permission denied) 2024-05-21T21:59:26.656 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70959 70960 log ceph txt unknown /proc/70959/task/70960/exe (readlink: Permission denied) 2024-05-21T21:59:26.656 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70959 70960 log ceph NOFD /proc/70959/task/70960/fd (opendir: Permission denied) 2024-05-21T21:59:26.656 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70959 70962 msgr-work ceph cwd unknown /proc/70959/task/70962/cwd (readlink: Permission denied) 2024-05-21T21:59:26.656 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70959 70962 msgr-work ceph rtd unknown /proc/70959/task/70962/root (readlink: Permission denied) 2024-05-21T21:59:26.656 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70959 70962 msgr-work ceph txt unknown /proc/70959/task/70962/exe (readlink: Permission denied) 2024-05-21T21:59:26.656 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70959 70962 msgr-work ceph NOFD /proc/70959/task/70962/fd (opendir: Permission denied) 2024-05-21T21:59:26.656 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70959 70963 msgr-work ceph cwd unknown /proc/70959/task/70963/cwd (readlink: Permission denied) 2024-05-21T21:59:26.656 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70959 70963 msgr-work ceph rtd unknown /proc/70959/task/70963/root (readlink: Permission denied) 2024-05-21T21:59:26.656 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70959 70963 msgr-work ceph txt unknown /proc/70959/task/70963/exe (readlink: Permission denied) 2024-05-21T21:59:26.656 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70959 70963 msgr-work ceph NOFD /proc/70959/task/70963/fd (opendir: Permission denied) 2024-05-21T21:59:26.657 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70959 70964 msgr-work ceph cwd unknown /proc/70959/task/70964/cwd (readlink: Permission denied) 2024-05-21T21:59:26.657 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70959 70964 msgr-work ceph rtd unknown /proc/70959/task/70964/root (readlink: Permission denied) 2024-05-21T21:59:26.657 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70959 70964 msgr-work ceph txt unknown /proc/70959/task/70964/exe (readlink: Permission denied) 2024-05-21T21:59:26.657 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70959 70964 msgr-work ceph NOFD /proc/70959/task/70964/fd (opendir: Permission denied) 2024-05-21T21:59:26.657 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70959 70968 service ceph cwd unknown /proc/70959/task/70968/cwd (readlink: Permission denied) 2024-05-21T21:59:26.657 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70959 70968 service ceph rtd unknown /proc/70959/task/70968/root (readlink: Permission denied) 2024-05-21T21:59:26.657 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70959 70968 service ceph txt unknown /proc/70959/task/70968/exe (readlink: Permission denied) 2024-05-21T21:59:26.657 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70959 70968 service ceph NOFD /proc/70959/task/70968/fd (opendir: Permission denied) 2024-05-21T21:59:26.657 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70959 70969 admin_soc ceph cwd unknown /proc/70959/task/70969/cwd (readlink: Permission denied) 2024-05-21T21:59:26.657 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70959 70969 admin_soc ceph rtd unknown /proc/70959/task/70969/root (readlink: Permission denied) 2024-05-21T21:59:26.657 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70959 70969 admin_soc ceph txt unknown /proc/70959/task/70969/exe (readlink: Permission denied) 2024-05-21T21:59:26.657 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70959 70969 admin_soc ceph NOFD /proc/70959/task/70969/fd (opendir: Permission denied) 2024-05-21T21:59:26.657 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70959 70970 signal_ha ceph cwd unknown /proc/70959/task/70970/cwd (readlink: Permission denied) 2024-05-21T21:59:26.657 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70959 70970 signal_ha ceph rtd unknown /proc/70959/task/70970/root (readlink: Permission denied) 2024-05-21T21:59:26.657 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70959 70970 signal_ha ceph txt unknown /proc/70959/task/70970/exe (readlink: Permission denied) 2024-05-21T21:59:26.658 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70959 70970 signal_ha ceph NOFD /proc/70959/task/70970/fd (opendir: Permission denied) 2024-05-21T21:59:26.658 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70959 70971 io_contex ceph cwd unknown /proc/70959/task/70971/cwd (readlink: Permission denied) 2024-05-21T21:59:26.658 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70959 70971 io_contex ceph rtd unknown /proc/70959/task/70971/root (readlink: Permission denied) 2024-05-21T21:59:26.658 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70959 70971 io_contex ceph txt unknown /proc/70959/task/70971/exe (readlink: Permission denied) 2024-05-21T21:59:26.658 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70959 70971 io_contex ceph NOFD /proc/70959/task/70971/fd (opendir: Permission denied) 2024-05-21T21:59:26.658 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70959 70972 io_contex ceph cwd unknown /proc/70959/task/70972/cwd (readlink: Permission denied) 2024-05-21T21:59:26.658 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70959 70972 io_contex ceph rtd unknown /proc/70959/task/70972/root (readlink: Permission denied) 2024-05-21T21:59:26.658 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70959 70972 io_contex ceph txt unknown /proc/70959/task/70972/exe (readlink: Permission denied) 2024-05-21T21:59:26.658 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70959 70972 io_contex ceph NOFD /proc/70959/task/70972/fd (opendir: Permission denied) 2024-05-21T21:59:26.658 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70959 70973 ms_dispat ceph cwd unknown /proc/70959/task/70973/cwd (readlink: Permission denied) 2024-05-21T21:59:26.658 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70959 70973 ms_dispat ceph rtd unknown /proc/70959/task/70973/root (readlink: Permission denied) 2024-05-21T21:59:26.658 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70959 70973 ms_dispat ceph txt unknown /proc/70959/task/70973/exe (readlink: Permission denied) 2024-05-21T21:59:26.658 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70959 70973 ms_dispat ceph NOFD /proc/70959/task/70973/fd (opendir: Permission denied) 2024-05-21T21:59:26.659 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70959 70974 ms_local ceph cwd unknown /proc/70959/task/70974/cwd (readlink: Permission denied) 2024-05-21T21:59:26.659 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70959 70974 ms_local ceph rtd unknown /proc/70959/task/70974/root (readlink: Permission denied) 2024-05-21T21:59:26.659 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70959 70974 ms_local ceph txt unknown /proc/70959/task/70974/exe (readlink: Permission denied) 2024-05-21T21:59:26.659 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70959 70974 ms_local ceph NOFD /proc/70959/task/70974/fd (opendir: Permission denied) 2024-05-21T21:59:26.659 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70959 70975 safe_time ceph cwd unknown /proc/70959/task/70975/cwd (readlink: Permission denied) 2024-05-21T21:59:26.659 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70959 70975 safe_time ceph rtd unknown /proc/70959/task/70975/root (readlink: Permission denied) 2024-05-21T21:59:26.659 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70959 70975 safe_time ceph txt unknown /proc/70959/task/70975/exe (readlink: Permission denied) 2024-05-21T21:59:26.659 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70959 70975 safe_time ceph NOFD /proc/70959/task/70975/fd (opendir: Permission denied) 2024-05-21T21:59:26.659 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70959 70976 safe_time ceph cwd unknown /proc/70959/task/70976/cwd (readlink: Permission denied) 2024-05-21T21:59:26.659 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70959 70976 safe_time ceph rtd unknown /proc/70959/task/70976/root (readlink: Permission denied) 2024-05-21T21:59:26.659 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70959 70976 safe_time ceph txt unknown /proc/70959/task/70976/exe (readlink: Permission denied) 2024-05-21T21:59:26.659 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70959 70976 safe_time ceph NOFD /proc/70959/task/70976/fd (opendir: Permission denied) 2024-05-21T21:59:26.659 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70959 70977 ceph-mds ceph cwd unknown /proc/70959/task/70977/cwd (readlink: Permission denied) 2024-05-21T21:59:26.659 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70959 70977 ceph-mds ceph rtd unknown /proc/70959/task/70977/root (readlink: Permission denied) 2024-05-21T21:59:26.659 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70959 70977 ceph-mds ceph txt unknown /proc/70959/task/70977/exe (readlink: Permission denied) 2024-05-21T21:59:26.660 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70959 70977 ceph-mds ceph NOFD /proc/70959/task/70977/fd (opendir: Permission denied) 2024-05-21T21:59:26.660 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70959 71014 safe_time ceph cwd unknown /proc/70959/task/71014/cwd (readlink: Permission denied) 2024-05-21T21:59:26.660 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70959 71014 safe_time ceph rtd unknown /proc/70959/task/71014/root (readlink: Permission denied) 2024-05-21T21:59:26.660 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70959 71014 safe_time ceph txt unknown /proc/70959/task/71014/exe (readlink: Permission denied) 2024-05-21T21:59:26.660 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 70959 71014 safe_time ceph NOFD /proc/70959/task/71014/fd (opendir: Permission denied) 2024-05-21T21:59:26.660 INFO:tasks.workunit.client.0.smithi151.stdout:conmon 71364 root cwd unknown /proc/71364/cwd (readlink: Permission denied) 2024-05-21T21:59:26.660 INFO:tasks.workunit.client.0.smithi151.stdout:conmon 71364 root rtd unknown /proc/71364/root (readlink: Permission denied) 2024-05-21T21:59:26.660 INFO:tasks.workunit.client.0.smithi151.stdout:conmon 71364 root txt unknown /proc/71364/exe (readlink: Permission denied) 2024-05-21T21:59:26.660 INFO:tasks.workunit.client.0.smithi151.stdout:conmon 71364 root NOFD /proc/71364/fd (opendir: Permission denied) 2024-05-21T21:59:26.660 INFO:tasks.workunit.client.0.smithi151.stdout:podman-in 71366 root cwd unknown /proc/71366/cwd (readlink: Permission denied) 2024-05-21T21:59:26.660 INFO:tasks.workunit.client.0.smithi151.stdout:podman-in 71366 root rtd unknown /proc/71366/root (readlink: Permission denied) 2024-05-21T21:59:26.660 INFO:tasks.workunit.client.0.smithi151.stdout:podman-in 71366 root txt unknown /proc/71366/exe (readlink: Permission denied) 2024-05-21T21:59:26.660 INFO:tasks.workunit.client.0.smithi151.stdout:podman-in 71366 root NOFD /proc/71366/fd (opendir: Permission denied) 2024-05-21T21:59:26.660 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 ceph cwd unknown /proc/71371/cwd (readlink: Permission denied) 2024-05-21T21:59:26.660 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 ceph rtd unknown /proc/71371/root (readlink: Permission denied) 2024-05-21T21:59:26.661 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 ceph txt unknown /proc/71371/exe (readlink: Permission denied) 2024-05-21T21:59:26.661 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 ceph NOFD /proc/71371/fd (opendir: Permission denied) 2024-05-21T21:59:26.661 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 71372 log ceph cwd unknown /proc/71371/task/71372/cwd (readlink: Permission denied) 2024-05-21T21:59:26.661 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 71372 log ceph rtd unknown /proc/71371/task/71372/root (readlink: Permission denied) 2024-05-21T21:59:26.661 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 71372 log ceph txt unknown /proc/71371/task/71372/exe (readlink: Permission denied) 2024-05-21T21:59:26.661 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 71372 log ceph NOFD /proc/71371/task/71372/fd (opendir: Permission denied) 2024-05-21T21:59:26.661 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 71374 msgr-work ceph cwd unknown /proc/71371/task/71374/cwd (readlink: Permission denied) 2024-05-21T21:59:26.661 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 71374 msgr-work ceph rtd unknown /proc/71371/task/71374/root (readlink: Permission denied) 2024-05-21T21:59:26.661 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 71374 msgr-work ceph txt unknown /proc/71371/task/71374/exe (readlink: Permission denied) 2024-05-21T21:59:26.661 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 71374 msgr-work ceph NOFD /proc/71371/task/71374/fd (opendir: Permission denied) 2024-05-21T21:59:26.661 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 71375 msgr-work ceph cwd unknown /proc/71371/task/71375/cwd (readlink: Permission denied) 2024-05-21T21:59:26.661 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 71375 msgr-work ceph rtd unknown /proc/71371/task/71375/root (readlink: Permission denied) 2024-05-21T21:59:26.661 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 71375 msgr-work ceph txt unknown /proc/71371/task/71375/exe (readlink: Permission denied) 2024-05-21T21:59:26.661 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 71375 msgr-work ceph NOFD /proc/71371/task/71375/fd (opendir: Permission denied) 2024-05-21T21:59:26.661 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 71376 msgr-work ceph cwd unknown /proc/71371/task/71376/cwd (readlink: Permission denied) 2024-05-21T21:59:26.662 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 71376 msgr-work ceph rtd unknown /proc/71371/task/71376/root (readlink: Permission denied) 2024-05-21T21:59:26.662 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 71376 msgr-work ceph txt unknown /proc/71371/task/71376/exe (readlink: Permission denied) 2024-05-21T21:59:26.662 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 71376 msgr-work ceph NOFD /proc/71371/task/71376/fd (opendir: Permission denied) 2024-05-21T21:59:26.662 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 71380 service ceph cwd unknown /proc/71371/task/71380/cwd (readlink: Permission denied) 2024-05-21T21:59:26.662 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 71380 service ceph rtd unknown /proc/71371/task/71380/root (readlink: Permission denied) 2024-05-21T21:59:26.662 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 71380 service ceph txt unknown /proc/71371/task/71380/exe (readlink: Permission denied) 2024-05-21T21:59:26.662 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 71380 service ceph NOFD /proc/71371/task/71380/fd (opendir: Permission denied) 2024-05-21T21:59:26.662 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 71381 admin_soc ceph cwd unknown /proc/71371/task/71381/cwd (readlink: Permission denied) 2024-05-21T21:59:26.662 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 71381 admin_soc ceph rtd unknown /proc/71371/task/71381/root (readlink: Permission denied) 2024-05-21T21:59:26.662 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 71381 admin_soc ceph txt unknown /proc/71371/task/71381/exe (readlink: Permission denied) 2024-05-21T21:59:26.662 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 71381 admin_soc ceph NOFD /proc/71371/task/71381/fd (opendir: Permission denied) 2024-05-21T21:59:26.663 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 71382 signal_ha ceph cwd unknown /proc/71371/task/71382/cwd (readlink: Permission denied) 2024-05-21T21:59:26.663 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 71382 signal_ha ceph rtd unknown /proc/71371/task/71382/root (readlink: Permission denied) 2024-05-21T21:59:26.663 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 71382 signal_ha ceph txt unknown /proc/71371/task/71382/exe (readlink: Permission denied) 2024-05-21T21:59:26.663 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 71382 signal_ha ceph NOFD /proc/71371/task/71382/fd (opendir: Permission denied) 2024-05-21T21:59:26.663 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 71383 io_contex ceph cwd unknown /proc/71371/task/71383/cwd (readlink: Permission denied) 2024-05-21T21:59:26.663 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 71383 io_contex ceph rtd unknown /proc/71371/task/71383/root (readlink: Permission denied) 2024-05-21T21:59:26.663 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 71383 io_contex ceph txt unknown /proc/71371/task/71383/exe (readlink: Permission denied) 2024-05-21T21:59:26.663 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 71383 io_contex ceph NOFD /proc/71371/task/71383/fd (opendir: Permission denied) 2024-05-21T21:59:26.663 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 71384 io_contex ceph cwd unknown /proc/71371/task/71384/cwd (readlink: Permission denied) 2024-05-21T21:59:26.663 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 71384 io_contex ceph rtd unknown /proc/71371/task/71384/root (readlink: Permission denied) 2024-05-21T21:59:26.663 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 71384 io_contex ceph txt unknown /proc/71371/task/71384/exe (readlink: Permission denied) 2024-05-21T21:59:26.663 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 71384 io_contex ceph NOFD /proc/71371/task/71384/fd (opendir: Permission denied) 2024-05-21T21:59:26.663 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 71385 ms_dispat ceph cwd unknown /proc/71371/task/71385/cwd (readlink: Permission denied) 2024-05-21T21:59:26.664 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 71385 ms_dispat ceph rtd unknown /proc/71371/task/71385/root (readlink: Permission denied) 2024-05-21T21:59:26.664 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 71385 ms_dispat ceph txt unknown /proc/71371/task/71385/exe (readlink: Permission denied) 2024-05-21T21:59:26.664 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 71385 ms_dispat ceph NOFD /proc/71371/task/71385/fd (opendir: Permission denied) 2024-05-21T21:59:26.664 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 71386 ms_local ceph cwd unknown /proc/71371/task/71386/cwd (readlink: Permission denied) 2024-05-21T21:59:26.664 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 71386 ms_local ceph rtd unknown /proc/71371/task/71386/root (readlink: Permission denied) 2024-05-21T21:59:26.664 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 71386 ms_local ceph txt unknown /proc/71371/task/71386/exe (readlink: Permission denied) 2024-05-21T21:59:26.664 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 71386 ms_local ceph NOFD /proc/71371/task/71386/fd (opendir: Permission denied) 2024-05-21T21:59:26.664 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 71387 safe_time ceph cwd unknown /proc/71371/task/71387/cwd (readlink: Permission denied) 2024-05-21T21:59:26.664 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 71387 safe_time ceph rtd unknown /proc/71371/task/71387/root (readlink: Permission denied) 2024-05-21T21:59:26.664 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 71387 safe_time ceph txt unknown /proc/71371/task/71387/exe (readlink: Permission denied) 2024-05-21T21:59:26.664 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 71387 safe_time ceph NOFD /proc/71371/task/71387/fd (opendir: Permission denied) 2024-05-21T21:59:26.664 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 71388 safe_time ceph cwd unknown /proc/71371/task/71388/cwd (readlink: Permission denied) 2024-05-21T21:59:26.664 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 71388 safe_time ceph rtd unknown /proc/71371/task/71388/root (readlink: Permission denied) 2024-05-21T21:59:26.664 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 71388 safe_time ceph txt unknown /proc/71371/task/71388/exe (readlink: Permission denied) 2024-05-21T21:59:26.665 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 71388 safe_time ceph NOFD /proc/71371/task/71388/fd (opendir: Permission denied) 2024-05-21T21:59:26.665 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 71389 ceph-mds ceph cwd unknown /proc/71371/task/71389/cwd (readlink: Permission denied) 2024-05-21T21:59:26.665 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 71389 ceph-mds ceph rtd unknown /proc/71371/task/71389/root (readlink: Permission denied) 2024-05-21T21:59:26.665 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 71389 ceph-mds ceph txt unknown /proc/71371/task/71389/exe (readlink: Permission denied) 2024-05-21T21:59:26.665 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 71389 ceph-mds ceph NOFD /proc/71371/task/71389/fd (opendir: Permission denied) 2024-05-21T21:59:26.665 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 71391 safe_time ceph cwd unknown /proc/71371/task/71391/cwd (readlink: Permission denied) 2024-05-21T21:59:26.665 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 71391 safe_time ceph rtd unknown /proc/71371/task/71391/root (readlink: Permission denied) 2024-05-21T21:59:26.665 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 71391 safe_time ceph txt unknown /proc/71371/task/71391/exe (readlink: Permission denied) 2024-05-21T21:59:26.665 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 71391 safe_time ceph NOFD /proc/71371/task/71391/fd (opendir: Permission denied) 2024-05-21T21:59:26.665 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 73759 ceph_time ceph cwd unknown /proc/71371/task/73759/cwd (readlink: Permission denied) 2024-05-21T21:59:26.665 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 73759 ceph_time ceph rtd unknown /proc/71371/task/73759/root (readlink: Permission denied) 2024-05-21T21:59:26.665 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 73759 ceph_time ceph txt unknown /proc/71371/task/73759/exe (readlink: Permission denied) 2024-05-21T21:59:26.665 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 73759 ceph_time ceph NOFD /proc/71371/task/73759/fd (opendir: Permission denied) 2024-05-21T21:59:26.665 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 73762 OpHistory ceph cwd unknown /proc/71371/task/73762/cwd (readlink: Permission denied) 2024-05-21T21:59:26.665 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 73762 OpHistory ceph rtd unknown /proc/71371/task/73762/root (readlink: Permission denied) 2024-05-21T21:59:26.666 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 73762 OpHistory ceph txt unknown /proc/71371/task/73762/exe (readlink: Permission denied) 2024-05-21T21:59:26.666 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 73762 OpHistory ceph NOFD /proc/71371/task/73762/fd (opendir: Permission denied) 2024-05-21T21:59:26.666 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 73763 ms_dispat ceph cwd unknown /proc/71371/task/73763/cwd (readlink: Permission denied) 2024-05-21T21:59:26.666 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 73763 ms_dispat ceph rtd unknown /proc/71371/task/73763/root (readlink: Permission denied) 2024-05-21T21:59:26.666 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 73763 ms_dispat ceph txt unknown /proc/71371/task/73763/exe (readlink: Permission denied) 2024-05-21T21:59:26.666 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 73763 ms_dispat ceph NOFD /proc/71371/task/73763/fd (opendir: Permission denied) 2024-05-21T21:59:26.666 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 73766 ms_dispat ceph cwd unknown /proc/71371/task/73766/cwd (readlink: Permission denied) 2024-05-21T21:59:26.666 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 73766 ms_dispat ceph rtd unknown /proc/71371/task/73766/root (readlink: Permission denied) 2024-05-21T21:59:26.666 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 73766 ms_dispat ceph txt unknown /proc/71371/task/73766/exe (readlink: Permission denied) 2024-05-21T21:59:26.666 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 73766 ms_dispat ceph NOFD /proc/71371/task/73766/fd (opendir: Permission denied) 2024-05-21T21:59:26.666 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 73767 mds_rank_ ceph cwd unknown /proc/71371/task/73767/cwd (readlink: Permission denied) 2024-05-21T21:59:26.666 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 73767 mds_rank_ ceph rtd unknown /proc/71371/task/73767/root (readlink: Permission denied) 2024-05-21T21:59:26.666 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 73767 mds_rank_ ceph txt unknown /proc/71371/task/73767/exe (readlink: Permission denied) 2024-05-21T21:59:26.666 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 73767 mds_rank_ ceph NOFD /proc/71371/task/73767/fd (opendir: Permission denied) 2024-05-21T21:59:26.667 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 73768 PQ_Finish ceph cwd unknown /proc/71371/task/73768/cwd (readlink: Permission denied) 2024-05-21T21:59:26.667 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 73768 PQ_Finish ceph rtd unknown /proc/71371/task/73768/root (readlink: Permission denied) 2024-05-21T21:59:26.667 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 73768 PQ_Finish ceph txt unknown /proc/71371/task/73768/exe (readlink: Permission denied) 2024-05-21T21:59:26.667 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 73768 PQ_Finish ceph NOFD /proc/71371/task/73768/fd (opendir: Permission denied) 2024-05-21T21:59:26.667 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 73770 safe_time ceph cwd unknown /proc/71371/task/73770/cwd (readlink: Permission denied) 2024-05-21T21:59:26.667 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 73770 safe_time ceph rtd unknown /proc/71371/task/73770/root (readlink: Permission denied) 2024-05-21T21:59:26.667 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 73770 safe_time ceph txt unknown /proc/71371/task/73770/exe (readlink: Permission denied) 2024-05-21T21:59:26.667 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 73770 safe_time ceph NOFD /proc/71371/task/73770/fd (opendir: Permission denied) 2024-05-21T21:59:26.667 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 73772 MR_Finish ceph cwd unknown /proc/71371/task/73772/cwd (readlink: Permission denied) 2024-05-21T21:59:26.667 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 73772 MR_Finish ceph rtd unknown /proc/71371/task/73772/root (readlink: Permission denied) 2024-05-21T21:59:26.667 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 73772 MR_Finish ceph txt unknown /proc/71371/task/73772/exe (readlink: Permission denied) 2024-05-21T21:59:26.667 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 73772 MR_Finish ceph NOFD /proc/71371/task/73772/fd (opendir: Permission denied) 2024-05-21T21:59:26.667 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 73776 md_submit ceph cwd unknown /proc/71371/task/73776/cwd (readlink: Permission denied) 2024-05-21T21:59:26.667 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 73776 md_submit ceph rtd unknown /proc/71371/task/73776/root (readlink: Permission denied) 2024-05-21T21:59:26.667 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 73776 md_submit ceph txt unknown /proc/71371/task/73776/exe (readlink: Permission denied) 2024-05-21T21:59:26.668 INFO:tasks.workunit.client.0.smithi151.stdout:ceph-mds 71371 73776 md_submit ceph NOFD /proc/71371/task/73776/fd (opendir: Permission denied) 2024-05-21T21:59:26.668 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/4 73054 root cwd unknown /proc/73054/cwd (readlink: Permission denied) 2024-05-21T21:59:26.668 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/4 73054 root rtd unknown /proc/73054/root (readlink: Permission denied) 2024-05-21T21:59:26.668 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/4 73054 root txt unknown /proc/73054/exe (readlink: Permission denied) 2024-05-21T21:59:26.668 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/4 73054 root NOFD /proc/73054/fd (opendir: Permission denied) 2024-05-21T21:59:26.668 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/u 73088 root cwd unknown /proc/73088/cwd (readlink: Permission denied) 2024-05-21T21:59:26.668 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/u 73088 root rtd unknown /proc/73088/root (readlink: Permission denied) 2024-05-21T21:59:26.668 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/u 73088 root txt unknown /proc/73088/exe (readlink: Permission denied) 2024-05-21T21:59:26.668 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/u 73088 root NOFD /proc/73088/fd (opendir: Permission denied) 2024-05-21T21:59:26.668 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/7 75934 root cwd unknown /proc/75934/cwd (readlink: Permission denied) 2024-05-21T21:59:26.668 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/7 75934 root rtd unknown /proc/75934/root (readlink: Permission denied) 2024-05-21T21:59:26.668 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/7 75934 root txt unknown /proc/75934/exe (readlink: Permission denied) 2024-05-21T21:59:26.668 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/7 75934 root NOFD /proc/75934/fd (opendir: Permission denied) 2024-05-21T21:59:26.669 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/3 77993 root cwd unknown /proc/77993/cwd (readlink: Permission denied) 2024-05-21T21:59:26.669 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/3 77993 root rtd unknown /proc/77993/root (readlink: Permission denied) 2024-05-21T21:59:26.669 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/3 77993 root txt unknown /proc/77993/exe (readlink: Permission denied) 2024-05-21T21:59:26.669 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/3 77993 root NOFD /proc/77993/fd (opendir: Permission denied) 2024-05-21T21:59:26.669 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/2 80003 root cwd unknown /proc/80003/cwd (readlink: Permission denied) 2024-05-21T21:59:26.669 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/2 80003 root rtd unknown /proc/80003/root (readlink: Permission denied) 2024-05-21T21:59:26.669 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/2 80003 root txt unknown /proc/80003/exe (readlink: Permission denied) 2024-05-21T21:59:26.669 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/2 80003 root NOFD /proc/80003/fd (opendir: Permission denied) 2024-05-21T21:59:26.669 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/7 81571 root cwd unknown /proc/81571/cwd (readlink: Permission denied) 2024-05-21T21:59:26.669 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/7 81571 root rtd unknown /proc/81571/root (readlink: Permission denied) 2024-05-21T21:59:26.669 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/7 81571 root txt unknown /proc/81571/exe (readlink: Permission denied) 2024-05-21T21:59:26.669 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/7 81571 root NOFD /proc/81571/fd (opendir: Permission denied) 2024-05-21T21:59:26.669 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/0 82900 root cwd unknown /proc/82900/cwd (readlink: Permission denied) 2024-05-21T21:59:26.670 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/0 82900 root rtd unknown /proc/82900/root (readlink: Permission denied) 2024-05-21T21:59:26.670 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/0 82900 root txt unknown /proc/82900/exe (readlink: Permission denied) 2024-05-21T21:59:26.670 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/0 82900 root NOFD /proc/82900/fd (opendir: Permission denied) 2024-05-21T21:59:26.670 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/6 83148 root cwd unknown /proc/83148/cwd (readlink: Permission denied) 2024-05-21T21:59:26.670 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/6 83148 root rtd unknown /proc/83148/root (readlink: Permission denied) 2024-05-21T21:59:26.670 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/6 83148 root txt unknown /proc/83148/exe (readlink: Permission denied) 2024-05-21T21:59:26.670 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/6 83148 root NOFD /proc/83148/fd (opendir: Permission denied) 2024-05-21T21:59:26.670 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/5 86148 root cwd unknown /proc/86148/cwd (readlink: Permission denied) 2024-05-21T21:59:26.670 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/5 86148 root rtd unknown /proc/86148/root (readlink: Permission denied) 2024-05-21T21:59:26.670 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/5 86148 root txt unknown /proc/86148/exe (readlink: Permission denied) 2024-05-21T21:59:26.670 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/5 86148 root NOFD /proc/86148/fd (opendir: Permission denied) 2024-05-21T21:59:26.670 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 86577 root cwd unknown /proc/86577/cwd (readlink: Permission denied) 2024-05-21T21:59:26.670 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 86577 root rtd unknown /proc/86577/root (readlink: Permission denied) 2024-05-21T21:59:26.671 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 86577 root txt unknown /proc/86577/exe (readlink: Permission denied) 2024-05-21T21:59:26.671 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 86577 root NOFD /proc/86577/fd (opendir: Permission denied) 2024-05-21T21:59:26.671 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 86596 root cwd unknown /proc/86596/cwd (readlink: Permission denied) 2024-05-21T21:59:26.671 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 86596 root rtd unknown /proc/86596/root (readlink: Permission denied) 2024-05-21T21:59:26.671 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 86596 root txt unknown /proc/86596/exe (readlink: Permission denied) 2024-05-21T21:59:26.671 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 86596 root NOFD /proc/86596/fd (opendir: Permission denied) 2024-05-21T21:59:26.671 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 86597 root cwd unknown /proc/86597/cwd (readlink: Permission denied) 2024-05-21T21:59:26.671 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 86597 root rtd unknown /proc/86597/root (readlink: Permission denied) 2024-05-21T21:59:26.671 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 86597 root txt unknown /proc/86597/exe (readlink: Permission denied) 2024-05-21T21:59:26.671 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/R 86597 root NOFD /proc/86597/fd (opendir: Permission denied) 2024-05-21T21:59:26.671 INFO:tasks.workunit.client.0.smithi151.stdout:sshd 86939 root cwd unknown /proc/86939/cwd (readlink: Permission denied) 2024-05-21T21:59:26.671 INFO:tasks.workunit.client.0.smithi151.stdout:sshd 86939 root rtd unknown /proc/86939/root (readlink: Permission denied) 2024-05-21T21:59:26.671 INFO:tasks.workunit.client.0.smithi151.stdout:sshd 86939 root txt unknown /proc/86939/exe (readlink: Permission denied) 2024-05-21T21:59:26.671 INFO:tasks.workunit.client.0.smithi151.stdout:sshd 86939 root NOFD /proc/86939/fd (opendir: Permission denied) 2024-05-21T21:59:26.671 INFO:tasks.workunit.client.0.smithi151.stdout:sshd 86999 root cwd unknown /proc/86999/cwd (readlink: Permission denied) 2024-05-21T21:59:26.672 INFO:tasks.workunit.client.0.smithi151.stdout:sshd 86999 root rtd unknown /proc/86999/root (readlink: Permission denied) 2024-05-21T21:59:26.672 INFO:tasks.workunit.client.0.smithi151.stdout:sshd 86999 root txt unknown /proc/86999/exe (readlink: Permission denied) 2024-05-21T21:59:26.672 INFO:tasks.workunit.client.0.smithi151.stdout:sshd 86999 root NOFD /proc/86999/fd (opendir: Permission denied) 2024-05-21T21:59:26.672 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/6 87429 root cwd unknown /proc/87429/cwd (readlink: Permission denied) 2024-05-21T21:59:26.672 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/6 87429 root rtd unknown /proc/87429/root (readlink: Permission denied) 2024-05-21T21:59:26.672 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/6 87429 root txt unknown /proc/87429/exe (readlink: Permission denied) 2024-05-21T21:59:26.672 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/6 87429 root NOFD /proc/87429/fd (opendir: Permission denied) 2024-05-21T21:59:26.672 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/4 94733 root cwd unknown /proc/94733/cwd (readlink: Permission denied) 2024-05-21T21:59:26.672 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/4 94733 root rtd unknown /proc/94733/root (readlink: Permission denied) 2024-05-21T21:59:26.672 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/4 94733 root txt unknown /proc/94733/exe (readlink: Permission denied) 2024-05-21T21:59:26.672 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/4 94733 root NOFD /proc/94733/fd (opendir: Permission denied) 2024-05-21T21:59:26.672 INFO:tasks.workunit.client.0.smithi151.stdout:timeout 96601 ubuntu cwd DIR 0,119 4 1099511627782 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-21T21:59:26.672 INFO:tasks.workunit.client.0.smithi151.stdout:timeout 96601 ubuntu rtd DIR 8,1 4096 2 / 2024-05-21T21:59:26.672 INFO:tasks.workunit.client.0.smithi151.stdout:timeout 96601 ubuntu txt REG 8,1 41144 7580 /usr/bin/timeout 2024-05-21T21:59:26.672 INFO:tasks.workunit.client.0.smithi151.stdout:timeout 96601 ubuntu mem REG 8,1 2586930 16078 /usr/lib/locale/en_US.utf8/LC_COLLATE 2024-05-21T21:59:26.673 INFO:tasks.workunit.client.0.smithi151.stdout:timeout 96601 ubuntu mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-05-21T21:59:26.673 INFO:tasks.workunit.client.0.smithi151.stdout:timeout 96601 ubuntu mem REG 8,1 346132 28859 /usr/lib/locale/C.utf8/LC_CTYPE 2024-05-21T21:59:26.673 INFO:tasks.workunit.client.0.smithi151.stdout:timeout 96601 ubuntu mem REG 8,1 54 16081 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2024-05-21T21:59:26.673 INFO:tasks.workunit.client.0.smithi151.stdout:timeout 96601 ubuntu mem REG 8,1 3284 28786 /usr/lib/locale/en_US.utf8/LC_TIME 2024-05-21T21:59:26.673 INFO:tasks.workunit.client.0.smithi151.stdout:timeout 96601 ubuntu mem REG 8,1 286 28783 /usr/lib/locale/en_US.utf8/LC_MONETARY 2024-05-21T21:59:26.673 INFO:tasks.workunit.client.0.smithi151.stdout:timeout 96601 ubuntu mem REG 8,1 57 26536 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2024-05-21T21:59:26.673 INFO:tasks.workunit.client.0.smithi151.stdout:timeout 96601 ubuntu mem REG 8,1 34 28836 /usr/lib/locale/en_US.utf8/LC_PAPER 2024-05-21T21:59:26.673 INFO:tasks.workunit.client.0.smithi151.stdout:timeout 96601 ubuntu mem REG 8,1 77 16080 /usr/lib/locale/en_US.utf8/LC_NAME 2024-05-21T21:59:26.673 INFO:tasks.workunit.client.0.smithi151.stdout:timeout 96601 ubuntu mem REG 8,1 167 28778 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2024-05-21T21:59:26.673 INFO:tasks.workunit.client.0.smithi151.stdout:timeout 96601 ubuntu mem REG 8,1 59 28785 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2024-05-21T21:59:26.673 INFO:tasks.workunit.client.0.smithi151.stdout:timeout 96601 ubuntu mem REG 8,1 23 28782 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2024-05-21T21:59:26.673 INFO:tasks.workunit.client.0.smithi151.stdout:timeout 96601 ubuntu mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-05-21T21:59:26.673 INFO:tasks.workunit.client.0.smithi151.stdout:timeout 96601 ubuntu mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-05-21T21:59:26.673 INFO:tasks.workunit.client.0.smithi151.stdout:timeout 96601 ubuntu mem REG 8,1 369 28781 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2024-05-21T21:59:26.673 INFO:tasks.workunit.client.0.smithi151.stdout:timeout 96601 ubuntu 0r FIFO 0,14 0t0 272751 pipe 2024-05-21T21:59:26.673 INFO:tasks.workunit.client.0.smithi151.stdout:timeout 96601 ubuntu 1w FIFO 0,14 0t0 272752 pipe 2024-05-21T21:59:26.674 INFO:tasks.workunit.client.0.smithi151.stdout:timeout 96601 ubuntu 2w FIFO 0,14 0t0 272753 pipe 2024-05-21T21:59:26.674 INFO:tasks.workunit.client.0.smithi151.stdout:fsync-tes 98005 ubuntu cwd DIR 0,119 4 1099511627782 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-21T21:59:26.674 INFO:tasks.workunit.client.0.smithi151.stdout:fsync-tes 98005 ubuntu rtd DIR 8,1 4096 2 / 2024-05-21T21:59:26.674 INFO:tasks.workunit.client.0.smithi151.stdout:fsync-tes 98005 ubuntu txt REG 8,1 1389072 7149 /usr/bin/bash 2024-05-21T21:59:26.674 INFO:tasks.workunit.client.0.smithi151.stdout:fsync-tes 98005 ubuntu mem REG 8,1 2586930 16078 /usr/lib/locale/en_US.utf8/LC_COLLATE 2024-05-21T21:59:26.674 INFO:tasks.workunit.client.0.smithi151.stdout:fsync-tes 98005 ubuntu mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-05-21T21:59:26.674 INFO:tasks.workunit.client.0.smithi151.stdout:fsync-tes 98005 ubuntu mem REG 8,1 346132 28859 /usr/lib/locale/C.utf8/LC_CTYPE 2024-05-21T21:59:26.674 INFO:tasks.workunit.client.0.smithi151.stdout:fsync-tes 98005 ubuntu mem REG 8,1 54 16081 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2024-05-21T21:59:26.674 INFO:tasks.workunit.client.0.smithi151.stdout:fsync-tes 98005 ubuntu mem REG 8,1 3284 28786 /usr/lib/locale/en_US.utf8/LC_TIME 2024-05-21T21:59:26.674 INFO:tasks.workunit.client.0.smithi151.stdout:fsync-tes 98005 ubuntu mem REG 8,1 286 28783 /usr/lib/locale/en_US.utf8/LC_MONETARY 2024-05-21T21:59:26.674 INFO:tasks.workunit.client.0.smithi151.stdout:fsync-tes 98005 ubuntu mem REG 8,1 57 26536 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2024-05-21T21:59:26.674 INFO:tasks.workunit.client.0.smithi151.stdout:fsync-tes 98005 ubuntu mem REG 8,1 34 28836 /usr/lib/locale/en_US.utf8/LC_PAPER 2024-05-21T21:59:26.675 INFO:tasks.workunit.client.0.smithi151.stdout:fsync-tes 98005 ubuntu mem REG 8,1 77 16080 /usr/lib/locale/en_US.utf8/LC_NAME 2024-05-21T21:59:26.675 INFO:tasks.workunit.client.0.smithi151.stdout:fsync-tes 98005 ubuntu mem REG 8,1 167 28778 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2024-05-21T21:59:26.675 INFO:tasks.workunit.client.0.smithi151.stdout:fsync-tes 98005 ubuntu mem REG 8,1 59 28785 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2024-05-21T21:59:26.675 INFO:tasks.workunit.client.0.smithi151.stdout:fsync-tes 98005 ubuntu mem REG 8,1 195080 7147 /usr/lib64/libtinfo.so.6.2 2024-05-21T21:59:26.675 INFO:tasks.workunit.client.0.smithi151.stdout:fsync-tes 98005 ubuntu mem REG 8,1 23 28782 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2024-05-21T21:59:26.675 INFO:tasks.workunit.client.0.smithi151.stdout:fsync-tes 98005 ubuntu mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-05-21T21:59:26.675 INFO:tasks.workunit.client.0.smithi151.stdout:fsync-tes 98005 ubuntu mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-05-21T21:59:26.675 INFO:tasks.workunit.client.0.smithi151.stdout:fsync-tes 98005 ubuntu mem REG 8,1 369 28781 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2024-05-21T21:59:26.675 INFO:tasks.workunit.client.0.smithi151.stdout:fsync-tes 98005 ubuntu 0r FIFO 0,14 0t0 272751 pipe 2024-05-21T21:59:26.675 INFO:tasks.workunit.client.0.smithi151.stdout:fsync-tes 98005 ubuntu 1w FIFO 0,14 0t0 272752 pipe 2024-05-21T21:59:26.675 INFO:tasks.workunit.client.0.smithi151.stdout:fsync-tes 98005 ubuntu 2w FIFO 0,14 0t0 272753 pipe 2024-05-21T21:59:26.675 INFO:tasks.workunit.client.0.smithi151.stdout:fsync-tes 98005 ubuntu 255r REG 8,1 207 662291 /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsync-tester.sh 2024-05-21T21:59:26.676 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/u 98211 root cwd unknown /proc/98211/cwd (readlink: Permission denied) 2024-05-21T21:59:26.676 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/u 98211 root rtd unknown /proc/98211/root (readlink: Permission denied) 2024-05-21T21:59:26.676 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/u 98211 root txt unknown /proc/98211/exe (readlink: Permission denied) 2024-05-21T21:59:26.676 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/u 98211 root NOFD /proc/98211/fd (opendir: Permission denied) 2024-05-21T21:59:26.676 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/u 98212 root cwd unknown /proc/98212/cwd (readlink: Permission denied) 2024-05-21T21:59:26.676 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/u 98212 root rtd unknown /proc/98212/root (readlink: Permission denied) 2024-05-21T21:59:26.676 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/u 98212 root txt unknown /proc/98212/exe (readlink: Permission denied) 2024-05-21T21:59:26.676 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/u 98212 root NOFD /proc/98212/fd (opendir: Permission denied) 2024-05-21T21:59:26.676 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/3 98213 root cwd unknown /proc/98213/cwd (readlink: Permission denied) 2024-05-21T21:59:26.676 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/3 98213 root rtd unknown /proc/98213/root (readlink: Permission denied) 2024-05-21T21:59:26.676 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/3 98213 root txt unknown /proc/98213/exe (readlink: Permission denied) 2024-05-21T21:59:26.676 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/3 98213 root NOFD /proc/98213/fd (opendir: Permission denied) 2024-05-21T21:59:26.676 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/3 98214 root cwd unknown /proc/98214/cwd (readlink: Permission denied) 2024-05-21T21:59:26.676 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/3 98214 root rtd unknown /proc/98214/root (readlink: Permission denied) 2024-05-21T21:59:26.676 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/3 98214 root txt unknown /proc/98214/exe (readlink: Permission denied) 2024-05-21T21:59:26.676 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/3 98214 root NOFD /proc/98214/fd (opendir: Permission denied) 2024-05-21T21:59:26.677 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/7 98399 root cwd unknown /proc/98399/cwd (readlink: Permission denied) 2024-05-21T21:59:26.677 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/7 98399 root rtd unknown /proc/98399/root (readlink: Permission denied) 2024-05-21T21:59:26.677 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/7 98399 root txt unknown /proc/98399/exe (readlink: Permission denied) 2024-05-21T21:59:26.677 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/7 98399 root NOFD /proc/98399/fd (opendir: Permission denied) 2024-05-21T21:59:26.677 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/7 98401 root cwd unknown /proc/98401/cwd (readlink: Permission denied) 2024-05-21T21:59:26.677 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/7 98401 root rtd unknown /proc/98401/root (readlink: Permission denied) 2024-05-21T21:59:26.677 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/7 98401 root txt unknown /proc/98401/exe (readlink: Permission denied) 2024-05-21T21:59:26.677 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/7 98401 root NOFD /proc/98401/fd (opendir: Permission denied) 2024-05-21T21:59:26.677 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/1 98456 root cwd unknown /proc/98456/cwd (readlink: Permission denied) 2024-05-21T21:59:26.677 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/1 98456 root rtd unknown /proc/98456/root (readlink: Permission denied) 2024-05-21T21:59:26.677 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/1 98456 root txt unknown /proc/98456/exe (readlink: Permission denied) 2024-05-21T21:59:26.677 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/1 98456 root NOFD /proc/98456/fd (opendir: Permission denied) 2024-05-21T21:59:26.677 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/5 99659 root cwd unknown /proc/99659/cwd (readlink: Permission denied) 2024-05-21T21:59:26.677 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/5 99659 root rtd unknown /proc/99659/root (readlink: Permission denied) 2024-05-21T21:59:26.678 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/5 99659 root txt unknown /proc/99659/exe (readlink: Permission denied) 2024-05-21T21:59:26.678 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/5 99659 root NOFD /proc/99659/fd (opendir: Permission denied) 2024-05-21T21:59:26.678 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/1 99660 root cwd unknown /proc/99660/cwd (readlink: Permission denied) 2024-05-21T21:59:26.678 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/1 99660 root rtd unknown /proc/99660/root (readlink: Permission denied) 2024-05-21T21:59:26.678 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/1 99660 root txt unknown /proc/99660/exe (readlink: Permission denied) 2024-05-21T21:59:26.678 INFO:tasks.workunit.client.0.smithi151.stdout:kworker/1 99660 root NOFD /proc/99660/fd (opendir: Permission denied) 2024-05-21T21:59:26.678 INFO:tasks.workunit.client.0.smithi151.stdout:lsof 99662 ubuntu cwd DIR 0,119 4 1099511627782 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-21T21:59:26.678 INFO:tasks.workunit.client.0.smithi151.stdout:lsof 99662 ubuntu rtd DIR 8,1 4096 2 / 2024-05-21T21:59:26.678 INFO:tasks.workunit.client.0.smithi151.stdout:lsof 99662 ubuntu txt REG 8,1 179232 9797 /usr/bin/lsof 2024-05-21T21:59:26.678 INFO:tasks.workunit.client.0.smithi151.stdout:lsof 99662 ubuntu mem REG 8,1 5431656 24515 /usr/lib64/libcrypto.so.3.2.1 2024-05-21T21:59:26.678 INFO:tasks.workunit.client.0.smithi151.stdout:lsof 99662 ubuntu mem REG 8,1 346132 28859 /usr/lib/locale/C.utf8/LC_CTYPE 2024-05-21T21:59:26.678 INFO:tasks.workunit.client.0.smithi151.stdout:lsof 99662 ubuntu mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-05-21T21:59:26.678 INFO:tasks.workunit.client.0.smithi151.stdout:lsof 99662 ubuntu mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-05-21T21:59:26.679 INFO:tasks.workunit.client.0.smithi151.stdout:lsof 99662 ubuntu mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-05-21T21:59:26.679 INFO:tasks.workunit.client.0.smithi151.stdout:lsof 99662 ubuntu mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-05-21T21:59:26.679 INFO:tasks.workunit.client.0.smithi151.stdout:lsof 99662 ubuntu mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-05-21T21:59:26.679 INFO:tasks.workunit.client.0.smithi151.stdout:lsof 99662 ubuntu mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-05-21T21:59:26.679 INFO:tasks.workunit.client.0.smithi151.stdout:lsof 99662 ubuntu mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-05-21T21:59:26.679 INFO:tasks.workunit.client.0.smithi151.stdout:lsof 99662 ubuntu mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-05-21T21:59:26.679 INFO:tasks.workunit.client.0.smithi151.stdout:lsof 99662 ubuntu mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-05-21T21:59:26.679 INFO:tasks.workunit.client.0.smithi151.stdout:lsof 99662 ubuntu mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-05-21T21:59:26.679 INFO:tasks.workunit.client.0.smithi151.stdout:lsof 99662 ubuntu mem REG 8,1 196224 8914 /usr/lib64/libtirpc.so.3.0.0 2024-05-21T21:59:26.679 INFO:tasks.workunit.client.0.smithi151.stdout:lsof 99662 ubuntu mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-05-21T21:59:26.679 INFO:tasks.workunit.client.0.smithi151.stdout:lsof 99662 ubuntu mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-05-21T21:59:26.679 INFO:tasks.workunit.client.0.smithi151.stdout:lsof 99662 ubuntu mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-05-21T21:59:26.679 INFO:tasks.workunit.client.0.smithi151.stdout:lsof 99662 ubuntu 0r FIFO 0,14 0t0 272751 pipe 2024-05-21T21:59:26.679 INFO:tasks.workunit.client.0.smithi151.stdout:lsof 99662 ubuntu 1w FIFO 0,14 0t0 272752 pipe 2024-05-21T21:59:26.680 INFO:tasks.workunit.client.0.smithi151.stdout:lsof 99662 ubuntu 2w FIFO 0,14 0t0 272753 pipe 2024-05-21T21:59:26.680 INFO:tasks.workunit.client.0.smithi151.stdout:lsof 99662 ubuntu 3r DIR 0,21 0 1 /proc 2024-05-21T21:59:26.680 INFO:tasks.workunit.client.0.smithi151.stdout:lsof 99662 ubuntu 4r DIR 0,21 7 279893 /proc/99662/fd 2024-05-21T21:59:26.680 INFO:tasks.workunit.client.0.smithi151.stdout:lsof 99662 ubuntu 5w FIFO 0,14 0t0 279898 pipe 2024-05-21T21:59:26.680 INFO:tasks.workunit.client.0.smithi151.stdout:lsof 99662 ubuntu 6r FIFO 0,14 0t0 279899 pipe 2024-05-21T21:59:26.680 INFO:tasks.workunit.client.0.smithi151.stdout:lsof 99663 ubuntu cwd DIR 0,119 4 1099511627782 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-21T21:59:26.680 INFO:tasks.workunit.client.0.smithi151.stdout:lsof 99663 ubuntu rtd DIR 8,1 4096 2 / 2024-05-21T21:59:26.680 INFO:tasks.workunit.client.0.smithi151.stdout:lsof 99663 ubuntu txt REG 8,1 179232 9797 /usr/bin/lsof 2024-05-21T21:59:26.680 INFO:tasks.workunit.client.0.smithi151.stdout:lsof 99663 ubuntu mem REG 8,1 5431656 24515 /usr/lib64/libcrypto.so.3.2.1 2024-05-21T21:59:26.680 INFO:tasks.workunit.client.0.smithi151.stdout:lsof 99663 ubuntu mem REG 8,1 346132 28859 /usr/lib/locale/C.utf8/LC_CTYPE 2024-05-21T21:59:26.680 INFO:tasks.workunit.client.0.smithi151.stdout:lsof 99663 ubuntu mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-05-21T21:59:26.680 INFO:tasks.workunit.client.0.smithi151.stdout:lsof 99663 ubuntu mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-05-21T21:59:26.680 INFO:tasks.workunit.client.0.smithi151.stdout:lsof 99663 ubuntu mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-05-21T21:59:26.681 INFO:tasks.workunit.client.0.smithi151.stdout:lsof 99663 ubuntu mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-05-21T21:59:26.681 INFO:tasks.workunit.client.0.smithi151.stdout:lsof 99663 ubuntu mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-05-21T21:59:26.681 INFO:tasks.workunit.client.0.smithi151.stdout:lsof 99663 ubuntu mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-05-21T21:59:26.681 INFO:tasks.workunit.client.0.smithi151.stdout:lsof 99663 ubuntu mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-05-21T21:59:26.681 INFO:tasks.workunit.client.0.smithi151.stdout:lsof 99663 ubuntu mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-05-21T21:59:26.681 INFO:tasks.workunit.client.0.smithi151.stdout:lsof 99663 ubuntu mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-05-21T21:59:26.681 INFO:tasks.workunit.client.0.smithi151.stdout:lsof 99663 ubuntu mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-05-21T21:59:26.681 INFO:tasks.workunit.client.0.smithi151.stdout:lsof 99663 ubuntu mem REG 8,1 196224 8914 /usr/lib64/libtirpc.so.3.0.0 2024-05-21T21:59:26.681 INFO:tasks.workunit.client.0.smithi151.stdout:lsof 99663 ubuntu mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-05-21T21:59:26.681 INFO:tasks.workunit.client.0.smithi151.stdout:lsof 99663 ubuntu mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-05-21T21:59:26.681 INFO:tasks.workunit.client.0.smithi151.stdout:lsof 99663 ubuntu mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-05-21T21:59:26.681 INFO:tasks.workunit.client.0.smithi151.stdout:lsof 99663 ubuntu 4r FIFO 0,14 0t0 279898 pipe 2024-05-21T21:59:26.681 INFO:tasks.workunit.client.0.smithi151.stdout:lsof 99663 ubuntu 7w FIFO 0,14 0t0 279899 pipe 2024-05-21T21:59:26.682 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-21T21:59:26.682 DEBUG:teuthology.orchestra.run.smithi151:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-21T21:59:26.789 INFO:tasks.workunit:Stopping ['suites/fsync-tester.sh'] on client.0... 2024-05-21T21:59:26.789 DEBUG:teuthology.orchestra.run.smithi151:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-05-21T21:59:27.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:59:27 smithi151 ceph-mon[31623]: pgmap v118: 97 pgs: 97 active+clean; 285 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 MiB/s wr, 1.14k op/s 2024-05-21T21:59:27.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:59:27 smithi184 ceph-mon[36942]: pgmap v118: 97 pgs: 97 active+clean; 285 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 MiB/s wr, 1.14k op/s 2024-05-21T21:59:27.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:59:27 smithi164 ceph-mon[36269]: pgmap v118: 97 pgs: 97 active+clean; 285 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 MiB/s wr, 1.14k op/s 2024-05-21T21:59:27.847 DEBUG:teuthology.parallel:result is None 2024-05-21T21:59:27.848 DEBUG:teuthology.orchestra.run.smithi151:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-05-21T21:59:27.943 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-05-21T21:59:27.943 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-05-21T21:59:27.956 INFO:tasks.quiescer:joining Quiescers 2024-05-21T21:59:27.957 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-05-21T21:59:27.957 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-21T21:59:28.393 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:59:29.375 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-21T21:59:29.621 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:59:29 smithi151 ceph-mon[31623]: pgmap v119: 97 pgs: 97 active+clean; 285 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 MiB/s wr, 1.15k op/s 2024-05-21T21:59:29.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:59:29 smithi184 ceph-mon[36942]: pgmap v119: 97 pgs: 97 active+clean; 285 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 MiB/s wr, 1.15k op/s 2024-05-21T21:59:29.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:59:29 smithi164 ceph-mon[36269]: pgmap v119: 97 pgs: 97 active+clean; 285 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 MiB/s wr, 1.15k op/s 2024-05-21T21:59:29.939 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:59:30.762 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:30.760+0000 7fa5fec00700 1 -- 172.21.15.151:0/589145893 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa5f810a440 msgr2=0x7fa5f81168e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:59:30.762 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:30.760+0000 7fa5fec00700 1 --2- 172.21.15.151:0/589145893 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa5f810a440 0x7fa5f81168e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa5f000b0a0 tx=0x7fa5f000b3b0 comp rx=0 tx=0).stop 2024-05-21T21:59:30.762 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:30.760+0000 7fa5fec00700 1 -- 172.21.15.151:0/589145893 shutdown_connections 2024-05-21T21:59:30.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:30.760+0000 7fa5fec00700 1 --2- 172.21.15.151:0/589145893 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa5f810a440 0x7fa5f81168e0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:30.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:30.760+0000 7fa5fec00700 1 --2- 172.21.15.151:0/589145893 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5f810e950 0x7fa5f810ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:30.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:30.760+0000 7fa5fec00700 1 --2- 172.21.15.151:0/589145893 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa5f8109a80 0x7fa5f8109f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:30.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:30.760+0000 7fa5fec00700 1 -- 172.21.15.151:0/589145893 >> 172.21.15.151:0/589145893 conn(0x7fa5f80792d0 msgr2=0x7fa5f8108360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:59:30.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:30.761+0000 7fa5fec00700 1 -- 172.21.15.151:0/589145893 shutdown_connections 2024-05-21T21:59:30.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:30.761+0000 7fa5fec00700 1 -- 172.21.15.151:0/589145893 wait complete. 2024-05-21T21:59:30.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:30.761+0000 7fa5fec00700 1 Processor -- start 2024-05-21T21:59:30.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:30.762+0000 7fa5fec00700 1 -- start start 2024-05-21T21:59:30.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:30.762+0000 7fa5fec00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa5f8109a80 0x7fa5f81a0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:30.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:30.762+0000 7fa5fec00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5f810a440 0x7fa5f81a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:30.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:30.762+0000 7fa5fec00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa5f810e950 0x7fa5f81a49e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:30.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:30.762+0000 7fa5fec00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f8119c80 con 0x7fa5f8109a80 2024-05-21T21:59:30.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:30.762+0000 7fa5fec00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f8119b00 con 0x7fa5f810e950 2024-05-21T21:59:30.766 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:30.762+0000 7fa5fec00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f8119e00 con 0x7fa5f810a440 2024-05-21T21:59:30.766 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:30.762+0000 7fa5fd800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa5f8109a80 0x7fa5f81a0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:30.766 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:30.762+0000 7fa5fd800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa5f8109a80 0x7fa5f81a0110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:42398/0 (socket says 172.21.15.151:42398) 2024-05-21T21:59:30.766 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:30.762+0000 7fa5fd800700 1 -- 172.21.15.151:0/1646075505 learned_addr learned my addr 172.21.15.151:0/1646075505 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:59:30.766 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:30.763+0000 7fa5fce00700 1 --2- 172.21.15.151:0/1646075505 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5f810a440 0x7fa5f81a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:30.766 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:30.763+0000 7fa5fe200700 1 --2- 172.21.15.151:0/1646075505 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa5f810e950 0x7fa5f81a49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:30.766 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:30.763+0000 7fa5fd800700 1 -- 172.21.15.151:0/1646075505 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5f810a440 msgr2=0x7fa5f81a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:59:30.766 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:30.763+0000 7fa5fd800700 1 --2- 172.21.15.151:0/1646075505 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5f810a440 0x7fa5f81a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:30.766 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:30.763+0000 7fa5fd800700 1 -- 172.21.15.151:0/1646075505 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa5f810e950 msgr2=0x7fa5f81a49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:59:30.766 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:30.763+0000 7fa5fd800700 1 --2- 172.21.15.151:0/1646075505 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa5f810e950 0x7fa5f81a49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:30.766 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:30.763+0000 7fa5fd800700 1 -- 172.21.15.151:0/1646075505 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5f81a5120 con 0x7fa5f8109a80 2024-05-21T21:59:30.766 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:30.763+0000 7fa5fd800700 1 --2- 172.21.15.151:0/1646075505 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa5f8109a80 0x7fa5f81a0110 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fa5ec00e9a0 tx=0x7fa5ec00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:59:30.767 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:30.763+0000 7fa5f6000700 1 -- 172.21.15.151:0/1646075505 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5ec00ca10 con 0x7fa5f8109a80 2024-05-21T21:59:30.767 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:30.764+0000 7fa5fec00700 1 -- 172.21.15.151:0/1646075505 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5f81a5410 con 0x7fa5f8109a80 2024-05-21T21:59:30.767 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:30.764+0000 7fa5f6000700 1 -- 172.21.15.151:0/1646075505 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5ec00cbb0 con 0x7fa5f8109a80 2024-05-21T21:59:30.767 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:30.764+0000 7fa5f6000700 1 -- 172.21.15.151:0/1646075505 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5ec018680 con 0x7fa5f8109a80 2024-05-21T21:59:30.767 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:30.764+0000 7fa5fec00700 1 -- 172.21.15.151:0/1646075505 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5f81acc90 con 0x7fa5f8109a80 2024-05-21T21:59:30.767 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:30.765+0000 7fa5fec00700 1 -- 172.21.15.151:0/1646075505 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5f806a230 con 0x7fa5f8109a80 2024-05-21T21:59:30.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:30.767+0000 7fa5f6000700 1 -- 172.21.15.151:0/1646075505 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7fa5ec010430 con 0x7fa5f8109a80 2024-05-21T21:59:30.770 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:30.769+0000 7fa5f6000700 1 --2- 172.21.15.151:0/1646075505 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fa5e00b29f0 0x7fa5e00b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:30.770 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:30.769+0000 7fa5f6000700 1 -- 172.21.15.151:0/1646075505 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fa5ec0e2720 con 0x7fa5f8109a80 2024-05-21T21:59:30.770 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:30.769+0000 7fa5fce00700 1 --2- 172.21.15.151:0/1646075505 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fa5e00b29f0 0x7fa5e00b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:30.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:30.770+0000 7fa5fce00700 1 --2- 172.21.15.151:0/1646075505 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fa5e00b29f0 0x7fa5e00b4eb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa5f8109630 tx=0x7fa5e4009260 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:59:30.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:30.773+0000 7fa5f6000700 1 -- 172.21.15.151:0/1646075505 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa5ec0aa110 con 0x7fa5f8109a80 2024-05-21T21:59:31.068 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:31.066+0000 7fa5fec00700 1 -- 172.21.15.151:0/1646075505 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa5f81a1470 con 0x7fa5f8109a80 2024-05-21T21:59:31.069 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:31.068+0000 7fa5f6000700 1 -- 172.21.15.151:0/1646075505 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11958 (secure 0 0 0) 0x7fa5ec0b2190 con 0x7fa5f8109a80 2024-05-21T21:59:31.070 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T21:59:31.070 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14358,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/158010529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":158010529},{"type":"v1","addr":"172.21.15.151:6835","nonce":158010529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6839/2069058115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":2069058115},{"type":"v1","addr":"172.21.15.151:6839","nonce":2069058115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T21:51:19.552069+0000","modified":"2024-05-21T21:55:34.643755+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24530,"mds_1":24493,"mds_2":24491,"mds_3":24478,"mds_4":24473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14403":{"gid":14403,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6841/1749723794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6840","nonce":1749723794},{"type":"v1","addr":"172.21.15.151:6841","nonce":1749723794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/2026266114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":2026266114},{"type":"v1","addr":"172.21.15.184:6833","nonce":2026266114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/2467900844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":2467900844},{"type":"v1","addr":"172.21.15.164:6833","nonce":2467900844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3367640960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3367640960},{"type":"v1","addr":"172.21.15.151:6837","nonce":3367640960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/3264142621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3264142621},{"type":"v1","addr":"172.21.15.184:6835","nonce":3264142621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":17,"addr":"172.21.15.164:6835/2340428470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2340428470},{"type":"v1","addr":"172.21.15.164:6835","nonce":2340428470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":16,"addr":"172.21.15.184:6837/2988194888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":2988194888},{"type":"v1","addr":"172.21.15.184:6837","nonce":2988194888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.164:6837/1554609572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1554609572},{"type":"v1","addr":"172.21.15.164:6837","nonce":1554609572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.184:6839/3059935143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":3059935143},{"type":"v1","addr":"172.21.15.184:6839","nonce":3059935143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.164:6839/2646401247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":2646401247},{"type":"v1","addr":"172.21.15.164:6839","nonce":2646401247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24530,"qdb_cluster":[24530,24493,24491,24473,24478]},"id":1}]} 2024-05-21T21:59:31.074 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:31.072+0000 7fa5fec00700 1 -- 172.21.15.151:0/1646075505 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fa5e00b29f0 msgr2=0x7fa5e00b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:59:31.074 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:31.072+0000 7fa5fec00700 1 --2- 172.21.15.151:0/1646075505 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fa5e00b29f0 0x7fa5e00b4eb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa5f8109630 tx=0x7fa5e4009260 comp rx=0 tx=0).stop 2024-05-21T21:59:31.074 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:31.072+0000 7fa5fec00700 1 -- 172.21.15.151:0/1646075505 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa5f8109a80 msgr2=0x7fa5f81a0110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:59:31.074 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:31.072+0000 7fa5fec00700 1 --2- 172.21.15.151:0/1646075505 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa5f8109a80 0x7fa5f81a0110 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fa5ec00e9a0 tx=0x7fa5ec00ecb0 comp rx=0 tx=0).stop 2024-05-21T21:59:31.074 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:31.072+0000 7fa5fec00700 1 -- 172.21.15.151:0/1646075505 shutdown_connections 2024-05-21T21:59:31.074 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:31.072+0000 7fa5fec00700 1 --2- 172.21.15.151:0/1646075505 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fa5e00b29f0 0x7fa5e00b4eb0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:31.074 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:31.072+0000 7fa5fec00700 1 --2- 172.21.15.151:0/1646075505 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa5f810e950 0x7fa5f81a49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:31.074 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:31.072+0000 7fa5fec00700 1 --2- 172.21.15.151:0/1646075505 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa5f8109a80 0x7fa5f81a0110 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:31.074 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:31.072+0000 7fa5fec00700 1 --2- 172.21.15.151:0/1646075505 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5f810a440 0x7fa5f81a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:31.074 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:31.072+0000 7fa5fec00700 1 -- 172.21.15.151:0/1646075505 >> 172.21.15.151:0/1646075505 conn(0x7fa5f80792d0 msgr2=0x7fa5f8079950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:59:31.074 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:31.072+0000 7fa5fec00700 1 -- 172.21.15.151:0/1646075505 shutdown_connections 2024-05-21T21:59:31.074 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:31.073+0000 7fa5fec00700 1 -- 172.21.15.151:0/1646075505 wait complete. 2024-05-21T21:59:31.075 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 29 2024-05-21T21:59:31.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:59:31 smithi151 ceph-mon[31623]: pgmap v120: 97 pgs: 97 active+clean; 285 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 MiB/s wr, 1.01k op/s 2024-05-21T21:59:31.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:59:31 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/1646075505' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T21:59:31.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:59:31 smithi184 ceph-mon[36942]: pgmap v120: 97 pgs: 97 active+clean; 285 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 MiB/s wr, 1.01k op/s 2024-05-21T21:59:31.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:59:31 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/1646075505' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T21:59:31.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:59:31 smithi164 ceph-mon[36269]: pgmap v120: 97 pgs: 97 active+clean; 285 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 MiB/s wr, 1.01k op/s 2024-05-21T21:59:31.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:59:31 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/1646075505' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T21:59:31.899 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24530 quiesce db --cancel --all' 2024-05-21T21:59:31.900 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- timeout 105 ceph tell mds.24530 quiesce db --cancel --all 2024-05-21T21:59:32.996 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:59:33.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.356+0000 7fefaee00700 1 -- 172.21.15.151:0/3436048540 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fefa810b760 msgr2=0x7fefa810bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:59:33.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.356+0000 7fefaee00700 1 --2- 172.21.15.151:0/3436048540 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fefa810b760 0x7fefa810bb40 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fef98009990 tx=0x7fef98009ca0 comp rx=0 tx=0).stop 2024-05-21T21:59:33.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.356+0000 7fefaee00700 1 -- 172.21.15.151:0/3436048540 shutdown_connections 2024-05-21T21:59:33.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.356+0000 7fefaee00700 1 --2- 172.21.15.151:0/3436048540 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fefa810b760 0x7fefa810bb40 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:33.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.356+0000 7fefaee00700 1 --2- 172.21.15.151:0/3436048540 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fefa8106120 0x7fefa8112630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:33.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.356+0000 7fefaee00700 1 --2- 172.21.15.151:0/3436048540 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fefa8105760 0x7fefa8105be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:33.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.356+0000 7fefaee00700 1 -- 172.21.15.151:0/3436048540 >> 172.21.15.151:0/3436048540 conn(0x7fefa8101250 msgr2=0x7fefa8103670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:59:33.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.356+0000 7fefaee00700 1 -- 172.21.15.151:0/3436048540 shutdown_connections 2024-05-21T21:59:33.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.356+0000 7fefaee00700 1 -- 172.21.15.151:0/3436048540 wait complete. 2024-05-21T21:59:33.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.358+0000 7fefaee00700 1 Processor -- start 2024-05-21T21:59:33.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.358+0000 7fefaee00700 1 -- start start 2024-05-21T21:59:33.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.359+0000 7fefaee00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fefa8105760 0x7fefa819be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:33.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.359+0000 7fefaee00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fefa8106120 0x7fefa819c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:33.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.359+0000 7fefaee00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fefa810b760 0x7fefa81a0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:33.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.359+0000 7fefaee00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fefa8115990 con 0x7fefa8106120 2024-05-21T21:59:33.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.359+0000 7fefaee00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fefa8115810 con 0x7fefa8105760 2024-05-21T21:59:33.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.359+0000 7fefaee00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fefa8115b10 con 0x7fefa810b760 2024-05-21T21:59:33.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.359+0000 7fefa7e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fefa8105760 0x7fefa819be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:33.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.359+0000 7fefa7400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fefa8106120 0x7fefa819c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:33.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.359+0000 7fefa7e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fefa8105760 0x7fefa819be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:60870/0 (socket says 172.21.15.151:60870) 2024-05-21T21:59:33.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.359+0000 7fefa7e00700 1 -- 172.21.15.151:0/4032238897 learned_addr learned my addr 172.21.15.151:0/4032238897 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:59:33.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.359+0000 7feface00700 1 --2- 172.21.15.151:0/4032238897 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fefa810b760 0x7fefa81a0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:33.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.359+0000 7fefa7e00700 1 -- 172.21.15.151:0/4032238897 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fefa810b760 msgr2=0x7fefa81a0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:59:33.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.360+0000 7fefa7e00700 1 --2- 172.21.15.151:0/4032238897 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fefa810b760 0x7fefa81a0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:33.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.360+0000 7fefa7e00700 1 -- 172.21.15.151:0/4032238897 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fefa8106120 msgr2=0x7fefa819c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:59:33.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.360+0000 7fefa7e00700 1 --2- 172.21.15.151:0/4032238897 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fefa8106120 0x7fefa819c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:33.363 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.360+0000 7fefa7e00700 1 -- 172.21.15.151:0/4032238897 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefa81a0e50 con 0x7fefa8105760 2024-05-21T21:59:33.363 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.360+0000 7fefa7400700 1 --2- 172.21.15.151:0/4032238897 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fefa8106120 0x7fefa819c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:59:33.363 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.360+0000 7fefa7e00700 1 --2- 172.21.15.151:0/4032238897 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fefa8105760 0x7fefa819be40 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fef98000c00 tx=0x7fef980057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:59:33.363 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.360+0000 7fefa4c00700 1 -- 172.21.15.151:0/4032238897 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef98027070 con 0x7fefa8105760 2024-05-21T21:59:33.363 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.360+0000 7fefaee00700 1 -- 172.21.15.151:0/4032238897 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefa81a10e0 con 0x7fefa8105760 2024-05-21T21:59:33.363 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.360+0000 7fefa4c00700 1 -- 172.21.15.151:0/4032238897 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef98022470 con 0x7fefa8105760 2024-05-21T21:59:33.363 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.360+0000 7fefa4c00700 1 -- 172.21.15.151:0/4032238897 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef98017650 con 0x7fefa8105760 2024-05-21T21:59:33.363 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.361+0000 7fefaee00700 1 -- 172.21.15.151:0/4032238897 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefa81a89c0 con 0x7fefa8105760 2024-05-21T21:59:33.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.364+0000 7fefa4c00700 1 -- 172.21.15.151:0/4032238897 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7fef9802a070 con 0x7fefa8105760 2024-05-21T21:59:33.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.365+0000 7fefb97f2b80 1 Processor -- start 2024-05-21T21:59:33.367 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.365+0000 7fefb97f2b80 1 -- start start 2024-05-21T21:59:33.368 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.366+0000 7fefa4c00700 1 --2- 172.21.15.151:0/4032238897 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fef8c0b29f0 0x7fef8c0b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:33.368 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.366+0000 7fefb97f2b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c9d8e93eb0 0x55c9d8e94290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:33.368 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.366+0000 7fefb97f2b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c9d8e947d0 0x55c9d8e8e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:33.368 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.366+0000 7fefb97f2b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c9d8e8e880 0x55c9d8e8ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:33.368 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.366+0000 7fefb97f2b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c9d8dece40 con 0x55c9d8e947d0 2024-05-21T21:59:33.368 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.366+0000 7fefb97f2b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55c9d8ddc9e0 con 0x55c9d8e8e880 2024-05-21T21:59:33.368 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.366+0000 7fefb97f2b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55c9d8de0a20 con 0x55c9d8e93eb0 2024-05-21T21:59:33.368 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.366+0000 7fefa4c00700 1 -- 172.21.15.151:0/4032238897 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fef980e33a0 con 0x7fefa8105760 2024-05-21T21:59:33.368 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.366+0000 7fefa7e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c9d8e93eb0 0x55c9d8e94290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:33.368 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.366+0000 7fefa7e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c9d8e93eb0 0x55c9d8e94290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:50970/0 (socket says 172.21.15.151:50970) 2024-05-21T21:59:33.368 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.366+0000 7fefa7e00700 1 -- 172.21.15.151:0/1966036288 learned_addr learned my addr 172.21.15.151:0/1966036288 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:59:33.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.366+0000 7feface00700 1 --2- 172.21.15.151:0/4032238897 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fef8c0b29f0 0x7fef8c0b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:33.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.367+0000 7fefa7400700 1 --2- 172.21.15.151:0/1966036288 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c9d8e947d0 0x55c9d8e8e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:33.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.367+0000 7feface00700 1 --2- 172.21.15.151:0/1966036288 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c9d8e8e880 0x55c9d8e8ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:33.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.367+0000 7fefa7e00700 1 -- 172.21.15.151:0/1966036288 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c9d8e8e880 msgr2=0x55c9d8e8ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:59:33.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.367+0000 7fefa7e00700 1 --2- 172.21.15.151:0/1966036288 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c9d8e8e880 0x55c9d8e8ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:33.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.367+0000 7fefa7e00700 1 -- 172.21.15.151:0/1966036288 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c9d8e947d0 msgr2=0x55c9d8e8e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:59:33.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.367+0000 7fefa7e00700 1 --2- 172.21.15.151:0/1966036288 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c9d8e947d0 0x55c9d8e8e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:33.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.367+0000 7fefa7e00700 1 -- 172.21.15.151:0/1966036288 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9d8da5420 con 0x55c9d8e93eb0 2024-05-21T21:59:33.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.367+0000 7feface00700 1 --2- 172.21.15.151:0/1966036288 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c9d8e8e880 0x55c9d8e8ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T21:59:33.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.367+0000 7fefa7e00700 1 --2- 172.21.15.151:0/1966036288 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c9d8e93eb0 0x55c9d8e94290 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fef980058b0 tx=0x7fef980aa5a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf60f37deea2adeb server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:59:33.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.367+0000 7fef9d600700 1 -- 172.21.15.151:0/1966036288 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef980aea70 con 0x55c9d8e93eb0 2024-05-21T21:59:33.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.368+0000 7feface00700 1 --2- 172.21.15.151:0/4032238897 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fef8c0b29f0 0x7fef8c0b4eb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fefa0009ec0 tx=0x7fefa000b040 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:59:33.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.368+0000 7fef9d600700 1 -- 172.21.15.151:0/1966036288 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef980b3490 con 0x55c9d8e93eb0 2024-05-21T21:59:33.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.368+0000 7fef9d600700 1 -- 172.21.15.151:0/1966036288 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef980e0d20 con 0x55c9d8e93eb0 2024-05-21T21:59:33.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.368+0000 7fefb97f2b80 1 -- 172.21.15.151:0/1966036288 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c9d8e93eb0 msgr2=0x55c9d8e94290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:59:33.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.368+0000 7fefb97f2b80 1 --2- 172.21.15.151:0/1966036288 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c9d8e93eb0 0x55c9d8e94290 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fef980058b0 tx=0x7fef980aa5a0 comp rx=0 tx=0).stop 2024-05-21T21:59:33.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.369+0000 7fefb97f2b80 1 -- 172.21.15.151:0/1966036288 shutdown_connections 2024-05-21T21:59:33.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.369+0000 7fefb97f2b80 1 --2- 172.21.15.151:0/1966036288 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c9d8e8e880 0x55c9d8e8ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:33.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.369+0000 7fefb97f2b80 1 --2- 172.21.15.151:0/1966036288 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c9d8e93eb0 0x55c9d8e94290 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:33.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.369+0000 7fefb97f2b80 1 --2- 172.21.15.151:0/1966036288 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c9d8e947d0 0x55c9d8e8e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:33.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.369+0000 7fefb97f2b80 1 -- 172.21.15.151:0/1966036288 >> 172.21.15.151:0/1966036288 conn(0x55c9d8dbcbe0 msgr2=0x55c9d8e89010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:59:33.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.369+0000 7fefb97f2b80 1 -- 172.21.15.151:0/1966036288 shutdown_connections 2024-05-21T21:59:33.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.369+0000 7fefb97f2b80 1 -- 172.21.15.151:0/1966036288 wait complete. 2024-05-21T21:59:33.372 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.369+0000 7fefb97f2b80 1 -- start start 2024-05-21T21:59:33.372 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.369+0000 7fefb97f2b80 1 Processor -- start 2024-05-21T21:59:33.373 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.370+0000 7fefb97f2b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c9d8e8e880 0x55c9d8e9a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:33.373 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.370+0000 7fefb97f2b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c9d8e93eb0 0x55c9d8e978c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:33.373 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.370+0000 7fefb97f2b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c9d8e947d0 0x55c9d8e97e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:33.373 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.370+0000 7fefb97f2b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c9d8ce2fa0 con 0x55c9d8e93eb0 2024-05-21T21:59:33.373 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.370+0000 7fefb97f2b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55c9d8d7d760 con 0x55c9d8e947d0 2024-05-21T21:59:33.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.370+0000 7fefb97f2b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55c9d8e9b000 con 0x55c9d8e8e880 2024-05-21T21:59:33.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.370+0000 7fefb97f2b80 10 client.0 _ll_register_callbacks cb 0x55c9d8d86c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:59:33.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.370+0000 7fefa7400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c9d8e93eb0 0x55c9d8e978c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:33.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.370+0000 7fefa7400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c9d8e93eb0 0x55c9d8e978c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:42434/0 (socket says 172.21.15.151:42434) 2024-05-21T21:59:33.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.370+0000 7fefa7400700 1 -- 172.21.15.151:0/1137540558 learned_addr learned my addr 172.21.15.151:0/1137540558 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:59:33.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.370+0000 7feface00700 1 --2- 172.21.15.151:0/1137540558 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c9d8e947d0 0x55c9d8e97e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:33.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.370+0000 7fefa7e00700 1 --2- 172.21.15.151:0/1137540558 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c9d8e8e880 0x55c9d8e9a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:33.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.371+0000 7fefa7400700 1 -- 172.21.15.151:0/1137540558 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c9d8e8e880 msgr2=0x55c9d8e9a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:59:33.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.371+0000 7fefa7400700 1 --2- 172.21.15.151:0/1137540558 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c9d8e8e880 0x55c9d8e9a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:33.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.371+0000 7fefa7400700 1 -- 172.21.15.151:0/1137540558 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c9d8e947d0 msgr2=0x55c9d8e97e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:59:33.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.371+0000 7fefa7400700 1 --2- 172.21.15.151:0/1137540558 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c9d8e947d0 0x55c9d8e97e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:33.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.371+0000 7fefa7400700 1 -- 172.21.15.151:0/1137540558 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9d8d2c8d0 con 0x55c9d8e93eb0 2024-05-21T21:59:33.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.371+0000 7feface00700 1 --2- 172.21.15.151:0/1137540558 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c9d8e947d0 0x55c9d8e97e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:59:33.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.371+0000 7fefa7400700 1 --2- 172.21.15.151:0/1137540558 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c9d8e93eb0 0x55c9d8e978c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fef94012d20 tx=0x7fef94012d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=626fd6022ed88fcb server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:59:33.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.371+0000 7fef92000700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-21T21:59:33.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.371+0000 7fef92000700 1 -- 172.21.15.151:0/1137540558 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef9400f830 con 0x55c9d8e93eb0 2024-05-21T21:59:33.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.372+0000 7fefb97f2b80 1 -- 172.21.15.151:0/1137540558 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef980b3490 con 0x55c9d8e93eb0 2024-05-21T21:59:33.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.372+0000 7fef92000700 1 -- 172.21.15.151:0/1137540558 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fef9400f9d0 con 0x55c9d8e93eb0 2024-05-21T21:59:33.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.372+0000 7fef92000700 1 -- 172.21.15.151:0/1137540558 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fef9400fbd0 con 0x55c9d8e93eb0 2024-05-21T21:59:33.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.372+0000 7fefb97f2b80 10 client.15960 fetch_fsmap learned FSMap version 29 2024-05-21T21:59:33.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.372+0000 7fefb97f2b80 1 -- 172.21.15.151:0/1137540558 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c9d8dad060 con 0x55c9d8e93eb0 2024-05-21T21:59:33.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.372+0000 7fef92000700 1 -- 172.21.15.151:0/1137540558 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef980b3490 con 0x55c9d8e93eb0 2024-05-21T21:59:33.377 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.372+0000 7fef92000700 1 -- 172.21.15.151:0/1137540558 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fef9401c420 con 0x55c9d8e93eb0 2024-05-21T21:59:33.377 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.373+0000 7fefb97f2b80 10 client.15960 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T21:59:33.377 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.373+0000 7fefb97f2b80 10 client.15960 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T21:59:33.377 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.373+0000 7fefb97f2b80 20 client.15960 populate_metadata read hostname 'smithi151' 2024-05-21T21:59:33.377 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.373+0000 7fefb97f2b80 1 --2- 172.21.15.151:0/1137540558 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55c9d8ea7a70 0x55c9d8ea9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:33.377 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.373+0000 7fefb97f2b80 4 client.15960 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T21:59:33.377 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.373+0000 7fefb97f2b80 1 -- 172.21.15.151:0/1137540558 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c9d8deed90 con 0x55c9d8ea7a70 2024-05-21T21:59:33.377 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.373+0000 7fefa7e00700 1 --2- 172.21.15.151:0/1137540558 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55c9d8ea7a70 0x55c9d8ea9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:33.377 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.374+0000 7fefa7e00700 1 --2- 172.21.15.151:0/1137540558 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55c9d8ea7a70 0x55c9d8ea9ec0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a0e667b069b3efe server_cookie=bbd1e46ab29195c7 in_seq=0 out_seq=0 2024-05-21T21:59:33.377 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.374+0000 7fef92000700 10 client.15960 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T21:59:33.377 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.374+0000 7fef92000700 1 -- 172.21.15.151:0/1137540558 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55c9d8deed90 con 0x55c9d8ea7a70 2024-05-21T21:59:33.377 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.374+0000 7fef92000700 10 client.15960 handle_command_reply: tid=0 2024-05-21T21:59:33.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.374+0000 7fefb97f2b80 1 client.15960 shutdown 2024-05-21T21:59:33.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.375+0000 7fefb97f2b80 1 -- 172.21.15.151:0/1137540558 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c9d8e93eb0 msgr2=0x55c9d8e978c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:59:33.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.375+0000 7fefb97f2b80 1 --2- 172.21.15.151:0/1137540558 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c9d8e93eb0 0x55c9d8e978c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fef94012d20 tx=0x7fef94012d50 comp rx=0 tx=0).stop 2024-05-21T21:59:33.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.375+0000 7fefb97f2b80 1 -- 172.21.15.151:0/1137540558 shutdown_connections 2024-05-21T21:59:33.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.375+0000 7fefb97f2b80 1 --2- 172.21.15.151:0/1137540558 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55c9d8ea7a70 0x55c9d8ea9ec0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:33.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.375+0000 7fefb97f2b80 1 --2- 172.21.15.151:0/1137540558 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c9d8e947d0 0x55c9d8e97e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:33.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.375+0000 7fefb97f2b80 1 --2- 172.21.15.151:0/1137540558 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c9d8e8e880 0x55c9d8e9a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:33.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.375+0000 7fefb97f2b80 1 --2- 172.21.15.151:0/1137540558 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c9d8e93eb0 0x55c9d8e978c0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:33.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.375+0000 7fefb97f2b80 1 -- 172.21.15.151:0/1137540558 >> 172.21.15.151:0/1137540558 conn(0x55c9d8dbcbe0 msgr2=0x55c9d8e8d250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:59:33.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.375+0000 7fef92000700 0 client.15960 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T21:59:33.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.375+0000 7fefb97f2b80 1 -- 172.21.15.151:0/1137540558 shutdown_connections 2024-05-21T21:59:33.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.375+0000 7fefb97f2b80 1 -- 172.21.15.151:0/1137540558 wait complete. 2024-05-21T21:59:33.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.375+0000 7fefb97f2b80 20 client.15960 trim_cache size 0 max 16384 2024-05-21T21:59:33.401 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.400+0000 7fefb97f2b80 1 Processor -- start 2024-05-21T21:59:33.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.401+0000 7fefb97f2b80 1 -- start start 2024-05-21T21:59:33.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.401+0000 7fefb97f2b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c9d8e8e880 0x55c9d8e8e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:33.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.401+0000 7fefb97f2b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c9d8e93eb0 0x55c9d8e97ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:33.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.401+0000 7fefb97f2b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c9d8e947d0 0x55c9d8e890f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:33.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.401+0000 7fefb97f2b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fefa81aff00 con 0x55c9d8e8e880 2024-05-21T21:59:33.403 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:59:33 smithi151 ceph-mon[31623]: pgmap v121: 97 pgs: 97 active+clean; 284 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 MiB/s wr, 1.01k op/s 2024-05-21T21:59:33.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.401+0000 7fefb97f2b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fefa81b1a40 con 0x55c9d8e93eb0 2024-05-21T21:59:33.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.401+0000 7fefb97f2b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fefa81b05d0 con 0x55c9d8e947d0 2024-05-21T21:59:33.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.402+0000 7fefa7400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c9d8e93eb0 0x55c9d8e97ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:33.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.402+0000 7fefa7400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c9d8e93eb0 0x55c9d8e97ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:60892/0 (socket says 172.21.15.151:60892) 2024-05-21T21:59:33.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.402+0000 7fefa7400700 1 -- 172.21.15.151:0/3228587152 learned_addr learned my addr 172.21.15.151:0/3228587152 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:59:33.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.402+0000 7feface00700 1 --2- 172.21.15.151:0/3228587152 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c9d8e947d0 0x55c9d8e890f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:33.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.402+0000 7fefa7400700 1 -- 172.21.15.151:0/3228587152 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c9d8e947d0 msgr2=0x55c9d8e890f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:59:33.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.402+0000 7fefa7400700 1 --2- 172.21.15.151:0/3228587152 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c9d8e947d0 0x55c9d8e890f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:33.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.402+0000 7fefa7400700 1 -- 172.21.15.151:0/3228587152 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c9d8e8e880 msgr2=0x55c9d8e8e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:59:33.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.402+0000 7feface00700 1 --2- 172.21.15.151:0/3228587152 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c9d8e947d0 0x55c9d8e890f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T21:59:33.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.402+0000 7fefa7400700 1 --2- 172.21.15.151:0/3228587152 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c9d8e8e880 0x55c9d8e8e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:33.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.402+0000 7fefa7400700 1 -- 172.21.15.151:0/3228587152 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9d8ea5490 con 0x55c9d8e93eb0 2024-05-21T21:59:33.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.403+0000 7fefa7400700 1 --2- 172.21.15.151:0/3228587152 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c9d8e93eb0 0x55c9d8e97ef0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fef9401c8e0 tx=0x7fef940123a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=915648822a0ddc33 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:59:33.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.403+0000 7fef9cc00700 1 -- 172.21.15.151:0/3228587152 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef9400fe30 con 0x55c9d8e93eb0 2024-05-21T21:59:33.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.403+0000 7fef9cc00700 1 -- 172.21.15.151:0/3228587152 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef94005520 con 0x55c9d8e93eb0 2024-05-21T21:59:33.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.403+0000 7fef9cc00700 1 -- 172.21.15.151:0/3228587152 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef940046d0 con 0x55c9d8e93eb0 2024-05-21T21:59:33.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.404+0000 7fefb97f2b80 1 -- 172.21.15.151:0/3228587152 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c9d8e93eb0 msgr2=0x55c9d8e97ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:59:33.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.404+0000 7fefb97f2b80 1 --2- 172.21.15.151:0/3228587152 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c9d8e93eb0 0x55c9d8e97ef0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fef9401c8e0 tx=0x7fef940123a0 comp rx=0 tx=0).stop 2024-05-21T21:59:33.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.404+0000 7fefb97f2b80 1 -- 172.21.15.151:0/3228587152 shutdown_connections 2024-05-21T21:59:33.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.404+0000 7fefb97f2b80 1 --2- 172.21.15.151:0/3228587152 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c9d8e947d0 0x55c9d8e890f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:33.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.404+0000 7fefb97f2b80 1 --2- 172.21.15.151:0/3228587152 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c9d8e8e880 0x55c9d8e8e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:33.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.404+0000 7fefb97f2b80 1 --2- 172.21.15.151:0/3228587152 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c9d8e93eb0 0x55c9d8e97ef0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:33.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.404+0000 7fefb97f2b80 1 -- 172.21.15.151:0/3228587152 >> 172.21.15.151:0/3228587152 conn(0x55c9d8dbcbe0 msgr2=0x55c9d8e889e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:59:33.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.404+0000 7fefb97f2b80 1 -- 172.21.15.151:0/3228587152 shutdown_connections 2024-05-21T21:59:33.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.405+0000 7fefb97f2b80 1 -- 172.21.15.151:0/3228587152 wait complete. 2024-05-21T21:59:33.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.405+0000 7fefb97f2b80 1 -- start start 2024-05-21T21:59:33.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.405+0000 7fefb97f2b80 1 Processor -- start 2024-05-21T21:59:33.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.406+0000 7fefb97f2b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c9d8e8e880 0x55c9d8ea0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:33.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.406+0000 7fefb97f2b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c9d8e93eb0 0x55c9d8eaf100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:33.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.406+0000 7fefb97f2b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c9d8e947d0 0x55c9d8eaf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:33.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.406+0000 7fefb97f2b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fefa81b1370 con 0x55c9d8e947d0 2024-05-21T21:59:33.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.406+0000 7fefb97f2b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fefa81b0ca0 con 0x55c9d8e8e880 2024-05-21T21:59:33.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.406+0000 7fefb97f2b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fefa81adea0 con 0x55c9d8e93eb0 2024-05-21T21:59:33.410 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.406+0000 7fefb97f2b80 10 client.0 _ll_register_callbacks cb 0x55c9d8d86c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:59:33.410 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.406+0000 7feface00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c9d8e947d0 0x55c9d8eaf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:33.410 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.406+0000 7fefa7e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c9d8e8e880 0x55c9d8ea0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:33.410 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.406+0000 7fefa7e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c9d8e8e880 0x55c9d8ea0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:60894/0 (socket says 172.21.15.151:60894) 2024-05-21T21:59:33.410 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.406+0000 7fefa7e00700 1 -- 172.21.15.151:0/1395711470 learned_addr learned my addr 172.21.15.151:0/1395711470 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:59:33.410 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.406+0000 7feface00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c9d8e947d0 0x55c9d8eaf640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:42464/0 (socket says 172.21.15.151:42464) 2024-05-21T21:59:33.410 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.406+0000 7fefa7400700 1 --2- 172.21.15.151:0/1395711470 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c9d8e93eb0 0x55c9d8eaf100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:33.410 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.407+0000 7fefa7e00700 1 -- 172.21.15.151:0/1395711470 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c9d8e93eb0 msgr2=0x55c9d8eaf100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:59:33.410 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.407+0000 7fefa7e00700 1 --2- 172.21.15.151:0/1395711470 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c9d8e93eb0 0x55c9d8eaf100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:33.410 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.407+0000 7fefa7e00700 1 -- 172.21.15.151:0/1395711470 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c9d8e947d0 msgr2=0x55c9d8eaf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:59:33.410 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.407+0000 7fefa7e00700 1 --2- 172.21.15.151:0/1395711470 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c9d8e947d0 0x55c9d8eaf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:33.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.407+0000 7fefa7e00700 1 -- 172.21.15.151:0/1395711470 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9d8eafe60 con 0x55c9d8e8e880 2024-05-21T21:59:33.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.407+0000 7fefa7e00700 1 --2- 172.21.15.151:0/1395711470 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c9d8e8e880 0x55c9d8ea0820 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fef980aec30 tx=0x7fef980a66b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ea30fd94c3f4273 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:59:33.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.407+0000 7fef92000700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-21T21:59:33.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.407+0000 7fef92000700 1 -- 172.21.15.151:0/1395711470 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef980b3490 con 0x55c9d8e8e880 2024-05-21T21:59:33.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.407+0000 7fef92000700 1 -- 172.21.15.151:0/1395711470 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fef98090e00 con 0x55c9d8e8e880 2024-05-21T21:59:33.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.407+0000 7fefb97f2b80 1 -- 172.21.15.151:0/1395711470 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef94005520 con 0x55c9d8e8e880 2024-05-21T21:59:33.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.407+0000 7fef92000700 1 -- 172.21.15.151:0/1395711470 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fef980920b0 con 0x55c9d8e8e880 2024-05-21T21:59:33.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.407+0000 7fefb97f2b80 10 client.26087 fetch_fsmap learned FSMap version 29 2024-05-21T21:59:33.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.407+0000 7fefb97f2b80 1 -- 172.21.15.151:0/1395711470 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c9d8f12690 con 0x55c9d8e8e880 2024-05-21T21:59:33.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.408+0000 7fef92000700 1 -- 172.21.15.151:0/1395711470 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef94005520 con 0x55c9d8e8e880 2024-05-21T21:59:33.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.408+0000 7fef92000700 1 -- 172.21.15.151:0/1395711470 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fef980aa920 con 0x55c9d8e8e880 2024-05-21T21:59:33.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.408+0000 7fefb97f2b80 10 client.26087 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T21:59:33.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.408+0000 7fefb97f2b80 10 client.26087 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T21:59:33.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.408+0000 7fefb97f2b80 20 client.26087 populate_metadata read hostname 'smithi151' 2024-05-21T21:59:33.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.408+0000 7fefb97f2b80 1 --2- 172.21.15.151:0/1395711470 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55c9d8f12b50 0x55c9d8f14f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:33.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.408+0000 7fefb97f2b80 4 client.26087 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-21T21:59:33.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.408+0000 7fefb97f2b80 1 -- 172.21.15.151:0/1395711470 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c9d8de8570 con 0x55c9d8f12b50 2024-05-21T21:59:33.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.408+0000 7fefa7400700 1 --2- 172.21.15.151:0/1395711470 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55c9d8f12b50 0x55c9d8f14f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:33.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.409+0000 7fefa7400700 1 --2- 172.21.15.151:0/1395711470 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55c9d8f12b50 0x55c9d8f14f40 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b21dff99567a32c4 server_cookie=b3a9251afce45048 in_seq=0 out_seq=0 2024-05-21T21:59:33.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.409+0000 7fef92000700 10 client.26087 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T21:59:33.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.409+0000 7fef92000700 1 -- 172.21.15.151:0/1395711470 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c9d8de8570 con 0x55c9d8f12b50 2024-05-21T21:59:33.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.409+0000 7fef92000700 10 client.26087 handle_command_reply: tid=0 2024-05-21T21:59:33.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.409+0000 7fefb97f2b80 1 client.26087 shutdown 2024-05-21T21:59:33.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.409+0000 7fefb97f2b80 1 -- 172.21.15.151:0/1395711470 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c9d8e8e880 msgr2=0x55c9d8ea0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:59:33.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.409+0000 7fefb97f2b80 1 --2- 172.21.15.151:0/1395711470 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c9d8e8e880 0x55c9d8ea0820 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fef980aec30 tx=0x7fef980a66b0 comp rx=0 tx=0).stop 2024-05-21T21:59:33.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.409+0000 7fefb97f2b80 1 -- 172.21.15.151:0/1395711470 shutdown_connections 2024-05-21T21:59:33.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.409+0000 7fefb97f2b80 1 --2- 172.21.15.151:0/1395711470 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55c9d8f12b50 0x55c9d8f14f40 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:33.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.409+0000 7fefb97f2b80 1 --2- 172.21.15.151:0/1395711470 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c9d8e8e880 0x55c9d8ea0820 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:33.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.409+0000 7fefb97f2b80 1 --2- 172.21.15.151:0/1395711470 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c9d8e947d0 0x55c9d8eaf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:33.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.409+0000 7fefb97f2b80 1 --2- 172.21.15.151:0/1395711470 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c9d8e93eb0 0x55c9d8eaf100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:33.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.409+0000 7fefb97f2b80 1 -- 172.21.15.151:0/1395711470 >> 172.21.15.151:0/1395711470 conn(0x55c9d8dbcbe0 msgr2=0x55c9d8e961b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:59:33.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.409+0000 7fef92000700 0 client.26087 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T21:59:33.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.410+0000 7fefb97f2b80 1 -- 172.21.15.151:0/1395711470 shutdown_connections 2024-05-21T21:59:33.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.410+0000 7fefb97f2b80 1 -- 172.21.15.151:0/1395711470 wait complete. 2024-05-21T21:59:33.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.410+0000 7fefb97f2b80 20 client.26087 trim_cache size 0 max 16384 2024-05-21T21:59:33.414 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-05-21T21:59:33.414 INFO:teuthology.orchestra.run.smithi151.stdout: "status": "no active scrubs running", 2024-05-21T21:59:33.414 INFO:teuthology.orchestra.run.smithi151.stdout: "scrubs": {} 2024-05-21T21:59:33.414 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-05-21T21:59:33.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.411+0000 7fefaee00700 1 -- 172.21.15.151:0/4032238897 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fef8c0b29f0 msgr2=0x7fef8c0b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:59:33.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.411+0000 7fefaee00700 1 --2- 172.21.15.151:0/4032238897 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fef8c0b29f0 0x7fef8c0b4eb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fefa0009ec0 tx=0x7fefa000b040 comp rx=0 tx=0).stop 2024-05-21T21:59:33.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.411+0000 7fefaee00700 1 -- 172.21.15.151:0/4032238897 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fefa8105760 msgr2=0x7fefa819be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:59:33.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.411+0000 7fefaee00700 1 --2- 172.21.15.151:0/4032238897 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fefa8105760 0x7fefa819be40 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fef98000c00 tx=0x7fef980057c0 comp rx=0 tx=0).stop 2024-05-21T21:59:33.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.411+0000 7fefaee00700 1 -- 172.21.15.151:0/4032238897 shutdown_connections 2024-05-21T21:59:33.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.411+0000 7fefaee00700 1 --2- 172.21.15.151:0/4032238897 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fef8c0b29f0 0x7fef8c0b4eb0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:33.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.411+0000 7fefaee00700 1 --2- 172.21.15.151:0/4032238897 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fefa8105760 0x7fefa819be40 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:33.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.411+0000 7fefaee00700 1 --2- 172.21.15.151:0/4032238897 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fefa810b760 0x7fefa81a0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:33.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.411+0000 7fefaee00700 1 --2- 172.21.15.151:0/4032238897 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fefa8106120 0x7fefa819c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:33.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.411+0000 7fefaee00700 1 -- 172.21.15.151:0/4032238897 >> 172.21.15.151:0/4032238897 conn(0x7fefa8101250 msgr2=0x7fefa8101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:59:33.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.411+0000 7fefaee00700 1 -- 172.21.15.151:0/4032238897 shutdown_connections 2024-05-21T21:59:33.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:33.412+0000 7fefaee00700 1 -- 172.21.15.151:0/4032238897 wait complete. 2024-05-21T21:59:33.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:59:33 smithi184 ceph-mon[36942]: pgmap v121: 97 pgs: 97 active+clean; 284 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 MiB/s wr, 1.01k op/s 2024-05-21T21:59:33.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:59:33 smithi164 ceph-mon[36269]: pgmap v121: 97 pgs: 97 active+clean; 284 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 MiB/s wr, 1.01k op/s 2024-05-21T21:59:35.217 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-21T21:59:35.218 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-21T21:59:35.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:59:35 smithi151 ceph-mon[31623]: pgmap v122: 97 pgs: 97 active+clean; 284 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 351 KiB/s wr, 11 op/s 2024-05-21T21:59:35.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:59:35 smithi184 ceph-mon[36942]: pgmap v122: 97 pgs: 97 active+clean; 284 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 351 KiB/s wr, 11 op/s 2024-05-21T21:59:35.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:59:35 smithi164 ceph-mon[36269]: pgmap v122: 97 pgs: 97 active+clean; 284 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 351 KiB/s wr, 11 op/s 2024-05-21T21:59:35.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.798+0000 7fb647e00700 1 -- 172.21.15.151:0/1990095180 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb648103490 msgr2=0x7fb648103910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:59:35.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.798+0000 7fb647e00700 1 --2- 172.21.15.151:0/1990095180 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb648103490 0x7fb648103910 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb6340099c0 tx=0x7fb634009cd0 comp rx=0 tx=0).stop 2024-05-21T21:59:35.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.799+0000 7fb647e00700 1 -- 172.21.15.151:0/1990095180 shutdown_connections 2024-05-21T21:59:35.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.799+0000 7fb647e00700 1 --2- 172.21.15.151:0/1990095180 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb648103e50 0x7fb648110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:35.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.799+0000 7fb647e00700 1 --2- 172.21.15.151:0/1990095180 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb648109540 0x7fb648109920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:35.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.799+0000 7fb647e00700 1 --2- 172.21.15.151:0/1990095180 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb648103490 0x7fb648103910 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:35.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.799+0000 7fb647e00700 1 -- 172.21.15.151:0/1990095180 >> 172.21.15.151:0/1990095180 conn(0x7fb6480fefc0 msgr2=0x7fb6481013e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:59:35.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.799+0000 7fb647e00700 1 -- 172.21.15.151:0/1990095180 shutdown_connections 2024-05-21T21:59:35.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.799+0000 7fb647e00700 1 -- 172.21.15.151:0/1990095180 wait complete. 2024-05-21T21:59:35.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.800+0000 7fb647e00700 1 Processor -- start 2024-05-21T21:59:35.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.800+0000 7fb647e00700 1 -- start start 2024-05-21T21:59:35.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.801+0000 7fb647e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb648103490 0x7fb648075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:35.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.801+0000 7fb647e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb648103e50 0x7fb648075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:35.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.801+0000 7fb647e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb648109540 0x7fb6480762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:35.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.801+0000 7fb647e00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb6481136a0 con 0x7fb648109540 2024-05-21T21:59:35.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.801+0000 7fb647e00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb648113520 con 0x7fb648103e50 2024-05-21T21:59:35.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.801+0000 7fb647e00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb648113820 con 0x7fb648103490 2024-05-21T21:59:35.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.801+0000 7fb646a00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb648103490 0x7fb648075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:35.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.801+0000 7fb646000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb648103e50 0x7fb648075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:35.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.801+0000 7fb646000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb648103e50 0x7fb648075d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:45354/0 (socket says 172.21.15.151:45354) 2024-05-21T21:59:35.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.801+0000 7fb646a00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb648103490 0x7fb648075840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:37256/0 (socket says 172.21.15.151:37256) 2024-05-21T21:59:35.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.801+0000 7fb646000700 1 -- 172.21.15.151:0/2896102489 learned_addr learned my addr 172.21.15.151:0/2896102489 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:59:35.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.801+0000 7fb647400700 1 --2- 172.21.15.151:0/2896102489 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb648109540 0x7fb6480762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:35.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.802+0000 7fb646a00700 1 -- 172.21.15.151:0/2896102489 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb648103e50 msgr2=0x7fb648075d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:59:35.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.802+0000 7fb646a00700 1 --2- 172.21.15.151:0/2896102489 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb648103e50 0x7fb648075d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:35.805 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.802+0000 7fb646a00700 1 -- 172.21.15.151:0/2896102489 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb648109540 msgr2=0x7fb6480762c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:59:35.805 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.802+0000 7fb646a00700 1 --2- 172.21.15.151:0/2896102489 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb648109540 0x7fb6480762c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:35.805 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.802+0000 7fb646a00700 1 -- 172.21.15.151:0/2896102489 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6481a8010 con 0x7fb648103490 2024-05-21T21:59:35.805 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.802+0000 7fb646a00700 1 --2- 172.21.15.151:0/2896102489 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb648103490 0x7fb648075840 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb63800e9f0 tx=0x7fb63800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:59:35.805 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.802+0000 7fb63f400700 1 -- 172.21.15.151:0/2896102489 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb63800c9c0 con 0x7fb648103490 2024-05-21T21:59:35.805 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.802+0000 7fb63f400700 1 -- 172.21.15.151:0/2896102489 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb63800cb60 con 0x7fb648103490 2024-05-21T21:59:35.805 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.802+0000 7fb63f400700 1 -- 172.21.15.151:0/2896102489 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb638018680 con 0x7fb648103490 2024-05-21T21:59:35.805 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.802+0000 7fb647e00700 1 -- 172.21.15.151:0/2896102489 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6481a8300 con 0x7fb648103490 2024-05-21T21:59:35.805 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.803+0000 7fb647e00700 1 -- 172.21.15.151:0/2896102489 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6481a8860 con 0x7fb648103490 2024-05-21T21:59:35.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.808+0000 7fb63f400700 1 -- 172.21.15.151:0/2896102489 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7fb638010430 con 0x7fb648103490 2024-05-21T21:59:35.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.809+0000 7fb658216b80 1 Processor -- start 2024-05-21T21:59:35.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.809+0000 7fb658216b80 1 -- start start 2024-05-21T21:59:35.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.810+0000 7fb658216b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ed1781ad10 0x55ed1781b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:35.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.810+0000 7fb658216b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ed1781b630 0x55ed17815240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:35.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.810+0000 7fb658216b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ed17815860 0x55ed17815cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:35.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.810+0000 7fb658216b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ed1776aeb0 con 0x55ed1781ad10 2024-05-21T21:59:35.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.810+0000 7fb658216b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55ed1775aa50 con 0x55ed1781b630 2024-05-21T21:59:35.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.810+0000 7fb658216b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55ed1775ea90 con 0x55ed17815860 2024-05-21T21:59:35.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.810+0000 7fb647400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ed1781ad10 0x55ed1781b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:35.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.810+0000 7fb647400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ed1781ad10 0x55ed1781b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:51816/0 (socket says 172.21.15.151:51816) 2024-05-21T21:59:35.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.810+0000 7fb647400700 1 -- 172.21.15.151:0/526926110 learned_addr learned my addr 172.21.15.151:0/526926110 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:59:35.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.810+0000 7fb63f400700 1 --2- 172.21.15.151:0/2896102489 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fb62c0b2a70 0x7fb62c0b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:35.813 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.810+0000 7fb646a00700 1 --2- 172.21.15.151:0/526926110 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ed1781b630 0x55ed17815240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:35.813 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.810+0000 7fb646000700 1 --2- 172.21.15.151:0/526926110 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ed17815860 0x55ed17815cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:35.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.811+0000 7fb63f400700 1 -- 172.21.15.151:0/2896102489 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fb6380e26d0 con 0x7fb648103490 2024-05-21T21:59:35.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.811+0000 7fb647400700 1 -- 172.21.15.151:0/526926110 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ed17815860 msgr2=0x55ed17815cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:59:35.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.811+0000 7fb647400700 1 --2- 172.21.15.151:0/526926110 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ed17815860 0x55ed17815cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:35.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.811+0000 7fb647400700 1 -- 172.21.15.151:0/526926110 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ed1781b630 msgr2=0x55ed17815240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:59:35.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.811+0000 7fb647400700 1 --2- 172.21.15.151:0/526926110 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ed1781b630 0x55ed17815240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:35.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.811+0000 7fb647400700 1 -- 172.21.15.151:0/526926110 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed17723490 con 0x55ed1781ad10 2024-05-21T21:59:35.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.811+0000 7fb647400700 1 --2- 172.21.15.151:0/2896102489 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fb62c0b2a70 0x7fb62c0b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:35.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.812+0000 7fb647400700 1 --2- 172.21.15.151:0/526926110 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ed1781ad10 0x55ed1781b0f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb64000f6b0 tx=0x7fb64000f9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c709c23f073bbfdd server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:59:35.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.812+0000 7fb633e00700 1 -- 172.21.15.151:0/526926110 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb64001c040 con 0x55ed1781ad10 2024-05-21T21:59:35.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.812+0000 7fb647400700 1 --2- 172.21.15.151:0/2896102489 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fb62c0b2a70 0x7fb62c0b4f30 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb640022000 tx=0x7fb64001f000 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:59:35.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.812+0000 7fb633e00700 1 -- 172.21.15.151:0/526926110 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb640018b30 con 0x55ed1781ad10 2024-05-21T21:59:35.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.812+0000 7fb633e00700 1 -- 172.21.15.151:0/526926110 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb640023910 con 0x55ed1781ad10 2024-05-21T21:59:35.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.813+0000 7fb658216b80 1 -- 172.21.15.151:0/526926110 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ed1781ad10 msgr2=0x55ed1781b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:59:35.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.813+0000 7fb658216b80 1 --2- 172.21.15.151:0/526926110 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ed1781ad10 0x55ed1781b0f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb64000f6b0 tx=0x7fb64000f9c0 comp rx=0 tx=0).stop 2024-05-21T21:59:35.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.813+0000 7fb658216b80 1 -- 172.21.15.151:0/526926110 shutdown_connections 2024-05-21T21:59:35.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.813+0000 7fb658216b80 1 --2- 172.21.15.151:0/526926110 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ed17815860 0x55ed17815cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:35.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.813+0000 7fb658216b80 1 --2- 172.21.15.151:0/526926110 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ed1781ad10 0x55ed1781b0f0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:35.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.814+0000 7fb658216b80 1 --2- 172.21.15.151:0/526926110 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ed1781b630 0x55ed17815240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:35.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.814+0000 7fb658216b80 1 -- 172.21.15.151:0/526926110 >> 172.21.15.151:0/526926110 conn(0x55ed1773ac50 msgr2=0x55ed178075a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:59:35.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.814+0000 7fb658216b80 1 -- 172.21.15.151:0/526926110 shutdown_connections 2024-05-21T21:59:35.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.814+0000 7fb658216b80 1 -- 172.21.15.151:0/526926110 wait complete. 2024-05-21T21:59:35.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.814+0000 7fb658216b80 1 -- start start 2024-05-21T21:59:35.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.815+0000 7fb658216b80 1 Processor -- start 2024-05-21T21:59:35.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.815+0000 7fb658216b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ed17815860 0x55ed17821400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:35.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.815+0000 7fb658216b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ed1781ad10 0x55ed1781e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:35.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.815+0000 7fb658216b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ed1781b630 0x55ed1781ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:35.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.815+0000 7fb658216b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ed17661010 con 0x55ed17815860 2024-05-21T21:59:35.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.815+0000 7fb658216b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55ed176fb7d0 con 0x55ed1781b630 2024-05-21T21:59:35.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.815+0000 7fb658216b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55ed17822130 con 0x55ed1781ad10 2024-05-21T21:59:35.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.815+0000 7fb658216b80 10 client.0 _ll_register_callbacks cb 0x55ed17704ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:59:35.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.815+0000 7fb646000700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ed1781ad10 0x55ed1781e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:35.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.815+0000 7fb646000700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ed1781ad10 0x55ed1781e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:37264/0 (socket says 172.21.15.151:37264) 2024-05-21T21:59:35.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.815+0000 7fb646000700 1 -- 172.21.15.151:0/2928996322 learned_addr learned my addr 172.21.15.151:0/2928996322 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:59:35.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.816+0000 7fb646a00700 1 --2- 172.21.15.151:0/2928996322 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ed17815860 0x55ed17821400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:35.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.816+0000 7fb647400700 1 --2- 172.21.15.151:0/2928996322 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ed1781b630 0x55ed1781ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:35.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.816+0000 7fb646000700 1 -- 172.21.15.151:0/2928996322 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ed1781b630 msgr2=0x55ed1781ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:59:35.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.816+0000 7fb646000700 1 --2- 172.21.15.151:0/2928996322 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ed1781b630 0x55ed1781ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:35.819 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.816+0000 7fb646000700 1 -- 172.21.15.151:0/2928996322 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ed17815860 msgr2=0x55ed17821400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:59:35.819 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.816+0000 7fb646000700 1 --2- 172.21.15.151:0/2928996322 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ed17815860 0x55ed17821400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:35.819 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.816+0000 7fb646000700 1 -- 172.21.15.151:0/2928996322 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed176aa940 con 0x55ed1781ad10 2024-05-21T21:59:35.819 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.816+0000 7fb646000700 1 --2- 172.21.15.151:0/2928996322 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ed1781ad10 0x55ed1781e7a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb63400b580 tx=0x7fb6340057a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b67c5fe065199c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:59:35.819 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.816+0000 7fb630c00700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-21T21:59:35.820 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.817+0000 7fb630c00700 1 -- 172.21.15.151:0/2928996322 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6340039c0 con 0x55ed1781ad10 2024-05-21T21:59:35.820 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.817+0000 7fb630c00700 1 -- 172.21.15.151:0/2928996322 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fb634003b60 con 0x55ed1781ad10 2024-05-21T21:59:35.820 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.817+0000 7fb658216b80 1 -- 172.21.15.151:0/2928996322 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb640018b30 con 0x55ed1781ad10 2024-05-21T21:59:35.820 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.817+0000 7fb630c00700 1 -- 172.21.15.151:0/2928996322 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb634032050 con 0x55ed1781ad10 2024-05-21T21:59:35.820 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.817+0000 7fb658216b80 10 client.26113 fetch_fsmap learned FSMap version 29 2024-05-21T21:59:35.820 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.817+0000 7fb658216b80 1 -- 172.21.15.151:0/2928996322 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ed1772b0d0 con 0x55ed1781ad10 2024-05-21T21:59:35.820 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.818+0000 7fb630c00700 1 -- 172.21.15.151:0/2928996322 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb640018b30 con 0x55ed1781ad10 2024-05-21T21:59:35.820 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.818+0000 7fb630c00700 1 -- 172.21.15.151:0/2928996322 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fb63400f5d0 con 0x55ed1781ad10 2024-05-21T21:59:35.820 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.818+0000 7fb658216b80 10 client.26113 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T21:59:35.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.818+0000 7fb658216b80 10 client.26113 resolve_mds: validated gid 24530 aka daemon mds.b 2024-05-21T21:59:35.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.818+0000 7fb658216b80 20 client.26113 populate_metadata read hostname 'smithi151' 2024-05-21T21:59:35.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.818+0000 7fb658216b80 1 --2- 172.21.15.151:0/2928996322 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55ed1782ed20 0x55ed17831110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:35.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.818+0000 7fb658216b80 4 client.26113 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T21:59:35.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.818+0000 7fb658216b80 1 -- 172.21.15.151:0/2928996322 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed1776ce00 con 0x55ed1782ed20 2024-05-21T21:59:35.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.819+0000 7fb646a00700 1 --2- 172.21.15.151:0/2928996322 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55ed1782ed20 0x55ed17831110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:35.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.819+0000 7fb646a00700 1 --2- 172.21.15.151:0/2928996322 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55ed1782ed20 0x55ed17831110 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4a12ad42f83e566 server_cookie=203af58d7cb2c61d in_seq=0 out_seq=0 2024-05-21T21:59:35.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.819+0000 7fb630c00700 10 client.26113 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T21:59:35.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.820+0000 7fb630c00700 1 -- 172.21.15.151:0/2928996322 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55ed1776ce00 con 0x55ed1782ed20 2024-05-21T21:59:35.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.820+0000 7fb630c00700 10 client.26113 handle_command_reply: tid=0 2024-05-21T21:59:35.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.820+0000 7fb658216b80 1 client.26113 shutdown 2024-05-21T21:59:35.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.820+0000 7fb658216b80 1 -- 172.21.15.151:0/2928996322 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ed1781ad10 msgr2=0x55ed1781e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:59:35.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.820+0000 7fb658216b80 1 --2- 172.21.15.151:0/2928996322 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ed1781ad10 0x55ed1781e7a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb63400b580 tx=0x7fb6340057a0 comp rx=0 tx=0).stop 2024-05-21T21:59:35.823 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.821+0000 7fb658216b80 1 -- 172.21.15.151:0/2928996322 shutdown_connections 2024-05-21T21:59:35.823 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.821+0000 7fb658216b80 1 --2- 172.21.15.151:0/2928996322 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55ed1782ed20 0x55ed17831110 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:35.823 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.821+0000 7fb658216b80 1 --2- 172.21.15.151:0/2928996322 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ed1781b630 0x55ed1781ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:35.823 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.821+0000 7fb658216b80 1 --2- 172.21.15.151:0/2928996322 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ed17815860 0x55ed17821400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:35.823 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.821+0000 7fb658216b80 1 --2- 172.21.15.151:0/2928996322 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ed1781ad10 0x55ed1781e7a0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:35.823 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.821+0000 7fb630c00700 0 client.26113 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T21:59:35.823 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.821+0000 7fb658216b80 1 -- 172.21.15.151:0/2928996322 >> 172.21.15.151:0/2928996322 conn(0x55ed1773ac50 msgr2=0x55ed1780ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:59:35.823 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.821+0000 7fb658216b80 1 -- 172.21.15.151:0/2928996322 shutdown_connections 2024-05-21T21:59:35.823 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.821+0000 7fb658216b80 1 -- 172.21.15.151:0/2928996322 wait complete. 2024-05-21T21:59:35.823 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.821+0000 7fb658216b80 20 client.26113 trim_cache size 0 max 16384 2024-05-21T21:59:35.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.847+0000 7fb658216b80 1 Processor -- start 2024-05-21T21:59:35.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.847+0000 7fb658216b80 1 -- start start 2024-05-21T21:59:35.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.848+0000 7fb658216b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ed17815860 0x55ed17801560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:35.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.848+0000 7fb658216b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ed1781ad10 0x55ed178369e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:35.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.848+0000 7fb658216b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ed1781b630 0x55ed17836f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:35.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.848+0000 7fb658216b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb6481aba40 con 0x55ed17815860 2024-05-21T21:59:35.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.848+0000 7fb658216b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb6481ad580 con 0x55ed1781b630 2024-05-21T21:59:35.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.848+0000 7fb658216b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb6481ac110 con 0x55ed1781ad10 2024-05-21T21:59:35.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.848+0000 7fb646a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ed17815860 0x55ed17801560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:35.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.848+0000 7fb646a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ed17815860 0x55ed17801560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:51838/0 (socket says 172.21.15.151:51838) 2024-05-21T21:59:35.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.848+0000 7fb646a00700 1 -- 172.21.15.151:0/3858620032 learned_addr learned my addr 172.21.15.151:0/3858620032 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:59:35.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.848+0000 7fb646000700 1 --2- 172.21.15.151:0/3858620032 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ed1781ad10 0x55ed178369e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:35.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.848+0000 7fb647400700 1 --2- 172.21.15.151:0/3858620032 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ed1781b630 0x55ed17836f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:35.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.849+0000 7fb646a00700 1 -- 172.21.15.151:0/3858620032 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ed1781ad10 msgr2=0x55ed178369e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:59:35.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.849+0000 7fb646a00700 1 --2- 172.21.15.151:0/3858620032 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ed1781ad10 0x55ed178369e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:35.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.849+0000 7fb646a00700 1 -- 172.21.15.151:0/3858620032 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ed1781b630 msgr2=0x55ed17836f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:59:35.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.849+0000 7fb646a00700 1 --2- 172.21.15.151:0/3858620032 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ed1781b630 0x55ed17836f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:35.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.849+0000 7fb646a00700 1 -- 172.21.15.151:0/3858620032 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed17821040 con 0x55ed17815860 2024-05-21T21:59:35.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.849+0000 7fb647400700 1 --2- 172.21.15.151:0/3858620032 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ed1781b630 0x55ed17836f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:59:35.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.849+0000 7fb646a00700 1 --2- 172.21.15.151:0/3858620032 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ed17815860 0x55ed17801560 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb63800e9c0 tx=0x7fb6380e4b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3305c3cd32dd37e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:59:35.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.849+0000 7fb633400700 1 -- 172.21.15.151:0/3858620032 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6380aa060 con 0x55ed17815860 2024-05-21T21:59:35.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.850+0000 7fb633400700 1 -- 172.21.15.151:0/3858620032 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6380a9960 con 0x55ed17815860 2024-05-21T21:59:35.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.850+0000 7fb633400700 1 -- 172.21.15.151:0/3858620032 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6380a9bc0 con 0x55ed17815860 2024-05-21T21:59:35.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.850+0000 7fb658216b80 1 -- 172.21.15.151:0/3858620032 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ed17815860 msgr2=0x55ed17801560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:59:35.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.850+0000 7fb658216b80 1 --2- 172.21.15.151:0/3858620032 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ed17815860 0x55ed17801560 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb63800e9c0 tx=0x7fb6380e4b90 comp rx=0 tx=0).stop 2024-05-21T21:59:35.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.850+0000 7fb658216b80 1 -- 172.21.15.151:0/3858620032 shutdown_connections 2024-05-21T21:59:35.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.850+0000 7fb658216b80 1 --2- 172.21.15.151:0/3858620032 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ed1781b630 0x55ed17836f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:35.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.850+0000 7fb658216b80 1 --2- 172.21.15.151:0/3858620032 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ed17815860 0x55ed17801560 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:35.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.850+0000 7fb658216b80 1 --2- 172.21.15.151:0/3858620032 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ed1781ad10 0x55ed178369e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:35.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.850+0000 7fb658216b80 1 -- 172.21.15.151:0/3858620032 >> 172.21.15.151:0/3858620032 conn(0x55ed1773ac50 msgr2=0x55ed17800e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:59:35.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.851+0000 7fb658216b80 1 -- 172.21.15.151:0/3858620032 shutdown_connections 2024-05-21T21:59:35.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.851+0000 7fb658216b80 1 -- 172.21.15.151:0/3858620032 wait complete. 2024-05-21T21:59:35.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.851+0000 7fb658216b80 1 -- start start 2024-05-21T21:59:35.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.851+0000 7fb658216b80 1 Processor -- start 2024-05-21T21:59:35.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.852+0000 7fb658216b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ed17815860 0x55ed17824b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:35.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.852+0000 7fb658216b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ed1781ad10 0x55ed178250b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:35.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.852+0000 7fb658216b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ed1781b630 0x55ed17827110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:35.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.852+0000 7fb658216b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb6481aceb0 con 0x55ed1781b630 2024-05-21T21:59:35.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.852+0000 7fb658216b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb6481ac7e0 con 0x55ed1781ad10 2024-05-21T21:59:35.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.852+0000 7fb658216b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb64819d8d0 con 0x55ed17815860 2024-05-21T21:59:35.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.852+0000 7fb658216b80 10 client.0 _ll_register_callbacks cb 0x55ed17704ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:59:35.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.852+0000 7fb647400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ed1781b630 0x55ed17827110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:35.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.852+0000 7fb646a00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ed17815860 0x55ed17824b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:35.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.852+0000 7fb646a00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ed17815860 0x55ed17824b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:37278/0 (socket says 172.21.15.151:37278) 2024-05-21T21:59:35.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.852+0000 7fb646a00700 1 -- 172.21.15.151:0/3189331343 learned_addr learned my addr 172.21.15.151:0/3189331343 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:59:35.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.852+0000 7fb646000700 1 --2- 172.21.15.151:0/3189331343 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ed1781ad10 0x55ed178250b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:35.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.853+0000 7fb646000700 1 -- 172.21.15.151:0/3189331343 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ed17815860 msgr2=0x55ed17824b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:59:35.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.853+0000 7fb646000700 1 --2- 172.21.15.151:0/3189331343 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ed17815860 0x55ed17824b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:35.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.853+0000 7fb646000700 1 -- 172.21.15.151:0/3189331343 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ed1781b630 msgr2=0x55ed17827110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:59:35.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.853+0000 7fb646000700 1 --2- 172.21.15.151:0/3189331343 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ed1781b630 0x55ed17827110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:35.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.853+0000 7fb646000700 1 -- 172.21.15.151:0/3189331343 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed17827780 con 0x55ed1781ad10 2024-05-21T21:59:35.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.853+0000 7fb647400700 1 --2- 172.21.15.151:0/3189331343 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ed1781b630 0x55ed17827110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:59:35.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.853+0000 7fb646000700 1 --2- 172.21.15.151:0/3189331343 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ed1781ad10 0x55ed178250b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb634009990 tx=0x7fb63400fea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e99a33c2ef13811f server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:59:35.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.853+0000 7fb630c00700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-21T21:59:35.858 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.853+0000 7fb630c00700 1 -- 172.21.15.151:0/3189331343 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6340037c0 con 0x55ed1781ad10 2024-05-21T21:59:35.858 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.854+0000 7fb658216b80 1 -- 172.21.15.151:0/3189331343 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6380a9960 con 0x55ed1781ad10 2024-05-21T21:59:35.858 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.854+0000 7fb630c00700 1 -- 172.21.15.151:0/3189331343 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fb634003960 con 0x55ed1781ad10 2024-05-21T21:59:35.858 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.854+0000 7fb630c00700 1 -- 172.21.15.151:0/3189331343 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb634003b80 con 0x55ed1781ad10 2024-05-21T21:59:35.858 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.854+0000 7fb658216b80 10 client.26105 fetch_fsmap learned FSMap version 29 2024-05-21T21:59:35.858 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.854+0000 7fb658216b80 1 -- 172.21.15.151:0/3189331343 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ed1789b410 con 0x55ed1781ad10 2024-05-21T21:59:35.858 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.854+0000 7fb630c00700 1 -- 172.21.15.151:0/3189331343 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6380a9960 con 0x55ed1781ad10 2024-05-21T21:59:35.858 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.854+0000 7fb630c00700 1 -- 172.21.15.151:0/3189331343 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fb63403d060 con 0x55ed1781ad10 2024-05-21T21:59:35.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.854+0000 7fb658216b80 10 client.26105 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T21:59:35.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.854+0000 7fb658216b80 10 client.26105 resolve_mds: validated gid 24530 aka daemon mds.b 2024-05-21T21:59:35.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.854+0000 7fb658216b80 20 client.26105 populate_metadata read hostname 'smithi151' 2024-05-21T21:59:35.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.854+0000 7fb658216b80 1 --2- 172.21.15.151:0/3189331343 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55ed1789ba90 0x55ed1789deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:35.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.854+0000 7fb658216b80 4 client.26105 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-05-21T21:59:35.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.854+0000 7fb658216b80 1 -- 172.21.15.151:0/3189331343 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x55ed177665e0 con 0x55ed1789ba90 2024-05-21T21:59:35.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.855+0000 7fb646a00700 1 --2- 172.21.15.151:0/3189331343 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55ed1789ba90 0x55ed1789deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:35.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.855+0000 7fb646a00700 1 --2- 172.21.15.151:0/3189331343 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55ed1789ba90 0x55ed1789deb0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f3e633c94d25f57 server_cookie=8fd205550b7268a3 in_seq=0 out_seq=0 2024-05-21T21:59:35.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.855+0000 7fb630c00700 10 client.26105 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T21:59:35.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.856+0000 7fb630c00700 1 -- 172.21.15.151:0/3189331343 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+79 (crc 0 0 0) 0x55ed177665e0 con 0x55ed1789ba90 2024-05-21T21:59:35.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.856+0000 7fb630c00700 10 client.26105 handle_command_reply: tid=0 2024-05-21T21:59:35.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.856+0000 7fb658216b80 1 client.26105 shutdown 2024-05-21T21:59:35.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.856+0000 7fb658216b80 1 -- 172.21.15.151:0/3189331343 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ed1781ad10 msgr2=0x55ed178250b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:59:35.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.856+0000 7fb658216b80 1 --2- 172.21.15.151:0/3189331343 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ed1781ad10 0x55ed178250b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb634009990 tx=0x7fb63400fea0 comp rx=0 tx=0).stop 2024-05-21T21:59:35.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.856+0000 7fb658216b80 1 -- 172.21.15.151:0/3189331343 shutdown_connections 2024-05-21T21:59:35.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.856+0000 7fb658216b80 1 --2- 172.21.15.151:0/3189331343 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ed1781b630 0x55ed17827110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:35.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.856+0000 7fb658216b80 1 --2- 172.21.15.151:0/3189331343 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ed17815860 0x55ed17824b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:35.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.856+0000 7fb658216b80 1 --2- 172.21.15.151:0/3189331343 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55ed1789ba90 0x55ed1789deb0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:35.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.856+0000 7fb658216b80 1 --2- 172.21.15.151:0/3189331343 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ed1781ad10 0x55ed178250b0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:35.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.856+0000 7fb630c00700 0 client.26105 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T21:59:35.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.856+0000 7fb658216b80 1 -- 172.21.15.151:0/3189331343 >> 172.21.15.151:0/3189331343 conn(0x55ed1773ac50 msgr2=0x55ed1780ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:59:35.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.856+0000 7fb658216b80 1 -- 172.21.15.151:0/3189331343 shutdown_connections 2024-05-21T21:59:35.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.856+0000 7fb658216b80 1 -- 172.21.15.151:0/3189331343 wait complete. 2024-05-21T21:59:35.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.857+0000 7fb658216b80 20 client.26105 trim_cache size 0 max 16384 2024-05-21T21:59:35.861 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-05-21T21:59:35.861 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 29, 2024-05-21T21:59:35.861 INFO:teuthology.orchestra.run.smithi151.stdout: "leader": 24530, 2024-05-21T21:59:35.861 INFO:teuthology.orchestra.run.smithi151.stdout: "set_version": 8, 2024-05-21T21:59:35.861 INFO:teuthology.orchestra.run.smithi151.stdout: "sets": {} 2024-05-21T21:59:35.861 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-05-21T21:59:35.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.857+0000 7fb647e00700 1 -- 172.21.15.151:0/2896102489 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fb62c0b2a70 msgr2=0x7fb62c0b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:59:35.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.857+0000 7fb647e00700 1 --2- 172.21.15.151:0/2896102489 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fb62c0b2a70 0x7fb62c0b4f30 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb640022000 tx=0x7fb64001f000 comp rx=0 tx=0).stop 2024-05-21T21:59:35.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.857+0000 7fb647e00700 1 -- 172.21.15.151:0/2896102489 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb648103490 msgr2=0x7fb648075840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:59:35.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.857+0000 7fb647e00700 1 --2- 172.21.15.151:0/2896102489 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb648103490 0x7fb648075840 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb63800e9f0 tx=0x7fb63800ed00 comp rx=0 tx=0).stop 2024-05-21T21:59:35.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.858+0000 7fb647e00700 1 -- 172.21.15.151:0/2896102489 shutdown_connections 2024-05-21T21:59:35.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.858+0000 7fb647e00700 1 --2- 172.21.15.151:0/2896102489 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fb62c0b2a70 0x7fb62c0b4f30 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:35.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.858+0000 7fb647e00700 1 --2- 172.21.15.151:0/2896102489 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb648109540 0x7fb6480762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:35.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.858+0000 7fb647e00700 1 --2- 172.21.15.151:0/2896102489 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb648103490 0x7fb648075840 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:35.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.858+0000 7fb647e00700 1 --2- 172.21.15.151:0/2896102489 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb648103e50 0x7fb648075d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:35.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.858+0000 7fb647e00700 1 -- 172.21.15.151:0/2896102489 >> 172.21.15.151:0/2896102489 conn(0x7fb6480fefc0 msgr2=0x7fb648100670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:59:35.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.858+0000 7fb647e00700 1 -- 172.21.15.151:0/2896102489 shutdown_connections 2024-05-21T21:59:35.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:35.858+0000 7fb647e00700 1 -- 172.21.15.151:0/2896102489 wait complete. 2024-05-21T21:59:36.738 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:59:36.778 INFO:tasks.quiescer:done joining Quiescers 2024-05-21T21:59:36.779 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-05-21T21:59:36.789 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-05-21T21:59:36.789 INFO:teuthology.task.exec:Executing custom commands... 2024-05-21T21:59:36.789 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi151.front.sepia.ceph.com 2024-05-21T21:59:36.790 DEBUG:teuthology.orchestra.run.smithi151:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-05-21T21:59:36.859 DEBUG:teuthology.orchestra.run.smithi151:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-05-21T21:59:37.635 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:59:37 smithi151 ceph-mon[31623]: pgmap v123: 97 pgs: 97 active+clean; 284 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 351 KiB/s wr, 11 op/s 2024-05-21T21:59:37.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:59:37 smithi184 ceph-mon[36942]: pgmap v123: 97 pgs: 97 active+clean; 284 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 351 KiB/s wr, 11 op/s 2024-05-21T21:59:37.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:59:37 smithi164 ceph-mon[36269]: pgmap v123: 97 pgs: 97 active+clean; 284 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 351 KiB/s wr, 11 op/s 2024-05-21T21:59:39.047 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:39.045+0000 7fd4b4e00700 1 -- 172.21.15.151:0/4065466779 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4b01041a0 msgr2=0x7fd4b01081c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:59:39.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:39.045+0000 7fd4b4e00700 1 --2- 172.21.15.151:0/4065466779 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4b01041a0 0x7fd4b01081c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fd49c009920 tx=0x7fd49c009c30 comp rx=0 tx=0).stop 2024-05-21T21:59:39.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:39.045+0000 7fd4b4e00700 1 -- 172.21.15.151:0/4065466779 shutdown_connections 2024-05-21T21:59:39.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:39.045+0000 7fd4b4e00700 1 --2- 172.21.15.151:0/4065466779 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd4b01088f0 0x7fd4b0110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:39.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:39.045+0000 7fd4b4e00700 1 --2- 172.21.15.151:0/4065466779 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4b01037f0 0x7fd4b0103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:39.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:39.045+0000 7fd4b4e00700 1 --2- 172.21.15.151:0/4065466779 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4b01041a0 0x7fd4b01081c0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:39.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:39.045+0000 7fd4b4e00700 1 -- 172.21.15.151:0/4065466779 >> 172.21.15.151:0/4065466779 conn(0x7fd4b007a6a0 msgr2=0x7fd4b007aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:59:39.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:39.045+0000 7fd4b4e00700 1 -- 172.21.15.151:0/4065466779 shutdown_connections 2024-05-21T21:59:39.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:39.045+0000 7fd4b4e00700 1 -- 172.21.15.151:0/4065466779 wait complete. 2024-05-21T21:59:39.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:39.046+0000 7fd4b4e00700 1 Processor -- start 2024-05-21T21:59:39.049 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:39.046+0000 7fd4b4e00700 1 -- start start 2024-05-21T21:59:39.051 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:39.047+0000 7fd4b4e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4b01037f0 0x7fd4b019be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:39.051 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:39.047+0000 7fd4b4e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4b01041a0 0x7fd4b019c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:39.051 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:39.047+0000 7fd4b4e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd4b01088f0 0x7fd4b01a0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:39.051 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:39.047+0000 7fd4b4e00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd4b0113720 con 0x7fd4b01088f0 2024-05-21T21:59:39.051 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:39.047+0000 7fd4b4e00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd4b01135a0 con 0x7fd4b01037f0 2024-05-21T21:59:39.051 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:39.047+0000 7fd4b4e00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd4b01138a0 con 0x7fd4b01041a0 2024-05-21T21:59:39.051 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:39.047+0000 7fd4ad400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4b01041a0 0x7fd4b019c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:39.051 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:39.047+0000 7fd4ad400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4b01041a0 0x7fd4b019c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:37296/0 (socket says 172.21.15.151:37296) 2024-05-21T21:59:39.051 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:39.047+0000 7fd4ad400700 1 -- 172.21.15.151:0/2903902723 learned_addr learned my addr 172.21.15.151:0/2903902723 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:59:39.051 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:39.047+0000 7fd4ae800700 1 --2- 172.21.15.151:0/2903902723 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd4b01088f0 0x7fd4b01a0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:39.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:39.048+0000 7fd4ade00700 1 --2- 172.21.15.151:0/2903902723 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4b01037f0 0x7fd4b019be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:39.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:39.048+0000 7fd4ad400700 1 -- 172.21.15.151:0/2903902723 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4b01037f0 msgr2=0x7fd4b019be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:59:39.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:39.048+0000 7fd4ad400700 1 --2- 172.21.15.151:0/2903902723 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4b01037f0 0x7fd4b019be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:39.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:39.048+0000 7fd4ad400700 1 -- 172.21.15.151:0/2903902723 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd4b01088f0 msgr2=0x7fd4b01a0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:59:39.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:39.048+0000 7fd4ad400700 1 --2- 172.21.15.151:0/2903902723 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd4b01088f0 0x7fd4b01a0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:39.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:39.048+0000 7fd4ad400700 1 -- 172.21.15.151:0/2903902723 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4b01a0e80 con 0x7fd4b01041a0 2024-05-21T21:59:39.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:39.048+0000 7fd4ade00700 1 --2- 172.21.15.151:0/2903902723 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4b01037f0 0x7fd4b019be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T21:59:39.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:39.048+0000 7fd4ae800700 1 --2- 172.21.15.151:0/2903902723 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd4b01088f0 0x7fd4b01a0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T21:59:39.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:39.048+0000 7fd4ad400700 1 --2- 172.21.15.151:0/2903902723 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4b01041a0 0x7fd4b019c3b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd49c000c00 tx=0x7fd49c00fa40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:59:39.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:39.048+0000 7fd4a6a00700 1 -- 172.21.15.151:0/2903902723 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd49c027070 con 0x7fd4b01041a0 2024-05-21T21:59:39.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:39.048+0000 7fd4a6a00700 1 -- 172.21.15.151:0/2903902723 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd49c005670 con 0x7fd4b01041a0 2024-05-21T21:59:39.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:39.048+0000 7fd4b4e00700 1 -- 172.21.15.151:0/2903902723 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4b01a1110 con 0x7fd4b01041a0 2024-05-21T21:59:39.053 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:39.048+0000 7fd4a6a00700 1 -- 172.21.15.151:0/2903902723 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd49c0049c0 con 0x7fd4b01041a0 2024-05-21T21:59:39.053 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:39.049+0000 7fd4b4e00700 1 -- 172.21.15.151:0/2903902723 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4b01a89f0 con 0x7fd4b01041a0 2024-05-21T21:59:39.053 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:39.050+0000 7fd4b4e00700 1 -- 172.21.15.151:0/2903902723 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4b0104c80 con 0x7fd4b01041a0 2024-05-21T21:59:39.054 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:39.053+0000 7fd4a6a00700 1 -- 172.21.15.151:0/2903902723 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7fd49c02a070 con 0x7fd4b01041a0 2024-05-21T21:59:39.056 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:39.055+0000 7fd4a6a00700 1 --2- 172.21.15.151:0/2903902723 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fd4940b29f0 0x7fd4940b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:39.056 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:39.055+0000 7fd4a6a00700 1 -- 172.21.15.151:0/2903902723 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fd49c0e3080 con 0x7fd4b01041a0 2024-05-21T21:59:39.056 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:39.055+0000 7fd4ade00700 1 --2- 172.21.15.151:0/2903902723 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fd4940b29f0 0x7fd4940b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:39.057 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:39.056+0000 7fd4ade00700 1 --2- 172.21.15.151:0/2903902723 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fd4940b29f0 0x7fd4940b4eb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd4a0009be0 tx=0x7fd4a0009260 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:59:39.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:39.059+0000 7fd4a6a00700 1 -- 172.21.15.151:0/2903902723 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd49c05fdf0 con 0x7fd4b01041a0 2024-05-21T21:59:39.324 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:39.322+0000 7fd4b4e00700 1 -- 172.21.15.151:0/2903902723 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd4b006a230 con 0x7fd4b01041a0 2024-05-21T21:59:39.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:39.324+0000 7fd4a6a00700 1 -- 172.21.15.151:0/2903902723 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11958 (secure 0 0 0) 0x7fd4b006a230 con 0x7fd4b01041a0 2024-05-21T21:59:39.325 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T21:59:39.326 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14358,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/158010529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":158010529},{"type":"v1","addr":"172.21.15.151:6835","nonce":158010529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6839/2069058115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":2069058115},{"type":"v1","addr":"172.21.15.151:6839","nonce":2069058115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T21:51:19.552069+0000","modified":"2024-05-21T21:55:34.643755+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24530,"mds_1":24493,"mds_2":24491,"mds_3":24478,"mds_4":24473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14403":{"gid":14403,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6841/1749723794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6840","nonce":1749723794},{"type":"v1","addr":"172.21.15.151:6841","nonce":1749723794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/2026266114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":2026266114},{"type":"v1","addr":"172.21.15.184:6833","nonce":2026266114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/2467900844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":2467900844},{"type":"v1","addr":"172.21.15.164:6833","nonce":2467900844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3367640960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3367640960},{"type":"v1","addr":"172.21.15.151:6837","nonce":3367640960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/3264142621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3264142621},{"type":"v1","addr":"172.21.15.184:6835","nonce":3264142621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":17,"addr":"172.21.15.164:6835/2340428470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2340428470},{"type":"v1","addr":"172.21.15.164:6835","nonce":2340428470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":16,"addr":"172.21.15.184:6837/2988194888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":2988194888},{"type":"v1","addr":"172.21.15.184:6837","nonce":2988194888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.164:6837/1554609572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1554609572},{"type":"v1","addr":"172.21.15.164:6837","nonce":1554609572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.184:6839/3059935143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":3059935143},{"type":"v1","addr":"172.21.15.184:6839","nonce":3059935143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.164:6839/2646401247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":2646401247},{"type":"v1","addr":"172.21.15.164:6839","nonce":2646401247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24530,"qdb_cluster":[24530,24493,24491,24473,24478]},"id":1}]} 2024-05-21T21:59:39.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:39.327+0000 7fd4b4e00700 1 -- 172.21.15.151:0/2903902723 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fd4940b29f0 msgr2=0x7fd4940b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:59:39.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:39.327+0000 7fd4b4e00700 1 --2- 172.21.15.151:0/2903902723 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fd4940b29f0 0x7fd4940b4eb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd4a0009be0 tx=0x7fd4a0009260 comp rx=0 tx=0).stop 2024-05-21T21:59:39.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:39.327+0000 7fd4b4e00700 1 -- 172.21.15.151:0/2903902723 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4b01041a0 msgr2=0x7fd4b019c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:59:39.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:39.327+0000 7fd4b4e00700 1 --2- 172.21.15.151:0/2903902723 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4b01041a0 0x7fd4b019c3b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd49c000c00 tx=0x7fd49c00fa40 comp rx=0 tx=0).stop 2024-05-21T21:59:39.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:39.327+0000 7fd4b4e00700 1 -- 172.21.15.151:0/2903902723 shutdown_connections 2024-05-21T21:59:39.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:39.327+0000 7fd4b4e00700 1 --2- 172.21.15.151:0/2903902723 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fd4940b29f0 0x7fd4940b4eb0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:39.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:39.327+0000 7fd4b4e00700 1 --2- 172.21.15.151:0/2903902723 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4b01037f0 0x7fd4b019be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:39.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:39.327+0000 7fd4b4e00700 1 --2- 172.21.15.151:0/2903902723 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd4b01088f0 0x7fd4b01a0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:39.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:39.327+0000 7fd4b4e00700 1 --2- 172.21.15.151:0/2903902723 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4b01041a0 0x7fd4b019c3b0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:39.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:39.327+0000 7fd4b4e00700 1 -- 172.21.15.151:0/2903902723 >> 172.21.15.151:0/2903902723 conn(0x7fd4b007a6a0 msgr2=0x7fd4b010fcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:59:39.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:39.327+0000 7fd4b4e00700 1 -- 172.21.15.151:0/2903902723 shutdown_connections 2024-05-21T21:59:39.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:39.327+0000 7fd4b4e00700 1 -- 172.21.15.151:0/2903902723 wait complete. 2024-05-21T21:59:39.330 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 29 2024-05-21T21:59:39.687 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:59:39 smithi151 ceph-mon[31623]: pgmap v124: 97 pgs: 97 active+clean; 284 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 353 KiB/s wr, 16 op/s 2024-05-21T21:59:39.688 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:59:39 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/2903902723' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T21:59:39.735 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:59:39 smithi184 ceph-mon[36942]: pgmap v124: 97 pgs: 97 active+clean; 284 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 353 KiB/s wr, 16 op/s 2024-05-21T21:59:39.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:59:39 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/2903902723' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T21:59:39.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:59:39 smithi164 ceph-mon[36269]: pgmap v124: 97 pgs: 97 active+clean; 284 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 353 KiB/s wr, 16 op/s 2024-05-21T21:59:39.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:59:39 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/2903902723' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T21:59:40.172 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-21T21:59:40.618 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:59:41.699 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:59:41 smithi151 ceph-mon[31623]: pgmap v125: 97 pgs: 97 active+clean; 284 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.4 KiB/s wr, 10 op/s 2024-05-21T21:59:41.735 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:59:41 smithi184 ceph-mon[36942]: pgmap v125: 97 pgs: 97 active+clean; 284 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.4 KiB/s wr, 10 op/s 2024-05-21T21:59:41.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:59:41 smithi164 ceph-mon[36269]: pgmap v125: 97 pgs: 97 active+clean; 284 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.4 KiB/s wr, 10 op/s 2024-05-21T21:59:42.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:42.987+0000 7f64bd400700 1 -- 172.21.15.151:0/1282070040 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f64b8106120 msgr2=0x7f64b8112630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:59:42.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:42.987+0000 7f64bd400700 1 --2- 172.21.15.151:0/1282070040 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f64b8106120 0x7f64b8112630 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f64b000b0a0 tx=0x7f64b000b3b0 comp rx=0 tx=0).stop 2024-05-21T21:59:42.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:42.987+0000 7f64bd400700 1 -- 172.21.15.151:0/1282070040 shutdown_connections 2024-05-21T21:59:42.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:42.987+0000 7f64bd400700 1 --2- 172.21.15.151:0/1282070040 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64b810b760 0x7f64b810bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:42.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:42.987+0000 7f64bd400700 1 --2- 172.21.15.151:0/1282070040 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f64b8106120 0x7f64b8112630 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:42.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:42.987+0000 7f64bd400700 1 --2- 172.21.15.151:0/1282070040 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f64b8105760 0x7f64b8105be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:42.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:42.987+0000 7f64bd400700 1 -- 172.21.15.151:0/1282070040 >> 172.21.15.151:0/1282070040 conn(0x7f64b8101250 msgr2=0x7f64b8103670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:59:42.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:42.987+0000 7f64bd400700 1 -- 172.21.15.151:0/1282070040 shutdown_connections 2024-05-21T21:59:42.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:42.987+0000 7f64bd400700 1 -- 172.21.15.151:0/1282070040 wait complete. 2024-05-21T21:59:42.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:42.988+0000 7f64bd400700 1 Processor -- start 2024-05-21T21:59:42.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:42.988+0000 7f64bd400700 1 -- start start 2024-05-21T21:59:42.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:42.988+0000 7f64bd400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f64b8105760 0x7f64b819bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:42.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:42.989+0000 7f64bd400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64b8106120 0x7f64b819c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:42.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:42.989+0000 7f64bd400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f64b810b760 0x7f64b81a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:42.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:42.989+0000 7f64bd400700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f64b8115940 con 0x7f64b8105760 2024-05-21T21:59:42.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:42.989+0000 7f64bd400700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f64b81157c0 con 0x7f64b8106120 2024-05-21T21:59:42.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:42.989+0000 7f64bd400700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f64b8115ac0 con 0x7f64b810b760 2024-05-21T21:59:42.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:42.989+0000 7f64b7200700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f64b810b760 0x7f64b81a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:42.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:42.989+0000 7f64b6800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f64b8105760 0x7f64b819bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:42.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:42.989+0000 7f64b5e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64b8106120 0x7f64b819c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:42.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:42.989+0000 7f64b7200700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f64b810b760 0x7f64b81a0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:37320/0 (socket says 172.21.15.151:37320) 2024-05-21T21:59:42.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:42.989+0000 7f64b7200700 1 -- 172.21.15.151:0/680149529 learned_addr learned my addr 172.21.15.151:0/680149529 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:59:42.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:42.989+0000 7f64b7200700 1 -- 172.21.15.151:0/680149529 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64b8106120 msgr2=0x7f64b819c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:59:42.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:42.989+0000 7f64b7200700 1 --2- 172.21.15.151:0/680149529 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64b8106120 0x7f64b819c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:42.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:42.989+0000 7f64b7200700 1 -- 172.21.15.151:0/680149529 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f64b8105760 msgr2=0x7f64b819bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:59:42.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:42.989+0000 7f64b7200700 1 --2- 172.21.15.151:0/680149529 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f64b8105760 0x7f64b819bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:42.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:42.989+0000 7f64b7200700 1 -- 172.21.15.151:0/680149529 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64b81a0da0 con 0x7f64b810b760 2024-05-21T21:59:42.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:42.990+0000 7f64b7200700 1 --2- 172.21.15.151:0/680149529 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f64b810b760 0x7f64b81a0660 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f64b000b780 tx=0x7f64b0004d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:59:42.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:42.990+0000 7f64af400700 1 -- 172.21.15.151:0/680149529 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b0021070 con 0x7f64b810b760 2024-05-21T21:59:42.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:42.990+0000 7f64af400700 1 -- 172.21.15.151:0/680149529 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64b0007e30 con 0x7f64b810b760 2024-05-21T21:59:42.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:42.990+0000 7f64bd400700 1 -- 172.21.15.151:0/680149529 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64b81a1030 con 0x7f64b810b760 2024-05-21T21:59:42.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:42.990+0000 7f64af400700 1 -- 172.21.15.151:0/680149529 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b0004700 con 0x7f64b810b760 2024-05-21T21:59:42.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:42.990+0000 7f64bd400700 1 -- 172.21.15.151:0/680149529 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64b81a8910 con 0x7f64b810b760 2024-05-21T21:59:42.995 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:42.994+0000 7f64af400700 1 -- 172.21.15.151:0/680149529 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7f64b0007980 con 0x7f64b810b760 2024-05-21T21:59:42.996 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:42.995+0000 7f64c7d38b80 1 Processor -- start 2024-05-21T21:59:42.997 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:42.996+0000 7f64c7d38b80 1 -- start start 2024-05-21T21:59:42.998 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:42.996+0000 7f64c7d38b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e03e182610 0x55e03e1829f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:42.998 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:42.996+0000 7f64c7d38b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e03e182f30 0x55e03e17cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:42.998 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:42.996+0000 7f64c7d38b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e03e17d140 0x55e03e17d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:42.998 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:42.996+0000 7f64c7d38b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e03e0dae40 con 0x55e03e17d140 2024-05-21T21:59:42.998 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:42.996+0000 7f64c7d38b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55e03e0ca9e0 con 0x55e03e182610 2024-05-21T21:59:42.998 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:42.996+0000 7f64c7d38b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55e03e0cea20 con 0x55e03e182f30 2024-05-21T21:59:42.998 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:42.996+0000 7f64af400700 1 --2- 172.21.15.151:0/680149529 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f649c0b29f0 0x7f649c0b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:42.998 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:42.997+0000 7f64b5e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e03e182610 0x55e03e1829f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:42.998 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:42.997+0000 7f64b5e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e03e17d140 0x55e03e17d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:42.999 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:42.997+0000 7f64b5e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e03e17d140 0x55e03e17d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:51878/0 (socket says 172.21.15.151:51878) 2024-05-21T21:59:42.999 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:42.997+0000 7f64b5e00700 1 -- 172.21.15.151:0/1794788268 learned_addr learned my addr 172.21.15.151:0/1794788268 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:59:43.000 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:42.997+0000 7f64af400700 1 -- 172.21.15.151:0/680149529 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f64b000f020 con 0x7f64b810b760 2024-05-21T21:59:43.000 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:42.997+0000 7f64b7200700 1 --2- 172.21.15.151:0/680149529 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f649c0b29f0 0x7f649c0b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:43.000 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:42.997+0000 7f64b6800700 1 --2- 172.21.15.151:0/1794788268 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e03e182f30 0x55e03e17cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:43.000 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:42.997+0000 7f64b5e00700 1 -- 172.21.15.151:0/1794788268 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e03e182f30 msgr2=0x55e03e17cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:59:43.000 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:42.997+0000 7f64b5e00700 1 --2- 172.21.15.151:0/1794788268 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e03e182f30 0x55e03e17cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:43.000 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:42.997+0000 7f64b5e00700 1 -- 172.21.15.151:0/1794788268 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e03e17d140 msgr2=0x55e03e17d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:59:43.000 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:42.997+0000 7f64b5e00700 1 --2- 172.21.15.151:0/1794788268 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e03e17d140 0x55e03e17d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:43.000 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:42.997+0000 7f64b5e00700 1 -- 172.21.15.151:0/1794788268 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e03e093420 con 0x55e03e182610 2024-05-21T21:59:43.000 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:42.998+0000 7f64b7200700 1 --2- 172.21.15.151:0/680149529 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f649c0b29f0 0x7f649c0b4eb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f64b00aac80 tx=0x7f64b0009040 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:59:43.000 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:42.998+0000 7f64b5e00700 1 --2- 172.21.15.151:0/1794788268 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e03e182610 0x55e03e1829f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f64a4009fd0 tx=0x7f64a4004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4e97ef12777c19b server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:59:43.000 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:42.998+0000 7f64a3e00700 1 -- 172.21.15.151:0/1794788268 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a4016030 con 0x55e03e182610 2024-05-21T21:59:43.000 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:42.998+0000 7f64a3e00700 1 -- 172.21.15.151:0/1794788268 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64a4005020 con 0x55e03e182610 2024-05-21T21:59:43.000 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:42.998+0000 7f64a3e00700 1 -- 172.21.15.151:0/1794788268 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a4019040 con 0x55e03e182610 2024-05-21T21:59:43.002 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:42.999+0000 7f64c7d38b80 1 -- 172.21.15.151:0/1794788268 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e03e182610 msgr2=0x55e03e1829f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:59:43.002 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:42.999+0000 7f64c7d38b80 1 --2- 172.21.15.151:0/1794788268 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e03e182610 0x55e03e1829f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f64a4009fd0 tx=0x7f64a4004750 comp rx=0 tx=0).stop 2024-05-21T21:59:43.002 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:42.999+0000 7f64c7d38b80 1 -- 172.21.15.151:0/1794788268 shutdown_connections 2024-05-21T21:59:43.002 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:42.999+0000 7f64c7d38b80 1 --2- 172.21.15.151:0/1794788268 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e03e182610 0x55e03e1829f0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:43.002 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:42.999+0000 7f64c7d38b80 1 --2- 172.21.15.151:0/1794788268 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e03e17d140 0x55e03e17d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:43.002 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:42.999+0000 7f64c7d38b80 1 --2- 172.21.15.151:0/1794788268 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e03e182f30 0x55e03e17cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:43.002 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:42.999+0000 7f64c7d38b80 1 -- 172.21.15.151:0/1794788268 >> 172.21.15.151:0/1794788268 conn(0x55e03e0aabe0 msgr2=0x55e03e177690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:59:43.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:42.999+0000 7f64c7d38b80 1 -- 172.21.15.151:0/1794788268 shutdown_connections 2024-05-21T21:59:43.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:42.999+0000 7f64c7d38b80 1 -- 172.21.15.151:0/1794788268 wait complete. 2024-05-21T21:59:43.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:42.999+0000 7f64c7d38b80 1 -- start start 2024-05-21T21:59:43.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.000+0000 7f64c7d38b80 1 Processor -- start 2024-05-21T21:59:43.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.000+0000 7f64c7d38b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e03e17d140 0x55e03e188d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:43.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.000+0000 7f64c7d38b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e03e182610 0x55e03e1860d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:43.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.000+0000 7f64c7d38b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e03e182f30 0x55e03e186610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:43.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.000+0000 7f64c7d38b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e03dfd0fa0 con 0x55e03e17d140 2024-05-21T21:59:43.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.000+0000 7f64c7d38b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55e03e06b760 con 0x55e03e182610 2024-05-21T21:59:43.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.000+0000 7f64c7d38b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55e03e1899e0 con 0x55e03e182f30 2024-05-21T21:59:43.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.000+0000 7f64c7d38b80 10 client.0 _ll_register_callbacks cb 0x55e03e074c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:59:43.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.001+0000 7f64b5e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e03e17d140 0x55e03e188d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:43.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.001+0000 7f64b5e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e03e17d140 0x55e03e188d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:51884/0 (socket says 172.21.15.151:51884) 2024-05-21T21:59:43.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.001+0000 7f64b5e00700 1 -- 172.21.15.151:0/4252820167 learned_addr learned my addr 172.21.15.151:0/4252820167 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:59:43.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.001+0000 7f64b6800700 1 --2- 172.21.15.151:0/4252820167 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e03e182610 0x55e03e1860d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:43.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.001+0000 7f64b5e00700 1 --2- 172.21.15.151:0/4252820167 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e03e182f30 0x55e03e186610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:43.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.001+0000 7f64b6800700 1 -- 172.21.15.151:0/4252820167 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e03e182f30 msgr2=0x55e03e186610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:59:43.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.001+0000 7f64b6800700 1 --2- 172.21.15.151:0/4252820167 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e03e182f30 0x55e03e186610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:43.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.001+0000 7f64b6800700 1 -- 172.21.15.151:0/4252820167 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e03e17d140 msgr2=0x55e03e188d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:59:43.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.001+0000 7f64b6800700 1 --2- 172.21.15.151:0/4252820167 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e03e17d140 0x55e03e188d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:43.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.001+0000 7f64b6800700 1 -- 172.21.15.151:0/4252820167 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e03e01a8d0 con 0x55e03e182610 2024-05-21T21:59:43.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.001+0000 7f64b5e00700 1 --2- 172.21.15.151:0/4252820167 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e03e182f30 0x55e03e186610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T21:59:43.006 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.001+0000 7f64b6800700 1 --2- 172.21.15.151:0/4252820167 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e03e182610 0x55e03e1860d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f64a80116f0 tx=0x7f64a8011a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dcc12ff9ab683f32 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:59:43.006 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.002+0000 7f64a0c00700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-21T21:59:43.006 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.002+0000 7f64a0c00700 1 -- 172.21.15.151:0/4252820167 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a801f060 con 0x55e03e182610 2024-05-21T21:59:43.006 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.002+0000 7f64c7d38b80 1 -- 172.21.15.151:0/4252820167 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64a4005020 con 0x55e03e182610 2024-05-21T21:59:43.006 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.002+0000 7f64a0c00700 1 -- 172.21.15.151:0/4252820167 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f64a8004320 con 0x55e03e182610 2024-05-21T21:59:43.006 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.002+0000 7f64a0c00700 1 -- 172.21.15.151:0/4252820167 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f64a8004520 con 0x55e03e182610 2024-05-21T21:59:43.006 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.002+0000 7f64c7d38b80 10 client.26120 fetch_fsmap learned FSMap version 29 2024-05-21T21:59:43.006 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.002+0000 7f64c7d38b80 1 -- 172.21.15.151:0/4252820167 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e03e09b060 con 0x55e03e182610 2024-05-21T21:59:43.006 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.003+0000 7f64a0c00700 1 -- 172.21.15.151:0/4252820167 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64a4005020 con 0x55e03e182610 2024-05-21T21:59:43.006 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.003+0000 7f64a0c00700 1 -- 172.21.15.151:0/4252820167 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f64a800f840 con 0x55e03e182610 2024-05-21T21:59:43.006 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.003+0000 7f64c7d38b80 10 client.26120 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T21:59:43.006 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.003+0000 7f64c7d38b80 10 client.26120 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-05-21T21:59:43.006 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.003+0000 7f64c7d38b80 20 client.26120 populate_metadata read hostname 'smithi151' 2024-05-21T21:59:43.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.003+0000 7f64c7d38b80 1 --2- 172.21.15.151:0/4252820167 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x55e03e1966c0 0x55e03e198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:43.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.003+0000 7f64c7d38b80 4 client.26120 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T21:59:43.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.003+0000 7f64c7d38b80 1 -- 172.21.15.151:0/4252820167 --> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e03e0dcd90 con 0x55e03e1966c0 2024-05-21T21:59:43.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.004+0000 7f64b5e00700 1 --2- 172.21.15.151:0/4252820167 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x55e03e1966c0 0x55e03e198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:43.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.004+0000 7f64b5e00700 1 --2- 172.21.15.151:0/4252820167 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x55e03e1966c0 0x55e03e198ab0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ce6f6f89324cbcd7 server_cookie=cef243c6cab9d14a in_seq=0 out_seq=0 2024-05-21T21:59:43.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.004+0000 7f64a0c00700 10 client.26120 ms_handle_connect on v2:172.21.15.164:6834/2340428470 2024-05-21T21:59:43.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.005+0000 7f64a0c00700 1 -- 172.21.15.151:0/4252820167 <== mds.4 v2:172.21.15.164:6834/2340428470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55e03e0dcd90 con 0x55e03e1966c0 2024-05-21T21:59:43.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.005+0000 7f64a0c00700 10 client.26120 handle_command_reply: tid=0 2024-05-21T21:59:43.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.005+0000 7f64c7d38b80 1 client.26120 shutdown 2024-05-21T21:59:43.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.005+0000 7f64c7d38b80 1 -- 172.21.15.151:0/4252820167 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e03e182610 msgr2=0x55e03e1860d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:59:43.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.005+0000 7f64c7d38b80 1 --2- 172.21.15.151:0/4252820167 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e03e182610 0x55e03e1860d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f64a80116f0 tx=0x7f64a8011a00 comp rx=0 tx=0).stop 2024-05-21T21:59:43.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.005+0000 7f64c7d38b80 1 -- 172.21.15.151:0/4252820167 shutdown_connections 2024-05-21T21:59:43.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.005+0000 7f64c7d38b80 1 --2- 172.21.15.151:0/4252820167 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x55e03e1966c0 0x55e03e198ab0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:43.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.005+0000 7f64c7d38b80 1 --2- 172.21.15.151:0/4252820167 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e03e182f30 0x55e03e186610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:43.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.005+0000 7f64c7d38b80 1 --2- 172.21.15.151:0/4252820167 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e03e17d140 0x55e03e188d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:43.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.005+0000 7f64c7d38b80 1 --2- 172.21.15.151:0/4252820167 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e03e182610 0x55e03e1860d0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:43.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.005+0000 7f64c7d38b80 1 -- 172.21.15.151:0/4252820167 >> 172.21.15.151:0/4252820167 conn(0x55e03e0aabe0 msgr2=0x55e03e176d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:59:43.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.005+0000 7f64a0c00700 0 client.26120 ms_handle_reset on v2:172.21.15.164:6834/2340428470 2024-05-21T21:59:43.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.006+0000 7f64c7d38b80 1 -- 172.21.15.151:0/4252820167 shutdown_connections 2024-05-21T21:59:43.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.006+0000 7f64c7d38b80 1 -- 172.21.15.151:0/4252820167 wait complete. 2024-05-21T21:59:43.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.006+0000 7f64c7d38b80 20 client.26120 trim_cache size 0 max 16384 2024-05-21T21:59:43.023 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.022+0000 7f64c7d38b80 1 Processor -- start 2024-05-21T21:59:43.023 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.022+0000 7f64c7d38b80 1 -- start start 2024-05-21T21:59:43.024 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.022+0000 7f64c7d38b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e03e17d140 0x55e03e17cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:43.024 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.022+0000 7f64c7d38b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e03e182610 0x55e03e192ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:43.024 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.022+0000 7f64c7d38b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e03e182f30 0x55e03e185030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:43.024 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.022+0000 7f64c7d38b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f64b81afe60 con 0x55e03e182610 2024-05-21T21:59:43.024 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.022+0000 7f64c7d38b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f64b81b19a0 con 0x55e03e17d140 2024-05-21T21:59:43.024 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.022+0000 7f64c7d38b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f64b81b0530 con 0x55e03e182f30 2024-05-21T21:59:43.025 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.023+0000 7f64b6800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e03e182610 0x55e03e192ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:43.025 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.023+0000 7f64b5e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e03e17d140 0x55e03e17cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:43.025 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.023+0000 7f64b5e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e03e17d140 0x55e03e17cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:45426/0 (socket says 172.21.15.151:45426) 2024-05-21T21:59:43.025 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.023+0000 7f64b6800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e03e182610 0x55e03e192ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:51886/0 (socket says 172.21.15.151:51886) 2024-05-21T21:59:43.025 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.023+0000 7f64b5e00700 1 -- 172.21.15.151:0/1028059354 learned_addr learned my addr 172.21.15.151:0/1028059354 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:59:43.025 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.023+0000 7f64b5e00700 1 --2- 172.21.15.151:0/1028059354 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e03e182f30 0x55e03e185030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:43.026 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.023+0000 7f64b6800700 1 -- 172.21.15.151:0/1028059354 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e03e182f30 msgr2=0x55e03e185030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:59:43.026 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.023+0000 7f64b6800700 1 --2- 172.21.15.151:0/1028059354 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e03e182f30 0x55e03e185030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:43.026 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.023+0000 7f64b6800700 1 -- 172.21.15.151:0/1028059354 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e03e17d140 msgr2=0x55e03e17cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:59:43.026 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.023+0000 7f64b6800700 1 --2- 172.21.15.151:0/1028059354 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e03e17d140 0x55e03e17cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:43.026 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.023+0000 7f64b6800700 1 -- 172.21.15.151:0/1028059354 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e03e1940e0 con 0x55e03e182610 2024-05-21T21:59:43.026 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.023+0000 7f64b6800700 1 --2- 172.21.15.151:0/1028059354 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e03e182610 0x55e03e192ab0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f64a800fce0 tx=0x7f64a8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e1cba1507884243 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:59:43.026 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.024+0000 7f64a3400700 1 -- 172.21.15.151:0/1028059354 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a8031070 con 0x55e03e182610 2024-05-21T21:59:43.026 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.024+0000 7f64a3400700 1 -- 172.21.15.151:0/1028059354 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64a8004320 con 0x55e03e182610 2024-05-21T21:59:43.026 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.024+0000 7f64a3400700 1 -- 172.21.15.151:0/1028059354 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a8003900 con 0x55e03e182610 2024-05-21T21:59:43.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.025+0000 7f64c7d38b80 1 -- 172.21.15.151:0/1028059354 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e03e182610 msgr2=0x55e03e192ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:59:43.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.025+0000 7f64c7d38b80 1 --2- 172.21.15.151:0/1028059354 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e03e182610 0x55e03e192ab0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f64a800fce0 tx=0x7f64a8009750 comp rx=0 tx=0).stop 2024-05-21T21:59:43.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.025+0000 7f64c7d38b80 1 -- 172.21.15.151:0/1028059354 shutdown_connections 2024-05-21T21:59:43.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.025+0000 7f64c7d38b80 1 --2- 172.21.15.151:0/1028059354 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e03e17d140 0x55e03e17cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:43.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.025+0000 7f64c7d38b80 1 --2- 172.21.15.151:0/1028059354 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e03e182f30 0x55e03e185030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:43.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.025+0000 7f64c7d38b80 1 --2- 172.21.15.151:0/1028059354 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e03e182610 0x55e03e192ab0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:43.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.025+0000 7f64c7d38b80 1 -- 172.21.15.151:0/1028059354 >> 172.21.15.151:0/1028059354 conn(0x55e03e0aabe0 msgr2=0x55e03e183310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:59:43.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.025+0000 7f64c7d38b80 1 -- 172.21.15.151:0/1028059354 shutdown_connections 2024-05-21T21:59:43.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.025+0000 7f64c7d38b80 1 -- 172.21.15.151:0/1028059354 wait complete. 2024-05-21T21:59:43.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.025+0000 7f64c7d38b80 1 -- start start 2024-05-21T21:59:43.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.026+0000 7f64c7d38b80 1 Processor -- start 2024-05-21T21:59:43.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.026+0000 7f64c7d38b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e03e17d140 0x55e03e18f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:43.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.026+0000 7f64c7d38b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e03e182610 0x55e03e19d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:43.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.026+0000 7f64c7d38b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e03e182f30 0x55e03e19de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:43.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.026+0000 7f64c7d38b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f64b81b12d0 con 0x55e03e182610 2024-05-21T21:59:43.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.026+0000 7f64c7d38b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f64b81b0c00 con 0x55e03e17d140 2024-05-21T21:59:43.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.026+0000 7f64c7d38b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f64b81ade00 con 0x55e03e182f30 2024-05-21T21:59:43.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.026+0000 7f64c7d38b80 10 client.0 _ll_register_callbacks cb 0x55e03e074c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:59:43.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.027+0000 7f64b6800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e03e182610 0x55e03e19d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:43.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.027+0000 7f64b5e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e03e17d140 0x55e03e18f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:43.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.027+0000 7f64b6800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e03e182610 0x55e03e19d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:51896/0 (socket says 172.21.15.151:51896) 2024-05-21T21:59:43.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.027+0000 7f64b6800700 1 -- 172.21.15.151:0/1319284735 learned_addr learned my addr 172.21.15.151:0/1319284735 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:59:43.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.027+0000 7f64b5e00700 1 --2- 172.21.15.151:0/1319284735 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e03e182f30 0x55e03e19de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:43.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.027+0000 7f64b5e00700 1 -- 172.21.15.151:0/1319284735 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e03e182f30 msgr2=0x55e03e19de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:59:43.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.027+0000 7f64b5e00700 1 --2- 172.21.15.151:0/1319284735 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e03e182f30 0x55e03e19de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:43.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.027+0000 7f64b5e00700 1 -- 172.21.15.151:0/1319284735 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e03e182610 msgr2=0x55e03e19d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:59:43.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.027+0000 7f64b5e00700 1 --2- 172.21.15.151:0/1319284735 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e03e182610 0x55e03e19d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:43.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.027+0000 7f64b5e00700 1 -- 172.21.15.151:0/1319284735 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e03e19e3a0 con 0x55e03e17d140 2024-05-21T21:59:43.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.027+0000 7f64b6800700 1 --2- 172.21.15.151:0/1319284735 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e03e182610 0x55e03e19d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:59:43.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.027+0000 7f64b5e00700 1 --2- 172.21.15.151:0/1319284735 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e03e17d140 0x55e03e18f060 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f64a40097c0 tx=0x7f64a4004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78d3eefee09156e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:59:43.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.027+0000 7f64a0c00700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-21T21:59:43.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.028+0000 7f64a0c00700 1 -- 172.21.15.151:0/1319284735 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a4016040 con 0x55e03e17d140 2024-05-21T21:59:43.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.028+0000 7f64a0c00700 1 -- 172.21.15.151:0/1319284735 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f64a40303c0 con 0x55e03e17d140 2024-05-21T21:59:43.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.028+0000 7f64c7d38b80 1 -- 172.21.15.151:0/1319284735 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64a8004320 con 0x55e03e17d140 2024-05-21T21:59:43.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.028+0000 7f64a0c00700 1 -- 172.21.15.151:0/1319284735 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f64a40305c0 con 0x55e03e17d140 2024-05-21T21:59:43.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.028+0000 7f64c7d38b80 10 client.26126 fetch_fsmap learned FSMap version 29 2024-05-21T21:59:43.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.028+0000 7f64c7d38b80 1 -- 172.21.15.151:0/1319284735 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e03e200f70 con 0x55e03e17d140 2024-05-21T21:59:43.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.028+0000 7f64a0c00700 1 -- 172.21.15.151:0/1319284735 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64a8004320 con 0x55e03e17d140 2024-05-21T21:59:43.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.028+0000 7f64a0c00700 1 -- 172.21.15.151:0/1319284735 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f64a402f040 con 0x55e03e17d140 2024-05-21T21:59:43.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.028+0000 7f64c7d38b80 10 client.26126 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T21:59:43.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.028+0000 7f64c7d38b80 10 client.26126 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-05-21T21:59:43.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.028+0000 7f64c7d38b80 20 client.26126 populate_metadata read hostname 'smithi151' 2024-05-21T21:59:43.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.028+0000 7f64c7d38b80 1 --2- 172.21.15.151:0/1319284735 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x55e03e201480 0x55e03e203870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:43.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.028+0000 7f64c7d38b80 4 client.26126 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T21:59:43.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.028+0000 7f64c7d38b80 1 -- 172.21.15.151:0/1319284735 --> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e03e0d6570 con 0x55e03e201480 2024-05-21T21:59:43.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.029+0000 7f64b6800700 1 --2- 172.21.15.151:0/1319284735 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x55e03e201480 0x55e03e203870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:43.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.029+0000 7f64b6800700 1 --2- 172.21.15.151:0/1319284735 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x55e03e201480 0x55e03e203870 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=392939e89e90ca2 server_cookie=5dca5708a7082a68 in_seq=0 out_seq=0 2024-05-21T21:59:43.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.029+0000 7f64a0c00700 10 client.26126 ms_handle_connect on v2:172.21.15.164:6834/2340428470 2024-05-21T21:59:43.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.030+0000 7f64a0c00700 1 -- 172.21.15.151:0/1319284735 <== mds.4 v2:172.21.15.164:6834/2340428470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e03e0d6570 con 0x55e03e201480 2024-05-21T21:59:43.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.030+0000 7f64a0c00700 10 client.26126 handle_command_reply: tid=0 2024-05-21T21:59:43.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.030+0000 7f64c7d38b80 1 client.26126 shutdown 2024-05-21T21:59:43.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.030+0000 7f64c7d38b80 1 -- 172.21.15.151:0/1319284735 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e03e17d140 msgr2=0x55e03e18f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:59:43.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.030+0000 7f64c7d38b80 1 --2- 172.21.15.151:0/1319284735 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e03e17d140 0x55e03e18f060 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f64a40097c0 tx=0x7f64a4004750 comp rx=0 tx=0).stop 2024-05-21T21:59:43.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.030+0000 7f64c7d38b80 1 -- 172.21.15.151:0/1319284735 shutdown_connections 2024-05-21T21:59:43.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.030+0000 7f64c7d38b80 1 --2- 172.21.15.151:0/1319284735 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x55e03e201480 0x55e03e203870 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:43.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.030+0000 7f64c7d38b80 1 --2- 172.21.15.151:0/1319284735 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e03e17d140 0x55e03e18f060 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:43.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.030+0000 7f64c7d38b80 1 --2- 172.21.15.151:0/1319284735 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e03e182f30 0x55e03e19de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:43.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.030+0000 7f64a0c00700 0 client.26126 ms_handle_reset on v2:172.21.15.164:6834/2340428470 2024-05-21T21:59:43.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.030+0000 7f64c7d38b80 1 --2- 172.21.15.151:0/1319284735 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e03e182610 0x55e03e19d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:43.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.030+0000 7f64c7d38b80 1 -- 172.21.15.151:0/1319284735 >> 172.21.15.151:0/1319284735 conn(0x55e03e0aabe0 msgr2=0x55e03e183ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:59:43.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.030+0000 7f64c7d38b80 1 -- 172.21.15.151:0/1319284735 shutdown_connections 2024-05-21T21:59:43.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.030+0000 7f64c7d38b80 1 -- 172.21.15.151:0/1319284735 wait complete. 2024-05-21T21:59:43.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.030+0000 7f64c7d38b80 20 client.26126 trim_cache size 0 max 16384 2024-05-21T21:59:43.035 INFO:teuthology.orchestra.run.smithi151.stdout:[] 2024-05-21T21:59:43.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.031+0000 7f64bd400700 1 -- 172.21.15.151:0/680149529 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f649c0b29f0 msgr2=0x7f649c0b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:59:43.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.031+0000 7f64bd400700 1 --2- 172.21.15.151:0/680149529 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f649c0b29f0 0x7f649c0b4eb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f64b00aac80 tx=0x7f64b0009040 comp rx=0 tx=0).stop 2024-05-21T21:59:43.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.031+0000 7f64bd400700 1 -- 172.21.15.151:0/680149529 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f64b810b760 msgr2=0x7f64b81a0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:59:43.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.031+0000 7f64bd400700 1 --2- 172.21.15.151:0/680149529 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f64b810b760 0x7f64b81a0660 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f64b000b780 tx=0x7f64b0004d00 comp rx=0 tx=0).stop 2024-05-21T21:59:43.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.031+0000 7f64bd400700 1 -- 172.21.15.151:0/680149529 shutdown_connections 2024-05-21T21:59:43.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.031+0000 7f64bd400700 1 --2- 172.21.15.151:0/680149529 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f649c0b29f0 0x7f649c0b4eb0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:43.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.031+0000 7f64bd400700 1 --2- 172.21.15.151:0/680149529 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f64b810b760 0x7f64b81a0660 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:43.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.031+0000 7f64bd400700 1 --2- 172.21.15.151:0/680149529 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f64b8105760 0x7f64b819bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:43.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.031+0000 7f64bd400700 1 --2- 172.21.15.151:0/680149529 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64b8106120 0x7f64b819c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:43.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.031+0000 7f64bd400700 1 -- 172.21.15.151:0/680149529 >> 172.21.15.151:0/680149529 conn(0x7f64b8101250 msgr2=0x7f64b81018c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:59:43.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.031+0000 7f64bd400700 1 -- 172.21.15.151:0/680149529 shutdown_connections 2024-05-21T21:59:43.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:43.031+0000 7f64bd400700 1 -- 172.21.15.151:0/680149529 wait complete. 2024-05-21T21:59:43.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:59:43 smithi151 ceph-mon[31623]: pgmap v126: 97 pgs: 97 active+clean; 515 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 21 MiB/s wr, 20 op/s 2024-05-21T21:59:43.735 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:59:43 smithi184 ceph-mon[36942]: pgmap v126: 97 pgs: 97 active+clean; 515 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 21 MiB/s wr, 20 op/s 2024-05-21T21:59:43.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:59:43 smithi164 ceph-mon[36269]: pgmap v126: 97 pgs: 97 active+clean; 515 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 21 MiB/s wr, 20 op/s 2024-05-21T21:59:43.857 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-21T21:59:44.309 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:59:45.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:59:45 smithi151 ceph-mon[31623]: pgmap v127: 97 pgs: 97 active+clean; 515 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 21 MiB/s wr, 14 op/s 2024-05-21T21:59:45.735 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:59:45 smithi184 ceph-mon[36942]: pgmap v127: 97 pgs: 97 active+clean; 515 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 21 MiB/s wr, 14 op/s 2024-05-21T21:59:45.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:59:45 smithi164 ceph-mon[36269]: pgmap v127: 97 pgs: 97 active+clean; 515 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 21 MiB/s wr, 14 op/s 2024-05-21T21:59:46.312 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.310+0000 7f0b6c000700 1 -- 172.21.15.151:0/2864260524 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b6406bab0 msgr2=0x7f0b6406bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:59:46.312 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.310+0000 7f0b6c000700 1 --2- 172.21.15.151:0/2864260524 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b6406bab0 0x7f0b6406bf30 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f0b50009990 tx=0x7f0b50009ca0 comp rx=0 tx=0).stop 2024-05-21T21:59:46.312 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.310+0000 7f0b6c000700 1 -- 172.21.15.151:0/2864260524 shutdown_connections 2024-05-21T21:59:46.312 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.310+0000 7f0b6c000700 1 --2- 172.21.15.151:0/2864260524 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0b6406c470 0x7f0b641103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:46.312 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.310+0000 7f0b6c000700 1 --2- 172.21.15.151:0/2864260524 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0b641095a0 0x7f0b64109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:46.312 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.310+0000 7f0b6c000700 1 --2- 172.21.15.151:0/2864260524 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b6406bab0 0x7f0b6406bf30 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:46.312 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.310+0000 7f0b6c000700 1 -- 172.21.15.151:0/2864260524 >> 172.21.15.151:0/2864260524 conn(0x7f0b640793b0 msgr2=0x7f0b640797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:59:46.312 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.310+0000 7f0b6c000700 1 -- 172.21.15.151:0/2864260524 shutdown_connections 2024-05-21T21:59:46.313 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.310+0000 7f0b6c000700 1 -- 172.21.15.151:0/2864260524 wait complete. 2024-05-21T21:59:46.314 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.311+0000 7f0b6c000700 1 Processor -- start 2024-05-21T21:59:46.314 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.312+0000 7f0b6c000700 1 -- start start 2024-05-21T21:59:46.314 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.312+0000 7f0b6c000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0b6406bab0 0x7f0b64102500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:46.314 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.312+0000 7f0b6c000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0b6406c470 0x7f0b64102a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:46.314 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.312+0000 7f0b6c000700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b641095a0 0x7f0b64102f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:46.314 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.312+0000 7f0b6c000700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0b64113700 con 0x7f0b6406c470 2024-05-21T21:59:46.314 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.312+0000 7f0b6c000700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0b64113580 con 0x7f0b6406bab0 2024-05-21T21:59:46.314 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.312+0000 7f0b6c000700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0b64113880 con 0x7f0b641095a0 2024-05-21T21:59:46.314 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.312+0000 7f0b68c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0b6406c470 0x7f0b64102a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:46.315 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.313+0000 7f0b68c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0b6406c470 0x7f0b64102a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:56464/0 (socket says 172.21.15.151:56464) 2024-05-21T21:59:46.315 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.313+0000 7f0b68c00700 1 -- 172.21.15.151:0/2287313639 learned_addr learned my addr 172.21.15.151:0/2287313639 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:59:46.315 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.313+0000 7f0b69600700 1 --2- 172.21.15.151:0/2287313639 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0b6406bab0 0x7f0b64102500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:46.315 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.313+0000 7f0b6a000700 1 --2- 172.21.15.151:0/2287313639 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b641095a0 0x7f0b64102f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:46.316 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.313+0000 7f0b69600700 1 -- 172.21.15.151:0/2287313639 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b641095a0 msgr2=0x7f0b64102f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:59:46.316 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.313+0000 7f0b69600700 1 --2- 172.21.15.151:0/2287313639 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b641095a0 0x7f0b64102f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:46.316 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.313+0000 7f0b69600700 1 -- 172.21.15.151:0/2287313639 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0b6406c470 msgr2=0x7f0b64102a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:59:46.316 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.313+0000 7f0b69600700 1 --2- 172.21.15.151:0/2287313639 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0b6406c470 0x7f0b64102a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:46.316 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.313+0000 7f0b69600700 1 -- 172.21.15.151:0/2287313639 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b641a8160 con 0x7f0b6406bab0 2024-05-21T21:59:46.316 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.313+0000 7f0b68c00700 1 --2- 172.21.15.151:0/2287313639 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0b6406c470 0x7f0b64102a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:59:46.316 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.313+0000 7f0b69600700 1 --2- 172.21.15.151:0/2287313639 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0b6406bab0 0x7f0b64102500 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f0b5800e9f0 tx=0x7f0b5800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:59:46.316 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.314+0000 7f0b62000700 1 -- 172.21.15.151:0/2287313639 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b5800c9c0 con 0x7f0b6406bab0 2024-05-21T21:59:46.316 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.314+0000 7f0b62000700 1 -- 172.21.15.151:0/2287313639 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b5800cb60 con 0x7f0b6406bab0 2024-05-21T21:59:46.316 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.314+0000 7f0b6c000700 1 -- 172.21.15.151:0/2287313639 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b641a8450 con 0x7f0b6406bab0 2024-05-21T21:59:46.317 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.314+0000 7f0b62000700 1 -- 172.21.15.151:0/2287313639 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b58018680 con 0x7f0b6406bab0 2024-05-21T21:59:46.317 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.314+0000 7f0b6c000700 1 -- 172.21.15.151:0/2287313639 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b641a8990 con 0x7f0b6406bab0 2024-05-21T21:59:46.318 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.318+0000 7f0b62000700 1 -- 172.21.15.151:0/2287313639 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7f0b58010430 con 0x7f0b6406bab0 2024-05-21T21:59:46.319 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.318+0000 7f0b76966b80 1 Processor -- start 2024-05-21T21:59:46.320 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.319+0000 7f0b76966b80 1 -- start start 2024-05-21T21:59:46.320 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.319+0000 7f0b76966b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55957e8dc610 0x55957e8dc9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:46.320 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.319+0000 7f0b76966b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55957e8dcf30 0x55957e8d6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:46.320 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.319+0000 7f0b76966b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55957e8d7140 0x55957e8d75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:46.320 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.319+0000 7f0b76966b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55957e834e40 con 0x55957e8dc610 2024-05-21T21:59:46.320 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.319+0000 7f0b76966b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55957e8249e0 con 0x55957e8dcf30 2024-05-21T21:59:46.321 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.319+0000 7f0b76966b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55957e828a20 con 0x55957e8d7140 2024-05-21T21:59:46.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.319+0000 7f0b6a000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55957e8dc610 0x55957e8dc9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:46.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.319+0000 7f0b6a000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55957e8dc610 0x55957e8dc9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:56474/0 (socket says 172.21.15.151:56474) 2024-05-21T21:59:46.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.319+0000 7f0b62000700 1 --2- 172.21.15.151:0/2287313639 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f0b4c0b29f0 0x7f0b4c0b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:46.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.319+0000 7f0b6a000700 1 -- 172.21.15.151:0/3148208832 learned_addr learned my addr 172.21.15.151:0/3148208832 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:59:46.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.319+0000 7f0b68c00700 1 --2- 172.21.15.151:0/3148208832 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55957e8d7140 0x55957e8d75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:46.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.320+0000 7f0b69600700 1 --2- 172.21.15.151:0/3148208832 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55957e8dcf30 0x55957e8d6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:46.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.320+0000 7f0b62000700 1 -- 172.21.15.151:0/2287313639 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f0b580e2750 con 0x7f0b6406bab0 2024-05-21T21:59:46.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.320+0000 7f0b68c00700 1 -- 172.21.15.151:0/3148208832 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55957e8dcf30 msgr2=0x55957e8d6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:59:46.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.320+0000 7f0b6a000700 1 --2- 172.21.15.151:0/2287313639 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f0b4c0b29f0 0x7f0b4c0b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:46.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.320+0000 7f0b68c00700 1 --2- 172.21.15.151:0/3148208832 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55957e8dcf30 0x55957e8d6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:46.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.320+0000 7f0b68c00700 1 -- 172.21.15.151:0/3148208832 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55957e8dc610 msgr2=0x55957e8dc9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:59:46.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.320+0000 7f0b68c00700 1 --2- 172.21.15.151:0/3148208832 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55957e8dc610 0x55957e8dc9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:46.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.320+0000 7f0b68c00700 1 -- 172.21.15.151:0/3148208832 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55957e7ed420 con 0x55957e8d7140 2024-05-21T21:59:46.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.321+0000 7f0b68c00700 1 --2- 172.21.15.151:0/3148208832 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55957e8d7140 0x55957e8d75c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0b64103f90 tx=0x7f0b500096a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a13ca9d826755304 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:59:46.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.321+0000 7f0b6a000700 1 --2- 172.21.15.151:0/2287313639 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f0b4c0b29f0 0x7f0b4c0b4eb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c00fb80 tx=0x7f0b5c004b40 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:59:46.324 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.321+0000 7f0b56a00700 1 -- 172.21.15.151:0/3148208832 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b50027070 con 0x55957e8d7140 2024-05-21T21:59:46.324 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.321+0000 7f0b56a00700 1 -- 172.21.15.151:0/3148208832 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b50003680 con 0x55957e8d7140 2024-05-21T21:59:46.324 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.321+0000 7f0b56a00700 1 -- 172.21.15.151:0/3148208832 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b5000f6c0 con 0x55957e8d7140 2024-05-21T21:59:46.324 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.322+0000 7f0b76966b80 1 -- 172.21.15.151:0/3148208832 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55957e8d7140 msgr2=0x55957e8d75c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:59:46.324 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.322+0000 7f0b76966b80 1 --2- 172.21.15.151:0/3148208832 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55957e8d7140 0x55957e8d75c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0b64103f90 tx=0x7f0b500096a0 comp rx=0 tx=0).stop 2024-05-21T21:59:46.324 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.322+0000 7f0b76966b80 1 -- 172.21.15.151:0/3148208832 shutdown_connections 2024-05-21T21:59:46.324 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.322+0000 7f0b76966b80 1 --2- 172.21.15.151:0/3148208832 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55957e8d7140 0x55957e8d75c0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:46.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.322+0000 7f0b76966b80 1 --2- 172.21.15.151:0/3148208832 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55957e8dc610 0x55957e8dc9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:46.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.322+0000 7f0b76966b80 1 --2- 172.21.15.151:0/3148208832 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55957e8dcf30 0x55957e8d6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:46.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.322+0000 7f0b76966b80 1 -- 172.21.15.151:0/3148208832 >> 172.21.15.151:0/3148208832 conn(0x55957e804be0 msgr2=0x55957e8d1690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:59:46.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.322+0000 7f0b76966b80 1 -- 172.21.15.151:0/3148208832 shutdown_connections 2024-05-21T21:59:46.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.322+0000 7f0b76966b80 1 -- 172.21.15.151:0/3148208832 wait complete. 2024-05-21T21:59:46.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.323+0000 7f0b76966b80 1 -- start start 2024-05-21T21:59:46.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.323+0000 7f0b76966b80 1 Processor -- start 2024-05-21T21:59:46.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.323+0000 7f0b76966b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55957e8d7140 0x55957e8e2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:46.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.323+0000 7f0b76966b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55957e8dc610 0x55957e8e0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:46.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.323+0000 7f0b76966b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55957e8dcf30 0x55957e8e0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:46.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.323+0000 7f0b76966b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55957e72afa0 con 0x55957e8dcf30 2024-05-21T21:59:46.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.323+0000 7f0b76966b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55957e7c5760 con 0x55957e8dc610 2024-05-21T21:59:46.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.323+0000 7f0b76966b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55957e8e3980 con 0x55957e8d7140 2024-05-21T21:59:46.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.323+0000 7f0b76966b80 10 client.0 _ll_register_callbacks cb 0x55957e7cec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:59:46.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.324+0000 7f0b69600700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55957e8d7140 0x55957e8e2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:46.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.324+0000 7f0b69600700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55957e8d7140 0x55957e8e2c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:41986/0 (socket says 172.21.15.151:41986) 2024-05-21T21:59:46.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.324+0000 7f0b69600700 1 -- 172.21.15.151:0/1135912959 learned_addr learned my addr 172.21.15.151:0/1135912959 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:59:46.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.324+0000 7f0b6a000700 1 --2- 172.21.15.151:0/1135912959 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55957e8dcf30 0x55957e8e0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:46.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.324+0000 7f0b68c00700 1 --2- 172.21.15.151:0/1135912959 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55957e8dc610 0x55957e8e0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:46.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.324+0000 7f0b69600700 1 -- 172.21.15.151:0/1135912959 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55957e8dc610 msgr2=0x55957e8e0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:59:46.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.324+0000 7f0b69600700 1 --2- 172.21.15.151:0/1135912959 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55957e8dc610 0x55957e8e0020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:46.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.324+0000 7f0b69600700 1 -- 172.21.15.151:0/1135912959 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55957e8dcf30 msgr2=0x55957e8e0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:59:46.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.324+0000 7f0b69600700 1 --2- 172.21.15.151:0/1135912959 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55957e8dcf30 0x55957e8e0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:46.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.324+0000 7f0b69600700 1 -- 172.21.15.151:0/1135912959 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55957e7748d0 con 0x55957e8d7140 2024-05-21T21:59:46.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.324+0000 7f0b68c00700 1 --2- 172.21.15.151:0/1135912959 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55957e8dc610 0x55957e8e0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:59:46.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.324+0000 7f0b6a000700 1 --2- 172.21.15.151:0/1135912959 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55957e8dcf30 0x55957e8e0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:59:46.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.324+0000 7f0b69600700 1 --2- 172.21.15.151:0/1135912959 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55957e8d7140 0x55957e8e2c30 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0b580a6c50 tx=0x7f0b5801dc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4f814fa42760d53 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:59:46.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.324+0000 7f0b4b400700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-21T21:59:46.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.325+0000 7f0b4b400700 1 -- 172.21.15.151:0/1135912959 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b580aa0e0 con 0x55957e8d7140 2024-05-21T21:59:46.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.325+0000 7f0b4b400700 1 -- 172.21.15.151:0/1135912959 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f0b580cc0c0 con 0x55957e8d7140 2024-05-21T21:59:46.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.325+0000 7f0b76966b80 1 -- 172.21.15.151:0/1135912959 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b50003680 con 0x55957e8d7140 2024-05-21T21:59:46.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.325+0000 7f0b4b400700 1 -- 172.21.15.151:0/1135912959 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0b580a99e0 con 0x55957e8d7140 2024-05-21T21:59:46.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.325+0000 7f0b76966b80 10 client.26155 fetch_fsmap learned FSMap version 29 2024-05-21T21:59:46.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.325+0000 7f0b76966b80 1 -- 172.21.15.151:0/1135912959 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55957e7f5060 con 0x55957e8d7140 2024-05-21T21:59:46.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.325+0000 7f0b4b400700 1 -- 172.21.15.151:0/1135912959 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b50003680 con 0x55957e8d7140 2024-05-21T21:59:46.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.325+0000 7f0b4b400700 1 -- 172.21.15.151:0/1135912959 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f0b580a9cd0 con 0x55957e8d7140 2024-05-21T21:59:46.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.326+0000 7f0b76966b80 10 client.26155 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T21:59:46.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.326+0000 7f0b76966b80 10 client.26155 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-21T21:59:46.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.326+0000 7f0b76966b80 20 client.26155 populate_metadata read hostname 'smithi151' 2024-05-21T21:59:46.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.326+0000 7f0b76966b80 1 --2- 172.21.15.151:0/1135912959 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x55957e8f0430 0x55957e8f2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:46.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.326+0000 7f0b76966b80 4 client.26155 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T21:59:46.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.326+0000 7f0b76966b80 1 -- 172.21.15.151:0/1135912959 --> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55957e836d90 con 0x55957e8f0430 2024-05-21T21:59:46.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.326+0000 7f0b68c00700 1 --2- 172.21.15.151:0/1135912959 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x55957e8f0430 0x55957e8f2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:46.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.327+0000 7f0b68c00700 1 --2- 172.21.15.151:0/1135912959 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x55957e8f0430 0x55957e8f2820 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e7ff3772a9295ec server_cookie=aafd1c50f2aa5eff in_seq=0 out_seq=0 2024-05-21T21:59:46.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.327+0000 7f0b4b400700 10 client.26155 ms_handle_connect on v2:172.21.15.184:6836/2988194888 2024-05-21T21:59:46.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.328+0000 7f0b4b400700 1 -- 172.21.15.151:0/1135912959 <== mds.3 v2:172.21.15.184:6836/2988194888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55957e836d90 con 0x55957e8f0430 2024-05-21T21:59:46.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.328+0000 7f0b4b400700 10 client.26155 handle_command_reply: tid=0 2024-05-21T21:59:46.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.328+0000 7f0b76966b80 1 client.26155 shutdown 2024-05-21T21:59:46.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.328+0000 7f0b76966b80 1 -- 172.21.15.151:0/1135912959 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55957e8d7140 msgr2=0x55957e8e2c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:59:46.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.328+0000 7f0b76966b80 1 --2- 172.21.15.151:0/1135912959 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55957e8d7140 0x55957e8e2c30 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0b580a6c50 tx=0x7f0b5801dc40 comp rx=0 tx=0).stop 2024-05-21T21:59:46.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.328+0000 7f0b76966b80 1 -- 172.21.15.151:0/1135912959 shutdown_connections 2024-05-21T21:59:46.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.328+0000 7f0b76966b80 1 --2- 172.21.15.151:0/1135912959 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x55957e8f0430 0x55957e8f2820 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:46.332 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.328+0000 7f0b76966b80 1 --2- 172.21.15.151:0/1135912959 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55957e8dcf30 0x55957e8e0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:46.332 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.328+0000 7f0b76966b80 1 --2- 172.21.15.151:0/1135912959 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55957e8d7140 0x55957e8e2c30 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:46.332 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.328+0000 7f0b76966b80 1 --2- 172.21.15.151:0/1135912959 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55957e8dc610 0x55957e8e0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:46.332 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.328+0000 7f0b76966b80 1 -- 172.21.15.151:0/1135912959 >> 172.21.15.151:0/1135912959 conn(0x55957e804be0 msgr2=0x55957e8d0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:59:46.332 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.328+0000 7f0b4b400700 0 client.26155 ms_handle_reset on v2:172.21.15.184:6836/2988194888 2024-05-21T21:59:46.332 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.328+0000 7f0b76966b80 1 -- 172.21.15.151:0/1135912959 shutdown_connections 2024-05-21T21:59:46.332 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.328+0000 7f0b76966b80 1 -- 172.21.15.151:0/1135912959 wait complete. 2024-05-21T21:59:46.332 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.328+0000 7f0b76966b80 20 client.26155 trim_cache size 0 max 16384 2024-05-21T21:59:46.346 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.345+0000 7f0b76966b80 1 Processor -- start 2024-05-21T21:59:46.346 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.345+0000 7f0b76966b80 1 -- start start 2024-05-21T21:59:46.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.345+0000 7f0b76966b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55957e8d7140 0x55957e8d6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:46.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.346+0000 7f0b76966b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55957e8dc610 0x55957e8ddba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:46.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.346+0000 7f0b76966b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55957e8dcf30 0x55957e8d14f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:46.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.346+0000 7f0b76966b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0b641affa0 con 0x55957e8dc610 2024-05-21T21:59:46.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.346+0000 7f0b76966b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0b641b1ae0 con 0x55957e8dcf30 2024-05-21T21:59:46.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.346+0000 7f0b76966b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0b641b0670 con 0x55957e8d7140 2024-05-21T21:59:46.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.346+0000 7f0b68c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55957e8dc610 0x55957e8ddba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:46.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.346+0000 7f0b68c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55957e8dc610 0x55957e8ddba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:56488/0 (socket says 172.21.15.151:56488) 2024-05-21T21:59:46.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.346+0000 7f0b68c00700 1 -- 172.21.15.151:0/1797608400 learned_addr learned my addr 172.21.15.151:0/1797608400 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:59:46.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.346+0000 7f0b69600700 1 --2- 172.21.15.151:0/1797608400 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55957e8d7140 0x55957e8d6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:46.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.346+0000 7f0b6a000700 1 --2- 172.21.15.151:0/1797608400 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55957e8dcf30 0x55957e8d14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:46.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.346+0000 7f0b68c00700 1 -- 172.21.15.151:0/1797608400 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55957e8d7140 msgr2=0x55957e8d6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:59:46.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.346+0000 7f0b68c00700 1 --2- 172.21.15.151:0/1797608400 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55957e8d7140 0x55957e8d6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:46.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.346+0000 7f0b68c00700 1 -- 172.21.15.151:0/1797608400 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55957e8dcf30 msgr2=0x55957e8d14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:59:46.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.346+0000 7f0b68c00700 1 --2- 172.21.15.151:0/1797608400 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55957e8dcf30 0x55957e8d14f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:46.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.346+0000 7f0b68c00700 1 -- 172.21.15.151:0/1797608400 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55957e8de590 con 0x55957e8dc610 2024-05-21T21:59:46.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.346+0000 7f0b69600700 1 --2- 172.21.15.151:0/1797608400 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55957e8d7140 0x55957e8d6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:59:46.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.347+0000 7f0b68c00700 1 --2- 172.21.15.151:0/1797608400 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55957e8dc610 0x55957e8ddba0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f0b50025040 tx=0x7f0b50005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b402dafb7eea27b server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:59:46.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.347+0000 7f0b56000700 1 -- 172.21.15.151:0/1797608400 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b50033040 con 0x55957e8dc610 2024-05-21T21:59:46.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.347+0000 7f0b56000700 1 -- 172.21.15.151:0/1797608400 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b50003b40 con 0x55957e8dc610 2024-05-21T21:59:46.350 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.347+0000 7f0b56000700 1 -- 172.21.15.151:0/1797608400 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b50027040 con 0x55957e8dc610 2024-05-21T21:59:46.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.348+0000 7f0b76966b80 1 -- 172.21.15.151:0/1797608400 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55957e8dc610 msgr2=0x55957e8ddba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:59:46.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.348+0000 7f0b76966b80 1 --2- 172.21.15.151:0/1797608400 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55957e8dc610 0x55957e8ddba0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f0b50025040 tx=0x7f0b50005fb0 comp rx=0 tx=0).stop 2024-05-21T21:59:46.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.348+0000 7f0b76966b80 1 -- 172.21.15.151:0/1797608400 shutdown_connections 2024-05-21T21:59:46.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.348+0000 7f0b76966b80 1 --2- 172.21.15.151:0/1797608400 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55957e8dcf30 0x55957e8d14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:46.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.348+0000 7f0b76966b80 1 --2- 172.21.15.151:0/1797608400 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55957e8d7140 0x55957e8d6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:46.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.348+0000 7f0b76966b80 1 --2- 172.21.15.151:0/1797608400 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55957e8dc610 0x55957e8ddba0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:46.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.348+0000 7f0b76966b80 1 -- 172.21.15.151:0/1797608400 >> 172.21.15.151:0/1797608400 conn(0x55957e804be0 msgr2=0x55957e8e1680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:59:46.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.348+0000 7f0b76966b80 1 -- 172.21.15.151:0/1797608400 shutdown_connections 2024-05-21T21:59:46.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.348+0000 7f0b76966b80 1 -- 172.21.15.151:0/1797608400 wait complete. 2024-05-21T21:59:46.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.349+0000 7f0b76966b80 1 -- start start 2024-05-21T21:59:46.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.349+0000 7f0b76966b80 1 Processor -- start 2024-05-21T21:59:46.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.349+0000 7f0b76966b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55957e8d7140 0x55957e8e92a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:46.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.349+0000 7f0b76966b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55957e8dc610 0x55957e8f7820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:46.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.349+0000 7f0b76966b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55957e8dcf30 0x55957e8f7d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:46.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.349+0000 7f0b76966b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0b641b1410 con 0x55957e8dcf30 2024-05-21T21:59:46.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.349+0000 7f0b76966b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0b641b0d40 con 0x55957e8dc610 2024-05-21T21:59:46.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.349+0000 7f0b76966b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0b641adf40 con 0x55957e8d7140 2024-05-21T21:59:46.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.350+0000 7f0b76966b80 10 client.0 _ll_register_callbacks cb 0x55957e7cec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:59:46.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.350+0000 7f0b6a000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55957e8dcf30 0x55957e8f7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:46.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.350+0000 7f0b69600700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55957e8d7140 0x55957e8e92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:46.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.350+0000 7f0b69600700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55957e8d7140 0x55957e8e92a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:42000/0 (socket says 172.21.15.151:42000) 2024-05-21T21:59:46.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.350+0000 7f0b69600700 1 -- 172.21.15.151:0/1993763193 learned_addr learned my addr 172.21.15.151:0/1993763193 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:59:46.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.350+0000 7f0b6a000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55957e8dcf30 0x55957e8f7d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:56504/0 (socket says 172.21.15.151:56504) 2024-05-21T21:59:46.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.350+0000 7f0b68c00700 1 --2- 172.21.15.151:0/1993763193 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55957e8dc610 0x55957e8f7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:46.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.350+0000 7f0b69600700 1 -- 172.21.15.151:0/1993763193 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55957e8dc610 msgr2=0x55957e8f7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:59:46.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.350+0000 7f0b69600700 1 --2- 172.21.15.151:0/1993763193 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55957e8dc610 0x55957e8f7820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:46.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.350+0000 7f0b69600700 1 -- 172.21.15.151:0/1993763193 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55957e8dcf30 msgr2=0x55957e8f7d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:59:46.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.350+0000 7f0b69600700 1 --2- 172.21.15.151:0/1993763193 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55957e8dcf30 0x55957e8f7d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:46.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.350+0000 7f0b69600700 1 -- 172.21.15.151:0/1993763193 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55957e8f8560 con 0x55957e8d7140 2024-05-21T21:59:46.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.351+0000 7f0b69600700 1 --2- 172.21.15.151:0/1993763193 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55957e8d7140 0x55957e8e92a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0b580a5a60 tx=0x7f0b5801df90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ebb6aeb09e369470 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:59:46.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.351+0000 7f0b4b400700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-21T21:59:46.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.351+0000 7f0b4b400700 1 -- 172.21.15.151:0/1993763193 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b580aa0e0 con 0x55957e8d7140 2024-05-21T21:59:46.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.351+0000 7f0b76966b80 1 -- 172.21.15.151:0/1993763193 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b50003b40 con 0x55957e8d7140 2024-05-21T21:59:46.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.351+0000 7f0b4b400700 1 -- 172.21.15.151:0/1993763193 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f0b580cc2c0 con 0x55957e8d7140 2024-05-21T21:59:46.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.351+0000 7f0b4b400700 1 -- 172.21.15.151:0/1993763193 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0b580add90 con 0x55957e8d7140 2024-05-21T21:59:46.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.351+0000 7f0b76966b80 10 client.26161 fetch_fsmap learned FSMap version 29 2024-05-21T21:59:46.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.351+0000 7f0b76966b80 1 -- 172.21.15.151:0/1993763193 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55957e95aec0 con 0x55957e8d7140 2024-05-21T21:59:46.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.351+0000 7f0b4b400700 1 -- 172.21.15.151:0/1993763193 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b50003b40 con 0x55957e8d7140 2024-05-21T21:59:46.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.351+0000 7f0b4b400700 1 -- 172.21.15.151:0/1993763193 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f0b580a5350 con 0x55957e8d7140 2024-05-21T21:59:46.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.352+0000 7f0b76966b80 10 client.26161 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T21:59:46.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.352+0000 7f0b76966b80 10 client.26161 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-21T21:59:46.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.352+0000 7f0b76966b80 20 client.26161 populate_metadata read hostname 'smithi151' 2024-05-21T21:59:46.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.352+0000 7f0b76966b80 1 --2- 172.21.15.151:0/1993763193 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x55957e95b470 0x55957e95d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:46.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.352+0000 7f0b76966b80 4 client.26161 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T21:59:46.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.352+0000 7f0b76966b80 1 -- 172.21.15.151:0/1993763193 --> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55957e830570 con 0x55957e95b470 2024-05-21T21:59:46.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.352+0000 7f0b68c00700 1 --2- 172.21.15.151:0/1993763193 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x55957e95b470 0x55957e95d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:46.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.353+0000 7f0b68c00700 1 --2- 172.21.15.151:0/1993763193 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x55957e95b470 0x55957e95d890 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d7267e71e3d85022 server_cookie=a18c520d008f11a8 in_seq=0 out_seq=0 2024-05-21T21:59:46.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.353+0000 7f0b4b400700 10 client.26161 ms_handle_connect on v2:172.21.15.184:6836/2988194888 2024-05-21T21:59:46.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.353+0000 7f0b4b400700 1 -- 172.21.15.151:0/1993763193 <== mds.3 v2:172.21.15.184:6836/2988194888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55957e830570 con 0x55957e95b470 2024-05-21T21:59:46.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.353+0000 7f0b4b400700 10 client.26161 handle_command_reply: tid=0 2024-05-21T21:59:46.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.353+0000 7f0b76966b80 1 client.26161 shutdown 2024-05-21T21:59:46.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.353+0000 7f0b76966b80 1 -- 172.21.15.151:0/1993763193 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55957e8d7140 msgr2=0x55957e8e92a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:59:46.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.353+0000 7f0b76966b80 1 --2- 172.21.15.151:0/1993763193 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55957e8d7140 0x55957e8e92a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0b580a5a60 tx=0x7f0b5801df90 comp rx=0 tx=0).stop 2024-05-21T21:59:46.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.353+0000 7f0b76966b80 1 -- 172.21.15.151:0/1993763193 shutdown_connections 2024-05-21T21:59:46.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.353+0000 7f0b76966b80 1 --2- 172.21.15.151:0/1993763193 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x55957e95b470 0x55957e95d890 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:46.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.353+0000 7f0b76966b80 1 --2- 172.21.15.151:0/1993763193 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55957e8dcf30 0x55957e8f7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:46.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.353+0000 7f0b76966b80 1 --2- 172.21.15.151:0/1993763193 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55957e8d7140 0x55957e8e92a0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:46.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.353+0000 7f0b4b400700 0 client.26161 ms_handle_reset on v2:172.21.15.184:6836/2988194888 2024-05-21T21:59:46.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.353+0000 7f0b76966b80 1 --2- 172.21.15.151:0/1993763193 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55957e8dc610 0x55957e8f7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:46.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.353+0000 7f0b76966b80 1 -- 172.21.15.151:0/1993763193 >> 172.21.15.151:0/1993763193 conn(0x55957e804be0 msgr2=0x55957e8dda20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:59:46.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.353+0000 7f0b76966b80 1 -- 172.21.15.151:0/1993763193 shutdown_connections 2024-05-21T21:59:46.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.353+0000 7f0b76966b80 1 -- 172.21.15.151:0/1993763193 wait complete. 2024-05-21T21:59:46.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.354+0000 7f0b76966b80 20 client.26161 trim_cache size 0 max 16384 2024-05-21T21:59:46.359 INFO:teuthology.orchestra.run.smithi151.stdout:[] 2024-05-21T21:59:46.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.354+0000 7f0b6c000700 1 -- 172.21.15.151:0/2287313639 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f0b4c0b29f0 msgr2=0x7f0b4c0b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:59:46.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.354+0000 7f0b6c000700 1 --2- 172.21.15.151:0/2287313639 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f0b4c0b29f0 0x7f0b4c0b4eb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c00fb80 tx=0x7f0b5c004b40 comp rx=0 tx=0).stop 2024-05-21T21:59:46.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.354+0000 7f0b6c000700 1 -- 172.21.15.151:0/2287313639 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0b6406bab0 msgr2=0x7f0b64102500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:59:46.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.354+0000 7f0b6c000700 1 --2- 172.21.15.151:0/2287313639 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0b6406bab0 0x7f0b64102500 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f0b5800e9f0 tx=0x7f0b5800ed00 comp rx=0 tx=0).stop 2024-05-21T21:59:46.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.354+0000 7f0b6c000700 1 -- 172.21.15.151:0/2287313639 shutdown_connections 2024-05-21T21:59:46.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.354+0000 7f0b6c000700 1 --2- 172.21.15.151:0/2287313639 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f0b4c0b29f0 0x7f0b4c0b4eb0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:46.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.354+0000 7f0b6c000700 1 --2- 172.21.15.151:0/2287313639 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0b6406bab0 0x7f0b64102500 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:46.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.355+0000 7f0b6c000700 1 --2- 172.21.15.151:0/2287313639 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b641095a0 0x7f0b64102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:46.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.355+0000 7f0b6c000700 1 --2- 172.21.15.151:0/2287313639 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0b6406c470 0x7f0b64102a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:46.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.355+0000 7f0b6c000700 1 -- 172.21.15.151:0/2287313639 >> 172.21.15.151:0/2287313639 conn(0x7f0b640793b0 msgr2=0x7f0b641011b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:59:46.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.355+0000 7f0b6c000700 1 -- 172.21.15.151:0/2287313639 shutdown_connections 2024-05-21T21:59:46.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:46.355+0000 7f0b6c000700 1 -- 172.21.15.151:0/2287313639 wait complete. 2024-05-21T21:59:47.161 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-21T21:59:47.616 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:59:47.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:59:47 smithi151 ceph-mon[31623]: pgmap v128: 97 pgs: 97 active+clean; 515 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 21 MiB/s wr, 14 op/s 2024-05-21T21:59:47.735 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:59:47 smithi184 ceph-mon[36942]: pgmap v128: 97 pgs: 97 active+clean; 515 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 21 MiB/s wr, 14 op/s 2024-05-21T21:59:47.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:59:47 smithi164 ceph-mon[36269]: pgmap v128: 97 pgs: 97 active+clean; 515 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 21 MiB/s wr, 14 op/s 2024-05-21T21:59:49.556 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:59:49 smithi151 ceph-mon[31623]: pgmap v129: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 21 MiB/s wr, 24 op/s 2024-05-21T21:59:49.733 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.730+0000 7f05f6e00700 1 -- 172.21.15.151:0/2382716860 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f05f006b250 msgr2=0x7f05f006b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:59:49.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.730+0000 7f05f6e00700 1 --2- 172.21.15.151:0/2382716860 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f05f006b250 0x7f05f006b630 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f05e0009990 tx=0x7f05e0009ca0 comp rx=0 tx=0).stop 2024-05-21T21:59:49.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.730+0000 7f05f6e00700 1 -- 172.21.15.151:0/2382716860 shutdown_connections 2024-05-21T21:59:49.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.730+0000 7f05f6e00700 1 --2- 172.21.15.151:0/2382716860 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05f010fff0 0x7f05f01123e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:49.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.730+0000 7f05f6e00700 1 --2- 172.21.15.151:0/2382716860 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f05f006b250 0x7f05f006b630 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:49.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.730+0000 7f05f6e00700 1 --2- 172.21.15.151:0/2382716860 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05f006bb70 0x7f05f010fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:49.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.730+0000 7f05f6e00700 1 -- 172.21.15.151:0/2382716860 >> 172.21.15.151:0/2382716860 conn(0x7f05f00793f0 msgr2=0x7f05f0079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:59:49.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.730+0000 7f05f6e00700 1 -- 172.21.15.151:0/2382716860 shutdown_connections 2024-05-21T21:59:49.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.731+0000 7f05f6e00700 1 -- 172.21.15.151:0/2382716860 wait complete. 2024-05-21T21:59:49.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.731+0000 7f05f6e00700 1 Processor -- start 2024-05-21T21:59:49.735 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.732+0000 7f05f6e00700 1 -- start start 2024-05-21T21:59:49.735 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.732+0000 7f05f6e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05f006b250 0x7f05f019beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:49.735 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.732+0000 7f05f6e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f05f006bb70 0x7f05f019c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:49.735 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.732+0000 7f05f6e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05f010fff0 0x7f05f01a0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:49.735 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.732+0000 7f05f6e00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f05f0115960 con 0x7f05f006bb70 2024-05-21T21:59:49.735 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.732+0000 7f05f6e00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f05f01157e0 con 0x7f05f010fff0 2024-05-21T21:59:49.735 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.732+0000 7f05f6e00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f05f0115ae0 con 0x7f05f006b250 2024-05-21T21:59:49.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:59:49 smithi184 ceph-mon[36942]: pgmap v129: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 21 MiB/s wr, 24 op/s 2024-05-21T21:59:49.737 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.732+0000 7f05efe00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05f006b250 0x7f05f019beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:49.737 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.732+0000 7f05ef400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f05f006bb70 0x7f05f019c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:49.737 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.733+0000 7f05efe00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05f006b250 0x7f05f019beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:42012/0 (socket says 172.21.15.151:42012) 2024-05-21T21:59:49.737 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.733+0000 7f05efe00700 1 -- 172.21.15.151:0/4074148342 learned_addr learned my addr 172.21.15.151:0/4074148342 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:59:49.737 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.733+0000 7f05f4e00700 1 --2- 172.21.15.151:0/4074148342 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05f010fff0 0x7f05f01a0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:49.737 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.733+0000 7f05efe00700 1 -- 172.21.15.151:0/4074148342 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05f010fff0 msgr2=0x7f05f01a0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:59:49.737 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.733+0000 7f05efe00700 1 --2- 172.21.15.151:0/4074148342 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05f010fff0 0x7f05f01a0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:49.737 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.733+0000 7f05efe00700 1 -- 172.21.15.151:0/4074148342 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f05f006bb70 msgr2=0x7f05f019c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:59:49.738 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.733+0000 7f05efe00700 1 --2- 172.21.15.151:0/4074148342 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f05f006bb70 0x7f05f019c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:49.738 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.733+0000 7f05efe00700 1 -- 172.21.15.151:0/4074148342 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05f01a0e70 con 0x7f05f006b250 2024-05-21T21:59:49.738 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.733+0000 7f05efe00700 1 --2- 172.21.15.151:0/4074148342 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05f006b250 0x7f05f019beb0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f05e0005910 tx=0x7f05e000b670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:59:49.738 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.734+0000 7f05ecc00700 1 -- 172.21.15.151:0/4074148342 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e0027070 con 0x7f05f006b250 2024-05-21T21:59:49.738 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.734+0000 7f05ecc00700 1 -- 172.21.15.151:0/4074148342 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05e0022470 con 0x7f05f006b250 2024-05-21T21:59:49.738 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.734+0000 7f05f6e00700 1 -- 172.21.15.151:0/4074148342 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05f01a1100 con 0x7f05f006b250 2024-05-21T21:59:49.738 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.734+0000 7f05ecc00700 1 -- 172.21.15.151:0/4074148342 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e0017650 con 0x7f05f006b250 2024-05-21T21:59:49.738 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.734+0000 7f05f6e00700 1 -- 172.21.15.151:0/4074148342 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05f01a89e0 con 0x7f05f006b250 2024-05-21T21:59:49.739 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.737+0000 7f05ecc00700 1 -- 172.21.15.151:0/4074148342 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7f05e002a070 con 0x7f05f006b250 2024-05-21T21:59:49.740 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.738+0000 7f0601735b80 1 Processor -- start 2024-05-21T21:59:49.740 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.739+0000 7f0601735b80 1 -- start start 2024-05-21T21:59:49.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.739+0000 7f0601735b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5592a980d610 0x5592a980d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:49.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.739+0000 7f0601735b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5592a980df30 0x5592a9807b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:49.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.739+0000 7f0601735b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5592a9808140 0x5592a98085c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:49.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.739+0000 7f0601735b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5592a9765e40 con 0x5592a9808140 2024-05-21T21:59:49.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.739+0000 7f0601735b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5592a97559e0 con 0x5592a980d610 2024-05-21T21:59:49.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.739+0000 7f0601735b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5592a9759a20 con 0x5592a980df30 2024-05-21T21:59:49.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.739+0000 7f05ecc00700 1 --2- 172.21.15.151:0/4074148342 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f05d40b29f0 0x7f05d40b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:49.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.740+0000 7f05ef400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5592a9808140 0x5592a98085c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:49.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.740+0000 7f05efe00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5592a980df30 0x5592a9807b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:49.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.740+0000 7f05efe00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5592a980df30 0x5592a9807b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:42014/0 (socket says 172.21.15.151:42014) 2024-05-21T21:59:49.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.740+0000 7f05efe00700 1 -- 172.21.15.151:0/1526015971 learned_addr learned my addr 172.21.15.151:0/1526015971 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:59:49.742 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.740+0000 7f05ef400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5592a9808140 0x5592a98085c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:56534/0 (socket says 172.21.15.151:56534) 2024-05-21T21:59:49.742 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.740+0000 7f05f4e00700 1 --2- 172.21.15.151:0/1526015971 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5592a980d610 0x5592a980d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:49.742 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.740+0000 7f05ecc00700 1 -- 172.21.15.151:0/4074148342 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f05e00e3400 con 0x7f05f006b250 2024-05-21T21:59:49.743 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.740+0000 7f05f4e00700 1 --2- 172.21.15.151:0/4074148342 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f05d40b29f0 0x7f05d40b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:49.743 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.740+0000 7f05ef400700 1 -- 172.21.15.151:0/1526015971 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5592a980df30 msgr2=0x5592a9807b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:59:49.743 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.740+0000 7f05ef400700 1 --2- 172.21.15.151:0/1526015971 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5592a980df30 0x5592a9807b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:49.743 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.740+0000 7f05ef400700 1 -- 172.21.15.151:0/1526015971 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5592a980d610 msgr2=0x5592a980d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:59:49.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.740+0000 7f05ef400700 1 --2- 172.21.15.151:0/1526015971 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5592a980d610 0x5592a980d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:49.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.740+0000 7f05ef400700 1 -- 172.21.15.151:0/1526015971 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592a971e420 con 0x5592a9808140 2024-05-21T21:59:49.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.740+0000 7f05f4e00700 1 --2- 172.21.15.151:0/1526015971 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5592a980d610 0x5592a980d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:59:49.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.741+0000 7f05ef400700 1 --2- 172.21.15.151:0/1526015971 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5592a9808140 0x5592a98085c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f05dc009c80 tx=0x7f05dc009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=327e11509b4de9dc server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:59:49.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.741+0000 7f05f4e00700 1 --2- 172.21.15.151:0/4074148342 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f05d40b29f0 0x7f05d40b4eb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f05e8009fd0 tx=0x7f05e800b040 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:59:49.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.741+0000 7f05e5600700 1 -- 172.21.15.151:0/1526015971 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05dc004310 con 0x5592a9808140 2024-05-21T21:59:49.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.741+0000 7f05e5600700 1 -- 172.21.15.151:0/1526015971 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05dc011de0 con 0x5592a9808140 2024-05-21T21:59:49.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.741+0000 7f05e5600700 1 -- 172.21.15.151:0/1526015971 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05dc005280 con 0x5592a9808140 2024-05-21T21:59:49.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.742+0000 7f0601735b80 1 -- 172.21.15.151:0/1526015971 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5592a9808140 msgr2=0x5592a98085c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:59:49.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.742+0000 7f0601735b80 1 --2- 172.21.15.151:0/1526015971 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5592a9808140 0x5592a98085c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f05dc009c80 tx=0x7f05dc009f90 comp rx=0 tx=0).stop 2024-05-21T21:59:49.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.742+0000 7f0601735b80 1 -- 172.21.15.151:0/1526015971 shutdown_connections 2024-05-21T21:59:49.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.742+0000 7f0601735b80 1 --2- 172.21.15.151:0/1526015971 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5592a980d610 0x5592a980d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:49.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.742+0000 7f0601735b80 1 --2- 172.21.15.151:0/1526015971 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5592a9808140 0x5592a98085c0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:49.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.742+0000 7f0601735b80 1 --2- 172.21.15.151:0/1526015971 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5592a980df30 0x5592a9807b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:49.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.742+0000 7f0601735b80 1 -- 172.21.15.151:0/1526015971 >> 172.21.15.151:0/1526015971 conn(0x5592a9735be0 msgr2=0x5592a9802690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:59:49.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.742+0000 7f0601735b80 1 -- 172.21.15.151:0/1526015971 shutdown_connections 2024-05-21T21:59:49.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.742+0000 7f0601735b80 1 -- 172.21.15.151:0/1526015971 wait complete. 2024-05-21T21:59:49.747 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.743+0000 7f0601735b80 1 -- start start 2024-05-21T21:59:49.747 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.743+0000 7f0601735b80 1 Processor -- start 2024-05-21T21:59:49.747 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.743+0000 7f0601735b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5592a9808140 0x5592a9813be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:49.747 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.744+0000 7f0601735b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5592a980d610 0x5592a9810f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:49.747 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.744+0000 7f0601735b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5592a980df30 0x5592a98114c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:49.747 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.744+0000 7f0601735b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5592a965bfa0 con 0x5592a980df30 2024-05-21T21:59:49.748 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.744+0000 7f0601735b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5592a96f6760 con 0x5592a980d610 2024-05-21T21:59:49.748 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.744+0000 7f0601735b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5592a98149e0 con 0x5592a9808140 2024-05-21T21:59:49.748 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.744+0000 7f0601735b80 10 client.0 _ll_register_callbacks cb 0x5592a96ffc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:59:49.748 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.744+0000 7f05ef400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5592a980d610 0x5592a9810f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:49.748 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.744+0000 7f05efe00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5592a9808140 0x5592a9813be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:49.748 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.744+0000 7f05f4e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5592a980df30 0x5592a98114c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:49.748 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.744+0000 7f05efe00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5592a9808140 0x5592a9813be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:42028/0 (socket says 172.21.15.151:42028) 2024-05-21T21:59:49.748 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.744+0000 7f05ef400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5592a980d610 0x5592a9810f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:35378/0 (socket says 172.21.15.151:35378) 2024-05-21T21:59:49.748 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.744+0000 7f05efe00700 1 -- 172.21.15.151:0/4146814206 learned_addr learned my addr 172.21.15.151:0/4146814206 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:59:49.748 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.744+0000 7f05efe00700 1 -- 172.21.15.151:0/4146814206 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5592a980d610 msgr2=0x5592a9810f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:59:49.748 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.744+0000 7f05efe00700 1 --2- 172.21.15.151:0/4146814206 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5592a980d610 0x5592a9810f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:49.748 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.744+0000 7f05efe00700 1 -- 172.21.15.151:0/4146814206 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5592a980df30 msgr2=0x5592a98114c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:59:49.748 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.744+0000 7f05efe00700 1 --2- 172.21.15.151:0/4146814206 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5592a980df30 0x5592a98114c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:49.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.744+0000 7f05efe00700 1 -- 172.21.15.151:0/4146814206 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592a96a58d0 con 0x5592a9808140 2024-05-21T21:59:49.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.745+0000 7f05efe00700 1 --2- 172.21.15.151:0/4146814206 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5592a9808140 0x5592a9813be0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f05e00e36a0 tx=0x7f05e0022cc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a7f8ae7ccf4f706 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:59:49.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.745+0000 7f05da000700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-21T21:59:49.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.745+0000 7f05da000700 1 -- 172.21.15.151:0/4146814206 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e00aa4f0 con 0x5592a9808140 2024-05-21T21:59:49.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.745+0000 7f05da000700 1 -- 172.21.15.151:0/4146814206 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f05e00aab90 con 0x5592a9808140 2024-05-21T21:59:49.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.745+0000 7f0601735b80 1 -- 172.21.15.151:0/4146814206 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05dc011de0 con 0x5592a9808140 2024-05-21T21:59:49.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.745+0000 7f05da000700 1 -- 172.21.15.151:0/4146814206 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f05e00a9e90 con 0x5592a9808140 2024-05-21T21:59:49.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.745+0000 7f0601735b80 10 client.26173 fetch_fsmap learned FSMap version 29 2024-05-21T21:59:49.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.745+0000 7f0601735b80 1 -- 172.21.15.151:0/4146814206 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5592a9726060 con 0x5592a9808140 2024-05-21T21:59:49.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.745+0000 7f05da000700 1 -- 172.21.15.151:0/4146814206 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05dc011de0 con 0x5592a9808140 2024-05-21T21:59:49.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.746+0000 7f05da000700 1 -- 172.21.15.151:0/4146814206 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f05e00b3060 con 0x5592a9808140 2024-05-21T21:59:49.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.746+0000 7f0601735b80 10 client.26173 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T21:59:49.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.746+0000 7f0601735b80 10 client.26173 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-05-21T21:59:49.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.746+0000 7f0601735b80 20 client.26173 populate_metadata read hostname 'smithi151' 2024-05-21T21:59:49.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.746+0000 7f0601735b80 1 --2- 172.21.15.151:0/4146814206 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x5592a9821320 0x5592a9823750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:49.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.746+0000 7f0601735b80 4 client.26173 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T21:59:49.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.746+0000 7f0601735b80 1 -- 172.21.15.151:0/4146814206 --> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5592a9767d90 con 0x5592a9821320 2024-05-21T21:59:49.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.746+0000 7f05ef400700 1 --2- 172.21.15.151:0/4146814206 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x5592a9821320 0x5592a9823750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:49.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.747+0000 7f05ef400700 1 --2- 172.21.15.151:0/4146814206 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x5592a9821320 0x5592a9823750 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6a3436a8eb11fd09 server_cookie=17f4f946c85fbdc0 in_seq=0 out_seq=0 2024-05-21T21:59:49.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.747+0000 7f05da000700 10 client.26173 ms_handle_connect on v2:172.21.15.164:6836/1554609572 2024-05-21T21:59:49.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.748+0000 7f05da000700 1 -- 172.21.15.151:0/4146814206 <== mds.2 v2:172.21.15.164:6836/1554609572 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5592a9767d90 con 0x5592a9821320 2024-05-21T21:59:49.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.748+0000 7f05da000700 10 client.26173 handle_command_reply: tid=0 2024-05-21T21:59:49.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.748+0000 7f0601735b80 1 client.26173 shutdown 2024-05-21T21:59:49.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.748+0000 7f0601735b80 1 -- 172.21.15.151:0/4146814206 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5592a9808140 msgr2=0x5592a9813be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:59:49.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.748+0000 7f0601735b80 1 --2- 172.21.15.151:0/4146814206 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5592a9808140 0x5592a9813be0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f05e00e36a0 tx=0x7f05e0022cc0 comp rx=0 tx=0).stop 2024-05-21T21:59:49.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.748+0000 7f0601735b80 1 -- 172.21.15.151:0/4146814206 shutdown_connections 2024-05-21T21:59:49.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.748+0000 7f0601735b80 1 --2- 172.21.15.151:0/4146814206 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x5592a9821320 0x5592a9823750 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:49.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.748+0000 7f0601735b80 1 --2- 172.21.15.151:0/4146814206 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5592a980df30 0x5592a98114c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:49.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.748+0000 7f0601735b80 1 --2- 172.21.15.151:0/4146814206 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5592a9808140 0x5592a9813be0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:49.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.748+0000 7f0601735b80 1 --2- 172.21.15.151:0/4146814206 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5592a980d610 0x5592a9810f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:49.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.748+0000 7f05da000700 0 client.26173 ms_handle_reset on v2:172.21.15.164:6836/1554609572 2024-05-21T21:59:49.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.748+0000 7f0601735b80 1 -- 172.21.15.151:0/4146814206 >> 172.21.15.151:0/4146814206 conn(0x5592a9735be0 msgr2=0x5592a9801d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:59:49.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.748+0000 7f0601735b80 1 -- 172.21.15.151:0/4146814206 shutdown_connections 2024-05-21T21:59:49.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.748+0000 7f0601735b80 1 -- 172.21.15.151:0/4146814206 wait complete. 2024-05-21T21:59:49.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.749+0000 7f0601735b80 20 client.26173 trim_cache size 0 max 16384 2024-05-21T21:59:49.766 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.765+0000 7f0601735b80 1 Processor -- start 2024-05-21T21:59:49.766 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.765+0000 7f0601735b80 1 -- start start 2024-05-21T21:59:49.767 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.765+0000 7f0601735b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5592a9808140 0x5592a9807b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:49.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.766+0000 7f0601735b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5592a980d610 0x5592a980eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:49.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.766+0000 7f0601735b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5592a980df30 0x5592a98113d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:49.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.766+0000 7f0601735b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f05f01aff10 con 0x5592a980df30 2024-05-21T21:59:49.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.766+0000 7f0601735b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f05f01b1a50 con 0x5592a980d610 2024-05-21T21:59:49.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.766+0000 7f0601735b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f05f01b05e0 con 0x5592a9808140 2024-05-21T21:59:49.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.766+0000 7f05f4e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5592a980df30 0x5592a98113d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:49.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.766+0000 7f05efe00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5592a9808140 0x5592a9807b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:49.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.766+0000 7f05f4e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5592a980df30 0x5592a98113d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:56550/0 (socket says 172.21.15.151:56550) 2024-05-21T21:59:49.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.766+0000 7f05f4e00700 1 -- 172.21.15.151:0/679803632 learned_addr learned my addr 172.21.15.151:0/679803632 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:59:49.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.766+0000 7f05efe00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5592a9808140 0x5592a9807b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:42032/0 (socket says 172.21.15.151:42032) 2024-05-21T21:59:49.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.766+0000 7f05ef400700 1 --2- 172.21.15.151:0/679803632 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5592a980d610 0x5592a980eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:49.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.766+0000 7f05ef400700 1 -- 172.21.15.151:0/679803632 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5592a9808140 msgr2=0x5592a9807b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:59:49.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.766+0000 7f05ef400700 1 --2- 172.21.15.151:0/679803632 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5592a9808140 0x5592a9807b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:49.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.766+0000 7f05ef400700 1 -- 172.21.15.151:0/679803632 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5592a980df30 msgr2=0x5592a98113d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:59:49.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.766+0000 7f05ef400700 1 --2- 172.21.15.151:0/679803632 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5592a980df30 0x5592a98113d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:49.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.766+0000 7f05ef400700 1 -- 172.21.15.151:0/679803632 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592a9823dd0 con 0x5592a980d610 2024-05-21T21:59:49.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.767+0000 7f05ef400700 1 --2- 172.21.15.151:0/679803632 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5592a980d610 0x5592a980eba0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f05dc004d80 tx=0x7f05dc011f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c23ce412ce52bc93 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:59:49.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.767+0000 7f05e4c00700 1 -- 172.21.15.151:0/679803632 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05dc005ce0 con 0x5592a980d610 2024-05-21T21:59:49.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.767+0000 7f05e4c00700 1 -- 172.21.15.151:0/679803632 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05dc011890 con 0x5592a980d610 2024-05-21T21:59:49.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.767+0000 7f05e4c00700 1 -- 172.21.15.151:0/679803632 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05dc003680 con 0x5592a980d610 2024-05-21T21:59:49.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.768+0000 7f0601735b80 1 -- 172.21.15.151:0/679803632 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5592a980d610 msgr2=0x5592a980eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:59:49.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.768+0000 7f0601735b80 1 --2- 172.21.15.151:0/679803632 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5592a980d610 0x5592a980eba0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f05dc004d80 tx=0x7f05dc011f90 comp rx=0 tx=0).stop 2024-05-21T21:59:49.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.768+0000 7f0601735b80 1 -- 172.21.15.151:0/679803632 shutdown_connections 2024-05-21T21:59:49.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.768+0000 7f0601735b80 1 --2- 172.21.15.151:0/679803632 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5592a980df30 0x5592a98113d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:49.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.768+0000 7f0601735b80 1 --2- 172.21.15.151:0/679803632 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5592a9808140 0x5592a9807b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:49.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.768+0000 7f0601735b80 1 --2- 172.21.15.151:0/679803632 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5592a980d610 0x5592a980eba0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:49.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.768+0000 7f0601735b80 1 -- 172.21.15.151:0/679803632 >> 172.21.15.151:0/679803632 conn(0x5592a9735be0 msgr2=0x5592a980f890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:59:49.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.768+0000 7f0601735b80 1 -- 172.21.15.151:0/679803632 shutdown_connections 2024-05-21T21:59:49.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.768+0000 7f0601735b80 1 -- 172.21.15.151:0/679803632 wait complete. 2024-05-21T21:59:49.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.768+0000 7f0601735b80 1 -- start start 2024-05-21T21:59:49.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.769+0000 7f0601735b80 1 Processor -- start 2024-05-21T21:59:49.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.769+0000 7f0601735b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5592a9808140 0x5592a981a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:49.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.769+0000 7f0601735b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5592a980d610 0x5592a9828670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:49.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.769+0000 7f0601735b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5592a980df30 0x5592a9828bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:49.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.769+0000 7f0601735b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f05f01b1380 con 0x5592a9808140 2024-05-21T21:59:49.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.769+0000 7f0601735b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f05f01b0cb0 con 0x5592a980d610 2024-05-21T21:59:49.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.769+0000 7f0601735b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f05f01adeb0 con 0x5592a980df30 2024-05-21T21:59:49.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.769+0000 7f0601735b80 10 client.0 _ll_register_callbacks cb 0x5592a96ffc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:59:49.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.770+0000 7f05efe00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5592a9808140 0x5592a981a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:49.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.770+0000 7f05ef400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5592a980d610 0x5592a9828670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:49.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.770+0000 7f05ef400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5592a980d610 0x5592a9828670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:35396/0 (socket says 172.21.15.151:35396) 2024-05-21T21:59:49.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.770+0000 7f05ef400700 1 -- 172.21.15.151:0/1732580141 learned_addr learned my addr 172.21.15.151:0/1732580141 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:59:49.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.770+0000 7f05f4e00700 1 --2- 172.21.15.151:0/1732580141 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5592a980df30 0x5592a9828bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:49.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.770+0000 7f05ef400700 1 -- 172.21.15.151:0/1732580141 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5592a980df30 msgr2=0x5592a9828bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:59:49.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.770+0000 7f05ef400700 1 --2- 172.21.15.151:0/1732580141 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5592a980df30 0x5592a9828bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:49.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.770+0000 7f05ef400700 1 -- 172.21.15.151:0/1732580141 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5592a9808140 msgr2=0x5592a981a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:59:49.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.770+0000 7f05ef400700 1 --2- 172.21.15.151:0/1732580141 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5592a9808140 0x5592a981a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:49.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.770+0000 7f05ef400700 1 -- 172.21.15.151:0/1732580141 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592a9829170 con 0x5592a980d610 2024-05-21T21:59:49.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.770+0000 7f05ef400700 1 --2- 172.21.15.151:0/1732580141 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5592a980d610 0x5592a9828670 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f05dc004c80 tx=0x7f05dc004b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0fc79a4bb3e0ff7 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:59:49.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.770+0000 7f05da000700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-21T21:59:49.776 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.771+0000 7f05da000700 1 -- 172.21.15.151:0/1732580141 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05dc011b00 con 0x5592a980d610 2024-05-21T21:59:49.776 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.771+0000 7f0601735b80 1 -- 172.21.15.151:0/1732580141 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05dc011890 con 0x5592a980d610 2024-05-21T21:59:49.776 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.771+0000 7f05da000700 1 -- 172.21.15.151:0/1732580141 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f05dc02a070 con 0x5592a980d610 2024-05-21T21:59:49.776 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.771+0000 7f05da000700 1 -- 172.21.15.151:0/1732580141 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f05dc011ca0 con 0x5592a980d610 2024-05-21T21:59:49.776 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.771+0000 7f0601735b80 10 client.26162 fetch_fsmap learned FSMap version 29 2024-05-21T21:59:49.776 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.771+0000 7f0601735b80 1 -- 172.21.15.151:0/1732580141 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5592a988bce0 con 0x5592a980d610 2024-05-21T21:59:49.776 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.771+0000 7f05da000700 1 -- 172.21.15.151:0/1732580141 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05dc011890 con 0x5592a980d610 2024-05-21T21:59:49.777 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.771+0000 7f05da000700 1 -- 172.21.15.151:0/1732580141 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f05dc018070 con 0x5592a980d610 2024-05-21T21:59:49.777 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.771+0000 7f0601735b80 10 client.26162 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T21:59:49.777 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.771+0000 7f0601735b80 10 client.26162 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-05-21T21:59:49.777 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.772+0000 7f0601735b80 20 client.26162 populate_metadata read hostname 'smithi151' 2024-05-21T21:59:49.777 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.772+0000 7f0601735b80 1 --2- 172.21.15.151:0/1732580141 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x5592a988c140 0x5592a988e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:49.777 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.772+0000 7f0601735b80 4 client.26162 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T21:59:49.777 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.772+0000 7f0601735b80 1 -- 172.21.15.151:0/1732580141 --> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5592a9761570 con 0x5592a988c140 2024-05-21T21:59:49.777 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.772+0000 7f05efe00700 1 --2- 172.21.15.151:0/1732580141 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x5592a988c140 0x5592a988e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:49.777 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.772+0000 7f05efe00700 1 --2- 172.21.15.151:0/1732580141 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x5592a988c140 0x5592a988e560 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=968549b93743a210 server_cookie=fa59f31fe56d7a2f in_seq=0 out_seq=0 2024-05-21T21:59:49.777 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.772+0000 7f05da000700 10 client.26162 ms_handle_connect on v2:172.21.15.164:6836/1554609572 2024-05-21T21:59:49.777 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.772+0000 7f05da000700 1 -- 172.21.15.151:0/1732580141 <== mds.2 v2:172.21.15.164:6836/1554609572 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5592a9761570 con 0x5592a988c140 2024-05-21T21:59:49.777 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.773+0000 7f05da000700 10 client.26162 handle_command_reply: tid=0 2024-05-21T21:59:49.777 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.773+0000 7f0601735b80 1 client.26162 shutdown 2024-05-21T21:59:49.777 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.773+0000 7f0601735b80 1 -- 172.21.15.151:0/1732580141 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5592a980d610 msgr2=0x5592a9828670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:59:49.778 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.773+0000 7f0601735b80 1 --2- 172.21.15.151:0/1732580141 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5592a980d610 0x5592a9828670 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f05dc004c80 tx=0x7f05dc004b50 comp rx=0 tx=0).stop 2024-05-21T21:59:49.778 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.773+0000 7f0601735b80 1 -- 172.21.15.151:0/1732580141 shutdown_connections 2024-05-21T21:59:49.778 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.773+0000 7f0601735b80 1 --2- 172.21.15.151:0/1732580141 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x5592a988c140 0x5592a988e560 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:49.778 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.773+0000 7f0601735b80 1 --2- 172.21.15.151:0/1732580141 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5592a980df30 0x5592a9828bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:49.778 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.773+0000 7f0601735b80 1 --2- 172.21.15.151:0/1732580141 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5592a9808140 0x5592a981a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:49.778 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.773+0000 7f0601735b80 1 --2- 172.21.15.151:0/1732580141 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5592a980d610 0x5592a9828670 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:49.778 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.773+0000 7f05da000700 0 client.26162 ms_handle_reset on v2:172.21.15.164:6836/1554609572 2024-05-21T21:59:49.778 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.773+0000 7f0601735b80 1 -- 172.21.15.151:0/1732580141 >> 172.21.15.151:0/1732580141 conn(0x5592a9735be0 msgr2=0x5592a9810120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:59:49.778 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.773+0000 7f0601735b80 1 -- 172.21.15.151:0/1732580141 shutdown_connections 2024-05-21T21:59:49.778 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.773+0000 7f0601735b80 1 -- 172.21.15.151:0/1732580141 wait complete. 2024-05-21T21:59:49.778 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.773+0000 7f0601735b80 20 client.26162 trim_cache size 0 max 16384 2024-05-21T21:59:49.779 INFO:teuthology.orchestra.run.smithi151.stdout:[] 2024-05-21T21:59:49.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.774+0000 7f05f6e00700 1 -- 172.21.15.151:0/4074148342 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f05d40b29f0 msgr2=0x7f05d40b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:59:49.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.774+0000 7f05f6e00700 1 --2- 172.21.15.151:0/4074148342 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f05d40b29f0 0x7f05d40b4eb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f05e8009fd0 tx=0x7f05e800b040 comp rx=0 tx=0).stop 2024-05-21T21:59:49.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.774+0000 7f05f6e00700 1 -- 172.21.15.151:0/4074148342 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05f006b250 msgr2=0x7f05f019beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:59:49.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.774+0000 7f05f6e00700 1 --2- 172.21.15.151:0/4074148342 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05f006b250 0x7f05f019beb0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f05e0005910 tx=0x7f05e000b670 comp rx=0 tx=0).stop 2024-05-21T21:59:49.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.774+0000 7f05f6e00700 1 -- 172.21.15.151:0/4074148342 shutdown_connections 2024-05-21T21:59:49.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.774+0000 7f05f6e00700 1 --2- 172.21.15.151:0/4074148342 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f05d40b29f0 0x7f05d40b4eb0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:49.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.774+0000 7f05f6e00700 1 --2- 172.21.15.151:0/4074148342 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05f010fff0 0x7f05f01a0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:49.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.774+0000 7f05f6e00700 1 --2- 172.21.15.151:0/4074148342 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05f006b250 0x7f05f019beb0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:49.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.774+0000 7f05f6e00700 1 --2- 172.21.15.151:0/4074148342 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f05f006bb70 0x7f05f019c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:49.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.774+0000 7f05f6e00700 1 -- 172.21.15.151:0/4074148342 >> 172.21.15.151:0/4074148342 conn(0x7f05f00793f0 msgr2=0x7f05f01114e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:59:49.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.774+0000 7f05f6e00700 1 -- 172.21.15.151:0/4074148342 shutdown_connections 2024-05-21T21:59:49.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:49.774+0000 7f05f6e00700 1 -- 172.21.15.151:0/4074148342 wait complete. 2024-05-21T21:59:49.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:59:49 smithi164 ceph-mon[36269]: pgmap v129: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 21 MiB/s wr, 24 op/s 2024-05-21T21:59:50.636 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-21T21:59:51.086 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:59:51.498 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:59:51 smithi151 ceph-mon[31623]: pgmap v130: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 21 MiB/s wr, 19 op/s 2024-05-21T21:59:51.735 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:59:51 smithi184 ceph-mon[36942]: pgmap v130: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 21 MiB/s wr, 19 op/s 2024-05-21T21:59:51.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:59:51 smithi164 ceph-mon[36269]: pgmap v130: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 21 MiB/s wr, 19 op/s 2024-05-21T21:59:53.318 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.317+0000 7f61e3200700 1 -- 172.21.15.151:0/4041326185 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f61e4075970 msgr2=0x7f61e4075d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:59:53.319 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.317+0000 7f61e3200700 1 --2- 172.21.15.151:0/4041326185 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f61e4075970 0x7f61e4075d50 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f61d400b0a0 tx=0x7f61d400b3b0 comp rx=0 tx=0).stop 2024-05-21T21:59:53.319 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.317+0000 7f61e3200700 1 -- 172.21.15.151:0/4041326185 shutdown_connections 2024-05-21T21:59:53.319 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.317+0000 7f61e3200700 1 --2- 172.21.15.151:0/4041326185 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61e410fcb0 0x7f61e4112120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:53.319 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.317+0000 7f61e3200700 1 --2- 172.21.15.151:0/4041326185 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f61e4075970 0x7f61e4075d50 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:53.319 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.317+0000 7f61e3200700 1 --2- 172.21.15.151:0/4041326185 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f61e4076290 0x7f61e410f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:53.319 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.317+0000 7f61e3200700 1 -- 172.21.15.151:0/4041326185 >> 172.21.15.151:0/4041326185 conn(0x7f61e40fefc0 msgr2=0x7f61e41013e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:59:53.319 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.317+0000 7f61e3200700 1 -- 172.21.15.151:0/4041326185 shutdown_connections 2024-05-21T21:59:53.319 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.317+0000 7f61e3200700 1 -- 172.21.15.151:0/4041326185 wait complete. 2024-05-21T21:59:53.320 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.318+0000 7f61e3200700 1 Processor -- start 2024-05-21T21:59:53.320 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.319+0000 7f61e3200700 1 -- start start 2024-05-21T21:59:53.320 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.319+0000 7f61e3200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f61e4075970 0x7f61e419bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:53.320 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.319+0000 7f61e3200700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61e4076290 0x7f61e419c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:53.320 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.319+0000 7f61e3200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f61e410fcb0 0x7f61e41a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:53.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.319+0000 7f61e3200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f61e4115840 con 0x7f61e4075970 2024-05-21T21:59:53.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.319+0000 7f61e3200700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f61e41156c0 con 0x7f61e410fcb0 2024-05-21T21:59:53.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.319+0000 7f61e3200700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f61e41159c0 con 0x7f61e4076290 2024-05-21T21:59:53.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.319+0000 7f61e1e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f61e4075970 0x7f61e419bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:53.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.319+0000 7f61e1e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f61e4075970 0x7f61e419bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:56580/0 (socket says 172.21.15.151:56580) 2024-05-21T21:59:53.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.319+0000 7f61e1e00700 1 -- 172.21.15.151:0/192689789 learned_addr learned my addr 172.21.15.151:0/192689789 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:59:53.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.319+0000 7f61d9400700 1 --2- 172.21.15.151:0/192689789 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61e4076290 0x7f61e419c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:53.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.319+0000 7f61e2800700 1 --2- 172.21.15.151:0/192689789 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f61e410fcb0 0x7f61e41a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:53.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.320+0000 7f61e1e00700 1 -- 172.21.15.151:0/192689789 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61e4076290 msgr2=0x7f61e419c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:59:53.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.320+0000 7f61e1e00700 1 --2- 172.21.15.151:0/192689789 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61e4076290 0x7f61e419c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:53.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.320+0000 7f61e1e00700 1 -- 172.21.15.151:0/192689789 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f61e410fcb0 msgr2=0x7f61e41a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:59:53.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.320+0000 7f61e1e00700 1 --2- 172.21.15.151:0/192689789 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f61e410fcb0 0x7f61e41a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:53.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.320+0000 7f61e1e00700 1 -- 172.21.15.151:0/192689789 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61e41a0d60 con 0x7f61e4075970 2024-05-21T21:59:53.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.320+0000 7f61e1e00700 1 --2- 172.21.15.151:0/192689789 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f61e4075970 0x7f61e419bd50 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f61d400b070 tx=0x7f61d4004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:59:53.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.320+0000 7f61db400700 1 -- 172.21.15.151:0/192689789 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d4021070 con 0x7f61e4075970 2024-05-21T21:59:53.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.320+0000 7f61db400700 1 -- 172.21.15.151:0/192689789 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61d4007e30 con 0x7f61e4075970 2024-05-21T21:59:53.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.321+0000 7f61db400700 1 -- 172.21.15.151:0/192689789 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d40046c0 con 0x7f61e4075970 2024-05-21T21:59:53.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.321+0000 7f61e3200700 1 -- 172.21.15.151:0/192689789 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61e41a0ff0 con 0x7f61e4075970 2024-05-21T21:59:53.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.321+0000 7f61e3200700 1 -- 172.21.15.151:0/192689789 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61e41a88d0 con 0x7f61e4075970 2024-05-21T21:59:53.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.324+0000 7f61db400700 1 -- 172.21.15.151:0/192689789 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7f61d4007980 con 0x7f61e4075970 2024-05-21T21:59:53.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.325+0000 7f61f354fb80 1 Processor -- start 2024-05-21T21:59:53.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.326+0000 7f61f354fb80 1 -- start start 2024-05-21T21:59:53.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.326+0000 7f61db400700 1 --2- 172.21.15.151:0/192689789 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f61c80b2a70 0x7f61c80b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:53.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.326+0000 7f61f354fb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a3594ff7c0 0x55a3594ffba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:53.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.326+0000 7f61f354fb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a3595000e0 0x55a3594f9d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:53.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.326+0000 7f61f354fb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a3594fa390 0x55a3594fa7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:53.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.326+0000 7f61f354fb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a359457e40 con 0x55a3594fa390 2024-05-21T21:59:53.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.326+0000 7f61f354fb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55a3594479e0 con 0x55a3595000e0 2024-05-21T21:59:53.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.326+0000 7f61f354fb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55a35944ba20 con 0x55a3594ff7c0 2024-05-21T21:59:53.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.326+0000 7f61db400700 1 -- 172.21.15.151:0/192689789 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f61d400f020 con 0x7f61e4075970 2024-05-21T21:59:53.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.326+0000 7f61e2800700 1 --2- 172.21.15.151:0/192689789 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f61c80b2a70 0x7f61c80b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:53.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.326+0000 7f61e1e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a3594ff7c0 0x55a3594ffba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:53.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.326+0000 7f61d9400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a3595000e0 0x55a3594f9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:53.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.326+0000 7f61d9400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a3595000e0 0x55a3594f9d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:35418/0 (socket says 172.21.15.151:35418) 2024-05-21T21:59:53.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.326+0000 7f61e1e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a3594ff7c0 0x55a3594ffba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:42066/0 (socket says 172.21.15.151:42066) 2024-05-21T21:59:53.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.326+0000 7f61d9400700 1 -- 172.21.15.151:0/2115598056 learned_addr learned my addr 172.21.15.151:0/2115598056 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:59:53.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.327+0000 7f61e2800700 1 --2- 172.21.15.151:0/2115598056 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a3594fa390 0x55a3594fa7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:53.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.327+0000 7f61d9400700 1 -- 172.21.15.151:0/2115598056 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a3594ff7c0 msgr2=0x55a3594ffba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:59:53.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.327+0000 7f61d9400700 1 --2- 172.21.15.151:0/2115598056 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a3594ff7c0 0x55a3594ffba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:53.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.327+0000 7f61d9400700 1 -- 172.21.15.151:0/2115598056 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a3594fa390 msgr2=0x55a3594fa7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:59:53.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.327+0000 7f61d9400700 1 --2- 172.21.15.151:0/2115598056 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a3594fa390 0x55a3594fa7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:53.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.327+0000 7f61d9400700 1 -- 172.21.15.151:0/2115598056 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a359410420 con 0x55a3595000e0 2024-05-21T21:59:53.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.327+0000 7f61e1e00700 1 --2- 172.21.15.151:0/2115598056 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a3594ff7c0 0x55a3594ffba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:59:53.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.327+0000 7f61e2800700 1 --2- 172.21.15.151:0/192689789 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f61c80b2a70 0x7f61c80b4f30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f61dc00b620 tx=0x7f61dc00bb50 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:59:53.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.327+0000 7f61d9400700 1 --2- 172.21.15.151:0/2115598056 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a3595000e0 0x55a3594f9d70 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f61cc009d20 tx=0x7f61cc011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=960249528830cec3 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:59:53.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.328+0000 7f61d3400700 1 -- 172.21.15.151:0/2115598056 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61cc01e070 con 0x55a3595000e0 2024-05-21T21:59:53.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.328+0000 7f61d3400700 1 -- 172.21.15.151:0/2115598056 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61cc019ae0 con 0x55a3595000e0 2024-05-21T21:59:53.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.328+0000 7f61d3400700 1 -- 172.21.15.151:0/2115598056 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61cc00e9f0 con 0x55a3595000e0 2024-05-21T21:59:53.332 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.329+0000 7f61f354fb80 1 -- 172.21.15.151:0/2115598056 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a3595000e0 msgr2=0x55a3594f9d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:59:53.332 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.329+0000 7f61f354fb80 1 --2- 172.21.15.151:0/2115598056 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a3595000e0 0x55a3594f9d70 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f61cc009d20 tx=0x7f61cc011720 comp rx=0 tx=0).stop 2024-05-21T21:59:53.332 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.329+0000 7f61f354fb80 1 -- 172.21.15.151:0/2115598056 shutdown_connections 2024-05-21T21:59:53.332 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.329+0000 7f61f354fb80 1 --2- 172.21.15.151:0/2115598056 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a3594fa390 0x55a3594fa7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:53.332 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.329+0000 7f61f354fb80 1 --2- 172.21.15.151:0/2115598056 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a3594ff7c0 0x55a3594ffba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:53.332 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.329+0000 7f61f354fb80 1 --2- 172.21.15.151:0/2115598056 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a3595000e0 0x55a3594f9d70 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:53.332 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.329+0000 7f61f354fb80 1 -- 172.21.15.151:0/2115598056 >> 172.21.15.151:0/2115598056 conn(0x55a359427be0 msgr2=0x55a3594f4810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:59:53.332 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.329+0000 7f61f354fb80 1 -- 172.21.15.151:0/2115598056 shutdown_connections 2024-05-21T21:59:53.332 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.329+0000 7f61f354fb80 1 -- 172.21.15.151:0/2115598056 wait complete. 2024-05-21T21:59:53.332 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.330+0000 7f61f354fb80 1 -- start start 2024-05-21T21:59:53.332 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.330+0000 7f61f354fb80 1 Processor -- start 2024-05-21T21:59:53.333 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.330+0000 7f61f354fb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a3594fa390 0x55a359505e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:53.333 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.330+0000 7f61f354fb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a3594ff7c0 0x55a359503260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:53.334 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.331+0000 7f61f354fb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a3595000e0 0x55a3595037a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:53.334 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.331+0000 7f61f354fb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a35934dfa0 con 0x55a3595000e0 2024-05-21T21:59:53.334 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.331+0000 7f61f354fb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55a3593e8760 con 0x55a3594fa390 2024-05-21T21:59:53.334 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.331+0000 7f61f354fb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55a359506c00 con 0x55a3594ff7c0 2024-05-21T21:59:53.334 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.331+0000 7f61f354fb80 10 client.0 _ll_register_callbacks cb 0x55a3593f1c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:59:53.334 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.331+0000 7f61e2800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a3595000e0 0x55a3595037a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:53.334 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.331+0000 7f61e2800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a3595000e0 0x55a3595037a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:56590/0 (socket says 172.21.15.151:56590) 2024-05-21T21:59:53.334 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.331+0000 7f61e2800700 1 -- 172.21.15.151:0/1014149116 learned_addr learned my addr 172.21.15.151:0/1014149116 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:59:53.334 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.331+0000 7f61d9400700 1 --2- 172.21.15.151:0/1014149116 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a3594ff7c0 0x55a359503260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:53.334 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.331+0000 7f61e1e00700 1 --2- 172.21.15.151:0/1014149116 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a3594fa390 0x55a359505e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:53.334 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.331+0000 7f61e2800700 1 -- 172.21.15.151:0/1014149116 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a3594ff7c0 msgr2=0x55a359503260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:59:53.334 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.331+0000 7f61e2800700 1 --2- 172.21.15.151:0/1014149116 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a3594ff7c0 0x55a359503260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:53.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.331+0000 7f61e2800700 1 -- 172.21.15.151:0/1014149116 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a3594fa390 msgr2=0x55a359505e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:59:53.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.331+0000 7f61e2800700 1 --2- 172.21.15.151:0/1014149116 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a3594fa390 0x55a359505e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:53.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.331+0000 7f61e2800700 1 -- 172.21.15.151:0/1014149116 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3593978d0 con 0x55a3595000e0 2024-05-21T21:59:53.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.332+0000 7f61e1e00700 1 --2- 172.21.15.151:0/1014149116 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a3594fa390 0x55a359505e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:59:53.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.332+0000 7f61e2800700 1 --2- 172.21.15.151:0/1014149116 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a3595000e0 0x55a3595037a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f61dc00b700 tx=0x7f61dc00b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c205fefaa7393b6b server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:59:53.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.332+0000 7f61c7e00700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-21T21:59:53.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.332+0000 7f61c7e00700 1 -- 172.21.15.151:0/1014149116 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61dc02d030 con 0x55a3595000e0 2024-05-21T21:59:53.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.332+0000 7f61c7e00700 1 -- 172.21.15.151:0/1014149116 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f61dc012e40 con 0x55a3595000e0 2024-05-21T21:59:53.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.332+0000 7f61f354fb80 1 -- 172.21.15.151:0/1014149116 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61cc019ae0 con 0x55a3595000e0 2024-05-21T21:59:53.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.333+0000 7f61c7e00700 1 -- 172.21.15.151:0/1014149116 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f61dc038050 con 0x55a3595000e0 2024-05-21T21:59:53.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.333+0000 7f61f354fb80 10 client.16053 fetch_fsmap learned FSMap version 29 2024-05-21T21:59:53.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.333+0000 7f61f354fb80 1 -- 172.21.15.151:0/1014149116 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a359418060 con 0x55a3595000e0 2024-05-21T21:59:53.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.333+0000 7f61c7e00700 1 -- 172.21.15.151:0/1014149116 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61cc019ae0 con 0x55a3595000e0 2024-05-21T21:59:53.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.334+0000 7f61c7e00700 1 -- 172.21.15.151:0/1014149116 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f61dc003680 con 0x55a3595000e0 2024-05-21T21:59:53.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.334+0000 7f61f354fb80 10 client.16053 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T21:59:53.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.334+0000 7f61f354fb80 10 client.16053 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-21T21:59:53.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.334+0000 7f61f354fb80 20 client.16053 populate_metadata read hostname 'smithi151' 2024-05-21T21:59:53.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.334+0000 7f61f354fb80 1 --2- 172.21.15.151:0/1014149116 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x55a359513800 0x55a359515bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:53.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.334+0000 7f61f354fb80 4 client.16053 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T21:59:53.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.334+0000 7f61f354fb80 1 -- 172.21.15.151:0/1014149116 --> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a359459d90 con 0x55a359513800 2024-05-21T21:59:53.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.334+0000 7f61e1e00700 1 --2- 172.21.15.151:0/1014149116 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x55a359513800 0x55a359515bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:53.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.335+0000 7f61e1e00700 1 --2- 172.21.15.151:0/1014149116 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x55a359513800 0x55a359515bf0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3de7b104748f3b99 server_cookie=f177886a775fb81b in_seq=0 out_seq=0 2024-05-21T21:59:53.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.335+0000 7f61c7e00700 10 client.16053 ms_handle_connect on v2:172.21.15.184:6838/3059935143 2024-05-21T21:59:53.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.335+0000 7f61c7e00700 1 -- 172.21.15.151:0/1014149116 <== mds.1 v2:172.21.15.184:6838/3059935143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55a359459d90 con 0x55a359513800 2024-05-21T21:59:53.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.335+0000 7f61c7e00700 10 client.16053 handle_command_reply: tid=0 2024-05-21T21:59:53.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.335+0000 7f61f354fb80 1 client.16053 shutdown 2024-05-21T21:59:53.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.336+0000 7f61f354fb80 1 -- 172.21.15.151:0/1014149116 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a3595000e0 msgr2=0x55a3595037a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:59:53.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.336+0000 7f61f354fb80 1 --2- 172.21.15.151:0/1014149116 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a3595000e0 0x55a3595037a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f61dc00b700 tx=0x7f61dc00b390 comp rx=0 tx=0).stop 2024-05-21T21:59:53.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.336+0000 7f61f354fb80 1 -- 172.21.15.151:0/1014149116 shutdown_connections 2024-05-21T21:59:53.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.336+0000 7f61f354fb80 1 --2- 172.21.15.151:0/1014149116 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x55a359513800 0x55a359515bf0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:53.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.336+0000 7f61f354fb80 1 --2- 172.21.15.151:0/1014149116 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a3594fa390 0x55a359505e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:53.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.336+0000 7f61f354fb80 1 --2- 172.21.15.151:0/1014149116 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a3595000e0 0x55a3595037a0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:53.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.336+0000 7f61f354fb80 1 --2- 172.21.15.151:0/1014149116 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a3594ff7c0 0x55a359503260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:53.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.336+0000 7f61c7e00700 0 client.16053 ms_handle_reset on v2:172.21.15.184:6838/3059935143 2024-05-21T21:59:53.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.336+0000 7f61f354fb80 1 -- 172.21.15.151:0/1014149116 >> 172.21.15.151:0/1014149116 conn(0x55a359427be0 msgr2=0x55a3594f3fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:59:53.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.336+0000 7f61f354fb80 1 -- 172.21.15.151:0/1014149116 shutdown_connections 2024-05-21T21:59:53.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.336+0000 7f61f354fb80 1 -- 172.21.15.151:0/1014149116 wait complete. 2024-05-21T21:59:53.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.336+0000 7f61f354fb80 20 client.16053 trim_cache size 0 max 16384 2024-05-21T21:59:53.354 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.353+0000 7f61f354fb80 1 Processor -- start 2024-05-21T21:59:53.354 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.353+0000 7f61f354fb80 1 -- start start 2024-05-21T21:59:53.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.353+0000 7f61f354fb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a3594fa390 0x55a3594f9d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:53.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.353+0000 7f61f354fb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a3594ff7c0 0x55a3594f4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:53.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.353+0000 7f61f354fb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a3595000e0 0x55a35951b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:53.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.353+0000 7f61f354fb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f61e41aba90 con 0x55a3594ff7c0 2024-05-21T21:59:53.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.353+0000 7f61f354fb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f61e41ad5d0 con 0x55a3595000e0 2024-05-21T21:59:53.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.353+0000 7f61f354fb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f61e41ac160 con 0x55a3594fa390 2024-05-21T21:59:53.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.353+0000 7f61d9400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a3594ff7c0 0x55a3594f4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:53.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.353+0000 7f61e1e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a3594fa390 0x55a3594f9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:53.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.353+0000 7f61d9400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a3594ff7c0 0x55a3594f4740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:56596/0 (socket says 172.21.15.151:56596) 2024-05-21T21:59:53.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.353+0000 7f61d9400700 1 -- 172.21.15.151:0/605960106 learned_addr learned my addr 172.21.15.151:0/605960106 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:59:53.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.354+0000 7f61e2800700 1 --2- 172.21.15.151:0/605960106 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a3595000e0 0x55a35951b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:53.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.354+0000 7f61e1e00700 1 -- 172.21.15.151:0/605960106 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a3595000e0 msgr2=0x55a35951b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:59:53.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.354+0000 7f61e1e00700 1 --2- 172.21.15.151:0/605960106 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a3595000e0 0x55a35951b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:53.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.354+0000 7f61e1e00700 1 -- 172.21.15.151:0/605960106 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a3594ff7c0 msgr2=0x55a3594f4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:59:53.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.354+0000 7f61e1e00700 1 --2- 172.21.15.151:0/605960106 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a3594ff7c0 0x55a3594f4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:53.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.354+0000 7f61e1e00700 1 -- 172.21.15.151:0/605960106 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3594ffee0 con 0x55a3594fa390 2024-05-21T21:59:53.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.354+0000 7f61d9400700 1 --2- 172.21.15.151:0/605960106 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a3594ff7c0 0x55a3594f4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:59:53.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.354+0000 7f61e1e00700 1 --2- 172.21.15.151:0/605960106 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a3594fa390 0x55a3594f9d70 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f61d40e1190 tx=0x7f61d4009040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=376ffaaf53a70165 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:59:53.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.354+0000 7f61d2a00700 1 -- 172.21.15.151:0/605960106 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d40ab4b0 con 0x55a3594fa390 2024-05-21T21:59:53.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.355+0000 7f61d2a00700 1 -- 172.21.15.151:0/605960106 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61d40aadb0 con 0x55a3594fa390 2024-05-21T21:59:53.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.355+0000 7f61d2a00700 1 -- 172.21.15.151:0/605960106 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d40074e0 con 0x55a3594fa390 2024-05-21T21:59:53.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.355+0000 7f61f354fb80 1 -- 172.21.15.151:0/605960106 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a3594fa390 msgr2=0x55a3594f9d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:59:53.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.355+0000 7f61f354fb80 1 --2- 172.21.15.151:0/605960106 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a3594fa390 0x55a3594f9d70 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f61d40e1190 tx=0x7f61d4009040 comp rx=0 tx=0).stop 2024-05-21T21:59:53.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.355+0000 7f61f354fb80 1 -- 172.21.15.151:0/605960106 shutdown_connections 2024-05-21T21:59:53.363 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.355+0000 7f61f354fb80 1 --2- 172.21.15.151:0/605960106 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a3595000e0 0x55a35951b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:53.363 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.355+0000 7f61f354fb80 1 --2- 172.21.15.151:0/605960106 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a3594fa390 0x55a3594f9d70 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:53.363 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.355+0000 7f61f354fb80 1 --2- 172.21.15.151:0/605960106 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a3594ff7c0 0x55a3594f4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:53.363 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.355+0000 7f61f354fb80 1 -- 172.21.15.151:0/605960106 >> 172.21.15.151:0/605960106 conn(0x55a359427be0 msgr2=0x55a3594fd310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:59:53.363 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.355+0000 7f61f354fb80 1 -- 172.21.15.151:0/605960106 shutdown_connections 2024-05-21T21:59:53.363 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.356+0000 7f61f354fb80 1 -- 172.21.15.151:0/605960106 wait complete. 2024-05-21T21:59:53.363 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.356+0000 7f61f354fb80 1 -- start start 2024-05-21T21:59:53.363 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.356+0000 7f61f354fb80 1 Processor -- start 2024-05-21T21:59:53.363 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.357+0000 7f61f354fb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a3594fa390 0x55a35950c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:53.363 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.357+0000 7f61f354fb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a3594ff7c0 0x55a35951ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:53.363 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.357+0000 7f61f354fb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a3595000e0 0x55a35951b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:53.364 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.357+0000 7f61f354fb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f61e41acf00 con 0x55a3595000e0 2024-05-21T21:59:53.364 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.357+0000 7f61f354fb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f61e41ac830 con 0x55a3594fa390 2024-05-21T21:59:53.364 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.357+0000 7f61f354fb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f61e4107490 con 0x55a3594ff7c0 2024-05-21T21:59:53.364 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.357+0000 7f61f354fb80 10 client.0 _ll_register_callbacks cb 0x55a3593f1c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:59:53.364 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.357+0000 7f61e1e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a3594fa390 0x55a35950c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:53.364 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.357+0000 7f61e1e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a3594fa390 0x55a35950c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:35448/0 (socket says 172.21.15.151:35448) 2024-05-21T21:59:53.364 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.357+0000 7f61e1e00700 1 -- 172.21.15.151:0/1283839935 learned_addr learned my addr 172.21.15.151:0/1283839935 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:59:53.364 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.357+0000 7f61d9400700 1 --2- 172.21.15.151:0/1283839935 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a3594ff7c0 0x55a35951ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:53.364 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.357+0000 7f61e2800700 1 --2- 172.21.15.151:0/1283839935 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a3595000e0 0x55a35951b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:53.364 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.357+0000 7f61e1e00700 1 -- 172.21.15.151:0/1283839935 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a3594ff7c0 msgr2=0x55a35951ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:59:53.364 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.357+0000 7f61e1e00700 1 --2- 172.21.15.151:0/1283839935 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a3594ff7c0 0x55a35951ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:53.364 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.357+0000 7f61e1e00700 1 -- 172.21.15.151:0/1283839935 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a3595000e0 msgr2=0x55a35951b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:59:53.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.357+0000 7f61e1e00700 1 --2- 172.21.15.151:0/1283839935 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a3595000e0 0x55a35951b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:53.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.357+0000 7f61e1e00700 1 -- 172.21.15.151:0/1283839935 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a35951bb60 con 0x55a3594fa390 2024-05-21T21:59:53.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.358+0000 7f61e1e00700 1 --2- 172.21.15.151:0/1283839935 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a3594fa390 0x55a35950c640 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f61d40aad40 tx=0x7f61d40aa9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e53c9db4c83cb0f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:59:53.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.358+0000 7f61c7e00700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-21T21:59:53.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.358+0000 7f61c7e00700 1 -- 172.21.15.151:0/1283839935 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d4091060 con 0x55a3594fa390 2024-05-21T21:59:53.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.358+0000 7f61c7e00700 1 -- 172.21.15.151:0/1283839935 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f61d40074e0 con 0x55a3594fa390 2024-05-21T21:59:53.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.358+0000 7f61f354fb80 1 -- 172.21.15.151:0/1283839935 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61d40aadb0 con 0x55a3594fa390 2024-05-21T21:59:53.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.358+0000 7f61c7e00700 1 -- 172.21.15.151:0/1283839935 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f61d40c1020 con 0x55a3594fa390 2024-05-21T21:59:53.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.358+0000 7f61f354fb80 10 client.26180 fetch_fsmap learned FSMap version 29 2024-05-21T21:59:53.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.358+0000 7f61f354fb80 1 -- 172.21.15.151:0/1283839935 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a35957e4a0 con 0x55a3594fa390 2024-05-21T21:59:53.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.358+0000 7f61c7e00700 1 -- 172.21.15.151:0/1283839935 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61d40aadb0 con 0x55a3594fa390 2024-05-21T21:59:53.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.359+0000 7f61c7e00700 1 -- 172.21.15.151:0/1283839935 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f61d40b1040 con 0x55a3594fa390 2024-05-21T21:59:53.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.359+0000 7f61f354fb80 10 client.26180 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T21:59:53.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.359+0000 7f61f354fb80 10 client.26180 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-21T21:59:53.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.359+0000 7f61f354fb80 20 client.26180 populate_metadata read hostname 'smithi151' 2024-05-21T21:59:53.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.359+0000 7f61f354fb80 1 --2- 172.21.15.151:0/1283839935 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x55a35957ea70 0x55a359580e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:53.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.359+0000 7f61f354fb80 4 client.26180 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T21:59:53.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.359+0000 7f61f354fb80 1 -- 172.21.15.151:0/1283839935 --> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a359453570 con 0x55a35957ea70 2024-05-21T21:59:53.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.359+0000 7f61d9400700 1 --2- 172.21.15.151:0/1283839935 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x55a35957ea70 0x55a359580e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:53.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.360+0000 7f61d9400700 1 --2- 172.21.15.151:0/1283839935 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x55a35957ea70 0x55a359580e90 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f038f5ce16434017 server_cookie=7c820c3909da7b6f in_seq=0 out_seq=0 2024-05-21T21:59:53.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.360+0000 7f61c7e00700 10 client.26180 ms_handle_connect on v2:172.21.15.184:6838/3059935143 2024-05-21T21:59:53.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.360+0000 7f61c7e00700 1 -- 172.21.15.151:0/1283839935 <== mds.1 v2:172.21.15.184:6838/3059935143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a359453570 con 0x55a35957ea70 2024-05-21T21:59:53.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.360+0000 7f61c7e00700 10 client.26180 handle_command_reply: tid=0 2024-05-21T21:59:53.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.360+0000 7f61f354fb80 1 client.26180 shutdown 2024-05-21T21:59:53.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.360+0000 7f61f354fb80 1 -- 172.21.15.151:0/1283839935 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a3594fa390 msgr2=0x55a35950c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:59:53.367 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.360+0000 7f61f354fb80 1 --2- 172.21.15.151:0/1283839935 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a3594fa390 0x55a35950c640 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f61d40aad40 tx=0x7f61d40aa9c0 comp rx=0 tx=0).stop 2024-05-21T21:59:53.367 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.360+0000 7f61f354fb80 1 -- 172.21.15.151:0/1283839935 shutdown_connections 2024-05-21T21:59:53.367 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.360+0000 7f61f354fb80 1 --2- 172.21.15.151:0/1283839935 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x55a35957ea70 0x55a359580e90 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:53.367 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.360+0000 7f61f354fb80 1 --2- 172.21.15.151:0/1283839935 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a3594fa390 0x55a35950c640 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:53.367 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.360+0000 7f61f354fb80 1 --2- 172.21.15.151:0/1283839935 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a3595000e0 0x55a35951b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:53.367 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.360+0000 7f61f354fb80 1 --2- 172.21.15.151:0/1283839935 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a3594ff7c0 0x55a35951ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:53.367 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.360+0000 7f61c7e00700 0 client.26180 ms_handle_reset on v2:172.21.15.184:6838/3059935143 2024-05-21T21:59:53.367 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.360+0000 7f61f354fb80 1 -- 172.21.15.151:0/1283839935 >> 172.21.15.151:0/1283839935 conn(0x55a359427be0 msgr2=0x55a3595004c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:59:53.367 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.360+0000 7f61f354fb80 1 -- 172.21.15.151:0/1283839935 shutdown_connections 2024-05-21T21:59:53.367 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.361+0000 7f61f354fb80 1 -- 172.21.15.151:0/1283839935 wait complete. 2024-05-21T21:59:53.367 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.361+0000 7f61f354fb80 20 client.26180 trim_cache size 0 max 16384 2024-05-21T21:59:53.368 INFO:teuthology.orchestra.run.smithi151.stdout:[] 2024-05-21T21:59:53.368 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.361+0000 7f61e3200700 1 -- 172.21.15.151:0/192689789 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f61c80b2a70 msgr2=0x7f61c80b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:59:53.368 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.361+0000 7f61e3200700 1 --2- 172.21.15.151:0/192689789 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f61c80b2a70 0x7f61c80b4f30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f61dc00b620 tx=0x7f61dc00bb50 comp rx=0 tx=0).stop 2024-05-21T21:59:53.368 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.361+0000 7f61e3200700 1 -- 172.21.15.151:0/192689789 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f61e4075970 msgr2=0x7f61e419bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:59:53.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.361+0000 7f61e3200700 1 --2- 172.21.15.151:0/192689789 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f61e4075970 0x7f61e419bd50 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f61d400b070 tx=0x7f61d4004d00 comp rx=0 tx=0).stop 2024-05-21T21:59:53.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.362+0000 7f61e3200700 1 -- 172.21.15.151:0/192689789 shutdown_connections 2024-05-21T21:59:53.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.362+0000 7f61e3200700 1 --2- 172.21.15.151:0/192689789 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f61c80b2a70 0x7f61c80b4f30 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:53.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.362+0000 7f61e3200700 1 --2- 172.21.15.151:0/192689789 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f61e410fcb0 0x7f61e41a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:53.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.362+0000 7f61e3200700 1 --2- 172.21.15.151:0/192689789 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f61e4075970 0x7f61e419bd50 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:53.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.362+0000 7f61e3200700 1 --2- 172.21.15.151:0/192689789 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61e4076290 0x7f61e419c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:53.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.362+0000 7f61e3200700 1 -- 172.21.15.151:0/192689789 >> 172.21.15.151:0/192689789 conn(0x7f61e40fefc0 msgr2=0x7f61e40ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:59:53.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.362+0000 7f61e3200700 1 -- 172.21.15.151:0/192689789 shutdown_connections 2024-05-21T21:59:53.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:53.362+0000 7f61e3200700 1 -- 172.21.15.151:0/192689789 wait complete. 2024-05-21T21:59:53.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:59:53 smithi151 ceph-mon[31623]: pgmap v131: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 21 MiB/s wr, 24 op/s 2024-05-21T21:59:53.735 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:59:53 smithi184 ceph-mon[36942]: pgmap v131: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 21 MiB/s wr, 24 op/s 2024-05-21T21:59:53.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:59:53 smithi164 ceph-mon[36269]: pgmap v131: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 21 MiB/s wr, 24 op/s 2024-05-21T21:59:54.270 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-21T21:59:54.726 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:59:55.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:59:55 smithi151 ceph-mon[31623]: pgmap v132: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 14 op/s 2024-05-21T21:59:55.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:59:55 smithi184 ceph-mon[36942]: pgmap v132: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 14 op/s 2024-05-21T21:59:55.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:59:55 smithi164 ceph-mon[36269]: pgmap v132: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 14 op/s 2024-05-21T21:59:56.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:56.987+0000 7f1158a00700 1 -- 172.21.15.151:0/4141782553 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1154103460 msgr2=0x7f11541038e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:59:56.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:56.987+0000 7f1158a00700 1 --2- 172.21.15.151:0/4141782553 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1154103460 0x7f11541038e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f1140009990 tx=0x7f1140009ca0 comp rx=0 tx=0).stop 2024-05-21T21:59:56.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:56.987+0000 7f1158a00700 1 -- 172.21.15.151:0/4141782553 shutdown_connections 2024-05-21T21:59:56.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:56.987+0000 7f1158a00700 1 --2- 172.21.15.151:0/4141782553 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f11541094f0 0x7f11541098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:56.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:56.987+0000 7f1158a00700 1 --2- 172.21.15.151:0/4141782553 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1154103e20 0x7f1154110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:56.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:56.987+0000 7f1158a00700 1 --2- 172.21.15.151:0/4141782553 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1154103460 0x7f11541038e0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:56.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:56.987+0000 7f1158a00700 1 -- 172.21.15.151:0/4141782553 >> 172.21.15.151:0/4141782553 conn(0x7f11540fef90 msgr2=0x7f11541013b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:59:56.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:56.988+0000 7f1158a00700 1 -- 172.21.15.151:0/4141782553 shutdown_connections 2024-05-21T21:59:56.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:56.988+0000 7f1158a00700 1 -- 172.21.15.151:0/4141782553 wait complete. 2024-05-21T21:59:56.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:56.989+0000 7f1158a00700 1 Processor -- start 2024-05-21T21:59:56.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:56.989+0000 7f1158a00700 1 -- start start 2024-05-21T21:59:56.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:56.990+0000 7f1158a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1154103460 0x7f115419f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:56.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:56.990+0000 7f1158a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1154103e20 0x7f115419fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:56.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:56.990+0000 7f1158a00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f11541094f0 0x7f1154199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:56.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:56.990+0000 7f1158a00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1154113600 con 0x7f1154103460 2024-05-21T21:59:56.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:56.990+0000 7f1158a00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1154113480 con 0x7f1154103e20 2024-05-21T21:59:56.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:56.990+0000 7f1158a00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1154113780 con 0x7f11541094f0 2024-05-21T21:59:56.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:56.990+0000 7f1151e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1154103e20 0x7f115419fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:56.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:56.990+0000 7f1151e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1154103e20 0x7f115419fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:37624/0 (socket says 172.21.15.151:37624) 2024-05-21T21:59:56.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:56.990+0000 7f1151e00700 1 -- 172.21.15.151:0/3004138496 learned_addr learned my addr 172.21.15.151:0/3004138496 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:59:56.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:56.990+0000 7f1152800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1154103460 0x7f115419f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:56.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:56.990+0000 7f1153200700 1 --2- 172.21.15.151:0/3004138496 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f11541094f0 0x7f1154199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:56.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:56.990+0000 7f1151e00700 1 -- 172.21.15.151:0/3004138496 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f11541094f0 msgr2=0x7f1154199960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:59:56.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:56.990+0000 7f1151e00700 1 --2- 172.21.15.151:0/3004138496 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f11541094f0 0x7f1154199960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:56.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:56.990+0000 7f1151e00700 1 -- 172.21.15.151:0/3004138496 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1154103460 msgr2=0x7f115419f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:59:56.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:56.990+0000 7f1151e00700 1 --2- 172.21.15.151:0/3004138496 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1154103460 0x7f115419f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:56.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:56.990+0000 7f1151e00700 1 -- 172.21.15.151:0/3004138496 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f115419a0c0 con 0x7f1154103e20 2024-05-21T21:59:56.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:56.991+0000 7f1151e00700 1 --2- 172.21.15.151:0/3004138496 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1154103e20 0x7f115419fce0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f11400057f0 tx=0x7f114000bfd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:59:56.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:56.991+0000 7f114b400700 1 -- 172.21.15.151:0/3004138496 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1140026070 con 0x7f1154103e20 2024-05-21T21:59:56.994 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:56.991+0000 7f114b400700 1 -- 172.21.15.151:0/3004138496 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1140021410 con 0x7f1154103e20 2024-05-21T21:59:56.994 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:56.991+0000 7f1158a00700 1 -- 172.21.15.151:0/3004138496 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f115419a350 con 0x7f1154103e20 2024-05-21T21:59:56.994 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:56.991+0000 7f114b400700 1 -- 172.21.15.151:0/3004138496 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1140017650 con 0x7f1154103e20 2024-05-21T21:59:56.994 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:56.991+0000 7f1158a00700 1 -- 172.21.15.151:0/3004138496 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11541a6670 con 0x7f1154103e20 2024-05-21T21:59:56.996 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:56.995+0000 7f114b400700 1 -- 172.21.15.151:0/3004138496 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7f1140029070 con 0x7f1154103e20 2024-05-21T21:59:56.997 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:56.996+0000 7f1163e39b80 1 Processor -- start 2024-05-21T21:59:56.997 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:56.996+0000 7f1163e39b80 1 -- start start 2024-05-21T21:59:56.998 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:56.996+0000 7f1163e39b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564450a6a4a0 0x564450a6a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:56.998 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:56.996+0000 7f1163e39b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564450a6adc0 0x564450a648c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:56.998 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:56.996+0000 7f1163e39b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564450a64ee0 0x564450a65340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:56.998 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:56.996+0000 7f1163e39b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5644509bee40 con 0x564450a6adc0 2024-05-21T21:59:56.998 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:56.996+0000 7f1163e39b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5644509ae9e0 con 0x564450a6a4a0 2024-05-21T21:59:56.998 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:56.996+0000 7f1163e39b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5644509b2a20 con 0x564450a64ee0 2024-05-21T21:59:56.999 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:56.997+0000 7f1153200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564450a6a4a0 0x564450a6a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:56.999 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:56.997+0000 7f1153200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564450a6a4a0 0x564450a6a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:37636/0 (socket says 172.21.15.151:37636) 2024-05-21T21:59:56.999 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:56.997+0000 7f1153200700 1 -- 172.21.15.151:0/247210614 learned_addr learned my addr 172.21.15.151:0/247210614 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:59:56.999 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:56.997+0000 7f1151e00700 1 --2- 172.21.15.151:0/247210614 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564450a64ee0 0x564450a65340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:56.999 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:56.997+0000 7f1152800700 1 --2- 172.21.15.151:0/247210614 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564450a6adc0 0x564450a648c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:56.999 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:56.997+0000 7f114b400700 1 --2- 172.21.15.151:0/3004138496 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f11380b29f0 0x7f11380b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:56.999 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:56.997+0000 7f114b400700 1 -- 172.21.15.151:0/3004138496 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f11400e32f0 con 0x7f1154103e20 2024-05-21T21:59:56.999 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:56.997+0000 7f1153200700 1 -- 172.21.15.151:0/247210614 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564450a64ee0 msgr2=0x564450a65340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:59:57.000 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:56.997+0000 7f1153200700 1 --2- 172.21.15.151:0/247210614 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564450a64ee0 0x564450a65340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:57.000 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:56.997+0000 7f1153200700 1 -- 172.21.15.151:0/247210614 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564450a6adc0 msgr2=0x564450a648c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:59:57.000 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:56.997+0000 7f1153200700 1 --2- 172.21.15.151:0/247210614 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564450a6adc0 0x564450a648c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:57.000 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:56.997+0000 7f1153200700 1 -- 172.21.15.151:0/247210614 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564450977420 con 0x564450a6a4a0 2024-05-21T21:59:57.000 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:56.998+0000 7f1153200700 1 --2- 172.21.15.151:0/3004138496 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f11380b29f0 0x7f11380b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:57.000 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:56.998+0000 7f1153200700 1 --2- 172.21.15.151:0/247210614 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564450a6a4a0 0x564450a6a880 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f114c011740 tx=0x7f114c011a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3897dc4011d1021 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:59:57.001 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:56.998+0000 7f1153200700 1 --2- 172.21.15.151:0/3004138496 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f11380b29f0 0x7f11380b4eb0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f114c022000 tx=0x7f114c023000 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:59:57.001 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:56.998+0000 7f113fe00700 1 -- 172.21.15.151:0/247210614 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f114c01c040 con 0x564450a6a4a0 2024-05-21T21:59:57.001 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:56.998+0000 7f113fe00700 1 -- 172.21.15.151:0/247210614 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f114c0047c0 con 0x564450a6a4a0 2024-05-21T21:59:57.001 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:56.998+0000 7f113fe00700 1 -- 172.21.15.151:0/247210614 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f114c023670 con 0x564450a6a4a0 2024-05-21T21:59:57.001 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:56.999+0000 7f1163e39b80 1 -- 172.21.15.151:0/247210614 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564450a6a4a0 msgr2=0x564450a6a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:59:57.002 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:56.999+0000 7f1163e39b80 1 --2- 172.21.15.151:0/247210614 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564450a6a4a0 0x564450a6a880 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f114c011740 tx=0x7f114c011a50 comp rx=0 tx=0).stop 2024-05-21T21:59:57.002 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:56.999+0000 7f1163e39b80 1 -- 172.21.15.151:0/247210614 shutdown_connections 2024-05-21T21:59:57.002 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:56.999+0000 7f1163e39b80 1 --2- 172.21.15.151:0/247210614 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564450a6a4a0 0x564450a6a880 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:57.002 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:56.999+0000 7f1163e39b80 1 --2- 172.21.15.151:0/247210614 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564450a64ee0 0x564450a65340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:57.002 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:56.999+0000 7f1163e39b80 1 --2- 172.21.15.151:0/247210614 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564450a6adc0 0x564450a648c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:57.002 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:56.999+0000 7f1163e39b80 1 -- 172.21.15.151:0/247210614 >> 172.21.15.151:0/247210614 conn(0x56445098ebe0 msgr2=0x564450a5b150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:59:57.002 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:56.999+0000 7f1163e39b80 1 -- 172.21.15.151:0/247210614 shutdown_connections 2024-05-21T21:59:57.002 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:56.999+0000 7f1163e39b80 1 -- 172.21.15.151:0/247210614 wait complete. 2024-05-21T21:59:57.002 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.000+0000 7f1163e39b80 1 -- start start 2024-05-21T21:59:57.002 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.000+0000 7f1163e39b80 1 Processor -- start 2024-05-21T21:59:57.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.001+0000 7f1163e39b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564450a64ee0 0x564450a70a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:57.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.001+0000 7f1163e39b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564450a6a4a0 0x564450a6de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:57.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.001+0000 7f1163e39b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564450a6adc0 0x564450a6e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:57.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.001+0000 7f1163e39b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5644508b4fa0 con 0x564450a6a4a0 2024-05-21T21:59:57.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.001+0000 7f1163e39b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56445094f760 con 0x564450a64ee0 2024-05-21T21:59:57.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.001+0000 7f1163e39b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x564450a71850 con 0x564450a6adc0 2024-05-21T21:59:57.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.001+0000 7f1163e39b80 10 client.0 _ll_register_callbacks cb 0x564450958c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:59:57.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.001+0000 7f1151e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564450a6a4a0 0x564450a6de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:57.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.001+0000 7f1153200700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564450a6adc0 0x564450a6e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:57.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.001+0000 7f1152800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564450a64ee0 0x564450a70a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:57.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.001+0000 7f1153200700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564450a6adc0 0x564450a6e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:37910/0 (socket says 172.21.15.151:37910) 2024-05-21T21:59:57.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.001+0000 7f1153200700 1 -- 172.21.15.151:0/3051584586 learned_addr learned my addr 172.21.15.151:0/3051584586 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:59:57.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.001+0000 7f1152800700 1 --2- 172.21.15.151:0/3051584586 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564450a64ee0 0x564450a70a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:37642/0 (socket says 172.21.15.151:37642) 2024-05-21T21:59:57.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.001+0000 7f1151e00700 1 -- 172.21.15.151:0/3051584586 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564450a6adc0 msgr2=0x564450a6e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:59:57.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.001+0000 7f1151e00700 1 --2- 172.21.15.151:0/3051584586 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564450a6adc0 0x564450a6e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:57.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.001+0000 7f1151e00700 1 -- 172.21.15.151:0/3051584586 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564450a64ee0 msgr2=0x564450a70a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:59:57.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.001+0000 7f1151e00700 1 --2- 172.21.15.151:0/3051584586 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564450a64ee0 0x564450a70a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:57.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.001+0000 7f1151e00700 1 -- 172.21.15.151:0/3051584586 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644508fe8d0 con 0x564450a6a4a0 2024-05-21T21:59:57.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.002+0000 7f1153200700 1 --2- 172.21.15.151:0/3051584586 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564450a6adc0 0x564450a6e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T21:59:57.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.002+0000 7f1151e00700 1 --2- 172.21.15.151:0/3051584586 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564450a6a4a0 0x564450a6de20 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f11400e09f0 tx=0x7f1140021c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e86d5a1ea4bd68dd server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:59:57.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.002+0000 7f113cc00700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-21T21:59:57.006 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.002+0000 7f113cc00700 1 -- 172.21.15.151:0/3051584586 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11400a9f60 con 0x564450a6a4a0 2024-05-21T21:59:57.006 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.002+0000 7f113cc00700 1 -- 172.21.15.151:0/3051584586 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f11400b2f50 con 0x564450a6a4a0 2024-05-21T21:59:57.006 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.002+0000 7f1163e39b80 1 -- 172.21.15.151:0/3051584586 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f114c0047c0 con 0x564450a6a4a0 2024-05-21T21:59:57.006 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.002+0000 7f113cc00700 1 -- 172.21.15.151:0/3051584586 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f11400a9d80 con 0x564450a6a4a0 2024-05-21T21:59:57.006 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.002+0000 7f1163e39b80 10 client.16071 fetch_fsmap learned FSMap version 29 2024-05-21T21:59:57.006 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.002+0000 7f1163e39b80 1 -- 172.21.15.151:0/3051584586 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56445097f060 con 0x564450a6a4a0 2024-05-21T21:59:57.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.003+0000 7f113cc00700 1 -- 172.21.15.151:0/3051584586 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f114c0047c0 con 0x564450a6a4a0 2024-05-21T21:59:57.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.003+0000 7f113cc00700 1 -- 172.21.15.151:0/3051584586 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f11400050e0 con 0x564450a6a4a0 2024-05-21T21:59:57.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.003+0000 7f1163e39b80 10 client.16071 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T21:59:57.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.003+0000 7f1163e39b80 10 client.16071 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T21:59:57.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.003+0000 7f1163e39b80 20 client.16071 populate_metadata read hostname 'smithi151' 2024-05-21T21:59:57.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.003+0000 7f1163e39b80 1 --2- 172.21.15.151:0/3051584586 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x564450a7e300 0x564450a80760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:57.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.003+0000 7f1163e39b80 4 client.16071 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T21:59:57.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.003+0000 7f1163e39b80 1 -- 172.21.15.151:0/3051584586 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5644509c0d90 con 0x564450a7e300 2024-05-21T21:59:57.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.004+0000 7f1152800700 1 --2- 172.21.15.151:0/3051584586 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x564450a7e300 0x564450a80760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:57.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.004+0000 7f1152800700 1 --2- 172.21.15.151:0/3051584586 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x564450a7e300 0x564450a80760 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7900b26255a640d server_cookie=738d627dc046b132 in_seq=0 out_seq=0 2024-05-21T21:59:57.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.004+0000 7f113cc00700 10 client.16071 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T21:59:57.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.005+0000 7f113cc00700 1 -- 172.21.15.151:0/3051584586 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5644509c0d90 con 0x564450a7e300 2024-05-21T21:59:57.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.005+0000 7f113cc00700 10 client.16071 handle_command_reply: tid=0 2024-05-21T21:59:57.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.005+0000 7f1163e39b80 1 client.16071 shutdown 2024-05-21T21:59:57.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.005+0000 7f1163e39b80 1 -- 172.21.15.151:0/3051584586 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564450a6a4a0 msgr2=0x564450a6de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:59:57.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.005+0000 7f1163e39b80 1 --2- 172.21.15.151:0/3051584586 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564450a6a4a0 0x564450a6de20 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f11400e09f0 tx=0x7f1140021c60 comp rx=0 tx=0).stop 2024-05-21T21:59:57.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.005+0000 7f1163e39b80 1 -- 172.21.15.151:0/3051584586 shutdown_connections 2024-05-21T21:59:57.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.005+0000 7f1163e39b80 1 --2- 172.21.15.151:0/3051584586 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x564450a7e300 0x564450a80760 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:57.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.005+0000 7f1163e39b80 1 --2- 172.21.15.151:0/3051584586 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564450a64ee0 0x564450a70a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:57.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.005+0000 7f1163e39b80 1 --2- 172.21.15.151:0/3051584586 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564450a6adc0 0x564450a6e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:57.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.005+0000 7f113cc00700 0 client.16071 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T21:59:57.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.005+0000 7f1163e39b80 1 --2- 172.21.15.151:0/3051584586 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564450a6a4a0 0x564450a6de20 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:57.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.005+0000 7f1163e39b80 1 -- 172.21.15.151:0/3051584586 >> 172.21.15.151:0/3051584586 conn(0x56445098ebe0 msgr2=0x564450a5f4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:59:57.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.006+0000 7f1163e39b80 1 -- 172.21.15.151:0/3051584586 shutdown_connections 2024-05-21T21:59:57.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.006+0000 7f1163e39b80 1 -- 172.21.15.151:0/3051584586 wait complete. 2024-05-21T21:59:57.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.006+0000 7f1163e39b80 20 client.16071 trim_cache size 0 max 16384 2024-05-21T21:59:57.024 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.022+0000 7f1163e39b80 1 Processor -- start 2024-05-21T21:59:57.024 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.022+0000 7f1163e39b80 1 -- start start 2024-05-21T21:59:57.025 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.023+0000 7f1163e39b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564450a64ee0 0x564450a652c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:57.025 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.023+0000 7f1163e39b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564450a6a4a0 0x564450a67e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:57.025 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.023+0000 7f1163e39b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564450a6adc0 0x564450a683d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:57.025 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.023+0000 7f1163e39b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f11541aba30 con 0x564450a6a4a0 2024-05-21T21:59:57.025 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.023+0000 7f1163e39b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f11541ad570 con 0x564450a6adc0 2024-05-21T21:59:57.025 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.023+0000 7f1163e39b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f11541ac100 con 0x564450a64ee0 2024-05-21T21:59:57.025 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.023+0000 7f1151e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564450a6a4a0 0x564450a67e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:57.025 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.023+0000 7f1153200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564450a6adc0 0x564450a683d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:57.025 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.023+0000 7f1151e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564450a6a4a0 0x564450a67e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:36228/0 (socket says 172.21.15.151:36228) 2024-05-21T21:59:57.025 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.023+0000 7f1151e00700 1 -- 172.21.15.151:0/2095616015 learned_addr learned my addr 172.21.15.151:0/2095616015 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:59:57.025 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.023+0000 7f1152800700 1 --2- 172.21.15.151:0/2095616015 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564450a64ee0 0x564450a652c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:57.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.024+0000 7f1153200700 1 -- 172.21.15.151:0/2095616015 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564450a64ee0 msgr2=0x564450a652c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:59:57.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.024+0000 7f1153200700 1 --2- 172.21.15.151:0/2095616015 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564450a64ee0 0x564450a652c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:57.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.024+0000 7f1153200700 1 -- 172.21.15.151:0/2095616015 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564450a6a4a0 msgr2=0x564450a67e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:59:57.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.024+0000 7f1153200700 1 --2- 172.21.15.151:0/2095616015 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564450a6a4a0 0x564450a67e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:57.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.024+0000 7f1153200700 1 -- 172.21.15.151:0/2095616015 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564450a5ce20 con 0x564450a6adc0 2024-05-21T21:59:57.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.024+0000 7f1153200700 1 --2- 172.21.15.151:0/2095616015 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564450a6adc0 0x564450a683d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f114c007f50 tx=0x7f114c0095a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5691183c45595662 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:59:57.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.024+0000 7f113f400700 1 -- 172.21.15.151:0/2095616015 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f114c01c040 con 0x564450a6adc0 2024-05-21T21:59:57.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.024+0000 7f113f400700 1 -- 172.21.15.151:0/2095616015 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f114c0049c0 con 0x564450a6adc0 2024-05-21T21:59:57.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.024+0000 7f113f400700 1 -- 172.21.15.151:0/2095616015 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f114c02db00 con 0x564450a6adc0 2024-05-21T21:59:57.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.025+0000 7f1163e39b80 1 -- 172.21.15.151:0/2095616015 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564450a6adc0 msgr2=0x564450a683d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:59:57.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.025+0000 7f1163e39b80 1 --2- 172.21.15.151:0/2095616015 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564450a6adc0 0x564450a683d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f114c007f50 tx=0x7f114c0095a0 comp rx=0 tx=0).stop 2024-05-21T21:59:57.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.025+0000 7f1163e39b80 1 -- 172.21.15.151:0/2095616015 shutdown_connections 2024-05-21T21:59:57.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.025+0000 7f1163e39b80 1 --2- 172.21.15.151:0/2095616015 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564450a6adc0 0x564450a683d0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:57.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.025+0000 7f1163e39b80 1 --2- 172.21.15.151:0/2095616015 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564450a64ee0 0x564450a652c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:57.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.025+0000 7f1163e39b80 1 --2- 172.21.15.151:0/2095616015 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564450a6a4a0 0x564450a67e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:57.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.025+0000 7f1163e39b80 1 -- 172.21.15.151:0/2095616015 >> 172.21.15.151:0/2095616015 conn(0x56445098ebe0 msgr2=0x564450a6c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:59:57.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.025+0000 7f1163e39b80 1 -- 172.21.15.151:0/2095616015 shutdown_connections 2024-05-21T21:59:57.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.025+0000 7f1163e39b80 1 -- 172.21.15.151:0/2095616015 wait complete. 2024-05-21T21:59:57.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.025+0000 7f1163e39b80 1 -- start start 2024-05-21T21:59:57.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.026+0000 7f1163e39b80 1 Processor -- start 2024-05-21T21:59:57.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.026+0000 7f1163e39b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564450a64ee0 0x564450a85b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:57.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.026+0000 7f1163e39b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564450a6a4a0 0x564450a860d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:57.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.026+0000 7f1163e39b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564450a6adc0 0x564450a86610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:57.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.026+0000 7f1163e39b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f11541acea0 con 0x564450a64ee0 2024-05-21T21:59:57.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.026+0000 7f1163e39b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f11541ac7d0 con 0x564450a6a4a0 2024-05-21T21:59:57.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.026+0000 7f1163e39b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f11541a99d0 con 0x564450a6adc0 2024-05-21T21:59:57.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.026+0000 7f1163e39b80 10 client.0 _ll_register_callbacks cb 0x564450958c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T21:59:57.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.026+0000 7f1152800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564450a64ee0 0x564450a85b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:57.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.027+0000 7f1152800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564450a64ee0 0x564450a85b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:36234/0 (socket says 172.21.15.151:36234) 2024-05-21T21:59:57.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.027+0000 7f1152800700 1 -- 172.21.15.151:0/3602435293 learned_addr learned my addr 172.21.15.151:0/3602435293 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T21:59:57.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.027+0000 7f1151e00700 1 --2- 172.21.15.151:0/3602435293 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564450a6a4a0 0x564450a860d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:57.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.027+0000 7f1153200700 1 --2- 172.21.15.151:0/3602435293 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564450a6adc0 0x564450a86610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:57.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.027+0000 7f1151e00700 1 -- 172.21.15.151:0/3602435293 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564450a6adc0 msgr2=0x564450a86610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:59:57.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.027+0000 7f1151e00700 1 --2- 172.21.15.151:0/3602435293 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564450a6adc0 0x564450a86610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:57.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.027+0000 7f1151e00700 1 -- 172.21.15.151:0/3602435293 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564450a64ee0 msgr2=0x564450a85b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T21:59:57.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.027+0000 7f1151e00700 1 --2- 172.21.15.151:0/3602435293 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564450a64ee0 0x564450a85b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:57.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.027+0000 7f1151e00700 1 -- 172.21.15.151:0/3602435293 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564450ae7f80 con 0x564450a6a4a0 2024-05-21T21:59:57.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.027+0000 7f1151e00700 1 --2- 172.21.15.151:0/3602435293 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564450a6a4a0 0x564450a860d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f11400aa5a0 tx=0x7f11400aabd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eaadc9bcc47b88d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T21:59:57.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.027+0000 7f113cc00700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-21T21:59:57.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.027+0000 7f113cc00700 1 -- 172.21.15.151:0/3602435293 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11400a9f60 con 0x564450a6a4a0 2024-05-21T21:59:57.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.028+0000 7f113cc00700 1 -- 172.21.15.151:0/3602435293 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f1140005110 con 0x564450a6a4a0 2024-05-21T21:59:57.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.028+0000 7f1163e39b80 1 -- 172.21.15.151:0/3602435293 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f114c0049c0 con 0x564450a6a4a0 2024-05-21T21:59:57.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.028+0000 7f113cc00700 1 -- 172.21.15.151:0/3602435293 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f11400aa120 con 0x564450a6a4a0 2024-05-21T21:59:57.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.028+0000 7f1163e39b80 10 client.26198 fetch_fsmap learned FSMap version 29 2024-05-21T21:59:57.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.028+0000 7f1163e39b80 1 -- 172.21.15.151:0/3602435293 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564450ae9240 con 0x564450a6a4a0 2024-05-21T21:59:57.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.028+0000 7f113cc00700 1 -- 172.21.15.151:0/3602435293 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f114c0049c0 con 0x564450a6a4a0 2024-05-21T21:59:57.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.028+0000 7f113cc00700 1 -- 172.21.15.151:0/3602435293 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f1140090bc0 con 0x564450a6a4a0 2024-05-21T21:59:57.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.028+0000 7f1163e39b80 10 client.26198 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T21:59:57.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.029+0000 7f1163e39b80 10 client.26198 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T21:59:57.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.029+0000 7f1163e39b80 20 client.26198 populate_metadata read hostname 'smithi151' 2024-05-21T21:59:57.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.029+0000 7f1163e39b80 1 --2- 172.21.15.151:0/3602435293 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x564450ae9710 0x564450aebb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T21:59:57.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.029+0000 7f1163e39b80 4 client.26198 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T21:59:57.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.029+0000 7f1163e39b80 1 -- 172.21.15.151:0/3602435293 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5644509ba570 con 0x564450ae9710 2024-05-21T21:59:57.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.029+0000 7f1152800700 1 --2- 172.21.15.151:0/3602435293 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x564450ae9710 0x564450aebb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T21:59:57.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.029+0000 7f1152800700 1 --2- 172.21.15.151:0/3602435293 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x564450ae9710 0x564450aebb30 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c40649b4618bc397 server_cookie=395e5edd5fd382ac in_seq=0 out_seq=0 2024-05-21T21:59:57.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.029+0000 7f113cc00700 10 client.26198 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T21:59:57.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.030+0000 7f113cc00700 1 -- 172.21.15.151:0/3602435293 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5644509ba570 con 0x564450ae9710 2024-05-21T21:59:57.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.030+0000 7f113cc00700 10 client.26198 handle_command_reply: tid=0 2024-05-21T21:59:57.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.030+0000 7f1163e39b80 1 client.26198 shutdown 2024-05-21T21:59:57.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.030+0000 7f1163e39b80 1 -- 172.21.15.151:0/3602435293 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564450a6a4a0 msgr2=0x564450a860d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:59:57.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.030+0000 7f1163e39b80 1 --2- 172.21.15.151:0/3602435293 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564450a6a4a0 0x564450a860d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f11400aa5a0 tx=0x7f11400aabd0 comp rx=0 tx=0).stop 2024-05-21T21:59:57.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.030+0000 7f1163e39b80 1 -- 172.21.15.151:0/3602435293 shutdown_connections 2024-05-21T21:59:57.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.030+0000 7f1163e39b80 1 --2- 172.21.15.151:0/3602435293 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564450a6adc0 0x564450a86610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:57.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.030+0000 7f1163e39b80 1 --2- 172.21.15.151:0/3602435293 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564450a64ee0 0x564450a85b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:57.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.030+0000 7f1163e39b80 1 --2- 172.21.15.151:0/3602435293 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x564450ae9710 0x564450aebb30 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:57.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.030+0000 7f1163e39b80 1 --2- 172.21.15.151:0/3602435293 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564450a6a4a0 0x564450a860d0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:57.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.030+0000 7f1163e39b80 1 -- 172.21.15.151:0/3602435293 >> 172.21.15.151:0/3602435293 conn(0x56445098ebe0 msgr2=0x564450a6c650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:59:57.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.030+0000 7f113cc00700 0 client.26198 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T21:59:57.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.030+0000 7f1163e39b80 1 -- 172.21.15.151:0/3602435293 shutdown_connections 2024-05-21T21:59:57.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.030+0000 7f1163e39b80 1 -- 172.21.15.151:0/3602435293 wait complete. 2024-05-21T21:59:57.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.030+0000 7f1163e39b80 20 client.26198 trim_cache size 0 max 16384 2024-05-21T21:59:57.036 INFO:teuthology.orchestra.run.smithi151.stdout:[] 2024-05-21T21:59:57.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.031+0000 7f1158a00700 1 -- 172.21.15.151:0/3004138496 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f11380b29f0 msgr2=0x7f11380b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:59:57.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.031+0000 7f1158a00700 1 --2- 172.21.15.151:0/3004138496 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f11380b29f0 0x7f11380b4eb0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f114c022000 tx=0x7f114c023000 comp rx=0 tx=0).stop 2024-05-21T21:59:57.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.031+0000 7f1158a00700 1 -- 172.21.15.151:0/3004138496 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1154103e20 msgr2=0x7f115419fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T21:59:57.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.031+0000 7f1158a00700 1 --2- 172.21.15.151:0/3004138496 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1154103e20 0x7f115419fce0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f11400057f0 tx=0x7f114000bfd0 comp rx=0 tx=0).stop 2024-05-21T21:59:57.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.031+0000 7f1158a00700 1 -- 172.21.15.151:0/3004138496 shutdown_connections 2024-05-21T21:59:57.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.031+0000 7f1158a00700 1 --2- 172.21.15.151:0/3004138496 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f11380b29f0 0x7f11380b4eb0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:57.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.031+0000 7f1158a00700 1 --2- 172.21.15.151:0/3004138496 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f11541094f0 0x7f1154199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:57.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.031+0000 7f1158a00700 1 --2- 172.21.15.151:0/3004138496 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1154103460 0x7f115419f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:57.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.031+0000 7f1158a00700 1 --2- 172.21.15.151:0/3004138496 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1154103e20 0x7f115419fce0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T21:59:57.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.031+0000 7f1158a00700 1 -- 172.21.15.151:0/3004138496 >> 172.21.15.151:0/3004138496 conn(0x7f11540fef90 msgr2=0x7f11540ff7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T21:59:57.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.031+0000 7f1158a00700 1 -- 172.21.15.151:0/3004138496 shutdown_connections 2024-05-21T21:59:57.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T21:59:57.031+0000 7f1158a00700 1 -- 172.21.15.151:0/3004138496 wait complete. 2024-05-21T21:59:57.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:59:57 smithi151 ceph-mon[31623]: pgmap v133: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 14 op/s 2024-05-21T21:59:57.735 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:59:57 smithi184 ceph-mon[36942]: pgmap v133: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 14 op/s 2024-05-21T21:59:57.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:59:57 smithi164 ceph-mon[36269]: pgmap v133: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 14 op/s 2024-05-21T21:59:58.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:59:58 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:59:58.735 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:59:58 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:59:58.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:59:58 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T21:59:58.956 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-21T21:59:58.957 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-21T21:59:59.396 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T21:59:59.716 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:59:59 smithi151 ceph-mon[31623]: pgmap v134: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 2.4 KiB/s wr, 18 op/s 2024-05-21T21:59:59.716 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:59:59 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:59:59.716 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:59:59 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:59:59.716 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 21:59:59 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T21:59:59.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:59:59 smithi184 ceph-mon[36942]: pgmap v134: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 2.4 KiB/s wr, 18 op/s 2024-05-21T21:59:59.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:59:59 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:59:59.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:59:59 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:59:59.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 21:59:59 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T21:59:59.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:59:59 smithi164 ceph-mon[36269]: pgmap v134: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 2.4 KiB/s wr, 18 op/s 2024-05-21T21:59:59.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:59:59 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T21:59:59.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:59:59 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T21:59:59.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 21:59:59 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T22:00:00.591 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:00:00 smithi151 ceph-mon[31623]: overall HEALTH_OK 2024-05-21T22:00:00.735 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:00:00 smithi184 ceph-mon[36942]: overall HEALTH_OK 2024-05-21T22:00:00.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:00:00 smithi164 ceph-mon[36269]: overall HEALTH_OK 2024-05-21T22:00:01.501 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:00:01 smithi151 ceph-mon[31623]: pgmap v135: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 682 B/s wr, 8 op/s 2024-05-21T22:00:01.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:00:01 smithi184 ceph-mon[36942]: pgmap v135: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 682 B/s wr, 8 op/s 2024-05-21T22:00:01.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:00:01 smithi164 ceph-mon[36269]: pgmap v135: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 682 B/s wr, 8 op/s 2024-05-21T22:00:01.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.786+0000 7fdf0c800700 1 -- 172.21.15.151:0/766158701 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdf041088f0 msgr2=0x7fdf04110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:01.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.786+0000 7fdf0c800700 1 --2- 172.21.15.151:0/766158701 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdf041088f0 0x7fdf04110470 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fdf0000b0d0 tx=0x7fdf0000b3e0 comp rx=0 tx=0).stop 2024-05-21T22:00:01.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.786+0000 7fdf0c800700 1 -- 172.21.15.151:0/766158701 shutdown_connections 2024-05-21T22:00:01.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.786+0000 7fdf0c800700 1 --2- 172.21.15.151:0/766158701 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf041037f0 0x7fdf04103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:01.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.786+0000 7fdf0c800700 1 --2- 172.21.15.151:0/766158701 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdf041088f0 0x7fdf04110470 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:01.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.786+0000 7fdf0c800700 1 --2- 172.21.15.151:0/766158701 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdf041041a0 0x7fdf041081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:01.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.786+0000 7fdf0c800700 1 -- 172.21.15.151:0/766158701 >> 172.21.15.151:0/766158701 conn(0x7fdf0407a6a0 msgr2=0x7fdf0407aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:00:01.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.787+0000 7fdf0c800700 1 -- 172.21.15.151:0/766158701 shutdown_connections 2024-05-21T22:00:01.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.787+0000 7fdf0c800700 1 -- 172.21.15.151:0/766158701 wait complete. 2024-05-21T22:00:01.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.788+0000 7fdf0c800700 1 Processor -- start 2024-05-21T22:00:01.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.788+0000 7fdf0c800700 1 -- start start 2024-05-21T22:00:01.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.788+0000 7fdf0c800700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdf041037f0 0x7fdf041a1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:01.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.788+0000 7fdf0c800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdf041041a0 0x7fdf041a1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:01.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.788+0000 7fdf0c800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf041088f0 0x7fdf0419bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:01.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.788+0000 7fdf0c800700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdf041136c0 con 0x7fdf041041a0 2024-05-21T22:00:01.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.788+0000 7fdf0c800700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdf04113540 con 0x7fdf041088f0 2024-05-21T22:00:01.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.788+0000 7fdf0c800700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fdf04113840 con 0x7fdf041037f0 2024-05-21T22:00:01.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.789+0000 7fdf09400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdf041041a0 0x7fdf041a1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:01.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.789+0000 7fdf09400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdf041041a0 0x7fdf041a1f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:36248/0 (socket says 172.21.15.151:36248) 2024-05-21T22:00:01.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.789+0000 7fdf09400700 1 -- 172.21.15.151:0/3313907893 learned_addr learned my addr 172.21.15.151:0/3313907893 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:00:01.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.789+0000 7fdf09e00700 1 --2- 172.21.15.151:0/3313907893 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdf041037f0 0x7fdf041a1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:01.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.789+0000 7fdf0a800700 1 --2- 172.21.15.151:0/3313907893 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf041088f0 0x7fdf0419bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:01.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.789+0000 7fdf09400700 1 -- 172.21.15.151:0/3313907893 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdf041037f0 msgr2=0x7fdf041a1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:01.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.789+0000 7fdf09400700 1 --2- 172.21.15.151:0/3313907893 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdf041037f0 0x7fdf041a1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:01.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.789+0000 7fdf09400700 1 -- 172.21.15.151:0/3313907893 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf041088f0 msgr2=0x7fdf0419bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:01.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.789+0000 7fdf09400700 1 --2- 172.21.15.151:0/3313907893 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf041088f0 0x7fdf0419bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:01.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.789+0000 7fdf09400700 1 -- 172.21.15.151:0/3313907893 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf0419c3f0 con 0x7fdf041041a0 2024-05-21T22:00:01.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.789+0000 7fdf09400700 1 --2- 172.21.15.151:0/3313907893 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdf041041a0 0x7fdf041a1f80 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fdef400e9a0 tx=0x7fdef400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:00:01.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.790+0000 7fdefea00700 1 -- 172.21.15.151:0/3313907893 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdef400ca10 con 0x7fdf041041a0 2024-05-21T22:00:01.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.790+0000 7fdefea00700 1 -- 172.21.15.151:0/3313907893 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdef400cbb0 con 0x7fdf041041a0 2024-05-21T22:00:01.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.790+0000 7fdf0c800700 1 -- 172.21.15.151:0/3313907893 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf0419c6e0 con 0x7fdf041041a0 2024-05-21T22:00:01.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.790+0000 7fdefea00700 1 -- 172.21.15.151:0/3313907893 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdef4018680 con 0x7fdf041041a0 2024-05-21T22:00:01.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.790+0000 7fdf0c800700 1 -- 172.21.15.151:0/3313907893 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf041a89c0 con 0x7fdf041041a0 2024-05-21T22:00:01.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.794+0000 7fdefea00700 1 -- 172.21.15.151:0/3313907893 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7fdef4010430 con 0x7fdf041041a0 2024-05-21T22:00:01.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.794+0000 7fdf170cab80 1 Processor -- start 2024-05-21T22:00:01.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.795+0000 7fdf170cab80 1 -- start start 2024-05-21T22:00:01.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.795+0000 7fdf170cab80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a7b0a50f30 0x55a7b0a51310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:01.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.795+0000 7fdf170cab80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a7b0a51850 0x55a7b0a4b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:01.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.795+0000 7fdf170cab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a7b0a4b900 0x55a7b0a4bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:01.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.795+0000 7fdf170cab80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a7b09a9ec0 con 0x55a7b0a50f30 2024-05-21T22:00:01.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.795+0000 7fdf170cab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55a7b0999a60 con 0x55a7b0a4b900 2024-05-21T22:00:01.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.795+0000 7fdf170cab80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55a7b099daa0 con 0x55a7b0a51850 2024-05-21T22:00:01.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.795+0000 7fdf0a800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a7b0a50f30 0x55a7b0a51310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:01.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.796+0000 7fdf09400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a7b0a4b900 0x55a7b0a4bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:01.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.796+0000 7fdf0a800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a7b0a50f30 0x55a7b0a51310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:36250/0 (socket says 172.21.15.151:36250) 2024-05-21T22:00:01.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.796+0000 7fdf0a800700 1 -- 172.21.15.151:0/2942737721 learned_addr learned my addr 172.21.15.151:0/2942737721 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:00:01.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.796+0000 7fdf09e00700 1 --2- 172.21.15.151:0/2942737721 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a7b0a51850 0x55a7b0a4b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:01.797 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.796+0000 7fdf09e00700 1 -- 172.21.15.151:0/2942737721 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a7b0a4b900 msgr2=0x55a7b0a4bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:01.797 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.796+0000 7fdf09e00700 1 --2- 172.21.15.151:0/2942737721 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a7b0a4b900 0x55a7b0a4bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:01.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.796+0000 7fdf09e00700 1 -- 172.21.15.151:0/2942737721 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a7b0a50f30 msgr2=0x55a7b0a51310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:01.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.796+0000 7fdf09e00700 1 --2- 172.21.15.151:0/2942737721 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a7b0a50f30 0x55a7b0a51310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:01.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.796+0000 7fdf09e00700 1 -- 172.21.15.151:0/2942737721 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7b09624a0 con 0x55a7b0a51850 2024-05-21T22:00:01.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.796+0000 7fdf09400700 1 --2- 172.21.15.151:0/2942737721 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a7b0a4b900 0x55a7b0a4bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:00:01.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.796+0000 7fdefea00700 1 --2- 172.21.15.151:0/3313907893 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fdeec0b29f0 0x7fdeec0b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:01.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.796+0000 7fdf09e00700 1 --2- 172.21.15.151:0/2942737721 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a7b0a51850 0x55a7b0a4b2e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fdef8009cb0 tx=0x7fdef8011440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48f0a43b399afb78 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:00:01.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.797+0000 7fdefea00700 1 -- 172.21.15.151:0/3313907893 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fdef40e2760 con 0x7fdf041041a0 2024-05-21T22:00:01.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.797+0000 7fdf0a800700 1 --2- 172.21.15.151:0/3313907893 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fdeec0b29f0 0x7fdeec0b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:01.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.797+0000 7fdef3400700 1 -- 172.21.15.151:0/2942737721 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdef8011d20 con 0x55a7b0a51850 2024-05-21T22:00:01.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.797+0000 7fdef3400700 1 -- 172.21.15.151:0/2942737721 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdef80044e0 con 0x55a7b0a51850 2024-05-21T22:00:01.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.797+0000 7fdef3400700 1 -- 172.21.15.151:0/2942737721 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdef8005260 con 0x55a7b0a51850 2024-05-21T22:00:01.799 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.798+0000 7fdf170cab80 1 -- 172.21.15.151:0/2942737721 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a7b0a51850 msgr2=0x55a7b0a4b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:01.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.798+0000 7fdf0a800700 1 --2- 172.21.15.151:0/3313907893 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fdeec0b29f0 0x7fdeec0b4eb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fdf00009dc0 tx=0x7fdf0000b040 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:00:01.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.798+0000 7fdf170cab80 1 --2- 172.21.15.151:0/2942737721 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a7b0a51850 0x55a7b0a4b2e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fdef8009cb0 tx=0x7fdef8011440 comp rx=0 tx=0).stop 2024-05-21T22:00:01.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.798+0000 7fdf170cab80 1 -- 172.21.15.151:0/2942737721 shutdown_connections 2024-05-21T22:00:01.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.798+0000 7fdf170cab80 1 --2- 172.21.15.151:0/2942737721 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a7b0a4b900 0x55a7b0a4bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:01.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.798+0000 7fdf170cab80 1 --2- 172.21.15.151:0/2942737721 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a7b0a50f30 0x55a7b0a51310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:01.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.798+0000 7fdf170cab80 1 --2- 172.21.15.151:0/2942737721 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a7b0a51850 0x55a7b0a4b2e0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:01.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.798+0000 7fdf170cab80 1 -- 172.21.15.151:0/2942737721 >> 172.21.15.151:0/2942737721 conn(0x55a7b0979c60 msgr2=0x55a7b0a46090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:00:01.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.798+0000 7fdf170cab80 1 -- 172.21.15.151:0/2942737721 shutdown_connections 2024-05-21T22:00:01.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.798+0000 7fdf170cab80 1 -- 172.21.15.151:0/2942737721 wait complete. 2024-05-21T22:00:01.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.799+0000 7fdf170cab80 1 -- start start 2024-05-21T22:00:01.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.799+0000 7fdf170cab80 1 Processor -- start 2024-05-21T22:00:01.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.799+0000 7fdf170cab80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a7b0a4b900 0x55a7b0a574d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:01.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.799+0000 7fdf170cab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a7b0a50f30 0x55a7b0a54870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:01.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.799+0000 7fdf170cab80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a7b0a51850 0x55a7b0a54db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:01.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.799+0000 7fdf170cab80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a7b08a0020 con 0x55a7b0a4b900 2024-05-21T22:00:01.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.799+0000 7fdf170cab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55a7b093a7e0 con 0x55a7b0a50f30 2024-05-21T22:00:01.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.799+0000 7fdf170cab80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55a7b0a582c0 con 0x55a7b0a51850 2024-05-21T22:00:01.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.800+0000 7fdf170cab80 10 client.0 _ll_register_callbacks cb 0x55a7b0943cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:00:01.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.800+0000 7fdf09e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a7b0a4b900 0x55a7b0a574d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:01.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.800+0000 7fdf09e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a7b0a4b900 0x55a7b0a574d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:36254/0 (socket says 172.21.15.151:36254) 2024-05-21T22:00:01.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.800+0000 7fdf09e00700 1 -- 172.21.15.151:0/1646776841 learned_addr learned my addr 172.21.15.151:0/1646776841 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:00:01.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.800+0000 7fdf0a800700 1 --2- 172.21.15.151:0/1646776841 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a7b0a51850 0x55a7b0a54db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:01.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.800+0000 7fdf09400700 1 --2- 172.21.15.151:0/1646776841 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a7b0a50f30 0x55a7b0a54870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:01.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.800+0000 7fdf09e00700 1 -- 172.21.15.151:0/1646776841 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a7b0a51850 msgr2=0x55a7b0a54db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:01.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.800+0000 7fdf09e00700 1 --2- 172.21.15.151:0/1646776841 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a7b0a51850 0x55a7b0a54db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:01.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.800+0000 7fdf09e00700 1 -- 172.21.15.151:0/1646776841 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a7b0a50f30 msgr2=0x55a7b0a54870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:01.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.800+0000 7fdf09e00700 1 --2- 172.21.15.151:0/1646776841 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a7b0a50f30 0x55a7b0a54870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:01.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.800+0000 7fdf09e00700 1 -- 172.21.15.151:0/1646776841 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7b08e9950 con 0x55a7b0a4b900 2024-05-21T22:00:01.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.801+0000 7fdf09e00700 1 --2- 172.21.15.151:0/1646776841 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a7b0a4b900 0x55a7b0a574d0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fdef8004e60 tx=0x7fdef8011d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd42cd83647c81d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:00:01.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.801+0000 7fdee7e00700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-21T22:00:01.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.801+0000 7fdee7e00700 1 -- 172.21.15.151:0/1646776841 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdef80052d0 con 0x55a7b0a4b900 2024-05-21T22:00:01.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.801+0000 7fdee7e00700 1 -- 172.21.15.151:0/1646776841 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fdef8005470 con 0x55a7b0a4b900 2024-05-21T22:00:01.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.801+0000 7fdf170cab80 1 -- 172.21.15.151:0/1646776841 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdef80044e0 con 0x55a7b0a4b900 2024-05-21T22:00:01.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.801+0000 7fdee7e00700 1 -- 172.21.15.151:0/1646776841 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdef8032050 con 0x55a7b0a4b900 2024-05-21T22:00:01.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.801+0000 7fdf170cab80 10 client.16089 fetch_fsmap learned FSMap version 29 2024-05-21T22:00:01.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.801+0000 7fdf170cab80 1 -- 172.21.15.151:0/1646776841 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a7b096a0e0 con 0x55a7b0a4b900 2024-05-21T22:00:01.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.802+0000 7fdee7e00700 1 -- 172.21.15.151:0/1646776841 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdef80044e0 con 0x55a7b0a4b900 2024-05-21T22:00:01.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.802+0000 7fdee7e00700 1 -- 172.21.15.151:0/1646776841 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fdef8018070 con 0x55a7b0a4b900 2024-05-21T22:00:01.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.802+0000 7fdf170cab80 10 client.16089 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:00:01.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.802+0000 7fdf170cab80 10 client.16089 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T22:00:01.805 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.802+0000 7fdf170cab80 20 client.16089 populate_metadata read hostname 'smithi151' 2024-05-21T22:00:01.805 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.802+0000 7fdf170cab80 1 --2- 172.21.15.151:0/1646776841 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55a7b0a64cf0 0x55a7b0a670e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:01.805 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.802+0000 7fdf170cab80 4 client.16089 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T22:00:01.805 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.802+0000 7fdf170cab80 1 -- 172.21.15.151:0/1646776841 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a7b09abe10 con 0x55a7b0a64cf0 2024-05-21T22:00:01.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.803+0000 7fdf09400700 1 --2- 172.21.15.151:0/1646776841 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55a7b0a64cf0 0x55a7b0a670e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:01.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.803+0000 7fdf09400700 1 --2- 172.21.15.151:0/1646776841 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55a7b0a64cf0 0x55a7b0a670e0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e60170a4b9127b1d server_cookie=b52282200492c43d in_seq=0 out_seq=0 2024-05-21T22:00:01.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.803+0000 7fdee7e00700 10 client.16089 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T22:00:01.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.804+0000 7fdee7e00700 1 -- 172.21.15.151:0/1646776841 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55a7b09abe10 con 0x55a7b0a64cf0 2024-05-21T22:00:01.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.804+0000 7fdee7e00700 10 client.16089 handle_command_reply: tid=0 2024-05-21T22:00:01.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.804+0000 7fdf170cab80 1 client.16089 shutdown 2024-05-21T22:00:01.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.804+0000 7fdf170cab80 1 -- 172.21.15.151:0/1646776841 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a7b0a4b900 msgr2=0x55a7b0a574d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:01.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.804+0000 7fdf170cab80 1 --2- 172.21.15.151:0/1646776841 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a7b0a4b900 0x55a7b0a574d0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fdef8004e60 tx=0x7fdef8011d50 comp rx=0 tx=0).stop 2024-05-21T22:00:01.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.804+0000 7fdf170cab80 1 -- 172.21.15.151:0/1646776841 shutdown_connections 2024-05-21T22:00:01.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.804+0000 7fdf170cab80 1 --2- 172.21.15.151:0/1646776841 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a7b0a51850 0x55a7b0a54db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:01.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.804+0000 7fdf170cab80 1 --2- 172.21.15.151:0/1646776841 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a7b0a4b900 0x55a7b0a574d0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:01.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.804+0000 7fdf170cab80 1 --2- 172.21.15.151:0/1646776841 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55a7b0a64cf0 0x55a7b0a670e0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:01.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.804+0000 7fdf170cab80 1 --2- 172.21.15.151:0/1646776841 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a7b0a50f30 0x55a7b0a54870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:01.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.804+0000 7fdf170cab80 1 -- 172.21.15.151:0/1646776841 >> 172.21.15.151:0/1646776841 conn(0x55a7b0979c60 msgr2=0x55a7b0a4a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:00:01.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.804+0000 7fdee7e00700 0 client.16089 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T22:00:01.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.804+0000 7fdf170cab80 1 -- 172.21.15.151:0/1646776841 shutdown_connections 2024-05-21T22:00:01.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.804+0000 7fdf170cab80 1 -- 172.21.15.151:0/1646776841 wait complete. 2024-05-21T22:00:01.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.805+0000 7fdf170cab80 20 client.16089 trim_cache size 0 max 16384 2024-05-21T22:00:01.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.827+0000 7fdf170cab80 1 Processor -- start 2024-05-21T22:00:01.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.827+0000 7fdf170cab80 1 -- start start 2024-05-21T22:00:01.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.828+0000 7fdf170cab80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a7b0979c60 0x55a7b0a4b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:01.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.828+0000 7fdf170cab80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a7b0a4b900 0x55a7b0a51cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:01.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.828+0000 7fdf170cab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a7b0a50f30 0x55a7b0a52230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:01.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.828+0000 7fdf170cab80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdf041aff60 con 0x55a7b0a4b900 2024-05-21T22:00:01.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.828+0000 7fdf170cab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdf041b1aa0 con 0x55a7b0a50f30 2024-05-21T22:00:01.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.828+0000 7fdf170cab80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fdf041b0630 con 0x55a7b0979c60 2024-05-21T22:00:01.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.828+0000 7fdf09400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a7b0a4b900 0x55a7b0a51cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:01.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.828+0000 7fdf09400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a7b0a4b900 0x55a7b0a51cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:36266/0 (socket says 172.21.15.151:36266) 2024-05-21T22:00:01.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.828+0000 7fdf09e00700 1 --2- 172.21.15.151:0/462708564 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a7b0979c60 0x55a7b0a4b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:01.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.828+0000 7fdf09400700 1 -- 172.21.15.151:0/462708564 learned_addr learned my addr 172.21.15.151:0/462708564 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:00:01.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.828+0000 7fdf0a800700 1 --2- 172.21.15.151:0/462708564 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a7b0a50f30 0x55a7b0a52230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:01.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.829+0000 7fdf09e00700 1 -- 172.21.15.151:0/462708564 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a7b0a50f30 msgr2=0x55a7b0a52230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:01.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.829+0000 7fdf09e00700 1 --2- 172.21.15.151:0/462708564 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a7b0a50f30 0x55a7b0a52230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:01.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.829+0000 7fdf09e00700 1 -- 172.21.15.151:0/462708564 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a7b0a4b900 msgr2=0x55a7b0a51cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:01.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.829+0000 7fdf09e00700 1 --2- 172.21.15.151:0/462708564 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a7b0a4b900 0x55a7b0a51cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:01.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.829+0000 7fdf09e00700 1 -- 172.21.15.151:0/462708564 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7b0a52e00 con 0x55a7b0979c60 2024-05-21T22:00:01.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.829+0000 7fdf09e00700 1 --2- 172.21.15.151:0/462708564 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a7b0979c60 0x55a7b0a4b2e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fdef8024e60 tx=0x7fdef8011440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4bd3b46ec08807d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:00:01.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.829+0000 7fdef2a00700 1 -- 172.21.15.151:0/462708564 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdef8015070 con 0x55a7b0979c60 2024-05-21T22:00:01.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.829+0000 7fdef2a00700 1 -- 172.21.15.151:0/462708564 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdef8029070 con 0x55a7b0979c60 2024-05-21T22:00:01.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.829+0000 7fdef2a00700 1 -- 172.21.15.151:0/462708564 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdef8005330 con 0x55a7b0979c60 2024-05-21T22:00:01.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.830+0000 7fdf170cab80 1 -- 172.21.15.151:0/462708564 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a7b0979c60 msgr2=0x55a7b0a4b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:01.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.830+0000 7fdf170cab80 1 --2- 172.21.15.151:0/462708564 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a7b0979c60 0x55a7b0a4b2e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fdef8024e60 tx=0x7fdef8011440 comp rx=0 tx=0).stop 2024-05-21T22:00:01.833 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.830+0000 7fdf170cab80 1 -- 172.21.15.151:0/462708564 shutdown_connections 2024-05-21T22:00:01.833 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.830+0000 7fdf170cab80 1 --2- 172.21.15.151:0/462708564 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a7b0a50f30 0x55a7b0a52230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:01.833 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.830+0000 7fdf170cab80 1 --2- 172.21.15.151:0/462708564 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a7b0979c60 0x55a7b0a4b2e0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:01.833 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.830+0000 7fdf170cab80 1 --2- 172.21.15.151:0/462708564 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a7b0a4b900 0x55a7b0a51cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:01.833 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.830+0000 7fdf170cab80 1 -- 172.21.15.151:0/462708564 >> 172.21.15.151:0/462708564 conn(0x55a7b0a673c0 msgr2=0x55a7b0a53180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:00:01.833 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.830+0000 7fdf170cab80 1 -- 172.21.15.151:0/462708564 shutdown_connections 2024-05-21T22:00:01.833 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.830+0000 7fdf170cab80 1 -- 172.21.15.151:0/462708564 wait complete. 2024-05-21T22:00:01.833 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.830+0000 7fdf170cab80 1 -- start start 2024-05-21T22:00:01.833 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.831+0000 7fdf170cab80 1 Processor -- start 2024-05-21T22:00:01.833 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.831+0000 7fdf170cab80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a7b0979c60 0x55a7b0a5a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:01.833 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.831+0000 7fdf170cab80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a7b0a4b900 0x55a7b0a5aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:01.833 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.831+0000 7fdf170cab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a7b0a50f30 0x55a7b0a5b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:01.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.831+0000 7fdf170cab80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdf041b13d0 con 0x55a7b0a4b900 2024-05-21T22:00:01.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.831+0000 7fdf170cab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdf041b0d00 con 0x55a7b0a50f30 2024-05-21T22:00:01.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.831+0000 7fdf170cab80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fdf041adf00 con 0x55a7b0979c60 2024-05-21T22:00:01.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.831+0000 7fdf170cab80 10 client.0 _ll_register_callbacks cb 0x55a7b0943cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:00:01.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.832+0000 7fdf09e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a7b0979c60 0x55a7b0a5a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:01.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.832+0000 7fdf09e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a7b0979c60 0x55a7b0a5a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:37964/0 (socket says 172.21.15.151:37964) 2024-05-21T22:00:01.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.832+0000 7fdf09e00700 1 -- 172.21.15.151:0/951888904 learned_addr learned my addr 172.21.15.151:0/951888904 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:00:01.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.832+0000 7fdf0a800700 1 --2- 172.21.15.151:0/951888904 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a7b0a50f30 0x55a7b0a5b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:01.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.832+0000 7fdf09400700 1 --2- 172.21.15.151:0/951888904 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a7b0a4b900 0x55a7b0a5aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:01.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.832+0000 7fdf09e00700 1 -- 172.21.15.151:0/951888904 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a7b0a50f30 msgr2=0x55a7b0a5b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:01.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.832+0000 7fdf09e00700 1 --2- 172.21.15.151:0/951888904 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a7b0a50f30 0x55a7b0a5b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:01.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.832+0000 7fdf09e00700 1 -- 172.21.15.151:0/951888904 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a7b0a4b900 msgr2=0x55a7b0a5aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:01.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.832+0000 7fdf09e00700 1 --2- 172.21.15.151:0/951888904 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a7b0a4b900 0x55a7b0a5aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:01.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.832+0000 7fdf09e00700 1 -- 172.21.15.151:0/951888904 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7b0acf910 con 0x55a7b0979c60 2024-05-21T22:00:01.837 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.832+0000 7fdf09e00700 1 --2- 172.21.15.151:0/951888904 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a7b0979c60 0x55a7b0a5a580 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fdef8024f50 tx=0x7fdef8019670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de4a23dd31cc2b08 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:00:01.837 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.832+0000 7fdee7e00700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-21T22:00:01.837 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.833+0000 7fdee7e00700 1 -- 172.21.15.151:0/951888904 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdef8015040 con 0x55a7b0979c60 2024-05-21T22:00:01.837 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.833+0000 7fdee7e00700 1 -- 172.21.15.151:0/951888904 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fdef8019ca0 con 0x55a7b0979c60 2024-05-21T22:00:01.837 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.833+0000 7fdf170cab80 1 -- 172.21.15.151:0/951888904 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdef8029070 con 0x55a7b0979c60 2024-05-21T22:00:01.837 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.833+0000 7fdee7e00700 1 -- 172.21.15.151:0/951888904 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdef8005ce0 con 0x55a7b0979c60 2024-05-21T22:00:01.837 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.833+0000 7fdf170cab80 10 client.26233 fetch_fsmap learned FSMap version 29 2024-05-21T22:00:01.837 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.833+0000 7fdf170cab80 1 -- 172.21.15.151:0/951888904 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a7b0ad0c30 con 0x55a7b0979c60 2024-05-21T22:00:01.837 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.833+0000 7fdee7e00700 1 -- 172.21.15.151:0/951888904 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdef8029070 con 0x55a7b0979c60 2024-05-21T22:00:01.837 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.833+0000 7fdee7e00700 1 -- 172.21.15.151:0/951888904 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fdef8018050 con 0x55a7b0979c60 2024-05-21T22:00:01.837 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.834+0000 7fdf170cab80 10 client.26233 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:00:01.837 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.834+0000 7fdf170cab80 10 client.26233 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T22:00:01.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.834+0000 7fdf170cab80 20 client.26233 populate_metadata read hostname 'smithi151' 2024-05-21T22:00:01.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.834+0000 7fdf170cab80 1 --2- 172.21.15.151:0/951888904 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55a7b0a51850 0x55a7b0ad3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:01.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.834+0000 7fdf170cab80 4 client.26233 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-21T22:00:01.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.834+0000 7fdf170cab80 1 -- 172.21.15.151:0/951888904 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a7b09a55f0 con 0x55a7b0a51850 2024-05-21T22:00:01.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.834+0000 7fdf09400700 1 --2- 172.21.15.151:0/951888904 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55a7b0a51850 0x55a7b0ad3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:01.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.834+0000 7fdf09400700 1 --2- 172.21.15.151:0/951888904 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55a7b0a51850 0x55a7b0ad3290 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1faba5d97d7c9f82 server_cookie=7ff2824d14c91e64 in_seq=0 out_seq=0 2024-05-21T22:00:01.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.834+0000 7fdee7e00700 10 client.26233 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T22:00:01.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.836+0000 7fdee7e00700 1 -- 172.21.15.151:0/951888904 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a7b09a55f0 con 0x55a7b0a51850 2024-05-21T22:00:01.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.836+0000 7fdee7e00700 10 client.26233 handle_command_reply: tid=0 2024-05-21T22:00:01.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.836+0000 7fdf170cab80 1 client.26233 shutdown 2024-05-21T22:00:01.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.836+0000 7fdf170cab80 1 -- 172.21.15.151:0/951888904 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a7b0979c60 msgr2=0x55a7b0a5a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:01.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.836+0000 7fdf170cab80 1 --2- 172.21.15.151:0/951888904 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a7b0979c60 0x55a7b0a5a580 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fdef8024f50 tx=0x7fdef8019670 comp rx=0 tx=0).stop 2024-05-21T22:00:01.839 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.836+0000 7fdf170cab80 1 -- 172.21.15.151:0/951888904 shutdown_connections 2024-05-21T22:00:01.839 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.836+0000 7fdf170cab80 1 --2- 172.21.15.151:0/951888904 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55a7b0a51850 0x55a7b0ad3290 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:01.839 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.836+0000 7fdf170cab80 1 --2- 172.21.15.151:0/951888904 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a7b0a50f30 0x55a7b0a5b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:01.839 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.836+0000 7fdf170cab80 1 --2- 172.21.15.151:0/951888904 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a7b0979c60 0x55a7b0a5a580 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:01.839 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.836+0000 7fdee7e00700 0 client.26233 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T22:00:01.839 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.836+0000 7fdf170cab80 1 --2- 172.21.15.151:0/951888904 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a7b0a4b900 0x55a7b0a5aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:01.839 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.836+0000 7fdf170cab80 1 -- 172.21.15.151:0/951888904 >> 172.21.15.151:0/951888904 conn(0x55a7b0a673c0 msgr2=0x55a7b0a51c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:00:01.839 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.836+0000 7fdf170cab80 1 -- 172.21.15.151:0/951888904 shutdown_connections 2024-05-21T22:00:01.839 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.836+0000 7fdf170cab80 1 -- 172.21.15.151:0/951888904 wait complete. 2024-05-21T22:00:01.839 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.837+0000 7fdf170cab80 20 client.26233 trim_cache size 0 max 16384 2024-05-21T22:00:01.839 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-05-21T22:00:01.840 INFO:teuthology.orchestra.run.smithi151.stdout: "return_code": 0, 2024-05-21T22:00:01.840 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_tag": "4164f3bf-00be-417b-aa5b-a5a51492acfa", 2024-05-21T22:00:01.840 INFO:teuthology.orchestra.run.smithi151.stdout: "mode": "asynchronous" 2024-05-21T22:00:01.840 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-05-21T22:00:01.840 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.837+0000 7fdf0c800700 1 -- 172.21.15.151:0/3313907893 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fdeec0b29f0 msgr2=0x7fdeec0b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:01.840 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.837+0000 7fdf0c800700 1 --2- 172.21.15.151:0/3313907893 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fdeec0b29f0 0x7fdeec0b4eb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fdf00009dc0 tx=0x7fdf0000b040 comp rx=0 tx=0).stop 2024-05-21T22:00:01.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.837+0000 7fdf0c800700 1 -- 172.21.15.151:0/3313907893 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdf041041a0 msgr2=0x7fdf041a1f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:01.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.837+0000 7fdf0c800700 1 --2- 172.21.15.151:0/3313907893 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdf041041a0 0x7fdf041a1f80 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fdef400e9a0 tx=0x7fdef400ecb0 comp rx=0 tx=0).stop 2024-05-21T22:00:01.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.837+0000 7fdf0c800700 1 -- 172.21.15.151:0/3313907893 shutdown_connections 2024-05-21T22:00:01.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.837+0000 7fdf0c800700 1 --2- 172.21.15.151:0/3313907893 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fdeec0b29f0 0x7fdeec0b4eb0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:01.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.837+0000 7fdf0c800700 1 --2- 172.21.15.151:0/3313907893 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf041088f0 0x7fdf0419bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:01.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.837+0000 7fdf0c800700 1 --2- 172.21.15.151:0/3313907893 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdf041037f0 0x7fdf041a1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:01.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.837+0000 7fdf0c800700 1 --2- 172.21.15.151:0/3313907893 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdf041041a0 0x7fdf041a1f80 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:01.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.837+0000 7fdf0c800700 1 -- 172.21.15.151:0/3313907893 >> 172.21.15.151:0/3313907893 conn(0x7fdf0407a6a0 msgr2=0x7fdf04101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:00:01.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.838+0000 7fdf0c800700 1 -- 172.21.15.151:0/3313907893 shutdown_connections 2024-05-21T22:00:01.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:01.838+0000 7fdf0c800700 1 -- 172.21.15.151:0/3313907893 wait complete. 2024-05-21T22:00:02.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:00:02 smithi151 ceph-mon[31623]: scrub queued for path: / 2024-05-21T22:00:02.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:00:02 smithi151 ceph-mon[31623]: scrub summary: idle+waiting paths [/] 2024-05-21T22:00:02.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:00:02 smithi151 ceph-mon[31623]: scrub summary: active paths [/] 2024-05-21T22:00:02.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:00:02 smithi151 ceph-mon[31623]: scrub summary: idle+waiting paths [/] 2024-05-21T22:00:02.735 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:00:02 smithi184 ceph-mon[36942]: scrub queued for path: / 2024-05-21T22:00:02.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:00:02 smithi184 ceph-mon[36942]: scrub summary: idle+waiting paths [/] 2024-05-21T22:00:02.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:00:02 smithi184 ceph-mon[36942]: scrub summary: active paths [/] 2024-05-21T22:00:02.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:00:02 smithi184 ceph-mon[36942]: scrub summary: idle+waiting paths [/] 2024-05-21T22:00:02.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:00:02 smithi164 ceph-mon[36269]: scrub queued for path: / 2024-05-21T22:00:02.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:00:02 smithi164 ceph-mon[36269]: scrub summary: idle+waiting paths [/] 2024-05-21T22:00:02.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:00:02 smithi164 ceph-mon[36269]: scrub summary: active paths [/] 2024-05-21T22:00:02.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:00:02 smithi164 ceph-mon[36269]: scrub summary: idle+waiting paths [/] 2024-05-21T22:00:02.908 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-21T22:00:03.361 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:00:03.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:00:03 smithi184 ceph-mon[36942]: pgmap v136: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 27 op/s 2024-05-21T22:00:03.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:00:03 smithi164 ceph-mon[36269]: pgmap v136: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 27 op/s 2024-05-21T22:00:03.788 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:00:03 smithi151 ceph-mon[31623]: pgmap v136: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 27 op/s 2024-05-21T22:00:04.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:00:04 smithi151 ceph-mon[31623]: scrub summary: idle 2024-05-21T22:00:04.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:00:04 smithi164 ceph-mon[36269]: scrub summary: idle 2024-05-21T22:00:04.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:00:04 smithi184 ceph-mon[36942]: scrub summary: idle 2024-05-21T22:00:05.547 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.546+0000 7f07c8a00700 1 -- 172.21.15.151:0/2941005606 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f07c410b650 msgr2=0x7f07c410ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:05.547 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.546+0000 7f07c8a00700 1 --2- 172.21.15.151:0/2941005606 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f07c410b650 0x7f07c410ba30 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f07b4009990 tx=0x7f07b4009ca0 comp rx=0 tx=0).stop 2024-05-21T22:00:05.547 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.546+0000 7f07c8a00700 1 -- 172.21.15.151:0/2941005606 shutdown_connections 2024-05-21T22:00:05.547 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.546+0000 7f07c8a00700 1 --2- 172.21.15.151:0/2941005606 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07c4106010 0x7f07c4112520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:05.548 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.546+0000 7f07c8a00700 1 --2- 172.21.15.151:0/2941005606 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f07c410b650 0x7f07c410ba30 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:05.548 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.546+0000 7f07c8a00700 1 --2- 172.21.15.151:0/2941005606 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07c4105650 0x7f07c4105ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:05.548 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.546+0000 7f07c8a00700 1 -- 172.21.15.151:0/2941005606 >> 172.21.15.151:0/2941005606 conn(0x7f07c4101120 msgr2=0x7f07c4103540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:00:05.548 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.547+0000 7f07c8a00700 1 -- 172.21.15.151:0/2941005606 shutdown_connections 2024-05-21T22:00:05.548 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.547+0000 7f07c8a00700 1 -- 172.21.15.151:0/2941005606 wait complete. 2024-05-21T22:00:05.549 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.548+0000 7f07c8a00700 1 Processor -- start 2024-05-21T22:00:05.549 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.548+0000 7f07c8a00700 1 -- start start 2024-05-21T22:00:05.549 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.549+0000 7f07c8a00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07c4105650 0x7f07c41a1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:05.549 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.549+0000 7f07c8a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07c4106010 0x7f07c41a1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:05.549 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.549+0000 7f07c8a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f07c410b650 0x7f07c419bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:05.549 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.549+0000 7f07c8a00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f07c4115850 con 0x7f07c410b650 2024-05-21T22:00:05.549 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.549+0000 7f07c8a00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f07c41156d0 con 0x7f07c4106010 2024-05-21T22:00:05.549 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.549+0000 7f07c8a00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f07c41159d0 con 0x7f07c4105650 2024-05-21T22:00:05.550 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.549+0000 7f07c3400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07c4105650 0x7f07c41a1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:05.550 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.549+0000 7f07c2a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07c4106010 0x7f07c41a1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:05.550 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.549+0000 7f07c3400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07c4105650 0x7f07c41a1a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:37984/0 (socket says 172.21.15.151:37984) 2024-05-21T22:00:05.550 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.549+0000 7f07c3400700 1 -- 172.21.15.151:0/4268807312 learned_addr learned my addr 172.21.15.151:0/4268807312 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:00:05.551 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.549+0000 7f07c3e00700 1 --2- 172.21.15.151:0/4268807312 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f07c410b650 0x7f07c419bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:05.551 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.550+0000 7f07c3400700 1 -- 172.21.15.151:0/4268807312 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07c4106010 msgr2=0x7f07c41a1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:05.551 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.550+0000 7f07c3400700 1 --2- 172.21.15.151:0/4268807312 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07c4106010 0x7f07c41a1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:05.551 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.550+0000 7f07c3400700 1 -- 172.21.15.151:0/4268807312 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f07c410b650 msgr2=0x7f07c419bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:05.551 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.550+0000 7f07c3400700 1 --2- 172.21.15.151:0/4268807312 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f07c410b650 0x7f07c419bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:05.551 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.550+0000 7f07c3400700 1 -- 172.21.15.151:0/4268807312 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07c419c410 con 0x7f07c4105650 2024-05-21T22:00:05.551 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.550+0000 7f07c3400700 1 --2- 172.21.15.151:0/4268807312 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07c4105650 0x7f07c41a1a60 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f07b40057f0 tx=0x7f07b400bfd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:00:05.552 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.550+0000 7f07bbe00700 1 -- 172.21.15.151:0/4268807312 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b4026070 con 0x7f07c4105650 2024-05-21T22:00:05.552 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.550+0000 7f07bbe00700 1 -- 172.21.15.151:0/4268807312 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07b4021410 con 0x7f07c4105650 2024-05-21T22:00:05.552 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.550+0000 7f07bbe00700 1 -- 172.21.15.151:0/4268807312 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b4017650 con 0x7f07c4105650 2024-05-21T22:00:05.552 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.551+0000 7f07c8a00700 1 -- 172.21.15.151:0/4268807312 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07c419c6a0 con 0x7f07c4105650 2024-05-21T22:00:05.552 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.551+0000 7f07c8a00700 1 -- 172.21.15.151:0/4268807312 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07c41a8810 con 0x7f07c4105650 2024-05-21T22:00:05.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.555+0000 7f07bbe00700 1 -- 172.21.15.151:0/4268807312 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7f07b4029070 con 0x7f07c4105650 2024-05-21T22:00:05.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.556+0000 7f07d4a58b80 1 Processor -- start 2024-05-21T22:00:05.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.556+0000 7f07d4a58b80 1 -- start start 2024-05-21T22:00:05.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.556+0000 7f07d4a58b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563a4fae97c0 0x563a4fae9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:05.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.556+0000 7f07d4a58b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563a4faea0e0 0x563a4fae3d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:05.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.556+0000 7f07d4a58b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a4fae4390 0x563a4fae47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:05.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.556+0000 7f07d4a58b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x563a4fa41e40 con 0x563a4fae97c0 2024-05-21T22:00:05.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.556+0000 7f07d4a58b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x563a4fa319e0 con 0x563a4faea0e0 2024-05-21T22:00:05.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.556+0000 7f07d4a58b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x563a4fa35a20 con 0x563a4fae4390 2024-05-21T22:00:05.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.556+0000 7f07bbe00700 1 --2- 172.21.15.151:0/4268807312 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f07a80b2a70 0x7f07a80b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:05.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.557+0000 7f07c3400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563a4faea0e0 0x563a4fae3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:05.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.557+0000 7f07c3400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563a4faea0e0 0x563a4fae3d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:37756/0 (socket says 172.21.15.151:37756) 2024-05-21T22:00:05.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.557+0000 7f07c3400700 1 -- 172.21.15.151:0/2154039346 learned_addr learned my addr 172.21.15.151:0/2154039346 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:00:05.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.557+0000 7f07c3e00700 1 --2- 172.21.15.151:0/2154039346 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563a4fae97c0 0x563a4fae9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:05.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.557+0000 7f07bbe00700 1 -- 172.21.15.151:0/4268807312 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f07b40e3160 con 0x7f07c4105650 2024-05-21T22:00:05.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.557+0000 7f07c3e00700 1 --2- 172.21.15.151:0/4268807312 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f07a80b2a70 0x7f07a80b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:05.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.557+0000 7f07c3400700 1 -- 172.21.15.151:0/2154039346 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a4fae4390 msgr2=0x563a4fae47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:05.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.557+0000 7f07c3400700 1 --2- 172.21.15.151:0/2154039346 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a4fae4390 0x563a4fae47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:05.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.557+0000 7f07c3400700 1 -- 172.21.15.151:0/2154039346 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563a4fae97c0 msgr2=0x563a4fae9ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:05.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.557+0000 7f07c3400700 1 --2- 172.21.15.151:0/2154039346 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563a4fae97c0 0x563a4fae9ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:05.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.557+0000 7f07c3400700 1 -- 172.21.15.151:0/2154039346 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a4f9fa420 con 0x563a4faea0e0 2024-05-21T22:00:05.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.557+0000 7f07c3e00700 1 --2- 172.21.15.151:0/2154039346 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563a4fae97c0 0x563a4fae9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T22:00:05.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.558+0000 7f07c3400700 1 --2- 172.21.15.151:0/2154039346 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563a4faea0e0 0x563a4fae3d70 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f07b40e0800 tx=0x7f07b4021c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b883722f99701aaf server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:00:05.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.558+0000 7f07b8c00700 1 -- 172.21.15.151:0/2154039346 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b40a9f90 con 0x563a4faea0e0 2024-05-21T22:00:05.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.558+0000 7f07b8c00700 1 -- 172.21.15.151:0/2154039346 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07b40aab80 con 0x563a4faea0e0 2024-05-21T22:00:05.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.558+0000 7f07b8c00700 1 -- 172.21.15.151:0/2154039346 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b40218c0 con 0x563a4faea0e0 2024-05-21T22:00:05.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.558+0000 7f07c3e00700 1 --2- 172.21.15.151:0/4268807312 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f07a80b2a70 0x7f07a80b4f30 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f07bc009fd0 tx=0x7f07bc00b040 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:00:05.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.559+0000 7f07d4a58b80 1 -- 172.21.15.151:0/2154039346 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563a4faea0e0 msgr2=0x563a4fae3d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:05.561 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.559+0000 7f07d4a58b80 1 --2- 172.21.15.151:0/2154039346 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563a4faea0e0 0x563a4fae3d70 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f07b40e0800 tx=0x7f07b4021c60 comp rx=0 tx=0).stop 2024-05-21T22:00:05.561 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.559+0000 7f07d4a58b80 1 -- 172.21.15.151:0/2154039346 shutdown_connections 2024-05-21T22:00:05.561 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.559+0000 7f07d4a58b80 1 --2- 172.21.15.151:0/2154039346 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a4fae4390 0x563a4fae47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:05.561 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.559+0000 7f07d4a58b80 1 --2- 172.21.15.151:0/2154039346 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563a4fae97c0 0x563a4fae9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:05.561 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.559+0000 7f07d4a58b80 1 --2- 172.21.15.151:0/2154039346 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563a4faea0e0 0x563a4fae3d70 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:05.561 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.559+0000 7f07d4a58b80 1 -- 172.21.15.151:0/2154039346 >> 172.21.15.151:0/2154039346 conn(0x563a4fa11be0 msgr2=0x563a4fade810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:00:05.561 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.559+0000 7f07d4a58b80 1 -- 172.21.15.151:0/2154039346 shutdown_connections 2024-05-21T22:00:05.561 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.559+0000 7f07d4a58b80 1 -- 172.21.15.151:0/2154039346 wait complete. 2024-05-21T22:00:05.561 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.560+0000 7f07d4a58b80 1 -- start start 2024-05-21T22:00:05.561 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.560+0000 7f07d4a58b80 1 Processor -- start 2024-05-21T22:00:05.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.561+0000 7f07d4a58b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a4fae4390 0x563a4faefe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:05.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.561+0000 7f07d4a58b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563a4fae97c0 0x563a4faed1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:05.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.561+0000 7f07d4a58b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563a4faea0e0 0x563a4faed6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:05.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.561+0000 7f07d4a58b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x563a4f937fa0 con 0x563a4fae97c0 2024-05-21T22:00:05.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.561+0000 7f07d4a58b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x563a4f9d2760 con 0x563a4faea0e0 2024-05-21T22:00:05.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.561+0000 7f07d4a58b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x563a4faf09c0 con 0x563a4fae4390 2024-05-21T22:00:05.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.561+0000 7f07d4a58b80 10 client.0 _ll_register_callbacks cb 0x563a4f9dbc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:00:05.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.561+0000 7f07c2a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563a4fae97c0 0x563a4faed1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:05.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.561+0000 7f07c2a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563a4fae97c0 0x563a4faed1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:36300/0 (socket says 172.21.15.151:36300) 2024-05-21T22:00:05.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.561+0000 7f07c2a00700 1 -- 172.21.15.151:0/3315495742 learned_addr learned my addr 172.21.15.151:0/3315495742 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:00:05.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.561+0000 7f07c3400700 1 --2- 172.21.15.151:0/3315495742 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a4fae4390 0x563a4faefe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:05.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.561+0000 7f07c3e00700 1 --2- 172.21.15.151:0/3315495742 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563a4faea0e0 0x563a4faed6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:05.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.561+0000 7f07c3400700 1 -- 172.21.15.151:0/3315495742 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563a4faea0e0 msgr2=0x563a4faed6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:05.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.561+0000 7f07c3400700 1 --2- 172.21.15.151:0/3315495742 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563a4faea0e0 0x563a4faed6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:05.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.561+0000 7f07c3400700 1 -- 172.21.15.151:0/3315495742 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563a4fae97c0 msgr2=0x563a4faed1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:05.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.561+0000 7f07c3400700 1 --2- 172.21.15.151:0/3315495742 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563a4fae97c0 0x563a4faed1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:05.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.561+0000 7f07c3400700 1 -- 172.21.15.151:0/3315495742 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a4f9818d0 con 0x563a4fae4390 2024-05-21T22:00:05.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.561+0000 7f07c2a00700 1 --2- 172.21.15.151:0/3315495742 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563a4fae97c0 0x563a4faed1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:00:05.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.562+0000 7f07c3400700 1 --2- 172.21.15.151:0/3315495742 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a4fae4390 0x563a4faefe10 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x563a4faf0850 tx=0x7f07b40e0830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d8cb5be6bae5ab6 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:00:05.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.562+0000 7f07ad600700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-21T22:00:05.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.562+0000 7f07ad600700 1 -- 172.21.15.151:0/3315495742 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b40a9f90 con 0x563a4fae4390 2024-05-21T22:00:05.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.562+0000 7f07ad600700 1 -- 172.21.15.151:0/3315495742 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f07b40aa6e0 con 0x563a4fae4390 2024-05-21T22:00:05.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.562+0000 7f07d4a58b80 1 -- 172.21.15.151:0/3315495742 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07b40aab80 con 0x563a4fae4390 2024-05-21T22:00:05.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.562+0000 7f07ad600700 1 -- 172.21.15.151:0/3315495742 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f07b40a6340 con 0x563a4fae4390 2024-05-21T22:00:05.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.562+0000 7f07d4a58b80 10 client.26242 fetch_fsmap learned FSMap version 29 2024-05-21T22:00:05.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.562+0000 7f07d4a58b80 1 -- 172.21.15.151:0/3315495742 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563a4fa02060 con 0x563a4fae4390 2024-05-21T22:00:05.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.563+0000 7f07ad600700 1 -- 172.21.15.151:0/3315495742 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07b40aab80 con 0x563a4fae4390 2024-05-21T22:00:05.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.563+0000 7f07ad600700 1 -- 172.21.15.151:0/3315495742 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f07b40c4ce0 con 0x563a4fae4390 2024-05-21T22:00:05.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.563+0000 7f07d4a58b80 10 client.26242 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:00:05.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.563+0000 7f07d4a58b80 10 client.26242 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T22:00:05.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.563+0000 7f07d4a58b80 20 client.26242 populate_metadata read hostname 'smithi151' 2024-05-21T22:00:05.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.563+0000 7f07d4a58b80 1 --2- 172.21.15.151:0/3315495742 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x563a4fafd340 0x563a4faff730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:05.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.563+0000 7f07d4a58b80 4 client.26242 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T22:00:05.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.563+0000 7f07d4a58b80 1 -- 172.21.15.151:0/3315495742 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563a4fa43d90 con 0x563a4fafd340 2024-05-21T22:00:05.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.564+0000 7f07c2a00700 1 --2- 172.21.15.151:0/3315495742 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x563a4fafd340 0x563a4faff730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:05.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.564+0000 7f07c2a00700 1 --2- 172.21.15.151:0/3315495742 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x563a4fafd340 0x563a4faff730 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26a8216cb5b4d9d0 server_cookie=abfbb10482593667 in_seq=0 out_seq=0 2024-05-21T22:00:05.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.564+0000 7f07ad600700 10 client.26242 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T22:00:05.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.565+0000 7f07ad600700 1 -- 172.21.15.151:0/3315495742 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x563a4fa43d90 con 0x563a4fafd340 2024-05-21T22:00:05.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.565+0000 7f07ad600700 10 client.26242 handle_command_reply: tid=0 2024-05-21T22:00:05.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.565+0000 7f07d4a58b80 1 client.26242 shutdown 2024-05-21T22:00:05.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.565+0000 7f07d4a58b80 1 -- 172.21.15.151:0/3315495742 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a4fae4390 msgr2=0x563a4faefe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:05.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.565+0000 7f07d4a58b80 1 --2- 172.21.15.151:0/3315495742 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a4fae4390 0x563a4faefe10 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x563a4faf0850 tx=0x7f07b40e0830 comp rx=0 tx=0).stop 2024-05-21T22:00:05.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.565+0000 7f07d4a58b80 1 -- 172.21.15.151:0/3315495742 shutdown_connections 2024-05-21T22:00:05.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.565+0000 7f07d4a58b80 1 --2- 172.21.15.151:0/3315495742 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x563a4fafd340 0x563a4faff730 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:05.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.566+0000 7f07d4a58b80 1 --2- 172.21.15.151:0/3315495742 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563a4faea0e0 0x563a4faed6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:05.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.566+0000 7f07d4a58b80 1 --2- 172.21.15.151:0/3315495742 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a4fae4390 0x563a4faefe10 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:05.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.566+0000 7f07ad600700 0 client.26242 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T22:00:05.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.566+0000 7f07d4a58b80 1 --2- 172.21.15.151:0/3315495742 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563a4fae97c0 0x563a4faed1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:05.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.566+0000 7f07d4a58b80 1 -- 172.21.15.151:0/3315495742 >> 172.21.15.151:0/3315495742 conn(0x563a4fa11be0 msgr2=0x563a4faddfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:00:05.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.566+0000 7f07d4a58b80 1 -- 172.21.15.151:0/3315495742 shutdown_connections 2024-05-21T22:00:05.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.566+0000 7f07d4a58b80 1 -- 172.21.15.151:0/3315495742 wait complete. 2024-05-21T22:00:05.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.566+0000 7f07d4a58b80 20 client.26242 trim_cache size 0 max 16384 2024-05-21T22:00:05.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.588+0000 7f07d4a58b80 1 Processor -- start 2024-05-21T22:00:05.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.588+0000 7f07d4a58b80 1 -- start start 2024-05-21T22:00:05.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.589+0000 7f07d4a58b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a4fae4390 0x563a4fae3d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:05.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.589+0000 7f07d4a58b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563a4fae97c0 0x563a4faeae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:05.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.589+0000 7f07d4a58b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563a4faea0e0 0x563a4fade740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:05.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.589+0000 7f07d4a58b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f07c41afda0 con 0x563a4faea0e0 2024-05-21T22:00:05.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.589+0000 7f07d4a58b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f07c41b18e0 con 0x563a4fae97c0 2024-05-21T22:00:05.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.589+0000 7f07d4a58b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f07c41b0470 con 0x563a4fae4390 2024-05-21T22:00:05.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.589+0000 7f07c3400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a4fae4390 0x563a4fae3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:05.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.589+0000 7f07c3e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563a4faea0e0 0x563a4fade740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:05.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.589+0000 7f07c3400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a4fae4390 0x563a4fae3d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:38018/0 (socket says 172.21.15.151:38018) 2024-05-21T22:00:05.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.589+0000 7f07c3400700 1 -- 172.21.15.151:0/2321287032 learned_addr learned my addr 172.21.15.151:0/2321287032 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:00:05.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.589+0000 7f07c2a00700 1 --2- 172.21.15.151:0/2321287032 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563a4fae97c0 0x563a4faeae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:05.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.589+0000 7f07c3400700 1 -- 172.21.15.151:0/2321287032 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563a4fae97c0 msgr2=0x563a4faeae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:05.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.589+0000 7f07c3400700 1 --2- 172.21.15.151:0/2321287032 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563a4fae97c0 0x563a4faeae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:05.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.589+0000 7f07c3400700 1 -- 172.21.15.151:0/2321287032 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563a4faea0e0 msgr2=0x563a4fade740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:05.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.589+0000 7f07c3400700 1 --2- 172.21.15.151:0/2321287032 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563a4faea0e0 0x563a4fade740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:05.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.589+0000 7f07c3400700 1 -- 172.21.15.151:0/2321287032 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a4fae9ee0 con 0x563a4fae4390 2024-05-21T22:00:05.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.590+0000 7f07c3400700 1 --2- 172.21.15.151:0/2321287032 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a4fae4390 0x563a4fae3d70 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f07b40a76f0 tx=0x7f07b4021c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a31da4af51dc6ca server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:00:05.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.590+0000 7f07afe00700 1 -- 172.21.15.151:0/2321287032 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b40a9f90 con 0x563a4fae4390 2024-05-21T22:00:05.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.590+0000 7f07afe00700 1 -- 172.21.15.151:0/2321287032 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07b40aab80 con 0x563a4fae4390 2024-05-21T22:00:05.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.590+0000 7f07afe00700 1 -- 172.21.15.151:0/2321287032 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b40ae830 con 0x563a4fae4390 2024-05-21T22:00:05.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.591+0000 7f07d4a58b80 1 -- 172.21.15.151:0/2321287032 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a4fae4390 msgr2=0x563a4fae3d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:05.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.591+0000 7f07d4a58b80 1 --2- 172.21.15.151:0/2321287032 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a4fae4390 0x563a4fae3d70 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f07b40a76f0 tx=0x7f07b4021c60 comp rx=0 tx=0).stop 2024-05-21T22:00:05.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.591+0000 7f07d4a58b80 1 -- 172.21.15.151:0/2321287032 shutdown_connections 2024-05-21T22:00:05.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.591+0000 7f07d4a58b80 1 --2- 172.21.15.151:0/2321287032 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563a4faea0e0 0x563a4fade740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:05.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.591+0000 7f07d4a58b80 1 --2- 172.21.15.151:0/2321287032 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a4fae4390 0x563a4fae3d70 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:05.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.591+0000 7f07d4a58b80 1 --2- 172.21.15.151:0/2321287032 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563a4fae97c0 0x563a4faeae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:05.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.591+0000 7f07d4a58b80 1 -- 172.21.15.151:0/2321287032 >> 172.21.15.151:0/2321287032 conn(0x563a4fa11be0 msgr2=0x563a4faebaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:00:05.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.591+0000 7f07d4a58b80 1 -- 172.21.15.151:0/2321287032 shutdown_connections 2024-05-21T22:00:05.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.591+0000 7f07d4a58b80 1 -- 172.21.15.151:0/2321287032 wait complete. 2024-05-21T22:00:05.595 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.591+0000 7f07d4a58b80 1 -- start start 2024-05-21T22:00:05.595 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.592+0000 7f07d4a58b80 1 Processor -- start 2024-05-21T22:00:05.595 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.592+0000 7f07d4a58b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563a4fae4390 0x563a4fb04720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:05.595 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.592+0000 7f07d4a58b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563a4fae97c0 0x563a4fb04c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:05.595 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.592+0000 7f07d4a58b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a4faea0e0 0x563a4fb051a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:05.595 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.592+0000 7f07d4a58b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f07c41b1210 con 0x563a4fae4390 2024-05-21T22:00:05.595 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.592+0000 7f07d4a58b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f07c41b0b40 con 0x563a4fae97c0 2024-05-21T22:00:05.595 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.592+0000 7f07d4a58b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f07c41add60 con 0x563a4faea0e0 2024-05-21T22:00:05.595 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.592+0000 7f07d4a58b80 10 client.0 _ll_register_callbacks cb 0x563a4f9dbc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:00:05.595 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.592+0000 7f07c3e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a4faea0e0 0x563a4fb051a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:05.595 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.592+0000 7f07c3400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563a4fae4390 0x563a4fb04720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:05.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.593+0000 7f07c3e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a4faea0e0 0x563a4fb051a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:38034/0 (socket says 172.21.15.151:38034) 2024-05-21T22:00:05.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.593+0000 7f07c3e00700 1 -- 172.21.15.151:0/3027638114 learned_addr learned my addr 172.21.15.151:0/3027638114 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:00:05.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.593+0000 7f07c2a00700 1 --2- 172.21.15.151:0/3027638114 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563a4fae97c0 0x563a4fb04c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:05.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.593+0000 7f07c3e00700 1 -- 172.21.15.151:0/3027638114 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563a4fae97c0 msgr2=0x563a4fb04c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:05.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.593+0000 7f07c3e00700 1 --2- 172.21.15.151:0/3027638114 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563a4fae97c0 0x563a4fb04c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:05.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.593+0000 7f07c3e00700 1 -- 172.21.15.151:0/3027638114 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563a4fae4390 msgr2=0x563a4fb04720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:05.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.593+0000 7f07c3e00700 1 --2- 172.21.15.151:0/3027638114 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563a4fae4390 0x563a4fb04720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:05.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.593+0000 7f07c3e00700 1 -- 172.21.15.151:0/3027638114 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a4fb66ba0 con 0x563a4faea0e0 2024-05-21T22:00:05.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.593+0000 7f07c3e00700 1 --2- 172.21.15.151:0/3027638114 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a4faea0e0 0x563a4fb051a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f07bc007be0 tx=0x7f07bc004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1184e0fa0fb41d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:00:05.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.593+0000 7f07ad600700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-21T22:00:05.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.593+0000 7f07ad600700 1 -- 172.21.15.151:0/3027638114 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07bc018d00 con 0x563a4faea0e0 2024-05-21T22:00:05.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.594+0000 7f07d4a58b80 1 -- 172.21.15.151:0/3027638114 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07b40aab80 con 0x563a4faea0e0 2024-05-21T22:00:05.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.594+0000 7f07ad600700 1 -- 172.21.15.151:0/3027638114 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f07bc02d470 con 0x563a4faea0e0 2024-05-21T22:00:05.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.594+0000 7f07ad600700 1 -- 172.21.15.151:0/3027638114 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f07bc004bb0 con 0x563a4faea0e0 2024-05-21T22:00:05.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.594+0000 7f07d4a58b80 10 client.26248 fetch_fsmap learned FSMap version 29 2024-05-21T22:00:05.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.594+0000 7f07d4a58b80 1 -- 172.21.15.151:0/3027638114 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563a4fb67ea0 con 0x563a4faea0e0 2024-05-21T22:00:05.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.594+0000 7f07ad600700 1 -- 172.21.15.151:0/3027638114 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07b40aab80 con 0x563a4faea0e0 2024-05-21T22:00:05.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.594+0000 7f07ad600700 1 -- 172.21.15.151:0/3027638114 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f07bc036500 con 0x563a4faea0e0 2024-05-21T22:00:05.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.594+0000 7f07d4a58b80 10 client.26248 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:00:05.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.595+0000 7f07d4a58b80 10 client.26248 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T22:00:05.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.595+0000 7f07d4a58b80 20 client.26248 populate_metadata read hostname 'smithi151' 2024-05-21T22:00:05.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.595+0000 7f07d4a58b80 1 --2- 172.21.15.151:0/3027638114 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x563a4fb68540 0x563a4fb6a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:05.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.595+0000 7f07d4a58b80 4 client.26248 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-21T22:00:05.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.595+0000 7f07d4a58b80 1 -- 172.21.15.151:0/3027638114 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563a4fa3d570 con 0x563a4fb68540 2024-05-21T22:00:05.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.595+0000 7f07c3400700 1 --2- 172.21.15.151:0/3027638114 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x563a4fb68540 0x563a4fb6a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:05.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.595+0000 7f07ad600700 10 client.26248 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T22:00:05.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.595+0000 7f07c3400700 1 --2- 172.21.15.151:0/3027638114 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x563a4fb68540 0x563a4fb6a960 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ccd74978e0d78a95 server_cookie=2251d50fcbaf2b7b in_seq=0 out_seq=0 2024-05-21T22:00:05.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.596+0000 7f07ad600700 1 -- 172.21.15.151:0/3027638114 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563a4fa3d570 con 0x563a4fb68540 2024-05-21T22:00:05.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.596+0000 7f07ad600700 10 client.26248 handle_command_reply: tid=0 2024-05-21T22:00:05.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.596+0000 7f07d4a58b80 1 client.26248 shutdown 2024-05-21T22:00:05.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.596+0000 7f07d4a58b80 1 -- 172.21.15.151:0/3027638114 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a4faea0e0 msgr2=0x563a4fb051a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:05.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.596+0000 7f07d4a58b80 1 --2- 172.21.15.151:0/3027638114 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a4faea0e0 0x563a4fb051a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f07bc007be0 tx=0x7f07bc004ad0 comp rx=0 tx=0).stop 2024-05-21T22:00:05.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.596+0000 7f07d4a58b80 1 -- 172.21.15.151:0/3027638114 shutdown_connections 2024-05-21T22:00:05.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.596+0000 7f07d4a58b80 1 --2- 172.21.15.151:0/3027638114 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a4faea0e0 0x563a4fb051a0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:05.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.596+0000 7f07d4a58b80 1 --2- 172.21.15.151:0/3027638114 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563a4fae4390 0x563a4fb04720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:05.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.596+0000 7f07d4a58b80 1 --2- 172.21.15.151:0/3027638114 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x563a4fb68540 0x563a4fb6a960 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:05.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.596+0000 7f07d4a58b80 1 --2- 172.21.15.151:0/3027638114 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563a4fae97c0 0x563a4fb04c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:05.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.596+0000 7f07ad600700 0 client.26248 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T22:00:05.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.596+0000 7f07d4a58b80 1 -- 172.21.15.151:0/3027638114 >> 172.21.15.151:0/3027638114 conn(0x563a4fa11be0 msgr2=0x563a4faec330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:00:05.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.596+0000 7f07d4a58b80 1 -- 172.21.15.151:0/3027638114 shutdown_connections 2024-05-21T22:00:05.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.596+0000 7f07d4a58b80 1 -- 172.21.15.151:0/3027638114 wait complete. 2024-05-21T22:00:05.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.596+0000 7f07d4a58b80 20 client.26248 trim_cache size 0 max 16384 2024-05-21T22:00:05.602 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-05-21T22:00:05.602 INFO:teuthology.orchestra.run.smithi151.stdout: "status": "no active scrubs running", 2024-05-21T22:00:05.602 INFO:teuthology.orchestra.run.smithi151.stdout: "scrubs": {} 2024-05-21T22:00:05.602 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-05-21T22:00:05.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.597+0000 7f07c8a00700 1 -- 172.21.15.151:0/4268807312 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f07a80b2a70 msgr2=0x7f07a80b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:05.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.597+0000 7f07c8a00700 1 --2- 172.21.15.151:0/4268807312 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f07a80b2a70 0x7f07a80b4f30 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f07bc009fd0 tx=0x7f07bc00b040 comp rx=0 tx=0).stop 2024-05-21T22:00:05.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.597+0000 7f07c8a00700 1 -- 172.21.15.151:0/4268807312 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07c4105650 msgr2=0x7f07c41a1a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:05.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.597+0000 7f07c8a00700 1 --2- 172.21.15.151:0/4268807312 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07c4105650 0x7f07c41a1a60 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f07b40057f0 tx=0x7f07b400bfd0 comp rx=0 tx=0).stop 2024-05-21T22:00:05.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.597+0000 7f07c8a00700 1 -- 172.21.15.151:0/4268807312 shutdown_connections 2024-05-21T22:00:05.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.597+0000 7f07c8a00700 1 --2- 172.21.15.151:0/4268807312 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f07a80b2a70 0x7f07a80b4f30 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:05.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.597+0000 7f07c8a00700 1 --2- 172.21.15.151:0/4268807312 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f07c410b650 0x7f07c419bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:05.604 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.597+0000 7f07c8a00700 1 --2- 172.21.15.151:0/4268807312 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07c4105650 0x7f07c41a1a60 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:05.604 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.597+0000 7f07c8a00700 1 --2- 172.21.15.151:0/4268807312 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07c4106010 0x7f07c41a1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:05.604 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.597+0000 7f07c8a00700 1 -- 172.21.15.151:0/4268807312 >> 172.21.15.151:0/4268807312 conn(0x7f07c4101120 msgr2=0x7f07c4101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:00:05.604 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.597+0000 7f07c8a00700 1 -- 172.21.15.151:0/4268807312 shutdown_connections 2024-05-21T22:00:05.604 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:05.597+0000 7f07c8a00700 1 -- 172.21.15.151:0/4268807312 wait complete. 2024-05-21T22:00:05.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:00:05 smithi151 ceph-mon[31623]: pgmap v137: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-05-21T22:00:05.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:00:05 smithi164 ceph-mon[36269]: pgmap v137: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-05-21T22:00:05.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:00:05 smithi184 ceph-mon[36942]: pgmap v137: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-05-21T22:00:06.488 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-21T22:00:06.489 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-21T22:00:06.932 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:00:07.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:00:07 smithi151 ceph-mon[31623]: pgmap v138: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-05-21T22:00:07.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:00:07 smithi164 ceph-mon[36269]: pgmap v138: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-05-21T22:00:07.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:00:07 smithi184 ceph-mon[36942]: pgmap v138: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-05-21T22:00:09.129 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:09.128+0000 7f9455000700 1 -- 172.21.15.151:0/1173159471 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9450075770 msgr2=0x7f9450075b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:09.129 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:09.128+0000 7f9455000700 1 --2- 172.21.15.151:0/1173159471 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9450075770 0x7f9450075b50 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f9440009f90 tx=0x7f944000b330 comp rx=0 tx=0).stop 2024-05-21T22:00:09.129 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:09.128+0000 7f9455000700 1 -- 172.21.15.151:0/1173159471 shutdown_connections 2024-05-21T22:00:09.129 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:09.128+0000 7f9455000700 1 --2- 172.21.15.151:0/1173159471 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f945010ff40 0x7f9450112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:09.129 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:09.128+0000 7f9455000700 1 --2- 172.21.15.151:0/1173159471 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9450075770 0x7f9450075b50 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:09.129 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:09.128+0000 7f9455000700 1 --2- 172.21.15.151:0/1173159471 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9450076090 0x7f945010fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:09.129 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:09.128+0000 7f9455000700 1 -- 172.21.15.151:0/1173159471 >> 172.21.15.151:0/1173159471 conn(0x7f945007a6a0 msgr2=0x7f945007aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:00:09.129 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:09.128+0000 7f9455000700 1 -- 172.21.15.151:0/1173159471 shutdown_connections 2024-05-21T22:00:09.130 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:09.128+0000 7f9455000700 1 -- 172.21.15.151:0/1173159471 wait complete. 2024-05-21T22:00:09.131 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:09.129+0000 7f9455000700 1 Processor -- start 2024-05-21T22:00:09.131 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:09.129+0000 7f9455000700 1 -- start start 2024-05-21T22:00:09.131 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:09.130+0000 7f9455000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9450075770 0x7f945019be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:09.131 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:09.130+0000 7f9455000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9450076090 0x7f945019c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:09.131 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:09.130+0000 7f9455000700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f945010ff40 0x7f94501a0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:09.131 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:09.130+0000 7f9455000700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f94501158f0 con 0x7f9450076090 2024-05-21T22:00:09.131 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:09.130+0000 7f9455000700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9450115770 con 0x7f9450075770 2024-05-21T22:00:09.131 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:09.130+0000 7f9455000700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9450115a70 con 0x7f945010ff40 2024-05-21T22:00:09.131 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:09.130+0000 7f944de00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9450075770 0x7f945019be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:09.131 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:09.130+0000 7f944de00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9450075770 0x7f945019be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:60526/0 (socket says 172.21.15.151:60526) 2024-05-21T22:00:09.132 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:09.130+0000 7f9445400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9450076090 0x7f945019c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:09.132 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:09.130+0000 7f944de00700 1 -- 172.21.15.151:0/3253559112 learned_addr learned my addr 172.21.15.151:0/3253559112 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:00:09.132 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:09.130+0000 7f944e800700 1 --2- 172.21.15.151:0/3253559112 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f945010ff40 0x7f94501a0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:09.132 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:09.130+0000 7f944de00700 1 -- 172.21.15.151:0/3253559112 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f945010ff40 msgr2=0x7f94501a0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:09.132 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:09.130+0000 7f944de00700 1 --2- 172.21.15.151:0/3253559112 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f945010ff40 0x7f94501a0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:09.132 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:09.130+0000 7f944de00700 1 -- 172.21.15.151:0/3253559112 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9450076090 msgr2=0x7f945019c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:09.132 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:09.130+0000 7f944de00700 1 --2- 172.21.15.151:0/3253559112 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9450076090 0x7f945019c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:09.132 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:09.130+0000 7f944de00700 1 -- 172.21.15.151:0/3253559112 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94501a0e50 con 0x7f9450075770 2024-05-21T22:00:09.132 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:09.131+0000 7f944de00700 1 --2- 172.21.15.151:0/3253559112 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9450075770 0x7f945019be40 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f9440009f60 tx=0x7f944000ccb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:00:09.133 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:09.131+0000 7f9447400700 1 -- 172.21.15.151:0/3253559112 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9440021070 con 0x7f9450075770 2024-05-21T22:00:09.133 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:09.131+0000 7f9447400700 1 -- 172.21.15.151:0/3253559112 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9440007ac0 con 0x7f9450075770 2024-05-21T22:00:09.133 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:09.131+0000 7f9455000700 1 -- 172.21.15.151:0/3253559112 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94501a10e0 con 0x7f9450075770 2024-05-21T22:00:09.133 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:09.131+0000 7f9447400700 1 -- 172.21.15.151:0/3253559112 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9440005040 con 0x7f9450075770 2024-05-21T22:00:09.133 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:09.131+0000 7f9455000700 1 -- 172.21.15.151:0/3253559112 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94501a89c0 con 0x7f9450075770 2024-05-21T22:00:09.133 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:09.132+0000 7f9455000700 1 -- 172.21.15.151:0/3253559112 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f945006a230 con 0x7f9450075770 2024-05-21T22:00:09.135 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:09.135+0000 7f9447400700 1 -- 172.21.15.151:0/3253559112 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7f9440024070 con 0x7f9450075770 2024-05-21T22:00:09.137 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:09.136+0000 7f9447400700 1 --2- 172.21.15.151:0/3253559112 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f94340b29f0 0x7f94340b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:09.137 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:09.137+0000 7f9447400700 1 -- 172.21.15.151:0/3253559112 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f944000f020 con 0x7f9450075770 2024-05-21T22:00:09.137 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:09.137+0000 7f9445400700 1 --2- 172.21.15.151:0/3253559112 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f94340b29f0 0x7f94340b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:09.137 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:09.137+0000 7f9445400700 1 --2- 172.21.15.151:0/3253559112 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f94340b29f0 0x7f94340b4eb0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9438009750 tx=0x7f9438009310 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:00:09.138 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:09.138+0000 7f9447400700 1 -- 172.21.15.151:0/3253559112 <== mon.2 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f944002b020 con 0x7f9450075770 2024-05-21T22:00:09.421 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:09.421+0000 7f9455000700 1 -- 172.21.15.151:0/3253559112 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f945002e560 con 0x7f9450075770 2024-05-21T22:00:09.423 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:09.422+0000 7f9447400700 1 -- 172.21.15.151:0/3253559112 <== mon.2 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11958 (secure 0 0 0) 0x7f94400ab8c0 con 0x7f9450075770 2024-05-21T22:00:09.424 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T22:00:09.424 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14358,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/158010529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":158010529},{"type":"v1","addr":"172.21.15.151:6835","nonce":158010529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6839/2069058115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":2069058115},{"type":"v1","addr":"172.21.15.151:6839","nonce":2069058115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T21:51:19.552069+0000","modified":"2024-05-21T21:55:34.643755+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24530,"mds_1":24493,"mds_2":24491,"mds_3":24478,"mds_4":24473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14403":{"gid":14403,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6841/1749723794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6840","nonce":1749723794},{"type":"v1","addr":"172.21.15.151:6841","nonce":1749723794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/2026266114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":2026266114},{"type":"v1","addr":"172.21.15.184:6833","nonce":2026266114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/2467900844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":2467900844},{"type":"v1","addr":"172.21.15.164:6833","nonce":2467900844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3367640960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3367640960},{"type":"v1","addr":"172.21.15.151:6837","nonce":3367640960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/3264142621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3264142621},{"type":"v1","addr":"172.21.15.184:6835","nonce":3264142621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":17,"addr":"172.21.15.164:6835/2340428470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2340428470},{"type":"v1","addr":"172.21.15.164:6835","nonce":2340428470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":16,"addr":"172.21.15.184:6837/2988194888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":2988194888},{"type":"v1","addr":"172.21.15.184:6837","nonce":2988194888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.164:6837/1554609572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1554609572},{"type":"v1","addr":"172.21.15.164:6837","nonce":1554609572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.184:6839/3059935143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":3059935143},{"type":"v1","addr":"172.21.15.184:6839","nonce":3059935143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.164:6839/2646401247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":2646401247},{"type":"v1","addr":"172.21.15.164:6839","nonce":2646401247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24530,"qdb_cluster":[24530,24493,24491,24473,24478]},"id":1}]} 2024-05-21T22:00:09.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:09.426+0000 7f9455000700 1 -- 172.21.15.151:0/3253559112 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f94340b29f0 msgr2=0x7f94340b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:09.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:09.426+0000 7f9455000700 1 --2- 172.21.15.151:0/3253559112 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f94340b29f0 0x7f94340b4eb0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9438009750 tx=0x7f9438009310 comp rx=0 tx=0).stop 2024-05-21T22:00:09.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:09.426+0000 7f9455000700 1 -- 172.21.15.151:0/3253559112 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9450075770 msgr2=0x7f945019be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:09.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:09.427+0000 7f9455000700 1 --2- 172.21.15.151:0/3253559112 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9450075770 0x7f945019be40 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f9440009f60 tx=0x7f944000ccb0 comp rx=0 tx=0).stop 2024-05-21T22:00:09.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:09.427+0000 7f9455000700 1 -- 172.21.15.151:0/3253559112 shutdown_connections 2024-05-21T22:00:09.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:09.427+0000 7f9455000700 1 --2- 172.21.15.151:0/3253559112 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f94340b29f0 0x7f94340b4eb0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:09.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:09.427+0000 7f9455000700 1 --2- 172.21.15.151:0/3253559112 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9450075770 0x7f945019be40 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:09.428 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:09.427+0000 7f9455000700 1 --2- 172.21.15.151:0/3253559112 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f945010ff40 0x7f94501a0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:09.428 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:09.427+0000 7f9455000700 1 --2- 172.21.15.151:0/3253559112 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9450076090 0x7f945019c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:09.428 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:09.427+0000 7f9455000700 1 -- 172.21.15.151:0/3253559112 >> 172.21.15.151:0/3253559112 conn(0x7f945007a6a0 msgr2=0x7f9450101ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:00:09.428 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:09.427+0000 7f9455000700 1 -- 172.21.15.151:0/3253559112 shutdown_connections 2024-05-21T22:00:09.428 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:09.427+0000 7f9455000700 1 -- 172.21.15.151:0/3253559112 wait complete. 2024-05-21T22:00:09.429 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 29 2024-05-21T22:00:09.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:00:09 smithi151 ceph-mon[31623]: pgmap v139: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 27 op/s 2024-05-21T22:00:09.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:00:09 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/3253559112' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T22:00:09.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:00:09 smithi164 ceph-mon[36269]: pgmap v139: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 27 op/s 2024-05-21T22:00:09.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:00:09 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/3253559112' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T22:00:09.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:00:09 smithi184 ceph-mon[36942]: pgmap v139: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 27 op/s 2024-05-21T22:00:09.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:00:09 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/3253559112' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T22:00:10.291 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-21T22:00:10.739 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:00:11.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:00:11 smithi164 ceph-mon[36269]: pgmap v140: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-05-21T22:00:11.910 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:00:11 smithi151 ceph-mon[31623]: pgmap v140: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-05-21T22:00:11.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:00:11 smithi184 ceph-mon[36942]: pgmap v140: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-05-21T22:00:12.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.811+0000 7ff260000700 1 -- 172.21.15.151:0/2635951185 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff258105730 msgr2=0x7ff258105bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:12.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.811+0000 7ff260000700 1 --2- 172.21.15.151:0/2635951185 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff258105730 0x7ff258105bb0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7ff244009990 tx=0x7ff244009ca0 comp rx=0 tx=0).stop 2024-05-21T22:00:12.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.811+0000 7ff260000700 1 -- 172.21.15.151:0/2635951185 shutdown_connections 2024-05-21T22:00:12.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.811+0000 7ff260000700 1 --2- 172.21.15.151:0/2635951185 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff25810b730 0x7ff25810bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:12.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.811+0000 7ff260000700 1 --2- 172.21.15.151:0/2635951185 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2581060f0 0x7ff258112600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:12.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.811+0000 7ff260000700 1 --2- 172.21.15.151:0/2635951185 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff258105730 0x7ff258105bb0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:12.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.811+0000 7ff260000700 1 -- 172.21.15.151:0/2635951185 >> 172.21.15.151:0/2635951185 conn(0x7ff258101220 msgr2=0x7ff258103640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:00:12.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.811+0000 7ff260000700 1 -- 172.21.15.151:0/2635951185 shutdown_connections 2024-05-21T22:00:12.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.811+0000 7ff260000700 1 -- 172.21.15.151:0/2635951185 wait complete. 2024-05-21T22:00:12.813 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.812+0000 7ff260000700 1 Processor -- start 2024-05-21T22:00:12.813 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.812+0000 7ff260000700 1 -- start start 2024-05-21T22:00:12.813 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.812+0000 7ff260000700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff258105730 0x7ff25819bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:12.813 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.812+0000 7ff260000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff2581060f0 0x7ff25819c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:12.813 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.812+0000 7ff260000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff25810b730 0x7ff2581a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:12.813 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.813+0000 7ff260000700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff258115930 con 0x7ff2581060f0 2024-05-21T22:00:12.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.813+0000 7ff260000700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff2581157b0 con 0x7ff25810b730 2024-05-21T22:00:12.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.813+0000 7ff260000700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff258115ab0 con 0x7ff258105730 2024-05-21T22:00:12.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.813+0000 7ff25cc00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff2581060f0 0x7ff25819c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:12.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.813+0000 7ff25cc00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff2581060f0 0x7ff25819c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:48852/0 (socket says 172.21.15.151:48852) 2024-05-21T22:00:12.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.813+0000 7ff25cc00700 1 -- 172.21.15.151:0/2454718514 learned_addr learned my addr 172.21.15.151:0/2454718514 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:00:12.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.813+0000 7ff25d600700 1 --2- 172.21.15.151:0/2454718514 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff258105730 0x7ff25819bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:12.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.813+0000 7ff25e000700 1 --2- 172.21.15.151:0/2454718514 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff25810b730 0x7ff2581a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:12.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.813+0000 7ff25cc00700 1 -- 172.21.15.151:0/2454718514 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff258105730 msgr2=0x7ff25819bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:12.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.813+0000 7ff25cc00700 1 --2- 172.21.15.151:0/2454718514 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff258105730 0x7ff25819bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:12.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.813+0000 7ff25cc00700 1 -- 172.21.15.151:0/2454718514 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff25810b730 msgr2=0x7ff2581a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:12.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.813+0000 7ff25cc00700 1 --2- 172.21.15.151:0/2454718514 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff25810b730 0x7ff2581a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:12.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.813+0000 7ff25cc00700 1 -- 172.21.15.151:0/2454718514 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2581a0df0 con 0x7ff2581060f0 2024-05-21T22:00:12.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.814+0000 7ff25cc00700 1 --2- 172.21.15.151:0/2454718514 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff2581060f0 0x7ff25819c320 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7ff2440057c0 tx=0x7ff244009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:00:12.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.814+0000 7ff256000700 1 -- 172.21.15.151:0/2454718514 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff244026070 con 0x7ff2581060f0 2024-05-21T22:00:12.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.814+0000 7ff260000700 1 -- 172.21.15.151:0/2454718514 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2581a1080 con 0x7ff2581060f0 2024-05-21T22:00:12.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.814+0000 7ff256000700 1 -- 172.21.15.151:0/2454718514 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff244021410 con 0x7ff2581060f0 2024-05-21T22:00:12.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.814+0000 7ff256000700 1 -- 172.21.15.151:0/2454718514 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff244017650 con 0x7ff2581060f0 2024-05-21T22:00:12.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.814+0000 7ff260000700 1 -- 172.21.15.151:0/2454718514 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2581a8960 con 0x7ff2581060f0 2024-05-21T22:00:12.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.817+0000 7ff256000700 1 -- 172.21.15.151:0/2454718514 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7ff244029070 con 0x7ff2581060f0 2024-05-21T22:00:12.819 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.819+0000 7ff26a8dcb80 1 Processor -- start 2024-05-21T22:00:12.819 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.819+0000 7ff26a8dcb80 1 -- start start 2024-05-21T22:00:12.819 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.819+0000 7ff26a8dcb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651a8bf4610 0x5651a8bf49f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:12.819 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.819+0000 7ff26a8dcb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5651a8bf4f30 0x5651a8beeb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:12.819 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.819+0000 7ff256000700 1 --2- 172.21.15.151:0/2454718514 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7ff2400b2a70 0x7ff2400b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:12.819 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.819+0000 7ff26a8dcb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5651a8bef140 0x5651a8bef5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:12.820 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.819+0000 7ff26a8dcb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5651a8b4ce40 con 0x5651a8bef140 2024-05-21T22:00:12.820 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.819+0000 7ff26a8dcb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5651a8b3c9e0 con 0x5651a8bf4f30 2024-05-21T22:00:12.820 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.819+0000 7ff26a8dcb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5651a8b40a20 con 0x5651a8bf4610 2024-05-21T22:00:12.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.819+0000 7ff256000700 1 -- 172.21.15.151:0/2454718514 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7ff2440e3220 con 0x7ff2581060f0 2024-05-21T22:00:12.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.820+0000 7ff25d600700 1 --2- 172.21.15.151:0/2454718514 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7ff2400b2a70 0x7ff2400b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:12.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.820+0000 7ff25e000700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651a8bf4610 0x5651a8bf49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:12.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.820+0000 7ff25e000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5651a8bef140 0x5651a8bef5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:12.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.820+0000 7ff25e000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5651a8bef140 0x5651a8bef5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:48868/0 (socket says 172.21.15.151:48868) 2024-05-21T22:00:12.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.820+0000 7ff25e000700 1 -- 172.21.15.151:0/2409084313 learned_addr learned my addr 172.21.15.151:0/2409084313 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:00:12.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.820+0000 7ff25cc00700 1 --2- 172.21.15.151:0/2409084313 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5651a8bf4f30 0x5651a8beeb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:12.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.820+0000 7ff25e000700 1 -- 172.21.15.151:0/2409084313 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651a8bf4610 msgr2=0x5651a8bf49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:12.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.820+0000 7ff25e000700 1 --2- 172.21.15.151:0/2409084313 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651a8bf4610 0x5651a8bf49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:12.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.820+0000 7ff25e000700 1 -- 172.21.15.151:0/2409084313 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5651a8bf4f30 msgr2=0x5651a8beeb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:12.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.820+0000 7ff25e000700 1 --2- 172.21.15.151:0/2409084313 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5651a8bf4f30 0x5651a8beeb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:12.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.820+0000 7ff25e000700 1 -- 172.21.15.151:0/2409084313 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651a8b05420 con 0x5651a8bef140 2024-05-21T22:00:12.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.820+0000 7ff25cc00700 1 --2- 172.21.15.151:0/2409084313 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5651a8bf4f30 0x5651a8beeb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:00:12.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.821+0000 7ff25e000700 1 --2- 172.21.15.151:0/2409084313 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5651a8bef140 0x5651a8bef5c0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7ff250008ca0 tx=0x7ff25000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69a816e68e5046c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:00:12.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.821+0000 7ff25d600700 1 --2- 172.21.15.151:0/2454718514 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7ff2400b2a70 0x7ff2400b4f30 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff24c007b60 tx=0x7ff24c0073d0 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:00:12.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.821+0000 7ff24aa00700 1 -- 172.21.15.151:0/2409084313 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff250020070 con 0x5651a8bef140 2024-05-21T22:00:12.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.821+0000 7ff24aa00700 1 -- 172.21.15.151:0/2409084313 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff250018b20 con 0x5651a8bef140 2024-05-21T22:00:12.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.821+0000 7ff24aa00700 1 -- 172.21.15.151:0/2409084313 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff250007a00 con 0x5651a8bef140 2024-05-21T22:00:12.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.822+0000 7ff26a8dcb80 1 -- 172.21.15.151:0/2409084313 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5651a8bef140 msgr2=0x5651a8bef5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:12.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.822+0000 7ff26a8dcb80 1 --2- 172.21.15.151:0/2409084313 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5651a8bef140 0x5651a8bef5c0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7ff250008ca0 tx=0x7ff25000b040 comp rx=0 tx=0).stop 2024-05-21T22:00:12.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.822+0000 7ff26a8dcb80 1 -- 172.21.15.151:0/2409084313 shutdown_connections 2024-05-21T22:00:12.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.822+0000 7ff26a8dcb80 1 --2- 172.21.15.151:0/2409084313 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5651a8bef140 0x5651a8bef5c0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:12.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.822+0000 7ff26a8dcb80 1 --2- 172.21.15.151:0/2409084313 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651a8bf4610 0x5651a8bf49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:12.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.822+0000 7ff26a8dcb80 1 --2- 172.21.15.151:0/2409084313 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5651a8bf4f30 0x5651a8beeb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:12.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.822+0000 7ff26a8dcb80 1 -- 172.21.15.151:0/2409084313 >> 172.21.15.151:0/2409084313 conn(0x5651a8b1cbe0 msgr2=0x5651a8be9690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:00:12.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.822+0000 7ff26a8dcb80 1 -- 172.21.15.151:0/2409084313 shutdown_connections 2024-05-21T22:00:12.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.822+0000 7ff26a8dcb80 1 -- 172.21.15.151:0/2409084313 wait complete. 2024-05-21T22:00:12.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.823+0000 7ff26a8dcb80 1 -- start start 2024-05-21T22:00:12.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.823+0000 7ff26a8dcb80 1 Processor -- start 2024-05-21T22:00:12.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.823+0000 7ff26a8dcb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5651a8bef140 0x5651a8bfacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:12.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.823+0000 7ff26a8dcb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651a8bf4610 0x5651a8bf8070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:12.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.823+0000 7ff26a8dcb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5651a8bf4f30 0x5651a8bf85b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:12.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.823+0000 7ff26a8dcb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5651a8a42fa0 con 0x5651a8bf4f30 2024-05-21T22:00:12.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.823+0000 7ff26a8dcb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5651a8add760 con 0x5651a8bef140 2024-05-21T22:00:12.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.823+0000 7ff26a8dcb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5651a8bfb9e0 con 0x5651a8bf4610 2024-05-21T22:00:12.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.823+0000 7ff26a8dcb80 10 client.0 _ll_register_callbacks cb 0x5651a8ae6c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:00:12.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.824+0000 7ff25cc00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651a8bf4610 0x5651a8bf8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:12.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.824+0000 7ff25e000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5651a8bf4f30 0x5651a8bf85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:12.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.824+0000 7ff25cc00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651a8bf4610 0x5651a8bf8070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:36940/0 (socket says 172.21.15.151:36940) 2024-05-21T22:00:12.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.824+0000 7ff25cc00700 1 -- 172.21.15.151:0/494196171 learned_addr learned my addr 172.21.15.151:0/494196171 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:00:12.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.824+0000 7ff25d600700 1 --2- 172.21.15.151:0/494196171 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5651a8bef140 0x5651a8bfacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:12.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.824+0000 7ff25cc00700 1 -- 172.21.15.151:0/494196171 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5651a8bef140 msgr2=0x5651a8bfacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:12.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.824+0000 7ff25cc00700 1 --2- 172.21.15.151:0/494196171 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5651a8bef140 0x5651a8bfacd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:12.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.824+0000 7ff25cc00700 1 -- 172.21.15.151:0/494196171 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5651a8bf4f30 msgr2=0x5651a8bf85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:12.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.824+0000 7ff25cc00700 1 --2- 172.21.15.151:0/494196171 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5651a8bf4f30 0x5651a8bf85b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:12.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.824+0000 7ff25cc00700 1 -- 172.21.15.151:0/494196171 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651a8a8c8d0 con 0x5651a8bf4610 2024-05-21T22:00:12.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.824+0000 7ff25cc00700 1 --2- 172.21.15.151:0/494196171 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651a8bf4610 0x5651a8bf8070 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff2440e0a70 tx=0x7ff244021ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9efdd0d2d73cc49f server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:00:12.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.824+0000 7ff23f400700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-21T22:00:12.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.825+0000 7ff23f400700 1 -- 172.21.15.151:0/494196171 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2440e3020 con 0x5651a8bf4610 2024-05-21T22:00:12.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.825+0000 7ff23f400700 1 -- 172.21.15.151:0/494196171 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7ff2440a6c20 con 0x5651a8bf4610 2024-05-21T22:00:12.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.825+0000 7ff26a8dcb80 1 -- 172.21.15.151:0/494196171 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff250018b20 con 0x5651a8bf4610 2024-05-21T22:00:12.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.825+0000 7ff23f400700 1 -- 172.21.15.151:0/494196171 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff2440a9da0 con 0x5651a8bf4610 2024-05-21T22:00:12.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.825+0000 7ff26a8dcb80 10 client.26266 fetch_fsmap learned FSMap version 29 2024-05-21T22:00:12.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.825+0000 7ff26a8dcb80 1 -- 172.21.15.151:0/494196171 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5651a8b0d060 con 0x5651a8bf4610 2024-05-21T22:00:12.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.825+0000 7ff23f400700 1 -- 172.21.15.151:0/494196171 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff250018b20 con 0x5651a8bf4610 2024-05-21T22:00:12.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.825+0000 7ff23f400700 1 -- 172.21.15.151:0/494196171 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7ff2440a8a00 con 0x5651a8bf4610 2024-05-21T22:00:12.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.825+0000 7ff26a8dcb80 10 client.26266 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:00:12.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.825+0000 7ff26a8dcb80 10 client.26266 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-05-21T22:00:12.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.826+0000 7ff26a8dcb80 20 client.26266 populate_metadata read hostname 'smithi151' 2024-05-21T22:00:12.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.826+0000 7ff26a8dcb80 1 --2- 172.21.15.151:0/494196171 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x5651a8c085a0 0x5651a8c0a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:12.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.826+0000 7ff26a8dcb80 4 client.26266 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T22:00:12.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.826+0000 7ff26a8dcb80 1 -- 172.21.15.151:0/494196171 --> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5651a8b4ed90 con 0x5651a8c085a0 2024-05-21T22:00:12.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.826+0000 7ff25d600700 1 --2- 172.21.15.151:0/494196171 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x5651a8c085a0 0x5651a8c0a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:12.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.827+0000 7ff25d600700 1 --2- 172.21.15.151:0/494196171 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x5651a8c085a0 0x5651a8c0a9c0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=89e2f680ffc19d45 server_cookie=92ba83865b3a3a6e in_seq=0 out_seq=0 2024-05-21T22:00:12.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.827+0000 7ff23f400700 10 client.26266 ms_handle_connect on v2:172.21.15.164:6834/2340428470 2024-05-21T22:00:12.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.828+0000 7ff23f400700 1 -- 172.21.15.151:0/494196171 <== mds.4 v2:172.21.15.164:6834/2340428470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5651a8b4ed90 con 0x5651a8c085a0 2024-05-21T22:00:12.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.828+0000 7ff23f400700 10 client.26266 handle_command_reply: tid=0 2024-05-21T22:00:12.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.828+0000 7ff26a8dcb80 1 client.26266 shutdown 2024-05-21T22:00:12.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.828+0000 7ff26a8dcb80 1 -- 172.21.15.151:0/494196171 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651a8bf4610 msgr2=0x5651a8bf8070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:12.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.828+0000 7ff26a8dcb80 1 --2- 172.21.15.151:0/494196171 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651a8bf4610 0x5651a8bf8070 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff2440e0a70 tx=0x7ff244021ba0 comp rx=0 tx=0).stop 2024-05-21T22:00:12.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.828+0000 7ff26a8dcb80 1 -- 172.21.15.151:0/494196171 shutdown_connections 2024-05-21T22:00:12.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.828+0000 7ff26a8dcb80 1 --2- 172.21.15.151:0/494196171 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x5651a8c085a0 0x5651a8c0a9c0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:12.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.828+0000 7ff26a8dcb80 1 --2- 172.21.15.151:0/494196171 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5651a8bef140 0x5651a8bfacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:12.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.828+0000 7ff26a8dcb80 1 --2- 172.21.15.151:0/494196171 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5651a8bf4f30 0x5651a8bf85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:12.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.828+0000 7ff26a8dcb80 1 --2- 172.21.15.151:0/494196171 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651a8bf4610 0x5651a8bf8070 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:12.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.828+0000 7ff26a8dcb80 1 -- 172.21.15.151:0/494196171 >> 172.21.15.151:0/494196171 conn(0x5651a8b1cbe0 msgr2=0x5651a8be8d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:00:12.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.828+0000 7ff23f400700 0 client.26266 ms_handle_reset on v2:172.21.15.164:6834/2340428470 2024-05-21T22:00:12.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.828+0000 7ff26a8dcb80 1 -- 172.21.15.151:0/494196171 shutdown_connections 2024-05-21T22:00:12.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.828+0000 7ff26a8dcb80 1 -- 172.21.15.151:0/494196171 wait complete. 2024-05-21T22:00:12.831 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.828+0000 7ff26a8dcb80 20 client.26266 trim_cache size 0 max 16384 2024-05-21T22:00:12.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.845+0000 7ff26a8dcb80 1 Processor -- start 2024-05-21T22:00:12.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.845+0000 7ff26a8dcb80 1 -- start start 2024-05-21T22:00:12.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.846+0000 7ff26a8dcb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651a8bef140 0x5651a8beeb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:12.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.846+0000 7ff26a8dcb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5651a8bf4610 0x5651a8bf27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:12.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.846+0000 7ff26a8dcb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5651a8bf4f30 0x5651a8bf5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:12.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.846+0000 7ff26a8dcb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff2581afe50 con 0x5651a8bf4f30 2024-05-21T22:00:12.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.846+0000 7ff26a8dcb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff2581b1990 con 0x5651a8bf4610 2024-05-21T22:00:12.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.846+0000 7ff26a8dcb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff2581b0520 con 0x5651a8bef140 2024-05-21T22:00:12.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.846+0000 7ff25e000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5651a8bf4f30 0x5651a8bf5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:12.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.846+0000 7ff25d600700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651a8bef140 0x5651a8beeb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:12.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.846+0000 7ff25cc00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5651a8bf4610 0x5651a8bf27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:12.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.846+0000 7ff25d600700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651a8bef140 0x5651a8beeb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:36946/0 (socket says 172.21.15.151:36946) 2024-05-21T22:00:12.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.846+0000 7ff25cc00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5651a8bf4610 0x5651a8bf27f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:60574/0 (socket says 172.21.15.151:60574) 2024-05-21T22:00:12.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.846+0000 7ff25d600700 1 -- 172.21.15.151:0/1161228017 learned_addr learned my addr 172.21.15.151:0/1161228017 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:00:12.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.846+0000 7ff25e000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5651a8bf4f30 0x5651a8bf5ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:48894/0 (socket says 172.21.15.151:48894) 2024-05-21T22:00:12.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.847+0000 7ff25d600700 1 -- 172.21.15.151:0/1161228017 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5651a8bf4610 msgr2=0x5651a8bf27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:12.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.847+0000 7ff25d600700 1 --2- 172.21.15.151:0/1161228017 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5651a8bf4610 0x5651a8bf27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:12.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.847+0000 7ff25d600700 1 -- 172.21.15.151:0/1161228017 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5651a8bf4f30 msgr2=0x5651a8bf5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:12.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.847+0000 7ff25d600700 1 --2- 172.21.15.151:0/1161228017 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5651a8bf4f30 0x5651a8bf5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:12.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.847+0000 7ff25d600700 1 -- 172.21.15.151:0/1161228017 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651a8bf6600 con 0x5651a8bef140 2024-05-21T22:00:12.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.847+0000 7ff25cc00700 1 --2- 172.21.15.151:0/1161228017 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5651a8bf4610 0x5651a8bf27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:00:12.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.847+0000 7ff25d600700 1 --2- 172.21.15.151:0/1161228017 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651a8bef140 0x5651a8beeb20 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff24c007940 tx=0x7ff24c013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e28974bbd27b6b34 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:00:12.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.847+0000 7ff24a000700 1 -- 172.21.15.151:0/1161228017 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff24c02b070 con 0x5651a8bef140 2024-05-21T22:00:12.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.847+0000 7ff24a000700 1 -- 172.21.15.151:0/1161228017 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff24c004450 con 0x5651a8bef140 2024-05-21T22:00:12.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.847+0000 7ff24a000700 1 -- 172.21.15.151:0/1161228017 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff24c0118d0 con 0x5651a8bef140 2024-05-21T22:00:12.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.848+0000 7ff26a8dcb80 1 -- 172.21.15.151:0/1161228017 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651a8bef140 msgr2=0x5651a8beeb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:12.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.848+0000 7ff26a8dcb80 1 --2- 172.21.15.151:0/1161228017 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651a8bef140 0x5651a8beeb20 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff24c007940 tx=0x7ff24c013000 comp rx=0 tx=0).stop 2024-05-21T22:00:12.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.848+0000 7ff26a8dcb80 1 -- 172.21.15.151:0/1161228017 shutdown_connections 2024-05-21T22:00:12.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.848+0000 7ff26a8dcb80 1 --2- 172.21.15.151:0/1161228017 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5651a8bf4f30 0x5651a8bf5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:12.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.848+0000 7ff26a8dcb80 1 --2- 172.21.15.151:0/1161228017 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651a8bef140 0x5651a8beeb20 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:12.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.848+0000 7ff26a8dcb80 1 --2- 172.21.15.151:0/1161228017 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5651a8bf4610 0x5651a8bf27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:12.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.848+0000 7ff26a8dcb80 1 -- 172.21.15.151:0/1161228017 >> 172.21.15.151:0/1161228017 conn(0x5651a8b1cbe0 msgr2=0x5651a8bf6980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:00:12.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.848+0000 7ff26a8dcb80 1 -- 172.21.15.151:0/1161228017 shutdown_connections 2024-05-21T22:00:12.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.848+0000 7ff26a8dcb80 1 -- 172.21.15.151:0/1161228017 wait complete. 2024-05-21T22:00:12.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.849+0000 7ff26a8dcb80 1 -- start start 2024-05-21T22:00:12.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.849+0000 7ff26a8dcb80 1 Processor -- start 2024-05-21T22:00:12.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.849+0000 7ff26a8dcb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651a8bef140 0x5651a8c01460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:12.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.849+0000 7ff26a8dcb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5651a8bf4610 0x5651a8c0fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:12.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.849+0000 7ff26a8dcb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5651a8bf4f30 0x5651a8c10040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:12.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.849+0000 7ff26a8dcb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff2581b12c0 con 0x5651a8bf4f30 2024-05-21T22:00:12.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.849+0000 7ff26a8dcb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff2581b0bf0 con 0x5651a8bf4610 2024-05-21T22:00:12.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.849+0000 7ff26a8dcb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff2581addf0 con 0x5651a8bef140 2024-05-21T22:00:12.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.850+0000 7ff26a8dcb80 10 client.0 _ll_register_callbacks cb 0x5651a8ae6c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:00:12.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.850+0000 7ff25e000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5651a8bf4f30 0x5651a8c10040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:12.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.850+0000 7ff25e000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5651a8bf4f30 0x5651a8c10040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:48896/0 (socket says 172.21.15.151:48896) 2024-05-21T22:00:12.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.850+0000 7ff25d600700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651a8bef140 0x5651a8c01460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:12.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.850+0000 7ff25e000700 1 -- 172.21.15.151:0/2495484867 learned_addr learned my addr 172.21.15.151:0/2495484867 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:00:12.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.850+0000 7ff25cc00700 1 --2- 172.21.15.151:0/2495484867 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5651a8bf4610 0x5651a8c0fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:12.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.850+0000 7ff25d600700 1 -- 172.21.15.151:0/2495484867 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5651a8bf4610 msgr2=0x5651a8c0fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:12.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.850+0000 7ff25d600700 1 --2- 172.21.15.151:0/2495484867 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5651a8bf4610 0x5651a8c0fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:12.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.850+0000 7ff25d600700 1 -- 172.21.15.151:0/2495484867 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5651a8bf4f30 msgr2=0x5651a8c10040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:12.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.850+0000 7ff25d600700 1 --2- 172.21.15.151:0/2495484867 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5651a8bf4f30 0x5651a8c10040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:12.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.850+0000 7ff25d600700 1 -- 172.21.15.151:0/2495484867 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651a8c10880 con 0x5651a8bef140 2024-05-21T22:00:12.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.851+0000 7ff25d600700 1 --2- 172.21.15.151:0/2495484867 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651a8bef140 0x5651a8c01460 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff24c004320 tx=0x7ff24c004680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92efc3f3aa96a126 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:00:12.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.851+0000 7ff23f400700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-21T22:00:12.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.851+0000 7ff23f400700 1 -- 172.21.15.151:0/2495484867 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff24c02b040 con 0x5651a8bef140 2024-05-21T22:00:12.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.851+0000 7ff26a8dcb80 1 -- 172.21.15.151:0/2495484867 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff24c004450 con 0x5651a8bef140 2024-05-21T22:00:12.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.851+0000 7ff23f400700 1 -- 172.21.15.151:0/2495484867 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7ff24c003680 con 0x5651a8bef140 2024-05-21T22:00:12.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.851+0000 7ff23f400700 1 -- 172.21.15.151:0/2495484867 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff24c003880 con 0x5651a8bef140 2024-05-21T22:00:12.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.851+0000 7ff26a8dcb80 10 client.26272 fetch_fsmap learned FSMap version 29 2024-05-21T22:00:12.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.851+0000 7ff26a8dcb80 1 -- 172.21.15.151:0/2495484867 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5651a8c731b0 con 0x5651a8bef140 2024-05-21T22:00:12.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.851+0000 7ff23f400700 1 -- 172.21.15.151:0/2495484867 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff24c004450 con 0x5651a8bef140 2024-05-21T22:00:12.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.852+0000 7ff23f400700 1 -- 172.21.15.151:0/2495484867 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7ff24c02e070 con 0x5651a8bef140 2024-05-21T22:00:12.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.852+0000 7ff26a8dcb80 10 client.26272 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:00:12.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.852+0000 7ff26a8dcb80 10 client.26272 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-05-21T22:00:12.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.852+0000 7ff26a8dcb80 20 client.26272 populate_metadata read hostname 'smithi151' 2024-05-21T22:00:12.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.852+0000 7ff26a8dcb80 1 --2- 172.21.15.151:0/2495484867 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x5651a8c73740 0x5651a8c75b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:12.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.852+0000 7ff26a8dcb80 4 client.26272 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T22:00:12.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.852+0000 7ff26a8dcb80 1 -- 172.21.15.151:0/2495484867 --> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5651a8b48570 con 0x5651a8c73740 2024-05-21T22:00:12.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.852+0000 7ff25cc00700 1 --2- 172.21.15.151:0/2495484867 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x5651a8c73740 0x5651a8c75b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:12.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.853+0000 7ff25cc00700 1 --2- 172.21.15.151:0/2495484867 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x5651a8c73740 0x5651a8c75b30 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=989b9673baf9419d server_cookie=26fea76c95499629 in_seq=0 out_seq=0 2024-05-21T22:00:12.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.853+0000 7ff23f400700 10 client.26272 ms_handle_connect on v2:172.21.15.164:6834/2340428470 2024-05-21T22:00:12.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.853+0000 7ff23f400700 1 -- 172.21.15.151:0/2495484867 <== mds.4 v2:172.21.15.164:6834/2340428470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5651a8b48570 con 0x5651a8c73740 2024-05-21T22:00:12.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.853+0000 7ff23f400700 10 client.26272 handle_command_reply: tid=0 2024-05-21T22:00:12.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.853+0000 7ff26a8dcb80 1 client.26272 shutdown 2024-05-21T22:00:12.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.853+0000 7ff26a8dcb80 1 -- 172.21.15.151:0/2495484867 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651a8bef140 msgr2=0x5651a8c01460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:12.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.853+0000 7ff26a8dcb80 1 --2- 172.21.15.151:0/2495484867 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651a8bef140 0x5651a8c01460 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff24c004320 tx=0x7ff24c004680 comp rx=0 tx=0).stop 2024-05-21T22:00:12.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.853+0000 7ff26a8dcb80 1 -- 172.21.15.151:0/2495484867 shutdown_connections 2024-05-21T22:00:12.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.853+0000 7ff26a8dcb80 1 --2- 172.21.15.151:0/2495484867 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x5651a8c73740 0x5651a8c75b30 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:12.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.853+0000 7ff26a8dcb80 1 --2- 172.21.15.151:0/2495484867 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5651a8bf4f30 0x5651a8c10040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:12.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.853+0000 7ff26a8dcb80 1 --2- 172.21.15.151:0/2495484867 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5651a8bef140 0x5651a8c01460 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:12.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.853+0000 7ff26a8dcb80 1 --2- 172.21.15.151:0/2495484867 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5651a8bf4610 0x5651a8c0fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:12.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.853+0000 7ff23f400700 0 client.26272 ms_handle_reset on v2:172.21.15.164:6834/2340428470 2024-05-21T22:00:12.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.853+0000 7ff26a8dcb80 1 -- 172.21.15.151:0/2495484867 >> 172.21.15.151:0/2495484867 conn(0x5651a8b1cbe0 msgr2=0x5651a8bf7210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:00:12.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.853+0000 7ff26a8dcb80 1 -- 172.21.15.151:0/2495484867 shutdown_connections 2024-05-21T22:00:12.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.853+0000 7ff26a8dcb80 1 -- 172.21.15.151:0/2495484867 wait complete. 2024-05-21T22:00:12.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.854+0000 7ff26a8dcb80 20 client.26272 trim_cache size 0 max 16384 2024-05-21T22:00:12.857 INFO:teuthology.orchestra.run.smithi151.stdout:[] 2024-05-21T22:00:12.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.854+0000 7ff260000700 1 -- 172.21.15.151:0/2454718514 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7ff2400b2a70 msgr2=0x7ff2400b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:12.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.854+0000 7ff260000700 1 --2- 172.21.15.151:0/2454718514 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7ff2400b2a70 0x7ff2400b4f30 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff24c007b60 tx=0x7ff24c0073d0 comp rx=0 tx=0).stop 2024-05-21T22:00:12.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.854+0000 7ff260000700 1 -- 172.21.15.151:0/2454718514 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff2581060f0 msgr2=0x7ff25819c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:12.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.854+0000 7ff260000700 1 --2- 172.21.15.151:0/2454718514 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff2581060f0 0x7ff25819c320 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7ff2440057c0 tx=0x7ff244009580 comp rx=0 tx=0).stop 2024-05-21T22:00:12.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.854+0000 7ff260000700 1 -- 172.21.15.151:0/2454718514 shutdown_connections 2024-05-21T22:00:12.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.854+0000 7ff260000700 1 --2- 172.21.15.151:0/2454718514 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7ff2400b2a70 0x7ff2400b4f30 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:12.858 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.854+0000 7ff260000700 1 --2- 172.21.15.151:0/2454718514 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff25810b730 0x7ff2581a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:12.858 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.854+0000 7ff260000700 1 --2- 172.21.15.151:0/2454718514 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff258105730 0x7ff25819bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:12.858 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.854+0000 7ff260000700 1 --2- 172.21.15.151:0/2454718514 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff2581060f0 0x7ff25819c320 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:12.858 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.854+0000 7ff260000700 1 -- 172.21.15.151:0/2454718514 >> 172.21.15.151:0/2454718514 conn(0x7ff258101220 msgr2=0x7ff2581018b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:00:12.858 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.855+0000 7ff260000700 1 -- 172.21.15.151:0/2454718514 shutdown_connections 2024-05-21T22:00:12.858 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:12.855+0000 7ff260000700 1 -- 172.21.15.151:0/2454718514 wait complete. 2024-05-21T22:00:13.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:00:13 smithi151 ceph-mon[31623]: pgmap v141: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-05-21T22:00:13.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:00:13 smithi164 ceph-mon[36269]: pgmap v141: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-05-21T22:00:13.805 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-21T22:00:13.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:00:13 smithi184 ceph-mon[36942]: pgmap v141: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-05-21T22:00:14.243 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:00:15.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:00:15 smithi164 ceph-mon[36269]: pgmap v142: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-05-21T22:00:15.928 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:00:15 smithi151 ceph-mon[31623]: pgmap v142: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-05-21T22:00:15.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:00:15 smithi184 ceph-mon[36942]: pgmap v142: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-05-21T22:00:16.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.411+0000 7f921f000700 1 -- 172.21.15.151:0/4033295270 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9218105760 msgr2=0x7f9218105be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:16.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.411+0000 7f921f000700 1 --2- 172.21.15.151:0/4033295270 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9218105760 0x7f9218105be0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9204009990 tx=0x7f9204009ca0 comp rx=0 tx=0).stop 2024-05-21T22:00:16.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.411+0000 7f921f000700 1 -- 172.21.15.151:0/4033295270 shutdown_connections 2024-05-21T22:00:16.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.411+0000 7f921f000700 1 --2- 172.21.15.151:0/4033295270 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9218106120 0x7f9218112630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:16.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.411+0000 7f921f000700 1 --2- 172.21.15.151:0/4033295270 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f921810b760 0x7f921810bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:16.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.411+0000 7f921f000700 1 --2- 172.21.15.151:0/4033295270 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9218105760 0x7f9218105be0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:16.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.411+0000 7f921f000700 1 -- 172.21.15.151:0/4033295270 >> 172.21.15.151:0/4033295270 conn(0x7f9218101250 msgr2=0x7f9218103670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:00:16.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.411+0000 7f921f000700 1 -- 172.21.15.151:0/4033295270 shutdown_connections 2024-05-21T22:00:16.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.411+0000 7f921f000700 1 -- 172.21.15.151:0/4033295270 wait complete. 2024-05-21T22:00:16.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.413+0000 7f921f000700 1 Processor -- start 2024-05-21T22:00:16.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.413+0000 7f921f000700 1 -- start start 2024-05-21T22:00:16.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.413+0000 7f921f000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9218105760 0x7f921819be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:16.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.413+0000 7f921f000700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9218106120 0x7f921819c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:16.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.413+0000 7f921f000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f921810b760 0x7f92181a0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:16.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.413+0000 7f921f000700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f92181159b0 con 0x7f9218105760 2024-05-21T22:00:16.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.413+0000 7f921f000700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9218115830 con 0x7f921810b760 2024-05-21T22:00:16.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.413+0000 7f921f000700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9218115b30 con 0x7f9218106120 2024-05-21T22:00:16.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.414+0000 7f9217e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9218105760 0x7f921819be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:16.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.414+0000 7f9217400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9218106120 0x7f921819c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:16.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.414+0000 7f9217e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9218105760 0x7f921819be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:59820/0 (socket says 172.21.15.151:59820) 2024-05-21T22:00:16.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.414+0000 7f9217e00700 1 -- 172.21.15.151:0/2090859324 learned_addr learned my addr 172.21.15.151:0/2090859324 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:00:16.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.414+0000 7f921d000700 1 --2- 172.21.15.151:0/2090859324 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f921810b760 0x7f92181a0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:16.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.414+0000 7f9217400700 1 -- 172.21.15.151:0/2090859324 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f921810b760 msgr2=0x7f92181a0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:16.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.414+0000 7f9217400700 1 --2- 172.21.15.151:0/2090859324 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f921810b760 0x7f92181a0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:16.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.414+0000 7f9217400700 1 -- 172.21.15.151:0/2090859324 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9218105760 msgr2=0x7f921819be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:16.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.414+0000 7f9217400700 1 --2- 172.21.15.151:0/2090859324 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9218105760 0x7f921819be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:16.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.414+0000 7f9217400700 1 -- 172.21.15.151:0/2090859324 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92181a0e70 con 0x7f9218106120 2024-05-21T22:00:16.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.414+0000 7f9217400700 1 --2- 172.21.15.151:0/2090859324 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9218106120 0x7f921819c3a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f92040098e0 tx=0x7f92040057c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:00:16.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.415+0000 7f9214c00700 1 -- 172.21.15.151:0/2090859324 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9204027070 con 0x7f9218106120 2024-05-21T22:00:16.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.415+0000 7f9214c00700 1 -- 172.21.15.151:0/2090859324 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9204022470 con 0x7f9218106120 2024-05-21T22:00:16.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.415+0000 7f921f000700 1 -- 172.21.15.151:0/2090859324 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92181a1100 con 0x7f9218106120 2024-05-21T22:00:16.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.415+0000 7f9214c00700 1 -- 172.21.15.151:0/2090859324 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9204017650 con 0x7f9218106120 2024-05-21T22:00:16.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.415+0000 7f921f000700 1 -- 172.21.15.151:0/2090859324 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92181a89e0 con 0x7f9218106120 2024-05-21T22:00:16.420 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.419+0000 7f9214c00700 1 -- 172.21.15.151:0/2090859324 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7f920402a070 con 0x7f9218106120 2024-05-21T22:00:16.420 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.420+0000 7f9229983b80 1 Processor -- start 2024-05-21T22:00:16.420 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.420+0000 7f9229983b80 1 -- start start 2024-05-21T22:00:16.421 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.420+0000 7f9229983b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f978a0a610 0x55f978a0a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:16.421 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.420+0000 7f9229983b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f978a0af30 0x55f978a04b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:16.421 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.420+0000 7f9229983b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f978a05140 0x55f978a055c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:16.421 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.420+0000 7f9229983b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55f978962e40 con 0x55f978a0af30 2024-05-21T22:00:16.421 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.420+0000 7f9229983b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55f9789529e0 con 0x55f978a0a610 2024-05-21T22:00:16.421 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.420+0000 7f9229983b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55f978956a20 con 0x55f978a05140 2024-05-21T22:00:16.421 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.421+0000 7f9217e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f978a0af30 0x55f978a04b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:16.421 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.421+0000 7f921d000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f978a0a610 0x55f978a0a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:16.422 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.421+0000 7f921d000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f978a0a610 0x55f978a0a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:33066/0 (socket says 172.21.15.151:33066) 2024-05-21T22:00:16.422 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.421+0000 7f921d000700 1 -- 172.21.15.151:0/1376294331 learned_addr learned my addr 172.21.15.151:0/1376294331 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:00:16.422 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.421+0000 7f9217400700 1 --2- 172.21.15.151:0/1376294331 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f978a05140 0x55f978a055c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:16.422 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.421+0000 7f921d000700 1 -- 172.21.15.151:0/1376294331 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f978a05140 msgr2=0x55f978a055c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:16.422 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.421+0000 7f921d000700 1 --2- 172.21.15.151:0/1376294331 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f978a05140 0x55f978a055c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:16.423 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.421+0000 7f921d000700 1 -- 172.21.15.151:0/1376294331 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f978a0af30 msgr2=0x55f978a04b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:16.423 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.421+0000 7f921d000700 1 --2- 172.21.15.151:0/1376294331 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f978a0af30 0x55f978a04b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:16.423 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.421+0000 7f921d000700 1 -- 172.21.15.151:0/1376294331 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f97891b420 con 0x55f978a0a610 2024-05-21T22:00:16.423 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.422+0000 7f921d000700 1 --2- 172.21.15.151:0/1376294331 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f978a0a610 0x55f978a0a9f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f9210011680 tx=0x7f9210009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d220bc717de7f556 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:00:16.423 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.422+0000 7f9214c00700 1 --2- 172.21.15.151:0/2090859324 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f91fc0b2a70 0x7f91fc0b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:16.423 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.422+0000 7f920d600700 1 -- 172.21.15.151:0/1376294331 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9210019030 con 0x55f978a0a610 2024-05-21T22:00:16.423 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.422+0000 7f9214c00700 1 -- 172.21.15.151:0/2090859324 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f92040e2ff0 con 0x7f9218106120 2024-05-21T22:00:16.423 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.422+0000 7f920d600700 1 -- 172.21.15.151:0/1376294331 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92100092e0 con 0x55f978a0a610 2024-05-21T22:00:16.423 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.422+0000 7f9217e00700 1 --2- 172.21.15.151:0/2090859324 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f91fc0b2a70 0x7f91fc0b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:16.423 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.422+0000 7f920d600700 1 -- 172.21.15.151:0/1376294331 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9210007500 con 0x55f978a0a610 2024-05-21T22:00:16.424 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.423+0000 7f9229983b80 1 -- 172.21.15.151:0/1376294331 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f978a0a610 msgr2=0x55f978a0a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:16.424 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.423+0000 7f9229983b80 1 --2- 172.21.15.151:0/1376294331 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f978a0a610 0x55f978a0a9f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f9210011680 tx=0x7f9210009920 comp rx=0 tx=0).stop 2024-05-21T22:00:16.424 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.423+0000 7f9217e00700 1 --2- 172.21.15.151:0/2090859324 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f91fc0b2a70 0x7f91fc0b4f30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f92080096b0 tx=0x7f9208009260 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:00:16.424 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.423+0000 7f9229983b80 1 -- 172.21.15.151:0/1376294331 shutdown_connections 2024-05-21T22:00:16.424 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.423+0000 7f9229983b80 1 --2- 172.21.15.151:0/1376294331 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f978a0a610 0x55f978a0a9f0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:16.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.423+0000 7f9229983b80 1 --2- 172.21.15.151:0/1376294331 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f978a05140 0x55f978a055c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:16.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.423+0000 7f9229983b80 1 --2- 172.21.15.151:0/1376294331 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f978a0af30 0x55f978a04b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:16.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.423+0000 7f9229983b80 1 -- 172.21.15.151:0/1376294331 >> 172.21.15.151:0/1376294331 conn(0x55f978932be0 msgr2=0x55f9789ff690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:00:16.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.423+0000 7f9229983b80 1 -- 172.21.15.151:0/1376294331 shutdown_connections 2024-05-21T22:00:16.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.423+0000 7f9229983b80 1 -- 172.21.15.151:0/1376294331 wait complete. 2024-05-21T22:00:16.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.424+0000 7f9229983b80 1 -- start start 2024-05-21T22:00:16.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.424+0000 7f9229983b80 1 Processor -- start 2024-05-21T22:00:16.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.424+0000 7f9229983b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f978a05140 0x55f978a10c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:16.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.424+0000 7f9229983b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f978a0a610 0x55f978a0e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:16.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.424+0000 7f9229983b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f978a0af30 0x55f978a0e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:16.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.424+0000 7f9229983b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55f978858fa0 con 0x55f978a0a610 2024-05-21T22:00:16.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.424+0000 7f9229983b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55f9788f3760 con 0x55f978a0af30 2024-05-21T22:00:16.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.424+0000 7f9229983b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55f978a119e0 con 0x55f978a05140 2024-05-21T22:00:16.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.425+0000 7f9229983b80 10 client.0 _ll_register_callbacks cb 0x55f9788fcc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:00:16.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.425+0000 7f9217e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f978a05140 0x55f978a10c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:16.428 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.425+0000 7f9217e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f978a05140 0x55f978a10c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:32954/0 (socket says 172.21.15.151:32954) 2024-05-21T22:00:16.428 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.425+0000 7f9217e00700 1 -- 172.21.15.151:0/308457564 learned_addr learned my addr 172.21.15.151:0/308457564 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:00:16.428 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.425+0000 7f921d000700 1 --2- 172.21.15.151:0/308457564 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f978a0af30 0x55f978a0e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:16.428 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.425+0000 7f9217400700 1 --2- 172.21.15.151:0/308457564 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f978a0a610 0x55f978a0e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:16.428 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.425+0000 7f9217e00700 1 -- 172.21.15.151:0/308457564 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f978a0af30 msgr2=0x55f978a0e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:16.428 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.425+0000 7f9217e00700 1 --2- 172.21.15.151:0/308457564 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f978a0af30 0x55f978a0e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:16.428 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.425+0000 7f9217e00700 1 -- 172.21.15.151:0/308457564 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f978a0a610 msgr2=0x55f978a0e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:16.428 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.425+0000 7f9217e00700 1 --2- 172.21.15.151:0/308457564 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f978a0a610 0x55f978a0e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:16.428 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.425+0000 7f9217e00700 1 -- 172.21.15.151:0/308457564 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9788a28d0 con 0x55f978a05140 2024-05-21T22:00:16.428 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.425+0000 7f9217400700 1 --2- 172.21.15.151:0/308457564 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f978a0a610 0x55f978a0e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T22:00:16.428 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.425+0000 7f921d000700 1 --2- 172.21.15.151:0/308457564 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f978a0af30 0x55f978a0e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:00:16.428 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.426+0000 7f9217e00700 1 --2- 172.21.15.151:0/308457564 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f978a05140 0x55f978a10c90 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9208004620 tx=0x7f9208004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63e1caddea15ab39 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:00:16.428 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.426+0000 7f9202000700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-21T22:00:16.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.426+0000 7f9202000700 1 -- 172.21.15.151:0/308457564 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f920802c060 con 0x55f978a05140 2024-05-21T22:00:16.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.426+0000 7f9202000700 1 -- 172.21.15.151:0/308457564 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f920801d440 con 0x55f978a05140 2024-05-21T22:00:16.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.426+0000 7f9229983b80 1 -- 172.21.15.151:0/308457564 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92100092e0 con 0x55f978a05140 2024-05-21T22:00:16.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.426+0000 7f9202000700 1 -- 172.21.15.151:0/308457564 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f920801d640 con 0x55f978a05140 2024-05-21T22:00:16.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.426+0000 7f9229983b80 10 client.26284 fetch_fsmap learned FSMap version 29 2024-05-21T22:00:16.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.426+0000 7f9229983b80 1 -- 172.21.15.151:0/308457564 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f978923060 con 0x55f978a05140 2024-05-21T22:00:16.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.427+0000 7f9202000700 1 -- 172.21.15.151:0/308457564 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92100092e0 con 0x55f978a05140 2024-05-21T22:00:16.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.427+0000 7f9202000700 1 -- 172.21.15.151:0/308457564 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f9208026030 con 0x55f978a05140 2024-05-21T22:00:16.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.427+0000 7f9229983b80 10 client.26284 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:00:16.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.427+0000 7f9229983b80 10 client.26284 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-21T22:00:16.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.427+0000 7f9229983b80 20 client.26284 populate_metadata read hostname 'smithi151' 2024-05-21T22:00:16.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.427+0000 7f9229983b80 1 --2- 172.21.15.151:0/308457564 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x55f978a1e5f0 0x55f978a209e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:16.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.427+0000 7f9229983b80 4 client.26284 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T22:00:16.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.427+0000 7f9229983b80 1 -- 172.21.15.151:0/308457564 --> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f978964d90 con 0x55f978a1e5f0 2024-05-21T22:00:16.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.428+0000 7f9217400700 1 --2- 172.21.15.151:0/308457564 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x55f978a1e5f0 0x55f978a209e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:16.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.428+0000 7f9217400700 1 --2- 172.21.15.151:0/308457564 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x55f978a1e5f0 0x55f978a209e0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c4101a0a69fdfe7d server_cookie=bb4d0ab510c284a2 in_seq=0 out_seq=0 2024-05-21T22:00:16.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.428+0000 7f9202000700 10 client.26284 ms_handle_connect on v2:172.21.15.184:6836/2988194888 2024-05-21T22:00:16.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.429+0000 7f9202000700 1 -- 172.21.15.151:0/308457564 <== mds.3 v2:172.21.15.184:6836/2988194888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55f978964d90 con 0x55f978a1e5f0 2024-05-21T22:00:16.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.429+0000 7f9202000700 10 client.26284 handle_command_reply: tid=0 2024-05-21T22:00:16.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.429+0000 7f9229983b80 1 client.26284 shutdown 2024-05-21T22:00:16.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.429+0000 7f9229983b80 1 -- 172.21.15.151:0/308457564 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f978a05140 msgr2=0x55f978a10c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:16.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.429+0000 7f9229983b80 1 --2- 172.21.15.151:0/308457564 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f978a05140 0x55f978a10c90 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9208004620 tx=0x7f9208004750 comp rx=0 tx=0).stop 2024-05-21T22:00:16.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.429+0000 7f9229983b80 1 -- 172.21.15.151:0/308457564 shutdown_connections 2024-05-21T22:00:16.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.429+0000 7f9229983b80 1 --2- 172.21.15.151:0/308457564 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x55f978a1e5f0 0x55f978a209e0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:16.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.429+0000 7f9229983b80 1 --2- 172.21.15.151:0/308457564 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f978a0af30 0x55f978a0e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:16.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.429+0000 7f9229983b80 1 --2- 172.21.15.151:0/308457564 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f978a05140 0x55f978a10c90 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:16.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.429+0000 7f9229983b80 1 --2- 172.21.15.151:0/308457564 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f978a0a610 0x55f978a0e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:16.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.429+0000 7f9202000700 0 client.26284 ms_handle_reset on v2:172.21.15.184:6836/2988194888 2024-05-21T22:00:16.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.429+0000 7f9229983b80 1 -- 172.21.15.151:0/308457564 >> 172.21.15.151:0/308457564 conn(0x55f978932be0 msgr2=0x55f9789fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:00:16.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.430+0000 7f9229983b80 1 -- 172.21.15.151:0/308457564 shutdown_connections 2024-05-21T22:00:16.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.430+0000 7f9229983b80 1 -- 172.21.15.151:0/308457564 wait complete. 2024-05-21T22:00:16.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.430+0000 7f9229983b80 20 client.26284 trim_cache size 0 max 16384 2024-05-21T22:00:16.447 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.446+0000 7f9229983b80 1 Processor -- start 2024-05-21T22:00:16.447 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.446+0000 7f9229983b80 1 -- start start 2024-05-21T22:00:16.447 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.447+0000 7f9229983b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f978a05140 0x55f978a04b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:16.447 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.447+0000 7f9229983b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f978a0a610 0x55f978a0e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:16.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.447+0000 7f9229983b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f978a0af30 0x55f9789ff4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:16.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.447+0000 7f9229983b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f92181aff10 con 0x55f978a0a610 2024-05-21T22:00:16.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.447+0000 7f9229983b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f92181b1a50 con 0x55f978a05140 2024-05-21T22:00:16.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.447+0000 7f9229983b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f92181b05e0 con 0x55f978a0af30 2024-05-21T22:00:16.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.447+0000 7f9217400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f978a0a610 0x55f978a0e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:16.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.447+0000 7f9217e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f978a05140 0x55f978a04b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:16.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.447+0000 7f9217400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f978a0a610 0x55f978a0e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:59854/0 (socket says 172.21.15.151:59854) 2024-05-21T22:00:16.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.447+0000 7f9217400700 1 -- 172.21.15.151:0/444198185 learned_addr learned my addr 172.21.15.151:0/444198185 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:00:16.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.447+0000 7f921d000700 1 --2- 172.21.15.151:0/444198185 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f978a0af30 0x55f9789ff4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:16.449 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.448+0000 7f9217e00700 1 -- 172.21.15.151:0/444198185 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f978a0af30 msgr2=0x55f9789ff4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:16.449 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.448+0000 7f9217e00700 1 --2- 172.21.15.151:0/444198185 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f978a0af30 0x55f9789ff4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:16.449 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.448+0000 7f9217e00700 1 -- 172.21.15.151:0/444198185 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f978a0a610 msgr2=0x55f978a0e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:16.449 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.448+0000 7f9217e00700 1 --2- 172.21.15.151:0/444198185 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f978a0a610 0x55f978a0e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:16.449 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.448+0000 7f9217e00700 1 -- 172.21.15.151:0/444198185 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f978a21010 con 0x55f978a05140 2024-05-21T22:00:16.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.448+0000 7f9217e00700 1 --2- 172.21.15.151:0/444198185 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f978a05140 0x55f978a04b20 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f9208019040 tx=0x7f9208003cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e647fca7e71cbee9 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:00:16.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.448+0000 7f920cc00700 1 -- 172.21.15.151:0/444198185 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f920801ed40 con 0x55f978a05140 2024-05-21T22:00:16.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.448+0000 7f920cc00700 1 -- 172.21.15.151:0/444198185 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92080054e0 con 0x55f978a05140 2024-05-21T22:00:16.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.448+0000 7f920cc00700 1 -- 172.21.15.151:0/444198185 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f920801dad0 con 0x55f978a05140 2024-05-21T22:00:16.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.449+0000 7f9229983b80 1 -- 172.21.15.151:0/444198185 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f978a05140 msgr2=0x55f978a04b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:16.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.449+0000 7f9229983b80 1 --2- 172.21.15.151:0/444198185 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f978a05140 0x55f978a04b20 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f9208019040 tx=0x7f9208003cf0 comp rx=0 tx=0).stop 2024-05-21T22:00:16.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.449+0000 7f9229983b80 1 -- 172.21.15.151:0/444198185 shutdown_connections 2024-05-21T22:00:16.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.449+0000 7f9229983b80 1 --2- 172.21.15.151:0/444198185 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f978a05140 0x55f978a04b20 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:16.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.449+0000 7f9229983b80 1 --2- 172.21.15.151:0/444198185 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f978a0af30 0x55f9789ff4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:16.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.449+0000 7f9229983b80 1 --2- 172.21.15.151:0/444198185 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f978a0a610 0x55f978a0e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:16.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.449+0000 7f9229983b80 1 -- 172.21.15.151:0/444198185 >> 172.21.15.151:0/444198185 conn(0x55f978932be0 msgr2=0x55f978a0de10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:00:16.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.449+0000 7f9229983b80 1 -- 172.21.15.151:0/444198185 shutdown_connections 2024-05-21T22:00:16.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.449+0000 7f9229983b80 1 -- 172.21.15.151:0/444198185 wait complete. 2024-05-21T22:00:16.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.450+0000 7f9229983b80 1 -- start start 2024-05-21T22:00:16.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.450+0000 7f9229983b80 1 Processor -- start 2024-05-21T22:00:16.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.450+0000 7f9229983b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f978a05140 0x55f978a17320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:16.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.450+0000 7f9229983b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f978a0a610 0x55f978a25b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:16.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.450+0000 7f9229983b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f978a0af30 0x55f978a26050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:16.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.450+0000 7f9229983b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f92181b1380 con 0x55f978a05140 2024-05-21T22:00:16.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.450+0000 7f9229983b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f92181b0cb0 con 0x55f978a0a610 2024-05-21T22:00:16.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.450+0000 7f9229983b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f92181adeb0 con 0x55f978a0af30 2024-05-21T22:00:16.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.450+0000 7f9229983b80 10 client.0 _ll_register_callbacks cb 0x55f9788fcc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:00:16.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.451+0000 7f9217e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f978a05140 0x55f978a17320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:16.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.451+0000 7f9217e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f978a05140 0x55f978a17320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:59862/0 (socket says 172.21.15.151:59862) 2024-05-21T22:00:16.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.451+0000 7f9217e00700 1 -- 172.21.15.151:0/824022628 learned_addr learned my addr 172.21.15.151:0/824022628 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:00:16.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.451+0000 7f921d000700 1 --2- 172.21.15.151:0/824022628 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f978a0af30 0x55f978a26050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:16.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.451+0000 7f9217400700 1 --2- 172.21.15.151:0/824022628 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f978a0a610 0x55f978a25b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:16.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.451+0000 7f9217e00700 1 -- 172.21.15.151:0/824022628 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f978a0af30 msgr2=0x55f978a26050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:16.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.451+0000 7f9217e00700 1 --2- 172.21.15.151:0/824022628 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f978a0af30 0x55f978a26050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:16.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.451+0000 7f9217e00700 1 -- 172.21.15.151:0/824022628 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f978a0a610 msgr2=0x55f978a25b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:16.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.451+0000 7f9217e00700 1 --2- 172.21.15.151:0/824022628 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f978a0a610 0x55f978a25b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:16.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.451+0000 7f9217e00700 1 -- 172.21.15.151:0/824022628 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f978a26870 con 0x55f978a05140 2024-05-21T22:00:16.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.451+0000 7f9217e00700 1 --2- 172.21.15.151:0/824022628 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f978a05140 0x55f978a17320 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f920801e9f0 tx=0x7f920801e6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4de63ac9d4a3b2e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:00:16.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.451+0000 7f9202000700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-21T22:00:16.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.452+0000 7f9202000700 1 -- 172.21.15.151:0/824022628 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f920802c070 con 0x55f978a05140 2024-05-21T22:00:16.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.452+0000 7f9229983b80 1 -- 172.21.15.151:0/824022628 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92080054e0 con 0x55f978a05140 2024-05-21T22:00:16.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.452+0000 7f9202000700 1 -- 172.21.15.151:0/824022628 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f920801d9f0 con 0x55f978a05140 2024-05-21T22:00:16.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.452+0000 7f9202000700 1 -- 172.21.15.151:0/824022628 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f920804a050 con 0x55f978a05140 2024-05-21T22:00:16.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.452+0000 7f9229983b80 10 client.16155 fetch_fsmap learned FSMap version 29 2024-05-21T22:00:16.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.452+0000 7f9229983b80 1 -- 172.21.15.151:0/824022628 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f978a891b0 con 0x55f978a05140 2024-05-21T22:00:16.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.452+0000 7f9202000700 1 -- 172.21.15.151:0/824022628 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92080054e0 con 0x55f978a05140 2024-05-21T22:00:16.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.453+0000 7f9202000700 1 -- 172.21.15.151:0/824022628 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f9208038070 con 0x55f978a05140 2024-05-21T22:00:16.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.453+0000 7f9229983b80 10 client.16155 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:00:16.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.453+0000 7f9229983b80 10 client.16155 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-21T22:00:16.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.453+0000 7f9229983b80 20 client.16155 populate_metadata read hostname 'smithi151' 2024-05-21T22:00:16.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.453+0000 7f9229983b80 1 --2- 172.21.15.151:0/824022628 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x55f978a897d0 0x55f978a8bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:16.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.453+0000 7f9229983b80 4 client.16155 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T22:00:16.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.453+0000 7f9229983b80 1 -- 172.21.15.151:0/824022628 --> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f97895e570 con 0x55f978a897d0 2024-05-21T22:00:16.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.453+0000 7f9217400700 1 --2- 172.21.15.151:0/824022628 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x55f978a897d0 0x55f978a8bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:16.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.453+0000 7f9217400700 1 --2- 172.21.15.151:0/824022628 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x55f978a897d0 0x55f978a8bbf0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a083275d8310eb5d server_cookie=df47efe85a92c83a in_seq=0 out_seq=0 2024-05-21T22:00:16.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.453+0000 7f9202000700 10 client.16155 ms_handle_connect on v2:172.21.15.184:6836/2988194888 2024-05-21T22:00:16.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.454+0000 7f9202000700 1 -- 172.21.15.151:0/824022628 <== mds.3 v2:172.21.15.184:6836/2988194888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f97895e570 con 0x55f978a897d0 2024-05-21T22:00:16.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.454+0000 7f9202000700 10 client.16155 handle_command_reply: tid=0 2024-05-21T22:00:16.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.454+0000 7f9229983b80 1 client.16155 shutdown 2024-05-21T22:00:16.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.454+0000 7f9229983b80 1 -- 172.21.15.151:0/824022628 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f978a05140 msgr2=0x55f978a17320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:16.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.454+0000 7f9229983b80 1 --2- 172.21.15.151:0/824022628 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f978a05140 0x55f978a17320 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f920801e9f0 tx=0x7f920801e6b0 comp rx=0 tx=0).stop 2024-05-21T22:00:16.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.454+0000 7f9229983b80 1 -- 172.21.15.151:0/824022628 shutdown_connections 2024-05-21T22:00:16.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.454+0000 7f9229983b80 1 --2- 172.21.15.151:0/824022628 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x55f978a897d0 0x55f978a8bbf0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:16.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.454+0000 7f9229983b80 1 --2- 172.21.15.151:0/824022628 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f978a0af30 0x55f978a26050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:16.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.454+0000 7f9229983b80 1 --2- 172.21.15.151:0/824022628 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f978a05140 0x55f978a17320 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:16.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.454+0000 7f9229983b80 1 --2- 172.21.15.151:0/824022628 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f978a0a610 0x55f978a25b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:16.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.454+0000 7f9202000700 0 client.16155 ms_handle_reset on v2:172.21.15.184:6836/2988194888 2024-05-21T22:00:16.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.454+0000 7f9229983b80 1 -- 172.21.15.151:0/824022628 >> 172.21.15.151:0/824022628 conn(0x55f978932be0 msgr2=0x55f978a0b310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:00:16.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.454+0000 7f9229983b80 1 -- 172.21.15.151:0/824022628 shutdown_connections 2024-05-21T22:00:16.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.454+0000 7f9229983b80 1 -- 172.21.15.151:0/824022628 wait complete. 2024-05-21T22:00:16.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.454+0000 7f9229983b80 20 client.16155 trim_cache size 0 max 16384 2024-05-21T22:00:16.458 INFO:teuthology.orchestra.run.smithi151.stdout:[] 2024-05-21T22:00:16.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.455+0000 7f921f000700 1 -- 172.21.15.151:0/2090859324 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f91fc0b2a70 msgr2=0x7f91fc0b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:16.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.455+0000 7f921f000700 1 --2- 172.21.15.151:0/2090859324 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f91fc0b2a70 0x7f91fc0b4f30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f92080096b0 tx=0x7f9208009260 comp rx=0 tx=0).stop 2024-05-21T22:00:16.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.455+0000 7f921f000700 1 -- 172.21.15.151:0/2090859324 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9218106120 msgr2=0x7f921819c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:16.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.455+0000 7f921f000700 1 --2- 172.21.15.151:0/2090859324 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9218106120 0x7f921819c3a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f92040098e0 tx=0x7f92040057c0 comp rx=0 tx=0).stop 2024-05-21T22:00:16.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.455+0000 7f921f000700 1 -- 172.21.15.151:0/2090859324 shutdown_connections 2024-05-21T22:00:16.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.455+0000 7f921f000700 1 --2- 172.21.15.151:0/2090859324 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f91fc0b2a70 0x7f91fc0b4f30 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:16.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.455+0000 7f921f000700 1 --2- 172.21.15.151:0/2090859324 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f921810b760 0x7f92181a0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:16.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.455+0000 7f921f000700 1 --2- 172.21.15.151:0/2090859324 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9218105760 0x7f921819be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:16.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.455+0000 7f921f000700 1 --2- 172.21.15.151:0/2090859324 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9218106120 0x7f921819c3a0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:16.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.455+0000 7f921f000700 1 -- 172.21.15.151:0/2090859324 >> 172.21.15.151:0/2090859324 conn(0x7f9218101250 msgr2=0x7f9218101980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:00:16.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.455+0000 7f921f000700 1 -- 172.21.15.151:0/2090859324 shutdown_connections 2024-05-21T22:00:16.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:16.456+0000 7f921f000700 1 -- 172.21.15.151:0/2090859324 wait complete. 2024-05-21T22:00:17.404 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-21T22:00:17.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:00:17 smithi151 ceph-mon[31623]: pgmap v143: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-05-21T22:00:17.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:00:17 smithi164 ceph-mon[36269]: pgmap v143: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-05-21T22:00:17.866 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:00:17.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:00:17 smithi184 ceph-mon[36942]: pgmap v143: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-05-21T22:00:19.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:00:19 smithi151 ceph-mon[31623]: pgmap v144: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 12 op/s 2024-05-21T22:00:19.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:00:19 smithi164 ceph-mon[36269]: pgmap v144: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 12 op/s 2024-05-21T22:00:19.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.891+0000 7f5df5a00700 1 -- 172.21.15.151:0/3798970954 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5df01095b0 msgr2=0x7f5df0109990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:19.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.891+0000 7f5df5a00700 1 --2- 172.21.15.151:0/3798970954 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5df01095b0 0x7f5df0109990 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f5de0009990 tx=0x7f5de0009ca0 comp rx=0 tx=0).stop 2024-05-21T22:00:19.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.892+0000 7f5df5a00700 1 -- 172.21.15.151:0/3798970954 shutdown_connections 2024-05-21T22:00:19.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.892+0000 7f5df5a00700 1 --2- 172.21.15.151:0/3798970954 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5df006bbe0 0x7f5df01103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:19.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.892+0000 7f5df5a00700 1 --2- 172.21.15.151:0/3798970954 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5df01095b0 0x7f5df0109990 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:19.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.892+0000 7f5df5a00700 1 --2- 172.21.15.151:0/3798970954 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5df006b220 0x7f5df006b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:19.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.892+0000 7f5df5a00700 1 -- 172.21.15.151:0/3798970954 >> 172.21.15.151:0/3798970954 conn(0x7f5df00793f0 msgr2=0x7f5df0079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:00:19.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.892+0000 7f5df5a00700 1 -- 172.21.15.151:0/3798970954 shutdown_connections 2024-05-21T22:00:19.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.892+0000 7f5df5a00700 1 -- 172.21.15.151:0/3798970954 wait complete. 2024-05-21T22:00:19.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.893+0000 7f5df5a00700 1 Processor -- start 2024-05-21T22:00:19.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.893+0000 7f5df5a00700 1 -- start start 2024-05-21T22:00:19.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.894+0000 7f5df5a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5df006b220 0x7f5df0102510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:19.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.894+0000 7f5df5a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5df006bbe0 0x7f5df0102a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:19.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.894+0000 7f5df5a00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5df01095b0 0x7f5df0102f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:19.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.894+0000 7f5df5a00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5df0113740 con 0x7f5df006bbe0 2024-05-21T22:00:19.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.894+0000 7f5df5a00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5df01135c0 con 0x7f5df006b220 2024-05-21T22:00:19.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.894+0000 7f5df5a00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5df01138c0 con 0x7f5df01095b0 2024-05-21T22:00:19.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.894+0000 7f5de7e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5df006bbe0 0x7f5df0102a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:19.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.894+0000 7f5deea00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5df006b220 0x7f5df0102510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:19.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.894+0000 7f5deea00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5df006b220 0x7f5df0102510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:33124/0 (socket says 172.21.15.151:33124) 2024-05-21T22:00:19.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.894+0000 7f5deea00700 1 -- 172.21.15.151:0/2815809301 learned_addr learned my addr 172.21.15.151:0/2815809301 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:00:19.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.894+0000 7f5def400700 1 --2- 172.21.15.151:0/2815809301 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5df01095b0 0x7f5df0102f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:19.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.895+0000 7f5deea00700 1 -- 172.21.15.151:0/2815809301 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5df01095b0 msgr2=0x7f5df0102f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:19.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.895+0000 7f5deea00700 1 --2- 172.21.15.151:0/2815809301 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5df01095b0 0x7f5df0102f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:19.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.895+0000 7f5deea00700 1 -- 172.21.15.151:0/2815809301 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5df006bbe0 msgr2=0x7f5df0102a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:19.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.895+0000 7f5deea00700 1 --2- 172.21.15.151:0/2815809301 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5df006bbe0 0x7f5df0102a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:19.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.895+0000 7f5deea00700 1 -- 172.21.15.151:0/2815809301 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5df01a8220 con 0x7f5df006b220 2024-05-21T22:00:19.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.895+0000 7f5deea00700 1 --2- 172.21.15.151:0/2815809301 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5df006b220 0x7f5df0102510 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f5de0005910 tx=0x7f5de000b670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:00:19.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.896+0000 7f5de7400700 1 -- 172.21.15.151:0/2815809301 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de0027070 con 0x7f5df006b220 2024-05-21T22:00:19.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.896+0000 7f5df5a00700 1 -- 172.21.15.151:0/2815809301 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5df01a84b0 con 0x7f5df006b220 2024-05-21T22:00:19.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.896+0000 7f5de7400700 1 -- 172.21.15.151:0/2815809301 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5de0022470 con 0x7f5df006b220 2024-05-21T22:00:19.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.896+0000 7f5de7400700 1 -- 172.21.15.151:0/2815809301 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de0017650 con 0x7f5df006b220 2024-05-21T22:00:19.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.896+0000 7f5df5a00700 1 -- 172.21.15.151:0/2815809301 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5df01a8960 con 0x7f5df006b220 2024-05-21T22:00:19.899 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.899+0000 7f5de7400700 1 -- 172.21.15.151:0/2815809301 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7f5de002a070 con 0x7f5df006b220 2024-05-21T22:00:19.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.900+0000 7f5e00421b80 1 Processor -- start 2024-05-21T22:00:19.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.901+0000 7f5e00421b80 1 -- start start 2024-05-21T22:00:19.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.901+0000 7f5de7400700 1 --2- 172.21.15.151:0/2815809301 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f5dd40b29f0 0x7f5dd40b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:19.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.901+0000 7f5e00421b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556d0cc46eb0 0x556d0cc47290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:19.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.901+0000 7f5e00421b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556d0cc477d0 0x556d0cc41260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:19.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.901+0000 7f5e00421b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556d0cc41880 0x556d0cc41ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:19.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.901+0000 7f5de7400700 1 -- 172.21.15.151:0/2815809301 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f5de00e3340 con 0x7f5df006b220 2024-05-21T22:00:19.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.901+0000 7f5e00421b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x556d0cb9fe40 con 0x556d0cc477d0 2024-05-21T22:00:19.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.901+0000 7f5e00421b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x556d0cb8f9e0 con 0x556d0cc46eb0 2024-05-21T22:00:19.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.901+0000 7f5e00421b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x556d0cb93a20 con 0x556d0cc41880 2024-05-21T22:00:19.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.901+0000 7f5def400700 1 --2- 172.21.15.151:0/2815809301 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f5dd40b29f0 0x7f5dd40b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:19.903 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.902+0000 7f5def400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556d0cc41880 0x556d0cc41ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:19.903 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.902+0000 7f5de7e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556d0cc477d0 0x556d0cc41260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:19.903 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.902+0000 7f5deea00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556d0cc46eb0 0x556d0cc47290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:19.903 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.902+0000 7f5def400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556d0cc41880 0x556d0cc41ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:33006/0 (socket says 172.21.15.151:33006) 2024-05-21T22:00:19.903 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.902+0000 7f5def400700 1 -- 172.21.15.151:0/2873775560 learned_addr learned my addr 172.21.15.151:0/2873775560 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:00:19.903 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.902+0000 7f5deea00700 1 -- 172.21.15.151:0/2873775560 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556d0cc41880 msgr2=0x556d0cc41ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:19.903 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.902+0000 7f5deea00700 1 --2- 172.21.15.151:0/2873775560 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556d0cc41880 0x556d0cc41ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:19.903 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.902+0000 7f5deea00700 1 -- 172.21.15.151:0/2873775560 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556d0cc477d0 msgr2=0x556d0cc41260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:19.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.902+0000 7f5deea00700 1 --2- 172.21.15.151:0/2873775560 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556d0cc477d0 0x556d0cc41260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:19.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.902+0000 7f5deea00700 1 -- 172.21.15.151:0/2873775560 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d0cb58420 con 0x556d0cc46eb0 2024-05-21T22:00:19.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.902+0000 7f5def400700 1 --2- 172.21.15.151:0/2815809301 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f5dd40b29f0 0x7f5dd40b4eb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5de800f940 tx=0x7f5de800fd00 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:00:19.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.902+0000 7f5deea00700 1 --2- 172.21.15.151:0/2873775560 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556d0cc46eb0 0x556d0cc47290 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f5de00e0a40 tx=0x7f5de002a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=437be703e908dd54 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:00:19.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.903+0000 7f5ddfe00700 1 -- 172.21.15.151:0/2873775560 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de00aa660 con 0x556d0cc46eb0 2024-05-21T22:00:19.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.903+0000 7f5ddfe00700 1 -- 172.21.15.151:0/2873775560 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5de00aea10 con 0x556d0cc46eb0 2024-05-21T22:00:19.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.903+0000 7f5ddfe00700 1 -- 172.21.15.151:0/2873775560 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de00aab20 con 0x556d0cc46eb0 2024-05-21T22:00:19.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.904+0000 7f5e00421b80 1 -- 172.21.15.151:0/2873775560 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556d0cc46eb0 msgr2=0x556d0cc47290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:19.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.904+0000 7f5e00421b80 1 --2- 172.21.15.151:0/2873775560 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556d0cc46eb0 0x556d0cc47290 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f5de00e0a40 tx=0x7f5de002a000 comp rx=0 tx=0).stop 2024-05-21T22:00:19.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.904+0000 7f5e00421b80 1 -- 172.21.15.151:0/2873775560 shutdown_connections 2024-05-21T22:00:19.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.904+0000 7f5e00421b80 1 --2- 172.21.15.151:0/2873775560 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556d0cc46eb0 0x556d0cc47290 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:19.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.904+0000 7f5e00421b80 1 --2- 172.21.15.151:0/2873775560 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556d0cc41880 0x556d0cc41ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:19.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.904+0000 7f5e00421b80 1 --2- 172.21.15.151:0/2873775560 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556d0cc477d0 0x556d0cc41260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:19.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.904+0000 7f5e00421b80 1 -- 172.21.15.151:0/2873775560 >> 172.21.15.151:0/2873775560 conn(0x556d0cb6fbe0 msgr2=0x556d0cc3c010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:00:19.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.904+0000 7f5e00421b80 1 -- 172.21.15.151:0/2873775560 shutdown_connections 2024-05-21T22:00:19.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.904+0000 7f5e00421b80 1 -- 172.21.15.151:0/2873775560 wait complete. 2024-05-21T22:00:19.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.904+0000 7f5e00421b80 1 -- start start 2024-05-21T22:00:19.906 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.905+0000 7f5e00421b80 1 Processor -- start 2024-05-21T22:00:19.907 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.905+0000 7f5e00421b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556d0cc41880 0x556d0cc4d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:19.907 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.905+0000 7f5e00421b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556d0cc46eb0 0x556d0cc4a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:19.907 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.905+0000 7f5e00421b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556d0cc477d0 0x556d0cc4ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:19.907 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.905+0000 7f5e00421b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x556d0ca95fa0 con 0x556d0cc41880 2024-05-21T22:00:19.907 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.905+0000 7f5e00421b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x556d0cb30760 con 0x556d0cc46eb0 2024-05-21T22:00:19.907 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.905+0000 7f5e00421b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x556d0cc4e000 con 0x556d0cc477d0 2024-05-21T22:00:19.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.905+0000 7f5e00421b80 10 client.0 _ll_register_callbacks cb 0x556d0cb39c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:00:19.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.906+0000 7f5deea00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556d0cc41880 0x556d0cc4d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:19.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.906+0000 7f5deea00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556d0cc41880 0x556d0cc4d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:59900/0 (socket says 172.21.15.151:59900) 2024-05-21T22:00:19.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.906+0000 7f5deea00700 1 -- 172.21.15.151:0/3427238332 learned_addr learned my addr 172.21.15.151:0/3427238332 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:00:19.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.906+0000 7f5de7e00700 1 --2- 172.21.15.151:0/3427238332 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556d0cc46eb0 0x556d0cc4a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:19.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.906+0000 7f5def400700 1 --2- 172.21.15.151:0/3427238332 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556d0cc477d0 0x556d0cc4ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:19.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.906+0000 7f5deea00700 1 -- 172.21.15.151:0/3427238332 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556d0cc477d0 msgr2=0x556d0cc4ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:19.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.906+0000 7f5deea00700 1 --2- 172.21.15.151:0/3427238332 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556d0cc477d0 0x556d0cc4ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:19.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.906+0000 7f5deea00700 1 -- 172.21.15.151:0/3427238332 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556d0cc46eb0 msgr2=0x556d0cc4a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:19.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.906+0000 7f5deea00700 1 --2- 172.21.15.151:0/3427238332 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556d0cc46eb0 0x556d0cc4a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:19.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.906+0000 7f5deea00700 1 -- 172.21.15.151:0/3427238332 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d0cadf8d0 con 0x556d0cc41880 2024-05-21T22:00:19.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.906+0000 7f5de7e00700 1 --2- 172.21.15.151:0/3427238332 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556d0cc46eb0 0x556d0cc4a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:00:19.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.906+0000 7f5deea00700 1 --2- 172.21.15.151:0/3427238332 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556d0cc41880 0x556d0cc4d4d0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f5de0005cd0 tx=0x7f5de00a6650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce6ee2f5ea34e159 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:00:19.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.907+0000 7f5ddcc00700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-21T22:00:19.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.907+0000 7f5ddcc00700 1 -- 172.21.15.151:0/3427238332 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de00b3430 con 0x556d0cc41880 2024-05-21T22:00:19.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.907+0000 7f5ddcc00700 1 -- 172.21.15.151:0/3427238332 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f5de00aaad0 con 0x556d0cc41880 2024-05-21T22:00:19.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.907+0000 7f5e00421b80 1 -- 172.21.15.151:0/3427238332 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5de00aea10 con 0x556d0cc41880 2024-05-21T22:00:19.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.907+0000 7f5ddcc00700 1 -- 172.21.15.151:0/3427238332 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5de00aa660 con 0x556d0cc41880 2024-05-21T22:00:19.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.907+0000 7f5e00421b80 10 client.16167 fetch_fsmap learned FSMap version 29 2024-05-21T22:00:19.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.907+0000 7f5e00421b80 1 -- 172.21.15.151:0/3427238332 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556d0cb60060 con 0x556d0cc41880 2024-05-21T22:00:19.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.907+0000 7f5ddcc00700 1 -- 172.21.15.151:0/3427238332 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5de00aea10 con 0x556d0cc41880 2024-05-21T22:00:19.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.908+0000 7f5ddcc00700 1 -- 172.21.15.151:0/3427238332 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f5de0090b80 con 0x556d0cc41880 2024-05-21T22:00:19.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.908+0000 7f5e00421b80 10 client.16167 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:00:19.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.908+0000 7f5e00421b80 10 client.16167 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-05-21T22:00:19.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.908+0000 7f5e00421b80 20 client.16167 populate_metadata read hostname 'smithi151' 2024-05-21T22:00:19.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.908+0000 7f5e00421b80 1 --2- 172.21.15.151:0/3427238332 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x556d0cc5a8e0 0x556d0cc5ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:19.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.908+0000 7f5e00421b80 4 client.16167 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T22:00:19.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.908+0000 7f5e00421b80 1 -- 172.21.15.151:0/3427238332 --> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556d0cba1d90 con 0x556d0cc5a8e0 2024-05-21T22:00:19.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.908+0000 7f5de7e00700 1 --2- 172.21.15.151:0/3427238332 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x556d0cc5a8e0 0x556d0cc5ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:19.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.909+0000 7f5de7e00700 1 --2- 172.21.15.151:0/3427238332 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x556d0cc5a8e0 0x556d0cc5ccd0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=55fcd029b99b08f1 server_cookie=29581d1c0a5ff5c in_seq=0 out_seq=0 2024-05-21T22:00:19.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.909+0000 7f5ddcc00700 10 client.16167 ms_handle_connect on v2:172.21.15.164:6836/1554609572 2024-05-21T22:00:19.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.910+0000 7f5ddcc00700 1 -- 172.21.15.151:0/3427238332 <== mds.2 v2:172.21.15.164:6836/1554609572 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x556d0cba1d90 con 0x556d0cc5a8e0 2024-05-21T22:00:19.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.910+0000 7f5ddcc00700 10 client.16167 handle_command_reply: tid=0 2024-05-21T22:00:19.912 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.910+0000 7f5e00421b80 1 client.16167 shutdown 2024-05-21T22:00:19.912 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.910+0000 7f5e00421b80 1 -- 172.21.15.151:0/3427238332 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556d0cc41880 msgr2=0x556d0cc4d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:19.912 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.910+0000 7f5e00421b80 1 --2- 172.21.15.151:0/3427238332 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556d0cc41880 0x556d0cc4d4d0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f5de0005cd0 tx=0x7f5de00a6650 comp rx=0 tx=0).stop 2024-05-21T22:00:19.912 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.910+0000 7f5e00421b80 1 -- 172.21.15.151:0/3427238332 shutdown_connections 2024-05-21T22:00:19.912 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.910+0000 7f5e00421b80 1 --2- 172.21.15.151:0/3427238332 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x556d0cc5a8e0 0x556d0cc5ccd0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:19.912 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.910+0000 7f5e00421b80 1 --2- 172.21.15.151:0/3427238332 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556d0cc477d0 0x556d0cc4ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:19.912 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.910+0000 7f5e00421b80 1 --2- 172.21.15.151:0/3427238332 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556d0cc41880 0x556d0cc4d4d0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:19.912 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.910+0000 7f5ddcc00700 0 client.16167 ms_handle_reset on v2:172.21.15.164:6836/1554609572 2024-05-21T22:00:19.912 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.910+0000 7f5e00421b80 1 --2- 172.21.15.151:0/3427238332 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556d0cc46eb0 0x556d0cc4a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:19.912 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.910+0000 7f5e00421b80 1 -- 172.21.15.151:0/3427238332 >> 172.21.15.151:0/3427238332 conn(0x556d0cb6fbe0 msgr2=0x556d0cc40250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:00:19.912 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.910+0000 7f5e00421b80 1 -- 172.21.15.151:0/3427238332 shutdown_connections 2024-05-21T22:00:19.912 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.910+0000 7f5e00421b80 1 -- 172.21.15.151:0/3427238332 wait complete. 2024-05-21T22:00:19.912 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.910+0000 7f5e00421b80 20 client.16167 trim_cache size 0 max 16384 2024-05-21T22:00:19.927 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.927+0000 7f5e00421b80 1 Processor -- start 2024-05-21T22:00:19.927 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.927+0000 7f5e00421b80 1 -- start start 2024-05-21T22:00:19.928 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.927+0000 7f5e00421b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556d0cc41880 0x556d0cc41260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:19.928 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.927+0000 7f5e00421b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556d0cc46eb0 0x556d0cc41c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:19.928 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.927+0000 7f5e00421b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556d0cc477d0 0x556d0cc624e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:19.928 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.927+0000 7f5e00421b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5df01afea0 con 0x556d0cc46eb0 2024-05-21T22:00:19.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.927+0000 7f5e00421b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5df01b19e0 con 0x556d0cc41880 2024-05-21T22:00:19.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.927+0000 7f5e00421b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5df01b0570 con 0x556d0cc477d0 2024-05-21T22:00:19.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.928+0000 7f5deea00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556d0cc41880 0x556d0cc41260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:19.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.928+0000 7f5de7e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556d0cc46eb0 0x556d0cc41c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:19.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.928+0000 7f5deea00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556d0cc41880 0x556d0cc41260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:33152/0 (socket says 172.21.15.151:33152) 2024-05-21T22:00:19.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.928+0000 7f5deea00700 1 -- 172.21.15.151:0/2196369814 learned_addr learned my addr 172.21.15.151:0/2196369814 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:00:19.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.928+0000 7f5def400700 1 --2- 172.21.15.151:0/2196369814 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556d0cc477d0 0x556d0cc624e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:19.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.928+0000 7f5de7e00700 1 -- 172.21.15.151:0/2196369814 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556d0cc477d0 msgr2=0x556d0cc624e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:19.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.928+0000 7f5de7e00700 1 --2- 172.21.15.151:0/2196369814 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556d0cc477d0 0x556d0cc624e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:19.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.928+0000 7f5de7e00700 1 -- 172.21.15.151:0/2196369814 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556d0cc41880 msgr2=0x556d0cc41260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:19.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.928+0000 7f5de7e00700 1 --2- 172.21.15.151:0/2196369814 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556d0cc41880 0x556d0cc41260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:19.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.928+0000 7f5de7e00700 1 -- 172.21.15.151:0/2196369814 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d0cc48e30 con 0x556d0cc46eb0 2024-05-21T22:00:19.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.928+0000 7f5def400700 1 --2- 172.21.15.151:0/2196369814 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556d0cc477d0 0x556d0cc624e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T22:00:19.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.928+0000 7f5deea00700 1 --2- 172.21.15.151:0/2196369814 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556d0cc41880 0x556d0cc41260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:00:19.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.929+0000 7f5de7e00700 1 --2- 172.21.15.151:0/2196369814 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556d0cc46eb0 0x556d0cc41c60 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x556d0cc34280 tx=0x7f5dd8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aa402b8e0cf3a85 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:00:19.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.929+0000 7f5ddf400700 1 -- 172.21.15.151:0/2196369814 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd8004250 con 0x556d0cc46eb0 2024-05-21T22:00:19.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.929+0000 7f5ddf400700 1 -- 172.21.15.151:0/2196369814 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5dd8012310 con 0x556d0cc46eb0 2024-05-21T22:00:19.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.929+0000 7f5ddf400700 1 -- 172.21.15.151:0/2196369814 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd8005520 con 0x556d0cc46eb0 2024-05-21T22:00:19.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.929+0000 7f5e00421b80 1 -- 172.21.15.151:0/2196369814 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556d0cc46eb0 msgr2=0x556d0cc41c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:19.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.929+0000 7f5e00421b80 1 --2- 172.21.15.151:0/2196369814 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556d0cc46eb0 0x556d0cc41c60 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x556d0cc34280 tx=0x7f5dd8009260 comp rx=0 tx=0).stop 2024-05-21T22:00:19.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.930+0000 7f5e00421b80 1 -- 172.21.15.151:0/2196369814 shutdown_connections 2024-05-21T22:00:19.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.930+0000 7f5e00421b80 1 --2- 172.21.15.151:0/2196369814 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556d0cc41880 0x556d0cc41260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:19.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.930+0000 7f5e00421b80 1 --2- 172.21.15.151:0/2196369814 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556d0cc477d0 0x556d0cc624e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:19.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.930+0000 7f5e00421b80 1 --2- 172.21.15.151:0/2196369814 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556d0cc46eb0 0x556d0cc41c60 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:19.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.930+0000 7f5e00421b80 1 -- 172.21.15.151:0/2196369814 >> 172.21.15.151:0/2196369814 conn(0x556d0cb6fbe0 msgr2=0x556d0cc449c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:00:19.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.930+0000 7f5e00421b80 1 -- 172.21.15.151:0/2196369814 shutdown_connections 2024-05-21T22:00:19.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.930+0000 7f5e00421b80 1 -- 172.21.15.151:0/2196369814 wait complete. 2024-05-21T22:00:19.932 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.930+0000 7f5e00421b80 1 -- start start 2024-05-21T22:00:19.932 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.930+0000 7f5e00421b80 1 Processor -- start 2024-05-21T22:00:19.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.931+0000 7f5e00421b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556d0cc41880 0x556d0cc534a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:19.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.931+0000 7f5e00421b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556d0cc46eb0 0x556d0cc539e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:19.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.931+0000 7f5e00421b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556d0cc477d0 0x556d0cc618f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:19.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.931+0000 7f5e00421b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5df01b1310 con 0x556d0cc46eb0 2024-05-21T22:00:19.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.931+0000 7f5e00421b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5df01b0c40 con 0x556d0cc477d0 2024-05-21T22:00:19.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.931+0000 7f5e00421b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5df01ade40 con 0x556d0cc41880 2024-05-21T22:00:19.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.931+0000 7f5e00421b80 10 client.0 _ll_register_callbacks cb 0x556d0cb39c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:00:19.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.931+0000 7f5def400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556d0cc477d0 0x556d0cc618f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:19.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.931+0000 7f5deea00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556d0cc41880 0x556d0cc534a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:19.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.931+0000 7f5def400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556d0cc477d0 0x556d0cc618f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:33168/0 (socket says 172.21.15.151:33168) 2024-05-21T22:00:19.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.931+0000 7f5def400700 1 -- 172.21.15.151:0/4079470192 learned_addr learned my addr 172.21.15.151:0/4079470192 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:00:19.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.931+0000 7f5de7e00700 1 --2- 172.21.15.151:0/4079470192 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556d0cc46eb0 0x556d0cc539e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:19.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.932+0000 7f5deea00700 1 -- 172.21.15.151:0/4079470192 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556d0cc477d0 msgr2=0x556d0cc618f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:19.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.932+0000 7f5deea00700 1 --2- 172.21.15.151:0/4079470192 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556d0cc477d0 0x556d0cc618f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:19.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.932+0000 7f5deea00700 1 -- 172.21.15.151:0/4079470192 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556d0cc46eb0 msgr2=0x556d0cc539e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:19.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.932+0000 7f5deea00700 1 --2- 172.21.15.151:0/4079470192 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556d0cc46eb0 0x556d0cc539e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:19.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.932+0000 7f5deea00700 1 -- 172.21.15.151:0/4079470192 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d0cc62180 con 0x556d0cc41880 2024-05-21T22:00:19.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.932+0000 7f5deea00700 1 --2- 172.21.15.151:0/4079470192 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556d0cc41880 0x556d0cc534a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5de00a78d0 tx=0x7f5de0009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c458b10e2440b1f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:00:19.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.932+0000 7f5ddcc00700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-21T22:00:19.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.932+0000 7f5ddcc00700 1 -- 172.21.15.151:0/4079470192 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de00aea10 con 0x556d0cc41880 2024-05-21T22:00:19.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.932+0000 7f5ddcc00700 1 -- 172.21.15.151:0/4079470192 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f5de00a61f0 con 0x556d0cc41880 2024-05-21T22:00:19.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.932+0000 7f5e00421b80 1 -- 172.21.15.151:0/4079470192 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556d0cb9b570 con 0x556d0cc41880 2024-05-21T22:00:19.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.933+0000 7f5ddcc00700 1 -- 172.21.15.151:0/4079470192 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5de00b3430 con 0x556d0cc41880 2024-05-21T22:00:19.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.933+0000 7f5e00421b80 10 client.26308 fetch_fsmap learned FSMap version 29 2024-05-21T22:00:19.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.933+0000 7f5e00421b80 1 -- 172.21.15.151:0/4079470192 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556d0cc62180 con 0x556d0cc41880 2024-05-21T22:00:19.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.933+0000 7f5ddcc00700 1 -- 172.21.15.151:0/4079470192 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x556d0cb9b570 con 0x556d0cc41880 2024-05-21T22:00:19.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.933+0000 7f5ddcc00700 1 -- 172.21.15.151:0/4079470192 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f5de00afba0 con 0x556d0cc41880 2024-05-21T22:00:19.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.933+0000 7f5e00421b80 10 client.26308 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:00:19.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.933+0000 7f5e00421b80 10 client.26308 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-05-21T22:00:19.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.933+0000 7f5e00421b80 20 client.26308 populate_metadata read hostname 'smithi151' 2024-05-21T22:00:19.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.933+0000 7f5e00421b80 1 --2- 172.21.15.151:0/4079470192 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x556d0ccc5200 0x556d0ccc75f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:19.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.933+0000 7f5e00421b80 4 client.26308 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T22:00:19.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.933+0000 7f5e00421b80 1 -- 172.21.15.151:0/4079470192 --> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556d0ca12250 con 0x556d0ccc5200 2024-05-21T22:00:19.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.934+0000 7f5de7e00700 1 --2- 172.21.15.151:0/4079470192 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x556d0ccc5200 0x556d0ccc75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:19.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.934+0000 7f5de7e00700 1 --2- 172.21.15.151:0/4079470192 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x556d0ccc5200 0x556d0ccc75f0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a7b53f5f39d4a7ad server_cookie=524b7888e8804d9d in_seq=0 out_seq=0 2024-05-21T22:00:19.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.934+0000 7f5ddcc00700 10 client.26308 ms_handle_connect on v2:172.21.15.164:6836/1554609572 2024-05-21T22:00:19.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.934+0000 7f5ddcc00700 1 -- 172.21.15.151:0/4079470192 <== mds.2 v2:172.21.15.164:6836/1554609572 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556d0ca12250 con 0x556d0ccc5200 2024-05-21T22:00:19.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.934+0000 7f5ddcc00700 10 client.26308 handle_command_reply: tid=0 2024-05-21T22:00:19.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.934+0000 7f5e00421b80 1 client.26308 shutdown 2024-05-21T22:00:19.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.935+0000 7f5e00421b80 1 -- 172.21.15.151:0/4079470192 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556d0cc41880 msgr2=0x556d0cc534a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:19.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.935+0000 7f5e00421b80 1 --2- 172.21.15.151:0/4079470192 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556d0cc41880 0x556d0cc534a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5de00a78d0 tx=0x7f5de0009ca0 comp rx=0 tx=0).stop 2024-05-21T22:00:19.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.935+0000 7f5e00421b80 1 -- 172.21.15.151:0/4079470192 shutdown_connections 2024-05-21T22:00:19.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.935+0000 7f5e00421b80 1 --2- 172.21.15.151:0/4079470192 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x556d0ccc5200 0x556d0ccc75f0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:19.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.935+0000 7f5e00421b80 1 --2- 172.21.15.151:0/4079470192 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556d0cc477d0 0x556d0cc618f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:19.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.935+0000 7f5e00421b80 1 --2- 172.21.15.151:0/4079470192 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556d0cc41880 0x556d0cc534a0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:19.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.935+0000 7f5e00421b80 1 --2- 172.21.15.151:0/4079470192 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556d0cc46eb0 0x556d0cc539e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:19.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.935+0000 7f5e00421b80 1 -- 172.21.15.151:0/4079470192 >> 172.21.15.151:0/4079470192 conn(0x556d0cb6fbe0 msgr2=0x556d0cc491b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:00:19.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.935+0000 7f5ddcc00700 0 client.26308 ms_handle_reset on v2:172.21.15.164:6836/1554609572 2024-05-21T22:00:19.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.935+0000 7f5e00421b80 1 -- 172.21.15.151:0/4079470192 shutdown_connections 2024-05-21T22:00:19.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.935+0000 7f5e00421b80 1 -- 172.21.15.151:0/4079470192 wait complete. 2024-05-21T22:00:19.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.935+0000 7f5e00421b80 20 client.26308 trim_cache size 0 max 16384 2024-05-21T22:00:19.938 INFO:teuthology.orchestra.run.smithi151.stdout:[] 2024-05-21T22:00:19.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.936+0000 7f5df5a00700 1 -- 172.21.15.151:0/2815809301 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f5dd40b29f0 msgr2=0x7f5dd40b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:19.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.936+0000 7f5df5a00700 1 --2- 172.21.15.151:0/2815809301 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f5dd40b29f0 0x7f5dd40b4eb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5de800f940 tx=0x7f5de800fd00 comp rx=0 tx=0).stop 2024-05-21T22:00:19.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.936+0000 7f5df5a00700 1 -- 172.21.15.151:0/2815809301 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5df006b220 msgr2=0x7f5df0102510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:19.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.936+0000 7f5df5a00700 1 --2- 172.21.15.151:0/2815809301 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5df006b220 0x7f5df0102510 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f5de0005910 tx=0x7f5de000b670 comp rx=0 tx=0).stop 2024-05-21T22:00:19.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.936+0000 7f5df5a00700 1 -- 172.21.15.151:0/2815809301 shutdown_connections 2024-05-21T22:00:19.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.936+0000 7f5df5a00700 1 --2- 172.21.15.151:0/2815809301 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f5dd40b29f0 0x7f5dd40b4eb0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:19.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.936+0000 7f5df5a00700 1 --2- 172.21.15.151:0/2815809301 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5df006b220 0x7f5df0102510 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:19.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.936+0000 7f5df5a00700 1 --2- 172.21.15.151:0/2815809301 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5df01095b0 0x7f5df0102f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:19.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.936+0000 7f5df5a00700 1 --2- 172.21.15.151:0/2815809301 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5df006bbe0 0x7f5df0102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:19.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.936+0000 7f5df5a00700 1 -- 172.21.15.151:0/2815809301 >> 172.21.15.151:0/2815809301 conn(0x7f5df00793f0 msgr2=0x7f5df0101240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:00:19.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.936+0000 7f5df5a00700 1 -- 172.21.15.151:0/2815809301 shutdown_connections 2024-05-21T22:00:19.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:19.936+0000 7f5df5a00700 1 -- 172.21.15.151:0/2815809301 wait complete. 2024-05-21T22:00:19.985 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:00:19 smithi184 ceph-mon[36942]: pgmap v144: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 12 op/s 2024-05-21T22:00:20.859 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-21T22:00:21.314 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:00:21.772 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:00:21 smithi151 ceph-mon[31623]: pgmap v145: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-05-21T22:00:21.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:00:21 smithi164 ceph-mon[36269]: pgmap v145: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-05-21T22:00:21.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:00:21 smithi184 ceph-mon[36942]: pgmap v145: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-05-21T22:00:23.401 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.400+0000 7f0fb1600700 1 -- 172.21.15.151:0/2444765384 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0fac1095a0 msgr2=0x7f0fac109980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:23.401 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.400+0000 7f0fb1600700 1 --2- 172.21.15.151:0/2444765384 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0fac1095a0 0x7f0fac109980 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c0098f0 tx=0x7f0f9c009c00 comp rx=0 tx=0).stop 2024-05-21T22:00:23.401 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.400+0000 7f0fb1600700 1 -- 172.21.15.151:0/2444765384 shutdown_connections 2024-05-21T22:00:23.401 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.400+0000 7f0fb1600700 1 --2- 172.21.15.151:0/2444765384 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0fac06c470 0x7f0fac1103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:23.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.400+0000 7f0fb1600700 1 --2- 172.21.15.151:0/2444765384 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0fac1095a0 0x7f0fac109980 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:23.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.400+0000 7f0fb1600700 1 --2- 172.21.15.151:0/2444765384 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0fac06bab0 0x7f0fac06bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:23.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.400+0000 7f0fb1600700 1 -- 172.21.15.151:0/2444765384 >> 172.21.15.151:0/2444765384 conn(0x7f0fac0793b0 msgr2=0x7f0fac0797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:00:23.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.401+0000 7f0fb1600700 1 -- 172.21.15.151:0/2444765384 shutdown_connections 2024-05-21T22:00:23.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.401+0000 7f0fb1600700 1 -- 172.21.15.151:0/2444765384 wait complete. 2024-05-21T22:00:23.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.402+0000 7f0fb1600700 1 Processor -- start 2024-05-21T22:00:23.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.402+0000 7f0fb1600700 1 -- start start 2024-05-21T22:00:23.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.402+0000 7f0fb1600700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0fac06bab0 0x7f0fac19bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:23.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.402+0000 7f0fb1600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0fac06c470 0x7f0fac19c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:23.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.402+0000 7f0fb1600700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0fac1095a0 0x7f0fac1a0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:23.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.402+0000 7f0fb1600700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0fac113720 con 0x7f0fac06c470 2024-05-21T22:00:23.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.402+0000 7f0fb1600700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0fac1135a0 con 0x7f0fac06bab0 2024-05-21T22:00:23.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.402+0000 7f0fb1600700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0fac1138a0 con 0x7f0fac1095a0 2024-05-21T22:00:23.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.403+0000 7f0faaa00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0fac06bab0 0x7f0fac19bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:23.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.403+0000 7f0faaa00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0fac06bab0 0x7f0fac19bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:33176/0 (socket says 172.21.15.151:33176) 2024-05-21T22:00:23.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.403+0000 7f0faaa00700 1 -- 172.21.15.151:0/718342511 learned_addr learned my addr 172.21.15.151:0/718342511 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:00:23.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.403+0000 7f0faa000700 1 --2- 172.21.15.151:0/718342511 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0fac06c470 0x7f0fac19c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:23.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.403+0000 7f0fab400700 1 --2- 172.21.15.151:0/718342511 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0fac1095a0 0x7f0fac1a0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:23.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.403+0000 7f0faaa00700 1 -- 172.21.15.151:0/718342511 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0fac1095a0 msgr2=0x7f0fac1a0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:23.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.403+0000 7f0faaa00700 1 --2- 172.21.15.151:0/718342511 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0fac1095a0 0x7f0fac1a0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:23.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.403+0000 7f0faaa00700 1 -- 172.21.15.151:0/718342511 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0fac06c470 msgr2=0x7f0fac19c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:23.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.403+0000 7f0faaa00700 1 --2- 172.21.15.151:0/718342511 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0fac06c470 0x7f0fac19c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:23.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.403+0000 7f0faaa00700 1 -- 172.21.15.151:0/718342511 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fac1a0e90 con 0x7f0fac06bab0 2024-05-21T22:00:23.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.403+0000 7f0faa000700 1 --2- 172.21.15.151:0/718342511 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0fac06c470 0x7f0fac19c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T22:00:23.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.404+0000 7f0faaa00700 1 --2- 172.21.15.151:0/718342511 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0fac06bab0 0x7f0fac19bf70 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c009230 tx=0x7f0f9c00fa70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:00:23.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.404+0000 7f0fa3400700 1 -- 172.21.15.151:0/718342511 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f9c005140 con 0x7f0fac06bab0 2024-05-21T22:00:23.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.404+0000 7f0fb1600700 1 -- 172.21.15.151:0/718342511 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fac1a1120 con 0x7f0fac06bab0 2024-05-21T22:00:23.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.404+0000 7f0fa3400700 1 -- 172.21.15.151:0/718342511 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f9c00b560 con 0x7f0fac06bab0 2024-05-21T22:00:23.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.404+0000 7f0fa3400700 1 -- 172.21.15.151:0/718342511 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f9c003ae0 con 0x7f0fac06bab0 2024-05-21T22:00:23.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.404+0000 7f0fb1600700 1 -- 172.21.15.151:0/718342511 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fac1a8a00 con 0x7f0fac06bab0 2024-05-21T22:00:23.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.408+0000 7f0fa3400700 1 -- 172.21.15.151:0/718342511 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7f0f9c003c80 con 0x7f0fac06bab0 2024-05-21T22:00:23.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.409+0000 7f0fbbf75b80 1 Processor -- start 2024-05-21T22:00:23.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.409+0000 7f0fbbf75b80 1 -- start start 2024-05-21T22:00:23.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.409+0000 7f0fbbf75b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55affa17b610 0x55affa17b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:23.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.409+0000 7f0fbbf75b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55affa17bf30 0x55affa175b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:23.410 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.409+0000 7f0fbbf75b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55affa176140 0x55affa1765c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:23.410 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.409+0000 7f0fbbf75b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55affa0d3e40 con 0x55affa176140 2024-05-21T22:00:23.410 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.409+0000 7f0fbbf75b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55affa0c39e0 con 0x55affa17bf30 2024-05-21T22:00:23.410 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.409+0000 7f0fbbf75b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55affa0c7a20 con 0x55affa17b610 2024-05-21T22:00:23.410 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.409+0000 7f0fa3400700 1 --2- 172.21.15.151:0/718342511 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f0f900b2a70 0x7f0f900b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:23.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.410+0000 7f0fa3400700 1 -- 172.21.15.151:0/718342511 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f0f9c0e35a0 con 0x7f0fac06bab0 2024-05-21T22:00:23.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.410+0000 7f0faa000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55affa176140 0x55affa1765c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:23.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.410+0000 7f0fab400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55affa17b610 0x55affa17b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:23.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.410+0000 7f0faaa00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55affa17bf30 0x55affa175b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:23.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.410+0000 7f0faa000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55affa176140 0x55affa1765c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:59948/0 (socket says 172.21.15.151:59948) 2024-05-21T22:00:23.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.410+0000 7f0faa000700 1 -- 172.21.15.151:0/2856965871 learned_addr learned my addr 172.21.15.151:0/2856965871 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:00:23.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.410+0000 7f0fab400700 1 --2- 172.21.15.151:0/718342511 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f0f900b2a70 0x7f0f900b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:23.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.410+0000 7f0faaa00700 1 -- 172.21.15.151:0/2856965871 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55affa17b610 msgr2=0x55affa17b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:23.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.410+0000 7f0faaa00700 1 --2- 172.21.15.151:0/2856965871 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55affa17b610 0x55affa17b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:23.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.410+0000 7f0faaa00700 1 -- 172.21.15.151:0/2856965871 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55affa176140 msgr2=0x55affa1765c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:23.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.410+0000 7f0faaa00700 1 --2- 172.21.15.151:0/2856965871 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55affa176140 0x55affa1765c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:23.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.410+0000 7f0faaa00700 1 -- 172.21.15.151:0/2856965871 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55affa08c420 con 0x55affa17bf30 2024-05-21T22:00:23.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.410+0000 7f0fab400700 1 --2- 172.21.15.151:0/2856965871 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55affa17b610 0x55affa17b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T22:00:23.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.411+0000 7f0faaa00700 1 --2- 172.21.15.151:0/2856965871 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55affa17bf30 0x55affa175b20 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c0e0ca0 tx=0x7f0f9c0aa710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d010cc9ebd93a73a server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:00:23.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.411+0000 7f0fab400700 1 --2- 172.21.15.151:0/718342511 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f0f900b2a70 0x7f0f900b4f30 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0fa4009550 tx=0x7f0fa400b040 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:00:23.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.411+0000 7f0f97e00700 1 -- 172.21.15.151:0/2856965871 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f9c0aa3d0 con 0x55affa17bf30 2024-05-21T22:00:23.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.411+0000 7f0f97e00700 1 -- 172.21.15.151:0/2856965871 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f9c0aafc0 con 0x55affa17bf30 2024-05-21T22:00:23.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.411+0000 7f0f97e00700 1 -- 172.21.15.151:0/2856965871 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f9c0b10b0 con 0x55affa17bf30 2024-05-21T22:00:23.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.412+0000 7f0fbbf75b80 1 -- 172.21.15.151:0/2856965871 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55affa17bf30 msgr2=0x55affa175b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:23.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.412+0000 7f0fbbf75b80 1 --2- 172.21.15.151:0/2856965871 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55affa17bf30 0x55affa175b20 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c0e0ca0 tx=0x7f0f9c0aa710 comp rx=0 tx=0).stop 2024-05-21T22:00:23.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.412+0000 7f0fbbf75b80 1 -- 172.21.15.151:0/2856965871 shutdown_connections 2024-05-21T22:00:23.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.412+0000 7f0fbbf75b80 1 --2- 172.21.15.151:0/2856965871 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55affa176140 0x55affa1765c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:23.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.412+0000 7f0fbbf75b80 1 --2- 172.21.15.151:0/2856965871 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55affa17b610 0x55affa17b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:23.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.412+0000 7f0fbbf75b80 1 --2- 172.21.15.151:0/2856965871 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55affa17bf30 0x55affa175b20 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:23.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.412+0000 7f0fbbf75b80 1 -- 172.21.15.151:0/2856965871 >> 172.21.15.151:0/2856965871 conn(0x55affa0a3be0 msgr2=0x55affa170690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:00:23.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.412+0000 7f0fbbf75b80 1 -- 172.21.15.151:0/2856965871 shutdown_connections 2024-05-21T22:00:23.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.412+0000 7f0fbbf75b80 1 -- 172.21.15.151:0/2856965871 wait complete. 2024-05-21T22:00:23.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.413+0000 7f0fbbf75b80 1 -- start start 2024-05-21T22:00:23.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.413+0000 7f0fbbf75b80 1 Processor -- start 2024-05-21T22:00:23.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.413+0000 7f0fbbf75b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55affa176140 0x55affa181c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:23.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.413+0000 7f0fbbf75b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55affa17b610 0x55affa17efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:23.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.413+0000 7f0fbbf75b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55affa17bf30 0x55affa17f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:23.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.413+0000 7f0fbbf75b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55aff9fc9fa0 con 0x55affa17b610 2024-05-21T22:00:23.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.413+0000 7f0fbbf75b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55affa064760 con 0x55affa17bf30 2024-05-21T22:00:23.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.413+0000 7f0fbbf75b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55affa1827a0 con 0x55affa176140 2024-05-21T22:00:23.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.413+0000 7f0fbbf75b80 10 client.0 _ll_register_callbacks cb 0x55affa06dc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:00:23.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.414+0000 7f0faa000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55affa17b610 0x55affa17efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:23.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.414+0000 7f0faaa00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55affa176140 0x55affa181c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:23.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.414+0000 7f0faaa00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55affa176140 0x55affa181c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:33068/0 (socket says 172.21.15.151:33068) 2024-05-21T22:00:23.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.414+0000 7f0faaa00700 1 -- 172.21.15.151:0/4130763843 learned_addr learned my addr 172.21.15.151:0/4130763843 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:00:23.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.414+0000 7f0fab400700 1 --2- 172.21.15.151:0/4130763843 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55affa17bf30 0x55affa17f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:23.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.414+0000 7f0faaa00700 1 -- 172.21.15.151:0/4130763843 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55affa17bf30 msgr2=0x55affa17f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:23.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.414+0000 7f0faaa00700 1 --2- 172.21.15.151:0/4130763843 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55affa17bf30 0x55affa17f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:23.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.414+0000 7f0faaa00700 1 -- 172.21.15.151:0/4130763843 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55affa17b610 msgr2=0x55affa17efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:23.417 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.414+0000 7f0faaa00700 1 --2- 172.21.15.151:0/4130763843 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55affa17b610 0x55affa17efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:23.417 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.414+0000 7f0faaa00700 1 -- 172.21.15.151:0/4130763843 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55affa0138d0 con 0x55affa176140 2024-05-21T22:00:23.417 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.414+0000 7f0faaa00700 1 --2- 172.21.15.151:0/4130763843 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55affa176140 0x55affa181c20 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c005950 tx=0x7f0f9c0a6de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6c6ebe42039ab2c server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:00:23.417 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.414+0000 7f0f94c00700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-21T22:00:23.417 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.415+0000 7f0f94c00700 1 -- 172.21.15.151:0/4130763843 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f9c0aa3d0 con 0x55affa176140 2024-05-21T22:00:23.417 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.415+0000 7f0f94c00700 1 -- 172.21.15.151:0/4130763843 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f0f9c0a6eb0 con 0x55affa176140 2024-05-21T22:00:23.417 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.415+0000 7f0fbbf75b80 1 -- 172.21.15.151:0/4130763843 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f9c0aafc0 con 0x55affa176140 2024-05-21T22:00:23.417 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.415+0000 7f0f94c00700 1 -- 172.21.15.151:0/4130763843 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0f9c0aa8c0 con 0x55affa176140 2024-05-21T22:00:23.417 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.415+0000 7f0fbbf75b80 10 client.26320 fetch_fsmap learned FSMap version 29 2024-05-21T22:00:23.417 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.415+0000 7f0fbbf75b80 1 -- 172.21.15.151:0/4130763843 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55affa094060 con 0x55affa176140 2024-05-21T22:00:23.417 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.415+0000 7f0f94c00700 1 -- 172.21.15.151:0/4130763843 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f9c0aafc0 con 0x55affa176140 2024-05-21T22:00:23.419 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.416+0000 7f0f94c00700 1 -- 172.21.15.151:0/4130763843 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f0f9c0aab20 con 0x55affa176140 2024-05-21T22:00:23.419 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.416+0000 7f0fbbf75b80 10 client.26320 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:00:23.419 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.416+0000 7f0fbbf75b80 10 client.26320 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-21T22:00:23.419 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.416+0000 7f0fbbf75b80 20 client.26320 populate_metadata read hostname 'smithi151' 2024-05-21T22:00:23.419 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.416+0000 7f0fbbf75b80 1 --2- 172.21.15.151:0/4130763843 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x55affa18f060 0x55affa191450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:23.419 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.416+0000 7f0fbbf75b80 4 client.26320 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T22:00:23.419 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.416+0000 7f0fbbf75b80 1 -- 172.21.15.151:0/4130763843 --> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55affa0d5d90 con 0x55affa18f060 2024-05-21T22:00:23.420 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.416+0000 7f0faa000700 1 --2- 172.21.15.151:0/4130763843 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x55affa18f060 0x55affa191450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:23.420 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.417+0000 7f0faa000700 1 --2- 172.21.15.151:0/4130763843 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x55affa18f060 0x55affa191450 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4dd8398da5f561d9 server_cookie=d7110ba563e67b6f in_seq=0 out_seq=0 2024-05-21T22:00:23.420 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.417+0000 7f0f94c00700 10 client.26320 ms_handle_connect on v2:172.21.15.184:6838/3059935143 2024-05-21T22:00:23.420 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.417+0000 7f0f94c00700 1 -- 172.21.15.151:0/4130763843 <== mds.1 v2:172.21.15.184:6838/3059935143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55affa0d5d90 con 0x55affa18f060 2024-05-21T22:00:23.420 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.417+0000 7f0f94c00700 10 client.26320 handle_command_reply: tid=0 2024-05-21T22:00:23.420 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.418+0000 7f0fbbf75b80 1 client.26320 shutdown 2024-05-21T22:00:23.420 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.418+0000 7f0fbbf75b80 1 -- 172.21.15.151:0/4130763843 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55affa176140 msgr2=0x55affa181c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:23.420 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.418+0000 7f0fbbf75b80 1 --2- 172.21.15.151:0/4130763843 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55affa176140 0x55affa181c20 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c005950 tx=0x7f0f9c0a6de0 comp rx=0 tx=0).stop 2024-05-21T22:00:23.420 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.418+0000 7f0fbbf75b80 1 -- 172.21.15.151:0/4130763843 shutdown_connections 2024-05-21T22:00:23.420 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.418+0000 7f0fbbf75b80 1 --2- 172.21.15.151:0/4130763843 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x55affa18f060 0x55affa191450 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:23.420 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.418+0000 7f0fbbf75b80 1 --2- 172.21.15.151:0/4130763843 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55affa17bf30 0x55affa17f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:23.420 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.418+0000 7f0fbbf75b80 1 --2- 172.21.15.151:0/4130763843 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55affa176140 0x55affa181c20 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:23.420 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.418+0000 7f0fbbf75b80 1 --2- 172.21.15.151:0/4130763843 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55affa17b610 0x55affa17efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:23.420 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.418+0000 7f0f94c00700 0 client.26320 ms_handle_reset on v2:172.21.15.184:6838/3059935143 2024-05-21T22:00:23.421 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.418+0000 7f0fbbf75b80 1 -- 172.21.15.151:0/4130763843 >> 172.21.15.151:0/4130763843 conn(0x55affa0a3be0 msgr2=0x55affa16fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:00:23.421 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.418+0000 7f0fbbf75b80 1 -- 172.21.15.151:0/4130763843 shutdown_connections 2024-05-21T22:00:23.421 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.418+0000 7f0fbbf75b80 1 -- 172.21.15.151:0/4130763843 wait complete. 2024-05-21T22:00:23.421 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.418+0000 7f0fbbf75b80 20 client.26320 trim_cache size 0 max 16384 2024-05-21T22:00:23.435 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.435+0000 7f0fbbf75b80 1 Processor -- start 2024-05-21T22:00:23.436 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.435+0000 7f0fbbf75b80 1 -- start start 2024-05-21T22:00:23.436 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.435+0000 7f0fbbf75b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55affa176140 0x55affa17b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:23.436 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.435+0000 7f0fbbf75b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55affa17b610 0x55affa17cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:23.436 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.435+0000 7f0fbbf75b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55affa17bf30 0x55affa1704f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:23.436 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.435+0000 7f0fbbf75b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0fac1afeb0 con 0x55affa17bf30 2024-05-21T22:00:23.436 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.435+0000 7f0fbbf75b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0fac1b19f0 con 0x55affa17b610 2024-05-21T22:00:23.436 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.435+0000 7f0fbbf75b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0fac1b0580 con 0x55affa176140 2024-05-21T22:00:23.437 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.436+0000 7f0faa000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55affa17b610 0x55affa17cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:23.437 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.436+0000 7f0faa000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55affa17b610 0x55affa17cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:33198/0 (socket says 172.21.15.151:33198) 2024-05-21T22:00:23.437 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.436+0000 7f0faa000700 1 -- 172.21.15.151:0/2507249043 learned_addr learned my addr 172.21.15.151:0/2507249043 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:00:23.437 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.436+0000 7f0fab400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55affa17bf30 0x55affa1704f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:23.437 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.436+0000 7f0faaa00700 1 --2- 172.21.15.151:0/2507249043 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55affa176140 0x55affa17b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:23.437 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.436+0000 7f0faa000700 1 -- 172.21.15.151:0/2507249043 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55affa176140 msgr2=0x55affa17b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:23.437 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.436+0000 7f0faa000700 1 --2- 172.21.15.151:0/2507249043 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55affa176140 0x55affa17b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:23.438 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.436+0000 7f0faa000700 1 -- 172.21.15.151:0/2507249043 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55affa17bf30 msgr2=0x55affa1704f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:23.438 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.436+0000 7f0faa000700 1 --2- 172.21.15.151:0/2507249043 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55affa17bf30 0x55affa1704f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:23.438 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.436+0000 7f0faa000700 1 -- 172.21.15.151:0/2507249043 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55affa17d020 con 0x55affa17b610 2024-05-21T22:00:23.438 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.436+0000 7f0faaa00700 1 --2- 172.21.15.151:0/2507249043 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55affa176140 0x55affa17b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:00:23.438 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.437+0000 7f0faa000700 1 --2- 172.21.15.151:0/2507249043 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55affa17b610 0x55affa17cae0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f0f98005920 tx=0x7f0f98014040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e59a2df8443d0aa server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:00:23.438 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.437+0000 7f0f97400700 1 -- 172.21.15.151:0/2507249043 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f98028630 con 0x55affa17b610 2024-05-21T22:00:23.438 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.437+0000 7f0f97400700 1 -- 172.21.15.151:0/2507249043 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f98028c80 con 0x55affa17b610 2024-05-21T22:00:23.438 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.437+0000 7f0f97400700 1 -- 172.21.15.151:0/2507249043 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f98011e10 con 0x55affa17b610 2024-05-21T22:00:23.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.438+0000 7f0fbbf75b80 1 -- 172.21.15.151:0/2507249043 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55affa17b610 msgr2=0x55affa17cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:23.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.438+0000 7f0fbbf75b80 1 --2- 172.21.15.151:0/2507249043 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55affa17b610 0x55affa17cae0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f0f98005920 tx=0x7f0f98014040 comp rx=0 tx=0).stop 2024-05-21T22:00:23.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.438+0000 7f0fbbf75b80 1 -- 172.21.15.151:0/2507249043 shutdown_connections 2024-05-21T22:00:23.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.438+0000 7f0fbbf75b80 1 --2- 172.21.15.151:0/2507249043 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55affa17bf30 0x55affa1704f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:23.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.438+0000 7f0fbbf75b80 1 --2- 172.21.15.151:0/2507249043 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55affa176140 0x55affa17b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:23.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.438+0000 7f0fbbf75b80 1 --2- 172.21.15.151:0/2507249043 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55affa17b610 0x55affa17cae0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:23.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.438+0000 7f0fbbf75b80 1 -- 172.21.15.151:0/2507249043 >> 172.21.15.151:0/2507249043 conn(0x55affa0a3be0 msgr2=0x55affa17c3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:00:23.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.438+0000 7f0fbbf75b80 1 -- 172.21.15.151:0/2507249043 shutdown_connections 2024-05-21T22:00:23.441 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.438+0000 7f0fbbf75b80 1 -- 172.21.15.151:0/2507249043 wait complete. 2024-05-21T22:00:23.441 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.438+0000 7f0fbbf75b80 1 -- start start 2024-05-21T22:00:23.441 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.438+0000 7f0fbbf75b80 1 Processor -- start 2024-05-21T22:00:23.441 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.439+0000 7f0fbbf75b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55affa176140 0x55affa191c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:23.441 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.439+0000 7f0fbbf75b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55affa17b610 0x55affa192150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:23.441 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.439+0000 7f0fbbf75b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55affa17bf30 0x55affa1fa070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:23.441 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.439+0000 7f0fbbf75b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0fac1b1320 con 0x55affa176140 2024-05-21T22:00:23.441 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.439+0000 7f0fbbf75b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0fac1b0c50 con 0x55affa17bf30 2024-05-21T22:00:23.441 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.439+0000 7f0fbbf75b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0fac1ade70 con 0x55affa17b610 2024-05-21T22:00:23.441 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.439+0000 7f0fbbf75b80 10 client.0 _ll_register_callbacks cb 0x55affa06dc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:00:23.441 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.439+0000 7f0faaa00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55affa176140 0x55affa191c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:23.441 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.439+0000 7f0faa000700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55affa17b610 0x55affa192150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:23.441 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.439+0000 7f0fab400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55affa17bf30 0x55affa1fa070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:23.442 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.439+0000 7f0faaa00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55affa176140 0x55affa191c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:59974/0 (socket says 172.21.15.151:59974) 2024-05-21T22:00:23.442 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.439+0000 7f0faaa00700 1 -- 172.21.15.151:0/3318105693 learned_addr learned my addr 172.21.15.151:0/3318105693 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:00:23.442 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.440+0000 7f0faa000700 1 -- 172.21.15.151:0/3318105693 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55affa17bf30 msgr2=0x55affa1fa070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:23.442 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.440+0000 7f0faa000700 1 --2- 172.21.15.151:0/3318105693 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55affa17bf30 0x55affa1fa070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:23.442 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.440+0000 7f0faa000700 1 -- 172.21.15.151:0/3318105693 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55affa176140 msgr2=0x55affa191c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:23.442 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.440+0000 7f0faa000700 1 --2- 172.21.15.151:0/3318105693 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55affa176140 0x55affa191c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:23.442 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.440+0000 7f0faa000700 1 -- 172.21.15.151:0/3318105693 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55affa1fa6e0 con 0x55affa17b610 2024-05-21T22:00:23.442 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.440+0000 7f0faaa00700 1 --2- 172.21.15.151:0/3318105693 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55affa176140 0x55affa191c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:00:23.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.440+0000 7f0faa000700 1 --2- 172.21.15.151:0/3318105693 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55affa17b610 0x55affa192150 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f0f98014010 tx=0x7f0f98028ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb5b891fffa3d09f server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:00:23.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.440+0000 7f0f94c00700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-21T22:00:23.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.440+0000 7f0f94c00700 1 -- 172.21.15.151:0/3318105693 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f98011a90 con 0x55affa17b610 2024-05-21T22:00:23.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.440+0000 7f0fbbf75b80 1 -- 172.21.15.151:0/3318105693 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f98028c80 con 0x55affa17b610 2024-05-21T22:00:23.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.440+0000 7f0f94c00700 1 -- 172.21.15.151:0/3318105693 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f0f98011c30 con 0x55affa17b610 2024-05-21T22:00:23.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.441+0000 7f0f94c00700 1 -- 172.21.15.151:0/3318105693 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0f9803e050 con 0x55affa17b610 2024-05-21T22:00:23.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.441+0000 7f0fbbf75b80 10 client.26326 fetch_fsmap learned FSMap version 29 2024-05-21T22:00:23.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.441+0000 7f0fbbf75b80 1 -- 172.21.15.151:0/3318105693 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55affa201a60 con 0x55affa17b610 2024-05-21T22:00:23.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.441+0000 7f0f94c00700 1 -- 172.21.15.151:0/3318105693 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f98028c80 con 0x55affa17b610 2024-05-21T22:00:23.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.441+0000 7f0f94c00700 1 -- 172.21.15.151:0/3318105693 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f0f980305b0 con 0x55affa17b610 2024-05-21T22:00:23.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.441+0000 7f0fbbf75b80 10 client.26326 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:00:23.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.441+0000 7f0fbbf75b80 10 client.26326 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-21T22:00:23.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.441+0000 7f0fbbf75b80 20 client.26326 populate_metadata read hostname 'smithi151' 2024-05-21T22:00:23.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.441+0000 7f0fbbf75b80 1 --2- 172.21.15.151:0/3318105693 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x55affa202090 0x55affa2044b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:23.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.441+0000 7f0fbbf75b80 4 client.26326 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T22:00:23.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.441+0000 7f0fbbf75b80 1 -- 172.21.15.151:0/3318105693 --> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55affa0cf570 con 0x55affa202090 2024-05-21T22:00:23.446 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.442+0000 7f0faaa00700 1 --2- 172.21.15.151:0/3318105693 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x55affa202090 0x55affa2044b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:23.446 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.442+0000 7f0faaa00700 1 --2- 172.21.15.151:0/3318105693 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x55affa202090 0x55affa2044b0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f8636d4e2db0951d server_cookie=6aac570e865f4b5b in_seq=0 out_seq=0 2024-05-21T22:00:23.446 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.442+0000 7f0f94c00700 10 client.26326 ms_handle_connect on v2:172.21.15.184:6838/3059935143 2024-05-21T22:00:23.446 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.442+0000 7f0f94c00700 1 -- 172.21.15.151:0/3318105693 <== mds.1 v2:172.21.15.184:6838/3059935143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55affa0cf570 con 0x55affa202090 2024-05-21T22:00:23.446 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.442+0000 7f0f94c00700 10 client.26326 handle_command_reply: tid=0 2024-05-21T22:00:23.446 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.442+0000 7f0fbbf75b80 1 client.26326 shutdown 2024-05-21T22:00:23.446 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.443+0000 7f0fbbf75b80 1 -- 172.21.15.151:0/3318105693 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55affa17b610 msgr2=0x55affa192150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:23.446 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.443+0000 7f0fbbf75b80 1 --2- 172.21.15.151:0/3318105693 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55affa17b610 0x55affa192150 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f0f98014010 tx=0x7f0f98028ec0 comp rx=0 tx=0).stop 2024-05-21T22:00:23.446 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.443+0000 7f0fbbf75b80 1 -- 172.21.15.151:0/3318105693 shutdown_connections 2024-05-21T22:00:23.446 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.443+0000 7f0fbbf75b80 1 --2- 172.21.15.151:0/3318105693 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x55affa202090 0x55affa2044b0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:23.446 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.443+0000 7f0fbbf75b80 1 --2- 172.21.15.151:0/3318105693 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55affa17bf30 0x55affa1fa070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:23.446 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.443+0000 7f0fbbf75b80 1 --2- 172.21.15.151:0/3318105693 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55affa176140 0x55affa191c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:23.446 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.443+0000 7f0fbbf75b80 1 --2- 172.21.15.151:0/3318105693 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55affa17b610 0x55affa192150 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:23.446 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.443+0000 7f0fbbf75b80 1 -- 172.21.15.151:0/3318105693 >> 172.21.15.151:0/3318105693 conn(0x55affa0a3be0 msgr2=0x55affa17d8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:00:23.447 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.443+0000 7f0f94c00700 0 client.26326 ms_handle_reset on v2:172.21.15.184:6838/3059935143 2024-05-21T22:00:23.447 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.443+0000 7f0fbbf75b80 1 -- 172.21.15.151:0/3318105693 shutdown_connections 2024-05-21T22:00:23.447 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.443+0000 7f0fbbf75b80 1 -- 172.21.15.151:0/3318105693 wait complete. 2024-05-21T22:00:23.447 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.443+0000 7f0fbbf75b80 20 client.26326 trim_cache size 0 max 16384 2024-05-21T22:00:23.447 INFO:teuthology.orchestra.run.smithi151.stdout:[] 2024-05-21T22:00:23.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.444+0000 7f0fb1600700 1 -- 172.21.15.151:0/718342511 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f0f900b2a70 msgr2=0x7f0f900b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:23.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.444+0000 7f0fb1600700 1 --2- 172.21.15.151:0/718342511 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f0f900b2a70 0x7f0f900b4f30 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0fa4009550 tx=0x7f0fa400b040 comp rx=0 tx=0).stop 2024-05-21T22:00:23.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.444+0000 7f0fb1600700 1 -- 172.21.15.151:0/718342511 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0fac06bab0 msgr2=0x7f0fac19bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:23.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.444+0000 7f0fb1600700 1 --2- 172.21.15.151:0/718342511 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0fac06bab0 0x7f0fac19bf70 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c009230 tx=0x7f0f9c00fa70 comp rx=0 tx=0).stop 2024-05-21T22:00:23.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.444+0000 7f0fb1600700 1 -- 172.21.15.151:0/718342511 shutdown_connections 2024-05-21T22:00:23.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.444+0000 7f0fb1600700 1 --2- 172.21.15.151:0/718342511 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f0f900b2a70 0x7f0f900b4f30 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:23.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.444+0000 7f0fb1600700 1 --2- 172.21.15.151:0/718342511 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0fac06bab0 0x7f0fac19bf70 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:23.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.444+0000 7f0fb1600700 1 --2- 172.21.15.151:0/718342511 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0fac1095a0 0x7f0fac1a0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:23.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.444+0000 7f0fb1600700 1 --2- 172.21.15.151:0/718342511 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0fac06c470 0x7f0fac19c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:23.449 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.444+0000 7f0fb1600700 1 -- 172.21.15.151:0/718342511 >> 172.21.15.151:0/718342511 conn(0x7f0fac0793b0 msgr2=0x7f0fac101030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:00:23.449 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.444+0000 7f0fb1600700 1 -- 172.21.15.151:0/718342511 shutdown_connections 2024-05-21T22:00:23.449 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:23.444+0000 7f0fb1600700 1 -- 172.21.15.151:0/718342511 wait complete. 2024-05-21T22:00:23.788 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:00:23 smithi184 ceph-mon[36942]: pgmap v146: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-05-21T22:00:23.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:00:23 smithi151 ceph-mon[31623]: pgmap v146: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-05-21T22:00:24.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:00:23 smithi164 ceph-mon[36269]: pgmap v146: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-05-21T22:00:24.519 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-21T22:00:24.972 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:00:25.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:00:25 smithi151 ceph-mon[31623]: pgmap v147: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:00:25.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:00:25 smithi184 ceph-mon[36942]: pgmap v147: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:00:26.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:00:25 smithi164 ceph-mon[36269]: pgmap v147: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:00:27.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.168+0000 7fa797800700 1 -- 172.21.15.151:0/2180811704 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa790075770 msgr2=0x7fa790075b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:27.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.168+0000 7fa797800700 1 --2- 172.21.15.151:0/2180811704 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa790075770 0x7fa790075b50 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fa780009990 tx=0x7fa780009ca0 comp rx=0 tx=0).stop 2024-05-21T22:00:27.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.168+0000 7fa797800700 1 -- 172.21.15.151:0/2180811704 shutdown_connections 2024-05-21T22:00:27.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.168+0000 7fa797800700 1 --2- 172.21.15.151:0/2180811704 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa79010ff40 0x7fa790112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:27.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.168+0000 7fa797800700 1 --2- 172.21.15.151:0/2180811704 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa790075770 0x7fa790075b50 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:27.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.168+0000 7fa797800700 1 --2- 172.21.15.151:0/2180811704 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa790076090 0x7fa79010fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:27.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.168+0000 7fa797800700 1 -- 172.21.15.151:0/2180811704 >> 172.21.15.151:0/2180811704 conn(0x7fa79007a6a0 msgr2=0x7fa79007aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:00:27.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.168+0000 7fa797800700 1 -- 172.21.15.151:0/2180811704 shutdown_connections 2024-05-21T22:00:27.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.169+0000 7fa797800700 1 -- 172.21.15.151:0/2180811704 wait complete. 2024-05-21T22:00:27.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.170+0000 7fa797800700 1 Processor -- start 2024-05-21T22:00:27.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.170+0000 7fa797800700 1 -- start start 2024-05-21T22:00:27.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.170+0000 7fa797800700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa790075770 0x7fa79019bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:27.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.170+0000 7fa797800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa790076090 0x7fa79019c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:27.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.170+0000 7fa797800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa79010ff40 0x7fa7901a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:27.172 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.170+0000 7fa797800700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa790115940 con 0x7fa790076090 2024-05-21T22:00:27.172 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.170+0000 7fa797800700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa7901157c0 con 0x7fa79010ff40 2024-05-21T22:00:27.172 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.170+0000 7fa797800700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa790115ac0 con 0x7fa790075770 2024-05-21T22:00:27.172 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.171+0000 7fa787e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa790076090 0x7fa79019c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:27.172 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.171+0000 7fa795800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa79010ff40 0x7fa7901a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:27.172 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.171+0000 7fa794e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa790075770 0x7fa79019bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:27.172 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.171+0000 7fa795800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa79010ff40 0x7fa7901a0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:36170/0 (socket says 172.21.15.151:36170) 2024-05-21T22:00:27.172 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.171+0000 7fa795800700 1 -- 172.21.15.151:0/4157444365 learned_addr learned my addr 172.21.15.151:0/4157444365 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:00:27.172 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.171+0000 7fa795800700 1 -- 172.21.15.151:0/4157444365 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa790075770 msgr2=0x7fa79019bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:27.172 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.171+0000 7fa795800700 1 --2- 172.21.15.151:0/4157444365 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa790075770 0x7fa79019bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:27.172 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.171+0000 7fa795800700 1 -- 172.21.15.151:0/4157444365 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa790076090 msgr2=0x7fa79019c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:27.172 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.171+0000 7fa795800700 1 --2- 172.21.15.151:0/4157444365 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa790076090 0x7fa79019c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:27.173 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.171+0000 7fa795800700 1 -- 172.21.15.151:0/4157444365 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7901a0db0 con 0x7fa79010ff40 2024-05-21T22:00:27.173 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.171+0000 7fa794e00700 1 --2- 172.21.15.151:0/4157444365 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa790075770 0x7fa79019bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:00:27.173 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.172+0000 7fa795800700 1 --2- 172.21.15.151:0/4157444365 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa79010ff40 0x7fa7901a0670 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa78800ef90 tx=0x7fa78800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:00:27.173 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.172+0000 7fa78e000700 1 -- 172.21.15.151:0/4157444365 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7880092e0 con 0x7fa79010ff40 2024-05-21T22:00:27.173 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.172+0000 7fa78e000700 1 -- 172.21.15.151:0/4157444365 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa788004bb0 con 0x7fa79010ff40 2024-05-21T22:00:27.173 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.172+0000 7fa797800700 1 -- 172.21.15.151:0/4157444365 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7901a1040 con 0x7fa79010ff40 2024-05-21T22:00:27.173 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.172+0000 7fa78e000700 1 -- 172.21.15.151:0/4157444365 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa788004280 con 0x7fa79010ff40 2024-05-21T22:00:27.173 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.172+0000 7fa797800700 1 -- 172.21.15.151:0/4157444365 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7901a8920 con 0x7fa79010ff40 2024-05-21T22:00:27.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.175+0000 7fa78e000700 1 -- 172.21.15.151:0/4157444365 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7fa78801b030 con 0x7fa79010ff40 2024-05-21T22:00:27.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.177+0000 7fa7a225cb80 1 Processor -- start 2024-05-21T22:00:27.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.177+0000 7fa7a225cb80 1 -- start start 2024-05-21T22:00:27.178 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.177+0000 7fa78e000700 1 --2- 172.21.15.151:0/4157444365 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fa7740b29f0 0x7fa7740b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:27.178 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.177+0000 7fa7a225cb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55df6381feb0 0x55df63820290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:27.178 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.177+0000 7fa7a225cb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df638207d0 0x55df6381a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:27.178 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.177+0000 7fa7a225cb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55df6381a880 0x55df6381ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:27.178 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.177+0000 7fa7a225cb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55df63778e40 con 0x55df6381feb0 2024-05-21T22:00:27.178 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.177+0000 7fa7a225cb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55df637689e0 con 0x55df6381a880 2024-05-21T22:00:27.178 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.177+0000 7fa7a225cb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55df6376ca20 con 0x55df638207d0 2024-05-21T22:00:27.178 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.177+0000 7fa78e000700 1 -- 172.21.15.151:0/4157444365 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fa7880e2190 con 0x7fa79010ff40 2024-05-21T22:00:27.178 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.178+0000 7fa794e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55df6381feb0 0x55df63820290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:27.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.178+0000 7fa794e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55df6381feb0 0x55df63820290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:45556/0 (socket says 172.21.15.151:45556) 2024-05-21T22:00:27.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.178+0000 7fa794e00700 1 -- 172.21.15.151:0/3337387011 learned_addr learned my addr 172.21.15.151:0/3337387011 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:00:27.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.178+0000 7fa787e00700 1 --2- 172.21.15.151:0/4157444365 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fa7740b29f0 0x7fa7740b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:27.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.178+0000 7fa795800700 1 --2- 172.21.15.151:0/3337387011 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55df6381a880 0x55df6381ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:27.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.178+0000 7fa787e00700 1 --2- 172.21.15.151:0/3337387011 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df638207d0 0x55df6381a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:27.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.178+0000 7fa794e00700 1 -- 172.21.15.151:0/3337387011 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df638207d0 msgr2=0x55df6381a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:27.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.178+0000 7fa794e00700 1 --2- 172.21.15.151:0/3337387011 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df638207d0 0x55df6381a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:27.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.178+0000 7fa794e00700 1 -- 172.21.15.151:0/3337387011 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55df6381a880 msgr2=0x55df6381ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:27.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.178+0000 7fa794e00700 1 --2- 172.21.15.151:0/3337387011 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55df6381a880 0x55df6381ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:27.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.178+0000 7fa794e00700 1 -- 172.21.15.151:0/3337387011 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df63731420 con 0x55df6381feb0 2024-05-21T22:00:27.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.179+0000 7fa794e00700 1 --2- 172.21.15.151:0/3337387011 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55df6381feb0 0x55df63820290 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fa7800096a0 tx=0x7fa7800096d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=550674a5618fc627 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:00:27.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.179+0000 7fa787e00700 1 --2- 172.21.15.151:0/4157444365 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fa7740b29f0 0x7fa7740b4eb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa790111b20 tx=0x7fa778009b60 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:00:27.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.179+0000 7fa786000700 1 -- 172.21.15.151:0/3337387011 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa780027070 con 0x55df6381feb0 2024-05-21T22:00:27.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.179+0000 7fa786000700 1 -- 172.21.15.151:0/3337387011 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa780003680 con 0x55df6381feb0 2024-05-21T22:00:27.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.179+0000 7fa786000700 1 -- 172.21.15.151:0/3337387011 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa78000f6a0 con 0x55df6381feb0 2024-05-21T22:00:27.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.180+0000 7fa7a225cb80 1 -- 172.21.15.151:0/3337387011 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55df6381feb0 msgr2=0x55df63820290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:27.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.180+0000 7fa7a225cb80 1 --2- 172.21.15.151:0/3337387011 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55df6381feb0 0x55df63820290 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fa7800096a0 tx=0x7fa7800096d0 comp rx=0 tx=0).stop 2024-05-21T22:00:27.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.180+0000 7fa7a225cb80 1 -- 172.21.15.151:0/3337387011 shutdown_connections 2024-05-21T22:00:27.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.180+0000 7fa7a225cb80 1 --2- 172.21.15.151:0/3337387011 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55df6381a880 0x55df6381ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:27.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.180+0000 7fa7a225cb80 1 --2- 172.21.15.151:0/3337387011 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55df6381feb0 0x55df63820290 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:27.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.180+0000 7fa7a225cb80 1 --2- 172.21.15.151:0/3337387011 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df638207d0 0x55df6381a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:27.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.180+0000 7fa7a225cb80 1 -- 172.21.15.151:0/3337387011 >> 172.21.15.151:0/3337387011 conn(0x55df63748be0 msgr2=0x55df63815010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:00:27.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.180+0000 7fa7a225cb80 1 -- 172.21.15.151:0/3337387011 shutdown_connections 2024-05-21T22:00:27.183 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.180+0000 7fa7a225cb80 1 -- 172.21.15.151:0/3337387011 wait complete. 2024-05-21T22:00:27.183 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.181+0000 7fa7a225cb80 1 -- start start 2024-05-21T22:00:27.183 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.181+0000 7fa7a225cb80 1 Processor -- start 2024-05-21T22:00:27.183 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.181+0000 7fa7a225cb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55df6381a880 0x55df638264c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:27.183 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.181+0000 7fa7a225cb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55df6381feb0 0x55df638238d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:27.183 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.181+0000 7fa7a225cb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df638207d0 0x55df63823e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:27.183 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.181+0000 7fa7a225cb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55df6366efa0 con 0x55df6381feb0 2024-05-21T22:00:27.183 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.181+0000 7fa7a225cb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55df63709760 con 0x55df6381a880 2024-05-21T22:00:27.183 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.181+0000 7fa7a225cb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55df638271e0 con 0x55df638207d0 2024-05-21T22:00:27.183 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.181+0000 7fa7a225cb80 10 client.0 _ll_register_callbacks cb 0x55df63712c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:00:27.183 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.182+0000 7fa787e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55df6381feb0 0x55df638238d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:27.183 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.182+0000 7fa787e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55df6381feb0 0x55df638238d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:45558/0 (socket says 172.21.15.151:45558) 2024-05-21T22:00:27.183 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.182+0000 7fa787e00700 1 -- 172.21.15.151:0/3155422139 learned_addr learned my addr 172.21.15.151:0/3155422139 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:00:27.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.182+0000 7fa795800700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df638207d0 0x55df63823e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:27.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.182+0000 7fa794e00700 1 --2- 172.21.15.151:0/3155422139 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55df6381a880 0x55df638264c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:27.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.182+0000 7fa795800700 1 -- 172.21.15.151:0/3155422139 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55df6381a880 msgr2=0x55df638264c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:27.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.182+0000 7fa795800700 1 --2- 172.21.15.151:0/3155422139 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55df6381a880 0x55df638264c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:27.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.182+0000 7fa795800700 1 -- 172.21.15.151:0/3155422139 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55df6381feb0 msgr2=0x55df638238d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:27.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.182+0000 7fa795800700 1 --2- 172.21.15.151:0/3155422139 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55df6381feb0 0x55df638238d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:27.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.182+0000 7fa795800700 1 -- 172.21.15.151:0/3155422139 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df636b88d0 con 0x55df638207d0 2024-05-21T22:00:27.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.182+0000 7fa787e00700 1 --2- 172.21.15.151:0/3155422139 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55df6381feb0 0x55df638238d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:00:27.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.182+0000 7fa795800700 1 --2- 172.21.15.151:0/3155422139 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df638207d0 0x55df63823e10 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa7880df890 tx=0x7fa7880a99f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83be8e839804cf31 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:00:27.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.182+0000 7fa77ea00700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-21T22:00:27.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.183+0000 7fa77ea00700 1 -- 172.21.15.151:0/3155422139 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7880c3060 con 0x55df638207d0 2024-05-21T22:00:27.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.183+0000 7fa77ea00700 1 -- 172.21.15.151:0/3155422139 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fa7880a3de0 con 0x55df638207d0 2024-05-21T22:00:27.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.183+0000 7fa7a225cb80 1 -- 172.21.15.151:0/3155422139 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa780003680 con 0x55df638207d0 2024-05-21T22:00:27.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.183+0000 7fa77ea00700 1 -- 172.21.15.151:0/3155422139 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa7880ca050 con 0x55df638207d0 2024-05-21T22:00:27.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.183+0000 7fa7a225cb80 10 client.26338 fetch_fsmap learned FSMap version 29 2024-05-21T22:00:27.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.183+0000 7fa7a225cb80 1 -- 172.21.15.151:0/3155422139 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55df63739060 con 0x55df638207d0 2024-05-21T22:00:27.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.183+0000 7fa77ea00700 1 -- 172.21.15.151:0/3155422139 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa780003680 con 0x55df638207d0 2024-05-21T22:00:27.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.183+0000 7fa77ea00700 1 -- 172.21.15.151:0/3155422139 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fa7880ca1f0 con 0x55df638207d0 2024-05-21T22:00:27.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.184+0000 7fa7a225cb80 10 client.26338 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:00:27.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.184+0000 7fa7a225cb80 10 client.26338 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T22:00:27.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.184+0000 7fa7a225cb80 20 client.26338 populate_metadata read hostname 'smithi151' 2024-05-21T22:00:27.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.184+0000 7fa7a225cb80 1 --2- 172.21.15.151:0/3155422139 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55df63833d10 0x55df63836100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:27.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.184+0000 7fa7a225cb80 4 client.26338 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T22:00:27.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.184+0000 7fa7a225cb80 1 -- 172.21.15.151:0/3155422139 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55df6377ad90 con 0x55df63833d10 2024-05-21T22:00:27.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.184+0000 7fa794e00700 1 --2- 172.21.15.151:0/3155422139 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55df63833d10 0x55df63836100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:27.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.185+0000 7fa794e00700 1 --2- 172.21.15.151:0/3155422139 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55df63833d10 0x55df63836100 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cecbd016126b5d8 server_cookie=2b3ed01ea4a873f9 in_seq=0 out_seq=0 2024-05-21T22:00:27.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.185+0000 7fa77ea00700 10 client.26338 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T22:00:27.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.185+0000 7fa77ea00700 1 -- 172.21.15.151:0/3155422139 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55df6377ad90 con 0x55df63833d10 2024-05-21T22:00:27.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.185+0000 7fa77ea00700 10 client.26338 handle_command_reply: tid=0 2024-05-21T22:00:27.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.186+0000 7fa7a225cb80 1 client.26338 shutdown 2024-05-21T22:00:27.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.186+0000 7fa7a225cb80 1 -- 172.21.15.151:0/3155422139 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df638207d0 msgr2=0x55df63823e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:27.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.186+0000 7fa7a225cb80 1 --2- 172.21.15.151:0/3155422139 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df638207d0 0x55df63823e10 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa7880df890 tx=0x7fa7880a99f0 comp rx=0 tx=0).stop 2024-05-21T22:00:27.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.186+0000 7fa7a225cb80 1 -- 172.21.15.151:0/3155422139 shutdown_connections 2024-05-21T22:00:27.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.186+0000 7fa7a225cb80 1 --2- 172.21.15.151:0/3155422139 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55df63833d10 0x55df63836100 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:27.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.186+0000 7fa7a225cb80 1 --2- 172.21.15.151:0/3155422139 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55df6381a880 0x55df638264c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:27.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.186+0000 7fa7a225cb80 1 --2- 172.21.15.151:0/3155422139 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df638207d0 0x55df63823e10 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:27.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.186+0000 7fa7a225cb80 1 --2- 172.21.15.151:0/3155422139 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55df6381feb0 0x55df638238d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:27.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.186+0000 7fa77ea00700 0 client.26338 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T22:00:27.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.186+0000 7fa7a225cb80 1 -- 172.21.15.151:0/3155422139 >> 172.21.15.151:0/3155422139 conn(0x55df63748be0 msgr2=0x55df63819250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:00:27.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.186+0000 7fa7a225cb80 1 -- 172.21.15.151:0/3155422139 shutdown_connections 2024-05-21T22:00:27.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.186+0000 7fa7a225cb80 1 -- 172.21.15.151:0/3155422139 wait complete. 2024-05-21T22:00:27.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.186+0000 7fa7a225cb80 20 client.26338 trim_cache size 0 max 16384 2024-05-21T22:00:27.203 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.202+0000 7fa7a225cb80 1 Processor -- start 2024-05-21T22:00:27.203 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.203+0000 7fa7a225cb80 1 -- start start 2024-05-21T22:00:27.204 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.203+0000 7fa7a225cb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55df6381a880 0x55df6381a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:27.204 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.203+0000 7fa7a225cb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df6381feb0 0x55df638264c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:27.204 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.203+0000 7fa7a225cb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55df638207d0 0x55df6381d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:27.204 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.203+0000 7fa7a225cb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa7901aff20 con 0x55df6381a880 2024-05-21T22:00:27.205 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.203+0000 7fa7a225cb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa7901b1a60 con 0x55df638207d0 2024-05-21T22:00:27.205 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.203+0000 7fa7a225cb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa7901b05f0 con 0x55df6381feb0 2024-05-21T22:00:27.205 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.203+0000 7fa794e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55df6381a880 0x55df6381a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:27.205 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.203+0000 7fa795800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55df638207d0 0x55df6381d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:27.205 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.203+0000 7fa787e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df6381feb0 0x55df638264c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:27.205 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.203+0000 7fa795800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55df638207d0 0x55df6381d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:36194/0 (socket says 172.21.15.151:36194) 2024-05-21T22:00:27.205 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.203+0000 7fa795800700 1 -- 172.21.15.151:0/822086913 learned_addr learned my addr 172.21.15.151:0/822086913 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:00:27.205 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.203+0000 7fa794e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55df6381a880 0x55df6381a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:45568/0 (socket says 172.21.15.151:45568) 2024-05-21T22:00:27.205 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.204+0000 7fa795800700 1 -- 172.21.15.151:0/822086913 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df6381feb0 msgr2=0x55df638264c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:27.205 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.204+0000 7fa795800700 1 --2- 172.21.15.151:0/822086913 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df6381feb0 0x55df638264c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:27.205 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.204+0000 7fa795800700 1 -- 172.21.15.151:0/822086913 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55df6381a880 msgr2=0x55df6381a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:27.205 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.204+0000 7fa795800700 1 --2- 172.21.15.151:0/822086913 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55df6381a880 0x55df6381a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:27.205 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.204+0000 7fa795800700 1 -- 172.21.15.151:0/822086913 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df63824140 con 0x55df638207d0 2024-05-21T22:00:27.206 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.204+0000 7fa795800700 1 --2- 172.21.15.151:0/822086913 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55df638207d0 0x55df6381d9c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa7880090a0 tx=0x7fa7880097c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a410cc180556313 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:00:27.206 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.204+0000 7fa785600700 1 -- 172.21.15.151:0/822086913 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7880c3030 con 0x55df638207d0 2024-05-21T22:00:27.206 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.204+0000 7fa785600700 1 -- 172.21.15.151:0/822086913 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7880a8fc0 con 0x55df638207d0 2024-05-21T22:00:27.206 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.204+0000 7fa785600700 1 -- 172.21.15.151:0/822086913 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7880c5e60 con 0x55df638207d0 2024-05-21T22:00:27.207 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.205+0000 7fa7a225cb80 1 -- 172.21.15.151:0/822086913 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55df638207d0 msgr2=0x55df6381d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:27.207 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.205+0000 7fa7a225cb80 1 --2- 172.21.15.151:0/822086913 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55df638207d0 0x55df6381d9c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa7880090a0 tx=0x7fa7880097c0 comp rx=0 tx=0).stop 2024-05-21T22:00:27.208 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.205+0000 7fa7a225cb80 1 -- 172.21.15.151:0/822086913 shutdown_connections 2024-05-21T22:00:27.208 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.205+0000 7fa7a225cb80 1 --2- 172.21.15.151:0/822086913 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55df638207d0 0x55df6381d9c0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:27.208 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.205+0000 7fa7a225cb80 1 --2- 172.21.15.151:0/822086913 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55df6381a880 0x55df6381a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:27.208 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.205+0000 7fa7a225cb80 1 --2- 172.21.15.151:0/822086913 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df6381feb0 0x55df638264c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:27.208 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.205+0000 7fa7a225cb80 1 -- 172.21.15.151:0/822086913 >> 172.21.15.151:0/822086913 conn(0x55df63748be0 msgr2=0x55df638221c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:00:27.208 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.205+0000 7fa7a225cb80 1 -- 172.21.15.151:0/822086913 shutdown_connections 2024-05-21T22:00:27.208 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.205+0000 7fa7a225cb80 1 -- 172.21.15.151:0/822086913 wait complete. 2024-05-21T22:00:27.208 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.205+0000 7fa7a225cb80 1 -- start start 2024-05-21T22:00:27.208 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.206+0000 7fa7a225cb80 1 Processor -- start 2024-05-21T22:00:27.208 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.206+0000 7fa7a225cb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55df6381a880 0x55df6382c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:27.208 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.206+0000 7fa7a225cb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df6381feb0 0x55df6382cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:27.208 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.206+0000 7fa7a225cb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55df638207d0 0x55df6383ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:27.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.206+0000 7fa7a225cb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa7901b1390 con 0x55df6381a880 2024-05-21T22:00:27.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.206+0000 7fa7a225cb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa7901b0cc0 con 0x55df638207d0 2024-05-21T22:00:27.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.206+0000 7fa7a225cb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa7901adec0 con 0x55df6381feb0 2024-05-21T22:00:27.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.206+0000 7fa7a225cb80 10 client.0 _ll_register_callbacks cb 0x55df63712c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:00:27.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.206+0000 7fa794e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55df6381a880 0x55df6382c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:27.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.206+0000 7fa787e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df6381feb0 0x55df6382cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:27.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.207+0000 7fa787e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df6381feb0 0x55df6382cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:38148/0 (socket says 172.21.15.151:38148) 2024-05-21T22:00:27.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.207+0000 7fa794e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55df6381a880 0x55df6382c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:45580/0 (socket says 172.21.15.151:45580) 2024-05-21T22:00:27.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.207+0000 7fa787e00700 1 -- 172.21.15.151:0/458771299 learned_addr learned my addr 172.21.15.151:0/458771299 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:00:27.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.207+0000 7fa795800700 1 --2- 172.21.15.151:0/458771299 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55df638207d0 0x55df6383ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:27.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.207+0000 7fa787e00700 1 -- 172.21.15.151:0/458771299 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55df638207d0 msgr2=0x55df6383ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:27.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.207+0000 7fa787e00700 1 --2- 172.21.15.151:0/458771299 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55df638207d0 0x55df6383ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:27.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.207+0000 7fa787e00700 1 -- 172.21.15.151:0/458771299 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55df6381a880 msgr2=0x55df6382c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:27.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.207+0000 7fa787e00700 1 --2- 172.21.15.151:0/458771299 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55df6381a880 0x55df6382c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:27.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.207+0000 7fa787e00700 1 -- 172.21.15.151:0/458771299 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df6383b650 con 0x55df6381feb0 2024-05-21T22:00:27.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.207+0000 7fa787e00700 1 --2- 172.21.15.151:0/458771299 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df6381feb0 0x55df6382cb90 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x55df6380d1f0 tx=0x7fa77801d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3041c05b987121bf server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:00:27.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.207+0000 7fa77ea00700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-21T22:00:27.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.208+0000 7fa77ea00700 1 -- 172.21.15.151:0/458771299 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa778031060 con 0x55df6381feb0 2024-05-21T22:00:27.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.208+0000 7fa7a225cb80 1 -- 172.21.15.151:0/458771299 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7880a8fc0 con 0x55df6381feb0 2024-05-21T22:00:27.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.208+0000 7fa77ea00700 1 -- 172.21.15.151:0/458771299 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fa7780106c0 con 0x55df6381feb0 2024-05-21T22:00:27.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.208+0000 7fa77ea00700 1 -- 172.21.15.151:0/458771299 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa7780108c0 con 0x55df6381feb0 2024-05-21T22:00:27.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.208+0000 7fa7a225cb80 10 client.26344 fetch_fsmap learned FSMap version 29 2024-05-21T22:00:27.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.208+0000 7fa7a225cb80 1 -- 172.21.15.151:0/458771299 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55df6389e400 con 0x55df6381feb0 2024-05-21T22:00:27.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.208+0000 7fa77ea00700 1 -- 172.21.15.151:0/458771299 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7880a8fc0 con 0x55df6381feb0 2024-05-21T22:00:27.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.209+0000 7fa77ea00700 1 -- 172.21.15.151:0/458771299 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fa77802b070 con 0x55df6381feb0 2024-05-21T22:00:27.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.209+0000 7fa7a225cb80 10 client.26344 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:00:27.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.209+0000 7fa7a225cb80 10 client.26344 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T22:00:27.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.209+0000 7fa7a225cb80 20 client.26344 populate_metadata read hostname 'smithi151' 2024-05-21T22:00:27.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.209+0000 7fa7a225cb80 1 --2- 172.21.15.151:0/458771299 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55df6389eae0 0x55df638a0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:27.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.209+0000 7fa7a225cb80 4 client.26344 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T22:00:27.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.209+0000 7fa7a225cb80 1 -- 172.21.15.151:0/458771299 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55df63774570 con 0x55df6389eae0 2024-05-21T22:00:27.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.209+0000 7fa794e00700 1 --2- 172.21.15.151:0/458771299 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55df6389eae0 0x55df638a0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:27.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.210+0000 7fa77ea00700 10 client.26344 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T22:00:27.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.210+0000 7fa794e00700 1 --2- 172.21.15.151:0/458771299 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55df6389eae0 0x55df638a0f00 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f65672b071b14756 server_cookie=d3453ef8014084e3 in_seq=0 out_seq=0 2024-05-21T22:00:27.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.210+0000 7fa77ea00700 1 -- 172.21.15.151:0/458771299 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55df63774570 con 0x55df6389eae0 2024-05-21T22:00:27.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.210+0000 7fa77ea00700 10 client.26344 handle_command_reply: tid=0 2024-05-21T22:00:27.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.210+0000 7fa7a225cb80 1 client.26344 shutdown 2024-05-21T22:00:27.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.210+0000 7fa7a225cb80 1 -- 172.21.15.151:0/458771299 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df6381feb0 msgr2=0x55df6382cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:27.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.210+0000 7fa7a225cb80 1 --2- 172.21.15.151:0/458771299 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df6381feb0 0x55df6382cb90 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x55df6380d1f0 tx=0x7fa77801d000 comp rx=0 tx=0).stop 2024-05-21T22:00:27.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.210+0000 7fa7a225cb80 1 -- 172.21.15.151:0/458771299 shutdown_connections 2024-05-21T22:00:27.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.210+0000 7fa7a225cb80 1 --2- 172.21.15.151:0/458771299 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55df6389eae0 0x55df638a0f00 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:27.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.210+0000 7fa7a225cb80 1 --2- 172.21.15.151:0/458771299 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55df638207d0 0x55df6383ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:27.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.210+0000 7fa7a225cb80 1 --2- 172.21.15.151:0/458771299 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55df6381a880 0x55df6382c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:27.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.210+0000 7fa7a225cb80 1 --2- 172.21.15.151:0/458771299 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df6381feb0 0x55df6382cb90 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:27.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.210+0000 7fa77ea00700 0 client.26344 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T22:00:27.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.210+0000 7fa7a225cb80 1 -- 172.21.15.151:0/458771299 >> 172.21.15.151:0/458771299 conn(0x55df63748be0 msgr2=0x55df638149e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:00:27.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.211+0000 7fa7a225cb80 1 -- 172.21.15.151:0/458771299 shutdown_connections 2024-05-21T22:00:27.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.211+0000 7fa7a225cb80 1 -- 172.21.15.151:0/458771299 wait complete. 2024-05-21T22:00:27.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.211+0000 7fa7a225cb80 20 client.26344 trim_cache size 0 max 16384 2024-05-21T22:00:27.215 INFO:teuthology.orchestra.run.smithi151.stdout:[] 2024-05-21T22:00:27.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.211+0000 7fa797800700 1 -- 172.21.15.151:0/4157444365 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fa7740b29f0 msgr2=0x7fa7740b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:27.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.211+0000 7fa797800700 1 --2- 172.21.15.151:0/4157444365 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fa7740b29f0 0x7fa7740b4eb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa790111b20 tx=0x7fa778009b60 comp rx=0 tx=0).stop 2024-05-21T22:00:27.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.211+0000 7fa797800700 1 -- 172.21.15.151:0/4157444365 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa79010ff40 msgr2=0x7fa7901a0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:27.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.211+0000 7fa797800700 1 --2- 172.21.15.151:0/4157444365 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa79010ff40 0x7fa7901a0670 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa78800ef90 tx=0x7fa78800c3a0 comp rx=0 tx=0).stop 2024-05-21T22:00:27.216 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.212+0000 7fa797800700 1 -- 172.21.15.151:0/4157444365 shutdown_connections 2024-05-21T22:00:27.216 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.212+0000 7fa797800700 1 --2- 172.21.15.151:0/4157444365 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fa7740b29f0 0x7fa7740b4eb0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:27.216 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.212+0000 7fa797800700 1 --2- 172.21.15.151:0/4157444365 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa79010ff40 0x7fa7901a0670 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:27.216 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.212+0000 7fa797800700 1 --2- 172.21.15.151:0/4157444365 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa790075770 0x7fa79019bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:27.216 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.212+0000 7fa797800700 1 --2- 172.21.15.151:0/4157444365 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa790076090 0x7fa79019c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:27.216 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.212+0000 7fa797800700 1 -- 172.21.15.151:0/4157444365 >> 172.21.15.151:0/4157444365 conn(0x7fa79007a6a0 msgr2=0x7fa7901006f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:00:27.216 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.212+0000 7fa797800700 1 -- 172.21.15.151:0/4157444365 shutdown_connections 2024-05-21T22:00:27.216 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:27.212+0000 7fa797800700 1 -- 172.21.15.151:0/4157444365 wait complete. 2024-05-21T22:00:27.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:00:27 smithi151 ceph-mon[31623]: pgmap v148: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:00:27.985 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:00:27 smithi184 ceph-mon[36942]: pgmap v148: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:00:28.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:00:27 smithi164 ceph-mon[36269]: pgmap v148: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:00:29.121 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-21T22:00:29.122 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-21T22:00:29.578 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:00:29.921 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:00:29 smithi151 ceph-mon[31623]: pgmap v149: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T22:00:29.985 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:00:29 smithi184 ceph-mon[36942]: pgmap v149: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T22:00:30.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:00:29 smithi164 ceph-mon[36269]: pgmap v149: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T22:00:31.630 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:00:31 smithi151 ceph-mon[31623]: pgmap v150: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:00:31.958 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.957+0000 7f9f15800700 1 -- 172.21.15.151:0/1176997728 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9f10075970 msgr2=0x7f9f10075d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:31.959 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.957+0000 7f9f15800700 1 --2- 172.21.15.151:0/1176997728 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9f10075970 0x7f9f10075d50 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f9f00009990 tx=0x7f9f00009ca0 comp rx=0 tx=0).stop 2024-05-21T22:00:31.959 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.957+0000 7f9f15800700 1 -- 172.21.15.151:0/1176997728 shutdown_connections 2024-05-21T22:00:31.959 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.957+0000 7f9f15800700 1 --2- 172.21.15.151:0/1176997728 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f1010fcb0 0x7f9f10112120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:31.959 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.957+0000 7f9f15800700 1 --2- 172.21.15.151:0/1176997728 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9f10075970 0x7f9f10075d50 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:31.959 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.957+0000 7f9f15800700 1 --2- 172.21.15.151:0/1176997728 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9f10076290 0x7f9f1010f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:31.959 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.957+0000 7f9f15800700 1 -- 172.21.15.151:0/1176997728 >> 172.21.15.151:0/1176997728 conn(0x7f9f100fefc0 msgr2=0x7f9f101013e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:00:31.959 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.957+0000 7f9f15800700 1 -- 172.21.15.151:0/1176997728 shutdown_connections 2024-05-21T22:00:31.959 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.958+0000 7f9f15800700 1 -- 172.21.15.151:0/1176997728 wait complete. 2024-05-21T22:00:31.959 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.958+0000 7f9f15800700 1 Processor -- start 2024-05-21T22:00:31.959 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.959+0000 7f9f15800700 1 -- start start 2024-05-21T22:00:31.961 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.959+0000 7f9f15800700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9f10075970 0x7f9f1019bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:31.961 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.959+0000 7f9f15800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9f10076290 0x7f9f1019c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:31.961 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.959+0000 7f9f15800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f1010fcb0 0x7f9f101a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:31.961 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.959+0000 7f9f15800700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9f101158b0 con 0x7f9f10076290 2024-05-21T22:00:31.961 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.959+0000 7f9f15800700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9f10115730 con 0x7f9f1010fcb0 2024-05-21T22:00:31.961 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.959+0000 7f9f15800700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9f10115a30 con 0x7f9f10075970 2024-05-21T22:00:31.961 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.959+0000 7f9f0f400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9f10076290 0x7f9f1019c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:31.962 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.959+0000 7f9f0f400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9f10076290 0x7f9f1019c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:45602/0 (socket says 172.21.15.151:45602) 2024-05-21T22:00:31.962 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.959+0000 7f9f0f400700 1 -- 172.21.15.151:0/2050052646 learned_addr learned my addr 172.21.15.151:0/2050052646 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:00:31.962 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.960+0000 7f9f0fe00700 1 --2- 172.21.15.151:0/2050052646 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9f10075970 0x7f9f1019bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:31.962 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.960+0000 7f9f14e00700 1 --2- 172.21.15.151:0/2050052646 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f1010fcb0 0x7f9f101a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:31.962 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.960+0000 7f9f0f400700 1 -- 172.21.15.151:0/2050052646 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9f10075970 msgr2=0x7f9f1019bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:31.962 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.960+0000 7f9f0f400700 1 --2- 172.21.15.151:0/2050052646 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9f10075970 0x7f9f1019bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:31.962 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.960+0000 7f9f0f400700 1 -- 172.21.15.151:0/2050052646 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f1010fcb0 msgr2=0x7f9f101a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:31.962 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.960+0000 7f9f0f400700 1 --2- 172.21.15.151:0/2050052646 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f1010fcb0 0x7f9f101a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:31.962 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.960+0000 7f9f0f400700 1 -- 172.21.15.151:0/2050052646 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f101a0db0 con 0x7f9f10076290 2024-05-21T22:00:31.962 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.960+0000 7f9f0f400700 1 --2- 172.21.15.151:0/2050052646 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9f10076290 0x7f9f1019c2e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f9efc00e9c0 tx=0x7f9efc00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:00:31.962 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.961+0000 7f9f0cc00700 1 -- 172.21.15.151:0/2050052646 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9efc00cbd0 con 0x7f9f10076290 2024-05-21T22:00:31.962 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.961+0000 7f9f0cc00700 1 -- 172.21.15.151:0/2050052646 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9efc00ee60 con 0x7f9f10076290 2024-05-21T22:00:31.962 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.961+0000 7f9f0cc00700 1 -- 172.21.15.151:0/2050052646 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9efc0055c0 con 0x7f9f10076290 2024-05-21T22:00:31.962 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.961+0000 7f9f15800700 1 -- 172.21.15.151:0/2050052646 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f101a10a0 con 0x7f9f10076290 2024-05-21T22:00:31.963 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.961+0000 7f9f15800700 1 -- 172.21.15.151:0/2050052646 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f101a8920 con 0x7f9f10076290 2024-05-21T22:00:31.966 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.966+0000 7f9f0cc00700 1 -- 172.21.15.151:0/2050052646 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7f9efc004750 con 0x7f9f10076290 2024-05-21T22:00:31.966 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.966+0000 7f9f21837b80 1 Processor -- start 2024-05-21T22:00:31.966 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.966+0000 7f9f21837b80 1 -- start start 2024-05-21T22:00:31.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.967+0000 7f9f21837b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561aa287c840 0x561aa287cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:31.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.967+0000 7f9f21837b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561aa287d160 0x561aa2876df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:31.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.967+0000 7f9f21837b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561aa2877410 0x561aa2877870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:31.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.967+0000 7f9f21837b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x561aa27d4ec0 con 0x561aa287d160 2024-05-21T22:00:31.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.967+0000 7f9f21837b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x561aa27c4a60 con 0x561aa2877410 2024-05-21T22:00:31.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.967+0000 7f9f21837b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x561aa27c8aa0 con 0x561aa287c840 2024-05-21T22:00:31.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.967+0000 7f9f0fe00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561aa287d160 0x561aa2876df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:31.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.967+0000 7f9f14e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561aa287c840 0x561aa287cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:31.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.967+0000 7f9f0fe00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561aa287d160 0x561aa2876df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:45614/0 (socket says 172.21.15.151:45614) 2024-05-21T22:00:31.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.967+0000 7f9f0fe00700 1 -- 172.21.15.151:0/3659243886 learned_addr learned my addr 172.21.15.151:0/3659243886 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:00:31.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.967+0000 7f9f0f400700 1 --2- 172.21.15.151:0/3659243886 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561aa2877410 0x561aa2877870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:31.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.967+0000 7f9f14e00700 1 -- 172.21.15.151:0/3659243886 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561aa2877410 msgr2=0x561aa2877870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:31.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.968+0000 7f9f14e00700 1 --2- 172.21.15.151:0/3659243886 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561aa2877410 0x561aa2877870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:31.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.968+0000 7f9f14e00700 1 -- 172.21.15.151:0/3659243886 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561aa287d160 msgr2=0x561aa2876df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:31.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.968+0000 7f9f14e00700 1 --2- 172.21.15.151:0/3659243886 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561aa287d160 0x561aa2876df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:31.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.968+0000 7f9f14e00700 1 -- 172.21.15.151:0/3659243886 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561aa278d4a0 con 0x561aa287c840 2024-05-21T22:00:31.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.968+0000 7f9f0fe00700 1 --2- 172.21.15.151:0/3659243886 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561aa287d160 0x561aa2876df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:00:31.970 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.968+0000 7f9f0cc00700 1 --2- 172.21.15.151:0/2050052646 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f9ef40b2a70 0x7f9ef40b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:31.970 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.968+0000 7f9f14e00700 1 --2- 172.21.15.151:0/3659243886 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561aa287c840 0x561aa287cc20 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f9f08011580 tx=0x7f9f08009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13822a25bf31f6a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:00:31.970 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.968+0000 7f9f0cc00700 1 -- 172.21.15.151:0/2050052646 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f9efc0e2810 con 0x7f9f10076290 2024-05-21T22:00:31.970 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.968+0000 7f9f0fe00700 1 --2- 172.21.15.151:0/2050052646 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f9ef40b2a70 0x7f9ef40b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:31.970 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.968+0000 7f9f05600700 1 -- 172.21.15.151:0/3659243886 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f08011e80 con 0x561aa287c840 2024-05-21T22:00:31.970 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.968+0000 7f9f05600700 1 -- 172.21.15.151:0/3659243886 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f08004bb0 con 0x561aa287c840 2024-05-21T22:00:31.970 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.968+0000 7f9f05600700 1 -- 172.21.15.151:0/3659243886 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f08004720 con 0x561aa287c840 2024-05-21T22:00:31.970 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.969+0000 7f9f21837b80 1 -- 172.21.15.151:0/3659243886 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561aa287c840 msgr2=0x561aa287cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:31.970 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.969+0000 7f9f0fe00700 1 --2- 172.21.15.151:0/2050052646 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f9ef40b2a70 0x7f9ef40b4f30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9f00015000 tx=0x7f9f00005fd0 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:00:31.970 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.969+0000 7f9f21837b80 1 --2- 172.21.15.151:0/3659243886 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561aa287c840 0x561aa287cc20 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f9f08011580 tx=0x7f9f08009920 comp rx=0 tx=0).stop 2024-05-21T22:00:31.970 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.969+0000 7f9f21837b80 1 -- 172.21.15.151:0/3659243886 shutdown_connections 2024-05-21T22:00:31.970 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.969+0000 7f9f21837b80 1 --2- 172.21.15.151:0/3659243886 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561aa2877410 0x561aa2877870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:31.971 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.969+0000 7f9f21837b80 1 --2- 172.21.15.151:0/3659243886 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561aa287c840 0x561aa287cc20 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:31.971 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.969+0000 7f9f21837b80 1 --2- 172.21.15.151:0/3659243886 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561aa287d160 0x561aa2876df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:31.971 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.969+0000 7f9f21837b80 1 -- 172.21.15.151:0/3659243886 >> 172.21.15.151:0/3659243886 conn(0x561aa27a4c60 msgr2=0x561aa2871890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:00:31.971 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.969+0000 7f9f21837b80 1 -- 172.21.15.151:0/3659243886 shutdown_connections 2024-05-21T22:00:31.971 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.970+0000 7f9f21837b80 1 -- 172.21.15.151:0/3659243886 wait complete. 2024-05-21T22:00:31.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.970+0000 7f9f21837b80 1 -- start start 2024-05-21T22:00:31.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.970+0000 7f9f21837b80 1 Processor -- start 2024-05-21T22:00:31.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.971+0000 7f9f21837b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561aa2877410 0x561aa2882f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:31.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.971+0000 7f9f21837b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561aa287c840 0x561aa28802d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:31.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.971+0000 7f9f21837b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561aa287d160 0x561aa2880810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:31.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.971+0000 7f9f21837b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x561aa26cb020 con 0x561aa287c840 2024-05-21T22:00:31.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.971+0000 7f9f21837b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x561aa27657e0 con 0x561aa287d160 2024-05-21T22:00:31.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.971+0000 7f9f21837b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x561aa2883c80 con 0x561aa2877410 2024-05-21T22:00:31.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.971+0000 7f9f21837b80 10 client.0 _ll_register_callbacks cb 0x561aa276ecf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:00:31.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.971+0000 7f9f0fe00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561aa2877410 0x561aa2882f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:31.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.971+0000 7f9f0f400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561aa287c840 0x561aa28802d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:31.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.971+0000 7f9f0fe00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561aa2877410 0x561aa2882f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:38176/0 (socket says 172.21.15.151:38176) 2024-05-21T22:00:31.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.971+0000 7f9f0fe00700 1 -- 172.21.15.151:0/1209653855 learned_addr learned my addr 172.21.15.151:0/1209653855 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:00:31.974 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.971+0000 7f9f14e00700 1 --2- 172.21.15.151:0/1209653855 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561aa287d160 0x561aa2880810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:31.974 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.972+0000 7f9f0f400700 1 -- 172.21.15.151:0/1209653855 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561aa2877410 msgr2=0x561aa2882f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:31.974 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.972+0000 7f9f0f400700 1 --2- 172.21.15.151:0/1209653855 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561aa2877410 0x561aa2882f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:31.974 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.972+0000 7f9f0f400700 1 -- 172.21.15.151:0/1209653855 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561aa287d160 msgr2=0x561aa2880810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:31.974 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.972+0000 7f9f0f400700 1 --2- 172.21.15.151:0/1209653855 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561aa287d160 0x561aa2880810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:31.974 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.972+0000 7f9f0f400700 1 -- 172.21.15.151:0/1209653855 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561aa2714950 con 0x561aa287c840 2024-05-21T22:00:31.974 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.972+0000 7f9f0f400700 1 --2- 172.21.15.151:0/1209653855 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561aa287c840 0x561aa28802d0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f9efc0dff70 tx=0x7f9efc0aa010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=353d7cc35a9ed4ec server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:00:31.974 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.972+0000 7f9efa000700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-21T22:00:31.974 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.972+0000 7f9efa000700 1 -- 172.21.15.151:0/1209653855 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9efc0a59c0 con 0x561aa287c840 2024-05-21T22:00:31.974 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.972+0000 7f9efa000700 1 -- 172.21.15.151:0/1209653855 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f9efc0a8f70 con 0x561aa287c840 2024-05-21T22:00:31.974 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.972+0000 7f9f21837b80 1 -- 172.21.15.151:0/1209653855 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f08004bb0 con 0x561aa287c840 2024-05-21T22:00:31.974 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.973+0000 7f9efa000700 1 -- 172.21.15.151:0/1209653855 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9efc0a5290 con 0x561aa287c840 2024-05-21T22:00:31.974 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.973+0000 7f9f21837b80 10 client.16221 fetch_fsmap learned FSMap version 29 2024-05-21T22:00:31.974 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.973+0000 7f9f21837b80 1 -- 172.21.15.151:0/1209653855 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561aa27950e0 con 0x561aa287c840 2024-05-21T22:00:31.976 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.973+0000 7f9efa000700 1 -- 172.21.15.151:0/1209653855 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f08004bb0 con 0x561aa287c840 2024-05-21T22:00:31.976 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.973+0000 7f9efa000700 1 -- 172.21.15.151:0/1209653855 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f9efc004a40 con 0x561aa287c840 2024-05-21T22:00:31.976 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.973+0000 7f9f21837b80 10 client.16221 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:00:31.976 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.973+0000 7f9f21837b80 10 client.16221 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T22:00:31.976 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.973+0000 7f9f21837b80 20 client.16221 populate_metadata read hostname 'smithi151' 2024-05-21T22:00:31.976 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.973+0000 7f9f21837b80 1 --2- 172.21.15.151:0/1209653855 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x561aa28907f0 0x561aa2892c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:31.976 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.973+0000 7f9f21837b80 4 client.16221 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T22:00:31.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.973+0000 7f9f21837b80 1 -- 172.21.15.151:0/1209653855 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561aa27d6e10 con 0x561aa28907f0 2024-05-21T22:00:31.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.974+0000 7f9f0fe00700 1 --2- 172.21.15.151:0/1209653855 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x561aa28907f0 0x561aa2892c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:31.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.974+0000 7f9f0fe00700 1 --2- 172.21.15.151:0/1209653855 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x561aa28907f0 0x561aa2892c10 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=407b11119d527b3 server_cookie=e28279f83e24e42a in_seq=0 out_seq=0 2024-05-21T22:00:31.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.974+0000 7f9efa000700 10 client.16221 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T22:00:31.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.975+0000 7f9efa000700 1 -- 172.21.15.151:0/1209653855 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x561aa27d6e10 con 0x561aa28907f0 2024-05-21T22:00:31.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.975+0000 7f9efa000700 10 client.16221 handle_command_reply: tid=0 2024-05-21T22:00:31.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.975+0000 7f9f21837b80 1 client.16221 shutdown 2024-05-21T22:00:31.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.975+0000 7f9f21837b80 1 -- 172.21.15.151:0/1209653855 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561aa287c840 msgr2=0x561aa28802d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:31.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.975+0000 7f9f21837b80 1 --2- 172.21.15.151:0/1209653855 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561aa287c840 0x561aa28802d0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f9efc0dff70 tx=0x7f9efc0aa010 comp rx=0 tx=0).stop 2024-05-21T22:00:31.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.976+0000 7f9f21837b80 1 -- 172.21.15.151:0/1209653855 shutdown_connections 2024-05-21T22:00:31.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.976+0000 7f9f21837b80 1 --2- 172.21.15.151:0/1209653855 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x561aa28907f0 0x561aa2892c10 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:31.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.976+0000 7f9f21837b80 1 --2- 172.21.15.151:0/1209653855 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561aa287d160 0x561aa2880810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:31.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.976+0000 7f9f21837b80 1 --2- 172.21.15.151:0/1209653855 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561aa2877410 0x561aa2882f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:31.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.976+0000 7f9f21837b80 1 --2- 172.21.15.151:0/1209653855 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561aa287c840 0x561aa28802d0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:31.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.976+0000 7f9efa000700 0 client.16221 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T22:00:31.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.976+0000 7f9f21837b80 1 -- 172.21.15.151:0/1209653855 >> 172.21.15.151:0/1209653855 conn(0x561aa27a4c60 msgr2=0x561aa2871040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:00:31.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.976+0000 7f9f21837b80 1 -- 172.21.15.151:0/1209653855 shutdown_connections 2024-05-21T22:00:31.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.976+0000 7f9f21837b80 1 -- 172.21.15.151:0/1209653855 wait complete. 2024-05-21T22:00:31.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.976+0000 7f9f21837b80 20 client.16221 trim_cache size 0 max 16384 2024-05-21T22:00:31.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:00:31 smithi184 ceph-mon[36942]: pgmap v150: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:00:31.999 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.999+0000 7f9f21837b80 1 Processor -- start 2024-05-21T22:00:31.999 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.999+0000 7f9f21837b80 1 -- start start 2024-05-21T22:00:31.999 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.999+0000 7f9f21837b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561aa2877410 0x561aa2876df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:31.999 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.999+0000 7f9f21837b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561aa287c840 0x561aa28717c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:32.000 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.999+0000 7f9f21837b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561aa287d160 0x561aa287bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:32.000 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.999+0000 7f9f21837b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9f101afee0 con 0x561aa287d160 2024-05-21T22:00:32.000 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.999+0000 7f9f21837b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9f101b1a20 con 0x561aa287c840 2024-05-21T22:00:32.000 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.999+0000 7f9f21837b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9f101b05b0 con 0x561aa2877410 2024-05-21T22:00:32.001 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.999+0000 7f9f14e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561aa287d160 0x561aa287bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:32.001 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.999+0000 7f9f0f400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561aa287c840 0x561aa28717c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:32.001 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:31.999+0000 7f9f0fe00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561aa2877410 0x561aa2876df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:32.001 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:32.000+0000 7f9f0f400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561aa287c840 0x561aa28717c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:36240/0 (socket says 172.21.15.151:36240) 2024-05-21T22:00:32.001 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:32.000+0000 7f9f14e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561aa287d160 0x561aa287bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:45638/0 (socket says 172.21.15.151:45638) 2024-05-21T22:00:32.001 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:32.000+0000 7f9f0f400700 1 -- 172.21.15.151:0/3403596315 learned_addr learned my addr 172.21.15.151:0/3403596315 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:00:32.001 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:32.000+0000 7f9f0f400700 1 -- 172.21.15.151:0/3403596315 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561aa2877410 msgr2=0x561aa2876df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:32.001 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:32.000+0000 7f9f0f400700 1 --2- 172.21.15.151:0/3403596315 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561aa2877410 0x561aa2876df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:32.001 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:32.000+0000 7f9f0f400700 1 -- 172.21.15.151:0/3403596315 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561aa287d160 msgr2=0x561aa287bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:32.001 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:32.000+0000 7f9f0f400700 1 --2- 172.21.15.151:0/3403596315 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561aa287d160 0x561aa287bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:32.001 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:32.000+0000 7f9f0f400700 1 -- 172.21.15.151:0/3403596315 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561aa2873250 con 0x561aa287c840 2024-05-21T22:00:32.002 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:32.000+0000 7f9f0fe00700 1 --2- 172.21.15.151:0/3403596315 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561aa2877410 0x561aa2876df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:00:32.002 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:32.000+0000 7f9f14e00700 1 --2- 172.21.15.151:0/3403596315 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561aa287d160 0x561aa287bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:00:32.002 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:32.000+0000 7f9f0f400700 1 --2- 172.21.15.151:0/3403596315 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561aa287c840 0x561aa28717c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f9efc0e81b0 tx=0x7f9efc0a9610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea8c1ba97eab7e38 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:00:32.002 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:32.000+0000 7f9f04c00700 1 -- 172.21.15.151:0/3403596315 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9efc0a9270 con 0x561aa287c840 2024-05-21T22:00:32.002 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:32.001+0000 7f9f04c00700 1 -- 172.21.15.151:0/3403596315 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9efc0b22a0 con 0x561aa287c840 2024-05-21T22:00:32.002 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:32.001+0000 7f9f04c00700 1 -- 172.21.15.151:0/3403596315 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9efc0a9450 con 0x561aa287c840 2024-05-21T22:00:32.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:32.001+0000 7f9f21837b80 1 -- 172.21.15.151:0/3403596315 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561aa287c840 msgr2=0x561aa28717c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:32.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:32.001+0000 7f9f21837b80 1 --2- 172.21.15.151:0/3403596315 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561aa287c840 0x561aa28717c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f9efc0e81b0 tx=0x7f9efc0a9610 comp rx=0 tx=0).stop 2024-05-21T22:00:32.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:32.001+0000 7f9f21837b80 1 -- 172.21.15.151:0/3403596315 shutdown_connections 2024-05-21T22:00:32.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:32.001+0000 7f9f21837b80 1 --2- 172.21.15.151:0/3403596315 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561aa287d160 0x561aa287bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:32.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:32.001+0000 7f9f21837b80 1 --2- 172.21.15.151:0/3403596315 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561aa2877410 0x561aa2876df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:32.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:32.001+0000 7f9f21837b80 1 --2- 172.21.15.151:0/3403596315 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561aa287c840 0x561aa28717c0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:32.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:32.001+0000 7f9f21837b80 1 -- 172.21.15.151:0/3403596315 >> 172.21.15.151:0/3403596315 conn(0x561aa27a4c60 msgr2=0x561aa287b400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:00:32.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:32.001+0000 7f9f21837b80 1 -- 172.21.15.151:0/3403596315 shutdown_connections 2024-05-21T22:00:32.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:32.001+0000 7f9f21837b80 1 -- 172.21.15.151:0/3403596315 wait complete. 2024-05-21T22:00:32.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:32.002+0000 7f9f21837b80 1 -- start start 2024-05-21T22:00:32.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:32.002+0000 7f9f21837b80 1 Processor -- start 2024-05-21T22:00:32.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:32.003+0000 7f9f21837b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561aa2877410 0x561aa2885ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:32.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:32.003+0000 7f9f21837b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561aa287c840 0x561aa2886530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:32.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:32.003+0000 7f9f21837b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561aa287d160 0x561aa28fb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:32.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:32.003+0000 7f9f21837b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9f101b1350 con 0x561aa2877410 2024-05-21T22:00:32.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:32.003+0000 7f9f21837b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9f101b0c80 con 0x561aa287c840 2024-05-21T22:00:32.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:32.003+0000 7f9f21837b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9f101ade80 con 0x561aa287d160 2024-05-21T22:00:32.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:32.003+0000 7f9f21837b80 10 client.0 _ll_register_callbacks cb 0x561aa276ecf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:00:32.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:32.003+0000 7f9f0fe00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561aa2877410 0x561aa2885ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:32.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:32.003+0000 7f9f0fe00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561aa2877410 0x561aa2885ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:45652/0 (socket says 172.21.15.151:45652) 2024-05-21T22:00:32.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:32.003+0000 7f9f14e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561aa287d160 0x561aa28fb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:32.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:32.003+0000 7f9f0fe00700 1 -- 172.21.15.151:0/1603421670 learned_addr learned my addr 172.21.15.151:0/1603421670 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:00:32.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:32.003+0000 7f9f0f400700 1 --2- 172.21.15.151:0/1603421670 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561aa287c840 0x561aa2886530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:32.006 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:32.003+0000 7f9f0f400700 1 -- 172.21.15.151:0/1603421670 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561aa287d160 msgr2=0x561aa28fb3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:32.006 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:32.003+0000 7f9f0f400700 1 --2- 172.21.15.151:0/1603421670 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561aa287d160 0x561aa28fb3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:32.006 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:32.003+0000 7f9f0f400700 1 -- 172.21.15.151:0/1603421670 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561aa2877410 msgr2=0x561aa2885ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:32.006 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:32.003+0000 7f9f0f400700 1 --2- 172.21.15.151:0/1603421670 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561aa2877410 0x561aa2885ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:32.006 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:32.003+0000 7f9f0f400700 1 -- 172.21.15.151:0/1603421670 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561aa28fb930 con 0x561aa287c840 2024-05-21T22:00:32.006 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:32.003+0000 7f9f0fe00700 1 --2- 172.21.15.151:0/1603421670 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561aa2877410 0x561aa2885ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:00:32.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:32.003+0000 7f9f14e00700 1 --2- 172.21.15.151:0/1603421670 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561aa287d160 0x561aa28fb3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:00:32.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:32.004+0000 7f9f0f400700 1 --2- 172.21.15.151:0/1603421670 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561aa287c840 0x561aa2886530 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x561aa28803e0 tx=0x7f9efc0aa2f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50c17232d55c752d server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:00:32.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:32.004+0000 7f9efa000700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-21T22:00:32.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:32.004+0000 7f9efa000700 1 -- 172.21.15.151:0/1603421670 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9efc0a98d0 con 0x561aa287c840 2024-05-21T22:00:32.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:32.004+0000 7f9efa000700 1 -- 172.21.15.151:0/1603421670 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f9efc0a5680 con 0x561aa287c840 2024-05-21T22:00:32.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:32.004+0000 7f9f21837b80 1 -- 172.21.15.151:0/1603421670 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561aa27d05f0 con 0x561aa287c840 2024-05-21T22:00:32.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:32.004+0000 7f9efa000700 1 -- 172.21.15.151:0/1603421670 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9efc0a9450 con 0x561aa287c840 2024-05-21T22:00:32.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:32.004+0000 7f9f21837b80 10 client.26348 fetch_fsmap learned FSMap version 29 2024-05-21T22:00:32.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:32.004+0000 7f9f21837b80 1 -- 172.21.15.151:0/1603421670 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561aa28fb930 con 0x561aa287c840 2024-05-21T22:00:32.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:32.005+0000 7f9efa000700 1 -- 172.21.15.151:0/1603421670 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x561aa27d05f0 con 0x561aa287c840 2024-05-21T22:00:32.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:32.005+0000 7f9efa000700 1 -- 172.21.15.151:0/1603421670 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f9efc087020 con 0x561aa287c840 2024-05-21T22:00:32.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:32.005+0000 7f9f21837b80 10 client.26348 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:00:32.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:32.005+0000 7f9f21837b80 10 client.26348 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T22:00:32.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:32.005+0000 7f9f21837b80 20 client.26348 populate_metadata read hostname 'smithi151' 2024-05-21T22:00:32.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:32.005+0000 7f9f21837b80 1 --2- 172.21.15.151:0/1603421670 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x561aa28fce00 0x561aa28ff1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:32.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:32.005+0000 7f9f21837b80 4 client.26348 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-21T22:00:32.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:32.005+0000 7f9f21837b80 1 -- 172.21.15.151:0/1603421670 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561aa26472d0 con 0x561aa28fce00 2024-05-21T22:00:32.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:32.005+0000 7f9f0fe00700 1 --2- 172.21.15.151:0/1603421670 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x561aa28fce00 0x561aa28ff1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:32.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:32.006+0000 7f9f0fe00700 1 --2- 172.21.15.151:0/1603421670 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x561aa28fce00 0x561aa28ff1f0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6edee12019a22d38 server_cookie=b3a8f4b46411df27 in_seq=0 out_seq=0 2024-05-21T22:00:32.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:32.006+0000 7f9efa000700 10 client.26348 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T22:00:32.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:32.007+0000 7f9efa000700 1 -- 172.21.15.151:0/1603421670 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561aa26472d0 con 0x561aa28fce00 2024-05-21T22:00:32.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:32.007+0000 7f9efa000700 10 client.26348 handle_command_reply: tid=0 2024-05-21T22:00:32.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:32.007+0000 7f9f21837b80 1 client.26348 shutdown 2024-05-21T22:00:32.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:32.007+0000 7f9f21837b80 1 -- 172.21.15.151:0/1603421670 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561aa287c840 msgr2=0x561aa2886530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:32.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:32.007+0000 7f9f21837b80 1 --2- 172.21.15.151:0/1603421670 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561aa287c840 0x561aa2886530 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x561aa28803e0 tx=0x7f9efc0aa2f0 comp rx=0 tx=0).stop 2024-05-21T22:00:32.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:32.007+0000 7f9f21837b80 1 -- 172.21.15.151:0/1603421670 shutdown_connections 2024-05-21T22:00:32.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:32.007+0000 7f9f21837b80 1 --2- 172.21.15.151:0/1603421670 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561aa287d160 0x561aa28fb3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:32.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:32.007+0000 7f9f21837b80 1 --2- 172.21.15.151:0/1603421670 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561aa2877410 0x561aa2885ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:32.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:32.007+0000 7f9f21837b80 1 --2- 172.21.15.151:0/1603421670 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x561aa28fce00 0x561aa28ff1f0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:32.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:32.007+0000 7f9f21837b80 1 --2- 172.21.15.151:0/1603421670 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x561aa287c840 0x561aa2886530 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:32.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:32.007+0000 7f9f21837b80 1 -- 172.21.15.151:0/1603421670 >> 172.21.15.151:0/1603421670 conn(0x561aa27a4c60 msgr2=0x561aa287bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:00:32.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:32.007+0000 7f9efa000700 0 client.26348 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T22:00:32.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:32.008+0000 7f9f21837b80 1 -- 172.21.15.151:0/1603421670 shutdown_connections 2024-05-21T22:00:32.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:32.008+0000 7f9f21837b80 1 -- 172.21.15.151:0/1603421670 wait complete. 2024-05-21T22:00:32.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:32.008+0000 7f9f21837b80 20 client.26348 trim_cache size 0 max 16384 2024-05-21T22:00:32.010 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-05-21T22:00:32.010 INFO:teuthology.orchestra.run.smithi151.stdout: "return_code": 0, 2024-05-21T22:00:32.010 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_tag": "77e818ea-4ec5-49a6-baf2-ad4a56e9e97b", 2024-05-21T22:00:32.010 INFO:teuthology.orchestra.run.smithi151.stdout: "mode": "asynchronous" 2024-05-21T22:00:32.010 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-05-21T22:00:32.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:32.009+0000 7f9f15800700 1 -- 172.21.15.151:0/2050052646 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f9ef40b2a70 msgr2=0x7f9ef40b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:32.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:32.009+0000 7f9f15800700 1 --2- 172.21.15.151:0/2050052646 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f9ef40b2a70 0x7f9ef40b4f30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9f00015000 tx=0x7f9f00005fd0 comp rx=0 tx=0).stop 2024-05-21T22:00:32.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:32.009+0000 7f9f15800700 1 -- 172.21.15.151:0/2050052646 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9f10076290 msgr2=0x7f9f1019c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:32.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:32.009+0000 7f9f15800700 1 --2- 172.21.15.151:0/2050052646 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9f10076290 0x7f9f1019c2e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f9efc00e9c0 tx=0x7f9efc00ecd0 comp rx=0 tx=0).stop 2024-05-21T22:00:32.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:32.009+0000 7f9f15800700 1 -- 172.21.15.151:0/2050052646 shutdown_connections 2024-05-21T22:00:32.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:32.009+0000 7f9f15800700 1 --2- 172.21.15.151:0/2050052646 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f9ef40b2a70 0x7f9ef40b4f30 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:32.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:32.009+0000 7f9f15800700 1 --2- 172.21.15.151:0/2050052646 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f1010fcb0 0x7f9f101a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:32.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:32.009+0000 7f9f15800700 1 --2- 172.21.15.151:0/2050052646 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9f10075970 0x7f9f1019bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:32.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:32.009+0000 7f9f15800700 1 --2- 172.21.15.151:0/2050052646 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9f10076290 0x7f9f1019c2e0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:32.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:32.009+0000 7f9f15800700 1 -- 172.21.15.151:0/2050052646 >> 172.21.15.151:0/2050052646 conn(0x7f9f100fefc0 msgr2=0x7f9f100ffc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:00:32.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:32.009+0000 7f9f15800700 1 -- 172.21.15.151:0/2050052646 shutdown_connections 2024-05-21T22:00:32.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:32.009+0000 7f9f15800700 1 -- 172.21.15.151:0/2050052646 wait complete. 2024-05-21T22:00:32.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:00:31 smithi164 ceph-mon[36269]: pgmap v150: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:00:32.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:00:32 smithi151 ceph-mon[31623]: scrub queued for path: / 2024-05-21T22:00:32.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:00:32 smithi151 ceph-mon[31623]: scrub summary: idle+waiting paths [/] 2024-05-21T22:00:32.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:00:32 smithi151 ceph-mon[31623]: scrub summary: active paths [/] 2024-05-21T22:00:32.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:00:32 smithi151 ceph-mon[31623]: scrub summary: idle+waiting paths [/] 2024-05-21T22:00:32.868 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-21T22:00:32.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:00:32 smithi184 ceph-mon[36942]: scrub queued for path: / 2024-05-21T22:00:32.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:00:32 smithi184 ceph-mon[36942]: scrub summary: idle+waiting paths [/] 2024-05-21T22:00:32.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:00:32 smithi184 ceph-mon[36942]: scrub summary: active paths [/] 2024-05-21T22:00:32.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:00:32 smithi184 ceph-mon[36942]: scrub summary: idle+waiting paths [/] 2024-05-21T22:00:33.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:00:32 smithi164 ceph-mon[36269]: scrub queued for path: / 2024-05-21T22:00:33.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:00:32 smithi164 ceph-mon[36269]: scrub summary: idle+waiting paths [/] 2024-05-21T22:00:33.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:00:32 smithi164 ceph-mon[36269]: scrub summary: active paths [/] 2024-05-21T22:00:33.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:00:32 smithi164 ceph-mon[36269]: scrub summary: idle+waiting paths [/] 2024-05-21T22:00:33.321 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:00:33.727 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:00:33 smithi151 ceph-mon[31623]: pgmap v151: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 0 B/s wr, 12 op/s 2024-05-21T22:00:33.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:00:33 smithi184 ceph-mon[36942]: pgmap v151: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 0 B/s wr, 12 op/s 2024-05-21T22:00:34.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:00:33 smithi164 ceph-mon[36269]: pgmap v151: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 0 B/s wr, 12 op/s 2024-05-21T22:00:35.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:00:35 smithi151 ceph-mon[31623]: scrub summary: idle 2024-05-21T22:00:35.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:00:35 smithi151 ceph-mon[31623]: pgmap v152: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 0 B/s wr, 8 op/s 2024-05-21T22:00:35.447 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.447+0000 7faf9fe00700 1 -- 172.21.15.151:0/2127505793 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf981037f0 msgr2=0x7faf98103bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:35.447 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.447+0000 7faf9fe00700 1 --2- 172.21.15.151:0/2127505793 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf981037f0 0x7faf98103bd0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7faf8c0099c0 tx=0x7faf8c009cd0 comp rx=0 tx=0).stop 2024-05-21T22:00:35.447 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.447+0000 7faf9fe00700 1 -- 172.21.15.151:0/2127505793 shutdown_connections 2024-05-21T22:00:35.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.447+0000 7faf9fe00700 1 --2- 172.21.15.151:0/2127505793 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faf981088f0 0x7faf98110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:35.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.447+0000 7faf9fe00700 1 --2- 172.21.15.151:0/2127505793 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf981037f0 0x7faf98103bd0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:35.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.447+0000 7faf9fe00700 1 --2- 172.21.15.151:0/2127505793 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf981041a0 0x7faf981081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:35.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.447+0000 7faf9fe00700 1 -- 172.21.15.151:0/2127505793 >> 172.21.15.151:0/2127505793 conn(0x7faf9807a6a0 msgr2=0x7faf9807aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:00:35.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.447+0000 7faf9fe00700 1 -- 172.21.15.151:0/2127505793 shutdown_connections 2024-05-21T22:00:35.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.447+0000 7faf9fe00700 1 -- 172.21.15.151:0/2127505793 wait complete. 2024-05-21T22:00:35.449 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.449+0000 7faf9fe00700 1 Processor -- start 2024-05-21T22:00:35.449 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.449+0000 7faf9fe00700 1 -- start start 2024-05-21T22:00:35.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.450+0000 7faf9fe00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faf981037f0 0x7faf98075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:35.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.450+0000 7faf9fe00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf981041a0 0x7faf98075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:35.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.450+0000 7faf9fe00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf981088f0 0x7faf980762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:35.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.450+0000 7faf9fe00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7faf981136e0 con 0x7faf981088f0 2024-05-21T22:00:35.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.450+0000 7faf9fe00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7faf98113560 con 0x7faf981041a0 2024-05-21T22:00:35.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.450+0000 7faf9fe00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7faf98113860 con 0x7faf981037f0 2024-05-21T22:00:35.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.450+0000 7faf9ca00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf981041a0 0x7faf98075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:35.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.450+0000 7faf9ca00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf981041a0 0x7faf98075d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:36264/0 (socket says 172.21.15.151:36264) 2024-05-21T22:00:35.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.450+0000 7faf9ca00700 1 -- 172.21.15.151:0/1461049151 learned_addr learned my addr 172.21.15.151:0/1461049151 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:00:35.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.450+0000 7faf9de00700 1 --2- 172.21.15.151:0/1461049151 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf981088f0 0x7faf980762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:35.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.450+0000 7faf9d400700 1 --2- 172.21.15.151:0/1461049151 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faf981037f0 0x7faf98075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:35.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.450+0000 7faf9ca00700 1 -- 172.21.15.151:0/1461049151 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faf981037f0 msgr2=0x7faf98075840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:35.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.450+0000 7faf9ca00700 1 --2- 172.21.15.151:0/1461049151 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faf981037f0 0x7faf98075840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:35.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.450+0000 7faf9ca00700 1 -- 172.21.15.151:0/1461049151 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf981088f0 msgr2=0x7faf980762c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:35.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.450+0000 7faf9ca00700 1 --2- 172.21.15.151:0/1461049151 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf981088f0 0x7faf980762c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:35.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.450+0000 7faf9ca00700 1 -- 172.21.15.151:0/1461049151 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf981a80c0 con 0x7faf981041a0 2024-05-21T22:00:35.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.450+0000 7faf9de00700 1 --2- 172.21.15.151:0/1461049151 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf981088f0 0x7faf980762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T22:00:35.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.451+0000 7faf9ca00700 1 --2- 172.21.15.151:0/1461049151 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf981041a0 0x7faf98075d80 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7faf8400eb30 tx=0x7faf8400ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:00:35.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.451+0000 7faf96000700 1 -- 172.21.15.151:0/1461049151 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf8400cc90 con 0x7faf981041a0 2024-05-21T22:00:35.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.451+0000 7faf9fe00700 1 -- 172.21.15.151:0/1461049151 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf981a83b0 con 0x7faf981041a0 2024-05-21T22:00:35.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.451+0000 7faf96000700 1 -- 172.21.15.151:0/1461049151 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf840044d0 con 0x7faf981041a0 2024-05-21T22:00:35.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.451+0000 7faf96000700 1 -- 172.21.15.151:0/1461049151 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf84010430 con 0x7faf981041a0 2024-05-21T22:00:35.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.451+0000 7faf9fe00700 1 -- 172.21.15.151:0/1461049151 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf981a8940 con 0x7faf981041a0 2024-05-21T22:00:35.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.455+0000 7faf96000700 1 -- 172.21.15.151:0/1461049151 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7faf84003ea0 con 0x7faf981041a0 2024-05-21T22:00:35.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.456+0000 7fafaa7e0b80 1 Processor -- start 2024-05-21T22:00:35.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.456+0000 7fafaa7e0b80 1 -- start start 2024-05-21T22:00:35.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.456+0000 7fafaa7e0b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5569dae99610 0x5569dae999f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:35.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.456+0000 7faf96000700 1 --2- 172.21.15.151:0/1461049151 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7faf800b29f0 0x7faf800b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:35.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.456+0000 7fafaa7e0b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5569dae99f30 0x5569dae93b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:35.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.456+0000 7fafaa7e0b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569dae94140 0x5569dae945c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:35.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.456+0000 7fafaa7e0b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5569dadf1e40 con 0x5569dae94140 2024-05-21T22:00:35.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.456+0000 7fafaa7e0b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5569dade19e0 con 0x5569dae99610 2024-05-21T22:00:35.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.456+0000 7fafaa7e0b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5569dade5a20 con 0x5569dae99f30 2024-05-21T22:00:35.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.457+0000 7faf96000700 1 -- 172.21.15.151:0/1461049151 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7faf840e2910 con 0x7faf981041a0 2024-05-21T22:00:35.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.457+0000 7faf9de00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5569dae99610 0x5569dae999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:35.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.457+0000 7faf9ca00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5569dae99f30 0x5569dae93b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:35.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.457+0000 7faf9ca00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5569dae99f30 0x5569dae93b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:38226/0 (socket says 172.21.15.151:38226) 2024-05-21T22:00:35.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.457+0000 7faf9d400700 1 --2- 172.21.15.151:0/1461049151 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7faf800b29f0 0x7faf800b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:35.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.457+0000 7faf9ca00700 1 -- 172.21.15.151:0/538921241 learned_addr learned my addr 172.21.15.151:0/538921241 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:00:35.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.457+0000 7faf9ca00700 1 -- 172.21.15.151:0/538921241 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5569dae99610 msgr2=0x5569dae999f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:35.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.457+0000 7faf9ca00700 1 --2- 172.21.15.151:0/538921241 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5569dae99610 0x5569dae999f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:35.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.457+0000 7faf9ca00700 1 -- 172.21.15.151:0/538921241 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569dae94140 msgr2=0x5569dae945c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:35.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.457+0000 7faf9ca00700 1 --2- 172.21.15.151:0/538921241 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569dae94140 0x5569dae945c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:35.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.457+0000 7faf9ca00700 1 -- 172.21.15.151:0/538921241 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569dadaa420 con 0x5569dae99f30 2024-05-21T22:00:35.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.458+0000 7faf9ca00700 1 --2- 172.21.15.151:0/538921241 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5569dae99f30 0x5569dae93b20 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7faf84009fd0 tx=0x7faf840a5b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0c88cd1ba0bb22d server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:00:35.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.458+0000 7faf8aa00700 1 -- 172.21.15.151:0/538921241 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf840aa2a0 con 0x5569dae99f30 2024-05-21T22:00:35.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.458+0000 7faf8aa00700 1 -- 172.21.15.151:0/538921241 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf840cae50 con 0x5569dae99f30 2024-05-21T22:00:35.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.458+0000 7faf8aa00700 1 -- 172.21.15.151:0/538921241 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf840aaed0 con 0x5569dae99f30 2024-05-21T22:00:35.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.459+0000 7fafaa7e0b80 1 -- 172.21.15.151:0/538921241 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5569dae99f30 msgr2=0x5569dae93b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:35.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.459+0000 7faf9d400700 1 --2- 172.21.15.151:0/1461049151 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7faf800b29f0 0x7faf800b4eb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7faf8c003760 tx=0x7faf8c005fb0 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:00:35.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.459+0000 7fafaa7e0b80 1 --2- 172.21.15.151:0/538921241 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5569dae99f30 0x5569dae93b20 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7faf84009fd0 tx=0x7faf840a5b90 comp rx=0 tx=0).stop 2024-05-21T22:00:35.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.459+0000 7fafaa7e0b80 1 -- 172.21.15.151:0/538921241 shutdown_connections 2024-05-21T22:00:35.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.459+0000 7fafaa7e0b80 1 --2- 172.21.15.151:0/538921241 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5569dae99610 0x5569dae999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:35.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.459+0000 7fafaa7e0b80 1 --2- 172.21.15.151:0/538921241 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569dae94140 0x5569dae945c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:35.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.459+0000 7fafaa7e0b80 1 --2- 172.21.15.151:0/538921241 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5569dae99f30 0x5569dae93b20 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:35.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.459+0000 7fafaa7e0b80 1 -- 172.21.15.151:0/538921241 >> 172.21.15.151:0/538921241 conn(0x5569dadc1be0 msgr2=0x5569dae8e690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:00:35.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.459+0000 7fafaa7e0b80 1 -- 172.21.15.151:0/538921241 shutdown_connections 2024-05-21T22:00:35.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.459+0000 7fafaa7e0b80 1 -- 172.21.15.151:0/538921241 wait complete. 2024-05-21T22:00:35.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.459+0000 7fafaa7e0b80 1 -- start start 2024-05-21T22:00:35.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.460+0000 7fafaa7e0b80 1 Processor -- start 2024-05-21T22:00:35.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.460+0000 7fafaa7e0b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5569dae94140 0x5569dae9fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:35.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.460+0000 7fafaa7e0b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5569dae99610 0x5569dae9d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:35.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.460+0000 7fafaa7e0b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569dae99f30 0x5569dae9d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:35.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.460+0000 7fafaa7e0b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5569dace7fa0 con 0x5569dae99f30 2024-05-21T22:00:35.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.460+0000 7fafaa7e0b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5569dad82760 con 0x5569dae99610 2024-05-21T22:00:35.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.460+0000 7fafaa7e0b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5569daea09e0 con 0x5569dae94140 2024-05-21T22:00:35.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.460+0000 7fafaa7e0b80 10 client.0 _ll_register_callbacks cb 0x5569dad8bc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:00:35.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.460+0000 7faf9d400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5569dae94140 0x5569dae9fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:35.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.461+0000 7faf9de00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569dae99f30 0x5569dae9d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:35.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.461+0000 7faf9d400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5569dae94140 0x5569dae9fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:38230/0 (socket says 172.21.15.151:38230) 2024-05-21T22:00:35.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.461+0000 7faf9d400700 1 -- 172.21.15.151:0/1308831842 learned_addr learned my addr 172.21.15.151:0/1308831842 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:00:35.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.461+0000 7faf9ca00700 1 --2- 172.21.15.151:0/1308831842 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5569dae99610 0x5569dae9d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:35.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.461+0000 7faf9d400700 1 -- 172.21.15.151:0/1308831842 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5569dae99610 msgr2=0x5569dae9d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:35.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.461+0000 7faf9d400700 1 --2- 172.21.15.151:0/1308831842 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5569dae99610 0x5569dae9d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:35.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.461+0000 7faf9d400700 1 -- 172.21.15.151:0/1308831842 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569dae99f30 msgr2=0x5569dae9d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:35.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.461+0000 7faf9d400700 1 --2- 172.21.15.151:0/1308831842 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569dae99f30 0x5569dae9d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:35.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.461+0000 7faf9d400700 1 -- 172.21.15.151:0/1308831842 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569dad318d0 con 0x5569dae94140 2024-05-21T22:00:35.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.461+0000 7faf9d400700 1 --2- 172.21.15.151:0/1308831842 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5569dae94140 0x5569dae9fca0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7faf8c009750 tx=0x7faf8c009cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=891c175c82156e6c server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:00:35.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.461+0000 7faf7f400700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-21T22:00:35.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.462+0000 7faf7f400700 1 -- 172.21.15.151:0/1308831842 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf8c033060 con 0x5569dae94140 2024-05-21T22:00:35.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.462+0000 7faf7f400700 1 -- 172.21.15.151:0/1308831842 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7faf8c00bba0 con 0x5569dae94140 2024-05-21T22:00:35.467 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.462+0000 7fafaa7e0b80 1 -- 172.21.15.151:0/1308831842 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf840cae50 con 0x5569dae94140 2024-05-21T22:00:35.467 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.462+0000 7faf7f400700 1 -- 172.21.15.151:0/1308831842 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7faf8c00bda0 con 0x5569dae94140 2024-05-21T22:00:35.467 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.462+0000 7fafaa7e0b80 10 client.26374 fetch_fsmap learned FSMap version 29 2024-05-21T22:00:35.467 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.462+0000 7fafaa7e0b80 1 -- 172.21.15.151:0/1308831842 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5569dadb2060 con 0x5569dae94140 2024-05-21T22:00:35.467 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.462+0000 7faf7f400700 1 -- 172.21.15.151:0/1308831842 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf840cae50 con 0x5569dae94140 2024-05-21T22:00:35.467 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.462+0000 7faf7f400700 1 -- 172.21.15.151:0/1308831842 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7faf8c010470 con 0x5569dae94140 2024-05-21T22:00:35.467 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.463+0000 7fafaa7e0b80 10 client.26374 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:00:35.467 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.463+0000 7fafaa7e0b80 10 client.26374 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T22:00:35.467 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.463+0000 7fafaa7e0b80 20 client.26374 populate_metadata read hostname 'smithi151' 2024-05-21T22:00:35.467 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.463+0000 7fafaa7e0b80 1 --2- 172.21.15.151:0/1308831842 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5569daead550 0x5569daeaf940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:35.467 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.463+0000 7fafaa7e0b80 4 client.26374 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T22:00:35.467 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.463+0000 7fafaa7e0b80 1 -- 172.21.15.151:0/1308831842 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5569dadf3d90 con 0x5569daead550 2024-05-21T22:00:35.467 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.463+0000 7faf9ca00700 1 --2- 172.21.15.151:0/1308831842 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5569daead550 0x5569daeaf940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:35.467 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.464+0000 7faf9ca00700 1 --2- 172.21.15.151:0/1308831842 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5569daead550 0x5569daeaf940 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a50b4a367c64b98 server_cookie=b6476c13bcf641f2 in_seq=0 out_seq=0 2024-05-21T22:00:35.467 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.464+0000 7faf7f400700 10 client.26374 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T22:00:35.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.465+0000 7faf7f400700 1 -- 172.21.15.151:0/1308831842 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5569dadf3d90 con 0x5569daead550 2024-05-21T22:00:35.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.465+0000 7faf7f400700 10 client.26374 handle_command_reply: tid=0 2024-05-21T22:00:35.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.465+0000 7fafaa7e0b80 1 client.26374 shutdown 2024-05-21T22:00:35.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.465+0000 7fafaa7e0b80 1 -- 172.21.15.151:0/1308831842 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5569dae94140 msgr2=0x5569dae9fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:35.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.465+0000 7fafaa7e0b80 1 --2- 172.21.15.151:0/1308831842 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5569dae94140 0x5569dae9fca0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7faf8c009750 tx=0x7faf8c009cd0 comp rx=0 tx=0).stop 2024-05-21T22:00:35.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.465+0000 7fafaa7e0b80 1 -- 172.21.15.151:0/1308831842 shutdown_connections 2024-05-21T22:00:35.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.465+0000 7fafaa7e0b80 1 --2- 172.21.15.151:0/1308831842 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569dae99f30 0x5569dae9d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:35.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.465+0000 7fafaa7e0b80 1 --2- 172.21.15.151:0/1308831842 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5569dae94140 0x5569dae9fca0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:35.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.465+0000 7fafaa7e0b80 1 --2- 172.21.15.151:0/1308831842 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5569daead550 0x5569daeaf940 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:35.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.465+0000 7fafaa7e0b80 1 --2- 172.21.15.151:0/1308831842 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5569dae99610 0x5569dae9d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:35.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.465+0000 7fafaa7e0b80 1 -- 172.21.15.151:0/1308831842 >> 172.21.15.151:0/1308831842 conn(0x5569dadc1be0 msgr2=0x5569dae8dd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:00:35.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.465+0000 7faf7f400700 0 client.26374 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T22:00:35.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.465+0000 7fafaa7e0b80 1 -- 172.21.15.151:0/1308831842 shutdown_connections 2024-05-21T22:00:35.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.465+0000 7fafaa7e0b80 1 -- 172.21.15.151:0/1308831842 wait complete. 2024-05-21T22:00:35.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.465+0000 7fafaa7e0b80 20 client.26374 trim_cache size 0 max 16384 2024-05-21T22:00:35.485 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:00:35 smithi184 ceph-mon[36942]: scrub summary: idle 2024-05-21T22:00:35.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:00:35 smithi184 ceph-mon[36942]: pgmap v152: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 0 B/s wr, 8 op/s 2024-05-21T22:00:35.488 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.488+0000 7fafaa7e0b80 1 Processor -- start 2024-05-21T22:00:35.488 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.488+0000 7fafaa7e0b80 1 -- start start 2024-05-21T22:00:35.488 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.488+0000 7fafaa7e0b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5569dae94140 0x5569dae93b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:35.489 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.488+0000 7fafaa7e0b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5569dae99610 0x5569dae9bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:35.489 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.488+0000 7fafaa7e0b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569dae99f30 0x5569dae9abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:35.489 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.488+0000 7fafaa7e0b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7faf981b0000 con 0x5569dae99f30 2024-05-21T22:00:35.489 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.488+0000 7fafaa7e0b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7faf981b1b40 con 0x5569dae94140 2024-05-21T22:00:35.489 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.488+0000 7fafaa7e0b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7faf981b06d0 con 0x5569dae99610 2024-05-21T22:00:35.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.489+0000 7faf9d400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5569dae94140 0x5569dae93b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:35.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.489+0000 7faf9ca00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5569dae99610 0x5569dae9bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:35.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.489+0000 7faf9de00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569dae99f30 0x5569dae9abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:35.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.489+0000 7faf9d400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5569dae94140 0x5569dae93b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:36296/0 (socket says 172.21.15.151:36296) 2024-05-21T22:00:35.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.489+0000 7faf9de00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569dae99f30 0x5569dae9abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:45704/0 (socket says 172.21.15.151:45704) 2024-05-21T22:00:35.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.489+0000 7faf9ca00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5569dae99610 0x5569dae9bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:38238/0 (socket says 172.21.15.151:38238) 2024-05-21T22:00:35.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.489+0000 7faf9d400700 1 -- 172.21.15.151:0/300426292 learned_addr learned my addr 172.21.15.151:0/300426292 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:00:35.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.489+0000 7faf9ca00700 1 -- 172.21.15.151:0/300426292 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5569dae94140 msgr2=0x5569dae93b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:35.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.489+0000 7faf9ca00700 1 --2- 172.21.15.151:0/300426292 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5569dae94140 0x5569dae93b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:35.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.489+0000 7faf9ca00700 1 -- 172.21.15.151:0/300426292 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569dae99f30 msgr2=0x5569dae9abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:35.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.489+0000 7faf9ca00700 1 --2- 172.21.15.151:0/300426292 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569dae99f30 0x5569dae9abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:35.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.489+0000 7faf9ca00700 1 -- 172.21.15.151:0/300426292 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569daeaffc0 con 0x5569dae99610 2024-05-21T22:00:35.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.490+0000 7faf9ca00700 1 --2- 172.21.15.151:0/300426292 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5569dae99610 0x5569dae9bff0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x5569daea0710 tx=0x7faf84003940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a974384b7480761 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:00:35.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.490+0000 7faf8a000700 1 -- 172.21.15.151:0/300426292 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf840cae50 con 0x5569dae99610 2024-05-21T22:00:35.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.490+0000 7faf8a000700 1 -- 172.21.15.151:0/300426292 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf840aa2a0 con 0x5569dae99610 2024-05-21T22:00:35.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.490+0000 7faf8a000700 1 -- 172.21.15.151:0/300426292 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf840e0290 con 0x5569dae99610 2024-05-21T22:00:35.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.490+0000 7fafaa7e0b80 1 -- 172.21.15.151:0/300426292 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5569dae99610 msgr2=0x5569dae9bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:35.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.490+0000 7fafaa7e0b80 1 --2- 172.21.15.151:0/300426292 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5569dae99610 0x5569dae9bff0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x5569daea0710 tx=0x7faf84003940 comp rx=0 tx=0).stop 2024-05-21T22:00:35.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.491+0000 7fafaa7e0b80 1 -- 172.21.15.151:0/300426292 shutdown_connections 2024-05-21T22:00:35.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.491+0000 7fafaa7e0b80 1 --2- 172.21.15.151:0/300426292 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5569dae94140 0x5569dae93b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:35.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.491+0000 7fafaa7e0b80 1 --2- 172.21.15.151:0/300426292 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569dae99f30 0x5569dae9abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:35.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.491+0000 7fafaa7e0b80 1 --2- 172.21.15.151:0/300426292 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5569dae99610 0x5569dae9bff0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:35.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.491+0000 7fafaa7e0b80 1 -- 172.21.15.151:0/300426292 >> 172.21.15.151:0/300426292 conn(0x5569dadc1be0 msgr2=0x5569dae98180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:00:35.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.491+0000 7fafaa7e0b80 1 -- 172.21.15.151:0/300426292 shutdown_connections 2024-05-21T22:00:35.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.491+0000 7fafaa7e0b80 1 -- 172.21.15.151:0/300426292 wait complete. 2024-05-21T22:00:35.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.491+0000 7fafaa7e0b80 1 -- start start 2024-05-21T22:00:35.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.491+0000 7fafaa7e0b80 1 Processor -- start 2024-05-21T22:00:35.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.492+0000 7fafaa7e0b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5569dae94140 0x5569daea5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:35.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.492+0000 7fafaa7e0b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5569dae99610 0x5569daea6470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:35.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.492+0000 7fafaa7e0b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569dae99f30 0x5569daea2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:35.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.492+0000 7fafaa7e0b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7faf981b1470 con 0x5569dae99f30 2024-05-21T22:00:35.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.492+0000 7fafaa7e0b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7faf981b0da0 con 0x5569dae94140 2024-05-21T22:00:35.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.492+0000 7fafaa7e0b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7faf981adfa0 con 0x5569dae99610 2024-05-21T22:00:35.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.492+0000 7fafaa7e0b80 10 client.0 _ll_register_callbacks cb 0x5569dad8bc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:00:35.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.492+0000 7faf9ca00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5569dae99610 0x5569daea6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:35.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.492+0000 7faf9d400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5569dae94140 0x5569daea5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:35.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.492+0000 7faf9de00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569dae99f30 0x5569daea2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:35.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.492+0000 7faf9ca00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5569dae99610 0x5569daea6470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:38242/0 (socket says 172.21.15.151:38242) 2024-05-21T22:00:35.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.492+0000 7faf9ca00700 1 -- 172.21.15.151:0/3974692854 learned_addr learned my addr 172.21.15.151:0/3974692854 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:00:35.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.492+0000 7faf9d400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5569dae94140 0x5569daea5f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:36304/0 (socket says 172.21.15.151:36304) 2024-05-21T22:00:35.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.493+0000 7faf9ca00700 1 -- 172.21.15.151:0/3974692854 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5569dae94140 msgr2=0x5569daea5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:35.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.493+0000 7faf9ca00700 1 --2- 172.21.15.151:0/3974692854 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5569dae94140 0x5569daea5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:35.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.493+0000 7faf9ca00700 1 -- 172.21.15.151:0/3974692854 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569dae99f30 msgr2=0x5569daea2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:35.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.493+0000 7faf9ca00700 1 --2- 172.21.15.151:0/3974692854 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569dae99f30 0x5569daea2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:35.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.493+0000 7faf9ca00700 1 -- 172.21.15.151:0/3974692854 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569daea3540 con 0x5569dae99610 2024-05-21T22:00:35.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.493+0000 7faf9ca00700 1 --2- 172.21.15.151:0/3974692854 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5569dae99610 0x5569daea6470 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7faf84000c00 tx=0x7faf84005b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62e838496bf19cb server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:00:35.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.493+0000 7faf7f400700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-21T22:00:35.495 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.493+0000 7faf7f400700 1 -- 172.21.15.151:0/3974692854 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf840adf50 con 0x5569dae99610 2024-05-21T22:00:35.495 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.493+0000 7faf7f400700 1 -- 172.21.15.151:0/3974692854 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7faf840a6190 con 0x5569dae99610 2024-05-21T22:00:35.495 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.493+0000 7fafaa7e0b80 1 -- 172.21.15.151:0/3974692854 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf840aa2a0 con 0x5569dae99610 2024-05-21T22:00:35.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.493+0000 7faf7f400700 1 -- 172.21.15.151:0/3974692854 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7faf840b2a00 con 0x5569dae99610 2024-05-21T22:00:35.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.494+0000 7fafaa7e0b80 10 client.26380 fetch_fsmap learned FSMap version 29 2024-05-21T22:00:35.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.494+0000 7fafaa7e0b80 1 -- 172.21.15.151:0/3974692854 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5569daf18d00 con 0x5569dae99610 2024-05-21T22:00:35.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.494+0000 7faf7f400700 1 -- 172.21.15.151:0/3974692854 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf840aa2a0 con 0x5569dae99610 2024-05-21T22:00:35.497 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.494+0000 7faf7f400700 1 -- 172.21.15.151:0/3974692854 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7faf840a9f00 con 0x5569dae99610 2024-05-21T22:00:35.497 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.494+0000 7fafaa7e0b80 10 client.26380 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:00:35.497 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.494+0000 7fafaa7e0b80 10 client.26380 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T22:00:35.497 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.494+0000 7fafaa7e0b80 20 client.26380 populate_metadata read hostname 'smithi151' 2024-05-21T22:00:35.497 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.494+0000 7fafaa7e0b80 1 --2- 172.21.15.151:0/3974692854 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5569daf18f70 0x5569daf1b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:35.497 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.494+0000 7fafaa7e0b80 4 client.26380 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-21T22:00:35.497 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.494+0000 7fafaa7e0b80 1 -- 172.21.15.151:0/3974692854 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5569daded570 con 0x5569daf18f70 2024-05-21T22:00:35.497 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.495+0000 7faf9d400700 1 --2- 172.21.15.151:0/3974692854 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5569daf18f70 0x5569daf1b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:35.497 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.495+0000 7faf9d400700 1 --2- 172.21.15.151:0/3974692854 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5569daf18f70 0x5569daf1b360 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdbfe13009b6bff1 server_cookie=d03eca4153a96188 in_seq=0 out_seq=0 2024-05-21T22:00:35.497 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.495+0000 7faf7f400700 10 client.26380 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T22:00:35.497 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.495+0000 7faf7f400700 1 -- 172.21.15.151:0/3974692854 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5569daded570 con 0x5569daf18f70 2024-05-21T22:00:35.497 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.495+0000 7faf7f400700 10 client.26380 handle_command_reply: tid=0 2024-05-21T22:00:35.497 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.495+0000 7fafaa7e0b80 1 client.26380 shutdown 2024-05-21T22:00:35.497 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.496+0000 7fafaa7e0b80 1 -- 172.21.15.151:0/3974692854 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5569dae99610 msgr2=0x5569daea6470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:35.498 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.496+0000 7fafaa7e0b80 1 --2- 172.21.15.151:0/3974692854 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5569dae99610 0x5569daea6470 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7faf84000c00 tx=0x7faf84005b40 comp rx=0 tx=0).stop 2024-05-21T22:00:35.498 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.496+0000 7fafaa7e0b80 1 -- 172.21.15.151:0/3974692854 shutdown_connections 2024-05-21T22:00:35.498 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.496+0000 7fafaa7e0b80 1 --2- 172.21.15.151:0/3974692854 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5569daf18f70 0x5569daf1b360 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:35.498 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.496+0000 7fafaa7e0b80 1 --2- 172.21.15.151:0/3974692854 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5569dae94140 0x5569daea5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:35.498 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.496+0000 7fafaa7e0b80 1 --2- 172.21.15.151:0/3974692854 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5569dae99f30 0x5569daea2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:35.498 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.496+0000 7fafaa7e0b80 1 --2- 172.21.15.151:0/3974692854 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5569dae99610 0x5569daea6470 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:35.498 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.496+0000 7faf7f400700 0 client.26380 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T22:00:35.498 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.496+0000 7fafaa7e0b80 1 -- 172.21.15.151:0/3974692854 >> 172.21.15.151:0/3974692854 conn(0x5569dadc1be0 msgr2=0x5569dae98180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:00:35.498 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.496+0000 7fafaa7e0b80 1 -- 172.21.15.151:0/3974692854 shutdown_connections 2024-05-21T22:00:35.498 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.496+0000 7fafaa7e0b80 1 -- 172.21.15.151:0/3974692854 wait complete. 2024-05-21T22:00:35.498 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.496+0000 7fafaa7e0b80 20 client.26380 trim_cache size 0 max 16384 2024-05-21T22:00:35.499 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-05-21T22:00:35.499 INFO:teuthology.orchestra.run.smithi151.stdout: "status": "no active scrubs running", 2024-05-21T22:00:35.499 INFO:teuthology.orchestra.run.smithi151.stdout: "scrubs": {} 2024-05-21T22:00:35.499 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-05-21T22:00:35.499 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.497+0000 7faf9fe00700 1 -- 172.21.15.151:0/1461049151 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7faf800b29f0 msgr2=0x7faf800b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:35.499 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.497+0000 7faf9fe00700 1 --2- 172.21.15.151:0/1461049151 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7faf800b29f0 0x7faf800b4eb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7faf8c003760 tx=0x7faf8c005fb0 comp rx=0 tx=0).stop 2024-05-21T22:00:35.500 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.497+0000 7faf9fe00700 1 -- 172.21.15.151:0/1461049151 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf981041a0 msgr2=0x7faf98075d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:35.500 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.497+0000 7faf9fe00700 1 --2- 172.21.15.151:0/1461049151 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf981041a0 0x7faf98075d80 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7faf8400eb30 tx=0x7faf8400ee40 comp rx=0 tx=0).stop 2024-05-21T22:00:35.500 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.497+0000 7faf9fe00700 1 -- 172.21.15.151:0/1461049151 shutdown_connections 2024-05-21T22:00:35.500 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.497+0000 7faf9fe00700 1 --2- 172.21.15.151:0/1461049151 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7faf800b29f0 0x7faf800b4eb0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:35.500 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.497+0000 7faf9fe00700 1 --2- 172.21.15.151:0/1461049151 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf981088f0 0x7faf980762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:35.500 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.497+0000 7faf9fe00700 1 --2- 172.21.15.151:0/1461049151 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faf981037f0 0x7faf98075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:35.500 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.497+0000 7faf9fe00700 1 --2- 172.21.15.151:0/1461049151 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf981041a0 0x7faf98075d80 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:35.500 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.497+0000 7faf9fe00700 1 -- 172.21.15.151:0/1461049151 >> 172.21.15.151:0/1461049151 conn(0x7faf9807a6a0 msgr2=0x7faf98109a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:00:35.500 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.497+0000 7faf9fe00700 1 -- 172.21.15.151:0/1461049151 shutdown_connections 2024-05-21T22:00:35.500 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:35.497+0000 7faf9fe00700 1 -- 172.21.15.151:0/1461049151 wait complete. 2024-05-21T22:00:35.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:00:35 smithi164 ceph-mon[36269]: scrub summary: idle 2024-05-21T22:00:35.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:00:35 smithi164 ceph-mon[36269]: pgmap v152: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 0 B/s wr, 8 op/s 2024-05-21T22:00:36.325 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-21T22:00:36.325 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-21T22:00:36.781 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:00:37.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:00:37 smithi151 ceph-mon[31623]: pgmap v153: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 0 B/s wr, 8 op/s 2024-05-21T22:00:37.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:00:37 smithi184 ceph-mon[36942]: pgmap v153: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 0 B/s wr, 8 op/s 2024-05-21T22:00:37.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:00:37 smithi164 ceph-mon[36269]: pgmap v153: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 0 B/s wr, 8 op/s 2024-05-21T22:00:38.917 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:38.916+0000 7f63c7e00700 1 -- 172.21.15.151:0/718545990 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f63c0076090 msgr2=0x7f63c010fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:38.917 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:38.916+0000 7f63c7e00700 1 --2- 172.21.15.151:0/718545990 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f63c0076090 0x7f63c010fa00 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f63ac0098f0 tx=0x7f63ac009c00 comp rx=0 tx=0).stop 2024-05-21T22:00:38.917 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:38.916+0000 7f63c7e00700 1 -- 172.21.15.151:0/718545990 shutdown_connections 2024-05-21T22:00:38.917 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:38.916+0000 7f63c7e00700 1 --2- 172.21.15.151:0/718545990 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f63c010ff40 0x7f63c0112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:38.917 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:38.916+0000 7f63c7e00700 1 --2- 172.21.15.151:0/718545990 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f63c0075770 0x7f63c0075b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:38.917 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:38.916+0000 7f63c7e00700 1 --2- 172.21.15.151:0/718545990 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f63c0076090 0x7f63c010fa00 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:38.918 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:38.916+0000 7f63c7e00700 1 -- 172.21.15.151:0/718545990 >> 172.21.15.151:0/718545990 conn(0x7f63c007a6a0 msgr2=0x7f63c007aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:00:38.918 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:38.916+0000 7f63c7e00700 1 -- 172.21.15.151:0/718545990 shutdown_connections 2024-05-21T22:00:38.918 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:38.916+0000 7f63c7e00700 1 -- 172.21.15.151:0/718545990 wait complete. 2024-05-21T22:00:38.919 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:38.917+0000 7f63c7e00700 1 Processor -- start 2024-05-21T22:00:38.919 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:38.918+0000 7f63c7e00700 1 -- start start 2024-05-21T22:00:38.919 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:38.918+0000 7f63c7e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f63c0075770 0x7f63c019bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:38.919 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:38.918+0000 7f63c7e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f63c0076090 0x7f63c019c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:38.919 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:38.918+0000 7f63c7e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f63c010ff40 0x7f63c01a0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:38.919 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:38.918+0000 7f63c7e00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f63c0115950 con 0x7f63c0075770 2024-05-21T22:00:38.919 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:38.918+0000 7f63c7e00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f63c01157d0 con 0x7f63c010ff40 2024-05-21T22:00:38.919 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:38.918+0000 7f63c7e00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f63c0115ad0 con 0x7f63c0076090 2024-05-21T22:00:38.919 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:38.918+0000 7f63c5400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f63c0075770 0x7f63c019bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:38.920 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:38.918+0000 7f63c4a00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f63c0076090 0x7f63c019c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:38.920 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:38.919+0000 7f63c5400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f63c0075770 0x7f63c019bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:47200/0 (socket says 172.21.15.151:47200) 2024-05-21T22:00:38.920 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:38.919+0000 7f63c5400700 1 -- 172.21.15.151:0/2140315065 learned_addr learned my addr 172.21.15.151:0/2140315065 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:00:38.920 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:38.919+0000 7f63c5e00700 1 --2- 172.21.15.151:0/2140315065 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f63c010ff40 0x7f63c01a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:38.920 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:38.919+0000 7f63c4a00700 1 -- 172.21.15.151:0/2140315065 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f63c010ff40 msgr2=0x7f63c01a0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:38.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:38.919+0000 7f63c4a00700 1 --2- 172.21.15.151:0/2140315065 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f63c010ff40 0x7f63c01a0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:38.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:38.919+0000 7f63c4a00700 1 -- 172.21.15.151:0/2140315065 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f63c0075770 msgr2=0x7f63c019bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:38.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:38.919+0000 7f63c4a00700 1 --2- 172.21.15.151:0/2140315065 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f63c0075770 0x7f63c019bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:38.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:38.919+0000 7f63c4a00700 1 -- 172.21.15.151:0/2140315065 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63c01a0eb0 con 0x7f63c0076090 2024-05-21T22:00:38.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:38.919+0000 7f63c5400700 1 --2- 172.21.15.151:0/2140315065 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f63c0075770 0x7f63c019bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:00:38.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:38.919+0000 7f63c4a00700 1 --2- 172.21.15.151:0/2140315065 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f63c0076090 0x7f63c019c3e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f63ac000c00 tx=0x7f63ac00fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:00:38.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:38.920+0000 7f63be000700 1 -- 172.21.15.151:0/2140315065 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63ac0050c0 con 0x7f63c0076090 2024-05-21T22:00:38.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:38.920+0000 7f63be000700 1 -- 172.21.15.151:0/2140315065 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63ac00b560 con 0x7f63c0076090 2024-05-21T22:00:38.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:38.920+0000 7f63c7e00700 1 -- 172.21.15.151:0/2140315065 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63c01a1140 con 0x7f63c0076090 2024-05-21T22:00:38.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:38.920+0000 7f63be000700 1 -- 172.21.15.151:0/2140315065 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63ac003a00 con 0x7f63c0076090 2024-05-21T22:00:38.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:38.920+0000 7f63c7e00700 1 -- 172.21.15.151:0/2140315065 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63c01a8a20 con 0x7f63c0076090 2024-05-21T22:00:38.922 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:38.921+0000 7f63c7e00700 1 -- 172.21.15.151:0/2140315065 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63c0076e60 con 0x7f63c0076090 2024-05-21T22:00:38.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:38.923+0000 7f63be000700 1 -- 172.21.15.151:0/2140315065 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7f63ac00b9c0 con 0x7f63c0076090 2024-05-21T22:00:38.925 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:38.925+0000 7f63be000700 1 --2- 172.21.15.151:0/2140315065 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f63a80b2a70 0x7f63a80b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:38.925 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:38.925+0000 7f63be000700 1 -- 172.21.15.151:0/2140315065 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f63ac0e3900 con 0x7f63c0076090 2024-05-21T22:00:38.925 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:38.925+0000 7f63c5400700 1 --2- 172.21.15.151:0/2140315065 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f63a80b2a70 0x7f63a80b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:38.925 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:38.926+0000 7f63c5400700 1 --2- 172.21.15.151:0/2140315065 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f63a80b2a70 0x7f63a80b4f30 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f63b4011550 tx=0x7f63b4009260 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:00:38.927 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:38.927+0000 7f63be000700 1 -- 172.21.15.151:0/2140315065 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f63ac013030 con 0x7f63c0076090 2024-05-21T22:00:39.207 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:39.207+0000 7f63c7e00700 1 -- 172.21.15.151:0/2140315065 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f63c019d200 con 0x7f63c0076090 2024-05-21T22:00:39.209 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T22:00:39.209 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14358,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/158010529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":158010529},{"type":"v1","addr":"172.21.15.151:6835","nonce":158010529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6839/2069058115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":2069058115},{"type":"v1","addr":"172.21.15.151:6839","nonce":2069058115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T21:51:19.552069+0000","modified":"2024-05-21T21:55:34.643755+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24530,"mds_1":24493,"mds_2":24491,"mds_3":24478,"mds_4":24473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14403":{"gid":14403,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6841/1749723794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6840","nonce":1749723794},{"type":"v1","addr":"172.21.15.151:6841","nonce":1749723794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/2026266114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":2026266114},{"type":"v1","addr":"172.21.15.184:6833","nonce":2026266114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/2467900844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":2467900844},{"type":"v1","addr":"172.21.15.164:6833","nonce":2467900844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3367640960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3367640960},{"type":"v1","addr":"172.21.15.151:6837","nonce":3367640960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/3264142621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3264142621},{"type":"v1","addr":"172.21.15.184:6835","nonce":3264142621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":17,"addr":"172.21.15.164:6835/2340428470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2340428470},{"type":"v1","addr":"172.21.15.164:6835","nonce":2340428470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":16,"addr":"172.21.15.184:6837/2988194888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":2988194888},{"type":"v1","addr":"172.21.15.184:6837","nonce":2988194888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.164:6837/1554609572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1554609572},{"type":"v1","addr":"172.21.15.164:6837","nonce":1554609572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.184:6839/3059935143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":3059935143},{"type":"v1","addr":"172.21.15.184:6839","nonce":3059935143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.164:6839/2646401247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":2646401247},{"type":"v1","addr":"172.21.15.164:6839","nonce":2646401247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24530,"qdb_cluster":[24530,24493,24491,24473,24478]},"id":1}]} 2024-05-21T22:00:39.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:39.208+0000 7f63be000700 1 -- 172.21.15.151:0/2140315065 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11958 (secure 0 0 0) 0x7f63ac0ab320 con 0x7f63c0076090 2024-05-21T22:00:39.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:39.211+0000 7f63c7e00700 1 -- 172.21.15.151:0/2140315065 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f63a80b2a70 msgr2=0x7f63a80b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:39.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:39.211+0000 7f63c7e00700 1 --2- 172.21.15.151:0/2140315065 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f63a80b2a70 0x7f63a80b4f30 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f63b4011550 tx=0x7f63b4009260 comp rx=0 tx=0).stop 2024-05-21T22:00:39.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:39.211+0000 7f63c7e00700 1 -- 172.21.15.151:0/2140315065 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f63c0076090 msgr2=0x7f63c019c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:39.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:39.211+0000 7f63c7e00700 1 --2- 172.21.15.151:0/2140315065 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f63c0076090 0x7f63c019c3e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f63ac000c00 tx=0x7f63ac00fa00 comp rx=0 tx=0).stop 2024-05-21T22:00:39.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:39.212+0000 7f63c7e00700 1 -- 172.21.15.151:0/2140315065 shutdown_connections 2024-05-21T22:00:39.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:39.212+0000 7f63c7e00700 1 --2- 172.21.15.151:0/2140315065 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f63a80b2a70 0x7f63a80b4f30 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:39.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:39.212+0000 7f63c7e00700 1 --2- 172.21.15.151:0/2140315065 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f63c010ff40 0x7f63c01a0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:39.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:39.212+0000 7f63c7e00700 1 --2- 172.21.15.151:0/2140315065 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f63c0075770 0x7f63c019bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:39.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:39.212+0000 7f63c7e00700 1 --2- 172.21.15.151:0/2140315065 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f63c0076090 0x7f63c019c3e0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:39.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:39.212+0000 7f63c7e00700 1 -- 172.21.15.151:0/2140315065 >> 172.21.15.151:0/2140315065 conn(0x7f63c007a6a0 msgr2=0x7f63c0100850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:00:39.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:39.212+0000 7f63c7e00700 1 -- 172.21.15.151:0/2140315065 shutdown_connections 2024-05-21T22:00:39.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:39.212+0000 7f63c7e00700 1 -- 172.21.15.151:0/2140315065 wait complete. 2024-05-21T22:00:39.214 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 29 2024-05-21T22:00:39.578 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:00:39 smithi151 ceph-mon[31623]: pgmap v154: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 0 B/s wr, 12 op/s 2024-05-21T22:00:39.578 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:00:39 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/2140315065' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T22:00:39.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:00:39 smithi184 ceph-mon[36942]: pgmap v154: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 0 B/s wr, 12 op/s 2024-05-21T22:00:39.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:00:39 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/2140315065' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T22:00:39.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:00:39 smithi164 ceph-mon[36269]: pgmap v154: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 0 B/s wr, 12 op/s 2024-05-21T22:00:39.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:00:39 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/2140315065' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T22:00:40.110 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-21T22:00:40.572 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:00:41.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:00:41 smithi151 ceph-mon[31623]: pgmap v155: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 0 B/s wr, 8 op/s 2024-05-21T22:00:41.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:00:41 smithi184 ceph-mon[36942]: pgmap v155: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 0 B/s wr, 8 op/s 2024-05-21T22:00:41.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:00:41 smithi164 ceph-mon[36269]: pgmap v155: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 0 B/s wr, 8 op/s 2024-05-21T22:00:42.843 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.842+0000 7f04b3400700 1 -- 172.21.15.151:0/294018412 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f04ac06b220 msgr2=0x7f04ac06b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:42.843 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.842+0000 7f04b3400700 1 --2- 172.21.15.151:0/294018412 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f04ac06b220 0x7f04ac06b6a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f0498009990 tx=0x7f0498009ca0 comp rx=0 tx=0).stop 2024-05-21T22:00:42.843 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.843+0000 7f04b3400700 1 -- 172.21.15.151:0/294018412 shutdown_connections 2024-05-21T22:00:42.844 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.843+0000 7f04b3400700 1 --2- 172.21.15.151:0/294018412 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f04ac06bbe0 0x7f04ac1103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:42.844 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.843+0000 7f04b3400700 1 --2- 172.21.15.151:0/294018412 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04ac1095b0 0x7f04ac109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:42.844 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.843+0000 7f04b3400700 1 --2- 172.21.15.151:0/294018412 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f04ac06b220 0x7f04ac06b6a0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:42.844 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.843+0000 7f04b3400700 1 -- 172.21.15.151:0/294018412 >> 172.21.15.151:0/294018412 conn(0x7f04ac0793f0 msgr2=0x7f04ac079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:00:42.844 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.843+0000 7f04b3400700 1 -- 172.21.15.151:0/294018412 shutdown_connections 2024-05-21T22:00:42.844 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.843+0000 7f04b3400700 1 -- 172.21.15.151:0/294018412 wait complete. 2024-05-21T22:00:42.845 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.845+0000 7f04b3400700 1 Processor -- start 2024-05-21T22:00:42.845 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.845+0000 7f04b3400700 1 -- start start 2024-05-21T22:00:42.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.845+0000 7f04b3400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04ac06b220 0x7f04ac19f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:42.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.846+0000 7f04b3400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f04ac06bbe0 0x7f04ac19fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:42.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.846+0000 7f04b3400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f04ac1095b0 0x7f04ac199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:42.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.846+0000 7f04b3400700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f04ac113740 con 0x7f04ac06bbe0 2024-05-21T22:00:42.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.846+0000 7f04b3400700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f04ac1135c0 con 0x7f04ac1095b0 2024-05-21T22:00:42.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.846+0000 7f04b3400700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f04ac1138c0 con 0x7f04ac06b220 2024-05-21T22:00:42.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.846+0000 7f04abe00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f04ac06bbe0 0x7f04ac19fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:42.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.846+0000 7f04b0a00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04ac06b220 0x7f04ac19f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:42.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.846+0000 7f04abe00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f04ac06bbe0 0x7f04ac19fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:47220/0 (socket says 172.21.15.151:47220) 2024-05-21T22:00:42.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.846+0000 7f04abe00700 1 -- 172.21.15.151:0/2999820108 learned_addr learned my addr 172.21.15.151:0/2999820108 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:00:42.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.846+0000 7f04b1400700 1 --2- 172.21.15.151:0/2999820108 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f04ac1095b0 0x7f04ac199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:42.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.846+0000 7f04abe00700 1 -- 172.21.15.151:0/2999820108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04ac06b220 msgr2=0x7f04ac19f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:42.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.846+0000 7f04abe00700 1 --2- 172.21.15.151:0/2999820108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04ac06b220 0x7f04ac19f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:42.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.846+0000 7f04abe00700 1 -- 172.21.15.151:0/2999820108 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f04ac1095b0 msgr2=0x7f04ac199ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:42.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.846+0000 7f04abe00700 1 --2- 172.21.15.151:0/2999820108 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f04ac1095b0 0x7f04ac199ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:42.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.846+0000 7f04abe00700 1 -- 172.21.15.151:0/2999820108 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04ac19a240 con 0x7f04ac06bbe0 2024-05-21T22:00:42.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.847+0000 7f04b0a00700 1 --2- 172.21.15.151:0/2999820108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04ac06b220 0x7f04ac19f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:00:42.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.847+0000 7f04abe00700 1 --2- 172.21.15.151:0/2999820108 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f04ac06bbe0 0x7f04ac19fe60 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f0498000c00 tx=0x7f04980057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:00:42.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.847+0000 7f04a9600700 1 -- 172.21.15.151:0/2999820108 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0498027070 con 0x7f04ac06bbe0 2024-05-21T22:00:42.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.847+0000 7f04b3400700 1 -- 172.21.15.151:0/2999820108 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04ac19a4d0 con 0x7f04ac06bbe0 2024-05-21T22:00:42.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.847+0000 7f04a9600700 1 -- 172.21.15.151:0/2999820108 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0498022470 con 0x7f04ac06bbe0 2024-05-21T22:00:42.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.847+0000 7f04a9600700 1 -- 172.21.15.151:0/2999820108 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0498017650 con 0x7f04ac06bbe0 2024-05-21T22:00:42.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.847+0000 7f04b3400700 1 -- 172.21.15.151:0/2999820108 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04ac1a67f0 con 0x7f04ac06bbe0 2024-05-21T22:00:42.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.851+0000 7f04a9600700 1 -- 172.21.15.151:0/2999820108 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7f049802a070 con 0x7f04ac06bbe0 2024-05-21T22:00:42.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.852+0000 7f04bde43b80 1 Processor -- start 2024-05-21T22:00:42.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.852+0000 7f04bde43b80 1 -- start start 2024-05-21T22:00:42.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.853+0000 7f04bde43b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558019946a20 0x558019946e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:42.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.853+0000 7f04bde43b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558019947340 0x558019940f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:42.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.853+0000 7f04bde43b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558019941550 0x5580199419d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:42.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.853+0000 7f04bde43b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55801989ae40 con 0x558019946a20 2024-05-21T22:00:42.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.853+0000 7f04bde43b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55801988a9e0 con 0x558019941550 2024-05-21T22:00:42.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.853+0000 7f04bde43b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55801988ea20 con 0x558019947340 2024-05-21T22:00:42.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.853+0000 7f04a9600700 1 --2- 172.21.15.151:0/2999820108 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f04940b29f0 0x7f04940b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:42.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.853+0000 7f04a9600700 1 -- 172.21.15.151:0/2999820108 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f04980e32f0 con 0x7f04ac06bbe0 2024-05-21T22:00:42.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.853+0000 7f04b1400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558019946a20 0x558019946e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:42.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.853+0000 7f04b0a00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558019947340 0x558019940f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:42.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.853+0000 7f04b1400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558019946a20 0x558019946e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:47226/0 (socket says 172.21.15.151:47226) 2024-05-21T22:00:42.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.853+0000 7f04b1400700 1 -- 172.21.15.151:0/3638945860 learned_addr learned my addr 172.21.15.151:0/3638945860 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:00:42.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.853+0000 7f04b1400700 1 --2- 172.21.15.151:0/2999820108 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f04940b29f0 0x7f04940b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:42.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.853+0000 7f04abe00700 1 --2- 172.21.15.151:0/3638945860 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558019941550 0x5580199419d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:42.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.854+0000 7f04b0a00700 1 -- 172.21.15.151:0/3638945860 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558019941550 msgr2=0x5580199419d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:42.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.854+0000 7f04b0a00700 1 --2- 172.21.15.151:0/3638945860 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558019941550 0x5580199419d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:42.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.854+0000 7f04b0a00700 1 -- 172.21.15.151:0/3638945860 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558019946a20 msgr2=0x558019946e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:42.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.854+0000 7f04b0a00700 1 --2- 172.21.15.151:0/3638945860 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558019946a20 0x558019946e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:42.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.854+0000 7f04b0a00700 1 -- 172.21.15.151:0/3638945860 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558019853420 con 0x558019947340 2024-05-21T22:00:42.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.854+0000 7f04b0a00700 1 --2- 172.21.15.151:0/3638945860 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558019947340 0x558019940f30 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f04a00114f0 tx=0x7f04a0009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1fde97ce1e06ef89 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:00:42.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.854+0000 7f04b1400700 1 --2- 172.21.15.151:0/2999820108 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f04940b29f0 0x7f04940b4eb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f04a4008d40 tx=0x7f04a4009100 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:00:42.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.854+0000 7f049e000700 1 -- 172.21.15.151:0/3638945860 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a0011230 con 0x558019947340 2024-05-21T22:00:42.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.855+0000 7f049e000700 1 -- 172.21.15.151:0/3638945860 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04a0003680 con 0x558019947340 2024-05-21T22:00:42.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.855+0000 7f049e000700 1 -- 172.21.15.151:0/3638945860 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a001a6e0 con 0x558019947340 2024-05-21T22:00:42.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.855+0000 7f04bde43b80 1 -- 172.21.15.151:0/3638945860 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558019947340 msgr2=0x558019940f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:42.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.855+0000 7f04bde43b80 1 --2- 172.21.15.151:0/3638945860 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558019947340 0x558019940f30 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f04a00114f0 tx=0x7f04a0009f90 comp rx=0 tx=0).stop 2024-05-21T22:00:42.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.856+0000 7f04bde43b80 1 -- 172.21.15.151:0/3638945860 shutdown_connections 2024-05-21T22:00:42.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.856+0000 7f04bde43b80 1 --2- 172.21.15.151:0/3638945860 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558019941550 0x5580199419d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:42.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.856+0000 7f04bde43b80 1 --2- 172.21.15.151:0/3638945860 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558019946a20 0x558019946e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:42.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.856+0000 7f04bde43b80 1 --2- 172.21.15.151:0/3638945860 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558019947340 0x558019940f30 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:42.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.856+0000 7f04bde43b80 1 -- 172.21.15.151:0/3638945860 >> 172.21.15.151:0/3638945860 conn(0x55801986abe0 msgr2=0x558019937690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:00:42.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.856+0000 7f04bde43b80 1 -- 172.21.15.151:0/3638945860 shutdown_connections 2024-05-21T22:00:42.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.856+0000 7f04bde43b80 1 -- 172.21.15.151:0/3638945860 wait complete. 2024-05-21T22:00:42.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.856+0000 7f04bde43b80 1 -- start start 2024-05-21T22:00:42.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.857+0000 7f04bde43b80 1 Processor -- start 2024-05-21T22:00:42.858 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.857+0000 7f04bde43b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558019941550 0x55801994d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:42.858 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.857+0000 7f04bde43b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558019946a20 0x55801994a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:42.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.857+0000 7f04bde43b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558019947340 0x55801994a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:42.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.857+0000 7f04bde43b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x558019790fa0 con 0x558019947340 2024-05-21T22:00:42.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.857+0000 7f04bde43b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55801982b760 con 0x558019946a20 2024-05-21T22:00:42.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.857+0000 7f04bde43b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55801994ddf0 con 0x558019941550 2024-05-21T22:00:42.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.857+0000 7f04bde43b80 10 client.0 _ll_register_callbacks cb 0x558019834c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:00:42.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.858+0000 7f04b0a00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558019941550 0x55801994d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:42.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.858+0000 7f04b1400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558019947340 0x55801994a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:42.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.858+0000 7f04b0a00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558019941550 0x55801994d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:57918/0 (socket says 172.21.15.151:57918) 2024-05-21T22:00:42.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.858+0000 7f04b0a00700 1 -- 172.21.15.151:0/2089847314 learned_addr learned my addr 172.21.15.151:0/2089847314 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:00:42.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.858+0000 7f04abe00700 1 --2- 172.21.15.151:0/2089847314 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558019946a20 0x55801994a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:42.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.858+0000 7f04b0a00700 1 -- 172.21.15.151:0/2089847314 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558019946a20 msgr2=0x55801994a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:42.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.858+0000 7f04b0a00700 1 --2- 172.21.15.151:0/2089847314 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558019946a20 0x55801994a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:42.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.858+0000 7f04b0a00700 1 -- 172.21.15.151:0/2089847314 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558019947340 msgr2=0x55801994a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:42.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.858+0000 7f04b0a00700 1 --2- 172.21.15.151:0/2089847314 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558019947340 0x55801994a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:42.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.858+0000 7f04b0a00700 1 -- 172.21.15.151:0/2089847314 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580197da8d0 con 0x558019941550 2024-05-21T22:00:42.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.858+0000 7f04b0a00700 1 --2- 172.21.15.151:0/2089847314 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558019941550 0x55801994d090 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f04a0014040 tx=0x7f04a00043f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed0145ce291a5cca server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:00:42.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.859+0000 7f0492a00700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-21T22:00:42.861 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.859+0000 7f0492a00700 1 -- 172.21.15.151:0/2089847314 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a0028060 con 0x558019941550 2024-05-21T22:00:42.861 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.859+0000 7f0492a00700 1 -- 172.21.15.151:0/2089847314 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f04a001a890 con 0x558019941550 2024-05-21T22:00:42.861 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.859+0000 7f04bde43b80 1 -- 172.21.15.151:0/2089847314 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04a0003680 con 0x558019941550 2024-05-21T22:00:42.861 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.859+0000 7f0492a00700 1 -- 172.21.15.151:0/2089847314 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f04a0005ce0 con 0x558019941550 2024-05-21T22:00:42.861 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.859+0000 7f04bde43b80 10 client.26398 fetch_fsmap learned FSMap version 29 2024-05-21T22:00:42.861 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.859+0000 7f04bde43b80 1 -- 172.21.15.151:0/2089847314 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55801985b060 con 0x558019941550 2024-05-21T22:00:42.861 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.859+0000 7f0492a00700 1 -- 172.21.15.151:0/2089847314 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04a0003680 con 0x558019941550 2024-05-21T22:00:42.861 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.860+0000 7f0492a00700 1 -- 172.21.15.151:0/2089847314 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f04a0022a60 con 0x558019941550 2024-05-21T22:00:42.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.860+0000 7f04bde43b80 10 client.26398 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:00:42.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.860+0000 7f04bde43b80 10 client.26398 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-05-21T22:00:42.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.860+0000 7f04bde43b80 20 client.26398 populate_metadata read hostname 'smithi151' 2024-05-21T22:00:42.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.860+0000 7f04bde43b80 1 --2- 172.21.15.151:0/2089847314 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x55801995a880 0x55801995cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:42.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.860+0000 7f04bde43b80 4 client.26398 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T22:00:42.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.860+0000 7f04bde43b80 1 -- 172.21.15.151:0/2089847314 --> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55801989cd90 con 0x55801995a880 2024-05-21T22:00:42.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.861+0000 7f04abe00700 1 --2- 172.21.15.151:0/2089847314 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x55801995a880 0x55801995cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:42.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.861+0000 7f04abe00700 1 --2- 172.21.15.151:0/2089847314 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x55801995a880 0x55801995cc70 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bda9e2b4c22073d server_cookie=4dbc3a6e4a50c5c7 in_seq=0 out_seq=0 2024-05-21T22:00:42.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.861+0000 7f0492a00700 10 client.26398 ms_handle_connect on v2:172.21.15.164:6834/2340428470 2024-05-21T22:00:42.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.862+0000 7f0492a00700 1 -- 172.21.15.151:0/2089847314 <== mds.4 v2:172.21.15.164:6834/2340428470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55801989cd90 con 0x55801995a880 2024-05-21T22:00:42.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.862+0000 7f0492a00700 10 client.26398 handle_command_reply: tid=0 2024-05-21T22:00:42.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.862+0000 7f04bde43b80 1 client.26398 shutdown 2024-05-21T22:00:42.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.862+0000 7f04bde43b80 1 -- 172.21.15.151:0/2089847314 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558019941550 msgr2=0x55801994d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:42.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.862+0000 7f04bde43b80 1 --2- 172.21.15.151:0/2089847314 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558019941550 0x55801994d090 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f04a0014040 tx=0x7f04a00043f0 comp rx=0 tx=0).stop 2024-05-21T22:00:42.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.862+0000 7f04bde43b80 1 -- 172.21.15.151:0/2089847314 shutdown_connections 2024-05-21T22:00:42.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.862+0000 7f04bde43b80 1 --2- 172.21.15.151:0/2089847314 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x55801995a880 0x55801995cc70 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:42.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.862+0000 7f04bde43b80 1 --2- 172.21.15.151:0/2089847314 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558019947340 0x55801994a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:42.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.862+0000 7f04bde43b80 1 --2- 172.21.15.151:0/2089847314 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558019941550 0x55801994d090 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:42.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.862+0000 7f0492a00700 0 client.26398 ms_handle_reset on v2:172.21.15.164:6834/2340428470 2024-05-21T22:00:42.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.862+0000 7f04bde43b80 1 --2- 172.21.15.151:0/2089847314 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558019946a20 0x55801994a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:42.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.862+0000 7f04bde43b80 1 -- 172.21.15.151:0/2089847314 >> 172.21.15.151:0/2089847314 conn(0x55801986abe0 msgr2=0x558019936d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:00:42.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.863+0000 7f04bde43b80 1 -- 172.21.15.151:0/2089847314 shutdown_connections 2024-05-21T22:00:42.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.863+0000 7f04bde43b80 1 -- 172.21.15.151:0/2089847314 wait complete. 2024-05-21T22:00:42.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.863+0000 7f04bde43b80 20 client.26398 trim_cache size 0 max 16384 2024-05-21T22:00:42.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.879+0000 7f04bde43b80 1 Processor -- start 2024-05-21T22:00:42.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.879+0000 7f04bde43b80 1 -- start start 2024-05-21T22:00:42.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.879+0000 7f04bde43b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558019941550 0x558019940f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:42.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.879+0000 7f04bde43b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558019946a20 0x558019948070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:42.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.880+0000 7f04bde43b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558019947340 0x5580199374f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:42.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.880+0000 7f04bde43b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f04ac1abb20 con 0x558019947340 2024-05-21T22:00:42.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.880+0000 7f04bde43b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f04ac1ad660 con 0x558019946a20 2024-05-21T22:00:42.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.880+0000 7f04bde43b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f04ac1ac1f0 con 0x558019941550 2024-05-21T22:00:42.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.880+0000 7f04b1400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558019947340 0x5580199374f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:42.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.880+0000 7f04b1400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558019947340 0x5580199374f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:47244/0 (socket says 172.21.15.151:47244) 2024-05-21T22:00:42.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.880+0000 7f04b1400700 1 -- 172.21.15.151:0/499467226 learned_addr learned my addr 172.21.15.151:0/499467226 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:00:42.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.880+0000 7f04b0a00700 1 --2- 172.21.15.151:0/499467226 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558019941550 0x558019940f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:42.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.880+0000 7f04abe00700 1 --2- 172.21.15.151:0/499467226 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558019946a20 0x558019948070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:42.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.880+0000 7f04b1400700 1 -- 172.21.15.151:0/499467226 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558019941550 msgr2=0x558019940f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:42.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.880+0000 7f04b1400700 1 --2- 172.21.15.151:0/499467226 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558019941550 0x558019940f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:42.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.880+0000 7f04b1400700 1 -- 172.21.15.151:0/499467226 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558019946a20 msgr2=0x558019948070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:42.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.880+0000 7f04b1400700 1 --2- 172.21.15.151:0/499467226 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558019946a20 0x558019948070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:42.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.880+0000 7f04b1400700 1 -- 172.21.15.151:0/499467226 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580199489a0 con 0x558019947340 2024-05-21T22:00:42.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.881+0000 7f04b1400700 1 --2- 172.21.15.151:0/499467226 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558019947340 0x5580199374f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f04a4008f50 tx=0x7f04a401d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be6c2ce46640b811 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:00:42.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.881+0000 7f049d600700 1 -- 172.21.15.151:0/499467226 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a4021040 con 0x558019947340 2024-05-21T22:00:42.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.881+0000 7f049d600700 1 -- 172.21.15.151:0/499467226 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04a4004bb0 con 0x558019947340 2024-05-21T22:00:42.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.881+0000 7f049d600700 1 -- 172.21.15.151:0/499467226 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a401bc00 con 0x558019947340 2024-05-21T22:00:42.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.882+0000 7f04bde43b80 1 -- 172.21.15.151:0/499467226 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558019947340 msgr2=0x5580199374f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:42.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.882+0000 7f04bde43b80 1 --2- 172.21.15.151:0/499467226 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558019947340 0x5580199374f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f04a4008f50 tx=0x7f04a401d000 comp rx=0 tx=0).stop 2024-05-21T22:00:42.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.882+0000 7f04bde43b80 1 -- 172.21.15.151:0/499467226 shutdown_connections 2024-05-21T22:00:42.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.882+0000 7f04bde43b80 1 --2- 172.21.15.151:0/499467226 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558019947340 0x5580199374f0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:42.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.882+0000 7f04bde43b80 1 --2- 172.21.15.151:0/499467226 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558019941550 0x558019940f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:42.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.882+0000 7f04bde43b80 1 --2- 172.21.15.151:0/499467226 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558019946a20 0x558019948070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:42.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.882+0000 7f04bde43b80 1 -- 172.21.15.151:0/499467226 >> 172.21.15.151:0/499467226 conn(0x55801986abe0 msgr2=0x558019948bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:00:42.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.882+0000 7f04bde43b80 1 -- 172.21.15.151:0/499467226 shutdown_connections 2024-05-21T22:00:42.884 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.882+0000 7f04bde43b80 1 -- 172.21.15.151:0/499467226 wait complete. 2024-05-21T22:00:42.886 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.883+0000 7f04bde43b80 1 -- start start 2024-05-21T22:00:42.886 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.883+0000 7f04bde43b80 1 Processor -- start 2024-05-21T22:00:42.886 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.883+0000 7f04bde43b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558019941550 0x558019962a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:42.886 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.883+0000 7f04bde43b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558019946a20 0x558019953080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:42.886 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.883+0000 7f04bde43b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558019947340 0x5580199535c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:42.886 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.883+0000 7f04bde43b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f04ac1acf90 con 0x558019941550 2024-05-21T22:00:42.886 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.883+0000 7f04bde43b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f04ac1ac8c0 con 0x558019946a20 2024-05-21T22:00:42.886 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.883+0000 7f04bde43b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f04ac1a9b00 con 0x558019947340 2024-05-21T22:00:42.886 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.883+0000 7f04bde43b80 10 client.0 _ll_register_callbacks cb 0x558019834c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:00:42.886 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.884+0000 7f04b0a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558019941550 0x558019962a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:42.886 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.884+0000 7f04b0a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558019941550 0x558019962a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:47248/0 (socket says 172.21.15.151:47248) 2024-05-21T22:00:42.886 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.884+0000 7f04b0a00700 1 -- 172.21.15.151:0/3541447433 learned_addr learned my addr 172.21.15.151:0/3541447433 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:00:42.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.884+0000 7f04abe00700 1 --2- 172.21.15.151:0/3541447433 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558019946a20 0x558019953080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:42.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.884+0000 7f04b1400700 1 --2- 172.21.15.151:0/3541447433 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558019947340 0x5580199535c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:42.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.884+0000 7f04abe00700 1 -- 172.21.15.151:0/3541447433 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558019947340 msgr2=0x5580199535c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:42.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.884+0000 7f04abe00700 1 --2- 172.21.15.151:0/3541447433 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558019947340 0x5580199535c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:42.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.884+0000 7f04abe00700 1 -- 172.21.15.151:0/3541447433 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558019941550 msgr2=0x558019962a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:42.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.884+0000 7f04abe00700 1 --2- 172.21.15.151:0/3541447433 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558019941550 0x558019962a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:42.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.884+0000 7f04abe00700 1 -- 172.21.15.151:0/3541447433 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55801993bb10 con 0x558019946a20 2024-05-21T22:00:42.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.884+0000 7f04b0a00700 1 --2- 172.21.15.151:0/3541447433 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558019941550 0x558019962a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:00:42.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.884+0000 7f04abe00700 1 --2- 172.21.15.151:0/3541447433 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558019946a20 0x558019953080 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f04980e0a50 tx=0x7f0498022cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4599498e0ef43b09 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:00:42.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.884+0000 7f0492a00700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-21T22:00:42.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.884+0000 7f0492a00700 1 -- 172.21.15.151:0/3541447433 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04980aad70 con 0x558019946a20 2024-05-21T22:00:42.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.885+0000 7f04bde43b80 1 -- 172.21.15.151:0/3541447433 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04a4004bb0 con 0x558019946a20 2024-05-21T22:00:42.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.885+0000 7f0492a00700 1 -- 172.21.15.151:0/3541447433 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f04980a6900 con 0x558019946a20 2024-05-21T22:00:42.888 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.885+0000 7f0492a00700 1 -- 172.21.15.151:0/3541447433 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f04980aa670 con 0x558019946a20 2024-05-21T22:00:42.888 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.885+0000 7f04bde43b80 10 client.26390 fetch_fsmap learned FSMap version 29 2024-05-21T22:00:42.888 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.885+0000 7f04bde43b80 1 -- 172.21.15.151:0/3541447433 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5580199c9c20 con 0x558019946a20 2024-05-21T22:00:42.888 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.885+0000 7f0492a00700 1 -- 172.21.15.151:0/3541447433 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04a4004bb0 con 0x558019946a20 2024-05-21T22:00:42.888 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.885+0000 7f0492a00700 1 -- 172.21.15.151:0/3541447433 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f04980aa8d0 con 0x558019946a20 2024-05-21T22:00:42.888 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.885+0000 7f04bde43b80 10 client.26390 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:00:42.888 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.885+0000 7f04bde43b80 10 client.26390 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-05-21T22:00:42.888 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.885+0000 7f04bde43b80 20 client.26390 populate_metadata read hostname 'smithi151' 2024-05-21T22:00:42.888 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.885+0000 7f04bde43b80 1 --2- 172.21.15.151:0/3541447433 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x5580199ca330 0x5580199cc750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:42.888 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.885+0000 7f04bde43b80 4 client.26390 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T22:00:42.888 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.885+0000 7f04bde43b80 1 -- 172.21.15.151:0/3541447433 --> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558019896570 con 0x5580199ca330 2024-05-21T22:00:42.888 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.886+0000 7f04b0a00700 1 --2- 172.21.15.151:0/3541447433 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x5580199ca330 0x5580199cc750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:42.890 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.886+0000 7f04b0a00700 1 --2- 172.21.15.151:0/3541447433 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x5580199ca330 0x5580199cc750 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=78e8bdaf8c59677f server_cookie=24ec92cfbb87207 in_seq=0 out_seq=0 2024-05-21T22:00:42.890 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.886+0000 7f0492a00700 10 client.26390 ms_handle_connect on v2:172.21.15.164:6834/2340428470 2024-05-21T22:00:42.890 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.886+0000 7f0492a00700 1 -- 172.21.15.151:0/3541447433 <== mds.4 v2:172.21.15.164:6834/2340428470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558019896570 con 0x5580199ca330 2024-05-21T22:00:42.890 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.886+0000 7f0492a00700 10 client.26390 handle_command_reply: tid=0 2024-05-21T22:00:42.890 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.886+0000 7f04bde43b80 1 client.26390 shutdown 2024-05-21T22:00:42.890 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.887+0000 7f04bde43b80 1 -- 172.21.15.151:0/3541447433 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558019946a20 msgr2=0x558019953080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:42.890 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.887+0000 7f04bde43b80 1 --2- 172.21.15.151:0/3541447433 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558019946a20 0x558019953080 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f04980e0a50 tx=0x7f0498022cc0 comp rx=0 tx=0).stop 2024-05-21T22:00:42.890 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.887+0000 7f04bde43b80 1 -- 172.21.15.151:0/3541447433 shutdown_connections 2024-05-21T22:00:42.890 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.887+0000 7f04bde43b80 1 --2- 172.21.15.151:0/3541447433 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x5580199ca330 0x5580199cc750 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:42.890 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.887+0000 7f04bde43b80 1 --2- 172.21.15.151:0/3541447433 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558019947340 0x5580199535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:42.890 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.887+0000 7f04bde43b80 1 --2- 172.21.15.151:0/3541447433 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558019941550 0x558019962a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:42.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.887+0000 7f04bde43b80 1 --2- 172.21.15.151:0/3541447433 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558019946a20 0x558019953080 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:42.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.887+0000 7f04bde43b80 1 -- 172.21.15.151:0/3541447433 >> 172.21.15.151:0/3541447433 conn(0x55801986abe0 msgr2=0x558019949480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:00:42.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.887+0000 7f0492a00700 0 client.26390 ms_handle_reset on v2:172.21.15.164:6834/2340428470 2024-05-21T22:00:42.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.887+0000 7f04bde43b80 1 -- 172.21.15.151:0/3541447433 shutdown_connections 2024-05-21T22:00:42.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.887+0000 7f04bde43b80 1 -- 172.21.15.151:0/3541447433 wait complete. 2024-05-21T22:00:42.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.887+0000 7f04bde43b80 20 client.26390 trim_cache size 0 max 16384 2024-05-21T22:00:42.891 INFO:teuthology.orchestra.run.smithi151.stdout:[] 2024-05-21T22:00:42.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.888+0000 7f04b3400700 1 -- 172.21.15.151:0/2999820108 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f04940b29f0 msgr2=0x7f04940b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:42.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.888+0000 7f04b3400700 1 --2- 172.21.15.151:0/2999820108 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f04940b29f0 0x7f04940b4eb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f04a4008d40 tx=0x7f04a4009100 comp rx=0 tx=0).stop 2024-05-21T22:00:42.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.888+0000 7f04b3400700 1 -- 172.21.15.151:0/2999820108 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f04ac06bbe0 msgr2=0x7f04ac19fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:42.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.888+0000 7f04b3400700 1 --2- 172.21.15.151:0/2999820108 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f04ac06bbe0 0x7f04ac19fe60 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f0498000c00 tx=0x7f04980057c0 comp rx=0 tx=0).stop 2024-05-21T22:00:42.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.888+0000 7f04b3400700 1 -- 172.21.15.151:0/2999820108 shutdown_connections 2024-05-21T22:00:42.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.888+0000 7f04b3400700 1 --2- 172.21.15.151:0/2999820108 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f04940b29f0 0x7f04940b4eb0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:42.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.888+0000 7f04b3400700 1 --2- 172.21.15.151:0/2999820108 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f04ac1095b0 0x7f04ac199ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:42.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.888+0000 7f04b3400700 1 --2- 172.21.15.151:0/2999820108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04ac06b220 0x7f04ac19f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:42.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.888+0000 7f04b3400700 1 --2- 172.21.15.151:0/2999820108 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f04ac06bbe0 0x7f04ac19fe60 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:42.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.888+0000 7f04b3400700 1 -- 172.21.15.151:0/2999820108 >> 172.21.15.151:0/2999820108 conn(0x7f04ac0793f0 msgr2=0x7f04ac1011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:00:42.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.888+0000 7f04b3400700 1 -- 172.21.15.151:0/2999820108 shutdown_connections 2024-05-21T22:00:42.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:42.888+0000 7f04b3400700 1 -- 172.21.15.151:0/2999820108 wait complete. 2024-05-21T22:00:43.682 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:00:43 smithi151 ceph-mon[31623]: pgmap v156: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 0 B/s wr, 12 op/s 2024-05-21T22:00:43.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:00:43 smithi184 ceph-mon[36942]: pgmap v156: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 0 B/s wr, 12 op/s 2024-05-21T22:00:43.744 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-21T22:00:43.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:00:43 smithi164 ceph-mon[36269]: pgmap v156: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 0 B/s wr, 12 op/s 2024-05-21T22:00:44.184 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:00:45.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:00:45 smithi151 ceph-mon[31623]: pgmap v157: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:00:45.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:00:45 smithi184 ceph-mon[36942]: pgmap v157: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:00:45.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:00:45 smithi164 ceph-mon[36269]: pgmap v157: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:00:46.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.323+0000 7f8380e00700 1 -- 172.21.15.151:0/3384736463 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f837c077a70 msgr2=0x7f837c077e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:46.324 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.323+0000 7f8380e00700 1 --2- 172.21.15.151:0/3384736463 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f837c077a70 0x7f837c077e50 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f836c009990 tx=0x7f836c009ca0 comp rx=0 tx=0).stop 2024-05-21T22:00:46.324 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.323+0000 7f8380e00700 1 -- 172.21.15.151:0/3384736463 shutdown_connections 2024-05-21T22:00:46.324 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.323+0000 7f8380e00700 1 --2- 172.21.15.151:0/3384736463 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f837c1142f0 0x7f837c1166e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:46.324 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.323+0000 7f8380e00700 1 --2- 172.21.15.151:0/3384736463 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f837c077a70 0x7f837c077e50 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:46.324 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.323+0000 7f8380e00700 1 --2- 172.21.15.151:0/3384736463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f837c078390 0x7f837c113db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:46.324 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.323+0000 7f8380e00700 1 -- 172.21.15.151:0/3384736463 >> 172.21.15.151:0/3384736463 conn(0x7f837c101180 msgr2=0x7f837c1035a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:00:46.324 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.323+0000 7f8380e00700 1 -- 172.21.15.151:0/3384736463 shutdown_connections 2024-05-21T22:00:46.324 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.323+0000 7f8380e00700 1 -- 172.21.15.151:0/3384736463 wait complete. 2024-05-21T22:00:46.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.324+0000 7f8380e00700 1 Processor -- start 2024-05-21T22:00:46.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.324+0000 7f8380e00700 1 -- start start 2024-05-21T22:00:46.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.325+0000 7f8380e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f837c077a70 0x7f837c1a0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:46.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.325+0000 7f8380e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f837c078390 0x7f837c1a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:46.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.325+0000 7f8380e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f837c1142f0 0x7f837c1a4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:46.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.325+0000 7f8380e00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f837c119ce0 con 0x7f837c077a70 2024-05-21T22:00:46.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.325+0000 7f8380e00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f837c119b60 con 0x7f837c1142f0 2024-05-21T22:00:46.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.325+0000 7f8380e00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f837c119e60 con 0x7f837c078390 2024-05-21T22:00:46.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.325+0000 7f837b400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f837c077a70 0x7f837c1a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:46.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.325+0000 7f837b400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f837c077a70 0x7f837c1a0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49954/0 (socket says 172.21.15.151:49954) 2024-05-21T22:00:46.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.325+0000 7f837b400700 1 -- 172.21.15.151:0/3014156936 learned_addr learned my addr 172.21.15.151:0/3014156936 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:00:46.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.325+0000 7f837be00700 1 --2- 172.21.15.151:0/3014156936 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f837c1142f0 0x7f837c1a4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:46.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.325+0000 7f837aa00700 1 --2- 172.21.15.151:0/3014156936 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f837c078390 0x7f837c1a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:46.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.325+0000 7f837b400700 1 -- 172.21.15.151:0/3014156936 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f837c078390 msgr2=0x7f837c1a06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:46.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.325+0000 7f837b400700 1 --2- 172.21.15.151:0/3014156936 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f837c078390 0x7f837c1a06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:46.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.325+0000 7f837b400700 1 -- 172.21.15.151:0/3014156936 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f837c1142f0 msgr2=0x7f837c1a4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:46.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.325+0000 7f837b400700 1 --2- 172.21.15.151:0/3014156936 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f837c1142f0 0x7f837c1a4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:46.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.325+0000 7f837b400700 1 -- 172.21.15.151:0/3014156936 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f837c1a51a0 con 0x7f837c077a70 2024-05-21T22:00:46.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.326+0000 7f837b400700 1 --2- 172.21.15.151:0/3014156936 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f837c077a70 0x7f837c1a0190 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f836c0057c0 tx=0x7f836c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:00:46.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.326+0000 7f8373e00700 1 -- 172.21.15.151:0/3014156936 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f836c00fb00 con 0x7f837c077a70 2024-05-21T22:00:46.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.326+0000 7f8373e00700 1 -- 172.21.15.151:0/3014156936 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f836c00fca0 con 0x7f837c077a70 2024-05-21T22:00:46.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.326+0000 7f8373e00700 1 -- 172.21.15.151:0/3014156936 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f836c017650 con 0x7f837c077a70 2024-05-21T22:00:46.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.326+0000 7f8380e00700 1 -- 172.21.15.151:0/3014156936 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f837c1a5430 con 0x7f837c077a70 2024-05-21T22:00:46.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.326+0000 7f8380e00700 1 -- 172.21.15.151:0/3014156936 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f837c1a5860 con 0x7f837c077a70 2024-05-21T22:00:46.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.330+0000 7f8373e00700 1 -- 172.21.15.151:0/3014156936 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7f836c009d10 con 0x7f837c077a70 2024-05-21T22:00:46.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.331+0000 7f838cdacb80 1 Processor -- start 2024-05-21T22:00:46.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.331+0000 7f838cdacb80 1 -- start start 2024-05-21T22:00:46.332 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.331+0000 7f838cdacb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b46afe3020 0x55b46afe3400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:46.332 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.331+0000 7f838cdacb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b46afe3940 0x55b46afdd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:46.332 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.331+0000 7f838cdacb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b46afddad0 0x55b46afddf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:46.332 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.331+0000 7f838cdacb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55b46af3be40 con 0x55b46afe3940 2024-05-21T22:00:46.332 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.331+0000 7f838cdacb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55b46af2b9e0 con 0x55b46afddad0 2024-05-21T22:00:46.332 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.331+0000 7f838cdacb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55b46af2fa20 con 0x55b46afe3020 2024-05-21T22:00:46.333 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.332+0000 7f837b400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b46afe3940 0x55b46afdd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:46.333 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.332+0000 7f837aa00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b46afddad0 0x55b46afddf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:46.333 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.332+0000 7f837be00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b46afe3020 0x55b46afe3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:46.333 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.332+0000 7f837b400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b46afe3940 0x55b46afdd4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49968/0 (socket says 172.21.15.151:49968) 2024-05-21T22:00:46.333 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.332+0000 7f837b400700 1 -- 172.21.15.151:0/3879974704 learned_addr learned my addr 172.21.15.151:0/3879974704 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:00:46.333 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.332+0000 7f837be00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b46afe3020 0x55b46afe3400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:44990/0 (socket says 172.21.15.151:44990) 2024-05-21T22:00:46.333 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.332+0000 7f8373e00700 1 --2- 172.21.15.151:0/3014156936 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f83600b2a70 0x7f83600b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:46.334 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.332+0000 7f837aa00700 1 -- 172.21.15.151:0/3879974704 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b46afe3020 msgr2=0x55b46afe3400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:46.334 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.332+0000 7f837aa00700 1 --2- 172.21.15.151:0/3879974704 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b46afe3020 0x55b46afe3400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:46.334 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.332+0000 7f837aa00700 1 -- 172.21.15.151:0/3879974704 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b46afe3940 msgr2=0x55b46afdd4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:46.334 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.332+0000 7f837aa00700 1 --2- 172.21.15.151:0/3879974704 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b46afe3940 0x55b46afdd4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:46.334 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.332+0000 7f8373e00700 1 -- 172.21.15.151:0/3014156936 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f836c0e3430 con 0x7f837c077a70 2024-05-21T22:00:46.334 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.333+0000 7f837aa00700 1 -- 172.21.15.151:0/3879974704 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b46aef4420 con 0x55b46afddad0 2024-05-21T22:00:46.334 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.333+0000 7f837be00700 1 --2- 172.21.15.151:0/3879974704 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b46afe3020 0x55b46afe3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T22:00:46.334 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.333+0000 7f837be00700 1 --2- 172.21.15.151:0/3014156936 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f83600b2a70 0x7f83600b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:46.334 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.333+0000 7f837aa00700 1 --2- 172.21.15.151:0/3879974704 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b46afddad0 0x55b46afddf30 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f8368009c60 tx=0x7f8368009f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e1083a5e425c98a server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:00:46.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.333+0000 7f8370c00700 1 -- 172.21.15.151:0/3879974704 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8368011d10 con 0x55b46afddad0 2024-05-21T22:00:46.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.333+0000 7f8370c00700 1 -- 172.21.15.151:0/3879974704 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8368004500 con 0x55b46afddad0 2024-05-21T22:00:46.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.333+0000 7f8370c00700 1 -- 172.21.15.151:0/3879974704 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8368005280 con 0x55b46afddad0 2024-05-21T22:00:46.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.334+0000 7f838cdacb80 1 -- 172.21.15.151:0/3879974704 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b46afddad0 msgr2=0x55b46afddf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:46.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.334+0000 7f838cdacb80 1 --2- 172.21.15.151:0/3879974704 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b46afddad0 0x55b46afddf30 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f8368009c60 tx=0x7f8368009f70 comp rx=0 tx=0).stop 2024-05-21T22:00:46.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.334+0000 7f837be00700 1 --2- 172.21.15.151:0/3014156936 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f83600b2a70 0x7f83600b4f30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8374009ec0 tx=0x7f8374009040 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:00:46.336 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.334+0000 7f838cdacb80 1 -- 172.21.15.151:0/3879974704 shutdown_connections 2024-05-21T22:00:46.336 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.334+0000 7f838cdacb80 1 --2- 172.21.15.151:0/3879974704 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b46afddad0 0x55b46afddf30 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:46.336 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.334+0000 7f838cdacb80 1 --2- 172.21.15.151:0/3879974704 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b46afe3020 0x55b46afe3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:46.336 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.334+0000 7f838cdacb80 1 --2- 172.21.15.151:0/3879974704 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b46afe3940 0x55b46afdd4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:46.336 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.334+0000 7f838cdacb80 1 -- 172.21.15.151:0/3879974704 >> 172.21.15.151:0/3879974704 conn(0x55b46af0bbe0 msgr2=0x55b46afd8150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:00:46.336 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.334+0000 7f838cdacb80 1 -- 172.21.15.151:0/3879974704 shutdown_connections 2024-05-21T22:00:46.336 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.334+0000 7f838cdacb80 1 -- 172.21.15.151:0/3879974704 wait complete. 2024-05-21T22:00:46.336 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.335+0000 7f838cdacb80 1 -- start start 2024-05-21T22:00:46.336 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.335+0000 7f838cdacb80 1 Processor -- start 2024-05-21T22:00:46.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.336+0000 7f838cdacb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b46afddad0 0x55b46afe96a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:46.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.336+0000 7f838cdacb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b46afe3020 0x55b46afe6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:46.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.336+0000 7f838cdacb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b46afe3940 0x55b46afe6f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:46.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.336+0000 7f838cdacb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55b46ae31fa0 con 0x55b46afe3940 2024-05-21T22:00:46.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.336+0000 7f838cdacb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55b46aecc760 con 0x55b46afe3020 2024-05-21T22:00:46.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.336+0000 7f838cdacb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55b46afea460 con 0x55b46afddad0 2024-05-21T22:00:46.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.336+0000 7f838cdacb80 10 client.0 _ll_register_callbacks cb 0x55b46aed5c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:00:46.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.336+0000 7f837be00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b46afe3940 0x55b46afe6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:46.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.336+0000 7f837b400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b46afddad0 0x55b46afe96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:46.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.336+0000 7f837b400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b46afddad0 0x55b46afe96a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:44998/0 (socket says 172.21.15.151:44998) 2024-05-21T22:00:46.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.336+0000 7f837b400700 1 -- 172.21.15.151:0/1462172841 learned_addr learned my addr 172.21.15.151:0/1462172841 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:00:46.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.336+0000 7f837aa00700 1 --2- 172.21.15.151:0/1462172841 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b46afe3020 0x55b46afe6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:46.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.337+0000 7f837b400700 1 -- 172.21.15.151:0/1462172841 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b46afe3020 msgr2=0x55b46afe6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:46.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.337+0000 7f837b400700 1 --2- 172.21.15.151:0/1462172841 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b46afe3020 0x55b46afe6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:46.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.337+0000 7f837b400700 1 -- 172.21.15.151:0/1462172841 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b46afe3940 msgr2=0x55b46afe6f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:46.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.337+0000 7f837b400700 1 --2- 172.21.15.151:0/1462172841 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b46afe3940 0x55b46afe6f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:46.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.337+0000 7f837b400700 1 -- 172.21.15.151:0/1462172841 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b46ae7b8d0 con 0x55b46afddad0 2024-05-21T22:00:46.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.337+0000 7f837b400700 1 --2- 172.21.15.151:0/1462172841 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b46afddad0 0x55b46afe96a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f836c0a6050 tx=0x7f836c009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c68dca7bf3bda7c server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:00:46.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.337+0000 7f8365600700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-21T22:00:46.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.337+0000 7f8365600700 1 -- 172.21.15.151:0/1462172841 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f836c0aa0a0 con 0x55b46afddad0 2024-05-21T22:00:46.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.337+0000 7f838cdacb80 1 -- 172.21.15.151:0/1462172841 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8368004500 con 0x55b46afddad0 2024-05-21T22:00:46.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.337+0000 7f8365600700 1 -- 172.21.15.151:0/1462172841 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f836c0aa9b0 con 0x55b46afddad0 2024-05-21T22:00:46.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.338+0000 7f8365600700 1 -- 172.21.15.151:0/1462172841 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f836c092320 con 0x55b46afddad0 2024-05-21T22:00:46.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.338+0000 7f838cdacb80 10 client.26416 fetch_fsmap learned FSMap version 29 2024-05-21T22:00:46.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.338+0000 7f838cdacb80 1 -- 172.21.15.151:0/1462172841 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b46aefc060 con 0x55b46afddad0 2024-05-21T22:00:46.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.338+0000 7f8365600700 1 -- 172.21.15.151:0/1462172841 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8368004500 con 0x55b46afddad0 2024-05-21T22:00:46.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.338+0000 7f8365600700 1 -- 172.21.15.151:0/1462172841 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f836c0b3090 con 0x55b46afddad0 2024-05-21T22:00:46.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.338+0000 7f838cdacb80 10 client.26416 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:00:46.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.338+0000 7f838cdacb80 10 client.26416 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-21T22:00:46.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.338+0000 7f838cdacb80 20 client.26416 populate_metadata read hostname 'smithi151' 2024-05-21T22:00:46.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.338+0000 7f838cdacb80 1 --2- 172.21.15.151:0/1462172841 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x55b46aff6fb0 0x55b46aff93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:46.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.338+0000 7f838cdacb80 4 client.26416 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T22:00:46.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.338+0000 7f838cdacb80 1 -- 172.21.15.151:0/1462172841 --> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b46af3dd90 con 0x55b46aff6fb0 2024-05-21T22:00:46.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.339+0000 7f837aa00700 1 --2- 172.21.15.151:0/1462172841 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x55b46aff6fb0 0x55b46aff93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:46.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.339+0000 7f837aa00700 1 --2- 172.21.15.151:0/1462172841 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x55b46aff6fb0 0x55b46aff93f0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=61299868df05339d server_cookie=14aacca060159ebf in_seq=0 out_seq=0 2024-05-21T22:00:46.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.339+0000 7f8365600700 10 client.26416 ms_handle_connect on v2:172.21.15.184:6836/2988194888 2024-05-21T22:00:46.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.340+0000 7f8365600700 1 -- 172.21.15.151:0/1462172841 <== mds.3 v2:172.21.15.184:6836/2988194888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55b46af3dd90 con 0x55b46aff6fb0 2024-05-21T22:00:46.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.340+0000 7f8365600700 10 client.26416 handle_command_reply: tid=0 2024-05-21T22:00:46.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.340+0000 7f838cdacb80 1 client.26416 shutdown 2024-05-21T22:00:46.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.340+0000 7f838cdacb80 1 -- 172.21.15.151:0/1462172841 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b46afddad0 msgr2=0x55b46afe96a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:46.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.340+0000 7f838cdacb80 1 --2- 172.21.15.151:0/1462172841 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b46afddad0 0x55b46afe96a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f836c0a6050 tx=0x7f836c009310 comp rx=0 tx=0).stop 2024-05-21T22:00:46.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.340+0000 7f838cdacb80 1 -- 172.21.15.151:0/1462172841 shutdown_connections 2024-05-21T22:00:46.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.340+0000 7f838cdacb80 1 --2- 172.21.15.151:0/1462172841 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x55b46aff6fb0 0x55b46aff93f0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:46.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.340+0000 7f838cdacb80 1 --2- 172.21.15.151:0/1462172841 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b46afe3940 0x55b46afe6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:46.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.340+0000 7f838cdacb80 1 --2- 172.21.15.151:0/1462172841 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b46afddad0 0x55b46afe96a0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:46.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.340+0000 7f838cdacb80 1 --2- 172.21.15.151:0/1462172841 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b46afe3020 0x55b46afe6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:46.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.340+0000 7f838cdacb80 1 -- 172.21.15.151:0/1462172841 >> 172.21.15.151:0/1462172841 conn(0x55b46af0bbe0 msgr2=0x55b46afdc4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:00:46.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.341+0000 7f8365600700 0 client.26416 ms_handle_reset on v2:172.21.15.184:6836/2988194888 2024-05-21T22:00:46.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.341+0000 7f838cdacb80 1 -- 172.21.15.151:0/1462172841 shutdown_connections 2024-05-21T22:00:46.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.341+0000 7f838cdacb80 1 -- 172.21.15.151:0/1462172841 wait complete. 2024-05-21T22:00:46.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.341+0000 7f838cdacb80 20 client.26416 trim_cache size 0 max 16384 2024-05-21T22:00:46.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.358+0000 7f838cdacb80 1 Processor -- start 2024-05-21T22:00:46.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.358+0000 7f838cdacb80 1 -- start start 2024-05-21T22:00:46.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.358+0000 7f838cdacb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b46afddad0 0x55b46afddeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:46.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.358+0000 7f838cdacb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b46afe3020 0x55b46afe5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:46.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.358+0000 7f838cdacb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b46afe3940 0x55b46afe5870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:46.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.358+0000 7f838cdacb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f837c1b4130 con 0x55b46afe3020 2024-05-21T22:00:46.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.358+0000 7f838cdacb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f837c1b5c70 con 0x55b46afddad0 2024-05-21T22:00:46.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.358+0000 7f838cdacb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f837c1b4800 con 0x55b46afe3940 2024-05-21T22:00:46.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.359+0000 7f837b400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b46afddad0 0x55b46afddeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:46.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.359+0000 7f837b400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b46afddad0 0x55b46afddeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:53220/0 (socket says 172.21.15.151:53220) 2024-05-21T22:00:46.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.359+0000 7f837b400700 1 -- 172.21.15.151:0/3974145007 learned_addr learned my addr 172.21.15.151:0/3974145007 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:00:46.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.359+0000 7f837aa00700 1 --2- 172.21.15.151:0/3974145007 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b46afe3020 0x55b46afe5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:46.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.359+0000 7f837be00700 1 --2- 172.21.15.151:0/3974145007 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b46afe3940 0x55b46afe5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:46.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.359+0000 7f837b400700 1 -- 172.21.15.151:0/3974145007 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b46afe3940 msgr2=0x55b46afe5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:46.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.359+0000 7f837b400700 1 --2- 172.21.15.151:0/3974145007 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b46afe3940 0x55b46afe5870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:46.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.359+0000 7f837b400700 1 -- 172.21.15.151:0/3974145007 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b46afe3020 msgr2=0x55b46afe5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:46.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.359+0000 7f837b400700 1 --2- 172.21.15.151:0/3974145007 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b46afe3020 0x55b46afe5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:46.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.359+0000 7f837b400700 1 -- 172.21.15.151:0/3974145007 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b46afd90f0 con 0x55b46afddad0 2024-05-21T22:00:46.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.359+0000 7f837aa00700 1 --2- 172.21.15.151:0/3974145007 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b46afe3020 0x55b46afe5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T22:00:46.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.359+0000 7f837b400700 1 --2- 172.21.15.151:0/3974145007 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b46afddad0 0x55b46afddeb0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f836c00bff0 tx=0x7f836c009510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3cc673a79e8d2834 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:00:46.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.359+0000 7f8367e00700 1 -- 172.21.15.151:0/3974145007 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f836c0a9ec0 con 0x55b46afddad0 2024-05-21T22:00:46.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.359+0000 7f8367e00700 1 -- 172.21.15.151:0/3974145007 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f836c0095a0 con 0x55b46afddad0 2024-05-21T22:00:46.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.360+0000 7f8367e00700 1 -- 172.21.15.151:0/3974145007 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f836c0aa9b0 con 0x55b46afddad0 2024-05-21T22:00:46.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.360+0000 7f838cdacb80 1 -- 172.21.15.151:0/3974145007 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b46afddad0 msgr2=0x55b46afddeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:46.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.360+0000 7f838cdacb80 1 --2- 172.21.15.151:0/3974145007 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b46afddad0 0x55b46afddeb0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f836c00bff0 tx=0x7f836c009510 comp rx=0 tx=0).stop 2024-05-21T22:00:46.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.360+0000 7f838cdacb80 1 -- 172.21.15.151:0/3974145007 shutdown_connections 2024-05-21T22:00:46.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.360+0000 7f838cdacb80 1 --2- 172.21.15.151:0/3974145007 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b46afddad0 0x55b46afddeb0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:46.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.360+0000 7f838cdacb80 1 --2- 172.21.15.151:0/3974145007 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b46afe3940 0x55b46afe5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:46.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.360+0000 7f838cdacb80 1 --2- 172.21.15.151:0/3974145007 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b46afe3020 0x55b46afe5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:46.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.360+0000 7f838cdacb80 1 -- 172.21.15.151:0/3974145007 >> 172.21.15.151:0/3974145007 conn(0x55b46af0bbe0 msgr2=0x55b46afdce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:00:46.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.361+0000 7f838cdacb80 1 -- 172.21.15.151:0/3974145007 shutdown_connections 2024-05-21T22:00:46.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.361+0000 7f838cdacb80 1 -- 172.21.15.151:0/3974145007 wait complete. 2024-05-21T22:00:46.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.361+0000 7f838cdacb80 1 -- start start 2024-05-21T22:00:46.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.361+0000 7f838cdacb80 1 Processor -- start 2024-05-21T22:00:46.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.362+0000 7f838cdacb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b46afddad0 0x55b46afefbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:46.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.362+0000 7f838cdacb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b46afe3020 0x55b46affe2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:46.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.362+0000 7f838cdacb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b46afe3940 0x55b46affe830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:46.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.362+0000 7f838cdacb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f837c1b55a0 con 0x55b46afddad0 2024-05-21T22:00:46.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.362+0000 7f838cdacb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f837c1b4ed0 con 0x55b46afe3940 2024-05-21T22:00:46.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.362+0000 7f838cdacb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f837c1b20f0 con 0x55b46afe3020 2024-05-21T22:00:46.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.362+0000 7f838cdacb80 10 client.0 _ll_register_callbacks cb 0x55b46afd2fa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:00:46.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.362+0000 7f837b400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b46afddad0 0x55b46afefbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:46.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.362+0000 7f837b400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b46afddad0 0x55b46afefbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49982/0 (socket says 172.21.15.151:49982) 2024-05-21T22:00:46.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.362+0000 7f837b400700 1 -- 172.21.15.151:0/948320829 learned_addr learned my addr 172.21.15.151:0/948320829 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:00:46.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.362+0000 7f837be00700 1 --2- 172.21.15.151:0/948320829 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b46afe3940 0x55b46affe830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:46.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.362+0000 7f837aa00700 1 --2- 172.21.15.151:0/948320829 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b46afe3020 0x55b46affe2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:46.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.363+0000 7f837b400700 1 -- 172.21.15.151:0/948320829 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b46afe3020 msgr2=0x55b46affe2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:46.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.363+0000 7f837b400700 1 --2- 172.21.15.151:0/948320829 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b46afe3020 0x55b46affe2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:46.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.363+0000 7f837b400700 1 -- 172.21.15.151:0/948320829 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b46afe3940 msgr2=0x55b46affe830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:46.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.363+0000 7f837b400700 1 --2- 172.21.15.151:0/948320829 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b46afe3940 0x55b46affe830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:46.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.363+0000 7f837b400700 1 -- 172.21.15.151:0/948320829 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b46affedf0 con 0x55b46afddad0 2024-05-21T22:00:46.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.363+0000 7f837be00700 1 --2- 172.21.15.151:0/948320829 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b46afe3940 0x55b46affe830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T22:00:46.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.363+0000 7f837b400700 1 --2- 172.21.15.151:0/948320829 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b46afddad0 0x55b46afefbf0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f836c0cd190 tx=0x7f836c0aa650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c66c829ee251194 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:00:46.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.363+0000 7f8365600700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-21T22:00:46.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.363+0000 7f8365600700 1 -- 172.21.15.151:0/948320829 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f836c0a9ec0 con 0x55b46afddad0 2024-05-21T22:00:46.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.363+0000 7f838cdacb80 1 -- 172.21.15.151:0/948320829 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f836c0095a0 con 0x55b46afddad0 2024-05-21T22:00:46.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.363+0000 7f8365600700 1 -- 172.21.15.151:0/948320829 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f836c0a9bc0 con 0x55b46afddad0 2024-05-21T22:00:46.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.363+0000 7f8365600700 1 -- 172.21.15.151:0/948320829 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f836c0aa0a0 con 0x55b46afddad0 2024-05-21T22:00:46.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.363+0000 7f838cdacb80 10 client.16284 fetch_fsmap learned FSMap version 29 2024-05-21T22:00:46.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.363+0000 7f838cdacb80 1 -- 172.21.15.151:0/948320829 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b46affedf0 con 0x55b46afddad0 2024-05-21T22:00:46.367 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.364+0000 7f8365600700 1 -- 172.21.15.151:0/948320829 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f836c0c9080 con 0x55b46afddad0 2024-05-21T22:00:46.367 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.364+0000 7f8365600700 1 -- 172.21.15.151:0/948320829 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f836c0a60c0 con 0x55b46afddad0 2024-05-21T22:00:46.367 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.364+0000 7f838cdacb80 10 client.16284 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:00:46.367 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.364+0000 7f838cdacb80 10 client.16284 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-21T22:00:46.367 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.364+0000 7f838cdacb80 20 client.16284 populate_metadata read hostname 'smithi151' 2024-05-21T22:00:46.367 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.364+0000 7f838cdacb80 1 --2- 172.21.15.151:0/948320829 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x55b46b061b50 0x55b46b063f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:46.367 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.364+0000 7f838cdacb80 4 client.16284 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T22:00:46.367 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.364+0000 7f838cdacb80 1 -- 172.21.15.151:0/948320829 --> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b46af37570 con 0x55b46b061b50 2024-05-21T22:00:46.367 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.365+0000 7f837aa00700 1 --2- 172.21.15.151:0/948320829 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x55b46b061b50 0x55b46b063f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:46.368 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.365+0000 7f837aa00700 1 --2- 172.21.15.151:0/948320829 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x55b46b061b50 0x55b46b063f40 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b0127eefdc50432c server_cookie=3cf8620939f7f516 in_seq=0 out_seq=0 2024-05-21T22:00:46.368 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.365+0000 7f8365600700 10 client.16284 ms_handle_connect on v2:172.21.15.184:6836/2988194888 2024-05-21T22:00:46.368 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.365+0000 7f8365600700 1 -- 172.21.15.151:0/948320829 <== mds.3 v2:172.21.15.184:6836/2988194888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b46af37570 con 0x55b46b061b50 2024-05-21T22:00:46.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.365+0000 7f8365600700 10 client.16284 handle_command_reply: tid=0 2024-05-21T22:00:46.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.365+0000 7f838cdacb80 1 client.16284 shutdown 2024-05-21T22:00:46.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.365+0000 7f838cdacb80 1 -- 172.21.15.151:0/948320829 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b46afddad0 msgr2=0x55b46afefbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:46.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.365+0000 7f838cdacb80 1 --2- 172.21.15.151:0/948320829 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b46afddad0 0x55b46afefbf0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f836c0cd190 tx=0x7f836c0aa650 comp rx=0 tx=0).stop 2024-05-21T22:00:46.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.366+0000 7f838cdacb80 1 -- 172.21.15.151:0/948320829 shutdown_connections 2024-05-21T22:00:46.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.366+0000 7f838cdacb80 1 --2- 172.21.15.151:0/948320829 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x55b46b061b50 0x55b46b063f40 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:46.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.366+0000 7f838cdacb80 1 --2- 172.21.15.151:0/948320829 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b46afe3940 0x55b46affe830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:46.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.366+0000 7f838cdacb80 1 --2- 172.21.15.151:0/948320829 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b46afddad0 0x55b46afefbf0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:46.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.366+0000 7f838cdacb80 1 --2- 172.21.15.151:0/948320829 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b46afe3020 0x55b46affe2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:46.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.366+0000 7f8365600700 0 client.16284 ms_handle_reset on v2:172.21.15.184:6836/2988194888 2024-05-21T22:00:46.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.366+0000 7f838cdacb80 1 -- 172.21.15.151:0/948320829 >> 172.21.15.151:0/948320829 conn(0x55b46af0bbe0 msgr2=0x55b46afd7c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:00:46.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.366+0000 7f838cdacb80 1 -- 172.21.15.151:0/948320829 shutdown_connections 2024-05-21T22:00:46.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.366+0000 7f838cdacb80 1 -- 172.21.15.151:0/948320829 wait complete. 2024-05-21T22:00:46.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.366+0000 7f838cdacb80 20 client.16284 trim_cache size 0 max 16384 2024-05-21T22:00:46.370 INFO:teuthology.orchestra.run.smithi151.stdout:[] 2024-05-21T22:00:46.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.367+0000 7f8380e00700 1 -- 172.21.15.151:0/3014156936 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f83600b2a70 msgr2=0x7f83600b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:46.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.367+0000 7f8380e00700 1 --2- 172.21.15.151:0/3014156936 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f83600b2a70 0x7f83600b4f30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8374009ec0 tx=0x7f8374009040 comp rx=0 tx=0).stop 2024-05-21T22:00:46.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.367+0000 7f8380e00700 1 -- 172.21.15.151:0/3014156936 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f837c077a70 msgr2=0x7f837c1a0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:46.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.367+0000 7f8380e00700 1 --2- 172.21.15.151:0/3014156936 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f837c077a70 0x7f837c1a0190 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f836c0057c0 tx=0x7f836c0057f0 comp rx=0 tx=0).stop 2024-05-21T22:00:46.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.367+0000 7f8380e00700 1 -- 172.21.15.151:0/3014156936 shutdown_connections 2024-05-21T22:00:46.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.367+0000 7f8380e00700 1 --2- 172.21.15.151:0/3014156936 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f83600b2a70 0x7f83600b4f30 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:46.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.367+0000 7f8380e00700 1 --2- 172.21.15.151:0/3014156936 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f837c1142f0 0x7f837c1a4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:46.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.367+0000 7f8380e00700 1 --2- 172.21.15.151:0/3014156936 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f837c077a70 0x7f837c1a0190 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:46.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.367+0000 7f8380e00700 1 --2- 172.21.15.151:0/3014156936 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f837c078390 0x7f837c1a06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:46.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.367+0000 7f8380e00700 1 -- 172.21.15.151:0/3014156936 >> 172.21.15.151:0/3014156936 conn(0x7f837c101180 msgr2=0x7f837c114f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:00:46.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.367+0000 7f8380e00700 1 -- 172.21.15.151:0/3014156936 shutdown_connections 2024-05-21T22:00:46.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:46.367+0000 7f8380e00700 1 -- 172.21.15.151:0/3014156936 wait complete. 2024-05-21T22:00:47.315 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-21T22:00:47.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:00:47 smithi151 ceph-mon[31623]: pgmap v158: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:00:47.735 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:00:47 smithi184 ceph-mon[36942]: pgmap v158: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:00:47.782 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:00:47.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:00:47 smithi164 ceph-mon[36269]: pgmap v158: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:00:49.678 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:00:49 smithi151 ceph-mon[31623]: pgmap v159: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T22:00:49.735 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:00:49 smithi184 ceph-mon[36942]: pgmap v159: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T22:00:49.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:00:49 smithi164 ceph-mon[36269]: pgmap v159: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T22:00:49.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.972+0000 7f6d7fe00700 1 -- 172.21.15.151:0/3115287811 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6d78105760 msgr2=0x7f6d78105be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:49.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.972+0000 7f6d7fe00700 1 --2- 172.21.15.151:0/3115287811 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6d78105760 0x7f6d78105be0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f6d64009990 tx=0x7f6d64009ca0 comp rx=0 tx=0).stop 2024-05-21T22:00:49.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.972+0000 7f6d7fe00700 1 -- 172.21.15.151:0/3115287811 shutdown_connections 2024-05-21T22:00:49.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.972+0000 7f6d7fe00700 1 --2- 172.21.15.151:0/3115287811 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d78106120 0x7f6d78112630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:49.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.972+0000 7f6d7fe00700 1 --2- 172.21.15.151:0/3115287811 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6d7810b760 0x7f6d7810bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:49.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.972+0000 7f6d7fe00700 1 --2- 172.21.15.151:0/3115287811 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6d78105760 0x7f6d78105be0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:49.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.972+0000 7f6d7fe00700 1 -- 172.21.15.151:0/3115287811 >> 172.21.15.151:0/3115287811 conn(0x7f6d78101250 msgr2=0x7f6d78103670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:00:49.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.972+0000 7f6d7fe00700 1 -- 172.21.15.151:0/3115287811 shutdown_connections 2024-05-21T22:00:49.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.972+0000 7f6d7fe00700 1 -- 172.21.15.151:0/3115287811 wait complete. 2024-05-21T22:00:49.975 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.973+0000 7f6d7fe00700 1 Processor -- start 2024-05-21T22:00:49.975 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.973+0000 7f6d7fe00700 1 -- start start 2024-05-21T22:00:49.975 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.973+0000 7f6d7fe00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6d78105760 0x7f6d7819be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:49.975 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.974+0000 7f6d7fe00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d78106120 0x7f6d7819c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:49.975 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.974+0000 7f6d7fe00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6d7810b760 0x7f6d781a0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:49.975 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.974+0000 7f6d7fe00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6d781159f0 con 0x7f6d78105760 2024-05-21T22:00:49.975 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.974+0000 7f6d7fe00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6d78115870 con 0x7f6d78106120 2024-05-21T22:00:49.975 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.974+0000 7f6d7fe00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6d78115b70 con 0x7f6d7810b760 2024-05-21T22:00:49.975 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.974+0000 7f6d7ca00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d78106120 0x7f6d7819c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:49.975 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.974+0000 7f6d7d400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6d78105760 0x7f6d7819be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:49.975 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.974+0000 7f6d7ca00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d78106120 0x7f6d7819c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:53252/0 (socket says 172.21.15.151:53252) 2024-05-21T22:00:49.975 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.974+0000 7f6d7ca00700 1 -- 172.21.15.151:0/1250780958 learned_addr learned my addr 172.21.15.151:0/1250780958 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:00:49.975 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.974+0000 7f6d7de00700 1 --2- 172.21.15.151:0/1250780958 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6d7810b760 0x7f6d781a0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:49.976 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.974+0000 7f6d7ca00700 1 -- 172.21.15.151:0/1250780958 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6d7810b760 msgr2=0x7f6d781a0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:49.976 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.974+0000 7f6d7ca00700 1 --2- 172.21.15.151:0/1250780958 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6d7810b760 0x7f6d781a0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:49.976 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.974+0000 7f6d7ca00700 1 -- 172.21.15.151:0/1250780958 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6d78105760 msgr2=0x7f6d7819be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:49.976 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.974+0000 7f6d7ca00700 1 --2- 172.21.15.151:0/1250780958 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6d78105760 0x7f6d7819be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:49.976 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.974+0000 7f6d7ca00700 1 -- 172.21.15.151:0/1250780958 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d781a0e70 con 0x7f6d78106120 2024-05-21T22:00:49.976 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.974+0000 7f6d7de00700 1 --2- 172.21.15.151:0/1250780958 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6d7810b760 0x7f6d781a0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T22:00:49.976 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.975+0000 7f6d7ca00700 1 --2- 172.21.15.151:0/1250780958 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d78106120 0x7f6d7819c3a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f6d64000c00 tx=0x7f6d640057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:00:49.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.975+0000 7f6d76000700 1 -- 172.21.15.151:0/1250780958 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d64027070 con 0x7f6d78106120 2024-05-21T22:00:49.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.975+0000 7f6d7fe00700 1 -- 172.21.15.151:0/1250780958 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d781a1100 con 0x7f6d78106120 2024-05-21T22:00:49.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.975+0000 7f6d76000700 1 -- 172.21.15.151:0/1250780958 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d64022470 con 0x7f6d78106120 2024-05-21T22:00:49.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.975+0000 7f6d76000700 1 -- 172.21.15.151:0/1250780958 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d64017650 con 0x7f6d78106120 2024-05-21T22:00:49.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.975+0000 7f6d7fe00700 1 -- 172.21.15.151:0/1250780958 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d781a89e0 con 0x7f6d78106120 2024-05-21T22:00:49.979 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.979+0000 7f6d76000700 1 -- 172.21.15.151:0/1250780958 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7f6d6402a070 con 0x7f6d78106120 2024-05-21T22:00:49.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.980+0000 7f6d8a6cdb80 1 Processor -- start 2024-05-21T22:00:49.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.980+0000 7f6d8a6cdb80 1 -- start start 2024-05-21T22:00:49.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.980+0000 7f6d76000700 1 --2- 172.21.15.151:0/1250780958 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f6d600b29f0 0x7f6d600b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:49.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.981+0000 7f6d8a6cdb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563a92885610 0x563a928859f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:49.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.981+0000 7f6d8a6cdb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a92885f30 0x563a9287fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:49.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.981+0000 7f6d8a6cdb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563a92880140 0x563a928805c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:49.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.981+0000 7f6d8a6cdb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x563a927dde40 con 0x563a92885610 2024-05-21T22:00:49.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.981+0000 7f6d8a6cdb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x563a927cd9e0 con 0x563a92880140 2024-05-21T22:00:49.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.981+0000 7f6d8a6cdb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x563a927d1a20 con 0x563a92885f30 2024-05-21T22:00:49.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.981+0000 7f6d76000700 1 -- 172.21.15.151:0/1250780958 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f6d640e33e0 con 0x7f6d78106120 2024-05-21T22:00:49.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.981+0000 7f6d7de00700 1 --2- 172.21.15.151:0/1250780958 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f6d600b29f0 0x7f6d600b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:49.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.981+0000 7f6d7ca00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a92885f30 0x563a9287fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:49.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.981+0000 7f6d7ca00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a92885f30 0x563a9287fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:45032/0 (socket says 172.21.15.151:45032) 2024-05-21T22:00:49.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.981+0000 7f6d7ca00700 1 -- 172.21.15.151:0/4108108026 learned_addr learned my addr 172.21.15.151:0/4108108026 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:00:49.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.981+0000 7f6d7de00700 1 --2- 172.21.15.151:0/4108108026 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563a92880140 0x563a928805c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:49.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.981+0000 7f6d7d400700 1 --2- 172.21.15.151:0/4108108026 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563a92885610 0x563a928859f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:49.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.981+0000 7f6d7ca00700 1 -- 172.21.15.151:0/4108108026 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563a92880140 msgr2=0x563a928805c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:49.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.981+0000 7f6d7ca00700 1 --2- 172.21.15.151:0/4108108026 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563a92880140 0x563a928805c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:49.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.981+0000 7f6d7ca00700 1 -- 172.21.15.151:0/4108108026 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563a92885610 msgr2=0x563a928859f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:49.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.981+0000 7f6d7ca00700 1 --2- 172.21.15.151:0/4108108026 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563a92885610 0x563a928859f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:49.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.981+0000 7f6d7ca00700 1 -- 172.21.15.151:0/4108108026 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a92796420 con 0x563a92885f30 2024-05-21T22:00:49.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.982+0000 7f6d7de00700 1 --2- 172.21.15.151:0/4108108026 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563a92880140 0x563a928805c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T22:00:49.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.982+0000 7f6d7ca00700 1 --2- 172.21.15.151:0/4108108026 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a92885f30 0x563a9287fb20 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f6d64006010 tx=0x7f6d64005f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=398abbed2045997a server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:00:49.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.982+0000 7f6d7de00700 1 --2- 172.21.15.151:0/1250780958 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f6d600b29f0 0x7f6d600b4eb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x563a92880b00 tx=0x7f6d70009920 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:00:49.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.982+0000 7f6d6aa00700 1 -- 172.21.15.151:0/4108108026 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d640a9e70 con 0x563a92885f30 2024-05-21T22:00:49.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.982+0000 7f6d6aa00700 1 -- 172.21.15.151:0/4108108026 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d640aa210 con 0x563a92885f30 2024-05-21T22:00:49.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.982+0000 7f6d6aa00700 1 -- 172.21.15.151:0/4108108026 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d640b34d0 con 0x563a92885f30 2024-05-21T22:00:49.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.983+0000 7f6d8a6cdb80 1 -- 172.21.15.151:0/4108108026 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a92885f30 msgr2=0x563a9287fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:49.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.983+0000 7f6d8a6cdb80 1 --2- 172.21.15.151:0/4108108026 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a92885f30 0x563a9287fb20 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f6d64006010 tx=0x7f6d64005f00 comp rx=0 tx=0).stop 2024-05-21T22:00:49.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.983+0000 7f6d8a6cdb80 1 -- 172.21.15.151:0/4108108026 shutdown_connections 2024-05-21T22:00:49.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.983+0000 7f6d8a6cdb80 1 --2- 172.21.15.151:0/4108108026 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563a92880140 0x563a928805c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:49.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.983+0000 7f6d8a6cdb80 1 --2- 172.21.15.151:0/4108108026 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563a92885610 0x563a928859f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:49.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.983+0000 7f6d8a6cdb80 1 --2- 172.21.15.151:0/4108108026 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a92885f30 0x563a9287fb20 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:49.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.983+0000 7f6d8a6cdb80 1 -- 172.21.15.151:0/4108108026 >> 172.21.15.151:0/4108108026 conn(0x563a927adbe0 msgr2=0x563a9287a690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:00:49.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.983+0000 7f6d8a6cdb80 1 -- 172.21.15.151:0/4108108026 shutdown_connections 2024-05-21T22:00:49.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.983+0000 7f6d8a6cdb80 1 -- 172.21.15.151:0/4108108026 wait complete. 2024-05-21T22:00:49.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.984+0000 7f6d8a6cdb80 1 -- start start 2024-05-21T22:00:49.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.984+0000 7f6d8a6cdb80 1 Processor -- start 2024-05-21T22:00:49.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.984+0000 7f6d8a6cdb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563a92880140 0x563a9288bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:49.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.985+0000 7f6d8a6cdb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a92885610 0x563a92889000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:49.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.985+0000 7f6d8a6cdb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563a92885f30 0x563a92889540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:49.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.985+0000 7f6d8a6cdb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x563a926d3fa0 con 0x563a92885f30 2024-05-21T22:00:49.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.985+0000 7f6d8a6cdb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x563a9276e760 con 0x563a92880140 2024-05-21T22:00:49.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.985+0000 7f6d8a6cdb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x563a9288c7a0 con 0x563a92885610 2024-05-21T22:00:49.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.985+0000 7f6d8a6cdb80 10 client.0 _ll_register_callbacks cb 0x563a92777c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:00:49.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.985+0000 7f6d7d400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563a92880140 0x563a9288bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:49.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.985+0000 7f6d7ca00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a92885610 0x563a92889000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:49.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.985+0000 7f6d7d400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563a92880140 0x563a9288bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:53282/0 (socket says 172.21.15.151:53282) 2024-05-21T22:00:49.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.985+0000 7f6d7d400700 1 -- 172.21.15.151:0/1091022172 learned_addr learned my addr 172.21.15.151:0/1091022172 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:00:49.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.985+0000 7f6d7de00700 1 --2- 172.21.15.151:0/1091022172 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563a92885f30 0x563a92889540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:49.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.985+0000 7f6d7ca00700 1 -- 172.21.15.151:0/1091022172 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563a92880140 msgr2=0x563a9288bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:49.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.985+0000 7f6d7ca00700 1 --2- 172.21.15.151:0/1091022172 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563a92880140 0x563a9288bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:49.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.985+0000 7f6d7ca00700 1 -- 172.21.15.151:0/1091022172 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563a92885f30 msgr2=0x563a92889540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:49.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.985+0000 7f6d7ca00700 1 --2- 172.21.15.151:0/1091022172 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563a92885f30 0x563a92889540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:49.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.985+0000 7f6d7ca00700 1 -- 172.21.15.151:0/1091022172 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a9271d8d0 con 0x563a92885610 2024-05-21T22:00:49.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.986+0000 7f6d7ca00700 1 --2- 172.21.15.151:0/1091022172 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a92885610 0x563a92889000 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f6d640aa690 tx=0x7f6d64092280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=629c594802e35435 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:00:49.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.986+0000 7f6d5f400700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-21T22:00:49.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.986+0000 7f6d5f400700 1 -- 172.21.15.151:0/1091022172 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d640c4060 con 0x563a92885610 2024-05-21T22:00:49.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.986+0000 7f6d8a6cdb80 1 -- 172.21.15.151:0/1091022172 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d640aa210 con 0x563a92885610 2024-05-21T22:00:49.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.986+0000 7f6d5f400700 1 -- 172.21.15.151:0/1091022172 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f6d640aab70 con 0x563a92885610 2024-05-21T22:00:49.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.986+0000 7f6d5f400700 1 -- 172.21.15.151:0/1091022172 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6d640a9e70 con 0x563a92885610 2024-05-21T22:00:49.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.986+0000 7f6d8a6cdb80 10 client.26434 fetch_fsmap learned FSMap version 29 2024-05-21T22:00:49.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.986+0000 7f6d8a6cdb80 1 -- 172.21.15.151:0/1091022172 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563a9279e060 con 0x563a92885610 2024-05-21T22:00:49.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.986+0000 7f6d5f400700 1 -- 172.21.15.151:0/1091022172 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d640aa210 con 0x563a92885610 2024-05-21T22:00:49.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.987+0000 7f6d5f400700 1 -- 172.21.15.151:0/1091022172 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f6d64005110 con 0x563a92885610 2024-05-21T22:00:49.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.987+0000 7f6d8a6cdb80 10 client.26434 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:00:49.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.987+0000 7f6d8a6cdb80 10 client.26434 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-05-21T22:00:49.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.987+0000 7f6d8a6cdb80 20 client.26434 populate_metadata read hostname 'smithi151' 2024-05-21T22:00:49.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.987+0000 7f6d8a6cdb80 1 --2- 172.21.15.151:0/1091022172 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x563a928990a0 0x563a9289b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:49.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.987+0000 7f6d8a6cdb80 4 client.26434 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T22:00:49.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.987+0000 7f6d8a6cdb80 1 -- 172.21.15.151:0/1091022172 --> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563a927dfd90 con 0x563a928990a0 2024-05-21T22:00:49.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.987+0000 7f6d7d400700 1 --2- 172.21.15.151:0/1091022172 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x563a928990a0 0x563a9289b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:49.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.988+0000 7f6d7d400700 1 --2- 172.21.15.151:0/1091022172 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x563a928990a0 0x563a9289b490 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=964a304ce82bc2f6 server_cookie=40824340ce12bae9 in_seq=0 out_seq=0 2024-05-21T22:00:49.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.988+0000 7f6d5f400700 10 client.26434 ms_handle_connect on v2:172.21.15.164:6836/1554609572 2024-05-21T22:00:49.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.989+0000 7f6d5f400700 1 -- 172.21.15.151:0/1091022172 <== mds.2 v2:172.21.15.164:6836/1554609572 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x563a927dfd90 con 0x563a928990a0 2024-05-21T22:00:49.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.989+0000 7f6d5f400700 10 client.26434 handle_command_reply: tid=0 2024-05-21T22:00:49.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.989+0000 7f6d8a6cdb80 1 client.26434 shutdown 2024-05-21T22:00:49.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.989+0000 7f6d8a6cdb80 1 -- 172.21.15.151:0/1091022172 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a92885610 msgr2=0x563a92889000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:49.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.989+0000 7f6d8a6cdb80 1 --2- 172.21.15.151:0/1091022172 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a92885610 0x563a92889000 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f6d640aa690 tx=0x7f6d64092280 comp rx=0 tx=0).stop 2024-05-21T22:00:49.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.989+0000 7f6d8a6cdb80 1 -- 172.21.15.151:0/1091022172 shutdown_connections 2024-05-21T22:00:49.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.989+0000 7f6d8a6cdb80 1 --2- 172.21.15.151:0/1091022172 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x563a928990a0 0x563a9289b490 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:49.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.989+0000 7f6d8a6cdb80 1 --2- 172.21.15.151:0/1091022172 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563a92880140 0x563a9288bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:49.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.989+0000 7f6d8a6cdb80 1 --2- 172.21.15.151:0/1091022172 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563a92885f30 0x563a92889540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:49.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.989+0000 7f6d8a6cdb80 1 --2- 172.21.15.151:0/1091022172 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a92885610 0x563a92889000 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:49.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.989+0000 7f6d5f400700 0 client.26434 ms_handle_reset on v2:172.21.15.164:6836/1554609572 2024-05-21T22:00:49.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.989+0000 7f6d8a6cdb80 1 -- 172.21.15.151:0/1091022172 >> 172.21.15.151:0/1091022172 conn(0x563a927adbe0 msgr2=0x563a92879d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:00:49.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.989+0000 7f6d8a6cdb80 1 -- 172.21.15.151:0/1091022172 shutdown_connections 2024-05-21T22:00:49.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.989+0000 7f6d8a6cdb80 1 -- 172.21.15.151:0/1091022172 wait complete. 2024-05-21T22:00:49.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:49.990+0000 7f6d8a6cdb80 20 client.26434 trim_cache size 0 max 16384 2024-05-21T22:00:50.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.006+0000 7f6d8a6cdb80 1 Processor -- start 2024-05-21T22:00:50.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.006+0000 7f6d8a6cdb80 1 -- start start 2024-05-21T22:00:50.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.006+0000 7f6d8a6cdb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a92880140 0x563a9287fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:50.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.006+0000 7f6d8a6cdb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563a92885610 0x563a9287b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:50.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.006+0000 7f6d8a6cdb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563a92885f30 0x563a928894f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:50.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.006+0000 7f6d8a6cdb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6d781afec0 con 0x563a92885610 2024-05-21T22:00:50.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.006+0000 7f6d8a6cdb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6d781b1a00 con 0x563a92885f30 2024-05-21T22:00:50.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.006+0000 7f6d8a6cdb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6d781b0590 con 0x563a92880140 2024-05-21T22:00:50.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.007+0000 7f6d7ca00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563a92885610 0x563a9287b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:50.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.007+0000 7f6d7de00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563a92885f30 0x563a928894f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:50.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.007+0000 7f6d7d400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a92880140 0x563a9287fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:50.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.007+0000 7f6d7ca00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563a92885610 0x563a9287b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:50028/0 (socket says 172.21.15.151:50028) 2024-05-21T22:00:50.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.007+0000 7f6d7ca00700 1 -- 172.21.15.151:0/3473564282 learned_addr learned my addr 172.21.15.151:0/3473564282 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:00:50.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.007+0000 7f6d7d400700 1 -- 172.21.15.151:0/3473564282 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563a92885f30 msgr2=0x563a928894f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:50.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.007+0000 7f6d7d400700 1 --2- 172.21.15.151:0/3473564282 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563a92885f30 0x563a928894f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:50.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.007+0000 7f6d7d400700 1 -- 172.21.15.151:0/3473564282 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563a92885610 msgr2=0x563a9287b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:50.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.007+0000 7f6d7d400700 1 --2- 172.21.15.151:0/3473564282 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563a92885610 0x563a9287b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:50.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.007+0000 7f6d7d400700 1 -- 172.21.15.151:0/3473564282 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a92885d30 con 0x563a92880140 2024-05-21T22:00:50.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.007+0000 7f6d7d400700 1 --2- 172.21.15.151:0/3473564282 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a92880140 0x563a9287fb20 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x563a9276e800 tx=0x7f6d6c009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27e592d0027d8c59 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:00:50.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.008+0000 7f6d6a000700 1 -- 172.21.15.151:0/3473564282 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c004a30 con 0x563a92880140 2024-05-21T22:00:50.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.008+0000 7f6d6a000700 1 -- 172.21.15.151:0/3473564282 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d6c0054d0 con 0x563a92880140 2024-05-21T22:00:50.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.008+0000 7f6d6a000700 1 -- 172.21.15.151:0/3473564282 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c003c10 con 0x563a92880140 2024-05-21T22:00:50.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.008+0000 7f6d8a6cdb80 1 -- 172.21.15.151:0/3473564282 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a92880140 msgr2=0x563a9287fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:50.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.008+0000 7f6d8a6cdb80 1 --2- 172.21.15.151:0/3473564282 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a92880140 0x563a9287fb20 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x563a9276e800 tx=0x7f6d6c009260 comp rx=0 tx=0).stop 2024-05-21T22:00:50.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.009+0000 7f6d8a6cdb80 1 -- 172.21.15.151:0/3473564282 shutdown_connections 2024-05-21T22:00:50.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.009+0000 7f6d8a6cdb80 1 --2- 172.21.15.151:0/3473564282 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563a92885f30 0x563a928894f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:50.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.009+0000 7f6d8a6cdb80 1 --2- 172.21.15.151:0/3473564282 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a92880140 0x563a9287fb20 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:50.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.009+0000 7f6d8a6cdb80 1 --2- 172.21.15.151:0/3473564282 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563a92885610 0x563a9287b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:50.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.009+0000 7f6d8a6cdb80 1 -- 172.21.15.151:0/3473564282 >> 172.21.15.151:0/3473564282 conn(0x563a927adbe0 msgr2=0x563a92888de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:00:50.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.009+0000 7f6d8a6cdb80 1 -- 172.21.15.151:0/3473564282 shutdown_connections 2024-05-21T22:00:50.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.009+0000 7f6d8a6cdb80 1 -- 172.21.15.151:0/3473564282 wait complete. 2024-05-21T22:00:50.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.009+0000 7f6d8a6cdb80 1 -- start start 2024-05-21T22:00:50.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.009+0000 7f6d8a6cdb80 1 Processor -- start 2024-05-21T22:00:50.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.010+0000 7f6d8a6cdb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563a92880140 0x563a92892050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:50.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.010+0000 7f6d8a6cdb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a92885610 0x563a928a0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:50.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.010+0000 7f6d8a6cdb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563a92885f30 0x563a928a0840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:50.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.010+0000 7f6d8a6cdb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6d781b1330 con 0x563a92885f30 2024-05-21T22:00:50.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.010+0000 7f6d8a6cdb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6d781b0c60 con 0x563a92880140 2024-05-21T22:00:50.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.010+0000 7f6d8a6cdb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6d781ade60 con 0x563a92885610 2024-05-21T22:00:50.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.010+0000 7f6d8a6cdb80 10 client.0 _ll_register_callbacks cb 0x563a92777c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:00:50.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.010+0000 7f6d7d400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563a92880140 0x563a92892050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:50.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.010+0000 7f6d7d400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563a92880140 0x563a92892050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:53308/0 (socket says 172.21.15.151:53308) 2024-05-21T22:00:50.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.010+0000 7f6d7de00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563a92885f30 0x563a928a0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:50.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.010+0000 7f6d7d400700 1 -- 172.21.15.151:0/1362117757 learned_addr learned my addr 172.21.15.151:0/1362117757 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:00:50.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.010+0000 7f6d7ca00700 1 --2- 172.21.15.151:0/1362117757 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a92885610 0x563a928a0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:50.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.011+0000 7f6d7d400700 1 -- 172.21.15.151:0/1362117757 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a92885610 msgr2=0x563a928a0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:50.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.011+0000 7f6d7d400700 1 --2- 172.21.15.151:0/1362117757 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a92885610 0x563a928a0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:50.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.011+0000 7f6d7d400700 1 -- 172.21.15.151:0/1362117757 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563a92885f30 msgr2=0x563a928a0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:50.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.011+0000 7f6d7d400700 1 --2- 172.21.15.151:0/1362117757 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563a92885f30 0x563a928a0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:50.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.011+0000 7f6d7d400700 1 -- 172.21.15.151:0/1362117757 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a928a1080 con 0x563a92880140 2024-05-21T22:00:50.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.011+0000 7f6d7d400700 1 --2- 172.21.15.151:0/1362117757 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563a92880140 0x563a92892050 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c005020 tx=0x7f6d6c005050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=400308578e26780d server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:00:50.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.011+0000 7f6d5f400700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-21T22:00:50.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.011+0000 7f6d5f400700 1 -- 172.21.15.151:0/1362117757 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c02b070 con 0x563a92880140 2024-05-21T22:00:50.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.011+0000 7f6d8a6cdb80 1 -- 172.21.15.151:0/1362117757 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d6c0054d0 con 0x563a92880140 2024-05-21T22:00:50.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.011+0000 7f6d5f400700 1 -- 172.21.15.151:0/1362117757 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f6d6c0047c0 con 0x563a92880140 2024-05-21T22:00:50.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.012+0000 7f6d5f400700 1 -- 172.21.15.151:0/1362117757 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6d6c037050 con 0x563a92880140 2024-05-21T22:00:50.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.012+0000 7f6d8a6cdb80 10 client.26426 fetch_fsmap learned FSMap version 29 2024-05-21T22:00:50.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.012+0000 7f6d8a6cdb80 1 -- 172.21.15.151:0/1362117757 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563a929039a0 con 0x563a92880140 2024-05-21T22:00:50.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.012+0000 7f6d5f400700 1 -- 172.21.15.151:0/1362117757 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d6c0054d0 con 0x563a92880140 2024-05-21T22:00:50.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.012+0000 7f6d5f400700 1 -- 172.21.15.151:0/1362117757 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f6d6c027430 con 0x563a92880140 2024-05-21T22:00:50.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.012+0000 7f6d8a6cdb80 10 client.26426 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:00:50.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.012+0000 7f6d8a6cdb80 10 client.26426 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-05-21T22:00:50.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.012+0000 7f6d8a6cdb80 20 client.26426 populate_metadata read hostname 'smithi151' 2024-05-21T22:00:50.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.012+0000 7f6d8a6cdb80 1 --2- 172.21.15.151:0/1362117757 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x563a929040b0 0x563a929064d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:50.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.012+0000 7f6d8a6cdb80 4 client.26426 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T22:00:50.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.012+0000 7f6d8a6cdb80 1 -- 172.21.15.151:0/1362117757 --> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563a927d9570 con 0x563a929040b0 2024-05-21T22:00:50.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.013+0000 7f6d7ca00700 1 --2- 172.21.15.151:0/1362117757 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x563a929040b0 0x563a929064d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:50.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.013+0000 7f6d5f400700 10 client.26426 ms_handle_connect on v2:172.21.15.164:6836/1554609572 2024-05-21T22:00:50.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.013+0000 7f6d7ca00700 1 --2- 172.21.15.151:0/1362117757 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x563a929040b0 0x563a929064d0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=369182d73c04d411 server_cookie=e8e3752236b402d9 in_seq=0 out_seq=0 2024-05-21T22:00:50.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.013+0000 7f6d5f400700 1 -- 172.21.15.151:0/1362117757 <== mds.2 v2:172.21.15.164:6836/1554609572 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563a927d9570 con 0x563a929040b0 2024-05-21T22:00:50.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.013+0000 7f6d5f400700 10 client.26426 handle_command_reply: tid=0 2024-05-21T22:00:50.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.013+0000 7f6d8a6cdb80 1 client.26426 shutdown 2024-05-21T22:00:50.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.013+0000 7f6d8a6cdb80 1 -- 172.21.15.151:0/1362117757 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563a92880140 msgr2=0x563a92892050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:50.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.013+0000 7f6d8a6cdb80 1 --2- 172.21.15.151:0/1362117757 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563a92880140 0x563a92892050 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c005020 tx=0x7f6d6c005050 comp rx=0 tx=0).stop 2024-05-21T22:00:50.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.013+0000 7f6d8a6cdb80 1 -- 172.21.15.151:0/1362117757 shutdown_connections 2024-05-21T22:00:50.017 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.013+0000 7f6d8a6cdb80 1 --2- 172.21.15.151:0/1362117757 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x563a929040b0 0x563a929064d0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:50.017 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.013+0000 7f6d8a6cdb80 1 --2- 172.21.15.151:0/1362117757 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563a92880140 0x563a92892050 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:50.017 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.014+0000 7f6d5f400700 0 client.26426 ms_handle_reset on v2:172.21.15.164:6836/1554609572 2024-05-21T22:00:50.017 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.014+0000 7f6d8a6cdb80 1 --2- 172.21.15.151:0/1362117757 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563a92885f30 0x563a928a0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:50.017 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.014+0000 7f6d8a6cdb80 1 --2- 172.21.15.151:0/1362117757 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563a92885610 0x563a928a0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:50.017 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.014+0000 7f6d8a6cdb80 1 -- 172.21.15.151:0/1362117757 >> 172.21.15.151:0/1362117757 conn(0x563a927adbe0 msgr2=0x563a92886350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:00:50.017 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.014+0000 7f6d8a6cdb80 1 -- 172.21.15.151:0/1362117757 shutdown_connections 2024-05-21T22:00:50.017 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.014+0000 7f6d8a6cdb80 1 -- 172.21.15.151:0/1362117757 wait complete. 2024-05-21T22:00:50.017 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.014+0000 7f6d8a6cdb80 20 client.26426 trim_cache size 0 max 16384 2024-05-21T22:00:50.017 INFO:teuthology.orchestra.run.smithi151.stdout:[] 2024-05-21T22:00:50.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.014+0000 7f6d7fe00700 1 -- 172.21.15.151:0/1250780958 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f6d600b29f0 msgr2=0x7f6d600b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:50.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.014+0000 7f6d7fe00700 1 --2- 172.21.15.151:0/1250780958 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f6d600b29f0 0x7f6d600b4eb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x563a92880b00 tx=0x7f6d70009920 comp rx=0 tx=0).stop 2024-05-21T22:00:50.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.015+0000 7f6d7fe00700 1 -- 172.21.15.151:0/1250780958 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d78106120 msgr2=0x7f6d7819c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:50.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.015+0000 7f6d7fe00700 1 --2- 172.21.15.151:0/1250780958 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d78106120 0x7f6d7819c3a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f6d64000c00 tx=0x7f6d640057c0 comp rx=0 tx=0).stop 2024-05-21T22:00:50.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.015+0000 7f6d7fe00700 1 -- 172.21.15.151:0/1250780958 shutdown_connections 2024-05-21T22:00:50.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.015+0000 7f6d7fe00700 1 --2- 172.21.15.151:0/1250780958 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f6d600b29f0 0x7f6d600b4eb0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:50.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.015+0000 7f6d7fe00700 1 --2- 172.21.15.151:0/1250780958 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6d7810b760 0x7f6d781a0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:50.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.015+0000 7f6d7fe00700 1 --2- 172.21.15.151:0/1250780958 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6d78105760 0x7f6d7819be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:50.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.015+0000 7f6d7fe00700 1 --2- 172.21.15.151:0/1250780958 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d78106120 0x7f6d7819c3a0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:50.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.015+0000 7f6d7fe00700 1 -- 172.21.15.151:0/1250780958 >> 172.21.15.151:0/1250780958 conn(0x7f6d78101250 msgr2=0x7f6d78101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:00:50.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.015+0000 7f6d7fe00700 1 -- 172.21.15.151:0/1250780958 shutdown_connections 2024-05-21T22:00:50.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:50.015+0000 7f6d7fe00700 1 -- 172.21.15.151:0/1250780958 wait complete. 2024-05-21T22:00:50.875 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-21T22:00:51.338 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:00:51.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:00:51 smithi151 ceph-mon[31623]: pgmap v160: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:00:51.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:00:51 smithi184 ceph-mon[36942]: pgmap v160: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:00:51.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:00:51 smithi164 ceph-mon[36269]: pgmap v160: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:00:53.544 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.542+0000 7f69c7c00700 1 -- 172.21.15.151:0/3101962256 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f69c010eeb0 msgr2=0x7f69c0116a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:53.544 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.542+0000 7f69c7c00700 1 --2- 172.21.15.151:0/3101962256 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f69c010eeb0 0x7f69c0116a80 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f69b800b0d0 tx=0x7f69b800b3e0 comp rx=0 tx=0).stop 2024-05-21T22:00:53.544 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.542+0000 7f69c7c00700 1 -- 172.21.15.151:0/3101962256 shutdown_connections 2024-05-21T22:00:53.544 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.542+0000 7f69c7c00700 1 --2- 172.21.15.151:0/3101962256 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69c0109dd0 0x7f69c010a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:53.544 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.542+0000 7f69c7c00700 1 --2- 172.21.15.151:0/3101962256 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f69c010eeb0 0x7f69c0116a80 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:53.544 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.542+0000 7f69c7c00700 1 --2- 172.21.15.151:0/3101962256 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69c010a780 0x7f69c010e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:53.544 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.542+0000 7f69c7c00700 1 -- 172.21.15.151:0/3101962256 >> 172.21.15.151:0/3101962256 conn(0x7f69c0076d10 msgr2=0x7f69c0108360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:00:53.545 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.542+0000 7f69c7c00700 1 -- 172.21.15.151:0/3101962256 shutdown_connections 2024-05-21T22:00:53.545 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.542+0000 7f69c7c00700 1 -- 172.21.15.151:0/3101962256 wait complete. 2024-05-21T22:00:53.546 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.543+0000 7f69c7c00700 1 Processor -- start 2024-05-21T22:00:53.546 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.543+0000 7f69c7c00700 1 -- start start 2024-05-21T22:00:53.546 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.544+0000 7f69c7c00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69c0109dd0 0x7f69c01a0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:53.546 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.544+0000 7f69c7c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f69c010a780 0x7f69c01a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:53.546 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.544+0000 7f69c7c00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69c010eeb0 0x7f69c01a4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:53.546 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.544+0000 7f69c7c00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f69c0119d40 con 0x7f69c010a780 2024-05-21T22:00:53.546 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.544+0000 7f69c7c00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f69c0119bc0 con 0x7f69c010eeb0 2024-05-21T22:00:53.547 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.544+0000 7f69c7c00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f69c0119ec0 con 0x7f69c0109dd0 2024-05-21T22:00:53.547 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.544+0000 7f69bfe00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f69c010a780 0x7f69c01a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:53.547 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.544+0000 7f69c5200700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69c0109dd0 0x7f69c01a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:53.547 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.544+0000 7f69c5200700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69c0109dd0 0x7f69c01a0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:45074/0 (socket says 172.21.15.151:45074) 2024-05-21T22:00:53.547 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.544+0000 7f69c5200700 1 -- 172.21.15.151:0/3419813788 learned_addr learned my addr 172.21.15.151:0/3419813788 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:00:53.547 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.544+0000 7f69bfe00700 1 --2- 172.21.15.151:0/3419813788 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f69c010a780 0x7f69c01a06d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:50060/0 (socket says 172.21.15.151:50060) 2024-05-21T22:00:53.547 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.544+0000 7f69c5c00700 1 --2- 172.21.15.151:0/3419813788 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69c010eeb0 0x7f69c01a4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:53.547 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.545+0000 7f69c5200700 1 -- 172.21.15.151:0/3419813788 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69c010eeb0 msgr2=0x7f69c01a4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:53.547 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.545+0000 7f69c5200700 1 --2- 172.21.15.151:0/3419813788 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69c010eeb0 0x7f69c01a4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:53.547 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.545+0000 7f69c5200700 1 -- 172.21.15.151:0/3419813788 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f69c010a780 msgr2=0x7f69c01a06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:53.547 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.545+0000 7f69c5200700 1 --2- 172.21.15.151:0/3419813788 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f69c010a780 0x7f69c01a06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:53.547 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.545+0000 7f69c5200700 1 -- 172.21.15.151:0/3419813788 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69c01a51a0 con 0x7f69c0109dd0 2024-05-21T22:00:53.547 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.545+0000 7f69c5c00700 1 --2- 172.21.15.151:0/3419813788 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69c010eeb0 0x7f69c01a4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:00:53.547 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.545+0000 7f69bfe00700 1 --2- 172.21.15.151:0/3419813788 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f69c010a780 0x7f69c01a06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:00:53.548 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.545+0000 7f69c5200700 1 --2- 172.21.15.151:0/3419813788 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69c0109dd0 0x7f69c01a0190 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f69b400e9a0 tx=0x7f69b400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:00:53.548 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.545+0000 7f69bd600700 1 -- 172.21.15.151:0/3419813788 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b400ca10 con 0x7f69c0109dd0 2024-05-21T22:00:53.548 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.545+0000 7f69bd600700 1 -- 172.21.15.151:0/3419813788 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69b400cbb0 con 0x7f69c0109dd0 2024-05-21T22:00:53.548 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.545+0000 7f69c7c00700 1 -- 172.21.15.151:0/3419813788 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69c01a5490 con 0x7f69c0109dd0 2024-05-21T22:00:53.548 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.545+0000 7f69bd600700 1 -- 172.21.15.151:0/3419813788 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b4018680 con 0x7f69c0109dd0 2024-05-21T22:00:53.548 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.545+0000 7f69c7c00700 1 -- 172.21.15.151:0/3419813788 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69c01acd10 con 0x7f69c0109dd0 2024-05-21T22:00:53.550 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.549+0000 7f69bd600700 1 -- 172.21.15.151:0/3419813788 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7f69b4010430 con 0x7f69c0109dd0 2024-05-21T22:00:53.551 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.550+0000 7f69d2518b80 1 Processor -- start 2024-05-21T22:00:53.551 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.550+0000 7f69d2518b80 1 -- start start 2024-05-21T22:00:53.552 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.551+0000 7f69bd600700 1 --2- 172.21.15.151:0/3419813788 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f69a80b2a70 0x7f69a80b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:53.552 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.551+0000 7f69d2518b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f78308beb0 0x55f78308c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:53.553 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.551+0000 7f69d2518b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f78308c7d0 0x55f783086260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:53.553 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.551+0000 7f69d2518b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f783086880 0x55f783086ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:53.553 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.551+0000 7f69d2518b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55f782fe4e40 con 0x55f78308c7d0 2024-05-21T22:00:53.553 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.551+0000 7f69d2518b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55f782fd49e0 con 0x55f783086880 2024-05-21T22:00:53.553 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.551+0000 7f69d2518b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55f782fd8a20 con 0x55f78308beb0 2024-05-21T22:00:53.553 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.551+0000 7f69bd600700 1 -- 172.21.15.151:0/3419813788 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f69b40e2760 con 0x7f69c0109dd0 2024-05-21T22:00:53.553 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.551+0000 7f69c5c00700 1 --2- 172.21.15.151:0/3419813788 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f69a80b2a70 0x7f69a80b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:53.553 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.551+0000 7f69bfe00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f78308c7d0 0x55f783086260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:53.553 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.551+0000 7f69c5200700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f78308beb0 0x55f78308c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:53.553 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.551+0000 7f69bfe00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f78308c7d0 0x55f783086260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:50070/0 (socket says 172.21.15.151:50070) 2024-05-21T22:00:53.553 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.551+0000 7f69bfe00700 1 -- 172.21.15.151:0/1227437047 learned_addr learned my addr 172.21.15.151:0/1227437047 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:00:53.553 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.551+0000 7f69c5c00700 1 --2- 172.21.15.151:0/1227437047 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f783086880 0x55f783086ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:53.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.552+0000 7f69c5200700 1 -- 172.21.15.151:0/1227437047 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f783086880 msgr2=0x55f783086ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:53.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.552+0000 7f69c5200700 1 --2- 172.21.15.151:0/1227437047 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f783086880 0x55f783086ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:53.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.552+0000 7f69c5200700 1 -- 172.21.15.151:0/1227437047 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f78308c7d0 msgr2=0x55f783086260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:53.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.552+0000 7f69c5200700 1 --2- 172.21.15.151:0/1227437047 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f78308c7d0 0x55f783086260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:53.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.552+0000 7f69c5200700 1 -- 172.21.15.151:0/1227437047 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f782f9d420 con 0x55f78308beb0 2024-05-21T22:00:53.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.552+0000 7f69bfe00700 1 --2- 172.21.15.151:0/1227437047 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f78308c7d0 0x55f783086260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T22:00:53.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.552+0000 7f69c5c00700 1 --2- 172.21.15.151:0/3419813788 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f69a80b2a70 0x7f69a80b4f30 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f69b8006130 tx=0x7f69b80093c0 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:00:53.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.552+0000 7f69c5200700 1 --2- 172.21.15.151:0/1227437047 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f78308beb0 0x55f78308c290 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f69b40dfe60 tx=0x7f69b40b2230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5780f07a4689a1f server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:00:53.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.552+0000 7f69b2000700 1 -- 172.21.15.151:0/1227437047 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b40a99f0 con 0x55f78308beb0 2024-05-21T22:00:53.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.553+0000 7f69b2000700 1 -- 172.21.15.151:0/1227437047 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69b40adda0 con 0x55f78308beb0 2024-05-21T22:00:53.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.553+0000 7f69b2000700 1 -- 172.21.15.151:0/1227437047 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b40a9c50 con 0x55f78308beb0 2024-05-21T22:00:53.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.553+0000 7f69d2518b80 1 -- 172.21.15.151:0/1227437047 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f78308beb0 msgr2=0x55f78308c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:53.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.553+0000 7f69d2518b80 1 --2- 172.21.15.151:0/1227437047 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f78308beb0 0x55f78308c290 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f69b40dfe60 tx=0x7f69b40b2230 comp rx=0 tx=0).stop 2024-05-21T22:00:53.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.554+0000 7f69d2518b80 1 -- 172.21.15.151:0/1227437047 shutdown_connections 2024-05-21T22:00:53.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.554+0000 7f69d2518b80 1 --2- 172.21.15.151:0/1227437047 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f783086880 0x55f783086ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:53.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.554+0000 7f69d2518b80 1 --2- 172.21.15.151:0/1227437047 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f78308beb0 0x55f78308c290 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:53.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.554+0000 7f69d2518b80 1 --2- 172.21.15.151:0/1227437047 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f78308c7d0 0x55f783086260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:53.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.554+0000 7f69d2518b80 1 -- 172.21.15.151:0/1227437047 >> 172.21.15.151:0/1227437047 conn(0x55f782fb4be0 msgr2=0x55f783081010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:00:53.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.554+0000 7f69d2518b80 1 -- 172.21.15.151:0/1227437047 shutdown_connections 2024-05-21T22:00:53.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.554+0000 7f69d2518b80 1 -- 172.21.15.151:0/1227437047 wait complete. 2024-05-21T22:00:53.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.554+0000 7f69d2518b80 1 -- start start 2024-05-21T22:00:53.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.555+0000 7f69d2518b80 1 Processor -- start 2024-05-21T22:00:53.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.555+0000 7f69d2518b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f783086880 0x55f783092530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:53.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.555+0000 7f69d2518b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f78308beb0 0x55f78308f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:53.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.555+0000 7f69d2518b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f78308c7d0 0x55f78308fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:53.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.555+0000 7f69d2518b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55f782edafa0 con 0x55f78308beb0 2024-05-21T22:00:53.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.555+0000 7f69d2518b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55f782f75760 con 0x55f78308c7d0 2024-05-21T22:00:53.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.555+0000 7f69d2518b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55f783093240 con 0x55f783086880 2024-05-21T22:00:53.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.555+0000 7f69d2518b80 10 client.0 _ll_register_callbacks cb 0x55f782f7ec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:00:53.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.555+0000 7f69bfe00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f78308beb0 0x55f78308f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:53.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.555+0000 7f69c5200700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f783086880 0x55f783092530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:53.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.556+0000 7f69c5200700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f783086880 0x55f783092530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:45092/0 (socket says 172.21.15.151:45092) 2024-05-21T22:00:53.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.556+0000 7f69c5200700 1 -- 172.21.15.151:0/82625107 learned_addr learned my addr 172.21.15.151:0/82625107 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:00:53.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.556+0000 7f69c5c00700 1 --2- 172.21.15.151:0/82625107 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f78308c7d0 0x55f78308fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:53.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.556+0000 7f69c5200700 1 -- 172.21.15.151:0/82625107 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f78308c7d0 msgr2=0x55f78308fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:53.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.556+0000 7f69c5200700 1 --2- 172.21.15.151:0/82625107 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f78308c7d0 0x55f78308fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:53.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.556+0000 7f69c5200700 1 -- 172.21.15.151:0/82625107 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f78308beb0 msgr2=0x55f78308f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:53.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.556+0000 7f69c5200700 1 --2- 172.21.15.151:0/82625107 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f78308beb0 0x55f78308f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:53.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.556+0000 7f69c5200700 1 -- 172.21.15.151:0/82625107 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f782f248d0 con 0x55f783086880 2024-05-21T22:00:53.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.556+0000 7f69c5c00700 1 --2- 172.21.15.151:0/82625107 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f78308c7d0 0x55f78308fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:00:53.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.556+0000 7f69c5200700 1 --2- 172.21.15.151:0/82625107 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f783086880 0x55f783092530 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f69b400e970 tx=0x7f69b40a9e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9a518888549ac29 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:00:53.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.556+0000 7f69a6a00700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-21T22:00:53.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.556+0000 7f69a6a00700 1 -- 172.21.15.151:0/82625107 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b40d1020 con 0x55f783086880 2024-05-21T22:00:53.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.557+0000 7f69a6a00700 1 -- 172.21.15.151:0/82625107 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f69b40d1950 con 0x55f783086880 2024-05-21T22:00:53.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.557+0000 7f69d2518b80 1 -- 172.21.15.151:0/82625107 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69b40adda0 con 0x55f783086880 2024-05-21T22:00:53.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.557+0000 7f69a6a00700 1 -- 172.21.15.151:0/82625107 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f69b40b2850 con 0x55f783086880 2024-05-21T22:00:53.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.557+0000 7f69d2518b80 10 client.26452 fetch_fsmap learned FSMap version 29 2024-05-21T22:00:53.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.557+0000 7f69d2518b80 1 -- 172.21.15.151:0/82625107 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f782fa5060 con 0x55f783086880 2024-05-21T22:00:53.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.557+0000 7f69a6a00700 1 -- 172.21.15.151:0/82625107 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69b40adda0 con 0x55f783086880 2024-05-21T22:00:53.561 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.557+0000 7f69a6a00700 1 -- 172.21.15.151:0/82625107 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f69b40d11c0 con 0x55f783086880 2024-05-21T22:00:53.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.558+0000 7f69d2518b80 10 client.26452 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:00:53.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.558+0000 7f69d2518b80 10 client.26452 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-21T22:00:53.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.558+0000 7f69d2518b80 20 client.26452 populate_metadata read hostname 'smithi151' 2024-05-21T22:00:53.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.558+0000 7f69d2518b80 1 --2- 172.21.15.151:0/82625107 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x55f78309fc40 0x55f7830a2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:53.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.558+0000 7f69d2518b80 4 client.26452 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T22:00:53.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.558+0000 7f69d2518b80 1 -- 172.21.15.151:0/82625107 --> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f782fe6d90 con 0x55f78309fc40 2024-05-21T22:00:53.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.558+0000 7f69bfe00700 1 --2- 172.21.15.151:0/82625107 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x55f78309fc40 0x55f7830a2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:53.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.558+0000 7f69bfe00700 1 --2- 172.21.15.151:0/82625107 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x55f78309fc40 0x55f7830a2030 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e67448179e849bd1 server_cookie=a4568d224b3fe796 in_seq=0 out_seq=0 2024-05-21T22:00:53.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.558+0000 7f69a6a00700 10 client.26452 ms_handle_connect on v2:172.21.15.184:6838/3059935143 2024-05-21T22:00:53.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.559+0000 7f69a6a00700 1 -- 172.21.15.151:0/82625107 <== mds.1 v2:172.21.15.184:6838/3059935143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55f782fe6d90 con 0x55f78309fc40 2024-05-21T22:00:53.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.559+0000 7f69a6a00700 10 client.26452 handle_command_reply: tid=0 2024-05-21T22:00:53.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.559+0000 7f69d2518b80 1 client.26452 shutdown 2024-05-21T22:00:53.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.559+0000 7f69d2518b80 1 -- 172.21.15.151:0/82625107 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f783086880 msgr2=0x55f783092530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:53.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.559+0000 7f69d2518b80 1 --2- 172.21.15.151:0/82625107 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f783086880 0x55f783092530 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f69b400e970 tx=0x7f69b40a9e60 comp rx=0 tx=0).stop 2024-05-21T22:00:53.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.560+0000 7f69d2518b80 1 -- 172.21.15.151:0/82625107 shutdown_connections 2024-05-21T22:00:53.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.560+0000 7f69d2518b80 1 --2- 172.21.15.151:0/82625107 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x55f78309fc40 0x55f7830a2030 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:53.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.560+0000 7f69d2518b80 1 --2- 172.21.15.151:0/82625107 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f78308c7d0 0x55f78308fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:53.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.560+0000 7f69d2518b80 1 --2- 172.21.15.151:0/82625107 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f783086880 0x55f783092530 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:53.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.560+0000 7f69d2518b80 1 --2- 172.21.15.151:0/82625107 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f78308beb0 0x55f78308f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:53.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.560+0000 7f69a6a00700 0 client.26452 ms_handle_reset on v2:172.21.15.184:6838/3059935143 2024-05-21T22:00:53.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.560+0000 7f69d2518b80 1 -- 172.21.15.151:0/82625107 >> 172.21.15.151:0/82625107 conn(0x55f782fb4be0 msgr2=0x55f783085250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:00:53.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.560+0000 7f69d2518b80 1 -- 172.21.15.151:0/82625107 shutdown_connections 2024-05-21T22:00:53.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.560+0000 7f69d2518b80 1 -- 172.21.15.151:0/82625107 wait complete. 2024-05-21T22:00:53.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.560+0000 7f69d2518b80 20 client.26452 trim_cache size 0 max 16384 2024-05-21T22:00:53.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.576+0000 7f69d2518b80 1 Processor -- start 2024-05-21T22:00:53.578 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.576+0000 7f69d2518b80 1 -- start start 2024-05-21T22:00:53.578 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.577+0000 7f69d2518b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f783086880 0x55f783086260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:53.578 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.577+0000 7f69d2518b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f78308beb0 0x55f783092530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:53.578 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.577+0000 7f69d2518b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f78308c7d0 0x55f7830899c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:53.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.577+0000 7f69d2518b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f69c01b43b0 con 0x55f783086880 2024-05-21T22:00:53.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.577+0000 7f69d2518b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f69c01b5ef0 con 0x55f78308c7d0 2024-05-21T22:00:53.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.577+0000 7f69d2518b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f69c01b4a80 con 0x55f78308beb0 2024-05-21T22:00:53.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.577+0000 7f69c5200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f783086880 0x55f783086260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:53.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.577+0000 7f69c5200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f783086880 0x55f783086260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:50088/0 (socket says 172.21.15.151:50088) 2024-05-21T22:00:53.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.577+0000 7f69c5200700 1 -- 172.21.15.151:0/3856206747 learned_addr learned my addr 172.21.15.151:0/3856206747 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:00:53.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.577+0000 7f69bfe00700 1 --2- 172.21.15.151:0/3856206747 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f78308beb0 0x55f783092530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:53.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.577+0000 7f69c5c00700 1 --2- 172.21.15.151:0/3856206747 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f78308c7d0 0x55f7830899c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:53.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.578+0000 7f69c5200700 1 -- 172.21.15.151:0/3856206747 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f78308beb0 msgr2=0x55f783092530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:53.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.578+0000 7f69c5200700 1 --2- 172.21.15.151:0/3856206747 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f78308beb0 0x55f783092530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:53.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.578+0000 7f69c5200700 1 -- 172.21.15.151:0/3856206747 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f78308c7d0 msgr2=0x55f7830899c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:53.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.578+0000 7f69c5200700 1 --2- 172.21.15.151:0/3856206747 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f78308c7d0 0x55f7830899c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:53.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.578+0000 7f69c5200700 1 -- 172.21.15.151:0/3856206747 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f78308de60 con 0x55f783086880 2024-05-21T22:00:53.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.578+0000 7f69c5c00700 1 --2- 172.21.15.151:0/3856206747 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f78308c7d0 0x55f7830899c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T22:00:53.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.578+0000 7f69c5200700 1 --2- 172.21.15.151:0/3856206747 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f783086880 0x55f783086260 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f69b40b2a10 tx=0x7f69b4009790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b844700e492c9a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:00:53.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.578+0000 7f69b1600700 1 -- 172.21.15.151:0/3856206747 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b40d1040 con 0x55f783086880 2024-05-21T22:00:53.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.578+0000 7f69b1600700 1 -- 172.21.15.151:0/3856206747 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69b40a99f0 con 0x55f783086880 2024-05-21T22:00:53.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.578+0000 7f69b1600700 1 -- 172.21.15.151:0/3856206747 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b40107a0 con 0x55f783086880 2024-05-21T22:00:53.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.579+0000 7f69d2518b80 1 -- 172.21.15.151:0/3856206747 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f783086880 msgr2=0x55f783086260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:53.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.579+0000 7f69d2518b80 1 --2- 172.21.15.151:0/3856206747 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f783086880 0x55f783086260 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f69b40b2a10 tx=0x7f69b4009790 comp rx=0 tx=0).stop 2024-05-21T22:00:53.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.579+0000 7f69d2518b80 1 -- 172.21.15.151:0/3856206747 shutdown_connections 2024-05-21T22:00:53.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.579+0000 7f69d2518b80 1 --2- 172.21.15.151:0/3856206747 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f78308c7d0 0x55f7830899c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:53.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.579+0000 7f69d2518b80 1 --2- 172.21.15.151:0/3856206747 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f783086880 0x55f783086260 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:53.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.579+0000 7f69d2518b80 1 --2- 172.21.15.151:0/3856206747 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f78308beb0 0x55f783092530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:53.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.579+0000 7f69d2518b80 1 -- 172.21.15.151:0/3856206747 >> 172.21.15.151:0/3856206747 conn(0x55f782fb4be0 msgr2=0x55f783082700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:00:53.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.579+0000 7f69d2518b80 1 -- 172.21.15.151:0/3856206747 shutdown_connections 2024-05-21T22:00:53.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.579+0000 7f69d2518b80 1 -- 172.21.15.151:0/3856206747 wait complete. 2024-05-21T22:00:53.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.580+0000 7f69d2518b80 1 -- start start 2024-05-21T22:00:53.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.580+0000 7f69d2518b80 1 Processor -- start 2024-05-21T22:00:53.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.580+0000 7f69d2518b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f783086880 0x55f7830987f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:53.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.580+0000 7f69d2518b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f78308beb0 0x55f783098d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:53.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.580+0000 7f69d2518b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f78308c7d0 0x55f7830a6d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:53.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.580+0000 7f69d2518b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f69c01b5820 con 0x55f783086880 2024-05-21T22:00:53.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.580+0000 7f69d2518b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f69c01b5150 con 0x55f78308beb0 2024-05-21T22:00:53.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.580+0000 7f69d2518b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f69c01b2350 con 0x55f78308c7d0 2024-05-21T22:00:53.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.580+0000 7f69d2518b80 10 client.0 _ll_register_callbacks cb 0x55f782f7ec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:00:53.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.581+0000 7f69c5c00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f78308c7d0 0x55f7830a6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:53.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.581+0000 7f69c5c00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f78308c7d0 0x55f7830a6d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:45112/0 (socket says 172.21.15.151:45112) 2024-05-21T22:00:53.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.581+0000 7f69c5c00700 1 -- 172.21.15.151:0/2637880278 learned_addr learned my addr 172.21.15.151:0/2637880278 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:00:53.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.581+0000 7f69bfe00700 1 --2- 172.21.15.151:0/2637880278 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f78308beb0 0x55f783098d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:53.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.581+0000 7f69c5200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f783086880 0x55f7830987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:53.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.581+0000 7f69c5c00700 1 -- 172.21.15.151:0/2637880278 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f78308beb0 msgr2=0x55f783098d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:53.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.581+0000 7f69c5c00700 1 --2- 172.21.15.151:0/2637880278 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f78308beb0 0x55f783098d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:53.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.581+0000 7f69c5c00700 1 -- 172.21.15.151:0/2637880278 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f783086880 msgr2=0x55f7830987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:53.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.581+0000 7f69c5c00700 1 --2- 172.21.15.151:0/2637880278 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f783086880 0x55f7830987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:53.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.581+0000 7f69c5c00700 1 -- 172.21.15.151:0/2637880278 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7830a75a0 con 0x55f78308c7d0 2024-05-21T22:00:53.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.581+0000 7f69bfe00700 1 --2- 172.21.15.151:0/2637880278 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f78308beb0 0x55f783098d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:00:53.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.581+0000 7f69c5c00700 1 --2- 172.21.15.151:0/2637880278 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f78308c7d0 0x55f7830a6d70 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f69b800b770 tx=0x7f69b800b3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5309d170870477cb server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:00:53.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.581+0000 7f69a6a00700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-21T22:00:53.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.582+0000 7f69a6a00700 1 -- 172.21.15.151:0/2637880278 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b80050c0 con 0x55f78308c7d0 2024-05-21T22:00:53.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.582+0000 7f69a6a00700 1 -- 172.21.15.151:0/2637880278 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f69b8005260 con 0x55f78308c7d0 2024-05-21T22:00:53.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.582+0000 7f69d2518b80 1 -- 172.21.15.151:0/2637880278 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69b40a99f0 con 0x55f78308c7d0 2024-05-21T22:00:53.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.582+0000 7f69a6a00700 1 -- 172.21.15.151:0/2637880278 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f69b8005460 con 0x55f78308c7d0 2024-05-21T22:00:53.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.582+0000 7f69d2518b80 10 client.26458 fetch_fsmap learned FSMap version 29 2024-05-21T22:00:53.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.582+0000 7f69d2518b80 1 -- 172.21.15.151:0/2637880278 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f78310a020 con 0x55f78308c7d0 2024-05-21T22:00:53.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.582+0000 7f69a6a00700 1 -- 172.21.15.151:0/2637880278 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69b40a99f0 con 0x55f78308c7d0 2024-05-21T22:00:53.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.583+0000 7f69a6a00700 1 -- 172.21.15.151:0/2637880278 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f69b8011070 con 0x55f78308c7d0 2024-05-21T22:00:53.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.583+0000 7f69d2518b80 10 client.26458 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:00:53.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.583+0000 7f69d2518b80 10 client.26458 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-21T22:00:53.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.583+0000 7f69d2518b80 20 client.26458 populate_metadata read hostname 'smithi151' 2024-05-21T22:00:53.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.583+0000 7f69d2518b80 1 --2- 172.21.15.151:0/2637880278 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x55f78310a7b0 0x55f78310cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:53.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.583+0000 7f69d2518b80 4 client.26458 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T22:00:53.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.583+0000 7f69d2518b80 1 -- 172.21.15.151:0/2637880278 --> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f782fe0570 con 0x55f78310a7b0 2024-05-21T22:00:53.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.583+0000 7f69c5200700 1 --2- 172.21.15.151:0/2637880278 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x55f78310a7b0 0x55f78310cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:53.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.583+0000 7f69c5200700 1 --2- 172.21.15.151:0/2637880278 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x55f78310a7b0 0x55f78310cbd0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=12e34d05461e691e server_cookie=1689e8bb95f59178 in_seq=0 out_seq=0 2024-05-21T22:00:53.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.583+0000 7f69a6a00700 10 client.26458 ms_handle_connect on v2:172.21.15.184:6838/3059935143 2024-05-21T22:00:53.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.584+0000 7f69a6a00700 1 -- 172.21.15.151:0/2637880278 <== mds.1 v2:172.21.15.184:6838/3059935143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f782fe0570 con 0x55f78310a7b0 2024-05-21T22:00:53.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.584+0000 7f69a6a00700 10 client.26458 handle_command_reply: tid=0 2024-05-21T22:00:53.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.584+0000 7f69d2518b80 1 client.26458 shutdown 2024-05-21T22:00:53.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.584+0000 7f69d2518b80 1 -- 172.21.15.151:0/2637880278 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f78308c7d0 msgr2=0x55f7830a6d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:53.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.584+0000 7f69d2518b80 1 --2- 172.21.15.151:0/2637880278 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f78308c7d0 0x55f7830a6d70 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f69b800b770 tx=0x7f69b800b3e0 comp rx=0 tx=0).stop 2024-05-21T22:00:53.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.584+0000 7f69d2518b80 1 -- 172.21.15.151:0/2637880278 shutdown_connections 2024-05-21T22:00:53.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.584+0000 7f69d2518b80 1 --2- 172.21.15.151:0/2637880278 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x55f78310a7b0 0x55f78310cbd0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:53.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.584+0000 7f69d2518b80 1 --2- 172.21.15.151:0/2637880278 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f78308c7d0 0x55f7830a6d70 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:53.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.584+0000 7f69a6a00700 0 client.26458 ms_handle_reset on v2:172.21.15.184:6838/3059935143 2024-05-21T22:00:53.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.584+0000 7f69d2518b80 1 --2- 172.21.15.151:0/2637880278 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f783086880 0x55f7830987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:53.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.584+0000 7f69d2518b80 1 --2- 172.21.15.151:0/2637880278 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f78308beb0 0x55f783098d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:53.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.584+0000 7f69d2518b80 1 -- 172.21.15.151:0/2637880278 >> 172.21.15.151:0/2637880278 conn(0x55f782fb4be0 msgr2=0x55f7830899c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:00:53.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.584+0000 7f69d2518b80 1 -- 172.21.15.151:0/2637880278 shutdown_connections 2024-05-21T22:00:53.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.584+0000 7f69d2518b80 1 -- 172.21.15.151:0/2637880278 wait complete. 2024-05-21T22:00:53.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.584+0000 7f69d2518b80 20 client.26458 trim_cache size 0 max 16384 2024-05-21T22:00:53.589 INFO:teuthology.orchestra.run.smithi151.stdout:[] 2024-05-21T22:00:53.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.585+0000 7f69c7c00700 1 -- 172.21.15.151:0/3419813788 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f69a80b2a70 msgr2=0x7f69a80b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:53.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.585+0000 7f69c7c00700 1 --2- 172.21.15.151:0/3419813788 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f69a80b2a70 0x7f69a80b4f30 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f69b8006130 tx=0x7f69b80093c0 comp rx=0 tx=0).stop 2024-05-21T22:00:53.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.585+0000 7f69c7c00700 1 -- 172.21.15.151:0/3419813788 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69c0109dd0 msgr2=0x7f69c01a0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:53.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.585+0000 7f69c7c00700 1 --2- 172.21.15.151:0/3419813788 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69c0109dd0 0x7f69c01a0190 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f69b400e9a0 tx=0x7f69b400ecb0 comp rx=0 tx=0).stop 2024-05-21T22:00:53.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.585+0000 7f69c7c00700 1 -- 172.21.15.151:0/3419813788 shutdown_connections 2024-05-21T22:00:53.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.585+0000 7f69c7c00700 1 --2- 172.21.15.151:0/3419813788 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f69a80b2a70 0x7f69a80b4f30 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:53.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.585+0000 7f69c7c00700 1 --2- 172.21.15.151:0/3419813788 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69c010eeb0 0x7f69c01a4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:53.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.585+0000 7f69c7c00700 1 --2- 172.21.15.151:0/3419813788 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69c0109dd0 0x7f69c01a0190 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:53.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.585+0000 7f69c7c00700 1 --2- 172.21.15.151:0/3419813788 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f69c010a780 0x7f69c01a06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:53.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.585+0000 7f69c7c00700 1 -- 172.21.15.151:0/3419813788 >> 172.21.15.151:0/3419813788 conn(0x7f69c0076d10 msgr2=0x7f69c0077290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:00:53.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.586+0000 7f69c7c00700 1 -- 172.21.15.151:0/3419813788 shutdown_connections 2024-05-21T22:00:53.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:53.586+0000 7f69c7c00700 1 -- 172.21.15.151:0/3419813788 wait complete. 2024-05-21T22:00:53.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:00:53 smithi151 ceph-mon[31623]: pgmap v161: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T22:00:53.735 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:00:53 smithi184 ceph-mon[36942]: pgmap v161: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T22:00:53.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:00:53 smithi164 ceph-mon[36269]: pgmap v161: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T22:00:54.580 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-21T22:00:55.030 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:00:55.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:00:55 smithi151 ceph-mon[31623]: pgmap v162: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:00:55.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:00:55 smithi184 ceph-mon[36942]: pgmap v162: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:00:55.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:00:55 smithi164 ceph-mon[36269]: pgmap v162: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:00:57.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.219+0000 7f9226800700 1 -- 172.21.15.151:0/3499335023 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f92201088f0 msgr2=0x7f9220110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:57.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.219+0000 7f9226800700 1 --2- 172.21.15.151:0/3499335023 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f92201088f0 0x7f9220110470 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f921800b0a0 tx=0x7f921800b3b0 comp rx=0 tx=0).stop 2024-05-21T22:00:57.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.219+0000 7f9226800700 1 -- 172.21.15.151:0/3499335023 shutdown_connections 2024-05-21T22:00:57.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.219+0000 7f9226800700 1 --2- 172.21.15.151:0/3499335023 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f92201037f0 0x7f9220103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:57.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.219+0000 7f9226800700 1 --2- 172.21.15.151:0/3499335023 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f92201088f0 0x7f9220110470 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:57.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.219+0000 7f9226800700 1 --2- 172.21.15.151:0/3499335023 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f92201041a0 0x7f92201081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:57.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.219+0000 7f9226800700 1 -- 172.21.15.151:0/3499335023 >> 172.21.15.151:0/3499335023 conn(0x7f922007a6a0 msgr2=0x7f922007aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:00:57.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.219+0000 7f9226800700 1 -- 172.21.15.151:0/3499335023 shutdown_connections 2024-05-21T22:00:57.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.219+0000 7f9226800700 1 -- 172.21.15.151:0/3499335023 wait complete. 2024-05-21T22:00:57.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.220+0000 7f9226800700 1 Processor -- start 2024-05-21T22:00:57.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.220+0000 7f9226800700 1 -- start start 2024-05-21T22:00:57.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.220+0000 7f9226800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f92201037f0 0x7f92201a1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:57.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.220+0000 7f9226800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f92201041a0 0x7f92201a1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:57.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.220+0000 7f9226800700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f92201088f0 0x7f922019bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:57.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.220+0000 7f9226800700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f92201136c0 con 0x7f92201037f0 2024-05-21T22:00:57.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.220+0000 7f9226800700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9220113540 con 0x7f92201041a0 2024-05-21T22:00:57.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.220+0000 7f9226800700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9220113840 con 0x7f92201088f0 2024-05-21T22:00:57.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.221+0000 7f921f400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f92201037f0 0x7f92201a1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:57.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.221+0000 7f921f400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f92201037f0 0x7f92201a1a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:46774/0 (socket says 172.21.15.151:46774) 2024-05-21T22:00:57.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.221+0000 7f921f400700 1 -- 172.21.15.151:0/977137579 learned_addr learned my addr 172.21.15.151:0/977137579 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:00:57.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.221+0000 7f921ea00700 1 --2- 172.21.15.151:0/977137579 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f92201041a0 0x7f92201a1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:57.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.221+0000 7f921fe00700 1 --2- 172.21.15.151:0/977137579 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f92201088f0 0x7f922019bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:57.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.221+0000 7f921f400700 1 -- 172.21.15.151:0/977137579 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f92201088f0 msgr2=0x7f922019bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:57.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.221+0000 7f921f400700 1 --2- 172.21.15.151:0/977137579 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f92201088f0 0x7f922019bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:57.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.221+0000 7f921f400700 1 -- 172.21.15.151:0/977137579 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f92201041a0 msgr2=0x7f92201a1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:57.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.221+0000 7f921f400700 1 --2- 172.21.15.151:0/977137579 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f92201041a0 0x7f92201a1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:57.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.221+0000 7f921f400700 1 -- 172.21.15.151:0/977137579 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f922019c460 con 0x7f92201037f0 2024-05-21T22:00:57.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.222+0000 7f921f400700 1 --2- 172.21.15.151:0/977137579 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f92201037f0 0x7f92201a1a60 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f921000e9a0 tx=0x7f921000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:00:57.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.222+0000 7f9217e00700 1 -- 172.21.15.151:0/977137579 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f921000ca10 con 0x7f92201037f0 2024-05-21T22:00:57.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.222+0000 7f9217e00700 1 -- 172.21.15.151:0/977137579 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f921000cbb0 con 0x7f92201037f0 2024-05-21T22:00:57.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.222+0000 7f9226800700 1 -- 172.21.15.151:0/977137579 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f922019c750 con 0x7f92201037f0 2024-05-21T22:00:57.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.222+0000 7f9217e00700 1 -- 172.21.15.151:0/977137579 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9210018680 con 0x7f92201037f0 2024-05-21T22:00:57.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.222+0000 7f9226800700 1 -- 172.21.15.151:0/977137579 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92201a89e0 con 0x7f92201037f0 2024-05-21T22:00:57.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.226+0000 7f9217e00700 1 -- 172.21.15.151:0/977137579 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7f9210010430 con 0x7f92201037f0 2024-05-21T22:00:57.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.227+0000 7f9231262b80 1 Processor -- start 2024-05-21T22:00:57.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.227+0000 7f9231262b80 1 -- start start 2024-05-21T22:00:57.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.227+0000 7f9217e00700 1 --2- 172.21.15.151:0/977137579 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f92040b2a70 0x7f92040b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:57.229 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.227+0000 7f9231262b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f993786610 0x55f9937869f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:57.229 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.227+0000 7f9231262b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f993786f30 0x55f993780b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:57.229 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.227+0000 7f9231262b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f993781140 0x55f9937815c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:57.229 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.227+0000 7f9231262b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55f9936dee40 con 0x55f993786610 2024-05-21T22:00:57.229 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.227+0000 7f9231262b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55f9936ce9e0 con 0x55f993786f30 2024-05-21T22:00:57.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.227+0000 7f9231262b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55f9936d2a20 con 0x55f993781140 2024-05-21T22:00:57.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.228+0000 7f9217e00700 1 -- 172.21.15.151:0/977137579 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f92100e2760 con 0x7f92201037f0 2024-05-21T22:00:57.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.228+0000 7f921fe00700 1 --2- 172.21.15.151:0/977137579 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f92040b2a70 0x7f92040b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:57.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.228+0000 7f921ea00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f993786f30 0x55f993780b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:57.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.228+0000 7f921ea00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f993786f30 0x55f993780b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:45134/0 (socket says 172.21.15.151:45134) 2024-05-21T22:00:57.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.228+0000 7f921ea00700 1 -- 172.21.15.151:0/3004140757 learned_addr learned my addr 172.21.15.151:0/3004140757 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:00:57.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.228+0000 7f921fe00700 1 --2- 172.21.15.151:0/3004140757 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f993781140 0x55f9937815c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:57.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.228+0000 7f921f400700 1 --2- 172.21.15.151:0/3004140757 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f993786610 0x55f9937869f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:57.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.228+0000 7f921ea00700 1 -- 172.21.15.151:0/3004140757 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f993781140 msgr2=0x55f9937815c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:57.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.228+0000 7f921ea00700 1 --2- 172.21.15.151:0/3004140757 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f993781140 0x55f9937815c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:57.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.228+0000 7f921ea00700 1 -- 172.21.15.151:0/3004140757 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f993786610 msgr2=0x55f9937869f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:57.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.228+0000 7f921ea00700 1 --2- 172.21.15.151:0/3004140757 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f993786610 0x55f9937869f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:57.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.228+0000 7f921ea00700 1 -- 172.21.15.151:0/3004140757 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f993697420 con 0x55f993786f30 2024-05-21T22:00:57.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.229+0000 7f921fe00700 1 --2- 172.21.15.151:0/977137579 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f92040b2a70 0x7f92040b4f30 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9218009fd0 tx=0x7f921800bbe0 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:00:57.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.229+0000 7f921ea00700 1 --2- 172.21.15.151:0/3004140757 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f993786f30 0x55f993780b20 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9208009cf0 tx=0x7f9208011440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8f5a15988953177 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:00:57.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.229+0000 7f9214c00700 1 -- 172.21.15.151:0/3004140757 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9208011d70 con 0x55f993786f30 2024-05-21T22:00:57.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.229+0000 7f9214c00700 1 -- 172.21.15.151:0/3004140757 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9208004500 con 0x55f993786f30 2024-05-21T22:00:57.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.229+0000 7f9214c00700 1 -- 172.21.15.151:0/3004140757 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9208019640 con 0x55f993786f30 2024-05-21T22:00:57.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.230+0000 7f9231262b80 1 -- 172.21.15.151:0/3004140757 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f993786f30 msgr2=0x55f993780b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:57.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.230+0000 7f9231262b80 1 --2- 172.21.15.151:0/3004140757 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f993786f30 0x55f993780b20 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9208009cf0 tx=0x7f9208011440 comp rx=0 tx=0).stop 2024-05-21T22:00:57.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.230+0000 7f9231262b80 1 -- 172.21.15.151:0/3004140757 shutdown_connections 2024-05-21T22:00:57.233 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.230+0000 7f9231262b80 1 --2- 172.21.15.151:0/3004140757 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f993781140 0x55f9937815c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:57.233 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.230+0000 7f9231262b80 1 --2- 172.21.15.151:0/3004140757 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f993786610 0x55f9937869f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:57.233 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.230+0000 7f9231262b80 1 --2- 172.21.15.151:0/3004140757 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f993786f30 0x55f993780b20 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:57.233 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.230+0000 7f9231262b80 1 -- 172.21.15.151:0/3004140757 >> 172.21.15.151:0/3004140757 conn(0x55f9936aebe0 msgr2=0x55f99377b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:00:57.233 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.230+0000 7f9231262b80 1 -- 172.21.15.151:0/3004140757 shutdown_connections 2024-05-21T22:00:57.233 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.230+0000 7f9231262b80 1 -- 172.21.15.151:0/3004140757 wait complete. 2024-05-21T22:00:57.233 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.231+0000 7f9231262b80 1 -- start start 2024-05-21T22:00:57.233 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.231+0000 7f9231262b80 1 Processor -- start 2024-05-21T22:00:57.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.232+0000 7f9231262b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f993781140 0x55f99378cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:57.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.232+0000 7f9231262b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f993786610 0x55f99378a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:57.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.232+0000 7f9231262b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f993786f30 0x55f99378a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:57.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.232+0000 7f9231262b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55f9935d4fa0 con 0x55f993786610 2024-05-21T22:00:57.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.232+0000 7f9231262b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55f99366f760 con 0x55f993781140 2024-05-21T22:00:57.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.232+0000 7f9231262b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55f99378d9e0 con 0x55f993786f30 2024-05-21T22:00:57.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.232+0000 7f9231262b80 10 client.0 _ll_register_callbacks cb 0x55f993678c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:00:57.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.232+0000 7f921ea00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f993786610 0x55f99378a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:57.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.232+0000 7f921ea00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f993786610 0x55f99378a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:46790/0 (socket says 172.21.15.151:46790) 2024-05-21T22:00:57.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.232+0000 7f921ea00700 1 -- 172.21.15.151:0/2219942991 learned_addr learned my addr 172.21.15.151:0/2219942991 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:00:57.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.232+0000 7f921fe00700 1 --2- 172.21.15.151:0/2219942991 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f993786f30 0x55f99378a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:57.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.232+0000 7f921f400700 1 --2- 172.21.15.151:0/2219942991 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f993781140 0x55f99378cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:57.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.233+0000 7f921ea00700 1 -- 172.21.15.151:0/2219942991 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f993786f30 msgr2=0x55f99378a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:57.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.233+0000 7f921ea00700 1 --2- 172.21.15.151:0/2219942991 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f993786f30 0x55f99378a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:57.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.233+0000 7f921ea00700 1 -- 172.21.15.151:0/2219942991 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f993781140 msgr2=0x55f99378cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:57.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.233+0000 7f921ea00700 1 --2- 172.21.15.151:0/2219942991 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f993781140 0x55f99378cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:57.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.233+0000 7f921ea00700 1 -- 172.21.15.151:0/2219942991 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f99361e8d0 con 0x55f993786610 2024-05-21T22:00:57.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.233+0000 7f921fe00700 1 --2- 172.21.15.151:0/2219942991 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f993786f30 0x55f99378a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:00:57.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.233+0000 7f921f400700 1 --2- 172.21.15.151:0/2219942991 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f993781140 0x55f99378cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:00:57.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.233+0000 7f921ea00700 1 --2- 172.21.15.151:0/2219942991 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f993786610 0x55f99378a000 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f9208014040 tx=0x7f9208009550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c01e424f04b2faa server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:00:57.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.233+0000 7f920d600700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-21T22:00:57.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.233+0000 7f920d600700 1 -- 172.21.15.151:0/2219942991 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9208027030 con 0x55f993786610 2024-05-21T22:00:57.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.234+0000 7f920d600700 1 -- 172.21.15.151:0/2219942991 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f920801f050 con 0x55f993786610 2024-05-21T22:00:57.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.234+0000 7f9231262b80 1 -- 172.21.15.151:0/2219942991 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9208004500 con 0x55f993786610 2024-05-21T22:00:57.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.234+0000 7f920d600700 1 -- 172.21.15.151:0/2219942991 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9208031050 con 0x55f993786610 2024-05-21T22:00:57.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.234+0000 7f9231262b80 10 client.16332 fetch_fsmap learned FSMap version 29 2024-05-21T22:00:57.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.234+0000 7f9231262b80 1 -- 172.21.15.151:0/2219942991 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f99369f060 con 0x55f993786610 2024-05-21T22:00:57.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.234+0000 7f920d600700 1 -- 172.21.15.151:0/2219942991 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9208004500 con 0x55f993786610 2024-05-21T22:00:57.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.234+0000 7f920d600700 1 -- 172.21.15.151:0/2219942991 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f9208018070 con 0x55f993786610 2024-05-21T22:00:57.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.235+0000 7f9231262b80 10 client.16332 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:00:57.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.235+0000 7f9231262b80 10 client.16332 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T22:00:57.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.235+0000 7f9231262b80 20 client.16332 populate_metadata read hostname 'smithi151' 2024-05-21T22:00:57.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.235+0000 7f9231262b80 1 --2- 172.21.15.151:0/2219942991 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55f99379a3d0 0x55f99379c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:57.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.235+0000 7f9231262b80 4 client.16332 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T22:00:57.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.235+0000 7f9231262b80 1 -- 172.21.15.151:0/2219942991 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f9936e0d90 con 0x55f99379a3d0 2024-05-21T22:00:57.238 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.235+0000 7f921f400700 1 --2- 172.21.15.151:0/2219942991 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55f99379a3d0 0x55f99379c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:57.238 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.235+0000 7f921f400700 1 --2- 172.21.15.151:0/2219942991 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55f99379a3d0 0x55f99379c7c0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf19c02aeaca5ea4 server_cookie=9ae03e6e84c701d9 in_seq=0 out_seq=0 2024-05-21T22:00:57.238 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.235+0000 7f920d600700 10 client.16332 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T22:00:57.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.236+0000 7f920d600700 1 -- 172.21.15.151:0/2219942991 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55f9936e0d90 con 0x55f99379a3d0 2024-05-21T22:00:57.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.236+0000 7f920d600700 10 client.16332 handle_command_reply: tid=0 2024-05-21T22:00:57.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.236+0000 7f9231262b80 1 client.16332 shutdown 2024-05-21T22:00:57.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.237+0000 7f9231262b80 1 -- 172.21.15.151:0/2219942991 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f993786610 msgr2=0x55f99378a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:57.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.237+0000 7f9231262b80 1 --2- 172.21.15.151:0/2219942991 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f993786610 0x55f99378a000 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f9208014040 tx=0x7f9208009550 comp rx=0 tx=0).stop 2024-05-21T22:00:57.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.237+0000 7f9231262b80 1 -- 172.21.15.151:0/2219942991 shutdown_connections 2024-05-21T22:00:57.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.237+0000 7f9231262b80 1 --2- 172.21.15.151:0/2219942991 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55f99379a3d0 0x55f99379c7c0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:57.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.237+0000 7f9231262b80 1 --2- 172.21.15.151:0/2219942991 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f993781140 0x55f99378cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:57.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.237+0000 7f9231262b80 1 --2- 172.21.15.151:0/2219942991 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f993786f30 0x55f99378a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:57.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.237+0000 7f920d600700 0 client.16332 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T22:00:57.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.237+0000 7f9231262b80 1 --2- 172.21.15.151:0/2219942991 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f993786610 0x55f99378a000 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:57.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.237+0000 7f9231262b80 1 -- 172.21.15.151:0/2219942991 >> 172.21.15.151:0/2219942991 conn(0x55f9936aebe0 msgr2=0x55f99377ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:00:57.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.237+0000 7f9231262b80 1 -- 172.21.15.151:0/2219942991 shutdown_connections 2024-05-21T22:00:57.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.237+0000 7f9231262b80 1 -- 172.21.15.151:0/2219942991 wait complete. 2024-05-21T22:00:57.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.237+0000 7f9231262b80 20 client.16332 trim_cache size 0 max 16384 2024-05-21T22:00:57.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.253+0000 7f9231262b80 1 Processor -- start 2024-05-21T22:00:57.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.254+0000 7f9231262b80 1 -- start start 2024-05-21T22:00:57.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.254+0000 7f9231262b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f993781140 0x55f993787ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:57.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.254+0000 7f9231262b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f993786610 0x55f99377b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:57.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.254+0000 7f9231262b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f993786f30 0x55f9937a1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:57.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.254+0000 7f9231262b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f92201afff0 con 0x55f993786610 2024-05-21T22:00:57.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.254+0000 7f9231262b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f92201b1b30 con 0x55f993786f30 2024-05-21T22:00:57.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.254+0000 7f9231262b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f92201b06c0 con 0x55f993781140 2024-05-21T22:00:57.257 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.254+0000 7f921f400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f993781140 0x55f993787ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:57.257 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.254+0000 7f921f400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f993781140 0x55f993787ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:48724/0 (socket says 172.21.15.151:48724) 2024-05-21T22:00:57.257 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.254+0000 7f921f400700 1 -- 172.21.15.151:0/1995605718 learned_addr learned my addr 172.21.15.151:0/1995605718 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:00:57.257 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.254+0000 7f921fe00700 1 --2- 172.21.15.151:0/1995605718 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f993786f30 0x55f9937a1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:57.257 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.254+0000 7f921ea00700 1 --2- 172.21.15.151:0/1995605718 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f993786610 0x55f99377b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:57.257 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.255+0000 7f921f400700 1 -- 172.21.15.151:0/1995605718 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f993786f30 msgr2=0x55f9937a1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:57.257 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.255+0000 7f921f400700 1 --2- 172.21.15.151:0/1995605718 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f993786f30 0x55f9937a1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:57.257 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.255+0000 7f921f400700 1 -- 172.21.15.151:0/1995605718 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f993786610 msgr2=0x55f99377b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:57.257 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.255+0000 7f921f400700 1 --2- 172.21.15.151:0/1995605718 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f993786610 0x55f99377b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:57.257 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.255+0000 7f921f400700 1 -- 172.21.15.151:0/1995605718 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f993780e60 con 0x55f993781140 2024-05-21T22:00:57.257 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.255+0000 7f921f400700 1 --2- 172.21.15.151:0/1995605718 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f993781140 0x55f993787ae0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f92100a96f0 tx=0x7f9210009790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88b80895b199537f server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:00:57.257 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.255+0000 7f920fe00700 1 -- 172.21.15.151:0/1995605718 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9210005520 con 0x55f993781140 2024-05-21T22:00:57.257 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.256+0000 7f920fe00700 1 -- 172.21.15.151:0/1995605718 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92100052f0 con 0x55f993781140 2024-05-21T22:00:57.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.256+0000 7f920fe00700 1 -- 172.21.15.151:0/1995605718 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92100b2620 con 0x55f993781140 2024-05-21T22:00:57.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.257+0000 7f9231262b80 1 -- 172.21.15.151:0/1995605718 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f993781140 msgr2=0x55f993787ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:57.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.257+0000 7f9231262b80 1 --2- 172.21.15.151:0/1995605718 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f993781140 0x55f993787ae0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f92100a96f0 tx=0x7f9210009790 comp rx=0 tx=0).stop 2024-05-21T22:00:57.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.257+0000 7f9231262b80 1 -- 172.21.15.151:0/1995605718 shutdown_connections 2024-05-21T22:00:57.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.257+0000 7f9231262b80 1 --2- 172.21.15.151:0/1995605718 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f993786f30 0x55f9937a1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:57.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.257+0000 7f9231262b80 1 --2- 172.21.15.151:0/1995605718 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f993781140 0x55f993787ae0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:57.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.257+0000 7f9231262b80 1 --2- 172.21.15.151:0/1995605718 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f993786610 0x55f99377b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:57.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.257+0000 7f9231262b80 1 -- 172.21.15.151:0/1995605718 >> 172.21.15.151:0/1995605718 conn(0x55f9936aebe0 msgr2=0x55f9937873d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:00:57.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.257+0000 7f9231262b80 1 -- 172.21.15.151:0/1995605718 shutdown_connections 2024-05-21T22:00:57.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.257+0000 7f9231262b80 1 -- 172.21.15.151:0/1995605718 wait complete. 2024-05-21T22:00:57.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.257+0000 7f9231262b80 1 -- start start 2024-05-21T22:00:57.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.257+0000 7f9231262b80 1 Processor -- start 2024-05-21T22:00:57.260 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.258+0000 7f9231262b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f993781140 0x55f9937932b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:57.260 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.258+0000 7f9231262b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f993786610 0x55f9937a1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:57.260 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.258+0000 7f9231262b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f993786f30 0x55f9937a1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:57.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.258+0000 7f9231262b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f92201b1460 con 0x55f993786f30 2024-05-21T22:00:57.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.258+0000 7f9231262b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f92201b0d90 con 0x55f993781140 2024-05-21T22:00:57.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.258+0000 7f9231262b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f92201adf90 con 0x55f993786610 2024-05-21T22:00:57.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.258+0000 7f9231262b80 10 client.0 _ll_register_callbacks cb 0x55f993678c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:00:57.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.258+0000 7f921fe00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f993786f30 0x55f9937a1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:57.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.258+0000 7f921f400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f993781140 0x55f9937932b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:57.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.258+0000 7f921fe00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f993786f30 0x55f9937a1c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:46812/0 (socket says 172.21.15.151:46812) 2024-05-21T22:00:57.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.258+0000 7f921f400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f993781140 0x55f9937932b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:45156/0 (socket says 172.21.15.151:45156) 2024-05-21T22:00:57.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.258+0000 7f921fe00700 1 -- 172.21.15.151:0/2174176817 learned_addr learned my addr 172.21.15.151:0/2174176817 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:00:57.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.258+0000 7f921ea00700 1 --2- 172.21.15.151:0/2174176817 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f993786610 0x55f9937a1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:57.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.259+0000 7f921fe00700 1 -- 172.21.15.151:0/2174176817 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f993786610 msgr2=0x55f9937a1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:57.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.259+0000 7f921fe00700 1 --2- 172.21.15.151:0/2174176817 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f993786610 0x55f9937a1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:57.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.259+0000 7f921fe00700 1 -- 172.21.15.151:0/2174176817 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f993781140 msgr2=0x55f9937932b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:00:57.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.259+0000 7f921fe00700 1 --2- 172.21.15.151:0/2174176817 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f993781140 0x55f9937932b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:57.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.259+0000 7f921fe00700 1 -- 172.21.15.151:0/2174176817 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9937a2400 con 0x55f993786f30 2024-05-21T22:00:57.262 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.259+0000 7f921f400700 1 --2- 172.21.15.151:0/2174176817 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f993781140 0x55f9937932b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:00:57.262 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.259+0000 7f921fe00700 1 --2- 172.21.15.151:0/2174176817 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f993786f30 0x55f9937a1c80 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f9218015000 tx=0x7f921800b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8b2ab79f1685d9f server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:00:57.262 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.259+0000 7f920d600700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-21T22:00:57.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.259+0000 7f920d600700 1 -- 172.21.15.151:0/2174176817 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9218030070 con 0x55f993786f30 2024-05-21T22:00:57.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.259+0000 7f9231262b80 1 -- 172.21.15.151:0/2174176817 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92100052f0 con 0x55f993786f30 2024-05-21T22:00:57.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.260+0000 7f920d600700 1 -- 172.21.15.151:0/2174176817 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f921801f850 con 0x55f993786f30 2024-05-21T22:00:57.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.260+0000 7f920d600700 1 -- 172.21.15.151:0/2174176817 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9218003e10 con 0x55f993786f30 2024-05-21T22:00:57.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.260+0000 7f9231262b80 10 client.16338 fetch_fsmap learned FSMap version 29 2024-05-21T22:00:57.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.260+0000 7f9231262b80 1 -- 172.21.15.151:0/2174176817 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f99380d5e0 con 0x55f993786f30 2024-05-21T22:00:57.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.260+0000 7f920d600700 1 -- 172.21.15.151:0/2174176817 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92100052f0 con 0x55f993786f30 2024-05-21T22:00:57.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.260+0000 7f920d600700 1 -- 172.21.15.151:0/2174176817 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f921802b040 con 0x55f993786f30 2024-05-21T22:00:57.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.260+0000 7f9231262b80 10 client.16338 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:00:57.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.260+0000 7f9231262b80 10 client.16338 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T22:00:57.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.260+0000 7f9231262b80 20 client.16338 populate_metadata read hostname 'smithi151' 2024-05-21T22:00:57.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.260+0000 7f9231262b80 1 --2- 172.21.15.151:0/2174176817 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55f99380dd00 0x55f993810120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:00:57.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.261+0000 7f9231262b80 4 client.16338 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T22:00:57.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.261+0000 7f9231262b80 1 -- 172.21.15.151:0/2174176817 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f9936da570 con 0x55f99380dd00 2024-05-21T22:00:57.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.261+0000 7f921f400700 1 --2- 172.21.15.151:0/2174176817 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55f99380dd00 0x55f993810120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:00:57.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.261+0000 7f921f400700 1 --2- 172.21.15.151:0/2174176817 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55f99380dd00 0x55f993810120 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea3e4dee97e754da server_cookie=a8df335307cb8cc in_seq=0 out_seq=0 2024-05-21T22:00:57.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.261+0000 7f920d600700 10 client.16338 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T22:00:57.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.262+0000 7f920d600700 1 -- 172.21.15.151:0/2174176817 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f9936da570 con 0x55f99380dd00 2024-05-21T22:00:57.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.262+0000 7f920d600700 10 client.16338 handle_command_reply: tid=0 2024-05-21T22:00:57.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.262+0000 7f9231262b80 1 client.16338 shutdown 2024-05-21T22:00:57.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.262+0000 7f9231262b80 1 -- 172.21.15.151:0/2174176817 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f993786f30 msgr2=0x55f9937a1c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:57.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.262+0000 7f9231262b80 1 --2- 172.21.15.151:0/2174176817 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f993786f30 0x55f9937a1c80 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f9218015000 tx=0x7f921800b3b0 comp rx=0 tx=0).stop 2024-05-21T22:00:57.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.262+0000 7f9231262b80 1 -- 172.21.15.151:0/2174176817 shutdown_connections 2024-05-21T22:00:57.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.262+0000 7f9231262b80 1 --2- 172.21.15.151:0/2174176817 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55f99380dd00 0x55f993810120 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:57.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.262+0000 7f9231262b80 1 --2- 172.21.15.151:0/2174176817 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f993781140 0x55f9937932b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:57.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.262+0000 7f9231262b80 1 --2- 172.21.15.151:0/2174176817 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f993786f30 0x55f9937a1c80 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:57.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.262+0000 7f9231262b80 1 --2- 172.21.15.151:0/2174176817 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f993786610 0x55f9937a1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:57.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.262+0000 7f920d600700 0 client.16338 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T22:00:57.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.262+0000 7f9231262b80 1 -- 172.21.15.151:0/2174176817 >> 172.21.15.151:0/2174176817 conn(0x55f9936aebe0 msgr2=0x55f99378b600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:00:57.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.262+0000 7f9231262b80 1 -- 172.21.15.151:0/2174176817 shutdown_connections 2024-05-21T22:00:57.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.262+0000 7f9231262b80 1 -- 172.21.15.151:0/2174176817 wait complete. 2024-05-21T22:00:57.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.262+0000 7f9231262b80 20 client.16338 trim_cache size 0 max 16384 2024-05-21T22:00:57.266 INFO:teuthology.orchestra.run.smithi151.stdout:[] 2024-05-21T22:00:57.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.263+0000 7f9226800700 1 -- 172.21.15.151:0/977137579 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f92040b2a70 msgr2=0x7f92040b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:57.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.263+0000 7f9226800700 1 --2- 172.21.15.151:0/977137579 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f92040b2a70 0x7f92040b4f30 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9218009fd0 tx=0x7f921800bbe0 comp rx=0 tx=0).stop 2024-05-21T22:00:57.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.263+0000 7f9226800700 1 -- 172.21.15.151:0/977137579 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f92201037f0 msgr2=0x7f92201a1a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:00:57.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.263+0000 7f9226800700 1 --2- 172.21.15.151:0/977137579 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f92201037f0 0x7f92201a1a60 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f921000e9a0 tx=0x7f921000ecb0 comp rx=0 tx=0).stop 2024-05-21T22:00:57.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.263+0000 7f9226800700 1 -- 172.21.15.151:0/977137579 shutdown_connections 2024-05-21T22:00:57.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.263+0000 7f9226800700 1 --2- 172.21.15.151:0/977137579 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f92040b2a70 0x7f92040b4f30 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:57.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.263+0000 7f9226800700 1 --2- 172.21.15.151:0/977137579 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f92201088f0 0x7f922019bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:57.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.263+0000 7f9226800700 1 --2- 172.21.15.151:0/977137579 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f92201037f0 0x7f92201a1a60 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:57.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.263+0000 7f9226800700 1 --2- 172.21.15.151:0/977137579 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f92201041a0 0x7f92201a1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:00:57.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.263+0000 7f9226800700 1 -- 172.21.15.151:0/977137579 >> 172.21.15.151:0/977137579 conn(0x7f922007a6a0 msgr2=0x7f9220101950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:00:57.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.263+0000 7f9226800700 1 -- 172.21.15.151:0/977137579 shutdown_connections 2024-05-21T22:00:57.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:00:57.263+0000 7f9226800700 1 -- 172.21.15.151:0/977137579 wait complete. 2024-05-21T22:00:57.638 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:00:57 smithi151 ceph-mon[31623]: pgmap v163: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:00:57.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:00:57 smithi184 ceph-mon[36942]: pgmap v163: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:00:57.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:00:57 smithi164 ceph-mon[36269]: pgmap v163: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:00:59.075 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-21T22:00:59.075 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-21T22:00:59.541 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:00:59.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:00:59 smithi151 ceph-mon[31623]: pgmap v164: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T22:00:59.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:00:59 smithi184 ceph-mon[36942]: pgmap v164: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T22:00:59.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:00:59 smithi164 ceph-mon[36269]: pgmap v164: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T22:01:00.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:01:00 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T22:01:00.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:01:00 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T22:01:00.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:01:00 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T22:01:01.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.722+0000 7fb470e00700 1 -- 172.21.15.151:0/3619481249 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb46c06c150 msgr2=0x7fb46c06c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:01.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.722+0000 7fb470e00700 1 --2- 172.21.15.151:0/3619481249 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb46c06c150 0x7fb46c06c5d0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fb454009990 tx=0x7fb454009ca0 comp rx=0 tx=0).stop 2024-05-21T22:01:01.724 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:01:01 smithi151 ceph-mon[31623]: pgmap v165: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:01:01.725 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:01:01 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T22:01:01.725 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:01:01 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T22:01:01.725 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:01:01 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T22:01:01.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.722+0000 7fb470e00700 1 -- 172.21.15.151:0/3619481249 shutdown_connections 2024-05-21T22:01:01.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.722+0000 7fb470e00700 1 --2- 172.21.15.151:0/3619481249 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb46c10fe90 0x7fb46c112280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:01.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.722+0000 7fb470e00700 1 --2- 172.21.15.151:0/3619481249 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb46c104480 0x7fb46c06bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:01.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.722+0000 7fb470e00700 1 --2- 172.21.15.151:0/3619481249 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb46c06c150 0x7fb46c06c5d0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:01.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.722+0000 7fb470e00700 1 -- 172.21.15.151:0/3619481249 >> 172.21.15.151:0/3619481249 conn(0x7fb46c0793f0 msgr2=0x7fb46c079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:01:01.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.722+0000 7fb470e00700 1 -- 172.21.15.151:0/3619481249 shutdown_connections 2024-05-21T22:01:01.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.722+0000 7fb470e00700 1 -- 172.21.15.151:0/3619481249 wait complete. 2024-05-21T22:01:01.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.723+0000 7fb470e00700 1 Processor -- start 2024-05-21T22:01:01.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.724+0000 7fb470e00700 1 -- start start 2024-05-21T22:01:01.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.724+0000 7fb470e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb46c06c150 0x7fb46c19be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:01.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.724+0000 7fb470e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb46c104480 0x7fb46c19c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:01.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.724+0000 7fb470e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb46c10fe90 0x7fb46c1a0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:01.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.724+0000 7fb470e00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb46c115970 con 0x7fb46c104480 2024-05-21T22:01:01.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.724+0000 7fb470e00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb46c1157f0 con 0x7fb46c10fe90 2024-05-21T22:01:01.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.724+0000 7fb470e00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb46c115af0 con 0x7fb46c06c150 2024-05-21T22:01:01.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.724+0000 7fb469e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb46c06c150 0x7fb46c19be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:01.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.725+0000 7fb463e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb46c104480 0x7fb46c19c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:01.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.725+0000 7fb469e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb46c06c150 0x7fb46c19be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:48746/0 (socket says 172.21.15.151:48746) 2024-05-21T22:01:01.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.725+0000 7fb469e00700 1 -- 172.21.15.151:0/2282153041 learned_addr learned my addr 172.21.15.151:0/2282153041 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:01:01.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.725+0000 7fb46a800700 1 --2- 172.21.15.151:0/2282153041 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb46c10fe90 0x7fb46c1a0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:01.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.725+0000 7fb469e00700 1 -- 172.21.15.151:0/2282153041 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb46c10fe90 msgr2=0x7fb46c1a0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:01.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.725+0000 7fb469e00700 1 --2- 172.21.15.151:0/2282153041 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb46c10fe90 0x7fb46c1a0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:01.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.725+0000 7fb469e00700 1 -- 172.21.15.151:0/2282153041 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb46c104480 msgr2=0x7fb46c19c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:01.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.725+0000 7fb469e00700 1 --2- 172.21.15.151:0/2282153041 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb46c104480 0x7fb46c19c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:01.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.725+0000 7fb469e00700 1 -- 172.21.15.151:0/2282153041 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb46c1a0e70 con 0x7fb46c06c150 2024-05-21T22:01:01.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.725+0000 7fb463e00700 1 --2- 172.21.15.151:0/2282153041 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb46c104480 0x7fb46c19c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:01:01.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.726+0000 7fb469e00700 1 --2- 172.21.15.151:0/2282153041 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb46c06c150 0x7fb46c19be60 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb45c00e9c0 tx=0x7fb45c00ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:01:01.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.726+0000 7fb462a00700 1 -- 172.21.15.151:0/2282153041 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb45c00cbd0 con 0x7fb46c06c150 2024-05-21T22:01:01.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.726+0000 7fb462a00700 1 -- 172.21.15.151:0/2282153041 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb45c00ee60 con 0x7fb46c06c150 2024-05-21T22:01:01.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.726+0000 7fb462a00700 1 -- 172.21.15.151:0/2282153041 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb45c0055c0 con 0x7fb46c06c150 2024-05-21T22:01:01.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.726+0000 7fb470e00700 1 -- 172.21.15.151:0/2282153041 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb46c1a1160 con 0x7fb46c06c150 2024-05-21T22:01:01.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.726+0000 7fb470e00700 1 -- 172.21.15.151:0/2282153041 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb46c1a89e0 con 0x7fb46c06c150 2024-05-21T22:01:01.733 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.731+0000 7fb462a00700 1 -- 172.21.15.151:0/2282153041 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7fb45c004750 con 0x7fb46c06c150 2024-05-21T22:01:01.733 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.732+0000 7fb47b72bb80 1 Processor -- start 2024-05-21T22:01:01.733 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.732+0000 7fb47b72bb80 1 -- start start 2024-05-21T22:01:01.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.733+0000 7fb47b72bb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5604cf2cb690 0x5604cf2cba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:01.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.733+0000 7fb47b72bb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5604cf2cbfb0 0x5604cf2c5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:01.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.733+0000 7fb47b72bb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5604cf2c61c0 0x5604cf2c6640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:01.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.733+0000 7fb47b72bb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5604cf223ec0 con 0x5604cf2cbfb0 2024-05-21T22:01:01.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.733+0000 7fb47b72bb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5604cf213a60 con 0x5604cf2c61c0 2024-05-21T22:01:01.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.733+0000 7fb47b72bb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5604cf217aa0 con 0x5604cf2cb690 2024-05-21T22:01:01.735 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.733+0000 7fb463e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5604cf2c61c0 0x5604cf2c6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:01.735 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.733+0000 7fb463e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5604cf2c61c0 0x5604cf2c6640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:45190/0 (socket says 172.21.15.151:45190) 2024-05-21T22:01:01.735 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.733+0000 7fb469e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5604cf2cbfb0 0x5604cf2c5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:01.736 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.733+0000 7fb46a800700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5604cf2cb690 0x5604cf2cba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:01.736 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.733+0000 7fb463e00700 1 -- 172.21.15.151:0/3141135411 learned_addr learned my addr 172.21.15.151:0/3141135411 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:01:01.736 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.733+0000 7fb463e00700 1 -- 172.21.15.151:0/3141135411 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5604cf2cb690 msgr2=0x5604cf2cba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:01.736 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.733+0000 7fb463e00700 1 --2- 172.21.15.151:0/3141135411 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5604cf2cb690 0x5604cf2cba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:01.736 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.733+0000 7fb463e00700 1 -- 172.21.15.151:0/3141135411 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5604cf2cbfb0 msgr2=0x5604cf2c5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:01.736 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.733+0000 7fb463e00700 1 --2- 172.21.15.151:0/3141135411 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5604cf2cbfb0 0x5604cf2c5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:01.736 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.733+0000 7fb463e00700 1 -- 172.21.15.151:0/3141135411 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604cf1dc4a0 con 0x5604cf2c61c0 2024-05-21T22:01:01.736 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.734+0000 7fb462a00700 1 --2- 172.21.15.151:0/2282153041 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fb4500b2a70 0x7fb4500b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:01.736 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.734+0000 7fb463e00700 1 --2- 172.21.15.151:0/3141135411 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5604cf2c61c0 0x5604cf2c6640 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb454015000 tx=0x7fb454005f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12b23a8f1b7c6f2a server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:01:01.736 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.734+0000 7fb462a00700 1 -- 172.21.15.151:0/2282153041 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fb45c0e2870 con 0x7fb46c06c150 2024-05-21T22:01:01.736 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.734+0000 7fb46a800700 1 --2- 172.21.15.151:0/2282153041 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fb4500b2a70 0x7fb4500b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:01.736 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.734+0000 7fb45b400700 1 -- 172.21.15.151:0/3141135411 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb454027070 con 0x5604cf2c61c0 2024-05-21T22:01:01.737 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:01:01 smithi184 ceph-mon[36942]: pgmap v165: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:01:01.737 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:01:01 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T22:01:01.737 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:01:01 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T22:01:01.737 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:01:01 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T22:01:01.738 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.734+0000 7fb45b400700 1 -- 172.21.15.151:0/3141135411 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb454003680 con 0x5604cf2c61c0 2024-05-21T22:01:01.738 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.734+0000 7fb45b400700 1 -- 172.21.15.151:0/3141135411 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb45400f6c0 con 0x5604cf2c61c0 2024-05-21T22:01:01.738 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.735+0000 7fb47b72bb80 1 -- 172.21.15.151:0/3141135411 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5604cf2c61c0 msgr2=0x5604cf2c6640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:01.738 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.735+0000 7fb47b72bb80 1 --2- 172.21.15.151:0/3141135411 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5604cf2c61c0 0x5604cf2c6640 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb454015000 tx=0x7fb454005f00 comp rx=0 tx=0).stop 2024-05-21T22:01:01.738 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.735+0000 7fb46a800700 1 --2- 172.21.15.151:0/2282153041 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fb4500b2a70 0x7fb4500b4f30 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb464009790 tx=0x7fb464009150 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:01:01.739 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.735+0000 7fb47b72bb80 1 -- 172.21.15.151:0/3141135411 shutdown_connections 2024-05-21T22:01:01.739 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.735+0000 7fb47b72bb80 1 --2- 172.21.15.151:0/3141135411 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5604cf2c61c0 0x5604cf2c6640 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:01.739 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.735+0000 7fb47b72bb80 1 --2- 172.21.15.151:0/3141135411 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5604cf2cb690 0x5604cf2cba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:01.739 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.735+0000 7fb47b72bb80 1 --2- 172.21.15.151:0/3141135411 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5604cf2cbfb0 0x5604cf2c5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:01.739 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.735+0000 7fb47b72bb80 1 -- 172.21.15.151:0/3141135411 >> 172.21.15.151:0/3141135411 conn(0x5604cf1f3c60 msgr2=0x5604cf2c0710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:01:01.739 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.736+0000 7fb47b72bb80 1 -- 172.21.15.151:0/3141135411 shutdown_connections 2024-05-21T22:01:01.739 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.736+0000 7fb47b72bb80 1 -- 172.21.15.151:0/3141135411 wait complete. 2024-05-21T22:01:01.739 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.736+0000 7fb47b72bb80 1 -- start start 2024-05-21T22:01:01.739 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.737+0000 7fb47b72bb80 1 Processor -- start 2024-05-21T22:01:01.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.737+0000 7fb47b72bb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5604cf2c61c0 0x5604cf2d1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:01.742 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.737+0000 7fb47b72bb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5604cf2cb690 0x5604cf2cf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:01.742 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.737+0000 7fb47b72bb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5604cf2cbfb0 0x5604cf2cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:01.742 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.737+0000 7fb47b72bb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5604cf11a020 con 0x5604cf2c61c0 2024-05-21T22:01:01.742 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.737+0000 7fb47b72bb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5604cf1b47e0 con 0x5604cf2cb690 2024-05-21T22:01:01.742 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.737+0000 7fb47b72bb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5604cf2d2a00 con 0x5604cf2cbfb0 2024-05-21T22:01:01.742 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.737+0000 7fb47b72bb80 10 client.0 _ll_register_callbacks cb 0x5604cf1bdcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:01:01.742 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.738+0000 7fb469e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5604cf2c61c0 0x5604cf2d1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:01.742 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.738+0000 7fb46a800700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5604cf2cbfb0 0x5604cf2cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:01.742 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.738+0000 7fb46a800700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5604cf2cbfb0 0x5604cf2cf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:48762/0 (socket says 172.21.15.151:48762) 2024-05-21T22:01:01.742 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.738+0000 7fb46a800700 1 -- 172.21.15.151:0/43093893 learned_addr learned my addr 172.21.15.151:0/43093893 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:01:01.742 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.738+0000 7fb463e00700 1 --2- 172.21.15.151:0/43093893 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5604cf2cb690 0x5604cf2cf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:01.742 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.738+0000 7fb46a800700 1 -- 172.21.15.151:0/43093893 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5604cf2cb690 msgr2=0x5604cf2cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:01.742 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.738+0000 7fb46a800700 1 --2- 172.21.15.151:0/43093893 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5604cf2cb690 0x5604cf2cf050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:01.743 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.738+0000 7fb46a800700 1 -- 172.21.15.151:0/43093893 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5604cf2c61c0 msgr2=0x5604cf2d1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:01.743 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.738+0000 7fb46a800700 1 --2- 172.21.15.151:0/43093893 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5604cf2c61c0 0x5604cf2d1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:01.743 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.738+0000 7fb46a800700 1 -- 172.21.15.151:0/43093893 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604cf163950 con 0x5604cf2cbfb0 2024-05-21T22:01:01.743 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.738+0000 7fb463e00700 1 --2- 172.21.15.151:0/43093893 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5604cf2cb690 0x5604cf2cf050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:01:01.743 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.738+0000 7fb469e00700 1 --2- 172.21.15.151:0/43093893 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5604cf2c61c0 0x5604cf2d1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:01:01.743 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.739+0000 7fb46a800700 1 --2- 172.21.15.151:0/43093893 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5604cf2cbfb0 0x5604cf2cf590 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb464007550 tx=0x7fb464004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e290034a2f538bec server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:01:01.743 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.739+0000 7fb44fe00700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-21T22:01:01.743 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.739+0000 7fb44fe00700 1 -- 172.21.15.151:0/43093893 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb464027020 con 0x5604cf2cbfb0 2024-05-21T22:01:01.743 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.739+0000 7fb44fe00700 1 -- 172.21.15.151:0/43093893 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fb464023440 con 0x5604cf2cbfb0 2024-05-21T22:01:01.743 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.739+0000 7fb47b72bb80 1 -- 172.21.15.151:0/43093893 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb454003680 con 0x5604cf2cbfb0 2024-05-21T22:01:01.743 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.739+0000 7fb44fe00700 1 -- 172.21.15.151:0/43093893 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb464032050 con 0x5604cf2cbfb0 2024-05-21T22:01:01.743 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.739+0000 7fb47b72bb80 10 client.26488 fetch_fsmap learned FSMap version 29 2024-05-21T22:01:01.743 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.739+0000 7fb47b72bb80 1 -- 172.21.15.151:0/43093893 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5604cf1e40e0 con 0x5604cf2cbfb0 2024-05-21T22:01:01.743 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.740+0000 7fb44fe00700 1 -- 172.21.15.151:0/43093893 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb454003680 con 0x5604cf2cbfb0 2024-05-21T22:01:01.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.740+0000 7fb44fe00700 1 -- 172.21.15.151:0/43093893 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fb464016660 con 0x5604cf2cbfb0 2024-05-21T22:01:01.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.740+0000 7fb47b72bb80 10 client.26488 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:01:01.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.740+0000 7fb47b72bb80 10 client.26488 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T22:01:01.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.740+0000 7fb47b72bb80 20 client.26488 populate_metadata read hostname 'smithi151' 2024-05-21T22:01:01.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.740+0000 7fb47b72bb80 1 --2- 172.21.15.151:0/43093893 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5604cf2df5c0 0x5604cf2e19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:01.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.740+0000 7fb47b72bb80 4 client.26488 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T22:01:01.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.740+0000 7fb47b72bb80 1 -- 172.21.15.151:0/43093893 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5604cf225e10 con 0x5604cf2df5c0 2024-05-21T22:01:01.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.741+0000 7fb469e00700 1 --2- 172.21.15.151:0/43093893 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5604cf2df5c0 0x5604cf2e19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:01.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.741+0000 7fb469e00700 1 --2- 172.21.15.151:0/43093893 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5604cf2df5c0 0x5604cf2e19d0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbde4ca64f7ecda6 server_cookie=4f335f3e2e9799d in_seq=0 out_seq=0 2024-05-21T22:01:01.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.741+0000 7fb44fe00700 10 client.26488 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T22:01:01.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.742+0000 7fb44fe00700 1 -- 172.21.15.151:0/43093893 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5604cf225e10 con 0x5604cf2df5c0 2024-05-21T22:01:01.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.742+0000 7fb44fe00700 10 client.26488 handle_command_reply: tid=0 2024-05-21T22:01:01.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.742+0000 7fb47b72bb80 1 client.26488 shutdown 2024-05-21T22:01:01.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.742+0000 7fb47b72bb80 1 -- 172.21.15.151:0/43093893 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5604cf2cbfb0 msgr2=0x5604cf2cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:01.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.742+0000 7fb47b72bb80 1 --2- 172.21.15.151:0/43093893 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5604cf2cbfb0 0x5604cf2cf590 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb464007550 tx=0x7fb464004ad0 comp rx=0 tx=0).stop 2024-05-21T22:01:01.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.742+0000 7fb47b72bb80 1 -- 172.21.15.151:0/43093893 shutdown_connections 2024-05-21T22:01:01.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.742+0000 7fb47b72bb80 1 --2- 172.21.15.151:0/43093893 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5604cf2cbfb0 0x5604cf2cf590 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:01.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.742+0000 7fb47b72bb80 1 --2- 172.21.15.151:0/43093893 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5604cf2c61c0 0x5604cf2d1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:01.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.742+0000 7fb47b72bb80 1 --2- 172.21.15.151:0/43093893 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5604cf2df5c0 0x5604cf2e19d0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:01.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.742+0000 7fb47b72bb80 1 --2- 172.21.15.151:0/43093893 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5604cf2cb690 0x5604cf2cf050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:01.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.742+0000 7fb47b72bb80 1 -- 172.21.15.151:0/43093893 >> 172.21.15.151:0/43093893 conn(0x5604cf1f3c60 msgr2=0x5604cf2bfdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:01:01.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.742+0000 7fb44fe00700 0 client.26488 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T22:01:01.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.742+0000 7fb47b72bb80 1 -- 172.21.15.151:0/43093893 shutdown_connections 2024-05-21T22:01:01.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.742+0000 7fb47b72bb80 1 -- 172.21.15.151:0/43093893 wait complete. 2024-05-21T22:01:01.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.743+0000 7fb47b72bb80 20 client.26488 trim_cache size 0 max 16384 2024-05-21T22:01:01.766 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.765+0000 7fb47b72bb80 1 Processor -- start 2024-05-21T22:01:01.766 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.765+0000 7fb47b72bb80 1 -- start start 2024-05-21T22:01:01.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.765+0000 7fb47b72bb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5604cf2c61c0 0x5604cf2c5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:01.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.765+0000 7fb47b72bb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5604cf2cb690 0x5604cf2ccaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:01.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.765+0000 7fb47b72bb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5604cf2cbfb0 0x5604cf2c0570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:01.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.765+0000 7fb47b72bb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb46c1afff0 con 0x5604cf2cbfb0 2024-05-21T22:01:01.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.765+0000 7fb47b72bb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb46c1b1b30 con 0x5604cf2cb690 2024-05-21T22:01:01.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.765+0000 7fb47b72bb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb46c1b06c0 con 0x5604cf2c61c0 2024-05-21T22:01:01.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.766+0000 7fb469e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5604cf2c61c0 0x5604cf2c5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:01.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.766+0000 7fb469e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5604cf2c61c0 0x5604cf2c5ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:48768/0 (socket says 172.21.15.151:48768) 2024-05-21T22:01:01.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.766+0000 7fb469e00700 1 -- 172.21.15.151:0/2323650538 learned_addr learned my addr 172.21.15.151:0/2323650538 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:01:01.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.766+0000 7fb46a800700 1 --2- 172.21.15.151:0/2323650538 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5604cf2cbfb0 0x5604cf2c0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:01.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.766+0000 7fb463e00700 1 --2- 172.21.15.151:0/2323650538 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5604cf2cb690 0x5604cf2ccaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:01.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.766+0000 7fb469e00700 1 -- 172.21.15.151:0/2323650538 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5604cf2cb690 msgr2=0x5604cf2ccaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:01.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.766+0000 7fb469e00700 1 --2- 172.21.15.151:0/2323650538 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5604cf2cb690 0x5604cf2ccaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:01.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.766+0000 7fb469e00700 1 -- 172.21.15.151:0/2323650538 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5604cf2cbfb0 msgr2=0x5604cf2c0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:01.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.766+0000 7fb469e00700 1 --2- 172.21.15.151:0/2323650538 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5604cf2cbfb0 0x5604cf2c0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:01.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.766+0000 7fb469e00700 1 -- 172.21.15.151:0/2323650538 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604cf2c1430 con 0x5604cf2c61c0 2024-05-21T22:01:01.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.766+0000 7fb46a800700 1 --2- 172.21.15.151:0/2323650538 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5604cf2cbfb0 0x5604cf2c0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T22:01:01.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.766+0000 7fb469e00700 1 --2- 172.21.15.151:0/2323650538 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5604cf2c61c0 0x5604cf2c5ba0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb45c0a9a90 tx=0x7fb45c0e4cc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=beee6206ad14bc10 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:01:01.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.767+0000 7fb45aa00700 1 -- 172.21.15.151:0/2323650538 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb45c0a59c0 con 0x5604cf2c61c0 2024-05-21T22:01:01.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.767+0000 7fb45aa00700 1 -- 172.21.15.151:0/2323650538 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb45c0e5e50 con 0x5604cf2c61c0 2024-05-21T22:01:01.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.767+0000 7fb45aa00700 1 -- 172.21.15.151:0/2323650538 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb45c0a5290 con 0x5604cf2c61c0 2024-05-21T22:01:01.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.767+0000 7fb47b72bb80 1 -- 172.21.15.151:0/2323650538 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5604cf2c61c0 msgr2=0x5604cf2c5ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:01.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.767+0000 7fb47b72bb80 1 --2- 172.21.15.151:0/2323650538 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5604cf2c61c0 0x5604cf2c5ba0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb45c0a9a90 tx=0x7fb45c0e4cc0 comp rx=0 tx=0).stop 2024-05-21T22:01:01.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.768+0000 7fb47b72bb80 1 -- 172.21.15.151:0/2323650538 shutdown_connections 2024-05-21T22:01:01.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.768+0000 7fb47b72bb80 1 --2- 172.21.15.151:0/2323650538 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5604cf2cbfb0 0x5604cf2c0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:01.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.768+0000 7fb47b72bb80 1 --2- 172.21.15.151:0/2323650538 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5604cf2c61c0 0x5604cf2c5ba0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:01.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.768+0000 7fb47b72bb80 1 --2- 172.21.15.151:0/2323650538 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5604cf2cb690 0x5604cf2ccaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:01.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.768+0000 7fb47b72bb80 1 -- 172.21.15.151:0/2323650538 >> 172.21.15.151:0/2323650538 conn(0x5604cf1f3c60 msgr2=0x5604cf2cc390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:01:01.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.768+0000 7fb47b72bb80 1 -- 172.21.15.151:0/2323650538 shutdown_connections 2024-05-21T22:01:01.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.768+0000 7fb47b72bb80 1 -- 172.21.15.151:0/2323650538 wait complete. 2024-05-21T22:01:01.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.768+0000 7fb47b72bb80 1 -- start start 2024-05-21T22:01:01.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.768+0000 7fb47b72bb80 1 Processor -- start 2024-05-21T22:01:01.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.769+0000 7fb47b72bb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5604cf2c61c0 0x5604cf2d7e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:01.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.769+0000 7fb47b72bb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5604cf2cb690 0x5604cf2d83b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:01.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.769+0000 7fb47b72bb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5604cf2cbfb0 0x5604cf2d4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:01.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.769+0000 7fb47b72bb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb46c1b1460 con 0x5604cf2cbfb0 2024-05-21T22:01:01.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.769+0000 7fb47b72bb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb46c1b0d90 con 0x5604cf2c61c0 2024-05-21T22:01:01.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.769+0000 7fb47b72bb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb46c1adf90 con 0x5604cf2cb690 2024-05-21T22:01:01.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.769+0000 7fb47b72bb80 10 client.0 _ll_register_callbacks cb 0x5604cf1bdcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:01:01.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.769+0000 7fb46a800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5604cf2cbfb0 0x5604cf2d4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:01.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.769+0000 7fb46a800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5604cf2cbfb0 0x5604cf2d4c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:46858/0 (socket says 172.21.15.151:46858) 2024-05-21T22:01:01.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.769+0000 7fb469e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5604cf2c61c0 0x5604cf2d7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:01.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.769+0000 7fb46a800700 1 -- 172.21.15.151:0/1385035296 learned_addr learned my addr 172.21.15.151:0/1385035296 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:01:01.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.769+0000 7fb463e00700 1 --2- 172.21.15.151:0/1385035296 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5604cf2cb690 0x5604cf2d83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:01.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.770+0000 7fb46a800700 1 -- 172.21.15.151:0/1385035296 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5604cf2cb690 msgr2=0x5604cf2d83b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:01.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.770+0000 7fb46a800700 1 --2- 172.21.15.151:0/1385035296 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5604cf2cb690 0x5604cf2d83b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:01.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.770+0000 7fb46a800700 1 -- 172.21.15.151:0/1385035296 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5604cf2c61c0 msgr2=0x5604cf2d7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:01.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.770+0000 7fb46a800700 1 --2- 172.21.15.151:0/1385035296 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5604cf2c61c0 0x5604cf2d7e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:01.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.770+0000 7fb46a800700 1 -- 172.21.15.151:0/1385035296 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604cf2d54a0 con 0x5604cf2cbfb0 2024-05-21T22:01:01.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.770+0000 7fb463e00700 1 --2- 172.21.15.151:0/1385035296 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5604cf2cb690 0x5604cf2d83b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:01:01.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.770+0000 7fb469e00700 1 --2- 172.21.15.151:0/1385035296 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5604cf2c61c0 0x5604cf2d7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:01:01.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.770+0000 7fb46a800700 1 --2- 172.21.15.151:0/1385035296 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5604cf2cbfb0 0x5604cf2d4c60 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fb464016b00 tx=0x7fb464004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fecbdf05fa15b6b server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:01:01.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.770+0000 7fb44fe00700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-21T22:01:01.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.771+0000 7fb44fe00700 1 -- 172.21.15.151:0/1385035296 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb464027040 con 0x5604cf2cbfb0 2024-05-21T22:01:01.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.771+0000 7fb47b72bb80 1 -- 172.21.15.151:0/1385035296 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb45c0e5e50 con 0x5604cf2cbfb0 2024-05-21T22:01:01.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.771+0000 7fb44fe00700 1 -- 172.21.15.151:0/1385035296 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fb464007d40 con 0x5604cf2cbfb0 2024-05-21T22:01:01.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.771+0000 7fb44fe00700 1 -- 172.21.15.151:0/1385035296 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb464046050 con 0x5604cf2cbfb0 2024-05-21T22:01:01.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.771+0000 7fb47b72bb80 10 client.16356 fetch_fsmap learned FSMap version 29 2024-05-21T22:01:01.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.771+0000 7fb47b72bb80 1 -- 172.21.15.151:0/1385035296 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5604cf34ace0 con 0x5604cf2cbfb0 2024-05-21T22:01:01.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.771+0000 7fb44fe00700 1 -- 172.21.15.151:0/1385035296 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb45c0e5e50 con 0x5604cf2cbfb0 2024-05-21T22:01:01.776 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.771+0000 7fb44fe00700 1 -- 172.21.15.151:0/1385035296 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fb4640052e0 con 0x5604cf2cbfb0 2024-05-21T22:01:01.776 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.771+0000 7fb47b72bb80 10 client.16356 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:01:01.776 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.772+0000 7fb47b72bb80 10 client.16356 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T22:01:01.776 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.772+0000 7fb47b72bb80 20 client.16356 populate_metadata read hostname 'smithi151' 2024-05-21T22:01:01.776 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.772+0000 7fb47b72bb80 1 --2- 172.21.15.151:0/1385035296 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5604cf34b3c0 0x5604cf34d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:01.776 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.772+0000 7fb47b72bb80 4 client.16356 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-21T22:01:01.776 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.772+0000 7fb47b72bb80 1 -- 172.21.15.151:0/1385035296 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5604cf21f5f0 con 0x5604cf34b3c0 2024-05-21T22:01:01.776 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.772+0000 7fb469e00700 1 --2- 172.21.15.151:0/1385035296 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5604cf34b3c0 0x5604cf34d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:01.776 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.773+0000 7fb469e00700 1 --2- 172.21.15.151:0/1385035296 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5604cf34b3c0 0x5604cf34d7e0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=396c792b9b5d48ec server_cookie=af922eac0e264300 in_seq=0 out_seq=0 2024-05-21T22:01:01.776 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.773+0000 7fb44fe00700 10 client.16356 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T22:01:01.776 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.774+0000 7fb44fe00700 1 -- 172.21.15.151:0/1385035296 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5604cf21f5f0 con 0x5604cf34b3c0 2024-05-21T22:01:01.776 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.774+0000 7fb44fe00700 10 client.16356 handle_command_reply: tid=0 2024-05-21T22:01:01.776 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.774+0000 7fb47b72bb80 1 client.16356 shutdown 2024-05-21T22:01:01.777 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.774+0000 7fb47b72bb80 1 -- 172.21.15.151:0/1385035296 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5604cf2cbfb0 msgr2=0x5604cf2d4c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:01.777 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.774+0000 7fb47b72bb80 1 --2- 172.21.15.151:0/1385035296 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5604cf2cbfb0 0x5604cf2d4c60 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fb464016b00 tx=0x7fb464004b40 comp rx=0 tx=0).stop 2024-05-21T22:01:01.777 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.774+0000 7fb47b72bb80 1 -- 172.21.15.151:0/1385035296 shutdown_connections 2024-05-21T22:01:01.777 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.774+0000 7fb47b72bb80 1 --2- 172.21.15.151:0/1385035296 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5604cf34b3c0 0x5604cf34d7e0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:01.777 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.774+0000 7fb47b72bb80 1 --2- 172.21.15.151:0/1385035296 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5604cf2c61c0 0x5604cf2d7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:01.778 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.774+0000 7fb47b72bb80 1 --2- 172.21.15.151:0/1385035296 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5604cf2cbfb0 0x5604cf2d4c60 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:01.778 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.774+0000 7fb47b72bb80 1 --2- 172.21.15.151:0/1385035296 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5604cf2cb690 0x5604cf2d83b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:01.778 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.774+0000 7fb47b72bb80 1 -- 172.21.15.151:0/1385035296 >> 172.21.15.151:0/1385035296 conn(0x5604cf1f3c60 msgr2=0x5604cf2d05f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:01:01.778 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.774+0000 7fb44fe00700 0 client.16356 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T22:01:01.778 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.775+0000 7fb47b72bb80 1 -- 172.21.15.151:0/1385035296 shutdown_connections 2024-05-21T22:01:01.778 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.775+0000 7fb47b72bb80 1 -- 172.21.15.151:0/1385035296 wait complete. 2024-05-21T22:01:01.778 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.775+0000 7fb47b72bb80 20 client.16356 trim_cache size 0 max 16384 2024-05-21T22:01:01.778 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-05-21T22:01:01.778 INFO:teuthology.orchestra.run.smithi151.stdout: "return_code": 0, 2024-05-21T22:01:01.778 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_tag": "15b07b4e-663c-484d-a87e-1ecdd030b26d", 2024-05-21T22:01:01.778 INFO:teuthology.orchestra.run.smithi151.stdout: "mode": "asynchronous" 2024-05-21T22:01:01.779 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-05-21T22:01:01.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.775+0000 7fb470e00700 1 -- 172.21.15.151:0/2282153041 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fb4500b2a70 msgr2=0x7fb4500b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:01.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.775+0000 7fb470e00700 1 --2- 172.21.15.151:0/2282153041 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fb4500b2a70 0x7fb4500b4f30 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb464009790 tx=0x7fb464009150 comp rx=0 tx=0).stop 2024-05-21T22:01:01.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.776+0000 7fb470e00700 1 -- 172.21.15.151:0/2282153041 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb46c06c150 msgr2=0x7fb46c19be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:01.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.776+0000 7fb470e00700 1 --2- 172.21.15.151:0/2282153041 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb46c06c150 0x7fb46c19be60 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb45c00e9c0 tx=0x7fb45c00ecd0 comp rx=0 tx=0).stop 2024-05-21T22:01:01.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.776+0000 7fb470e00700 1 -- 172.21.15.151:0/2282153041 shutdown_connections 2024-05-21T22:01:01.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.776+0000 7fb470e00700 1 --2- 172.21.15.151:0/2282153041 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fb4500b2a70 0x7fb4500b4f30 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:01.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.776+0000 7fb470e00700 1 --2- 172.21.15.151:0/2282153041 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb46c10fe90 0x7fb46c1a0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:01.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.776+0000 7fb470e00700 1 --2- 172.21.15.151:0/2282153041 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb46c06c150 0x7fb46c19be60 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:01.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.776+0000 7fb470e00700 1 --2- 172.21.15.151:0/2282153041 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb46c104480 0x7fb46c19c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:01.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.776+0000 7fb470e00700 1 -- 172.21.15.151:0/2282153041 >> 172.21.15.151:0/2282153041 conn(0x7fb46c0793f0 msgr2=0x7fb46c101810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:01:01.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.776+0000 7fb470e00700 1 -- 172.21.15.151:0/2282153041 shutdown_connections 2024-05-21T22:01:01.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:01.776+0000 7fb470e00700 1 -- 172.21.15.151:0/2282153041 wait complete. 2024-05-21T22:01:01.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:01:01 smithi164 ceph-mon[36269]: pgmap v165: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:01:01.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:01:01 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T22:01:01.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:01:01 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T22:01:01.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:01:01 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T22:01:02.699 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:01:02 smithi151 ceph-mon[31623]: scrub queued for path: / 2024-05-21T22:01:02.699 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:01:02 smithi151 ceph-mon[31623]: scrub summary: idle+waiting paths [/] 2024-05-21T22:01:02.699 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:01:02 smithi151 ceph-mon[31623]: scrub summary: active paths [/] 2024-05-21T22:01:02.699 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:01:02 smithi151 ceph-mon[31623]: scrub summary: idle+waiting paths [/] 2024-05-21T22:01:02.735 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:01:02 smithi184 ceph-mon[36942]: scrub queued for path: / 2024-05-21T22:01:02.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:01:02 smithi184 ceph-mon[36942]: scrub summary: idle+waiting paths [/] 2024-05-21T22:01:02.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:01:02 smithi184 ceph-mon[36942]: scrub summary: active paths [/] 2024-05-21T22:01:02.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:01:02 smithi184 ceph-mon[36942]: scrub summary: idle+waiting paths [/] 2024-05-21T22:01:02.752 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-21T22:01:02.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:01:02 smithi164 ceph-mon[36269]: scrub queued for path: / 2024-05-21T22:01:02.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:01:02 smithi164 ceph-mon[36269]: scrub summary: idle+waiting paths [/] 2024-05-21T22:01:02.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:01:02 smithi164 ceph-mon[36269]: scrub summary: active paths [/] 2024-05-21T22:01:02.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:01:02 smithi164 ceph-mon[36269]: scrub summary: idle+waiting paths [/] 2024-05-21T22:01:03.193 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:01:03.661 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:01:03 smithi151 ceph-mon[31623]: pgmap v166: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-05-21T22:01:03.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:01:03 smithi184 ceph-mon[36942]: pgmap v166: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-05-21T22:01:03.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:01:03 smithi164 ceph-mon[36269]: pgmap v166: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-05-21T22:01:04.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:01:04 smithi184 ceph-mon[36942]: scrub summary: idle 2024-05-21T22:01:04.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:01:04 smithi164 ceph-mon[36269]: scrub summary: idle 2024-05-21T22:01:04.874 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:01:04 smithi151 ceph-mon[31623]: scrub summary: idle 2024-05-21T22:01:05.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.338+0000 7f5246200700 1 -- 172.21.15.151:0/3418019476 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5240076090 msgr2=0x7f524010fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:05.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.338+0000 7f5246200700 1 --2- 172.21.15.151:0/3418019476 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5240076090 0x7f524010fa00 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f522c009990 tx=0x7f522c009ca0 comp rx=0 tx=0).stop 2024-05-21T22:01:05.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.338+0000 7f5246200700 1 -- 172.21.15.151:0/3418019476 shutdown_connections 2024-05-21T22:01:05.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.338+0000 7f5246200700 1 --2- 172.21.15.151:0/3418019476 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f524010ff40 0x7f5240112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:05.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.338+0000 7f5246200700 1 --2- 172.21.15.151:0/3418019476 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5240075770 0x7f5240075b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:05.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.338+0000 7f5246200700 1 --2- 172.21.15.151:0/3418019476 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5240076090 0x7f524010fa00 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:05.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.338+0000 7f5246200700 1 -- 172.21.15.151:0/3418019476 >> 172.21.15.151:0/3418019476 conn(0x7f524007a6a0 msgr2=0x7f524007aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:01:05.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.338+0000 7f5246200700 1 -- 172.21.15.151:0/3418019476 shutdown_connections 2024-05-21T22:01:05.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.338+0000 7f5246200700 1 -- 172.21.15.151:0/3418019476 wait complete. 2024-05-21T22:01:05.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.339+0000 7f5246200700 1 Processor -- start 2024-05-21T22:01:05.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.340+0000 7f5246200700 1 -- start start 2024-05-21T22:01:05.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.340+0000 7f5246200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5240075770 0x7f524019bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:05.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.340+0000 7f5246200700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5240076090 0x7f524019c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:05.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.340+0000 7f5246200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f524010ff40 0x7f52401a0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:05.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.340+0000 7f5246200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5240115970 con 0x7f524010ff40 2024-05-21T22:01:05.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.340+0000 7f5246200700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f52401157f0 con 0x7f5240075770 2024-05-21T22:01:05.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.340+0000 7f5246200700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5240115af0 con 0x7f5240076090 2024-05-21T22:01:05.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.340+0000 7f523fe00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f524010ff40 0x7f52401a0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:05.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.340+0000 7f523f400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5240075770 0x7f524019bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:05.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.341+0000 7f523ea00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5240076090 0x7f524019c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:05.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.341+0000 7f523f400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5240075770 0x7f524019bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:45232/0 (socket says 172.21.15.151:45232) 2024-05-21T22:01:05.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.341+0000 7f523f400700 1 -- 172.21.15.151:0/1977119477 learned_addr learned my addr 172.21.15.151:0/1977119477 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:01:05.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.341+0000 7f523f400700 1 -- 172.21.15.151:0/1977119477 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5240076090 msgr2=0x7f524019c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:05.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.341+0000 7f523f400700 1 --2- 172.21.15.151:0/1977119477 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5240076090 0x7f524019c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:05.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.341+0000 7f523f400700 1 -- 172.21.15.151:0/1977119477 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f524010ff40 msgr2=0x7f52401a0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:05.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.341+0000 7f523f400700 1 --2- 172.21.15.151:0/1977119477 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f524010ff40 0x7f52401a0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:05.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.341+0000 7f523f400700 1 -- 172.21.15.151:0/1977119477 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52401a0e80 con 0x7f5240075770 2024-05-21T22:01:05.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.341+0000 7f523f400700 1 --2- 172.21.15.151:0/1977119477 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5240075770 0x7f524019bec0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f523000e9c0 tx=0x7f523000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:01:05.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.341+0000 7f5237e00700 1 -- 172.21.15.151:0/1977119477 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f523000cbd0 con 0x7f5240075770 2024-05-21T22:01:05.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.342+0000 7f5237e00700 1 -- 172.21.15.151:0/1977119477 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f523000ee60 con 0x7f5240075770 2024-05-21T22:01:05.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.342+0000 7f5237e00700 1 -- 172.21.15.151:0/1977119477 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52300055c0 con 0x7f5240075770 2024-05-21T22:01:05.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.342+0000 7f5246200700 1 -- 172.21.15.151:0/1977119477 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52401a1170 con 0x7f5240075770 2024-05-21T22:01:05.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.342+0000 7f5246200700 1 -- 172.21.15.151:0/1977119477 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52401a89f0 con 0x7f5240075770 2024-05-21T22:01:05.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.347+0000 7f5237e00700 1 -- 172.21.15.151:0/1977119477 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7f5230004750 con 0x7f5240075770 2024-05-21T22:01:05.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.347+0000 7f5250a9eb80 1 Processor -- start 2024-05-21T22:01:05.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.348+0000 7f5250a9eb80 1 -- start start 2024-05-21T22:01:05.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.348+0000 7f5250a9eb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a575faceb0 0x55a575fad290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:05.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.348+0000 7f5250a9eb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a575fad7d0 0x55a575fa7260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:05.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.348+0000 7f5250a9eb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a575fa7880 0x55a575fa7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:05.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.348+0000 7f5250a9eb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a575f05e40 con 0x55a575fa7880 2024-05-21T22:01:05.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.348+0000 7f5250a9eb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55a575ef59e0 con 0x55a575fad7d0 2024-05-21T22:01:05.350 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.348+0000 7f5250a9eb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55a575ef9a20 con 0x55a575faceb0 2024-05-21T22:01:05.350 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.348+0000 7f523f400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a575fad7d0 0x55a575fa7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:05.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.348+0000 7f523ea00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a575fa7880 0x55a575fa7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:05.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.348+0000 7f523fe00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a575faceb0 0x55a575fad290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:05.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.348+0000 7f523f400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a575fad7d0 0x55a575fa7260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:45238/0 (socket says 172.21.15.151:45238) 2024-05-21T22:01:05.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.348+0000 7f523f400700 1 -- 172.21.15.151:0/3490464054 learned_addr learned my addr 172.21.15.151:0/3490464054 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:01:05.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.349+0000 7f523f400700 1 -- 172.21.15.151:0/3490464054 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a575faceb0 msgr2=0x55a575fad290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:05.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.349+0000 7f523f400700 1 --2- 172.21.15.151:0/3490464054 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a575faceb0 0x55a575fad290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:05.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.349+0000 7f523f400700 1 -- 172.21.15.151:0/3490464054 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a575fa7880 msgr2=0x55a575fa7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:05.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.349+0000 7f523f400700 1 --2- 172.21.15.151:0/3490464054 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a575fa7880 0x55a575fa7ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:05.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.349+0000 7f523f400700 1 -- 172.21.15.151:0/3490464054 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a575ebe420 con 0x55a575fad7d0 2024-05-21T22:01:05.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.349+0000 7f523ea00700 1 --2- 172.21.15.151:0/3490464054 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a575fa7880 0x55a575fa7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:01:05.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.349+0000 7f523f400700 1 --2- 172.21.15.151:0/3490464054 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a575fad7d0 0x55a575fa7260 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f52300dff30 tx=0x7f52300e8a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf8a500fc4f843f server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:01:05.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.349+0000 7f5237e00700 1 --2- 172.21.15.151:0/1977119477 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f52240b29f0 0x7f52240b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:05.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.350+0000 7f523fe00700 1 --2- 172.21.15.151:0/1977119477 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f52240b29f0 0x7f52240b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:05.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.350+0000 7f5234c00700 1 -- 172.21.15.151:0/3490464054 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52300f1070 con 0x55a575fad7d0 2024-05-21T22:01:05.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.350+0000 7f5234c00700 1 -- 172.21.15.151:0/3490464054 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52300a9690 con 0x55a575fad7d0 2024-05-21T22:01:05.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.350+0000 7f5234c00700 1 -- 172.21.15.151:0/3490464054 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52300a94d0 con 0x55a575fad7d0 2024-05-21T22:01:05.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.350+0000 7f5237e00700 1 -- 172.21.15.151:0/1977119477 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f52300e28f0 con 0x7f5240075770 2024-05-21T22:01:05.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.351+0000 7f5250a9eb80 1 -- 172.21.15.151:0/3490464054 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a575fad7d0 msgr2=0x55a575fa7260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:05.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.351+0000 7f523fe00700 1 --2- 172.21.15.151:0/1977119477 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f52240b29f0 0x7f52240b4eb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f52380127f0 tx=0x7f523800b040 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:01:05.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.351+0000 7f5250a9eb80 1 --2- 172.21.15.151:0/3490464054 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a575fad7d0 0x55a575fa7260 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f52300dff30 tx=0x7f52300e8a60 comp rx=0 tx=0).stop 2024-05-21T22:01:05.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.351+0000 7f5250a9eb80 1 -- 172.21.15.151:0/3490464054 shutdown_connections 2024-05-21T22:01:05.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.351+0000 7f5250a9eb80 1 --2- 172.21.15.151:0/3490464054 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a575fa7880 0x55a575fa7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:05.354 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.351+0000 7f5250a9eb80 1 --2- 172.21.15.151:0/3490464054 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a575faceb0 0x55a575fad290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:05.354 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.351+0000 7f5250a9eb80 1 --2- 172.21.15.151:0/3490464054 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a575fad7d0 0x55a575fa7260 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:05.354 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.351+0000 7f5250a9eb80 1 -- 172.21.15.151:0/3490464054 >> 172.21.15.151:0/3490464054 conn(0x55a575ed5be0 msgr2=0x55a575fa2010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:01:05.354 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.351+0000 7f5250a9eb80 1 -- 172.21.15.151:0/3490464054 shutdown_connections 2024-05-21T22:01:05.354 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.351+0000 7f5250a9eb80 1 -- 172.21.15.151:0/3490464054 wait complete. 2024-05-21T22:01:05.354 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.352+0000 7f5250a9eb80 1 -- start start 2024-05-21T22:01:05.354 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.352+0000 7f5250a9eb80 1 Processor -- start 2024-05-21T22:01:05.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.352+0000 7f5250a9eb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a575fa7880 0x55a575fb34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:05.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.352+0000 7f5250a9eb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a575faceb0 0x55a575fb08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:05.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.352+0000 7f5250a9eb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a575fad7d0 0x55a575fb0de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:05.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.352+0000 7f5250a9eb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a575dfbfa0 con 0x55a575faceb0 2024-05-21T22:01:05.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.352+0000 7f5250a9eb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55a575e96760 con 0x55a575fa7880 2024-05-21T22:01:05.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.352+0000 7f5250a9eb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55a575fb4240 con 0x55a575fad7d0 2024-05-21T22:01:05.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.352+0000 7f5250a9eb80 10 client.0 _ll_register_callbacks cb 0x55a575e9fc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:01:05.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.353+0000 7f523ea00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a575faceb0 0x55a575fb08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:05.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.353+0000 7f523f400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a575fa7880 0x55a575fb34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:05.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.353+0000 7f523f400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a575fa7880 0x55a575fb34b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:45242/0 (socket says 172.21.15.151:45242) 2024-05-21T22:01:05.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.353+0000 7f523f400700 1 -- 172.21.15.151:0/140849150 learned_addr learned my addr 172.21.15.151:0/140849150 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:01:05.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.353+0000 7f523fe00700 1 --2- 172.21.15.151:0/140849150 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a575fad7d0 0x55a575fb0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:05.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.353+0000 7f523f400700 1 -- 172.21.15.151:0/140849150 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a575fad7d0 msgr2=0x55a575fb0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:05.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.353+0000 7f523f400700 1 --2- 172.21.15.151:0/140849150 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a575fad7d0 0x55a575fb0de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:05.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.353+0000 7f523f400700 1 -- 172.21.15.151:0/140849150 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a575faceb0 msgr2=0x55a575fb08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:05.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.353+0000 7f523f400700 1 --2- 172.21.15.151:0/140849150 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a575faceb0 0x55a575fb08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:05.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.353+0000 7f523f400700 1 -- 172.21.15.151:0/140849150 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a575e458d0 con 0x55a575fa7880 2024-05-21T22:01:05.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.353+0000 7f523f400700 1 --2- 172.21.15.151:0/140849150 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a575fa7880 0x55a575fb34b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f5230009fd0 tx=0x7f52300e04a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69a44614995a7408 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:01:05.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.353+0000 7f5229600700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-21T22:01:05.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.354+0000 7f5229600700 1 -- 172.21.15.151:0/140849150 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52300f1020 con 0x55a575fa7880 2024-05-21T22:01:05.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.354+0000 7f5229600700 1 -- 172.21.15.151:0/140849150 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f52300a9ff0 con 0x55a575fa7880 2024-05-21T22:01:05.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.354+0000 7f5250a9eb80 1 -- 172.21.15.151:0/140849150 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52300a9690 con 0x55a575fa7880 2024-05-21T22:01:05.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.354+0000 7f5229600700 1 -- 172.21.15.151:0/140849150 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5230051e90 con 0x55a575fa7880 2024-05-21T22:01:05.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.354+0000 7f5250a9eb80 10 client.26492 fetch_fsmap learned FSMap version 29 2024-05-21T22:01:05.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.354+0000 7f5250a9eb80 1 -- 172.21.15.151:0/140849150 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a575ec6060 con 0x55a575fa7880 2024-05-21T22:01:05.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.354+0000 7f5229600700 1 -- 172.21.15.151:0/140849150 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52300a9690 con 0x55a575fa7880 2024-05-21T22:01:05.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.355+0000 7f5229600700 1 -- 172.21.15.151:0/140849150 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f52300e2ce0 con 0x55a575fa7880 2024-05-21T22:01:05.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.355+0000 7f5250a9eb80 10 client.26492 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:01:05.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.355+0000 7f5250a9eb80 10 client.26492 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T22:01:05.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.355+0000 7f5250a9eb80 20 client.26492 populate_metadata read hostname 'smithi151' 2024-05-21T22:01:05.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.355+0000 7f5250a9eb80 1 --2- 172.21.15.151:0/140849150 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55a575fc0bc0 0x55a575fc2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:05.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.355+0000 7f5250a9eb80 4 client.26492 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T22:01:05.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.355+0000 7f5250a9eb80 1 -- 172.21.15.151:0/140849150 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a575f07d90 con 0x55a575fc0bc0 2024-05-21T22:01:05.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.355+0000 7f523ea00700 1 --2- 172.21.15.151:0/140849150 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55a575fc0bc0 0x55a575fc2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:05.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.356+0000 7f523ea00700 1 --2- 172.21.15.151:0/140849150 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55a575fc0bc0 0x55a575fc2fb0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81c1a0d9974a1c03 server_cookie=2ec166487b3f5db7 in_seq=0 out_seq=0 2024-05-21T22:01:05.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.356+0000 7f5229600700 10 client.26492 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T22:01:05.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.356+0000 7f5229600700 1 -- 172.21.15.151:0/140849150 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55a575f07d90 con 0x55a575fc0bc0 2024-05-21T22:01:05.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.356+0000 7f5229600700 10 client.26492 handle_command_reply: tid=0 2024-05-21T22:01:05.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.356+0000 7f5250a9eb80 1 client.26492 shutdown 2024-05-21T22:01:05.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.357+0000 7f5250a9eb80 1 -- 172.21.15.151:0/140849150 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a575fa7880 msgr2=0x55a575fb34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:05.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.357+0000 7f5250a9eb80 1 --2- 172.21.15.151:0/140849150 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a575fa7880 0x55a575fb34b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f5230009fd0 tx=0x7f52300e04a0 comp rx=0 tx=0).stop 2024-05-21T22:01:05.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.357+0000 7f5250a9eb80 1 -- 172.21.15.151:0/140849150 shutdown_connections 2024-05-21T22:01:05.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.357+0000 7f5250a9eb80 1 --2- 172.21.15.151:0/140849150 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55a575fc0bc0 0x55a575fc2fb0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:05.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.357+0000 7f5250a9eb80 1 --2- 172.21.15.151:0/140849150 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a575fa7880 0x55a575fb34b0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:05.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.357+0000 7f5250a9eb80 1 --2- 172.21.15.151:0/140849150 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a575fad7d0 0x55a575fb0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:05.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.357+0000 7f5250a9eb80 1 --2- 172.21.15.151:0/140849150 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a575faceb0 0x55a575fb08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:05.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.357+0000 7f5229600700 0 client.26492 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T22:01:05.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.357+0000 7f5250a9eb80 1 -- 172.21.15.151:0/140849150 >> 172.21.15.151:0/140849150 conn(0x55a575ed5be0 msgr2=0x55a575fa6250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:01:05.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.357+0000 7f5250a9eb80 1 -- 172.21.15.151:0/140849150 shutdown_connections 2024-05-21T22:01:05.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.357+0000 7f5250a9eb80 1 -- 172.21.15.151:0/140849150 wait complete. 2024-05-21T22:01:05.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.357+0000 7f5250a9eb80 20 client.26492 trim_cache size 0 max 16384 2024-05-21T22:01:05.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.379+0000 7f5250a9eb80 1 Processor -- start 2024-05-21T22:01:05.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.379+0000 7f5250a9eb80 1 -- start start 2024-05-21T22:01:05.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.380+0000 7f5250a9eb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a575fa7880 0x55a575fa7260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:05.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.380+0000 7f5250a9eb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a575faceb0 0x55a575faa9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:05.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.380+0000 7f5250a9eb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a575fad7d0 0x55a575faaf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:05.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.380+0000 7f5250a9eb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f52401b00d0 con 0x55a575fa7880 2024-05-21T22:01:05.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.380+0000 7f5250a9eb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f52401b1c10 con 0x55a575fad7d0 2024-05-21T22:01:05.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.380+0000 7f5250a9eb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f52401b07a0 con 0x55a575faceb0 2024-05-21T22:01:05.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.380+0000 7f523f400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a575fa7880 0x55a575fa7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:05.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.380+0000 7f523f400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a575fa7880 0x55a575fa7260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:46914/0 (socket says 172.21.15.151:46914) 2024-05-21T22:01:05.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.380+0000 7f523ea00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a575faceb0 0x55a575faa9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:05.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.380+0000 7f523f400700 1 -- 172.21.15.151:0/1312924582 learned_addr learned my addr 172.21.15.151:0/1312924582 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:01:05.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.380+0000 7f523fe00700 1 --2- 172.21.15.151:0/1312924582 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a575fad7d0 0x55a575faaf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:05.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.381+0000 7f523ea00700 1 -- 172.21.15.151:0/1312924582 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a575fad7d0 msgr2=0x55a575faaf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:05.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.381+0000 7f523ea00700 1 --2- 172.21.15.151:0/1312924582 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a575fad7d0 0x55a575faaf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:05.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.381+0000 7f523ea00700 1 -- 172.21.15.151:0/1312924582 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a575fa7880 msgr2=0x55a575fa7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:05.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.381+0000 7f523ea00700 1 --2- 172.21.15.151:0/1312924582 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a575fa7880 0x55a575fa7260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:05.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.381+0000 7f523ea00700 1 -- 172.21.15.151:0/1312924582 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a575fa82d0 con 0x55a575faceb0 2024-05-21T22:01:05.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.381+0000 7f523f400700 1 --2- 172.21.15.151:0/1312924582 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a575fa7880 0x55a575fa7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:01:05.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.381+0000 7f523ea00700 1 --2- 172.21.15.151:0/1312924582 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a575faceb0 0x55a575faa9c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f522c000c00 tx=0x7f522c00b260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4fd92cb236a759f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:01:05.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.381+0000 7f522be00700 1 -- 172.21.15.151:0/1312924582 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f522c00b920 con 0x55a575faceb0 2024-05-21T22:01:05.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.381+0000 7f522be00700 1 -- 172.21.15.151:0/1312924582 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f522c003680 con 0x55a575faceb0 2024-05-21T22:01:05.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.381+0000 7f522be00700 1 -- 172.21.15.151:0/1312924582 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f522c00f6c0 con 0x55a575faceb0 2024-05-21T22:01:05.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.382+0000 7f5250a9eb80 1 -- 172.21.15.151:0/1312924582 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a575faceb0 msgr2=0x55a575faa9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:05.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.382+0000 7f5250a9eb80 1 --2- 172.21.15.151:0/1312924582 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a575faceb0 0x55a575faa9c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f522c000c00 tx=0x7f522c00b260 comp rx=0 tx=0).stop 2024-05-21T22:01:05.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.382+0000 7f5250a9eb80 1 -- 172.21.15.151:0/1312924582 shutdown_connections 2024-05-21T22:01:05.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.382+0000 7f5250a9eb80 1 --2- 172.21.15.151:0/1312924582 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a575fad7d0 0x55a575faaf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:05.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.382+0000 7f5250a9eb80 1 --2- 172.21.15.151:0/1312924582 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a575fa7880 0x55a575fa7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:05.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.382+0000 7f5250a9eb80 1 --2- 172.21.15.151:0/1312924582 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a575faceb0 0x55a575faa9c0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:05.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.382+0000 7f5250a9eb80 1 -- 172.21.15.151:0/1312924582 >> 172.21.15.151:0/1312924582 conn(0x55a575ed5be0 msgr2=0x55a575fb0860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:01:05.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.382+0000 7f5250a9eb80 1 -- 172.21.15.151:0/1312924582 shutdown_connections 2024-05-21T22:01:05.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.382+0000 7f5250a9eb80 1 -- 172.21.15.151:0/1312924582 wait complete. 2024-05-21T22:01:05.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.383+0000 7f5250a9eb80 1 -- start start 2024-05-21T22:01:05.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.383+0000 7f5250a9eb80 1 Processor -- start 2024-05-21T22:01:05.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.383+0000 7f5250a9eb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a575fa7880 0x55a575fc40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:05.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.383+0000 7f5250a9eb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a575faceb0 0x55a575fc4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:05.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.383+0000 7f5250a9eb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a575fad7d0 0x55a57602a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:05.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.383+0000 7f5250a9eb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f52401b1540 con 0x55a575fa7880 2024-05-21T22:01:05.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.383+0000 7f5250a9eb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f52401b0e70 con 0x55a575fad7d0 2024-05-21T22:01:05.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.383+0000 7f5250a9eb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f52401ae070 con 0x55a575faceb0 2024-05-21T22:01:05.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.384+0000 7f5250a9eb80 10 client.0 _ll_register_callbacks cb 0x55a575e9fc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:01:05.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.384+0000 7f523f400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a575fa7880 0x55a575fc40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:05.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.384+0000 7f523f400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a575fa7880 0x55a575fc40f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:46924/0 (socket says 172.21.15.151:46924) 2024-05-21T22:01:05.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.384+0000 7f523f400700 1 -- 172.21.15.151:0/1096616973 learned_addr learned my addr 172.21.15.151:0/1096616973 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:01:05.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.384+0000 7f523ea00700 1 --2- 172.21.15.151:0/1096616973 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a575faceb0 0x55a575fc4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:05.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.384+0000 7f523fe00700 1 --2- 172.21.15.151:0/1096616973 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a575fad7d0 0x55a57602a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:05.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.384+0000 7f523f400700 1 -- 172.21.15.151:0/1096616973 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a575faceb0 msgr2=0x55a575fc4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:05.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.384+0000 7f523f400700 1 --2- 172.21.15.151:0/1096616973 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a575faceb0 0x55a575fc4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:05.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.384+0000 7f523f400700 1 -- 172.21.15.151:0/1096616973 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a575fad7d0 msgr2=0x55a57602a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:05.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.384+0000 7f523f400700 1 --2- 172.21.15.151:0/1096616973 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a575fad7d0 0x55a57602a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:05.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.384+0000 7f523f400700 1 -- 172.21.15.151:0/1096616973 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a57602a940 con 0x55a575fa7880 2024-05-21T22:01:05.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.384+0000 7f523ea00700 1 --2- 172.21.15.151:0/1096616973 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a575faceb0 0x55a575fc4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:01:05.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.385+0000 7f523f400700 1 --2- 172.21.15.151:0/1096616973 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a575fa7880 0x55a575fc40f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f52300a9950 tx=0x7f52300e89b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1874a4fac2247c4b server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:01:05.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.385+0000 7f5229600700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-21T22:01:05.390 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.385+0000 7f5229600700 1 -- 172.21.15.151:0/1096616973 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52300f1040 con 0x55a575fa7880 2024-05-21T22:01:05.390 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.385+0000 7f5229600700 1 -- 172.21.15.151:0/1096616973 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f52300a6170 con 0x55a575fa7880 2024-05-21T22:01:05.390 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.385+0000 7f5250a9eb80 1 -- 172.21.15.151:0/1096616973 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f522c003680 con 0x55a575fa7880 2024-05-21T22:01:05.390 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.385+0000 7f5229600700 1 -- 172.21.15.151:0/1096616973 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f52300a92f0 con 0x55a575fa7880 2024-05-21T22:01:05.390 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.385+0000 7f5250a9eb80 10 client.16374 fetch_fsmap learned FSMap version 29 2024-05-21T22:01:05.390 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.385+0000 7f5250a9eb80 1 -- 172.21.15.151:0/1096616973 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a57602a940 con 0x55a575fa7880 2024-05-21T22:01:05.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.385+0000 7f5229600700 1 -- 172.21.15.151:0/1096616973 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f522c003680 con 0x55a575fa7880 2024-05-21T22:01:05.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.385+0000 7f5229600700 1 -- 172.21.15.151:0/1096616973 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f52300e2b70 con 0x55a575fa7880 2024-05-21T22:01:05.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.386+0000 7f5250a9eb80 10 client.16374 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:01:05.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.386+0000 7f5250a9eb80 10 client.16374 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T22:01:05.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.386+0000 7f5250a9eb80 20 client.16374 populate_metadata read hostname 'smithi151' 2024-05-21T22:01:05.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.386+0000 7f5250a9eb80 1 --2- 172.21.15.151:0/1096616973 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55a57602bdd0 0x55a57602e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:05.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.386+0000 7f5250a9eb80 4 client.16374 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-21T22:01:05.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.386+0000 7f5250a9eb80 1 -- 172.21.15.151:0/1096616973 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a575f01570 con 0x55a57602bdd0 2024-05-21T22:01:05.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.386+0000 7f523ea00700 1 --2- 172.21.15.151:0/1096616973 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55a57602bdd0 0x55a57602e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:05.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.387+0000 7f5229600700 10 client.16374 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T22:01:05.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.387+0000 7f523ea00700 1 --2- 172.21.15.151:0/1096616973 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55a57602bdd0 0x55a57602e1c0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f6ffd0d64de76e5 server_cookie=314ee33a1f1512a4 in_seq=0 out_seq=0 2024-05-21T22:01:05.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.387+0000 7f5229600700 1 -- 172.21.15.151:0/1096616973 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55a575f01570 con 0x55a57602bdd0 2024-05-21T22:01:05.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.387+0000 7f5229600700 10 client.16374 handle_command_reply: tid=0 2024-05-21T22:01:05.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.387+0000 7f5250a9eb80 1 client.16374 shutdown 2024-05-21T22:01:05.392 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.387+0000 7f5250a9eb80 1 -- 172.21.15.151:0/1096616973 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a575fa7880 msgr2=0x55a575fc40f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:05.392 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.387+0000 7f5250a9eb80 1 --2- 172.21.15.151:0/1096616973 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a575fa7880 0x55a575fc40f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f52300a9950 tx=0x7f52300e89b0 comp rx=0 tx=0).stop 2024-05-21T22:01:05.392 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.387+0000 7f5250a9eb80 1 -- 172.21.15.151:0/1096616973 shutdown_connections 2024-05-21T22:01:05.392 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.387+0000 7f5250a9eb80 1 --2- 172.21.15.151:0/1096616973 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55a57602bdd0 0x55a57602e1c0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:05.392 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.387+0000 7f5250a9eb80 1 --2- 172.21.15.151:0/1096616973 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a575fad7d0 0x55a57602a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:05.392 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.387+0000 7f5250a9eb80 1 --2- 172.21.15.151:0/1096616973 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a575fa7880 0x55a575fc40f0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:05.392 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.387+0000 7f5250a9eb80 1 --2- 172.21.15.151:0/1096616973 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a575faceb0 0x55a575fc4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:05.392 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.387+0000 7f5250a9eb80 1 -- 172.21.15.151:0/1096616973 >> 172.21.15.151:0/1096616973 conn(0x55a575ed5be0 msgr2=0x55a575fb0840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:01:05.392 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.387+0000 7f5229600700 0 client.16374 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T22:01:05.392 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.388+0000 7f5250a9eb80 1 -- 172.21.15.151:0/1096616973 shutdown_connections 2024-05-21T22:01:05.392 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.388+0000 7f5250a9eb80 1 -- 172.21.15.151:0/1096616973 wait complete. 2024-05-21T22:01:05.392 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.388+0000 7f5250a9eb80 20 client.16374 trim_cache size 0 max 16384 2024-05-21T22:01:05.393 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-05-21T22:01:05.393 INFO:teuthology.orchestra.run.smithi151.stdout: "status": "no active scrubs running", 2024-05-21T22:01:05.393 INFO:teuthology.orchestra.run.smithi151.stdout: "scrubs": {} 2024-05-21T22:01:05.393 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-05-21T22:01:05.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.388+0000 7f5246200700 1 -- 172.21.15.151:0/1977119477 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f52240b29f0 msgr2=0x7f52240b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:05.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.388+0000 7f5246200700 1 --2- 172.21.15.151:0/1977119477 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f52240b29f0 0x7f52240b4eb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f52380127f0 tx=0x7f523800b040 comp rx=0 tx=0).stop 2024-05-21T22:01:05.394 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.388+0000 7f5246200700 1 -- 172.21.15.151:0/1977119477 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5240075770 msgr2=0x7f524019bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:05.394 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.388+0000 7f5246200700 1 --2- 172.21.15.151:0/1977119477 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5240075770 0x7f524019bec0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f523000e9c0 tx=0x7f523000ecd0 comp rx=0 tx=0).stop 2024-05-21T22:01:05.394 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.389+0000 7f5246200700 1 -- 172.21.15.151:0/1977119477 shutdown_connections 2024-05-21T22:01:05.394 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.389+0000 7f5246200700 1 --2- 172.21.15.151:0/1977119477 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f52240b29f0 0x7f52240b4eb0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:05.394 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.389+0000 7f5246200700 1 --2- 172.21.15.151:0/1977119477 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5240075770 0x7f524019bec0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:05.394 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.389+0000 7f5246200700 1 --2- 172.21.15.151:0/1977119477 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f524010ff40 0x7f52401a0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:05.394 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.389+0000 7f5246200700 1 --2- 172.21.15.151:0/1977119477 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5240076090 0x7f524019c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:05.394 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.389+0000 7f5246200700 1 -- 172.21.15.151:0/1977119477 >> 172.21.15.151:0/1977119477 conn(0x7f524007a6a0 msgr2=0x7f52401007f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:01:05.394 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.389+0000 7f5246200700 1 -- 172.21.15.151:0/1977119477 shutdown_connections 2024-05-21T22:01:05.394 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:05.389+0000 7f5246200700 1 -- 172.21.15.151:0/1977119477 wait complete. 2024-05-21T22:01:05.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:01:05 smithi184 ceph-mon[36942]: pgmap v167: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-05-21T22:01:05.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:01:05 smithi164 ceph-mon[36269]: pgmap v167: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-05-21T22:01:05.926 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:01:05 smithi151 ceph-mon[31623]: pgmap v167: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-05-21T22:01:06.470 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-21T22:01:06.470 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-21T22:01:06.924 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:01:07.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:01:07 smithi151 ceph-mon[31623]: pgmap v168: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-05-21T22:01:07.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:01:07 smithi184 ceph-mon[36942]: pgmap v168: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-05-21T22:01:07.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:01:07 smithi164 ceph-mon[36269]: pgmap v168: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-05-21T22:01:09.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:09.219+0000 7faed3200700 1 -- 172.21.15.151:0/766042785 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faed410fd00 msgr2=0x7faed4112130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:09.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:09.219+0000 7faed3200700 1 --2- 172.21.15.151:0/766042785 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faed410fd00 0x7faed4112130 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7faecc00b0a0 tx=0x7faecc00b3b0 comp rx=0 tx=0).stop 2024-05-21T22:01:09.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:09.219+0000 7faed3200700 1 -- 172.21.15.151:0/766042785 shutdown_connections 2024-05-21T22:01:09.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:09.219+0000 7faed3200700 1 --2- 172.21.15.151:0/766042785 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faed4104400 0x7faed406bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:09.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:09.219+0000 7faed3200700 1 --2- 172.21.15.151:0/766042785 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faed410fd00 0x7faed4112130 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:09.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:09.219+0000 7faed3200700 1 --2- 172.21.15.151:0/766042785 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faed406bff0 0x7faed406c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:09.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:09.219+0000 7faed3200700 1 -- 172.21.15.151:0/766042785 >> 172.21.15.151:0/766042785 conn(0x7faed4079280 msgr2=0x7faed4079690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:01:09.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:09.219+0000 7faed3200700 1 -- 172.21.15.151:0/766042785 shutdown_connections 2024-05-21T22:01:09.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:09.219+0000 7faed3200700 1 -- 172.21.15.151:0/766042785 wait complete. 2024-05-21T22:01:09.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:09.220+0000 7faed3200700 1 Processor -- start 2024-05-21T22:01:09.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:09.221+0000 7faed3200700 1 -- start start 2024-05-21T22:01:09.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:09.221+0000 7faed3200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faed406bff0 0x7faed41a1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:09.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:09.221+0000 7faed3200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faed4104400 0x7faed41a1da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:09.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:09.221+0000 7faed3200700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faed410fd00 0x7faed419ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:09.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:09.221+0000 7faed3200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7faed4115840 con 0x7faed406bff0 2024-05-21T22:01:09.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:09.221+0000 7faed3200700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7faed41156c0 con 0x7faed4104400 2024-05-21T22:01:09.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:09.221+0000 7faed3200700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7faed41159c0 con 0x7faed410fd00 2024-05-21T22:01:09.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:09.221+0000 7faed1e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faed406bff0 0x7faed41a1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:09.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:09.221+0000 7faed1e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faed406bff0 0x7faed41a1860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:50446/0 (socket says 172.21.15.151:50446) 2024-05-21T22:01:09.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:09.221+0000 7faed1e00700 1 -- 172.21.15.151:0/1758901523 learned_addr learned my addr 172.21.15.151:0/1758901523 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:01:09.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:09.222+0000 7faed1400700 1 --2- 172.21.15.151:0/1758901523 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faed4104400 0x7faed41a1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:09.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:09.222+0000 7faed2800700 1 --2- 172.21.15.151:0/1758901523 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faed410fd00 0x7faed419ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:09.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:09.222+0000 7faed1e00700 1 -- 172.21.15.151:0/1758901523 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faed410fd00 msgr2=0x7faed419ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:09.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:09.222+0000 7faed1e00700 1 --2- 172.21.15.151:0/1758901523 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faed410fd00 0x7faed419ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:09.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:09.222+0000 7faed1e00700 1 -- 172.21.15.151:0/1758901523 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faed4104400 msgr2=0x7faed41a1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:09.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:09.222+0000 7faed1e00700 1 --2- 172.21.15.151:0/1758901523 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faed4104400 0x7faed41a1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:09.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:09.222+0000 7faed1e00700 1 -- 172.21.15.151:0/1758901523 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faed419c180 con 0x7faed406bff0 2024-05-21T22:01:09.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:09.222+0000 7faed1e00700 1 --2- 172.21.15.151:0/1758901523 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faed406bff0 0x7faed41a1860 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7faec400e9a0 tx=0x7faec400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:01:09.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:09.223+0000 7faecaa00700 1 -- 172.21.15.151:0/1758901523 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec400ca10 con 0x7faed406bff0 2024-05-21T22:01:09.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:09.223+0000 7faed3200700 1 -- 172.21.15.151:0/1758901523 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faed419c470 con 0x7faed406bff0 2024-05-21T22:01:09.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:09.223+0000 7faecaa00700 1 -- 172.21.15.151:0/1758901523 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faec400cbb0 con 0x7faed406bff0 2024-05-21T22:01:09.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:09.223+0000 7faecaa00700 1 -- 172.21.15.151:0/1758901523 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec4018680 con 0x7faed406bff0 2024-05-21T22:01:09.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:09.223+0000 7faed3200700 1 -- 172.21.15.151:0/1758901523 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faed41a8730 con 0x7faed406bff0 2024-05-21T22:01:09.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:09.224+0000 7faed3200700 1 -- 172.21.15.151:0/1758901523 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faed406a230 con 0x7faed406bff0 2024-05-21T22:01:09.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:09.227+0000 7faecaa00700 1 -- 172.21.15.151:0/1758901523 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7faec4010430 con 0x7faed406bff0 2024-05-21T22:01:09.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:09.228+0000 7faecaa00700 1 --2- 172.21.15.151:0/1758901523 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7faeb80b29f0 0x7faeb80b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:09.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:09.229+0000 7faecaa00700 1 -- 172.21.15.151:0/1758901523 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7faec40e2b20 con 0x7faed406bff0 2024-05-21T22:01:09.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:09.229+0000 7faed1400700 1 --2- 172.21.15.151:0/1758901523 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7faeb80b29f0 0x7faeb80b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:09.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:09.230+0000 7faed1400700 1 --2- 172.21.15.151:0/1758901523 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7faeb80b29f0 0x7faeb80b4eb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7faed4079a10 tx=0x7faec0009260 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:01:09.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:09.231+0000 7faecaa00700 1 -- 172.21.15.151:0/1758901523 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7faec40aa4b0 con 0x7faed406bff0 2024-05-21T22:01:09.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:09.511+0000 7faed3200700 1 -- 172.21.15.151:0/1758901523 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faed419cf50 con 0x7faed406bff0 2024-05-21T22:01:09.514 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:09.512+0000 7faecaa00700 1 -- 172.21.15.151:0/1758901523 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11958 (secure 0 0 0) 0x7faec40a9db0 con 0x7faed406bff0 2024-05-21T22:01:09.514 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T22:01:09.514 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14358,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/158010529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":158010529},{"type":"v1","addr":"172.21.15.151:6835","nonce":158010529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6839/2069058115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":2069058115},{"type":"v1","addr":"172.21.15.151:6839","nonce":2069058115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T21:51:19.552069+0000","modified":"2024-05-21T21:55:34.643755+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24530,"mds_1":24493,"mds_2":24491,"mds_3":24478,"mds_4":24473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14403":{"gid":14403,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6841/1749723794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6840","nonce":1749723794},{"type":"v1","addr":"172.21.15.151:6841","nonce":1749723794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/2026266114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":2026266114},{"type":"v1","addr":"172.21.15.184:6833","nonce":2026266114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/2467900844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":2467900844},{"type":"v1","addr":"172.21.15.164:6833","nonce":2467900844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3367640960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3367640960},{"type":"v1","addr":"172.21.15.151:6837","nonce":3367640960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/3264142621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3264142621},{"type":"v1","addr":"172.21.15.184:6835","nonce":3264142621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":17,"addr":"172.21.15.164:6835/2340428470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2340428470},{"type":"v1","addr":"172.21.15.164:6835","nonce":2340428470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":16,"addr":"172.21.15.184:6837/2988194888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":2988194888},{"type":"v1","addr":"172.21.15.184:6837","nonce":2988194888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.164:6837/1554609572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1554609572},{"type":"v1","addr":"172.21.15.164:6837","nonce":1554609572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.184:6839/3059935143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":3059935143},{"type":"v1","addr":"172.21.15.184:6839","nonce":3059935143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.164:6839/2646401247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":2646401247},{"type":"v1","addr":"172.21.15.164:6839","nonce":2646401247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24530,"qdb_cluster":[24530,24493,24491,24473,24478]},"id":1}]} 2024-05-21T22:01:09.517 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:09.516+0000 7faed3200700 1 -- 172.21.15.151:0/1758901523 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7faeb80b29f0 msgr2=0x7faeb80b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:09.517 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:09.516+0000 7faed3200700 1 --2- 172.21.15.151:0/1758901523 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7faeb80b29f0 0x7faeb80b4eb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7faed4079a10 tx=0x7faec0009260 comp rx=0 tx=0).stop 2024-05-21T22:01:09.517 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:09.516+0000 7faed3200700 1 -- 172.21.15.151:0/1758901523 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faed406bff0 msgr2=0x7faed41a1860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:09.517 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:09.516+0000 7faed3200700 1 --2- 172.21.15.151:0/1758901523 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faed406bff0 0x7faed41a1860 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7faec400e9a0 tx=0x7faec400ecb0 comp rx=0 tx=0).stop 2024-05-21T22:01:09.518 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:09.516+0000 7faed3200700 1 -- 172.21.15.151:0/1758901523 shutdown_connections 2024-05-21T22:01:09.518 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:09.516+0000 7faed3200700 1 --2- 172.21.15.151:0/1758901523 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7faeb80b29f0 0x7faeb80b4eb0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:09.518 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:09.516+0000 7faed3200700 1 --2- 172.21.15.151:0/1758901523 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faed410fd00 0x7faed419ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:09.518 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:09.516+0000 7faed3200700 1 --2- 172.21.15.151:0/1758901523 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faed406bff0 0x7faed41a1860 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:09.518 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:09.516+0000 7faed3200700 1 --2- 172.21.15.151:0/1758901523 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faed4104400 0x7faed41a1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:09.518 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:09.516+0000 7faed3200700 1 -- 172.21.15.151:0/1758901523 >> 172.21.15.151:0/1758901523 conn(0x7faed4079280 msgr2=0x7faed41015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:01:09.518 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:09.516+0000 7faed3200700 1 -- 172.21.15.151:0/1758901523 shutdown_connections 2024-05-21T22:01:09.518 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:09.516+0000 7faed3200700 1 -- 172.21.15.151:0/1758901523 wait complete. 2024-05-21T22:01:09.519 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 29 2024-05-21T22:01:09.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:01:09 smithi164 ceph-mon[36269]: pgmap v169: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-05-21T22:01:09.900 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:01:09 smithi151 ceph-mon[31623]: pgmap v169: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-05-21T22:01:09.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:01:09 smithi184 ceph-mon[36942]: pgmap v169: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-05-21T22:01:10.393 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-21T22:01:10.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:01:10 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/1758901523' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T22:01:10.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:01:10 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/1758901523' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T22:01:10.829 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:01:10.985 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:01:10 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/1758901523' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T22:01:11.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:01:11 smithi151 ceph-mon[31623]: pgmap v170: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-05-21T22:01:11.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:01:11 smithi164 ceph-mon[36269]: pgmap v170: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-05-21T22:01:11.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:01:11 smithi184 ceph-mon[36942]: pgmap v170: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-05-21T22:01:12.999 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:12.997+0000 7f020d800700 1 -- 172.21.15.151:0/1072891286 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f020810b760 msgr2=0x7f020810bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:12.999 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:12.997+0000 7f020d800700 1 --2- 172.21.15.151:0/1072891286 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f020810b760 0x7f020810bb40 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f01f8009990 tx=0x7f01f8009ca0 comp rx=0 tx=0).stop 2024-05-21T22:01:12.999 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:12.997+0000 7f020d800700 1 -- 172.21.15.151:0/1072891286 shutdown_connections 2024-05-21T22:01:12.999 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:12.997+0000 7f020d800700 1 --2- 172.21.15.151:0/1072891286 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f020810b760 0x7f020810bb40 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:12.999 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:12.997+0000 7f020d800700 1 --2- 172.21.15.151:0/1072891286 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0208106120 0x7f0208112630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:12.999 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:12.997+0000 7f020d800700 1 --2- 172.21.15.151:0/1072891286 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0208105760 0x7f0208105be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:12.999 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:12.997+0000 7f020d800700 1 -- 172.21.15.151:0/1072891286 >> 172.21.15.151:0/1072891286 conn(0x7f0208101250 msgr2=0x7f0208103670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:01:13.000 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:12.997+0000 7f020d800700 1 -- 172.21.15.151:0/1072891286 shutdown_connections 2024-05-21T22:01:13.000 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:12.997+0000 7f020d800700 1 -- 172.21.15.151:0/1072891286 wait complete. 2024-05-21T22:01:13.000 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:12.999+0000 7f020d800700 1 Processor -- start 2024-05-21T22:01:13.001 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:12.999+0000 7f020d800700 1 -- start start 2024-05-21T22:01:13.001 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:12.999+0000 7f020d800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0208105760 0x7f020819bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:13.001 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:12.999+0000 7f020d800700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0208106120 0x7f020819c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:13.001 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:12.999+0000 7f020d800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f020810b760 0x7f02081a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:13.001 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:12.999+0000 7f020d800700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0208115940 con 0x7f0208105760 2024-05-21T22:01:13.001 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:12.999+0000 7f020d800700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f02081157c0 con 0x7f020810b760 2024-05-21T22:01:13.001 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:12.999+0000 7f020d800700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0208115ac0 con 0x7f0208106120 2024-05-21T22:01:13.001 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.000+0000 7f0206a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0208105760 0x7f020819bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:13.002 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.000+0000 7f0207400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f020810b760 0x7f02081a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:13.002 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.000+0000 7f0206a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0208105760 0x7f020819bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:50466/0 (socket says 172.21.15.151:50466) 2024-05-21T22:01:13.002 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.000+0000 7f0206a00700 1 -- 172.21.15.151:0/2833718685 learned_addr learned my addr 172.21.15.151:0/2833718685 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:01:13.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.000+0000 7f0206000700 1 --2- 172.21.15.151:0/2833718685 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0208106120 0x7f020819c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:13.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.000+0000 7f0207400700 1 -- 172.21.15.151:0/2833718685 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0208106120 msgr2=0x7f020819c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:13.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.000+0000 7f0207400700 1 --2- 172.21.15.151:0/2833718685 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0208106120 0x7f020819c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:13.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.000+0000 7f0207400700 1 -- 172.21.15.151:0/2833718685 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0208105760 msgr2=0x7f020819bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:13.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.000+0000 7f0207400700 1 --2- 172.21.15.151:0/2833718685 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0208105760 0x7f020819bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:13.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.000+0000 7f0207400700 1 -- 172.21.15.151:0/2833718685 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02081a0e00 con 0x7f020810b760 2024-05-21T22:01:13.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.000+0000 7f0207400700 1 --2- 172.21.15.151:0/2833718685 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f020810b760 0x7f02081a06c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f020000efc0 tx=0x7f020000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:01:13.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.001+0000 7f01ff400700 1 -- 172.21.15.151:0/2833718685 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02000092e0 con 0x7f020810b760 2024-05-21T22:01:13.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.001+0000 7f01ff400700 1 -- 172.21.15.151:0/2833718685 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0200004bb0 con 0x7f020810b760 2024-05-21T22:01:13.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.001+0000 7f01ff400700 1 -- 172.21.15.151:0/2833718685 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0200004020 con 0x7f020810b760 2024-05-21T22:01:13.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.001+0000 7f020d800700 1 -- 172.21.15.151:0/2833718685 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02081a1090 con 0x7f020810b760 2024-05-21T22:01:13.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.001+0000 7f020d800700 1 -- 172.21.15.151:0/2833718685 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02081a8970 con 0x7f020810b760 2024-05-21T22:01:13.006 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.004+0000 7f01ff400700 1 -- 172.21.15.151:0/2833718685 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7f020001b030 con 0x7f020810b760 2024-05-21T22:01:13.006 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.005+0000 7f0218181b80 1 Processor -- start 2024-05-21T22:01:13.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.006+0000 7f0218181b80 1 -- start start 2024-05-21T22:01:13.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.006+0000 7f0218181b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55793d9c4610 0x55793d9c49f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:13.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.006+0000 7f0218181b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55793d9c4f30 0x55793d9beb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:13.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.006+0000 7f0218181b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55793d9bf140 0x55793d9bf5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:13.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.006+0000 7f01ff400700 1 --2- 172.21.15.151:0/2833718685 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f01ec0b29f0 0x7f01ec0b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:13.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.006+0000 7f0218181b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55793d91ce40 con 0x55793d9c4610 2024-05-21T22:01:13.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.006+0000 7f0218181b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55793d90c9e0 con 0x55793d9c4f30 2024-05-21T22:01:13.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.006+0000 7f0218181b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55793d910a20 con 0x55793d9bf140 2024-05-21T22:01:13.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.007+0000 7f0206a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55793d9c4f30 0x55793d9beb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:13.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.007+0000 7f0206000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55793d9c4610 0x55793d9c49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:13.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.007+0000 7f0207400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55793d9bf140 0x55793d9bf5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:13.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.007+0000 7f0206a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55793d9c4f30 0x55793d9beb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:60816/0 (socket says 172.21.15.151:60816) 2024-05-21T22:01:13.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.007+0000 7f0206a00700 1 -- 172.21.15.151:0/791361233 learned_addr learned my addr 172.21.15.151:0/791361233 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:01:13.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.007+0000 7f01ff400700 1 -- 172.21.15.151:0/2833718685 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f02000e20d0 con 0x7f020810b760 2024-05-21T22:01:13.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.007+0000 7f0206a00700 1 -- 172.21.15.151:0/791361233 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55793d9bf140 msgr2=0x55793d9bf5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:13.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.007+0000 7f0206a00700 1 --2- 172.21.15.151:0/791361233 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55793d9bf140 0x55793d9bf5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:13.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.007+0000 7f0206000700 1 --2- 172.21.15.151:0/2833718685 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f01ec0b29f0 0x7f01ec0b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:13.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.007+0000 7f0206a00700 1 -- 172.21.15.151:0/791361233 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55793d9c4610 msgr2=0x55793d9c49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:13.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.007+0000 7f0206a00700 1 --2- 172.21.15.151:0/791361233 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55793d9c4610 0x55793d9c49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:13.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.007+0000 7f0206a00700 1 -- 172.21.15.151:0/791361233 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55793d8d5420 con 0x55793d9c4f30 2024-05-21T22:01:13.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.007+0000 7f0206a00700 1 --2- 172.21.15.151:0/791361233 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55793d9c4f30 0x55793d9beb20 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f01f80096d0 tx=0x7f01f8009700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1abc9d823f49376 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:01:13.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.008+0000 7f0206000700 1 --2- 172.21.15.151:0/2833718685 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f01ec0b29f0 0x7f01ec0b4eb0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f01f4005950 tx=0x7f01f4009260 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:01:13.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.008+0000 7f01f3e00700 1 -- 172.21.15.151:0/791361233 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f8027070 con 0x55793d9c4f30 2024-05-21T22:01:13.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.008+0000 7f01f3e00700 1 -- 172.21.15.151:0/791361233 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01f80056d0 con 0x55793d9c4f30 2024-05-21T22:01:13.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.008+0000 7f01f3e00700 1 -- 172.21.15.151:0/791361233 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f8003c70 con 0x55793d9c4f30 2024-05-21T22:01:13.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.009+0000 7f0218181b80 1 -- 172.21.15.151:0/791361233 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55793d9c4f30 msgr2=0x55793d9beb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:13.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.009+0000 7f0218181b80 1 --2- 172.21.15.151:0/791361233 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55793d9c4f30 0x55793d9beb20 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f01f80096d0 tx=0x7f01f8009700 comp rx=0 tx=0).stop 2024-05-21T22:01:13.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.009+0000 7f0218181b80 1 -- 172.21.15.151:0/791361233 shutdown_connections 2024-05-21T22:01:13.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.009+0000 7f0218181b80 1 --2- 172.21.15.151:0/791361233 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55793d9bf140 0x55793d9bf5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:13.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.009+0000 7f0218181b80 1 --2- 172.21.15.151:0/791361233 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55793d9c4610 0x55793d9c49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:13.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.009+0000 7f0218181b80 1 --2- 172.21.15.151:0/791361233 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55793d9c4f30 0x55793d9beb20 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:13.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.009+0000 7f0218181b80 1 -- 172.21.15.151:0/791361233 >> 172.21.15.151:0/791361233 conn(0x55793d8ecbe0 msgr2=0x55793d9b9690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:01:13.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.009+0000 7f0218181b80 1 -- 172.21.15.151:0/791361233 shutdown_connections 2024-05-21T22:01:13.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.009+0000 7f0218181b80 1 -- 172.21.15.151:0/791361233 wait complete. 2024-05-21T22:01:13.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.010+0000 7f0218181b80 1 -- start start 2024-05-21T22:01:13.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.010+0000 7f0218181b80 1 Processor -- start 2024-05-21T22:01:13.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.011+0000 7f0218181b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55793d9bf140 0x55793d9cac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:13.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.011+0000 7f0218181b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55793d9c4610 0x55793d9c8000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:13.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.011+0000 7f0218181b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55793d9c4f30 0x55793d9c8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:13.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.011+0000 7f0218181b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55793d812fa0 con 0x55793d9bf140 2024-05-21T22:01:13.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.011+0000 7f0218181b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55793d8ad760 con 0x55793d9c4610 2024-05-21T22:01:13.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.011+0000 7f0218181b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55793d9cb980 con 0x55793d9c4f30 2024-05-21T22:01:13.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.011+0000 7f0218181b80 10 client.0 _ll_register_callbacks cb 0x55793d8b6c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:01:13.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.011+0000 7f0206000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55793d9c4610 0x55793d9c8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:13.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.011+0000 7f0206a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55793d9bf140 0x55793d9cac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:13.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.011+0000 7f0206000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55793d9c4610 0x55793d9c8000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:60830/0 (socket says 172.21.15.151:60830) 2024-05-21T22:01:13.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.011+0000 7f0206000700 1 -- 172.21.15.151:0/1456033512 learned_addr learned my addr 172.21.15.151:0/1456033512 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:01:13.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.011+0000 7f0207400700 1 --2- 172.21.15.151:0/1456033512 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55793d9c4f30 0x55793d9c8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:13.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.011+0000 7f0206000700 1 -- 172.21.15.151:0/1456033512 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55793d9c4f30 msgr2=0x55793d9c8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:13.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.011+0000 7f0206000700 1 --2- 172.21.15.151:0/1456033512 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55793d9c4f30 0x55793d9c8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:13.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.011+0000 7f0206000700 1 -- 172.21.15.151:0/1456033512 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55793d9bf140 msgr2=0x55793d9cac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:13.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.011+0000 7f0206000700 1 --2- 172.21.15.151:0/1456033512 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55793d9bf140 0x55793d9cac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:13.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.011+0000 7f0206000700 1 -- 172.21.15.151:0/1456033512 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55793d85c8d0 con 0x55793d9c4610 2024-05-21T22:01:13.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.012+0000 7f0206000700 1 --2- 172.21.15.151:0/1456033512 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55793d9c4610 0x55793d9c8000 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f01f4010b80 tx=0x7f01f4004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb749e99bce22886 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:01:13.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.012+0000 7f01f0c00700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-21T22:01:13.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.012+0000 7f01f0c00700 1 -- 172.21.15.151:0/1456033512 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f400e870 con 0x55793d9c4610 2024-05-21T22:01:13.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.012+0000 7f0218181b80 1 -- 172.21.15.151:0/1456033512 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01f80056d0 con 0x55793d9c4610 2024-05-21T22:01:13.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.012+0000 7f01f0c00700 1 -- 172.21.15.151:0/1456033512 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f01f400ea10 con 0x55793d9c4610 2024-05-21T22:01:13.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.012+0000 7f01f0c00700 1 -- 172.21.15.151:0/1456033512 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f01f400ec10 con 0x55793d9c4610 2024-05-21T22:01:13.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.012+0000 7f0218181b80 10 client.26516 fetch_fsmap learned FSMap version 29 2024-05-21T22:01:13.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.012+0000 7f0218181b80 1 -- 172.21.15.151:0/1456033512 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55793d8dd060 con 0x55793d9c4610 2024-05-21T22:01:13.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.013+0000 7f01f0c00700 1 -- 172.21.15.151:0/1456033512 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01f80056d0 con 0x55793d9c4610 2024-05-21T22:01:13.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.013+0000 7f01f0c00700 1 -- 172.21.15.151:0/1456033512 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f01f401d030 con 0x55793d9c4610 2024-05-21T22:01:13.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.013+0000 7f0218181b80 10 client.26516 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:01:13.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.013+0000 7f0218181b80 10 client.26516 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-05-21T22:01:13.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.013+0000 7f0218181b80 20 client.26516 populate_metadata read hostname 'smithi151' 2024-05-21T22:01:13.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.013+0000 7f0218181b80 1 --2- 172.21.15.151:0/1456033512 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x55793d9e0cc0 0x55793d9e30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:13.017 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.013+0000 7f0218181b80 4 client.26516 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T22:01:13.017 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.013+0000 7f0218181b80 1 -- 172.21.15.151:0/1456033512 --> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55793d91ed90 con 0x55793d9e0cc0 2024-05-21T22:01:13.017 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.014+0000 7f0206a00700 1 --2- 172.21.15.151:0/1456033512 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x55793d9e0cc0 0x55793d9e30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:13.017 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.014+0000 7f0206a00700 1 --2- 172.21.15.151:0/1456033512 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x55793d9e0cc0 0x55793d9e30d0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=46a948688cfdb6c9 server_cookie=5380034aa4b33ae in_seq=0 out_seq=0 2024-05-21T22:01:13.017 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.014+0000 7f01f0c00700 10 client.26516 ms_handle_connect on v2:172.21.15.164:6834/2340428470 2024-05-21T22:01:13.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.015+0000 7f01f0c00700 1 -- 172.21.15.151:0/1456033512 <== mds.4 v2:172.21.15.164:6834/2340428470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55793d91ed90 con 0x55793d9e0cc0 2024-05-21T22:01:13.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.015+0000 7f01f0c00700 10 client.26516 handle_command_reply: tid=0 2024-05-21T22:01:13.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.015+0000 7f0218181b80 1 client.26516 shutdown 2024-05-21T22:01:13.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.015+0000 7f0218181b80 1 -- 172.21.15.151:0/1456033512 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55793d9c4610 msgr2=0x55793d9c8000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:13.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.015+0000 7f0218181b80 1 --2- 172.21.15.151:0/1456033512 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55793d9c4610 0x55793d9c8000 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f01f4010b80 tx=0x7f01f4004750 comp rx=0 tx=0).stop 2024-05-21T22:01:13.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.015+0000 7f0218181b80 1 -- 172.21.15.151:0/1456033512 shutdown_connections 2024-05-21T22:01:13.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.015+0000 7f0218181b80 1 --2- 172.21.15.151:0/1456033512 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x55793d9e0cc0 0x55793d9e30d0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:13.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.015+0000 7f0218181b80 1 --2- 172.21.15.151:0/1456033512 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55793d9c4f30 0x55793d9c8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:13.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.015+0000 7f0218181b80 1 --2- 172.21.15.151:0/1456033512 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55793d9bf140 0x55793d9cac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:13.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.015+0000 7f0218181b80 1 --2- 172.21.15.151:0/1456033512 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55793d9c4610 0x55793d9c8000 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:13.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.015+0000 7f01f0c00700 0 client.26516 ms_handle_reset on v2:172.21.15.164:6834/2340428470 2024-05-21T22:01:13.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.015+0000 7f0218181b80 1 -- 172.21.15.151:0/1456033512 >> 172.21.15.151:0/1456033512 conn(0x55793d8ecbe0 msgr2=0x55793d9b8d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:01:13.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.015+0000 7f0218181b80 1 -- 172.21.15.151:0/1456033512 shutdown_connections 2024-05-21T22:01:13.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.016+0000 7f0218181b80 1 -- 172.21.15.151:0/1456033512 wait complete. 2024-05-21T22:01:13.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.016+0000 7f0218181b80 20 client.26516 trim_cache size 0 max 16384 2024-05-21T22:01:13.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.032+0000 7f0218181b80 1 Processor -- start 2024-05-21T22:01:13.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.032+0000 7f0218181b80 1 -- start start 2024-05-21T22:01:13.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.032+0000 7f0218181b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55793d9bf140 0x55793d9beb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:13.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.032+0000 7f0218181b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55793d9c4610 0x55793d9e30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:13.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.032+0000 7f0218181b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55793d9c4f30 0x55793d9b94f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:13.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.032+0000 7f0218181b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f02081abc00 con 0x55793d9c4610 2024-05-21T22:01:13.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.032+0000 7f0218181b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f02081ad740 con 0x55793d9bf140 2024-05-21T22:01:13.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.032+0000 7f0218181b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f02081ac2d0 con 0x55793d9c4f30 2024-05-21T22:01:13.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.033+0000 7f0207400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55793d9c4f30 0x55793d9b94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:13.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.033+0000 7f0206a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55793d9bf140 0x55793d9beb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:13.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.033+0000 7f0207400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55793d9c4f30 0x55793d9b94f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:54364/0 (socket says 172.21.15.151:54364) 2024-05-21T22:01:13.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.033+0000 7f0207400700 1 -- 172.21.15.151:0/967044397 learned_addr learned my addr 172.21.15.151:0/967044397 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:01:13.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.033+0000 7f0206000700 1 --2- 172.21.15.151:0/967044397 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55793d9c4610 0x55793d9e30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:13.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.033+0000 7f0207400700 1 -- 172.21.15.151:0/967044397 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55793d9bf140 msgr2=0x55793d9beb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:13.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.033+0000 7f0207400700 1 --2- 172.21.15.151:0/967044397 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55793d9bf140 0x55793d9beb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:13.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.033+0000 7f0207400700 1 -- 172.21.15.151:0/967044397 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55793d9c4610 msgr2=0x55793d9e30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:13.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.033+0000 7f0207400700 1 --2- 172.21.15.151:0/967044397 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55793d9c4610 0x55793d9e30b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:13.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.033+0000 7f0207400700 1 -- 172.21.15.151:0/967044397 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55793d9c6590 con 0x55793d9c4f30 2024-05-21T22:01:13.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.033+0000 7f0206a00700 1 --2- 172.21.15.151:0/967044397 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55793d9bf140 0x55793d9beb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:01:13.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.034+0000 7f0207400700 1 --2- 172.21.15.151:0/967044397 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55793d9c4f30 0x55793d9b94f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f020000c6f0 tx=0x7f02000b7240 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15e439469d47f9fc server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:01:13.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.034+0000 7f01f3400700 1 -- 172.21.15.151:0/967044397 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f020007f040 con 0x55793d9c4f30 2024-05-21T22:01:13.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.034+0000 7f01f3400700 1 -- 172.21.15.151:0/967044397 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02000a9af0 con 0x55793d9c4f30 2024-05-21T22:01:13.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.034+0000 7f01f3400700 1 -- 172.21.15.151:0/967044397 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02000a9650 con 0x55793d9c4f30 2024-05-21T22:01:13.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.034+0000 7f0218181b80 1 -- 172.21.15.151:0/967044397 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55793d9c4f30 msgr2=0x55793d9b94f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:13.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.034+0000 7f0218181b80 1 --2- 172.21.15.151:0/967044397 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55793d9c4f30 0x55793d9b94f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f020000c6f0 tx=0x7f02000b7240 comp rx=0 tx=0).stop 2024-05-21T22:01:13.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.035+0000 7f0218181b80 1 -- 172.21.15.151:0/967044397 shutdown_connections 2024-05-21T22:01:13.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.035+0000 7f0218181b80 1 --2- 172.21.15.151:0/967044397 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55793d9bf140 0x55793d9beb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:13.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.035+0000 7f0218181b80 1 --2- 172.21.15.151:0/967044397 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55793d9c4f30 0x55793d9b94f0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:13.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.035+0000 7f0218181b80 1 --2- 172.21.15.151:0/967044397 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55793d9c4610 0x55793d9e30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:13.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.035+0000 7f0218181b80 1 -- 172.21.15.151:0/967044397 >> 172.21.15.151:0/967044397 conn(0x55793d8ecbe0 msgr2=0x55793d9c6910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:01:13.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.035+0000 7f0218181b80 1 -- 172.21.15.151:0/967044397 shutdown_connections 2024-05-21T22:01:13.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.035+0000 7f0218181b80 1 -- 172.21.15.151:0/967044397 wait complete. 2024-05-21T22:01:13.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.035+0000 7f0218181b80 1 -- start start 2024-05-21T22:01:13.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.035+0000 7f0218181b80 1 Processor -- start 2024-05-21T22:01:13.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.036+0000 7f0218181b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55793d9bf140 0x55793d9d9680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:13.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.036+0000 7f0218181b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55793d9c4610 0x55793d9d9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:13.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.036+0000 7f0218181b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55793d9c4f30 0x55793d9e7d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:13.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.036+0000 7f0218181b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f02081ad070 con 0x55793d9bf140 2024-05-21T22:01:13.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.036+0000 7f0218181b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f02081ac9a0 con 0x55793d9c4610 2024-05-21T22:01:13.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.036+0000 7f0218181b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f020807bbd0 con 0x55793d9c4f30 2024-05-21T22:01:13.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.036+0000 7f0218181b80 10 client.0 _ll_register_callbacks cb 0x55793d8b6c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:01:13.040 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.036+0000 7f0206a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55793d9bf140 0x55793d9d9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:13.040 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.036+0000 7f0206a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55793d9bf140 0x55793d9d9680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:50510/0 (socket says 172.21.15.151:50510) 2024-05-21T22:01:13.040 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.036+0000 7f0206a00700 1 -- 172.21.15.151:0/1322921670 learned_addr learned my addr 172.21.15.151:0/1322921670 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:01:13.040 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.036+0000 7f0207400700 1 --2- 172.21.15.151:0/1322921670 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55793d9c4f30 0x55793d9e7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:13.040 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.036+0000 7f0206000700 1 --2- 172.21.15.151:0/1322921670 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55793d9c4610 0x55793d9d9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:13.040 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.037+0000 7f0206a00700 1 -- 172.21.15.151:0/1322921670 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55793d9c4f30 msgr2=0x55793d9e7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:13.040 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.037+0000 7f0206a00700 1 --2- 172.21.15.151:0/1322921670 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55793d9c4f30 0x55793d9e7d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:13.040 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.037+0000 7f0206a00700 1 -- 172.21.15.151:0/1322921670 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55793d9c4610 msgr2=0x55793d9d9bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:13.040 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.037+0000 7f0206a00700 1 --2- 172.21.15.151:0/1322921670 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55793d9c4610 0x55793d9d9bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:13.040 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.037+0000 7f0206a00700 1 -- 172.21.15.151:0/1322921670 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55793d9e85a0 con 0x55793d9bf140 2024-05-21T22:01:13.040 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.037+0000 7f0207400700 1 --2- 172.21.15.151:0/1322921670 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55793d9c4f30 0x55793d9e7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:01:13.040 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.037+0000 7f0206a00700 1 --2- 172.21.15.151:0/1322921670 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55793d9bf140 0x55793d9d9680 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f01f8003820 tx=0x7f01f8009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46b0fd7b80e0c74f server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:01:13.040 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.037+0000 7f01f0c00700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-21T22:01:13.042 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.037+0000 7f01f0c00700 1 -- 172.21.15.151:0/1322921670 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f803b060 con 0x55793d9bf140 2024-05-21T22:01:13.042 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.037+0000 7f01f0c00700 1 -- 172.21.15.151:0/1322921670 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f01f8003c20 con 0x55793d9bf140 2024-05-21T22:01:13.042 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.037+0000 7f0218181b80 1 -- 172.21.15.151:0/1322921670 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02000a9af0 con 0x55793d9bf140 2024-05-21T22:01:13.043 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.038+0000 7f01f0c00700 1 -- 172.21.15.151:0/1322921670 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f01f8033070 con 0x55793d9bf140 2024-05-21T22:01:13.043 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.038+0000 7f0218181b80 10 client.16398 fetch_fsmap learned FSMap version 29 2024-05-21T22:01:13.043 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.038+0000 7f0218181b80 1 -- 172.21.15.151:0/1322921670 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55793da4b3b0 con 0x55793d9bf140 2024-05-21T22:01:13.043 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.038+0000 7f01f0c00700 1 -- 172.21.15.151:0/1322921670 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02000a9af0 con 0x55793d9bf140 2024-05-21T22:01:13.043 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.038+0000 7f01f0c00700 1 -- 172.21.15.151:0/1322921670 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f01f8021070 con 0x55793d9bf140 2024-05-21T22:01:13.043 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.038+0000 7f0218181b80 10 client.16398 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:01:13.043 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.038+0000 7f0218181b80 10 client.16398 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-05-21T22:01:13.043 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.038+0000 7f0218181b80 20 client.16398 populate_metadata read hostname 'smithi151' 2024-05-21T22:01:13.043 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.038+0000 7f0218181b80 1 --2- 172.21.15.151:0/1322921670 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x55793da4b640 0x55793da4da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:13.043 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.038+0000 7f0218181b80 4 client.16398 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T22:01:13.043 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.038+0000 7f0218181b80 1 -- 172.21.15.151:0/1322921670 --> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55793d918570 con 0x55793da4b640 2024-05-21T22:01:13.044 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.039+0000 7f0206000700 1 --2- 172.21.15.151:0/1322921670 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x55793da4b640 0x55793da4da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:13.044 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.039+0000 7f0206000700 1 --2- 172.21.15.151:0/1322921670 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x55793da4b640 0x55793da4da30 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=877e529ccecc4d43 server_cookie=5670462e705f7434 in_seq=0 out_seq=0 2024-05-21T22:01:13.044 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.039+0000 7f01f0c00700 10 client.16398 ms_handle_connect on v2:172.21.15.164:6834/2340428470 2024-05-21T22:01:13.044 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.039+0000 7f01f0c00700 1 -- 172.21.15.151:0/1322921670 <== mds.4 v2:172.21.15.164:6834/2340428470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55793d918570 con 0x55793da4b640 2024-05-21T22:01:13.044 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.039+0000 7f01f0c00700 10 client.16398 handle_command_reply: tid=0 2024-05-21T22:01:13.044 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.039+0000 7f0218181b80 1 client.16398 shutdown 2024-05-21T22:01:13.044 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.040+0000 7f0218181b80 1 -- 172.21.15.151:0/1322921670 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55793d9bf140 msgr2=0x55793d9d9680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:13.044 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.040+0000 7f0218181b80 1 --2- 172.21.15.151:0/1322921670 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55793d9bf140 0x55793d9d9680 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f01f8003820 tx=0x7f01f8009310 comp rx=0 tx=0).stop 2024-05-21T22:01:13.044 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.040+0000 7f0218181b80 1 -- 172.21.15.151:0/1322921670 shutdown_connections 2024-05-21T22:01:13.044 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.040+0000 7f0218181b80 1 --2- 172.21.15.151:0/1322921670 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x55793da4b640 0x55793da4da30 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:13.044 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.040+0000 7f0218181b80 1 --2- 172.21.15.151:0/1322921670 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55793d9c4f30 0x55793d9e7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:13.044 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.040+0000 7f0218181b80 1 --2- 172.21.15.151:0/1322921670 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55793d9bf140 0x55793d9d9680 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:13.045 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.040+0000 7f0218181b80 1 --2- 172.21.15.151:0/1322921670 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55793d9c4610 0x55793d9d9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:13.045 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.040+0000 7f0218181b80 1 -- 172.21.15.151:0/1322921670 >> 172.21.15.151:0/1322921670 conn(0x55793d8ecbe0 msgr2=0x55793d9c53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:01:13.045 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.040+0000 7f01f0c00700 0 client.16398 ms_handle_reset on v2:172.21.15.164:6834/2340428470 2024-05-21T22:01:13.045 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.040+0000 7f0218181b80 1 -- 172.21.15.151:0/1322921670 shutdown_connections 2024-05-21T22:01:13.045 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.040+0000 7f0218181b80 1 -- 172.21.15.151:0/1322921670 wait complete. 2024-05-21T22:01:13.045 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.040+0000 7f0218181b80 20 client.16398 trim_cache size 0 max 16384 2024-05-21T22:01:13.045 INFO:teuthology.orchestra.run.smithi151.stdout:[] 2024-05-21T22:01:13.046 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.041+0000 7f020d800700 1 -- 172.21.15.151:0/2833718685 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f01ec0b29f0 msgr2=0x7f01ec0b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:13.046 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.041+0000 7f020d800700 1 --2- 172.21.15.151:0/2833718685 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f01ec0b29f0 0x7f01ec0b4eb0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f01f4005950 tx=0x7f01f4009260 comp rx=0 tx=0).stop 2024-05-21T22:01:13.046 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.041+0000 7f020d800700 1 -- 172.21.15.151:0/2833718685 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f020810b760 msgr2=0x7f02081a06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:13.046 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.041+0000 7f020d800700 1 --2- 172.21.15.151:0/2833718685 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f020810b760 0x7f02081a06c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f020000efc0 tx=0x7f020000c3a0 comp rx=0 tx=0).stop 2024-05-21T22:01:13.046 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.041+0000 7f020d800700 1 -- 172.21.15.151:0/2833718685 shutdown_connections 2024-05-21T22:01:13.046 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.041+0000 7f020d800700 1 --2- 172.21.15.151:0/2833718685 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f01ec0b29f0 0x7f01ec0b4eb0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:13.046 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.041+0000 7f020d800700 1 --2- 172.21.15.151:0/2833718685 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f020810b760 0x7f02081a06c0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:13.046 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.041+0000 7f020d800700 1 --2- 172.21.15.151:0/2833718685 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0208105760 0x7f020819bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:13.047 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.041+0000 7f020d800700 1 --2- 172.21.15.151:0/2833718685 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0208106120 0x7f020819c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:13.047 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.041+0000 7f020d800700 1 -- 172.21.15.151:0/2833718685 >> 172.21.15.151:0/2833718685 conn(0x7f0208101250 msgr2=0x7f0208101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:01:13.047 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.041+0000 7f020d800700 1 -- 172.21.15.151:0/2833718685 shutdown_connections 2024-05-21T22:01:13.047 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:13.041+0000 7f020d800700 1 -- 172.21.15.151:0/2833718685 wait complete. 2024-05-21T22:01:13.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:01:13 smithi164 ceph-mon[36269]: pgmap v171: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-05-21T22:01:13.940 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:01:13 smithi151 ceph-mon[31623]: pgmap v171: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-05-21T22:01:13.985 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:01:13 smithi184 ceph-mon[36942]: pgmap v171: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-05-21T22:01:14.022 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-21T22:01:14.457 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:01:15.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:01:15 smithi151 ceph-mon[31623]: pgmap v172: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2024-05-21T22:01:15.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:01:15 smithi164 ceph-mon[36269]: pgmap v172: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2024-05-21T22:01:15.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:01:15 smithi184 ceph-mon[36942]: pgmap v172: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2024-05-21T22:01:16.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.633+0000 7fd224800700 1 -- 172.21.15.151:0/3918357286 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd21c1041a0 msgr2=0x7fd21c1081c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:16.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.633+0000 7fd224800700 1 --2- 172.21.15.151:0/3918357286 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd21c1041a0 0x7fd21c1081c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fd20c0098f0 tx=0x7fd20c009c00 comp rx=0 tx=0).stop 2024-05-21T22:01:16.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.633+0000 7fd224800700 1 -- 172.21.15.151:0/3918357286 shutdown_connections 2024-05-21T22:01:16.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.633+0000 7fd224800700 1 --2- 172.21.15.151:0/3918357286 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd21c1037f0 0x7fd21c103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:16.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.633+0000 7fd224800700 1 --2- 172.21.15.151:0/3918357286 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd21c1088f0 0x7fd21c110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:16.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.633+0000 7fd224800700 1 --2- 172.21.15.151:0/3918357286 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd21c1041a0 0x7fd21c1081c0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:16.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.633+0000 7fd224800700 1 -- 172.21.15.151:0/3918357286 >> 172.21.15.151:0/3918357286 conn(0x7fd21c07a6a0 msgr2=0x7fd21c07aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:01:16.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.633+0000 7fd224800700 1 -- 172.21.15.151:0/3918357286 shutdown_connections 2024-05-21T22:01:16.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.634+0000 7fd224800700 1 -- 172.21.15.151:0/3918357286 wait complete. 2024-05-21T22:01:16.637 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.635+0000 7fd224800700 1 Processor -- start 2024-05-21T22:01:16.637 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.635+0000 7fd224800700 1 -- start start 2024-05-21T22:01:16.637 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.635+0000 7fd224800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd21c1037f0 0x7fd21c19f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:16.637 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.636+0000 7fd224800700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd21c1041a0 0x7fd21c19fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:16.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.636+0000 7fd224800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd21c1088f0 0x7fd21c199a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:16.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.636+0000 7fd224800700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd21c113720 con 0x7fd21c1088f0 2024-05-21T22:01:16.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.636+0000 7fd224800700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd21c1135a0 con 0x7fd21c1037f0 2024-05-21T22:01:16.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.636+0000 7fd224800700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd21c1138a0 con 0x7fd21c1041a0 2024-05-21T22:01:16.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.636+0000 7fd221e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd21c1037f0 0x7fd21c19f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:16.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.636+0000 7fd222800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd21c1088f0 0x7fd21c199a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:16.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.636+0000 7fd221400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd21c1041a0 0x7fd21c19fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:16.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.636+0000 7fd221e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd21c1037f0 0x7fd21c19f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:57728/0 (socket says 172.21.15.151:57728) 2024-05-21T22:01:16.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.636+0000 7fd222800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd21c1088f0 0x7fd21c199a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:50968/0 (socket says 172.21.15.151:50968) 2024-05-21T22:01:16.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.636+0000 7fd221400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd21c1041a0 0x7fd21c19fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:47874/0 (socket says 172.21.15.151:47874) 2024-05-21T22:01:16.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.636+0000 7fd221e00700 1 -- 172.21.15.151:0/4148485831 learned_addr learned my addr 172.21.15.151:0/4148485831 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:01:16.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.636+0000 7fd221e00700 1 -- 172.21.15.151:0/4148485831 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd21c1041a0 msgr2=0x7fd21c19fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:16.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.636+0000 7fd221e00700 1 --2- 172.21.15.151:0/4148485831 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd21c1041a0 0x7fd21c19fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:16.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.636+0000 7fd221e00700 1 -- 172.21.15.151:0/4148485831 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd21c1088f0 msgr2=0x7fd21c199a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:16.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.636+0000 7fd221e00700 1 --2- 172.21.15.151:0/4148485831 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd21c1088f0 0x7fd21c199a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:16.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.637+0000 7fd221e00700 1 -- 172.21.15.151:0/4148485831 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd21c19a1c0 con 0x7fd21c1037f0 2024-05-21T22:01:16.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.637+0000 7fd221400700 1 --2- 172.21.15.151:0/4148485831 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd21c1041a0 0x7fd21c19fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:01:16.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.637+0000 7fd222800700 1 --2- 172.21.15.151:0/4148485831 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd21c1088f0 0x7fd21c199a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:01:16.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.637+0000 7fd221e00700 1 --2- 172.21.15.151:0/4148485831 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd21c1037f0 0x7fd21c19f8a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd21000e9c0 tx=0x7fd21000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:01:16.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.637+0000 7fd216a00700 1 -- 172.21.15.151:0/4148485831 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd21000cbd0 con 0x7fd21c1037f0 2024-05-21T22:01:16.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.637+0000 7fd224800700 1 -- 172.21.15.151:0/4148485831 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd21c19a4b0 con 0x7fd21c1037f0 2024-05-21T22:01:16.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.637+0000 7fd216a00700 1 -- 172.21.15.151:0/4148485831 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd21000ee60 con 0x7fd21c1037f0 2024-05-21T22:01:16.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.637+0000 7fd216a00700 1 -- 172.21.15.151:0/4148485831 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2100055c0 con 0x7fd21c1037f0 2024-05-21T22:01:16.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.637+0000 7fd224800700 1 -- 172.21.15.151:0/4148485831 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd21c1a6770 con 0x7fd21c1037f0 2024-05-21T22:01:16.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.641+0000 7fd216a00700 1 -- 172.21.15.151:0/4148485831 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7fd210004750 con 0x7fd21c1037f0 2024-05-21T22:01:16.643 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.642+0000 7fd22f23fb80 1 Processor -- start 2024-05-21T22:01:16.643 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.642+0000 7fd22f23fb80 1 -- start start 2024-05-21T22:01:16.644 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.642+0000 7fd22f23fb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5557f1bb3610 0x5557f1bb39f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:16.644 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.642+0000 7fd22f23fb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557f1bb3f30 0x5557f1badb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:16.644 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.642+0000 7fd22f23fb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5557f1bae140 0x5557f1bae5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:16.644 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.642+0000 7fd22f23fb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5557f1b0be40 con 0x5557f1bae140 2024-05-21T22:01:16.644 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.642+0000 7fd22f23fb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5557f1afb9e0 con 0x5557f1bb3f30 2024-05-21T22:01:16.644 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.642+0000 7fd22f23fb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5557f1affa20 con 0x5557f1bb3610 2024-05-21T22:01:16.644 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.643+0000 7fd216a00700 1 --2- 172.21.15.151:0/4148485831 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fd2040b2a70 0x7fd2040b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:16.645 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.643+0000 7fd221e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557f1bb3f30 0x5557f1badb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:16.645 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.643+0000 7fd221400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5557f1bae140 0x5557f1bae5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:16.646 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.643+0000 7fd222800700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5557f1bb3610 0x5557f1bb39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:16.646 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.643+0000 7fd221e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557f1bb3f30 0x5557f1badb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:57742/0 (socket says 172.21.15.151:57742) 2024-05-21T22:01:16.646 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.643+0000 7fd221e00700 1 -- 172.21.15.151:0/1377521998 learned_addr learned my addr 172.21.15.151:0/1377521998 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:01:16.646 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.643+0000 7fd216a00700 1 -- 172.21.15.151:0/4148485831 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fd2100e2870 con 0x7fd21c1037f0 2024-05-21T22:01:16.646 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.643+0000 7fd222800700 1 --2- 172.21.15.151:0/4148485831 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fd2040b2a70 0x7fd2040b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:16.646 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.643+0000 7fd221400700 1 -- 172.21.15.151:0/1377521998 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5557f1bb3610 msgr2=0x5557f1bb39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:16.646 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.643+0000 7fd221400700 1 --2- 172.21.15.151:0/1377521998 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5557f1bb3610 0x5557f1bb39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:16.646 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.643+0000 7fd221400700 1 -- 172.21.15.151:0/1377521998 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557f1bb3f30 msgr2=0x5557f1badb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:16.646 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.644+0000 7fd221400700 1 --2- 172.21.15.151:0/1377521998 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557f1bb3f30 0x5557f1badb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:16.646 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.644+0000 7fd221400700 1 -- 172.21.15.151:0/1377521998 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557f1ac4420 con 0x5557f1bae140 2024-05-21T22:01:16.646 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.644+0000 7fd222800700 1 --2- 172.21.15.151:0/1377521998 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5557f1bb3610 0x5557f1bb39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T22:01:16.646 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.644+0000 7fd221400700 1 --2- 172.21.15.151:0/1377521998 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5557f1bae140 0x5557f1bae5c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fd20c00c010 tx=0x7fd20c00bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dd5511aa82d233 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:01:16.647 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.644+0000 7fd20b400700 1 -- 172.21.15.151:0/1377521998 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd20c00f660 con 0x5557f1bae140 2024-05-21T22:01:16.647 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.644+0000 7fd222800700 1 --2- 172.21.15.151:0/4148485831 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fd2040b2a70 0x7fd2040b4f30 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd218009610 tx=0x7fd21800b040 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:01:16.647 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.644+0000 7fd20b400700 1 -- 172.21.15.151:0/1377521998 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd20c0054f0 con 0x5557f1bae140 2024-05-21T22:01:16.647 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.644+0000 7fd20b400700 1 -- 172.21.15.151:0/1377521998 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd20c004ae0 con 0x5557f1bae140 2024-05-21T22:01:16.648 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.645+0000 7fd22f23fb80 1 -- 172.21.15.151:0/1377521998 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5557f1bae140 msgr2=0x5557f1bae5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:16.648 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.645+0000 7fd22f23fb80 1 --2- 172.21.15.151:0/1377521998 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5557f1bae140 0x5557f1bae5c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fd20c00c010 tx=0x7fd20c00bf60 comp rx=0 tx=0).stop 2024-05-21T22:01:16.648 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.645+0000 7fd22f23fb80 1 -- 172.21.15.151:0/1377521998 shutdown_connections 2024-05-21T22:01:16.648 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.645+0000 7fd22f23fb80 1 --2- 172.21.15.151:0/1377521998 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5557f1bae140 0x5557f1bae5c0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:16.648 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.645+0000 7fd22f23fb80 1 --2- 172.21.15.151:0/1377521998 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5557f1bb3610 0x5557f1bb39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:16.648 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.645+0000 7fd22f23fb80 1 --2- 172.21.15.151:0/1377521998 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557f1bb3f30 0x5557f1badb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:16.648 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.645+0000 7fd22f23fb80 1 -- 172.21.15.151:0/1377521998 >> 172.21.15.151:0/1377521998 conn(0x5557f1adbbe0 msgr2=0x5557f1ba8690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:01:16.648 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.645+0000 7fd22f23fb80 1 -- 172.21.15.151:0/1377521998 shutdown_connections 2024-05-21T22:01:16.649 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.645+0000 7fd22f23fb80 1 -- 172.21.15.151:0/1377521998 wait complete. 2024-05-21T22:01:16.649 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.646+0000 7fd22f23fb80 1 -- start start 2024-05-21T22:01:16.649 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.646+0000 7fd22f23fb80 1 Processor -- start 2024-05-21T22:01:16.649 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.647+0000 7fd22f23fb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5557f1bae140 0x5557f1bb9c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:16.649 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.647+0000 7fd22f23fb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5557f1bb3610 0x5557f1bb6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:16.649 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.647+0000 7fd22f23fb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557f1bb3f30 0x5557f1bb7500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:16.649 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.647+0000 7fd22f23fb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5557f1a01fa0 con 0x5557f1bae140 2024-05-21T22:01:16.649 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.647+0000 7fd22f23fb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5557f1a9c760 con 0x5557f1bb3f30 2024-05-21T22:01:16.649 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.647+0000 7fd22f23fb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5557f1bba980 con 0x5557f1bb3610 2024-05-21T22:01:16.649 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.647+0000 7fd22f23fb80 10 client.0 _ll_register_callbacks cb 0x5557f1aa5c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:01:16.651 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.647+0000 7fd221e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5557f1bae140 0x5557f1bb9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:16.651 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.647+0000 7fd221e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5557f1bae140 0x5557f1bb9c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:50982/0 (socket says 172.21.15.151:50982) 2024-05-21T22:01:16.651 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.647+0000 7fd221e00700 1 -- 172.21.15.151:0/2386589027 learned_addr learned my addr 172.21.15.151:0/2386589027 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:01:16.651 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.647+0000 7fd222800700 1 --2- 172.21.15.151:0/2386589027 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557f1bb3f30 0x5557f1bb7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:16.651 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.647+0000 7fd221400700 1 --2- 172.21.15.151:0/2386589027 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5557f1bb3610 0x5557f1bb6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:16.651 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.647+0000 7fd221e00700 1 -- 172.21.15.151:0/2386589027 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5557f1bb3610 msgr2=0x5557f1bb6fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:16.652 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.647+0000 7fd221e00700 1 --2- 172.21.15.151:0/2386589027 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5557f1bb3610 0x5557f1bb6fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:16.652 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.647+0000 7fd221e00700 1 -- 172.21.15.151:0/2386589027 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557f1bb3f30 msgr2=0x5557f1bb7500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:16.652 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.647+0000 7fd221e00700 1 --2- 172.21.15.151:0/2386589027 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557f1bb3f30 0x5557f1bb7500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:16.652 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.648+0000 7fd221e00700 1 -- 172.21.15.151:0/2386589027 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557f1a4b8d0 con 0x5557f1bae140 2024-05-21T22:01:16.652 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.648+0000 7fd221e00700 1 --2- 172.21.15.151:0/2386589027 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5557f1bae140 0x5557f1bb9c20 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fd210009fd0 tx=0x7fd210064680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b3981308bbbe991 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:01:16.652 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.648+0000 7fd1ffe00700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-21T22:01:16.652 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.648+0000 7fd1ffe00700 1 -- 172.21.15.151:0/2386589027 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2100c9060 con 0x5557f1bae140 2024-05-21T22:01:16.652 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.648+0000 7fd1ffe00700 1 -- 172.21.15.151:0/2386589027 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fd2100a9f70 con 0x5557f1bae140 2024-05-21T22:01:16.652 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.648+0000 7fd22f23fb80 1 -- 172.21.15.151:0/2386589027 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd20c0054f0 con 0x5557f1bae140 2024-05-21T22:01:16.652 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.648+0000 7fd1ffe00700 1 -- 172.21.15.151:0/2386589027 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd2100a5290 con 0x5557f1bae140 2024-05-21T22:01:16.652 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.648+0000 7fd22f23fb80 10 client.16410 fetch_fsmap learned FSMap version 29 2024-05-21T22:01:16.652 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.648+0000 7fd22f23fb80 1 -- 172.21.15.151:0/2386589027 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5557f1acc060 con 0x5557f1bae140 2024-05-21T22:01:16.652 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.649+0000 7fd1ffe00700 1 -- 172.21.15.151:0/2386589027 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd20c0054f0 con 0x5557f1bae140 2024-05-21T22:01:16.652 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.649+0000 7fd1ffe00700 1 -- 172.21.15.151:0/2386589027 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fd21006f060 con 0x5557f1bae140 2024-05-21T22:01:16.653 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.649+0000 7fd22f23fb80 10 client.16410 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:01:16.653 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.649+0000 7fd22f23fb80 10 client.16410 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-21T22:01:16.653 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.649+0000 7fd22f23fb80 20 client.16410 populate_metadata read hostname 'smithi151' 2024-05-21T22:01:16.653 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.649+0000 7fd22f23fb80 1 --2- 172.21.15.151:0/2386589027 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x5557f1bc74d0 0x5557f1bc98c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:16.653 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.649+0000 7fd22f23fb80 4 client.16410 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T22:01:16.653 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.649+0000 7fd22f23fb80 1 -- 172.21.15.151:0/2386589027 --> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5557f1b0dd90 con 0x5557f1bc74d0 2024-05-21T22:01:16.653 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.650+0000 7fd221400700 1 --2- 172.21.15.151:0/2386589027 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x5557f1bc74d0 0x5557f1bc98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:16.653 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.650+0000 7fd221400700 1 --2- 172.21.15.151:0/2386589027 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x5557f1bc74d0 0x5557f1bc98c0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=58ac4f89ce83fd72 server_cookie=43fd6fddd781fadd in_seq=0 out_seq=0 2024-05-21T22:01:16.653 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.650+0000 7fd1ffe00700 10 client.16410 ms_handle_connect on v2:172.21.15.184:6836/2988194888 2024-05-21T22:01:16.654 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.651+0000 7fd1ffe00700 1 -- 172.21.15.151:0/2386589027 <== mds.3 v2:172.21.15.184:6836/2988194888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5557f1b0dd90 con 0x5557f1bc74d0 2024-05-21T22:01:16.654 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.651+0000 7fd1ffe00700 10 client.16410 handle_command_reply: tid=0 2024-05-21T22:01:16.654 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.651+0000 7fd22f23fb80 1 client.16410 shutdown 2024-05-21T22:01:16.654 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.651+0000 7fd22f23fb80 1 -- 172.21.15.151:0/2386589027 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5557f1bae140 msgr2=0x5557f1bb9c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:16.654 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.651+0000 7fd22f23fb80 1 --2- 172.21.15.151:0/2386589027 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5557f1bae140 0x5557f1bb9c20 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fd210009fd0 tx=0x7fd210064680 comp rx=0 tx=0).stop 2024-05-21T22:01:16.654 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.651+0000 7fd22f23fb80 1 -- 172.21.15.151:0/2386589027 shutdown_connections 2024-05-21T22:01:16.655 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.651+0000 7fd22f23fb80 1 --2- 172.21.15.151:0/2386589027 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x5557f1bc74d0 0x5557f1bc98c0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:16.655 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.651+0000 7fd22f23fb80 1 --2- 172.21.15.151:0/2386589027 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557f1bb3f30 0x5557f1bb7500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:16.655 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.651+0000 7fd22f23fb80 1 --2- 172.21.15.151:0/2386589027 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5557f1bae140 0x5557f1bb9c20 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:16.655 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.651+0000 7fd22f23fb80 1 --2- 172.21.15.151:0/2386589027 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5557f1bb3610 0x5557f1bb6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:16.655 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.651+0000 7fd1ffe00700 0 client.16410 ms_handle_reset on v2:172.21.15.184:6836/2988194888 2024-05-21T22:01:16.655 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.651+0000 7fd22f23fb80 1 -- 172.21.15.151:0/2386589027 >> 172.21.15.151:0/2386589027 conn(0x5557f1adbbe0 msgr2=0x5557f1ba7d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:01:16.655 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.651+0000 7fd22f23fb80 1 -- 172.21.15.151:0/2386589027 shutdown_connections 2024-05-21T22:01:16.655 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.652+0000 7fd22f23fb80 1 -- 172.21.15.151:0/2386589027 wait complete. 2024-05-21T22:01:16.655 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.652+0000 7fd22f23fb80 20 client.16410 trim_cache size 0 max 16384 2024-05-21T22:01:16.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.669+0000 7fd22f23fb80 1 Processor -- start 2024-05-21T22:01:16.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.669+0000 7fd22f23fb80 1 -- start start 2024-05-21T22:01:16.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.669+0000 7fd22f23fb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5557f1bae140 0x5557f1badb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:16.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.669+0000 7fd22f23fb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5557f1bb3610 0x5557f1bb5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:16.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.669+0000 7fd22f23fb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557f1bb3f30 0x5557f1bb4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:16.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.669+0000 7fd22f23fb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd21c1abb20 con 0x5557f1bae140 2024-05-21T22:01:16.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.669+0000 7fd22f23fb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd21c1ad660 con 0x5557f1bb3f30 2024-05-21T22:01:16.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.669+0000 7fd22f23fb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd21c1ac1f0 con 0x5557f1bb3610 2024-05-21T22:01:16.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.669+0000 7fd221e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5557f1bae140 0x5557f1badb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:16.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.669+0000 7fd221400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5557f1bb3610 0x5557f1bb5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:16.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.669+0000 7fd221400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5557f1bb3610 0x5557f1bb5f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:47900/0 (socket says 172.21.15.151:47900) 2024-05-21T22:01:16.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.669+0000 7fd221400700 1 -- 172.21.15.151:0/2967408609 learned_addr learned my addr 172.21.15.151:0/2967408609 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:01:16.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.669+0000 7fd222800700 1 --2- 172.21.15.151:0/2967408609 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557f1bb3f30 0x5557f1bb4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:16.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.670+0000 7fd221400700 1 -- 172.21.15.151:0/2967408609 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557f1bb3f30 msgr2=0x5557f1bb4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:16.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.670+0000 7fd221400700 1 --2- 172.21.15.151:0/2967408609 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557f1bb3f30 0x5557f1bb4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:16.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.670+0000 7fd221400700 1 -- 172.21.15.151:0/2967408609 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5557f1bae140 msgr2=0x5557f1badb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:16.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.670+0000 7fd221400700 1 --2- 172.21.15.151:0/2967408609 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5557f1bae140 0x5557f1badb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:16.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.670+0000 7fd221400700 1 -- 172.21.15.151:0/2967408609 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557f1bb3d30 con 0x5557f1bb3610 2024-05-21T22:01:16.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.670+0000 7fd221400700 1 --2- 172.21.15.151:0/2967408609 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5557f1bb3610 0x5557f1bb5f20 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fd20c00c010 tx=0x7fd20c0056e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=430dce68729e2d70 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:01:16.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.670+0000 7fd20aa00700 1 -- 172.21.15.151:0/2967408609 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd20c00f8a0 con 0x5557f1bb3610 2024-05-21T22:01:16.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.670+0000 7fd20aa00700 1 -- 172.21.15.151:0/2967408609 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd20c0058e0 con 0x5557f1bb3610 2024-05-21T22:01:16.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.670+0000 7fd20aa00700 1 -- 172.21.15.151:0/2967408609 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd20c003e00 con 0x5557f1bb3610 2024-05-21T22:01:16.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.671+0000 7fd22f23fb80 1 -- 172.21.15.151:0/2967408609 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5557f1bb3610 msgr2=0x5557f1bb5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:16.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.671+0000 7fd22f23fb80 1 --2- 172.21.15.151:0/2967408609 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5557f1bb3610 0x5557f1bb5f20 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fd20c00c010 tx=0x7fd20c0056e0 comp rx=0 tx=0).stop 2024-05-21T22:01:16.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.671+0000 7fd22f23fb80 1 -- 172.21.15.151:0/2967408609 shutdown_connections 2024-05-21T22:01:16.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.671+0000 7fd22f23fb80 1 --2- 172.21.15.151:0/2967408609 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557f1bb3f30 0x5557f1bb4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:16.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.671+0000 7fd22f23fb80 1 --2- 172.21.15.151:0/2967408609 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5557f1bae140 0x5557f1badb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:16.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.671+0000 7fd22f23fb80 1 --2- 172.21.15.151:0/2967408609 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5557f1bb3610 0x5557f1bb5f20 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:16.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.671+0000 7fd22f23fb80 1 -- 172.21.15.151:0/2967408609 >> 172.21.15.151:0/2967408609 conn(0x5557f1adbbe0 msgr2=0x5557f1bb4310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:01:16.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.671+0000 7fd22f23fb80 1 -- 172.21.15.151:0/2967408609 shutdown_connections 2024-05-21T22:01:16.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.671+0000 7fd22f23fb80 1 -- 172.21.15.151:0/2967408609 wait complete. 2024-05-21T22:01:16.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.672+0000 7fd22f23fb80 1 -- start start 2024-05-21T22:01:16.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.672+0000 7fd22f23fb80 1 Processor -- start 2024-05-21T22:01:16.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.672+0000 7fd22f23fb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5557f1bae140 0x5557f1bbfc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:16.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.672+0000 7fd22f23fb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5557f1bb3610 0x5557f1bc0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:16.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.672+0000 7fd22f23fb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557f1bb3f30 0x5557f1bbcbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:16.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.672+0000 7fd22f23fb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd21c1acf90 con 0x5557f1bb3610 2024-05-21T22:01:16.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.673+0000 7fd22f23fb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd21c1ac8c0 con 0x5557f1bb3f30 2024-05-21T22:01:16.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.673+0000 7fd22f23fb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd21c1a9ac0 con 0x5557f1bae140 2024-05-21T22:01:16.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.673+0000 7fd22f23fb80 10 client.0 _ll_register_callbacks cb 0x5557f1aa5c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:01:16.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.673+0000 7fd221400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5557f1bb3610 0x5557f1bc0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:16.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.673+0000 7fd221e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5557f1bae140 0x5557f1bbfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:16.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.673+0000 7fd221400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5557f1bb3610 0x5557f1bc0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:50986/0 (socket says 172.21.15.151:50986) 2024-05-21T22:01:16.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.673+0000 7fd221400700 1 -- 172.21.15.151:0/2239648927 learned_addr learned my addr 172.21.15.151:0/2239648927 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:01:16.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.673+0000 7fd222800700 1 --2- 172.21.15.151:0/2239648927 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557f1bb3f30 0x5557f1bbcbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:16.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.673+0000 7fd221e00700 1 -- 172.21.15.151:0/2239648927 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557f1bb3f30 msgr2=0x5557f1bbcbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:16.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.673+0000 7fd221e00700 1 --2- 172.21.15.151:0/2239648927 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557f1bb3f30 0x5557f1bbcbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:16.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.673+0000 7fd221e00700 1 -- 172.21.15.151:0/2239648927 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5557f1bb3610 msgr2=0x5557f1bc0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:16.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.673+0000 7fd221e00700 1 --2- 172.21.15.151:0/2239648927 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5557f1bb3610 0x5557f1bc0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:16.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.673+0000 7fd221e00700 1 -- 172.21.15.151:0/2239648927 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557f1bcfb70 con 0x5557f1bae140 2024-05-21T22:01:16.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.674+0000 7fd221e00700 1 --2- 172.21.15.151:0/2239648927 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5557f1bae140 0x5557f1bbfc10 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd2100a6d70 tx=0x7fd2100e04d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9fc8e45ffd1be33 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:01:16.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.674+0000 7fd1ffe00700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-21T22:01:16.679 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.674+0000 7fd1ffe00700 1 -- 172.21.15.151:0/2239648927 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2100b2730 con 0x5557f1bae140 2024-05-21T22:01:16.679 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.674+0000 7fd22f23fb80 1 -- 172.21.15.151:0/2239648927 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd20c0058e0 con 0x5557f1bae140 2024-05-21T22:01:16.679 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.674+0000 7fd1ffe00700 1 -- 172.21.15.151:0/2239648927 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fd2100a60f0 con 0x5557f1bae140 2024-05-21T22:01:16.679 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.674+0000 7fd1ffe00700 1 -- 172.21.15.151:0/2239648927 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd2100a9610 con 0x5557f1bae140 2024-05-21T22:01:16.679 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.674+0000 7fd22f23fb80 10 client.26554 fetch_fsmap learned FSMap version 29 2024-05-21T22:01:16.679 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.674+0000 7fd22f23fb80 1 -- 172.21.15.151:0/2239648927 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5557f1bcfb70 con 0x5557f1bae140 2024-05-21T22:01:16.679 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.674+0000 7fd1ffe00700 1 -- 172.21.15.151:0/2239648927 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd20c0058e0 con 0x5557f1bae140 2024-05-21T22:01:16.679 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.675+0000 7fd1ffe00700 1 -- 172.21.15.151:0/2239648927 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fd210004bd0 con 0x5557f1bae140 2024-05-21T22:01:16.679 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.675+0000 7fd22f23fb80 10 client.26554 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:01:16.679 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.675+0000 7fd22f23fb80 10 client.26554 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-21T22:01:16.679 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.675+0000 7fd22f23fb80 20 client.26554 populate_metadata read hostname 'smithi151' 2024-05-21T22:01:16.679 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.675+0000 7fd22f23fb80 1 --2- 172.21.15.151:0/2239648927 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x5557f1c3af70 0x5557f1c3d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:16.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.675+0000 7fd22f23fb80 4 client.26554 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T22:01:16.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.675+0000 7fd22f23fb80 1 -- 172.21.15.151:0/2239648927 --> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5557f1b07570 con 0x5557f1c3af70 2024-05-21T22:01:16.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.675+0000 7fd221400700 1 --2- 172.21.15.151:0/2239648927 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x5557f1c3af70 0x5557f1c3d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:16.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.675+0000 7fd221400700 1 --2- 172.21.15.151:0/2239648927 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x5557f1c3af70 0x5557f1c3d360 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=62f57fb73b533da6 server_cookie=e522227a3386226f in_seq=0 out_seq=0 2024-05-21T22:01:16.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.675+0000 7fd1ffe00700 10 client.26554 ms_handle_connect on v2:172.21.15.184:6836/2988194888 2024-05-21T22:01:16.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.676+0000 7fd1ffe00700 1 -- 172.21.15.151:0/2239648927 <== mds.3 v2:172.21.15.184:6836/2988194888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5557f1b07570 con 0x5557f1c3af70 2024-05-21T22:01:16.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.676+0000 7fd1ffe00700 10 client.26554 handle_command_reply: tid=0 2024-05-21T22:01:16.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.676+0000 7fd22f23fb80 1 client.26554 shutdown 2024-05-21T22:01:16.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.676+0000 7fd22f23fb80 1 -- 172.21.15.151:0/2239648927 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5557f1bae140 msgr2=0x5557f1bbfc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:16.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.676+0000 7fd22f23fb80 1 --2- 172.21.15.151:0/2239648927 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5557f1bae140 0x5557f1bbfc10 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd2100a6d70 tx=0x7fd2100e04d0 comp rx=0 tx=0).stop 2024-05-21T22:01:16.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.676+0000 7fd22f23fb80 1 -- 172.21.15.151:0/2239648927 shutdown_connections 2024-05-21T22:01:16.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.676+0000 7fd22f23fb80 1 --2- 172.21.15.151:0/2239648927 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x5557f1c3af70 0x5557f1c3d360 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:16.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.676+0000 7fd22f23fb80 1 --2- 172.21.15.151:0/2239648927 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5557f1bb3f30 0x5557f1bbcbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:16.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.676+0000 7fd22f23fb80 1 --2- 172.21.15.151:0/2239648927 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5557f1bae140 0x5557f1bbfc10 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:16.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.676+0000 7fd1ffe00700 0 client.26554 ms_handle_reset on v2:172.21.15.184:6836/2988194888 2024-05-21T22:01:16.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.676+0000 7fd22f23fb80 1 --2- 172.21.15.151:0/2239648927 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5557f1bb3610 0x5557f1bc0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:16.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.676+0000 7fd22f23fb80 1 -- 172.21.15.151:0/2239648927 >> 172.21.15.151:0/2239648927 conn(0x5557f1adbbe0 msgr2=0x5557f1bb2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:01:16.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.676+0000 7fd22f23fb80 1 -- 172.21.15.151:0/2239648927 shutdown_connections 2024-05-21T22:01:16.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.676+0000 7fd22f23fb80 1 -- 172.21.15.151:0/2239648927 wait complete. 2024-05-21T22:01:16.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.676+0000 7fd22f23fb80 20 client.26554 trim_cache size 0 max 16384 2024-05-21T22:01:16.681 INFO:teuthology.orchestra.run.smithi151.stdout:[] 2024-05-21T22:01:16.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.677+0000 7fd224800700 1 -- 172.21.15.151:0/4148485831 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fd2040b2a70 msgr2=0x7fd2040b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:16.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.677+0000 7fd224800700 1 --2- 172.21.15.151:0/4148485831 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fd2040b2a70 0x7fd2040b4f30 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd218009610 tx=0x7fd21800b040 comp rx=0 tx=0).stop 2024-05-21T22:01:16.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.677+0000 7fd224800700 1 -- 172.21.15.151:0/4148485831 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd21c1037f0 msgr2=0x7fd21c19f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:16.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.677+0000 7fd224800700 1 --2- 172.21.15.151:0/4148485831 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd21c1037f0 0x7fd21c19f8a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd21000e9c0 tx=0x7fd21000ecd0 comp rx=0 tx=0).stop 2024-05-21T22:01:16.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.677+0000 7fd224800700 1 -- 172.21.15.151:0/4148485831 shutdown_connections 2024-05-21T22:01:16.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.677+0000 7fd224800700 1 --2- 172.21.15.151:0/4148485831 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fd2040b2a70 0x7fd2040b4f30 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:16.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.677+0000 7fd224800700 1 --2- 172.21.15.151:0/4148485831 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd21c1037f0 0x7fd21c19f8a0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:16.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.677+0000 7fd224800700 1 --2- 172.21.15.151:0/4148485831 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd21c1088f0 0x7fd21c199a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:16.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.677+0000 7fd224800700 1 --2- 172.21.15.151:0/4148485831 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd21c1041a0 0x7fd21c19fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:16.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.677+0000 7fd224800700 1 -- 172.21.15.151:0/4148485831 >> 172.21.15.151:0/4148485831 conn(0x7fd21c07a6a0 msgr2=0x7fd21c101a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:01:16.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.678+0000 7fd224800700 1 -- 172.21.15.151:0/4148485831 shutdown_connections 2024-05-21T22:01:16.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:16.678+0000 7fd224800700 1 -- 172.21.15.151:0/4148485831 wait complete. 2024-05-21T22:01:17.690 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-21T22:01:17.770 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:01:17 smithi151 ceph-mon[31623]: pgmap v173: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2024-05-21T22:01:17.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:01:17 smithi164 ceph-mon[36269]: pgmap v173: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2024-05-21T22:01:17.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:01:17 smithi184 ceph-mon[36942]: pgmap v173: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2024-05-21T22:01:18.144 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:01:19.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:01:19 smithi164 ceph-mon[36269]: pgmap v174: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 170 B/s wr, 12 op/s 2024-05-21T22:01:19.842 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:01:19 smithi151 ceph-mon[31623]: pgmap v174: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 170 B/s wr, 12 op/s 2024-05-21T22:01:19.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:01:19 smithi184 ceph-mon[36942]: pgmap v174: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 170 B/s wr, 12 op/s 2024-05-21T22:01:20.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.326+0000 7ffa1a000700 1 -- 172.21.15.151:0/2163144975 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa14075770 msgr2=0x7ffa14075b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:20.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.326+0000 7ffa1a000700 1 --2- 172.21.15.151:0/2163144975 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa14075770 0x7ffa14075b50 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7ffa0400b0a0 tx=0x7ffa0400b3b0 comp rx=0 tx=0).stop 2024-05-21T22:01:20.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.326+0000 7ffa1a000700 1 -- 172.21.15.151:0/2163144975 shutdown_connections 2024-05-21T22:01:20.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.326+0000 7ffa1a000700 1 --2- 172.21.15.151:0/2163144975 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffa14114710 0x7ffa14116b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:20.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.326+0000 7ffa1a000700 1 --2- 172.21.15.151:0/2163144975 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa14075770 0x7ffa14075b50 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:20.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.326+0000 7ffa1a000700 1 --2- 172.21.15.151:0/2163144975 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffa14076090 0x7ffa141141d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:20.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.326+0000 7ffa1a000700 1 -- 172.21.15.151:0/2163144975 >> 172.21.15.151:0/2163144975 conn(0x7ffa1407a6a0 msgr2=0x7ffa1407aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:01:20.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.326+0000 7ffa1a000700 1 -- 172.21.15.151:0/2163144975 shutdown_connections 2024-05-21T22:01:20.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.326+0000 7ffa1a000700 1 -- 172.21.15.151:0/2163144975 wait complete. 2024-05-21T22:01:20.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.327+0000 7ffa1a000700 1 Processor -- start 2024-05-21T22:01:20.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.327+0000 7ffa1a000700 1 -- start start 2024-05-21T22:01:20.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.327+0000 7ffa1a000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa14075770 0x7ffa141a0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:20.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.327+0000 7ffa1a000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffa14076090 0x7ffa141a0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:20.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.327+0000 7ffa1a000700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffa14114710 0x7ffa141a4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:20.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.327+0000 7ffa1a000700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ffa14119d70 con 0x7ffa14075770 2024-05-21T22:01:20.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.327+0000 7ffa1a000700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ffa14119bf0 con 0x7ffa14076090 2024-05-21T22:01:20.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.327+0000 7ffa1a000700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ffa14119ef0 con 0x7ffa14114710 2024-05-21T22:01:20.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.328+0000 7ffa13400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa14075770 0x7ffa141a0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:20.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.328+0000 7ffa0aa00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffa14076090 0x7ffa141a0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:20.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.328+0000 7ffa13400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa14075770 0x7ffa141a0320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:51006/0 (socket says 172.21.15.151:51006) 2024-05-21T22:01:20.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.328+0000 7ffa13400700 1 -- 172.21.15.151:0/3269814722 learned_addr learned my addr 172.21.15.151:0/3269814722 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:01:20.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.328+0000 7ffa0aa00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffa14076090 0x7ffa141a0860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:57786/0 (socket says 172.21.15.151:57786) 2024-05-21T22:01:20.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.328+0000 7ffa13e00700 1 --2- 172.21.15.151:0/3269814722 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffa14114710 0x7ffa141a4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:20.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.328+0000 7ffa13400700 1 -- 172.21.15.151:0/3269814722 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffa14114710 msgr2=0x7ffa141a4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:20.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.328+0000 7ffa13400700 1 --2- 172.21.15.151:0/3269814722 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffa14114710 0x7ffa141a4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:20.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.328+0000 7ffa13400700 1 -- 172.21.15.151:0/3269814722 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffa14076090 msgr2=0x7ffa141a0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:20.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.328+0000 7ffa13400700 1 --2- 172.21.15.151:0/3269814722 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffa14076090 0x7ffa141a0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:20.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.328+0000 7ffa13400700 1 -- 172.21.15.151:0/3269814722 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa141a5330 con 0x7ffa14075770 2024-05-21T22:01:20.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.328+0000 7ffa0aa00700 1 --2- 172.21.15.151:0/3269814722 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffa14076090 0x7ffa141a0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:01:20.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.329+0000 7ffa13400700 1 --2- 172.21.15.151:0/3269814722 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa14075770 0x7ffa141a0320 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7ffa0400d010 tx=0x7ffa04004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:01:20.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.329+0000 7ffa10c00700 1 -- 172.21.15.151:0/3269814722 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa04021070 con 0x7ffa14075770 2024-05-21T22:01:20.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.329+0000 7ffa10c00700 1 -- 172.21.15.151:0/3269814722 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa04007e30 con 0x7ffa14075770 2024-05-21T22:01:20.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.329+0000 7ffa1a000700 1 -- 172.21.15.151:0/3269814722 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa141a55c0 con 0x7ffa14075770 2024-05-21T22:01:20.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.329+0000 7ffa10c00700 1 -- 172.21.15.151:0/3269814722 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa04004640 con 0x7ffa14075770 2024-05-21T22:01:20.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.329+0000 7ffa1a000700 1 -- 172.21.15.151:0/3269814722 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa141acea0 con 0x7ffa14075770 2024-05-21T22:01:20.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.334+0000 7ffa10c00700 1 -- 172.21.15.151:0/3269814722 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7ffa04007980 con 0x7ffa14075770 2024-05-21T22:01:20.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.335+0000 7ffa249cbb80 1 Processor -- start 2024-05-21T22:01:20.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.335+0000 7ffa249cbb80 1 -- start start 2024-05-21T22:01:20.336 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.335+0000 7ffa249cbb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561606f1eb0 0x5561606f2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:20.336 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.336+0000 7ffa249cbb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5561606f27d0 0x5561606ec260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:20.336 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.336+0000 7ffa249cbb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561606ec880 0x5561606ecce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:20.336 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.336+0000 7ffa249cbb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55616064ae40 con 0x5561606f1eb0 2024-05-21T22:01:20.336 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.336+0000 7ffa249cbb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55616063a9e0 con 0x5561606f27d0 2024-05-21T22:01:20.336 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.336+0000 7ffa249cbb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55616063ea20 con 0x5561606ec880 2024-05-21T22:01:20.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.336+0000 7ffa0aa00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561606ec880 0x5561606ecce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:20.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.336+0000 7ffa0aa00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561606ec880 0x5561606ecce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:47926/0 (socket says 172.21.15.151:47926) 2024-05-21T22:01:20.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.336+0000 7ffa0aa00700 1 -- 172.21.15.151:0/2742429851 learned_addr learned my addr 172.21.15.151:0/2742429851 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:01:20.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.336+0000 7ffa13e00700 1 --2- 172.21.15.151:0/2742429851 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561606f1eb0 0x5561606f2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:20.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.336+0000 7ffa13400700 1 --2- 172.21.15.151:0/2742429851 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5561606f27d0 0x5561606ec260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:20.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.336+0000 7ffa10c00700 1 --2- 172.21.15.151:0/3269814722 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7ff9f80b29f0 0x7ff9f80b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:20.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.336+0000 7ffa0aa00700 1 -- 172.21.15.151:0/2742429851 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5561606f27d0 msgr2=0x5561606ec260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:20.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.336+0000 7ffa0aa00700 1 --2- 172.21.15.151:0/2742429851 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5561606f27d0 0x5561606ec260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:20.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.336+0000 7ffa0aa00700 1 -- 172.21.15.151:0/2742429851 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561606f1eb0 msgr2=0x5561606f2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:20.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.336+0000 7ffa0aa00700 1 --2- 172.21.15.151:0/2742429851 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561606f1eb0 0x5561606f2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:20.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.336+0000 7ffa0aa00700 1 -- 172.21.15.151:0/2742429851 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556160603420 con 0x5561606ec880 2024-05-21T22:01:20.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.336+0000 7ffa13400700 1 --2- 172.21.15.151:0/2742429851 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5561606f27d0 0x5561606ec260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T22:01:20.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.337+0000 7ffa10c00700 1 -- 172.21.15.151:0/3269814722 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7ffa0400f020 con 0x7ffa14075770 2024-05-21T22:01:20.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.337+0000 7ffa13e00700 1 --2- 172.21.15.151:0/3269814722 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7ff9f80b29f0 0x7ff9f80b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:20.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.337+0000 7ffa0aa00700 1 --2- 172.21.15.151:0/2742429851 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561606ec880 0x5561606ecce0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc0115f0 tx=0x7ff9fc011900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4e6874341de715b server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:01:20.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.337+0000 7ffa08c00700 1 -- 172.21.15.151:0/2742429851 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc01e070 con 0x5561606ec880 2024-05-21T22:01:20.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.337+0000 7ffa08c00700 1 -- 172.21.15.151:0/2742429851 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9fc0044e0 con 0x5561606ec880 2024-05-21T22:01:20.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.337+0000 7ffa08c00700 1 -- 172.21.15.151:0/2742429851 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc0196b0 con 0x5561606ec880 2024-05-21T22:01:20.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.338+0000 7ffa249cbb80 1 -- 172.21.15.151:0/2742429851 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561606ec880 msgr2=0x5561606ecce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:20.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.338+0000 7ffa13e00700 1 --2- 172.21.15.151:0/3269814722 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7ff9f80b29f0 0x7ff9f80b4eb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c009230 tx=0x7ffa0c014040 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:01:20.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.338+0000 7ffa249cbb80 1 --2- 172.21.15.151:0/2742429851 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561606ec880 0x5561606ecce0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc0115f0 tx=0x7ff9fc011900 comp rx=0 tx=0).stop 2024-05-21T22:01:20.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.338+0000 7ffa249cbb80 1 -- 172.21.15.151:0/2742429851 shutdown_connections 2024-05-21T22:01:20.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.338+0000 7ffa249cbb80 1 --2- 172.21.15.151:0/2742429851 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561606ec880 0x5561606ecce0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:20.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.338+0000 7ffa249cbb80 1 --2- 172.21.15.151:0/2742429851 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561606f1eb0 0x5561606f2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:20.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.338+0000 7ffa249cbb80 1 --2- 172.21.15.151:0/2742429851 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5561606f27d0 0x5561606ec260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:20.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.338+0000 7ffa249cbb80 1 -- 172.21.15.151:0/2742429851 >> 172.21.15.151:0/2742429851 conn(0x55616061abe0 msgr2=0x5561606e7010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:01:20.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.338+0000 7ffa249cbb80 1 -- 172.21.15.151:0/2742429851 shutdown_connections 2024-05-21T22:01:20.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.338+0000 7ffa249cbb80 1 -- 172.21.15.151:0/2742429851 wait complete. 2024-05-21T22:01:20.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.339+0000 7ffa249cbb80 1 -- start start 2024-05-21T22:01:20.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.339+0000 7ffa249cbb80 1 Processor -- start 2024-05-21T22:01:20.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.340+0000 7ffa249cbb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561606ec880 0x5561606f8450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:20.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.340+0000 7ffa249cbb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5561606f1eb0 0x5561606f57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:20.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.340+0000 7ffa249cbb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561606f27d0 0x5561606f5d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:20.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.340+0000 7ffa249cbb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x556160540fa0 con 0x5561606f27d0 2024-05-21T22:01:20.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.340+0000 7ffa249cbb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5561605db760 con 0x5561606f1eb0 2024-05-21T22:01:20.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.340+0000 7ffa249cbb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5561606f9240 con 0x5561606ec880 2024-05-21T22:01:20.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.340+0000 7ffa249cbb80 10 client.0 _ll_register_callbacks cb 0x5561605e4c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:01:20.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.340+0000 7ffa0aa00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5561606f1eb0 0x5561606f57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:20.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.340+0000 7ffa13400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561606ec880 0x5561606f8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:20.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.340+0000 7ffa0aa00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5561606f1eb0 0x5561606f57f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:57798/0 (socket says 172.21.15.151:57798) 2024-05-21T22:01:20.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.340+0000 7ffa0aa00700 1 -- 172.21.15.151:0/1301544194 learned_addr learned my addr 172.21.15.151:0/1301544194 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:01:20.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.340+0000 7ffa13e00700 1 --2- 172.21.15.151:0/1301544194 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561606f27d0 0x5561606f5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:20.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.340+0000 7ffa0aa00700 1 -- 172.21.15.151:0/1301544194 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561606ec880 msgr2=0x5561606f8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:20.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.340+0000 7ffa0aa00700 1 --2- 172.21.15.151:0/1301544194 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561606ec880 0x5561606f8450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:20.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.340+0000 7ffa0aa00700 1 -- 172.21.15.151:0/1301544194 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561606f27d0 msgr2=0x5561606f5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:20.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.341+0000 7ffa0aa00700 1 --2- 172.21.15.151:0/1301544194 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561606f27d0 0x5561606f5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:20.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.341+0000 7ffa0aa00700 1 -- 172.21.15.151:0/1301544194 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55616058a8d0 con 0x5561606f1eb0 2024-05-21T22:01:20.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.341+0000 7ffa13e00700 1 --2- 172.21.15.151:0/1301544194 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561606f27d0 0x5561606f5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T22:01:20.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.341+0000 7ffa0aa00700 1 --2- 172.21.15.151:0/1301544194 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5561606f1eb0 0x5561606f57f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc003d30 tx=0x7ff9fc003d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c809e5d9e37e5cb2 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:01:20.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.341+0000 7ffa01600700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-21T22:01:20.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.341+0000 7ffa01600700 1 -- 172.21.15.151:0/1301544194 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc01e040 con 0x5561606f1eb0 2024-05-21T22:01:20.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.341+0000 7ffa01600700 1 -- 172.21.15.151:0/1301544194 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7ff9fc004030 con 0x5561606f1eb0 2024-05-21T22:01:20.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.341+0000 7ffa249cbb80 1 -- 172.21.15.151:0/1301544194 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9fc0044e0 con 0x5561606f1eb0 2024-05-21T22:01:20.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.342+0000 7ffa01600700 1 -- 172.21.15.151:0/1301544194 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff9fc030050 con 0x5561606f1eb0 2024-05-21T22:01:20.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.342+0000 7ffa249cbb80 10 client.26552 fetch_fsmap learned FSMap version 29 2024-05-21T22:01:20.345 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.342+0000 7ffa249cbb80 1 -- 172.21.15.151:0/1301544194 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55616060b060 con 0x5561606f1eb0 2024-05-21T22:01:20.345 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.342+0000 7ffa01600700 1 -- 172.21.15.151:0/1301544194 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9fc0044e0 con 0x5561606f1eb0 2024-05-21T22:01:20.345 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.342+0000 7ffa01600700 1 -- 172.21.15.151:0/1301544194 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7ff9fc021070 con 0x5561606f1eb0 2024-05-21T22:01:20.345 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.342+0000 7ffa249cbb80 10 client.26552 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:01:20.345 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.342+0000 7ffa249cbb80 10 client.26552 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-05-21T22:01:20.345 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.342+0000 7ffa249cbb80 20 client.26552 populate_metadata read hostname 'smithi151' 2024-05-21T22:01:20.345 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.342+0000 7ffa249cbb80 1 --2- 172.21.15.151:0/1301544194 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x556160705c40 0x556160708030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:20.345 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.342+0000 7ffa249cbb80 4 client.26552 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T22:01:20.345 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.342+0000 7ffa249cbb80 1 -- 172.21.15.151:0/1301544194 --> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55616064cd90 con 0x556160705c40 2024-05-21T22:01:20.345 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.343+0000 7ffa13400700 1 --2- 172.21.15.151:0/1301544194 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x556160705c40 0x556160708030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:20.346 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.343+0000 7ffa13400700 1 --2- 172.21.15.151:0/1301544194 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x556160705c40 0x556160708030 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=90cbd7952a3ff023 server_cookie=3aba2006b8319690 in_seq=0 out_seq=0 2024-05-21T22:01:20.346 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.343+0000 7ffa01600700 10 client.26552 ms_handle_connect on v2:172.21.15.164:6836/1554609572 2024-05-21T22:01:20.346 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.344+0000 7ffa01600700 1 -- 172.21.15.151:0/1301544194 <== mds.2 v2:172.21.15.164:6836/1554609572 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55616064cd90 con 0x556160705c40 2024-05-21T22:01:20.346 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.344+0000 7ffa01600700 10 client.26552 handle_command_reply: tid=0 2024-05-21T22:01:20.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.344+0000 7ffa249cbb80 1 client.26552 shutdown 2024-05-21T22:01:20.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.345+0000 7ffa249cbb80 1 -- 172.21.15.151:0/1301544194 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5561606f1eb0 msgr2=0x5561606f57f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:20.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.345+0000 7ffa249cbb80 1 --2- 172.21.15.151:0/1301544194 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5561606f1eb0 0x5561606f57f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc003d30 tx=0x7ff9fc003d60 comp rx=0 tx=0).stop 2024-05-21T22:01:20.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.345+0000 7ffa249cbb80 1 -- 172.21.15.151:0/1301544194 shutdown_connections 2024-05-21T22:01:20.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.345+0000 7ffa249cbb80 1 --2- 172.21.15.151:0/1301544194 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x556160705c40 0x556160708030 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:20.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.345+0000 7ffa249cbb80 1 --2- 172.21.15.151:0/1301544194 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561606f27d0 0x5561606f5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:20.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.345+0000 7ffa249cbb80 1 --2- 172.21.15.151:0/1301544194 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561606ec880 0x5561606f8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:20.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.345+0000 7ffa249cbb80 1 --2- 172.21.15.151:0/1301544194 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5561606f1eb0 0x5561606f57f0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:20.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.345+0000 7ffa01600700 0 client.26552 ms_handle_reset on v2:172.21.15.164:6836/1554609572 2024-05-21T22:01:20.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.345+0000 7ffa249cbb80 1 -- 172.21.15.151:0/1301544194 >> 172.21.15.151:0/1301544194 conn(0x55616061abe0 msgr2=0x5561606eb250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:01:20.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.345+0000 7ffa249cbb80 1 -- 172.21.15.151:0/1301544194 shutdown_connections 2024-05-21T22:01:20.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.345+0000 7ffa249cbb80 1 -- 172.21.15.151:0/1301544194 wait complete. 2024-05-21T22:01:20.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.345+0000 7ffa249cbb80 20 client.26552 trim_cache size 0 max 16384 2024-05-21T22:01:20.363 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.362+0000 7ffa249cbb80 1 Processor -- start 2024-05-21T22:01:20.363 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.362+0000 7ffa249cbb80 1 -- start start 2024-05-21T22:01:20.363 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.362+0000 7ffa249cbb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5561606ec880 0x5561606ec260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:20.363 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.362+0000 7ffa249cbb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561606f1eb0 0x5561606e69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:20.363 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.362+0000 7ffa249cbb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561606f27d0 0x5561606e6f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:20.363 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.362+0000 7ffa249cbb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ffa141b43a0 con 0x5561606f27d0 2024-05-21T22:01:20.363 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.362+0000 7ffa249cbb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ffa141b5ee0 con 0x5561606ec880 2024-05-21T22:01:20.363 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.362+0000 7ffa249cbb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ffa141b4a70 con 0x5561606f1eb0 2024-05-21T22:01:20.363 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.362+0000 7ffa13e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561606f27d0 0x5561606e6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:20.363 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.362+0000 7ffa13e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561606f27d0 0x5561606e6f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:51038/0 (socket says 172.21.15.151:51038) 2024-05-21T22:01:20.363 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.363+0000 7ffa13e00700 1 -- 172.21.15.151:0/3310762572 learned_addr learned my addr 172.21.15.151:0/3310762572 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:01:20.364 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.362+0000 7ffa0aa00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561606f1eb0 0x5561606e69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:20.364 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.363+0000 7ffa13400700 1 --2- 172.21.15.151:0/3310762572 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5561606ec880 0x5561606ec260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:20.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.363+0000 7ffa0aa00700 1 -- 172.21.15.151:0/3310762572 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5561606ec880 msgr2=0x5561606ec260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:20.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.363+0000 7ffa0aa00700 1 --2- 172.21.15.151:0/3310762572 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5561606ec880 0x5561606ec260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:20.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.363+0000 7ffa0aa00700 1 -- 172.21.15.151:0/3310762572 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561606f27d0 msgr2=0x5561606e6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:20.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.363+0000 7ffa0aa00700 1 --2- 172.21.15.151:0/3310762572 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561606f27d0 0x5561606e6f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:20.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.363+0000 7ffa0aa00700 1 -- 172.21.15.151:0/3310762572 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561606e14c0 con 0x5561606f1eb0 2024-05-21T22:01:20.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.363+0000 7ffa0aa00700 1 --2- 172.21.15.151:0/3310762572 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561606f1eb0 0x5561606e69e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc011510 tx=0x7ff9fc011900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68825cddeb36259e server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:01:20.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.363+0000 7ffa03e00700 1 -- 172.21.15.151:0/3310762572 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc015020 con 0x5561606f1eb0 2024-05-21T22:01:20.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.364+0000 7ffa03e00700 1 -- 172.21.15.151:0/3310762572 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9fc027070 con 0x5561606f1eb0 2024-05-21T22:01:20.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.364+0000 7ffa03e00700 1 -- 172.21.15.151:0/3310762572 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc01e070 con 0x5561606f1eb0 2024-05-21T22:01:20.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.364+0000 7ffa249cbb80 1 -- 172.21.15.151:0/3310762572 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561606f1eb0 msgr2=0x5561606e69e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:20.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.364+0000 7ffa249cbb80 1 --2- 172.21.15.151:0/3310762572 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561606f1eb0 0x5561606e69e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc011510 tx=0x7ff9fc011900 comp rx=0 tx=0).stop 2024-05-21T22:01:20.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.364+0000 7ffa249cbb80 1 -- 172.21.15.151:0/3310762572 shutdown_connections 2024-05-21T22:01:20.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.364+0000 7ffa249cbb80 1 --2- 172.21.15.151:0/3310762572 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5561606ec880 0x5561606ec260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:20.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.364+0000 7ffa249cbb80 1 --2- 172.21.15.151:0/3310762572 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561606f27d0 0x5561606e6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:20.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.364+0000 7ffa249cbb80 1 --2- 172.21.15.151:0/3310762572 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561606f1eb0 0x5561606e69e0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:20.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.364+0000 7ffa249cbb80 1 -- 172.21.15.151:0/3310762572 >> 172.21.15.151:0/3310762572 conn(0x55616061abe0 msgr2=0x5561606f40e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:01:20.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.365+0000 7ffa249cbb80 1 -- 172.21.15.151:0/3310762572 shutdown_connections 2024-05-21T22:01:20.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.365+0000 7ffa249cbb80 1 -- 172.21.15.151:0/3310762572 wait complete. 2024-05-21T22:01:20.368 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.365+0000 7ffa249cbb80 1 -- start start 2024-05-21T22:01:20.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.365+0000 7ffa249cbb80 1 Processor -- start 2024-05-21T22:01:20.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.366+0000 7ffa249cbb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5561606ec880 0x55616070d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:20.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.366+0000 7ffa249cbb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561606f1eb0 0x55616070d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:20.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.366+0000 7ffa249cbb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561606f27d0 0x55616070de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:20.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.366+0000 7ffa249cbb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ffa141b5810 con 0x5561606f1eb0 2024-05-21T22:01:20.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.366+0000 7ffa249cbb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ffa141b5140 con 0x5561606ec880 2024-05-21T22:01:20.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.366+0000 7ffa249cbb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ffa141b2340 con 0x5561606f27d0 2024-05-21T22:01:20.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.366+0000 7ffa249cbb80 10 client.0 _ll_register_callbacks cb 0x5561605e4c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:01:20.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.366+0000 7ffa13400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5561606ec880 0x55616070d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:20.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.366+0000 7ffa0aa00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561606f1eb0 0x55616070d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:20.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.366+0000 7ffa13400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5561606ec880 0x55616070d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:57812/0 (socket says 172.21.15.151:57812) 2024-05-21T22:01:20.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.366+0000 7ffa13400700 1 -- 172.21.15.151:0/632986479 learned_addr learned my addr 172.21.15.151:0/632986479 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:01:20.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.366+0000 7ffa13e00700 1 --2- 172.21.15.151:0/632986479 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561606f27d0 0x55616070de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:20.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.367+0000 7ffa13400700 1 -- 172.21.15.151:0/632986479 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561606f27d0 msgr2=0x55616070de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:20.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.367+0000 7ffa13400700 1 --2- 172.21.15.151:0/632986479 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561606f27d0 0x55616070de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:20.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.367+0000 7ffa13400700 1 -- 172.21.15.151:0/632986479 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561606f1eb0 msgr2=0x55616070d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:20.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.367+0000 7ffa13400700 1 --2- 172.21.15.151:0/632986479 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561606f1eb0 0x55616070d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:20.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.367+0000 7ffa13400700 1 -- 172.21.15.151:0/632986479 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55616076f750 con 0x5561606ec880 2024-05-21T22:01:20.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.367+0000 7ffa0aa00700 1 --2- 172.21.15.151:0/632986479 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561606f1eb0 0x55616070d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:01:20.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.367+0000 7ffa13400700 1 --2- 172.21.15.151:0/632986479 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5561606ec880 0x55616070d3b0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ffa04009820 tx=0x7ffa04004eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=550bf9b3b63ea7de server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:01:20.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.367+0000 7ffa01600700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-21T22:01:20.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.367+0000 7ffa01600700 1 -- 172.21.15.151:0/632986479 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa040e9040 con 0x5561606ec880 2024-05-21T22:01:20.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.367+0000 7ffa01600700 1 -- 172.21.15.151:0/632986479 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7ffa040074e0 con 0x5561606ec880 2024-05-21T22:01:20.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.367+0000 7ffa249cbb80 1 -- 172.21.15.151:0/632986479 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9fc027070 con 0x5561606ec880 2024-05-21T22:01:20.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.368+0000 7ffa01600700 1 -- 172.21.15.151:0/632986479 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ffa040aa8f0 con 0x5561606ec880 2024-05-21T22:01:20.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.368+0000 7ffa249cbb80 10 client.26558 fetch_fsmap learned FSMap version 29 2024-05-21T22:01:20.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.368+0000 7ffa249cbb80 1 -- 172.21.15.151:0/632986479 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556160770a70 con 0x5561606ec880 2024-05-21T22:01:20.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.368+0000 7ffa01600700 1 -- 172.21.15.151:0/632986479 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9fc027070 con 0x5561606ec880 2024-05-21T22:01:20.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.368+0000 7ffa01600700 1 -- 172.21.15.151:0/632986479 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7ffa040c3020 con 0x5561606ec880 2024-05-21T22:01:20.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.368+0000 7ffa249cbb80 10 client.26558 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:01:20.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.368+0000 7ffa249cbb80 10 client.26558 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-05-21T22:01:20.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.368+0000 7ffa249cbb80 20 client.26558 populate_metadata read hostname 'smithi151' 2024-05-21T22:01:20.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.368+0000 7ffa249cbb80 1 --2- 172.21.15.151:0/632986479 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x556160771010 0x556160773430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:20.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.368+0000 7ffa249cbb80 4 client.26558 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T22:01:20.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.368+0000 7ffa249cbb80 1 -- 172.21.15.151:0/632986479 --> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556160646570 con 0x556160771010 2024-05-21T22:01:20.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.368+0000 7ffa0aa00700 1 --2- 172.21.15.151:0/632986479 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x556160771010 0x556160773430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:20.372 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.369+0000 7ffa0aa00700 1 --2- 172.21.15.151:0/632986479 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x556160771010 0x556160773430 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bb14d96d3f4caf3e server_cookie=27626434bbb8805f in_seq=0 out_seq=0 2024-05-21T22:01:20.372 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.369+0000 7ffa01600700 10 client.26558 ms_handle_connect on v2:172.21.15.164:6836/1554609572 2024-05-21T22:01:20.372 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.369+0000 7ffa01600700 1 -- 172.21.15.151:0/632986479 <== mds.2 v2:172.21.15.164:6836/1554609572 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556160646570 con 0x556160771010 2024-05-21T22:01:20.372 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.369+0000 7ffa01600700 10 client.26558 handle_command_reply: tid=0 2024-05-21T22:01:20.372 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.369+0000 7ffa249cbb80 1 client.26558 shutdown 2024-05-21T22:01:20.373 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.369+0000 7ffa249cbb80 1 -- 172.21.15.151:0/632986479 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5561606ec880 msgr2=0x55616070d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:20.373 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.369+0000 7ffa249cbb80 1 --2- 172.21.15.151:0/632986479 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5561606ec880 0x55616070d3b0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ffa04009820 tx=0x7ffa04004eb0 comp rx=0 tx=0).stop 2024-05-21T22:01:20.373 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.370+0000 7ffa249cbb80 1 -- 172.21.15.151:0/632986479 shutdown_connections 2024-05-21T22:01:20.373 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.370+0000 7ffa249cbb80 1 --2- 172.21.15.151:0/632986479 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x556160771010 0x556160773430 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:20.373 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.370+0000 7ffa249cbb80 1 --2- 172.21.15.151:0/632986479 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5561606ec880 0x55616070d3b0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:20.373 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.370+0000 7ffa249cbb80 1 --2- 172.21.15.151:0/632986479 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561606f27d0 0x55616070de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:20.373 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.370+0000 7ffa249cbb80 1 --2- 172.21.15.151:0/632986479 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561606f1eb0 0x55616070d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:20.373 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.370+0000 7ffa01600700 0 client.26558 ms_handle_reset on v2:172.21.15.164:6836/1554609572 2024-05-21T22:01:20.373 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.370+0000 7ffa249cbb80 1 -- 172.21.15.151:0/632986479 >> 172.21.15.151:0/632986479 conn(0x55616061abe0 msgr2=0x5561606e69e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:01:20.373 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.370+0000 7ffa249cbb80 1 -- 172.21.15.151:0/632986479 shutdown_connections 2024-05-21T22:01:20.373 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.370+0000 7ffa249cbb80 1 -- 172.21.15.151:0/632986479 wait complete. 2024-05-21T22:01:20.373 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.370+0000 7ffa249cbb80 20 client.26558 trim_cache size 0 max 16384 2024-05-21T22:01:20.374 INFO:teuthology.orchestra.run.smithi151.stdout:[] 2024-05-21T22:01:20.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.371+0000 7ffa1a000700 1 -- 172.21.15.151:0/3269814722 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7ff9f80b29f0 msgr2=0x7ff9f80b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:20.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.371+0000 7ffa1a000700 1 --2- 172.21.15.151:0/3269814722 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7ff9f80b29f0 0x7ff9f80b4eb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c009230 tx=0x7ffa0c014040 comp rx=0 tx=0).stop 2024-05-21T22:01:20.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.371+0000 7ffa1a000700 1 -- 172.21.15.151:0/3269814722 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa14075770 msgr2=0x7ffa141a0320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:20.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.371+0000 7ffa1a000700 1 --2- 172.21.15.151:0/3269814722 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa14075770 0x7ffa141a0320 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7ffa0400d010 tx=0x7ffa04004d00 comp rx=0 tx=0).stop 2024-05-21T22:01:20.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.371+0000 7ffa1a000700 1 -- 172.21.15.151:0/3269814722 shutdown_connections 2024-05-21T22:01:20.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.371+0000 7ffa1a000700 1 --2- 172.21.15.151:0/3269814722 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7ff9f80b29f0 0x7ff9f80b4eb0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:20.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.371+0000 7ffa1a000700 1 --2- 172.21.15.151:0/3269814722 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffa14114710 0x7ffa141a4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:20.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.371+0000 7ffa1a000700 1 --2- 172.21.15.151:0/3269814722 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa14075770 0x7ffa141a0320 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:20.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.371+0000 7ffa1a000700 1 --2- 172.21.15.151:0/3269814722 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffa14076090 0x7ffa141a0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:20.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.371+0000 7ffa1a000700 1 -- 172.21.15.151:0/3269814722 >> 172.21.15.151:0/3269814722 conn(0x7ffa1407a6a0 msgr2=0x7ffa14113540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:01:20.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.371+0000 7ffa1a000700 1 -- 172.21.15.151:0/3269814722 shutdown_connections 2024-05-21T22:01:20.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:20.371+0000 7ffa1a000700 1 -- 172.21.15.151:0/3269814722 wait complete. 2024-05-21T22:01:21.428 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-21T22:01:21.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:01:21 smithi151 ceph-mon[31623]: pgmap v175: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-05-21T22:01:21.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:01:21 smithi164 ceph-mon[36269]: pgmap v175: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-05-21T22:01:21.881 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:01:21.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:01:21 smithi184 ceph-mon[36942]: pgmap v175: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-05-21T22:01:23.593 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:01:23 smithi151 ceph-mon[31623]: pgmap v176: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2024-05-21T22:01:23.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:01:23 smithi184 ceph-mon[36942]: pgmap v176: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2024-05-21T22:01:24.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:01:23 smithi164 ceph-mon[36269]: pgmap v176: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2024-05-21T22:01:24.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.138+0000 7f997a200700 1 -- 172.21.15.151:0/4030262731 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9974105760 msgr2=0x7f9974105be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:24.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.138+0000 7f997a200700 1 --2- 172.21.15.151:0/4030262731 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9974105760 0x7f9974105be0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f9960009990 tx=0x7f9960009ca0 comp rx=0 tx=0).stop 2024-05-21T22:01:24.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.138+0000 7f997a200700 1 -- 172.21.15.151:0/4030262731 shutdown_connections 2024-05-21T22:01:24.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.138+0000 7f997a200700 1 --2- 172.21.15.151:0/4030262731 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f997410b760 0x7f997410bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:24.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.138+0000 7f997a200700 1 --2- 172.21.15.151:0/4030262731 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9974106120 0x7f9974112630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:24.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.138+0000 7f997a200700 1 --2- 172.21.15.151:0/4030262731 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9974105760 0x7f9974105be0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:24.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.138+0000 7f997a200700 1 -- 172.21.15.151:0/4030262731 >> 172.21.15.151:0/4030262731 conn(0x7f9974101250 msgr2=0x7f9974103670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:01:24.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.138+0000 7f997a200700 1 -- 172.21.15.151:0/4030262731 shutdown_connections 2024-05-21T22:01:24.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.138+0000 7f997a200700 1 -- 172.21.15.151:0/4030262731 wait complete. 2024-05-21T22:01:24.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.140+0000 7f997a200700 1 Processor -- start 2024-05-21T22:01:24.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.140+0000 7f997a200700 1 -- start start 2024-05-21T22:01:24.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.140+0000 7f997a200700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9974105760 0x7f997419bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:24.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.140+0000 7f997a200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9974106120 0x7f997419c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:24.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.140+0000 7f997a200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f997410b760 0x7f99741a0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:24.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.140+0000 7f997a200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f99741159f0 con 0x7f9974106120 2024-05-21T22:01:24.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.140+0000 7f997a200700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9974115870 con 0x7f997410b760 2024-05-21T22:01:24.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.140+0000 7f997a200700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f9974115b70 con 0x7f9974105760 2024-05-21T22:01:24.142 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.141+0000 7f9972a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9974106120 0x7f997419c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:24.142 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.141+0000 7f9972a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9974106120 0x7f997419c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:51058/0 (socket says 172.21.15.151:51058) 2024-05-21T22:01:24.142 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.141+0000 7f9973400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9974105760 0x7f997419bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:24.142 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.141+0000 7f9972a00700 1 -- 172.21.15.151:0/115555456 learned_addr learned my addr 172.21.15.151:0/115555456 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:01:24.142 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.141+0000 7f9973e00700 1 --2- 172.21.15.151:0/115555456 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f997410b760 0x7f99741a0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:24.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.141+0000 7f9973e00700 1 -- 172.21.15.151:0/115555456 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9974105760 msgr2=0x7f997419bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:24.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.141+0000 7f9973e00700 1 --2- 172.21.15.151:0/115555456 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9974105760 0x7f997419bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:24.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.141+0000 7f9973e00700 1 -- 172.21.15.151:0/115555456 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9974106120 msgr2=0x7f997419c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:24.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.141+0000 7f9973e00700 1 --2- 172.21.15.151:0/115555456 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9974106120 0x7f997419c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:24.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.141+0000 7f9973e00700 1 -- 172.21.15.151:0/115555456 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99741a0e60 con 0x7f997410b760 2024-05-21T22:01:24.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.141+0000 7f9973400700 1 --2- 172.21.15.151:0/115555456 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9974105760 0x7f997419bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:01:24.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.142+0000 7f9973e00700 1 --2- 172.21.15.151:0/115555456 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f997410b760 0x7f99741a0720 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f996c00efc0 tx=0x7f996c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:01:24.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.142+0000 7f996be00700 1 -- 172.21.15.151:0/115555456 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f996c0092e0 con 0x7f997410b760 2024-05-21T22:01:24.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.142+0000 7f997a200700 1 -- 172.21.15.151:0/115555456 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99741a10f0 con 0x7f997410b760 2024-05-21T22:01:24.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.142+0000 7f996be00700 1 -- 172.21.15.151:0/115555456 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f996c004bb0 con 0x7f997410b760 2024-05-21T22:01:24.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.142+0000 7f996be00700 1 -- 172.21.15.151:0/115555456 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f996c004020 con 0x7f997410b760 2024-05-21T22:01:24.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.142+0000 7f997a200700 1 -- 172.21.15.151:0/115555456 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99741a89d0 con 0x7f997410b760 2024-05-21T22:01:24.146 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.146+0000 7f996be00700 1 -- 172.21.15.151:0/115555456 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7f996c01b030 con 0x7f997410b760 2024-05-21T22:01:24.147 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.147+0000 7f9984b3fb80 1 Processor -- start 2024-05-21T22:01:24.147 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.147+0000 7f9984b3fb80 1 -- start start 2024-05-21T22:01:24.147 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.147+0000 7f996be00700 1 --2- 172.21.15.151:0/115555456 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f99580b29f0 0x7f99580b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:24.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.147+0000 7f9984b3fb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dfb1a06eb0 0x55dfb1a07290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:24.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.147+0000 7f9984b3fb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dfb1a077d0 0x55dfb1a01260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:24.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.147+0000 7f9984b3fb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dfb1a01880 0x55dfb1a01ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:24.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.147+0000 7f9984b3fb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55dfb195fe40 con 0x55dfb1a01880 2024-05-21T22:01:24.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.147+0000 7f9984b3fb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55dfb194f9e0 con 0x55dfb1a077d0 2024-05-21T22:01:24.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.147+0000 7f9984b3fb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55dfb1953a20 con 0x55dfb1a06eb0 2024-05-21T22:01:24.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.147+0000 7f996be00700 1 -- 172.21.15.151:0/115555456 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f996c0e2150 con 0x7f997410b760 2024-05-21T22:01:24.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.148+0000 7f9972a00700 1 --2- 172.21.15.151:0/115555456 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f99580b29f0 0x7f99580b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:24.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.148+0000 7f9973400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dfb1a06eb0 0x55dfb1a07290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:24.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.148+0000 7f9972a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dfb1a077d0 0x55dfb1a01260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:24.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.148+0000 7f9973400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dfb1a06eb0 0x55dfb1a07290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:48006/0 (socket says 172.21.15.151:48006) 2024-05-21T22:01:24.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.148+0000 7f9973400700 1 -- 172.21.15.151:0/364353329 learned_addr learned my addr 172.21.15.151:0/364353329 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:01:24.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.148+0000 7f9973e00700 1 --2- 172.21.15.151:0/364353329 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dfb1a01880 0x55dfb1a01ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:24.150 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.148+0000 7f9973400700 1 -- 172.21.15.151:0/364353329 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dfb1a077d0 msgr2=0x55dfb1a01260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:24.150 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.148+0000 7f9973400700 1 --2- 172.21.15.151:0/364353329 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dfb1a077d0 0x55dfb1a01260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:24.150 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.148+0000 7f9973400700 1 -- 172.21.15.151:0/364353329 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dfb1a01880 msgr2=0x55dfb1a01ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:24.150 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.148+0000 7f9973400700 1 --2- 172.21.15.151:0/364353329 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dfb1a01880 0x55dfb1a01ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:24.150 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.148+0000 7f9973400700 1 -- 172.21.15.151:0/364353329 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfb1918420 con 0x55dfb1a06eb0 2024-05-21T22:01:24.150 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.149+0000 7f9972a00700 1 --2- 172.21.15.151:0/115555456 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f99580b29f0 0x7f99580b4eb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f997410ad60 tx=0x7f9960005790 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:01:24.150 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.149+0000 7f9973400700 1 --2- 172.21.15.151:0/364353329 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dfb1a06eb0 0x55dfb1a07290 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f99640115f0 tx=0x7f9964011900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f189869f31a9dc7 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:01:24.150 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.149+0000 7f9968c00700 1 -- 172.21.15.151:0/364353329 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f996401e070 con 0x55dfb1a06eb0 2024-05-21T22:01:24.150 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.149+0000 7f9968c00700 1 -- 172.21.15.151:0/364353329 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9964019ae0 con 0x55dfb1a06eb0 2024-05-21T22:01:24.150 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.149+0000 7f9968c00700 1 -- 172.21.15.151:0/364353329 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f996400ea70 con 0x55dfb1a06eb0 2024-05-21T22:01:24.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.150+0000 7f9984b3fb80 1 -- 172.21.15.151:0/364353329 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dfb1a06eb0 msgr2=0x55dfb1a07290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:24.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.150+0000 7f9984b3fb80 1 --2- 172.21.15.151:0/364353329 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dfb1a06eb0 0x55dfb1a07290 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f99640115f0 tx=0x7f9964011900 comp rx=0 tx=0).stop 2024-05-21T22:01:24.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.150+0000 7f9984b3fb80 1 -- 172.21.15.151:0/364353329 shutdown_connections 2024-05-21T22:01:24.152 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.150+0000 7f9984b3fb80 1 --2- 172.21.15.151:0/364353329 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dfb1a01880 0x55dfb1a01ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:24.152 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.150+0000 7f9984b3fb80 1 --2- 172.21.15.151:0/364353329 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dfb1a06eb0 0x55dfb1a07290 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:24.152 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.150+0000 7f9984b3fb80 1 --2- 172.21.15.151:0/364353329 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dfb1a077d0 0x55dfb1a01260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:24.152 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.150+0000 7f9984b3fb80 1 -- 172.21.15.151:0/364353329 >> 172.21.15.151:0/364353329 conn(0x55dfb192fbe0 msgr2=0x55dfb19fc010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:01:24.152 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.150+0000 7f9984b3fb80 1 -- 172.21.15.151:0/364353329 shutdown_connections 2024-05-21T22:01:24.152 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.150+0000 7f9984b3fb80 1 -- 172.21.15.151:0/364353329 wait complete. 2024-05-21T22:01:24.152 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.151+0000 7f9984b3fb80 1 -- start start 2024-05-21T22:01:24.152 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.151+0000 7f9984b3fb80 1 Processor -- start 2024-05-21T22:01:24.153 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.152+0000 7f9984b3fb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dfb1a01880 0x55dfb1a0d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:24.153 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.152+0000 7f9984b3fb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dfb1a06eb0 0x55dfb1a0a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:24.153 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.152+0000 7f9984b3fb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dfb1a077d0 0x55dfb1a0ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:24.153 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.152+0000 7f9984b3fb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55dfb1855fa0 con 0x55dfb1a077d0 2024-05-21T22:01:24.153 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.152+0000 7f9984b3fb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55dfb18f0760 con 0x55dfb1a06eb0 2024-05-21T22:01:24.153 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.152+0000 7f9984b3fb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55dfb1a0e240 con 0x55dfb1a01880 2024-05-21T22:01:24.153 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.152+0000 7f9984b3fb80 10 client.0 _ll_register_callbacks cb 0x55dfb18f9c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:01:24.153 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.152+0000 7f9972a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dfb1a06eb0 0x55dfb1a0a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:24.153 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.152+0000 7f9972a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dfb1a06eb0 0x55dfb1a0a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:57840/0 (socket says 172.21.15.151:57840) 2024-05-21T22:01:24.153 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.152+0000 7f9972a00700 1 -- 172.21.15.151:0/2090367956 learned_addr learned my addr 172.21.15.151:0/2090367956 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:01:24.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.152+0000 7f9973e00700 1 --2- 172.21.15.151:0/2090367956 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dfb1a077d0 0x55dfb1a0ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:24.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.152+0000 7f9973400700 1 --2- 172.21.15.151:0/2090367956 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dfb1a01880 0x55dfb1a0d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:24.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.152+0000 7f9972a00700 1 -- 172.21.15.151:0/2090367956 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dfb1a01880 msgr2=0x55dfb1a0d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:24.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.152+0000 7f9972a00700 1 --2- 172.21.15.151:0/2090367956 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dfb1a01880 0x55dfb1a0d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:24.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.152+0000 7f9972a00700 1 -- 172.21.15.151:0/2090367956 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dfb1a077d0 msgr2=0x55dfb1a0ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:24.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.152+0000 7f9972a00700 1 --2- 172.21.15.151:0/2090367956 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dfb1a077d0 0x55dfb1a0ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:24.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.152+0000 7f9972a00700 1 -- 172.21.15.151:0/2090367956 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfb189f8d0 con 0x55dfb1a06eb0 2024-05-21T22:01:24.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.152+0000 7f9973400700 1 --2- 172.21.15.151:0/2090367956 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dfb1a01880 0x55dfb1a0d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:01:24.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.153+0000 7f9972a00700 1 --2- 172.21.15.151:0/2090367956 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dfb1a06eb0 0x55dfb1a0a840 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f9960005bc0 tx=0x7f9960028000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8cb68c20967b7ec6 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:01:24.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.153+0000 7f995d600700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-21T22:01:24.155 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.153+0000 7f995d600700 1 -- 172.21.15.151:0/2090367956 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f996002e070 con 0x55dfb1a06eb0 2024-05-21T22:01:24.155 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.153+0000 7f9984b3fb80 1 -- 172.21.15.151:0/2090367956 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9964019ae0 con 0x55dfb1a06eb0 2024-05-21T22:01:24.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.153+0000 7f995d600700 1 -- 172.21.15.151:0/2090367956 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f996000f460 con 0x55dfb1a06eb0 2024-05-21T22:01:24.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.153+0000 7f995d600700 1 -- 172.21.15.151:0/2090367956 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f996000f660 con 0x55dfb1a06eb0 2024-05-21T22:01:24.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.153+0000 7f9984b3fb80 10 client.26570 fetch_fsmap learned FSMap version 29 2024-05-21T22:01:24.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.153+0000 7f9984b3fb80 1 -- 172.21.15.151:0/2090367956 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55dfb1920060 con 0x55dfb1a06eb0 2024-05-21T22:01:24.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.154+0000 7f995d600700 1 -- 172.21.15.151:0/2090367956 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9964019ae0 con 0x55dfb1a06eb0 2024-05-21T22:01:24.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.154+0000 7f995d600700 1 -- 172.21.15.151:0/2090367956 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f99600327b0 con 0x55dfb1a06eb0 2024-05-21T22:01:24.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.154+0000 7f9984b3fb80 10 client.26570 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:01:24.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.154+0000 7f9984b3fb80 10 client.26570 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-21T22:01:24.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.154+0000 7f9984b3fb80 20 client.26570 populate_metadata read hostname 'smithi151' 2024-05-21T22:01:24.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.154+0000 7f9984b3fb80 1 --2- 172.21.15.151:0/2090367956 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x55dfb1a1add0 0x55dfb1a1d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:24.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.154+0000 7f9984b3fb80 4 client.26570 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T22:01:24.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.154+0000 7f9984b3fb80 1 -- 172.21.15.151:0/2090367956 --> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dfb1961d90 con 0x55dfb1a1add0 2024-05-21T22:01:24.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.155+0000 7f9973400700 1 --2- 172.21.15.151:0/2090367956 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x55dfb1a1add0 0x55dfb1a1d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:24.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.155+0000 7f995d600700 10 client.26570 ms_handle_connect on v2:172.21.15.184:6838/3059935143 2024-05-21T22:01:24.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.155+0000 7f9973400700 1 --2- 172.21.15.151:0/2090367956 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x55dfb1a1add0 0x55dfb1a1d1c0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=804701bcea6c451c server_cookie=32b05d81e6d544df in_seq=0 out_seq=0 2024-05-21T22:01:24.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.156+0000 7f995d600700 1 -- 172.21.15.151:0/2090367956 <== mds.1 v2:172.21.15.184:6838/3059935143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55dfb1961d90 con 0x55dfb1a1add0 2024-05-21T22:01:24.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.156+0000 7f995d600700 10 client.26570 handle_command_reply: tid=0 2024-05-21T22:01:24.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.157+0000 7f9984b3fb80 1 client.26570 shutdown 2024-05-21T22:01:24.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.157+0000 7f9984b3fb80 1 -- 172.21.15.151:0/2090367956 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dfb1a06eb0 msgr2=0x55dfb1a0a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:24.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.157+0000 7f9984b3fb80 1 --2- 172.21.15.151:0/2090367956 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dfb1a06eb0 0x55dfb1a0a840 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f9960005bc0 tx=0x7f9960028000 comp rx=0 tx=0).stop 2024-05-21T22:01:24.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.157+0000 7f9984b3fb80 1 -- 172.21.15.151:0/2090367956 shutdown_connections 2024-05-21T22:01:24.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.157+0000 7f9984b3fb80 1 --2- 172.21.15.151:0/2090367956 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x55dfb1a1add0 0x55dfb1a1d1c0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:24.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.157+0000 7f9984b3fb80 1 --2- 172.21.15.151:0/2090367956 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dfb1a077d0 0x55dfb1a0ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:24.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.157+0000 7f9984b3fb80 1 --2- 172.21.15.151:0/2090367956 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dfb1a01880 0x55dfb1a0d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:24.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.157+0000 7f9984b3fb80 1 --2- 172.21.15.151:0/2090367956 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dfb1a06eb0 0x55dfb1a0a840 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:24.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.157+0000 7f995d600700 0 client.26570 ms_handle_reset on v2:172.21.15.184:6838/3059935143 2024-05-21T22:01:24.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.157+0000 7f9984b3fb80 1 -- 172.21.15.151:0/2090367956 >> 172.21.15.151:0/2090367956 conn(0x55dfb192fbe0 msgr2=0x55dfb1a00250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:01:24.159 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.157+0000 7f9984b3fb80 1 -- 172.21.15.151:0/2090367956 shutdown_connections 2024-05-21T22:01:24.159 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.157+0000 7f9984b3fb80 1 -- 172.21.15.151:0/2090367956 wait complete. 2024-05-21T22:01:24.159 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.157+0000 7f9984b3fb80 20 client.26570 trim_cache size 0 max 16384 2024-05-21T22:01:24.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.174+0000 7f9984b3fb80 1 Processor -- start 2024-05-21T22:01:24.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.174+0000 7f9984b3fb80 1 -- start start 2024-05-21T22:01:24.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.174+0000 7f9984b3fb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dfb1a01880 0x55dfb1a01260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:24.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.174+0000 7f9984b3fb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dfb1a06eb0 0x55dfb1a049c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:24.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.174+0000 7f9984b3fb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dfb1a077d0 0x55dfb1a09130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:24.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.174+0000 7f9984b3fb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f99741b0020 con 0x55dfb1a06eb0 2024-05-21T22:01:24.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.174+0000 7f9984b3fb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f99741b1b60 con 0x55dfb1a077d0 2024-05-21T22:01:24.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.174+0000 7f9984b3fb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f99741b06f0 con 0x55dfb1a01880 2024-05-21T22:01:24.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.174+0000 7f9972a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dfb1a06eb0 0x55dfb1a049c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:24.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.174+0000 7f9972a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dfb1a06eb0 0x55dfb1a049c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:51094/0 (socket says 172.21.15.151:51094) 2024-05-21T22:01:24.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.174+0000 7f9972a00700 1 -- 172.21.15.151:0/833958959 learned_addr learned my addr 172.21.15.151:0/833958959 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:01:24.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.175+0000 7f9973e00700 1 --2- 172.21.15.151:0/833958959 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dfb1a077d0 0x55dfb1a09130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:24.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.175+0000 7f9973400700 1 --2- 172.21.15.151:0/833958959 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dfb1a01880 0x55dfb1a01260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:24.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.175+0000 7f9972a00700 1 -- 172.21.15.151:0/833958959 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dfb1a01880 msgr2=0x55dfb1a01260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:24.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.175+0000 7f9972a00700 1 --2- 172.21.15.151:0/833958959 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dfb1a01880 0x55dfb1a01260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:24.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.175+0000 7f9972a00700 1 -- 172.21.15.151:0/833958959 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dfb1a077d0 msgr2=0x55dfb1a09130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:24.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.175+0000 7f9972a00700 1 --2- 172.21.15.151:0/833958959 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dfb1a077d0 0x55dfb1a09130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:24.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.175+0000 7f9972a00700 1 -- 172.21.15.151:0/833958959 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfb1a0d0e0 con 0x55dfb1a06eb0 2024-05-21T22:01:24.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.175+0000 7f9973400700 1 --2- 172.21.15.151:0/833958959 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dfb1a01880 0x55dfb1a01260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T22:01:24.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.175+0000 7f9972a00700 1 --2- 172.21.15.151:0/833958959 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dfb1a06eb0 0x55dfb1a049c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f9960009ce0 tx=0x7f9960005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b99b9bd3c7bb52dc server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:01:24.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.175+0000 7f995fe00700 1 -- 172.21.15.151:0/833958959 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f996002e070 con 0x55dfb1a06eb0 2024-05-21T22:01:24.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.175+0000 7f995fe00700 1 -- 172.21.15.151:0/833958959 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9960028a20 con 0x55dfb1a06eb0 2024-05-21T22:01:24.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.175+0000 7f995fe00700 1 -- 172.21.15.151:0/833958959 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f996000fd90 con 0x55dfb1a06eb0 2024-05-21T22:01:24.183 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.176+0000 7f9984b3fb80 1 -- 172.21.15.151:0/833958959 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dfb1a06eb0 msgr2=0x55dfb1a049c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:24.183 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.176+0000 7f9984b3fb80 1 --2- 172.21.15.151:0/833958959 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dfb1a06eb0 0x55dfb1a049c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f9960009ce0 tx=0x7f9960005720 comp rx=0 tx=0).stop 2024-05-21T22:01:24.183 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.176+0000 7f9984b3fb80 1 -- 172.21.15.151:0/833958959 shutdown_connections 2024-05-21T22:01:24.183 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.176+0000 7f9984b3fb80 1 --2- 172.21.15.151:0/833958959 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dfb1a077d0 0x55dfb1a09130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:24.183 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.176+0000 7f9984b3fb80 1 --2- 172.21.15.151:0/833958959 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dfb1a01880 0x55dfb1a01260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:24.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.176+0000 7f9984b3fb80 1 --2- 172.21.15.151:0/833958959 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dfb1a06eb0 0x55dfb1a049c0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:24.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.176+0000 7f9984b3fb80 1 -- 172.21.15.151:0/833958959 >> 172.21.15.151:0/833958959 conn(0x55dfb192fbe0 msgr2=0x55dfb1a01c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:01:24.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.176+0000 7f9984b3fb80 1 -- 172.21.15.151:0/833958959 shutdown_connections 2024-05-21T22:01:24.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.176+0000 7f9984b3fb80 1 -- 172.21.15.151:0/833958959 wait complete. 2024-05-21T22:01:24.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.177+0000 7f9984b3fb80 1 -- start start 2024-05-21T22:01:24.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.177+0000 7f9984b3fb80 1 Processor -- start 2024-05-21T22:01:24.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.177+0000 7f9984b3fb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dfb1a01880 0x55dfb1a138d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:24.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.177+0000 7f9984b3fb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dfb1a06eb0 0x55dfb1a22020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:24.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.177+0000 7f9984b3fb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dfb1a077d0 0x55dfb1a22560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:24.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.177+0000 7f9984b3fb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f99741b1490 con 0x55dfb1a077d0 2024-05-21T22:01:24.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.177+0000 7f9984b3fb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f99741b0dc0 con 0x55dfb1a06eb0 2024-05-21T22:01:24.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.177+0000 7f9984b3fb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f99741adfc0 con 0x55dfb1a01880 2024-05-21T22:01:24.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.178+0000 7f9984b3fb80 10 client.0 _ll_register_callbacks cb 0x55dfb18f9c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:01:24.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.178+0000 7f9973e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dfb1a077d0 0x55dfb1a22560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:24.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.178+0000 7f9972a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dfb1a06eb0 0x55dfb1a22020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:24.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.178+0000 7f9972a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dfb1a06eb0 0x55dfb1a22020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:57858/0 (socket says 172.21.15.151:57858) 2024-05-21T22:01:24.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.178+0000 7f9973400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dfb1a01880 0x55dfb1a138d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:24.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.178+0000 7f9972a00700 1 -- 172.21.15.151:0/717592388 learned_addr learned my addr 172.21.15.151:0/717592388 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:01:24.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.178+0000 7f9973400700 1 -- 172.21.15.151:0/717592388 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dfb1a06eb0 msgr2=0x55dfb1a22020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:24.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.178+0000 7f9973400700 1 --2- 172.21.15.151:0/717592388 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dfb1a06eb0 0x55dfb1a22020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:24.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.178+0000 7f9973400700 1 -- 172.21.15.151:0/717592388 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dfb1a077d0 msgr2=0x55dfb1a22560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:24.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.178+0000 7f9973400700 1 --2- 172.21.15.151:0/717592388 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dfb1a077d0 0x55dfb1a22560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:24.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.178+0000 7f9973400700 1 -- 172.21.15.151:0/717592388 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfb1a22aa0 con 0x55dfb1a01880 2024-05-21T22:01:24.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.178+0000 7f9972a00700 1 --2- 172.21.15.151:0/717592388 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dfb1a06eb0 0x55dfb1a22020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:01:24.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.178+0000 7f9973e00700 1 --2- 172.21.15.151:0/717592388 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dfb1a077d0 0x55dfb1a22560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:01:24.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.179+0000 7f9973400700 1 --2- 172.21.15.151:0/717592388 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dfb1a01880 0x55dfb1a138d0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f9964005920 tx=0x7f9964019550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=158d44f70520ac28 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:01:24.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.179+0000 7f995d600700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-21T22:01:24.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.179+0000 7f995d600700 1 -- 172.21.15.151:0/717592388 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f996401e040 con 0x55dfb1a01880 2024-05-21T22:01:24.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.179+0000 7f9984b3fb80 1 -- 172.21.15.151:0/717592388 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9960028a20 con 0x55dfb1a01880 2024-05-21T22:01:24.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.179+0000 7f995d600700 1 -- 172.21.15.151:0/717592388 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f9964019c80 con 0x55dfb1a01880 2024-05-21T22:01:24.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.179+0000 7f995d600700 1 -- 172.21.15.151:0/717592388 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f99640050b0 con 0x55dfb1a01880 2024-05-21T22:01:24.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.179+0000 7f9984b3fb80 10 client.26590 fetch_fsmap learned FSMap version 29 2024-05-21T22:01:24.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.179+0000 7f9984b3fb80 1 -- 172.21.15.151:0/717592388 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55dfb1a85670 con 0x55dfb1a01880 2024-05-21T22:01:24.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.180+0000 7f995d600700 1 -- 172.21.15.151:0/717592388 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9960028a20 con 0x55dfb1a01880 2024-05-21T22:01:24.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.180+0000 7f995d600700 1 -- 172.21.15.151:0/717592388 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f99640039f0 con 0x55dfb1a01880 2024-05-21T22:01:24.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.180+0000 7f9984b3fb80 10 client.26590 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:01:24.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.180+0000 7f9984b3fb80 10 client.26590 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-21T22:01:24.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.180+0000 7f9984b3fb80 20 client.26590 populate_metadata read hostname 'smithi151' 2024-05-21T22:01:24.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.180+0000 7f9984b3fb80 1 --2- 172.21.15.151:0/717592388 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x55dfb1a85b20 0x55dfb1a87f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:24.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.180+0000 7f9984b3fb80 4 client.26590 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T22:01:24.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.180+0000 7f9984b3fb80 1 -- 172.21.15.151:0/717592388 --> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dfb195b570 con 0x55dfb1a85b20 2024-05-21T22:01:24.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.180+0000 7f9972a00700 1 --2- 172.21.15.151:0/717592388 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x55dfb1a85b20 0x55dfb1a87f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:24.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.181+0000 7f9972a00700 1 --2- 172.21.15.151:0/717592388 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x55dfb1a85b20 0x55dfb1a87f10 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=73fbab5e4f72a408 server_cookie=55454ac39b4b0513 in_seq=0 out_seq=0 2024-05-21T22:01:24.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.181+0000 7f995d600700 10 client.26590 ms_handle_connect on v2:172.21.15.184:6838/3059935143 2024-05-21T22:01:24.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.181+0000 7f995d600700 1 -- 172.21.15.151:0/717592388 <== mds.1 v2:172.21.15.184:6838/3059935143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dfb195b570 con 0x55dfb1a85b20 2024-05-21T22:01:24.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.181+0000 7f995d600700 10 client.26590 handle_command_reply: tid=0 2024-05-21T22:01:24.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.181+0000 7f9984b3fb80 1 client.26590 shutdown 2024-05-21T22:01:24.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.181+0000 7f9984b3fb80 1 -- 172.21.15.151:0/717592388 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dfb1a01880 msgr2=0x55dfb1a138d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:24.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.181+0000 7f9984b3fb80 1 --2- 172.21.15.151:0/717592388 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dfb1a01880 0x55dfb1a138d0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f9964005920 tx=0x7f9964019550 comp rx=0 tx=0).stop 2024-05-21T22:01:24.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.181+0000 7f9984b3fb80 1 -- 172.21.15.151:0/717592388 shutdown_connections 2024-05-21T22:01:24.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.181+0000 7f9984b3fb80 1 --2- 172.21.15.151:0/717592388 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x55dfb1a85b20 0x55dfb1a87f10 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:24.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.181+0000 7f9984b3fb80 1 --2- 172.21.15.151:0/717592388 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dfb1a077d0 0x55dfb1a22560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:24.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.181+0000 7f995d600700 0 client.26590 ms_handle_reset on v2:172.21.15.184:6838/3059935143 2024-05-21T22:01:24.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.181+0000 7f9984b3fb80 1 --2- 172.21.15.151:0/717592388 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55dfb1a01880 0x55dfb1a138d0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:24.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.181+0000 7f9984b3fb80 1 --2- 172.21.15.151:0/717592388 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55dfb1a06eb0 0x55dfb1a22020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:24.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.181+0000 7f9984b3fb80 1 -- 172.21.15.151:0/717592388 >> 172.21.15.151:0/717592388 conn(0x55dfb192fbe0 msgr2=0x55dfb1a049c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:01:24.193 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.182+0000 7f9984b3fb80 1 -- 172.21.15.151:0/717592388 shutdown_connections 2024-05-21T22:01:24.193 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.182+0000 7f9984b3fb80 1 -- 172.21.15.151:0/717592388 wait complete. 2024-05-21T22:01:24.193 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.182+0000 7f9984b3fb80 20 client.26590 trim_cache size 0 max 16384 2024-05-21T22:01:24.193 INFO:teuthology.orchestra.run.smithi151.stdout:[] 2024-05-21T22:01:24.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.182+0000 7f997a200700 1 -- 172.21.15.151:0/115555456 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f99580b29f0 msgr2=0x7f99580b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:24.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.182+0000 7f997a200700 1 --2- 172.21.15.151:0/115555456 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f99580b29f0 0x7f99580b4eb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f997410ad60 tx=0x7f9960005790 comp rx=0 tx=0).stop 2024-05-21T22:01:24.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.182+0000 7f997a200700 1 -- 172.21.15.151:0/115555456 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f997410b760 msgr2=0x7f99741a0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:24.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.182+0000 7f997a200700 1 --2- 172.21.15.151:0/115555456 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f997410b760 0x7f99741a0720 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f996c00efc0 tx=0x7f996c00c3a0 comp rx=0 tx=0).stop 2024-05-21T22:01:24.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.182+0000 7f997a200700 1 -- 172.21.15.151:0/115555456 shutdown_connections 2024-05-21T22:01:24.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.182+0000 7f997a200700 1 --2- 172.21.15.151:0/115555456 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f99580b29f0 0x7f99580b4eb0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:24.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.182+0000 7f997a200700 1 --2- 172.21.15.151:0/115555456 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f997410b760 0x7f99741a0720 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:24.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.182+0000 7f997a200700 1 --2- 172.21.15.151:0/115555456 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9974105760 0x7f997419bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:24.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.182+0000 7f997a200700 1 --2- 172.21.15.151:0/115555456 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9974106120 0x7f997419c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:24.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.182+0000 7f997a200700 1 -- 172.21.15.151:0/115555456 >> 172.21.15.151:0/115555456 conn(0x7f9974101250 msgr2=0x7f9974101870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:01:24.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.183+0000 7f997a200700 1 -- 172.21.15.151:0/115555456 shutdown_connections 2024-05-21T22:01:24.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:24.183+0000 7f997a200700 1 -- 172.21.15.151:0/115555456 wait complete. 2024-05-21T22:01:25.038 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-21T22:01:25.483 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:01:25.828 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:01:25 smithi151 ceph-mon[31623]: pgmap v177: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:01:25.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:01:25 smithi184 ceph-mon[36942]: pgmap v177: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:01:26.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:01:25 smithi164 ceph-mon[36269]: pgmap v177: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:01:27.626 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.624+0000 7f8b15800700 1 -- 172.21.15.151:0/826816486 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8b10075770 msgr2=0x7f8b10075b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:27.626 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.624+0000 7f8b15800700 1 --2- 172.21.15.151:0/826816486 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8b10075770 0x7f8b10075b50 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f8b00009990 tx=0x7f8b00009ca0 comp rx=0 tx=0).stop 2024-05-21T22:01:27.626 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.625+0000 7f8b15800700 1 -- 172.21.15.151:0/826816486 shutdown_connections 2024-05-21T22:01:27.626 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.625+0000 7f8b15800700 1 --2- 172.21.15.151:0/826816486 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b1010ff40 0x7f8b10112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:27.626 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.625+0000 7f8b15800700 1 --2- 172.21.15.151:0/826816486 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8b10075770 0x7f8b10075b50 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:27.626 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.625+0000 7f8b15800700 1 --2- 172.21.15.151:0/826816486 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8b10076090 0x7f8b1010fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:27.626 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.625+0000 7f8b15800700 1 -- 172.21.15.151:0/826816486 >> 172.21.15.151:0/826816486 conn(0x7f8b1007a6a0 msgr2=0x7f8b1007aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:01:27.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.625+0000 7f8b15800700 1 -- 172.21.15.151:0/826816486 shutdown_connections 2024-05-21T22:01:27.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.625+0000 7f8b15800700 1 -- 172.21.15.151:0/826816486 wait complete. 2024-05-21T22:01:27.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.626+0000 7f8b15800700 1 Processor -- start 2024-05-21T22:01:27.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.626+0000 7f8b15800700 1 -- start start 2024-05-21T22:01:27.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.626+0000 7f8b15800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8b10075770 0x7f8b1019be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:27.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.626+0000 7f8b15800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8b10076090 0x7f8b1019c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:27.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.626+0000 7f8b15800700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b1010ff40 0x7f8b101a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:27.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.626+0000 7f8b15800700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8b10115940 con 0x7f8b10075770 2024-05-21T22:01:27.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.626+0000 7f8b15800700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8b101157c0 con 0x7f8b10076090 2024-05-21T22:01:27.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.626+0000 7f8b15800700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8b10115ac0 con 0x7f8b1010ff40 2024-05-21T22:01:27.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.627+0000 7f8b0ea00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8b10075770 0x7f8b1019be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:27.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.627+0000 7f8b0ea00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8b10075770 0x7f8b1019be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:45154/0 (socket says 172.21.15.151:45154) 2024-05-21T22:01:27.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.627+0000 7f8b0ea00700 1 -- 172.21.15.151:0/3745516339 learned_addr learned my addr 172.21.15.151:0/3745516339 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:01:27.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.627+0000 7f8b0f400700 1 --2- 172.21.15.151:0/3745516339 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b1010ff40 0x7f8b101a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:27.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.627+0000 7f8b0e000700 1 --2- 172.21.15.151:0/3745516339 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8b10076090 0x7f8b1019c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:27.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.627+0000 7f8b0ea00700 1 -- 172.21.15.151:0/3745516339 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b1010ff40 msgr2=0x7f8b101a06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:27.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.627+0000 7f8b0ea00700 1 --2- 172.21.15.151:0/3745516339 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b1010ff40 0x7f8b101a06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:27.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.627+0000 7f8b0ea00700 1 -- 172.21.15.151:0/3745516339 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8b10076090 msgr2=0x7f8b1019c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:27.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.627+0000 7f8b0ea00700 1 --2- 172.21.15.151:0/3745516339 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8b10076090 0x7f8b1019c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:27.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.627+0000 7f8b0ea00700 1 -- 172.21.15.151:0/3745516339 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b101a0e10 con 0x7f8b10075770 2024-05-21T22:01:27.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.627+0000 7f8b0ea00700 1 --2- 172.21.15.151:0/3745516339 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8b10075770 0x7f8b1019be50 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f8b00000c00 tx=0x7f8b000057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:01:27.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.628+0000 7f8b07400700 1 -- 172.21.15.151:0/3745516339 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b00027070 con 0x7f8b10075770 2024-05-21T22:01:27.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.628+0000 7f8b07400700 1 -- 172.21.15.151:0/3745516339 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b00022470 con 0x7f8b10075770 2024-05-21T22:01:27.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.628+0000 7f8b15800700 1 -- 172.21.15.151:0/3745516339 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b101a10a0 con 0x7f8b10075770 2024-05-21T22:01:27.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.628+0000 7f8b07400700 1 -- 172.21.15.151:0/3745516339 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b00017650 con 0x7f8b10075770 2024-05-21T22:01:27.630 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.628+0000 7f8b15800700 1 -- 172.21.15.151:0/3745516339 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b101a8980 con 0x7f8b10075770 2024-05-21T22:01:27.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.632+0000 7f8b07400700 1 -- 172.21.15.151:0/3745516339 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7f8b0002a070 con 0x7f8b10075770 2024-05-21T22:01:27.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.633+0000 7f8b2013fb80 1 Processor -- start 2024-05-21T22:01:27.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.633+0000 7f8b2013fb80 1 -- start start 2024-05-21T22:01:27.634 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.633+0000 7f8b2013fb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5577e4cfbeb0 0x5577e4cfc290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:27.634 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.633+0000 7f8b07400700 1 --2- 172.21.15.151:0/3745516339 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f8af40b29f0 0x7f8af40b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:27.634 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.633+0000 7f8b2013fb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5577e4cfc7d0 0x5577e4cf6260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:27.634 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.633+0000 7f8b2013fb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577e4cf6880 0x5577e4cf6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:27.634 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.633+0000 7f8b2013fb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5577e4c54e40 con 0x5577e4cf6880 2024-05-21T22:01:27.634 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.633+0000 7f8b2013fb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5577e4c449e0 con 0x5577e4cfbeb0 2024-05-21T22:01:27.634 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.633+0000 7f8b2013fb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5577e4c48a20 con 0x5577e4cfc7d0 2024-05-21T22:01:27.634 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.634+0000 7f8b07400700 1 -- 172.21.15.151:0/3745516339 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f8b000e3350 con 0x7f8b10075770 2024-05-21T22:01:27.634 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.634+0000 7f8b0e000700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5577e4cfc7d0 0x5577e4cf6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:27.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.634+0000 7f8b0e000700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5577e4cfc7d0 0x5577e4cf6260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:46902/0 (socket says 172.21.15.151:46902) 2024-05-21T22:01:27.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.634+0000 7f8b0f400700 1 --2- 172.21.15.151:0/3745516339 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f8af40b29f0 0x7f8af40b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:27.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.634+0000 7f8b0e000700 1 -- 172.21.15.151:0/1023027512 learned_addr learned my addr 172.21.15.151:0/1023027512 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:01:27.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.634+0000 7f8b0ea00700 1 --2- 172.21.15.151:0/1023027512 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5577e4cfbeb0 0x5577e4cfc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:27.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.634+0000 7f8b0f400700 1 --2- 172.21.15.151:0/1023027512 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577e4cf6880 0x5577e4cf6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:27.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.634+0000 7f8b0e000700 1 -- 172.21.15.151:0/1023027512 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5577e4cfbeb0 msgr2=0x5577e4cfc290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:27.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.634+0000 7f8b0e000700 1 --2- 172.21.15.151:0/1023027512 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5577e4cfbeb0 0x5577e4cfc290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:27.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.634+0000 7f8b0e000700 1 -- 172.21.15.151:0/1023027512 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577e4cf6880 msgr2=0x5577e4cf6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:27.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.634+0000 7f8b0e000700 1 --2- 172.21.15.151:0/1023027512 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577e4cf6880 0x5577e4cf6ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:27.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.634+0000 7f8b0e000700 1 -- 172.21.15.151:0/1023027512 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577e4c0d420 con 0x5577e4cfc7d0 2024-05-21T22:01:27.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.634+0000 7f8b0f400700 1 --2- 172.21.15.151:0/1023027512 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577e4cf6880 0x5577e4cf6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T22:01:27.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.635+0000 7f8b0e000700 1 --2- 172.21.15.151:0/1023027512 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5577e4cfc7d0 0x5577e4cf6260 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8afc009cf0 tx=0x7f8afc011440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d42f233774b1f86 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:01:27.637 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.635+0000 7f8b0f400700 1 --2- 172.21.15.151:0/3745516339 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f8af40b29f0 0x7f8af40b4eb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8b08009ec0 tx=0x7f8b08009920 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:01:27.637 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.635+0000 7f8afbe00700 1 -- 172.21.15.151:0/1023027512 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8afc011e30 con 0x5577e4cfc7d0 2024-05-21T22:01:27.637 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.635+0000 7f8afbe00700 1 -- 172.21.15.151:0/1023027512 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8afc004ce0 con 0x5577e4cfc7d0 2024-05-21T22:01:27.637 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.635+0000 7f8afbe00700 1 -- 172.21.15.151:0/1023027512 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8afc019690 con 0x5577e4cfc7d0 2024-05-21T22:01:27.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.636+0000 7f8b2013fb80 1 -- 172.21.15.151:0/1023027512 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5577e4cfc7d0 msgr2=0x5577e4cf6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:27.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.636+0000 7f8b2013fb80 1 --2- 172.21.15.151:0/1023027512 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5577e4cfc7d0 0x5577e4cf6260 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8afc009cf0 tx=0x7f8afc011440 comp rx=0 tx=0).stop 2024-05-21T22:01:27.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.636+0000 7f8b2013fb80 1 -- 172.21.15.151:0/1023027512 shutdown_connections 2024-05-21T22:01:27.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.636+0000 7f8b2013fb80 1 --2- 172.21.15.151:0/1023027512 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5577e4cfbeb0 0x5577e4cfc290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:27.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.636+0000 7f8b2013fb80 1 --2- 172.21.15.151:0/1023027512 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577e4cf6880 0x5577e4cf6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:27.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.636+0000 7f8b2013fb80 1 --2- 172.21.15.151:0/1023027512 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5577e4cfc7d0 0x5577e4cf6260 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:27.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.636+0000 7f8b2013fb80 1 -- 172.21.15.151:0/1023027512 >> 172.21.15.151:0/1023027512 conn(0x5577e4c24be0 msgr2=0x5577e4cf1010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:01:27.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.636+0000 7f8b2013fb80 1 -- 172.21.15.151:0/1023027512 shutdown_connections 2024-05-21T22:01:27.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.637+0000 7f8b2013fb80 1 -- 172.21.15.151:0/1023027512 wait complete. 2024-05-21T22:01:27.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.637+0000 7f8b2013fb80 1 -- start start 2024-05-21T22:01:27.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.637+0000 7f8b2013fb80 1 Processor -- start 2024-05-21T22:01:27.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.638+0000 7f8b2013fb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5577e4cf6880 0x5577e4d02450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:27.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.638+0000 7f8b2013fb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577e4cfbeb0 0x5577e4cff7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:27.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.638+0000 7f8b2013fb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5577e4cfc7d0 0x5577e4cffd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:27.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.638+0000 7f8b2013fb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5577e4b4afa0 con 0x5577e4cfbeb0 2024-05-21T22:01:27.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.638+0000 7f8b2013fb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5577e4be5760 con 0x5577e4cf6880 2024-05-21T22:01:27.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.638+0000 7f8b2013fb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5577e4d03240 con 0x5577e4cfc7d0 2024-05-21T22:01:27.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.638+0000 7f8b2013fb80 10 client.0 _ll_register_callbacks cb 0x5577e4beec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:01:27.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.638+0000 7f8b0e000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577e4cfbeb0 0x5577e4cff7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:27.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.638+0000 7f8b0e000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577e4cfbeb0 0x5577e4cff7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:45164/0 (socket says 172.21.15.151:45164) 2024-05-21T22:01:27.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.638+0000 7f8b0e000700 1 -- 172.21.15.151:0/2760614472 learned_addr learned my addr 172.21.15.151:0/2760614472 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:01:27.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.638+0000 7f8b0ea00700 1 --2- 172.21.15.151:0/2760614472 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5577e4cf6880 0x5577e4d02450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:27.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.638+0000 7f8b0f400700 1 --2- 172.21.15.151:0/2760614472 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5577e4cfc7d0 0x5577e4cffd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:27.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.639+0000 7f8b0ea00700 1 -- 172.21.15.151:0/2760614472 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5577e4cfc7d0 msgr2=0x5577e4cffd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:27.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.639+0000 7f8b0ea00700 1 --2- 172.21.15.151:0/2760614472 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5577e4cfc7d0 0x5577e4cffd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:27.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.639+0000 7f8b0ea00700 1 -- 172.21.15.151:0/2760614472 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577e4cfbeb0 msgr2=0x5577e4cff7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:27.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.639+0000 7f8b0ea00700 1 --2- 172.21.15.151:0/2760614472 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577e4cfbeb0 0x5577e4cff7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:27.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.639+0000 7f8b0ea00700 1 -- 172.21.15.151:0/2760614472 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577e4b948d0 con 0x5577e4cf6880 2024-05-21T22:01:27.643 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.639+0000 7f8b0ea00700 1 --2- 172.21.15.151:0/2760614472 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5577e4cf6880 0x5577e4d02450 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f8b000e0a50 tx=0x7f8b000a9fc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9bb545cac5517a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:01:27.643 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.639+0000 7f8af8c00700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-21T22:01:27.643 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.639+0000 7f8af8c00700 1 -- 172.21.15.151:0/2760614472 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b000afbb0 con 0x5577e4cf6880 2024-05-21T22:01:27.643 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.639+0000 7f8af8c00700 1 -- 172.21.15.151:0/2760614472 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f8b000e0cd0 con 0x5577e4cf6880 2024-05-21T22:01:27.644 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.639+0000 7f8b2013fb80 1 -- 172.21.15.151:0/2760614472 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8afc004ce0 con 0x5577e4cf6880 2024-05-21T22:01:27.644 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.640+0000 7f8af8c00700 1 -- 172.21.15.151:0/2760614472 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8b000ab7a0 con 0x5577e4cf6880 2024-05-21T22:01:27.644 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.640+0000 7f8b2013fb80 10 client.26588 fetch_fsmap learned FSMap version 29 2024-05-21T22:01:27.644 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.640+0000 7f8b2013fb80 1 -- 172.21.15.151:0/2760614472 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5577e4c15060 con 0x5577e4cf6880 2024-05-21T22:01:27.644 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.640+0000 7f8af8c00700 1 -- 172.21.15.151:0/2760614472 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8afc004ce0 con 0x5577e4cf6880 2024-05-21T22:01:27.644 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.640+0000 7f8af8c00700 1 -- 172.21.15.151:0/2760614472 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f8b000aa8d0 con 0x5577e4cf6880 2024-05-21T22:01:27.644 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.640+0000 7f8b2013fb80 10 client.26588 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:01:27.644 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.640+0000 7f8b2013fb80 10 client.26588 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T22:01:27.644 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.640+0000 7f8b2013fb80 20 client.26588 populate_metadata read hostname 'smithi151' 2024-05-21T22:01:27.644 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.640+0000 7f8b2013fb80 1 --2- 172.21.15.151:0/2760614472 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5577e4d0fcf0 0x5577e4d12170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:27.644 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.640+0000 7f8b2013fb80 4 client.26588 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T22:01:27.644 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.640+0000 7f8b2013fb80 1 -- 172.21.15.151:0/2760614472 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5577e4c56d90 con 0x5577e4d0fcf0 2024-05-21T22:01:27.644 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.641+0000 7f8b0e000700 1 --2- 172.21.15.151:0/2760614472 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5577e4d0fcf0 0x5577e4d12170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:27.645 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.641+0000 7f8b0e000700 1 --2- 172.21.15.151:0/2760614472 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5577e4d0fcf0 0x5577e4d12170 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9732dad3bab023eb server_cookie=f30ea089be64d158 in_seq=0 out_seq=0 2024-05-21T22:01:27.645 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.641+0000 7f8af8c00700 10 client.26588 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T22:01:27.645 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.642+0000 7f8af8c00700 1 -- 172.21.15.151:0/2760614472 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5577e4c56d90 con 0x5577e4d0fcf0 2024-05-21T22:01:27.645 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.642+0000 7f8af8c00700 10 client.26588 handle_command_reply: tid=0 2024-05-21T22:01:27.645 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.642+0000 7f8b2013fb80 1 client.26588 shutdown 2024-05-21T22:01:27.645 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.642+0000 7f8b2013fb80 1 -- 172.21.15.151:0/2760614472 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5577e4cf6880 msgr2=0x5577e4d02450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:27.645 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.642+0000 7f8b2013fb80 1 --2- 172.21.15.151:0/2760614472 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5577e4cf6880 0x5577e4d02450 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f8b000e0a50 tx=0x7f8b000a9fc0 comp rx=0 tx=0).stop 2024-05-21T22:01:27.645 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.642+0000 7f8b2013fb80 1 -- 172.21.15.151:0/2760614472 shutdown_connections 2024-05-21T22:01:27.645 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.642+0000 7f8b2013fb80 1 --2- 172.21.15.151:0/2760614472 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5577e4d0fcf0 0x5577e4d12170 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:27.645 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.642+0000 7f8b2013fb80 1 --2- 172.21.15.151:0/2760614472 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5577e4cf6880 0x5577e4d02450 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:27.645 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.642+0000 7f8b2013fb80 1 --2- 172.21.15.151:0/2760614472 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5577e4cfc7d0 0x5577e4cffd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:27.645 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.642+0000 7f8b2013fb80 1 --2- 172.21.15.151:0/2760614472 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577e4cfbeb0 0x5577e4cff7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:27.645 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.642+0000 7f8af8c00700 0 client.26588 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T22:01:27.645 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.642+0000 7f8b2013fb80 1 -- 172.21.15.151:0/2760614472 >> 172.21.15.151:0/2760614472 conn(0x5577e4c24be0 msgr2=0x5577e4cf5250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:01:27.646 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.643+0000 7f8b2013fb80 1 -- 172.21.15.151:0/2760614472 shutdown_connections 2024-05-21T22:01:27.646 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.643+0000 7f8b2013fb80 1 -- 172.21.15.151:0/2760614472 wait complete. 2024-05-21T22:01:27.646 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.643+0000 7f8b2013fb80 20 client.26588 trim_cache size 0 max 16384 2024-05-21T22:01:27.660 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.660+0000 7f8b2013fb80 1 Processor -- start 2024-05-21T22:01:27.660 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.660+0000 7f8b2013fb80 1 -- start start 2024-05-21T22:01:27.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.660+0000 7f8b2013fb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5577e4cf6880 0x5577e4cf6260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:27.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.660+0000 7f8b2013fb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5577e4cfbeb0 0x5577e4cf09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:27.662 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.660+0000 7f8b2013fb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577e4cfc7d0 0x5577e4cf0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:27.662 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.660+0000 7f8b2013fb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8b101afe90 con 0x5577e4cfc7d0 2024-05-21T22:01:27.662 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.660+0000 7f8b2013fb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8b101b19d0 con 0x5577e4cf6880 2024-05-21T22:01:27.662 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.660+0000 7f8b2013fb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8b101b0560 con 0x5577e4cfbeb0 2024-05-21T22:01:27.662 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.661+0000 7f8b0f400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577e4cfc7d0 0x5577e4cf0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:27.662 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.661+0000 7f8b0ea00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5577e4cf6880 0x5577e4cf6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:27.662 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.661+0000 7f8b0ea00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5577e4cf6880 0x5577e4cf6260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:44282/0 (socket says 172.21.15.151:44282) 2024-05-21T22:01:27.662 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.661+0000 7f8b0f400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577e4cfc7d0 0x5577e4cf0f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:45166/0 (socket says 172.21.15.151:45166) 2024-05-21T22:01:27.662 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.661+0000 7f8b0ea00700 1 -- 172.21.15.151:0/2596721060 learned_addr learned my addr 172.21.15.151:0/2596721060 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:01:27.662 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.661+0000 7f8b0e000700 1 --2- 172.21.15.151:0/2596721060 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5577e4cfbeb0 0x5577e4cf09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:27.662 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.661+0000 7f8b0ea00700 1 -- 172.21.15.151:0/2596721060 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5577e4cfbeb0 msgr2=0x5577e4cf09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:27.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.661+0000 7f8b0ea00700 1 --2- 172.21.15.151:0/2596721060 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5577e4cfbeb0 0x5577e4cf09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:27.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.661+0000 7f8b0ea00700 1 -- 172.21.15.151:0/2596721060 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577e4cfc7d0 msgr2=0x5577e4cf0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:27.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.661+0000 7f8b0ea00700 1 --2- 172.21.15.151:0/2596721060 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577e4cfc7d0 0x5577e4cf0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:27.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.661+0000 7f8b0ea00700 1 -- 172.21.15.151:0/2596721060 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577e4cfb9b0 con 0x5577e4cf6880 2024-05-21T22:01:27.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.662+0000 7f8b0ea00700 1 --2- 172.21.15.151:0/2596721060 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5577e4cf6880 0x5577e4cf6260 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8b000afdc0 tx=0x7f8b00009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70a9bb346bddb67c server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:01:27.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.662+0000 7f8afb400700 1 -- 172.21.15.151:0/2596721060 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b000aa180 con 0x5577e4cf6880 2024-05-21T22:01:27.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.662+0000 7f8afb400700 1 -- 172.21.15.151:0/2596721060 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b0006d050 con 0x5577e4cf6880 2024-05-21T22:01:27.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.662+0000 7f8afb400700 1 -- 172.21.15.151:0/2596721060 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b000aa670 con 0x5577e4cf6880 2024-05-21T22:01:27.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.663+0000 7f8b2013fb80 1 -- 172.21.15.151:0/2596721060 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5577e4cf6880 msgr2=0x5577e4cf6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:27.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.663+0000 7f8b2013fb80 1 --2- 172.21.15.151:0/2596721060 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5577e4cf6880 0x5577e4cf6260 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8b000afdc0 tx=0x7f8b00009ca0 comp rx=0 tx=0).stop 2024-05-21T22:01:27.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.663+0000 7f8b2013fb80 1 -- 172.21.15.151:0/2596721060 shutdown_connections 2024-05-21T22:01:27.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.663+0000 7f8b2013fb80 1 --2- 172.21.15.151:0/2596721060 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5577e4cf6880 0x5577e4cf6260 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:27.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.663+0000 7f8b2013fb80 1 --2- 172.21.15.151:0/2596721060 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577e4cfc7d0 0x5577e4cf0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:27.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.663+0000 7f8b2013fb80 1 --2- 172.21.15.151:0/2596721060 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5577e4cfbeb0 0x5577e4cf09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:27.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.663+0000 7f8b2013fb80 1 -- 172.21.15.151:0/2596721060 >> 172.21.15.151:0/2596721060 conn(0x5577e4c24be0 msgr2=0x5577e4cf99c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:01:27.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.663+0000 7f8b2013fb80 1 -- 172.21.15.151:0/2596721060 shutdown_connections 2024-05-21T22:01:27.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.663+0000 7f8b2013fb80 1 -- 172.21.15.151:0/2596721060 wait complete. 2024-05-21T22:01:27.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.663+0000 7f8b2013fb80 1 -- start start 2024-05-21T22:01:27.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.664+0000 7f8b2013fb80 1 Processor -- start 2024-05-21T22:01:27.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.664+0000 7f8b2013fb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577e4cf6880 0x5577e4d08d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:27.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.664+0000 7f8b2013fb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5577e4cfbeb0 0x5577e4d17430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:27.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.664+0000 7f8b2013fb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5577e4cfc7d0 0x5577e4d17b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:27.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.664+0000 7f8b2013fb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8b101b1300 con 0x5577e4cf6880 2024-05-21T22:01:27.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.664+0000 7f8b2013fb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8b101b0c30 con 0x5577e4cfc7d0 2024-05-21T22:01:27.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.664+0000 7f8b2013fb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8b101ade30 con 0x5577e4cfbeb0 2024-05-21T22:01:27.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.664+0000 7f8b2013fb80 10 client.0 _ll_register_callbacks cb 0x5577e4beec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:01:27.667 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.664+0000 7f8b0ea00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577e4cf6880 0x5577e4d08d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:27.667 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.664+0000 7f8b0ea00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577e4cf6880 0x5577e4d08d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:45180/0 (socket says 172.21.15.151:45180) 2024-05-21T22:01:27.667 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.664+0000 7f8b0ea00700 1 -- 172.21.15.151:0/1890364711 learned_addr learned my addr 172.21.15.151:0/1890364711 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:01:27.667 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.665+0000 7f8b0e000700 1 --2- 172.21.15.151:0/1890364711 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5577e4cfbeb0 0x5577e4d17430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:27.667 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.665+0000 7f8b0f400700 1 --2- 172.21.15.151:0/1890364711 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5577e4cfc7d0 0x5577e4d17b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:27.667 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.665+0000 7f8b0ea00700 1 -- 172.21.15.151:0/1890364711 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5577e4cfbeb0 msgr2=0x5577e4d17430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:27.667 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.665+0000 7f8b0ea00700 1 --2- 172.21.15.151:0/1890364711 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5577e4cfbeb0 0x5577e4d17430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:27.667 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.665+0000 7f8b0ea00700 1 -- 172.21.15.151:0/1890364711 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5577e4cfc7d0 msgr2=0x5577e4d17b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:27.667 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.665+0000 7f8b0ea00700 1 --2- 172.21.15.151:0/1890364711 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5577e4cfc7d0 0x5577e4d17b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:27.667 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.665+0000 7f8b0ea00700 1 -- 172.21.15.151:0/1890364711 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577e4d79910 con 0x5577e4cf6880 2024-05-21T22:01:27.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.665+0000 7f8b0ea00700 1 --2- 172.21.15.151:0/1890364711 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577e4cf6880 0x5577e4d08d40 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f8b00022c70 tx=0x7f8b000aa4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feab51c8486314d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:01:27.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.665+0000 7f8af8c00700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-21T22:01:27.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.665+0000 7f8af8c00700 1 -- 172.21.15.151:0/1890364711 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b00089060 con 0x5577e4cf6880 2024-05-21T22:01:27.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.666+0000 7f8af8c00700 1 -- 172.21.15.151:0/1890364711 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f8b000a6c60 con 0x5577e4cf6880 2024-05-21T22:01:27.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.666+0000 7f8b2013fb80 1 -- 172.21.15.151:0/1890364711 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b0006d050 con 0x5577e4cf6880 2024-05-21T22:01:27.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.666+0000 7f8af8c00700 1 -- 172.21.15.151:0/1890364711 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8b000aa180 con 0x5577e4cf6880 2024-05-21T22:01:27.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.666+0000 7f8b2013fb80 10 client.16470 fetch_fsmap learned FSMap version 29 2024-05-21T22:01:27.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.666+0000 7f8b2013fb80 1 -- 172.21.15.151:0/1890364711 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5577e4d79910 con 0x5577e4cf6880 2024-05-21T22:01:27.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.666+0000 7f8af8c00700 1 -- 172.21.15.151:0/1890364711 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b0006d050 con 0x5577e4cf6880 2024-05-21T22:01:27.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.666+0000 7f8af8c00700 1 -- 172.21.15.151:0/1890364711 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f8b00090b90 con 0x5577e4cf6880 2024-05-21T22:01:27.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.666+0000 7f8b2013fb80 10 client.16470 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:01:27.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.666+0000 7f8b2013fb80 10 client.16470 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T22:01:27.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.666+0000 7f8b2013fb80 20 client.16470 populate_metadata read hostname 'smithi151' 2024-05-21T22:01:27.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.666+0000 7f8b2013fb80 1 --2- 172.21.15.151:0/1890364711 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5577e4d7ae60 0x5577e4d7d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:27.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.666+0000 7f8b2013fb80 4 client.16470 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T22:01:27.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.666+0000 7f8b2013fb80 1 -- 172.21.15.151:0/1890364711 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5577e4c50570 con 0x5577e4d7ae60 2024-05-21T22:01:27.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.667+0000 7f8b0e000700 1 --2- 172.21.15.151:0/1890364711 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5577e4d7ae60 0x5577e4d7d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:27.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.667+0000 7f8b0e000700 1 --2- 172.21.15.151:0/1890364711 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5577e4d7ae60 0x5577e4d7d280 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aec0c44a282d5bd5 server_cookie=e982c32a69aeeb72 in_seq=0 out_seq=0 2024-05-21T22:01:27.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.667+0000 7f8af8c00700 10 client.16470 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T22:01:27.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.668+0000 7f8af8c00700 1 -- 172.21.15.151:0/1890364711 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5577e4c50570 con 0x5577e4d7ae60 2024-05-21T22:01:27.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.668+0000 7f8af8c00700 10 client.16470 handle_command_reply: tid=0 2024-05-21T22:01:27.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.668+0000 7f8b2013fb80 1 client.16470 shutdown 2024-05-21T22:01:27.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.668+0000 7f8b2013fb80 1 -- 172.21.15.151:0/1890364711 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577e4cf6880 msgr2=0x5577e4d08d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:27.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.668+0000 7f8b2013fb80 1 --2- 172.21.15.151:0/1890364711 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577e4cf6880 0x5577e4d08d40 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f8b00022c70 tx=0x7f8b000aa4f0 comp rx=0 tx=0).stop 2024-05-21T22:01:27.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.668+0000 7f8b2013fb80 1 -- 172.21.15.151:0/1890364711 shutdown_connections 2024-05-21T22:01:27.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.668+0000 7f8b2013fb80 1 --2- 172.21.15.151:0/1890364711 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5577e4d7ae60 0x5577e4d7d280 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:27.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.668+0000 7f8b2013fb80 1 --2- 172.21.15.151:0/1890364711 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5577e4cfc7d0 0x5577e4d17b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:27.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.668+0000 7f8b2013fb80 1 --2- 172.21.15.151:0/1890364711 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577e4cf6880 0x5577e4d08d40 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:27.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.668+0000 7f8b2013fb80 1 --2- 172.21.15.151:0/1890364711 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5577e4cfbeb0 0x5577e4d17430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:27.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.668+0000 7f8af8c00700 0 client.16470 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T22:01:27.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.668+0000 7f8b2013fb80 1 -- 172.21.15.151:0/1890364711 >> 172.21.15.151:0/1890364711 conn(0x5577e4c24be0 msgr2=0x5577e4cf09e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:01:27.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.668+0000 7f8b2013fb80 1 -- 172.21.15.151:0/1890364711 shutdown_connections 2024-05-21T22:01:27.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.668+0000 7f8b2013fb80 1 -- 172.21.15.151:0/1890364711 wait complete. 2024-05-21T22:01:27.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.668+0000 7f8b2013fb80 20 client.16470 trim_cache size 0 max 16384 2024-05-21T22:01:27.674 INFO:teuthology.orchestra.run.smithi151.stdout:[] 2024-05-21T22:01:27.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.669+0000 7f8b15800700 1 -- 172.21.15.151:0/3745516339 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f8af40b29f0 msgr2=0x7f8af40b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:27.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.669+0000 7f8b15800700 1 --2- 172.21.15.151:0/3745516339 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f8af40b29f0 0x7f8af40b4eb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8b08009ec0 tx=0x7f8b08009920 comp rx=0 tx=0).stop 2024-05-21T22:01:27.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.669+0000 7f8b15800700 1 -- 172.21.15.151:0/3745516339 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8b10075770 msgr2=0x7f8b1019be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:27.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.669+0000 7f8b15800700 1 --2- 172.21.15.151:0/3745516339 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8b10075770 0x7f8b1019be50 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f8b00000c00 tx=0x7f8b000057c0 comp rx=0 tx=0).stop 2024-05-21T22:01:27.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.669+0000 7f8b15800700 1 -- 172.21.15.151:0/3745516339 shutdown_connections 2024-05-21T22:01:27.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.669+0000 7f8b15800700 1 --2- 172.21.15.151:0/3745516339 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f8af40b29f0 0x7f8af40b4eb0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:27.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.669+0000 7f8b15800700 1 --2- 172.21.15.151:0/3745516339 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b1010ff40 0x7f8b101a06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:27.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.669+0000 7f8b15800700 1 --2- 172.21.15.151:0/3745516339 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8b10075770 0x7f8b1019be50 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:27.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.669+0000 7f8b15800700 1 --2- 172.21.15.151:0/3745516339 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8b10076090 0x7f8b1019c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:27.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.669+0000 7f8b15800700 1 -- 172.21.15.151:0/3745516339 >> 172.21.15.151:0/3745516339 conn(0x7f8b1007a6a0 msgr2=0x7f8b101007f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:01:27.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.669+0000 7f8b15800700 1 -- 172.21.15.151:0/3745516339 shutdown_connections 2024-05-21T22:01:27.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:27.669+0000 7f8b15800700 1 -- 172.21.15.151:0/3745516339 wait complete. 2024-05-21T22:01:27.741 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:01:27 smithi151 ceph-mon[31623]: pgmap v178: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:01:27.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:01:27 smithi184 ceph-mon[36942]: pgmap v178: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:01:28.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:01:27 smithi164 ceph-mon[36269]: pgmap v178: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:01:29.627 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-21T22:01:29.628 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-21T22:01:29.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:01:29 smithi151 ceph-mon[31623]: pgmap v179: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T22:01:29.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:01:29 smithi184 ceph-mon[36942]: pgmap v179: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T22:01:30.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:01:29 smithi164 ceph-mon[36269]: pgmap v179: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T22:01:30.090 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:01:31.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:01:31 smithi151 ceph-mon[31623]: pgmap v180: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:01:31.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:01:31 smithi184 ceph-mon[36942]: pgmap v180: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:01:32.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:01:31 smithi164 ceph-mon[36269]: pgmap v180: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:01:32.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.247+0000 7f2915400700 1 -- 172.21.15.151:0/155396351 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f29101037f0 msgr2=0x7f2910103bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:32.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.247+0000 7f2915400700 1 --2- 172.21.15.151:0/155396351 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f29101037f0 0x7f2910103bd0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f290000b0d0 tx=0x7f290000b3e0 comp rx=0 tx=0).stop 2024-05-21T22:01:32.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.247+0000 7f2915400700 1 -- 172.21.15.151:0/155396351 shutdown_connections 2024-05-21T22:01:32.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.247+0000 7f2915400700 1 --2- 172.21.15.151:0/155396351 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f29101037f0 0x7f2910103bd0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:32.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.247+0000 7f2915400700 1 --2- 172.21.15.151:0/155396351 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f29101088f0 0x7f2910110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:32.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.247+0000 7f2915400700 1 --2- 172.21.15.151:0/155396351 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f29101041a0 0x7f29101081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:32.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.247+0000 7f2915400700 1 -- 172.21.15.151:0/155396351 >> 172.21.15.151:0/155396351 conn(0x7f291007a6a0 msgr2=0x7f291007aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:01:32.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.247+0000 7f2915400700 1 -- 172.21.15.151:0/155396351 shutdown_connections 2024-05-21T22:01:32.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.247+0000 7f2915400700 1 -- 172.21.15.151:0/155396351 wait complete. 2024-05-21T22:01:32.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.248+0000 7f2915400700 1 Processor -- start 2024-05-21T22:01:32.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.249+0000 7f2915400700 1 -- start start 2024-05-21T22:01:32.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.249+0000 7f2915400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f29101037f0 0x7f291019be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:32.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.249+0000 7f2915400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f29101041a0 0x7f291019c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:32.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.249+0000 7f2915400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f29101088f0 0x7f29101a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:32.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.249+0000 7f2915400700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2910113710 con 0x7f29101088f0 2024-05-21T22:01:32.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.249+0000 7f2915400700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2910113590 con 0x7f29101041a0 2024-05-21T22:01:32.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.249+0000 7f2915400700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2910113890 con 0x7f29101037f0 2024-05-21T22:01:32.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.249+0000 7f290f200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f29101088f0 0x7f29101a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:32.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.249+0000 7f290e800700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f29101037f0 0x7f291019be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:32.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.249+0000 7f290f200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f29101088f0 0x7f29101a06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:45200/0 (socket says 172.21.15.151:45200) 2024-05-21T22:01:32.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.249+0000 7f290f200700 1 -- 172.21.15.151:0/3932679597 learned_addr learned my addr 172.21.15.151:0/3932679597 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:01:32.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.250+0000 7f2905e00700 1 --2- 172.21.15.151:0/3932679597 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f29101041a0 0x7f291019c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:32.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.250+0000 7f290e800700 1 -- 172.21.15.151:0/3932679597 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f29101041a0 msgr2=0x7f291019c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:32.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.250+0000 7f290e800700 1 --2- 172.21.15.151:0/3932679597 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f29101041a0 0x7f291019c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:32.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.250+0000 7f290e800700 1 -- 172.21.15.151:0/3932679597 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f29101088f0 msgr2=0x7f29101a06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:32.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.250+0000 7f290e800700 1 --2- 172.21.15.151:0/3932679597 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f29101088f0 0x7f29101a06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:32.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.250+0000 7f290e800700 1 -- 172.21.15.151:0/3932679597 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29101a0e30 con 0x7f29101037f0 2024-05-21T22:01:32.253 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.250+0000 7f290e800700 1 --2- 172.21.15.151:0/3932679597 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f29101037f0 0x7f291019be20 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f2900000f80 tx=0x7f2900007a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:01:32.253 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.251+0000 7f2907e00700 1 -- 172.21.15.151:0/3932679597 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f290001f020 con 0x7f29101037f0 2024-05-21T22:01:32.253 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.251+0000 7f2907e00700 1 -- 172.21.15.151:0/3932679597 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2900003c00 con 0x7f29101037f0 2024-05-21T22:01:32.253 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.251+0000 7f2915400700 1 -- 172.21.15.151:0/3932679597 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29101a10c0 con 0x7f29101037f0 2024-05-21T22:01:32.253 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.251+0000 7f2907e00700 1 -- 172.21.15.151:0/3932679597 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f290001b430 con 0x7f29101037f0 2024-05-21T22:01:32.253 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.251+0000 7f2915400700 1 -- 172.21.15.151:0/3932679597 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29101a89a0 con 0x7f29101037f0 2024-05-21T22:01:32.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.254+0000 7f2907e00700 1 -- 172.21.15.151:0/3932679597 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7f2900022020 con 0x7f29101037f0 2024-05-21T22:01:32.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.255+0000 7f291fdb5b80 1 Processor -- start 2024-05-21T22:01:32.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.256+0000 7f291fdb5b80 1 -- start start 2024-05-21T22:01:32.257 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.256+0000 7f2907e00700 1 --2- 172.21.15.151:0/3932679597 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f28f40b29f0 0x7f28f40b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:32.257 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.256+0000 7f291fdb5b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bd8d92690 0x563bd8d92a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:32.257 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.256+0000 7f291fdb5b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563bd8d92fb0 0x563bd8d8cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:32.257 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.256+0000 7f291fdb5b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563bd8d8d1c0 0x563bd8d8d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:32.257 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.256+0000 7f291fdb5b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x563bd8ceaec0 con 0x563bd8d92fb0 2024-05-21T22:01:32.257 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.256+0000 7f291fdb5b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x563bd8cdaa60 con 0x563bd8d92690 2024-05-21T22:01:32.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.256+0000 7f291fdb5b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x563bd8cdeaa0 con 0x563bd8d8d1c0 2024-05-21T22:01:32.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.256+0000 7f2907e00700 1 -- 172.21.15.151:0/3932679597 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f290000f020 con 0x7f29101037f0 2024-05-21T22:01:32.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.256+0000 7f290f200700 1 --2- 172.21.15.151:0/3932679597 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f28f40b29f0 0x7f28f40b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:32.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.257+0000 7f290e800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bd8d92690 0x563bd8d92a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:32.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.257+0000 7f290e800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bd8d92690 0x563bd8d92a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:44328/0 (socket says 172.21.15.151:44328) 2024-05-21T22:01:32.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.257+0000 7f290e800700 1 -- 172.21.15.151:0/2617991533 learned_addr learned my addr 172.21.15.151:0/2617991533 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:01:32.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.257+0000 7f290f200700 1 --2- 172.21.15.151:0/2617991533 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563bd8d8d1c0 0x563bd8d8d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:32.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.257+0000 7f2905e00700 1 --2- 172.21.15.151:0/2617991533 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563bd8d92fb0 0x563bd8d8cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:32.260 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.257+0000 7f290e800700 1 -- 172.21.15.151:0/2617991533 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563bd8d8d1c0 msgr2=0x563bd8d8d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:32.260 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.257+0000 7f290e800700 1 --2- 172.21.15.151:0/2617991533 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563bd8d8d1c0 0x563bd8d8d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:32.260 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.257+0000 7f290e800700 1 -- 172.21.15.151:0/2617991533 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563bd8d92fb0 msgr2=0x563bd8d8cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:32.260 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.257+0000 7f290e800700 1 --2- 172.21.15.151:0/2617991533 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563bd8d92fb0 0x563bd8d8cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:32.260 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.257+0000 7f290e800700 1 -- 172.21.15.151:0/2617991533 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563bd8ca34a0 con 0x563bd8d92690 2024-05-21T22:01:32.260 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.257+0000 7f2905e00700 1 --2- 172.21.15.151:0/2617991533 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563bd8d92fb0 0x563bd8d8cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:01:32.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.257+0000 7f290e800700 1 --2- 172.21.15.151:0/2617991533 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bd8d92690 0x563bd8d92a70 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f29000aa1f0 tx=0x7f2900007ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9af586f06f0a8ac server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:01:32.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.258+0000 7f290f200700 1 --2- 172.21.15.151:0/3932679597 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f28f40b29f0 0x7f28f40b4eb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x563bd8d8db80 tx=0x7f29080099d0 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:01:32.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.258+0000 7f28ffe00700 1 -- 172.21.15.151:0/2617991533 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29000aabf0 con 0x563bd8d92690 2024-05-21T22:01:32.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.258+0000 7f28ffe00700 1 -- 172.21.15.151:0/2617991533 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29000aa4f0 con 0x563bd8d92690 2024-05-21T22:01:32.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.258+0000 7f28ffe00700 1 -- 172.21.15.151:0/2617991533 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29000a6780 con 0x563bd8d92690 2024-05-21T22:01:32.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.259+0000 7f291fdb5b80 1 -- 172.21.15.151:0/2617991533 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bd8d92690 msgr2=0x563bd8d92a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:32.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.259+0000 7f291fdb5b80 1 --2- 172.21.15.151:0/2617991533 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bd8d92690 0x563bd8d92a70 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f29000aa1f0 tx=0x7f2900007ed0 comp rx=0 tx=0).stop 2024-05-21T22:01:32.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.259+0000 7f291fdb5b80 1 -- 172.21.15.151:0/2617991533 shutdown_connections 2024-05-21T22:01:32.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.259+0000 7f291fdb5b80 1 --2- 172.21.15.151:0/2617991533 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bd8d92690 0x563bd8d92a70 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:32.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.259+0000 7f291fdb5b80 1 --2- 172.21.15.151:0/2617991533 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563bd8d8d1c0 0x563bd8d8d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:32.262 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.259+0000 7f291fdb5b80 1 --2- 172.21.15.151:0/2617991533 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563bd8d92fb0 0x563bd8d8cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:32.262 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.259+0000 7f291fdb5b80 1 -- 172.21.15.151:0/2617991533 >> 172.21.15.151:0/2617991533 conn(0x563bd8cbac60 msgr2=0x563bd8d87710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:01:32.262 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.259+0000 7f291fdb5b80 1 -- 172.21.15.151:0/2617991533 shutdown_connections 2024-05-21T22:01:32.262 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.259+0000 7f291fdb5b80 1 -- 172.21.15.151:0/2617991533 wait complete. 2024-05-21T22:01:32.262 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.259+0000 7f291fdb5b80 1 -- start start 2024-05-21T22:01:32.262 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.260+0000 7f291fdb5b80 1 Processor -- start 2024-05-21T22:01:32.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.260+0000 7f291fdb5b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563bd8d8d1c0 0x563bd8d98d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:32.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.260+0000 7f291fdb5b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bd8d92690 0x563bd8d960d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:32.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.260+0000 7f291fdb5b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563bd8d92fb0 0x563bd8d96610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:32.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.260+0000 7f291fdb5b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x563bd8be1020 con 0x563bd8d92fb0 2024-05-21T22:01:32.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.260+0000 7f291fdb5b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x563bd8c7b7e0 con 0x563bd8d92690 2024-05-21T22:01:32.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.260+0000 7f291fdb5b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x563bd8cdaa60 con 0x563bd8d8d1c0 2024-05-21T22:01:32.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.260+0000 7f291fdb5b80 10 client.0 _ll_register_callbacks cb 0x563bd8c84cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:01:32.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.261+0000 7f2905e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bd8d92690 0x563bd8d960d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:32.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.261+0000 7f290e800700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563bd8d8d1c0 0x563bd8d98d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:32.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.261+0000 7f290f200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563bd8d92fb0 0x563bd8d96610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:32.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.261+0000 7f290e800700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563bd8d8d1c0 0x563bd8d98d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:46972/0 (socket says 172.21.15.151:46972) 2024-05-21T22:01:32.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.261+0000 7f2905e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bd8d92690 0x563bd8d960d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:44334/0 (socket says 172.21.15.151:44334) 2024-05-21T22:01:32.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.261+0000 7f290e800700 1 -- 172.21.15.151:0/612297667 learned_addr learned my addr 172.21.15.151:0/612297667 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:01:32.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.261+0000 7f290e800700 1 -- 172.21.15.151:0/612297667 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bd8d92690 msgr2=0x563bd8d960d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:32.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.261+0000 7f290e800700 1 --2- 172.21.15.151:0/612297667 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bd8d92690 0x563bd8d960d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:32.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.261+0000 7f290e800700 1 -- 172.21.15.151:0/612297667 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563bd8d92fb0 msgr2=0x563bd8d96610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:32.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.261+0000 7f290e800700 1 --2- 172.21.15.151:0/612297667 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563bd8d92fb0 0x563bd8d96610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:32.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.261+0000 7f290e800700 1 -- 172.21.15.151:0/612297667 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563bd8c2a950 con 0x563bd8d8d1c0 2024-05-21T22:01:32.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.261+0000 7f290e800700 1 --2- 172.21.15.151:0/612297667 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563bd8d8d1c0 0x563bd8d98d30 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f2900005040 tx=0x7f2900007e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d664e00d97d6f19 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:01:32.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.261+0000 7f28fcc00700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-21T22:01:32.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.261+0000 7f28fcc00700 1 -- 172.21.15.151:0/612297667 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2900097020 con 0x563bd8d8d1c0 2024-05-21T22:01:32.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.262+0000 7f28fcc00700 1 -- 172.21.15.151:0/612297667 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f29000a6ae0 con 0x563bd8d8d1c0 2024-05-21T22:01:32.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.262+0000 7f291fdb5b80 1 -- 172.21.15.151:0/612297667 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29000aa4f0 con 0x563bd8d8d1c0 2024-05-21T22:01:32.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.262+0000 7f28fcc00700 1 -- 172.21.15.151:0/612297667 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f29000971c0 con 0x563bd8d8d1c0 2024-05-21T22:01:32.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.262+0000 7f291fdb5b80 10 client.26620 fetch_fsmap learned FSMap version 29 2024-05-21T22:01:32.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.262+0000 7f291fdb5b80 1 -- 172.21.15.151:0/612297667 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563bd8cab0e0 con 0x563bd8d8d1c0 2024-05-21T22:01:32.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.262+0000 7f28fcc00700 1 -- 172.21.15.151:0/612297667 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29000aa4f0 con 0x563bd8d8d1c0 2024-05-21T22:01:32.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.262+0000 7f28fcc00700 1 -- 172.21.15.151:0/612297667 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f290002c060 con 0x563bd8d8d1c0 2024-05-21T22:01:32.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.262+0000 7f291fdb5b80 10 client.26620 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:01:32.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.262+0000 7f291fdb5b80 10 client.26620 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T22:01:32.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.262+0000 7f291fdb5b80 20 client.26620 populate_metadata read hostname 'smithi151' 2024-05-21T22:01:32.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.262+0000 7f291fdb5b80 1 --2- 172.21.15.151:0/612297667 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x563bd8da5ff0 0x563bd8da83e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:32.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.262+0000 7f291fdb5b80 4 client.26620 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T22:01:32.269 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.262+0000 7f291fdb5b80 1 -- 172.21.15.151:0/612297667 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563bd8cece10 con 0x563bd8da5ff0 2024-05-21T22:01:32.269 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.263+0000 7f2905e00700 1 --2- 172.21.15.151:0/612297667 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x563bd8da5ff0 0x563bd8da83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:32.269 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.263+0000 7f2905e00700 1 --2- 172.21.15.151:0/612297667 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x563bd8da5ff0 0x563bd8da83e0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6c09bfc4381c314 server_cookie=498595569c09c084 in_seq=0 out_seq=0 2024-05-21T22:01:32.269 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.263+0000 7f28fcc00700 10 client.26620 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T22:01:32.269 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.264+0000 7f28fcc00700 1 -- 172.21.15.151:0/612297667 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x563bd8cece10 con 0x563bd8da5ff0 2024-05-21T22:01:32.269 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.264+0000 7f28fcc00700 10 client.26620 handle_command_reply: tid=0 2024-05-21T22:01:32.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.264+0000 7f291fdb5b80 1 client.26620 shutdown 2024-05-21T22:01:32.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.264+0000 7f291fdb5b80 1 -- 172.21.15.151:0/612297667 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563bd8d8d1c0 msgr2=0x563bd8d98d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:32.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.265+0000 7f291fdb5b80 1 --2- 172.21.15.151:0/612297667 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563bd8d8d1c0 0x563bd8d98d30 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f2900005040 tx=0x7f2900007e40 comp rx=0 tx=0).stop 2024-05-21T22:01:32.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.265+0000 7f291fdb5b80 1 -- 172.21.15.151:0/612297667 shutdown_connections 2024-05-21T22:01:32.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.265+0000 7f291fdb5b80 1 --2- 172.21.15.151:0/612297667 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563bd8d92fb0 0x563bd8d96610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:32.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.265+0000 7f291fdb5b80 1 --2- 172.21.15.151:0/612297667 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563bd8d8d1c0 0x563bd8d98d30 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:32.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.265+0000 7f291fdb5b80 1 --2- 172.21.15.151:0/612297667 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x563bd8da5ff0 0x563bd8da83e0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:32.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.265+0000 7f291fdb5b80 1 --2- 172.21.15.151:0/612297667 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bd8d92690 0x563bd8d960d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:32.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.265+0000 7f291fdb5b80 1 -- 172.21.15.151:0/612297667 >> 172.21.15.151:0/612297667 conn(0x563bd8cbac60 msgr2=0x563bd8d86df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:01:32.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.265+0000 7f28fcc00700 0 client.26620 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T22:01:32.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.265+0000 7f291fdb5b80 1 -- 172.21.15.151:0/612297667 shutdown_connections 2024-05-21T22:01:32.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.265+0000 7f291fdb5b80 1 -- 172.21.15.151:0/612297667 wait complete. 2024-05-21T22:01:32.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.265+0000 7f291fdb5b80 20 client.26620 trim_cache size 0 max 16384 2024-05-21T22:01:32.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.288+0000 7f291fdb5b80 1 Processor -- start 2024-05-21T22:01:32.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.288+0000 7f291fdb5b80 1 -- start start 2024-05-21T22:01:32.289 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.288+0000 7f291fdb5b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563bd8d8d1c0 0x563bd8d8cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:32.289 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.288+0000 7f291fdb5b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bd8d92690 0x563bd8d87570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:32.289 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.288+0000 7f291fdb5b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563bd8d92fb0 0x563bd8d93a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:32.289 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.288+0000 7f291fdb5b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f29101aff20 con 0x563bd8d8d1c0 2024-05-21T22:01:32.289 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.288+0000 7f291fdb5b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f29101b1a60 con 0x563bd8d92690 2024-05-21T22:01:32.289 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.288+0000 7f291fdb5b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f29101b05f0 con 0x563bd8d92fb0 2024-05-21T22:01:32.289 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.288+0000 7f290e800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563bd8d8d1c0 0x563bd8d8cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:32.289 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.289+0000 7f290e800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563bd8d8d1c0 0x563bd8d8cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:45232/0 (socket says 172.21.15.151:45232) 2024-05-21T22:01:32.289 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.289+0000 7f290e800700 1 -- 172.21.15.151:0/3095794217 learned_addr learned my addr 172.21.15.151:0/3095794217 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:01:32.290 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.289+0000 7f2905e00700 1 --2- 172.21.15.151:0/3095794217 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bd8d92690 0x563bd8d87570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:32.290 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.289+0000 7f290f200700 1 --2- 172.21.15.151:0/3095794217 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563bd8d92fb0 0x563bd8d93a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:32.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.289+0000 7f290e800700 1 -- 172.21.15.151:0/3095794217 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563bd8d92fb0 msgr2=0x563bd8d93a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:32.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.289+0000 7f290e800700 1 --2- 172.21.15.151:0/3095794217 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563bd8d92fb0 0x563bd8d93a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:32.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.289+0000 7f290e800700 1 -- 172.21.15.151:0/3095794217 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bd8d92690 msgr2=0x563bd8d87570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:32.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.289+0000 7f290e800700 1 --2- 172.21.15.151:0/3095794217 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bd8d92690 0x563bd8d87570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:32.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.289+0000 7f290e800700 1 -- 172.21.15.151:0/3095794217 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563bd8d94640 con 0x563bd8d8d1c0 2024-05-21T22:01:32.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.289+0000 7f290f200700 1 --2- 172.21.15.151:0/3095794217 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563bd8d92fb0 0x563bd8d93a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T22:01:32.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.289+0000 7f2905e00700 1 --2- 172.21.15.151:0/3095794217 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bd8d92690 0x563bd8d87570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:01:32.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.289+0000 7f290e800700 1 --2- 172.21.15.151:0/3095794217 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563bd8d8d1c0 0x563bd8d8cba0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f29000a9fb0 tx=0x7f290000b3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a08c4ab432ca65e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:01:32.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.290+0000 7f28ff400700 1 -- 172.21.15.151:0/3095794217 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29000971c0 con 0x563bd8d8d1c0 2024-05-21T22:01:32.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.290+0000 7f28ff400700 1 -- 172.21.15.151:0/3095794217 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29000aabf0 con 0x563bd8d8d1c0 2024-05-21T22:01:32.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.290+0000 7f28ff400700 1 -- 172.21.15.151:0/3095794217 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29000aa960 con 0x563bd8d8d1c0 2024-05-21T22:01:32.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.291+0000 7f291fdb5b80 1 -- 172.21.15.151:0/3095794217 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563bd8d8d1c0 msgr2=0x563bd8d8cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:32.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.291+0000 7f291fdb5b80 1 --2- 172.21.15.151:0/3095794217 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563bd8d8d1c0 0x563bd8d8cba0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f29000a9fb0 tx=0x7f290000b3e0 comp rx=0 tx=0).stop 2024-05-21T22:01:32.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.291+0000 7f291fdb5b80 1 -- 172.21.15.151:0/3095794217 shutdown_connections 2024-05-21T22:01:32.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.291+0000 7f291fdb5b80 1 --2- 172.21.15.151:0/3095794217 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563bd8d92fb0 0x563bd8d93a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:32.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.291+0000 7f291fdb5b80 1 --2- 172.21.15.151:0/3095794217 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563bd8d8d1c0 0x563bd8d8cba0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:32.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.291+0000 7f291fdb5b80 1 --2- 172.21.15.151:0/3095794217 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bd8d92690 0x563bd8d87570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:32.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.291+0000 7f291fdb5b80 1 -- 172.21.15.151:0/3095794217 >> 172.21.15.151:0/3095794217 conn(0x563bd8cbac60 msgr2=0x563bd8d95120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:01:32.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.291+0000 7f291fdb5b80 1 -- 172.21.15.151:0/3095794217 shutdown_connections 2024-05-21T22:01:32.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.291+0000 7f291fdb5b80 1 -- 172.21.15.151:0/3095794217 wait complete. 2024-05-21T22:01:32.294 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.291+0000 7f291fdb5b80 1 -- start start 2024-05-21T22:01:32.294 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.292+0000 7f291fdb5b80 1 Processor -- start 2024-05-21T22:01:32.294 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.292+0000 7f291fdb5b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563bd8d8d1c0 0x563bd8d9ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:32.294 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.292+0000 7f291fdb5b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bd8d92690 0x563bd8d9f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:32.294 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.292+0000 7f291fdb5b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563bd8d92fb0 0x563bd8d9b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:32.294 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.292+0000 7f291fdb5b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f29101b1390 con 0x563bd8d8d1c0 2024-05-21T22:01:32.294 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.292+0000 7f291fdb5b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f29101b0cc0 con 0x563bd8d92690 2024-05-21T22:01:32.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.292+0000 7f291fdb5b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f29101adec0 con 0x563bd8d92fb0 2024-05-21T22:01:32.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.292+0000 7f291fdb5b80 10 client.0 _ll_register_callbacks cb 0x563bd8c84cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:01:32.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.292+0000 7f290e800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563bd8d8d1c0 0x563bd8d9ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:32.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.292+0000 7f290e800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563bd8d8d1c0 0x563bd8d9ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:45246/0 (socket says 172.21.15.151:45246) 2024-05-21T22:01:32.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.293+0000 7f290e800700 1 -- 172.21.15.151:0/2124629724 learned_addr learned my addr 172.21.15.151:0/2124629724 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:01:32.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.293+0000 7f290f200700 1 --2- 172.21.15.151:0/2124629724 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563bd8d92fb0 0x563bd8d9b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:32.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.293+0000 7f2905e00700 1 --2- 172.21.15.151:0/2124629724 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bd8d92690 0x563bd8d9f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:32.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.293+0000 7f290e800700 1 -- 172.21.15.151:0/2124629724 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563bd8d92fb0 msgr2=0x563bd8d9b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:32.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.293+0000 7f290e800700 1 --2- 172.21.15.151:0/2124629724 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563bd8d92fb0 0x563bd8d9b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:32.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.293+0000 7f290e800700 1 -- 172.21.15.151:0/2124629724 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bd8d92690 msgr2=0x563bd8d9f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:32.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.293+0000 7f290e800700 1 --2- 172.21.15.151:0/2124629724 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bd8d92690 0x563bd8d9f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:32.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.293+0000 7f290e800700 1 -- 172.21.15.151:0/2124629724 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563bd8d9c120 con 0x563bd8d8d1c0 2024-05-21T22:01:32.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.293+0000 7f290e800700 1 --2- 172.21.15.151:0/2124629724 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563bd8d8d1c0 0x563bd8d9ead0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f29000aa480 tx=0x7f2900007e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f74d7fed24e152f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:01:32.296 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.293+0000 7f28fcc00700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-21T22:01:32.296 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.294+0000 7f28fcc00700 1 -- 172.21.15.151:0/2124629724 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29000b32c0 con 0x563bd8d8d1c0 2024-05-21T22:01:32.296 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.294+0000 7f28fcc00700 1 -- 172.21.15.151:0/2124629724 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f29000971c0 con 0x563bd8d8d1c0 2024-05-21T22:01:32.296 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.294+0000 7f291fdb5b80 1 -- 172.21.15.151:0/2124629724 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29000aabf0 con 0x563bd8d8d1c0 2024-05-21T22:01:32.296 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.294+0000 7f28fcc00700 1 -- 172.21.15.151:0/2124629724 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2900092090 con 0x563bd8d8d1c0 2024-05-21T22:01:32.296 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.294+0000 7f291fdb5b80 10 client.16488 fetch_fsmap learned FSMap version 29 2024-05-21T22:01:32.296 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.294+0000 7f291fdb5b80 1 -- 172.21.15.151:0/2124629724 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563bd8d9c120 con 0x563bd8d8d1c0 2024-05-21T22:01:32.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.294+0000 7f28fcc00700 1 -- 172.21.15.151:0/2124629724 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29000aabf0 con 0x563bd8d8d1c0 2024-05-21T22:01:32.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.294+0000 7f28fcc00700 1 -- 172.21.15.151:0/2124629724 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f2900011070 con 0x563bd8d8d1c0 2024-05-21T22:01:32.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.294+0000 7f291fdb5b80 10 client.16488 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:01:32.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.294+0000 7f291fdb5b80 10 client.16488 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T22:01:32.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.294+0000 7f291fdb5b80 20 client.16488 populate_metadata read hostname 'smithi151' 2024-05-21T22:01:32.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.294+0000 7f291fdb5b80 1 --2- 172.21.15.151:0/2124629724 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x563bd8e1a060 0x563bd8e1c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:32.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.294+0000 7f291fdb5b80 4 client.16488 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-21T22:01:32.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.294+0000 7f291fdb5b80 1 -- 172.21.15.151:0/2124629724 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563bd8ce65f0 con 0x563bd8e1a060 2024-05-21T22:01:32.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.295+0000 7f2905e00700 1 --2- 172.21.15.151:0/2124629724 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x563bd8e1a060 0x563bd8e1c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:32.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.295+0000 7f2905e00700 1 --2- 172.21.15.151:0/2124629724 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x563bd8e1a060 0x563bd8e1c450 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2de1144570ad68f server_cookie=bc643e4540da5f06 in_seq=0 out_seq=0 2024-05-21T22:01:32.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.295+0000 7f28fcc00700 10 client.16488 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T22:01:32.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.297+0000 7f28fcc00700 1 -- 172.21.15.151:0/2124629724 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563bd8ce65f0 con 0x563bd8e1a060 2024-05-21T22:01:32.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.297+0000 7f28fcc00700 10 client.16488 handle_command_reply: tid=0 2024-05-21T22:01:32.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.297+0000 7f291fdb5b80 1 client.16488 shutdown 2024-05-21T22:01:32.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.297+0000 7f291fdb5b80 1 -- 172.21.15.151:0/2124629724 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563bd8d8d1c0 msgr2=0x563bd8d9ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:32.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.297+0000 7f291fdb5b80 1 --2- 172.21.15.151:0/2124629724 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563bd8d8d1c0 0x563bd8d9ead0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f29000aa480 tx=0x7f2900007e40 comp rx=0 tx=0).stop 2024-05-21T22:01:32.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.297+0000 7f291fdb5b80 1 -- 172.21.15.151:0/2124629724 shutdown_connections 2024-05-21T22:01:32.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.297+0000 7f291fdb5b80 1 --2- 172.21.15.151:0/2124629724 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563bd8d92fb0 0x563bd8d9b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:32.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.297+0000 7f291fdb5b80 1 --2- 172.21.15.151:0/2124629724 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563bd8d8d1c0 0x563bd8d9ead0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:32.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.297+0000 7f291fdb5b80 1 --2- 172.21.15.151:0/2124629724 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x563bd8e1a060 0x563bd8e1c450 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:32.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.297+0000 7f291fdb5b80 1 --2- 172.21.15.151:0/2124629724 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563bd8d92690 0x563bd8d9f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:32.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.297+0000 7f291fdb5b80 1 -- 172.21.15.151:0/2124629724 >> 172.21.15.151:0/2124629724 conn(0x563bd8cbac60 msgr2=0x563bd8d95120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:01:32.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.297+0000 7f28fcc00700 0 client.16488 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T22:01:32.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.297+0000 7f291fdb5b80 1 -- 172.21.15.151:0/2124629724 shutdown_connections 2024-05-21T22:01:32.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.297+0000 7f291fdb5b80 1 -- 172.21.15.151:0/2124629724 wait complete. 2024-05-21T22:01:32.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.297+0000 7f291fdb5b80 20 client.16488 trim_cache size 0 max 16384 2024-05-21T22:01:32.300 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-05-21T22:01:32.300 INFO:teuthology.orchestra.run.smithi151.stdout: "return_code": 0, 2024-05-21T22:01:32.300 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_tag": "69c76694-8180-44ab-a1c6-70a9f34fd111", 2024-05-21T22:01:32.300 INFO:teuthology.orchestra.run.smithi151.stdout: "mode": "asynchronous" 2024-05-21T22:01:32.301 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-05-21T22:01:32.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.298+0000 7f2915400700 1 -- 172.21.15.151:0/3932679597 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f28f40b29f0 msgr2=0x7f28f40b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:32.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.298+0000 7f2915400700 1 --2- 172.21.15.151:0/3932679597 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f28f40b29f0 0x7f28f40b4eb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x563bd8d8db80 tx=0x7f29080099d0 comp rx=0 tx=0).stop 2024-05-21T22:01:32.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.298+0000 7f2915400700 1 -- 172.21.15.151:0/3932679597 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f29101037f0 msgr2=0x7f291019be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:32.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.298+0000 7f2915400700 1 --2- 172.21.15.151:0/3932679597 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f29101037f0 0x7f291019be20 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f2900000f80 tx=0x7f2900007a00 comp rx=0 tx=0).stop 2024-05-21T22:01:32.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.298+0000 7f2915400700 1 -- 172.21.15.151:0/3932679597 shutdown_connections 2024-05-21T22:01:32.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.298+0000 7f2915400700 1 --2- 172.21.15.151:0/3932679597 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f28f40b29f0 0x7f28f40b4eb0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:32.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.298+0000 7f2915400700 1 --2- 172.21.15.151:0/3932679597 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f29101088f0 0x7f29101a06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:32.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.298+0000 7f2915400700 1 --2- 172.21.15.151:0/3932679597 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f29101037f0 0x7f291019be20 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:32.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.298+0000 7f2915400700 1 --2- 172.21.15.151:0/3932679597 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f29101041a0 0x7f291019c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:32.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.298+0000 7f2915400700 1 -- 172.21.15.151:0/3932679597 >> 172.21.15.151:0/3932679597 conn(0x7f291007a6a0 msgr2=0x7f2910101650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:01:32.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.298+0000 7f2915400700 1 -- 172.21.15.151:0/3932679597 shutdown_connections 2024-05-21T22:01:32.303 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:32.299+0000 7f2915400700 1 -- 172.21.15.151:0/3932679597 wait complete. 2024-05-21T22:01:33.265 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-21T22:01:33.723 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:01:33.876 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:01:33 smithi151 ceph-mon[31623]: scrub queued for path: / 2024-05-21T22:01:33.876 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:01:33 smithi151 ceph-mon[31623]: scrub summary: idle+waiting paths [/] 2024-05-21T22:01:33.876 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:01:33 smithi151 ceph-mon[31623]: scrub summary: active paths [/] 2024-05-21T22:01:33.876 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:01:33 smithi151 ceph-mon[31623]: scrub summary: idle+waiting paths [/] 2024-05-21T22:01:33.876 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:01:33 smithi151 ceph-mon[31623]: pgmap v181: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T22:01:33.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:01:33 smithi184 ceph-mon[36942]: scrub queued for path: / 2024-05-21T22:01:33.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:01:33 smithi184 ceph-mon[36942]: scrub summary: idle+waiting paths [/] 2024-05-21T22:01:33.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:01:33 smithi184 ceph-mon[36942]: scrub summary: active paths [/] 2024-05-21T22:01:33.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:01:33 smithi184 ceph-mon[36942]: scrub summary: idle+waiting paths [/] 2024-05-21T22:01:33.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:01:33 smithi184 ceph-mon[36942]: pgmap v181: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T22:01:34.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:01:33 smithi164 ceph-mon[36269]: scrub queued for path: / 2024-05-21T22:01:34.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:01:33 smithi164 ceph-mon[36269]: scrub summary: idle+waiting paths [/] 2024-05-21T22:01:34.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:01:33 smithi164 ceph-mon[36269]: scrub summary: active paths [/] 2024-05-21T22:01:34.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:01:33 smithi164 ceph-mon[36269]: scrub summary: idle+waiting paths [/] 2024-05-21T22:01:34.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:01:33 smithi164 ceph-mon[36269]: pgmap v181: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T22:01:34.904 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:01:34 smithi151 ceph-mon[31623]: scrub summary: idle 2024-05-21T22:01:34.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:01:34 smithi184 ceph-mon[36942]: scrub summary: idle 2024-05-21T22:01:35.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:01:34 smithi164 ceph-mon[36269]: scrub summary: idle 2024-05-21T22:01:35.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.807+0000 7fd0d2200700 1 -- 172.21.15.151:0/869499855 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd0cc1088f0 msgr2=0x7fd0cc110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:35.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.807+0000 7fd0d2200700 1 --2- 172.21.15.151:0/869499855 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd0cc1088f0 0x7fd0cc110470 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fd0c400b0d0 tx=0x7fd0c400b3e0 comp rx=0 tx=0).stop 2024-05-21T22:01:35.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.807+0000 7fd0d2200700 1 -- 172.21.15.151:0/869499855 shutdown_connections 2024-05-21T22:01:35.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.807+0000 7fd0d2200700 1 --2- 172.21.15.151:0/869499855 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0cc1037f0 0x7fd0cc103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:35.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.807+0000 7fd0d2200700 1 --2- 172.21.15.151:0/869499855 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd0cc1088f0 0x7fd0cc110470 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:35.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.807+0000 7fd0d2200700 1 --2- 172.21.15.151:0/869499855 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0cc1041a0 0x7fd0cc1081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:35.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.807+0000 7fd0d2200700 1 -- 172.21.15.151:0/869499855 >> 172.21.15.151:0/869499855 conn(0x7fd0cc07a6a0 msgr2=0x7fd0cc07aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:01:35.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.807+0000 7fd0d2200700 1 -- 172.21.15.151:0/869499855 shutdown_connections 2024-05-21T22:01:35.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.807+0000 7fd0d2200700 1 -- 172.21.15.151:0/869499855 wait complete. 2024-05-21T22:01:35.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.808+0000 7fd0d2200700 1 Processor -- start 2024-05-21T22:01:35.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.808+0000 7fd0d2200700 1 -- start start 2024-05-21T22:01:35.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.808+0000 7fd0d2200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0cc1037f0 0x7fd0cc19bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:35.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.809+0000 7fd0d2200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd0cc1041a0 0x7fd0cc19c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:35.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.809+0000 7fd0d2200700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0cc1088f0 0x7fd0cc1a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:35.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.809+0000 7fd0d2200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd0cc1136e0 con 0x7fd0cc1041a0 2024-05-21T22:01:35.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.809+0000 7fd0d2200700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd0cc113560 con 0x7fd0cc1037f0 2024-05-21T22:01:35.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.809+0000 7fd0d2200700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd0cc113860 con 0x7fd0cc1088f0 2024-05-21T22:01:35.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.809+0000 7fd0cb400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0cc1037f0 0x7fd0cc19bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:35.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.809+0000 7fd0caa00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd0cc1041a0 0x7fd0cc19c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:35.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.809+0000 7fd0cb400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0cc1037f0 0x7fd0cc19bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:55402/0 (socket says 172.21.15.151:55402) 2024-05-21T22:01:35.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.809+0000 7fd0cbe00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0cc1088f0 0x7fd0cc1a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:35.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.809+0000 7fd0cb400700 1 -- 172.21.15.151:0/1758775605 learned_addr learned my addr 172.21.15.151:0/1758775605 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:01:35.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.809+0000 7fd0cb400700 1 -- 172.21.15.151:0/1758775605 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0cc1088f0 msgr2=0x7fd0cc1a0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:35.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.809+0000 7fd0cb400700 1 --2- 172.21.15.151:0/1758775605 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0cc1088f0 0x7fd0cc1a0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:35.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.809+0000 7fd0cb400700 1 -- 172.21.15.151:0/1758775605 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd0cc1041a0 msgr2=0x7fd0cc19c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:35.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.809+0000 7fd0cb400700 1 --2- 172.21.15.151:0/1758775605 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd0cc1041a0 0x7fd0cc19c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:35.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.809+0000 7fd0cb400700 1 -- 172.21.15.151:0/1758775605 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0cc1a0d80 con 0x7fd0cc1037f0 2024-05-21T22:01:35.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.810+0000 7fd0cb400700 1 --2- 172.21.15.151:0/1758775605 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0cc1037f0 0x7fd0cc19bd70 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc00e9a0 tx=0x7fd0bc00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:01:35.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.810+0000 7fd0c3e00700 1 -- 172.21.15.151:0/1758775605 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0bc00ca10 con 0x7fd0cc1037f0 2024-05-21T22:01:35.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.810+0000 7fd0c3e00700 1 -- 172.21.15.151:0/1758775605 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0bc00cbb0 con 0x7fd0cc1037f0 2024-05-21T22:01:35.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.810+0000 7fd0d2200700 1 -- 172.21.15.151:0/1758775605 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0cc1a1070 con 0x7fd0cc1037f0 2024-05-21T22:01:35.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.810+0000 7fd0c3e00700 1 -- 172.21.15.151:0/1758775605 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0bc018680 con 0x7fd0cc1037f0 2024-05-21T22:01:35.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.810+0000 7fd0d2200700 1 -- 172.21.15.151:0/1758775605 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0cc1a88f0 con 0x7fd0cc1037f0 2024-05-21T22:01:35.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.814+0000 7fd0c3e00700 1 -- 172.21.15.151:0/1758775605 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7fd0bc010430 con 0x7fd0cc1037f0 2024-05-21T22:01:35.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.815+0000 7fd0dcc26b80 1 Processor -- start 2024-05-21T22:01:35.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.815+0000 7fd0dcc26b80 1 -- start start 2024-05-21T22:01:35.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.815+0000 7fd0dcc26b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556b55052610 0x556b550529f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:35.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.815+0000 7fd0dcc26b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556b55052f30 0x556b5504cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:35.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.815+0000 7fd0dcc26b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556b5504d140 0x556b5504d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:35.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.815+0000 7fd0dcc26b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x556b54faae40 con 0x556b55052f30 2024-05-21T22:01:35.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.815+0000 7fd0dcc26b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x556b54f9a9e0 con 0x556b55052610 2024-05-21T22:01:35.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.815+0000 7fd0dcc26b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x556b54f9ea20 con 0x556b5504d140 2024-05-21T22:01:35.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.816+0000 7fd0cbe00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556b55052610 0x556b550529f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:35.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.816+0000 7fd0caa00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556b5504d140 0x556b5504d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:35.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.816+0000 7fd0cb400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556b55052f30 0x556b5504cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:35.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.816+0000 7fd0cbe00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556b55052610 0x556b550529f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:55416/0 (socket says 172.21.15.151:55416) 2024-05-21T22:01:35.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.816+0000 7fd0cbe00700 1 -- 172.21.15.151:0/865512595 learned_addr learned my addr 172.21.15.151:0/865512595 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:01:35.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.816+0000 7fd0c3e00700 1 --2- 172.21.15.151:0/1758775605 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fd0b00b2a70 0x7fd0b00b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:35.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.816+0000 7fd0caa00700 1 -- 172.21.15.151:0/865512595 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556b55052610 msgr2=0x556b550529f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:35.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.816+0000 7fd0caa00700 1 --2- 172.21.15.151:0/865512595 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556b55052610 0x556b550529f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:35.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.816+0000 7fd0caa00700 1 -- 172.21.15.151:0/865512595 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556b55052f30 msgr2=0x556b5504cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:35.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.816+0000 7fd0caa00700 1 --2- 172.21.15.151:0/865512595 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556b55052f30 0x556b5504cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:35.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.816+0000 7fd0caa00700 1 -- 172.21.15.151:0/865512595 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b54f63420 con 0x556b5504d140 2024-05-21T22:01:35.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.816+0000 7fd0cbe00700 1 --2- 172.21.15.151:0/865512595 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556b55052610 0x556b550529f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:01:35.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.816+0000 7fd0c3e00700 1 -- 172.21.15.151:0/1758775605 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fd0bc0e2760 con 0x7fd0cc1037f0 2024-05-21T22:01:35.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.817+0000 7fd0cbe00700 1 --2- 172.21.15.151:0/1758775605 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fd0b00b2a70 0x7fd0b00b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:35.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.817+0000 7fd0caa00700 1 --2- 172.21.15.151:0/865512595 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556b5504d140 0x556b5504d5c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fd0b8009c60 tx=0x7fd0b8009f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b9905c66831f57e server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:01:35.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.817+0000 7fd0c0c00700 1 -- 172.21.15.151:0/865512595 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b8011cb0 con 0x556b5504d140 2024-05-21T22:01:35.819 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.817+0000 7fd0c0c00700 1 -- 172.21.15.151:0/865512595 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0b8011e50 con 0x556b5504d140 2024-05-21T22:01:35.819 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.817+0000 7fd0c0c00700 1 -- 172.21.15.151:0/865512595 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b8005280 con 0x556b5504d140 2024-05-21T22:01:35.820 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.818+0000 7fd0dcc26b80 1 -- 172.21.15.151:0/865512595 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556b5504d140 msgr2=0x556b5504d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:35.820 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.818+0000 7fd0cbe00700 1 --2- 172.21.15.151:0/1758775605 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fd0b00b2a70 0x7fd0b00b4f30 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd0c4009e60 tx=0x7fd0c40099e0 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:01:35.820 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.818+0000 7fd0dcc26b80 1 --2- 172.21.15.151:0/865512595 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556b5504d140 0x556b5504d5c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fd0b8009c60 tx=0x7fd0b8009f70 comp rx=0 tx=0).stop 2024-05-21T22:01:35.820 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.818+0000 7fd0dcc26b80 1 -- 172.21.15.151:0/865512595 shutdown_connections 2024-05-21T22:01:35.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.818+0000 7fd0dcc26b80 1 --2- 172.21.15.151:0/865512595 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556b55052610 0x556b550529f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:35.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.818+0000 7fd0dcc26b80 1 --2- 172.21.15.151:0/865512595 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556b5504d140 0x556b5504d5c0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:35.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.818+0000 7fd0dcc26b80 1 --2- 172.21.15.151:0/865512595 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556b55052f30 0x556b5504cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:35.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.818+0000 7fd0dcc26b80 1 -- 172.21.15.151:0/865512595 >> 172.21.15.151:0/865512595 conn(0x556b54f7abe0 msgr2=0x556b55047690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:01:35.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.818+0000 7fd0dcc26b80 1 -- 172.21.15.151:0/865512595 shutdown_connections 2024-05-21T22:01:35.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.818+0000 7fd0dcc26b80 1 -- 172.21.15.151:0/865512595 wait complete. 2024-05-21T22:01:35.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.819+0000 7fd0dcc26b80 1 -- start start 2024-05-21T22:01:35.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.819+0000 7fd0dcc26b80 1 Processor -- start 2024-05-21T22:01:35.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.820+0000 7fd0dcc26b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556b5504d140 0x556b55058bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:35.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.820+0000 7fd0dcc26b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556b55052610 0x556b55055f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:35.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.820+0000 7fd0dcc26b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556b55052f30 0x556b550564b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:35.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.820+0000 7fd0dcc26b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x556b54ea0fa0 con 0x556b5504d140 2024-05-21T22:01:35.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.820+0000 7fd0dcc26b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x556b54f3b760 con 0x556b55052f30 2024-05-21T22:01:35.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.820+0000 7fd0dcc26b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x556b550599e0 con 0x556b55052610 2024-05-21T22:01:35.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.820+0000 7fd0dcc26b80 10 client.0 _ll_register_callbacks cb 0x556b54f44c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:01:35.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.820+0000 7fd0cb400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556b5504d140 0x556b55058bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:35.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.820+0000 7fd0caa00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556b55052610 0x556b55055f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:35.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.820+0000 7fd0caa00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556b55052610 0x556b55055f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:40358/0 (socket says 172.21.15.151:40358) 2024-05-21T22:01:35.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.820+0000 7fd0cb400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556b5504d140 0x556b55058bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:34004/0 (socket says 172.21.15.151:34004) 2024-05-21T22:01:35.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.820+0000 7fd0caa00700 1 -- 172.21.15.151:0/877003140 learned_addr learned my addr 172.21.15.151:0/877003140 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:01:35.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.820+0000 7fd0cbe00700 1 --2- 172.21.15.151:0/877003140 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556b55052f30 0x556b550564b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:35.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.821+0000 7fd0caa00700 1 -- 172.21.15.151:0/877003140 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556b55052f30 msgr2=0x556b550564b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:35.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.821+0000 7fd0caa00700 1 --2- 172.21.15.151:0/877003140 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556b55052f30 0x556b550564b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:35.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.821+0000 7fd0caa00700 1 -- 172.21.15.151:0/877003140 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556b5504d140 msgr2=0x556b55058bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:35.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.821+0000 7fd0caa00700 1 --2- 172.21.15.151:0/877003140 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556b5504d140 0x556b55058bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:35.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.821+0000 7fd0caa00700 1 -- 172.21.15.151:0/877003140 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b54eea8d0 con 0x556b55052610 2024-05-21T22:01:35.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.821+0000 7fd0caa00700 1 --2- 172.21.15.151:0/877003140 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556b55052610 0x556b55055f70 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fd0b8014040 tx=0x7fd0b8005d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ee0267b207cb386 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:01:35.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.821+0000 7fd0b5600700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-21T22:01:35.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.821+0000 7fd0b5600700 1 -- 172.21.15.151:0/877003140 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b8026020 con 0x556b55052610 2024-05-21T22:01:35.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.821+0000 7fd0b5600700 1 -- 172.21.15.151:0/877003140 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fd0b8004050 con 0x556b55052610 2024-05-21T22:01:35.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.821+0000 7fd0dcc26b80 1 -- 172.21.15.151:0/877003140 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0b8011e50 con 0x556b55052610 2024-05-21T22:01:35.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.822+0000 7fd0b5600700 1 -- 172.21.15.151:0/877003140 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd0b8031050 con 0x556b55052610 2024-05-21T22:01:35.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.822+0000 7fd0dcc26b80 10 client.26638 fetch_fsmap learned FSMap version 29 2024-05-21T22:01:35.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.822+0000 7fd0dcc26b80 1 -- 172.21.15.151:0/877003140 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556b54f6b060 con 0x556b55052610 2024-05-21T22:01:35.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.822+0000 7fd0b5600700 1 -- 172.21.15.151:0/877003140 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0b8011e50 con 0x556b55052610 2024-05-21T22:01:35.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.822+0000 7fd0b5600700 1 -- 172.21.15.151:0/877003140 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fd0b8018070 con 0x556b55052610 2024-05-21T22:01:35.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.822+0000 7fd0dcc26b80 10 client.26638 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:01:35.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.822+0000 7fd0dcc26b80 10 client.26638 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T22:01:35.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.822+0000 7fd0dcc26b80 20 client.26638 populate_metadata read hostname 'smithi151' 2024-05-21T22:01:35.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.822+0000 7fd0dcc26b80 1 --2- 172.21.15.151:0/877003140 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x556b55066400 0x556b550687f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:35.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.822+0000 7fd0dcc26b80 4 client.26638 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T22:01:35.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.822+0000 7fd0dcc26b80 1 -- 172.21.15.151:0/877003140 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b54facd90 con 0x556b55066400 2024-05-21T22:01:35.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.823+0000 7fd0cb400700 1 --2- 172.21.15.151:0/877003140 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x556b55066400 0x556b550687f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:35.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.823+0000 7fd0cb400700 1 --2- 172.21.15.151:0/877003140 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x556b55066400 0x556b550687f0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=765ea6e0a6c321b4 server_cookie=6fb632b986a150f4 in_seq=0 out_seq=0 2024-05-21T22:01:35.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.823+0000 7fd0b5600700 10 client.26638 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T22:01:35.826 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.824+0000 7fd0b5600700 1 -- 172.21.15.151:0/877003140 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x556b54facd90 con 0x556b55066400 2024-05-21T22:01:35.826 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.824+0000 7fd0b5600700 10 client.26638 handle_command_reply: tid=0 2024-05-21T22:01:35.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.824+0000 7fd0dcc26b80 1 client.26638 shutdown 2024-05-21T22:01:35.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.825+0000 7fd0dcc26b80 1 -- 172.21.15.151:0/877003140 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556b55052610 msgr2=0x556b55055f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:35.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.825+0000 7fd0dcc26b80 1 --2- 172.21.15.151:0/877003140 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556b55052610 0x556b55055f70 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fd0b8014040 tx=0x7fd0b8005d90 comp rx=0 tx=0).stop 2024-05-21T22:01:35.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.825+0000 7fd0dcc26b80 1 -- 172.21.15.151:0/877003140 shutdown_connections 2024-05-21T22:01:35.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.825+0000 7fd0dcc26b80 1 --2- 172.21.15.151:0/877003140 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x556b55066400 0x556b550687f0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:35.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.825+0000 7fd0dcc26b80 1 --2- 172.21.15.151:0/877003140 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556b55052f30 0x556b550564b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:35.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.825+0000 7fd0dcc26b80 1 --2- 172.21.15.151:0/877003140 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556b5504d140 0x556b55058bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:35.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.825+0000 7fd0b5600700 0 client.26638 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T22:01:35.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.825+0000 7fd0dcc26b80 1 --2- 172.21.15.151:0/877003140 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556b55052610 0x556b55055f70 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:35.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.825+0000 7fd0dcc26b80 1 -- 172.21.15.151:0/877003140 >> 172.21.15.151:0/877003140 conn(0x556b54f7abe0 msgr2=0x556b55046d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:01:35.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.825+0000 7fd0dcc26b80 1 -- 172.21.15.151:0/877003140 shutdown_connections 2024-05-21T22:01:35.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.825+0000 7fd0dcc26b80 1 -- 172.21.15.151:0/877003140 wait complete. 2024-05-21T22:01:35.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.825+0000 7fd0dcc26b80 20 client.26638 trim_cache size 0 max 16384 2024-05-21T22:01:35.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.847+0000 7fd0dcc26b80 1 Processor -- start 2024-05-21T22:01:35.848 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:01:35 smithi151 ceph-mon[31623]: pgmap v182: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:01:35.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.847+0000 7fd0dcc26b80 1 -- start start 2024-05-21T22:01:35.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.848+0000 7fd0dcc26b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556b5504d140 0x556b5504cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:35.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.848+0000 7fd0dcc26b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556b55052610 0x556b55053a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:35.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.848+0000 7fd0dcc26b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556b55052f30 0x556b550474f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:35.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.848+0000 7fd0dcc26b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd0cc1aff40 con 0x556b55052610 2024-05-21T22:01:35.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.848+0000 7fd0dcc26b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd0cc1b1a80 con 0x556b55052f30 2024-05-21T22:01:35.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.848+0000 7fd0dcc26b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd0cc1b0610 con 0x556b5504d140 2024-05-21T22:01:35.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.848+0000 7fd0caa00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556b55052610 0x556b55053a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:35.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.848+0000 7fd0cb400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556b5504d140 0x556b5504cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:35.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.848+0000 7fd0cb400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556b5504d140 0x556b5504cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:40370/0 (socket says 172.21.15.151:40370) 2024-05-21T22:01:35.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.848+0000 7fd0cb400700 1 -- 172.21.15.151:0/3408660698 learned_addr learned my addr 172.21.15.151:0/3408660698 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:01:35.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.849+0000 7fd0cbe00700 1 --2- 172.21.15.151:0/3408660698 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556b55052f30 0x556b550474f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:35.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.849+0000 7fd0cb400700 1 -- 172.21.15.151:0/3408660698 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556b55052f30 msgr2=0x556b550474f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:35.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.849+0000 7fd0cb400700 1 --2- 172.21.15.151:0/3408660698 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556b55052f30 0x556b550474f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:35.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.849+0000 7fd0cb400700 1 -- 172.21.15.151:0/3408660698 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556b55052610 msgr2=0x556b55053a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:35.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.849+0000 7fd0cb400700 1 --2- 172.21.15.151:0/3408660698 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556b55052610 0x556b55053a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:35.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.849+0000 7fd0cb400700 1 -- 172.21.15.151:0/3408660698 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b55049230 con 0x556b5504d140 2024-05-21T22:01:35.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.849+0000 7fd0caa00700 1 --2- 172.21.15.151:0/3408660698 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556b55052610 0x556b55053a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:01:35.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.849+0000 7fd0cb400700 1 --2- 172.21.15.151:0/3408660698 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556b5504d140 0x556b5504cb20 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc0a6c60 tx=0x7fd0bc0e4bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b95027fa95a3892f server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:01:35.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.849+0000 7fd0b7e00700 1 -- 172.21.15.151:0/3408660698 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0bc0a9500 con 0x556b5504d140 2024-05-21T22:01:35.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.850+0000 7fd0b7e00700 1 -- 172.21.15.151:0/3408660698 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0bc0aa0f0 con 0x556b5504d140 2024-05-21T22:01:35.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.850+0000 7fd0b7e00700 1 -- 172.21.15.151:0/3408660698 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0bc0cc0c0 con 0x556b5504d140 2024-05-21T22:01:35.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.850+0000 7fd0dcc26b80 1 -- 172.21.15.151:0/3408660698 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556b5504d140 msgr2=0x556b5504cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:35.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.850+0000 7fd0dcc26b80 1 --2- 172.21.15.151:0/3408660698 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556b5504d140 0x556b5504cb20 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc0a6c60 tx=0x7fd0bc0e4bb0 comp rx=0 tx=0).stop 2024-05-21T22:01:35.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.850+0000 7fd0dcc26b80 1 -- 172.21.15.151:0/3408660698 shutdown_connections 2024-05-21T22:01:35.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.850+0000 7fd0dcc26b80 1 --2- 172.21.15.151:0/3408660698 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556b55052f30 0x556b550474f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:35.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.850+0000 7fd0dcc26b80 1 --2- 172.21.15.151:0/3408660698 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556b5504d140 0x556b5504cb20 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:35.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.850+0000 7fd0dcc26b80 1 --2- 172.21.15.151:0/3408660698 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556b55052610 0x556b55053a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:35.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.850+0000 7fd0dcc26b80 1 -- 172.21.15.151:0/3408660698 >> 172.21.15.151:0/3408660698 conn(0x556b54f7abe0 msgr2=0x556b55053310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:01:35.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.850+0000 7fd0dcc26b80 1 -- 172.21.15.151:0/3408660698 shutdown_connections 2024-05-21T22:01:35.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.850+0000 7fd0dcc26b80 1 -- 172.21.15.151:0/3408660698 wait complete. 2024-05-21T22:01:35.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.851+0000 7fd0dcc26b80 1 -- start start 2024-05-21T22:01:35.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.851+0000 7fd0dcc26b80 1 Processor -- start 2024-05-21T22:01:35.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.852+0000 7fd0dcc26b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556b5504d140 0x556b5505ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:35.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.852+0000 7fd0dcc26b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556b55052610 0x556b5505f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:35.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.852+0000 7fd0dcc26b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556b55052f30 0x556b5505bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:35.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.852+0000 7fd0dcc26b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd0cc1b13b0 con 0x556b55052610 2024-05-21T22:01:35.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.852+0000 7fd0dcc26b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd0cc1b0ce0 con 0x556b5504d140 2024-05-21T22:01:35.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.852+0000 7fd0dcc26b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd0cc1adee0 con 0x556b55052f30 2024-05-21T22:01:35.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.852+0000 7fd0dcc26b80 10 client.0 _ll_register_callbacks cb 0x556b54f44c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:01:35.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.852+0000 7fd0cb400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556b5504d140 0x556b5505ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:35.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.852+0000 7fd0cb400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556b5504d140 0x556b5505ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:55434/0 (socket says 172.21.15.151:55434) 2024-05-21T22:01:35.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.852+0000 7fd0cb400700 1 -- 172.21.15.151:0/1881521457 learned_addr learned my addr 172.21.15.151:0/1881521457 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:01:35.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.852+0000 7fd0caa00700 1 --2- 172.21.15.151:0/1881521457 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556b55052610 0x556b5505f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:35.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.852+0000 7fd0cbe00700 1 --2- 172.21.15.151:0/1881521457 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556b55052f30 0x556b5505bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:35.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.852+0000 7fd0cb400700 1 -- 172.21.15.151:0/1881521457 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556b55052f30 msgr2=0x556b5505bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:35.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.852+0000 7fd0cb400700 1 --2- 172.21.15.151:0/1881521457 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556b55052f30 0x556b5505bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:35.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.852+0000 7fd0cb400700 1 -- 172.21.15.151:0/1881521457 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556b55052610 msgr2=0x556b5505f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:35.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.852+0000 7fd0cb400700 1 --2- 172.21.15.151:0/1881521457 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556b55052610 0x556b5505f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:35.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.852+0000 7fd0cb400700 1 -- 172.21.15.151:0/1881521457 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b5505c2f0 con 0x556b5504d140 2024-05-21T22:01:35.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.852+0000 7fd0caa00700 1 --2- 172.21.15.151:0/1881521457 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556b55052610 0x556b5505f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T22:01:35.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.852+0000 7fd0cbe00700 1 --2- 172.21.15.151:0/1881521457 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556b55052f30 0x556b5505bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T22:01:35.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.853+0000 7fd0cb400700 1 --2- 172.21.15.151:0/1881521457 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556b5504d140 0x556b5505ee80 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc009230 tx=0x7fd0bc0a6050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37386d125620baca server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:01:35.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.853+0000 7fd0b5600700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-21T22:01:35.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.853+0000 7fd0b5600700 1 -- 172.21.15.151:0/1881521457 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0bc0b2850 con 0x556b5504d140 2024-05-21T22:01:35.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.853+0000 7fd0dcc26b80 1 -- 172.21.15.151:0/1881521457 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0bc0aa0f0 con 0x556b5504d140 2024-05-21T22:01:35.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.853+0000 7fd0b5600700 1 -- 172.21.15.151:0/1881521457 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fd0bc0b23c0 con 0x556b5504d140 2024-05-21T22:01:35.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.853+0000 7fd0b5600700 1 -- 172.21.15.151:0/1881521457 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd0bc0a9500 con 0x556b5504d140 2024-05-21T22:01:35.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.853+0000 7fd0dcc26b80 10 client.26630 fetch_fsmap learned FSMap version 29 2024-05-21T22:01:35.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.853+0000 7fd0dcc26b80 1 -- 172.21.15.151:0/1881521457 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556b550d1780 con 0x556b5504d140 2024-05-21T22:01:35.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.854+0000 7fd0b5600700 1 -- 172.21.15.151:0/1881521457 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0bc0aa0f0 con 0x556b5504d140 2024-05-21T22:01:35.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.854+0000 7fd0b5600700 1 -- 172.21.15.151:0/1881521457 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fd0bc010840 con 0x556b5504d140 2024-05-21T22:01:35.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.854+0000 7fd0dcc26b80 10 client.26630 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:01:35.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.854+0000 7fd0dcc26b80 10 client.26630 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T22:01:35.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.854+0000 7fd0dcc26b80 20 client.26630 populate_metadata read hostname 'smithi151' 2024-05-21T22:01:35.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.854+0000 7fd0dcc26b80 1 --2- 172.21.15.151:0/1881521457 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x556b550d1d40 0x556b550d4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:35.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.854+0000 7fd0dcc26b80 4 client.26630 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-21T22:01:35.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.854+0000 7fd0dcc26b80 1 -- 172.21.15.151:0/1881521457 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556b54fa6570 con 0x556b550d1d40 2024-05-21T22:01:35.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.854+0000 7fd0caa00700 1 --2- 172.21.15.151:0/1881521457 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x556b550d1d40 0x556b550d4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:35.858 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.855+0000 7fd0caa00700 1 --2- 172.21.15.151:0/1881521457 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x556b550d1d40 0x556b550d4160 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d8e58912278d100 server_cookie=f45444b8bf8a553c in_seq=0 out_seq=0 2024-05-21T22:01:35.858 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.855+0000 7fd0b5600700 10 client.26630 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T22:01:35.858 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.855+0000 7fd0b5600700 1 -- 172.21.15.151:0/1881521457 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x556b54fa6570 con 0x556b550d1d40 2024-05-21T22:01:35.858 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.855+0000 7fd0b5600700 10 client.26630 handle_command_reply: tid=0 2024-05-21T22:01:35.858 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.855+0000 7fd0dcc26b80 1 client.26630 shutdown 2024-05-21T22:01:35.858 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.855+0000 7fd0dcc26b80 1 -- 172.21.15.151:0/1881521457 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556b5504d140 msgr2=0x556b5505ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:35.858 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.855+0000 7fd0dcc26b80 1 --2- 172.21.15.151:0/1881521457 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556b5504d140 0x556b5505ee80 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc009230 tx=0x7fd0bc0a6050 comp rx=0 tx=0).stop 2024-05-21T22:01:35.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.855+0000 7fd0dcc26b80 1 -- 172.21.15.151:0/1881521457 shutdown_connections 2024-05-21T22:01:35.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.855+0000 7fd0dcc26b80 1 --2- 172.21.15.151:0/1881521457 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x556b550d1d40 0x556b550d4160 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:35.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.855+0000 7fd0dcc26b80 1 --2- 172.21.15.151:0/1881521457 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556b5504d140 0x556b5505ee80 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:35.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.855+0000 7fd0dcc26b80 1 --2- 172.21.15.151:0/1881521457 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556b55052f30 0x556b5505bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:35.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.855+0000 7fd0dcc26b80 1 --2- 172.21.15.151:0/1881521457 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556b55052610 0x556b5505f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:35.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.855+0000 7fd0b5600700 0 client.26630 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T22:01:35.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.855+0000 7fd0dcc26b80 1 -- 172.21.15.151:0/1881521457 >> 172.21.15.151:0/1881521457 conn(0x556b54f7abe0 msgr2=0x556b55057570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:01:35.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.856+0000 7fd0dcc26b80 1 -- 172.21.15.151:0/1881521457 shutdown_connections 2024-05-21T22:01:35.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.856+0000 7fd0dcc26b80 1 -- 172.21.15.151:0/1881521457 wait complete. 2024-05-21T22:01:35.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.856+0000 7fd0dcc26b80 20 client.26630 trim_cache size 0 max 16384 2024-05-21T22:01:35.859 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-05-21T22:01:35.859 INFO:teuthology.orchestra.run.smithi151.stdout: "status": "no active scrubs running", 2024-05-21T22:01:35.860 INFO:teuthology.orchestra.run.smithi151.stdout: "scrubs": {} 2024-05-21T22:01:35.860 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-05-21T22:01:35.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.856+0000 7fd0d2200700 1 -- 172.21.15.151:0/1758775605 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fd0b00b2a70 msgr2=0x7fd0b00b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:35.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.856+0000 7fd0d2200700 1 --2- 172.21.15.151:0/1758775605 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fd0b00b2a70 0x7fd0b00b4f30 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd0c4009e60 tx=0x7fd0c40099e0 comp rx=0 tx=0).stop 2024-05-21T22:01:35.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.856+0000 7fd0d2200700 1 -- 172.21.15.151:0/1758775605 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0cc1037f0 msgr2=0x7fd0cc19bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:35.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.856+0000 7fd0d2200700 1 --2- 172.21.15.151:0/1758775605 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0cc1037f0 0x7fd0cc19bd70 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc00e9a0 tx=0x7fd0bc00ecb0 comp rx=0 tx=0).stop 2024-05-21T22:01:35.861 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.857+0000 7fd0d2200700 1 -- 172.21.15.151:0/1758775605 shutdown_connections 2024-05-21T22:01:35.861 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.857+0000 7fd0d2200700 1 --2- 172.21.15.151:0/1758775605 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fd0b00b2a70 0x7fd0b00b4f30 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:35.861 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.857+0000 7fd0d2200700 1 --2- 172.21.15.151:0/1758775605 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0cc1037f0 0x7fd0cc19bd70 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:35.861 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.857+0000 7fd0d2200700 1 --2- 172.21.15.151:0/1758775605 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0cc1088f0 0x7fd0cc1a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:35.861 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.857+0000 7fd0d2200700 1 --2- 172.21.15.151:0/1758775605 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd0cc1041a0 0x7fd0cc19c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:35.861 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.857+0000 7fd0d2200700 1 -- 172.21.15.151:0/1758775605 >> 172.21.15.151:0/1758775605 conn(0x7fd0cc07a6a0 msgr2=0x7fd0cc101830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:01:35.861 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.857+0000 7fd0d2200700 1 -- 172.21.15.151:0/1758775605 shutdown_connections 2024-05-21T22:01:35.861 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:35.857+0000 7fd0d2200700 1 -- 172.21.15.151:0/1758775605 wait complete. 2024-05-21T22:01:35.985 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:01:35 smithi184 ceph-mon[36942]: pgmap v182: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:01:36.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:01:35 smithi164 ceph-mon[36269]: pgmap v182: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:01:36.763 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-21T22:01:36.763 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-21T22:01:37.218 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:01:37.688 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:01:37 smithi151 ceph-mon[31623]: pgmap v183: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:01:37.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:01:37 smithi184 ceph-mon[36942]: pgmap v183: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:01:38.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:01:37 smithi164 ceph-mon[36269]: pgmap v183: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:01:39.447 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:39.446+0000 7f48b2200700 1 -- 172.21.15.151:0/1135874414 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f48ac103e20 msgr2=0x7f48ac110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:39.447 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:39.446+0000 7f48b2200700 1 --2- 172.21.15.151:0/1135874414 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f48ac103e20 0x7f48ac110330 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f48a400b0a0 tx=0x7f48a400b3b0 comp rx=0 tx=0).stop 2024-05-21T22:01:39.447 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:39.446+0000 7f48b2200700 1 -- 172.21.15.151:0/1135874414 shutdown_connections 2024-05-21T22:01:39.447 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:39.446+0000 7f48b2200700 1 --2- 172.21.15.151:0/1135874414 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48ac1094f0 0x7f48ac1098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:39.447 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:39.446+0000 7f48b2200700 1 --2- 172.21.15.151:0/1135874414 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f48ac103e20 0x7f48ac110330 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:39.447 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:39.446+0000 7f48b2200700 1 --2- 172.21.15.151:0/1135874414 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f48ac103460 0x7f48ac1038e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:39.447 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:39.446+0000 7f48b2200700 1 -- 172.21.15.151:0/1135874414 >> 172.21.15.151:0/1135874414 conn(0x7f48ac0fef50 msgr2=0x7f48ac101370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:01:39.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:39.446+0000 7f48b2200700 1 -- 172.21.15.151:0/1135874414 shutdown_connections 2024-05-21T22:01:39.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:39.446+0000 7f48b2200700 1 -- 172.21.15.151:0/1135874414 wait complete. 2024-05-21T22:01:39.449 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:39.447+0000 7f48b2200700 1 Processor -- start 2024-05-21T22:01:39.449 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:39.447+0000 7f48b2200700 1 -- start start 2024-05-21T22:01:39.449 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:39.448+0000 7f48b2200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f48ac103460 0x7f48ac1a1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:39.449 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:39.448+0000 7f48b2200700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f48ac103e20 0x7f48ac1a1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:39.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:39.448+0000 7f48b2200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48ac1094f0 0x7f48ac19bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:39.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:39.448+0000 7f48b2200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f48ac113680 con 0x7f48ac103460 2024-05-21T22:01:39.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:39.448+0000 7f48b2200700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f48ac113500 con 0x7f48ac1094f0 2024-05-21T22:01:39.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:39.448+0000 7f48b2200700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f48ac113800 con 0x7f48ac103e20 2024-05-21T22:01:39.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:39.448+0000 7f48b0e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f48ac103460 0x7f48ac1a1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:39.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:39.448+0000 7f48b0e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f48ac103460 0x7f48ac1a1a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:34054/0 (socket says 172.21.15.151:34054) 2024-05-21T22:01:39.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:39.448+0000 7f48b0e00700 1 -- 172.21.15.151:0/2784746414 learned_addr learned my addr 172.21.15.151:0/2784746414 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:01:39.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:39.448+0000 7f48abe00700 1 --2- 172.21.15.151:0/2784746414 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f48ac103e20 0x7f48ac1a1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:39.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:39.448+0000 7f48b1800700 1 --2- 172.21.15.151:0/2784746414 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48ac1094f0 0x7f48ac19bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:39.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:39.449+0000 7f48abe00700 1 -- 172.21.15.151:0/2784746414 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48ac1094f0 msgr2=0x7f48ac19bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:39.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:39.449+0000 7f48abe00700 1 --2- 172.21.15.151:0/2784746414 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48ac1094f0 0x7f48ac19bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:39.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:39.449+0000 7f48abe00700 1 -- 172.21.15.151:0/2784746414 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f48ac103460 msgr2=0x7f48ac1a1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:39.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:39.449+0000 7f48abe00700 1 --2- 172.21.15.151:0/2784746414 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f48ac103460 0x7f48ac1a1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:39.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:39.449+0000 7f48abe00700 1 -- 172.21.15.151:0/2784746414 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48ac19c410 con 0x7f48ac103e20 2024-05-21T22:01:39.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:39.449+0000 7f48b1800700 1 --2- 172.21.15.151:0/2784746414 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48ac1094f0 0x7f48ac19bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:01:39.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:39.449+0000 7f48b0e00700 1 --2- 172.21.15.151:0/2784746414 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f48ac103460 0x7f48ac1a1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:01:39.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:39.449+0000 7f48abe00700 1 --2- 172.21.15.151:0/2784746414 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f48ac103e20 0x7f48ac1a1fa0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f489800e9f0 tx=0x7f489800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:01:39.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:39.449+0000 7f48a9600700 1 -- 172.21.15.151:0/2784746414 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4898004220 con 0x7f48ac103e20 2024-05-21T22:01:39.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:39.449+0000 7f48a9600700 1 -- 172.21.15.151:0/2784746414 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48980043c0 con 0x7f48ac103e20 2024-05-21T22:01:39.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:39.449+0000 7f48b2200700 1 -- 172.21.15.151:0/2784746414 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48ac19c700 con 0x7f48ac103e20 2024-05-21T22:01:39.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:39.449+0000 7f48a9600700 1 -- 172.21.15.151:0/2784746414 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4898005020 con 0x7f48ac103e20 2024-05-21T22:01:39.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:39.449+0000 7f48b2200700 1 -- 172.21.15.151:0/2784746414 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48ac1a89e0 con 0x7f48ac103e20 2024-05-21T22:01:39.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:39.450+0000 7f48b2200700 1 -- 172.21.15.151:0/2784746414 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48ac104ba0 con 0x7f48ac103e20 2024-05-21T22:01:39.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:39.453+0000 7f48a9600700 1 -- 172.21.15.151:0/2784746414 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7f4898004750 con 0x7f48ac103e20 2024-05-21T22:01:39.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:39.455+0000 7f48a9600700 1 --2- 172.21.15.151:0/2784746414 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f48940b29f0 0x7f48940b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:39.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:39.455+0000 7f48a9600700 1 -- 172.21.15.151:0/2784746414 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f48980e2680 con 0x7f48ac103e20 2024-05-21T22:01:39.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:39.455+0000 7f48b0e00700 1 --2- 172.21.15.151:0/2784746414 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f48940b29f0 0x7f48940b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:39.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:39.456+0000 7f48b0e00700 1 --2- 172.21.15.151:0/2784746414 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f48940b29f0 0x7f48940b4eb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f48a00103d0 tx=0x7f48a0007480 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:01:39.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:39.457+0000 7f48a9600700 1 -- 172.21.15.151:0/2784746414 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f48980aa0a0 con 0x7f48ac103e20 2024-05-21T22:01:39.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:39.745+0000 7f48b2200700 1 -- 172.21.15.151:0/2784746414 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f48ac1a8bb0 con 0x7f48ac103e20 2024-05-21T22:01:39.747 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:39.747+0000 7f48a9600700 1 -- 172.21.15.151:0/2784746414 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11958 (secure 0 0 0) 0x7f48980a99a0 con 0x7f48ac103e20 2024-05-21T22:01:39.747 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T22:01:39.748 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14358,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/158010529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":158010529},{"type":"v1","addr":"172.21.15.151:6835","nonce":158010529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6839/2069058115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":2069058115},{"type":"v1","addr":"172.21.15.151:6839","nonce":2069058115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T21:51:19.552069+0000","modified":"2024-05-21T21:55:34.643755+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24530,"mds_1":24493,"mds_2":24491,"mds_3":24478,"mds_4":24473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14403":{"gid":14403,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6841/1749723794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6840","nonce":1749723794},{"type":"v1","addr":"172.21.15.151:6841","nonce":1749723794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/2026266114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":2026266114},{"type":"v1","addr":"172.21.15.184:6833","nonce":2026266114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/2467900844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":2467900844},{"type":"v1","addr":"172.21.15.164:6833","nonce":2467900844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3367640960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3367640960},{"type":"v1","addr":"172.21.15.151:6837","nonce":3367640960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/3264142621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3264142621},{"type":"v1","addr":"172.21.15.184:6835","nonce":3264142621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":17,"addr":"172.21.15.164:6835/2340428470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2340428470},{"type":"v1","addr":"172.21.15.164:6835","nonce":2340428470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":16,"addr":"172.21.15.184:6837/2988194888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":2988194888},{"type":"v1","addr":"172.21.15.184:6837","nonce":2988194888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.164:6837/1554609572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1554609572},{"type":"v1","addr":"172.21.15.164:6837","nonce":1554609572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.184:6839/3059935143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":3059935143},{"type":"v1","addr":"172.21.15.184:6839","nonce":3059935143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.164:6839/2646401247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":2646401247},{"type":"v1","addr":"172.21.15.164:6839","nonce":2646401247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24530,"qdb_cluster":[24530,24493,24491,24473,24478]},"id":1}]} 2024-05-21T22:01:39.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:39.750+0000 7f48b2200700 1 -- 172.21.15.151:0/2784746414 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f48940b29f0 msgr2=0x7f48940b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:39.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:39.750+0000 7f48b2200700 1 --2- 172.21.15.151:0/2784746414 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f48940b29f0 0x7f48940b4eb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f48a00103d0 tx=0x7f48a0007480 comp rx=0 tx=0).stop 2024-05-21T22:01:39.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:39.750+0000 7f48b2200700 1 -- 172.21.15.151:0/2784746414 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f48ac103e20 msgr2=0x7f48ac1a1fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:39.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:39.750+0000 7f48b2200700 1 --2- 172.21.15.151:0/2784746414 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f48ac103e20 0x7f48ac1a1fa0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f489800e9f0 tx=0x7f489800ed00 comp rx=0 tx=0).stop 2024-05-21T22:01:39.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:39.750+0000 7f48b2200700 1 -- 172.21.15.151:0/2784746414 shutdown_connections 2024-05-21T22:01:39.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:39.750+0000 7f48b2200700 1 --2- 172.21.15.151:0/2784746414 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f48940b29f0 0x7f48940b4eb0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:39.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:39.750+0000 7f48b2200700 1 --2- 172.21.15.151:0/2784746414 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48ac1094f0 0x7f48ac19bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:39.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:39.750+0000 7f48b2200700 1 --2- 172.21.15.151:0/2784746414 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f48ac103460 0x7f48ac1a1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:39.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:39.750+0000 7f48b2200700 1 --2- 172.21.15.151:0/2784746414 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f48ac103e20 0x7f48ac1a1fa0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:39.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:39.750+0000 7f48b2200700 1 -- 172.21.15.151:0/2784746414 >> 172.21.15.151:0/2784746414 conn(0x7f48ac0fef50 msgr2=0x7f48ac0ff330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:01:39.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:39.750+0000 7f48b2200700 1 -- 172.21.15.151:0/2784746414 shutdown_connections 2024-05-21T22:01:39.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:39.750+0000 7f48b2200700 1 -- 172.21.15.151:0/2784746414 wait complete. 2024-05-21T22:01:39.752 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 29 2024-05-21T22:01:39.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:01:39 smithi151 ceph-mon[31623]: pgmap v184: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-05-21T22:01:39.985 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:01:39 smithi184 ceph-mon[36942]: pgmap v184: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-05-21T22:01:40.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:01:39 smithi164 ceph-mon[36269]: pgmap v184: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-05-21T22:01:40.594 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-21T22:01:40.952 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:01:40 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/2784746414' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T22:01:40.985 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:01:40 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/2784746414' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T22:01:41.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:01:40 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/2784746414' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T22:01:41.036 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:01:41.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:01:41 smithi151 ceph-mon[31623]: pgmap v185: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-05-21T22:01:41.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:01:41 smithi184 ceph-mon[36942]: pgmap v185: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-05-21T22:01:42.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:01:41 smithi164 ceph-mon[36269]: pgmap v185: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-05-21T22:01:43.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.208+0000 7f99fc000700 1 -- 172.21.15.151:0/2888495221 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f99f4103f90 msgr2=0x7f99f4110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:43.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.208+0000 7f99fc000700 1 --2- 172.21.15.151:0/2888495221 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f99f4103f90 0x7f99f4110470 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f99ec00b0a0 tx=0x7f99ec00b3b0 comp rx=0 tx=0).stop 2024-05-21T22:01:43.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.208+0000 7f99fc000700 1 -- 172.21.15.151:0/2888495221 shutdown_connections 2024-05-21T22:01:43.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.208+0000 7f99fc000700 1 --2- 172.21.15.151:0/2888495221 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f99f4103f90 0x7f99f4110470 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:43.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.208+0000 7f99fc000700 1 --2- 172.21.15.151:0/2888495221 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f99f4108500 0x7f99f41088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:43.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.208+0000 7f99fc000700 1 --2- 172.21.15.151:0/2888495221 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f99f41035f0 0x7f99f4103a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:43.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.208+0000 7f99fc000700 1 -- 172.21.15.151:0/2888495221 >> 172.21.15.151:0/2888495221 conn(0x7f99f407a6a0 msgr2=0x7f99f407aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:01:43.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.209+0000 7f99fc000700 1 -- 172.21.15.151:0/2888495221 shutdown_connections 2024-05-21T22:01:43.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.209+0000 7f99fc000700 1 -- 172.21.15.151:0/2888495221 wait complete. 2024-05-21T22:01:43.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.210+0000 7f99fc000700 1 Processor -- start 2024-05-21T22:01:43.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.210+0000 7f99fc000700 1 -- start start 2024-05-21T22:01:43.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.210+0000 7f99fc000700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f99f41035f0 0x7f99f419bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:43.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.211+0000 7f99fc000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f99f4103f90 0x7f99f419c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:43.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.211+0000 7f99fc000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f99f4108500 0x7f99f41a0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:43.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.211+0000 7f99fc000700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f99f4113790 con 0x7f99f4103f90 2024-05-21T22:01:43.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.211+0000 7f99fc000700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f99f4113610 con 0x7f99f4108500 2024-05-21T22:01:43.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.211+0000 7f99fc000700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f99f4113910 con 0x7f99f41035f0 2024-05-21T22:01:43.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.211+0000 7f99f8c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f99f4103f90 0x7f99f419c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:43.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.211+0000 7f99f9600700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f99f41035f0 0x7f99f419bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:43.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.211+0000 7f99f8c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f99f4103f90 0x7f99f419c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:34066/0 (socket says 172.21.15.151:34066) 2024-05-21T22:01:43.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.211+0000 7f99f8c00700 1 -- 172.21.15.151:0/2266142516 learned_addr learned my addr 172.21.15.151:0/2266142516 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:01:43.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.211+0000 7f99fa000700 1 --2- 172.21.15.151:0/2266142516 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f99f4108500 0x7f99f41a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:43.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.211+0000 7f99f8c00700 1 -- 172.21.15.151:0/2266142516 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f99f41035f0 msgr2=0x7f99f419bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:43.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.211+0000 7f99f8c00700 1 --2- 172.21.15.151:0/2266142516 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f99f41035f0 0x7f99f419bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:43.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.211+0000 7f99f8c00700 1 -- 172.21.15.151:0/2266142516 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f99f4108500 msgr2=0x7f99f41a0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:43.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.211+0000 7f99f8c00700 1 --2- 172.21.15.151:0/2266142516 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f99f4108500 0x7f99f41a0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:43.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.211+0000 7f99f8c00700 1 -- 172.21.15.151:0/2266142516 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99f41a0eb0 con 0x7f99f4103f90 2024-05-21T22:01:43.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.211+0000 7f99fa000700 1 --2- 172.21.15.151:0/2266142516 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f99f4108500 0x7f99f41a0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:01:43.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.212+0000 7f99f8c00700 1 --2- 172.21.15.151:0/2266142516 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f99f4103f90 0x7f99f419c3e0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f99e000e9a0 tx=0x7f99e000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:01:43.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.212+0000 7f99f2000700 1 -- 172.21.15.151:0/2266142516 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e000ca10 con 0x7f99f4103f90 2024-05-21T22:01:43.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.212+0000 7f99f2000700 1 -- 172.21.15.151:0/2266142516 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99e000cbb0 con 0x7f99f4103f90 2024-05-21T22:01:43.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.212+0000 7f99fc000700 1 -- 172.21.15.151:0/2266142516 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99f41a11a0 con 0x7f99f4103f90 2024-05-21T22:01:43.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.212+0000 7f99f2000700 1 -- 172.21.15.151:0/2266142516 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e0018680 con 0x7f99f4103f90 2024-05-21T22:01:43.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.212+0000 7f99fc000700 1 -- 172.21.15.151:0/2266142516 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99f41a8a20 con 0x7f99f4103f90 2024-05-21T22:01:43.216 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.216+0000 7f99f2000700 1 -- 172.21.15.151:0/2266142516 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7f99e0010430 con 0x7f99f4103f90 2024-05-21T22:01:43.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.217+0000 7f9a06968b80 1 Processor -- start 2024-05-21T22:01:43.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.217+0000 7f9a06968b80 1 -- start start 2024-05-21T22:01:43.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.218+0000 7f9a06968b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d5632d4eb0 0x55d5632d5290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:43.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.218+0000 7f9a06968b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d5632d57d0 0x55d5632cf260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:43.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.218+0000 7f9a06968b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d5632cf880 0x55d5632cfce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:43.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.218+0000 7f9a06968b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d56322de40 con 0x55d5632d4eb0 2024-05-21T22:01:43.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.218+0000 7f9a06968b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55d56321d9e0 con 0x55d5632cf880 2024-05-21T22:01:43.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.218+0000 7f9a06968b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55d563221a20 con 0x55d5632d57d0 2024-05-21T22:01:43.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.218+0000 7f99f2000700 1 --2- 172.21.15.151:0/2266142516 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f99dc0b29f0 0x7f99dc0b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:43.219 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.218+0000 7f99f9600700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d5632d57d0 0x55d5632cf260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:43.219 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.218+0000 7f99f2000700 1 -- 172.21.15.151:0/2266142516 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f99e00e27c0 con 0x7f99f4103f90 2024-05-21T22:01:43.219 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.218+0000 7f99f9600700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d5632d57d0 0x55d5632cf260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:40408/0 (socket says 172.21.15.151:40408) 2024-05-21T22:01:43.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.218+0000 7f99f9600700 1 -- 172.21.15.151:0/1491271408 learned_addr learned my addr 172.21.15.151:0/1491271408 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:01:43.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.218+0000 7f99fa000700 1 --2- 172.21.15.151:0/1491271408 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d5632d4eb0 0x55d5632d5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:43.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.218+0000 7f99f8c00700 1 --2- 172.21.15.151:0/1491271408 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d5632cf880 0x55d5632cfce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:43.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.219+0000 7f99f9600700 1 -- 172.21.15.151:0/1491271408 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d5632cf880 msgr2=0x55d5632cfce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:43.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.219+0000 7f99f9600700 1 --2- 172.21.15.151:0/1491271408 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d5632cf880 0x55d5632cfce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:43.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.219+0000 7f99f9600700 1 -- 172.21.15.151:0/1491271408 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d5632d4eb0 msgr2=0x55d5632d5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:43.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.219+0000 7f99f9600700 1 --2- 172.21.15.151:0/1491271408 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d5632d4eb0 0x55d5632d5290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:43.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.219+0000 7f99fa000700 1 --2- 172.21.15.151:0/2266142516 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f99dc0b29f0 0x7f99dc0b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:43.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.219+0000 7f99f9600700 1 -- 172.21.15.151:0/1491271408 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5631e6420 con 0x55d5632d57d0 2024-05-21T22:01:43.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.219+0000 7f99f9600700 1 --2- 172.21.15.151:0/1491271408 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d5632d57d0 0x55d5632cf260 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f99e8009cb0 tx=0x7f99e8011440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=833edd769b81f9ec server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:01:43.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.219+0000 7f99e6a00700 1 -- 172.21.15.151:0/1491271408 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e801e070 con 0x55d5632d57d0 2024-05-21T22:01:43.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.219+0000 7f99e6a00700 1 -- 172.21.15.151:0/1491271408 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99e8011cc0 con 0x55d5632d57d0 2024-05-21T22:01:43.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.219+0000 7f99e6a00700 1 -- 172.21.15.151:0/1491271408 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e800e730 con 0x55d5632d57d0 2024-05-21T22:01:43.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.220+0000 7f9a06968b80 1 -- 172.21.15.151:0/1491271408 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d5632d57d0 msgr2=0x55d5632cf260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:43.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.220+0000 7f9a06968b80 1 --2- 172.21.15.151:0/1491271408 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d5632d57d0 0x55d5632cf260 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f99e8009cb0 tx=0x7f99e8011440 comp rx=0 tx=0).stop 2024-05-21T22:01:43.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.220+0000 7f99fa000700 1 --2- 172.21.15.151:0/2266142516 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f99dc0b29f0 0x7f99dc0b4eb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f99ec00b780 tx=0x7f99ec006210 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:01:43.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.220+0000 7f9a06968b80 1 -- 172.21.15.151:0/1491271408 shutdown_connections 2024-05-21T22:01:43.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.220+0000 7f9a06968b80 1 --2- 172.21.15.151:0/1491271408 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d5632cf880 0x55d5632cfce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:43.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.220+0000 7f9a06968b80 1 --2- 172.21.15.151:0/1491271408 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d5632d4eb0 0x55d5632d5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:43.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.220+0000 7f9a06968b80 1 --2- 172.21.15.151:0/1491271408 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d5632d57d0 0x55d5632cf260 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:43.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.220+0000 7f9a06968b80 1 -- 172.21.15.151:0/1491271408 >> 172.21.15.151:0/1491271408 conn(0x55d5631fdbe0 msgr2=0x55d5632ca010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:01:43.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.220+0000 7f9a06968b80 1 -- 172.21.15.151:0/1491271408 shutdown_connections 2024-05-21T22:01:43.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.220+0000 7f9a06968b80 1 -- 172.21.15.151:0/1491271408 wait complete. 2024-05-21T22:01:43.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.221+0000 7f9a06968b80 1 -- start start 2024-05-21T22:01:43.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.221+0000 7f9a06968b80 1 Processor -- start 2024-05-21T22:01:43.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.222+0000 7f9a06968b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d5632cf880 0x55d5632db4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:43.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.222+0000 7f9a06968b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d5632d4eb0 0x55d5632d8840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:43.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.222+0000 7f9a06968b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d5632d57d0 0x55d5632d8d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:43.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.222+0000 7f9a06968b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d563123fa0 con 0x55d5632d57d0 2024-05-21T22:01:43.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.222+0000 7f9a06968b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55d5631be760 con 0x55d5632d4eb0 2024-05-21T22:01:43.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.222+0000 7f9a06968b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55d5632dc240 con 0x55d5632cf880 2024-05-21T22:01:43.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.222+0000 7f9a06968b80 10 client.0 _ll_register_callbacks cb 0x55d5631c7c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:01:43.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.222+0000 7f99f9600700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d5632cf880 0x55d5632db4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:43.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.222+0000 7f99f9600700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d5632cf880 0x55d5632db4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:40418/0 (socket says 172.21.15.151:40418) 2024-05-21T22:01:43.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.222+0000 7f99f9600700 1 -- 172.21.15.151:0/1611580789 learned_addr learned my addr 172.21.15.151:0/1611580789 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:01:43.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.222+0000 7f99fa000700 1 --2- 172.21.15.151:0/1611580789 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d5632d57d0 0x55d5632d8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:43.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.222+0000 7f99f8c00700 1 --2- 172.21.15.151:0/1611580789 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d5632d4eb0 0x55d5632d8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:43.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.223+0000 7f99f9600700 1 -- 172.21.15.151:0/1611580789 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d5632d4eb0 msgr2=0x55d5632d8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:43.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.223+0000 7f99f9600700 1 --2- 172.21.15.151:0/1611580789 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d5632d4eb0 0x55d5632d8840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:43.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.223+0000 7f99f9600700 1 -- 172.21.15.151:0/1611580789 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d5632d57d0 msgr2=0x55d5632d8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:43.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.223+0000 7f99f9600700 1 --2- 172.21.15.151:0/1611580789 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d5632d57d0 0x55d5632d8d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:43.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.223+0000 7f99f9600700 1 -- 172.21.15.151:0/1611580789 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d56316d8d0 con 0x55d5632cf880 2024-05-21T22:01:43.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.223+0000 7f99f9600700 1 --2- 172.21.15.151:0/1611580789 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d5632cf880 0x55d5632db4a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f99e8000c00 tx=0x7f99e8005710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f74fb301c5a79465 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:01:43.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.223+0000 7f99db400700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-21T22:01:43.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.223+0000 7f99db400700 1 -- 172.21.15.151:0/1611580789 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e801e050 con 0x55d5632cf880 2024-05-21T22:01:43.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.223+0000 7f99db400700 1 -- 172.21.15.151:0/1611580789 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f99e8019640 con 0x55d5632cf880 2024-05-21T22:01:43.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.223+0000 7f9a06968b80 1 -- 172.21.15.151:0/1611580789 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99e8011cc0 con 0x55d5632cf880 2024-05-21T22:01:43.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.224+0000 7f99db400700 1 -- 172.21.15.151:0/1611580789 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f99e8033050 con 0x55d5632cf880 2024-05-21T22:01:43.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.224+0000 7f9a06968b80 10 client.26662 fetch_fsmap learned FSMap version 29 2024-05-21T22:01:43.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.224+0000 7f9a06968b80 1 -- 172.21.15.151:0/1611580789 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d5631ee060 con 0x55d5632cf880 2024-05-21T22:01:43.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.224+0000 7f99db400700 1 -- 172.21.15.151:0/1611580789 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99e8011cc0 con 0x55d5632cf880 2024-05-21T22:01:43.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.224+0000 7f99db400700 1 -- 172.21.15.151:0/1611580789 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f99e8019960 con 0x55d5632cf880 2024-05-21T22:01:43.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.224+0000 7f9a06968b80 10 client.26662 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:01:43.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.224+0000 7f9a06968b80 10 client.26662 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-05-21T22:01:43.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.224+0000 7f9a06968b80 20 client.26662 populate_metadata read hostname 'smithi151' 2024-05-21T22:01:43.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.224+0000 7f9a06968b80 1 --2- 172.21.15.151:0/1611580789 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x55d5632e8c70 0x55d5632eb060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:43.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.224+0000 7f9a06968b80 4 client.26662 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T22:01:43.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.224+0000 7f9a06968b80 1 -- 172.21.15.151:0/1611580789 --> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d56322fd90 con 0x55d5632e8c70 2024-05-21T22:01:43.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.225+0000 7f99f8c00700 1 --2- 172.21.15.151:0/1611580789 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x55d5632e8c70 0x55d5632eb060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:43.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.225+0000 7f99f8c00700 1 --2- 172.21.15.151:0/1611580789 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x55d5632e8c70 0x55d5632eb060 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=881966fe570e1691 server_cookie=524fd2fca40b8d84 in_seq=0 out_seq=0 2024-05-21T22:01:43.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.225+0000 7f99db400700 10 client.26662 ms_handle_connect on v2:172.21.15.164:6834/2340428470 2024-05-21T22:01:43.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.226+0000 7f99db400700 1 -- 172.21.15.151:0/1611580789 <== mds.4 v2:172.21.15.164:6834/2340428470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55d56322fd90 con 0x55d5632e8c70 2024-05-21T22:01:43.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.226+0000 7f99db400700 10 client.26662 handle_command_reply: tid=0 2024-05-21T22:01:43.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.226+0000 7f9a06968b80 1 client.26662 shutdown 2024-05-21T22:01:43.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.226+0000 7f9a06968b80 1 -- 172.21.15.151:0/1611580789 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d5632cf880 msgr2=0x55d5632db4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:43.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.226+0000 7f9a06968b80 1 --2- 172.21.15.151:0/1611580789 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d5632cf880 0x55d5632db4a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f99e8000c00 tx=0x7f99e8005710 comp rx=0 tx=0).stop 2024-05-21T22:01:43.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.227+0000 7f9a06968b80 1 -- 172.21.15.151:0/1611580789 shutdown_connections 2024-05-21T22:01:43.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.227+0000 7f9a06968b80 1 --2- 172.21.15.151:0/1611580789 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x55d5632e8c70 0x55d5632eb060 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:43.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.227+0000 7f9a06968b80 1 --2- 172.21.15.151:0/1611580789 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d5632d57d0 0x55d5632d8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:43.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.227+0000 7f9a06968b80 1 --2- 172.21.15.151:0/1611580789 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d5632cf880 0x55d5632db4a0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:43.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.227+0000 7f9a06968b80 1 --2- 172.21.15.151:0/1611580789 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d5632d4eb0 0x55d5632d8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:43.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.227+0000 7f9a06968b80 1 -- 172.21.15.151:0/1611580789 >> 172.21.15.151:0/1611580789 conn(0x55d5631fdbe0 msgr2=0x55d5632ce250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:01:43.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.227+0000 7f99db400700 0 client.26662 ms_handle_reset on v2:172.21.15.164:6834/2340428470 2024-05-21T22:01:43.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.227+0000 7f9a06968b80 1 -- 172.21.15.151:0/1611580789 shutdown_connections 2024-05-21T22:01:43.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.227+0000 7f9a06968b80 1 -- 172.21.15.151:0/1611580789 wait complete. 2024-05-21T22:01:43.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.227+0000 7f9a06968b80 20 client.26662 trim_cache size 0 max 16384 2024-05-21T22:01:43.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.243+0000 7f9a06968b80 1 Processor -- start 2024-05-21T22:01:43.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.243+0000 7f9a06968b80 1 -- start start 2024-05-21T22:01:43.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.244+0000 7f9a06968b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d5632cf880 0x55d5632cf260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:43.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.244+0000 7f9a06968b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d5632d4eb0 0x55d5632dacb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:43.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.244+0000 7f9a06968b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d5632d57d0 0x55d5632c99e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:43.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.244+0000 7f9a06968b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f99f41b0100 con 0x55d5632d4eb0 2024-05-21T22:01:43.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.244+0000 7f9a06968b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f99f41b1c40 con 0x55d5632d57d0 2024-05-21T22:01:43.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.244+0000 7f9a06968b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f99f41b07d0 con 0x55d5632cf880 2024-05-21T22:01:43.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.244+0000 7f99fa000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d5632d57d0 0x55d5632c99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:43.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.244+0000 7f99f8c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d5632d4eb0 0x55d5632dacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:43.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.244+0000 7f99fa000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d5632d57d0 0x55d5632c99e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:55494/0 (socket says 172.21.15.151:55494) 2024-05-21T22:01:43.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.244+0000 7f99fa000700 1 -- 172.21.15.151:0/1631266458 learned_addr learned my addr 172.21.15.151:0/1631266458 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:01:43.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.244+0000 7f99f9600700 1 --2- 172.21.15.151:0/1631266458 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d5632cf880 0x55d5632cf260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:43.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.245+0000 7f99fa000700 1 -- 172.21.15.151:0/1631266458 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d5632cf880 msgr2=0x55d5632cf260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:43.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.245+0000 7f99fa000700 1 --2- 172.21.15.151:0/1631266458 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d5632cf880 0x55d5632cf260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:43.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.245+0000 7f99fa000700 1 -- 172.21.15.151:0/1631266458 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d5632d4eb0 msgr2=0x55d5632dacb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:43.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.245+0000 7f99fa000700 1 --2- 172.21.15.151:0/1631266458 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d5632d4eb0 0x55d5632dacb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:43.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.245+0000 7f99fa000700 1 -- 172.21.15.151:0/1631266458 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5632d6dd0 con 0x55d5632d57d0 2024-05-21T22:01:43.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.245+0000 7f99fa000700 1 --2- 172.21.15.151:0/1631266458 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d5632d57d0 0x55d5632c99e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f99ec00d010 tx=0x7f99ec006020 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1aed0d859671d802 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:01:43.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.245+0000 7f99e6000700 1 -- 172.21.15.151:0/1631266458 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99ec007b40 con 0x55d5632d57d0 2024-05-21T22:01:43.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.245+0000 7f99e6000700 1 -- 172.21.15.151:0/1631266458 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99ec023040 con 0x55d5632d57d0 2024-05-21T22:01:43.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.245+0000 7f99e6000700 1 -- 172.21.15.151:0/1631266458 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99ec01e430 con 0x55d5632d57d0 2024-05-21T22:01:43.247 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.246+0000 7f9a06968b80 1 -- 172.21.15.151:0/1631266458 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d5632d57d0 msgr2=0x55d5632c99e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:43.247 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.246+0000 7f9a06968b80 1 --2- 172.21.15.151:0/1631266458 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d5632d57d0 0x55d5632c99e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f99ec00d010 tx=0x7f99ec006020 comp rx=0 tx=0).stop 2024-05-21T22:01:43.247 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.246+0000 7f9a06968b80 1 -- 172.21.15.151:0/1631266458 shutdown_connections 2024-05-21T22:01:43.247 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.246+0000 7f9a06968b80 1 --2- 172.21.15.151:0/1631266458 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d5632d57d0 0x55d5632c99e0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:43.247 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.246+0000 7f9a06968b80 1 --2- 172.21.15.151:0/1631266458 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d5632cf880 0x55d5632cf260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:43.247 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.246+0000 7f9a06968b80 1 --2- 172.21.15.151:0/1631266458 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d5632d4eb0 0x55d5632dacb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:43.247 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.246+0000 7f9a06968b80 1 -- 172.21.15.151:0/1631266458 >> 172.21.15.151:0/1631266458 conn(0x55d5631fdbe0 msgr2=0x55d5632d29c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:01:43.247 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.246+0000 7f9a06968b80 1 -- 172.21.15.151:0/1631266458 shutdown_connections 2024-05-21T22:01:43.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.246+0000 7f9a06968b80 1 -- 172.21.15.151:0/1631266458 wait complete. 2024-05-21T22:01:43.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.247+0000 7f9a06968b80 1 -- start start 2024-05-21T22:01:43.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.247+0000 7f9a06968b80 1 Processor -- start 2024-05-21T22:01:43.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.247+0000 7f9a06968b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d5632cf880 0x55d5632e1a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:43.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.247+0000 7f9a06968b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d5632d4eb0 0x55d5632eff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:43.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.247+0000 7f9a06968b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d5632d57d0 0x55d5632f0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:43.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.247+0000 7f9a06968b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f99f41b1570 con 0x55d5632d4eb0 2024-05-21T22:01:43.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.247+0000 7f9a06968b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f99f41b0ea0 con 0x55d5632d57d0 2024-05-21T22:01:43.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.247+0000 7f9a06968b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f99f41ae0a0 con 0x55d5632cf880 2024-05-21T22:01:43.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.247+0000 7f9a06968b80 10 client.0 _ll_register_callbacks cb 0x55d5631c7c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:01:43.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.248+0000 7f99f9600700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d5632cf880 0x55d5632e1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:43.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.248+0000 7f99f8c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d5632d4eb0 0x55d5632eff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:43.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.248+0000 7f99f9600700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d5632cf880 0x55d5632e1a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:40436/0 (socket says 172.21.15.151:40436) 2024-05-21T22:01:43.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.248+0000 7f99f9600700 1 -- 172.21.15.151:0/154253746 learned_addr learned my addr 172.21.15.151:0/154253746 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:01:43.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.248+0000 7f99fa000700 1 --2- 172.21.15.151:0/154253746 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d5632d57d0 0x55d5632f0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:43.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.248+0000 7f99f9600700 1 -- 172.21.15.151:0/154253746 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d5632d57d0 msgr2=0x55d5632f0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:43.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.248+0000 7f99f9600700 1 --2- 172.21.15.151:0/154253746 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d5632d57d0 0x55d5632f0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:43.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.248+0000 7f99f9600700 1 -- 172.21.15.151:0/154253746 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d5632d4eb0 msgr2=0x55d5632eff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:43.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.248+0000 7f99f9600700 1 --2- 172.21.15.151:0/154253746 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d5632d4eb0 0x55d5632eff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:43.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.248+0000 7f99f9600700 1 -- 172.21.15.151:0/154253746 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5632f0b60 con 0x55d5632cf880 2024-05-21T22:01:43.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.248+0000 7f99f8c00700 1 --2- 172.21.15.151:0/154253746 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d5632d4eb0 0x55d5632eff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:01:43.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.248+0000 7f99f9600700 1 --2- 172.21.15.151:0/154253746 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d5632cf880 0x55d5632e1a30 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f99e8000c00 tx=0x7f99e80259d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f0bee75f47aa7ca server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:01:43.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.248+0000 7f99db400700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-21T22:01:43.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.249+0000 7f99db400700 1 -- 172.21.15.151:0/154253746 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e801e070 con 0x55d5632cf880 2024-05-21T22:01:43.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.249+0000 7f9a06968b80 1 -- 172.21.15.151:0/154253746 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99ec023040 con 0x55d5632cf880 2024-05-21T22:01:43.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.249+0000 7f99db400700 1 -- 172.21.15.151:0/154253746 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f99e8011020 con 0x55d5632cf880 2024-05-21T22:01:43.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.249+0000 7f99db400700 1 -- 172.21.15.151:0/154253746 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f99e8011220 con 0x55d5632cf880 2024-05-21T22:01:43.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.249+0000 7f9a06968b80 10 client.26668 fetch_fsmap learned FSMap version 29 2024-05-21T22:01:43.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.249+0000 7f9a06968b80 1 -- 172.21.15.151:0/154253746 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d563353540 con 0x55d5632cf880 2024-05-21T22:01:43.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.249+0000 7f99db400700 1 -- 172.21.15.151:0/154253746 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99ec023040 con 0x55d5632cf880 2024-05-21T22:01:43.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.249+0000 7f99db400700 1 -- 172.21.15.151:0/154253746 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f99e8003b80 con 0x55d5632cf880 2024-05-21T22:01:43.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.249+0000 7f9a06968b80 10 client.26668 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:01:43.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.249+0000 7f9a06968b80 10 client.26668 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-05-21T22:01:43.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.249+0000 7f9a06968b80 20 client.26668 populate_metadata read hostname 'smithi151' 2024-05-21T22:01:43.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.250+0000 7f9a06968b80 1 --2- 172.21.15.151:0/154253746 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x55d563353bb0 0x55d563355fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:43.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.250+0000 7f9a06968b80 4 client.26668 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T22:01:43.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.250+0000 7f9a06968b80 1 -- 172.21.15.151:0/154253746 --> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d563229570 con 0x55d563353bb0 2024-05-21T22:01:43.253 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.250+0000 7f99f8c00700 1 --2- 172.21.15.151:0/154253746 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x55d563353bb0 0x55d563355fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:43.253 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.250+0000 7f99f8c00700 1 --2- 172.21.15.151:0/154253746 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x55d563353bb0 0x55d563355fd0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ec6f5f8cd61a8a6b server_cookie=e98c53395129fb2c in_seq=0 out_seq=0 2024-05-21T22:01:43.253 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.250+0000 7f99db400700 10 client.26668 ms_handle_connect on v2:172.21.15.164:6834/2340428470 2024-05-21T22:01:43.253 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.251+0000 7f99db400700 1 -- 172.21.15.151:0/154253746 <== mds.4 v2:172.21.15.164:6834/2340428470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d563229570 con 0x55d563353bb0 2024-05-21T22:01:43.253 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.251+0000 7f99db400700 10 client.26668 handle_command_reply: tid=0 2024-05-21T22:01:43.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.251+0000 7f9a06968b80 1 client.26668 shutdown 2024-05-21T22:01:43.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.251+0000 7f9a06968b80 1 -- 172.21.15.151:0/154253746 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d5632cf880 msgr2=0x55d5632e1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:43.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.251+0000 7f9a06968b80 1 --2- 172.21.15.151:0/154253746 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d5632cf880 0x55d5632e1a30 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f99e8000c00 tx=0x7f99e80259d0 comp rx=0 tx=0).stop 2024-05-21T22:01:43.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.251+0000 7f9a06968b80 1 -- 172.21.15.151:0/154253746 shutdown_connections 2024-05-21T22:01:43.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.251+0000 7f9a06968b80 1 --2- 172.21.15.151:0/154253746 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x55d563353bb0 0x55d563355fd0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:43.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.251+0000 7f9a06968b80 1 --2- 172.21.15.151:0/154253746 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d5632d57d0 0x55d5632f0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:43.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.251+0000 7f9a06968b80 1 --2- 172.21.15.151:0/154253746 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d5632cf880 0x55d5632e1a30 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:43.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.251+0000 7f99db400700 0 client.26668 ms_handle_reset on v2:172.21.15.164:6834/2340428470 2024-05-21T22:01:43.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.251+0000 7f9a06968b80 1 --2- 172.21.15.151:0/154253746 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d5632d4eb0 0x55d5632eff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:43.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.251+0000 7f9a06968b80 1 -- 172.21.15.151:0/154253746 >> 172.21.15.151:0/154253746 conn(0x55d5631fdbe0 msgr2=0x55d5632c99e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:01:43.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.251+0000 7f9a06968b80 1 -- 172.21.15.151:0/154253746 shutdown_connections 2024-05-21T22:01:43.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.251+0000 7f9a06968b80 1 -- 172.21.15.151:0/154253746 wait complete. 2024-05-21T22:01:43.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.251+0000 7f9a06968b80 20 client.26668 trim_cache size 0 max 16384 2024-05-21T22:01:43.255 INFO:teuthology.orchestra.run.smithi151.stdout:[] 2024-05-21T22:01:43.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.252+0000 7f99fc000700 1 -- 172.21.15.151:0/2266142516 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f99dc0b29f0 msgr2=0x7f99dc0b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:43.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.252+0000 7f99fc000700 1 --2- 172.21.15.151:0/2266142516 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f99dc0b29f0 0x7f99dc0b4eb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f99ec00b780 tx=0x7f99ec006210 comp rx=0 tx=0).stop 2024-05-21T22:01:43.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.252+0000 7f99fc000700 1 -- 172.21.15.151:0/2266142516 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f99f4103f90 msgr2=0x7f99f419c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:43.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.252+0000 7f99fc000700 1 --2- 172.21.15.151:0/2266142516 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f99f4103f90 0x7f99f419c3e0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f99e000e9a0 tx=0x7f99e000ecb0 comp rx=0 tx=0).stop 2024-05-21T22:01:43.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.252+0000 7f99fc000700 1 -- 172.21.15.151:0/2266142516 shutdown_connections 2024-05-21T22:01:43.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.252+0000 7f99fc000700 1 --2- 172.21.15.151:0/2266142516 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f99dc0b29f0 0x7f99dc0b4eb0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:43.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.252+0000 7f99fc000700 1 --2- 172.21.15.151:0/2266142516 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f99f4108500 0x7f99f41a0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:43.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.252+0000 7f99fc000700 1 --2- 172.21.15.151:0/2266142516 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f99f41035f0 0x7f99f419bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:43.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.252+0000 7f99fc000700 1 --2- 172.21.15.151:0/2266142516 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f99f4103f90 0x7f99f419c3e0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:43.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.252+0000 7f99fc000700 1 -- 172.21.15.151:0/2266142516 >> 172.21.15.151:0/2266142516 conn(0x7f99f407a6a0 msgr2=0x7f99f4101460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:01:43.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.252+0000 7f99fc000700 1 -- 172.21.15.151:0/2266142516 shutdown_connections 2024-05-21T22:01:43.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:43.252+0000 7f99fc000700 1 -- 172.21.15.151:0/2266142516 wait complete. 2024-05-21T22:01:43.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:01:43 smithi151 ceph-mon[31623]: pgmap v186: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-05-21T22:01:43.985 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:01:43 smithi184 ceph-mon[36942]: pgmap v186: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-05-21T22:01:44.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:01:43 smithi164 ceph-mon[36269]: pgmap v186: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-05-21T22:01:44.148 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-21T22:01:44.592 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:01:45.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:01:45 smithi151 ceph-mon[31623]: pgmap v187: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-05-21T22:01:45.985 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:01:45 smithi184 ceph-mon[36942]: pgmap v187: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-05-21T22:01:46.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:01:45 smithi164 ceph-mon[36269]: pgmap v187: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-05-21T22:01:46.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.849+0000 7efe06400700 1 -- 172.21.15.151:0/557098906 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe000758e0 msgr2=0x7efe00075cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:46.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.849+0000 7efe06400700 1 --2- 172.21.15.151:0/557098906 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe000758e0 0x7efe00075cc0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7efdf400b0a0 tx=0x7efdf400b3b0 comp rx=0 tx=0).stop 2024-05-21T22:01:46.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.849+0000 7efe06400700 1 -- 172.21.15.151:0/557098906 shutdown_connections 2024-05-21T22:01:46.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.849+0000 7efe06400700 1 --2- 172.21.15.151:0/557098906 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efe00114480 0x7efe00116870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:46.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.849+0000 7efe06400700 1 --2- 172.21.15.151:0/557098906 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe000758e0 0x7efe00075cc0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:46.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.849+0000 7efe06400700 1 --2- 172.21.15.151:0/557098906 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe00076200 0x7efe00113f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:46.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.849+0000 7efe06400700 1 -- 172.21.15.151:0/557098906 >> 172.21.15.151:0/557098906 conn(0x7efe000fefc0 msgr2=0x7efe001013e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:01:46.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.849+0000 7efe06400700 1 -- 172.21.15.151:0/557098906 shutdown_connections 2024-05-21T22:01:46.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.849+0000 7efe06400700 1 -- 172.21.15.151:0/557098906 wait complete. 2024-05-21T22:01:46.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.850+0000 7efe06400700 1 Processor -- start 2024-05-21T22:01:46.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.851+0000 7efe06400700 1 -- start start 2024-05-21T22:01:46.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.851+0000 7efe06400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efe000758e0 0x7efe001a0210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:46.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.851+0000 7efe06400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe00076200 0x7efe001a0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:46.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.851+0000 7efe06400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe00114480 0x7efe001a4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:46.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.851+0000 7efe06400700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7efe00119cc0 con 0x7efe000758e0 2024-05-21T22:01:46.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.851+0000 7efe06400700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7efe00119b40 con 0x7efe00076200 2024-05-21T22:01:46.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.851+0000 7efe06400700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7efe00119e40 con 0x7efe00114480 2024-05-21T22:01:46.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.851+0000 7efe05000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efe000758e0 0x7efe001a0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:46.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.851+0000 7efdffe00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe00076200 0x7efe001a0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:46.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.851+0000 7efe05a00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe00114480 0x7efe001a4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:46.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.851+0000 7efe05000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efe000758e0 0x7efe001a0210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:51848/0 (socket says 172.21.15.151:51848) 2024-05-21T22:01:46.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.851+0000 7efe05000700 1 -- 172.21.15.151:0/3402911013 learned_addr learned my addr 172.21.15.151:0/3402911013 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:01:46.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.851+0000 7efdffe00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe00076200 0x7efe001a0750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:44014/0 (socket says 172.21.15.151:44014) 2024-05-21T22:01:46.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.852+0000 7efdffe00700 1 -- 172.21.15.151:0/3402911013 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe00114480 msgr2=0x7efe001a4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:46.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.852+0000 7efdffe00700 1 --2- 172.21.15.151:0/3402911013 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe00114480 0x7efe001a4ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:46.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.852+0000 7efdffe00700 1 -- 172.21.15.151:0/3402911013 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efe000758e0 msgr2=0x7efe001a0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:46.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.852+0000 7efdffe00700 1 --2- 172.21.15.151:0/3402911013 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efe000758e0 0x7efe001a0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:46.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.852+0000 7efdffe00700 1 -- 172.21.15.151:0/3402911013 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe001a5220 con 0x7efe00076200 2024-05-21T22:01:46.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.852+0000 7efe05a00700 1 --2- 172.21.15.151:0/3402911013 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe00114480 0x7efe001a4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:01:46.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.852+0000 7efdffe00700 1 --2- 172.21.15.151:0/3402911013 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe00076200 0x7efe001a0750 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7efdec00e9f0 tx=0x7efdec00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:01:46.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.852+0000 7efdfd600700 1 -- 172.21.15.151:0/3402911013 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdec004220 con 0x7efe00076200 2024-05-21T22:01:46.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.852+0000 7efdfd600700 1 -- 172.21.15.151:0/3402911013 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efdec0043c0 con 0x7efe00076200 2024-05-21T22:01:46.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.852+0000 7efe06400700 1 -- 172.21.15.151:0/3402911013 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe001a5510 con 0x7efe00076200 2024-05-21T22:01:46.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.853+0000 7efdfd600700 1 -- 172.21.15.151:0/3402911013 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdec00ce60 con 0x7efe00076200 2024-05-21T22:01:46.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.853+0000 7efe06400700 1 -- 172.21.15.151:0/3402911013 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe001acd90 con 0x7efe00076200 2024-05-21T22:01:46.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.856+0000 7efdfd600700 1 -- 172.21.15.151:0/3402911013 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7efdec0052d0 con 0x7efe00076200 2024-05-21T22:01:46.858 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.857+0000 7efe1240cb80 1 Processor -- start 2024-05-21T22:01:46.858 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.857+0000 7efe1240cb80 1 -- start start 2024-05-21T22:01:46.858 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.858+0000 7efdfd600700 1 --2- 172.21.15.151:0/3402911013 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7efde80b2a70 0x7efde80b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:46.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.858+0000 7efe1240cb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b0382157c0 0x55b038215ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:46.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.858+0000 7efe1240cb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b0382160e0 0x55b03820fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:46.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.858+0000 7efe1240cb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b038210390 0x55b0382107f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:46.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.858+0000 7efe1240cb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55b03816de40 con 0x55b0382160e0 2024-05-21T22:01:46.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.858+0000 7efe1240cb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55b03815d9e0 con 0x55b0382157c0 2024-05-21T22:01:46.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.858+0000 7efe1240cb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55b038161a20 con 0x55b038210390 2024-05-21T22:01:46.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.858+0000 7efdfd600700 1 -- 172.21.15.151:0/3402911013 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7efdec0e2750 con 0x7efe00076200 2024-05-21T22:01:46.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.858+0000 7efe05a00700 1 --2- 172.21.15.151:0/3402911013 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7efde80b2a70 0x7efde80b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:46.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.858+0000 7efe05000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b0382157c0 0x55b038215ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:46.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.858+0000 7efdffe00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b0382160e0 0x55b03820fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:46.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.858+0000 7efe05000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b0382157c0 0x55b038215ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:44020/0 (socket says 172.21.15.151:44020) 2024-05-21T22:01:46.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.858+0000 7efe05000700 1 -- 172.21.15.151:0/3791743834 learned_addr learned my addr 172.21.15.151:0/3791743834 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:01:46.861 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.859+0000 7efe05a00700 1 --2- 172.21.15.151:0/3791743834 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b038210390 0x55b0382107f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:46.861 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.859+0000 7efdffe00700 1 -- 172.21.15.151:0/3791743834 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b038210390 msgr2=0x55b0382107f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:46.861 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.859+0000 7efdffe00700 1 --2- 172.21.15.151:0/3791743834 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b038210390 0x55b0382107f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:46.861 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.859+0000 7efdffe00700 1 -- 172.21.15.151:0/3791743834 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b0382157c0 msgr2=0x55b038215ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:46.861 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.859+0000 7efdffe00700 1 --2- 172.21.15.151:0/3791743834 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b0382157c0 0x55b038215ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:46.861 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.859+0000 7efdffe00700 1 -- 172.21.15.151:0/3791743834 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b038126420 con 0x55b0382160e0 2024-05-21T22:01:46.861 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.859+0000 7efe05000700 1 --2- 172.21.15.151:0/3791743834 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b0382157c0 0x55b038215ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:01:46.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.859+0000 7efe05a00700 1 --2- 172.21.15.151:0/3791743834 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b038210390 0x55b0382107f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T22:01:46.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.859+0000 7efdffe00700 1 --2- 172.21.15.151:0/3791743834 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b0382160e0 0x55b03820fd70 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7efdec0dfe50 tx=0x7efdec0b22d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a4b9e54ebe2f8d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:01:46.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.859+0000 7efe05a00700 1 --2- 172.21.15.151:0/3402911013 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7efde80b2a70 0x7efde80b4f30 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7efdf8009860 tx=0x7efdf80097f0 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:01:46.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.859+0000 7efdf2000700 1 -- 172.21.15.151:0/3791743834 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdec01c040 con 0x55b0382160e0 2024-05-21T22:01:46.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.860+0000 7efdf2000700 1 -- 172.21.15.151:0/3791743834 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efdec0a99e0 con 0x55b0382160e0 2024-05-21T22:01:46.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.860+0000 7efdf2000700 1 -- 172.21.15.151:0/3791743834 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdec0a9ea0 con 0x55b0382160e0 2024-05-21T22:01:46.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.860+0000 7efe1240cb80 1 -- 172.21.15.151:0/3791743834 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b0382160e0 msgr2=0x55b03820fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:46.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.860+0000 7efe1240cb80 1 --2- 172.21.15.151:0/3791743834 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b0382160e0 0x55b03820fd70 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7efdec0dfe50 tx=0x7efdec0b22d0 comp rx=0 tx=0).stop 2024-05-21T22:01:46.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.860+0000 7efe1240cb80 1 -- 172.21.15.151:0/3791743834 shutdown_connections 2024-05-21T22:01:46.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.860+0000 7efe1240cb80 1 --2- 172.21.15.151:0/3791743834 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b0382157c0 0x55b038215ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:46.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.860+0000 7efe1240cb80 1 --2- 172.21.15.151:0/3791743834 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b038210390 0x55b0382107f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:46.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.860+0000 7efe1240cb80 1 --2- 172.21.15.151:0/3791743834 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b0382160e0 0x55b03820fd70 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:46.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.860+0000 7efe1240cb80 1 -- 172.21.15.151:0/3791743834 >> 172.21.15.151:0/3791743834 conn(0x55b03813dbe0 msgr2=0x55b03820a810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:01:46.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.861+0000 7efe1240cb80 1 -- 172.21.15.151:0/3791743834 shutdown_connections 2024-05-21T22:01:46.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.861+0000 7efe1240cb80 1 -- 172.21.15.151:0/3791743834 wait complete. 2024-05-21T22:01:46.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.861+0000 7efe1240cb80 1 -- start start 2024-05-21T22:01:46.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.861+0000 7efe1240cb80 1 Processor -- start 2024-05-21T22:01:46.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.862+0000 7efe1240cb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b038210390 0x55b03821bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:46.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.862+0000 7efe1240cb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b0382157c0 0x55b038219160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:46.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.862+0000 7efe1240cb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b0382160e0 0x55b0382196a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:46.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.862+0000 7efe1240cb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55b038063fa0 con 0x55b0382157c0 2024-05-21T22:01:46.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.862+0000 7efe1240cb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55b0380fe760 con 0x55b0382160e0 2024-05-21T22:01:46.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.862+0000 7efe1240cb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55b03821cc00 con 0x55b038210390 2024-05-21T22:01:46.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.862+0000 7efe1240cb80 10 client.0 _ll_register_callbacks cb 0x55b038107c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:01:46.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.862+0000 7efdffe00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b0382157c0 0x55b038219160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:46.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.862+0000 7efe05000700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b038210390 0x55b03821bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:46.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.862+0000 7efe05000700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b038210390 0x55b03821bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:40442/0 (socket says 172.21.15.151:40442) 2024-05-21T22:01:46.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.862+0000 7efe05000700 1 -- 172.21.15.151:0/2274060729 learned_addr learned my addr 172.21.15.151:0/2274060729 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:01:46.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.862+0000 7efe05a00700 1 --2- 172.21.15.151:0/2274060729 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b0382160e0 0x55b0382196a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:46.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.863+0000 7efdffe00700 1 -- 172.21.15.151:0/2274060729 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b038210390 msgr2=0x55b03821bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:46.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.863+0000 7efdffe00700 1 --2- 172.21.15.151:0/2274060729 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b038210390 0x55b03821bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:46.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.863+0000 7efdffe00700 1 -- 172.21.15.151:0/2274060729 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b0382160e0 msgr2=0x55b0382196a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:46.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.863+0000 7efdffe00700 1 --2- 172.21.15.151:0/2274060729 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b0382160e0 0x55b0382196a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:46.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.863+0000 7efdffe00700 1 -- 172.21.15.151:0/2274060729 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0380ad8d0 con 0x55b0382157c0 2024-05-21T22:01:46.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.863+0000 7efe05000700 1 --2- 172.21.15.151:0/2274060729 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b038210390 0x55b03821bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:01:46.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.863+0000 7efdffe00700 1 --2- 172.21.15.151:0/2274060729 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b0382157c0 0x55b038219160 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7efdec0adf50 tx=0x7efdec0a9e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f3d7336d9b9cac7 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:01:46.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.863+0000 7efde6a00700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-21T22:01:46.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.863+0000 7efde6a00700 1 -- 172.21.15.151:0/2274060729 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdec01c040 con 0x55b0382157c0 2024-05-21T22:01:46.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.863+0000 7efde6a00700 1 -- 172.21.15.151:0/2274060729 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7efdec0b25c0 con 0x55b0382157c0 2024-05-21T22:01:46.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.863+0000 7efe1240cb80 1 -- 172.21.15.151:0/2274060729 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efdec0a99e0 con 0x55b0382157c0 2024-05-21T22:01:46.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.864+0000 7efde6a00700 1 -- 172.21.15.151:0/2274060729 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7efdec0add90 con 0x55b0382157c0 2024-05-21T22:01:46.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.864+0000 7efe1240cb80 10 client.16542 fetch_fsmap learned FSMap version 29 2024-05-21T22:01:46.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.864+0000 7efe1240cb80 1 -- 172.21.15.151:0/2274060729 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b03812e060 con 0x55b0382157c0 2024-05-21T22:01:46.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.864+0000 7efde6a00700 1 -- 172.21.15.151:0/2274060729 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efdec0a99e0 con 0x55b0382157c0 2024-05-21T22:01:46.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.864+0000 7efde6a00700 1 -- 172.21.15.151:0/2274060729 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7efdec0a86b0 con 0x55b0382157c0 2024-05-21T22:01:46.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.864+0000 7efe1240cb80 10 client.16542 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:01:46.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.864+0000 7efe1240cb80 10 client.16542 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-21T22:01:46.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.864+0000 7efe1240cb80 20 client.16542 populate_metadata read hostname 'smithi151' 2024-05-21T22:01:46.871 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.864+0000 7efe1240cb80 1 --2- 172.21.15.151:0/2274060729 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x55b038229580 0x55b03822b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:46.871 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.864+0000 7efe1240cb80 4 client.16542 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T22:01:46.871 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.864+0000 7efe1240cb80 1 -- 172.21.15.151:0/2274060729 --> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b03816fd90 con 0x55b038229580 2024-05-21T22:01:46.871 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.865+0000 7efe05000700 1 --2- 172.21.15.151:0/2274060729 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x55b038229580 0x55b03822b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:46.871 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.865+0000 7efe05000700 1 --2- 172.21.15.151:0/2274060729 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x55b038229580 0x55b03822b970 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7b4154150eead3be server_cookie=8f63ce27855a2c8b in_seq=0 out_seq=0 2024-05-21T22:01:46.871 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.865+0000 7efde6a00700 10 client.16542 ms_handle_connect on v2:172.21.15.184:6836/2988194888 2024-05-21T22:01:46.871 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.866+0000 7efde6a00700 1 -- 172.21.15.151:0/2274060729 <== mds.3 v2:172.21.15.184:6836/2988194888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55b03816fd90 con 0x55b038229580 2024-05-21T22:01:46.871 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.866+0000 7efde6a00700 10 client.16542 handle_command_reply: tid=0 2024-05-21T22:01:46.871 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.866+0000 7efe1240cb80 1 client.16542 shutdown 2024-05-21T22:01:46.871 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.866+0000 7efe1240cb80 1 -- 172.21.15.151:0/2274060729 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b0382157c0 msgr2=0x55b038219160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:46.871 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.866+0000 7efe1240cb80 1 --2- 172.21.15.151:0/2274060729 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b0382157c0 0x55b038219160 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7efdec0adf50 tx=0x7efdec0a9e50 comp rx=0 tx=0).stop 2024-05-21T22:01:46.871 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.866+0000 7efe1240cb80 1 -- 172.21.15.151:0/2274060729 shutdown_connections 2024-05-21T22:01:46.871 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.866+0000 7efe1240cb80 1 --2- 172.21.15.151:0/2274060729 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x55b038229580 0x55b03822b970 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:46.872 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.866+0000 7efe1240cb80 1 --2- 172.21.15.151:0/2274060729 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b0382160e0 0x55b0382196a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:46.872 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.866+0000 7efe1240cb80 1 --2- 172.21.15.151:0/2274060729 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b038210390 0x55b03821bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:46.872 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.866+0000 7efe1240cb80 1 --2- 172.21.15.151:0/2274060729 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b0382157c0 0x55b038219160 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:46.872 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.866+0000 7efe1240cb80 1 -- 172.21.15.151:0/2274060729 >> 172.21.15.151:0/2274060729 conn(0x55b03813dbe0 msgr2=0x55b038209fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:01:46.872 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.866+0000 7efde6a00700 0 client.16542 ms_handle_reset on v2:172.21.15.184:6836/2988194888 2024-05-21T22:01:46.872 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.867+0000 7efe1240cb80 1 -- 172.21.15.151:0/2274060729 shutdown_connections 2024-05-21T22:01:46.872 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.867+0000 7efe1240cb80 1 -- 172.21.15.151:0/2274060729 wait complete. 2024-05-21T22:01:46.872 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.867+0000 7efe1240cb80 20 client.16542 trim_cache size 0 max 16384 2024-05-21T22:01:46.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.883+0000 7efe1240cb80 1 Processor -- start 2024-05-21T22:01:46.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.883+0000 7efe1240cb80 1 -- start start 2024-05-21T22:01:46.884 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.884+0000 7efe1240cb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b038210390 0x55b03820fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:46.884 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.884+0000 7efe1240cb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b0382157c0 0x55b0382135b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:46.884 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.884+0000 7efe1240cb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b0382160e0 0x55b038216c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:46.884 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.884+0000 7efe1240cb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7efe001b4330 con 0x55b0382160e0 2024-05-21T22:01:46.884 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.884+0000 7efe1240cb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7efe001b5e70 con 0x55b038210390 2024-05-21T22:01:46.884 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.884+0000 7efe1240cb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7efe001b4a00 con 0x55b0382157c0 2024-05-21T22:01:46.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.884+0000 7efe05a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b0382160e0 0x55b038216c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:46.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.884+0000 7efe05000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b038210390 0x55b03820fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:46.886 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.884+0000 7efdffe00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b0382157c0 0x55b0382135b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:46.886 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.884+0000 7efe05a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b0382160e0 0x55b038216c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:51876/0 (socket says 172.21.15.151:51876) 2024-05-21T22:01:46.886 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.884+0000 7efe05a00700 1 -- 172.21.15.151:0/533955996 learned_addr learned my addr 172.21.15.151:0/533955996 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:01:46.886 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.885+0000 7efe05000700 1 -- 172.21.15.151:0/533955996 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b0382157c0 msgr2=0x55b0382135b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:46.886 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.885+0000 7efe05000700 1 --2- 172.21.15.151:0/533955996 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b0382157c0 0x55b0382135b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:46.886 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.885+0000 7efe05000700 1 -- 172.21.15.151:0/533955996 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b0382160e0 msgr2=0x55b038216c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:46.886 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.885+0000 7efe05000700 1 --2- 172.21.15.151:0/533955996 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b0382160e0 0x55b038216c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:46.886 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.885+0000 7efe05000700 1 -- 172.21.15.151:0/533955996 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0382152b0 con 0x55b038210390 2024-05-21T22:01:46.886 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.885+0000 7efe05a00700 1 --2- 172.21.15.151:0/533955996 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b0382160e0 0x55b038216c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:01:46.886 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.885+0000 7efdffe00700 1 --2- 172.21.15.151:0/533955996 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b0382157c0 0x55b0382135b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:01:46.886 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.885+0000 7efe05000700 1 --2- 172.21.15.151:0/533955996 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b038210390 0x55b03820fd70 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x55b03815db10 tx=0x7efdf4006230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=922f99789d4d67d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:01:46.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.885+0000 7efdf1600700 1 -- 172.21.15.151:0/533955996 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf4004bd0 con 0x55b038210390 2024-05-21T22:01:46.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.885+0000 7efdf1600700 1 -- 172.21.15.151:0/533955996 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efdf402b070 con 0x55b038210390 2024-05-21T22:01:46.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.885+0000 7efdf1600700 1 -- 172.21.15.151:0/533955996 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf4026520 con 0x55b038210390 2024-05-21T22:01:46.888 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.886+0000 7efe1240cb80 1 -- 172.21.15.151:0/533955996 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b038210390 msgr2=0x55b03820fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:46.888 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.886+0000 7efe1240cb80 1 --2- 172.21.15.151:0/533955996 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b038210390 0x55b03820fd70 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x55b03815db10 tx=0x7efdf4006230 comp rx=0 tx=0).stop 2024-05-21T22:01:46.888 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.886+0000 7efe1240cb80 1 -- 172.21.15.151:0/533955996 shutdown_connections 2024-05-21T22:01:46.888 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.886+0000 7efe1240cb80 1 --2- 172.21.15.151:0/533955996 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b038210390 0x55b03820fd70 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:46.888 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.886+0000 7efe1240cb80 1 --2- 172.21.15.151:0/533955996 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b0382160e0 0x55b038216c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:46.888 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.886+0000 7efe1240cb80 1 --2- 172.21.15.151:0/533955996 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b0382157c0 0x55b0382135b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:46.888 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.886+0000 7efe1240cb80 1 -- 172.21.15.151:0/533955996 >> 172.21.15.151:0/533955996 conn(0x55b03813dbe0 msgr2=0x55b038216580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:01:46.888 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.886+0000 7efe1240cb80 1 -- 172.21.15.151:0/533955996 shutdown_connections 2024-05-21T22:01:46.888 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.886+0000 7efe1240cb80 1 -- 172.21.15.151:0/533955996 wait complete. 2024-05-21T22:01:46.888 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.887+0000 7efe1240cb80 1 -- start start 2024-05-21T22:01:46.889 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.887+0000 7efe1240cb80 1 Processor -- start 2024-05-21T22:01:46.889 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.888+0000 7efe1240cb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b038210390 0x55b03822bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:46.889 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.888+0000 7efe1240cb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b0382157c0 0x55b03822c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:46.889 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.888+0000 7efe1240cb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b0382160e0 0x55b03822e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:46.889 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.888+0000 7efe1240cb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7efe001b57a0 con 0x55b038210390 2024-05-21T22:01:46.889 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.888+0000 7efe1240cb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7efe001b50d0 con 0x55b0382157c0 2024-05-21T22:01:46.889 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.888+0000 7efe1240cb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7efe001b22d0 con 0x55b0382160e0 2024-05-21T22:01:46.889 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.888+0000 7efe1240cb80 10 client.0 _ll_register_callbacks cb 0x55b038107c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:01:46.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.888+0000 7efe05a00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b0382160e0 0x55b03822e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:46.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.888+0000 7efe05000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b038210390 0x55b03822bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:46.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.888+0000 7efe05a00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b0382160e0 0x55b03822e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:40466/0 (socket says 172.21.15.151:40466) 2024-05-21T22:01:46.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.888+0000 7efdffe00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b0382157c0 0x55b03822c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:46.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.888+0000 7efe05a00700 1 -- 172.21.15.151:0/2974561218 learned_addr learned my addr 172.21.15.151:0/2974561218 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:01:46.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.888+0000 7efe05a00700 1 -- 172.21.15.151:0/2974561218 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b0382157c0 msgr2=0x55b03822c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:46.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.888+0000 7efe05a00700 1 --2- 172.21.15.151:0/2974561218 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b0382157c0 0x55b03822c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:46.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.888+0000 7efe05a00700 1 -- 172.21.15.151:0/2974561218 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b038210390 msgr2=0x55b03822bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:46.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.888+0000 7efe05a00700 1 --2- 172.21.15.151:0/2974561218 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b038210390 0x55b03822bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:46.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.888+0000 7efe05a00700 1 -- 172.21.15.151:0/2974561218 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b03822e980 con 0x55b0382160e0 2024-05-21T22:01:46.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.888+0000 7efe05000700 1 --2- 172.21.15.151:0/2974561218 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b038210390 0x55b03822bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:01:46.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.888+0000 7efdffe00700 1 --2- 172.21.15.151:0/2974561218 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b0382157c0 0x55b03822c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:01:46.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.889+0000 7efe05a00700 1 --2- 172.21.15.151:0/2974561218 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b0382160e0 0x55b03822e310 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7efdf8012000 tx=0x7efdf8004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d416eab508c0f53 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:01:46.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.889+0000 7efde6a00700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-21T22:01:46.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.889+0000 7efde6a00700 1 -- 172.21.15.151:0/2974561218 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf801fdb0 con 0x55b0382160e0 2024-05-21T22:01:46.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.889+0000 7efe1240cb80 1 -- 172.21.15.151:0/2974561218 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efdf402b070 con 0x55b0382160e0 2024-05-21T22:01:46.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.889+0000 7efde6a00700 1 -- 172.21.15.151:0/2974561218 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7efdf8009040 con 0x55b0382160e0 2024-05-21T22:01:46.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.889+0000 7efde6a00700 1 -- 172.21.15.151:0/2974561218 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7efdf8004830 con 0x55b0382160e0 2024-05-21T22:01:46.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.889+0000 7efe1240cb80 10 client.26683 fetch_fsmap learned FSMap version 29 2024-05-21T22:01:46.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.889+0000 7efe1240cb80 1 -- 172.21.15.151:0/2974561218 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b038293980 con 0x55b0382160e0 2024-05-21T22:01:46.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.890+0000 7efde6a00700 1 -- 172.21.15.151:0/2974561218 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efdf402b070 con 0x55b0382160e0 2024-05-21T22:01:46.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.890+0000 7efde6a00700 1 -- 172.21.15.151:0/2974561218 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7efdf801b070 con 0x55b0382160e0 2024-05-21T22:01:46.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.890+0000 7efe1240cb80 10 client.26683 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:01:46.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.890+0000 7efe1240cb80 10 client.26683 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-21T22:01:46.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.890+0000 7efe1240cb80 20 client.26683 populate_metadata read hostname 'smithi151' 2024-05-21T22:01:46.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.890+0000 7efe1240cb80 1 --2- 172.21.15.151:0/2974561218 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x55b0382940a0 0x55b0382964c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:46.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.890+0000 7efe1240cb80 4 client.26683 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T22:01:46.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.890+0000 7efe1240cb80 1 -- 172.21.15.151:0/2974561218 --> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b038169570 con 0x55b0382940a0 2024-05-21T22:01:46.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.890+0000 7efe05000700 1 --2- 172.21.15.151:0/2974561218 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x55b0382940a0 0x55b0382964c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:46.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.891+0000 7efe05000700 1 --2- 172.21.15.151:0/2974561218 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x55b0382940a0 0x55b0382964c0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=86cdbe87bffabf7c server_cookie=e827b5d56d8a907c in_seq=0 out_seq=0 2024-05-21T22:01:46.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.891+0000 7efde6a00700 10 client.26683 ms_handle_connect on v2:172.21.15.184:6836/2988194888 2024-05-21T22:01:46.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.891+0000 7efde6a00700 1 -- 172.21.15.151:0/2974561218 <== mds.3 v2:172.21.15.184:6836/2988194888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b038169570 con 0x55b0382940a0 2024-05-21T22:01:46.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.891+0000 7efde6a00700 10 client.26683 handle_command_reply: tid=0 2024-05-21T22:01:46.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.891+0000 7efe1240cb80 1 client.26683 shutdown 2024-05-21T22:01:46.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.891+0000 7efe1240cb80 1 -- 172.21.15.151:0/2974561218 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b0382160e0 msgr2=0x55b03822e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:46.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.891+0000 7efe1240cb80 1 --2- 172.21.15.151:0/2974561218 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b0382160e0 0x55b03822e310 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7efdf8012000 tx=0x7efdf8004750 comp rx=0 tx=0).stop 2024-05-21T22:01:46.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.891+0000 7efe1240cb80 1 -- 172.21.15.151:0/2974561218 shutdown_connections 2024-05-21T22:01:46.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.891+0000 7efe1240cb80 1 --2- 172.21.15.151:0/2974561218 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x55b0382940a0 0x55b0382964c0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:46.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.891+0000 7efe1240cb80 1 --2- 172.21.15.151:0/2974561218 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b0382160e0 0x55b03822e310 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:46.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.891+0000 7efe1240cb80 1 --2- 172.21.15.151:0/2974561218 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b038210390 0x55b03822bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:46.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.891+0000 7efde6a00700 0 client.26683 ms_handle_reset on v2:172.21.15.184:6836/2988194888 2024-05-21T22:01:46.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.891+0000 7efe1240cb80 1 --2- 172.21.15.151:0/2974561218 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b0382157c0 0x55b03822c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:46.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.891+0000 7efe1240cb80 1 -- 172.21.15.151:0/2974561218 >> 172.21.15.151:0/2974561218 conn(0x55b03813dbe0 msgr2=0x55b0382142a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:01:46.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.892+0000 7efe1240cb80 1 -- 172.21.15.151:0/2974561218 shutdown_connections 2024-05-21T22:01:46.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.892+0000 7efe1240cb80 1 -- 172.21.15.151:0/2974561218 wait complete. 2024-05-21T22:01:46.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.892+0000 7efe1240cb80 20 client.26683 trim_cache size 0 max 16384 2024-05-21T22:01:46.895 INFO:teuthology.orchestra.run.smithi151.stdout:[] 2024-05-21T22:01:46.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.893+0000 7efe06400700 1 -- 172.21.15.151:0/3402911013 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7efde80b2a70 msgr2=0x7efde80b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:46.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.893+0000 7efe06400700 1 --2- 172.21.15.151:0/3402911013 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7efde80b2a70 0x7efde80b4f30 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7efdf8009860 tx=0x7efdf80097f0 comp rx=0 tx=0).stop 2024-05-21T22:01:46.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.893+0000 7efe06400700 1 -- 172.21.15.151:0/3402911013 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe00076200 msgr2=0x7efe001a0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:46.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.893+0000 7efe06400700 1 --2- 172.21.15.151:0/3402911013 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe00076200 0x7efe001a0750 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7efdec00e9f0 tx=0x7efdec00ed00 comp rx=0 tx=0).stop 2024-05-21T22:01:46.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.893+0000 7efe06400700 1 -- 172.21.15.151:0/3402911013 shutdown_connections 2024-05-21T22:01:46.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.893+0000 7efe06400700 1 --2- 172.21.15.151:0/3402911013 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7efde80b2a70 0x7efde80b4f30 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:46.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.893+0000 7efe06400700 1 --2- 172.21.15.151:0/3402911013 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe00114480 0x7efe001a4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:46.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.893+0000 7efe06400700 1 --2- 172.21.15.151:0/3402911013 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efe000758e0 0x7efe001a0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:46.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.893+0000 7efe06400700 1 --2- 172.21.15.151:0/3402911013 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe00076200 0x7efe001a0750 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:46.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.893+0000 7efe06400700 1 -- 172.21.15.151:0/3402911013 >> 172.21.15.151:0/3402911013 conn(0x7efe000fefc0 msgr2=0x7efe000ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:01:46.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.893+0000 7efe06400700 1 -- 172.21.15.151:0/3402911013 shutdown_connections 2024-05-21T22:01:46.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:46.893+0000 7efe06400700 1 -- 172.21.15.151:0/3402911013 wait complete. 2024-05-21T22:01:47.887 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:01:47 smithi151 ceph-mon[31623]: pgmap v188: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-05-21T22:01:47.891 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-21T22:01:47.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:01:47 smithi184 ceph-mon[36942]: pgmap v188: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-05-21T22:01:48.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:01:47 smithi164 ceph-mon[36269]: pgmap v188: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-05-21T22:01:48.332 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:01:49.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:01:49 smithi151 ceph-mon[31623]: pgmap v189: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-05-21T22:01:49.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:01:49 smithi184 ceph-mon[36942]: pgmap v189: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-05-21T22:01:50.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:01:49 smithi164 ceph-mon[36269]: pgmap v189: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-05-21T22:01:50.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.676+0000 7f1dd9600700 1 -- 172.21.15.151:0/2236734672 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1dd41041a0 msgr2=0x7f1dd41081c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:50.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.676+0000 7f1dd9600700 1 --2- 172.21.15.151:0/2236734672 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1dd41041a0 0x7f1dd41081c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0009990 tx=0x7f1dc0009ca0 comp rx=0 tx=0).stop 2024-05-21T22:01:50.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.676+0000 7f1dd9600700 1 -- 172.21.15.151:0/2236734672 shutdown_connections 2024-05-21T22:01:50.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.676+0000 7f1dd9600700 1 --2- 172.21.15.151:0/2236734672 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1dd41088f0 0x7f1dd4110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:50.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.676+0000 7f1dd9600700 1 --2- 172.21.15.151:0/2236734672 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1dd41037f0 0x7f1dd4103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:50.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.676+0000 7f1dd9600700 1 --2- 172.21.15.151:0/2236734672 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1dd41041a0 0x7f1dd41081c0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:50.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.676+0000 7f1dd9600700 1 -- 172.21.15.151:0/2236734672 >> 172.21.15.151:0/2236734672 conn(0x7f1dd407a6a0 msgr2=0x7f1dd407aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:01:50.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.676+0000 7f1dd9600700 1 -- 172.21.15.151:0/2236734672 shutdown_connections 2024-05-21T22:01:50.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.676+0000 7f1dd9600700 1 -- 172.21.15.151:0/2236734672 wait complete. 2024-05-21T22:01:50.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.677+0000 7f1dd9600700 1 Processor -- start 2024-05-21T22:01:50.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.678+0000 7f1dd9600700 1 -- start start 2024-05-21T22:01:50.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.678+0000 7f1dd9600700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1dd41037f0 0x7f1dd4075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:50.679 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.678+0000 7f1dd9600700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1dd41041a0 0x7f1dd4075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:50.679 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.678+0000 7f1dd9600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1dd41088f0 0x7f1dd40762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:50.679 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.678+0000 7f1dd9600700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd4113720 con 0x7f1dd41088f0 2024-05-21T22:01:50.679 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.678+0000 7f1dd9600700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd41135a0 con 0x7f1dd41037f0 2024-05-21T22:01:50.679 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.678+0000 7f1dd9600700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd41138a0 con 0x7f1dd41041a0 2024-05-21T22:01:50.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.679+0000 7f1dd2000700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1dd41041a0 0x7f1dd4075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:50.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.679+0000 7f1dd3400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1dd41088f0 0x7f1dd40762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:50.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.679+0000 7f1dd2a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1dd41037f0 0x7f1dd4075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:50.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.679+0000 7f1dd2000700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1dd41041a0 0x7f1dd4075d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:40490/0 (socket says 172.21.15.151:40490) 2024-05-21T22:01:50.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.679+0000 7f1dd2000700 1 -- 172.21.15.151:0/4129369380 learned_addr learned my addr 172.21.15.151:0/4129369380 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:01:50.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.679+0000 7f1dd2a00700 1 -- 172.21.15.151:0/4129369380 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1dd41041a0 msgr2=0x7f1dd4075d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:50.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.679+0000 7f1dd2a00700 1 --2- 172.21.15.151:0/4129369380 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1dd41041a0 0x7f1dd4075d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:50.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.679+0000 7f1dd2a00700 1 -- 172.21.15.151:0/4129369380 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1dd41088f0 msgr2=0x7f1dd40762c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:50.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.679+0000 7f1dd2a00700 1 --2- 172.21.15.151:0/4129369380 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1dd41088f0 0x7f1dd40762c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:50.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.679+0000 7f1dd2a00700 1 -- 172.21.15.151:0/4129369380 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dd41a8100 con 0x7f1dd41037f0 2024-05-21T22:01:50.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.679+0000 7f1dd2a00700 1 --2- 172.21.15.151:0/4129369380 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1dd41037f0 0x7f1dd4075840 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f1dc400e9f0 tx=0x7f1dc400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:01:50.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.680+0000 7f1dcb400700 1 -- 172.21.15.151:0/4129369380 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc400cb70 con 0x7f1dd41037f0 2024-05-21T22:01:50.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.680+0000 7f1dcb400700 1 -- 172.21.15.151:0/4129369380 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1dc400ee90 con 0x7f1dd41037f0 2024-05-21T22:01:50.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.680+0000 7f1dcb400700 1 -- 172.21.15.151:0/4129369380 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc4003d40 con 0x7f1dd41037f0 2024-05-21T22:01:50.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.680+0000 7f1dd9600700 1 -- 172.21.15.151:0/4129369380 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dd41a83f0 con 0x7f1dd41037f0 2024-05-21T22:01:50.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.680+0000 7f1dd9600700 1 -- 172.21.15.151:0/4129369380 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dd41a88e0 con 0x7f1dd41037f0 2024-05-21T22:01:50.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.683+0000 7f1dcb400700 1 -- 172.21.15.151:0/4129369380 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7f1dc4003ee0 con 0x7f1dd41037f0 2024-05-21T22:01:50.685 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.685+0000 7f1de3fefb80 1 Processor -- start 2024-05-21T22:01:50.685 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.685+0000 7f1dcb400700 1 --2- 172.21.15.151:0/4129369380 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f1db80b2a70 0x7f1db80b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:50.685 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.685+0000 7f1de3fefb80 1 -- start start 2024-05-21T22:01:50.685 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.685+0000 7f1dcb400700 1 -- 172.21.15.151:0/4129369380 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f1dc40e26e0 con 0x7f1dd41037f0 2024-05-21T22:01:50.686 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.685+0000 7f1dd3400700 1 --2- 172.21.15.151:0/4129369380 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f1db80b2a70 0x7f1db80b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:50.686 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.685+0000 7f1de3fefb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56292dda0eb0 0x56292dda1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:50.686 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.685+0000 7f1de3fefb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56292dda17d0 0x56292dd9b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:50.686 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.685+0000 7f1de3fefb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56292dd9b880 0x56292dd9bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:50.686 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.685+0000 7f1de3fefb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56292dcf9e40 con 0x56292dda17d0 2024-05-21T22:01:50.686 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.685+0000 7f1de3fefb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56292dce99e0 con 0x56292dda0eb0 2024-05-21T22:01:50.687 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.685+0000 7f1de3fefb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56292dceda20 con 0x56292dd9b880 2024-05-21T22:01:50.687 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.686+0000 7f1dd2000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56292dda17d0 0x56292dd9b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:50.687 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.686+0000 7f1dd2a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56292dda0eb0 0x56292dda1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:50.687 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.686+0000 7f1dd2000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56292dda17d0 0x56292dd9b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:51928/0 (socket says 172.21.15.151:51928) 2024-05-21T22:01:50.687 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.686+0000 7f1dd2000700 1 -- 172.21.15.151:0/2278112218 learned_addr learned my addr 172.21.15.151:0/2278112218 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:01:50.687 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.686+0000 7f1dd3400700 1 --2- 172.21.15.151:0/2278112218 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56292dd9b880 0x56292dd9bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:50.688 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.686+0000 7f1dd2a00700 1 -- 172.21.15.151:0/2278112218 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56292dd9b880 msgr2=0x56292dd9bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:50.688 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.686+0000 7f1dd2a00700 1 --2- 172.21.15.151:0/2278112218 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56292dd9b880 0x56292dd9bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:50.688 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.686+0000 7f1dd2a00700 1 -- 172.21.15.151:0/2278112218 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56292dda17d0 msgr2=0x56292dd9b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:50.688 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.686+0000 7f1dd2a00700 1 --2- 172.21.15.151:0/2278112218 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56292dda17d0 0x56292dd9b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:50.688 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.686+0000 7f1dd2a00700 1 -- 172.21.15.151:0/2278112218 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56292dcb2420 con 0x56292dda0eb0 2024-05-21T22:01:50.688 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.687+0000 7f1dd2a00700 1 --2- 172.21.15.151:0/2278112218 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56292dda0eb0 0x56292dda1290 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f1dc40dfde0 tx=0x7f1dc40a5180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47be262e0a81e37c server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:01:50.688 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.687+0000 7f1dd3400700 1 --2- 172.21.15.151:0/4129369380 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f1db80b2a70 0x7f1db80b4f30 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc00f870 tx=0x7f1dcc00fc30 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:01:50.689 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.687+0000 7f1dbfe00700 1 -- 172.21.15.151:0/2278112218 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc40aa070 con 0x56292dda0eb0 2024-05-21T22:01:50.689 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.687+0000 7f1dbfe00700 1 -- 172.21.15.151:0/2278112218 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1dc40094d0 con 0x56292dda0eb0 2024-05-21T22:01:50.689 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.687+0000 7f1dbfe00700 1 -- 172.21.15.151:0/2278112218 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc40a9bd0 con 0x56292dda0eb0 2024-05-21T22:01:50.689 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.688+0000 7f1de3fefb80 1 -- 172.21.15.151:0/2278112218 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56292dda0eb0 msgr2=0x56292dda1290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:50.689 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.688+0000 7f1de3fefb80 1 --2- 172.21.15.151:0/2278112218 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56292dda0eb0 0x56292dda1290 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f1dc40dfde0 tx=0x7f1dc40a5180 comp rx=0 tx=0).stop 2024-05-21T22:01:50.689 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.688+0000 7f1de3fefb80 1 -- 172.21.15.151:0/2278112218 shutdown_connections 2024-05-21T22:01:50.689 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.688+0000 7f1de3fefb80 1 --2- 172.21.15.151:0/2278112218 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56292dda0eb0 0x56292dda1290 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:50.689 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.688+0000 7f1de3fefb80 1 --2- 172.21.15.151:0/2278112218 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56292dd9b880 0x56292dd9bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:50.689 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.688+0000 7f1de3fefb80 1 --2- 172.21.15.151:0/2278112218 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56292dda17d0 0x56292dd9b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:50.689 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.688+0000 7f1de3fefb80 1 -- 172.21.15.151:0/2278112218 >> 172.21.15.151:0/2278112218 conn(0x56292dcc9be0 msgr2=0x56292dd96010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:01:50.689 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.688+0000 7f1de3fefb80 1 -- 172.21.15.151:0/2278112218 shutdown_connections 2024-05-21T22:01:50.689 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.688+0000 7f1de3fefb80 1 -- 172.21.15.151:0/2278112218 wait complete. 2024-05-21T22:01:50.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.688+0000 7f1de3fefb80 1 -- start start 2024-05-21T22:01:50.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.689+0000 7f1de3fefb80 1 Processor -- start 2024-05-21T22:01:50.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.689+0000 7f1de3fefb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56292dd9b880 0x56292dda7610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:50.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.689+0000 7f1de3fefb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56292dda0eb0 0x56292dda49b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:50.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.689+0000 7f1de3fefb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56292dda17d0 0x56292dda4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:50.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.689+0000 7f1de3fefb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56292dbeffa0 con 0x56292dd9b880 2024-05-21T22:01:50.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.689+0000 7f1de3fefb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56292dc8a760 con 0x56292dda17d0 2024-05-21T22:01:50.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.689+0000 7f1de3fefb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56292dda8240 con 0x56292dda0eb0 2024-05-21T22:01:50.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.689+0000 7f1de3fefb80 10 client.0 _ll_register_callbacks cb 0x56292dc93c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:01:50.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.690+0000 7f1dd2a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56292dd9b880 0x56292dda7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:50.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.690+0000 7f1dd3400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56292dda17d0 0x56292dda4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:50.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.690+0000 7f1dd2a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56292dd9b880 0x56292dda7610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:51938/0 (socket says 172.21.15.151:51938) 2024-05-21T22:01:50.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.690+0000 7f1dd2a00700 1 -- 172.21.15.151:0/2076876812 learned_addr learned my addr 172.21.15.151:0/2076876812 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:01:50.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.690+0000 7f1dd2000700 1 --2- 172.21.15.151:0/2076876812 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56292dda0eb0 0x56292dda49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:50.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.690+0000 7f1dd2a00700 1 -- 172.21.15.151:0/2076876812 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56292dda0eb0 msgr2=0x56292dda49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:50.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.690+0000 7f1dd2a00700 1 --2- 172.21.15.151:0/2076876812 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56292dda0eb0 0x56292dda49b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:50.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.690+0000 7f1dd2a00700 1 -- 172.21.15.151:0/2076876812 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56292dda17d0 msgr2=0x56292dda4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:50.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.690+0000 7f1dd2a00700 1 --2- 172.21.15.151:0/2076876812 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56292dda17d0 0x56292dda4ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:50.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.690+0000 7f1dd2a00700 1 -- 172.21.15.151:0/2076876812 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56292dc398d0 con 0x56292dd9b880 2024-05-21T22:01:50.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.690+0000 7f1dd3400700 1 --2- 172.21.15.151:0/2076876812 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56292dda17d0 0x56292dda4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:01:50.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.690+0000 7f1dd2a00700 1 --2- 172.21.15.151:0/2076876812 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56292dd9b880 0x56292dda7610 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f1dc400c5b0 tx=0x7f1dc40cc3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6de82561103d2a05 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:01:50.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.690+0000 7f1dbcc00700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-21T22:01:50.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.691+0000 7f1dbcc00700 1 -- 172.21.15.151:0/2076876812 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc40aa070 con 0x56292dd9b880 2024-05-21T22:01:50.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.691+0000 7f1dbcc00700 1 -- 172.21.15.151:0/2076876812 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f1dc40b2550 con 0x56292dd9b880 2024-05-21T22:01:50.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.691+0000 7f1de3fefb80 1 -- 172.21.15.151:0/2076876812 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1dc40094d0 con 0x56292dd9b880 2024-05-21T22:01:50.694 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.691+0000 7f1dbcc00700 1 -- 172.21.15.151:0/2076876812 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1dc40a9970 con 0x56292dd9b880 2024-05-21T22:01:50.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.691+0000 7f1de3fefb80 10 client.16560 fetch_fsmap learned FSMap version 29 2024-05-21T22:01:50.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.691+0000 7f1de3fefb80 1 -- 172.21.15.151:0/2076876812 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56292dcba060 con 0x56292dd9b880 2024-05-21T22:01:50.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.691+0000 7f1dbcc00700 1 -- 172.21.15.151:0/2076876812 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1dc40094d0 con 0x56292dd9b880 2024-05-21T22:01:50.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.692+0000 7f1dbcc00700 1 -- 172.21.15.151:0/2076876812 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f1dc40a5400 con 0x56292dd9b880 2024-05-21T22:01:50.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.692+0000 7f1de3fefb80 10 client.16560 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:01:50.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.692+0000 7f1de3fefb80 10 client.16560 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-05-21T22:01:50.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.692+0000 7f1de3fefb80 20 client.16560 populate_metadata read hostname 'smithi151' 2024-05-21T22:01:50.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.692+0000 7f1de3fefb80 1 --2- 172.21.15.151:0/2076876812 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x56292ddb4d40 0x56292ddb7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:50.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.692+0000 7f1de3fefb80 4 client.16560 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T22:01:50.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.692+0000 7f1de3fefb80 1 -- 172.21.15.151:0/2076876812 --> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56292dcfbd90 con 0x56292ddb4d40 2024-05-21T22:01:50.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.692+0000 7f1dd2000700 1 --2- 172.21.15.151:0/2076876812 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x56292ddb4d40 0x56292ddb7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:50.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.693+0000 7f1dd2000700 1 --2- 172.21.15.151:0/2076876812 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x56292ddb4d40 0x56292ddb7130 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3be5d514ec7a17cf server_cookie=dbbb25b8486b0b0 in_seq=0 out_seq=0 2024-05-21T22:01:50.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.693+0000 7f1dbcc00700 10 client.16560 ms_handle_connect on v2:172.21.15.164:6836/1554609572 2024-05-21T22:01:50.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.693+0000 7f1dbcc00700 1 -- 172.21.15.151:0/2076876812 <== mds.2 v2:172.21.15.164:6836/1554609572 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x56292dcfbd90 con 0x56292ddb4d40 2024-05-21T22:01:50.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.693+0000 7f1dbcc00700 10 client.16560 handle_command_reply: tid=0 2024-05-21T22:01:50.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.694+0000 7f1de3fefb80 1 client.16560 shutdown 2024-05-21T22:01:50.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.694+0000 7f1de3fefb80 1 -- 172.21.15.151:0/2076876812 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56292dd9b880 msgr2=0x56292dda7610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:50.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.694+0000 7f1de3fefb80 1 --2- 172.21.15.151:0/2076876812 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56292dd9b880 0x56292dda7610 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f1dc400c5b0 tx=0x7f1dc40cc3c0 comp rx=0 tx=0).stop 2024-05-21T22:01:50.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.694+0000 7f1de3fefb80 1 -- 172.21.15.151:0/2076876812 shutdown_connections 2024-05-21T22:01:50.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.694+0000 7f1de3fefb80 1 --2- 172.21.15.151:0/2076876812 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x56292ddb4d40 0x56292ddb7130 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:50.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.694+0000 7f1de3fefb80 1 --2- 172.21.15.151:0/2076876812 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56292dda17d0 0x56292dda4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:50.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.694+0000 7f1de3fefb80 1 --2- 172.21.15.151:0/2076876812 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56292dd9b880 0x56292dda7610 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:50.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.694+0000 7f1de3fefb80 1 --2- 172.21.15.151:0/2076876812 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56292dda0eb0 0x56292dda49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:50.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.694+0000 7f1dbcc00700 0 client.16560 ms_handle_reset on v2:172.21.15.164:6836/1554609572 2024-05-21T22:01:50.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.694+0000 7f1de3fefb80 1 -- 172.21.15.151:0/2076876812 >> 172.21.15.151:0/2076876812 conn(0x56292dcc9be0 msgr2=0x56292dd9a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:01:50.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.694+0000 7f1de3fefb80 1 -- 172.21.15.151:0/2076876812 shutdown_connections 2024-05-21T22:01:50.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.694+0000 7f1de3fefb80 1 -- 172.21.15.151:0/2076876812 wait complete. 2024-05-21T22:01:50.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.694+0000 7f1de3fefb80 20 client.16560 trim_cache size 0 max 16384 2024-05-21T22:01:50.711 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.711+0000 7f1de3fefb80 1 Processor -- start 2024-05-21T22:01:50.711 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.711+0000 7f1de3fefb80 1 -- start start 2024-05-21T22:01:50.712 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.711+0000 7f1de3fefb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56292dcc9be0 0x56292dda7610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:50.712 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.711+0000 7f1de3fefb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56292dd9b880 0x56292dd9b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:50.712 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.711+0000 7f1de3fefb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56292dda0eb0 0x56292dda22c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:50.712 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.711+0000 7f1de3fefb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd41b0000 con 0x56292dcc9be0 2024-05-21T22:01:50.712 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.711+0000 7f1de3fefb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd41b1b40 con 0x56292dda0eb0 2024-05-21T22:01:50.712 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.711+0000 7f1de3fefb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd41b06d0 con 0x56292dd9b880 2024-05-21T22:01:50.712 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.712+0000 7f1dd2a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56292dcc9be0 0x56292dda7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:50.712 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.712+0000 7f1dd2a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56292dcc9be0 0x56292dda7610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:51940/0 (socket says 172.21.15.151:51940) 2024-05-21T22:01:50.712 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.712+0000 7f1dd2a00700 1 -- 172.21.15.151:0/1093281061 learned_addr learned my addr 172.21.15.151:0/1093281061 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:01:50.712 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.712+0000 7f1dd3400700 1 --2- 172.21.15.151:0/1093281061 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56292dda0eb0 0x56292dda22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:50.713 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.712+0000 7f1dd2000700 1 --2- 172.21.15.151:0/1093281061 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56292dd9b880 0x56292dd9b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:50.714 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.712+0000 7f1dd2a00700 1 -- 172.21.15.151:0/1093281061 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56292dd9b880 msgr2=0x56292dd9b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:50.714 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.712+0000 7f1dd2a00700 1 --2- 172.21.15.151:0/1093281061 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56292dd9b880 0x56292dd9b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:50.714 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.712+0000 7f1dd2a00700 1 -- 172.21.15.151:0/1093281061 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56292dda0eb0 msgr2=0x56292dda22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:50.714 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.712+0000 7f1dd2a00700 1 --2- 172.21.15.151:0/1093281061 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56292dda0eb0 0x56292dda22c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:50.714 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.712+0000 7f1dd2a00700 1 -- 172.21.15.151:0/1093281061 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56292dd90510 con 0x56292dcc9be0 2024-05-21T22:01:50.714 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.712+0000 7f1dd2a00700 1 --2- 172.21.15.151:0/1093281061 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56292dcc9be0 0x56292dda7610 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f1dc40a59f0 tx=0x7f1dc40a9740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67999ba38d0d5a46 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:01:50.714 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.713+0000 7f1dbf400700 1 -- 172.21.15.151:0/1093281061 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc40094d0 con 0x56292dcc9be0 2024-05-21T22:01:50.714 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.713+0000 7f1dbf400700 1 -- 172.21.15.151:0/1093281061 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1dc4087030 con 0x56292dcc9be0 2024-05-21T22:01:50.714 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.713+0000 7f1dbf400700 1 -- 172.21.15.151:0/1093281061 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc40a89b0 con 0x56292dcc9be0 2024-05-21T22:01:50.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.713+0000 7f1de3fefb80 1 -- 172.21.15.151:0/1093281061 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56292dcc9be0 msgr2=0x56292dda7610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:50.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.713+0000 7f1de3fefb80 1 --2- 172.21.15.151:0/1093281061 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56292dcc9be0 0x56292dda7610 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f1dc40a59f0 tx=0x7f1dc40a9740 comp rx=0 tx=0).stop 2024-05-21T22:01:50.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.713+0000 7f1de3fefb80 1 -- 172.21.15.151:0/1093281061 shutdown_connections 2024-05-21T22:01:50.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.713+0000 7f1de3fefb80 1 --2- 172.21.15.151:0/1093281061 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56292dda0eb0 0x56292dda22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:50.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.713+0000 7f1de3fefb80 1 --2- 172.21.15.151:0/1093281061 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56292dcc9be0 0x56292dda7610 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:50.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.713+0000 7f1de3fefb80 1 --2- 172.21.15.151:0/1093281061 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56292dd9b880 0x56292dd9b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:50.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.713+0000 7f1de3fefb80 1 -- 172.21.15.151:0/1093281061 >> 172.21.15.151:0/1093281061 conn(0x56292dd90130 msgr2=0x56292dda1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:01:50.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.714+0000 7f1de3fefb80 1 -- 172.21.15.151:0/1093281061 shutdown_connections 2024-05-21T22:01:50.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.714+0000 7f1de3fefb80 1 -- 172.21.15.151:0/1093281061 wait complete. 2024-05-21T22:01:50.717 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.714+0000 7f1de3fefb80 1 -- start start 2024-05-21T22:01:50.717 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.714+0000 7f1de3fefb80 1 Processor -- start 2024-05-21T22:01:50.717 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.715+0000 7f1de3fefb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56292dcc9be0 0x56292ddb7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:50.717 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.715+0000 7f1de3fefb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56292dd9b880 0x56292ddb7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:50.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.715+0000 7f1de3fefb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56292dda0eb0 0x56292ddb9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:50.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.715+0000 7f1de3fefb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd41b1470 con 0x56292dcc9be0 2024-05-21T22:01:50.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.715+0000 7f1de3fefb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd41b0da0 con 0x56292dda0eb0 2024-05-21T22:01:50.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.715+0000 7f1de3fefb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd41adfa0 con 0x56292dd9b880 2024-05-21T22:01:50.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.715+0000 7f1de3fefb80 10 client.0 _ll_register_callbacks cb 0x56292dc93c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:01:50.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.715+0000 7f1dd2a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56292dcc9be0 0x56292ddb7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:50.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.715+0000 7f1dd2a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56292dcc9be0 0x56292ddb7990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:51944/0 (socket says 172.21.15.151:51944) 2024-05-21T22:01:50.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.715+0000 7f1dd2a00700 1 -- 172.21.15.151:0/3416705449 learned_addr learned my addr 172.21.15.151:0/3416705449 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:01:50.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.715+0000 7f1dd3400700 1 --2- 172.21.15.151:0/3416705449 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56292dda0eb0 0x56292ddb9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:50.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.715+0000 7f1dd2000700 1 --2- 172.21.15.151:0/3416705449 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56292dd9b880 0x56292ddb7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:50.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.716+0000 7f1dd2a00700 1 -- 172.21.15.151:0/3416705449 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56292dd9b880 msgr2=0x56292ddb7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:50.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.716+0000 7f1dd2a00700 1 --2- 172.21.15.151:0/3416705449 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56292dd9b880 0x56292ddb7ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:50.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.716+0000 7f1dd2a00700 1 -- 172.21.15.151:0/3416705449 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56292dda0eb0 msgr2=0x56292ddb9eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:50.719 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.716+0000 7f1dd2a00700 1 --2- 172.21.15.151:0/3416705449 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56292dda0eb0 0x56292ddb9eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:50.719 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.716+0000 7f1dd2a00700 1 -- 172.21.15.151:0/3416705449 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56292dd9c2f0 con 0x56292dcc9be0 2024-05-21T22:01:50.719 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.716+0000 7f1dd2a00700 1 --2- 172.21.15.151:0/3416705449 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56292dcc9be0 0x56292ddb7990 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f1dc40a8640 tx=0x7f1dc4003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23201c02159dc5e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:01:50.719 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.716+0000 7f1dbcc00700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-21T22:01:50.719 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.716+0000 7f1dbcc00700 1 -- 172.21.15.151:0/3416705449 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc40094d0 con 0x56292dcc9be0 2024-05-21T22:01:50.719 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.716+0000 7f1dbcc00700 1 -- 172.21.15.151:0/3416705449 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f1dc40e2250 con 0x56292dcc9be0 2024-05-21T22:01:50.719 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.716+0000 7f1de3fefb80 1 -- 172.21.15.151:0/3416705449 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1dc4087030 con 0x56292dcc9be0 2024-05-21T22:01:50.719 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.716+0000 7f1dbcc00700 1 -- 172.21.15.151:0/3416705449 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1dc40aa070 con 0x56292dcc9be0 2024-05-21T22:01:50.719 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.716+0000 7f1de3fefb80 10 client.16566 fetch_fsmap learned FSMap version 29 2024-05-21T22:01:50.719 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.716+0000 7f1de3fefb80 1 -- 172.21.15.151:0/3416705449 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56292dd9c2f0 con 0x56292dcc9be0 2024-05-21T22:01:50.719 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.717+0000 7f1dbcc00700 1 -- 172.21.15.151:0/3416705449 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1dc4087030 con 0x56292dcc9be0 2024-05-21T22:01:50.719 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.717+0000 7f1dbcc00700 1 -- 172.21.15.151:0/3416705449 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f1dc40aeeb0 con 0x56292dcc9be0 2024-05-21T22:01:50.719 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.717+0000 7f1de3fefb80 10 client.16566 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:01:50.719 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.717+0000 7f1de3fefb80 10 client.16566 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-05-21T22:01:50.720 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.717+0000 7f1de3fefb80 20 client.16566 populate_metadata read hostname 'smithi151' 2024-05-21T22:01:50.720 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.717+0000 7f1de3fefb80 1 --2- 172.21.15.151:0/3416705449 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x56292dda17d0 0x56292de21220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:50.720 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.717+0000 7f1de3fefb80 4 client.16566 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T22:01:50.720 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.717+0000 7f1de3fefb80 1 -- 172.21.15.151:0/3416705449 --> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56292dcf5570 con 0x56292dda17d0 2024-05-21T22:01:50.720 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.717+0000 7f1dd2000700 1 --2- 172.21.15.151:0/3416705449 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x56292dda17d0 0x56292de21220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:50.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.718+0000 7f1dd2000700 1 --2- 172.21.15.151:0/3416705449 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x56292dda17d0 0x56292de21220 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=216ff62875544528 server_cookie=4a147aa2e4ae2eb4 in_seq=0 out_seq=0 2024-05-21T22:01:50.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.718+0000 7f1dbcc00700 10 client.16566 ms_handle_connect on v2:172.21.15.164:6836/1554609572 2024-05-21T22:01:50.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.718+0000 7f1dbcc00700 1 -- 172.21.15.151:0/3416705449 <== mds.2 v2:172.21.15.164:6836/1554609572 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56292dcf5570 con 0x56292dda17d0 2024-05-21T22:01:50.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.718+0000 7f1dbcc00700 10 client.16566 handle_command_reply: tid=0 2024-05-21T22:01:50.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.718+0000 7f1de3fefb80 1 client.16566 shutdown 2024-05-21T22:01:50.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.718+0000 7f1de3fefb80 1 -- 172.21.15.151:0/3416705449 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56292dcc9be0 msgr2=0x56292ddb7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:50.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.718+0000 7f1de3fefb80 1 --2- 172.21.15.151:0/3416705449 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56292dcc9be0 0x56292ddb7990 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f1dc40a8640 tx=0x7f1dc4003680 comp rx=0 tx=0).stop 2024-05-21T22:01:50.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.718+0000 7f1de3fefb80 1 -- 172.21.15.151:0/3416705449 shutdown_connections 2024-05-21T22:01:50.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.718+0000 7f1de3fefb80 1 --2- 172.21.15.151:0/3416705449 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x56292dda17d0 0x56292de21220 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:50.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.718+0000 7f1de3fefb80 1 --2- 172.21.15.151:0/3416705449 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56292dda0eb0 0x56292ddb9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:50.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.718+0000 7f1de3fefb80 1 --2- 172.21.15.151:0/3416705449 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56292dcc9be0 0x56292ddb7990 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:50.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.718+0000 7f1de3fefb80 1 --2- 172.21.15.151:0/3416705449 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56292dd9b880 0x56292ddb7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:50.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.718+0000 7f1de3fefb80 1 -- 172.21.15.151:0/3416705449 >> 172.21.15.151:0/3416705449 conn(0x56292dd90130 msgr2=0x56292dda3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:01:50.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.718+0000 7f1dbcc00700 0 client.16566 ms_handle_reset on v2:172.21.15.164:6836/1554609572 2024-05-21T22:01:50.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.718+0000 7f1de3fefb80 1 -- 172.21.15.151:0/3416705449 shutdown_connections 2024-05-21T22:01:50.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.719+0000 7f1de3fefb80 1 -- 172.21.15.151:0/3416705449 wait complete. 2024-05-21T22:01:50.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.719+0000 7f1de3fefb80 20 client.16566 trim_cache size 0 max 16384 2024-05-21T22:01:50.722 INFO:teuthology.orchestra.run.smithi151.stdout:[] 2024-05-21T22:01:50.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.719+0000 7f1dd9600700 1 -- 172.21.15.151:0/4129369380 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f1db80b2a70 msgr2=0x7f1db80b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:50.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.719+0000 7f1dd9600700 1 --2- 172.21.15.151:0/4129369380 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f1db80b2a70 0x7f1db80b4f30 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc00f870 tx=0x7f1dcc00fc30 comp rx=0 tx=0).stop 2024-05-21T22:01:50.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.719+0000 7f1dd9600700 1 -- 172.21.15.151:0/4129369380 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1dd41037f0 msgr2=0x7f1dd4075840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:50.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.719+0000 7f1dd9600700 1 --2- 172.21.15.151:0/4129369380 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1dd41037f0 0x7f1dd4075840 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f1dc400e9f0 tx=0x7f1dc400ed00 comp rx=0 tx=0).stop 2024-05-21T22:01:50.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.719+0000 7f1dd9600700 1 -- 172.21.15.151:0/4129369380 shutdown_connections 2024-05-21T22:01:50.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.719+0000 7f1dd9600700 1 --2- 172.21.15.151:0/4129369380 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f1db80b2a70 0x7f1db80b4f30 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:50.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.719+0000 7f1dd9600700 1 --2- 172.21.15.151:0/4129369380 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1dd41037f0 0x7f1dd4075840 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:50.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.719+0000 7f1dd9600700 1 --2- 172.21.15.151:0/4129369380 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1dd41088f0 0x7f1dd40762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:50.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.719+0000 7f1dd9600700 1 --2- 172.21.15.151:0/4129369380 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1dd41041a0 0x7f1dd4075d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:50.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.719+0000 7f1dd9600700 1 -- 172.21.15.151:0/4129369380 >> 172.21.15.151:0/4129369380 conn(0x7f1dd407a6a0 msgr2=0x7f1dd41018e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:01:50.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.720+0000 7f1dd9600700 1 -- 172.21.15.151:0/4129369380 shutdown_connections 2024-05-21T22:01:50.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:50.720+0000 7f1dd9600700 1 -- 172.21.15.151:0/4129369380 wait complete. 2024-05-21T22:01:51.621 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-21T22:01:51.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:01:51 smithi151 ceph-mon[31623]: pgmap v190: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:01:51.985 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:01:51 smithi184 ceph-mon[36942]: pgmap v190: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:01:52.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:01:51 smithi164 ceph-mon[36269]: pgmap v190: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:01:52.072 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:01:53.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:01:53 smithi151 ceph-mon[31623]: pgmap v191: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T22:01:53.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:01:53 smithi184 ceph-mon[36942]: pgmap v191: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T22:01:54.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:01:53 smithi164 ceph-mon[36269]: pgmap v191: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T22:01:54.262 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.262+0000 7fe3df200700 1 -- 172.21.15.151:0/1891487511 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe3d806c470 msgr2=0x7fe3d81103c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:54.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.262+0000 7fe3df200700 1 --2- 172.21.15.151:0/1891487511 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe3d806c470 0x7fe3d81103c0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fe3d000b0a0 tx=0x7fe3d000b3b0 comp rx=0 tx=0).stop 2024-05-21T22:01:54.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.262+0000 7fe3df200700 1 -- 172.21.15.151:0/1891487511 shutdown_connections 2024-05-21T22:01:54.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.262+0000 7fe3df200700 1 --2- 172.21.15.151:0/1891487511 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe3d806c470 0x7fe3d81103c0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:54.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.262+0000 7fe3df200700 1 --2- 172.21.15.151:0/1891487511 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe3d81095a0 0x7fe3d8109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:54.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.262+0000 7fe3df200700 1 --2- 172.21.15.151:0/1891487511 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe3d806bab0 0x7fe3d806bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:54.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.262+0000 7fe3df200700 1 -- 172.21.15.151:0/1891487511 >> 172.21.15.151:0/1891487511 conn(0x7fe3d80793b0 msgr2=0x7fe3d80797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:01:54.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.262+0000 7fe3df200700 1 -- 172.21.15.151:0/1891487511 shutdown_connections 2024-05-21T22:01:54.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.262+0000 7fe3df200700 1 -- 172.21.15.151:0/1891487511 wait complete. 2024-05-21T22:01:54.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.264+0000 7fe3df200700 1 Processor -- start 2024-05-21T22:01:54.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.264+0000 7fe3df200700 1 -- start start 2024-05-21T22:01:54.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.264+0000 7fe3df200700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe3d806bab0 0x7fe3d8102500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:54.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.264+0000 7fe3df200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe3d806c470 0x7fe3d8102a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:54.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.264+0000 7fe3df200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe3d81095a0 0x7fe3d8102f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:54.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.265+0000 7fe3df200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d8113730 con 0x7fe3d806c470 2024-05-21T22:01:54.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.265+0000 7fe3df200700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d81135b0 con 0x7fe3d81095a0 2024-05-21T22:01:54.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.265+0000 7fe3df200700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d81138b0 con 0x7fe3d806bab0 2024-05-21T22:01:54.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.265+0000 7fe3d7400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe3d806c470 0x7fe3d8102a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:54.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.265+0000 7fe3dd200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe3d81095a0 0x7fe3d8102f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:54.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.265+0000 7fe3d7400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe3d806c470 0x7fe3d8102a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:51952/0 (socket says 172.21.15.151:51952) 2024-05-21T22:01:54.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.265+0000 7fe3d7400700 1 -- 172.21.15.151:0/1381700696 learned_addr learned my addr 172.21.15.151:0/1381700696 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:01:54.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.265+0000 7fe3d7e00700 1 --2- 172.21.15.151:0/1381700696 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe3d806bab0 0x7fe3d8102500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:54.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.265+0000 7fe3dd200700 1 -- 172.21.15.151:0/1381700696 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe3d806bab0 msgr2=0x7fe3d8102500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:54.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.265+0000 7fe3dd200700 1 --2- 172.21.15.151:0/1381700696 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe3d806bab0 0x7fe3d8102500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:54.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.265+0000 7fe3dd200700 1 -- 172.21.15.151:0/1381700696 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe3d806c470 msgr2=0x7fe3d8102a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:54.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.265+0000 7fe3dd200700 1 --2- 172.21.15.151:0/1381700696 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe3d806c470 0x7fe3d8102a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:54.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.265+0000 7fe3dd200700 1 -- 172.21.15.151:0/1381700696 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3d81a8150 con 0x7fe3d81095a0 2024-05-21T22:01:54.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.265+0000 7fe3d7400700 1 --2- 172.21.15.151:0/1381700696 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe3d806c470 0x7fe3d8102a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:01:54.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.266+0000 7fe3dd200700 1 --2- 172.21.15.151:0/1381700696 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe3d81095a0 0x7fe3d8102f80 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe3d000b070 tx=0x7fe3d0004c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:01:54.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.266+0000 7fe3d4c00700 1 -- 172.21.15.151:0/1381700696 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d0021070 con 0x7fe3d81095a0 2024-05-21T22:01:54.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.266+0000 7fe3d4c00700 1 -- 172.21.15.151:0/1381700696 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3d0007e30 con 0x7fe3d81095a0 2024-05-21T22:01:54.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.266+0000 7fe3d4c00700 1 -- 172.21.15.151:0/1381700696 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d00045f0 con 0x7fe3d81095a0 2024-05-21T22:01:54.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.266+0000 7fe3df200700 1 -- 172.21.15.151:0/1381700696 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3d81a83e0 con 0x7fe3d81095a0 2024-05-21T22:01:54.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.266+0000 7fe3df200700 1 -- 172.21.15.151:0/1381700696 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3d81a8840 con 0x7fe3d81095a0 2024-05-21T22:01:54.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.270+0000 7fe3d4c00700 1 -- 172.21.15.151:0/1381700696 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7fe3d0007980 con 0x7fe3d81095a0 2024-05-21T22:01:54.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.271+0000 7fe3e9c57b80 1 Processor -- start 2024-05-21T22:01:54.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.271+0000 7fe3e9c57b80 1 -- start start 2024-05-21T22:01:54.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.272+0000 7fe3e9c57b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a24cb5f610 0x55a24cb5f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:54.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.272+0000 7fe3e9c57b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a24cb5ff30 0x55a24cb59b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:54.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.272+0000 7fe3e9c57b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a24cb5a140 0x55a24cb5a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:54.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.272+0000 7fe3e9c57b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a24cab7e40 con 0x55a24cb5a140 2024-05-21T22:01:54.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.272+0000 7fe3e9c57b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55a24caa79e0 con 0x55a24cb5ff30 2024-05-21T22:01:54.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.272+0000 7fe3e9c57b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55a24caaba20 con 0x55a24cb5f610 2024-05-21T22:01:54.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.272+0000 7fe3d4c00700 1 --2- 172.21.15.151:0/1381700696 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fe3bc0b29f0 0x7fe3bc0b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:54.273 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.272+0000 7fe3d7e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a24cb5ff30 0x55a24cb59b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:54.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.272+0000 7fe3d7400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a24cb5f610 0x55a24cb5f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:54.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.272+0000 7fe3d7e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a24cb5ff30 0x55a24cb59b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:44148/0 (socket says 172.21.15.151:44148) 2024-05-21T22:01:54.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.272+0000 7fe3d7e00700 1 -- 172.21.15.151:0/1505399530 learned_addr learned my addr 172.21.15.151:0/1505399530 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:01:54.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.272+0000 7fe3d7400700 1 --2- 172.21.15.151:0/1505399530 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a24cb5a140 0x55a24cb5a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:54.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.272+0000 7fe3d4c00700 1 -- 172.21.15.151:0/1381700696 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fe3d000f020 con 0x7fe3d81095a0 2024-05-21T22:01:54.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.272+0000 7fe3d7e00700 1 -- 172.21.15.151:0/1505399530 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a24cb5f610 msgr2=0x55a24cb5f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:54.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.272+0000 7fe3d7e00700 1 --2- 172.21.15.151:0/1505399530 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a24cb5f610 0x55a24cb5f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:54.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.272+0000 7fe3dd200700 1 --2- 172.21.15.151:0/1381700696 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fe3bc0b29f0 0x7fe3bc0b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:54.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.272+0000 7fe3d7e00700 1 -- 172.21.15.151:0/1505399530 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a24cb5a140 msgr2=0x55a24cb5a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:54.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.272+0000 7fe3d7e00700 1 --2- 172.21.15.151:0/1505399530 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a24cb5a140 0x55a24cb5a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:54.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.273+0000 7fe3d7e00700 1 -- 172.21.15.151:0/1505399530 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a24ca70420 con 0x55a24cb5ff30 2024-05-21T22:01:54.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.273+0000 7fe3d7400700 1 --2- 172.21.15.151:0/1505399530 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a24cb5a140 0x55a24cb5a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T22:01:54.275 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.273+0000 7fe3d7e00700 1 --2- 172.21.15.151:0/1505399530 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a24cb5ff30 0x55a24cb59b20 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fe3c8009d30 tx=0x7fe3c8011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b607706f6cb256a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:01:54.275 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.273+0000 7fe3cd600700 1 -- 172.21.15.151:0/1505399530 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c8004230 con 0x55a24cb5ff30 2024-05-21T22:01:54.275 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.273+0000 7fe3dd200700 1 --2- 172.21.15.151:0/1381700696 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fe3bc0b29f0 0x7fe3bc0b4eb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe3d00a6cc0 tx=0x7fe3d000b3b0 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:01:54.275 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.273+0000 7fe3cd600700 1 -- 172.21.15.151:0/1505399530 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3c80043d0 con 0x55a24cb5ff30 2024-05-21T22:01:54.275 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.273+0000 7fe3cd600700 1 -- 172.21.15.151:0/1505399530 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c80197e0 con 0x55a24cb5ff30 2024-05-21T22:01:54.276 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.274+0000 7fe3e9c57b80 1 -- 172.21.15.151:0/1505399530 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a24cb5ff30 msgr2=0x55a24cb59b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:54.276 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.274+0000 7fe3e9c57b80 1 --2- 172.21.15.151:0/1505399530 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a24cb5ff30 0x55a24cb59b20 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fe3c8009d30 tx=0x7fe3c8011720 comp rx=0 tx=0).stop 2024-05-21T22:01:54.276 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.274+0000 7fe3e9c57b80 1 -- 172.21.15.151:0/1505399530 shutdown_connections 2024-05-21T22:01:54.276 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.274+0000 7fe3e9c57b80 1 --2- 172.21.15.151:0/1505399530 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a24cb5a140 0x55a24cb5a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:54.277 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.274+0000 7fe3e9c57b80 1 --2- 172.21.15.151:0/1505399530 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a24cb5f610 0x55a24cb5f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:54.277 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.274+0000 7fe3e9c57b80 1 --2- 172.21.15.151:0/1505399530 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a24cb5ff30 0x55a24cb59b20 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:54.277 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.274+0000 7fe3e9c57b80 1 -- 172.21.15.151:0/1505399530 >> 172.21.15.151:0/1505399530 conn(0x55a24ca87be0 msgr2=0x55a24cb54690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:01:54.277 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.274+0000 7fe3e9c57b80 1 -- 172.21.15.151:0/1505399530 shutdown_connections 2024-05-21T22:01:54.277 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.274+0000 7fe3e9c57b80 1 -- 172.21.15.151:0/1505399530 wait complete. 2024-05-21T22:01:54.277 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.275+0000 7fe3e9c57b80 1 -- start start 2024-05-21T22:01:54.277 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.275+0000 7fe3e9c57b80 1 Processor -- start 2024-05-21T22:01:54.277 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.276+0000 7fe3e9c57b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a24cb5a140 0x55a24cb65c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:54.277 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.276+0000 7fe3e9c57b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a24cb5f610 0x55a24cb63030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:54.277 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.276+0000 7fe3e9c57b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a24cb5ff30 0x55a24cb63570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:54.277 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.276+0000 7fe3e9c57b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a24c9adfa0 con 0x55a24cb5f610 2024-05-21T22:01:54.277 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.276+0000 7fe3e9c57b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55a24ca48760 con 0x55a24cb5a140 2024-05-21T22:01:54.278 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.276+0000 7fe3e9c57b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55a24cb669e0 con 0x55a24cb5ff30 2024-05-21T22:01:54.278 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.276+0000 7fe3e9c57b80 10 client.0 _ll_register_callbacks cb 0x55a24ca51c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:01:54.278 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.276+0000 7fe3d7400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a24cb5a140 0x55a24cb65c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:54.278 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.276+0000 7fe3d7400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a24cb5a140 0x55a24cb65c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:44150/0 (socket says 172.21.15.151:44150) 2024-05-21T22:01:54.278 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.276+0000 7fe3d7400700 1 -- 172.21.15.151:0/2833448084 learned_addr learned my addr 172.21.15.151:0/2833448084 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:01:54.278 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.276+0000 7fe3d7e00700 1 --2- 172.21.15.151:0/2833448084 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a24cb5f610 0x55a24cb63030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:54.278 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.276+0000 7fe3d7400700 1 --2- 172.21.15.151:0/2833448084 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a24cb5ff30 0x55a24cb63570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:54.280 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.276+0000 7fe3d7400700 1 -- 172.21.15.151:0/2833448084 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a24cb5ff30 msgr2=0x55a24cb63570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:54.280 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.276+0000 7fe3d7400700 1 --2- 172.21.15.151:0/2833448084 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a24cb5ff30 0x55a24cb63570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:54.280 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.276+0000 7fe3d7400700 1 -- 172.21.15.151:0/2833448084 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a24cb5f610 msgr2=0x55a24cb63030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:54.280 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.276+0000 7fe3d7400700 1 --2- 172.21.15.151:0/2833448084 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a24cb5f610 0x55a24cb63030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:54.280 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.276+0000 7fe3d7400700 1 -- 172.21.15.151:0/2833448084 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a24c9f78d0 con 0x55a24cb5a140 2024-05-21T22:01:54.280 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.277+0000 7fe3d7400700 1 --2- 172.21.15.151:0/2833448084 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a24cb5a140 0x55a24cb65c90 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe3c4011e00 tx=0x7fe3c40047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a44eca1bd045df6 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:01:54.280 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.277+0000 7fe3c2000700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-21T22:01:54.280 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.277+0000 7fe3c2000700 1 -- 172.21.15.151:0/2833448084 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c4004020 con 0x55a24cb5a140 2024-05-21T22:01:54.280 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.277+0000 7fe3c2000700 1 -- 172.21.15.151:0/2833448084 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fe3c40041c0 con 0x55a24cb5a140 2024-05-21T22:01:54.280 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.277+0000 7fe3e9c57b80 1 -- 172.21.15.151:0/2833448084 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3c80043d0 con 0x55a24cb5a140 2024-05-21T22:01:54.280 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.277+0000 7fe3c2000700 1 -- 172.21.15.151:0/2833448084 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe3c40043c0 con 0x55a24cb5a140 2024-05-21T22:01:54.280 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.277+0000 7fe3e9c57b80 10 client.26702 fetch_fsmap learned FSMap version 29 2024-05-21T22:01:54.280 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.277+0000 7fe3e9c57b80 1 -- 172.21.15.151:0/2833448084 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a24ca78060 con 0x55a24cb5a140 2024-05-21T22:01:54.281 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.278+0000 7fe3c2000700 1 -- 172.21.15.151:0/2833448084 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3c80043d0 con 0x55a24cb5a140 2024-05-21T22:01:54.281 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.278+0000 7fe3c2000700 1 -- 172.21.15.151:0/2833448084 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fe3c4020040 con 0x55a24cb5a140 2024-05-21T22:01:54.281 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.278+0000 7fe3e9c57b80 10 client.26702 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:01:54.281 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.278+0000 7fe3e9c57b80 10 client.26702 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-21T22:01:54.281 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.278+0000 7fe3e9c57b80 20 client.26702 populate_metadata read hostname 'smithi151' 2024-05-21T22:01:54.281 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.278+0000 7fe3e9c57b80 1 --2- 172.21.15.151:0/2833448084 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x55a24cb735c0 0x55a24cb759b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:54.281 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.278+0000 7fe3e9c57b80 4 client.26702 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T22:01:54.281 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.278+0000 7fe3e9c57b80 1 -- 172.21.15.151:0/2833448084 --> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a24cab9d90 con 0x55a24cb735c0 2024-05-21T22:01:54.281 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.279+0000 7fe3d7e00700 1 --2- 172.21.15.151:0/2833448084 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x55a24cb735c0 0x55a24cb759b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:54.281 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.279+0000 7fe3c2000700 10 client.26702 ms_handle_connect on v2:172.21.15.184:6838/3059935143 2024-05-21T22:01:54.281 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.279+0000 7fe3d7e00700 1 --2- 172.21.15.151:0/2833448084 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x55a24cb735c0 0x55a24cb759b0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7822954c6c671116 server_cookie=9995e7af9ae671f6 in_seq=0 out_seq=0 2024-05-21T22:01:54.282 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.280+0000 7fe3c2000700 1 -- 172.21.15.151:0/2833448084 <== mds.1 v2:172.21.15.184:6838/3059935143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55a24cab9d90 con 0x55a24cb735c0 2024-05-21T22:01:54.282 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.280+0000 7fe3c2000700 10 client.26702 handle_command_reply: tid=0 2024-05-21T22:01:54.282 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.280+0000 7fe3e9c57b80 1 client.26702 shutdown 2024-05-21T22:01:54.282 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.280+0000 7fe3e9c57b80 1 -- 172.21.15.151:0/2833448084 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a24cb5a140 msgr2=0x55a24cb65c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:54.283 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.280+0000 7fe3e9c57b80 1 --2- 172.21.15.151:0/2833448084 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a24cb5a140 0x55a24cb65c90 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe3c4011e00 tx=0x7fe3c40047c0 comp rx=0 tx=0).stop 2024-05-21T22:01:54.283 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.280+0000 7fe3e9c57b80 1 -- 172.21.15.151:0/2833448084 shutdown_connections 2024-05-21T22:01:54.283 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.280+0000 7fe3e9c57b80 1 --2- 172.21.15.151:0/2833448084 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x55a24cb735c0 0x55a24cb759b0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:54.283 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.280+0000 7fe3e9c57b80 1 --2- 172.21.15.151:0/2833448084 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a24cb5a140 0x55a24cb65c90 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:54.283 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.280+0000 7fe3e9c57b80 1 --2- 172.21.15.151:0/2833448084 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a24cb5ff30 0x55a24cb63570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:54.283 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.280+0000 7fe3e9c57b80 1 --2- 172.21.15.151:0/2833448084 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a24cb5f610 0x55a24cb63030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:54.283 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.280+0000 7fe3e9c57b80 1 -- 172.21.15.151:0/2833448084 >> 172.21.15.151:0/2833448084 conn(0x55a24ca87be0 msgr2=0x55a24cb53d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:01:54.283 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.280+0000 7fe3c2000700 0 client.26702 ms_handle_reset on v2:172.21.15.184:6838/3059935143 2024-05-21T22:01:54.283 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.281+0000 7fe3e9c57b80 1 -- 172.21.15.151:0/2833448084 shutdown_connections 2024-05-21T22:01:54.283 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.281+0000 7fe3e9c57b80 1 -- 172.21.15.151:0/2833448084 wait complete. 2024-05-21T22:01:54.283 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.281+0000 7fe3e9c57b80 20 client.26702 trim_cache size 0 max 16384 2024-05-21T22:01:54.297 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.297+0000 7fe3e9c57b80 1 Processor -- start 2024-05-21T22:01:54.297 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.297+0000 7fe3e9c57b80 1 -- start start 2024-05-21T22:01:54.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.298+0000 7fe3e9c57b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a24cb5a140 0x55a24cb59b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:54.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.298+0000 7fe3e9c57b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a24cb5f610 0x55a24cb636a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:54.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.298+0000 7fe3e9c57b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a24cb5ff30 0x55a24cb5d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:54.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.298+0000 7fe3e9c57b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d81afec0 con 0x55a24cb5a140 2024-05-21T22:01:54.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.298+0000 7fe3e9c57b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d81b1a00 con 0x55a24cb5ff30 2024-05-21T22:01:54.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.298+0000 7fe3e9c57b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d81b0590 con 0x55a24cb5f610 2024-05-21T22:01:54.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.298+0000 7fe3d7400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a24cb5a140 0x55a24cb59b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:54.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.298+0000 7fe3d7400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a24cb5a140 0x55a24cb59b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:51972/0 (socket says 172.21.15.151:51972) 2024-05-21T22:01:54.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.298+0000 7fe3d7400700 1 -- 172.21.15.151:0/2409607711 learned_addr learned my addr 172.21.15.151:0/2409607711 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:01:54.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.298+0000 7fe3d7e00700 1 --2- 172.21.15.151:0/2409607711 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a24cb5f610 0x55a24cb636a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:54.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.298+0000 7fe3d7400700 1 --2- 172.21.15.151:0/2409607711 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a24cb5ff30 0x55a24cb5d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:54.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.298+0000 7fe3d7e00700 1 -- 172.21.15.151:0/2409607711 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a24cb5ff30 msgr2=0x55a24cb5d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:54.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.298+0000 7fe3d7e00700 1 --2- 172.21.15.151:0/2409607711 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a24cb5ff30 0x55a24cb5d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:54.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.298+0000 7fe3d7e00700 1 -- 172.21.15.151:0/2409607711 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a24cb5a140 msgr2=0x55a24cb59b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:54.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.298+0000 7fe3d7e00700 1 --2- 172.21.15.151:0/2409607711 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a24cb5a140 0x55a24cb59b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:54.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.299+0000 7fe3d7e00700 1 -- 172.21.15.151:0/2409607711 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a24cb76010 con 0x55a24cb5f610 2024-05-21T22:01:54.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.299+0000 7fe3d7400700 1 --2- 172.21.15.151:0/2409607711 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a24cb5a140 0x55a24cb59b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:01:54.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.299+0000 7fe3d7e00700 1 --2- 172.21.15.151:0/2409607711 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a24cb5f610 0x55a24cb636a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe3c8009d00 tx=0x7fe3c8009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54c91ef0ecf9a23b server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:01:54.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.299+0000 7fe3ccc00700 1 -- 172.21.15.151:0/2409607711 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c8030070 con 0x55a24cb5f610 2024-05-21T22:01:54.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.299+0000 7fe3ccc00700 1 -- 172.21.15.151:0/2409607711 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3c8004270 con 0x55a24cb5f610 2024-05-21T22:01:54.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.299+0000 7fe3ccc00700 1 -- 172.21.15.151:0/2409607711 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c802a550 con 0x55a24cb5f610 2024-05-21T22:01:54.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.300+0000 7fe3e9c57b80 1 -- 172.21.15.151:0/2409607711 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a24cb5f610 msgr2=0x55a24cb636a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:54.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.300+0000 7fe3e9c57b80 1 --2- 172.21.15.151:0/2409607711 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a24cb5f610 0x55a24cb636a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe3c8009d00 tx=0x7fe3c8009c30 comp rx=0 tx=0).stop 2024-05-21T22:01:54.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.300+0000 7fe3e9c57b80 1 -- 172.21.15.151:0/2409607711 shutdown_connections 2024-05-21T22:01:54.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.300+0000 7fe3e9c57b80 1 --2- 172.21.15.151:0/2409607711 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a24cb5ff30 0x55a24cb5d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:54.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.300+0000 7fe3e9c57b80 1 --2- 172.21.15.151:0/2409607711 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a24cb5a140 0x55a24cb59b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:54.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.300+0000 7fe3e9c57b80 1 --2- 172.21.15.151:0/2409607711 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a24cb5f610 0x55a24cb636a0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:54.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.300+0000 7fe3e9c57b80 1 -- 172.21.15.151:0/2409607711 >> 172.21.15.151:0/2409607711 conn(0x55a24ca87be0 msgr2=0x55a24cb60310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:01:54.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.300+0000 7fe3e9c57b80 1 -- 172.21.15.151:0/2409607711 shutdown_connections 2024-05-21T22:01:54.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.300+0000 7fe3e9c57b80 1 -- 172.21.15.151:0/2409607711 wait complete. 2024-05-21T22:01:54.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.301+0000 7fe3e9c57b80 1 -- start start 2024-05-21T22:01:54.304 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.301+0000 7fe3e9c57b80 1 Processor -- start 2024-05-21T22:01:54.304 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.301+0000 7fe3e9c57b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a24cb5a140 0x55a24cb7ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:54.304 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.301+0000 7fe3e9c57b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a24cb5f610 0x55a24cb7b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:54.304 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.301+0000 7fe3e9c57b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a24cb5ff30 0x55a24cb7b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:54.305 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.301+0000 7fe3e9c57b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d81b1330 con 0x55a24cb5a140 2024-05-21T22:01:54.305 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.301+0000 7fe3e9c57b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d81b0c60 con 0x55a24cb5f610 2024-05-21T22:01:54.305 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.301+0000 7fe3e9c57b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d81ade60 con 0x55a24cb5ff30 2024-05-21T22:01:54.305 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.301+0000 7fe3e9c57b80 10 client.0 _ll_register_callbacks cb 0x55a24ca51c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:01:54.305 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.302+0000 7fe3d7400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a24cb5a140 0x55a24cb7ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:54.305 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.302+0000 7fe3d7400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a24cb5a140 0x55a24cb7ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:51976/0 (socket says 172.21.15.151:51976) 2024-05-21T22:01:54.305 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.302+0000 7fe3d7400700 1 -- 172.21.15.151:0/1217570717 learned_addr learned my addr 172.21.15.151:0/1217570717 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:01:54.305 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.302+0000 7fe3d7e00700 1 --2- 172.21.15.151:0/1217570717 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a24cb5f610 0x55a24cb7b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:54.305 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.302+0000 7fe3d7400700 1 --2- 172.21.15.151:0/1217570717 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a24cb5ff30 0x55a24cb7b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:54.305 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.302+0000 7fe3d7400700 1 -- 172.21.15.151:0/1217570717 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a24cb5ff30 msgr2=0x55a24cb7b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:54.305 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.302+0000 7fe3d7400700 1 --2- 172.21.15.151:0/1217570717 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a24cb5ff30 0x55a24cb7b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:54.305 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.302+0000 7fe3d7400700 1 -- 172.21.15.151:0/1217570717 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a24cb5f610 msgr2=0x55a24cb7b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:54.305 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.302+0000 7fe3d7400700 1 --2- 172.21.15.151:0/1217570717 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a24cb5f610 0x55a24cb7b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:54.306 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.302+0000 7fe3d7400700 1 -- 172.21.15.151:0/1217570717 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a24cbdd1c0 con 0x55a24cb5a140 2024-05-21T22:01:54.306 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.302+0000 7fe3d7400700 1 --2- 172.21.15.151:0/1217570717 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a24cb5a140 0x55a24cb7ae20 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fe3c4011e00 tx=0x7fe3c4001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d621dc8739a681ce server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:01:54.306 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.302+0000 7fe3c2000700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-21T22:01:54.306 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.303+0000 7fe3c2000700 1 -- 172.21.15.151:0/1217570717 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c4004150 con 0x55a24cb5a140 2024-05-21T22:01:54.306 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.303+0000 7fe3c2000700 1 -- 172.21.15.151:0/1217570717 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fe3c4004830 con 0x55a24cb5a140 2024-05-21T22:01:54.306 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.303+0000 7fe3e9c57b80 1 -- 172.21.15.151:0/1217570717 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3c8004270 con 0x55a24cb5a140 2024-05-21T22:01:54.306 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.303+0000 7fe3c2000700 1 -- 172.21.15.151:0/1217570717 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe3c400e870 con 0x55a24cb5a140 2024-05-21T22:01:54.306 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.303+0000 7fe3e9c57b80 10 client.16584 fetch_fsmap learned FSMap version 29 2024-05-21T22:01:54.306 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.303+0000 7fe3e9c57b80 1 -- 172.21.15.151:0/1217570717 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a24cbde480 con 0x55a24cb5a140 2024-05-21T22:01:54.306 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.303+0000 7fe3c2000700 1 -- 172.21.15.151:0/1217570717 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3c8004270 con 0x55a24cb5a140 2024-05-21T22:01:54.306 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.303+0000 7fe3c2000700 1 -- 172.21.15.151:0/1217570717 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fe3c4022820 con 0x55a24cb5a140 2024-05-21T22:01:54.306 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.303+0000 7fe3e9c57b80 10 client.16584 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:01:54.306 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.303+0000 7fe3e9c57b80 10 client.16584 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-21T22:01:54.307 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.303+0000 7fe3e9c57b80 20 client.16584 populate_metadata read hostname 'smithi151' 2024-05-21T22:01:54.307 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.304+0000 7fe3e9c57b80 1 --2- 172.21.15.151:0/1217570717 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x55a24cbdeb60 0x55a24cbe0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:54.307 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.304+0000 7fe3e9c57b80 4 client.16584 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T22:01:54.307 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.304+0000 7fe3e9c57b80 1 -- 172.21.15.151:0/1217570717 --> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a24cab3570 con 0x55a24cbdeb60 2024-05-21T22:01:54.307 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.304+0000 7fe3d7e00700 1 --2- 172.21.15.151:0/1217570717 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x55a24cbdeb60 0x55a24cbe0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:54.308 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.305+0000 7fe3d7e00700 1 --2- 172.21.15.151:0/1217570717 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x55a24cbdeb60 0x55a24cbe0f80 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fc91174dc0e17bc1 server_cookie=d1b52a5c530e36f in_seq=0 out_seq=0 2024-05-21T22:01:54.308 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.305+0000 7fe3c2000700 10 client.16584 ms_handle_connect on v2:172.21.15.184:6838/3059935143 2024-05-21T22:01:54.308 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.305+0000 7fe3c2000700 1 -- 172.21.15.151:0/1217570717 <== mds.1 v2:172.21.15.184:6838/3059935143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a24cab3570 con 0x55a24cbdeb60 2024-05-21T22:01:54.308 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.305+0000 7fe3c2000700 10 client.16584 handle_command_reply: tid=0 2024-05-21T22:01:54.308 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.305+0000 7fe3e9c57b80 1 client.16584 shutdown 2024-05-21T22:01:54.308 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.305+0000 7fe3e9c57b80 1 -- 172.21.15.151:0/1217570717 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a24cb5a140 msgr2=0x55a24cb7ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:54.308 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.305+0000 7fe3e9c57b80 1 --2- 172.21.15.151:0/1217570717 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a24cb5a140 0x55a24cb7ae20 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fe3c4011e00 tx=0x7fe3c4001e50 comp rx=0 tx=0).stop 2024-05-21T22:01:54.308 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.305+0000 7fe3e9c57b80 1 -- 172.21.15.151:0/1217570717 shutdown_connections 2024-05-21T22:01:54.308 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.305+0000 7fe3e9c57b80 1 --2- 172.21.15.151:0/1217570717 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x55a24cbdeb60 0x55a24cbe0f80 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:54.308 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.305+0000 7fe3e9c57b80 1 --2- 172.21.15.151:0/1217570717 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a24cb5ff30 0x55a24cb7b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:54.309 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.305+0000 7fe3e9c57b80 1 --2- 172.21.15.151:0/1217570717 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a24cb5a140 0x55a24cb7ae20 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:54.309 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.305+0000 7fe3e9c57b80 1 --2- 172.21.15.151:0/1217570717 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a24cb5f610 0x55a24cb7b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:54.309 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.305+0000 7fe3c2000700 0 client.16584 ms_handle_reset on v2:172.21.15.184:6838/3059935143 2024-05-21T22:01:54.309 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.305+0000 7fe3e9c57b80 1 -- 172.21.15.151:0/1217570717 >> 172.21.15.151:0/1217570717 conn(0x55a24ca87be0 msgr2=0x55a24cb60ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:01:54.309 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.305+0000 7fe3e9c57b80 1 -- 172.21.15.151:0/1217570717 shutdown_connections 2024-05-21T22:01:54.309 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.305+0000 7fe3e9c57b80 1 -- 172.21.15.151:0/1217570717 wait complete. 2024-05-21T22:01:54.309 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.306+0000 7fe3e9c57b80 20 client.16584 trim_cache size 0 max 16384 2024-05-21T22:01:54.309 INFO:teuthology.orchestra.run.smithi151.stdout:[] 2024-05-21T22:01:54.310 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.306+0000 7fe3df200700 1 -- 172.21.15.151:0/1381700696 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fe3bc0b29f0 msgr2=0x7fe3bc0b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:54.310 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.306+0000 7fe3df200700 1 --2- 172.21.15.151:0/1381700696 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fe3bc0b29f0 0x7fe3bc0b4eb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe3d00a6cc0 tx=0x7fe3d000b3b0 comp rx=0 tx=0).stop 2024-05-21T22:01:54.310 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.306+0000 7fe3df200700 1 -- 172.21.15.151:0/1381700696 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe3d81095a0 msgr2=0x7fe3d8102f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:54.310 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.306+0000 7fe3df200700 1 --2- 172.21.15.151:0/1381700696 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe3d81095a0 0x7fe3d8102f80 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe3d000b070 tx=0x7fe3d0004c80 comp rx=0 tx=0).stop 2024-05-21T22:01:54.310 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.306+0000 7fe3df200700 1 -- 172.21.15.151:0/1381700696 shutdown_connections 2024-05-21T22:01:54.310 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.306+0000 7fe3df200700 1 --2- 172.21.15.151:0/1381700696 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fe3bc0b29f0 0x7fe3bc0b4eb0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:54.310 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.306+0000 7fe3df200700 1 --2- 172.21.15.151:0/1381700696 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe3d81095a0 0x7fe3d8102f80 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:54.310 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.306+0000 7fe3df200700 1 --2- 172.21.15.151:0/1381700696 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe3d806bab0 0x7fe3d8102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:54.310 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.306+0000 7fe3df200700 1 --2- 172.21.15.151:0/1381700696 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe3d806c470 0x7fe3d8102a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:54.311 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.306+0000 7fe3df200700 1 -- 172.21.15.151:0/1381700696 >> 172.21.15.151:0/1381700696 conn(0x7fe3d80793b0 msgr2=0x7fe3d8101030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:01:54.311 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.307+0000 7fe3df200700 1 -- 172.21.15.151:0/1381700696 shutdown_connections 2024-05-21T22:01:54.311 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:54.307+0000 7fe3df200700 1 -- 172.21.15.151:0/1381700696 wait complete. 2024-05-21T22:01:55.315 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-21T22:01:55.392 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:01:55 smithi151 ceph-mon[31623]: pgmap v192: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:01:55.485 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:01:55 smithi184 ceph-mon[36942]: pgmap v192: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:01:55.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:01:55 smithi164 ceph-mon[36269]: pgmap v192: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:01:55.768 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:01:57.619 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:01:57 smithi151 ceph-mon[31623]: pgmap v193: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:01:57.735 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:01:57 smithi184 ceph-mon[36942]: pgmap v193: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:01:57.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:01:57 smithi164 ceph-mon[36269]: pgmap v193: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:01:57.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.792+0000 7f8802400700 1 -- 172.21.15.151:0/2585502814 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87fc105760 msgr2=0x7f87fc105be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:57.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.792+0000 7f8802400700 1 --2- 172.21.15.151:0/2585502814 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87fc105760 0x7f87fc105be0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f87e8009990 tx=0x7f87e8009ca0 comp rx=0 tx=0).stop 2024-05-21T22:01:57.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.792+0000 7f8802400700 1 -- 172.21.15.151:0/2585502814 shutdown_connections 2024-05-21T22:01:57.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.792+0000 7f8802400700 1 --2- 172.21.15.151:0/2585502814 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f87fc10b760 0x7f87fc10bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:57.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.792+0000 7f8802400700 1 --2- 172.21.15.151:0/2585502814 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f87fc106120 0x7f87fc112630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:57.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.792+0000 7f8802400700 1 --2- 172.21.15.151:0/2585502814 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87fc105760 0x7f87fc105be0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:57.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.792+0000 7f8802400700 1 -- 172.21.15.151:0/2585502814 >> 172.21.15.151:0/2585502814 conn(0x7f87fc101250 msgr2=0x7f87fc103670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:01:57.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.792+0000 7f8802400700 1 -- 172.21.15.151:0/2585502814 shutdown_connections 2024-05-21T22:01:57.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.792+0000 7f8802400700 1 -- 172.21.15.151:0/2585502814 wait complete. 2024-05-21T22:01:57.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.794+0000 7f8802400700 1 Processor -- start 2024-05-21T22:01:57.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.794+0000 7f8802400700 1 -- start start 2024-05-21T22:01:57.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.795+0000 7f8802400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f87fc105760 0x7f87fc19be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:57.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.795+0000 7f8802400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87fc106120 0x7f87fc19c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:57.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.795+0000 7f8802400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f87fc10b760 0x7f87fc1a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:57.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.795+0000 7f8802400700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f87fc115940 con 0x7f87fc105760 2024-05-21T22:01:57.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.795+0000 7f8802400700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f87fc1157c0 con 0x7f87fc10b760 2024-05-21T22:01:57.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.795+0000 7f8802400700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f87fc115ac0 con 0x7f87fc106120 2024-05-21T22:01:57.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.795+0000 7f87fb400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f87fc105760 0x7f87fc19be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:57.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.795+0000 7f87fb400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f87fc105760 0x7f87fc19be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:43896/0 (socket says 172.21.15.151:43896) 2024-05-21T22:01:57.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.795+0000 7f87fb400700 1 -- 172.21.15.151:0/3262881967 learned_addr learned my addr 172.21.15.151:0/3262881967 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:01:57.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.795+0000 7f87faa00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87fc106120 0x7f87fc19c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:57.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.795+0000 7f87fbe00700 1 --2- 172.21.15.151:0/3262881967 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f87fc10b760 0x7f87fc1a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:57.797 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.796+0000 7f87fbe00700 1 -- 172.21.15.151:0/3262881967 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87fc106120 msgr2=0x7f87fc19c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:57.797 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.796+0000 7f87fbe00700 1 --2- 172.21.15.151:0/3262881967 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87fc106120 0x7f87fc19c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:57.797 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.796+0000 7f87fbe00700 1 -- 172.21.15.151:0/3262881967 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f87fc105760 msgr2=0x7f87fc19be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:57.797 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.796+0000 7f87fbe00700 1 --2- 172.21.15.151:0/3262881967 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f87fc105760 0x7f87fc19be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:57.797 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.796+0000 7f87fbe00700 1 -- 172.21.15.151:0/3262881967 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87fc1a0e20 con 0x7f87fc10b760 2024-05-21T22:01:57.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.796+0000 7f87faa00700 1 --2- 172.21.15.151:0/3262881967 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87fc106120 0x7f87fc19c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:01:57.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.796+0000 7f87fbe00700 1 --2- 172.21.15.151:0/3262881967 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f87fc10b760 0x7f87fc1a06e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f87f400efc0 tx=0x7f87f400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:01:57.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.796+0000 7f87f3e00700 1 -- 172.21.15.151:0/3262881967 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f40092e0 con 0x7f87fc10b760 2024-05-21T22:01:57.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.796+0000 7f87f3e00700 1 -- 172.21.15.151:0/3262881967 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87f4004bb0 con 0x7f87fc10b760 2024-05-21T22:01:57.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.796+0000 7f8802400700 1 -- 172.21.15.151:0/3262881967 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87fc1a10b0 con 0x7f87fc10b760 2024-05-21T22:01:57.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.796+0000 7f87f3e00700 1 -- 172.21.15.151:0/3262881967 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f4004020 con 0x7f87fc10b760 2024-05-21T22:01:57.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.797+0000 7f8802400700 1 -- 172.21.15.151:0/3262881967 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87fc1a8990 con 0x7f87fc10b760 2024-05-21T22:01:57.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.801+0000 7f87f3e00700 1 -- 172.21.15.151:0/3262881967 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7f87f401b030 con 0x7f87fc10b760 2024-05-21T22:01:57.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.801+0000 7f880ce8db80 1 Processor -- start 2024-05-21T22:01:57.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.801+0000 7f880ce8db80 1 -- start start 2024-05-21T22:01:57.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.802+0000 7f880ce8db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5566ce42c610 0x5566ce42c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:57.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.802+0000 7f880ce8db80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5566ce42cf30 0x5566ce426b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:57.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.802+0000 7f880ce8db80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5566ce427140 0x5566ce4275c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:57.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.802+0000 7f880ce8db80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5566ce384e40 con 0x5566ce427140 2024-05-21T22:01:57.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.802+0000 7f880ce8db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5566ce3749e0 con 0x5566ce42c610 2024-05-21T22:01:57.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.802+0000 7f880ce8db80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5566ce378a20 con 0x5566ce42cf30 2024-05-21T22:01:57.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.802+0000 7f87faa00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5566ce427140 0x5566ce4275c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:57.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.802+0000 7f87faa00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5566ce42c610 0x5566ce42c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:57.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.802+0000 7f87fb400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5566ce42cf30 0x5566ce426b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:57.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.802+0000 7f87faa00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5566ce42c610 0x5566ce42c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:38844/0 (socket says 172.21.15.151:38844) 2024-05-21T22:01:57.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.802+0000 7f87faa00700 1 -- 172.21.15.151:0/2447238034 learned_addr learned my addr 172.21.15.151:0/2447238034 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:01:57.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.803+0000 7f87faa00700 1 -- 172.21.15.151:0/2447238034 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5566ce42cf30 msgr2=0x5566ce426b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:57.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.803+0000 7f87faa00700 1 --2- 172.21.15.151:0/2447238034 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5566ce42cf30 0x5566ce426b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:57.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.803+0000 7f87faa00700 1 -- 172.21.15.151:0/2447238034 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5566ce427140 msgr2=0x5566ce4275c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:57.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.803+0000 7f87faa00700 1 --2- 172.21.15.151:0/2447238034 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5566ce427140 0x5566ce4275c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:57.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.803+0000 7f87faa00700 1 -- 172.21.15.151:0/2447238034 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566ce33d420 con 0x5566ce42c610 2024-05-21T22:01:57.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.803+0000 7f87faa00700 1 --2- 172.21.15.151:0/2447238034 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5566ce42c610 0x5566ce42c9f0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f87e8000c00 tx=0x7f87e8005960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7bb9048770f87b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:01:57.805 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.803+0000 7f87f3e00700 1 --2- 172.21.15.151:0/3262881967 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f87e00b29f0 0x7f87e00b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:57.805 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.803+0000 7f87f3e00700 1 -- 172.21.15.151:0/3262881967 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f87f40e2150 con 0x7f87fc10b760 2024-05-21T22:01:57.805 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.803+0000 7f87f0c00700 1 -- 172.21.15.151:0/2447238034 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e8005330 con 0x5566ce42c610 2024-05-21T22:01:57.805 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.804+0000 7f87f0c00700 1 -- 172.21.15.151:0/2447238034 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87e80054d0 con 0x5566ce42c610 2024-05-21T22:01:57.805 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.804+0000 7f87fb400700 1 --2- 172.21.15.151:0/3262881967 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f87e00b29f0 0x7f87e00b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:57.805 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.804+0000 7f87f0c00700 1 -- 172.21.15.151:0/2447238034 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e800f6c0 con 0x5566ce42c610 2024-05-21T22:01:57.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.804+0000 7f880ce8db80 1 -- 172.21.15.151:0/2447238034 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5566ce42c610 msgr2=0x5566ce42c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:57.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.804+0000 7f87fb400700 1 --2- 172.21.15.151:0/3262881967 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f87e00b29f0 0x7f87e00b4eb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f87ec0096b0 tx=0x7f87ec009260 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:01:57.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.804+0000 7f880ce8db80 1 --2- 172.21.15.151:0/2447238034 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5566ce42c610 0x5566ce42c9f0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f87e8000c00 tx=0x7f87e8005960 comp rx=0 tx=0).stop 2024-05-21T22:01:57.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.805+0000 7f880ce8db80 1 -- 172.21.15.151:0/2447238034 shutdown_connections 2024-05-21T22:01:57.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.805+0000 7f880ce8db80 1 --2- 172.21.15.151:0/2447238034 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5566ce42c610 0x5566ce42c9f0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:57.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.805+0000 7f880ce8db80 1 --2- 172.21.15.151:0/2447238034 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5566ce427140 0x5566ce4275c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:57.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.805+0000 7f880ce8db80 1 --2- 172.21.15.151:0/2447238034 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5566ce42cf30 0x5566ce426b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:57.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.805+0000 7f880ce8db80 1 -- 172.21.15.151:0/2447238034 >> 172.21.15.151:0/2447238034 conn(0x5566ce354be0 msgr2=0x5566ce421690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:01:57.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.805+0000 7f880ce8db80 1 -- 172.21.15.151:0/2447238034 shutdown_connections 2024-05-21T22:01:57.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.805+0000 7f880ce8db80 1 -- 172.21.15.151:0/2447238034 wait complete. 2024-05-21T22:01:57.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.805+0000 7f880ce8db80 1 -- start start 2024-05-21T22:01:57.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.805+0000 7f880ce8db80 1 Processor -- start 2024-05-21T22:01:57.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.806+0000 7f880ce8db80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5566ce427140 0x5566ce432c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:57.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.806+0000 7f880ce8db80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5566ce42c610 0x5566ce430000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:57.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.806+0000 7f880ce8db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5566ce42cf30 0x5566ce430540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:57.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.806+0000 7f880ce8db80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5566ce27afa0 con 0x5566ce42c610 2024-05-21T22:01:57.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.806+0000 7f880ce8db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5566ce315760 con 0x5566ce42cf30 2024-05-21T22:01:57.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.806+0000 7f880ce8db80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5566ce433980 con 0x5566ce427140 2024-05-21T22:01:57.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.806+0000 7f880ce8db80 10 client.0 _ll_register_callbacks cb 0x5566ce31ec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:01:57.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.806+0000 7f87faa00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5566ce42c610 0x5566ce430000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:57.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.806+0000 7f87fbe00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5566ce42cf30 0x5566ce430540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:57.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.806+0000 7f87faa00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5566ce42c610 0x5566ce430000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:43920/0 (socket says 172.21.15.151:43920) 2024-05-21T22:01:57.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.806+0000 7f87faa00700 1 -- 172.21.15.151:0/1644352364 learned_addr learned my addr 172.21.15.151:0/1644352364 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:01:57.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.806+0000 7f87fb400700 1 --2- 172.21.15.151:0/1644352364 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5566ce427140 0x5566ce432c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:57.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.807+0000 7f87fbe00700 1 -- 172.21.15.151:0/1644352364 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5566ce427140 msgr2=0x5566ce432c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:57.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.807+0000 7f87fbe00700 1 --2- 172.21.15.151:0/1644352364 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5566ce427140 0x5566ce432c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:57.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.807+0000 7f87fbe00700 1 -- 172.21.15.151:0/1644352364 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5566ce42c610 msgr2=0x5566ce430000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:57.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.807+0000 7f87fbe00700 1 --2- 172.21.15.151:0/1644352364 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5566ce42c610 0x5566ce430000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:57.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.807+0000 7f87fbe00700 1 -- 172.21.15.151:0/1644352364 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566ce2c48d0 con 0x5566ce42cf30 2024-05-21T22:01:57.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.807+0000 7f87faa00700 1 --2- 172.21.15.151:0/1644352364 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5566ce42c610 0x5566ce430000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:01:57.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.807+0000 7f87fbe00700 1 --2- 172.21.15.151:0/1644352364 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5566ce42cf30 0x5566ce430540 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f87f40df850 tx=0x7f87f40a99b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d3e4be734db8232 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:01:57.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.807+0000 7f87e5600700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-21T22:01:57.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.807+0000 7f87e5600700 1 -- 172.21.15.151:0/1644352364 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f40a8f80 con 0x5566ce42cf30 2024-05-21T22:01:57.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.807+0000 7f87e5600700 1 -- 172.21.15.151:0/1644352364 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f87f40a3da0 con 0x5566ce42cf30 2024-05-21T22:01:57.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.807+0000 7f880ce8db80 1 -- 172.21.15.151:0/1644352364 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87e80054d0 con 0x5566ce42cf30 2024-05-21T22:01:57.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.808+0000 7f87e5600700 1 -- 172.21.15.151:0/1644352364 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f87f40a9b70 con 0x5566ce42cf30 2024-05-21T22:01:57.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.808+0000 7f880ce8db80 10 client.26720 fetch_fsmap learned FSMap version 29 2024-05-21T22:01:57.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.808+0000 7f880ce8db80 1 -- 172.21.15.151:0/1644352364 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5566ce345060 con 0x5566ce42cf30 2024-05-21T22:01:57.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.808+0000 7f87e5600700 1 -- 172.21.15.151:0/1644352364 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87e80054d0 con 0x5566ce42cf30 2024-05-21T22:01:57.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.808+0000 7f87e5600700 1 -- 172.21.15.151:0/1644352364 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f87f40a4de0 con 0x5566ce42cf30 2024-05-21T22:01:57.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.808+0000 7f880ce8db80 10 client.26720 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:01:57.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.808+0000 7f880ce8db80 10 client.26720 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T22:01:57.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.808+0000 7f880ce8db80 20 client.26720 populate_metadata read hostname 'smithi151' 2024-05-21T22:01:57.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.808+0000 7f880ce8db80 1 --2- 172.21.15.151:0/1644352364 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5566ce4404a0 0x5566ce442890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:57.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.808+0000 7f880ce8db80 4 client.26720 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T22:01:57.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.808+0000 7f880ce8db80 1 -- 172.21.15.151:0/1644352364 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5566ce386d90 con 0x5566ce4404a0 2024-05-21T22:01:57.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.809+0000 7f87fb400700 1 --2- 172.21.15.151:0/1644352364 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5566ce4404a0 0x5566ce442890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:57.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.809+0000 7f87fb400700 1 --2- 172.21.15.151:0/1644352364 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5566ce4404a0 0x5566ce442890 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74c3799747bb6c4f server_cookie=866991705ff2afca in_seq=0 out_seq=0 2024-05-21T22:01:57.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.809+0000 7f87e5600700 10 client.26720 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T22:01:57.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.810+0000 7f87e5600700 1 -- 172.21.15.151:0/1644352364 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5566ce386d90 con 0x5566ce4404a0 2024-05-21T22:01:57.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.810+0000 7f87e5600700 10 client.26720 handle_command_reply: tid=0 2024-05-21T22:01:57.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.810+0000 7f880ce8db80 1 client.26720 shutdown 2024-05-21T22:01:57.813 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.810+0000 7f880ce8db80 1 -- 172.21.15.151:0/1644352364 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5566ce42cf30 msgr2=0x5566ce430540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:57.813 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.810+0000 7f880ce8db80 1 --2- 172.21.15.151:0/1644352364 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5566ce42cf30 0x5566ce430540 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f87f40df850 tx=0x7f87f40a99b0 comp rx=0 tx=0).stop 2024-05-21T22:01:57.813 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.810+0000 7f880ce8db80 1 -- 172.21.15.151:0/1644352364 shutdown_connections 2024-05-21T22:01:57.813 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.810+0000 7f880ce8db80 1 --2- 172.21.15.151:0/1644352364 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5566ce4404a0 0x5566ce442890 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:57.813 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.810+0000 7f880ce8db80 1 --2- 172.21.15.151:0/1644352364 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5566ce42cf30 0x5566ce430540 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:57.813 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.810+0000 7f880ce8db80 1 --2- 172.21.15.151:0/1644352364 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5566ce427140 0x5566ce432c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:57.813 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.810+0000 7f880ce8db80 1 --2- 172.21.15.151:0/1644352364 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5566ce42c610 0x5566ce430000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:57.813 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.810+0000 7f87e5600700 0 client.26720 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T22:01:57.813 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.810+0000 7f880ce8db80 1 -- 172.21.15.151:0/1644352364 >> 172.21.15.151:0/1644352364 conn(0x5566ce354be0 msgr2=0x5566ce420d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:01:57.813 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.811+0000 7f880ce8db80 1 -- 172.21.15.151:0/1644352364 shutdown_connections 2024-05-21T22:01:57.813 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.811+0000 7f880ce8db80 1 -- 172.21.15.151:0/1644352364 wait complete. 2024-05-21T22:01:57.813 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.811+0000 7f880ce8db80 20 client.26720 trim_cache size 0 max 16384 2024-05-21T22:01:57.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.827+0000 7f880ce8db80 1 Processor -- start 2024-05-21T22:01:57.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.827+0000 7f880ce8db80 1 -- start start 2024-05-21T22:01:57.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.828+0000 7f880ce8db80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5566ce427140 0x5566ce426b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:57.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.828+0000 7f880ce8db80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5566ce42c610 0x5566ce42da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:57.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.828+0000 7f880ce8db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5566ce42cf30 0x5566ce4214f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:57.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.828+0000 7f880ce8db80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f87fc1aff40 con 0x5566ce427140 2024-05-21T22:01:57.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.828+0000 7f880ce8db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f87fc1b1a80 con 0x5566ce42cf30 2024-05-21T22:01:57.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.828+0000 7f880ce8db80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f87fc1b0610 con 0x5566ce42c610 2024-05-21T22:01:57.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.828+0000 7f87fb400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5566ce427140 0x5566ce426b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:57.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.828+0000 7f87fb400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5566ce427140 0x5566ce426b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:43936/0 (socket says 172.21.15.151:43936) 2024-05-21T22:01:57.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.828+0000 7f87fb400700 1 -- 172.21.15.151:0/3249459091 learned_addr learned my addr 172.21.15.151:0/3249459091 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:01:57.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.828+0000 7f87faa00700 1 --2- 172.21.15.151:0/3249459091 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5566ce42c610 0x5566ce42da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:57.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.828+0000 7f87fbe00700 1 --2- 172.21.15.151:0/3249459091 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5566ce42cf30 0x5566ce4214f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:57.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.828+0000 7f87fb400700 1 -- 172.21.15.151:0/3249459091 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5566ce42c610 msgr2=0x5566ce42da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:57.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.828+0000 7f87fb400700 1 --2- 172.21.15.151:0/3249459091 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5566ce42c610 0x5566ce42da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:57.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.828+0000 7f87fb400700 1 -- 172.21.15.151:0/3249459091 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5566ce42cf30 msgr2=0x5566ce4214f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:57.831 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.828+0000 7f87fb400700 1 --2- 172.21.15.151:0/3249459091 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5566ce42cf30 0x5566ce4214f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:57.831 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.829+0000 7f87fb400700 1 -- 172.21.15.151:0/3249459091 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566ce4328a0 con 0x5566ce427140 2024-05-21T22:01:57.831 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.829+0000 7f87fbe00700 1 --2- 172.21.15.151:0/3249459091 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5566ce42cf30 0x5566ce4214f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T22:01:57.831 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.829+0000 7f87fb400700 1 --2- 172.21.15.151:0/3249459091 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5566ce427140 0x5566ce426b20 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x5566ce443100 tx=0x7f87ec009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d67d54b9a298d69f server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:01:57.831 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.829+0000 7f87e7e00700 1 -- 172.21.15.151:0/3249459091 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87ec003ac0 con 0x5566ce427140 2024-05-21T22:01:57.831 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.829+0000 7f87e7e00700 1 -- 172.21.15.151:0/3249459091 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87ec02f070 con 0x5566ce427140 2024-05-21T22:01:57.831 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.829+0000 7f87e7e00700 1 -- 172.21.15.151:0/3249459091 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87ec01e680 con 0x5566ce427140 2024-05-21T22:01:57.831 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.830+0000 7f880ce8db80 1 -- 172.21.15.151:0/3249459091 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5566ce427140 msgr2=0x5566ce426b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:57.831 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.830+0000 7f880ce8db80 1 --2- 172.21.15.151:0/3249459091 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5566ce427140 0x5566ce426b20 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x5566ce443100 tx=0x7f87ec009f90 comp rx=0 tx=0).stop 2024-05-21T22:01:57.831 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.830+0000 7f880ce8db80 1 -- 172.21.15.151:0/3249459091 shutdown_connections 2024-05-21T22:01:57.831 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.830+0000 7f880ce8db80 1 --2- 172.21.15.151:0/3249459091 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5566ce42cf30 0x5566ce4214f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:57.831 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.830+0000 7f880ce8db80 1 --2- 172.21.15.151:0/3249459091 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5566ce427140 0x5566ce426b20 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:57.831 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.830+0000 7f880ce8db80 1 --2- 172.21.15.151:0/3249459091 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5566ce42c610 0x5566ce42da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:57.831 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.830+0000 7f880ce8db80 1 -- 172.21.15.151:0/3249459091 >> 172.21.15.151:0/3249459091 conn(0x5566ce354be0 msgr2=0x5566ce42d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:01:57.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.830+0000 7f880ce8db80 1 -- 172.21.15.151:0/3249459091 shutdown_connections 2024-05-21T22:01:57.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.830+0000 7f880ce8db80 1 -- 172.21.15.151:0/3249459091 wait complete. 2024-05-21T22:01:57.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.830+0000 7f880ce8db80 1 -- start start 2024-05-21T22:01:57.833 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.831+0000 7f880ce8db80 1 Processor -- start 2024-05-21T22:01:57.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.831+0000 7f880ce8db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5566ce427140 0x5566ce438c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:57.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.831+0000 7f880ce8db80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5566ce42c610 0x5566ce439150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:57.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.831+0000 7f880ce8db80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5566ce42cf30 0x5566ce435be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:57.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.831+0000 7f880ce8db80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f87fc1b13b0 con 0x5566ce42c610 2024-05-21T22:01:57.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.831+0000 7f880ce8db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f87fc1b0ce0 con 0x5566ce427140 2024-05-21T22:01:57.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.831+0000 7f880ce8db80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f87fc1adee0 con 0x5566ce42cf30 2024-05-21T22:01:57.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.831+0000 7f880ce8db80 10 client.0 _ll_register_callbacks cb 0x5566ce31ec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:01:57.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.832+0000 7f87faa00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5566ce42c610 0x5566ce439150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:57.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.832+0000 7f87faa00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5566ce42c610 0x5566ce439150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:43942/0 (socket says 172.21.15.151:43942) 2024-05-21T22:01:57.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.832+0000 7f87fbe00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5566ce42cf30 0x5566ce435be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:57.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.832+0000 7f87faa00700 1 -- 172.21.15.151:0/269289175 learned_addr learned my addr 172.21.15.151:0/269289175 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:01:57.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.832+0000 7f87fb400700 1 --2- 172.21.15.151:0/269289175 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5566ce427140 0x5566ce438c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:57.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.832+0000 7f87fbe00700 1 -- 172.21.15.151:0/269289175 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5566ce427140 msgr2=0x5566ce438c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:57.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.832+0000 7f87fbe00700 1 --2- 172.21.15.151:0/269289175 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5566ce427140 0x5566ce438c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:57.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.832+0000 7f87fbe00700 1 -- 172.21.15.151:0/269289175 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5566ce42c610 msgr2=0x5566ce439150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:01:57.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.832+0000 7f87fbe00700 1 --2- 172.21.15.151:0/269289175 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5566ce42c610 0x5566ce439150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:57.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.832+0000 7f87fbe00700 1 -- 172.21.15.151:0/269289175 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566ce439820 con 0x5566ce42cf30 2024-05-21T22:01:57.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.832+0000 7f87fbe00700 1 --2- 172.21.15.151:0/269289175 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5566ce42cf30 0x5566ce435be0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f87f40a9400 tx=0x7f87f40a83f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1bd3120b0d69866 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:01:57.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.832+0000 7f87e5600700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-21T22:01:57.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.833+0000 7f87e5600700 1 -- 172.21.15.151:0/269289175 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f40a8f80 con 0x5566ce42cf30 2024-05-21T22:01:57.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.833+0000 7f880ce8db80 1 -- 172.21.15.151:0/269289175 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87ec02f070 con 0x5566ce42cf30 2024-05-21T22:01:57.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.833+0000 7f87e5600700 1 -- 172.21.15.151:0/269289175 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f87f40a5a60 con 0x5566ce42cf30 2024-05-21T22:01:57.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.833+0000 7f87e5600700 1 -- 172.21.15.151:0/269289175 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f87f40a9470 con 0x5566ce42cf30 2024-05-21T22:01:57.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.833+0000 7f880ce8db80 10 client.26737 fetch_fsmap learned FSMap version 29 2024-05-21T22:01:57.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.833+0000 7f880ce8db80 1 -- 172.21.15.151:0/269289175 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5566ce4ab680 con 0x5566ce42cf30 2024-05-21T22:01:57.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.833+0000 7f87e5600700 1 -- 172.21.15.151:0/269289175 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87ec02f070 con 0x5566ce42cf30 2024-05-21T22:01:57.836 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.833+0000 7f87e5600700 1 -- 172.21.15.151:0/269289175 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f87f4003a00 con 0x5566ce42cf30 2024-05-21T22:01:57.836 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.833+0000 7f880ce8db80 10 client.26737 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:01:57.836 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.833+0000 7f880ce8db80 10 client.26737 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T22:01:57.836 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.833+0000 7f880ce8db80 20 client.26737 populate_metadata read hostname 'smithi151' 2024-05-21T22:01:57.836 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.833+0000 7f880ce8db80 1 --2- 172.21.15.151:0/269289175 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5566ce4abd30 0x5566ce4ae150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:01:57.836 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.833+0000 7f880ce8db80 4 client.26737 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T22:01:57.836 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.833+0000 7f880ce8db80 1 -- 172.21.15.151:0/269289175 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5566ce380570 con 0x5566ce4abd30 2024-05-21T22:01:57.837 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.834+0000 7f87fb400700 1 --2- 172.21.15.151:0/269289175 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5566ce4abd30 0x5566ce4ae150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:01:57.837 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.834+0000 7f87fb400700 1 --2- 172.21.15.151:0/269289175 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5566ce4abd30 0x5566ce4ae150 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8254da19186486 server_cookie=4c56726c4d8bac1e in_seq=0 out_seq=0 2024-05-21T22:01:57.837 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.834+0000 7f87e5600700 10 client.26737 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T22:01:57.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.834+0000 7f87e5600700 1 -- 172.21.15.151:0/269289175 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5566ce380570 con 0x5566ce4abd30 2024-05-21T22:01:57.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.834+0000 7f87e5600700 10 client.26737 handle_command_reply: tid=0 2024-05-21T22:01:57.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.834+0000 7f880ce8db80 1 client.26737 shutdown 2024-05-21T22:01:57.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.835+0000 7f880ce8db80 1 -- 172.21.15.151:0/269289175 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5566ce42cf30 msgr2=0x5566ce435be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:57.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.835+0000 7f880ce8db80 1 --2- 172.21.15.151:0/269289175 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5566ce42cf30 0x5566ce435be0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f87f40a9400 tx=0x7f87f40a83f0 comp rx=0 tx=0).stop 2024-05-21T22:01:57.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.835+0000 7f880ce8db80 1 -- 172.21.15.151:0/269289175 shutdown_connections 2024-05-21T22:01:57.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.835+0000 7f880ce8db80 1 --2- 172.21.15.151:0/269289175 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5566ce4abd30 0x5566ce4ae150 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:57.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.835+0000 7f880ce8db80 1 --2- 172.21.15.151:0/269289175 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5566ce427140 0x5566ce438c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:57.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.835+0000 7f880ce8db80 1 --2- 172.21.15.151:0/269289175 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5566ce42cf30 0x5566ce435be0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:57.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.835+0000 7f87e5600700 0 client.26737 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T22:01:57.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.835+0000 7f880ce8db80 1 --2- 172.21.15.151:0/269289175 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5566ce42c610 0x5566ce439150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:57.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.835+0000 7f880ce8db80 1 -- 172.21.15.151:0/269289175 >> 172.21.15.151:0/269289175 conn(0x5566ce354be0 msgr2=0x5566ce431600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:01:57.839 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.835+0000 7f880ce8db80 1 -- 172.21.15.151:0/269289175 shutdown_connections 2024-05-21T22:01:57.839 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.835+0000 7f880ce8db80 1 -- 172.21.15.151:0/269289175 wait complete. 2024-05-21T22:01:57.839 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.835+0000 7f880ce8db80 20 client.26737 trim_cache size 0 max 16384 2024-05-21T22:01:57.839 INFO:teuthology.orchestra.run.smithi151.stdout:[] 2024-05-21T22:01:57.840 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.836+0000 7f8802400700 1 -- 172.21.15.151:0/3262881967 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f87e00b29f0 msgr2=0x7f87e00b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:57.840 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.836+0000 7f8802400700 1 --2- 172.21.15.151:0/3262881967 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f87e00b29f0 0x7f87e00b4eb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f87ec0096b0 tx=0x7f87ec009260 comp rx=0 tx=0).stop 2024-05-21T22:01:57.840 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.836+0000 7f8802400700 1 -- 172.21.15.151:0/3262881967 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f87fc10b760 msgr2=0x7f87fc1a06e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:01:57.840 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.836+0000 7f8802400700 1 --2- 172.21.15.151:0/3262881967 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f87fc10b760 0x7f87fc1a06e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f87f400efc0 tx=0x7f87f400c3a0 comp rx=0 tx=0).stop 2024-05-21T22:01:57.840 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.836+0000 7f8802400700 1 -- 172.21.15.151:0/3262881967 shutdown_connections 2024-05-21T22:01:57.840 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.836+0000 7f8802400700 1 --2- 172.21.15.151:0/3262881967 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f87e00b29f0 0x7f87e00b4eb0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:57.840 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.836+0000 7f8802400700 1 --2- 172.21.15.151:0/3262881967 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f87fc10b760 0x7f87fc1a06e0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:57.840 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.836+0000 7f8802400700 1 --2- 172.21.15.151:0/3262881967 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f87fc105760 0x7f87fc19be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:57.840 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.836+0000 7f8802400700 1 --2- 172.21.15.151:0/3262881967 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87fc106120 0x7f87fc19c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:01:57.840 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.836+0000 7f8802400700 1 -- 172.21.15.151:0/3262881967 >> 172.21.15.151:0/3262881967 conn(0x7f87fc101250 msgr2=0x7f87fc1019a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:01:57.840 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.836+0000 7f8802400700 1 -- 172.21.15.151:0/3262881967 shutdown_connections 2024-05-21T22:01:57.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:01:57.836+0000 7f8802400700 1 -- 172.21.15.151:0/3262881967 wait complete. 2024-05-21T22:01:59.694 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-21T22:01:59.695 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-21T22:01:59.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:01:59 smithi151 ceph-mon[31623]: pgmap v194: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T22:01:59.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:01:59 smithi184 ceph-mon[36942]: pgmap v194: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T22:01:59.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:01:59 smithi164 ceph-mon[36269]: pgmap v194: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T22:02:00.145 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:02:01.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:02:01 smithi151 ceph-mon[31623]: pgmap v195: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:02:01.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:02:01 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T22:02:01.735 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:02:01 smithi184 ceph-mon[36942]: pgmap v195: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:02:01.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:02:01 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T22:02:01.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:02:01 smithi164 ceph-mon[36269]: pgmap v195: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:02:01.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:02:01 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T22:02:02.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.221+0000 7fe461800700 1 -- 172.21.15.151:0/156787281 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe45c10fff0 msgr2=0x7fe45c1123e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:02.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.221+0000 7fe461800700 1 --2- 172.21.15.151:0/156787281 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe45c10fff0 0x7fe45c1123e0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fe45400b0a0 tx=0x7fe45400b3b0 comp rx=0 tx=0).stop 2024-05-21T22:02:02.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.221+0000 7fe461800700 1 -- 172.21.15.151:0/156787281 shutdown_connections 2024-05-21T22:02:02.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.221+0000 7fe461800700 1 --2- 172.21.15.151:0/156787281 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe45c10fff0 0x7fe45c1123e0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:02.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.221+0000 7fe461800700 1 --2- 172.21.15.151:0/156787281 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe45c06b250 0x7fe45c06b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:02.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.221+0000 7fe461800700 1 --2- 172.21.15.151:0/156787281 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe45c06bb70 0x7fe45c10fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:02.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.221+0000 7fe461800700 1 -- 172.21.15.151:0/156787281 >> 172.21.15.151:0/156787281 conn(0x7fe45c0793f0 msgr2=0x7fe45c079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:02:02.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.221+0000 7fe461800700 1 -- 172.21.15.151:0/156787281 shutdown_connections 2024-05-21T22:02:02.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.221+0000 7fe461800700 1 -- 172.21.15.151:0/156787281 wait complete. 2024-05-21T22:02:02.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.222+0000 7fe461800700 1 Processor -- start 2024-05-21T22:02:02.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.222+0000 7fe461800700 1 -- start start 2024-05-21T22:02:02.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.222+0000 7fe461800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe45c06b250 0x7fe45c19beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:02.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.222+0000 7fe461800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe45c06bb70 0x7fe45c19c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:02.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.223+0000 7fe461800700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe45c10fff0 0x7fe45c1a0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:02.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.223+0000 7fe461800700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe45c115960 con 0x7fe45c06bb70 2024-05-21T22:02:02.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.223+0000 7fe461800700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe45c1157e0 con 0x7fe45c06b250 2024-05-21T22:02:02.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.223+0000 7fe461800700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe45c115ae0 con 0x7fe45c10fff0 2024-05-21T22:02:02.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.223+0000 7fe453e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe45c06bb70 0x7fe45c19c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:02.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.223+0000 7fe45aa00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe45c06b250 0x7fe45c19beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:02.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.223+0000 7fe453e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe45c06bb70 0x7fe45c19c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:43954/0 (socket says 172.21.15.151:43954) 2024-05-21T22:02:02.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.223+0000 7fe453e00700 1 -- 172.21.15.151:0/2572817705 learned_addr learned my addr 172.21.15.151:0/2572817705 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:02:02.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.223+0000 7fe45b400700 1 --2- 172.21.15.151:0/2572817705 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe45c10fff0 0x7fe45c1a0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:02.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.223+0000 7fe45aa00700 1 -- 172.21.15.151:0/2572817705 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe45c10fff0 msgr2=0x7fe45c1a0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:02.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.223+0000 7fe45aa00700 1 --2- 172.21.15.151:0/2572817705 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe45c10fff0 0x7fe45c1a0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:02.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.223+0000 7fe45aa00700 1 -- 172.21.15.151:0/2572817705 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe45c06bb70 msgr2=0x7fe45c19c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:02.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.223+0000 7fe45aa00700 1 --2- 172.21.15.151:0/2572817705 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe45c06bb70 0x7fe45c19c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:02.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.223+0000 7fe45aa00700 1 -- 172.21.15.151:0/2572817705 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe45c1a0e70 con 0x7fe45c06b250 2024-05-21T22:02:02.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.224+0000 7fe45aa00700 1 --2- 172.21.15.151:0/2572817705 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe45c06b250 0x7fe45c19beb0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe44c00e9f0 tx=0x7fe44c00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:02:02.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.224+0000 7fe453400700 1 -- 172.21.15.151:0/2572817705 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe44c00c9c0 con 0x7fe45c06b250 2024-05-21T22:02:02.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.224+0000 7fe453400700 1 -- 172.21.15.151:0/2572817705 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe44c00cb60 con 0x7fe45c06b250 2024-05-21T22:02:02.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.224+0000 7fe461800700 1 -- 172.21.15.151:0/2572817705 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe45c1a1160 con 0x7fe45c06b250 2024-05-21T22:02:02.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.224+0000 7fe453400700 1 -- 172.21.15.151:0/2572817705 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe44c018680 con 0x7fe45c06b250 2024-05-21T22:02:02.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.224+0000 7fe461800700 1 -- 172.21.15.151:0/2572817705 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe45c1a89e0 con 0x7fe45c06b250 2024-05-21T22:02:02.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.228+0000 7fe453400700 1 -- 172.21.15.151:0/2572817705 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7fe44c010430 con 0x7fe45c06b250 2024-05-21T22:02:02.229 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.229+0000 7fe46c1a6b80 1 Processor -- start 2024-05-21T22:02:02.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.229+0000 7fe453400700 1 --2- 172.21.15.151:0/2572817705 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fe4400b2a70 0x7fe4400b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:02.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.229+0000 7fe46c1a6b80 1 -- start start 2024-05-21T22:02:02.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.229+0000 7fe453400700 1 -- 172.21.15.151:0/2572817705 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fe44c0e2750 con 0x7fe45c06b250 2024-05-21T22:02:02.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.230+0000 7fe45b400700 1 --2- 172.21.15.151:0/2572817705 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fe4400b2a70 0x7fe4400b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:02.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.230+0000 7fe46c1a6b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56159a795690 0x56159a795a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:02.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.230+0000 7fe46c1a6b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56159a795fb0 0x56159a78fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:02.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.230+0000 7fe46c1a6b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56159a7901c0 0x56159a790640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:02.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.230+0000 7fe46c1a6b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56159a6edec0 con 0x56159a7901c0 2024-05-21T22:02:02.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.230+0000 7fe46c1a6b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56159a6dda60 con 0x56159a795fb0 2024-05-21T22:02:02.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.230+0000 7fe46c1a6b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56159a6e1aa0 con 0x56159a795690 2024-05-21T22:02:02.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.230+0000 7fe45aa00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56159a795690 0x56159a795a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:02.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.230+0000 7fe45b400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56159a7901c0 0x56159a790640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:02.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.230+0000 7fe45aa00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56159a795690 0x56159a795a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:47666/0 (socket says 172.21.15.151:47666) 2024-05-21T22:02:02.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.230+0000 7fe45aa00700 1 -- 172.21.15.151:0/1997323318 learned_addr learned my addr 172.21.15.151:0/1997323318 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:02:02.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.230+0000 7fe453e00700 1 --2- 172.21.15.151:0/1997323318 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56159a795fb0 0x56159a78fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:02.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.231+0000 7fe45aa00700 1 -- 172.21.15.151:0/1997323318 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56159a795fb0 msgr2=0x56159a78fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:02.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.231+0000 7fe45aa00700 1 --2- 172.21.15.151:0/1997323318 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56159a795fb0 0x56159a78fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:02.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.231+0000 7fe45aa00700 1 -- 172.21.15.151:0/1997323318 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56159a7901c0 msgr2=0x56159a790640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:02.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.231+0000 7fe45aa00700 1 --2- 172.21.15.151:0/1997323318 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56159a7901c0 0x56159a790640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:02.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.231+0000 7fe45aa00700 1 -- 172.21.15.151:0/1997323318 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56159a6a64a0 con 0x56159a795690 2024-05-21T22:02:02.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.231+0000 7fe45b400700 1 --2- 172.21.15.151:0/1997323318 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56159a7901c0 0x56159a790640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T22:02:02.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.231+0000 7fe45b400700 1 --2- 172.21.15.151:0/2572817705 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fe4400b2a70 0x7fe4400b4f30 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe454009fd0 tx=0x7fe45400bb90 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:02:02.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.231+0000 7fe45aa00700 1 --2- 172.21.15.151:0/1997323318 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56159a795690 0x56159a795a70 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe44c0a9970 tx=0x7fe44c0a9460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d33fcef6375d5fef server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:02:02.233 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.231+0000 7fe44be00700 1 -- 172.21.15.151:0/1997323318 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe44c0a99e0 con 0x56159a795690 2024-05-21T22:02:02.233 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.231+0000 7fe44be00700 1 -- 172.21.15.151:0/1997323318 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe44c0add90 con 0x56159a795690 2024-05-21T22:02:02.233 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.231+0000 7fe44be00700 1 -- 172.21.15.151:0/1997323318 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe44c0b23b0 con 0x56159a795690 2024-05-21T22:02:02.233 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.232+0000 7fe46c1a6b80 1 -- 172.21.15.151:0/1997323318 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56159a795690 msgr2=0x56159a795a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:02.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.232+0000 7fe46c1a6b80 1 --2- 172.21.15.151:0/1997323318 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56159a795690 0x56159a795a70 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe44c0a9970 tx=0x7fe44c0a9460 comp rx=0 tx=0).stop 2024-05-21T22:02:02.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.232+0000 7fe46c1a6b80 1 -- 172.21.15.151:0/1997323318 shutdown_connections 2024-05-21T22:02:02.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.232+0000 7fe46c1a6b80 1 --2- 172.21.15.151:0/1997323318 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56159a7901c0 0x56159a790640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:02.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.232+0000 7fe46c1a6b80 1 --2- 172.21.15.151:0/1997323318 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56159a795690 0x56159a795a70 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:02.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.232+0000 7fe46c1a6b80 1 --2- 172.21.15.151:0/1997323318 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56159a795fb0 0x56159a78fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:02.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.232+0000 7fe46c1a6b80 1 -- 172.21.15.151:0/1997323318 >> 172.21.15.151:0/1997323318 conn(0x56159a6bdc60 msgr2=0x56159a78a710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:02:02.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.232+0000 7fe46c1a6b80 1 -- 172.21.15.151:0/1997323318 shutdown_connections 2024-05-21T22:02:02.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.232+0000 7fe46c1a6b80 1 -- 172.21.15.151:0/1997323318 wait complete. 2024-05-21T22:02:02.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.233+0000 7fe46c1a6b80 1 -- start start 2024-05-21T22:02:02.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.233+0000 7fe46c1a6b80 1 Processor -- start 2024-05-21T22:02:02.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.234+0000 7fe46c1a6b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56159a7901c0 0x56159a79be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:02.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.234+0000 7fe46c1a6b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56159a795690 0x56159a7991d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:02.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.234+0000 7fe46c1a6b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56159a795fb0 0x56159a799710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:02.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.234+0000 7fe46c1a6b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56159a5e4020 con 0x56159a7901c0 2024-05-21T22:02:02.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.234+0000 7fe46c1a6b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56159a67e7e0 con 0x56159a795690 2024-05-21T22:02:02.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.234+0000 7fe46c1a6b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56159a79ca60 con 0x56159a795fb0 2024-05-21T22:02:02.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.234+0000 7fe46c1a6b80 10 client.0 _ll_register_callbacks cb 0x56159a687cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:02:02.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.234+0000 7fe45b400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56159a795fb0 0x56159a799710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:02.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.234+0000 7fe453e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56159a795690 0x56159a7991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:02.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.234+0000 7fe45aa00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56159a7901c0 0x56159a79be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:02.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.234+0000 7fe453e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56159a795690 0x56159a7991d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:38892/0 (socket says 172.21.15.151:38892) 2024-05-21T22:02:02.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.234+0000 7fe453e00700 1 -- 172.21.15.151:0/4205260132 learned_addr learned my addr 172.21.15.151:0/4205260132 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:02:02.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.235+0000 7fe453e00700 1 -- 172.21.15.151:0/4205260132 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56159a795fb0 msgr2=0x56159a799710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:02.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.235+0000 7fe453e00700 1 --2- 172.21.15.151:0/4205260132 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56159a795fb0 0x56159a799710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:02.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.235+0000 7fe453e00700 1 -- 172.21.15.151:0/4205260132 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56159a7901c0 msgr2=0x56159a79be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:02.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.235+0000 7fe453e00700 1 --2- 172.21.15.151:0/4205260132 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56159a7901c0 0x56159a79be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:02.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.235+0000 7fe453e00700 1 -- 172.21.15.151:0/4205260132 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56159a62d950 con 0x56159a795690 2024-05-21T22:02:02.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.235+0000 7fe453e00700 1 --2- 172.21.15.151:0/4205260132 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56159a795690 0x56159a7991d0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe444012840 tx=0x7fe444012b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d485008f7de2a7a server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:02:02.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.235+0000 7fe448c00700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-21T22:02:02.238 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.236+0000 7fe448c00700 1 -- 172.21.15.151:0/4205260132 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe44400f4e0 con 0x56159a795690 2024-05-21T22:02:02.238 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.236+0000 7fe46c1a6b80 1 -- 172.21.15.151:0/4205260132 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe44c0add90 con 0x56159a795690 2024-05-21T22:02:02.238 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.236+0000 7fe448c00700 1 -- 172.21.15.151:0/4205260132 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fe44400f680 con 0x56159a795690 2024-05-21T22:02:02.238 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.236+0000 7fe448c00700 1 -- 172.21.15.151:0/4205260132 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe44400f880 con 0x56159a795690 2024-05-21T22:02:02.238 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.236+0000 7fe46c1a6b80 10 client.26738 fetch_fsmap learned FSMap version 29 2024-05-21T22:02:02.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.236+0000 7fe46c1a6b80 1 -- 172.21.15.151:0/4205260132 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56159a6ae0e0 con 0x56159a795690 2024-05-21T22:02:02.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.236+0000 7fe448c00700 1 -- 172.21.15.151:0/4205260132 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe44c0add90 con 0x56159a795690 2024-05-21T22:02:02.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.237+0000 7fe448c00700 1 -- 172.21.15.151:0/4205260132 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fe444010ac0 con 0x56159a795690 2024-05-21T22:02:02.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.237+0000 7fe46c1a6b80 10 client.26738 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:02:02.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.237+0000 7fe46c1a6b80 10 client.26738 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T22:02:02.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.237+0000 7fe46c1a6b80 20 client.26738 populate_metadata read hostname 'smithi151' 2024-05-21T22:02:02.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.237+0000 7fe46c1a6b80 1 --2- 172.21.15.151:0/4205260132 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x56159a7a9700 0x56159a7abaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:02.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.237+0000 7fe46c1a6b80 4 client.26738 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T22:02:02.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.237+0000 7fe46c1a6b80 1 -- 172.21.15.151:0/4205260132 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56159a6efe10 con 0x56159a7a9700 2024-05-21T22:02:02.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.237+0000 7fe45aa00700 1 --2- 172.21.15.151:0/4205260132 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x56159a7a9700 0x56159a7abaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:02.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.238+0000 7fe45aa00700 1 --2- 172.21.15.151:0/4205260132 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x56159a7a9700 0x56159a7abaf0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a4224d28ff70afe server_cookie=107947bd52c507d4 in_seq=0 out_seq=0 2024-05-21T22:02:02.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.238+0000 7fe448c00700 10 client.26738 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T22:02:02.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.239+0000 7fe448c00700 1 -- 172.21.15.151:0/4205260132 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x56159a6efe10 con 0x56159a7a9700 2024-05-21T22:02:02.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.239+0000 7fe448c00700 10 client.26738 handle_command_reply: tid=0 2024-05-21T22:02:02.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.239+0000 7fe46c1a6b80 1 client.26738 shutdown 2024-05-21T22:02:02.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.239+0000 7fe46c1a6b80 1 -- 172.21.15.151:0/4205260132 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56159a795690 msgr2=0x56159a7991d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:02.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.239+0000 7fe46c1a6b80 1 --2- 172.21.15.151:0/4205260132 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56159a795690 0x56159a7991d0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe444012840 tx=0x7fe444012b50 comp rx=0 tx=0).stop 2024-05-21T22:02:02.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.239+0000 7fe46c1a6b80 1 -- 172.21.15.151:0/4205260132 shutdown_connections 2024-05-21T22:02:02.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.239+0000 7fe46c1a6b80 1 --2- 172.21.15.151:0/4205260132 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56159a795fb0 0x56159a799710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:02.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.239+0000 7fe46c1a6b80 1 --2- 172.21.15.151:0/4205260132 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56159a7901c0 0x56159a79be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:02.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.239+0000 7fe46c1a6b80 1 --2- 172.21.15.151:0/4205260132 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x56159a7a9700 0x56159a7abaf0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:02.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.239+0000 7fe46c1a6b80 1 --2- 172.21.15.151:0/4205260132 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56159a795690 0x56159a7991d0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:02.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.239+0000 7fe46c1a6b80 1 -- 172.21.15.151:0/4205260132 >> 172.21.15.151:0/4205260132 conn(0x56159a6bdc60 msgr2=0x56159a789df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:02:02.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.239+0000 7fe448c00700 0 client.26738 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T22:02:02.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.239+0000 7fe46c1a6b80 1 -- 172.21.15.151:0/4205260132 shutdown_connections 2024-05-21T22:02:02.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.239+0000 7fe46c1a6b80 1 -- 172.21.15.151:0/4205260132 wait complete. 2024-05-21T22:02:02.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.239+0000 7fe46c1a6b80 20 client.26738 trim_cache size 0 max 16384 2024-05-21T22:02:02.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.262+0000 7fe46c1a6b80 1 Processor -- start 2024-05-21T22:02:02.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.262+0000 7fe46c1a6b80 1 -- start start 2024-05-21T22:02:02.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.263+0000 7fe46c1a6b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56159a7901c0 0x56159a7a5b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:02.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.263+0000 7fe46c1a6b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56159a795690 0x56159a78a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:02.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.263+0000 7fe46c1a6b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56159a795fb0 0x56159a796c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:02.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.263+0000 7fe46c1a6b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe45c1b00f0 con 0x56159a795690 2024-05-21T22:02:02.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.263+0000 7fe46c1a6b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe45c1b1c30 con 0x56159a795fb0 2024-05-21T22:02:02.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.263+0000 7fe46c1a6b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe45c1b07c0 con 0x56159a7901c0 2024-05-21T22:02:02.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.263+0000 7fe453e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56159a795690 0x56159a78a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:02.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.263+0000 7fe45b400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56159a795fb0 0x56159a796c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:02.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.263+0000 7fe45b400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56159a795fb0 0x56159a796c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:38894/0 (socket says 172.21.15.151:38894) 2024-05-21T22:02:02.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.263+0000 7fe45b400700 1 -- 172.21.15.151:0/4051279652 learned_addr learned my addr 172.21.15.151:0/4051279652 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:02:02.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.263+0000 7fe45aa00700 1 --2- 172.21.15.151:0/4051279652 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56159a7901c0 0x56159a7a5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:02.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.264+0000 7fe45b400700 1 -- 172.21.15.151:0/4051279652 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56159a7901c0 msgr2=0x56159a7a5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:02.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.264+0000 7fe45b400700 1 --2- 172.21.15.151:0/4051279652 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56159a7901c0 0x56159a7a5b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:02.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.264+0000 7fe45b400700 1 -- 172.21.15.151:0/4051279652 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56159a795690 msgr2=0x56159a78a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:02.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.264+0000 7fe45b400700 1 --2- 172.21.15.151:0/4051279652 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56159a795690 0x56159a78a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:02.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.264+0000 7fe45b400700 1 -- 172.21.15.151:0/4051279652 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56159a7a71d0 con 0x56159a795fb0 2024-05-21T22:02:02.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.264+0000 7fe45b400700 1 --2- 172.21.15.151:0/4051279652 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56159a795fb0 0x56159a796c20 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fe45c06ae10 tx=0x7fe454024000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3889d283de912373 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:02:02.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.264+0000 7fe44b400700 1 -- 172.21.15.151:0/4051279652 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe454030070 con 0x56159a795fb0 2024-05-21T22:02:02.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.264+0000 7fe44b400700 1 -- 172.21.15.151:0/4051279652 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe454003de0 con 0x56159a795fb0 2024-05-21T22:02:02.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.264+0000 7fe44b400700 1 -- 172.21.15.151:0/4051279652 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe45401f840 con 0x56159a795fb0 2024-05-21T22:02:02.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.265+0000 7fe46c1a6b80 1 -- 172.21.15.151:0/4051279652 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56159a795fb0 msgr2=0x56159a796c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:02.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.265+0000 7fe46c1a6b80 1 --2- 172.21.15.151:0/4051279652 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56159a795fb0 0x56159a796c20 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fe45c06ae10 tx=0x7fe454024000 comp rx=0 tx=0).stop 2024-05-21T22:02:02.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.265+0000 7fe46c1a6b80 1 -- 172.21.15.151:0/4051279652 shutdown_connections 2024-05-21T22:02:02.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.265+0000 7fe46c1a6b80 1 --2- 172.21.15.151:0/4051279652 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56159a795fb0 0x56159a796c20 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:02.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.265+0000 7fe46c1a6b80 1 --2- 172.21.15.151:0/4051279652 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56159a7901c0 0x56159a7a5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:02.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.265+0000 7fe46c1a6b80 1 --2- 172.21.15.151:0/4051279652 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56159a795690 0x56159a78a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:02.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.265+0000 7fe46c1a6b80 1 -- 172.21.15.151:0/4051279652 >> 172.21.15.151:0/4051279652 conn(0x56159a6bdc60 msgr2=0x56159a794960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:02:02.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.266+0000 7fe46c1a6b80 1 -- 172.21.15.151:0/4051279652 shutdown_connections 2024-05-21T22:02:02.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.266+0000 7fe46c1a6b80 1 -- 172.21.15.151:0/4051279652 wait complete. 2024-05-21T22:02:02.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.266+0000 7fe46c1a6b80 1 -- start start 2024-05-21T22:02:02.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.266+0000 7fe46c1a6b80 1 Processor -- start 2024-05-21T22:02:02.269 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.267+0000 7fe46c1a6b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56159a7901c0 0x56159a7a2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:02.269 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.267+0000 7fe46c1a6b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56159a795690 0x56159a79ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:02.269 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.267+0000 7fe46c1a6b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56159a795fb0 0x56159a79f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:02.269 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.267+0000 7fe46c1a6b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe45c1b1560 con 0x56159a795fb0 2024-05-21T22:02:02.269 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.267+0000 7fe46c1a6b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe45c1b0e90 con 0x56159a795690 2024-05-21T22:02:02.269 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.267+0000 7fe46c1a6b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fe45c1ae090 con 0x56159a7901c0 2024-05-21T22:02:02.269 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.267+0000 7fe46c1a6b80 10 client.0 _ll_register_callbacks cb 0x56159a799ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:02:02.269 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.267+0000 7fe45aa00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56159a7901c0 0x56159a7a2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:02.269 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.267+0000 7fe453e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56159a795690 0x56159a79ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:02.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.267+0000 7fe45aa00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56159a7901c0 0x56159a7a2550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:47686/0 (socket says 172.21.15.151:47686) 2024-05-21T22:02:02.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.267+0000 7fe45aa00700 1 -- 172.21.15.151:0/2519358203 learned_addr learned my addr 172.21.15.151:0/2519358203 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:02:02.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.267+0000 7fe453e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56159a795690 0x56159a79ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:38908/0 (socket says 172.21.15.151:38908) 2024-05-21T22:02:02.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.267+0000 7fe45b400700 1 --2- 172.21.15.151:0/2519358203 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56159a795fb0 0x56159a79f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:02.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.268+0000 7fe45aa00700 1 -- 172.21.15.151:0/2519358203 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56159a795690 msgr2=0x56159a79ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:02.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.268+0000 7fe45aa00700 1 --2- 172.21.15.151:0/2519358203 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56159a795690 0x56159a79ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:02.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.268+0000 7fe45aa00700 1 -- 172.21.15.151:0/2519358203 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56159a795fb0 msgr2=0x56159a79f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:02.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.268+0000 7fe45aa00700 1 --2- 172.21.15.151:0/2519358203 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56159a795fb0 0x56159a79f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:02.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.268+0000 7fe45aa00700 1 -- 172.21.15.151:0/2519358203 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56159a814250 con 0x56159a7901c0 2024-05-21T22:02:02.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.268+0000 7fe45b400700 1 --2- 172.21.15.151:0/2519358203 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56159a795fb0 0x56159a79f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T22:02:02.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.268+0000 7fe45aa00700 1 --2- 172.21.15.151:0/2519358203 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56159a7901c0 0x56159a7a2550 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe44c0b2980 tx=0x7fe44c009790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9289b1c051c54500 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:02:02.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.268+0000 7fe448c00700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-21T22:02:02.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.268+0000 7fe448c00700 1 -- 172.21.15.151:0/2519358203 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe44c0add90 con 0x56159a7901c0 2024-05-21T22:02:02.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.268+0000 7fe448c00700 1 -- 172.21.15.151:0/2519358203 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fe44c0a5fd0 con 0x56159a7901c0 2024-05-21T22:02:02.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.268+0000 7fe46c1a6b80 1 -- 172.21.15.151:0/2519358203 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe454003de0 con 0x56159a7901c0 2024-05-21T22:02:02.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.268+0000 7fe448c00700 1 -- 172.21.15.151:0/2519358203 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe44c0a9c40 con 0x56159a7901c0 2024-05-21T22:02:02.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.269+0000 7fe46c1a6b80 10 client.26755 fetch_fsmap learned FSMap version 29 2024-05-21T22:02:02.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.269+0000 7fe46c1a6b80 1 -- 172.21.15.151:0/2519358203 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56159a815450 con 0x56159a7901c0 2024-05-21T22:02:02.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.269+0000 7fe448c00700 1 -- 172.21.15.151:0/2519358203 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe454003de0 con 0x56159a7901c0 2024-05-21T22:02:02.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.269+0000 7fe448c00700 1 -- 172.21.15.151:0/2519358203 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fe44c010720 con 0x56159a7901c0 2024-05-21T22:02:02.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.269+0000 7fe46c1a6b80 10 client.26755 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:02:02.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.269+0000 7fe46c1a6b80 10 client.26755 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T22:02:02.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.269+0000 7fe46c1a6b80 20 client.26755 populate_metadata read hostname 'smithi151' 2024-05-21T22:02:02.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.270+0000 7fe46c1a6b80 1 --2- 172.21.15.151:0/2519358203 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x56159a815680 0x56159a817a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:02.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.270+0000 7fe46c1a6b80 4 client.26755 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-21T22:02:02.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.270+0000 7fe46c1a6b80 1 -- 172.21.15.151:0/2519358203 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56159a6e95f0 con 0x56159a815680 2024-05-21T22:02:02.273 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.270+0000 7fe453e00700 1 --2- 172.21.15.151:0/2519358203 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x56159a815680 0x56159a817a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:02.273 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.271+0000 7fe453e00700 1 --2- 172.21.15.151:0/2519358203 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x56159a815680 0x56159a817a70 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9c26b490f1d2004 server_cookie=4846117dbf359782 in_seq=0 out_seq=0 2024-05-21T22:02:02.273 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.271+0000 7fe448c00700 10 client.26755 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T22:02:02.273 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.272+0000 7fe448c00700 1 -- 172.21.15.151:0/2519358203 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56159a6e95f0 con 0x56159a815680 2024-05-21T22:02:02.273 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.272+0000 7fe448c00700 10 client.26755 handle_command_reply: tid=0 2024-05-21T22:02:02.273 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.272+0000 7fe46c1a6b80 1 client.26755 shutdown 2024-05-21T22:02:02.273 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.272+0000 7fe46c1a6b80 1 -- 172.21.15.151:0/2519358203 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56159a7901c0 msgr2=0x56159a7a2550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:02.273 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.272+0000 7fe46c1a6b80 1 --2- 172.21.15.151:0/2519358203 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56159a7901c0 0x56159a7a2550 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe44c0b2980 tx=0x7fe44c009790 comp rx=0 tx=0).stop 2024-05-21T22:02:02.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.272+0000 7fe46c1a6b80 1 -- 172.21.15.151:0/2519358203 shutdown_connections 2024-05-21T22:02:02.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.272+0000 7fe46c1a6b80 1 --2- 172.21.15.151:0/2519358203 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56159a795fb0 0x56159a79f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:02.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.272+0000 7fe46c1a6b80 1 --2- 172.21.15.151:0/2519358203 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56159a7901c0 0x56159a7a2550 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:02.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.272+0000 7fe46c1a6b80 1 --2- 172.21.15.151:0/2519358203 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x56159a815680 0x56159a817a70 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:02.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.272+0000 7fe46c1a6b80 1 --2- 172.21.15.151:0/2519358203 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56159a795690 0x56159a79ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:02.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.272+0000 7fe46c1a6b80 1 -- 172.21.15.151:0/2519358203 >> 172.21.15.151:0/2519358203 conn(0x56159a6bdc60 msgr2=0x56159a79b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:02:02.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.272+0000 7fe448c00700 0 client.26755 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T22:02:02.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.272+0000 7fe46c1a6b80 1 -- 172.21.15.151:0/2519358203 shutdown_connections 2024-05-21T22:02:02.275 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.272+0000 7fe46c1a6b80 1 -- 172.21.15.151:0/2519358203 wait complete. 2024-05-21T22:02:02.275 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.273+0000 7fe46c1a6b80 20 client.26755 trim_cache size 0 max 16384 2024-05-21T22:02:02.275 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-05-21T22:02:02.275 INFO:teuthology.orchestra.run.smithi151.stdout: "return_code": 0, 2024-05-21T22:02:02.275 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_tag": "4faa4a36-7ba1-43a8-8b2b-9ab70af57064", 2024-05-21T22:02:02.275 INFO:teuthology.orchestra.run.smithi151.stdout: "mode": "asynchronous" 2024-05-21T22:02:02.275 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-05-21T22:02:02.276 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.273+0000 7fe461800700 1 -- 172.21.15.151:0/2572817705 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fe4400b2a70 msgr2=0x7fe4400b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:02.276 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.273+0000 7fe461800700 1 --2- 172.21.15.151:0/2572817705 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fe4400b2a70 0x7fe4400b4f30 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe454009fd0 tx=0x7fe45400bb90 comp rx=0 tx=0).stop 2024-05-21T22:02:02.276 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.273+0000 7fe461800700 1 -- 172.21.15.151:0/2572817705 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe45c06b250 msgr2=0x7fe45c19beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:02.276 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.273+0000 7fe461800700 1 --2- 172.21.15.151:0/2572817705 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe45c06b250 0x7fe45c19beb0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe44c00e9f0 tx=0x7fe44c00ed00 comp rx=0 tx=0).stop 2024-05-21T22:02:02.276 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.273+0000 7fe461800700 1 -- 172.21.15.151:0/2572817705 shutdown_connections 2024-05-21T22:02:02.276 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.273+0000 7fe461800700 1 --2- 172.21.15.151:0/2572817705 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fe4400b2a70 0x7fe4400b4f30 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:02.276 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.273+0000 7fe461800700 1 --2- 172.21.15.151:0/2572817705 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe45c06b250 0x7fe45c19beb0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:02.276 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.273+0000 7fe461800700 1 --2- 172.21.15.151:0/2572817705 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe45c10fff0 0x7fe45c1a0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:02.277 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.273+0000 7fe461800700 1 --2- 172.21.15.151:0/2572817705 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe45c06bb70 0x7fe45c19c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:02.277 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.273+0000 7fe461800700 1 -- 172.21.15.151:0/2572817705 >> 172.21.15.151:0/2572817705 conn(0x7fe45c0793f0 msgr2=0x7fe45c1114e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:02:02.277 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.274+0000 7fe461800700 1 -- 172.21.15.151:0/2572817705 shutdown_connections 2024-05-21T22:02:02.277 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:02.274+0000 7fe461800700 1 -- 172.21.15.151:0/2572817705 wait complete. 2024-05-21T22:02:02.701 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:02:02 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T22:02:02.701 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:02:02 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T22:02:02.701 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:02:02 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T22:02:02.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:02:02 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T22:02:02.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:02:02 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T22:02:02.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:02:02 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T22:02:02.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:02:02 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T22:02:02.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:02:02 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T22:02:02.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:02:02 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T22:02:03.178 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-21T22:02:03.631 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:02:03.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:02:03 smithi151 ceph-mon[31623]: scrub queued for path: / 2024-05-21T22:02:03.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:02:03 smithi151 ceph-mon[31623]: scrub summary: idle+waiting paths [/] 2024-05-21T22:02:03.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:02:03 smithi151 ceph-mon[31623]: scrub summary: active paths [/] 2024-05-21T22:02:03.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:02:03 smithi151 ceph-mon[31623]: scrub summary: idle+waiting paths [/] 2024-05-21T22:02:03.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:02:03 smithi151 ceph-mon[31623]: pgmap v196: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-05-21T22:02:03.735 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:02:03 smithi184 ceph-mon[36942]: scrub queued for path: / 2024-05-21T22:02:03.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:02:03 smithi184 ceph-mon[36942]: scrub summary: idle+waiting paths [/] 2024-05-21T22:02:03.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:02:03 smithi184 ceph-mon[36942]: scrub summary: active paths [/] 2024-05-21T22:02:03.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:02:03 smithi184 ceph-mon[36942]: scrub summary: idle+waiting paths [/] 2024-05-21T22:02:03.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:02:03 smithi184 ceph-mon[36942]: pgmap v196: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-05-21T22:02:03.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:02:03 smithi164 ceph-mon[36269]: scrub queued for path: / 2024-05-21T22:02:03.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:02:03 smithi164 ceph-mon[36269]: scrub summary: idle+waiting paths [/] 2024-05-21T22:02:03.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:02:03 smithi164 ceph-mon[36269]: scrub summary: active paths [/] 2024-05-21T22:02:03.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:02:03 smithi164 ceph-mon[36269]: scrub summary: idle+waiting paths [/] 2024-05-21T22:02:03.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:02:03 smithi164 ceph-mon[36269]: pgmap v196: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-05-21T22:02:04.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:02:04 smithi184 ceph-mon[36942]: scrub summary: idle 2024-05-21T22:02:04.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:02:04 smithi164 ceph-mon[36269]: scrub summary: idle 2024-05-21T22:02:04.819 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:02:04 smithi151 ceph-mon[31623]: scrub summary: idle 2024-05-21T22:02:05.522 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:02:05 smithi151 ceph-mon[31623]: pgmap v197: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-05-21T22:02:05.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:02:05 smithi184 ceph-mon[36942]: pgmap v197: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-05-21T22:02:05.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:02:05 smithi164 ceph-mon[36269]: pgmap v197: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-05-21T22:02:05.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.797+0000 7f2b91a00700 1 -- 172.21.15.151:0/567063268 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2b8c103e20 msgr2=0x7f2b8c110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:05.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.797+0000 7f2b91a00700 1 --2- 172.21.15.151:0/567063268 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2b8c103e20 0x7f2b8c110330 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f2b8400b0a0 tx=0x7f2b8400b3b0 comp rx=0 tx=0).stop 2024-05-21T22:02:05.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.798+0000 7f2b91a00700 1 -- 172.21.15.151:0/567063268 shutdown_connections 2024-05-21T22:02:05.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.798+0000 7f2b91a00700 1 --2- 172.21.15.151:0/567063268 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2b8c103e20 0x7f2b8c110330 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:05.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.798+0000 7f2b91a00700 1 --2- 172.21.15.151:0/567063268 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2b8c1094f0 0x7f2b8c1098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:05.799 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.798+0000 7f2b91a00700 1 --2- 172.21.15.151:0/567063268 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b8c103460 0x7f2b8c1038e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:05.799 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.798+0000 7f2b91a00700 1 -- 172.21.15.151:0/567063268 >> 172.21.15.151:0/567063268 conn(0x7f2b8c0fef50 msgr2=0x7f2b8c101370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:02:05.799 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.798+0000 7f2b91a00700 1 -- 172.21.15.151:0/567063268 shutdown_connections 2024-05-21T22:02:05.799 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.798+0000 7f2b91a00700 1 -- 172.21.15.151:0/567063268 wait complete. 2024-05-21T22:02:05.799 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.799+0000 7f2b91a00700 1 Processor -- start 2024-05-21T22:02:05.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.799+0000 7f2b91a00700 1 -- start start 2024-05-21T22:02:05.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.799+0000 7f2b91a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2b8c103460 0x7f2b8c19bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:05.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.799+0000 7f2b91a00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2b8c103e20 0x7f2b8c19c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:05.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.799+0000 7f2b91a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b8c1094f0 0x7f2b8c1a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:05.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.799+0000 7f2b91a00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2b8c113600 con 0x7f2b8c103460 2024-05-21T22:02:05.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.799+0000 7f2b91a00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2b8c113480 con 0x7f2b8c1094f0 2024-05-21T22:02:05.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.799+0000 7f2b91a00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2b8c113780 con 0x7f2b8c103e20 2024-05-21T22:02:05.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.800+0000 7f2b8be00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2b8c103460 0x7f2b8c19bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:05.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.800+0000 7f2b91000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b8c1094f0 0x7f2b8c1a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:05.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.800+0000 7f2b8be00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2b8c103460 0x7f2b8c19bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:55014/0 (socket says 172.21.15.151:55014) 2024-05-21T22:02:05.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.800+0000 7f2b91000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b8c1094f0 0x7f2b8c1a05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:39968/0 (socket says 172.21.15.151:39968) 2024-05-21T22:02:05.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.800+0000 7f2b8be00700 1 -- 172.21.15.151:0/1645091143 learned_addr learned my addr 172.21.15.151:0/1645091143 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:02:05.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.800+0000 7f2b8b400700 1 --2- 172.21.15.151:0/1645091143 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2b8c103e20 0x7f2b8c19c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:05.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.800+0000 7f2b91000700 1 -- 172.21.15.151:0/1645091143 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2b8c103e20 msgr2=0x7f2b8c19c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:05.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.800+0000 7f2b91000700 1 --2- 172.21.15.151:0/1645091143 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2b8c103e20 0x7f2b8c19c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:05.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.800+0000 7f2b91000700 1 -- 172.21.15.151:0/1645091143 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2b8c103460 msgr2=0x7f2b8c19bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:05.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.800+0000 7f2b91000700 1 --2- 172.21.15.151:0/1645091143 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2b8c103460 0x7f2b8c19bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:05.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.800+0000 7f2b91000700 1 -- 172.21.15.151:0/1645091143 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b8c1a0d30 con 0x7f2b8c1094f0 2024-05-21T22:02:05.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.801+0000 7f2b91000700 1 --2- 172.21.15.151:0/1645091143 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b8c1094f0 0x7f2b8c1a05f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f2b8400b070 tx=0x7f2b84004d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:02:05.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.801+0000 7f2b88c00700 1 -- 172.21.15.151:0/1645091143 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b84021070 con 0x7f2b8c1094f0 2024-05-21T22:02:05.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.801+0000 7f2b91a00700 1 -- 172.21.15.151:0/1645091143 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b8c1a0fc0 con 0x7f2b8c1094f0 2024-05-21T22:02:05.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.801+0000 7f2b88c00700 1 -- 172.21.15.151:0/1645091143 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b84007e30 con 0x7f2b8c1094f0 2024-05-21T22:02:05.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.801+0000 7f2b88c00700 1 -- 172.21.15.151:0/1645091143 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b840045d0 con 0x7f2b8c1094f0 2024-05-21T22:02:05.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.801+0000 7f2b91a00700 1 -- 172.21.15.151:0/1645091143 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b8c1a88a0 con 0x7f2b8c1094f0 2024-05-21T22:02:05.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.805+0000 7f2b88c00700 1 -- 172.21.15.151:0/1645091143 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7f2b84003c00 con 0x7f2b8c1094f0 2024-05-21T22:02:05.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.806+0000 7f2b9d8dbb80 1 Processor -- start 2024-05-21T22:02:05.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.806+0000 7f2b9d8dbb80 1 -- start start 2024-05-21T22:02:05.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.806+0000 7f2b88c00700 1 --2- 172.21.15.151:0/1645091143 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f2b700b2a70 0x7f2b700b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:05.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.806+0000 7f2b88c00700 1 -- 172.21.15.151:0/1645091143 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f2b8400f020 con 0x7f2b8c1094f0 2024-05-21T22:02:05.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.806+0000 7f2b9d8dbb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564e60271020 0x564e60271400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:05.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.806+0000 7f2b9d8dbb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564e60271940 0x564e6026b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:05.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.806+0000 7f2b9d8dbb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564e6026bad0 0x564e6026bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:05.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.806+0000 7f2b9d8dbb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x564e601c9e40 con 0x564e60271020 2024-05-21T22:02:05.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.806+0000 7f2b9d8dbb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x564e601b99e0 con 0x564e60271940 2024-05-21T22:02:05.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.806+0000 7f2b9d8dbb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x564e601bda20 con 0x564e6026bad0 2024-05-21T22:02:05.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.807+0000 7f2b8b400700 1 --2- 172.21.15.151:0/1645091143 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f2b700b2a70 0x7f2b700b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:05.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.807+0000 7f2b8be00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564e60271020 0x564e60271400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:05.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.807+0000 7f2b91000700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564e6026bad0 0x564e6026bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:05.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.807+0000 7f2b8be00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564e60271020 0x564e60271400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:55016/0 (socket says 172.21.15.151:55016) 2024-05-21T22:02:05.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.807+0000 7f2b8be00700 1 -- 172.21.15.151:0/2448805019 learned_addr learned my addr 172.21.15.151:0/2448805019 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:02:05.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.807+0000 7f2b91000700 1 --2- 172.21.15.151:0/2448805019 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564e6026bad0 0x564e6026bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:32816/0 (socket says 172.21.15.151:32816) 2024-05-21T22:02:05.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.807+0000 7f2b8b400700 1 --2- 172.21.15.151:0/2448805019 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564e60271940 0x564e6026b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:05.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.807+0000 7f2b91000700 1 -- 172.21.15.151:0/2448805019 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564e60271940 msgr2=0x564e6026b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:05.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.807+0000 7f2b91000700 1 --2- 172.21.15.151:0/2448805019 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564e60271940 0x564e6026b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:05.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.807+0000 7f2b91000700 1 -- 172.21.15.151:0/2448805019 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564e60271020 msgr2=0x564e60271400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:05.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.807+0000 7f2b91000700 1 --2- 172.21.15.151:0/2448805019 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564e60271020 0x564e60271400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:05.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.807+0000 7f2b91000700 1 -- 172.21.15.151:0/2448805019 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e60182420 con 0x564e6026bad0 2024-05-21T22:02:05.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.807+0000 7f2b8be00700 1 --2- 172.21.15.151:0/2448805019 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564e60271020 0x564e60271400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:02:05.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.808+0000 7f2b8b400700 1 --2- 172.21.15.151:0/1645091143 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f2b700b2a70 0x7f2b700b4f30 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c1107e0 tx=0x7f2b740118e0 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:02:05.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.808+0000 7f2b91000700 1 --2- 172.21.15.151:0/2448805019 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564e6026bad0 0x564e6026bf30 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2b840e0f90 tx=0x7f2b840aa6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bdabdedbc6bbcf57 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:02:05.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.808+0000 7f2b81600700 1 -- 172.21.15.151:0/2448805019 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b840aabb0 con 0x564e6026bad0 2024-05-21T22:02:05.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.808+0000 7f2b81600700 1 -- 172.21.15.151:0/2448805019 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b840aef60 con 0x564e6026bad0 2024-05-21T22:02:05.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.808+0000 7f2b81600700 1 -- 172.21.15.151:0/2448805019 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b840a6e40 con 0x564e6026bad0 2024-05-21T22:02:05.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.809+0000 7f2b9d8dbb80 1 -- 172.21.15.151:0/2448805019 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564e6026bad0 msgr2=0x564e6026bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:05.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.809+0000 7f2b9d8dbb80 1 --2- 172.21.15.151:0/2448805019 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564e6026bad0 0x564e6026bf30 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2b840e0f90 tx=0x7f2b840aa6c0 comp rx=0 tx=0).stop 2024-05-21T22:02:05.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.809+0000 7f2b9d8dbb80 1 -- 172.21.15.151:0/2448805019 shutdown_connections 2024-05-21T22:02:05.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.809+0000 7f2b9d8dbb80 1 --2- 172.21.15.151:0/2448805019 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564e6026bad0 0x564e6026bf30 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:05.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.809+0000 7f2b9d8dbb80 1 --2- 172.21.15.151:0/2448805019 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564e60271020 0x564e60271400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:05.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.809+0000 7f2b9d8dbb80 1 --2- 172.21.15.151:0/2448805019 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564e60271940 0x564e6026b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:05.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.809+0000 7f2b9d8dbb80 1 -- 172.21.15.151:0/2448805019 >> 172.21.15.151:0/2448805019 conn(0x564e60199be0 msgr2=0x564e60266150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:02:05.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.809+0000 7f2b9d8dbb80 1 -- 172.21.15.151:0/2448805019 shutdown_connections 2024-05-21T22:02:05.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.809+0000 7f2b9d8dbb80 1 -- 172.21.15.151:0/2448805019 wait complete. 2024-05-21T22:02:05.813 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.810+0000 7f2b9d8dbb80 1 -- start start 2024-05-21T22:02:05.813 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.810+0000 7f2b9d8dbb80 1 Processor -- start 2024-05-21T22:02:05.813 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.810+0000 7f2b9d8dbb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564e6026bad0 0x564e602776d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:05.813 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.810+0000 7f2b9d8dbb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564e60271020 0x564e60274a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:05.813 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.810+0000 7f2b9d8dbb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564e60271940 0x564e60274fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:05.813 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.810+0000 7f2b9d8dbb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x564e600bffa0 con 0x564e6026bad0 2024-05-21T22:02:05.813 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.810+0000 7f2b9d8dbb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x564e6015a760 con 0x564e60271020 2024-05-21T22:02:05.813 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.810+0000 7f2b9d8dbb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x564e60278220 con 0x564e60271940 2024-05-21T22:02:05.813 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.811+0000 7f2b9d8dbb80 10 client.0 _ll_register_callbacks cb 0x564e60163c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:02:05.813 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.811+0000 7f2b8be00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564e6026bad0 0x564e602776d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:05.813 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.811+0000 7f2b8be00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564e6026bad0 0x564e602776d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:55024/0 (socket says 172.21.15.151:55024) 2024-05-21T22:02:05.813 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.811+0000 7f2b91000700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564e60271940 0x564e60274fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:05.813 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.811+0000 7f2b8be00700 1 -- 172.21.15.151:0/1757191901 learned_addr learned my addr 172.21.15.151:0/1757191901 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:02:05.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.811+0000 7f2b8b400700 1 --2- 172.21.15.151:0/1757191901 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564e60271020 0x564e60274a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:05.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.811+0000 7f2b91000700 1 -- 172.21.15.151:0/1757191901 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564e60271020 msgr2=0x564e60274a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:05.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.811+0000 7f2b91000700 1 --2- 172.21.15.151:0/1757191901 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564e60271020 0x564e60274a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:05.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.811+0000 7f2b91000700 1 -- 172.21.15.151:0/1757191901 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564e6026bad0 msgr2=0x564e602776d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:05.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.811+0000 7f2b91000700 1 --2- 172.21.15.151:0/1757191901 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564e6026bad0 0x564e602776d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:05.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.811+0000 7f2b91000700 1 -- 172.21.15.151:0/1757191901 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e601098d0 con 0x564e60271940 2024-05-21T22:02:05.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.811+0000 7f2b8b400700 1 --2- 172.21.15.151:0/1757191901 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564e60271020 0x564e60274a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T22:02:05.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.811+0000 7f2b8be00700 1 --2- 172.21.15.151:0/1757191901 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564e6026bad0 0x564e602776d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:02:05.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.812+0000 7f2b91000700 1 --2- 172.21.15.151:0/1757191901 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564e60271940 0x564e60274fb0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x564e60278130 tx=0x7f2b84004f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a7fe04998bd103a server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:02:05.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.812+0000 7f2b7a000700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-21T22:02:05.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.812+0000 7f2b7a000700 1 -- 172.21.15.151:0/1757191901 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b840c5210 con 0x564e60271940 2024-05-21T22:02:05.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.812+0000 7f2b9d8dbb80 1 -- 172.21.15.151:0/1757191901 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b840aef60 con 0x564e60271940 2024-05-21T22:02:05.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.812+0000 7f2b7a000700 1 -- 172.21.15.151:0/1757191901 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f2b840a6e40 con 0x564e60271940 2024-05-21T22:02:05.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.812+0000 7f2b7a000700 1 -- 172.21.15.151:0/1757191901 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2b840aabb0 con 0x564e60271940 2024-05-21T22:02:05.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.812+0000 7f2b9d8dbb80 10 client.26767 fetch_fsmap learned FSMap version 29 2024-05-21T22:02:05.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.812+0000 7f2b9d8dbb80 1 -- 172.21.15.151:0/1757191901 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564e6018a060 con 0x564e60271940 2024-05-21T22:02:05.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.812+0000 7f2b7a000700 1 -- 172.21.15.151:0/1757191901 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b840aef60 con 0x564e60271940 2024-05-21T22:02:05.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.813+0000 7f2b7a000700 1 -- 172.21.15.151:0/1757191901 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f2b84011070 con 0x564e60271940 2024-05-21T22:02:05.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.813+0000 7f2b9d8dbb80 10 client.26767 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:02:05.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.813+0000 7f2b9d8dbb80 10 client.26767 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T22:02:05.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.813+0000 7f2b9d8dbb80 20 client.26767 populate_metadata read hostname 'smithi151' 2024-05-21T22:02:05.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.813+0000 7f2b9d8dbb80 1 --2- 172.21.15.151:0/1757191901 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x564e60284b70 0x564e60286f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:05.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.813+0000 7f2b9d8dbb80 4 client.26767 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T22:02:05.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.813+0000 7f2b9d8dbb80 1 -- 172.21.15.151:0/1757191901 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564e601cbd90 con 0x564e60284b70 2024-05-21T22:02:05.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.813+0000 7f2b8be00700 1 --2- 172.21.15.151:0/1757191901 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x564e60284b70 0x564e60286f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:05.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.814+0000 7f2b8be00700 1 --2- 172.21.15.151:0/1757191901 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x564e60284b70 0x564e60286f60 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f6f3a4a20fc71f0 server_cookie=1fe741f80f9b9db6 in_seq=0 out_seq=0 2024-05-21T22:02:05.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.814+0000 7f2b7a000700 10 client.26767 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T22:02:05.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.815+0000 7f2b7a000700 1 -- 172.21.15.151:0/1757191901 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x564e601cbd90 con 0x564e60284b70 2024-05-21T22:02:05.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.815+0000 7f2b7a000700 10 client.26767 handle_command_reply: tid=0 2024-05-21T22:02:05.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.815+0000 7f2b9d8dbb80 1 client.26767 shutdown 2024-05-21T22:02:05.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.815+0000 7f2b9d8dbb80 1 -- 172.21.15.151:0/1757191901 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564e60271940 msgr2=0x564e60274fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:05.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.815+0000 7f2b9d8dbb80 1 --2- 172.21.15.151:0/1757191901 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564e60271940 0x564e60274fb0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x564e60278130 tx=0x7f2b84004f30 comp rx=0 tx=0).stop 2024-05-21T22:02:05.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.815+0000 7f2b9d8dbb80 1 -- 172.21.15.151:0/1757191901 shutdown_connections 2024-05-21T22:02:05.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.815+0000 7f2b9d8dbb80 1 --2- 172.21.15.151:0/1757191901 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564e60271940 0x564e60274fb0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:05.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.815+0000 7f2b9d8dbb80 1 --2- 172.21.15.151:0/1757191901 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564e6026bad0 0x564e602776d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:05.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.815+0000 7f2b9d8dbb80 1 --2- 172.21.15.151:0/1757191901 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x564e60284b70 0x564e60286f60 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:05.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.815+0000 7f2b9d8dbb80 1 --2- 172.21.15.151:0/1757191901 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564e60271020 0x564e60274a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:05.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.815+0000 7f2b7a000700 0 client.26767 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T22:02:05.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.815+0000 7f2b9d8dbb80 1 -- 172.21.15.151:0/1757191901 >> 172.21.15.151:0/1757191901 conn(0x564e60199be0 msgr2=0x564e6026a4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:02:05.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.815+0000 7f2b9d8dbb80 1 -- 172.21.15.151:0/1757191901 shutdown_connections 2024-05-21T22:02:05.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.815+0000 7f2b9d8dbb80 1 -- 172.21.15.151:0/1757191901 wait complete. 2024-05-21T22:02:05.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.815+0000 7f2b9d8dbb80 20 client.26767 trim_cache size 0 max 16384 2024-05-21T22:02:05.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.838+0000 7f2b9d8dbb80 1 Processor -- start 2024-05-21T22:02:05.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.838+0000 7f2b9d8dbb80 1 -- start start 2024-05-21T22:02:05.839 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.838+0000 7f2b9d8dbb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564e6026bad0 0x564e60274850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:05.839 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.838+0000 7f2b9d8dbb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564e60271020 0x564e60274d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:05.839 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.838+0000 7f2b9d8dbb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564e60271940 0x564e6028c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:05.839 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.838+0000 7f2b9d8dbb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2b8c1afd20 con 0x564e60271940 2024-05-21T22:02:05.839 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.838+0000 7f2b9d8dbb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2b8c1b1860 con 0x564e6026bad0 2024-05-21T22:02:05.839 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.838+0000 7f2b9d8dbb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2b8c1b03f0 con 0x564e60271020 2024-05-21T22:02:05.839 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.838+0000 7f2b91000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564e60271940 0x564e6028c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:05.839 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.838+0000 7f2b8b400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564e60271020 0x564e60274d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:05.839 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.838+0000 7f2b91000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564e60271940 0x564e6028c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:55034/0 (socket says 172.21.15.151:55034) 2024-05-21T22:02:05.840 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.838+0000 7f2b91000700 1 -- 172.21.15.151:0/529981707 learned_addr learned my addr 172.21.15.151:0/529981707 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:02:05.840 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.839+0000 7f2b8be00700 1 --2- 172.21.15.151:0/529981707 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564e6026bad0 0x564e60274850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:05.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.839+0000 7f2b91000700 1 -- 172.21.15.151:0/529981707 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564e60271020 msgr2=0x564e60274d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:05.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.839+0000 7f2b91000700 1 --2- 172.21.15.151:0/529981707 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564e60271020 0x564e60274d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:05.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.839+0000 7f2b91000700 1 -- 172.21.15.151:0/529981707 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564e6026bad0 msgr2=0x564e60274850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:05.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.839+0000 7f2b91000700 1 --2- 172.21.15.151:0/529981707 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564e6026bad0 0x564e60274850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:05.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.839+0000 7f2b91000700 1 -- 172.21.15.151:0/529981707 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e60273010 con 0x564e60271940 2024-05-21T22:02:05.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.839+0000 7f2b8b400700 1 --2- 172.21.15.151:0/529981707 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564e60271020 0x564e60274d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:02:05.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.839+0000 7f2b91000700 1 --2- 172.21.15.151:0/529981707 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564e60271940 0x564e6028c6f0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f2b840aa8b0 tx=0x7f2b840099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86730f89be382fb3 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:02:05.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.839+0000 7f2b80c00700 1 -- 172.21.15.151:0/529981707 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b840c5210 con 0x564e60271940 2024-05-21T22:02:05.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.840+0000 7f2b80c00700 1 -- 172.21.15.151:0/529981707 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b840aef60 con 0x564e60271940 2024-05-21T22:02:05.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.840+0000 7f2b80c00700 1 -- 172.21.15.151:0/529981707 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b840a9ed0 con 0x564e60271940 2024-05-21T22:02:05.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.840+0000 7f2b9d8dbb80 1 -- 172.21.15.151:0/529981707 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564e60271940 msgr2=0x564e6028c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:05.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.840+0000 7f2b9d8dbb80 1 --2- 172.21.15.151:0/529981707 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564e60271940 0x564e6028c6f0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f2b840aa8b0 tx=0x7f2b840099e0 comp rx=0 tx=0).stop 2024-05-21T22:02:05.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.840+0000 7f2b9d8dbb80 1 -- 172.21.15.151:0/529981707 shutdown_connections 2024-05-21T22:02:05.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.840+0000 7f2b9d8dbb80 1 --2- 172.21.15.151:0/529981707 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564e6026bad0 0x564e60274850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:05.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.840+0000 7f2b9d8dbb80 1 --2- 172.21.15.151:0/529981707 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564e60271940 0x564e6028c6f0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:05.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.840+0000 7f2b9d8dbb80 1 --2- 172.21.15.151:0/529981707 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564e60271020 0x564e60274d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:05.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.840+0000 7f2b9d8dbb80 1 -- 172.21.15.151:0/529981707 >> 172.21.15.151:0/529981707 conn(0x564e60199be0 msgr2=0x564e60265c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:02:05.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.840+0000 7f2b9d8dbb80 1 -- 172.21.15.151:0/529981707 shutdown_connections 2024-05-21T22:02:05.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.841+0000 7f2b9d8dbb80 1 -- 172.21.15.151:0/529981707 wait complete. 2024-05-21T22:02:05.845 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.841+0000 7f2b9d8dbb80 1 -- start start 2024-05-21T22:02:05.845 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.841+0000 7f2b9d8dbb80 1 Processor -- start 2024-05-21T22:02:05.845 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.842+0000 7f2b9d8dbb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564e6026bad0 0x564e6027d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:05.845 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.842+0000 7f2b9d8dbb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564e60271020 0x564e6027d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:05.845 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.842+0000 7f2b9d8dbb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564e60271940 0x564e6027a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:05.845 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.842+0000 7f2b9d8dbb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2b8c1b1190 con 0x564e6026bad0 2024-05-21T22:02:05.845 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.842+0000 7f2b9d8dbb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2b8c1b0ac0 con 0x564e60271020 2024-05-21T22:02:05.845 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.842+0000 7f2b9d8dbb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2b8c1adcc0 con 0x564e60271940 2024-05-21T22:02:05.845 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.842+0000 7f2b9d8dbb80 10 client.0 _ll_register_callbacks cb 0x564e6026c5a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:02:05.845 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.842+0000 7f2b8be00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564e6026bad0 0x564e6027d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:05.845 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.842+0000 7f2b8b400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564e60271020 0x564e6027d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:05.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.842+0000 7f2b91000700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564e60271940 0x564e6027a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:05.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.842+0000 7f2b8b400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564e60271020 0x564e6027d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:40018/0 (socket says 172.21.15.151:40018) 2024-05-21T22:02:05.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.842+0000 7f2b8b400700 1 -- 172.21.15.151:0/2595003634 learned_addr learned my addr 172.21.15.151:0/2595003634 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:02:05.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.842+0000 7f2b91000700 1 -- 172.21.15.151:0/2595003634 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564e60271020 msgr2=0x564e6027d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:05.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.842+0000 7f2b91000700 1 --2- 172.21.15.151:0/2595003634 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564e60271020 0x564e6027d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:05.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.842+0000 7f2b91000700 1 -- 172.21.15.151:0/2595003634 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564e6026bad0 msgr2=0x564e6027d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:05.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.842+0000 7f2b91000700 1 --2- 172.21.15.151:0/2595003634 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564e6026bad0 0x564e6027d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:05.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.842+0000 7f2b91000700 1 -- 172.21.15.151:0/2595003634 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e6027aa00 con 0x564e60271940 2024-05-21T22:02:05.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.843+0000 7f2b91000700 1 --2- 172.21.15.151:0/2595003634 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564e60271940 0x564e6027a440 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f2b840aab40 tx=0x7f2b840e1040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=941c45ad47d6501f server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:02:05.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.843+0000 7f2b7a000700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-21T22:02:05.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.843+0000 7f2b7a000700 1 -- 172.21.15.151:0/2595003634 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b840c5210 con 0x564e60271940 2024-05-21T22:02:05.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.843+0000 7f2b7a000700 1 -- 172.21.15.151:0/2595003634 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f2b840ab020 con 0x564e60271940 2024-05-21T22:02:05.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.843+0000 7f2b9d8dbb80 1 -- 172.21.15.151:0/2595003634 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b840aef60 con 0x564e60271940 2024-05-21T22:02:05.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.843+0000 7f2b7a000700 1 -- 172.21.15.151:0/2595003634 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2b840b3bd0 con 0x564e60271940 2024-05-21T22:02:05.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.843+0000 7f2b9d8dbb80 10 client.26773 fetch_fsmap learned FSMap version 29 2024-05-21T22:02:05.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.843+0000 7f2b9d8dbb80 1 -- 172.21.15.151:0/2595003634 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564e6027aa00 con 0x564e60271940 2024-05-21T22:02:05.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.844+0000 7f2b7a000700 1 -- 172.21.15.151:0/2595003634 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b840aef60 con 0x564e60271940 2024-05-21T22:02:05.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.844+0000 7f2b7a000700 1 -- 172.21.15.151:0/2595003634 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f2b84011040 con 0x564e60271940 2024-05-21T22:02:05.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.844+0000 7f2b9d8dbb80 10 client.26773 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:02:05.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.844+0000 7f2b9d8dbb80 10 client.26773 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T22:02:05.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.844+0000 7f2b9d8dbb80 20 client.26773 populate_metadata read hostname 'smithi151' 2024-05-21T22:02:05.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.844+0000 7f2b9d8dbb80 1 --2- 172.21.15.151:0/2595003634 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x564e602f0110 0x564e602f2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:05.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.844+0000 7f2b9d8dbb80 4 client.26773 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-21T22:02:05.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.844+0000 7f2b9d8dbb80 1 -- 172.21.15.151:0/2595003634 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564e601c5570 con 0x564e602f0110 2024-05-21T22:02:05.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.844+0000 7f2b8be00700 1 --2- 172.21.15.151:0/2595003634 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x564e602f0110 0x564e602f2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:05.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.845+0000 7f2b8be00700 1 --2- 172.21.15.151:0/2595003634 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x564e602f0110 0x564e602f2500 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab203ef9f78edefd server_cookie=cc74713aaf9d0c9e in_seq=0 out_seq=0 2024-05-21T22:02:05.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.845+0000 7f2b7a000700 10 client.26773 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T22:02:05.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.845+0000 7f2b7a000700 1 -- 172.21.15.151:0/2595003634 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x564e601c5570 con 0x564e602f0110 2024-05-21T22:02:05.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.845+0000 7f2b7a000700 10 client.26773 handle_command_reply: tid=0 2024-05-21T22:02:05.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.845+0000 7f2b9d8dbb80 1 client.26773 shutdown 2024-05-21T22:02:05.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.845+0000 7f2b9d8dbb80 1 -- 172.21.15.151:0/2595003634 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564e60271940 msgr2=0x564e6027a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:05.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.845+0000 7f2b9d8dbb80 1 --2- 172.21.15.151:0/2595003634 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564e60271940 0x564e6027a440 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f2b840aab40 tx=0x7f2b840e1040 comp rx=0 tx=0).stop 2024-05-21T22:02:05.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.845+0000 7f2b9d8dbb80 1 -- 172.21.15.151:0/2595003634 shutdown_connections 2024-05-21T22:02:05.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.845+0000 7f2b9d8dbb80 1 --2- 172.21.15.151:0/2595003634 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564e60271940 0x564e6027a440 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:05.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.845+0000 7f2b9d8dbb80 1 --2- 172.21.15.151:0/2595003634 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564e6026bad0 0x564e6027d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:05.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.845+0000 7f2b9d8dbb80 1 --2- 172.21.15.151:0/2595003634 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x564e602f0110 0x564e602f2500 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:05.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.845+0000 7f2b9d8dbb80 1 --2- 172.21.15.151:0/2595003634 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564e60271020 0x564e6027d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:05.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.845+0000 7f2b9d8dbb80 1 -- 172.21.15.151:0/2595003634 >> 172.21.15.151:0/2595003634 conn(0x564e60199be0 msgr2=0x564e60274850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:02:05.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.845+0000 7f2b7a000700 0 client.26773 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T22:02:05.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.845+0000 7f2b9d8dbb80 1 -- 172.21.15.151:0/2595003634 shutdown_connections 2024-05-21T22:02:05.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.845+0000 7f2b9d8dbb80 1 -- 172.21.15.151:0/2595003634 wait complete. 2024-05-21T22:02:05.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.846+0000 7f2b9d8dbb80 20 client.26773 trim_cache size 0 max 16384 2024-05-21T22:02:05.850 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-05-21T22:02:05.850 INFO:teuthology.orchestra.run.smithi151.stdout: "status": "no active scrubs running", 2024-05-21T22:02:05.850 INFO:teuthology.orchestra.run.smithi151.stdout: "scrubs": {} 2024-05-21T22:02:05.850 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-05-21T22:02:05.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.846+0000 7f2b91a00700 1 -- 172.21.15.151:0/1645091143 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f2b700b2a70 msgr2=0x7f2b700b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:05.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.846+0000 7f2b91a00700 1 --2- 172.21.15.151:0/1645091143 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f2b700b2a70 0x7f2b700b4f30 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c1107e0 tx=0x7f2b740118e0 comp rx=0 tx=0).stop 2024-05-21T22:02:05.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.846+0000 7f2b91a00700 1 -- 172.21.15.151:0/1645091143 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b8c1094f0 msgr2=0x7f2b8c1a05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:05.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.846+0000 7f2b91a00700 1 --2- 172.21.15.151:0/1645091143 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b8c1094f0 0x7f2b8c1a05f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f2b8400b070 tx=0x7f2b84004d00 comp rx=0 tx=0).stop 2024-05-21T22:02:05.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.846+0000 7f2b91a00700 1 -- 172.21.15.151:0/1645091143 shutdown_connections 2024-05-21T22:02:05.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.846+0000 7f2b91a00700 1 --2- 172.21.15.151:0/1645091143 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f2b700b2a70 0x7f2b700b4f30 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:05.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.846+0000 7f2b91a00700 1 --2- 172.21.15.151:0/1645091143 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b8c1094f0 0x7f2b8c1a05f0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:05.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.846+0000 7f2b91a00700 1 --2- 172.21.15.151:0/1645091143 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2b8c103460 0x7f2b8c19bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:05.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.846+0000 7f2b91a00700 1 --2- 172.21.15.151:0/1645091143 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2b8c103e20 0x7f2b8c19c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:05.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.846+0000 7f2b91a00700 1 -- 172.21.15.151:0/1645091143 >> 172.21.15.151:0/1645091143 conn(0x7f2b8c0fef50 msgr2=0x7f2b8c0ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:02:05.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.846+0000 7f2b91a00700 1 -- 172.21.15.151:0/1645091143 shutdown_connections 2024-05-21T22:02:05.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:05.847+0000 7f2b91a00700 1 -- 172.21.15.151:0/1645091143 wait complete. 2024-05-21T22:02:06.693 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-21T22:02:06.693 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-21T22:02:07.125 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:02:07.609 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:02:07 smithi151 ceph-mon[31623]: pgmap v198: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-05-21T22:02:07.735 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:02:07 smithi184 ceph-mon[36942]: pgmap v198: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-05-21T22:02:07.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:02:07 smithi164 ceph-mon[36269]: pgmap v198: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-05-21T22:02:09.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:09.345+0000 7f98a4000700 1 -- 172.21.15.151:0/1572331835 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f989c1041a0 msgr2=0x7f989c1081c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:09.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:09.345+0000 7f98a4000700 1 --2- 172.21.15.151:0/1572331835 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f989c1041a0 0x7f989c1081c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f98880099c0 tx=0x7f9888009cd0 comp rx=0 tx=0).stop 2024-05-21T22:02:09.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:09.346+0000 7f98a4000700 1 -- 172.21.15.151:0/1572331835 shutdown_connections 2024-05-21T22:02:09.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:09.346+0000 7f98a4000700 1 --2- 172.21.15.151:0/1572331835 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f989c1088f0 0x7f989c110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:09.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:09.346+0000 7f98a4000700 1 --2- 172.21.15.151:0/1572331835 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f989c1037f0 0x7f989c103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:09.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:09.346+0000 7f98a4000700 1 --2- 172.21.15.151:0/1572331835 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f989c1041a0 0x7f989c1081c0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:09.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:09.346+0000 7f98a4000700 1 -- 172.21.15.151:0/1572331835 >> 172.21.15.151:0/1572331835 conn(0x7f989c07a6a0 msgr2=0x7f989c07aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:02:09.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:09.346+0000 7f98a4000700 1 -- 172.21.15.151:0/1572331835 shutdown_connections 2024-05-21T22:02:09.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:09.346+0000 7f98a4000700 1 -- 172.21.15.151:0/1572331835 wait complete. 2024-05-21T22:02:09.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:09.347+0000 7f98a4000700 1 Processor -- start 2024-05-21T22:02:09.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:09.347+0000 7f98a4000700 1 -- start start 2024-05-21T22:02:09.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:09.348+0000 7f98a4000700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f989c1037f0 0x7f989c19bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:09.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:09.348+0000 7f98a4000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f989c1041a0 0x7f989c19c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:09.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:09.348+0000 7f98a4000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f989c1088f0 0x7f989c1a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:09.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:09.348+0000 7f98a4000700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f989c113720 con 0x7f989c1088f0 2024-05-21T22:02:09.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:09.348+0000 7f98a4000700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f989c1135a0 con 0x7f989c1041a0 2024-05-21T22:02:09.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:09.348+0000 7f98a4000700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f989c1138a0 con 0x7f989c1037f0 2024-05-21T22:02:09.350 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:09.348+0000 7f98a2000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f989c1088f0 0x7f989c1a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:09.350 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:09.348+0000 7f98a1600700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f989c1037f0 0x7f989c19bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:09.350 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:09.348+0000 7f98a2000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f989c1088f0 0x7f989c1a06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:55042/0 (socket says 172.21.15.151:55042) 2024-05-21T22:02:09.350 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:09.348+0000 7f98a0c00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f989c1041a0 0x7f989c19c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:09.350 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:09.348+0000 7f98a2000700 1 -- 172.21.15.151:0/582982064 learned_addr learned my addr 172.21.15.151:0/582982064 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:02:09.350 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:09.348+0000 7f98a1600700 1 -- 172.21.15.151:0/582982064 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f989c1041a0 msgr2=0x7f989c19c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:09.350 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:09.348+0000 7f98a1600700 1 --2- 172.21.15.151:0/582982064 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f989c1041a0 0x7f989c19c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:09.350 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:09.348+0000 7f98a1600700 1 -- 172.21.15.151:0/582982064 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f989c1088f0 msgr2=0x7f989c1a06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:09.350 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:09.348+0000 7f98a1600700 1 --2- 172.21.15.151:0/582982064 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f989c1088f0 0x7f989c1a06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:09.350 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:09.348+0000 7f98a1600700 1 -- 172.21.15.151:0/582982064 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f989c1a0de0 con 0x7f989c1037f0 2024-05-21T22:02:09.350 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:09.349+0000 7f98a1600700 1 --2- 172.21.15.151:0/582982064 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f989c1037f0 0x7f989c19bdd0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f989000e9f0 tx=0x7f989000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:02:09.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:09.349+0000 7f989a000700 1 -- 172.21.15.151:0/582982064 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9890004220 con 0x7f989c1037f0 2024-05-21T22:02:09.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:09.349+0000 7f989a000700 1 -- 172.21.15.151:0/582982064 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98900043c0 con 0x7f989c1037f0 2024-05-21T22:02:09.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:09.349+0000 7f989a000700 1 -- 172.21.15.151:0/582982064 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9890005020 con 0x7f989c1037f0 2024-05-21T22:02:09.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:09.349+0000 7f98a4000700 1 -- 172.21.15.151:0/582982064 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f989c1a10d0 con 0x7f989c1037f0 2024-05-21T22:02:09.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:09.349+0000 7f98a4000700 1 -- 172.21.15.151:0/582982064 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f989c1a8950 con 0x7f989c1037f0 2024-05-21T22:02:09.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:09.350+0000 7f98a4000700 1 -- 172.21.15.151:0/582982064 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f989c06a230 con 0x7f989c1037f0 2024-05-21T22:02:09.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:09.353+0000 7f989a000700 1 -- 172.21.15.151:0/582982064 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7f9890004750 con 0x7f989c1037f0 2024-05-21T22:02:09.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:09.354+0000 7f989a000700 1 --2- 172.21.15.151:0/582982064 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f98840b29f0 0x7f98840b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:09.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:09.355+0000 7f989a000700 1 -- 172.21.15.151:0/582982064 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f98900e2680 con 0x7f989c1037f0 2024-05-21T22:02:09.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:09.355+0000 7f98a0c00700 1 --2- 172.21.15.151:0/582982064 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f98840b29f0 0x7f98840b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:09.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:09.355+0000 7f98a0c00700 1 --2- 172.21.15.151:0/582982064 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f98840b29f0 0x7f98840b4eb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f989c103520 tx=0x7f9888005fb0 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:02:09.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:09.356+0000 7f989a000700 1 -- 172.21.15.151:0/582982064 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f98900aa0a0 con 0x7f989c1037f0 2024-05-21T22:02:09.649 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:09.649+0000 7f98a4000700 1 -- 172.21.15.151:0/582982064 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f989c19d130 con 0x7f989c1037f0 2024-05-21T22:02:09.653 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:09.653+0000 7f989a000700 1 -- 172.21.15.151:0/582982064 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11958 (secure 0 0 0) 0x7f98900a99a0 con 0x7f989c1037f0 2024-05-21T22:02:09.658 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T22:02:09.658 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14358,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/158010529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":158010529},{"type":"v1","addr":"172.21.15.151:6835","nonce":158010529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6839/2069058115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":2069058115},{"type":"v1","addr":"172.21.15.151:6839","nonce":2069058115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T21:51:19.552069+0000","modified":"2024-05-21T21:55:34.643755+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24530,"mds_1":24493,"mds_2":24491,"mds_3":24478,"mds_4":24473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14403":{"gid":14403,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6841/1749723794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6840","nonce":1749723794},{"type":"v1","addr":"172.21.15.151:6841","nonce":1749723794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/2026266114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":2026266114},{"type":"v1","addr":"172.21.15.184:6833","nonce":2026266114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/2467900844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":2467900844},{"type":"v1","addr":"172.21.15.164:6833","nonce":2467900844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3367640960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3367640960},{"type":"v1","addr":"172.21.15.151:6837","nonce":3367640960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/3264142621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3264142621},{"type":"v1","addr":"172.21.15.184:6835","nonce":3264142621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":17,"addr":"172.21.15.164:6835/2340428470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2340428470},{"type":"v1","addr":"172.21.15.164:6835","nonce":2340428470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":16,"addr":"172.21.15.184:6837/2988194888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":2988194888},{"type":"v1","addr":"172.21.15.184:6837","nonce":2988194888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.164:6837/1554609572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1554609572},{"type":"v1","addr":"172.21.15.164:6837","nonce":1554609572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.184:6839/3059935143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":3059935143},{"type":"v1","addr":"172.21.15.184:6839","nonce":3059935143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.164:6839/2646401247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":2646401247},{"type":"v1","addr":"172.21.15.164:6839","nonce":2646401247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24530,"qdb_cluster":[24530,24493,24491,24473,24478]},"id":1}]} 2024-05-21T22:02:09.659 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:09.658+0000 7f98a4000700 1 -- 172.21.15.151:0/582982064 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f98840b29f0 msgr2=0x7f98840b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:09.659 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:09.658+0000 7f98a4000700 1 --2- 172.21.15.151:0/582982064 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f98840b29f0 0x7f98840b4eb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f989c103520 tx=0x7f9888005fb0 comp rx=0 tx=0).stop 2024-05-21T22:02:09.659 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:09.659+0000 7f98a4000700 1 -- 172.21.15.151:0/582982064 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f989c1037f0 msgr2=0x7f989c19bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:09.660 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:09.659+0000 7f98a4000700 1 --2- 172.21.15.151:0/582982064 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f989c1037f0 0x7f989c19bdd0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f989000e9f0 tx=0x7f989000ed00 comp rx=0 tx=0).stop 2024-05-21T22:02:09.660 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:09.659+0000 7f98a4000700 1 -- 172.21.15.151:0/582982064 shutdown_connections 2024-05-21T22:02:09.660 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:09.659+0000 7f98a4000700 1 --2- 172.21.15.151:0/582982064 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f98840b29f0 0x7f98840b4eb0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:09.660 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:09.659+0000 7f98a4000700 1 --2- 172.21.15.151:0/582982064 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f989c1088f0 0x7f989c1a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:09.660 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:09.659+0000 7f98a4000700 1 --2- 172.21.15.151:0/582982064 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f989c1037f0 0x7f989c19bdd0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:09.660 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:09.659+0000 7f98a4000700 1 --2- 172.21.15.151:0/582982064 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f989c1041a0 0x7f989c19c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:09.660 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:09.659+0000 7f98a4000700 1 -- 172.21.15.151:0/582982064 >> 172.21.15.151:0/582982064 conn(0x7f989c07a6a0 msgr2=0x7f989c1019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:02:09.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:09.661+0000 7f98a4000700 1 -- 172.21.15.151:0/582982064 shutdown_connections 2024-05-21T22:02:09.662 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:09.661+0000 7f98a4000700 1 -- 172.21.15.151:0/582982064 wait complete. 2024-05-21T22:02:09.665 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 29 2024-05-21T22:02:09.735 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:02:09 smithi184 ceph-mon[36942]: pgmap v199: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-05-21T22:02:09.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:02:09 smithi164 ceph-mon[36269]: pgmap v199: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-05-21T22:02:09.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:02:09 smithi151 ceph-mon[31623]: pgmap v199: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-05-21T22:02:10.564 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-21T22:02:10.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:02:10 smithi164 ceph-mon[36269]: osdmap e84: 12 total, 12 up, 12 in 2024-05-21T22:02:10.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:02:10 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/582982064' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T22:02:10.952 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:02:10 smithi151 ceph-mon[31623]: osdmap e84: 12 total, 12 up, 12 in 2024-05-21T22:02:10.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:02:10 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/582982064' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T22:02:10.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:02:10 smithi184 ceph-mon[36942]: osdmap e84: 12 total, 12 up, 12 in 2024-05-21T22:02:10.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:02:10 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/582982064' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T22:02:11.025 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:02:11.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:02:11 smithi151 ceph-mon[31623]: pgmap v201: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-05-21T22:02:11.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:02:11 smithi164 ceph-mon[36269]: pgmap v201: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-05-21T22:02:11.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:02:11 smithi184 ceph-mon[36942]: pgmap v201: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-05-21T22:02:13.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.214+0000 7fdf5a800700 1 -- 172.21.15.151:0/415509132 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdf5406bab0 msgr2=0x7fdf5406bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:13.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.214+0000 7fdf5a800700 1 --2- 172.21.15.151:0/415509132 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdf5406bab0 0x7fdf5406bf30 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fdf400099c0 tx=0x7fdf40009cd0 comp rx=0 tx=0).stop 2024-05-21T22:02:13.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.214+0000 7fdf5a800700 1 -- 172.21.15.151:0/415509132 shutdown_connections 2024-05-21T22:02:13.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.214+0000 7fdf5a800700 1 --2- 172.21.15.151:0/415509132 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf54109580 0x7fdf54109960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:13.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.214+0000 7fdf5a800700 1 --2- 172.21.15.151:0/415509132 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdf5406c470 0x7fdf54110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:13.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.214+0000 7fdf5a800700 1 --2- 172.21.15.151:0/415509132 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdf5406bab0 0x7fdf5406bf30 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:13.216 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.214+0000 7fdf5a800700 1 -- 172.21.15.151:0/415509132 >> 172.21.15.151:0/415509132 conn(0x7fdf54079280 msgr2=0x7fdf54079690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:02:13.216 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.215+0000 7fdf5a800700 1 -- 172.21.15.151:0/415509132 shutdown_connections 2024-05-21T22:02:13.216 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.215+0000 7fdf5a800700 1 -- 172.21.15.151:0/415509132 wait complete. 2024-05-21T22:02:13.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.216+0000 7fdf5a800700 1 Processor -- start 2024-05-21T22:02:13.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.216+0000 7fdf5a800700 1 -- start start 2024-05-21T22:02:13.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.216+0000 7fdf5a800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdf5406bab0 0x7fdf541a1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:13.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.216+0000 7fdf5a800700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdf5406c470 0x7fdf541a1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:13.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.216+0000 7fdf5a800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf54109580 0x7fdf5419baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:13.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.216+0000 7fdf5a800700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdf54113680 con 0x7fdf5406bab0 2024-05-21T22:02:13.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.216+0000 7fdf5a800700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdf54113500 con 0x7fdf54109580 2024-05-21T22:02:13.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.216+0000 7fdf5a800700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fdf54113800 con 0x7fdf5406c470 2024-05-21T22:02:13.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.217+0000 7fdf59400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdf5406bab0 0x7fdf541a1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:13.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.217+0000 7fdf59400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdf5406bab0 0x7fdf541a1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:55056/0 (socket says 172.21.15.151:55056) 2024-05-21T22:02:13.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.217+0000 7fdf59400700 1 -- 172.21.15.151:0/56278383 learned_addr learned my addr 172.21.15.151:0/56278383 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:02:13.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.217+0000 7fdf58a00700 1 --2- 172.21.15.151:0/56278383 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdf5406c470 0x7fdf541a1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:13.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.217+0000 7fdf59e00700 1 --2- 172.21.15.151:0/56278383 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf54109580 0x7fdf5419baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:13.219 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.217+0000 7fdf59400700 1 -- 172.21.15.151:0/56278383 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdf5406c470 msgr2=0x7fdf541a1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:13.219 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.217+0000 7fdf59400700 1 --2- 172.21.15.151:0/56278383 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdf5406c470 0x7fdf541a1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:13.219 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.217+0000 7fdf59400700 1 -- 172.21.15.151:0/56278383 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf54109580 msgr2=0x7fdf5419baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:13.219 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.217+0000 7fdf59400700 1 --2- 172.21.15.151:0/56278383 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf54109580 0x7fdf5419baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:13.219 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.217+0000 7fdf59400700 1 -- 172.21.15.151:0/56278383 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf5419c250 con 0x7fdf5406bab0 2024-05-21T22:02:13.219 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.217+0000 7fdf59400700 1 --2- 172.21.15.151:0/56278383 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdf5406bab0 0x7fdf541a1930 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fdf4800d950 tx=0x7fdf4800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:02:13.219 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.218+0000 7fdf52000700 1 -- 172.21.15.151:0/56278383 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf48016070 con 0x7fdf5406bab0 2024-05-21T22:02:13.219 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.218+0000 7fdf5a800700 1 -- 172.21.15.151:0/56278383 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf5419c540 con 0x7fdf5406bab0 2024-05-21T22:02:13.219 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.218+0000 7fdf52000700 1 -- 172.21.15.151:0/56278383 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf4800fd40 con 0x7fdf5406bab0 2024-05-21T22:02:13.219 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.218+0000 7fdf52000700 1 -- 172.21.15.151:0/56278383 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf4800e3b0 con 0x7fdf5406bab0 2024-05-21T22:02:13.219 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.218+0000 7fdf5a800700 1 -- 172.21.15.151:0/56278383 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf541a8800 con 0x7fdf5406bab0 2024-05-21T22:02:13.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.221+0000 7fdf52000700 1 -- 172.21.15.151:0/56278383 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7fdf48019070 con 0x7fdf5406bab0 2024-05-21T22:02:13.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.223+0000 7fdf6677db80 1 Processor -- start 2024-05-21T22:02:13.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.223+0000 7fdf6677db80 1 -- start start 2024-05-21T22:02:13.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.224+0000 7fdf6677db80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b9a2c7e7c0 0x55b9a2c7eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:13.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.224+0000 7fdf6677db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b9a2c7f0e0 0x55b9a2c78d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:13.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.224+0000 7fdf6677db80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b9a2c79390 0x55b9a2c797f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:13.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.224+0000 7fdf6677db80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55b9a2bd6e40 con 0x55b9a2c7e7c0 2024-05-21T22:02:13.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.224+0000 7fdf6677db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55b9a2bc69e0 con 0x55b9a2c7f0e0 2024-05-21T22:02:13.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.224+0000 7fdf6677db80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55b9a2bcaa20 con 0x55b9a2c79390 2024-05-21T22:02:13.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.224+0000 7fdf52000700 1 --2- 172.21.15.151:0/56278383 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fdf3c0b2a70 0x7fdf3c0b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:13.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.224+0000 7fdf52000700 1 -- 172.21.15.151:0/56278383 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8586+0+0 (secure 0 0 0) 0x7fdf480e2130 con 0x7fdf5406bab0 2024-05-21T22:02:13.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.224+0000 7fdf59e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b9a2c7e7c0 0x55b9a2c7eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:13.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.224+0000 7fdf59400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b9a2c7f0e0 0x55b9a2c78d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:13.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.224+0000 7fdf59e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b9a2c7e7c0 0x55b9a2c7eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:55068/0 (socket says 172.21.15.151:55068) 2024-05-21T22:02:13.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.224+0000 7fdf59e00700 1 -- 172.21.15.151:0/1783126574 learned_addr learned my addr 172.21.15.151:0/1783126574 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:02:13.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.224+0000 7fdf58a00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b9a2c79390 0x55b9a2c797f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:13.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.225+0000 7fdf59e00700 1 --2- 172.21.15.151:0/56278383 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fdf3c0b2a70 0x7fdf3c0b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:13.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.225+0000 7fdf58a00700 1 -- 172.21.15.151:0/1783126574 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b9a2c7f0e0 msgr2=0x55b9a2c78d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:13.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.225+0000 7fdf58a00700 1 --2- 172.21.15.151:0/1783126574 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b9a2c7f0e0 0x55b9a2c78d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:13.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.225+0000 7fdf58a00700 1 -- 172.21.15.151:0/1783126574 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b9a2c7e7c0 msgr2=0x55b9a2c7eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:13.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.225+0000 7fdf58a00700 1 --2- 172.21.15.151:0/1783126574 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b9a2c7e7c0 0x55b9a2c7eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:13.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.225+0000 7fdf58a00700 1 -- 172.21.15.151:0/1783126574 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9a2b8f420 con 0x55b9a2c79390 2024-05-21T22:02:13.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.225+0000 7fdf58a00700 1 --2- 172.21.15.151:0/1783126574 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b9a2c79390 0x55b9a2c797f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fdf40009990 tx=0x7fdf40005bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4118fa8d196fdd21 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:02:13.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.226+0000 7fdf59e00700 1 --2- 172.21.15.151:0/56278383 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fdf3c0b2a70 0x7fdf3c0b4f30 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c008d70 tx=0x7fdf4c00b040 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:02:13.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.226+0000 7fdf46a00700 1 -- 172.21.15.151:0/1783126574 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf40028070 con 0x55b9a2c79390 2024-05-21T22:02:13.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.226+0000 7fdf46a00700 1 -- 172.21.15.151:0/1783126574 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf400037c0 con 0x55b9a2c79390 2024-05-21T22:02:13.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.226+0000 7fdf46a00700 1 -- 172.21.15.151:0/1783126574 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf4000b990 con 0x55b9a2c79390 2024-05-21T22:02:13.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.226+0000 7fdf6677db80 1 -- 172.21.15.151:0/1783126574 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b9a2c79390 msgr2=0x55b9a2c797f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:13.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.226+0000 7fdf6677db80 1 --2- 172.21.15.151:0/1783126574 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b9a2c79390 0x55b9a2c797f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fdf40009990 tx=0x7fdf40005bc0 comp rx=0 tx=0).stop 2024-05-21T22:02:13.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.227+0000 7fdf6677db80 1 -- 172.21.15.151:0/1783126574 shutdown_connections 2024-05-21T22:02:13.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.227+0000 7fdf6677db80 1 --2- 172.21.15.151:0/1783126574 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b9a2c79390 0x55b9a2c797f0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:13.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.227+0000 7fdf6677db80 1 --2- 172.21.15.151:0/1783126574 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b9a2c7e7c0 0x55b9a2c7eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:13.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.227+0000 7fdf6677db80 1 --2- 172.21.15.151:0/1783126574 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b9a2c7f0e0 0x55b9a2c78d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:13.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.227+0000 7fdf6677db80 1 -- 172.21.15.151:0/1783126574 >> 172.21.15.151:0/1783126574 conn(0x55b9a2ba6be0 msgr2=0x55b9a2c73810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:02:13.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.227+0000 7fdf6677db80 1 -- 172.21.15.151:0/1783126574 shutdown_connections 2024-05-21T22:02:13.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.227+0000 7fdf6677db80 1 -- 172.21.15.151:0/1783126574 wait complete. 2024-05-21T22:02:13.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.227+0000 7fdf6677db80 1 -- start start 2024-05-21T22:02:13.229 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.228+0000 7fdf6677db80 1 Processor -- start 2024-05-21T22:02:13.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.228+0000 7fdf6677db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b9a2c79390 0x55b9a2c84ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:13.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.228+0000 7fdf6677db80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b9a2c7e7c0 0x55b9a2c82240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:13.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.228+0000 7fdf6677db80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b9a2c7f0e0 0x55b9a2c82780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:13.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.228+0000 7fdf6677db80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55b9a2accfa0 con 0x55b9a2c7e7c0 2024-05-21T22:02:13.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.228+0000 7fdf6677db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55b9a2b67760 con 0x55b9a2c79390 2024-05-21T22:02:13.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.228+0000 7fdf6677db80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55b9a2c85ba0 con 0x55b9a2c7f0e0 2024-05-21T22:02:13.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.228+0000 7fdf6677db80 10 client.0 _ll_register_callbacks cb 0x55b9a2b70c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:02:13.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.228+0000 7fdf58a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b9a2c7e7c0 0x55b9a2c82240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:13.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.228+0000 7fdf58a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b9a2c7e7c0 0x55b9a2c82240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:55074/0 (socket says 172.21.15.151:55074) 2024-05-21T22:02:13.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.229+0000 7fdf58a00700 1 -- 172.21.15.151:0/2566199576 learned_addr learned my addr 172.21.15.151:0/2566199576 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:02:13.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.229+0000 7fdf59400700 1 --2- 172.21.15.151:0/2566199576 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b9a2c79390 0x55b9a2c84ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:13.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.229+0000 7fdf59e00700 1 --2- 172.21.15.151:0/2566199576 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b9a2c7f0e0 0x55b9a2c82780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:13.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.229+0000 7fdf59400700 1 -- 172.21.15.151:0/2566199576 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b9a2c7f0e0 msgr2=0x55b9a2c82780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:13.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.229+0000 7fdf59400700 1 --2- 172.21.15.151:0/2566199576 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b9a2c7f0e0 0x55b9a2c82780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:13.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.229+0000 7fdf59400700 1 -- 172.21.15.151:0/2566199576 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b9a2c7e7c0 msgr2=0x55b9a2c82240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:13.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.229+0000 7fdf59400700 1 --2- 172.21.15.151:0/2566199576 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b9a2c7e7c0 0x55b9a2c82240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:13.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.229+0000 7fdf59400700 1 -- 172.21.15.151:0/2566199576 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9a2b168d0 con 0x55b9a2c79390 2024-05-21T22:02:13.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.229+0000 7fdf59400700 1 --2- 172.21.15.151:0/2566199576 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b9a2c79390 0x55b9a2c84ea0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fdf480df7e0 tx=0x7fdf480a8fe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc89a81c90d49008 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:02:13.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.229+0000 7fdf3b400700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-21T22:02:13.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.229+0000 7fdf3b400700 1 -- 172.21.15.151:0/2566199576 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf48020070 con 0x55b9a2c79390 2024-05-21T22:02:13.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.229+0000 7fdf3b400700 1 -- 172.21.15.151:0/2566199576 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8586+0+0 (secure 0 0 0) 0x7fdf480a97e0 con 0x55b9a2c79390 2024-05-21T22:02:13.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.229+0000 7fdf6677db80 1 -- 172.21.15.151:0/2566199576 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf400037c0 con 0x55b9a2c79390 2024-05-21T22:02:13.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.230+0000 7fdf3b400700 1 -- 172.21.15.151:0/2566199576 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdf480ad720 con 0x55b9a2c79390 2024-05-21T22:02:13.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.230+0000 7fdf6677db80 10 client.26780 fetch_fsmap learned FSMap version 29 2024-05-21T22:02:13.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.230+0000 7fdf6677db80 1 -- 172.21.15.151:0/2566199576 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b9a2b97060 con 0x55b9a2c79390 2024-05-21T22:02:13.233 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.230+0000 7fdf3b400700 1 -- 172.21.15.151:0/2566199576 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf400037c0 con 0x55b9a2c79390 2024-05-21T22:02:13.233 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.230+0000 7fdf3b400700 1 -- 172.21.15.151:0/2566199576 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fdf480a85d0 con 0x55b9a2c79390 2024-05-21T22:02:13.233 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.230+0000 7fdf6677db80 10 client.26780 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:02:13.233 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.230+0000 7fdf6677db80 10 client.26780 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-05-21T22:02:13.233 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.230+0000 7fdf6677db80 20 client.26780 populate_metadata read hostname 'smithi151' 2024-05-21T22:02:13.233 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.230+0000 7fdf6677db80 1 --2- 172.21.15.151:0/2566199576 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x55b9a2c92700 0x55b9a2c94b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:13.233 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.230+0000 7fdf6677db80 4 client.26780 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T22:02:13.233 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.230+0000 7fdf6677db80 1 -- 172.21.15.151:0/2566199576 --> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b9a2bd8d90 con 0x55b9a2c92700 2024-05-21T22:02:13.233 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.231+0000 7fdf58a00700 1 --2- 172.21.15.151:0/2566199576 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x55b9a2c92700 0x55b9a2c94b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:13.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.231+0000 7fdf58a00700 1 --2- 172.21.15.151:0/2566199576 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x55b9a2c92700 0x55b9a2c94b80 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=fb3e4ff182f0078 server_cookie=22066e69205c4b8d in_seq=0 out_seq=0 2024-05-21T22:02:13.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.231+0000 7fdf3b400700 10 client.26780 ms_handle_connect on v2:172.21.15.164:6834/2340428470 2024-05-21T22:02:13.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.232+0000 7fdf3b400700 1 -- 172.21.15.151:0/2566199576 <== mds.4 v2:172.21.15.164:6834/2340428470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55b9a2bd8d90 con 0x55b9a2c92700 2024-05-21T22:02:13.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.232+0000 7fdf3b400700 10 client.26780 handle_command_reply: tid=0 2024-05-21T22:02:13.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.232+0000 7fdf6677db80 1 client.26780 shutdown 2024-05-21T22:02:13.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.232+0000 7fdf6677db80 1 -- 172.21.15.151:0/2566199576 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b9a2c79390 msgr2=0x55b9a2c84ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:13.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.232+0000 7fdf6677db80 1 --2- 172.21.15.151:0/2566199576 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b9a2c79390 0x55b9a2c84ea0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fdf480df7e0 tx=0x7fdf480a8fe0 comp rx=0 tx=0).stop 2024-05-21T22:02:13.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.232+0000 7fdf6677db80 1 -- 172.21.15.151:0/2566199576 shutdown_connections 2024-05-21T22:02:13.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.232+0000 7fdf6677db80 1 --2- 172.21.15.151:0/2566199576 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x55b9a2c92700 0x55b9a2c94b80 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:13.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.232+0000 7fdf6677db80 1 --2- 172.21.15.151:0/2566199576 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b9a2c79390 0x55b9a2c84ea0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:13.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.232+0000 7fdf6677db80 1 --2- 172.21.15.151:0/2566199576 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b9a2c7f0e0 0x55b9a2c82780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:13.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.232+0000 7fdf6677db80 1 --2- 172.21.15.151:0/2566199576 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b9a2c7e7c0 0x55b9a2c82240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:13.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.232+0000 7fdf3b400700 0 client.26780 ms_handle_reset on v2:172.21.15.164:6834/2340428470 2024-05-21T22:02:13.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.232+0000 7fdf6677db80 1 -- 172.21.15.151:0/2566199576 >> 172.21.15.151:0/2566199576 conn(0x55b9a2ba6be0 msgr2=0x55b9a2c72fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:02:13.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.233+0000 7fdf6677db80 1 -- 172.21.15.151:0/2566199576 shutdown_connections 2024-05-21T22:02:13.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.233+0000 7fdf6677db80 1 -- 172.21.15.151:0/2566199576 wait complete. 2024-05-21T22:02:13.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.233+0000 7fdf6677db80 20 client.26780 trim_cache size 0 max 16384 2024-05-21T22:02:13.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.249+0000 7fdf6677db80 1 Processor -- start 2024-05-21T22:02:13.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.249+0000 7fdf6677db80 1 -- start start 2024-05-21T22:02:13.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.250+0000 7fdf6677db80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b9a2c79390 0x55b9a2c78d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:13.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.250+0000 7fdf6677db80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b9a2c7e7c0 0x55b9a2c7fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:13.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.250+0000 7fdf6677db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b9a2c7f0e0 0x55b9a2c82690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:13.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.250+0000 7fdf6677db80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdf541afed0 con 0x55b9a2c79390 2024-05-21T22:02:13.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.250+0000 7fdf6677db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdf541b1a10 con 0x55b9a2c7f0e0 2024-05-21T22:02:13.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.250+0000 7fdf6677db80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fdf541b05a0 con 0x55b9a2c7e7c0 2024-05-21T22:02:13.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.250+0000 7fdf59400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b9a2c79390 0x55b9a2c78d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:13.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.250+0000 7fdf59400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b9a2c79390 0x55b9a2c78d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:55078/0 (socket says 172.21.15.151:55078) 2024-05-21T22:02:13.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.250+0000 7fdf59400700 1 -- 172.21.15.151:0/2375455845 learned_addr learned my addr 172.21.15.151:0/2375455845 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:02:13.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.250+0000 7fdf58a00700 1 --2- 172.21.15.151:0/2375455845 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b9a2c7e7c0 0x55b9a2c7fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:13.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.250+0000 7fdf59e00700 1 --2- 172.21.15.151:0/2375455845 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b9a2c7f0e0 0x55b9a2c82690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:13.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.251+0000 7fdf59400700 1 -- 172.21.15.151:0/2375455845 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b9a2c7e7c0 msgr2=0x55b9a2c7fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:13.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.251+0000 7fdf59400700 1 --2- 172.21.15.151:0/2375455845 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b9a2c7e7c0 0x55b9a2c7fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:13.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.251+0000 7fdf59400700 1 -- 172.21.15.151:0/2375455845 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b9a2c7f0e0 msgr2=0x55b9a2c82690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:13.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.251+0000 7fdf59400700 1 --2- 172.21.15.151:0/2375455845 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b9a2c7f0e0 0x55b9a2c82690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:13.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.251+0000 7fdf59400700 1 -- 172.21.15.151:0/2375455845 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9a2c95200 con 0x55b9a2c79390 2024-05-21T22:02:13.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.251+0000 7fdf59400700 1 --2- 172.21.15.151:0/2375455845 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b9a2c79390 0x55b9a2c78d70 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fdf4800b2f0 tx=0x7fdf480a5360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=394ff9f17e059265 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:02:13.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.251+0000 7fdf46000700 1 -- 172.21.15.151:0/2375455845 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf48020070 con 0x55b9a2c79390 2024-05-21T22:02:13.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.251+0000 7fdf46000700 1 -- 172.21.15.151:0/2375455845 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf480a5960 con 0x55b9a2c79390 2024-05-21T22:02:13.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.251+0000 7fdf46000700 1 -- 172.21.15.151:0/2375455845 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf480b1fa0 con 0x55b9a2c79390 2024-05-21T22:02:13.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.252+0000 7fdf6677db80 1 -- 172.21.15.151:0/2375455845 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b9a2c79390 msgr2=0x55b9a2c78d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:13.253 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.252+0000 7fdf6677db80 1 --2- 172.21.15.151:0/2375455845 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b9a2c79390 0x55b9a2c78d70 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fdf4800b2f0 tx=0x7fdf480a5360 comp rx=0 tx=0).stop 2024-05-21T22:02:13.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.252+0000 7fdf6677db80 1 -- 172.21.15.151:0/2375455845 shutdown_connections 2024-05-21T22:02:13.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.252+0000 7fdf6677db80 1 --2- 172.21.15.151:0/2375455845 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b9a2c7f0e0 0x55b9a2c82690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:13.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.252+0000 7fdf6677db80 1 --2- 172.21.15.151:0/2375455845 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b9a2c79390 0x55b9a2c78d70 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:13.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.252+0000 7fdf6677db80 1 --2- 172.21.15.151:0/2375455845 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b9a2c7e7c0 0x55b9a2c7fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:13.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.252+0000 7fdf6677db80 1 -- 172.21.15.151:0/2375455845 >> 172.21.15.151:0/2375455845 conn(0x55b9a2ba6be0 msgr2=0x55b9a2c80a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:02:13.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.252+0000 7fdf6677db80 1 -- 172.21.15.151:0/2375455845 shutdown_connections 2024-05-21T22:02:13.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.252+0000 7fdf6677db80 1 -- 172.21.15.151:0/2375455845 wait complete. 2024-05-21T22:02:13.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.253+0000 7fdf6677db80 1 -- start start 2024-05-21T22:02:13.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.253+0000 7fdf6677db80 1 Processor -- start 2024-05-21T22:02:13.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.253+0000 7fdf6677db80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b9a2c79390 0x55b9a2c8b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:13.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.253+0000 7fdf6677db80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b9a2c7e7c0 0x55b9a2c99c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:13.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.253+0000 7fdf6677db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b9a2c7f0e0 0x55b9a2c9a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:13.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.254+0000 7fdf6677db80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdf541b1340 con 0x55b9a2c79390 2024-05-21T22:02:13.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.254+0000 7fdf6677db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdf541b0c70 con 0x55b9a2c7f0e0 2024-05-21T22:02:13.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.254+0000 7fdf6677db80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fdf541ade90 con 0x55b9a2c7e7c0 2024-05-21T22:02:13.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.254+0000 7fdf6677db80 10 client.0 _ll_register_callbacks cb 0x55b9a2b70c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:02:13.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.254+0000 7fdf59400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b9a2c79390 0x55b9a2c8b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:13.257 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.254+0000 7fdf58a00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b9a2c7e7c0 0x55b9a2c99c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:13.257 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.254+0000 7fdf59400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b9a2c79390 0x55b9a2c8b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:55094/0 (socket says 172.21.15.151:55094) 2024-05-21T22:02:13.257 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.254+0000 7fdf59400700 1 -- 172.21.15.151:0/3019081522 learned_addr learned my addr 172.21.15.151:0/3019081522 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:02:13.257 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.254+0000 7fdf59e00700 1 --2- 172.21.15.151:0/3019081522 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b9a2c7f0e0 0x55b9a2c9a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:13.257 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.254+0000 7fdf58a00700 1 -- 172.21.15.151:0/3019081522 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b9a2c7f0e0 msgr2=0x55b9a2c9a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:13.257 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.254+0000 7fdf58a00700 1 --2- 172.21.15.151:0/3019081522 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b9a2c7f0e0 0x55b9a2c9a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:13.257 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.254+0000 7fdf58a00700 1 -- 172.21.15.151:0/3019081522 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b9a2c79390 msgr2=0x55b9a2c8b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:13.257 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.254+0000 7fdf58a00700 1 --2- 172.21.15.151:0/3019081522 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b9a2c79390 0x55b9a2c8b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:13.257 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.254+0000 7fdf58a00700 1 -- 172.21.15.151:0/3019081522 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9a2c9a8b0 con 0x55b9a2c7e7c0 2024-05-21T22:02:13.257 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.254+0000 7fdf58a00700 1 --2- 172.21.15.151:0/3019081522 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b9a2c7e7c0 0x55b9a2c99c30 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fdf4000b580 tx=0x7fdf4000b630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1510a8619d24f65 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:02:13.257 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.254+0000 7fdf3b400700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-21T22:02:13.257 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.255+0000 7fdf3b400700 1 -- 172.21.15.151:0/3019081522 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf40034070 con 0x55b9a2c7e7c0 2024-05-21T22:02:13.257 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.255+0000 7fdf6677db80 1 -- 172.21.15.151:0/3019081522 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf480a5960 con 0x55b9a2c7e7c0 2024-05-21T22:02:13.257 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.255+0000 7fdf3b400700 1 -- 172.21.15.151:0/3019081522 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8586+0+0 (secure 0 0 0) 0x7fdf400052c0 con 0x55b9a2c7e7c0 2024-05-21T22:02:13.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.255+0000 7fdf3b400700 1 -- 172.21.15.151:0/3019081522 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdf4003f050 con 0x55b9a2c7e7c0 2024-05-21T22:02:13.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.255+0000 7fdf6677db80 10 client.26797 fetch_fsmap learned FSMap version 29 2024-05-21T22:02:13.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.255+0000 7fdf6677db80 1 -- 172.21.15.151:0/3019081522 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b9a2cfd2c0 con 0x55b9a2c7e7c0 2024-05-21T22:02:13.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.255+0000 7fdf3b400700 1 -- 172.21.15.151:0/3019081522 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf480a5960 con 0x55b9a2c7e7c0 2024-05-21T22:02:13.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.256+0000 7fdf3b400700 1 -- 172.21.15.151:0/3019081522 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fdf4000f460 con 0x55b9a2c7e7c0 2024-05-21T22:02:13.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.256+0000 7fdf6677db80 10 client.26797 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:02:13.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.256+0000 7fdf6677db80 10 client.26797 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-05-21T22:02:13.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.256+0000 7fdf6677db80 20 client.26797 populate_metadata read hostname 'smithi151' 2024-05-21T22:02:13.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.256+0000 7fdf6677db80 1 --2- 172.21.15.151:0/3019081522 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x55b9a2cfd550 0x55b9a2cff940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:13.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.256+0000 7fdf6677db80 4 client.26797 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T22:02:13.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.256+0000 7fdf6677db80 1 -- 172.21.15.151:0/3019081522 --> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b9a2bd2570 con 0x55b9a2cfd550 2024-05-21T22:02:13.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.256+0000 7fdf59400700 1 --2- 172.21.15.151:0/3019081522 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x55b9a2cfd550 0x55b9a2cff940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:13.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.257+0000 7fdf59400700 1 --2- 172.21.15.151:0/3019081522 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x55b9a2cfd550 0x55b9a2cff940 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8f8f56d1d8deed81 server_cookie=f11e75c649166858 in_seq=0 out_seq=0 2024-05-21T22:02:13.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.257+0000 7fdf3b400700 10 client.26797 ms_handle_connect on v2:172.21.15.164:6834/2340428470 2024-05-21T22:02:13.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.257+0000 7fdf3b400700 1 -- 172.21.15.151:0/3019081522 <== mds.4 v2:172.21.15.164:6834/2340428470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b9a2bd2570 con 0x55b9a2cfd550 2024-05-21T22:02:13.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.257+0000 7fdf3b400700 10 client.26797 handle_command_reply: tid=0 2024-05-21T22:02:13.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.257+0000 7fdf6677db80 1 client.26797 shutdown 2024-05-21T22:02:13.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.257+0000 7fdf6677db80 1 -- 172.21.15.151:0/3019081522 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b9a2c7e7c0 msgr2=0x55b9a2c99c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:13.260 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.257+0000 7fdf6677db80 1 --2- 172.21.15.151:0/3019081522 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b9a2c7e7c0 0x55b9a2c99c30 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fdf4000b580 tx=0x7fdf4000b630 comp rx=0 tx=0).stop 2024-05-21T22:02:13.260 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.257+0000 7fdf6677db80 1 -- 172.21.15.151:0/3019081522 shutdown_connections 2024-05-21T22:02:13.260 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.257+0000 7fdf6677db80 1 --2- 172.21.15.151:0/3019081522 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x55b9a2cfd550 0x55b9a2cff940 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:13.260 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.257+0000 7fdf6677db80 1 --2- 172.21.15.151:0/3019081522 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b9a2c7f0e0 0x55b9a2c9a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:13.260 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.257+0000 7fdf6677db80 1 --2- 172.21.15.151:0/3019081522 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b9a2c79390 0x55b9a2c8b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:13.260 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.257+0000 7fdf6677db80 1 --2- 172.21.15.151:0/3019081522 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b9a2c7e7c0 0x55b9a2c99c30 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:13.260 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.257+0000 7fdf6677db80 1 -- 172.21.15.151:0/3019081522 >> 172.21.15.151:0/3019081522 conn(0x55b9a2ba6be0 msgr2=0x55b9a2c7d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:02:13.260 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.257+0000 7fdf3b400700 0 client.26797 ms_handle_reset on v2:172.21.15.164:6834/2340428470 2024-05-21T22:02:13.260 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.257+0000 7fdf6677db80 1 -- 172.21.15.151:0/3019081522 shutdown_connections 2024-05-21T22:02:13.260 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.257+0000 7fdf6677db80 1 -- 172.21.15.151:0/3019081522 wait complete. 2024-05-21T22:02:13.260 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.257+0000 7fdf6677db80 20 client.26797 trim_cache size 0 max 16384 2024-05-21T22:02:13.261 INFO:teuthology.orchestra.run.smithi151.stdout:[] 2024-05-21T22:02:13.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.258+0000 7fdf5a800700 1 -- 172.21.15.151:0/56278383 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fdf3c0b2a70 msgr2=0x7fdf3c0b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:13.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.258+0000 7fdf5a800700 1 --2- 172.21.15.151:0/56278383 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fdf3c0b2a70 0x7fdf3c0b4f30 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c008d70 tx=0x7fdf4c00b040 comp rx=0 tx=0).stop 2024-05-21T22:02:13.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.258+0000 7fdf5a800700 1 -- 172.21.15.151:0/56278383 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdf5406bab0 msgr2=0x7fdf541a1930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:13.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.258+0000 7fdf5a800700 1 --2- 172.21.15.151:0/56278383 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdf5406bab0 0x7fdf541a1930 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fdf4800d950 tx=0x7fdf4800dc60 comp rx=0 tx=0).stop 2024-05-21T22:02:13.262 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.258+0000 7fdf5a800700 1 -- 172.21.15.151:0/56278383 shutdown_connections 2024-05-21T22:02:13.262 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.258+0000 7fdf5a800700 1 --2- 172.21.15.151:0/56278383 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fdf3c0b2a70 0x7fdf3c0b4f30 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:13.262 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.258+0000 7fdf5a800700 1 --2- 172.21.15.151:0/56278383 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf54109580 0x7fdf5419baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:13.262 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.258+0000 7fdf5a800700 1 --2- 172.21.15.151:0/56278383 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdf5406bab0 0x7fdf541a1930 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:13.262 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.258+0000 7fdf5a800700 1 --2- 172.21.15.151:0/56278383 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdf5406c470 0x7fdf541a1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:13.262 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.258+0000 7fdf5a800700 1 -- 172.21.15.151:0/56278383 >> 172.21.15.151:0/56278383 conn(0x7fdf54079280 msgr2=0x7fdf5410e580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:02:13.262 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.258+0000 7fdf5a800700 1 -- 172.21.15.151:0/56278383 shutdown_connections 2024-05-21T22:02:13.262 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:13.258+0000 7fdf5a800700 1 -- 172.21.15.151:0/56278383 wait complete. 2024-05-21T22:02:13.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:02:13 smithi164 ceph-mon[36269]: pgmap v202: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-21T22:02:13.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:02:13 smithi151 ceph-mon[31623]: pgmap v202: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-21T22:02:13.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:02:13 smithi184 ceph-mon[36942]: pgmap v202: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-21T22:02:14.176 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-21T22:02:14.617 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:02:15.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:02:15 smithi151 ceph-mon[31623]: osdmap e85: 12 total, 12 up, 12 in 2024-05-21T22:02:15.454 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:02:15 smithi151 ceph-mon[31623]: pgmap v204: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-05-21T22:02:15.485 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:02:15 smithi184 ceph-mon[36942]: osdmap e85: 12 total, 12 up, 12 in 2024-05-21T22:02:15.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:02:15 smithi184 ceph-mon[36942]: pgmap v204: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-05-21T22:02:15.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:02:15 smithi164 ceph-mon[36269]: osdmap e85: 12 total, 12 up, 12 in 2024-05-21T22:02:15.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:02:15 smithi164 ceph-mon[36269]: pgmap v204: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-05-21T22:02:16.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.895+0000 7f4bb5c00700 1 -- 172.21.15.151:0/2127294491 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4bb0104f50 msgr2=0x7f4bb010d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:16.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.895+0000 7f4bb5c00700 1 --2- 172.21.15.151:0/2127294491 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4bb0104f50 0x7f4bb010d440 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f4b9c009990 tx=0x7f4b9c009ca0 comp rx=0 tx=0).stop 2024-05-21T22:02:16.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.895+0000 7f4bb5c00700 1 -- 172.21.15.151:0/2127294491 shutdown_connections 2024-05-21T22:02:16.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.896+0000 7f4bb5c00700 1 --2- 172.21.15.151:0/2127294491 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4bb0104630 0x7f4bb0104a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:16.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.896+0000 7f4bb5c00700 1 --2- 172.21.15.151:0/2127294491 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4bb010da60 0x7f4bb010ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:16.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.896+0000 7f4bb5c00700 1 --2- 172.21.15.151:0/2127294491 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4bb0104f50 0x7f4bb010d440 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:16.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.896+0000 7f4bb5c00700 1 -- 172.21.15.151:0/2127294491 >> 172.21.15.151:0/2127294491 conn(0x7f4bb00fde60 msgr2=0x7f4bb0100280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:02:16.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.896+0000 7f4bb5c00700 1 -- 172.21.15.151:0/2127294491 shutdown_connections 2024-05-21T22:02:16.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.896+0000 7f4bb5c00700 1 -- 172.21.15.151:0/2127294491 wait complete. 2024-05-21T22:02:16.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.897+0000 7f4bb5c00700 1 Processor -- start 2024-05-21T22:02:16.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.897+0000 7f4bb5c00700 1 -- start start 2024-05-21T22:02:16.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.897+0000 7f4bb5c00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4bb0104630 0x7f4bb019bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:16.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.897+0000 7f4bb5c00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4bb0104f50 0x7f4bb019c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:16.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.897+0000 7f4bb5c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4bb010da60 0x7f4bb01a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:16.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.897+0000 7f4bb5c00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb0113680 con 0x7f4bb010da60 2024-05-21T22:02:16.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.897+0000 7f4bb5c00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb0113500 con 0x7f4bb0104630 2024-05-21T22:02:16.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.897+0000 7f4bb5c00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb0113800 con 0x7f4bb0104f50 2024-05-21T22:02:16.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.898+0000 7f4bb5200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4bb010da60 0x7f4bb01a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:16.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.898+0000 7f4bafe00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4bb0104630 0x7f4bb019bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:16.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.898+0000 7f4bb5200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4bb010da60 0x7f4bb01a06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52426/0 (socket says 172.21.15.151:52426) 2024-05-21T22:02:16.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.898+0000 7f4bb5200700 1 -- 172.21.15.151:0/4169756937 learned_addr learned my addr 172.21.15.151:0/4169756937 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:02:16.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.898+0000 7f4baf400700 1 --2- 172.21.15.151:0/4169756937 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4bb0104f50 0x7f4bb019c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:16.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.898+0000 7f4bafe00700 1 -- 172.21.15.151:0/4169756937 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4bb0104f50 msgr2=0x7f4bb019c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:16.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.898+0000 7f4bafe00700 1 --2- 172.21.15.151:0/4169756937 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4bb0104f50 0x7f4bb019c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:16.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.898+0000 7f4bafe00700 1 -- 172.21.15.151:0/4169756937 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4bb010da60 msgr2=0x7f4bb01a06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:16.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.898+0000 7f4bafe00700 1 --2- 172.21.15.151:0/4169756937 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4bb010da60 0x7f4bb01a06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:16.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.898+0000 7f4bafe00700 1 -- 172.21.15.151:0/4169756937 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bb01a0de0 con 0x7f4bb0104630 2024-05-21T22:02:16.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.898+0000 7f4bb5200700 1 --2- 172.21.15.151:0/4169756937 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4bb010da60 0x7f4bb01a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:02:16.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.899+0000 7f4bafe00700 1 --2- 172.21.15.151:0/4169756937 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4bb0104630 0x7f4bb019bdd0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f4ba000e9f0 tx=0x7f4ba000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:02:16.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.899+0000 7f4bacc00700 1 -- 172.21.15.151:0/4169756937 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ba000c9c0 con 0x7f4bb0104630 2024-05-21T22:02:16.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.899+0000 7f4bb5c00700 1 -- 172.21.15.151:0/4169756937 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bb01a10d0 con 0x7f4bb0104630 2024-05-21T22:02:16.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.899+0000 7f4bb5c00700 1 -- 172.21.15.151:0/4169756937 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bb01a8950 con 0x7f4bb0104630 2024-05-21T22:02:16.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.899+0000 7f4bacc00700 1 -- 172.21.15.151:0/4169756937 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ba000cb60 con 0x7f4bb0104630 2024-05-21T22:02:16.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.899+0000 7f4bacc00700 1 -- 172.21.15.151:0/4169756937 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ba0018680 con 0x7f4bb0104630 2024-05-21T22:02:16.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.903+0000 7f4bacc00700 1 -- 172.21.15.151:0/4169756937 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7f4ba0018820 con 0x7f4bb0104630 2024-05-21T22:02:16.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.904+0000 7f4bc1bd6b80 1 Processor -- start 2024-05-21T22:02:16.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.904+0000 7f4bc1bd6b80 1 -- start start 2024-05-21T22:02:16.907 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.904+0000 7f4bacc00700 1 --2- 172.21.15.151:0/4169756937 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f4b940b2a70 0x7f4b940b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:16.907 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.904+0000 7f4bc1bd6b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eb13ae17c0 0x55eb13ae1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:16.907 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.904+0000 7f4bc1bd6b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eb13ae20e0 0x55eb13adbd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:16.907 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.904+0000 7f4bc1bd6b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eb13adc390 0x55eb13adc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:16.907 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.904+0000 7f4bc1bd6b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55eb13a39e40 con 0x55eb13ae20e0 2024-05-21T22:02:16.907 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.904+0000 7f4bc1bd6b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55eb13a299e0 con 0x55eb13ae17c0 2024-05-21T22:02:16.907 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.904+0000 7f4bc1bd6b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55eb13a2da20 con 0x55eb13adc390 2024-05-21T22:02:16.907 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.904+0000 7f4bacc00700 1 -- 172.21.15.151:0/4169756937 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8542+0+0 (secure 0 0 0) 0x7f4ba00e2760 con 0x7f4bb0104630 2024-05-21T22:02:16.907 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.905+0000 7f4baf400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eb13ae20e0 0x55eb13adbd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:16.907 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.905+0000 7f4bb5200700 1 --2- 172.21.15.151:0/4169756937 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f4b940b2a70 0x7f4b940b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:16.907 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.905+0000 7f4bafe00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eb13ae17c0 0x55eb13ae1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:16.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.905+0000 7f4baf400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eb13ae20e0 0x55eb13adbd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52438/0 (socket says 172.21.15.151:52438) 2024-05-21T22:02:16.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.905+0000 7f4baf400700 1 -- 172.21.15.151:0/1748474141 learned_addr learned my addr 172.21.15.151:0/1748474141 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:02:16.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.905+0000 7f4bb5200700 1 --2- 172.21.15.151:0/1748474141 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eb13adc390 0x55eb13adc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:16.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.905+0000 7f4bafe00700 1 -- 172.21.15.151:0/1748474141 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eb13adc390 msgr2=0x55eb13adc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:16.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.905+0000 7f4bafe00700 1 --2- 172.21.15.151:0/1748474141 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eb13adc390 0x55eb13adc7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:16.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.905+0000 7f4bafe00700 1 -- 172.21.15.151:0/1748474141 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eb13ae20e0 msgr2=0x55eb13adbd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:16.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.905+0000 7f4bafe00700 1 --2- 172.21.15.151:0/1748474141 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eb13ae20e0 0x55eb13adbd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:16.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.905+0000 7f4bafe00700 1 -- 172.21.15.151:0/1748474141 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb139f2420 con 0x55eb13ae17c0 2024-05-21T22:02:16.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.905+0000 7f4bb5200700 1 --2- 172.21.15.151:0/1748474141 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eb13adc390 0x55eb13adc7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T22:02:16.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.905+0000 7f4baf400700 1 --2- 172.21.15.151:0/1748474141 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eb13ae20e0 0x55eb13adbd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:02:16.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.905+0000 7f4bafe00700 1 --2- 172.21.15.151:0/1748474141 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eb13ae17c0 0x55eb13ae1ba0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f4ba00dfe40 tx=0x7f4ba00aa150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb7197ec4f96bd53 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:02:16.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.906+0000 7f4bb5200700 1 --2- 172.21.15.151:0/4169756937 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f4b940b2a70 0x7f4b940b4f30 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4ba8009fd0 tx=0x7f4ba8009920 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:02:16.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.906+0000 7f4ba5600700 1 -- 172.21.15.151:0/1748474141 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ba00a94e0 con 0x55eb13ae17c0 2024-05-21T22:02:16.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.906+0000 7f4ba5600700 1 -- 172.21.15.151:0/1748474141 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ba00a99d0 con 0x55eb13ae17c0 2024-05-21T22:02:16.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.906+0000 7f4ba5600700 1 -- 172.21.15.151:0/1748474141 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ba00ba1a0 con 0x55eb13ae17c0 2024-05-21T22:02:16.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.906+0000 7f4bc1bd6b80 1 -- 172.21.15.151:0/1748474141 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eb13ae17c0 msgr2=0x55eb13ae1ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:16.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.907+0000 7f4bc1bd6b80 1 --2- 172.21.15.151:0/1748474141 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eb13ae17c0 0x55eb13ae1ba0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f4ba00dfe40 tx=0x7f4ba00aa150 comp rx=0 tx=0).stop 2024-05-21T22:02:16.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.907+0000 7f4bc1bd6b80 1 -- 172.21.15.151:0/1748474141 shutdown_connections 2024-05-21T22:02:16.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.907+0000 7f4bc1bd6b80 1 --2- 172.21.15.151:0/1748474141 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eb13ae17c0 0x55eb13ae1ba0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:16.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.907+0000 7f4bc1bd6b80 1 --2- 172.21.15.151:0/1748474141 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eb13adc390 0x55eb13adc7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:16.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.907+0000 7f4bc1bd6b80 1 --2- 172.21.15.151:0/1748474141 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eb13ae20e0 0x55eb13adbd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:16.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.907+0000 7f4bc1bd6b80 1 -- 172.21.15.151:0/1748474141 >> 172.21.15.151:0/1748474141 conn(0x55eb13a09be0 msgr2=0x55eb13ad6810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:02:16.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.907+0000 7f4bc1bd6b80 1 -- 172.21.15.151:0/1748474141 shutdown_connections 2024-05-21T22:02:16.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.907+0000 7f4bc1bd6b80 1 -- 172.21.15.151:0/1748474141 wait complete. 2024-05-21T22:02:16.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.907+0000 7f4bc1bd6b80 1 -- start start 2024-05-21T22:02:16.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.908+0000 7f4bc1bd6b80 1 Processor -- start 2024-05-21T22:02:16.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.908+0000 7f4bc1bd6b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eb13adc390 0x55eb13ae7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:16.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.908+0000 7f4bc1bd6b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eb13ae17c0 0x55eb13ae51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:16.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.908+0000 7f4bc1bd6b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eb13ae20e0 0x55eb13ae56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:16.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.908+0000 7f4bc1bd6b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55eb1392ffa0 con 0x55eb13ae20e0 2024-05-21T22:02:16.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.908+0000 7f4bc1bd6b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55eb139ca760 con 0x55eb13adc390 2024-05-21T22:02:16.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.908+0000 7f4bc1bd6b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55eb13ae8c00 con 0x55eb13ae17c0 2024-05-21T22:02:16.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.908+0000 7f4bc1bd6b80 10 client.0 _ll_register_callbacks cb 0x55eb139d3c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:02:16.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.908+0000 7f4bafe00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eb13adc390 0x55eb13ae7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:16.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.909+0000 7f4bafe00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eb13adc390 0x55eb13ae7dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:33242/0 (socket says 172.21.15.151:33242) 2024-05-21T22:02:16.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.909+0000 7f4bafe00700 1 -- 172.21.15.151:0/175050945 learned_addr learned my addr 172.21.15.151:0/175050945 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:02:16.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.909+0000 7f4baf400700 1 --2- 172.21.15.151:0/175050945 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eb13ae17c0 0x55eb13ae51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:16.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.909+0000 7f4bb5200700 1 --2- 172.21.15.151:0/175050945 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eb13ae20e0 0x55eb13ae56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:16.913 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.909+0000 7f4bafe00700 1 -- 172.21.15.151:0/175050945 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eb13ae17c0 msgr2=0x55eb13ae51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:16.913 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.909+0000 7f4bafe00700 1 --2- 172.21.15.151:0/175050945 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eb13ae17c0 0x55eb13ae51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:16.913 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.909+0000 7f4bafe00700 1 -- 172.21.15.151:0/175050945 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eb13ae20e0 msgr2=0x55eb13ae56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:16.913 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.909+0000 7f4bafe00700 1 --2- 172.21.15.151:0/175050945 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eb13ae20e0 0x55eb13ae56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:16.913 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.909+0000 7f4bafe00700 1 -- 172.21.15.151:0/175050945 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb139798d0 con 0x55eb13adc390 2024-05-21T22:02:16.913 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.909+0000 7f4bb5200700 1 --2- 172.21.15.151:0/175050945 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eb13ae20e0 0x55eb13ae56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T22:02:16.913 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.909+0000 7f4bafe00700 1 --2- 172.21.15.151:0/175050945 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eb13adc390 0x55eb13ae7dc0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f4ba00a6c40 tx=0x7f4ba00a59c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d6e6f0e24c1fc8a server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:02:16.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.909+0000 7f4b9a000700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-21T22:02:16.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.910+0000 7f4b9a000700 1 -- 172.21.15.151:0/175050945 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ba00a94e0 con 0x55eb13adc390 2024-05-21T22:02:16.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.910+0000 7f4b9a000700 1 -- 172.21.15.151:0/175050945 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8542+0+0 (secure 0 0 0) 0x7f4ba00a9c30 con 0x55eb13adc390 2024-05-21T22:02:16.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.910+0000 7f4bc1bd6b80 1 -- 172.21.15.151:0/175050945 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ba00a99d0 con 0x55eb13adc390 2024-05-21T22:02:16.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.910+0000 7f4b9a000700 1 -- 172.21.15.151:0/175050945 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4ba00ba3b0 con 0x55eb13adc390 2024-05-21T22:02:16.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.910+0000 7f4bc1bd6b80 10 client.26798 fetch_fsmap learned FSMap version 29 2024-05-21T22:02:16.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.910+0000 7f4bc1bd6b80 1 -- 172.21.15.151:0/175050945 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55eb139fa060 con 0x55eb13adc390 2024-05-21T22:02:16.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.910+0000 7f4b9a000700 1 -- 172.21.15.151:0/175050945 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ba00a99d0 con 0x55eb13adc390 2024-05-21T22:02:16.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.910+0000 7f4b9a000700 1 -- 172.21.15.151:0/175050945 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f4ba00a5340 con 0x55eb13adc390 2024-05-21T22:02:16.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.911+0000 7f4bc1bd6b80 10 client.26798 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:02:16.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.911+0000 7f4bc1bd6b80 10 client.26798 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-21T22:02:16.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.911+0000 7f4bc1bd6b80 20 client.26798 populate_metadata read hostname 'smithi151' 2024-05-21T22:02:16.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.911+0000 7f4bc1bd6b80 1 --2- 172.21.15.151:0/175050945 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x55eb13af55b0 0x55eb13af79a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:16.915 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.911+0000 7f4bc1bd6b80 4 client.26798 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T22:02:16.915 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.911+0000 7f4bc1bd6b80 1 -- 172.21.15.151:0/175050945 --> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eb13a3bd90 con 0x55eb13af55b0 2024-05-21T22:02:16.915 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.911+0000 7f4baf400700 1 --2- 172.21.15.151:0/175050945 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x55eb13af55b0 0x55eb13af79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:16.915 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.912+0000 7f4baf400700 1 --2- 172.21.15.151:0/175050945 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x55eb13af55b0 0x55eb13af79a0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9db67111729b10e0 server_cookie=86d03549037617ee in_seq=0 out_seq=0 2024-05-21T22:02:16.915 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.912+0000 7f4b9a000700 10 client.26798 ms_handle_connect on v2:172.21.15.184:6836/2988194888 2024-05-21T22:02:16.916 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.912+0000 7f4b9a000700 1 -- 172.21.15.151:0/175050945 <== mds.3 v2:172.21.15.184:6836/2988194888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55eb13a3bd90 con 0x55eb13af55b0 2024-05-21T22:02:16.916 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.912+0000 7f4b9a000700 10 client.26798 handle_command_reply: tid=0 2024-05-21T22:02:16.916 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.913+0000 7f4bc1bd6b80 1 client.26798 shutdown 2024-05-21T22:02:16.916 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.913+0000 7f4bc1bd6b80 1 -- 172.21.15.151:0/175050945 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eb13adc390 msgr2=0x55eb13ae7dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:16.916 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.913+0000 7f4bc1bd6b80 1 --2- 172.21.15.151:0/175050945 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eb13adc390 0x55eb13ae7dc0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f4ba00a6c40 tx=0x7f4ba00a59c0 comp rx=0 tx=0).stop 2024-05-21T22:02:16.916 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.913+0000 7f4bc1bd6b80 1 -- 172.21.15.151:0/175050945 shutdown_connections 2024-05-21T22:02:16.916 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.913+0000 7f4bc1bd6b80 1 --2- 172.21.15.151:0/175050945 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x55eb13af55b0 0x55eb13af79a0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:16.916 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.913+0000 7f4bc1bd6b80 1 --2- 172.21.15.151:0/175050945 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eb13adc390 0x55eb13ae7dc0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:16.916 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.913+0000 7f4bc1bd6b80 1 --2- 172.21.15.151:0/175050945 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eb13ae20e0 0x55eb13ae56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:16.916 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.913+0000 7f4bc1bd6b80 1 --2- 172.21.15.151:0/175050945 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eb13ae17c0 0x55eb13ae51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:16.917 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.913+0000 7f4b9a000700 0 client.26798 ms_handle_reset on v2:172.21.15.184:6836/2988194888 2024-05-21T22:02:16.917 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.913+0000 7f4bc1bd6b80 1 -- 172.21.15.151:0/175050945 >> 172.21.15.151:0/175050945 conn(0x55eb13a09be0 msgr2=0x55eb13ad5fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:02:16.917 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.913+0000 7f4bc1bd6b80 1 -- 172.21.15.151:0/175050945 shutdown_connections 2024-05-21T22:02:16.917 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.913+0000 7f4bc1bd6b80 1 -- 172.21.15.151:0/175050945 wait complete. 2024-05-21T22:02:16.917 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.913+0000 7f4bc1bd6b80 20 client.26798 trim_cache size 0 max 16384 2024-05-21T22:02:16.932 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.930+0000 7f4bc1bd6b80 1 Processor -- start 2024-05-21T22:02:16.932 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.930+0000 7f4bc1bd6b80 1 -- start start 2024-05-21T22:02:16.932 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.930+0000 7f4bc1bd6b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eb13a09be0 0x55eb13adbd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:16.932 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.930+0000 7f4bc1bd6b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eb13adc390 0x55eb13ae2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:16.932 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.930+0000 7f4bc1bd6b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eb13ae17c0 0x55eb13ad6740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:16.932 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.930+0000 7f4bc1bd6b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4b9000d610 con 0x55eb13a09be0 2024-05-21T22:02:16.932 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.930+0000 7f4bc1bd6b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4b9000f150 con 0x55eb13adc390 2024-05-21T22:02:16.932 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.930+0000 7f4bc1bd6b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4b9000dce0 con 0x55eb13ae17c0 2024-05-21T22:02:16.932 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.931+0000 7f4bafe00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eb13a09be0 0x55eb13adbd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:16.932 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.931+0000 7f4baf400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eb13adc390 0x55eb13ae2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:16.932 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.931+0000 7f4bafe00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eb13a09be0 0x55eb13adbd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52446/0 (socket says 172.21.15.151:52446) 2024-05-21T22:02:16.932 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.931+0000 7f4bafe00700 1 -- 172.21.15.151:0/351410162 learned_addr learned my addr 172.21.15.151:0/351410162 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:02:16.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.931+0000 7f4bb5200700 1 --2- 172.21.15.151:0/351410162 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eb13ae17c0 0x55eb13ad6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:16.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.931+0000 7f4baf400700 1 -- 172.21.15.151:0/351410162 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eb13ae17c0 msgr2=0x55eb13ad6740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:16.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.931+0000 7f4baf400700 1 --2- 172.21.15.151:0/351410162 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eb13ae17c0 0x55eb13ad6740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:16.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.931+0000 7f4baf400700 1 -- 172.21.15.151:0/351410162 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eb13a09be0 msgr2=0x55eb13adbd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:16.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.931+0000 7f4baf400700 1 --2- 172.21.15.151:0/351410162 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eb13a09be0 0x55eb13adbd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:16.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.931+0000 7f4baf400700 1 -- 172.21.15.151:0/351410162 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb13ae3720 con 0x55eb13adc390 2024-05-21T22:02:16.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.931+0000 7f4bafe00700 1 --2- 172.21.15.151:0/351410162 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eb13a09be0 0x55eb13adbd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:02:16.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.931+0000 7f4baf400700 1 --2- 172.21.15.151:0/351410162 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eb13adc390 0x55eb13ae2bd0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f4b9c00b2e0 tx=0x7f4b9c005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51820b7e9b7a6845 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:02:16.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.932+0000 7f4ba4c00700 1 -- 172.21.15.151:0/351410162 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b9c00b920 con 0x55eb13adc390 2024-05-21T22:02:16.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.932+0000 7f4ba4c00700 1 -- 172.21.15.151:0/351410162 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b9c003680 con 0x55eb13adc390 2024-05-21T22:02:16.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.932+0000 7f4ba4c00700 1 -- 172.21.15.151:0/351410162 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b9c00f6c0 con 0x55eb13adc390 2024-05-21T22:02:16.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.932+0000 7f4bc1bd6b80 1 -- 172.21.15.151:0/351410162 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eb13adc390 msgr2=0x55eb13ae2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:16.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.932+0000 7f4bc1bd6b80 1 --2- 172.21.15.151:0/351410162 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eb13adc390 0x55eb13ae2bd0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f4b9c00b2e0 tx=0x7f4b9c005fd0 comp rx=0 tx=0).stop 2024-05-21T22:02:16.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.932+0000 7f4bc1bd6b80 1 -- 172.21.15.151:0/351410162 shutdown_connections 2024-05-21T22:02:16.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.932+0000 7f4bc1bd6b80 1 --2- 172.21.15.151:0/351410162 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eb13ae17c0 0x55eb13ad6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:16.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.932+0000 7f4bc1bd6b80 1 --2- 172.21.15.151:0/351410162 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eb13a09be0 0x55eb13adbd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:16.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.932+0000 7f4bc1bd6b80 1 --2- 172.21.15.151:0/351410162 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eb13adc390 0x55eb13ae2bd0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:16.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.932+0000 7f4bc1bd6b80 1 -- 172.21.15.151:0/351410162 >> 172.21.15.151:0/351410162 conn(0x55eb13ae6050 msgr2=0x55eb13ae24c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:02:16.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.933+0000 7f4bc1bd6b80 1 -- 172.21.15.151:0/351410162 shutdown_connections 2024-05-21T22:02:16.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.933+0000 7f4bc1bd6b80 1 -- 172.21.15.151:0/351410162 wait complete. 2024-05-21T22:02:16.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.933+0000 7f4bc1bd6b80 1 -- start start 2024-05-21T22:02:16.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.933+0000 7f4bc1bd6b80 1 Processor -- start 2024-05-21T22:02:16.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.934+0000 7f4bc1bd6b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eb13a09be0 0x55eb13afaea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:16.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.934+0000 7f4bc1bd6b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eb13adc390 0x55eb13af8240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:16.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.934+0000 7f4bc1bd6b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eb13ae17c0 0x55eb13af8780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:16.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.934+0000 7f4bc1bd6b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4b9000ea80 con 0x55eb13adc390 2024-05-21T22:02:16.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.934+0000 7f4bc1bd6b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4b9000e3b0 con 0x55eb13ae17c0 2024-05-21T22:02:16.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.934+0000 7f4bc1bd6b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4b9000b5d0 con 0x55eb13a09be0 2024-05-21T22:02:16.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.934+0000 7f4bc1bd6b80 10 client.0 _ll_register_callbacks cb 0x55eb139d3c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:02:16.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.934+0000 7f4baf400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eb13adc390 0x55eb13af8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:16.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.934+0000 7f4bafe00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eb13a09be0 0x55eb13afaea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:16.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.934+0000 7f4bb5200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eb13ae17c0 0x55eb13af8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:16.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.934+0000 7f4baf400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eb13adc390 0x55eb13af8240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52462/0 (socket says 172.21.15.151:52462) 2024-05-21T22:02:16.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.934+0000 7f4bafe00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eb13a09be0 0x55eb13afaea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:56660/0 (socket says 172.21.15.151:56660) 2024-05-21T22:02:16.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.934+0000 7f4baf400700 1 -- 172.21.15.151:0/304445262 learned_addr learned my addr 172.21.15.151:0/304445262 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:02:16.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.935+0000 7f4bb5200700 1 -- 172.21.15.151:0/304445262 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eb13a09be0 msgr2=0x55eb13afaea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:16.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.935+0000 7f4bb5200700 1 --2- 172.21.15.151:0/304445262 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eb13a09be0 0x55eb13afaea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:16.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.935+0000 7f4bb5200700 1 -- 172.21.15.151:0/304445262 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eb13adc390 msgr2=0x55eb13af8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:16.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.935+0000 7f4bb5200700 1 --2- 172.21.15.151:0/304445262 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eb13adc390 0x55eb13af8240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:16.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.935+0000 7f4bb5200700 1 -- 172.21.15.151:0/304445262 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb13af8dc0 con 0x55eb13ae17c0 2024-05-21T22:02:16.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.935+0000 7f4baf400700 1 --2- 172.21.15.151:0/304445262 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eb13adc390 0x55eb13af8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:02:16.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.935+0000 7f4bafe00700 1 --2- 172.21.15.151:0/304445262 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eb13a09be0 0x55eb13afaea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:02:16.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.935+0000 7f4bb5200700 1 --2- 172.21.15.151:0/304445262 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eb13ae17c0 0x55eb13af8780 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f4ba8009f00 tx=0x7f4ba800b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1600abeab42b4b27 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:02:16.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.935+0000 7f4b9a000700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-21T22:02:16.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.935+0000 7f4b9a000700 1 -- 172.21.15.151:0/304445262 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ba801ab80 con 0x55eb13ae17c0 2024-05-21T22:02:16.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.935+0000 7f4b9a000700 1 -- 172.21.15.151:0/304445262 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8542+0+0 (secure 0 0 0) 0x7f4ba801ad20 con 0x55eb13ae17c0 2024-05-21T22:02:16.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.935+0000 7f4bc1bd6b80 1 -- 172.21.15.151:0/304445262 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b9c003680 con 0x55eb13ae17c0 2024-05-21T22:02:16.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.936+0000 7f4b9a000700 1 -- 172.21.15.151:0/304445262 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4ba8034050 con 0x55eb13ae17c0 2024-05-21T22:02:16.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.936+0000 7f4bc1bd6b80 10 client.26804 fetch_fsmap learned FSMap version 29 2024-05-21T22:02:16.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.936+0000 7f4bc1bd6b80 1 -- 172.21.15.151:0/304445262 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55eb13b5fc20 con 0x55eb13ae17c0 2024-05-21T22:02:16.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.936+0000 7f4b9a000700 1 -- 172.21.15.151:0/304445262 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b9c003680 con 0x55eb13ae17c0 2024-05-21T22:02:16.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.936+0000 7f4b9a000700 1 -- 172.21.15.151:0/304445262 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f4ba8027480 con 0x55eb13ae17c0 2024-05-21T22:02:16.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.936+0000 7f4bc1bd6b80 10 client.26804 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:02:16.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.936+0000 7f4bc1bd6b80 10 client.26804 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-21T22:02:16.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.936+0000 7f4bc1bd6b80 20 client.26804 populate_metadata read hostname 'smithi151' 2024-05-21T22:02:16.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.936+0000 7f4bc1bd6b80 1 --2- 172.21.15.151:0/304445262 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x55eb13ae20e0 0x55eb13b623c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:16.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.936+0000 7f4bc1bd6b80 4 client.26804 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T22:02:16.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.936+0000 7f4bc1bd6b80 1 -- 172.21.15.151:0/304445262 --> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55eb13a35570 con 0x55eb13ae20e0 2024-05-21T22:02:16.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.937+0000 7f4bafe00700 1 --2- 172.21.15.151:0/304445262 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x55eb13ae20e0 0x55eb13b623c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:16.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.937+0000 7f4bafe00700 1 --2- 172.21.15.151:0/304445262 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x55eb13ae20e0 0x55eb13b623c0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a96655bbd57675a6 server_cookie=994c39a0518832af in_seq=0 out_seq=0 2024-05-21T22:02:16.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.937+0000 7f4b9a000700 10 client.26804 ms_handle_connect on v2:172.21.15.184:6836/2988194888 2024-05-21T22:02:16.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.937+0000 7f4b9a000700 1 -- 172.21.15.151:0/304445262 <== mds.3 v2:172.21.15.184:6836/2988194888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55eb13a35570 con 0x55eb13ae20e0 2024-05-21T22:02:16.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.937+0000 7f4b9a000700 10 client.26804 handle_command_reply: tid=0 2024-05-21T22:02:16.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.937+0000 7f4bc1bd6b80 1 client.26804 shutdown 2024-05-21T22:02:16.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.937+0000 7f4bc1bd6b80 1 -- 172.21.15.151:0/304445262 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eb13ae17c0 msgr2=0x55eb13af8780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:16.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.937+0000 7f4bc1bd6b80 1 --2- 172.21.15.151:0/304445262 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eb13ae17c0 0x55eb13af8780 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f4ba8009f00 tx=0x7f4ba800b040 comp rx=0 tx=0).stop 2024-05-21T22:02:16.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.938+0000 7f4bc1bd6b80 1 -- 172.21.15.151:0/304445262 shutdown_connections 2024-05-21T22:02:16.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.938+0000 7f4bc1bd6b80 1 --2- 172.21.15.151:0/304445262 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x55eb13ae20e0 0x55eb13b623c0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:16.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.938+0000 7f4bc1bd6b80 1 --2- 172.21.15.151:0/304445262 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55eb13ae17c0 0x55eb13af8780 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:16.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.938+0000 7f4bc1bd6b80 1 --2- 172.21.15.151:0/304445262 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eb13a09be0 0x55eb13afaea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:16.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.938+0000 7f4bc1bd6b80 1 --2- 172.21.15.151:0/304445262 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eb13adc390 0x55eb13af8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:16.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.938+0000 7f4b9a000700 0 client.26804 ms_handle_reset on v2:172.21.15.184:6836/2988194888 2024-05-21T22:02:16.942 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.938+0000 7f4bc1bd6b80 1 -- 172.21.15.151:0/304445262 >> 172.21.15.151:0/304445262 conn(0x55eb13ae6050 msgr2=0x55eb13ae3aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:02:16.942 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.938+0000 7f4bc1bd6b80 1 -- 172.21.15.151:0/304445262 shutdown_connections 2024-05-21T22:02:16.942 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.938+0000 7f4bc1bd6b80 1 -- 172.21.15.151:0/304445262 wait complete. 2024-05-21T22:02:16.942 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.938+0000 7f4bc1bd6b80 20 client.26804 trim_cache size 0 max 16384 2024-05-21T22:02:16.942 INFO:teuthology.orchestra.run.smithi151.stdout:[] 2024-05-21T22:02:16.943 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.939+0000 7f4bb5c00700 1 -- 172.21.15.151:0/4169756937 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f4b940b2a70 msgr2=0x7f4b940b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:16.943 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.939+0000 7f4bb5c00700 1 --2- 172.21.15.151:0/4169756937 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f4b940b2a70 0x7f4b940b4f30 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4ba8009fd0 tx=0x7f4ba8009920 comp rx=0 tx=0).stop 2024-05-21T22:02:16.943 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.939+0000 7f4bb5c00700 1 -- 172.21.15.151:0/4169756937 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4bb0104630 msgr2=0x7f4bb019bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:16.943 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.939+0000 7f4bb5c00700 1 --2- 172.21.15.151:0/4169756937 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4bb0104630 0x7f4bb019bdd0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f4ba000e9f0 tx=0x7f4ba000ed00 comp rx=0 tx=0).stop 2024-05-21T22:02:16.943 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.939+0000 7f4bb5c00700 1 -- 172.21.15.151:0/4169756937 shutdown_connections 2024-05-21T22:02:16.943 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.939+0000 7f4bb5c00700 1 --2- 172.21.15.151:0/4169756937 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f4b940b2a70 0x7f4b940b4f30 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:16.943 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.939+0000 7f4bb5c00700 1 --2- 172.21.15.151:0/4169756937 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4bb0104630 0x7f4bb019bdd0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:16.943 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.939+0000 7f4bb5c00700 1 --2- 172.21.15.151:0/4169756937 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4bb010da60 0x7f4bb01a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:16.943 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.939+0000 7f4bb5c00700 1 --2- 172.21.15.151:0/4169756937 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4bb0104f50 0x7f4bb019c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:16.943 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.939+0000 7f4bb5c00700 1 -- 172.21.15.151:0/4169756937 >> 172.21.15.151:0/4169756937 conn(0x7f4bb00fde60 msgr2=0x7f4bb0100250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:02:16.943 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.939+0000 7f4bb5c00700 1 -- 172.21.15.151:0/4169756937 shutdown_connections 2024-05-21T22:02:16.943 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:16.939+0000 7f4bb5c00700 1 -- 172.21.15.151:0/4169756937 wait complete. 2024-05-21T22:02:17.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:02:17 smithi151 ceph-mon[31623]: pgmap v205: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-05-21T22:02:17.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:02:17 smithi184 ceph-mon[36942]: pgmap v205: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-05-21T22:02:17.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:02:17 smithi164 ceph-mon[36269]: pgmap v205: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-05-21T22:02:17.894 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-21T22:02:18.324 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:02:19.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:02:19 smithi151 ceph-mon[31623]: pgmap v206: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 230 B/s wr, 11 op/s 2024-05-21T22:02:19.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:02:19 smithi184 ceph-mon[36942]: pgmap v206: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 230 B/s wr, 11 op/s 2024-05-21T22:02:19.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:02:19 smithi164 ceph-mon[36269]: pgmap v206: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 230 B/s wr, 11 op/s 2024-05-21T22:02:20.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.501+0000 7fee3e400700 1 -- 172.21.15.151:0/486974231 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fee381041a0 msgr2=0x7fee381081c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:20.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.501+0000 7fee3e400700 1 --2- 172.21.15.151:0/486974231 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fee381041a0 0x7fee381081c0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fee200099c0 tx=0x7fee20009cd0 comp rx=0 tx=0).stop 2024-05-21T22:02:20.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.501+0000 7fee3e400700 1 -- 172.21.15.151:0/486974231 shutdown_connections 2024-05-21T22:02:20.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.501+0000 7fee3e400700 1 --2- 172.21.15.151:0/486974231 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fee381037f0 0x7fee38103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:20.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.501+0000 7fee3e400700 1 --2- 172.21.15.151:0/486974231 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee381088f0 0x7fee38110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:20.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.501+0000 7fee3e400700 1 --2- 172.21.15.151:0/486974231 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fee381041a0 0x7fee381081c0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:20.504 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.501+0000 7fee3e400700 1 -- 172.21.15.151:0/486974231 >> 172.21.15.151:0/486974231 conn(0x7fee3807a6a0 msgr2=0x7fee3807aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:02:20.504 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.501+0000 7fee3e400700 1 -- 172.21.15.151:0/486974231 shutdown_connections 2024-05-21T22:02:20.504 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.501+0000 7fee3e400700 1 -- 172.21.15.151:0/486974231 wait complete. 2024-05-21T22:02:20.504 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.503+0000 7fee3e400700 1 Processor -- start 2024-05-21T22:02:20.505 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.503+0000 7fee3e400700 1 -- start start 2024-05-21T22:02:20.505 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.503+0000 7fee3e400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fee381037f0 0x7fee3819bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:20.505 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.503+0000 7fee3e400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fee381041a0 0x7fee3819c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:20.505 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.503+0000 7fee3e400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee381088f0 0x7fee381a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:20.505 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.503+0000 7fee3e400700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fee38113720 con 0x7fee381037f0 2024-05-21T22:02:20.505 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.503+0000 7fee3e400700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fee381135a0 con 0x7fee381041a0 2024-05-21T22:02:20.505 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.503+0000 7fee3e400700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fee381138a0 con 0x7fee381088f0 2024-05-21T22:02:20.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.504+0000 7fee2ea00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fee381041a0 0x7fee3819c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:20.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.504+0000 7fee37400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fee381037f0 0x7fee3819bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:20.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.504+0000 7fee2ea00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fee381041a0 0x7fee3819c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:33274/0 (socket says 172.21.15.151:33274) 2024-05-21T22:02:20.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.504+0000 7fee2ea00700 1 -- 172.21.15.151:0/4160549700 learned_addr learned my addr 172.21.15.151:0/4160549700 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:02:20.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.504+0000 7fee37e00700 1 --2- 172.21.15.151:0/4160549700 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee381088f0 0x7fee381a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:20.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.504+0000 7fee37400700 1 -- 172.21.15.151:0/4160549700 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee381088f0 msgr2=0x7fee381a0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:20.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.504+0000 7fee37400700 1 --2- 172.21.15.151:0/4160549700 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee381088f0 0x7fee381a0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:20.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.504+0000 7fee37400700 1 -- 172.21.15.151:0/4160549700 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fee381041a0 msgr2=0x7fee3819c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:20.507 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.504+0000 7fee37400700 1 --2- 172.21.15.151:0/4160549700 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fee381041a0 0x7fee3819c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:20.507 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.504+0000 7fee37400700 1 -- 172.21.15.151:0/4160549700 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee381a0dc0 con 0x7fee381037f0 2024-05-21T22:02:20.507 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.504+0000 7fee2ea00700 1 --2- 172.21.15.151:0/4160549700 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fee381041a0 0x7fee3819c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:02:20.507 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.504+0000 7fee37400700 1 --2- 172.21.15.151:0/4160549700 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fee381037f0 0x7fee3819bdb0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fee2800ef10 tx=0x7fee2800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:02:20.507 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.505+0000 7fee34c00700 1 -- 172.21.15.151:0/4160549700 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee2800c9c0 con 0x7fee381037f0 2024-05-21T22:02:20.507 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.505+0000 7fee34c00700 1 -- 172.21.15.151:0/4160549700 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee28019070 con 0x7fee381037f0 2024-05-21T22:02:20.507 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.505+0000 7fee3e400700 1 -- 172.21.15.151:0/4160549700 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee381a1050 con 0x7fee381037f0 2024-05-21T22:02:20.507 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.505+0000 7fee34c00700 1 -- 172.21.15.151:0/4160549700 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee28014480 con 0x7fee381037f0 2024-05-21T22:02:20.507 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.505+0000 7fee3e400700 1 -- 172.21.15.151:0/4160549700 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee381a8930 con 0x7fee381037f0 2024-05-21T22:02:20.509 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.508+0000 7fee34c00700 1 -- 172.21.15.151:0/4160549700 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7fee28012050 con 0x7fee381037f0 2024-05-21T22:02:20.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.510+0000 7fee48dd5b80 1 Processor -- start 2024-05-21T22:02:20.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.510+0000 7fee48dd5b80 1 -- start start 2024-05-21T22:02:20.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.510+0000 7fee34c00700 1 --2- 172.21.15.151:0/4160549700 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fee1c0b2a70 0x7fee1c0b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:20.512 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.510+0000 7fee48dd5b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f95cd4aeb0 0x55f95cd4b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:20.512 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.510+0000 7fee48dd5b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f95cd4b7d0 0x55f95cd45260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:20.512 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.510+0000 7fee34c00700 1 -- 172.21.15.151:0/4160549700 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8542+0+0 (secure 0 0 0) 0x7fee280e2580 con 0x7fee381037f0 2024-05-21T22:02:20.512 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.510+0000 7fee48dd5b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f95cd45880 0x55f95cd45ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:20.512 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.510+0000 7fee48dd5b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55f95cca3e40 con 0x55f95cd4b7d0 2024-05-21T22:02:20.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.510+0000 7fee48dd5b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55f95cc939e0 con 0x55f95cd45880 2024-05-21T22:02:20.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.510+0000 7fee48dd5b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55f95cc97a20 con 0x55f95cd4aeb0 2024-05-21T22:02:20.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.510+0000 7fee37e00700 1 --2- 172.21.15.151:0/4160549700 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fee1c0b2a70 0x7fee1c0b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:20.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.511+0000 7fee2ea00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f95cd4b7d0 0x55f95cd45260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:20.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.511+0000 7fee37400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f95cd4aeb0 0x55f95cd4b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:20.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.511+0000 7fee2ea00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f95cd4b7d0 0x55f95cd45260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52484/0 (socket says 172.21.15.151:52484) 2024-05-21T22:02:20.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.511+0000 7fee2ea00700 1 -- 172.21.15.151:0/3402915880 learned_addr learned my addr 172.21.15.151:0/3402915880 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:02:20.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.511+0000 7fee37e00700 1 --2- 172.21.15.151:0/3402915880 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f95cd45880 0x55f95cd45ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:20.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.511+0000 7fee37400700 1 -- 172.21.15.151:0/3402915880 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f95cd45880 msgr2=0x55f95cd45ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:20.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.511+0000 7fee37400700 1 --2- 172.21.15.151:0/3402915880 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f95cd45880 0x55f95cd45ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:20.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.511+0000 7fee37400700 1 -- 172.21.15.151:0/3402915880 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f95cd4b7d0 msgr2=0x55f95cd45260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:20.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.511+0000 7fee37400700 1 --2- 172.21.15.151:0/3402915880 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f95cd4b7d0 0x55f95cd45260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:20.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.511+0000 7fee37400700 1 -- 172.21.15.151:0/3402915880 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f95cc5c420 con 0x55f95cd4aeb0 2024-05-21T22:02:20.514 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.511+0000 7fee37400700 1 --2- 172.21.15.151:0/3402915880 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f95cd4aeb0 0x55f95cd4b290 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fee280dfc60 tx=0x7fee280a9690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75209b198acca242 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:02:20.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.511+0000 7fee37e00700 1 --2- 172.21.15.151:0/4160549700 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fee1c0b2a70 0x7fee1c0b4f30 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fee3000b620 tx=0x7fee300041e0 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:02:20.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.511+0000 7fee2cc00700 1 -- 172.21.15.151:0/3402915880 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee280a9ef0 con 0x55f95cd4aeb0 2024-05-21T22:02:20.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.512+0000 7fee2cc00700 1 -- 172.21.15.151:0/3402915880 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee280a97f0 con 0x55f95cd4aeb0 2024-05-21T22:02:20.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.512+0000 7fee2cc00700 1 -- 172.21.15.151:0/3402915880 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee280a9cb0 con 0x55f95cd4aeb0 2024-05-21T22:02:20.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.512+0000 7fee48dd5b80 1 -- 172.21.15.151:0/3402915880 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f95cd4aeb0 msgr2=0x55f95cd4b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:20.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.512+0000 7fee48dd5b80 1 --2- 172.21.15.151:0/3402915880 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f95cd4aeb0 0x55f95cd4b290 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fee280dfc60 tx=0x7fee280a9690 comp rx=0 tx=0).stop 2024-05-21T22:02:20.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.512+0000 7fee48dd5b80 1 -- 172.21.15.151:0/3402915880 shutdown_connections 2024-05-21T22:02:20.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.512+0000 7fee48dd5b80 1 --2- 172.21.15.151:0/3402915880 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f95cd45880 0x55f95cd45ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:20.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.512+0000 7fee48dd5b80 1 --2- 172.21.15.151:0/3402915880 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f95cd4aeb0 0x55f95cd4b290 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:20.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.512+0000 7fee48dd5b80 1 --2- 172.21.15.151:0/3402915880 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f95cd4b7d0 0x55f95cd45260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:20.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.512+0000 7fee48dd5b80 1 -- 172.21.15.151:0/3402915880 >> 172.21.15.151:0/3402915880 conn(0x55f95cc73be0 msgr2=0x55f95cd40010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:02:20.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.513+0000 7fee48dd5b80 1 -- 172.21.15.151:0/3402915880 shutdown_connections 2024-05-21T22:02:20.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.513+0000 7fee48dd5b80 1 -- 172.21.15.151:0/3402915880 wait complete. 2024-05-21T22:02:20.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.513+0000 7fee48dd5b80 1 -- start start 2024-05-21T22:02:20.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.513+0000 7fee48dd5b80 1 Processor -- start 2024-05-21T22:02:20.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.514+0000 7fee48dd5b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f95cd45880 0x55f95cd51580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:20.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.514+0000 7fee48dd5b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f95cd4aeb0 0x55f95cd4e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:20.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.514+0000 7fee48dd5b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f95cd4b7d0 0x55f95cd4ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:20.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.514+0000 7fee48dd5b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55f95cb99fa0 con 0x55f95cd4aeb0 2024-05-21T22:02:20.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.514+0000 7fee48dd5b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55f95cc34760 con 0x55f95cd45880 2024-05-21T22:02:20.517 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.514+0000 7fee48dd5b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55f95cd521e0 con 0x55f95cd4b7d0 2024-05-21T22:02:20.517 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.514+0000 7fee48dd5b80 10 client.0 _ll_register_callbacks cb 0x55f95cc3dc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:02:20.517 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.514+0000 7fee2ea00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f95cd4aeb0 0x55f95cd4e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:20.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.514+0000 7fee2ea00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f95cd4aeb0 0x55f95cd4e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52500/0 (socket says 172.21.15.151:52500) 2024-05-21T22:02:20.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.514+0000 7fee37e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f95cd4b7d0 0x55f95cd4ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:20.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.514+0000 7fee2ea00700 1 -- 172.21.15.151:0/3432401469 learned_addr learned my addr 172.21.15.151:0/3432401469 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:02:20.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.514+0000 7fee37400700 1 --2- 172.21.15.151:0/3432401469 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f95cd45880 0x55f95cd51580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:20.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.514+0000 7fee37400700 1 -- 172.21.15.151:0/3432401469 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f95cd4b7d0 msgr2=0x55f95cd4ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:20.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.514+0000 7fee37400700 1 --2- 172.21.15.151:0/3432401469 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f95cd4b7d0 0x55f95cd4ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:20.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.514+0000 7fee37400700 1 -- 172.21.15.151:0/3432401469 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f95cd4aeb0 msgr2=0x55f95cd4e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:20.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.514+0000 7fee37400700 1 --2- 172.21.15.151:0/3432401469 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f95cd4aeb0 0x55f95cd4e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:20.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.515+0000 7fee37400700 1 -- 172.21.15.151:0/3432401469 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f95cbe38d0 con 0x55f95cd45880 2024-05-21T22:02:20.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.515+0000 7fee37400700 1 --2- 172.21.15.151:0/3432401469 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f95cd45880 0x55f95cd51580 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fee280cc2c0 tx=0x7fee2801e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1dc6d8175c453b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:02:20.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.515+0000 7fee25600700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-21T22:02:20.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.515+0000 7fee25600700 1 -- 172.21.15.151:0/3432401469 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee280b2650 con 0x55f95cd45880 2024-05-21T22:02:20.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.515+0000 7fee25600700 1 -- 172.21.15.151:0/3432401469 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8542+0+0 (secure 0 0 0) 0x7fee280b23d0 con 0x55f95cd45880 2024-05-21T22:02:20.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.515+0000 7fee48dd5b80 1 -- 172.21.15.151:0/3432401469 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee280a97f0 con 0x55f95cd45880 2024-05-21T22:02:20.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.515+0000 7fee25600700 1 -- 172.21.15.151:0/3432401469 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fee280adba0 con 0x55f95cd45880 2024-05-21T22:02:20.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.515+0000 7fee48dd5b80 10 client.26816 fetch_fsmap learned FSMap version 29 2024-05-21T22:02:20.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.515+0000 7fee48dd5b80 1 -- 172.21.15.151:0/3432401469 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f95cc64060 con 0x55f95cd45880 2024-05-21T22:02:20.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.515+0000 7fee25600700 1 -- 172.21.15.151:0/3432401469 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee280a97f0 con 0x55f95cd45880 2024-05-21T22:02:20.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.516+0000 7fee25600700 1 -- 172.21.15.151:0/3432401469 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fee280e2040 con 0x55f95cd45880 2024-05-21T22:02:20.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.516+0000 7fee48dd5b80 10 client.26816 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:02:20.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.516+0000 7fee48dd5b80 10 client.26816 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-05-21T22:02:20.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.516+0000 7fee48dd5b80 20 client.26816 populate_metadata read hostname 'smithi151' 2024-05-21T22:02:20.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.516+0000 7fee48dd5b80 1 --2- 172.21.15.151:0/3432401469 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x55f95cd5ec30 0x55f95cd61020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:20.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.516+0000 7fee48dd5b80 4 client.26816 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T22:02:20.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.516+0000 7fee48dd5b80 1 -- 172.21.15.151:0/3432401469 --> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f95cca5d90 con 0x55f95cd5ec30 2024-05-21T22:02:20.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.516+0000 7fee2ea00700 1 --2- 172.21.15.151:0/3432401469 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x55f95cd5ec30 0x55f95cd61020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:20.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.517+0000 7fee2ea00700 1 --2- 172.21.15.151:0/3432401469 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x55f95cd5ec30 0x55f95cd61020 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d7c8c1c41b32bc7c server_cookie=11af7b28e8bfe8a5 in_seq=0 out_seq=0 2024-05-21T22:02:20.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.517+0000 7fee25600700 10 client.26816 ms_handle_connect on v2:172.21.15.164:6836/1554609572 2024-05-21T22:02:20.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.517+0000 7fee25600700 1 -- 172.21.15.151:0/3432401469 <== mds.2 v2:172.21.15.164:6836/1554609572 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55f95cca5d90 con 0x55f95cd5ec30 2024-05-21T22:02:20.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.517+0000 7fee25600700 10 client.26816 handle_command_reply: tid=0 2024-05-21T22:02:20.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.517+0000 7fee48dd5b80 1 client.26816 shutdown 2024-05-21T22:02:20.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.518+0000 7fee48dd5b80 1 -- 172.21.15.151:0/3432401469 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f95cd45880 msgr2=0x55f95cd51580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:20.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.518+0000 7fee48dd5b80 1 --2- 172.21.15.151:0/3432401469 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f95cd45880 0x55f95cd51580 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fee280cc2c0 tx=0x7fee2801e000 comp rx=0 tx=0).stop 2024-05-21T22:02:20.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.518+0000 7fee48dd5b80 1 -- 172.21.15.151:0/3432401469 shutdown_connections 2024-05-21T22:02:20.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.518+0000 7fee48dd5b80 1 --2- 172.21.15.151:0/3432401469 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x55f95cd5ec30 0x55f95cd61020 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:20.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.518+0000 7fee48dd5b80 1 --2- 172.21.15.151:0/3432401469 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f95cd45880 0x55f95cd51580 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:20.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.518+0000 7fee48dd5b80 1 --2- 172.21.15.151:0/3432401469 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f95cd4b7d0 0x55f95cd4ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:20.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.518+0000 7fee25600700 0 client.26816 ms_handle_reset on v2:172.21.15.164:6836/1554609572 2024-05-21T22:02:20.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.518+0000 7fee48dd5b80 1 --2- 172.21.15.151:0/3432401469 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f95cd4aeb0 0x55f95cd4e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:20.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.518+0000 7fee48dd5b80 1 -- 172.21.15.151:0/3432401469 >> 172.21.15.151:0/3432401469 conn(0x55f95cc73be0 msgr2=0x55f95cd44250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:02:20.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.518+0000 7fee48dd5b80 1 -- 172.21.15.151:0/3432401469 shutdown_connections 2024-05-21T22:02:20.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.518+0000 7fee48dd5b80 1 -- 172.21.15.151:0/3432401469 wait complete. 2024-05-21T22:02:20.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.518+0000 7fee48dd5b80 20 client.26816 trim_cache size 0 max 16384 2024-05-21T22:02:20.535 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.534+0000 7fee48dd5b80 1 Processor -- start 2024-05-21T22:02:20.535 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.534+0000 7fee48dd5b80 1 -- start start 2024-05-21T22:02:20.536 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.534+0000 7fee48dd5b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f95cd45880 0x55f95cd45260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:20.536 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.534+0000 7fee48dd5b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f95cd4aeb0 0x55f95cd489c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:20.536 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.534+0000 7fee48dd5b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f95cd4b7d0 0x55f95cd48f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:20.536 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.534+0000 7fee48dd5b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fee381affc0 con 0x55f95cd4aeb0 2024-05-21T22:02:20.536 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.534+0000 7fee48dd5b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fee381b1b00 con 0x55f95cd4b7d0 2024-05-21T22:02:20.536 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.534+0000 7fee48dd5b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fee381b0690 con 0x55f95cd45880 2024-05-21T22:02:20.536 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.534+0000 7fee2ea00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f95cd4aeb0 0x55f95cd489c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:20.536 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.534+0000 7fee37400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f95cd45880 0x55f95cd45260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:20.536 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.534+0000 7fee37e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f95cd4b7d0 0x55f95cd48f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:20.536 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.534+0000 7fee37400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f95cd45880 0x55f95cd45260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:56718/0 (socket says 172.21.15.151:56718) 2024-05-21T22:02:20.536 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.534+0000 7fee37400700 1 -- 172.21.15.151:0/1759189599 learned_addr learned my addr 172.21.15.151:0/1759189599 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:02:20.536 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.534+0000 7fee2ea00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f95cd4aeb0 0x55f95cd489c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52506/0 (socket says 172.21.15.151:52506) 2024-05-21T22:02:20.539 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.535+0000 7fee37400700 1 -- 172.21.15.151:0/1759189599 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f95cd4b7d0 msgr2=0x55f95cd48f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:20.539 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.535+0000 7fee37400700 1 --2- 172.21.15.151:0/1759189599 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f95cd4b7d0 0x55f95cd48f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:20.539 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.535+0000 7fee37400700 1 -- 172.21.15.151:0/1759189599 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f95cd4aeb0 msgr2=0x55f95cd489c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:20.539 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.535+0000 7fee37400700 1 --2- 172.21.15.151:0/1759189599 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f95cd4aeb0 0x55f95cd489c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:20.539 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.535+0000 7fee37400700 1 -- 172.21.15.151:0/1759189599 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f95cd4ab00 con 0x55f95cd45880 2024-05-21T22:02:20.539 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.535+0000 7fee37400700 1 --2- 172.21.15.151:0/1759189599 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f95cd45880 0x55f95cd45260 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fee280a5870 tx=0x7fee28007ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e948f115c27b094 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:02:20.539 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.535+0000 7fee27e00700 1 -- 172.21.15.151:0/1759189599 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee28087030 con 0x55f95cd45880 2024-05-21T22:02:20.540 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.535+0000 7fee27e00700 1 -- 172.21.15.151:0/1759189599 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee280b2650 con 0x55f95cd45880 2024-05-21T22:02:20.540 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.535+0000 7fee27e00700 1 -- 172.21.15.151:0/1759189599 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee280aab20 con 0x55f95cd45880 2024-05-21T22:02:20.540 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.536+0000 7fee48dd5b80 1 -- 172.21.15.151:0/1759189599 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f95cd45880 msgr2=0x55f95cd45260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:20.540 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.536+0000 7fee48dd5b80 1 --2- 172.21.15.151:0/1759189599 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f95cd45880 0x55f95cd45260 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fee280a5870 tx=0x7fee28007ab0 comp rx=0 tx=0).stop 2024-05-21T22:02:20.540 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.536+0000 7fee48dd5b80 1 -- 172.21.15.151:0/1759189599 shutdown_connections 2024-05-21T22:02:20.540 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.536+0000 7fee48dd5b80 1 --2- 172.21.15.151:0/1759189599 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f95cd4b7d0 0x55f95cd48f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:20.540 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.536+0000 7fee48dd5b80 1 --2- 172.21.15.151:0/1759189599 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f95cd45880 0x55f95cd45260 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:20.540 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.536+0000 7fee48dd5b80 1 --2- 172.21.15.151:0/1759189599 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f95cd4aeb0 0x55f95cd489c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:20.540 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.536+0000 7fee48dd5b80 1 -- 172.21.15.151:0/1759189599 >> 172.21.15.151:0/1759189599 conn(0x55f95cc73be0 msgr2=0x55f95cd4d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:02:20.540 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.536+0000 7fee48dd5b80 1 -- 172.21.15.151:0/1759189599 shutdown_connections 2024-05-21T22:02:20.540 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.536+0000 7fee48dd5b80 1 -- 172.21.15.151:0/1759189599 wait complete. 2024-05-21T22:02:20.540 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.536+0000 7fee48dd5b80 1 -- start start 2024-05-21T22:02:20.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.537+0000 7fee48dd5b80 1 Processor -- start 2024-05-21T22:02:20.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.537+0000 7fee48dd5b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f95cd45880 0x55f95cd61d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:20.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.537+0000 7fee48dd5b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f95cd4aeb0 0x55f95cd62260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:20.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.537+0000 7fee48dd5b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f95cd4b7d0 0x55f95cd627a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:20.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.537+0000 7fee48dd5b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fee381b1430 con 0x55f95cd4aeb0 2024-05-21T22:02:20.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.537+0000 7fee48dd5b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fee381b0d60 con 0x55f95cd45880 2024-05-21T22:02:20.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.537+0000 7fee48dd5b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fee381adf80 con 0x55f95cd4b7d0 2024-05-21T22:02:20.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.537+0000 7fee48dd5b80 10 client.0 _ll_register_callbacks cb 0x55f95cc3dc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:02:20.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.537+0000 7fee2ea00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f95cd4aeb0 0x55f95cd62260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:20.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.537+0000 7fee37400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f95cd45880 0x55f95cd61d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:20.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.537+0000 7fee2ea00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f95cd4aeb0 0x55f95cd62260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52514/0 (socket says 172.21.15.151:52514) 2024-05-21T22:02:20.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.538+0000 7fee2ea00700 1 -- 172.21.15.151:0/2525594956 learned_addr learned my addr 172.21.15.151:0/2525594956 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:02:20.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.538+0000 7fee37e00700 1 --2- 172.21.15.151:0/2525594956 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f95cd4b7d0 0x55f95cd627a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:20.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.538+0000 7fee2ea00700 1 -- 172.21.15.151:0/2525594956 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f95cd4b7d0 msgr2=0x55f95cd627a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:20.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.538+0000 7fee2ea00700 1 --2- 172.21.15.151:0/2525594956 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f95cd4b7d0 0x55f95cd627a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:20.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.538+0000 7fee2ea00700 1 -- 172.21.15.151:0/2525594956 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f95cd45880 msgr2=0x55f95cd61d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:20.542 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.538+0000 7fee2ea00700 1 --2- 172.21.15.151:0/2525594956 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f95cd45880 0x55f95cd61d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:20.542 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.538+0000 7fee2ea00700 1 -- 172.21.15.151:0/2525594956 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f95cdc8470 con 0x55f95cd4aeb0 2024-05-21T22:02:20.542 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.538+0000 7fee37400700 1 --2- 172.21.15.151:0/2525594956 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f95cd45880 0x55f95cd61d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:02:20.543 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.538+0000 7fee2ea00700 1 --2- 172.21.15.151:0/2525594956 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f95cd4aeb0 0x55f95cd62260 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fee20009990 tx=0x7fee20009290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f7e7798d1c345c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:02:20.544 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.538+0000 7fee25600700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-21T22:02:20.544 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.539+0000 7fee25600700 1 -- 172.21.15.151:0/2525594956 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee200052b0 con 0x55f95cd4aeb0 2024-05-21T22:02:20.544 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.539+0000 7fee25600700 1 -- 172.21.15.151:0/2525594956 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8542+0+0 (secure 0 0 0) 0x7fee20005450 con 0x55f95cd4aeb0 2024-05-21T22:02:20.544 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.539+0000 7fee48dd5b80 1 -- 172.21.15.151:0/2525594956 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee280b2650 con 0x55f95cd4aeb0 2024-05-21T22:02:20.544 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.539+0000 7fee25600700 1 -- 172.21.15.151:0/2525594956 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fee20005650 con 0x55f95cd4aeb0 2024-05-21T22:02:20.544 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.539+0000 7fee48dd5b80 10 client.16698 fetch_fsmap learned FSMap version 29 2024-05-21T22:02:20.544 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.539+0000 7fee48dd5b80 1 -- 172.21.15.151:0/2525594956 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f95cdc97a0 con 0x55f95cd4aeb0 2024-05-21T22:02:20.544 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.539+0000 7fee25600700 1 -- 172.21.15.151:0/2525594956 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee280b2650 con 0x55f95cd4aeb0 2024-05-21T22:02:20.544 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.539+0000 7fee25600700 1 -- 172.21.15.151:0/2525594956 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fee2000f700 con 0x55f95cd4aeb0 2024-05-21T22:02:20.544 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.539+0000 7fee48dd5b80 10 client.16698 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:02:20.544 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.539+0000 7fee48dd5b80 10 client.16698 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-05-21T22:02:20.544 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.540+0000 7fee48dd5b80 20 client.16698 populate_metadata read hostname 'smithi151' 2024-05-21T22:02:20.544 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.540+0000 7fee48dd5b80 1 --2- 172.21.15.151:0/2525594956 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x55f95cdc9f30 0x55f95cdcc350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:20.544 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.540+0000 7fee48dd5b80 4 client.16698 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T22:02:20.545 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.540+0000 7fee48dd5b80 1 -- 172.21.15.151:0/2525594956 --> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f95cc9f570 con 0x55f95cdc9f30 2024-05-21T22:02:20.545 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.540+0000 7fee37400700 1 --2- 172.21.15.151:0/2525594956 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x55f95cdc9f30 0x55f95cdcc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:20.545 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.540+0000 7fee37400700 1 --2- 172.21.15.151:0/2525594956 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x55f95cdc9f30 0x55f95cdcc350 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ec4a1e203adad9c2 server_cookie=8e84cd84a7909388 in_seq=0 out_seq=0 2024-05-21T22:02:20.545 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.540+0000 7fee25600700 10 client.16698 ms_handle_connect on v2:172.21.15.164:6836/1554609572 2024-05-21T22:02:20.545 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.540+0000 7fee25600700 1 -- 172.21.15.151:0/2525594956 <== mds.2 v2:172.21.15.164:6836/1554609572 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f95cc9f570 con 0x55f95cdc9f30 2024-05-21T22:02:20.545 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.540+0000 7fee25600700 10 client.16698 handle_command_reply: tid=0 2024-05-21T22:02:20.545 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.541+0000 7fee48dd5b80 1 client.16698 shutdown 2024-05-21T22:02:20.545 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.541+0000 7fee48dd5b80 1 -- 172.21.15.151:0/2525594956 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f95cd4aeb0 msgr2=0x55f95cd62260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:20.545 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.541+0000 7fee48dd5b80 1 --2- 172.21.15.151:0/2525594956 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f95cd4aeb0 0x55f95cd62260 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fee20009990 tx=0x7fee20009290 comp rx=0 tx=0).stop 2024-05-21T22:02:20.545 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.541+0000 7fee48dd5b80 1 -- 172.21.15.151:0/2525594956 shutdown_connections 2024-05-21T22:02:20.545 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.541+0000 7fee48dd5b80 1 --2- 172.21.15.151:0/2525594956 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x55f95cdc9f30 0x55f95cdcc350 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:20.545 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.541+0000 7fee48dd5b80 1 --2- 172.21.15.151:0/2525594956 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55f95cd45880 0x55f95cd61d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:20.545 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.541+0000 7fee48dd5b80 1 --2- 172.21.15.151:0/2525594956 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f95cd4b7d0 0x55f95cd627a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:20.545 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.541+0000 7fee48dd5b80 1 --2- 172.21.15.151:0/2525594956 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f95cd4aeb0 0x55f95cd62260 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:20.545 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.541+0000 7fee48dd5b80 1 -- 172.21.15.151:0/2525594956 >> 172.21.15.151:0/2525594956 conn(0x55f95cc73be0 msgr2=0x55f95cd489c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:02:20.545 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.541+0000 7fee25600700 0 client.16698 ms_handle_reset on v2:172.21.15.164:6836/1554609572 2024-05-21T22:02:20.546 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.541+0000 7fee48dd5b80 1 -- 172.21.15.151:0/2525594956 shutdown_connections 2024-05-21T22:02:20.546 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.541+0000 7fee48dd5b80 1 -- 172.21.15.151:0/2525594956 wait complete. 2024-05-21T22:02:20.546 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.541+0000 7fee48dd5b80 20 client.16698 trim_cache size 0 max 16384 2024-05-21T22:02:20.546 INFO:teuthology.orchestra.run.smithi151.stdout:[] 2024-05-21T22:02:20.547 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.542+0000 7fee3e400700 1 -- 172.21.15.151:0/4160549700 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fee1c0b2a70 msgr2=0x7fee1c0b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:20.547 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.542+0000 7fee3e400700 1 --2- 172.21.15.151:0/4160549700 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fee1c0b2a70 0x7fee1c0b4f30 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fee3000b620 tx=0x7fee300041e0 comp rx=0 tx=0).stop 2024-05-21T22:02:20.547 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.542+0000 7fee3e400700 1 -- 172.21.15.151:0/4160549700 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fee381037f0 msgr2=0x7fee3819bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:20.547 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.542+0000 7fee3e400700 1 --2- 172.21.15.151:0/4160549700 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fee381037f0 0x7fee3819bdb0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fee2800ef10 tx=0x7fee2800c3a0 comp rx=0 tx=0).stop 2024-05-21T22:02:20.547 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.542+0000 7fee3e400700 1 -- 172.21.15.151:0/4160549700 shutdown_connections 2024-05-21T22:02:20.547 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.542+0000 7fee3e400700 1 --2- 172.21.15.151:0/4160549700 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fee1c0b2a70 0x7fee1c0b4f30 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:20.547 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.542+0000 7fee3e400700 1 --2- 172.21.15.151:0/4160549700 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee381088f0 0x7fee381a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:20.547 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.542+0000 7fee3e400700 1 --2- 172.21.15.151:0/4160549700 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fee381037f0 0x7fee3819bdb0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:20.547 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.542+0000 7fee3e400700 1 --2- 172.21.15.151:0/4160549700 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fee381041a0 0x7fee3819c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:20.547 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.542+0000 7fee3e400700 1 -- 172.21.15.151:0/4160549700 >> 172.21.15.151:0/4160549700 conn(0x7fee3807a6a0 msgr2=0x7fee38101a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:02:20.547 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.542+0000 7fee3e400700 1 -- 172.21.15.151:0/4160549700 shutdown_connections 2024-05-21T22:02:20.548 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:20.542+0000 7fee3e400700 1 -- 172.21.15.151:0/4160549700 wait complete. 2024-05-21T22:02:21.511 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-21T22:02:21.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:02:21 smithi151 ceph-mon[31623]: pgmap v207: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-21T22:02:21.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:02:21 smithi184 ceph-mon[36942]: pgmap v207: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-21T22:02:21.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:02:21 smithi164 ceph-mon[36269]: pgmap v207: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-21T22:02:21.958 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:02:23.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:02:23 smithi151 ceph-mon[31623]: pgmap v208: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-05-21T22:02:23.735 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:02:23 smithi184 ceph-mon[36942]: pgmap v208: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-05-21T22:02:23.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:02:23 smithi164 ceph-mon[36269]: pgmap v208: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-05-21T22:02:24.297 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.295+0000 7fd621600700 1 -- 172.21.15.151:0/406945091 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd61c1037f0 msgr2=0x7fd61c103bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:24.297 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.295+0000 7fd621600700 1 --2- 172.21.15.151:0/406945091 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd61c1037f0 0x7fd61c103bd0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fd60c00b0d0 tx=0x7fd60c00b3e0 comp rx=0 tx=0).stop 2024-05-21T22:02:24.297 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.295+0000 7fd621600700 1 -- 172.21.15.151:0/406945091 shutdown_connections 2024-05-21T22:02:24.297 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.295+0000 7fd621600700 1 --2- 172.21.15.151:0/406945091 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd61c1037f0 0x7fd61c103bd0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:24.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.295+0000 7fd621600700 1 --2- 172.21.15.151:0/406945091 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd61c1088f0 0x7fd61c110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:24.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.295+0000 7fd621600700 1 --2- 172.21.15.151:0/406945091 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd61c1041a0 0x7fd61c1081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:24.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.295+0000 7fd621600700 1 -- 172.21.15.151:0/406945091 >> 172.21.15.151:0/406945091 conn(0x7fd61c07a6a0 msgr2=0x7fd61c07aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:02:24.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.296+0000 7fd621600700 1 -- 172.21.15.151:0/406945091 shutdown_connections 2024-05-21T22:02:24.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.296+0000 7fd621600700 1 -- 172.21.15.151:0/406945091 wait complete. 2024-05-21T22:02:24.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.297+0000 7fd621600700 1 Processor -- start 2024-05-21T22:02:24.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.297+0000 7fd621600700 1 -- start start 2024-05-21T22:02:24.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.297+0000 7fd621600700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd61c1037f0 0x7fd61c19f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:24.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.297+0000 7fd621600700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd61c1041a0 0x7fd61c19fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:24.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.297+0000 7fd621600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd61c1088f0 0x7fd61c199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:24.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.297+0000 7fd621600700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd61c113710 con 0x7fd61c1088f0 2024-05-21T22:02:24.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.297+0000 7fd621600700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd61c113590 con 0x7fd61c1037f0 2024-05-21T22:02:24.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.297+0000 7fd621600700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd61c113890 con 0x7fd61c1041a0 2024-05-21T22:02:24.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.297+0000 7fd61aa00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd61c1037f0 0x7fd61c19f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:24.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.297+0000 7fd61aa00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd61c1037f0 0x7fd61c19f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:33312/0 (socket says 172.21.15.151:33312) 2024-05-21T22:02:24.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.297+0000 7fd61aa00700 1 -- 172.21.15.151:0/1814579314 learned_addr learned my addr 172.21.15.151:0/1814579314 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:02:24.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.297+0000 7fd61b400700 1 --2- 172.21.15.151:0/1814579314 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd61c1088f0 0x7fd61c199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:24.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.298+0000 7fd612000700 1 --2- 172.21.15.151:0/1814579314 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd61c1041a0 0x7fd61c19fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:24.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.298+0000 7fd61aa00700 1 -- 172.21.15.151:0/1814579314 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd61c1041a0 msgr2=0x7fd61c19fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:24.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.298+0000 7fd61aa00700 1 --2- 172.21.15.151:0/1814579314 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd61c1041a0 0x7fd61c19fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:24.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.298+0000 7fd61aa00700 1 -- 172.21.15.151:0/1814579314 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd61c1088f0 msgr2=0x7fd61c199a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:24.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.298+0000 7fd61aa00700 1 --2- 172.21.15.151:0/1814579314 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd61c1088f0 0x7fd61c199a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:24.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.298+0000 7fd61aa00700 1 -- 172.21.15.151:0/1814579314 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd61c19a1d0 con 0x7fd61c1037f0 2024-05-21T22:02:24.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.298+0000 7fd61aa00700 1 --2- 172.21.15.151:0/1814579314 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd61c1037f0 0x7fd61c19f860 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fd60c007a20 tx=0x7fd60c007a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:02:24.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.299+0000 7fd613e00700 1 -- 172.21.15.151:0/1814579314 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd60c01f070 con 0x7fd61c1037f0 2024-05-21T22:02:24.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.299+0000 7fd613e00700 1 -- 172.21.15.151:0/1814579314 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd60c004c50 con 0x7fd61c1037f0 2024-05-21T22:02:24.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.299+0000 7fd621600700 1 -- 172.21.15.151:0/1814579314 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd61c19a460 con 0x7fd61c1037f0 2024-05-21T22:02:24.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.299+0000 7fd613e00700 1 -- 172.21.15.151:0/1814579314 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd60c01b690 con 0x7fd61c1037f0 2024-05-21T22:02:24.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.299+0000 7fd621600700 1 -- 172.21.15.151:0/1814579314 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd61c1a6730 con 0x7fd61c1037f0 2024-05-21T22:02:24.303 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.302+0000 7fd613e00700 1 -- 172.21.15.151:0/1814579314 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7fd60c022070 con 0x7fd61c1037f0 2024-05-21T22:02:24.304 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.303+0000 7fd62c045b80 1 Processor -- start 2024-05-21T22:02:24.305 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.303+0000 7fd62c045b80 1 -- start start 2024-05-21T22:02:24.305 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.304+0000 7fd62c045b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563ae6ab7a20 0x563ae6ab7e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:24.305 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.304+0000 7fd62c045b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563ae6ab8340 0x563ae6ab1f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:24.305 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.304+0000 7fd62c045b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563ae6ab2550 0x563ae6ab29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:24.305 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.304+0000 7fd62c045b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x563ae6a0be40 con 0x563ae6ab2550 2024-05-21T22:02:24.305 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.304+0000 7fd62c045b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x563ae69fb9e0 con 0x563ae6ab8340 2024-05-21T22:02:24.305 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.304+0000 7fd62c045b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x563ae69ffa20 con 0x563ae6ab7a20 2024-05-21T22:02:24.306 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.304+0000 7fd613e00700 1 --2- 172.21.15.151:0/1814579314 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fd6000b2a70 0x7fd6000b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:24.307 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.304+0000 7fd612000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563ae6ab2550 0x563ae6ab29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:24.307 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.304+0000 7fd613e00700 1 -- 172.21.15.151:0/1814579314 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8542+0+0 (secure 0 0 0) 0x7fd60c00f020 con 0x7fd61c1037f0 2024-05-21T22:02:24.307 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.304+0000 7fd61b400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563ae6ab7a20 0x563ae6ab7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:24.307 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.304+0000 7fd612000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563ae6ab2550 0x563ae6ab29b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52554/0 (socket says 172.21.15.151:52554) 2024-05-21T22:02:24.307 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.304+0000 7fd61aa00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563ae6ab8340 0x563ae6ab1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:24.307 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.304+0000 7fd612000700 1 -- 172.21.15.151:0/4247262577 learned_addr learned my addr 172.21.15.151:0/4247262577 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:02:24.307 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.305+0000 7fd61b400700 1 --2- 172.21.15.151:0/1814579314 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fd6000b2a70 0x7fd6000b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:24.307 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.305+0000 7fd61aa00700 1 -- 172.21.15.151:0/4247262577 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563ae6ab7a20 msgr2=0x563ae6ab7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:24.307 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.305+0000 7fd61aa00700 1 --2- 172.21.15.151:0/4247262577 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563ae6ab7a20 0x563ae6ab7e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:24.307 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.305+0000 7fd61aa00700 1 -- 172.21.15.151:0/4247262577 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563ae6ab2550 msgr2=0x563ae6ab29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:24.307 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.305+0000 7fd61aa00700 1 --2- 172.21.15.151:0/4247262577 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563ae6ab2550 0x563ae6ab29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:24.308 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.305+0000 7fd61aa00700 1 -- 172.21.15.151:0/4247262577 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ae69c4420 con 0x563ae6ab8340 2024-05-21T22:02:24.308 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.305+0000 7fd61b400700 1 --2- 172.21.15.151:0/4247262577 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563ae6ab7a20 0x563ae6ab7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T22:02:24.308 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.305+0000 7fd61aa00700 1 --2- 172.21.15.151:0/4247262577 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563ae6ab8340 0x563ae6ab1f30 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd60c0e0dc0 tx=0x7fd60c005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58129b457ad6d387 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:02:24.308 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.305+0000 7fd60be00700 1 -- 172.21.15.151:0/4247262577 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd60c0ab0e0 con 0x563ae6ab8340 2024-05-21T22:02:24.308 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.305+0000 7fd61b400700 1 --2- 172.21.15.151:0/1814579314 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fd6000b2a70 0x7fd6000b4f30 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd614005940 tx=0x7fd614009260 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:02:24.308 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.306+0000 7fd60be00700 1 -- 172.21.15.151:0/4247262577 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd60c0aa9e0 con 0x563ae6ab8340 2024-05-21T22:02:24.308 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.306+0000 7fd60be00700 1 -- 172.21.15.151:0/4247262577 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd60c0b1110 con 0x563ae6ab8340 2024-05-21T22:02:24.309 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.306+0000 7fd62c045b80 1 -- 172.21.15.151:0/4247262577 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563ae6ab8340 msgr2=0x563ae6ab1f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:24.309 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.306+0000 7fd62c045b80 1 --2- 172.21.15.151:0/4247262577 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563ae6ab8340 0x563ae6ab1f30 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd60c0e0dc0 tx=0x7fd60c005850 comp rx=0 tx=0).stop 2024-05-21T22:02:24.309 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.306+0000 7fd62c045b80 1 -- 172.21.15.151:0/4247262577 shutdown_connections 2024-05-21T22:02:24.309 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.306+0000 7fd62c045b80 1 --2- 172.21.15.151:0/4247262577 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563ae6ab2550 0x563ae6ab29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:24.309 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.306+0000 7fd62c045b80 1 --2- 172.21.15.151:0/4247262577 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563ae6ab7a20 0x563ae6ab7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:24.310 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.306+0000 7fd62c045b80 1 --2- 172.21.15.151:0/4247262577 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563ae6ab8340 0x563ae6ab1f30 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:24.310 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.306+0000 7fd62c045b80 1 -- 172.21.15.151:0/4247262577 >> 172.21.15.151:0/4247262577 conn(0x563ae69dbbe0 msgr2=0x563ae6aa8690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:02:24.310 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.307+0000 7fd62c045b80 1 -- 172.21.15.151:0/4247262577 shutdown_connections 2024-05-21T22:02:24.310 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.307+0000 7fd62c045b80 1 -- 172.21.15.151:0/4247262577 wait complete. 2024-05-21T22:02:24.310 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.307+0000 7fd62c045b80 1 -- start start 2024-05-21T22:02:24.310 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.307+0000 7fd62c045b80 1 Processor -- start 2024-05-21T22:02:24.310 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.308+0000 7fd62c045b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563ae6ab2550 0x563ae6abe060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:24.310 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.308+0000 7fd62c045b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563ae6ab7a20 0x563ae6abb450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:24.310 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.308+0000 7fd62c045b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563ae6ab8340 0x563ae6abb990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:24.310 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.308+0000 7fd62c045b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x563ae6901fa0 con 0x563ae6ab7a20 2024-05-21T22:02:24.310 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.308+0000 7fd62c045b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x563ae699c760 con 0x563ae6ab8340 2024-05-21T22:02:24.310 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.308+0000 7fd62c045b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x563ae6abec20 con 0x563ae6ab2550 2024-05-21T22:02:24.310 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.308+0000 7fd62c045b80 10 client.0 _ll_register_callbacks cb 0x563ae69a5c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:02:24.312 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.308+0000 7fd61aa00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563ae6ab2550 0x563ae6abe060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:24.312 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.308+0000 7fd61b400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563ae6ab8340 0x563ae6abb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:24.312 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.308+0000 7fd61aa00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563ae6ab2550 0x563ae6abe060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:56778/0 (socket says 172.21.15.151:56778) 2024-05-21T22:02:24.313 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.308+0000 7fd61aa00700 1 -- 172.21.15.151:0/1447782282 learned_addr learned my addr 172.21.15.151:0/1447782282 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:02:24.313 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.308+0000 7fd612000700 1 --2- 172.21.15.151:0/1447782282 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563ae6ab7a20 0x563ae6abb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:24.313 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.309+0000 7fd61aa00700 1 -- 172.21.15.151:0/1447782282 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563ae6ab8340 msgr2=0x563ae6abb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:24.313 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.309+0000 7fd61aa00700 1 --2- 172.21.15.151:0/1447782282 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563ae6ab8340 0x563ae6abb990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:24.313 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.309+0000 7fd61aa00700 1 -- 172.21.15.151:0/1447782282 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563ae6ab7a20 msgr2=0x563ae6abb450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:24.313 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.309+0000 7fd61aa00700 1 --2- 172.21.15.151:0/1447782282 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563ae6ab7a20 0x563ae6abb450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:24.313 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.309+0000 7fd61aa00700 1 -- 172.21.15.151:0/1447782282 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ae694b8d0 con 0x563ae6ab2550 2024-05-21T22:02:24.313 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.309+0000 7fd61aa00700 1 --2- 172.21.15.151:0/1447782282 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563ae6ab2550 0x563ae6abe060 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fd60c0aa970 tx=0x7fd60c0a69d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=efb9fc34f7d5ce5b server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:02:24.313 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.309+0000 7fd608c00700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-21T22:02:24.313 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.309+0000 7fd608c00700 1 -- 172.21.15.151:0/1447782282 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd60c0c3060 con 0x563ae6ab2550 2024-05-21T22:02:24.313 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.309+0000 7fd608c00700 1 -- 172.21.15.151:0/1447782282 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8542+0+0 (secure 0 0 0) 0x7fd60c0233b0 con 0x563ae6ab2550 2024-05-21T22:02:24.313 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.309+0000 7fd62c045b80 1 -- 172.21.15.151:0/1447782282 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd60c0aa9e0 con 0x563ae6ab2550 2024-05-21T22:02:24.314 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.310+0000 7fd608c00700 1 -- 172.21.15.151:0/1447782282 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd60c0e1250 con 0x563ae6ab2550 2024-05-21T22:02:24.314 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.310+0000 7fd62c045b80 10 client.26845 fetch_fsmap learned FSMap version 29 2024-05-21T22:02:24.314 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.310+0000 7fd62c045b80 1 -- 172.21.15.151:0/1447782282 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563ae69cc060 con 0x563ae6ab2550 2024-05-21T22:02:24.314 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.310+0000 7fd608c00700 1 -- 172.21.15.151:0/1447782282 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd60c0aa9e0 con 0x563ae6ab2550 2024-05-21T22:02:24.314 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.310+0000 7fd608c00700 1 -- 172.21.15.151:0/1447782282 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fd60c011070 con 0x563ae6ab2550 2024-05-21T22:02:24.314 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.310+0000 7fd62c045b80 10 client.26845 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:02:24.314 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.310+0000 7fd62c045b80 10 client.26845 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-21T22:02:24.314 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.310+0000 7fd62c045b80 20 client.26845 populate_metadata read hostname 'smithi151' 2024-05-21T22:02:24.314 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.310+0000 7fd62c045b80 1 --2- 172.21.15.151:0/1447782282 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x563ae6acb4b0 0x563ae6acd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:24.314 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.310+0000 7fd62c045b80 4 client.26845 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T22:02:24.314 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.310+0000 7fd62c045b80 1 -- 172.21.15.151:0/1447782282 --> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563ae6a0dd90 con 0x563ae6acb4b0 2024-05-21T22:02:24.314 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.311+0000 7fd612000700 1 --2- 172.21.15.151:0/1447782282 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x563ae6acb4b0 0x563ae6acd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:24.314 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.311+0000 7fd612000700 1 --2- 172.21.15.151:0/1447782282 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x563ae6acb4b0 0x563ae6acd8a0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a51b936cd66b97d server_cookie=b8c1a99c01c8a5cc in_seq=0 out_seq=0 2024-05-21T22:02:24.314 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.311+0000 7fd608c00700 10 client.26845 ms_handle_connect on v2:172.21.15.184:6838/3059935143 2024-05-21T22:02:24.315 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.312+0000 7fd608c00700 1 -- 172.21.15.151:0/1447782282 <== mds.1 v2:172.21.15.184:6838/3059935143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x563ae6a0dd90 con 0x563ae6acb4b0 2024-05-21T22:02:24.315 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.312+0000 7fd608c00700 10 client.26845 handle_command_reply: tid=0 2024-05-21T22:02:24.316 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.312+0000 7fd62c045b80 1 client.26845 shutdown 2024-05-21T22:02:24.316 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.312+0000 7fd62c045b80 1 -- 172.21.15.151:0/1447782282 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563ae6ab2550 msgr2=0x563ae6abe060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:24.316 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.312+0000 7fd62c045b80 1 --2- 172.21.15.151:0/1447782282 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563ae6ab2550 0x563ae6abe060 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fd60c0aa970 tx=0x7fd60c0a69d0 comp rx=0 tx=0).stop 2024-05-21T22:02:24.316 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.312+0000 7fd62c045b80 1 -- 172.21.15.151:0/1447782282 shutdown_connections 2024-05-21T22:02:24.316 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.312+0000 7fd62c045b80 1 --2- 172.21.15.151:0/1447782282 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x563ae6acb4b0 0x563ae6acd8a0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:24.316 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.312+0000 7fd62c045b80 1 --2- 172.21.15.151:0/1447782282 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563ae6ab8340 0x563ae6abb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:24.316 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.312+0000 7fd62c045b80 1 --2- 172.21.15.151:0/1447782282 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563ae6ab2550 0x563ae6abe060 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:24.316 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.312+0000 7fd62c045b80 1 --2- 172.21.15.151:0/1447782282 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563ae6ab7a20 0x563ae6abb450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:24.316 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.312+0000 7fd608c00700 0 client.26845 ms_handle_reset on v2:172.21.15.184:6838/3059935143 2024-05-21T22:02:24.316 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.312+0000 7fd62c045b80 1 -- 172.21.15.151:0/1447782282 >> 172.21.15.151:0/1447782282 conn(0x563ae69dbbe0 msgr2=0x563ae6aa7d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:02:24.316 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.313+0000 7fd62c045b80 1 -- 172.21.15.151:0/1447782282 shutdown_connections 2024-05-21T22:02:24.316 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.313+0000 7fd62c045b80 1 -- 172.21.15.151:0/1447782282 wait complete. 2024-05-21T22:02:24.316 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.313+0000 7fd62c045b80 20 client.26845 trim_cache size 0 max 16384 2024-05-21T22:02:24.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.329+0000 7fd62c045b80 1 Processor -- start 2024-05-21T22:02:24.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.329+0000 7fd62c045b80 1 -- start start 2024-05-21T22:02:24.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.330+0000 7fd62c045b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563ae6ab2550 0x563ae6abb940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:24.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.330+0000 7fd62c045b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563ae6ab7a20 0x563ae6aa84f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:24.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.330+0000 7fd62c045b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563ae6ab8340 0x563ae6ad30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:24.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.330+0000 7fd62c045b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd61c1aba90 con 0x563ae6ab2550 2024-05-21T22:02:24.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.330+0000 7fd62c045b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd61c1ad5d0 con 0x563ae6ab8340 2024-05-21T22:02:24.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.330+0000 7fd62c045b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd61c1ac160 con 0x563ae6ab7a20 2024-05-21T22:02:24.332 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.330+0000 7fd61aa00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563ae6ab2550 0x563ae6abb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:24.333 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.330+0000 7fd612000700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563ae6ab7a20 0x563ae6aa84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:24.333 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.330+0000 7fd61aa00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563ae6ab2550 0x563ae6abb940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52580/0 (socket says 172.21.15.151:52580) 2024-05-21T22:02:24.333 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.330+0000 7fd61aa00700 1 -- 172.21.15.151:0/2464191112 learned_addr learned my addr 172.21.15.151:0/2464191112 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:02:24.333 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.330+0000 7fd61b400700 1 --2- 172.21.15.151:0/2464191112 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563ae6ab8340 0x563ae6ad30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:24.333 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.331+0000 7fd61aa00700 1 -- 172.21.15.151:0/2464191112 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563ae6ab7a20 msgr2=0x563ae6aa84f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:24.333 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.331+0000 7fd61aa00700 1 --2- 172.21.15.151:0/2464191112 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563ae6ab7a20 0x563ae6aa84f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:24.333 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.331+0000 7fd61aa00700 1 -- 172.21.15.151:0/2464191112 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563ae6ab8340 msgr2=0x563ae6ad30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:24.333 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.331+0000 7fd61aa00700 1 --2- 172.21.15.151:0/2464191112 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563ae6ab8340 0x563ae6ad30b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:24.333 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.331+0000 7fd61aa00700 1 -- 172.21.15.151:0/2464191112 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ae6abe060 con 0x563ae6ab2550 2024-05-21T22:02:24.333 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.331+0000 7fd612000700 1 --2- 172.21.15.151:0/2464191112 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563ae6ab7a20 0x563ae6aa84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:02:24.333 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.331+0000 7fd61aa00700 1 --2- 172.21.15.151:0/2464191112 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563ae6ab2550 0x563ae6abb940 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fd60c00bfd0 tx=0x7fd60c0b1420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2df36456d4b43895 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:02:24.333 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.331+0000 7fd60b400700 1 -- 172.21.15.151:0/2464191112 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd60c0c3070 con 0x563ae6ab2550 2024-05-21T22:02:24.333 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.331+0000 7fd60b400700 1 -- 172.21.15.151:0/2464191112 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd60c0b35a0 con 0x563ae6ab2550 2024-05-21T22:02:24.333 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.331+0000 7fd60b400700 1 -- 172.21.15.151:0/2464191112 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd60c0233b0 con 0x563ae6ab2550 2024-05-21T22:02:24.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.332+0000 7fd62c045b80 1 -- 172.21.15.151:0/2464191112 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563ae6ab2550 msgr2=0x563ae6abb940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:24.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.332+0000 7fd62c045b80 1 --2- 172.21.15.151:0/2464191112 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563ae6ab2550 0x563ae6abb940 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fd60c00bfd0 tx=0x7fd60c0b1420 comp rx=0 tx=0).stop 2024-05-21T22:02:24.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.332+0000 7fd62c045b80 1 -- 172.21.15.151:0/2464191112 shutdown_connections 2024-05-21T22:02:24.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.332+0000 7fd62c045b80 1 --2- 172.21.15.151:0/2464191112 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563ae6ab8340 0x563ae6ad30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:24.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.332+0000 7fd62c045b80 1 --2- 172.21.15.151:0/2464191112 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563ae6ab2550 0x563ae6abb940 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:24.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.332+0000 7fd62c045b80 1 --2- 172.21.15.151:0/2464191112 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563ae6ab7a20 0x563ae6aa84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:24.336 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.332+0000 7fd62c045b80 1 -- 172.21.15.151:0/2464191112 >> 172.21.15.151:0/2464191112 conn(0x563ae69dbbe0 msgr2=0x563ae6abb230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:02:24.336 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.332+0000 7fd62c045b80 1 -- 172.21.15.151:0/2464191112 shutdown_connections 2024-05-21T22:02:24.336 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.332+0000 7fd62c045b80 1 -- 172.21.15.151:0/2464191112 wait complete. 2024-05-21T22:02:24.336 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.333+0000 7fd62c045b80 1 -- start start 2024-05-21T22:02:24.336 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.333+0000 7fd62c045b80 1 Processor -- start 2024-05-21T22:02:24.336 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.333+0000 7fd62c045b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563ae6ab2550 0x563ae6ad2d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:24.336 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.333+0000 7fd62c045b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563ae6ab7a20 0x563ae6ad3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:24.336 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.333+0000 7fd62c045b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563ae6ab8340 0x563ae6ad3780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:24.336 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.333+0000 7fd62c045b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd61c1acf00 con 0x563ae6ab7a20 2024-05-21T22:02:24.336 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.333+0000 7fd62c045b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd61c1ac830 con 0x563ae6ab8340 2024-05-21T22:02:24.336 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.333+0000 7fd62c045b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd61c1a9a70 con 0x563ae6ab2550 2024-05-21T22:02:24.336 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.333+0000 7fd62c045b80 10 client.0 _ll_register_callbacks cb 0x563ae69a5c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:02:24.336 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.334+0000 7fd61b400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563ae6ab8340 0x563ae6ad3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:24.336 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.334+0000 7fd61aa00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563ae6ab2550 0x563ae6ad2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:24.336 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.334+0000 7fd61b400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563ae6ab8340 0x563ae6ad3780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:33354/0 (socket says 172.21.15.151:33354) 2024-05-21T22:02:24.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.334+0000 7fd612000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563ae6ab7a20 0x563ae6ad3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:24.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.334+0000 7fd61b400700 1 -- 172.21.15.151:0/1190613693 learned_addr learned my addr 172.21.15.151:0/1190613693 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:02:24.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.334+0000 7fd61aa00700 1 -- 172.21.15.151:0/1190613693 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563ae6ab8340 msgr2=0x563ae6ad3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:24.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.334+0000 7fd61aa00700 1 --2- 172.21.15.151:0/1190613693 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563ae6ab8340 0x563ae6ad3780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:24.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.334+0000 7fd61aa00700 1 -- 172.21.15.151:0/1190613693 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563ae6ab7a20 msgr2=0x563ae6ad3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:24.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.334+0000 7fd61aa00700 1 --2- 172.21.15.151:0/1190613693 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563ae6ab7a20 0x563ae6ad3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:24.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.334+0000 7fd61aa00700 1 -- 172.21.15.151:0/1190613693 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ae6b35000 con 0x563ae6ab2550 2024-05-21T22:02:24.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.335+0000 7fd61aa00700 1 --2- 172.21.15.151:0/1190613693 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563ae6ab2550 0x563ae6ad2d00 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fd60c0b3280 tx=0x7fd60c009bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57ee67c06a02dff0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:02:24.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.335+0000 7fd608c00700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-21T22:02:24.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.335+0000 7fd608c00700 1 -- 172.21.15.151:0/1190613693 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd60c0c3070 con 0x563ae6ab2550 2024-05-21T22:02:24.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.335+0000 7fd608c00700 1 -- 172.21.15.151:0/1190613693 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8542+0+0 (secure 0 0 0) 0x7fd60c005db0 con 0x563ae6ab2550 2024-05-21T22:02:24.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.335+0000 7fd62c045b80 1 -- 172.21.15.151:0/1190613693 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd60c0b35a0 con 0x563ae6ab2550 2024-05-21T22:02:24.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.335+0000 7fd608c00700 1 -- 172.21.15.151:0/1190613693 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd60c0aa9e0 con 0x563ae6ab2550 2024-05-21T22:02:24.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.335+0000 7fd62c045b80 10 client.26851 fetch_fsmap learned FSMap version 29 2024-05-21T22:02:24.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.335+0000 7fd62c045b80 1 -- 172.21.15.151:0/1190613693 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563ae6b35000 con 0x563ae6ab2550 2024-05-21T22:02:24.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.335+0000 7fd608c00700 1 -- 172.21.15.151:0/1190613693 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd60c0b35a0 con 0x563ae6ab2550 2024-05-21T22:02:24.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.335+0000 7fd608c00700 1 -- 172.21.15.151:0/1190613693 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fd60c011070 con 0x563ae6ab2550 2024-05-21T22:02:24.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.336+0000 7fd62c045b80 10 client.26851 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:02:24.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.336+0000 7fd62c045b80 10 client.26851 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-21T22:02:24.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.336+0000 7fd62c045b80 20 client.26851 populate_metadata read hostname 'smithi151' 2024-05-21T22:02:24.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.336+0000 7fd62c045b80 1 --2- 172.21.15.151:0/1190613693 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x563ae6b364f0 0x563ae6b388e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:24.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.336+0000 7fd62c045b80 4 client.26851 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T22:02:24.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.336+0000 7fd62c045b80 1 -- 172.21.15.151:0/1190613693 --> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563ae6a07570 con 0x563ae6b364f0 2024-05-21T22:02:24.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.336+0000 7fd612000700 1 --2- 172.21.15.151:0/1190613693 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x563ae6b364f0 0x563ae6b388e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:24.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.336+0000 7fd612000700 1 --2- 172.21.15.151:0/1190613693 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x563ae6b364f0 0x563ae6b388e0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9c2dbbe38c602601 server_cookie=c71aa630b0c2383d in_seq=0 out_seq=0 2024-05-21T22:02:24.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.336+0000 7fd608c00700 10 client.26851 ms_handle_connect on v2:172.21.15.184:6838/3059935143 2024-05-21T22:02:24.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.337+0000 7fd608c00700 1 -- 172.21.15.151:0/1190613693 <== mds.1 v2:172.21.15.184:6838/3059935143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563ae6a07570 con 0x563ae6b364f0 2024-05-21T22:02:24.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.337+0000 7fd608c00700 10 client.26851 handle_command_reply: tid=0 2024-05-21T22:02:24.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.337+0000 7fd62c045b80 1 client.26851 shutdown 2024-05-21T22:02:24.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.337+0000 7fd62c045b80 1 -- 172.21.15.151:0/1190613693 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563ae6ab2550 msgr2=0x563ae6ad2d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:24.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.337+0000 7fd62c045b80 1 --2- 172.21.15.151:0/1190613693 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563ae6ab2550 0x563ae6ad2d00 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fd60c0b3280 tx=0x7fd60c009bf0 comp rx=0 tx=0).stop 2024-05-21T22:02:24.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.337+0000 7fd62c045b80 1 -- 172.21.15.151:0/1190613693 shutdown_connections 2024-05-21T22:02:24.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.337+0000 7fd62c045b80 1 --2- 172.21.15.151:0/1190613693 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x563ae6b364f0 0x563ae6b388e0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:24.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.337+0000 7fd62c045b80 1 --2- 172.21.15.151:0/1190613693 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563ae6ab8340 0x563ae6ad3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:24.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.337+0000 7fd62c045b80 1 --2- 172.21.15.151:0/1190613693 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563ae6ab2550 0x563ae6ad2d00 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:24.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.337+0000 7fd62c045b80 1 --2- 172.21.15.151:0/1190613693 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563ae6ab7a20 0x563ae6ad3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:24.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.337+0000 7fd608c00700 0 client.26851 ms_handle_reset on v2:172.21.15.184:6838/3059935143 2024-05-21T22:02:24.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.337+0000 7fd62c045b80 1 -- 172.21.15.151:0/1190613693 >> 172.21.15.151:0/1190613693 conn(0x563ae69dbbe0 msgr2=0x563ae6ab8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:02:24.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.337+0000 7fd62c045b80 1 -- 172.21.15.151:0/1190613693 shutdown_connections 2024-05-21T22:02:24.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.337+0000 7fd62c045b80 1 -- 172.21.15.151:0/1190613693 wait complete. 2024-05-21T22:02:24.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.337+0000 7fd62c045b80 20 client.26851 trim_cache size 0 max 16384 2024-05-21T22:02:24.342 INFO:teuthology.orchestra.run.smithi151.stdout:[] 2024-05-21T22:02:24.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.338+0000 7fd621600700 1 -- 172.21.15.151:0/1814579314 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fd6000b2a70 msgr2=0x7fd6000b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:24.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.338+0000 7fd621600700 1 --2- 172.21.15.151:0/1814579314 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fd6000b2a70 0x7fd6000b4f30 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd614005940 tx=0x7fd614009260 comp rx=0 tx=0).stop 2024-05-21T22:02:24.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.338+0000 7fd621600700 1 -- 172.21.15.151:0/1814579314 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd61c1037f0 msgr2=0x7fd61c19f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:24.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.338+0000 7fd621600700 1 --2- 172.21.15.151:0/1814579314 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd61c1037f0 0x7fd61c19f860 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fd60c007a20 tx=0x7fd60c007a50 comp rx=0 tx=0).stop 2024-05-21T22:02:24.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.338+0000 7fd621600700 1 -- 172.21.15.151:0/1814579314 shutdown_connections 2024-05-21T22:02:24.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.338+0000 7fd621600700 1 --2- 172.21.15.151:0/1814579314 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fd6000b2a70 0x7fd6000b4f30 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:24.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.338+0000 7fd621600700 1 --2- 172.21.15.151:0/1814579314 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd61c1037f0 0x7fd61c19f860 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:24.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.338+0000 7fd621600700 1 --2- 172.21.15.151:0/1814579314 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd61c1088f0 0x7fd61c199a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:24.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.338+0000 7fd621600700 1 --2- 172.21.15.151:0/1814579314 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd61c1041a0 0x7fd61c19fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:24.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.338+0000 7fd621600700 1 -- 172.21.15.151:0/1814579314 >> 172.21.15.151:0/1814579314 conn(0x7fd61c07a6a0 msgr2=0x7fd61c1017f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:02:24.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.338+0000 7fd621600700 1 -- 172.21.15.151:0/1814579314 shutdown_connections 2024-05-21T22:02:24.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:24.338+0000 7fd621600700 1 -- 172.21.15.151:0/1814579314 wait complete. 2024-05-21T22:02:25.211 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-21T22:02:25.661 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:02:25.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:02:25 smithi151 ceph-mon[31623]: pgmap v209: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 99 B/s wr, 9 op/s 2024-05-21T22:02:25.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:02:25 smithi184 ceph-mon[36942]: pgmap v209: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 99 B/s wr, 9 op/s 2024-05-21T22:02:25.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:02:25 smithi164 ceph-mon[36269]: pgmap v209: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 99 B/s wr, 9 op/s 2024-05-21T22:02:27.693 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:02:27 smithi151 ceph-mon[31623]: pgmap v210: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-05-21T22:02:27.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:02:27 smithi184 ceph-mon[36942]: pgmap v210: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-05-21T22:02:27.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:02:27 smithi164 ceph-mon[36269]: pgmap v210: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-05-21T22:02:27.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.864+0000 7f0cd3e00700 1 -- 172.21.15.151:0/2633023728 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0ccc076090 msgr2=0x7f0ccc1141d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:27.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.864+0000 7f0cd3e00700 1 --2- 172.21.15.151:0/2633023728 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0ccc076090 0x7f0ccc1141d0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f0cb8009990 tx=0x7f0cb8009ca0 comp rx=0 tx=0).stop 2024-05-21T22:02:27.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.865+0000 7f0cd3e00700 1 -- 172.21.15.151:0/2633023728 shutdown_connections 2024-05-21T22:02:27.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.865+0000 7f0cd3e00700 1 --2- 172.21.15.151:0/2633023728 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0ccc075770 0x7f0ccc075b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:27.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.865+0000 7f0cd3e00700 1 --2- 172.21.15.151:0/2633023728 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ccc114710 0x7f0ccc116b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:27.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.865+0000 7f0cd3e00700 1 --2- 172.21.15.151:0/2633023728 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0ccc076090 0x7f0ccc1141d0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:27.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.865+0000 7f0cd3e00700 1 -- 172.21.15.151:0/2633023728 >> 172.21.15.151:0/2633023728 conn(0x7f0ccc07a6a0 msgr2=0x7f0ccc07aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:02:27.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.865+0000 7f0cd3e00700 1 -- 172.21.15.151:0/2633023728 shutdown_connections 2024-05-21T22:02:27.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.865+0000 7f0cd3e00700 1 -- 172.21.15.151:0/2633023728 wait complete. 2024-05-21T22:02:27.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.866+0000 7f0cd3e00700 1 Processor -- start 2024-05-21T22:02:27.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.866+0000 7f0cd3e00700 1 -- start start 2024-05-21T22:02:27.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.866+0000 7f0cd3e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ccc075770 0x7f0ccc1a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:27.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.866+0000 7f0cd3e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0ccc076090 0x7f0ccc1a0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:27.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.866+0000 7f0cd3e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0ccc114710 0x7f0ccc1a4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:27.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.866+0000 7f0cd3e00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0ccc119d20 con 0x7f0ccc076090 2024-05-21T22:02:27.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.866+0000 7f0cd3e00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0ccc119ba0 con 0x7f0ccc114710 2024-05-21T22:02:27.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.866+0000 7f0cd3e00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0ccc119ea0 con 0x7f0ccc075770 2024-05-21T22:02:27.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.867+0000 7f0cd0a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0ccc076090 0x7f0ccc1a0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:27.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.867+0000 7f0cd1e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0ccc114710 0x7f0ccc1a4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:27.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.867+0000 7f0cd0a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0ccc076090 0x7f0ccc1a0730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:58168/0 (socket says 172.21.15.151:58168) 2024-05-21T22:02:27.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.867+0000 7f0cd1e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0ccc114710 0x7f0ccc1a4ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:55248/0 (socket says 172.21.15.151:55248) 2024-05-21T22:02:27.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.867+0000 7f0cd0a00700 1 -- 172.21.15.151:0/2262095451 learned_addr learned my addr 172.21.15.151:0/2262095451 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:02:27.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.867+0000 7f0cd1400700 1 --2- 172.21.15.151:0/2262095451 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ccc075770 0x7f0ccc1a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:27.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.867+0000 7f0cd1e00700 1 -- 172.21.15.151:0/2262095451 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ccc075770 msgr2=0x7f0ccc1a01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:27.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.867+0000 7f0cd1e00700 1 --2- 172.21.15.151:0/2262095451 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ccc075770 0x7f0ccc1a01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:27.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.867+0000 7f0cd1e00700 1 -- 172.21.15.151:0/2262095451 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0ccc076090 msgr2=0x7f0ccc1a0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:27.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.867+0000 7f0cd1e00700 1 --2- 172.21.15.151:0/2262095451 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0ccc076090 0x7f0ccc1a0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:27.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.867+0000 7f0cd1e00700 1 -- 172.21.15.151:0/2262095451 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ccc1a5200 con 0x7f0ccc114710 2024-05-21T22:02:27.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.867+0000 7f0cd0a00700 1 --2- 172.21.15.151:0/2262095451 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0ccc076090 0x7f0ccc1a0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:02:27.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.868+0000 7f0cd1e00700 1 --2- 172.21.15.151:0/2262095451 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0ccc114710 0x7f0ccc1a4ac0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f0cc400ef90 tx=0x7f0cc400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:02:27.871 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.868+0000 7f0cca000700 1 -- 172.21.15.151:0/2262095451 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc40092e0 con 0x7f0ccc114710 2024-05-21T22:02:27.871 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.868+0000 7f0cca000700 1 -- 172.21.15.151:0/2262095451 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cc4004bb0 con 0x7f0ccc114710 2024-05-21T22:02:27.871 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.868+0000 7f0cd3e00700 1 -- 172.21.15.151:0/2262095451 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ccc1a5490 con 0x7f0ccc114710 2024-05-21T22:02:27.871 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.868+0000 7f0cca000700 1 -- 172.21.15.151:0/2262095451 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc4004020 con 0x7f0ccc114710 2024-05-21T22:02:27.871 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.868+0000 7f0cd3e00700 1 -- 172.21.15.151:0/2262095451 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ccc1acd70 con 0x7f0ccc114710 2024-05-21T22:02:27.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.872+0000 7f0cca000700 1 -- 172.21.15.151:0/2262095451 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7f0cc401b030 con 0x7f0ccc114710 2024-05-21T22:02:27.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.873+0000 7f0cde79db80 1 Processor -- start 2024-05-21T22:02:27.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.873+0000 7f0cde79db80 1 -- start start 2024-05-21T22:02:27.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.873+0000 7f0cca000700 1 --2- 172.21.15.151:0/2262095451 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f0cb40b29f0 0x7f0cb40b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:27.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.873+0000 7f0cde79db80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558eb587d610 0x558eb587d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:27.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.873+0000 7f0cde79db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558eb587df30 0x558eb5877b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:27.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.873+0000 7f0cde79db80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558eb5878140 0x558eb58785c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:27.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.873+0000 7f0cde79db80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x558eb57d5e40 con 0x558eb5878140 2024-05-21T22:02:27.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.873+0000 7f0cde79db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x558eb57c59e0 con 0x558eb587df30 2024-05-21T22:02:27.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.873+0000 7f0cde79db80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x558eb57c9a20 con 0x558eb587d610 2024-05-21T22:02:27.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.873+0000 7f0cca000700 1 -- 172.21.15.151:0/2262095451 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8542+0+0 (secure 0 0 0) 0x7f0cc40e2120 con 0x7f0ccc114710 2024-05-21T22:02:27.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.874+0000 7f0cd0a00700 1 --2- 172.21.15.151:0/2262095451 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f0cb40b29f0 0x7f0cb40b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:27.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.874+0000 7f0cd1400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558eb587d610 0x558eb587d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:27.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.874+0000 7f0cd1400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558eb587d610 0x558eb587d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:54722/0 (socket says 172.21.15.151:54722) 2024-05-21T22:02:27.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.874+0000 7f0cd1400700 1 -- 172.21.15.151:0/1616416330 learned_addr learned my addr 172.21.15.151:0/1616416330 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:02:27.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.874+0000 7f0cd0a00700 1 --2- 172.21.15.151:0/1616416330 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558eb587df30 0x558eb5877b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:27.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.874+0000 7f0cd1e00700 1 --2- 172.21.15.151:0/1616416330 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558eb5878140 0x558eb58785c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:27.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.874+0000 7f0cd1400700 1 -- 172.21.15.151:0/1616416330 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558eb587df30 msgr2=0x558eb5877b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:27.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.874+0000 7f0cd1400700 1 --2- 172.21.15.151:0/1616416330 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558eb587df30 0x558eb5877b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:27.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.874+0000 7f0cd1400700 1 -- 172.21.15.151:0/1616416330 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558eb5878140 msgr2=0x558eb58785c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:27.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.874+0000 7f0cd1400700 1 --2- 172.21.15.151:0/1616416330 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558eb5878140 0x558eb58785c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:27.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.874+0000 7f0cd1400700 1 -- 172.21.15.151:0/1616416330 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558eb578e420 con 0x558eb587d610 2024-05-21T22:02:27.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.875+0000 7f0cd1400700 1 --2- 172.21.15.151:0/1616416330 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558eb587d610 0x558eb587d9f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f0cc0009d80 tx=0x7f0cc0011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9a87c562257a442 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:02:27.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.875+0000 7f0cd0a00700 1 --2- 172.21.15.151:0/2262095451 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f0cb40b29f0 0x7f0cb40b4eb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc1110c0 tx=0x7f0cb80096d0 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:02:27.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.875+0000 7f0cbea00700 1 -- 172.21.15.151:0/1616416330 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc0004230 con 0x558eb587d610 2024-05-21T22:02:27.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.875+0000 7f0cbea00700 1 -- 172.21.15.151:0/1616416330 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cc00043d0 con 0x558eb587d610 2024-05-21T22:02:27.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.875+0000 7f0cbea00700 1 -- 172.21.15.151:0/1616416330 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc0019820 con 0x558eb587d610 2024-05-21T22:02:27.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.876+0000 7f0cde79db80 1 -- 172.21.15.151:0/1616416330 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558eb587d610 msgr2=0x558eb587d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:27.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.876+0000 7f0cde79db80 1 --2- 172.21.15.151:0/1616416330 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558eb587d610 0x558eb587d9f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f0cc0009d80 tx=0x7f0cc0011720 comp rx=0 tx=0).stop 2024-05-21T22:02:27.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.876+0000 7f0cde79db80 1 -- 172.21.15.151:0/1616416330 shutdown_connections 2024-05-21T22:02:27.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.876+0000 7f0cde79db80 1 --2- 172.21.15.151:0/1616416330 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558eb5878140 0x558eb58785c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:27.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.876+0000 7f0cde79db80 1 --2- 172.21.15.151:0/1616416330 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558eb587d610 0x558eb587d9f0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:27.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.876+0000 7f0cde79db80 1 --2- 172.21.15.151:0/1616416330 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558eb587df30 0x558eb5877b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:27.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.876+0000 7f0cde79db80 1 -- 172.21.15.151:0/1616416330 >> 172.21.15.151:0/1616416330 conn(0x558eb57a5be0 msgr2=0x558eb5872690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:02:27.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.876+0000 7f0cde79db80 1 -- 172.21.15.151:0/1616416330 shutdown_connections 2024-05-21T22:02:27.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.876+0000 7f0cde79db80 1 -- 172.21.15.151:0/1616416330 wait complete. 2024-05-21T22:02:27.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.876+0000 7f0cde79db80 1 -- start start 2024-05-21T22:02:27.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.877+0000 7f0cde79db80 1 Processor -- start 2024-05-21T22:02:27.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.877+0000 7f0cde79db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558eb5878140 0x558eb5883b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:27.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.877+0000 7f0cde79db80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558eb587d610 0x558eb5880f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:27.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.877+0000 7f0cde79db80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558eb587df30 0x558eb5881470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:27.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.877+0000 7f0cde79db80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x558eb56cbfa0 con 0x558eb587d610 2024-05-21T22:02:27.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.877+0000 7f0cde79db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x558eb5766760 con 0x558eb5878140 2024-05-21T22:02:27.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.877+0000 7f0cde79db80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x558eb58849e0 con 0x558eb587df30 2024-05-21T22:02:27.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.877+0000 7f0cde79db80 10 client.0 _ll_register_callbacks cb 0x558eb576fc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:02:27.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.878+0000 7f0cd0a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558eb587d610 0x558eb5880f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:27.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.878+0000 7f0cd1400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558eb5878140 0x558eb5883b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:27.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.878+0000 7f0cd0a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558eb587d610 0x558eb5880f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:58180/0 (socket says 172.21.15.151:58180) 2024-05-21T22:02:27.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.878+0000 7f0cd0a00700 1 -- 172.21.15.151:0/590364596 learned_addr learned my addr 172.21.15.151:0/590364596 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:02:27.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.878+0000 7f0cd1e00700 1 --2- 172.21.15.151:0/590364596 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558eb587df30 0x558eb5881470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:27.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.878+0000 7f0cd0a00700 1 -- 172.21.15.151:0/590364596 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558eb587df30 msgr2=0x558eb5881470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:27.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.878+0000 7f0cd0a00700 1 --2- 172.21.15.151:0/590364596 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558eb587df30 0x558eb5881470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:27.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.878+0000 7f0cd0a00700 1 -- 172.21.15.151:0/590364596 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558eb5878140 msgr2=0x558eb5883b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:27.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.878+0000 7f0cd0a00700 1 --2- 172.21.15.151:0/590364596 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558eb5878140 0x558eb5883b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:27.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.878+0000 7f0cd0a00700 1 -- 172.21.15.151:0/590364596 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558eb57158d0 con 0x558eb587d610 2024-05-21T22:02:27.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.878+0000 7f0cd1400700 1 --2- 172.21.15.151:0/590364596 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558eb5878140 0x558eb5883b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:02:27.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.878+0000 7f0cd0a00700 1 --2- 172.21.15.151:0/590364596 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558eb587d610 0x558eb5880f30 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f0cb8003930 tx=0x7f0cb8009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54bb759aeb5fad51 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:02:27.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.878+0000 7f0cb3400700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-21T22:02:27.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.879+0000 7f0cb3400700 1 -- 172.21.15.151:0/590364596 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb8037060 con 0x558eb587d610 2024-05-21T22:02:27.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.879+0000 7f0cb3400700 1 -- 172.21.15.151:0/590364596 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8542+0+0 (secure 0 0 0) 0x7f0cb800f730 con 0x558eb587d610 2024-05-21T22:02:27.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.879+0000 7f0cde79db80 1 -- 172.21.15.151:0/590364596 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0cc00043d0 con 0x558eb587d610 2024-05-21T22:02:27.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.879+0000 7f0cb3400700 1 -- 172.21.15.151:0/590364596 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0cb800f930 con 0x558eb587d610 2024-05-21T22:02:27.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.879+0000 7f0cde79db80 10 client.16728 fetch_fsmap learned FSMap version 29 2024-05-21T22:02:27.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.879+0000 7f0cde79db80 1 -- 172.21.15.151:0/590364596 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558eb5796060 con 0x558eb587d610 2024-05-21T22:02:27.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.879+0000 7f0cb3400700 1 -- 172.21.15.151:0/590364596 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cc00043d0 con 0x558eb587d610 2024-05-21T22:02:27.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.880+0000 7f0cb3400700 1 -- 172.21.15.151:0/590364596 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f0cb8027040 con 0x558eb587d610 2024-05-21T22:02:27.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.880+0000 7f0cde79db80 10 client.16728 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:02:27.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.880+0000 7f0cde79db80 10 client.16728 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T22:02:27.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.880+0000 7f0cde79db80 20 client.16728 populate_metadata read hostname 'smithi151' 2024-05-21T22:02:27.884 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.880+0000 7f0cde79db80 1 --2- 172.21.15.151:0/590364596 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x558eb5891450 0x558eb5893860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:27.884 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.880+0000 7f0cde79db80 4 client.16728 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T22:02:27.884 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.880+0000 7f0cde79db80 1 -- 172.21.15.151:0/590364596 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558eb57d7d90 con 0x558eb5891450 2024-05-21T22:02:27.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.880+0000 7f0cd1400700 1 --2- 172.21.15.151:0/590364596 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x558eb5891450 0x558eb5893860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:27.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.881+0000 7f0cd1400700 1 --2- 172.21.15.151:0/590364596 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x558eb5891450 0x558eb5893860 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2154959c6b02b894 server_cookie=69abe051ff7a8257 in_seq=0 out_seq=0 2024-05-21T22:02:27.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.881+0000 7f0cb3400700 10 client.16728 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T22:02:27.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.882+0000 7f0cb3400700 1 -- 172.21.15.151:0/590364596 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x558eb57d7d90 con 0x558eb5891450 2024-05-21T22:02:27.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.882+0000 7f0cb3400700 10 client.16728 handle_command_reply: tid=0 2024-05-21T22:02:27.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.882+0000 7f0cde79db80 1 client.16728 shutdown 2024-05-21T22:02:27.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.882+0000 7f0cde79db80 1 -- 172.21.15.151:0/590364596 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558eb587d610 msgr2=0x558eb5880f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:27.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.882+0000 7f0cde79db80 1 --2- 172.21.15.151:0/590364596 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558eb587d610 0x558eb5880f30 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f0cb8003930 tx=0x7f0cb8009ca0 comp rx=0 tx=0).stop 2024-05-21T22:02:27.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.882+0000 7f0cde79db80 1 -- 172.21.15.151:0/590364596 shutdown_connections 2024-05-21T22:02:27.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.882+0000 7f0cde79db80 1 --2- 172.21.15.151:0/590364596 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x558eb5891450 0x558eb5893860 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:27.886 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.882+0000 7f0cde79db80 1 --2- 172.21.15.151:0/590364596 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558eb5878140 0x558eb5883b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:27.886 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.882+0000 7f0cde79db80 1 --2- 172.21.15.151:0/590364596 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558eb587df30 0x558eb5881470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:27.886 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.882+0000 7f0cde79db80 1 --2- 172.21.15.151:0/590364596 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558eb587d610 0x558eb5880f30 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:27.886 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.882+0000 7f0cb3400700 0 client.16728 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T22:02:27.886 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.882+0000 7f0cde79db80 1 -- 172.21.15.151:0/590364596 >> 172.21.15.151:0/590364596 conn(0x558eb57a5be0 msgr2=0x558eb5871d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:02:27.886 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.882+0000 7f0cde79db80 1 -- 172.21.15.151:0/590364596 shutdown_connections 2024-05-21T22:02:27.886 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.882+0000 7f0cde79db80 1 -- 172.21.15.151:0/590364596 wait complete. 2024-05-21T22:02:27.886 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.883+0000 7f0cde79db80 20 client.16728 trim_cache size 0 max 16384 2024-05-21T22:02:27.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.899+0000 7f0cde79db80 1 Processor -- start 2024-05-21T22:02:27.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.899+0000 7f0cde79db80 1 -- start start 2024-05-21T22:02:27.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.899+0000 7f0cde79db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558eb5878140 0x558eb5877b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:27.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.899+0000 7f0cde79db80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558eb587d610 0x558eb587ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:27.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.899+0000 7f0cde79db80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558eb587df30 0x558eb58724f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:27.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.899+0000 7f0cde79db80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0ccc1b4410 con 0x558eb587df30 2024-05-21T22:02:27.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.899+0000 7f0cde79db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0ccc1b5f50 con 0x558eb5878140 2024-05-21T22:02:27.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.899+0000 7f0cde79db80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0ccc1b4ae0 con 0x558eb587d610 2024-05-21T22:02:27.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.899+0000 7f0cd1400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558eb5878140 0x558eb5877b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:27.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.900+0000 7f0cd0a00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558eb587d610 0x558eb587ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:27.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.899+0000 7f0cd1e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558eb587df30 0x558eb58724f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:27.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.900+0000 7f0cd1400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558eb5878140 0x558eb5877b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:55286/0 (socket says 172.21.15.151:55286) 2024-05-21T22:02:27.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.900+0000 7f0cd1400700 1 -- 172.21.15.151:0/417999360 learned_addr learned my addr 172.21.15.151:0/417999360 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:02:27.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.900+0000 7f0cd0a00700 1 -- 172.21.15.151:0/417999360 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558eb5878140 msgr2=0x558eb5877b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:27.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.900+0000 7f0cd0a00700 1 --2- 172.21.15.151:0/417999360 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558eb5878140 0x558eb5877b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:27.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.900+0000 7f0cd0a00700 1 -- 172.21.15.151:0/417999360 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558eb587df30 msgr2=0x558eb58724f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:27.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.900+0000 7f0cd0a00700 1 --2- 172.21.15.151:0/417999360 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558eb587df30 0x558eb58724f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:27.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.900+0000 7f0cd0a00700 1 -- 172.21.15.151:0/417999360 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558eb5893ee0 con 0x558eb587d610 2024-05-21T22:02:27.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.900+0000 7f0cd1400700 1 --2- 172.21.15.151:0/417999360 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558eb5878140 0x558eb5877b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:02:27.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.900+0000 7f0cd0a00700 1 --2- 172.21.15.151:0/417999360 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558eb587d610 0x558eb587ea20 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f0cb8003900 tx=0x7f0cb800be90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4d71ff91b8f13c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:02:27.903 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.901+0000 7f0cbe000700 1 -- 172.21.15.151:0/417999360 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb8037040 con 0x558eb587d610 2024-05-21T22:02:27.903 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.901+0000 7f0cbe000700 1 -- 172.21.15.151:0/417999360 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cb8022040 con 0x558eb587d610 2024-05-21T22:02:27.903 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.901+0000 7f0cbe000700 1 -- 172.21.15.151:0/417999360 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb80284c0 con 0x558eb587d610 2024-05-21T22:02:27.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.901+0000 7f0cde79db80 1 -- 172.21.15.151:0/417999360 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558eb587d610 msgr2=0x558eb587ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:27.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.901+0000 7f0cde79db80 1 --2- 172.21.15.151:0/417999360 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558eb587d610 0x558eb587ea20 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f0cb8003900 tx=0x7f0cb800be90 comp rx=0 tx=0).stop 2024-05-21T22:02:27.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.901+0000 7f0cde79db80 1 -- 172.21.15.151:0/417999360 shutdown_connections 2024-05-21T22:02:27.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.901+0000 7f0cde79db80 1 --2- 172.21.15.151:0/417999360 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558eb5878140 0x558eb5877b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:27.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.901+0000 7f0cde79db80 1 --2- 172.21.15.151:0/417999360 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558eb587df30 0x558eb58724f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:27.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.901+0000 7f0cde79db80 1 --2- 172.21.15.151:0/417999360 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558eb587d610 0x558eb587ea20 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:27.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.901+0000 7f0cde79db80 1 -- 172.21.15.151:0/417999360 >> 172.21.15.151:0/417999360 conn(0x558eb57a5be0 msgr2=0x558eb587e310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:02:27.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.902+0000 7f0cde79db80 1 -- 172.21.15.151:0/417999360 shutdown_connections 2024-05-21T22:02:27.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.902+0000 7f0cde79db80 1 -- 172.21.15.151:0/417999360 wait complete. 2024-05-21T22:02:27.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.902+0000 7f0cde79db80 1 -- start start 2024-05-21T22:02:27.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.902+0000 7f0cde79db80 1 Processor -- start 2024-05-21T22:02:27.906 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.903+0000 7f0cde79db80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558eb5878140 0x558eb588a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:27.906 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.903+0000 7f0cde79db80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558eb587d610 0x558eb58986e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:27.906 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.903+0000 7f0cde79db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558eb587df30 0x558eb5898c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:27.906 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.903+0000 7f0cde79db80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0ccc1b5880 con 0x558eb5878140 2024-05-21T22:02:27.906 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.903+0000 7f0cde79db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0ccc1b51b0 con 0x558eb587df30 2024-05-21T22:02:27.906 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.903+0000 7f0cde79db80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0ccc1b23d0 con 0x558eb587d610 2024-05-21T22:02:27.906 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.903+0000 7f0cde79db80 10 client.0 _ll_register_callbacks cb 0x558eb576fc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:02:27.907 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.903+0000 7f0cd1400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558eb5878140 0x558eb588a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:27.907 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.903+0000 7f0cd1400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558eb5878140 0x558eb588a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:58196/0 (socket says 172.21.15.151:58196) 2024-05-21T22:02:27.907 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.903+0000 7f0cd1400700 1 -- 172.21.15.151:0/3246229001 learned_addr learned my addr 172.21.15.151:0/3246229001 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:02:27.907 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.903+0000 7f0cd0a00700 1 --2- 172.21.15.151:0/3246229001 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558eb587d610 0x558eb58986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:27.907 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.903+0000 7f0cd1e00700 1 --2- 172.21.15.151:0/3246229001 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558eb587df30 0x558eb5898c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:27.907 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.904+0000 7f0cd0a00700 1 -- 172.21.15.151:0/3246229001 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558eb587df30 msgr2=0x558eb5898c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:27.907 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.904+0000 7f0cd0a00700 1 --2- 172.21.15.151:0/3246229001 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558eb587df30 0x558eb5898c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:27.907 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.904+0000 7f0cd0a00700 1 -- 172.21.15.151:0/3246229001 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558eb5878140 msgr2=0x558eb588a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:27.907 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.904+0000 7f0cd0a00700 1 --2- 172.21.15.151:0/3246229001 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558eb5878140 0x558eb588a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:27.907 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.904+0000 7f0cd0a00700 1 -- 172.21.15.151:0/3246229001 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558eb5899160 con 0x558eb587d610 2024-05-21T22:02:27.907 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.904+0000 7f0cd0a00700 1 --2- 172.21.15.151:0/3246229001 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558eb587d610 0x558eb58986e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f0cb80095c0 tx=0x7f0cb8004f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=afddbf4aceff6e44 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:02:27.907 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.904+0000 7f0cb3400700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-21T22:02:27.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.904+0000 7f0cb3400700 1 -- 172.21.15.151:0/3246229001 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb8037070 con 0x558eb587d610 2024-05-21T22:02:27.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.904+0000 7f0cde79db80 1 -- 172.21.15.151:0/3246229001 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0cb8022040 con 0x558eb587d610 2024-05-21T22:02:27.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.904+0000 7f0cb3400700 1 -- 172.21.15.151:0/3246229001 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8542+0+0 (secure 0 0 0) 0x7f0cb8032640 con 0x558eb587d610 2024-05-21T22:02:27.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.905+0000 7f0cb3400700 1 -- 172.21.15.151:0/3246229001 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0cb8032840 con 0x558eb587d610 2024-05-21T22:02:27.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.905+0000 7f0cde79db80 10 client.26869 fetch_fsmap learned FSMap version 29 2024-05-21T22:02:27.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.905+0000 7f0cde79db80 1 -- 172.21.15.151:0/3246229001 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558eb58fbcf0 con 0x558eb587d610 2024-05-21T22:02:27.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.905+0000 7f0cb3400700 1 -- 172.21.15.151:0/3246229001 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cb8022040 con 0x558eb587d610 2024-05-21T22:02:27.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.905+0000 7f0cb3400700 1 -- 172.21.15.151:0/3246229001 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f0cb8027070 con 0x558eb587d610 2024-05-21T22:02:27.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.905+0000 7f0cde79db80 10 client.26869 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:02:27.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.905+0000 7f0cde79db80 10 client.26869 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T22:02:27.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.905+0000 7f0cde79db80 20 client.26869 populate_metadata read hostname 'smithi151' 2024-05-21T22:02:27.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.905+0000 7f0cde79db80 1 --2- 172.21.15.151:0/3246229001 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x558eb58fc2b0 0x558eb58fe6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:27.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.905+0000 7f0cde79db80 4 client.26869 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T22:02:27.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.905+0000 7f0cde79db80 1 -- 172.21.15.151:0/3246229001 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558eb57d1570 con 0x558eb58fc2b0 2024-05-21T22:02:27.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.906+0000 7f0cd1400700 1 --2- 172.21.15.151:0/3246229001 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x558eb58fc2b0 0x558eb58fe6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:27.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.906+0000 7f0cd1400700 1 --2- 172.21.15.151:0/3246229001 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x558eb58fc2b0 0x558eb58fe6d0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=385450c26db39ccd server_cookie=29a9f84c05315801 in_seq=0 out_seq=0 2024-05-21T22:02:27.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.906+0000 7f0cb3400700 10 client.26869 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T22:02:27.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.906+0000 7f0cb3400700 1 -- 172.21.15.151:0/3246229001 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558eb57d1570 con 0x558eb58fc2b0 2024-05-21T22:02:27.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.906+0000 7f0cb3400700 10 client.26869 handle_command_reply: tid=0 2024-05-21T22:02:27.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.907+0000 7f0cde79db80 1 client.26869 shutdown 2024-05-21T22:02:27.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.907+0000 7f0cde79db80 1 -- 172.21.15.151:0/3246229001 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558eb587d610 msgr2=0x558eb58986e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:27.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.907+0000 7f0cde79db80 1 --2- 172.21.15.151:0/3246229001 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558eb587d610 0x558eb58986e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f0cb80095c0 tx=0x7f0cb8004f30 comp rx=0 tx=0).stop 2024-05-21T22:02:27.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.907+0000 7f0cde79db80 1 -- 172.21.15.151:0/3246229001 shutdown_connections 2024-05-21T22:02:27.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.907+0000 7f0cde79db80 1 --2- 172.21.15.151:0/3246229001 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x558eb58fc2b0 0x558eb58fe6d0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:27.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.907+0000 7f0cde79db80 1 --2- 172.21.15.151:0/3246229001 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558eb587df30 0x558eb5898c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:27.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.907+0000 7f0cde79db80 1 --2- 172.21.15.151:0/3246229001 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558eb5878140 0x558eb588a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:27.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.907+0000 7f0cb3400700 0 client.26869 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T22:02:27.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.907+0000 7f0cde79db80 1 --2- 172.21.15.151:0/3246229001 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558eb587d610 0x558eb58986e0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:27.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.907+0000 7f0cde79db80 1 -- 172.21.15.151:0/3246229001 >> 172.21.15.151:0/3246229001 conn(0x558eb57a5be0 msgr2=0x558eb5882530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:02:27.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.907+0000 7f0cde79db80 1 -- 172.21.15.151:0/3246229001 shutdown_connections 2024-05-21T22:02:27.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.907+0000 7f0cde79db80 1 -- 172.21.15.151:0/3246229001 wait complete. 2024-05-21T22:02:27.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.907+0000 7f0cde79db80 20 client.26869 trim_cache size 0 max 16384 2024-05-21T22:02:27.911 INFO:teuthology.orchestra.run.smithi151.stdout:[] 2024-05-21T22:02:27.912 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.908+0000 7f0cd3e00700 1 -- 172.21.15.151:0/2262095451 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f0cb40b29f0 msgr2=0x7f0cb40b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:27.912 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.908+0000 7f0cd3e00700 1 --2- 172.21.15.151:0/2262095451 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f0cb40b29f0 0x7f0cb40b4eb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc1110c0 tx=0x7f0cb80096d0 comp rx=0 tx=0).stop 2024-05-21T22:02:27.912 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.908+0000 7f0cd3e00700 1 -- 172.21.15.151:0/2262095451 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0ccc114710 msgr2=0x7f0ccc1a4ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:27.912 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.908+0000 7f0cd3e00700 1 --2- 172.21.15.151:0/2262095451 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0ccc114710 0x7f0ccc1a4ac0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f0cc400ef90 tx=0x7f0cc400c3a0 comp rx=0 tx=0).stop 2024-05-21T22:02:27.912 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.908+0000 7f0cd3e00700 1 -- 172.21.15.151:0/2262095451 shutdown_connections 2024-05-21T22:02:27.912 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.908+0000 7f0cd3e00700 1 --2- 172.21.15.151:0/2262095451 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f0cb40b29f0 0x7f0cb40b4eb0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:27.913 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.908+0000 7f0cd3e00700 1 --2- 172.21.15.151:0/2262095451 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0ccc114710 0x7f0ccc1a4ac0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:27.913 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.908+0000 7f0cd3e00700 1 --2- 172.21.15.151:0/2262095451 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ccc075770 0x7f0ccc1a01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:27.913 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.908+0000 7f0cd3e00700 1 --2- 172.21.15.151:0/2262095451 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0ccc076090 0x7f0ccc1a0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:27.913 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.908+0000 7f0cd3e00700 1 -- 172.21.15.151:0/2262095451 >> 172.21.15.151:0/2262095451 conn(0x7f0ccc07a6a0 msgr2=0x7f0ccc1134d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:02:27.913 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.908+0000 7f0cd3e00700 1 -- 172.21.15.151:0/2262095451 shutdown_connections 2024-05-21T22:02:27.913 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:27.908+0000 7f0cd3e00700 1 -- 172.21.15.151:0/2262095451 wait complete. 2024-05-21T22:02:29.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:02:29 smithi184 ceph-mon[36942]: pgmap v211: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-05-21T22:02:29.743 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:02:29 smithi151 ceph-mon[31623]: pgmap v211: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-05-21T22:02:29.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:02:29 smithi164 ceph-mon[36269]: pgmap v211: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-05-21T22:02:29.932 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-21T22:02:29.932 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-21T22:02:30.394 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:02:31.596 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:02:31 smithi151 ceph-mon[31623]: pgmap v212: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:02:31.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:02:31 smithi184 ceph-mon[36942]: pgmap v212: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:02:31.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:02:31 smithi164 ceph-mon[36269]: pgmap v212: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:02:32.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.567+0000 7f72ef600700 1 -- 172.21.15.151:0/3648625130 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f72e810b760 msgr2=0x7f72e810bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:32.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.567+0000 7f72ef600700 1 --2- 172.21.15.151:0/3648625130 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f72e810b760 0x7f72e810bb40 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f72dc0098f0 tx=0x7f72dc009c00 comp rx=0 tx=0).stop 2024-05-21T22:02:32.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.567+0000 7f72ef600700 1 -- 172.21.15.151:0/3648625130 shutdown_connections 2024-05-21T22:02:32.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.567+0000 7f72ef600700 1 --2- 172.21.15.151:0/3648625130 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f72e810b760 0x7f72e810bb40 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:32.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.567+0000 7f72ef600700 1 --2- 172.21.15.151:0/3648625130 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72e8106120 0x7f72e8112630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:32.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.567+0000 7f72ef600700 1 --2- 172.21.15.151:0/3648625130 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f72e8105760 0x7f72e8105be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:32.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.567+0000 7f72ef600700 1 -- 172.21.15.151:0/3648625130 >> 172.21.15.151:0/3648625130 conn(0x7f72e8101250 msgr2=0x7f72e8103670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:02:32.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.567+0000 7f72ef600700 1 -- 172.21.15.151:0/3648625130 shutdown_connections 2024-05-21T22:02:32.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.567+0000 7f72ef600700 1 -- 172.21.15.151:0/3648625130 wait complete. 2024-05-21T22:02:32.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.569+0000 7f72ef600700 1 Processor -- start 2024-05-21T22:02:32.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.569+0000 7f72ef600700 1 -- start start 2024-05-21T22:02:32.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.570+0000 7f72ef600700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72e8105760 0x7f72e819bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:32.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.570+0000 7f72ef600700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f72e8106120 0x7f72e819c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:32.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.570+0000 7f72ef600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f72e810b760 0x7f72e81a0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:32.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.570+0000 7f72ef600700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f72e8115990 con 0x7f72e810b760 2024-05-21T22:02:32.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.570+0000 7f72ef600700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f72e8115810 con 0x7f72e8106120 2024-05-21T22:02:32.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.570+0000 7f72ef600700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f72e8115b10 con 0x7f72e8105760 2024-05-21T22:02:32.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.570+0000 7f72e7e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f72e8106120 0x7f72e819c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:32.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.570+0000 7f72e7e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f72e8106120 0x7f72e819c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:55318/0 (socket says 172.21.15.151:55318) 2024-05-21T22:02:32.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.570+0000 7f72e7e00700 1 -- 172.21.15.151:0/1360018022 learned_addr learned my addr 172.21.15.151:0/1360018022 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:02:32.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.570+0000 7f72ecc00700 1 --2- 172.21.15.151:0/1360018022 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72e8105760 0x7f72e819bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:32.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.570+0000 7f72ed600700 1 --2- 172.21.15.151:0/1360018022 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f72e810b760 0x7f72e81a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:32.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.570+0000 7f72e7e00700 1 -- 172.21.15.151:0/1360018022 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72e8105760 msgr2=0x7f72e819bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:32.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.570+0000 7f72e7e00700 1 --2- 172.21.15.151:0/1360018022 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72e8105760 0x7f72e819bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:32.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.570+0000 7f72e7e00700 1 -- 172.21.15.151:0/1360018022 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f72e810b760 msgr2=0x7f72e81a0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:32.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.570+0000 7f72e7e00700 1 --2- 172.21.15.151:0/1360018022 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f72e810b760 0x7f72e81a0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:32.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.570+0000 7f72e7e00700 1 -- 172.21.15.151:0/1360018022 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72e81a0eb0 con 0x7f72e8106120 2024-05-21T22:02:32.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.570+0000 7f72ecc00700 1 --2- 172.21.15.151:0/1360018022 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72e8105760 0x7f72e819bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:02:32.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.571+0000 7f72e7e00700 1 --2- 172.21.15.151:0/1360018022 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f72e8106120 0x7f72e819c3e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f72d800d980 tx=0x7f72d800dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:02:32.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.571+0000 7f72e5600700 1 -- 172.21.15.151:0/1360018022 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d8016070 con 0x7f72e8106120 2024-05-21T22:02:32.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.571+0000 7f72e5600700 1 -- 172.21.15.151:0/1360018022 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72d8004830 con 0x7f72e8106120 2024-05-21T22:02:32.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.571+0000 7f72ef600700 1 -- 172.21.15.151:0/1360018022 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72e81a11a0 con 0x7f72e8106120 2024-05-21T22:02:32.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.571+0000 7f72e5600700 1 -- 172.21.15.151:0/1360018022 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d8005540 con 0x7f72e8106120 2024-05-21T22:02:32.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.571+0000 7f72ef600700 1 -- 172.21.15.151:0/1360018022 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72e81a8a20 con 0x7f72e8106120 2024-05-21T22:02:32.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.574+0000 7f72e5600700 1 -- 172.21.15.151:0/1360018022 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7f72d8004220 con 0x7f72e8106120 2024-05-21T22:02:32.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.576+0000 7f72e5600700 1 --2- 172.21.15.151:0/1360018022 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f72d00b2a70 0x7f72d00b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:32.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.576+0000 7f72f9eb3b80 1 Processor -- start 2024-05-21T22:02:32.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.576+0000 7f72e5600700 1 -- 172.21.15.151:0/1360018022 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8542+0+0 (secure 0 0 0) 0x7f72d80e23e0 con 0x7f72e8106120 2024-05-21T22:02:32.578 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.576+0000 7f72f9eb3b80 1 -- start start 2024-05-21T22:02:32.578 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.576+0000 7f72ecc00700 1 --2- 172.21.15.151:0/1360018022 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f72d00b2a70 0x7f72d00b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:32.578 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.577+0000 7f72f9eb3b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c5a80ab690 0x55c5a80aba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:32.578 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.577+0000 7f72f9eb3b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c5a80abfb0 0x55c5a80a5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:32.578 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.577+0000 7f72f9eb3b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c5a80a61c0 0x55c5a80a6640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:32.578 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.577+0000 7f72f9eb3b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c5a8003ec0 con 0x55c5a80ab690 2024-05-21T22:02:32.578 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.577+0000 7f72f9eb3b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55c5a7ff3a60 con 0x55c5a80abfb0 2024-05-21T22:02:32.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.577+0000 7f72f9eb3b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55c5a7ff7aa0 con 0x55c5a80a61c0 2024-05-21T22:02:32.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.577+0000 7f72ed600700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c5a80a61c0 0x55c5a80a6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:32.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.577+0000 7f72e7e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c5a80abfb0 0x55c5a80a5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:32.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.577+0000 7f72ed600700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c5a80a61c0 0x55c5a80a6640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:54776/0 (socket says 172.21.15.151:54776) 2024-05-21T22:02:32.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.577+0000 7f72ed600700 1 -- 172.21.15.151:0/651736876 learned_addr learned my addr 172.21.15.151:0/651736876 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:02:32.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.577+0000 7f72ecc00700 1 --2- 172.21.15.151:0/651736876 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c5a80ab690 0x55c5a80aba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:32.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.578+0000 7f72e7e00700 1 -- 172.21.15.151:0/651736876 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c5a80a61c0 msgr2=0x55c5a80a6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:32.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.578+0000 7f72e7e00700 1 --2- 172.21.15.151:0/651736876 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c5a80a61c0 0x55c5a80a6640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:32.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.578+0000 7f72e7e00700 1 -- 172.21.15.151:0/651736876 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c5a80ab690 msgr2=0x55c5a80aba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:32.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.578+0000 7f72e7e00700 1 --2- 172.21.15.151:0/651736876 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c5a80ab690 0x55c5a80aba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:32.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.578+0000 7f72e7e00700 1 -- 172.21.15.151:0/651736876 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5a7fbc4a0 con 0x55c5a80abfb0 2024-05-21T22:02:32.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.578+0000 7f72ecc00700 1 --2- 172.21.15.151:0/651736876 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c5a80ab690 0x55c5a80aba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T22:02:32.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.578+0000 7f72ed600700 1 --2- 172.21.15.151:0/651736876 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c5a80a61c0 0x55c5a80a6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:02:32.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.578+0000 7f72ecc00700 1 --2- 172.21.15.151:0/1360018022 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f72d00b2a70 0x7f72d00b4f30 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f72dc0098c0 tx=0x7f72dc00bf60 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:02:32.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.578+0000 7f72e7e00700 1 --2- 172.21.15.151:0/651736876 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c5a80abfb0 0x55c5a80a5ba0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f72d80dfac0 tx=0x7f72d80a8ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53e69f35c4240445 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:02:32.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.578+0000 7f72d6000700 1 -- 172.21.15.151:0/651736876 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d80ada00 con 0x55c5a80abfb0 2024-05-21T22:02:32.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.578+0000 7f72d6000700 1 -- 172.21.15.151:0/651736876 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72d80b24b0 con 0x55c5a80abfb0 2024-05-21T22:02:32.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.578+0000 7f72d6000700 1 -- 172.21.15.151:0/651736876 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d8022550 con 0x55c5a80abfb0 2024-05-21T22:02:32.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.579+0000 7f72f9eb3b80 1 -- 172.21.15.151:0/651736876 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c5a80abfb0 msgr2=0x55c5a80a5ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:32.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.579+0000 7f72f9eb3b80 1 --2- 172.21.15.151:0/651736876 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c5a80abfb0 0x55c5a80a5ba0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f72d80dfac0 tx=0x7f72d80a8ed0 comp rx=0 tx=0).stop 2024-05-21T22:02:32.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.579+0000 7f72f9eb3b80 1 -- 172.21.15.151:0/651736876 shutdown_connections 2024-05-21T22:02:32.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.579+0000 7f72f9eb3b80 1 --2- 172.21.15.151:0/651736876 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c5a80a61c0 0x55c5a80a6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:32.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.579+0000 7f72f9eb3b80 1 --2- 172.21.15.151:0/651736876 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c5a80ab690 0x55c5a80aba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:32.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.579+0000 7f72f9eb3b80 1 --2- 172.21.15.151:0/651736876 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c5a80abfb0 0x55c5a80a5ba0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:32.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.579+0000 7f72f9eb3b80 1 -- 172.21.15.151:0/651736876 >> 172.21.15.151:0/651736876 conn(0x55c5a7fd3c60 msgr2=0x55c5a80a0710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:02:32.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.579+0000 7f72f9eb3b80 1 -- 172.21.15.151:0/651736876 shutdown_connections 2024-05-21T22:02:32.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.579+0000 7f72f9eb3b80 1 -- 172.21.15.151:0/651736876 wait complete. 2024-05-21T22:02:32.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.580+0000 7f72f9eb3b80 1 -- start start 2024-05-21T22:02:32.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.580+0000 7f72f9eb3b80 1 Processor -- start 2024-05-21T22:02:32.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.581+0000 7f72f9eb3b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c5a80a61c0 0x55c5a80b1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:32.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.581+0000 7f72f9eb3b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c5a80ab690 0x55c5a80af210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:32.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.581+0000 7f72f9eb3b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c5a80abfb0 0x55c5a80af750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:32.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.581+0000 7f72f9eb3b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c5a7efa020 con 0x55c5a80ab690 2024-05-21T22:02:32.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.581+0000 7f72f9eb3b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55c5a7f947e0 con 0x55c5a80abfb0 2024-05-21T22:02:32.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.581+0000 7f72f9eb3b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55c5a80b2a00 con 0x55c5a80a61c0 2024-05-21T22:02:32.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.581+0000 7f72f9eb3b80 10 client.0 _ll_register_callbacks cb 0x55c5a7f9dcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:02:32.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.581+0000 7f72ecc00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c5a80a61c0 0x55c5a80b1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:32.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.581+0000 7f72ecc00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c5a80a61c0 0x55c5a80b1e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:54782/0 (socket says 172.21.15.151:54782) 2024-05-21T22:02:32.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.581+0000 7f72ecc00700 1 -- 172.21.15.151:0/1332108160 learned_addr learned my addr 172.21.15.151:0/1332108160 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:02:32.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.581+0000 7f72ed600700 1 --2- 172.21.15.151:0/1332108160 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c5a80abfb0 0x55c5a80af750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:32.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.581+0000 7f72e7e00700 1 --2- 172.21.15.151:0/1332108160 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c5a80ab690 0x55c5a80af210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:32.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.582+0000 7f72ecc00700 1 -- 172.21.15.151:0/1332108160 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c5a80abfb0 msgr2=0x55c5a80af750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:32.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.582+0000 7f72ecc00700 1 --2- 172.21.15.151:0/1332108160 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c5a80abfb0 0x55c5a80af750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:32.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.582+0000 7f72ecc00700 1 -- 172.21.15.151:0/1332108160 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c5a80ab690 msgr2=0x55c5a80af210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:32.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.582+0000 7f72ecc00700 1 --2- 172.21.15.151:0/1332108160 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c5a80ab690 0x55c5a80af210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:32.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.582+0000 7f72ecc00700 1 -- 172.21.15.151:0/1332108160 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5a7f43950 con 0x55c5a80a61c0 2024-05-21T22:02:32.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.582+0000 7f72ecc00700 1 --2- 172.21.15.151:0/1332108160 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c5a80a61c0 0x55c5a80b1e20 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f72dc0098f0 tx=0x7f72dc009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b684bc29aeaaaa8d server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:02:32.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.582+0000 7f72caa00700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-21T22:02:32.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.582+0000 7f72caa00700 1 -- 172.21.15.151:0/1332108160 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72dc032060 con 0x55c5a80a61c0 2024-05-21T22:02:32.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.582+0000 7f72f9eb3b80 1 -- 172.21.15.151:0/1332108160 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72d80b24b0 con 0x55c5a80a61c0 2024-05-21T22:02:32.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.582+0000 7f72caa00700 1 -- 172.21.15.151:0/1332108160 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8542+0+0 (secure 0 0 0) 0x7f72dc0055e0 con 0x55c5a80a61c0 2024-05-21T22:02:32.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.583+0000 7f72caa00700 1 -- 172.21.15.151:0/1332108160 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f72dc029040 con 0x55c5a80a61c0 2024-05-21T22:02:32.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.583+0000 7f72f9eb3b80 10 client.26881 fetch_fsmap learned FSMap version 29 2024-05-21T22:02:32.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.583+0000 7f72f9eb3b80 1 -- 172.21.15.151:0/1332108160 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c5a7fc40e0 con 0x55c5a80a61c0 2024-05-21T22:02:32.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.583+0000 7f72caa00700 1 -- 172.21.15.151:0/1332108160 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72d80b24b0 con 0x55c5a80a61c0 2024-05-21T22:02:32.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.583+0000 7f72caa00700 1 -- 172.21.15.151:0/1332108160 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f72dc024050 con 0x55c5a80a61c0 2024-05-21T22:02:32.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.583+0000 7f72f9eb3b80 10 client.26881 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:02:32.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.583+0000 7f72f9eb3b80 10 client.26881 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T22:02:32.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.583+0000 7f72f9eb3b80 20 client.26881 populate_metadata read hostname 'smithi151' 2024-05-21T22:02:32.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.583+0000 7f72f9eb3b80 1 --2- 172.21.15.151:0/1332108160 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55c5a80bf6a0 0x55c5a80c1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:32.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.583+0000 7f72f9eb3b80 4 client.26881 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T22:02:32.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.583+0000 7f72f9eb3b80 1 -- 172.21.15.151:0/1332108160 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c5a8005e10 con 0x55c5a80bf6a0 2024-05-21T22:02:32.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.584+0000 7f72e7e00700 1 --2- 172.21.15.151:0/1332108160 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55c5a80bf6a0 0x55c5a80c1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:32.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.584+0000 7f72e7e00700 1 --2- 172.21.15.151:0/1332108160 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55c5a80bf6a0 0x55c5a80c1ab0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d07805a92d9a842d server_cookie=826d78beb870b3e5 in_seq=0 out_seq=0 2024-05-21T22:02:32.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.584+0000 7f72caa00700 10 client.26881 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T22:02:32.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.585+0000 7f72caa00700 1 -- 172.21.15.151:0/1332108160 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55c5a8005e10 con 0x55c5a80bf6a0 2024-05-21T22:02:32.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.585+0000 7f72caa00700 10 client.26881 handle_command_reply: tid=0 2024-05-21T22:02:32.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.585+0000 7f72f9eb3b80 1 client.26881 shutdown 2024-05-21T22:02:32.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.585+0000 7f72f9eb3b80 1 -- 172.21.15.151:0/1332108160 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c5a80a61c0 msgr2=0x55c5a80b1e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:32.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.585+0000 7f72f9eb3b80 1 --2- 172.21.15.151:0/1332108160 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c5a80a61c0 0x55c5a80b1e20 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f72dc0098f0 tx=0x7f72dc009c00 comp rx=0 tx=0).stop 2024-05-21T22:02:32.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.585+0000 7f72f9eb3b80 1 -- 172.21.15.151:0/1332108160 shutdown_connections 2024-05-21T22:02:32.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.585+0000 7f72f9eb3b80 1 --2- 172.21.15.151:0/1332108160 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55c5a80bf6a0 0x55c5a80c1ab0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:32.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.585+0000 7f72f9eb3b80 1 --2- 172.21.15.151:0/1332108160 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c5a80abfb0 0x55c5a80af750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:32.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.585+0000 7f72f9eb3b80 1 --2- 172.21.15.151:0/1332108160 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c5a80a61c0 0x55c5a80b1e20 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:32.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.585+0000 7f72f9eb3b80 1 --2- 172.21.15.151:0/1332108160 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c5a80ab690 0x55c5a80af210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:32.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.585+0000 7f72f9eb3b80 1 -- 172.21.15.151:0/1332108160 >> 172.21.15.151:0/1332108160 conn(0x55c5a7fd3c60 msgr2=0x55c5a809fdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:02:32.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.585+0000 7f72caa00700 0 client.26881 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T22:02:32.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.586+0000 7f72f9eb3b80 1 -- 172.21.15.151:0/1332108160 shutdown_connections 2024-05-21T22:02:32.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.586+0000 7f72f9eb3b80 1 -- 172.21.15.151:0/1332108160 wait complete. 2024-05-21T22:02:32.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.586+0000 7f72f9eb3b80 20 client.26881 trim_cache size 0 max 16384 2024-05-21T22:02:32.609 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.608+0000 7f72f9eb3b80 1 Processor -- start 2024-05-21T22:02:32.609 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.608+0000 7f72f9eb3b80 1 -- start start 2024-05-21T22:02:32.610 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.608+0000 7f72f9eb3b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c5a80a61c0 0x55c5a80a5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:32.610 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.608+0000 7f72f9eb3b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c5a80ab690 0x55c5a80b1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:32.610 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.608+0000 7f72f9eb3b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c5a80abfb0 0x55c5a80ae390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:32.610 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.608+0000 7f72f9eb3b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f72e81b0110 con 0x55c5a80a61c0 2024-05-21T22:02:32.611 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.608+0000 7f72f9eb3b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f72e81b1c50 con 0x55c5a80ab690 2024-05-21T22:02:32.611 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.608+0000 7f72f9eb3b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f72e81b07e0 con 0x55c5a80abfb0 2024-05-21T22:02:32.611 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.609+0000 7f72e7e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c5a80ab690 0x55c5a80b1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:32.611 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.609+0000 7f72ecc00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c5a80a61c0 0x55c5a80a5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:32.611 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.609+0000 7f72e7e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c5a80ab690 0x55c5a80b1660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:55342/0 (socket says 172.21.15.151:55342) 2024-05-21T22:02:32.611 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.609+0000 7f72e7e00700 1 -- 172.21.15.151:0/4035937080 learned_addr learned my addr 172.21.15.151:0/4035937080 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:02:32.611 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.609+0000 7f72ecc00700 1 --2- 172.21.15.151:0/4035937080 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c5a80a61c0 0x55c5a80a5ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:58240/0 (socket says 172.21.15.151:58240) 2024-05-21T22:02:32.611 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.609+0000 7f72ed600700 1 --2- 172.21.15.151:0/4035937080 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c5a80abfb0 0x55c5a80ae390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:32.612 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.609+0000 7f72ecc00700 1 -- 172.21.15.151:0/4035937080 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c5a80abfb0 msgr2=0x55c5a80ae390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:32.612 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.609+0000 7f72ecc00700 1 --2- 172.21.15.151:0/4035937080 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c5a80abfb0 0x55c5a80ae390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:32.612 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.609+0000 7f72ecc00700 1 -- 172.21.15.151:0/4035937080 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c5a80ab690 msgr2=0x55c5a80b1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:32.613 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.609+0000 7f72ecc00700 1 --2- 172.21.15.151:0/4035937080 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c5a80ab690 0x55c5a80b1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:32.613 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.609+0000 7f72ecc00700 1 -- 172.21.15.151:0/4035937080 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5a80a1520 con 0x55c5a80a61c0 2024-05-21T22:02:32.613 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.609+0000 7f72e7e00700 1 --2- 172.21.15.151:0/4035937080 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c5a80ab690 0x55c5a80b1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:02:32.613 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.610+0000 7f72ecc00700 1 --2- 172.21.15.151:0/4035937080 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c5a80a61c0 0x55c5a80a5ba0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f72dc00b8f0 tx=0x7f72dc005440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8fcff42efc4e3e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:02:32.613 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.610+0000 7f72d5600700 1 -- 172.21.15.151:0/4035937080 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72dc032040 con 0x55c5a80a61c0 2024-05-21T22:02:32.613 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.610+0000 7f72d5600700 1 -- 172.21.15.151:0/4035937080 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72dc00b560 con 0x55c5a80a61c0 2024-05-21T22:02:32.613 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.610+0000 7f72d5600700 1 -- 172.21.15.151:0/4035937080 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72dc003e50 con 0x55c5a80a61c0 2024-05-21T22:02:32.613 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.611+0000 7f72f9eb3b80 1 -- 172.21.15.151:0/4035937080 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c5a80a61c0 msgr2=0x55c5a80a5ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:32.613 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.611+0000 7f72f9eb3b80 1 --2- 172.21.15.151:0/4035937080 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c5a80a61c0 0x55c5a80a5ba0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f72dc00b8f0 tx=0x7f72dc005440 comp rx=0 tx=0).stop 2024-05-21T22:02:32.613 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.611+0000 7f72f9eb3b80 1 -- 172.21.15.151:0/4035937080 shutdown_connections 2024-05-21T22:02:32.613 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.611+0000 7f72f9eb3b80 1 --2- 172.21.15.151:0/4035937080 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c5a80abfb0 0x55c5a80ae390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:32.613 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.611+0000 7f72f9eb3b80 1 --2- 172.21.15.151:0/4035937080 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c5a80a61c0 0x55c5a80a5ba0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:32.613 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.611+0000 7f72f9eb3b80 1 --2- 172.21.15.151:0/4035937080 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c5a80ab690 0x55c5a80b1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:32.613 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.611+0000 7f72f9eb3b80 1 -- 172.21.15.151:0/4035937080 >> 172.21.15.151:0/4035937080 conn(0x55c5a7fd3c60 msgr2=0x55c5a80ac390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:02:32.613 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.611+0000 7f72f9eb3b80 1 -- 172.21.15.151:0/4035937080 shutdown_connections 2024-05-21T22:02:32.614 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.611+0000 7f72f9eb3b80 1 -- 172.21.15.151:0/4035937080 wait complete. 2024-05-21T22:02:32.614 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.611+0000 7f72f9eb3b80 1 -- start start 2024-05-21T22:02:32.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.612+0000 7f72f9eb3b80 1 Processor -- start 2024-05-21T22:02:32.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.612+0000 7f72f9eb3b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c5a80a61c0 0x55c5a80b7c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:32.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.612+0000 7f72f9eb3b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c5a80ab690 0x55c5a80b81d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:32.616 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.612+0000 7f72f9eb3b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c5a80abfb0 0x55c5a80b4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:32.616 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.612+0000 7f72f9eb3b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f72e81b1580 con 0x55c5a80a61c0 2024-05-21T22:02:32.616 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.612+0000 7f72f9eb3b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f72e81b0eb0 con 0x55c5a80ab690 2024-05-21T22:02:32.616 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.612+0000 7f72f9eb3b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f72e81ae0d0 con 0x55c5a80abfb0 2024-05-21T22:02:32.616 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.612+0000 7f72f9eb3b80 10 client.0 _ll_register_callbacks cb 0x55c5a7f9dcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:02:32.616 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.612+0000 7f72ecc00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c5a80a61c0 0x55c5a80b7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:32.616 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.612+0000 7f72ecc00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c5a80a61c0 0x55c5a80b7c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:58252/0 (socket says 172.21.15.151:58252) 2024-05-21T22:02:32.616 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.612+0000 7f72ecc00700 1 -- 172.21.15.151:0/188399071 learned_addr learned my addr 172.21.15.151:0/188399071 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:02:32.616 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.613+0000 7f72ed600700 1 --2- 172.21.15.151:0/188399071 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c5a80abfb0 0x55c5a80b4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:32.616 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.613+0000 7f72e7e00700 1 --2- 172.21.15.151:0/188399071 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c5a80ab690 0x55c5a80b81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:32.616 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.613+0000 7f72ecc00700 1 -- 172.21.15.151:0/188399071 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c5a80abfb0 msgr2=0x55c5a80b4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:32.616 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.613+0000 7f72ecc00700 1 --2- 172.21.15.151:0/188399071 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c5a80abfb0 0x55c5a80b4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:32.616 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.613+0000 7f72ecc00700 1 -- 172.21.15.151:0/188399071 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c5a80ab690 msgr2=0x55c5a80b81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:32.616 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.613+0000 7f72ecc00700 1 --2- 172.21.15.151:0/188399071 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c5a80ab690 0x55c5a80b81d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:32.617 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.613+0000 7f72ecc00700 1 -- 172.21.15.151:0/188399071 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5a80b51a0 con 0x55c5a80a61c0 2024-05-21T22:02:32.617 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.613+0000 7f72ecc00700 1 --2- 172.21.15.151:0/188399071 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c5a80a61c0 0x55c5a80b7c90 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f72dc00fb60 tx=0x7f72dc00b780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d742dae56f5a2422 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:02:32.617 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.613+0000 7f72caa00700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-21T22:02:32.617 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.614+0000 7f72caa00700 1 -- 172.21.15.151:0/188399071 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72dc032030 con 0x55c5a80a61c0 2024-05-21T22:02:32.617 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.614+0000 7f72f9eb3b80 1 -- 172.21.15.151:0/188399071 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72dc00b560 con 0x55c5a80a61c0 2024-05-21T22:02:32.617 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.614+0000 7f72caa00700 1 -- 172.21.15.151:0/188399071 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8542+0+0 (secure 0 0 0) 0x7f72dc003680 con 0x55c5a80a61c0 2024-05-21T22:02:32.617 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.614+0000 7f72caa00700 1 -- 172.21.15.151:0/188399071 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f72dc003880 con 0x55c5a80a61c0 2024-05-21T22:02:32.617 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.614+0000 7f72f9eb3b80 10 client.16752 fetch_fsmap learned FSMap version 29 2024-05-21T22:02:32.617 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.614+0000 7f72f9eb3b80 1 -- 172.21.15.151:0/188399071 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c5a812a980 con 0x55c5a80a61c0 2024-05-21T22:02:32.617 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.614+0000 7f72caa00700 1 -- 172.21.15.151:0/188399071 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72dc00b560 con 0x55c5a80a61c0 2024-05-21T22:02:32.617 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.614+0000 7f72caa00700 1 -- 172.21.15.151:0/188399071 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f72dc03fa40 con 0x55c5a80a61c0 2024-05-21T22:02:32.619 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.614+0000 7f72f9eb3b80 10 client.16752 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:02:32.619 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.615+0000 7f72f9eb3b80 10 client.16752 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T22:02:32.619 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.615+0000 7f72f9eb3b80 20 client.16752 populate_metadata read hostname 'smithi151' 2024-05-21T22:02:32.619 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.615+0000 7f72f9eb3b80 1 --2- 172.21.15.151:0/188399071 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55c5a812aea0 0x55c5a812d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:32.619 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.615+0000 7f72f9eb3b80 4 client.16752 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-21T22:02:32.619 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.615+0000 7f72f9eb3b80 1 -- 172.21.15.151:0/188399071 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c5a7fff5f0 con 0x55c5a812aea0 2024-05-21T22:02:32.619 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.615+0000 7f72e7e00700 1 --2- 172.21.15.151:0/188399071 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55c5a812aea0 0x55c5a812d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:32.619 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.615+0000 7f72e7e00700 1 --2- 172.21.15.151:0/188399071 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55c5a812aea0 0x55c5a812d2c0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be3b232a4019ba1b server_cookie=6788d18d70c9add9 in_seq=0 out_seq=0 2024-05-21T22:02:32.619 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.615+0000 7f72caa00700 10 client.16752 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T22:02:32.619 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.616+0000 7f72caa00700 1 -- 172.21.15.151:0/188399071 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c5a7fff5f0 con 0x55c5a812aea0 2024-05-21T22:02:32.619 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.616+0000 7f72caa00700 10 client.16752 handle_command_reply: tid=0 2024-05-21T22:02:32.620 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.617+0000 7f72f9eb3b80 1 client.16752 shutdown 2024-05-21T22:02:32.620 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.617+0000 7f72f9eb3b80 1 -- 172.21.15.151:0/188399071 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c5a80a61c0 msgr2=0x55c5a80b7c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:32.620 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.617+0000 7f72f9eb3b80 1 --2- 172.21.15.151:0/188399071 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c5a80a61c0 0x55c5a80b7c90 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f72dc00fb60 tx=0x7f72dc00b780 comp rx=0 tx=0).stop 2024-05-21T22:02:32.620 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.617+0000 7f72f9eb3b80 1 -- 172.21.15.151:0/188399071 shutdown_connections 2024-05-21T22:02:32.620 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.617+0000 7f72f9eb3b80 1 --2- 172.21.15.151:0/188399071 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c5a80abfb0 0x55c5a80b4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:32.620 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.617+0000 7f72f9eb3b80 1 --2- 172.21.15.151:0/188399071 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c5a80a61c0 0x55c5a80b7c90 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:32.620 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.617+0000 7f72f9eb3b80 1 --2- 172.21.15.151:0/188399071 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55c5a812aea0 0x55c5a812d2c0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:32.620 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.617+0000 7f72f9eb3b80 1 --2- 172.21.15.151:0/188399071 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55c5a80ab690 0x55c5a80b81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:32.620 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.617+0000 7f72f9eb3b80 1 -- 172.21.15.151:0/188399071 >> 172.21.15.151:0/188399071 conn(0x55c5a7fd3c60 msgr2=0x55c5a80a4b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:02:32.620 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.617+0000 7f72caa00700 0 client.16752 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T22:02:32.620 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.617+0000 7f72f9eb3b80 1 -- 172.21.15.151:0/188399071 shutdown_connections 2024-05-21T22:02:32.620 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.617+0000 7f72f9eb3b80 1 -- 172.21.15.151:0/188399071 wait complete. 2024-05-21T22:02:32.620 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.617+0000 7f72f9eb3b80 20 client.16752 trim_cache size 0 max 16384 2024-05-21T22:02:32.621 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-05-21T22:02:32.621 INFO:teuthology.orchestra.run.smithi151.stdout: "return_code": 0, 2024-05-21T22:02:32.621 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_tag": "5df3acd1-afad-45c5-8a12-f79b68a19934", 2024-05-21T22:02:32.621 INFO:teuthology.orchestra.run.smithi151.stdout: "mode": "asynchronous" 2024-05-21T22:02:32.621 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-05-21T22:02:32.621 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.618+0000 7f72ef600700 1 -- 172.21.15.151:0/1360018022 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f72d00b2a70 msgr2=0x7f72d00b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:32.621 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.618+0000 7f72ef600700 1 --2- 172.21.15.151:0/1360018022 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f72d00b2a70 0x7f72d00b4f30 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f72dc0098c0 tx=0x7f72dc00bf60 comp rx=0 tx=0).stop 2024-05-21T22:02:32.622 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.618+0000 7f72ef600700 1 -- 172.21.15.151:0/1360018022 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f72e8106120 msgr2=0x7f72e819c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:32.622 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.618+0000 7f72ef600700 1 --2- 172.21.15.151:0/1360018022 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f72e8106120 0x7f72e819c3e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f72d800d980 tx=0x7f72d800dc90 comp rx=0 tx=0).stop 2024-05-21T22:02:32.622 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.618+0000 7f72ef600700 1 -- 172.21.15.151:0/1360018022 shutdown_connections 2024-05-21T22:02:32.622 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.618+0000 7f72ef600700 1 --2- 172.21.15.151:0/1360018022 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f72d00b2a70 0x7f72d00b4f30 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:32.622 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.618+0000 7f72ef600700 1 --2- 172.21.15.151:0/1360018022 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f72e810b760 0x7f72e81a0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:32.622 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.618+0000 7f72ef600700 1 --2- 172.21.15.151:0/1360018022 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72e8105760 0x7f72e819bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:32.622 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.618+0000 7f72ef600700 1 --2- 172.21.15.151:0/1360018022 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f72e8106120 0x7f72e819c3e0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:32.622 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.618+0000 7f72ef600700 1 -- 172.21.15.151:0/1360018022 >> 172.21.15.151:0/1360018022 conn(0x7f72e8101250 msgr2=0x7f72e8101b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:02:32.622 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.618+0000 7f72ef600700 1 -- 172.21.15.151:0/1360018022 shutdown_connections 2024-05-21T22:02:32.622 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:32.618+0000 7f72ef600700 1 -- 172.21.15.151:0/1360018022 wait complete. 2024-05-21T22:02:33.539 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-21T22:02:33.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:02:33 smithi184 ceph-mon[36942]: pgmap v213: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T22:02:33.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:02:33 smithi184 ceph-mon[36942]: scrub queued for path: / 2024-05-21T22:02:33.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:02:33 smithi184 ceph-mon[36942]: scrub summary: idle+waiting paths [/] 2024-05-21T22:02:33.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:02:33 smithi184 ceph-mon[36942]: scrub summary: active paths [/] 2024-05-21T22:02:33.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:02:33 smithi184 ceph-mon[36942]: scrub summary: idle+waiting paths [/] 2024-05-21T22:02:33.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:02:33 smithi164 ceph-mon[36269]: pgmap v213: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T22:02:33.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:02:33 smithi164 ceph-mon[36269]: scrub queued for path: / 2024-05-21T22:02:33.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:02:33 smithi164 ceph-mon[36269]: scrub summary: idle+waiting paths [/] 2024-05-21T22:02:33.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:02:33 smithi164 ceph-mon[36269]: scrub summary: active paths [/] 2024-05-21T22:02:33.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:02:33 smithi164 ceph-mon[36269]: scrub summary: idle+waiting paths [/] 2024-05-21T22:02:33.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:02:33 smithi151 ceph-mon[31623]: pgmap v213: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T22:02:33.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:02:33 smithi151 ceph-mon[31623]: scrub queued for path: / 2024-05-21T22:02:33.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:02:33 smithi151 ceph-mon[31623]: scrub summary: idle+waiting paths [/] 2024-05-21T22:02:33.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:02:33 smithi151 ceph-mon[31623]: scrub summary: active paths [/] 2024-05-21T22:02:33.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:02:33 smithi151 ceph-mon[31623]: scrub summary: idle+waiting paths [/] 2024-05-21T22:02:33.987 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:02:34.701 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:02:34 smithi151 ceph-mon[31623]: scrub summary: idle 2024-05-21T22:02:34.735 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:02:34 smithi184 ceph-mon[36942]: scrub summary: idle 2024-05-21T22:02:34.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:02:34 smithi164 ceph-mon[36269]: scrub summary: idle 2024-05-21T22:02:35.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:02:35 smithi184 ceph-mon[36942]: pgmap v214: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:02:35.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:02:35 smithi164 ceph-mon[36269]: pgmap v214: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:02:35.851 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:02:35 smithi151 ceph-mon[31623]: pgmap v214: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:02:36.333 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.331+0000 7f2d21600700 1 -- 172.21.15.151:0/3782302886 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d1c103490 msgr2=0x7f2d1c103910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:36.334 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.331+0000 7f2d21600700 1 --2- 172.21.15.151:0/3782302886 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d1c103490 0x7f2d1c103910 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f2d080099c0 tx=0x7f2d08009cd0 comp rx=0 tx=0).stop 2024-05-21T22:02:36.334 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.332+0000 7f2d21600700 1 -- 172.21.15.151:0/3782302886 shutdown_connections 2024-05-21T22:02:36.334 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.332+0000 7f2d21600700 1 --2- 172.21.15.151:0/3782302886 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d1c103e50 0x7f2d1c110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:36.334 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.332+0000 7f2d21600700 1 --2- 172.21.15.151:0/3782302886 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d1c109540 0x7f2d1c109920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:36.334 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.332+0000 7f2d21600700 1 --2- 172.21.15.151:0/3782302886 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d1c103490 0x7f2d1c103910 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:36.334 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.332+0000 7f2d21600700 1 -- 172.21.15.151:0/3782302886 >> 172.21.15.151:0/3782302886 conn(0x7f2d1c0fefc0 msgr2=0x7f2d1c1013e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:02:36.334 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.332+0000 7f2d21600700 1 -- 172.21.15.151:0/3782302886 shutdown_connections 2024-05-21T22:02:36.334 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.332+0000 7f2d21600700 1 -- 172.21.15.151:0/3782302886 wait complete. 2024-05-21T22:02:36.336 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.333+0000 7f2d21600700 1 Processor -- start 2024-05-21T22:02:36.336 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.333+0000 7f2d21600700 1 -- start start 2024-05-21T22:02:36.336 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.333+0000 7f2d21600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d1c103490 0x7f2d1c19f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:36.336 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.333+0000 7f2d21600700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d1c103e50 0x7f2d1c19fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:36.336 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.333+0000 7f2d21600700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d1c109540 0x7f2d1c1998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:36.336 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.333+0000 7f2d21600700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2d1c113620 con 0x7f2d1c103490 2024-05-21T22:02:36.336 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.333+0000 7f2d21600700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2d1c1134a0 con 0x7f2d1c103e50 2024-05-21T22:02:36.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.333+0000 7f2d21600700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2d1c1137a0 con 0x7f2d1c109540 2024-05-21T22:02:36.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.334+0000 7f2d1be00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d1c103490 0x7f2d1c19f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:36.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.334+0000 7f2d1be00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d1c103490 0x7f2d1c19f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:33196/0 (socket says 172.21.15.151:33196) 2024-05-21T22:02:36.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.334+0000 7f2d1be00700 1 -- 172.21.15.151:0/3228903771 learned_addr learned my addr 172.21.15.151:0/3228903771 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:02:36.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.334+0000 7f2d1b400700 1 --2- 172.21.15.151:0/3228903771 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d1c103e50 0x7f2d1c19fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:36.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.334+0000 7f2d20c00700 1 --2- 172.21.15.151:0/3228903771 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d1c109540 0x7f2d1c1998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:36.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.334+0000 7f2d1be00700 1 -- 172.21.15.151:0/3228903771 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d1c109540 msgr2=0x7f2d1c1998a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:36.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.334+0000 7f2d1be00700 1 --2- 172.21.15.151:0/3228903771 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d1c109540 0x7f2d1c1998a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:36.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.334+0000 7f2d1be00700 1 -- 172.21.15.151:0/3228903771 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d1c103e50 msgr2=0x7f2d1c19fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:36.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.334+0000 7f2d1be00700 1 --2- 172.21.15.151:0/3228903771 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d1c103e50 0x7f2d1c19fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:36.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.334+0000 7f2d1be00700 1 -- 172.21.15.151:0/3228903771 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d1c19a000 con 0x7f2d1c103490 2024-05-21T22:02:36.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.335+0000 7f2d1be00700 1 --2- 172.21.15.151:0/3228903771 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d1c103490 0x7f2d1c19f6e0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c00eb30 tx=0x7f2d0c00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:02:36.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.335+0000 7f2d18c00700 1 -- 172.21.15.151:0/3228903771 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c00cc90 con 0x7f2d1c103490 2024-05-21T22:02:36.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.335+0000 7f2d18c00700 1 -- 172.21.15.151:0/3228903771 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d0c0044d0 con 0x7f2d1c103490 2024-05-21T22:02:36.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.335+0000 7f2d18c00700 1 -- 172.21.15.151:0/3228903771 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c010430 con 0x7f2d1c103490 2024-05-21T22:02:36.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.335+0000 7f2d21600700 1 -- 172.21.15.151:0/3228903771 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d1c19a2f0 con 0x7f2d1c103490 2024-05-21T22:02:36.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.335+0000 7f2d21600700 1 -- 172.21.15.151:0/3228903771 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d1c1a65b0 con 0x7f2d1c103490 2024-05-21T22:02:36.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.339+0000 7f2d18c00700 1 -- 172.21.15.151:0/3228903771 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7f2d0c003ea0 con 0x7f2d1c103490 2024-05-21T22:02:36.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.340+0000 7f2d2d598b80 1 Processor -- start 2024-05-21T22:02:36.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.340+0000 7f2d2d598b80 1 -- start start 2024-05-21T22:02:36.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.341+0000 7f2d2d598b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555fae037020 0x555fae037400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:36.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.341+0000 7f2d2d598b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555fae037940 0x555fae0314b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:36.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.341+0000 7f2d2d598b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555fae031ad0 0x555fae031f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:36.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.341+0000 7f2d2d598b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x555fadf8fe40 con 0x555fae031ad0 2024-05-21T22:02:36.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.341+0000 7f2d2d598b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x555fadf7f9e0 con 0x555fae037020 2024-05-21T22:02:36.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.341+0000 7f2d2d598b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x555fadf83a20 con 0x555fae037940 2024-05-21T22:02:36.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.341+0000 7f2d18c00700 1 --2- 172.21.15.151:0/3228903771 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f2d000b2af0 0x7f2d000b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:36.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.341+0000 7f2d20c00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555fae037020 0x555fae037400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:36.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.341+0000 7f2d20c00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555fae037020 0x555fae037400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:42774/0 (socket says 172.21.15.151:42774) 2024-05-21T22:02:36.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.341+0000 7f2d20c00700 1 -- 172.21.15.151:0/204259457 learned_addr learned my addr 172.21.15.151:0/204259457 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:02:36.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.341+0000 7f2d1be00700 1 --2- 172.21.15.151:0/204259457 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555fae037940 0x555fae0314b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:36.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.341+0000 7f2d1b400700 1 --2- 172.21.15.151:0/204259457 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555fae031ad0 0x555fae031f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:36.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.341+0000 7f2d20c00700 1 --2- 172.21.15.151:0/3228903771 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f2d000b2af0 0x7f2d000b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:36.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.341+0000 7f2d18c00700 1 -- 172.21.15.151:0/3228903771 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8542+0+0 (secure 0 0 0) 0x7f2d0c0e28a0 con 0x7f2d1c103490 2024-05-21T22:02:36.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.341+0000 7f2d1be00700 1 -- 172.21.15.151:0/204259457 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555fae037020 msgr2=0x555fae037400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:36.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.341+0000 7f2d1be00700 1 --2- 172.21.15.151:0/204259457 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555fae037020 0x555fae037400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:36.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.341+0000 7f2d1be00700 1 -- 172.21.15.151:0/204259457 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555fae031ad0 msgr2=0x555fae031f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:36.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.341+0000 7f2d1be00700 1 --2- 172.21.15.151:0/204259457 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555fae031ad0 0x555fae031f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:36.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.341+0000 7f2d1be00700 1 -- 172.21.15.151:0/204259457 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555fadf48420 con 0x555fae037940 2024-05-21T22:02:36.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.342+0000 7f2d1be00700 1 --2- 172.21.15.151:0/204259457 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555fae037940 0x555fae0314b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c0dff80 tx=0x7f2d0c009640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31f29a35950aaa7f server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:02:36.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.342+0000 7f2d20c00700 1 --2- 172.21.15.151:0/3228903771 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f2d000b2af0 0x7f2d000b4fb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2d14009fd0 tx=0x7f2d1400b040 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:02:36.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.342+0000 7f2d11600700 1 -- 172.21.15.151:0/204259457 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c0a9280 con 0x555fae037940 2024-05-21T22:02:36.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.342+0000 7f2d11600700 1 -- 172.21.15.151:0/204259457 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d0c0a9620 con 0x555fae037940 2024-05-21T22:02:36.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.342+0000 7f2d11600700 1 -- 172.21.15.151:0/204259457 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c0b2970 con 0x555fae037940 2024-05-21T22:02:36.345 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.343+0000 7f2d2d598b80 1 -- 172.21.15.151:0/204259457 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555fae037940 msgr2=0x555fae0314b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:36.345 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.343+0000 7f2d2d598b80 1 --2- 172.21.15.151:0/204259457 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555fae037940 0x555fae0314b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c0dff80 tx=0x7f2d0c009640 comp rx=0 tx=0).stop 2024-05-21T22:02:36.346 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.343+0000 7f2d2d598b80 1 -- 172.21.15.151:0/204259457 shutdown_connections 2024-05-21T22:02:36.346 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.343+0000 7f2d2d598b80 1 --2- 172.21.15.151:0/204259457 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555fae037020 0x555fae037400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:36.346 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.343+0000 7f2d2d598b80 1 --2- 172.21.15.151:0/204259457 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555fae031ad0 0x555fae031f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:36.346 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.343+0000 7f2d2d598b80 1 --2- 172.21.15.151:0/204259457 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555fae037940 0x555fae0314b0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:36.346 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.343+0000 7f2d2d598b80 1 -- 172.21.15.151:0/204259457 >> 172.21.15.151:0/204259457 conn(0x555fadf5fbe0 msgr2=0x555fae02c150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:02:36.346 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.343+0000 7f2d2d598b80 1 -- 172.21.15.151:0/204259457 shutdown_connections 2024-05-21T22:02:36.346 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.343+0000 7f2d2d598b80 1 -- 172.21.15.151:0/204259457 wait complete. 2024-05-21T22:02:36.346 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.344+0000 7f2d2d598b80 1 -- start start 2024-05-21T22:02:36.346 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.344+0000 7f2d2d598b80 1 Processor -- start 2024-05-21T22:02:36.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.345+0000 7f2d2d598b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555fae031ad0 0x555fae03d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:36.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.345+0000 7f2d2d598b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555fae037020 0x555fae03aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:36.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.345+0000 7f2d2d598b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555fae037940 0x555fae03afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:36.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.345+0000 7f2d2d598b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x555fade85fa0 con 0x555fae037020 2024-05-21T22:02:36.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.345+0000 7f2d2d598b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x555fadf20760 con 0x555fae031ad0 2024-05-21T22:02:36.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.345+0000 7f2d2d598b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x555fae03e400 con 0x555fae037940 2024-05-21T22:02:36.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.345+0000 7f2d2d598b80 10 client.0 _ll_register_callbacks cb 0x555fadf29c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:02:36.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.345+0000 7f2d1be00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555fae031ad0 0x555fae03d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:36.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.345+0000 7f2d1b400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555fae037020 0x555fae03aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:36.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.345+0000 7f2d1b400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555fae037020 0x555fae03aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:33210/0 (socket says 172.21.15.151:33210) 2024-05-21T22:02:36.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.345+0000 7f2d1be00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555fae031ad0 0x555fae03d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:42776/0 (socket says 172.21.15.151:42776) 2024-05-21T22:02:36.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.345+0000 7f2d1b400700 1 -- 172.21.15.151:0/2806024317 learned_addr learned my addr 172.21.15.151:0/2806024317 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:02:36.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.345+0000 7f2d20c00700 1 --2- 172.21.15.151:0/2806024317 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555fae037940 0x555fae03afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:36.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.346+0000 7f2d1be00700 1 -- 172.21.15.151:0/2806024317 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555fae037940 msgr2=0x555fae03afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:36.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.346+0000 7f2d1be00700 1 --2- 172.21.15.151:0/2806024317 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555fae037940 0x555fae03afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:36.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.346+0000 7f2d1be00700 1 -- 172.21.15.151:0/2806024317 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555fae037020 msgr2=0x555fae03aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:36.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.346+0000 7f2d1be00700 1 --2- 172.21.15.151:0/2806024317 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555fae037020 0x555fae03aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:36.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.346+0000 7f2d1be00700 1 -- 172.21.15.151:0/2806024317 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555fadecf8d0 con 0x555fae031ad0 2024-05-21T22:02:36.350 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.346+0000 7f2d1be00700 1 --2- 172.21.15.151:0/2806024317 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555fae031ad0 0x555fae03d700 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c0a9cd0 tx=0x7f2d0c0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb11a20899063eae server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:02:36.350 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.346+0000 7f2d06000700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-21T22:02:36.350 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.346+0000 7f2d06000700 1 -- 172.21.15.151:0/2806024317 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c0a9b10 con 0x555fae031ad0 2024-05-21T22:02:36.350 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.346+0000 7f2d06000700 1 -- 172.21.15.151:0/2806024317 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8542+0+0 (secure 0 0 0) 0x7f2d0c0a9d70 con 0x555fae031ad0 2024-05-21T22:02:36.350 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.346+0000 7f2d2d598b80 1 -- 172.21.15.151:0/2806024317 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d0c0a9620 con 0x555fae031ad0 2024-05-21T22:02:36.350 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.346+0000 7f2d06000700 1 -- 172.21.15.151:0/2806024317 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2d0c0aa210 con 0x555fae031ad0 2024-05-21T22:02:36.350 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.346+0000 7f2d2d598b80 10 client.26888 fetch_fsmap learned FSMap version 29 2024-05-21T22:02:36.350 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.346+0000 7f2d2d598b80 1 -- 172.21.15.151:0/2806024317 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555fadf50060 con 0x555fae031ad0 2024-05-21T22:02:36.350 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.347+0000 7f2d06000700 1 -- 172.21.15.151:0/2806024317 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d0c0a9620 con 0x555fae031ad0 2024-05-21T22:02:36.350 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.347+0000 7f2d06000700 1 -- 172.21.15.151:0/2806024317 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f2d0c0b24e0 con 0x555fae031ad0 2024-05-21T22:02:36.350 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.347+0000 7f2d2d598b80 10 client.26888 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:02:36.350 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.347+0000 7f2d2d598b80 10 client.26888 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T22:02:36.350 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.347+0000 7f2d2d598b80 20 client.26888 populate_metadata read hostname 'smithi151' 2024-05-21T22:02:36.350 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.347+0000 7f2d2d598b80 1 --2- 172.21.15.151:0/2806024317 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x555fae04ae50 0x555fae04d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:36.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.347+0000 7f2d2d598b80 4 client.26888 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T22:02:36.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.347+0000 7f2d2d598b80 1 -- 172.21.15.151:0/2806024317 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555fadf91d90 con 0x555fae04ae50 2024-05-21T22:02:36.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.347+0000 7f2d1b400700 1 --2- 172.21.15.151:0/2806024317 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x555fae04ae50 0x555fae04d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:36.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.348+0000 7f2d1b400700 1 --2- 172.21.15.151:0/2806024317 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x555fae04ae50 0x555fae04d240 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2cf255a1ffdf583b server_cookie=b501413e76c29e6c in_seq=0 out_seq=0 2024-05-21T22:02:36.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.348+0000 7f2d06000700 10 client.26888 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T22:02:36.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.349+0000 7f2d06000700 1 -- 172.21.15.151:0/2806024317 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x555fadf91d90 con 0x555fae04ae50 2024-05-21T22:02:36.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.349+0000 7f2d06000700 10 client.26888 handle_command_reply: tid=0 2024-05-21T22:02:36.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.349+0000 7f2d2d598b80 1 client.26888 shutdown 2024-05-21T22:02:36.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.349+0000 7f2d2d598b80 1 -- 172.21.15.151:0/2806024317 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555fae031ad0 msgr2=0x555fae03d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:36.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.349+0000 7f2d2d598b80 1 --2- 172.21.15.151:0/2806024317 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555fae031ad0 0x555fae03d700 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c0a9cd0 tx=0x7f2d0c0047c0 comp rx=0 tx=0).stop 2024-05-21T22:02:36.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.349+0000 7f2d2d598b80 1 -- 172.21.15.151:0/2806024317 shutdown_connections 2024-05-21T22:02:36.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.350+0000 7f2d2d598b80 1 --2- 172.21.15.151:0/2806024317 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x555fae04ae50 0x555fae04d240 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:36.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.350+0000 7f2d2d598b80 1 --2- 172.21.15.151:0/2806024317 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555fae031ad0 0x555fae03d700 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:36.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.350+0000 7f2d2d598b80 1 --2- 172.21.15.151:0/2806024317 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555fae037940 0x555fae03afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:36.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.350+0000 7f2d2d598b80 1 --2- 172.21.15.151:0/2806024317 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555fae037020 0x555fae03aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:36.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.350+0000 7f2d06000700 0 client.26888 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T22:02:36.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.350+0000 7f2d2d598b80 1 -- 172.21.15.151:0/2806024317 >> 172.21.15.151:0/2806024317 conn(0x555fadf5fbe0 msgr2=0x555fae0304a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:02:36.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.350+0000 7f2d2d598b80 1 -- 172.21.15.151:0/2806024317 shutdown_connections 2024-05-21T22:02:36.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.350+0000 7f2d2d598b80 1 -- 172.21.15.151:0/2806024317 wait complete. 2024-05-21T22:02:36.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.350+0000 7f2d2d598b80 20 client.26888 trim_cache size 0 max 16384 2024-05-21T22:02:36.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.372+0000 7f2d2d598b80 1 Processor -- start 2024-05-21T22:02:36.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.372+0000 7f2d2d598b80 1 -- start start 2024-05-21T22:02:36.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.373+0000 7f2d2d598b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555fae031ad0 0x555fae039390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:36.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.373+0000 7f2d2d598b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555fae037020 0x555fae0398d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:36.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.373+0000 7f2d2d598b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555fae037940 0x555fae02bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:36.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.373+0000 7f2d2d598b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2d1c1abab0 con 0x555fae031ad0 2024-05-21T22:02:36.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.373+0000 7f2d2d598b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2d1c1ad5f0 con 0x555fae037020 2024-05-21T22:02:36.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.373+0000 7f2d2d598b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2d1c1ac180 con 0x555fae037940 2024-05-21T22:02:36.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.373+0000 7f2d1be00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555fae031ad0 0x555fae039390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:36.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.373+0000 7f2d1b400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555fae037020 0x555fae0398d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:36.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.373+0000 7f2d20c00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555fae037940 0x555fae02bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:36.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.373+0000 7f2d1b400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555fae037020 0x555fae0398d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:42782/0 (socket says 172.21.15.151:42782) 2024-05-21T22:02:36.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.373+0000 7f2d1b400700 1 -- 172.21.15.151:0/2290547262 learned_addr learned my addr 172.21.15.151:0/2290547262 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:02:36.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.374+0000 7f2d20c00700 1 -- 172.21.15.151:0/2290547262 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555fae037020 msgr2=0x555fae0398d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:36.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.374+0000 7f2d20c00700 1 --2- 172.21.15.151:0/2290547262 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555fae037020 0x555fae0398d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:36.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.374+0000 7f2d20c00700 1 -- 172.21.15.151:0/2290547262 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555fae031ad0 msgr2=0x555fae039390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:36.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.374+0000 7f2d20c00700 1 --2- 172.21.15.151:0/2290547262 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555fae031ad0 0x555fae039390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:36.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.374+0000 7f2d20c00700 1 -- 172.21.15.151:0/2290547262 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555fae02df90 con 0x555fae037940 2024-05-21T22:02:36.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.374+0000 7f2d20c00700 1 --2- 172.21.15.151:0/2290547262 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555fae037940 0x555fae02bc30 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f2d14008b40 tx=0x7f2d14009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=deb4321115b8e27d server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:02:36.377 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.374+0000 7f2d10c00700 1 -- 172.21.15.151:0/2290547262 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d14004bb0 con 0x555fae037940 2024-05-21T22:02:36.377 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.374+0000 7f2d10c00700 1 -- 172.21.15.151:0/2290547262 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d14004d50 con 0x555fae037940 2024-05-21T22:02:36.377 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.374+0000 7f2d10c00700 1 -- 172.21.15.151:0/2290547262 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d14004030 con 0x555fae037940 2024-05-21T22:02:36.377 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.375+0000 7f2d2d598b80 1 -- 172.21.15.151:0/2290547262 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555fae037940 msgr2=0x555fae02bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:36.377 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.375+0000 7f2d2d598b80 1 --2- 172.21.15.151:0/2290547262 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555fae037940 0x555fae02bc30 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f2d14008b40 tx=0x7f2d14009920 comp rx=0 tx=0).stop 2024-05-21T22:02:36.377 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.375+0000 7f2d2d598b80 1 -- 172.21.15.151:0/2290547262 shutdown_connections 2024-05-21T22:02:36.377 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.375+0000 7f2d2d598b80 1 --2- 172.21.15.151:0/2290547262 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555fae037940 0x555fae02bc30 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:36.377 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.375+0000 7f2d2d598b80 1 --2- 172.21.15.151:0/2290547262 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555fae031ad0 0x555fae039390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:36.377 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.375+0000 7f2d2d598b80 1 --2- 172.21.15.151:0/2290547262 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555fae037020 0x555fae0398d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:36.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.375+0000 7f2d2d598b80 1 -- 172.21.15.151:0/2290547262 >> 172.21.15.151:0/2290547262 conn(0x555fadf5fbe0 msgr2=0x555fae03aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:02:36.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.375+0000 7f2d2d598b80 1 -- 172.21.15.151:0/2290547262 shutdown_connections 2024-05-21T22:02:36.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.375+0000 7f2d2d598b80 1 -- 172.21.15.151:0/2290547262 wait complete. 2024-05-21T22:02:36.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.376+0000 7f2d2d598b80 1 -- start start 2024-05-21T22:02:36.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.376+0000 7f2d2d598b80 1 Processor -- start 2024-05-21T22:02:36.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.376+0000 7f2d2d598b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555fae031ad0 0x555fae050830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:36.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.377+0000 7f2d2d598b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555fae037020 0x555fae04dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:36.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.377+0000 7f2d2d598b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555fae037940 0x555fae04e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:36.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.377+0000 7f2d2d598b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2d1c1acf20 con 0x555fae037940 2024-05-21T22:02:36.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.377+0000 7f2d2d598b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2d1c1ac850 con 0x555fae031ad0 2024-05-21T22:02:36.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.377+0000 7f2d2d598b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2d1c1a9a70 con 0x555fae037020 2024-05-21T22:02:36.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.377+0000 7f2d2d598b80 10 client.0 _ll_register_callbacks cb 0x555fadf29c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:02:36.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.377+0000 7f2d1be00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555fae031ad0 0x555fae050830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:36.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.377+0000 7f2d1b400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555fae037020 0x555fae04dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:36.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.377+0000 7f2d1be00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555fae031ad0 0x555fae050830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:42792/0 (socket says 172.21.15.151:42792) 2024-05-21T22:02:36.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.377+0000 7f2d1be00700 1 -- 172.21.15.151:0/3521111308 learned_addr learned my addr 172.21.15.151:0/3521111308 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:02:36.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.377+0000 7f2d20c00700 1 --2- 172.21.15.151:0/3521111308 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555fae037940 0x555fae04e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:36.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.377+0000 7f2d1be00700 1 -- 172.21.15.151:0/3521111308 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555fae037020 msgr2=0x555fae04dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:36.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.377+0000 7f2d1be00700 1 --2- 172.21.15.151:0/3521111308 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555fae037020 0x555fae04dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:36.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.377+0000 7f2d1be00700 1 -- 172.21.15.151:0/3521111308 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555fae037940 msgr2=0x555fae04e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:36.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.377+0000 7f2d1be00700 1 --2- 172.21.15.151:0/3521111308 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555fae037940 0x555fae04e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:36.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.377+0000 7f2d1be00700 1 -- 172.21.15.151:0/3521111308 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555fae04e860 con 0x555fae031ad0 2024-05-21T22:02:36.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.377+0000 7f2d20c00700 1 --2- 172.21.15.151:0/3521111308 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555fae037940 0x555fae04e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T22:02:36.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.378+0000 7f2d1be00700 1 --2- 172.21.15.151:0/3521111308 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555fae031ad0 0x555fae050830 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c0a9aa0 tx=0x7f2d0c0aa050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c8bb5c8fe6a2f19 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:02:36.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.378+0000 7f2d06000700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-21T22:02:36.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.378+0000 7f2d06000700 1 -- 172.21.15.151:0/3521111308 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c0a9280 con 0x555fae031ad0 2024-05-21T22:02:36.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.378+0000 7f2d06000700 1 -- 172.21.15.151:0/3521111308 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8542+0+0 (secure 0 0 0) 0x7f2d0c0a6100 con 0x555fae031ad0 2024-05-21T22:02:36.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.378+0000 7f2d2d598b80 1 -- 172.21.15.151:0/3521111308 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d14004d50 con 0x555fae031ad0 2024-05-21T22:02:36.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.378+0000 7f2d06000700 1 -- 172.21.15.151:0/3521111308 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2d0c0b6070 con 0x555fae031ad0 2024-05-21T22:02:36.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.378+0000 7f2d2d598b80 10 client.26894 fetch_fsmap learned FSMap version 29 2024-05-21T22:02:36.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.378+0000 7f2d2d598b80 1 -- 172.21.15.151:0/3521111308 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555fae0b55b0 con 0x555fae031ad0 2024-05-21T22:02:36.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.379+0000 7f2d06000700 1 -- 172.21.15.151:0/3521111308 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d14004d50 con 0x555fae031ad0 2024-05-21T22:02:36.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.379+0000 7f2d06000700 1 -- 172.21.15.151:0/3521111308 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f2d0c0a5480 con 0x555fae031ad0 2024-05-21T22:02:36.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.379+0000 7f2d2d598b80 10 client.26894 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:02:36.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.379+0000 7f2d2d598b80 10 client.26894 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T22:02:36.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.379+0000 7f2d2d598b80 20 client.26894 populate_metadata read hostname 'smithi151' 2024-05-21T22:02:36.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.379+0000 7f2d2d598b80 1 --2- 172.21.15.151:0/3521111308 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x555fae0b5a10 0x555fae0b7e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:36.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.379+0000 7f2d2d598b80 4 client.26894 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-21T22:02:36.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.379+0000 7f2d2d598b80 1 -- 172.21.15.151:0/3521111308 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555fadf8b570 con 0x555fae0b5a10 2024-05-21T22:02:36.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.379+0000 7f2d1b400700 1 --2- 172.21.15.151:0/3521111308 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x555fae0b5a10 0x555fae0b7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:36.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.380+0000 7f2d1b400700 1 --2- 172.21.15.151:0/3521111308 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x555fae0b5a10 0x555fae0b7e00 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b668cd685b82fea server_cookie=187325eafdb091a9 in_seq=0 out_seq=0 2024-05-21T22:02:36.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.380+0000 7f2d06000700 10 client.26894 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T22:02:36.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.380+0000 7f2d06000700 1 -- 172.21.15.151:0/3521111308 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x555fadf8b570 con 0x555fae0b5a10 2024-05-21T22:02:36.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.380+0000 7f2d06000700 10 client.26894 handle_command_reply: tid=0 2024-05-21T22:02:36.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.380+0000 7f2d2d598b80 1 client.26894 shutdown 2024-05-21T22:02:36.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.380+0000 7f2d2d598b80 1 -- 172.21.15.151:0/3521111308 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555fae031ad0 msgr2=0x555fae050830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:36.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.380+0000 7f2d2d598b80 1 --2- 172.21.15.151:0/3521111308 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555fae031ad0 0x555fae050830 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c0a9aa0 tx=0x7f2d0c0aa050 comp rx=0 tx=0).stop 2024-05-21T22:02:36.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.380+0000 7f2d2d598b80 1 -- 172.21.15.151:0/3521111308 shutdown_connections 2024-05-21T22:02:36.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.380+0000 7f2d2d598b80 1 --2- 172.21.15.151:0/3521111308 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x555fae0b5a10 0x555fae0b7e00 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:36.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.380+0000 7f2d2d598b80 1 --2- 172.21.15.151:0/3521111308 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555fae031ad0 0x555fae050830 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:36.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.380+0000 7f2d2d598b80 1 --2- 172.21.15.151:0/3521111308 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555fae037940 0x555fae04e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:36.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.380+0000 7f2d06000700 0 client.26894 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T22:02:36.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.380+0000 7f2d2d598b80 1 --2- 172.21.15.151:0/3521111308 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555fae037020 0x555fae04dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:36.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.380+0000 7f2d2d598b80 1 -- 172.21.15.151:0/3521111308 >> 172.21.15.151:0/3521111308 conn(0x555fadf5fbe0 msgr2=0x555fae034aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:02:36.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.380+0000 7f2d2d598b80 1 -- 172.21.15.151:0/3521111308 shutdown_connections 2024-05-21T22:02:36.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.381+0000 7f2d2d598b80 1 -- 172.21.15.151:0/3521111308 wait complete. 2024-05-21T22:02:36.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.381+0000 7f2d2d598b80 20 client.26894 trim_cache size 0 max 16384 2024-05-21T22:02:36.384 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-05-21T22:02:36.385 INFO:teuthology.orchestra.run.smithi151.stdout: "status": "no active scrubs running", 2024-05-21T22:02:36.385 INFO:teuthology.orchestra.run.smithi151.stdout: "scrubs": {} 2024-05-21T22:02:36.385 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-05-21T22:02:36.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.381+0000 7f2d21600700 1 -- 172.21.15.151:0/3228903771 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f2d000b2af0 msgr2=0x7f2d000b4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:36.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.381+0000 7f2d21600700 1 --2- 172.21.15.151:0/3228903771 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f2d000b2af0 0x7f2d000b4fb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2d14009fd0 tx=0x7f2d1400b040 comp rx=0 tx=0).stop 2024-05-21T22:02:36.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.381+0000 7f2d21600700 1 -- 172.21.15.151:0/3228903771 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d1c103490 msgr2=0x7f2d1c19f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:36.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.381+0000 7f2d21600700 1 --2- 172.21.15.151:0/3228903771 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d1c103490 0x7f2d1c19f6e0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c00eb30 tx=0x7f2d0c00ee40 comp rx=0 tx=0).stop 2024-05-21T22:02:36.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.382+0000 7f2d21600700 1 -- 172.21.15.151:0/3228903771 shutdown_connections 2024-05-21T22:02:36.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.382+0000 7f2d21600700 1 --2- 172.21.15.151:0/3228903771 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f2d000b2af0 0x7f2d000b4fb0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:36.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.382+0000 7f2d21600700 1 --2- 172.21.15.151:0/3228903771 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d1c109540 0x7f2d1c1998a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:36.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.382+0000 7f2d21600700 1 --2- 172.21.15.151:0/3228903771 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d1c103490 0x7f2d1c19f6e0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:36.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.382+0000 7f2d21600700 1 --2- 172.21.15.151:0/3228903771 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d1c103e50 0x7f2d1c19fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:36.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.382+0000 7f2d21600700 1 -- 172.21.15.151:0/3228903771 >> 172.21.15.151:0/3228903771 conn(0x7f2d1c0fefc0 msgr2=0x7f2d1c100770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:02:36.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.382+0000 7f2d21600700 1 -- 172.21.15.151:0/3228903771 shutdown_connections 2024-05-21T22:02:36.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:36.382+0000 7f2d21600700 1 -- 172.21.15.151:0/3228903771 wait complete. 2024-05-21T22:02:37.269 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-21T22:02:37.269 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-21T22:02:37.718 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:02:37.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:02:37 smithi164 ceph-mon[36269]: pgmap v215: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:02:37.871 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:02:37 smithi151 ceph-mon[31623]: pgmap v215: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:02:37.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:02:37 smithi184 ceph-mon[36942]: pgmap v215: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:02:39.746 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:02:39 smithi151 ceph-mon[31623]: pgmap v216: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-05-21T22:02:39.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:02:39 smithi164 ceph-mon[36269]: pgmap v216: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-05-21T22:02:39.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:02:39 smithi184 ceph-mon[36942]: pgmap v216: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-05-21T22:02:40.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:40.006+0000 7f431aa00700 1 -- 172.21.15.151:0/3353429591 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43141035d0 msgr2=0x7f4314103a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:40.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:40.006+0000 7f431aa00700 1 --2- 172.21.15.151:0/3353429591 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43141035d0 0x7f4314103a50 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f4300009920 tx=0x7f4300009c30 comp rx=0 tx=0).stop 2024-05-21T22:02:40.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:40.006+0000 7f431aa00700 1 -- 172.21.15.151:0/3353429591 shutdown_connections 2024-05-21T22:02:40.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:40.006+0000 7f431aa00700 1 --2- 172.21.15.151:0/3353429591 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4314103f90 0x7f4314110440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:40.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:40.006+0000 7f431aa00700 1 --2- 172.21.15.151:0/3353429591 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4314109670 0x7f4314109a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:40.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:40.006+0000 7f431aa00700 1 --2- 172.21.15.151:0/3353429591 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43141035d0 0x7f4314103a50 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:40.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:40.006+0000 7f431aa00700 1 -- 172.21.15.151:0/3353429591 >> 172.21.15.151:0/3353429591 conn(0x7f431407a730 msgr2=0x7f431407ab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:02:40.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:40.006+0000 7f431aa00700 1 -- 172.21.15.151:0/3353429591 shutdown_connections 2024-05-21T22:02:40.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:40.006+0000 7f431aa00700 1 -- 172.21.15.151:0/3353429591 wait complete. 2024-05-21T22:02:40.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:40.007+0000 7f431aa00700 1 Processor -- start 2024-05-21T22:02:40.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:40.007+0000 7f431aa00700 1 -- start start 2024-05-21T22:02:40.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:40.008+0000 7f431aa00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f43141035d0 0x7f4314075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:40.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:40.008+0000 7f431aa00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4314103f90 0x7f4314075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:40.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:40.008+0000 7f431aa00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4314109670 0x7f43140762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:40.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:40.008+0000 7f431aa00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f43141137d0 con 0x7f43141035d0 2024-05-21T22:02:40.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:40.008+0000 7f431aa00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4314113650 con 0x7f4314109670 2024-05-21T22:02:40.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:40.008+0000 7f431aa00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4314113950 con 0x7f4314103f90 2024-05-21T22:02:40.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:40.008+0000 7f4313400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4314103f90 0x7f4314075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:40.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:40.008+0000 7f4313e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f43141035d0 0x7f4314075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:40.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:40.008+0000 7f4313400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4314103f90 0x7f4314075d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:51922/0 (socket says 172.21.15.151:51922) 2024-05-21T22:02:40.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:40.008+0000 7f4313400700 1 -- 172.21.15.151:0/3314624293 learned_addr learned my addr 172.21.15.151:0/3314624293 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:02:40.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:40.008+0000 7f4318a00700 1 --2- 172.21.15.151:0/3314624293 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4314109670 0x7f43140762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:40.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:40.009+0000 7f4313400700 1 -- 172.21.15.151:0/3314624293 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4314109670 msgr2=0x7f43140762c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:40.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:40.009+0000 7f4313400700 1 --2- 172.21.15.151:0/3314624293 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4314109670 0x7f43140762c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:40.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:40.009+0000 7f4313400700 1 -- 172.21.15.151:0/3314624293 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f43141035d0 msgr2=0x7f4314075840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:40.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:40.009+0000 7f4313400700 1 --2- 172.21.15.151:0/3314624293 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f43141035d0 0x7f4314075840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:40.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:40.009+0000 7f4313400700 1 -- 172.21.15.151:0/3314624293 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43141a81f0 con 0x7f4314103f90 2024-05-21T22:02:40.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:40.009+0000 7f4313400700 1 --2- 172.21.15.151:0/3314624293 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4314103f90 0x7f4314075d80 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f4300009230 tx=0x7f430000fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:02:40.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:40.009+0000 7f4310c00700 1 -- 172.21.15.151:0/3314624293 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4300027070 con 0x7f4314103f90 2024-05-21T22:02:40.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:40.009+0000 7f4310c00700 1 -- 172.21.15.151:0/3314624293 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4300005570 con 0x7f4314103f90 2024-05-21T22:02:40.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:40.009+0000 7f431aa00700 1 -- 172.21.15.151:0/3314624293 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43141a8480 con 0x7f4314103f90 2024-05-21T22:02:40.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:40.009+0000 7f4310c00700 1 -- 172.21.15.151:0/3314624293 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4300004730 con 0x7f4314103f90 2024-05-21T22:02:40.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:40.010+0000 7f431aa00700 1 -- 172.21.15.151:0/3314624293 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43141a88d0 con 0x7f4314103f90 2024-05-21T22:02:40.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:40.011+0000 7f431aa00700 1 -- 172.21.15.151:0/3314624293 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f431410bb00 con 0x7f4314103f90 2024-05-21T22:02:40.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:40.014+0000 7f4310c00700 1 -- 172.21.15.151:0/3314624293 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7f430002a070 con 0x7f4314103f90 2024-05-21T22:02:40.017 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:40.016+0000 7f4310c00700 1 --2- 172.21.15.151:0/3314624293 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f42f80b29f0 0x7f42f80b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:40.017 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:40.016+0000 7f4310c00700 1 -- 172.21.15.151:0/3314624293 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8542+0+0 (secure 0 0 0) 0x7f43000e33e0 con 0x7f4314103f90 2024-05-21T22:02:40.017 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:40.016+0000 7f4313e00700 1 --2- 172.21.15.151:0/3314624293 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f42f80b29f0 0x7f42f80b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:40.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:40.017+0000 7f4313e00700 1 --2- 172.21.15.151:0/3314624293 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f42f80b29f0 0x7f42f80b4eb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4304009be0 tx=0x7f4304009260 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:02:40.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:40.017+0000 7f4310c00700 1 -- 172.21.15.151:0/3314624293 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4300031020 con 0x7f4314103f90 2024-05-21T22:02:40.306 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:40.304+0000 7f431aa00700 1 -- 172.21.15.151:0/3314624293 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f431406a230 con 0x7f4314103f90 2024-05-21T22:02:40.307 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:40.305+0000 7f4310c00700 1 -- 172.21.15.151:0/3314624293 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11958 (secure 0 0 0) 0x7f4300013030 con 0x7f4314103f90 2024-05-21T22:02:40.307 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T22:02:40.308 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14358,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/158010529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":158010529},{"type":"v1","addr":"172.21.15.151:6835","nonce":158010529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6839/2069058115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":2069058115},{"type":"v1","addr":"172.21.15.151:6839","nonce":2069058115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T21:51:19.552069+0000","modified":"2024-05-21T21:55:34.643755+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24530,"mds_1":24493,"mds_2":24491,"mds_3":24478,"mds_4":24473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14403":{"gid":14403,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6841/1749723794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6840","nonce":1749723794},{"type":"v1","addr":"172.21.15.151:6841","nonce":1749723794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/2026266114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":2026266114},{"type":"v1","addr":"172.21.15.184:6833","nonce":2026266114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/2467900844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":2467900844},{"type":"v1","addr":"172.21.15.164:6833","nonce":2467900844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3367640960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3367640960},{"type":"v1","addr":"172.21.15.151:6837","nonce":3367640960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/3264142621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3264142621},{"type":"v1","addr":"172.21.15.184:6835","nonce":3264142621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":17,"addr":"172.21.15.164:6835/2340428470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2340428470},{"type":"v1","addr":"172.21.15.164:6835","nonce":2340428470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":16,"addr":"172.21.15.184:6837/2988194888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":2988194888},{"type":"v1","addr":"172.21.15.184:6837","nonce":2988194888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.164:6837/1554609572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1554609572},{"type":"v1","addr":"172.21.15.164:6837","nonce":1554609572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.184:6839/3059935143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":3059935143},{"type":"v1","addr":"172.21.15.184:6839","nonce":3059935143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.164:6839/2646401247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":2646401247},{"type":"v1","addr":"172.21.15.164:6839","nonce":2646401247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24530,"qdb_cluster":[24530,24493,24491,24473,24478]},"id":1}]} 2024-05-21T22:02:40.311 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:40.309+0000 7f431aa00700 1 -- 172.21.15.151:0/3314624293 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f42f80b29f0 msgr2=0x7f42f80b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:40.311 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:40.309+0000 7f431aa00700 1 --2- 172.21.15.151:0/3314624293 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f42f80b29f0 0x7f42f80b4eb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4304009be0 tx=0x7f4304009260 comp rx=0 tx=0).stop 2024-05-21T22:02:40.311 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:40.309+0000 7f431aa00700 1 -- 172.21.15.151:0/3314624293 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4314103f90 msgr2=0x7f4314075d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:40.311 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:40.309+0000 7f431aa00700 1 --2- 172.21.15.151:0/3314624293 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4314103f90 0x7f4314075d80 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f4300009230 tx=0x7f430000fa00 comp rx=0 tx=0).stop 2024-05-21T22:02:40.311 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:40.309+0000 7f431aa00700 1 -- 172.21.15.151:0/3314624293 shutdown_connections 2024-05-21T22:02:40.311 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:40.309+0000 7f431aa00700 1 --2- 172.21.15.151:0/3314624293 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f42f80b29f0 0x7f42f80b4eb0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:40.311 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:40.309+0000 7f431aa00700 1 --2- 172.21.15.151:0/3314624293 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4314109670 0x7f43140762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:40.311 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:40.309+0000 7f431aa00700 1 --2- 172.21.15.151:0/3314624293 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f43141035d0 0x7f4314075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:40.311 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:40.309+0000 7f431aa00700 1 --2- 172.21.15.151:0/3314624293 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4314103f90 0x7f4314075d80 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:40.311 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:40.309+0000 7f431aa00700 1 -- 172.21.15.151:0/3314624293 >> 172.21.15.151:0/3314624293 conn(0x7f431407a730 msgr2=0x7f43141024c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:02:40.311 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:40.309+0000 7f431aa00700 1 -- 172.21.15.151:0/3314624293 shutdown_connections 2024-05-21T22:02:40.311 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:40.310+0000 7f431aa00700 1 -- 172.21.15.151:0/3314624293 wait complete. 2024-05-21T22:02:40.312 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 29 2024-05-21T22:02:40.651 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:02:40 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/3314624293' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T22:02:40.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:02:40 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/3314624293' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T22:02:40.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:02:40 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/3314624293' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T22:02:41.133 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-21T22:02:41.579 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:02:41.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:02:41 smithi164 ceph-mon[36269]: pgmap v217: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-05-21T22:02:41.903 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:02:41 smithi151 ceph-mon[31623]: pgmap v217: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-05-21T22:02:41.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:02:41 smithi184 ceph-mon[36942]: pgmap v217: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-05-21T22:02:43.651 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:02:43 smithi151 ceph-mon[31623]: pgmap v218: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-05-21T22:02:43.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:02:43 smithi164 ceph-mon[36269]: pgmap v218: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-05-21T22:02:43.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.821+0000 7f01c4e00700 1 -- 172.21.15.151:0/2473588225 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01c01037f0 msgr2=0x7f01c0103bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:43.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.821+0000 7f01c4e00700 1 --2- 172.21.15.151:0/2473588225 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01c01037f0 0x7f01c0103bd0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f01b000b0a0 tx=0x7f01b000b3b0 comp rx=0 tx=0).stop 2024-05-21T22:02:43.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.821+0000 7f01c4e00700 1 -- 172.21.15.151:0/2473588225 shutdown_connections 2024-05-21T22:02:43.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.821+0000 7f01c4e00700 1 --2- 172.21.15.151:0/2473588225 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f01c01088f0 0x7f01c0110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:43.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.821+0000 7f01c4e00700 1 --2- 172.21.15.151:0/2473588225 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01c01037f0 0x7f01c0103bd0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:43.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.821+0000 7f01c4e00700 1 --2- 172.21.15.151:0/2473588225 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f01c01041a0 0x7f01c01081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:43.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.821+0000 7f01c4e00700 1 -- 172.21.15.151:0/2473588225 >> 172.21.15.151:0/2473588225 conn(0x7f01c007a6a0 msgr2=0x7f01c007aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:02:43.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.821+0000 7f01c4e00700 1 -- 172.21.15.151:0/2473588225 shutdown_connections 2024-05-21T22:02:43.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.821+0000 7f01c4e00700 1 -- 172.21.15.151:0/2473588225 wait complete. 2024-05-21T22:02:43.823 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.823+0000 7f01c4e00700 1 Processor -- start 2024-05-21T22:02:43.823 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.823+0000 7f01c4e00700 1 -- start start 2024-05-21T22:02:43.823 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.823+0000 7f01c4e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f01c01037f0 0x7f01c019be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:43.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.823+0000 7f01c4e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f01c01041a0 0x7f01c019c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:43.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.823+0000 7f01c4e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01c01088f0 0x7f01c01a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:43.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.823+0000 7f01c4e00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f01c0113710 con 0x7f01c01041a0 2024-05-21T22:02:43.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.823+0000 7f01c4e00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f01c0113590 con 0x7f01c01037f0 2024-05-21T22:02:43.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.823+0000 7f01c4e00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f01c0113890 con 0x7f01c01088f0 2024-05-21T22:02:43.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.824+0000 7f01b5400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f01c01041a0 0x7f01c019c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:43.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.824+0000 7f01bde00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f01c01037f0 0x7f01c019be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:43.826 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.824+0000 7f01b5400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f01c01041a0 0x7f01c019c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:33268/0 (socket says 172.21.15.151:33268) 2024-05-21T22:02:43.826 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.824+0000 7f01b5400700 1 -- 172.21.15.151:0/848010184 learned_addr learned my addr 172.21.15.151:0/848010184 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:02:43.826 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.824+0000 7f01be800700 1 --2- 172.21.15.151:0/848010184 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01c01088f0 0x7f01c01a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:43.826 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.824+0000 7f01bde00700 1 -- 172.21.15.151:0/848010184 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01c01088f0 msgr2=0x7f01c01a06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:43.826 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.824+0000 7f01bde00700 1 --2- 172.21.15.151:0/848010184 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01c01088f0 0x7f01c01a06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:43.826 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.824+0000 7f01bde00700 1 -- 172.21.15.151:0/848010184 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f01c01041a0 msgr2=0x7f01c019c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:43.826 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.824+0000 7f01bde00700 1 --2- 172.21.15.151:0/848010184 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f01c01041a0 0x7f01c019c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:43.826 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.824+0000 7f01bde00700 1 -- 172.21.15.151:0/848010184 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01c01a0e10 con 0x7f01c01037f0 2024-05-21T22:02:43.826 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.825+0000 7f01bde00700 1 --2- 172.21.15.151:0/848010184 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f01c01037f0 0x7f01c019be00 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f01b000b780 tx=0x7f01b0004d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:02:43.826 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.825+0000 7f01b7400700 1 -- 172.21.15.151:0/848010184 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b0021070 con 0x7f01c01037f0 2024-05-21T22:02:43.826 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.825+0000 7f01c4e00700 1 -- 172.21.15.151:0/848010184 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01c01a10a0 con 0x7f01c01037f0 2024-05-21T22:02:43.826 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.825+0000 7f01b7400700 1 -- 172.21.15.151:0/848010184 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01b0007b80 con 0x7f01c01037f0 2024-05-21T22:02:43.826 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.825+0000 7f01b7400700 1 -- 172.21.15.151:0/848010184 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b0005040 con 0x7f01c01037f0 2024-05-21T22:02:43.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.825+0000 7f01c4e00700 1 -- 172.21.15.151:0/848010184 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01c01a8980 con 0x7f01c01037f0 2024-05-21T22:02:43.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.828+0000 7f01b7400700 1 -- 172.21.15.151:0/848010184 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7f01b0024070 con 0x7f01c01037f0 2024-05-21T22:02:43.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.830+0000 7f01cf826b80 1 Processor -- start 2024-05-21T22:02:43.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.830+0000 7f01cf826b80 1 -- start start 2024-05-21T22:02:43.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.830+0000 7f01b7400700 1 --2- 172.21.15.151:0/848010184 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f01a40b29f0 0x7f01a40b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:43.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.830+0000 7f01cf826b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5616b124eeb0 0x5616b124f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:43.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.830+0000 7f01cf826b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5616b124f7d0 0x5616b1249260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:43.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.830+0000 7f01cf826b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5616b1249880 0x5616b1249ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:43.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.830+0000 7f01cf826b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5616b11a7e40 con 0x5616b124eeb0 2024-05-21T22:02:43.833 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.830+0000 7f01cf826b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5616b11979e0 con 0x5616b124f7d0 2024-05-21T22:02:43.833 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.830+0000 7f01cf826b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5616b119ba20 con 0x5616b1249880 2024-05-21T22:02:43.833 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.831+0000 7f01b7400700 1 -- 172.21.15.151:0/848010184 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8542+0+0 (secure 0 0 0) 0x7f01b000f020 con 0x7f01c01037f0 2024-05-21T22:02:43.833 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.831+0000 7f01b5400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5616b124f7d0 0x5616b1249260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:43.833 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.831+0000 7f01be800700 1 --2- 172.21.15.151:0/848010184 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f01a40b29f0 0x7f01a40b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:43.833 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.831+0000 7f01b5400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5616b124f7d0 0x5616b1249260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:42824/0 (socket says 172.21.15.151:42824) 2024-05-21T22:02:43.833 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.831+0000 7f01b5400700 1 -- 172.21.15.151:0/3037861085 learned_addr learned my addr 172.21.15.151:0/3037861085 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:02:43.833 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.831+0000 7f01bde00700 1 --2- 172.21.15.151:0/3037861085 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5616b124eeb0 0x5616b124f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:43.833 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.831+0000 7f01be800700 1 --2- 172.21.15.151:0/3037861085 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5616b1249880 0x5616b1249ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:43.833 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.831+0000 7f01b5400700 1 -- 172.21.15.151:0/3037861085 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5616b1249880 msgr2=0x5616b1249ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:43.833 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.831+0000 7f01b5400700 1 --2- 172.21.15.151:0/3037861085 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5616b1249880 0x5616b1249ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:43.833 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.831+0000 7f01b5400700 1 -- 172.21.15.151:0/3037861085 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5616b124eeb0 msgr2=0x5616b124f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:43.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.831+0000 7f01b5400700 1 --2- 172.21.15.151:0/3037861085 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5616b124eeb0 0x5616b124f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:43.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.831+0000 7f01b5400700 1 -- 172.21.15.151:0/3037861085 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616b1160420 con 0x5616b124f7d0 2024-05-21T22:02:43.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.831+0000 7f01bde00700 1 --2- 172.21.15.151:0/3037861085 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5616b124eeb0 0x5616b124f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T22:02:43.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.832+0000 7f01b5400700 1 --2- 172.21.15.151:0/3037861085 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5616b124f7d0 0x5616b1249260 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f01a8009e30 tx=0x7f01a8011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cae89e62b04d9098 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:02:43.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.832+0000 7f01be800700 1 --2- 172.21.15.151:0/848010184 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f01a40b29f0 0x7f01a40b4eb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f01b80097a0 tx=0x7f01b80099b0 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:02:43.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.832+0000 7f01af400700 1 -- 172.21.15.151:0/3037861085 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a801e070 con 0x5616b124f7d0 2024-05-21T22:02:43.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.832+0000 7f01af400700 1 -- 172.21.15.151:0/3037861085 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01a8019ae0 con 0x5616b124f7d0 2024-05-21T22:02:43.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.832+0000 7f01af400700 1 -- 172.21.15.151:0/3037861085 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a800fb50 con 0x5616b124f7d0 2024-05-21T22:02:43.836 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.833+0000 7f01cf826b80 1 -- 172.21.15.151:0/3037861085 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5616b124f7d0 msgr2=0x5616b1249260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:43.836 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.833+0000 7f01cf826b80 1 --2- 172.21.15.151:0/3037861085 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5616b124f7d0 0x5616b1249260 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f01a8009e30 tx=0x7f01a8011720 comp rx=0 tx=0).stop 2024-05-21T22:02:43.837 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.833+0000 7f01cf826b80 1 -- 172.21.15.151:0/3037861085 shutdown_connections 2024-05-21T22:02:43.837 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.833+0000 7f01cf826b80 1 --2- 172.21.15.151:0/3037861085 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5616b1249880 0x5616b1249ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:43.837 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.833+0000 7f01cf826b80 1 --2- 172.21.15.151:0/3037861085 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5616b124eeb0 0x5616b124f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:43.837 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.833+0000 7f01cf826b80 1 --2- 172.21.15.151:0/3037861085 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5616b124f7d0 0x5616b1249260 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:43.837 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.833+0000 7f01cf826b80 1 -- 172.21.15.151:0/3037861085 >> 172.21.15.151:0/3037861085 conn(0x5616b1177be0 msgr2=0x5616b1244010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:02:43.837 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.833+0000 7f01cf826b80 1 -- 172.21.15.151:0/3037861085 shutdown_connections 2024-05-21T22:02:43.837 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.833+0000 7f01cf826b80 1 -- 172.21.15.151:0/3037861085 wait complete. 2024-05-21T22:02:43.837 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.833+0000 7f01cf826b80 1 -- start start 2024-05-21T22:02:43.837 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.834+0000 7f01cf826b80 1 Processor -- start 2024-05-21T22:02:43.837 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.834+0000 7f01cf826b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5616b1249880 0x5616b1255450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:43.837 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.834+0000 7f01cf826b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5616b124eeb0 0x5616b12527f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:43.837 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.834+0000 7f01cf826b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5616b124f7d0 0x5616b1252d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:43.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.834+0000 7f01cf826b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5616b109dfa0 con 0x5616b124f7d0 2024-05-21T22:02:43.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.834+0000 7f01cf826b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5616b1138760 con 0x5616b124eeb0 2024-05-21T22:02:43.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.834+0000 7f01cf826b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5616b1256240 con 0x5616b1249880 2024-05-21T22:02:43.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.834+0000 7f01cf826b80 10 client.0 _ll_register_callbacks cb 0x5616b1141c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:02:43.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.835+0000 7f01be800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5616b124f7d0 0x5616b1252d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:43.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.835+0000 7f01b5400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5616b124eeb0 0x5616b12527f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:43.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.835+0000 7f01b5400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5616b124eeb0 0x5616b12527f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:42838/0 (socket says 172.21.15.151:42838) 2024-05-21T22:02:43.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.835+0000 7f01b5400700 1 -- 172.21.15.151:0/723065713 learned_addr learned my addr 172.21.15.151:0/723065713 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:02:43.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.835+0000 7f01bde00700 1 --2- 172.21.15.151:0/723065713 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5616b1249880 0x5616b1255450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:43.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.835+0000 7f01b5400700 1 -- 172.21.15.151:0/723065713 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5616b1249880 msgr2=0x5616b1255450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:43.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.835+0000 7f01b5400700 1 --2- 172.21.15.151:0/723065713 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5616b1249880 0x5616b1255450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:43.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.835+0000 7f01b5400700 1 -- 172.21.15.151:0/723065713 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5616b124f7d0 msgr2=0x5616b1252d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:43.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.835+0000 7f01b5400700 1 --2- 172.21.15.151:0/723065713 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5616b124f7d0 0x5616b1252d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:43.839 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.835+0000 7f01b5400700 1 -- 172.21.15.151:0/723065713 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616b10e78d0 con 0x5616b124eeb0 2024-05-21T22:02:43.839 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.835+0000 7f01b5400700 1 --2- 172.21.15.151:0/723065713 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5616b124eeb0 0x5616b12527f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f01a800f6b0 tx=0x7f01a8019660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b3aca1ef46ce94 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:02:43.839 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.836+0000 7f01a3e00700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-21T22:02:43.839 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.836+0000 7f01a3e00700 1 -- 172.21.15.151:0/723065713 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a8027020 con 0x5616b124eeb0 2024-05-21T22:02:43.839 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.836+0000 7f01cf826b80 1 -- 172.21.15.151:0/723065713 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01a8019ae0 con 0x5616b124eeb0 2024-05-21T22:02:43.839 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.836+0000 7f01a3e00700 1 -- 172.21.15.151:0/723065713 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8542+0+0 (secure 0 0 0) 0x7f01a8004450 con 0x5616b124eeb0 2024-05-21T22:02:43.839 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.836+0000 7f01a3e00700 1 -- 172.21.15.151:0/723065713 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f01a8032050 con 0x5616b124eeb0 2024-05-21T22:02:43.839 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.836+0000 7f01cf826b80 10 client.26912 fetch_fsmap learned FSMap version 29 2024-05-21T22:02:43.839 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.836+0000 7f01cf826b80 1 -- 172.21.15.151:0/723065713 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5616b1168060 con 0x5616b124eeb0 2024-05-21T22:02:43.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.837+0000 7f01a3e00700 1 -- 172.21.15.151:0/723065713 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01a8019ae0 con 0x5616b124eeb0 2024-05-21T22:02:43.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.837+0000 7f01a3e00700 1 -- 172.21.15.151:0/723065713 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f01a800f970 con 0x5616b124eeb0 2024-05-21T22:02:43.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.837+0000 7f01cf826b80 10 client.26912 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:02:43.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.837+0000 7f01cf826b80 10 client.26912 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-05-21T22:02:43.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.837+0000 7f01cf826b80 20 client.26912 populate_metadata read hostname 'smithi151' 2024-05-21T22:02:43.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.837+0000 7f01cf826b80 1 --2- 172.21.15.151:0/723065713 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x5616b1262cd0 0x5616b12650c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:43.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.837+0000 7f01cf826b80 4 client.26912 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T22:02:43.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.837+0000 7f01cf826b80 1 -- 172.21.15.151:0/723065713 --> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5616b11a9d90 con 0x5616b1262cd0 2024-05-21T22:02:43.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.837+0000 7f01bde00700 1 --2- 172.21.15.151:0/723065713 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x5616b1262cd0 0x5616b12650c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:43.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.838+0000 7f01bde00700 1 --2- 172.21.15.151:0/723065713 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x5616b1262cd0 0x5616b12650c0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6384660e84809a04 server_cookie=f42e7b7d5ea4b762 in_seq=0 out_seq=0 2024-05-21T22:02:43.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.838+0000 7f01a3e00700 10 client.26912 ms_handle_connect on v2:172.21.15.164:6834/2340428470 2024-05-21T22:02:43.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.839+0000 7f01a3e00700 1 -- 172.21.15.151:0/723065713 <== mds.4 v2:172.21.15.164:6834/2340428470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5616b11a9d90 con 0x5616b1262cd0 2024-05-21T22:02:43.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.839+0000 7f01a3e00700 10 client.26912 handle_command_reply: tid=0 2024-05-21T22:02:43.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.839+0000 7f01cf826b80 1 client.26912 shutdown 2024-05-21T22:02:43.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.839+0000 7f01cf826b80 1 -- 172.21.15.151:0/723065713 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5616b124eeb0 msgr2=0x5616b12527f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:43.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.839+0000 7f01cf826b80 1 --2- 172.21.15.151:0/723065713 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5616b124eeb0 0x5616b12527f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f01a800f6b0 tx=0x7f01a8019660 comp rx=0 tx=0).stop 2024-05-21T22:02:43.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.839+0000 7f01cf826b80 1 -- 172.21.15.151:0/723065713 shutdown_connections 2024-05-21T22:02:43.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.839+0000 7f01cf826b80 1 --2- 172.21.15.151:0/723065713 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x5616b1262cd0 0x5616b12650c0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:43.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.839+0000 7f01cf826b80 1 --2- 172.21.15.151:0/723065713 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5616b124f7d0 0x5616b1252d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:43.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.839+0000 7f01cf826b80 1 --2- 172.21.15.151:0/723065713 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5616b1249880 0x5616b1255450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:43.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.839+0000 7f01cf826b80 1 --2- 172.21.15.151:0/723065713 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5616b124eeb0 0x5616b12527f0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:43.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.839+0000 7f01cf826b80 1 -- 172.21.15.151:0/723065713 >> 172.21.15.151:0/723065713 conn(0x5616b1177be0 msgr2=0x5616b1248250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:02:43.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.839+0000 7f01a3e00700 0 client.26912 ms_handle_reset on v2:172.21.15.164:6834/2340428470 2024-05-21T22:02:43.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.839+0000 7f01cf826b80 1 -- 172.21.15.151:0/723065713 shutdown_connections 2024-05-21T22:02:43.843 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.839+0000 7f01cf826b80 1 -- 172.21.15.151:0/723065713 wait complete. 2024-05-21T22:02:43.843 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.840+0000 7f01cf826b80 20 client.26912 trim_cache size 0 max 16384 2024-05-21T22:02:43.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.856+0000 7f01cf826b80 1 Processor -- start 2024-05-21T22:02:43.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.856+0000 7f01cf826b80 1 -- start start 2024-05-21T22:02:43.858 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.857+0000 7f01cf826b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5616b1249880 0x5616b1249260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:43.858 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.857+0000 7f01cf826b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5616b124eeb0 0x5616b1249c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:43.858 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.857+0000 7f01cf826b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5616b124f7d0 0x5616b124c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:43.858 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.857+0000 7f01cf826b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f01c01aff10 con 0x5616b1249880 2024-05-21T22:02:43.858 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.857+0000 7f01cf826b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f01c01b1a50 con 0x5616b124eeb0 2024-05-21T22:02:43.858 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.857+0000 7f01cf826b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f01c01b05e0 con 0x5616b124f7d0 2024-05-21T22:02:43.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.857+0000 7f01bde00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5616b1249880 0x5616b1249260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:43.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.857+0000 7f01b5400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5616b124eeb0 0x5616b1249c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:43.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.857+0000 7f01bde00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5616b1249880 0x5616b1249260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:33292/0 (socket says 172.21.15.151:33292) 2024-05-21T22:02:43.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.857+0000 7f01bde00700 1 -- 172.21.15.151:0/2757242534 learned_addr learned my addr 172.21.15.151:0/2757242534 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:02:43.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.857+0000 7f01b5400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5616b124eeb0 0x5616b1249c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:42846/0 (socket says 172.21.15.151:42846) 2024-05-21T22:02:43.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.857+0000 7f01be800700 1 --2- 172.21.15.151:0/2757242534 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5616b124f7d0 0x5616b124c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:43.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.858+0000 7f01b5400700 1 -- 172.21.15.151:0/2757242534 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5616b124f7d0 msgr2=0x5616b124c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:43.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.858+0000 7f01b5400700 1 --2- 172.21.15.151:0/2757242534 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5616b124f7d0 0x5616b124c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:43.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.858+0000 7f01b5400700 1 -- 172.21.15.151:0/2757242534 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5616b1249880 msgr2=0x5616b1249260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:43.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.858+0000 7f01b5400700 1 --2- 172.21.15.151:0/2757242534 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5616b1249880 0x5616b1249260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:43.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.858+0000 7f01b5400700 1 -- 172.21.15.151:0/2757242534 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616b1255090 con 0x5616b124eeb0 2024-05-21T22:02:43.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.858+0000 7f01bde00700 1 --2- 172.21.15.151:0/2757242534 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5616b1249880 0x5616b1249260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:02:43.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.858+0000 7f01b5400700 1 --2- 172.21.15.151:0/2757242534 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5616b124eeb0 0x5616b1249c60 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f01a8009d50 tx=0x7f01a8011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=107895a97771520b server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:02:43.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.858+0000 7f01aea00700 1 -- 172.21.15.151:0/2757242534 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a8027030 con 0x5616b124eeb0 2024-05-21T22:02:43.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.858+0000 7f01aea00700 1 -- 172.21.15.151:0/2757242534 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01a801f040 con 0x5616b124eeb0 2024-05-21T22:02:43.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.858+0000 7f01aea00700 1 -- 172.21.15.151:0/2757242534 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a801e070 con 0x5616b124eeb0 2024-05-21T22:02:43.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.859+0000 7f01cf826b80 1 -- 172.21.15.151:0/2757242534 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5616b124eeb0 msgr2=0x5616b1249c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:43.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.859+0000 7f01cf826b80 1 --2- 172.21.15.151:0/2757242534 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5616b124eeb0 0x5616b1249c60 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f01a8009d50 tx=0x7f01a8011720 comp rx=0 tx=0).stop 2024-05-21T22:02:43.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.859+0000 7f01cf826b80 1 -- 172.21.15.151:0/2757242534 shutdown_connections 2024-05-21T22:02:43.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.859+0000 7f01cf826b80 1 --2- 172.21.15.151:0/2757242534 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5616b124f7d0 0x5616b124c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:43.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.859+0000 7f01cf826b80 1 --2- 172.21.15.151:0/2757242534 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5616b1249880 0x5616b1249260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:43.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.859+0000 7f01cf826b80 1 --2- 172.21.15.151:0/2757242534 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5616b124eeb0 0x5616b1249c60 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:43.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.859+0000 7f01cf826b80 1 -- 172.21.15.151:0/2757242534 >> 172.21.15.151:0/2757242534 conn(0x5616b1177be0 msgr2=0x5616b12527b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:02:43.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.859+0000 7f01cf826b80 1 -- 172.21.15.151:0/2757242534 shutdown_connections 2024-05-21T22:02:43.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.859+0000 7f01cf826b80 1 -- 172.21.15.151:0/2757242534 wait complete. 2024-05-21T22:02:43.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.860+0000 7f01cf826b80 1 -- start start 2024-05-21T22:02:43.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.860+0000 7f01cf826b80 1 Processor -- start 2024-05-21T22:02:43.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.861+0000 7f01cf826b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5616b1249880 0x5616b125b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:43.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.861+0000 7f01cf826b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5616b124eeb0 0x5616b125bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:43.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.861+0000 7f01cf826b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5616b124f7d0 0x5616b1269c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:43.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.861+0000 7f01cf826b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f01c01b1380 con 0x5616b124f7d0 2024-05-21T22:02:43.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.861+0000 7f01cf826b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f01c01b0cb0 con 0x5616b124eeb0 2024-05-21T22:02:43.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.861+0000 7f01cf826b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f01c01aded0 con 0x5616b1249880 2024-05-21T22:02:43.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.861+0000 7f01cf826b80 10 client.0 _ll_register_callbacks cb 0x5616b1141c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:02:43.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.861+0000 7f01bde00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5616b1249880 0x5616b125b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:43.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.861+0000 7f01bde00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5616b1249880 0x5616b125b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:51998/0 (socket says 172.21.15.151:51998) 2024-05-21T22:02:43.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.861+0000 7f01bde00700 1 -- 172.21.15.151:0/2064292957 learned_addr learned my addr 172.21.15.151:0/2064292957 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:02:43.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.861+0000 7f01b5400700 1 --2- 172.21.15.151:0/2064292957 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5616b124eeb0 0x5616b125bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:43.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.861+0000 7f01be800700 1 --2- 172.21.15.151:0/2064292957 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5616b124f7d0 0x5616b1269c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:43.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.861+0000 7f01bde00700 1 -- 172.21.15.151:0/2064292957 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5616b124eeb0 msgr2=0x5616b125bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:43.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.861+0000 7f01bde00700 1 --2- 172.21.15.151:0/2064292957 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5616b124eeb0 0x5616b125bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:43.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.861+0000 7f01bde00700 1 -- 172.21.15.151:0/2064292957 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5616b124f7d0 msgr2=0x5616b1269c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:43.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.861+0000 7f01bde00700 1 --2- 172.21.15.151:0/2064292957 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5616b124f7d0 0x5616b1269c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:43.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.861+0000 7f01bde00700 1 -- 172.21.15.151:0/2064292957 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616b126a370 con 0x5616b1249880 2024-05-21T22:02:43.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.861+0000 7f01be800700 1 --2- 172.21.15.151:0/2064292957 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5616b124f7d0 0x5616b1269c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T22:02:43.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.862+0000 7f01bde00700 1 --2- 172.21.15.151:0/2064292957 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5616b1249880 0x5616b125b710 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f01b0009e60 tx=0x7f01b001da70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f620dc17f623b58a server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:02:43.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.862+0000 7f01a3e00700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-21T22:02:43.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.862+0000 7f01a3e00700 1 -- 172.21.15.151:0/2064292957 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b00aad10 con 0x5616b1249880 2024-05-21T22:02:43.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.862+0000 7f01cf826b80 1 -- 172.21.15.151:0/2064292957 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01a801f040 con 0x5616b1249880 2024-05-21T22:02:43.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.862+0000 7f01a3e00700 1 -- 172.21.15.151:0/2064292957 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8542+0+0 (secure 0 0 0) 0x7f01b00cc6b0 con 0x5616b1249880 2024-05-21T22:02:43.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.862+0000 7f01a3e00700 1 -- 172.21.15.151:0/2064292957 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f01b00af0c0 con 0x5616b1249880 2024-05-21T22:02:43.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.862+0000 7f01cf826b80 10 client.26929 fetch_fsmap learned FSMap version 29 2024-05-21T22:02:43.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.862+0000 7f01cf826b80 1 -- 172.21.15.151:0/2064292957 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5616b12cd200 con 0x5616b1249880 2024-05-21T22:02:43.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.863+0000 7f01a3e00700 1 -- 172.21.15.151:0/2064292957 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01a801f040 con 0x5616b1249880 2024-05-21T22:02:43.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.863+0000 7f01a3e00700 1 -- 172.21.15.151:0/2064292957 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f01b008f050 con 0x5616b1249880 2024-05-21T22:02:43.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.863+0000 7f01cf826b80 10 client.26929 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:02:43.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.863+0000 7f01cf826b80 10 client.26929 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-05-21T22:02:43.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.863+0000 7f01cf826b80 20 client.26929 populate_metadata read hostname 'smithi151' 2024-05-21T22:02:43.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.863+0000 7f01cf826b80 1 --2- 172.21.15.151:0/2064292957 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x5616b12cd910 0x5616b12cfd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:43.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.863+0000 7f01cf826b80 4 client.26929 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T22:02:43.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.863+0000 7f01cf826b80 1 -- 172.21.15.151:0/2064292957 --> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5616b11a3570 con 0x5616b12cd910 2024-05-21T22:02:43.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.863+0000 7f01b5400700 1 --2- 172.21.15.151:0/2064292957 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x5616b12cd910 0x5616b12cfd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:43.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.864+0000 7f01b5400700 1 --2- 172.21.15.151:0/2064292957 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x5616b12cd910 0x5616b12cfd30 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=43bd27fb60244ba2 server_cookie=ec0fe6fadfcef9d5 in_seq=0 out_seq=0 2024-05-21T22:02:43.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.864+0000 7f01a3e00700 10 client.26929 ms_handle_connect on v2:172.21.15.164:6834/2340428470 2024-05-21T22:02:43.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.864+0000 7f01a3e00700 1 -- 172.21.15.151:0/2064292957 <== mds.4 v2:172.21.15.164:6834/2340428470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5616b11a3570 con 0x5616b12cd910 2024-05-21T22:02:43.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.864+0000 7f01a3e00700 10 client.26929 handle_command_reply: tid=0 2024-05-21T22:02:43.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.864+0000 7f01cf826b80 1 client.26929 shutdown 2024-05-21T22:02:43.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.864+0000 7f01cf826b80 1 -- 172.21.15.151:0/2064292957 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5616b1249880 msgr2=0x5616b125b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:43.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.864+0000 7f01cf826b80 1 --2- 172.21.15.151:0/2064292957 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5616b1249880 0x5616b125b710 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f01b0009e60 tx=0x7f01b001da70 comp rx=0 tx=0).stop 2024-05-21T22:02:43.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.864+0000 7f01cf826b80 1 -- 172.21.15.151:0/2064292957 shutdown_connections 2024-05-21T22:02:43.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.864+0000 7f01cf826b80 1 --2- 172.21.15.151:0/2064292957 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x5616b12cd910 0x5616b12cfd30 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:43.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.864+0000 7f01cf826b80 1 --2- 172.21.15.151:0/2064292957 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5616b124f7d0 0x5616b1269c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:43.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.864+0000 7f01cf826b80 1 --2- 172.21.15.151:0/2064292957 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5616b1249880 0x5616b125b710 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:43.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.864+0000 7f01cf826b80 1 --2- 172.21.15.151:0/2064292957 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5616b124eeb0 0x5616b125bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:43.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.864+0000 7f01cf826b80 1 -- 172.21.15.151:0/2064292957 >> 172.21.15.151:0/2064292957 conn(0x5616b1177be0 msgr2=0x5616b124c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:02:43.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.864+0000 7f01a3e00700 0 client.26929 ms_handle_reset on v2:172.21.15.164:6834/2340428470 2024-05-21T22:02:43.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.865+0000 7f01cf826b80 1 -- 172.21.15.151:0/2064292957 shutdown_connections 2024-05-21T22:02:43.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.865+0000 7f01cf826b80 1 -- 172.21.15.151:0/2064292957 wait complete. 2024-05-21T22:02:43.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.865+0000 7f01cf826b80 20 client.26929 trim_cache size 0 max 16384 2024-05-21T22:02:43.868 INFO:teuthology.orchestra.run.smithi151.stdout:[] 2024-05-21T22:02:43.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.866+0000 7f01c4e00700 1 -- 172.21.15.151:0/848010184 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f01a40b29f0 msgr2=0x7f01a40b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:43.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.866+0000 7f01c4e00700 1 --2- 172.21.15.151:0/848010184 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f01a40b29f0 0x7f01a40b4eb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f01b80097a0 tx=0x7f01b80099b0 comp rx=0 tx=0).stop 2024-05-21T22:02:43.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.866+0000 7f01c4e00700 1 -- 172.21.15.151:0/848010184 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f01c01037f0 msgr2=0x7f01c019be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:43.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.866+0000 7f01c4e00700 1 --2- 172.21.15.151:0/848010184 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f01c01037f0 0x7f01c019be00 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f01b000b780 tx=0x7f01b0004d00 comp rx=0 tx=0).stop 2024-05-21T22:02:43.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.866+0000 7f01c4e00700 1 -- 172.21.15.151:0/848010184 shutdown_connections 2024-05-21T22:02:43.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.866+0000 7f01c4e00700 1 --2- 172.21.15.151:0/848010184 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f01a40b29f0 0x7f01a40b4eb0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:43.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.866+0000 7f01c4e00700 1 --2- 172.21.15.151:0/848010184 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f01c01037f0 0x7f01c019be00 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:43.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.866+0000 7f01c4e00700 1 --2- 172.21.15.151:0/848010184 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01c01088f0 0x7f01c01a06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:43.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.866+0000 7f01c4e00700 1 --2- 172.21.15.151:0/848010184 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f01c01041a0 0x7f01c019c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:43.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.866+0000 7f01c4e00700 1 -- 172.21.15.151:0/848010184 >> 172.21.15.151:0/848010184 conn(0x7f01c007a6a0 msgr2=0x7f01c01019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:02:43.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.866+0000 7f01c4e00700 1 -- 172.21.15.151:0/848010184 shutdown_connections 2024-05-21T22:02:43.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:43.866+0000 7f01c4e00700 1 -- 172.21.15.151:0/848010184 wait complete. 2024-05-21T22:02:43.985 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:02:43 smithi184 ceph-mon[36942]: pgmap v218: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-05-21T22:02:44.719 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-21T22:02:45.161 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:02:45.600 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:02:45 smithi151 ceph-mon[31623]: pgmap v219: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-05-21T22:02:45.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:02:45 smithi164 ceph-mon[36269]: pgmap v219: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-05-21T22:02:45.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:02:45 smithi184 ceph-mon[36942]: pgmap v219: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-05-21T22:02:47.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.444+0000 7fa651e00700 1 -- 172.21.15.151:0/3395847979 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa64c1088f0 msgr2=0x7fa64c110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:47.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.444+0000 7fa651e00700 1 --2- 172.21.15.151:0/3395847979 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa64c1088f0 0x7fa64c110470 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fa64400b0d0 tx=0x7fa64400b3e0 comp rx=0 tx=0).stop 2024-05-21T22:02:47.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.444+0000 7fa651e00700 1 -- 172.21.15.151:0/3395847979 shutdown_connections 2024-05-21T22:02:47.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.444+0000 7fa651e00700 1 --2- 172.21.15.151:0/3395847979 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa64c1037f0 0x7fa64c103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:47.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.444+0000 7fa651e00700 1 --2- 172.21.15.151:0/3395847979 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa64c1088f0 0x7fa64c110470 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:47.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.444+0000 7fa651e00700 1 --2- 172.21.15.151:0/3395847979 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa64c1041a0 0x7fa64c1081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:47.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.444+0000 7fa651e00700 1 -- 172.21.15.151:0/3395847979 >> 172.21.15.151:0/3395847979 conn(0x7fa64c07a6a0 msgr2=0x7fa64c07aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:02:47.446 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.444+0000 7fa651e00700 1 -- 172.21.15.151:0/3395847979 shutdown_connections 2024-05-21T22:02:47.446 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.444+0000 7fa651e00700 1 -- 172.21.15.151:0/3395847979 wait complete. 2024-05-21T22:02:47.446 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.445+0000 7fa651e00700 1 Processor -- start 2024-05-21T22:02:47.447 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.446+0000 7fa651e00700 1 -- start start 2024-05-21T22:02:47.447 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.446+0000 7fa651e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa64c1037f0 0x7fa64c19bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:47.447 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.446+0000 7fa651e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa64c1041a0 0x7fa64c19c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:47.447 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.446+0000 7fa651e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa64c1088f0 0x7fa64c1a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:47.447 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.446+0000 7fa651e00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa64c113710 con 0x7fa64c1041a0 2024-05-21T22:02:47.447 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.446+0000 7fa651e00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa64c113590 con 0x7fa64c1037f0 2024-05-21T22:02:47.447 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.446+0000 7fa651e00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa64c113890 con 0x7fa64c1088f0 2024-05-21T22:02:47.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.447+0000 7fa64aa00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa64c1037f0 0x7fa64c19bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:47.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.447+0000 7fa64a000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa64c1041a0 0x7fa64c19c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:47.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.447+0000 7fa64a000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa64c1041a0 0x7fa64c19c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:39420/0 (socket says 172.21.15.151:39420) 2024-05-21T22:02:47.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.447+0000 7fa64a000700 1 -- 172.21.15.151:0/4264878659 learned_addr learned my addr 172.21.15.151:0/4264878659 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:02:47.449 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.447+0000 7fa64b400700 1 --2- 172.21.15.151:0/4264878659 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa64c1088f0 0x7fa64c1a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:47.449 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.447+0000 7fa64aa00700 1 -- 172.21.15.151:0/4264878659 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa64c1088f0 msgr2=0x7fa64c1a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:47.449 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.447+0000 7fa64aa00700 1 --2- 172.21.15.151:0/4264878659 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa64c1088f0 0x7fa64c1a0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:47.449 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.447+0000 7fa64aa00700 1 -- 172.21.15.151:0/4264878659 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa64c1041a0 msgr2=0x7fa64c19c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:47.449 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.447+0000 7fa64aa00700 1 --2- 172.21.15.151:0/4264878659 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa64c1041a0 0x7fa64c19c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:47.449 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.447+0000 7fa64aa00700 1 -- 172.21.15.151:0/4264878659 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa64c1a0d50 con 0x7fa64c1037f0 2024-05-21T22:02:47.449 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.447+0000 7fa64a000700 1 --2- 172.21.15.151:0/4264878659 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa64c1041a0 0x7fa64c19c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:02:47.449 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.448+0000 7fa64aa00700 1 --2- 172.21.15.151:0/4264878659 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa64c1037f0 0x7fa64c19bd40 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa63c00e9f0 tx=0x7fa63c00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:02:47.449 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.448+0000 7fa650a00700 1 -- 172.21.15.151:0/4264878659 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa63c00c9c0 con 0x7fa64c1037f0 2024-05-21T22:02:47.449 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.448+0000 7fa650a00700 1 -- 172.21.15.151:0/4264878659 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa63c00cb60 con 0x7fa64c1037f0 2024-05-21T22:02:47.449 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.448+0000 7fa651e00700 1 -- 172.21.15.151:0/4264878659 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa64c1a1040 con 0x7fa64c1037f0 2024-05-21T22:02:47.449 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.448+0000 7fa650a00700 1 -- 172.21.15.151:0/4264878659 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa63c018680 con 0x7fa64c1037f0 2024-05-21T22:02:47.449 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.448+0000 7fa651e00700 1 -- 172.21.15.151:0/4264878659 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa64c1a88c0 con 0x7fa64c1037f0 2024-05-21T22:02:47.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.452+0000 7fa650a00700 1 -- 172.21.15.151:0/4264878659 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7fa63c010430 con 0x7fa64c1037f0 2024-05-21T22:02:47.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.454+0000 7fa650a00700 1 --2- 172.21.15.151:0/4264878659 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fa6300b29f0 0x7fa6300b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:47.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.454+0000 7fa650a00700 1 -- 172.21.15.151:0/4264878659 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8542+0+0 (secure 0 0 0) 0x7fa63c0e27e0 con 0x7fa64c1037f0 2024-05-21T22:02:47.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.454+0000 7fa64a000700 1 --2- 172.21.15.151:0/4264878659 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fa6300b29f0 0x7fa6300b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:47.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.455+0000 7fa64a000700 1 --2- 172.21.15.151:0/4264878659 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fa6300b29f0 0x7fa6300b4eb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa64c110820 tx=0x7fa638009260 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:02:47.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.456+0000 7fa65c7eab80 1 Processor -- start 2024-05-21T22:02:47.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.456+0000 7fa65c7eab80 1 -- start start 2024-05-21T22:02:47.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.456+0000 7fa65c7eab80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5610b5b6deb0 0x5610b5b6e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:47.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.456+0000 7fa65c7eab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5610b5b6e7d0 0x5610b5b68260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:47.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.456+0000 7fa65c7eab80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5610b5b68880 0x5610b5b68ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:47.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.457+0000 7fa65c7eab80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5610b5ac6e40 con 0x5610b5b68880 2024-05-21T22:02:47.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.457+0000 7fa65c7eab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5610b5ab69e0 con 0x5610b5b6e7d0 2024-05-21T22:02:47.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.457+0000 7fa65c7eab80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5610b5abaa20 con 0x5610b5b6deb0 2024-05-21T22:02:47.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.457+0000 7fa64aa00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5610b5b6deb0 0x5610b5b6e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:47.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.457+0000 7fa64a000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5610b5b6e7d0 0x5610b5b68260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:47.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.457+0000 7fa64aa00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5610b5b6deb0 0x5610b5b6e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:46776/0 (socket says 172.21.15.151:46776) 2024-05-21T22:02:47.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.457+0000 7fa64aa00700 1 -- 172.21.15.151:0/3810795106 learned_addr learned my addr 172.21.15.151:0/3810795106 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:02:47.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.457+0000 7fa64b400700 1 --2- 172.21.15.151:0/3810795106 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5610b5b68880 0x5610b5b68ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:47.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.457+0000 7fa64a000700 1 -- 172.21.15.151:0/3810795106 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5610b5b6deb0 msgr2=0x5610b5b6e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:47.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.457+0000 7fa64a000700 1 --2- 172.21.15.151:0/3810795106 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5610b5b6deb0 0x5610b5b6e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:47.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.457+0000 7fa64a000700 1 -- 172.21.15.151:0/3810795106 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5610b5b68880 msgr2=0x5610b5b68ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:47.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.457+0000 7fa64a000700 1 --2- 172.21.15.151:0/3810795106 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5610b5b68880 0x5610b5b68ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:47.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.457+0000 7fa64a000700 1 -- 172.21.15.151:0/3810795106 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610b5a7f420 con 0x5610b5b6e7d0 2024-05-21T22:02:47.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.457+0000 7fa64b400700 1 --2- 172.21.15.151:0/3810795106 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5610b5b68880 0x5610b5b68ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T22:02:47.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.458+0000 7fa64a000700 1 --2- 172.21.15.151:0/3810795106 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5610b5b6e7d0 0x5610b5b68260 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa638011c70 tx=0x7fa638014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eaebcb0711110821 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:02:47.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.458+0000 7fa640c00700 1 -- 172.21.15.151:0/3810795106 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa638004830 con 0x5610b5b6e7d0 2024-05-21T22:02:47.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.458+0000 7fa640c00700 1 -- 172.21.15.151:0/3810795106 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa638005020 con 0x5610b5b6e7d0 2024-05-21T22:02:47.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.458+0000 7fa640c00700 1 -- 172.21.15.151:0/3810795106 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa63801c780 con 0x5610b5b6e7d0 2024-05-21T22:02:47.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.459+0000 7fa65c7eab80 1 -- 172.21.15.151:0/3810795106 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5610b5b6e7d0 msgr2=0x5610b5b68260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:47.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.459+0000 7fa65c7eab80 1 --2- 172.21.15.151:0/3810795106 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5610b5b6e7d0 0x5610b5b68260 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa638011c70 tx=0x7fa638014000 comp rx=0 tx=0).stop 2024-05-21T22:02:47.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.459+0000 7fa65c7eab80 1 -- 172.21.15.151:0/3810795106 shutdown_connections 2024-05-21T22:02:47.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.459+0000 7fa65c7eab80 1 --2- 172.21.15.151:0/3810795106 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5610b5b68880 0x5610b5b68ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:47.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.459+0000 7fa65c7eab80 1 --2- 172.21.15.151:0/3810795106 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5610b5b6deb0 0x5610b5b6e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:47.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.459+0000 7fa65c7eab80 1 --2- 172.21.15.151:0/3810795106 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5610b5b6e7d0 0x5610b5b68260 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:47.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.459+0000 7fa65c7eab80 1 -- 172.21.15.151:0/3810795106 >> 172.21.15.151:0/3810795106 conn(0x5610b5a96be0 msgr2=0x5610b5b63010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:02:47.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.459+0000 7fa65c7eab80 1 -- 172.21.15.151:0/3810795106 shutdown_connections 2024-05-21T22:02:47.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.459+0000 7fa65c7eab80 1 -- 172.21.15.151:0/3810795106 wait complete. 2024-05-21T22:02:47.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.460+0000 7fa65c7eab80 1 -- start start 2024-05-21T22:02:47.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.460+0000 7fa65c7eab80 1 Processor -- start 2024-05-21T22:02:47.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.461+0000 7fa65c7eab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5610b5b68880 0x5610b5b744f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:47.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.461+0000 7fa65c7eab80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5610b5b6deb0 0x5610b5b71950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:47.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.461+0000 7fa65c7eab80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5610b5b6e7d0 0x5610b5b71e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:47.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.461+0000 7fa65c7eab80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5610b59bcfa0 con 0x5610b5b6e7d0 2024-05-21T22:02:47.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.461+0000 7fa65c7eab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5610b5a57760 con 0x5610b5b68880 2024-05-21T22:02:47.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.461+0000 7fa65c7eab80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5610b5b75240 con 0x5610b5b6deb0 2024-05-21T22:02:47.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.461+0000 7fa65c7eab80 10 client.0 _ll_register_callbacks cb 0x5610b5a60c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:02:47.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.461+0000 7fa64b400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5610b5b6e7d0 0x5610b5b71e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:47.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.461+0000 7fa64aa00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5610b5b68880 0x5610b5b744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:47.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.461+0000 7fa64b400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5610b5b6e7d0 0x5610b5b71e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:39444/0 (socket says 172.21.15.151:39444) 2024-05-21T22:02:47.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.461+0000 7fa64a000700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5610b5b6deb0 0x5610b5b71950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:47.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.461+0000 7fa64b400700 1 -- 172.21.15.151:0/1238512364 learned_addr learned my addr 172.21.15.151:0/1238512364 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:02:47.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.461+0000 7fa64aa00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5610b5b68880 0x5610b5b744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:55966/0 (socket says 172.21.15.151:55966) 2024-05-21T22:02:47.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.461+0000 7fa64aa00700 1 -- 172.21.15.151:0/1238512364 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5610b5b6deb0 msgr2=0x5610b5b71950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:47.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.461+0000 7fa64aa00700 1 --2- 172.21.15.151:0/1238512364 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5610b5b6deb0 0x5610b5b71950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:47.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.461+0000 7fa64aa00700 1 -- 172.21.15.151:0/1238512364 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5610b5b6e7d0 msgr2=0x5610b5b71e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:47.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.461+0000 7fa64aa00700 1 --2- 172.21.15.151:0/1238512364 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5610b5b6e7d0 0x5610b5b71e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:47.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.461+0000 7fa64aa00700 1 -- 172.21.15.151:0/1238512364 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610b5a068d0 con 0x5610b5b68880 2024-05-21T22:02:47.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.461+0000 7fa64a000700 1 --2- 172.21.15.151:0/1238512364 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5610b5b6deb0 0x5610b5b71950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:02:47.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.462+0000 7fa64aa00700 1 --2- 172.21.15.151:0/1238512364 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5610b5b68880 0x5610b5b744f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa63c0a99e0 tx=0x7fa63c0a9f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c857b3654a29e82f server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:02:47.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.462+0000 7fa635600700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-21T22:02:47.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.462+0000 7fa635600700 1 -- 172.21.15.151:0/1238512364 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa63c0a5040 con 0x5610b5b68880 2024-05-21T22:02:47.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.462+0000 7fa635600700 1 -- 172.21.15.151:0/1238512364 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8542+0+0 (secure 0 0 0) 0x7fa63c0cc2d0 con 0x5610b5b68880 2024-05-21T22:02:47.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.462+0000 7fa65c7eab80 1 -- 172.21.15.151:0/1238512364 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa638005020 con 0x5610b5b68880 2024-05-21T22:02:47.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.462+0000 7fa635600700 1 -- 172.21.15.151:0/1238512364 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa63c0b21f0 con 0x5610b5b68880 2024-05-21T22:02:47.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.462+0000 7fa65c7eab80 10 client.26930 fetch_fsmap learned FSMap version 29 2024-05-21T22:02:47.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.462+0000 7fa65c7eab80 1 -- 172.21.15.151:0/1238512364 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5610b5a87060 con 0x5610b5b68880 2024-05-21T22:02:47.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.463+0000 7fa635600700 1 -- 172.21.15.151:0/1238512364 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa638005020 con 0x5610b5b68880 2024-05-21T22:02:47.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.463+0000 7fa635600700 1 -- 172.21.15.151:0/1238512364 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fa63c06f2b0 con 0x5610b5b68880 2024-05-21T22:02:47.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.463+0000 7fa65c7eab80 10 client.26930 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:02:47.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.463+0000 7fa65c7eab80 10 client.26930 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-21T22:02:47.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.463+0000 7fa65c7eab80 20 client.26930 populate_metadata read hostname 'smithi151' 2024-05-21T22:02:47.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.463+0000 7fa65c7eab80 1 --2- 172.21.15.151:0/1238512364 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x5610b5b81ea0 0x5610b5b84290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:47.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.463+0000 7fa65c7eab80 4 client.26930 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T22:02:47.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.463+0000 7fa65c7eab80 1 -- 172.21.15.151:0/1238512364 --> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5610b5ac8d90 con 0x5610b5b81ea0 2024-05-21T22:02:47.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.464+0000 7fa64a000700 1 --2- 172.21.15.151:0/1238512364 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x5610b5b81ea0 0x5610b5b84290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:47.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.464+0000 7fa64a000700 1 --2- 172.21.15.151:0/1238512364 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x5610b5b81ea0 0x5610b5b84290 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=54c02f81d7de9f1c server_cookie=5068313a0f5041d7 in_seq=0 out_seq=0 2024-05-21T22:02:47.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.464+0000 7fa635600700 10 client.26930 ms_handle_connect on v2:172.21.15.184:6836/2988194888 2024-05-21T22:02:47.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.465+0000 7fa635600700 1 -- 172.21.15.151:0/1238512364 <== mds.3 v2:172.21.15.184:6836/2988194888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5610b5ac8d90 con 0x5610b5b81ea0 2024-05-21T22:02:47.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.465+0000 7fa635600700 10 client.26930 handle_command_reply: tid=0 2024-05-21T22:02:47.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.465+0000 7fa65c7eab80 1 client.26930 shutdown 2024-05-21T22:02:47.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.465+0000 7fa65c7eab80 1 -- 172.21.15.151:0/1238512364 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5610b5b68880 msgr2=0x5610b5b744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:47.467 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.465+0000 7fa65c7eab80 1 --2- 172.21.15.151:0/1238512364 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5610b5b68880 0x5610b5b744f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa63c0a99e0 tx=0x7fa63c0a9f90 comp rx=0 tx=0).stop 2024-05-21T22:02:47.467 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.465+0000 7fa65c7eab80 1 -- 172.21.15.151:0/1238512364 shutdown_connections 2024-05-21T22:02:47.467 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.465+0000 7fa65c7eab80 1 --2- 172.21.15.151:0/1238512364 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x5610b5b81ea0 0x5610b5b84290 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:47.467 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.465+0000 7fa65c7eab80 1 --2- 172.21.15.151:0/1238512364 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5610b5b68880 0x5610b5b744f0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:47.467 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.465+0000 7fa635600700 0 client.26930 ms_handle_reset on v2:172.21.15.184:6836/2988194888 2024-05-21T22:02:47.467 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.465+0000 7fa65c7eab80 1 --2- 172.21.15.151:0/1238512364 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5610b5b6e7d0 0x5610b5b71e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:47.467 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.465+0000 7fa65c7eab80 1 --2- 172.21.15.151:0/1238512364 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5610b5b6deb0 0x5610b5b71950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:47.467 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.465+0000 7fa65c7eab80 1 -- 172.21.15.151:0/1238512364 >> 172.21.15.151:0/1238512364 conn(0x5610b5a96be0 msgr2=0x5610b5b67250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:02:47.467 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.466+0000 7fa65c7eab80 1 -- 172.21.15.151:0/1238512364 shutdown_connections 2024-05-21T22:02:47.467 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.466+0000 7fa65c7eab80 1 -- 172.21.15.151:0/1238512364 wait complete. 2024-05-21T22:02:47.467 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.466+0000 7fa65c7eab80 20 client.26930 trim_cache size 0 max 16384 2024-05-21T22:02:47.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.483+0000 7fa65c7eab80 1 Processor -- start 2024-05-21T22:02:47.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.483+0000 7fa65c7eab80 1 -- start start 2024-05-21T22:02:47.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.483+0000 7fa65c7eab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5610b5b68880 0x5610b5b68260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:47.484 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.483+0000 7fa65c7eab80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5610b5b6deb0 0x5610b5b6c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:47.484 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.483+0000 7fa65c7eab80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5610b5b6e7d0 0x5610b5b73d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:47.484 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.483+0000 7fa65c7eab80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa64c1b0070 con 0x5610b5b6deb0 2024-05-21T22:02:47.484 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.483+0000 7fa65c7eab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa64c1b1bb0 con 0x5610b5b68880 2024-05-21T22:02:47.484 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.483+0000 7fa65c7eab80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa64c1b0740 con 0x5610b5b6e7d0 2024-05-21T22:02:47.484 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.483+0000 7fa64a000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5610b5b6deb0 0x5610b5b6c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:47.485 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.484+0000 7fa64aa00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5610b5b68880 0x5610b5b68260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:47.485 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.484+0000 7fa64aa00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5610b5b68880 0x5610b5b68260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:55974/0 (socket says 172.21.15.151:55974) 2024-05-21T22:02:47.485 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.484+0000 7fa64aa00700 1 -- 172.21.15.151:0/451730964 learned_addr learned my addr 172.21.15.151:0/451730964 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:02:47.485 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.484+0000 7fa64b400700 1 --2- 172.21.15.151:0/451730964 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5610b5b6e7d0 0x5610b5b73d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:47.485 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.484+0000 7fa64aa00700 1 -- 172.21.15.151:0/451730964 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5610b5b6e7d0 msgr2=0x5610b5b73d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:47.485 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.484+0000 7fa64aa00700 1 --2- 172.21.15.151:0/451730964 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5610b5b6e7d0 0x5610b5b73d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:47.485 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.484+0000 7fa64aa00700 1 -- 172.21.15.151:0/451730964 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5610b5b6deb0 msgr2=0x5610b5b6c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:47.485 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.484+0000 7fa64aa00700 1 --2- 172.21.15.151:0/451730964 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5610b5b6deb0 0x5610b5b6c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:47.485 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.484+0000 7fa64aa00700 1 -- 172.21.15.151:0/451730964 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610b5b72860 con 0x5610b5b68880 2024-05-21T22:02:47.485 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.484+0000 7fa64aa00700 1 --2- 172.21.15.151:0/451730964 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5610b5b68880 0x5610b5b68260 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fa63c0a9750 tx=0x7fa63c009790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ba658973d6c9750 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:02:47.486 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.484+0000 7fa637e00700 1 -- 172.21.15.151:0/451730964 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa63c0a9560 con 0x5610b5b68880 2024-05-21T22:02:47.486 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.485+0000 7fa637e00700 1 -- 172.21.15.151:0/451730964 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa63c0b2040 con 0x5610b5b68880 2024-05-21T22:02:47.486 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.485+0000 7fa637e00700 1 -- 172.21.15.151:0/451730964 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa63c0aa150 con 0x5610b5b68880 2024-05-21T22:02:47.487 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.485+0000 7fa65c7eab80 1 -- 172.21.15.151:0/451730964 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5610b5b68880 msgr2=0x5610b5b68260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:47.487 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.485+0000 7fa65c7eab80 1 --2- 172.21.15.151:0/451730964 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5610b5b68880 0x5610b5b68260 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fa63c0a9750 tx=0x7fa63c009790 comp rx=0 tx=0).stop 2024-05-21T22:02:47.487 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.485+0000 7fa65c7eab80 1 -- 172.21.15.151:0/451730964 shutdown_connections 2024-05-21T22:02:47.487 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.485+0000 7fa65c7eab80 1 --2- 172.21.15.151:0/451730964 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5610b5b68880 0x5610b5b68260 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:47.487 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.485+0000 7fa65c7eab80 1 --2- 172.21.15.151:0/451730964 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5610b5b6e7d0 0x5610b5b73d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:47.487 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.485+0000 7fa65c7eab80 1 --2- 172.21.15.151:0/451730964 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5610b5b6deb0 0x5610b5b6c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:47.487 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.486+0000 7fa65c7eab80 1 -- 172.21.15.151:0/451730964 >> 172.21.15.151:0/451730964 conn(0x5610b5a96be0 msgr2=0x5610b5b6b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:02:47.487 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.486+0000 7fa65c7eab80 1 -- 172.21.15.151:0/451730964 shutdown_connections 2024-05-21T22:02:47.487 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.486+0000 7fa65c7eab80 1 -- 172.21.15.151:0/451730964 wait complete. 2024-05-21T22:02:47.487 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.486+0000 7fa65c7eab80 1 -- start start 2024-05-21T22:02:47.487 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.486+0000 7fa65c7eab80 1 Processor -- start 2024-05-21T22:02:47.489 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.487+0000 7fa65c7eab80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5610b5b68880 0x5610b5b7aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:47.489 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.487+0000 7fa65c7eab80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5610b5b6deb0 0x5610b5b89200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:47.489 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.487+0000 7fa65c7eab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5610b5b6e7d0 0x5610b5b89740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:47.489 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.487+0000 7fa65c7eab80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa64c1b14e0 con 0x5610b5b68880 2024-05-21T22:02:47.489 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.487+0000 7fa65c7eab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa64c1b0e10 con 0x5610b5b6e7d0 2024-05-21T22:02:47.489 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.487+0000 7fa65c7eab80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa64c1ae030 con 0x5610b5b6deb0 2024-05-21T22:02:47.489 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.487+0000 7fa65c7eab80 10 client.0 _ll_register_callbacks cb 0x5610b5a60c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:02:47.489 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.487+0000 7fa64aa00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5610b5b68880 0x5610b5b7aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:47.489 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.487+0000 7fa64a000700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5610b5b6deb0 0x5610b5b89200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:47.489 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.487+0000 7fa64aa00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5610b5b68880 0x5610b5b7aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:39466/0 (socket says 172.21.15.151:39466) 2024-05-21T22:02:47.489 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.487+0000 7fa64aa00700 1 -- 172.21.15.151:0/1762367564 learned_addr learned my addr 172.21.15.151:0/1762367564 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:02:47.489 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.487+0000 7fa64b400700 1 --2- 172.21.15.151:0/1762367564 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5610b5b6e7d0 0x5610b5b89740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:47.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.488+0000 7fa64a000700 1 -- 172.21.15.151:0/1762367564 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5610b5b6e7d0 msgr2=0x5610b5b89740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:47.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.488+0000 7fa64a000700 1 --2- 172.21.15.151:0/1762367564 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5610b5b6e7d0 0x5610b5b89740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:47.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.488+0000 7fa64a000700 1 -- 172.21.15.151:0/1762367564 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5610b5b68880 msgr2=0x5610b5b7aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:47.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.488+0000 7fa64a000700 1 --2- 172.21.15.151:0/1762367564 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5610b5b68880 0x5610b5b7aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:47.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.488+0000 7fa64a000700 1 -- 172.21.15.151:0/1762367564 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610b5b89e00 con 0x5610b5b6deb0 2024-05-21T22:02:47.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.488+0000 7fa64aa00700 1 --2- 172.21.15.151:0/1762367564 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5610b5b68880 0x5610b5b7aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:02:47.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.488+0000 7fa64a000700 1 --2- 172.21.15.151:0/1762367564 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5610b5b6deb0 0x5610b5b89200 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x5610b5b5b1f0 tx=0x7fa638009f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2e3cf9729180184 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:02:47.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.488+0000 7fa635600700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-21T22:02:47.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.488+0000 7fa635600700 1 -- 172.21.15.151:0/1762367564 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa638036070 con 0x5610b5b6deb0 2024-05-21T22:02:47.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.489+0000 7fa635600700 1 -- 172.21.15.151:0/1762367564 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8542+0+0 (secure 0 0 0) 0x7fa638005330 con 0x5610b5b6deb0 2024-05-21T22:02:47.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.489+0000 7fa65c7eab80 1 -- 172.21.15.151:0/1762367564 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa63c0b2040 con 0x5610b5b6deb0 2024-05-21T22:02:47.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.489+0000 7fa635600700 1 -- 172.21.15.151:0/1762367564 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa638042050 con 0x5610b5b6deb0 2024-05-21T22:02:47.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.489+0000 7fa65c7eab80 10 client.26947 fetch_fsmap learned FSMap version 29 2024-05-21T22:02:47.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.489+0000 7fa65c7eab80 1 -- 172.21.15.151:0/1762367564 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5610b5bf5080 con 0x5610b5b6deb0 2024-05-21T22:02:47.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.489+0000 7fa635600700 1 -- 172.21.15.151:0/1762367564 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa63c0b2040 con 0x5610b5b6deb0 2024-05-21T22:02:47.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.489+0000 7fa635600700 1 -- 172.21.15.151:0/1762367564 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fa638032410 con 0x5610b5b6deb0 2024-05-21T22:02:47.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.489+0000 7fa65c7eab80 10 client.26947 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:02:47.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.489+0000 7fa65c7eab80 10 client.26947 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-21T22:02:47.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.490+0000 7fa65c7eab80 20 client.26947 populate_metadata read hostname 'smithi151' 2024-05-21T22:02:47.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.490+0000 7fa65c7eab80 1 --2- 172.21.15.151:0/1762367564 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x5610b5bf5400 0x5610b5bf7820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:47.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.490+0000 7fa65c7eab80 4 client.26947 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T22:02:47.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.490+0000 7fa65c7eab80 1 -- 172.21.15.151:0/1762367564 --> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5610b5ac2570 con 0x5610b5bf5400 2024-05-21T22:02:47.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.490+0000 7fa64aa00700 1 --2- 172.21.15.151:0/1762367564 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x5610b5bf5400 0x5610b5bf7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:47.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.490+0000 7fa64aa00700 1 --2- 172.21.15.151:0/1762367564 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x5610b5bf5400 0x5610b5bf7820 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c38d3150470f8a33 server_cookie=976d79c4caf768d1 in_seq=0 out_seq=0 2024-05-21T22:02:47.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.490+0000 7fa635600700 10 client.26947 ms_handle_connect on v2:172.21.15.184:6836/2988194888 2024-05-21T22:02:47.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.491+0000 7fa635600700 1 -- 172.21.15.151:0/1762367564 <== mds.3 v2:172.21.15.184:6836/2988194888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5610b5ac2570 con 0x5610b5bf5400 2024-05-21T22:02:47.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.491+0000 7fa635600700 10 client.26947 handle_command_reply: tid=0 2024-05-21T22:02:47.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.491+0000 7fa65c7eab80 1 client.26947 shutdown 2024-05-21T22:02:47.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.491+0000 7fa65c7eab80 1 -- 172.21.15.151:0/1762367564 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5610b5b6deb0 msgr2=0x5610b5b89200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:47.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.491+0000 7fa65c7eab80 1 --2- 172.21.15.151:0/1762367564 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5610b5b6deb0 0x5610b5b89200 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x5610b5b5b1f0 tx=0x7fa638009f80 comp rx=0 tx=0).stop 2024-05-21T22:02:47.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.491+0000 7fa65c7eab80 1 -- 172.21.15.151:0/1762367564 shutdown_connections 2024-05-21T22:02:47.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.491+0000 7fa65c7eab80 1 --2- 172.21.15.151:0/1762367564 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x5610b5bf5400 0x5610b5bf7820 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:47.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.491+0000 7fa65c7eab80 1 --2- 172.21.15.151:0/1762367564 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5610b5b6e7d0 0x5610b5b89740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:47.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.491+0000 7fa65c7eab80 1 --2- 172.21.15.151:0/1762367564 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5610b5b68880 0x5610b5b7aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:47.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.491+0000 7fa65c7eab80 1 --2- 172.21.15.151:0/1762367564 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5610b5b6deb0 0x5610b5b89200 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:47.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.491+0000 7fa635600700 0 client.26947 ms_handle_reset on v2:172.21.15.184:6836/2988194888 2024-05-21T22:02:47.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.491+0000 7fa65c7eab80 1 -- 172.21.15.151:0/1762367564 >> 172.21.15.151:0/1762367564 conn(0x5610b5a96be0 msgr2=0x5610b5b629e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:02:47.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.491+0000 7fa65c7eab80 1 -- 172.21.15.151:0/1762367564 shutdown_connections 2024-05-21T22:02:47.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.491+0000 7fa65c7eab80 1 -- 172.21.15.151:0/1762367564 wait complete. 2024-05-21T22:02:47.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.491+0000 7fa65c7eab80 20 client.26947 trim_cache size 0 max 16384 2024-05-21T22:02:47.494 INFO:teuthology.orchestra.run.smithi151.stdout:[] 2024-05-21T22:02:47.495 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.492+0000 7fa651e00700 1 -- 172.21.15.151:0/4264878659 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fa6300b29f0 msgr2=0x7fa6300b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:47.495 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.492+0000 7fa651e00700 1 --2- 172.21.15.151:0/4264878659 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fa6300b29f0 0x7fa6300b4eb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa64c110820 tx=0x7fa638009260 comp rx=0 tx=0).stop 2024-05-21T22:02:47.495 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.492+0000 7fa651e00700 1 -- 172.21.15.151:0/4264878659 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa64c1037f0 msgr2=0x7fa64c19bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:47.495 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.492+0000 7fa651e00700 1 --2- 172.21.15.151:0/4264878659 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa64c1037f0 0x7fa64c19bd40 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa63c00e9f0 tx=0x7fa63c00ed00 comp rx=0 tx=0).stop 2024-05-21T22:02:47.495 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.492+0000 7fa651e00700 1 -- 172.21.15.151:0/4264878659 shutdown_connections 2024-05-21T22:02:47.495 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.492+0000 7fa651e00700 1 --2- 172.21.15.151:0/4264878659 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fa6300b29f0 0x7fa6300b4eb0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:47.495 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.492+0000 7fa651e00700 1 --2- 172.21.15.151:0/4264878659 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa64c1037f0 0x7fa64c19bd40 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:47.495 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.492+0000 7fa651e00700 1 --2- 172.21.15.151:0/4264878659 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa64c1088f0 0x7fa64c1a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:47.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.492+0000 7fa651e00700 1 --2- 172.21.15.151:0/4264878659 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa64c1041a0 0x7fa64c19c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:47.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.492+0000 7fa651e00700 1 -- 172.21.15.151:0/4264878659 >> 172.21.15.151:0/4264878659 conn(0x7fa64c07a6a0 msgr2=0x7fa64c1014a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:02:47.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.493+0000 7fa651e00700 1 -- 172.21.15.151:0/4264878659 shutdown_connections 2024-05-21T22:02:47.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:47.493+0000 7fa651e00700 1 -- 172.21.15.151:0/4264878659 wait complete. 2024-05-21T22:02:47.702 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:02:47 smithi151 ceph-mon[31623]: pgmap v220: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-05-21T22:02:47.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:02:47 smithi164 ceph-mon[36269]: pgmap v220: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-05-21T22:02:47.985 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:02:47 smithi184 ceph-mon[36942]: pgmap v220: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-05-21T22:02:48.417 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-21T22:02:48.843 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:02:49.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:02:49 smithi164 ceph-mon[36269]: pgmap v221: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-05-21T22:02:49.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:02:49 smithi151 ceph-mon[31623]: pgmap v221: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-05-21T22:02:49.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:02:49 smithi184 ceph-mon[36942]: pgmap v221: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-05-21T22:02:51.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.031+0000 7f44c7200700 1 -- 172.21.15.151:0/3836865495 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f44c010da60 msgr2=0x7f44c010ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:51.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.031+0000 7f44c7200700 1 --2- 172.21.15.151:0/3836865495 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f44c010da60 0x7f44c010ff20 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f44bc00b0a0 tx=0x7f44bc00b3b0 comp rx=0 tx=0).stop 2024-05-21T22:02:51.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.032+0000 7f44c7200700 1 -- 172.21.15.151:0/3836865495 shutdown_connections 2024-05-21T22:02:51.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.032+0000 7f44c7200700 1 --2- 172.21.15.151:0/3836865495 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f44c0104630 0x7f44c0104a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:51.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.032+0000 7f44c7200700 1 --2- 172.21.15.151:0/3836865495 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f44c010da60 0x7f44c010ff20 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:51.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.032+0000 7f44c7200700 1 --2- 172.21.15.151:0/3836865495 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f44c0104f50 0x7f44c010d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:51.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.032+0000 7f44c7200700 1 -- 172.21.15.151:0/3836865495 >> 172.21.15.151:0/3836865495 conn(0x7f44c00fde60 msgr2=0x7f44c0100280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:02:51.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.032+0000 7f44c7200700 1 -- 172.21.15.151:0/3836865495 shutdown_connections 2024-05-21T22:02:51.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.032+0000 7f44c7200700 1 -- 172.21.15.151:0/3836865495 wait complete. 2024-05-21T22:02:51.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.033+0000 7f44c7200700 1 Processor -- start 2024-05-21T22:02:51.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.033+0000 7f44c7200700 1 -- start start 2024-05-21T22:02:51.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.033+0000 7f44c7200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f44c0104630 0x7f44c0102290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:51.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.033+0000 7f44c7200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f44c0104f50 0x7f44c01027d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:51.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.033+0000 7f44c7200700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f44c010da60 0x7f44c0102d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:51.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.033+0000 7f44c7200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f44c01136b0 con 0x7f44c0104630 2024-05-21T22:02:51.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.033+0000 7f44c7200700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f44c0113530 con 0x7f44c0104f50 2024-05-21T22:02:51.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.034+0000 7f44c7200700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f44c0113830 con 0x7f44c010da60 2024-05-21T22:02:51.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.034+0000 7f44c5e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f44c0104630 0x7f44c0102290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:51.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.034+0000 7f44c5400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f44c0104f50 0x7f44c01027d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:51.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.034+0000 7f44c5e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f44c0104630 0x7f44c0102290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:39492/0 (socket says 172.21.15.151:39492) 2024-05-21T22:02:51.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.034+0000 7f44c5400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f44c0104f50 0x7f44c01027d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:55996/0 (socket says 172.21.15.151:55996) 2024-05-21T22:02:51.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.034+0000 7f44c5e00700 1 -- 172.21.15.151:0/3320137202 learned_addr learned my addr 172.21.15.151:0/3320137202 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:02:51.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.034+0000 7f44c6800700 1 --2- 172.21.15.151:0/3320137202 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f44c010da60 0x7f44c0102d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:51.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.034+0000 7f44c5400700 1 -- 172.21.15.151:0/3320137202 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f44c010da60 msgr2=0x7f44c0102d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:51.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.034+0000 7f44c5400700 1 --2- 172.21.15.151:0/3320137202 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f44c010da60 0x7f44c0102d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:51.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.034+0000 7f44c5400700 1 -- 172.21.15.151:0/3320137202 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f44c0104630 msgr2=0x7f44c0102290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:51.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.034+0000 7f44c5400700 1 --2- 172.21.15.151:0/3320137202 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f44c0104630 0x7f44c0102290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:51.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.034+0000 7f44c5400700 1 -- 172.21.15.151:0/3320137202 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44c01a8050 con 0x7f44c0104f50 2024-05-21T22:02:51.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.035+0000 7f44c5400700 1 --2- 172.21.15.151:0/3320137202 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f44c0104f50 0x7f44c01027d0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f44b000e9c0 tx=0x7f44b000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:02:51.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.035+0000 7f44baa00700 1 -- 172.21.15.151:0/3320137202 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b000c9c0 con 0x7f44c0104f50 2024-05-21T22:02:51.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.035+0000 7f44baa00700 1 -- 172.21.15.151:0/3320137202 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44b000cb60 con 0x7f44c0104f50 2024-05-21T22:02:51.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.035+0000 7f44c7200700 1 -- 172.21.15.151:0/3320137202 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44c01a8340 con 0x7f44c0104f50 2024-05-21T22:02:51.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.035+0000 7f44baa00700 1 -- 172.21.15.151:0/3320137202 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b0018420 con 0x7f44c0104f50 2024-05-21T22:02:51.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.035+0000 7f44c7200700 1 -- 172.21.15.151:0/3320137202 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44c01a8880 con 0x7f44c0104f50 2024-05-21T22:02:51.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.039+0000 7f44baa00700 1 -- 172.21.15.151:0/3320137202 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7f44b0010430 con 0x7f44c0104f50 2024-05-21T22:02:51.040 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.040+0000 7f44d3224b80 1 Processor -- start 2024-05-21T22:02:51.040 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.040+0000 7f44d3224b80 1 -- start start 2024-05-21T22:02:51.041 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.040+0000 7f44d3224b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f7cf08020 0x560f7cf08400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:51.041 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.040+0000 7f44d3224b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f7cf08940 0x560f7cf024b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:51.041 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.040+0000 7f44d3224b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f7cf02ad0 0x560f7cf02f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:51.041 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.040+0000 7f44d3224b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x560f7ce60e40 con 0x560f7cf08020 2024-05-21T22:02:51.041 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.040+0000 7f44d3224b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x560f7ce509e0 con 0x560f7cf02ad0 2024-05-21T22:02:51.041 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.040+0000 7f44d3224b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x560f7ce54a20 con 0x560f7cf08940 2024-05-21T22:02:51.041 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.041+0000 7f44c6800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f7cf08020 0x560f7cf08400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:51.043 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.041+0000 7f44c6800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f7cf08020 0x560f7cf08400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:39500/0 (socket says 172.21.15.151:39500) 2024-05-21T22:02:51.043 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.041+0000 7f44c6800700 1 -- 172.21.15.151:0/2846671774 learned_addr learned my addr 172.21.15.151:0/2846671774 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:02:51.043 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.041+0000 7f44c5400700 1 --2- 172.21.15.151:0/2846671774 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f7cf02ad0 0x560f7cf02f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:51.043 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.041+0000 7f44c5e00700 1 --2- 172.21.15.151:0/2846671774 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f7cf08940 0x560f7cf024b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:51.043 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.041+0000 7f44baa00700 1 --2- 172.21.15.151:0/3320137202 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f44a80b29f0 0x7f44a80b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:51.043 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.041+0000 7f44c5400700 1 -- 172.21.15.151:0/2846671774 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f7cf08940 msgr2=0x560f7cf024b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:51.043 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.042+0000 7f44c5400700 1 --2- 172.21.15.151:0/2846671774 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f7cf08940 0x560f7cf024b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:51.043 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.042+0000 7f44c5400700 1 -- 172.21.15.151:0/2846671774 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f7cf08020 msgr2=0x560f7cf08400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:51.043 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.042+0000 7f44c5400700 1 --2- 172.21.15.151:0/2846671774 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f7cf08020 0x560f7cf08400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:51.043 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.042+0000 7f44baa00700 1 -- 172.21.15.151:0/3320137202 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8542+0+0 (secure 0 0 0) 0x7f44b00e2770 con 0x7f44c0104f50 2024-05-21T22:02:51.043 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.042+0000 7f44c5400700 1 -- 172.21.15.151:0/2846671774 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f7ce19420 con 0x560f7cf02ad0 2024-05-21T22:02:51.043 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.042+0000 7f44c5e00700 1 --2- 172.21.15.151:0/2846671774 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f7cf08940 0x560f7cf024b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:02:51.044 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.042+0000 7f44c6800700 1 --2- 172.21.15.151:0/2846671774 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f7cf08020 0x560f7cf08400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:02:51.044 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.042+0000 7f44c6800700 1 --2- 172.21.15.151:0/3320137202 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f44a80b29f0 0x7f44a80b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:51.044 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.042+0000 7f44c5400700 1 --2- 172.21.15.151:0/2846671774 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f7cf02ad0 0x560f7cf02f30 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f44b00dfe50 tx=0x7f44b0009f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1f425835fc8ecd5 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:02:51.044 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.042+0000 7f44af400700 1 -- 172.21.15.151:0/2846671774 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b00a9330 con 0x560f7cf02ad0 2024-05-21T22:02:51.044 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.042+0000 7f44af400700 1 -- 172.21.15.151:0/2846671774 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44b00b2190 con 0x560f7cf02ad0 2024-05-21T22:02:51.044 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.043+0000 7f44af400700 1 -- 172.21.15.151:0/2846671774 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b00a9150 con 0x560f7cf02ad0 2024-05-21T22:02:51.045 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.043+0000 7f44d3224b80 1 -- 172.21.15.151:0/2846671774 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f7cf02ad0 msgr2=0x560f7cf02f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:51.045 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.043+0000 7f44d3224b80 1 --2- 172.21.15.151:0/2846671774 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f7cf02ad0 0x560f7cf02f30 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f44b00dfe50 tx=0x7f44b0009f60 comp rx=0 tx=0).stop 2024-05-21T22:02:51.045 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.043+0000 7f44c6800700 1 --2- 172.21.15.151:0/3320137202 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f44a80b29f0 0x7f44a80b4eb0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f44bc00d010 tx=0x7f44bc0099e0 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:02:51.045 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.043+0000 7f44d3224b80 1 -- 172.21.15.151:0/2846671774 shutdown_connections 2024-05-21T22:02:51.045 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.043+0000 7f44d3224b80 1 --2- 172.21.15.151:0/2846671774 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f7cf02ad0 0x560f7cf02f30 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:51.045 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.043+0000 7f44d3224b80 1 --2- 172.21.15.151:0/2846671774 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f7cf08020 0x560f7cf08400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:51.045 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.043+0000 7f44d3224b80 1 --2- 172.21.15.151:0/2846671774 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f7cf08940 0x560f7cf024b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:51.045 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.043+0000 7f44d3224b80 1 -- 172.21.15.151:0/2846671774 >> 172.21.15.151:0/2846671774 conn(0x560f7ce30be0 msgr2=0x560f7cefd150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:02:51.045 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.044+0000 7f44d3224b80 1 -- 172.21.15.151:0/2846671774 shutdown_connections 2024-05-21T22:02:51.045 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.044+0000 7f44d3224b80 1 -- 172.21.15.151:0/2846671774 wait complete. 2024-05-21T22:02:51.045 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.044+0000 7f44d3224b80 1 -- start start 2024-05-21T22:02:51.046 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.044+0000 7f44d3224b80 1 Processor -- start 2024-05-21T22:02:51.046 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.045+0000 7f44d3224b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f7cf02ad0 0x560f7cf0e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:51.046 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.045+0000 7f44d3224b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f7cf08020 0x560f7cf0b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:51.047 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.045+0000 7f44d3224b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f7cf08940 0x560f7cf0bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:51.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.045+0000 7f44d3224b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x560f7cd56fa0 con 0x560f7cf08020 2024-05-21T22:02:51.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.045+0000 7f44d3224b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x560f7cdf1760 con 0x560f7cf08940 2024-05-21T22:02:51.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.045+0000 7f44d3224b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x560f7cf0f3a0 con 0x560f7cf02ad0 2024-05-21T22:02:51.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.045+0000 7f44d3224b80 10 client.0 _ll_register_callbacks cb 0x560f7cdfac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:02:51.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.045+0000 7f44c5400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f7cf08020 0x560f7cf0b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:51.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.045+0000 7f44c6800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f7cf08940 0x560f7cf0bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:51.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.045+0000 7f44c5400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f7cf08020 0x560f7cf0b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:39502/0 (socket says 172.21.15.151:39502) 2024-05-21T22:02:51.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.045+0000 7f44c6800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f7cf08940 0x560f7cf0bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:56014/0 (socket says 172.21.15.151:56014) 2024-05-21T22:02:51.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.045+0000 7f44c5400700 1 -- 172.21.15.151:0/3623963613 learned_addr learned my addr 172.21.15.151:0/3623963613 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:02:51.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.045+0000 7f44c5e00700 1 --2- 172.21.15.151:0/3623963613 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f7cf02ad0 0x560f7cf0e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:51.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.046+0000 7f44c6800700 1 -- 172.21.15.151:0/3623963613 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f7cf02ad0 msgr2=0x560f7cf0e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:51.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.046+0000 7f44c6800700 1 --2- 172.21.15.151:0/3623963613 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f7cf02ad0 0x560f7cf0e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:51.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.046+0000 7f44c6800700 1 -- 172.21.15.151:0/3623963613 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f7cf08020 msgr2=0x560f7cf0b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:51.049 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.046+0000 7f44c6800700 1 --2- 172.21.15.151:0/3623963613 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f7cf08020 0x560f7cf0b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:51.049 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.046+0000 7f44c6800700 1 -- 172.21.15.151:0/3623963613 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f7cda08d0 con 0x560f7cf08940 2024-05-21T22:02:51.049 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.046+0000 7f44c6800700 1 --2- 172.21.15.151:0/3623963613 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f7cf08940 0x560f7cf0bef0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f44bc009fd0 tx=0x7f44bc00b3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aaee4de9dac4002a server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:02:51.049 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.046+0000 7f44a3e00700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-21T22:02:51.049 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.046+0000 7f44a3e00700 1 -- 172.21.15.151:0/3623963613 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44bc02d020 con 0x560f7cf08940 2024-05-21T22:02:51.049 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.046+0000 7f44d3224b80 1 -- 172.21.15.151:0/3623963613 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44b00b2190 con 0x560f7cf08940 2024-05-21T22:02:51.049 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.047+0000 7f44a3e00700 1 -- 172.21.15.151:0/3623963613 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8542+0+0 (secure 0 0 0) 0x7f44bc007b50 con 0x560f7cf08940 2024-05-21T22:02:51.049 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.047+0000 7f44a3e00700 1 -- 172.21.15.151:0/3623963613 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f44bc038050 con 0x560f7cf08940 2024-05-21T22:02:51.049 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.047+0000 7f44d3224b80 10 client.26948 fetch_fsmap learned FSMap version 29 2024-05-21T22:02:51.049 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.047+0000 7f44d3224b80 1 -- 172.21.15.151:0/3623963613 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560f7ce21060 con 0x560f7cf08940 2024-05-21T22:02:51.049 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.047+0000 7f44a3e00700 1 -- 172.21.15.151:0/3623963613 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44b00b2190 con 0x560f7cf08940 2024-05-21T22:02:51.049 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.047+0000 7f44a3e00700 1 -- 172.21.15.151:0/3623963613 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f44bc011070 con 0x560f7cf08940 2024-05-21T22:02:51.049 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.047+0000 7f44d3224b80 10 client.26948 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:02:51.049 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.047+0000 7f44d3224b80 10 client.26948 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-05-21T22:02:51.049 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.047+0000 7f44d3224b80 20 client.26948 populate_metadata read hostname 'smithi151' 2024-05-21T22:02:51.049 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.047+0000 7f44d3224b80 1 --2- 172.21.15.151:0/3623963613 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x560f7cf1be60 0x560f7cf1e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:51.050 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.047+0000 7f44d3224b80 4 client.26948 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T22:02:51.050 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.047+0000 7f44d3224b80 1 -- 172.21.15.151:0/3623963613 --> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560f7ce62d90 con 0x560f7cf1be60 2024-05-21T22:02:51.051 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.048+0000 7f44c5e00700 1 --2- 172.21.15.151:0/3623963613 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x560f7cf1be60 0x560f7cf1e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:51.051 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.048+0000 7f44c5e00700 1 --2- 172.21.15.151:0/3623963613 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x560f7cf1be60 0x560f7cf1e250 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e88a3f5ff668326a server_cookie=a091b4bbf99ce297 in_seq=0 out_seq=0 2024-05-21T22:02:51.051 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.048+0000 7f44a3e00700 10 client.26948 ms_handle_connect on v2:172.21.15.164:6836/1554609572 2024-05-21T22:02:51.051 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.049+0000 7f44a3e00700 1 -- 172.21.15.151:0/3623963613 <== mds.2 v2:172.21.15.164:6836/1554609572 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x560f7ce62d90 con 0x560f7cf1be60 2024-05-21T22:02:51.051 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.049+0000 7f44a3e00700 10 client.26948 handle_command_reply: tid=0 2024-05-21T22:02:51.051 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.049+0000 7f44d3224b80 1 client.26948 shutdown 2024-05-21T22:02:51.051 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.049+0000 7f44d3224b80 1 -- 172.21.15.151:0/3623963613 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f7cf08940 msgr2=0x560f7cf0bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:51.051 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.049+0000 7f44d3224b80 1 --2- 172.21.15.151:0/3623963613 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f7cf08940 0x560f7cf0bef0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f44bc009fd0 tx=0x7f44bc00b3b0 comp rx=0 tx=0).stop 2024-05-21T22:02:51.051 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.049+0000 7f44d3224b80 1 -- 172.21.15.151:0/3623963613 shutdown_connections 2024-05-21T22:02:51.051 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.049+0000 7f44d3224b80 1 --2- 172.21.15.151:0/3623963613 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x560f7cf1be60 0x560f7cf1e250 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:51.051 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.049+0000 7f44d3224b80 1 --2- 172.21.15.151:0/3623963613 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f7cf08940 0x560f7cf0bef0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:51.051 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.049+0000 7f44d3224b80 1 --2- 172.21.15.151:0/3623963613 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f7cf02ad0 0x560f7cf0e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:51.051 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.049+0000 7f44d3224b80 1 --2- 172.21.15.151:0/3623963613 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f7cf08020 0x560f7cf0b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:51.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.049+0000 7f44a3e00700 0 client.26948 ms_handle_reset on v2:172.21.15.164:6836/1554609572 2024-05-21T22:02:51.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.049+0000 7f44d3224b80 1 -- 172.21.15.151:0/3623963613 >> 172.21.15.151:0/3623963613 conn(0x560f7ce30be0 msgr2=0x560f7cf014a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:02:51.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.050+0000 7f44d3224b80 1 -- 172.21.15.151:0/3623963613 shutdown_connections 2024-05-21T22:02:51.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.050+0000 7f44d3224b80 1 -- 172.21.15.151:0/3623963613 wait complete. 2024-05-21T22:02:51.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.050+0000 7f44d3224b80 20 client.26948 trim_cache size 0 max 16384 2024-05-21T22:02:51.067 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.067+0000 7f44d3224b80 1 Processor -- start 2024-05-21T22:02:51.067 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.067+0000 7f44d3224b80 1 -- start start 2024-05-21T22:02:51.068 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.067+0000 7f44d3224b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f7cf02ad0 0x560f7cf1e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:51.068 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.067+0000 7f44d3224b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f7cf08020 0x560f7cf0b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:51.068 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.067+0000 7f44d3224b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f7cf08940 0x560f7cf0a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:51.068 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.067+0000 7f44d3224b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f44c01aff50 con 0x560f7cf08020 2024-05-21T22:02:51.068 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.067+0000 7f44d3224b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f44c01b1a90 con 0x560f7cf02ad0 2024-05-21T22:02:51.068 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.067+0000 7f44d3224b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f44c01b0620 con 0x560f7cf08940 2024-05-21T22:02:51.069 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.068+0000 7f44c5e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f7cf02ad0 0x560f7cf1e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:51.069 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.068+0000 7f44c5400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f7cf08020 0x560f7cf0b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:51.069 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.068+0000 7f44c5e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f7cf02ad0 0x560f7cf1e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:56028/0 (socket says 172.21.15.151:56028) 2024-05-21T22:02:51.069 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.068+0000 7f44c5e00700 1 -- 172.21.15.151:0/2408386196 learned_addr learned my addr 172.21.15.151:0/2408386196 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:02:51.069 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.068+0000 7f44c6800700 1 --2- 172.21.15.151:0/2408386196 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f7cf08940 0x560f7cf0a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:51.069 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.068+0000 7f44c5e00700 1 -- 172.21.15.151:0/2408386196 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f7cf08940 msgr2=0x560f7cf0a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:51.069 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.068+0000 7f44c5e00700 1 --2- 172.21.15.151:0/2408386196 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f7cf08940 0x560f7cf0a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:51.069 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.068+0000 7f44c5e00700 1 -- 172.21.15.151:0/2408386196 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f7cf08020 msgr2=0x560f7cf0b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:51.069 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.068+0000 7f44c5e00700 1 --2- 172.21.15.151:0/2408386196 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f7cf08020 0x560f7cf0b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:51.069 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.068+0000 7f44c5e00700 1 -- 172.21.15.151:0/2408386196 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f7cefe8b0 con 0x560f7cf02ad0 2024-05-21T22:02:51.070 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.068+0000 7f44c5e00700 1 --2- 172.21.15.151:0/2408386196 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f7cf02ad0 0x560f7cf1e250 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x560f7cf1ea90 tx=0x7f44b4009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20df44abc0b00cb server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:02:51.070 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.069+0000 7f44aea00700 1 -- 172.21.15.151:0/2408386196 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b4004050 con 0x560f7cf02ad0 2024-05-21T22:02:51.070 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.069+0000 7f44aea00700 1 -- 172.21.15.151:0/2408386196 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44b4004830 con 0x560f7cf02ad0 2024-05-21T22:02:51.070 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.069+0000 7f44aea00700 1 -- 172.21.15.151:0/2408386196 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b4005440 con 0x560f7cf02ad0 2024-05-21T22:02:51.070 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.069+0000 7f44d3224b80 1 -- 172.21.15.151:0/2408386196 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f7cf02ad0 msgr2=0x560f7cf1e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:51.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.069+0000 7f44d3224b80 1 --2- 172.21.15.151:0/2408386196 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f7cf02ad0 0x560f7cf1e250 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x560f7cf1ea90 tx=0x7f44b4009260 comp rx=0 tx=0).stop 2024-05-21T22:02:51.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.069+0000 7f44d3224b80 1 -- 172.21.15.151:0/2408386196 shutdown_connections 2024-05-21T22:02:51.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.069+0000 7f44d3224b80 1 --2- 172.21.15.151:0/2408386196 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f7cf02ad0 0x560f7cf1e250 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:51.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.069+0000 7f44d3224b80 1 --2- 172.21.15.151:0/2408386196 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f7cf08940 0x560f7cf0a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:51.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.069+0000 7f44d3224b80 1 --2- 172.21.15.151:0/2408386196 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f7cf08020 0x560f7cf0b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:51.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.069+0000 7f44d3224b80 1 -- 172.21.15.151:0/2408386196 >> 172.21.15.151:0/2408386196 conn(0x560f7ce30be0 msgr2=0x560f7cef6fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:02:51.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.070+0000 7f44d3224b80 1 -- 172.21.15.151:0/2408386196 shutdown_connections 2024-05-21T22:02:51.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.070+0000 7f44d3224b80 1 -- 172.21.15.151:0/2408386196 wait complete. 2024-05-21T22:02:51.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.070+0000 7f44d3224b80 1 -- start start 2024-05-21T22:02:51.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.070+0000 7f44d3224b80 1 Processor -- start 2024-05-21T22:02:51.073 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.071+0000 7f44d3224b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f7cf02ad0 0x560f7cf14ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:51.073 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.071+0000 7f44d3224b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f7cf08020 0x560f7cf23270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:51.073 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.071+0000 7f44d3224b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f7cf08940 0x560f7cf237b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:51.073 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.071+0000 7f44d3224b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f44c01b13c0 con 0x560f7cf08940 2024-05-21T22:02:51.073 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.071+0000 7f44d3224b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f44c01b0cf0 con 0x560f7cf02ad0 2024-05-21T22:02:51.073 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.071+0000 7f44d3224b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f44c01adf10 con 0x560f7cf08020 2024-05-21T22:02:51.073 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.071+0000 7f44d3224b80 10 client.0 _ll_register_callbacks cb 0x560f7cdfac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:02:51.073 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.071+0000 7f44c5e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f7cf02ad0 0x560f7cf14ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:51.073 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.071+0000 7f44c5e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f7cf02ad0 0x560f7cf14ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:56030/0 (socket says 172.21.15.151:56030) 2024-05-21T22:02:51.073 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.071+0000 7f44c5400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f7cf08020 0x560f7cf23270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:51.073 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.071+0000 7f44c5e00700 1 -- 172.21.15.151:0/2434074295 learned_addr learned my addr 172.21.15.151:0/2434074295 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:02:51.073 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.071+0000 7f44c6800700 1 --2- 172.21.15.151:0/2434074295 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f7cf08940 0x560f7cf237b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:51.073 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.072+0000 7f44c5400700 1 -- 172.21.15.151:0/2434074295 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f7cf02ad0 msgr2=0x560f7cf14ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:51.074 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.072+0000 7f44c5400700 1 --2- 172.21.15.151:0/2434074295 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f7cf02ad0 0x560f7cf14ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:51.074 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.072+0000 7f44c5400700 1 -- 172.21.15.151:0/2434074295 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f7cf08940 msgr2=0x560f7cf237b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:51.074 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.072+0000 7f44c5400700 1 --2- 172.21.15.151:0/2434074295 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f7cf08940 0x560f7cf237b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:51.074 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.072+0000 7f44c5400700 1 -- 172.21.15.151:0/2434074295 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f7cf23ff0 con 0x560f7cf08020 2024-05-21T22:02:51.074 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.072+0000 7f44c5400700 1 --2- 172.21.15.151:0/2434074295 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f7cf08020 0x560f7cf23270 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f44b0009230 tx=0x7f44b00a98b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8a58d219715bac6 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:02:51.074 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.072+0000 7f44a3e00700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-21T22:02:51.074 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.072+0000 7f44a3e00700 1 -- 172.21.15.151:0/2434074295 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b00a9330 con 0x560f7cf08020 2024-05-21T22:02:51.074 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.072+0000 7f44d3224b80 1 -- 172.21.15.151:0/2434074295 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44b4004830 con 0x560f7cf08020 2024-05-21T22:02:51.074 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.072+0000 7f44a3e00700 1 -- 172.21.15.151:0/2434074295 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8542+0+0 (secure 0 0 0) 0x7f44b00cc2b0 con 0x560f7cf08020 2024-05-21T22:02:51.074 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.072+0000 7f44a3e00700 1 -- 172.21.15.151:0/2434074295 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f44b00b2190 con 0x560f7cf08020 2024-05-21T22:02:51.074 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.072+0000 7f44d3224b80 10 client.26965 fetch_fsmap learned FSMap version 29 2024-05-21T22:02:51.074 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.072+0000 7f44d3224b80 1 -- 172.21.15.151:0/2434074295 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560f7cf86920 con 0x560f7cf08020 2024-05-21T22:02:51.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.073+0000 7f44a3e00700 1 -- 172.21.15.151:0/2434074295 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44b4004830 con 0x560f7cf08020 2024-05-21T22:02:51.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.073+0000 7f44a3e00700 1 -- 172.21.15.151:0/2434074295 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f44b00b23b0 con 0x560f7cf08020 2024-05-21T22:02:51.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.073+0000 7f44d3224b80 10 client.26965 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:02:51.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.073+0000 7f44d3224b80 10 client.26965 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-05-21T22:02:51.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.073+0000 7f44d3224b80 20 client.26965 populate_metadata read hostname 'smithi151' 2024-05-21T22:02:51.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.073+0000 7f44d3224b80 1 --2- 172.21.15.151:0/2434074295 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x560f7cf86cb0 0x560f7cf890a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:51.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.073+0000 7f44d3224b80 4 client.26965 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T22:02:51.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.073+0000 7f44d3224b80 1 -- 172.21.15.151:0/2434074295 --> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560f7ce5c570 con 0x560f7cf86cb0 2024-05-21T22:02:51.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.073+0000 7f44c5e00700 1 --2- 172.21.15.151:0/2434074295 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x560f7cf86cb0 0x560f7cf890a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:51.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.074+0000 7f44c5e00700 1 --2- 172.21.15.151:0/2434074295 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x560f7cf86cb0 0x560f7cf890a0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f4fb4197e3358b5d server_cookie=7f2b1104cc2a4522 in_seq=0 out_seq=0 2024-05-21T22:02:51.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.074+0000 7f44a3e00700 10 client.26965 ms_handle_connect on v2:172.21.15.164:6836/1554609572 2024-05-21T22:02:51.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.074+0000 7f44a3e00700 1 -- 172.21.15.151:0/2434074295 <== mds.2 v2:172.21.15.164:6836/1554609572 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560f7ce5c570 con 0x560f7cf86cb0 2024-05-21T22:02:51.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.074+0000 7f44a3e00700 10 client.26965 handle_command_reply: tid=0 2024-05-21T22:02:51.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.074+0000 7f44d3224b80 1 client.26965 shutdown 2024-05-21T22:02:51.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.074+0000 7f44d3224b80 1 -- 172.21.15.151:0/2434074295 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f7cf08020 msgr2=0x560f7cf23270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:51.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.074+0000 7f44d3224b80 1 --2- 172.21.15.151:0/2434074295 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f7cf08020 0x560f7cf23270 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f44b0009230 tx=0x7f44b00a98b0 comp rx=0 tx=0).stop 2024-05-21T22:02:51.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.074+0000 7f44d3224b80 1 -- 172.21.15.151:0/2434074295 shutdown_connections 2024-05-21T22:02:51.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.074+0000 7f44d3224b80 1 --2- 172.21.15.151:0/2434074295 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x560f7cf86cb0 0x560f7cf890a0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:51.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.074+0000 7f44d3224b80 1 --2- 172.21.15.151:0/2434074295 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f7cf02ad0 0x560f7cf14ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:51.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.074+0000 7f44d3224b80 1 --2- 172.21.15.151:0/2434074295 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f7cf08940 0x560f7cf237b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:51.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.074+0000 7f44a3e00700 0 client.26965 ms_handle_reset on v2:172.21.15.164:6836/1554609572 2024-05-21T22:02:51.078 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.074+0000 7f44d3224b80 1 --2- 172.21.15.151:0/2434074295 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f7cf08020 0x560f7cf23270 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:51.078 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.074+0000 7f44d3224b80 1 -- 172.21.15.151:0/2434074295 >> 172.21.15.151:0/2434074295 conn(0x560f7ce30be0 msgr2=0x560f7cefcc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:02:51.078 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.075+0000 7f44d3224b80 1 -- 172.21.15.151:0/2434074295 shutdown_connections 2024-05-21T22:02:51.078 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.075+0000 7f44d3224b80 1 -- 172.21.15.151:0/2434074295 wait complete. 2024-05-21T22:02:51.078 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.075+0000 7f44d3224b80 20 client.26965 trim_cache size 0 max 16384 2024-05-21T22:02:51.078 INFO:teuthology.orchestra.run.smithi151.stdout:[] 2024-05-21T22:02:51.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.075+0000 7f44c7200700 1 -- 172.21.15.151:0/3320137202 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f44a80b29f0 msgr2=0x7f44a80b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:51.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.075+0000 7f44c7200700 1 --2- 172.21.15.151:0/3320137202 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f44a80b29f0 0x7f44a80b4eb0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f44bc00d010 tx=0x7f44bc0099e0 comp rx=0 tx=0).stop 2024-05-21T22:02:51.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.075+0000 7f44c7200700 1 -- 172.21.15.151:0/3320137202 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f44c0104f50 msgr2=0x7f44c01027d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:51.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.075+0000 7f44c7200700 1 --2- 172.21.15.151:0/3320137202 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f44c0104f50 0x7f44c01027d0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f44b000e9c0 tx=0x7f44b000ecd0 comp rx=0 tx=0).stop 2024-05-21T22:02:51.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.075+0000 7f44c7200700 1 -- 172.21.15.151:0/3320137202 shutdown_connections 2024-05-21T22:02:51.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.075+0000 7f44c7200700 1 --2- 172.21.15.151:0/3320137202 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f44a80b29f0 0x7f44a80b4eb0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:51.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.075+0000 7f44c7200700 1 --2- 172.21.15.151:0/3320137202 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f44c010da60 0x7f44c0102d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:51.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.075+0000 7f44c7200700 1 --2- 172.21.15.151:0/3320137202 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f44c0104630 0x7f44c0102290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:51.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.075+0000 7f44c7200700 1 --2- 172.21.15.151:0/3320137202 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f44c0104f50 0x7f44c01027d0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:51.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.075+0000 7f44c7200700 1 -- 172.21.15.151:0/3320137202 >> 172.21.15.151:0/3320137202 conn(0x7f44c00fde60 msgr2=0x7f44c00feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:02:51.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.076+0000 7f44c7200700 1 -- 172.21.15.151:0/3320137202 shutdown_connections 2024-05-21T22:02:51.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:51.076+0000 7f44c7200700 1 -- 172.21.15.151:0/3320137202 wait complete. 2024-05-21T22:02:51.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:02:51 smithi164 ceph-mon[36269]: pgmap v222: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:02:51.907 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:02:51 smithi151 ceph-mon[31623]: pgmap v222: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:02:51.967 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-21T22:02:51.985 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:02:51 smithi184 ceph-mon[36942]: pgmap v222: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:02:52.412 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:02:53.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:02:53 smithi151 ceph-mon[31623]: pgmap v223: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T22:02:53.788 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:02:53 smithi184 ceph-mon[36942]: pgmap v223: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T22:02:54.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:02:53 smithi164 ceph-mon[36269]: pgmap v223: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T22:02:54.614 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.613+0000 7f1375200700 1 -- 172.21.15.151:0/2136573934 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1370106030 msgr2=0x7f1370112540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:54.614 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.613+0000 7f1375200700 1 --2- 172.21.15.151:0/2136573934 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1370106030 0x7f1370112540 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f136800b0a0 tx=0x7f136800b3b0 comp rx=0 tx=0).stop 2024-05-21T22:02:54.614 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.613+0000 7f1375200700 1 -- 172.21.15.151:0/2136573934 shutdown_connections 2024-05-21T22:02:54.614 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.613+0000 7f1375200700 1 --2- 172.21.15.151:0/2136573934 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1370106030 0x7f1370112540 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:54.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.613+0000 7f1375200700 1 --2- 172.21.15.151:0/2136573934 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f137010b670 0x7f137010ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:54.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.613+0000 7f1375200700 1 --2- 172.21.15.151:0/2136573934 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1370105670 0x7f1370105af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:54.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.613+0000 7f1375200700 1 -- 172.21.15.151:0/2136573934 >> 172.21.15.151:0/2136573934 conn(0x7f1370101160 msgr2=0x7f1370103580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:02:54.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.613+0000 7f1375200700 1 -- 172.21.15.151:0/2136573934 shutdown_connections 2024-05-21T22:02:54.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.613+0000 7f1375200700 1 -- 172.21.15.151:0/2136573934 wait complete. 2024-05-21T22:02:54.617 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.614+0000 7f1375200700 1 Processor -- start 2024-05-21T22:02:54.617 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.615+0000 7f1375200700 1 -- start start 2024-05-21T22:02:54.617 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.615+0000 7f1375200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1370105670 0x7f13701a1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:54.617 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.615+0000 7f1375200700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1370106030 0x7f13701a1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:54.617 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.615+0000 7f1375200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f137010b670 0x7f137019bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:54.617 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.615+0000 7f1375200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1370115900 con 0x7f1370105670 2024-05-21T22:02:54.618 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.615+0000 7f1375200700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1370115780 con 0x7f137010b670 2024-05-21T22:02:54.618 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.615+0000 7f1375200700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1370115a80 con 0x7f1370106030 2024-05-21T22:02:54.618 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.615+0000 7f136f400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1370105670 0x7f13701a1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:54.618 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.615+0000 7f136f400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1370105670 0x7f13701a1940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:39542/0 (socket says 172.21.15.151:39542) 2024-05-21T22:02:54.618 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.615+0000 7f136f400700 1 -- 172.21.15.151:0/1034917711 learned_addr learned my addr 172.21.15.151:0/1034917711 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:02:54.618 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.615+0000 7f136ea00700 1 --2- 172.21.15.151:0/1034917711 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1370106030 0x7f13701a1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:54.618 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.615+0000 7f136fe00700 1 --2- 172.21.15.151:0/1034917711 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f137010b670 0x7f137019bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:54.618 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.616+0000 7f136f400700 1 -- 172.21.15.151:0/1034917711 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1370106030 msgr2=0x7f13701a1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:54.618 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.616+0000 7f136f400700 1 --2- 172.21.15.151:0/1034917711 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1370106030 0x7f13701a1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:54.618 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.616+0000 7f136f400700 1 -- 172.21.15.151:0/1034917711 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f137010b670 msgr2=0x7f137019bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:54.618 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.616+0000 7f136f400700 1 --2- 172.21.15.151:0/1034917711 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f137010b670 0x7f137019bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:54.619 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.616+0000 7f136f400700 1 -- 172.21.15.151:0/1034917711 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f137019c260 con 0x7f1370105670 2024-05-21T22:02:54.619 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.616+0000 7f136f400700 1 --2- 172.21.15.151:0/1034917711 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1370105670 0x7f13701a1940 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f136000e9a0 tx=0x7f136000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:02:54.619 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.616+0000 7f1367e00700 1 -- 172.21.15.151:0/1034917711 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f136000ca10 con 0x7f1370105670 2024-05-21T22:02:54.619 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.617+0000 7f1367e00700 1 -- 172.21.15.151:0/1034917711 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f136000cbb0 con 0x7f1370105670 2024-05-21T22:02:54.619 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.617+0000 7f1375200700 1 -- 172.21.15.151:0/1034917711 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f137019c550 con 0x7f1370105670 2024-05-21T22:02:54.619 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.617+0000 7f1367e00700 1 -- 172.21.15.151:0/1034917711 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1360018680 con 0x7f1370105670 2024-05-21T22:02:54.619 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.617+0000 7f1375200700 1 -- 172.21.15.151:0/1034917711 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13701a8810 con 0x7f1370105670 2024-05-21T22:02:54.620 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.620+0000 7f1367e00700 1 -- 172.21.15.151:0/1034917711 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7f1360010430 con 0x7f1370105670 2024-05-21T22:02:54.622 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.621+0000 7f13811fbb80 1 Processor -- start 2024-05-21T22:02:54.622 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.622+0000 7f13811fbb80 1 -- start start 2024-05-21T22:02:54.622 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.622+0000 7f1367e00700 1 --2- 172.21.15.151:0/1034917711 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f13540b29f0 0x7f13540b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:54.623 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.622+0000 7f13811fbb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ab7f017c0 0x559ab7f01ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:54.623 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.622+0000 7f1367e00700 1 -- 172.21.15.151:0/1034917711 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8542+0+0 (secure 0 0 0) 0x7f13600e27a0 con 0x7f1370105670 2024-05-21T22:02:54.623 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.622+0000 7f13811fbb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ab7f020e0 0x559ab7efbd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:54.623 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.622+0000 7f13811fbb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559ab7efc390 0x559ab7efc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:54.623 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.622+0000 7f13811fbb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x559ab7e59e40 con 0x559ab7f020e0 2024-05-21T22:02:54.623 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.622+0000 7f13811fbb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x559ab7e499e0 con 0x559ab7efc390 2024-05-21T22:02:54.623 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.622+0000 7f13811fbb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x559ab7e4da20 con 0x559ab7f017c0 2024-05-21T22:02:54.623 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.622+0000 7f136fe00700 1 --2- 172.21.15.151:0/1034917711 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f13540b29f0 0x7f13540b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:54.623 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.622+0000 7f136f400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ab7f017c0 0x559ab7f01ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:54.623 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.622+0000 7f136f400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ab7f017c0 0x559ab7f01ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:46872/0 (socket says 172.21.15.151:46872) 2024-05-21T22:02:54.623 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.622+0000 7f136f400700 1 -- 172.21.15.151:0/1842863702 learned_addr learned my addr 172.21.15.151:0/1842863702 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:02:54.623 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.622+0000 7f136ea00700 1 --2- 172.21.15.151:0/1842863702 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ab7f020e0 0x559ab7efbd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:54.624 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.622+0000 7f136fe00700 1 --2- 172.21.15.151:0/1842863702 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559ab7efc390 0x559ab7efc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:54.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.623+0000 7f136f400700 1 -- 172.21.15.151:0/1842863702 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559ab7efc390 msgr2=0x559ab7efc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:54.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.623+0000 7f136f400700 1 --2- 172.21.15.151:0/1842863702 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559ab7efc390 0x559ab7efc7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:54.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.623+0000 7f136f400700 1 -- 172.21.15.151:0/1842863702 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ab7f020e0 msgr2=0x559ab7efbd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:54.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.623+0000 7f136f400700 1 --2- 172.21.15.151:0/1842863702 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ab7f020e0 0x559ab7efbd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:54.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.623+0000 7f136f400700 1 -- 172.21.15.151:0/1842863702 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ab7e12420 con 0x559ab7f017c0 2024-05-21T22:02:54.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.623+0000 7f136fe00700 1 --2- 172.21.15.151:0/1842863702 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559ab7efc390 0x559ab7efc7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T22:02:54.626 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.623+0000 7f136f400700 1 --2- 172.21.15.151:0/1842863702 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ab7f017c0 0x559ab7f01ba0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f1360009230 tx=0x7f13600a95a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a25dcec26fedbdd0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:02:54.626 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.623+0000 7f136fe00700 1 --2- 172.21.15.151:0/1034917711 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f13540b29f0 0x7f13540b4eb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f136800b780 tx=0x7f1368009d40 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:02:54.626 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.623+0000 7f1364c00700 1 -- 172.21.15.151:0/1842863702 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13600a9a10 con 0x559ab7f017c0 2024-05-21T22:02:54.626 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.624+0000 7f1364c00700 1 -- 172.21.15.151:0/1842863702 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13600addc0 con 0x559ab7f017c0 2024-05-21T22:02:54.626 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.624+0000 7f1364c00700 1 -- 172.21.15.151:0/1842863702 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13600cc0b0 con 0x559ab7f017c0 2024-05-21T22:02:54.626 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.624+0000 7f13811fbb80 1 -- 172.21.15.151:0/1842863702 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ab7f017c0 msgr2=0x559ab7f01ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:54.626 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.624+0000 7f13811fbb80 1 --2- 172.21.15.151:0/1842863702 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ab7f017c0 0x559ab7f01ba0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f1360009230 tx=0x7f13600a95a0 comp rx=0 tx=0).stop 2024-05-21T22:02:54.626 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.625+0000 7f13811fbb80 1 -- 172.21.15.151:0/1842863702 shutdown_connections 2024-05-21T22:02:54.626 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.625+0000 7f13811fbb80 1 --2- 172.21.15.151:0/1842863702 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559ab7efc390 0x559ab7efc7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:54.626 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.625+0000 7f13811fbb80 1 --2- 172.21.15.151:0/1842863702 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ab7f017c0 0x559ab7f01ba0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:54.626 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.625+0000 7f13811fbb80 1 --2- 172.21.15.151:0/1842863702 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ab7f020e0 0x559ab7efbd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:54.626 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.625+0000 7f13811fbb80 1 -- 172.21.15.151:0/1842863702 >> 172.21.15.151:0/1842863702 conn(0x559ab7e29be0 msgr2=0x559ab7ef6810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:02:54.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.625+0000 7f13811fbb80 1 -- 172.21.15.151:0/1842863702 shutdown_connections 2024-05-21T22:02:54.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.625+0000 7f13811fbb80 1 -- 172.21.15.151:0/1842863702 wait complete. 2024-05-21T22:02:54.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.625+0000 7f13811fbb80 1 -- start start 2024-05-21T22:02:54.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.626+0000 7f13811fbb80 1 Processor -- start 2024-05-21T22:02:54.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.626+0000 7f13811fbb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559ab7efc390 0x559ab7f07ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:54.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.626+0000 7f13811fbb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ab7f017c0 0x559ab7f05240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:54.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.626+0000 7f13811fbb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ab7f020e0 0x559ab7f05780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:54.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.626+0000 7f13811fbb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x559ab7d4ffa0 con 0x559ab7f017c0 2024-05-21T22:02:54.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.626+0000 7f13811fbb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x559ab7dea760 con 0x559ab7efc390 2024-05-21T22:02:54.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.626+0000 7f13811fbb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x559ab7f08c00 con 0x559ab7f020e0 2024-05-21T22:02:54.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.626+0000 7f13811fbb80 10 client.0 _ll_register_callbacks cb 0x559ab7df3c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:02:54.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.626+0000 7f136ea00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ab7f017c0 0x559ab7f05240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:54.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.626+0000 7f136f400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559ab7efc390 0x559ab7f07ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:54.630 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.626+0000 7f136ea00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ab7f017c0 0x559ab7f05240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:39556/0 (socket says 172.21.15.151:39556) 2024-05-21T22:02:54.630 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.626+0000 7f136ea00700 1 -- 172.21.15.151:0/1669904261 learned_addr learned my addr 172.21.15.151:0/1669904261 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:02:54.630 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.627+0000 7f136fe00700 1 --2- 172.21.15.151:0/1669904261 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ab7f020e0 0x559ab7f05780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:54.630 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.627+0000 7f136f400700 1 -- 172.21.15.151:0/1669904261 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ab7f020e0 msgr2=0x559ab7f05780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:54.630 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.627+0000 7f136f400700 1 --2- 172.21.15.151:0/1669904261 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ab7f020e0 0x559ab7f05780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:54.630 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.627+0000 7f136f400700 1 -- 172.21.15.151:0/1669904261 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ab7f017c0 msgr2=0x559ab7f05240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:54.630 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.627+0000 7f136f400700 1 --2- 172.21.15.151:0/1669904261 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ab7f017c0 0x559ab7f05240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:54.630 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.627+0000 7f136f400700 1 -- 172.21.15.151:0/1669904261 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ab7d998d0 con 0x559ab7efc390 2024-05-21T22:02:54.630 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.627+0000 7f136f400700 1 --2- 172.21.15.151:0/1669904261 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559ab7efc390 0x559ab7f07ea0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f13600a9710 tx=0x7f13600a9ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f8f5dd5fc7603e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:02:54.630 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.627+0000 7f1359600700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-21T22:02:54.630 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.628+0000 7f1359600700 1 -- 172.21.15.151:0/1669904261 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13600bee60 con 0x559ab7efc390 2024-05-21T22:02:54.630 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.628+0000 7f1359600700 1 -- 172.21.15.151:0/1669904261 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8542+0+0 (secure 0 0 0) 0x7f13600a8e80 con 0x559ab7efc390 2024-05-21T22:02:54.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.628+0000 7f13811fbb80 1 -- 172.21.15.151:0/1669904261 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13600addc0 con 0x559ab7efc390 2024-05-21T22:02:54.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.628+0000 7f1359600700 1 -- 172.21.15.151:0/1669904261 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f13600b2870 con 0x559ab7efc390 2024-05-21T22:02:54.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.628+0000 7f13811fbb80 10 client.26966 fetch_fsmap learned FSMap version 29 2024-05-21T22:02:54.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.628+0000 7f13811fbb80 1 -- 172.21.15.151:0/1669904261 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559ab7e1a060 con 0x559ab7efc390 2024-05-21T22:02:54.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.628+0000 7f1359600700 1 -- 172.21.15.151:0/1669904261 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13600addc0 con 0x559ab7efc390 2024-05-21T22:02:54.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.628+0000 7f1359600700 1 -- 172.21.15.151:0/1669904261 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f13600a5380 con 0x559ab7efc390 2024-05-21T22:02:54.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.628+0000 7f13811fbb80 10 client.26966 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:02:54.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.628+0000 7f13811fbb80 10 client.26966 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-21T22:02:54.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.629+0000 7f13811fbb80 20 client.26966 populate_metadata read hostname 'smithi151' 2024-05-21T22:02:54.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.629+0000 7f13811fbb80 1 --2- 172.21.15.151:0/1669904261 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x559ab7f15650 0x559ab7f17a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:54.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.629+0000 7f13811fbb80 4 client.26966 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T22:02:54.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.629+0000 7f13811fbb80 1 -- 172.21.15.151:0/1669904261 --> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559ab7e5bd90 con 0x559ab7f15650 2024-05-21T22:02:54.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.629+0000 7f136ea00700 1 --2- 172.21.15.151:0/1669904261 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x559ab7f15650 0x559ab7f17a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:54.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.630+0000 7f136ea00700 1 --2- 172.21.15.151:0/1669904261 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x559ab7f15650 0x559ab7f17a40 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=26e9775cb7918173 server_cookie=a3f306f4c0d1d589 in_seq=0 out_seq=0 2024-05-21T22:02:54.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.630+0000 7f1359600700 10 client.26966 ms_handle_connect on v2:172.21.15.184:6838/3059935143 2024-05-21T22:02:54.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.631+0000 7f1359600700 1 -- 172.21.15.151:0/1669904261 <== mds.1 v2:172.21.15.184:6838/3059935143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x559ab7e5bd90 con 0x559ab7f15650 2024-05-21T22:02:54.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.631+0000 7f1359600700 10 client.26966 handle_command_reply: tid=0 2024-05-21T22:02:54.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.631+0000 7f13811fbb80 1 client.26966 shutdown 2024-05-21T22:02:54.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.631+0000 7f13811fbb80 1 -- 172.21.15.151:0/1669904261 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559ab7efc390 msgr2=0x559ab7f07ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:54.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.631+0000 7f13811fbb80 1 --2- 172.21.15.151:0/1669904261 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559ab7efc390 0x559ab7f07ea0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f13600a9710 tx=0x7f13600a9ea0 comp rx=0 tx=0).stop 2024-05-21T22:02:54.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.631+0000 7f13811fbb80 1 -- 172.21.15.151:0/1669904261 shutdown_connections 2024-05-21T22:02:54.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.631+0000 7f13811fbb80 1 --2- 172.21.15.151:0/1669904261 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x559ab7f15650 0x559ab7f17a40 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:54.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.631+0000 7f13811fbb80 1 --2- 172.21.15.151:0/1669904261 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559ab7efc390 0x559ab7f07ea0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:54.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.631+0000 7f13811fbb80 1 --2- 172.21.15.151:0/1669904261 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ab7f020e0 0x559ab7f05780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:54.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.631+0000 7f13811fbb80 1 --2- 172.21.15.151:0/1669904261 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ab7f017c0 0x559ab7f05240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:54.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.631+0000 7f13811fbb80 1 -- 172.21.15.151:0/1669904261 >> 172.21.15.151:0/1669904261 conn(0x559ab7e29be0 msgr2=0x559ab7ef5fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:02:54.634 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.631+0000 7f1359600700 0 client.26966 ms_handle_reset on v2:172.21.15.184:6838/3059935143 2024-05-21T22:02:54.634 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.631+0000 7f13811fbb80 1 -- 172.21.15.151:0/1669904261 shutdown_connections 2024-05-21T22:02:54.634 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.631+0000 7f13811fbb80 1 -- 172.21.15.151:0/1669904261 wait complete. 2024-05-21T22:02:54.634 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.631+0000 7f13811fbb80 20 client.26966 trim_cache size 0 max 16384 2024-05-21T22:02:54.648 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.648+0000 7f13811fbb80 1 Processor -- start 2024-05-21T22:02:54.648 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.648+0000 7f13811fbb80 1 -- start start 2024-05-21T22:02:54.648 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.648+0000 7f13811fbb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ab7efc390 0x559ab7efbd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:54.649 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.648+0000 7f13811fbb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ab7f017c0 0x559ab7ef6740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:54.649 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.648+0000 7f13811fbb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559ab7f020e0 0x559ab7f02bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:54.649 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.648+0000 7f13811fbb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f13701b0020 con 0x559ab7f017c0 2024-05-21T22:02:54.649 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.648+0000 7f13811fbb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f13701b1b60 con 0x559ab7f020e0 2024-05-21T22:02:54.649 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.648+0000 7f13811fbb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f13701b06f0 con 0x559ab7efc390 2024-05-21T22:02:54.650 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.648+0000 7f136ea00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ab7f017c0 0x559ab7ef6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:54.650 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.648+0000 7f136fe00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559ab7f020e0 0x559ab7f02bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:54.650 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.649+0000 7f136ea00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ab7f017c0 0x559ab7ef6740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:39566/0 (socket says 172.21.15.151:39566) 2024-05-21T22:02:54.650 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.649+0000 7f136ea00700 1 -- 172.21.15.151:0/836954369 learned_addr learned my addr 172.21.15.151:0/836954369 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:02:54.650 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.649+0000 7f136fe00700 1 --2- 172.21.15.151:0/836954369 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559ab7f020e0 0x559ab7f02bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:56064/0 (socket says 172.21.15.151:56064) 2024-05-21T22:02:54.650 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.649+0000 7f136f400700 1 --2- 172.21.15.151:0/836954369 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ab7efc390 0x559ab7efbd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:54.651 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.649+0000 7f136fe00700 1 -- 172.21.15.151:0/836954369 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ab7efc390 msgr2=0x559ab7efbd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:54.651 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.649+0000 7f136fe00700 1 --2- 172.21.15.151:0/836954369 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ab7efc390 0x559ab7efbd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:54.651 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.649+0000 7f136fe00700 1 -- 172.21.15.151:0/836954369 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ab7f017c0 msgr2=0x559ab7ef6740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:54.651 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.649+0000 7f136fe00700 1 --2- 172.21.15.151:0/836954369 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ab7f017c0 0x559ab7ef6740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:54.651 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.649+0000 7f136fe00700 1 -- 172.21.15.151:0/836954369 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ab7f037b0 con 0x559ab7f020e0 2024-05-21T22:02:54.651 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.649+0000 7f136ea00700 1 --2- 172.21.15.151:0/836954369 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ab7f017c0 0x559ab7ef6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:02:54.651 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.649+0000 7f136fe00700 1 --2- 172.21.15.151:0/836954369 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559ab7f020e0 0x559ab7f02bd0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f13680062a0 tx=0x7f136800b3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c954e092a17ca38 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:02:54.651 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.650+0000 7f135be00700 1 -- 172.21.15.151:0/836954369 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1368031020 con 0x559ab7f020e0 2024-05-21T22:02:54.651 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.650+0000 7f135be00700 1 -- 172.21.15.151:0/836954369 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1368007b90 con 0x559ab7f020e0 2024-05-21T22:02:54.651 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.650+0000 7f135be00700 1 -- 172.21.15.151:0/836954369 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f136802daa0 con 0x559ab7f020e0 2024-05-21T22:02:54.653 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.650+0000 7f13811fbb80 1 -- 172.21.15.151:0/836954369 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559ab7f020e0 msgr2=0x559ab7f02bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:54.653 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.650+0000 7f13811fbb80 1 --2- 172.21.15.151:0/836954369 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559ab7f020e0 0x559ab7f02bd0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f13680062a0 tx=0x7f136800b3b0 comp rx=0 tx=0).stop 2024-05-21T22:02:54.653 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.650+0000 7f13811fbb80 1 -- 172.21.15.151:0/836954369 shutdown_connections 2024-05-21T22:02:54.654 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.650+0000 7f13811fbb80 1 --2- 172.21.15.151:0/836954369 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559ab7f020e0 0x559ab7f02bd0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:54.654 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.650+0000 7f13811fbb80 1 --2- 172.21.15.151:0/836954369 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ab7efc390 0x559ab7efbd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:54.654 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.650+0000 7f13811fbb80 1 --2- 172.21.15.151:0/836954369 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ab7f017c0 0x559ab7ef6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:54.654 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.650+0000 7f13811fbb80 1 -- 172.21.15.151:0/836954369 >> 172.21.15.151:0/836954369 conn(0x559ab7e29be0 msgr2=0x559ab7f024c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:02:54.654 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.650+0000 7f13811fbb80 1 -- 172.21.15.151:0/836954369 shutdown_connections 2024-05-21T22:02:54.654 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.651+0000 7f13811fbb80 1 -- 172.21.15.151:0/836954369 wait complete. 2024-05-21T22:02:54.654 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.651+0000 7f13811fbb80 1 -- start start 2024-05-21T22:02:54.654 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.651+0000 7f13811fbb80 1 Processor -- start 2024-05-21T22:02:54.654 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.652+0000 7f13811fbb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ab7efc390 0x559ab7f18e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:54.654 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.652+0000 7f13811fbb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559ab7f017c0 0x559ab7f193b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:54.654 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.652+0000 7f13811fbb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ab7f020e0 0x559ab7f7ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:54.655 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.652+0000 7f13811fbb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f13701b1490 con 0x559ab7efc390 2024-05-21T22:02:54.655 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.652+0000 7f13811fbb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f13701b0dc0 con 0x559ab7f017c0 2024-05-21T22:02:54.655 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.652+0000 7f13811fbb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f13701adfe0 con 0x559ab7f020e0 2024-05-21T22:02:54.655 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.652+0000 7f13811fbb80 10 client.0 _ll_register_callbacks cb 0x559ab7df3c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:02:54.655 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.652+0000 7f136f400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ab7efc390 0x559ab7f18e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:54.655 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.652+0000 7f136f400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ab7efc390 0x559ab7f18e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:39570/0 (socket says 172.21.15.151:39570) 2024-05-21T22:02:54.655 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.652+0000 7f136f400700 1 -- 172.21.15.151:0/2447218867 learned_addr learned my addr 172.21.15.151:0/2447218867 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:02:54.655 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.652+0000 7f136ea00700 1 --2- 172.21.15.151:0/2447218867 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559ab7f017c0 0x559ab7f193b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:54.655 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.652+0000 7f136fe00700 1 --2- 172.21.15.151:0/2447218867 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ab7f020e0 0x559ab7f7ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:54.655 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.652+0000 7f136f400700 1 -- 172.21.15.151:0/2447218867 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ab7f020e0 msgr2=0x559ab7f7ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:54.655 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.652+0000 7f136f400700 1 --2- 172.21.15.151:0/2447218867 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ab7f020e0 0x559ab7f7ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:54.655 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.652+0000 7f136f400700 1 -- 172.21.15.151:0/2447218867 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559ab7f017c0 msgr2=0x559ab7f193b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:54.655 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.652+0000 7f136f400700 1 --2- 172.21.15.151:0/2447218867 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559ab7f017c0 0x559ab7f193b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:54.656 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.652+0000 7f136f400700 1 -- 172.21.15.151:0/2447218867 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ab7f7f3d0 con 0x559ab7efc390 2024-05-21T22:02:54.656 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.653+0000 7f136f400700 1 --2- 172.21.15.151:0/2447218867 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ab7efc390 0x559ab7f18e70 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f13600a5870 tx=0x7f13600e03e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbcbb97ca8d1b505 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:02:54.656 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.653+0000 7f1359600700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-21T22:02:54.656 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.653+0000 7f1359600700 1 -- 172.21.15.151:0/2447218867 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13600bee60 con 0x559ab7efc390 2024-05-21T22:02:54.656 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.653+0000 7f1359600700 1 -- 172.21.15.151:0/2447218867 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8542+0+0 (secure 0 0 0) 0x7f13600a5380 con 0x559ab7efc390 2024-05-21T22:02:54.656 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.653+0000 7f13811fbb80 1 -- 172.21.15.151:0/2447218867 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1368007b90 con 0x559ab7efc390 2024-05-21T22:02:54.656 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.653+0000 7f1359600700 1 -- 172.21.15.151:0/2447218867 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f13600a5580 con 0x559ab7efc390 2024-05-21T22:02:54.656 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.653+0000 7f13811fbb80 10 client.16845 fetch_fsmap learned FSMap version 29 2024-05-21T22:02:54.656 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.653+0000 7f13811fbb80 1 -- 172.21.15.151:0/2447218867 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559ab7f80630 con 0x559ab7efc390 2024-05-21T22:02:54.658 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.654+0000 7f1359600700 1 -- 172.21.15.151:0/2447218867 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1368007b90 con 0x559ab7efc390 2024-05-21T22:02:54.658 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.654+0000 7f1359600700 1 -- 172.21.15.151:0/2447218867 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f13600107b0 con 0x559ab7efc390 2024-05-21T22:02:54.658 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.654+0000 7f13811fbb80 10 client.16845 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:02:54.658 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.654+0000 7f13811fbb80 10 client.16845 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-21T22:02:54.658 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.654+0000 7f13811fbb80 20 client.16845 populate_metadata read hostname 'smithi151' 2024-05-21T22:02:54.658 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.654+0000 7f13811fbb80 1 --2- 172.21.15.151:0/2447218867 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x559ab7f80af0 0x559ab7f82ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:54.658 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.654+0000 7f13811fbb80 4 client.16845 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T22:02:54.659 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.654+0000 7f13811fbb80 1 -- 172.21.15.151:0/2447218867 --> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559ab7e55570 con 0x559ab7f80af0 2024-05-21T22:02:54.659 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.654+0000 7f136ea00700 1 --2- 172.21.15.151:0/2447218867 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x559ab7f80af0 0x559ab7f82ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:54.659 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.655+0000 7f136ea00700 1 --2- 172.21.15.151:0/2447218867 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x559ab7f80af0 0x559ab7f82ee0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9d33b1bd64e7c612 server_cookie=6934e6b4fb5b037b in_seq=0 out_seq=0 2024-05-21T22:02:54.659 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.655+0000 7f1359600700 10 client.16845 ms_handle_connect on v2:172.21.15.184:6838/3059935143 2024-05-21T22:02:54.659 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.655+0000 7f1359600700 1 -- 172.21.15.151:0/2447218867 <== mds.1 v2:172.21.15.184:6838/3059935143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559ab7e55570 con 0x559ab7f80af0 2024-05-21T22:02:54.659 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.655+0000 7f1359600700 10 client.16845 handle_command_reply: tid=0 2024-05-21T22:02:54.659 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.655+0000 7f13811fbb80 1 client.16845 shutdown 2024-05-21T22:02:54.659 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.655+0000 7f13811fbb80 1 -- 172.21.15.151:0/2447218867 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ab7efc390 msgr2=0x559ab7f18e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:54.659 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.655+0000 7f13811fbb80 1 --2- 172.21.15.151:0/2447218867 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ab7efc390 0x559ab7f18e70 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f13600a5870 tx=0x7f13600e03e0 comp rx=0 tx=0).stop 2024-05-21T22:02:54.659 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.655+0000 7f13811fbb80 1 -- 172.21.15.151:0/2447218867 shutdown_connections 2024-05-21T22:02:54.659 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.655+0000 7f13811fbb80 1 --2- 172.21.15.151:0/2447218867 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x559ab7f80af0 0x559ab7f82ee0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:54.659 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.655+0000 7f13811fbb80 1 --2- 172.21.15.151:0/2447218867 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ab7f020e0 0x559ab7f7ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:54.659 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.655+0000 7f13811fbb80 1 --2- 172.21.15.151:0/2447218867 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559ab7efc390 0x559ab7f18e70 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:54.660 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.655+0000 7f13811fbb80 1 --2- 172.21.15.151:0/2447218867 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559ab7f017c0 0x559ab7f193b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:54.660 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.655+0000 7f1359600700 0 client.16845 ms_handle_reset on v2:172.21.15.184:6838/3059935143 2024-05-21T22:02:54.660 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.655+0000 7f13811fbb80 1 -- 172.21.15.151:0/2447218867 >> 172.21.15.151:0/2447218867 conn(0x559ab7e29be0 msgr2=0x559ab7f02d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:02:54.660 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.656+0000 7f13811fbb80 1 -- 172.21.15.151:0/2447218867 shutdown_connections 2024-05-21T22:02:54.660 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.656+0000 7f13811fbb80 1 -- 172.21.15.151:0/2447218867 wait complete. 2024-05-21T22:02:54.660 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.656+0000 7f13811fbb80 20 client.16845 trim_cache size 0 max 16384 2024-05-21T22:02:54.660 INFO:teuthology.orchestra.run.smithi151.stdout:[] 2024-05-21T22:02:54.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.657+0000 7f1375200700 1 -- 172.21.15.151:0/1034917711 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f13540b29f0 msgr2=0x7f13540b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:54.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.657+0000 7f1375200700 1 --2- 172.21.15.151:0/1034917711 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f13540b29f0 0x7f13540b4eb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f136800b780 tx=0x7f1368009d40 comp rx=0 tx=0).stop 2024-05-21T22:02:54.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.657+0000 7f1375200700 1 -- 172.21.15.151:0/1034917711 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1370105670 msgr2=0x7f13701a1940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:54.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.657+0000 7f1375200700 1 --2- 172.21.15.151:0/1034917711 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1370105670 0x7f13701a1940 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f136000e9a0 tx=0x7f136000ecb0 comp rx=0 tx=0).stop 2024-05-21T22:02:54.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.657+0000 7f1375200700 1 -- 172.21.15.151:0/1034917711 shutdown_connections 2024-05-21T22:02:54.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.657+0000 7f1375200700 1 --2- 172.21.15.151:0/1034917711 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f13540b29f0 0x7f13540b4eb0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:54.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.657+0000 7f1375200700 1 --2- 172.21.15.151:0/1034917711 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f137010b670 0x7f137019bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:54.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.657+0000 7f1375200700 1 --2- 172.21.15.151:0/1034917711 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1370105670 0x7f13701a1940 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:54.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.657+0000 7f1375200700 1 --2- 172.21.15.151:0/1034917711 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1370106030 0x7f13701a1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:54.662 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.657+0000 7f1375200700 1 -- 172.21.15.151:0/1034917711 >> 172.21.15.151:0/1034917711 conn(0x7f1370101160 msgr2=0x7f1370101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:02:54.662 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.657+0000 7f1375200700 1 -- 172.21.15.151:0/1034917711 shutdown_connections 2024-05-21T22:02:54.662 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:54.657+0000 7f1375200700 1 -- 172.21.15.151:0/1034917711 wait complete. 2024-05-21T22:02:55.472 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-21T22:02:55.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:02:55 smithi151 ceph-mon[31623]: pgmap v224: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:02:55.939 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:02:55.985 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:02:55 smithi184 ceph-mon[36942]: pgmap v224: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:02:56.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:02:55 smithi164 ceph-mon[36269]: pgmap v224: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:02:57.805 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:02:57 smithi151 ceph-mon[31623]: pgmap v225: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:02:57.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:02:57 smithi184 ceph-mon[36942]: pgmap v225: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:02:58.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:02:57 smithi164 ceph-mon[36269]: pgmap v225: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:02:58.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.079+0000 7f832b600700 1 -- 172.21.15.151:0/3874764156 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8324075770 msgr2=0x7f8324075b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:58.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.079+0000 7f832b600700 1 --2- 172.21.15.151:0/3874764156 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8324075770 0x7f8324075b50 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f8314009990 tx=0x7f8314009ca0 comp rx=0 tx=0).stop 2024-05-21T22:02:58.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.079+0000 7f832b600700 1 -- 172.21.15.151:0/3874764156 shutdown_connections 2024-05-21T22:02:58.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.079+0000 7f832b600700 1 --2- 172.21.15.151:0/3874764156 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f832410ff40 0x7f8324112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:58.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.079+0000 7f832b600700 1 --2- 172.21.15.151:0/3874764156 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8324075770 0x7f8324075b50 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:58.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.079+0000 7f832b600700 1 --2- 172.21.15.151:0/3874764156 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8324076090 0x7f832410fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:58.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.079+0000 7f832b600700 1 -- 172.21.15.151:0/3874764156 >> 172.21.15.151:0/3874764156 conn(0x7f832407a6a0 msgr2=0x7f832407aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:02:58.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.079+0000 7f832b600700 1 -- 172.21.15.151:0/3874764156 shutdown_connections 2024-05-21T22:02:58.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.080+0000 7f832b600700 1 -- 172.21.15.151:0/3874764156 wait complete. 2024-05-21T22:02:58.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.081+0000 7f832b600700 1 Processor -- start 2024-05-21T22:02:58.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.081+0000 7f832b600700 1 -- start start 2024-05-21T22:02:58.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.082+0000 7f832b600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8324075770 0x7f832419bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:58.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.082+0000 7f832b600700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8324076090 0x7f832419c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:58.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.082+0000 7f832b600700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f832410ff40 0x7f83241a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:58.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.082+0000 7f832b600700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8324115960 con 0x7f8324075770 2024-05-21T22:02:58.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.082+0000 7f832b600700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f83241157e0 con 0x7f8324076090 2024-05-21T22:02:58.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.082+0000 7f832b600700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8324115ae0 con 0x7f832410ff40 2024-05-21T22:02:58.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.082+0000 7f8328c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8324075770 0x7f832419bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:58.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.082+0000 7f8329600700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f832410ff40 0x7f83241a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:58.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.082+0000 7f8328c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8324075770 0x7f832419bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49180/0 (socket says 172.21.15.151:49180) 2024-05-21T22:02:58.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.082+0000 7f8328c00700 1 -- 172.21.15.151:0/981146899 learned_addr learned my addr 172.21.15.151:0/981146899 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:02:58.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.082+0000 7f831be00700 1 --2- 172.21.15.151:0/981146899 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8324076090 0x7f832419c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:58.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.082+0000 7f8329600700 1 -- 172.21.15.151:0/981146899 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8324076090 msgr2=0x7f832419c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:58.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.082+0000 7f8329600700 1 --2- 172.21.15.151:0/981146899 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8324076090 0x7f832419c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:58.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.082+0000 7f8329600700 1 -- 172.21.15.151:0/981146899 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8324075770 msgr2=0x7f832419bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:58.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.082+0000 7f8329600700 1 --2- 172.21.15.151:0/981146899 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8324075770 0x7f832419bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:58.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.082+0000 7f8329600700 1 -- 172.21.15.151:0/981146899 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83241a0e00 con 0x7f832410ff40 2024-05-21T22:02:58.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.083+0000 7f8329600700 1 --2- 172.21.15.151:0/981146899 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f832410ff40 0x7f83241a06c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f831c00ef90 tx=0x7f831c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:02:58.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.083+0000 7f8322000700 1 -- 172.21.15.151:0/981146899 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f831c0092e0 con 0x7f832410ff40 2024-05-21T22:02:58.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.083+0000 7f8322000700 1 -- 172.21.15.151:0/981146899 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f831c004bb0 con 0x7f832410ff40 2024-05-21T22:02:58.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.083+0000 7f832b600700 1 -- 172.21.15.151:0/981146899 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83241a1090 con 0x7f832410ff40 2024-05-21T22:02:58.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.083+0000 7f8322000700 1 -- 172.21.15.151:0/981146899 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f831c004020 con 0x7f832410ff40 2024-05-21T22:02:58.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.083+0000 7f832b600700 1 -- 172.21.15.151:0/981146899 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83241a8970 con 0x7f832410ff40 2024-05-21T22:02:58.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.087+0000 7f8322000700 1 -- 172.21.15.151:0/981146899 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7f831c01b030 con 0x7f832410ff40 2024-05-21T22:02:58.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.088+0000 7f8335f2fb80 1 Processor -- start 2024-05-21T22:02:58.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.088+0000 7f8335f2fb80 1 -- start start 2024-05-21T22:02:58.089 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.088+0000 7f8335f2fb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557357dd9610 0x557357dd99f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:58.089 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.088+0000 7f8335f2fb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557357dd9f30 0x557357dd3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:58.089 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.088+0000 7f8335f2fb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557357dd4140 0x557357dd45c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:58.089 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.088+0000 7f8335f2fb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x557357d31e40 con 0x557357dd9610 2024-05-21T22:02:58.089 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.088+0000 7f8335f2fb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x557357d219e0 con 0x557357dd4140 2024-05-21T22:02:58.089 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.088+0000 7f8335f2fb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x557357d25a20 con 0x557357dd9f30 2024-05-21T22:02:58.089 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.089+0000 7f8322000700 1 --2- 172.21.15.151:0/981146899 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f83080b29f0 0x7f83080b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:58.090 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.089+0000 7f8328c00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557357dd9f30 0x557357dd3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:58.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.089+0000 7f8328c00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557357dd9f30 0x557357dd3b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:53270/0 (socket says 172.21.15.151:53270) 2024-05-21T22:02:58.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.089+0000 7f8328c00700 1 -- 172.21.15.151:0/2813828778 learned_addr learned my addr 172.21.15.151:0/2813828778 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:02:58.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.089+0000 7f831be00700 1 --2- 172.21.15.151:0/2813828778 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557357dd9610 0x557357dd99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:58.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.089+0000 7f8322000700 1 -- 172.21.15.151:0/981146899 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8542+0+0 (secure 0 0 0) 0x7f831c0e2190 con 0x7f832410ff40 2024-05-21T22:02:58.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.089+0000 7f8329600700 1 --2- 172.21.15.151:0/981146899 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f83080b29f0 0x7f83080b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:58.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.089+0000 7f831be00700 1 --2- 172.21.15.151:0/2813828778 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557357dd4140 0x557357dd45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:58.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.089+0000 7f8328c00700 1 -- 172.21.15.151:0/2813828778 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557357dd4140 msgr2=0x557357dd45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:58.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.089+0000 7f8328c00700 1 --2- 172.21.15.151:0/2813828778 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557357dd4140 0x557357dd45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:58.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.089+0000 7f8328c00700 1 -- 172.21.15.151:0/2813828778 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557357dd9610 msgr2=0x557357dd99f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:58.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.089+0000 7f8328c00700 1 --2- 172.21.15.151:0/2813828778 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557357dd9610 0x557357dd99f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:58.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.090+0000 7f8328c00700 1 -- 172.21.15.151:0/2813828778 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557357cea420 con 0x557357dd9f30 2024-05-21T22:02:58.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.090+0000 7f831be00700 1 --2- 172.21.15.151:0/2813828778 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557357dd4140 0x557357dd45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T22:02:58.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.090+0000 7f8328c00700 1 --2- 172.21.15.151:0/2813828778 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557357dd9f30 0x557357dd3b20 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f83140096d0 tx=0x7f8314009700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28048fa0e6d193e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:02:58.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.090+0000 7f8329600700 1 --2- 172.21.15.151:0/981146899 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f83080b29f0 0x7f83080b4eb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f831c00c6f0 tx=0x7f831c009480 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:02:58.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.090+0000 7f831a000700 1 -- 172.21.15.151:0/2813828778 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8314027070 con 0x557357dd9f30 2024-05-21T22:02:58.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.090+0000 7f831a000700 1 -- 172.21.15.151:0/2813828778 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83140056d0 con 0x557357dd9f30 2024-05-21T22:02:58.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.090+0000 7f831a000700 1 -- 172.21.15.151:0/2813828778 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8314003c70 con 0x557357dd9f30 2024-05-21T22:02:58.093 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.091+0000 7f8335f2fb80 1 -- 172.21.15.151:0/2813828778 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557357dd9f30 msgr2=0x557357dd3b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:58.094 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.091+0000 7f8335f2fb80 1 --2- 172.21.15.151:0/2813828778 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557357dd9f30 0x557357dd3b20 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f83140096d0 tx=0x7f8314009700 comp rx=0 tx=0).stop 2024-05-21T22:02:58.094 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.091+0000 7f8335f2fb80 1 -- 172.21.15.151:0/2813828778 shutdown_connections 2024-05-21T22:02:58.094 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.091+0000 7f8335f2fb80 1 --2- 172.21.15.151:0/2813828778 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557357dd4140 0x557357dd45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:58.094 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.091+0000 7f8335f2fb80 1 --2- 172.21.15.151:0/2813828778 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557357dd9610 0x557357dd99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:58.094 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.091+0000 7f8335f2fb80 1 --2- 172.21.15.151:0/2813828778 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557357dd9f30 0x557357dd3b20 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:58.094 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.091+0000 7f8335f2fb80 1 -- 172.21.15.151:0/2813828778 >> 172.21.15.151:0/2813828778 conn(0x557357d01be0 msgr2=0x557357dce690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:02:58.094 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.091+0000 7f8335f2fb80 1 -- 172.21.15.151:0/2813828778 shutdown_connections 2024-05-21T22:02:58.094 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.091+0000 7f8335f2fb80 1 -- 172.21.15.151:0/2813828778 wait complete. 2024-05-21T22:02:58.094 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.092+0000 7f8335f2fb80 1 -- start start 2024-05-21T22:02:58.094 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.092+0000 7f8335f2fb80 1 Processor -- start 2024-05-21T22:02:58.094 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.092+0000 7f8335f2fb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557357dd4140 0x557357ddfcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:58.094 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.092+0000 7f8335f2fb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557357dd9610 0x557357ddd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:58.094 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.093+0000 7f8335f2fb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557357dd9f30 0x557357ddd590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:58.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.093+0000 7f8335f2fb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x557357c27fa0 con 0x557357dd4140 2024-05-21T22:02:58.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.093+0000 7f8335f2fb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x557357cc2760 con 0x557357dd9f30 2024-05-21T22:02:58.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.093+0000 7f8335f2fb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x557357de0980 con 0x557357dd9610 2024-05-21T22:02:58.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.093+0000 7f8335f2fb80 10 client.0 _ll_register_callbacks cb 0x557357ccbc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:02:58.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.093+0000 7f831be00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557357dd4140 0x557357ddfcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:58.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.093+0000 7f8328c00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557357dd9610 0x557357ddd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:58.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.093+0000 7f8328c00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557357dd9610 0x557357ddd050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:53286/0 (socket says 172.21.15.151:53286) 2024-05-21T22:02:58.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.093+0000 7f8328c00700 1 -- 172.21.15.151:0/2438533518 learned_addr learned my addr 172.21.15.151:0/2438533518 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:02:58.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.093+0000 7f831be00700 1 --2- 172.21.15.151:0/2438533518 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557357dd9f30 0x557357ddd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:58.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.093+0000 7f8328c00700 1 -- 172.21.15.151:0/2438533518 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557357dd9f30 msgr2=0x557357ddd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:58.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.093+0000 7f8328c00700 1 --2- 172.21.15.151:0/2438533518 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557357dd9f30 0x557357ddd590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:58.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.093+0000 7f8328c00700 1 -- 172.21.15.151:0/2438533518 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557357dd4140 msgr2=0x557357ddfcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:58.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.093+0000 7f8328c00700 1 --2- 172.21.15.151:0/2438533518 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557357dd4140 0x557357ddfcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:58.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.093+0000 7f8328c00700 1 -- 172.21.15.151:0/2438533518 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557357c718d0 con 0x557357dd9610 2024-05-21T22:02:58.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.094+0000 7f8328c00700 1 --2- 172.21.15.151:0/2438533518 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557357dd9610 0x557357ddd050 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f83140096a0 tx=0x7f8314003e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c46f4e923d64c62f server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:02:58.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.094+0000 7f8312a00700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-21T22:02:58.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.094+0000 7f8312a00700 1 -- 172.21.15.151:0/2438533518 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8314033060 con 0x557357dd9610 2024-05-21T22:02:58.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.094+0000 7f8312a00700 1 -- 172.21.15.151:0/2438533518 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8542+0+0 (secure 0 0 0) 0x7f831400b920 con 0x557357dd9610 2024-05-21T22:02:58.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.094+0000 7f8335f2fb80 1 -- 172.21.15.151:0/2438533518 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83140056d0 con 0x557357dd9610 2024-05-21T22:02:58.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.094+0000 7f8312a00700 1 -- 172.21.15.151:0/2438533518 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8314027070 con 0x557357dd9610 2024-05-21T22:02:58.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.094+0000 7f8335f2fb80 10 client.26995 fetch_fsmap learned FSMap version 29 2024-05-21T22:02:58.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.095+0000 7f8335f2fb80 1 -- 172.21.15.151:0/2438533518 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557357cf2060 con 0x557357dd9610 2024-05-21T22:02:58.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.095+0000 7f8312a00700 1 -- 172.21.15.151:0/2438533518 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83140056d0 con 0x557357dd9610 2024-05-21T22:02:58.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.095+0000 7f8312a00700 1 -- 172.21.15.151:0/2438533518 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f8314021070 con 0x557357dd9610 2024-05-21T22:02:58.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.095+0000 7f8335f2fb80 10 client.26995 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:02:58.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.095+0000 7f8335f2fb80 10 client.26995 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T22:02:58.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.095+0000 7f8335f2fb80 20 client.26995 populate_metadata read hostname 'smithi151' 2024-05-21T22:02:58.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.095+0000 7f8335f2fb80 1 --2- 172.21.15.151:0/2438533518 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x557357ded390 0x557357def780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:58.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.095+0000 7f8335f2fb80 4 client.26995 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T22:02:58.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.095+0000 7f8335f2fb80 1 -- 172.21.15.151:0/2438533518 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557357d33d90 con 0x557357ded390 2024-05-21T22:02:58.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.096+0000 7f831be00700 1 --2- 172.21.15.151:0/2438533518 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x557357ded390 0x557357def780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:58.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.096+0000 7f831be00700 1 --2- 172.21.15.151:0/2438533518 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x557357ded390 0x557357def780 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8cf1d44b7d8c5c71 server_cookie=14e750b3e929c720 in_seq=0 out_seq=0 2024-05-21T22:02:58.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.096+0000 7f8312a00700 10 client.26995 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T22:02:58.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.097+0000 7f8312a00700 1 -- 172.21.15.151:0/2438533518 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x557357d33d90 con 0x557357ded390 2024-05-21T22:02:58.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.097+0000 7f8312a00700 10 client.26995 handle_command_reply: tid=0 2024-05-21T22:02:58.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.097+0000 7f8335f2fb80 1 client.26995 shutdown 2024-05-21T22:02:58.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.097+0000 7f8335f2fb80 1 -- 172.21.15.151:0/2438533518 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557357dd9610 msgr2=0x557357ddd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:58.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.097+0000 7f8335f2fb80 1 --2- 172.21.15.151:0/2438533518 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557357dd9610 0x557357ddd050 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f83140096a0 tx=0x7f8314003e70 comp rx=0 tx=0).stop 2024-05-21T22:02:58.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.097+0000 7f8335f2fb80 1 -- 172.21.15.151:0/2438533518 shutdown_connections 2024-05-21T22:02:58.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.097+0000 7f8335f2fb80 1 --2- 172.21.15.151:0/2438533518 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x557357ded390 0x557357def780 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:58.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.097+0000 7f8335f2fb80 1 --2- 172.21.15.151:0/2438533518 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557357dd9f30 0x557357ddd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:58.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.097+0000 7f8335f2fb80 1 --2- 172.21.15.151:0/2438533518 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557357dd4140 0x557357ddfcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:58.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.097+0000 7f8335f2fb80 1 --2- 172.21.15.151:0/2438533518 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557357dd9610 0x557357ddd050 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:58.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.097+0000 7f8312a00700 0 client.26995 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T22:02:58.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.097+0000 7f8335f2fb80 1 -- 172.21.15.151:0/2438533518 >> 172.21.15.151:0/2438533518 conn(0x557357d01be0 msgr2=0x557357dcdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:02:58.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.098+0000 7f8335f2fb80 1 -- 172.21.15.151:0/2438533518 shutdown_connections 2024-05-21T22:02:58.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.098+0000 7f8335f2fb80 1 -- 172.21.15.151:0/2438533518 wait complete. 2024-05-21T22:02:58.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.098+0000 7f8335f2fb80 20 client.26995 trim_cache size 0 max 16384 2024-05-21T22:02:58.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.114+0000 7f8335f2fb80 1 Processor -- start 2024-05-21T22:02:58.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.114+0000 7f8335f2fb80 1 -- start start 2024-05-21T22:02:58.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.115+0000 7f8335f2fb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557357dd4140 0x557357dd3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:58.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.115+0000 7f8335f2fb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557357dd9610 0x557357dce4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:58.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.115+0000 7f8335f2fb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557357dd9f30 0x557357df27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:58.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.115+0000 7f8335f2fb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f83241b0010 con 0x557357dd9f30 2024-05-21T22:02:58.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.115+0000 7f8335f2fb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f83241b1b50 con 0x557357dd4140 2024-05-21T22:02:58.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.115+0000 7f8335f2fb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f83241b06e0 con 0x557357dd9610 2024-05-21T22:02:58.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.115+0000 7f831be00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557357dd4140 0x557357dd3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:58.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.115+0000 7f831be00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557357dd9f30 0x557357df27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:58.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.115+0000 7f831be00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557357dd9f30 0x557357df27a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49216/0 (socket says 172.21.15.151:49216) 2024-05-21T22:02:58.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.115+0000 7f831be00700 1 -- 172.21.15.151:0/2234601088 learned_addr learned my addr 172.21.15.151:0/2234601088 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:02:58.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.115+0000 7f8328c00700 1 --2- 172.21.15.151:0/2234601088 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557357dd9610 0x557357dce4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:58.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.116+0000 7f8328c00700 1 -- 172.21.15.151:0/2234601088 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557357dd4140 msgr2=0x557357dd3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:58.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.116+0000 7f8328c00700 1 --2- 172.21.15.151:0/2234601088 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557357dd4140 0x557357dd3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:58.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.116+0000 7f8328c00700 1 -- 172.21.15.151:0/2234601088 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557357dd9f30 msgr2=0x557357df27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:58.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.116+0000 7f8328c00700 1 --2- 172.21.15.151:0/2234601088 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557357dd9f30 0x557357df27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:58.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.116+0000 7f8328c00700 1 -- 172.21.15.151:0/2234601088 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557357dddf30 con 0x557357dd9610 2024-05-21T22:02:58.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.116+0000 7f831be00700 1 --2- 172.21.15.151:0/2234601088 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557357dd9f30 0x557357df27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:02:58.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.116+0000 7f8328c00700 1 --2- 172.21.15.151:0/2234601088 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557357dd9610 0x557357dce4f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f8314005010 tx=0x7f83140098c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=866eda1d15aa38d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:02:58.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.116+0000 7f8319600700 1 -- 172.21.15.151:0/2234601088 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8314033040 con 0x557357dd9610 2024-05-21T22:02:58.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.116+0000 7f8319600700 1 -- 172.21.15.151:0/2234601088 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8314028040 con 0x557357dd9610 2024-05-21T22:02:58.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.116+0000 7f8319600700 1 -- 172.21.15.151:0/2234601088 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f831400bbf0 con 0x557357dd9610 2024-05-21T22:02:58.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.117+0000 7f8335f2fb80 1 -- 172.21.15.151:0/2234601088 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557357dd9610 msgr2=0x557357dce4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:58.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.117+0000 7f8335f2fb80 1 --2- 172.21.15.151:0/2234601088 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557357dd9610 0x557357dce4f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f8314005010 tx=0x7f83140098c0 comp rx=0 tx=0).stop 2024-05-21T22:02:58.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.117+0000 7f8335f2fb80 1 -- 172.21.15.151:0/2234601088 shutdown_connections 2024-05-21T22:02:58.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.117+0000 7f8335f2fb80 1 --2- 172.21.15.151:0/2234601088 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557357dd4140 0x557357dd3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:58.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.117+0000 7f8335f2fb80 1 --2- 172.21.15.151:0/2234601088 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557357dd9f30 0x557357df27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:58.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.117+0000 7f8335f2fb80 1 --2- 172.21.15.151:0/2234601088 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557357dd9610 0x557357dce4f0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:58.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.117+0000 7f8335f2fb80 1 -- 172.21.15.151:0/2234601088 >> 172.21.15.151:0/2234601088 conn(0x557357d01be0 msgr2=0x557357ddd010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:02:58.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.117+0000 7f8335f2fb80 1 -- 172.21.15.151:0/2234601088 shutdown_connections 2024-05-21T22:02:58.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.117+0000 7f8335f2fb80 1 -- 172.21.15.151:0/2234601088 wait complete. 2024-05-21T22:02:58.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.118+0000 7f8335f2fb80 1 -- start start 2024-05-21T22:02:58.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.118+0000 7f8335f2fb80 1 Processor -- start 2024-05-21T22:02:58.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.118+0000 7f8335f2fb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557357dd4140 0x557357defd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:58.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.118+0000 7f8335f2fb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557357dd9610 0x557357df0260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:58.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.118+0000 7f8335f2fb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557357dd9f30 0x557357e5e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:58.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.118+0000 7f8335f2fb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f83241b1480 con 0x557357dd9f30 2024-05-21T22:02:58.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.118+0000 7f8335f2fb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f83241b0db0 con 0x557357dd9610 2024-05-21T22:02:58.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.119+0000 7f8335f2fb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f83241adfd0 con 0x557357dd4140 2024-05-21T22:02:58.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.119+0000 7f8335f2fb80 10 client.0 _ll_register_callbacks cb 0x557357ccbc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:02:58.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.119+0000 7f8328c00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557357dd9610 0x557357df0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:58.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.119+0000 7f8328c00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557357dd9610 0x557357df0260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:43282/0 (socket says 172.21.15.151:43282) 2024-05-21T22:02:58.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.119+0000 7f8328c00700 1 -- 172.21.15.151:0/3215077855 learned_addr learned my addr 172.21.15.151:0/3215077855 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:02:58.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.119+0000 7f831be00700 1 --2- 172.21.15.151:0/3215077855 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557357dd4140 0x557357defd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:58.121 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.119+0000 7f831be00700 1 --2- 172.21.15.151:0/3215077855 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557357dd9f30 0x557357e5e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:58.122 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.119+0000 7f8328c00700 1 -- 172.21.15.151:0/3215077855 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557357dd4140 msgr2=0x557357defd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:58.122 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.119+0000 7f8328c00700 1 --2- 172.21.15.151:0/3215077855 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557357dd4140 0x557357defd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:58.122 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.119+0000 7f8328c00700 1 -- 172.21.15.151:0/3215077855 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557357dd9f30 msgr2=0x557357e5e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:02:58.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.119+0000 7f8328c00700 1 --2- 172.21.15.151:0/3215077855 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557357dd9f30 0x557357e5e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:58.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.119+0000 7f8328c00700 1 -- 172.21.15.151:0/3215077855 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557357e5e9c0 con 0x557357dd9610 2024-05-21T22:02:58.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.119+0000 7f831be00700 1 --2- 172.21.15.151:0/3215077855 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557357dd9f30 0x557357e5e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T22:02:58.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.120+0000 7f8328c00700 1 --2- 172.21.15.151:0/3215077855 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557357dd9610 0x557357df0260 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f8314009ce0 tx=0x7f831400f710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f48c890d1ac511bd server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:02:58.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.120+0000 7f8312a00700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-21T22:02:58.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.120+0000 7f8312a00700 1 -- 172.21.15.151:0/3215077855 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8314033040 con 0x557357dd9610 2024-05-21T22:02:58.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.120+0000 7f8335f2fb80 1 -- 172.21.15.151:0/3215077855 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8314028040 con 0x557357dd9610 2024-05-21T22:02:58.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.120+0000 7f8312a00700 1 -- 172.21.15.151:0/3215077855 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8542+0+0 (secure 0 0 0) 0x7f831400b720 con 0x557357dd9610 2024-05-21T22:02:58.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.120+0000 7f8312a00700 1 -- 172.21.15.151:0/3215077855 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f831404d050 con 0x557357dd9610 2024-05-21T22:02:58.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.120+0000 7f8335f2fb80 10 client.26990 fetch_fsmap learned FSMap version 29 2024-05-21T22:02:58.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.120+0000 7f8335f2fb80 1 -- 172.21.15.151:0/3215077855 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557357e5fca0 con 0x557357dd9610 2024-05-21T22:02:58.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.121+0000 7f8312a00700 1 -- 172.21.15.151:0/3215077855 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8314028040 con 0x557357dd9610 2024-05-21T22:02:58.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.121+0000 7f8312a00700 1 -- 172.21.15.151:0/3215077855 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f8314021070 con 0x557357dd9610 2024-05-21T22:02:58.124 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.121+0000 7f8335f2fb80 10 client.26990 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:02:58.124 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.121+0000 7f8335f2fb80 10 client.26990 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T22:02:58.124 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.121+0000 7f8335f2fb80 20 client.26990 populate_metadata read hostname 'smithi151' 2024-05-21T22:02:58.124 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.121+0000 7f8335f2fb80 1 --2- 172.21.15.151:0/3215077855 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x557357e602e0 0x557357e62700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:02:58.124 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.121+0000 7f8335f2fb80 4 client.26990 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T22:02:58.124 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.121+0000 7f8335f2fb80 1 -- 172.21.15.151:0/3215077855 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557357d2d570 con 0x557357e602e0 2024-05-21T22:02:58.124 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.122+0000 7f831be00700 1 --2- 172.21.15.151:0/3215077855 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x557357e602e0 0x557357e62700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:02:58.124 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.122+0000 7f8312a00700 10 client.26990 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T22:02:58.124 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.122+0000 7f831be00700 1 --2- 172.21.15.151:0/3215077855 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x557357e602e0 0x557357e62700 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80cadd948a095a14 server_cookie=531faa74bb6280af in_seq=0 out_seq=0 2024-05-21T22:02:58.124 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.122+0000 7f8312a00700 1 -- 172.21.15.151:0/3215077855 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557357d2d570 con 0x557357e602e0 2024-05-21T22:02:58.124 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.122+0000 7f8312a00700 10 client.26990 handle_command_reply: tid=0 2024-05-21T22:02:58.124 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.122+0000 7f8335f2fb80 1 client.26990 shutdown 2024-05-21T22:02:58.125 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.122+0000 7f8335f2fb80 1 -- 172.21.15.151:0/3215077855 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557357dd9610 msgr2=0x557357df0260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:58.125 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.122+0000 7f8335f2fb80 1 --2- 172.21.15.151:0/3215077855 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557357dd9610 0x557357df0260 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f8314009ce0 tx=0x7f831400f710 comp rx=0 tx=0).stop 2024-05-21T22:02:58.125 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.123+0000 7f8335f2fb80 1 -- 172.21.15.151:0/3215077855 shutdown_connections 2024-05-21T22:02:58.125 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.123+0000 7f8335f2fb80 1 --2- 172.21.15.151:0/3215077855 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557357dd9f30 0x557357e5e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:58.125 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.123+0000 7f8335f2fb80 1 --2- 172.21.15.151:0/3215077855 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557357dd4140 0x557357defd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:58.125 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.123+0000 7f8335f2fb80 1 --2- 172.21.15.151:0/3215077855 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x557357e602e0 0x557357e62700 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:58.125 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.123+0000 7f8335f2fb80 1 --2- 172.21.15.151:0/3215077855 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557357dd9610 0x557357df0260 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:58.126 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.123+0000 7f8312a00700 0 client.26990 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T22:02:58.126 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.123+0000 7f8335f2fb80 1 -- 172.21.15.151:0/3215077855 >> 172.21.15.151:0/3215077855 conn(0x557357d01be0 msgr2=0x557357dda310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:02:58.126 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.123+0000 7f8335f2fb80 1 -- 172.21.15.151:0/3215077855 shutdown_connections 2024-05-21T22:02:58.126 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.123+0000 7f8335f2fb80 1 -- 172.21.15.151:0/3215077855 wait complete. 2024-05-21T22:02:58.126 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.123+0000 7f8335f2fb80 20 client.26990 trim_cache size 0 max 16384 2024-05-21T22:02:58.126 INFO:teuthology.orchestra.run.smithi151.stdout:[] 2024-05-21T22:02:58.127 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.124+0000 7f832b600700 1 -- 172.21.15.151:0/981146899 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f83080b29f0 msgr2=0x7f83080b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:58.127 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.124+0000 7f832b600700 1 --2- 172.21.15.151:0/981146899 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f83080b29f0 0x7f83080b4eb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f831c00c6f0 tx=0x7f831c009480 comp rx=0 tx=0).stop 2024-05-21T22:02:58.127 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.124+0000 7f832b600700 1 -- 172.21.15.151:0/981146899 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f832410ff40 msgr2=0x7f83241a06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:02:58.127 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.124+0000 7f832b600700 1 --2- 172.21.15.151:0/981146899 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f832410ff40 0x7f83241a06c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f831c00ef90 tx=0x7f831c00c3a0 comp rx=0 tx=0).stop 2024-05-21T22:02:58.127 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.124+0000 7f832b600700 1 -- 172.21.15.151:0/981146899 shutdown_connections 2024-05-21T22:02:58.127 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.124+0000 7f832b600700 1 --2- 172.21.15.151:0/981146899 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f83080b29f0 0x7f83080b4eb0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:58.127 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.124+0000 7f832b600700 1 --2- 172.21.15.151:0/981146899 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f832410ff40 0x7f83241a06c0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:58.127 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.124+0000 7f832b600700 1 --2- 172.21.15.151:0/981146899 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8324075770 0x7f832419bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:58.127 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.124+0000 7f832b600700 1 --2- 172.21.15.151:0/981146899 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8324076090 0x7f832419c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:02:58.127 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.124+0000 7f832b600700 1 -- 172.21.15.151:0/981146899 >> 172.21.15.151:0/981146899 conn(0x7f832407a6a0 msgr2=0x7f83241006d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:02:58.127 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.124+0000 7f832b600700 1 -- 172.21.15.151:0/981146899 shutdown_connections 2024-05-21T22:02:58.128 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:02:58.124+0000 7f832b600700 1 -- 172.21.15.151:0/981146899 wait complete. 2024-05-21T22:02:59.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:02:59 smithi151 ceph-mon[31623]: pgmap v226: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T22:02:59.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:02:59 smithi184 ceph-mon[36942]: pgmap v226: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T22:03:00.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:02:59 smithi164 ceph-mon[36269]: pgmap v226: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T22:03:00.165 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-21T22:03:00.166 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-21T22:03:00.629 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:03:01.824 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:03:01 smithi151 ceph-mon[31623]: pgmap v227: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:03:01.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:03:01 smithi184 ceph-mon[36942]: pgmap v227: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:03:02.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:03:01 smithi164 ceph-mon[36269]: pgmap v227: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:03:02.920 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.919+0000 7ff4eae00700 1 -- 172.21.15.151:0/2531572615 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff4e41095b0 msgr2=0x7ff4e4109990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:02.920 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.919+0000 7ff4eae00700 1 --2- 172.21.15.151:0/2531572615 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff4e41095b0 0x7ff4e4109990 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ff4d4009990 tx=0x7ff4d4009ca0 comp rx=0 tx=0).stop 2024-05-21T22:03:02.920 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.919+0000 7ff4eae00700 1 -- 172.21.15.151:0/2531572615 shutdown_connections 2024-05-21T22:03:02.920 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.919+0000 7ff4eae00700 1 --2- 172.21.15.151:0/2531572615 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff4e41095b0 0x7ff4e4109990 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:02.920 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.919+0000 7ff4eae00700 1 --2- 172.21.15.151:0/2531572615 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff4e406bbe0 0x7ff4e41103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:02.920 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.919+0000 7ff4eae00700 1 --2- 172.21.15.151:0/2531572615 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff4e406b220 0x7ff4e406b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:02.920 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.919+0000 7ff4eae00700 1 -- 172.21.15.151:0/2531572615 >> 172.21.15.151:0/2531572615 conn(0x7ff4e40793f0 msgr2=0x7ff4e4079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:03:02.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.919+0000 7ff4eae00700 1 -- 172.21.15.151:0/2531572615 shutdown_connections 2024-05-21T22:03:02.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.919+0000 7ff4eae00700 1 -- 172.21.15.151:0/2531572615 wait complete. 2024-05-21T22:03:02.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.920+0000 7ff4eae00700 1 Processor -- start 2024-05-21T22:03:02.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.921+0000 7ff4eae00700 1 -- start start 2024-05-21T22:03:02.922 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.921+0000 7ff4eae00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff4e406b220 0x7ff4e4102510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:02.922 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.921+0000 7ff4eae00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff4e406bbe0 0x7ff4e4102a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:02.922 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.921+0000 7ff4eae00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff4e41095b0 0x7ff4e4102f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:02.922 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.921+0000 7ff4eae00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e41136e0 con 0x7ff4e41095b0 2024-05-21T22:03:02.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.921+0000 7ff4eae00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e4113560 con 0x7ff4e406b220 2024-05-21T22:03:02.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.921+0000 7ff4eae00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e4113860 con 0x7ff4e406bbe0 2024-05-21T22:03:02.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.922+0000 7ff4e3e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff4e406b220 0x7ff4e4102510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:02.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.922+0000 7ff4e3e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff4e406b220 0x7ff4e4102510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:43298/0 (socket says 172.21.15.151:43298) 2024-05-21T22:03:02.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.922+0000 7ff4e3e00700 1 -- 172.21.15.151:0/2470426196 learned_addr learned my addr 172.21.15.151:0/2470426196 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:03:02.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.922+0000 7ff4e3400700 1 --2- 172.21.15.151:0/2470426196 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff4e406bbe0 0x7ff4e4102a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:02.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.922+0000 7ff4e8e00700 1 --2- 172.21.15.151:0/2470426196 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff4e41095b0 0x7ff4e4102f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:02.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.922+0000 7ff4e3e00700 1 -- 172.21.15.151:0/2470426196 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff4e406bbe0 msgr2=0x7ff4e4102a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:02.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.922+0000 7ff4e3e00700 1 --2- 172.21.15.151:0/2470426196 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff4e406bbe0 0x7ff4e4102a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:02.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.922+0000 7ff4e3e00700 1 -- 172.21.15.151:0/2470426196 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff4e41095b0 msgr2=0x7ff4e4102f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:02.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.922+0000 7ff4e3e00700 1 --2- 172.21.15.151:0/2470426196 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff4e41095b0 0x7ff4e4102f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:02.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.922+0000 7ff4e3e00700 1 -- 172.21.15.151:0/2470426196 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4e41a8130 con 0x7ff4e406b220 2024-05-21T22:03:02.924 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.922+0000 7ff4e3e00700 1 --2- 172.21.15.151:0/2470426196 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff4e406b220 0x7ff4e4102510 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7ff4d40057c0 tx=0x7ff4d4009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:03:02.924 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.923+0000 7ff4e0c00700 1 -- 172.21.15.151:0/2470426196 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d4026070 con 0x7ff4e406b220 2024-05-21T22:03:02.924 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.923+0000 7ff4e0c00700 1 -- 172.21.15.151:0/2470426196 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4d4021410 con 0x7ff4e406b220 2024-05-21T22:03:02.924 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.923+0000 7ff4e0c00700 1 -- 172.21.15.151:0/2470426196 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d4017650 con 0x7ff4e406b220 2024-05-21T22:03:02.924 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.923+0000 7ff4eae00700 1 -- 172.21.15.151:0/2470426196 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4e41a83c0 con 0x7ff4e406b220 2024-05-21T22:03:02.924 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.923+0000 7ff4eae00700 1 -- 172.21.15.151:0/2470426196 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4e41a8850 con 0x7ff4e406b220 2024-05-21T22:03:02.928 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.928+0000 7ff4f582db80 1 Processor -- start 2024-05-21T22:03:02.928 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.928+0000 7ff4f582db80 1 -- start start 2024-05-21T22:03:02.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.928+0000 7ff4f582db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e259e3e690 0x55e259e3ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:02.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.929+0000 7ff4f582db80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e259e3efb0 0x55e259e38ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:02.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.929+0000 7ff4f582db80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e259e391c0 0x55e259e39640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:02.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.929+0000 7ff4f582db80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e259d96ec0 con 0x55e259e391c0 2024-05-21T22:03:02.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.929+0000 7ff4f582db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55e259d86a60 con 0x55e259e3e690 2024-05-21T22:03:02.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.929+0000 7ff4f582db80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55e259d8aaa0 con 0x55e259e3efb0 2024-05-21T22:03:02.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.929+0000 7ff4e8e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e259e3e690 0x55e259e3ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:02.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.929+0000 7ff4e8e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e259e3e690 0x55e259e3ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:43308/0 (socket says 172.21.15.151:43308) 2024-05-21T22:03:02.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.929+0000 7ff4e8e00700 1 -- 172.21.15.151:0/3427615350 learned_addr learned my addr 172.21.15.151:0/3427615350 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:03:02.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.929+0000 7ff4e3400700 1 --2- 172.21.15.151:0/3427615350 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e259e391c0 0x55e259e39640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:02.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.929+0000 7ff4e8e00700 1 -- 172.21.15.151:0/3427615350 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e259e3efb0 msgr2=0x55e259e38ba0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-21T22:03:02.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.929+0000 7ff4e8e00700 1 --2- 172.21.15.151:0/3427615350 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e259e3efb0 0x55e259e38ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:02.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.929+0000 7ff4e8e00700 1 -- 172.21.15.151:0/3427615350 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e259e391c0 msgr2=0x55e259e39640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:02.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.929+0000 7ff4e8e00700 1 --2- 172.21.15.151:0/3427615350 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e259e391c0 0x55e259e39640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:02.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.929+0000 7ff4e8e00700 1 -- 172.21.15.151:0/3427615350 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e259d4f4a0 con 0x55e259e3e690 2024-05-21T22:03:02.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.930+0000 7ff4e0c00700 1 -- 172.21.15.151:0/2470426196 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7ff4d4029070 con 0x7ff4e406b220 2024-05-21T22:03:02.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.930+0000 7ff4e8e00700 1 --2- 172.21.15.151:0/3427615350 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e259e3e690 0x55e259e3ea70 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc009440 tx=0x7ff4dc009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3e098a34d0ac0c server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:03:02.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.930+0000 7ff4d9600700 1 -- 172.21.15.151:0/3427615350 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4dc013750 con 0x55e259e3e690 2024-05-21T22:03:02.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.930+0000 7ff4d9600700 1 -- 172.21.15.151:0/3427615350 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4dc013da0 con 0x55e259e3e690 2024-05-21T22:03:02.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.930+0000 7ff4d9600700 1 -- 172.21.15.151:0/3427615350 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4dc0143e0 con 0x55e259e3e690 2024-05-21T22:03:02.932 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.931+0000 7ff4f582db80 1 -- 172.21.15.151:0/3427615350 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e259e3e690 msgr2=0x55e259e3ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:02.932 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.931+0000 7ff4f582db80 1 --2- 172.21.15.151:0/3427615350 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e259e3e690 0x55e259e3ea70 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc009440 tx=0x7ff4dc009920 comp rx=0 tx=0).stop 2024-05-21T22:03:02.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.931+0000 7ff4f582db80 1 -- 172.21.15.151:0/3427615350 shutdown_connections 2024-05-21T22:03:02.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.931+0000 7ff4f582db80 1 --2- 172.21.15.151:0/3427615350 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e259e3e690 0x55e259e3ea70 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:02.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.931+0000 7ff4f582db80 1 --2- 172.21.15.151:0/3427615350 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e259e391c0 0x55e259e39640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:02.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.931+0000 7ff4f582db80 1 --2- 172.21.15.151:0/3427615350 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e259e3efb0 0x55e259e38ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:02.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.931+0000 7ff4f582db80 1 -- 172.21.15.151:0/3427615350 >> 172.21.15.151:0/3427615350 conn(0x55e259d66c60 msgr2=0x55e259e33710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:03:02.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.931+0000 7ff4f582db80 1 -- 172.21.15.151:0/3427615350 shutdown_connections 2024-05-21T22:03:02.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.931+0000 7ff4f582db80 1 -- 172.21.15.151:0/3427615350 wait complete. 2024-05-21T22:03:02.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.932+0000 7ff4e0c00700 1 --2- 172.21.15.151:0/2470426196 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7ff4c80b2910 0x7ff4c80b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:02.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.932+0000 7ff4f582db80 1 -- start start 2024-05-21T22:03:02.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.932+0000 7ff4e0c00700 1 -- 172.21.15.151:0/2470426196 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8542+0+0 (secure 0 0 0) 0x7ff4d40e3460 con 0x7ff4e406b220 2024-05-21T22:03:02.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.932+0000 7ff4f582db80 1 Processor -- start 2024-05-21T22:03:02.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.932+0000 7ff4e8e00700 1 --2- 172.21.15.151:0/2470426196 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7ff4c80b2910 0x7ff4c80b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:02.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.933+0000 7ff4f582db80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e259e391c0 0x55e259e44ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:02.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.933+0000 7ff4f582db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e259e3e690 0x55e259e420a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:02.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.933+0000 7ff4f582db80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e259e3efb0 0x55e259e425e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:02.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.933+0000 7ff4f582db80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e259c8d020 con 0x55e259e391c0 2024-05-21T22:03:02.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.933+0000 7ff4f582db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55e259d277e0 con 0x55e259e3e690 2024-05-21T22:03:02.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.933+0000 7ff4f582db80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55e259e45a60 con 0x55e259e3efb0 2024-05-21T22:03:02.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.933+0000 7ff4f582db80 10 client.0 _ll_register_callbacks cb 0x55e259d30cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:03:02.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.933+0000 7ff4e3400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e259e3e690 0x55e259e420a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:02.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.933+0000 7ff4e3e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e259e391c0 0x55e259e44ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:02.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.933+0000 7ff4e3400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e259e3e690 0x55e259e420a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:43324/0 (socket says 172.21.15.151:43324) 2024-05-21T22:03:02.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.933+0000 7ff4e3400700 1 -- 172.21.15.151:0/1878984409 learned_addr learned my addr 172.21.15.151:0/1878984409 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:03:02.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.933+0000 7ff4e8e00700 1 --2- 172.21.15.151:0/1878984409 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e259e3efb0 0x55e259e425e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:02.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.934+0000 7ff4e3400700 1 -- 172.21.15.151:0/1878984409 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e259e3efb0 msgr2=0x55e259e425e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:02.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.934+0000 7ff4e3400700 1 --2- 172.21.15.151:0/1878984409 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e259e3efb0 0x55e259e425e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:02.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.934+0000 7ff4e3400700 1 -- 172.21.15.151:0/1878984409 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e259e391c0 msgr2=0x55e259e44ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:02.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.934+0000 7ff4e3400700 1 --2- 172.21.15.151:0/1878984409 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e259e391c0 0x55e259e44ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:02.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.934+0000 7ff4e3400700 1 -- 172.21.15.151:0/1878984409 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e259cd6950 con 0x55e259e3e690 2024-05-21T22:03:02.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.934+0000 7ff4e8e00700 1 --2- 172.21.15.151:0/1878984409 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e259e3efb0 0x55e259e425e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T22:03:02.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.934+0000 7ff4e3400700 1 --2- 172.21.15.151:0/1878984409 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e259e3e690 0x55e259e420a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7ff4d0011440 tx=0x7ff4d0011750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19e4ca498fa671f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:03:02.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.934+0000 7ff4ce000700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-21T22:03:02.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.934+0000 7ff4e8e00700 1 --2- 172.21.15.151:0/2470426196 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7ff4c80b2910 0x7ff4c80b4dd0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc013f90 tx=0x7ff4dc0159a0 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:03:02.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.934+0000 7ff4ce000700 1 -- 172.21.15.151:0/1878984409 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d0004030 con 0x55e259e3e690 2024-05-21T22:03:02.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.934+0000 7ff4ce000700 1 -- 172.21.15.151:0/1878984409 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8542+0+0 (secure 0 0 0) 0x7ff4d0011e00 con 0x55e259e3e690 2024-05-21T22:03:02.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.934+0000 7ff4f582db80 1 -- 172.21.15.151:0/1878984409 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4dc013da0 con 0x55e259e3e690 2024-05-21T22:03:02.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.935+0000 7ff4ce000700 1 -- 172.21.15.151:0/1878984409 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff4d00041d0 con 0x55e259e3e690 2024-05-21T22:03:02.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.935+0000 7ff4f582db80 10 client.27002 fetch_fsmap learned FSMap version 29 2024-05-21T22:03:02.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.935+0000 7ff4f582db80 1 -- 172.21.15.151:0/1878984409 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e259d570e0 con 0x55e259e3e690 2024-05-21T22:03:02.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.935+0000 7ff4ce000700 1 -- 172.21.15.151:0/1878984409 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4dc013da0 con 0x55e259e3e690 2024-05-21T22:03:02.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.935+0000 7ff4ce000700 1 -- 172.21.15.151:0/1878984409 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7ff4d00036f0 con 0x55e259e3e690 2024-05-21T22:03:02.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.935+0000 7ff4f582db80 10 client.27002 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:03:02.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.935+0000 7ff4f582db80 10 client.27002 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T22:03:02.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.935+0000 7ff4f582db80 20 client.27002 populate_metadata read hostname 'smithi151' 2024-05-21T22:03:02.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.935+0000 7ff4f582db80 1 --2- 172.21.15.151:0/1878984409 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55e259e52600 0x55e259e549f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:02.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.936+0000 7ff4f582db80 4 client.27002 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T22:03:02.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.936+0000 7ff4f582db80 1 -- 172.21.15.151:0/1878984409 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e259d98e10 con 0x55e259e52600 2024-05-21T22:03:02.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.936+0000 7ff4e3e00700 1 --2- 172.21.15.151:0/1878984409 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55e259e52600 0x55e259e549f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:02.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.936+0000 7ff4e3e00700 1 --2- 172.21.15.151:0/1878984409 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55e259e52600 0x55e259e549f0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=deb1ef9fed11992b server_cookie=f63f7a3629e705fb in_seq=0 out_seq=0 2024-05-21T22:03:02.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.936+0000 7ff4ce000700 10 client.27002 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T22:03:02.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.937+0000 7ff4ce000700 1 -- 172.21.15.151:0/1878984409 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55e259d98e10 con 0x55e259e52600 2024-05-21T22:03:02.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.937+0000 7ff4ce000700 10 client.27002 handle_command_reply: tid=0 2024-05-21T22:03:02.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.937+0000 7ff4f582db80 1 client.27002 shutdown 2024-05-21T22:03:02.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.937+0000 7ff4f582db80 1 -- 172.21.15.151:0/1878984409 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e259e3e690 msgr2=0x55e259e420a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:02.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.937+0000 7ff4f582db80 1 --2- 172.21.15.151:0/1878984409 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e259e3e690 0x55e259e420a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7ff4d0011440 tx=0x7ff4d0011750 comp rx=0 tx=0).stop 2024-05-21T22:03:02.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.937+0000 7ff4f582db80 1 -- 172.21.15.151:0/1878984409 shutdown_connections 2024-05-21T22:03:02.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.937+0000 7ff4f582db80 1 --2- 172.21.15.151:0/1878984409 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e259e3efb0 0x55e259e425e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:02.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.937+0000 7ff4f582db80 1 --2- 172.21.15.151:0/1878984409 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e259e391c0 0x55e259e44ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:02.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.937+0000 7ff4f582db80 1 --2- 172.21.15.151:0/1878984409 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55e259e52600 0x55e259e549f0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:02.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.937+0000 7ff4f582db80 1 --2- 172.21.15.151:0/1878984409 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e259e3e690 0x55e259e420a0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:02.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.937+0000 7ff4f582db80 1 -- 172.21.15.151:0/1878984409 >> 172.21.15.151:0/1878984409 conn(0x55e259d66c60 msgr2=0x55e259e32df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:03:02.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.937+0000 7ff4ce000700 0 client.27002 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T22:03:02.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.938+0000 7ff4f582db80 1 -- 172.21.15.151:0/1878984409 shutdown_connections 2024-05-21T22:03:02.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.938+0000 7ff4f582db80 1 -- 172.21.15.151:0/1878984409 wait complete. 2024-05-21T22:03:02.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.938+0000 7ff4f582db80 20 client.27002 trim_cache size 0 max 16384 2024-05-21T22:03:02.961 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.961+0000 7ff4f582db80 1 Processor -- start 2024-05-21T22:03:02.961 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.961+0000 7ff4f582db80 1 -- start start 2024-05-21T22:03:02.961 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.961+0000 7ff4f582db80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e259e391c0 0x55e259e3fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:02.961 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.961+0000 7ff4f582db80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e259e3e690 0x55e259e33570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:02.962 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.961+0000 7ff4f582db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e259e3efb0 0x55e259e5a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:02.962 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.961+0000 7ff4f582db80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e41b0080 con 0x55e259e391c0 2024-05-21T22:03:02.962 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.961+0000 7ff4f582db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e41b1bc0 con 0x55e259e3efb0 2024-05-21T22:03:02.962 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.961+0000 7ff4f582db80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e41b0750 con 0x55e259e3e690 2024-05-21T22:03:02.963 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.961+0000 7ff4e3400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e259e3e690 0x55e259e33570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:02.963 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.961+0000 7ff4e8e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e259e3efb0 0x55e259e5a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:02.963 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.961+0000 7ff4e3400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e259e3e690 0x55e259e33570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:53336/0 (socket says 172.21.15.151:53336) 2024-05-21T22:03:02.963 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.961+0000 7ff4e3e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e259e391c0 0x55e259e3fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:02.963 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.961+0000 7ff4e3400700 1 -- 172.21.15.151:0/442029165 learned_addr learned my addr 172.21.15.151:0/442029165 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:03:02.963 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.962+0000 7ff4e3400700 1 -- 172.21.15.151:0/442029165 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e259e3efb0 msgr2=0x55e259e5a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:02.963 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.962+0000 7ff4e3400700 1 --2- 172.21.15.151:0/442029165 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e259e3efb0 0x55e259e5a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:02.963 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.962+0000 7ff4e3400700 1 -- 172.21.15.151:0/442029165 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e259e391c0 msgr2=0x55e259e3fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:02.963 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.962+0000 7ff4e3400700 1 --2- 172.21.15.151:0/442029165 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e259e391c0 0x55e259e3fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:02.963 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.962+0000 7ff4e3400700 1 -- 172.21.15.151:0/442029165 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e259e40640 con 0x55e259e3e690 2024-05-21T22:03:02.963 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.962+0000 7ff4e3400700 1 --2- 172.21.15.151:0/442029165 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e259e3e690 0x55e259e33570 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ff4d0007cd0 tx=0x7ff4d0011bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dde35ca67c256fb8 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:03:02.963 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.962+0000 7ff4d8c00700 1 -- 172.21.15.151:0/442029165 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d0032070 con 0x55e259e3e690 2024-05-21T22:03:02.963 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.963+0000 7ff4d8c00700 1 -- 172.21.15.151:0/442029165 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4d0029040 con 0x55e259e3e690 2024-05-21T22:03:02.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.963+0000 7ff4d8c00700 1 -- 172.21.15.151:0/442029165 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d0028070 con 0x55e259e3e690 2024-05-21T22:03:02.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.963+0000 7ff4f582db80 1 -- 172.21.15.151:0/442029165 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e259e3e690 msgr2=0x55e259e33570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:02.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.963+0000 7ff4f582db80 1 --2- 172.21.15.151:0/442029165 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e259e3e690 0x55e259e33570 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ff4d0007cd0 tx=0x7ff4d0011bd0 comp rx=0 tx=0).stop 2024-05-21T22:03:02.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.963+0000 7ff4f582db80 1 -- 172.21.15.151:0/442029165 shutdown_connections 2024-05-21T22:03:02.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.963+0000 7ff4f582db80 1 --2- 172.21.15.151:0/442029165 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e259e3efb0 0x55e259e5a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:02.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.963+0000 7ff4f582db80 1 --2- 172.21.15.151:0/442029165 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e259e391c0 0x55e259e3fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:02.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.963+0000 7ff4f582db80 1 --2- 172.21.15.151:0/442029165 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e259e3e690 0x55e259e33570 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:02.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.963+0000 7ff4f582db80 1 -- 172.21.15.151:0/442029165 >> 172.21.15.151:0/442029165 conn(0x55e259d66c60 msgr2=0x55e259e3f3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:03:02.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.964+0000 7ff4f582db80 1 -- 172.21.15.151:0/442029165 shutdown_connections 2024-05-21T22:03:02.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.964+0000 7ff4f582db80 1 -- 172.21.15.151:0/442029165 wait complete. 2024-05-21T22:03:02.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.964+0000 7ff4f582db80 1 -- start start 2024-05-21T22:03:02.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.964+0000 7ff4f582db80 1 Processor -- start 2024-05-21T22:03:02.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.965+0000 7ff4f582db80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e259e391c0 0x55e259e4b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:02.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.965+0000 7ff4f582db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e259e3e690 0x55e259e47cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:02.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.965+0000 7ff4f582db80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e259e3efb0 0x55e259e48230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:02.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.965+0000 7ff4f582db80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e41b14f0 con 0x55e259e3efb0 2024-05-21T22:03:02.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.965+0000 7ff4f582db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e41b0e20 con 0x55e259e3e690 2024-05-21T22:03:02.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.965+0000 7ff4f582db80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e41ae040 con 0x55e259e391c0 2024-05-21T22:03:02.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.965+0000 7ff4f582db80 10 client.0 _ll_register_callbacks cb 0x55e259d30cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:03:02.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.965+0000 7ff4e3e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e259e391c0 0x55e259e4b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:02.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.965+0000 7ff4e3400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e259e3e690 0x55e259e47cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:02.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.965+0000 7ff4e3e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e259e391c0 0x55e259e4b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:53346/0 (socket says 172.21.15.151:53346) 2024-05-21T22:03:02.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.965+0000 7ff4e3e00700 1 -- 172.21.15.151:0/2422825116 learned_addr learned my addr 172.21.15.151:0/2422825116 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:03:02.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.965+0000 7ff4e8e00700 1 --2- 172.21.15.151:0/2422825116 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e259e3efb0 0x55e259e48230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:02.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.965+0000 7ff4e3e00700 1 -- 172.21.15.151:0/2422825116 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e259e3e690 msgr2=0x55e259e47cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:02.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.965+0000 7ff4e3e00700 1 --2- 172.21.15.151:0/2422825116 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e259e3e690 0x55e259e47cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:02.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.965+0000 7ff4e3e00700 1 -- 172.21.15.151:0/2422825116 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e259e3efb0 msgr2=0x55e259e48230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:02.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.965+0000 7ff4e3e00700 1 --2- 172.21.15.151:0/2422825116 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e259e3efb0 0x55e259e48230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:02.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.965+0000 7ff4e3e00700 1 -- 172.21.15.151:0/2422825116 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e259e48770 con 0x55e259e391c0 2024-05-21T22:03:02.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.966+0000 7ff4e3e00700 1 --2- 172.21.15.151:0/2422825116 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e259e391c0 0x55e259e4b100 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ff4d40a60f0 tx=0x7ff4d4009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2285afc5e397864c server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:03:02.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.966+0000 7ff4ce000700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-21T22:03:02.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.966+0000 7ff4ce000700 1 -- 172.21.15.151:0/2422825116 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d40aa7f0 con 0x55e259e391c0 2024-05-21T22:03:02.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.966+0000 7ff4ce000700 1 -- 172.21.15.151:0/2422825116 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8542+0+0 (secure 0 0 0) 0x7ff4d40218c0 con 0x55e259e391c0 2024-05-21T22:03:02.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.966+0000 7ff4f582db80 1 -- 172.21.15.151:0/2422825116 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4d0029040 con 0x55e259e391c0 2024-05-21T22:03:02.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.966+0000 7ff4ce000700 1 -- 172.21.15.151:0/2422825116 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff4d40aeba0 con 0x55e259e391c0 2024-05-21T22:03:02.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.966+0000 7ff4f582db80 10 client.27013 fetch_fsmap learned FSMap version 29 2024-05-21T22:03:02.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.966+0000 7ff4f582db80 1 -- 172.21.15.151:0/2422825116 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e259ebdf30 con 0x55e259e391c0 2024-05-21T22:03:02.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.966+0000 7ff4ce000700 1 -- 172.21.15.151:0/2422825116 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4d0029040 con 0x55e259e391c0 2024-05-21T22:03:02.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.967+0000 7ff4ce000700 1 -- 172.21.15.151:0/2422825116 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7ff4d40b3130 con 0x55e259e391c0 2024-05-21T22:03:02.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.967+0000 7ff4f582db80 10 client.27013 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:03:02.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.967+0000 7ff4f582db80 10 client.27013 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T22:03:02.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.967+0000 7ff4f582db80 20 client.27013 populate_metadata read hostname 'smithi151' 2024-05-21T22:03:02.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.967+0000 7ff4f582db80 1 --2- 172.21.15.151:0/2422825116 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55e259ebe550 0x55e259ec0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:02.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.967+0000 7ff4f582db80 4 client.27013 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-21T22:03:02.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.967+0000 7ff4f582db80 1 -- 172.21.15.151:0/2422825116 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e259d925f0 con 0x55e259ebe550 2024-05-21T22:03:02.970 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.967+0000 7ff4e3400700 1 --2- 172.21.15.151:0/2422825116 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55e259ebe550 0x55e259ec0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:02.970 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.968+0000 7ff4e3400700 1 --2- 172.21.15.151:0/2422825116 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55e259ebe550 0x55e259ec0a10 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83888f8e66f17b82 server_cookie=3ce3e3d037744d87 in_seq=0 out_seq=0 2024-05-21T22:03:02.970 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.968+0000 7ff4ce000700 10 client.27013 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T22:03:02.970 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.968+0000 7ff4ce000700 1 -- 172.21.15.151:0/2422825116 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e259d925f0 con 0x55e259ebe550 2024-05-21T22:03:02.971 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.968+0000 7ff4ce000700 10 client.27013 handle_command_reply: tid=0 2024-05-21T22:03:02.971 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.969+0000 7ff4f582db80 1 client.27013 shutdown 2024-05-21T22:03:02.971 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.969+0000 7ff4f582db80 1 -- 172.21.15.151:0/2422825116 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e259e391c0 msgr2=0x55e259e4b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:02.971 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.969+0000 7ff4f582db80 1 --2- 172.21.15.151:0/2422825116 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e259e391c0 0x55e259e4b100 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ff4d40a60f0 tx=0x7ff4d4009ca0 comp rx=0 tx=0).stop 2024-05-21T22:03:02.971 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.969+0000 7ff4f582db80 1 -- 172.21.15.151:0/2422825116 shutdown_connections 2024-05-21T22:03:02.971 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.969+0000 7ff4f582db80 1 --2- 172.21.15.151:0/2422825116 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e259e3efb0 0x55e259e48230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:02.971 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.969+0000 7ff4f582db80 1 --2- 172.21.15.151:0/2422825116 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e259e391c0 0x55e259e4b100 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:02.971 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.969+0000 7ff4f582db80 1 --2- 172.21.15.151:0/2422825116 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55e259ebe550 0x55e259ec0a10 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:02.971 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.969+0000 7ff4f582db80 1 --2- 172.21.15.151:0/2422825116 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e259e3e690 0x55e259e47cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:02.971 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.969+0000 7ff4f582db80 1 -- 172.21.15.151:0/2422825116 >> 172.21.15.151:0/2422825116 conn(0x55e259d66c60 msgr2=0x55e259e3d1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:03:02.971 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.969+0000 7ff4ce000700 0 client.27013 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T22:03:02.971 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.969+0000 7ff4f582db80 1 -- 172.21.15.151:0/2422825116 shutdown_connections 2024-05-21T22:03:02.971 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.969+0000 7ff4f582db80 1 -- 172.21.15.151:0/2422825116 wait complete. 2024-05-21T22:03:02.971 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.969+0000 7ff4f582db80 20 client.27013 trim_cache size 0 max 16384 2024-05-21T22:03:02.972 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-05-21T22:03:02.972 INFO:teuthology.orchestra.run.smithi151.stdout: "return_code": 0, 2024-05-21T22:03:02.972 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_tag": "43fb6632-b3c3-46ca-9923-47080c11544f", 2024-05-21T22:03:02.972 INFO:teuthology.orchestra.run.smithi151.stdout: "mode": "asynchronous" 2024-05-21T22:03:02.972 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-05-21T22:03:02.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.970+0000 7ff4eae00700 1 -- 172.21.15.151:0/2470426196 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7ff4c80b2910 msgr2=0x7ff4c80b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:02.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.970+0000 7ff4eae00700 1 --2- 172.21.15.151:0/2470426196 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7ff4c80b2910 0x7ff4c80b4dd0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc013f90 tx=0x7ff4dc0159a0 comp rx=0 tx=0).stop 2024-05-21T22:03:02.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.970+0000 7ff4eae00700 1 -- 172.21.15.151:0/2470426196 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff4e406b220 msgr2=0x7ff4e4102510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:02.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.970+0000 7ff4eae00700 1 --2- 172.21.15.151:0/2470426196 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff4e406b220 0x7ff4e4102510 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7ff4d40057c0 tx=0x7ff4d4009580 comp rx=0 tx=0).stop 2024-05-21T22:03:02.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.970+0000 7ff4eae00700 1 -- 172.21.15.151:0/2470426196 shutdown_connections 2024-05-21T22:03:02.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.970+0000 7ff4eae00700 1 --2- 172.21.15.151:0/2470426196 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7ff4c80b2910 0x7ff4c80b4dd0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:02.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.970+0000 7ff4eae00700 1 --2- 172.21.15.151:0/2470426196 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff4e406b220 0x7ff4e4102510 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:02.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.970+0000 7ff4eae00700 1 --2- 172.21.15.151:0/2470426196 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff4e41095b0 0x7ff4e4102f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:02.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.970+0000 7ff4eae00700 1 --2- 172.21.15.151:0/2470426196 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff4e406bbe0 0x7ff4e4102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:02.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.970+0000 7ff4eae00700 1 -- 172.21.15.151:0/2470426196 >> 172.21.15.151:0/2470426196 conn(0x7ff4e40793f0 msgr2=0x7ff4e4101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:03:02.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.970+0000 7ff4eae00700 1 -- 172.21.15.151:0/2470426196 shutdown_connections 2024-05-21T22:03:02.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:02.971+0000 7ff4eae00700 1 -- 172.21.15.151:0/2470426196 wait complete. 2024-05-21T22:03:03.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:03:03 smithi151 ceph-mon[31623]: pgmap v228: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-05-21T22:03:03.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:03:03 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T22:03:03.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:03:03 smithi151 ceph-mon[31623]: scrub queued for path: / 2024-05-21T22:03:03.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:03:03 smithi151 ceph-mon[31623]: scrub summary: idle+waiting paths [/] 2024-05-21T22:03:03.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:03:03 smithi151 ceph-mon[31623]: scrub summary: active paths [/] 2024-05-21T22:03:03.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:03:03 smithi151 ceph-mon[31623]: scrub summary: idle+waiting paths [/] 2024-05-21T22:03:03.791 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-21T22:03:03.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:03:03 smithi184 ceph-mon[36942]: pgmap v228: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-05-21T22:03:03.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:03:03 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T22:03:03.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:03:03 smithi184 ceph-mon[36942]: scrub queued for path: / 2024-05-21T22:03:03.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:03:03 smithi184 ceph-mon[36942]: scrub summary: idle+waiting paths [/] 2024-05-21T22:03:03.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:03:03 smithi184 ceph-mon[36942]: scrub summary: active paths [/] 2024-05-21T22:03:03.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:03:03 smithi184 ceph-mon[36942]: scrub summary: idle+waiting paths [/] 2024-05-21T22:03:04.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:03:03 smithi164 ceph-mon[36269]: pgmap v228: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-05-21T22:03:04.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:03:03 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T22:03:04.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:03:03 smithi164 ceph-mon[36269]: scrub queued for path: / 2024-05-21T22:03:04.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:03:03 smithi164 ceph-mon[36269]: scrub summary: idle+waiting paths [/] 2024-05-21T22:03:04.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:03:03 smithi164 ceph-mon[36269]: scrub summary: active paths [/] 2024-05-21T22:03:04.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:03:03 smithi164 ceph-mon[36269]: scrub summary: idle+waiting paths [/] 2024-05-21T22:03:04.218 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:03:04.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:03:04 smithi151 ceph-mon[31623]: scrub summary: idle 2024-05-21T22:03:04.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:03:04 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T22:03:04.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:03:04 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T22:03:04.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:03:04 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T22:03:04.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:03:04 smithi184 ceph-mon[36942]: scrub summary: idle 2024-05-21T22:03:04.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:03:04 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T22:03:04.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:03:04 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T22:03:04.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:03:04 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T22:03:05.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:03:04 smithi164 ceph-mon[36269]: scrub summary: idle 2024-05-21T22:03:05.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:03:04 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T22:03:05.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:03:04 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T22:03:05.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:03:04 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T22:03:05.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:03:05 smithi151 ceph-mon[31623]: pgmap v229: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-05-21T22:03:05.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:03:05 smithi184 ceph-mon[36942]: pgmap v229: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-05-21T22:03:06.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:03:05 smithi164 ceph-mon[36269]: pgmap v229: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-05-21T22:03:06.662 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.661+0000 7f4b34000700 1 -- 172.21.15.151:0/2721272179 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4b2c076090 msgr2=0x7f4b2c10fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:06.662 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.661+0000 7f4b34000700 1 --2- 172.21.15.151:0/2721272179 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4b2c076090 0x7f4b2c10fa30 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f4b18009990 tx=0x7f4b18009ca0 comp rx=0 tx=0).stop 2024-05-21T22:03:06.662 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.661+0000 7f4b34000700 1 -- 172.21.15.151:0/2721272179 shutdown_connections 2024-05-21T22:03:06.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.661+0000 7f4b34000700 1 --2- 172.21.15.151:0/2721272179 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4b2c075770 0x7f4b2c075b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:06.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.661+0000 7f4b34000700 1 --2- 172.21.15.151:0/2721272179 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4b2c10ff70 0x7f4b2c112360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:06.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.661+0000 7f4b34000700 1 --2- 172.21.15.151:0/2721272179 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4b2c076090 0x7f4b2c10fa30 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:06.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.661+0000 7f4b34000700 1 -- 172.21.15.151:0/2721272179 >> 172.21.15.151:0/2721272179 conn(0x7f4b2c07a6f0 msgr2=0x7f4b2c07ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:03:06.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.662+0000 7f4b34000700 1 -- 172.21.15.151:0/2721272179 shutdown_connections 2024-05-21T22:03:06.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.662+0000 7f4b34000700 1 -- 172.21.15.151:0/2721272179 wait complete. 2024-05-21T22:03:06.664 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.663+0000 7f4b34000700 1 Processor -- start 2024-05-21T22:03:06.664 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.663+0000 7f4b34000700 1 -- start start 2024-05-21T22:03:06.664 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.663+0000 7f4b34000700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4b2c075770 0x7f4b2c19be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:06.664 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.663+0000 7f4b34000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4b2c076090 0x7f4b2c19c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:06.664 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.663+0000 7f4b34000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4b2c10ff70 0x7f4b2c1a0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:06.664 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.663+0000 7f4b34000700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4b2c115990 con 0x7f4b2c076090 2024-05-21T22:03:06.664 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.663+0000 7f4b34000700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4b2c115810 con 0x7f4b2c10ff70 2024-05-21T22:03:06.664 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.663+0000 7f4b34000700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4b2c115b10 con 0x7f4b2c075770 2024-05-21T22:03:06.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.664+0000 7f4b30c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4b2c076090 0x7f4b2c19c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:06.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.664+0000 7f4b30c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4b2c076090 0x7f4b2c19c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:59488/0 (socket says 172.21.15.151:59488) 2024-05-21T22:03:06.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.664+0000 7f4b30c00700 1 -- 172.21.15.151:0/2276425443 learned_addr learned my addr 172.21.15.151:0/2276425443 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:03:06.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.664+0000 7f4b32000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4b2c10ff70 0x7f4b2c1a0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:06.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.664+0000 7f4b31600700 1 --2- 172.21.15.151:0/2276425443 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4b2c075770 0x7f4b2c19be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:06.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.664+0000 7f4b32000700 1 -- 172.21.15.151:0/2276425443 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4b2c075770 msgr2=0x7f4b2c19be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:06.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.664+0000 7f4b32000700 1 --2- 172.21.15.151:0/2276425443 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4b2c075770 0x7f4b2c19be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:06.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.664+0000 7f4b32000700 1 -- 172.21.15.151:0/2276425443 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4b2c076090 msgr2=0x7f4b2c19c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:06.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.664+0000 7f4b32000700 1 --2- 172.21.15.151:0/2276425443 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4b2c076090 0x7f4b2c19c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:06.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.664+0000 7f4b32000700 1 -- 172.21.15.151:0/2276425443 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b2c1a0e90 con 0x7f4b2c10ff70 2024-05-21T22:03:06.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.665+0000 7f4b32000700 1 --2- 172.21.15.151:0/2276425443 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4b2c10ff70 0x7f4b2c1a0750 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f4b2400ef90 tx=0x7f4b2400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:03:06.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.665+0000 7f4b2a000700 1 -- 172.21.15.151:0/2276425443 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b240092e0 con 0x7f4b2c10ff70 2024-05-21T22:03:06.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.665+0000 7f4b2a000700 1 -- 172.21.15.151:0/2276425443 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b24004bb0 con 0x7f4b2c10ff70 2024-05-21T22:03:06.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.665+0000 7f4b2a000700 1 -- 172.21.15.151:0/2276425443 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b24004020 con 0x7f4b2c10ff70 2024-05-21T22:03:06.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.665+0000 7f4b34000700 1 -- 172.21.15.151:0/2276425443 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b2c1a1120 con 0x7f4b2c10ff70 2024-05-21T22:03:06.667 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.665+0000 7f4b34000700 1 -- 172.21.15.151:0/2276425443 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b2c1a8a00 con 0x7f4b2c10ff70 2024-05-21T22:03:06.668 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.668+0000 7f4b2a000700 1 -- 172.21.15.151:0/2276425443 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7f4b2401b030 con 0x7f4b2c10ff70 2024-05-21T22:03:06.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.670+0000 7f4b3ea8db80 1 Processor -- start 2024-05-21T22:03:06.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.670+0000 7f4b3ea8db80 1 -- start start 2024-05-21T22:03:06.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.670+0000 7f4b2a000700 1 --2- 172.21.15.151:0/2276425443 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f4b140b2a70 0x7f4b140b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:06.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.670+0000 7f4b3ea8db80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560803f8ceb0 0x560803f8d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:06.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.670+0000 7f4b3ea8db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560803f8d7d0 0x560803f87260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:06.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.670+0000 7f4b3ea8db80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560803f87880 0x560803f87ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:06.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.670+0000 7f4b3ea8db80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x560803ee5e40 con 0x560803f8ceb0 2024-05-21T22:03:06.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.670+0000 7f4b3ea8db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x560803ed59e0 con 0x560803f8d7d0 2024-05-21T22:03:06.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.670+0000 7f4b3ea8db80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x560803ed9a20 con 0x560803f87880 2024-05-21T22:03:06.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.670+0000 7f4b2a000700 1 -- 172.21.15.151:0/2276425443 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8542+0+0 (secure 0 0 0) 0x7f4b240e2150 con 0x7f4b2c10ff70 2024-05-21T22:03:06.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.670+0000 7f4b30c00700 1 --2- 172.21.15.151:0/2276425443 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f4b140b2a70 0x7f4b140b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:06.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.671+0000 7f4b31600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560803f8ceb0 0x560803f8d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:06.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.671+0000 7f4b30c00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560803f8d7d0 0x560803f87260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:06.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.671+0000 7f4b31600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560803f8ceb0 0x560803f8d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:59494/0 (socket says 172.21.15.151:59494) 2024-05-21T22:03:06.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.671+0000 7f4b30c00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560803f8d7d0 0x560803f87260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:52224/0 (socket says 172.21.15.151:52224) 2024-05-21T22:03:06.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.671+0000 7f4b31600700 1 -- 172.21.15.151:0/2344704819 learned_addr learned my addr 172.21.15.151:0/2344704819 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:03:06.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.671+0000 7f4b32000700 1 --2- 172.21.15.151:0/2344704819 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560803f87880 0x560803f87ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:06.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.671+0000 7f4b30c00700 1 -- 172.21.15.151:0/2344704819 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560803f87880 msgr2=0x560803f87ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:06.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.671+0000 7f4b30c00700 1 --2- 172.21.15.151:0/2344704819 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560803f87880 0x560803f87ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:06.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.671+0000 7f4b30c00700 1 -- 172.21.15.151:0/2344704819 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560803f8ceb0 msgr2=0x560803f8d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:06.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.671+0000 7f4b30c00700 1 --2- 172.21.15.151:0/2344704819 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560803f8ceb0 0x560803f8d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:06.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.671+0000 7f4b30c00700 1 -- 172.21.15.151:0/2344704819 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560803e9e420 con 0x560803f8d7d0 2024-05-21T22:03:06.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.671+0000 7f4b32000700 1 --2- 172.21.15.151:0/2344704819 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560803f87880 0x560803f87ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:03:06.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.672+0000 7f4b30c00700 1 --2- 172.21.15.151:0/2276425443 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f4b140b2a70 0x7f4b140b4f30 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c111c90 tx=0x7f4b18009550 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:03:06.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.672+0000 7f4b30c00700 1 --2- 172.21.15.151:0/2344704819 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560803f8d7d0 0x560803f87260 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f4b18000c00 tx=0x7f4b18028000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2972240b63a924b server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:03:06.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.672+0000 7f4b1ea00700 1 -- 172.21.15.151:0/2344704819 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b1802e070 con 0x560803f8d7d0 2024-05-21T22:03:06.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.672+0000 7f4b1ea00700 1 -- 172.21.15.151:0/2344704819 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b18022040 con 0x560803f8d7d0 2024-05-21T22:03:06.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.672+0000 7f4b1ea00700 1 -- 172.21.15.151:0/2344704819 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b1800f960 con 0x560803f8d7d0 2024-05-21T22:03:06.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.673+0000 7f4b3ea8db80 1 -- 172.21.15.151:0/2344704819 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560803f8d7d0 msgr2=0x560803f87260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:06.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.673+0000 7f4b3ea8db80 1 --2- 172.21.15.151:0/2344704819 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560803f8d7d0 0x560803f87260 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f4b18000c00 tx=0x7f4b18028000 comp rx=0 tx=0).stop 2024-05-21T22:03:06.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.673+0000 7f4b3ea8db80 1 -- 172.21.15.151:0/2344704819 shutdown_connections 2024-05-21T22:03:06.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.673+0000 7f4b3ea8db80 1 --2- 172.21.15.151:0/2344704819 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560803f87880 0x560803f87ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:06.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.673+0000 7f4b3ea8db80 1 --2- 172.21.15.151:0/2344704819 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560803f8ceb0 0x560803f8d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:06.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.673+0000 7f4b3ea8db80 1 --2- 172.21.15.151:0/2344704819 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560803f8d7d0 0x560803f87260 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:06.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.673+0000 7f4b3ea8db80 1 -- 172.21.15.151:0/2344704819 >> 172.21.15.151:0/2344704819 conn(0x560803eb5be0 msgr2=0x560803f82010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:03:06.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.673+0000 7f4b3ea8db80 1 -- 172.21.15.151:0/2344704819 shutdown_connections 2024-05-21T22:03:06.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.673+0000 7f4b3ea8db80 1 -- 172.21.15.151:0/2344704819 wait complete. 2024-05-21T22:03:06.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.674+0000 7f4b3ea8db80 1 -- start start 2024-05-21T22:03:06.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.674+0000 7f4b3ea8db80 1 Processor -- start 2024-05-21T22:03:06.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.674+0000 7f4b3ea8db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560803f87880 0x560803f934d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:06.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.674+0000 7f4b3ea8db80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560803f8ceb0 0x560803f908c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:06.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.674+0000 7f4b3ea8db80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560803f8d7d0 0x560803f90e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:06.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.674+0000 7f4b3ea8db80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x560803ddbfa0 con 0x560803f8ceb0 2024-05-21T22:03:06.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.674+0000 7f4b3ea8db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x560803e76760 con 0x560803f87880 2024-05-21T22:03:06.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.674+0000 7f4b3ea8db80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x560803f941e0 con 0x560803f8d7d0 2024-05-21T22:03:06.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.674+0000 7f4b3ea8db80 10 client.0 _ll_register_callbacks cb 0x560803e7fc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:03:06.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.675+0000 7f4b30c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560803f8ceb0 0x560803f908c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:06.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.675+0000 7f4b30c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560803f8ceb0 0x560803f908c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:59504/0 (socket says 172.21.15.151:59504) 2024-05-21T22:03:06.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.675+0000 7f4b30c00700 1 -- 172.21.15.151:0/2031360832 learned_addr learned my addr 172.21.15.151:0/2031360832 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:03:06.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.675+0000 7f4b31600700 1 --2- 172.21.15.151:0/2031360832 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560803f87880 0x560803f934d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:06.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.675+0000 7f4b32000700 1 --2- 172.21.15.151:0/2031360832 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560803f8d7d0 0x560803f90e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:06.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.675+0000 7f4b30c00700 1 -- 172.21.15.151:0/2031360832 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560803f8d7d0 msgr2=0x560803f90e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:06.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.675+0000 7f4b30c00700 1 --2- 172.21.15.151:0/2031360832 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560803f8d7d0 0x560803f90e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:06.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.675+0000 7f4b30c00700 1 -- 172.21.15.151:0/2031360832 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560803f87880 msgr2=0x560803f934d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:06.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.675+0000 7f4b30c00700 1 --2- 172.21.15.151:0/2031360832 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560803f87880 0x560803f934d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:06.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.675+0000 7f4b30c00700 1 -- 172.21.15.151:0/2031360832 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560803e258d0 con 0x560803f8ceb0 2024-05-21T22:03:06.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.675+0000 7f4b31600700 1 --2- 172.21.15.151:0/2031360832 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560803f87880 0x560803f934d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:03:06.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.675+0000 7f4b30c00700 1 --2- 172.21.15.151:0/2031360832 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560803f8ceb0 0x560803f908c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f4b180058c0 tx=0x7f4b1800f810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d02b934c7910e53 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:03:06.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.675+0000 7f4b13400700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-21T22:03:06.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.676+0000 7f4b13400700 1 -- 172.21.15.151:0/2031360832 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b1802e070 con 0x560803f8ceb0 2024-05-21T22:03:06.679 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.676+0000 7f4b3ea8db80 1 -- 172.21.15.151:0/2031360832 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b18022040 con 0x560803f8ceb0 2024-05-21T22:03:06.679 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.676+0000 7f4b13400700 1 -- 172.21.15.151:0/2031360832 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8542+0+0 (secure 0 0 0) 0x7f4b18003890 con 0x560803f8ceb0 2024-05-21T22:03:06.679 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.676+0000 7f4b13400700 1 -- 172.21.15.151:0/2031360832 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4b18005040 con 0x560803f8ceb0 2024-05-21T22:03:06.679 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.676+0000 7f4b3ea8db80 10 client.16893 fetch_fsmap learned FSMap version 29 2024-05-21T22:03:06.679 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.676+0000 7f4b3ea8db80 1 -- 172.21.15.151:0/2031360832 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560803ea6060 con 0x560803f8ceb0 2024-05-21T22:03:06.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.676+0000 7f4b13400700 1 -- 172.21.15.151:0/2031360832 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b18022040 con 0x560803f8ceb0 2024-05-21T22:03:06.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.677+0000 7f4b13400700 1 -- 172.21.15.151:0/2031360832 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f4b18027040 con 0x560803f8ceb0 2024-05-21T22:03:06.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.677+0000 7f4b3ea8db80 10 client.16893 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:03:06.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.677+0000 7f4b3ea8db80 10 client.16893 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T22:03:06.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.677+0000 7f4b3ea8db80 20 client.16893 populate_metadata read hostname 'smithi151' 2024-05-21T22:03:06.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.677+0000 7f4b3ea8db80 1 --2- 172.21.15.151:0/2031360832 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x560803fa0b80 0x560803fa2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:06.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.677+0000 7f4b3ea8db80 4 client.16893 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T22:03:06.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.677+0000 7f4b3ea8db80 1 -- 172.21.15.151:0/2031360832 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560803ee7d90 con 0x560803fa0b80 2024-05-21T22:03:06.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.677+0000 7f4b31600700 1 --2- 172.21.15.151:0/2031360832 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x560803fa0b80 0x560803fa2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:06.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.678+0000 7f4b31600700 1 --2- 172.21.15.151:0/2031360832 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x560803fa0b80 0x560803fa2f70 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd564624a0d8a939 server_cookie=1301a97b58358559 in_seq=0 out_seq=0 2024-05-21T22:03:06.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.678+0000 7f4b13400700 10 client.16893 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T22:03:06.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.679+0000 7f4b13400700 1 -- 172.21.15.151:0/2031360832 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x560803ee7d90 con 0x560803fa0b80 2024-05-21T22:03:06.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.679+0000 7f4b13400700 10 client.16893 handle_command_reply: tid=0 2024-05-21T22:03:06.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.679+0000 7f4b3ea8db80 1 client.16893 shutdown 2024-05-21T22:03:06.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.679+0000 7f4b3ea8db80 1 -- 172.21.15.151:0/2031360832 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560803f8ceb0 msgr2=0x560803f908c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:06.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.679+0000 7f4b3ea8db80 1 --2- 172.21.15.151:0/2031360832 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560803f8ceb0 0x560803f908c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f4b180058c0 tx=0x7f4b1800f810 comp rx=0 tx=0).stop 2024-05-21T22:03:06.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.679+0000 7f4b3ea8db80 1 -- 172.21.15.151:0/2031360832 shutdown_connections 2024-05-21T22:03:06.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.679+0000 7f4b3ea8db80 1 --2- 172.21.15.151:0/2031360832 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x560803fa0b80 0x560803fa2f70 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:06.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.679+0000 7f4b3ea8db80 1 --2- 172.21.15.151:0/2031360832 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560803f87880 0x560803f934d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:06.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.679+0000 7f4b3ea8db80 1 --2- 172.21.15.151:0/2031360832 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560803f8d7d0 0x560803f90e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:06.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.679+0000 7f4b3ea8db80 1 --2- 172.21.15.151:0/2031360832 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560803f8ceb0 0x560803f908c0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:06.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.679+0000 7f4b3ea8db80 1 -- 172.21.15.151:0/2031360832 >> 172.21.15.151:0/2031360832 conn(0x560803eb5be0 msgr2=0x560803f86250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:03:06.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.679+0000 7f4b13400700 0 client.16893 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T22:03:06.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.680+0000 7f4b3ea8db80 1 -- 172.21.15.151:0/2031360832 shutdown_connections 2024-05-21T22:03:06.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.680+0000 7f4b3ea8db80 1 -- 172.21.15.151:0/2031360832 wait complete. 2024-05-21T22:03:06.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.680+0000 7f4b3ea8db80 20 client.16893 trim_cache size 0 max 16384 2024-05-21T22:03:06.702 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.702+0000 7f4b3ea8db80 1 Processor -- start 2024-05-21T22:03:06.702 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.702+0000 7f4b3ea8db80 1 -- start start 2024-05-21T22:03:06.703 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.702+0000 7f4b3ea8db80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560803f87880 0x560803f90ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:06.703 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.702+0000 7f4b3ea8db80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560803f8ceb0 0x560803f87260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:06.703 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.702+0000 7f4b3ea8db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560803f8d7d0 0x560803f8e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:06.703 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.702+0000 7f4b3ea8db80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4b2c1aff70 con 0x560803f87880 2024-05-21T22:03:06.703 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.702+0000 7f4b3ea8db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4b2c1b1ab0 con 0x560803f8d7d0 2024-05-21T22:03:06.703 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.702+0000 7f4b3ea8db80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4b2c1b0640 con 0x560803f8ceb0 2024-05-21T22:03:06.703 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.702+0000 7f4b31600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560803f87880 0x560803f90ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:06.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.703+0000 7f4b31600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560803f87880 0x560803f90ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:59506/0 (socket says 172.21.15.151:59506) 2024-05-21T22:03:06.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.703+0000 7f4b31600700 1 -- 172.21.15.151:0/3631211889 learned_addr learned my addr 172.21.15.151:0/3631211889 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:03:06.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.703+0000 7f4b32000700 1 --2- 172.21.15.151:0/3631211889 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560803f8d7d0 0x560803f8e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:06.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.703+0000 7f4b30c00700 1 --2- 172.21.15.151:0/3631211889 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560803f8ceb0 0x560803f87260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:06.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.703+0000 7f4b31600700 1 -- 172.21.15.151:0/3631211889 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560803f8ceb0 msgr2=0x560803f87260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:06.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.703+0000 7f4b31600700 1 --2- 172.21.15.151:0/3631211889 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560803f8ceb0 0x560803f87260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:06.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.703+0000 7f4b31600700 1 -- 172.21.15.151:0/3631211889 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560803f8d7d0 msgr2=0x560803f8e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:06.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.703+0000 7f4b31600700 1 --2- 172.21.15.151:0/3631211889 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560803f8d7d0 0x560803f8e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:06.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.703+0000 7f4b31600700 1 -- 172.21.15.151:0/3631211889 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560803f7a190 con 0x560803f87880 2024-05-21T22:03:06.705 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.703+0000 7f4b31600700 1 --2- 172.21.15.151:0/3631211889 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560803f87880 0x560803f90ef0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x560803f94270 tx=0x7f4b20009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90994e80bf7f1428 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:03:06.705 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.704+0000 7f4b1e000700 1 -- 172.21.15.151:0/3631211889 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b20004830 con 0x560803f87880 2024-05-21T22:03:06.705 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.704+0000 7f4b1e000700 1 -- 172.21.15.151:0/3631211889 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b20005020 con 0x560803f87880 2024-05-21T22:03:06.705 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.704+0000 7f4b1e000700 1 -- 172.21.15.151:0/3631211889 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b20010690 con 0x560803f87880 2024-05-21T22:03:06.705 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.704+0000 7f4b3ea8db80 1 -- 172.21.15.151:0/3631211889 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560803f87880 msgr2=0x560803f90ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:06.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.704+0000 7f4b3ea8db80 1 --2- 172.21.15.151:0/3631211889 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560803f87880 0x560803f90ef0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x560803f94270 tx=0x7f4b20009260 comp rx=0 tx=0).stop 2024-05-21T22:03:06.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.704+0000 7f4b3ea8db80 1 -- 172.21.15.151:0/3631211889 shutdown_connections 2024-05-21T22:03:06.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.704+0000 7f4b3ea8db80 1 --2- 172.21.15.151:0/3631211889 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560803f8d7d0 0x560803f8e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:06.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.704+0000 7f4b3ea8db80 1 --2- 172.21.15.151:0/3631211889 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560803f87880 0x560803f90ef0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:06.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.704+0000 7f4b3ea8db80 1 --2- 172.21.15.151:0/3631211889 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560803f8ceb0 0x560803f87260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:06.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.704+0000 7f4b3ea8db80 1 -- 172.21.15.151:0/3631211889 >> 172.21.15.151:0/3631211889 conn(0x560803eb5be0 msgr2=0x560803f8dc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:03:06.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.704+0000 7f4b3ea8db80 1 -- 172.21.15.151:0/3631211889 shutdown_connections 2024-05-21T22:03:06.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.705+0000 7f4b3ea8db80 1 -- 172.21.15.151:0/3631211889 wait complete. 2024-05-21T22:03:06.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.705+0000 7f4b3ea8db80 1 -- start start 2024-05-21T22:03:06.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.705+0000 7f4b3ea8db80 1 Processor -- start 2024-05-21T22:03:06.708 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.706+0000 7f4b3ea8db80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560803f87880 0x560803fa3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:06.708 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.706+0000 7f4b3ea8db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560803f8ceb0 0x560803fa42f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:06.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.706+0000 7f4b3ea8db80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560803f8d7d0 0x560803fa4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:06.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.706+0000 7f4b3ea8db80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4b2c1b13e0 con 0x560803f87880 2024-05-21T22:03:06.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.706+0000 7f4b3ea8db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4b2c1b0d10 con 0x560803f8ceb0 2024-05-21T22:03:06.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.706+0000 7f4b3ea8db80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4b2c1adf30 con 0x560803f8d7d0 2024-05-21T22:03:06.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.706+0000 7f4b3ea8db80 10 client.0 _ll_register_callbacks cb 0x560803f7c610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:03:06.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.706+0000 7f4b30c00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560803f8ceb0 0x560803fa42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:06.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.706+0000 7f4b30c00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560803f8ceb0 0x560803fa42f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:52260/0 (socket says 172.21.15.151:52260) 2024-05-21T22:03:06.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.706+0000 7f4b30c00700 1 -- 172.21.15.151:0/2225821002 learned_addr learned my addr 172.21.15.151:0/2225821002 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:03:06.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.706+0000 7f4b32000700 1 --2- 172.21.15.151:0/2225821002 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560803f8d7d0 0x560803fa4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:06.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.706+0000 7f4b31600700 1 --2- 172.21.15.151:0/2225821002 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560803f87880 0x560803fa3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:06.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.706+0000 7f4b30c00700 1 -- 172.21.15.151:0/2225821002 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560803f8d7d0 msgr2=0x560803fa4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:06.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.706+0000 7f4b30c00700 1 --2- 172.21.15.151:0/2225821002 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560803f8d7d0 0x560803fa4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:06.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.706+0000 7f4b30c00700 1 -- 172.21.15.151:0/2225821002 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560803f87880 msgr2=0x560803fa3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:06.710 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.706+0000 7f4b30c00700 1 --2- 172.21.15.151:0/2225821002 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560803f87880 0x560803fa3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:06.710 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.706+0000 7f4b30c00700 1 -- 172.21.15.151:0/2225821002 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56080400a490 con 0x560803f8ceb0 2024-05-21T22:03:06.710 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.706+0000 7f4b32000700 1 --2- 172.21.15.151:0/2225821002 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560803f8d7d0 0x560803fa4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T22:03:06.710 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.706+0000 7f4b31600700 1 --2- 172.21.15.151:0/2225821002 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560803f87880 0x560803fa3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T22:03:06.710 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.706+0000 7f4b30c00700 1 --2- 172.21.15.151:0/2225821002 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560803f8ceb0 0x560803fa42f0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f4b1800b2e0 tx=0x7f4b18033730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aac80ae5bb695335 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:03:06.710 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.706+0000 7f4b13400700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-21T22:03:06.710 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.707+0000 7f4b13400700 1 -- 172.21.15.151:0/2225821002 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b1804a020 con 0x560803f8ceb0 2024-05-21T22:03:06.710 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.707+0000 7f4b3ea8db80 1 -- 172.21.15.151:0/2225821002 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b20005020 con 0x560803f8ceb0 2024-05-21T22:03:06.710 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.707+0000 7f4b13400700 1 -- 172.21.15.151:0/2225821002 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8542+0+0 (secure 0 0 0) 0x7f4b18003aa0 con 0x560803f8ceb0 2024-05-21T22:03:06.710 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.707+0000 7f4b13400700 1 -- 172.21.15.151:0/2225821002 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4b1802e040 con 0x560803f8ceb0 2024-05-21T22:03:06.710 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.707+0000 7f4b3ea8db80 10 client.27026 fetch_fsmap learned FSMap version 29 2024-05-21T22:03:06.710 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.707+0000 7f4b3ea8db80 1 -- 172.21.15.151:0/2225821002 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56080400b7a0 con 0x560803f8ceb0 2024-05-21T22:03:06.711 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.707+0000 7f4b13400700 1 -- 172.21.15.151:0/2225821002 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b20005020 con 0x560803f8ceb0 2024-05-21T22:03:06.711 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.708+0000 7f4b13400700 1 -- 172.21.15.151:0/2225821002 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f4b18027070 con 0x560803f8ceb0 2024-05-21T22:03:06.711 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.708+0000 7f4b3ea8db80 10 client.27026 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:03:06.711 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.708+0000 7f4b3ea8db80 10 client.27026 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T22:03:06.711 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.708+0000 7f4b3ea8db80 20 client.27026 populate_metadata read hostname 'smithi151' 2024-05-21T22:03:06.711 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.708+0000 7f4b3ea8db80 1 --2- 172.21.15.151:0/2225821002 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x56080400be40 0x56080400e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:06.711 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.708+0000 7f4b3ea8db80 4 client.27026 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-21T22:03:06.711 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.708+0000 7f4b3ea8db80 1 -- 172.21.15.151:0/2225821002 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560803fa9100 con 0x56080400be40 2024-05-21T22:03:06.711 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.708+0000 7f4b31600700 1 --2- 172.21.15.151:0/2225821002 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x56080400be40 0x56080400e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:06.712 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.709+0000 7f4b31600700 1 --2- 172.21.15.151:0/2225821002 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x56080400be40 0x56080400e260 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90ae8da1a4f0f522 server_cookie=c1dbf8de19f2e183 in_seq=0 out_seq=0 2024-05-21T22:03:06.712 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.709+0000 7f4b13400700 10 client.27026 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T22:03:06.712 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.709+0000 7f4b13400700 1 -- 172.21.15.151:0/2225821002 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560803fa9100 con 0x56080400be40 2024-05-21T22:03:06.713 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.709+0000 7f4b13400700 10 client.27026 handle_command_reply: tid=0 2024-05-21T22:03:06.713 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.709+0000 7f4b3ea8db80 1 client.27026 shutdown 2024-05-21T22:03:06.713 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.709+0000 7f4b3ea8db80 1 -- 172.21.15.151:0/2225821002 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560803f8ceb0 msgr2=0x560803fa42f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:06.713 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.709+0000 7f4b3ea8db80 1 --2- 172.21.15.151:0/2225821002 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560803f8ceb0 0x560803fa42f0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f4b1800b2e0 tx=0x7f4b18033730 comp rx=0 tx=0).stop 2024-05-21T22:03:06.713 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.709+0000 7f4b3ea8db80 1 -- 172.21.15.151:0/2225821002 shutdown_connections 2024-05-21T22:03:06.713 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.709+0000 7f4b3ea8db80 1 --2- 172.21.15.151:0/2225821002 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560803f8d7d0 0x560803fa4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:06.713 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.709+0000 7f4b3ea8db80 1 --2- 172.21.15.151:0/2225821002 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560803f87880 0x560803fa3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:06.713 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.709+0000 7f4b3ea8db80 1 --2- 172.21.15.151:0/2225821002 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x56080400be40 0x56080400e260 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:06.713 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.709+0000 7f4b3ea8db80 1 --2- 172.21.15.151:0/2225821002 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560803f8ceb0 0x560803fa42f0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:06.713 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.709+0000 7f4b13400700 0 client.27026 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T22:03:06.713 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.709+0000 7f4b3ea8db80 1 -- 172.21.15.151:0/2225821002 >> 172.21.15.151:0/2225821002 conn(0x560803eb5be0 msgr2=0x560803f8f1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:03:06.713 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.710+0000 7f4b3ea8db80 1 -- 172.21.15.151:0/2225821002 shutdown_connections 2024-05-21T22:03:06.713 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.710+0000 7f4b3ea8db80 1 -- 172.21.15.151:0/2225821002 wait complete. 2024-05-21T22:03:06.713 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.710+0000 7f4b3ea8db80 20 client.27026 trim_cache size 0 max 16384 2024-05-21T22:03:06.714 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-05-21T22:03:06.714 INFO:teuthology.orchestra.run.smithi151.stdout: "status": "no active scrubs running", 2024-05-21T22:03:06.714 INFO:teuthology.orchestra.run.smithi151.stdout: "scrubs": {} 2024-05-21T22:03:06.714 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-05-21T22:03:06.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.711+0000 7f4b34000700 1 -- 172.21.15.151:0/2276425443 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f4b140b2a70 msgr2=0x7f4b140b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:06.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.711+0000 7f4b34000700 1 --2- 172.21.15.151:0/2276425443 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f4b140b2a70 0x7f4b140b4f30 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c111c90 tx=0x7f4b18009550 comp rx=0 tx=0).stop 2024-05-21T22:03:06.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.711+0000 7f4b34000700 1 -- 172.21.15.151:0/2276425443 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4b2c10ff70 msgr2=0x7f4b2c1a0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:06.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.711+0000 7f4b34000700 1 --2- 172.21.15.151:0/2276425443 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4b2c10ff70 0x7f4b2c1a0750 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f4b2400ef90 tx=0x7f4b2400c3a0 comp rx=0 tx=0).stop 2024-05-21T22:03:06.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.711+0000 7f4b34000700 1 -- 172.21.15.151:0/2276425443 shutdown_connections 2024-05-21T22:03:06.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.711+0000 7f4b34000700 1 --2- 172.21.15.151:0/2276425443 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f4b140b2a70 0x7f4b140b4f30 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:06.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.711+0000 7f4b34000700 1 --2- 172.21.15.151:0/2276425443 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4b2c10ff70 0x7f4b2c1a0750 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:06.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.711+0000 7f4b34000700 1 --2- 172.21.15.151:0/2276425443 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4b2c075770 0x7f4b2c19be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:06.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.711+0000 7f4b34000700 1 --2- 172.21.15.151:0/2276425443 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4b2c076090 0x7f4b2c19c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:06.716 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.711+0000 7f4b34000700 1 -- 172.21.15.151:0/2276425443 >> 172.21.15.151:0/2276425443 conn(0x7f4b2c07a6f0 msgr2=0x7f4b2c1007e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:03:06.716 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.711+0000 7f4b34000700 1 -- 172.21.15.151:0/2276425443 shutdown_connections 2024-05-21T22:03:06.716 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:06.711+0000 7f4b34000700 1 -- 172.21.15.151:0/2276425443 wait complete. 2024-05-21T22:03:07.631 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-21T22:03:07.631 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-21T22:03:07.952 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:03:07 smithi151 ceph-mon[31623]: pgmap v230: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-05-21T22:03:07.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:03:07 smithi184 ceph-mon[36942]: pgmap v230: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-05-21T22:03:08.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:03:07 smithi164 ceph-mon[36269]: pgmap v230: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-05-21T22:03:08.073 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:03:09.804 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:03:09 smithi151 ceph-mon[31623]: pgmap v231: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-05-21T22:03:09.838 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:03:09 smithi164 ceph-mon[36269]: pgmap v231: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-05-21T22:03:09.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:03:09 smithi184 ceph-mon[36942]: pgmap v231: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-05-21T22:03:10.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:10.358+0000 7f7eef000700 1 -- 172.21.15.151:0/2228484776 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7ee8103f50 msgr2=0x7f7ee8110430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:10.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:10.358+0000 7f7eef000700 1 --2- 172.21.15.151:0/2228484776 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7ee8103f50 0x7f7ee8110430 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f7ee000b0a0 tx=0x7f7ee000b3b0 comp rx=0 tx=0).stop 2024-05-21T22:03:10.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:10.358+0000 7f7eef000700 1 -- 172.21.15.151:0/2228484776 shutdown_connections 2024-05-21T22:03:10.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:10.358+0000 7f7eef000700 1 --2- 172.21.15.151:0/2228484776 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7ee8103f50 0x7f7ee8110430 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:10.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:10.358+0000 7f7eef000700 1 --2- 172.21.15.151:0/2228484776 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ee81096a0 0x7f7ee8109a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:10.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:10.358+0000 7f7eef000700 1 --2- 172.21.15.151:0/2228484776 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7ee8103590 0x7f7ee8103a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:10.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:10.358+0000 7f7eef000700 1 -- 172.21.15.151:0/2228484776 >> 172.21.15.151:0/2228484776 conn(0x7f7ee807a6f0 msgr2=0x7f7ee807ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:03:10.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:10.358+0000 7f7eef000700 1 -- 172.21.15.151:0/2228484776 shutdown_connections 2024-05-21T22:03:10.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:10.358+0000 7f7eef000700 1 -- 172.21.15.151:0/2228484776 wait complete. 2024-05-21T22:03:10.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:10.359+0000 7f7eef000700 1 Processor -- start 2024-05-21T22:03:10.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:10.360+0000 7f7eef000700 1 -- start start 2024-05-21T22:03:10.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:10.360+0000 7f7eef000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7ee8103590 0x7f7ee819bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:10.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:10.360+0000 7f7eef000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7ee8103f50 0x7f7ee819c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:10.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:10.360+0000 7f7eef000700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ee81096a0 0x7f7ee81a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:10.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:10.360+0000 7f7eef000700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee81137c0 con 0x7f7ee8103f50 2024-05-21T22:03:10.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:10.360+0000 7f7eef000700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee8113640 con 0x7f7ee8103590 2024-05-21T22:03:10.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:10.360+0000 7f7eef000700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee8113940 con 0x7f7ee81096a0 2024-05-21T22:03:10.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:10.360+0000 7f7ee7400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7ee8103f50 0x7f7ee819c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:10.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:10.361+0000 7f7eed000700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ee81096a0 0x7f7ee81a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:10.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:10.361+0000 7f7ee7e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7ee8103590 0x7f7ee819bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:10.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:10.361+0000 7f7eed000700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ee81096a0 0x7f7ee81a0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:55370/0 (socket says 172.21.15.151:55370) 2024-05-21T22:03:10.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:10.361+0000 7f7eed000700 1 -- 172.21.15.151:0/3422074896 learned_addr learned my addr 172.21.15.151:0/3422074896 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:03:10.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:10.361+0000 7f7eed000700 1 -- 172.21.15.151:0/3422074896 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7ee8103590 msgr2=0x7f7ee819bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:10.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:10.361+0000 7f7eed000700 1 --2- 172.21.15.151:0/3422074896 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7ee8103590 0x7f7ee819bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:10.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:10.361+0000 7f7eed000700 1 -- 172.21.15.151:0/3422074896 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7ee8103f50 msgr2=0x7f7ee819c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:10.363 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:10.361+0000 7f7eed000700 1 --2- 172.21.15.151:0/3422074896 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7ee8103f50 0x7f7ee819c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:10.363 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:10.361+0000 7f7eed000700 1 -- 172.21.15.151:0/3422074896 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ee81a0dc0 con 0x7f7ee81096a0 2024-05-21T22:03:10.363 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:10.361+0000 7f7eed000700 1 --2- 172.21.15.151:0/3422074896 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ee81096a0 0x7f7ee81a0680 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f7ee000b070 tx=0x7f7ee0004c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:03:10.363 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:10.361+0000 7f7ee4c00700 1 -- 172.21.15.151:0/3422074896 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee0021070 con 0x7f7ee81096a0 2024-05-21T22:03:10.363 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:10.362+0000 7f7eef000700 1 -- 172.21.15.151:0/3422074896 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ee81a1050 con 0x7f7ee81096a0 2024-05-21T22:03:10.363 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:10.362+0000 7f7ee4c00700 1 -- 172.21.15.151:0/3422074896 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ee0007e30 con 0x7f7ee81096a0 2024-05-21T22:03:10.363 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:10.362+0000 7f7ee4c00700 1 -- 172.21.15.151:0/3422074896 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee0004570 con 0x7f7ee81096a0 2024-05-21T22:03:10.363 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:10.362+0000 7f7eef000700 1 -- 172.21.15.151:0/3422074896 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ee81a8930 con 0x7f7ee81096a0 2024-05-21T22:03:10.363 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:10.363+0000 7f7eef000700 1 -- 172.21.15.151:0/3422074896 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ee8104cb0 con 0x7f7ee81096a0 2024-05-21T22:03:10.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:10.365+0000 7f7ee4c00700 1 -- 172.21.15.151:0/3422074896 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7f7ee0004710 con 0x7f7ee81096a0 2024-05-21T22:03:10.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:10.367+0000 7f7ee4c00700 1 --2- 172.21.15.151:0/3422074896 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f7ecc0b2a70 0x7f7ecc0b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:10.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:10.367+0000 7f7ee4c00700 1 -- 172.21.15.151:0/3422074896 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8586+0+0 (secure 0 0 0) 0x7f7ee00e3380 con 0x7f7ee81096a0 2024-05-21T22:03:10.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:10.367+0000 7f7ee7e00700 1 --2- 172.21.15.151:0/3422074896 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f7ecc0b2a70 0x7f7ecc0b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:10.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:10.368+0000 7f7ee7e00700 1 --2- 172.21.15.151:0/3422074896 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f7ecc0b2a70 0x7f7ecc0b4f30 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7ed8009c60 tx=0x7f7ed8009260 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:03:10.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:10.369+0000 7f7ee4c00700 1 -- 172.21.15.151:0/3422074896 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7ee00aad50 con 0x7f7ee81096a0 2024-05-21T22:03:10.652 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:10.652+0000 7f7eef000700 1 -- 172.21.15.151:0/3422074896 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7ee806a230 con 0x7f7ee81096a0 2024-05-21T22:03:10.653 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:10.653+0000 7f7ee4c00700 1 -- 172.21.15.151:0/3422074896 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11958 (secure 0 0 0) 0x7f7ee806a230 con 0x7f7ee81096a0 2024-05-21T22:03:10.654 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T22:03:10.654 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14358,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/158010529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":158010529},{"type":"v1","addr":"172.21.15.151:6835","nonce":158010529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6839/2069058115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":2069058115},{"type":"v1","addr":"172.21.15.151:6839","nonce":2069058115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T21:51:19.552069+0000","modified":"2024-05-21T21:55:34.643755+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24530,"mds_1":24493,"mds_2":24491,"mds_3":24478,"mds_4":24473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14403":{"gid":14403,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6841/1749723794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6840","nonce":1749723794},{"type":"v1","addr":"172.21.15.151:6841","nonce":1749723794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/2026266114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":2026266114},{"type":"v1","addr":"172.21.15.184:6833","nonce":2026266114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/2467900844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":2467900844},{"type":"v1","addr":"172.21.15.164:6833","nonce":2467900844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3367640960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3367640960},{"type":"v1","addr":"172.21.15.151:6837","nonce":3367640960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/3264142621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3264142621},{"type":"v1","addr":"172.21.15.184:6835","nonce":3264142621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":17,"addr":"172.21.15.164:6835/2340428470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2340428470},{"type":"v1","addr":"172.21.15.164:6835","nonce":2340428470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":16,"addr":"172.21.15.184:6837/2988194888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":2988194888},{"type":"v1","addr":"172.21.15.184:6837","nonce":2988194888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.164:6837/1554609572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1554609572},{"type":"v1","addr":"172.21.15.164:6837","nonce":1554609572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.184:6839/3059935143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":3059935143},{"type":"v1","addr":"172.21.15.184:6839","nonce":3059935143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.164:6839/2646401247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":2646401247},{"type":"v1","addr":"172.21.15.164:6839","nonce":2646401247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24530,"qdb_cluster":[24530,24493,24491,24473,24478]},"id":1}]} 2024-05-21T22:03:10.656 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:10.656+0000 7f7eef000700 1 -- 172.21.15.151:0/3422074896 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f7ecc0b2a70 msgr2=0x7f7ecc0b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:10.656 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:10.656+0000 7f7eef000700 1 --2- 172.21.15.151:0/3422074896 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f7ecc0b2a70 0x7f7ecc0b4f30 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7ed8009c60 tx=0x7f7ed8009260 comp rx=0 tx=0).stop 2024-05-21T22:03:10.656 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:10.656+0000 7f7eef000700 1 -- 172.21.15.151:0/3422074896 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ee81096a0 msgr2=0x7f7ee81a0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:10.656 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:10.656+0000 7f7eef000700 1 --2- 172.21.15.151:0/3422074896 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ee81096a0 0x7f7ee81a0680 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f7ee000b070 tx=0x7f7ee0004c80 comp rx=0 tx=0).stop 2024-05-21T22:03:10.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:10.656+0000 7f7eef000700 1 -- 172.21.15.151:0/3422074896 shutdown_connections 2024-05-21T22:03:10.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:10.656+0000 7f7eef000700 1 --2- 172.21.15.151:0/3422074896 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f7ecc0b2a70 0x7f7ecc0b4f30 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:10.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:10.656+0000 7f7eef000700 1 --2- 172.21.15.151:0/3422074896 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7ee8103590 0x7f7ee819bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:10.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:10.656+0000 7f7eef000700 1 --2- 172.21.15.151:0/3422074896 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ee81096a0 0x7f7ee81a0680 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:10.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:10.656+0000 7f7eef000700 1 --2- 172.21.15.151:0/3422074896 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7ee8103f50 0x7f7ee819c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:10.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:10.656+0000 7f7eef000700 1 -- 172.21.15.151:0/3422074896 >> 172.21.15.151:0/3422074896 conn(0x7f7ee807a6f0 msgr2=0x7f7ee810e990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:03:10.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:10.656+0000 7f7eef000700 1 -- 172.21.15.151:0/3422074896 shutdown_connections 2024-05-21T22:03:10.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:10.656+0000 7f7eef000700 1 -- 172.21.15.151:0/3422074896 wait complete. 2024-05-21T22:03:10.658 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 29 2024-05-21T22:03:10.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:03:10 smithi151 ceph-mon[31623]: osdmap e86: 12 total, 12 up, 12 in 2024-05-21T22:03:10.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:03:10 smithi184 ceph-mon[36942]: osdmap e86: 12 total, 12 up, 12 in 2024-05-21T22:03:11.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:03:10 smithi164 ceph-mon[36269]: osdmap e86: 12 total, 12 up, 12 in 2024-05-21T22:03:11.533 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-21T22:03:11.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:03:11 smithi151 ceph-mon[31623]: pgmap v233: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-05-21T22:03:11.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:03:11 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/3422074896' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T22:03:11.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:03:11 smithi184 ceph-mon[36942]: pgmap v233: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-05-21T22:03:11.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:03:11 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/3422074896' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T22:03:11.996 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:03:12.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:03:11 smithi164 ceph-mon[36269]: pgmap v233: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-05-21T22:03:12.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:03:11 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/3422074896' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T22:03:13.888 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:03:13 smithi151 ceph-mon[31623]: pgmap v234: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-21T22:03:13.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:03:13 smithi184 ceph-mon[36942]: pgmap v234: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-21T22:03:14.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:03:13 smithi164 ceph-mon[36269]: pgmap v234: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-21T22:03:14.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.226+0000 7fab3be00700 1 -- 172.21.15.151:0/3452857847 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fab3c103e20 msgr2=0x7fab3c110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:14.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.226+0000 7fab3be00700 1 --2- 172.21.15.151:0/3452857847 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fab3c103e20 0x7fab3c110330 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fab34009990 tx=0x7fab34009ca0 comp rx=0 tx=0).stop 2024-05-21T22:03:14.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.226+0000 7fab3be00700 1 -- 172.21.15.151:0/3452857847 shutdown_connections 2024-05-21T22:03:14.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.226+0000 7fab3be00700 1 --2- 172.21.15.151:0/3452857847 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab3c1094f0 0x7fab3c1098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:14.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.226+0000 7fab3be00700 1 --2- 172.21.15.151:0/3452857847 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fab3c103e20 0x7fab3c110330 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:14.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.226+0000 7fab3be00700 1 --2- 172.21.15.151:0/3452857847 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fab3c103460 0x7fab3c1038e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:14.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.226+0000 7fab3be00700 1 -- 172.21.15.151:0/3452857847 >> 172.21.15.151:0/3452857847 conn(0x7fab3c0fef50 msgr2=0x7fab3c101370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:03:14.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.227+0000 7fab3be00700 1 -- 172.21.15.151:0/3452857847 shutdown_connections 2024-05-21T22:03:14.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.227+0000 7fab3be00700 1 -- 172.21.15.151:0/3452857847 wait complete. 2024-05-21T22:03:14.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.228+0000 7fab3be00700 1 Processor -- start 2024-05-21T22:03:14.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.228+0000 7fab3be00700 1 -- start start 2024-05-21T22:03:14.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.228+0000 7fab3be00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab3c103460 0x7fab3c1a1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:14.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.228+0000 7fab3be00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fab3c103e20 0x7fab3c1a1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:14.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.228+0000 7fab3be00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fab3c1094f0 0x7fab3c19bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:14.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.228+0000 7fab3be00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fab3c113630 con 0x7fab3c1094f0 2024-05-21T22:03:14.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.228+0000 7fab3be00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fab3c1134b0 con 0x7fab3c103460 2024-05-21T22:03:14.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.228+0000 7fab3be00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fab3c1137b0 con 0x7fab3c103e20 2024-05-21T22:03:14.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.229+0000 7fab3b400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fab3c1094f0 0x7fab3c19bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:14.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.229+0000 7fab3b400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fab3c1094f0 0x7fab3c19bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:59552/0 (socket says 172.21.15.151:59552) 2024-05-21T22:03:14.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.229+0000 7fab3b400700 1 -- 172.21.15.151:0/3145791038 learned_addr learned my addr 172.21.15.151:0/3145791038 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:03:14.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.229+0000 7fab3aa00700 1 --2- 172.21.15.151:0/3145791038 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab3c103460 0x7fab3c1a1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:14.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.229+0000 7fab32000700 1 --2- 172.21.15.151:0/3145791038 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fab3c103e20 0x7fab3c1a1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:14.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.229+0000 7fab3b400700 1 -- 172.21.15.151:0/3145791038 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fab3c103e20 msgr2=0x7fab3c1a1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:14.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.229+0000 7fab3b400700 1 --2- 172.21.15.151:0/3145791038 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fab3c103e20 0x7fab3c1a1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:14.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.229+0000 7fab3b400700 1 -- 172.21.15.151:0/3145791038 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab3c103460 msgr2=0x7fab3c1a1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:14.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.229+0000 7fab3b400700 1 --2- 172.21.15.151:0/3145791038 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab3c103460 0x7fab3c1a1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:14.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.229+0000 7fab3b400700 1 -- 172.21.15.151:0/3145791038 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab3c19c3f0 con 0x7fab3c1094f0 2024-05-21T22:03:14.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.229+0000 7fab3aa00700 1 --2- 172.21.15.151:0/3145791038 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab3c103460 0x7fab3c1a1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:03:14.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.230+0000 7fab3b400700 1 --2- 172.21.15.151:0/3145791038 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fab3c1094f0 0x7fab3c19bb10 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fab3400c010 tx=0x7fab340057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:03:14.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.230+0000 7fab33e00700 1 -- 172.21.15.151:0/3145791038 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab34027070 con 0x7fab3c1094f0 2024-05-21T22:03:14.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.230+0000 7fab33e00700 1 -- 172.21.15.151:0/3145791038 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab34022470 con 0x7fab3c1094f0 2024-05-21T22:03:14.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.230+0000 7fab3be00700 1 -- 172.21.15.151:0/3145791038 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab3c19c680 con 0x7fab3c1094f0 2024-05-21T22:03:14.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.230+0000 7fab33e00700 1 -- 172.21.15.151:0/3145791038 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab34017630 con 0x7fab3c1094f0 2024-05-21T22:03:14.233 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.230+0000 7fab3be00700 1 -- 172.21.15.151:0/3145791038 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab3c1a87f0 con 0x7fab3c1094f0 2024-05-21T22:03:14.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.235+0000 7fab33e00700 1 -- 172.21.15.151:0/3145791038 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7fab3402a070 con 0x7fab3c1094f0 2024-05-21T22:03:14.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.237+0000 7fab4c4a8b80 1 Processor -- start 2024-05-21T22:03:14.238 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.237+0000 7fab4c4a8b80 1 -- start start 2024-05-21T22:03:14.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.238+0000 7fab4c4a8b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f77ed0fd0 0x557f77ed13b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:14.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.238+0000 7fab4c4a8b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557f77ed18f0 0x557f77ecb580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:14.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.238+0000 7fab4c4a8b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557f77ecbba0 0x557f77ecc000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:14.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.238+0000 7fab4c4a8b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x557f77e20e40 con 0x557f77ecbba0 2024-05-21T22:03:14.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.238+0000 7fab4c4a8b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x557f77e109e0 con 0x557f77ed0fd0 2024-05-21T22:03:14.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.238+0000 7fab4c4a8b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x557f77e14a20 con 0x557f77ed18f0 2024-05-21T22:03:14.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.239+0000 7fab3aa00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557f77ed18f0 0x557f77ecb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:14.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.239+0000 7fab32000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f77ed0fd0 0x557f77ed13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:14.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.239+0000 7fab3aa00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557f77ed18f0 0x557f77ecb580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:55402/0 (socket says 172.21.15.151:55402) 2024-05-21T22:03:14.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.239+0000 7fab3aa00700 1 -- 172.21.15.151:0/1946805257 learned_addr learned my addr 172.21.15.151:0/1946805257 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:03:14.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.239+0000 7fab32000700 1 --2- 172.21.15.151:0/1946805257 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557f77ecbba0 0x557f77ecc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:14.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.239+0000 7fab3aa00700 1 -- 172.21.15.151:0/1946805257 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f77ed0fd0 msgr2=0x557f77ed13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:14.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.239+0000 7fab3aa00700 1 --2- 172.21.15.151:0/1946805257 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f77ed0fd0 0x557f77ed13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:14.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.239+0000 7fab3aa00700 1 -- 172.21.15.151:0/1946805257 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557f77ecbba0 msgr2=0x557f77ecc000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:14.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.239+0000 7fab3aa00700 1 --2- 172.21.15.151:0/1946805257 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557f77ecbba0 0x557f77ecc000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:14.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.239+0000 7fab3aa00700 1 -- 172.21.15.151:0/1946805257 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f77dd9420 con 0x557f77ed18f0 2024-05-21T22:03:14.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.240+0000 7fab3aa00700 1 --2- 172.21.15.151:0/1946805257 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557f77ed18f0 0x557f77ecb580 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fab2c0078d0 tx=0x7fab2c007be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2e371047897b3ab server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:03:14.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.240+0000 7fab33e00700 1 --2- 172.21.15.151:0/3145791038 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fab200b2a70 0x7fab200b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:14.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.240+0000 7fab2be00700 1 -- 172.21.15.151:0/1946805257 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab2c019030 con 0x557f77ed18f0 2024-05-21T22:03:14.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.240+0000 7fab2be00700 1 -- 172.21.15.151:0/1946805257 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab2c015b00 con 0x557f77ed18f0 2024-05-21T22:03:14.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.240+0000 7fab2be00700 1 -- 172.21.15.151:0/1946805257 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab2c014ae0 con 0x557f77ed18f0 2024-05-21T22:03:14.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.241+0000 7fab33e00700 1 -- 172.21.15.151:0/3145791038 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7fab340e3400 con 0x7fab3c1094f0 2024-05-21T22:03:14.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.241+0000 7fab32000700 1 --2- 172.21.15.151:0/3145791038 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fab200b2a70 0x7fab200b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:14.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.242+0000 7fab4c4a8b80 1 -- 172.21.15.151:0/1946805257 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557f77ed18f0 msgr2=0x557f77ecb580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:14.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.242+0000 7fab4c4a8b80 1 --2- 172.21.15.151:0/1946805257 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557f77ed18f0 0x557f77ecb580 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fab2c0078d0 tx=0x7fab2c007be0 comp rx=0 tx=0).stop 2024-05-21T22:03:14.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.242+0000 7fab32000700 1 --2- 172.21.15.151:0/3145791038 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fab200b2a70 0x7fab200b4f30 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fab3c19d260 tx=0x7fab240047c0 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:03:14.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.243+0000 7fab4c4a8b80 1 -- 172.21.15.151:0/1946805257 shutdown_connections 2024-05-21T22:03:14.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.243+0000 7fab4c4a8b80 1 --2- 172.21.15.151:0/1946805257 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f77ed0fd0 0x557f77ed13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:14.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.243+0000 7fab4c4a8b80 1 --2- 172.21.15.151:0/1946805257 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557f77ecbba0 0x557f77ecc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:14.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.243+0000 7fab4c4a8b80 1 --2- 172.21.15.151:0/1946805257 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557f77ed18f0 0x557f77ecb580 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:14.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.243+0000 7fab4c4a8b80 1 -- 172.21.15.151:0/1946805257 >> 172.21.15.151:0/1946805257 conn(0x557f77df0be0 msgr2=0x557f77ebd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:03:14.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.243+0000 7fab4c4a8b80 1 -- 172.21.15.151:0/1946805257 shutdown_connections 2024-05-21T22:03:14.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.243+0000 7fab4c4a8b80 1 -- 172.21.15.151:0/1946805257 wait complete. 2024-05-21T22:03:14.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.244+0000 7fab4c4a8b80 1 -- start start 2024-05-21T22:03:14.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.244+0000 7fab4c4a8b80 1 Processor -- start 2024-05-21T22:03:14.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.244+0000 7fab4c4a8b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557f77ecbba0 0x557f77ed75b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:14.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.244+0000 7fab4c4a8b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f77ed0fd0 0x557f77ed4950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:14.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.245+0000 7fab4c4a8b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557f77ed18f0 0x557f77ed4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:14.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.245+0000 7fab4c4a8b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x557f77d16fa0 con 0x557f77ed18f0 2024-05-21T22:03:14.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.245+0000 7fab4c4a8b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x557f77db1760 con 0x557f77ed0fd0 2024-05-21T22:03:14.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.245+0000 7fab4c4a8b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x557f77ed8410 con 0x557f77ecbba0 2024-05-21T22:03:14.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.245+0000 7fab4c4a8b80 10 client.0 _ll_register_callbacks cb 0x557f77dbac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:03:14.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.245+0000 7fab3aa00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557f77ecbba0 0x557f77ed75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:14.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.245+0000 7fab32000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f77ed0fd0 0x557f77ed4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:14.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.245+0000 7fab3aa00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557f77ecbba0 0x557f77ed75b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:55414/0 (socket says 172.21.15.151:55414) 2024-05-21T22:03:14.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.245+0000 7fab3aa00700 1 -- 172.21.15.151:0/380603276 learned_addr learned my addr 172.21.15.151:0/380603276 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:03:14.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.245+0000 7fab3b400700 1 --2- 172.21.15.151:0/380603276 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557f77ed18f0 0x557f77ed4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:14.247 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.245+0000 7fab3aa00700 1 -- 172.21.15.151:0/380603276 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f77ed0fd0 msgr2=0x557f77ed4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:14.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.245+0000 7fab3aa00700 1 --2- 172.21.15.151:0/380603276 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f77ed0fd0 0x557f77ed4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:14.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.245+0000 7fab3aa00700 1 -- 172.21.15.151:0/380603276 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557f77ed18f0 msgr2=0x557f77ed4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:14.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.245+0000 7fab3aa00700 1 --2- 172.21.15.151:0/380603276 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557f77ed18f0 0x557f77ed4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:14.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.245+0000 7fab3aa00700 1 -- 172.21.15.151:0/380603276 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f77d608d0 con 0x557f77ecbba0 2024-05-21T22:03:14.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.246+0000 7fab3aa00700 1 --2- 172.21.15.151:0/380603276 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557f77ecbba0 0x557f77ed75b0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fab2c0078a0 tx=0x7fab2c015880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5fdcbafea97ff3a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:03:14.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.246+0000 7fab28c00700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-21T22:03:14.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.246+0000 7fab28c00700 1 -- 172.21.15.151:0/380603276 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab2c022070 con 0x557f77ecbba0 2024-05-21T22:03:14.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.246+0000 7fab28c00700 1 -- 172.21.15.151:0/380603276 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7fab2c003a00 con 0x557f77ecbba0 2024-05-21T22:03:14.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.246+0000 7fab4c4a8b80 1 -- 172.21.15.151:0/380603276 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab2c015b00 con 0x557f77ecbba0 2024-05-21T22:03:14.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.246+0000 7fab28c00700 1 -- 172.21.15.151:0/380603276 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fab2c02e050 con 0x557f77ecbba0 2024-05-21T22:03:14.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.246+0000 7fab4c4a8b80 10 client.27049 fetch_fsmap learned FSMap version 29 2024-05-21T22:03:14.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.246+0000 7fab4c4a8b80 1 -- 172.21.15.151:0/380603276 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557f77de1060 con 0x557f77ecbba0 2024-05-21T22:03:14.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.247+0000 7fab28c00700 1 -- 172.21.15.151:0/380603276 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab2c015b00 con 0x557f77ecbba0 2024-05-21T22:03:14.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.247+0000 7fab28c00700 1 -- 172.21.15.151:0/380603276 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fab2c004030 con 0x557f77ecbba0 2024-05-21T22:03:14.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.247+0000 7fab4c4a8b80 10 client.27049 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:03:14.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.247+0000 7fab4c4a8b80 10 client.27049 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-05-21T22:03:14.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.247+0000 7fab4c4a8b80 20 client.27049 populate_metadata read hostname 'smithi151' 2024-05-21T22:03:14.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.247+0000 7fab4c4a8b80 1 --2- 172.21.15.151:0/380603276 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x557f77ee4d80 0x557f77ee7170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:14.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.247+0000 7fab4c4a8b80 4 client.27049 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T22:03:14.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.247+0000 7fab4c4a8b80 1 -- 172.21.15.151:0/380603276 --> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557f77e22d90 con 0x557f77ee4d80 2024-05-21T22:03:14.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.247+0000 7fab32000700 1 --2- 172.21.15.151:0/380603276 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x557f77ee4d80 0x557f77ee7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:14.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.248+0000 7fab32000700 1 --2- 172.21.15.151:0/380603276 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x557f77ee4d80 0x557f77ee7170 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5fe6c271ce35f3ec server_cookie=4c69502d7a79caa1 in_seq=0 out_seq=0 2024-05-21T22:03:14.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.248+0000 7fab28c00700 10 client.27049 ms_handle_connect on v2:172.21.15.164:6834/2340428470 2024-05-21T22:03:14.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.249+0000 7fab28c00700 1 -- 172.21.15.151:0/380603276 <== mds.4 v2:172.21.15.164:6834/2340428470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x557f77e22d90 con 0x557f77ee4d80 2024-05-21T22:03:14.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.249+0000 7fab28c00700 10 client.27049 handle_command_reply: tid=0 2024-05-21T22:03:14.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.249+0000 7fab4c4a8b80 1 client.27049 shutdown 2024-05-21T22:03:14.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.249+0000 7fab4c4a8b80 1 -- 172.21.15.151:0/380603276 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557f77ecbba0 msgr2=0x557f77ed75b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:14.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.249+0000 7fab4c4a8b80 1 --2- 172.21.15.151:0/380603276 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557f77ecbba0 0x557f77ed75b0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fab2c0078a0 tx=0x7fab2c015880 comp rx=0 tx=0).stop 2024-05-21T22:03:14.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.249+0000 7fab4c4a8b80 1 -- 172.21.15.151:0/380603276 shutdown_connections 2024-05-21T22:03:14.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.249+0000 7fab4c4a8b80 1 --2- 172.21.15.151:0/380603276 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x557f77ee4d80 0x557f77ee7170 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:14.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.249+0000 7fab4c4a8b80 1 --2- 172.21.15.151:0/380603276 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557f77ed18f0 0x557f77ed4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:14.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.249+0000 7fab4c4a8b80 1 --2- 172.21.15.151:0/380603276 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557f77ecbba0 0x557f77ed75b0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:14.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.249+0000 7fab28c00700 0 client.27049 ms_handle_reset on v2:172.21.15.164:6834/2340428470 2024-05-21T22:03:14.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.249+0000 7fab4c4a8b80 1 --2- 172.21.15.151:0/380603276 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f77ed0fd0 0x557f77ed4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:14.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.249+0000 7fab4c4a8b80 1 -- 172.21.15.151:0/380603276 >> 172.21.15.151:0/380603276 conn(0x557f77df0be0 msgr2=0x557f77ebcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:03:14.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.249+0000 7fab4c4a8b80 1 -- 172.21.15.151:0/380603276 shutdown_connections 2024-05-21T22:03:14.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.249+0000 7fab4c4a8b80 1 -- 172.21.15.151:0/380603276 wait complete. 2024-05-21T22:03:14.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.250+0000 7fab4c4a8b80 20 client.27049 trim_cache size 0 max 16384 2024-05-21T22:03:14.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.266+0000 7fab4c4a8b80 1 Processor -- start 2024-05-21T22:03:14.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.266+0000 7fab4c4a8b80 1 -- start start 2024-05-21T22:03:14.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.266+0000 7fab4c4a8b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557f77ecbba0 0x557f77ecb580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:14.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.267+0000 7fab4c4a8b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557f77ed0fd0 0x557f77ed4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:14.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.267+0000 7fab4c4a8b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f77ed18f0 0x557f77ebd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:14.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.267+0000 7fab4c4a8b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fab3c1abac0 con 0x557f77ed0fd0 2024-05-21T22:03:14.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.267+0000 7fab4c4a8b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fab3c1ad600 con 0x557f77ed18f0 2024-05-21T22:03:14.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.267+0000 7fab4c4a8b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fab3c1ac190 con 0x557f77ecbba0 2024-05-21T22:03:14.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.267+0000 7fab32000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557f77ed0fd0 0x557f77ed4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:14.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.267+0000 7fab3aa00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557f77ecbba0 0x557f77ecb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:14.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.267+0000 7fab32000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557f77ed0fd0 0x557f77ed4e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:59586/0 (socket says 172.21.15.151:59586) 2024-05-21T22:03:14.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.267+0000 7fab32000700 1 -- 172.21.15.151:0/3176674883 learned_addr learned my addr 172.21.15.151:0/3176674883 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:03:14.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.267+0000 7fab3b400700 1 --2- 172.21.15.151:0/3176674883 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f77ed18f0 0x557f77ebd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:14.269 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.267+0000 7fab32000700 1 -- 172.21.15.151:0/3176674883 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557f77ecbba0 msgr2=0x557f77ecb580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:14.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.267+0000 7fab32000700 1 --2- 172.21.15.151:0/3176674883 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557f77ecbba0 0x557f77ecb580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:14.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.267+0000 7fab32000700 1 -- 172.21.15.151:0/3176674883 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f77ed18f0 msgr2=0x557f77ebd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:14.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.267+0000 7fab32000700 1 --2- 172.21.15.151:0/3176674883 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f77ed18f0 0x557f77ebd740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:14.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.267+0000 7fab32000700 1 -- 172.21.15.151:0/3176674883 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f77ed2ec0 con 0x557f77ed0fd0 2024-05-21T22:03:14.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.267+0000 7fab3b400700 1 --2- 172.21.15.151:0/3176674883 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f77ed18f0 0x557f77ebd740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:03:14.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.268+0000 7fab32000700 1 --2- 172.21.15.151:0/3176674883 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557f77ed0fd0 0x557f77ed4e40 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x557f77ed5f20 tx=0x7fab24016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81964fe160558a81 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:03:14.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.268+0000 7fab2b400700 1 -- 172.21.15.151:0/3176674883 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab24003c80 con 0x557f77ed0fd0 2024-05-21T22:03:14.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.268+0000 7fab2b400700 1 -- 172.21.15.151:0/3176674883 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab24011e60 con 0x557f77ed0fd0 2024-05-21T22:03:14.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.268+0000 7fab2b400700 1 -- 172.21.15.151:0/3176674883 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab24005020 con 0x557f77ed0fd0 2024-05-21T22:03:14.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.269+0000 7fab4c4a8b80 1 -- 172.21.15.151:0/3176674883 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557f77ed0fd0 msgr2=0x557f77ed4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:14.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.269+0000 7fab4c4a8b80 1 --2- 172.21.15.151:0/3176674883 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557f77ed0fd0 0x557f77ed4e40 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x557f77ed5f20 tx=0x7fab24016000 comp rx=0 tx=0).stop 2024-05-21T22:03:14.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.269+0000 7fab4c4a8b80 1 -- 172.21.15.151:0/3176674883 shutdown_connections 2024-05-21T22:03:14.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.269+0000 7fab4c4a8b80 1 --2- 172.21.15.151:0/3176674883 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f77ed18f0 0x557f77ebd740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:14.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.269+0000 7fab4c4a8b80 1 --2- 172.21.15.151:0/3176674883 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557f77ecbba0 0x557f77ecb580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:14.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.269+0000 7fab4c4a8b80 1 --2- 172.21.15.151:0/3176674883 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557f77ed0fd0 0x557f77ed4e40 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:14.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.269+0000 7fab4c4a8b80 1 -- 172.21.15.151:0/3176674883 >> 172.21.15.151:0/3176674883 conn(0x557f77df0be0 msgr2=0x557f77ed4730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:03:14.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.269+0000 7fab4c4a8b80 1 -- 172.21.15.151:0/3176674883 shutdown_connections 2024-05-21T22:03:14.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.269+0000 7fab4c4a8b80 1 -- 172.21.15.151:0/3176674883 wait complete. 2024-05-21T22:03:14.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.270+0000 7fab4c4a8b80 1 -- start start 2024-05-21T22:03:14.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.270+0000 7fab4c4a8b80 1 Processor -- start 2024-05-21T22:03:14.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.270+0000 7fab4c4a8b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557f77ecbba0 0x557f77eec3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:14.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.270+0000 7fab4c4a8b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f77ed0fd0 0x557f77eec8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:14.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.270+0000 7fab4c4a8b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557f77ed18f0 0x557f77eece30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:14.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.270+0000 7fab4c4a8b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fab3c1acf30 con 0x557f77ecbba0 2024-05-21T22:03:14.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.270+0000 7fab4c4a8b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fab3c1ac860 con 0x557f77ed0fd0 2024-05-21T22:03:14.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.270+0000 7fab4c4a8b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fab3c10e540 con 0x557f77ed18f0 2024-05-21T22:03:14.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.270+0000 7fab4c4a8b80 10 client.0 _ll_register_callbacks cb 0x557f77dbac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:03:14.275 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.271+0000 7fab3aa00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557f77ecbba0 0x557f77eec3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:14.275 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.271+0000 7fab3aa00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557f77ecbba0 0x557f77eec3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:59600/0 (socket says 172.21.15.151:59600) 2024-05-21T22:03:14.275 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.271+0000 7fab3aa00700 1 -- 172.21.15.151:0/405546332 learned_addr learned my addr 172.21.15.151:0/405546332 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:03:14.275 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.271+0000 7fab32000700 1 --2- 172.21.15.151:0/405546332 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f77ed0fd0 0x557f77eec8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:14.275 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.271+0000 7fab3b400700 1 --2- 172.21.15.151:0/405546332 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557f77ed18f0 0x557f77eece30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:14.275 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.271+0000 7fab3aa00700 1 -- 172.21.15.151:0/405546332 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557f77ed18f0 msgr2=0x557f77eece30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:14.275 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.271+0000 7fab3aa00700 1 --2- 172.21.15.151:0/405546332 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557f77ed18f0 0x557f77eece30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:14.275 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.271+0000 7fab3aa00700 1 -- 172.21.15.151:0/405546332 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f77ed0fd0 msgr2=0x557f77eec8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:14.275 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.271+0000 7fab3aa00700 1 --2- 172.21.15.151:0/405546332 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f77ed0fd0 0x557f77eec8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:14.275 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.271+0000 7fab3aa00700 1 -- 172.21.15.151:0/405546332 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f77f4e770 con 0x557f77ecbba0 2024-05-21T22:03:14.275 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.271+0000 7fab3aa00700 1 --2- 172.21.15.151:0/405546332 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557f77ecbba0 0x557f77eec3b0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fab2c007c20 tx=0x7fab2c0049a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1f10c61a9062bfc server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:03:14.275 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.271+0000 7fab28c00700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-21T22:03:14.275 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.272+0000 7fab28c00700 1 -- 172.21.15.151:0/405546332 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab2c019050 con 0x557f77ecbba0 2024-05-21T22:03:14.276 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.272+0000 7fab28c00700 1 -- 172.21.15.151:0/405546332 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7fab2c015b00 con 0x557f77ecbba0 2024-05-21T22:03:14.276 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.272+0000 7fab4c4a8b80 1 -- 172.21.15.151:0/405546332 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab24011e60 con 0x557f77ecbba0 2024-05-21T22:03:14.276 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.272+0000 7fab28c00700 1 -- 172.21.15.151:0/405546332 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fab2c022040 con 0x557f77ecbba0 2024-05-21T22:03:14.276 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.272+0000 7fab4c4a8b80 10 client.16923 fetch_fsmap learned FSMap version 29 2024-05-21T22:03:14.276 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.272+0000 7fab4c4a8b80 1 -- 172.21.15.151:0/405546332 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557f77f4fa50 con 0x557f77ecbba0 2024-05-21T22:03:14.276 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.272+0000 7fab28c00700 1 -- 172.21.15.151:0/405546332 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab24011e60 con 0x557f77ecbba0 2024-05-21T22:03:14.276 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.272+0000 7fab28c00700 1 -- 172.21.15.151:0/405546332 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fab2c014a70 con 0x557f77ecbba0 2024-05-21T22:03:14.276 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.273+0000 7fab4c4a8b80 10 client.16923 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:03:14.276 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.273+0000 7fab4c4a8b80 10 client.16923 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-05-21T22:03:14.276 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.273+0000 7fab4c4a8b80 20 client.16923 populate_metadata read hostname 'smithi151' 2024-05-21T22:03:14.276 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.273+0000 7fab4c4a8b80 1 --2- 172.21.15.151:0/405546332 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x557f77f50120 0x557f77f52540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:14.276 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.273+0000 7fab4c4a8b80 4 client.16923 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T22:03:14.276 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.273+0000 7fab4c4a8b80 1 -- 172.21.15.151:0/405546332 --> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557f77e1c570 con 0x557f77f50120 2024-05-21T22:03:14.277 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.273+0000 7fab32000700 1 --2- 172.21.15.151:0/405546332 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x557f77f50120 0x557f77f52540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:14.277 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.273+0000 7fab32000700 1 --2- 172.21.15.151:0/405546332 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x557f77f50120 0x557f77f52540 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3731d186f29f61f0 server_cookie=62d6042415fd4c08 in_seq=0 out_seq=0 2024-05-21T22:03:14.277 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.273+0000 7fab28c00700 10 client.16923 ms_handle_connect on v2:172.21.15.164:6834/2340428470 2024-05-21T22:03:14.277 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.274+0000 7fab28c00700 1 -- 172.21.15.151:0/405546332 <== mds.4 v2:172.21.15.164:6834/2340428470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557f77e1c570 con 0x557f77f50120 2024-05-21T22:03:14.277 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.274+0000 7fab28c00700 10 client.16923 handle_command_reply: tid=0 2024-05-21T22:03:14.277 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.274+0000 7fab4c4a8b80 1 client.16923 shutdown 2024-05-21T22:03:14.277 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.274+0000 7fab4c4a8b80 1 -- 172.21.15.151:0/405546332 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557f77ecbba0 msgr2=0x557f77eec3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:14.277 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.274+0000 7fab4c4a8b80 1 --2- 172.21.15.151:0/405546332 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557f77ecbba0 0x557f77eec3b0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fab2c007c20 tx=0x7fab2c0049a0 comp rx=0 tx=0).stop 2024-05-21T22:03:14.278 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.274+0000 7fab4c4a8b80 1 -- 172.21.15.151:0/405546332 shutdown_connections 2024-05-21T22:03:14.278 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.274+0000 7fab4c4a8b80 1 --2- 172.21.15.151:0/405546332 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x557f77f50120 0x557f77f52540 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:14.278 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.274+0000 7fab4c4a8b80 1 --2- 172.21.15.151:0/405546332 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557f77ed18f0 0x557f77eece30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:14.278 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.274+0000 7fab4c4a8b80 1 --2- 172.21.15.151:0/405546332 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557f77ecbba0 0x557f77eec3b0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:14.278 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.274+0000 7fab28c00700 0 client.16923 ms_handle_reset on v2:172.21.15.164:6834/2340428470 2024-05-21T22:03:14.278 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.274+0000 7fab4c4a8b80 1 --2- 172.21.15.151:0/405546332 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557f77ed0fd0 0x557f77eec8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:14.278 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.274+0000 7fab4c4a8b80 1 -- 172.21.15.151:0/405546332 >> 172.21.15.151:0/405546332 conn(0x557f77df0be0 msgr2=0x557f77ed1d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:03:14.278 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.274+0000 7fab4c4a8b80 1 -- 172.21.15.151:0/405546332 shutdown_connections 2024-05-21T22:03:14.278 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.274+0000 7fab4c4a8b80 1 -- 172.21.15.151:0/405546332 wait complete. 2024-05-21T22:03:14.278 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.274+0000 7fab4c4a8b80 20 client.16923 trim_cache size 0 max 16384 2024-05-21T22:03:14.279 INFO:teuthology.orchestra.run.smithi151.stdout:[] 2024-05-21T22:03:14.279 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.275+0000 7fab3be00700 1 -- 172.21.15.151:0/3145791038 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fab200b2a70 msgr2=0x7fab200b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:14.279 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.275+0000 7fab3be00700 1 --2- 172.21.15.151:0/3145791038 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fab200b2a70 0x7fab200b4f30 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fab3c19d260 tx=0x7fab240047c0 comp rx=0 tx=0).stop 2024-05-21T22:03:14.279 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.275+0000 7fab3be00700 1 -- 172.21.15.151:0/3145791038 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fab3c1094f0 msgr2=0x7fab3c19bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:14.279 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.275+0000 7fab3be00700 1 --2- 172.21.15.151:0/3145791038 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fab3c1094f0 0x7fab3c19bb10 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fab3400c010 tx=0x7fab340057c0 comp rx=0 tx=0).stop 2024-05-21T22:03:14.280 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.275+0000 7fab3be00700 1 -- 172.21.15.151:0/3145791038 shutdown_connections 2024-05-21T22:03:14.280 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.275+0000 7fab3be00700 1 --2- 172.21.15.151:0/3145791038 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fab200b2a70 0x7fab200b4f30 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:14.280 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.275+0000 7fab3be00700 1 --2- 172.21.15.151:0/3145791038 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab3c103460 0x7fab3c1a1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:14.280 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.275+0000 7fab3be00700 1 --2- 172.21.15.151:0/3145791038 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fab3c1094f0 0x7fab3c19bb10 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:14.280 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.275+0000 7fab3be00700 1 --2- 172.21.15.151:0/3145791038 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fab3c103e20 0x7fab3c1a1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:14.280 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.275+0000 7fab3be00700 1 -- 172.21.15.151:0/3145791038 >> 172.21.15.151:0/3145791038 conn(0x7fab3c0fef50 msgr2=0x7fab3c1087b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:03:14.280 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.275+0000 7fab3be00700 1 -- 172.21.15.151:0/3145791038 shutdown_connections 2024-05-21T22:03:14.280 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:14.275+0000 7fab3be00700 1 -- 172.21.15.151:0/3145791038 wait complete. 2024-05-21T22:03:15.057 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-21T22:03:15.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:03:15 smithi151 ceph-mon[31623]: osdmap e87: 12 total, 12 up, 12 in 2024-05-21T22:03:15.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:03:15 smithi151 ceph-mon[31623]: pgmap v236: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 255 B/s wr, 13 op/s 2024-05-21T22:03:15.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:03:15 smithi184 ceph-mon[36942]: osdmap e87: 12 total, 12 up, 12 in 2024-05-21T22:03:15.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:03:15 smithi184 ceph-mon[36942]: pgmap v236: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 255 B/s wr, 13 op/s 2024-05-21T22:03:15.501 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:03:15.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:03:15 smithi164 ceph-mon[36269]: osdmap e87: 12 total, 12 up, 12 in 2024-05-21T22:03:15.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:03:15 smithi164 ceph-mon[36269]: pgmap v236: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 255 B/s wr, 13 op/s 2024-05-21T22:03:17.590 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:03:17 smithi151 ceph-mon[31623]: pgmap v237: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-05-21T22:03:17.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:03:17 smithi184 ceph-mon[36942]: pgmap v237: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-05-21T22:03:17.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:03:17 smithi164 ceph-mon[36269]: pgmap v237: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-05-21T22:03:17.890 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.889+0000 7f139ba00700 1 -- 172.21.15.151:0/643609659 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1394106120 msgr2=0x7f1394112630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:17.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.889+0000 7f139ba00700 1 --2- 172.21.15.151:0/643609659 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1394106120 0x7f1394112630 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f138c009f90 tx=0x7f138c00b330 comp rx=0 tx=0).stop 2024-05-21T22:03:17.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.889+0000 7f139ba00700 1 -- 172.21.15.151:0/643609659 shutdown_connections 2024-05-21T22:03:17.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.889+0000 7f139ba00700 1 --2- 172.21.15.151:0/643609659 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f139410b760 0x7f139410bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:17.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.889+0000 7f139ba00700 1 --2- 172.21.15.151:0/643609659 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1394106120 0x7f1394112630 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:17.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.889+0000 7f139ba00700 1 --2- 172.21.15.151:0/643609659 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1394105760 0x7f1394105be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:17.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.889+0000 7f139ba00700 1 -- 172.21.15.151:0/643609659 >> 172.21.15.151:0/643609659 conn(0x7f1394101250 msgr2=0x7f1394103670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:03:17.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.889+0000 7f139ba00700 1 -- 172.21.15.151:0/643609659 shutdown_connections 2024-05-21T22:03:17.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.890+0000 7f139ba00700 1 -- 172.21.15.151:0/643609659 wait complete. 2024-05-21T22:03:17.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.891+0000 7f139ba00700 1 Processor -- start 2024-05-21T22:03:17.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.891+0000 7f139ba00700 1 -- start start 2024-05-21T22:03:17.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.891+0000 7f139ba00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1394105760 0x7f139419be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:17.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.891+0000 7f139ba00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1394106120 0x7f139419c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:17.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.891+0000 7f139ba00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f139410b760 0x7f13941a0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:17.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.891+0000 7f139ba00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f13941159f0 con 0x7f1394106120 2024-05-21T22:03:17.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.891+0000 7f139ba00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1394115870 con 0x7f139410b760 2024-05-21T22:03:17.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.891+0000 7f139ba00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1394115b70 con 0x7f1394105760 2024-05-21T22:03:17.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.892+0000 7f1393e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1394106120 0x7f139419c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:17.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.892+0000 7f1399000700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1394105760 0x7f139419be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:17.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.892+0000 7f1393e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1394106120 0x7f139419c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:55380/0 (socket says 172.21.15.151:55380) 2024-05-21T22:03:17.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.892+0000 7f1399a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f139410b760 0x7f13941a0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:17.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.892+0000 7f1399000700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1394105760 0x7f139419be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:44822/0 (socket says 172.21.15.151:44822) 2024-05-21T22:03:17.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.892+0000 7f1393e00700 1 -- 172.21.15.151:0/430521610 learned_addr learned my addr 172.21.15.151:0/430521610 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:03:17.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.892+0000 7f1399a00700 1 -- 172.21.15.151:0/430521610 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1394105760 msgr2=0x7f139419be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:17.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.892+0000 7f1399a00700 1 --2- 172.21.15.151:0/430521610 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1394105760 0x7f139419be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:17.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.892+0000 7f1399a00700 1 -- 172.21.15.151:0/430521610 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1394106120 msgr2=0x7f139419c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:17.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.892+0000 7f1399a00700 1 --2- 172.21.15.151:0/430521610 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1394106120 0x7f139419c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:17.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.892+0000 7f1399a00700 1 -- 172.21.15.151:0/430521610 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13941a0e50 con 0x7f139410b760 2024-05-21T22:03:17.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.892+0000 7f1399000700 1 --2- 172.21.15.151:0/430521610 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1394105760 0x7f139419be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:03:17.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.893+0000 7f1399a00700 1 --2- 172.21.15.151:0/430521610 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f139410b760 0x7f13941a0710 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f138c00b6c0 tx=0x7f138c00cc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:03:17.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.893+0000 7f1391600700 1 -- 172.21.15.151:0/430521610 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f138c021070 con 0x7f139410b760 2024-05-21T22:03:17.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.893+0000 7f139ba00700 1 -- 172.21.15.151:0/430521610 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13941a10e0 con 0x7f139410b760 2024-05-21T22:03:17.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.893+0000 7f1391600700 1 -- 172.21.15.151:0/430521610 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f138c007d60 con 0x7f139410b760 2024-05-21T22:03:17.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.893+0000 7f1391600700 1 -- 172.21.15.151:0/430521610 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f138c005040 con 0x7f139410b760 2024-05-21T22:03:17.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.893+0000 7f139ba00700 1 -- 172.21.15.151:0/430521610 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13941a89c0 con 0x7f139410b760 2024-05-21T22:03:17.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.897+0000 7f1391600700 1 -- 172.21.15.151:0/430521610 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7f138c024070 con 0x7f139410b760 2024-05-21T22:03:17.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.898+0000 7f13a633eb80 1 Processor -- start 2024-05-21T22:03:17.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.898+0000 7f13a633eb80 1 -- start start 2024-05-21T22:03:17.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.898+0000 7f13a633eb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ffe7314610 0x55ffe73149f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:17.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.898+0000 7f13a633eb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ffe7314f30 0x55ffe730eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:17.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.898+0000 7f13a633eb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ffe730f140 0x55ffe730f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:17.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.898+0000 7f13a633eb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ffe726ce40 con 0x55ffe730f140 2024-05-21T22:03:17.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.898+0000 7f13a633eb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55ffe725c9e0 con 0x55ffe7314f30 2024-05-21T22:03:17.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.898+0000 7f13a633eb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55ffe7260a20 con 0x55ffe7314610 2024-05-21T22:03:17.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.899+0000 7f1393e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ffe7314610 0x55ffe73149f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:17.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.899+0000 7f1399000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ffe7314f30 0x55ffe730eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:17.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.899+0000 7f1393e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ffe7314610 0x55ffe73149f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:44838/0 (socket says 172.21.15.151:44838) 2024-05-21T22:03:17.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.899+0000 7f1393e00700 1 -- 172.21.15.151:0/1242288427 learned_addr learned my addr 172.21.15.151:0/1242288427 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:03:17.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.899+0000 7f1393e00700 1 --2- 172.21.15.151:0/1242288427 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ffe730f140 0x55ffe730f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:17.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.899+0000 7f1391600700 1 --2- 172.21.15.151:0/430521610 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f137c0b29f0 0x7f137c0b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:17.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.899+0000 7f1393e00700 1 -- 172.21.15.151:0/1242288427 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ffe7314f30 msgr2=0x55ffe730eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:17.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.899+0000 7f1393e00700 1 --2- 172.21.15.151:0/1242288427 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ffe7314f30 0x55ffe730eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:17.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.899+0000 7f1393e00700 1 -- 172.21.15.151:0/1242288427 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ffe730f140 msgr2=0x55ffe730f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:17.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.899+0000 7f1393e00700 1 --2- 172.21.15.151:0/1242288427 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ffe730f140 0x55ffe730f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:17.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.899+0000 7f1393e00700 1 -- 172.21.15.151:0/1242288427 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffe7225420 con 0x55ffe7314610 2024-05-21T22:03:17.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.899+0000 7f1399000700 1 --2- 172.21.15.151:0/1242288427 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ffe7314f30 0x55ffe730eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:03:17.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.900+0000 7f1391600700 1 -- 172.21.15.151:0/430521610 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7f138c00f020 con 0x7f139410b760 2024-05-21T22:03:17.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.900+0000 7f1393e00700 1 --2- 172.21.15.151:0/1242288427 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ffe7314610 0x55ffe73149f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f1380009fd0 tx=0x7f1380009930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8bf49a5d9fb4d2a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:03:17.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.900+0000 7f1399a00700 1 --2- 172.21.15.151:0/430521610 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f137c0b29f0 0x7f137c0b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:17.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.900+0000 7f1386000700 1 -- 172.21.15.151:0/1242288427 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f138001f440 con 0x55ffe7314610 2024-05-21T22:03:17.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.900+0000 7f1386000700 1 -- 172.21.15.151:0/1242288427 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f138001fa90 con 0x55ffe7314610 2024-05-21T22:03:17.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.900+0000 7f1386000700 1 -- 172.21.15.151:0/1242288427 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f138000f920 con 0x55ffe7314610 2024-05-21T22:03:17.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.901+0000 7f13a633eb80 1 -- 172.21.15.151:0/1242288427 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ffe7314610 msgr2=0x55ffe73149f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:17.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.901+0000 7f1399a00700 1 --2- 172.21.15.151:0/430521610 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f137c0b29f0 0x7f137c0b4eb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f138c0a6d00 tx=0x7f138c009040 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:03:17.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.901+0000 7f13a633eb80 1 --2- 172.21.15.151:0/1242288427 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ffe7314610 0x55ffe73149f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f1380009fd0 tx=0x7f1380009930 comp rx=0 tx=0).stop 2024-05-21T22:03:17.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.901+0000 7f13a633eb80 1 -- 172.21.15.151:0/1242288427 shutdown_connections 2024-05-21T22:03:17.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.901+0000 7f13a633eb80 1 --2- 172.21.15.151:0/1242288427 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ffe730f140 0x55ffe730f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:17.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.901+0000 7f13a633eb80 1 --2- 172.21.15.151:0/1242288427 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ffe7314610 0x55ffe73149f0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:17.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.901+0000 7f13a633eb80 1 --2- 172.21.15.151:0/1242288427 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ffe7314f30 0x55ffe730eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:17.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.901+0000 7f13a633eb80 1 -- 172.21.15.151:0/1242288427 >> 172.21.15.151:0/1242288427 conn(0x55ffe723cbe0 msgr2=0x55ffe7309690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:03:17.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.901+0000 7f13a633eb80 1 -- 172.21.15.151:0/1242288427 shutdown_connections 2024-05-21T22:03:17.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.901+0000 7f13a633eb80 1 -- 172.21.15.151:0/1242288427 wait complete. 2024-05-21T22:03:17.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.902+0000 7f13a633eb80 1 -- start start 2024-05-21T22:03:17.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.902+0000 7f13a633eb80 1 Processor -- start 2024-05-21T22:03:17.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.903+0000 7f13a633eb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ffe730f140 0x55ffe731ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:17.906 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.903+0000 7f13a633eb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ffe7314610 0x55ffe7318110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:17.906 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.903+0000 7f13a633eb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ffe7314f30 0x55ffe7318650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:17.906 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.903+0000 7f13a633eb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ffe7162fa0 con 0x55ffe7314610 2024-05-21T22:03:17.906 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.903+0000 7f13a633eb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55ffe71fd760 con 0x55ffe7314f30 2024-05-21T22:03:17.906 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.903+0000 7f13a633eb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55ffe731b9e0 con 0x55ffe730f140 2024-05-21T22:03:17.906 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.903+0000 7f13a633eb80 10 client.0 _ll_register_callbacks cb 0x55ffe7206c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:03:17.906 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.903+0000 7f1399000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ffe7314610 0x55ffe7318110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:17.906 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.903+0000 7f1399000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ffe7314610 0x55ffe7318110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:55408/0 (socket says 172.21.15.151:55408) 2024-05-21T22:03:17.906 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.903+0000 7f1399000700 1 -- 172.21.15.151:0/3979842896 learned_addr learned my addr 172.21.15.151:0/3979842896 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:03:17.906 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.903+0000 7f1393e00700 1 --2- 172.21.15.151:0/3979842896 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ffe730f140 0x55ffe731ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:17.906 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.903+0000 7f1393e00700 1 --2- 172.21.15.151:0/3979842896 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ffe7314f30 0x55ffe7318650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:17.906 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.903+0000 7f1399000700 1 -- 172.21.15.151:0/3979842896 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ffe730f140 msgr2=0x55ffe731ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:17.906 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.903+0000 7f1399000700 1 --2- 172.21.15.151:0/3979842896 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ffe730f140 0x55ffe731ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:17.907 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.903+0000 7f1399000700 1 -- 172.21.15.151:0/3979842896 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ffe7314f30 msgr2=0x55ffe7318650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:17.907 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.903+0000 7f1399000700 1 --2- 172.21.15.151:0/3979842896 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ffe7314f30 0x55ffe7318650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:17.907 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.903+0000 7f1399000700 1 -- 172.21.15.151:0/3979842896 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffe71ac8d0 con 0x55ffe7314610 2024-05-21T22:03:17.907 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.903+0000 7f1393e00700 1 --2- 172.21.15.151:0/3979842896 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ffe7314f30 0x55ffe7318650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T22:03:17.907 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.904+0000 7f1399000700 1 --2- 172.21.15.151:0/3979842896 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ffe7314610 0x55ffe7318110 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f1388012d20 tx=0x7f138800f6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3da1ef3bae3b9ddd server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:03:17.907 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.904+0000 7f137aa00700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-21T22:03:17.907 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.904+0000 7f137aa00700 1 -- 172.21.15.151:0/3979842896 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f138800fc30 con 0x55ffe7314610 2024-05-21T22:03:17.907 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.904+0000 7f137aa00700 1 -- 172.21.15.151:0/3979842896 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7f138800fdd0 con 0x55ffe7314610 2024-05-21T22:03:17.907 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.904+0000 7f13a633eb80 1 -- 172.21.15.151:0/3979842896 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f138001fa90 con 0x55ffe7314610 2024-05-21T22:03:17.907 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.904+0000 7f137aa00700 1 -- 172.21.15.151:0/3979842896 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1388010430 con 0x55ffe7314610 2024-05-21T22:03:17.907 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.905+0000 7f13a633eb80 10 client.16935 fetch_fsmap learned FSMap version 29 2024-05-21T22:03:17.907 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.905+0000 7f13a633eb80 1 -- 172.21.15.151:0/3979842896 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ffe722d060 con 0x55ffe7314610 2024-05-21T22:03:17.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.905+0000 7f137aa00700 1 -- 172.21.15.151:0/3979842896 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f138001fa90 con 0x55ffe7314610 2024-05-21T22:03:17.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.905+0000 7f137aa00700 1 -- 172.21.15.151:0/3979842896 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f138801c590 con 0x55ffe7314610 2024-05-21T22:03:17.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.905+0000 7f13a633eb80 10 client.16935 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:03:17.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.905+0000 7f13a633eb80 10 client.16935 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-21T22:03:17.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.905+0000 7f13a633eb80 20 client.16935 populate_metadata read hostname 'smithi151' 2024-05-21T22:03:17.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.905+0000 7f13a633eb80 1 --2- 172.21.15.151:0/3979842896 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x55ffe7328720 0x55ffe732ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:17.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.905+0000 7f13a633eb80 4 client.16935 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T22:03:17.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.905+0000 7f13a633eb80 1 -- 172.21.15.151:0/3979842896 --> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ffe726ed90 con 0x55ffe7328720 2024-05-21T22:03:17.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.906+0000 7f1393e00700 1 --2- 172.21.15.151:0/3979842896 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x55ffe7328720 0x55ffe732ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:17.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.906+0000 7f1393e00700 1 --2- 172.21.15.151:0/3979842896 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x55ffe7328720 0x55ffe732ab40 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3b7a363a3b145cb8 server_cookie=1a5dd74aa44bf961 in_seq=0 out_seq=0 2024-05-21T22:03:17.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.906+0000 7f137aa00700 10 client.16935 ms_handle_connect on v2:172.21.15.184:6836/2988194888 2024-05-21T22:03:17.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.907+0000 7f137aa00700 1 -- 172.21.15.151:0/3979842896 <== mds.3 v2:172.21.15.184:6836/2988194888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55ffe726ed90 con 0x55ffe7328720 2024-05-21T22:03:17.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.907+0000 7f137aa00700 10 client.16935 handle_command_reply: tid=0 2024-05-21T22:03:17.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.907+0000 7f13a633eb80 1 client.16935 shutdown 2024-05-21T22:03:17.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.907+0000 7f13a633eb80 1 -- 172.21.15.151:0/3979842896 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ffe7314610 msgr2=0x55ffe7318110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:17.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.907+0000 7f13a633eb80 1 --2- 172.21.15.151:0/3979842896 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ffe7314610 0x55ffe7318110 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f1388012d20 tx=0x7f138800f6a0 comp rx=0 tx=0).stop 2024-05-21T22:03:17.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.907+0000 7f13a633eb80 1 -- 172.21.15.151:0/3979842896 shutdown_connections 2024-05-21T22:03:17.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.907+0000 7f13a633eb80 1 --2- 172.21.15.151:0/3979842896 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x55ffe7328720 0x55ffe732ab40 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:17.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.907+0000 7f13a633eb80 1 --2- 172.21.15.151:0/3979842896 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ffe7314f30 0x55ffe7318650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:17.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.907+0000 7f13a633eb80 1 --2- 172.21.15.151:0/3979842896 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ffe730f140 0x55ffe731ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:17.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.907+0000 7f13a633eb80 1 --2- 172.21.15.151:0/3979842896 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ffe7314610 0x55ffe7318110 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:17.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.908+0000 7f13a633eb80 1 -- 172.21.15.151:0/3979842896 >> 172.21.15.151:0/3979842896 conn(0x55ffe723cbe0 msgr2=0x55ffe7308d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:03:17.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.908+0000 7f137aa00700 0 client.16935 ms_handle_reset on v2:172.21.15.184:6836/2988194888 2024-05-21T22:03:17.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.908+0000 7f13a633eb80 1 -- 172.21.15.151:0/3979842896 shutdown_connections 2024-05-21T22:03:17.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.908+0000 7f13a633eb80 1 -- 172.21.15.151:0/3979842896 wait complete. 2024-05-21T22:03:17.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.908+0000 7f13a633eb80 20 client.16935 trim_cache size 0 max 16384 2024-05-21T22:03:17.925 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.925+0000 7f13a633eb80 1 Processor -- start 2024-05-21T22:03:17.925 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.925+0000 7f13a633eb80 1 -- start start 2024-05-21T22:03:17.925 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.925+0000 7f13a633eb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ffe730f140 0x55ffe730eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:17.925 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.925+0000 7f13a633eb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ffe7314610 0x55ffe7318260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:17.925 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.925+0000 7f13a633eb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ffe7314f30 0x55ffe7324a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:17.925 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.925+0000 7f13a633eb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f13941affa0 con 0x55ffe730f140 2024-05-21T22:03:17.926 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.925+0000 7f13a633eb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f13941b1ae0 con 0x55ffe7314f30 2024-05-21T22:03:17.926 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.925+0000 7f13a633eb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f13941b0670 con 0x55ffe7314610 2024-05-21T22:03:17.927 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.925+0000 7f1393e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ffe730f140 0x55ffe730eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:17.927 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.925+0000 7f1393e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ffe730f140 0x55ffe730eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:55418/0 (socket says 172.21.15.151:55418) 2024-05-21T22:03:17.927 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.925+0000 7f1393e00700 1 -- 172.21.15.151:0/3247191337 learned_addr learned my addr 172.21.15.151:0/3247191337 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:03:17.927 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.926+0000 7f1399000700 1 --2- 172.21.15.151:0/3247191337 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ffe7314610 0x55ffe7318260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:17.927 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.926+0000 7f1393e00700 1 --2- 172.21.15.151:0/3247191337 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ffe7314f30 0x55ffe7324a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:17.927 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.926+0000 7f1399000700 1 -- 172.21.15.151:0/3247191337 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ffe7314f30 msgr2=0x55ffe7324a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:17.927 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.926+0000 7f1399000700 1 --2- 172.21.15.151:0/3247191337 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ffe7314f30 0x55ffe7324a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:17.927 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.926+0000 7f1399000700 1 -- 172.21.15.151:0/3247191337 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ffe730f140 msgr2=0x55ffe730eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:17.927 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.926+0000 7f1399000700 1 --2- 172.21.15.151:0/3247191337 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ffe730f140 0x55ffe730eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:17.927 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.926+0000 7f1399000700 1 -- 172.21.15.151:0/3247191337 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffe730a6d0 con 0x55ffe7314610 2024-05-21T22:03:17.928 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.926+0000 7f1393e00700 1 --2- 172.21.15.151:0/3247191337 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ffe7314f30 0x55ffe7324a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T22:03:17.928 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.926+0000 7f1399000700 1 --2- 172.21.15.151:0/3247191337 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ffe7314610 0x55ffe7318260 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f138801ca00 tx=0x7f138801ded0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e12992e5dc339fef server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:03:17.928 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.926+0000 7f1385600700 1 -- 172.21.15.151:0/3247191337 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f138800fc30 con 0x55ffe7314610 2024-05-21T22:03:17.928 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.926+0000 7f1385600700 1 -- 172.21.15.151:0/3247191337 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f138800fdd0 con 0x55ffe7314610 2024-05-21T22:03:17.928 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.926+0000 7f1385600700 1 -- 172.21.15.151:0/3247191337 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1388003680 con 0x55ffe7314610 2024-05-21T22:03:17.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.927+0000 7f13a633eb80 1 -- 172.21.15.151:0/3247191337 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ffe7314610 msgr2=0x55ffe7318260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:17.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.927+0000 7f13a633eb80 1 --2- 172.21.15.151:0/3247191337 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ffe7314610 0x55ffe7318260 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f138801ca00 tx=0x7f138801ded0 comp rx=0 tx=0).stop 2024-05-21T22:03:17.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.927+0000 7f13a633eb80 1 -- 172.21.15.151:0/3247191337 shutdown_connections 2024-05-21T22:03:17.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.927+0000 7f13a633eb80 1 --2- 172.21.15.151:0/3247191337 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ffe7314f30 0x55ffe7324a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:17.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.927+0000 7f13a633eb80 1 --2- 172.21.15.151:0/3247191337 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ffe730f140 0x55ffe730eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:17.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.927+0000 7f13a633eb80 1 --2- 172.21.15.151:0/3247191337 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ffe7314610 0x55ffe7318260 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:17.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.927+0000 7f13a633eb80 1 -- 172.21.15.151:0/3247191337 >> 172.21.15.151:0/3247191337 conn(0x55ffe723cbe0 msgr2=0x55ffe73168d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:03:17.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.927+0000 7f13a633eb80 1 -- 172.21.15.151:0/3247191337 shutdown_connections 2024-05-21T22:03:17.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.927+0000 7f13a633eb80 1 -- 172.21.15.151:0/3247191337 wait complete. 2024-05-21T22:03:17.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.928+0000 7f13a633eb80 1 -- start start 2024-05-21T22:03:17.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.928+0000 7f13a633eb80 1 Processor -- start 2024-05-21T22:03:17.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.929+0000 7f13a633eb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ffe730f140 0x55ffe7320e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:17.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.929+0000 7f13a633eb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ffe7314610 0x55ffe7321390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:17.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.929+0000 7f13a633eb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ffe7314f30 0x55ffe732f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:17.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.929+0000 7f13a633eb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f13941b1410 con 0x55ffe7314f30 2024-05-21T22:03:17.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.929+0000 7f13a633eb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f13941b0d40 con 0x55ffe7314610 2024-05-21T22:03:17.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.929+0000 7f13a633eb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f13941adf60 con 0x55ffe730f140 2024-05-21T22:03:17.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.929+0000 7f13a633eb80 10 client.0 _ll_register_callbacks cb 0x55ffe7206c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:03:17.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.929+0000 7f1393e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ffe7314f30 0x55ffe732f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:17.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.929+0000 7f1399000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ffe7314610 0x55ffe7321390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:17.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.929+0000 7f1399000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ffe7314610 0x55ffe7321390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:51018/0 (socket says 172.21.15.151:51018) 2024-05-21T22:03:17.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.929+0000 7f1393e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ffe7314f30 0x55ffe732f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:55426/0 (socket says 172.21.15.151:55426) 2024-05-21T22:03:17.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.929+0000 7f1399000700 1 -- 172.21.15.151:0/1957635528 learned_addr learned my addr 172.21.15.151:0/1957635528 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:03:17.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.929+0000 7f1393e00700 1 --2- 172.21.15.151:0/1957635528 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ffe730f140 0x55ffe7320e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:17.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.930+0000 7f1393e00700 1 -- 172.21.15.151:0/1957635528 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ffe730f140 msgr2=0x55ffe7320e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:17.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.930+0000 7f1393e00700 1 --2- 172.21.15.151:0/1957635528 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ffe730f140 0x55ffe7320e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:17.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.930+0000 7f1393e00700 1 -- 172.21.15.151:0/1957635528 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ffe7314610 msgr2=0x55ffe7321390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:17.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.930+0000 7f1393e00700 1 --2- 172.21.15.151:0/1957635528 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ffe7314610 0x55ffe7321390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:17.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.930+0000 7f1393e00700 1 -- 172.21.15.151:0/1957635528 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffe7330070 con 0x55ffe7314f30 2024-05-21T22:03:17.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.930+0000 7f1393e00700 1 --2- 172.21.15.151:0/1957635528 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ffe7314f30 0x55ffe732f800 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f1380001eb0 tx=0x7f138000f480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df712dfc0142b487 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:03:17.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.930+0000 7f137aa00700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-21T22:03:17.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.930+0000 7f137aa00700 1 -- 172.21.15.151:0/1957635528 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1380033040 con 0x55ffe7314f30 2024-05-21T22:03:17.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.930+0000 7f137aa00700 1 -- 172.21.15.151:0/1957635528 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7f138001f900 con 0x55ffe7314f30 2024-05-21T22:03:17.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.930+0000 7f13a633eb80 1 -- 172.21.15.151:0/1957635528 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f138800fdd0 con 0x55ffe7314f30 2024-05-21T22:03:17.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.931+0000 7f137aa00700 1 -- 172.21.15.151:0/1957635528 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1380005020 con 0x55ffe7314f30 2024-05-21T22:03:17.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.931+0000 7f13a633eb80 10 client.16941 fetch_fsmap learned FSMap version 29 2024-05-21T22:03:17.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.931+0000 7f13a633eb80 1 -- 172.21.15.151:0/1957635528 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ffe7392d30 con 0x55ffe7314f30 2024-05-21T22:03:17.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.931+0000 7f137aa00700 1 -- 172.21.15.151:0/1957635528 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f138800fdd0 con 0x55ffe7314f30 2024-05-21T22:03:17.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.931+0000 7f137aa00700 1 -- 172.21.15.151:0/1957635528 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f1380039070 con 0x55ffe7314f30 2024-05-21T22:03:17.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.931+0000 7f13a633eb80 10 client.16941 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:03:17.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.931+0000 7f13a633eb80 10 client.16941 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-21T22:03:17.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.931+0000 7f13a633eb80 20 client.16941 populate_metadata read hostname 'smithi151' 2024-05-21T22:03:17.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.931+0000 7f13a633eb80 1 --2- 172.21.15.151:0/1957635528 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x55ffe7393270 0x55ffe7395690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:17.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.931+0000 7f13a633eb80 4 client.16941 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T22:03:17.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.931+0000 7f13a633eb80 1 -- 172.21.15.151:0/1957635528 --> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ffe7268570 con 0x55ffe7393270 2024-05-21T22:03:17.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.932+0000 7f1399000700 1 --2- 172.21.15.151:0/1957635528 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x55ffe7393270 0x55ffe7395690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:17.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.932+0000 7f1399000700 1 --2- 172.21.15.151:0/1957635528 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x55ffe7393270 0x55ffe7395690 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=66660c8651e9e203 server_cookie=3116f03b6d90995a in_seq=0 out_seq=0 2024-05-21T22:03:17.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.932+0000 7f137aa00700 10 client.16941 ms_handle_connect on v2:172.21.15.184:6836/2988194888 2024-05-21T22:03:17.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.932+0000 7f137aa00700 1 -- 172.21.15.151:0/1957635528 <== mds.3 v2:172.21.15.184:6836/2988194888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ffe7268570 con 0x55ffe7393270 2024-05-21T22:03:17.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.932+0000 7f137aa00700 10 client.16941 handle_command_reply: tid=0 2024-05-21T22:03:17.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.932+0000 7f13a633eb80 1 client.16941 shutdown 2024-05-21T22:03:17.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.932+0000 7f13a633eb80 1 -- 172.21.15.151:0/1957635528 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ffe7314f30 msgr2=0x55ffe732f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:17.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.932+0000 7f13a633eb80 1 --2- 172.21.15.151:0/1957635528 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ffe7314f30 0x55ffe732f800 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f1380001eb0 tx=0x7f138000f480 comp rx=0 tx=0).stop 2024-05-21T22:03:17.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.933+0000 7f13a633eb80 1 -- 172.21.15.151:0/1957635528 shutdown_connections 2024-05-21T22:03:17.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.933+0000 7f13a633eb80 1 --2- 172.21.15.151:0/1957635528 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x55ffe7393270 0x55ffe7395690 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:17.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.933+0000 7f13a633eb80 1 --2- 172.21.15.151:0/1957635528 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ffe7314f30 0x55ffe732f800 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:17.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.933+0000 7f13a633eb80 1 --2- 172.21.15.151:0/1957635528 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ffe730f140 0x55ffe7320e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:17.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.933+0000 7f13a633eb80 1 --2- 172.21.15.151:0/1957635528 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ffe7314610 0x55ffe7321390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:17.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.933+0000 7f137aa00700 0 client.16941 ms_handle_reset on v2:172.21.15.184:6836/2988194888 2024-05-21T22:03:17.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.933+0000 7f13a633eb80 1 -- 172.21.15.151:0/1957635528 >> 172.21.15.151:0/1957635528 conn(0x55ffe723cbe0 msgr2=0x55ffe7315310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:03:17.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.933+0000 7f13a633eb80 1 -- 172.21.15.151:0/1957635528 shutdown_connections 2024-05-21T22:03:17.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.933+0000 7f13a633eb80 1 -- 172.21.15.151:0/1957635528 wait complete. 2024-05-21T22:03:17.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.933+0000 7f13a633eb80 20 client.16941 trim_cache size 0 max 16384 2024-05-21T22:03:17.936 INFO:teuthology.orchestra.run.smithi151.stdout:[] 2024-05-21T22:03:17.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.934+0000 7f139ba00700 1 -- 172.21.15.151:0/430521610 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f137c0b29f0 msgr2=0x7f137c0b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:17.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.934+0000 7f139ba00700 1 --2- 172.21.15.151:0/430521610 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f137c0b29f0 0x7f137c0b4eb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f138c0a6d00 tx=0x7f138c009040 comp rx=0 tx=0).stop 2024-05-21T22:03:17.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.934+0000 7f139ba00700 1 -- 172.21.15.151:0/430521610 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f139410b760 msgr2=0x7f13941a0710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:17.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.934+0000 7f139ba00700 1 --2- 172.21.15.151:0/430521610 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f139410b760 0x7f13941a0710 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f138c00b6c0 tx=0x7f138c00cc30 comp rx=0 tx=0).stop 2024-05-21T22:03:17.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.934+0000 7f139ba00700 1 -- 172.21.15.151:0/430521610 shutdown_connections 2024-05-21T22:03:17.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.934+0000 7f139ba00700 1 --2- 172.21.15.151:0/430521610 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f137c0b29f0 0x7f137c0b4eb0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:17.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.934+0000 7f139ba00700 1 --2- 172.21.15.151:0/430521610 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f139410b760 0x7f13941a0710 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:17.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.934+0000 7f139ba00700 1 --2- 172.21.15.151:0/430521610 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1394105760 0x7f139419be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:17.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.934+0000 7f139ba00700 1 --2- 172.21.15.151:0/430521610 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1394106120 0x7f139419c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:17.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.934+0000 7f139ba00700 1 -- 172.21.15.151:0/430521610 >> 172.21.15.151:0/430521610 conn(0x7f1394101250 msgr2=0x7f1394101830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:03:17.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.934+0000 7f139ba00700 1 -- 172.21.15.151:0/430521610 shutdown_connections 2024-05-21T22:03:17.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:17.934+0000 7f139ba00700 1 -- 172.21.15.151:0/430521610 wait complete. 2024-05-21T22:03:18.894 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-21T22:03:19.327 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:03:19.685 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:03:19 smithi151 ceph-mon[31623]: pgmap v238: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 232 B/s wr, 11 op/s 2024-05-21T22:03:19.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:03:19 smithi184 ceph-mon[36942]: pgmap v238: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 232 B/s wr, 11 op/s 2024-05-21T22:03:19.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:03:19 smithi164 ceph-mon[36269]: pgmap v238: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 232 B/s wr, 11 op/s 2024-05-21T22:03:21.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.680+0000 7f6439600700 1 -- 172.21.15.151:0/2611565496 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f64341088f0 msgr2=0x7f6434110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:21.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.680+0000 7f6439600700 1 --2- 172.21.15.151:0/2611565496 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f64341088f0 0x7f6434110470 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f642c009f90 tx=0x7f642c00b330 comp rx=0 tx=0).stop 2024-05-21T22:03:21.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.680+0000 7f6439600700 1 -- 172.21.15.151:0/2611565496 shutdown_connections 2024-05-21T22:03:21.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.680+0000 7f6439600700 1 --2- 172.21.15.151:0/2611565496 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f64341088f0 0x7f6434110470 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:21.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.680+0000 7f6439600700 1 --2- 172.21.15.151:0/2611565496 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f64341037f0 0x7f6434103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:21.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.680+0000 7f6439600700 1 --2- 172.21.15.151:0/2611565496 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64341041a0 0x7f64341081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:21.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.680+0000 7f6439600700 1 -- 172.21.15.151:0/2611565496 >> 172.21.15.151:0/2611565496 conn(0x7f643407a6a0 msgr2=0x7f643407aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:03:21.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.680+0000 7f6439600700 1 -- 172.21.15.151:0/2611565496 shutdown_connections 2024-05-21T22:03:21.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.680+0000 7f6439600700 1 -- 172.21.15.151:0/2611565496 wait complete. 2024-05-21T22:03:21.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.681+0000 7f6439600700 1 Processor -- start 2024-05-21T22:03:21.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.681+0000 7f6439600700 1 -- start start 2024-05-21T22:03:21.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.682+0000 7f6439600700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64341037f0 0x7f64341a1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:21.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.682+0000 7f6439600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f64341041a0 0x7f64341a1e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:21.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.682+0000 7f6439600700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f64341088f0 0x7f643419bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:21.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.682+0000 7f6439600700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f64341136c0 con 0x7f64341041a0 2024-05-21T22:03:21.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.682+0000 7f6439600700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6434113540 con 0x7f64341037f0 2024-05-21T22:03:21.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.682+0000 7f6439600700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6434113840 con 0x7f64341088f0 2024-05-21T22:03:21.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.682+0000 7f6432a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64341037f0 0x7f64341a1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:21.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.682+0000 7f6432000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f64341041a0 0x7f64341a1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:21.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.682+0000 7f6432a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64341037f0 0x7f64341a1920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:51040/0 (socket says 172.21.15.151:51040) 2024-05-21T22:03:21.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.682+0000 7f6432a00700 1 -- 172.21.15.151:0/3183555471 learned_addr learned my addr 172.21.15.151:0/3183555471 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:03:21.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.682+0000 7f6433400700 1 --2- 172.21.15.151:0/3183555471 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f64341088f0 0x7f643419bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:21.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.683+0000 7f6432a00700 1 -- 172.21.15.151:0/3183555471 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f64341088f0 msgr2=0x7f643419bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:21.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.683+0000 7f6432a00700 1 --2- 172.21.15.151:0/3183555471 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f64341088f0 0x7f643419bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:21.685 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.683+0000 7f6432a00700 1 -- 172.21.15.151:0/3183555471 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f64341041a0 msgr2=0x7f64341a1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:21.685 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.683+0000 7f6432a00700 1 --2- 172.21.15.151:0/3183555471 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f64341041a0 0x7f64341a1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:21.685 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.683+0000 7f6432a00700 1 -- 172.21.15.151:0/3183555471 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f643419c240 con 0x7f64341037f0 2024-05-21T22:03:21.685 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.683+0000 7f6433400700 1 --2- 172.21.15.151:0/3183555471 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f64341088f0 0x7f643419bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T22:03:21.685 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.683+0000 7f6432a00700 1 --2- 172.21.15.151:0/3183555471 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64341037f0 0x7f64341a1920 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f642400e9a0 tx=0x7f642400ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:03:21.685 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.683+0000 7f642b400700 1 -- 172.21.15.151:0/3183555471 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f642400cbd0 con 0x7f64341037f0 2024-05-21T22:03:21.685 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.683+0000 7f642b400700 1 -- 172.21.15.151:0/3183555471 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f642400cd70 con 0x7f64341037f0 2024-05-21T22:03:21.685 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.683+0000 7f642b400700 1 -- 172.21.15.151:0/3183555471 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6424018680 con 0x7f64341037f0 2024-05-21T22:03:21.685 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.683+0000 7f6439600700 1 -- 172.21.15.151:0/3183555471 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f643419c530 con 0x7f64341037f0 2024-05-21T22:03:21.685 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.684+0000 7f6439600700 1 -- 172.21.15.151:0/3183555471 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64341a87f0 con 0x7f64341037f0 2024-05-21T22:03:21.687 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.687+0000 7f642b400700 1 -- 172.21.15.151:0/3183555471 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7f6424010430 con 0x7f64341037f0 2024-05-21T22:03:21.688 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.688+0000 7f6443ffab80 1 Processor -- start 2024-05-21T22:03:21.688 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.688+0000 7f6443ffab80 1 -- start start 2024-05-21T22:03:21.689 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.688+0000 7f642b400700 1 --2- 172.21.15.151:0/3183555471 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f64180b29f0 0x7f64180b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:21.689 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.689+0000 7f642b400700 1 -- 172.21.15.151:0/3183555471 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7f64240e28d0 con 0x7f64341037f0 2024-05-21T22:03:21.689 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.689+0000 7f6443ffab80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5612f6334eb0 0x5612f6335290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:21.689 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.689+0000 7f6443ffab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612f63357d0 0x5612f632f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:21.689 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.689+0000 7f6443ffab80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5612f632f880 0x5612f632fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:21.689 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.689+0000 7f6443ffab80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5612f628de40 con 0x5612f632f880 2024-05-21T22:03:21.689 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.689+0000 7f6443ffab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5612f627d9e0 con 0x5612f63357d0 2024-05-21T22:03:21.689 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.689+0000 7f6443ffab80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5612f6281a20 con 0x5612f6334eb0 2024-05-21T22:03:21.690 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.689+0000 7f6433400700 1 --2- 172.21.15.151:0/3183555471 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f64180b29f0 0x7f64180b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:21.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.689+0000 7f6432a00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5612f6334eb0 0x5612f6335290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:21.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.689+0000 7f6432000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612f63357d0 0x5612f632f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:21.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.689+0000 7f6432a00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5612f6334eb0 0x5612f6335290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:44878/0 (socket says 172.21.15.151:44878) 2024-05-21T22:03:21.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.689+0000 7f6432a00700 1 -- 172.21.15.151:0/4084047504 learned_addr learned my addr 172.21.15.151:0/4084047504 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:03:21.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.689+0000 7f6432000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612f63357d0 0x5612f632f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:51050/0 (socket says 172.21.15.151:51050) 2024-05-21T22:03:21.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.689+0000 7f6433400700 1 --2- 172.21.15.151:0/4084047504 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5612f632f880 0x5612f632fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:21.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.690+0000 7f6432000700 1 -- 172.21.15.151:0/4084047504 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5612f6334eb0 msgr2=0x5612f6335290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:21.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.690+0000 7f6432000700 1 --2- 172.21.15.151:0/4084047504 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5612f6334eb0 0x5612f6335290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:21.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.690+0000 7f6432000700 1 -- 172.21.15.151:0/4084047504 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5612f632f880 msgr2=0x5612f632fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:21.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.690+0000 7f6432000700 1 --2- 172.21.15.151:0/4084047504 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5612f632f880 0x5612f632fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:21.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.690+0000 7f6432000700 1 -- 172.21.15.151:0/4084047504 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612f6246420 con 0x5612f63357d0 2024-05-21T22:03:21.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.690+0000 7f6433400700 1 --2- 172.21.15.151:0/3183555471 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f64180b29f0 0x7f64180b4eb0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f642c009550 tx=0x7f642c003d20 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:03:21.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.690+0000 7f6432000700 1 --2- 172.21.15.151:0/4084047504 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612f63357d0 0x5612f632f260 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f6420005fd0 tx=0x7f6420011440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e8e20c90fdf04ee server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:03:21.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.690+0000 7f641fe00700 1 -- 172.21.15.151:0/4084047504 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6420011cc0 con 0x5612f63357d0 2024-05-21T22:03:21.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.690+0000 7f641fe00700 1 -- 172.21.15.151:0/4084047504 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6420004500 con 0x5612f63357d0 2024-05-21T22:03:21.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.690+0000 7f641fe00700 1 -- 172.21.15.151:0/4084047504 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6420005280 con 0x5612f63357d0 2024-05-21T22:03:21.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.691+0000 7f6443ffab80 1 -- 172.21.15.151:0/4084047504 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612f63357d0 msgr2=0x5612f632f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:21.693 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.691+0000 7f6443ffab80 1 --2- 172.21.15.151:0/4084047504 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612f63357d0 0x5612f632f260 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f6420005fd0 tx=0x7f6420011440 comp rx=0 tx=0).stop 2024-05-21T22:03:21.693 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.691+0000 7f6443ffab80 1 -- 172.21.15.151:0/4084047504 shutdown_connections 2024-05-21T22:03:21.693 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.691+0000 7f6443ffab80 1 --2- 172.21.15.151:0/4084047504 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5612f632f880 0x5612f632fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:21.693 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.691+0000 7f6443ffab80 1 --2- 172.21.15.151:0/4084047504 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5612f6334eb0 0x5612f6335290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:21.693 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.691+0000 7f6443ffab80 1 --2- 172.21.15.151:0/4084047504 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612f63357d0 0x5612f632f260 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:21.693 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.691+0000 7f6443ffab80 1 -- 172.21.15.151:0/4084047504 >> 172.21.15.151:0/4084047504 conn(0x5612f625dbe0 msgr2=0x5612f632a010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:03:21.693 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.691+0000 7f6443ffab80 1 -- 172.21.15.151:0/4084047504 shutdown_connections 2024-05-21T22:03:21.693 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.691+0000 7f6443ffab80 1 -- 172.21.15.151:0/4084047504 wait complete. 2024-05-21T22:03:21.693 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.692+0000 7f6443ffab80 1 -- start start 2024-05-21T22:03:21.693 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.692+0000 7f6443ffab80 1 Processor -- start 2024-05-21T22:03:21.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.693+0000 7f6443ffab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612f632f880 0x5612f633b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:21.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.693+0000 7f6443ffab80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5612f6334eb0 0x5612f63387e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:21.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.693+0000 7f6443ffab80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5612f63357d0 0x5612f6338d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:21.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.693+0000 7f6443ffab80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5612f6183fa0 con 0x5612f6334eb0 2024-05-21T22:03:21.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.693+0000 7f6443ffab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5612f621e760 con 0x5612f632f880 2024-05-21T22:03:21.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.693+0000 7f6443ffab80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5612f633c240 con 0x5612f63357d0 2024-05-21T22:03:21.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.693+0000 7f6443ffab80 10 client.0 _ll_register_callbacks cb 0x5612f6227c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:03:21.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.693+0000 7f6432000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5612f6334eb0 0x5612f63387e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:21.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.693+0000 7f6433400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5612f63357d0 0x5612f6338d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:21.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.693+0000 7f6432000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5612f6334eb0 0x5612f63387e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:55448/0 (socket says 172.21.15.151:55448) 2024-05-21T22:03:21.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.693+0000 7f6432000700 1 -- 172.21.15.151:0/109635616 learned_addr learned my addr 172.21.15.151:0/109635616 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:03:21.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.693+0000 7f6432a00700 1 --2- 172.21.15.151:0/109635616 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612f632f880 0x5612f633b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:21.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.694+0000 7f6433400700 1 -- 172.21.15.151:0/109635616 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612f632f880 msgr2=0x5612f633b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:21.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.694+0000 7f6433400700 1 --2- 172.21.15.151:0/109635616 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612f632f880 0x5612f633b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:21.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.694+0000 7f6433400700 1 -- 172.21.15.151:0/109635616 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5612f6334eb0 msgr2=0x5612f63387e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:21.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.694+0000 7f6433400700 1 --2- 172.21.15.151:0/109635616 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5612f6334eb0 0x5612f63387e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:21.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.694+0000 7f6433400700 1 -- 172.21.15.151:0/109635616 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612f61cd8d0 con 0x5612f63357d0 2024-05-21T22:03:21.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.694+0000 7f6432a00700 1 --2- 172.21.15.151:0/109635616 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612f632f880 0x5612f633b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:03:21.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.694+0000 7f6432000700 1 --2- 172.21.15.151:0/109635616 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5612f6334eb0 0x5612f63387e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:03:21.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.694+0000 7f6433400700 1 --2- 172.21.15.151:0/109635616 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5612f63357d0 0x5612f6338d20 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f642c006130 tx=0x7f642c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=652d6444f96ffaf9 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:03:21.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.694+0000 7f641cc00700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-21T22:03:21.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.694+0000 7f641cc00700 1 -- 172.21.15.151:0/109635616 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f642c02e030 con 0x5612f63357d0 2024-05-21T22:03:21.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.694+0000 7f641cc00700 1 -- 172.21.15.151:0/109635616 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7f642c007730 con 0x5612f63357d0 2024-05-21T22:03:21.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.694+0000 7f6443ffab80 1 -- 172.21.15.151:0/109635616 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6420004500 con 0x5612f63357d0 2024-05-21T22:03:21.698 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.695+0000 7f641cc00700 1 -- 172.21.15.151:0/109635616 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f642c007930 con 0x5612f63357d0 2024-05-21T22:03:21.698 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.695+0000 7f6443ffab80 10 client.27085 fetch_fsmap learned FSMap version 29 2024-05-21T22:03:21.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.695+0000 7f6443ffab80 1 -- 172.21.15.151:0/109635616 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5612f624e060 con 0x5612f63357d0 2024-05-21T22:03:21.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.695+0000 7f641cc00700 1 -- 172.21.15.151:0/109635616 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6420004500 con 0x5612f63357d0 2024-05-21T22:03:21.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.695+0000 7f641cc00700 1 -- 172.21.15.151:0/109635616 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f642c011070 con 0x5612f63357d0 2024-05-21T22:03:21.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.696+0000 7f6443ffab80 10 client.27085 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:03:21.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.696+0000 7f6443ffab80 10 client.27085 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-05-21T22:03:21.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.696+0000 7f6443ffab80 20 client.27085 populate_metadata read hostname 'smithi151' 2024-05-21T22:03:21.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.696+0000 7f6443ffab80 1 --2- 172.21.15.151:0/109635616 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x5612f6348df0 0x5612f634b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:21.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.696+0000 7f6443ffab80 4 client.27085 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T22:03:21.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.696+0000 7f6443ffab80 1 -- 172.21.15.151:0/109635616 --> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5612f628fd90 con 0x5612f6348df0 2024-05-21T22:03:21.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.696+0000 7f6432a00700 1 --2- 172.21.15.151:0/109635616 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x5612f6348df0 0x5612f634b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:21.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.697+0000 7f6432a00700 1 --2- 172.21.15.151:0/109635616 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x5612f6348df0 0x5612f634b220 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bf71ac1381ec89b3 server_cookie=613ac825af6ab751 in_seq=0 out_seq=0 2024-05-21T22:03:21.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.697+0000 7f641cc00700 10 client.27085 ms_handle_connect on v2:172.21.15.164:6836/1554609572 2024-05-21T22:03:21.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.697+0000 7f641cc00700 1 -- 172.21.15.151:0/109635616 <== mds.2 v2:172.21.15.164:6836/1554609572 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5612f628fd90 con 0x5612f6348df0 2024-05-21T22:03:21.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.697+0000 7f641cc00700 10 client.27085 handle_command_reply: tid=0 2024-05-21T22:03:21.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.697+0000 7f6443ffab80 1 client.27085 shutdown 2024-05-21T22:03:21.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.698+0000 7f6443ffab80 1 -- 172.21.15.151:0/109635616 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5612f63357d0 msgr2=0x5612f6338d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:21.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.698+0000 7f6443ffab80 1 --2- 172.21.15.151:0/109635616 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5612f63357d0 0x5612f6338d20 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f642c006130 tx=0x7f642c00b330 comp rx=0 tx=0).stop 2024-05-21T22:03:21.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.698+0000 7f6443ffab80 1 -- 172.21.15.151:0/109635616 shutdown_connections 2024-05-21T22:03:21.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.698+0000 7f6443ffab80 1 --2- 172.21.15.151:0/109635616 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x5612f6348df0 0x5612f634b220 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:21.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.698+0000 7f6443ffab80 1 --2- 172.21.15.151:0/109635616 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612f632f880 0x5612f633b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:21.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.698+0000 7f6443ffab80 1 --2- 172.21.15.151:0/109635616 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5612f63357d0 0x5612f6338d20 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:21.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.698+0000 7f641cc00700 0 client.27085 ms_handle_reset on v2:172.21.15.164:6836/1554609572 2024-05-21T22:03:21.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.698+0000 7f6443ffab80 1 --2- 172.21.15.151:0/109635616 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5612f6334eb0 0x5612f63387e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:21.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.698+0000 7f6443ffab80 1 -- 172.21.15.151:0/109635616 >> 172.21.15.151:0/109635616 conn(0x5612f625dbe0 msgr2=0x5612f632e250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:03:21.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.698+0000 7f6443ffab80 1 -- 172.21.15.151:0/109635616 shutdown_connections 2024-05-21T22:03:21.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.698+0000 7f6443ffab80 1 -- 172.21.15.151:0/109635616 wait complete. 2024-05-21T22:03:21.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.698+0000 7f6443ffab80 20 client.27085 trim_cache size 0 max 16384 2024-05-21T22:03:21.714 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.714+0000 7f6443ffab80 1 Processor -- start 2024-05-21T22:03:21.715 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:03:21 smithi151 ceph-mon[31623]: pgmap v239: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-21T22:03:21.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.714+0000 7f6443ffab80 1 -- start start 2024-05-21T22:03:21.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.715+0000 7f6443ffab80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5612f632f880 0x5612f632f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:21.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.715+0000 7f6443ffab80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5612f6334eb0 0x5612f6339380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:21.716 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.715+0000 7f6443ffab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612f63357d0 0x5612f63329c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:21.716 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.715+0000 7f6443ffab80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f64341affa0 con 0x5612f6334eb0 2024-05-21T22:03:21.716 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.715+0000 7f6443ffab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f64341b1ae0 con 0x5612f63357d0 2024-05-21T22:03:21.716 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.715+0000 7f6443ffab80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f64341b0670 con 0x5612f632f880 2024-05-21T22:03:21.716 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.715+0000 7f6432000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5612f6334eb0 0x5612f6339380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:21.716 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.715+0000 7f6432000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5612f6334eb0 0x5612f6339380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:55456/0 (socket says 172.21.15.151:55456) 2024-05-21T22:03:21.716 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.715+0000 7f6432a00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5612f632f880 0x5612f632f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:21.716 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.715+0000 7f6432000700 1 -- 172.21.15.151:0/2568660207 learned_addr learned my addr 172.21.15.151:0/2568660207 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:03:21.716 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.715+0000 7f6433400700 1 --2- 172.21.15.151:0/2568660207 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612f63357d0 0x5612f63329c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:21.717 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.716+0000 7f6432a00700 1 -- 172.21.15.151:0/2568660207 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612f63357d0 msgr2=0x5612f63329c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:21.717 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.716+0000 7f6432a00700 1 --2- 172.21.15.151:0/2568660207 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612f63357d0 0x5612f63329c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:21.717 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.716+0000 7f6432a00700 1 -- 172.21.15.151:0/2568660207 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5612f6334eb0 msgr2=0x5612f6339380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:21.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.716+0000 7f6432a00700 1 --2- 172.21.15.151:0/2568660207 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5612f6334eb0 0x5612f6339380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:21.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.716+0000 7f6432a00700 1 -- 172.21.15.151:0/2568660207 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612f6336d50 con 0x5612f632f880 2024-05-21T22:03:21.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.716+0000 7f6432a00700 1 --2- 172.21.15.151:0/2568660207 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5612f632f880 0x5612f632f260 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f64240a9ad0 tx=0x7f6424010cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61d2938b086073c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:03:21.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.716+0000 7f641f400700 1 -- 172.21.15.151:0/2568660207 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f642408e070 con 0x5612f632f880 2024-05-21T22:03:21.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.716+0000 7f641f400700 1 -- 172.21.15.151:0/2568660207 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64240b22e0 con 0x5612f632f880 2024-05-21T22:03:21.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.716+0000 7f641f400700 1 -- 172.21.15.151:0/2568660207 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64240b2770 con 0x5612f632f880 2024-05-21T22:03:21.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.717+0000 7f6443ffab80 1 -- 172.21.15.151:0/2568660207 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5612f632f880 msgr2=0x5612f632f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:21.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.717+0000 7f6443ffab80 1 --2- 172.21.15.151:0/2568660207 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5612f632f880 0x5612f632f260 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f64240a9ad0 tx=0x7f6424010cb0 comp rx=0 tx=0).stop 2024-05-21T22:03:21.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.717+0000 7f6443ffab80 1 -- 172.21.15.151:0/2568660207 shutdown_connections 2024-05-21T22:03:21.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.717+0000 7f6443ffab80 1 --2- 172.21.15.151:0/2568660207 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612f63357d0 0x5612f63329c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:21.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.717+0000 7f6443ffab80 1 --2- 172.21.15.151:0/2568660207 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5612f632f880 0x5612f632f260 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:21.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.717+0000 7f6443ffab80 1 --2- 172.21.15.151:0/2568660207 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5612f6334eb0 0x5612f6339380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:21.719 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.717+0000 7f6443ffab80 1 -- 172.21.15.151:0/2568660207 >> 172.21.15.151:0/2568660207 conn(0x5612f625dbe0 msgr2=0x5612f63387a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:03:21.719 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.717+0000 7f6443ffab80 1 -- 172.21.15.151:0/2568660207 shutdown_connections 2024-05-21T22:03:21.719 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.717+0000 7f6443ffab80 1 -- 172.21.15.151:0/2568660207 wait complete. 2024-05-21T22:03:21.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.718+0000 7f6443ffab80 1 -- start start 2024-05-21T22:03:21.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.718+0000 7f6443ffab80 1 Processor -- start 2024-05-21T22:03:21.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.718+0000 7f6443ffab80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5612f632f880 0x5612f6341a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:21.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.718+0000 7f6443ffab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612f6334eb0 0x5612f6350260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:21.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.718+0000 7f6443ffab80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5612f63357d0 0x5612f63507a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:21.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.718+0000 7f6443ffab80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f64341b1410 con 0x5612f63357d0 2024-05-21T22:03:21.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.718+0000 7f6443ffab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f64341b0d40 con 0x5612f6334eb0 2024-05-21T22:03:21.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.718+0000 7f6443ffab80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f64341adf60 con 0x5612f632f880 2024-05-21T22:03:21.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.718+0000 7f6443ffab80 10 client.0 _ll_register_callbacks cb 0x5612f6227c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:03:21.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.719+0000 7f6432000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612f6334eb0 0x5612f6350260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:21.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.719+0000 7f6432000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612f6334eb0 0x5612f6350260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:51086/0 (socket says 172.21.15.151:51086) 2024-05-21T22:03:21.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.719+0000 7f6432000700 1 -- 172.21.15.151:0/2261256854 learned_addr learned my addr 172.21.15.151:0/2261256854 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:03:21.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.719+0000 7f6433400700 1 --2- 172.21.15.151:0/2261256854 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5612f63357d0 0x5612f63507a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:21.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.719+0000 7f6432a00700 1 --2- 172.21.15.151:0/2261256854 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5612f632f880 0x5612f6341a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:21.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.719+0000 7f6432000700 1 -- 172.21.15.151:0/2261256854 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5612f632f880 msgr2=0x5612f6341a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:21.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.719+0000 7f6432000700 1 --2- 172.21.15.151:0/2261256854 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5612f632f880 0x5612f6341a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:21.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.719+0000 7f6432000700 1 -- 172.21.15.151:0/2261256854 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5612f63357d0 msgr2=0x5612f63507a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:21.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.719+0000 7f6432000700 1 --2- 172.21.15.151:0/2261256854 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5612f63357d0 0x5612f63507a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:21.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.719+0000 7f6432000700 1 -- 172.21.15.151:0/2261256854 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612f6350e80 con 0x5612f6334eb0 2024-05-21T22:03:21.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.719+0000 7f6432a00700 1 --2- 172.21.15.151:0/2261256854 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5612f632f880 0x5612f6341a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:03:21.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.719+0000 7f6433400700 1 --2- 172.21.15.151:0/2261256854 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5612f63357d0 0x5612f63507a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:03:21.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.720+0000 7f6432000700 1 --2- 172.21.15.151:0/2261256854 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612f6334eb0 0x5612f6350260 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f6420009c10 tx=0x7f6420012d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44881d6476643972 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:03:21.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.720+0000 7f641cc00700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-21T22:03:21.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.720+0000 7f641cc00700 1 -- 172.21.15.151:0/2261256854 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6420004260 con 0x5612f6334eb0 2024-05-21T22:03:21.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.720+0000 7f641cc00700 1 -- 172.21.15.151:0/2261256854 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7f6420004400 con 0x5612f6334eb0 2024-05-21T22:03:21.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.720+0000 7f6443ffab80 1 -- 172.21.15.151:0/2261256854 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5612f6289570 con 0x5612f6334eb0 2024-05-21T22:03:21.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.720+0000 7f641cc00700 1 -- 172.21.15.151:0/2261256854 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6420003680 con 0x5612f6334eb0 2024-05-21T22:03:21.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.720+0000 7f6443ffab80 10 client.27083 fetch_fsmap learned FSMap version 29 2024-05-21T22:03:21.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.720+0000 7f6443ffab80 1 -- 172.21.15.151:0/2261256854 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5612f63b38b0 con 0x5612f6334eb0 2024-05-21T22:03:21.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.720+0000 7f641cc00700 1 -- 172.21.15.151:0/2261256854 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5612f6289570 con 0x5612f6334eb0 2024-05-21T22:03:21.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.721+0000 7f641cc00700 1 -- 172.21.15.151:0/2261256854 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f6420018070 con 0x5612f6334eb0 2024-05-21T22:03:21.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.721+0000 7f6443ffab80 10 client.27083 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:03:21.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.721+0000 7f6443ffab80 10 client.27083 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-05-21T22:03:21.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.721+0000 7f6443ffab80 20 client.27083 populate_metadata read hostname 'smithi151' 2024-05-21T22:03:21.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.721+0000 7f6443ffab80 1 --2- 172.21.15.151:0/2261256854 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x5612f63b3d70 0x5612f63b6190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:21.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.721+0000 7f6443ffab80 4 client.27083 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T22:03:21.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.721+0000 7f6443ffab80 1 -- 172.21.15.151:0/2261256854 --> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5612f6100250 con 0x5612f63b3d70 2024-05-21T22:03:21.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.721+0000 7f6432a00700 1 --2- 172.21.15.151:0/2261256854 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x5612f63b3d70 0x5612f63b6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:21.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.722+0000 7f6432a00700 1 --2- 172.21.15.151:0/2261256854 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x5612f63b3d70 0x5612f63b6190 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7f1a6563bf854587 server_cookie=c1fa0be88662c22b in_seq=0 out_seq=0 2024-05-21T22:03:21.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.722+0000 7f641cc00700 10 client.27083 ms_handle_connect on v2:172.21.15.164:6836/1554609572 2024-05-21T22:03:21.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.722+0000 7f641cc00700 1 -- 172.21.15.151:0/2261256854 <== mds.2 v2:172.21.15.164:6836/1554609572 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5612f6100250 con 0x5612f63b3d70 2024-05-21T22:03:21.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.722+0000 7f641cc00700 10 client.27083 handle_command_reply: tid=0 2024-05-21T22:03:21.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.722+0000 7f6443ffab80 1 client.27083 shutdown 2024-05-21T22:03:21.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.722+0000 7f6443ffab80 1 -- 172.21.15.151:0/2261256854 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612f6334eb0 msgr2=0x5612f6350260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:21.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.722+0000 7f6443ffab80 1 --2- 172.21.15.151:0/2261256854 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612f6334eb0 0x5612f6350260 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f6420009c10 tx=0x7f6420012d00 comp rx=0 tx=0).stop 2024-05-21T22:03:21.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.722+0000 7f6443ffab80 1 -- 172.21.15.151:0/2261256854 shutdown_connections 2024-05-21T22:03:21.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.722+0000 7f6443ffab80 1 --2- 172.21.15.151:0/2261256854 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x5612f63b3d70 0x5612f63b6190 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:21.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.722+0000 7f6443ffab80 1 --2- 172.21.15.151:0/2261256854 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5612f63357d0 0x5612f63507a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:21.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.722+0000 7f6443ffab80 1 --2- 172.21.15.151:0/2261256854 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5612f632f880 0x5612f6341a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:21.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.722+0000 7f6443ffab80 1 --2- 172.21.15.151:0/2261256854 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5612f6334eb0 0x5612f6350260 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:21.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.722+0000 7f641cc00700 0 client.27083 ms_handle_reset on v2:172.21.15.164:6836/1554609572 2024-05-21T22:03:21.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.722+0000 7f6443ffab80 1 -- 172.21.15.151:0/2261256854 >> 172.21.15.151:0/2261256854 conn(0x5612f625dbe0 msgr2=0x5612f63329c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:03:21.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.722+0000 7f6443ffab80 1 -- 172.21.15.151:0/2261256854 shutdown_connections 2024-05-21T22:03:21.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.722+0000 7f6443ffab80 1 -- 172.21.15.151:0/2261256854 wait complete. 2024-05-21T22:03:21.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.722+0000 7f6443ffab80 20 client.27083 trim_cache size 0 max 16384 2024-05-21T22:03:21.726 INFO:teuthology.orchestra.run.smithi151.stdout:[] 2024-05-21T22:03:21.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.723+0000 7f6439600700 1 -- 172.21.15.151:0/3183555471 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f64180b29f0 msgr2=0x7f64180b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:21.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.723+0000 7f6439600700 1 --2- 172.21.15.151:0/3183555471 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f64180b29f0 0x7f64180b4eb0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f642c009550 tx=0x7f642c003d20 comp rx=0 tx=0).stop 2024-05-21T22:03:21.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.723+0000 7f6439600700 1 -- 172.21.15.151:0/3183555471 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64341037f0 msgr2=0x7f64341a1920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:21.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.723+0000 7f6439600700 1 --2- 172.21.15.151:0/3183555471 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64341037f0 0x7f64341a1920 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f642400e9a0 tx=0x7f642400ed60 comp rx=0 tx=0).stop 2024-05-21T22:03:21.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.723+0000 7f6439600700 1 -- 172.21.15.151:0/3183555471 shutdown_connections 2024-05-21T22:03:21.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.723+0000 7f6439600700 1 --2- 172.21.15.151:0/3183555471 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f64180b29f0 0x7f64180b4eb0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:21.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.723+0000 7f6439600700 1 --2- 172.21.15.151:0/3183555471 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64341037f0 0x7f64341a1920 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:21.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.723+0000 7f6439600700 1 --2- 172.21.15.151:0/3183555471 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f64341088f0 0x7f643419bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:21.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.723+0000 7f6439600700 1 --2- 172.21.15.151:0/3183555471 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f64341041a0 0x7f64341a1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:21.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.723+0000 7f6439600700 1 -- 172.21.15.151:0/3183555471 >> 172.21.15.151:0/3183555471 conn(0x7f643407a6a0 msgr2=0x7f64341019a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:03:21.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.723+0000 7f6439600700 1 -- 172.21.15.151:0/3183555471 shutdown_connections 2024-05-21T22:03:21.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:21.723+0000 7f6439600700 1 -- 172.21.15.151:0/3183555471 wait complete. 2024-05-21T22:03:21.735 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:03:21 smithi184 ceph-mon[36942]: pgmap v239: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-21T22:03:21.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:03:21 smithi164 ceph-mon[36269]: pgmap v239: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-21T22:03:22.416 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-21T22:03:22.868 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:03:23.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:03:23 smithi151 ceph-mon[31623]: pgmap v240: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-05-21T22:03:23.735 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:03:23 smithi184 ceph-mon[36942]: pgmap v240: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-05-21T22:03:23.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:03:23 smithi164 ceph-mon[36269]: pgmap v240: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-05-21T22:03:25.133 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.132+0000 7f7850600700 1 -- 172.21.15.151:0/2423502019 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f78481041a0 msgr2=0x7f78481081c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:25.134 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.132+0000 7f7850600700 1 --2- 172.21.15.151:0/2423502019 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f78481041a0 0x7f78481081c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f7834009920 tx=0x7f7834009c30 comp rx=0 tx=0).stop 2024-05-21T22:03:25.134 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.133+0000 7f7850600700 1 -- 172.21.15.151:0/2423502019 shutdown_connections 2024-05-21T22:03:25.134 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.133+0000 7f7850600700 1 --2- 172.21.15.151:0/2423502019 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f78481088f0 0x7f7848110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:25.134 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.133+0000 7f7850600700 1 --2- 172.21.15.151:0/2423502019 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78481037f0 0x7f7848103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:25.134 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.133+0000 7f7850600700 1 --2- 172.21.15.151:0/2423502019 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f78481041a0 0x7f78481081c0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:25.134 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.133+0000 7f7850600700 1 -- 172.21.15.151:0/2423502019 >> 172.21.15.151:0/2423502019 conn(0x7f784807a6a0 msgr2=0x7f784807aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:03:25.134 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.133+0000 7f7850600700 1 -- 172.21.15.151:0/2423502019 shutdown_connections 2024-05-21T22:03:25.134 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.133+0000 7f7850600700 1 -- 172.21.15.151:0/2423502019 wait complete. 2024-05-21T22:03:25.135 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.134+0000 7f7850600700 1 Processor -- start 2024-05-21T22:03:25.135 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.134+0000 7f7850600700 1 -- start start 2024-05-21T22:03:25.135 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.135+0000 7f7850600700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f78481037f0 0x7f784819beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:25.136 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.135+0000 7f7850600700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78481041a0 0x7f784819c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:25.136 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.135+0000 7f7850600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f78481088f0 0x7f78481a0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:25.136 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.135+0000 7f7850600700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7848113720 con 0x7f78481088f0 2024-05-21T22:03:25.136 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.135+0000 7f7850600700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f78481135a0 con 0x7f78481037f0 2024-05-21T22:03:25.136 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.135+0000 7f7850600700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f78481138a0 con 0x7f78481041a0 2024-05-21T22:03:25.136 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.135+0000 7f784e600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f78481088f0 0x7f78481a0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:25.136 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.135+0000 7f784e600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f78481088f0 0x7f78481a0780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:55480/0 (socket says 172.21.15.151:55480) 2024-05-21T22:03:25.136 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.135+0000 7f784e600700 1 -- 172.21.15.151:0/3425558542 learned_addr learned my addr 172.21.15.151:0/3425558542 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:03:25.136 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.135+0000 7f784dc00700 1 --2- 172.21.15.151:0/3425558542 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f78481037f0 0x7f784819beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:25.136 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.135+0000 7f784d200700 1 --2- 172.21.15.151:0/3425558542 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78481041a0 0x7f784819c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:25.137 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.136+0000 7f784e600700 1 -- 172.21.15.151:0/3425558542 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78481041a0 msgr2=0x7f784819c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:25.137 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.136+0000 7f784e600700 1 --2- 172.21.15.151:0/3425558542 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78481041a0 0x7f784819c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:25.137 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.136+0000 7f784e600700 1 -- 172.21.15.151:0/3425558542 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f78481037f0 msgr2=0x7f784819beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:25.137 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.136+0000 7f784e600700 1 --2- 172.21.15.151:0/3425558542 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f78481037f0 0x7f784819beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:25.138 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.136+0000 7f784e600700 1 -- 172.21.15.151:0/3425558542 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78481a0ec0 con 0x7f78481088f0 2024-05-21T22:03:25.138 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.136+0000 7f784d200700 1 --2- 172.21.15.151:0/3425558542 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78481041a0 0x7f784819c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T22:03:25.138 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.136+0000 7f784dc00700 1 --2- 172.21.15.151:0/3425558542 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f78481037f0 0x7f784819beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:03:25.138 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.136+0000 7f784e600700 1 --2- 172.21.15.151:0/3425558542 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f78481088f0 0x7f78481a0780 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f784000c110 tx=0x7f784000c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:03:25.138 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.136+0000 7f7846000700 1 -- 172.21.15.151:0/3425558542 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78400092e0 con 0x7f78481088f0 2024-05-21T22:03:25.138 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.136+0000 7f7850600700 1 -- 172.21.15.151:0/3425558542 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78481a1150 con 0x7f78481088f0 2024-05-21T22:03:25.138 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.136+0000 7f7846000700 1 -- 172.21.15.151:0/3425558542 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7840004bb0 con 0x7f78481088f0 2024-05-21T22:03:25.138 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.137+0000 7f7846000700 1 -- 172.21.15.151:0/3425558542 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7840007ce0 con 0x7f78481088f0 2024-05-21T22:03:25.138 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.137+0000 7f7850600700 1 -- 172.21.15.151:0/3425558542 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78481a8a30 con 0x7f78481088f0 2024-05-21T22:03:25.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.140+0000 7f7846000700 1 -- 172.21.15.151:0/3425558542 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7f784001a020 con 0x7f78481088f0 2024-05-21T22:03:25.142 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.142+0000 7f785aedcb80 1 Processor -- start 2024-05-21T22:03:25.142 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.142+0000 7f785aedcb80 1 -- start start 2024-05-21T22:03:25.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.142+0000 7f785aedcb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfa4a6eeb0 0x55bfa4a6f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:25.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.142+0000 7f785aedcb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bfa4a6f7d0 0x55bfa4a69260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:25.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.142+0000 7f785aedcb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bfa4a69880 0x55bfa4a69ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:25.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.142+0000 7f785aedcb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55bfa49c7e40 con 0x55bfa4a6f7d0 2024-05-21T22:03:25.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.142+0000 7f785aedcb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55bfa49b79e0 con 0x55bfa4a69880 2024-05-21T22:03:25.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.142+0000 7f785aedcb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55bfa49bba20 con 0x55bfa4a6eeb0 2024-05-21T22:03:25.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.143+0000 7f7846000700 1 --2- 172.21.15.151:0/3425558542 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f78300b29f0 0x7f78300b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:25.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.143+0000 7f784dc00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bfa4a6f7d0 0x55bfa4a69260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:25.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.143+0000 7f784d200700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfa4a6eeb0 0x55bfa4a6f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:25.145 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.143+0000 7f784d200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bfa4a69880 0x55bfa4a69ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:25.145 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.143+0000 7f784d200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bfa4a69880 0x55bfa4a69ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:51114/0 (socket says 172.21.15.151:51114) 2024-05-21T22:03:25.145 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.143+0000 7f784dc00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bfa4a6f7d0 0x55bfa4a69260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:55496/0 (socket says 172.21.15.151:55496) 2024-05-21T22:03:25.145 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.143+0000 7f784d200700 1 -- 172.21.15.151:0/1241479784 learned_addr learned my addr 172.21.15.151:0/1241479784 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:03:25.145 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.143+0000 7f7846000700 1 -- 172.21.15.151:0/3425558542 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7f78400e21f0 con 0x7f78481088f0 2024-05-21T22:03:25.145 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.143+0000 7f784e600700 1 --2- 172.21.15.151:0/3425558542 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f78300b29f0 0x7f78300b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:25.145 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.143+0000 7f784d200700 1 -- 172.21.15.151:0/1241479784 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfa4a6eeb0 msgr2=0x55bfa4a6f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:25.145 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.143+0000 7f784d200700 1 --2- 172.21.15.151:0/1241479784 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfa4a6eeb0 0x55bfa4a6f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:25.145 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.143+0000 7f784d200700 1 -- 172.21.15.151:0/1241479784 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bfa4a6f7d0 msgr2=0x55bfa4a69260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:25.145 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.143+0000 7f784d200700 1 --2- 172.21.15.151:0/1241479784 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bfa4a6f7d0 0x55bfa4a69260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:25.145 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.143+0000 7f784d200700 1 -- 172.21.15.151:0/1241479784 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfa4980420 con 0x55bfa4a69880 2024-05-21T22:03:25.145 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.144+0000 7f784d200700 1 --2- 172.21.15.151:0/1241479784 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bfa4a69880 0x55bfa4a69ce0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f7834009920 tx=0x7f7834009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6e2fd94e5634472 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:03:25.145 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.144+0000 7f784e600700 1 --2- 172.21.15.151:0/3425558542 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f78300b29f0 0x7f78300b4eb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f78400a9160 tx=0x7f7840006240 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:03:25.146 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.144+0000 7f783aa00700 1 -- 172.21.15.151:0/1241479784 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f783400fbc0 con 0x55bfa4a69880 2024-05-21T22:03:25.146 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.144+0000 7f783aa00700 1 -- 172.21.15.151:0/1241479784 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f783400fd60 con 0x55bfa4a69880 2024-05-21T22:03:25.146 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.144+0000 7f783aa00700 1 -- 172.21.15.151:0/1241479784 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7834004930 con 0x55bfa4a69880 2024-05-21T22:03:25.147 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.145+0000 7f785aedcb80 1 -- 172.21.15.151:0/1241479784 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bfa4a69880 msgr2=0x55bfa4a69ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:25.147 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.145+0000 7f785aedcb80 1 --2- 172.21.15.151:0/1241479784 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bfa4a69880 0x55bfa4a69ce0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f7834009920 tx=0x7f7834009260 comp rx=0 tx=0).stop 2024-05-21T22:03:25.147 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.145+0000 7f785aedcb80 1 -- 172.21.15.151:0/1241479784 shutdown_connections 2024-05-21T22:03:25.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.145+0000 7f785aedcb80 1 --2- 172.21.15.151:0/1241479784 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bfa4a69880 0x55bfa4a69ce0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:25.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.145+0000 7f785aedcb80 1 --2- 172.21.15.151:0/1241479784 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfa4a6eeb0 0x55bfa4a6f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:25.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.145+0000 7f785aedcb80 1 --2- 172.21.15.151:0/1241479784 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bfa4a6f7d0 0x55bfa4a69260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:25.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.145+0000 7f785aedcb80 1 -- 172.21.15.151:0/1241479784 >> 172.21.15.151:0/1241479784 conn(0x55bfa4997be0 msgr2=0x55bfa4a64010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:03:25.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.145+0000 7f785aedcb80 1 -- 172.21.15.151:0/1241479784 shutdown_connections 2024-05-21T22:03:25.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.145+0000 7f785aedcb80 1 -- 172.21.15.151:0/1241479784 wait complete. 2024-05-21T22:03:25.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.146+0000 7f785aedcb80 1 -- start start 2024-05-21T22:03:25.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.146+0000 7f785aedcb80 1 Processor -- start 2024-05-21T22:03:25.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.147+0000 7f785aedcb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bfa4a69880 0x55bfa4a75440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:25.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.147+0000 7f785aedcb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfa4a6eeb0 0x55bfa4a72830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:25.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.147+0000 7f785aedcb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bfa4a6f7d0 0x55bfa4a72d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:25.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.147+0000 7f785aedcb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55bfa48bdfa0 con 0x55bfa4a6f7d0 2024-05-21T22:03:25.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.147+0000 7f785aedcb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55bfa4958760 con 0x55bfa4a69880 2024-05-21T22:03:25.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.147+0000 7f785aedcb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55bfa4a761e0 con 0x55bfa4a6eeb0 2024-05-21T22:03:25.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.147+0000 7f785aedcb80 10 client.0 _ll_register_callbacks cb 0x55bfa4961c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:03:25.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.147+0000 7f784d200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bfa4a6f7d0 0x55bfa4a72d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:25.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.147+0000 7f784dc00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfa4a6eeb0 0x55bfa4a72830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:25.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.147+0000 7f784d200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bfa4a6f7d0 0x55bfa4a72d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:55498/0 (socket says 172.21.15.151:55498) 2024-05-21T22:03:25.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.147+0000 7f784d200700 1 -- 172.21.15.151:0/1405116431 learned_addr learned my addr 172.21.15.151:0/1405116431 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:03:25.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.147+0000 7f784d200700 1 --2- 172.21.15.151:0/1405116431 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bfa4a69880 0x55bfa4a75440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:25.150 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.148+0000 7f784d200700 1 -- 172.21.15.151:0/1405116431 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfa4a6eeb0 msgr2=0x55bfa4a72830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:25.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.148+0000 7f784d200700 1 --2- 172.21.15.151:0/1405116431 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfa4a6eeb0 0x55bfa4a72830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:25.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.148+0000 7f784d200700 1 -- 172.21.15.151:0/1405116431 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bfa4a69880 msgr2=0x55bfa4a75440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:25.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.148+0000 7f784d200700 1 --2- 172.21.15.151:0/1405116431 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bfa4a69880 0x55bfa4a75440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:25.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.148+0000 7f784d200700 1 -- 172.21.15.151:0/1405116431 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfa49078d0 con 0x55bfa4a6f7d0 2024-05-21T22:03:25.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.148+0000 7f784d200700 1 --2- 172.21.15.151:0/1405116431 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bfa4a6f7d0 0x55bfa4a72d70 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f78340098f0 tx=0x7f78340054b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=299008f9b83af28d server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:03:25.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.148+0000 7f782f400700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-21T22:03:25.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.148+0000 7f782f400700 1 -- 172.21.15.151:0/1405116431 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f783400f5f0 con 0x55bfa4a6f7d0 2024-05-21T22:03:25.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.148+0000 7f782f400700 1 -- 172.21.15.151:0/1405116431 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7f783401d070 con 0x55bfa4a6f7d0 2024-05-21T22:03:25.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.148+0000 7f785aedcb80 1 -- 172.21.15.151:0/1405116431 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f783400fd60 con 0x55bfa4a6f7d0 2024-05-21T22:03:25.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.149+0000 7f782f400700 1 -- 172.21.15.151:0/1405116431 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f783403d050 con 0x55bfa4a6f7d0 2024-05-21T22:03:25.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.149+0000 7f785aedcb80 10 client.16971 fetch_fsmap learned FSMap version 29 2024-05-21T22:03:25.152 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.149+0000 7f785aedcb80 1 -- 172.21.15.151:0/1405116431 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bfa4988060 con 0x55bfa4a6f7d0 2024-05-21T22:03:25.152 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.149+0000 7f782f400700 1 -- 172.21.15.151:0/1405116431 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f783400fd60 con 0x55bfa4a6f7d0 2024-05-21T22:03:25.152 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.149+0000 7f782f400700 1 -- 172.21.15.151:0/1405116431 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f783400bc40 con 0x55bfa4a6f7d0 2024-05-21T22:03:25.152 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.149+0000 7f785aedcb80 10 client.16971 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:03:25.152 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.149+0000 7f785aedcb80 10 client.16971 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-21T22:03:25.152 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.149+0000 7f785aedcb80 20 client.16971 populate_metadata read hostname 'smithi151' 2024-05-21T22:03:25.152 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.149+0000 7f785aedcb80 1 --2- 172.21.15.151:0/1405116431 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x55bfa4a82b00 0x55bfa4a84ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:25.152 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.149+0000 7f785aedcb80 4 client.16971 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T22:03:25.152 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.149+0000 7f785aedcb80 1 -- 172.21.15.151:0/1405116431 --> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bfa49c9d90 con 0x55bfa4a82b00 2024-05-21T22:03:25.152 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.150+0000 7f784dc00700 1 --2- 172.21.15.151:0/1405116431 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x55bfa4a82b00 0x55bfa4a84ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:25.152 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.150+0000 7f784dc00700 1 --2- 172.21.15.151:0/1405116431 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x55bfa4a82b00 0x55bfa4a84ef0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a75dda2267c36e57 server_cookie=c5860042aae175d2 in_seq=0 out_seq=0 2024-05-21T22:03:25.152 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.150+0000 7f782f400700 10 client.16971 ms_handle_connect on v2:172.21.15.184:6838/3059935143 2024-05-21T22:03:25.153 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.151+0000 7f782f400700 1 -- 172.21.15.151:0/1405116431 <== mds.1 v2:172.21.15.184:6838/3059935143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55bfa49c9d90 con 0x55bfa4a82b00 2024-05-21T22:03:25.153 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.151+0000 7f782f400700 10 client.16971 handle_command_reply: tid=0 2024-05-21T22:03:25.153 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.151+0000 7f785aedcb80 1 client.16971 shutdown 2024-05-21T22:03:25.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.151+0000 7f785aedcb80 1 -- 172.21.15.151:0/1405116431 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bfa4a6f7d0 msgr2=0x55bfa4a72d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:25.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.151+0000 7f785aedcb80 1 --2- 172.21.15.151:0/1405116431 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bfa4a6f7d0 0x55bfa4a72d70 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f78340098f0 tx=0x7f78340054b0 comp rx=0 tx=0).stop 2024-05-21T22:03:25.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.152+0000 7f785aedcb80 1 -- 172.21.15.151:0/1405116431 shutdown_connections 2024-05-21T22:03:25.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.152+0000 7f785aedcb80 1 --2- 172.21.15.151:0/1405116431 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x55bfa4a82b00 0x55bfa4a84ef0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:25.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.152+0000 7f785aedcb80 1 --2- 172.21.15.151:0/1405116431 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bfa4a69880 0x55bfa4a75440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:25.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.152+0000 7f785aedcb80 1 --2- 172.21.15.151:0/1405116431 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bfa4a6f7d0 0x55bfa4a72d70 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:25.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.152+0000 7f785aedcb80 1 --2- 172.21.15.151:0/1405116431 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfa4a6eeb0 0x55bfa4a72830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:25.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.152+0000 7f785aedcb80 1 -- 172.21.15.151:0/1405116431 >> 172.21.15.151:0/1405116431 conn(0x55bfa4997be0 msgr2=0x55bfa4a68250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:03:25.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.152+0000 7f782f400700 0 client.16971 ms_handle_reset on v2:172.21.15.184:6838/3059935143 2024-05-21T22:03:25.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.152+0000 7f785aedcb80 1 -- 172.21.15.151:0/1405116431 shutdown_connections 2024-05-21T22:03:25.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.152+0000 7f785aedcb80 1 -- 172.21.15.151:0/1405116431 wait complete. 2024-05-21T22:03:25.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.152+0000 7f785aedcb80 20 client.16971 trim_cache size 0 max 16384 2024-05-21T22:03:25.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.168+0000 7f785aedcb80 1 Processor -- start 2024-05-21T22:03:25.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.169+0000 7f785aedcb80 1 -- start start 2024-05-21T22:03:25.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.169+0000 7f785aedcb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfa4a69880 0x55bfa4a69260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:25.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.169+0000 7f785aedcb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bfa4a6eeb0 0x55bfa4a639e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:25.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.169+0000 7f785aedcb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bfa4a6f7d0 0x55bfa4a63f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:25.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.169+0000 7f785aedcb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f78481b0010 con 0x55bfa4a6eeb0 2024-05-21T22:03:25.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.169+0000 7f785aedcb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f78481b1b50 con 0x55bfa4a6f7d0 2024-05-21T22:03:25.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.169+0000 7f785aedcb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f78481b06e0 con 0x55bfa4a69880 2024-05-21T22:03:25.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.169+0000 7f784dc00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bfa4a6eeb0 0x55bfa4a639e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:25.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.169+0000 7f784dc00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bfa4a6eeb0 0x55bfa4a639e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:55514/0 (socket says 172.21.15.151:55514) 2024-05-21T22:03:25.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.169+0000 7f784dc00700 1 -- 172.21.15.151:0/72280532 learned_addr learned my addr 172.21.15.151:0/72280532 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:03:25.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.169+0000 7f784d200700 1 --2- 172.21.15.151:0/72280532 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfa4a69880 0x55bfa4a69260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:25.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.169+0000 7f784d200700 1 --2- 172.21.15.151:0/72280532 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bfa4a6f7d0 0x55bfa4a63f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:25.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.170+0000 7f784dc00700 1 -- 172.21.15.151:0/72280532 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfa4a69880 msgr2=0x55bfa4a69260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:25.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.170+0000 7f784dc00700 1 --2- 172.21.15.151:0/72280532 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfa4a69880 0x55bfa4a69260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:25.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.170+0000 7f784dc00700 1 -- 172.21.15.151:0/72280532 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bfa4a6f7d0 msgr2=0x55bfa4a63f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:25.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.170+0000 7f784dc00700 1 --2- 172.21.15.151:0/72280532 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bfa4a6f7d0 0x55bfa4a63f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:25.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.170+0000 7f784dc00700 1 -- 172.21.15.151:0/72280532 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfa4a64d30 con 0x55bfa4a6eeb0 2024-05-21T22:03:25.172 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.170+0000 7f784d200700 1 --2- 172.21.15.151:0/72280532 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bfa4a6f7d0 0x55bfa4a63f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T22:03:25.172 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.170+0000 7f784dc00700 1 --2- 172.21.15.151:0/72280532 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bfa4a6eeb0 0x55bfa4a639e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f783c012360 tx=0x7f783c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4a3b53d5d14574b server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:03:25.172 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.170+0000 7f783a000700 1 -- 172.21.15.151:0/72280532 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f783c024070 con 0x55bfa4a6eeb0 2024-05-21T22:03:25.172 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.170+0000 7f783a000700 1 -- 172.21.15.151:0/72280532 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f783c004500 con 0x55bfa4a6eeb0 2024-05-21T22:03:25.172 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.170+0000 7f783a000700 1 -- 172.21.15.151:0/72280532 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f783c005020 con 0x55bfa4a6eeb0 2024-05-21T22:03:25.174 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.171+0000 7f785aedcb80 1 -- 172.21.15.151:0/72280532 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bfa4a6eeb0 msgr2=0x55bfa4a639e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:25.174 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.171+0000 7f785aedcb80 1 --2- 172.21.15.151:0/72280532 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bfa4a6eeb0 0x55bfa4a639e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f783c012360 tx=0x7f783c009260 comp rx=0 tx=0).stop 2024-05-21T22:03:25.174 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.171+0000 7f785aedcb80 1 -- 172.21.15.151:0/72280532 shutdown_connections 2024-05-21T22:03:25.174 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.171+0000 7f785aedcb80 1 --2- 172.21.15.151:0/72280532 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bfa4a6f7d0 0x55bfa4a63f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:25.174 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.171+0000 7f785aedcb80 1 --2- 172.21.15.151:0/72280532 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfa4a69880 0x55bfa4a69260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:25.174 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.171+0000 7f785aedcb80 1 --2- 172.21.15.151:0/72280532 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bfa4a6eeb0 0x55bfa4a639e0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:25.174 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.171+0000 7f785aedcb80 1 -- 172.21.15.151:0/72280532 >> 172.21.15.151:0/72280532 conn(0x55bfa4997be0 msgr2=0x55bfa4a6c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:03:25.174 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.171+0000 7f785aedcb80 1 -- 172.21.15.151:0/72280532 shutdown_connections 2024-05-21T22:03:25.174 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.171+0000 7f785aedcb80 1 -- 172.21.15.151:0/72280532 wait complete. 2024-05-21T22:03:25.174 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.172+0000 7f785aedcb80 1 -- start start 2024-05-21T22:03:25.174 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.172+0000 7f785aedcb80 1 Processor -- start 2024-05-21T22:03:25.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.172+0000 7f785aedcb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bfa4a69880 0x55bfa4a852b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:25.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.172+0000 7f785aedcb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfa4a6eeb0 0x55bfa4a857f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:25.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.172+0000 7f785aedcb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bfa4a6f7d0 0x55bfa4aedb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:25.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.173+0000 7f785aedcb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f78481b1480 con 0x55bfa4a69880 2024-05-21T22:03:25.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.173+0000 7f785aedcb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f78481b0db0 con 0x55bfa4a6f7d0 2024-05-21T22:03:25.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.173+0000 7f785aedcb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f78481adfd0 con 0x55bfa4a6eeb0 2024-05-21T22:03:25.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.173+0000 7f785aedcb80 10 client.0 _ll_register_callbacks cb 0x55bfa4961c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:03:25.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.173+0000 7f784dc00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfa4a6eeb0 0x55bfa4a857f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:25.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.173+0000 7f784dc00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfa4a6eeb0 0x55bfa4a857f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:44966/0 (socket says 172.21.15.151:44966) 2024-05-21T22:03:25.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.173+0000 7f784dc00700 1 -- 172.21.15.151:0/245318203 learned_addr learned my addr 172.21.15.151:0/245318203 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:03:25.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.173+0000 7f784d200700 1 --2- 172.21.15.151:0/245318203 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bfa4a69880 0x55bfa4a852b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:25.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.173+0000 7f784d200700 1 --2- 172.21.15.151:0/245318203 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bfa4a6f7d0 0x55bfa4aedb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:25.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.173+0000 7f784dc00700 1 -- 172.21.15.151:0/245318203 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bfa4a6f7d0 msgr2=0x55bfa4aedb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:25.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.173+0000 7f784dc00700 1 --2- 172.21.15.151:0/245318203 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bfa4a6f7d0 0x55bfa4aedb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:25.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.173+0000 7f784dc00700 1 -- 172.21.15.151:0/245318203 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bfa4a69880 msgr2=0x55bfa4a852b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:25.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.173+0000 7f784dc00700 1 --2- 172.21.15.151:0/245318203 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bfa4a69880 0x55bfa4a852b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:25.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.173+0000 7f784dc00700 1 -- 172.21.15.151:0/245318203 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfa4aee180 con 0x55bfa4a6eeb0 2024-05-21T22:03:25.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.174+0000 7f784dc00700 1 --2- 172.21.15.151:0/245318203 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfa4a6eeb0 0x55bfa4a857f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f783c005920 tx=0x7f783c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b280cafb011be1d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:03:25.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.174+0000 7f782f400700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-21T22:03:25.178 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.174+0000 7f782f400700 1 -- 172.21.15.151:0/245318203 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f783c024040 con 0x55bfa4a6eeb0 2024-05-21T22:03:25.178 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.174+0000 7f785aedcb80 1 -- 172.21.15.151:0/245318203 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f783c004500 con 0x55bfa4a6eeb0 2024-05-21T22:03:25.178 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.174+0000 7f782f400700 1 -- 172.21.15.151:0/245318203 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7f783c012b20 con 0x55bfa4a6eeb0 2024-05-21T22:03:25.178 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.174+0000 7f782f400700 1 -- 172.21.15.151:0/245318203 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f783c012d20 con 0x55bfa4a6eeb0 2024-05-21T22:03:25.178 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.174+0000 7f785aedcb80 10 client.27109 fetch_fsmap learned FSMap version 29 2024-05-21T22:03:25.178 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.174+0000 7f785aedcb80 1 -- 172.21.15.151:0/245318203 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bfa4af5500 con 0x55bfa4a6eeb0 2024-05-21T22:03:25.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.175+0000 7f782f400700 1 -- 172.21.15.151:0/245318203 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f783c004500 con 0x55bfa4a6eeb0 2024-05-21T22:03:25.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.175+0000 7f782f400700 1 -- 172.21.15.151:0/245318203 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f783c01f7d0 con 0x55bfa4a6eeb0 2024-05-21T22:03:25.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.175+0000 7f785aedcb80 10 client.27109 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:03:25.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.175+0000 7f785aedcb80 10 client.27109 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-21T22:03:25.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.175+0000 7f785aedcb80 20 client.27109 populate_metadata read hostname 'smithi151' 2024-05-21T22:03:25.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.175+0000 7f785aedcb80 1 --2- 172.21.15.151:0/245318203 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x55bfa4af5bc0 0x55bfa4af7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:25.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.175+0000 7f785aedcb80 4 client.27109 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T22:03:25.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.175+0000 7f785aedcb80 1 -- 172.21.15.151:0/245318203 --> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bfa49c3570 con 0x55bfa4af5bc0 2024-05-21T22:03:25.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.175+0000 7f784d200700 1 --2- 172.21.15.151:0/245318203 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x55bfa4af5bc0 0x55bfa4af7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:25.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.176+0000 7f784d200700 1 --2- 172.21.15.151:0/245318203 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x55bfa4af5bc0 0x55bfa4af7fe0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=efcd64db6bfebed server_cookie=8f22b71d23bf23bf in_seq=0 out_seq=0 2024-05-21T22:03:25.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.176+0000 7f782f400700 10 client.27109 ms_handle_connect on v2:172.21.15.184:6838/3059935143 2024-05-21T22:03:25.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.176+0000 7f782f400700 1 -- 172.21.15.151:0/245318203 <== mds.1 v2:172.21.15.184:6838/3059935143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bfa49c3570 con 0x55bfa4af5bc0 2024-05-21T22:03:25.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.176+0000 7f782f400700 10 client.27109 handle_command_reply: tid=0 2024-05-21T22:03:25.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.176+0000 7f785aedcb80 1 client.27109 shutdown 2024-05-21T22:03:25.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.176+0000 7f785aedcb80 1 -- 172.21.15.151:0/245318203 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfa4a6eeb0 msgr2=0x55bfa4a857f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:25.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.176+0000 7f785aedcb80 1 --2- 172.21.15.151:0/245318203 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfa4a6eeb0 0x55bfa4a857f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f783c005920 tx=0x7f783c004750 comp rx=0 tx=0).stop 2024-05-21T22:03:25.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.176+0000 7f785aedcb80 1 -- 172.21.15.151:0/245318203 shutdown_connections 2024-05-21T22:03:25.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.176+0000 7f785aedcb80 1 --2- 172.21.15.151:0/245318203 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x55bfa4af5bc0 0x55bfa4af7fe0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:25.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.176+0000 7f785aedcb80 1 --2- 172.21.15.151:0/245318203 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55bfa4a6f7d0 0x55bfa4aedb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:25.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.176+0000 7f785aedcb80 1 --2- 172.21.15.151:0/245318203 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bfa4a69880 0x55bfa4a852b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:25.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.176+0000 7f782f400700 0 client.27109 ms_handle_reset on v2:172.21.15.184:6838/3059935143 2024-05-21T22:03:25.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.176+0000 7f785aedcb80 1 --2- 172.21.15.151:0/245318203 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bfa4a6eeb0 0x55bfa4a857f0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:25.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.176+0000 7f785aedcb80 1 -- 172.21.15.151:0/245318203 >> 172.21.15.151:0/245318203 conn(0x55bfa4997be0 msgr2=0x55bfa4a639e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:03:25.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.177+0000 7f785aedcb80 1 -- 172.21.15.151:0/245318203 shutdown_connections 2024-05-21T22:03:25.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.177+0000 7f785aedcb80 1 -- 172.21.15.151:0/245318203 wait complete. 2024-05-21T22:03:25.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.177+0000 7f785aedcb80 20 client.27109 trim_cache size 0 max 16384 2024-05-21T22:03:25.181 INFO:teuthology.orchestra.run.smithi151.stdout:[] 2024-05-21T22:03:25.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.177+0000 7f7850600700 1 -- 172.21.15.151:0/3425558542 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f78300b29f0 msgr2=0x7f78300b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:25.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.177+0000 7f7850600700 1 --2- 172.21.15.151:0/3425558542 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f78300b29f0 0x7f78300b4eb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f78400a9160 tx=0x7f7840006240 comp rx=0 tx=0).stop 2024-05-21T22:03:25.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.177+0000 7f7850600700 1 -- 172.21.15.151:0/3425558542 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f78481088f0 msgr2=0x7f78481a0780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:25.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.177+0000 7f7850600700 1 --2- 172.21.15.151:0/3425558542 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f78481088f0 0x7f78481a0780 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f784000c110 tx=0x7f784000c420 comp rx=0 tx=0).stop 2024-05-21T22:03:25.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.178+0000 7f7850600700 1 -- 172.21.15.151:0/3425558542 shutdown_connections 2024-05-21T22:03:25.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.178+0000 7f7850600700 1 --2- 172.21.15.151:0/3425558542 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f78300b29f0 0x7f78300b4eb0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:25.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.178+0000 7f7850600700 1 --2- 172.21.15.151:0/3425558542 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f78481037f0 0x7f784819beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:25.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.178+0000 7f7850600700 1 --2- 172.21.15.151:0/3425558542 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f78481088f0 0x7f78481a0780 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:25.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.178+0000 7f7850600700 1 --2- 172.21.15.151:0/3425558542 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78481041a0 0x7f784819c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:25.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.178+0000 7f7850600700 1 -- 172.21.15.151:0/3425558542 >> 172.21.15.151:0/3425558542 conn(0x7f784807a6a0 msgr2=0x7f784810fcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:03:25.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.178+0000 7f7850600700 1 -- 172.21.15.151:0/3425558542 shutdown_connections 2024-05-21T22:03:25.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:25.178+0000 7f7850600700 1 -- 172.21.15.151:0/3425558542 wait complete. 2024-05-21T22:03:25.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:03:25 smithi184 ceph-mon[36942]: pgmap v241: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 99 B/s wr, 9 op/s 2024-05-21T22:03:25.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:03:25 smithi164 ceph-mon[36269]: pgmap v241: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 99 B/s wr, 9 op/s 2024-05-21T22:03:25.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:03:25 smithi151 ceph-mon[31623]: pgmap v241: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 99 B/s wr, 9 op/s 2024-05-21T22:03:26.122 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-21T22:03:26.569 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:03:27.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:03:27 smithi184 ceph-mon[36942]: pgmap v242: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-05-21T22:03:27.737 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:03:27 smithi151 ceph-mon[31623]: pgmap v242: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-05-21T22:03:27.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:03:27 smithi164 ceph-mon[36269]: pgmap v242: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-05-21T22:03:28.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.784+0000 7f8d1e000700 1 -- 172.21.15.151:0/2186540970 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8d1810ffe0 msgr2=0x7f8d181123d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:28.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.784+0000 7f8d1e000700 1 --2- 172.21.15.151:0/2186540970 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8d1810ffe0 0x7f8d181123d0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f8d1000b0a0 tx=0x7f8d1000b3b0 comp rx=0 tx=0).stop 2024-05-21T22:03:28.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.784+0000 7f8d1e000700 1 -- 172.21.15.151:0/2186540970 shutdown_connections 2024-05-21T22:03:28.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.784+0000 7f8d1e000700 1 --2- 172.21.15.151:0/2186540970 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8d1806bae0 0x7f8d1806bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:28.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.784+0000 7f8d1e000700 1 --2- 172.21.15.151:0/2186540970 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8d1810ffe0 0x7f8d181123d0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:28.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.784+0000 7f8d1e000700 1 --2- 172.21.15.151:0/2186540970 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8d1806c400 0x7f8d1810faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:28.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.784+0000 7f8d1e000700 1 -- 172.21.15.151:0/2186540970 >> 172.21.15.151:0/2186540970 conn(0x7f8d180793b0 msgr2=0x7f8d180797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:03:28.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.784+0000 7f8d1e000700 1 -- 172.21.15.151:0/2186540970 shutdown_connections 2024-05-21T22:03:28.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.784+0000 7f8d1e000700 1 -- 172.21.15.151:0/2186540970 wait complete. 2024-05-21T22:03:28.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.785+0000 7f8d1e000700 1 Processor -- start 2024-05-21T22:03:28.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.785+0000 7f8d1e000700 1 -- start start 2024-05-21T22:03:28.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.786+0000 7f8d1e000700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8d1806bae0 0x7f8d1819be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:28.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.786+0000 7f8d1e000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8d1806c400 0x7f8d1819c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:28.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.786+0000 7f8d1e000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8d1810ffe0 0x7f8d181a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:28.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.786+0000 7f8d1e000700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8d18115950 con 0x7f8d1806c400 2024-05-21T22:03:28.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.786+0000 7f8d1e000700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8d181157d0 con 0x7f8d1810ffe0 2024-05-21T22:03:28.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.786+0000 7f8d1e000700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8d18115ad0 con 0x7f8d1806bae0 2024-05-21T22:03:28.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.786+0000 7f8d16a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8d1806c400 0x7f8d1819c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:28.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.786+0000 7f8d17400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8d1806bae0 0x7f8d1819be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:28.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.786+0000 7f8d16a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8d1806c400 0x7f8d1819c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41834/0 (socket says 172.21.15.151:41834) 2024-05-21T22:03:28.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.786+0000 7f8d16a00700 1 -- 172.21.15.151:0/1183297801 learned_addr learned my addr 172.21.15.151:0/1183297801 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:03:28.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.786+0000 7f8d17400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8d1806bae0 0x7f8d1819be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:39180/0 (socket says 172.21.15.151:39180) 2024-05-21T22:03:28.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.786+0000 7f8d17e00700 1 --2- 172.21.15.151:0/1183297801 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8d1810ffe0 0x7f8d181a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:28.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.787+0000 7f8d17400700 1 -- 172.21.15.151:0/1183297801 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8d1810ffe0 msgr2=0x7f8d181a06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:28.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.787+0000 7f8d17400700 1 --2- 172.21.15.151:0/1183297801 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8d1810ffe0 0x7f8d181a06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:28.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.787+0000 7f8d17400700 1 -- 172.21.15.151:0/1183297801 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8d1806c400 msgr2=0x7f8d1819c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:28.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.787+0000 7f8d17400700 1 --2- 172.21.15.151:0/1183297801 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8d1806c400 0x7f8d1819c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:28.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.787+0000 7f8d17400700 1 -- 172.21.15.151:0/1183297801 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d181a0e10 con 0x7f8d1806bae0 2024-05-21T22:03:28.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.787+0000 7f8d17400700 1 --2- 172.21.15.151:0/1183297801 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8d1806bae0 0x7f8d1819be00 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f8d0800e9f0 tx=0x7f8d0800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:03:28.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.787+0000 7f8d0fe00700 1 -- 172.21.15.151:0/1183297801 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d0800c9c0 con 0x7f8d1806bae0 2024-05-21T22:03:28.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.787+0000 7f8d0fe00700 1 -- 172.21.15.151:0/1183297801 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d0800cb60 con 0x7f8d1806bae0 2024-05-21T22:03:28.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.787+0000 7f8d1e000700 1 -- 172.21.15.151:0/1183297801 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d181a1100 con 0x7f8d1806bae0 2024-05-21T22:03:28.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.787+0000 7f8d0fe00700 1 -- 172.21.15.151:0/1183297801 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d08018680 con 0x7f8d1806bae0 2024-05-21T22:03:28.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.788+0000 7f8d1e000700 1 -- 172.21.15.151:0/1183297801 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d181a8980 con 0x7f8d1806bae0 2024-05-21T22:03:28.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.791+0000 7f8d0fe00700 1 -- 172.21.15.151:0/1183297801 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7f8d08010430 con 0x7f8d1806bae0 2024-05-21T22:03:28.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.792+0000 7f8d28a0db80 1 Processor -- start 2024-05-21T22:03:28.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.792+0000 7f8d28a0db80 1 -- start start 2024-05-21T22:03:28.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.793+0000 7f8d28a0db80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5650a4725eb0 0x5650a4726290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:28.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.793+0000 7f8d28a0db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650a47267d0 0x5650a4720260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:28.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.793+0000 7f8d28a0db80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650a4720880 0x5650a4720ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:28.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.793+0000 7f8d0fe00700 1 --2- 172.21.15.151:0/1183297801 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f8cfc0b29f0 0x7f8cfc0b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:28.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.793+0000 7f8d28a0db80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5650a467ee40 con 0x5650a4720880 2024-05-21T22:03:28.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.793+0000 7f8d28a0db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5650a466e9e0 con 0x5650a47267d0 2024-05-21T22:03:28.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.793+0000 7f8d28a0db80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5650a4672a20 con 0x5650a4725eb0 2024-05-21T22:03:28.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.793+0000 7f8d0fe00700 1 -- 172.21.15.151:0/1183297801 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7f8d080e27e0 con 0x7f8d1806bae0 2024-05-21T22:03:28.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.793+0000 7f8d17e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5650a4725eb0 0x5650a4726290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:28.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.793+0000 7f8d17e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5650a4725eb0 0x5650a4726290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:39182/0 (socket says 172.21.15.151:39182) 2024-05-21T22:03:28.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.793+0000 7f8d17e00700 1 -- 172.21.15.151:0/3236614640 learned_addr learned my addr 172.21.15.151:0/3236614640 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:03:28.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.793+0000 7f8d17400700 1 --2- 172.21.15.151:0/3236614640 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650a47267d0 0x5650a4720260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:28.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.793+0000 7f8d17e00700 1 --2- 172.21.15.151:0/3236614640 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650a4720880 0x5650a4720ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:28.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.793+0000 7f8d16a00700 1 --2- 172.21.15.151:0/1183297801 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f8cfc0b29f0 0x7f8cfc0b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:28.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.794+0000 7f8d17400700 1 -- 172.21.15.151:0/3236614640 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5650a4725eb0 msgr2=0x5650a4726290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:28.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.794+0000 7f8d17400700 1 --2- 172.21.15.151:0/3236614640 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5650a4725eb0 0x5650a4726290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:28.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.794+0000 7f8d17400700 1 -- 172.21.15.151:0/3236614640 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650a4720880 msgr2=0x5650a4720ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:28.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.794+0000 7f8d17400700 1 --2- 172.21.15.151:0/3236614640 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650a4720880 0x5650a4720ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:28.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.794+0000 7f8d17400700 1 -- 172.21.15.151:0/3236614640 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650a4637420 con 0x5650a47267d0 2024-05-21T22:03:28.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.794+0000 7f8d17400700 1 --2- 172.21.15.151:0/3236614640 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650a47267d0 0x5650a4720260 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f8d080dfec0 tx=0x7f8d080a9950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5819ba8a21e62c81 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:03:28.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.794+0000 7f8d16a00700 1 --2- 172.21.15.151:0/1183297801 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f8cfc0b29f0 0x7f8cfc0b4eb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8d18103900 tx=0x7f8d04009260 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:03:28.797 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.794+0000 7f8d0cc00700 1 -- 172.21.15.151:0/3236614640 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d080a5050 con 0x5650a47267d0 2024-05-21T22:03:28.797 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.794+0000 7f8d0cc00700 1 -- 172.21.15.151:0/3236614640 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d080b2040 con 0x5650a47267d0 2024-05-21T22:03:28.797 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.794+0000 7f8d0cc00700 1 -- 172.21.15.151:0/3236614640 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d080a9cb0 con 0x5650a47267d0 2024-05-21T22:03:28.797 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.795+0000 7f8d28a0db80 1 -- 172.21.15.151:0/3236614640 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650a47267d0 msgr2=0x5650a4720260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:28.797 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.795+0000 7f8d28a0db80 1 --2- 172.21.15.151:0/3236614640 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650a47267d0 0x5650a4720260 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f8d080dfec0 tx=0x7f8d080a9950 comp rx=0 tx=0).stop 2024-05-21T22:03:28.797 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.796+0000 7f8d28a0db80 1 -- 172.21.15.151:0/3236614640 shutdown_connections 2024-05-21T22:03:28.797 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.796+0000 7f8d28a0db80 1 --2- 172.21.15.151:0/3236614640 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650a4720880 0x5650a4720ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:28.797 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.796+0000 7f8d28a0db80 1 --2- 172.21.15.151:0/3236614640 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5650a4725eb0 0x5650a4726290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:28.797 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.796+0000 7f8d28a0db80 1 --2- 172.21.15.151:0/3236614640 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650a47267d0 0x5650a4720260 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:28.797 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.796+0000 7f8d28a0db80 1 -- 172.21.15.151:0/3236614640 >> 172.21.15.151:0/3236614640 conn(0x5650a464ebe0 msgr2=0x5650a471b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:03:28.797 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.796+0000 7f8d28a0db80 1 -- 172.21.15.151:0/3236614640 shutdown_connections 2024-05-21T22:03:28.797 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.796+0000 7f8d28a0db80 1 -- 172.21.15.151:0/3236614640 wait complete. 2024-05-21T22:03:28.799 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.796+0000 7f8d28a0db80 1 -- start start 2024-05-21T22:03:28.799 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.797+0000 7f8d28a0db80 1 Processor -- start 2024-05-21T22:03:28.799 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.797+0000 7f8d28a0db80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5650a4720880 0x5650a472c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:28.799 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.797+0000 7f8d28a0db80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650a4725eb0 0x5650a47298d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:28.799 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.797+0000 7f8d28a0db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650a47267d0 0x5650a4729e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:28.799 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.797+0000 7f8d28a0db80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5650a4574fa0 con 0x5650a4725eb0 2024-05-21T22:03:28.799 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.797+0000 7f8d28a0db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5650a460f760 con 0x5650a47267d0 2024-05-21T22:03:28.799 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.797+0000 7f8d28a0db80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5650a472d240 con 0x5650a4720880 2024-05-21T22:03:28.799 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.797+0000 7f8d28a0db80 10 client.0 _ll_register_callbacks cb 0x5650a4618c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:03:28.799 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.798+0000 7f8d16a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650a4725eb0 0x5650a47298d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:28.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.798+0000 7f8d17400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5650a4720880 0x5650a472c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:28.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.798+0000 7f8d17400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5650a4720880 0x5650a472c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:39188/0 (socket says 172.21.15.151:39188) 2024-05-21T22:03:28.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.798+0000 7f8d17400700 1 -- 172.21.15.151:0/3985601215 learned_addr learned my addr 172.21.15.151:0/3985601215 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:03:28.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.798+0000 7f8d17e00700 1 --2- 172.21.15.151:0/3985601215 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650a47267d0 0x5650a4729e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:28.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.798+0000 7f8d17400700 1 -- 172.21.15.151:0/3985601215 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650a47267d0 msgr2=0x5650a4729e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:28.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.798+0000 7f8d17400700 1 --2- 172.21.15.151:0/3985601215 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650a47267d0 0x5650a4729e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:28.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.798+0000 7f8d17400700 1 -- 172.21.15.151:0/3985601215 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650a4725eb0 msgr2=0x5650a47298d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:28.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.798+0000 7f8d17400700 1 --2- 172.21.15.151:0/3985601215 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650a4725eb0 0x5650a47298d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:28.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.798+0000 7f8d17400700 1 -- 172.21.15.151:0/3985601215 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650a45be8d0 con 0x5650a4720880 2024-05-21T22:03:28.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.798+0000 7f8d17400700 1 --2- 172.21.15.151:0/3985601215 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5650a4720880 0x5650a472c530 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f8d080dff20 tx=0x7f8d080a9ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d8abb5a67facdcc server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:03:28.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.798+0000 7f8d01600700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-21T22:03:28.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.799+0000 7f8d01600700 1 -- 172.21.15.151:0/3985601215 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d080aa150 con 0x5650a4720880 2024-05-21T22:03:28.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.799+0000 7f8d01600700 1 -- 172.21.15.151:0/3985601215 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7f8d080b2630 con 0x5650a4720880 2024-05-21T22:03:28.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.799+0000 7f8d28a0db80 1 -- 172.21.15.151:0/3985601215 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d080b2040 con 0x5650a4720880 2024-05-21T22:03:28.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.799+0000 7f8d01600700 1 -- 172.21.15.151:0/3985601215 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8d080a9a50 con 0x5650a4720880 2024-05-21T22:03:28.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.799+0000 7f8d28a0db80 10 client.27121 fetch_fsmap learned FSMap version 29 2024-05-21T22:03:28.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.799+0000 7f8d28a0db80 1 -- 172.21.15.151:0/3985601215 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5650a463f060 con 0x5650a4720880 2024-05-21T22:03:28.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.799+0000 7f8d01600700 1 -- 172.21.15.151:0/3985601215 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d080b2040 con 0x5650a4720880 2024-05-21T22:03:28.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.800+0000 7f8d01600700 1 -- 172.21.15.151:0/3985601215 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f8d08010720 con 0x5650a4720880 2024-05-21T22:03:28.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.800+0000 7f8d28a0db80 10 client.27121 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:03:28.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.800+0000 7f8d28a0db80 10 client.27121 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T22:03:28.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.800+0000 7f8d28a0db80 20 client.27121 populate_metadata read hostname 'smithi151' 2024-05-21T22:03:28.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.800+0000 7f8d28a0db80 1 --2- 172.21.15.151:0/3985601215 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5650a4739c40 0x5650a473c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:28.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.800+0000 7f8d28a0db80 4 client.27121 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T22:03:28.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.800+0000 7f8d28a0db80 1 -- 172.21.15.151:0/3985601215 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5650a4680d90 con 0x5650a4739c40 2024-05-21T22:03:28.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.800+0000 7f8d16a00700 1 --2- 172.21.15.151:0/3985601215 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5650a4739c40 0x5650a473c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:28.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.801+0000 7f8d16a00700 1 --2- 172.21.15.151:0/3985601215 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5650a4739c40 0x5650a473c030 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76b6aa0c49d34f21 server_cookie=fb6ea554cc9a1d9d in_seq=0 out_seq=0 2024-05-21T22:03:28.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.801+0000 7f8d01600700 10 client.27121 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T22:03:28.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.801+0000 7f8d01600700 1 -- 172.21.15.151:0/3985601215 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5650a4680d90 con 0x5650a4739c40 2024-05-21T22:03:28.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.801+0000 7f8d01600700 10 client.27121 handle_command_reply: tid=0 2024-05-21T22:03:28.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.802+0000 7f8d28a0db80 1 client.27121 shutdown 2024-05-21T22:03:28.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.802+0000 7f8d28a0db80 1 -- 172.21.15.151:0/3985601215 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5650a4720880 msgr2=0x5650a472c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:28.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.802+0000 7f8d28a0db80 1 --2- 172.21.15.151:0/3985601215 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5650a4720880 0x5650a472c530 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f8d080dff20 tx=0x7f8d080a9ec0 comp rx=0 tx=0).stop 2024-05-21T22:03:28.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.802+0000 7f8d28a0db80 1 -- 172.21.15.151:0/3985601215 shutdown_connections 2024-05-21T22:03:28.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.802+0000 7f8d28a0db80 1 --2- 172.21.15.151:0/3985601215 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5650a4739c40 0x5650a473c030 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:28.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.802+0000 7f8d28a0db80 1 --2- 172.21.15.151:0/3985601215 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650a47267d0 0x5650a4729e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:28.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.802+0000 7f8d28a0db80 1 --2- 172.21.15.151:0/3985601215 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5650a4720880 0x5650a472c530 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:28.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.802+0000 7f8d28a0db80 1 --2- 172.21.15.151:0/3985601215 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650a4725eb0 0x5650a47298d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:28.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.802+0000 7f8d01600700 0 client.27121 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T22:03:28.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.802+0000 7f8d28a0db80 1 -- 172.21.15.151:0/3985601215 >> 172.21.15.151:0/3985601215 conn(0x5650a464ebe0 msgr2=0x5650a471f250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:03:28.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.802+0000 7f8d28a0db80 1 -- 172.21.15.151:0/3985601215 shutdown_connections 2024-05-21T22:03:28.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.802+0000 7f8d28a0db80 1 -- 172.21.15.151:0/3985601215 wait complete. 2024-05-21T22:03:28.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.802+0000 7f8d28a0db80 20 client.27121 trim_cache size 0 max 16384 2024-05-21T22:03:28.819 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.819+0000 7f8d28a0db80 1 Processor -- start 2024-05-21T22:03:28.819 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.819+0000 7f8d28a0db80 1 -- start start 2024-05-21T22:03:28.820 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.819+0000 7f8d28a0db80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5650a4720880 0x5650a4720260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:28.820 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.819+0000 7f8d28a0db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650a4725eb0 0x5650a47238f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:28.820 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.819+0000 7f8d28a0db80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650a47267d0 0x5650a4723e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:28.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.819+0000 7f8d28a0db80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8d181b0110 con 0x5650a47267d0 2024-05-21T22:03:28.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.819+0000 7f8d28a0db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8d181b1c50 con 0x5650a4725eb0 2024-05-21T22:03:28.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.819+0000 7f8d28a0db80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8d181b07e0 con 0x5650a4720880 2024-05-21T22:03:28.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.820+0000 7f8d17e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650a47267d0 0x5650a4723e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:28.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.820+0000 7f8d16a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650a4725eb0 0x5650a47238f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:28.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.820+0000 7f8d16a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650a4725eb0 0x5650a47238f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:35054/0 (socket says 172.21.15.151:35054) 2024-05-21T22:03:28.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.820+0000 7f8d17e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650a47267d0 0x5650a4723e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41862/0 (socket says 172.21.15.151:41862) 2024-05-21T22:03:28.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.820+0000 7f8d16a00700 1 -- 172.21.15.151:0/335325972 learned_addr learned my addr 172.21.15.151:0/335325972 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:03:28.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.820+0000 7f8d16a00700 1 -- 172.21.15.151:0/335325972 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5650a4720880 msgr2=0x5650a4720260 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-21T22:03:28.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.820+0000 7f8d16a00700 1 --2- 172.21.15.151:0/335325972 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5650a4720880 0x5650a4720260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:28.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.820+0000 7f8d16a00700 1 -- 172.21.15.151:0/335325972 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650a47267d0 msgr2=0x5650a4723e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:28.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.820+0000 7f8d16a00700 1 --2- 172.21.15.151:0/335325972 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650a47267d0 0x5650a4723e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:28.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.820+0000 7f8d16a00700 1 -- 172.21.15.151:0/335325972 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650a4727e40 con 0x5650a4725eb0 2024-05-21T22:03:28.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.820+0000 7f8d17e00700 1 --2- 172.21.15.151:0/335325972 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650a47267d0 0x5650a4723e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:03:28.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.820+0000 7f8d16a00700 1 --2- 172.21.15.151:0/335325972 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650a4725eb0 0x5650a47238f0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f8d040098a0 tx=0x7f8d04017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85ddf25fc53142f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:03:28.823 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.821+0000 7f8d03e00700 1 -- 172.21.15.151:0/335325972 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d0401f6d0 con 0x5650a4725eb0 2024-05-21T22:03:28.823 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.821+0000 7f8d03e00700 1 -- 172.21.15.151:0/335325972 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d0402c070 con 0x5650a4725eb0 2024-05-21T22:03:28.823 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.821+0000 7f8d03e00700 1 -- 172.21.15.151:0/335325972 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d04027ac0 con 0x5650a4725eb0 2024-05-21T22:03:28.823 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.821+0000 7f8d28a0db80 1 -- 172.21.15.151:0/335325972 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650a4725eb0 msgr2=0x5650a47238f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:28.823 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.821+0000 7f8d28a0db80 1 --2- 172.21.15.151:0/335325972 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650a4725eb0 0x5650a47238f0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f8d040098a0 tx=0x7f8d04017000 comp rx=0 tx=0).stop 2024-05-21T22:03:28.823 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.822+0000 7f8d28a0db80 1 -- 172.21.15.151:0/335325972 shutdown_connections 2024-05-21T22:03:28.823 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.822+0000 7f8d28a0db80 1 --2- 172.21.15.151:0/335325972 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650a47267d0 0x5650a4723e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:28.823 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.822+0000 7f8d28a0db80 1 --2- 172.21.15.151:0/335325972 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5650a4720880 0x5650a4720260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:28.823 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.822+0000 7f8d28a0db80 1 --2- 172.21.15.151:0/335325972 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650a4725eb0 0x5650a47238f0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:28.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.822+0000 7f8d28a0db80 1 -- 172.21.15.151:0/335325972 >> 172.21.15.151:0/335325972 conn(0x5650a464ebe0 msgr2=0x5650a47281c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:03:28.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.822+0000 7f8d28a0db80 1 -- 172.21.15.151:0/335325972 shutdown_connections 2024-05-21T22:03:28.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.822+0000 7f8d28a0db80 1 -- 172.21.15.151:0/335325972 wait complete. 2024-05-21T22:03:28.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.822+0000 7f8d28a0db80 1 -- start start 2024-05-21T22:03:28.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.823+0000 7f8d28a0db80 1 Processor -- start 2024-05-21T22:03:28.826 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.823+0000 7f8d28a0db80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650a4720880 0x5650a473cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:28.826 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.823+0000 7f8d28a0db80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650a4725eb0 0x5650a473d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:28.826 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.823+0000 7f8d28a0db80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5650a47267d0 0x5650a473d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:28.826 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.823+0000 7f8d28a0db80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8d181b1580 con 0x5650a4725eb0 2024-05-21T22:03:28.826 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.823+0000 7f8d28a0db80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8d181b0eb0 con 0x5650a4720880 2024-05-21T22:03:28.826 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.823+0000 7f8d28a0db80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8d181ae0d0 con 0x5650a47267d0 2024-05-21T22:03:28.826 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.823+0000 7f8d28a0db80 10 client.0 _ll_register_callbacks cb 0x5650a4618c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:03:28.826 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.823+0000 7f8d16a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650a4725eb0 0x5650a473d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:28.826 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.824+0000 7f8d17e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5650a47267d0 0x5650a473d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:28.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.824+0000 7f8d17400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650a4720880 0x5650a473cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:28.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.824+0000 7f8d16a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650a4725eb0 0x5650a473d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41868/0 (socket says 172.21.15.151:41868) 2024-05-21T22:03:28.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.824+0000 7f8d16a00700 1 -- 172.21.15.151:0/672108247 learned_addr learned my addr 172.21.15.151:0/672108247 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:03:28.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.824+0000 7f8d17400700 1 -- 172.21.15.151:0/672108247 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5650a47267d0 msgr2=0x5650a473d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:28.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.824+0000 7f8d17400700 1 --2- 172.21.15.151:0/672108247 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5650a47267d0 0x5650a473d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:28.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.824+0000 7f8d17400700 1 -- 172.21.15.151:0/672108247 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650a4725eb0 msgr2=0x5650a473d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:28.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.824+0000 7f8d17400700 1 --2- 172.21.15.151:0/672108247 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650a4725eb0 0x5650a473d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:28.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.824+0000 7f8d17400700 1 -- 172.21.15.151:0/672108247 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650a47a34b0 con 0x5650a4720880 2024-05-21T22:03:28.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.824+0000 7f8d17400700 1 --2- 172.21.15.151:0/672108247 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650a4720880 0x5650a473cdc0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f8d080a99e0 tx=0x7f8d080a96a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b14a546448cf270 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:03:28.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.824+0000 7f8d01600700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-21T22:03:28.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.824+0000 7f8d01600700 1 -- 172.21.15.151:0/672108247 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d080a5910 con 0x5650a4720880 2024-05-21T22:03:28.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.824+0000 7f8d01600700 1 -- 172.21.15.151:0/672108247 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7f8d080a6040 con 0x5650a4720880 2024-05-21T22:03:28.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.824+0000 7f8d28a0db80 1 -- 172.21.15.151:0/672108247 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d0402c070 con 0x5650a4720880 2024-05-21T22:03:28.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.825+0000 7f8d01600700 1 -- 172.21.15.151:0/672108247 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8d080a9f30 con 0x5650a4720880 2024-05-21T22:03:28.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.825+0000 7f8d28a0db80 10 client.27119 fetch_fsmap learned FSMap version 29 2024-05-21T22:03:28.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.825+0000 7f8d28a0db80 1 -- 172.21.15.151:0/672108247 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5650a47a4790 con 0x5650a4720880 2024-05-21T22:03:28.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.825+0000 7f8d01600700 1 -- 172.21.15.151:0/672108247 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d0402c070 con 0x5650a4720880 2024-05-21T22:03:28.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.825+0000 7f8d01600700 1 -- 172.21.15.151:0/672108247 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f8d080a8720 con 0x5650a4720880 2024-05-21T22:03:28.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.825+0000 7f8d28a0db80 10 client.27119 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:03:28.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.825+0000 7f8d28a0db80 10 client.27119 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T22:03:28.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.825+0000 7f8d28a0db80 20 client.27119 populate_metadata read hostname 'smithi151' 2024-05-21T22:03:28.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.825+0000 7f8d28a0db80 1 --2- 172.21.15.151:0/672108247 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5650a47a4d00 0x5650a47a7120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:28.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.825+0000 7f8d28a0db80 4 client.27119 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T22:03:28.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.825+0000 7f8d28a0db80 1 -- 172.21.15.151:0/672108247 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5650a467a570 con 0x5650a47a4d00 2024-05-21T22:03:28.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.826+0000 7f8d16a00700 1 --2- 172.21.15.151:0/672108247 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5650a47a4d00 0x5650a47a7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:28.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.826+0000 7f8d16a00700 1 --2- 172.21.15.151:0/672108247 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5650a47a4d00 0x5650a47a7120 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b56f638ef227bba2 server_cookie=19903f72342a4a79 in_seq=0 out_seq=0 2024-05-21T22:03:28.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.826+0000 7f8d01600700 10 client.27119 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T22:03:28.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.826+0000 7f8d01600700 1 -- 172.21.15.151:0/672108247 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5650a467a570 con 0x5650a47a4d00 2024-05-21T22:03:28.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.826+0000 7f8d01600700 10 client.27119 handle_command_reply: tid=0 2024-05-21T22:03:28.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.826+0000 7f8d28a0db80 1 client.27119 shutdown 2024-05-21T22:03:28.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.827+0000 7f8d28a0db80 1 -- 172.21.15.151:0/672108247 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650a4720880 msgr2=0x5650a473cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:28.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.827+0000 7f8d28a0db80 1 --2- 172.21.15.151:0/672108247 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650a4720880 0x5650a473cdc0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f8d080a99e0 tx=0x7f8d080a96a0 comp rx=0 tx=0).stop 2024-05-21T22:03:28.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.827+0000 7f8d28a0db80 1 -- 172.21.15.151:0/672108247 shutdown_connections 2024-05-21T22:03:28.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.827+0000 7f8d28a0db80 1 --2- 172.21.15.151:0/672108247 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5650a47a4d00 0x5650a47a7120 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:28.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.827+0000 7f8d28a0db80 1 --2- 172.21.15.151:0/672108247 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5650a4720880 0x5650a473cdc0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:28.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.827+0000 7f8d28a0db80 1 --2- 172.21.15.151:0/672108247 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5650a47267d0 0x5650a473d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:28.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.827+0000 7f8d28a0db80 1 --2- 172.21.15.151:0/672108247 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650a4725eb0 0x5650a473d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:28.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.827+0000 7f8d01600700 0 client.27119 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T22:03:28.831 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.827+0000 7f8d28a0db80 1 -- 172.21.15.151:0/672108247 >> 172.21.15.151:0/672108247 conn(0x5650a464ebe0 msgr2=0x5650a4728050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:03:28.831 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.827+0000 7f8d28a0db80 1 -- 172.21.15.151:0/672108247 shutdown_connections 2024-05-21T22:03:28.831 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.827+0000 7f8d28a0db80 1 -- 172.21.15.151:0/672108247 wait complete. 2024-05-21T22:03:28.831 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.827+0000 7f8d28a0db80 20 client.27119 trim_cache size 0 max 16384 2024-05-21T22:03:28.831 INFO:teuthology.orchestra.run.smithi151.stdout:[] 2024-05-21T22:03:28.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.828+0000 7f8d1e000700 1 -- 172.21.15.151:0/1183297801 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f8cfc0b29f0 msgr2=0x7f8cfc0b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:28.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.828+0000 7f8d1e000700 1 --2- 172.21.15.151:0/1183297801 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f8cfc0b29f0 0x7f8cfc0b4eb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8d18103900 tx=0x7f8d04009260 comp rx=0 tx=0).stop 2024-05-21T22:03:28.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.828+0000 7f8d1e000700 1 -- 172.21.15.151:0/1183297801 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8d1806bae0 msgr2=0x7f8d1819be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:28.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.828+0000 7f8d1e000700 1 --2- 172.21.15.151:0/1183297801 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8d1806bae0 0x7f8d1819be00 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f8d0800e9f0 tx=0x7f8d0800ed00 comp rx=0 tx=0).stop 2024-05-21T22:03:28.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.828+0000 7f8d1e000700 1 -- 172.21.15.151:0/1183297801 shutdown_connections 2024-05-21T22:03:28.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.828+0000 7f8d1e000700 1 --2- 172.21.15.151:0/1183297801 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f8cfc0b29f0 0x7f8cfc0b4eb0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:28.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.828+0000 7f8d1e000700 1 --2- 172.21.15.151:0/1183297801 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8d1810ffe0 0x7f8d181a06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:28.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.828+0000 7f8d1e000700 1 --2- 172.21.15.151:0/1183297801 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8d1806bae0 0x7f8d1819be00 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:28.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.828+0000 7f8d1e000700 1 --2- 172.21.15.151:0/1183297801 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8d1806c400 0x7f8d1819c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:28.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.828+0000 7f8d1e000700 1 -- 172.21.15.151:0/1183297801 >> 172.21.15.151:0/1183297801 conn(0x7f8d180793b0 msgr2=0x7f8d18111430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:03:28.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.828+0000 7f8d1e000700 1 -- 172.21.15.151:0/1183297801 shutdown_connections 2024-05-21T22:03:28.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:28.828+0000 7f8d1e000700 1 -- 172.21.15.151:0/1183297801 wait complete. 2024-05-21T22:03:29.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:03:29 smithi151 ceph-mon[31623]: pgmap v243: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-05-21T22:03:29.735 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:03:29 smithi184 ceph-mon[36942]: pgmap v243: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-05-21T22:03:29.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:03:29 smithi164 ceph-mon[36269]: pgmap v243: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-05-21T22:03:30.824 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-21T22:03:30.824 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-21T22:03:31.278 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:03:31.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:03:31 smithi151 ceph-mon[31623]: pgmap v244: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:03:31.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:03:31 smithi164 ceph-mon[36269]: pgmap v244: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:03:31.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:03:31 smithi184 ceph-mon[36942]: pgmap v244: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:03:33.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.587+0000 7f4f91200700 1 -- 172.21.15.151:0/411508735 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4f8c1041a0 msgr2=0x7f4f8c1081c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:33.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.587+0000 7f4f91200700 1 --2- 172.21.15.151:0/411508735 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4f8c1041a0 0x7f4f8c1081c0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f4f74009990 tx=0x7f4f74009ca0 comp rx=0 tx=0).stop 2024-05-21T22:03:33.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.588+0000 7f4f91200700 1 -- 172.21.15.151:0/411508735 shutdown_connections 2024-05-21T22:03:33.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.588+0000 7f4f91200700 1 --2- 172.21.15.151:0/411508735 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4f8c1037f0 0x7f4f8c103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:33.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.588+0000 7f4f91200700 1 --2- 172.21.15.151:0/411508735 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4f8c1088f0 0x7f4f8c110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:33.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.588+0000 7f4f91200700 1 --2- 172.21.15.151:0/411508735 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4f8c1041a0 0x7f4f8c1081c0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:33.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.588+0000 7f4f91200700 1 -- 172.21.15.151:0/411508735 >> 172.21.15.151:0/411508735 conn(0x7f4f8c07a6a0 msgr2=0x7f4f8c07aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:03:33.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.588+0000 7f4f91200700 1 -- 172.21.15.151:0/411508735 shutdown_connections 2024-05-21T22:03:33.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.588+0000 7f4f91200700 1 -- 172.21.15.151:0/411508735 wait complete. 2024-05-21T22:03:33.590 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:03:33 smithi151 ceph-mon[31623]: pgmap v245: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T22:03:33.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.589+0000 7f4f91200700 1 Processor -- start 2024-05-21T22:03:33.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.589+0000 7f4f91200700 1 -- start start 2024-05-21T22:03:33.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.590+0000 7f4f91200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4f8c1037f0 0x7f4f8c19f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:33.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.590+0000 7f4f91200700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4f8c1041a0 0x7f4f8c19fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:33.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.590+0000 7f4f91200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4f8c1088f0 0x7f4f8c199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:33.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.590+0000 7f4f91200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4f8c1136b0 con 0x7f4f8c1088f0 2024-05-21T22:03:33.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.590+0000 7f4f91200700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4f8c113530 con 0x7f4f8c1037f0 2024-05-21T22:03:33.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.590+0000 7f4f91200700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4f8c113830 con 0x7f4f8c1041a0 2024-05-21T22:03:33.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.590+0000 7f4f89e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4f8c1037f0 0x7f4f8c19f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:33.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.590+0000 7f4f81400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4f8c1041a0 0x7f4f8c19fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:33.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.590+0000 7f4f89e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4f8c1037f0 0x7f4f8c19f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:35092/0 (socket says 172.21.15.151:35092) 2024-05-21T22:03:33.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.590+0000 7f4f89e00700 1 -- 172.21.15.151:0/3832259536 learned_addr learned my addr 172.21.15.151:0/3832259536 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:03:33.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.590+0000 7f4f8a800700 1 --2- 172.21.15.151:0/3832259536 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4f8c1088f0 0x7f4f8c199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:33.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.590+0000 7f4f81400700 1 -- 172.21.15.151:0/3832259536 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4f8c1037f0 msgr2=0x7f4f8c19f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:33.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.590+0000 7f4f81400700 1 --2- 172.21.15.151:0/3832259536 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4f8c1037f0 0x7f4f8c19f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:33.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.590+0000 7f4f81400700 1 -- 172.21.15.151:0/3832259536 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4f8c1088f0 msgr2=0x7f4f8c199970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:33.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.590+0000 7f4f81400700 1 --2- 172.21.15.151:0/3832259536 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4f8c1088f0 0x7f4f8c199970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:33.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.590+0000 7f4f81400700 1 -- 172.21.15.151:0/3832259536 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f8c19a0d0 con 0x7f4f8c1041a0 2024-05-21T22:03:33.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.590+0000 7f4f8a800700 1 --2- 172.21.15.151:0/3832259536 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4f8c1088f0 0x7f4f8c199970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:03:33.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.590+0000 7f4f89e00700 1 --2- 172.21.15.151:0/3832259536 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4f8c1037f0 0x7f4f8c19f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:03:33.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.591+0000 7f4f81400700 1 --2- 172.21.15.151:0/3832259536 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4f8c1041a0 0x7f4f8c19fcf0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f4f74006010 tx=0x7f4f740057c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:03:33.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.591+0000 7f4f83400700 1 -- 172.21.15.151:0/3832259536 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f74027070 con 0x7f4f8c1041a0 2024-05-21T22:03:33.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.591+0000 7f4f83400700 1 -- 172.21.15.151:0/3832259536 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f7400b610 con 0x7f4f8c1041a0 2024-05-21T22:03:33.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.591+0000 7f4f91200700 1 -- 172.21.15.151:0/3832259536 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f8c19a360 con 0x7f4f8c1041a0 2024-05-21T22:03:33.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.591+0000 7f4f83400700 1 -- 172.21.15.151:0/3832259536 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f74022e30 con 0x7f4f8c1041a0 2024-05-21T22:03:33.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.591+0000 7f4f91200700 1 -- 172.21.15.151:0/3832259536 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f8c1a6680 con 0x7f4f8c1041a0 2024-05-21T22:03:33.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.596+0000 7f4f83400700 1 -- 172.21.15.151:0/3832259536 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7f4f7402a070 con 0x7f4f8c1041a0 2024-05-21T22:03:33.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.597+0000 7f4f9baabb80 1 Processor -- start 2024-05-21T22:03:33.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.597+0000 7f4f9baabb80 1 -- start start 2024-05-21T22:03:33.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.598+0000 7f4f9baabb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564badbaef30 0x564badbaf310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:33.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.598+0000 7f4f9baabb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564badbaf850 0x564badba92e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:33.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.598+0000 7f4f9baabb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564badba9900 0x564badba9d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:33.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.598+0000 7f4f9baabb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x564badb07ec0 con 0x564badbaf850 2024-05-21T22:03:33.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.598+0000 7f4f9baabb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x564badaf7a60 con 0x564badbaef30 2024-05-21T22:03:33.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.598+0000 7f4f9baabb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x564badafbaa0 con 0x564badba9900 2024-05-21T22:03:33.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.598+0000 7f4f83400700 1 --2- 172.21.15.151:0/3832259536 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f4f700b29f0 0x7f4f700b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:33.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.598+0000 7f4f89e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564badbaf850 0x564badba92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:33.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.598+0000 7f4f8a800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564badbaef30 0x564badbaf310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:33.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.598+0000 7f4f89e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564badbaf850 0x564badba92e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41886/0 (socket says 172.21.15.151:41886) 2024-05-21T22:03:33.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.598+0000 7f4f89e00700 1 -- 172.21.15.151:0/2793783542 learned_addr learned my addr 172.21.15.151:0/2793783542 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:03:33.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.598+0000 7f4f81400700 1 --2- 172.21.15.151:0/2793783542 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564badba9900 0x564badba9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:33.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.598+0000 7f4f83400700 1 -- 172.21.15.151:0/3832259536 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7f4f740e3420 con 0x7f4f8c1041a0 2024-05-21T22:03:33.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.599+0000 7f4f8a800700 1 --2- 172.21.15.151:0/3832259536 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f4f700b29f0 0x7f4f700b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:33.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.599+0000 7f4f81400700 1 -- 172.21.15.151:0/2793783542 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564badbaef30 msgr2=0x564badbaf310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:33.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.599+0000 7f4f81400700 1 --2- 172.21.15.151:0/2793783542 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564badbaef30 0x564badbaf310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:33.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.599+0000 7f4f81400700 1 -- 172.21.15.151:0/2793783542 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564badbaf850 msgr2=0x564badba92e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:33.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.599+0000 7f4f81400700 1 --2- 172.21.15.151:0/2793783542 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564badbaf850 0x564badba92e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:33.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.599+0000 7f4f81400700 1 -- 172.21.15.151:0/2793783542 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564badac04a0 con 0x564badba9900 2024-05-21T22:03:33.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.599+0000 7f4f89e00700 1 --2- 172.21.15.151:0/2793783542 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564badbaf850 0x564badba92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:03:33.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.599+0000 7f4f81400700 1 --2- 172.21.15.151:0/2793783542 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564badba9900 0x564badba9d60 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f4f740e0b00 tx=0x7f4f740d9ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee0c6fc92ccff16a server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:03:33.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.599+0000 7f4f8a800700 1 --2- 172.21.15.151:0/3832259536 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f4f700b29f0 0x7f4f700b4eb0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4f840060b0 tx=0x7f4f84004b40 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:03:33.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.599+0000 7f4f7b400700 1 -- 172.21.15.151:0/2793783542 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f74078130 con 0x564badba9900 2024-05-21T22:03:33.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.599+0000 7f4f7b400700 1 -- 172.21.15.151:0/2793783542 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f740aa230 con 0x564badba9900 2024-05-21T22:03:33.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.599+0000 7f4f7b400700 1 -- 172.21.15.151:0/2793783542 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f740b34f0 con 0x564badba9900 2024-05-21T22:03:33.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.600+0000 7f4f9baabb80 1 -- 172.21.15.151:0/2793783542 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564badba9900 msgr2=0x564badba9d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:33.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.600+0000 7f4f9baabb80 1 --2- 172.21.15.151:0/2793783542 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564badba9900 0x564badba9d60 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f4f740e0b00 tx=0x7f4f740d9ca0 comp rx=0 tx=0).stop 2024-05-21T22:03:33.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.600+0000 7f4f9baabb80 1 -- 172.21.15.151:0/2793783542 shutdown_connections 2024-05-21T22:03:33.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.600+0000 7f4f9baabb80 1 --2- 172.21.15.151:0/2793783542 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564badbaef30 0x564badbaf310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:33.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.600+0000 7f4f9baabb80 1 --2- 172.21.15.151:0/2793783542 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564badba9900 0x564badba9d60 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:33.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.600+0000 7f4f9baabb80 1 --2- 172.21.15.151:0/2793783542 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564badbaf850 0x564badba92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:33.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.600+0000 7f4f9baabb80 1 -- 172.21.15.151:0/2793783542 >> 172.21.15.151:0/2793783542 conn(0x564badad7c60 msgr2=0x564badba4090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:03:33.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.600+0000 7f4f9baabb80 1 -- 172.21.15.151:0/2793783542 shutdown_connections 2024-05-21T22:03:33.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.600+0000 7f4f9baabb80 1 -- 172.21.15.151:0/2793783542 wait complete. 2024-05-21T22:03:33.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.601+0000 7f4f9baabb80 1 -- start start 2024-05-21T22:03:33.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.601+0000 7f4f9baabb80 1 Processor -- start 2024-05-21T22:03:33.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.602+0000 7f4f9baabb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564badba9900 0x564badbb5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:33.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.602+0000 7f4f9baabb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564badbaef30 0x564badbb29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:33.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.602+0000 7f4f9baabb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564badbaf850 0x564badbb2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:33.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.602+0000 7f4f9baabb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x564bad9fe020 con 0x564badbaf850 2024-05-21T22:03:33.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.602+0000 7f4f9baabb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x564bada987e0 con 0x564badbaef30 2024-05-21T22:03:33.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.602+0000 7f4f9baabb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x564badbb6080 con 0x564badba9900 2024-05-21T22:03:33.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.602+0000 7f4f9baabb80 10 client.0 _ll_register_callbacks cb 0x564badaa1cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:03:33.606 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.602+0000 7f4f8a800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564badbaf850 0x564badbb2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:33.606 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.602+0000 7f4f8a800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564badbaf850 0x564badbb2f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41894/0 (socket says 172.21.15.151:41894) 2024-05-21T22:03:33.606 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.602+0000 7f4f8a800700 1 -- 172.21.15.151:0/2376950628 learned_addr learned my addr 172.21.15.151:0/2376950628 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:03:33.606 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.602+0000 7f4f89e00700 1 --2- 172.21.15.151:0/2376950628 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564badba9900 0x564badbb5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:33.606 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.602+0000 7f4f81400700 1 --2- 172.21.15.151:0/2376950628 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564badbaef30 0x564badbb29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:33.606 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.603+0000 7f4f8a800700 1 -- 172.21.15.151:0/2376950628 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564badba9900 msgr2=0x564badbb5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:33.606 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.603+0000 7f4f8a800700 1 --2- 172.21.15.151:0/2376950628 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564badba9900 0x564badbb5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:33.606 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.603+0000 7f4f8a800700 1 -- 172.21.15.151:0/2376950628 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564badbaef30 msgr2=0x564badbb29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:33.606 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.603+0000 7f4f8a800700 1 --2- 172.21.15.151:0/2376950628 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564badbaef30 0x564badbb29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:33.606 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.603+0000 7f4f8a800700 1 -- 172.21.15.151:0/2376950628 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bada47950 con 0x564badbaf850 2024-05-21T22:03:33.606 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.603+0000 7f4f81400700 1 --2- 172.21.15.151:0/2376950628 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564badbaef30 0x564badbb29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:03:33.606 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.603+0000 7f4f8a800700 1 --2- 172.21.15.151:0/2376950628 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564badbaf850 0x564badbb2f10 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f4f8400eb10 tx=0x7f4f84004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e50b519a000b2198 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:03:33.607 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.603+0000 7f4f6fe00700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-21T22:03:33.607 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.603+0000 7f4f6fe00700 1 -- 172.21.15.151:0/2376950628 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f84007b20 con 0x564badbaf850 2024-05-21T22:03:33.607 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.604+0000 7f4f6fe00700 1 -- 172.21.15.151:0/2376950628 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7f4f84007cc0 con 0x564badbaf850 2024-05-21T22:03:33.607 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.604+0000 7f4f9baabb80 1 -- 172.21.15.151:0/2376950628 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f740aa230 con 0x564badbaf850 2024-05-21T22:03:33.607 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.604+0000 7f4f6fe00700 1 -- 172.21.15.151:0/2376950628 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4f8401b450 con 0x564badbaf850 2024-05-21T22:03:33.607 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.604+0000 7f4f9baabb80 10 client.17007 fetch_fsmap learned FSMap version 29 2024-05-21T22:03:33.607 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.604+0000 7f4f9baabb80 1 -- 172.21.15.151:0/2376950628 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564badac80e0 con 0x564badbaf850 2024-05-21T22:03:33.607 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.604+0000 7f4f6fe00700 1 -- 172.21.15.151:0/2376950628 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f740aa230 con 0x564badbaf850 2024-05-21T22:03:33.607 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.604+0000 7f4f6fe00700 1 -- 172.21.15.151:0/2376950628 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f4f8401a570 con 0x564badbaf850 2024-05-21T22:03:33.607 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.605+0000 7f4f9baabb80 10 client.17007 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:03:33.607 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.605+0000 7f4f9baabb80 10 client.17007 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T22:03:33.607 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.605+0000 7f4f9baabb80 20 client.17007 populate_metadata read hostname 'smithi151' 2024-05-21T22:03:33.607 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.605+0000 7f4f9baabb80 1 --2- 172.21.15.151:0/2376950628 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x564badbc2b60 0x564badbc4f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:33.608 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.605+0000 7f4f9baabb80 4 client.17007 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T22:03:33.608 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.605+0000 7f4f9baabb80 1 -- 172.21.15.151:0/2376950628 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564badb09e10 con 0x564badbc2b60 2024-05-21T22:03:33.609 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.605+0000 7f4f89e00700 1 --2- 172.21.15.151:0/2376950628 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x564badbc2b60 0x564badbc4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:33.609 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.606+0000 7f4f89e00700 1 --2- 172.21.15.151:0/2376950628 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x564badbc2b60 0x564badbc4f80 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa8b8f025689715b server_cookie=d73cfcf2ed14e483 in_seq=0 out_seq=0 2024-05-21T22:03:33.609 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.606+0000 7f4f6fe00700 10 client.17007 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T22:03:33.609 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.606+0000 7f4f6fe00700 1 -- 172.21.15.151:0/2376950628 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x564badb09e10 con 0x564badbc2b60 2024-05-21T22:03:33.609 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.606+0000 7f4f6fe00700 10 client.17007 handle_command_reply: tid=0 2024-05-21T22:03:33.609 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.606+0000 7f4f9baabb80 1 client.17007 shutdown 2024-05-21T22:03:33.609 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.607+0000 7f4f9baabb80 1 -- 172.21.15.151:0/2376950628 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564badbaf850 msgr2=0x564badbb2f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:33.609 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.607+0000 7f4f9baabb80 1 --2- 172.21.15.151:0/2376950628 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564badbaf850 0x564badbb2f10 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f4f8400eb10 tx=0x7f4f84004ad0 comp rx=0 tx=0).stop 2024-05-21T22:03:33.609 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.607+0000 7f4f9baabb80 1 -- 172.21.15.151:0/2376950628 shutdown_connections 2024-05-21T22:03:33.610 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.607+0000 7f4f9baabb80 1 --2- 172.21.15.151:0/2376950628 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564badbaf850 0x564badbb2f10 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:33.610 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.607+0000 7f4f9baabb80 1 --2- 172.21.15.151:0/2376950628 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564badba9900 0x564badbb5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:33.610 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.607+0000 7f4f9baabb80 1 --2- 172.21.15.151:0/2376950628 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x564badbc2b60 0x564badbc4f80 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:33.610 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.607+0000 7f4f9baabb80 1 --2- 172.21.15.151:0/2376950628 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564badbaef30 0x564badbb29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:33.610 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.607+0000 7f4f9baabb80 1 -- 172.21.15.151:0/2376950628 >> 172.21.15.151:0/2376950628 conn(0x564badad7c60 msgr2=0x564badba82d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:03:33.610 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.607+0000 7f4f6fe00700 0 client.17007 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T22:03:33.610 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.607+0000 7f4f9baabb80 1 -- 172.21.15.151:0/2376950628 shutdown_connections 2024-05-21T22:03:33.610 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.607+0000 7f4f9baabb80 1 -- 172.21.15.151:0/2376950628 wait complete. 2024-05-21T22:03:33.610 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.607+0000 7f4f9baabb80 20 client.17007 trim_cache size 0 max 16384 2024-05-21T22:03:33.630 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.630+0000 7f4f9baabb80 1 Processor -- start 2024-05-21T22:03:33.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.630+0000 7f4f9baabb80 1 -- start start 2024-05-21T22:03:33.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.630+0000 7f4f9baabb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564badba9900 0x564badba92e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:33.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.630+0000 7f4f9baabb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564badbaef30 0x564badbb04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:33.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.630+0000 7f4f9baabb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564badbaf850 0x564badba40d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:33.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.630+0000 7f4f9baabb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4f8c1abaf0 con 0x564badba9900 2024-05-21T22:03:33.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.630+0000 7f4f9baabb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4f8c1ad630 con 0x564badbaef30 2024-05-21T22:03:33.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.630+0000 7f4f9baabb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4f8c1ac1c0 con 0x564badbaf850 2024-05-21T22:03:33.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.631+0000 7f4f89e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564badba9900 0x564badba92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:33.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.631+0000 7f4f89e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564badba9900 0x564badba92e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41910/0 (socket says 172.21.15.151:41910) 2024-05-21T22:03:33.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.631+0000 7f4f8a800700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564badbaf850 0x564badba40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:33.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.631+0000 7f4f89e00700 1 -- 172.21.15.151:0/1289383390 learned_addr learned my addr 172.21.15.151:0/1289383390 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:03:33.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.631+0000 7f4f81400700 1 --2- 172.21.15.151:0/1289383390 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564badbaef30 0x564badbb04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:33.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.631+0000 7f4f8a800700 1 -- 172.21.15.151:0/1289383390 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564badbaef30 msgr2=0x564badbb04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:33.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.631+0000 7f4f8a800700 1 --2- 172.21.15.151:0/1289383390 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564badbaef30 0x564badbb04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:33.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.631+0000 7f4f8a800700 1 -- 172.21.15.151:0/1289383390 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564badba9900 msgr2=0x564badba92e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:33.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.631+0000 7f4f8a800700 1 --2- 172.21.15.151:0/1289383390 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564badba9900 0x564badba92e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:33.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.631+0000 7f4f8a800700 1 -- 172.21.15.151:0/1289383390 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564badbaf650 con 0x564badbaf850 2024-05-21T22:03:33.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.632+0000 7f4f8a800700 1 --2- 172.21.15.151:0/1289383390 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564badbaf850 0x564badba40d0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f4f8401a940 tx=0x7f4f8401adc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a23d557748794d01 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:03:33.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.632+0000 7f4f7aa00700 1 -- 172.21.15.151:0/1289383390 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f8401f070 con 0x564badbaf850 2024-05-21T22:03:33.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.632+0000 7f4f7aa00700 1 -- 172.21.15.151:0/1289383390 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f84034020 con 0x564badbaf850 2024-05-21T22:03:33.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.632+0000 7f4f7aa00700 1 -- 172.21.15.151:0/1289383390 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f84004bb0 con 0x564badbaf850 2024-05-21T22:03:33.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.633+0000 7f4f9baabb80 1 -- 172.21.15.151:0/1289383390 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564badbaf850 msgr2=0x564badba40d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:33.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.633+0000 7f4f9baabb80 1 --2- 172.21.15.151:0/1289383390 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564badbaf850 0x564badba40d0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f4f8401a940 tx=0x7f4f8401adc0 comp rx=0 tx=0).stop 2024-05-21T22:03:33.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.633+0000 7f4f9baabb80 1 -- 172.21.15.151:0/1289383390 shutdown_connections 2024-05-21T22:03:33.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.633+0000 7f4f9baabb80 1 --2- 172.21.15.151:0/1289383390 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564badbaf850 0x564badba40d0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:33.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.633+0000 7f4f9baabb80 1 --2- 172.21.15.151:0/1289383390 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564badba9900 0x564badba92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:33.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.633+0000 7f4f9baabb80 1 --2- 172.21.15.151:0/1289383390 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564badbaef30 0x564badbb04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:33.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.633+0000 7f4f9baabb80 1 -- 172.21.15.151:0/1289383390 >> 172.21.15.151:0/1289383390 conn(0x564badad7c60 msgr2=0x564badbb12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:03:33.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.633+0000 7f4f9baabb80 1 -- 172.21.15.151:0/1289383390 shutdown_connections 2024-05-21T22:03:33.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.633+0000 7f4f9baabb80 1 -- 172.21.15.151:0/1289383390 wait complete. 2024-05-21T22:03:33.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.633+0000 7f4f9baabb80 1 -- start start 2024-05-21T22:03:33.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.634+0000 7f4f9baabb80 1 Processor -- start 2024-05-21T22:03:33.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.634+0000 7f4f9baabb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564badba9900 0x564badbbba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:33.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.634+0000 7f4f9baabb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564badbaef30 0x564badbb82a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:33.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.634+0000 7f4f9baabb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564badbaf850 0x564badbb87e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:33.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.634+0000 7f4f9baabb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4f8c1acf60 con 0x564badbaef30 2024-05-21T22:03:33.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.634+0000 7f4f9baabb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4f8c1ac890 con 0x564badba9900 2024-05-21T22:03:33.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.634+0000 7f4f9baabb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4f8c1a9ab0 con 0x564badbaf850 2024-05-21T22:03:33.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.634+0000 7f4f9baabb80 10 client.0 _ll_register_callbacks cb 0x564badaa1cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:03:33.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.634+0000 7f4f81400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564badbaef30 0x564badbb82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:33.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.635+0000 7f4f81400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564badbaef30 0x564badbb82a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41914/0 (socket says 172.21.15.151:41914) 2024-05-21T22:03:33.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.635+0000 7f4f81400700 1 -- 172.21.15.151:0/23633049 learned_addr learned my addr 172.21.15.151:0/23633049 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:03:33.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.635+0000 7f4f8a800700 1 --2- 172.21.15.151:0/23633049 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564badbaf850 0x564badbb87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:33.637 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.635+0000 7f4f89e00700 1 --2- 172.21.15.151:0/23633049 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564badba9900 0x564badbbba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:33.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.635+0000 7f4f81400700 1 -- 172.21.15.151:0/23633049 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564badbaf850 msgr2=0x564badbb87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:33.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.635+0000 7f4f81400700 1 --2- 172.21.15.151:0/23633049 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564badbaf850 0x564badbb87e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:33.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.635+0000 7f4f81400700 1 -- 172.21.15.151:0/23633049 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564badba9900 msgr2=0x564badbbba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:33.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.635+0000 7f4f81400700 1 --2- 172.21.15.151:0/23633049 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564badba9900 0x564badbbba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:33.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.635+0000 7f4f81400700 1 -- 172.21.15.151:0/23633049 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564badbb90a0 con 0x564badbaef30 2024-05-21T22:03:33.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.635+0000 7f4f81400700 1 --2- 172.21.15.151:0/23633049 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564badbaef30 0x564badbb82a0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f4f74000c00 tx=0x7f4f740228e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7804b27318db956 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:03:33.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.635+0000 7f4f6fe00700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-21T22:03:33.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.635+0000 7f4f6fe00700 1 -- 172.21.15.151:0/23633049 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f740aa230 con 0x564badbaef30 2024-05-21T22:03:33.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.636+0000 7f4f6fe00700 1 -- 172.21.15.151:0/23633049 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7f4f740b3290 con 0x564badbaef30 2024-05-21T22:03:33.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.636+0000 7f4f9baabb80 1 -- 172.21.15.151:0/23633049 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f84034020 con 0x564badbaef30 2024-05-21T22:03:33.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.636+0000 7f4f6fe00700 1 -- 172.21.15.151:0/23633049 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4f740aae20 con 0x564badbaef30 2024-05-21T22:03:33.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.636+0000 7f4f9baabb80 10 client.17013 fetch_fsmap learned FSMap version 29 2024-05-21T22:03:33.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.636+0000 7f4f9baabb80 1 -- 172.21.15.151:0/23633049 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564badc2e7b0 con 0x564badbaef30 2024-05-21T22:03:33.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.636+0000 7f4f6fe00700 1 -- 172.21.15.151:0/23633049 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f84034020 con 0x564badbaef30 2024-05-21T22:03:33.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.636+0000 7f4f6fe00700 1 -- 172.21.15.151:0/23633049 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f4f740050b0 con 0x564badbaef30 2024-05-21T22:03:33.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.636+0000 7f4f9baabb80 10 client.17013 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:03:33.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.636+0000 7f4f9baabb80 10 client.17013 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T22:03:33.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.636+0000 7f4f9baabb80 20 client.17013 populate_metadata read hostname 'smithi151' 2024-05-21T22:03:33.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.636+0000 7f4f9baabb80 1 --2- 172.21.15.151:0/23633049 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x564badc2ecf0 0x564badc31110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:33.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.636+0000 7f4f9baabb80 4 client.17013 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-21T22:03:33.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.636+0000 7f4f9baabb80 1 -- 172.21.15.151:0/23633049 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x564badb035f0 con 0x564badc2ecf0 2024-05-21T22:03:33.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.637+0000 7f4f89e00700 1 --2- 172.21.15.151:0/23633049 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x564badc2ecf0 0x564badc31110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:33.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.637+0000 7f4f89e00700 1 --2- 172.21.15.151:0/23633049 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x564badc2ecf0 0x564badc31110 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1092aca5d8bf2ca3 server_cookie=2493a7caa18bc8da in_seq=0 out_seq=0 2024-05-21T22:03:33.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.637+0000 7f4f6fe00700 10 client.17013 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T22:03:33.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.639+0000 7f4f6fe00700 1 -- 172.21.15.151:0/23633049 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x564badb035f0 con 0x564badc2ecf0 2024-05-21T22:03:33.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.639+0000 7f4f6fe00700 10 client.17013 handle_command_reply: tid=0 2024-05-21T22:03:33.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.639+0000 7f4f9baabb80 1 client.17013 shutdown 2024-05-21T22:03:33.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.639+0000 7f4f9baabb80 1 -- 172.21.15.151:0/23633049 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564badbaef30 msgr2=0x564badbb82a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:33.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.639+0000 7f4f9baabb80 1 --2- 172.21.15.151:0/23633049 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564badbaef30 0x564badbb82a0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f4f74000c00 tx=0x7f4f740228e0 comp rx=0 tx=0).stop 2024-05-21T22:03:33.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.639+0000 7f4f9baabb80 1 -- 172.21.15.151:0/23633049 shutdown_connections 2024-05-21T22:03:33.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.639+0000 7f4f9baabb80 1 --2- 172.21.15.151:0/23633049 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x564badc2ecf0 0x564badc31110 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:33.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.639+0000 7f4f9baabb80 1 --2- 172.21.15.151:0/23633049 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x564badba9900 0x564badbbba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:33.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.639+0000 7f4f9baabb80 1 --2- 172.21.15.151:0/23633049 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564badbaf850 0x564badbb87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:33.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.639+0000 7f4f9baabb80 1 --2- 172.21.15.151:0/23633049 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564badbaef30 0x564badbb82a0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:33.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.639+0000 7f4f9baabb80 1 -- 172.21.15.151:0/23633049 >> 172.21.15.151:0/23633049 conn(0x564badad7c60 msgr2=0x564badbb0340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:03:33.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.639+0000 7f4f6fe00700 0 client.17013 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T22:03:33.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.640+0000 7f4f9baabb80 1 -- 172.21.15.151:0/23633049 shutdown_connections 2024-05-21T22:03:33.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.640+0000 7f4f9baabb80 1 -- 172.21.15.151:0/23633049 wait complete. 2024-05-21T22:03:33.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.640+0000 7f4f9baabb80 20 client.17013 trim_cache size 0 max 16384 2024-05-21T22:03:33.642 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-05-21T22:03:33.642 INFO:teuthology.orchestra.run.smithi151.stdout: "return_code": 0, 2024-05-21T22:03:33.642 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_tag": "6286ad72-9657-4200-8178-a12a9934b0d0", 2024-05-21T22:03:33.642 INFO:teuthology.orchestra.run.smithi151.stdout: "mode": "asynchronous" 2024-05-21T22:03:33.643 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-05-21T22:03:33.643 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.640+0000 7f4f91200700 1 -- 172.21.15.151:0/3832259536 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f4f700b29f0 msgr2=0x7f4f700b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:33.643 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.640+0000 7f4f91200700 1 --2- 172.21.15.151:0/3832259536 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f4f700b29f0 0x7f4f700b4eb0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4f840060b0 tx=0x7f4f84004b40 comp rx=0 tx=0).stop 2024-05-21T22:03:33.643 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.641+0000 7f4f91200700 1 -- 172.21.15.151:0/3832259536 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4f8c1041a0 msgr2=0x7f4f8c19fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:33.643 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.641+0000 7f4f91200700 1 --2- 172.21.15.151:0/3832259536 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4f8c1041a0 0x7f4f8c19fcf0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f4f74006010 tx=0x7f4f740057c0 comp rx=0 tx=0).stop 2024-05-21T22:03:33.644 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.641+0000 7f4f91200700 1 -- 172.21.15.151:0/3832259536 shutdown_connections 2024-05-21T22:03:33.644 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.641+0000 7f4f91200700 1 --2- 172.21.15.151:0/3832259536 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f4f700b29f0 0x7f4f700b4eb0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:33.644 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.641+0000 7f4f91200700 1 --2- 172.21.15.151:0/3832259536 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4f8c1037f0 0x7f4f8c19f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:33.644 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.641+0000 7f4f91200700 1 --2- 172.21.15.151:0/3832259536 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4f8c1088f0 0x7f4f8c199970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:33.644 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.641+0000 7f4f91200700 1 --2- 172.21.15.151:0/3832259536 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4f8c1041a0 0x7f4f8c19fcf0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:33.644 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.641+0000 7f4f91200700 1 -- 172.21.15.151:0/3832259536 >> 172.21.15.151:0/3832259536 conn(0x7f4f8c07a6a0 msgr2=0x7f4f8c1018b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:03:33.644 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.641+0000 7f4f91200700 1 -- 172.21.15.151:0/3832259536 shutdown_connections 2024-05-21T22:03:33.644 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:33.641+0000 7f4f91200700 1 -- 172.21.15.151:0/3832259536 wait complete. 2024-05-21T22:03:33.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:03:33 smithi164 ceph-mon[36269]: pgmap v245: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T22:03:33.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:03:33 smithi184 ceph-mon[36942]: pgmap v245: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T22:03:34.549 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-21T22:03:34.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:03:34 smithi164 ceph-mon[36269]: scrub queued for path: / 2024-05-21T22:03:34.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:03:34 smithi164 ceph-mon[36269]: scrub summary: idle+waiting paths [/] 2024-05-21T22:03:34.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:03:34 smithi164 ceph-mon[36269]: scrub summary: active paths [/] 2024-05-21T22:03:34.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:03:34 smithi164 ceph-mon[36269]: scrub summary: idle+waiting paths [/] 2024-05-21T22:03:34.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:03:34 smithi164 ceph-mon[36269]: scrub summary: idle 2024-05-21T22:03:34.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:03:34 smithi151 ceph-mon[31623]: scrub queued for path: / 2024-05-21T22:03:34.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:03:34 smithi151 ceph-mon[31623]: scrub summary: idle+waiting paths [/] 2024-05-21T22:03:34.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:03:34 smithi151 ceph-mon[31623]: scrub summary: active paths [/] 2024-05-21T22:03:34.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:03:34 smithi151 ceph-mon[31623]: scrub summary: idle+waiting paths [/] 2024-05-21T22:03:34.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:03:34 smithi151 ceph-mon[31623]: scrub summary: idle 2024-05-21T22:03:34.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:03:34 smithi184 ceph-mon[36942]: scrub queued for path: / 2024-05-21T22:03:34.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:03:34 smithi184 ceph-mon[36942]: scrub summary: idle+waiting paths [/] 2024-05-21T22:03:34.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:03:34 smithi184 ceph-mon[36942]: scrub summary: active paths [/] 2024-05-21T22:03:34.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:03:34 smithi184 ceph-mon[36942]: scrub summary: idle+waiting paths [/] 2024-05-21T22:03:34.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:03:34 smithi184 ceph-mon[36942]: scrub summary: idle 2024-05-21T22:03:34.997 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:03:35.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:03:35 smithi164 ceph-mon[36269]: pgmap v246: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:03:35.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:03:35 smithi151 ceph-mon[31623]: pgmap v246: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:03:35.985 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:03:35 smithi184 ceph-mon[36942]: pgmap v246: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:03:37.435 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.433+0000 7fd4d8a00700 1 -- 172.21.15.151:0/2071430377 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4d406c400 msgr2=0x7fd4d410faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:37.435 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.433+0000 7fd4d8a00700 1 --2- 172.21.15.151:0/2071430377 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4d406c400 0x7fd4d410faa0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd4c00098f0 tx=0x7fd4c0009c00 comp rx=0 tx=0).stop 2024-05-21T22:03:37.435 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.433+0000 7fd4d8a00700 1 -- 172.21.15.151:0/2071430377 shutdown_connections 2024-05-21T22:03:37.435 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.433+0000 7fd4d8a00700 1 --2- 172.21.15.151:0/2071430377 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4d410ffe0 0x7fd4d41123d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:37.435 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.433+0000 7fd4d8a00700 1 --2- 172.21.15.151:0/2071430377 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd4d406bae0 0x7fd4d406bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:37.435 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.433+0000 7fd4d8a00700 1 --2- 172.21.15.151:0/2071430377 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4d406c400 0x7fd4d410faa0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:37.435 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.433+0000 7fd4d8a00700 1 -- 172.21.15.151:0/2071430377 >> 172.21.15.151:0/2071430377 conn(0x7fd4d40793b0 msgr2=0x7fd4d40797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:03:37.435 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.433+0000 7fd4d8a00700 1 -- 172.21.15.151:0/2071430377 shutdown_connections 2024-05-21T22:03:37.435 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.433+0000 7fd4d8a00700 1 -- 172.21.15.151:0/2071430377 wait complete. 2024-05-21T22:03:37.436 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.435+0000 7fd4d8a00700 1 Processor -- start 2024-05-21T22:03:37.436 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.435+0000 7fd4d8a00700 1 -- start start 2024-05-21T22:03:37.437 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.436+0000 7fd4d8a00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4d406bae0 0x7fd4d419bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:37.437 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.436+0000 7fd4d8a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4d406c400 0x7fd4d419c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:37.437 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.436+0000 7fd4d8a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd4d410ffe0 0x7fd4d41a0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:37.438 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.436+0000 7fd4d8a00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d41158f0 con 0x7fd4d410ffe0 2024-05-21T22:03:37.438 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.436+0000 7fd4d8a00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d4115770 con 0x7fd4d406c400 2024-05-21T22:03:37.438 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.436+0000 7fd4d8a00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d4115a70 con 0x7fd4d406bae0 2024-05-21T22:03:37.438 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.436+0000 7fd4d2800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd4d410ffe0 0x7fd4d41a0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:37.438 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.436+0000 7fd4d1e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4d406bae0 0x7fd4d419bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:37.438 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.436+0000 7fd4d2800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd4d410ffe0 0x7fd4d41a0790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:33274/0 (socket says 172.21.15.151:33274) 2024-05-21T22:03:37.438 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.436+0000 7fd4d2800700 1 -- 172.21.15.151:0/4045928373 learned_addr learned my addr 172.21.15.151:0/4045928373 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:03:37.438 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.436+0000 7fd4d1400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4d406c400 0x7fd4d419c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:37.439 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.437+0000 7fd4d2800700 1 -- 172.21.15.151:0/4045928373 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4d406bae0 msgr2=0x7fd4d419bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:37.439 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.437+0000 7fd4d2800700 1 --2- 172.21.15.151:0/4045928373 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4d406bae0 0x7fd4d419bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:37.439 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.437+0000 7fd4d2800700 1 -- 172.21.15.151:0/4045928373 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4d406c400 msgr2=0x7fd4d419c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:37.439 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.437+0000 7fd4d2800700 1 --2- 172.21.15.151:0/4045928373 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4d406c400 0x7fd4d419c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:37.439 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.437+0000 7fd4d2800700 1 -- 172.21.15.151:0/4045928373 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4d41a0ed0 con 0x7fd4d410ffe0 2024-05-21T22:03:37.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.437+0000 7fd4d1400700 1 --2- 172.21.15.151:0/4045928373 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4d406c400 0x7fd4d419c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:03:37.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.437+0000 7fd4d2800700 1 --2- 172.21.15.151:0/4045928373 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd4d410ffe0 0x7fd4d41a0790 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc00ef90 tx=0x7fd4cc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:03:37.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.437+0000 7fd4caa00700 1 -- 172.21.15.151:0/4045928373 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4cc0092e0 con 0x7fd4d410ffe0 2024-05-21T22:03:37.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.437+0000 7fd4d8a00700 1 -- 172.21.15.151:0/4045928373 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4d41a1160 con 0x7fd4d410ffe0 2024-05-21T22:03:37.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.437+0000 7fd4caa00700 1 -- 172.21.15.151:0/4045928373 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4cc004bb0 con 0x7fd4d410ffe0 2024-05-21T22:03:37.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.437+0000 7fd4caa00700 1 -- 172.21.15.151:0/4045928373 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4cc004280 con 0x7fd4d410ffe0 2024-05-21T22:03:37.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.438+0000 7fd4d8a00700 1 -- 172.21.15.151:0/4045928373 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4d41a8a40 con 0x7fd4d410ffe0 2024-05-21T22:03:37.442 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.441+0000 7fd4caa00700 1 -- 172.21.15.151:0/4045928373 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7fd4cc01b030 con 0x7fd4d410ffe0 2024-05-21T22:03:37.443 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.442+0000 7fd4e3361b80 1 Processor -- start 2024-05-21T22:03:37.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.442+0000 7fd4e3361b80 1 -- start start 2024-05-21T22:03:37.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.443+0000 7fd4e3361b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d0ab000eb0 0x55d0ab001290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:37.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.443+0000 7fd4e3361b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0ab0017d0 0x55d0aaffb260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:37.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.443+0000 7fd4e3361b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d0aaffb880 0x55d0aaffbce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:37.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.443+0000 7fd4e3361b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d0aaf59e40 con 0x55d0ab000eb0 2024-05-21T22:03:37.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.443+0000 7fd4e3361b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55d0aaf499e0 con 0x55d0aaffb880 2024-05-21T22:03:37.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.443+0000 7fd4e3361b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55d0aaf4da20 con 0x55d0ab0017d0 2024-05-21T22:03:37.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.443+0000 7fd4caa00700 1 --2- 172.21.15.151:0/4045928373 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fd4b80b2a70 0x7fd4b80b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:37.446 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.443+0000 7fd4d1e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0ab0017d0 0x55d0aaffb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:37.446 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.443+0000 7fd4d1400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d0ab000eb0 0x55d0ab001290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:37.446 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.443+0000 7fd4d1e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0ab0017d0 0x55d0aaffb260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:59382/0 (socket says 172.21.15.151:59382) 2024-05-21T22:03:37.446 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.443+0000 7fd4d1e00700 1 -- 172.21.15.151:0/613904491 learned_addr learned my addr 172.21.15.151:0/613904491 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:03:37.446 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.443+0000 7fd4d2800700 1 --2- 172.21.15.151:0/4045928373 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fd4b80b2a70 0x7fd4b80b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:37.446 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.443+0000 7fd4d1400700 1 --2- 172.21.15.151:0/613904491 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d0aaffb880 0x55d0aaffbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:37.446 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.443+0000 7fd4caa00700 1 -- 172.21.15.151:0/4045928373 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7fd4cc0e2060 con 0x7fd4d410ffe0 2024-05-21T22:03:37.446 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.444+0000 7fd4d1400700 1 -- 172.21.15.151:0/613904491 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0ab0017d0 msgr2=0x55d0aaffb260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:37.446 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.444+0000 7fd4d1400700 1 --2- 172.21.15.151:0/613904491 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0ab0017d0 0x55d0aaffb260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:37.446 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.444+0000 7fd4d1400700 1 -- 172.21.15.151:0/613904491 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d0aaffb880 msgr2=0x55d0aaffbce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:37.446 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.444+0000 7fd4d1400700 1 --2- 172.21.15.151:0/613904491 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d0aaffb880 0x55d0aaffbce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:37.447 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.444+0000 7fd4d1400700 1 -- 172.21.15.151:0/613904491 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0aaf12420 con 0x55d0ab000eb0 2024-05-21T22:03:37.447 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.444+0000 7fd4d1400700 1 --2- 172.21.15.151:0/613904491 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d0ab000eb0 0x55d0ab001290 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fd4c00098c0 tx=0x7fd4c0019040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b102b3098c7f32c server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:03:37.447 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.444+0000 7fd4d2800700 1 --2- 172.21.15.151:0/4045928373 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fd4b80b2a70 0x7fd4b80b4f30 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc009510 tx=0x7fd4cc009e70 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:03:37.447 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.445+0000 7fd4bf400700 1 -- 172.21.15.151:0/613904491 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c000f750 con 0x55d0ab000eb0 2024-05-21T22:03:37.447 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.445+0000 7fd4bf400700 1 -- 172.21.15.151:0/613904491 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4c000f8f0 con 0x55d0ab000eb0 2024-05-21T22:03:37.447 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.445+0000 7fd4bf400700 1 -- 172.21.15.151:0/613904491 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c0003c50 con 0x55d0ab000eb0 2024-05-21T22:03:37.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.446+0000 7fd4e3361b80 1 -- 172.21.15.151:0/613904491 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d0ab000eb0 msgr2=0x55d0ab001290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:37.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.446+0000 7fd4e3361b80 1 --2- 172.21.15.151:0/613904491 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d0ab000eb0 0x55d0ab001290 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fd4c00098c0 tx=0x7fd4c0019040 comp rx=0 tx=0).stop 2024-05-21T22:03:37.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.446+0000 7fd4e3361b80 1 -- 172.21.15.151:0/613904491 shutdown_connections 2024-05-21T22:03:37.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.446+0000 7fd4e3361b80 1 --2- 172.21.15.151:0/613904491 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d0aaffb880 0x55d0aaffbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:37.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.446+0000 7fd4e3361b80 1 --2- 172.21.15.151:0/613904491 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d0ab000eb0 0x55d0ab001290 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:37.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.446+0000 7fd4e3361b80 1 --2- 172.21.15.151:0/613904491 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0ab0017d0 0x55d0aaffb260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:37.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.446+0000 7fd4e3361b80 1 -- 172.21.15.151:0/613904491 >> 172.21.15.151:0/613904491 conn(0x55d0aaf29be0 msgr2=0x55d0aaff6010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:03:37.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.446+0000 7fd4e3361b80 1 -- 172.21.15.151:0/613904491 shutdown_connections 2024-05-21T22:03:37.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.446+0000 7fd4e3361b80 1 -- 172.21.15.151:0/613904491 wait complete. 2024-05-21T22:03:37.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.446+0000 7fd4e3361b80 1 -- start start 2024-05-21T22:03:37.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.447+0000 7fd4e3361b80 1 Processor -- start 2024-05-21T22:03:37.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.447+0000 7fd4e3361b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d0aaffb880 0x55d0ab007550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:37.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.447+0000 7fd4e3361b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d0ab000eb0 0x55d0ab0048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:37.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.447+0000 7fd4e3361b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0ab0017d0 0x55d0ab004e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:37.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.447+0000 7fd4e3361b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d0aae4ffa0 con 0x55d0aaffb880 2024-05-21T22:03:37.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.447+0000 7fd4e3361b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55d0aaeea760 con 0x55d0ab000eb0 2024-05-21T22:03:37.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.447+0000 7fd4e3361b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55d0ab0081e0 con 0x55d0ab0017d0 2024-05-21T22:03:37.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.447+0000 7fd4e3361b80 10 client.0 _ll_register_callbacks cb 0x55d0aaef3c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:03:37.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.448+0000 7fd4d1e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d0ab000eb0 0x55d0ab0048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:37.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.448+0000 7fd4d1400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d0aaffb880 0x55d0ab007550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:37.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.448+0000 7fd4d1e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d0ab000eb0 0x55d0ab0048f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:46962/0 (socket says 172.21.15.151:46962) 2024-05-21T22:03:37.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.448+0000 7fd4d1e00700 1 -- 172.21.15.151:0/349996151 learned_addr learned my addr 172.21.15.151:0/349996151 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:03:37.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.448+0000 7fd4d1400700 1 --2- 172.21.15.151:0/349996151 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0ab0017d0 0x55d0ab004e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:37.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.448+0000 7fd4d1e00700 1 -- 172.21.15.151:0/349996151 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0ab0017d0 msgr2=0x55d0ab004e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:37.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.448+0000 7fd4d1e00700 1 --2- 172.21.15.151:0/349996151 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0ab0017d0 0x55d0ab004e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:37.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.448+0000 7fd4d1e00700 1 -- 172.21.15.151:0/349996151 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d0aaffb880 msgr2=0x55d0ab007550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:37.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.448+0000 7fd4d1e00700 1 --2- 172.21.15.151:0/349996151 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d0aaffb880 0x55d0ab007550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:37.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.448+0000 7fd4d1e00700 1 -- 172.21.15.151:0/349996151 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0aae998d0 con 0x55d0ab000eb0 2024-05-21T22:03:37.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.448+0000 7fd4d1400700 1 --2- 172.21.15.151:0/349996151 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d0aaffb880 0x55d0ab007550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:03:37.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.448+0000 7fd4d1e00700 1 --2- 172.21.15.151:0/349996151 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d0ab000eb0 0x55d0ab0048f0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fd4c40128a0 tx=0x7fd4c4012bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3531cc5fdbfe84c server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:03:37.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.448+0000 7fd4b3e00700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-21T22:03:37.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.449+0000 7fd4b3e00700 1 -- 172.21.15.151:0/349996151 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c4004020 con 0x55d0ab000eb0 2024-05-21T22:03:37.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.449+0000 7fd4e3361b80 1 -- 172.21.15.151:0/349996151 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4c000f8f0 con 0x55d0ab000eb0 2024-05-21T22:03:37.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.449+0000 7fd4b3e00700 1 -- 172.21.15.151:0/349996151 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7fd4c40041c0 con 0x55d0ab000eb0 2024-05-21T22:03:37.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.449+0000 7fd4b3e00700 1 -- 172.21.15.151:0/349996151 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd4c4004400 con 0x55d0ab000eb0 2024-05-21T22:03:37.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.449+0000 7fd4e3361b80 10 client.27149 fetch_fsmap learned FSMap version 29 2024-05-21T22:03:37.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.449+0000 7fd4e3361b80 1 -- 172.21.15.151:0/349996151 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d0aaf1a060 con 0x55d0ab000eb0 2024-05-21T22:03:37.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.449+0000 7fd4b3e00700 1 -- 172.21.15.151:0/349996151 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4c000f8f0 con 0x55d0ab000eb0 2024-05-21T22:03:37.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.450+0000 7fd4b3e00700 1 -- 172.21.15.151:0/349996151 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fd4c400f700 con 0x55d0ab000eb0 2024-05-21T22:03:37.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.450+0000 7fd4e3361b80 10 client.27149 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:03:37.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.450+0000 7fd4e3361b80 10 client.27149 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T22:03:37.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.450+0000 7fd4e3361b80 20 client.27149 populate_metadata read hostname 'smithi151' 2024-05-21T22:03:37.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.450+0000 7fd4e3361b80 1 --2- 172.21.15.151:0/349996151 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55d0ab014de0 0x55d0ab0171d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:37.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.450+0000 7fd4e3361b80 4 client.27149 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T22:03:37.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.450+0000 7fd4e3361b80 1 -- 172.21.15.151:0/349996151 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d0aaf5bd90 con 0x55d0ab014de0 2024-05-21T22:03:37.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.450+0000 7fd4d1400700 1 --2- 172.21.15.151:0/349996151 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55d0ab014de0 0x55d0ab0171d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:37.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.451+0000 7fd4d1400700 1 --2- 172.21.15.151:0/349996151 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55d0ab014de0 0x55d0ab0171d0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db538240a1ce5b41 server_cookie=9401afd22577ac10 in_seq=0 out_seq=0 2024-05-21T22:03:37.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.451+0000 7fd4b3e00700 10 client.27149 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T22:03:37.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.451+0000 7fd4b3e00700 1 -- 172.21.15.151:0/349996151 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55d0aaf5bd90 con 0x55d0ab014de0 2024-05-21T22:03:37.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.451+0000 7fd4b3e00700 10 client.27149 handle_command_reply: tid=0 2024-05-21T22:03:37.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.451+0000 7fd4e3361b80 1 client.27149 shutdown 2024-05-21T22:03:37.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.452+0000 7fd4e3361b80 1 -- 172.21.15.151:0/349996151 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d0ab000eb0 msgr2=0x55d0ab0048f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:37.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.452+0000 7fd4e3361b80 1 --2- 172.21.15.151:0/349996151 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d0ab000eb0 0x55d0ab0048f0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fd4c40128a0 tx=0x7fd4c4012bb0 comp rx=0 tx=0).stop 2024-05-21T22:03:37.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.452+0000 7fd4e3361b80 1 -- 172.21.15.151:0/349996151 shutdown_connections 2024-05-21T22:03:37.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.452+0000 7fd4e3361b80 1 --2- 172.21.15.151:0/349996151 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0ab0017d0 0x55d0ab004e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:37.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.452+0000 7fd4e3361b80 1 --2- 172.21.15.151:0/349996151 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d0aaffb880 0x55d0ab007550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:37.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.452+0000 7fd4e3361b80 1 --2- 172.21.15.151:0/349996151 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55d0ab014de0 0x55d0ab0171d0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:37.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.452+0000 7fd4e3361b80 1 --2- 172.21.15.151:0/349996151 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d0ab000eb0 0x55d0ab0048f0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:37.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.452+0000 7fd4b3e00700 0 client.27149 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T22:03:37.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.452+0000 7fd4e3361b80 1 -- 172.21.15.151:0/349996151 >> 172.21.15.151:0/349996151 conn(0x55d0aaf29be0 msgr2=0x55d0aaffa250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:03:37.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.452+0000 7fd4e3361b80 1 -- 172.21.15.151:0/349996151 shutdown_connections 2024-05-21T22:03:37.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.452+0000 7fd4e3361b80 1 -- 172.21.15.151:0/349996151 wait complete. 2024-05-21T22:03:37.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.452+0000 7fd4e3361b80 20 client.27149 trim_cache size 0 max 16384 2024-05-21T22:03:37.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.474+0000 7fd4e3361b80 1 Processor -- start 2024-05-21T22:03:37.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.475+0000 7fd4e3361b80 1 -- start start 2024-05-21T22:03:37.477 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.475+0000 7fd4e3361b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d0aaffb880 0x55d0aaffb260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:37.477 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.475+0000 7fd4e3361b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0ab000eb0 0x55d0aafff0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:37.477 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.475+0000 7fd4e3361b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d0ab0017d0 0x55d0ab004f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:37.477 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.475+0000 7fd4e3361b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d41b0070 con 0x55d0ab0017d0 2024-05-21T22:03:37.477 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.475+0000 7fd4e3361b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d41b1bb0 con 0x55d0aaffb880 2024-05-21T22:03:37.477 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.475+0000 7fd4e3361b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d41b0740 con 0x55d0ab000eb0 2024-05-21T22:03:37.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.475+0000 7fd4d1e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0ab000eb0 0x55d0aafff0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:37.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.476+0000 7fd4d1e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0ab000eb0 0x55d0aafff0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:59398/0 (socket says 172.21.15.151:59398) 2024-05-21T22:03:37.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.476+0000 7fd4d1e00700 1 -- 172.21.15.151:0/1380678659 learned_addr learned my addr 172.21.15.151:0/1380678659 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:03:37.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.475+0000 7fd4d1400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d0ab0017d0 0x55d0ab004f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:37.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.476+0000 7fd4d1400700 1 --2- 172.21.15.151:0/1380678659 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d0aaffb880 0x55d0aaffb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:37.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.476+0000 7fd4d1e00700 1 -- 172.21.15.151:0/1380678659 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d0aaffb880 msgr2=0x55d0aaffb260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:37.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.476+0000 7fd4d1e00700 1 --2- 172.21.15.151:0/1380678659 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d0aaffb880 0x55d0aaffb260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:37.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.476+0000 7fd4d1e00700 1 -- 172.21.15.151:0/1380678659 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d0ab0017d0 msgr2=0x55d0ab004f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:37.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.476+0000 7fd4d1e00700 1 --2- 172.21.15.151:0/1380678659 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d0ab0017d0 0x55d0ab004f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:37.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.476+0000 7fd4d1e00700 1 -- 172.21.15.151:0/1380678659 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0aaff7d00 con 0x55d0ab000eb0 2024-05-21T22:03:37.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.476+0000 7fd4d1e00700 1 --2- 172.21.15.151:0/1380678659 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0ab000eb0 0x55d0aafff0d0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fd4c4005a80 tx=0x7fd4c4009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42d0dff132fa59cc server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:03:37.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.477+0000 7fd4bea00700 1 -- 172.21.15.151:0/1380678659 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c4029070 con 0x55d0ab000eb0 2024-05-21T22:03:37.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.477+0000 7fd4bea00700 1 -- 172.21.15.151:0/1380678659 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4c4005070 con 0x55d0ab000eb0 2024-05-21T22:03:37.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.477+0000 7fd4bea00700 1 -- 172.21.15.151:0/1380678659 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c4004930 con 0x55d0ab000eb0 2024-05-21T22:03:37.481 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.477+0000 7fd4e3361b80 1 -- 172.21.15.151:0/1380678659 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0ab000eb0 msgr2=0x55d0aafff0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:37.481 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.477+0000 7fd4e3361b80 1 --2- 172.21.15.151:0/1380678659 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0ab000eb0 0x55d0aafff0d0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fd4c4005a80 tx=0x7fd4c4009920 comp rx=0 tx=0).stop 2024-05-21T22:03:37.481 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.477+0000 7fd4e3361b80 1 -- 172.21.15.151:0/1380678659 shutdown_connections 2024-05-21T22:03:37.481 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.477+0000 7fd4e3361b80 1 --2- 172.21.15.151:0/1380678659 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d0aaffb880 0x55d0aaffb260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:37.481 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.477+0000 7fd4e3361b80 1 --2- 172.21.15.151:0/1380678659 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d0ab0017d0 0x55d0ab004f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:37.481 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.477+0000 7fd4e3361b80 1 --2- 172.21.15.151:0/1380678659 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0ab000eb0 0x55d0aafff0d0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:37.481 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.477+0000 7fd4e3361b80 1 -- 172.21.15.151:0/1380678659 >> 172.21.15.151:0/1380678659 conn(0x55d0aaf29be0 msgr2=0x55d0aaffe9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:03:37.481 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.477+0000 7fd4e3361b80 1 -- 172.21.15.151:0/1380678659 shutdown_connections 2024-05-21T22:03:37.481 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.478+0000 7fd4e3361b80 1 -- 172.21.15.151:0/1380678659 wait complete. 2024-05-21T22:03:37.481 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.478+0000 7fd4e3361b80 1 -- start start 2024-05-21T22:03:37.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.478+0000 7fd4e3361b80 1 Processor -- start 2024-05-21T22:03:37.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.479+0000 7fd4e3361b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d0aaffb880 0x55d0ab00d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:37.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.479+0000 7fd4e3361b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0ab000eb0 0x55d0ab00a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:37.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.479+0000 7fd4e3361b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d0ab0017d0 0x55d0ab00a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:37.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.479+0000 7fd4e3361b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d41b14e0 con 0x55d0ab0017d0 2024-05-21T22:03:37.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.479+0000 7fd4e3361b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d41b0e10 con 0x55d0aaffb880 2024-05-21T22:03:37.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.479+0000 7fd4e3361b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d41ae030 con 0x55d0ab000eb0 2024-05-21T22:03:37.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.479+0000 7fd4e3361b80 10 client.0 _ll_register_callbacks cb 0x55d0aaef3c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:03:37.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.479+0000 7fd4d1400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d0ab0017d0 0x55d0ab00a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:37.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.479+0000 7fd4d1400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d0ab0017d0 0x55d0ab00a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:33316/0 (socket says 172.21.15.151:33316) 2024-05-21T22:03:37.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.479+0000 7fd4d1400700 1 -- 172.21.15.151:0/3294448506 learned_addr learned my addr 172.21.15.151:0/3294448506 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:03:37.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.479+0000 7fd4d1400700 1 --2- 172.21.15.151:0/3294448506 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d0aaffb880 0x55d0ab00d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:37.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.479+0000 7fd4d1e00700 1 --2- 172.21.15.151:0/3294448506 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0ab000eb0 0x55d0ab00a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:37.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.480+0000 7fd4d1400700 1 -- 172.21.15.151:0/3294448506 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0ab000eb0 msgr2=0x55d0ab00a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:37.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.480+0000 7fd4d1400700 1 --2- 172.21.15.151:0/3294448506 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0ab000eb0 0x55d0ab00a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:37.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.480+0000 7fd4d1400700 1 -- 172.21.15.151:0/3294448506 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d0aaffb880 msgr2=0x55d0ab00d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:37.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.480+0000 7fd4d1400700 1 --2- 172.21.15.151:0/3294448506 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d0aaffb880 0x55d0ab00d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:37.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.480+0000 7fd4d1400700 1 -- 172.21.15.151:0/3294448506 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0ab00af40 con 0x55d0ab0017d0 2024-05-21T22:03:37.484 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.480+0000 7fd4d1400700 1 --2- 172.21.15.151:0/3294448506 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d0ab0017d0 0x55d0ab00a980 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fd4c0005e20 tx=0x7fd4c0019040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6548071522737b34 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:03:37.485 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.480+0000 7fd4b3e00700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-21T22:03:37.485 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.480+0000 7fd4b3e00700 1 -- 172.21.15.151:0/3294448506 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c0003a70 con 0x55d0ab0017d0 2024-05-21T22:03:37.485 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.480+0000 7fd4b3e00700 1 -- 172.21.15.151:0/3294448506 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7fd4c0020040 con 0x55d0ab0017d0 2024-05-21T22:03:37.485 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.480+0000 7fd4e3361b80 1 -- 172.21.15.151:0/3294448506 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4c4005070 con 0x55d0ab0017d0 2024-05-21T22:03:37.485 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.480+0000 7fd4b3e00700 1 -- 172.21.15.151:0/3294448506 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd4c0003c10 con 0x55d0ab0017d0 2024-05-21T22:03:37.485 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.480+0000 7fd4e3361b80 10 client.17031 fetch_fsmap learned FSMap version 29 2024-05-21T22:03:37.485 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.480+0000 7fd4e3361b80 1 -- 172.21.15.151:0/3294448506 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d0ab0806c0 con 0x55d0ab0017d0 2024-05-21T22:03:37.485 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.481+0000 7fd4b3e00700 1 -- 172.21.15.151:0/3294448506 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4c4005070 con 0x55d0ab0017d0 2024-05-21T22:03:37.485 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.481+0000 7fd4b3e00700 1 -- 172.21.15.151:0/3294448506 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fd4c0026040 con 0x55d0ab0017d0 2024-05-21T22:03:37.485 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.481+0000 7fd4e3361b80 10 client.17031 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:03:37.485 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.481+0000 7fd4e3361b80 10 client.17031 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T22:03:37.485 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.481+0000 7fd4e3361b80 20 client.17031 populate_metadata read hostname 'smithi151' 2024-05-21T22:03:37.485 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.481+0000 7fd4e3361b80 1 --2- 172.21.15.151:0/3294448506 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55d0ab0809b0 0x55d0ab082da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:37.485 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.481+0000 7fd4e3361b80 4 client.17031 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-21T22:03:37.485 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.481+0000 7fd4e3361b80 1 -- 172.21.15.151:0/3294448506 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d0aaf55570 con 0x55d0ab0809b0 2024-05-21T22:03:37.486 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.482+0000 7fd4d1e00700 1 --2- 172.21.15.151:0/3294448506 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55d0ab0809b0 0x55d0ab082da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:37.486 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.482+0000 7fd4d1e00700 1 --2- 172.21.15.151:0/3294448506 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55d0ab0809b0 0x55d0ab082da0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=343728d251d63d35 server_cookie=8905a41c7e9e5dd5 in_seq=0 out_seq=0 2024-05-21T22:03:37.486 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.482+0000 7fd4b3e00700 10 client.17031 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T22:03:37.486 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.482+0000 7fd4b3e00700 1 -- 172.21.15.151:0/3294448506 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55d0aaf55570 con 0x55d0ab0809b0 2024-05-21T22:03:37.486 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.482+0000 7fd4b3e00700 10 client.17031 handle_command_reply: tid=0 2024-05-21T22:03:37.486 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.482+0000 7fd4e3361b80 1 client.17031 shutdown 2024-05-21T22:03:37.486 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.482+0000 7fd4e3361b80 1 -- 172.21.15.151:0/3294448506 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d0ab0017d0 msgr2=0x55d0ab00a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:37.486 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.482+0000 7fd4e3361b80 1 --2- 172.21.15.151:0/3294448506 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d0ab0017d0 0x55d0ab00a980 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fd4c0005e20 tx=0x7fd4c0019040 comp rx=0 tx=0).stop 2024-05-21T22:03:37.486 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.483+0000 7fd4e3361b80 1 -- 172.21.15.151:0/3294448506 shutdown_connections 2024-05-21T22:03:37.486 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.483+0000 7fd4e3361b80 1 --2- 172.21.15.151:0/3294448506 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55d0ab0809b0 0x55d0ab082da0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:37.486 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.483+0000 7fd4e3361b80 1 --2- 172.21.15.151:0/3294448506 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d0aaffb880 0x55d0ab00d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:37.486 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.483+0000 7fd4e3361b80 1 --2- 172.21.15.151:0/3294448506 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d0ab0017d0 0x55d0ab00a980 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:37.487 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.483+0000 7fd4e3361b80 1 --2- 172.21.15.151:0/3294448506 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0ab000eb0 0x55d0ab00a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:37.487 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.483+0000 7fd4e3361b80 1 -- 172.21.15.151:0/3294448506 >> 172.21.15.151:0/3294448506 conn(0x55d0aaf29be0 msgr2=0x55d0ab001bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:03:37.487 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.483+0000 7fd4b3e00700 0 client.17031 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T22:03:37.487 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.483+0000 7fd4e3361b80 1 -- 172.21.15.151:0/3294448506 shutdown_connections 2024-05-21T22:03:37.487 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.483+0000 7fd4e3361b80 1 -- 172.21.15.151:0/3294448506 wait complete. 2024-05-21T22:03:37.487 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.483+0000 7fd4e3361b80 20 client.17031 trim_cache size 0 max 16384 2024-05-21T22:03:37.487 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-05-21T22:03:37.487 INFO:teuthology.orchestra.run.smithi151.stdout: "status": "no active scrubs running", 2024-05-21T22:03:37.487 INFO:teuthology.orchestra.run.smithi151.stdout: "scrubs": {} 2024-05-21T22:03:37.487 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-05-21T22:03:37.488 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.484+0000 7fd4d8a00700 1 -- 172.21.15.151:0/4045928373 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fd4b80b2a70 msgr2=0x7fd4b80b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:37.488 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.484+0000 7fd4d8a00700 1 --2- 172.21.15.151:0/4045928373 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fd4b80b2a70 0x7fd4b80b4f30 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc009510 tx=0x7fd4cc009e70 comp rx=0 tx=0).stop 2024-05-21T22:03:37.488 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.484+0000 7fd4d8a00700 1 -- 172.21.15.151:0/4045928373 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd4d410ffe0 msgr2=0x7fd4d41a0790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:37.488 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.484+0000 7fd4d8a00700 1 --2- 172.21.15.151:0/4045928373 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd4d410ffe0 0x7fd4d41a0790 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc00ef90 tx=0x7fd4cc00c3a0 comp rx=0 tx=0).stop 2024-05-21T22:03:37.488 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.484+0000 7fd4d8a00700 1 -- 172.21.15.151:0/4045928373 shutdown_connections 2024-05-21T22:03:37.489 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.484+0000 7fd4d8a00700 1 --2- 172.21.15.151:0/4045928373 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fd4b80b2a70 0x7fd4b80b4f30 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:37.489 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.484+0000 7fd4d8a00700 1 --2- 172.21.15.151:0/4045928373 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd4d410ffe0 0x7fd4d41a0790 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:37.489 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.484+0000 7fd4d8a00700 1 --2- 172.21.15.151:0/4045928373 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4d406bae0 0x7fd4d419bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:37.489 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.484+0000 7fd4d8a00700 1 --2- 172.21.15.151:0/4045928373 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4d406c400 0x7fd4d419c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:37.489 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.484+0000 7fd4d8a00700 1 -- 172.21.15.151:0/4045928373 >> 172.21.15.151:0/4045928373 conn(0x7fd4d40793b0 msgr2=0x7fd4d4111710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:03:37.489 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.484+0000 7fd4d8a00700 1 -- 172.21.15.151:0/4045928373 shutdown_connections 2024-05-21T22:03:37.489 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:37.484+0000 7fd4d8a00700 1 -- 172.21.15.151:0/4045928373 wait complete. 2024-05-21T22:03:37.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:03:37 smithi164 ceph-mon[36269]: pgmap v247: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:03:37.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:03:37 smithi151 ceph-mon[31623]: pgmap v247: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:03:37.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:03:37 smithi184 ceph-mon[36942]: pgmap v247: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:03:38.488 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-21T22:03:38.489 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-21T22:03:38.934 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:03:39.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:03:39 smithi151 ceph-mon[31623]: pgmap v248: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-05-21T22:03:39.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:03:39 smithi164 ceph-mon[36269]: pgmap v248: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-05-21T22:03:39.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:03:39 smithi184 ceph-mon[36942]: pgmap v248: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-05-21T22:03:41.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:41.237+0000 7f017fe00700 1 -- 172.21.15.151:0/2506281841 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0178103f90 msgr2=0x7f0178110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:41.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:41.237+0000 7f017fe00700 1 --2- 172.21.15.151:0/2506281841 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0178103f90 0x7f0178110470 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f0170009f90 tx=0x7f017000b330 comp rx=0 tx=0).stop 2024-05-21T22:03:41.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:41.238+0000 7f017fe00700 1 -- 172.21.15.151:0/2506281841 shutdown_connections 2024-05-21T22:03:41.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:41.238+0000 7f017fe00700 1 --2- 172.21.15.151:0/2506281841 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0178103f90 0x7f0178110470 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:41.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:41.238+0000 7f017fe00700 1 --2- 172.21.15.151:0/2506281841 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0178108500 0x7f01781088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:41.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:41.238+0000 7f017fe00700 1 --2- 172.21.15.151:0/2506281841 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f01781035f0 0x7f0178103a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:41.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:41.238+0000 7f017fe00700 1 -- 172.21.15.151:0/2506281841 >> 172.21.15.151:0/2506281841 conn(0x7f017807a6a0 msgr2=0x7f017807aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:03:41.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:41.238+0000 7f017fe00700 1 -- 172.21.15.151:0/2506281841 shutdown_connections 2024-05-21T22:03:41.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:41.238+0000 7f017fe00700 1 -- 172.21.15.151:0/2506281841 wait complete. 2024-05-21T22:03:41.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:41.239+0000 7f017fe00700 1 Processor -- start 2024-05-21T22:03:41.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:41.239+0000 7f017fe00700 1 -- start start 2024-05-21T22:03:41.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:41.239+0000 7f017fe00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f01781035f0 0x7f017819bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:41.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:41.239+0000 7f017fe00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0178103f90 0x7f017819c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:41.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:41.239+0000 7f017fe00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0178108500 0x7f01781a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:41.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:41.239+0000 7f017fe00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0178113790 con 0x7f01781035f0 2024-05-21T22:03:41.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:41.239+0000 7f017fe00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0178113610 con 0x7f0178103f90 2024-05-21T22:03:41.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:41.239+0000 7f017fe00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0178113910 con 0x7f0178108500 2024-05-21T22:03:41.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:41.240+0000 7f017d400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f01781035f0 0x7f017819bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:41.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:41.240+0000 7f017d400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f01781035f0 0x7f017819bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:33322/0 (socket says 172.21.15.151:33322) 2024-05-21T22:03:41.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:41.240+0000 7f017d400700 1 -- 172.21.15.151:0/1960871984 learned_addr learned my addr 172.21.15.151:0/1960871984 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:03:41.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:41.240+0000 7f017ca00700 1 --2- 172.21.15.151:0/1960871984 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0178103f90 0x7f017819c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:41.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:41.240+0000 7f017de00700 1 --2- 172.21.15.151:0/1960871984 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0178108500 0x7f01781a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:41.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:41.240+0000 7f017ca00700 1 -- 172.21.15.151:0/1960871984 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0178108500 msgr2=0x7f01781a06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:41.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:41.240+0000 7f017ca00700 1 --2- 172.21.15.151:0/1960871984 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0178108500 0x7f01781a06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:41.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:41.240+0000 7f017ca00700 1 -- 172.21.15.151:0/1960871984 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f01781035f0 msgr2=0x7f017819bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:41.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:41.240+0000 7f017ca00700 1 --2- 172.21.15.151:0/1960871984 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f01781035f0 0x7f017819bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:41.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:41.240+0000 7f017ca00700 1 -- 172.21.15.151:0/1960871984 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01781a0de0 con 0x7f0178103f90 2024-05-21T22:03:41.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:41.240+0000 7f017d400700 1 --2- 172.21.15.151:0/1960871984 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f01781035f0 0x7f017819bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:03:41.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:41.240+0000 7f017de00700 1 --2- 172.21.15.151:0/1960871984 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0178108500 0x7f01781a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:03:41.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:41.241+0000 7f017ca00700 1 --2- 172.21.15.151:0/1960871984 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0178103f90 0x7f017819c310 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f016400e9f0 tx=0x7f016400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:03:41.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:41.241+0000 7f0176000700 1 -- 172.21.15.151:0/1960871984 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f016400c9c0 con 0x7f0178103f90 2024-05-21T22:03:41.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:41.241+0000 7f017fe00700 1 -- 172.21.15.151:0/1960871984 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01781a10d0 con 0x7f0178103f90 2024-05-21T22:03:41.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:41.241+0000 7f0176000700 1 -- 172.21.15.151:0/1960871984 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f016400cb60 con 0x7f0178103f90 2024-05-21T22:03:41.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:41.241+0000 7f0176000700 1 -- 172.21.15.151:0/1960871984 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0164018680 con 0x7f0178103f90 2024-05-21T22:03:41.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:41.241+0000 7f017fe00700 1 -- 172.21.15.151:0/1960871984 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01781a8950 con 0x7f0178103f90 2024-05-21T22:03:41.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:41.242+0000 7f017fe00700 1 -- 172.21.15.151:0/1960871984 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f017806a230 con 0x7f0178103f90 2024-05-21T22:03:41.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:41.244+0000 7f0176000700 1 -- 172.21.15.151:0/1960871984 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7f0164010430 con 0x7f0178103f90 2024-05-21T22:03:41.247 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:41.246+0000 7f0176000700 1 --2- 172.21.15.151:0/1960871984 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f01600b29f0 0x7f01600b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:41.247 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:41.246+0000 7f0176000700 1 -- 172.21.15.151:0/1960871984 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7f01640e27e0 con 0x7f0178103f90 2024-05-21T22:03:41.247 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:41.246+0000 7f017d400700 1 --2- 172.21.15.151:0/1960871984 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f01600b29f0 0x7f01600b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:41.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:41.247+0000 7f017d400700 1 --2- 172.21.15.151:0/1960871984 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f01600b29f0 0x7f01600b4eb0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f016c009750 tx=0x7f016c009310 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:03:41.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:41.251+0000 7f0176000700 1 -- 172.21.15.151:0/1960871984 <== mon.2 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f01640aa150 con 0x7f0178103f90 2024-05-21T22:03:41.540 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:41.538+0000 7f017fe00700 1 -- 172.21.15.151:0/1960871984 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f01781a8b50 con 0x7f0178103f90 2024-05-21T22:03:41.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:41.540+0000 7f0176000700 1 -- 172.21.15.151:0/1960871984 <== mon.2 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11958 (secure 0 0 0) 0x7f01640b21d0 con 0x7f0178103f90 2024-05-21T22:03:41.542 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T22:03:41.542 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14358,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/158010529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":158010529},{"type":"v1","addr":"172.21.15.151:6835","nonce":158010529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6839/2069058115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":2069058115},{"type":"v1","addr":"172.21.15.151:6839","nonce":2069058115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T21:51:19.552069+0000","modified":"2024-05-21T21:55:34.643755+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24530,"mds_1":24493,"mds_2":24491,"mds_3":24478,"mds_4":24473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14403":{"gid":14403,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6841/1749723794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6840","nonce":1749723794},{"type":"v1","addr":"172.21.15.151:6841","nonce":1749723794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/2026266114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":2026266114},{"type":"v1","addr":"172.21.15.184:6833","nonce":2026266114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/2467900844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":2467900844},{"type":"v1","addr":"172.21.15.164:6833","nonce":2467900844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3367640960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3367640960},{"type":"v1","addr":"172.21.15.151:6837","nonce":3367640960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/3264142621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3264142621},{"type":"v1","addr":"172.21.15.184:6835","nonce":3264142621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":17,"addr":"172.21.15.164:6835/2340428470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2340428470},{"type":"v1","addr":"172.21.15.164:6835","nonce":2340428470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":16,"addr":"172.21.15.184:6837/2988194888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":2988194888},{"type":"v1","addr":"172.21.15.184:6837","nonce":2988194888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.164:6837/1554609572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1554609572},{"type":"v1","addr":"172.21.15.164:6837","nonce":1554609572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.184:6839/3059935143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":3059935143},{"type":"v1","addr":"172.21.15.184:6839","nonce":3059935143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.164:6839/2646401247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":2646401247},{"type":"v1","addr":"172.21.15.164:6839","nonce":2646401247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24530,"qdb_cluster":[24530,24493,24491,24473,24478]},"id":1}]} 2024-05-21T22:03:41.544 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:41.543+0000 7f017fe00700 1 -- 172.21.15.151:0/1960871984 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f01600b29f0 msgr2=0x7f01600b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:41.544 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:41.543+0000 7f017fe00700 1 --2- 172.21.15.151:0/1960871984 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f01600b29f0 0x7f01600b4eb0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f016c009750 tx=0x7f016c009310 comp rx=0 tx=0).stop 2024-05-21T22:03:41.544 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:41.543+0000 7f017fe00700 1 -- 172.21.15.151:0/1960871984 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0178103f90 msgr2=0x7f017819c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:41.544 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:41.543+0000 7f017fe00700 1 --2- 172.21.15.151:0/1960871984 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0178103f90 0x7f017819c310 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f016400e9f0 tx=0x7f016400ed00 comp rx=0 tx=0).stop 2024-05-21T22:03:41.545 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:41.543+0000 7f017fe00700 1 -- 172.21.15.151:0/1960871984 shutdown_connections 2024-05-21T22:03:41.545 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:41.543+0000 7f017fe00700 1 --2- 172.21.15.151:0/1960871984 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f01600b29f0 0x7f01600b4eb0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:41.545 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:41.543+0000 7f017fe00700 1 --2- 172.21.15.151:0/1960871984 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0178108500 0x7f01781a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:41.545 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:41.543+0000 7f017fe00700 1 --2- 172.21.15.151:0/1960871984 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f01781035f0 0x7f017819bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:41.545 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:41.543+0000 7f017fe00700 1 --2- 172.21.15.151:0/1960871984 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0178103f90 0x7f017819c310 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:41.545 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:41.543+0000 7f017fe00700 1 -- 172.21.15.151:0/1960871984 >> 172.21.15.151:0/1960871984 conn(0x7f017807a6a0 msgr2=0x7f0178101460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:03:41.545 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:41.543+0000 7f017fe00700 1 -- 172.21.15.151:0/1960871984 shutdown_connections 2024-05-21T22:03:41.545 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:41.543+0000 7f017fe00700 1 -- 172.21.15.151:0/1960871984 wait complete. 2024-05-21T22:03:41.546 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 29 2024-05-21T22:03:41.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:03:41 smithi164 ceph-mon[36269]: pgmap v249: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-05-21T22:03:41.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:03:41 smithi151 ceph-mon[31623]: pgmap v249: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-05-21T22:03:41.985 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:03:41 smithi184 ceph-mon[36942]: pgmap v249: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-05-21T22:03:42.483 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-21T22:03:42.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:03:42 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/1960871984' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T22:03:42.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:03:42 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/1960871984' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T22:03:42.929 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:03:42.985 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:03:42 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/1960871984' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T22:03:43.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:03:43 smithi151 ceph-mon[31623]: pgmap v250: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-05-21T22:03:43.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:03:43 smithi164 ceph-mon[36269]: pgmap v250: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-05-21T22:03:43.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:03:43 smithi184 ceph-mon[36942]: pgmap v250: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-05-21T22:03:44.966 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.964+0000 7fdabb200700 1 -- 172.21.15.151:0/2442459632 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdab41060f0 msgr2=0x7fdab4112600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:44.966 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.964+0000 7fdabb200700 1 --2- 172.21.15.151:0/2442459632 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdab41060f0 0x7fdab4112600 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fdaac00b0a0 tx=0x7fdaac00b3b0 comp rx=0 tx=0).stop 2024-05-21T22:03:44.966 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.964+0000 7fdabb200700 1 -- 172.21.15.151:0/2442459632 shutdown_connections 2024-05-21T22:03:44.966 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.964+0000 7fdabb200700 1 --2- 172.21.15.151:0/2442459632 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdab410b730 0x7fdab410bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:44.966 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.964+0000 7fdabb200700 1 --2- 172.21.15.151:0/2442459632 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdab41060f0 0x7fdab4112600 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:44.966 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.964+0000 7fdabb200700 1 --2- 172.21.15.151:0/2442459632 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdab4105730 0x7fdab4105bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:44.966 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.964+0000 7fdabb200700 1 -- 172.21.15.151:0/2442459632 >> 172.21.15.151:0/2442459632 conn(0x7fdab4101220 msgr2=0x7fdab4103640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:03:44.966 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.964+0000 7fdabb200700 1 -- 172.21.15.151:0/2442459632 shutdown_connections 2024-05-21T22:03:44.966 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.964+0000 7fdabb200700 1 -- 172.21.15.151:0/2442459632 wait complete. 2024-05-21T22:03:44.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.965+0000 7fdabb200700 1 Processor -- start 2024-05-21T22:03:44.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.965+0000 7fdabb200700 1 -- start start 2024-05-21T22:03:44.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.966+0000 7fdabb200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdab4105730 0x7fdab4077a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:44.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.966+0000 7fdabb200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdab41060f0 0x7fdab4077f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:44.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.966+0000 7fdabb200700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdab410b730 0x7fdab40784d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:44.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.966+0000 7fdabb200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdab4115980 con 0x7fdab41060f0 2024-05-21T22:03:44.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.966+0000 7fdabb200700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdab4115800 con 0x7fdab4105730 2024-05-21T22:03:44.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.966+0000 7fdabb200700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fdab4115b00 con 0x7fdab410b730 2024-05-21T22:03:44.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.966+0000 7fdab3e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdab4105730 0x7fdab4077a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:44.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.966+0000 7fdab3400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdab41060f0 0x7fdab4077f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:44.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.966+0000 7fdab9200700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdab410b730 0x7fdab40784d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:44.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.966+0000 7fdab3e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdab4105730 0x7fdab4077a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:47000/0 (socket says 172.21.15.151:47000) 2024-05-21T22:03:44.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.966+0000 7fdab3e00700 1 -- 172.21.15.151:0/1828888261 learned_addr learned my addr 172.21.15.151:0/1828888261 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:03:44.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.966+0000 7fdab3e00700 1 -- 172.21.15.151:0/1828888261 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdab410b730 msgr2=0x7fdab40784d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:44.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.966+0000 7fdab3e00700 1 --2- 172.21.15.151:0/1828888261 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdab410b730 0x7fdab40784d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:44.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.966+0000 7fdab3e00700 1 -- 172.21.15.151:0/1828888261 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdab41060f0 msgr2=0x7fdab4077f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:44.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.966+0000 7fdab3e00700 1 --2- 172.21.15.151:0/1828888261 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdab41060f0 0x7fdab4077f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:44.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.966+0000 7fdab3e00700 1 -- 172.21.15.151:0/1828888261 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdab41ac4a0 con 0x7fdab4105730 2024-05-21T22:03:44.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.966+0000 7fdab9200700 1 --2- 172.21.15.151:0/1828888261 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdab410b730 0x7fdab40784d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:03:44.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.967+0000 7fdab3e00700 1 --2- 172.21.15.151:0/1828888261 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdab4105730 0x7fdab4077a50 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fdaa400eb30 tx=0x7fdaa400ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:03:44.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.967+0000 7fdab0c00700 1 -- 172.21.15.151:0/1828888261 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa400cbd0 con 0x7fdab4105730 2024-05-21T22:03:44.970 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.967+0000 7fdabb200700 1 -- 172.21.15.151:0/1828888261 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdab41ac790 con 0x7fdab4105730 2024-05-21T22:03:44.970 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.967+0000 7fdab0c00700 1 -- 172.21.15.151:0/1828888261 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdaa40044d0 con 0x7fdab4105730 2024-05-21T22:03:44.970 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.967+0000 7fdab0c00700 1 -- 172.21.15.151:0/1828888261 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa4003e30 con 0x7fdab4105730 2024-05-21T22:03:44.970 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.967+0000 7fdabb200700 1 -- 172.21.15.151:0/1828888261 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdab41acd20 con 0x7fdab4105730 2024-05-21T22:03:44.972 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.971+0000 7fdab0c00700 1 -- 172.21.15.151:0/1828888261 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7fdaa4004000 con 0x7fdab4105730 2024-05-21T22:03:44.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.972+0000 7fdac5b7eb80 1 Processor -- start 2024-05-21T22:03:44.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.972+0000 7fdac5b7eb80 1 -- start start 2024-05-21T22:03:44.974 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.972+0000 7fdac5b7eb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562ca5548610 0x562ca55489f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:44.974 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.972+0000 7fdac5b7eb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562ca5548f30 0x562ca5542b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:44.974 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.972+0000 7fdac5b7eb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ca5543140 0x562ca55435c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:44.974 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.972+0000 7fdac5b7eb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x562ca54a0e40 con 0x562ca5548f30 2024-05-21T22:03:44.974 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.972+0000 7fdac5b7eb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x562ca54909e0 con 0x562ca5543140 2024-05-21T22:03:44.974 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.972+0000 7fdac5b7eb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x562ca5494a20 con 0x562ca5548610 2024-05-21T22:03:44.974 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.972+0000 7fdab0c00700 1 --2- 172.21.15.151:0/1828888261 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fda980b2b20 0x7fda980b4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:44.974 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.973+0000 7fdab3e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562ca5548f30 0x562ca5542b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:44.975 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.973+0000 7fdab3e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562ca5548f30 0x562ca5542b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:33354/0 (socket says 172.21.15.151:33354) 2024-05-21T22:03:44.975 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.973+0000 7fdab3e00700 1 -- 172.21.15.151:0/3058587729 learned_addr learned my addr 172.21.15.151:0/3058587729 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:03:44.976 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.973+0000 7fdab3400700 1 --2- 172.21.15.151:0/3058587729 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ca5543140 0x562ca55435c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:44.976 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.973+0000 7fdab0c00700 1 -- 172.21.15.151:0/1828888261 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7fdaa40e2930 con 0x7fdab4105730 2024-05-21T22:03:44.976 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.973+0000 7fdab9200700 1 --2- 172.21.15.151:0/3058587729 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562ca5548610 0x562ca55489f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:44.976 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.973+0000 7fdab9200700 1 --2- 172.21.15.151:0/1828888261 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fda980b2b20 0x7fda980b4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:44.976 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.973+0000 7fdab3400700 1 -- 172.21.15.151:0/3058587729 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562ca5548610 msgr2=0x562ca55489f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:44.976 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.973+0000 7fdab3400700 1 --2- 172.21.15.151:0/3058587729 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562ca5548610 0x562ca55489f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:44.976 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.973+0000 7fdab3400700 1 -- 172.21.15.151:0/3058587729 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562ca5548f30 msgr2=0x562ca5542b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:44.976 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.973+0000 7fdab3400700 1 --2- 172.21.15.151:0/3058587729 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562ca5548f30 0x562ca5542b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:44.976 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.973+0000 7fdab3400700 1 -- 172.21.15.151:0/3058587729 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ca5459420 con 0x562ca5543140 2024-05-21T22:03:44.976 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.974+0000 7fdab3400700 1 --2- 172.21.15.151:0/3058587729 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ca5543140 0x562ca55435c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fdab4079250 tx=0x7fdaa0009f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec2345d94ef40014 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:03:44.976 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.974+0000 7fdaa9600700 1 -- 172.21.15.151:0/3058587729 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa0011d00 con 0x562ca5543140 2024-05-21T22:03:44.976 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.974+0000 7fdaa9600700 1 -- 172.21.15.151:0/3058587729 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdaa0004500 con 0x562ca5543140 2024-05-21T22:03:44.976 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.974+0000 7fdaa9600700 1 -- 172.21.15.151:0/3058587729 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa0005280 con 0x562ca5543140 2024-05-21T22:03:44.976 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.974+0000 7fdab9200700 1 --2- 172.21.15.151:0/1828888261 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fda980b2b20 0x7fda980b4fe0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fdaac00b780 tx=0x7fdaac006210 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:03:44.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.975+0000 7fdac5b7eb80 1 -- 172.21.15.151:0/3058587729 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ca5543140 msgr2=0x562ca55435c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:44.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.975+0000 7fdac5b7eb80 1 --2- 172.21.15.151:0/3058587729 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ca5543140 0x562ca55435c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fdab4079250 tx=0x7fdaa0009f40 comp rx=0 tx=0).stop 2024-05-21T22:03:44.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.975+0000 7fdac5b7eb80 1 -- 172.21.15.151:0/3058587729 shutdown_connections 2024-05-21T22:03:44.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.975+0000 7fdac5b7eb80 1 --2- 172.21.15.151:0/3058587729 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ca5543140 0x562ca55435c0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:44.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.975+0000 7fdac5b7eb80 1 --2- 172.21.15.151:0/3058587729 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562ca5548610 0x562ca55489f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:44.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.975+0000 7fdac5b7eb80 1 --2- 172.21.15.151:0/3058587729 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562ca5548f30 0x562ca5542b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:44.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.975+0000 7fdac5b7eb80 1 -- 172.21.15.151:0/3058587729 >> 172.21.15.151:0/3058587729 conn(0x562ca5470be0 msgr2=0x562ca553d690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:03:44.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.975+0000 7fdac5b7eb80 1 -- 172.21.15.151:0/3058587729 shutdown_connections 2024-05-21T22:03:44.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.975+0000 7fdac5b7eb80 1 -- 172.21.15.151:0/3058587729 wait complete. 2024-05-21T22:03:44.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.976+0000 7fdac5b7eb80 1 -- start start 2024-05-21T22:03:44.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.976+0000 7fdac5b7eb80 1 Processor -- start 2024-05-21T22:03:44.979 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.977+0000 7fdac5b7eb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562ca5543140 0x562ca554ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:44.979 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.977+0000 7fdac5b7eb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ca5548610 0x562ca554c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:44.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.977+0000 7fdac5b7eb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562ca5548f30 0x562ca554c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:44.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.977+0000 7fdac5b7eb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x562ca5396fa0 con 0x562ca5543140 2024-05-21T22:03:44.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.977+0000 7fdac5b7eb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x562ca5431760 con 0x562ca5548610 2024-05-21T22:03:44.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.977+0000 7fdac5b7eb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x562ca554f9e0 con 0x562ca5548f30 2024-05-21T22:03:44.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.977+0000 7fdac5b7eb80 10 client.0 _ll_register_callbacks cb 0x562ca543ac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:03:44.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.977+0000 7fdab3400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ca5548610 0x562ca554c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:44.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.977+0000 7fdab9200700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562ca5548f30 0x562ca554c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:44.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.977+0000 7fdab3400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ca5548610 0x562ca554c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:47028/0 (socket says 172.21.15.151:47028) 2024-05-21T22:03:44.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.977+0000 7fdab3400700 1 -- 172.21.15.151:0/3028808114 learned_addr learned my addr 172.21.15.151:0/3028808114 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:03:44.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.977+0000 7fdab3e00700 1 --2- 172.21.15.151:0/3028808114 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562ca5543140 0x562ca554ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:44.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.977+0000 7fdab3400700 1 -- 172.21.15.151:0/3028808114 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562ca5548f30 msgr2=0x562ca554c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:44.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.977+0000 7fdab3400700 1 --2- 172.21.15.151:0/3028808114 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562ca5548f30 0x562ca554c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:44.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.977+0000 7fdab3400700 1 -- 172.21.15.151:0/3028808114 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562ca5543140 msgr2=0x562ca554ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:44.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.977+0000 7fdab3400700 1 --2- 172.21.15.151:0/3028808114 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562ca5543140 0x562ca554ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:44.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.977+0000 7fdab3400700 1 -- 172.21.15.151:0/3028808114 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ca53e08d0 con 0x562ca5548610 2024-05-21T22:03:44.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.978+0000 7fdab3400700 1 --2- 172.21.15.151:0/3028808114 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ca5548610 0x562ca554c020 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fdaa000a010 tx=0x7fdaa0011c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7cae1b8d22a68ba server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:03:44.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.978+0000 7fda9e000700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-21T22:03:44.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.978+0000 7fda9e000700 1 -- 172.21.15.151:0/3028808114 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa0026020 con 0x562ca5548610 2024-05-21T22:03:44.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.978+0000 7fda9e000700 1 -- 172.21.15.151:0/3028808114 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7fdaa0004960 con 0x562ca5548610 2024-05-21T22:03:44.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.978+0000 7fdac5b7eb80 1 -- 172.21.15.151:0/3028808114 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdaa0004500 con 0x562ca5548610 2024-05-21T22:03:44.982 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.979+0000 7fda9e000700 1 -- 172.21.15.151:0/3028808114 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdaa0004ba0 con 0x562ca5548610 2024-05-21T22:03:44.982 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.979+0000 7fdac5b7eb80 10 client.27173 fetch_fsmap learned FSMap version 29 2024-05-21T22:03:44.982 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.979+0000 7fdac5b7eb80 1 -- 172.21.15.151:0/3028808114 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562ca5461060 con 0x562ca5548610 2024-05-21T22:03:44.982 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.979+0000 7fda9e000700 1 -- 172.21.15.151:0/3028808114 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdaa0004500 con 0x562ca5548610 2024-05-21T22:03:44.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.980+0000 7fda9e000700 1 -- 172.21.15.151:0/3028808114 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fdaa0018070 con 0x562ca5548610 2024-05-21T22:03:44.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.980+0000 7fdac5b7eb80 10 client.27173 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:03:44.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.980+0000 7fdac5b7eb80 10 client.27173 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-05-21T22:03:44.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.980+0000 7fdac5b7eb80 20 client.27173 populate_metadata read hostname 'smithi151' 2024-05-21T22:03:44.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.980+0000 7fdac5b7eb80 1 --2- 172.21.15.151:0/3028808114 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x562ca555c3c0 0x562ca555e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:44.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.980+0000 7fdac5b7eb80 4 client.27173 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T22:03:44.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.980+0000 7fdac5b7eb80 1 -- 172.21.15.151:0/3028808114 --> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562ca54a2d90 con 0x562ca555c3c0 2024-05-21T22:03:44.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.980+0000 7fdab3e00700 1 --2- 172.21.15.151:0/3028808114 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x562ca555c3c0 0x562ca555e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:44.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.981+0000 7fdab3e00700 1 --2- 172.21.15.151:0/3028808114 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x562ca555c3c0 0x562ca555e7b0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=32674f936f963223 server_cookie=67ac36de97dd6a67 in_seq=0 out_seq=0 2024-05-21T22:03:44.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.981+0000 7fda9e000700 10 client.27173 ms_handle_connect on v2:172.21.15.164:6834/2340428470 2024-05-21T22:03:44.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.982+0000 7fda9e000700 1 -- 172.21.15.151:0/3028808114 <== mds.4 v2:172.21.15.164:6834/2340428470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x562ca54a2d90 con 0x562ca555c3c0 2024-05-21T22:03:44.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.982+0000 7fda9e000700 10 client.27173 handle_command_reply: tid=0 2024-05-21T22:03:44.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.982+0000 7fdac5b7eb80 1 client.27173 shutdown 2024-05-21T22:03:44.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.982+0000 7fdac5b7eb80 1 -- 172.21.15.151:0/3028808114 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ca5548610 msgr2=0x562ca554c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:44.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.982+0000 7fdac5b7eb80 1 --2- 172.21.15.151:0/3028808114 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ca5548610 0x562ca554c020 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fdaa000a010 tx=0x7fdaa0011c40 comp rx=0 tx=0).stop 2024-05-21T22:03:44.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.982+0000 7fdac5b7eb80 1 -- 172.21.15.151:0/3028808114 shutdown_connections 2024-05-21T22:03:44.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.982+0000 7fdac5b7eb80 1 --2- 172.21.15.151:0/3028808114 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x562ca555c3c0 0x562ca555e7b0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:44.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.982+0000 7fdac5b7eb80 1 --2- 172.21.15.151:0/3028808114 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562ca5548f30 0x562ca554c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:44.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.982+0000 7fdac5b7eb80 1 --2- 172.21.15.151:0/3028808114 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562ca5543140 0x562ca554ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:44.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.982+0000 7fda9e000700 0 client.27173 ms_handle_reset on v2:172.21.15.164:6834/2340428470 2024-05-21T22:03:44.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.982+0000 7fdac5b7eb80 1 --2- 172.21.15.151:0/3028808114 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ca5548610 0x562ca554c020 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:44.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.982+0000 7fdac5b7eb80 1 -- 172.21.15.151:0/3028808114 >> 172.21.15.151:0/3028808114 conn(0x562ca5470be0 msgr2=0x562ca553cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:03:44.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.982+0000 7fdac5b7eb80 1 -- 172.21.15.151:0/3028808114 shutdown_connections 2024-05-21T22:03:44.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.982+0000 7fdac5b7eb80 1 -- 172.21.15.151:0/3028808114 wait complete. 2024-05-21T22:03:44.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.983+0000 7fdac5b7eb80 20 client.27173 trim_cache size 0 max 16384 2024-05-21T22:03:45.000 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.999+0000 7fdac5b7eb80 1 Processor -- start 2024-05-21T22:03:45.000 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:44.999+0000 7fdac5b7eb80 1 -- start start 2024-05-21T22:03:45.001 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.000+0000 7fdac5b7eb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562ca5543140 0x562ca5542b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:45.001 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.000+0000 7fdac5b7eb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ca5548610 0x562ca554b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:45.001 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.000+0000 7fdac5b7eb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562ca5548f30 0x562ca553d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:45.002 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.000+0000 7fdac5b7eb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdab41b44e0 con 0x562ca5548f30 2024-05-21T22:03:45.002 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.000+0000 7fdac5b7eb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdab41b6020 con 0x562ca5548610 2024-05-21T22:03:45.002 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.000+0000 7fdac5b7eb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fdab41b4bb0 con 0x562ca5543140 2024-05-21T22:03:45.002 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.000+0000 7fdab9200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562ca5548f30 0x562ca553d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:45.002 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.000+0000 7fdab9200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562ca5548f30 0x562ca553d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:33376/0 (socket says 172.21.15.151:33376) 2024-05-21T22:03:45.002 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.000+0000 7fdab9200700 1 -- 172.21.15.151:0/3391059148 learned_addr learned my addr 172.21.15.151:0/3391059148 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:03:45.002 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.000+0000 7fdab3e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562ca5543140 0x562ca5542b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:45.002 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.000+0000 7fdab3400700 1 --2- 172.21.15.151:0/3391059148 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ca5548610 0x562ca554b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:45.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.001+0000 7fdab9200700 1 -- 172.21.15.151:0/3391059148 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562ca5543140 msgr2=0x562ca5542b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:45.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.001+0000 7fdab9200700 1 --2- 172.21.15.151:0/3391059148 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562ca5543140 0x562ca5542b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:45.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.001+0000 7fdab9200700 1 -- 172.21.15.151:0/3391059148 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ca5548610 msgr2=0x562ca554b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:45.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.001+0000 7fdab9200700 1 --2- 172.21.15.151:0/3391059148 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ca5548610 0x562ca554b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:45.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.001+0000 7fdab9200700 1 -- 172.21.15.151:0/3391059148 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ca554a590 con 0x562ca5548f30 2024-05-21T22:03:45.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.001+0000 7fdab3e00700 1 --2- 172.21.15.151:0/3391059148 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562ca5543140 0x562ca5542b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T22:03:45.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.001+0000 7fdab3400700 1 --2- 172.21.15.151:0/3391059148 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ca5548610 0x562ca554b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:03:45.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.001+0000 7fdab9200700 1 --2- 172.21.15.151:0/3391059148 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562ca5548f30 0x562ca553d4f0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fdaac015000 tx=0x7fdaac00be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b23ce2d0adbf3ae server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:03:45.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.002+0000 7fdaa8c00700 1 -- 172.21.15.151:0/3391059148 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaac007b70 con 0x562ca5548f30 2024-05-21T22:03:45.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.002+0000 7fdaa8c00700 1 -- 172.21.15.151:0/3391059148 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdaac023040 con 0x562ca5548f30 2024-05-21T22:03:45.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.002+0000 7fdaa8c00700 1 -- 172.21.15.151:0/3391059148 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaac01e430 con 0x562ca5548f30 2024-05-21T22:03:45.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.002+0000 7fdac5b7eb80 1 -- 172.21.15.151:0/3391059148 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562ca5548f30 msgr2=0x562ca553d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:45.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.002+0000 7fdac5b7eb80 1 --2- 172.21.15.151:0/3391059148 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562ca5548f30 0x562ca553d4f0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fdaac015000 tx=0x7fdaac00be60 comp rx=0 tx=0).stop 2024-05-21T22:03:45.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.003+0000 7fdac5b7eb80 1 -- 172.21.15.151:0/3391059148 shutdown_connections 2024-05-21T22:03:45.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.003+0000 7fdac5b7eb80 1 --2- 172.21.15.151:0/3391059148 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562ca5548f30 0x562ca553d4f0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:45.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.003+0000 7fdac5b7eb80 1 --2- 172.21.15.151:0/3391059148 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562ca5543140 0x562ca5542b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:45.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.003+0000 7fdac5b7eb80 1 --2- 172.21.15.151:0/3391059148 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ca5548610 0x562ca554b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:45.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.003+0000 7fdac5b7eb80 1 -- 172.21.15.151:0/3391059148 >> 172.21.15.151:0/3391059148 conn(0x562ca5470be0 msgr2=0x562ca554a910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:03:45.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.003+0000 7fdac5b7eb80 1 -- 172.21.15.151:0/3391059148 shutdown_connections 2024-05-21T22:03:45.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.003+0000 7fdac5b7eb80 1 -- 172.21.15.151:0/3391059148 wait complete. 2024-05-21T22:03:45.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.003+0000 7fdac5b7eb80 1 -- start start 2024-05-21T22:03:45.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.003+0000 7fdac5b7eb80 1 Processor -- start 2024-05-21T22:03:45.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.004+0000 7fdac5b7eb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562ca5543140 0x562ca5554e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:45.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.004+0000 7fdac5b7eb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ca5548610 0x562ca5555390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:45.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.004+0000 7fdac5b7eb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562ca5548f30 0x562ca5551c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:45.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.004+0000 7fdac5b7eb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdab41b5950 con 0x562ca5548f30 2024-05-21T22:03:45.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.004+0000 7fdac5b7eb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdab41b5280 con 0x562ca5548610 2024-05-21T22:03:45.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.004+0000 7fdac5b7eb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fdab41b24a0 con 0x562ca5543140 2024-05-21T22:03:45.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.004+0000 7fdac5b7eb80 10 client.0 _ll_register_callbacks cb 0x562ca543ac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:03:45.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.004+0000 7fdab3400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ca5548610 0x562ca5555390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:45.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.004+0000 7fdab3e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562ca5543140 0x562ca5554e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:45.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.004+0000 7fdab9200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562ca5548f30 0x562ca5551c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:45.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.004+0000 7fdab3400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ca5548610 0x562ca5555390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:47050/0 (socket says 172.21.15.151:47050) 2024-05-21T22:03:45.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.004+0000 7fdab3400700 1 -- 172.21.15.151:0/3662779529 learned_addr learned my addr 172.21.15.151:0/3662779529 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:03:45.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.004+0000 7fdab3e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562ca5543140 0x562ca5554e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:59504/0 (socket says 172.21.15.151:59504) 2024-05-21T22:03:45.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.005+0000 7fdab3400700 1 -- 172.21.15.151:0/3662779529 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562ca5543140 msgr2=0x562ca5554e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:45.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.005+0000 7fdab3400700 1 --2- 172.21.15.151:0/3662779529 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562ca5543140 0x562ca5554e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:45.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.005+0000 7fdab3400700 1 -- 172.21.15.151:0/3662779529 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562ca5548f30 msgr2=0x562ca5551c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:45.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.005+0000 7fdab3400700 1 --2- 172.21.15.151:0/3662779529 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562ca5548f30 0x562ca5551c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:45.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.005+0000 7fdab3400700 1 -- 172.21.15.151:0/3662779529 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ca5552480 con 0x562ca5548610 2024-05-21T22:03:45.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.005+0000 7fdab3400700 1 --2- 172.21.15.151:0/3662779529 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ca5548610 0x562ca5555390 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x562ca554d5f0 tx=0x7fdaa0005800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3b76e66d0886ae7 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:03:45.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.005+0000 7fda9e000700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-21T22:03:45.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.005+0000 7fda9e000700 1 -- 172.21.15.151:0/3662779529 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa0026040 con 0x562ca5548610 2024-05-21T22:03:45.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.005+0000 7fda9e000700 1 -- 172.21.15.151:0/3662779529 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7fdaa0003680 con 0x562ca5548610 2024-05-21T22:03:45.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.005+0000 7fdac5b7eb80 1 -- 172.21.15.151:0/3662779529 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdaac023040 con 0x562ca5548610 2024-05-21T22:03:45.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.006+0000 7fda9e000700 1 -- 172.21.15.151:0/3662779529 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdaa0005550 con 0x562ca5548610 2024-05-21T22:03:45.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.006+0000 7fdac5b7eb80 10 client.27179 fetch_fsmap learned FSMap version 29 2024-05-21T22:03:45.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.006+0000 7fdac5b7eb80 1 -- 172.21.15.151:0/3662779529 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562ca55c7880 con 0x562ca5548610 2024-05-21T22:03:45.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.006+0000 7fda9e000700 1 -- 172.21.15.151:0/3662779529 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdaac023040 con 0x562ca5548610 2024-05-21T22:03:45.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.006+0000 7fda9e000700 1 -- 172.21.15.151:0/3662779529 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fdaa0018040 con 0x562ca5548610 2024-05-21T22:03:45.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.006+0000 7fdac5b7eb80 10 client.27179 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:03:45.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.006+0000 7fdac5b7eb80 10 client.27179 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-05-21T22:03:45.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.006+0000 7fdac5b7eb80 20 client.27179 populate_metadata read hostname 'smithi151' 2024-05-21T22:03:45.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.006+0000 7fdac5b7eb80 1 --2- 172.21.15.151:0/3662779529 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x562ca55c7ed0 0x562ca55ca2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:45.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.006+0000 7fdac5b7eb80 4 client.27179 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T22:03:45.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.006+0000 7fdac5b7eb80 1 -- 172.21.15.151:0/3662779529 --> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562ca549c570 con 0x562ca55c7ed0 2024-05-21T22:03:45.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.007+0000 7fdab3e00700 1 --2- 172.21.15.151:0/3662779529 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x562ca55c7ed0 0x562ca55ca2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:45.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.007+0000 7fdab3e00700 1 --2- 172.21.15.151:0/3662779529 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x562ca55c7ed0 0x562ca55ca2f0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=995b480e0a4466e5 server_cookie=bf069e121acff9fc in_seq=0 out_seq=0 2024-05-21T22:03:45.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.007+0000 7fda9e000700 10 client.27179 ms_handle_connect on v2:172.21.15.164:6834/2340428470 2024-05-21T22:03:45.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.007+0000 7fda9e000700 1 -- 172.21.15.151:0/3662779529 <== mds.4 v2:172.21.15.164:6834/2340428470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562ca549c570 con 0x562ca55c7ed0 2024-05-21T22:03:45.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.007+0000 7fda9e000700 10 client.27179 handle_command_reply: tid=0 2024-05-21T22:03:45.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.007+0000 7fdac5b7eb80 1 client.27179 shutdown 2024-05-21T22:03:45.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.008+0000 7fdac5b7eb80 1 -- 172.21.15.151:0/3662779529 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ca5548610 msgr2=0x562ca5555390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:45.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.008+0000 7fdac5b7eb80 1 --2- 172.21.15.151:0/3662779529 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ca5548610 0x562ca5555390 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x562ca554d5f0 tx=0x7fdaa0005800 comp rx=0 tx=0).stop 2024-05-21T22:03:45.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.008+0000 7fdac5b7eb80 1 -- 172.21.15.151:0/3662779529 shutdown_connections 2024-05-21T22:03:45.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.008+0000 7fdac5b7eb80 1 --2- 172.21.15.151:0/3662779529 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x562ca55c7ed0 0x562ca55ca2f0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:45.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.008+0000 7fdac5b7eb80 1 --2- 172.21.15.151:0/3662779529 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562ca5548f30 0x562ca5551c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:45.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.008+0000 7fdac5b7eb80 1 --2- 172.21.15.151:0/3662779529 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562ca5543140 0x562ca5554e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:45.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.008+0000 7fda9e000700 0 client.27179 ms_handle_reset on v2:172.21.15.164:6834/2340428470 2024-05-21T22:03:45.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.008+0000 7fdac5b7eb80 1 --2- 172.21.15.151:0/3662779529 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562ca5548610 0x562ca5555390 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:45.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.008+0000 7fdac5b7eb80 1 -- 172.21.15.151:0/3662779529 >> 172.21.15.151:0/3662779529 conn(0x562ca5470be0 msgr2=0x562ca554b1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:03:45.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.008+0000 7fdac5b7eb80 1 -- 172.21.15.151:0/3662779529 shutdown_connections 2024-05-21T22:03:45.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.008+0000 7fdac5b7eb80 1 -- 172.21.15.151:0/3662779529 wait complete. 2024-05-21T22:03:45.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.008+0000 7fdac5b7eb80 20 client.27179 trim_cache size 0 max 16384 2024-05-21T22:03:45.013 INFO:teuthology.orchestra.run.smithi151.stdout:[] 2024-05-21T22:03:45.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.009+0000 7fdabb200700 1 -- 172.21.15.151:0/1828888261 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fda980b2b20 msgr2=0x7fda980b4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:45.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.009+0000 7fdabb200700 1 --2- 172.21.15.151:0/1828888261 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fda980b2b20 0x7fda980b4fe0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fdaac00b780 tx=0x7fdaac006210 comp rx=0 tx=0).stop 2024-05-21T22:03:45.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.009+0000 7fdabb200700 1 -- 172.21.15.151:0/1828888261 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdab4105730 msgr2=0x7fdab4077a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:45.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.009+0000 7fdabb200700 1 --2- 172.21.15.151:0/1828888261 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdab4105730 0x7fdab4077a50 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fdaa400eb30 tx=0x7fdaa400ee40 comp rx=0 tx=0).stop 2024-05-21T22:03:45.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.009+0000 7fdabb200700 1 -- 172.21.15.151:0/1828888261 shutdown_connections 2024-05-21T22:03:45.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.009+0000 7fdabb200700 1 --2- 172.21.15.151:0/1828888261 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fda980b2b20 0x7fda980b4fe0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:45.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.009+0000 7fdabb200700 1 --2- 172.21.15.151:0/1828888261 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdab4105730 0x7fdab4077a50 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:45.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.009+0000 7fdabb200700 1 --2- 172.21.15.151:0/1828888261 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdab410b730 0x7fdab40784d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:45.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.009+0000 7fdabb200700 1 --2- 172.21.15.151:0/1828888261 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdab41060f0 0x7fdab4077f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:45.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.009+0000 7fdabb200700 1 -- 172.21.15.151:0/1828888261 >> 172.21.15.151:0/1828888261 conn(0x7fdab4101220 msgr2=0x7fdab4101800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:03:45.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.009+0000 7fdabb200700 1 -- 172.21.15.151:0/1828888261 shutdown_connections 2024-05-21T22:03:45.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:45.009+0000 7fdabb200700 1 -- 172.21.15.151:0/1828888261 wait complete. 2024-05-21T22:03:45.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:03:45 smithi151 ceph-mon[31623]: pgmap v251: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-05-21T22:03:45.985 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:03:45 smithi184 ceph-mon[36942]: pgmap v251: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-05-21T22:03:46.026 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-21T22:03:46.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:03:45 smithi164 ceph-mon[36269]: pgmap v251: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-05-21T22:03:46.496 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:03:47.899 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:03:47 smithi151 ceph-mon[31623]: pgmap v252: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-05-21T22:03:47.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:03:47 smithi184 ceph-mon[36942]: pgmap v252: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-05-21T22:03:48.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:03:47 smithi164 ceph-mon[36269]: pgmap v252: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-05-21T22:03:48.717 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.715+0000 7fb4fec00700 1 -- 172.21.15.151:0/3487799160 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb4f806b220 msgr2=0x7fb4f806b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:48.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.715+0000 7fb4fec00700 1 --2- 172.21.15.151:0/3487799160 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb4f806b220 0x7fb4f806b6a0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fb4e4009990 tx=0x7fb4e4009ca0 comp rx=0 tx=0).stop 2024-05-21T22:03:48.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.716+0000 7fb4fec00700 1 -- 172.21.15.151:0/3487799160 shutdown_connections 2024-05-21T22:03:48.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.716+0000 7fb4fec00700 1 --2- 172.21.15.151:0/3487799160 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb4f81095b0 0x7fb4f8109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:48.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.716+0000 7fb4fec00700 1 --2- 172.21.15.151:0/3487799160 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4f806bbe0 0x7fb4f81103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:48.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.716+0000 7fb4fec00700 1 --2- 172.21.15.151:0/3487799160 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb4f806b220 0x7fb4f806b6a0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:48.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.716+0000 7fb4fec00700 1 -- 172.21.15.151:0/3487799160 >> 172.21.15.151:0/3487799160 conn(0x7fb4f80793f0 msgr2=0x7fb4f8079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:03:48.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.716+0000 7fb4fec00700 1 -- 172.21.15.151:0/3487799160 shutdown_connections 2024-05-21T22:03:48.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.716+0000 7fb4fec00700 1 -- 172.21.15.151:0/3487799160 wait complete. 2024-05-21T22:03:48.720 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.717+0000 7fb4fec00700 1 Processor -- start 2024-05-21T22:03:48.720 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.717+0000 7fb4fec00700 1 -- start start 2024-05-21T22:03:48.720 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.717+0000 7fb4fec00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4f806b220 0x7fb4f8102510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:48.720 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.717+0000 7fb4fec00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb4f806bbe0 0x7fb4f8102a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:48.720 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.717+0000 7fb4fec00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb4f81095b0 0x7fb4f8102f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:48.720 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.717+0000 7fb4fec00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f8113740 con 0x7fb4f806bbe0 2024-05-21T22:03:48.720 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.717+0000 7fb4fec00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f81135c0 con 0x7fb4f81095b0 2024-05-21T22:03:48.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.717+0000 7fb4fec00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f81138c0 con 0x7fb4f806b220 2024-05-21T22:03:48.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.718+0000 7fb4f7400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb4f806bbe0 0x7fb4f8102a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:48.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.718+0000 7fb4f7e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4f806b220 0x7fb4f8102510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:48.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.718+0000 7fb4f7400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb4f806bbe0 0x7fb4f8102a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52060/0 (socket says 172.21.15.151:52060) 2024-05-21T22:03:48.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.718+0000 7fb4f7400700 1 -- 172.21.15.151:0/3670664635 learned_addr learned my addr 172.21.15.151:0/3670664635 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:03:48.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.718+0000 7fb4fcc00700 1 --2- 172.21.15.151:0/3670664635 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb4f81095b0 0x7fb4f8102f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:48.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.718+0000 7fb4f7e00700 1 -- 172.21.15.151:0/3670664635 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb4f81095b0 msgr2=0x7fb4f8102f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:48.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.718+0000 7fb4f7e00700 1 --2- 172.21.15.151:0/3670664635 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb4f81095b0 0x7fb4f8102f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:48.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.718+0000 7fb4f7e00700 1 -- 172.21.15.151:0/3670664635 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb4f806bbe0 msgr2=0x7fb4f8102a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:48.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.718+0000 7fb4f7e00700 1 --2- 172.21.15.151:0/3670664635 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb4f806bbe0 0x7fb4f8102a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:48.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.718+0000 7fb4f7e00700 1 -- 172.21.15.151:0/3670664635 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4f81a81c0 con 0x7fb4f806b220 2024-05-21T22:03:48.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.719+0000 7fb4f7e00700 1 --2- 172.21.15.151:0/3670664635 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4f806b220 0x7fb4f8102510 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fb4e800d750 tx=0x7fb4e800da60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:03:48.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.719+0000 7fb4f4c00700 1 -- 172.21.15.151:0/3670664635 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e8014030 con 0x7fb4f806b220 2024-05-21T22:03:48.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.719+0000 7fb4f4c00700 1 -- 172.21.15.151:0/3670664635 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4e80044d0 con 0x7fb4f806b220 2024-05-21T22:03:48.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.719+0000 7fb4fec00700 1 -- 172.21.15.151:0/3670664635 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4f81a84b0 con 0x7fb4f806b220 2024-05-21T22:03:48.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.719+0000 7fb4f4c00700 1 -- 172.21.15.151:0/3670664635 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e8005420 con 0x7fb4f806b220 2024-05-21T22:03:48.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.719+0000 7fb4fec00700 1 -- 172.21.15.151:0/3670664635 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4f81a8a40 con 0x7fb4f806b220 2024-05-21T22:03:48.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.722+0000 7fb4f4c00700 1 -- 172.21.15.151:0/3670664635 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7fb4e8004a50 con 0x7fb4f806b220 2024-05-21T22:03:48.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.724+0000 7fb509534b80 1 Processor -- start 2024-05-21T22:03:48.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.724+0000 7fb509534b80 1 -- start start 2024-05-21T22:03:48.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.724+0000 7fb4f4c00700 1 --2- 172.21.15.151:0/3670664635 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fb4dc0b2a70 0x7fb4dc0b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:48.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.724+0000 7fb4f4c00700 1 -- 172.21.15.151:0/3670664635 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7fb4e80e2180 con 0x7fb4f806b220 2024-05-21T22:03:48.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.724+0000 7fb509534b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b01a959610 0x55b01a9599f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:48.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.724+0000 7fb509534b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b01a959f30 0x55b01a953b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:48.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.724+0000 7fb509534b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b01a954140 0x55b01a9545c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:48.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.724+0000 7fb509534b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55b01a8b1e40 con 0x55b01a959610 2024-05-21T22:03:48.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.724+0000 7fb509534b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55b01a8a19e0 con 0x55b01a954140 2024-05-21T22:03:48.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.724+0000 7fb509534b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55b01a8a5a20 con 0x55b01a959f30 2024-05-21T22:03:48.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.724+0000 7fb4fcc00700 1 --2- 172.21.15.151:0/3670664635 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fb4dc0b2a70 0x7fb4dc0b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:48.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.725+0000 7fb4f7e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b01a959610 0x55b01a9599f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:48.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.725+0000 7fb4f7e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b01a959610 0x55b01a9599f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52062/0 (socket says 172.21.15.151:52062) 2024-05-21T22:03:48.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.725+0000 7fb4f7e00700 1 -- 172.21.15.151:0/583965469 learned_addr learned my addr 172.21.15.151:0/583965469 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:03:48.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.725+0000 7fb4f7400700 1 --2- 172.21.15.151:0/583965469 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b01a959f30 0x55b01a953b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:48.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.725+0000 7fb4fcc00700 1 --2- 172.21.15.151:0/583965469 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b01a954140 0x55b01a9545c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:48.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.725+0000 7fb4f7e00700 1 -- 172.21.15.151:0/583965469 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b01a959f30 msgr2=0x55b01a953b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:48.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.725+0000 7fb4f7e00700 1 --2- 172.21.15.151:0/583965469 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b01a959f30 0x55b01a953b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:48.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.725+0000 7fb4f7e00700 1 -- 172.21.15.151:0/583965469 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b01a954140 msgr2=0x55b01a9545c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:48.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.725+0000 7fb4f7e00700 1 --2- 172.21.15.151:0/583965469 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b01a954140 0x55b01a9545c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:48.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.725+0000 7fb4f7e00700 1 -- 172.21.15.151:0/583965469 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b01a86a420 con 0x55b01a959610 2024-05-21T22:03:48.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.726+0000 7fb4fcc00700 1 --2- 172.21.15.151:0/3670664635 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fb4dc0b2a70 0x7fb4dc0b4f30 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb4f000f870 tx=0x7fb4f000fc30 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:03:48.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.726+0000 7fb4f7e00700 1 --2- 172.21.15.151:0/583965469 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b01a959610 0x55b01a9599f0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fb4e80a5470 tx=0x7fb4e80a8e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f83b2e3409deb4e server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:03:48.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.726+0000 7fb4ed600700 1 -- 172.21.15.151:0/583965469 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e801d040 con 0x55b01a959610 2024-05-21T22:03:48.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.726+0000 7fb4ed600700 1 -- 172.21.15.151:0/583965469 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4e80ad7a0 con 0x55b01a959610 2024-05-21T22:03:48.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.726+0000 7fb4ed600700 1 -- 172.21.15.151:0/583965469 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e80a9650 con 0x55b01a959610 2024-05-21T22:03:48.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.727+0000 7fb509534b80 1 -- 172.21.15.151:0/583965469 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b01a959610 msgr2=0x55b01a9599f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:48.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.727+0000 7fb509534b80 1 --2- 172.21.15.151:0/583965469 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b01a959610 0x55b01a9599f0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fb4e80a5470 tx=0x7fb4e80a8e70 comp rx=0 tx=0).stop 2024-05-21T22:03:48.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.727+0000 7fb509534b80 1 -- 172.21.15.151:0/583965469 shutdown_connections 2024-05-21T22:03:48.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.727+0000 7fb509534b80 1 --2- 172.21.15.151:0/583965469 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b01a954140 0x55b01a9545c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:48.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.727+0000 7fb509534b80 1 --2- 172.21.15.151:0/583965469 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b01a959610 0x55b01a9599f0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:48.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.727+0000 7fb509534b80 1 --2- 172.21.15.151:0/583965469 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b01a959f30 0x55b01a953b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:48.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.727+0000 7fb509534b80 1 -- 172.21.15.151:0/583965469 >> 172.21.15.151:0/583965469 conn(0x55b01a881be0 msgr2=0x55b01a94e690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:03:48.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.727+0000 7fb509534b80 1 -- 172.21.15.151:0/583965469 shutdown_connections 2024-05-21T22:03:48.731 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.727+0000 7fb509534b80 1 -- 172.21.15.151:0/583965469 wait complete. 2024-05-21T22:03:48.731 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.728+0000 7fb509534b80 1 -- start start 2024-05-21T22:03:48.731 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.728+0000 7fb509534b80 1 Processor -- start 2024-05-21T22:03:48.731 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.728+0000 7fb509534b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b01a954140 0x55b01a95fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:48.731 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.728+0000 7fb509534b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b01a959610 0x55b01a95d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:48.731 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.728+0000 7fb509534b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b01a959f30 0x55b01a95d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:48.731 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.728+0000 7fb509534b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55b01a7a7fa0 con 0x55b01a959f30 2024-05-21T22:03:48.731 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.728+0000 7fb509534b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55b01a842760 con 0x55b01a959610 2024-05-21T22:03:48.731 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.728+0000 7fb509534b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55b01a9609e0 con 0x55b01a954140 2024-05-21T22:03:48.731 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.728+0000 7fb509534b80 10 client.0 _ll_register_callbacks cb 0x55b01a84bc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:03:48.733 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.729+0000 7fb4fcc00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b01a959f30 0x55b01a95d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:48.733 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.729+0000 7fb4f7e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b01a954140 0x55b01a95fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:48.733 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.729+0000 7fb4f7e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b01a954140 0x55b01a95fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:52814/0 (socket says 172.21.15.151:52814) 2024-05-21T22:03:48.733 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.729+0000 7fb4f7e00700 1 -- 172.21.15.151:0/3814025830 learned_addr learned my addr 172.21.15.151:0/3814025830 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:03:48.733 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.729+0000 7fb4f7400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b01a959610 0x55b01a95d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:48.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.729+0000 7fb4fcc00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b01a959f30 0x55b01a95d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52072/0 (socket says 172.21.15.151:52072) 2024-05-21T22:03:48.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.729+0000 7fb4f7e00700 1 -- 172.21.15.151:0/3814025830 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b01a959610 msgr2=0x55b01a95d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:48.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.729+0000 7fb4f7e00700 1 --2- 172.21.15.151:0/3814025830 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b01a959610 0x55b01a95d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:48.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.729+0000 7fb4f7e00700 1 -- 172.21.15.151:0/3814025830 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b01a959f30 msgr2=0x55b01a95d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:48.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.729+0000 7fb4f7e00700 1 --2- 172.21.15.151:0/3814025830 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b01a959f30 0x55b01a95d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:48.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.729+0000 7fb4f7e00700 1 -- 172.21.15.151:0/3814025830 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b01a7f18d0 con 0x55b01a954140 2024-05-21T22:03:48.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.729+0000 7fb4fcc00700 1 --2- 172.21.15.151:0/3814025830 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b01a959f30 0x55b01a95d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:03:48.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.729+0000 7fb4f7e00700 1 --2- 172.21.15.151:0/3814025830 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b01a954140 0x55b01a95fe50 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fb4e8012040 tx=0x7fb4e80a92f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48443df74f709f1f server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:03:48.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.729+0000 7fb4e2000700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-21T22:03:48.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.730+0000 7fb4e2000700 1 -- 172.21.15.151:0/3814025830 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e80b2250 con 0x55b01a954140 2024-05-21T22:03:48.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.730+0000 7fb4e2000700 1 -- 172.21.15.151:0/3814025830 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7fb4e80a9650 con 0x55b01a954140 2024-05-21T22:03:48.735 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.730+0000 7fb509534b80 1 -- 172.21.15.151:0/3814025830 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4e80ad7a0 con 0x55b01a954140 2024-05-21T22:03:48.735 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.730+0000 7fb4e2000700 1 -- 172.21.15.151:0/3814025830 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb4e80a9850 con 0x55b01a954140 2024-05-21T22:03:48.735 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.730+0000 7fb509534b80 10 client.27196 fetch_fsmap learned FSMap version 29 2024-05-21T22:03:48.735 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.730+0000 7fb509534b80 1 -- 172.21.15.151:0/3814025830 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b01a872060 con 0x55b01a954140 2024-05-21T22:03:48.735 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.730+0000 7fb4e2000700 1 -- 172.21.15.151:0/3814025830 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4e80ad7a0 con 0x55b01a954140 2024-05-21T22:03:48.735 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.731+0000 7fb4e2000700 1 -- 172.21.15.151:0/3814025830 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fb4e80c3ce0 con 0x55b01a954140 2024-05-21T22:03:48.735 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.731+0000 7fb509534b80 10 client.27196 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:03:48.735 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.731+0000 7fb509534b80 10 client.27196 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-21T22:03:48.735 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.731+0000 7fb509534b80 20 client.27196 populate_metadata read hostname 'smithi151' 2024-05-21T22:03:48.735 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.731+0000 7fb509534b80 1 --2- 172.21.15.151:0/3814025830 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x55b01a96d590 0x55b01a96f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:48.735 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.731+0000 7fb509534b80 4 client.27196 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T22:03:48.735 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.731+0000 7fb509534b80 1 -- 172.21.15.151:0/3814025830 --> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b01a8b3d90 con 0x55b01a96d590 2024-05-21T22:03:48.735 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.731+0000 7fb4f7400700 1 --2- 172.21.15.151:0/3814025830 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x55b01a96d590 0x55b01a96f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:48.735 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.732+0000 7fb4f7400700 1 --2- 172.21.15.151:0/3814025830 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x55b01a96d590 0x55b01a96f980 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2097e9d9a8948a1a server_cookie=16316e8ab5db6d99 in_seq=0 out_seq=0 2024-05-21T22:03:48.735 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.732+0000 7fb4e2000700 10 client.27196 ms_handle_connect on v2:172.21.15.184:6836/2988194888 2024-05-21T22:03:48.736 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.732+0000 7fb4e2000700 1 -- 172.21.15.151:0/3814025830 <== mds.3 v2:172.21.15.184:6836/2988194888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55b01a8b3d90 con 0x55b01a96d590 2024-05-21T22:03:48.737 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.732+0000 7fb4e2000700 10 client.27196 handle_command_reply: tid=0 2024-05-21T22:03:48.737 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.732+0000 7fb509534b80 1 client.27196 shutdown 2024-05-21T22:03:48.737 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.733+0000 7fb509534b80 1 -- 172.21.15.151:0/3814025830 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b01a954140 msgr2=0x55b01a95fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:48.737 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.733+0000 7fb509534b80 1 --2- 172.21.15.151:0/3814025830 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b01a954140 0x55b01a95fe50 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fb4e8012040 tx=0x7fb4e80a92f0 comp rx=0 tx=0).stop 2024-05-21T22:03:48.737 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.733+0000 7fb509534b80 1 -- 172.21.15.151:0/3814025830 shutdown_connections 2024-05-21T22:03:48.737 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.733+0000 7fb509534b80 1 --2- 172.21.15.151:0/3814025830 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x55b01a96d590 0x55b01a96f980 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:48.737 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.733+0000 7fb509534b80 1 --2- 172.21.15.151:0/3814025830 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b01a959f30 0x55b01a95d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:48.737 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.733+0000 7fb509534b80 1 --2- 172.21.15.151:0/3814025830 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b01a954140 0x55b01a95fe50 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:48.737 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.733+0000 7fb509534b80 1 --2- 172.21.15.151:0/3814025830 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b01a959610 0x55b01a95d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:48.737 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.733+0000 7fb509534b80 1 -- 172.21.15.151:0/3814025830 >> 172.21.15.151:0/3814025830 conn(0x55b01a881be0 msgr2=0x55b01a94dd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:03:48.737 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.733+0000 7fb4e2000700 0 client.27196 ms_handle_reset on v2:172.21.15.184:6836/2988194888 2024-05-21T22:03:48.737 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.733+0000 7fb509534b80 1 -- 172.21.15.151:0/3814025830 shutdown_connections 2024-05-21T22:03:48.737 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.733+0000 7fb509534b80 1 -- 172.21.15.151:0/3814025830 wait complete. 2024-05-21T22:03:48.737 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.733+0000 7fb509534b80 20 client.27196 trim_cache size 0 max 16384 2024-05-21T22:03:48.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.749+0000 7fb509534b80 1 Processor -- start 2024-05-21T22:03:48.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.749+0000 7fb509534b80 1 -- start start 2024-05-21T22:03:48.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.750+0000 7fb509534b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b01a954140 0x55b01a953b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:48.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.750+0000 7fb509534b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b01a959610 0x55b01a957830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:48.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.750+0000 7fb509534b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b01a959f30 0x55b01a94e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:48.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.750+0000 7fb509534b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f81b0140 con 0x55b01a954140 2024-05-21T22:03:48.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.750+0000 7fb509534b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f81b1c80 con 0x55b01a959f30 2024-05-21T22:03:48.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.750+0000 7fb509534b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f81b0810 con 0x55b01a959610 2024-05-21T22:03:48.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.750+0000 7fb4f7e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b01a954140 0x55b01a953b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:48.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.750+0000 7fb4f7e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b01a954140 0x55b01a953b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52074/0 (socket says 172.21.15.151:52074) 2024-05-21T22:03:48.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.750+0000 7fb4f7e00700 1 -- 172.21.15.151:0/4016896218 learned_addr learned my addr 172.21.15.151:0/4016896218 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:03:48.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.750+0000 7fb4f7400700 1 --2- 172.21.15.151:0/4016896218 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b01a959610 0x55b01a957830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:48.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.750+0000 7fb4fcc00700 1 --2- 172.21.15.151:0/4016896218 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b01a959f30 0x55b01a94e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:48.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.751+0000 7fb4f7e00700 1 -- 172.21.15.151:0/4016896218 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b01a959610 msgr2=0x55b01a957830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:48.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.751+0000 7fb4f7e00700 1 --2- 172.21.15.151:0/4016896218 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b01a959610 0x55b01a957830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:48.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.751+0000 7fb4f7e00700 1 -- 172.21.15.151:0/4016896218 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b01a959f30 msgr2=0x55b01a94e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:48.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.751+0000 7fb4f7e00700 1 --2- 172.21.15.151:0/4016896218 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b01a959f30 0x55b01a94e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:48.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.751+0000 7fb4f7e00700 1 -- 172.21.15.151:0/4016896218 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b01a94f4a0 con 0x55b01a954140 2024-05-21T22:03:48.754 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.751+0000 7fb4f7e00700 1 --2- 172.21.15.151:0/4016896218 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b01a954140 0x55b01a953b20 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fb4e80a5470 tx=0x7fb4e8004d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91bf40c9ff72796f server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:03:48.754 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.751+0000 7fb4ecc00700 1 -- 172.21.15.151:0/4016896218 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e801d070 con 0x55b01a954140 2024-05-21T22:03:48.754 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.751+0000 7fb4ecc00700 1 -- 172.21.15.151:0/4016896218 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4e80b2250 con 0x55b01a954140 2024-05-21T22:03:48.754 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.751+0000 7fb4ecc00700 1 -- 172.21.15.151:0/4016896218 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e80ca470 con 0x55b01a954140 2024-05-21T22:03:48.754 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.752+0000 7fb509534b80 1 -- 172.21.15.151:0/4016896218 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b01a954140 msgr2=0x55b01a953b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:48.754 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.752+0000 7fb509534b80 1 --2- 172.21.15.151:0/4016896218 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b01a954140 0x55b01a953b20 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fb4e80a5470 tx=0x7fb4e8004d40 comp rx=0 tx=0).stop 2024-05-21T22:03:48.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.752+0000 7fb509534b80 1 -- 172.21.15.151:0/4016896218 shutdown_connections 2024-05-21T22:03:48.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.752+0000 7fb509534b80 1 --2- 172.21.15.151:0/4016896218 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b01a959f30 0x55b01a94e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:48.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.752+0000 7fb509534b80 1 --2- 172.21.15.151:0/4016896218 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b01a954140 0x55b01a953b20 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:48.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.752+0000 7fb509534b80 1 --2- 172.21.15.151:0/4016896218 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b01a959610 0x55b01a957830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:48.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.752+0000 7fb509534b80 1 -- 172.21.15.151:0/4016896218 >> 172.21.15.151:0/4016896218 conn(0x55b01a881be0 msgr2=0x55b01a957120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:03:48.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.752+0000 7fb509534b80 1 -- 172.21.15.151:0/4016896218 shutdown_connections 2024-05-21T22:03:48.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.752+0000 7fb509534b80 1 -- 172.21.15.151:0/4016896218 wait complete. 2024-05-21T22:03:48.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.753+0000 7fb509534b80 1 -- start start 2024-05-21T22:03:48.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.753+0000 7fb509534b80 1 Processor -- start 2024-05-21T22:03:48.757 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.753+0000 7fb509534b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b01a954140 0x55b01a970140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:48.757 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.753+0000 7fb509534b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b01a959610 0x55b01a970680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:48.757 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.753+0000 7fb509534b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b01a959f30 0x55b01a9d8610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:48.757 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.753+0000 7fb509534b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f81b15b0 con 0x55b01a959610 2024-05-21T22:03:48.757 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.753+0000 7fb509534b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f81b0ee0 con 0x55b01a954140 2024-05-21T22:03:48.757 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.753+0000 7fb509534b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f81ae100 con 0x55b01a959f30 2024-05-21T22:03:48.757 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.753+0000 7fb509534b80 10 client.0 _ll_register_callbacks cb 0x55b01a84bc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:03:48.757 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.754+0000 7fb4f7400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b01a959610 0x55b01a970680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:48.757 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.754+0000 7fb4f7e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b01a954140 0x55b01a970140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:48.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.754+0000 7fb4f7400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b01a959610 0x55b01a970680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52078/0 (socket says 172.21.15.151:52078) 2024-05-21T22:03:48.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.754+0000 7fb4f7400700 1 -- 172.21.15.151:0/3681306430 learned_addr learned my addr 172.21.15.151:0/3681306430 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:03:48.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.754+0000 7fb4fcc00700 1 --2- 172.21.15.151:0/3681306430 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b01a959f30 0x55b01a9d8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:48.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.754+0000 7fb4f7e00700 1 -- 172.21.15.151:0/3681306430 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b01a959f30 msgr2=0x55b01a9d8610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:48.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.754+0000 7fb4f7e00700 1 --2- 172.21.15.151:0/3681306430 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b01a959f30 0x55b01a9d8610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:48.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.754+0000 7fb4f7e00700 1 -- 172.21.15.151:0/3681306430 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b01a959610 msgr2=0x55b01a970680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:48.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.754+0000 7fb4f7e00700 1 --2- 172.21.15.151:0/3681306430 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b01a959610 0x55b01a970680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:48.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.754+0000 7fb4f7e00700 1 -- 172.21.15.151:0/3681306430 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b01a9d8c50 con 0x55b01a954140 2024-05-21T22:03:48.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.754+0000 7fb4fcc00700 1 --2- 172.21.15.151:0/3681306430 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b01a959f30 0x55b01a9d8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:03:48.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.754+0000 7fb4f7e00700 1 --2- 172.21.15.151:0/3681306430 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b01a954140 0x55b01a970140 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb4e80a9380 tx=0x7fb4e80b2160 comp rx=0 tx=0).ready entity=mon.2 client_cookie=195443ccf10ddbe4 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:03:48.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.754+0000 7fb4e2000700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-21T22:03:48.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.755+0000 7fb4e2000700 1 -- 172.21.15.151:0/3681306430 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e801d050 con 0x55b01a954140 2024-05-21T22:03:48.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.755+0000 7fb4e2000700 1 -- 172.21.15.151:0/3681306430 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7fb4e80a9650 con 0x55b01a954140 2024-05-21T22:03:48.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.755+0000 7fb509534b80 1 -- 172.21.15.151:0/3681306430 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4e80b2250 con 0x55b01a954140 2024-05-21T22:03:48.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.755+0000 7fb4e2000700 1 -- 172.21.15.151:0/3681306430 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb4e80a91c0 con 0x55b01a954140 2024-05-21T22:03:48.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.755+0000 7fb509534b80 10 client.27197 fetch_fsmap learned FSMap version 29 2024-05-21T22:03:48.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.755+0000 7fb509534b80 1 -- 172.21.15.151:0/3681306430 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b01a9d8c50 con 0x55b01a954140 2024-05-21T22:03:48.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.755+0000 7fb4e2000700 1 -- 172.21.15.151:0/3681306430 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4e80b2250 con 0x55b01a954140 2024-05-21T22:03:48.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.755+0000 7fb4e2000700 1 -- 172.21.15.151:0/3681306430 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fb4e80c3ce0 con 0x55b01a954140 2024-05-21T22:03:48.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.755+0000 7fb509534b80 10 client.27197 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:03:48.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.755+0000 7fb509534b80 10 client.27197 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-21T22:03:48.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.755+0000 7fb509534b80 20 client.27197 populate_metadata read hostname 'smithi151' 2024-05-21T22:03:48.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.756+0000 7fb509534b80 1 --2- 172.21.15.151:0/3681306430 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x55b01a9e01a0 0x55b01a9e25c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:48.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.756+0000 7fb509534b80 4 client.27197 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T22:03:48.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.756+0000 7fb509534b80 1 -- 172.21.15.151:0/3681306430 --> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b01a8ad570 con 0x55b01a9e01a0 2024-05-21T22:03:48.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.756+0000 7fb4f7400700 1 --2- 172.21.15.151:0/3681306430 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x55b01a9e01a0 0x55b01a9e25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:48.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.756+0000 7fb4f7400700 1 --2- 172.21.15.151:0/3681306430 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x55b01a9e01a0 0x55b01a9e25c0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a33ccf384da042d1 server_cookie=1da63ccda32b5d9d in_seq=0 out_seq=0 2024-05-21T22:03:48.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.756+0000 7fb4e2000700 10 client.27197 ms_handle_connect on v2:172.21.15.184:6836/2988194888 2024-05-21T22:03:48.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.757+0000 7fb4e2000700 1 -- 172.21.15.151:0/3681306430 <== mds.3 v2:172.21.15.184:6836/2988194888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b01a8ad570 con 0x55b01a9e01a0 2024-05-21T22:03:48.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.757+0000 7fb4e2000700 10 client.27197 handle_command_reply: tid=0 2024-05-21T22:03:48.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.757+0000 7fb509534b80 1 client.27197 shutdown 2024-05-21T22:03:48.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.757+0000 7fb509534b80 1 -- 172.21.15.151:0/3681306430 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b01a954140 msgr2=0x55b01a970140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:48.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.757+0000 7fb509534b80 1 --2- 172.21.15.151:0/3681306430 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b01a954140 0x55b01a970140 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb4e80a9380 tx=0x7fb4e80b2160 comp rx=0 tx=0).stop 2024-05-21T22:03:48.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.757+0000 7fb509534b80 1 -- 172.21.15.151:0/3681306430 shutdown_connections 2024-05-21T22:03:48.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.757+0000 7fb509534b80 1 --2- 172.21.15.151:0/3681306430 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x55b01a9e01a0 0x55b01a9e25c0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:48.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.757+0000 7fb509534b80 1 --2- 172.21.15.151:0/3681306430 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b01a954140 0x55b01a970140 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:48.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.757+0000 7fb509534b80 1 --2- 172.21.15.151:0/3681306430 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b01a959f30 0x55b01a9d8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:48.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.757+0000 7fb509534b80 1 --2- 172.21.15.151:0/3681306430 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b01a959610 0x55b01a970680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:48.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.757+0000 7fb4e2000700 0 client.27197 ms_handle_reset on v2:172.21.15.184:6836/2988194888 2024-05-21T22:03:48.762 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.757+0000 7fb509534b80 1 -- 172.21.15.151:0/3681306430 >> 172.21.15.151:0/3681306430 conn(0x55b01a881be0 msgr2=0x55b01a95a310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:03:48.762 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.757+0000 7fb509534b80 1 -- 172.21.15.151:0/3681306430 shutdown_connections 2024-05-21T22:03:48.762 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.757+0000 7fb509534b80 1 -- 172.21.15.151:0/3681306430 wait complete. 2024-05-21T22:03:48.762 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.757+0000 7fb509534b80 20 client.27197 trim_cache size 0 max 16384 2024-05-21T22:03:48.762 INFO:teuthology.orchestra.run.smithi151.stdout:[] 2024-05-21T22:03:48.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.758+0000 7fb4fec00700 1 -- 172.21.15.151:0/3670664635 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fb4dc0b2a70 msgr2=0x7fb4dc0b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:48.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.758+0000 7fb4fec00700 1 --2- 172.21.15.151:0/3670664635 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fb4dc0b2a70 0x7fb4dc0b4f30 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb4f000f870 tx=0x7fb4f000fc30 comp rx=0 tx=0).stop 2024-05-21T22:03:48.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.758+0000 7fb4fec00700 1 -- 172.21.15.151:0/3670664635 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4f806b220 msgr2=0x7fb4f8102510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:48.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.758+0000 7fb4fec00700 1 --2- 172.21.15.151:0/3670664635 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4f806b220 0x7fb4f8102510 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fb4e800d750 tx=0x7fb4e800da60 comp rx=0 tx=0).stop 2024-05-21T22:03:48.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.758+0000 7fb4fec00700 1 -- 172.21.15.151:0/3670664635 shutdown_connections 2024-05-21T22:03:48.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.758+0000 7fb4fec00700 1 --2- 172.21.15.151:0/3670664635 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fb4dc0b2a70 0x7fb4dc0b4f30 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:48.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.758+0000 7fb4fec00700 1 --2- 172.21.15.151:0/3670664635 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb4f81095b0 0x7fb4f8102f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:48.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.758+0000 7fb4fec00700 1 --2- 172.21.15.151:0/3670664635 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4f806b220 0x7fb4f8102510 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:48.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.758+0000 7fb4fec00700 1 --2- 172.21.15.151:0/3670664635 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb4f806bbe0 0x7fb4f8102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:48.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.758+0000 7fb4fec00700 1 -- 172.21.15.151:0/3670664635 >> 172.21.15.151:0/3670664635 conn(0x7fb4f80793f0 msgr2=0x7fb4f8101100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:03:48.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.758+0000 7fb4fec00700 1 -- 172.21.15.151:0/3670664635 shutdown_connections 2024-05-21T22:03:48.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:48.758+0000 7fb4fec00700 1 -- 172.21.15.151:0/3670664635 wait complete. 2024-05-21T22:03:49.790 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-21T22:03:49.819 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:03:49 smithi151 ceph-mon[31623]: pgmap v253: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-05-21T22:03:49.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:03:49 smithi184 ceph-mon[36942]: pgmap v253: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-05-21T22:03:50.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:03:49 smithi164 ceph-mon[36269]: pgmap v253: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-05-21T22:03:50.264 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:03:51.926 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:03:51 smithi151 ceph-mon[31623]: pgmap v254: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:03:51.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:03:51 smithi184 ceph-mon[36942]: pgmap v254: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:03:52.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:03:51 smithi164 ceph-mon[36269]: pgmap v254: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:03:52.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.462+0000 7f1bdcc00700 1 -- 172.21.15.151:0/1601512611 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1bd8109dd0 msgr2=0x7f1bd810a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:52.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.462+0000 7f1bdcc00700 1 --2- 172.21.15.151:0/1601512611 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1bd8109dd0 0x7f1bd810a1b0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f1bc80099c0 tx=0x7f1bc8009cd0 comp rx=0 tx=0).stop 2024-05-21T22:03:52.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.463+0000 7f1bdcc00700 1 -- 172.21.15.151:0/1601512611 shutdown_connections 2024-05-21T22:03:52.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.463+0000 7f1bdcc00700 1 --2- 172.21.15.151:0/1601512611 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1bd810eeb0 0x7f1bd8116a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:52.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.463+0000 7f1bdcc00700 1 --2- 172.21.15.151:0/1601512611 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1bd8109dd0 0x7f1bd810a1b0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:52.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.463+0000 7f1bdcc00700 1 --2- 172.21.15.151:0/1601512611 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1bd810a780 0x7f1bd810e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:52.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.463+0000 7f1bdcc00700 1 -- 172.21.15.151:0/1601512611 >> 172.21.15.151:0/1601512611 conn(0x7f1bd8076d10 msgr2=0x7f1bd8108360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:03:52.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.463+0000 7f1bdcc00700 1 -- 172.21.15.151:0/1601512611 shutdown_connections 2024-05-21T22:03:52.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.463+0000 7f1bdcc00700 1 -- 172.21.15.151:0/1601512611 wait complete. 2024-05-21T22:03:52.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.464+0000 7f1bdcc00700 1 Processor -- start 2024-05-21T22:03:52.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.464+0000 7f1bdcc00700 1 -- start start 2024-05-21T22:03:52.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.464+0000 7f1bdcc00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1bd8109dd0 0x7f1bd81a0240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:52.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.464+0000 7f1bdcc00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1bd810a780 0x7f1bd81a0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:52.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.464+0000 7f1bdcc00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1bd810eeb0 0x7f1bd81a4b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:52.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.464+0000 7f1bdcc00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd8119d70 con 0x7f1bd8109dd0 2024-05-21T22:03:52.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.464+0000 7f1bdcc00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd8119bf0 con 0x7f1bd810eeb0 2024-05-21T22:03:52.467 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.464+0000 7f1bdcc00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd8119ef0 con 0x7f1bd810a780 2024-05-21T22:03:52.467 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.465+0000 7f1bd5e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1bd8109dd0 0x7f1bd81a0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:52.467 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.465+0000 7f1bd5e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1bd8109dd0 0x7f1bd81a0240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52104/0 (socket says 172.21.15.151:52104) 2024-05-21T22:03:52.467 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.465+0000 7f1bd6800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1bd810eeb0 0x7f1bd81a4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:52.467 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.465+0000 7f1bd5e00700 1 -- 172.21.15.151:0/2362197839 learned_addr learned my addr 172.21.15.151:0/2362197839 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:03:52.467 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.465+0000 7f1bd5400700 1 --2- 172.21.15.151:0/2362197839 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1bd810a780 0x7f1bd81a0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:52.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.465+0000 7f1bd6800700 1 -- 172.21.15.151:0/2362197839 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1bd810a780 msgr2=0x7f1bd81a0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:52.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.465+0000 7f1bd6800700 1 --2- 172.21.15.151:0/2362197839 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1bd810a780 0x7f1bd81a0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:52.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.465+0000 7f1bd6800700 1 -- 172.21.15.151:0/2362197839 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1bd8109dd0 msgr2=0x7f1bd81a0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:52.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.465+0000 7f1bd6800700 1 --2- 172.21.15.151:0/2362197839 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1bd8109dd0 0x7f1bd81a0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:52.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.465+0000 7f1bd6800700 1 -- 172.21.15.151:0/2362197839 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bd81a5250 con 0x7f1bd810eeb0 2024-05-21T22:03:52.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.465+0000 7f1bd5400700 1 --2- 172.21.15.151:0/2362197839 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1bd810a780 0x7f1bd81a0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:03:52.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.466+0000 7f1bd6800700 1 --2- 172.21.15.151:0/2362197839 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1bd810eeb0 0x7f1bd81a4b10 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f1bd0007f10 tx=0x7f1bd000d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:03:52.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.466+0000 7f1bcea00700 1 -- 172.21.15.151:0/2362197839 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd0004db0 con 0x7f1bd810eeb0 2024-05-21T22:03:52.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.466+0000 7f1bdcc00700 1 -- 172.21.15.151:0/2362197839 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bd81a5540 con 0x7f1bd810eeb0 2024-05-21T22:03:52.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.466+0000 7f1bcea00700 1 -- 172.21.15.151:0/2362197839 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1bd0003a00 con 0x7f1bd810eeb0 2024-05-21T22:03:52.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.466+0000 7f1bcea00700 1 -- 172.21.15.151:0/2362197839 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd0004580 con 0x7f1bd810eeb0 2024-05-21T22:03:52.469 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.466+0000 7f1bdcc00700 1 -- 172.21.15.151:0/2362197839 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bd81acdc0 con 0x7f1bd810eeb0 2024-05-21T22:03:52.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.470+0000 7f1bcea00700 1 -- 172.21.15.151:0/2362197839 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7f1bd001c020 con 0x7f1bd810eeb0 2024-05-21T22:03:52.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.471+0000 7f1be7551b80 1 Processor -- start 2024-05-21T22:03:52.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.471+0000 7f1be7551b80 1 -- start start 2024-05-21T22:03:52.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.471+0000 7f1be7551b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55953163d610 0x55953163d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:52.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.471+0000 7f1be7551b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55953163df30 0x559531637b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:52.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.471+0000 7f1be7551b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559531638140 0x5595316385c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:52.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.471+0000 7f1be7551b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x559531595e40 con 0x559531638140 2024-05-21T22:03:52.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.471+0000 7f1be7551b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5595315859e0 con 0x55953163d610 2024-05-21T22:03:52.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.471+0000 7f1be7551b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x559531589a20 con 0x55953163df30 2024-05-21T22:03:52.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.472+0000 7f1bd5400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55953163d610 0x55953163d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:52.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.472+0000 7f1bd5400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559531638140 0x5595316385c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:52.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.472+0000 7f1bd5e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55953163df30 0x559531637b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:52.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.472+0000 7f1bd5400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55953163d610 0x55953163d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:59610/0 (socket says 172.21.15.151:59610) 2024-05-21T22:03:52.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.472+0000 7f1bd5400700 1 -- 172.21.15.151:0/3384402386 learned_addr learned my addr 172.21.15.151:0/3384402386 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:03:52.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.472+0000 7f1bcea00700 1 --2- 172.21.15.151:0/2362197839 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f1bbc0b2a70 0x7f1bbc0b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:52.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.472+0000 7f1bd5e00700 1 -- 172.21.15.151:0/3384402386 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55953163d610 msgr2=0x55953163d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:52.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.472+0000 7f1bd5e00700 1 --2- 172.21.15.151:0/3384402386 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55953163d610 0x55953163d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:52.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.472+0000 7f1bd5e00700 1 -- 172.21.15.151:0/3384402386 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559531638140 msgr2=0x5595316385c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:52.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.472+0000 7f1bd5e00700 1 --2- 172.21.15.151:0/3384402386 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559531638140 0x5595316385c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:52.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.472+0000 7f1bd5e00700 1 -- 172.21.15.151:0/3384402386 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55953154e420 con 0x55953163df30 2024-05-21T22:03:52.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.472+0000 7f1bd6800700 1 --2- 172.21.15.151:0/2362197839 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f1bbc0b2a70 0x7f1bbc0b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:52.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.472+0000 7f1bcea00700 1 -- 172.21.15.151:0/2362197839 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7f1bd00e1cc0 con 0x7f1bd810eeb0 2024-05-21T22:03:52.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.473+0000 7f1bd5e00700 1 --2- 172.21.15.151:0/3384402386 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55953163df30 0x559531637b20 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f1bc80059d0 tx=0x7f1bc8005c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=79a0cb647090fd8e server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:03:52.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.473+0000 7f1bc3400700 1 -- 172.21.15.151:0/3384402386 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc8028070 con 0x55953163df30 2024-05-21T22:03:52.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.473+0000 7f1bc3400700 1 -- 172.21.15.151:0/3384402386 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1bc80037c0 con 0x55953163df30 2024-05-21T22:03:52.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.473+0000 7f1bc3400700 1 -- 172.21.15.151:0/3384402386 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc800b9d0 con 0x55953163df30 2024-05-21T22:03:52.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.474+0000 7f1be7551b80 1 -- 172.21.15.151:0/3384402386 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55953163df30 msgr2=0x559531637b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:52.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.474+0000 7f1bd6800700 1 --2- 172.21.15.151:0/2362197839 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f1bbc0b2a70 0x7f1bbc0b4f30 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1bd00a8ec0 tx=0x7f1bd0009530 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:03:52.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.474+0000 7f1be7551b80 1 --2- 172.21.15.151:0/3384402386 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55953163df30 0x559531637b20 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f1bc80059d0 tx=0x7f1bc8005c70 comp rx=0 tx=0).stop 2024-05-21T22:03:52.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.474+0000 7f1be7551b80 1 -- 172.21.15.151:0/3384402386 shutdown_connections 2024-05-21T22:03:52.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.474+0000 7f1be7551b80 1 --2- 172.21.15.151:0/3384402386 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55953163d610 0x55953163d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:52.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.474+0000 7f1be7551b80 1 --2- 172.21.15.151:0/3384402386 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559531638140 0x5595316385c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:52.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.474+0000 7f1be7551b80 1 --2- 172.21.15.151:0/3384402386 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55953163df30 0x559531637b20 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:52.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.474+0000 7f1be7551b80 1 -- 172.21.15.151:0/3384402386 >> 172.21.15.151:0/3384402386 conn(0x559531565be0 msgr2=0x559531632690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:03:52.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.474+0000 7f1be7551b80 1 -- 172.21.15.151:0/3384402386 shutdown_connections 2024-05-21T22:03:52.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.474+0000 7f1be7551b80 1 -- 172.21.15.151:0/3384402386 wait complete. 2024-05-21T22:03:52.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.475+0000 7f1be7551b80 1 -- start start 2024-05-21T22:03:52.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.475+0000 7f1be7551b80 1 Processor -- start 2024-05-21T22:03:52.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.475+0000 7f1be7551b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559531638140 0x559531643be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:52.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.475+0000 7f1be7551b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55953163d610 0x559531640f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:52.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.475+0000 7f1be7551b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55953163df30 0x5595316414c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:52.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.475+0000 7f1be7551b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55953148bfa0 con 0x55953163d610 2024-05-21T22:03:52.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.475+0000 7f1be7551b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x559531526760 con 0x55953163df30 2024-05-21T22:03:52.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.475+0000 7f1be7551b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x559531644980 con 0x559531638140 2024-05-21T22:03:52.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.475+0000 7f1be7551b80 10 client.0 _ll_register_callbacks cb 0x55953152fc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:03:52.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.476+0000 7f1bd5e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55953163d610 0x559531640f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:52.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.476+0000 7f1bd5400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559531638140 0x559531643be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:52.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.476+0000 7f1bd5e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55953163d610 0x559531640f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52124/0 (socket says 172.21.15.151:52124) 2024-05-21T22:03:52.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.476+0000 7f1bd5e00700 1 -- 172.21.15.151:0/1573449897 learned_addr learned my addr 172.21.15.151:0/1573449897 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:03:52.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.476+0000 7f1bd5400700 1 --2- 172.21.15.151:0/1573449897 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55953163df30 0x5595316414c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:52.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.476+0000 7f1bd5400700 1 -- 172.21.15.151:0/1573449897 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55953163df30 msgr2=0x5595316414c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:52.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.476+0000 7f1bd5400700 1 --2- 172.21.15.151:0/1573449897 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55953163df30 0x5595316414c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:52.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.476+0000 7f1bd5400700 1 -- 172.21.15.151:0/1573449897 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55953163d610 msgr2=0x559531640f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:52.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.476+0000 7f1bd5400700 1 --2- 172.21.15.151:0/1573449897 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55953163d610 0x559531640f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:52.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.476+0000 7f1bd5400700 1 -- 172.21.15.151:0/1573449897 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595314d58d0 con 0x559531638140 2024-05-21T22:03:52.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.476+0000 7f1bd5e00700 1 --2- 172.21.15.151:0/1573449897 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55953163d610 0x559531640f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:03:52.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.476+0000 7f1bd5400700 1 --2- 172.21.15.151:0/1573449897 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559531638140 0x559531643be0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f1bc4010fd0 tx=0x7f1bc4010430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74c94fa5c5f4b806 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:03:52.481 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.476+0000 7f1bb7e00700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-21T22:03:52.481 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.477+0000 7f1bb7e00700 1 -- 172.21.15.151:0/1573449897 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc4019040 con 0x559531638140 2024-05-21T22:03:52.481 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.477+0000 7f1be7551b80 1 -- 172.21.15.151:0/1573449897 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1bc80037c0 con 0x559531638140 2024-05-21T22:03:52.481 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.477+0000 7f1bb7e00700 1 -- 172.21.15.151:0/1573449897 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7f1bc4004830 con 0x559531638140 2024-05-21T22:03:52.481 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.477+0000 7f1bb7e00700 1 -- 172.21.15.151:0/1573449897 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1bc4004a30 con 0x559531638140 2024-05-21T22:03:52.481 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.477+0000 7f1be7551b80 10 client.27214 fetch_fsmap learned FSMap version 29 2024-05-21T22:03:52.481 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.477+0000 7f1be7551b80 1 -- 172.21.15.151:0/1573449897 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559531556060 con 0x559531638140 2024-05-21T22:03:52.481 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.477+0000 7f1bb7e00700 1 -- 172.21.15.151:0/1573449897 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1bc80037c0 con 0x559531638140 2024-05-21T22:03:52.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.478+0000 7f1bb7e00700 1 -- 172.21.15.151:0/1573449897 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f1bc402a470 con 0x559531638140 2024-05-21T22:03:52.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.478+0000 7f1be7551b80 10 client.27214 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:03:52.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.478+0000 7f1be7551b80 10 client.27214 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-05-21T22:03:52.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.478+0000 7f1be7551b80 20 client.27214 populate_metadata read hostname 'smithi151' 2024-05-21T22:03:52.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.478+0000 7f1be7551b80 1 --2- 172.21.15.151:0/1573449897 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x559531651450 0x559531653870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:52.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.478+0000 7f1be7551b80 4 client.27214 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T22:03:52.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.478+0000 7f1be7551b80 1 -- 172.21.15.151:0/1573449897 --> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559531597d90 con 0x559531651450 2024-05-21T22:03:52.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.478+0000 7f1bd5e00700 1 --2- 172.21.15.151:0/1573449897 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x559531651450 0x559531653870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:52.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.479+0000 7f1bd5e00700 1 --2- 172.21.15.151:0/1573449897 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x559531651450 0x559531653870 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b936212abca1629a server_cookie=d65010abebb0420a in_seq=0 out_seq=0 2024-05-21T22:03:52.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.479+0000 7f1bb7e00700 10 client.27214 ms_handle_connect on v2:172.21.15.164:6836/1554609572 2024-05-21T22:03:52.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.479+0000 7f1bb7e00700 1 -- 172.21.15.151:0/1573449897 <== mds.2 v2:172.21.15.164:6836/1554609572 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x559531597d90 con 0x559531651450 2024-05-21T22:03:52.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.479+0000 7f1bb7e00700 10 client.27214 handle_command_reply: tid=0 2024-05-21T22:03:52.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.480+0000 7f1be7551b80 1 client.27214 shutdown 2024-05-21T22:03:52.484 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.480+0000 7f1be7551b80 1 -- 172.21.15.151:0/1573449897 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559531638140 msgr2=0x559531643be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:52.484 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.480+0000 7f1be7551b80 1 --2- 172.21.15.151:0/1573449897 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559531638140 0x559531643be0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f1bc4010fd0 tx=0x7f1bc4010430 comp rx=0 tx=0).stop 2024-05-21T22:03:52.484 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.480+0000 7f1be7551b80 1 -- 172.21.15.151:0/1573449897 shutdown_connections 2024-05-21T22:03:52.484 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.480+0000 7f1be7551b80 1 --2- 172.21.15.151:0/1573449897 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x559531651450 0x559531653870 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:52.484 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.480+0000 7f1be7551b80 1 --2- 172.21.15.151:0/1573449897 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55953163df30 0x5595316414c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:52.484 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.480+0000 7f1be7551b80 1 --2- 172.21.15.151:0/1573449897 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559531638140 0x559531643be0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:52.484 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.480+0000 7f1be7551b80 1 --2- 172.21.15.151:0/1573449897 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55953163d610 0x559531640f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:52.484 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.480+0000 7f1bb7e00700 0 client.27214 ms_handle_reset on v2:172.21.15.164:6836/1554609572 2024-05-21T22:03:52.484 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.480+0000 7f1be7551b80 1 -- 172.21.15.151:0/1573449897 >> 172.21.15.151:0/1573449897 conn(0x559531565be0 msgr2=0x559531631d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:03:52.484 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.480+0000 7f1be7551b80 1 -- 172.21.15.151:0/1573449897 shutdown_connections 2024-05-21T22:03:52.484 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.480+0000 7f1be7551b80 1 -- 172.21.15.151:0/1573449897 wait complete. 2024-05-21T22:03:52.484 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.480+0000 7f1be7551b80 20 client.27214 trim_cache size 0 max 16384 2024-05-21T22:03:52.498 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.497+0000 7f1be7551b80 1 Processor -- start 2024-05-21T22:03:52.498 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.497+0000 7f1be7551b80 1 -- start start 2024-05-21T22:03:52.499 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.497+0000 7f1be7551b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559531638140 0x559531637b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:52.499 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.497+0000 7f1be7551b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55953163d610 0x559531641470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:52.499 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.497+0000 7f1be7551b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55953163df30 0x5595316324f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:52.499 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.497+0000 7f1be7551b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd81b4470 con 0x559531638140 2024-05-21T22:03:52.499 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.497+0000 7f1be7551b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd81b5fb0 con 0x55953163d610 2024-05-21T22:03:52.500 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.497+0000 7f1be7551b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd81b4b40 con 0x55953163df30 2024-05-21T22:03:52.500 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.497+0000 7f1bd5400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559531638140 0x559531637b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:52.500 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.498+0000 7f1bd5400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559531638140 0x559531637b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52140/0 (socket says 172.21.15.151:52140) 2024-05-21T22:03:52.500 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.498+0000 7f1bd5400700 1 -- 172.21.15.151:0/1245495307 learned_addr learned my addr 172.21.15.151:0/1245495307 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:03:52.500 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.498+0000 7f1bd5e00700 1 --2- 172.21.15.151:0/1245495307 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55953163d610 0x559531641470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:52.500 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.498+0000 7f1bd5400700 1 --2- 172.21.15.151:0/1245495307 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55953163df30 0x5595316324f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:52.500 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.498+0000 7f1bd5e00700 1 -- 172.21.15.151:0/1245495307 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55953163df30 msgr2=0x5595316324f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:52.500 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.498+0000 7f1bd5e00700 1 --2- 172.21.15.151:0/1245495307 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55953163df30 0x5595316324f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:52.500 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.498+0000 7f1bd5e00700 1 -- 172.21.15.151:0/1245495307 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559531638140 msgr2=0x559531637b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:52.500 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.498+0000 7f1bd5e00700 1 --2- 172.21.15.151:0/1245495307 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559531638140 0x559531637b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:52.500 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.498+0000 7f1bd5e00700 1 -- 172.21.15.151:0/1245495307 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55953163f510 con 0x55953163d610 2024-05-21T22:03:52.500 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.498+0000 7f1bd5400700 1 --2- 172.21.15.151:0/1245495307 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55953163df30 0x5595316324f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T22:03:52.500 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.498+0000 7f1bd5e00700 1 --2- 172.21.15.151:0/1245495307 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55953163d610 0x559531641470 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x559531633940 tx=0x7f1bc801a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61a65b385ac5b298 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:03:52.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.498+0000 7f1bc2a00700 1 -- 172.21.15.151:0/1245495307 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc8028040 con 0x55953163d610 2024-05-21T22:03:52.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.498+0000 7f1bc2a00700 1 -- 172.21.15.151:0/1245495307 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1bc8032070 con 0x55953163d610 2024-05-21T22:03:52.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.498+0000 7f1bc2a00700 1 -- 172.21.15.151:0/1245495307 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc800be00 con 0x55953163d610 2024-05-21T22:03:52.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.499+0000 7f1be7551b80 1 -- 172.21.15.151:0/1245495307 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55953163d610 msgr2=0x559531641470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:52.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.499+0000 7f1be7551b80 1 --2- 172.21.15.151:0/1245495307 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55953163d610 0x559531641470 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x559531633940 tx=0x7f1bc801a040 comp rx=0 tx=0).stop 2024-05-21T22:03:52.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.499+0000 7f1be7551b80 1 -- 172.21.15.151:0/1245495307 shutdown_connections 2024-05-21T22:03:52.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.499+0000 7f1be7551b80 1 --2- 172.21.15.151:0/1245495307 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55953163df30 0x5595316324f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:52.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.499+0000 7f1be7551b80 1 --2- 172.21.15.151:0/1245495307 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559531638140 0x559531637b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:52.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.499+0000 7f1be7551b80 1 --2- 172.21.15.151:0/1245495307 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55953163d610 0x559531641470 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:52.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.499+0000 7f1be7551b80 1 -- 172.21.15.151:0/1245495307 >> 172.21.15.151:0/1245495307 conn(0x559531565be0 msgr2=0x559531640d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:03:52.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.499+0000 7f1be7551b80 1 -- 172.21.15.151:0/1245495307 shutdown_connections 2024-05-21T22:03:52.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.499+0000 7f1be7551b80 1 -- 172.21.15.151:0/1245495307 wait complete. 2024-05-21T22:03:52.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.500+0000 7f1be7551b80 1 -- start start 2024-05-21T22:03:52.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.500+0000 7f1be7551b80 1 Processor -- start 2024-05-21T22:03:52.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.500+0000 7f1be7551b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559531638140 0x55953164a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:52.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.500+0000 7f1be7551b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55953163d610 0x5595316589f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:52.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.500+0000 7f1be7551b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55953163df30 0x559531658f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:52.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.500+0000 7f1be7551b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd81b58e0 con 0x55953163df30 2024-05-21T22:03:52.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.500+0000 7f1be7551b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd81b5210 con 0x55953163d610 2024-05-21T22:03:52.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.500+0000 7f1be7551b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd81b2430 con 0x559531638140 2024-05-21T22:03:52.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.500+0000 7f1be7551b80 10 client.0 _ll_register_callbacks cb 0x55953152fc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:03:52.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.501+0000 7f1bd5e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55953163d610 0x5595316589f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:52.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.501+0000 7f1bd5400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559531638140 0x55953164a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:52.504 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.501+0000 7f1bd5e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55953163d610 0x5595316589f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:59634/0 (socket says 172.21.15.151:59634) 2024-05-21T22:03:52.504 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.501+0000 7f1bd5e00700 1 -- 172.21.15.151:0/2586654556 learned_addr learned my addr 172.21.15.151:0/2586654556 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:03:52.504 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.501+0000 7f1bd5400700 1 --2- 172.21.15.151:0/2586654556 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55953163df30 0x559531658f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:52.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.501+0000 7f1bd5e00700 1 -- 172.21.15.151:0/2586654556 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559531638140 msgr2=0x55953164a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:52.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.501+0000 7f1bd5e00700 1 --2- 172.21.15.151:0/2586654556 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559531638140 0x55953164a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:52.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.501+0000 7f1bd5e00700 1 -- 172.21.15.151:0/2586654556 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55953163df30 msgr2=0x559531658f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:52.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.501+0000 7f1bd5e00700 1 --2- 172.21.15.151:0/2586654556 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55953163df30 0x559531658f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:52.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.501+0000 7f1bd5e00700 1 -- 172.21.15.151:0/2586654556 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559531659720 con 0x55953163d610 2024-05-21T22:03:52.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.501+0000 7f1bd5400700 1 --2- 172.21.15.151:0/2586654556 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55953163df30 0x559531658f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T22:03:52.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.502+0000 7f1bd5e00700 1 --2- 172.21.15.151:0/2586654556 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55953163d610 0x5595316589f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f1bc80098e0 tx=0x7f1bc8003e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40e2931332c285a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:03:52.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.502+0000 7f1bb7e00700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-21T22:03:52.507 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.502+0000 7f1bb7e00700 1 -- 172.21.15.151:0/2586654556 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc803d070 con 0x55953163d610 2024-05-21T22:03:52.507 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.502+0000 7f1bb7e00700 1 -- 172.21.15.151:0/2586654556 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7f1bc800ba80 con 0x55953163d610 2024-05-21T22:03:52.507 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.502+0000 7f1be7551b80 1 -- 172.21.15.151:0/2586654556 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1bc8032070 con 0x55953163d610 2024-05-21T22:03:52.507 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.502+0000 7f1bb7e00700 1 -- 172.21.15.151:0/2586654556 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1bc8049050 con 0x55953163d610 2024-05-21T22:03:52.507 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.502+0000 7f1be7551b80 10 client.27215 fetch_fsmap learned FSMap version 29 2024-05-21T22:03:52.507 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.502+0000 7f1be7551b80 1 -- 172.21.15.151:0/2586654556 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5595316bbfb0 con 0x55953163d610 2024-05-21T22:03:52.507 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.502+0000 7f1bb7e00700 1 -- 172.21.15.151:0/2586654556 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1bc8032070 con 0x55953163d610 2024-05-21T22:03:52.507 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.503+0000 7f1bb7e00700 1 -- 172.21.15.151:0/2586654556 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f1bc8023760 con 0x55953163d610 2024-05-21T22:03:52.507 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.503+0000 7f1be7551b80 10 client.27215 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:03:52.507 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.503+0000 7f1be7551b80 10 client.27215 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-05-21T22:03:52.507 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.503+0000 7f1be7551b80 20 client.27215 populate_metadata read hostname 'smithi151' 2024-05-21T22:03:52.507 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.503+0000 7f1be7551b80 1 --2- 172.21.15.151:0/2586654556 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x5595316bc1e0 0x5595316be5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:52.507 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.503+0000 7f1be7551b80 4 client.27215 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T22:03:52.507 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.503+0000 7f1be7551b80 1 -- 172.21.15.151:0/2586654556 --> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559531591570 con 0x5595316bc1e0 2024-05-21T22:03:52.508 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.503+0000 7f1bd5400700 1 --2- 172.21.15.151:0/2586654556 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x5595316bc1e0 0x5595316be5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:52.508 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.504+0000 7f1bd5400700 1 --2- 172.21.15.151:0/2586654556 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x5595316bc1e0 0x5595316be5d0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7e7c18be108a6c8e server_cookie=258ba66cd0884aac in_seq=0 out_seq=0 2024-05-21T22:03:52.508 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.504+0000 7f1bb7e00700 10 client.27215 ms_handle_connect on v2:172.21.15.164:6836/1554609572 2024-05-21T22:03:52.508 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.504+0000 7f1bb7e00700 1 -- 172.21.15.151:0/2586654556 <== mds.2 v2:172.21.15.164:6836/1554609572 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559531591570 con 0x5595316bc1e0 2024-05-21T22:03:52.508 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.504+0000 7f1bb7e00700 10 client.27215 handle_command_reply: tid=0 2024-05-21T22:03:52.508 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.504+0000 7f1be7551b80 1 client.27215 shutdown 2024-05-21T22:03:52.508 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.504+0000 7f1be7551b80 1 -- 172.21.15.151:0/2586654556 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55953163d610 msgr2=0x5595316589f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:52.508 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.504+0000 7f1be7551b80 1 --2- 172.21.15.151:0/2586654556 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55953163d610 0x5595316589f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f1bc80098e0 tx=0x7f1bc8003e20 comp rx=0 tx=0).stop 2024-05-21T22:03:52.508 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.504+0000 7f1be7551b80 1 -- 172.21.15.151:0/2586654556 shutdown_connections 2024-05-21T22:03:52.508 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.504+0000 7f1be7551b80 1 --2- 172.21.15.151:0/2586654556 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x5595316bc1e0 0x5595316be5d0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:52.508 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.504+0000 7f1be7551b80 1 --2- 172.21.15.151:0/2586654556 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55953163df30 0x559531658f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:52.508 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.504+0000 7f1be7551b80 1 --2- 172.21.15.151:0/2586654556 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559531638140 0x55953164a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:52.508 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.504+0000 7f1be7551b80 1 --2- 172.21.15.151:0/2586654556 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55953163d610 0x5595316589f0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:52.509 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.504+0000 7f1bb7e00700 0 client.27215 ms_handle_reset on v2:172.21.15.164:6836/1554609572 2024-05-21T22:03:52.509 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.504+0000 7f1be7551b80 1 -- 172.21.15.151:0/2586654556 >> 172.21.15.151:0/2586654556 conn(0x559531565be0 msgr2=0x55953163e310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:03:52.509 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.505+0000 7f1be7551b80 1 -- 172.21.15.151:0/2586654556 shutdown_connections 2024-05-21T22:03:52.509 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.505+0000 7f1be7551b80 1 -- 172.21.15.151:0/2586654556 wait complete. 2024-05-21T22:03:52.509 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.505+0000 7f1be7551b80 20 client.27215 trim_cache size 0 max 16384 2024-05-21T22:03:52.509 INFO:teuthology.orchestra.run.smithi151.stdout:[] 2024-05-21T22:03:52.510 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.505+0000 7f1bdcc00700 1 -- 172.21.15.151:0/2362197839 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f1bbc0b2a70 msgr2=0x7f1bbc0b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:52.510 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.505+0000 7f1bdcc00700 1 --2- 172.21.15.151:0/2362197839 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f1bbc0b2a70 0x7f1bbc0b4f30 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1bd00a8ec0 tx=0x7f1bd0009530 comp rx=0 tx=0).stop 2024-05-21T22:03:52.510 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.505+0000 7f1bdcc00700 1 -- 172.21.15.151:0/2362197839 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1bd810eeb0 msgr2=0x7f1bd81a4b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:52.510 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.505+0000 7f1bdcc00700 1 --2- 172.21.15.151:0/2362197839 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1bd810eeb0 0x7f1bd81a4b10 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f1bd0007f10 tx=0x7f1bd000d3b0 comp rx=0 tx=0).stop 2024-05-21T22:03:52.510 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.506+0000 7f1bdcc00700 1 -- 172.21.15.151:0/2362197839 shutdown_connections 2024-05-21T22:03:52.510 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.506+0000 7f1bdcc00700 1 --2- 172.21.15.151:0/2362197839 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f1bbc0b2a70 0x7f1bbc0b4f30 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:52.510 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.506+0000 7f1bdcc00700 1 --2- 172.21.15.151:0/2362197839 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1bd810eeb0 0x7f1bd81a4b10 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:52.510 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.506+0000 7f1bdcc00700 1 --2- 172.21.15.151:0/2362197839 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1bd8109dd0 0x7f1bd81a0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:52.510 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.506+0000 7f1bdcc00700 1 --2- 172.21.15.151:0/2362197839 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1bd810a780 0x7f1bd81a0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:52.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.506+0000 7f1bdcc00700 1 -- 172.21.15.151:0/2362197839 >> 172.21.15.151:0/2362197839 conn(0x7f1bd8076d10 msgr2=0x7f1bd810daf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:03:52.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.506+0000 7f1bdcc00700 1 -- 172.21.15.151:0/2362197839 shutdown_connections 2024-05-21T22:03:52.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:52.506+0000 7f1bdcc00700 1 -- 172.21.15.151:0/2362197839 wait complete. 2024-05-21T22:03:53.223 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-21T22:03:53.730 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:03:53.874 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:03:53 smithi151 ceph-mon[31623]: pgmap v255: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T22:03:53.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:03:53 smithi184 ceph-mon[36942]: pgmap v255: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T22:03:54.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:03:53 smithi164 ceph-mon[36269]: pgmap v255: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T22:03:55.358 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:03:55 smithi151 ceph-mon[31623]: pgmap v256: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:03:55.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:03:55 smithi184 ceph-mon[36942]: pgmap v256: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:03:55.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:03:55 smithi164 ceph-mon[36269]: pgmap v256: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:03:55.840 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.838+0000 7f477ca00700 1 -- 172.21.15.151:0/358727299 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4778104690 msgr2=0x7f4778104a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:55.840 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.838+0000 7f477ca00700 1 --2- 172.21.15.151:0/358727299 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4778104690 0x7f4778104a70 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f4768009990 tx=0x7f4768009ca0 comp rx=0 tx=0).stop 2024-05-21T22:03:55.840 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.839+0000 7f477ca00700 1 -- 172.21.15.151:0/358727299 shutdown_connections 2024-05-21T22:03:55.840 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.839+0000 7f477ca00700 1 --2- 172.21.15.151:0/358727299 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f477810da70 0x7f477810ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:55.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.839+0000 7f477ca00700 1 --2- 172.21.15.151:0/358727299 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4778104690 0x7f4778104a70 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:55.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.839+0000 7f477ca00700 1 --2- 172.21.15.151:0/358727299 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4778104fb0 0x7f477810d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:55.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.839+0000 7f477ca00700 1 -- 172.21.15.151:0/358727299 >> 172.21.15.151:0/358727299 conn(0x7f47780fde40 msgr2=0x7f4778100260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:03:55.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.839+0000 7f477ca00700 1 -- 172.21.15.151:0/358727299 shutdown_connections 2024-05-21T22:03:55.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.839+0000 7f477ca00700 1 -- 172.21.15.151:0/358727299 wait complete. 2024-05-21T22:03:55.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.840+0000 7f477ca00700 1 Processor -- start 2024-05-21T22:03:55.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.840+0000 7f477ca00700 1 -- start start 2024-05-21T22:03:55.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.840+0000 7f477ca00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4778104690 0x7f47781022f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:55.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.840+0000 7f477ca00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4778104fb0 0x7f4778102830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:55.843 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.840+0000 7f477ca00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f477810da70 0x7f4778102d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:55.843 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.840+0000 7f477ca00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f47781136c0 con 0x7f4778104690 2024-05-21T22:03:55.843 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.840+0000 7f477ca00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4778113540 con 0x7f477810da70 2024-05-21T22:03:55.843 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.840+0000 7f477ca00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f4778113840 con 0x7f4778104fb0 2024-05-21T22:03:55.843 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.841+0000 7f4777400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4778104690 0x7f47781022f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:55.843 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.841+0000 7f4777400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4778104690 0x7f47781022f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49814/0 (socket says 172.21.15.151:49814) 2024-05-21T22:03:55.843 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.841+0000 7f4777400700 1 -- 172.21.15.151:0/1160932069 learned_addr learned my addr 172.21.15.151:0/1160932069 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:03:55.843 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.841+0000 7f4776a00700 1 --2- 172.21.15.151:0/1160932069 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4778104fb0 0x7f4778102830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:55.843 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.841+0000 7f4777e00700 1 --2- 172.21.15.151:0/1160932069 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f477810da70 0x7f4778102d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:55.843 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.841+0000 7f4777400700 1 -- 172.21.15.151:0/1160932069 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4778104fb0 msgr2=0x7f4778102830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:55.843 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.841+0000 7f4777400700 1 --2- 172.21.15.151:0/1160932069 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4778104fb0 0x7f4778102830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:55.843 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.841+0000 7f4777400700 1 -- 172.21.15.151:0/1160932069 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f477810da70 msgr2=0x7f4778102d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:55.843 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.841+0000 7f4777400700 1 --2- 172.21.15.151:0/1160932069 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f477810da70 0x7f4778102d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:55.843 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.841+0000 7f4777400700 1 -- 172.21.15.151:0/1160932069 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47781a8120 con 0x7f4778104690 2024-05-21T22:03:55.844 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.841+0000 7f4777400700 1 --2- 172.21.15.151:0/1160932069 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4778104690 0x7f47781022f0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f4768000c00 tx=0x7f47680057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:03:55.844 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.842+0000 7f476fe00700 1 -- 172.21.15.151:0/1160932069 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4768027070 con 0x7f4778104690 2024-05-21T22:03:55.844 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.842+0000 7f476fe00700 1 -- 172.21.15.151:0/1160932069 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4768022470 con 0x7f4778104690 2024-05-21T22:03:55.844 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.842+0000 7f477ca00700 1 -- 172.21.15.151:0/1160932069 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47781a83b0 con 0x7f4778104690 2024-05-21T22:03:55.844 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.842+0000 7f476fe00700 1 -- 172.21.15.151:0/1160932069 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4768017650 con 0x7f4778104690 2024-05-21T22:03:55.844 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.842+0000 7f477ca00700 1 -- 172.21.15.151:0/1160932069 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47781a88e0 con 0x7f4778104690 2024-05-21T22:03:55.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.845+0000 7f476fe00700 1 -- 172.21.15.151:0/1160932069 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7f476802a070 con 0x7f4778104690 2024-05-21T22:03:55.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.847+0000 7f47889c5b80 1 Processor -- start 2024-05-21T22:03:55.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.847+0000 7f47889c5b80 1 -- start start 2024-05-21T22:03:55.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.847+0000 7f476fe00700 1 --2- 172.21.15.151:0/1160932069 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f475c0b2a70 0x7f475c0b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:55.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.847+0000 7f47889c5b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557970f55020 0x557970f55400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:55.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.847+0000 7f47889c5b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557970f55940 0x557970f4f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:55.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.847+0000 7f47889c5b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557970f4fad0 0x557970f4ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:55.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.847+0000 7f47889c5b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x557970eade40 con 0x557970f55940 2024-05-21T22:03:55.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.847+0000 7f47889c5b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x557970e9d9e0 con 0x557970f4fad0 2024-05-21T22:03:55.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.847+0000 7f47889c5b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x557970ea1a20 con 0x557970f55020 2024-05-21T22:03:55.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.847+0000 7f476fe00700 1 -- 172.21.15.151:0/1160932069 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7f47680e32a0 con 0x7f4778104690 2024-05-21T22:03:55.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.848+0000 7f4777e00700 1 --2- 172.21.15.151:0/1160932069 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f475c0b2a70 0x7f475c0b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:55.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.848+0000 7f4777400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557970f55020 0x557970f55400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:55.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.848+0000 7f4777400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557970f55020 0x557970f55400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:54268/0 (socket says 172.21.15.151:54268) 2024-05-21T22:03:55.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.848+0000 7f4777400700 1 -- 172.21.15.151:0/1992780442 learned_addr learned my addr 172.21.15.151:0/1992780442 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:03:55.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.848+0000 7f4776a00700 1 --2- 172.21.15.151:0/1992780442 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557970f55940 0x557970f4f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:55.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.848+0000 7f4777e00700 1 --2- 172.21.15.151:0/1992780442 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557970f4fad0 0x557970f4ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:55.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.848+0000 7f4777400700 1 -- 172.21.15.151:0/1992780442 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557970f4fad0 msgr2=0x557970f4ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:55.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.848+0000 7f4777400700 1 --2- 172.21.15.151:0/1992780442 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557970f4fad0 0x557970f4ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:55.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.848+0000 7f4777400700 1 -- 172.21.15.151:0/1992780442 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557970f55940 msgr2=0x557970f4f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:55.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.848+0000 7f4777400700 1 --2- 172.21.15.151:0/1992780442 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557970f55940 0x557970f4f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:55.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.848+0000 7f4777400700 1 -- 172.21.15.151:0/1992780442 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557970e66420 con 0x557970f55020 2024-05-21T22:03:55.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.848+0000 7f4777e00700 1 --2- 172.21.15.151:0/1992780442 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557970f4fad0 0x557970f4ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T22:03:55.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.848+0000 7f4777400700 1 --2- 172.21.15.151:0/1992780442 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557970f55020 0x557970f55400 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f47680e0920 tx=0x7f47680a6590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72a2f65c072e086d server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:03:55.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.849+0000 7f4777e00700 1 --2- 172.21.15.151:0/1160932069 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f475c0b2a70 0x7f475c0b4f30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4770009da0 tx=0x7f4770009920 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:03:55.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.849+0000 7f476cc00700 1 -- 172.21.15.151:0/1992780442 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47680e3070 con 0x557970f55020 2024-05-21T22:03:55.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.849+0000 7f476cc00700 1 -- 172.21.15.151:0/1992780442 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47680aa0b0 con 0x557970f55020 2024-05-21T22:03:55.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.849+0000 7f476cc00700 1 -- 172.21.15.151:0/1992780442 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47680b3370 con 0x557970f55020 2024-05-21T22:03:55.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.850+0000 7f47889c5b80 1 -- 172.21.15.151:0/1992780442 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557970f55020 msgr2=0x557970f55400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:55.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.850+0000 7f47889c5b80 1 --2- 172.21.15.151:0/1992780442 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557970f55020 0x557970f55400 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f47680e0920 tx=0x7f47680a6590 comp rx=0 tx=0).stop 2024-05-21T22:03:55.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.850+0000 7f47889c5b80 1 -- 172.21.15.151:0/1992780442 shutdown_connections 2024-05-21T22:03:55.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.850+0000 7f47889c5b80 1 --2- 172.21.15.151:0/1992780442 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557970f4fad0 0x557970f4ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:55.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.850+0000 7f47889c5b80 1 --2- 172.21.15.151:0/1992780442 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557970f55020 0x557970f55400 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:55.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.850+0000 7f47889c5b80 1 --2- 172.21.15.151:0/1992780442 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557970f55940 0x557970f4f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:55.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.850+0000 7f47889c5b80 1 -- 172.21.15.151:0/1992780442 >> 172.21.15.151:0/1992780442 conn(0x557970e7dbe0 msgr2=0x557970f4a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:03:55.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.850+0000 7f47889c5b80 1 -- 172.21.15.151:0/1992780442 shutdown_connections 2024-05-21T22:03:55.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.850+0000 7f47889c5b80 1 -- 172.21.15.151:0/1992780442 wait complete. 2024-05-21T22:03:55.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.850+0000 7f47889c5b80 1 -- start start 2024-05-21T22:03:55.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.851+0000 7f47889c5b80 1 Processor -- start 2024-05-21T22:03:55.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.851+0000 7f47889c5b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557970f4fad0 0x557970f5b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:55.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.851+0000 7f47889c5b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557970f55020 0x557970f58a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:55.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.851+0000 7f47889c5b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557970f55940 0x557970f58fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:55.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.851+0000 7f47889c5b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x557970da3fa0 con 0x557970f55020 2024-05-21T22:03:55.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.851+0000 7f47889c5b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x557970e3e760 con 0x557970f4fad0 2024-05-21T22:03:55.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.851+0000 7f47889c5b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x557970f5c220 con 0x557970f55940 2024-05-21T22:03:55.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.851+0000 7f47889c5b80 10 client.0 _ll_register_callbacks cb 0x557970e47c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:03:55.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.851+0000 7f4776a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557970f55020 0x557970f58a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:55.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.851+0000 7f4777e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557970f55940 0x557970f58fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:55.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.852+0000 7f4776a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557970f55020 0x557970f58a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49828/0 (socket says 172.21.15.151:49828) 2024-05-21T22:03:55.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.852+0000 7f4776a00700 1 -- 172.21.15.151:0/3023096048 learned_addr learned my addr 172.21.15.151:0/3023096048 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:03:55.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.852+0000 7f4777400700 1 --2- 172.21.15.151:0/3023096048 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557970f4fad0 0x557970f5b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:55.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.852+0000 7f4776a00700 1 -- 172.21.15.151:0/3023096048 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557970f55940 msgr2=0x557970f58fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:55.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.852+0000 7f4776a00700 1 --2- 172.21.15.151:0/3023096048 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557970f55940 0x557970f58fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:55.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.852+0000 7f4776a00700 1 -- 172.21.15.151:0/3023096048 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557970f4fad0 msgr2=0x557970f5b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:55.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.852+0000 7f4776a00700 1 --2- 172.21.15.151:0/3023096048 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557970f4fad0 0x557970f5b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:55.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.852+0000 7f4776a00700 1 -- 172.21.15.151:0/3023096048 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557970ded8d0 con 0x557970f55020 2024-05-21T22:03:55.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.852+0000 7f4777400700 1 --2- 172.21.15.151:0/3023096048 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557970f4fad0 0x557970f5b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:03:55.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.852+0000 7f4776a00700 1 --2- 172.21.15.151:0/3023096048 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557970f55020 0x557970f58a80 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f4764012770 tx=0x7f4764012a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef83d99c347b82e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:03:55.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.852+0000 7f4761600700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-21T22:03:55.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.853+0000 7f4761600700 1 -- 172.21.15.151:0/3023096048 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4764004020 con 0x557970f55020 2024-05-21T22:03:55.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.853+0000 7f4761600700 1 -- 172.21.15.151:0/3023096048 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7f47640041c0 con 0x557970f55020 2024-05-21T22:03:55.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.853+0000 7f47889c5b80 1 -- 172.21.15.151:0/3023096048 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47680aa0b0 con 0x557970f55020 2024-05-21T22:03:55.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.853+0000 7f4761600700 1 -- 172.21.15.151:0/3023096048 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f47640043c0 con 0x557970f55020 2024-05-21T22:03:55.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.853+0000 7f47889c5b80 10 client.17103 fetch_fsmap learned FSMap version 29 2024-05-21T22:03:55.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.853+0000 7f47889c5b80 1 -- 172.21.15.151:0/3023096048 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557970e6e060 con 0x557970f55020 2024-05-21T22:03:55.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.853+0000 7f4761600700 1 -- 172.21.15.151:0/3023096048 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47680aa0b0 con 0x557970f55020 2024-05-21T22:03:55.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.853+0000 7f4761600700 1 -- 172.21.15.151:0/3023096048 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f476400f510 con 0x557970f55020 2024-05-21T22:03:55.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.854+0000 7f47889c5b80 10 client.17103 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:03:55.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.854+0000 7f47889c5b80 10 client.17103 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-21T22:03:55.858 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.854+0000 7f47889c5b80 20 client.17103 populate_metadata read hostname 'smithi151' 2024-05-21T22:03:55.858 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.854+0000 7f47889c5b80 1 --2- 172.21.15.151:0/3023096048 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x557970f68d60 0x557970f6b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:55.858 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.854+0000 7f47889c5b80 4 client.17103 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T22:03:55.858 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.854+0000 7f47889c5b80 1 -- 172.21.15.151:0/3023096048 --> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557970eafd90 con 0x557970f68d60 2024-05-21T22:03:55.858 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.854+0000 7f4777400700 1 --2- 172.21.15.151:0/3023096048 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x557970f68d60 0x557970f6b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:55.858 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.854+0000 7f4777400700 1 --2- 172.21.15.151:0/3023096048 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x557970f68d60 0x557970f6b150 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e7b4e2438a15b1d2 server_cookie=6bd9fecdba6886c6 in_seq=0 out_seq=0 2024-05-21T22:03:55.858 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.854+0000 7f4761600700 10 client.17103 ms_handle_connect on v2:172.21.15.184:6838/3059935143 2024-05-21T22:03:55.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.855+0000 7f4761600700 1 -- 172.21.15.151:0/3023096048 <== mds.1 v2:172.21.15.184:6838/3059935143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x557970eafd90 con 0x557970f68d60 2024-05-21T22:03:55.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.855+0000 7f4761600700 10 client.17103 handle_command_reply: tid=0 2024-05-21T22:03:55.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.855+0000 7f47889c5b80 1 client.17103 shutdown 2024-05-21T22:03:55.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.855+0000 7f47889c5b80 1 -- 172.21.15.151:0/3023096048 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557970f55020 msgr2=0x557970f58a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:55.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.855+0000 7f47889c5b80 1 --2- 172.21.15.151:0/3023096048 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557970f55020 0x557970f58a80 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f4764012770 tx=0x7f4764012a80 comp rx=0 tx=0).stop 2024-05-21T22:03:55.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.856+0000 7f47889c5b80 1 -- 172.21.15.151:0/3023096048 shutdown_connections 2024-05-21T22:03:55.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.856+0000 7f47889c5b80 1 --2- 172.21.15.151:0/3023096048 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x557970f68d60 0x557970f6b150 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:55.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.856+0000 7f47889c5b80 1 --2- 172.21.15.151:0/3023096048 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557970f4fad0 0x557970f5b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:55.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.856+0000 7f47889c5b80 1 --2- 172.21.15.151:0/3023096048 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557970f55940 0x557970f58fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:55.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.856+0000 7f47889c5b80 1 --2- 172.21.15.151:0/3023096048 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557970f55020 0x557970f58a80 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:55.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.856+0000 7f47889c5b80 1 -- 172.21.15.151:0/3023096048 >> 172.21.15.151:0/3023096048 conn(0x557970e7dbe0 msgr2=0x557970f49fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:03:55.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.856+0000 7f4761600700 0 client.17103 ms_handle_reset on v2:172.21.15.184:6838/3059935143 2024-05-21T22:03:55.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.856+0000 7f47889c5b80 1 -- 172.21.15.151:0/3023096048 shutdown_connections 2024-05-21T22:03:55.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.856+0000 7f47889c5b80 1 -- 172.21.15.151:0/3023096048 wait complete. 2024-05-21T22:03:55.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.856+0000 7f47889c5b80 20 client.17103 trim_cache size 0 max 16384 2024-05-21T22:03:55.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.873+0000 7f47889c5b80 1 Processor -- start 2024-05-21T22:03:55.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.873+0000 7f47889c5b80 1 -- start start 2024-05-21T22:03:55.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.873+0000 7f47889c5b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557970f4fad0 0x557970f4f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:55.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.873+0000 7f47889c5b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557970f55020 0x557970f58a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:55.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.873+0000 7f47889c5b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557970f55940 0x557970f58f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:55.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.873+0000 7f47889c5b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f47781aff90 con 0x557970f4fad0 2024-05-21T22:03:55.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.873+0000 7f47889c5b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f47781b1ad0 con 0x557970f55940 2024-05-21T22:03:55.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.873+0000 7f47889c5b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f47781b0660 con 0x557970f55020 2024-05-21T22:03:55.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.874+0000 7f4777400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557970f4fad0 0x557970f4f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:55.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.874+0000 7f4776a00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557970f55020 0x557970f58a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:55.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.874+0000 7f4777400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557970f4fad0 0x557970f4f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49832/0 (socket says 172.21.15.151:49832) 2024-05-21T22:03:55.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.874+0000 7f4777400700 1 -- 172.21.15.151:0/3859236700 learned_addr learned my addr 172.21.15.151:0/3859236700 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:03:55.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.874+0000 7f4777e00700 1 --2- 172.21.15.151:0/3859236700 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557970f55940 0x557970f58f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:55.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.874+0000 7f4776a00700 1 -- 172.21.15.151:0/3859236700 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557970f55940 msgr2=0x557970f58f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:55.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.874+0000 7f4776a00700 1 --2- 172.21.15.151:0/3859236700 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557970f55940 0x557970f58f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:55.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.874+0000 7f4776a00700 1 -- 172.21.15.151:0/3859236700 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557970f4fad0 msgr2=0x557970f4f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:55.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.874+0000 7f4776a00700 1 --2- 172.21.15.151:0/3859236700 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557970f4fad0 0x557970f4f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:55.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.874+0000 7f4776a00700 1 -- 172.21.15.151:0/3859236700 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557970f423e0 con 0x557970f55020 2024-05-21T22:03:55.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.874+0000 7f4777400700 1 --2- 172.21.15.151:0/3859236700 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557970f4fad0 0x557970f4f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:03:55.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.874+0000 7f4776a00700 1 --2- 172.21.15.151:0/3859236700 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557970f55020 0x557970f58a40 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f476400fb20 tx=0x7f4764005780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58894689d6c8634e server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:03:55.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.875+0000 7f4763e00700 1 -- 172.21.15.151:0/3859236700 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4764029040 con 0x557970f55020 2024-05-21T22:03:55.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.875+0000 7f4763e00700 1 -- 172.21.15.151:0/3859236700 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f476400fb50 con 0x557970f55020 2024-05-21T22:03:55.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.875+0000 7f4763e00700 1 -- 172.21.15.151:0/3859236700 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f476401cb80 con 0x557970f55020 2024-05-21T22:03:55.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.875+0000 7f47889c5b80 1 -- 172.21.15.151:0/3859236700 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557970f55020 msgr2=0x557970f58a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:55.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.875+0000 7f47889c5b80 1 --2- 172.21.15.151:0/3859236700 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557970f55020 0x557970f58a40 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f476400fb20 tx=0x7f4764005780 comp rx=0 tx=0).stop 2024-05-21T22:03:55.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.876+0000 7f47889c5b80 1 -- 172.21.15.151:0/3859236700 shutdown_connections 2024-05-21T22:03:55.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.876+0000 7f47889c5b80 1 --2- 172.21.15.151:0/3859236700 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557970f55940 0x557970f58f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:55.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.876+0000 7f47889c5b80 1 --2- 172.21.15.151:0/3859236700 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557970f4fad0 0x557970f4f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:55.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.876+0000 7f47889c5b80 1 --2- 172.21.15.151:0/3859236700 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557970f55020 0x557970f58a40 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:55.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.876+0000 7f47889c5b80 1 -- 172.21.15.151:0/3859236700 >> 172.21.15.151:0/3859236700 conn(0x557970e7dbe0 msgr2=0x557970f57370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:03:55.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.876+0000 7f47889c5b80 1 -- 172.21.15.151:0/3859236700 shutdown_connections 2024-05-21T22:03:55.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.876+0000 7f47889c5b80 1 -- 172.21.15.151:0/3859236700 wait complete. 2024-05-21T22:03:55.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.876+0000 7f47889c5b80 1 -- start start 2024-05-21T22:03:55.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.876+0000 7f47889c5b80 1 Processor -- start 2024-05-21T22:03:55.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.877+0000 7f47889c5b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557970f4fad0 0x557970f61880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:55.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.877+0000 7f47889c5b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557970f55020 0x557970f6fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:55.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.877+0000 7f47889c5b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557970f55940 0x557970f70530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:55.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.877+0000 7f47889c5b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f47781b1400 con 0x557970f55020 2024-05-21T22:03:55.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.877+0000 7f47889c5b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f47781b0d30 con 0x557970f4fad0 2024-05-21T22:03:55.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.877+0000 7f47889c5b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f47781adf50 con 0x557970f55940 2024-05-21T22:03:55.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.877+0000 7f47889c5b80 10 client.0 _ll_register_callbacks cb 0x557970e47c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:03:55.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.877+0000 7f4777400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557970f4fad0 0x557970f61880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:55.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.877+0000 7f4776a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557970f55020 0x557970f6fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:55.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.877+0000 7f4777400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557970f4fad0 0x557970f61880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:35932/0 (socket says 172.21.15.151:35932) 2024-05-21T22:03:55.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.877+0000 7f4777400700 1 -- 172.21.15.151:0/3857561290 learned_addr learned my addr 172.21.15.151:0/3857561290 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:03:55.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.877+0000 7f4777e00700 1 --2- 172.21.15.151:0/3857561290 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557970f55940 0x557970f70530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:55.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.878+0000 7f4776a00700 1 -- 172.21.15.151:0/3857561290 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557970f55940 msgr2=0x557970f70530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:55.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.878+0000 7f4776a00700 1 --2- 172.21.15.151:0/3857561290 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557970f55940 0x557970f70530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:55.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.878+0000 7f4776a00700 1 -- 172.21.15.151:0/3857561290 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557970f4fad0 msgr2=0x557970f61880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:55.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.878+0000 7f4776a00700 1 --2- 172.21.15.151:0/3857561290 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557970f4fad0 0x557970f61880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:55.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.878+0000 7f4776a00700 1 -- 172.21.15.151:0/3857561290 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557970f70a70 con 0x557970f55020 2024-05-21T22:03:55.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.878+0000 7f4776a00700 1 --2- 172.21.15.151:0/3857561290 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557970f55020 0x557970f6fff0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f476400f9b0 tx=0x7f4764009ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ed57a89a76c785c server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:03:55.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.878+0000 7f4761600700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-21T22:03:55.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.878+0000 7f4761600700 1 -- 172.21.15.151:0/3857561290 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4764029040 con 0x557970f55020 2024-05-21T22:03:55.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.879+0000 7f4761600700 1 -- 172.21.15.151:0/3857561290 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7f4764005020 con 0x557970f55020 2024-05-21T22:03:55.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.879+0000 7f47889c5b80 1 -- 172.21.15.151:0/3857561290 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f476400fb50 con 0x557970f55020 2024-05-21T22:03:55.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.879+0000 7f4761600700 1 -- 172.21.15.151:0/3857561290 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f476403c050 con 0x557970f55020 2024-05-21T22:03:55.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.879+0000 7f47889c5b80 10 client.17109 fetch_fsmap learned FSMap version 29 2024-05-21T22:03:55.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.879+0000 7f47889c5b80 1 -- 172.21.15.151:0/3857561290 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557970fdbe90 con 0x557970f55020 2024-05-21T22:03:55.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.879+0000 7f4761600700 1 -- 172.21.15.151:0/3857561290 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f476400fb50 con 0x557970f55020 2024-05-21T22:03:55.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.879+0000 7f4761600700 1 -- 172.21.15.151:0/3857561290 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f4764035060 con 0x557970f55020 2024-05-21T22:03:55.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.880+0000 7f47889c5b80 10 client.17109 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:03:55.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.880+0000 7f47889c5b80 10 client.17109 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-21T22:03:55.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.880+0000 7f47889c5b80 20 client.17109 populate_metadata read hostname 'smithi151' 2024-05-21T22:03:55.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.880+0000 7f47889c5b80 1 --2- 172.21.15.151:0/3857561290 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x557970fdc3d0 0x557970fde830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:55.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.880+0000 7f47889c5b80 4 client.17109 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T22:03:55.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.880+0000 7f47889c5b80 1 -- 172.21.15.151:0/3857561290 --> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557970ea9570 con 0x557970fdc3d0 2024-05-21T22:03:55.884 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.880+0000 7f4777400700 1 --2- 172.21.15.151:0/3857561290 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x557970fdc3d0 0x557970fde830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:55.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.880+0000 7f4777400700 1 --2- 172.21.15.151:0/3857561290 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x557970fdc3d0 0x557970fde830 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3de3d947fc0d6998 server_cookie=991fc90d07142d5e in_seq=0 out_seq=0 2024-05-21T22:03:55.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.880+0000 7f4761600700 10 client.17109 ms_handle_connect on v2:172.21.15.184:6838/3059935143 2024-05-21T22:03:55.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.881+0000 7f4761600700 1 -- 172.21.15.151:0/3857561290 <== mds.1 v2:172.21.15.184:6838/3059935143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557970ea9570 con 0x557970fdc3d0 2024-05-21T22:03:55.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.881+0000 7f4761600700 10 client.17109 handle_command_reply: tid=0 2024-05-21T22:03:55.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.881+0000 7f47889c5b80 1 client.17109 shutdown 2024-05-21T22:03:55.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.881+0000 7f47889c5b80 1 -- 172.21.15.151:0/3857561290 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557970f55020 msgr2=0x557970f6fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:55.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.881+0000 7f47889c5b80 1 --2- 172.21.15.151:0/3857561290 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557970f55020 0x557970f6fff0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f476400f9b0 tx=0x7f4764009ea0 comp rx=0 tx=0).stop 2024-05-21T22:03:55.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.881+0000 7f47889c5b80 1 -- 172.21.15.151:0/3857561290 shutdown_connections 2024-05-21T22:03:55.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.881+0000 7f47889c5b80 1 --2- 172.21.15.151:0/3857561290 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x557970fdc3d0 0x557970fde830 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:55.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.881+0000 7f47889c5b80 1 --2- 172.21.15.151:0/3857561290 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557970f4fad0 0x557970f61880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:55.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.881+0000 7f47889c5b80 1 --2- 172.21.15.151:0/3857561290 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557970f55940 0x557970f70530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:55.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.881+0000 7f47889c5b80 1 --2- 172.21.15.151:0/3857561290 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557970f55020 0x557970f6fff0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:55.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.881+0000 7f4761600700 0 client.17109 ms_handle_reset on v2:172.21.15.184:6838/3059935143 2024-05-21T22:03:55.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.881+0000 7f47889c5b80 1 -- 172.21.15.151:0/3857561290 >> 172.21.15.151:0/3857561290 conn(0x557970e7dbe0 msgr2=0x557970f572d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:03:55.886 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.881+0000 7f47889c5b80 1 -- 172.21.15.151:0/3857561290 shutdown_connections 2024-05-21T22:03:55.886 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.881+0000 7f47889c5b80 1 -- 172.21.15.151:0/3857561290 wait complete. 2024-05-21T22:03:55.886 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.881+0000 7f47889c5b80 20 client.17109 trim_cache size 0 max 16384 2024-05-21T22:03:55.886 INFO:teuthology.orchestra.run.smithi151.stdout:[] 2024-05-21T22:03:55.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.882+0000 7f477ca00700 1 -- 172.21.15.151:0/1160932069 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f475c0b2a70 msgr2=0x7f475c0b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:55.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.882+0000 7f477ca00700 1 --2- 172.21.15.151:0/1160932069 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f475c0b2a70 0x7f475c0b4f30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4770009da0 tx=0x7f4770009920 comp rx=0 tx=0).stop 2024-05-21T22:03:55.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.882+0000 7f477ca00700 1 -- 172.21.15.151:0/1160932069 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4778104690 msgr2=0x7f47781022f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:55.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.882+0000 7f477ca00700 1 --2- 172.21.15.151:0/1160932069 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4778104690 0x7f47781022f0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f4768000c00 tx=0x7f47680057c0 comp rx=0 tx=0).stop 2024-05-21T22:03:55.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.882+0000 7f477ca00700 1 -- 172.21.15.151:0/1160932069 shutdown_connections 2024-05-21T22:03:55.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.882+0000 7f477ca00700 1 --2- 172.21.15.151:0/1160932069 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f475c0b2a70 0x7f475c0b4f30 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:55.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.882+0000 7f477ca00700 1 --2- 172.21.15.151:0/1160932069 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f477810da70 0x7f4778102d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:55.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.882+0000 7f477ca00700 1 --2- 172.21.15.151:0/1160932069 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4778104690 0x7f47781022f0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:55.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.882+0000 7f477ca00700 1 --2- 172.21.15.151:0/1160932069 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4778104fb0 0x7f4778102830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:55.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.882+0000 7f477ca00700 1 -- 172.21.15.151:0/1160932069 >> 172.21.15.151:0/1160932069 conn(0x7f47780fde40 msgr2=0x7f47780febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:03:55.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.882+0000 7f477ca00700 1 -- 172.21.15.151:0/1160932069 shutdown_connections 2024-05-21T22:03:55.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:55.882+0000 7f477ca00700 1 -- 172.21.15.151:0/1160932069 wait complete. 2024-05-21T22:03:56.893 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-21T22:03:57.350 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:03:57.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:03:57 smithi184 ceph-mon[36942]: pgmap v257: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:03:57.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:03:57 smithi164 ceph-mon[36269]: pgmap v257: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:03:57.796 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:03:57 smithi151 ceph-mon[31623]: pgmap v257: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:03:59.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.554+0000 7f24bec00700 1 -- 172.21.15.151:0/1306478658 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f24b81095a0 msgr2=0x7f24b8109980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:59.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.554+0000 7f24bec00700 1 --2- 172.21.15.151:0/1306478658 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f24b81095a0 0x7f24b8109980 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f24a8009990 tx=0x7f24a8009ca0 comp rx=0 tx=0).stop 2024-05-21T22:03:59.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.554+0000 7f24bec00700 1 -- 172.21.15.151:0/1306478658 shutdown_connections 2024-05-21T22:03:59.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.554+0000 7f24bec00700 1 --2- 172.21.15.151:0/1306478658 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f24b806c470 0x7f24b81103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:59.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.554+0000 7f24bec00700 1 --2- 172.21.15.151:0/1306478658 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f24b81095a0 0x7f24b8109980 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:59.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.554+0000 7f24bec00700 1 --2- 172.21.15.151:0/1306478658 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f24b806bab0 0x7f24b806bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:59.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.554+0000 7f24bec00700 1 -- 172.21.15.151:0/1306478658 >> 172.21.15.151:0/1306478658 conn(0x7f24b80793b0 msgr2=0x7f24b80797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:03:59.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.554+0000 7f24bec00700 1 -- 172.21.15.151:0/1306478658 shutdown_connections 2024-05-21T22:03:59.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.554+0000 7f24bec00700 1 -- 172.21.15.151:0/1306478658 wait complete. 2024-05-21T22:03:59.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.555+0000 7f24bec00700 1 Processor -- start 2024-05-21T22:03:59.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.556+0000 7f24bec00700 1 -- start start 2024-05-21T22:03:59.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.556+0000 7f24bec00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f24b806bab0 0x7f24b819f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:59.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.556+0000 7f24bec00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f24b806c470 0x7f24b819fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:59.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.556+0000 7f24bec00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f24b81095a0 0x7f24b8199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:59.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.556+0000 7f24bec00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f24b81136c0 con 0x7f24b806bab0 2024-05-21T22:03:59.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.556+0000 7f24bec00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f24b8113540 con 0x7f24b806c470 2024-05-21T22:03:59.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.556+0000 7f24bec00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f24b8113840 con 0x7f24b81095a0 2024-05-21T22:03:59.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.557+0000 7f24b7e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f24b806bab0 0x7f24b819f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:59.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.557+0000 7f24b7400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f24b806c470 0x7f24b819fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:59.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.557+0000 7f24b7400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f24b806c470 0x7f24b819fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:35958/0 (socket says 172.21.15.151:35958) 2024-05-21T22:03:59.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.557+0000 7f24b7400700 1 -- 172.21.15.151:0/2940140653 learned_addr learned my addr 172.21.15.151:0/2940140653 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:03:59.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.557+0000 7f24bcc00700 1 --2- 172.21.15.151:0/2940140653 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f24b81095a0 0x7f24b8199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:59.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.557+0000 7f24b7400700 1 -- 172.21.15.151:0/2940140653 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f24b81095a0 msgr2=0x7f24b8199a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:59.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.557+0000 7f24b7400700 1 --2- 172.21.15.151:0/2940140653 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f24b81095a0 0x7f24b8199a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:59.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.557+0000 7f24b7400700 1 -- 172.21.15.151:0/2940140653 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f24b806bab0 msgr2=0x7f24b819f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:59.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.557+0000 7f24b7400700 1 --2- 172.21.15.151:0/2940140653 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f24b806bab0 0x7f24b819f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:59.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.557+0000 7f24b7400700 1 -- 172.21.15.151:0/2940140653 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24b819a160 con 0x7f24b806c470 2024-05-21T22:03:59.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.557+0000 7f24b7e00700 1 --2- 172.21.15.151:0/2940140653 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f24b806bab0 0x7f24b819f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:03:59.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.557+0000 7f24b7400700 1 --2- 172.21.15.151:0/2940140653 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f24b806c470 0x7f24b819fd80 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f24a400eb30 tx=0x7f24a400ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:03:59.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.558+0000 7f24b4c00700 1 -- 172.21.15.151:0/2940140653 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a400cc90 con 0x7f24b806c470 2024-05-21T22:03:59.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.558+0000 7f24bec00700 1 -- 172.21.15.151:0/2940140653 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24b819a450 con 0x7f24b806c470 2024-05-21T22:03:59.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.558+0000 7f24b4c00700 1 -- 172.21.15.151:0/2940140653 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24a40044d0 con 0x7f24b806c470 2024-05-21T22:03:59.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.558+0000 7f24b4c00700 1 -- 172.21.15.151:0/2940140653 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a4010430 con 0x7f24b806c470 2024-05-21T22:03:59.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.558+0000 7f24bec00700 1 -- 172.21.15.151:0/2940140653 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24b81a6710 con 0x7f24b806c470 2024-05-21T22:03:59.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.562+0000 7f24b4c00700 1 -- 172.21.15.151:0/2940140653 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7f24a4003ea0 con 0x7f24b806c470 2024-05-21T22:03:59.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.563+0000 7f24c958cb80 1 Processor -- start 2024-05-21T22:03:59.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.563+0000 7f24c958cb80 1 -- start start 2024-05-21T22:03:59.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.563+0000 7f24c958cb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f0bbb02c0 0x560f0bbb06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:59.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.563+0000 7f24c958cb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f0bbb0be0 0x560f0bbaa670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:59.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.563+0000 7f24c958cb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f0bbaac90 0x560f0bbab0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:59.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.563+0000 7f24c958cb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x560f0bb04e40 con 0x560f0bbaac90 2024-05-21T22:03:59.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.563+0000 7f24c958cb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x560f0baf49e0 con 0x560f0bbb02c0 2024-05-21T22:03:59.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.563+0000 7f24c958cb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x560f0baf8a20 con 0x560f0bbb0be0 2024-05-21T22:03:59.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.564+0000 7f24bcc00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f0bbb02c0 0x560f0bbb06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:59.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.564+0000 7f24bcc00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f0bbb02c0 0x560f0bbb06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:35974/0 (socket says 172.21.15.151:35974) 2024-05-21T22:03:59.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.564+0000 7f24bcc00700 1 -- 172.21.15.151:0/2091346097 learned_addr learned my addr 172.21.15.151:0/2091346097 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:03:59.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.564+0000 7f24b7e00700 1 --2- 172.21.15.151:0/2091346097 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f0bbb0be0 0x560f0bbaa670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:59.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.564+0000 7f24b7400700 1 --2- 172.21.15.151:0/2091346097 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f0bbaac90 0x560f0bbab0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:59.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.564+0000 7f24bcc00700 1 -- 172.21.15.151:0/2091346097 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f0bbb0be0 msgr2=0x560f0bbaa670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:59.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.564+0000 7f24bcc00700 1 --2- 172.21.15.151:0/2091346097 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f0bbb0be0 0x560f0bbaa670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:59.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.564+0000 7f24bcc00700 1 -- 172.21.15.151:0/2091346097 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f0bbaac90 msgr2=0x560f0bbab0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:59.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.564+0000 7f24bcc00700 1 --2- 172.21.15.151:0/2091346097 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f0bbaac90 0x560f0bbab0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:59.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.564+0000 7f24bcc00700 1 -- 172.21.15.151:0/2091346097 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f0babd420 con 0x560f0bbb02c0 2024-05-21T22:03:59.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.565+0000 7f24bcc00700 1 --2- 172.21.15.151:0/2091346097 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f0bbb02c0 0x560f0bbb06a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f24b0009440 tx=0x7f24b0009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4641e09f84fddb45 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:03:59.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.565+0000 7f24ad600700 1 -- 172.21.15.151:0/2091346097 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b0013750 con 0x560f0bbb02c0 2024-05-21T22:03:59.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.565+0000 7f24ad600700 1 -- 172.21.15.151:0/2091346097 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24b0013da0 con 0x560f0bbb02c0 2024-05-21T22:03:59.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.565+0000 7f24ad600700 1 -- 172.21.15.151:0/2091346097 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b00143e0 con 0x560f0bbb02c0 2024-05-21T22:03:59.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.565+0000 7f24b4c00700 1 --2- 172.21.15.151:0/2940140653 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f249c0b29f0 0x7f249c0b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:59.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.566+0000 7f24b7e00700 1 --2- 172.21.15.151:0/2940140653 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f249c0b29f0 0x7f249c0b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:59.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.566+0000 7f24b4c00700 1 -- 172.21.15.151:0/2940140653 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7f24a40e2930 con 0x7f24b806c470 2024-05-21T22:03:59.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.566+0000 7f24c958cb80 1 -- 172.21.15.151:0/2091346097 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f0bbb02c0 msgr2=0x560f0bbb06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:59.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.566+0000 7f24c958cb80 1 --2- 172.21.15.151:0/2091346097 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f0bbb02c0 0x560f0bbb06a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f24b0009440 tx=0x7f24b0009920 comp rx=0 tx=0).stop 2024-05-21T22:03:59.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.566+0000 7f24c958cb80 1 -- 172.21.15.151:0/2091346097 shutdown_connections 2024-05-21T22:03:59.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.566+0000 7f24c958cb80 1 --2- 172.21.15.151:0/2091346097 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f0bbb02c0 0x560f0bbb06a0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:59.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.566+0000 7f24c958cb80 1 --2- 172.21.15.151:0/2091346097 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f0bbaac90 0x560f0bbab0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:59.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.566+0000 7f24c958cb80 1 --2- 172.21.15.151:0/2091346097 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f0bbb0be0 0x560f0bbaa670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:59.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.566+0000 7f24c958cb80 1 -- 172.21.15.151:0/2091346097 >> 172.21.15.151:0/2091346097 conn(0x560f0bad4be0 msgr2=0x560f0bba1010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:03:59.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.566+0000 7f24c958cb80 1 -- 172.21.15.151:0/2091346097 shutdown_connections 2024-05-21T22:03:59.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.566+0000 7f24c958cb80 1 -- 172.21.15.151:0/2091346097 wait complete. 2024-05-21T22:03:59.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.567+0000 7f24b7e00700 1 --2- 172.21.15.151:0/2940140653 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f249c0b29f0 0x7f249c0b4eb0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f24a800b2e0 tx=0x7f24a8005040 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:03:59.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.567+0000 7f24c958cb80 1 -- start start 2024-05-21T22:03:59.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.567+0000 7f24c958cb80 1 Processor -- start 2024-05-21T22:03:59.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.568+0000 7f24c958cb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f0bbaac90 0x560f0bbb68f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:59.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.568+0000 7f24c958cb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f0bbb02c0 0x560f0bbb3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:59.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.568+0000 7f24c958cb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f0bbb0be0 0x560f0bbb41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:59.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.568+0000 7f24c958cb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x560f0b9fafa0 con 0x560f0bbb0be0 2024-05-21T22:03:59.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.568+0000 7f24c958cb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x560f0ba95760 con 0x560f0bbb02c0 2024-05-21T22:03:59.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.568+0000 7f24c958cb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x560f0bbb7650 con 0x560f0bbaac90 2024-05-21T22:03:59.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.568+0000 7f24c958cb80 10 client.0 _ll_register_callbacks cb 0x560f0ba9ec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:03:59.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.568+0000 7f24bcc00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f0bbb0be0 0x560f0bbb41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:59.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.568+0000 7f24b7e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f0bbaac90 0x560f0bbb68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:59.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.568+0000 7f24bcc00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f0bbb0be0 0x560f0bbb41d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49870/0 (socket says 172.21.15.151:49870) 2024-05-21T22:03:59.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.568+0000 7f24bcc00700 1 -- 172.21.15.151:0/1086364745 learned_addr learned my addr 172.21.15.151:0/1086364745 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:03:59.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.568+0000 7f24b7400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f0bbb02c0 0x560f0bbb3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:59.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.569+0000 7f24bcc00700 1 -- 172.21.15.151:0/1086364745 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f0bbaac90 msgr2=0x560f0bbb68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:59.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.569+0000 7f24bcc00700 1 --2- 172.21.15.151:0/1086364745 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f0bbaac90 0x560f0bbb68f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:59.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.569+0000 7f24bcc00700 1 -- 172.21.15.151:0/1086364745 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f0bbb02c0 msgr2=0x560f0bbb3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:59.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.569+0000 7f24bcc00700 1 --2- 172.21.15.151:0/1086364745 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f0bbb02c0 0x560f0bbb3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:59.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.569+0000 7f24bcc00700 1 -- 172.21.15.151:0/1086364745 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f0ba448d0 con 0x560f0bbb0be0 2024-05-21T22:03:59.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.569+0000 7f24b7e00700 1 --2- 172.21.15.151:0/1086364745 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f0bbaac90 0x560f0bbb68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:03:59.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.569+0000 7f24b7400700 1 --2- 172.21.15.151:0/1086364745 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f0bbb02c0 0x560f0bbb3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:03:59.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.569+0000 7f24bcc00700 1 --2- 172.21.15.151:0/1086364745 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f0bbb0be0 0x560f0bbb41d0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f24b0013f90 tx=0x7f24b0016d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=276c0800ad533d61 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:03:59.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.569+0000 7f24a2000700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-21T22:03:59.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.569+0000 7f24a2000700 1 -- 172.21.15.151:0/1086364745 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b00095e0 con 0x560f0bbb0be0 2024-05-21T22:03:59.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.569+0000 7f24a2000700 1 -- 172.21.15.151:0/1086364745 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7f24b0013550 con 0x560f0bbb0be0 2024-05-21T22:03:59.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.569+0000 7f24c958cb80 1 -- 172.21.15.151:0/1086364745 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24b0013da0 con 0x560f0bbb0be0 2024-05-21T22:03:59.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.570+0000 7f24a2000700 1 -- 172.21.15.151:0/1086364745 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f24b0013750 con 0x560f0bbb0be0 2024-05-21T22:03:59.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.570+0000 7f24c958cb80 10 client.17121 fetch_fsmap learned FSMap version 29 2024-05-21T22:03:59.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.570+0000 7f24c958cb80 1 -- 172.21.15.151:0/1086364745 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560f0bac5060 con 0x560f0bbb0be0 2024-05-21T22:03:59.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.570+0000 7f24a2000700 1 -- 172.21.15.151:0/1086364745 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24b0013da0 con 0x560f0bbb0be0 2024-05-21T22:03:59.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.570+0000 7f24a2000700 1 -- 172.21.15.151:0/1086364745 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f24b0005680 con 0x560f0bbb0be0 2024-05-21T22:03:59.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.570+0000 7f24c958cb80 10 client.17121 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:03:59.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.570+0000 7f24c958cb80 10 client.17121 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T22:03:59.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.570+0000 7f24c958cb80 20 client.17121 populate_metadata read hostname 'smithi151' 2024-05-21T22:03:59.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.570+0000 7f24c958cb80 1 --2- 172.21.15.151:0/1086364745 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x560f0bbc40c0 0x560f0bbc64b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:59.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.570+0000 7f24c958cb80 4 client.17121 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T22:03:59.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.570+0000 7f24c958cb80 1 -- 172.21.15.151:0/1086364745 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560f0bb06d90 con 0x560f0bbc40c0 2024-05-21T22:03:59.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.571+0000 7f24b7e00700 1 --2- 172.21.15.151:0/1086364745 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x560f0bbc40c0 0x560f0bbc64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:59.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.572+0000 7f24b7e00700 1 --2- 172.21.15.151:0/1086364745 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x560f0bbc40c0 0x560f0bbc64b0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7e758b3eb467973 server_cookie=2a76794fe653affb in_seq=0 out_seq=0 2024-05-21T22:03:59.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.572+0000 7f24a2000700 10 client.17121 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T22:03:59.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.572+0000 7f24a2000700 1 -- 172.21.15.151:0/1086364745 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x560f0bb06d90 con 0x560f0bbc40c0 2024-05-21T22:03:59.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.572+0000 7f24a2000700 10 client.17121 handle_command_reply: tid=0 2024-05-21T22:03:59.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.573+0000 7f24c958cb80 1 client.17121 shutdown 2024-05-21T22:03:59.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.573+0000 7f24c958cb80 1 -- 172.21.15.151:0/1086364745 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f0bbb0be0 msgr2=0x560f0bbb41d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:59.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.573+0000 7f24c958cb80 1 --2- 172.21.15.151:0/1086364745 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f0bbb0be0 0x560f0bbb41d0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f24b0013f90 tx=0x7f24b0016d20 comp rx=0 tx=0).stop 2024-05-21T22:03:59.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.573+0000 7f24c958cb80 1 -- 172.21.15.151:0/1086364745 shutdown_connections 2024-05-21T22:03:59.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.573+0000 7f24c958cb80 1 --2- 172.21.15.151:0/1086364745 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f0bbb0be0 0x560f0bbb41d0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:59.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.573+0000 7f24c958cb80 1 --2- 172.21.15.151:0/1086364745 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f0bbaac90 0x560f0bbb68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:59.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.573+0000 7f24c958cb80 1 --2- 172.21.15.151:0/1086364745 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x560f0bbc40c0 0x560f0bbc64b0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:59.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.573+0000 7f24c958cb80 1 --2- 172.21.15.151:0/1086364745 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f0bbb02c0 0x560f0bbb3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:59.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.573+0000 7f24c958cb80 1 -- 172.21.15.151:0/1086364745 >> 172.21.15.151:0/1086364745 conn(0x560f0bad4be0 msgr2=0x560f0bba9660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:03:59.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.573+0000 7f24a2000700 0 client.17121 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T22:03:59.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.573+0000 7f24c958cb80 1 -- 172.21.15.151:0/1086364745 shutdown_connections 2024-05-21T22:03:59.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.573+0000 7f24c958cb80 1 -- 172.21.15.151:0/1086364745 wait complete. 2024-05-21T22:03:59.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.573+0000 7f24c958cb80 20 client.17121 trim_cache size 0 max 16384 2024-05-21T22:03:59.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.590+0000 7f24c958cb80 1 Processor -- start 2024-05-21T22:03:59.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.590+0000 7f24c958cb80 1 -- start start 2024-05-21T22:03:59.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.590+0000 7f24c958cb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f0bbaac90 0x560f0bbaa670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:59.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.590+0000 7f24c958cb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f0bbb02c0 0x560f0bbb68f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:59.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.590+0000 7f24c958cb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f0bbb0be0 0x560f0bbb6130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:59.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.590+0000 7f24c958cb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f24b81abd30 con 0x560f0bbb0be0 2024-05-21T22:03:59.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.590+0000 7f24c958cb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f24b81ad870 con 0x560f0bbaac90 2024-05-21T22:03:59.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.590+0000 7f24c958cb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f24b81ac400 con 0x560f0bbb02c0 2024-05-21T22:03:59.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.591+0000 7f24bcc00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f0bbb0be0 0x560f0bbb6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:59.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.591+0000 7f24bcc00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f0bbb0be0 0x560f0bbb6130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49886/0 (socket says 172.21.15.151:49886) 2024-05-21T22:03:59.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.591+0000 7f24bcc00700 1 -- 172.21.15.151:0/4187788967 learned_addr learned my addr 172.21.15.151:0/4187788967 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:03:59.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.591+0000 7f24b7e00700 1 --2- 172.21.15.151:0/4187788967 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f0bbaac90 0x560f0bbaa670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:59.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.591+0000 7f24b7400700 1 --2- 172.21.15.151:0/4187788967 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f0bbb02c0 0x560f0bbb68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:59.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.591+0000 7f24bcc00700 1 -- 172.21.15.151:0/4187788967 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f0bbb02c0 msgr2=0x560f0bbb68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:59.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.591+0000 7f24bcc00700 1 --2- 172.21.15.151:0/4187788967 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f0bbb02c0 0x560f0bbb68f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:59.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.591+0000 7f24bcc00700 1 -- 172.21.15.151:0/4187788967 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f0bbaac90 msgr2=0x560f0bbaa670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:59.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.591+0000 7f24bcc00700 1 --2- 172.21.15.151:0/4187788967 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f0bbaac90 0x560f0bbaa670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:59.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.591+0000 7f24bcc00700 1 -- 172.21.15.151:0/4187788967 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f0bbb2240 con 0x560f0bbb0be0 2024-05-21T22:03:59.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.591+0000 7f24b7e00700 1 --2- 172.21.15.151:0/4187788967 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f0bbaac90 0x560f0bbaa670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:03:59.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.591+0000 7f24bcc00700 1 --2- 172.21.15.151:0/4187788967 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f0bbb0be0 0x560f0bbb6130 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f24b0014040 tx=0x7f24b0029380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2de2465d0d6aa03f server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:03:59.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.592+0000 7f24acc00700 1 -- 172.21.15.151:0/4187788967 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b0035030 con 0x560f0bbb0be0 2024-05-21T22:03:59.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.592+0000 7f24acc00700 1 -- 172.21.15.151:0/4187788967 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24b00095e0 con 0x560f0bbb0be0 2024-05-21T22:03:59.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.592+0000 7f24acc00700 1 -- 172.21.15.151:0/4187788967 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b0029b30 con 0x560f0bbb0be0 2024-05-21T22:03:59.595 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.592+0000 7f24c958cb80 1 -- 172.21.15.151:0/4187788967 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f0bbb0be0 msgr2=0x560f0bbb6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:59.595 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.592+0000 7f24c958cb80 1 --2- 172.21.15.151:0/4187788967 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f0bbb0be0 0x560f0bbb6130 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f24b0014040 tx=0x7f24b0029380 comp rx=0 tx=0).stop 2024-05-21T22:03:59.595 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.592+0000 7f24c958cb80 1 -- 172.21.15.151:0/4187788967 shutdown_connections 2024-05-21T22:03:59.595 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.592+0000 7f24c958cb80 1 --2- 172.21.15.151:0/4187788967 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f0bbaac90 0x560f0bbaa670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:59.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.592+0000 7f24c958cb80 1 --2- 172.21.15.151:0/4187788967 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f0bbb0be0 0x560f0bbb6130 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:59.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.592+0000 7f24c958cb80 1 --2- 172.21.15.151:0/4187788967 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f0bbb02c0 0x560f0bbb68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:59.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.592+0000 7f24c958cb80 1 -- 172.21.15.151:0/4187788967 >> 172.21.15.151:0/4187788967 conn(0x560f0bad4be0 msgr2=0x560f0bba0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:03:59.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.593+0000 7f24c958cb80 1 -- 172.21.15.151:0/4187788967 shutdown_connections 2024-05-21T22:03:59.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.593+0000 7f24c958cb80 1 -- 172.21.15.151:0/4187788967 wait complete. 2024-05-21T22:03:59.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.593+0000 7f24c958cb80 1 -- start start 2024-05-21T22:03:59.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.593+0000 7f24c958cb80 1 Processor -- start 2024-05-21T22:03:59.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.594+0000 7f24c958cb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f0bbaac90 0x560f0bbc66b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:59.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.594+0000 7f24c958cb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f0bbb02c0 0x560f0bbc6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:59.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.594+0000 7f24c958cb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f0bbb0be0 0x560f0bc2c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:59.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.594+0000 7f24c958cb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f24b81ad1a0 con 0x560f0bbb0be0 2024-05-21T22:03:59.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.594+0000 7f24c958cb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f24b81acad0 con 0x560f0bbb02c0 2024-05-21T22:03:59.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.594+0000 7f24c958cb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f24b81a9cf0 con 0x560f0bbaac90 2024-05-21T22:03:59.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.594+0000 7f24c958cb80 10 client.0 _ll_register_callbacks cb 0x560f0ba9ec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:03:59.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.594+0000 7f24bcc00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f0bbb0be0 0x560f0bc2c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:59.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.594+0000 7f24b7400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f0bbb02c0 0x560f0bbc6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:59.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.594+0000 7f24b7e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f0bbaac90 0x560f0bbc66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:59.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.594+0000 7f24b7400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f0bbb02c0 0x560f0bbc6bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:35996/0 (socket says 172.21.15.151:35996) 2024-05-21T22:03:59.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.594+0000 7f24bcc00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f0bbb0be0 0x560f0bc2c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49902/0 (socket says 172.21.15.151:49902) 2024-05-21T22:03:59.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.594+0000 7f24b7e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f0bbaac90 0x560f0bbc66b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:54340/0 (socket says 172.21.15.151:54340) 2024-05-21T22:03:59.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.594+0000 7f24b7400700 1 -- 172.21.15.151:0/3591492641 learned_addr learned my addr 172.21.15.151:0/3591492641 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:03:59.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.595+0000 7f24b7400700 1 -- 172.21.15.151:0/3591492641 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f0bbaac90 msgr2=0x560f0bbc66b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:59.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.595+0000 7f24b7400700 1 --2- 172.21.15.151:0/3591492641 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f0bbaac90 0x560f0bbc66b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:59.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.595+0000 7f24b7400700 1 -- 172.21.15.151:0/3591492641 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f0bbb0be0 msgr2=0x560f0bc2c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:03:59.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.595+0000 7f24b7400700 1 --2- 172.21.15.151:0/3591492641 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f0bbb0be0 0x560f0bc2c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:59.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.595+0000 7f24b7400700 1 -- 172.21.15.151:0/3591492641 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f0bc2ce30 con 0x560f0bbb02c0 2024-05-21T22:03:59.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.595+0000 7f24bcc00700 1 --2- 172.21.15.151:0/3591492641 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f0bbb0be0 0x560f0bc2c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:03:59.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.595+0000 7f24b7e00700 1 --2- 172.21.15.151:0/3591492641 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f0bbaac90 0x560f0bbc66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:03:59.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.595+0000 7f24b7400700 1 --2- 172.21.15.151:0/3591492641 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f0bbb02c0 0x560f0bbc6bf0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f24a40a6e10 tx=0x7f24a40038b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=724548987c2ec30a server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:03:59.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.595+0000 7f24a2000700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-21T22:03:59.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.595+0000 7f24a2000700 1 -- 172.21.15.151:0/3591492641 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a40a5330 con 0x560f0bbb02c0 2024-05-21T22:03:59.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.595+0000 7f24c958cb80 1 -- 172.21.15.151:0/3591492641 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24b00095e0 con 0x560f0bbb02c0 2024-05-21T22:03:59.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.595+0000 7f24a2000700 1 -- 172.21.15.151:0/3591492641 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7f24a40b2780 con 0x560f0bbb02c0 2024-05-21T22:03:59.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.595+0000 7f24a2000700 1 -- 172.21.15.151:0/3591492641 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f24a40b2340 con 0x560f0bbb02c0 2024-05-21T22:03:59.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.596+0000 7f24c958cb80 10 client.27251 fetch_fsmap learned FSMap version 29 2024-05-21T22:03:59.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.596+0000 7f24c958cb80 1 -- 172.21.15.151:0/3591492641 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560f0bc2e1b0 con 0x560f0bbb02c0 2024-05-21T22:03:59.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.596+0000 7f24a2000700 1 -- 172.21.15.151:0/3591492641 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24b00095e0 con 0x560f0bbb02c0 2024-05-21T22:03:59.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.596+0000 7f24a2000700 1 -- 172.21.15.151:0/3591492641 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f24a40a5510 con 0x560f0bbb02c0 2024-05-21T22:03:59.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.596+0000 7f24c958cb80 10 client.27251 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:03:59.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.596+0000 7f24c958cb80 10 client.27251 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T22:03:59.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.596+0000 7f24c958cb80 20 client.27251 populate_metadata read hostname 'smithi151' 2024-05-21T22:03:59.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.596+0000 7f24c958cb80 1 --2- 172.21.15.151:0/3591492641 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x560f0bc2e870 0x560f0bc30c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:03:59.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.596+0000 7f24c958cb80 4 client.27251 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T22:03:59.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.596+0000 7f24c958cb80 1 -- 172.21.15.151:0/3591492641 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560f0bb00570 con 0x560f0bc2e870 2024-05-21T22:03:59.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.596+0000 7f24b7e00700 1 --2- 172.21.15.151:0/3591492641 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x560f0bc2e870 0x560f0bc30c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:03:59.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.597+0000 7f24b7e00700 1 --2- 172.21.15.151:0/3591492641 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x560f0bc2e870 0x560f0bc30c90 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=663e212158eb27a8 server_cookie=50ec0b5c2e0cb632 in_seq=0 out_seq=0 2024-05-21T22:03:59.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.597+0000 7f24a2000700 10 client.27251 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T22:03:59.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.597+0000 7f24a2000700 1 -- 172.21.15.151:0/3591492641 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560f0bb00570 con 0x560f0bc2e870 2024-05-21T22:03:59.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.597+0000 7f24a2000700 10 client.27251 handle_command_reply: tid=0 2024-05-21T22:03:59.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.597+0000 7f24c958cb80 1 client.27251 shutdown 2024-05-21T22:03:59.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.597+0000 7f24c958cb80 1 -- 172.21.15.151:0/3591492641 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f0bbb02c0 msgr2=0x560f0bbc6bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:59.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.597+0000 7f24c958cb80 1 --2- 172.21.15.151:0/3591492641 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f0bbb02c0 0x560f0bbc6bf0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f24a40a6e10 tx=0x7f24a40038b0 comp rx=0 tx=0).stop 2024-05-21T22:03:59.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.597+0000 7f24c958cb80 1 -- 172.21.15.151:0/3591492641 shutdown_connections 2024-05-21T22:03:59.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.597+0000 7f24c958cb80 1 --2- 172.21.15.151:0/3591492641 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f0bbb0be0 0x560f0bc2c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:59.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.597+0000 7f24c958cb80 1 --2- 172.21.15.151:0/3591492641 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f0bbaac90 0x560f0bbc66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:59.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.597+0000 7f24c958cb80 1 --2- 172.21.15.151:0/3591492641 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x560f0bc2e870 0x560f0bc30c90 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:59.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.597+0000 7f24c958cb80 1 --2- 172.21.15.151:0/3591492641 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x560f0bbb02c0 0x560f0bbc6bf0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:59.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.597+0000 7f24a2000700 0 client.27251 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T22:03:59.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.597+0000 7f24c958cb80 1 -- 172.21.15.151:0/3591492641 >> 172.21.15.151:0/3591492641 conn(0x560f0bad4be0 msgr2=0x560f0bbaddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:03:59.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.598+0000 7f24c958cb80 1 -- 172.21.15.151:0/3591492641 shutdown_connections 2024-05-21T22:03:59.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.598+0000 7f24c958cb80 1 -- 172.21.15.151:0/3591492641 wait complete. 2024-05-21T22:03:59.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.598+0000 7f24c958cb80 20 client.27251 trim_cache size 0 max 16384 2024-05-21T22:03:59.602 INFO:teuthology.orchestra.run.smithi151.stdout:[] 2024-05-21T22:03:59.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.598+0000 7f24bec00700 1 -- 172.21.15.151:0/2940140653 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f249c0b29f0 msgr2=0x7f249c0b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:59.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.598+0000 7f24bec00700 1 --2- 172.21.15.151:0/2940140653 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f249c0b29f0 0x7f249c0b4eb0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f24a800b2e0 tx=0x7f24a8005040 comp rx=0 tx=0).stop 2024-05-21T22:03:59.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.598+0000 7f24bec00700 1 -- 172.21.15.151:0/2940140653 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f24b806c470 msgr2=0x7f24b819fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:03:59.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.598+0000 7f24bec00700 1 --2- 172.21.15.151:0/2940140653 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f24b806c470 0x7f24b819fd80 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f24a400eb30 tx=0x7f24a400ee40 comp rx=0 tx=0).stop 2024-05-21T22:03:59.604 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.598+0000 7f24bec00700 1 -- 172.21.15.151:0/2940140653 shutdown_connections 2024-05-21T22:03:59.604 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.598+0000 7f24bec00700 1 --2- 172.21.15.151:0/2940140653 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f249c0b29f0 0x7f249c0b4eb0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:59.604 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.598+0000 7f24bec00700 1 --2- 172.21.15.151:0/2940140653 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f24b81095a0 0x7f24b8199a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:59.604 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.598+0000 7f24bec00700 1 --2- 172.21.15.151:0/2940140653 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f24b806bab0 0x7f24b819f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:59.604 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.598+0000 7f24bec00700 1 --2- 172.21.15.151:0/2940140653 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f24b806c470 0x7f24b819fd80 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:03:59.604 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.599+0000 7f24bec00700 1 -- 172.21.15.151:0/2940140653 >> 172.21.15.151:0/2940140653 conn(0x7f24b80793b0 msgr2=0x7f24b8101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:03:59.604 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.599+0000 7f24bec00700 1 -- 172.21.15.151:0/2940140653 shutdown_connections 2024-05-21T22:03:59.604 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:03:59.599+0000 7f24bec00700 1 -- 172.21.15.151:0/2940140653 wait complete. 2024-05-21T22:03:59.665 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:03:59 smithi151 ceph-mon[31623]: pgmap v258: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T22:03:59.735 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:03:59 smithi184 ceph-mon[36942]: pgmap v258: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T22:03:59.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:03:59 smithi164 ceph-mon[36269]: pgmap v258: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T22:04:01.541 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-21T22:04:01.541 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-21T22:04:01.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:04:01 smithi184 ceph-mon[36942]: pgmap v259: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:04:01.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:04:01 smithi164 ceph-mon[36269]: pgmap v259: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:04:01.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:04:01 smithi151 ceph-mon[31623]: pgmap v259: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:04:01.986 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:04:03.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:04:03 smithi184 ceph-mon[36942]: pgmap v260: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-05-21T22:04:03.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:04:03 smithi164 ceph-mon[36269]: pgmap v260: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-05-21T22:04:03.880 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:04:03 smithi151 ceph-mon[31623]: pgmap v260: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-05-21T22:04:04.056 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.055+0000 7fd052600700 1 -- 172.21.15.151:0/226572468 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd04c10a780 msgr2=0x7fd04c10e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:04:04.056 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.055+0000 7fd052600700 1 --2- 172.21.15.151:0/226572468 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd04c10a780 0x7fd04c10e7d0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fd0380099c0 tx=0x7fd038009cd0 comp rx=0 tx=0).stop 2024-05-21T22:04:04.056 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.055+0000 7fd052600700 1 -- 172.21.15.151:0/226572468 shutdown_connections 2024-05-21T22:04:04.056 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.055+0000 7fd052600700 1 --2- 172.21.15.151:0/226572468 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd04c10eeb0 0x7fd04c116a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:04.056 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.055+0000 7fd052600700 1 --2- 172.21.15.151:0/226572468 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd04c109dd0 0x7fd04c10a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:04.056 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.055+0000 7fd052600700 1 --2- 172.21.15.151:0/226572468 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd04c10a780 0x7fd04c10e7d0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:04.057 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.055+0000 7fd052600700 1 -- 172.21.15.151:0/226572468 >> 172.21.15.151:0/226572468 conn(0x7fd04c076d10 msgr2=0x7fd04c108360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:04:04.057 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.055+0000 7fd052600700 1 -- 172.21.15.151:0/226572468 shutdown_connections 2024-05-21T22:04:04.057 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.055+0000 7fd052600700 1 -- 172.21.15.151:0/226572468 wait complete. 2024-05-21T22:04:04.057 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.056+0000 7fd052600700 1 Processor -- start 2024-05-21T22:04:04.058 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.057+0000 7fd052600700 1 -- start start 2024-05-21T22:04:04.058 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.057+0000 7fd052600700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd04c109dd0 0x7fd04c1a0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:04.058 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.057+0000 7fd052600700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd04c10a780 0x7fd04c1a0850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:04.058 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.057+0000 7fd052600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd04c10eeb0 0x7fd04c1a4b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:04.058 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.057+0000 7fd052600700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd04c119da0 con 0x7fd04c10eeb0 2024-05-21T22:04:04.058 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.057+0000 7fd052600700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd04c119c20 con 0x7fd04c10a780 2024-05-21T22:04:04.058 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.057+0000 7fd052600700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd04c119f20 con 0x7fd04c109dd0 2024-05-21T22:04:04.058 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.057+0000 7fd04b400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd04c109dd0 0x7fd04c1a0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:04.058 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.057+0000 7fd04b400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd04c109dd0 0x7fd04c1a0310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:54348/0 (socket says 172.21.15.151:54348) 2024-05-21T22:04:04.058 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.057+0000 7fd04b400700 1 -- 172.21.15.151:0/895174031 learned_addr learned my addr 172.21.15.151:0/895174031 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:04:04.058 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.057+0000 7fd04aa00700 1 --2- 172.21.15.151:0/895174031 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd04c10a780 0x7fd04c1a0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:04.058 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.057+0000 7fd04be00700 1 --2- 172.21.15.151:0/895174031 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd04c10eeb0 0x7fd04c1a4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:04.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.058+0000 7fd04b400700 1 -- 172.21.15.151:0/895174031 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd04c10a780 msgr2=0x7fd04c1a0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:04:04.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.058+0000 7fd04b400700 1 --2- 172.21.15.151:0/895174031 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd04c10a780 0x7fd04c1a0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:04.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.058+0000 7fd04b400700 1 -- 172.21.15.151:0/895174031 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd04c10eeb0 msgr2=0x7fd04c1a4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:04:04.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.058+0000 7fd04b400700 1 --2- 172.21.15.151:0/895174031 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd04c10eeb0 0x7fd04c1a4b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:04.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.058+0000 7fd04b400700 1 -- 172.21.15.151:0/895174031 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd04c1a52d0 con 0x7fd04c109dd0 2024-05-21T22:04:04.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.058+0000 7fd04b400700 1 --2- 172.21.15.151:0/895174031 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd04c109dd0 0x7fd04c1a0310 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fd03c00e9c0 tx=0x7fd03c00ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:04:04.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.058+0000 7fd043e00700 1 -- 172.21.15.151:0/895174031 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd03c00cbd0 con 0x7fd04c109dd0 2024-05-21T22:04:04.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.058+0000 7fd052600700 1 -- 172.21.15.151:0/895174031 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd04c1a55c0 con 0x7fd04c109dd0 2024-05-21T22:04:04.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.058+0000 7fd043e00700 1 -- 172.21.15.151:0/895174031 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd03c00ee50 con 0x7fd04c109dd0 2024-05-21T22:04:04.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.059+0000 7fd043e00700 1 -- 172.21.15.151:0/895174031 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd03c005550 con 0x7fd04c109dd0 2024-05-21T22:04:04.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.059+0000 7fd052600700 1 -- 172.21.15.151:0/895174031 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd04c1ace40 con 0x7fd04c109dd0 2024-05-21T22:04:04.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.062+0000 7fd043e00700 1 -- 172.21.15.151:0/895174031 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7fd03c004750 con 0x7fd04c109dd0 2024-05-21T22:04:04.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.063+0000 7fd05ceb0b80 1 Processor -- start 2024-05-21T22:04:04.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.063+0000 7fd05ceb0b80 1 -- start start 2024-05-21T22:04:04.064 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.064+0000 7fd05ceb0b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559fe4201f30 0x559fe4202310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:04.064 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.064+0000 7fd05ceb0b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559fe4202850 0x559fe41fc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:04.064 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.064+0000 7fd05ceb0b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559fe41fc900 0x559fe41fcd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:04.064 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.064+0000 7fd05ceb0b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x559fe415aec0 con 0x559fe4202850 2024-05-21T22:04:04.064 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.064+0000 7fd05ceb0b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x559fe414aa60 con 0x559fe4201f30 2024-05-21T22:04:04.064 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.064+0000 7fd05ceb0b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x559fe414eaa0 con 0x559fe41fc900 2024-05-21T22:04:04.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.064+0000 7fd04aa00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559fe41fc900 0x559fe41fcd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:04.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.064+0000 7fd04b400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559fe4202850 0x559fe41fc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:04.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.064+0000 7fd04aa00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559fe41fc900 0x559fe41fcd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:54364/0 (socket says 172.21.15.151:54364) 2024-05-21T22:04:04.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.064+0000 7fd04aa00700 1 -- 172.21.15.151:0/3824785020 learned_addr learned my addr 172.21.15.151:0/3824785020 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:04:04.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.064+0000 7fd04be00700 1 --2- 172.21.15.151:0/3824785020 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559fe4201f30 0x559fe4202310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:04.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.065+0000 7fd043e00700 1 --2- 172.21.15.151:0/895174031 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fd0300b2a70 0x7fd0300b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:04.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.065+0000 7fd04aa00700 1 -- 172.21.15.151:0/3824785020 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559fe4201f30 msgr2=0x559fe4202310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:04:04.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.065+0000 7fd04aa00700 1 --2- 172.21.15.151:0/3824785020 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559fe4201f30 0x559fe4202310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:04.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.065+0000 7fd04aa00700 1 -- 172.21.15.151:0/3824785020 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559fe4202850 msgr2=0x559fe41fc2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:04:04.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.065+0000 7fd04aa00700 1 --2- 172.21.15.151:0/3824785020 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559fe4202850 0x559fe41fc2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:04.067 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.065+0000 7fd04aa00700 1 -- 172.21.15.151:0/3824785020 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fe41134a0 con 0x559fe41fc900 2024-05-21T22:04:04.067 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.065+0000 7fd043e00700 1 -- 172.21.15.151:0/895174031 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7fd03c0e28b0 con 0x7fd04c109dd0 2024-05-21T22:04:04.067 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.065+0000 7fd04aa00700 1 --2- 172.21.15.151:0/3824785020 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559fe41fc900 0x559fe41fcd60 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fd0380099c0 tx=0x7fd038003680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc7725f7f2e2b6ca server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:04:04.067 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.065+0000 7fd04be00700 1 --2- 172.21.15.151:0/895174031 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fd0300b2a70 0x7fd0300b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:04.067 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.065+0000 7fd040c00700 1 -- 172.21.15.151:0/3824785020 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd038028070 con 0x559fe41fc900 2024-05-21T22:04:04.067 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.065+0000 7fd040c00700 1 -- 172.21.15.151:0/3824785020 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd038009590 con 0x559fe41fc900 2024-05-21T22:04:04.067 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.065+0000 7fd040c00700 1 -- 172.21.15.151:0/3824785020 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd038003e60 con 0x559fe41fc900 2024-05-21T22:04:04.068 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.066+0000 7fd05ceb0b80 1 -- 172.21.15.151:0/3824785020 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559fe41fc900 msgr2=0x559fe41fcd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:04:04.068 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.066+0000 7fd04be00700 1 --2- 172.21.15.151:0/895174031 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fd0300b2a70 0x7fd0300b4f30 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd044009fd0 tx=0x7fd044009040 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:04:04.068 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.066+0000 7fd05ceb0b80 1 --2- 172.21.15.151:0/3824785020 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559fe41fc900 0x559fe41fcd60 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fd0380099c0 tx=0x7fd038003680 comp rx=0 tx=0).stop 2024-05-21T22:04:04.068 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.066+0000 7fd05ceb0b80 1 -- 172.21.15.151:0/3824785020 shutdown_connections 2024-05-21T22:04:04.068 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.066+0000 7fd05ceb0b80 1 --2- 172.21.15.151:0/3824785020 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559fe4201f30 0x559fe4202310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:04.068 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.066+0000 7fd05ceb0b80 1 --2- 172.21.15.151:0/3824785020 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559fe41fc900 0x559fe41fcd60 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:04.068 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.066+0000 7fd05ceb0b80 1 --2- 172.21.15.151:0/3824785020 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559fe4202850 0x559fe41fc2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:04.068 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.066+0000 7fd05ceb0b80 1 -- 172.21.15.151:0/3824785020 >> 172.21.15.151:0/3824785020 conn(0x559fe412ac60 msgr2=0x559fe41f7090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:04:04.068 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.067+0000 7fd05ceb0b80 1 -- 172.21.15.151:0/3824785020 shutdown_connections 2024-05-21T22:04:04.068 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.067+0000 7fd05ceb0b80 1 -- 172.21.15.151:0/3824785020 wait complete. 2024-05-21T22:04:04.069 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.067+0000 7fd05ceb0b80 1 -- start start 2024-05-21T22:04:04.069 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.068+0000 7fd05ceb0b80 1 Processor -- start 2024-05-21T22:04:04.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.068+0000 7fd05ceb0b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559fe41fc900 0x559fe42084f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:04.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.068+0000 7fd05ceb0b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559fe4201f30 0x559fe4205890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:04.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.068+0000 7fd05ceb0b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559fe4202850 0x559fe4205dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:04.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.068+0000 7fd05ceb0b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x559fe4051020 con 0x559fe4202850 2024-05-21T22:04:04.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.068+0000 7fd05ceb0b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x559fe40eb7e0 con 0x559fe41fc900 2024-05-21T22:04:04.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.068+0000 7fd05ceb0b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x559fe4209260 con 0x559fe4201f30 2024-05-21T22:04:04.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.068+0000 7fd05ceb0b80 10 client.0 _ll_register_callbacks cb 0x559fe40f4cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:04:04.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.068+0000 7fd04be00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559fe4202850 0x559fe4205dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:04.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.069+0000 7fd04be00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559fe4202850 0x559fe4205dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49938/0 (socket says 172.21.15.151:49938) 2024-05-21T22:04:04.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.069+0000 7fd04be00700 1 -- 172.21.15.151:0/2784368767 learned_addr learned my addr 172.21.15.151:0/2784368767 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:04:04.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.069+0000 7fd04b400700 1 --2- 172.21.15.151:0/2784368767 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559fe41fc900 0x559fe42084f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:04.072 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.069+0000 7fd04aa00700 1 --2- 172.21.15.151:0/2784368767 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559fe4201f30 0x559fe4205890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:04.072 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.069+0000 7fd04b400700 1 -- 172.21.15.151:0/2784368767 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559fe4201f30 msgr2=0x559fe4205890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:04:04.072 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.069+0000 7fd04b400700 1 --2- 172.21.15.151:0/2784368767 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559fe4201f30 0x559fe4205890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:04.072 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.069+0000 7fd04b400700 1 -- 172.21.15.151:0/2784368767 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559fe4202850 msgr2=0x559fe4205dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:04:04.072 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.069+0000 7fd04b400700 1 --2- 172.21.15.151:0/2784368767 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559fe4202850 0x559fe4205dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:04.072 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.069+0000 7fd04b400700 1 -- 172.21.15.151:0/2784368767 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fe409a950 con 0x559fe41fc900 2024-05-21T22:04:04.072 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.069+0000 7fd04b400700 1 --2- 172.21.15.151:0/2784368767 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559fe41fc900 0x559fe42084f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fd03c0dff90 tx=0x7fd03c009870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0d0208b5cd43181 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:04:04.072 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.069+0000 7fd035600700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-21T22:04:04.072 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.069+0000 7fd035600700 1 -- 172.21.15.151:0/2784368767 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd03c0a98f0 con 0x559fe41fc900 2024-05-21T22:04:04.072 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.070+0000 7fd035600700 1 -- 172.21.15.151:0/2784368767 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7fd03c0b2de0 con 0x559fe41fc900 2024-05-21T22:04:04.072 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.070+0000 7fd05ceb0b80 1 -- 172.21.15.151:0/2784368767 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd038009590 con 0x559fe41fc900 2024-05-21T22:04:04.072 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.070+0000 7fd035600700 1 -- 172.21.15.151:0/2784368767 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd03c0a9290 con 0x559fe41fc900 2024-05-21T22:04:04.072 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.070+0000 7fd05ceb0b80 10 client.27263 fetch_fsmap learned FSMap version 29 2024-05-21T22:04:04.073 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.070+0000 7fd05ceb0b80 1 -- 172.21.15.151:0/2784368767 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559fe411b0e0 con 0x559fe41fc900 2024-05-21T22:04:04.073 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.070+0000 7fd035600700 1 -- 172.21.15.151:0/2784368767 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd038009590 con 0x559fe41fc900 2024-05-21T22:04:04.073 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.070+0000 7fd035600700 1 -- 172.21.15.151:0/2784368767 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fd03c0b24f0 con 0x559fe41fc900 2024-05-21T22:04:04.073 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.070+0000 7fd05ceb0b80 10 client.27263 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:04:04.073 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.070+0000 7fd05ceb0b80 10 client.27263 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T22:04:04.073 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.070+0000 7fd05ceb0b80 20 client.27263 populate_metadata read hostname 'smithi151' 2024-05-21T22:04:04.073 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.070+0000 7fd05ceb0b80 1 --2- 172.21.15.151:0/2784368767 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x559fe42160e0 0x559fe42184d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:04.073 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.071+0000 7fd05ceb0b80 4 client.27263 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T22:04:04.073 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.071+0000 7fd05ceb0b80 1 -- 172.21.15.151:0/2784368767 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559fe415ce10 con 0x559fe42160e0 2024-05-21T22:04:04.073 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.071+0000 7fd04aa00700 1 --2- 172.21.15.151:0/2784368767 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x559fe42160e0 0x559fe42184d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:04.074 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.071+0000 7fd04aa00700 1 --2- 172.21.15.151:0/2784368767 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x559fe42160e0 0x559fe42184d0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a822768a3ce449 server_cookie=4d0ff1f0ef6b984f in_seq=0 out_seq=0 2024-05-21T22:04:04.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.071+0000 7fd035600700 10 client.27263 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T22:04:04.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.072+0000 7fd035600700 1 -- 172.21.15.151:0/2784368767 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x559fe415ce10 con 0x559fe42160e0 2024-05-21T22:04:04.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.072+0000 7fd035600700 10 client.27263 handle_command_reply: tid=0 2024-05-21T22:04:04.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.072+0000 7fd05ceb0b80 1 client.27263 shutdown 2024-05-21T22:04:04.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.073+0000 7fd05ceb0b80 1 -- 172.21.15.151:0/2784368767 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559fe41fc900 msgr2=0x559fe42084f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:04:04.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.073+0000 7fd05ceb0b80 1 --2- 172.21.15.151:0/2784368767 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559fe41fc900 0x559fe42084f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fd03c0dff90 tx=0x7fd03c009870 comp rx=0 tx=0).stop 2024-05-21T22:04:04.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.073+0000 7fd05ceb0b80 1 -- 172.21.15.151:0/2784368767 shutdown_connections 2024-05-21T22:04:04.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.073+0000 7fd05ceb0b80 1 --2- 172.21.15.151:0/2784368767 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x559fe42160e0 0x559fe42184d0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:04.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.073+0000 7fd05ceb0b80 1 --2- 172.21.15.151:0/2784368767 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559fe41fc900 0x559fe42084f0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:04.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.073+0000 7fd05ceb0b80 1 --2- 172.21.15.151:0/2784368767 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559fe4202850 0x559fe4205dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:04.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.073+0000 7fd05ceb0b80 1 --2- 172.21.15.151:0/2784368767 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559fe4201f30 0x559fe4205890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:04.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.073+0000 7fd035600700 0 client.27263 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T22:04:04.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.073+0000 7fd05ceb0b80 1 -- 172.21.15.151:0/2784368767 >> 172.21.15.151:0/2784368767 conn(0x559fe412ac60 msgr2=0x559fe41fb2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:04:04.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.073+0000 7fd05ceb0b80 1 -- 172.21.15.151:0/2784368767 shutdown_connections 2024-05-21T22:04:04.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.073+0000 7fd05ceb0b80 1 -- 172.21.15.151:0/2784368767 wait complete. 2024-05-21T22:04:04.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.073+0000 7fd05ceb0b80 20 client.27263 trim_cache size 0 max 16384 2024-05-21T22:04:04.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.096+0000 7fd05ceb0b80 1 Processor -- start 2024-05-21T22:04:04.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.096+0000 7fd05ceb0b80 1 -- start start 2024-05-21T22:04:04.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.096+0000 7fd05ceb0b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559fe41fc900 0x559fe41fc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:04.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.096+0000 7fd05ceb0b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559fe4201f30 0x559fe41f6a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:04.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.096+0000 7fd05ceb0b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559fe4202850 0x559fe41f6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:04.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.096+0000 7fd05ceb0b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd04c1b44d0 con 0x559fe41fc900 2024-05-21T22:04:04.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.096+0000 7fd05ceb0b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd04c1b6010 con 0x559fe4202850 2024-05-21T22:04:04.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.096+0000 7fd05ceb0b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd04c1b4ba0 con 0x559fe4201f30 2024-05-21T22:04:04.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.096+0000 7fd04b400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559fe41fc900 0x559fe41fc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:04.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.096+0000 7fd04aa00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559fe4201f30 0x559fe41f6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:04.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.096+0000 7fd04b400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559fe41fc900 0x559fe41fc2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49948/0 (socket says 172.21.15.151:49948) 2024-05-21T22:04:04.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.096+0000 7fd04b400700 1 -- 172.21.15.151:0/1027746061 learned_addr learned my addr 172.21.15.151:0/1027746061 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:04:04.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.096+0000 7fd04be00700 1 --2- 172.21.15.151:0/1027746061 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559fe4202850 0x559fe41f6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:04.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.097+0000 7fd04aa00700 1 -- 172.21.15.151:0/1027746061 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559fe4202850 msgr2=0x559fe41f6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:04:04.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.097+0000 7fd04aa00700 1 --2- 172.21.15.151:0/1027746061 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559fe4202850 0x559fe41f6fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:04.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.097+0000 7fd04aa00700 1 -- 172.21.15.151:0/1027746061 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559fe41fc900 msgr2=0x559fe41fc2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:04:04.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.097+0000 7fd04aa00700 1 --2- 172.21.15.151:0/1027746061 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559fe41fc900 0x559fe41fc2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:04.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.097+0000 7fd04aa00700 1 -- 172.21.15.151:0/1027746061 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fe4212500 con 0x559fe4201f30 2024-05-21T22:04:04.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.097+0000 7fd04aa00700 1 --2- 172.21.15.151:0/1027746061 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559fe4201f30 0x559fe41f6a60 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fd03800b580 tx=0x7fd038005fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=145bd5b3e0e73891 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:04:04.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.097+0000 7fd037e00700 1 -- 172.21.15.151:0/1027746061 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd038028040 con 0x559fe4201f30 2024-05-21T22:04:04.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.097+0000 7fd037e00700 1 -- 172.21.15.151:0/1027746061 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd038033070 con 0x559fe4201f30 2024-05-21T22:04:04.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.097+0000 7fd037e00700 1 -- 172.21.15.151:0/1027746061 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd03800bcb0 con 0x559fe4201f30 2024-05-21T22:04:04.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.098+0000 7fd05ceb0b80 1 -- 172.21.15.151:0/1027746061 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559fe4201f30 msgr2=0x559fe41f6a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:04:04.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.098+0000 7fd05ceb0b80 1 --2- 172.21.15.151:0/1027746061 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559fe4201f30 0x559fe41f6a60 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fd03800b580 tx=0x7fd038005fb0 comp rx=0 tx=0).stop 2024-05-21T22:04:04.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.098+0000 7fd05ceb0b80 1 -- 172.21.15.151:0/1027746061 shutdown_connections 2024-05-21T22:04:04.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.098+0000 7fd05ceb0b80 1 --2- 172.21.15.151:0/1027746061 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559fe4202850 0x559fe41f6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:04.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.098+0000 7fd05ceb0b80 1 --2- 172.21.15.151:0/1027746061 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559fe41fc900 0x559fe41fc2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:04.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.098+0000 7fd05ceb0b80 1 --2- 172.21.15.151:0/1027746061 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559fe4201f30 0x559fe41f6a60 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:04.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.098+0000 7fd05ceb0b80 1 -- 172.21.15.151:0/1027746061 >> 172.21.15.151:0/1027746061 conn(0x559fe412ac60 msgr2=0x559fe42041a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:04:04.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.098+0000 7fd05ceb0b80 1 -- 172.21.15.151:0/1027746061 shutdown_connections 2024-05-21T22:04:04.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.098+0000 7fd05ceb0b80 1 -- 172.21.15.151:0/1027746061 wait complete. 2024-05-21T22:04:04.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.099+0000 7fd05ceb0b80 1 -- start start 2024-05-21T22:04:04.102 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.099+0000 7fd05ceb0b80 1 Processor -- start 2024-05-21T22:04:04.102 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.099+0000 7fd05ceb0b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559fe41fc900 0x559fe421b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:04.102 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.100+0000 7fd05ceb0b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559fe4201f30 0x559fe421bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:04.102 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.100+0000 7fd05ceb0b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559fe4202850 0x559fe4219a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:04.102 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.100+0000 7fd05ceb0b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd04c1b5940 con 0x559fe4201f30 2024-05-21T22:04:04.102 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.100+0000 7fd05ceb0b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd04c1b5270 con 0x559fe4202850 2024-05-21T22:04:04.102 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.100+0000 7fd05ceb0b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd04c1b2490 con 0x559fe41fc900 2024-05-21T22:04:04.102 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.100+0000 7fd05ceb0b80 10 client.0 _ll_register_callbacks cb 0x559fe40f4cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:04:04.102 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.100+0000 7fd04be00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559fe4202850 0x559fe4219a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:04.102 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.100+0000 7fd04aa00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559fe4201f30 0x559fe421bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:04.102 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.100+0000 7fd04be00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559fe4202850 0x559fe4219a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:36030/0 (socket says 172.21.15.151:36030) 2024-05-21T22:04:04.102 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.100+0000 7fd04be00700 1 -- 172.21.15.151:0/2100468291 learned_addr learned my addr 172.21.15.151:0/2100468291 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:04:04.102 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.100+0000 7fd04b400700 1 --2- 172.21.15.151:0/2100468291 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559fe41fc900 0x559fe421b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:04.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.100+0000 7fd04be00700 1 -- 172.21.15.151:0/2100468291 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559fe41fc900 msgr2=0x559fe421b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:04:04.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.100+0000 7fd04be00700 1 --2- 172.21.15.151:0/2100468291 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559fe41fc900 0x559fe421b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:04.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.100+0000 7fd04be00700 1 -- 172.21.15.151:0/2100468291 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559fe4201f30 msgr2=0x559fe421bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:04:04.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.100+0000 7fd04be00700 1 --2- 172.21.15.151:0/2100468291 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559fe4201f30 0x559fe421bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:04.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.100+0000 7fd04be00700 1 -- 172.21.15.151:0/2100468291 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fe421a290 con 0x559fe4202850 2024-05-21T22:04:04.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.101+0000 7fd04be00700 1 --2- 172.21.15.151:0/2100468291 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559fe4202850 0x559fe4219a70 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fd044017e90 tx=0x7fd044004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4acaf194f84a6763 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:04:04.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.101+0000 7fd035600700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-21T22:04:04.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.101+0000 7fd035600700 1 -- 172.21.15.151:0/2100468291 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd044024d70 con 0x559fe4202850 2024-05-21T22:04:04.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.101+0000 7fd035600700 1 -- 172.21.15.151:0/2100468291 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7fd044003a00 con 0x559fe4202850 2024-05-21T22:04:04.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.101+0000 7fd05ceb0b80 1 -- 172.21.15.151:0/2100468291 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd038033070 con 0x559fe4202850 2024-05-21T22:04:04.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.101+0000 7fd035600700 1 -- 172.21.15.151:0/2100468291 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd044003c00 con 0x559fe4202850 2024-05-21T22:04:04.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.101+0000 7fd05ceb0b80 10 client.27269 fetch_fsmap learned FSMap version 29 2024-05-21T22:04:04.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.101+0000 7fd05ceb0b80 1 -- 172.21.15.151:0/2100468291 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559fe4281500 con 0x559fe4202850 2024-05-21T22:04:04.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.102+0000 7fd035600700 1 -- 172.21.15.151:0/2100468291 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd038033070 con 0x559fe4202850 2024-05-21T22:04:04.105 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.102+0000 7fd035600700 1 -- 172.21.15.151:0/2100468291 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fd04402b030 con 0x559fe4202850 2024-05-21T22:04:04.105 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.102+0000 7fd05ceb0b80 10 client.27269 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:04:04.105 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.102+0000 7fd05ceb0b80 10 client.27269 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T22:04:04.105 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.102+0000 7fd05ceb0b80 20 client.27269 populate_metadata read hostname 'smithi151' 2024-05-21T22:04:04.105 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.102+0000 7fd05ceb0b80 1 --2- 172.21.15.151:0/2100468291 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x559fe4281c10 0x559fe4284030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:04.105 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.102+0000 7fd05ceb0b80 4 client.27269 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-21T22:04:04.105 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.102+0000 7fd05ceb0b80 1 -- 172.21.15.151:0/2100468291 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559fe41565f0 con 0x559fe4281c10 2024-05-21T22:04:04.105 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.103+0000 7fd04b400700 1 --2- 172.21.15.151:0/2100468291 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x559fe4281c10 0x559fe4284030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:04.105 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.103+0000 7fd04b400700 1 --2- 172.21.15.151:0/2100468291 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x559fe4281c10 0x559fe4284030 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93e8aaf3d0bea9cb server_cookie=9802e548ec2d0992 in_seq=0 out_seq=0 2024-05-21T22:04:04.105 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.103+0000 7fd035600700 10 client.27269 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T22:04:04.105 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.104+0000 7fd035600700 1 -- 172.21.15.151:0/2100468291 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559fe41565f0 con 0x559fe4281c10 2024-05-21T22:04:04.105 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.104+0000 7fd035600700 10 client.27269 handle_command_reply: tid=0 2024-05-21T22:04:04.105 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.104+0000 7fd05ceb0b80 1 client.27269 shutdown 2024-05-21T22:04:04.105 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.104+0000 7fd05ceb0b80 1 -- 172.21.15.151:0/2100468291 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559fe4202850 msgr2=0x559fe4219a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:04:04.106 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.104+0000 7fd05ceb0b80 1 --2- 172.21.15.151:0/2100468291 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559fe4202850 0x559fe4219a70 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fd044017e90 tx=0x7fd044004ad0 comp rx=0 tx=0).stop 2024-05-21T22:04:04.106 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.105+0000 7fd05ceb0b80 1 -- 172.21.15.151:0/2100468291 shutdown_connections 2024-05-21T22:04:04.106 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.105+0000 7fd05ceb0b80 1 --2- 172.21.15.151:0/2100468291 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x559fe4281c10 0x559fe4284030 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:04.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.105+0000 7fd05ceb0b80 1 --2- 172.21.15.151:0/2100468291 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559fe4202850 0x559fe4219a70 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:04.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.105+0000 7fd05ceb0b80 1 --2- 172.21.15.151:0/2100468291 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559fe41fc900 0x559fe421b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:04.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.105+0000 7fd05ceb0b80 1 --2- 172.21.15.151:0/2100468291 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559fe4201f30 0x559fe421bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:04.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.105+0000 7fd05ceb0b80 1 -- 172.21.15.151:0/2100468291 >> 172.21.15.151:0/2100468291 conn(0x559fe412ac60 msgr2=0x559fe41f6a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:04:04.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.105+0000 7fd035600700 0 client.27269 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T22:04:04.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.105+0000 7fd05ceb0b80 1 -- 172.21.15.151:0/2100468291 shutdown_connections 2024-05-21T22:04:04.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.105+0000 7fd05ceb0b80 1 -- 172.21.15.151:0/2100468291 wait complete. 2024-05-21T22:04:04.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.105+0000 7fd05ceb0b80 20 client.27269 trim_cache size 0 max 16384 2024-05-21T22:04:04.107 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-05-21T22:04:04.107 INFO:teuthology.orchestra.run.smithi151.stdout: "return_code": 0, 2024-05-21T22:04:04.108 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_tag": "54b77533-77c0-4e4b-b681-a7e0f714d71a", 2024-05-21T22:04:04.108 INFO:teuthology.orchestra.run.smithi151.stdout: "mode": "asynchronous" 2024-05-21T22:04:04.108 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-05-21T22:04:04.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.106+0000 7fd052600700 1 -- 172.21.15.151:0/895174031 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fd0300b2a70 msgr2=0x7fd0300b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:04:04.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.106+0000 7fd052600700 1 --2- 172.21.15.151:0/895174031 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fd0300b2a70 0x7fd0300b4f30 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd044009fd0 tx=0x7fd044009040 comp rx=0 tx=0).stop 2024-05-21T22:04:04.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.106+0000 7fd052600700 1 -- 172.21.15.151:0/895174031 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd04c109dd0 msgr2=0x7fd04c1a0310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:04:04.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.106+0000 7fd052600700 1 --2- 172.21.15.151:0/895174031 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd04c109dd0 0x7fd04c1a0310 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fd03c00e9c0 tx=0x7fd03c00ecd0 comp rx=0 tx=0).stop 2024-05-21T22:04:04.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.106+0000 7fd052600700 1 -- 172.21.15.151:0/895174031 shutdown_connections 2024-05-21T22:04:04.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.106+0000 7fd052600700 1 --2- 172.21.15.151:0/895174031 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fd0300b2a70 0x7fd0300b4f30 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:04.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.106+0000 7fd052600700 1 --2- 172.21.15.151:0/895174031 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd04c10eeb0 0x7fd04c1a4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:04.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.106+0000 7fd052600700 1 --2- 172.21.15.151:0/895174031 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd04c109dd0 0x7fd04c1a0310 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:04.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.106+0000 7fd052600700 1 --2- 172.21.15.151:0/895174031 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd04c10a780 0x7fd04c1a0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:04.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.106+0000 7fd052600700 1 -- 172.21.15.151:0/895174031 >> 172.21.15.151:0/895174031 conn(0x7fd04c076d10 msgr2=0x7fd04c110080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:04:04.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.106+0000 7fd052600700 1 -- 172.21.15.151:0/895174031 shutdown_connections 2024-05-21T22:04:04.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:04.106+0000 7fd052600700 1 -- 172.21.15.151:0/895174031 wait complete. 2024-05-21T22:04:05.013 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-21T22:04:05.444 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:04:05.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:04:05 smithi164 ceph-mon[36269]: scrub queued for path: / 2024-05-21T22:04:05.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:04:05 smithi164 ceph-mon[36269]: scrub summary: idle+waiting paths [/] 2024-05-21T22:04:05.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:04:05 smithi164 ceph-mon[36269]: scrub summary: active paths [/] 2024-05-21T22:04:05.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:04:05 smithi164 ceph-mon[36269]: scrub summary: idle+waiting paths [/] 2024-05-21T22:04:05.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:04:05 smithi164 ceph-mon[36269]: pgmap v261: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-05-21T22:04:05.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:04:05 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T22:04:05.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:04:05 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T22:04:05.787 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:04:05 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T22:04:05.813 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:04:05 smithi151 ceph-mon[31623]: scrub queued for path: / 2024-05-21T22:04:05.813 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:04:05 smithi151 ceph-mon[31623]: scrub summary: idle+waiting paths [/] 2024-05-21T22:04:05.813 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:04:05 smithi151 ceph-mon[31623]: scrub summary: active paths [/] 2024-05-21T22:04:05.813 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:04:05 smithi151 ceph-mon[31623]: scrub summary: idle+waiting paths [/] 2024-05-21T22:04:05.813 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:04:05 smithi151 ceph-mon[31623]: pgmap v261: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-05-21T22:04:05.813 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:04:05 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T22:04:05.813 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:04:05 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T22:04:05.813 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:04:05 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T22:04:05.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:04:05 smithi184 ceph-mon[36942]: scrub queued for path: / 2024-05-21T22:04:05.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:04:05 smithi184 ceph-mon[36942]: scrub summary: idle+waiting paths [/] 2024-05-21T22:04:05.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:04:05 smithi184 ceph-mon[36942]: scrub summary: active paths [/] 2024-05-21T22:04:05.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:04:05 smithi184 ceph-mon[36942]: scrub summary: idle+waiting paths [/] 2024-05-21T22:04:05.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:04:05 smithi184 ceph-mon[36942]: pgmap v261: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-05-21T22:04:05.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:04:05 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T22:04:05.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:04:05 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T22:04:05.987 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:04:05 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T22:04:07.612 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.610+0000 7f8e77e00700 1 -- 172.21.15.151:0/474053367 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e78106010 msgr2=0x7f8e78112520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:04:07.612 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.610+0000 7f8e77e00700 1 --2- 172.21.15.151:0/474053367 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e78106010 0x7f8e78112520 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f8e7000b0a0 tx=0x7f8e7000b3b0 comp rx=0 tx=0).stop 2024-05-21T22:04:07.612 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.610+0000 7f8e77e00700 1 -- 172.21.15.151:0/474053367 shutdown_connections 2024-05-21T22:04:07.612 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.610+0000 7f8e77e00700 1 --2- 172.21.15.151:0/474053367 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e78106010 0x7f8e78112520 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:07.612 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.611+0000 7f8e77e00700 1 --2- 172.21.15.151:0/474053367 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8e7810b650 0x7f8e7810ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:07.612 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.611+0000 7f8e77e00700 1 --2- 172.21.15.151:0/474053367 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8e78105650 0x7f8e78105ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:07.613 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.611+0000 7f8e77e00700 1 -- 172.21.15.151:0/474053367 >> 172.21.15.151:0/474053367 conn(0x7f8e78101120 msgr2=0x7f8e78103540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:04:07.613 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.611+0000 7f8e77e00700 1 -- 172.21.15.151:0/474053367 shutdown_connections 2024-05-21T22:04:07.613 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.611+0000 7f8e77e00700 1 -- 172.21.15.151:0/474053367 wait complete. 2024-05-21T22:04:07.613 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.612+0000 7f8e77e00700 1 Processor -- start 2024-05-21T22:04:07.613 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.612+0000 7f8e77e00700 1 -- start start 2024-05-21T22:04:07.614 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.612+0000 7f8e77e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8e78105650 0x7f8e781a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:07.614 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.612+0000 7f8e77e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e78106010 0x7f8e781a1e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:07.614 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.612+0000 7f8e77e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8e7810b650 0x7f8e7819ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:07.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.612+0000 7f8e77e00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8e78115890 con 0x7f8e78106010 2024-05-21T22:04:07.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.612+0000 7f8e77e00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8e78115710 con 0x7f8e78105650 2024-05-21T22:04:07.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.612+0000 7f8e77e00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8e78115a10 con 0x7f8e7810b650 2024-05-21T22:04:07.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.613+0000 7f8e76000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e78106010 0x7f8e781a1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:07.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.613+0000 7f8e76000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e78106010 0x7f8e781a1e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:44946/0 (socket says 172.21.15.151:44946) 2024-05-21T22:04:07.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.613+0000 7f8e76000700 1 -- 172.21.15.151:0/1188894442 learned_addr learned my addr 172.21.15.151:0/1188894442 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:04:07.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.613+0000 7f8e76a00700 1 --2- 172.21.15.151:0/1188894442 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8e78105650 0x7f8e781a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:07.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.613+0000 7f8e77400700 1 --2- 172.21.15.151:0/1188894442 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8e7810b650 0x7f8e7819ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:07.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.613+0000 7f8e76a00700 1 -- 172.21.15.151:0/1188894442 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8e7810b650 msgr2=0x7f8e7819ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:04:07.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.613+0000 7f8e76a00700 1 --2- 172.21.15.151:0/1188894442 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8e7810b650 0x7f8e7819ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:07.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.613+0000 7f8e76a00700 1 -- 172.21.15.151:0/1188894442 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e78106010 msgr2=0x7f8e781a1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:04:07.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.613+0000 7f8e76a00700 1 --2- 172.21.15.151:0/1188894442 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e78106010 0x7f8e781a1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:07.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.613+0000 7f8e76a00700 1 -- 172.21.15.151:0/1188894442 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e7819c1f0 con 0x7f8e78105650 2024-05-21T22:04:07.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.613+0000 7f8e76a00700 1 --2- 172.21.15.151:0/1188894442 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8e78105650 0x7f8e781a18d0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f8e6800e9f0 tx=0x7f8e6800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:04:07.616 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.614+0000 7f8e6f400700 1 -- 172.21.15.151:0/1188894442 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e6800c9c0 con 0x7f8e78105650 2024-05-21T22:04:07.616 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.614+0000 7f8e77e00700 1 -- 172.21.15.151:0/1188894442 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e7819c4e0 con 0x7f8e78105650 2024-05-21T22:04:07.616 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.614+0000 7f8e6f400700 1 -- 172.21.15.151:0/1188894442 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e6800cb60 con 0x7f8e78105650 2024-05-21T22:04:07.616 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.614+0000 7f8e6f400700 1 -- 172.21.15.151:0/1188894442 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e68018680 con 0x7f8e78105650 2024-05-21T22:04:07.616 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.614+0000 7f8e77e00700 1 -- 172.21.15.151:0/1188894442 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e781a87a0 con 0x7f8e78105650 2024-05-21T22:04:07.617 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.617+0000 7f8e6f400700 1 -- 172.21.15.151:0/1188894442 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7f8e68010430 con 0x7f8e78105650 2024-05-21T22:04:07.622 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.618+0000 7f8e88071b80 1 Processor -- start 2024-05-21T22:04:07.622 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.619+0000 7f8e88071b80 1 -- start start 2024-05-21T22:04:07.622 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.619+0000 7f8e6f400700 1 --2- 172.21.15.151:0/1188894442 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f8e5c0b29f0 0x7f8e5c0b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:07.626 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.619+0000 7f8e88071b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5563cd47c020 0x5563cd47c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:07.626 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.619+0000 7f8e88071b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5563cd47c940 0x5563cd4764b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:07.626 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.619+0000 7f8e6f400700 1 -- 172.21.15.151:0/1188894442 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7f8e680e2770 con 0x7f8e78105650 2024-05-21T22:04:07.626 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.619+0000 7f8e88071b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5563cd476ad0 0x5563cd476f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:07.626 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.619+0000 7f8e88071b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5563cd3d4e40 con 0x5563cd476ad0 2024-05-21T22:04:07.626 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.619+0000 7f8e88071b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5563cd3c49e0 con 0x5563cd47c020 2024-05-21T22:04:07.626 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.619+0000 7f8e88071b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5563cd3c8a20 con 0x5563cd47c940 2024-05-21T22:04:07.626 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.619+0000 7f8e77400700 1 --2- 172.21.15.151:0/1188894442 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f8e5c0b29f0 0x7f8e5c0b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:07.626 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.620+0000 7f8e76a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5563cd47c020 0x5563cd47c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:07.626 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.620+0000 7f8e76a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5563cd47c020 0x5563cd47c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:36096/0 (socket says 172.21.15.151:36096) 2024-05-21T22:04:07.626 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.620+0000 7f8e76a00700 1 -- 172.21.15.151:0/690459154 learned_addr learned my addr 172.21.15.151:0/690459154 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:04:07.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.620+0000 7f8e77400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5563cd476ad0 0x5563cd476f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:07.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.620+0000 7f8e76000700 1 --2- 172.21.15.151:0/690459154 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5563cd47c940 0x5563cd4764b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:07.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.620+0000 7f8e76a00700 1 -- 172.21.15.151:0/690459154 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5563cd47c940 msgr2=0x5563cd4764b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:04:07.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.620+0000 7f8e76a00700 1 --2- 172.21.15.151:0/690459154 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5563cd47c940 0x5563cd4764b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:07.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.620+0000 7f8e76a00700 1 -- 172.21.15.151:0/690459154 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5563cd476ad0 msgr2=0x5563cd476f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:04:07.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.620+0000 7f8e76a00700 1 --2- 172.21.15.151:0/690459154 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5563cd476ad0 0x5563cd476f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:07.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.620+0000 7f8e76a00700 1 -- 172.21.15.151:0/690459154 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563cd38d420 con 0x5563cd47c020 2024-05-21T22:04:07.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.620+0000 7f8e77400700 1 --2- 172.21.15.151:0/690459154 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5563cd476ad0 0x5563cd476f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T22:04:07.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.620+0000 7f8e76a00700 1 --2- 172.21.15.151:0/690459154 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5563cd47c020 0x5563cd47c400 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f8e68009230 tx=0x7f8e680a9570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d161cb7f116a36f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:04:07.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.621+0000 7f8e77400700 1 --2- 172.21.15.151:0/1188894442 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f8e5c0b29f0 0x7f8e5c0b4eb0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8e70009e60 tx=0x7f8e70006000 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:04:07.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.621+0000 7f8e63e00700 1 -- 172.21.15.151:0/690459154 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e680a99e0 con 0x5563cd47c020 2024-05-21T22:04:07.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.621+0000 7f8e63e00700 1 -- 172.21.15.151:0/690459154 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e680add90 con 0x5563cd47c020 2024-05-21T22:04:07.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.621+0000 7f8e63e00700 1 -- 172.21.15.151:0/690459154 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e680b23b0 con 0x5563cd47c020 2024-05-21T22:04:07.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.621+0000 7f8e88071b80 1 -- 172.21.15.151:0/690459154 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5563cd47c020 msgr2=0x5563cd47c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:04:07.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.621+0000 7f8e88071b80 1 --2- 172.21.15.151:0/690459154 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5563cd47c020 0x5563cd47c400 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f8e68009230 tx=0x7f8e680a9570 comp rx=0 tx=0).stop 2024-05-21T22:04:07.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.621+0000 7f8e88071b80 1 -- 172.21.15.151:0/690459154 shutdown_connections 2024-05-21T22:04:07.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.621+0000 7f8e88071b80 1 --2- 172.21.15.151:0/690459154 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5563cd47c020 0x5563cd47c400 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:07.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.621+0000 7f8e88071b80 1 --2- 172.21.15.151:0/690459154 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5563cd476ad0 0x5563cd476f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:07.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.621+0000 7f8e88071b80 1 --2- 172.21.15.151:0/690459154 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5563cd47c940 0x5563cd4764b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:07.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.621+0000 7f8e88071b80 1 -- 172.21.15.151:0/690459154 >> 172.21.15.151:0/690459154 conn(0x5563cd3a4be0 msgr2=0x5563cd471150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:04:07.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.622+0000 7f8e88071b80 1 -- 172.21.15.151:0/690459154 shutdown_connections 2024-05-21T22:04:07.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.622+0000 7f8e88071b80 1 -- 172.21.15.151:0/690459154 wait complete. 2024-05-21T22:04:07.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.622+0000 7f8e88071b80 1 -- start start 2024-05-21T22:04:07.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.622+0000 7f8e88071b80 1 Processor -- start 2024-05-21T22:04:07.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.623+0000 7f8e88071b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5563cd476ad0 0x5563cd4826b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:07.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.623+0000 7f8e88071b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5563cd47c020 0x5563cd47fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:07.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.623+0000 7f8e88071b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5563cd47c940 0x5563cd47ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:07.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.623+0000 7f8e88071b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5563cd2cafa0 con 0x5563cd476ad0 2024-05-21T22:04:07.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.623+0000 7f8e88071b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5563cd365760 con 0x5563cd47c020 2024-05-21T22:04:07.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.623+0000 7f8e88071b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5563cd4833a0 con 0x5563cd47c940 2024-05-21T22:04:07.630 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.623+0000 7f8e88071b80 10 client.0 _ll_register_callbacks cb 0x5563cd36ec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:04:07.630 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.623+0000 7f8e76a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5563cd476ad0 0x5563cd4826b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:07.630 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.623+0000 7f8e76000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5563cd47c020 0x5563cd47fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:07.630 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.623+0000 7f8e76000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5563cd47c020 0x5563cd47fa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:36106/0 (socket says 172.21.15.151:36106) 2024-05-21T22:04:07.630 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.623+0000 7f8e76000700 1 -- 172.21.15.151:0/4267153372 learned_addr learned my addr 172.21.15.151:0/4267153372 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:04:07.630 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.623+0000 7f8e77400700 1 --2- 172.21.15.151:0/4267153372 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5563cd47c940 0x5563cd47ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:07.630 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.624+0000 7f8e76000700 1 -- 172.21.15.151:0/4267153372 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5563cd47c940 msgr2=0x5563cd47ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:04:07.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.624+0000 7f8e76000700 1 --2- 172.21.15.151:0/4267153372 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5563cd47c940 0x5563cd47ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:07.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.624+0000 7f8e76000700 1 -- 172.21.15.151:0/4267153372 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5563cd476ad0 msgr2=0x5563cd4826b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:04:07.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.624+0000 7f8e76000700 1 --2- 172.21.15.151:0/4267153372 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5563cd476ad0 0x5563cd4826b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:07.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.624+0000 7f8e76000700 1 -- 172.21.15.151:0/4267153372 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563cd3148d0 con 0x5563cd47c020 2024-05-21T22:04:07.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.624+0000 7f8e76000700 1 --2- 172.21.15.151:0/4267153372 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5563cd47c020 0x5563cd47fa50 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f8e64012840 tx=0x7f8e64012b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=576bb665e37a4bc1 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:04:07.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.624+0000 7f8e60c00700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-21T22:04:07.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.624+0000 7f8e60c00700 1 -- 172.21.15.151:0/4267153372 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e64004050 con 0x5563cd47c020 2024-05-21T22:04:07.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.624+0000 7f8e60c00700 1 -- 172.21.15.151:0/4267153372 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7f8e640041f0 con 0x5563cd47c020 2024-05-21T22:04:07.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.624+0000 7f8e88071b80 1 -- 172.21.15.151:0/4267153372 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e680add90 con 0x5563cd47c020 2024-05-21T22:04:07.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.625+0000 7f8e60c00700 1 -- 172.21.15.151:0/4267153372 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8e640043f0 con 0x5563cd47c020 2024-05-21T22:04:07.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.625+0000 7f8e88071b80 10 client.27281 fetch_fsmap learned FSMap version 29 2024-05-21T22:04:07.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.625+0000 7f8e88071b80 1 -- 172.21.15.151:0/4267153372 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5563cd395060 con 0x5563cd47c020 2024-05-21T22:04:07.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.625+0000 7f8e60c00700 1 -- 172.21.15.151:0/4267153372 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e680add90 con 0x5563cd47c020 2024-05-21T22:04:07.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.625+0000 7f8e60c00700 1 -- 172.21.15.151:0/4267153372 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f8e6400f7b0 con 0x5563cd47c020 2024-05-21T22:04:07.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.625+0000 7f8e88071b80 10 client.27281 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:04:07.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.625+0000 7f8e88071b80 10 client.27281 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T22:04:07.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.625+0000 7f8e88071b80 20 client.27281 populate_metadata read hostname 'smithi151' 2024-05-21T22:04:07.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.625+0000 7f8e88071b80 1 --2- 172.21.15.151:0/4267153372 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5563cd48ff60 0x5563cd492350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:07.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.625+0000 7f8e88071b80 4 client.27281 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T22:04:07.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.625+0000 7f8e88071b80 1 -- 172.21.15.151:0/4267153372 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5563cd3d6d90 con 0x5563cd48ff60 2024-05-21T22:04:07.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.626+0000 7f8e76a00700 1 --2- 172.21.15.151:0/4267153372 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5563cd48ff60 0x5563cd492350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:07.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.626+0000 7f8e76a00700 1 --2- 172.21.15.151:0/4267153372 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5563cd48ff60 0x5563cd492350 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fea602148f11306a server_cookie=190f913b35d1c90f in_seq=0 out_seq=0 2024-05-21T22:04:07.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.626+0000 7f8e60c00700 10 client.27281 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T22:04:07.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.627+0000 7f8e60c00700 1 -- 172.21.15.151:0/4267153372 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5563cd3d6d90 con 0x5563cd48ff60 2024-05-21T22:04:07.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.627+0000 7f8e60c00700 10 client.27281 handle_command_reply: tid=0 2024-05-21T22:04:07.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.627+0000 7f8e88071b80 1 client.27281 shutdown 2024-05-21T22:04:07.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.628+0000 7f8e88071b80 1 -- 172.21.15.151:0/4267153372 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5563cd47c020 msgr2=0x5563cd47fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:04:07.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.628+0000 7f8e88071b80 1 --2- 172.21.15.151:0/4267153372 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5563cd47c020 0x5563cd47fa50 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f8e64012840 tx=0x7f8e64012b50 comp rx=0 tx=0).stop 2024-05-21T22:04:07.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.628+0000 7f8e88071b80 1 -- 172.21.15.151:0/4267153372 shutdown_connections 2024-05-21T22:04:07.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.628+0000 7f8e88071b80 1 --2- 172.21.15.151:0/4267153372 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5563cd47c940 0x5563cd47ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:07.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.628+0000 7f8e88071b80 1 --2- 172.21.15.151:0/4267153372 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5563cd476ad0 0x5563cd4826b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:07.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.628+0000 7f8e88071b80 1 --2- 172.21.15.151:0/4267153372 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5563cd48ff60 0x5563cd492350 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:07.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.628+0000 7f8e88071b80 1 --2- 172.21.15.151:0/4267153372 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5563cd47c020 0x5563cd47fa50 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:07.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.628+0000 7f8e88071b80 1 -- 172.21.15.151:0/4267153372 >> 172.21.15.151:0/4267153372 conn(0x5563cd3a4be0 msgr2=0x5563cd4754a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:04:07.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.628+0000 7f8e60c00700 0 client.27281 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T22:04:07.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.628+0000 7f8e88071b80 1 -- 172.21.15.151:0/4267153372 shutdown_connections 2024-05-21T22:04:07.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.628+0000 7f8e88071b80 1 -- 172.21.15.151:0/4267153372 wait complete. 2024-05-21T22:04:07.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.628+0000 7f8e88071b80 20 client.27281 trim_cache size 0 max 16384 2024-05-21T22:04:07.651 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.651+0000 7f8e88071b80 1 Processor -- start 2024-05-21T22:04:07.651 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.651+0000 7f8e88071b80 1 -- start start 2024-05-21T22:04:07.651 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.651+0000 7f8e88071b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5563cd476ad0 0x5563cd4764b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:07.652 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.651+0000 7f8e88071b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5563cd47c020 0x5563cd470c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:07.652 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.651+0000 7f8e88071b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5563cd47c940 0x5563cd471170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:07.652 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.651+0000 7f8e88071b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8e781aff00 con 0x5563cd47c020 2024-05-21T22:04:07.652 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.651+0000 7f8e88071b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8e781b1a40 con 0x5563cd47c940 2024-05-21T22:04:07.652 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.651+0000 7f8e88071b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8e781b05d0 con 0x5563cd476ad0 2024-05-21T22:04:07.653 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.652+0000 7f8e76000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5563cd47c020 0x5563cd470c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:07.653 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.652+0000 7f8e76000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5563cd47c020 0x5563cd470c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:44966/0 (socket says 172.21.15.151:44966) 2024-05-21T22:04:07.653 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.652+0000 7f8e76000700 1 -- 172.21.15.151:0/34704920 learned_addr learned my addr 172.21.15.151:0/34704920 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:04:07.653 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.652+0000 7f8e76a00700 1 --2- 172.21.15.151:0/34704920 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5563cd476ad0 0x5563cd4764b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:07.653 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.652+0000 7f8e77400700 1 --2- 172.21.15.151:0/34704920 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5563cd47c940 0x5563cd471170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:07.654 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.652+0000 7f8e76a00700 1 -- 172.21.15.151:0/34704920 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5563cd47c940 msgr2=0x5563cd471170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:04:07.654 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.652+0000 7f8e76a00700 1 --2- 172.21.15.151:0/34704920 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5563cd47c940 0x5563cd471170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:07.654 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.652+0000 7f8e76a00700 1 -- 172.21.15.151:0/34704920 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5563cd47c020 msgr2=0x5563cd470c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:04:07.654 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.652+0000 7f8e76a00700 1 --2- 172.21.15.151:0/34704920 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5563cd47c020 0x5563cd470c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:07.654 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.652+0000 7f8e76a00700 1 -- 172.21.15.151:0/34704920 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563cd482a20 con 0x5563cd476ad0 2024-05-21T22:04:07.654 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.652+0000 7f8e76000700 1 --2- 172.21.15.151:0/34704920 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5563cd47c020 0x5563cd470c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:04:07.654 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.652+0000 7f8e76a00700 1 --2- 172.21.15.151:0/34704920 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5563cd476ad0 0x5563cd4764b0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f8e6800e9c0 tx=0x7f8e680b2250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f45cb70f947da5c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:04:07.654 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.653+0000 7f8e63400700 1 -- 172.21.15.151:0/34704920 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e6808f070 con 0x5563cd476ad0 2024-05-21T22:04:07.654 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.653+0000 7f8e63400700 1 -- 172.21.15.151:0/34704920 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e680add90 con 0x5563cd476ad0 2024-05-21T22:04:07.654 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.653+0000 7f8e63400700 1 -- 172.21.15.151:0/34704920 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e680b25f0 con 0x5563cd476ad0 2024-05-21T22:04:07.656 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.653+0000 7f8e88071b80 1 -- 172.21.15.151:0/34704920 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5563cd476ad0 msgr2=0x5563cd4764b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:04:07.656 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.653+0000 7f8e88071b80 1 --2- 172.21.15.151:0/34704920 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5563cd476ad0 0x5563cd4764b0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f8e6800e9c0 tx=0x7f8e680b2250 comp rx=0 tx=0).stop 2024-05-21T22:04:07.656 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.654+0000 7f8e88071b80 1 -- 172.21.15.151:0/34704920 shutdown_connections 2024-05-21T22:04:07.656 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.654+0000 7f8e88071b80 1 --2- 172.21.15.151:0/34704920 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5563cd47c940 0x5563cd471170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:07.656 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.654+0000 7f8e88071b80 1 --2- 172.21.15.151:0/34704920 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5563cd476ad0 0x5563cd4764b0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:07.656 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.654+0000 7f8e88071b80 1 --2- 172.21.15.151:0/34704920 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5563cd47c020 0x5563cd470c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:07.656 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.654+0000 7f8e88071b80 1 -- 172.21.15.151:0/34704920 >> 172.21.15.151:0/34704920 conn(0x5563cd3a4be0 msgr2=0x5563cd47e380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:04:07.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.654+0000 7f8e88071b80 1 -- 172.21.15.151:0/34704920 shutdown_connections 2024-05-21T22:04:07.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.654+0000 7f8e88071b80 1 -- 172.21.15.151:0/34704920 wait complete. 2024-05-21T22:04:07.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.654+0000 7f8e88071b80 1 -- start start 2024-05-21T22:04:07.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.654+0000 7f8e88071b80 1 Processor -- start 2024-05-21T22:04:07.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.655+0000 7f8e88071b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5563cd476ad0 0x5563cd488630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:07.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.655+0000 7f8e88071b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5563cd47c020 0x5563cd488b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:07.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.655+0000 7f8e88071b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5563cd47c940 0x5563cd485600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:07.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.655+0000 7f8e88071b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8e781b1370 con 0x5563cd476ad0 2024-05-21T22:04:07.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.655+0000 7f8e88071b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8e781b0ca0 con 0x5563cd47c020 2024-05-21T22:04:07.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.655+0000 7f8e88071b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8e781adec0 con 0x5563cd47c940 2024-05-21T22:04:07.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.655+0000 7f8e88071b80 10 client.0 _ll_register_callbacks cb 0x5563cd36ec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:04:07.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.655+0000 7f8e76a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5563cd476ad0 0x5563cd488630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:07.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.655+0000 7f8e76a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5563cd476ad0 0x5563cd488630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:44972/0 (socket says 172.21.15.151:44972) 2024-05-21T22:04:07.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.655+0000 7f8e76a00700 1 -- 172.21.15.151:0/1460027623 learned_addr learned my addr 172.21.15.151:0/1460027623 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:04:07.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.655+0000 7f8e77400700 1 --2- 172.21.15.151:0/1460027623 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5563cd47c940 0x5563cd485600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:07.658 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.655+0000 7f8e76000700 1 --2- 172.21.15.151:0/1460027623 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5563cd47c020 0x5563cd488b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:07.658 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.656+0000 7f8e76a00700 1 -- 172.21.15.151:0/1460027623 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5563cd47c940 msgr2=0x5563cd485600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:04:07.658 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.656+0000 7f8e76a00700 1 --2- 172.21.15.151:0/1460027623 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5563cd47c940 0x5563cd485600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:07.658 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.656+0000 7f8e76a00700 1 -- 172.21.15.151:0/1460027623 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5563cd47c020 msgr2=0x5563cd488b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:04:07.658 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.656+0000 7f8e76a00700 1 --2- 172.21.15.151:0/1460027623 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5563cd47c020 0x5563cd488b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:07.658 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.656+0000 7f8e76a00700 1 -- 172.21.15.151:0/1460027623 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563cd489240 con 0x5563cd476ad0 2024-05-21T22:04:07.658 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.656+0000 7f8e76000700 1 --2- 172.21.15.151:0/1460027623 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5563cd47c020 0x5563cd488b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:04:07.660 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.656+0000 7f8e76a00700 1 --2- 172.21.15.151:0/1460027623 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5563cd476ad0 0x5563cd488630 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f8e68009230 tx=0x7f8e680a9830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34cab15e7b5b5a44 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:04:07.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.656+0000 7f8e60c00700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-21T22:04:07.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.656+0000 7f8e60c00700 1 -- 172.21.15.151:0/1460027623 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e680c4060 con 0x5563cd476ad0 2024-05-21T22:04:07.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.656+0000 7f8e60c00700 1 -- 172.21.15.151:0/1460027623 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8542+0+0 (secure 0 0 0) 0x7f8e680cc2b0 con 0x5563cd476ad0 2024-05-21T22:04:07.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.656+0000 7f8e88071b80 1 -- 172.21.15.151:0/1460027623 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e680add90 con 0x5563cd476ad0 2024-05-21T22:04:07.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.657+0000 7f8e60c00700 1 -- 172.21.15.151:0/1460027623 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8e6808f070 con 0x5563cd476ad0 2024-05-21T22:04:07.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.657+0000 7f8e88071b80 10 client.17163 fetch_fsmap learned FSMap version 29 2024-05-21T22:04:07.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.657+0000 7f8e88071b80 1 -- 172.21.15.151:0/1460027623 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5563cd503970 con 0x5563cd476ad0 2024-05-21T22:04:07.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.657+0000 7f8e60c00700 1 -- 172.21.15.151:0/1460027623 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e680add90 con 0x5563cd476ad0 2024-05-21T22:04:07.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.657+0000 7f8e60c00700 1 -- 172.21.15.151:0/1460027623 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f8e68010720 con 0x5563cd476ad0 2024-05-21T22:04:07.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.657+0000 7f8e88071b80 10 client.17163 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:04:07.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.657+0000 7f8e88071b80 10 client.17163 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T22:04:07.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.657+0000 7f8e88071b80 20 client.17163 populate_metadata read hostname 'smithi151' 2024-05-21T22:04:07.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.657+0000 7f8e88071b80 1 --2- 172.21.15.151:0/1460027623 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5563cd503b40 0x5563cd505f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:07.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.657+0000 7f8e88071b80 4 client.17163 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-21T22:04:07.662 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.657+0000 7f8e88071b80 1 -- 172.21.15.151:0/1460027623 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5563cd3d0570 con 0x5563cd503b40 2024-05-21T22:04:07.662 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.658+0000 7f8e76000700 1 --2- 172.21.15.151:0/1460027623 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5563cd503b40 0x5563cd505f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:07.662 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.658+0000 7f8e76000700 1 --2- 172.21.15.151:0/1460027623 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5563cd503b40 0x5563cd505f30 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be27d4d6597019bf server_cookie=189a0ee172260c92 in_seq=0 out_seq=0 2024-05-21T22:04:07.662 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.658+0000 7f8e60c00700 10 client.17163 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T22:04:07.662 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.658+0000 7f8e60c00700 1 -- 172.21.15.151:0/1460027623 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5563cd3d0570 con 0x5563cd503b40 2024-05-21T22:04:07.662 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.658+0000 7f8e60c00700 10 client.17163 handle_command_reply: tid=0 2024-05-21T22:04:07.662 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.658+0000 7f8e88071b80 1 client.17163 shutdown 2024-05-21T22:04:07.662 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.659+0000 7f8e88071b80 1 -- 172.21.15.151:0/1460027623 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5563cd476ad0 msgr2=0x5563cd488630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:04:07.662 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.659+0000 7f8e88071b80 1 --2- 172.21.15.151:0/1460027623 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5563cd476ad0 0x5563cd488630 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f8e68009230 tx=0x7f8e680a9830 comp rx=0 tx=0).stop 2024-05-21T22:04:07.662 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.659+0000 7f8e88071b80 1 -- 172.21.15.151:0/1460027623 shutdown_connections 2024-05-21T22:04:07.662 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.659+0000 7f8e88071b80 1 --2- 172.21.15.151:0/1460027623 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5563cd47c940 0x5563cd485600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:07.662 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.659+0000 7f8e88071b80 1 --2- 172.21.15.151:0/1460027623 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5563cd476ad0 0x5563cd488630 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:07.662 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.659+0000 7f8e88071b80 1 --2- 172.21.15.151:0/1460027623 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5563cd503b40 0x5563cd505f30 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:07.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.659+0000 7f8e88071b80 1 --2- 172.21.15.151:0/1460027623 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5563cd47c020 0x5563cd488b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:07.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.659+0000 7f8e88071b80 1 -- 172.21.15.151:0/1460027623 >> 172.21.15.151:0/1460027623 conn(0x5563cd3a4be0 msgr2=0x5563cd470c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:04:07.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.659+0000 7f8e60c00700 0 client.17163 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T22:04:07.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.659+0000 7f8e88071b80 1 -- 172.21.15.151:0/1460027623 shutdown_connections 2024-05-21T22:04:07.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.659+0000 7f8e88071b80 1 -- 172.21.15.151:0/1460027623 wait complete. 2024-05-21T22:04:07.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.659+0000 7f8e88071b80 20 client.17163 trim_cache size 0 max 16384 2024-05-21T22:04:07.663 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-05-21T22:04:07.663 INFO:teuthology.orchestra.run.smithi151.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-21T22:04:07.664 INFO:teuthology.orchestra.run.smithi151.stdout: "scrubs": { 2024-05-21T22:04:07.664 INFO:teuthology.orchestra.run.smithi151.stdout: "54b77533-77c0-4e4b-b681-a7e0f714d71a": { 2024-05-21T22:04:07.664 INFO:teuthology.orchestra.run.smithi151.stdout: "path": "/", 2024-05-21T22:04:07.664 INFO:teuthology.orchestra.run.smithi151.stdout: "tag": "54b77533-77c0-4e4b-b681-a7e0f714d71a", 2024-05-21T22:04:07.664 INFO:teuthology.orchestra.run.smithi151.stdout: "options": "recursive,force" 2024-05-21T22:04:07.664 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:04:07.664 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:04:07.664 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-05-21T22:04:07.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.660+0000 7f8e77e00700 1 -- 172.21.15.151:0/1188894442 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f8e5c0b29f0 msgr2=0x7f8e5c0b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:04:07.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.660+0000 7f8e77e00700 1 --2- 172.21.15.151:0/1188894442 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f8e5c0b29f0 0x7f8e5c0b4eb0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8e70009e60 tx=0x7f8e70006000 comp rx=0 tx=0).stop 2024-05-21T22:04:07.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.660+0000 7f8e77e00700 1 -- 172.21.15.151:0/1188894442 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8e78105650 msgr2=0x7f8e781a18d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:04:07.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.660+0000 7f8e77e00700 1 --2- 172.21.15.151:0/1188894442 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8e78105650 0x7f8e781a18d0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f8e6800e9f0 tx=0x7f8e6800ed00 comp rx=0 tx=0).stop 2024-05-21T22:04:07.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.660+0000 7f8e77e00700 1 -- 172.21.15.151:0/1188894442 shutdown_connections 2024-05-21T22:04:07.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.660+0000 7f8e77e00700 1 --2- 172.21.15.151:0/1188894442 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f8e5c0b29f0 0x7f8e5c0b4eb0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:07.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.660+0000 7f8e77e00700 1 --2- 172.21.15.151:0/1188894442 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8e78105650 0x7f8e781a18d0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:07.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.660+0000 7f8e77e00700 1 --2- 172.21.15.151:0/1188894442 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8e7810b650 0x7f8e7819ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:07.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.660+0000 7f8e77e00700 1 --2- 172.21.15.151:0/1188894442 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e78106010 0x7f8e781a1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:07.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.660+0000 7f8e77e00700 1 -- 172.21.15.151:0/1188894442 >> 172.21.15.151:0/1188894442 conn(0x7f8e78101120 msgr2=0x7f8e78101660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:04:07.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.660+0000 7f8e77e00700 1 -- 172.21.15.151:0/1188894442 shutdown_connections 2024-05-21T22:04:07.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:07.660+0000 7f8e77e00700 1 -- 172.21.15.151:0/1188894442 wait complete. 2024-05-21T22:04:07.702 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:04:07 smithi151 ceph-mon[31623]: pgmap v262: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-05-21T22:04:07.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:04:07 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T22:04:07.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:04:07 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T22:04:07.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:04:07 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T22:04:07.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:04:07 smithi164 ceph-mon[36269]: pgmap v262: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-05-21T22:04:07.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:04:07 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T22:04:07.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:04:07 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T22:04:07.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:04:07 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T22:04:07.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:04:07 smithi184 ceph-mon[36942]: pgmap v262: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-05-21T22:04:07.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:04:07 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T22:04:07.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:04:07 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T22:04:07.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:04:07 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T22:04:08.512 INFO:tasks.cephfs.filesystem:scrub status for tag:54b77533-77c0-4e4b-b681-a7e0f714d71a - {'path': '/', 'tag': '54b77533-77c0-4e4b-b681-a7e0f714d71a', 'options': 'recursive,force'} 2024-05-21T22:04:09.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:04:09 smithi164 ceph-mon[36269]: pgmap v263: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-05-21T22:04:09.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:04:09 smithi164 ceph-mon[36269]: scrub summary: idle 2024-05-21T22:04:09.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:04:09 smithi151 ceph-mon[31623]: pgmap v263: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-05-21T22:04:09.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:04:09 smithi151 ceph-mon[31623]: scrub summary: idle 2024-05-21T22:04:09.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:04:09 smithi184 ceph-mon[36942]: pgmap v263: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-05-21T22:04:09.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:04:09 smithi184 ceph-mon[36942]: scrub summary: idle 2024-05-21T22:04:10.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:04:10 smithi164 ceph-mon[36269]: osdmap e88: 12 total, 12 up, 12 in 2024-05-21T22:04:10.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:04:10 smithi151 ceph-mon[31623]: osdmap e88: 12 total, 12 up, 12 in 2024-05-21T22:04:10.985 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:04:10 smithi184 ceph-mon[36942]: osdmap e88: 12 total, 12 up, 12 in 2024-05-21T22:04:11.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:04:11 smithi164 ceph-mon[36269]: pgmap v265: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-05-21T22:04:11.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:04:11 smithi151 ceph-mon[31623]: pgmap v265: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-05-21T22:04:11.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:04:11 smithi184 ceph-mon[36942]: pgmap v265: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-05-21T22:04:13.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:04:13 smithi164 ceph-mon[36269]: pgmap v266: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-21T22:04:13.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:04:13 smithi151 ceph-mon[31623]: pgmap v266: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-21T22:04:13.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:04:13 smithi184 ceph-mon[36942]: pgmap v266: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-21T22:04:15.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:04:15 smithi151 ceph-mon[31623]: osdmap e89: 12 total, 12 up, 12 in 2024-05-21T22:04:15.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:04:15 smithi151 ceph-mon[31623]: pgmap v268: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 255 B/s wr, 13 op/s 2024-05-21T22:04:15.485 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:04:15 smithi184 ceph-mon[36942]: osdmap e89: 12 total, 12 up, 12 in 2024-05-21T22:04:15.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:04:15 smithi184 ceph-mon[36942]: pgmap v268: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 255 B/s wr, 13 op/s 2024-05-21T22:04:15.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:04:15 smithi164 ceph-mon[36269]: osdmap e89: 12 total, 12 up, 12 in 2024-05-21T22:04:15.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:04:15 smithi164 ceph-mon[36269]: pgmap v268: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 255 B/s wr, 13 op/s 2024-05-21T22:04:17.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:04:17 smithi184 ceph-mon[36942]: pgmap v269: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-05-21T22:04:17.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:04:17 smithi164 ceph-mon[36269]: pgmap v269: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-05-21T22:04:17.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:04:17 smithi151 ceph-mon[31623]: pgmap v269: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-05-21T22:04:19.658 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:04:19 smithi151 ceph-mon[31623]: pgmap v270: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 229 B/s wr, 11 op/s 2024-05-21T22:04:19.735 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:04:19 smithi184 ceph-mon[36942]: pgmap v270: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 229 B/s wr, 11 op/s 2024-05-21T22:04:19.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:04:19 smithi164 ceph-mon[36269]: pgmap v270: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 229 B/s wr, 11 op/s 2024-05-21T22:04:21.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:04:21 smithi164 ceph-mon[36269]: pgmap v271: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-21T22:04:21.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:04:21 smithi151 ceph-mon[31623]: pgmap v271: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-21T22:04:21.985 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:04:21 smithi184 ceph-mon[36942]: pgmap v271: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-21T22:04:23.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:04:23 smithi164 ceph-mon[36269]: pgmap v272: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-05-21T22:04:23.788 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:04:23 smithi184 ceph-mon[36942]: pgmap v272: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-05-21T22:04:23.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:04:23 smithi151 ceph-mon[31623]: pgmap v272: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-05-21T22:04:25.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:04:25 smithi164 ceph-mon[36269]: pgmap v273: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 99 B/s wr, 9 op/s 2024-05-21T22:04:25.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:04:25 smithi151 ceph-mon[31623]: pgmap v273: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 99 B/s wr, 9 op/s 2024-05-21T22:04:25.985 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:04:25 smithi184 ceph-mon[36942]: pgmap v273: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 99 B/s wr, 9 op/s 2024-05-21T22:04:27.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:04:27 smithi164 ceph-mon[36269]: pgmap v274: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-05-21T22:04:27.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:04:27 smithi151 ceph-mon[31623]: pgmap v274: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-05-21T22:04:27.985 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:04:27 smithi184 ceph-mon[36942]: pgmap v274: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-05-21T22:04:29.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:04:29 smithi164 ceph-mon[36269]: pgmap v275: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-05-21T22:04:29.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:04:29 smithi151 ceph-mon[31623]: pgmap v275: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-05-21T22:04:29.985 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:04:29 smithi184 ceph-mon[36942]: pgmap v275: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-05-21T22:04:31.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:04:31 smithi164 ceph-mon[36269]: pgmap v276: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:04:31.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:04:31 smithi151 ceph-mon[31623]: pgmap v276: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:04:31.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:04:31 smithi184 ceph-mon[36942]: pgmap v276: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:04:33.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:04:33 smithi164 ceph-mon[36269]: pgmap v277: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T22:04:33.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:04:33 smithi151 ceph-mon[31623]: pgmap v277: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T22:04:33.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:04:33 smithi184 ceph-mon[36942]: pgmap v277: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T22:04:35.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:04:35 smithi164 ceph-mon[36269]: pgmap v278: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:04:35.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:04:35 smithi151 ceph-mon[31623]: pgmap v278: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:04:35.985 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:04:35 smithi184 ceph-mon[36942]: pgmap v278: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:04:37.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:04:37 smithi164 ceph-mon[36269]: pgmap v279: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:04:37.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:04:37 smithi151 ceph-mon[31623]: pgmap v279: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:04:37.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:04:37 smithi184 ceph-mon[36942]: pgmap v279: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:04:38.513 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-21T22:04:38.976 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:04:39.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:04:39 smithi164 ceph-mon[36269]: pgmap v280: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T22:04:39.902 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:04:39 smithi151 ceph-mon[31623]: pgmap v280: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T22:04:39.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:04:39 smithi184 ceph-mon[36942]: pgmap v280: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T22:04:41.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.182+0000 7fafcea00700 1 -- 172.21.15.151:0/1459293306 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fafc810b780 msgr2=0x7fafc810bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:04:41.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.182+0000 7fafcea00700 1 --2- 172.21.15.151:0/1459293306 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fafc810b780 0x7fafc810bb60 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fafb8009990 tx=0x7fafb8009ca0 comp rx=0 tx=0).stop 2024-05-21T22:04:41.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.182+0000 7fafcea00700 1 -- 172.21.15.151:0/1459293306 shutdown_connections 2024-05-21T22:04:41.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.182+0000 7fafcea00700 1 --2- 172.21.15.151:0/1459293306 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fafc8106140 0x7fafc8112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:41.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.182+0000 7fafcea00700 1 --2- 172.21.15.151:0/1459293306 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fafc810b780 0x7fafc810bb60 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:41.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.182+0000 7fafcea00700 1 --2- 172.21.15.151:0/1459293306 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fafc8105780 0x7fafc8105c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:41.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.182+0000 7fafcea00700 1 -- 172.21.15.151:0/1459293306 >> 172.21.15.151:0/1459293306 conn(0x7fafc8101270 msgr2=0x7fafc8103690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:04:41.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.183+0000 7fafcea00700 1 -- 172.21.15.151:0/1459293306 shutdown_connections 2024-05-21T22:04:41.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.183+0000 7fafcea00700 1 -- 172.21.15.151:0/1459293306 wait complete. 2024-05-21T22:04:41.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.184+0000 7fafcea00700 1 Processor -- start 2024-05-21T22:04:41.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.184+0000 7fafcea00700 1 -- start start 2024-05-21T22:04:41.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.184+0000 7fafcea00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fafc8105780 0x7fafc819be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:41.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.184+0000 7fafcea00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fafc8106140 0x7fafc819c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:41.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.184+0000 7fafcea00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fafc810b780 0x7fafc81a0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:41.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.184+0000 7fafcea00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fafc81159d0 con 0x7fafc8105780 2024-05-21T22:04:41.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.184+0000 7fafcea00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fafc8115850 con 0x7fafc8106140 2024-05-21T22:04:41.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.184+0000 7fafcea00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fafc8115b50 con 0x7fafc810b780 2024-05-21T22:04:41.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.185+0000 7fafc7e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fafc8105780 0x7fafc819be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:41.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.185+0000 7fafc7e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fafc8105780 0x7fafc819be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52642/0 (socket says 172.21.15.151:52642) 2024-05-21T22:04:41.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.185+0000 7fafc7e00700 1 -- 172.21.15.151:0/693863525 learned_addr learned my addr 172.21.15.151:0/693863525 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:04:41.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.185+0000 7fafcca00700 1 --2- 172.21.15.151:0/693863525 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fafc810b780 0x7fafc81a0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:41.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.185+0000 7fafc7400700 1 --2- 172.21.15.151:0/693863525 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fafc8106140 0x7fafc819c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:41.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.185+0000 7fafc7400700 1 -- 172.21.15.151:0/693863525 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fafc810b780 msgr2=0x7fafc81a0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:04:41.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.185+0000 7fafc7400700 1 --2- 172.21.15.151:0/693863525 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fafc810b780 0x7fafc81a0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:41.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.185+0000 7fafc7400700 1 -- 172.21.15.151:0/693863525 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fafc8105780 msgr2=0x7fafc819be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:04:41.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.185+0000 7fafc7400700 1 --2- 172.21.15.151:0/693863525 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fafc8105780 0x7fafc819be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:41.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.185+0000 7fafc7400700 1 -- 172.21.15.151:0/693863525 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafc81a0e90 con 0x7fafc8106140 2024-05-21T22:04:41.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.185+0000 7fafc7400700 1 --2- 172.21.15.151:0/693863525 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fafc8106140 0x7fafc819c3c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fafb400e9f0 tx=0x7fafb400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:04:41.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.186+0000 7fafc4c00700 1 -- 172.21.15.151:0/693863525 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb4004220 con 0x7fafc8106140 2024-05-21T22:04:41.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.186+0000 7fafcea00700 1 -- 172.21.15.151:0/693863525 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafc81a1180 con 0x7fafc8106140 2024-05-21T22:04:41.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.186+0000 7fafc4c00700 1 -- 172.21.15.151:0/693863525 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fafb40043c0 con 0x7fafc8106140 2024-05-21T22:04:41.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.186+0000 7fafc4c00700 1 -- 172.21.15.151:0/693863525 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb4005020 con 0x7fafc8106140 2024-05-21T22:04:41.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.186+0000 7fafcea00700 1 -- 172.21.15.151:0/693863525 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafc81a8a00 con 0x7fafc8106140 2024-05-21T22:04:41.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.190+0000 7fafc4c00700 1 -- 172.21.15.151:0/693863525 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7fafb4004750 con 0x7fafc8106140 2024-05-21T22:04:41.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.190+0000 7fafd93deb80 1 Processor -- start 2024-05-21T22:04:41.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.191+0000 7fafd93deb80 1 -- start start 2024-05-21T22:04:41.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.191+0000 7fafd93deb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cf47c4610 0x562cf47c49f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:41.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.191+0000 7fafd93deb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cf47c4f30 0x562cf47beb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:41.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.191+0000 7fafd93deb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562cf47bf140 0x562cf47bf5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:41.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.191+0000 7fafd93deb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x562cf471ce40 con 0x562cf47c4f30 2024-05-21T22:04:41.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.191+0000 7fafd93deb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x562cf470c9e0 con 0x562cf47c4610 2024-05-21T22:04:41.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.191+0000 7fafd93deb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x562cf4710a20 con 0x562cf47bf140 2024-05-21T22:04:41.193 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.191+0000 7fafc7e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cf47c4f30 0x562cf47beb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:41.193 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.191+0000 7fafc7e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cf47c4f30 0x562cf47beb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52652/0 (socket says 172.21.15.151:52652) 2024-05-21T22:04:41.193 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.191+0000 7fafc7e00700 1 -- 172.21.15.151:0/2185484089 learned_addr learned my addr 172.21.15.151:0/2185484089 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:04:41.193 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.191+0000 7fafcca00700 1 --2- 172.21.15.151:0/2185484089 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cf47c4610 0x562cf47c49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:41.193 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.192+0000 7fafc7400700 1 --2- 172.21.15.151:0/2185484089 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562cf47bf140 0x562cf47bf5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:41.193 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.192+0000 7fafc7e00700 1 -- 172.21.15.151:0/2185484089 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562cf47bf140 msgr2=0x562cf47bf5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:04:41.193 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.192+0000 7fafc7e00700 1 --2- 172.21.15.151:0/2185484089 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562cf47bf140 0x562cf47bf5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:41.193 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.192+0000 7fafc7e00700 1 -- 172.21.15.151:0/2185484089 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cf47c4610 msgr2=0x562cf47c49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:04:41.193 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.192+0000 7fafc7e00700 1 --2- 172.21.15.151:0/2185484089 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cf47c4610 0x562cf47c49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:41.193 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.192+0000 7fafc7e00700 1 -- 172.21.15.151:0/2185484089 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562cf46d5420 con 0x562cf47c4f30 2024-05-21T22:04:41.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.192+0000 7fafc7e00700 1 --2- 172.21.15.151:0/2185484089 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cf47c4f30 0x562cf47beb20 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fafb80096d0 tx=0x7fafb8009700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=695502bff2fcbfca server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:04:41.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.192+0000 7fafc4c00700 1 --2- 172.21.15.151:0/693863525 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fafac0b29f0 0x7fafac0b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:41.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.193+0000 7fafbd600700 1 -- 172.21.15.151:0/2185484089 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb8027070 con 0x562cf47c4f30 2024-05-21T22:04:41.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.193+0000 7fafbd600700 1 -- 172.21.15.151:0/2185484089 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fafb80056d0 con 0x562cf47c4f30 2024-05-21T22:04:41.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.193+0000 7fafbd600700 1 -- 172.21.15.151:0/2185484089 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb8003bb0 con 0x562cf47c4f30 2024-05-21T22:04:41.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.193+0000 7fafc4c00700 1 -- 172.21.15.151:0/693863525 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7fafb40e2610 con 0x7fafc8106140 2024-05-21T22:04:41.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.193+0000 7fafcca00700 1 --2- 172.21.15.151:0/693863525 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fafac0b29f0 0x7fafac0b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:41.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.194+0000 7fafd93deb80 1 -- 172.21.15.151:0/2185484089 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cf47c4f30 msgr2=0x562cf47beb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:04:41.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.194+0000 7fafd93deb80 1 --2- 172.21.15.151:0/2185484089 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cf47c4f30 0x562cf47beb20 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fafb80096d0 tx=0x7fafb8009700 comp rx=0 tx=0).stop 2024-05-21T22:04:41.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.194+0000 7fafcca00700 1 --2- 172.21.15.151:0/693863525 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fafac0b29f0 0x7fafac0b4eb0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fafc00127f0 tx=0x7fafc000b040 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:04:41.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.194+0000 7fafd93deb80 1 -- 172.21.15.151:0/2185484089 shutdown_connections 2024-05-21T22:04:41.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.194+0000 7fafd93deb80 1 --2- 172.21.15.151:0/2185484089 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cf47c4610 0x562cf47c49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:41.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.194+0000 7fafd93deb80 1 --2- 172.21.15.151:0/2185484089 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562cf47bf140 0x562cf47bf5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:41.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.194+0000 7fafd93deb80 1 --2- 172.21.15.151:0/2185484089 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cf47c4f30 0x562cf47beb20 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:41.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.194+0000 7fafd93deb80 1 -- 172.21.15.151:0/2185484089 >> 172.21.15.151:0/2185484089 conn(0x562cf46ecbe0 msgr2=0x562cf47b9690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:04:41.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.194+0000 7fafd93deb80 1 -- 172.21.15.151:0/2185484089 shutdown_connections 2024-05-21T22:04:41.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.194+0000 7fafd93deb80 1 -- 172.21.15.151:0/2185484089 wait complete. 2024-05-21T22:04:41.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.195+0000 7fafd93deb80 1 -- start start 2024-05-21T22:04:41.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.195+0000 7fafd93deb80 1 Processor -- start 2024-05-21T22:04:41.198 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.195+0000 7fafd93deb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cf47bf140 0x562cf47cabf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:41.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.195+0000 7fafd93deb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562cf47c4610 0x562cf47c7f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:41.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.195+0000 7fafd93deb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cf47c4f30 0x562cf47c84d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:41.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.196+0000 7fafd93deb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x562cf4612fa0 con 0x562cf47c4f30 2024-05-21T22:04:41.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.196+0000 7fafd93deb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x562cf46ad760 con 0x562cf47bf140 2024-05-21T22:04:41.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.196+0000 7fafd93deb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x562cf47cb980 con 0x562cf47c4610 2024-05-21T22:04:41.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.196+0000 7fafd93deb80 10 client.0 _ll_register_callbacks cb 0x562cf46b6c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:04:41.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.196+0000 7fafc7e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cf47bf140 0x562cf47cabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:41.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.196+0000 7fafcca00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cf47c4f30 0x562cf47c84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:41.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.196+0000 7fafc7e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cf47bf140 0x562cf47cabf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:58384/0 (socket says 172.21.15.151:58384) 2024-05-21T22:04:41.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.196+0000 7fafc7e00700 1 -- 172.21.15.151:0/1427884335 learned_addr learned my addr 172.21.15.151:0/1427884335 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:04:41.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.196+0000 7fafc7400700 1 --2- 172.21.15.151:0/1427884335 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562cf47c4610 0x562cf47c7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:41.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.196+0000 7fafc7e00700 1 -- 172.21.15.151:0/1427884335 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562cf47c4610 msgr2=0x562cf47c7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:04:41.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.196+0000 7fafc7e00700 1 --2- 172.21.15.151:0/1427884335 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562cf47c4610 0x562cf47c7f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:41.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.196+0000 7fafc7e00700 1 -- 172.21.15.151:0/1427884335 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cf47c4f30 msgr2=0x562cf47c84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:04:41.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.196+0000 7fafc7e00700 1 --2- 172.21.15.151:0/1427884335 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cf47c4f30 0x562cf47c84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:41.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.196+0000 7fafc7e00700 1 -- 172.21.15.151:0/1427884335 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562cf465c8d0 con 0x562cf47bf140 2024-05-21T22:04:41.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.197+0000 7fafc7e00700 1 --2- 172.21.15.151:0/1427884335 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cf47bf140 0x562cf47cabf0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fafb800bd10 tx=0x7fafb800bd40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc63058d607a75f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:04:41.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.197+0000 7fafb2000700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-21T22:04:41.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.197+0000 7fafb2000700 1 -- 172.21.15.151:0/1427884335 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb8032060 con 0x562cf47bf140 2024-05-21T22:04:41.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.197+0000 7fafb2000700 1 -- 172.21.15.151:0/1427884335 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7fafb80039a0 con 0x562cf47bf140 2024-05-21T22:04:41.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.197+0000 7fafd93deb80 1 -- 172.21.15.151:0/1427884335 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fafb80056d0 con 0x562cf47bf140 2024-05-21T22:04:41.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.197+0000 7fafb2000700 1 -- 172.21.15.151:0/1427884335 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fafb8027070 con 0x562cf47bf140 2024-05-21T22:04:41.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.197+0000 7fafd93deb80 10 client.27299 fetch_fsmap learned FSMap version 29 2024-05-21T22:04:41.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.197+0000 7fafd93deb80 1 -- 172.21.15.151:0/1427884335 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562cf46dd060 con 0x562cf47bf140 2024-05-21T22:04:41.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.198+0000 7fafb2000700 1 -- 172.21.15.151:0/1427884335 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fafb80056d0 con 0x562cf47bf140 2024-05-21T22:04:41.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.198+0000 7fafb2000700 1 -- 172.21.15.151:0/1427884335 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fafb8021070 con 0x562cf47bf140 2024-05-21T22:04:41.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.198+0000 7fafd93deb80 10 client.27299 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:04:41.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.198+0000 7fafd93deb80 10 client.27299 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T22:04:41.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.198+0000 7fafd93deb80 20 client.27299 populate_metadata read hostname 'smithi151' 2024-05-21T22:04:41.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.198+0000 7fafd93deb80 1 --2- 172.21.15.151:0/1427884335 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x562cf47d8310 0x562cf47da700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:41.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.198+0000 7fafd93deb80 4 client.27299 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T22:04:41.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.198+0000 7fafd93deb80 1 -- 172.21.15.151:0/1427884335 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562cf471ed90 con 0x562cf47d8310 2024-05-21T22:04:41.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.199+0000 7fafc7400700 1 --2- 172.21.15.151:0/1427884335 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x562cf47d8310 0x562cf47da700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:41.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.199+0000 7fafc7400700 1 --2- 172.21.15.151:0/1427884335 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x562cf47d8310 0x562cf47da700 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97cfd7463d3fb525 server_cookie=92bb4e30c32a0f38 in_seq=0 out_seq=0 2024-05-21T22:04:41.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.199+0000 7fafb2000700 10 client.27299 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T22:04:41.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.200+0000 7fafb2000700 1 -- 172.21.15.151:0/1427884335 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x562cf471ed90 con 0x562cf47d8310 2024-05-21T22:04:41.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.200+0000 7fafb2000700 10 client.27299 handle_command_reply: tid=0 2024-05-21T22:04:41.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.200+0000 7fafd93deb80 1 client.27299 shutdown 2024-05-21T22:04:41.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.200+0000 7fafd93deb80 1 -- 172.21.15.151:0/1427884335 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cf47bf140 msgr2=0x562cf47cabf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:04:41.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.200+0000 7fafd93deb80 1 --2- 172.21.15.151:0/1427884335 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cf47bf140 0x562cf47cabf0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fafb800bd10 tx=0x7fafb800bd40 comp rx=0 tx=0).stop 2024-05-21T22:04:41.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.200+0000 7fafd93deb80 1 -- 172.21.15.151:0/1427884335 shutdown_connections 2024-05-21T22:04:41.203 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.200+0000 7fafd93deb80 1 --2- 172.21.15.151:0/1427884335 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x562cf47d8310 0x562cf47da700 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:41.203 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.200+0000 7fafd93deb80 1 --2- 172.21.15.151:0/1427884335 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cf47bf140 0x562cf47cabf0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:41.203 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.200+0000 7fafd93deb80 1 --2- 172.21.15.151:0/1427884335 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cf47c4f30 0x562cf47c84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:41.203 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.200+0000 7fafb2000700 0 client.27299 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T22:04:41.203 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.200+0000 7fafd93deb80 1 --2- 172.21.15.151:0/1427884335 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562cf47c4610 0x562cf47c7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:41.203 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.200+0000 7fafd93deb80 1 -- 172.21.15.151:0/1427884335 >> 172.21.15.151:0/1427884335 conn(0x562cf46ecbe0 msgr2=0x562cf47b8d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:04:41.203 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.201+0000 7fafd93deb80 1 -- 172.21.15.151:0/1427884335 shutdown_connections 2024-05-21T22:04:41.203 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.201+0000 7fafd93deb80 1 -- 172.21.15.151:0/1427884335 wait complete. 2024-05-21T22:04:41.203 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.201+0000 7fafd93deb80 20 client.27299 trim_cache size 0 max 16384 2024-05-21T22:04:41.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.223+0000 7fafd93deb80 1 Processor -- start 2024-05-21T22:04:41.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.223+0000 7fafd93deb80 1 -- start start 2024-05-21T22:04:41.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.223+0000 7fafd93deb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562cf47bf140 0x562cf47beb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:41.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.223+0000 7fafd93deb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cf47c4610 0x562cf47c83e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:41.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.223+0000 7fafd93deb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cf47c4f30 0x562cf47b94f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:41.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.223+0000 7fafd93deb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fafc81b00b0 con 0x562cf47c4610 2024-05-21T22:04:41.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.223+0000 7fafd93deb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fafc81b1bf0 con 0x562cf47c4f30 2024-05-21T22:04:41.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.223+0000 7fafd93deb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fafc81b0780 con 0x562cf47bf140 2024-05-21T22:04:41.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.223+0000 7fafcca00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cf47c4f30 0x562cf47b94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:41.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.223+0000 7fafc7400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cf47c4610 0x562cf47c83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:41.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.223+0000 7fafcca00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cf47c4f30 0x562cf47b94f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:58388/0 (socket says 172.21.15.151:58388) 2024-05-21T22:04:41.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.223+0000 7fafcca00700 1 -- 172.21.15.151:0/3588092840 learned_addr learned my addr 172.21.15.151:0/3588092840 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:04:41.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.224+0000 7fafc7e00700 1 --2- 172.21.15.151:0/3588092840 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562cf47bf140 0x562cf47beb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:41.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.224+0000 7fafcca00700 1 -- 172.21.15.151:0/3588092840 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562cf47bf140 msgr2=0x562cf47beb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:04:41.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.224+0000 7fafcca00700 1 --2- 172.21.15.151:0/3588092840 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562cf47bf140 0x562cf47beb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:41.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.224+0000 7fafcca00700 1 -- 172.21.15.151:0/3588092840 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cf47c4610 msgr2=0x562cf47c83e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:04:41.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.224+0000 7fafcca00700 1 --2- 172.21.15.151:0/3588092840 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cf47c4610 0x562cf47c83e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:41.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.224+0000 7fafcca00700 1 -- 172.21.15.151:0/3588092840 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562cf47c6500 con 0x562cf47c4f30 2024-05-21T22:04:41.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.224+0000 7fafcca00700 1 --2- 172.21.15.151:0/3588092840 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cf47c4f30 0x562cf47b94f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fafc0003e10 tx=0x7fafc0017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2ed1192488874b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:04:41.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.225+0000 7fafbcc00700 1 -- 172.21.15.151:0/3588092840 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc0004db0 con 0x562cf47c4f30 2024-05-21T22:04:41.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.225+0000 7fafbcc00700 1 -- 172.21.15.151:0/3588092840 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fafc0028070 con 0x562cf47c4f30 2024-05-21T22:04:41.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.225+0000 7fafbcc00700 1 -- 172.21.15.151:0/3588092840 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc002c680 con 0x562cf47c4f30 2024-05-21T22:04:41.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.225+0000 7fafd93deb80 1 -- 172.21.15.151:0/3588092840 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cf47c4f30 msgr2=0x562cf47b94f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:04:41.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.225+0000 7fafd93deb80 1 --2- 172.21.15.151:0/3588092840 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cf47c4f30 0x562cf47b94f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fafc0003e10 tx=0x7fafc0017000 comp rx=0 tx=0).stop 2024-05-21T22:04:41.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.226+0000 7fafd93deb80 1 -- 172.21.15.151:0/3588092840 shutdown_connections 2024-05-21T22:04:41.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.226+0000 7fafd93deb80 1 --2- 172.21.15.151:0/3588092840 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cf47c4f30 0x562cf47b94f0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:41.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.226+0000 7fafd93deb80 1 --2- 172.21.15.151:0/3588092840 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562cf47bf140 0x562cf47beb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:41.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.226+0000 7fafd93deb80 1 --2- 172.21.15.151:0/3588092840 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cf47c4610 0x562cf47c83e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:41.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.226+0000 7fafd93deb80 1 -- 172.21.15.151:0/3588092840 >> 172.21.15.151:0/3588092840 conn(0x562cf46ecbe0 msgr2=0x562cf47c30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:04:41.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.226+0000 7fafd93deb80 1 -- 172.21.15.151:0/3588092840 shutdown_connections 2024-05-21T22:04:41.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.226+0000 7fafd93deb80 1 -- 172.21.15.151:0/3588092840 wait complete. 2024-05-21T22:04:41.229 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.226+0000 7fafd93deb80 1 -- start start 2024-05-21T22:04:41.229 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.227+0000 7fafd93deb80 1 Processor -- start 2024-05-21T22:04:41.229 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.227+0000 7fafd93deb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cf47bf140 0x562cf47db5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:41.229 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.227+0000 7fafd93deb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562cf47c4610 0x562cf47dbb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:41.229 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.227+0000 7fafd93deb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cf47c4f30 0x562cf47dc070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:41.229 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.227+0000 7fafd93deb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fafc81b1520 con 0x562cf47bf140 2024-05-21T22:04:41.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.227+0000 7fafd93deb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fafc81b0e50 con 0x562cf47c4f30 2024-05-21T22:04:41.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.227+0000 7fafd93deb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fafc81ae070 con 0x562cf47c4610 2024-05-21T22:04:41.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.227+0000 7fafd93deb80 10 client.0 _ll_register_callbacks cb 0x562cf46b6c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:04:41.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.227+0000 7fafc7400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562cf47c4610 0x562cf47dbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:41.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.227+0000 7fafc7e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cf47bf140 0x562cf47db5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:41.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.227+0000 7fafcca00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cf47c4f30 0x562cf47dc070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:41.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.227+0000 7fafc7400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562cf47c4610 0x562cf47dbb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:55448/0 (socket says 172.21.15.151:55448) 2024-05-21T22:04:41.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.227+0000 7fafc7400700 1 -- 172.21.15.151:0/4032627655 learned_addr learned my addr 172.21.15.151:0/4032627655 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:04:41.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.227+0000 7fafcca00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cf47c4f30 0x562cf47dc070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:58390/0 (socket says 172.21.15.151:58390) 2024-05-21T22:04:41.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.228+0000 7fafc7400700 1 -- 172.21.15.151:0/4032627655 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cf47c4f30 msgr2=0x562cf47dc070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:04:41.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.228+0000 7fafc7400700 1 --2- 172.21.15.151:0/4032627655 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cf47c4f30 0x562cf47dc070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:41.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.228+0000 7fafc7400700 1 -- 172.21.15.151:0/4032627655 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cf47bf140 msgr2=0x562cf47db5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:04:41.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.228+0000 7fafc7400700 1 --2- 172.21.15.151:0/4032627655 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cf47bf140 0x562cf47db5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:41.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.228+0000 7fafc7400700 1 -- 172.21.15.151:0/4032627655 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562cf4841c90 con 0x562cf47c4610 2024-05-21T22:04:41.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.228+0000 7fafc7400700 1 --2- 172.21.15.151:0/4032627655 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562cf47c4610 0x562cf47dbb30 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fafb40a9810 tx=0x7fafb4091040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=249ffa48735570d server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:04:41.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.228+0000 7fafb2000700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-21T22:04:41.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.228+0000 7fafb2000700 1 -- 172.21.15.151:0/4032627655 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb401c040 con 0x562cf47c4610 2024-05-21T22:04:41.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.228+0000 7fafd93deb80 1 -- 172.21.15.151:0/4032627655 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fafc0028070 con 0x562cf47c4610 2024-05-21T22:04:41.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.228+0000 7fafb2000700 1 -- 172.21.15.151:0/4032627655 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7fafb40dff70 con 0x562cf47c4610 2024-05-21T22:04:41.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.229+0000 7fafb2000700 1 -- 172.21.15.151:0/4032627655 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fafb40a9650 con 0x562cf47c4610 2024-05-21T22:04:41.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.229+0000 7fafd93deb80 10 client.27307 fetch_fsmap learned FSMap version 29 2024-05-21T22:04:41.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.229+0000 7fafd93deb80 1 -- 172.21.15.151:0/4032627655 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562cf4842fe0 con 0x562cf47c4610 2024-05-21T22:04:41.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.229+0000 7fafb2000700 1 -- 172.21.15.151:0/4032627655 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fafc0028070 con 0x562cf47c4610 2024-05-21T22:04:41.233 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.229+0000 7fafb2000700 1 -- 172.21.15.151:0/4032627655 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fafb40a9ae0 con 0x562cf47c4610 2024-05-21T22:04:41.233 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.229+0000 7fafd93deb80 10 client.27307 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:04:41.233 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.229+0000 7fafd93deb80 10 client.27307 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T22:04:41.233 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.229+0000 7fafd93deb80 20 client.27307 populate_metadata read hostname 'smithi151' 2024-05-21T22:04:41.233 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.229+0000 7fafd93deb80 1 --2- 172.21.15.151:0/4032627655 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x562cf48436a0 0x562cf4845ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:41.233 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.229+0000 7fafd93deb80 4 client.27307 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-21T22:04:41.233 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.229+0000 7fafd93deb80 1 -- 172.21.15.151:0/4032627655 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562cf4718570 con 0x562cf48436a0 2024-05-21T22:04:41.233 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.230+0000 7fafc7e00700 1 --2- 172.21.15.151:0/4032627655 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x562cf48436a0 0x562cf4845ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:41.233 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.230+0000 7fafc7e00700 1 --2- 172.21.15.151:0/4032627655 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x562cf48436a0 0x562cf4845ac0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2b517ebd39a7950 server_cookie=7fb719131cc40563 in_seq=0 out_seq=0 2024-05-21T22:04:41.233 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.230+0000 7fafb2000700 10 client.27307 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T22:04:41.233 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.230+0000 7fafb2000700 1 -- 172.21.15.151:0/4032627655 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x562cf4718570 con 0x562cf48436a0 2024-05-21T22:04:41.233 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.230+0000 7fafb2000700 10 client.27307 handle_command_reply: tid=0 2024-05-21T22:04:41.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.230+0000 7fafd93deb80 1 client.27307 shutdown 2024-05-21T22:04:41.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.231+0000 7fafd93deb80 1 -- 172.21.15.151:0/4032627655 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562cf47c4610 msgr2=0x562cf47dbb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:04:41.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.231+0000 7fafd93deb80 1 --2- 172.21.15.151:0/4032627655 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562cf47c4610 0x562cf47dbb30 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fafb40a9810 tx=0x7fafb4091040 comp rx=0 tx=0).stop 2024-05-21T22:04:41.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.231+0000 7fafd93deb80 1 -- 172.21.15.151:0/4032627655 shutdown_connections 2024-05-21T22:04:41.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.231+0000 7fafd93deb80 1 --2- 172.21.15.151:0/4032627655 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x562cf48436a0 0x562cf4845ac0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:41.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.231+0000 7fafd93deb80 1 --2- 172.21.15.151:0/4032627655 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x562cf47c4f30 0x562cf47dc070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:41.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.231+0000 7fafd93deb80 1 --2- 172.21.15.151:0/4032627655 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cf47bf140 0x562cf47db5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:41.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.231+0000 7fafd93deb80 1 --2- 172.21.15.151:0/4032627655 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562cf47c4610 0x562cf47dbb30 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:41.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.231+0000 7fafb2000700 0 client.27307 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T22:04:41.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.231+0000 7fafd93deb80 1 -- 172.21.15.151:0/4032627655 >> 172.21.15.151:0/4032627655 conn(0x562cf46ecbe0 msgr2=0x562cf47c9dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:04:41.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.231+0000 7fafd93deb80 1 -- 172.21.15.151:0/4032627655 shutdown_connections 2024-05-21T22:04:41.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.231+0000 7fafd93deb80 1 -- 172.21.15.151:0/4032627655 wait complete. 2024-05-21T22:04:41.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.231+0000 7fafd93deb80 20 client.27307 trim_cache size 0 max 16384 2024-05-21T22:04:41.235 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-05-21T22:04:41.235 INFO:teuthology.orchestra.run.smithi151.stdout: "status": "no active scrubs running", 2024-05-21T22:04:41.235 INFO:teuthology.orchestra.run.smithi151.stdout: "scrubs": {} 2024-05-21T22:04:41.235 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-05-21T22:04:41.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.232+0000 7fafcea00700 1 -- 172.21.15.151:0/693863525 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fafac0b29f0 msgr2=0x7fafac0b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:04:41.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.232+0000 7fafcea00700 1 --2- 172.21.15.151:0/693863525 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fafac0b29f0 0x7fafac0b4eb0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fafc00127f0 tx=0x7fafc000b040 comp rx=0 tx=0).stop 2024-05-21T22:04:41.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.232+0000 7fafcea00700 1 -- 172.21.15.151:0/693863525 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fafc8106140 msgr2=0x7fafc819c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:04:41.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.232+0000 7fafcea00700 1 --2- 172.21.15.151:0/693863525 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fafc8106140 0x7fafc819c3c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fafb400e9f0 tx=0x7fafb400ed00 comp rx=0 tx=0).stop 2024-05-21T22:04:41.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.232+0000 7fafcea00700 1 -- 172.21.15.151:0/693863525 shutdown_connections 2024-05-21T22:04:41.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.232+0000 7fafcea00700 1 --2- 172.21.15.151:0/693863525 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fafac0b29f0 0x7fafac0b4eb0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:41.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.232+0000 7fafcea00700 1 --2- 172.21.15.151:0/693863525 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fafc810b780 0x7fafc81a0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:41.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.232+0000 7fafcea00700 1 --2- 172.21.15.151:0/693863525 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fafc8105780 0x7fafc819be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:41.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.232+0000 7fafcea00700 1 --2- 172.21.15.151:0/693863525 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fafc8106140 0x7fafc819c3c0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:41.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.232+0000 7fafcea00700 1 -- 172.21.15.151:0/693863525 >> 172.21.15.151:0/693863525 conn(0x7fafc8101270 msgr2=0x7fafc8101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:04:41.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.232+0000 7fafcea00700 1 -- 172.21.15.151:0/693863525 shutdown_connections 2024-05-21T22:04:41.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:41.232+0000 7fafcea00700 1 -- 172.21.15.151:0/693863525 wait complete. 2024-05-21T22:04:41.630 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:04:41 smithi151 ceph-mon[31623]: pgmap v281: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:04:41.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:04:41 smithi184 ceph-mon[36942]: pgmap v281: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:04:42.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:04:41 smithi164 ceph-mon[36269]: pgmap v281: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:04:42.207 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-21T22:04:42.208 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-21T22:04:42.630 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:04:43.954 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:04:43 smithi151 ceph-mon[31623]: pgmap v282: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T22:04:43.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:04:43 smithi184 ceph-mon[36942]: pgmap v282: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T22:04:44.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:04:43 smithi164 ceph-mon[36269]: pgmap v282: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T22:04:44.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:44.930+0000 7ffac1200700 1 -- 172.21.15.151:0/4232605623 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffabc102440 msgr2=0x7ffabc1028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:04:44.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:44.930+0000 7ffac1200700 1 --2- 172.21.15.151:0/4232605623 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffabc102440 0x7ffabc1028c0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7ffaa4009990 tx=0x7ffaa4009ca0 comp rx=0 tx=0).stop 2024-05-21T22:04:44.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:44.930+0000 7ffac1200700 1 -- 172.21.15.151:0/4232605623 shutdown_connections 2024-05-21T22:04:44.932 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:44.930+0000 7ffac1200700 1 --2- 172.21.15.151:0/4232605623 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffabc102e00 0x7ffabc10f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:44.932 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:44.930+0000 7ffac1200700 1 --2- 172.21.15.151:0/4232605623 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffabc108520 0x7ffabc108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:44.932 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:44.930+0000 7ffac1200700 1 --2- 172.21.15.151:0/4232605623 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffabc102440 0x7ffabc1028c0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:44.932 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:44.930+0000 7ffac1200700 1 -- 172.21.15.151:0/4232605623 >> 172.21.15.151:0/4232605623 conn(0x7ffabc0793f0 msgr2=0x7ffabc079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:04:44.932 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:44.930+0000 7ffac1200700 1 -- 172.21.15.151:0/4232605623 shutdown_connections 2024-05-21T22:04:44.932 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:44.930+0000 7ffac1200700 1 -- 172.21.15.151:0/4232605623 wait complete. 2024-05-21T22:04:44.932 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:44.931+0000 7ffac1200700 1 Processor -- start 2024-05-21T22:04:44.932 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:44.931+0000 7ffac1200700 1 -- start start 2024-05-21T22:04:44.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:44.932+0000 7ffac1200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffabc102440 0x7ffabc19bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:44.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:44.932+0000 7ffac1200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffabc102e00 0x7ffabc19c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:44.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:44.932+0000 7ffac1200700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffabc108520 0x7ffabc1a07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:44.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:44.932+0000 7ffac1200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ffabc1125f0 con 0x7ffabc102440 2024-05-21T22:04:44.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:44.932+0000 7ffac1200700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ffabc112470 con 0x7ffabc102e00 2024-05-21T22:04:44.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:44.932+0000 7ffac1200700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ffabc112770 con 0x7ffabc108520 2024-05-21T22:04:44.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:44.932+0000 7ffab9e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffabc102440 0x7ffabc19bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:44.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:44.932+0000 7ffab9e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffabc102440 0x7ffabc19bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52704/0 (socket says 172.21.15.151:52704) 2024-05-21T22:04:44.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:44.932+0000 7ffab9e00700 1 -- 172.21.15.151:0/910877695 learned_addr learned my addr 172.21.15.151:0/910877695 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:04:44.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:44.932+0000 7ffab1400700 1 --2- 172.21.15.151:0/910877695 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffabc102e00 0x7ffabc19c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:44.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:44.932+0000 7ffaba800700 1 --2- 172.21.15.151:0/910877695 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffabc108520 0x7ffabc1a07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:44.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:44.932+0000 7ffab1400700 1 -- 172.21.15.151:0/910877695 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffabc108520 msgr2=0x7ffabc1a07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:04:44.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:44.932+0000 7ffab1400700 1 --2- 172.21.15.151:0/910877695 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffabc108520 0x7ffabc1a07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:44.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:44.932+0000 7ffab1400700 1 -- 172.21.15.151:0/910877695 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffabc102440 msgr2=0x7ffabc19bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:04:44.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:44.932+0000 7ffab1400700 1 --2- 172.21.15.151:0/910877695 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffabc102440 0x7ffabc19bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:44.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:44.932+0000 7ffab1400700 1 -- 172.21.15.151:0/910877695 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffabc1a0f00 con 0x7ffabc102e00 2024-05-21T22:04:44.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:44.933+0000 7ffab1400700 1 --2- 172.21.15.151:0/910877695 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffabc102e00 0x7ffabc19c430 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7ffaa40059d0 tx=0x7ffaa40047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:04:44.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:44.933+0000 7ffab3400700 1 -- 172.21.15.151:0/910877695 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaa4027070 con 0x7ffabc102e00 2024-05-21T22:04:44.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:44.933+0000 7ffab3400700 1 -- 172.21.15.151:0/910877695 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffaa4022470 con 0x7ffabc102e00 2024-05-21T22:04:44.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:44.933+0000 7ffac1200700 1 -- 172.21.15.151:0/910877695 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffabc1a1190 con 0x7ffabc102e00 2024-05-21T22:04:44.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:44.933+0000 7ffab3400700 1 -- 172.21.15.151:0/910877695 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaa4017630 con 0x7ffabc102e00 2024-05-21T22:04:44.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:44.933+0000 7ffac1200700 1 -- 172.21.15.151:0/910877695 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffabc1a8a70 con 0x7ffabc102e00 2024-05-21T22:04:44.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:44.935+0000 7ffac1200700 1 -- 172.21.15.151:0/910877695 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffabc06a230 con 0x7ffabc102e00 2024-05-21T22:04:44.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:44.937+0000 7ffab3400700 1 -- 172.21.15.151:0/910877695 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7ffaa402a070 con 0x7ffabc102e00 2024-05-21T22:04:44.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:44.938+0000 7ffab3400700 1 --2- 172.21.15.151:0/910877695 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7ffaa00b29f0 0x7ffaa00b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:44.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:44.939+0000 7ffab3400700 1 -- 172.21.15.151:0/910877695 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7ffaa40e3290 con 0x7ffabc102e00 2024-05-21T22:04:44.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:44.939+0000 7ffab9e00700 1 --2- 172.21.15.151:0/910877695 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7ffaa00b29f0 0x7ffaa00b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:44.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:44.939+0000 7ffab9e00700 1 --2- 172.21.15.151:0/910877695 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7ffaa00b29f0 0x7ffaa00b4eb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ffaac009ce0 tx=0x7ffaac009260 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:04:44.943 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:44.943+0000 7ffab3400700 1 -- 172.21.15.151:0/910877695 <== mon.2 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ffaa4013070 con 0x7ffabc102e00 2024-05-21T22:04:45.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:45.235+0000 7ffac1200700 1 -- 172.21.15.151:0/910877695 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffabc1a8c40 con 0x7ffabc102e00 2024-05-21T22:04:45.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:45.236+0000 7ffab3400700 1 -- 172.21.15.151:0/910877695 <== mon.2 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11958 (secure 0 0 0) 0x7ffaa40aac90 con 0x7ffabc102e00 2024-05-21T22:04:45.237 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T22:04:45.237 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14358,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/158010529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":158010529},{"type":"v1","addr":"172.21.15.151:6835","nonce":158010529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6839/2069058115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":2069058115},{"type":"v1","addr":"172.21.15.151:6839","nonce":2069058115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T21:51:19.552069+0000","modified":"2024-05-21T21:55:34.643755+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24530,"mds_1":24493,"mds_2":24491,"mds_3":24478,"mds_4":24473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14403":{"gid":14403,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6841/1749723794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6840","nonce":1749723794},{"type":"v1","addr":"172.21.15.151:6841","nonce":1749723794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/2026266114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":2026266114},{"type":"v1","addr":"172.21.15.184:6833","nonce":2026266114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/2467900844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":2467900844},{"type":"v1","addr":"172.21.15.164:6833","nonce":2467900844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3367640960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3367640960},{"type":"v1","addr":"172.21.15.151:6837","nonce":3367640960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/3264142621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3264142621},{"type":"v1","addr":"172.21.15.184:6835","nonce":3264142621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":17,"addr":"172.21.15.164:6835/2340428470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2340428470},{"type":"v1","addr":"172.21.15.164:6835","nonce":2340428470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":16,"addr":"172.21.15.184:6837/2988194888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":2988194888},{"type":"v1","addr":"172.21.15.184:6837","nonce":2988194888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.164:6837/1554609572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1554609572},{"type":"v1","addr":"172.21.15.164:6837","nonce":1554609572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.184:6839/3059935143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":3059935143},{"type":"v1","addr":"172.21.15.184:6839","nonce":3059935143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.164:6839/2646401247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":2646401247},{"type":"v1","addr":"172.21.15.164:6839","nonce":2646401247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24530,"qdb_cluster":[24530,24493,24491,24473,24478]},"id":1}]} 2024-05-21T22:04:45.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:45.239+0000 7ffac1200700 1 -- 172.21.15.151:0/910877695 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7ffaa00b29f0 msgr2=0x7ffaa00b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:04:45.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:45.239+0000 7ffac1200700 1 --2- 172.21.15.151:0/910877695 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7ffaa00b29f0 0x7ffaa00b4eb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ffaac009ce0 tx=0x7ffaac009260 comp rx=0 tx=0).stop 2024-05-21T22:04:45.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:45.239+0000 7ffac1200700 1 -- 172.21.15.151:0/910877695 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffabc102e00 msgr2=0x7ffabc19c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:04:45.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:45.239+0000 7ffac1200700 1 --2- 172.21.15.151:0/910877695 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffabc102e00 0x7ffabc19c430 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7ffaa40059d0 tx=0x7ffaa40047b0 comp rx=0 tx=0).stop 2024-05-21T22:04:45.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:45.239+0000 7ffac1200700 1 -- 172.21.15.151:0/910877695 shutdown_connections 2024-05-21T22:04:45.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:45.239+0000 7ffac1200700 1 --2- 172.21.15.151:0/910877695 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7ffaa00b29f0 0x7ffaa00b4eb0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:45.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:45.239+0000 7ffac1200700 1 --2- 172.21.15.151:0/910877695 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffabc108520 0x7ffabc1a07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:45.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:45.239+0000 7ffac1200700 1 --2- 172.21.15.151:0/910877695 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffabc102440 0x7ffabc19bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:45.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:45.239+0000 7ffac1200700 1 --2- 172.21.15.151:0/910877695 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffabc102e00 0x7ffabc19c430 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:45.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:45.239+0000 7ffac1200700 1 -- 172.21.15.151:0/910877695 >> 172.21.15.151:0/910877695 conn(0x7ffabc0793f0 msgr2=0x7ffabc06b500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:04:45.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:45.239+0000 7ffac1200700 1 -- 172.21.15.151:0/910877695 shutdown_connections 2024-05-21T22:04:45.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:45.239+0000 7ffac1200700 1 -- 172.21.15.151:0/910877695 wait complete. 2024-05-21T22:04:45.241 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 29 2024-05-21T22:04:45.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:04:45 smithi151 ceph-mon[31623]: pgmap v283: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:04:45.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:04:45 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/910877695' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T22:04:45.985 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:04:45 smithi184 ceph-mon[36942]: pgmap v283: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:04:45.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:04:45 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/910877695' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T22:04:46.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:04:45 smithi164 ceph-mon[36269]: pgmap v283: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:04:46.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:04:45 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/910877695' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T22:04:46.041 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-21T22:04:46.490 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:04:47.880 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:04:47 smithi151 ceph-mon[31623]: pgmap v284: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:04:47.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:04:47 smithi184 ceph-mon[36942]: pgmap v284: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:04:48.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:04:47 smithi164 ceph-mon[36269]: pgmap v284: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:04:48.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.681+0000 7f6761e00700 1 -- 172.21.15.151:0/3220042619 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f675c10a440 msgr2=0x7f675c1168e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:04:48.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.681+0000 7f6761e00700 1 --2- 172.21.15.151:0/3220042619 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f675c10a440 0x7f675c1168e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f6754009f90 tx=0x7f675400b330 comp rx=0 tx=0).stop 2024-05-21T22:04:48.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.681+0000 7f6761e00700 1 -- 172.21.15.151:0/3220042619 shutdown_connections 2024-05-21T22:04:48.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.681+0000 7f6761e00700 1 --2- 172.21.15.151:0/3220042619 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f675c10e950 0x7f675c10ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:48.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.681+0000 7f6761e00700 1 --2- 172.21.15.151:0/3220042619 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f675c10a440 0x7f675c1168e0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:48.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.681+0000 7f6761e00700 1 --2- 172.21.15.151:0/3220042619 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f675c109a80 0x7f675c109f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:48.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.681+0000 7f6761e00700 1 -- 172.21.15.151:0/3220042619 >> 172.21.15.151:0/3220042619 conn(0x7f675c0792d0 msgr2=0x7f675c108360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:04:48.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.682+0000 7f6761e00700 1 -- 172.21.15.151:0/3220042619 shutdown_connections 2024-05-21T22:04:48.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.682+0000 7f6761e00700 1 -- 172.21.15.151:0/3220042619 wait complete. 2024-05-21T22:04:48.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.683+0000 7f6761e00700 1 Processor -- start 2024-05-21T22:04:48.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.683+0000 7f6761e00700 1 -- start start 2024-05-21T22:04:48.685 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.684+0000 7f6761e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f675c109a80 0x7f675c1a0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:48.685 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.684+0000 7f6761e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f675c10a440 0x7f675c1a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:48.685 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.684+0000 7f6761e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f675c10e950 0x7f675c1a49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:48.685 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.684+0000 7f6761e00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f675c119c80 con 0x7f675c10a440 2024-05-21T22:04:48.685 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.684+0000 7f6761e00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f675c119b00 con 0x7f675c10e950 2024-05-21T22:04:48.685 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.684+0000 7f6761e00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f675c119e00 con 0x7f675c109a80 2024-05-21T22:04:48.685 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.684+0000 7f6760a00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f675c109a80 0x7f675c1a0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:48.685 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.685+0000 7f6761400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f675c10e950 0x7f675c1a49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:48.685 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.685+0000 7f675be00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f675c10a440 0x7f675c1a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:48.685 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.685+0000 7f6761400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f675c10e950 0x7f675c1a49d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:33750/0 (socket says 172.21.15.151:33750) 2024-05-21T22:04:48.686 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.685+0000 7f6761400700 1 -- 172.21.15.151:0/2039680758 learned_addr learned my addr 172.21.15.151:0/2039680758 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:04:48.686 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.685+0000 7f6761400700 1 -- 172.21.15.151:0/2039680758 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f675c109a80 msgr2=0x7f675c1a0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:04:48.686 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.685+0000 7f6761400700 1 --2- 172.21.15.151:0/2039680758 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f675c109a80 0x7f675c1a0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:48.687 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.685+0000 7f6761400700 1 -- 172.21.15.151:0/2039680758 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f675c10a440 msgr2=0x7f675c1a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:04:48.687 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.685+0000 7f6761400700 1 --2- 172.21.15.151:0/2039680758 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f675c10a440 0x7f675c1a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:48.687 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.685+0000 7f6761400700 1 -- 172.21.15.151:0/2039680758 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f675c1a5110 con 0x7f675c10e950 2024-05-21T22:04:48.687 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.685+0000 7f6761400700 1 --2- 172.21.15.151:0/2039680758 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f675c10e950 0x7f675c1a49d0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f6754009c10 tx=0x7f675400cca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:04:48.687 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.686+0000 7f6759600700 1 -- 172.21.15.151:0/2039680758 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6754021070 con 0x7f675c10e950 2024-05-21T22:04:48.687 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.686+0000 7f6759600700 1 -- 172.21.15.151:0/2039680758 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f675400ce90 con 0x7f675c10e950 2024-05-21T22:04:48.687 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.686+0000 7f6759600700 1 -- 172.21.15.151:0/2039680758 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6754005040 con 0x7f675c10e950 2024-05-21T22:04:48.687 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.686+0000 7f6761e00700 1 -- 172.21.15.151:0/2039680758 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f675c1a53a0 con 0x7f675c10e950 2024-05-21T22:04:48.687 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.686+0000 7f6761e00700 1 -- 172.21.15.151:0/2039680758 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f675c1acc80 con 0x7f675c10e950 2024-05-21T22:04:48.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.691+0000 7f6759600700 1 -- 172.21.15.151:0/2039680758 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7f6754024070 con 0x7f675c10e950 2024-05-21T22:04:48.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.692+0000 7f676dd50b80 1 Processor -- start 2024-05-21T22:04:48.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.692+0000 7f676dd50b80 1 -- start start 2024-05-21T22:04:48.693 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.692+0000 7f676dd50b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563674e33020 0x563674e33400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:48.693 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.692+0000 7f676dd50b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563674e33940 0x563674e2d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:48.693 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.692+0000 7f676dd50b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563674e2dad0 0x563674e2df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:48.694 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.692+0000 7f676dd50b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x563674d8be40 con 0x563674e33020 2024-05-21T22:04:48.694 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.692+0000 7f676dd50b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x563674d7b9e0 con 0x563674e33940 2024-05-21T22:04:48.694 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.692+0000 7f676dd50b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x563674d7fa20 con 0x563674e2dad0 2024-05-21T22:04:48.694 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.693+0000 7f6759600700 1 --2- 172.21.15.151:0/2039680758 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f67440b2a70 0x7f67440b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:48.694 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.693+0000 7f675be00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563674e33020 0x563674e33400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:48.694 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.693+0000 7f6760a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563674e33940 0x563674e2d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:48.694 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.693+0000 7f675be00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563674e33020 0x563674e33400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:33646/0 (socket says 172.21.15.151:33646) 2024-05-21T22:04:48.694 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.693+0000 7f675be00700 1 -- 172.21.15.151:0/1703437571 learned_addr learned my addr 172.21.15.151:0/1703437571 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:04:48.694 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.693+0000 7f6760a00700 1 --2- 172.21.15.151:0/1703437571 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563674e33940 0x563674e2d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:33760/0 (socket says 172.21.15.151:33760) 2024-05-21T22:04:48.694 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.693+0000 7f6759600700 1 -- 172.21.15.151:0/2039680758 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7f675400f020 con 0x7f675c10e950 2024-05-21T22:04:48.694 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.693+0000 7f675be00700 1 --2- 172.21.15.151:0/1703437571 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563674e2dad0 0x563674e2df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:48.694 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.693+0000 7f675be00700 1 -- 172.21.15.151:0/1703437571 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563674e2dad0 msgr2=0x563674e2df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:04:48.694 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.693+0000 7f675be00700 1 --2- 172.21.15.151:0/1703437571 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563674e2dad0 0x563674e2df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:48.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.693+0000 7f675be00700 1 -- 172.21.15.151:0/1703437571 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563674e33940 msgr2=0x563674e2d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:04:48.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.693+0000 7f675be00700 1 --2- 172.21.15.151:0/1703437571 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563674e33940 0x563674e2d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:48.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.693+0000 7f675be00700 1 -- 172.21.15.151:0/1703437571 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563674d44420 con 0x563674e33020 2024-05-21T22:04:48.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.694+0000 7f675be00700 1 --2- 172.21.15.151:0/1703437571 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563674e33020 0x563674e33400 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f67480136b0 tx=0x7f6748009b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27e1ec05a315b64c server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:04:48.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.694+0000 7f6761400700 1 --2- 172.21.15.151:0/2039680758 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f67440b2a70 0x7f67440b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:48.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.694+0000 7f674e000700 1 -- 172.21.15.151:0/1703437571 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6748004050 con 0x563674e33020 2024-05-21T22:04:48.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.694+0000 7f674e000700 1 -- 172.21.15.151:0/1703437571 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6748003680 con 0x563674e33020 2024-05-21T22:04:48.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.694+0000 7f674e000700 1 -- 172.21.15.151:0/1703437571 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6748010690 con 0x563674e33020 2024-05-21T22:04:48.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.695+0000 7f676dd50b80 1 -- 172.21.15.151:0/1703437571 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563674e33020 msgr2=0x563674e33400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:04:48.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.695+0000 7f6761400700 1 --2- 172.21.15.151:0/2039680758 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f67440b2a70 0x7f67440b4f30 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f67540aac10 tx=0x7f6754009920 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:04:48.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.695+0000 7f676dd50b80 1 --2- 172.21.15.151:0/1703437571 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563674e33020 0x563674e33400 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f67480136b0 tx=0x7f6748009b70 comp rx=0 tx=0).stop 2024-05-21T22:04:48.697 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.695+0000 7f676dd50b80 1 -- 172.21.15.151:0/1703437571 shutdown_connections 2024-05-21T22:04:48.697 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.695+0000 7f676dd50b80 1 --2- 172.21.15.151:0/1703437571 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563674e2dad0 0x563674e2df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:48.697 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.695+0000 7f676dd50b80 1 --2- 172.21.15.151:0/1703437571 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563674e33020 0x563674e33400 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:48.697 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.695+0000 7f676dd50b80 1 --2- 172.21.15.151:0/1703437571 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563674e33940 0x563674e2d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:48.697 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.695+0000 7f676dd50b80 1 -- 172.21.15.151:0/1703437571 >> 172.21.15.151:0/1703437571 conn(0x563674d5bbe0 msgr2=0x563674e28150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:04:48.697 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.695+0000 7f676dd50b80 1 -- 172.21.15.151:0/1703437571 shutdown_connections 2024-05-21T22:04:48.697 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.695+0000 7f676dd50b80 1 -- 172.21.15.151:0/1703437571 wait complete. 2024-05-21T22:04:48.697 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.696+0000 7f676dd50b80 1 -- start start 2024-05-21T22:04:48.697 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.696+0000 7f676dd50b80 1 Processor -- start 2024-05-21T22:04:48.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.696+0000 7f676dd50b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563674e2dad0 0x563674e39620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:48.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.696+0000 7f676dd50b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563674e33020 0x563674e369c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:48.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.696+0000 7f676dd50b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563674e33940 0x563674e36f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:48.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.697+0000 7f676dd50b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x563674c81fa0 con 0x563674e33020 2024-05-21T22:04:48.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.697+0000 7f676dd50b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x563674d1c760 con 0x563674e2dad0 2024-05-21T22:04:48.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.697+0000 7f676dd50b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x563674e3a460 con 0x563674e33940 2024-05-21T22:04:48.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.697+0000 7f676dd50b80 10 client.0 _ll_register_callbacks cb 0x563674d25c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:04:48.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.697+0000 7f6760a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563674e33020 0x563674e369c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:48.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.697+0000 7f675be00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563674e2dad0 0x563674e39620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:48.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.697+0000 7f6760a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563674e33020 0x563674e369c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:33654/0 (socket says 172.21.15.151:33654) 2024-05-21T22:04:48.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.697+0000 7f6760a00700 1 -- 172.21.15.151:0/350832567 learned_addr learned my addr 172.21.15.151:0/350832567 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:04:48.701 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.697+0000 7f675be00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563674e2dad0 0x563674e39620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:33774/0 (socket says 172.21.15.151:33774) 2024-05-21T22:04:48.701 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.697+0000 7f675be00700 1 --2- 172.21.15.151:0/350832567 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563674e33940 0x563674e36f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:48.701 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.697+0000 7f6760a00700 1 -- 172.21.15.151:0/350832567 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563674e33940 msgr2=0x563674e36f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:04:48.701 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.697+0000 7f6760a00700 1 --2- 172.21.15.151:0/350832567 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563674e33940 0x563674e36f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:48.701 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.697+0000 7f6760a00700 1 -- 172.21.15.151:0/350832567 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563674e2dad0 msgr2=0x563674e39620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:04:48.701 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.697+0000 7f6760a00700 1 --2- 172.21.15.151:0/350832567 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563674e2dad0 0x563674e39620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:48.701 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.697+0000 7f6760a00700 1 -- 172.21.15.151:0/350832567 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563674ccb8d0 con 0x563674e33020 2024-05-21T22:04:48.701 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.697+0000 7f675be00700 1 --2- 172.21.15.151:0/350832567 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563674e33940 0x563674e36f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T22:04:48.701 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.698+0000 7f6760a00700 1 --2- 172.21.15.151:0/350832567 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563674e33020 0x563674e369c0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f67500127f0 tx=0x7f6750012b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cf403269c27f8c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:04:48.701 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.698+0000 7f6742a00700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-21T22:04:48.701 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.698+0000 7f6742a00700 1 -- 172.21.15.151:0/350832567 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f675000f5c0 con 0x563674e33020 2024-05-21T22:04:48.701 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.698+0000 7f6742a00700 1 -- 172.21.15.151:0/350832567 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7f675000f760 con 0x563674e33020 2024-05-21T22:04:48.701 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.698+0000 7f676dd50b80 1 -- 172.21.15.151:0/350832567 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6748003680 con 0x563674e33020 2024-05-21T22:04:48.701 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.698+0000 7f6742a00700 1 -- 172.21.15.151:0/350832567 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f675000f960 con 0x563674e33020 2024-05-21T22:04:48.701 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.698+0000 7f676dd50b80 10 client.17199 fetch_fsmap learned FSMap version 29 2024-05-21T22:04:48.702 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.698+0000 7f676dd50b80 1 -- 172.21.15.151:0/350832567 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563674d4c060 con 0x563674e33020 2024-05-21T22:04:48.702 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.699+0000 7f6742a00700 1 -- 172.21.15.151:0/350832567 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6748003680 con 0x563674e33020 2024-05-21T22:04:48.702 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.699+0000 7f6742a00700 1 -- 172.21.15.151:0/350832567 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f6750010b80 con 0x563674e33020 2024-05-21T22:04:48.702 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.699+0000 7f676dd50b80 10 client.17199 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:04:48.702 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.699+0000 7f676dd50b80 10 client.17199 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-05-21T22:04:48.702 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.699+0000 7f676dd50b80 20 client.17199 populate_metadata read hostname 'smithi151' 2024-05-21T22:04:48.702 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.699+0000 7f676dd50b80 1 --2- 172.21.15.151:0/350832567 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x563674e47030 0x563674e49420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:48.702 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.699+0000 7f676dd50b80 4 client.17199 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T22:04:48.702 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.699+0000 7f676dd50b80 1 -- 172.21.15.151:0/350832567 --> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563674d8dd90 con 0x563674e47030 2024-05-21T22:04:48.702 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.700+0000 7f675be00700 1 --2- 172.21.15.151:0/350832567 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x563674e47030 0x563674e49420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:48.703 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.700+0000 7f675be00700 1 --2- 172.21.15.151:0/350832567 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x563674e47030 0x563674e49420 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c632177e6c83e1db server_cookie=9e72934aee8062c0 in_seq=0 out_seq=0 2024-05-21T22:04:48.703 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.700+0000 7f6742a00700 10 client.17199 ms_handle_connect on v2:172.21.15.164:6834/2340428470 2024-05-21T22:04:48.703 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.701+0000 7f6742a00700 1 -- 172.21.15.151:0/350832567 <== mds.4 v2:172.21.15.164:6834/2340428470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x563674d8dd90 con 0x563674e47030 2024-05-21T22:04:48.703 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.701+0000 7f6742a00700 10 client.17199 handle_command_reply: tid=0 2024-05-21T22:04:48.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.701+0000 7f676dd50b80 1 client.17199 shutdown 2024-05-21T22:04:48.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.701+0000 7f676dd50b80 1 -- 172.21.15.151:0/350832567 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563674e33020 msgr2=0x563674e369c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:04:48.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.701+0000 7f676dd50b80 1 --2- 172.21.15.151:0/350832567 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563674e33020 0x563674e369c0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f67500127f0 tx=0x7f6750012b00 comp rx=0 tx=0).stop 2024-05-21T22:04:48.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.701+0000 7f676dd50b80 1 -- 172.21.15.151:0/350832567 shutdown_connections 2024-05-21T22:04:48.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.701+0000 7f676dd50b80 1 --2- 172.21.15.151:0/350832567 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x563674e47030 0x563674e49420 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:48.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.701+0000 7f676dd50b80 1 --2- 172.21.15.151:0/350832567 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563674e2dad0 0x563674e39620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:48.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.701+0000 7f676dd50b80 1 --2- 172.21.15.151:0/350832567 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563674e33940 0x563674e36f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:48.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.701+0000 7f676dd50b80 1 --2- 172.21.15.151:0/350832567 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563674e33020 0x563674e369c0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:48.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.701+0000 7f676dd50b80 1 -- 172.21.15.151:0/350832567 >> 172.21.15.151:0/350832567 conn(0x563674d5bbe0 msgr2=0x563674e2c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:04:48.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.701+0000 7f6742a00700 0 client.17199 ms_handle_reset on v2:172.21.15.164:6834/2340428470 2024-05-21T22:04:48.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.701+0000 7f676dd50b80 1 -- 172.21.15.151:0/350832567 shutdown_connections 2024-05-21T22:04:48.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.702+0000 7f676dd50b80 1 -- 172.21.15.151:0/350832567 wait complete. 2024-05-21T22:04:48.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.702+0000 7f676dd50b80 20 client.17199 trim_cache size 0 max 16384 2024-05-21T22:04:48.720 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.718+0000 7f676dd50b80 1 Processor -- start 2024-05-21T22:04:48.720 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.718+0000 7f676dd50b80 1 -- start start 2024-05-21T22:04:48.720 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.718+0000 7f676dd50b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563674e2dad0 0x563674e2d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:48.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.718+0000 7f676dd50b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563674e33020 0x563674e33400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:48.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.718+0000 7f676dd50b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563674e33940 0x563674e352b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:48.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.719+0000 7f676dd50b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f675c1b4260 con 0x563674e33020 2024-05-21T22:04:48.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.719+0000 7f676dd50b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f675c1b5da0 con 0x563674e33940 2024-05-21T22:04:48.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.719+0000 7f676dd50b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f675c1b4930 con 0x563674e2dad0 2024-05-21T22:04:48.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.719+0000 7f6760a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563674e33020 0x563674e33400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:48.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.719+0000 7f675be00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563674e2dad0 0x563674e2d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:48.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.719+0000 7f6760a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563674e33020 0x563674e33400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:33668/0 (socket says 172.21.15.151:33668) 2024-05-21T22:04:48.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.719+0000 7f6760a00700 1 -- 172.21.15.151:0/2295531743 learned_addr learned my addr 172.21.15.151:0/2295531743 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:04:48.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.719+0000 7f675be00700 1 --2- 172.21.15.151:0/2295531743 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563674e33940 0x563674e352b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:48.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.719+0000 7f675be00700 1 -- 172.21.15.151:0/2295531743 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563674e33940 msgr2=0x563674e352b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:04:48.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.719+0000 7f675be00700 1 --2- 172.21.15.151:0/2295531743 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563674e33940 0x563674e352b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:48.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.719+0000 7f675be00700 1 -- 172.21.15.151:0/2295531743 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563674e33020 msgr2=0x563674e33400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:04:48.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.719+0000 7f675be00700 1 --2- 172.21.15.151:0/2295531743 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563674e33020 0x563674e33400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:48.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.719+0000 7f675be00700 1 -- 172.21.15.151:0/2295531743 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563674e44a50 con 0x563674e2dad0 2024-05-21T22:04:48.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.719+0000 7f6760a00700 1 --2- 172.21.15.151:0/2295531743 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563674e33020 0x563674e33400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:04:48.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.720+0000 7f675be00700 1 --2- 172.21.15.151:0/2295531743 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563674e2dad0 0x563674e2d4b0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x563674e49c90 tx=0x7f6748003fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96e1fff9c3535294 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:04:48.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.720+0000 7f674d600700 1 -- 172.21.15.151:0/2295531743 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67480052c0 con 0x563674e2dad0 2024-05-21T22:04:48.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.720+0000 7f674d600700 1 -- 172.21.15.151:0/2295531743 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6748005460 con 0x563674e2dad0 2024-05-21T22:04:48.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.720+0000 7f674d600700 1 -- 172.21.15.151:0/2295531743 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6748031420 con 0x563674e2dad0 2024-05-21T22:04:48.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.721+0000 7f676dd50b80 1 -- 172.21.15.151:0/2295531743 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563674e2dad0 msgr2=0x563674e2d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:04:48.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.721+0000 7f676dd50b80 1 --2- 172.21.15.151:0/2295531743 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563674e2dad0 0x563674e2d4b0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x563674e49c90 tx=0x7f6748003fb0 comp rx=0 tx=0).stop 2024-05-21T22:04:48.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.721+0000 7f676dd50b80 1 -- 172.21.15.151:0/2295531743 shutdown_connections 2024-05-21T22:04:48.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.721+0000 7f676dd50b80 1 --2- 172.21.15.151:0/2295531743 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563674e33940 0x563674e352b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:48.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.721+0000 7f676dd50b80 1 --2- 172.21.15.151:0/2295531743 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563674e2dad0 0x563674e2d4b0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:48.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.721+0000 7f676dd50b80 1 --2- 172.21.15.151:0/2295531743 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563674e33020 0x563674e33400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:48.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.721+0000 7f676dd50b80 1 -- 172.21.15.151:0/2295531743 >> 172.21.15.151:0/2295531743 conn(0x563674d5bbe0 msgr2=0x563674e36980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:04:48.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.721+0000 7f676dd50b80 1 -- 172.21.15.151:0/2295531743 shutdown_connections 2024-05-21T22:04:48.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.721+0000 7f676dd50b80 1 -- 172.21.15.151:0/2295531743 wait complete. 2024-05-21T22:04:48.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.721+0000 7f676dd50b80 1 -- start start 2024-05-21T22:04:48.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.722+0000 7f676dd50b80 1 Processor -- start 2024-05-21T22:04:48.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.722+0000 7f676dd50b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563674e2dad0 0x563674e3f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:48.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.722+0000 7f676dd50b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563674e33020 0x563674e3fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:48.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.722+0000 7f676dd50b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563674e33940 0x563674e4dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:48.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.722+0000 7f676dd50b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f675c1b56d0 con 0x563674e33020 2024-05-21T22:04:48.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.722+0000 7f676dd50b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f675c1b5000 con 0x563674e2dad0 2024-05-21T22:04:48.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.722+0000 7f676dd50b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f675c1b2220 con 0x563674e33940 2024-05-21T22:04:48.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.722+0000 7f676dd50b80 10 client.0 _ll_register_callbacks cb 0x563674d25c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:04:48.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.722+0000 7f6760a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563674e33020 0x563674e3fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:48.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.722+0000 7f6760a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563674e33020 0x563674e3fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:33674/0 (socket says 172.21.15.151:33674) 2024-05-21T22:04:48.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.722+0000 7f6760a00700 1 -- 172.21.15.151:0/2552834202 learned_addr learned my addr 172.21.15.151:0/2552834202 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:04:48.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.723+0000 7f675be00700 1 --2- 172.21.15.151:0/2552834202 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563674e2dad0 0x563674e3f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:48.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.723+0000 7f675be00700 1 --2- 172.21.15.151:0/2552834202 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563674e33940 0x563674e4dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:48.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.723+0000 7f6760a00700 1 -- 172.21.15.151:0/2552834202 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563674e33940 msgr2=0x563674e4dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:04:48.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.723+0000 7f6760a00700 1 --2- 172.21.15.151:0/2552834202 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563674e33940 0x563674e4dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:48.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.723+0000 7f6760a00700 1 -- 172.21.15.151:0/2552834202 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563674e2dad0 msgr2=0x563674e3f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:04:48.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.723+0000 7f6760a00700 1 --2- 172.21.15.151:0/2552834202 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563674e2dad0 0x563674e3f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:48.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.723+0000 7f6760a00700 1 -- 172.21.15.151:0/2552834202 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563674e22650 con 0x563674e33020 2024-05-21T22:04:48.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.723+0000 7f6760a00700 1 --2- 172.21.15.151:0/2552834202 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563674e33020 0x563674e3fc60 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f675001c790 tx=0x7f675001c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=186d1c1f64ee9a50 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:04:48.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.723+0000 7f6742a00700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-21T22:04:48.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.724+0000 7f6742a00700 1 -- 172.21.15.151:0/2552834202 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f675000fe10 con 0x563674e33020 2024-05-21T22:04:48.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.724+0000 7f676dd50b80 1 -- 172.21.15.151:0/2552834202 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6748005460 con 0x563674e33020 2024-05-21T22:04:48.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.724+0000 7f6742a00700 1 -- 172.21.15.151:0/2552834202 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7f675001c910 con 0x563674e33020 2024-05-21T22:04:48.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.724+0000 7f6742a00700 1 -- 172.21.15.151:0/2552834202 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f675001cb10 con 0x563674e33020 2024-05-21T22:04:48.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.724+0000 7f676dd50b80 10 client.17205 fetch_fsmap learned FSMap version 29 2024-05-21T22:04:48.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.724+0000 7f676dd50b80 1 -- 172.21.15.151:0/2552834202 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563674eb11c0 con 0x563674e33020 2024-05-21T22:04:48.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.724+0000 7f6742a00700 1 -- 172.21.15.151:0/2552834202 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6748005460 con 0x563674e33020 2024-05-21T22:04:48.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.724+0000 7f6742a00700 1 -- 172.21.15.151:0/2552834202 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f6750003680 con 0x563674e33020 2024-05-21T22:04:48.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.724+0000 7f676dd50b80 10 client.17205 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:04:48.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.724+0000 7f676dd50b80 10 client.17205 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-05-21T22:04:48.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.724+0000 7f676dd50b80 20 client.17205 populate_metadata read hostname 'smithi151' 2024-05-21T22:04:48.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.725+0000 7f676dd50b80 1 --2- 172.21.15.151:0/2552834202 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x563674eb18a0 0x563674eb3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:48.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.725+0000 7f676dd50b80 4 client.17205 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T22:04:48.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.725+0000 7f676dd50b80 1 -- 172.21.15.151:0/2552834202 --> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563674d87570 con 0x563674eb18a0 2024-05-21T22:04:48.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.725+0000 7f675be00700 1 --2- 172.21.15.151:0/2552834202 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x563674eb18a0 0x563674eb3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:48.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.725+0000 7f675be00700 1 --2- 172.21.15.151:0/2552834202 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x563674eb18a0 0x563674eb3cc0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6f8861850754fa18 server_cookie=7decd4a09745a2f4 in_seq=0 out_seq=0 2024-05-21T22:04:48.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.725+0000 7f6742a00700 10 client.17205 ms_handle_connect on v2:172.21.15.164:6834/2340428470 2024-05-21T22:04:48.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.726+0000 7f6742a00700 1 -- 172.21.15.151:0/2552834202 <== mds.4 v2:172.21.15.164:6834/2340428470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563674d87570 con 0x563674eb18a0 2024-05-21T22:04:48.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.726+0000 7f6742a00700 10 client.17205 handle_command_reply: tid=0 2024-05-21T22:04:48.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.726+0000 7f676dd50b80 1 client.17205 shutdown 2024-05-21T22:04:48.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.726+0000 7f676dd50b80 1 -- 172.21.15.151:0/2552834202 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563674e33020 msgr2=0x563674e3fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:04:48.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.726+0000 7f676dd50b80 1 --2- 172.21.15.151:0/2552834202 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563674e33020 0x563674e3fc60 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f675001c790 tx=0x7f675001c7c0 comp rx=0 tx=0).stop 2024-05-21T22:04:48.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.726+0000 7f676dd50b80 1 -- 172.21.15.151:0/2552834202 shutdown_connections 2024-05-21T22:04:48.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.726+0000 7f676dd50b80 1 --2- 172.21.15.151:0/2552834202 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x563674eb18a0 0x563674eb3cc0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:48.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.726+0000 7f676dd50b80 1 --2- 172.21.15.151:0/2552834202 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563674e2dad0 0x563674e3f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:48.731 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.726+0000 7f676dd50b80 1 --2- 172.21.15.151:0/2552834202 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563674e33940 0x563674e4dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:48.731 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.726+0000 7f676dd50b80 1 --2- 172.21.15.151:0/2552834202 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563674e33020 0x563674e3fc60 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:48.731 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.726+0000 7f676dd50b80 1 -- 172.21.15.151:0/2552834202 >> 172.21.15.151:0/2552834202 conn(0x563674d5bbe0 msgr2=0x563674e27c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:04:48.731 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.726+0000 7f6742a00700 0 client.17205 ms_handle_reset on v2:172.21.15.164:6834/2340428470 2024-05-21T22:04:48.731 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.726+0000 7f676dd50b80 1 -- 172.21.15.151:0/2552834202 shutdown_connections 2024-05-21T22:04:48.731 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.726+0000 7f676dd50b80 1 -- 172.21.15.151:0/2552834202 wait complete. 2024-05-21T22:04:48.731 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.726+0000 7f676dd50b80 20 client.17205 trim_cache size 0 max 16384 2024-05-21T22:04:48.731 INFO:teuthology.orchestra.run.smithi151.stdout:[] 2024-05-21T22:04:48.732 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.727+0000 7f6761e00700 1 -- 172.21.15.151:0/2039680758 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f67440b2a70 msgr2=0x7f67440b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:04:48.732 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.727+0000 7f6761e00700 1 --2- 172.21.15.151:0/2039680758 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f67440b2a70 0x7f67440b4f30 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f67540aac10 tx=0x7f6754009920 comp rx=0 tx=0).stop 2024-05-21T22:04:48.732 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.727+0000 7f6761e00700 1 -- 172.21.15.151:0/2039680758 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f675c10e950 msgr2=0x7f675c1a49d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:04:48.732 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.727+0000 7f6761e00700 1 --2- 172.21.15.151:0/2039680758 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f675c10e950 0x7f675c1a49d0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f6754009c10 tx=0x7f675400cca0 comp rx=0 tx=0).stop 2024-05-21T22:04:48.732 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.727+0000 7f6761e00700 1 -- 172.21.15.151:0/2039680758 shutdown_connections 2024-05-21T22:04:48.732 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.727+0000 7f6761e00700 1 --2- 172.21.15.151:0/2039680758 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f67440b2a70 0x7f67440b4f30 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:48.732 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.727+0000 7f6761e00700 1 --2- 172.21.15.151:0/2039680758 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f675c10e950 0x7f675c1a49d0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:48.733 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.727+0000 7f6761e00700 1 --2- 172.21.15.151:0/2039680758 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f675c109a80 0x7f675c1a0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:48.733 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.727+0000 7f6761e00700 1 --2- 172.21.15.151:0/2039680758 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f675c10a440 0x7f675c1a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:48.733 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.727+0000 7f6761e00700 1 -- 172.21.15.151:0/2039680758 >> 172.21.15.151:0/2039680758 conn(0x7f675c0792d0 msgr2=0x7f675c079930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:04:48.733 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.727+0000 7f6761e00700 1 -- 172.21.15.151:0/2039680758 shutdown_connections 2024-05-21T22:04:48.733 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:48.727+0000 7f6761e00700 1 -- 172.21.15.151:0/2039680758 wait complete. 2024-05-21T22:04:49.729 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-21T22:04:49.809 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:04:49 smithi151 ceph-mon[31623]: pgmap v285: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T22:04:49.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:04:49 smithi184 ceph-mon[36942]: pgmap v285: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T22:04:50.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:04:49 smithi164 ceph-mon[36269]: pgmap v285: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T22:04:50.203 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:04:51.912 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:04:51 smithi151 ceph-mon[31623]: pgmap v286: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:04:51.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:04:51 smithi184 ceph-mon[36942]: pgmap v286: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:04:52.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:04:51 smithi164 ceph-mon[36269]: pgmap v286: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:04:52.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.383+0000 7fd00d000700 1 -- 172.21.15.151:0/2514127895 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd0081037f0 msgr2=0x7fd008103bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:04:52.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.383+0000 7fd00d000700 1 --2- 172.21.15.151:0/2514127895 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd0081037f0 0x7fd008103bd0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fcff80099c0 tx=0x7fcff8009cd0 comp rx=0 tx=0).stop 2024-05-21T22:04:52.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.384+0000 7fd00d000700 1 -- 172.21.15.151:0/2514127895 shutdown_connections 2024-05-21T22:04:52.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.384+0000 7fd00d000700 1 --2- 172.21.15.151:0/2514127895 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0081088f0 0x7fd008110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:52.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.384+0000 7fd00d000700 1 --2- 172.21.15.151:0/2514127895 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd0081037f0 0x7fd008103bd0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:52.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.384+0000 7fd00d000700 1 --2- 172.21.15.151:0/2514127895 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0081041a0 0x7fd0081081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:52.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.384+0000 7fd00d000700 1 -- 172.21.15.151:0/2514127895 >> 172.21.15.151:0/2514127895 conn(0x7fd00807a6a0 msgr2=0x7fd00807aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:04:52.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.384+0000 7fd00d000700 1 -- 172.21.15.151:0/2514127895 shutdown_connections 2024-05-21T22:04:52.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.384+0000 7fd00d000700 1 -- 172.21.15.151:0/2514127895 wait complete. 2024-05-21T22:04:52.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.385+0000 7fd00d000700 1 Processor -- start 2024-05-21T22:04:52.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.385+0000 7fd00d000700 1 -- start start 2024-05-21T22:04:52.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.385+0000 7fd00d000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd0081037f0 0x7fd00819bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:52.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.385+0000 7fd00d000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0081041a0 0x7fd00819c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:52.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.385+0000 7fd00d000700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0081088f0 0x7fd0081a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:52.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.385+0000 7fd00d000700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd008113710 con 0x7fd0081037f0 2024-05-21T22:04:52.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.385+0000 7fd00d000700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd008113590 con 0x7fd0081041a0 2024-05-21T22:04:52.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.385+0000 7fd00d000700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd008113890 con 0x7fd0081088f0 2024-05-21T22:04:52.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.386+0000 7fd005e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd0081037f0 0x7fd00819bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:52.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.386+0000 7fd005e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd0081037f0 0x7fd00819bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:33700/0 (socket says 172.21.15.151:33700) 2024-05-21T22:04:52.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.386+0000 7fd005e00700 1 -- 172.21.15.151:0/1877286613 learned_addr learned my addr 172.21.15.151:0/1877286613 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:04:52.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.386+0000 7fd005400700 1 --2- 172.21.15.151:0/1877286613 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0081041a0 0x7fd00819c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:52.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.386+0000 7fd006800700 1 --2- 172.21.15.151:0/1877286613 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0081088f0 0x7fd0081a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:52.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.386+0000 7fd005e00700 1 -- 172.21.15.151:0/1877286613 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0081088f0 msgr2=0x7fd0081a06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:04:52.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.386+0000 7fd005e00700 1 --2- 172.21.15.151:0/1877286613 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0081088f0 0x7fd0081a06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:52.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.386+0000 7fd005e00700 1 -- 172.21.15.151:0/1877286613 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0081041a0 msgr2=0x7fd00819c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:04:52.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.386+0000 7fd005e00700 1 --2- 172.21.15.151:0/1877286613 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0081041a0 0x7fd00819c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:52.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.386+0000 7fd005e00700 1 -- 172.21.15.151:0/1877286613 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0081a0de0 con 0x7fd0081037f0 2024-05-21T22:04:52.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.386+0000 7fd005400700 1 --2- 172.21.15.151:0/1877286613 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0081041a0 0x7fd00819c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:04:52.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.387+0000 7fd005e00700 1 --2- 172.21.15.151:0/1877286613 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd0081037f0 0x7fd00819bdd0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fcff8006010 tx=0x7fcff8005280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:04:52.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.387+0000 7fcffea00700 1 -- 172.21.15.151:0/1877286613 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff8026070 con 0x7fd0081037f0 2024-05-21T22:04:52.389 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.387+0000 7fcffea00700 1 -- 172.21.15.151:0/1877286613 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcff8003c70 con 0x7fd0081037f0 2024-05-21T22:04:52.389 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.387+0000 7fd00d000700 1 -- 172.21.15.151:0/1877286613 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0081a1070 con 0x7fd0081037f0 2024-05-21T22:04:52.389 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.387+0000 7fcffea00700 1 -- 172.21.15.151:0/1877286613 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff800fad0 con 0x7fd0081037f0 2024-05-21T22:04:52.389 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.387+0000 7fd00d000700 1 -- 172.21.15.151:0/1877286613 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0081a8950 con 0x7fd0081037f0 2024-05-21T22:04:52.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.391+0000 7fcffea00700 1 -- 172.21.15.151:0/1877286613 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7fcff8029070 con 0x7fd0081037f0 2024-05-21T22:04:52.392 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.392+0000 7fd01797eb80 1 Processor -- start 2024-05-21T22:04:52.392 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.392+0000 7fd01797eb80 1 -- start start 2024-05-21T22:04:52.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.392+0000 7fd01797eb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1a511c610 0x55d1a511c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:52.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.393+0000 7fd01797eb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d1a511cf30 0x55d1a5116b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:52.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.393+0000 7fd01797eb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1a5117140 0x55d1a51175c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:52.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.393+0000 7fd01797eb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d1a5074e40 con 0x55d1a511c610 2024-05-21T22:04:52.394 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.393+0000 7fd01797eb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55d1a50649e0 con 0x55d1a5117140 2024-05-21T22:04:52.394 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.393+0000 7fd01797eb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55d1a5068a20 con 0x55d1a511cf30 2024-05-21T22:04:52.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.393+0000 7fcffea00700 1 --2- 172.21.15.151:0/1877286613 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fcfec0b29f0 0x7fcfec0b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:52.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.393+0000 7fd005e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d1a511cf30 0x55d1a5116b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:52.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.393+0000 7fd005400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1a5117140 0x55d1a51175c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:52.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.393+0000 7fd005e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d1a511cf30 0x55d1a5116b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:43224/0 (socket says 172.21.15.151:43224) 2024-05-21T22:04:52.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.393+0000 7fd005e00700 1 -- 172.21.15.151:0/993273243 learned_addr learned my addr 172.21.15.151:0/993273243 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:04:52.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.393+0000 7fd006800700 1 --2- 172.21.15.151:0/993273243 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1a511c610 0x55d1a511c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:52.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.393+0000 7fd006800700 1 --2- 172.21.15.151:0/1877286613 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fcfec0b29f0 0x7fcfec0b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:52.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.393+0000 7fcffea00700 1 -- 172.21.15.151:0/1877286613 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7fcff80e2e40 con 0x7fd0081037f0 2024-05-21T22:04:52.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.393+0000 7fd005400700 1 -- 172.21.15.151:0/993273243 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d1a511cf30 msgr2=0x55d1a5116b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:04:52.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.393+0000 7fd005400700 1 --2- 172.21.15.151:0/993273243 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d1a511cf30 0x55d1a5116b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:52.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.393+0000 7fd005400700 1 -- 172.21.15.151:0/993273243 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1a511c610 msgr2=0x55d1a511c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:04:52.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.393+0000 7fd005400700 1 --2- 172.21.15.151:0/993273243 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1a511c610 0x55d1a511c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:52.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.393+0000 7fd005400700 1 -- 172.21.15.151:0/993273243 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1a502d420 con 0x55d1a5117140 2024-05-21T22:04:52.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.394+0000 7fd005e00700 1 --2- 172.21.15.151:0/993273243 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d1a511cf30 0x55d1a5116b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:04:52.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.394+0000 7fd006800700 1 --2- 172.21.15.151:0/993273243 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1a511c610 0x55d1a511c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T22:04:52.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.394+0000 7fd005400700 1 --2- 172.21.15.151:0/993273243 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1a5117140 0x55d1a51175c0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fcff40114c0 tx=0x7fcff40117d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e845202a837af3c server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:04:52.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.394+0000 7fcff3400700 1 -- 172.21.15.151:0/993273243 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff4004220 con 0x55d1a5117140 2024-05-21T22:04:52.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.394+0000 7fcff3400700 1 -- 172.21.15.151:0/993273243 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcff4011e50 con 0x55d1a5117140 2024-05-21T22:04:52.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.394+0000 7fcff3400700 1 -- 172.21.15.151:0/993273243 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff40054d0 con 0x55d1a5117140 2024-05-21T22:04:52.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.395+0000 7fd01797eb80 1 -- 172.21.15.151:0/993273243 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1a5117140 msgr2=0x55d1a51175c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:04:52.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.395+0000 7fd006800700 1 --2- 172.21.15.151:0/1877286613 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fcfec0b29f0 0x7fcfec0b4eb0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd000009da0 tx=0x7fd00000b040 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:04:52.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.395+0000 7fd01797eb80 1 --2- 172.21.15.151:0/993273243 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1a5117140 0x55d1a51175c0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fcff40114c0 tx=0x7fcff40117d0 comp rx=0 tx=0).stop 2024-05-21T22:04:52.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.395+0000 7fd01797eb80 1 -- 172.21.15.151:0/993273243 shutdown_connections 2024-05-21T22:04:52.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.395+0000 7fd01797eb80 1 --2- 172.21.15.151:0/993273243 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1a5117140 0x55d1a51175c0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:52.398 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.395+0000 7fd01797eb80 1 --2- 172.21.15.151:0/993273243 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1a511c610 0x55d1a511c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:52.398 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.395+0000 7fd01797eb80 1 --2- 172.21.15.151:0/993273243 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d1a511cf30 0x55d1a5116b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:52.398 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.395+0000 7fd01797eb80 1 -- 172.21.15.151:0/993273243 >> 172.21.15.151:0/993273243 conn(0x55d1a5044be0 msgr2=0x55d1a5111690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:04:52.398 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.395+0000 7fd01797eb80 1 -- 172.21.15.151:0/993273243 shutdown_connections 2024-05-21T22:04:52.398 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.395+0000 7fd01797eb80 1 -- 172.21.15.151:0/993273243 wait complete. 2024-05-21T22:04:52.398 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.396+0000 7fd01797eb80 1 -- start start 2024-05-21T22:04:52.398 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.396+0000 7fd01797eb80 1 Processor -- start 2024-05-21T22:04:52.398 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.397+0000 7fd01797eb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1a5117140 0x55d1a5122c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:52.398 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.397+0000 7fd01797eb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1a511c610 0x55d1a511ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:52.398 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.397+0000 7fd01797eb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d1a511cf30 0x55d1a51204e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:52.398 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.397+0000 7fd01797eb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d1a4f6afa0 con 0x55d1a5117140 2024-05-21T22:04:52.398 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.397+0000 7fd01797eb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55d1a5005760 con 0x55d1a511c610 2024-05-21T22:04:52.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.397+0000 7fd01797eb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55d1a51239e0 con 0x55d1a511cf30 2024-05-21T22:04:52.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.397+0000 7fd01797eb80 10 client.0 _ll_register_callbacks cb 0x55d1a500ec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:04:52.401 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.397+0000 7fd005e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1a5117140 0x55d1a5122c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:52.401 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.397+0000 7fd005e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1a5117140 0x55d1a5122c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:33714/0 (socket says 172.21.15.151:33714) 2024-05-21T22:04:52.401 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.397+0000 7fd005e00700 1 -- 172.21.15.151:0/534008993 learned_addr learned my addr 172.21.15.151:0/534008993 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:04:52.401 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.397+0000 7fd005400700 1 --2- 172.21.15.151:0/534008993 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1a511c610 0x55d1a511ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:52.401 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.397+0000 7fd006800700 1 --2- 172.21.15.151:0/534008993 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d1a511cf30 0x55d1a51204e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:52.401 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.398+0000 7fd005e00700 1 -- 172.21.15.151:0/534008993 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d1a511cf30 msgr2=0x55d1a51204e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:04:52.401 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.398+0000 7fd005e00700 1 --2- 172.21.15.151:0/534008993 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d1a511cf30 0x55d1a51204e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:52.401 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.398+0000 7fd005e00700 1 -- 172.21.15.151:0/534008993 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1a511c610 msgr2=0x55d1a511ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:04:52.401 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.398+0000 7fd005e00700 1 --2- 172.21.15.151:0/534008993 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1a511c610 0x55d1a511ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:52.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.398+0000 7fd005e00700 1 -- 172.21.15.151:0/534008993 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1a4fb48d0 con 0x55d1a5117140 2024-05-21T22:04:52.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.398+0000 7fd005400700 1 --2- 172.21.15.151:0/534008993 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1a511c610 0x55d1a511ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:04:52.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.398+0000 7fd005e00700 1 --2- 172.21.15.151:0/534008993 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1a5117140 0x55d1a5122c00 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fcff80a66e0 tx=0x7fcff80e0a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=940d46a797008653 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:04:52.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.398+0000 7fcfe7e00700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-21T22:04:52.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.398+0000 7fcfe7e00700 1 -- 172.21.15.151:0/534008993 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff80aa840 con 0x55d1a5117140 2024-05-21T22:04:52.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.398+0000 7fcfe7e00700 1 -- 172.21.15.151:0/534008993 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7fcff80aa5b0 con 0x55d1a5117140 2024-05-21T22:04:52.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.398+0000 7fd01797eb80 1 -- 172.21.15.151:0/534008993 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcff4011e50 con 0x55d1a5117140 2024-05-21T22:04:52.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.399+0000 7fcfe7e00700 1 -- 172.21.15.151:0/534008993 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcff8003e20 con 0x55d1a5117140 2024-05-21T22:04:52.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.399+0000 7fd01797eb80 10 client.17217 fetch_fsmap learned FSMap version 29 2024-05-21T22:04:52.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.399+0000 7fd01797eb80 1 -- 172.21.15.151:0/534008993 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d1a5035060 con 0x55d1a5117140 2024-05-21T22:04:52.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.399+0000 7fcfe7e00700 1 -- 172.21.15.151:0/534008993 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcff4011e50 con 0x55d1a5117140 2024-05-21T22:04:52.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.399+0000 7fcfe7e00700 1 -- 172.21.15.151:0/534008993 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fcff80ae140 con 0x55d1a5117140 2024-05-21T22:04:52.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.399+0000 7fd01797eb80 10 client.17217 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:04:52.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.399+0000 7fd01797eb80 10 client.17217 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-21T22:04:52.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.399+0000 7fd01797eb80 20 client.17217 populate_metadata read hostname 'smithi151' 2024-05-21T22:04:52.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.399+0000 7fd01797eb80 1 --2- 172.21.15.151:0/534008993 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x55d1a51303b0 0x55d1a5132800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:52.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.399+0000 7fd01797eb80 4 client.17217 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T22:04:52.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.399+0000 7fd01797eb80 1 -- 172.21.15.151:0/534008993 --> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d1a5076d90 con 0x55d1a51303b0 2024-05-21T22:04:52.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.400+0000 7fd005400700 1 --2- 172.21.15.151:0/534008993 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x55d1a51303b0 0x55d1a5132800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:52.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.400+0000 7fd005400700 1 --2- 172.21.15.151:0/534008993 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x55d1a51303b0 0x55d1a5132800 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=dd7b7d5bb6dbc445 server_cookie=50c90257b64ecd41 in_seq=0 out_seq=0 2024-05-21T22:04:52.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.400+0000 7fcfe7e00700 10 client.17217 ms_handle_connect on v2:172.21.15.184:6836/2988194888 2024-05-21T22:04:52.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.401+0000 7fcfe7e00700 1 -- 172.21.15.151:0/534008993 <== mds.3 v2:172.21.15.184:6836/2988194888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55d1a5076d90 con 0x55d1a51303b0 2024-05-21T22:04:52.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.401+0000 7fcfe7e00700 10 client.17217 handle_command_reply: tid=0 2024-05-21T22:04:52.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.401+0000 7fd01797eb80 1 client.17217 shutdown 2024-05-21T22:04:52.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.401+0000 7fd01797eb80 1 -- 172.21.15.151:0/534008993 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1a5117140 msgr2=0x55d1a5122c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:04:52.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.401+0000 7fd01797eb80 1 --2- 172.21.15.151:0/534008993 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1a5117140 0x55d1a5122c00 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fcff80a66e0 tx=0x7fcff80e0a80 comp rx=0 tx=0).stop 2024-05-21T22:04:52.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.401+0000 7fd01797eb80 1 -- 172.21.15.151:0/534008993 shutdown_connections 2024-05-21T22:04:52.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.401+0000 7fd01797eb80 1 --2- 172.21.15.151:0/534008993 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x55d1a51303b0 0x55d1a5132800 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:52.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.401+0000 7fd01797eb80 1 --2- 172.21.15.151:0/534008993 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d1a511cf30 0x55d1a51204e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:52.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.401+0000 7fd01797eb80 1 --2- 172.21.15.151:0/534008993 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1a5117140 0x55d1a5122c00 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:52.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.401+0000 7fd01797eb80 1 --2- 172.21.15.151:0/534008993 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1a511c610 0x55d1a511ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:52.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.401+0000 7fd01797eb80 1 -- 172.21.15.151:0/534008993 >> 172.21.15.151:0/534008993 conn(0x55d1a5044be0 msgr2=0x55d1a5110d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:04:52.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.402+0000 7fcfe7e00700 0 client.17217 ms_handle_reset on v2:172.21.15.184:6836/2988194888 2024-05-21T22:04:52.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.402+0000 7fd01797eb80 1 -- 172.21.15.151:0/534008993 shutdown_connections 2024-05-21T22:04:52.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.402+0000 7fd01797eb80 1 -- 172.21.15.151:0/534008993 wait complete. 2024-05-21T22:04:52.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.402+0000 7fd01797eb80 20 client.17217 trim_cache size 0 max 16384 2024-05-21T22:04:52.419 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.419+0000 7fd01797eb80 1 Processor -- start 2024-05-21T22:04:52.419 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.419+0000 7fd01797eb80 1 -- start start 2024-05-21T22:04:52.419 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.419+0000 7fd01797eb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1a5117140 0x55d1a5116b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:52.419 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.419+0000 7fd01797eb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1a511c610 0x55d1a511dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:52.420 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.419+0000 7fd01797eb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d1a511cf30 0x55d1a51203f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:52.420 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.419+0000 7fd01797eb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd0081abb20 con 0x55d1a5117140 2024-05-21T22:04:52.420 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.419+0000 7fd01797eb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd0081ad660 con 0x55d1a511c610 2024-05-21T22:04:52.420 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.419+0000 7fd01797eb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd0081ac1f0 con 0x55d1a511cf30 2024-05-21T22:04:52.421 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.419+0000 7fd005e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1a5117140 0x55d1a5116b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:52.421 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.419+0000 7fd005e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1a5117140 0x55d1a5116b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:33718/0 (socket says 172.21.15.151:33718) 2024-05-21T22:04:52.421 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.419+0000 7fd005e00700 1 -- 172.21.15.151:0/1930913585 learned_addr learned my addr 172.21.15.151:0/1930913585 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:04:52.421 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.419+0000 7fd005400700 1 --2- 172.21.15.151:0/1930913585 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1a511c610 0x55d1a511dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:52.421 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.420+0000 7fd006800700 1 --2- 172.21.15.151:0/1930913585 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d1a511cf30 0x55d1a51203f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:52.421 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.420+0000 7fd005e00700 1 -- 172.21.15.151:0/1930913585 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d1a511cf30 msgr2=0x55d1a51203f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:04:52.421 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.420+0000 7fd005e00700 1 --2- 172.21.15.151:0/1930913585 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d1a511cf30 0x55d1a51203f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:52.421 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.420+0000 7fd005e00700 1 -- 172.21.15.151:0/1930913585 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1a511c610 msgr2=0x55d1a511dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:04:52.422 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.420+0000 7fd005e00700 1 --2- 172.21.15.151:0/1930913585 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1a511c610 0x55d1a511dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:52.422 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.420+0000 7fd005e00700 1 -- 172.21.15.151:0/1930913585 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1a51130e0 con 0x55d1a5117140 2024-05-21T22:04:52.422 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.420+0000 7fd005e00700 1 --2- 172.21.15.151:0/1930913585 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1a5117140 0x55d1a5116b20 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fcff8083040 tx=0x7fcff8005870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea20f76d8c285072 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:04:52.422 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.420+0000 7fcff2a00700 1 -- 172.21.15.151:0/1930913585 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff80aa840 con 0x55d1a5117140 2024-05-21T22:04:52.422 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.420+0000 7fcff2a00700 1 -- 172.21.15.151:0/1930913585 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcff80aa140 con 0x55d1a5117140 2024-05-21T22:04:52.422 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.421+0000 7fcff2a00700 1 -- 172.21.15.151:0/1930913585 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff80ae2f0 con 0x55d1a5117140 2024-05-21T22:04:52.423 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.421+0000 7fd01797eb80 1 -- 172.21.15.151:0/1930913585 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1a5117140 msgr2=0x55d1a5116b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:04:52.424 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.421+0000 7fd01797eb80 1 --2- 172.21.15.151:0/1930913585 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1a5117140 0x55d1a5116b20 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fcff8083040 tx=0x7fcff8005870 comp rx=0 tx=0).stop 2024-05-21T22:04:52.424 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.421+0000 7fd01797eb80 1 -- 172.21.15.151:0/1930913585 shutdown_connections 2024-05-21T22:04:52.424 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.421+0000 7fd01797eb80 1 --2- 172.21.15.151:0/1930913585 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d1a511cf30 0x55d1a51203f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:52.424 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.421+0000 7fd01797eb80 1 --2- 172.21.15.151:0/1930913585 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1a5117140 0x55d1a5116b20 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:52.424 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.421+0000 7fd01797eb80 1 --2- 172.21.15.151:0/1930913585 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1a511c610 0x55d1a511dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:52.424 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.421+0000 7fd01797eb80 1 -- 172.21.15.151:0/1930913585 >> 172.21.15.151:0/1930913585 conn(0x55d1a5044be0 msgr2=0x55d1a511e890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:04:52.424 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.421+0000 7fd01797eb80 1 -- 172.21.15.151:0/1930913585 shutdown_connections 2024-05-21T22:04:52.424 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.421+0000 7fd01797eb80 1 -- 172.21.15.151:0/1930913585 wait complete. 2024-05-21T22:04:52.424 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.422+0000 7fd01797eb80 1 -- start start 2024-05-21T22:04:52.424 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.422+0000 7fd01797eb80 1 Processor -- start 2024-05-21T22:04:52.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.423+0000 7fd01797eb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1a5117140 0x55d1a5128c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:52.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.423+0000 7fd01797eb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1a511c610 0x55d1a51291b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:52.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.423+0000 7fd01797eb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d1a511cf30 0x55d1a5137120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:52.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.423+0000 7fd01797eb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd0081acf90 con 0x55d1a5117140 2024-05-21T22:04:52.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.423+0000 7fd01797eb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd0081ac8c0 con 0x55d1a511c610 2024-05-21T22:04:52.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.423+0000 7fd01797eb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd00810e560 con 0x55d1a511cf30 2024-05-21T22:04:52.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.423+0000 7fd01797eb80 10 client.0 _ll_register_callbacks cb 0x55d1a500ec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:04:52.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.423+0000 7fd005e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1a5117140 0x55d1a5128c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:52.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.423+0000 7fd006800700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d1a511cf30 0x55d1a5137120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:52.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.423+0000 7fd005400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1a511c610 0x55d1a51291b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:52.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.423+0000 7fd006800700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d1a511cf30 0x55d1a5137120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:43250/0 (socket says 172.21.15.151:43250) 2024-05-21T22:04:52.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.423+0000 7fd005e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1a5117140 0x55d1a5128c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:33732/0 (socket says 172.21.15.151:33732) 2024-05-21T22:04:52.426 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.423+0000 7fd006800700 1 -- 172.21.15.151:0/409335822 learned_addr learned my addr 172.21.15.151:0/409335822 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:04:52.426 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.423+0000 7fd005400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1a511c610 0x55d1a51291b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:33842/0 (socket says 172.21.15.151:33842) 2024-05-21T22:04:52.426 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.423+0000 7fd006800700 1 -- 172.21.15.151:0/409335822 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1a511c610 msgr2=0x55d1a51291b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:04:52.426 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.423+0000 7fd006800700 1 --2- 172.21.15.151:0/409335822 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1a511c610 0x55d1a51291b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:52.426 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.423+0000 7fd006800700 1 -- 172.21.15.151:0/409335822 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1a5117140 msgr2=0x55d1a5128c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:04:52.426 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.423+0000 7fd006800700 1 --2- 172.21.15.151:0/409335822 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1a5117140 0x55d1a5128c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:52.426 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.423+0000 7fd006800700 1 -- 172.21.15.151:0/409335822 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1a5137660 con 0x55d1a511cf30 2024-05-21T22:04:52.426 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.423+0000 7fd005400700 1 --2- 172.21.15.151:0/409335822 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1a511c610 0x55d1a51291b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:04:52.426 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.423+0000 7fd005e00700 1 --2- 172.21.15.151:0/409335822 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1a5117140 0x55d1a5128c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:04:52.428 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.424+0000 7fd006800700 1 --2- 172.21.15.151:0/409335822 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d1a511cf30 0x55d1a5137120 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fd000028870 tx=0x7fd000004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=796f952733992c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:04:52.428 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.424+0000 7fcfe7e00700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-21T22:04:52.428 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.424+0000 7fcfe7e00700 1 -- 172.21.15.151:0/409335822 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd000029900 con 0x55d1a511cf30 2024-05-21T22:04:52.428 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.424+0000 7fcfe7e00700 1 -- 172.21.15.151:0/409335822 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7fd000029aa0 con 0x55d1a511cf30 2024-05-21T22:04:52.428 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.424+0000 7fd01797eb80 1 -- 172.21.15.151:0/409335822 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcff80aa140 con 0x55d1a511cf30 2024-05-21T22:04:52.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.425+0000 7fcfe7e00700 1 -- 172.21.15.151:0/409335822 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd000029ca0 con 0x55d1a511cf30 2024-05-21T22:04:52.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.425+0000 7fd01797eb80 10 client.27349 fetch_fsmap learned FSMap version 29 2024-05-21T22:04:52.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.425+0000 7fd01797eb80 1 -- 172.21.15.151:0/409335822 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d1a519a840 con 0x55d1a511cf30 2024-05-21T22:04:52.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.425+0000 7fcfe7e00700 1 -- 172.21.15.151:0/409335822 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcff80aa140 con 0x55d1a511cf30 2024-05-21T22:04:52.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.425+0000 7fcfe7e00700 1 -- 172.21.15.151:0/409335822 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fd000027430 con 0x55d1a511cf30 2024-05-21T22:04:52.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.425+0000 7fd01797eb80 10 client.27349 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:04:52.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.425+0000 7fd01797eb80 10 client.27349 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-21T22:04:52.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.425+0000 7fd01797eb80 20 client.27349 populate_metadata read hostname 'smithi151' 2024-05-21T22:04:52.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.425+0000 7fd01797eb80 1 --2- 172.21.15.151:0/409335822 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x55d1a519af80 0x55d1a519d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:52.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.425+0000 7fd01797eb80 4 client.27349 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T22:04:52.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.425+0000 7fd01797eb80 1 -- 172.21.15.151:0/409335822 --> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d1a5070570 con 0x55d1a519af80 2024-05-21T22:04:52.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.426+0000 7fd005e00700 1 --2- 172.21.15.151:0/409335822 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x55d1a519af80 0x55d1a519d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:52.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.426+0000 7fd005e00700 1 --2- 172.21.15.151:0/409335822 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x55d1a519af80 0x55d1a519d3a0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=906effae8cee93a3 server_cookie=c2fc4e12f8e5a2fd in_seq=0 out_seq=0 2024-05-21T22:04:52.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.426+0000 7fcfe7e00700 10 client.27349 ms_handle_connect on v2:172.21.15.184:6836/2988194888 2024-05-21T22:04:52.430 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.426+0000 7fcfe7e00700 1 -- 172.21.15.151:0/409335822 <== mds.3 v2:172.21.15.184:6836/2988194888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d1a5070570 con 0x55d1a519af80 2024-05-21T22:04:52.430 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.426+0000 7fcfe7e00700 10 client.27349 handle_command_reply: tid=0 2024-05-21T22:04:52.430 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.426+0000 7fd01797eb80 1 client.27349 shutdown 2024-05-21T22:04:52.430 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.426+0000 7fd01797eb80 1 -- 172.21.15.151:0/409335822 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d1a511cf30 msgr2=0x55d1a5137120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:04:52.430 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.426+0000 7fd01797eb80 1 --2- 172.21.15.151:0/409335822 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d1a511cf30 0x55d1a5137120 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fd000028870 tx=0x7fd000004ad0 comp rx=0 tx=0).stop 2024-05-21T22:04:52.430 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.427+0000 7fd01797eb80 1 -- 172.21.15.151:0/409335822 shutdown_connections 2024-05-21T22:04:52.430 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.427+0000 7fd01797eb80 1 --2- 172.21.15.151:0/409335822 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x55d1a519af80 0x55d1a519d3a0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:52.430 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.427+0000 7fd01797eb80 1 --2- 172.21.15.151:0/409335822 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d1a511cf30 0x55d1a5137120 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:52.430 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.427+0000 7fd01797eb80 1 --2- 172.21.15.151:0/409335822 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1a5117140 0x55d1a5128c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:52.430 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.427+0000 7fcfe7e00700 0 client.27349 ms_handle_reset on v2:172.21.15.184:6836/2988194888 2024-05-21T22:04:52.430 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.427+0000 7fd01797eb80 1 --2- 172.21.15.151:0/409335822 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d1a511c610 0x55d1a51291b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:52.430 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.427+0000 7fd01797eb80 1 -- 172.21.15.151:0/409335822 >> 172.21.15.151:0/409335822 conn(0x55d1a5044be0 msgr2=0x55d1a511da20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:04:52.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.427+0000 7fd01797eb80 1 -- 172.21.15.151:0/409335822 shutdown_connections 2024-05-21T22:04:52.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.427+0000 7fd01797eb80 1 -- 172.21.15.151:0/409335822 wait complete. 2024-05-21T22:04:52.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.427+0000 7fd01797eb80 20 client.27349 trim_cache size 0 max 16384 2024-05-21T22:04:52.431 INFO:teuthology.orchestra.run.smithi151.stdout:[] 2024-05-21T22:04:52.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.428+0000 7fd00d000700 1 -- 172.21.15.151:0/1877286613 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fcfec0b29f0 msgr2=0x7fcfec0b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:04:52.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.428+0000 7fd00d000700 1 --2- 172.21.15.151:0/1877286613 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fcfec0b29f0 0x7fcfec0b4eb0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd000009da0 tx=0x7fd00000b040 comp rx=0 tx=0).stop 2024-05-21T22:04:52.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.428+0000 7fd00d000700 1 -- 172.21.15.151:0/1877286613 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd0081037f0 msgr2=0x7fd00819bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:04:52.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.428+0000 7fd00d000700 1 --2- 172.21.15.151:0/1877286613 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd0081037f0 0x7fd00819bdd0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fcff8006010 tx=0x7fcff8005280 comp rx=0 tx=0).stop 2024-05-21T22:04:52.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.428+0000 7fd00d000700 1 -- 172.21.15.151:0/1877286613 shutdown_connections 2024-05-21T22:04:52.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.428+0000 7fd00d000700 1 --2- 172.21.15.151:0/1877286613 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fcfec0b29f0 0x7fcfec0b4eb0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:52.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.428+0000 7fd00d000700 1 --2- 172.21.15.151:0/1877286613 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0081088f0 0x7fd0081a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:52.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.428+0000 7fd00d000700 1 --2- 172.21.15.151:0/1877286613 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd0081037f0 0x7fd00819bdd0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:52.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.428+0000 7fd00d000700 1 --2- 172.21.15.151:0/1877286613 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0081041a0 0x7fd00819c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:52.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.428+0000 7fd00d000700 1 -- 172.21.15.151:0/1877286613 >> 172.21.15.151:0/1877286613 conn(0x7fd00807a6a0 msgr2=0x7fd008101550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:04:52.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.428+0000 7fd00d000700 1 -- 172.21.15.151:0/1877286613 shutdown_connections 2024-05-21T22:04:52.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:52.428+0000 7fd00d000700 1 -- 172.21.15.151:0/1877286613 wait complete. 2024-05-21T22:04:53.231 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-21T22:04:53.680 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:04:53.829 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:04:53 smithi151 ceph-mon[31623]: pgmap v287: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T22:04:53.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:04:53 smithi184 ceph-mon[36942]: pgmap v287: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T22:04:54.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:04:53 smithi164 ceph-mon[36269]: pgmap v287: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T22:04:55.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.781+0000 7f79a1800700 1 -- 172.21.15.151:0/3316556387 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f799c077520 msgr2=0x7f799c075770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:04:55.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.781+0000 7f79a1800700 1 --2- 172.21.15.151:0/3316556387 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f799c077520 0x7f799c075770 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f798c009990 tx=0x7f798c009ca0 comp rx=0 tx=0).stop 2024-05-21T22:04:55.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.782+0000 7f79a1800700 1 -- 172.21.15.151:0/3316556387 shutdown_connections 2024-05-21T22:04:55.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.782+0000 7f79a1800700 1 --2- 172.21.15.151:0/3316556387 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f799c114240 0x7f799c116630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:55.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.782+0000 7f79a1800700 1 --2- 172.21.15.151:0/3316556387 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f799c077520 0x7f799c075770 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:55.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.782+0000 7f79a1800700 1 --2- 172.21.15.151:0/3316556387 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f799c075cb0 0x7f799c076130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:55.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.782+0000 7f79a1800700 1 -- 172.21.15.151:0/3316556387 >> 172.21.15.151:0/3316556387 conn(0x7f799c07a700 msgr2=0x7f799c07ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:04:55.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.782+0000 7f79a1800700 1 -- 172.21.15.151:0/3316556387 shutdown_connections 2024-05-21T22:04:55.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.782+0000 7f79a1800700 1 -- 172.21.15.151:0/3316556387 wait complete. 2024-05-21T22:04:55.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.783+0000 7f79a1800700 1 Processor -- start 2024-05-21T22:04:55.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.783+0000 7f79a1800700 1 -- start start 2024-05-21T22:04:55.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.783+0000 7f79a1800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f799c075cb0 0x7f799c1a02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:55.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.783+0000 7f79a1800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f799c077520 0x7f799c1a07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:55.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.783+0000 7f79a1800700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f799c114240 0x7f799c1a4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:55.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.783+0000 7f79a1800700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f799c119da0 con 0x7f799c075cb0 2024-05-21T22:04:55.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.783+0000 7f79a1800700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f799c119c20 con 0x7f799c077520 2024-05-21T22:04:55.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.784+0000 7f79a1800700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f799c119f20 con 0x7f799c114240 2024-05-21T22:04:55.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.784+0000 7f799a000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f799c077520 0x7f799c1a07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:55.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.784+0000 7f799aa00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f799c075cb0 0x7f799c1a02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:55.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.784+0000 7f799a000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f799c077520 0x7f799c1a07f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:33852/0 (socket says 172.21.15.151:33852) 2024-05-21T22:04:55.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.784+0000 7f799a000700 1 -- 172.21.15.151:0/1859548335 learned_addr learned my addr 172.21.15.151:0/1859548335 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:04:55.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.784+0000 7f799b400700 1 --2- 172.21.15.151:0/1859548335 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f799c114240 0x7f799c1a4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:55.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.784+0000 7f799a000700 1 -- 172.21.15.151:0/1859548335 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f799c114240 msgr2=0x7f799c1a4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:04:55.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.784+0000 7f799a000700 1 --2- 172.21.15.151:0/1859548335 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f799c114240 0x7f799c1a4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:55.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.784+0000 7f799a000700 1 -- 172.21.15.151:0/1859548335 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f799c075cb0 msgr2=0x7f799c1a02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:04:55.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.784+0000 7f799a000700 1 --2- 172.21.15.151:0/1859548335 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f799c075cb0 0x7f799c1a02b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:55.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.784+0000 7f799a000700 1 -- 172.21.15.151:0/1859548335 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f799c1a52c0 con 0x7f799c077520 2024-05-21T22:04:55.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.784+0000 7f799b400700 1 --2- 172.21.15.151:0/1859548335 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f799c114240 0x7f799c1a4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:04:55.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.785+0000 7f799a000700 1 --2- 172.21.15.151:0/1859548335 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f799c077520 0x7f799c1a07f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f798800e9c0 tx=0x7f798800ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:04:55.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.785+0000 7f7993400700 1 -- 172.21.15.151:0/1859548335 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f798800cbd0 con 0x7f799c077520 2024-05-21T22:04:55.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.785+0000 7f79a1800700 1 -- 172.21.15.151:0/1859548335 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f799c1a55b0 con 0x7f799c077520 2024-05-21T22:04:55.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.785+0000 7f7993400700 1 -- 172.21.15.151:0/1859548335 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f798800ee60 con 0x7f799c077520 2024-05-21T22:04:55.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.785+0000 7f79a1800700 1 -- 172.21.15.151:0/1859548335 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f799c1ace30 con 0x7f799c077520 2024-05-21T22:04:55.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.785+0000 7f7993400700 1 -- 172.21.15.151:0/1859548335 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7988005550 con 0x7f799c077520 2024-05-21T22:04:55.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.789+0000 7f7993400700 1 -- 172.21.15.151:0/1859548335 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7f7988004750 con 0x7f799c077520 2024-05-21T22:04:55.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.790+0000 7f79ac17fb80 1 Processor -- start 2024-05-21T22:04:55.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.790+0000 7f79ac17fb80 1 -- start start 2024-05-21T22:04:55.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.790+0000 7f79ac17fb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558df7685610 0x558df76859f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:55.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.790+0000 7f79ac17fb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558df7685f30 0x558df767fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:55.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.790+0000 7f79ac17fb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558df7680140 0x558df76805c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:55.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.790+0000 7f79ac17fb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x558df75dde40 con 0x558df7685f30 2024-05-21T22:04:55.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.790+0000 7f79ac17fb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x558df75cd9e0 con 0x558df7680140 2024-05-21T22:04:55.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.790+0000 7f79ac17fb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x558df75d1a20 con 0x558df7685610 2024-05-21T22:04:55.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.791+0000 7f799aa00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558df7685f30 0x558df767fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:55.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.791+0000 7f799aa00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558df7685f30 0x558df767fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:33758/0 (socket says 172.21.15.151:33758) 2024-05-21T22:04:55.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.791+0000 7f799aa00700 1 -- 172.21.15.151:0/1730912326 learned_addr learned my addr 172.21.15.151:0/1730912326 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:04:55.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.791+0000 7f799b400700 1 --2- 172.21.15.151:0/1730912326 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558df7685610 0x558df76859f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:55.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.791+0000 7f799a000700 1 --2- 172.21.15.151:0/1730912326 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558df7680140 0x558df76805c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:55.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.791+0000 7f799aa00700 1 -- 172.21.15.151:0/1730912326 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558df7685610 msgr2=0x558df76859f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:04:55.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.791+0000 7f799aa00700 1 --2- 172.21.15.151:0/1730912326 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558df7685610 0x558df76859f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:55.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.791+0000 7f799aa00700 1 -- 172.21.15.151:0/1730912326 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558df7680140 msgr2=0x558df76805c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:04:55.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.791+0000 7f799aa00700 1 --2- 172.21.15.151:0/1730912326 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558df7680140 0x558df76805c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:55.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.791+0000 7f799aa00700 1 -- 172.21.15.151:0/1730912326 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558df7596420 con 0x558df7685f30 2024-05-21T22:04:55.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.791+0000 7f799b400700 1 --2- 172.21.15.151:0/1730912326 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558df7685610 0x558df76859f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:04:55.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.791+0000 7f799a000700 1 --2- 172.21.15.151:0/1730912326 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558df7680140 0x558df76805c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T22:04:55.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.791+0000 7f799aa00700 1 --2- 172.21.15.151:0/1730912326 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558df7685f30 0x558df767fb20 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f798c0096a0 tx=0x7f798c0096d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adc78c2dd86ccebe server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:04:55.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.792+0000 7f7993400700 1 --2- 172.21.15.151:0/1859548335 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f79800b2a70 0x7f79800b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:55.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.792+0000 7f7993400700 1 -- 172.21.15.151:0/1859548335 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7f79880e2810 con 0x7f799c077520 2024-05-21T22:04:55.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.792+0000 7f7987e00700 1 -- 172.21.15.151:0/1730912326 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f798c027070 con 0x558df7685f30 2024-05-21T22:04:55.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.792+0000 7f7987e00700 1 -- 172.21.15.151:0/1730912326 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f798c0056d0 con 0x558df7685f30 2024-05-21T22:04:55.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.792+0000 7f7987e00700 1 -- 172.21.15.151:0/1730912326 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f798c003be0 con 0x558df7685f30 2024-05-21T22:04:55.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.792+0000 7f799b400700 1 --2- 172.21.15.151:0/1859548335 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f79800b2a70 0x7f79800b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:55.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.793+0000 7f79ac17fb80 1 -- 172.21.15.151:0/1730912326 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558df7685f30 msgr2=0x558df767fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:04:55.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.793+0000 7f79ac17fb80 1 --2- 172.21.15.151:0/1730912326 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558df7685f30 0x558df767fb20 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f798c0096a0 tx=0x7f798c0096d0 comp rx=0 tx=0).stop 2024-05-21T22:04:55.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.793+0000 7f799b400700 1 --2- 172.21.15.151:0/1859548335 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f79800b2a70 0x7f79800b4f30 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7994003fd0 tx=0x7f799400b040 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:04:55.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.793+0000 7f79ac17fb80 1 -- 172.21.15.151:0/1730912326 shutdown_connections 2024-05-21T22:04:55.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.793+0000 7f79ac17fb80 1 --2- 172.21.15.151:0/1730912326 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558df7680140 0x558df76805c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:55.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.793+0000 7f79ac17fb80 1 --2- 172.21.15.151:0/1730912326 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558df7685610 0x558df76859f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:55.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.793+0000 7f79ac17fb80 1 --2- 172.21.15.151:0/1730912326 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558df7685f30 0x558df767fb20 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:55.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.793+0000 7f79ac17fb80 1 -- 172.21.15.151:0/1730912326 >> 172.21.15.151:0/1730912326 conn(0x558df75adbe0 msgr2=0x558df767a690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:04:55.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.793+0000 7f79ac17fb80 1 -- 172.21.15.151:0/1730912326 shutdown_connections 2024-05-21T22:04:55.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.793+0000 7f79ac17fb80 1 -- 172.21.15.151:0/1730912326 wait complete. 2024-05-21T22:04:55.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.794+0000 7f79ac17fb80 1 -- start start 2024-05-21T22:04:55.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.794+0000 7f79ac17fb80 1 Processor -- start 2024-05-21T22:04:55.797 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.794+0000 7f79ac17fb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558df7680140 0x558df768bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:55.797 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.794+0000 7f79ac17fb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558df7685610 0x558df7688ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:55.797 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.794+0000 7f79ac17fb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558df7685f30 0x558df7689400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:55.797 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.794+0000 7f79ac17fb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x558df74d3fa0 con 0x558df7680140 2024-05-21T22:04:55.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.794+0000 7f79ac17fb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x558df756e760 con 0x558df7685f30 2024-05-21T22:04:55.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.794+0000 7f79ac17fb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x558df768c980 con 0x558df7685610 2024-05-21T22:04:55.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.795+0000 7f79ac17fb80 10 client.0 _ll_register_callbacks cb 0x558df7577c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:04:55.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.795+0000 7f799b400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558df7685f30 0x558df7689400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:55.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.795+0000 7f799aa00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558df7680140 0x558df768bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:55.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.795+0000 7f799b400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558df7685f30 0x558df7689400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:32936/0 (socket says 172.21.15.151:32936) 2024-05-21T22:04:55.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.795+0000 7f799a000700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558df7685610 0x558df7688ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:55.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.795+0000 7f799b400700 1 -- 172.21.15.151:0/3150199278 learned_addr learned my addr 172.21.15.151:0/3150199278 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:04:55.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.795+0000 7f799a000700 1 -- 172.21.15.151:0/3150199278 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558df7685f30 msgr2=0x558df7689400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:04:55.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.795+0000 7f799a000700 1 --2- 172.21.15.151:0/3150199278 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558df7685f30 0x558df7689400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:55.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.795+0000 7f799a000700 1 -- 172.21.15.151:0/3150199278 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558df7680140 msgr2=0x558df768bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:04:55.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.795+0000 7f799a000700 1 --2- 172.21.15.151:0/3150199278 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558df7680140 0x558df768bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:55.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.795+0000 7f799a000700 1 -- 172.21.15.151:0/3150199278 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558df751d8d0 con 0x558df7685610 2024-05-21T22:04:55.799 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.795+0000 7f799b400700 1 --2- 172.21.15.151:0/3150199278 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558df7685f30 0x558df7689400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:04:55.799 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.795+0000 7f799aa00700 1 --2- 172.21.15.151:0/3150199278 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558df7680140 0x558df768bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:04:55.799 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.796+0000 7f799a000700 1 --2- 172.21.15.151:0/3150199278 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558df7685610 0x558df7688ec0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f79880dfef0 tx=0x7f79880a9270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3397a40182fd1b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:04:55.799 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.796+0000 7f7984c00700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-21T22:04:55.799 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.796+0000 7f7984c00700 1 -- 172.21.15.151:0/3150199278 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79880aa180 con 0x558df7685610 2024-05-21T22:04:55.799 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.796+0000 7f79ac17fb80 1 -- 172.21.15.151:0/3150199278 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f798c0056d0 con 0x558df7685610 2024-05-21T22:04:55.799 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.796+0000 7f7984c00700 1 -- 172.21.15.151:0/3150199278 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7f79880a9ef0 con 0x558df7685610 2024-05-21T22:04:55.799 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.796+0000 7f7984c00700 1 -- 172.21.15.151:0/3150199278 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f798800cd70 con 0x558df7685610 2024-05-21T22:04:55.799 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.796+0000 7f79ac17fb80 10 client.27361 fetch_fsmap learned FSMap version 29 2024-05-21T22:04:55.799 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.796+0000 7f79ac17fb80 1 -- 172.21.15.151:0/3150199278 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558df759e060 con 0x558df7685610 2024-05-21T22:04:55.799 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.796+0000 7f7984c00700 1 -- 172.21.15.151:0/3150199278 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f798c0056d0 con 0x558df7685610 2024-05-21T22:04:55.799 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.797+0000 7f7984c00700 1 -- 172.21.15.151:0/3150199278 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f79880b2450 con 0x558df7685610 2024-05-21T22:04:55.799 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.797+0000 7f79ac17fb80 10 client.27361 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:04:55.799 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.797+0000 7f79ac17fb80 10 client.27361 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-05-21T22:04:55.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.797+0000 7f79ac17fb80 20 client.27361 populate_metadata read hostname 'smithi151' 2024-05-21T22:04:55.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.797+0000 7f79ac17fb80 1 --2- 172.21.15.151:0/3150199278 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x558df7699400 0x558df769b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:55.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.797+0000 7f79ac17fb80 4 client.27361 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T22:04:55.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.797+0000 7f79ac17fb80 1 -- 172.21.15.151:0/3150199278 --> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558df75dfd90 con 0x558df7699400 2024-05-21T22:04:55.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.797+0000 7f799aa00700 1 --2- 172.21.15.151:0/3150199278 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x558df7699400 0x558df769b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:55.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.798+0000 7f799aa00700 1 --2- 172.21.15.151:0/3150199278 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x558df7699400 0x558df769b7f0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1a9de25f7c7aab0f server_cookie=29d90fcdabd56820 in_seq=0 out_seq=0 2024-05-21T22:04:55.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.798+0000 7f7984c00700 10 client.27361 ms_handle_connect on v2:172.21.15.164:6836/1554609572 2024-05-21T22:04:55.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.799+0000 7f7984c00700 1 -- 172.21.15.151:0/3150199278 <== mds.2 v2:172.21.15.164:6836/1554609572 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x558df75dfd90 con 0x558df7699400 2024-05-21T22:04:55.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.799+0000 7f7984c00700 10 client.27361 handle_command_reply: tid=0 2024-05-21T22:04:55.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.799+0000 7f79ac17fb80 1 client.27361 shutdown 2024-05-21T22:04:55.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.799+0000 7f79ac17fb80 1 -- 172.21.15.151:0/3150199278 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558df7685610 msgr2=0x558df7688ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:04:55.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.799+0000 7f79ac17fb80 1 --2- 172.21.15.151:0/3150199278 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558df7685610 0x558df7688ec0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f79880dfef0 tx=0x7f79880a9270 comp rx=0 tx=0).stop 2024-05-21T22:04:55.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.799+0000 7f79ac17fb80 1 -- 172.21.15.151:0/3150199278 shutdown_connections 2024-05-21T22:04:55.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.799+0000 7f79ac17fb80 1 --2- 172.21.15.151:0/3150199278 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x558df7699400 0x558df769b7f0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:55.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.799+0000 7f79ac17fb80 1 --2- 172.21.15.151:0/3150199278 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558df7685f30 0x558df7689400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:55.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.799+0000 7f79ac17fb80 1 --2- 172.21.15.151:0/3150199278 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558df7680140 0x558df768bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:55.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.799+0000 7f7984c00700 0 client.27361 ms_handle_reset on v2:172.21.15.164:6836/1554609572 2024-05-21T22:04:55.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.799+0000 7f79ac17fb80 1 --2- 172.21.15.151:0/3150199278 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558df7685610 0x558df7688ec0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:55.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.799+0000 7f79ac17fb80 1 -- 172.21.15.151:0/3150199278 >> 172.21.15.151:0/3150199278 conn(0x558df75adbe0 msgr2=0x558df7679d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:04:55.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.799+0000 7f79ac17fb80 1 -- 172.21.15.151:0/3150199278 shutdown_connections 2024-05-21T22:04:55.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.800+0000 7f79ac17fb80 1 -- 172.21.15.151:0/3150199278 wait complete. 2024-05-21T22:04:55.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.800+0000 7f79ac17fb80 20 client.27361 trim_cache size 0 max 16384 2024-05-21T22:04:55.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.816+0000 7f79ac17fb80 1 Processor -- start 2024-05-21T22:04:55.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.816+0000 7f79ac17fb80 1 -- start start 2024-05-21T22:04:55.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.816+0000 7f79ac17fb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558df7680140 0x558df7686a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:55.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.816+0000 7f79ac17fb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558df7685610 0x558df767a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:55.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.816+0000 7f79ac17fb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558df7685f30 0x558df76a1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:55.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.816+0000 7f79ac17fb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f797c00d5c0 con 0x558df7680140 2024-05-21T22:04:55.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.816+0000 7f79ac17fb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f797c00f100 con 0x558df7685f30 2024-05-21T22:04:55.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.816+0000 7f79ac17fb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f797c00dc90 con 0x558df7685610 2024-05-21T22:04:55.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.817+0000 7f799aa00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558df7680140 0x558df7686a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:55.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.817+0000 7f799aa00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558df7680140 0x558df7686a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49706/0 (socket says 172.21.15.151:49706) 2024-05-21T22:04:55.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.817+0000 7f799aa00700 1 -- 172.21.15.151:0/2357230985 learned_addr learned my addr 172.21.15.151:0/2357230985 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:04:55.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.817+0000 7f799b400700 1 --2- 172.21.15.151:0/2357230985 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558df7685f30 0x558df76a1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:55.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.817+0000 7f799a000700 1 --2- 172.21.15.151:0/2357230985 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558df7685610 0x558df767a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:55.819 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.817+0000 7f799aa00700 1 -- 172.21.15.151:0/2357230985 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558df7685610 msgr2=0x558df767a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:04:55.819 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.817+0000 7f799aa00700 1 --2- 172.21.15.151:0/2357230985 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558df7685610 0x558df767a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:55.819 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.817+0000 7f799aa00700 1 -- 172.21.15.151:0/2357230985 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558df7685f30 msgr2=0x558df76a1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:04:55.819 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.817+0000 7f799aa00700 1 --2- 172.21.15.151:0/2357230985 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558df7685f30 0x558df76a1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:55.819 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.817+0000 7f799aa00700 1 -- 172.21.15.151:0/2357230985 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558df7687450 con 0x558df7680140 2024-05-21T22:04:55.819 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.818+0000 7f799aa00700 1 --2- 172.21.15.151:0/2357230985 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558df7680140 0x558df7686a20 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f798c0036f0 tx=0x7f798c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12329169a3075950 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:04:55.819 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.818+0000 7f7987400700 1 -- 172.21.15.151:0/2357230985 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f798c027070 con 0x558df7680140 2024-05-21T22:04:55.819 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.818+0000 7f7987400700 1 -- 172.21.15.151:0/2357230985 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f798c028040 con 0x558df7680140 2024-05-21T22:04:55.819 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.818+0000 7f7987400700 1 -- 172.21.15.151:0/2357230985 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f798c0342a0 con 0x558df7680140 2024-05-21T22:04:55.820 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.819+0000 7f79ac17fb80 1 -- 172.21.15.151:0/2357230985 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558df7680140 msgr2=0x558df7686a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:04:55.820 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.819+0000 7f79ac17fb80 1 --2- 172.21.15.151:0/2357230985 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558df7680140 0x558df7686a20 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f798c0036f0 tx=0x7f798c00b260 comp rx=0 tx=0).stop 2024-05-21T22:04:55.820 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.819+0000 7f79ac17fb80 1 -- 172.21.15.151:0/2357230985 shutdown_connections 2024-05-21T22:04:55.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.819+0000 7f79ac17fb80 1 --2- 172.21.15.151:0/2357230985 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558df7685f30 0x558df76a1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:55.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.819+0000 7f79ac17fb80 1 --2- 172.21.15.151:0/2357230985 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558df7680140 0x558df7686a20 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:55.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.819+0000 7f79ac17fb80 1 --2- 172.21.15.151:0/2357230985 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558df7685610 0x558df767a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:55.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.819+0000 7f79ac17fb80 1 -- 172.21.15.151:0/2357230985 >> 172.21.15.151:0/2357230985 conn(0x558df75adbe0 msgr2=0x558df7686310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:04:55.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.819+0000 7f79ac17fb80 1 -- 172.21.15.151:0/2357230985 shutdown_connections 2024-05-21T22:04:55.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.819+0000 7f79ac17fb80 1 -- 172.21.15.151:0/2357230985 wait complete. 2024-05-21T22:04:55.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.819+0000 7f79ac17fb80 1 -- start start 2024-05-21T22:04:55.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.820+0000 7f79ac17fb80 1 Processor -- start 2024-05-21T22:04:55.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.820+0000 7f79ac17fb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558df7680140 0x558df76a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:55.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.820+0000 7f79ac17fb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558df7685610 0x558df76a12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:55.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.820+0000 7f79ac17fb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558df7685f30 0x558df76a17e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:55.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.820+0000 7f79ac17fb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f797c00ea30 con 0x558df7685f30 2024-05-21T22:04:55.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.820+0000 7f79ac17fb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f797c00e360 con 0x558df7685610 2024-05-21T22:04:55.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.820+0000 7f79ac17fb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f797c00b580 con 0x558df7680140 2024-05-21T22:04:55.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.820+0000 7f79ac17fb80 10 client.0 _ll_register_callbacks cb 0x558df7577c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:04:55.822 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:04:55 smithi151 ceph-mon[31623]: pgmap v288: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:04:55.823 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.820+0000 7f799aa00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558df7680140 0x558df76a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:55.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.820+0000 7f799b400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558df7685f30 0x558df76a17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:55.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.820+0000 7f799aa00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558df7680140 0x558df76a0d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:51660/0 (socket says 172.21.15.151:51660) 2024-05-21T22:04:55.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.820+0000 7f799a000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558df7685610 0x558df76a12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:55.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.821+0000 7f799aa00700 1 -- 172.21.15.151:0/3001816838 learned_addr learned my addr 172.21.15.151:0/3001816838 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:04:55.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.821+0000 7f799aa00700 1 -- 172.21.15.151:0/3001816838 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558df7685610 msgr2=0x558df76a12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:04:55.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.821+0000 7f799aa00700 1 --2- 172.21.15.151:0/3001816838 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558df7685610 0x558df76a12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:55.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.821+0000 7f799aa00700 1 -- 172.21.15.151:0/3001816838 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558df7685f30 msgr2=0x558df76a17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:04:55.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.821+0000 7f799aa00700 1 --2- 172.21.15.151:0/3001816838 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558df7685f30 0x558df76a17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:55.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.821+0000 7f799aa00700 1 -- 172.21.15.151:0/3001816838 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558df7703130 con 0x558df7680140 2024-05-21T22:04:55.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.821+0000 7f799aa00700 1 --2- 172.21.15.151:0/3001816838 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558df7680140 0x558df76a0d60 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f798c009710 tx=0x7f798c0058e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cbcde8124749acbe server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:04:55.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.821+0000 7f7984c00700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-21T22:04:55.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.822+0000 7f7984c00700 1 -- 172.21.15.151:0/3001816838 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f798c027040 con 0x558df7680140 2024-05-21T22:04:55.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.822+0000 7f7984c00700 1 -- 172.21.15.151:0/3001816838 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7f798c00b920 con 0x558df7680140 2024-05-21T22:04:55.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.822+0000 7f79ac17fb80 1 -- 172.21.15.151:0/3001816838 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f798c028040 con 0x558df7680140 2024-05-21T22:04:55.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.822+0000 7f7984c00700 1 -- 172.21.15.151:0/3001816838 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f798c04c050 con 0x558df7680140 2024-05-21T22:04:55.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.822+0000 7f79ac17fb80 10 client.27367 fetch_fsmap learned FSMap version 29 2024-05-21T22:04:55.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.822+0000 7f79ac17fb80 1 -- 172.21.15.151:0/3001816838 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558df77043f0 con 0x558df7680140 2024-05-21T22:04:55.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.822+0000 7f7984c00700 1 -- 172.21.15.151:0/3001816838 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f798c028040 con 0x558df7680140 2024-05-21T22:04:55.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.822+0000 7f7984c00700 1 -- 172.21.15.151:0/3001816838 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f798c02d040 con 0x558df7680140 2024-05-21T22:04:55.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.823+0000 7f79ac17fb80 10 client.27367 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:04:55.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.823+0000 7f79ac17fb80 10 client.27367 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-05-21T22:04:55.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.823+0000 7f79ac17fb80 20 client.27367 populate_metadata read hostname 'smithi151' 2024-05-21T22:04:55.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.823+0000 7f79ac17fb80 1 --2- 172.21.15.151:0/3001816838 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x558df7704620 0x558df7706a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:55.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.823+0000 7f79ac17fb80 4 client.27367 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T22:04:55.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.823+0000 7f79ac17fb80 1 -- 172.21.15.151:0/3001816838 --> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558df75d9570 con 0x558df7704620 2024-05-21T22:04:55.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.823+0000 7f799a000700 1 --2- 172.21.15.151:0/3001816838 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x558df7704620 0x558df7706a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:55.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.824+0000 7f799a000700 1 --2- 172.21.15.151:0/3001816838 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x558df7704620 0x558df7706a10 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8d093dc47f49c49 server_cookie=6158b5d688b2ff9a in_seq=0 out_seq=0 2024-05-21T22:04:55.826 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.824+0000 7f7984c00700 10 client.27367 ms_handle_connect on v2:172.21.15.164:6836/1554609572 2024-05-21T22:04:55.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.824+0000 7f7984c00700 1 -- 172.21.15.151:0/3001816838 <== mds.2 v2:172.21.15.164:6836/1554609572 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558df75d9570 con 0x558df7704620 2024-05-21T22:04:55.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.824+0000 7f7984c00700 10 client.27367 handle_command_reply: tid=0 2024-05-21T22:04:55.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.824+0000 7f79ac17fb80 1 client.27367 shutdown 2024-05-21T22:04:55.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.824+0000 7f79ac17fb80 1 -- 172.21.15.151:0/3001816838 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558df7680140 msgr2=0x558df76a0d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:04:55.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.824+0000 7f79ac17fb80 1 --2- 172.21.15.151:0/3001816838 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558df7680140 0x558df76a0d60 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f798c009710 tx=0x7f798c0058e0 comp rx=0 tx=0).stop 2024-05-21T22:04:55.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.824+0000 7f79ac17fb80 1 -- 172.21.15.151:0/3001816838 shutdown_connections 2024-05-21T22:04:55.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.824+0000 7f79ac17fb80 1 --2- 172.21.15.151:0/3001816838 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x558df7704620 0x558df7706a10 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:55.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.824+0000 7f79ac17fb80 1 --2- 172.21.15.151:0/3001816838 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558df7685f30 0x558df76a17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:55.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.824+0000 7f7984c00700 0 client.27367 ms_handle_reset on v2:172.21.15.164:6836/1554609572 2024-05-21T22:04:55.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.824+0000 7f79ac17fb80 1 --2- 172.21.15.151:0/3001816838 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558df7680140 0x558df76a0d60 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:55.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.824+0000 7f79ac17fb80 1 --2- 172.21.15.151:0/3001816838 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x558df7685610 0x558df76a12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:55.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.824+0000 7f79ac17fb80 1 -- 172.21.15.151:0/3001816838 >> 172.21.15.151:0/3001816838 conn(0x558df75adbe0 msgr2=0x558df768a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:04:55.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.824+0000 7f79ac17fb80 1 -- 172.21.15.151:0/3001816838 shutdown_connections 2024-05-21T22:04:55.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.825+0000 7f79ac17fb80 1 -- 172.21.15.151:0/3001816838 wait complete. 2024-05-21T22:04:55.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.825+0000 7f79ac17fb80 20 client.27367 trim_cache size 0 max 16384 2024-05-21T22:04:55.828 INFO:teuthology.orchestra.run.smithi151.stdout:[] 2024-05-21T22:04:55.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.825+0000 7f79a1800700 1 -- 172.21.15.151:0/1859548335 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f79800b2a70 msgr2=0x7f79800b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:04:55.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.825+0000 7f79a1800700 1 --2- 172.21.15.151:0/1859548335 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f79800b2a70 0x7f79800b4f30 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7994003fd0 tx=0x7f799400b040 comp rx=0 tx=0).stop 2024-05-21T22:04:55.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.825+0000 7f79a1800700 1 -- 172.21.15.151:0/1859548335 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f799c077520 msgr2=0x7f799c1a07f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:04:55.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.825+0000 7f79a1800700 1 --2- 172.21.15.151:0/1859548335 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f799c077520 0x7f799c1a07f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f798800e9c0 tx=0x7f798800ecd0 comp rx=0 tx=0).stop 2024-05-21T22:04:55.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.826+0000 7f79a1800700 1 -- 172.21.15.151:0/1859548335 shutdown_connections 2024-05-21T22:04:55.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.826+0000 7f79a1800700 1 --2- 172.21.15.151:0/1859548335 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f79800b2a70 0x7f79800b4f30 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:55.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.826+0000 7f79a1800700 1 --2- 172.21.15.151:0/1859548335 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f799c114240 0x7f799c1a4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:55.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.826+0000 7f79a1800700 1 --2- 172.21.15.151:0/1859548335 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f799c075cb0 0x7f799c1a02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:55.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.826+0000 7f79a1800700 1 --2- 172.21.15.151:0/1859548335 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f799c077520 0x7f799c1a07f0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:55.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.826+0000 7f79a1800700 1 -- 172.21.15.151:0/1859548335 >> 172.21.15.151:0/1859548335 conn(0x7f799c07a700 msgr2=0x7f799c112230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:04:55.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.826+0000 7f79a1800700 1 -- 172.21.15.151:0/1859548335 shutdown_connections 2024-05-21T22:04:55.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:55.826+0000 7f79a1800700 1 -- 172.21.15.151:0/1859548335 wait complete. 2024-05-21T22:04:55.985 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:04:55 smithi184 ceph-mon[36942]: pgmap v288: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:04:56.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:04:55 smithi164 ceph-mon[36269]: pgmap v288: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:04:56.760 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-21T22:04:57.212 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:04:57.622 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:04:57 smithi151 ceph-mon[31623]: pgmap v289: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:04:57.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:04:57 smithi184 ceph-mon[36942]: pgmap v289: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:04:58.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:04:57 smithi164 ceph-mon[36269]: pgmap v289: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:04:59.423 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.421+0000 7f6722c00700 1 -- 172.21.15.151:0/2405439401 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f671c10eeb0 msgr2=0x7f671c116a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:04:59.423 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.421+0000 7f6722c00700 1 --2- 172.21.15.151:0/2405439401 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f671c10eeb0 0x7f671c116a80 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f671400b0d0 tx=0x7f671400b3e0 comp rx=0 tx=0).stop 2024-05-21T22:04:59.423 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.421+0000 7f6722c00700 1 -- 172.21.15.151:0/2405439401 shutdown_connections 2024-05-21T22:04:59.423 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.421+0000 7f6722c00700 1 --2- 172.21.15.151:0/2405439401 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f671c10eeb0 0x7f671c116a80 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:59.423 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.421+0000 7f6722c00700 1 --2- 172.21.15.151:0/2405439401 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f671c109dd0 0x7f671c10a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:59.423 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.421+0000 7f6722c00700 1 --2- 172.21.15.151:0/2405439401 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f671c10a780 0x7f671c10e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:59.423 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.421+0000 7f6722c00700 1 -- 172.21.15.151:0/2405439401 >> 172.21.15.151:0/2405439401 conn(0x7f671c076d10 msgr2=0x7f671c108360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:04:59.423 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.421+0000 7f6722c00700 1 -- 172.21.15.151:0/2405439401 shutdown_connections 2024-05-21T22:04:59.423 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.421+0000 7f6722c00700 1 -- 172.21.15.151:0/2405439401 wait complete. 2024-05-21T22:04:59.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.422+0000 7f6722c00700 1 Processor -- start 2024-05-21T22:04:59.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.422+0000 7f6722c00700 1 -- start start 2024-05-21T22:04:59.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.423+0000 7f6722c00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f671c109dd0 0x7f671c1a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:59.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.423+0000 7f6722c00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f671c10a780 0x7f671c1a0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:59.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.423+0000 7f6722c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f671c10eeb0 0x7f671c1a4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:59.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.423+0000 7f6722c00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f671c119d60 con 0x7f671c10eeb0 2024-05-21T22:04:59.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.423+0000 7f6722c00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f671c119be0 con 0x7f671c109dd0 2024-05-21T22:04:59.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.423+0000 7f6722c00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f671c119ee0 con 0x7f671c10a780 2024-05-21T22:04:59.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.423+0000 7f671b400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f671c10a780 0x7f671c1a0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:59.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.423+0000 7f6720c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f671c10eeb0 0x7f671c1a4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:59.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.423+0000 7f671be00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f671c109dd0 0x7f671c1a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:59.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.423+0000 7f671b400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f671c10a780 0x7f671c1a0730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:51682/0 (socket says 172.21.15.151:51682) 2024-05-21T22:04:59.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.423+0000 7f671b400700 1 -- 172.21.15.151:0/2891407996 learned_addr learned my addr 172.21.15.151:0/2891407996 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:04:59.426 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.424+0000 7f671b400700 1 -- 172.21.15.151:0/2891407996 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f671c109dd0 msgr2=0x7f671c1a01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:04:59.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.424+0000 7f671b400700 1 --2- 172.21.15.151:0/2891407996 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f671c109dd0 0x7f671c1a01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:59.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.424+0000 7f671b400700 1 -- 172.21.15.151:0/2891407996 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f671c10eeb0 msgr2=0x7f671c1a4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:04:59.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.424+0000 7f671b400700 1 --2- 172.21.15.151:0/2891407996 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f671c10eeb0 0x7f671c1a4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:59.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.424+0000 7f671b400700 1 -- 172.21.15.151:0/2891407996 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f671c1a5200 con 0x7f671c10a780 2024-05-21T22:04:59.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.424+0000 7f671b400700 1 --2- 172.21.15.151:0/2891407996 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f671c10a780 0x7f671c1a0730 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f670800e970 tx=0x7f670800ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:04:59.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.424+0000 7f6718c00700 1 -- 172.21.15.151:0/2891407996 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6708004a30 con 0x7f671c10a780 2024-05-21T22:04:59.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.424+0000 7f6718c00700 1 -- 172.21.15.151:0/2891407996 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f670800cc90 con 0x7f671c10a780 2024-05-21T22:04:59.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.424+0000 7f6722c00700 1 -- 172.21.15.151:0/2891407996 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f671c1a54f0 con 0x7f671c10a780 2024-05-21T22:04:59.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.425+0000 7f6722c00700 1 -- 172.21.15.151:0/2891407996 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f671c1acd70 con 0x7f671c10a780 2024-05-21T22:04:59.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.425+0000 7f6718c00700 1 -- 172.21.15.151:0/2891407996 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6708005610 con 0x7f671c10a780 2024-05-21T22:04:59.430 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.429+0000 7f6718c00700 1 -- 172.21.15.151:0/2891407996 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7f670800c7e0 con 0x7f671c10a780 2024-05-21T22:04:59.430 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.429+0000 7f672d4a3b80 1 Processor -- start 2024-05-21T22:04:59.430 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.429+0000 7f672d4a3b80 1 -- start start 2024-05-21T22:04:59.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.430+0000 7f672d4a3b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7cd614610 0x55e7cd6149f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:59.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.430+0000 7f672d4a3b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e7cd614f30 0x55e7cd60eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:59.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.430+0000 7f672d4a3b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e7cd60f140 0x55e7cd60f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:59.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.430+0000 7f672d4a3b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e7cd56ce40 con 0x55e7cd60f140 2024-05-21T22:04:59.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.430+0000 7f672d4a3b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55e7cd55c9e0 con 0x55e7cd614610 2024-05-21T22:04:59.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.430+0000 7f672d4a3b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55e7cd560a20 con 0x55e7cd614f30 2024-05-21T22:04:59.433 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.430+0000 7f671be00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e7cd614f30 0x55e7cd60eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:59.433 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.430+0000 7f671b400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e7cd60f140 0x55e7cd60f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:59.433 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.430+0000 7f6720c00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7cd614610 0x55e7cd6149f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:59.433 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.430+0000 7f671be00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e7cd614f30 0x55e7cd60eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:51692/0 (socket says 172.21.15.151:51692) 2024-05-21T22:04:59.433 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.430+0000 7f671be00700 1 -- 172.21.15.151:0/296701348 learned_addr learned my addr 172.21.15.151:0/296701348 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:04:59.433 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.431+0000 7f671be00700 1 -- 172.21.15.151:0/296701348 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7cd614610 msgr2=0x55e7cd6149f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:04:59.433 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.431+0000 7f671be00700 1 --2- 172.21.15.151:0/296701348 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7cd614610 0x55e7cd6149f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:59.433 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.431+0000 7f671be00700 1 -- 172.21.15.151:0/296701348 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e7cd60f140 msgr2=0x55e7cd60f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:04:59.433 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.431+0000 7f671be00700 1 --2- 172.21.15.151:0/296701348 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e7cd60f140 0x55e7cd60f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:59.433 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.431+0000 7f671be00700 1 -- 172.21.15.151:0/296701348 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7cd525420 con 0x55e7cd614f30 2024-05-21T22:04:59.433 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.431+0000 7f671b400700 1 --2- 172.21.15.151:0/296701348 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e7cd60f140 0x55e7cd60f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T22:04:59.434 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.431+0000 7f6720c00700 1 --2- 172.21.15.151:0/296701348 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7cd614610 0x55e7cd6149f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:04:59.434 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.431+0000 7f6718c00700 1 --2- 172.21.15.151:0/2891407996 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f67000b2a70 0x7f67000b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:59.434 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.431+0000 7f6718c00700 1 -- 172.21.15.151:0/2891407996 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7f67080e28c0 con 0x7f671c10a780 2024-05-21T22:04:59.434 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.431+0000 7f671be00700 1 --2- 172.21.15.151:0/296701348 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e7cd614f30 0x55e7cd60eb20 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f670c009d80 tx=0x7f670c011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff3aed04ee58bc9 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:04:59.434 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.431+0000 7f6720c00700 1 --2- 172.21.15.151:0/2891407996 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f67000b2a70 0x7f67000b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:59.434 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.431+0000 7f6711600700 1 -- 172.21.15.151:0/296701348 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f670c01e070 con 0x55e7cd614f30 2024-05-21T22:04:59.434 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.431+0000 7f6711600700 1 -- 172.21.15.151:0/296701348 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f670c019ae0 con 0x55e7cd614f30 2024-05-21T22:04:59.434 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.431+0000 7f6711600700 1 -- 172.21.15.151:0/296701348 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f670c00eac0 con 0x55e7cd614f30 2024-05-21T22:04:59.435 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.432+0000 7f672d4a3b80 1 -- 172.21.15.151:0/296701348 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e7cd614f30 msgr2=0x55e7cd60eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:04:59.435 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.432+0000 7f6720c00700 1 --2- 172.21.15.151:0/2891407996 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f67000b2a70 0x7f67000b4f30 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6714005bd0 tx=0x7f671400b040 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:04:59.436 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.432+0000 7f672d4a3b80 1 --2- 172.21.15.151:0/296701348 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e7cd614f30 0x55e7cd60eb20 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f670c009d80 tx=0x7f670c011720 comp rx=0 tx=0).stop 2024-05-21T22:04:59.436 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.432+0000 7f672d4a3b80 1 -- 172.21.15.151:0/296701348 shutdown_connections 2024-05-21T22:04:59.436 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.432+0000 7f672d4a3b80 1 --2- 172.21.15.151:0/296701348 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7cd614610 0x55e7cd6149f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:59.436 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.432+0000 7f672d4a3b80 1 --2- 172.21.15.151:0/296701348 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e7cd60f140 0x55e7cd60f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:59.436 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.432+0000 7f672d4a3b80 1 --2- 172.21.15.151:0/296701348 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e7cd614f30 0x55e7cd60eb20 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:59.436 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.432+0000 7f672d4a3b80 1 -- 172.21.15.151:0/296701348 >> 172.21.15.151:0/296701348 conn(0x55e7cd53cbe0 msgr2=0x55e7cd609690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:04:59.436 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.433+0000 7f672d4a3b80 1 -- 172.21.15.151:0/296701348 shutdown_connections 2024-05-21T22:04:59.436 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.433+0000 7f672d4a3b80 1 -- 172.21.15.151:0/296701348 wait complete. 2024-05-21T22:04:59.436 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.433+0000 7f672d4a3b80 1 -- start start 2024-05-21T22:04:59.436 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.433+0000 7f672d4a3b80 1 Processor -- start 2024-05-21T22:04:59.436 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.434+0000 7f672d4a3b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e7cd60f140 0x55e7cd61ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:59.436 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.434+0000 7f672d4a3b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7cd614610 0x55e7cd617fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:59.437 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.434+0000 7f672d4a3b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e7cd614f30 0x55e7cd618510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:59.437 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.434+0000 7f672d4a3b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e7cd462fa0 con 0x55e7cd614f30 2024-05-21T22:04:59.437 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.434+0000 7f672d4a3b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55e7cd4fd760 con 0x55e7cd614610 2024-05-21T22:04:59.437 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.434+0000 7f672d4a3b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55e7cd61b9e0 con 0x55e7cd60f140 2024-05-21T22:04:59.437 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.434+0000 7f672d4a3b80 10 client.0 _ll_register_callbacks cb 0x55e7cd506c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:04:59.437 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.434+0000 7f671be00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e7cd60f140 0x55e7cd61ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:59.437 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.434+0000 7f671be00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e7cd60f140 0x55e7cd61ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:51698/0 (socket says 172.21.15.151:51698) 2024-05-21T22:04:59.437 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.434+0000 7f6720c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e7cd614f30 0x55e7cd618510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:59.437 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.434+0000 7f671be00700 1 -- 172.21.15.151:0/2303747810 learned_addr learned my addr 172.21.15.151:0/2303747810 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:04:59.437 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.434+0000 7f671b400700 1 --2- 172.21.15.151:0/2303747810 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7cd614610 0x55e7cd617fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:59.439 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.435+0000 7f671b400700 1 -- 172.21.15.151:0/2303747810 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e7cd60f140 msgr2=0x55e7cd61ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:04:59.439 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.435+0000 7f671b400700 1 --2- 172.21.15.151:0/2303747810 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e7cd60f140 0x55e7cd61ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:59.439 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.435+0000 7f671b400700 1 -- 172.21.15.151:0/2303747810 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e7cd614f30 msgr2=0x55e7cd618510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:04:59.439 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.435+0000 7f671b400700 1 --2- 172.21.15.151:0/2303747810 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e7cd614f30 0x55e7cd618510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:59.439 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.435+0000 7f671b400700 1 -- 172.21.15.151:0/2303747810 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7cd4ac8d0 con 0x55e7cd614610 2024-05-21T22:04:59.439 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.435+0000 7f671b400700 1 --2- 172.21.15.151:0/2303747810 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7cd614610 0x55e7cd617fd0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f67080dffa0 tx=0x7f67080aa090 comp rx=0 tx=0).ready entity=mon.2 client_cookie=198443ec8efe03b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:04:59.439 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.435+0000 7f6706000700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-21T22:04:59.439 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.435+0000 7f6706000700 1 -- 172.21.15.151:0/2303747810 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67080aae60 con 0x55e7cd614610 2024-05-21T22:04:59.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.435+0000 7f672d4a3b80 1 -- 172.21.15.151:0/2303747810 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f670c019ae0 con 0x55e7cd614610 2024-05-21T22:04:59.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.435+0000 7f6706000700 1 -- 172.21.15.151:0/2303747810 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7f67080a8fa0 con 0x55e7cd614610 2024-05-21T22:04:59.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.436+0000 7f6706000700 1 -- 172.21.15.151:0/2303747810 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f67080a59f0 con 0x55e7cd614610 2024-05-21T22:04:59.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.436+0000 7f672d4a3b80 10 client.27377 fetch_fsmap learned FSMap version 29 2024-05-21T22:04:59.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.436+0000 7f672d4a3b80 1 -- 172.21.15.151:0/2303747810 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e7cd52d060 con 0x55e7cd614610 2024-05-21T22:04:59.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.436+0000 7f6706000700 1 -- 172.21.15.151:0/2303747810 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f670c019ae0 con 0x55e7cd614610 2024-05-21T22:04:59.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.436+0000 7f6706000700 1 -- 172.21.15.151:0/2303747810 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f67080cc0f0 con 0x55e7cd614610 2024-05-21T22:04:59.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.436+0000 7f672d4a3b80 10 client.27377 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:04:59.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.436+0000 7f672d4a3b80 10 client.27377 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-21T22:04:59.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.436+0000 7f672d4a3b80 20 client.27377 populate_metadata read hostname 'smithi151' 2024-05-21T22:04:59.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.436+0000 7f672d4a3b80 1 --2- 172.21.15.151:0/2303747810 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x55e7cd628550 0x55e7cd62a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:59.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.436+0000 7f672d4a3b80 4 client.27377 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T22:04:59.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.436+0000 7f672d4a3b80 1 -- 172.21.15.151:0/2303747810 --> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e7cd56ed90 con 0x55e7cd628550 2024-05-21T22:04:59.441 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.437+0000 7f671be00700 1 --2- 172.21.15.151:0/2303747810 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x55e7cd628550 0x55e7cd62a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:59.441 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.437+0000 7f6706000700 10 client.27377 ms_handle_connect on v2:172.21.15.184:6838/3059935143 2024-05-21T22:04:59.441 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.437+0000 7f671be00700 1 --2- 172.21.15.151:0/2303747810 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x55e7cd628550 0x55e7cd62a940 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=71a2b6329c463ee7 server_cookie=5879600faf4269d in_seq=0 out_seq=0 2024-05-21T22:04:59.442 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.438+0000 7f6706000700 1 -- 172.21.15.151:0/2303747810 <== mds.1 v2:172.21.15.184:6838/3059935143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55e7cd56ed90 con 0x55e7cd628550 2024-05-21T22:04:59.442 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.438+0000 7f6706000700 10 client.27377 handle_command_reply: tid=0 2024-05-21T22:04:59.442 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.438+0000 7f672d4a3b80 1 client.27377 shutdown 2024-05-21T22:04:59.442 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.439+0000 7f672d4a3b80 1 -- 172.21.15.151:0/2303747810 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7cd614610 msgr2=0x55e7cd617fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:04:59.442 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.439+0000 7f672d4a3b80 1 --2- 172.21.15.151:0/2303747810 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7cd614610 0x55e7cd617fd0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f67080dffa0 tx=0x7f67080aa090 comp rx=0 tx=0).stop 2024-05-21T22:04:59.442 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.439+0000 7f672d4a3b80 1 -- 172.21.15.151:0/2303747810 shutdown_connections 2024-05-21T22:04:59.442 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.439+0000 7f672d4a3b80 1 --2- 172.21.15.151:0/2303747810 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x55e7cd628550 0x55e7cd62a940 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:59.442 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.439+0000 7f672d4a3b80 1 --2- 172.21.15.151:0/2303747810 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e7cd614f30 0x55e7cd618510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:59.442 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.439+0000 7f672d4a3b80 1 --2- 172.21.15.151:0/2303747810 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e7cd60f140 0x55e7cd61ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:59.442 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.439+0000 7f672d4a3b80 1 --2- 172.21.15.151:0/2303747810 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7cd614610 0x55e7cd617fd0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:59.443 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.439+0000 7f6706000700 0 client.27377 ms_handle_reset on v2:172.21.15.184:6838/3059935143 2024-05-21T22:04:59.443 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.439+0000 7f672d4a3b80 1 -- 172.21.15.151:0/2303747810 >> 172.21.15.151:0/2303747810 conn(0x55e7cd53cbe0 msgr2=0x55e7cd608d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:04:59.443 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.439+0000 7f672d4a3b80 1 -- 172.21.15.151:0/2303747810 shutdown_connections 2024-05-21T22:04:59.443 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.439+0000 7f672d4a3b80 1 -- 172.21.15.151:0/2303747810 wait complete. 2024-05-21T22:04:59.443 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.439+0000 7f672d4a3b80 20 client.27377 trim_cache size 0 max 16384 2024-05-21T22:04:59.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.456+0000 7f672d4a3b80 1 Processor -- start 2024-05-21T22:04:59.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.456+0000 7f672d4a3b80 1 -- start start 2024-05-21T22:04:59.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.456+0000 7f672d4a3b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e7cd60f140 0x55e7cd61ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:59.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.456+0000 7f672d4a3b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e7cd614610 0x55e7cd612790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:59.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.456+0000 7f672d4a3b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7cd614f30 0x55e7cd615a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:59.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.456+0000 7f672d4a3b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f671c1b4520 con 0x55e7cd60f140 2024-05-21T22:04:59.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.456+0000 7f672d4a3b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f671c1b6060 con 0x55e7cd614f30 2024-05-21T22:04:59.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.456+0000 7f672d4a3b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f671c1b4bf0 con 0x55e7cd614610 2024-05-21T22:04:59.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.456+0000 7f671be00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e7cd60f140 0x55e7cd61ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:59.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.456+0000 7f671be00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e7cd60f140 0x55e7cd61ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49752/0 (socket says 172.21.15.151:49752) 2024-05-21T22:04:59.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.456+0000 7f6720c00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7cd614f30 0x55e7cd615a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:59.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.456+0000 7f671b400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e7cd614610 0x55e7cd612790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:59.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.456+0000 7f671be00700 1 -- 172.21.15.151:0/3597247656 learned_addr learned my addr 172.21.15.151:0/3597247656 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:04:59.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.457+0000 7f671b400700 1 -- 172.21.15.151:0/3597247656 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7cd614f30 msgr2=0x55e7cd615a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:04:59.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.457+0000 7f671b400700 1 --2- 172.21.15.151:0/3597247656 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7cd614f30 0x55e7cd615a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:59.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.457+0000 7f671b400700 1 -- 172.21.15.151:0/3597247656 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e7cd60f140 msgr2=0x55e7cd61ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:04:59.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.457+0000 7f671b400700 1 --2- 172.21.15.151:0/3597247656 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e7cd60f140 0x55e7cd61ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:59.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.457+0000 7f671b400700 1 -- 172.21.15.151:0/3597247656 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7cd61a870 con 0x55e7cd614610 2024-05-21T22:04:59.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.457+0000 7f671be00700 1 --2- 172.21.15.151:0/3597247656 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e7cd60f140 0x55e7cd61ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:04:59.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.457+0000 7f671b400700 1 --2- 172.21.15.151:0/3597247656 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e7cd614610 0x55e7cd612790 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f67080a9ac0 tx=0x7f67080e2b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a512fd326f16a49 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:04:59.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.457+0000 7f6710c00700 1 -- 172.21.15.151:0/3597247656 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67080c9070 con 0x55e7cd614610 2024-05-21T22:04:59.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.457+0000 7f6710c00700 1 -- 172.21.15.151:0/3597247656 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67080a9640 con 0x55e7cd614610 2024-05-21T22:04:59.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.457+0000 7f6710c00700 1 -- 172.21.15.151:0/3597247656 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67080aae60 con 0x55e7cd614610 2024-05-21T22:04:59.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.458+0000 7f672d4a3b80 1 -- 172.21.15.151:0/3597247656 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e7cd614610 msgr2=0x55e7cd612790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:04:59.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.458+0000 7f672d4a3b80 1 --2- 172.21.15.151:0/3597247656 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e7cd614610 0x55e7cd612790 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f67080a9ac0 tx=0x7f67080e2b40 comp rx=0 tx=0).stop 2024-05-21T22:04:59.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.458+0000 7f672d4a3b80 1 -- 172.21.15.151:0/3597247656 shutdown_connections 2024-05-21T22:04:59.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.458+0000 7f672d4a3b80 1 --2- 172.21.15.151:0/3597247656 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7cd614f30 0x55e7cd615a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:59.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.458+0000 7f672d4a3b80 1 --2- 172.21.15.151:0/3597247656 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e7cd60f140 0x55e7cd61ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:59.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.458+0000 7f672d4a3b80 1 --2- 172.21.15.151:0/3597247656 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e7cd614610 0x55e7cd612790 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:59.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.458+0000 7f672d4a3b80 1 -- 172.21.15.151:0/3597247656 >> 172.21.15.151:0/3597247656 conn(0x55e7cd53cbe0 msgr2=0x55e7cd615310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:04:59.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.458+0000 7f672d4a3b80 1 -- 172.21.15.151:0/3597247656 shutdown_connections 2024-05-21T22:04:59.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.458+0000 7f672d4a3b80 1 -- 172.21.15.151:0/3597247656 wait complete. 2024-05-21T22:04:59.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.459+0000 7f672d4a3b80 1 -- start start 2024-05-21T22:04:59.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.459+0000 7f672d4a3b80 1 Processor -- start 2024-05-21T22:04:59.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.459+0000 7f672d4a3b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7cd60f140 0x55e7cd62fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:59.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.459+0000 7f672d4a3b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e7cd614f30 0x55e7cd630360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:59.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.459+0000 7f672d4a3b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e7cd6308a0 0x55e7cd692090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:59.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.459+0000 7f672d4a3b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f671c1b5990 con 0x55e7cd6308a0 2024-05-21T22:04:59.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.459+0000 7f672d4a3b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f671c1b52c0 con 0x55e7cd60f140 2024-05-21T22:04:59.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.459+0000 7f672d4a3b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f671c1b24e0 con 0x55e7cd614f30 2024-05-21T22:04:59.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.460+0000 7f672d4a3b80 10 client.0 _ll_register_callbacks cb 0x55e7cd506c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:04:59.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.460+0000 7f6720c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e7cd6308a0 0x55e7cd692090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:59.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.460+0000 7f671be00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7cd60f140 0x55e7cd62fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:59.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.460+0000 7f671be00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7cd60f140 0x55e7cd62fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:32996/0 (socket says 172.21.15.151:32996) 2024-05-21T22:04:59.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.460+0000 7f671be00700 1 -- 172.21.15.151:0/1703229886 learned_addr learned my addr 172.21.15.151:0/1703229886 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:04:59.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.460+0000 7f671b400700 1 --2- 172.21.15.151:0/1703229886 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e7cd614f30 0x55e7cd630360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:59.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.460+0000 7f671be00700 1 -- 172.21.15.151:0/1703229886 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e7cd614f30 msgr2=0x55e7cd630360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:04:59.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.460+0000 7f671be00700 1 --2- 172.21.15.151:0/1703229886 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e7cd614f30 0x55e7cd630360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:59.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.460+0000 7f671be00700 1 -- 172.21.15.151:0/1703229886 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e7cd6308a0 msgr2=0x55e7cd692090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:04:59.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.460+0000 7f671be00700 1 --2- 172.21.15.151:0/1703229886 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e7cd6308a0 0x55e7cd692090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:59.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.460+0000 7f671be00700 1 -- 172.21.15.151:0/1703229886 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7cd692680 con 0x55e7cd60f140 2024-05-21T22:04:59.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.461+0000 7f671be00700 1 --2- 172.21.15.151:0/1703229886 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7cd60f140 0x55e7cd62fe20 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f670c00a010 tx=0x7f670c005e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1779be8f95b4ddb6 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:04:59.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.461+0000 7f6706000700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-21T22:04:59.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.461+0000 7f6706000700 1 -- 172.21.15.151:0/1703229886 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f670c01e040 con 0x55e7cd60f140 2024-05-21T22:04:59.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.461+0000 7f6706000700 1 -- 172.21.15.151:0/1703229886 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7f670c0194b0 con 0x55e7cd60f140 2024-05-21T22:04:59.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.461+0000 7f672d4a3b80 1 -- 172.21.15.151:0/1703229886 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67080a9640 con 0x55e7cd60f140 2024-05-21T22:04:59.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.461+0000 7f6706000700 1 -- 172.21.15.151:0/1703229886 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f670c0196b0 con 0x55e7cd60f140 2024-05-21T22:04:59.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.461+0000 7f672d4a3b80 10 client.27383 fetch_fsmap learned FSMap version 29 2024-05-21T22:04:59.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.461+0000 7f672d4a3b80 1 -- 172.21.15.151:0/1703229886 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e7cd693940 con 0x55e7cd60f140 2024-05-21T22:04:59.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.461+0000 7f6706000700 1 -- 172.21.15.151:0/1703229886 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67080a9640 con 0x55e7cd60f140 2024-05-21T22:04:59.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.462+0000 7f6706000700 1 -- 172.21.15.151:0/1703229886 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f670c00e710 con 0x55e7cd60f140 2024-05-21T22:04:59.467 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.462+0000 7f672d4a3b80 10 client.27383 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:04:59.467 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.462+0000 7f672d4a3b80 10 client.27383 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-21T22:04:59.467 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.462+0000 7f672d4a3b80 20 client.27383 populate_metadata read hostname 'smithi151' 2024-05-21T22:04:59.467 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.462+0000 7f672d4a3b80 1 --2- 172.21.15.151:0/1703229886 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x55e7cd693e50 0x55e7cd696240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:04:59.467 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.462+0000 7f672d4a3b80 4 client.27383 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T22:04:59.467 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.462+0000 7f672d4a3b80 1 -- 172.21.15.151:0/1703229886 --> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e7cd568570 con 0x55e7cd693e50 2024-05-21T22:04:59.467 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.462+0000 7f671b400700 1 --2- 172.21.15.151:0/1703229886 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x55e7cd693e50 0x55e7cd696240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:04:59.467 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.463+0000 7f671b400700 1 --2- 172.21.15.151:0/1703229886 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x55e7cd693e50 0x55e7cd696240 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c6e310d9af4587de server_cookie=d7f7490ff7e1bca4 in_seq=0 out_seq=0 2024-05-21T22:04:59.467 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.463+0000 7f6706000700 10 client.27383 ms_handle_connect on v2:172.21.15.184:6838/3059935143 2024-05-21T22:04:59.467 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.463+0000 7f6706000700 1 -- 172.21.15.151:0/1703229886 <== mds.1 v2:172.21.15.184:6838/3059935143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e7cd568570 con 0x55e7cd693e50 2024-05-21T22:04:59.467 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.463+0000 7f6706000700 10 client.27383 handle_command_reply: tid=0 2024-05-21T22:04:59.467 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.463+0000 7f672d4a3b80 1 client.27383 shutdown 2024-05-21T22:04:59.467 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.463+0000 7f672d4a3b80 1 -- 172.21.15.151:0/1703229886 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7cd60f140 msgr2=0x55e7cd62fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:04:59.467 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.463+0000 7f672d4a3b80 1 --2- 172.21.15.151:0/1703229886 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7cd60f140 0x55e7cd62fe20 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f670c00a010 tx=0x7f670c005e40 comp rx=0 tx=0).stop 2024-05-21T22:04:59.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.463+0000 7f672d4a3b80 1 -- 172.21.15.151:0/1703229886 shutdown_connections 2024-05-21T22:04:59.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.463+0000 7f672d4a3b80 1 --2- 172.21.15.151:0/1703229886 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x55e7cd693e50 0x55e7cd696240 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:59.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.463+0000 7f672d4a3b80 1 --2- 172.21.15.151:0/1703229886 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e7cd60f140 0x55e7cd62fe20 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:59.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.463+0000 7f672d4a3b80 1 --2- 172.21.15.151:0/1703229886 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e7cd6308a0 0x55e7cd692090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:59.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.463+0000 7f6706000700 0 client.27383 ms_handle_reset on v2:172.21.15.184:6838/3059935143 2024-05-21T22:04:59.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.463+0000 7f672d4a3b80 1 --2- 172.21.15.151:0/1703229886 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e7cd614f30 0x55e7cd630360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:59.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.463+0000 7f672d4a3b80 1 -- 172.21.15.151:0/1703229886 >> 172.21.15.151:0/1703229886 conn(0x55e7cd53cbe0 msgr2=0x55e7cd613180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:04:59.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.463+0000 7f672d4a3b80 1 -- 172.21.15.151:0/1703229886 shutdown_connections 2024-05-21T22:04:59.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.463+0000 7f672d4a3b80 1 -- 172.21.15.151:0/1703229886 wait complete. 2024-05-21T22:04:59.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.464+0000 7f672d4a3b80 20 client.27383 trim_cache size 0 max 16384 2024-05-21T22:04:59.468 INFO:teuthology.orchestra.run.smithi151.stdout:[] 2024-05-21T22:04:59.469 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.464+0000 7f6722c00700 1 -- 172.21.15.151:0/2891407996 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f67000b2a70 msgr2=0x7f67000b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:04:59.469 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.464+0000 7f6722c00700 1 --2- 172.21.15.151:0/2891407996 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f67000b2a70 0x7f67000b4f30 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6714005bd0 tx=0x7f671400b040 comp rx=0 tx=0).stop 2024-05-21T22:04:59.469 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.464+0000 7f6722c00700 1 -- 172.21.15.151:0/2891407996 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f671c10a780 msgr2=0x7f671c1a0730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:04:59.469 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.464+0000 7f6722c00700 1 --2- 172.21.15.151:0/2891407996 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f671c10a780 0x7f671c1a0730 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f670800e970 tx=0x7f670800ec80 comp rx=0 tx=0).stop 2024-05-21T22:04:59.469 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.464+0000 7f6722c00700 1 -- 172.21.15.151:0/2891407996 shutdown_connections 2024-05-21T22:04:59.470 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.464+0000 7f6722c00700 1 --2- 172.21.15.151:0/2891407996 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f67000b2a70 0x7f67000b4f30 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:59.470 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.464+0000 7f6722c00700 1 --2- 172.21.15.151:0/2891407996 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f671c109dd0 0x7f671c1a01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:59.470 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.464+0000 7f6722c00700 1 --2- 172.21.15.151:0/2891407996 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f671c10eeb0 0x7f671c1a4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:59.470 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.464+0000 7f6722c00700 1 --2- 172.21.15.151:0/2891407996 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f671c10a780 0x7f671c1a0730 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:04:59.470 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.464+0000 7f6722c00700 1 -- 172.21.15.151:0/2891407996 >> 172.21.15.151:0/2891407996 conn(0x7f671c076d10 msgr2=0x7f671c077290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:04:59.470 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.465+0000 7f6722c00700 1 -- 172.21.15.151:0/2891407996 shutdown_connections 2024-05-21T22:04:59.470 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:04:59.465+0000 7f6722c00700 1 -- 172.21.15.151:0/2891407996 wait complete. 2024-05-21T22:04:59.954 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:04:59 smithi151 ceph-mon[31623]: pgmap v290: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T22:04:59.985 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:04:59 smithi184 ceph-mon[36942]: pgmap v290: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T22:05:00.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:04:59 smithi164 ceph-mon[36269]: pgmap v290: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T22:05:00.542 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-21T22:05:00.982 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:05:01.910 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:05:01 smithi151 ceph-mon[31623]: pgmap v291: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:05:01.985 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:05:01 smithi184 ceph-mon[36942]: pgmap v291: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:05:02.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:05:01 smithi164 ceph-mon[36269]: pgmap v291: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:05:03.137 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.135+0000 7f0756a00700 1 -- 172.21.15.151:0/1948276874 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0750078390 msgr2=0x7f0750113db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:03.137 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.135+0000 7f0756a00700 1 --2- 172.21.15.151:0/1948276874 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0750078390 0x7f0750113db0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f073c009990 tx=0x7f073c009ca0 comp rx=0 tx=0).stop 2024-05-21T22:05:03.137 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.135+0000 7f0756a00700 1 -- 172.21.15.151:0/1948276874 shutdown_connections 2024-05-21T22:05:03.137 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.135+0000 7f0756a00700 1 --2- 172.21.15.151:0/1948276874 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0750077a70 0x7f0750077e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:03.137 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.135+0000 7f0756a00700 1 --2- 172.21.15.151:0/1948276874 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07501142f0 0x7f07501166e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:03.137 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.135+0000 7f0756a00700 1 --2- 172.21.15.151:0/1948276874 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0750078390 0x7f0750113db0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:03.138 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.135+0000 7f0756a00700 1 -- 172.21.15.151:0/1948276874 >> 172.21.15.151:0/1948276874 conn(0x7f0750101180 msgr2=0x7f07501035a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:05:03.138 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.135+0000 7f0756a00700 1 -- 172.21.15.151:0/1948276874 shutdown_connections 2024-05-21T22:05:03.138 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.135+0000 7f0756a00700 1 -- 172.21.15.151:0/1948276874 wait complete. 2024-05-21T22:05:03.138 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.137+0000 7f0756a00700 1 Processor -- start 2024-05-21T22:05:03.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.137+0000 7f0756a00700 1 -- start start 2024-05-21T22:05:03.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.137+0000 7f0756a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0750077a70 0x7f07501a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:03.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.137+0000 7f0756a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0750078390 0x7f07501a0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:03.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.137+0000 7f0756a00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07501142f0 0x7f07501a4a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:03.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.137+0000 7f0756a00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0750119c80 con 0x7f0750077a70 2024-05-21T22:05:03.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.137+0000 7f0756a00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0750119b00 con 0x7f0750078390 2024-05-21T22:05:03.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.137+0000 7f0756a00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0750119e00 con 0x7f07501142f0 2024-05-21T22:05:03.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.138+0000 7f0755600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0750077a70 0x7f07501a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:03.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.138+0000 7f0755600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0750077a70 0x7f07501a01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49782/0 (socket says 172.21.15.151:49782) 2024-05-21T22:05:03.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.138+0000 7f0755600700 1 -- 172.21.15.151:0/3323980280 learned_addr learned my addr 172.21.15.151:0/3323980280 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:05:03.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.138+0000 7f0754c00700 1 --2- 172.21.15.151:0/3323980280 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0750078390 0x7f07501a0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:03.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.138+0000 7f0756000700 1 --2- 172.21.15.151:0/3323980280 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07501142f0 0x7f07501a4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:03.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.138+0000 7f0754c00700 1 -- 172.21.15.151:0/3323980280 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07501142f0 msgr2=0x7f07501a4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:03.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.138+0000 7f0754c00700 1 --2- 172.21.15.151:0/3323980280 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07501142f0 0x7f07501a4a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:03.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.138+0000 7f0754c00700 1 -- 172.21.15.151:0/3323980280 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0750077a70 msgr2=0x7f07501a01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:03.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.138+0000 7f0754c00700 1 --2- 172.21.15.151:0/3323980280 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0750077a70 0x7f07501a01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:03.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.138+0000 7f0754c00700 1 -- 172.21.15.151:0/3323980280 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07501a51b0 con 0x7f0750078390 2024-05-21T22:05:03.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.138+0000 7f0755600700 1 --2- 172.21.15.151:0/3323980280 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0750077a70 0x7f07501a01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:05:03.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.139+0000 7f0754c00700 1 --2- 172.21.15.151:0/3323980280 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0750078390 0x7f07501a0730 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f073c006010 tx=0x7f073c0057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:05:03.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.139+0000 7f074e000700 1 -- 172.21.15.151:0/3323980280 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f073c027070 con 0x7f0750078390 2024-05-21T22:05:03.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.139+0000 7f074e000700 1 -- 172.21.15.151:0/3323980280 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f073c022470 con 0x7f0750078390 2024-05-21T22:05:03.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.139+0000 7f074e000700 1 -- 172.21.15.151:0/3323980280 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f073c017650 con 0x7f0750078390 2024-05-21T22:05:03.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.139+0000 7f0756a00700 1 -- 172.21.15.151:0/3323980280 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07501a5440 con 0x7f0750078390 2024-05-21T22:05:03.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.139+0000 7f0756a00700 1 -- 172.21.15.151:0/3323980280 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07501acd20 con 0x7f0750078390 2024-05-21T22:05:03.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.143+0000 7f074e000700 1 -- 172.21.15.151:0/3323980280 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7f073c02a070 con 0x7f0750078390 2024-05-21T22:05:03.145 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.144+0000 7f07628ccb80 1 Processor -- start 2024-05-21T22:05:03.145 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.144+0000 7f07628ccb80 1 -- start start 2024-05-21T22:05:03.146 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.144+0000 7f074e000700 1 --2- 172.21.15.151:0/3323980280 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f07380b29f0 0x7f07380b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:03.146 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.144+0000 7f07628ccb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557cda16c7c0 0x557cda16cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:03.146 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.144+0000 7f07628ccb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557cda16d0e0 0x557cda166d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:03.146 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.144+0000 7f07628ccb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557cda167390 0x557cda1677f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:03.146 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.144+0000 7f07628ccb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x557cda0c4e40 con 0x557cda16d0e0 2024-05-21T22:05:03.146 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.144+0000 7f07628ccb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x557cda0b49e0 con 0x557cda16c7c0 2024-05-21T22:05:03.146 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.144+0000 7f07628ccb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x557cda0b8a20 con 0x557cda167390 2024-05-21T22:05:03.146 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.145+0000 7f074e000700 1 -- 172.21.15.151:0/3323980280 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7f073c0e3270 con 0x7f0750078390 2024-05-21T22:05:03.147 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.145+0000 7f0755600700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557cda16c7c0 0x557cda16cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:03.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.145+0000 7f0755600700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557cda16c7c0 0x557cda16cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:33028/0 (socket says 172.21.15.151:33028) 2024-05-21T22:05:03.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.145+0000 7f0755600700 1 -- 172.21.15.151:0/2390636811 learned_addr learned my addr 172.21.15.151:0/2390636811 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:05:03.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.145+0000 7f0754c00700 1 --2- 172.21.15.151:0/2390636811 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557cda16d0e0 0x557cda166d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:03.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.145+0000 7f0756000700 1 --2- 172.21.15.151:0/3323980280 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f07380b29f0 0x7f07380b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:03.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.145+0000 7f0756000700 1 --2- 172.21.15.151:0/2390636811 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557cda167390 0x557cda1677f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:03.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.145+0000 7f0755600700 1 -- 172.21.15.151:0/2390636811 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557cda167390 msgr2=0x557cda1677f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:03.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.145+0000 7f0755600700 1 --2- 172.21.15.151:0/2390636811 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557cda167390 0x557cda1677f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:03.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.145+0000 7f0755600700 1 -- 172.21.15.151:0/2390636811 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557cda16d0e0 msgr2=0x557cda166d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:03.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.145+0000 7f0755600700 1 --2- 172.21.15.151:0/2390636811 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557cda16d0e0 0x557cda166d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:03.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.145+0000 7f0755600700 1 -- 172.21.15.151:0/2390636811 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557cda07d420 con 0x557cda16c7c0 2024-05-21T22:05:03.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.146+0000 7f0755600700 1 --2- 172.21.15.151:0/2390636811 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557cda16c7c0 0x557cda16cba0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f07440117e0 tx=0x7f0744011af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58648b30804b6ec9 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:05:03.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.146+0000 7f0756000700 1 --2- 172.21.15.151:0/3323980280 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f07380b29f0 0x7f07380b4eb0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0748009ec0 tx=0x7f0748009920 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:05:03.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.146+0000 7f0742a00700 1 -- 172.21.15.151:0/2390636811 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f074401e070 con 0x557cda16c7c0 2024-05-21T22:05:03.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.146+0000 7f0742a00700 1 -- 172.21.15.151:0/2390636811 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f074400f930 con 0x557cda16c7c0 2024-05-21T22:05:03.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.146+0000 7f0742a00700 1 -- 172.21.15.151:0/2390636811 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0744019970 con 0x557cda16c7c0 2024-05-21T22:05:03.150 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.147+0000 7f07628ccb80 1 -- 172.21.15.151:0/2390636811 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557cda16c7c0 msgr2=0x557cda16cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:03.150 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.147+0000 7f07628ccb80 1 --2- 172.21.15.151:0/2390636811 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557cda16c7c0 0x557cda16cba0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f07440117e0 tx=0x7f0744011af0 comp rx=0 tx=0).stop 2024-05-21T22:05:03.150 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.147+0000 7f07628ccb80 1 -- 172.21.15.151:0/2390636811 shutdown_connections 2024-05-21T22:05:03.150 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.147+0000 7f07628ccb80 1 --2- 172.21.15.151:0/2390636811 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557cda16c7c0 0x557cda16cba0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:03.150 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.147+0000 7f07628ccb80 1 --2- 172.21.15.151:0/2390636811 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557cda167390 0x557cda1677f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:03.150 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.147+0000 7f07628ccb80 1 --2- 172.21.15.151:0/2390636811 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557cda16d0e0 0x557cda166d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:03.150 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.147+0000 7f07628ccb80 1 -- 172.21.15.151:0/2390636811 >> 172.21.15.151:0/2390636811 conn(0x557cda094be0 msgr2=0x557cda161810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:05:03.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.147+0000 7f07628ccb80 1 -- 172.21.15.151:0/2390636811 shutdown_connections 2024-05-21T22:05:03.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.147+0000 7f07628ccb80 1 -- 172.21.15.151:0/2390636811 wait complete. 2024-05-21T22:05:03.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.148+0000 7f07628ccb80 1 -- start start 2024-05-21T22:05:03.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.148+0000 7f07628ccb80 1 Processor -- start 2024-05-21T22:05:03.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.148+0000 7f07628ccb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557cda167390 0x557cda172e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:03.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.148+0000 7f07628ccb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557cda16c7c0 0x557cda1701d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:03.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.148+0000 7f07628ccb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557cda16d0e0 0x557cda170710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:03.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.148+0000 7f07628ccb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x557cd9fbafa0 con 0x557cda16c7c0 2024-05-21T22:05:03.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.148+0000 7f07628ccb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x557cda055760 con 0x557cda16d0e0 2024-05-21T22:05:03.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.148+0000 7f07628ccb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x557cda173c00 con 0x557cda167390 2024-05-21T22:05:03.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.149+0000 7f07628ccb80 10 client.0 _ll_register_callbacks cb 0x557cda05ec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:05:03.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.149+0000 7f0756000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557cda16d0e0 0x557cda170710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:03.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.149+0000 7f0756000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557cda16d0e0 0x557cda170710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:33036/0 (socket says 172.21.15.151:33036) 2024-05-21T22:05:03.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.149+0000 7f0756000700 1 -- 172.21.15.151:0/1202903925 learned_addr learned my addr 172.21.15.151:0/1202903925 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:05:03.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.149+0000 7f0754c00700 1 --2- 172.21.15.151:0/1202903925 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557cda16c7c0 0x557cda1701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:03.152 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.149+0000 7f0755600700 1 --2- 172.21.15.151:0/1202903925 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557cda167390 0x557cda172e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:03.153 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.149+0000 7f0755600700 1 -- 172.21.15.151:0/1202903925 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557cda16d0e0 msgr2=0x557cda170710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:03.153 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.149+0000 7f0755600700 1 --2- 172.21.15.151:0/1202903925 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557cda16d0e0 0x557cda170710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:03.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.149+0000 7f0755600700 1 -- 172.21.15.151:0/1202903925 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557cda16c7c0 msgr2=0x557cda1701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:03.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.149+0000 7f0755600700 1 --2- 172.21.15.151:0/1202903925 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557cda16c7c0 0x557cda1701d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:03.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.149+0000 7f0755600700 1 -- 172.21.15.151:0/1202903925 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557cda0048d0 con 0x557cda167390 2024-05-21T22:05:03.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.149+0000 7f0754c00700 1 --2- 172.21.15.151:0/1202903925 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557cda16c7c0 0x557cda1701d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:05:03.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.149+0000 7f0756000700 1 --2- 172.21.15.151:0/1202903925 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557cda16d0e0 0x557cda170710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:05:03.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.150+0000 7f0755600700 1 --2- 172.21.15.151:0/1202903925 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557cda167390 0x557cda172e30 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f0744000c00 tx=0x7f074400fd90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99f7ab263e31a9f server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:05:03.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.150+0000 7f0737400700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-21T22:05:03.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.150+0000 7f0737400700 1 -- 172.21.15.151:0/1202903925 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f074401e040 con 0x557cda167390 2024-05-21T22:05:03.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.150+0000 7f07628ccb80 1 -- 172.21.15.151:0/1202903925 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f074400f930 con 0x557cda167390 2024-05-21T22:05:03.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.150+0000 7f0737400700 1 -- 172.21.15.151:0/1202903925 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7f07440050d0 con 0x557cda167390 2024-05-21T22:05:03.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.150+0000 7f0737400700 1 -- 172.21.15.151:0/1202903925 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0744005ce0 con 0x557cda167390 2024-05-21T22:05:03.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.150+0000 7f07628ccb80 10 client.27397 fetch_fsmap learned FSMap version 29 2024-05-21T22:05:03.155 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.150+0000 7f07628ccb80 1 -- 172.21.15.151:0/1202903925 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557cda085060 con 0x557cda167390 2024-05-21T22:05:03.155 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.151+0000 7f0737400700 1 -- 172.21.15.151:0/1202903925 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f074400f930 con 0x557cda167390 2024-05-21T22:05:03.155 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.151+0000 7f0737400700 1 -- 172.21.15.151:0/1202903925 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f0744021070 con 0x557cda167390 2024-05-21T22:05:03.155 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.151+0000 7f07628ccb80 10 client.27397 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:05:03.155 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.151+0000 7f07628ccb80 10 client.27397 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T22:05:03.155 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.151+0000 7f07628ccb80 20 client.27397 populate_metadata read hostname 'smithi151' 2024-05-21T22:05:03.155 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.151+0000 7f07628ccb80 1 --2- 172.21.15.151:0/1202903925 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x557cda180640 0x557cda182a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:03.155 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.151+0000 7f07628ccb80 4 client.27397 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T22:05:03.155 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.151+0000 7f07628ccb80 1 -- 172.21.15.151:0/1202903925 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557cda0c6d90 con 0x557cda180640 2024-05-21T22:05:03.155 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.152+0000 7f0754c00700 1 --2- 172.21.15.151:0/1202903925 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x557cda180640 0x557cda182a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:03.155 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.152+0000 7f0754c00700 1 --2- 172.21.15.151:0/1202903925 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x557cda180640 0x557cda182a30 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a6dbf2d6c8e3a79 server_cookie=9649a2af1f8af949 in_seq=0 out_seq=0 2024-05-21T22:05:03.155 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.152+0000 7f0737400700 10 client.27397 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T22:05:03.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.153+0000 7f0737400700 1 -- 172.21.15.151:0/1202903925 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x557cda0c6d90 con 0x557cda180640 2024-05-21T22:05:03.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.153+0000 7f0737400700 10 client.27397 handle_command_reply: tid=0 2024-05-21T22:05:03.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.153+0000 7f07628ccb80 1 client.27397 shutdown 2024-05-21T22:05:03.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.153+0000 7f07628ccb80 1 -- 172.21.15.151:0/1202903925 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557cda167390 msgr2=0x557cda172e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:03.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.153+0000 7f07628ccb80 1 --2- 172.21.15.151:0/1202903925 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557cda167390 0x557cda172e30 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f0744000c00 tx=0x7f074400fd90 comp rx=0 tx=0).stop 2024-05-21T22:05:03.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.153+0000 7f07628ccb80 1 -- 172.21.15.151:0/1202903925 shutdown_connections 2024-05-21T22:05:03.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.153+0000 7f07628ccb80 1 --2- 172.21.15.151:0/1202903925 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x557cda180640 0x557cda182a30 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:03.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.153+0000 7f07628ccb80 1 --2- 172.21.15.151:0/1202903925 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557cda16d0e0 0x557cda170710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:03.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.153+0000 7f07628ccb80 1 --2- 172.21.15.151:0/1202903925 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557cda167390 0x557cda172e30 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:03.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.153+0000 7f0737400700 0 client.27397 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T22:05:03.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.153+0000 7f07628ccb80 1 --2- 172.21.15.151:0/1202903925 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557cda16c7c0 0x557cda1701d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:03.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.153+0000 7f07628ccb80 1 -- 172.21.15.151:0/1202903925 >> 172.21.15.151:0/1202903925 conn(0x557cda094be0 msgr2=0x557cda160fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:05:03.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.153+0000 7f07628ccb80 1 -- 172.21.15.151:0/1202903925 shutdown_connections 2024-05-21T22:05:03.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.154+0000 7f07628ccb80 1 -- 172.21.15.151:0/1202903925 wait complete. 2024-05-21T22:05:03.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.154+0000 7f07628ccb80 20 client.27397 trim_cache size 0 max 16384 2024-05-21T22:05:03.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.170+0000 7f07628ccb80 1 Processor -- start 2024-05-21T22:05:03.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.170+0000 7f07628ccb80 1 -- start start 2024-05-21T22:05:03.172 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.170+0000 7f07628ccb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557cda167390 0x557cda166d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:03.172 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.170+0000 7f07628ccb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557cda16c7c0 0x557cda16dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:03.173 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.170+0000 7f07628ccb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557cda16d0e0 0x557cda161740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:03.173 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.170+0000 7f07628ccb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f07501b4220 con 0x557cda16d0e0 2024-05-21T22:05:03.173 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.170+0000 7f07628ccb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f07501b5d60 con 0x557cda167390 2024-05-21T22:05:03.173 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.170+0000 7f07628ccb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f07501b48f0 con 0x557cda16c7c0 2024-05-21T22:05:03.173 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.171+0000 7f0756000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557cda16d0e0 0x557cda161740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:03.173 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.171+0000 7f0756000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557cda16d0e0 0x557cda161740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49822/0 (socket says 172.21.15.151:49822) 2024-05-21T22:05:03.173 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.171+0000 7f0756000700 1 -- 172.21.15.151:0/1713481018 learned_addr learned my addr 172.21.15.151:0/1713481018 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:05:03.173 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.171+0000 7f0755600700 1 --2- 172.21.15.151:0/1713481018 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557cda167390 0x557cda166d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:03.173 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.171+0000 7f0754c00700 1 --2- 172.21.15.151:0/1713481018 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557cda16c7c0 0x557cda16dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:03.173 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.171+0000 7f0756000700 1 -- 172.21.15.151:0/1713481018 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557cda16c7c0 msgr2=0x557cda16dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:03.173 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.171+0000 7f0756000700 1 --2- 172.21.15.151:0/1713481018 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557cda16c7c0 0x557cda16dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:03.173 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.171+0000 7f0756000700 1 -- 172.21.15.151:0/1713481018 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557cda167390 msgr2=0x557cda166d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:03.173 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.171+0000 7f0756000700 1 --2- 172.21.15.151:0/1713481018 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557cda167390 0x557cda166d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:03.173 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.171+0000 7f0756000700 1 -- 172.21.15.151:0/1713481018 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557cda172a70 con 0x557cda16d0e0 2024-05-21T22:05:03.174 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.171+0000 7f0755600700 1 --2- 172.21.15.151:0/1713481018 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557cda167390 0x557cda166d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:05:03.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.172+0000 7f0756000700 1 --2- 172.21.15.151:0/1713481018 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557cda16d0e0 0x557cda161740 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f0748009020 tx=0x7f074800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56e57e238de4007e server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:05:03.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.172+0000 7f0742000700 1 -- 172.21.15.151:0/1713481018 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0748003a00 con 0x557cda16d0e0 2024-05-21T22:05:03.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.172+0000 7f0742000700 1 -- 172.21.15.151:0/1713481018 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0748003ba0 con 0x557cda16d0e0 2024-05-21T22:05:03.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.172+0000 7f0742000700 1 -- 172.21.15.151:0/1713481018 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f074801a470 con 0x557cda16d0e0 2024-05-21T22:05:03.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.173+0000 7f07628ccb80 1 -- 172.21.15.151:0/1713481018 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557cda16d0e0 msgr2=0x557cda161740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:03.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.173+0000 7f07628ccb80 1 --2- 172.21.15.151:0/1713481018 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557cda16d0e0 0x557cda161740 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f0748009020 tx=0x7f074800b040 comp rx=0 tx=0).stop 2024-05-21T22:05:03.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.173+0000 7f07628ccb80 1 -- 172.21.15.151:0/1713481018 shutdown_connections 2024-05-21T22:05:03.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.173+0000 7f07628ccb80 1 --2- 172.21.15.151:0/1713481018 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557cda167390 0x557cda166d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:03.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.173+0000 7f07628ccb80 1 --2- 172.21.15.151:0/1713481018 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557cda16d0e0 0x557cda161740 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:03.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.173+0000 7f07628ccb80 1 --2- 172.21.15.151:0/1713481018 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557cda16c7c0 0x557cda16dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:03.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.173+0000 7f07628ccb80 1 -- 172.21.15.151:0/1713481018 >> 172.21.15.151:0/1713481018 conn(0x557cda094be0 msgr2=0x557cda16e990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:05:03.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.173+0000 7f07628ccb80 1 -- 172.21.15.151:0/1713481018 shutdown_connections 2024-05-21T22:05:03.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.173+0000 7f07628ccb80 1 -- 172.21.15.151:0/1713481018 wait complete. 2024-05-21T22:05:03.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.173+0000 7f07628ccb80 1 -- start start 2024-05-21T22:05:03.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.174+0000 7f07628ccb80 1 Processor -- start 2024-05-21T22:05:03.178 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.174+0000 7f07628ccb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557cda167390 0x557cda1792a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:03.178 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.174+0000 7f07628ccb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557cda16c7c0 0x557cda187750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:03.178 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.174+0000 7f07628ccb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557cda16d0e0 0x557cda187c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:03.178 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.174+0000 7f07628ccb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f07501b5690 con 0x557cda16c7c0 2024-05-21T22:05:03.178 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.174+0000 7f07628ccb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f07501b4fc0 con 0x557cda16d0e0 2024-05-21T22:05:03.178 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.174+0000 7f07628ccb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f07501b21e0 con 0x557cda167390 2024-05-21T22:05:03.178 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.174+0000 7f07628ccb80 10 client.0 _ll_register_callbacks cb 0x557cda05ec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:05:03.178 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.174+0000 7f0754c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557cda16c7c0 0x557cda187750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:03.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.174+0000 7f0754c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557cda16c7c0 0x557cda187750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49826/0 (socket says 172.21.15.151:49826) 2024-05-21T22:05:03.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.174+0000 7f0754c00700 1 -- 172.21.15.151:0/2988709416 learned_addr learned my addr 172.21.15.151:0/2988709416 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:05:03.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.175+0000 7f0756000700 1 --2- 172.21.15.151:0/2988709416 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557cda16d0e0 0x557cda187c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:03.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.175+0000 7f0755600700 1 --2- 172.21.15.151:0/2988709416 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557cda167390 0x557cda1792a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:03.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.175+0000 7f0754c00700 1 -- 172.21.15.151:0/2988709416 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557cda167390 msgr2=0x557cda1792a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:03.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.175+0000 7f0754c00700 1 --2- 172.21.15.151:0/2988709416 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557cda167390 0x557cda1792a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:03.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.175+0000 7f0754c00700 1 -- 172.21.15.151:0/2988709416 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557cda16d0e0 msgr2=0x557cda187c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:03.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.175+0000 7f0754c00700 1 --2- 172.21.15.151:0/2988709416 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557cda16d0e0 0x557cda187c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:03.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.175+0000 7f0754c00700 1 -- 172.21.15.151:0/2988709416 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557cda1881d0 con 0x557cda16c7c0 2024-05-21T22:05:03.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.175+0000 7f0754c00700 1 --2- 172.21.15.151:0/2988709416 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557cda16c7c0 0x557cda187750 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f073c0a6b10 tx=0x7f073c009290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce8b627307aeb298 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:05:03.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.175+0000 7f0737400700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-21T22:05:03.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.175+0000 7f0737400700 1 -- 172.21.15.151:0/2988709416 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f073c0aa570 con 0x557cda16c7c0 2024-05-21T22:05:03.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.176+0000 7f0737400700 1 -- 172.21.15.151:0/2988709416 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7f073c0d50e0 con 0x557cda16c7c0 2024-05-21T22:05:03.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.176+0000 7f07628ccb80 1 -- 172.21.15.151:0/2988709416 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0748003ba0 con 0x557cda16c7c0 2024-05-21T22:05:03.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.176+0000 7f0737400700 1 -- 172.21.15.151:0/2988709416 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f073c0ae920 con 0x557cda16c7c0 2024-05-21T22:05:03.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.176+0000 7f07628ccb80 10 client.17277 fetch_fsmap learned FSMap version 29 2024-05-21T22:05:03.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.176+0000 7f07628ccb80 1 -- 172.21.15.151:0/2988709416 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557cda1eae40 con 0x557cda16c7c0 2024-05-21T22:05:03.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.176+0000 7f0737400700 1 -- 172.21.15.151:0/2988709416 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0748003ba0 con 0x557cda16c7c0 2024-05-21T22:05:03.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.176+0000 7f0737400700 1 -- 172.21.15.151:0/2988709416 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f073c0b2eb0 con 0x557cda16c7c0 2024-05-21T22:05:03.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.176+0000 7f07628ccb80 10 client.17277 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:05:03.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.176+0000 7f07628ccb80 10 client.17277 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T22:05:03.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.176+0000 7f07628ccb80 20 client.17277 populate_metadata read hostname 'smithi151' 2024-05-21T22:05:03.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.176+0000 7f07628ccb80 1 --2- 172.21.15.151:0/2988709416 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x557cda1eb5a0 0x557cda1ed9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:03.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.176+0000 7f07628ccb80 4 client.17277 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T22:05:03.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.176+0000 7f07628ccb80 1 -- 172.21.15.151:0/2988709416 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557cda0c0570 con 0x557cda1eb5a0 2024-05-21T22:05:03.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.177+0000 7f0755600700 1 --2- 172.21.15.151:0/2988709416 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x557cda1eb5a0 0x557cda1ed9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:03.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.177+0000 7f0755600700 1 --2- 172.21.15.151:0/2988709416 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x557cda1eb5a0 0x557cda1ed9c0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2ddc0bd2dbe38e3 server_cookie=cce33987002218ad in_seq=0 out_seq=0 2024-05-21T22:05:03.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.177+0000 7f0737400700 10 client.17277 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T22:05:03.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.177+0000 7f0737400700 1 -- 172.21.15.151:0/2988709416 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557cda0c0570 con 0x557cda1eb5a0 2024-05-21T22:05:03.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.177+0000 7f0737400700 10 client.17277 handle_command_reply: tid=0 2024-05-21T22:05:03.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.177+0000 7f07628ccb80 1 client.17277 shutdown 2024-05-21T22:05:03.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.178+0000 7f07628ccb80 1 -- 172.21.15.151:0/2988709416 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557cda16c7c0 msgr2=0x557cda187750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:03.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.178+0000 7f07628ccb80 1 --2- 172.21.15.151:0/2988709416 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557cda16c7c0 0x557cda187750 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f073c0a6b10 tx=0x7f073c009290 comp rx=0 tx=0).stop 2024-05-21T22:05:03.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.178+0000 7f07628ccb80 1 -- 172.21.15.151:0/2988709416 shutdown_connections 2024-05-21T22:05:03.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.178+0000 7f07628ccb80 1 --2- 172.21.15.151:0/2988709416 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x557cda1eb5a0 0x557cda1ed9c0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:03.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.178+0000 7f07628ccb80 1 --2- 172.21.15.151:0/2988709416 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x557cda16d0e0 0x557cda187c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:03.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.178+0000 7f07628ccb80 1 --2- 172.21.15.151:0/2988709416 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557cda167390 0x557cda1792a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:03.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.178+0000 7f07628ccb80 1 --2- 172.21.15.151:0/2988709416 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557cda16c7c0 0x557cda187750 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:03.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.178+0000 7f0737400700 0 client.17277 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T22:05:03.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.178+0000 7f07628ccb80 1 -- 172.21.15.151:0/2988709416 >> 172.21.15.151:0/2988709416 conn(0x557cda094be0 msgr2=0x557cda16f220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:05:03.183 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.178+0000 7f07628ccb80 1 -- 172.21.15.151:0/2988709416 shutdown_connections 2024-05-21T22:05:03.183 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.178+0000 7f07628ccb80 1 -- 172.21.15.151:0/2988709416 wait complete. 2024-05-21T22:05:03.183 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.178+0000 7f07628ccb80 20 client.17277 trim_cache size 0 max 16384 2024-05-21T22:05:03.183 INFO:teuthology.orchestra.run.smithi151.stdout:[] 2024-05-21T22:05:03.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.179+0000 7f0756a00700 1 -- 172.21.15.151:0/3323980280 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f07380b29f0 msgr2=0x7f07380b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:03.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.179+0000 7f0756a00700 1 --2- 172.21.15.151:0/3323980280 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f07380b29f0 0x7f07380b4eb0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0748009ec0 tx=0x7f0748009920 comp rx=0 tx=0).stop 2024-05-21T22:05:03.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.179+0000 7f0756a00700 1 -- 172.21.15.151:0/3323980280 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0750078390 msgr2=0x7f07501a0730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:03.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.179+0000 7f0756a00700 1 --2- 172.21.15.151:0/3323980280 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0750078390 0x7f07501a0730 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f073c006010 tx=0x7f073c0057c0 comp rx=0 tx=0).stop 2024-05-21T22:05:03.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.179+0000 7f0756a00700 1 -- 172.21.15.151:0/3323980280 shutdown_connections 2024-05-21T22:05:03.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.179+0000 7f0756a00700 1 --2- 172.21.15.151:0/3323980280 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f07380b29f0 0x7f07380b4eb0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:03.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.179+0000 7f0756a00700 1 --2- 172.21.15.151:0/3323980280 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07501142f0 0x7f07501a4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:03.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.179+0000 7f0756a00700 1 --2- 172.21.15.151:0/3323980280 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0750077a70 0x7f07501a01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:03.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.179+0000 7f0756a00700 1 --2- 172.21.15.151:0/3323980280 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0750078390 0x7f07501a0730 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:03.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.179+0000 7f0756a00700 1 -- 172.21.15.151:0/3323980280 >> 172.21.15.151:0/3323980280 conn(0x7f0750101180 msgr2=0x7f0750103570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:05:03.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.179+0000 7f0756a00700 1 -- 172.21.15.151:0/3323980280 shutdown_connections 2024-05-21T22:05:03.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:03.179+0000 7f0756a00700 1 -- 172.21.15.151:0/3323980280 wait complete. 2024-05-21T22:05:03.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:05:03 smithi151 ceph-mon[31623]: pgmap v292: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-05-21T22:05:03.985 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:05:03 smithi184 ceph-mon[36942]: pgmap v292: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-05-21T22:05:04.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:05:03 smithi164 ceph-mon[36269]: pgmap v292: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-05-21T22:05:05.140 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-21T22:05:05.141 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-21T22:05:05.589 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:05:05.912 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:05:05 smithi151 ceph-mon[31623]: pgmap v293: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-05-21T22:05:05.985 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:05:05 smithi184 ceph-mon[36942]: pgmap v293: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-05-21T22:05:06.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:05:05 smithi164 ceph-mon[36269]: pgmap v293: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-05-21T22:05:07.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.801+0000 7f265f600700 1 -- 172.21.15.151:0/3988390859 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f265806bc70 msgr2=0x7f265806c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:07.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.801+0000 7f265f600700 1 --2- 172.21.15.151:0/3988390859 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f265806bc70 0x7f265806c0f0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f2644009990 tx=0x7f2644009ca0 comp rx=0 tx=0).stop 2024-05-21T22:05:07.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.802+0000 7f265f600700 1 -- 172.21.15.151:0/3988390859 shutdown_connections 2024-05-21T22:05:07.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.802+0000 7f265f600700 1 --2- 172.21.15.151:0/3988390859 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f265806c630 0x7f26581103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:07.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.802+0000 7f265f600700 1 --2- 172.21.15.151:0/3988390859 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f26581095b0 0x7f2658109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:07.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.802+0000 7f265f600700 1 --2- 172.21.15.151:0/3988390859 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f265806bc70 0x7f265806c0f0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:07.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.802+0000 7f265f600700 1 -- 172.21.15.151:0/3988390859 >> 172.21.15.151:0/3988390859 conn(0x7f26580793f0 msgr2=0x7f2658079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:05:07.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.802+0000 7f265f600700 1 -- 172.21.15.151:0/3988390859 shutdown_connections 2024-05-21T22:05:07.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.802+0000 7f265f600700 1 -- 172.21.15.151:0/3988390859 wait complete. 2024-05-21T22:05:07.805 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.803+0000 7f265f600700 1 Processor -- start 2024-05-21T22:05:07.805 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.803+0000 7f265f600700 1 -- start start 2024-05-21T22:05:07.805 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.804+0000 7f265f600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f265806bc70 0x7f2658102540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:07.805 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.804+0000 7f265f600700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f265806c630 0x7f2658102a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:07.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.804+0000 7f265f600700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f26581095b0 0x7f2658102fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:07.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.804+0000 7f265f600700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2658113730 con 0x7f265806bc70 2024-05-21T22:05:07.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.804+0000 7f265f600700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f26581135b0 con 0x7f26581095b0 2024-05-21T22:05:07.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.804+0000 7f265f600700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f26581138b0 con 0x7f265806c630 2024-05-21T22:05:07.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.804+0000 7f2657e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f265806c630 0x7f2658102a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:07.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.804+0000 7f265cc00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f265806bc70 0x7f2658102540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:07.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.804+0000 7f2657e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f265806c630 0x7f2658102a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:53424/0 (socket says 172.21.15.151:53424) 2024-05-21T22:05:07.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.804+0000 7f2657e00700 1 -- 172.21.15.151:0/152636319 learned_addr learned my addr 172.21.15.151:0/152636319 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:05:07.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.804+0000 7f265d600700 1 --2- 172.21.15.151:0/152636319 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f26581095b0 0x7f2658102fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:07.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.805+0000 7f2657e00700 1 -- 172.21.15.151:0/152636319 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f26581095b0 msgr2=0x7f2658102fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:07.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.805+0000 7f2657e00700 1 --2- 172.21.15.151:0/152636319 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f26581095b0 0x7f2658102fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:07.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.805+0000 7f2657e00700 1 -- 172.21.15.151:0/152636319 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f265806bc70 msgr2=0x7f2658102540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:07.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.805+0000 7f2657e00700 1 --2- 172.21.15.151:0/152636319 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f265806bc70 0x7f2658102540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:07.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.805+0000 7f2657e00700 1 -- 172.21.15.151:0/152636319 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26581a81f0 con 0x7f265806c630 2024-05-21T22:05:07.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.805+0000 7f2657e00700 1 --2- 172.21.15.151:0/152636319 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f265806c630 0x7f2658102a80 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f2644000c00 tx=0x7f26440057c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:05:07.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.805+0000 7f2655600700 1 -- 172.21.15.151:0/152636319 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2644027070 con 0x7f265806c630 2024-05-21T22:05:07.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.805+0000 7f2655600700 1 -- 172.21.15.151:0/152636319 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2644022470 con 0x7f265806c630 2024-05-21T22:05:07.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.805+0000 7f2655600700 1 -- 172.21.15.151:0/152636319 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2644017650 con 0x7f265806c630 2024-05-21T22:05:07.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.805+0000 7f265f600700 1 -- 172.21.15.151:0/152636319 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26581a8480 con 0x7f265806c630 2024-05-21T22:05:07.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.806+0000 7f265f600700 1 -- 172.21.15.151:0/152636319 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26581a8930 con 0x7f265806c630 2024-05-21T22:05:07.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.809+0000 7f2655600700 1 -- 172.21.15.151:0/152636319 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7f264402a070 con 0x7f265806c630 2024-05-21T22:05:07.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.810+0000 7f266a050b80 1 Processor -- start 2024-05-21T22:05:07.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.810+0000 7f266a050b80 1 -- start start 2024-05-21T22:05:07.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.811+0000 7f2655600700 1 --2- 172.21.15.151:0/152636319 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f26400b2a70 0x7f26400b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:07.813 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.811+0000 7f266a050b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563b2b6f2690 0x563b2b6f2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:07.813 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.811+0000 7f266a050b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b2b6f2fb0 0x563b2b6ecba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:07.813 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.811+0000 7f2655600700 1 -- 172.21.15.151:0/152636319 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7f26440e32d0 con 0x7f265806c630 2024-05-21T22:05:07.813 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.811+0000 7f266a050b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563b2b6ed1c0 0x563b2b6ed640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:07.813 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.811+0000 7f266a050b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x563b2b64aec0 con 0x563b2b6ed1c0 2024-05-21T22:05:07.813 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.811+0000 7f266a050b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x563b2b63aa60 con 0x563b2b6f2690 2024-05-21T22:05:07.813 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.811+0000 7f266a050b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x563b2b63eaa0 con 0x563b2b6f2fb0 2024-05-21T22:05:07.813 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.811+0000 7f265cc00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563b2b6f2690 0x563b2b6f2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:07.813 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.811+0000 7f265cc00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563b2b6f2690 0x563b2b6f2a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:51114/0 (socket says 172.21.15.151:51114) 2024-05-21T22:05:07.813 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.811+0000 7f2657e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b2b6f2fb0 0x563b2b6ecba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:07.813 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.811+0000 7f265cc00700 1 -- 172.21.15.151:0/2450477241 learned_addr learned my addr 172.21.15.151:0/2450477241 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:05:07.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.812+0000 7f2657e00700 1 -- 172.21.15.151:0/2450477241 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563b2b6f2690 msgr2=0x563b2b6f2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:07.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.812+0000 7f2657e00700 1 --2- 172.21.15.151:0/2450477241 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563b2b6f2690 0x563b2b6f2a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:07.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.812+0000 7f2657e00700 1 -- 172.21.15.151:0/2450477241 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563b2b6ed1c0 msgr2=0x563b2b6ed640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:07.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.812+0000 7f2657e00700 1 --2- 172.21.15.151:0/2450477241 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563b2b6ed1c0 0x563b2b6ed640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:07.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.812+0000 7f265d600700 1 --2- 172.21.15.151:0/152636319 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f26400b2a70 0x7f26400b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:07.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.812+0000 7f2657e00700 1 -- 172.21.15.151:0/2450477241 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b2b6034a0 con 0x563b2b6f2fb0 2024-05-21T22:05:07.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.812+0000 7f265cc00700 1 --2- 172.21.15.151:0/2450477241 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563b2b6f2690 0x563b2b6f2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:05:07.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.812+0000 7f2657e00700 1 --2- 172.21.15.151:0/2450477241 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b2b6f2fb0 0x563b2b6ecba0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f26440e09b0 tx=0x7f26440a9dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9610cc86798657da server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:05:07.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.812+0000 7f264a000700 1 -- 172.21.15.151:0/2450477241 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26440aa5d0 con 0x563b2b6f2fb0 2024-05-21T22:05:07.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.812+0000 7f264a000700 1 -- 172.21.15.151:0/2450477241 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26440ae980 con 0x563b2b6f2fb0 2024-05-21T22:05:07.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.812+0000 7f264a000700 1 -- 172.21.15.151:0/2450477241 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26440aa830 con 0x563b2b6f2fb0 2024-05-21T22:05:07.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.813+0000 7f266a050b80 1 -- 172.21.15.151:0/2450477241 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b2b6f2fb0 msgr2=0x563b2b6ecba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:07.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.813+0000 7f266a050b80 1 --2- 172.21.15.151:0/2450477241 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b2b6f2fb0 0x563b2b6ecba0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f26440e09b0 tx=0x7f26440a9dc0 comp rx=0 tx=0).stop 2024-05-21T22:05:07.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.813+0000 7f265d600700 1 --2- 172.21.15.151:0/152636319 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f26400b2a70 0x7f26400b4f30 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x563b2b6edb80 tx=0x7f2650009920 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:05:07.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.813+0000 7f266a050b80 1 -- 172.21.15.151:0/2450477241 shutdown_connections 2024-05-21T22:05:07.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.813+0000 7f266a050b80 1 --2- 172.21.15.151:0/2450477241 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563b2b6f2690 0x563b2b6f2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:07.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.813+0000 7f266a050b80 1 --2- 172.21.15.151:0/2450477241 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563b2b6ed1c0 0x563b2b6ed640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:07.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.813+0000 7f266a050b80 1 --2- 172.21.15.151:0/2450477241 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b2b6f2fb0 0x563b2b6ecba0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:07.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.813+0000 7f266a050b80 1 -- 172.21.15.151:0/2450477241 >> 172.21.15.151:0/2450477241 conn(0x563b2b61ac60 msgr2=0x563b2b6e7710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:05:07.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.813+0000 7f266a050b80 1 -- 172.21.15.151:0/2450477241 shutdown_connections 2024-05-21T22:05:07.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.814+0000 7f266a050b80 1 -- 172.21.15.151:0/2450477241 wait complete. 2024-05-21T22:05:07.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.814+0000 7f266a050b80 1 -- start start 2024-05-21T22:05:07.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.814+0000 7f266a050b80 1 Processor -- start 2024-05-21T22:05:07.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.815+0000 7f266a050b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563b2b6ed1c0 0x563b2b6f8e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:07.819 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.815+0000 7f266a050b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b2b6f2690 0x563b2b6f61f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:07.819 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.815+0000 7f266a050b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563b2b6f2fb0 0x563b2b6f6730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:07.819 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.815+0000 7f266a050b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x563b2b541020 con 0x563b2b6f2fb0 2024-05-21T22:05:07.819 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.815+0000 7f266a050b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x563b2b5db7e0 con 0x563b2b6ed1c0 2024-05-21T22:05:07.819 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.815+0000 7f266a050b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x563b2b6f9820 con 0x563b2b6f2690 2024-05-21T22:05:07.819 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.815+0000 7f266a050b80 10 client.0 _ll_register_callbacks cb 0x563b2b5e4cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:05:07.819 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.815+0000 7f265cc00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563b2b6ed1c0 0x563b2b6f8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:07.819 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.815+0000 7f2657e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b2b6f2690 0x563b2b6f61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:07.819 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.815+0000 7f2657e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b2b6f2690 0x563b2b6f61f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:53448/0 (socket says 172.21.15.151:53448) 2024-05-21T22:05:07.819 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.815+0000 7f265cc00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563b2b6ed1c0 0x563b2b6f8e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:51116/0 (socket says 172.21.15.151:51116) 2024-05-21T22:05:07.819 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.815+0000 7f2657e00700 1 -- 172.21.15.151:0/4044401602 learned_addr learned my addr 172.21.15.151:0/4044401602 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:05:07.819 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.815+0000 7f265d600700 1 --2- 172.21.15.151:0/4044401602 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563b2b6f2fb0 0x563b2b6f6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:07.820 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.816+0000 7f2657e00700 1 -- 172.21.15.151:0/4044401602 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563b2b6ed1c0 msgr2=0x563b2b6f8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:07.820 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.816+0000 7f2657e00700 1 --2- 172.21.15.151:0/4044401602 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563b2b6ed1c0 0x563b2b6f8e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:07.820 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.816+0000 7f2657e00700 1 -- 172.21.15.151:0/4044401602 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563b2b6f2fb0 msgr2=0x563b2b6f6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:07.820 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.816+0000 7f2657e00700 1 --2- 172.21.15.151:0/4044401602 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563b2b6f2fb0 0x563b2b6f6730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:07.820 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.816+0000 7f2657e00700 1 -- 172.21.15.151:0/4044401602 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b2b58a950 con 0x563b2b6f2690 2024-05-21T22:05:07.820 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.816+0000 7f265d600700 1 --2- 172.21.15.151:0/4044401602 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563b2b6f2fb0 0x563b2b6f6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T22:05:07.820 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.816+0000 7f2657e00700 1 --2- 172.21.15.151:0/4044401602 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b2b6f2690 0x563b2b6f61f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f26440aa560 tx=0x7f26440096e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f6505d4a530c4f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:05:07.820 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.816+0000 7f263ea00700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-21T22:05:07.820 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.816+0000 7f263ea00700 1 -- 172.21.15.151:0/4044401602 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26440c3070 con 0x563b2b6f2690 2024-05-21T22:05:07.820 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.816+0000 7f263ea00700 1 -- 172.21.15.151:0/4044401602 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7f2644022920 con 0x563b2b6f2690 2024-05-21T22:05:07.820 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.816+0000 7f266a050b80 1 -- 172.21.15.151:0/4044401602 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26440ae980 con 0x563b2b6f2690 2024-05-21T22:05:07.820 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.816+0000 7f263ea00700 1 -- 172.21.15.151:0/4044401602 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f26440cb370 con 0x563b2b6f2690 2024-05-21T22:05:07.820 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.816+0000 7f266a050b80 10 client.27415 fetch_fsmap learned FSMap version 29 2024-05-21T22:05:07.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.816+0000 7f266a050b80 1 -- 172.21.15.151:0/4044401602 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563b2b60b0e0 con 0x563b2b6f2690 2024-05-21T22:05:07.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.817+0000 7f263ea00700 1 -- 172.21.15.151:0/4044401602 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26440ae980 con 0x563b2b6f2690 2024-05-21T22:05:07.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.817+0000 7f263ea00700 1 -- 172.21.15.151:0/4044401602 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f26440b2fa0 con 0x563b2b6f2690 2024-05-21T22:05:07.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.817+0000 7f266a050b80 10 client.27415 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:05:07.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.817+0000 7f266a050b80 10 client.27415 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T22:05:07.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.817+0000 7f266a050b80 20 client.27415 populate_metadata read hostname 'smithi151' 2024-05-21T22:05:07.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.817+0000 7f266a050b80 1 --2- 172.21.15.151:0/4044401602 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x563b2b7062a0 0x563b2b708690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:07.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.817+0000 7f266a050b80 4 client.27415 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T22:05:07.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.817+0000 7f266a050b80 1 -- 172.21.15.151:0/4044401602 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563b2b64ce10 con 0x563b2b7062a0 2024-05-21T22:05:07.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.817+0000 7f265cc00700 1 --2- 172.21.15.151:0/4044401602 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x563b2b7062a0 0x563b2b708690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:07.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.818+0000 7f265cc00700 1 --2- 172.21.15.151:0/4044401602 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x563b2b7062a0 0x563b2b708690 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=567da003c933ac79 server_cookie=552a445c3f347632 in_seq=0 out_seq=0 2024-05-21T22:05:07.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.818+0000 7f263ea00700 10 client.27415 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T22:05:07.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.819+0000 7f263ea00700 1 -- 172.21.15.151:0/4044401602 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x563b2b64ce10 con 0x563b2b7062a0 2024-05-21T22:05:07.823 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.819+0000 7f263ea00700 10 client.27415 handle_command_reply: tid=0 2024-05-21T22:05:07.823 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.819+0000 7f266a050b80 1 client.27415 shutdown 2024-05-21T22:05:07.823 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.819+0000 7f266a050b80 1 -- 172.21.15.151:0/4044401602 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b2b6f2690 msgr2=0x563b2b6f61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:07.823 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.819+0000 7f266a050b80 1 --2- 172.21.15.151:0/4044401602 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b2b6f2690 0x563b2b6f61f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f26440aa560 tx=0x7f26440096e0 comp rx=0 tx=0).stop 2024-05-21T22:05:07.823 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.819+0000 7f266a050b80 1 -- 172.21.15.151:0/4044401602 shutdown_connections 2024-05-21T22:05:07.823 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.819+0000 7f266a050b80 1 --2- 172.21.15.151:0/4044401602 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x563b2b7062a0 0x563b2b708690 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:07.823 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.819+0000 7f266a050b80 1 --2- 172.21.15.151:0/4044401602 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563b2b6ed1c0 0x563b2b6f8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:07.823 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.819+0000 7f266a050b80 1 --2- 172.21.15.151:0/4044401602 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563b2b6f2fb0 0x563b2b6f6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:07.823 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.819+0000 7f266a050b80 1 --2- 172.21.15.151:0/4044401602 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b2b6f2690 0x563b2b6f61f0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:07.823 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.819+0000 7f266a050b80 1 -- 172.21.15.151:0/4044401602 >> 172.21.15.151:0/4044401602 conn(0x563b2b61ac60 msgr2=0x563b2b6e6df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:05:07.823 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.819+0000 7f263ea00700 0 client.27415 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T22:05:07.823 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.819+0000 7f266a050b80 1 -- 172.21.15.151:0/4044401602 shutdown_connections 2024-05-21T22:05:07.823 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.819+0000 7f266a050b80 1 -- 172.21.15.151:0/4044401602 wait complete. 2024-05-21T22:05:07.823 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.819+0000 7f266a050b80 20 client.27415 trim_cache size 0 max 16384 2024-05-21T22:05:07.843 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.842+0000 7f266a050b80 1 Processor -- start 2024-05-21T22:05:07.843 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.842+0000 7f266a050b80 1 -- start start 2024-05-21T22:05:07.843 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.842+0000 7f266a050b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563b2b6ed1c0 0x563b2b6ecba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:07.844 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.842+0000 7f266a050b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563b2b6f2690 0x563b2b6e7570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:07.844 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.842+0000 7f266a050b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b2b6f2fb0 0x563b2b6f3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:07.844 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.842+0000 7f266a050b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f26581affe0 con 0x563b2b6f2690 2024-05-21T22:05:07.844 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.842+0000 7f266a050b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f26581b1b20 con 0x563b2b6ed1c0 2024-05-21T22:05:07.844 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.842+0000 7f266a050b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f26581b06b0 con 0x563b2b6f2fb0 2024-05-21T22:05:07.845 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.842+0000 7f265cc00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563b2b6ed1c0 0x563b2b6ecba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:07.845 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.842+0000 7f265d600700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b2b6f2fb0 0x563b2b6f3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:07.845 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.843+0000 7f2657e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563b2b6f2690 0x563b2b6e7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:07.845 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.843+0000 7f265cc00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563b2b6ed1c0 0x563b2b6ecba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:51118/0 (socket says 172.21.15.151:51118) 2024-05-21T22:05:07.845 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.843+0000 7f265cc00700 1 -- 172.21.15.151:0/1117713632 learned_addr learned my addr 172.21.15.151:0/1117713632 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:05:07.845 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.843+0000 7f265cc00700 1 -- 172.21.15.151:0/1117713632 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b2b6f2fb0 msgr2=0x563b2b6f3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:07.845 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.843+0000 7f265cc00700 1 --2- 172.21.15.151:0/1117713632 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b2b6f2fb0 0x563b2b6f3ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:07.845 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.843+0000 7f265cc00700 1 -- 172.21.15.151:0/1117713632 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563b2b6f2690 msgr2=0x563b2b6e7570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:07.845 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.843+0000 7f265cc00700 1 --2- 172.21.15.151:0/1117713632 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563b2b6f2690 0x563b2b6e7570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:07.845 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.843+0000 7f265cc00700 1 -- 172.21.15.151:0/1117713632 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b2b60b0e0 con 0x563b2b6ed1c0 2024-05-21T22:05:07.845 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.843+0000 7f265cc00700 1 --2- 172.21.15.151:0/1117713632 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563b2b6ed1c0 0x563b2b6ecba0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f264c005c20 tx=0x7f264c009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=877e3d65c3eb065d server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:05:07.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.843+0000 7f2649600700 1 -- 172.21.15.151:0/1117713632 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f264c003b40 con 0x563b2b6ed1c0 2024-05-21T22:05:07.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.844+0000 7f2649600700 1 -- 172.21.15.151:0/1117713632 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f264c004ce0 con 0x563b2b6ed1c0 2024-05-21T22:05:07.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.844+0000 7f2649600700 1 -- 172.21.15.151:0/1117713632 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f264c005620 con 0x563b2b6ed1c0 2024-05-21T22:05:07.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.844+0000 7f266a050b80 1 -- 172.21.15.151:0/1117713632 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563b2b6ed1c0 msgr2=0x563b2b6ecba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:07.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.844+0000 7f266a050b80 1 --2- 172.21.15.151:0/1117713632 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563b2b6ed1c0 0x563b2b6ecba0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f264c005c20 tx=0x7f264c009260 comp rx=0 tx=0).stop 2024-05-21T22:05:07.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.844+0000 7f266a050b80 1 -- 172.21.15.151:0/1117713632 shutdown_connections 2024-05-21T22:05:07.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.844+0000 7f266a050b80 1 --2- 172.21.15.151:0/1117713632 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563b2b6ed1c0 0x563b2b6ecba0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:07.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.844+0000 7f266a050b80 1 --2- 172.21.15.151:0/1117713632 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b2b6f2fb0 0x563b2b6f3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:07.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.844+0000 7f266a050b80 1 --2- 172.21.15.151:0/1117713632 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563b2b6f2690 0x563b2b6e7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:07.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.844+0000 7f266a050b80 1 -- 172.21.15.151:0/1117713632 >> 172.21.15.151:0/1117713632 conn(0x563b2b61ac60 msgr2=0x563b2b6f1930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:05:07.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.844+0000 7f266a050b80 1 -- 172.21.15.151:0/1117713632 shutdown_connections 2024-05-21T22:05:07.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.844+0000 7f266a050b80 1 -- 172.21.15.151:0/1117713632 wait complete. 2024-05-21T22:05:07.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.845+0000 7f266a050b80 1 -- start start 2024-05-21T22:05:07.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.845+0000 7f266a050b80 1 Processor -- start 2024-05-21T22:05:07.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.846+0000 7f266a050b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563b2b6ed1c0 0x563b2b708fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:07.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.846+0000 7f266a050b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563b2b6f2690 0x563b2b709520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:07.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.846+0000 7f266a050b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b2b6f2fb0 0x563b2b709a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:07.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.846+0000 7f266a050b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f26581b1450 con 0x563b2b6ed1c0 2024-05-21T22:05:07.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.846+0000 7f266a050b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f26581b0d80 con 0x563b2b6f2690 2024-05-21T22:05:07.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.846+0000 7f266a050b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f26581adfa0 con 0x563b2b6f2fb0 2024-05-21T22:05:07.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.846+0000 7f266a050b80 10 client.0 _ll_register_callbacks cb 0x563b2b5e4cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:05:07.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.846+0000 7f265cc00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563b2b6ed1c0 0x563b2b708fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:07.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.846+0000 7f2657e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563b2b6f2690 0x563b2b709520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:07.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.846+0000 7f2657e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563b2b6f2690 0x563b2b709520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:51124/0 (socket says 172.21.15.151:51124) 2024-05-21T22:05:07.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.846+0000 7f2657e00700 1 -- 172.21.15.151:0/2590448576 learned_addr learned my addr 172.21.15.151:0/2590448576 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:05:07.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.846+0000 7f265d600700 1 --2- 172.21.15.151:0/2590448576 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b2b6f2fb0 0x563b2b709a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:07.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.846+0000 7f2657e00700 1 -- 172.21.15.151:0/2590448576 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b2b6f2fb0 msgr2=0x563b2b709a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:07.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.846+0000 7f2657e00700 1 --2- 172.21.15.151:0/2590448576 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b2b6f2fb0 0x563b2b709a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:07.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.846+0000 7f2657e00700 1 -- 172.21.15.151:0/2590448576 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563b2b6ed1c0 msgr2=0x563b2b708fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:07.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.846+0000 7f2657e00700 1 --2- 172.21.15.151:0/2590448576 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563b2b6ed1c0 0x563b2b708fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:07.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.846+0000 7f2657e00700 1 -- 172.21.15.151:0/2590448576 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b2b6f2cc0 con 0x563b2b6f2690 2024-05-21T22:05:07.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.847+0000 7f2657e00700 1 --2- 172.21.15.151:0/2590448576 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563b2b6f2690 0x563b2b709520 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f26440b3310 tx=0x7f2644005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ccd48394f4c6aaaa server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:05:07.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.847+0000 7f263ea00700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-21T22:05:07.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.847+0000 7f263ea00700 1 -- 172.21.15.151:0/2590448576 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26440c3050 con 0x563b2b6f2690 2024-05-21T22:05:07.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.847+0000 7f263ea00700 1 -- 172.21.15.151:0/2590448576 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7f26440a6bc0 con 0x563b2b6f2690 2024-05-21T22:05:07.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.847+0000 7f266a050b80 1 -- 172.21.15.151:0/2590448576 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f264c004ce0 con 0x563b2b6f2690 2024-05-21T22:05:07.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.847+0000 7f263ea00700 1 -- 172.21.15.151:0/2590448576 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f26440ae980 con 0x563b2b6f2690 2024-05-21T22:05:07.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.847+0000 7f266a050b80 10 client.27419 fetch_fsmap learned FSMap version 29 2024-05-21T22:05:07.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.847+0000 7f266a050b80 1 -- 172.21.15.151:0/2590448576 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563b2b6f2cc0 con 0x563b2b6f2690 2024-05-21T22:05:07.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.847+0000 7f263ea00700 1 -- 172.21.15.151:0/2590448576 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f264c004ce0 con 0x563b2b6f2690 2024-05-21T22:05:07.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.848+0000 7f263ea00700 1 -- 172.21.15.151:0/2590448576 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f26440aa830 con 0x563b2b6f2690 2024-05-21T22:05:07.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.848+0000 7f266a050b80 10 client.27419 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:05:07.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.848+0000 7f266a050b80 10 client.27419 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T22:05:07.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.848+0000 7f266a050b80 20 client.27419 populate_metadata read hostname 'smithi151' 2024-05-21T22:05:07.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.848+0000 7f266a050b80 1 --2- 172.21.15.151:0/2590448576 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x563b2b779740 0x563b2b77bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:07.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.848+0000 7f266a050b80 4 client.27419 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-21T22:05:07.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.848+0000 7f266a050b80 1 -- 172.21.15.151:0/2590448576 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563b2b6465f0 con 0x563b2b779740 2024-05-21T22:05:07.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.848+0000 7f265cc00700 1 --2- 172.21.15.151:0/2590448576 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x563b2b779740 0x563b2b77bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:07.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.849+0000 7f265cc00700 1 --2- 172.21.15.151:0/2590448576 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x563b2b779740 0x563b2b77bb60 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3051e2dd5ec3a164 server_cookie=a6aadacb9b5c1979 in_seq=0 out_seq=0 2024-05-21T22:05:07.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.849+0000 7f263ea00700 10 client.27419 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T22:05:07.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.850+0000 7f263ea00700 1 -- 172.21.15.151:0/2590448576 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563b2b6465f0 con 0x563b2b779740 2024-05-21T22:05:07.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.850+0000 7f263ea00700 10 client.27419 handle_command_reply: tid=0 2024-05-21T22:05:07.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.850+0000 7f266a050b80 1 client.27419 shutdown 2024-05-21T22:05:07.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.850+0000 7f266a050b80 1 -- 172.21.15.151:0/2590448576 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563b2b6f2690 msgr2=0x563b2b709520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:07.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.850+0000 7f266a050b80 1 --2- 172.21.15.151:0/2590448576 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563b2b6f2690 0x563b2b709520 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f26440b3310 tx=0x7f2644005e80 comp rx=0 tx=0).stop 2024-05-21T22:05:07.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.850+0000 7f266a050b80 1 -- 172.21.15.151:0/2590448576 shutdown_connections 2024-05-21T22:05:07.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.850+0000 7f266a050b80 1 --2- 172.21.15.151:0/2590448576 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b2b6f2fb0 0x563b2b709a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:07.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.850+0000 7f266a050b80 1 --2- 172.21.15.151:0/2590448576 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563b2b6ed1c0 0x563b2b708fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:07.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.850+0000 7f266a050b80 1 --2- 172.21.15.151:0/2590448576 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x563b2b779740 0x563b2b77bb60 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:07.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.850+0000 7f266a050b80 1 --2- 172.21.15.151:0/2590448576 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563b2b6f2690 0x563b2b709520 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:07.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.850+0000 7f266a050b80 1 -- 172.21.15.151:0/2590448576 >> 172.21.15.151:0/2590448576 conn(0x563b2b61ac60 msgr2=0x563b2b6f3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:05:07.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.851+0000 7f263ea00700 0 client.27419 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T22:05:07.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.851+0000 7f266a050b80 1 -- 172.21.15.151:0/2590448576 shutdown_connections 2024-05-21T22:05:07.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.851+0000 7f266a050b80 1 -- 172.21.15.151:0/2590448576 wait complete. 2024-05-21T22:05:07.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.851+0000 7f266a050b80 20 client.27419 trim_cache size 0 max 16384 2024-05-21T22:05:07.855 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-05-21T22:05:07.855 INFO:teuthology.orchestra.run.smithi151.stdout: "return_code": 0, 2024-05-21T22:05:07.855 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_tag": "bdba9d63-5068-4494-a692-44fa1be4fe98", 2024-05-21T22:05:07.855 INFO:teuthology.orchestra.run.smithi151.stdout: "mode": "asynchronous" 2024-05-21T22:05:07.855 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-05-21T22:05:07.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.852+0000 7f265f600700 1 -- 172.21.15.151:0/152636319 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f26400b2a70 msgr2=0x7f26400b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:07.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.852+0000 7f265f600700 1 --2- 172.21.15.151:0/152636319 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f26400b2a70 0x7f26400b4f30 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x563b2b6edb80 tx=0x7f2650009920 comp rx=0 tx=0).stop 2024-05-21T22:05:07.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.852+0000 7f265f600700 1 -- 172.21.15.151:0/152636319 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f265806c630 msgr2=0x7f2658102a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:07.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.852+0000 7f265f600700 1 --2- 172.21.15.151:0/152636319 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f265806c630 0x7f2658102a80 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f2644000c00 tx=0x7f26440057c0 comp rx=0 tx=0).stop 2024-05-21T22:05:07.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.852+0000 7f265f600700 1 -- 172.21.15.151:0/152636319 shutdown_connections 2024-05-21T22:05:07.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.852+0000 7f265f600700 1 --2- 172.21.15.151:0/152636319 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f26400b2a70 0x7f26400b4f30 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:07.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.852+0000 7f265f600700 1 --2- 172.21.15.151:0/152636319 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f26581095b0 0x7f2658102fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:07.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.852+0000 7f265f600700 1 --2- 172.21.15.151:0/152636319 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f265806bc70 0x7f2658102540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:07.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.852+0000 7f265f600700 1 --2- 172.21.15.151:0/152636319 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f265806c630 0x7f2658102a80 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:07.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.852+0000 7f265f600700 1 -- 172.21.15.151:0/152636319 >> 172.21.15.151:0/152636319 conn(0x7f26580793f0 msgr2=0x7f2658101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:05:07.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.852+0000 7f265f600700 1 -- 172.21.15.151:0/152636319 shutdown_connections 2024-05-21T22:05:07.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:07.852+0000 7f265f600700 1 -- 172.21.15.151:0/152636319 wait complete. 2024-05-21T22:05:07.869 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:05:07 smithi151 ceph-mon[31623]: pgmap v294: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-05-21T22:05:07.870 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:05:07 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T22:05:07.870 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:05:07 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T22:05:07.870 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:05:07 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T22:05:07.870 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:05:07 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T22:05:07.870 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:05:07 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T22:05:07.985 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:05:07 smithi184 ceph-mon[36942]: pgmap v294: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-05-21T22:05:07.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:05:07 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T22:05:07.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:05:07 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T22:05:07.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:05:07 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T22:05:07.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:05:07 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T22:05:07.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:05:07 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T22:05:08.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:05:07 smithi164 ceph-mon[36269]: pgmap v294: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-05-21T22:05:08.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:05:07 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T22:05:08.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:05:07 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T22:05:08.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:05:07 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T22:05:08.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:05:07 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T22:05:08.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:05:07 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T22:05:08.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:05:08 smithi151 ceph-mon[31623]: scrub queued for path: / 2024-05-21T22:05:08.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:05:08 smithi151 ceph-mon[31623]: scrub summary: idle+waiting paths [/] 2024-05-21T22:05:08.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:05:08 smithi151 ceph-mon[31623]: scrub summary: active paths [/] 2024-05-21T22:05:08.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:05:08 smithi151 ceph-mon[31623]: scrub summary: idle+waiting paths [/] 2024-05-21T22:05:08.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:05:08 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T22:05:08.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:05:08 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T22:05:08.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:05:08 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T22:05:08.878 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-21T22:05:08.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:05:08 smithi184 ceph-mon[36942]: scrub queued for path: / 2024-05-21T22:05:08.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:05:08 smithi184 ceph-mon[36942]: scrub summary: idle+waiting paths [/] 2024-05-21T22:05:08.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:05:08 smithi184 ceph-mon[36942]: scrub summary: active paths [/] 2024-05-21T22:05:08.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:05:08 smithi184 ceph-mon[36942]: scrub summary: idle+waiting paths [/] 2024-05-21T22:05:08.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:05:08 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T22:05:08.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:05:08 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T22:05:08.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:05:08 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T22:05:09.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:05:08 smithi164 ceph-mon[36269]: scrub queued for path: / 2024-05-21T22:05:09.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:05:08 smithi164 ceph-mon[36269]: scrub summary: idle+waiting paths [/] 2024-05-21T22:05:09.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:05:08 smithi164 ceph-mon[36269]: scrub summary: active paths [/] 2024-05-21T22:05:09.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:05:08 smithi164 ceph-mon[36269]: scrub summary: idle+waiting paths [/] 2024-05-21T22:05:09.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:05:08 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T22:05:09.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:05:08 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T22:05:09.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:05:08 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T22:05:09.330 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:05:09.801 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:05:09 smithi151 ceph-mon[31623]: pgmap v295: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-05-21T22:05:09.801 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:05:09 smithi151 ceph-mon[31623]: scrub summary: idle 2024-05-21T22:05:09.937 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:05:09 smithi164 ceph-mon[36269]: pgmap v295: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-05-21T22:05:09.937 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:05:09 smithi164 ceph-mon[36269]: scrub summary: idle 2024-05-21T22:05:09.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:05:09 smithi184 ceph-mon[36942]: pgmap v295: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-05-21T22:05:09.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:05:09 smithi184 ceph-mon[36942]: scrub summary: idle 2024-05-21T22:05:11.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.522+0000 7f5890a00700 1 -- 172.21.15.151:0/877006779 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f588c1088f0 msgr2=0x7f588c110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:11.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.522+0000 7f5890a00700 1 --2- 172.21.15.151:0/877006779 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f588c1088f0 0x7f588c110470 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f5884009fc0 tx=0x7f588400b330 comp rx=0 tx=0).stop 2024-05-21T22:05:11.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.522+0000 7f5890a00700 1 -- 172.21.15.151:0/877006779 shutdown_connections 2024-05-21T22:05:11.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.522+0000 7f5890a00700 1 --2- 172.21.15.151:0/877006779 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f588c1088f0 0x7f588c110470 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:11.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.522+0000 7f5890a00700 1 --2- 172.21.15.151:0/877006779 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f588c1037f0 0x7f588c103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:11.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.522+0000 7f5890a00700 1 --2- 172.21.15.151:0/877006779 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f588c1041a0 0x7f588c1081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:11.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.522+0000 7f5890a00700 1 -- 172.21.15.151:0/877006779 >> 172.21.15.151:0/877006779 conn(0x7f588c07a6a0 msgr2=0x7f588c07aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:05:11.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.522+0000 7f5890a00700 1 -- 172.21.15.151:0/877006779 shutdown_connections 2024-05-21T22:05:11.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.522+0000 7f5890a00700 1 -- 172.21.15.151:0/877006779 wait complete. 2024-05-21T22:05:11.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.523+0000 7f5890a00700 1 Processor -- start 2024-05-21T22:05:11.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.523+0000 7f5890a00700 1 -- start start 2024-05-21T22:05:11.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.524+0000 7f5890a00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f588c1037f0 0x7f588c19bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:11.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.524+0000 7f5890a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f588c1041a0 0x7f588c19c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:11.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.524+0000 7f5890a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f588c1088f0 0x7f588c1a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:11.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.524+0000 7f5890a00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f588c113730 con 0x7f588c1041a0 2024-05-21T22:05:11.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.524+0000 7f5890a00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f588c1135b0 con 0x7f588c1088f0 2024-05-21T22:05:11.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.524+0000 7f5890a00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f588c1138b0 con 0x7f588c1037f0 2024-05-21T22:05:11.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.524+0000 7f5889400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f588c1041a0 0x7f588c19c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:11.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.524+0000 7f5889400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f588c1041a0 0x7f588c19c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:46860/0 (socket says 172.21.15.151:46860) 2024-05-21T22:05:11.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.524+0000 7f5889400700 1 -- 172.21.15.151:0/2748545783 learned_addr learned my addr 172.21.15.151:0/2748545783 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:05:11.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.524+0000 7f5889e00700 1 --2- 172.21.15.151:0/2748545783 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f588c1037f0 0x7f588c19bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:11.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.524+0000 7f588a800700 1 --2- 172.21.15.151:0/2748545783 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f588c1088f0 0x7f588c1a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:11.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.525+0000 7f5889e00700 1 -- 172.21.15.151:0/2748545783 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f588c1088f0 msgr2=0x7f588c1a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:11.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.525+0000 7f5889e00700 1 --2- 172.21.15.151:0/2748545783 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f588c1088f0 0x7f588c1a0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:11.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.525+0000 7f5889e00700 1 -- 172.21.15.151:0/2748545783 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f588c1041a0 msgr2=0x7f588c19c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:11.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.525+0000 7f5889e00700 1 --2- 172.21.15.151:0/2748545783 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f588c1041a0 0x7f588c19c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:11.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.525+0000 7f5889e00700 1 -- 172.21.15.151:0/2748545783 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f588c1a0d50 con 0x7f588c1037f0 2024-05-21T22:05:11.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.525+0000 7f588a800700 1 --2- 172.21.15.151:0/2748545783 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f588c1088f0 0x7f588c1a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:05:11.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.525+0000 7f5889400700 1 --2- 172.21.15.151:0/2748545783 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f588c1041a0 0x7f588c19c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:05:11.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.525+0000 7f5889e00700 1 --2- 172.21.15.151:0/2748545783 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f588c1037f0 0x7f588c19bd40 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f587c00e9c0 tx=0x7f587c00ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:05:11.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.525+0000 7f5882a00700 1 -- 172.21.15.151:0/2748545783 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f587c00c9c0 con 0x7f588c1037f0 2024-05-21T22:05:11.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.525+0000 7f5890a00700 1 -- 172.21.15.151:0/2748545783 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f588c1a1040 con 0x7f588c1037f0 2024-05-21T22:05:11.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.525+0000 7f5882a00700 1 -- 172.21.15.151:0/2748545783 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f587c00cb60 con 0x7f588c1037f0 2024-05-21T22:05:11.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.525+0000 7f5882a00700 1 -- 172.21.15.151:0/2748545783 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f587c018420 con 0x7f588c1037f0 2024-05-21T22:05:11.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.525+0000 7f5890a00700 1 -- 172.21.15.151:0/2748545783 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f588c1a88c0 con 0x7f588c1037f0 2024-05-21T22:05:11.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.529+0000 7f5882a00700 1 -- 172.21.15.151:0/2748545783 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7f587c010430 con 0x7f588c1037f0 2024-05-21T22:05:11.531 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.530+0000 7f589b3c6b80 1 Processor -- start 2024-05-21T22:05:11.531 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.530+0000 7f589b3c6b80 1 -- start start 2024-05-21T22:05:11.532 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.530+0000 7f589b3c6b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1af9ff610 0x55b1af9ff9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:11.532 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.530+0000 7f589b3c6b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b1af9fff30 0x55b1af9f9b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:11.532 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.530+0000 7f589b3c6b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b1af9fa140 0x55b1af9fa5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:11.532 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.530+0000 7f589b3c6b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55b1af957e40 con 0x55b1af9fa140 2024-05-21T22:05:11.532 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.530+0000 7f589b3c6b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55b1af9479e0 con 0x55b1af9ff610 2024-05-21T22:05:11.532 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.531+0000 7f589b3c6b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55b1af94ba20 con 0x55b1af9fff30 2024-05-21T22:05:11.533 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.531+0000 7f5889e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b1af9fff30 0x55b1af9f9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:11.533 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.531+0000 7f5889400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b1af9fa140 0x55b1af9fa5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:11.533 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.531+0000 7f5889e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b1af9fff30 0x55b1af9f9b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:53486/0 (socket says 172.21.15.151:53486) 2024-05-21T22:05:11.533 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.531+0000 7f588a800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1af9ff610 0x55b1af9ff9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:11.533 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.531+0000 7f5889e00700 1 -- 172.21.15.151:0/2713373238 learned_addr learned my addr 172.21.15.151:0/2713373238 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:05:11.533 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.531+0000 7f5889e00700 1 -- 172.21.15.151:0/2713373238 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1af9ff610 msgr2=0x55b1af9ff9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:11.533 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.531+0000 7f5889e00700 1 --2- 172.21.15.151:0/2713373238 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1af9ff610 0x55b1af9ff9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:11.533 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.531+0000 7f5889e00700 1 -- 172.21.15.151:0/2713373238 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b1af9fa140 msgr2=0x55b1af9fa5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:11.533 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.531+0000 7f5889e00700 1 --2- 172.21.15.151:0/2713373238 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b1af9fa140 0x55b1af9fa5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:11.533 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.531+0000 7f5889e00700 1 -- 172.21.15.151:0/2713373238 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1af910420 con 0x55b1af9fff30 2024-05-21T22:05:11.533 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.532+0000 7f5882a00700 1 --2- 172.21.15.151:0/2748545783 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f58700b29f0 0x7f58700b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:11.534 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.532+0000 7f5889e00700 1 --2- 172.21.15.151:0/2713373238 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b1af9fff30 0x55b1af9f9b20 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f587c0dfe20 tx=0x7f587c0e8ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e33063336ee4125 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:05:11.534 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.532+0000 7f588a800700 1 --2- 172.21.15.151:0/2748545783 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f58700b29f0 0x7f58700b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:11.534 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.532+0000 7f5882a00700 1 -- 172.21.15.151:0/2748545783 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7f587c0e2800 con 0x7f588c1037f0 2024-05-21T22:05:11.534 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.532+0000 7f5877400700 1 -- 172.21.15.151:0/2713373238 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f587c0a5930 con 0x55b1af9fff30 2024-05-21T22:05:11.534 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.532+0000 7f5877400700 1 -- 172.21.15.151:0/2713373238 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f587c0a51f0 con 0x55b1af9fff30 2024-05-21T22:05:11.534 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.532+0000 7f5877400700 1 -- 172.21.15.151:0/2713373238 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f587c0a91e0 con 0x55b1af9fff30 2024-05-21T22:05:11.535 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.533+0000 7f589b3c6b80 1 -- 172.21.15.151:0/2713373238 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b1af9fff30 msgr2=0x55b1af9f9b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:11.536 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.533+0000 7f588a800700 1 --2- 172.21.15.151:0/2748545783 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f58700b29f0 0x7f58700b4eb0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f588400b6c0 tx=0x7f5884009f30 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:05:11.536 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.533+0000 7f589b3c6b80 1 --2- 172.21.15.151:0/2713373238 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b1af9fff30 0x55b1af9f9b20 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f587c0dfe20 tx=0x7f587c0e8ed0 comp rx=0 tx=0).stop 2024-05-21T22:05:11.536 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.534+0000 7f589b3c6b80 1 -- 172.21.15.151:0/2713373238 shutdown_connections 2024-05-21T22:05:11.536 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.534+0000 7f589b3c6b80 1 --2- 172.21.15.151:0/2713373238 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1af9ff610 0x55b1af9ff9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:11.536 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.534+0000 7f589b3c6b80 1 --2- 172.21.15.151:0/2713373238 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b1af9fa140 0x55b1af9fa5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:11.536 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.534+0000 7f589b3c6b80 1 --2- 172.21.15.151:0/2713373238 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b1af9fff30 0x55b1af9f9b20 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:11.536 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.534+0000 7f589b3c6b80 1 -- 172.21.15.151:0/2713373238 >> 172.21.15.151:0/2713373238 conn(0x55b1af927be0 msgr2=0x55b1af9f4690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:05:11.536 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.534+0000 7f589b3c6b80 1 -- 172.21.15.151:0/2713373238 shutdown_connections 2024-05-21T22:05:11.536 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.534+0000 7f589b3c6b80 1 -- 172.21.15.151:0/2713373238 wait complete. 2024-05-21T22:05:11.536 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.534+0000 7f589b3c6b80 1 -- start start 2024-05-21T22:05:11.536 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.535+0000 7f589b3c6b80 1 Processor -- start 2024-05-21T22:05:11.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.535+0000 7f589b3c6b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b1af9fa140 0x55b1afa05c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:11.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.535+0000 7f589b3c6b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b1af9ff610 0x55b1afa02fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:11.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.535+0000 7f589b3c6b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1af9fff30 0x55b1afa03500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:11.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.535+0000 7f589b3c6b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55b1af84dfa0 con 0x55b1af9ff610 2024-05-21T22:05:11.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.535+0000 7f589b3c6b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55b1af8e8760 con 0x55b1af9fff30 2024-05-21T22:05:11.538 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.535+0000 7f589b3c6b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55b1afa069e0 con 0x55b1af9fa140 2024-05-21T22:05:11.538 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.535+0000 7f589b3c6b80 10 client.0 _ll_register_callbacks cb 0x55b1af8f1c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:05:11.538 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.535+0000 7f5889400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b1af9ff610 0x55b1afa02fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:11.538 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.536+0000 7f5889e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b1af9fa140 0x55b1afa05c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:11.538 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.536+0000 7f5889400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b1af9ff610 0x55b1afa02fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:46870/0 (socket says 172.21.15.151:46870) 2024-05-21T22:05:11.538 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.536+0000 7f5889e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b1af9fa140 0x55b1afa05c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:53488/0 (socket says 172.21.15.151:53488) 2024-05-21T22:05:11.538 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.536+0000 7f5889400700 1 -- 172.21.15.151:0/3335249430 learned_addr learned my addr 172.21.15.151:0/3335249430 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:05:11.538 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.536+0000 7f588a800700 1 --2- 172.21.15.151:0/3335249430 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1af9fff30 0x55b1afa03500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:11.538 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.536+0000 7f5889e00700 1 -- 172.21.15.151:0/3335249430 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1af9fff30 msgr2=0x55b1afa03500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:11.538 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.536+0000 7f5889e00700 1 --2- 172.21.15.151:0/3335249430 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1af9fff30 0x55b1afa03500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:11.538 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.536+0000 7f5889e00700 1 -- 172.21.15.151:0/3335249430 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b1af9ff610 msgr2=0x55b1afa02fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:11.538 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.536+0000 7f5889e00700 1 --2- 172.21.15.151:0/3335249430 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b1af9ff610 0x55b1afa02fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:11.538 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.536+0000 7f5889e00700 1 -- 172.21.15.151:0/3335249430 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1af8978d0 con 0x55b1af9fa140 2024-05-21T22:05:11.539 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.536+0000 7f588a800700 1 --2- 172.21.15.151:0/3335249430 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1af9fff30 0x55b1afa03500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T22:05:11.540 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.536+0000 7f5889e00700 1 --2- 172.21.15.151:0/3335249430 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b1af9fa140 0x55b1afa05c20 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f587c0a6010 tx=0x7f587c0e2a80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85910a5d543cea6e server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:05:11.540 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.536+0000 7f586be00700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-21T22:05:11.540 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.537+0000 7f586be00700 1 -- 172.21.15.151:0/3335249430 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f587c0a93c0 con 0x55b1af9fa140 2024-05-21T22:05:11.540 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.537+0000 7f586be00700 1 -- 172.21.15.151:0/3335249430 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7f587c0a6060 con 0x55b1af9fa140 2024-05-21T22:05:11.540 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.537+0000 7f589b3c6b80 1 -- 172.21.15.151:0/3335249430 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f587c0a51f0 con 0x55b1af9fa140 2024-05-21T22:05:11.540 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.537+0000 7f586be00700 1 -- 172.21.15.151:0/3335249430 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f587c0a9840 con 0x55b1af9fa140 2024-05-21T22:05:11.540 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.537+0000 7f589b3c6b80 10 client.27433 fetch_fsmap learned FSMap version 29 2024-05-21T22:05:11.540 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.537+0000 7f589b3c6b80 1 -- 172.21.15.151:0/3335249430 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b1af918060 con 0x55b1af9fa140 2024-05-21T22:05:11.540 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.537+0000 7f586be00700 1 -- 172.21.15.151:0/3335249430 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f587c0a51f0 con 0x55b1af9fa140 2024-05-21T22:05:11.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.538+0000 7f586be00700 1 -- 172.21.15.151:0/3335249430 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f587c0b2440 con 0x55b1af9fa140 2024-05-21T22:05:11.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.538+0000 7f589b3c6b80 10 client.27433 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:05:11.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.538+0000 7f589b3c6b80 10 client.27433 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T22:05:11.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.538+0000 7f589b3c6b80 20 client.27433 populate_metadata read hostname 'smithi151' 2024-05-21T22:05:11.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.538+0000 7f589b3c6b80 1 --2- 172.21.15.151:0/3335249430 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55b1afa13390 0x55b1afa15780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:11.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.538+0000 7f589b3c6b80 4 client.27433 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T22:05:11.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.538+0000 7f589b3c6b80 1 -- 172.21.15.151:0/3335249430 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b1af959d90 con 0x55b1afa13390 2024-05-21T22:05:11.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.538+0000 7f5889400700 1 --2- 172.21.15.151:0/3335249430 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55b1afa13390 0x55b1afa15780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:11.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.539+0000 7f5889400700 1 --2- 172.21.15.151:0/3335249430 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55b1afa13390 0x55b1afa15780 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ade77602364de226 server_cookie=4a1ab1529475cbf5 in_seq=0 out_seq=0 2024-05-21T22:05:11.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.539+0000 7f586be00700 10 client.27433 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T22:05:11.542 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.540+0000 7f586be00700 1 -- 172.21.15.151:0/3335249430 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55b1af959d90 con 0x55b1afa13390 2024-05-21T22:05:11.542 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.540+0000 7f586be00700 10 client.27433 handle_command_reply: tid=0 2024-05-21T22:05:11.542 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.540+0000 7f589b3c6b80 1 client.27433 shutdown 2024-05-21T22:05:11.542 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.540+0000 7f589b3c6b80 1 -- 172.21.15.151:0/3335249430 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b1af9fa140 msgr2=0x55b1afa05c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:11.542 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.540+0000 7f589b3c6b80 1 --2- 172.21.15.151:0/3335249430 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b1af9fa140 0x55b1afa05c20 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f587c0a6010 tx=0x7f587c0e2a80 comp rx=0 tx=0).stop 2024-05-21T22:05:11.543 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.540+0000 7f589b3c6b80 1 -- 172.21.15.151:0/3335249430 shutdown_connections 2024-05-21T22:05:11.543 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.540+0000 7f589b3c6b80 1 --2- 172.21.15.151:0/3335249430 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55b1afa13390 0x55b1afa15780 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:11.543 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.540+0000 7f589b3c6b80 1 --2- 172.21.15.151:0/3335249430 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1af9fff30 0x55b1afa03500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:11.543 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.540+0000 7f589b3c6b80 1 --2- 172.21.15.151:0/3335249430 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b1af9fa140 0x55b1afa05c20 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:11.543 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.540+0000 7f589b3c6b80 1 --2- 172.21.15.151:0/3335249430 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b1af9ff610 0x55b1afa02fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:11.543 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.540+0000 7f589b3c6b80 1 -- 172.21.15.151:0/3335249430 >> 172.21.15.151:0/3335249430 conn(0x55b1af927be0 msgr2=0x55b1af9f3d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:05:11.543 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.540+0000 7f586be00700 0 client.27433 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T22:05:11.543 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.540+0000 7f589b3c6b80 1 -- 172.21.15.151:0/3335249430 shutdown_connections 2024-05-21T22:05:11.543 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.540+0000 7f589b3c6b80 1 -- 172.21.15.151:0/3335249430 wait complete. 2024-05-21T22:05:11.543 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.541+0000 7f589b3c6b80 20 client.27433 trim_cache size 0 max 16384 2024-05-21T22:05:11.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.578+0000 7f589b3c6b80 1 Processor -- start 2024-05-21T22:05:11.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.578+0000 7f589b3c6b80 1 -- start start 2024-05-21T22:05:11.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.578+0000 7f589b3c6b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b1af9fa140 0x55b1afa00c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:11.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.578+0000 7f589b3c6b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1af9ff610 0x55b1afa02000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:11.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.578+0000 7f589b3c6b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b1af9fff30 0x55b1af9f44f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:11.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.578+0000 7f589b3c6b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f588c1b0140 con 0x55b1af9fa140 2024-05-21T22:05:11.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.578+0000 7f589b3c6b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f588c1b1c80 con 0x55b1af9ff610 2024-05-21T22:05:11.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.578+0000 7f589b3c6b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f588c1b0810 con 0x55b1af9fff30 2024-05-21T22:05:11.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.579+0000 7f5889400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1af9ff610 0x55b1afa02000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:11.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.579+0000 7f5889e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b1af9fa140 0x55b1afa00c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:11.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.579+0000 7f5889400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1af9ff610 0x55b1afa02000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:51176/0 (socket says 172.21.15.151:51176) 2024-05-21T22:05:11.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.579+0000 7f5889400700 1 -- 172.21.15.151:0/2386086997 learned_addr learned my addr 172.21.15.151:0/2386086997 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:05:11.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.579+0000 7f588a800700 1 --2- 172.21.15.151:0/2386086997 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b1af9fff30 0x55b1af9f44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:11.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.579+0000 7f5889400700 1 -- 172.21.15.151:0/2386086997 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b1af9fff30 msgr2=0x55b1af9f44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:11.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.579+0000 7f5889400700 1 --2- 172.21.15.151:0/2386086997 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b1af9fff30 0x55b1af9f44f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:11.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.579+0000 7f5889400700 1 -- 172.21.15.151:0/2386086997 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b1af9fa140 msgr2=0x55b1afa00c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:11.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.579+0000 7f5889400700 1 --2- 172.21.15.151:0/2386086997 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b1af9fa140 0x55b1afa00c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:11.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.579+0000 7f5889400700 1 -- 172.21.15.151:0/2386086997 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1af9f53c0 con 0x55b1af9ff610 2024-05-21T22:05:11.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.580+0000 7f5889400700 1 --2- 172.21.15.151:0/2386086997 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1af9ff610 0x55b1afa02000 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x55b1afa04590 tx=0x7f5878009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=498d42968be9950c server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:05:11.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.580+0000 7f5876a00700 1 -- 172.21.15.151:0/2386086997 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5878004830 con 0x55b1af9ff610 2024-05-21T22:05:11.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.580+0000 7f5876a00700 1 -- 172.21.15.151:0/2386086997 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5878005020 con 0x55b1af9ff610 2024-05-21T22:05:11.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.580+0000 7f5876a00700 1 -- 172.21.15.151:0/2386086997 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5878010690 con 0x55b1af9ff610 2024-05-21T22:05:11.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.581+0000 7f589b3c6b80 1 -- 172.21.15.151:0/2386086997 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1af9ff610 msgr2=0x55b1afa02000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:11.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.581+0000 7f589b3c6b80 1 --2- 172.21.15.151:0/2386086997 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1af9ff610 0x55b1afa02000 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x55b1afa04590 tx=0x7f5878009260 comp rx=0 tx=0).stop 2024-05-21T22:05:11.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.581+0000 7f589b3c6b80 1 -- 172.21.15.151:0/2386086997 shutdown_connections 2024-05-21T22:05:11.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.581+0000 7f589b3c6b80 1 --2- 172.21.15.151:0/2386086997 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b1af9fff30 0x55b1af9f44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:11.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.581+0000 7f589b3c6b80 1 --2- 172.21.15.151:0/2386086997 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b1af9fa140 0x55b1afa00c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:11.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.581+0000 7f589b3c6b80 1 --2- 172.21.15.151:0/2386086997 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1af9ff610 0x55b1afa02000 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:11.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.581+0000 7f589b3c6b80 1 -- 172.21.15.151:0/2386086997 >> 172.21.15.151:0/2386086997 conn(0x55b1af927be0 msgr2=0x55b1afa018f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:05:11.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.581+0000 7f589b3c6b80 1 -- 172.21.15.151:0/2386086997 shutdown_connections 2024-05-21T22:05:11.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.581+0000 7f589b3c6b80 1 -- 172.21.15.151:0/2386086997 wait complete. 2024-05-21T22:05:11.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.581+0000 7f589b3c6b80 1 -- start start 2024-05-21T22:05:11.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.582+0000 7f589b3c6b80 1 Processor -- start 2024-05-21T22:05:11.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.582+0000 7f589b3c6b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1af9fa140 0x55b1afa16170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:11.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.582+0000 7f589b3c6b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b1af9ff610 0x55b1afa166b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:11.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.582+0000 7f589b3c6b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b1af9fff30 0x55b1afa16bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:11.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.582+0000 7f589b3c6b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f588c1b15b0 con 0x55b1af9ff610 2024-05-21T22:05:11.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.582+0000 7f589b3c6b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f588c1b0ee0 con 0x55b1af9fa140 2024-05-21T22:05:11.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.582+0000 7f589b3c6b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f588c1ae100 con 0x55b1af9fff30 2024-05-21T22:05:11.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.582+0000 7f589b3c6b80 10 client.0 _ll_register_callbacks cb 0x55b1af8f1c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:05:11.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.582+0000 7f5889400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b1af9ff610 0x55b1afa166b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:11.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.583+0000 7f5889400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b1af9ff610 0x55b1afa166b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:46884/0 (socket says 172.21.15.151:46884) 2024-05-21T22:05:11.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.583+0000 7f5889400700 1 -- 172.21.15.151:0/3526927589 learned_addr learned my addr 172.21.15.151:0/3526927589 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:05:11.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.583+0000 7f5889e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1af9fa140 0x55b1afa16170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:11.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.583+0000 7f588a800700 1 --2- 172.21.15.151:0/3526927589 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b1af9fff30 0x55b1afa16bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:11.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.583+0000 7f5889e00700 1 -- 172.21.15.151:0/3526927589 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b1af9fff30 msgr2=0x55b1afa16bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:11.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.583+0000 7f5889e00700 1 --2- 172.21.15.151:0/3526927589 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b1af9fff30 0x55b1afa16bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:11.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.583+0000 7f5889e00700 1 -- 172.21.15.151:0/3526927589 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b1af9ff610 msgr2=0x55b1afa166b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:11.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.583+0000 7f5889e00700 1 --2- 172.21.15.151:0/3526927589 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b1af9ff610 0x55b1afa166b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:11.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.583+0000 7f5889e00700 1 -- 172.21.15.151:0/3526927589 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1afa7cc50 con 0x55b1af9fa140 2024-05-21T22:05:11.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.583+0000 7f5889e00700 1 --2- 172.21.15.151:0/3526927589 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1af9fa140 0x55b1afa16170 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f587c005fd0 tx=0x7f587c01f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f532245e891eac76 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:05:11.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.583+0000 7f586be00700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-21T22:05:11.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.584+0000 7f586be00700 1 -- 172.21.15.151:0/3526927589 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f587c0a93c0 con 0x55b1af9fa140 2024-05-21T22:05:11.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.584+0000 7f586be00700 1 -- 172.21.15.151:0/3526927589 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7f587c0e8300 con 0x55b1af9fa140 2024-05-21T22:05:11.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.584+0000 7f589b3c6b80 1 -- 172.21.15.151:0/3526927589 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5878005020 con 0x55b1af9fa140 2024-05-21T22:05:11.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.584+0000 7f586be00700 1 -- 172.21.15.151:0/3526927589 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f587c0a91e0 con 0x55b1af9fa140 2024-05-21T22:05:11.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.584+0000 7f589b3c6b80 10 client.27437 fetch_fsmap learned FSMap version 29 2024-05-21T22:05:11.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.584+0000 7f589b3c6b80 1 -- 172.21.15.151:0/3526927589 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b1afa7cc50 con 0x55b1af9fa140 2024-05-21T22:05:11.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.584+0000 7f586be00700 1 -- 172.21.15.151:0/3526927589 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5878005020 con 0x55b1af9fa140 2024-05-21T22:05:11.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.584+0000 7f586be00700 1 -- 172.21.15.151:0/3526927589 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f587c0cc110 con 0x55b1af9fa140 2024-05-21T22:05:11.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.584+0000 7f589b3c6b80 10 client.27437 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:05:11.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.585+0000 7f589b3c6b80 10 client.27437 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T22:05:11.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.585+0000 7f589b3c6b80 20 client.27437 populate_metadata read hostname 'smithi151' 2024-05-21T22:05:11.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.585+0000 7f589b3c6b80 1 --2- 172.21.15.151:0/3526927589 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55b1afa7e220 0x55b1afa80640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:11.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.585+0000 7f589b3c6b80 4 client.27437 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-21T22:05:11.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.585+0000 7f589b3c6b80 1 -- 172.21.15.151:0/3526927589 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b1af953570 con 0x55b1afa7e220 2024-05-21T22:05:11.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.585+0000 7f5889400700 1 --2- 172.21.15.151:0/3526927589 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55b1afa7e220 0x55b1afa80640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:11.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.586+0000 7f586be00700 10 client.27437 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T22:05:11.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.586+0000 7f5889400700 1 --2- 172.21.15.151:0/3526927589 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55b1afa7e220 0x55b1afa80640 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6678fd3b8890570e server_cookie=ab2af1d5f42d26a0 in_seq=0 out_seq=0 2024-05-21T22:05:11.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.586+0000 7f586be00700 1 -- 172.21.15.151:0/3526927589 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b1af953570 con 0x55b1afa7e220 2024-05-21T22:05:11.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.586+0000 7f586be00700 10 client.27437 handle_command_reply: tid=0 2024-05-21T22:05:11.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.586+0000 7f589b3c6b80 1 client.27437 shutdown 2024-05-21T22:05:11.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.586+0000 7f589b3c6b80 1 -- 172.21.15.151:0/3526927589 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1af9fa140 msgr2=0x55b1afa16170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:11.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.586+0000 7f589b3c6b80 1 --2- 172.21.15.151:0/3526927589 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1af9fa140 0x55b1afa16170 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f587c005fd0 tx=0x7f587c01f000 comp rx=0 tx=0).stop 2024-05-21T22:05:11.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.586+0000 7f589b3c6b80 1 -- 172.21.15.151:0/3526927589 shutdown_connections 2024-05-21T22:05:11.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.586+0000 7f589b3c6b80 1 --2- 172.21.15.151:0/3526927589 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55b1afa7e220 0x55b1afa80640 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:11.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.586+0000 7f589b3c6b80 1 --2- 172.21.15.151:0/3526927589 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55b1af9fa140 0x55b1afa16170 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:11.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.586+0000 7f589b3c6b80 1 --2- 172.21.15.151:0/3526927589 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b1af9fff30 0x55b1afa16bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:11.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.586+0000 7f586be00700 0 client.27437 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T22:05:11.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.586+0000 7f589b3c6b80 1 --2- 172.21.15.151:0/3526927589 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b1af9ff610 0x55b1afa166b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:11.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.586+0000 7f589b3c6b80 1 -- 172.21.15.151:0/3526927589 >> 172.21.15.151:0/3526927589 conn(0x55b1af927be0 msgr2=0x55b1afa00a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:05:11.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.586+0000 7f589b3c6b80 1 -- 172.21.15.151:0/3526927589 shutdown_connections 2024-05-21T22:05:11.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.586+0000 7f589b3c6b80 1 -- 172.21.15.151:0/3526927589 wait complete. 2024-05-21T22:05:11.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.586+0000 7f589b3c6b80 20 client.27437 trim_cache size 0 max 16384 2024-05-21T22:05:11.590 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-05-21T22:05:11.590 INFO:teuthology.orchestra.run.smithi151.stdout: "status": "no active scrubs running", 2024-05-21T22:05:11.591 INFO:teuthology.orchestra.run.smithi151.stdout: "scrubs": {} 2024-05-21T22:05:11.591 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-05-21T22:05:11.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.587+0000 7f5890a00700 1 -- 172.21.15.151:0/2748545783 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f58700b29f0 msgr2=0x7f58700b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:11.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.587+0000 7f5890a00700 1 --2- 172.21.15.151:0/2748545783 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f58700b29f0 0x7f58700b4eb0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f588400b6c0 tx=0x7f5884009f30 comp rx=0 tx=0).stop 2024-05-21T22:05:11.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.587+0000 7f5890a00700 1 -- 172.21.15.151:0/2748545783 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f588c1037f0 msgr2=0x7f588c19bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:11.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.587+0000 7f5890a00700 1 --2- 172.21.15.151:0/2748545783 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f588c1037f0 0x7f588c19bd40 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f587c00e9c0 tx=0x7f587c00ecd0 comp rx=0 tx=0).stop 2024-05-21T22:05:11.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.587+0000 7f5890a00700 1 -- 172.21.15.151:0/2748545783 shutdown_connections 2024-05-21T22:05:11.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.587+0000 7f5890a00700 1 --2- 172.21.15.151:0/2748545783 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f58700b29f0 0x7f58700b4eb0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:11.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.587+0000 7f5890a00700 1 --2- 172.21.15.151:0/2748545783 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f588c1088f0 0x7f588c1a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:11.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.587+0000 7f5890a00700 1 --2- 172.21.15.151:0/2748545783 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f588c1037f0 0x7f588c19bd40 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:11.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.587+0000 7f5890a00700 1 --2- 172.21.15.151:0/2748545783 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f588c1041a0 0x7f588c19c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:11.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.587+0000 7f5890a00700 1 -- 172.21.15.151:0/2748545783 >> 172.21.15.151:0/2748545783 conn(0x7f588c07a6a0 msgr2=0x7f588c1017a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:05:11.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.587+0000 7f5890a00700 1 -- 172.21.15.151:0/2748545783 shutdown_connections 2024-05-21T22:05:11.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:11.588+0000 7f5890a00700 1 -- 172.21.15.151:0/2748545783 wait complete. 2024-05-21T22:05:11.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:05:11 smithi151 ceph-mon[31623]: pgmap v296: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-05-21T22:05:11.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:05:11 smithi184 ceph-mon[36942]: pgmap v296: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-05-21T22:05:12.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:05:11 smithi164 ceph-mon[36269]: pgmap v296: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-05-21T22:05:12.539 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-21T22:05:12.539 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-21T22:05:12.977 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:05:13.910 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:05:13 smithi151 ceph-mon[31623]: pgmap v297: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-05-21T22:05:13.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:05:13 smithi184 ceph-mon[36942]: pgmap v297: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-05-21T22:05:14.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:05:13 smithi164 ceph-mon[36269]: pgmap v297: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-05-21T22:05:15.161 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:15.159+0000 7ff62fe00700 1 -- 172.21.15.151:0/3228266493 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6281095b0 msgr2=0x7ff628109990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:15.161 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:15.159+0000 7ff62fe00700 1 --2- 172.21.15.151:0/3228266493 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6281095b0 0x7ff628109990 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7ff61c00b0a0 tx=0x7ff61c00b3b0 comp rx=0 tx=0).stop 2024-05-21T22:05:15.161 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:15.159+0000 7ff62fe00700 1 -- 172.21.15.151:0/3228266493 shutdown_connections 2024-05-21T22:05:15.162 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:15.159+0000 7ff62fe00700 1 --2- 172.21.15.151:0/3228266493 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff62806bbe0 0x7ff6281103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:15.162 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:15.159+0000 7ff62fe00700 1 --2- 172.21.15.151:0/3228266493 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6281095b0 0x7ff628109990 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:15.162 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:15.159+0000 7ff62fe00700 1 --2- 172.21.15.151:0/3228266493 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff62806b220 0x7ff62806b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:15.162 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:15.159+0000 7ff62fe00700 1 -- 172.21.15.151:0/3228266493 >> 172.21.15.151:0/3228266493 conn(0x7ff6280793f0 msgr2=0x7ff628079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:05:15.162 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:15.160+0000 7ff62fe00700 1 -- 172.21.15.151:0/3228266493 shutdown_connections 2024-05-21T22:05:15.162 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:15.160+0000 7ff62fe00700 1 -- 172.21.15.151:0/3228266493 wait complete. 2024-05-21T22:05:15.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:15.161+0000 7ff62fe00700 1 Processor -- start 2024-05-21T22:05:15.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:15.161+0000 7ff62fe00700 1 -- start start 2024-05-21T22:05:15.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:15.162+0000 7ff62fe00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff62806b220 0x7ff628102510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:15.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:15.162+0000 7ff62fe00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff62806bbe0 0x7ff628102a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:15.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:15.162+0000 7ff62fe00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6281095b0 0x7ff628102f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:15.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:15.162+0000 7ff62fe00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff628113710 con 0x7ff62806bbe0 2024-05-21T22:05:15.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:15.162+0000 7ff62fe00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff628113590 con 0x7ff62806b220 2024-05-21T22:05:15.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:15.162+0000 7ff62fe00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff628113890 con 0x7ff6281095b0 2024-05-21T22:05:15.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:15.162+0000 7ff62d400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff62806b220 0x7ff628102510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:15.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:15.162+0000 7ff62ca00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff62806bbe0 0x7ff628102a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:15.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:15.162+0000 7ff62d400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff62806b220 0x7ff628102510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:51208/0 (socket says 172.21.15.151:51208) 2024-05-21T22:05:15.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:15.162+0000 7ff62d400700 1 -- 172.21.15.151:0/661165153 learned_addr learned my addr 172.21.15.151:0/661165153 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:05:15.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:15.162+0000 7ff62de00700 1 --2- 172.21.15.151:0/661165153 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6281095b0 0x7ff628102f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:15.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:15.162+0000 7ff62d400700 1 -- 172.21.15.151:0/661165153 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6281095b0 msgr2=0x7ff628102f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:15.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:15.162+0000 7ff62d400700 1 --2- 172.21.15.151:0/661165153 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6281095b0 0x7ff628102f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:15.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:15.162+0000 7ff62d400700 1 -- 172.21.15.151:0/661165153 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff62806bbe0 msgr2=0x7ff628102a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:15.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:15.162+0000 7ff62d400700 1 --2- 172.21.15.151:0/661165153 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff62806bbe0 0x7ff628102a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:15.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:15.162+0000 7ff62d400700 1 -- 172.21.15.151:0/661165153 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6281a8170 con 0x7ff62806b220 2024-05-21T22:05:15.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:15.163+0000 7ff62d400700 1 --2- 172.21.15.151:0/661165153 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff62806b220 0x7ff628102510 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ff61c00d010 tx=0x7ff61c004d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:05:15.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:15.163+0000 7ff626000700 1 -- 172.21.15.151:0/661165153 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff61c021070 con 0x7ff62806b220 2024-05-21T22:05:15.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:15.163+0000 7ff62fe00700 1 -- 172.21.15.151:0/661165153 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6281a8400 con 0x7ff62806b220 2024-05-21T22:05:15.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:15.163+0000 7ff626000700 1 -- 172.21.15.151:0/661165153 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff61c007e30 con 0x7ff62806b220 2024-05-21T22:05:15.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:15.163+0000 7ff626000700 1 -- 172.21.15.151:0/661165153 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff61c004640 con 0x7ff62806b220 2024-05-21T22:05:15.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:15.163+0000 7ff62fe00700 1 -- 172.21.15.151:0/661165153 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6281a8900 con 0x7ff62806b220 2024-05-21T22:05:15.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:15.164+0000 7ff62fe00700 1 -- 172.21.15.151:0/661165153 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff62810ba90 con 0x7ff62806b220 2024-05-21T22:05:15.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:15.167+0000 7ff626000700 1 -- 172.21.15.151:0/661165153 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7ff61c007980 con 0x7ff62806b220 2024-05-21T22:05:15.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:15.168+0000 7ff626000700 1 --2- 172.21.15.151:0/661165153 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7ff6100b29f0 0x7ff6100b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:15.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:15.168+0000 7ff626000700 1 -- 172.21.15.151:0/661165153 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7ff61c00f020 con 0x7ff62806b220 2024-05-21T22:05:15.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:15.169+0000 7ff62ca00700 1 --2- 172.21.15.151:0/661165153 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7ff6100b29f0 0x7ff6100b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:15.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:15.169+0000 7ff62ca00700 1 --2- 172.21.15.151:0/661165153 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7ff6100b29f0 0x7ff6100b4eb0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff614009680 tx=0x7ff614009260 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:05:15.172 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:15.171+0000 7ff626000700 1 -- 172.21.15.151:0/661165153 <== mon.2 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff61c02a020 con 0x7ff62806b220 2024-05-21T22:05:15.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:15.453+0000 7ff62fe00700 1 -- 172.21.15.151:0/661165153 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff628103a40 con 0x7ff62806b220 2024-05-21T22:05:15.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:15.454+0000 7ff626000700 1 -- 172.21.15.151:0/661165153 <== mon.2 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11958 (secure 0 0 0) 0x7ff61c02e3e0 con 0x7ff62806b220 2024-05-21T22:05:15.456 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T22:05:15.456 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14358,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/158010529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":158010529},{"type":"v1","addr":"172.21.15.151:6835","nonce":158010529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6839/2069058115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":2069058115},{"type":"v1","addr":"172.21.15.151:6839","nonce":2069058115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T21:51:19.552069+0000","modified":"2024-05-21T21:55:34.643755+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24530,"mds_1":24493,"mds_2":24491,"mds_3":24478,"mds_4":24473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14403":{"gid":14403,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6841/1749723794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6840","nonce":1749723794},{"type":"v1","addr":"172.21.15.151:6841","nonce":1749723794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/2026266114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":2026266114},{"type":"v1","addr":"172.21.15.184:6833","nonce":2026266114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/2467900844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":2467900844},{"type":"v1","addr":"172.21.15.164:6833","nonce":2467900844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3367640960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3367640960},{"type":"v1","addr":"172.21.15.151:6837","nonce":3367640960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/3264142621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3264142621},{"type":"v1","addr":"172.21.15.184:6835","nonce":3264142621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":17,"addr":"172.21.15.164:6835/2340428470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2340428470},{"type":"v1","addr":"172.21.15.164:6835","nonce":2340428470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":16,"addr":"172.21.15.184:6837/2988194888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":2988194888},{"type":"v1","addr":"172.21.15.184:6837","nonce":2988194888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.164:6837/1554609572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1554609572},{"type":"v1","addr":"172.21.15.164:6837","nonce":1554609572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.184:6839/3059935143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":3059935143},{"type":"v1","addr":"172.21.15.184:6839","nonce":3059935143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.164:6839/2646401247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":2646401247},{"type":"v1","addr":"172.21.15.164:6839","nonce":2646401247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24530,"qdb_cluster":[24530,24493,24491,24473,24478]},"id":1}]} 2024-05-21T22:05:15.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:15.458+0000 7ff62fe00700 1 -- 172.21.15.151:0/661165153 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7ff6100b29f0 msgr2=0x7ff6100b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:15.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:15.458+0000 7ff62fe00700 1 --2- 172.21.15.151:0/661165153 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7ff6100b29f0 0x7ff6100b4eb0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff614009680 tx=0x7ff614009260 comp rx=0 tx=0).stop 2024-05-21T22:05:15.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:15.458+0000 7ff62fe00700 1 -- 172.21.15.151:0/661165153 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff62806b220 msgr2=0x7ff628102510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:15.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:15.458+0000 7ff62fe00700 1 --2- 172.21.15.151:0/661165153 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff62806b220 0x7ff628102510 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ff61c00d010 tx=0x7ff61c004d00 comp rx=0 tx=0).stop 2024-05-21T22:05:15.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:15.458+0000 7ff62fe00700 1 -- 172.21.15.151:0/661165153 shutdown_connections 2024-05-21T22:05:15.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:15.458+0000 7ff62fe00700 1 --2- 172.21.15.151:0/661165153 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7ff6100b29f0 0x7ff6100b4eb0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:15.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:15.458+0000 7ff62fe00700 1 --2- 172.21.15.151:0/661165153 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff62806b220 0x7ff628102510 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:15.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:15.458+0000 7ff62fe00700 1 --2- 172.21.15.151:0/661165153 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6281095b0 0x7ff628102f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:15.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:15.458+0000 7ff62fe00700 1 --2- 172.21.15.151:0/661165153 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff62806bbe0 0x7ff628102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:15.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:15.458+0000 7ff62fe00700 1 -- 172.21.15.151:0/661165153 >> 172.21.15.151:0/661165153 conn(0x7ff6280793f0 msgr2=0x7ff6281010e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:05:15.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:15.458+0000 7ff62fe00700 1 -- 172.21.15.151:0/661165153 shutdown_connections 2024-05-21T22:05:15.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:15.458+0000 7ff62fe00700 1 -- 172.21.15.151:0/661165153 wait complete. 2024-05-21T22:05:15.461 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 29 2024-05-21T22:05:15.777 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:05:15 smithi151 ceph-mon[31623]: pgmap v298: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 170 B/s wr, 8 op/s 2024-05-21T22:05:15.777 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:05:15 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/661165153' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T22:05:15.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:05:15 smithi184 ceph-mon[36942]: pgmap v298: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 170 B/s wr, 8 op/s 2024-05-21T22:05:15.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:05:15 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/661165153' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T22:05:16.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:05:15 smithi164 ceph-mon[36269]: pgmap v298: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 170 B/s wr, 8 op/s 2024-05-21T22:05:16.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:05:15 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/661165153' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T22:05:16.256 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-21T22:05:16.704 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:05:17.925 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:05:17 smithi151 ceph-mon[31623]: pgmap v299: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 170 B/s wr, 8 op/s 2024-05-21T22:05:17.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:05:17 smithi184 ceph-mon[36942]: pgmap v299: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 170 B/s wr, 8 op/s 2024-05-21T22:05:18.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:05:17 smithi164 ceph-mon[36269]: pgmap v299: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 170 B/s wr, 8 op/s 2024-05-21T22:05:18.839 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.837+0000 7fc2e6a00700 1 -- 172.21.15.151:0/3997352034 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc2e0103f80 msgr2=0x7fc2e0110410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:18.839 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.837+0000 7fc2e6a00700 1 --2- 172.21.15.151:0/3997352034 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc2e0103f80 0x7fc2e0110410 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fc2d800b0a0 tx=0x7fc2d800b3b0 comp rx=0 tx=0).stop 2024-05-21T22:05:18.839 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.837+0000 7fc2e6a00700 1 -- 172.21.15.151:0/3997352034 shutdown_connections 2024-05-21T22:05:18.839 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.837+0000 7fc2e6a00700 1 --2- 172.21.15.151:0/3997352034 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc2e01095d0 0x7fc2e01099b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:18.839 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.837+0000 7fc2e6a00700 1 --2- 172.21.15.151:0/3997352034 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc2e0103f80 0x7fc2e0110410 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:18.839 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.837+0000 7fc2e6a00700 1 --2- 172.21.15.151:0/3997352034 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2e01035c0 0x7fc2e0103a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:18.840 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.837+0000 7fc2e6a00700 1 -- 172.21.15.151:0/3997352034 >> 172.21.15.151:0/3997352034 conn(0x7fc2e007a610 msgr2=0x7fc2e007aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:05:18.840 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.837+0000 7fc2e6a00700 1 -- 172.21.15.151:0/3997352034 shutdown_connections 2024-05-21T22:05:18.840 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.837+0000 7fc2e6a00700 1 -- 172.21.15.151:0/3997352034 wait complete. 2024-05-21T22:05:18.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.838+0000 7fc2e6a00700 1 Processor -- start 2024-05-21T22:05:18.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.839+0000 7fc2e6a00700 1 -- start start 2024-05-21T22:05:18.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.839+0000 7fc2e6a00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2e01035c0 0x7fc2e019bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:18.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.839+0000 7fc2e6a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc2e0103f80 0x7fc2e019c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:18.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.839+0000 7fc2e6a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc2e01095d0 0x7fc2e01a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:18.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.839+0000 7fc2e6a00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc2e01137d0 con 0x7fc2e0103f80 2024-05-21T22:05:18.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.839+0000 7fc2e6a00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc2e0113650 con 0x7fc2e01095d0 2024-05-21T22:05:18.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.839+0000 7fc2e6a00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc2e0113950 con 0x7fc2e01035c0 2024-05-21T22:05:18.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.839+0000 7fc2d7400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc2e0103f80 0x7fc2e019c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:18.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.839+0000 7fc2d7400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc2e0103f80 0x7fc2e019c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:36096/0 (socket says 172.21.15.151:36096) 2024-05-21T22:05:18.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.839+0000 7fc2d7400700 1 -- 172.21.15.151:0/46145558 learned_addr learned my addr 172.21.15.151:0/46145558 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:05:18.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.840+0000 7fc2dfe00700 1 --2- 172.21.15.151:0/46145558 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2e01035c0 0x7fc2e019bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:18.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.839+0000 7fc2e4a00700 1 --2- 172.21.15.151:0/46145558 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc2e01095d0 0x7fc2e01a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:18.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.840+0000 7fc2e4a00700 1 -- 172.21.15.151:0/46145558 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2e01035c0 msgr2=0x7fc2e019bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:18.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.840+0000 7fc2e4a00700 1 --2- 172.21.15.151:0/46145558 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2e01035c0 0x7fc2e019bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:18.843 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.840+0000 7fc2e4a00700 1 -- 172.21.15.151:0/46145558 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc2e0103f80 msgr2=0x7fc2e019c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:18.843 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.840+0000 7fc2e4a00700 1 --2- 172.21.15.151:0/46145558 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc2e0103f80 0x7fc2e019c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:18.843 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.840+0000 7fc2e4a00700 1 -- 172.21.15.151:0/46145558 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2e01a0d90 con 0x7fc2e01095d0 2024-05-21T22:05:18.843 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.840+0000 7fc2d7400700 1 --2- 172.21.15.151:0/46145558 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc2e0103f80 0x7fc2e019c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:05:18.843 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.840+0000 7fc2e4a00700 1 --2- 172.21.15.151:0/46145558 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc2e01095d0 0x7fc2e01a0650 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fc2d800b070 tx=0x7fc2d8004c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:05:18.843 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.841+0000 7fc2dd600700 1 -- 172.21.15.151:0/46145558 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2d8021070 con 0x7fc2e01095d0 2024-05-21T22:05:18.843 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.841+0000 7fc2e6a00700 1 -- 172.21.15.151:0/46145558 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2e01a1020 con 0x7fc2e01095d0 2024-05-21T22:05:18.844 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.841+0000 7fc2e6a00700 1 -- 172.21.15.151:0/46145558 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2e01a8900 con 0x7fc2e01095d0 2024-05-21T22:05:18.844 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.841+0000 7fc2dd600700 1 -- 172.21.15.151:0/46145558 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2d8007e30 con 0x7fc2e01095d0 2024-05-21T22:05:18.844 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.841+0000 7fc2dd600700 1 -- 172.21.15.151:0/46145558 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2d8004430 con 0x7fc2e01095d0 2024-05-21T22:05:18.845 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.844+0000 7fc2dd600700 1 -- 172.21.15.151:0/46145558 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7fc2d8003c00 con 0x7fc2e01095d0 2024-05-21T22:05:18.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.846+0000 7fc2f1369b80 1 Processor -- start 2024-05-21T22:05:18.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.846+0000 7fc2f1369b80 1 -- start start 2024-05-21T22:05:18.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.847+0000 7fc2f1369b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556826dbd610 0x556826dbd9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:18.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.847+0000 7fc2f1369b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556826dbdf30 0x556826db7b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:18.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.847+0000 7fc2f1369b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556826db8140 0x556826db85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:18.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.847+0000 7fc2f1369b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x556826d15e40 con 0x556826db8140 2024-05-21T22:05:18.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.847+0000 7fc2f1369b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x556826d059e0 con 0x556826dbd610 2024-05-21T22:05:18.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.847+0000 7fc2f1369b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x556826d09a20 con 0x556826dbdf30 2024-05-21T22:05:18.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.847+0000 7fc2dd600700 1 --2- 172.21.15.151:0/46145558 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fc2c40b2a70 0x7fc2c40b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:18.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.847+0000 7fc2d7400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556826dbd610 0x556826dbd9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:18.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.847+0000 7fc2d7400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556826dbd610 0x556826dbd9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:48574/0 (socket says 172.21.15.151:48574) 2024-05-21T22:05:18.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.847+0000 7fc2d7400700 1 -- 172.21.15.151:0/2796961740 learned_addr learned my addr 172.21.15.151:0/2796961740 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:05:18.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.847+0000 7fc2dfe00700 1 --2- 172.21.15.151:0/2796961740 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556826dbdf30 0x556826db7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:18.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.847+0000 7fc2d7400700 1 --2- 172.21.15.151:0/2796961740 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556826db8140 0x556826db85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:18.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.847+0000 7fc2dd600700 1 -- 172.21.15.151:0/46145558 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7fc2d80e2ff0 con 0x7fc2e01095d0 2024-05-21T22:05:18.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.847+0000 7fc2e4a00700 1 --2- 172.21.15.151:0/46145558 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fc2c40b2a70 0x7fc2c40b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:18.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.848+0000 7fc2d7400700 1 -- 172.21.15.151:0/2796961740 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556826dbdf30 msgr2=0x556826db7b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:18.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.848+0000 7fc2d7400700 1 --2- 172.21.15.151:0/2796961740 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556826dbdf30 0x556826db7b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:18.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.848+0000 7fc2d7400700 1 -- 172.21.15.151:0/2796961740 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556826db8140 msgr2=0x556826db85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:18.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.848+0000 7fc2d7400700 1 --2- 172.21.15.151:0/2796961740 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556826db8140 0x556826db85c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:18.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.848+0000 7fc2d7400700 1 -- 172.21.15.151:0/2796961740 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556826cce420 con 0x556826dbd610 2024-05-21T22:05:18.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.848+0000 7fc2d7400700 1 --2- 172.21.15.151:0/2796961740 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556826dbd610 0x556826dbd9f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fc2c80098a0 tx=0x7fc2c8009d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=154ccc6f74d07e3b server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:05:18.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.848+0000 7fc2e4a00700 1 --2- 172.21.15.151:0/46145558 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fc2c40b2a70 0x7fc2c40b4f30 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc2d80e0750 tx=0x7fc2d8009040 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:05:18.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.848+0000 7fc2d5600700 1 -- 172.21.15.151:0/2796961740 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c8011030 con 0x556826dbd610 2024-05-21T22:05:18.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.848+0000 7fc2d5600700 1 -- 172.21.15.151:0/2796961740 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2c800fe60 con 0x556826dbd610 2024-05-21T22:05:18.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.848+0000 7fc2d5600700 1 -- 172.21.15.151:0/2796961740 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c8010ad0 con 0x556826dbd610 2024-05-21T22:05:18.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.849+0000 7fc2f1369b80 1 -- 172.21.15.151:0/2796961740 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556826dbd610 msgr2=0x556826dbd9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:18.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.849+0000 7fc2f1369b80 1 --2- 172.21.15.151:0/2796961740 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556826dbd610 0x556826dbd9f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fc2c80098a0 tx=0x7fc2c8009d10 comp rx=0 tx=0).stop 2024-05-21T22:05:18.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.849+0000 7fc2f1369b80 1 -- 172.21.15.151:0/2796961740 shutdown_connections 2024-05-21T22:05:18.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.849+0000 7fc2f1369b80 1 --2- 172.21.15.151:0/2796961740 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556826dbd610 0x556826dbd9f0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:18.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.849+0000 7fc2f1369b80 1 --2- 172.21.15.151:0/2796961740 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556826db8140 0x556826db85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:18.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.849+0000 7fc2f1369b80 1 --2- 172.21.15.151:0/2796961740 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556826dbdf30 0x556826db7b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:18.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.849+0000 7fc2f1369b80 1 -- 172.21.15.151:0/2796961740 >> 172.21.15.151:0/2796961740 conn(0x556826ce5be0 msgr2=0x556826db2690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:05:18.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.850+0000 7fc2f1369b80 1 -- 172.21.15.151:0/2796961740 shutdown_connections 2024-05-21T22:05:18.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.850+0000 7fc2f1369b80 1 -- 172.21.15.151:0/2796961740 wait complete. 2024-05-21T22:05:18.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.850+0000 7fc2f1369b80 1 -- start start 2024-05-21T22:05:18.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.850+0000 7fc2f1369b80 1 Processor -- start 2024-05-21T22:05:18.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.851+0000 7fc2f1369b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556826db8140 0x556826dc3c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:18.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.851+0000 7fc2f1369b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556826dbd610 0x556826dc0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:18.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.851+0000 7fc2f1369b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556826dbdf30 0x556826dc1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:18.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.851+0000 7fc2f1369b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x556826c0bfa0 con 0x556826db8140 2024-05-21T22:05:18.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.851+0000 7fc2f1369b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x556826ca6760 con 0x556826dbdf30 2024-05-21T22:05:18.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.851+0000 7fc2f1369b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x556826dc49e0 con 0x556826dbd610 2024-05-21T22:05:18.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.851+0000 7fc2f1369b80 10 client.0 _ll_register_callbacks cb 0x556826cafc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:05:18.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.851+0000 7fc2d7400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556826db8140 0x556826dc3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:18.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.851+0000 7fc2dfe00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556826dbd610 0x556826dc0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:18.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.851+0000 7fc2d7400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556826db8140 0x556826dc3c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:36122/0 (socket says 172.21.15.151:36122) 2024-05-21T22:05:18.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.851+0000 7fc2d7400700 1 -- 172.21.15.151:0/2975984832 learned_addr learned my addr 172.21.15.151:0/2975984832 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:05:18.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.852+0000 7fc2d7400700 1 --2- 172.21.15.151:0/2975984832 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556826dbdf30 0x556826dc1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:18.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.852+0000 7fc2dfe00700 1 -- 172.21.15.151:0/2975984832 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556826dbdf30 msgr2=0x556826dc1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:18.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.852+0000 7fc2dfe00700 1 --2- 172.21.15.151:0/2975984832 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556826dbdf30 0x556826dc1530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:18.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.852+0000 7fc2dfe00700 1 -- 172.21.15.151:0/2975984832 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556826db8140 msgr2=0x556826dc3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:18.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.852+0000 7fc2dfe00700 1 --2- 172.21.15.151:0/2975984832 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556826db8140 0x556826dc3c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:18.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.852+0000 7fc2dfe00700 1 -- 172.21.15.151:0/2975984832 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556826c558d0 con 0x556826dbd610 2024-05-21T22:05:18.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.852+0000 7fc2dfe00700 1 --2- 172.21.15.151:0/2975984832 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556826dbd610 0x556826dc0ff0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fc2d0012770 tx=0x7fc2d0012a80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99abc26dd026e6f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:05:18.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.852+0000 7fc2ce000700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-21T22:05:18.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.852+0000 7fc2ce000700 1 -- 172.21.15.151:0/2975984832 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2d000f540 con 0x556826dbd610 2024-05-21T22:05:18.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.853+0000 7fc2f1369b80 1 -- 172.21.15.151:0/2975984832 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2c800fe60 con 0x556826dbd610 2024-05-21T22:05:18.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.853+0000 7fc2ce000700 1 -- 172.21.15.151:0/2975984832 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7fc2d000f6e0 con 0x556826dbd610 2024-05-21T22:05:18.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.853+0000 7fc2ce000700 1 -- 172.21.15.151:0/2975984832 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc2d000f8e0 con 0x556826dbd610 2024-05-21T22:05:18.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.853+0000 7fc2f1369b80 10 client.27457 fetch_fsmap learned FSMap version 29 2024-05-21T22:05:18.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.853+0000 7fc2f1369b80 1 -- 172.21.15.151:0/2975984832 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556826cd6060 con 0x556826dbd610 2024-05-21T22:05:18.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.853+0000 7fc2ce000700 1 -- 172.21.15.151:0/2975984832 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2c800fe60 con 0x556826dbd610 2024-05-21T22:05:18.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.853+0000 7fc2ce000700 1 -- 172.21.15.151:0/2975984832 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fc2d0010b30 con 0x556826dbd610 2024-05-21T22:05:18.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.853+0000 7fc2f1369b80 10 client.27457 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:05:18.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.853+0000 7fc2f1369b80 10 client.27457 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-05-21T22:05:18.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.854+0000 7fc2f1369b80 20 client.27457 populate_metadata read hostname 'smithi151' 2024-05-21T22:05:18.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.854+0000 7fc2f1369b80 1 --2- 172.21.15.151:0/2975984832 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x556826dd15d0 0x556826dd3a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:18.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.854+0000 7fc2f1369b80 4 client.27457 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T22:05:18.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.854+0000 7fc2f1369b80 1 -- 172.21.15.151:0/2975984832 --> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556826d17d90 con 0x556826dd15d0 2024-05-21T22:05:18.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.854+0000 7fc2d7400700 1 --2- 172.21.15.151:0/2975984832 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x556826dd15d0 0x556826dd3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:18.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.854+0000 7fc2d7400700 1 --2- 172.21.15.151:0/2975984832 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x556826dd15d0 0x556826dd3a30 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=996108357d4228d6 server_cookie=a2a715ab04657876 in_seq=0 out_seq=0 2024-05-21T22:05:18.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.854+0000 7fc2ce000700 10 client.27457 ms_handle_connect on v2:172.21.15.164:6834/2340428470 2024-05-21T22:05:18.858 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.855+0000 7fc2ce000700 1 -- 172.21.15.151:0/2975984832 <== mds.4 v2:172.21.15.164:6834/2340428470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x556826d17d90 con 0x556826dd15d0 2024-05-21T22:05:18.858 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.855+0000 7fc2ce000700 10 client.27457 handle_command_reply: tid=0 2024-05-21T22:05:18.858 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.855+0000 7fc2f1369b80 1 client.27457 shutdown 2024-05-21T22:05:18.858 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.856+0000 7fc2f1369b80 1 -- 172.21.15.151:0/2975984832 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556826dbd610 msgr2=0x556826dc0ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:18.858 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.856+0000 7fc2f1369b80 1 --2- 172.21.15.151:0/2975984832 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556826dbd610 0x556826dc0ff0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fc2d0012770 tx=0x7fc2d0012a80 comp rx=0 tx=0).stop 2024-05-21T22:05:18.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.856+0000 7fc2f1369b80 1 -- 172.21.15.151:0/2975984832 shutdown_connections 2024-05-21T22:05:18.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.856+0000 7fc2f1369b80 1 --2- 172.21.15.151:0/2975984832 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x556826dd15d0 0x556826dd3a30 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:18.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.856+0000 7fc2f1369b80 1 --2- 172.21.15.151:0/2975984832 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556826dbdf30 0x556826dc1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:18.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.856+0000 7fc2f1369b80 1 --2- 172.21.15.151:0/2975984832 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556826db8140 0x556826dc3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:18.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.856+0000 7fc2ce000700 0 client.27457 ms_handle_reset on v2:172.21.15.164:6834/2340428470 2024-05-21T22:05:18.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.856+0000 7fc2f1369b80 1 --2- 172.21.15.151:0/2975984832 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556826dbd610 0x556826dc0ff0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:18.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.856+0000 7fc2f1369b80 1 -- 172.21.15.151:0/2975984832 >> 172.21.15.151:0/2975984832 conn(0x556826ce5be0 msgr2=0x556826db1d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:05:18.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.856+0000 7fc2f1369b80 1 -- 172.21.15.151:0/2975984832 shutdown_connections 2024-05-21T22:05:18.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.856+0000 7fc2f1369b80 1 -- 172.21.15.151:0/2975984832 wait complete. 2024-05-21T22:05:18.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.856+0000 7fc2f1369b80 20 client.27457 trim_cache size 0 max 16384 2024-05-21T22:05:18.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.876+0000 7fc2f1369b80 1 Processor -- start 2024-05-21T22:05:18.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.877+0000 7fc2f1369b80 1 -- start start 2024-05-21T22:05:18.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.877+0000 7fc2f1369b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556826db8140 0x556826db7b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:18.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.877+0000 7fc2f1369b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556826dbd610 0x556826dbea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:18.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.877+0000 7fc2f1369b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556826dbdf30 0x556826db24f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:18.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.877+0000 7fc2f1369b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc2bc00d610 con 0x556826dbd610 2024-05-21T22:05:18.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.877+0000 7fc2f1369b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc2bc00f150 con 0x556826db8140 2024-05-21T22:05:18.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.877+0000 7fc2f1369b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc2bc00dce0 con 0x556826dbdf30 2024-05-21T22:05:18.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.877+0000 7fc2dfe00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556826dbd610 0x556826dbea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:18.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.877+0000 7fc2d7400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556826dbdf30 0x556826db24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:18.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.877+0000 7fc2dfe00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556826dbd610 0x556826dbea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:36130/0 (socket says 172.21.15.151:36130) 2024-05-21T22:05:18.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.877+0000 7fc2dfe00700 1 -- 172.21.15.151:0/1066895646 learned_addr learned my addr 172.21.15.151:0/1066895646 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:05:18.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.877+0000 7fc2d7400700 1 --2- 172.21.15.151:0/1066895646 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556826db8140 0x556826db7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:18.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.878+0000 7fc2d7400700 1 -- 172.21.15.151:0/1066895646 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556826db8140 msgr2=0x556826db7b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:18.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.878+0000 7fc2d7400700 1 --2- 172.21.15.151:0/1066895646 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556826db8140 0x556826db7b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:18.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.878+0000 7fc2d7400700 1 -- 172.21.15.151:0/1066895646 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556826dbd610 msgr2=0x556826dbea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:18.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.878+0000 7fc2d7400700 1 --2- 172.21.15.151:0/1066895646 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556826dbd610 0x556826dbea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:18.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.878+0000 7fc2d7400700 1 -- 172.21.15.151:0/1066895646 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556826dd40b0 con 0x556826dbdf30 2024-05-21T22:05:18.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.878+0000 7fc2dfe00700 1 --2- 172.21.15.151:0/1066895646 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556826dbd610 0x556826dbea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:05:18.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.878+0000 7fc2d7400700 1 --2- 172.21.15.151:0/1066895646 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556826dbdf30 0x556826db24f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fc2c8009320 tx=0x7fc2c8011b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=113336492d955f65 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:05:18.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.878+0000 7fc2d4c00700 1 -- 172.21.15.151:0/1066895646 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c8037030 con 0x556826dbdf30 2024-05-21T22:05:18.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.878+0000 7fc2d4c00700 1 -- 172.21.15.151:0/1066895646 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2c802a040 con 0x556826dbdf30 2024-05-21T22:05:18.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.878+0000 7fc2d4c00700 1 -- 172.21.15.151:0/1066895646 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c8029040 con 0x556826dbdf30 2024-05-21T22:05:18.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.879+0000 7fc2f1369b80 1 -- 172.21.15.151:0/1066895646 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556826dbdf30 msgr2=0x556826db24f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:18.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.879+0000 7fc2f1369b80 1 --2- 172.21.15.151:0/1066895646 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556826dbdf30 0x556826db24f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fc2c8009320 tx=0x7fc2c8011b50 comp rx=0 tx=0).stop 2024-05-21T22:05:18.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.879+0000 7fc2f1369b80 1 -- 172.21.15.151:0/1066895646 shutdown_connections 2024-05-21T22:05:18.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.879+0000 7fc2f1369b80 1 --2- 172.21.15.151:0/1066895646 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556826db8140 0x556826db7b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:18.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.879+0000 7fc2f1369b80 1 --2- 172.21.15.151:0/1066895646 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556826dbdf30 0x556826db24f0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:18.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.879+0000 7fc2f1369b80 1 --2- 172.21.15.151:0/1066895646 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556826dbd610 0x556826dbea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:18.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.879+0000 7fc2f1369b80 1 -- 172.21.15.151:0/1066895646 >> 172.21.15.151:0/1066895646 conn(0x556826ce5be0 msgr2=0x556826dbe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:05:18.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.879+0000 7fc2f1369b80 1 -- 172.21.15.151:0/1066895646 shutdown_connections 2024-05-21T22:05:18.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.879+0000 7fc2f1369b80 1 -- 172.21.15.151:0/1066895646 wait complete. 2024-05-21T22:05:18.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.880+0000 7fc2f1369b80 1 -- start start 2024-05-21T22:05:18.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.880+0000 7fc2f1369b80 1 Processor -- start 2024-05-21T22:05:18.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.880+0000 7fc2f1369b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556826db8140 0x556826dd8e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:18.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.880+0000 7fc2f1369b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556826dbd610 0x556826dd93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:18.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.880+0000 7fc2f1369b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556826dbdf30 0x556826dd9900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:18.886 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.880+0000 7fc2f1369b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc2bc00ea80 con 0x556826db8140 2024-05-21T22:05:18.886 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.880+0000 7fc2f1369b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc2bc00e3b0 con 0x556826dbd610 2024-05-21T22:05:18.886 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.880+0000 7fc2f1369b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc2bc00b5d0 con 0x556826dbdf30 2024-05-21T22:05:18.886 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.881+0000 7fc2f1369b80 10 client.0 _ll_register_callbacks cb 0x556826cafc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:05:18.886 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.881+0000 7fc2d7400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556826db8140 0x556826dd8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:18.886 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.881+0000 7fc2d7400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556826db8140 0x556826dd8e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:36146/0 (socket says 172.21.15.151:36146) 2024-05-21T22:05:18.886 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.881+0000 7fc2d7400700 1 -- 172.21.15.151:0/118263779 learned_addr learned my addr 172.21.15.151:0/118263779 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:05:18.886 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.881+0000 7fc2dfe00700 1 --2- 172.21.15.151:0/118263779 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556826dbd610 0x556826dd93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:18.886 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.881+0000 7fc2d7400700 1 --2- 172.21.15.151:0/118263779 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556826dbdf30 0x556826dd9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:18.886 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.881+0000 7fc2d7400700 1 -- 172.21.15.151:0/118263779 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556826dbdf30 msgr2=0x556826dd9900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:18.886 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.881+0000 7fc2d7400700 1 --2- 172.21.15.151:0/118263779 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556826dbdf30 0x556826dd9900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:18.886 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.881+0000 7fc2d7400700 1 -- 172.21.15.151:0/118263779 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556826dbd610 msgr2=0x556826dd93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:18.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.881+0000 7fc2d7400700 1 --2- 172.21.15.151:0/118263779 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556826dbd610 0x556826dd93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:18.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.881+0000 7fc2d7400700 1 -- 172.21.15.151:0/118263779 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556826e3b240 con 0x556826db8140 2024-05-21T22:05:18.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.881+0000 7fc2dfe00700 1 --2- 172.21.15.151:0/118263779 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556826dbd610 0x556826dd93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:05:18.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.882+0000 7fc2d7400700 1 --2- 172.21.15.151:0/118263779 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556826db8140 0x556826dd8e80 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fc2c8011ac0 tx=0x7fc2c8011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fba2017ea39f122 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:05:18.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.882+0000 7fc2ce000700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-21T22:05:18.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.882+0000 7fc2ce000700 1 -- 172.21.15.151:0/118263779 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c8037070 con 0x556826db8140 2024-05-21T22:05:18.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.882+0000 7fc2f1369b80 1 -- 172.21.15.151:0/118263779 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2c802a040 con 0x556826db8140 2024-05-21T22:05:18.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.882+0000 7fc2ce000700 1 -- 172.21.15.151:0/118263779 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7fc2c800fb00 con 0x556826db8140 2024-05-21T22:05:18.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.882+0000 7fc2ce000700 1 -- 172.21.15.151:0/118263779 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc2c8029040 con 0x556826db8140 2024-05-21T22:05:18.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.882+0000 7fc2f1369b80 10 client.17331 fetch_fsmap learned FSMap version 29 2024-05-21T22:05:18.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.882+0000 7fc2f1369b80 1 -- 172.21.15.151:0/118263779 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556826e3c520 con 0x556826db8140 2024-05-21T22:05:18.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.883+0000 7fc2ce000700 1 -- 172.21.15.151:0/118263779 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2c802a040 con 0x556826db8140 2024-05-21T22:05:18.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.883+0000 7fc2ce000700 1 -- 172.21.15.151:0/118263779 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fc2c8004980 con 0x556826db8140 2024-05-21T22:05:18.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.883+0000 7fc2f1369b80 10 client.17331 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:05:18.888 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.883+0000 7fc2f1369b80 10 client.17331 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-05-21T22:05:18.888 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.883+0000 7fc2f1369b80 20 client.17331 populate_metadata read hostname 'smithi151' 2024-05-21T22:05:18.888 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.883+0000 7fc2f1369b80 1 --2- 172.21.15.151:0/118263779 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x556826e3c9c0 0x556826e3ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:18.888 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.883+0000 7fc2f1369b80 4 client.17331 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T22:05:18.888 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.883+0000 7fc2f1369b80 1 -- 172.21.15.151:0/118263779 --> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556826d11570 con 0x556826e3c9c0 2024-05-21T22:05:18.888 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.883+0000 7fc2dfe00700 1 --2- 172.21.15.151:0/118263779 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x556826e3c9c0 0x556826e3ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:18.888 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.884+0000 7fc2dfe00700 1 --2- 172.21.15.151:0/118263779 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x556826e3c9c0 0x556826e3ede0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=239eb88a8450df5f server_cookie=bd32ac1a9eba6011 in_seq=0 out_seq=0 2024-05-21T22:05:18.888 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.884+0000 7fc2ce000700 10 client.17331 ms_handle_connect on v2:172.21.15.164:6834/2340428470 2024-05-21T22:05:18.888 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.884+0000 7fc2ce000700 1 -- 172.21.15.151:0/118263779 <== mds.4 v2:172.21.15.164:6834/2340428470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556826d11570 con 0x556826e3c9c0 2024-05-21T22:05:18.888 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.884+0000 7fc2ce000700 10 client.17331 handle_command_reply: tid=0 2024-05-21T22:05:18.888 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.884+0000 7fc2f1369b80 1 client.17331 shutdown 2024-05-21T22:05:18.888 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.884+0000 7fc2f1369b80 1 -- 172.21.15.151:0/118263779 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556826db8140 msgr2=0x556826dd8e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:18.888 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.884+0000 7fc2f1369b80 1 --2- 172.21.15.151:0/118263779 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556826db8140 0x556826dd8e80 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fc2c8011ac0 tx=0x7fc2c8011920 comp rx=0 tx=0).stop 2024-05-21T22:05:18.889 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.884+0000 7fc2f1369b80 1 -- 172.21.15.151:0/118263779 shutdown_connections 2024-05-21T22:05:18.889 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.884+0000 7fc2f1369b80 1 --2- 172.21.15.151:0/118263779 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x556826e3c9c0 0x556826e3ede0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:18.889 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.884+0000 7fc2f1369b80 1 --2- 172.21.15.151:0/118263779 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556826dbdf30 0x556826dd9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:18.889 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.884+0000 7fc2f1369b80 1 --2- 172.21.15.151:0/118263779 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556826db8140 0x556826dd8e80 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:18.890 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.884+0000 7fc2f1369b80 1 --2- 172.21.15.151:0/118263779 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x556826dbd610 0x556826dd93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:18.890 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.884+0000 7fc2f1369b80 1 -- 172.21.15.151:0/118263779 >> 172.21.15.151:0/118263779 conn(0x556826ce5be0 msgr2=0x556826dbc110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:05:18.890 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.884+0000 7fc2ce000700 0 client.17331 ms_handle_reset on v2:172.21.15.164:6834/2340428470 2024-05-21T22:05:18.890 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.885+0000 7fc2f1369b80 1 -- 172.21.15.151:0/118263779 shutdown_connections 2024-05-21T22:05:18.890 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.885+0000 7fc2f1369b80 1 -- 172.21.15.151:0/118263779 wait complete. 2024-05-21T22:05:18.890 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.885+0000 7fc2f1369b80 20 client.17331 trim_cache size 0 max 16384 2024-05-21T22:05:18.890 INFO:teuthology.orchestra.run.smithi151.stdout:[] 2024-05-21T22:05:18.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.886+0000 7fc2e6a00700 1 -- 172.21.15.151:0/46145558 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fc2c40b2a70 msgr2=0x7fc2c40b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:18.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.886+0000 7fc2e6a00700 1 --2- 172.21.15.151:0/46145558 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fc2c40b2a70 0x7fc2c40b4f30 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc2d80e0750 tx=0x7fc2d8009040 comp rx=0 tx=0).stop 2024-05-21T22:05:18.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.886+0000 7fc2e6a00700 1 -- 172.21.15.151:0/46145558 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc2e01095d0 msgr2=0x7fc2e01a0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:18.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.886+0000 7fc2e6a00700 1 --2- 172.21.15.151:0/46145558 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc2e01095d0 0x7fc2e01a0650 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fc2d800b070 tx=0x7fc2d8004c80 comp rx=0 tx=0).stop 2024-05-21T22:05:18.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.886+0000 7fc2e6a00700 1 -- 172.21.15.151:0/46145558 shutdown_connections 2024-05-21T22:05:18.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.886+0000 7fc2e6a00700 1 --2- 172.21.15.151:0/46145558 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fc2c40b2a70 0x7fc2c40b4f30 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:18.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.886+0000 7fc2e6a00700 1 --2- 172.21.15.151:0/46145558 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc2e01095d0 0x7fc2e01a0650 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:18.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.886+0000 7fc2e6a00700 1 --2- 172.21.15.151:0/46145558 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2e01035c0 0x7fc2e019bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:18.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.886+0000 7fc2e6a00700 1 --2- 172.21.15.151:0/46145558 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc2e0103f80 0x7fc2e019c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:18.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.886+0000 7fc2e6a00700 1 -- 172.21.15.151:0/46145558 >> 172.21.15.151:0/46145558 conn(0x7fc2e007a610 msgr2=0x7fc2e0101f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:05:18.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.886+0000 7fc2e6a00700 1 -- 172.21.15.151:0/46145558 shutdown_connections 2024-05-21T22:05:18.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:18.886+0000 7fc2e6a00700 1 -- 172.21.15.151:0/46145558 wait complete. 2024-05-21T22:05:19.782 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-21T22:05:19.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:05:19 smithi151 ceph-mon[31623]: pgmap v300: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 170 B/s wr, 12 op/s 2024-05-21T22:05:19.985 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:05:19 smithi184 ceph-mon[36942]: pgmap v300: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 170 B/s wr, 12 op/s 2024-05-21T22:05:20.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:05:19 smithi164 ceph-mon[36269]: pgmap v300: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 170 B/s wr, 12 op/s 2024-05-21T22:05:20.238 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:05:21.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:05:21 smithi151 ceph-mon[31623]: pgmap v301: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 85 B/s wr, 8 op/s 2024-05-21T22:05:21.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:05:21 smithi184 ceph-mon[36942]: pgmap v301: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 85 B/s wr, 8 op/s 2024-05-21T22:05:22.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:05:21 smithi164 ceph-mon[36269]: pgmap v301: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 85 B/s wr, 8 op/s 2024-05-21T22:05:22.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.375+0000 7efec0c00700 1 -- 172.21.15.151:0/2158231114 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efebc105760 msgr2=0x7efebc105be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:22.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.375+0000 7efec0c00700 1 --2- 172.21.15.151:0/2158231114 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efebc105760 0x7efebc105be0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7efea8009990 tx=0x7efea8009ca0 comp rx=0 tx=0).stop 2024-05-21T22:05:22.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.375+0000 7efec0c00700 1 -- 172.21.15.151:0/2158231114 shutdown_connections 2024-05-21T22:05:22.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.375+0000 7efec0c00700 1 --2- 172.21.15.151:0/2158231114 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efebc106120 0x7efebc112630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:22.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.375+0000 7efec0c00700 1 --2- 172.21.15.151:0/2158231114 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efebc10b760 0x7efebc10bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:22.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.375+0000 7efec0c00700 1 --2- 172.21.15.151:0/2158231114 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efebc105760 0x7efebc105be0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:22.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.375+0000 7efec0c00700 1 -- 172.21.15.151:0/2158231114 >> 172.21.15.151:0/2158231114 conn(0x7efebc101250 msgr2=0x7efebc103670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:05:22.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.375+0000 7efec0c00700 1 -- 172.21.15.151:0/2158231114 shutdown_connections 2024-05-21T22:05:22.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.375+0000 7efec0c00700 1 -- 172.21.15.151:0/2158231114 wait complete. 2024-05-21T22:05:22.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.376+0000 7efec0c00700 1 Processor -- start 2024-05-21T22:05:22.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.376+0000 7efec0c00700 1 -- start start 2024-05-21T22:05:22.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.377+0000 7efec0c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efebc105760 0x7efebc077a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:22.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.377+0000 7efec0c00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efebc106120 0x7efebc077f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:22.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.377+0000 7efec0c00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efebc10b760 0x7efebc0784d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:22.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.377+0000 7efec0c00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7efebc1159a0 con 0x7efebc105760 2024-05-21T22:05:22.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.377+0000 7efec0c00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7efebc115820 con 0x7efebc10b760 2024-05-21T22:05:22.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.377+0000 7efec0c00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7efebc115b20 con 0x7efebc106120 2024-05-21T22:05:22.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.377+0000 7efeb9e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efebc105760 0x7efebc077a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:22.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.377+0000 7efeb9e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efebc105760 0x7efebc077a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:36162/0 (socket says 172.21.15.151:36162) 2024-05-21T22:05:22.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.377+0000 7efeb9e00700 1 -- 172.21.15.151:0/1958469741 learned_addr learned my addr 172.21.15.151:0/1958469741 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:05:22.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.377+0000 7efeb9400700 1 --2- 172.21.15.151:0/1958469741 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efebc106120 0x7efebc077f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:22.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.378+0000 7efeba800700 1 --2- 172.21.15.151:0/1958469741 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efebc10b760 0x7efebc0784d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:22.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.378+0000 7efeb9e00700 1 -- 172.21.15.151:0/1958469741 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efebc106120 msgr2=0x7efebc077f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:22.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.378+0000 7efeb9e00700 1 --2- 172.21.15.151:0/1958469741 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efebc106120 0x7efebc077f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:22.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.378+0000 7efeb9e00700 1 -- 172.21.15.151:0/1958469741 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efebc10b760 msgr2=0x7efebc0784d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:22.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.378+0000 7efeb9e00700 1 --2- 172.21.15.151:0/1958469741 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efebc10b760 0x7efebc0784d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:22.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.378+0000 7efeb9e00700 1 -- 172.21.15.151:0/1958469741 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efebc1ac4e0 con 0x7efebc105760 2024-05-21T22:05:22.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.378+0000 7efeba800700 1 --2- 172.21.15.151:0/1958469741 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efebc10b760 0x7efebc0784d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T22:05:22.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.378+0000 7efeb9e00700 1 --2- 172.21.15.151:0/1958469741 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efebc105760 0x7efebc077a50 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7efeac00e9c0 tx=0x7efeac00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:05:22.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.378+0000 7efeb2a00700 1 -- 172.21.15.151:0/1958469741 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeac00cbd0 con 0x7efebc105760 2024-05-21T22:05:22.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.378+0000 7efec0c00700 1 -- 172.21.15.151:0/1958469741 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efebc1ac7d0 con 0x7efebc105760 2024-05-21T22:05:22.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.378+0000 7efeb2a00700 1 -- 172.21.15.151:0/1958469741 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efeac00cd70 con 0x7efebc105760 2024-05-21T22:05:22.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.378+0000 7efeb2a00700 1 -- 172.21.15.151:0/1958469741 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeac005680 con 0x7efebc105760 2024-05-21T22:05:22.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.379+0000 7efec0c00700 1 -- 172.21.15.151:0/1958469741 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efebc1acd10 con 0x7efebc105760 2024-05-21T22:05:22.389 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.382+0000 7efeb2a00700 1 -- 172.21.15.151:0/1958469741 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7efeac004750 con 0x7efebc105760 2024-05-21T22:05:22.390 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.383+0000 7efecb59bb80 1 Processor -- start 2024-05-21T22:05:22.390 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.384+0000 7efecb59bb80 1 -- start start 2024-05-21T22:05:22.390 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.384+0000 7efecb59bb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5644168ef610 0x5644168ef9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:22.390 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.384+0000 7efecb59bb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5644168eff30 0x5644168e9b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:22.390 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.384+0000 7efecb59bb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644168ea140 0x5644168ea5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:22.390 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.384+0000 7efecb59bb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x564416847e40 con 0x5644168ef610 2024-05-21T22:05:22.390 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.384+0000 7efecb59bb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5644168379e0 con 0x5644168ea140 2024-05-21T22:05:22.390 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.384+0000 7efecb59bb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56441683ba20 con 0x5644168eff30 2024-05-21T22:05:22.390 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.385+0000 7efeb2a00700 1 --2- 172.21.15.151:0/1958469741 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7efea00b2a70 0x7efea00b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:22.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.385+0000 7efeba800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5644168ef610 0x5644168ef9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:22.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.385+0000 7efeba800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5644168ef610 0x5644168ef9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:36178/0 (socket says 172.21.15.151:36178) 2024-05-21T22:05:22.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.385+0000 7efeba800700 1 -- 172.21.15.151:0/917311811 learned_addr learned my addr 172.21.15.151:0/917311811 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:05:22.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.385+0000 7efeb9e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5644168eff30 0x5644168e9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:22.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.385+0000 7efeb9400700 1 --2- 172.21.15.151:0/917311811 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644168ea140 0x5644168ea5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:22.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.385+0000 7efeb2a00700 1 -- 172.21.15.151:0/1958469741 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7efeac0e2820 con 0x7efebc105760 2024-05-21T22:05:22.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.385+0000 7efeba800700 1 --2- 172.21.15.151:0/1958469741 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7efea00b2a70 0x7efea00b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:22.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.385+0000 7efeb9400700 1 -- 172.21.15.151:0/917311811 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5644168eff30 msgr2=0x5644168e9b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:22.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.385+0000 7efeb9400700 1 --2- 172.21.15.151:0/917311811 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5644168eff30 0x5644168e9b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:22.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.385+0000 7efeb9400700 1 -- 172.21.15.151:0/917311811 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5644168ef610 msgr2=0x5644168ef9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:22.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.385+0000 7efeb9400700 1 --2- 172.21.15.151:0/917311811 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5644168ef610 0x5644168ef9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:22.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.385+0000 7efeb9400700 1 -- 172.21.15.151:0/917311811 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564416800420 con 0x5644168ea140 2024-05-21T22:05:22.392 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.386+0000 7efeb9400700 1 --2- 172.21.15.151:0/917311811 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644168ea140 0x5644168ea5c0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7efea8006010 tx=0x7efea8005940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b015907c6f2e3b4c server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:05:22.392 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.386+0000 7efeba800700 1 --2- 172.21.15.151:0/1958469741 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7efea00b2a70 0x7efea00b4f30 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7efeb40094f0 tx=0x7efeb4004b40 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:05:22.392 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.386+0000 7efea7400700 1 -- 172.21.15.151:0/917311811 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea8027070 con 0x5644168ea140 2024-05-21T22:05:22.392 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.386+0000 7efea7400700 1 -- 172.21.15.151:0/917311811 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efea80056a0 con 0x5644168ea140 2024-05-21T22:05:22.392 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.386+0000 7efea7400700 1 -- 172.21.15.151:0/917311811 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea8003a00 con 0x5644168ea140 2024-05-21T22:05:22.392 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.387+0000 7efecb59bb80 1 -- 172.21.15.151:0/917311811 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644168ea140 msgr2=0x5644168ea5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:22.392 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.387+0000 7efecb59bb80 1 --2- 172.21.15.151:0/917311811 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644168ea140 0x5644168ea5c0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7efea8006010 tx=0x7efea8005940 comp rx=0 tx=0).stop 2024-05-21T22:05:22.392 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.387+0000 7efecb59bb80 1 -- 172.21.15.151:0/917311811 shutdown_connections 2024-05-21T22:05:22.392 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.387+0000 7efecb59bb80 1 --2- 172.21.15.151:0/917311811 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644168ea140 0x5644168ea5c0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:22.392 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.387+0000 7efecb59bb80 1 --2- 172.21.15.151:0/917311811 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5644168ef610 0x5644168ef9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:22.392 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.387+0000 7efecb59bb80 1 --2- 172.21.15.151:0/917311811 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5644168eff30 0x5644168e9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:22.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.387+0000 7efecb59bb80 1 -- 172.21.15.151:0/917311811 >> 172.21.15.151:0/917311811 conn(0x564416817be0 msgr2=0x5644168e4690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:05:22.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.387+0000 7efecb59bb80 1 -- 172.21.15.151:0/917311811 shutdown_connections 2024-05-21T22:05:22.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.387+0000 7efecb59bb80 1 -- 172.21.15.151:0/917311811 wait complete. 2024-05-21T22:05:22.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.388+0000 7efecb59bb80 1 -- start start 2024-05-21T22:05:22.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.388+0000 7efecb59bb80 1 Processor -- start 2024-05-21T22:05:22.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.389+0000 7efecb59bb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5644168ea140 0x5644168f5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:22.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.389+0000 7efecb59bb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5644168ef610 0x5644168f3000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:22.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.389+0000 7efecb59bb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644168eff30 0x5644168f3540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:22.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.389+0000 7efecb59bb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56441673dfa0 con 0x5644168ef610 2024-05-21T22:05:22.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.389+0000 7efecb59bb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5644167d8760 con 0x5644168eff30 2024-05-21T22:05:22.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.389+0000 7efecb59bb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5644168f6980 con 0x5644168ea140 2024-05-21T22:05:22.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.389+0000 7efecb59bb80 10 client.0 _ll_register_callbacks cb 0x5644167e1c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:05:22.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.389+0000 7efeb9400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5644168ef610 0x5644168f3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:22.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.389+0000 7efeb9400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5644168ef610 0x5644168f3000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:36182/0 (socket says 172.21.15.151:36182) 2024-05-21T22:05:22.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.389+0000 7efeb9400700 1 -- 172.21.15.151:0/4231256812 learned_addr learned my addr 172.21.15.151:0/4231256812 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:05:22.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.389+0000 7efeba800700 1 --2- 172.21.15.151:0/4231256812 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644168eff30 0x5644168f3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:22.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.389+0000 7efeb9e00700 1 --2- 172.21.15.151:0/4231256812 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5644168ea140 0x5644168f5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:22.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.390+0000 7efeb9400700 1 -- 172.21.15.151:0/4231256812 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5644168ea140 msgr2=0x5644168f5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:22.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.390+0000 7efeb9400700 1 --2- 172.21.15.151:0/4231256812 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5644168ea140 0x5644168f5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:22.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.390+0000 7efeb9400700 1 -- 172.21.15.151:0/4231256812 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644168eff30 msgr2=0x5644168f3540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:22.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.390+0000 7efeb9400700 1 --2- 172.21.15.151:0/4231256812 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644168eff30 0x5644168f3540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:22.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.390+0000 7efeb9400700 1 -- 172.21.15.151:0/4231256812 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644167878d0 con 0x5644168ef610 2024-05-21T22:05:22.398 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.390+0000 7efeba800700 1 --2- 172.21.15.151:0/4231256812 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644168eff30 0x5644168f3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:05:22.398 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.390+0000 7efeb9e00700 1 --2- 172.21.15.151:0/4231256812 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5644168ea140 0x5644168f5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:05:22.398 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.390+0000 7efeb9400700 1 --2- 172.21.15.151:0/4231256812 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5644168ef610 0x5644168f3000 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7efea8005f50 tx=0x7efea800bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cf5f564bbae0abe server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:05:22.398 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.390+0000 7efe9be00700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-21T22:05:22.398 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.390+0000 7efe9be00700 1 -- 172.21.15.151:0/4231256812 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea802f020 con 0x5644168ef610 2024-05-21T22:05:22.398 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.390+0000 7efe9be00700 1 -- 172.21.15.151:0/4231256812 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7efea8003890 con 0x5644168ef610 2024-05-21T22:05:22.398 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.390+0000 7efecb59bb80 1 -- 172.21.15.151:0/4231256812 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efea80056a0 con 0x5644168ef610 2024-05-21T22:05:22.398 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.391+0000 7efe9be00700 1 -- 172.21.15.151:0/4231256812 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7efea8027040 con 0x5644168ef610 2024-05-21T22:05:22.398 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.391+0000 7efecb59bb80 10 client.17343 fetch_fsmap learned FSMap version 29 2024-05-21T22:05:22.398 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.391+0000 7efecb59bb80 1 -- 172.21.15.151:0/4231256812 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564416808060 con 0x5644168ef610 2024-05-21T22:05:22.398 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.391+0000 7efe9be00700 1 -- 172.21.15.151:0/4231256812 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efea80056a0 con 0x5644168ef610 2024-05-21T22:05:22.398 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.391+0000 7efe9be00700 1 -- 172.21.15.151:0/4231256812 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7efea8021070 con 0x5644168ef610 2024-05-21T22:05:22.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.391+0000 7efecb59bb80 10 client.17343 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:05:22.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.391+0000 7efecb59bb80 10 client.17343 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-21T22:05:22.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.391+0000 7efecb59bb80 20 client.17343 populate_metadata read hostname 'smithi151' 2024-05-21T22:05:22.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.391+0000 7efecb59bb80 1 --2- 172.21.15.151:0/4231256812 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x564416903340 0x564416905730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:22.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.391+0000 7efecb59bb80 4 client.17343 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T22:05:22.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.391+0000 7efecb59bb80 1 -- 172.21.15.151:0/4231256812 --> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564416849d90 con 0x564416903340 2024-05-21T22:05:22.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.392+0000 7efeb9e00700 1 --2- 172.21.15.151:0/4231256812 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x564416903340 0x564416905730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:22.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.392+0000 7efeb9e00700 1 --2- 172.21.15.151:0/4231256812 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x564416903340 0x564416905730 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d2690728910a7bd server_cookie=b2d358ead79ab98d in_seq=0 out_seq=0 2024-05-21T22:05:22.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.392+0000 7efe9be00700 10 client.17343 ms_handle_connect on v2:172.21.15.184:6836/2988194888 2024-05-21T22:05:22.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.393+0000 7efe9be00700 1 -- 172.21.15.151:0/4231256812 <== mds.3 v2:172.21.15.184:6836/2988194888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x564416849d90 con 0x564416903340 2024-05-21T22:05:22.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.393+0000 7efe9be00700 10 client.17343 handle_command_reply: tid=0 2024-05-21T22:05:22.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.393+0000 7efecb59bb80 1 client.17343 shutdown 2024-05-21T22:05:22.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.394+0000 7efecb59bb80 1 -- 172.21.15.151:0/4231256812 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5644168ef610 msgr2=0x5644168f3000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:22.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.394+0000 7efecb59bb80 1 --2- 172.21.15.151:0/4231256812 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5644168ef610 0x5644168f3000 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7efea8005f50 tx=0x7efea800bcd0 comp rx=0 tx=0).stop 2024-05-21T22:05:22.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.394+0000 7efecb59bb80 1 -- 172.21.15.151:0/4231256812 shutdown_connections 2024-05-21T22:05:22.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.394+0000 7efecb59bb80 1 --2- 172.21.15.151:0/4231256812 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x564416903340 0x564416905730 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:22.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.394+0000 7efecb59bb80 1 --2- 172.21.15.151:0/4231256812 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644168eff30 0x5644168f3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:22.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.394+0000 7efecb59bb80 1 --2- 172.21.15.151:0/4231256812 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5644168ea140 0x5644168f5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:22.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.394+0000 7efecb59bb80 1 --2- 172.21.15.151:0/4231256812 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5644168ef610 0x5644168f3000 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:22.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.394+0000 7efe9be00700 0 client.17343 ms_handle_reset on v2:172.21.15.184:6836/2988194888 2024-05-21T22:05:22.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.394+0000 7efecb59bb80 1 -- 172.21.15.151:0/4231256812 >> 172.21.15.151:0/4231256812 conn(0x564416817be0 msgr2=0x5644168e3d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:05:22.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.394+0000 7efecb59bb80 1 -- 172.21.15.151:0/4231256812 shutdown_connections 2024-05-21T22:05:22.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.394+0000 7efecb59bb80 1 -- 172.21.15.151:0/4231256812 wait complete. 2024-05-21T22:05:22.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.394+0000 7efecb59bb80 20 client.17343 trim_cache size 0 max 16384 2024-05-21T22:05:22.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.410+0000 7efecb59bb80 1 Processor -- start 2024-05-21T22:05:22.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.411+0000 7efecb59bb80 1 -- start start 2024-05-21T22:05:22.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.411+0000 7efecb59bb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5644168ea140 0x5644168e9b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:22.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.411+0000 7efecb59bb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644168ef610 0x5644168f0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:22.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.411+0000 7efecb59bb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5644168eff30 0x5644168f3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:22.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.411+0000 7efecb59bb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7efebc1b44f0 con 0x5644168ea140 2024-05-21T22:05:22.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.411+0000 7efecb59bb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7efebc1b6030 con 0x5644168ef610 2024-05-21T22:05:22.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.411+0000 7efecb59bb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7efebc1b4bc0 con 0x5644168eff30 2024-05-21T22:05:22.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.411+0000 7efeb9e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5644168ea140 0x5644168e9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:22.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.411+0000 7efeb9e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5644168ea140 0x5644168e9b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:36190/0 (socket says 172.21.15.151:36190) 2024-05-21T22:05:22.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.411+0000 7efeb9e00700 1 -- 172.21.15.151:0/3075093260 learned_addr learned my addr 172.21.15.151:0/3075093260 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:05:22.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.411+0000 7efeb9400700 1 --2- 172.21.15.151:0/3075093260 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644168ef610 0x5644168f0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:22.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.411+0000 7efeba800700 1 --2- 172.21.15.151:0/3075093260 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5644168eff30 0x5644168f3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:22.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.412+0000 7efeb9e00700 1 -- 172.21.15.151:0/3075093260 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5644168eff30 msgr2=0x5644168f3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:22.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.412+0000 7efeb9e00700 1 --2- 172.21.15.151:0/3075093260 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5644168eff30 0x5644168f3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:22.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.412+0000 7efeb9e00700 1 -- 172.21.15.151:0/3075093260 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644168ef610 msgr2=0x5644168f0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:22.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.412+0000 7efeb9e00700 1 --2- 172.21.15.151:0/3075093260 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644168ef610 0x5644168f0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:22.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.412+0000 7efeb9e00700 1 -- 172.21.15.151:0/3075093260 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644168e60e0 con 0x5644168ea140 2024-05-21T22:05:22.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.412+0000 7efeb9e00700 1 --2- 172.21.15.151:0/3075093260 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5644168ea140 0x5644168e9b20 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7efeac0050d0 tx=0x7efeac077660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=923ce2bf6da67bf server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:05:22.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.412+0000 7efea6a00700 1 -- 172.21.15.151:0/3075093260 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeac0a5220 con 0x5644168ea140 2024-05-21T22:05:22.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.412+0000 7efea6a00700 1 -- 172.21.15.151:0/3075093260 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efeac0b2230 con 0x5644168ea140 2024-05-21T22:05:22.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.412+0000 7efea6a00700 1 -- 172.21.15.151:0/3075093260 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeac0a9f50 con 0x5644168ea140 2024-05-21T22:05:22.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.413+0000 7efecb59bb80 1 -- 172.21.15.151:0/3075093260 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5644168ea140 msgr2=0x5644168e9b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:22.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.413+0000 7efecb59bb80 1 --2- 172.21.15.151:0/3075093260 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5644168ea140 0x5644168e9b20 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7efeac0050d0 tx=0x7efeac077660 comp rx=0 tx=0).stop 2024-05-21T22:05:22.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.413+0000 7efecb59bb80 1 -- 172.21.15.151:0/3075093260 shutdown_connections 2024-05-21T22:05:22.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.413+0000 7efecb59bb80 1 --2- 172.21.15.151:0/3075093260 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5644168eff30 0x5644168f3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:22.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.413+0000 7efecb59bb80 1 --2- 172.21.15.151:0/3075093260 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5644168ea140 0x5644168e9b20 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:22.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.413+0000 7efecb59bb80 1 --2- 172.21.15.151:0/3075093260 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644168ef610 0x5644168f0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:22.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.413+0000 7efecb59bb80 1 -- 172.21.15.151:0/3075093260 >> 172.21.15.151:0/3075093260 conn(0x564416817be0 msgr2=0x5644168f46b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:05:22.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.413+0000 7efecb59bb80 1 -- 172.21.15.151:0/3075093260 shutdown_connections 2024-05-21T22:05:22.417 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.413+0000 7efecb59bb80 1 -- 172.21.15.151:0/3075093260 wait complete. 2024-05-21T22:05:22.420 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.414+0000 7efecb59bb80 1 -- start start 2024-05-21T22:05:22.421 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.414+0000 7efecb59bb80 1 Processor -- start 2024-05-21T22:05:22.421 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.414+0000 7efecb59bb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644168ea140 0x564416908a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:22.421 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.414+0000 7efecb59bb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5644168ef610 0x564416908f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:22.421 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.414+0000 7efecb59bb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5644168eff30 0x564416906030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:22.421 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.414+0000 7efecb59bb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7efebc1b5960 con 0x5644168ef610 2024-05-21T22:05:22.421 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.414+0000 7efecb59bb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7efebc1b5290 con 0x5644168ea140 2024-05-21T22:05:22.421 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.414+0000 7efecb59bb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7efebc1b24b0 con 0x5644168eff30 2024-05-21T22:05:22.421 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.415+0000 7efecb59bb80 10 client.0 _ll_register_callbacks cb 0x5644167e1c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:05:22.421 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.415+0000 7efeb9e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644168ea140 0x564416908a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:22.421 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.415+0000 7efeb9400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5644168ef610 0x564416908f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:22.421 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.415+0000 7efeb9e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644168ea140 0x564416908a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:48656/0 (socket says 172.21.15.151:48656) 2024-05-21T22:05:22.421 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.415+0000 7efeb9400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5644168ef610 0x564416908f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:36194/0 (socket says 172.21.15.151:36194) 2024-05-21T22:05:22.421 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.415+0000 7efeb9e00700 1 -- 172.21.15.151:0/2508250705 learned_addr learned my addr 172.21.15.151:0/2508250705 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:05:22.421 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.415+0000 7efeba800700 1 --2- 172.21.15.151:0/2508250705 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5644168eff30 0x564416906030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:22.422 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.415+0000 7efeb9e00700 1 -- 172.21.15.151:0/2508250705 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5644168eff30 msgr2=0x564416906030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:22.422 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.415+0000 7efeb9e00700 1 --2- 172.21.15.151:0/2508250705 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5644168eff30 0x564416906030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:22.422 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.415+0000 7efeb9e00700 1 -- 172.21.15.151:0/2508250705 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5644168ef610 msgr2=0x564416908f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:22.422 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.415+0000 7efeb9e00700 1 --2- 172.21.15.151:0/2508250705 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5644168ef610 0x564416908f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:22.422 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.415+0000 7efeb9e00700 1 -- 172.21.15.151:0/2508250705 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564416906870 con 0x5644168ea140 2024-05-21T22:05:22.422 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.415+0000 7efeb9400700 1 --2- 172.21.15.151:0/2508250705 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5644168ef610 0x564416908f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:05:22.422 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.416+0000 7efeb9e00700 1 --2- 172.21.15.151:0/2508250705 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644168ea140 0x564416908a40 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7efeac0a6d00 tx=0x7efeac0775f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16964a29ef6546f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:05:22.422 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.416+0000 7efe9be00700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-21T22:05:22.422 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.416+0000 7efe9be00700 1 -- 172.21.15.151:0/2508250705 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeac0c2020 con 0x5644168ea140 2024-05-21T22:05:22.422 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.416+0000 7efe9be00700 1 -- 172.21.15.151:0/2508250705 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7efeac0a9f00 con 0x5644168ea140 2024-05-21T22:05:22.422 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.416+0000 7efecb59bb80 1 -- 172.21.15.151:0/2508250705 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564416843570 con 0x5644168ea140 2024-05-21T22:05:22.422 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.416+0000 7efe9be00700 1 -- 172.21.15.151:0/2508250705 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7efeac0aa190 con 0x5644168ea140 2024-05-21T22:05:22.422 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.416+0000 7efecb59bb80 10 client.27479 fetch_fsmap learned FSMap version 29 2024-05-21T22:05:22.423 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.416+0000 7efecb59bb80 1 -- 172.21.15.151:0/2508250705 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56441696d7c0 con 0x5644168ea140 2024-05-21T22:05:22.423 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.416+0000 7efe9be00700 1 -- 172.21.15.151:0/2508250705 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x564416843570 con 0x5644168ea140 2024-05-21T22:05:22.423 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.417+0000 7efe9be00700 1 -- 172.21.15.151:0/2508250705 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7efeac0cc070 con 0x5644168ea140 2024-05-21T22:05:22.423 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.417+0000 7efecb59bb80 10 client.27479 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:05:22.423 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.417+0000 7efecb59bb80 10 client.27479 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-21T22:05:22.423 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.417+0000 7efecb59bb80 20 client.27479 populate_metadata read hostname 'smithi151' 2024-05-21T22:05:22.423 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.417+0000 7efecb59bb80 1 --2- 172.21.15.151:0/2508250705 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x56441696de10 0x564416970230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:22.423 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.417+0000 7efecb59bb80 4 client.27479 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T22:05:22.423 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.417+0000 7efecb59bb80 1 -- 172.21.15.151:0/2508250705 --> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5644166ba250 con 0x56441696de10 2024-05-21T22:05:22.423 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.417+0000 7efeb9400700 1 --2- 172.21.15.151:0/2508250705 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x56441696de10 0x564416970230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:22.423 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.418+0000 7efeb9400700 1 --2- 172.21.15.151:0/2508250705 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x56441696de10 0x564416970230 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c55cd3810d138139 server_cookie=8edf14eaf432ab06 in_seq=0 out_seq=0 2024-05-21T22:05:22.423 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.418+0000 7efe9be00700 10 client.27479 ms_handle_connect on v2:172.21.15.184:6836/2988194888 2024-05-21T22:05:22.423 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.418+0000 7efe9be00700 1 -- 172.21.15.151:0/2508250705 <== mds.3 v2:172.21.15.184:6836/2988194888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5644166ba250 con 0x56441696de10 2024-05-21T22:05:22.423 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.418+0000 7efe9be00700 10 client.27479 handle_command_reply: tid=0 2024-05-21T22:05:22.424 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.418+0000 7efecb59bb80 1 client.27479 shutdown 2024-05-21T22:05:22.424 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.418+0000 7efecb59bb80 1 -- 172.21.15.151:0/2508250705 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644168ea140 msgr2=0x564416908a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:22.424 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.418+0000 7efecb59bb80 1 --2- 172.21.15.151:0/2508250705 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644168ea140 0x564416908a40 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7efeac0a6d00 tx=0x7efeac0775f0 comp rx=0 tx=0).stop 2024-05-21T22:05:22.424 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.418+0000 7efecb59bb80 1 -- 172.21.15.151:0/2508250705 shutdown_connections 2024-05-21T22:05:22.424 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.418+0000 7efecb59bb80 1 --2- 172.21.15.151:0/2508250705 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x56441696de10 0x564416970230 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:22.424 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.418+0000 7efecb59bb80 1 --2- 172.21.15.151:0/2508250705 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5644168ea140 0x564416908a40 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:22.424 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.418+0000 7efecb59bb80 1 --2- 172.21.15.151:0/2508250705 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5644168eff30 0x564416906030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:22.424 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.418+0000 7efe9be00700 0 client.27479 ms_handle_reset on v2:172.21.15.184:6836/2988194888 2024-05-21T22:05:22.424 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.418+0000 7efecb59bb80 1 --2- 172.21.15.151:0/2508250705 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5644168ef610 0x564416908f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:22.424 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.418+0000 7efecb59bb80 1 -- 172.21.15.151:0/2508250705 >> 172.21.15.151:0/2508250705 conn(0x564416817be0 msgr2=0x5644168ee9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:05:22.424 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.418+0000 7efecb59bb80 1 -- 172.21.15.151:0/2508250705 shutdown_connections 2024-05-21T22:05:22.424 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.418+0000 7efecb59bb80 1 -- 172.21.15.151:0/2508250705 wait complete. 2024-05-21T22:05:22.424 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.419+0000 7efecb59bb80 20 client.27479 trim_cache size 0 max 16384 2024-05-21T22:05:22.425 INFO:teuthology.orchestra.run.smithi151.stdout:[] 2024-05-21T22:05:22.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.419+0000 7efec0c00700 1 -- 172.21.15.151:0/1958469741 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7efea00b2a70 msgr2=0x7efea00b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:22.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.419+0000 7efec0c00700 1 --2- 172.21.15.151:0/1958469741 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7efea00b2a70 0x7efea00b4f30 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7efeb40094f0 tx=0x7efeb4004b40 comp rx=0 tx=0).stop 2024-05-21T22:05:22.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.419+0000 7efec0c00700 1 -- 172.21.15.151:0/1958469741 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efebc105760 msgr2=0x7efebc077a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:22.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.419+0000 7efec0c00700 1 --2- 172.21.15.151:0/1958469741 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efebc105760 0x7efebc077a50 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7efeac00e9c0 tx=0x7efeac00ecd0 comp rx=0 tx=0).stop 2024-05-21T22:05:22.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.419+0000 7efec0c00700 1 -- 172.21.15.151:0/1958469741 shutdown_connections 2024-05-21T22:05:22.426 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.419+0000 7efec0c00700 1 --2- 172.21.15.151:0/1958469741 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7efea00b2a70 0x7efea00b4f30 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:22.426 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.419+0000 7efec0c00700 1 --2- 172.21.15.151:0/1958469741 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efebc10b760 0x7efebc0784d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:22.426 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.419+0000 7efec0c00700 1 --2- 172.21.15.151:0/1958469741 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efebc105760 0x7efebc077a50 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:22.426 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.419+0000 7efec0c00700 1 --2- 172.21.15.151:0/1958469741 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efebc106120 0x7efebc077f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:22.426 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.419+0000 7efec0c00700 1 -- 172.21.15.151:0/1958469741 >> 172.21.15.151:0/1958469741 conn(0x7efebc101250 msgr2=0x7efebc101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:05:22.426 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.420+0000 7efec0c00700 1 -- 172.21.15.151:0/1958469741 shutdown_connections 2024-05-21T22:05:22.426 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:22.420+0000 7efec0c00700 1 -- 172.21.15.151:0/1958469741 wait complete. 2024-05-21T22:05:23.425 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-21T22:05:23.864 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:05:23.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:05:23 smithi151 ceph-mon[31623]: pgmap v302: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 85 B/s wr, 12 op/s 2024-05-21T22:05:23.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:05:23 smithi184 ceph-mon[36942]: pgmap v302: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 85 B/s wr, 12 op/s 2024-05-21T22:05:24.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:05:23 smithi164 ceph-mon[36269]: pgmap v302: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 85 B/s wr, 12 op/s 2024-05-21T22:05:25.929 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:05:25 smithi151 ceph-mon[31623]: pgmap v303: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:05:25.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.927+0000 7f3622600700 1 -- 172.21.15.151:0/4163747422 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f361c10eeb0 msgr2=0x7f361c116a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:25.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.927+0000 7f3622600700 1 --2- 172.21.15.151:0/4163747422 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f361c10eeb0 0x7f361c116a80 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f361400b0d0 tx=0x7f361400b3e0 comp rx=0 tx=0).stop 2024-05-21T22:05:25.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.927+0000 7f3622600700 1 -- 172.21.15.151:0/4163747422 shutdown_connections 2024-05-21T22:05:25.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.927+0000 7f3622600700 1 --2- 172.21.15.151:0/4163747422 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f361c10eeb0 0x7f361c116a80 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:25.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.927+0000 7f3622600700 1 --2- 172.21.15.151:0/4163747422 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f361c109dd0 0x7f361c10a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:25.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.927+0000 7f3622600700 1 --2- 172.21.15.151:0/4163747422 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f361c10a780 0x7f361c10e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:25.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.927+0000 7f3622600700 1 -- 172.21.15.151:0/4163747422 >> 172.21.15.151:0/4163747422 conn(0x7f361c076d10 msgr2=0x7f361c108360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:05:25.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.927+0000 7f3622600700 1 -- 172.21.15.151:0/4163747422 shutdown_connections 2024-05-21T22:05:25.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.927+0000 7f3622600700 1 -- 172.21.15.151:0/4163747422 wait complete. 2024-05-21T22:05:25.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.928+0000 7f3622600700 1 Processor -- start 2024-05-21T22:05:25.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.929+0000 7f3622600700 1 -- start start 2024-05-21T22:05:25.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.929+0000 7f3622600700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f361c109dd0 0x7f361c1a0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:25.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.929+0000 7f3622600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f361c10a780 0x7f361c1a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:25.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.929+0000 7f3622600700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f361c10eeb0 0x7f361c1a4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:25.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.929+0000 7f3622600700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f361c119d40 con 0x7f361c10a780 2024-05-21T22:05:25.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.929+0000 7f3622600700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f361c119bc0 con 0x7f361c10eeb0 2024-05-21T22:05:25.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.929+0000 7f3622600700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f361c119ec0 con 0x7f361c109dd0 2024-05-21T22:05:25.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.929+0000 7f361b400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f361c109dd0 0x7f361c1a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:25.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.929+0000 7f361b400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f361c109dd0 0x7f361c1a0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:53368/0 (socket says 172.21.15.151:53368) 2024-05-21T22:05:25.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.929+0000 7f361b400700 1 -- 172.21.15.151:0/535504747 learned_addr learned my addr 172.21.15.151:0/535504747 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:05:25.932 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.929+0000 7f361be00700 1 --2- 172.21.15.151:0/535504747 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f361c10eeb0 0x7f361c1a4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:25.932 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.929+0000 7f3613e00700 1 --2- 172.21.15.151:0/535504747 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f361c10a780 0x7f361c1a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:25.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.930+0000 7f361be00700 1 -- 172.21.15.151:0/535504747 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f361c109dd0 msgr2=0x7f361c1a0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:25.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.930+0000 7f361be00700 1 --2- 172.21.15.151:0/535504747 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f361c109dd0 0x7f361c1a0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:25.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.930+0000 7f361be00700 1 -- 172.21.15.151:0/535504747 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f361c10a780 msgr2=0x7f361c1a06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:25.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.930+0000 7f361be00700 1 --2- 172.21.15.151:0/535504747 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f361c10a780 0x7f361c1a06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:25.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.930+0000 7f361be00700 1 -- 172.21.15.151:0/535504747 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f361c1a51a0 con 0x7f361c10eeb0 2024-05-21T22:05:25.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.930+0000 7f361b400700 1 --2- 172.21.15.151:0/535504747 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f361c109dd0 0x7f361c1a0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:05:25.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.930+0000 7f361be00700 1 --2- 172.21.15.151:0/535504747 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f361c10eeb0 0x7f361c1a4a60 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f361400b0d0 tx=0x7f3614007a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:05:25.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.930+0000 7f3618c00700 1 -- 172.21.15.151:0/535504747 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f361401f070 con 0x7f361c10eeb0 2024-05-21T22:05:25.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.931+0000 7f3622600700 1 -- 172.21.15.151:0/535504747 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f361c1a5430 con 0x7f361c10eeb0 2024-05-21T22:05:25.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.931+0000 7f3618c00700 1 -- 172.21.15.151:0/535504747 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3614004c50 con 0x7f361c10eeb0 2024-05-21T22:05:25.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.931+0000 7f3618c00700 1 -- 172.21.15.151:0/535504747 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f361401b430 con 0x7f361c10eeb0 2024-05-21T22:05:25.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.931+0000 7f3622600700 1 -- 172.21.15.151:0/535504747 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f361c1acd10 con 0x7f361c10eeb0 2024-05-21T22:05:25.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.937+0000 7f3618c00700 1 -- 172.21.15.151:0/535504747 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7f3614022070 con 0x7f361c10eeb0 2024-05-21T22:05:25.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.937+0000 7f362d051b80 1 Processor -- start 2024-05-21T22:05:25.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.937+0000 7f362d051b80 1 -- start start 2024-05-21T22:05:25.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.937+0000 7f362d051b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56228b768610 0x56228b7689f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:25.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.937+0000 7f362d051b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56228b768f30 0x56228b762b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:25.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.937+0000 7f362d051b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56228b763140 0x56228b7635c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:25.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.937+0000 7f362d051b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56228b6c0e40 con 0x56228b763140 2024-05-21T22:05:25.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.937+0000 7f362d051b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56228b6b09e0 con 0x56228b768610 2024-05-21T22:05:25.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.937+0000 7f362d051b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56228b6b4a20 con 0x56228b768f30 2024-05-21T22:05:25.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.938+0000 7f3618c00700 1 --2- 172.21.15.151:0/535504747 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f36000b29f0 0x7f36000b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:25.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.938+0000 7f3613e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56228b763140 0x56228b7635c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:25.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.938+0000 7f3613e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56228b763140 0x56228b7635c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52452/0 (socket says 172.21.15.151:52452) 2024-05-21T22:05:25.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.938+0000 7f3613e00700 1 -- 172.21.15.151:0/1086951118 learned_addr learned my addr 172.21.15.151:0/1086951118 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:05:25.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.938+0000 7f3613e00700 1 --2- 172.21.15.151:0/1086951118 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56228b768610 0x56228b7689f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:25.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.938+0000 7f361b400700 1 --2- 172.21.15.151:0/1086951118 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56228b768f30 0x56228b762b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:25.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.938+0000 7f3618c00700 1 -- 172.21.15.151:0/535504747 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7f36140e30c0 con 0x7f361c10eeb0 2024-05-21T22:05:25.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.938+0000 7f361be00700 1 --2- 172.21.15.151:0/535504747 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f36000b29f0 0x7f36000b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:25.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.938+0000 7f3613e00700 1 -- 172.21.15.151:0/1086951118 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56228b768f30 msgr2=0x56228b762b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:25.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.938+0000 7f3613e00700 1 --2- 172.21.15.151:0/1086951118 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56228b768f30 0x56228b762b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:25.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.938+0000 7f3613e00700 1 -- 172.21.15.151:0/1086951118 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56228b768610 msgr2=0x56228b7689f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:25.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.938+0000 7f3613e00700 1 --2- 172.21.15.151:0/1086951118 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56228b768610 0x56228b7689f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:25.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.938+0000 7f3613e00700 1 -- 172.21.15.151:0/1086951118 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56228b679420 con 0x56228b763140 2024-05-21T22:05:25.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.939+0000 7f3613e00700 1 --2- 172.21.15.151:0/1086951118 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56228b763140 0x56228b7635c0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f3604005950 tx=0x7f36040058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e2e63e3b6885595 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:05:25.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.939+0000 7f361be00700 1 --2- 172.21.15.151:0/535504747 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f36000b29f0 0x7f36000b4eb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f36140e08b0 tx=0x7f361400b3e0 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:05:25.942 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.939+0000 7f3610c00700 1 -- 172.21.15.151:0/1086951118 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3604013e20 con 0x56228b763140 2024-05-21T22:05:25.942 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.939+0000 7f3610c00700 1 -- 172.21.15.151:0/1086951118 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36040108d0 con 0x56228b763140 2024-05-21T22:05:25.942 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.939+0000 7f3610c00700 1 -- 172.21.15.151:0/1086951118 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f360400f700 con 0x56228b763140 2024-05-21T22:05:25.942 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.940+0000 7f362d051b80 1 -- 172.21.15.151:0/1086951118 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56228b763140 msgr2=0x56228b7635c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:25.942 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.940+0000 7f362d051b80 1 --2- 172.21.15.151:0/1086951118 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56228b763140 0x56228b7635c0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f3604005950 tx=0x7f36040058e0 comp rx=0 tx=0).stop 2024-05-21T22:05:25.942 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.940+0000 7f362d051b80 1 -- 172.21.15.151:0/1086951118 shutdown_connections 2024-05-21T22:05:25.942 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.940+0000 7f362d051b80 1 --2- 172.21.15.151:0/1086951118 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56228b768610 0x56228b7689f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:25.942 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.940+0000 7f362d051b80 1 --2- 172.21.15.151:0/1086951118 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56228b763140 0x56228b7635c0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:25.942 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.940+0000 7f362d051b80 1 --2- 172.21.15.151:0/1086951118 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56228b768f30 0x56228b762b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:25.942 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.940+0000 7f362d051b80 1 -- 172.21.15.151:0/1086951118 >> 172.21.15.151:0/1086951118 conn(0x56228b690be0 msgr2=0x56228b75d690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:05:25.942 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.940+0000 7f362d051b80 1 -- 172.21.15.151:0/1086951118 shutdown_connections 2024-05-21T22:05:25.943 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.940+0000 7f362d051b80 1 -- 172.21.15.151:0/1086951118 wait complete. 2024-05-21T22:05:25.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.941+0000 7f362d051b80 1 -- start start 2024-05-21T22:05:25.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.941+0000 7f362d051b80 1 Processor -- start 2024-05-21T22:05:25.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.941+0000 7f362d051b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56228b763140 0x56228b76ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:25.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.941+0000 7f362d051b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56228b768610 0x56228b76bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:25.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.942+0000 7f362d051b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56228b768f30 0x56228b76c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:25.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.942+0000 7f362d051b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56228b5b6fa0 con 0x56228b763140 2024-05-21T22:05:25.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.942+0000 7f362d051b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x56228b651760 con 0x56228b768610 2024-05-21T22:05:25.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.942+0000 7f362d051b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x56228b76f9e0 con 0x56228b768f30 2024-05-21T22:05:25.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.942+0000 7f362d051b80 10 client.0 _ll_register_callbacks cb 0x56228b65ac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:05:25.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.942+0000 7f3613e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56228b763140 0x56228b76ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:25.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.942+0000 7f361b400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56228b768610 0x56228b76bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:25.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.942+0000 7f3613e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56228b763140 0x56228b76ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52462/0 (socket says 172.21.15.151:52462) 2024-05-21T22:05:25.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.942+0000 7f3613e00700 1 -- 172.21.15.151:0/1590484491 learned_addr learned my addr 172.21.15.151:0/1590484491 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:05:25.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.942+0000 7f3613e00700 1 --2- 172.21.15.151:0/1590484491 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56228b768f30 0x56228b76c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:25.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.942+0000 7f3613e00700 1 -- 172.21.15.151:0/1590484491 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56228b768f30 msgr2=0x56228b76c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:25.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.942+0000 7f3613e00700 1 --2- 172.21.15.151:0/1590484491 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56228b768f30 0x56228b76c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:25.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.942+0000 7f3613e00700 1 -- 172.21.15.151:0/1590484491 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56228b768610 msgr2=0x56228b76bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:25.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.942+0000 7f3613e00700 1 --2- 172.21.15.151:0/1590484491 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56228b768610 0x56228b76bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:25.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.942+0000 7f3613e00700 1 -- 172.21.15.151:0/1590484491 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56228b6008d0 con 0x56228b763140 2024-05-21T22:05:25.946 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.942+0000 7f361b400700 1 --2- 172.21.15.151:0/1590484491 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56228b768610 0x56228b76bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:05:25.946 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.943+0000 7f3613e00700 1 --2- 172.21.15.151:0/1590484491 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56228b763140 0x56228b76ec40 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f360400f400 tx=0x7f360400f500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e583cc3ac9ba680 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:05:25.946 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.943+0000 7f3609600700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-21T22:05:25.946 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.943+0000 7f3609600700 1 -- 172.21.15.151:0/1590484491 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36040050d0 con 0x56228b763140 2024-05-21T22:05:25.946 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.943+0000 7f3609600700 1 -- 172.21.15.151:0/1590484491 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7f3604005270 con 0x56228b763140 2024-05-21T22:05:25.946 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.943+0000 7f362d051b80 1 -- 172.21.15.151:0/1590484491 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36040108d0 con 0x56228b763140 2024-05-21T22:05:25.949 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.943+0000 7f3609600700 1 -- 172.21.15.151:0/1590484491 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3604005470 con 0x56228b763140 2024-05-21T22:05:25.949 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.943+0000 7f362d051b80 10 client.17361 fetch_fsmap learned FSMap version 29 2024-05-21T22:05:25.949 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.943+0000 7f362d051b80 1 -- 172.21.15.151:0/1590484491 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56228b681060 con 0x56228b763140 2024-05-21T22:05:25.949 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.944+0000 7f3609600700 1 -- 172.21.15.151:0/1590484491 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36040108d0 con 0x56228b763140 2024-05-21T22:05:25.949 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.944+0000 7f3609600700 1 -- 172.21.15.151:0/1590484491 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f360400f5a0 con 0x56228b763140 2024-05-21T22:05:25.949 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.944+0000 7f362d051b80 10 client.17361 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:05:25.949 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.944+0000 7f362d051b80 10 client.17361 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-05-21T22:05:25.949 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.944+0000 7f362d051b80 20 client.17361 populate_metadata read hostname 'smithi151' 2024-05-21T22:05:25.949 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.944+0000 7f362d051b80 1 --2- 172.21.15.151:0/1590484491 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x56228b77c3d0 0x56228b77e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:25.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.944+0000 7f362d051b80 4 client.17361 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T22:05:25.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.944+0000 7f362d051b80 1 -- 172.21.15.151:0/1590484491 --> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56228b6c2d90 con 0x56228b77c3d0 2024-05-21T22:05:25.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.944+0000 7f361b400700 1 --2- 172.21.15.151:0/1590484491 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x56228b77c3d0 0x56228b77e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:25.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.945+0000 7f361b400700 1 --2- 172.21.15.151:0/1590484491 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x56228b77c3d0 0x56228b77e7c0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=74e7b4f0dc9360d3 server_cookie=860d2997fc1922fc in_seq=0 out_seq=0 2024-05-21T22:05:25.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.945+0000 7f3609600700 10 client.17361 ms_handle_connect on v2:172.21.15.164:6836/1554609572 2024-05-21T22:05:25.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.946+0000 7f3609600700 1 -- 172.21.15.151:0/1590484491 <== mds.2 v2:172.21.15.164:6836/1554609572 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x56228b6c2d90 con 0x56228b77c3d0 2024-05-21T22:05:25.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.946+0000 7f3609600700 10 client.17361 handle_command_reply: tid=0 2024-05-21T22:05:25.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.946+0000 7f362d051b80 1 client.17361 shutdown 2024-05-21T22:05:25.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.946+0000 7f362d051b80 1 -- 172.21.15.151:0/1590484491 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56228b763140 msgr2=0x56228b76ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:25.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.946+0000 7f362d051b80 1 --2- 172.21.15.151:0/1590484491 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56228b763140 0x56228b76ec40 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f360400f400 tx=0x7f360400f500 comp rx=0 tx=0).stop 2024-05-21T22:05:25.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.946+0000 7f362d051b80 1 -- 172.21.15.151:0/1590484491 shutdown_connections 2024-05-21T22:05:25.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.946+0000 7f362d051b80 1 --2- 172.21.15.151:0/1590484491 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x56228b77c3d0 0x56228b77e7c0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:25.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.946+0000 7f362d051b80 1 --2- 172.21.15.151:0/1590484491 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56228b768f30 0x56228b76c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:25.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.946+0000 7f362d051b80 1 --2- 172.21.15.151:0/1590484491 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56228b763140 0x56228b76ec40 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:25.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.946+0000 7f362d051b80 1 --2- 172.21.15.151:0/1590484491 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56228b768610 0x56228b76bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:25.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.946+0000 7f3609600700 0 client.17361 ms_handle_reset on v2:172.21.15.164:6836/1554609572 2024-05-21T22:05:25.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.946+0000 7f362d051b80 1 -- 172.21.15.151:0/1590484491 >> 172.21.15.151:0/1590484491 conn(0x56228b690be0 msgr2=0x56228b75cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:05:25.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.946+0000 7f362d051b80 1 -- 172.21.15.151:0/1590484491 shutdown_connections 2024-05-21T22:05:25.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.946+0000 7f362d051b80 1 -- 172.21.15.151:0/1590484491 wait complete. 2024-05-21T22:05:25.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.947+0000 7f362d051b80 20 client.17361 trim_cache size 0 max 16384 2024-05-21T22:05:25.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.963+0000 7f362d051b80 1 Processor -- start 2024-05-21T22:05:25.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.963+0000 7f362d051b80 1 -- start start 2024-05-21T22:05:25.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.963+0000 7f362d051b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56228b763140 0x56228b762b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:25.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.963+0000 7f362d051b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56228b768610 0x56228b769a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:25.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.963+0000 7f362d051b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56228b768f30 0x56228b75d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:25.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.963+0000 7f362d051b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f361c1b4350 con 0x56228b768f30 2024-05-21T22:05:25.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.963+0000 7f362d051b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f361c1b5e90 con 0x56228b768610 2024-05-21T22:05:25.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.963+0000 7f362d051b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f361c1b4a20 con 0x56228b763140 2024-05-21T22:05:25.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.964+0000 7f361b400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56228b768610 0x56228b769a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:25.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.964+0000 7f3613e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56228b763140 0x56228b762b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:25.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.964+0000 7f361b400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56228b768610 0x56228b769a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:56674/0 (socket says 172.21.15.151:56674) 2024-05-21T22:05:25.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.964+0000 7f361b400700 1 -- 172.21.15.151:0/2006133252 learned_addr learned my addr 172.21.15.151:0/2006133252 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:05:25.966 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.964+0000 7f3613e00700 1 --2- 172.21.15.151:0/2006133252 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56228b768f30 0x56228b75d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:25.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.964+0000 7f361b400700 1 -- 172.21.15.151:0/2006133252 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56228b763140 msgr2=0x56228b762b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:25.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.964+0000 7f361b400700 1 --2- 172.21.15.151:0/2006133252 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56228b763140 0x56228b762b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:25.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.964+0000 7f361b400700 1 -- 172.21.15.151:0/2006133252 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56228b768f30 msgr2=0x56228b75d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:25.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.964+0000 7f361b400700 1 --2- 172.21.15.151:0/2006133252 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56228b768f30 0x56228b75d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:25.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.964+0000 7f361b400700 1 -- 172.21.15.151:0/2006133252 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56228b76a550 con 0x56228b768610 2024-05-21T22:05:25.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.964+0000 7f361b400700 1 --2- 172.21.15.151:0/2006133252 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56228b768610 0x56228b769a20 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x56228b77ee50 tx=0x7f360c009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e6aab7c6097aeef server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:05:25.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.964+0000 7f360be00700 1 -- 172.21.15.151:0/2006133252 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f360c004020 con 0x56228b768610 2024-05-21T22:05:25.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.965+0000 7f360be00700 1 -- 172.21.15.151:0/2006133252 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f360c00f980 con 0x56228b768610 2024-05-21T22:05:25.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.965+0000 7f360be00700 1 -- 172.21.15.151:0/2006133252 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f360c003c10 con 0x56228b768610 2024-05-21T22:05:25.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.965+0000 7f362d051b80 1 -- 172.21.15.151:0/2006133252 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56228b768610 msgr2=0x56228b769a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:25.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.965+0000 7f362d051b80 1 --2- 172.21.15.151:0/2006133252 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56228b768610 0x56228b769a20 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x56228b77ee50 tx=0x7f360c009260 comp rx=0 tx=0).stop 2024-05-21T22:05:25.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.965+0000 7f362d051b80 1 -- 172.21.15.151:0/2006133252 shutdown_connections 2024-05-21T22:05:25.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.965+0000 7f362d051b80 1 --2- 172.21.15.151:0/2006133252 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56228b768f30 0x56228b75d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:25.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.965+0000 7f362d051b80 1 --2- 172.21.15.151:0/2006133252 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56228b763140 0x56228b762b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:25.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.965+0000 7f362d051b80 1 --2- 172.21.15.151:0/2006133252 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56228b768610 0x56228b769a20 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:25.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.965+0000 7f362d051b80 1 -- 172.21.15.151:0/2006133252 >> 172.21.15.151:0/2006133252 conn(0x56228b690be0 msgr2=0x56228b769310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:05:25.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.965+0000 7f362d051b80 1 -- 172.21.15.151:0/2006133252 shutdown_connections 2024-05-21T22:05:25.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.965+0000 7f362d051b80 1 -- 172.21.15.151:0/2006133252 wait complete. 2024-05-21T22:05:25.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.966+0000 7f362d051b80 1 -- start start 2024-05-21T22:05:25.971 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.966+0000 7f362d051b80 1 Processor -- start 2024-05-21T22:05:25.971 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.967+0000 7f362d051b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56228b763140 0x56228b774ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:25.971 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.967+0000 7f362d051b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56228b768610 0x56228b775420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:25.971 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.967+0000 7f362d051b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56228b768f30 0x56228b7834a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:25.971 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.967+0000 7f362d051b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f361c1b57c0 con 0x56228b763140 2024-05-21T22:05:25.971 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.967+0000 7f362d051b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f361c1b50f0 con 0x56228b768f30 2024-05-21T22:05:25.971 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.967+0000 7f362d051b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f361c1b2310 con 0x56228b768610 2024-05-21T22:05:25.972 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.967+0000 7f362d051b80 10 client.0 _ll_register_callbacks cb 0x56228b65ac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:05:25.972 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.967+0000 7f361b400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56228b768610 0x56228b775420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:25.972 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.967+0000 7f361b400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56228b768610 0x56228b775420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:53408/0 (socket says 172.21.15.151:53408) 2024-05-21T22:05:25.972 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.967+0000 7f361b400700 1 -- 172.21.15.151:0/1037656860 learned_addr learned my addr 172.21.15.151:0/1037656860 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:05:25.972 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.967+0000 7f3613e00700 1 --2- 172.21.15.151:0/1037656860 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56228b763140 0x56228b774ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:25.972 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.967+0000 7f3613e00700 1 --2- 172.21.15.151:0/1037656860 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56228b768f30 0x56228b7834a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:25.972 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.967+0000 7f361b400700 1 -- 172.21.15.151:0/1037656860 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56228b768f30 msgr2=0x56228b7834a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:25.972 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.967+0000 7f361b400700 1 --2- 172.21.15.151:0/1037656860 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56228b768f30 0x56228b7834a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:25.972 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.967+0000 7f361b400700 1 -- 172.21.15.151:0/1037656860 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56228b763140 msgr2=0x56228b774ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:25.972 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.967+0000 7f361b400700 1 --2- 172.21.15.151:0/1037656860 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56228b763140 0x56228b774ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:25.972 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.967+0000 7f361b400700 1 -- 172.21.15.151:0/1037656860 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56228b783ce0 con 0x56228b768610 2024-05-21T22:05:25.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.967+0000 7f3613e00700 1 --2- 172.21.15.151:0/1037656860 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56228b768f30 0x56228b7834a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T22:05:25.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.968+0000 7f361b400700 1 --2- 172.21.15.151:0/1037656860 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56228b768610 0x56228b775420 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f360c012a30 tx=0x7f360c0121a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=490317b6b0088fbe server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:05:25.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.968+0000 7f3609600700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-21T22:05:25.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.968+0000 7f3609600700 1 -- 172.21.15.151:0/1037656860 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f360c00fd30 con 0x56228b768610 2024-05-21T22:05:25.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.968+0000 7f3609600700 1 -- 172.21.15.151:0/1037656860 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7f360c004030 con 0x56228b768610 2024-05-21T22:05:25.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.968+0000 7f362d051b80 1 -- 172.21.15.151:0/1037656860 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f360c00f980 con 0x56228b768610 2024-05-21T22:05:25.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.968+0000 7f3609600700 1 -- 172.21.15.151:0/1037656860 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f360c004230 con 0x56228b768610 2024-05-21T22:05:25.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.968+0000 7f362d051b80 10 client.27499 fetch_fsmap learned FSMap version 29 2024-05-21T22:05:25.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.968+0000 7f362d051b80 1 -- 172.21.15.151:0/1037656860 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56228b7e68e0 con 0x56228b768610 2024-05-21T22:05:25.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.969+0000 7f3609600700 1 -- 172.21.15.151:0/1037656860 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f360c00f980 con 0x56228b768610 2024-05-21T22:05:25.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.969+0000 7f3609600700 1 -- 172.21.15.151:0/1037656860 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f360c033480 con 0x56228b768610 2024-05-21T22:05:25.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.969+0000 7f362d051b80 10 client.27499 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:05:25.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.969+0000 7f362d051b80 10 client.27499 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-05-21T22:05:25.974 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.969+0000 7f362d051b80 20 client.27499 populate_metadata read hostname 'smithi151' 2024-05-21T22:05:25.974 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.969+0000 7f362d051b80 1 --2- 172.21.15.151:0/1037656860 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x56228b7e6f70 0x56228b7e9390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:25.974 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.969+0000 7f362d051b80 4 client.27499 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T22:05:25.974 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.969+0000 7f362d051b80 1 -- 172.21.15.151:0/1037656860 --> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56228b6bc570 con 0x56228b7e6f70 2024-05-21T22:05:25.974 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.969+0000 7f3613e00700 1 --2- 172.21.15.151:0/1037656860 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x56228b7e6f70 0x56228b7e9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:25.974 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.970+0000 7f3613e00700 1 --2- 172.21.15.151:0/1037656860 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x56228b7e6f70 0x56228b7e9390 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=107d06e8bead8ceb server_cookie=23874c428122ba77 in_seq=0 out_seq=0 2024-05-21T22:05:25.974 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.970+0000 7f3609600700 10 client.27499 ms_handle_connect on v2:172.21.15.164:6836/1554609572 2024-05-21T22:05:25.975 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.970+0000 7f3609600700 1 -- 172.21.15.151:0/1037656860 <== mds.2 v2:172.21.15.164:6836/1554609572 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56228b6bc570 con 0x56228b7e6f70 2024-05-21T22:05:25.975 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.970+0000 7f3609600700 10 client.27499 handle_command_reply: tid=0 2024-05-21T22:05:25.975 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.970+0000 7f362d051b80 1 client.27499 shutdown 2024-05-21T22:05:25.975 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.970+0000 7f362d051b80 1 -- 172.21.15.151:0/1037656860 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56228b768610 msgr2=0x56228b775420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:25.976 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.970+0000 7f362d051b80 1 --2- 172.21.15.151:0/1037656860 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56228b768610 0x56228b775420 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f360c012a30 tx=0x7f360c0121a0 comp rx=0 tx=0).stop 2024-05-21T22:05:25.976 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.970+0000 7f362d051b80 1 -- 172.21.15.151:0/1037656860 shutdown_connections 2024-05-21T22:05:25.976 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.970+0000 7f362d051b80 1 --2- 172.21.15.151:0/1037656860 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x56228b7e6f70 0x56228b7e9390 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:25.976 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.970+0000 7f362d051b80 1 --2- 172.21.15.151:0/1037656860 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x56228b768f30 0x56228b7834a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:25.976 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.970+0000 7f362d051b80 1 --2- 172.21.15.151:0/1037656860 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56228b763140 0x56228b774ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:25.976 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.970+0000 7f362d051b80 1 --2- 172.21.15.151:0/1037656860 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56228b768610 0x56228b775420 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:25.976 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.970+0000 7f3609600700 0 client.27499 ms_handle_reset on v2:172.21.15.164:6836/1554609572 2024-05-21T22:05:25.976 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.970+0000 7f362d051b80 1 -- 172.21.15.151:0/1037656860 >> 172.21.15.151:0/1037656860 conn(0x56228b690be0 msgr2=0x56228b76a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:05:25.976 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.971+0000 7f362d051b80 1 -- 172.21.15.151:0/1037656860 shutdown_connections 2024-05-21T22:05:25.976 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.971+0000 7f362d051b80 1 -- 172.21.15.151:0/1037656860 wait complete. 2024-05-21T22:05:25.976 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.971+0000 7f362d051b80 20 client.27499 trim_cache size 0 max 16384 2024-05-21T22:05:25.977 INFO:teuthology.orchestra.run.smithi151.stdout:[] 2024-05-21T22:05:25.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.971+0000 7f3622600700 1 -- 172.21.15.151:0/535504747 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f36000b29f0 msgr2=0x7f36000b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:25.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.971+0000 7f3622600700 1 --2- 172.21.15.151:0/535504747 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f36000b29f0 0x7f36000b4eb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f36140e08b0 tx=0x7f361400b3e0 comp rx=0 tx=0).stop 2024-05-21T22:05:25.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.971+0000 7f3622600700 1 -- 172.21.15.151:0/535504747 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f361c10eeb0 msgr2=0x7f361c1a4a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:25.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.971+0000 7f3622600700 1 --2- 172.21.15.151:0/535504747 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f361c10eeb0 0x7f361c1a4a60 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f361400b0d0 tx=0x7f3614007a00 comp rx=0 tx=0).stop 2024-05-21T22:05:25.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.972+0000 7f3622600700 1 -- 172.21.15.151:0/535504747 shutdown_connections 2024-05-21T22:05:25.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.972+0000 7f3622600700 1 --2- 172.21.15.151:0/535504747 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f36000b29f0 0x7f36000b4eb0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:25.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.972+0000 7f3622600700 1 --2- 172.21.15.151:0/535504747 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f361c10eeb0 0x7f361c1a4a60 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:25.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.972+0000 7f3622600700 1 --2- 172.21.15.151:0/535504747 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f361c109dd0 0x7f361c1a0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:25.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.972+0000 7f3622600700 1 --2- 172.21.15.151:0/535504747 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f361c10a780 0x7f361c1a06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:25.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.972+0000 7f3622600700 1 -- 172.21.15.151:0/535504747 >> 172.21.15.151:0/535504747 conn(0x7f361c076d10 msgr2=0x7f361c077230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:05:25.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.972+0000 7f3622600700 1 -- 172.21.15.151:0/535504747 shutdown_connections 2024-05-21T22:05:25.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:25.972+0000 7f3622600700 1 -- 172.21.15.151:0/535504747 wait complete. 2024-05-21T22:05:25.985 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:05:25 smithi184 ceph-mon[36942]: pgmap v303: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:05:26.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:05:25 smithi164 ceph-mon[36269]: pgmap v303: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:05:26.902 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-21T22:05:27.369 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:05:27.922 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:05:27 smithi151 ceph-mon[31623]: pgmap v304: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:05:27.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:05:27 smithi184 ceph-mon[36942]: pgmap v304: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:05:28.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:05:27 smithi164 ceph-mon[36269]: pgmap v304: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:05:29.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.554+0000 7fce86200700 1 -- 172.21.15.151:0/2419571482 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce801035f0 msgr2=0x7fce80103a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:29.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.554+0000 7fce86200700 1 --2- 172.21.15.151:0/2419571482 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce801035f0 0x7fce80103a50 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fce6c009990 tx=0x7fce6c009ca0 comp rx=0 tx=0).stop 2024-05-21T22:05:29.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.554+0000 7fce86200700 1 -- 172.21.15.151:0/2419571482 shutdown_connections 2024-05-21T22:05:29.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.554+0000 7fce86200700 1 --2- 172.21.15.151:0/2419571482 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce80103f90 0x7fce80110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:29.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.554+0000 7fce86200700 1 --2- 172.21.15.151:0/2419571482 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce80108500 0x7fce801088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:29.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.554+0000 7fce86200700 1 --2- 172.21.15.151:0/2419571482 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce801035f0 0x7fce80103a50 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:29.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.554+0000 7fce86200700 1 -- 172.21.15.151:0/2419571482 >> 172.21.15.151:0/2419571482 conn(0x7fce8007a6a0 msgr2=0x7fce8007aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:05:29.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.554+0000 7fce86200700 1 -- 172.21.15.151:0/2419571482 shutdown_connections 2024-05-21T22:05:29.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.554+0000 7fce86200700 1 -- 172.21.15.151:0/2419571482 wait complete. 2024-05-21T22:05:29.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.555+0000 7fce86200700 1 Processor -- start 2024-05-21T22:05:29.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.555+0000 7fce86200700 1 -- start start 2024-05-21T22:05:29.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.556+0000 7fce86200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce801035f0 0x7fce8019be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:29.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.556+0000 7fce86200700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce80103f90 0x7fce8019c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:29.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.556+0000 7fce86200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce80108500 0x7fce801a0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:29.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.556+0000 7fce86200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fce80113790 con 0x7fce80108500 2024-05-21T22:05:29.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.556+0000 7fce86200700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fce80113610 con 0x7fce801035f0 2024-05-21T22:05:29.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.556+0000 7fce86200700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fce80113910 con 0x7fce80103f90 2024-05-21T22:05:29.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.556+0000 7fce7fe00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce80108500 0x7fce801a0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:29.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.556+0000 7fce7fe00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce80108500 0x7fce801a0710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52490/0 (socket says 172.21.15.151:52490) 2024-05-21T22:05:29.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.556+0000 7fce7fe00700 1 -- 172.21.15.151:0/3547749348 learned_addr learned my addr 172.21.15.151:0/3547749348 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:05:29.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.556+0000 7fce7ea00700 1 --2- 172.21.15.151:0/3547749348 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce80103f90 0x7fce8019c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:29.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.556+0000 7fce7f400700 1 --2- 172.21.15.151:0/3547749348 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce801035f0 0x7fce8019be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:29.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.557+0000 7fce7fe00700 1 -- 172.21.15.151:0/3547749348 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce80103f90 msgr2=0x7fce8019c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:29.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.557+0000 7fce7fe00700 1 --2- 172.21.15.151:0/3547749348 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce80103f90 0x7fce8019c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:29.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.557+0000 7fce7fe00700 1 -- 172.21.15.151:0/3547749348 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce801035f0 msgr2=0x7fce8019be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:29.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.557+0000 7fce7fe00700 1 --2- 172.21.15.151:0/3547749348 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce801035f0 0x7fce8019be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:29.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.557+0000 7fce7fe00700 1 -- 172.21.15.151:0/3547749348 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce801a0e50 con 0x7fce80108500 2024-05-21T22:05:29.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.557+0000 7fce7fe00700 1 --2- 172.21.15.151:0/3547749348 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce80108500 0x7fce801a0710 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fce7800efc0 tx=0x7fce7800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:05:29.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.557+0000 7fce77e00700 1 -- 172.21.15.151:0/3547749348 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce780092e0 con 0x7fce80108500 2024-05-21T22:05:29.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.557+0000 7fce77e00700 1 -- 172.21.15.151:0/3547749348 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce78004bb0 con 0x7fce80108500 2024-05-21T22:05:29.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.557+0000 7fce86200700 1 -- 172.21.15.151:0/3547749348 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce801a10e0 con 0x7fce80108500 2024-05-21T22:05:29.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.557+0000 7fce77e00700 1 -- 172.21.15.151:0/3547749348 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce78004020 con 0x7fce80108500 2024-05-21T22:05:29.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.557+0000 7fce86200700 1 -- 172.21.15.151:0/3547749348 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce801a89c0 con 0x7fce80108500 2024-05-21T22:05:29.561 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.561+0000 7fce77e00700 1 -- 172.21.15.151:0/3547749348 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7fce7801b030 con 0x7fce80108500 2024-05-21T22:05:29.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.562+0000 7fce90aabb80 1 Processor -- start 2024-05-21T22:05:29.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.563+0000 7fce90aabb80 1 -- start start 2024-05-21T22:05:29.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.563+0000 7fce77e00700 1 --2- 172.21.15.151:0/3547749348 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fce640b2a70 0x7fce640b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:29.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.563+0000 7fce90aabb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5646c4551610 0x5646c45519f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:29.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.563+0000 7fce90aabb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5646c4551f30 0x5646c454bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:29.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.563+0000 7fce77e00700 1 -- 172.21.15.151:0/3547749348 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7fce780e1fc0 con 0x7fce80108500 2024-05-21T22:05:29.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.563+0000 7fce90aabb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5646c454c140 0x5646c454c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:29.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.563+0000 7fce90aabb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5646c44a9e40 con 0x5646c4551f30 2024-05-21T22:05:29.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.563+0000 7fce90aabb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5646c44999e0 con 0x5646c4551610 2024-05-21T22:05:29.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.563+0000 7fce90aabb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5646c449da20 con 0x5646c454c140 2024-05-21T22:05:29.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.563+0000 7fce7ea00700 1 --2- 172.21.15.151:0/3547749348 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fce640b2a70 0x7fce640b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:29.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.563+0000 7fce7ea00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5646c4551f30 0x5646c454bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:29.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.563+0000 7fce7ea00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5646c4551f30 0x5646c454bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52502/0 (socket says 172.21.15.151:52502) 2024-05-21T22:05:29.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.563+0000 7fce7ea00700 1 -- 172.21.15.151:0/256801133 learned_addr learned my addr 172.21.15.151:0/256801133 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:05:29.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.563+0000 7fce7f400700 1 --2- 172.21.15.151:0/256801133 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5646c4551610 0x5646c45519f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:29.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.564+0000 7fce7fe00700 1 --2- 172.21.15.151:0/256801133 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5646c454c140 0x5646c454c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:29.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.564+0000 7fce7fe00700 1 -- 172.21.15.151:0/256801133 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5646c4551610 msgr2=0x5646c45519f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:29.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.564+0000 7fce7fe00700 1 --2- 172.21.15.151:0/256801133 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5646c4551610 0x5646c45519f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:29.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.564+0000 7fce7fe00700 1 -- 172.21.15.151:0/256801133 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5646c4551f30 msgr2=0x5646c454bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:29.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.564+0000 7fce7fe00700 1 --2- 172.21.15.151:0/256801133 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5646c4551f30 0x5646c454bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:29.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.564+0000 7fce7fe00700 1 -- 172.21.15.151:0/256801133 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646c4462420 con 0x5646c454c140 2024-05-21T22:05:29.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.564+0000 7fce7f400700 1 --2- 172.21.15.151:0/256801133 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5646c4551610 0x5646c45519f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:05:29.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.564+0000 7fce7fe00700 1 --2- 172.21.15.151:0/256801133 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5646c454c140 0x5646c454c5c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fce780df6a0 tx=0x7fce780a8e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aaf5ec0542df01c server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:05:29.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.564+0000 7fce7ea00700 1 --2- 172.21.15.151:0/3547749348 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fce640b2a70 0x7fce640b4f30 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fce6c000c00 tx=0x7fce6c009570 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:05:29.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.565+0000 7fce74c00700 1 -- 172.21.15.151:0/256801133 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce780ad5d0 con 0x5646c454c140 2024-05-21T22:05:29.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.565+0000 7fce74c00700 1 -- 172.21.15.151:0/256801133 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce780b1ff0 con 0x5646c454c140 2024-05-21T22:05:29.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.565+0000 7fce74c00700 1 -- 172.21.15.151:0/256801133 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce780ad3d0 con 0x5646c454c140 2024-05-21T22:05:29.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.565+0000 7fce90aabb80 1 -- 172.21.15.151:0/256801133 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5646c454c140 msgr2=0x5646c454c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:29.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.565+0000 7fce90aabb80 1 --2- 172.21.15.151:0/256801133 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5646c454c140 0x5646c454c5c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fce780df6a0 tx=0x7fce780a8e00 comp rx=0 tx=0).stop 2024-05-21T22:05:29.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.566+0000 7fce90aabb80 1 -- 172.21.15.151:0/256801133 shutdown_connections 2024-05-21T22:05:29.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.566+0000 7fce90aabb80 1 --2- 172.21.15.151:0/256801133 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5646c4551610 0x5646c45519f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:29.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.566+0000 7fce90aabb80 1 --2- 172.21.15.151:0/256801133 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5646c454c140 0x5646c454c5c0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:29.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.566+0000 7fce90aabb80 1 --2- 172.21.15.151:0/256801133 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5646c4551f30 0x5646c454bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:29.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.566+0000 7fce90aabb80 1 -- 172.21.15.151:0/256801133 >> 172.21.15.151:0/256801133 conn(0x5646c4479be0 msgr2=0x5646c4546690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:05:29.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.566+0000 7fce90aabb80 1 -- 172.21.15.151:0/256801133 shutdown_connections 2024-05-21T22:05:29.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.566+0000 7fce90aabb80 1 -- 172.21.15.151:0/256801133 wait complete. 2024-05-21T22:05:29.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.566+0000 7fce90aabb80 1 -- start start 2024-05-21T22:05:29.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.566+0000 7fce90aabb80 1 Processor -- start 2024-05-21T22:05:29.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.567+0000 7fce90aabb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5646c454c140 0x5646c4557cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:29.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.567+0000 7fce90aabb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5646c4551610 0x5646c4555090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:29.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.567+0000 7fce90aabb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5646c4551f30 0x5646c45555d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:29.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.567+0000 7fce90aabb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5646c439ffa0 con 0x5646c4551f30 2024-05-21T22:05:29.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.567+0000 7fce90aabb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5646c443a760 con 0x5646c4551610 2024-05-21T22:05:29.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.567+0000 7fce90aabb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5646c45589e0 con 0x5646c454c140 2024-05-21T22:05:29.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.567+0000 7fce90aabb80 10 client.0 _ll_register_callbacks cb 0x5646c4443c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:05:29.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.567+0000 7fce7fe00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5646c4551f30 0x5646c45555d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:29.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.567+0000 7fce7fe00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5646c4551f30 0x5646c45555d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52518/0 (socket says 172.21.15.151:52518) 2024-05-21T22:05:29.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.567+0000 7fce7fe00700 1 -- 172.21.15.151:0/2758359772 learned_addr learned my addr 172.21.15.151:0/2758359772 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:05:29.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.568+0000 7fce7f400700 1 --2- 172.21.15.151:0/2758359772 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5646c454c140 0x5646c4557cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:29.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.568+0000 7fce7ea00700 1 --2- 172.21.15.151:0/2758359772 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5646c4551610 0x5646c4555090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:29.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.568+0000 7fce7fe00700 1 -- 172.21.15.151:0/2758359772 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5646c454c140 msgr2=0x5646c4557cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:29.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.568+0000 7fce7fe00700 1 --2- 172.21.15.151:0/2758359772 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5646c454c140 0x5646c4557cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:29.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.568+0000 7fce7fe00700 1 -- 172.21.15.151:0/2758359772 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5646c4551610 msgr2=0x5646c4555090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:29.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.568+0000 7fce7fe00700 1 --2- 172.21.15.151:0/2758359772 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5646c4551610 0x5646c4555090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:29.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.568+0000 7fce7fe00700 1 -- 172.21.15.151:0/2758359772 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646c43e98d0 con 0x5646c4551f30 2024-05-21T22:05:29.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.568+0000 7fce7fe00700 1 --2- 172.21.15.151:0/2758359772 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5646c4551f30 0x5646c45555d0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fce780df700 tx=0x7fce78009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=788be2966d6446e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:05:29.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.568+0000 7fce69600700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-21T22:05:29.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.568+0000 7fce69600700 1 -- 172.21.15.151:0/2758359772 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce780c3060 con 0x5646c4551f30 2024-05-21T22:05:29.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.569+0000 7fce69600700 1 -- 172.21.15.151:0/2758359772 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7fce780a9680 con 0x5646c4551f30 2024-05-21T22:05:29.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.569+0000 7fce90aabb80 1 -- 172.21.15.151:0/2758359772 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce780b1ff0 con 0x5646c4551f30 2024-05-21T22:05:29.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.569+0000 7fce69600700 1 -- 172.21.15.151:0/2758359772 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fce780b1b60 con 0x5646c4551f30 2024-05-21T22:05:29.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.569+0000 7fce90aabb80 10 client.17379 fetch_fsmap learned FSMap version 29 2024-05-21T22:05:29.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.569+0000 7fce90aabb80 1 -- 172.21.15.151:0/2758359772 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5646c446a060 con 0x5646c4551f30 2024-05-21T22:05:29.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.569+0000 7fce69600700 1 -- 172.21.15.151:0/2758359772 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce780b1ff0 con 0x5646c4551f30 2024-05-21T22:05:29.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.569+0000 7fce69600700 1 -- 172.21.15.151:0/2758359772 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fce780a4d00 con 0x5646c4551f30 2024-05-21T22:05:29.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.569+0000 7fce90aabb80 10 client.17379 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:05:29.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.569+0000 7fce90aabb80 10 client.17379 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-21T22:05:29.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.570+0000 7fce90aabb80 20 client.17379 populate_metadata read hostname 'smithi151' 2024-05-21T22:05:29.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.570+0000 7fce90aabb80 1 --2- 172.21.15.151:0/2758359772 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x5646c4565430 0x5646c4567820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:29.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.570+0000 7fce90aabb80 4 client.17379 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T22:05:29.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.570+0000 7fce90aabb80 1 -- 172.21.15.151:0/2758359772 --> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5646c44abd90 con 0x5646c4565430 2024-05-21T22:05:29.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.570+0000 7fce7f400700 1 --2- 172.21.15.151:0/2758359772 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x5646c4565430 0x5646c4567820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:29.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.570+0000 7fce7f400700 1 --2- 172.21.15.151:0/2758359772 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x5646c4565430 0x5646c4567820 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6c9def0f36fd7efe server_cookie=b6e96cc552254a67 in_seq=0 out_seq=0 2024-05-21T22:05:29.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.570+0000 7fce69600700 10 client.17379 ms_handle_connect on v2:172.21.15.184:6838/3059935143 2024-05-21T22:05:29.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.571+0000 7fce69600700 1 -- 172.21.15.151:0/2758359772 <== mds.1 v2:172.21.15.184:6838/3059935143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5646c44abd90 con 0x5646c4565430 2024-05-21T22:05:29.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.571+0000 7fce69600700 10 client.17379 handle_command_reply: tid=0 2024-05-21T22:05:29.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.571+0000 7fce90aabb80 1 client.17379 shutdown 2024-05-21T22:05:29.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.572+0000 7fce90aabb80 1 -- 172.21.15.151:0/2758359772 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5646c4551f30 msgr2=0x5646c45555d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:29.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.572+0000 7fce90aabb80 1 --2- 172.21.15.151:0/2758359772 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5646c4551f30 0x5646c45555d0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fce780df700 tx=0x7fce78009da0 comp rx=0 tx=0).stop 2024-05-21T22:05:29.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.572+0000 7fce90aabb80 1 -- 172.21.15.151:0/2758359772 shutdown_connections 2024-05-21T22:05:29.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.572+0000 7fce90aabb80 1 --2- 172.21.15.151:0/2758359772 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x5646c4565430 0x5646c4567820 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:29.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.572+0000 7fce90aabb80 1 --2- 172.21.15.151:0/2758359772 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5646c4551f30 0x5646c45555d0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:29.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.572+0000 7fce90aabb80 1 --2- 172.21.15.151:0/2758359772 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5646c454c140 0x5646c4557cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:29.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.572+0000 7fce69600700 0 client.17379 ms_handle_reset on v2:172.21.15.184:6838/3059935143 2024-05-21T22:05:29.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.572+0000 7fce90aabb80 1 --2- 172.21.15.151:0/2758359772 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5646c4551610 0x5646c4555090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:29.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.572+0000 7fce90aabb80 1 -- 172.21.15.151:0/2758359772 >> 172.21.15.151:0/2758359772 conn(0x5646c4479be0 msgr2=0x5646c4545d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:05:29.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.572+0000 7fce90aabb80 1 -- 172.21.15.151:0/2758359772 shutdown_connections 2024-05-21T22:05:29.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.572+0000 7fce90aabb80 1 -- 172.21.15.151:0/2758359772 wait complete. 2024-05-21T22:05:29.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.572+0000 7fce90aabb80 20 client.17379 trim_cache size 0 max 16384 2024-05-21T22:05:29.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.588+0000 7fce90aabb80 1 Processor -- start 2024-05-21T22:05:29.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.588+0000 7fce90aabb80 1 -- start start 2024-05-21T22:05:29.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.589+0000 7fce90aabb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5646c454c140 0x5646c454bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:29.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.589+0000 7fce90aabb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5646c4551610 0x5646c45464f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:29.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.589+0000 7fce90aabb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5646c4551f30 0x5646c4553f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:29.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.589+0000 7fce90aabb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fce801b0140 con 0x5646c454c140 2024-05-21T22:05:29.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.589+0000 7fce90aabb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fce801b1c80 con 0x5646c4551610 2024-05-21T22:05:29.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.589+0000 7fce90aabb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fce801b0810 con 0x5646c4551f30 2024-05-21T22:05:29.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.589+0000 7fce7f400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5646c454c140 0x5646c454bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:29.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.589+0000 7fce7f400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5646c454c140 0x5646c454bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52522/0 (socket says 172.21.15.151:52522) 2024-05-21T22:05:29.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.589+0000 7fce7f400700 1 -- 172.21.15.151:0/4166590402 learned_addr learned my addr 172.21.15.151:0/4166590402 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:05:29.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.589+0000 7fce7ea00700 1 --2- 172.21.15.151:0/4166590402 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5646c4551610 0x5646c45464f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:29.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.589+0000 7fce7fe00700 1 --2- 172.21.15.151:0/4166590402 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5646c4551f30 0x5646c4553f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:29.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.590+0000 7fce7f400700 1 -- 172.21.15.151:0/4166590402 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5646c4551f30 msgr2=0x5646c4553f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:29.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.590+0000 7fce7f400700 1 --2- 172.21.15.151:0/4166590402 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5646c4551f30 0x5646c4553f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:29.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.590+0000 7fce7f400700 1 -- 172.21.15.151:0/4166590402 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5646c4551610 msgr2=0x5646c45464f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:29.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.590+0000 7fce7f400700 1 --2- 172.21.15.151:0/4166590402 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5646c4551610 0x5646c45464f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:29.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.590+0000 7fce7f400700 1 -- 172.21.15.151:0/4166590402 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646c4555f90 con 0x5646c454c140 2024-05-21T22:05:29.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.590+0000 7fce7ea00700 1 --2- 172.21.15.151:0/4166590402 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5646c4551610 0x5646c45464f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:05:29.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.590+0000 7fce7f400700 1 --2- 172.21.15.151:0/4166590402 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5646c454c140 0x5646c454bb20 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x5646c4558b30 tx=0x7fce70009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f52909fe5dfdecad server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:05:29.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.590+0000 7fce6be00700 1 -- 172.21.15.151:0/4166590402 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce70003c60 con 0x5646c454c140 2024-05-21T22:05:29.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.590+0000 7fce6be00700 1 -- 172.21.15.151:0/4166590402 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce70003e00 con 0x5646c454c140 2024-05-21T22:05:29.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.590+0000 7fce6be00700 1 -- 172.21.15.151:0/4166590402 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce70010830 con 0x5646c454c140 2024-05-21T22:05:29.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.591+0000 7fce90aabb80 1 -- 172.21.15.151:0/4166590402 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5646c454c140 msgr2=0x5646c454bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:29.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.591+0000 7fce90aabb80 1 --2- 172.21.15.151:0/4166590402 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5646c454c140 0x5646c454bb20 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x5646c4558b30 tx=0x7fce70009260 comp rx=0 tx=0).stop 2024-05-21T22:05:29.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.591+0000 7fce90aabb80 1 -- 172.21.15.151:0/4166590402 shutdown_connections 2024-05-21T22:05:29.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.591+0000 7fce90aabb80 1 --2- 172.21.15.151:0/4166590402 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5646c4551f30 0x5646c4553f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:29.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.591+0000 7fce90aabb80 1 --2- 172.21.15.151:0/4166590402 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5646c454c140 0x5646c454bb20 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:29.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.591+0000 7fce90aabb80 1 --2- 172.21.15.151:0/4166590402 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5646c4551610 0x5646c45464f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:29.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.591+0000 7fce90aabb80 1 -- 172.21.15.151:0/4166590402 >> 172.21.15.151:0/4166590402 conn(0x5646c4479be0 msgr2=0x5646c4553850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:05:29.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.591+0000 7fce90aabb80 1 -- 172.21.15.151:0/4166590402 shutdown_connections 2024-05-21T22:05:29.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.591+0000 7fce90aabb80 1 -- 172.21.15.151:0/4166590402 wait complete. 2024-05-21T22:05:29.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.592+0000 7fce90aabb80 1 -- start start 2024-05-21T22:05:29.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.592+0000 7fce90aabb80 1 Processor -- start 2024-05-21T22:05:29.595 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.592+0000 7fce90aabb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5646c454c140 0x5646c456b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:29.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.592+0000 7fce90aabb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5646c4551610 0x5646c45683c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:29.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.592+0000 7fce90aabb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5646c4551f30 0x5646c45689b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:29.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.592+0000 7fce90aabb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fce801b15b0 con 0x5646c4551f30 2024-05-21T22:05:29.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.592+0000 7fce90aabb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fce801b0ee0 con 0x5646c454c140 2024-05-21T22:05:29.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.592+0000 7fce90aabb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fce801ae100 con 0x5646c4551610 2024-05-21T22:05:29.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.593+0000 7fce90aabb80 10 client.0 _ll_register_callbacks cb 0x5646c4443c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:05:29.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.593+0000 7fce7f400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5646c454c140 0x5646c456b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:29.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.593+0000 7fce7f400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5646c454c140 0x5646c456b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:56718/0 (socket says 172.21.15.151:56718) 2024-05-21T22:05:29.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.593+0000 7fce7f400700 1 -- 172.21.15.151:0/1865531429 learned_addr learned my addr 172.21.15.151:0/1865531429 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:05:29.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.593+0000 7fce7ea00700 1 --2- 172.21.15.151:0/1865531429 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5646c4551610 0x5646c45683c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:29.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.593+0000 7fce7fe00700 1 --2- 172.21.15.151:0/1865531429 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5646c4551f30 0x5646c45689b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:29.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.593+0000 7fce7f400700 1 -- 172.21.15.151:0/1865531429 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5646c4551610 msgr2=0x5646c45683c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:29.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.593+0000 7fce7f400700 1 --2- 172.21.15.151:0/1865531429 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5646c4551610 0x5646c45683c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:29.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.593+0000 7fce7f400700 1 -- 172.21.15.151:0/1865531429 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5646c4551f30 msgr2=0x5646c45689b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:29.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.593+0000 7fce7f400700 1 --2- 172.21.15.151:0/1865531429 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5646c4551f30 0x5646c45689b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:29.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.593+0000 7fce7f400700 1 -- 172.21.15.151:0/1865531429 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646c45691f0 con 0x5646c454c140 2024-05-21T22:05:29.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.593+0000 7fce7fe00700 1 --2- 172.21.15.151:0/1865531429 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5646c4551f30 0x5646c45689b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T22:05:29.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.594+0000 7fce7f400700 1 --2- 172.21.15.151:0/1865531429 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5646c454c140 0x5646c456b020 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fce70000c00 tx=0x7fce700039e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c5d5643f62120ff server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:05:29.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.594+0000 7fce69600700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-21T22:05:29.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.594+0000 7fce69600700 1 -- 172.21.15.151:0/1865531429 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce70004a30 con 0x5646c454c140 2024-05-21T22:05:29.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.594+0000 7fce90aabb80 1 -- 172.21.15.151:0/1865531429 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce70003e00 con 0x5646c454c140 2024-05-21T22:05:29.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.594+0000 7fce69600700 1 -- 172.21.15.151:0/1865531429 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7fce70004bd0 con 0x5646c454c140 2024-05-21T22:05:29.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.594+0000 7fce69600700 1 -- 172.21.15.151:0/1865531429 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fce70004dd0 con 0x5646c454c140 2024-05-21T22:05:29.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.594+0000 7fce90aabb80 10 client.27512 fetch_fsmap learned FSMap version 29 2024-05-21T22:05:29.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.594+0000 7fce90aabb80 1 -- 172.21.15.151:0/1865531429 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5646c45cfda0 con 0x5646c454c140 2024-05-21T22:05:29.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.595+0000 7fce69600700 1 -- 172.21.15.151:0/1865531429 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce70003e00 con 0x5646c454c140 2024-05-21T22:05:29.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.595+0000 7fce69600700 1 -- 172.21.15.151:0/1865531429 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fce7001e840 con 0x5646c454c140 2024-05-21T22:05:29.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.595+0000 7fce90aabb80 10 client.27512 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:05:29.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.595+0000 7fce90aabb80 10 client.27512 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-21T22:05:29.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.595+0000 7fce90aabb80 20 client.27512 populate_metadata read hostname 'smithi151' 2024-05-21T22:05:29.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.595+0000 7fce90aabb80 1 --2- 172.21.15.151:0/1865531429 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x5646c45d0400 0x5646c45d2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:29.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.595+0000 7fce90aabb80 4 client.27512 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T22:05:29.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.595+0000 7fce90aabb80 1 -- 172.21.15.151:0/1865531429 --> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5646c44a5570 con 0x5646c45d0400 2024-05-21T22:05:29.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.595+0000 7fce7ea00700 1 --2- 172.21.15.151:0/1865531429 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x5646c45d0400 0x5646c45d2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:29.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.596+0000 7fce69600700 10 client.27512 ms_handle_connect on v2:172.21.15.184:6838/3059935143 2024-05-21T22:05:29.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.596+0000 7fce7ea00700 1 --2- 172.21.15.151:0/1865531429 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x5646c45d0400 0x5646c45d2820 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=17d3aca8b48abdf2 server_cookie=7951d632724586c9 in_seq=0 out_seq=0 2024-05-21T22:05:29.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.596+0000 7fce69600700 1 -- 172.21.15.151:0/1865531429 <== mds.1 v2:172.21.15.184:6838/3059935143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5646c44a5570 con 0x5646c45d0400 2024-05-21T22:05:29.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.596+0000 7fce69600700 10 client.27512 handle_command_reply: tid=0 2024-05-21T22:05:29.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.596+0000 7fce90aabb80 1 client.27512 shutdown 2024-05-21T22:05:29.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.596+0000 7fce90aabb80 1 -- 172.21.15.151:0/1865531429 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5646c454c140 msgr2=0x5646c456b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:29.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.596+0000 7fce90aabb80 1 --2- 172.21.15.151:0/1865531429 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5646c454c140 0x5646c456b020 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fce70000c00 tx=0x7fce700039e0 comp rx=0 tx=0).stop 2024-05-21T22:05:29.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.596+0000 7fce90aabb80 1 -- 172.21.15.151:0/1865531429 shutdown_connections 2024-05-21T22:05:29.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.596+0000 7fce90aabb80 1 --2- 172.21.15.151:0/1865531429 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x5646c45d0400 0x5646c45d2820 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:29.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.597+0000 7fce90aabb80 1 --2- 172.21.15.151:0/1865531429 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5646c454c140 0x5646c456b020 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:29.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.597+0000 7fce90aabb80 1 --2- 172.21.15.151:0/1865531429 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5646c4551f30 0x5646c45689b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:29.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.597+0000 7fce90aabb80 1 --2- 172.21.15.151:0/1865531429 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5646c4551610 0x5646c45683c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:29.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.597+0000 7fce69600700 0 client.27512 ms_handle_reset on v2:172.21.15.184:6838/3059935143 2024-05-21T22:05:29.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.597+0000 7fce90aabb80 1 -- 172.21.15.151:0/1865531429 >> 172.21.15.151:0/1865531429 conn(0x5646c4479be0 msgr2=0x5646c45540e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:05:29.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.597+0000 7fce90aabb80 1 -- 172.21.15.151:0/1865531429 shutdown_connections 2024-05-21T22:05:29.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.597+0000 7fce90aabb80 1 -- 172.21.15.151:0/1865531429 wait complete. 2024-05-21T22:05:29.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.597+0000 7fce90aabb80 20 client.27512 trim_cache size 0 max 16384 2024-05-21T22:05:29.601 INFO:teuthology.orchestra.run.smithi151.stdout:[] 2024-05-21T22:05:29.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.597+0000 7fce86200700 1 -- 172.21.15.151:0/3547749348 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fce640b2a70 msgr2=0x7fce640b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:29.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.597+0000 7fce86200700 1 --2- 172.21.15.151:0/3547749348 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fce640b2a70 0x7fce640b4f30 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fce6c000c00 tx=0x7fce6c009570 comp rx=0 tx=0).stop 2024-05-21T22:05:29.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.597+0000 7fce86200700 1 -- 172.21.15.151:0/3547749348 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce80108500 msgr2=0x7fce801a0710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:29.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.597+0000 7fce86200700 1 --2- 172.21.15.151:0/3547749348 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce80108500 0x7fce801a0710 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fce7800efc0 tx=0x7fce7800c3a0 comp rx=0 tx=0).stop 2024-05-21T22:05:29.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.598+0000 7fce86200700 1 -- 172.21.15.151:0/3547749348 shutdown_connections 2024-05-21T22:05:29.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.598+0000 7fce86200700 1 --2- 172.21.15.151:0/3547749348 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fce640b2a70 0x7fce640b4f30 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:29.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.598+0000 7fce86200700 1 --2- 172.21.15.151:0/3547749348 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce801035f0 0x7fce8019be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:29.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.598+0000 7fce86200700 1 --2- 172.21.15.151:0/3547749348 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce80108500 0x7fce801a0710 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:29.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.598+0000 7fce86200700 1 --2- 172.21.15.151:0/3547749348 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce80103f90 0x7fce8019c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:29.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.598+0000 7fce86200700 1 -- 172.21.15.151:0/3547749348 >> 172.21.15.151:0/3547749348 conn(0x7fce8007a6a0 msgr2=0x7fce80101460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:05:29.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.598+0000 7fce86200700 1 -- 172.21.15.151:0/3547749348 shutdown_connections 2024-05-21T22:05:29.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:29.598+0000 7fce86200700 1 -- 172.21.15.151:0/3547749348 wait complete. 2024-05-21T22:05:29.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:05:29 smithi151 ceph-mon[31623]: pgmap v305: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T22:05:29.985 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:05:29 smithi184 ceph-mon[36942]: pgmap v305: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T22:05:30.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:05:29 smithi164 ceph-mon[36269]: pgmap v305: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T22:05:30.678 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-21T22:05:31.128 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:05:31.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:05:31 smithi151 ceph-mon[31623]: pgmap v306: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:05:31.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:05:31 smithi184 ceph-mon[36942]: pgmap v306: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:05:32.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:05:31 smithi164 ceph-mon[36269]: pgmap v306: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:05:33.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.397+0000 7f215ea00700 1 -- 172.21.15.151:0/300386308 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2158109a80 msgr2=0x7f2158109f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:33.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.397+0000 7f215ea00700 1 --2- 172.21.15.151:0/300386308 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2158109a80 0x7f2158109f00 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f21480098f0 tx=0x7f2148009c00 comp rx=0 tx=0).stop 2024-05-21T22:05:33.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.398+0000 7f215ea00700 1 -- 172.21.15.151:0/300386308 shutdown_connections 2024-05-21T22:05:33.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.398+0000 7f215ea00700 1 --2- 172.21.15.151:0/300386308 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f215810a440 0x7f21581168e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:33.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.398+0000 7f215ea00700 1 --2- 172.21.15.151:0/300386308 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f215810e950 0x7f215810ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:33.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.398+0000 7f215ea00700 1 --2- 172.21.15.151:0/300386308 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2158109a80 0x7f2158109f00 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:33.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.398+0000 7f215ea00700 1 -- 172.21.15.151:0/300386308 >> 172.21.15.151:0/300386308 conn(0x7f21580792d0 msgr2=0x7f2158108360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:05:33.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.398+0000 7f215ea00700 1 -- 172.21.15.151:0/300386308 shutdown_connections 2024-05-21T22:05:33.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.398+0000 7f215ea00700 1 -- 172.21.15.151:0/300386308 wait complete. 2024-05-21T22:05:33.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.399+0000 7f215ea00700 1 Processor -- start 2024-05-21T22:05:33.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.399+0000 7f215ea00700 1 -- start start 2024-05-21T22:05:33.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.399+0000 7f215ea00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2158109a80 0x7f21581a01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:33.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.400+0000 7f215ea00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f215810a440 0x7f21581a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:33.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.400+0000 7f215ea00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f215810e950 0x7f21581a4a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:33.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.400+0000 7f215ea00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2158119c30 con 0x7f215810a440 2024-05-21T22:05:33.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.400+0000 7f215ea00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2158119ab0 con 0x7f215810e950 2024-05-21T22:05:33.401 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.400+0000 7f215ea00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f2158119db0 con 0x7f2158109a80 2024-05-21T22:05:33.401 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.400+0000 7f215cc00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f215810a440 0x7f21581a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:33.401 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.400+0000 7f215d600700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2158109a80 0x7f21581a01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:33.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.400+0000 7f215cc00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f215810a440 0x7f21581a06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52554/0 (socket says 172.21.15.151:52554) 2024-05-21T22:05:33.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.400+0000 7f215cc00700 1 -- 172.21.15.151:0/2170804605 learned_addr learned my addr 172.21.15.151:0/2170804605 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:05:33.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.400+0000 7f215e000700 1 --2- 172.21.15.151:0/2170804605 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f215810e950 0x7f21581a4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:33.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.400+0000 7f215cc00700 1 -- 172.21.15.151:0/2170804605 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2158109a80 msgr2=0x7f21581a01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:33.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.400+0000 7f215cc00700 1 --2- 172.21.15.151:0/2170804605 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2158109a80 0x7f21581a01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:33.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.400+0000 7f215cc00700 1 -- 172.21.15.151:0/2170804605 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f215810e950 msgr2=0x7f21581a4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:33.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.400+0000 7f215cc00700 1 --2- 172.21.15.151:0/2170804605 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f215810e950 0x7f21581a4a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:33.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.400+0000 7f215cc00700 1 -- 172.21.15.151:0/2170804605 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21581a51b0 con 0x7f215810a440 2024-05-21T22:05:33.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.401+0000 7f215cc00700 1 --2- 172.21.15.151:0/2170804605 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f215810a440 0x7f21581a06e0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f214800f990 tx=0x7f214800f9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:05:33.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.401+0000 7f2156000700 1 -- 172.21.15.151:0/2170804605 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2148027070 con 0x7f215810a440 2024-05-21T22:05:33.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.401+0000 7f2156000700 1 -- 172.21.15.151:0/2170804605 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2148005570 con 0x7f215810a440 2024-05-21T22:05:33.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.401+0000 7f2156000700 1 -- 172.21.15.151:0/2170804605 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2148004930 con 0x7f215810a440 2024-05-21T22:05:33.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.401+0000 7f215ea00700 1 -- 172.21.15.151:0/2170804605 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21581a5440 con 0x7f215810a440 2024-05-21T22:05:33.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.402+0000 7f215ea00700 1 -- 172.21.15.151:0/2170804605 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21581acd20 con 0x7f215810a440 2024-05-21T22:05:33.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.405+0000 7f2156000700 1 -- 172.21.15.151:0/2170804605 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7f214802a070 con 0x7f215810a440 2024-05-21T22:05:33.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.407+0000 7f216aa70b80 1 Processor -- start 2024-05-21T22:05:33.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.407+0000 7f216aa70b80 1 -- start start 2024-05-21T22:05:33.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.407+0000 7f2156000700 1 --2- 172.21.15.151:0/2170804605 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f21400b29f0 0x7f21400b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:33.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.407+0000 7f216aa70b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ecfd102020 0x55ecfd102400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:33.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.407+0000 7f2156000700 1 -- 172.21.15.151:0/2170804605 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7f21480e3070 con 0x7f215810a440 2024-05-21T22:05:33.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.407+0000 7f216aa70b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ecfd102940 0x55ecfd0fc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:33.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.407+0000 7f216aa70b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ecfd0fcad0 0x55ecfd0fcf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:33.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.407+0000 7f216aa70b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ecfd05ae40 con 0x55ecfd0fcad0 2024-05-21T22:05:33.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.407+0000 7f216aa70b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55ecfd04a9e0 con 0x55ecfd102020 2024-05-21T22:05:33.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.407+0000 7f216aa70b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55ecfd04ea20 con 0x55ecfd102940 2024-05-21T22:05:33.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.407+0000 7f215e000700 1 --2- 172.21.15.151:0/2170804605 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f21400b29f0 0x7f21400b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:33.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.408+0000 7f215d600700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ecfd102020 0x55ecfd102400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:33.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.408+0000 7f215d600700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ecfd102020 0x55ecfd102400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:56760/0 (socket says 172.21.15.151:56760) 2024-05-21T22:05:33.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.408+0000 7f215d600700 1 -- 172.21.15.151:0/3266761179 learned_addr learned my addr 172.21.15.151:0/3266761179 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:05:33.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.408+0000 7f215cc00700 1 --2- 172.21.15.151:0/3266761179 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ecfd102940 0x55ecfd0fc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:33.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.408+0000 7f215e000700 1 --2- 172.21.15.151:0/3266761179 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ecfd0fcad0 0x55ecfd0fcf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:33.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.408+0000 7f215d600700 1 -- 172.21.15.151:0/3266761179 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ecfd102940 msgr2=0x55ecfd0fc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:33.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.408+0000 7f215d600700 1 --2- 172.21.15.151:0/3266761179 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ecfd102940 0x55ecfd0fc4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:33.410 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.408+0000 7f215d600700 1 -- 172.21.15.151:0/3266761179 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ecfd0fcad0 msgr2=0x55ecfd0fcf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:33.410 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.408+0000 7f215d600700 1 --2- 172.21.15.151:0/3266761179 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ecfd0fcad0 0x55ecfd0fcf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:33.410 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.408+0000 7f215d600700 1 -- 172.21.15.151:0/3266761179 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ecfd013420 con 0x55ecfd102020 2024-05-21T22:05:33.410 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.408+0000 7f215d600700 1 --2- 172.21.15.151:0/3266761179 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ecfd102020 0x55ecfd102400 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f214c009d30 tx=0x7f214c011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d304eac68392d9d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:05:33.410 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.409+0000 7f215e000700 1 --2- 172.21.15.151:0/2170804605 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f21400b29f0 0x7f21400b4eb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2150009fd0 tx=0x7f2150007a60 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:05:33.410 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.409+0000 7f2146a00700 1 -- 172.21.15.151:0/3266761179 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f214c004220 con 0x55ecfd102020 2024-05-21T22:05:33.410 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.409+0000 7f2146a00700 1 -- 172.21.15.151:0/3266761179 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f214c0043c0 con 0x55ecfd102020 2024-05-21T22:05:33.410 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.409+0000 7f2146a00700 1 -- 172.21.15.151:0/3266761179 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f214c0197a0 con 0x55ecfd102020 2024-05-21T22:05:33.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.410+0000 7f216aa70b80 1 -- 172.21.15.151:0/3266761179 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ecfd102020 msgr2=0x55ecfd102400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:33.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.410+0000 7f216aa70b80 1 --2- 172.21.15.151:0/3266761179 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ecfd102020 0x55ecfd102400 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f214c009d30 tx=0x7f214c011720 comp rx=0 tx=0).stop 2024-05-21T22:05:33.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.410+0000 7f216aa70b80 1 -- 172.21.15.151:0/3266761179 shutdown_connections 2024-05-21T22:05:33.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.410+0000 7f216aa70b80 1 --2- 172.21.15.151:0/3266761179 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ecfd102020 0x55ecfd102400 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:33.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.410+0000 7f216aa70b80 1 --2- 172.21.15.151:0/3266761179 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ecfd0fcad0 0x55ecfd0fcf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:33.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.410+0000 7f216aa70b80 1 --2- 172.21.15.151:0/3266761179 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ecfd102940 0x55ecfd0fc4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:33.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.410+0000 7f216aa70b80 1 -- 172.21.15.151:0/3266761179 >> 172.21.15.151:0/3266761179 conn(0x55ecfd02abe0 msgr2=0x55ecfd0f7150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:05:33.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.410+0000 7f216aa70b80 1 -- 172.21.15.151:0/3266761179 shutdown_connections 2024-05-21T22:05:33.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.410+0000 7f216aa70b80 1 -- 172.21.15.151:0/3266761179 wait complete. 2024-05-21T22:05:33.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.410+0000 7f216aa70b80 1 -- start start 2024-05-21T22:05:33.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.411+0000 7f216aa70b80 1 Processor -- start 2024-05-21T22:05:33.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.411+0000 7f216aa70b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ecfd0fcad0 0x55ecfd1085c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:33.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.411+0000 7f216aa70b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ecfd102020 0x55ecfd105960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:33.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.411+0000 7f216aa70b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ecfd102940 0x55ecfd105ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:33.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.411+0000 7f216aa70b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ecfcf50fa0 con 0x55ecfd102020 2024-05-21T22:05:33.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.411+0000 7f216aa70b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55ecfcfeb760 con 0x55ecfd0fcad0 2024-05-21T22:05:33.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.411+0000 7f216aa70b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55ecfd109460 con 0x55ecfd102940 2024-05-21T22:05:33.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.411+0000 7f216aa70b80 10 client.0 _ll_register_callbacks cb 0x55ecfcff4c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:05:33.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.412+0000 7f215cc00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ecfd102020 0x55ecfd105960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:33.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.412+0000 7f215e000700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ecfd102940 0x55ecfd105ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:33.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.412+0000 7f215d600700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ecfd0fcad0 0x55ecfd1085c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:33.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.412+0000 7f215e000700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ecfd102940 0x55ecfd105ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:53536/0 (socket says 172.21.15.151:53536) 2024-05-21T22:05:33.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.412+0000 7f215e000700 1 -- 172.21.15.151:0/3990973347 learned_addr learned my addr 172.21.15.151:0/3990973347 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:05:33.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.412+0000 7f215d600700 1 -- 172.21.15.151:0/3990973347 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ecfd102940 msgr2=0x55ecfd105ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:33.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.412+0000 7f215d600700 1 --2- 172.21.15.151:0/3990973347 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ecfd102940 0x55ecfd105ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:33.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.412+0000 7f215d600700 1 -- 172.21.15.151:0/3990973347 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ecfd102020 msgr2=0x55ecfd105960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:33.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.412+0000 7f215d600700 1 --2- 172.21.15.151:0/3990973347 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ecfd102020 0x55ecfd105960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:33.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.412+0000 7f215d600700 1 -- 172.21.15.151:0/3990973347 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ecfcf9a8d0 con 0x55ecfd0fcad0 2024-05-21T22:05:33.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.412+0000 7f215d600700 1 --2- 172.21.15.151:0/3990973347 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ecfd0fcad0 0x55ecfd1085c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f214c009900 tx=0x7f214c0057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bcc21a3c638b59cc server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:05:33.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.412+0000 7f213b400700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-21T22:05:33.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.413+0000 7f213b400700 1 -- 172.21.15.151:0/3990973347 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f214c027070 con 0x55ecfd0fcad0 2024-05-21T22:05:33.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.413+0000 7f216aa70b80 1 -- 172.21.15.151:0/3990973347 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f214c0043c0 con 0x55ecfd0fcad0 2024-05-21T22:05:33.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.413+0000 7f213b400700 1 -- 172.21.15.151:0/3990973347 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7f214c004b60 con 0x55ecfd0fcad0 2024-05-21T22:05:33.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.413+0000 7f213b400700 1 -- 172.21.15.151:0/3990973347 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f214c033050 con 0x55ecfd0fcad0 2024-05-21T22:05:33.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.413+0000 7f216aa70b80 10 client.27524 fetch_fsmap learned FSMap version 29 2024-05-21T22:05:33.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.413+0000 7f216aa70b80 1 -- 172.21.15.151:0/3990973347 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ecfd01b060 con 0x55ecfd0fcad0 2024-05-21T22:05:33.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.413+0000 7f213b400700 1 -- 172.21.15.151:0/3990973347 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f214c0043c0 con 0x55ecfd0fcad0 2024-05-21T22:05:33.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.413+0000 7f213b400700 1 -- 172.21.15.151:0/3990973347 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f214c019430 con 0x55ecfd0fcad0 2024-05-21T22:05:33.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.413+0000 7f216aa70b80 10 client.27524 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:05:33.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.413+0000 7f216aa70b80 10 client.27524 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T22:05:33.417 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.414+0000 7f216aa70b80 20 client.27524 populate_metadata read hostname 'smithi151' 2024-05-21T22:05:33.417 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.414+0000 7f216aa70b80 1 --2- 172.21.15.151:0/3990973347 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55ecfd115dc0 0x55ecfd1181b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:33.417 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.414+0000 7f216aa70b80 4 client.27524 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T22:05:33.417 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.414+0000 7f216aa70b80 1 -- 172.21.15.151:0/3990973347 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ecfd05cd90 con 0x55ecfd115dc0 2024-05-21T22:05:33.417 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.414+0000 7f215cc00700 1 --2- 172.21.15.151:0/3990973347 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55ecfd115dc0 0x55ecfd1181b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:33.417 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.414+0000 7f215cc00700 1 --2- 172.21.15.151:0/3990973347 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55ecfd115dc0 0x55ecfd1181b0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2c5988f6a329e9c server_cookie=16114db73bc5b5d5 in_seq=0 out_seq=0 2024-05-21T22:05:33.417 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.414+0000 7f213b400700 10 client.27524 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T22:05:33.417 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.415+0000 7f213b400700 1 -- 172.21.15.151:0/3990973347 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55ecfd05cd90 con 0x55ecfd115dc0 2024-05-21T22:05:33.417 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.415+0000 7f213b400700 10 client.27524 handle_command_reply: tid=0 2024-05-21T22:05:33.417 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.415+0000 7f216aa70b80 1 client.27524 shutdown 2024-05-21T22:05:33.417 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.415+0000 7f216aa70b80 1 -- 172.21.15.151:0/3990973347 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ecfd0fcad0 msgr2=0x55ecfd1085c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:33.417 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.415+0000 7f216aa70b80 1 --2- 172.21.15.151:0/3990973347 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ecfd0fcad0 0x55ecfd1085c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f214c009900 tx=0x7f214c0057f0 comp rx=0 tx=0).stop 2024-05-21T22:05:33.417 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.416+0000 7f216aa70b80 1 -- 172.21.15.151:0/3990973347 shutdown_connections 2024-05-21T22:05:33.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.416+0000 7f216aa70b80 1 --2- 172.21.15.151:0/3990973347 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55ecfd115dc0 0x55ecfd1181b0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:33.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.416+0000 7f216aa70b80 1 --2- 172.21.15.151:0/3990973347 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ecfd0fcad0 0x55ecfd1085c0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:33.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.416+0000 7f213b400700 0 client.27524 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T22:05:33.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.416+0000 7f216aa70b80 1 --2- 172.21.15.151:0/3990973347 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ecfd102940 0x55ecfd105ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:33.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.416+0000 7f216aa70b80 1 --2- 172.21.15.151:0/3990973347 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ecfd102020 0x55ecfd105960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:33.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.416+0000 7f216aa70b80 1 -- 172.21.15.151:0/3990973347 >> 172.21.15.151:0/3990973347 conn(0x55ecfd02abe0 msgr2=0x55ecfd0fb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:05:33.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.416+0000 7f216aa70b80 1 -- 172.21.15.151:0/3990973347 shutdown_connections 2024-05-21T22:05:33.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.416+0000 7f216aa70b80 1 -- 172.21.15.151:0/3990973347 wait complete. 2024-05-21T22:05:33.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.416+0000 7f216aa70b80 20 client.27524 trim_cache size 0 max 16384 2024-05-21T22:05:33.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.432+0000 7f216aa70b80 1 Processor -- start 2024-05-21T22:05:33.433 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.432+0000 7f216aa70b80 1 -- start start 2024-05-21T22:05:33.433 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.433+0000 7f216aa70b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ecfd0fcad0 0x55ecfd0fc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:33.434 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.433+0000 7f216aa70b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ecfd102020 0x55ecfd11d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:33.434 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.433+0000 7f216aa70b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ecfd102940 0x55ecfd11df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:33.434 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.433+0000 7f216aa70b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f21581b42f0 con 0x55ecfd102020 2024-05-21T22:05:33.434 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.433+0000 7f216aa70b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f21581b5e30 con 0x55ecfd102940 2024-05-21T22:05:33.434 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.433+0000 7f216aa70b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f21581b49c0 con 0x55ecfd0fcad0 2024-05-21T22:05:33.434 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.433+0000 7f215cc00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ecfd102020 0x55ecfd11d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:33.434 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.433+0000 7f215cc00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ecfd102020 0x55ecfd11d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52578/0 (socket says 172.21.15.151:52578) 2024-05-21T22:05:33.434 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.433+0000 7f215d600700 1 --2- 172.21.15.151:0/3001420460 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ecfd0fcad0 0x55ecfd0fc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:33.434 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.433+0000 7f215cc00700 1 -- 172.21.15.151:0/3001420460 learned_addr learned my addr 172.21.15.151:0/3001420460 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:05:33.434 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.433+0000 7f215e000700 1 --2- 172.21.15.151:0/3001420460 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ecfd102940 0x55ecfd11df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:33.436 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.434+0000 7f215d600700 1 -- 172.21.15.151:0/3001420460 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ecfd102940 msgr2=0x55ecfd11df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:33.436 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.434+0000 7f215d600700 1 --2- 172.21.15.151:0/3001420460 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ecfd102940 0x55ecfd11df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:33.436 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.434+0000 7f215d600700 1 -- 172.21.15.151:0/3001420460 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ecfd102020 msgr2=0x55ecfd11d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:33.436 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.434+0000 7f215d600700 1 --2- 172.21.15.151:0/3001420460 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ecfd102020 0x55ecfd11d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:33.436 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.434+0000 7f215d600700 1 -- 172.21.15.151:0/3001420460 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ecfd118740 con 0x55ecfd0fcad0 2024-05-21T22:05:33.436 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.434+0000 7f215cc00700 1 --2- 172.21.15.151:0/3001420460 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ecfd102020 0x55ecfd11d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:05:33.436 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.434+0000 7f215d600700 1 --2- 172.21.15.151:0/3001420460 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ecfd0fcad0 0x55ecfd0fc4b0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f214c019880 tx=0x7f214c01b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c0b3ad4ba5c1a1e server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:05:33.436 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.434+0000 7f2146000700 1 -- 172.21.15.151:0/3001420460 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f214c027040 con 0x55ecfd0fcad0 2024-05-21T22:05:33.436 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.434+0000 7f2146000700 1 -- 172.21.15.151:0/3001420460 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f214c004960 con 0x55ecfd0fcad0 2024-05-21T22:05:33.436 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.434+0000 7f2146000700 1 -- 172.21.15.151:0/3001420460 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f214c0236c0 con 0x55ecfd0fcad0 2024-05-21T22:05:33.436 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.435+0000 7f216aa70b80 1 -- 172.21.15.151:0/3001420460 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ecfd0fcad0 msgr2=0x55ecfd0fc4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:33.436 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.435+0000 7f216aa70b80 1 --2- 172.21.15.151:0/3001420460 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ecfd0fcad0 0x55ecfd0fc4b0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f214c019880 tx=0x7f214c01b000 comp rx=0 tx=0).stop 2024-05-21T22:05:33.436 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.435+0000 7f216aa70b80 1 -- 172.21.15.151:0/3001420460 shutdown_connections 2024-05-21T22:05:33.436 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.435+0000 7f216aa70b80 1 --2- 172.21.15.151:0/3001420460 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ecfd102940 0x55ecfd11df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:33.437 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.435+0000 7f216aa70b80 1 --2- 172.21.15.151:0/3001420460 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ecfd0fcad0 0x55ecfd0fc4b0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:33.437 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.435+0000 7f216aa70b80 1 --2- 172.21.15.151:0/3001420460 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ecfd102020 0x55ecfd11d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:33.437 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.435+0000 7f216aa70b80 1 -- 172.21.15.151:0/3001420460 >> 172.21.15.151:0/3001420460 conn(0x55ecfd02abe0 msgr2=0x55ecfd0f6c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:05:33.437 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.435+0000 7f216aa70b80 1 -- 172.21.15.151:0/3001420460 shutdown_connections 2024-05-21T22:05:33.437 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.435+0000 7f216aa70b80 1 -- 172.21.15.151:0/3001420460 wait complete. 2024-05-21T22:05:33.439 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.436+0000 7f216aa70b80 1 -- start start 2024-05-21T22:05:33.439 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.436+0000 7f216aa70b80 1 Processor -- start 2024-05-21T22:05:33.439 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.437+0000 7f216aa70b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ecfd0fcad0 0x55ecfd11d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:33.439 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.437+0000 7f216aa70b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ecfd102020 0x55ecfd11d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:33.439 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.437+0000 7f216aa70b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ecfd102940 0x55ecfd11df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:33.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.437+0000 7f216aa70b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f21581b5760 con 0x55ecfd102020 2024-05-21T22:05:33.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.437+0000 7f216aa70b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f21581b5090 con 0x55ecfd0fcad0 2024-05-21T22:05:33.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.437+0000 7f216aa70b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f21581b22b0 con 0x55ecfd102940 2024-05-21T22:05:33.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.437+0000 7f216aa70b80 10 client.0 _ll_register_callbacks cb 0x55ecfcff4c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:05:33.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.437+0000 7f215d600700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ecfd0fcad0 0x55ecfd11d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:33.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.437+0000 7f215d600700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ecfd0fcad0 0x55ecfd11d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:56776/0 (socket says 172.21.15.151:56776) 2024-05-21T22:05:33.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.437+0000 7f215d600700 1 -- 172.21.15.151:0/3074826334 learned_addr learned my addr 172.21.15.151:0/3074826334 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:05:33.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.437+0000 7f215cc00700 1 --2- 172.21.15.151:0/3074826334 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ecfd102020 0x55ecfd11d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:33.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.437+0000 7f215e000700 1 --2- 172.21.15.151:0/3074826334 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ecfd102940 0x55ecfd11df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:33.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.437+0000 7f215d600700 1 -- 172.21.15.151:0/3074826334 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ecfd102940 msgr2=0x55ecfd11df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:33.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.437+0000 7f215d600700 1 --2- 172.21.15.151:0/3074826334 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ecfd102940 0x55ecfd11df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:33.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.437+0000 7f215d600700 1 -- 172.21.15.151:0/3074826334 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ecfd102020 msgr2=0x55ecfd11d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:33.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.437+0000 7f215d600700 1 --2- 172.21.15.151:0/3074826334 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ecfd102020 0x55ecfd11d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:33.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.437+0000 7f215d600700 1 -- 172.21.15.151:0/3074826334 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ecfd17f890 con 0x55ecfd0fcad0 2024-05-21T22:05:33.441 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.438+0000 7f215d600700 1 --2- 172.21.15.151:0/3074826334 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ecfd0fcad0 0x55ecfd11d490 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f214c011760 tx=0x7f214c0196e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=589bdd815f68ad6c server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:05:33.441 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.438+0000 7f213b400700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-21T22:05:33.441 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.438+0000 7f213b400700 1 -- 172.21.15.151:0/3074826334 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f214c027040 con 0x55ecfd0fcad0 2024-05-21T22:05:33.441 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.438+0000 7f213b400700 1 -- 172.21.15.151:0/3074826334 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7f214c004d20 con 0x55ecfd0fcad0 2024-05-21T22:05:33.441 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.438+0000 7f216aa70b80 1 -- 172.21.15.151:0/3074826334 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f214c004960 con 0x55ecfd0fcad0 2024-05-21T22:05:33.441 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.438+0000 7f213b400700 1 -- 172.21.15.151:0/3074826334 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f214c04d050 con 0x55ecfd0fcad0 2024-05-21T22:05:33.441 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.438+0000 7f216aa70b80 10 client.27530 fetch_fsmap learned FSMap version 29 2024-05-21T22:05:33.441 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.438+0000 7f216aa70b80 1 -- 172.21.15.151:0/3074826334 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ecfd180ad0 con 0x55ecfd0fcad0 2024-05-21T22:05:33.441 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.439+0000 7f213b400700 1 -- 172.21.15.151:0/3074826334 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f214c004960 con 0x55ecfd0fcad0 2024-05-21T22:05:33.441 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.439+0000 7f213b400700 1 -- 172.21.15.151:0/3074826334 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f214c0238f0 con 0x55ecfd0fcad0 2024-05-21T22:05:33.441 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.439+0000 7f216aa70b80 10 client.27530 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:05:33.441 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.439+0000 7f216aa70b80 10 client.27530 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T22:05:33.441 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.439+0000 7f216aa70b80 20 client.27530 populate_metadata read hostname 'smithi151' 2024-05-21T22:05:33.441 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.439+0000 7f216aa70b80 1 --2- 172.21.15.151:0/3074826334 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55ecfd1810e0 0x55ecfd183500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:33.441 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.439+0000 7f216aa70b80 4 client.27530 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T22:05:33.442 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.439+0000 7f216aa70b80 1 -- 172.21.15.151:0/3074826334 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ecfd056570 con 0x55ecfd1810e0 2024-05-21T22:05:33.442 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.439+0000 7f215cc00700 1 --2- 172.21.15.151:0/3074826334 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55ecfd1810e0 0x55ecfd183500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:33.443 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.440+0000 7f215cc00700 1 --2- 172.21.15.151:0/3074826334 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55ecfd1810e0 0x55ecfd183500 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d184e8441349570f server_cookie=f41725c4670844d0 in_seq=0 out_seq=0 2024-05-21T22:05:33.443 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.440+0000 7f213b400700 10 client.27530 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T22:05:33.443 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.440+0000 7f213b400700 1 -- 172.21.15.151:0/3074826334 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ecfd056570 con 0x55ecfd1810e0 2024-05-21T22:05:33.443 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.440+0000 7f213b400700 10 client.27530 handle_command_reply: tid=0 2024-05-21T22:05:33.443 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.440+0000 7f216aa70b80 1 client.27530 shutdown 2024-05-21T22:05:33.443 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.440+0000 7f216aa70b80 1 -- 172.21.15.151:0/3074826334 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ecfd0fcad0 msgr2=0x55ecfd11d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:33.443 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.440+0000 7f216aa70b80 1 --2- 172.21.15.151:0/3074826334 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ecfd0fcad0 0x55ecfd11d490 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f214c011760 tx=0x7f214c0196e0 comp rx=0 tx=0).stop 2024-05-21T22:05:33.443 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.440+0000 7f216aa70b80 1 -- 172.21.15.151:0/3074826334 shutdown_connections 2024-05-21T22:05:33.443 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.440+0000 7f216aa70b80 1 --2- 172.21.15.151:0/3074826334 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55ecfd1810e0 0x55ecfd183500 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:33.443 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.440+0000 7f216aa70b80 1 --2- 172.21.15.151:0/3074826334 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55ecfd0fcad0 0x55ecfd11d490 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:33.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.440+0000 7f216aa70b80 1 --2- 172.21.15.151:0/3074826334 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ecfd102940 0x55ecfd11df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:33.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.440+0000 7f216aa70b80 1 --2- 172.21.15.151:0/3074826334 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ecfd102020 0x55ecfd11d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:33.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.440+0000 7f213b400700 0 client.27530 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T22:05:33.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.440+0000 7f216aa70b80 1 -- 172.21.15.151:0/3074826334 >> 172.21.15.151:0/3074826334 conn(0x55ecfd02abe0 msgr2=0x55ecfd102d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:05:33.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.440+0000 7f216aa70b80 1 -- 172.21.15.151:0/3074826334 shutdown_connections 2024-05-21T22:05:33.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.440+0000 7f216aa70b80 1 -- 172.21.15.151:0/3074826334 wait complete. 2024-05-21T22:05:33.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.441+0000 7f216aa70b80 20 client.27530 trim_cache size 0 max 16384 2024-05-21T22:05:33.444 INFO:teuthology.orchestra.run.smithi151.stdout:[] 2024-05-21T22:05:33.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.441+0000 7f215ea00700 1 -- 172.21.15.151:0/2170804605 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f21400b29f0 msgr2=0x7f21400b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:33.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.441+0000 7f215ea00700 1 --2- 172.21.15.151:0/2170804605 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f21400b29f0 0x7f21400b4eb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2150009fd0 tx=0x7f2150007a60 comp rx=0 tx=0).stop 2024-05-21T22:05:33.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.441+0000 7f215ea00700 1 -- 172.21.15.151:0/2170804605 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f215810a440 msgr2=0x7f21581a06e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:33.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.441+0000 7f215ea00700 1 --2- 172.21.15.151:0/2170804605 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f215810a440 0x7f21581a06e0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f214800f990 tx=0x7f214800f9c0 comp rx=0 tx=0).stop 2024-05-21T22:05:33.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.441+0000 7f215ea00700 1 -- 172.21.15.151:0/2170804605 shutdown_connections 2024-05-21T22:05:33.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.441+0000 7f215ea00700 1 --2- 172.21.15.151:0/2170804605 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f21400b29f0 0x7f21400b4eb0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:33.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.441+0000 7f215ea00700 1 --2- 172.21.15.151:0/2170804605 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f215810e950 0x7f21581a4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:33.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.441+0000 7f215ea00700 1 --2- 172.21.15.151:0/2170804605 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2158109a80 0x7f21581a01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:33.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.441+0000 7f215ea00700 1 --2- 172.21.15.151:0/2170804605 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f215810a440 0x7f21581a06e0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:33.446 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.441+0000 7f215ea00700 1 -- 172.21.15.151:0/2170804605 >> 172.21.15.151:0/2170804605 conn(0x7f21580792d0 msgr2=0x7f21580799f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:05:33.446 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.442+0000 7f215ea00700 1 -- 172.21.15.151:0/2170804605 shutdown_connections 2024-05-21T22:05:33.446 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:33.442+0000 7f215ea00700 1 -- 172.21.15.151:0/2170804605 wait complete. 2024-05-21T22:05:33.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:05:33 smithi151 ceph-mon[31623]: pgmap v307: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T22:05:33.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:05:33 smithi184 ceph-mon[36942]: pgmap v307: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T22:05:34.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:05:33 smithi164 ceph-mon[36269]: pgmap v307: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T22:05:35.351 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-21T22:05:35.351 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-21T22:05:35.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:05:35 smithi151 ceph-mon[31623]: pgmap v308: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:05:35.819 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:05:35.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:05:35 smithi184 ceph-mon[36942]: pgmap v308: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:05:36.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:05:35 smithi164 ceph-mon[36269]: pgmap v308: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:05:37.841 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:05:37 smithi151 ceph-mon[31623]: pgmap v309: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:05:37.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:05:37 smithi184 ceph-mon[36942]: pgmap v309: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:05:38.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.010+0000 7ff04a400700 1 -- 172.21.15.151:0/1816244682 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff044109a80 msgr2=0x7ff044109f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:38.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.010+0000 7ff04a400700 1 --2- 172.21.15.151:0/1816244682 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff044109a80 0x7ff044109f00 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7ff030009990 tx=0x7ff030009ca0 comp rx=0 tx=0).stop 2024-05-21T22:05:38.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.010+0000 7ff04a400700 1 -- 172.21.15.151:0/1816244682 shutdown_connections 2024-05-21T22:05:38.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.010+0000 7ff04a400700 1 --2- 172.21.15.151:0/1816244682 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff04410e950 0x7ff04410ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:38.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.010+0000 7ff04a400700 1 --2- 172.21.15.151:0/1816244682 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff04410a440 0x7ff0441168e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:38.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.010+0000 7ff04a400700 1 --2- 172.21.15.151:0/1816244682 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff044109a80 0x7ff044109f00 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:38.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.010+0000 7ff04a400700 1 -- 172.21.15.151:0/1816244682 >> 172.21.15.151:0/1816244682 conn(0x7ff0440792d0 msgr2=0x7ff044108360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:05:38.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.010+0000 7ff04a400700 1 -- 172.21.15.151:0/1816244682 shutdown_connections 2024-05-21T22:05:38.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.010+0000 7ff04a400700 1 -- 172.21.15.151:0/1816244682 wait complete. 2024-05-21T22:05:38.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.011+0000 7ff04a400700 1 Processor -- start 2024-05-21T22:05:38.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.011+0000 7ff04a400700 1 -- start start 2024-05-21T22:05:38.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.012+0000 7ff04a400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff044109a80 0x7ff0441a01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:38.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.012+0000 7ff04a400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff04410a440 0x7ff0441a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:38.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.012+0000 7ff04a400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff04410e950 0x7ff0441a4a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:38.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.012+0000 7ff04a400700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff044119ca0 con 0x7ff044109a80 2024-05-21T22:05:38.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.012+0000 7ff04a400700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff044119b20 con 0x7ff04410a440 2024-05-21T22:05:38.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.012+0000 7ff04a400700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff044119e20 con 0x7ff04410e950 2024-05-21T22:05:38.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.012+0000 7ff049000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff044109a80 0x7ff0441a01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:38.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.012+0000 7ff043e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff04410a440 0x7ff0441a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:38.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.012+0000 7ff049000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff044109a80 0x7ff0441a01b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:53224/0 (socket says 172.21.15.151:53224) 2024-05-21T22:05:38.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.012+0000 7ff043e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff04410a440 0x7ff0441a06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:48664/0 (socket says 172.21.15.151:48664) 2024-05-21T22:05:38.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.012+0000 7ff043e00700 1 -- 172.21.15.151:0/3996058403 learned_addr learned my addr 172.21.15.151:0/3996058403 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:05:38.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.012+0000 7ff049a00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff04410e950 0x7ff0441a4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:38.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.013+0000 7ff043e00700 1 -- 172.21.15.151:0/3996058403 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff04410e950 msgr2=0x7ff0441a4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:38.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.013+0000 7ff043e00700 1 --2- 172.21.15.151:0/3996058403 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff04410e950 0x7ff0441a4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:38.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.013+0000 7ff043e00700 1 -- 172.21.15.151:0/3996058403 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff044109a80 msgr2=0x7ff0441a01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:38.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.013+0000 7ff043e00700 1 --2- 172.21.15.151:0/3996058403 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff044109a80 0x7ff0441a01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:38.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.013+0000 7ff043e00700 1 -- 172.21.15.151:0/3996058403 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0441a5170 con 0x7ff04410a440 2024-05-21T22:05:38.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.013+0000 7ff049000700 1 --2- 172.21.15.151:0/3996058403 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff044109a80 0x7ff0441a01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:05:38.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.013+0000 7ff043e00700 1 --2- 172.21.15.151:0/3996058403 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff04410a440 0x7ff0441a06f0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7ff030000c00 tx=0x7ff0300057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:05:38.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.013+0000 7ff041600700 1 -- 172.21.15.151:0/3996058403 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff030027070 con 0x7ff04410a440 2024-05-21T22:05:38.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.013+0000 7ff04a400700 1 -- 172.21.15.151:0/3996058403 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0441a5400 con 0x7ff04410a440 2024-05-21T22:05:38.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.014+0000 7ff04a400700 1 -- 172.21.15.151:0/3996058403 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0441acce0 con 0x7ff04410a440 2024-05-21T22:05:38.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.014+0000 7ff041600700 1 -- 172.21.15.151:0/3996058403 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff030022470 con 0x7ff04410a440 2024-05-21T22:05:38.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.014+0000 7ff041600700 1 -- 172.21.15.151:0/3996058403 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff030017650 con 0x7ff04410a440 2024-05-21T22:05:38.017 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.017+0000 7ff041600700 1 -- 172.21.15.151:0/3996058403 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7ff03002a070 con 0x7ff04410a440 2024-05-21T22:05:38.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.019+0000 7ff056426b80 1 Processor -- start 2024-05-21T22:05:38.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.019+0000 7ff056426b80 1 -- start start 2024-05-21T22:05:38.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.019+0000 7ff056426b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4474940a0 0x55a447494480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:38.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.019+0000 7ff056426b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a4474949c0 0x55a44748e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:38.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.019+0000 7ff056426b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a44748eb50 0x55a44748efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:38.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.019+0000 7ff056426b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a4473ecec0 con 0x55a44748eb50 2024-05-21T22:05:38.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.019+0000 7ff056426b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55a4473dca60 con 0x55a4474949c0 2024-05-21T22:05:38.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.019+0000 7ff056426b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55a4473e0aa0 con 0x55a4474940a0 2024-05-21T22:05:38.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.020+0000 7ff041600700 1 --2- 172.21.15.151:0/3996058403 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7ff02c0b29f0 0x7ff02c0b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:38.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.020+0000 7ff049000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a4474949c0 0x55a44748e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:38.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.020+0000 7ff049a00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4474940a0 0x55a447494480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:38.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.020+0000 7ff049000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a4474949c0 0x55a44748e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:48668/0 (socket says 172.21.15.151:48668) 2024-05-21T22:05:38.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.020+0000 7ff049000700 1 -- 172.21.15.151:0/3208743384 learned_addr learned my addr 172.21.15.151:0/3208743384 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:05:38.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.020+0000 7ff043e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a44748eb50 0x55a44748efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:38.022 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.020+0000 7ff049a00700 1 --2- 172.21.15.151:0/3996058403 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7ff02c0b29f0 0x7ff02c0b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:38.022 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.020+0000 7ff041600700 1 -- 172.21.15.151:0/3996058403 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7ff0300e3330 con 0x7ff04410a440 2024-05-21T22:05:38.022 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.020+0000 7ff049000700 1 -- 172.21.15.151:0/3208743384 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4474940a0 msgr2=0x55a447494480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:38.022 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.020+0000 7ff049000700 1 --2- 172.21.15.151:0/3208743384 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4474940a0 0x55a447494480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:38.022 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.020+0000 7ff049000700 1 -- 172.21.15.151:0/3208743384 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a44748eb50 msgr2=0x55a44748efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:38.022 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.020+0000 7ff049000700 1 --2- 172.21.15.151:0/3208743384 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a44748eb50 0x55a44748efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:38.022 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.020+0000 7ff049000700 1 -- 172.21.15.151:0/3208743384 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4473a54a0 con 0x55a4474949c0 2024-05-21T22:05:38.022 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.021+0000 7ff049000700 1 --2- 172.21.15.151:0/3208743384 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a4474949c0 0x55a44748e530 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7ff0380115c0 tx=0x7ff038009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=638477608d6291b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:05:38.022 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.021+0000 7ff049a00700 1 --2- 172.21.15.151:0/3996058403 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7ff02c0b29f0 0x7ff02c0b4eb0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff03c008c10 tx=0x7ff03c009f90 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:05:38.022 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.021+0000 7ff036000700 1 -- 172.21.15.151:0/3208743384 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff03801e070 con 0x55a4474949c0 2024-05-21T22:05:38.022 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.021+0000 7ff036000700 1 -- 172.21.15.151:0/3208743384 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff038019ae0 con 0x55a4474949c0 2024-05-21T22:05:38.022 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.021+0000 7ff036000700 1 -- 172.21.15.151:0/3208743384 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff03800eb60 con 0x55a4474949c0 2024-05-21T22:05:38.024 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.022+0000 7ff056426b80 1 -- 172.21.15.151:0/3208743384 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a4474949c0 msgr2=0x55a44748e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:38.024 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.022+0000 7ff056426b80 1 --2- 172.21.15.151:0/3208743384 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a4474949c0 0x55a44748e530 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7ff0380115c0 tx=0x7ff038009f90 comp rx=0 tx=0).stop 2024-05-21T22:05:38.024 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.022+0000 7ff056426b80 1 -- 172.21.15.151:0/3208743384 shutdown_connections 2024-05-21T22:05:38.024 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.022+0000 7ff056426b80 1 --2- 172.21.15.151:0/3208743384 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a44748eb50 0x55a44748efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:38.024 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.022+0000 7ff056426b80 1 --2- 172.21.15.151:0/3208743384 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4474940a0 0x55a447494480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:38.024 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.022+0000 7ff056426b80 1 --2- 172.21.15.151:0/3208743384 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a4474949c0 0x55a44748e530 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:38.024 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.022+0000 7ff056426b80 1 -- 172.21.15.151:0/3208743384 >> 172.21.15.151:0/3208743384 conn(0x55a4473bcc60 msgr2=0x55a4474891d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:05:38.024 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.022+0000 7ff056426b80 1 -- 172.21.15.151:0/3208743384 shutdown_connections 2024-05-21T22:05:38.024 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.022+0000 7ff056426b80 1 -- 172.21.15.151:0/3208743384 wait complete. 2024-05-21T22:05:38.024 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.023+0000 7ff056426b80 1 -- start start 2024-05-21T22:05:38.024 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.023+0000 7ff056426b80 1 Processor -- start 2024-05-21T22:05:38.025 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.023+0000 7ff056426b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a44748eb50 0x55a44749a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:38.025 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.023+0000 7ff056426b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a4474940a0 0x55a447497ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:38.025 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.023+0000 7ff056426b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a4474949c0 0x55a447498000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:38.025 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.023+0000 7ff056426b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a4472e3020 con 0x55a4474949c0 2024-05-21T22:05:38.025 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.023+0000 7ff056426b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55a44737d7e0 con 0x55a4474940a0 2024-05-21T22:05:38.025 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.023+0000 7ff056426b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55a44749b4e0 con 0x55a44748eb50 2024-05-21T22:05:38.025 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.024+0000 7ff056426b80 10 client.0 _ll_register_callbacks cb 0x55a447386cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:05:38.025 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.024+0000 7ff043e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a4474940a0 0x55a447497ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:38.025 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.024+0000 7ff049a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a4474949c0 0x55a447498000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:38.025 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.024+0000 7ff043e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a4474940a0 0x55a447497ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:48678/0 (socket says 172.21.15.151:48678) 2024-05-21T22:05:38.025 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.024+0000 7ff043e00700 1 -- 172.21.15.151:0/422716450 learned_addr learned my addr 172.21.15.151:0/422716450 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:05:38.025 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.024+0000 7ff049000700 1 --2- 172.21.15.151:0/422716450 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a44748eb50 0x55a44749a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:38.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.024+0000 7ff043e00700 1 -- 172.21.15.151:0/422716450 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a44748eb50 msgr2=0x55a44749a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:38.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.024+0000 7ff043e00700 1 --2- 172.21.15.151:0/422716450 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a44748eb50 0x55a44749a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:38.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.024+0000 7ff043e00700 1 -- 172.21.15.151:0/422716450 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a4474949c0 msgr2=0x55a447498000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:38.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.024+0000 7ff043e00700 1 --2- 172.21.15.151:0/422716450 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a4474949c0 0x55a447498000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:38.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.024+0000 7ff043e00700 1 -- 172.21.15.151:0/422716450 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a44732c950 con 0x55a4474940a0 2024-05-21T22:05:38.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.025+0000 7ff043e00700 1 --2- 172.21.15.151:0/422716450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a4474940a0 0x55a447497ac0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7ff0300a78a0 tx=0x7ff030070070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d12917e07d239f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:05:38.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.025+0000 7ff02aa00700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-21T22:05:38.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.025+0000 7ff02aa00700 1 -- 172.21.15.151:0/422716450 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0300a9da0 con 0x55a4474940a0 2024-05-21T22:05:38.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.025+0000 7ff02aa00700 1 -- 172.21.15.151:0/422716450 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7ff0300cbfd0 con 0x55a4474940a0 2024-05-21T22:05:38.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.025+0000 7ff056426b80 1 -- 172.21.15.151:0/422716450 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff038019ae0 con 0x55a4474940a0 2024-05-21T22:05:38.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.025+0000 7ff02aa00700 1 -- 172.21.15.151:0/422716450 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff0300aa140 con 0x55a4474940a0 2024-05-21T22:05:38.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.025+0000 7ff056426b80 10 client.27542 fetch_fsmap learned FSMap version 29 2024-05-21T22:05:38.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.025+0000 7ff056426b80 1 -- 172.21.15.151:0/422716450 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a4473ad0e0 con 0x55a4474940a0 2024-05-21T22:05:38.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.025+0000 7ff02aa00700 1 -- 172.21.15.151:0/422716450 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff038019ae0 con 0x55a4474940a0 2024-05-21T22:05:38.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.026+0000 7ff02aa00700 1 -- 172.21.15.151:0/422716450 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7ff0300e0c90 con 0x55a4474940a0 2024-05-21T22:05:38.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.026+0000 7ff056426b80 10 client.27542 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:05:38.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.026+0000 7ff056426b80 10 client.27542 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T22:05:38.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.026+0000 7ff056426b80 20 client.27542 populate_metadata read hostname 'smithi151' 2024-05-21T22:05:38.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.026+0000 7ff056426b80 1 --2- 172.21.15.151:0/422716450 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55a4474a8000 0x55a4474aa420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:38.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.026+0000 7ff056426b80 4 client.27542 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T22:05:38.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.026+0000 7ff056426b80 1 -- 172.21.15.151:0/422716450 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a4473eee10 con 0x55a4474a8000 2024-05-21T22:05:38.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.026+0000 7ff049000700 1 --2- 172.21.15.151:0/422716450 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55a4474a8000 0x55a4474aa420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:38.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.027+0000 7ff049000700 1 --2- 172.21.15.151:0/422716450 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55a4474a8000 0x55a4474aa420 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6bf022e1156f3f0 server_cookie=b0d85f42574d5350 in_seq=0 out_seq=0 2024-05-21T22:05:38.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.027+0000 7ff02aa00700 10 client.27542 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T22:05:38.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.027+0000 7ff02aa00700 1 -- 172.21.15.151:0/422716450 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55a4473eee10 con 0x55a4474a8000 2024-05-21T22:05:38.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.027+0000 7ff02aa00700 10 client.27542 handle_command_reply: tid=0 2024-05-21T22:05:38.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.028+0000 7ff056426b80 1 client.27542 shutdown 2024-05-21T22:05:38.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.028+0000 7ff056426b80 1 -- 172.21.15.151:0/422716450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a4474940a0 msgr2=0x55a447497ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:38.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.028+0000 7ff056426b80 1 --2- 172.21.15.151:0/422716450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a4474940a0 0x55a447497ac0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7ff0300a78a0 tx=0x7ff030070070 comp rx=0 tx=0).stop 2024-05-21T22:05:38.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.028+0000 7ff056426b80 1 -- 172.21.15.151:0/422716450 shutdown_connections 2024-05-21T22:05:38.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.028+0000 7ff056426b80 1 --2- 172.21.15.151:0/422716450 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a4474949c0 0x55a447498000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:38.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.028+0000 7ff056426b80 1 --2- 172.21.15.151:0/422716450 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a44748eb50 0x55a44749a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:38.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.028+0000 7ff056426b80 1 --2- 172.21.15.151:0/422716450 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55a4474a8000 0x55a4474aa420 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:38.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.028+0000 7ff056426b80 1 --2- 172.21.15.151:0/422716450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a4474940a0 0x55a447497ac0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:38.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.028+0000 7ff056426b80 1 -- 172.21.15.151:0/422716450 >> 172.21.15.151:0/422716450 conn(0x55a4473bcc60 msgr2=0x55a44748d520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:05:38.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.028+0000 7ff02aa00700 0 client.27542 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T22:05:38.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.028+0000 7ff056426b80 1 -- 172.21.15.151:0/422716450 shutdown_connections 2024-05-21T22:05:38.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.028+0000 7ff056426b80 1 -- 172.21.15.151:0/422716450 wait complete. 2024-05-21T22:05:38.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.028+0000 7ff056426b80 20 client.27542 trim_cache size 0 max 16384 2024-05-21T22:05:38.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:05:37 smithi164 ceph-mon[36269]: pgmap v309: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:05:38.051 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.051+0000 7ff056426b80 1 Processor -- start 2024-05-21T22:05:38.051 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.051+0000 7ff056426b80 1 -- start start 2024-05-21T22:05:38.051 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.051+0000 7ff056426b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a44748eb50 0x55a44748e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:38.051 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.051+0000 7ff056426b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a4474940a0 0x55a447495630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:38.053 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.051+0000 7ff056426b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a4474949c0 0x55a447496ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:38.053 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.051+0000 7ff056426b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff0441b41c0 con 0x55a4474949c0 2024-05-21T22:05:38.053 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.051+0000 7ff056426b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff0441b5d00 con 0x55a4474940a0 2024-05-21T22:05:38.053 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.051+0000 7ff056426b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff0441b4890 con 0x55a44748eb50 2024-05-21T22:05:38.053 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.051+0000 7ff049a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a4474949c0 0x55a447496ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:38.053 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.052+0000 7ff049a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a4474949c0 0x55a447496ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:53252/0 (socket says 172.21.15.151:53252) 2024-05-21T22:05:38.053 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.052+0000 7ff049a00700 1 -- 172.21.15.151:0/2335886932 learned_addr learned my addr 172.21.15.151:0/2335886932 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:05:38.053 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.052+0000 7ff049000700 1 --2- 172.21.15.151:0/2335886932 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a44748eb50 0x55a44748e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:38.053 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.052+0000 7ff043e00700 1 --2- 172.21.15.151:0/2335886932 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a4474940a0 0x55a447495630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:38.053 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.052+0000 7ff049000700 1 -- 172.21.15.151:0/2335886932 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a4474940a0 msgr2=0x55a447495630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:38.053 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.052+0000 7ff049000700 1 --2- 172.21.15.151:0/2335886932 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a4474940a0 0x55a447495630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:38.053 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.052+0000 7ff049000700 1 -- 172.21.15.151:0/2335886932 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a4474949c0 msgr2=0x55a447496ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:38.054 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.052+0000 7ff049000700 1 --2- 172.21.15.151:0/2335886932 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a4474949c0 0x55a447496ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:38.054 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.052+0000 7ff049000700 1 -- 172.21.15.151:0/2335886932 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a44748ef30 con 0x55a44748eb50 2024-05-21T22:05:38.054 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.052+0000 7ff049a00700 1 --2- 172.21.15.151:0/2335886932 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a4474949c0 0x55a447496ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:05:38.054 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.052+0000 7ff049000700 1 --2- 172.21.15.151:0/2335886932 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a44748eb50 0x55a44748e530 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff038005950 tx=0x7ff0380058e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da47dcd35ec98007 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:05:38.055 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.053+0000 7ff035600700 1 -- 172.21.15.151:0/2335886932 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff03801e040 con 0x55a44748eb50 2024-05-21T22:05:38.055 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.053+0000 7ff035600700 1 -- 172.21.15.151:0/2335886932 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff03802b040 con 0x55a44748eb50 2024-05-21T22:05:38.055 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.053+0000 7ff035600700 1 -- 172.21.15.151:0/2335886932 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff038003e10 con 0x55a44748eb50 2024-05-21T22:05:38.055 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.053+0000 7ff056426b80 1 -- 172.21.15.151:0/2335886932 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a44748eb50 msgr2=0x55a44748e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:38.056 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.053+0000 7ff056426b80 1 --2- 172.21.15.151:0/2335886932 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a44748eb50 0x55a44748e530 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff038005950 tx=0x7ff0380058e0 comp rx=0 tx=0).stop 2024-05-21T22:05:38.056 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.054+0000 7ff056426b80 1 -- 172.21.15.151:0/2335886932 shutdown_connections 2024-05-21T22:05:38.056 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.054+0000 7ff056426b80 1 --2- 172.21.15.151:0/2335886932 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a4474949c0 0x55a447496ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:38.056 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.054+0000 7ff056426b80 1 --2- 172.21.15.151:0/2335886932 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a44748eb50 0x55a44748e530 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:38.056 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.054+0000 7ff056426b80 1 --2- 172.21.15.151:0/2335886932 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a4474940a0 0x55a447495630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:38.056 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.054+0000 7ff056426b80 1 -- 172.21.15.151:0/2335886932 >> 172.21.15.151:0/2335886932 conn(0x55a4473bcc60 msgr2=0x55a4474963b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:05:38.056 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.054+0000 7ff056426b80 1 -- 172.21.15.151:0/2335886932 shutdown_connections 2024-05-21T22:05:38.056 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.054+0000 7ff056426b80 1 -- 172.21.15.151:0/2335886932 wait complete. 2024-05-21T22:05:38.056 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.054+0000 7ff056426b80 1 -- start start 2024-05-21T22:05:38.056 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.054+0000 7ff056426b80 1 Processor -- start 2024-05-21T22:05:38.056 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.055+0000 7ff056426b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a44748eb50 0x55a44749df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:38.056 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.055+0000 7ff056426b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a4474940a0 0x55a44749e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:38.056 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.055+0000 7ff056426b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4474949c0 0x55a44750cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:38.056 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.055+0000 7ff056426b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff0441b5630 con 0x55a4474940a0 2024-05-21T22:05:38.056 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.055+0000 7ff056426b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff0441b4f60 con 0x55a44748eb50 2024-05-21T22:05:38.057 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.055+0000 7ff056426b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff0441b2180 con 0x55a4474949c0 2024-05-21T22:05:38.057 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.055+0000 7ff056426b80 10 client.0 _ll_register_callbacks cb 0x55a447386cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:05:38.058 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.055+0000 7ff043e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a4474940a0 0x55a44749e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:38.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.055+0000 7ff043e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a4474940a0 0x55a44749e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:53262/0 (socket says 172.21.15.151:53262) 2024-05-21T22:05:38.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.055+0000 7ff043e00700 1 -- 172.21.15.151:0/1140201813 learned_addr learned my addr 172.21.15.151:0/1140201813 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:05:38.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.056+0000 7ff049000700 1 --2- 172.21.15.151:0/1140201813 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a44748eb50 0x55a44749df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:38.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.056+0000 7ff049a00700 1 --2- 172.21.15.151:0/1140201813 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4474949c0 0x55a44750cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:38.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.056+0000 7ff043e00700 1 -- 172.21.15.151:0/1140201813 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4474949c0 msgr2=0x55a44750cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:38.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.056+0000 7ff043e00700 1 --2- 172.21.15.151:0/1140201813 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4474949c0 0x55a44750cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:38.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.056+0000 7ff043e00700 1 -- 172.21.15.151:0/1140201813 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a44748eb50 msgr2=0x55a44749df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:38.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.056+0000 7ff043e00700 1 --2- 172.21.15.151:0/1140201813 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a44748eb50 0x55a44749df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:38.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.056+0000 7ff043e00700 1 -- 172.21.15.151:0/1140201813 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a44750d220 con 0x55a4474940a0 2024-05-21T22:05:38.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.056+0000 7ff049000700 1 --2- 172.21.15.151:0/1140201813 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a44748eb50 0x55a44749df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:05:38.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.056+0000 7ff043e00700 1 --2- 172.21.15.151:0/1140201813 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a4474940a0 0x55a44749e440 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7ff0300aa330 tx=0x7ff0300e0fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4158c3c34fd6a8fe server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:05:38.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.056+0000 7ff02aa00700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-21T22:05:38.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.056+0000 7ff02aa00700 1 -- 172.21.15.151:0/1140201813 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0300a9da0 con 0x55a4474940a0 2024-05-21T22:05:38.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.056+0000 7ff02aa00700 1 -- 172.21.15.151:0/1140201813 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7ff0300afd80 con 0x55a4474940a0 2024-05-21T22:05:38.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.056+0000 7ff056426b80 1 -- 172.21.15.151:0/1140201813 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff03802b040 con 0x55a4474940a0 2024-05-21T22:05:38.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.057+0000 7ff02aa00700 1 -- 172.21.15.151:0/1140201813 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff0300aad30 con 0x55a4474940a0 2024-05-21T22:05:38.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.057+0000 7ff056426b80 10 client.17421 fetch_fsmap learned FSMap version 29 2024-05-21T22:05:38.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.057+0000 7ff056426b80 1 -- 172.21.15.151:0/1140201813 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a44750d220 con 0x55a4474940a0 2024-05-21T22:05:38.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.057+0000 7ff02aa00700 1 -- 172.21.15.151:0/1140201813 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff03802b040 con 0x55a4474940a0 2024-05-21T22:05:38.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.057+0000 7ff02aa00700 1 -- 172.21.15.151:0/1140201813 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7ff0300a9890 con 0x55a4474940a0 2024-05-21T22:05:38.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.057+0000 7ff056426b80 10 client.17421 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:05:38.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.057+0000 7ff056426b80 10 client.17421 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T22:05:38.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.057+0000 7ff056426b80 20 client.17421 populate_metadata read hostname 'smithi151' 2024-05-21T22:05:38.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.057+0000 7ff056426b80 1 --2- 172.21.15.151:0/1140201813 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55a447514900 0x55a447516cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:38.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.057+0000 7ff056426b80 4 client.17421 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-21T22:05:38.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.057+0000 7ff056426b80 1 -- 172.21.15.151:0/1140201813 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a4473e85f0 con 0x55a447514900 2024-05-21T22:05:38.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.058+0000 7ff049000700 1 --2- 172.21.15.151:0/1140201813 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55a447514900 0x55a447516cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:38.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.058+0000 7ff049000700 1 --2- 172.21.15.151:0/1140201813 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55a447514900 0x55a447516cf0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5134ee358f65141 server_cookie=9871ac5a3c108993 in_seq=0 out_seq=0 2024-05-21T22:05:38.061 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.058+0000 7ff02aa00700 10 client.17421 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T22:05:38.061 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.059+0000 7ff02aa00700 1 -- 172.21.15.151:0/1140201813 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a4473e85f0 con 0x55a447514900 2024-05-21T22:05:38.062 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.059+0000 7ff02aa00700 10 client.17421 handle_command_reply: tid=0 2024-05-21T22:05:38.062 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.059+0000 7ff056426b80 1 client.17421 shutdown 2024-05-21T22:05:38.062 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.059+0000 7ff056426b80 1 -- 172.21.15.151:0/1140201813 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a4474940a0 msgr2=0x55a44749e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:38.062 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.059+0000 7ff056426b80 1 --2- 172.21.15.151:0/1140201813 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a4474940a0 0x55a44749e440 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7ff0300aa330 tx=0x7ff0300e0fa0 comp rx=0 tx=0).stop 2024-05-21T22:05:38.062 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.060+0000 7ff056426b80 1 -- 172.21.15.151:0/1140201813 shutdown_connections 2024-05-21T22:05:38.062 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.060+0000 7ff056426b80 1 --2- 172.21.15.151:0/1140201813 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x55a447514900 0x55a447516cf0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:38.062 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.060+0000 7ff056426b80 1 --2- 172.21.15.151:0/1140201813 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55a44748eb50 0x55a44749df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:38.062 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.060+0000 7ff056426b80 1 --2- 172.21.15.151:0/1140201813 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4474949c0 0x55a44750cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:38.062 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.060+0000 7ff056426b80 1 --2- 172.21.15.151:0/1140201813 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a4474940a0 0x55a44749e440 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:38.062 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.060+0000 7ff02aa00700 0 client.17421 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T22:05:38.062 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.060+0000 7ff056426b80 1 -- 172.21.15.151:0/1140201813 >> 172.21.15.151:0/1140201813 conn(0x55a4473bcc60 msgr2=0x55a447496c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:05:38.062 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.060+0000 7ff056426b80 1 -- 172.21.15.151:0/1140201813 shutdown_connections 2024-05-21T22:05:38.062 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.060+0000 7ff056426b80 1 -- 172.21.15.151:0/1140201813 wait complete. 2024-05-21T22:05:38.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.060+0000 7ff056426b80 20 client.17421 trim_cache size 0 max 16384 2024-05-21T22:05:38.063 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-05-21T22:05:38.063 INFO:teuthology.orchestra.run.smithi151.stdout: "return_code": 0, 2024-05-21T22:05:38.063 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_tag": "df06f6da-d821-402b-9042-f7bc996f7d7c", 2024-05-21T22:05:38.063 INFO:teuthology.orchestra.run.smithi151.stdout: "mode": "asynchronous" 2024-05-21T22:05:38.063 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-05-21T22:05:38.064 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.061+0000 7ff04a400700 1 -- 172.21.15.151:0/3996058403 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7ff02c0b29f0 msgr2=0x7ff02c0b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:38.064 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.061+0000 7ff04a400700 1 --2- 172.21.15.151:0/3996058403 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7ff02c0b29f0 0x7ff02c0b4eb0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff03c008c10 tx=0x7ff03c009f90 comp rx=0 tx=0).stop 2024-05-21T22:05:38.064 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.061+0000 7ff04a400700 1 -- 172.21.15.151:0/3996058403 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff04410a440 msgr2=0x7ff0441a06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:38.064 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.061+0000 7ff04a400700 1 --2- 172.21.15.151:0/3996058403 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff04410a440 0x7ff0441a06f0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7ff030000c00 tx=0x7ff0300057c0 comp rx=0 tx=0).stop 2024-05-21T22:05:38.064 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.061+0000 7ff04a400700 1 -- 172.21.15.151:0/3996058403 shutdown_connections 2024-05-21T22:05:38.064 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.061+0000 7ff04a400700 1 --2- 172.21.15.151:0/3996058403 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7ff02c0b29f0 0x7ff02c0b4eb0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:38.064 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.061+0000 7ff04a400700 1 --2- 172.21.15.151:0/3996058403 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff04410e950 0x7ff0441a4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:38.064 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.061+0000 7ff04a400700 1 --2- 172.21.15.151:0/3996058403 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff044109a80 0x7ff0441a01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:38.064 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.061+0000 7ff04a400700 1 --2- 172.21.15.151:0/3996058403 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff04410a440 0x7ff0441a06f0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:38.064 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.061+0000 7ff04a400700 1 -- 172.21.15.151:0/3996058403 >> 172.21.15.151:0/3996058403 conn(0x7ff0440792d0 msgr2=0x7ff044079930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:05:38.064 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.061+0000 7ff04a400700 1 -- 172.21.15.151:0/3996058403 shutdown_connections 2024-05-21T22:05:38.065 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:38.061+0000 7ff04a400700 1 -- 172.21.15.151:0/3996058403 wait complete. 2024-05-21T22:05:38.927 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:05:38 smithi151 ceph-mon[31623]: scrub queued for path: / 2024-05-21T22:05:38.928 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:05:38 smithi151 ceph-mon[31623]: scrub summary: idle+waiting paths [/] 2024-05-21T22:05:38.928 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:05:38 smithi151 ceph-mon[31623]: scrub summary: active paths [/] 2024-05-21T22:05:38.928 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:05:38 smithi151 ceph-mon[31623]: scrub summary: idle+waiting paths [/] 2024-05-21T22:05:38.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:05:38 smithi184 ceph-mon[36942]: scrub queued for path: / 2024-05-21T22:05:38.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:05:38 smithi184 ceph-mon[36942]: scrub summary: idle+waiting paths [/] 2024-05-21T22:05:38.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:05:38 smithi184 ceph-mon[36942]: scrub summary: active paths [/] 2024-05-21T22:05:38.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:05:38 smithi184 ceph-mon[36942]: scrub summary: idle+waiting paths [/] 2024-05-21T22:05:38.993 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-21T22:05:39.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:05:38 smithi164 ceph-mon[36269]: scrub queued for path: / 2024-05-21T22:05:39.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:05:38 smithi164 ceph-mon[36269]: scrub summary: idle+waiting paths [/] 2024-05-21T22:05:39.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:05:38 smithi164 ceph-mon[36269]: scrub summary: active paths [/] 2024-05-21T22:05:39.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:05:38 smithi164 ceph-mon[36269]: scrub summary: idle+waiting paths [/] 2024-05-21T22:05:39.470 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:05:39.948 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:05:39 smithi151 ceph-mon[31623]: pgmap v310: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T22:05:39.948 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:05:39 smithi151 ceph-mon[31623]: scrub summary: idle 2024-05-21T22:05:39.985 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:05:39 smithi184 ceph-mon[36942]: pgmap v310: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T22:05:39.985 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:05:39 smithi184 ceph-mon[36942]: scrub summary: idle 2024-05-21T22:05:40.037 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:05:39 smithi164 ceph-mon[36269]: pgmap v310: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T22:05:40.038 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:05:39 smithi164 ceph-mon[36269]: scrub summary: idle 2024-05-21T22:05:41.659 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.658+0000 7f8cf7200700 1 -- 172.21.15.151:0/2672152264 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8cf8109540 msgr2=0x7f8cf8109920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:41.659 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.658+0000 7f8cf7200700 1 --2- 172.21.15.151:0/2672152264 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8cf8109540 0x7f8cf8109920 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f8ce80099c0 tx=0x7f8ce8009cd0 comp rx=0 tx=0).stop 2024-05-21T22:05:41.659 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.658+0000 7f8cf7200700 1 -- 172.21.15.151:0/2672152264 shutdown_connections 2024-05-21T22:05:41.659 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.658+0000 7f8cf7200700 1 --2- 172.21.15.151:0/2672152264 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8cf8109540 0x7f8cf8109920 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:41.659 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.658+0000 7f8cf7200700 1 --2- 172.21.15.151:0/2672152264 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8cf8103e50 0x7f8cf8110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:41.660 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.658+0000 7f8cf7200700 1 --2- 172.21.15.151:0/2672152264 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8cf8103490 0x7f8cf8103910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:41.660 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.658+0000 7f8cf7200700 1 -- 172.21.15.151:0/2672152264 >> 172.21.15.151:0/2672152264 conn(0x7f8cf80fefc0 msgr2=0x7f8cf81013e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:05:41.660 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.658+0000 7f8cf7200700 1 -- 172.21.15.151:0/2672152264 shutdown_connections 2024-05-21T22:05:41.660 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.658+0000 7f8cf7200700 1 -- 172.21.15.151:0/2672152264 wait complete. 2024-05-21T22:05:41.660 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.659+0000 7f8cf7200700 1 Processor -- start 2024-05-21T22:05:41.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.660+0000 7f8cf7200700 1 -- start start 2024-05-21T22:05:41.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.660+0000 7f8cf7200700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8cf8103490 0x7f8cf8075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:41.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.660+0000 7f8cf7200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8cf8103e50 0x7f8cf8075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:41.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.660+0000 7f8cf7200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8cf8109540 0x7f8cf80762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:41.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.660+0000 7f8cf7200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf81136a0 con 0x7f8cf8109540 2024-05-21T22:05:41.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.660+0000 7f8cf7200700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf8113520 con 0x7f8cf8103e50 2024-05-21T22:05:41.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.660+0000 7f8cf7200700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf8113820 con 0x7f8cf8103490 2024-05-21T22:05:41.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.660+0000 7f8cf5e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8cf8103490 0x7f8cf8075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:41.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.660+0000 7f8cf5e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8cf8103490 0x7f8cf8075840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:37986/0 (socket says 172.21.15.151:37986) 2024-05-21T22:05:41.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.660+0000 7f8cf5e00700 1 -- 172.21.15.151:0/1895577038 learned_addr learned my addr 172.21.15.151:0/1895577038 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:05:41.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.660+0000 7f8cf5400700 1 --2- 172.21.15.151:0/1895577038 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8cf8103e50 0x7f8cf8075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:41.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.660+0000 7f8cf6800700 1 --2- 172.21.15.151:0/1895577038 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8cf8109540 0x7f8cf80762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:41.662 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.661+0000 7f8cf5e00700 1 -- 172.21.15.151:0/1895577038 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8cf8103e50 msgr2=0x7f8cf8075d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:41.662 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.661+0000 7f8cf5e00700 1 --2- 172.21.15.151:0/1895577038 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8cf8103e50 0x7f8cf8075d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:41.662 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.661+0000 7f8cf5e00700 1 -- 172.21.15.151:0/1895577038 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8cf8109540 msgr2=0x7f8cf80762c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:41.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.661+0000 7f8cf5e00700 1 --2- 172.21.15.151:0/1895577038 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8cf8109540 0x7f8cf80762c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:41.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.661+0000 7f8cf5e00700 1 -- 172.21.15.151:0/1895577038 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cf81a8040 con 0x7f8cf8103490 2024-05-21T22:05:41.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.661+0000 7f8cf6800700 1 --2- 172.21.15.151:0/1895577038 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8cf8109540 0x7f8cf80762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T22:05:41.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.661+0000 7f8cf5400700 1 --2- 172.21.15.151:0/1895577038 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8cf8103e50 0x7f8cf8075d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:05:41.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.661+0000 7f8cf5e00700 1 --2- 172.21.15.151:0/1895577038 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8cf8103490 0x7f8cf8075840 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f8ce8005910 tx=0x7f8ce8004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:05:41.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.661+0000 7f8ceea00700 1 -- 172.21.15.151:0/1895577038 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce8026070 con 0x7f8cf8103490 2024-05-21T22:05:41.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.662+0000 7f8ceea00700 1 -- 172.21.15.151:0/1895577038 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ce8003c70 con 0x7f8cf8103490 2024-05-21T22:05:41.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.662+0000 7f8ceea00700 1 -- 172.21.15.151:0/1895577038 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce800fb50 con 0x7f8cf8103490 2024-05-21T22:05:41.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.662+0000 7f8cf7200700 1 -- 172.21.15.151:0/1895577038 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cf81a82d0 con 0x7f8cf8103490 2024-05-21T22:05:41.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.662+0000 7f8cf7200700 1 -- 172.21.15.151:0/1895577038 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cf81a86d0 con 0x7f8cf8103490 2024-05-21T22:05:41.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.665+0000 7f8ceea00700 1 -- 172.21.15.151:0/1895577038 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7f8ce8029070 con 0x7f8cf8103490 2024-05-21T22:05:41.667 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.667+0000 7f8d07719b80 1 Processor -- start 2024-05-21T22:05:41.667 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.667+0000 7f8d07719b80 1 -- start start 2024-05-21T22:05:41.668 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.668+0000 7f8d07719b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5561d4ffa7c0 0x5561d4ffaba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:41.668 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.668+0000 7f8d07719b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561d4ffb0e0 0x5561d4ff4d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:41.668 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.668+0000 7f8d07719b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561d4ff5390 0x5561d4ff57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:41.668 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.668+0000 7f8d07719b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5561d4f52e40 con 0x5561d4ff5390 2024-05-21T22:05:41.668 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.668+0000 7f8d07719b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5561d4f429e0 con 0x5561d4ffa7c0 2024-05-21T22:05:41.668 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.668+0000 7f8d07719b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5561d4f46a20 con 0x5561d4ffb0e0 2024-05-21T22:05:41.669 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.668+0000 7f8ceea00700 1 --2- 172.21.15.151:0/1895577038 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f8cdc0b29f0 0x7f8cdc0b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:41.669 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.668+0000 7f8cf5400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561d4ff5390 0x5561d4ff57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:41.669 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.668+0000 7f8cf5e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561d4ffb0e0 0x5561d4ff4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:41.669 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.668+0000 7f8cf6800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5561d4ffa7c0 0x5561d4ffaba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:41.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.668+0000 7f8cf5400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561d4ff5390 0x5561d4ff57f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:53284/0 (socket says 172.21.15.151:53284) 2024-05-21T22:05:41.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.668+0000 7f8cf5400700 1 -- 172.21.15.151:0/2308169840 learned_addr learned my addr 172.21.15.151:0/2308169840 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:05:41.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.669+0000 7f8cf6800700 1 --2- 172.21.15.151:0/1895577038 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f8cdc0b29f0 0x7f8cdc0b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:41.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.669+0000 7f8ceea00700 1 -- 172.21.15.151:0/1895577038 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7f8ce80e2f10 con 0x7f8cf8103490 2024-05-21T22:05:41.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.669+0000 7f8cf5400700 1 -- 172.21.15.151:0/2308169840 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561d4ffb0e0 msgr2=0x5561d4ff4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:41.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.669+0000 7f8cf5400700 1 --2- 172.21.15.151:0/2308169840 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561d4ffb0e0 0x5561d4ff4d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:41.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.669+0000 7f8cf5400700 1 -- 172.21.15.151:0/2308169840 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5561d4ffa7c0 msgr2=0x5561d4ffaba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:41.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.669+0000 7f8cf5400700 1 --2- 172.21.15.151:0/2308169840 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5561d4ffa7c0 0x5561d4ffaba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:41.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.669+0000 7f8cf5400700 1 -- 172.21.15.151:0/2308169840 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561d4f0b420 con 0x5561d4ff5390 2024-05-21T22:05:41.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.669+0000 7f8cf5400700 1 --2- 172.21.15.151:0/2308169840 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561d4ff5390 0x5561d4ff57f0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f8ce4011540 tx=0x7f8ce4011850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47b783a9236cda03 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:05:41.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.670+0000 7f8ce3400700 1 -- 172.21.15.151:0/2308169840 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce4004a30 con 0x5561d4ff5390 2024-05-21T22:05:41.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.670+0000 7f8cf6800700 1 --2- 172.21.15.151:0/1895577038 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f8cdc0b29f0 0x7f8cdc0b4eb0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8cf0009fd0 tx=0x7f8cf000b040 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:05:41.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.670+0000 7f8ce3400700 1 -- 172.21.15.151:0/2308169840 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ce40054d0 con 0x5561d4ff5390 2024-05-21T22:05:41.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.670+0000 7f8ce3400700 1 -- 172.21.15.151:0/2308169840 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce400e890 con 0x5561d4ff5390 2024-05-21T22:05:41.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.671+0000 7f8d07719b80 1 -- 172.21.15.151:0/2308169840 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561d4ff5390 msgr2=0x5561d4ff57f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:41.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.671+0000 7f8d07719b80 1 --2- 172.21.15.151:0/2308169840 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561d4ff5390 0x5561d4ff57f0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f8ce4011540 tx=0x7f8ce4011850 comp rx=0 tx=0).stop 2024-05-21T22:05:41.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.671+0000 7f8d07719b80 1 -- 172.21.15.151:0/2308169840 shutdown_connections 2024-05-21T22:05:41.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.671+0000 7f8d07719b80 1 --2- 172.21.15.151:0/2308169840 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5561d4ffa7c0 0x5561d4ffaba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:41.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.671+0000 7f8d07719b80 1 --2- 172.21.15.151:0/2308169840 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561d4ff5390 0x5561d4ff57f0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:41.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.671+0000 7f8d07719b80 1 --2- 172.21.15.151:0/2308169840 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561d4ffb0e0 0x5561d4ff4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:41.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.671+0000 7f8d07719b80 1 -- 172.21.15.151:0/2308169840 >> 172.21.15.151:0/2308169840 conn(0x5561d4f22be0 msgr2=0x5561d4fef810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:05:41.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.671+0000 7f8d07719b80 1 -- 172.21.15.151:0/2308169840 shutdown_connections 2024-05-21T22:05:41.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.671+0000 7f8d07719b80 1 -- 172.21.15.151:0/2308169840 wait complete. 2024-05-21T22:05:41.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.672+0000 7f8d07719b80 1 -- start start 2024-05-21T22:05:41.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.672+0000 7f8d07719b80 1 Processor -- start 2024-05-21T22:05:41.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.672+0000 7f8d07719b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561d4ff5390 0x5561d5000d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:41.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.672+0000 7f8d07719b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561d4ffa7c0 0x5561d4ffe120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:41.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.672+0000 7f8d07719b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5561d4ffb0e0 0x5561d4ffe660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:41.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.672+0000 7f8d07719b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5561d4e48fa0 con 0x5561d4ffa7c0 2024-05-21T22:05:41.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.672+0000 7f8d07719b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5561d4ee3760 con 0x5561d4ffb0e0 2024-05-21T22:05:41.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.672+0000 7f8d07719b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5561d5001c00 con 0x5561d4ff5390 2024-05-21T22:05:41.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.673+0000 7f8d07719b80 10 client.0 _ll_register_callbacks cb 0x5561d4eecc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:05:41.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.673+0000 7f8cf5400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561d4ffa7c0 0x5561d4ffe120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:41.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.673+0000 7f8cf5e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561d4ff5390 0x5561d5000d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:41.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.673+0000 7f8cf6800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5561d4ffb0e0 0x5561d4ffe660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:41.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.673+0000 7f8cf5e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561d4ff5390 0x5561d5000d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:37998/0 (socket says 172.21.15.151:37998) 2024-05-21T22:05:41.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.673+0000 7f8cf5e00700 1 -- 172.21.15.151:0/788646899 learned_addr learned my addr 172.21.15.151:0/788646899 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:05:41.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.673+0000 7f8cf5e00700 1 -- 172.21.15.151:0/788646899 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5561d4ffb0e0 msgr2=0x5561d4ffe660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:41.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.673+0000 7f8cf5e00700 1 --2- 172.21.15.151:0/788646899 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5561d4ffb0e0 0x5561d4ffe660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:41.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.673+0000 7f8cf5e00700 1 -- 172.21.15.151:0/788646899 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561d4ffa7c0 msgr2=0x5561d4ffe120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:41.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.673+0000 7f8cf5e00700 1 --2- 172.21.15.151:0/788646899 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561d4ffa7c0 0x5561d4ffe120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:41.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.673+0000 7f8cf5e00700 1 -- 172.21.15.151:0/788646899 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561d4e928d0 con 0x5561d4ff5390 2024-05-21T22:05:41.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.673+0000 7f8cf6800700 1 --2- 172.21.15.151:0/788646899 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5561d4ffb0e0 0x5561d4ffe660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:05:41.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.674+0000 7f8cf5e00700 1 --2- 172.21.15.151:0/788646899 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561d4ff5390 0x5561d5000d80 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f8ce80a6290 tx=0x7f8ce8073ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6ef433df0958678 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:05:41.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.674+0000 7f8cd7e00700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-21T22:05:41.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.674+0000 7f8cd7e00700 1 -- 172.21.15.151:0/788646899 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce80a9b60 con 0x5561d4ff5390 2024-05-21T22:05:41.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.674+0000 7f8d07719b80 1 -- 172.21.15.151:0/788646899 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ce40054d0 con 0x5561d4ff5390 2024-05-21T22:05:41.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.674+0000 7f8cd7e00700 1 -- 172.21.15.151:0/788646899 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7f8ce80b2d60 con 0x5561d4ff5390 2024-05-21T22:05:41.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.674+0000 7f8cd7e00700 1 -- 172.21.15.151:0/788646899 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8ce8009e60 con 0x5561d4ff5390 2024-05-21T22:05:41.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.674+0000 7f8d07719b80 10 client.27565 fetch_fsmap learned FSMap version 29 2024-05-21T22:05:41.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.674+0000 7f8d07719b80 1 -- 172.21.15.151:0/788646899 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5561d4f13060 con 0x5561d4ff5390 2024-05-21T22:05:41.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.674+0000 7f8cd7e00700 1 -- 172.21.15.151:0/788646899 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ce40054d0 con 0x5561d4ff5390 2024-05-21T22:05:41.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.675+0000 7f8cd7e00700 1 -- 172.21.15.151:0/788646899 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f8ce8076bb0 con 0x5561d4ff5390 2024-05-21T22:05:41.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.675+0000 7f8d07719b80 10 client.27565 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:05:41.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.675+0000 7f8d07719b80 10 client.27565 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T22:05:41.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.675+0000 7f8d07719b80 20 client.27565 populate_metadata read hostname 'smithi151' 2024-05-21T22:05:41.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.675+0000 7f8d07719b80 1 --2- 172.21.15.151:0/788646899 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5561d500e610 0x5561d5010a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:41.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.675+0000 7f8d07719b80 4 client.27565 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T22:05:41.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.675+0000 7f8d07719b80 1 -- 172.21.15.151:0/788646899 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5561d4f54d90 con 0x5561d500e610 2024-05-21T22:05:41.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.675+0000 7f8cf5400700 1 --2- 172.21.15.151:0/788646899 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5561d500e610 0x5561d5010a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:41.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.676+0000 7f8cf5400700 1 --2- 172.21.15.151:0/788646899 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5561d500e610 0x5561d5010a00 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a734778f2e5b7509 server_cookie=428c455284068c35 in_seq=0 out_seq=0 2024-05-21T22:05:41.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.676+0000 7f8cd7e00700 10 client.27565 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T22:05:41.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.677+0000 7f8cd7e00700 1 -- 172.21.15.151:0/788646899 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5561d4f54d90 con 0x5561d500e610 2024-05-21T22:05:41.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.677+0000 7f8cd7e00700 10 client.27565 handle_command_reply: tid=0 2024-05-21T22:05:41.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.677+0000 7f8d07719b80 1 client.27565 shutdown 2024-05-21T22:05:41.679 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.677+0000 7f8d07719b80 1 -- 172.21.15.151:0/788646899 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561d4ff5390 msgr2=0x5561d5000d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:41.679 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.677+0000 7f8d07719b80 1 --2- 172.21.15.151:0/788646899 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561d4ff5390 0x5561d5000d80 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f8ce80a6290 tx=0x7f8ce8073ab0 comp rx=0 tx=0).stop 2024-05-21T22:05:41.679 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.677+0000 7f8d07719b80 1 -- 172.21.15.151:0/788646899 shutdown_connections 2024-05-21T22:05:41.679 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.677+0000 7f8d07719b80 1 --2- 172.21.15.151:0/788646899 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5561d500e610 0x5561d5010a00 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:41.679 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.677+0000 7f8d07719b80 1 --2- 172.21.15.151:0/788646899 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5561d4ffb0e0 0x5561d4ffe660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:41.679 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.677+0000 7f8d07719b80 1 --2- 172.21.15.151:0/788646899 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561d4ff5390 0x5561d5000d80 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:41.679 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.677+0000 7f8d07719b80 1 --2- 172.21.15.151:0/788646899 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561d4ffa7c0 0x5561d4ffe120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:41.679 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.677+0000 7f8cd7e00700 0 client.27565 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T22:05:41.679 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.677+0000 7f8d07719b80 1 -- 172.21.15.151:0/788646899 >> 172.21.15.151:0/788646899 conn(0x5561d4f22be0 msgr2=0x5561d4feefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:05:41.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.677+0000 7f8d07719b80 1 -- 172.21.15.151:0/788646899 shutdown_connections 2024-05-21T22:05:41.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.677+0000 7f8d07719b80 1 -- 172.21.15.151:0/788646899 wait complete. 2024-05-21T22:05:41.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.677+0000 7f8d07719b80 20 client.27565 trim_cache size 0 max 16384 2024-05-21T22:05:41.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.700+0000 7f8d07719b80 1 Processor -- start 2024-05-21T22:05:41.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.700+0000 7f8d07719b80 1 -- start start 2024-05-21T22:05:41.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.700+0000 7f8d07719b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5561d4ff5390 0x5561d4ff4d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:41.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.700+0000 7f8d07719b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561d4ffa7c0 0x5561d4ffbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:41.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.700+0000 7f8d07719b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561d4ffb0e0 0x5561d4fef740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:41.701 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.700+0000 7f8d07719b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf81afdd0 con 0x5561d4ffb0e0 2024-05-21T22:05:41.701 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.700+0000 7f8d07719b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf81b1910 con 0x5561d4ff5390 2024-05-21T22:05:41.701 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.700+0000 7f8d07719b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf81b04a0 con 0x5561d4ffa7c0 2024-05-21T22:05:41.701 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.701+0000 7f8cf6800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561d4ffb0e0 0x5561d4fef740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:41.702 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.701+0000 7f8cf5400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561d4ffa7c0 0x5561d4ffbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:41.702 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.701+0000 7f8cf5400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561d4ffa7c0 0x5561d4ffbbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:38010/0 (socket says 172.21.15.151:38010) 2024-05-21T22:05:41.702 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.701+0000 7f8cf5400700 1 -- 172.21.15.151:0/894557788 learned_addr learned my addr 172.21.15.151:0/894557788 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:05:41.702 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.701+0000 7f8cf6800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561d4ffb0e0 0x5561d4fef740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:53298/0 (socket says 172.21.15.151:53298) 2024-05-21T22:05:41.702 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.701+0000 7f8cf5e00700 1 --2- 172.21.15.151:0/894557788 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5561d4ff5390 0x5561d4ff4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:41.702 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.701+0000 7f8cf5400700 1 -- 172.21.15.151:0/894557788 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5561d4ff5390 msgr2=0x5561d4ff4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:41.702 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.701+0000 7f8cf5400700 1 --2- 172.21.15.151:0/894557788 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5561d4ff5390 0x5561d4ff4d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:41.702 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.701+0000 7f8cf5400700 1 -- 172.21.15.151:0/894557788 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561d4ffb0e0 msgr2=0x5561d4fef740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:41.702 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.701+0000 7f8cf5400700 1 --2- 172.21.15.151:0/894557788 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561d4ffb0e0 0x5561d4fef740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:41.702 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.701+0000 7f8cf5400700 1 -- 172.21.15.151:0/894557788 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561d4ff04b0 con 0x5561d4ffa7c0 2024-05-21T22:05:41.702 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.701+0000 7f8cf5400700 1 --2- 172.21.15.151:0/894557788 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561d4ffa7c0 0x5561d4ffbbd0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f8ce4011460 tx=0x7f8ce40058e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4df073a292e2f9b server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:05:41.703 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.702+0000 7f8ce2a00700 1 -- 172.21.15.151:0/894557788 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce4005110 con 0x5561d4ffa7c0 2024-05-21T22:05:41.703 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.702+0000 7f8ce2a00700 1 -- 172.21.15.151:0/894557788 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ce400ec60 con 0x5561d4ffa7c0 2024-05-21T22:05:41.703 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.702+0000 7f8ce2a00700 1 -- 172.21.15.151:0/894557788 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce4003680 con 0x5561d4ffa7c0 2024-05-21T22:05:41.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.703+0000 7f8d07719b80 1 -- 172.21.15.151:0/894557788 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561d4ffa7c0 msgr2=0x5561d4ffbbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:41.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.703+0000 7f8d07719b80 1 --2- 172.21.15.151:0/894557788 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561d4ffa7c0 0x5561d4ffbbd0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f8ce4011460 tx=0x7f8ce40058e0 comp rx=0 tx=0).stop 2024-05-21T22:05:41.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.703+0000 7f8d07719b80 1 -- 172.21.15.151:0/894557788 shutdown_connections 2024-05-21T22:05:41.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.703+0000 7f8d07719b80 1 --2- 172.21.15.151:0/894557788 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5561d4ff5390 0x5561d4ff4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:41.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.703+0000 7f8d07719b80 1 --2- 172.21.15.151:0/894557788 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561d4ffb0e0 0x5561d4fef740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:41.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.703+0000 7f8d07719b80 1 --2- 172.21.15.151:0/894557788 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561d4ffa7c0 0x5561d4ffbbd0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:41.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.703+0000 7f8d07719b80 1 -- 172.21.15.151:0/894557788 >> 172.21.15.151:0/894557788 conn(0x5561d4f22be0 msgr2=0x5561d4ffb4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:05:41.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.703+0000 7f8d07719b80 1 -- 172.21.15.151:0/894557788 shutdown_connections 2024-05-21T22:05:41.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.703+0000 7f8d07719b80 1 -- 172.21.15.151:0/894557788 wait complete. 2024-05-21T22:05:41.705 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.704+0000 7f8d07719b80 1 -- start start 2024-05-21T22:05:41.705 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.704+0000 7f8d07719b80 1 Processor -- start 2024-05-21T22:05:41.705 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.705+0000 7f8d07719b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5561d4ff5390 0x5561d5006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:41.705 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.705+0000 7f8d07719b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561d4ffa7c0 0x5561d50073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:41.705 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.705+0000 7f8d07719b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561d4ffb0e0 0x5561d5003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:41.705 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.705+0000 7f8d07719b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf81b1240 con 0x5561d4ffa7c0 2024-05-21T22:05:41.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.705+0000 7f8d07719b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf81b0b70 con 0x5561d4ff5390 2024-05-21T22:05:41.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.705+0000 7f8d07719b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf81add90 con 0x5561d4ffb0e0 2024-05-21T22:05:41.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.705+0000 7f8d07719b80 10 client.0 _ll_register_callbacks cb 0x5561d4eecc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:05:41.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.705+0000 7f8cf5e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5561d4ff5390 0x5561d5006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:41.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.705+0000 7f8cf5e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5561d4ff5390 0x5561d5006e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:48774/0 (socket says 172.21.15.151:48774) 2024-05-21T22:05:41.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.705+0000 7f8cf5e00700 1 -- 172.21.15.151:0/962542397 learned_addr learned my addr 172.21.15.151:0/962542397 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:05:41.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.705+0000 7f8cf5400700 1 --2- 172.21.15.151:0/962542397 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561d4ffa7c0 0x5561d50073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:41.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.705+0000 7f8cf6800700 1 --2- 172.21.15.151:0/962542397 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561d4ffb0e0 0x5561d5003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:41.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.706+0000 7f8cf5e00700 1 -- 172.21.15.151:0/962542397 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561d4ffb0e0 msgr2=0x5561d5003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:41.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.706+0000 7f8cf5e00700 1 --2- 172.21.15.151:0/962542397 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561d4ffb0e0 0x5561d5003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:41.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.706+0000 7f8cf5e00700 1 -- 172.21.15.151:0/962542397 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561d4ffa7c0 msgr2=0x5561d50073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:41.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.706+0000 7f8cf5e00700 1 --2- 172.21.15.151:0/962542397 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561d4ffa7c0 0x5561d50073d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:41.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.706+0000 7f8cf5e00700 1 -- 172.21.15.151:0/962542397 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561d50043d0 con 0x5561d4ff5390 2024-05-21T22:05:41.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.706+0000 7f8cf6800700 1 --2- 172.21.15.151:0/962542397 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561d4ffb0e0 0x5561d5003e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:05:41.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.706+0000 7f8cf5e00700 1 --2- 172.21.15.151:0/962542397 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5561d4ff5390 0x5561d5006e90 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f8ce80a67b0 tx=0x7f8ce80b3060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=150c6dcf71c943b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:05:41.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.706+0000 7f8cd7e00700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-21T22:05:41.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.707+0000 7f8cd7e00700 1 -- 172.21.15.151:0/962542397 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce80a9980 con 0x5561d4ff5390 2024-05-21T22:05:41.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.707+0000 7f8cd7e00700 1 -- 172.21.15.151:0/962542397 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7f8ce80ae310 con 0x5561d4ff5390 2024-05-21T22:05:41.708 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.707+0000 7f8d07719b80 1 -- 172.21.15.151:0/962542397 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ce400ec60 con 0x5561d4ff5390 2024-05-21T22:05:41.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.708+0000 7f8cd7e00700 1 -- 172.21.15.151:0/962542397 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8ce80a9b60 con 0x5561d4ff5390 2024-05-21T22:05:41.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.708+0000 7f8d07719b80 10 client.27566 fetch_fsmap learned FSMap version 29 2024-05-21T22:05:41.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.708+0000 7f8d07719b80 1 -- 172.21.15.151:0/962542397 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5561d5079a10 con 0x5561d4ff5390 2024-05-21T22:05:41.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.709+0000 7f8cd7e00700 1 -- 172.21.15.151:0/962542397 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ce400ec60 con 0x5561d4ff5390 2024-05-21T22:05:41.710 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.709+0000 7f8cd7e00700 1 -- 172.21.15.151:0/962542397 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f8ce80a9600 con 0x5561d4ff5390 2024-05-21T22:05:41.710 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.709+0000 7f8d07719b80 10 client.27566 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:05:41.710 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.709+0000 7f8d07719b80 10 client.27566 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T22:05:41.710 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.709+0000 7f8d07719b80 20 client.27566 populate_metadata read hostname 'smithi151' 2024-05-21T22:05:41.710 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.709+0000 7f8d07719b80 1 --2- 172.21.15.151:0/962542397 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5561d5079ec0 0x5561d507c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:41.710 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.709+0000 7f8d07719b80 4 client.27566 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-21T22:05:41.710 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.709+0000 7f8d07719b80 1 -- 172.21.15.151:0/962542397 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5561d4f4e570 con 0x5561d5079ec0 2024-05-21T22:05:41.710 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.710+0000 7f8cf5400700 1 --2- 172.21.15.151:0/962542397 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5561d5079ec0 0x5561d507c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:41.712 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.710+0000 7f8cf5400700 1 --2- 172.21.15.151:0/962542397 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5561d5079ec0 0x5561d507c2b0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=787f7f1792e58723 server_cookie=d92bbd01a86017d5 in_seq=0 out_seq=0 2024-05-21T22:05:41.712 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.710+0000 7f8cd7e00700 10 client.27566 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T22:05:41.712 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.710+0000 7f8cd7e00700 1 -- 172.21.15.151:0/962542397 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5561d4f4e570 con 0x5561d5079ec0 2024-05-21T22:05:41.712 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.710+0000 7f8cd7e00700 10 client.27566 handle_command_reply: tid=0 2024-05-21T22:05:41.712 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.710+0000 7f8d07719b80 1 client.27566 shutdown 2024-05-21T22:05:41.712 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.711+0000 7f8d07719b80 1 -- 172.21.15.151:0/962542397 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5561d4ff5390 msgr2=0x5561d5006e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:41.712 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.711+0000 7f8d07719b80 1 --2- 172.21.15.151:0/962542397 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5561d4ff5390 0x5561d5006e90 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f8ce80a67b0 tx=0x7f8ce80b3060 comp rx=0 tx=0).stop 2024-05-21T22:05:41.712 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.711+0000 7f8d07719b80 1 -- 172.21.15.151:0/962542397 shutdown_connections 2024-05-21T22:05:41.712 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.711+0000 7f8d07719b80 1 --2- 172.21.15.151:0/962542397 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5561d5079ec0 0x5561d507c2b0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:41.712 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.711+0000 7f8d07719b80 1 --2- 172.21.15.151:0/962542397 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5561d4ff5390 0x5561d5006e90 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:41.712 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.711+0000 7f8d07719b80 1 --2- 172.21.15.151:0/962542397 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5561d4ffb0e0 0x5561d5003e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:41.712 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.711+0000 7f8d07719b80 1 --2- 172.21.15.151:0/962542397 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5561d4ffa7c0 0x5561d50073d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:41.712 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.711+0000 7f8d07719b80 1 -- 172.21.15.151:0/962542397 >> 172.21.15.151:0/962542397 conn(0x5561d4f22be0 msgr2=0x5561d4ff9260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:05:41.713 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.711+0000 7f8cd7e00700 0 client.27566 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T22:05:41.713 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.711+0000 7f8d07719b80 1 -- 172.21.15.151:0/962542397 shutdown_connections 2024-05-21T22:05:41.713 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.711+0000 7f8d07719b80 1 -- 172.21.15.151:0/962542397 wait complete. 2024-05-21T22:05:41.713 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.711+0000 7f8d07719b80 20 client.27566 trim_cache size 0 max 16384 2024-05-21T22:05:41.713 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-05-21T22:05:41.713 INFO:teuthology.orchestra.run.smithi151.stdout: "status": "no active scrubs running", 2024-05-21T22:05:41.713 INFO:teuthology.orchestra.run.smithi151.stdout: "scrubs": {} 2024-05-21T22:05:41.714 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-05-21T22:05:41.714 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.712+0000 7f8cf7200700 1 -- 172.21.15.151:0/1895577038 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f8cdc0b29f0 msgr2=0x7f8cdc0b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:41.714 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.712+0000 7f8cf7200700 1 --2- 172.21.15.151:0/1895577038 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f8cdc0b29f0 0x7f8cdc0b4eb0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8cf0009fd0 tx=0x7f8cf000b040 comp rx=0 tx=0).stop 2024-05-21T22:05:41.714 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.712+0000 7f8cf7200700 1 -- 172.21.15.151:0/1895577038 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8cf8103490 msgr2=0x7f8cf8075840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:41.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.712+0000 7f8cf7200700 1 --2- 172.21.15.151:0/1895577038 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8cf8103490 0x7f8cf8075840 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f8ce8005910 tx=0x7f8ce8004270 comp rx=0 tx=0).stop 2024-05-21T22:05:41.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.712+0000 7f8cf7200700 1 -- 172.21.15.151:0/1895577038 shutdown_connections 2024-05-21T22:05:41.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.712+0000 7f8cf7200700 1 --2- 172.21.15.151:0/1895577038 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f8cdc0b29f0 0x7f8cdc0b4eb0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:41.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.712+0000 7f8cf7200700 1 --2- 172.21.15.151:0/1895577038 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8cf8109540 0x7f8cf80762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:41.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.712+0000 7f8cf7200700 1 --2- 172.21.15.151:0/1895577038 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8cf8103490 0x7f8cf8075840 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:41.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.712+0000 7f8cf7200700 1 --2- 172.21.15.151:0/1895577038 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8cf8103e50 0x7f8cf8075d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:41.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.712+0000 7f8cf7200700 1 -- 172.21.15.151:0/1895577038 >> 172.21.15.151:0/1895577038 conn(0x7f8cf80fefc0 msgr2=0x7f8cf8100750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:05:41.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.713+0000 7f8cf7200700 1 -- 172.21.15.151:0/1895577038 shutdown_connections 2024-05-21T22:05:41.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:41.713+0000 7f8cf7200700 1 -- 172.21.15.151:0/1895577038 wait complete. 2024-05-21T22:05:41.751 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:05:41 smithi151 ceph-mon[31623]: pgmap v311: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:05:41.985 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:05:41 smithi184 ceph-mon[36942]: pgmap v311: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:05:42.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:05:41 smithi164 ceph-mon[36269]: pgmap v311: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:05:42.682 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-21T22:05:42.682 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-21T22:05:43.110 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:05:43.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:05:43 smithi151 ceph-mon[31623]: pgmap v312: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-05-21T22:05:43.985 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:05:43 smithi184 ceph-mon[36942]: pgmap v312: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-05-21T22:05:44.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:05:43 smithi164 ceph-mon[36269]: pgmap v312: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-05-21T22:05:45.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:45.339+0000 7f6569c00700 1 -- 172.21.15.151:0/1071047431 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f656410eeb0 msgr2=0x7f6564116a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:45.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:45.339+0000 7f6569c00700 1 --2- 172.21.15.151:0/1071047431 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f656410eeb0 0x7f6564116a80 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f655c009fc0 tx=0x7f655c00b330 comp rx=0 tx=0).stop 2024-05-21T22:05:45.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:45.339+0000 7f6569c00700 1 -- 172.21.15.151:0/1071047431 shutdown_connections 2024-05-21T22:05:45.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:45.339+0000 7f6569c00700 1 --2- 172.21.15.151:0/1071047431 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f656410eeb0 0x7f6564116a80 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:45.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:45.339+0000 7f6569c00700 1 --2- 172.21.15.151:0/1071047431 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6564109dd0 0x7f656410a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:45.345 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:45.339+0000 7f6569c00700 1 --2- 172.21.15.151:0/1071047431 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f656410a780 0x7f656410e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:45.345 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:45.340+0000 7f6569c00700 1 -- 172.21.15.151:0/1071047431 >> 172.21.15.151:0/1071047431 conn(0x7f6564076d10 msgr2=0x7f6564108360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:05:45.345 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:45.340+0000 7f6569c00700 1 -- 172.21.15.151:0/1071047431 shutdown_connections 2024-05-21T22:05:45.345 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:45.340+0000 7f6569c00700 1 -- 172.21.15.151:0/1071047431 wait complete. 2024-05-21T22:05:45.345 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:45.341+0000 7f6569c00700 1 Processor -- start 2024-05-21T22:05:45.345 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:45.341+0000 7f6569c00700 1 -- start start 2024-05-21T22:05:45.345 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:45.342+0000 7f6569c00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6564109dd0 0x7f65641a0240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:45.345 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:45.342+0000 7f6569c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f656410a780 0x7f65641a0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:45.345 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:45.342+0000 7f6569c00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f656410eeb0 0x7f65641a4b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:45.345 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:45.342+0000 7f6569c00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6564119db0 con 0x7f656410a780 2024-05-21T22:05:45.345 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:45.342+0000 7f6569c00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6564119c30 con 0x7f656410eeb0 2024-05-21T22:05:45.345 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:45.342+0000 7f6569c00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f6564119f30 con 0x7f6564109dd0 2024-05-21T22:05:45.345 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:45.342+0000 7f6562a00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6564109dd0 0x7f65641a0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:45.345 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:45.342+0000 7f6562000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f656410a780 0x7f65641a0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:45.346 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:45.342+0000 7f6562a00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6564109dd0 0x7f65641a0240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:38038/0 (socket says 172.21.15.151:38038) 2024-05-21T22:05:45.346 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:45.342+0000 7f6562a00700 1 -- 172.21.15.151:0/1451069679 learned_addr learned my addr 172.21.15.151:0/1451069679 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:05:45.346 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:45.342+0000 7f6563400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f656410eeb0 0x7f65641a4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:45.346 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:45.342+0000 7f6562a00700 1 -- 172.21.15.151:0/1451069679 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f656410eeb0 msgr2=0x7f65641a4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:45.346 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:45.342+0000 7f6562a00700 1 --2- 172.21.15.151:0/1451069679 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f656410eeb0 0x7f65641a4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:45.346 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:45.342+0000 7f6562a00700 1 -- 172.21.15.151:0/1451069679 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f656410a780 msgr2=0x7f65641a0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:45.346 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:45.342+0000 7f6562a00700 1 --2- 172.21.15.151:0/1451069679 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f656410a780 0x7f65641a0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:45.346 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:45.342+0000 7f6562a00700 1 -- 172.21.15.151:0/1451069679 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65641a5250 con 0x7f6564109dd0 2024-05-21T22:05:45.346 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:45.342+0000 7f6563400700 1 --2- 172.21.15.151:0/1451069679 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f656410eeb0 0x7f65641a4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:05:45.346 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:45.343+0000 7f6562a00700 1 --2- 172.21.15.151:0/1451069679 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6564109dd0 0x7f65641a0240 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f655400e9c0 tx=0x7f655400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:05:45.346 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:45.343+0000 7f655b400700 1 -- 172.21.15.151:0/1451069679 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6554004a30 con 0x7f6564109dd0 2024-05-21T22:05:45.346 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:45.343+0000 7f655b400700 1 -- 172.21.15.151:0/1451069679 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f655400cc70 con 0x7f6564109dd0 2024-05-21T22:05:45.346 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:45.343+0000 7f6569c00700 1 -- 172.21.15.151:0/1451069679 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65641a5540 con 0x7f6564109dd0 2024-05-21T22:05:45.346 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:45.343+0000 7f655b400700 1 -- 172.21.15.151:0/1451069679 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6554005490 con 0x7f6564109dd0 2024-05-21T22:05:45.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:45.343+0000 7f6569c00700 1 -- 172.21.15.151:0/1451069679 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65641acdc0 con 0x7f6564109dd0 2024-05-21T22:05:45.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:45.344+0000 7f6569c00700 1 -- 172.21.15.151:0/1451069679 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f656406a230 con 0x7f6564109dd0 2024-05-21T22:05:45.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:45.347+0000 7f655b400700 1 -- 172.21.15.151:0/1451069679 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7f6554004bd0 con 0x7f6564109dd0 2024-05-21T22:05:45.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:45.348+0000 7f655b400700 1 --2- 172.21.15.151:0/1451069679 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f65480b2a70 0x7f65480b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:45.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:45.348+0000 7f655b400700 1 -- 172.21.15.151:0/1451069679 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7f65540e2940 con 0x7f6564109dd0 2024-05-21T22:05:45.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:45.348+0000 7f6562000700 1 --2- 172.21.15.151:0/1451069679 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f65480b2a70 0x7f65480b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:45.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:45.349+0000 7f6562000700 1 --2- 172.21.15.151:0/1451069679 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f65480b2a70 0x7f65480b4f30 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f65641101c0 tx=0x7f6550009260 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:05:45.350 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:45.350+0000 7f655b400700 1 -- 172.21.15.151:0/1451069679 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6554054df0 con 0x7f6564109dd0 2024-05-21T22:05:45.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:45.627+0000 7f6569c00700 1 -- 172.21.15.151:0/1451069679 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f656402e560 con 0x7f6564109dd0 2024-05-21T22:05:45.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:45.628+0000 7f655b400700 1 -- 172.21.15.151:0/1451069679 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11958 (secure 0 0 0) 0x7f65540aa2b0 con 0x7f6564109dd0 2024-05-21T22:05:45.629 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T22:05:45.629 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14358,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/158010529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":158010529},{"type":"v1","addr":"172.21.15.151:6835","nonce":158010529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6839/2069058115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":2069058115},{"type":"v1","addr":"172.21.15.151:6839","nonce":2069058115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T21:51:19.552069+0000","modified":"2024-05-21T21:55:34.643755+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24530,"mds_1":24493,"mds_2":24491,"mds_3":24478,"mds_4":24473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14403":{"gid":14403,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6841/1749723794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6840","nonce":1749723794},{"type":"v1","addr":"172.21.15.151:6841","nonce":1749723794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/2026266114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":2026266114},{"type":"v1","addr":"172.21.15.184:6833","nonce":2026266114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/2467900844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":2467900844},{"type":"v1","addr":"172.21.15.164:6833","nonce":2467900844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3367640960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3367640960},{"type":"v1","addr":"172.21.15.151:6837","nonce":3367640960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/3264142621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3264142621},{"type":"v1","addr":"172.21.15.184:6835","nonce":3264142621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":17,"addr":"172.21.15.164:6835/2340428470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2340428470},{"type":"v1","addr":"172.21.15.164:6835","nonce":2340428470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":16,"addr":"172.21.15.184:6837/2988194888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":2988194888},{"type":"v1","addr":"172.21.15.184:6837","nonce":2988194888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.164:6837/1554609572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1554609572},{"type":"v1","addr":"172.21.15.164:6837","nonce":1554609572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.184:6839/3059935143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":3059935143},{"type":"v1","addr":"172.21.15.184:6839","nonce":3059935143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.164:6839/2646401247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":2646401247},{"type":"v1","addr":"172.21.15.164:6839","nonce":2646401247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24530,"qdb_cluster":[24530,24493,24491,24473,24478]},"id":1}]} 2024-05-21T22:05:45.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:45.631+0000 7f6569c00700 1 -- 172.21.15.151:0/1451069679 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f65480b2a70 msgr2=0x7f65480b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:45.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:45.631+0000 7f6569c00700 1 --2- 172.21.15.151:0/1451069679 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f65480b2a70 0x7f65480b4f30 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f65641101c0 tx=0x7f6550009260 comp rx=0 tx=0).stop 2024-05-21T22:05:45.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:45.631+0000 7f6569c00700 1 -- 172.21.15.151:0/1451069679 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6564109dd0 msgr2=0x7f65641a0240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:45.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:45.631+0000 7f6569c00700 1 --2- 172.21.15.151:0/1451069679 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6564109dd0 0x7f65641a0240 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f655400e9c0 tx=0x7f655400ecd0 comp rx=0 tx=0).stop 2024-05-21T22:05:45.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:45.631+0000 7f6569c00700 1 -- 172.21.15.151:0/1451069679 shutdown_connections 2024-05-21T22:05:45.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:45.631+0000 7f6569c00700 1 --2- 172.21.15.151:0/1451069679 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f65480b2a70 0x7f65480b4f30 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:45.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:45.631+0000 7f6569c00700 1 --2- 172.21.15.151:0/1451069679 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f656410eeb0 0x7f65641a4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:45.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:45.631+0000 7f6569c00700 1 --2- 172.21.15.151:0/1451069679 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6564109dd0 0x7f65641a0240 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:45.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:45.631+0000 7f6569c00700 1 --2- 172.21.15.151:0/1451069679 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f656410a780 0x7f65641a0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:45.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:45.631+0000 7f6569c00700 1 -- 172.21.15.151:0/1451069679 >> 172.21.15.151:0/1451069679 conn(0x7f6564076d10 msgr2=0x7f65640772f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:05:45.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:45.631+0000 7f6569c00700 1 -- 172.21.15.151:0/1451069679 shutdown_connections 2024-05-21T22:05:45.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:45.632+0000 7f6569c00700 1 -- 172.21.15.151:0/1451069679 wait complete. 2024-05-21T22:05:45.633 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 29 2024-05-21T22:05:45.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:05:45 smithi151 ceph-mon[31623]: pgmap v313: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-05-21T22:05:45.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:05:45 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/1451069679' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T22:05:45.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:05:45 smithi184 ceph-mon[36942]: pgmap v313: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-05-21T22:05:45.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:05:45 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/1451069679' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T22:05:46.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:05:45 smithi164 ceph-mon[36269]: pgmap v313: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-05-21T22:05:46.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:05:45 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/1451069679' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T22:05:46.497 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-21T22:05:46.963 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:05:47.969 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:05:47 smithi151 ceph-mon[31623]: pgmap v314: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-05-21T22:05:47.985 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:05:47 smithi184 ceph-mon[36942]: pgmap v314: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-05-21T22:05:48.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:05:47 smithi164 ceph-mon[36269]: pgmap v314: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-05-21T22:05:49.198 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.197+0000 7fd4a1600700 1 -- 172.21.15.151:0/1528154633 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd49c1095b0 msgr2=0x7fd49c109990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:49.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.197+0000 7fd4a1600700 1 --2- 172.21.15.151:0/1528154633 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd49c1095b0 0x7fd49c109990 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fd48c009990 tx=0x7fd48c009ca0 comp rx=0 tx=0).stop 2024-05-21T22:05:49.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.198+0000 7fd4a1600700 1 -- 172.21.15.151:0/1528154633 shutdown_connections 2024-05-21T22:05:49.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.198+0000 7fd4a1600700 1 --2- 172.21.15.151:0/1528154633 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd49c06bbe0 0x7fd49c1103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:49.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.198+0000 7fd4a1600700 1 --2- 172.21.15.151:0/1528154633 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd49c1095b0 0x7fd49c109990 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:49.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.198+0000 7fd4a1600700 1 --2- 172.21.15.151:0/1528154633 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd49c06b220 0x7fd49c06b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:49.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.198+0000 7fd4a1600700 1 -- 172.21.15.151:0/1528154633 >> 172.21.15.151:0/1528154633 conn(0x7fd49c0793f0 msgr2=0x7fd49c079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:05:49.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.198+0000 7fd4a1600700 1 -- 172.21.15.151:0/1528154633 shutdown_connections 2024-05-21T22:05:49.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.198+0000 7fd4a1600700 1 -- 172.21.15.151:0/1528154633 wait complete. 2024-05-21T22:05:49.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.199+0000 7fd4a1600700 1 Processor -- start 2024-05-21T22:05:49.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.199+0000 7fd4a1600700 1 -- start start 2024-05-21T22:05:49.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.199+0000 7fd4a1600700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd49c06b220 0x7fd49c19bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:49.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.199+0000 7fd4a1600700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd49c06bbe0 0x7fd49c19c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:49.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.200+0000 7fd4a1600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd49c1095b0 0x7fd49c1a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:49.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.200+0000 7fd4a1600700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd49c1136e0 con 0x7fd49c1095b0 2024-05-21T22:05:49.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.200+0000 7fd4a1600700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd49c113560 con 0x7fd49c06b220 2024-05-21T22:05:49.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.200+0000 7fd4a1600700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd49c113860 con 0x7fd49c06bbe0 2024-05-21T22:05:49.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.200+0000 7fd49aa00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd49c06b220 0x7fd49c19bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:49.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.200+0000 7fd49aa00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd49c06b220 0x7fd49c19bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:57322/0 (socket says 172.21.15.151:57322) 2024-05-21T22:05:49.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.200+0000 7fd49aa00700 1 -- 172.21.15.151:0/688455593 learned_addr learned my addr 172.21.15.151:0/688455593 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:05:49.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.200+0000 7fd492000700 1 --2- 172.21.15.151:0/688455593 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd49c06bbe0 0x7fd49c19c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:49.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.200+0000 7fd49b400700 1 --2- 172.21.15.151:0/688455593 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd49c1095b0 0x7fd49c1a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:49.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.200+0000 7fd49aa00700 1 -- 172.21.15.151:0/688455593 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd49c06bbe0 msgr2=0x7fd49c19c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:49.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.200+0000 7fd49aa00700 1 --2- 172.21.15.151:0/688455593 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd49c06bbe0 0x7fd49c19c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:49.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.200+0000 7fd49aa00700 1 -- 172.21.15.151:0/688455593 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd49c1095b0 msgr2=0x7fd49c1a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:49.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.200+0000 7fd49aa00700 1 --2- 172.21.15.151:0/688455593 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd49c1095b0 0x7fd49c1a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:49.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.200+0000 7fd49aa00700 1 -- 172.21.15.151:0/688455593 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd49c1a0db0 con 0x7fd49c06b220 2024-05-21T22:05:49.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.201+0000 7fd49aa00700 1 --2- 172.21.15.151:0/688455593 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd49c06b220 0x7fd49c19bdf0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fd48c000c00 tx=0x7fd48c00bfd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:05:49.203 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.201+0000 7fd493e00700 1 -- 172.21.15.151:0/688455593 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd48c027070 con 0x7fd49c06b220 2024-05-21T22:05:49.203 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.201+0000 7fd4a1600700 1 -- 172.21.15.151:0/688455593 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd49c1a1040 con 0x7fd49c06b220 2024-05-21T22:05:49.203 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.201+0000 7fd493e00700 1 -- 172.21.15.151:0/688455593 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd48c022470 con 0x7fd49c06b220 2024-05-21T22:05:49.203 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.201+0000 7fd493e00700 1 -- 172.21.15.151:0/688455593 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd48c017650 con 0x7fd49c06b220 2024-05-21T22:05:49.203 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.201+0000 7fd4a1600700 1 -- 172.21.15.151:0/688455593 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd49c1a8920 con 0x7fd49c06b220 2024-05-21T22:05:49.205 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.205+0000 7fd493e00700 1 -- 172.21.15.151:0/688455593 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7fd48c02a070 con 0x7fd49c06b220 2024-05-21T22:05:49.206 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.206+0000 7fd4abf40b80 1 Processor -- start 2024-05-21T22:05:49.206 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.206+0000 7fd4abf40b80 1 -- start start 2024-05-21T22:05:49.207 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.206+0000 7fd493e00700 1 --2- 172.21.15.151:0/688455593 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fd4800b2a70 0x7fd4800b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:49.207 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.206+0000 7fd4abf40b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559e9065ceb0 0x559e9065d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:49.207 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.206+0000 7fd4abf40b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559e9065d7d0 0x559e90657260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:49.207 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.206+0000 7fd4abf40b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559e90657880 0x559e90657ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:49.207 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.206+0000 7fd4abf40b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x559e905b5e40 con 0x559e90657880 2024-05-21T22:05:49.207 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.206+0000 7fd4abf40b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x559e905a59e0 con 0x559e9065ceb0 2024-05-21T22:05:49.207 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.206+0000 7fd4abf40b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x559e905a9a20 con 0x559e9065d7d0 2024-05-21T22:05:49.207 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.206+0000 7fd493e00700 1 -- 172.21.15.151:0/688455593 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7fd48c0e3370 con 0x7fd49c06b220 2024-05-21T22:05:49.208 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.207+0000 7fd49aa00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559e9065ceb0 0x559e9065d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:49.208 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.207+0000 7fd49b400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559e90657880 0x559e90657ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:49.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.207+0000 7fd49aa00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559e9065ceb0 0x559e9065d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:57336/0 (socket says 172.21.15.151:57336) 2024-05-21T22:05:49.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.207+0000 7fd49aa00700 1 -- 172.21.15.151:0/3887245513 learned_addr learned my addr 172.21.15.151:0/3887245513 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:05:49.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.207+0000 7fd49b400700 1 --2- 172.21.15.151:0/688455593 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fd4800b2a70 0x7fd4800b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:49.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.207+0000 7fd492000700 1 --2- 172.21.15.151:0/3887245513 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559e9065d7d0 0x559e90657260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:49.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.207+0000 7fd49aa00700 1 -- 172.21.15.151:0/3887245513 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559e9065d7d0 msgr2=0x559e90657260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:49.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.207+0000 7fd49aa00700 1 --2- 172.21.15.151:0/3887245513 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559e9065d7d0 0x559e90657260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:49.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.207+0000 7fd49aa00700 1 -- 172.21.15.151:0/3887245513 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559e90657880 msgr2=0x559e90657ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:49.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.207+0000 7fd49aa00700 1 --2- 172.21.15.151:0/3887245513 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559e90657880 0x559e90657ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:49.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.207+0000 7fd49aa00700 1 -- 172.21.15.151:0/3887245513 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e9056e420 con 0x559e9065ceb0 2024-05-21T22:05:49.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.208+0000 7fd49aa00700 1 --2- 172.21.15.151:0/3887245513 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559e9065ceb0 0x559e9065d290 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fd48c024000 tx=0x7fd48c00b2a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9542194bf84116d server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:05:49.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.208+0000 7fd49b400700 1 --2- 172.21.15.151:0/688455593 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fd4800b2a70 0x7fd4800b4f30 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd494009fd0 tx=0x7fd49400b040 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:05:49.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.208+0000 7fd48be00700 1 -- 172.21.15.151:0/3887245513 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd48c083150 con 0x559e9065ceb0 2024-05-21T22:05:49.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.208+0000 7fd48be00700 1 -- 172.21.15.151:0/3887245513 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd48c0aa180 con 0x559e9065ceb0 2024-05-21T22:05:49.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.208+0000 7fd48be00700 1 -- 172.21.15.151:0/3887245513 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd48c082f70 con 0x559e9065ceb0 2024-05-21T22:05:49.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.209+0000 7fd4abf40b80 1 -- 172.21.15.151:0/3887245513 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559e9065ceb0 msgr2=0x559e9065d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:49.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.209+0000 7fd4abf40b80 1 --2- 172.21.15.151:0/3887245513 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559e9065ceb0 0x559e9065d290 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fd48c024000 tx=0x7fd48c00b2a0 comp rx=0 tx=0).stop 2024-05-21T22:05:49.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.209+0000 7fd4abf40b80 1 -- 172.21.15.151:0/3887245513 shutdown_connections 2024-05-21T22:05:49.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.209+0000 7fd4abf40b80 1 --2- 172.21.15.151:0/3887245513 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559e9065ceb0 0x559e9065d290 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:49.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.209+0000 7fd4abf40b80 1 --2- 172.21.15.151:0/3887245513 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559e90657880 0x559e90657ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:49.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.209+0000 7fd4abf40b80 1 --2- 172.21.15.151:0/3887245513 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559e9065d7d0 0x559e90657260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:49.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.209+0000 7fd4abf40b80 1 -- 172.21.15.151:0/3887245513 >> 172.21.15.151:0/3887245513 conn(0x559e90585be0 msgr2=0x559e90652010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:05:49.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.209+0000 7fd4abf40b80 1 -- 172.21.15.151:0/3887245513 shutdown_connections 2024-05-21T22:05:49.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.209+0000 7fd4abf40b80 1 -- 172.21.15.151:0/3887245513 wait complete. 2024-05-21T22:05:49.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.210+0000 7fd4abf40b80 1 -- start start 2024-05-21T22:05:49.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.210+0000 7fd4abf40b80 1 Processor -- start 2024-05-21T22:05:49.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.210+0000 7fd4abf40b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559e90657880 0x559e906634c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:49.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.210+0000 7fd4abf40b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559e9065ceb0 0x559e90660860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:49.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.210+0000 7fd4abf40b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559e9065d7d0 0x559e90660da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:49.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.210+0000 7fd4abf40b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x559e904abfa0 con 0x559e9065ceb0 2024-05-21T22:05:49.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.210+0000 7fd4abf40b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x559e90546760 con 0x559e9065d7d0 2024-05-21T22:05:49.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.210+0000 7fd4abf40b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x559e90664000 con 0x559e90657880 2024-05-21T22:05:49.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.210+0000 7fd4abf40b80 10 client.0 _ll_register_callbacks cb 0x559e9054fc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:05:49.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.211+0000 7fd492000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559e9065ceb0 0x559e90660860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:49.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.211+0000 7fd49aa00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559e90657880 0x559e906634c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:49.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.211+0000 7fd49aa00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559e90657880 0x559e906634c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:52968/0 (socket says 172.21.15.151:52968) 2024-05-21T22:05:49.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.211+0000 7fd49aa00700 1 -- 172.21.15.151:0/2173951320 learned_addr learned my addr 172.21.15.151:0/2173951320 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:05:49.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.211+0000 7fd49b400700 1 --2- 172.21.15.151:0/2173951320 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559e9065d7d0 0x559e90660da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:49.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.211+0000 7fd49aa00700 1 -- 172.21.15.151:0/2173951320 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559e9065d7d0 msgr2=0x559e90660da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:49.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.211+0000 7fd49aa00700 1 --2- 172.21.15.151:0/2173951320 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559e9065d7d0 0x559e90660da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:49.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.211+0000 7fd49aa00700 1 -- 172.21.15.151:0/2173951320 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559e9065ceb0 msgr2=0x559e90660860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:49.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.211+0000 7fd49aa00700 1 --2- 172.21.15.151:0/2173951320 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559e9065ceb0 0x559e90660860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:49.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.211+0000 7fd49aa00700 1 -- 172.21.15.151:0/2173951320 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e904f58d0 con 0x559e90657880 2024-05-21T22:05:49.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.211+0000 7fd49aa00700 1 --2- 172.21.15.151:0/2173951320 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559e90657880 0x559e906634c0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fd48c005f50 tx=0x7fd48c0e0f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e052653f01ac2c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:05:49.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.211+0000 7fd488c00700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-21T22:05:49.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.212+0000 7fd488c00700 1 -- 172.21.15.151:0/2173951320 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd48c0aa670 con 0x559e90657880 2024-05-21T22:05:49.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.212+0000 7fd488c00700 1 -- 172.21.15.151:0/2173951320 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7fd48c082d00 con 0x559e90657880 2024-05-21T22:05:49.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.212+0000 7fd4abf40b80 1 -- 172.21.15.151:0/2173951320 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd48c0aa180 con 0x559e90657880 2024-05-21T22:05:49.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.212+0000 7fd488c00700 1 -- 172.21.15.151:0/2173951320 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd48c082f30 con 0x559e90657880 2024-05-21T22:05:49.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.212+0000 7fd4abf40b80 10 client.27589 fetch_fsmap learned FSMap version 29 2024-05-21T22:05:49.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.212+0000 7fd4abf40b80 1 -- 172.21.15.151:0/2173951320 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559e90576060 con 0x559e90657880 2024-05-21T22:05:49.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.212+0000 7fd488c00700 1 -- 172.21.15.151:0/2173951320 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd48c0aa180 con 0x559e90657880 2024-05-21T22:05:49.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.212+0000 7fd488c00700 1 -- 172.21.15.151:0/2173951320 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fd48c0b2fb0 con 0x559e90657880 2024-05-21T22:05:49.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.212+0000 7fd4abf40b80 10 client.27589 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:05:49.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.212+0000 7fd4abf40b80 10 client.27589 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-05-21T22:05:49.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.212+0000 7fd4abf40b80 20 client.27589 populate_metadata read hostname 'smithi151' 2024-05-21T22:05:49.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.213+0000 7fd4abf40b80 1 --2- 172.21.15.151:0/2173951320 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x559e906708e0 0x559e90672cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:49.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.213+0000 7fd4abf40b80 4 client.27589 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T22:05:49.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.213+0000 7fd4abf40b80 1 -- 172.21.15.151:0/2173951320 --> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e905b7d90 con 0x559e906708e0 2024-05-21T22:05:49.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.213+0000 7fd492000700 1 --2- 172.21.15.151:0/2173951320 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x559e906708e0 0x559e90672cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:49.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.214+0000 7fd492000700 1 --2- 172.21.15.151:0/2173951320 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x559e906708e0 0x559e90672cd0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6b02a25747d72828 server_cookie=24b7394d5f49a037 in_seq=0 out_seq=0 2024-05-21T22:05:49.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.214+0000 7fd488c00700 10 client.27589 ms_handle_connect on v2:172.21.15.164:6834/2340428470 2024-05-21T22:05:49.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.214+0000 7fd488c00700 1 -- 172.21.15.151:0/2173951320 <== mds.4 v2:172.21.15.164:6834/2340428470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x559e905b7d90 con 0x559e906708e0 2024-05-21T22:05:49.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.214+0000 7fd488c00700 10 client.27589 handle_command_reply: tid=0 2024-05-21T22:05:49.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.214+0000 7fd4abf40b80 1 client.27589 shutdown 2024-05-21T22:05:49.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.215+0000 7fd4abf40b80 1 -- 172.21.15.151:0/2173951320 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559e90657880 msgr2=0x559e906634c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:49.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.215+0000 7fd4abf40b80 1 --2- 172.21.15.151:0/2173951320 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559e90657880 0x559e906634c0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fd48c005f50 tx=0x7fd48c0e0f00 comp rx=0 tx=0).stop 2024-05-21T22:05:49.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.215+0000 7fd4abf40b80 1 -- 172.21.15.151:0/2173951320 shutdown_connections 2024-05-21T22:05:49.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.215+0000 7fd4abf40b80 1 --2- 172.21.15.151:0/2173951320 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x559e906708e0 0x559e90672cd0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:49.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.215+0000 7fd4abf40b80 1 --2- 172.21.15.151:0/2173951320 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559e9065d7d0 0x559e90660da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:49.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.215+0000 7fd4abf40b80 1 --2- 172.21.15.151:0/2173951320 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559e90657880 0x559e906634c0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:49.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.215+0000 7fd4abf40b80 1 --2- 172.21.15.151:0/2173951320 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559e9065ceb0 0x559e90660860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:49.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.215+0000 7fd488c00700 0 client.27589 ms_handle_reset on v2:172.21.15.164:6834/2340428470 2024-05-21T22:05:49.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.215+0000 7fd4abf40b80 1 -- 172.21.15.151:0/2173951320 >> 172.21.15.151:0/2173951320 conn(0x559e90585be0 msgr2=0x559e90656250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:05:49.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.215+0000 7fd4abf40b80 1 -- 172.21.15.151:0/2173951320 shutdown_connections 2024-05-21T22:05:49.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.215+0000 7fd4abf40b80 1 -- 172.21.15.151:0/2173951320 wait complete. 2024-05-21T22:05:49.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.215+0000 7fd4abf40b80 20 client.27589 trim_cache size 0 max 16384 2024-05-21T22:05:49.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.232+0000 7fd4abf40b80 1 Processor -- start 2024-05-21T22:05:49.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.232+0000 7fd4abf40b80 1 -- start start 2024-05-21T22:05:49.233 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.232+0000 7fd4abf40b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559e90657880 0x559e90657260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:49.233 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.232+0000 7fd4abf40b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559e9065ceb0 0x559e9065d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:49.233 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.232+0000 7fd4abf40b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559e9065d7d0 0x559e9065a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:49.233 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.232+0000 7fd4abf40b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd49c1aff60 con 0x559e90657880 2024-05-21T22:05:49.233 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.232+0000 7fd4abf40b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd49c1b1aa0 con 0x559e9065ceb0 2024-05-21T22:05:49.233 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.232+0000 7fd4abf40b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd49c1b0630 con 0x559e9065d7d0 2024-05-21T22:05:49.233 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.233+0000 7fd492000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559e9065ceb0 0x559e9065d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:49.233 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.233+0000 7fd492000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559e9065ceb0 0x559e9065d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:57350/0 (socket says 172.21.15.151:57350) 2024-05-21T22:05:49.233 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.233+0000 7fd492000700 1 -- 172.21.15.151:0/2257619178 learned_addr learned my addr 172.21.15.151:0/2257619178 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:05:49.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.233+0000 7fd49b400700 1 --2- 172.21.15.151:0/2257619178 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559e9065d7d0 0x559e9065a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:49.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.233+0000 7fd49aa00700 1 --2- 172.21.15.151:0/2257619178 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559e90657880 0x559e90657260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:49.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.233+0000 7fd492000700 1 -- 172.21.15.151:0/2257619178 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559e9065d7d0 msgr2=0x559e9065a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:49.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.233+0000 7fd492000700 1 --2- 172.21.15.151:0/2257619178 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559e9065d7d0 0x559e9065a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:49.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.233+0000 7fd492000700 1 -- 172.21.15.151:0/2257619178 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559e90657880 msgr2=0x559e90657260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:49.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.233+0000 7fd492000700 1 --2- 172.21.15.151:0/2257619178 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559e90657880 0x559e90657260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:49.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.233+0000 7fd492000700 1 -- 172.21.15.151:0/2257619178 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e90652f40 con 0x559e9065ceb0 2024-05-21T22:05:49.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.233+0000 7fd492000700 1 --2- 172.21.15.151:0/2257619178 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559e9065ceb0 0x559e9065d290 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x559e90664110 tx=0x7fd484009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d65b7674c45d72 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:05:49.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.234+0000 7fd48b400700 1 -- 172.21.15.151:0/2257619178 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd484010cb0 con 0x559e9065ceb0 2024-05-21T22:05:49.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.234+0000 7fd48b400700 1 -- 172.21.15.151:0/2257619178 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd484010e50 con 0x559e9065ceb0 2024-05-21T22:05:49.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.234+0000 7fd48b400700 1 -- 172.21.15.151:0/2257619178 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd48401f430 con 0x559e9065ceb0 2024-05-21T22:05:49.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.234+0000 7fd4abf40b80 1 -- 172.21.15.151:0/2257619178 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559e9065ceb0 msgr2=0x559e9065d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:49.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.234+0000 7fd4abf40b80 1 --2- 172.21.15.151:0/2257619178 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559e9065ceb0 0x559e9065d290 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x559e90664110 tx=0x7fd484009260 comp rx=0 tx=0).stop 2024-05-21T22:05:49.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.234+0000 7fd4abf40b80 1 -- 172.21.15.151:0/2257619178 shutdown_connections 2024-05-21T22:05:49.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.234+0000 7fd4abf40b80 1 --2- 172.21.15.151:0/2257619178 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559e9065d7d0 0x559e9065a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:49.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.234+0000 7fd4abf40b80 1 --2- 172.21.15.151:0/2257619178 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559e90657880 0x559e90657260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:49.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.234+0000 7fd4abf40b80 1 --2- 172.21.15.151:0/2257619178 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559e9065ceb0 0x559e9065d290 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:49.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.234+0000 7fd4abf40b80 1 -- 172.21.15.151:0/2257619178 >> 172.21.15.151:0/2257619178 conn(0x559e90585be0 msgr2=0x559e90660820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:05:49.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.235+0000 7fd4abf40b80 1 -- 172.21.15.151:0/2257619178 shutdown_connections 2024-05-21T22:05:49.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.235+0000 7fd4abf40b80 1 -- 172.21.15.151:0/2257619178 wait complete. 2024-05-21T22:05:49.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.235+0000 7fd4abf40b80 1 -- start start 2024-05-21T22:05:49.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.235+0000 7fd4abf40b80 1 Processor -- start 2024-05-21T22:05:49.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.236+0000 7fd4abf40b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559e90657880 0x559e90676530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:49.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.236+0000 7fd4abf40b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559e9065ceb0 0x559e906738d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:49.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.236+0000 7fd4abf40b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559e9065d7d0 0x559e90674010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:49.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.236+0000 7fd4abf40b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd49c1b13d0 con 0x559e90657880 2024-05-21T22:05:49.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.236+0000 7fd4abf40b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd49c1b0d00 con 0x559e9065d7d0 2024-05-21T22:05:49.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.236+0000 7fd4abf40b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fd49c1adf20 con 0x559e9065ceb0 2024-05-21T22:05:49.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.236+0000 7fd4abf40b80 10 client.0 _ll_register_callbacks cb 0x559e9054fc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:05:49.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.236+0000 7fd49aa00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559e90657880 0x559e90676530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:49.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.236+0000 7fd49aa00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559e90657880 0x559e90676530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:43160/0 (socket says 172.21.15.151:43160) 2024-05-21T22:05:49.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.236+0000 7fd49aa00700 1 -- 172.21.15.151:0/1783104740 learned_addr learned my addr 172.21.15.151:0/1783104740 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:05:49.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.236+0000 7fd492000700 1 --2- 172.21.15.151:0/1783104740 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559e9065ceb0 0x559e906738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:49.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.236+0000 7fd49b400700 1 --2- 172.21.15.151:0/1783104740 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559e9065d7d0 0x559e90674010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:49.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.236+0000 7fd49aa00700 1 -- 172.21.15.151:0/1783104740 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559e9065ceb0 msgr2=0x559e906738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:49.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.236+0000 7fd49aa00700 1 --2- 172.21.15.151:0/1783104740 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559e9065ceb0 0x559e906738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:49.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.236+0000 7fd49aa00700 1 -- 172.21.15.151:0/1783104740 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559e9065d7d0 msgr2=0x559e90674010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:49.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.236+0000 7fd49aa00700 1 --2- 172.21.15.151:0/1783104740 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559e9065d7d0 0x559e90674010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:49.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.236+0000 7fd49aa00700 1 -- 172.21.15.151:0/1783104740 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e906da120 con 0x559e90657880 2024-05-21T22:05:49.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.237+0000 7fd49aa00700 1 --2- 172.21.15.151:0/1783104740 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559e90657880 0x559e90676530 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fd48c0098e0 tx=0x7fd48c0095f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d5487c29be91ba4 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:05:49.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.237+0000 7fd488c00700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-21T22:05:49.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.237+0000 7fd488c00700 1 -- 172.21.15.151:0/1783104740 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd48c0aa180 con 0x559e90657880 2024-05-21T22:05:49.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.237+0000 7fd488c00700 1 -- 172.21.15.151:0/1783104740 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7fd48c0821a0 con 0x559e90657880 2024-05-21T22:05:49.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.237+0000 7fd4abf40b80 1 -- 172.21.15.151:0/1783104740 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd484010e50 con 0x559e90657880 2024-05-21T22:05:49.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.237+0000 7fd488c00700 1 -- 172.21.15.151:0/1783104740 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd48c0aad70 con 0x559e90657880 2024-05-21T22:05:49.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.237+0000 7fd4abf40b80 10 client.17463 fetch_fsmap learned FSMap version 29 2024-05-21T22:05:49.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.237+0000 7fd4abf40b80 1 -- 172.21.15.151:0/1783104740 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559e906da120 con 0x559e90657880 2024-05-21T22:05:49.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.238+0000 7fd488c00700 1 -- 172.21.15.151:0/1783104740 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd484010e50 con 0x559e90657880 2024-05-21T22:05:49.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.238+0000 7fd488c00700 1 -- 172.21.15.151:0/1783104740 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fd48c090cb0 con 0x559e90657880 2024-05-21T22:05:49.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.238+0000 7fd4abf40b80 10 client.17463 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:05:49.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.238+0000 7fd4abf40b80 10 client.17463 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-05-21T22:05:49.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.238+0000 7fd4abf40b80 20 client.17463 populate_metadata read hostname 'smithi151' 2024-05-21T22:05:49.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.238+0000 7fd4abf40b80 1 --2- 172.21.15.151:0/1783104740 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x559e906db510 0x559e906dd930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:49.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.238+0000 7fd4abf40b80 4 client.17463 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T22:05:49.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.238+0000 7fd4abf40b80 1 -- 172.21.15.151:0/1783104740 --> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559e905b1570 con 0x559e906db510 2024-05-21T22:05:49.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.238+0000 7fd492000700 1 --2- 172.21.15.151:0/1783104740 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x559e906db510 0x559e906dd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:49.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.239+0000 7fd492000700 1 --2- 172.21.15.151:0/1783104740 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x559e906db510 0x559e906dd930 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=53dc1a68df24b9f8 server_cookie=8c50159c8a37b035 in_seq=0 out_seq=0 2024-05-21T22:05:49.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.239+0000 7fd488c00700 10 client.17463 ms_handle_connect on v2:172.21.15.164:6834/2340428470 2024-05-21T22:05:49.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.239+0000 7fd488c00700 1 -- 172.21.15.151:0/1783104740 <== mds.4 v2:172.21.15.164:6834/2340428470 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559e905b1570 con 0x559e906db510 2024-05-21T22:05:49.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.239+0000 7fd488c00700 10 client.17463 handle_command_reply: tid=0 2024-05-21T22:05:49.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.239+0000 7fd4abf40b80 1 client.17463 shutdown 2024-05-21T22:05:49.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.239+0000 7fd4abf40b80 1 -- 172.21.15.151:0/1783104740 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559e90657880 msgr2=0x559e90676530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:49.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.239+0000 7fd4abf40b80 1 --2- 172.21.15.151:0/1783104740 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559e90657880 0x559e90676530 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fd48c0098e0 tx=0x7fd48c0095f0 comp rx=0 tx=0).stop 2024-05-21T22:05:49.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.239+0000 7fd4abf40b80 1 -- 172.21.15.151:0/1783104740 shutdown_connections 2024-05-21T22:05:49.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.239+0000 7fd4abf40b80 1 --2- 172.21.15.151:0/1783104740 >> [v2:172.21.15.164:6834/2340428470,v1:172.21.15.164:6835/2340428470] conn(0x559e906db510 0x559e906dd930 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:49.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.239+0000 7fd4abf40b80 1 --2- 172.21.15.151:0/1783104740 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x559e9065d7d0 0x559e90674010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:49.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.239+0000 7fd4abf40b80 1 --2- 172.21.15.151:0/1783104740 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559e90657880 0x559e90676530 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:49.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.240+0000 7fd4abf40b80 1 --2- 172.21.15.151:0/1783104740 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559e9065ceb0 0x559e906738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:49.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.240+0000 7fd4abf40b80 1 -- 172.21.15.151:0/1783104740 >> 172.21.15.151:0/1783104740 conn(0x559e90585be0 msgr2=0x559e9065a9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:05:49.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.240+0000 7fd488c00700 0 client.17463 ms_handle_reset on v2:172.21.15.164:6834/2340428470 2024-05-21T22:05:49.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.240+0000 7fd4abf40b80 1 -- 172.21.15.151:0/1783104740 shutdown_connections 2024-05-21T22:05:49.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.240+0000 7fd4abf40b80 1 -- 172.21.15.151:0/1783104740 wait complete. 2024-05-21T22:05:49.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.240+0000 7fd4abf40b80 20 client.17463 trim_cache size 0 max 16384 2024-05-21T22:05:49.244 INFO:teuthology.orchestra.run.smithi151.stdout:[] 2024-05-21T22:05:49.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.241+0000 7fd4a1600700 1 -- 172.21.15.151:0/688455593 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fd4800b2a70 msgr2=0x7fd4800b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:49.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.241+0000 7fd4a1600700 1 --2- 172.21.15.151:0/688455593 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fd4800b2a70 0x7fd4800b4f30 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd494009fd0 tx=0x7fd49400b040 comp rx=0 tx=0).stop 2024-05-21T22:05:49.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.241+0000 7fd4a1600700 1 -- 172.21.15.151:0/688455593 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd49c06b220 msgr2=0x7fd49c19bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:49.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.241+0000 7fd4a1600700 1 --2- 172.21.15.151:0/688455593 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd49c06b220 0x7fd49c19bdf0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fd48c000c00 tx=0x7fd48c00bfd0 comp rx=0 tx=0).stop 2024-05-21T22:05:49.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.241+0000 7fd4a1600700 1 -- 172.21.15.151:0/688455593 shutdown_connections 2024-05-21T22:05:49.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.241+0000 7fd4a1600700 1 --2- 172.21.15.151:0/688455593 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fd4800b2a70 0x7fd4800b4f30 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:49.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.241+0000 7fd4a1600700 1 --2- 172.21.15.151:0/688455593 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd49c06b220 0x7fd49c19bdf0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:49.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.241+0000 7fd4a1600700 1 --2- 172.21.15.151:0/688455593 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd49c1095b0 0x7fd49c1a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:49.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.241+0000 7fd4a1600700 1 --2- 172.21.15.151:0/688455593 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd49c06bbe0 0x7fd49c19c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:49.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.241+0000 7fd4a1600700 1 -- 172.21.15.151:0/688455593 >> 172.21.15.151:0/688455593 conn(0x7fd49c0793f0 msgr2=0x7fd49c100f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:05:49.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.241+0000 7fd4a1600700 1 -- 172.21.15.151:0/688455593 shutdown_connections 2024-05-21T22:05:49.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:49.241+0000 7fd4a1600700 1 -- 172.21.15.151:0/688455593 wait complete. 2024-05-21T22:05:49.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:05:49 smithi151 ceph-mon[31623]: pgmap v315: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-05-21T22:05:49.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:05:49 smithi184 ceph-mon[36942]: pgmap v315: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-05-21T22:05:50.032 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-21T22:05:50.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:05:49 smithi164 ceph-mon[36269]: pgmap v315: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-05-21T22:05:50.488 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:05:51.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:05:51 smithi151 ceph-mon[31623]: pgmap v316: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-05-21T22:05:51.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:05:51 smithi184 ceph-mon[36942]: pgmap v316: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-05-21T22:05:52.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:05:51 smithi164 ceph-mon[36269]: pgmap v316: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-05-21T22:05:52.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.588+0000 7feaa6800700 1 -- 172.21.15.151:0/3433535232 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feaa0102e00 msgr2=0x7feaa010f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:52.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.588+0000 7feaa6800700 1 --2- 172.21.15.151:0/3433535232 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feaa0102e00 0x7feaa010f2f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fea9800b0a0 tx=0x7fea9800b3b0 comp rx=0 tx=0).stop 2024-05-21T22:05:52.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.588+0000 7feaa6800700 1 -- 172.21.15.151:0/3433535232 shutdown_connections 2024-05-21T22:05:52.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.588+0000 7feaa6800700 1 --2- 172.21.15.151:0/3433535232 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feaa0102e00 0x7feaa010f2f0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:52.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.588+0000 7feaa6800700 1 --2- 172.21.15.151:0/3433535232 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feaa0108520 0x7feaa0108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:52.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.588+0000 7feaa6800700 1 --2- 172.21.15.151:0/3433535232 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feaa0102440 0x7feaa01028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:52.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.588+0000 7feaa6800700 1 -- 172.21.15.151:0/3433535232 >> 172.21.15.151:0/3433535232 conn(0x7feaa00793f0 msgr2=0x7feaa0079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:05:52.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.588+0000 7feaa6800700 1 -- 172.21.15.151:0/3433535232 shutdown_connections 2024-05-21T22:05:52.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.588+0000 7feaa6800700 1 -- 172.21.15.151:0/3433535232 wait complete. 2024-05-21T22:05:52.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.589+0000 7feaa6800700 1 Processor -- start 2024-05-21T22:05:52.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.590+0000 7feaa6800700 1 -- start start 2024-05-21T22:05:52.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.590+0000 7feaa6800700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feaa0102440 0x7feaa019beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:52.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.590+0000 7feaa6800700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feaa0102e00 0x7feaa019c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:52.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.590+0000 7feaa6800700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feaa0108520 0x7feaa01a0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:52.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.590+0000 7feaa6800700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7feaa0112640 con 0x7feaa0108520 2024-05-21T22:05:52.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.590+0000 7feaa6800700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7feaa01124c0 con 0x7feaa0102440 2024-05-21T22:05:52.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.590+0000 7feaa6800700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7feaa01127c0 con 0x7feaa0102e00 2024-05-21T22:05:52.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.590+0000 7fea9fe00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feaa0108520 0x7feaa01a0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:52.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.590+0000 7fea9fe00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feaa0108520 0x7feaa01a0780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:43184/0 (socket says 172.21.15.151:43184) 2024-05-21T22:05:52.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.590+0000 7fea9f400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feaa0102440 0x7feaa019beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:52.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.590+0000 7fea9fe00700 1 -- 172.21.15.151:0/535908654 learned_addr learned my addr 172.21.15.151:0/535908654 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:05:52.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.591+0000 7fea9ea00700 1 --2- 172.21.15.151:0/535908654 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feaa0102e00 0x7feaa019c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:52.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.591+0000 7fea9fe00700 1 -- 172.21.15.151:0/535908654 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feaa0102e00 msgr2=0x7feaa019c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:52.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.591+0000 7fea9fe00700 1 --2- 172.21.15.151:0/535908654 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feaa0102e00 0x7feaa019c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:52.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.591+0000 7fea9fe00700 1 -- 172.21.15.151:0/535908654 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feaa0102440 msgr2=0x7feaa019beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:52.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.591+0000 7fea9fe00700 1 --2- 172.21.15.151:0/535908654 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feaa0102440 0x7feaa019beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:52.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.591+0000 7fea9fe00700 1 -- 172.21.15.151:0/535908654 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaa01a0ec0 con 0x7feaa0108520 2024-05-21T22:05:52.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.591+0000 7fea9f400700 1 --2- 172.21.15.151:0/535908654 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feaa0102440 0x7feaa019beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:05:52.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.591+0000 7fea9fe00700 1 --2- 172.21.15.151:0/535908654 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feaa0108520 0x7feaa01a0780 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fea9800b070 tx=0x7fea98009c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:05:52.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.592+0000 7fea97e00700 1 -- 172.21.15.151:0/535908654 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea98021070 con 0x7feaa0108520 2024-05-21T22:05:52.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.592+0000 7fea97e00700 1 -- 172.21.15.151:0/535908654 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea98004e40 con 0x7feaa0108520 2024-05-21T22:05:52.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.592+0000 7feaa6800700 1 -- 172.21.15.151:0/535908654 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feaa01a1150 con 0x7feaa0108520 2024-05-21T22:05:52.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.592+0000 7fea97e00700 1 -- 172.21.15.151:0/535908654 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea98004630 con 0x7feaa0108520 2024-05-21T22:05:52.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.592+0000 7feaa6800700 1 -- 172.21.15.151:0/535908654 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feaa01a8a30 con 0x7feaa0108520 2024-05-21T22:05:52.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.595+0000 7fea97e00700 1 -- 172.21.15.151:0/535908654 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7fea98007980 con 0x7feaa0108520 2024-05-21T22:05:52.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.597+0000 7feab111bb80 1 Processor -- start 2024-05-21T22:05:52.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.597+0000 7feab111bb80 1 -- start start 2024-05-21T22:05:52.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.597+0000 7fea97e00700 1 --2- 172.21.15.151:0/535908654 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fea840b29f0 0x7fea840b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:52.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.597+0000 7feab111bb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55911817e610 0x55911817e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:52.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.597+0000 7feab111bb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55911817ef30 0x559118178b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:52.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.597+0000 7feab111bb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559118179140 0x5591181795c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:52.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.597+0000 7fea97e00700 1 -- 172.21.15.151:0/535908654 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7fea9800f020 con 0x7feaa0108520 2024-05-21T22:05:52.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.597+0000 7feab111bb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5591180d6e40 con 0x55911817e610 2024-05-21T22:05:52.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.597+0000 7feab111bb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5591180c69e0 con 0x55911817ef30 2024-05-21T22:05:52.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.597+0000 7feab111bb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5591180caa20 con 0x559118179140 2024-05-21T22:05:52.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.597+0000 7fea9ea00700 1 --2- 172.21.15.151:0/535908654 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fea840b29f0 0x7fea840b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:52.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.598+0000 7fea9f400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55911817e610 0x55911817e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:52.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.598+0000 7fea9f400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55911817e610 0x55911817e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:43190/0 (socket says 172.21.15.151:43190) 2024-05-21T22:05:52.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.598+0000 7fea9f400700 1 -- 172.21.15.151:0/2868057143 learned_addr learned my addr 172.21.15.151:0/2868057143 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:05:52.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.598+0000 7fea9fe00700 1 --2- 172.21.15.151:0/2868057143 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559118179140 0x5591181795c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:52.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.598+0000 7fea9ea00700 1 --2- 172.21.15.151:0/2868057143 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55911817ef30 0x559118178b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:52.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.598+0000 7fea9fe00700 1 -- 172.21.15.151:0/2868057143 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55911817ef30 msgr2=0x559118178b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:52.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.598+0000 7fea9fe00700 1 --2- 172.21.15.151:0/2868057143 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55911817ef30 0x559118178b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:52.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.598+0000 7fea9fe00700 1 -- 172.21.15.151:0/2868057143 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55911817e610 msgr2=0x55911817e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:52.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.598+0000 7fea9fe00700 1 --2- 172.21.15.151:0/2868057143 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55911817e610 0x55911817e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:52.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.598+0000 7fea9fe00700 1 -- 172.21.15.151:0/2868057143 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55911808f420 con 0x559118179140 2024-05-21T22:05:52.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.598+0000 7fea9f400700 1 --2- 172.21.15.151:0/2868057143 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55911817e610 0x55911817e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:05:52.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.598+0000 7fea9fe00700 1 --2- 172.21.15.151:0/2868057143 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559118179140 0x5591181795c0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fea980e1090 tx=0x7fea980aa420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1b2361e6f37b1b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:05:52.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.599+0000 7fea9ea00700 1 --2- 172.21.15.151:0/535908654 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fea840b29f0 0x7fea840b4eb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fea8c0097a0 tx=0x7fea8c009b80 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:05:52.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.599+0000 7fea94c00700 1 -- 172.21.15.151:0/2868057143 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea980ab3b0 con 0x559118179140 2024-05-21T22:05:52.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.599+0000 7fea94c00700 1 -- 172.21.15.151:0/2868057143 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea980aacb0 con 0x559118179140 2024-05-21T22:05:52.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.599+0000 7fea94c00700 1 -- 172.21.15.151:0/2868057143 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea98003d20 con 0x559118179140 2024-05-21T22:05:52.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.600+0000 7feab111bb80 1 -- 172.21.15.151:0/2868057143 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559118179140 msgr2=0x5591181795c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:52.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.600+0000 7feab111bb80 1 --2- 172.21.15.151:0/2868057143 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559118179140 0x5591181795c0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fea980e1090 tx=0x7fea980aa420 comp rx=0 tx=0).stop 2024-05-21T22:05:52.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.600+0000 7feab111bb80 1 -- 172.21.15.151:0/2868057143 shutdown_connections 2024-05-21T22:05:52.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.600+0000 7feab111bb80 1 --2- 172.21.15.151:0/2868057143 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559118179140 0x5591181795c0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:52.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.600+0000 7feab111bb80 1 --2- 172.21.15.151:0/2868057143 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55911817e610 0x55911817e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:52.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.600+0000 7feab111bb80 1 --2- 172.21.15.151:0/2868057143 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55911817ef30 0x559118178b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:52.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.600+0000 7feab111bb80 1 -- 172.21.15.151:0/2868057143 >> 172.21.15.151:0/2868057143 conn(0x5591180a6be0 msgr2=0x559118173690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:05:52.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.600+0000 7feab111bb80 1 -- 172.21.15.151:0/2868057143 shutdown_connections 2024-05-21T22:05:52.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.600+0000 7feab111bb80 1 -- 172.21.15.151:0/2868057143 wait complete. 2024-05-21T22:05:52.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.600+0000 7feab111bb80 1 -- start start 2024-05-21T22:05:52.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.601+0000 7feab111bb80 1 Processor -- start 2024-05-21T22:05:52.604 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.601+0000 7feab111bb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559118179140 0x559118184de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:52.604 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.601+0000 7feab111bb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55911817e610 0x559118182180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:52.604 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.601+0000 7feab111bb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55911817ef30 0x5591181826c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:52.604 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.601+0000 7feab111bb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x559117fccfa0 con 0x55911817e610 2024-05-21T22:05:52.604 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.601+0000 7feab111bb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x559118067760 con 0x55911817ef30 2024-05-21T22:05:52.604 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.601+0000 7feab111bb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5591181857a0 con 0x559118179140 2024-05-21T22:05:52.604 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.601+0000 7feab111bb80 10 client.0 _ll_register_callbacks cb 0x559118070c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:05:52.604 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.601+0000 7fea9ea00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55911817e610 0x559118182180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:52.604 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.601+0000 7fea9ea00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55911817e610 0x559118182180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:43202/0 (socket says 172.21.15.151:43202) 2024-05-21T22:05:52.604 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.601+0000 7fea9ea00700 1 -- 172.21.15.151:0/593138545 learned_addr learned my addr 172.21.15.151:0/593138545 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:05:52.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.602+0000 7fea9fe00700 1 --2- 172.21.15.151:0/593138545 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55911817ef30 0x5591181826c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:52.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.602+0000 7fea9f400700 1 --2- 172.21.15.151:0/593138545 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559118179140 0x559118184de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:52.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.602+0000 7fea9ea00700 1 -- 172.21.15.151:0/593138545 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559118179140 msgr2=0x559118184de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:52.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.602+0000 7fea9ea00700 1 --2- 172.21.15.151:0/593138545 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559118179140 0x559118184de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:52.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.602+0000 7fea9ea00700 1 -- 172.21.15.151:0/593138545 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55911817ef30 msgr2=0x5591181826c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:52.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.602+0000 7fea9ea00700 1 --2- 172.21.15.151:0/593138545 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55911817ef30 0x5591181826c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:52.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.602+0000 7fea9ea00700 1 -- 172.21.15.151:0/593138545 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591180168d0 con 0x55911817e610 2024-05-21T22:05:52.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.602+0000 7fea9ea00700 1 --2- 172.21.15.151:0/593138545 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55911817e610 0x559118182180 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fea8c013460 tx=0x7fea8c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a25bff36025808c server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:05:52.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.602+0000 7fea89600700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-21T22:05:52.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.603+0000 7fea89600700 1 -- 172.21.15.151:0/593138545 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea8c00fa90 con 0x55911817e610 2024-05-21T22:05:52.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.603+0000 7fea89600700 1 -- 172.21.15.151:0/593138545 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7fea8c00fc30 con 0x55911817e610 2024-05-21T22:05:52.606 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.603+0000 7feab111bb80 1 -- 172.21.15.151:0/593138545 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea980aacb0 con 0x55911817e610 2024-05-21T22:05:52.606 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.603+0000 7fea89600700 1 -- 172.21.15.151:0/593138545 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fea8c00fe30 con 0x55911817e610 2024-05-21T22:05:52.606 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.603+0000 7feab111bb80 10 client.17475 fetch_fsmap learned FSMap version 29 2024-05-21T22:05:52.606 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.603+0000 7feab111bb80 1 -- 172.21.15.151:0/593138545 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559118097060 con 0x55911817e610 2024-05-21T22:05:52.606 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.603+0000 7fea89600700 1 -- 172.21.15.151:0/593138545 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea980aacb0 con 0x55911817e610 2024-05-21T22:05:52.606 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.603+0000 7fea89600700 1 -- 172.21.15.151:0/593138545 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fea8c020050 con 0x55911817e610 2024-05-21T22:05:52.606 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.604+0000 7feab111bb80 10 client.17475 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:05:52.606 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.604+0000 7feab111bb80 10 client.17475 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-21T22:05:52.606 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.604+0000 7feab111bb80 20 client.17475 populate_metadata read hostname 'smithi151' 2024-05-21T22:05:52.606 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.604+0000 7feab111bb80 1 --2- 172.21.15.151:0/593138545 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x5591181923f0 0x5591181947e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:52.606 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.604+0000 7feab111bb80 4 client.17475 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T22:05:52.606 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.604+0000 7feab111bb80 1 -- 172.21.15.151:0/593138545 --> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5591180d8d90 con 0x5591181923f0 2024-05-21T22:05:52.607 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.604+0000 7fea9f400700 1 --2- 172.21.15.151:0/593138545 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x5591181923f0 0x5591181947e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:52.607 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.605+0000 7fea9f400700 1 --2- 172.21.15.151:0/593138545 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x5591181923f0 0x5591181947e0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b418426679006b0d server_cookie=f9acfbbbb0d505b in_seq=0 out_seq=0 2024-05-21T22:05:52.608 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.605+0000 7fea89600700 10 client.17475 ms_handle_connect on v2:172.21.15.184:6836/2988194888 2024-05-21T22:05:52.608 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.605+0000 7fea89600700 1 -- 172.21.15.151:0/593138545 <== mds.3 v2:172.21.15.184:6836/2988194888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5591180d8d90 con 0x5591181923f0 2024-05-21T22:05:52.608 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.605+0000 7fea89600700 10 client.17475 handle_command_reply: tid=0 2024-05-21T22:05:52.608 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.605+0000 7feab111bb80 1 client.17475 shutdown 2024-05-21T22:05:52.608 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.606+0000 7feab111bb80 1 -- 172.21.15.151:0/593138545 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55911817e610 msgr2=0x559118182180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:52.608 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.606+0000 7feab111bb80 1 --2- 172.21.15.151:0/593138545 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55911817e610 0x559118182180 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fea8c013460 tx=0x7fea8c0047c0 comp rx=0 tx=0).stop 2024-05-21T22:05:52.608 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.606+0000 7feab111bb80 1 -- 172.21.15.151:0/593138545 shutdown_connections 2024-05-21T22:05:52.608 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.606+0000 7feab111bb80 1 --2- 172.21.15.151:0/593138545 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x5591181923f0 0x5591181947e0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:52.608 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.606+0000 7feab111bb80 1 --2- 172.21.15.151:0/593138545 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55911817ef30 0x5591181826c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:52.608 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.606+0000 7feab111bb80 1 --2- 172.21.15.151:0/593138545 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559118179140 0x559118184de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:52.608 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.606+0000 7feab111bb80 1 --2- 172.21.15.151:0/593138545 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55911817e610 0x559118182180 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:52.608 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.606+0000 7feab111bb80 1 -- 172.21.15.151:0/593138545 >> 172.21.15.151:0/593138545 conn(0x5591180a6be0 msgr2=0x559118172d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:05:52.608 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.606+0000 7fea89600700 0 client.17475 ms_handle_reset on v2:172.21.15.184:6836/2988194888 2024-05-21T22:05:52.609 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.606+0000 7feab111bb80 1 -- 172.21.15.151:0/593138545 shutdown_connections 2024-05-21T22:05:52.609 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.606+0000 7feab111bb80 1 -- 172.21.15.151:0/593138545 wait complete. 2024-05-21T22:05:52.609 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.606+0000 7feab111bb80 20 client.17475 trim_cache size 0 max 16384 2024-05-21T22:05:52.623 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.623+0000 7feab111bb80 1 Processor -- start 2024-05-21T22:05:52.623 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.623+0000 7feab111bb80 1 -- start start 2024-05-21T22:05:52.623 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.623+0000 7feab111bb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559118179140 0x559118178b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:52.623 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.623+0000 7feab111bb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55911817e610 0x55911818e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:52.623 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.623+0000 7feab111bb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55911817ef30 0x55911817c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:52.624 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.623+0000 7feab111bb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7feaa01b0010 con 0x559118179140 2024-05-21T22:05:52.624 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.623+0000 7feab111bb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7feaa01b1b50 con 0x55911817ef30 2024-05-21T22:05:52.624 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.623+0000 7feab111bb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7feaa01b06e0 con 0x55911817e610 2024-05-21T22:05:52.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.623+0000 7fea9f400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559118179140 0x559118178b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:52.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.623+0000 7fea9f400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559118179140 0x559118178b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:43212/0 (socket says 172.21.15.151:43212) 2024-05-21T22:05:52.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.623+0000 7fea9f400700 1 -- 172.21.15.151:0/191898432 learned_addr learned my addr 172.21.15.151:0/191898432 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:05:52.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.624+0000 7fea9ea00700 1 --2- 172.21.15.151:0/191898432 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55911817e610 0x55911818e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:52.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.624+0000 7fea9fe00700 1 --2- 172.21.15.151:0/191898432 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55911817ef30 0x55911817c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:52.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.624+0000 7fea9ea00700 1 -- 172.21.15.151:0/191898432 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55911817ef30 msgr2=0x55911817c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:52.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.624+0000 7fea9ea00700 1 --2- 172.21.15.151:0/191898432 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55911817ef30 0x55911817c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:52.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.624+0000 7fea9ea00700 1 -- 172.21.15.151:0/191898432 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559118179140 msgr2=0x559118178b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:52.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.624+0000 7fea9ea00700 1 --2- 172.21.15.151:0/191898432 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559118179140 0x559118178b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:52.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.624+0000 7fea9ea00700 1 -- 172.21.15.151:0/191898432 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559118184a20 con 0x55911817e610 2024-05-21T22:05:52.626 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.624+0000 7fea9ea00700 1 --2- 172.21.15.151:0/191898432 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55911817e610 0x55911818e800 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fea8c00ffd0 tx=0x7fea8c00f6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f880d22cc0cd990c server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:05:52.626 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.624+0000 7fea8be00700 1 -- 172.21.15.151:0/191898432 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea8c036070 con 0x55911817e610 2024-05-21T22:05:52.626 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.624+0000 7fea8be00700 1 -- 172.21.15.151:0/191898432 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea8c0045f0 con 0x55911817e610 2024-05-21T22:05:52.626 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.624+0000 7fea8be00700 1 -- 172.21.15.151:0/191898432 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea8c02aaa0 con 0x55911817e610 2024-05-21T22:05:52.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.625+0000 7feab111bb80 1 -- 172.21.15.151:0/191898432 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55911817e610 msgr2=0x55911818e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:52.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.625+0000 7feab111bb80 1 --2- 172.21.15.151:0/191898432 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55911817e610 0x55911818e800 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fea8c00ffd0 tx=0x7fea8c00f6a0 comp rx=0 tx=0).stop 2024-05-21T22:05:52.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.625+0000 7feab111bb80 1 -- 172.21.15.151:0/191898432 shutdown_connections 2024-05-21T22:05:52.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.625+0000 7feab111bb80 1 --2- 172.21.15.151:0/191898432 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55911817ef30 0x55911817c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:52.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.625+0000 7feab111bb80 1 --2- 172.21.15.151:0/191898432 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559118179140 0x559118178b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:52.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.625+0000 7feab111bb80 1 --2- 172.21.15.151:0/191898432 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55911817e610 0x55911818e800 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:52.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.625+0000 7feab111bb80 1 -- 172.21.15.151:0/191898432 >> 172.21.15.151:0/191898432 conn(0x5591180a6be0 msgr2=0x55911817c120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:05:52.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.625+0000 7feab111bb80 1 -- 172.21.15.151:0/191898432 shutdown_connections 2024-05-21T22:05:52.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.625+0000 7feab111bb80 1 -- 172.21.15.151:0/191898432 wait complete. 2024-05-21T22:05:52.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.626+0000 7feab111bb80 1 -- start start 2024-05-21T22:05:52.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.626+0000 7feab111bb80 1 Processor -- start 2024-05-21T22:05:52.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.626+0000 7feab111bb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559118179140 0x55911818af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:52.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.627+0000 7feab111bb80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55911817e610 0x559118199780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:52.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.627+0000 7feab111bb80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55911817ef30 0x559118199cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:52.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.627+0000 7feab111bb80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7feaa01b1480 con 0x55911817ef30 2024-05-21T22:05:52.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.627+0000 7feab111bb80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7feaa01b0db0 con 0x55911817e610 2024-05-21T22:05:52.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.627+0000 7feab111bb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7feaa01adfd0 con 0x559118179140 2024-05-21T22:05:52.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.627+0000 7feab111bb80 10 client.0 _ll_register_callbacks cb 0x559118070c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:05:52.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.627+0000 7fea9ea00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55911817e610 0x559118199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:52.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.627+0000 7fea9f400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559118179140 0x55911818af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:52.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.627+0000 7fea9fe00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55911817ef30 0x559118199cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:52.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.627+0000 7fea9ea00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55911817e610 0x559118199780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:57424/0 (socket says 172.21.15.151:57424) 2024-05-21T22:05:52.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.627+0000 7fea9f400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559118179140 0x55911818af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:53058/0 (socket says 172.21.15.151:53058) 2024-05-21T22:05:52.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.627+0000 7fea9ea00700 1 -- 172.21.15.151:0/82800637 learned_addr learned my addr 172.21.15.151:0/82800637 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:05:52.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.627+0000 7fea9ea00700 1 -- 172.21.15.151:0/82800637 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559118179140 msgr2=0x55911818af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:52.630 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.627+0000 7fea9ea00700 1 --2- 172.21.15.151:0/82800637 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559118179140 0x55911818af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:52.630 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.627+0000 7fea9ea00700 1 -- 172.21.15.151:0/82800637 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55911817ef30 msgr2=0x559118199cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:52.630 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.627+0000 7fea9ea00700 1 --2- 172.21.15.151:0/82800637 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55911817ef30 0x559118199cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:52.630 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.627+0000 7fea9ea00700 1 -- 172.21.15.151:0/82800637 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55911819a3c0 con 0x55911817e610 2024-05-21T22:05:52.630 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.627+0000 7fea9fe00700 1 --2- 172.21.15.151:0/82800637 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55911817ef30 0x559118199cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:05:52.630 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.627+0000 7fea9f400700 1 --2- 172.21.15.151:0/82800637 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559118179140 0x55911818af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:05:52.630 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.628+0000 7fea9ea00700 1 --2- 172.21.15.151:0/82800637 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55911817e610 0x559118199780 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fea8c000c00 tx=0x7fea8c00fd20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c13bb532e8fe8ecb server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:05:52.630 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.628+0000 7fea89600700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-21T22:05:52.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.628+0000 7fea89600700 1 -- 172.21.15.151:0/82800637 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea8c036070 con 0x55911817e610 2024-05-21T22:05:52.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.628+0000 7feab111bb80 1 -- 172.21.15.151:0/82800637 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea8c0045f0 con 0x55911817e610 2024-05-21T22:05:52.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.628+0000 7fea89600700 1 -- 172.21.15.151:0/82800637 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7fea8c005030 con 0x55911817e610 2024-05-21T22:05:52.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.628+0000 7fea89600700 1 -- 172.21.15.151:0/82800637 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fea8c005230 con 0x55911817e610 2024-05-21T22:05:52.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.628+0000 7feab111bb80 10 client.27608 fetch_fsmap learned FSMap version 29 2024-05-21T22:05:52.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.628+0000 7feab111bb80 1 -- 172.21.15.151:0/82800637 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559118205630 con 0x55911817e610 2024-05-21T22:05:52.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.628+0000 7fea89600700 1 -- 172.21.15.151:0/82800637 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea8c0045f0 con 0x55911817e610 2024-05-21T22:05:52.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.629+0000 7fea89600700 1 -- 172.21.15.151:0/82800637 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fea8c003680 con 0x55911817e610 2024-05-21T22:05:52.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.629+0000 7feab111bb80 10 client.27608 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:05:52.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.629+0000 7feab111bb80 10 client.27608 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-05-21T22:05:52.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.629+0000 7feab111bb80 20 client.27608 populate_metadata read hostname 'smithi151' 2024-05-21T22:05:52.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.629+0000 7feab111bb80 1 --2- 172.21.15.151:0/82800637 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x559118205c80 0x5591182080a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:52.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.629+0000 7feab111bb80 4 client.27608 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T22:05:52.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.629+0000 7feab111bb80 1 -- 172.21.15.151:0/82800637 --> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5591180d2570 con 0x559118205c80 2024-05-21T22:05:52.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.629+0000 7fea9f400700 1 --2- 172.21.15.151:0/82800637 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x559118205c80 0x5591182080a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:52.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.630+0000 7fea9f400700 1 --2- 172.21.15.151:0/82800637 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x559118205c80 0x5591182080a0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=328ebf862945a5e4 server_cookie=c8fae689fc82a089 in_seq=0 out_seq=0 2024-05-21T22:05:52.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.630+0000 7fea89600700 10 client.27608 ms_handle_connect on v2:172.21.15.184:6836/2988194888 2024-05-21T22:05:52.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.630+0000 7fea89600700 1 -- 172.21.15.151:0/82800637 <== mds.3 v2:172.21.15.184:6836/2988194888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5591180d2570 con 0x559118205c80 2024-05-21T22:05:52.634 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.630+0000 7fea89600700 10 client.27608 handle_command_reply: tid=0 2024-05-21T22:05:52.634 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.630+0000 7feab111bb80 1 client.27608 shutdown 2024-05-21T22:05:52.634 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.630+0000 7feab111bb80 1 -- 172.21.15.151:0/82800637 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55911817e610 msgr2=0x559118199780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:52.634 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.630+0000 7feab111bb80 1 --2- 172.21.15.151:0/82800637 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55911817e610 0x559118199780 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fea8c000c00 tx=0x7fea8c00fd20 comp rx=0 tx=0).stop 2024-05-21T22:05:52.634 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.630+0000 7feab111bb80 1 -- 172.21.15.151:0/82800637 shutdown_connections 2024-05-21T22:05:52.634 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.630+0000 7feab111bb80 1 --2- 172.21.15.151:0/82800637 >> [v2:172.21.15.184:6836/2988194888,v1:172.21.15.184:6837/2988194888] conn(0x559118205c80 0x5591182080a0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:52.634 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.630+0000 7feab111bb80 1 --2- 172.21.15.151:0/82800637 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55911817ef30 0x559118199cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:52.634 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.630+0000 7feab111bb80 1 --2- 172.21.15.151:0/82800637 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559118179140 0x55911818af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:52.634 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.630+0000 7feab111bb80 1 --2- 172.21.15.151:0/82800637 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55911817e610 0x559118199780 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:52.634 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.630+0000 7fea89600700 0 client.27608 ms_handle_reset on v2:172.21.15.184:6836/2988194888 2024-05-21T22:05:52.634 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.630+0000 7feab111bb80 1 -- 172.21.15.151:0/82800637 >> 172.21.15.151:0/82800637 conn(0x5591180a6be0 msgr2=0x55911817f310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:05:52.634 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.630+0000 7feab111bb80 1 -- 172.21.15.151:0/82800637 shutdown_connections 2024-05-21T22:05:52.634 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.631+0000 7feab111bb80 1 -- 172.21.15.151:0/82800637 wait complete. 2024-05-21T22:05:52.634 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.631+0000 7feab111bb80 20 client.27608 trim_cache size 0 max 16384 2024-05-21T22:05:52.635 INFO:teuthology.orchestra.run.smithi151.stdout:[] 2024-05-21T22:05:52.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.631+0000 7feaa6800700 1 -- 172.21.15.151:0/535908654 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fea840b29f0 msgr2=0x7fea840b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:52.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.631+0000 7feaa6800700 1 --2- 172.21.15.151:0/535908654 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fea840b29f0 0x7fea840b4eb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fea8c0097a0 tx=0x7fea8c009b80 comp rx=0 tx=0).stop 2024-05-21T22:05:52.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.631+0000 7feaa6800700 1 -- 172.21.15.151:0/535908654 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feaa0108520 msgr2=0x7feaa01a0780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:52.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.631+0000 7feaa6800700 1 --2- 172.21.15.151:0/535908654 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feaa0108520 0x7feaa01a0780 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fea9800b070 tx=0x7fea98009c90 comp rx=0 tx=0).stop 2024-05-21T22:05:52.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.631+0000 7feaa6800700 1 -- 172.21.15.151:0/535908654 shutdown_connections 2024-05-21T22:05:52.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.631+0000 7feaa6800700 1 --2- 172.21.15.151:0/535908654 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fea840b29f0 0x7fea840b4eb0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:52.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.631+0000 7feaa6800700 1 --2- 172.21.15.151:0/535908654 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feaa0102440 0x7feaa019beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:52.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.631+0000 7feaa6800700 1 --2- 172.21.15.151:0/535908654 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feaa0108520 0x7feaa01a0780 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:52.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.631+0000 7feaa6800700 1 --2- 172.21.15.151:0/535908654 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feaa0102e00 0x7feaa019c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:52.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.631+0000 7feaa6800700 1 -- 172.21.15.151:0/535908654 >> 172.21.15.151:0/535908654 conn(0x7feaa00793f0 msgr2=0x7feaa006b400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:05:52.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.632+0000 7feaa6800700 1 -- 172.21.15.151:0/535908654 shutdown_connections 2024-05-21T22:05:52.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:52.632+0000 7feaa6800700 1 -- 172.21.15.151:0/535908654 wait complete. 2024-05-21T22:05:53.503 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-21T22:05:53.959 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:05:53.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:05:53 smithi184 ceph-mon[36942]: pgmap v317: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 12 op/s 2024-05-21T22:05:54.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:05:53 smithi164 ceph-mon[36269]: pgmap v317: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 12 op/s 2024-05-21T22:05:54.110 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:05:53 smithi151 ceph-mon[31623]: pgmap v317: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 12 op/s 2024-05-21T22:05:54.947 DEBUG:teuthology.orchestra.run.smithi151:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-05-21T22:05:55.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.148+0000 7f44a4c00640 1 -- 172.21.15.151:0/3780169173 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f44a0101580 msgr2=0x7f44a010a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:55.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.148+0000 7f44a4c00640 1 --2- 172.21.15.151:0/3780169173 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f44a0101580 0x7f44a010a9b0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f4488009890 tx=0x7f448802a380 comp rx=0 tx=0).stop 2024-05-21T22:05:55.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.148+0000 7f44a4c00640 1 -- 172.21.15.151:0/3780169173 shutdown_connections 2024-05-21T22:05:55.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.148+0000 7f44a4c00640 1 --2- 172.21.15.151:0/3780169173 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f44a010b0e0 0x7f44a0111a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:55.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.148+0000 7f44a4c00640 1 --2- 172.21.15.151:0/3780169173 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f44a0101580 0x7f44a010a9b0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:55.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.148+0000 7f44a4c00640 1 --2- 172.21.15.151:0/3780169173 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f44a0100bd0 0x7f44a0100fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:55.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.148+0000 7f44a4c00640 1 -- 172.21.15.151:0/3780169173 >> 172.21.15.151:0/3780169173 conn(0x7f44a00778b0 msgr2=0x7f44a00fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:05:55.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.148+0000 7f44a4c00640 1 -- 172.21.15.151:0/3780169173 shutdown_connections 2024-05-21T22:05:55.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.148+0000 7f44a4c00640 1 -- 172.21.15.151:0/3780169173 wait complete. 2024-05-21T22:05:55.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.149+0000 7f44a4c00640 1 Processor -- start 2024-05-21T22:05:55.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.149+0000 7f44a4c00640 1 -- start start 2024-05-21T22:05:55.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.150+0000 7f44a4c00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f44a0100bd0 0x7f44a019b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:55.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.150+0000 7f44a4c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f44a010b0e0 0x7f44a019b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:55.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.150+0000 7f44a4c00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f44a019fb70 0x7f44a01a0020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:55.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.150+0000 7f44a4c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f44a0114d30 con 0x7f44a010b0e0 2024-05-21T22:05:55.152 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.150+0000 7f44a4c00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f44a0114bb0 con 0x7f44a019fb70 2024-05-21T22:05:55.152 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.150+0000 7f44a4c00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f44a0114eb0 con 0x7f44a0100bd0 2024-05-21T22:05:55.152 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.150+0000 7f449f400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f44a0100bd0 0x7f44a019b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:55.152 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.150+0000 7f449f400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f44a0100bd0 0x7f44a019b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:53078/0 (socket says 172.21.15.151:53078) 2024-05-21T22:05:55.152 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.150+0000 7f449f400640 1 -- 172.21.15.151:0/1823500171 learned_addr learned my addr 172.21.15.151:0/1823500171 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:05:55.152 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.150+0000 7f449ea00640 1 --2- 172.21.15.151:0/1823500171 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f44a010b0e0 0x7f44a019b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:55.152 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.150+0000 7f449fe00640 1 --2- 172.21.15.151:0/1823500171 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f44a019fb70 0x7f44a01a0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:55.152 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.151+0000 7f449f400640 1 -- 172.21.15.151:0/1823500171 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f44a019fb70 msgr2=0x7f44a01a0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:55.152 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.151+0000 7f449f400640 1 --2- 172.21.15.151:0/1823500171 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f44a019fb70 0x7f44a01a0020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:55.152 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.151+0000 7f449f400640 1 -- 172.21.15.151:0/1823500171 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f44a010b0e0 msgr2=0x7f44a019b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:55.152 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.151+0000 7f449f400640 1 --2- 172.21.15.151:0/1823500171 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f44a010b0e0 0x7f44a019b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:55.152 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.151+0000 7f449f400640 1 -- 172.21.15.151:0/1823500171 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44a01a0660 con 0x7f44a0100bd0 2024-05-21T22:05:55.153 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.151+0000 7f449fe00640 1 --2- 172.21.15.151:0/1823500171 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f44a019fb70 0x7f44a01a0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:05:55.153 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.151+0000 7f449f400640 1 --2- 172.21.15.151:0/1823500171 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f44a0100bd0 0x7f44a019b2b0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f449000efb0 tx=0x7f449000c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:05:55.153 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.151+0000 7f4497e00640 1 -- 172.21.15.151:0/1823500171 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f449001b070 con 0x7f44a0100bd0 2024-05-21T22:05:55.153 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.151+0000 7f44a4c00640 1 -- 172.21.15.151:0/1823500171 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44a01a0950 con 0x7f44a0100bd0 2024-05-21T22:05:55.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.151+0000 7f4497e00640 1 -- 172.21.15.151:0/1823500171 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f449000ba20 con 0x7f44a0100bd0 2024-05-21T22:05:55.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.151+0000 7f4497e00640 1 -- 172.21.15.151:0/1823500171 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4490016b30 con 0x7f44a0100bd0 2024-05-21T22:05:55.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.151+0000 7f44a4c00640 1 -- 172.21.15.151:0/1823500171 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44a01a8160 con 0x7f44a0100bd0 2024-05-21T22:05:55.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.152+0000 7f44a4c00640 1 -- 172.21.15.151:0/1823500171 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44a0051030 con 0x7f44a0100bd0 2024-05-21T22:05:55.155 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.155+0000 7f4497e00640 1 -- 172.21.15.151:0/1823500171 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7f449001e070 con 0x7f44a0100bd0 2024-05-21T22:05:55.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.157+0000 7f4497e00640 1 --2- 172.21.15.151:0/1823500171 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f44840b29d0 0x7f44840b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:55.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.157+0000 7f4497e00640 1 -- 172.21.15.151:0/1823500171 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7f44900e2b10 con 0x7f44a0100bd0 2024-05-21T22:05:55.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.157+0000 7f449ea00640 1 --2- 172.21.15.151:0/1823500171 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f44840b29d0 0x7f44840b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:55.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.158+0000 7f449ea00640 1 --2- 172.21.15.151:0/1823500171 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f44840b29d0 0x7f44840b4e90 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f44880303f0 tx=0x7f4488030340 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:05:55.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.158+0000 7f4497e00640 1 -- 172.21.15.151:0/1823500171 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f44900a9e30 con 0x7f44a0100bd0 2024-05-21T22:05:55.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.406+0000 7f44a4c00640 1 -- 172.21.15.151:0/1823500171 --> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f44a0101580 con 0x7f44840b29d0 2024-05-21T22:05:55.422 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.422+0000 7f4497e00640 1 -- 172.21.15.151:0/1823500171 <== mgr.25138 v2:172.21.15.151:6800/958417056 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7f44a0101580 con 0x7f44840b29d0 2024-05-21T22:05:55.422 INFO:teuthology.orchestra.run.smithi151.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-05-21T00:00:00", "created": "2024-05-21T21:55:52", "first": "2024-05-21T21:56:00", "last": "2024-05-21T22:05:00", "last_pruned": "2024-05-21T22:04:00", "created_count": 10, "pruned_count": 3, "active": true} 2024-05-21T22:05:55.426 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.426+0000 7f44a4c00640 1 -- 172.21.15.151:0/1823500171 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f44840b29d0 msgr2=0x7f44840b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:55.426 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.426+0000 7f44a4c00640 1 --2- 172.21.15.151:0/1823500171 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f44840b29d0 0x7f44840b4e90 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f44880303f0 tx=0x7f4488030340 comp rx=0 tx=0).stop 2024-05-21T22:05:55.426 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.426+0000 7f44a4c00640 1 -- 172.21.15.151:0/1823500171 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f44a0100bd0 msgr2=0x7f44a019b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:55.426 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.426+0000 7f44a4c00640 1 --2- 172.21.15.151:0/1823500171 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f44a0100bd0 0x7f44a019b2b0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f449000efb0 tx=0x7f449000c800 comp rx=0 tx=0).stop 2024-05-21T22:05:55.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.426+0000 7f44a4c00640 1 -- 172.21.15.151:0/1823500171 shutdown_connections 2024-05-21T22:05:55.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.426+0000 7f44a4c00640 1 --2- 172.21.15.151:0/1823500171 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f44840b29d0 0x7f44840b4e90 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:55.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.426+0000 7f44a4c00640 1 --2- 172.21.15.151:0/1823500171 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f44a019fb70 0x7f44a01a0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:55.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.426+0000 7f44a4c00640 1 --2- 172.21.15.151:0/1823500171 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f44a010b0e0 0x7f44a019b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:55.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.426+0000 7f44a4c00640 1 --2- 172.21.15.151:0/1823500171 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f44a0100bd0 0x7f44a019b2b0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:55.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.426+0000 7f44a4c00640 1 -- 172.21.15.151:0/1823500171 >> 172.21.15.151:0/1823500171 conn(0x7f44a00778b0 msgr2=0x7f44a00fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:05:55.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.426+0000 7f44a4c00640 1 -- 172.21.15.151:0/1823500171 shutdown_connections 2024-05-21T22:05:55.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.426+0000 7f44a4c00640 1 -- 172.21.15.151:0/1823500171 wait complete. 2024-05-21T22:05:55.447 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:05:55 smithi151 ceph-mon[31623]: pgmap v318: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-05-21T22:05:55.450 DEBUG:teuthology.orchestra.run.smithi151:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-05-21T22:05:55.485 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:05:55 smithi184 ceph-mon[36942]: pgmap v318: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-05-21T22:05:55.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:05:55 smithi164 ceph-mon[36269]: pgmap v318: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-05-21T22:05:55.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.634+0000 7f8b6d400640 1 -- 172.21.15.151:0/3625132978 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b68101860 msgr2=0x7f8b68101c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:55.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.634+0000 7f8b6d400640 1 --2- 172.21.15.151:0/3625132978 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b68101860 0x7f8b68101c40 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f8b58009900 tx=0x7f8b5802a410 comp rx=0 tx=0).stop 2024-05-21T22:05:55.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.634+0000 7f8b6d400640 1 -- 172.21.15.151:0/3625132978 shutdown_connections 2024-05-21T22:05:55.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.634+0000 7f8b6d400640 1 --2- 172.21.15.151:0/3625132978 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8b6810f410 0x7f8b68111800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:55.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.634+0000 7f8b6d400640 1 --2- 172.21.15.151:0/3625132978 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8b68102180 0x7f8b6810eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:55.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.634+0000 7f8b6d400640 1 --2- 172.21.15.151:0/3625132978 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b68101860 0x7f8b68101c40 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:55.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.634+0000 7f8b6d400640 1 -- 172.21.15.151:0/3625132978 >> 172.21.15.151:0/3625132978 conn(0x7f8b68078ad0 msgr2=0x7f8b68078ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:05:55.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.634+0000 7f8b6d400640 1 -- 172.21.15.151:0/3625132978 shutdown_connections 2024-05-21T22:05:55.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.634+0000 7f8b6d400640 1 -- 172.21.15.151:0/3625132978 wait complete. 2024-05-21T22:05:55.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.636+0000 7f8b6d400640 1 Processor -- start 2024-05-21T22:05:55.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.636+0000 7f8b6d400640 1 -- start start 2024-05-21T22:05:55.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.636+0000 7f8b6d400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b68101860 0x7f8b6819b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:55.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.636+0000 7f8b6d400640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8b68102180 0x7f8b6819b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:55.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.636+0000 7f8b6d400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8b6810f410 0x7f8b6819fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:55.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.636+0000 7f8b6d400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8b68114f40 con 0x7f8b6810f410 2024-05-21T22:05:55.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.636+0000 7f8b6d400640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8b68114dc0 con 0x7f8b68102180 2024-05-21T22:05:55.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.636+0000 7f8b6d400640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f8b681150c0 con 0x7f8b68101860 2024-05-21T22:05:55.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.637+0000 7f8b67200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8b6810f410 0x7f8b6819fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:55.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.637+0000 7f8b66800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b68101860 0x7f8b6819b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:55.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.637+0000 7f8b66800640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b68101860 0x7f8b6819b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:53090/0 (socket says 172.21.15.151:53090) 2024-05-21T22:05:55.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.637+0000 7f8b5fe00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8b68102180 0x7f8b6819b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:55.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.637+0000 7f8b66800640 1 -- 172.21.15.151:0/3810208159 learned_addr learned my addr 172.21.15.151:0/3810208159 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:05:55.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.637+0000 7f8b66800640 1 -- 172.21.15.151:0/3810208159 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8b68102180 msgr2=0x7f8b6819b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:55.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.637+0000 7f8b66800640 1 --2- 172.21.15.151:0/3810208159 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8b68102180 0x7f8b6819b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:55.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.637+0000 7f8b66800640 1 -- 172.21.15.151:0/3810208159 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8b6810f410 msgr2=0x7f8b6819fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:55.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.637+0000 7f8b66800640 1 --2- 172.21.15.151:0/3810208159 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8b6810f410 0x7f8b6819fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:55.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.637+0000 7f8b66800640 1 -- 172.21.15.151:0/3810208159 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b681a0470 con 0x7f8b68101860 2024-05-21T22:05:55.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.637+0000 7f8b66800640 1 --2- 172.21.15.151:0/3810208159 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b68101860 0x7f8b6819b480 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f8b58005ab0 tx=0x7f8b58003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:05:55.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.638+0000 7f8b5f400640 1 -- 172.21.15.151:0/3810208159 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b5803f070 con 0x7f8b68101860 2024-05-21T22:05:55.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.638+0000 7f8b5f400640 1 -- 172.21.15.151:0/3810208159 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b58031b60 con 0x7f8b68101860 2024-05-21T22:05:55.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.638+0000 7f8b5f400640 1 -- 172.21.15.151:0/3810208159 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b58035030 con 0x7f8b68101860 2024-05-21T22:05:55.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.638+0000 7f8b6d400640 1 -- 172.21.15.151:0/3810208159 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b681a0700 con 0x7f8b68101860 2024-05-21T22:05:55.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.638+0000 7f8b6d400640 1 -- 172.21.15.151:0/3810208159 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b681a7ff0 con 0x7f8b68101860 2024-05-21T22:05:55.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.639+0000 7f8b6d400640 1 -- 172.21.15.151:0/3810208159 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b6806a900 con 0x7f8b68101860 2024-05-21T22:05:55.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.641+0000 7f8b5f400640 1 -- 172.21.15.151:0/3810208159 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7f8b58042070 con 0x7f8b68101860 2024-05-21T22:05:55.643 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.643+0000 7f8b5f400640 1 --2- 172.21.15.151:0/3810208159 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f8b4c0b2a90 0x7f8b4c0b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:55.643 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.643+0000 7f8b5f400640 1 -- 172.21.15.151:0/3810208159 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7f8b58103300 con 0x7f8b68101860 2024-05-21T22:05:55.643 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.643+0000 7f8b5fe00640 1 --2- 172.21.15.151:0/3810208159 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f8b4c0b2a90 0x7f8b4c0b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:55.644 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.644+0000 7f8b5fe00640 1 --2- 172.21.15.151:0/3810208159 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f8b4c0b2a90 0x7f8b4c0b4f50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8b681033e0 tx=0x7f8b50005f90 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:05:55.645 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.645+0000 7f8b5f400640 1 -- 172.21.15.151:0/3810208159 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8b580ca620 con 0x7f8b68101860 2024-05-21T22:05:55.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.887+0000 7f8b6d400640 1 -- 172.21.15.151:0/3810208159 --> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f8b6810a610 con 0x7f8b4c0b2a90 2024-05-21T22:05:55.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.902+0000 7f8b5f400640 1 -- 172.21.15.151:0/3810208159 <== mgr.25138 v2:172.21.15.151:6800/958417056 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f8b5001b3f0 con 0x7f8b4c0b2a90 2024-05-21T22:05:55.902 INFO:teuthology.orchestra.run.smithi151.stdout:/ 1m 6m3h 2024-05-21T22:05:55.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.904+0000 7f8b6d400640 1 -- 172.21.15.151:0/3810208159 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f8b4c0b2a90 msgr2=0x7f8b4c0b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:55.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.904+0000 7f8b6d400640 1 --2- 172.21.15.151:0/3810208159 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f8b4c0b2a90 0x7f8b4c0b4f50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8b681033e0 tx=0x7f8b50005f90 comp rx=0 tx=0).stop 2024-05-21T22:05:55.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.904+0000 7f8b6d400640 1 -- 172.21.15.151:0/3810208159 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b68101860 msgr2=0x7f8b6819b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:55.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.904+0000 7f8b6d400640 1 --2- 172.21.15.151:0/3810208159 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b68101860 0x7f8b6819b480 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f8b58005ab0 tx=0x7f8b58003c10 comp rx=0 tx=0).stop 2024-05-21T22:05:55.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.905+0000 7f8b6d400640 1 -- 172.21.15.151:0/3810208159 shutdown_connections 2024-05-21T22:05:55.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.905+0000 7f8b6d400640 1 --2- 172.21.15.151:0/3810208159 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f8b4c0b2a90 0x7f8b4c0b4f50 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:55.906 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.905+0000 7f8b6d400640 1 --2- 172.21.15.151:0/3810208159 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8b6810f410 0x7f8b6819fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:55.906 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.905+0000 7f8b6d400640 1 --2- 172.21.15.151:0/3810208159 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8b68102180 0x7f8b6819b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:55.906 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.905+0000 7f8b6d400640 1 --2- 172.21.15.151:0/3810208159 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b68101860 0x7f8b6819b480 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:55.906 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.905+0000 7f8b6d400640 1 -- 172.21.15.151:0/3810208159 >> 172.21.15.151:0/3810208159 conn(0x7f8b68078ad0 msgr2=0x7f8b68110a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:05:55.906 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.905+0000 7f8b6d400640 1 -- 172.21.15.151:0/3810208159 shutdown_connections 2024-05-21T22:05:55.906 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:55.905+0000 7f8b6d400640 1 -- 172.21.15.151:0/3810208159 wait complete. 2024-05-21T22:05:55.926 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-05-21T22:05:55.938 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-05-21T22:05:56.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.079+0000 7f74e6200700 1 -- 172.21.15.151:0/3700973991 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74e01035c0 msgr2=0x7f74e0103a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:56.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.079+0000 7f74e6200700 1 --2- 172.21.15.151:0/3700973991 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74e01035c0 0x7f74e0103a40 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f74c8009990 tx=0x7f74c8009ca0 comp rx=0 tx=0).stop 2024-05-21T22:05:56.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.079+0000 7f74e6200700 1 -- 172.21.15.151:0/3700973991 shutdown_connections 2024-05-21T22:05:56.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.079+0000 7f74e6200700 1 --2- 172.21.15.151:0/3700973991 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74e01095d0 0x7f74e01099b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:56.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.079+0000 7f74e6200700 1 --2- 172.21.15.151:0/3700973991 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74e0103f80 0x7f74e0110410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:56.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.079+0000 7f74e6200700 1 --2- 172.21.15.151:0/3700973991 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74e01035c0 0x7f74e0103a40 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:56.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.079+0000 7f74e6200700 1 -- 172.21.15.151:0/3700973991 >> 172.21.15.151:0/3700973991 conn(0x7f74e007a610 msgr2=0x7f74e007aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:05:56.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.079+0000 7f74e6200700 1 -- 172.21.15.151:0/3700973991 shutdown_connections 2024-05-21T22:05:56.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.080+0000 7f74e6200700 1 -- 172.21.15.151:0/3700973991 wait complete. 2024-05-21T22:05:56.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.080+0000 7f74e6200700 1 Processor -- start 2024-05-21T22:05:56.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.081+0000 7f74e6200700 1 -- start start 2024-05-21T22:05:56.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.081+0000 7f74e6200700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74e01035c0 0x7f74e0075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:56.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.081+0000 7f74e6200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74e0103f80 0x7f74e0075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:56.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.081+0000 7f74e6200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74e01095d0 0x7f74e00762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:56.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.081+0000 7f74e6200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f74e0113780 con 0x7f74e0103f80 2024-05-21T22:05:56.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.081+0000 7f74e6200700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f74e0113600 con 0x7f74e01095d0 2024-05-21T22:05:56.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.081+0000 7f74e6200700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f74e0113900 con 0x7f74e01035c0 2024-05-21T22:05:56.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.081+0000 7f74d6a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74e0103f80 0x7f74e0075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:56.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.081+0000 7f74d6a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74e0103f80 0x7f74e0075d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:53266/0 (socket says 172.21.15.151:53266) 2024-05-21T22:05:56.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.081+0000 7f74d6a00700 1 -- 172.21.15.151:0/2555483587 learned_addr learned my addr 172.21.15.151:0/2555483587 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:05:56.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.081+0000 7f74df400700 1 --2- 172.21.15.151:0/2555483587 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74e01035c0 0x7f74e0075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:56.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.082+0000 7f74dfe00700 1 --2- 172.21.15.151:0/2555483587 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74e01095d0 0x7f74e00762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:56.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.082+0000 7f74df400700 1 -- 172.21.15.151:0/2555483587 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74e01095d0 msgr2=0x7f74e00762c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:56.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.082+0000 7f74df400700 1 --2- 172.21.15.151:0/2555483587 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74e01095d0 0x7f74e00762c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:56.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.082+0000 7f74df400700 1 -- 172.21.15.151:0/2555483587 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74e0103f80 msgr2=0x7f74e0075d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:56.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.082+0000 7f74df400700 1 --2- 172.21.15.151:0/2555483587 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74e0103f80 0x7f74e0075d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:56.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.082+0000 7f74df400700 1 -- 172.21.15.151:0/2555483587 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74e01a8160 con 0x7f74e01035c0 2024-05-21T22:05:56.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.082+0000 7f74d6a00700 1 --2- 172.21.15.151:0/2555483587 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74e0103f80 0x7f74e0075d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:05:56.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.082+0000 7f74df400700 1 --2- 172.21.15.151:0/2555483587 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74e01035c0 0x7f74e0075840 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f74d000ef10 tx=0x7f74d000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:05:56.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.082+0000 7f74dcc00700 1 -- 172.21.15.151:0/2555483587 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d000c9c0 con 0x7f74e01035c0 2024-05-21T22:05:56.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.083+0000 7f74e6200700 1 -- 172.21.15.151:0/2555483587 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74e01a83f0 con 0x7f74e01035c0 2024-05-21T22:05:56.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.083+0000 7f74dcc00700 1 -- 172.21.15.151:0/2555483587 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74d0019070 con 0x7f74e01035c0 2024-05-21T22:05:56.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.083+0000 7f74dcc00700 1 -- 172.21.15.151:0/2555483587 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d0014480 con 0x7f74e01035c0 2024-05-21T22:05:56.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.083+0000 7f74e6200700 1 -- 172.21.15.151:0/2555483587 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74e01a8930 con 0x7f74e01035c0 2024-05-21T22:05:56.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.088+0000 7f74dcc00700 1 -- 172.21.15.151:0/2555483587 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7f74d0012050 con 0x7f74e01035c0 2024-05-21T22:05:56.089 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.088+0000 7f74f0bbab80 1 Processor -- start 2024-05-21T22:05:56.089 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.089+0000 7f74f0bbab80 1 -- start start 2024-05-21T22:05:56.090 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.089+0000 7f74f0bbab80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d72ac48610 0x55d72ac489f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:56.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.089+0000 7f74f0bbab80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d72ac48f30 0x55d72ac42b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:56.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.089+0000 7f74f0bbab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d72ac43140 0x55d72ac435c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:56.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.089+0000 7f74f0bbab80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d72aba0e40 con 0x55d72ac48610 2024-05-21T22:05:56.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.089+0000 7f74f0bbab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55d72ab909e0 con 0x55d72ac43140 2024-05-21T22:05:56.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.089+0000 7f74f0bbab80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55d72ab94a20 con 0x55d72ac48f30 2024-05-21T22:05:56.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.089+0000 7f74dfe00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d72ac48610 0x55d72ac489f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:56.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.089+0000 7f74df400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d72ac48f30 0x55d72ac42b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:56.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.089+0000 7f74dfe00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d72ac48610 0x55d72ac489f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:53286/0 (socket says 172.21.15.151:53286) 2024-05-21T22:05:56.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.089+0000 7f74dfe00700 1 -- 172.21.15.151:0/3254733237 learned_addr learned my addr 172.21.15.151:0/3254733237 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:05:56.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.089+0000 7f74df400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d72ac48f30 0x55d72ac42b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:52918/0 (socket says 172.21.15.151:52918) 2024-05-21T22:05:56.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.090+0000 7f74d6a00700 1 --2- 172.21.15.151:0/3254733237 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d72ac43140 0x55d72ac435c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:56.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.090+0000 7f74dcc00700 1 --2- 172.21.15.151:0/2555483587 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f74c40b29f0 0x7f74c40b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:56.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.090+0000 7f74df400700 1 -- 172.21.15.151:0/3254733237 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d72ac43140 msgr2=0x55d72ac435c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:56.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.090+0000 7f74df400700 1 --2- 172.21.15.151:0/3254733237 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d72ac43140 0x55d72ac435c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:56.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.090+0000 7f74df400700 1 -- 172.21.15.151:0/3254733237 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d72ac48610 msgr2=0x55d72ac489f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:56.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.090+0000 7f74df400700 1 --2- 172.21.15.151:0/3254733237 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d72ac48610 0x55d72ac489f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:56.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.090+0000 7f74df400700 1 -- 172.21.15.151:0/3254733237 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d72ab59420 con 0x55d72ac48f30 2024-05-21T22:05:56.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.090+0000 7f74dcc00700 1 -- 172.21.15.151:0/2555483587 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7f74d00e2590 con 0x7f74e01035c0 2024-05-21T22:05:56.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.090+0000 7f74dfe00700 1 --2- 172.21.15.151:0/2555483587 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f74c40b29f0 0x7f74c40b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:56.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.090+0000 7f74df400700 1 --2- 172.21.15.151:0/3254733237 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d72ac48f30 0x55d72ac42b20 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f74d00dfc70 tx=0x7f74d0009eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d49b8ceb0e2bf20 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:05:56.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.091+0000 7f74d4c00700 1 -- 172.21.15.151:0/3254733237 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d00a95d0 con 0x55d72ac48f30 2024-05-21T22:05:56.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.091+0000 7f74d4c00700 1 -- 172.21.15.151:0/3254733237 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74d00a9310 con 0x55d72ac48f30 2024-05-21T22:05:56.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.091+0000 7f74d4c00700 1 -- 172.21.15.151:0/3254733237 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d00a8f70 con 0x55d72ac48f30 2024-05-21T22:05:56.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.091+0000 7f74f0bbab80 1 -- 172.21.15.151:0/3254733237 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d72ac48f30 msgr2=0x55d72ac42b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:56.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.091+0000 7f74dfe00700 1 --2- 172.21.15.151:0/2555483587 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f74c40b29f0 0x7f74c40b4eb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f74d80117e0 tx=0x7f74d80047c0 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:05:56.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.091+0000 7f74f0bbab80 1 --2- 172.21.15.151:0/3254733237 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d72ac48f30 0x55d72ac42b20 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f74d00dfc70 tx=0x7f74d0009eb0 comp rx=0 tx=0).stop 2024-05-21T22:05:56.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.092+0000 7f74f0bbab80 1 -- 172.21.15.151:0/3254733237 shutdown_connections 2024-05-21T22:05:56.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.092+0000 7f74f0bbab80 1 --2- 172.21.15.151:0/3254733237 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d72ac43140 0x55d72ac435c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:56.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.092+0000 7f74f0bbab80 1 --2- 172.21.15.151:0/3254733237 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d72ac48610 0x55d72ac489f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:56.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.092+0000 7f74f0bbab80 1 --2- 172.21.15.151:0/3254733237 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d72ac48f30 0x55d72ac42b20 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:56.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.092+0000 7f74f0bbab80 1 -- 172.21.15.151:0/3254733237 >> 172.21.15.151:0/3254733237 conn(0x55d72ab70be0 msgr2=0x55d72ac3d690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:05:56.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.092+0000 7f74f0bbab80 1 -- 172.21.15.151:0/3254733237 shutdown_connections 2024-05-21T22:05:56.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.092+0000 7f74f0bbab80 1 -- 172.21.15.151:0/3254733237 wait complete. 2024-05-21T22:05:56.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.092+0000 7f74f0bbab80 1 -- start start 2024-05-21T22:05:56.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.093+0000 7f74f0bbab80 1 Processor -- start 2024-05-21T22:05:56.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.093+0000 7f74f0bbab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d72ac43140 0x55d72ac4ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:56.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.093+0000 7f74f0bbab80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d72ac48610 0x55d72ac4bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:56.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.093+0000 7f74f0bbab80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d72ac48f30 0x55d72ac4c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:56.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.093+0000 7f74f0bbab80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d72aa96fa0 con 0x55d72ac48610 2024-05-21T22:05:56.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.093+0000 7f74f0bbab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55d72ab31760 con 0x55d72ac43140 2024-05-21T22:05:56.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.093+0000 7f74f0bbab80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55d72ac4f980 con 0x55d72ac48f30 2024-05-21T22:05:56.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.093+0000 7f74f0bbab80 10 client.0 _ll_register_callbacks cb 0x55d72ab3ac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:05:56.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.094+0000 7f74d6a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d72ac48610 0x55d72ac4bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:56.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.094+0000 7f74dfe00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d72ac48f30 0x55d72ac4c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:56.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.094+0000 7f74df400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d72ac43140 0x55d72ac4ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:56.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.094+0000 7f74d6a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d72ac48610 0x55d72ac4bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:53290/0 (socket says 172.21.15.151:53290) 2024-05-21T22:05:56.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.094+0000 7f74d6a00700 1 -- 172.21.15.151:0/2437403774 learned_addr learned my addr 172.21.15.151:0/2437403774 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:05:56.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.094+0000 7f74df400700 1 -- 172.21.15.151:0/2437403774 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d72ac48f30 msgr2=0x55d72ac4c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:56.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.094+0000 7f74df400700 1 --2- 172.21.15.151:0/2437403774 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d72ac48f30 0x55d72ac4c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:56.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.094+0000 7f74df400700 1 -- 172.21.15.151:0/2437403774 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d72ac48610 msgr2=0x55d72ac4bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:56.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.094+0000 7f74df400700 1 --2- 172.21.15.151:0/2437403774 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d72ac48610 0x55d72ac4bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:56.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.094+0000 7f74df400700 1 -- 172.21.15.151:0/2437403774 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d72aae08d0 con 0x55d72ac43140 2024-05-21T22:05:56.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.094+0000 7f74d6a00700 1 --2- 172.21.15.151:0/2437403774 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d72ac48610 0x55d72ac4bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:05:56.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.094+0000 7f74df400700 1 --2- 172.21.15.151:0/2437403774 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d72ac43140 0x55d72ac4ebf0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f74d00a9500 tx=0x7f74d000cf40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee4d7c96892acd94 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:05:56.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.094+0000 7f74cd600700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-21T22:05:56.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.095+0000 7f74cd600700 1 -- 172.21.15.151:0/2437403774 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d00a9150 con 0x55d72ac43140 2024-05-21T22:05:56.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.095+0000 7f74f0bbab80 1 -- 172.21.15.151:0/2437403774 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74d00a9310 con 0x55d72ac43140 2024-05-21T22:05:56.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.095+0000 7f74cd600700 1 -- 172.21.15.151:0/2437403774 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7f74d00a9c60 con 0x55d72ac43140 2024-05-21T22:05:56.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.095+0000 7f74cd600700 1 -- 172.21.15.151:0/2437403774 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f74d00a8f70 con 0x55d72ac43140 2024-05-21T22:05:56.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.095+0000 7f74f0bbab80 10 client.27632 fetch_fsmap learned FSMap version 29 2024-05-21T22:05:56.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.095+0000 7f74f0bbab80 1 -- 172.21.15.151:0/2437403774 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d72ab61060 con 0x55d72ac43140 2024-05-21T22:05:56.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.095+0000 7f74cd600700 1 -- 172.21.15.151:0/2437403774 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74d00a9310 con 0x55d72ac43140 2024-05-21T22:05:56.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.096+0000 7f74cd600700 1 -- 172.21.15.151:0/2437403774 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f74d00a8a60 con 0x55d72ac43140 2024-05-21T22:05:56.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.096+0000 7f74f0bbab80 10 client.27632 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:05:56.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.096+0000 7f74f0bbab80 10 client.27632 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-05-21T22:05:56.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.096+0000 7f74f0bbab80 20 client.27632 populate_metadata read hostname 'smithi151' 2024-05-21T22:05:56.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.096+0000 7f74f0bbab80 1 --2- 172.21.15.151:0/2437403774 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x55d72ac5c2e0 0x55d72ac5e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:56.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.096+0000 7f74f0bbab80 4 client.27632 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T22:05:56.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.096+0000 7f74f0bbab80 1 -- 172.21.15.151:0/2437403774 --> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d72aba2d90 con 0x55d72ac5c2e0 2024-05-21T22:05:56.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.096+0000 7f74d6a00700 1 --2- 172.21.15.151:0/2437403774 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x55d72ac5c2e0 0x55d72ac5e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:56.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.097+0000 7f74d6a00700 1 --2- 172.21.15.151:0/2437403774 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x55d72ac5c2e0 0x55d72ac5e6d0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2f0298056a8b759c server_cookie=7405f3d0157da961 in_seq=0 out_seq=0 2024-05-21T22:05:56.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.097+0000 7f74cd600700 10 client.27632 ms_handle_connect on v2:172.21.15.164:6836/1554609572 2024-05-21T22:05:56.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.097+0000 7f74cd600700 1 -- 172.21.15.151:0/2437403774 <== mds.2 v2:172.21.15.164:6836/1554609572 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55d72aba2d90 con 0x55d72ac5c2e0 2024-05-21T22:05:56.102 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.097+0000 7f74cd600700 10 client.27632 handle_command_reply: tid=0 2024-05-21T22:05:56.102 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.098+0000 7f74f0bbab80 1 client.27632 shutdown 2024-05-21T22:05:56.102 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.098+0000 7f74f0bbab80 1 -- 172.21.15.151:0/2437403774 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d72ac43140 msgr2=0x55d72ac4ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:56.102 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.098+0000 7f74f0bbab80 1 --2- 172.21.15.151:0/2437403774 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d72ac43140 0x55d72ac4ebf0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f74d00a9500 tx=0x7f74d000cf40 comp rx=0 tx=0).stop 2024-05-21T22:05:56.102 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.098+0000 7f74f0bbab80 1 -- 172.21.15.151:0/2437403774 shutdown_connections 2024-05-21T22:05:56.102 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.098+0000 7f74f0bbab80 1 --2- 172.21.15.151:0/2437403774 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x55d72ac5c2e0 0x55d72ac5e6d0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:56.102 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.098+0000 7f74f0bbab80 1 --2- 172.21.15.151:0/2437403774 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d72ac43140 0x55d72ac4ebf0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:56.102 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.098+0000 7f74f0bbab80 1 --2- 172.21.15.151:0/2437403774 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d72ac48f30 0x55d72ac4c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:56.102 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.098+0000 7f74f0bbab80 1 --2- 172.21.15.151:0/2437403774 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d72ac48610 0x55d72ac4bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:56.102 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.098+0000 7f74f0bbab80 1 -- 172.21.15.151:0/2437403774 >> 172.21.15.151:0/2437403774 conn(0x55d72ab70be0 msgr2=0x55d72ac3cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:05:56.102 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.098+0000 7f74cd600700 0 client.27632 ms_handle_reset on v2:172.21.15.164:6836/1554609572 2024-05-21T22:05:56.102 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.098+0000 7f74f0bbab80 1 -- 172.21.15.151:0/2437403774 shutdown_connections 2024-05-21T22:05:56.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.098+0000 7f74f0bbab80 1 -- 172.21.15.151:0/2437403774 wait complete. 2024-05-21T22:05:56.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.098+0000 7f74f0bbab80 20 client.27632 trim_cache size 0 max 16384 2024-05-21T22:05:56.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.114+0000 7f74f0bbab80 1 Processor -- start 2024-05-21T22:05:56.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.115+0000 7f74f0bbab80 1 -- start start 2024-05-21T22:05:56.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.115+0000 7f74f0bbab80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d72ac43140 0x55d72ac49ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:56.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.115+0000 7f74f0bbab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d72ac48610 0x55d72ac3d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:56.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.115+0000 7f74f0bbab80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d72ac48f30 0x55d72ac57c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:56.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.115+0000 7f74f0bbab80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f74e01b01a0 con 0x55d72ac43140 2024-05-21T22:05:56.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.115+0000 7f74f0bbab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f74e01b1ce0 con 0x55d72ac48610 2024-05-21T22:05:56.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.115+0000 7f74f0bbab80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f74e01b0870 con 0x55d72ac48f30 2024-05-21T22:05:56.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.115+0000 7f74df400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d72ac43140 0x55d72ac49ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:56.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.115+0000 7f74df400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d72ac43140 0x55d72ac49ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:53306/0 (socket says 172.21.15.151:53306) 2024-05-21T22:05:56.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.115+0000 7f74df400700 1 -- 172.21.15.151:0/1419489728 learned_addr learned my addr 172.21.15.151:0/1419489728 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:05:56.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.115+0000 7f74dfe00700 1 --2- 172.21.15.151:0/1419489728 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d72ac48f30 0x55d72ac57c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:56.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.115+0000 7f74d6a00700 1 --2- 172.21.15.151:0/1419489728 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d72ac48610 0x55d72ac3d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:56.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.116+0000 7f74df400700 1 -- 172.21.15.151:0/1419489728 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d72ac48f30 msgr2=0x55d72ac57c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:56.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.116+0000 7f74df400700 1 --2- 172.21.15.151:0/1419489728 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d72ac48f30 0x55d72ac57c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:56.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.116+0000 7f74df400700 1 -- 172.21.15.151:0/1419489728 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d72ac48610 msgr2=0x55d72ac3d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:56.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.116+0000 7f74df400700 1 --2- 172.21.15.151:0/1419489728 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d72ac48610 0x55d72ac3d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:56.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.116+0000 7f74df400700 1 -- 172.21.15.151:0/1419489728 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d72ac35620 con 0x55d72ac43140 2024-05-21T22:05:56.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.116+0000 7f74df400700 1 --2- 172.21.15.151:0/1419489728 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d72ac43140 0x55d72ac49ae0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f74d000c6f0 tx=0x7f74d00e2890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d87be80dfb93ac98 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:05:56.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.116+0000 7f74cfe00700 1 -- 172.21.15.151:0/1419489728 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d0018e50 con 0x55d72ac43140 2024-05-21T22:05:56.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.116+0000 7f74cfe00700 1 -- 172.21.15.151:0/1419489728 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74d00a9310 con 0x55d72ac43140 2024-05-21T22:05:56.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.116+0000 7f74cfe00700 1 -- 172.21.15.151:0/1419489728 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d00cc0e0 con 0x55d72ac43140 2024-05-21T22:05:56.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.117+0000 7f74f0bbab80 1 -- 172.21.15.151:0/1419489728 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d72ac43140 msgr2=0x55d72ac49ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:56.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.117+0000 7f74f0bbab80 1 --2- 172.21.15.151:0/1419489728 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d72ac43140 0x55d72ac49ae0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f74d000c6f0 tx=0x7f74d00e2890 comp rx=0 tx=0).stop 2024-05-21T22:05:56.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.117+0000 7f74f0bbab80 1 -- 172.21.15.151:0/1419489728 shutdown_connections 2024-05-21T22:05:56.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.117+0000 7f74f0bbab80 1 --2- 172.21.15.151:0/1419489728 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d72ac48f30 0x55d72ac57c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:56.124 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.117+0000 7f74f0bbab80 1 --2- 172.21.15.151:0/1419489728 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d72ac43140 0x55d72ac49ae0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:56.124 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.117+0000 7f74f0bbab80 1 --2- 172.21.15.151:0/1419489728 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d72ac48610 0x55d72ac3d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:56.124 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.117+0000 7f74f0bbab80 1 -- 172.21.15.151:0/1419489728 >> 172.21.15.151:0/1419489728 conn(0x55d72ab70be0 msgr2=0x55d72ac493d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:05:56.124 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.117+0000 7f74f0bbab80 1 -- 172.21.15.151:0/1419489728 shutdown_connections 2024-05-21T22:05:56.124 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.117+0000 7f74f0bbab80 1 -- 172.21.15.151:0/1419489728 wait complete. 2024-05-21T22:05:56.124 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.118+0000 7f74f0bbab80 1 -- start start 2024-05-21T22:05:56.124 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.118+0000 7f74f0bbab80 1 Processor -- start 2024-05-21T22:05:56.124 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.118+0000 7f74f0bbab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d72ac43140 0x55d72ac5eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:56.124 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.118+0000 7f74f0bbab80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d72ac48610 0x55d72ac5eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:56.124 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.118+0000 7f74f0bbab80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d72ac48f30 0x55d72acc73d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:56.124 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.118+0000 7f74f0bbab80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f74e01b1610 con 0x55d72ac48610 2024-05-21T22:05:56.125 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.118+0000 7f74f0bbab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f74e01b0f40 con 0x55d72ac43140 2024-05-21T22:05:56.125 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.118+0000 7f74f0bbab80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f74e01ae160 con 0x55d72ac48f30 2024-05-21T22:05:56.125 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.118+0000 7f74f0bbab80 10 client.0 _ll_register_callbacks cb 0x55d72ab3ac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:05:56.125 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.119+0000 7f74d6a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d72ac48610 0x55d72ac5eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:56.125 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.119+0000 7f74d6a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d72ac48610 0x55d72ac5eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:53318/0 (socket says 172.21.15.151:53318) 2024-05-21T22:05:56.125 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.119+0000 7f74d6a00700 1 -- 172.21.15.151:0/3649660431 learned_addr learned my addr 172.21.15.151:0/3649660431 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:05:56.125 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.119+0000 7f74df400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d72ac43140 0x55d72ac5eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:56.125 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.119+0000 7f74dfe00700 1 --2- 172.21.15.151:0/3649660431 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d72ac48f30 0x55d72acc73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:56.125 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.119+0000 7f74d6a00700 1 -- 172.21.15.151:0/3649660431 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d72ac48f30 msgr2=0x55d72acc73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:56.125 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.119+0000 7f74d6a00700 1 --2- 172.21.15.151:0/3649660431 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d72ac48f30 0x55d72acc73d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:56.125 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.119+0000 7f74d6a00700 1 -- 172.21.15.151:0/3649660431 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d72ac43140 msgr2=0x55d72ac5eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:56.126 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.119+0000 7f74d6a00700 1 --2- 172.21.15.151:0/3649660431 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d72ac43140 0x55d72ac5eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:56.126 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.119+0000 7f74d6a00700 1 -- 172.21.15.151:0/3649660431 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d72acc7a10 con 0x55d72ac48610 2024-05-21T22:05:56.126 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.119+0000 7f74d6a00700 1 --2- 172.21.15.151:0/3649660431 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d72ac48610 0x55d72ac5eff0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x55d72ac4d560 tx=0x7f74c8009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24b1899ef4becde6 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:05:56.126 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.119+0000 7f74cd600700 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-05-21T22:05:56.126 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.119+0000 7f74cd600700 1 -- 172.21.15.151:0/3649660431 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c8003680 con 0x55d72ac48610 2024-05-21T22:05:56.126 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.119+0000 7f74cd600700 1 -- 172.21.15.151:0/3649660431 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7f74c8003820 con 0x55d72ac48610 2024-05-21T22:05:56.126 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.119+0000 7f74f0bbab80 1 -- 172.21.15.151:0/3649660431 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74d00a9310 con 0x55d72ac48610 2024-05-21T22:05:56.126 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.120+0000 7f74cd600700 1 -- 172.21.15.151:0/3649660431 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f74c8003a20 con 0x55d72ac48610 2024-05-21T22:05:56.126 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.120+0000 7f74f0bbab80 10 client.17511 fetch_fsmap learned FSMap version 29 2024-05-21T22:05:56.126 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.120+0000 7f74f0bbab80 1 -- 172.21.15.151:0/3649660431 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d72accedd0 con 0x55d72ac48610 2024-05-21T22:05:56.126 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.120+0000 7f74cd600700 1 -- 172.21.15.151:0/3649660431 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74d00a9310 con 0x55d72ac48610 2024-05-21T22:05:56.126 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.120+0000 7f74cd600700 1 -- 172.21.15.151:0/3649660431 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f74c8037410 con 0x55d72ac48610 2024-05-21T22:05:56.127 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.120+0000 7f74f0bbab80 10 client.17511 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:05:56.127 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.120+0000 7f74f0bbab80 10 client.17511 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-05-21T22:05:56.127 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.120+0000 7f74f0bbab80 20 client.17511 populate_metadata read hostname 'smithi151' 2024-05-21T22:05:56.127 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.120+0000 7f74f0bbab80 1 --2- 172.21.15.151:0/3649660431 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x55d72accf510 0x55d72acd1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:56.127 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.120+0000 7f74f0bbab80 4 client.17511 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T22:05:56.127 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.120+0000 7f74f0bbab80 1 -- 172.21.15.151:0/3649660431 --> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d72ab9c570 con 0x55d72accf510 2024-05-21T22:05:56.127 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.121+0000 7f74df400700 1 --2- 172.21.15.151:0/3649660431 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x55d72accf510 0x55d72acd1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:56.127 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.121+0000 7f74df400700 1 --2- 172.21.15.151:0/3649660431 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x55d72accf510 0x55d72acd1930 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d8462f69ee9060f9 server_cookie=ac018f7704e01b06 in_seq=0 out_seq=0 2024-05-21T22:05:56.127 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.121+0000 7f74cd600700 10 client.17511 ms_handle_connect on v2:172.21.15.164:6836/1554609572 2024-05-21T22:05:56.127 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.121+0000 7f74cd600700 1 -- 172.21.15.151:0/3649660431 <== mds.2 v2:172.21.15.164:6836/1554609572 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d72ab9c570 con 0x55d72accf510 2024-05-21T22:05:56.127 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.121+0000 7f74cd600700 10 client.17511 handle_command_reply: tid=0 2024-05-21T22:05:56.128 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.122+0000 7f74f0bbab80 1 client.17511 shutdown 2024-05-21T22:05:56.128 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.122+0000 7f74f0bbab80 1 -- 172.21.15.151:0/3649660431 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d72ac48610 msgr2=0x55d72ac5eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:56.128 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.122+0000 7f74f0bbab80 1 --2- 172.21.15.151:0/3649660431 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d72ac48610 0x55d72ac5eff0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x55d72ac4d560 tx=0x7f74c8009500 comp rx=0 tx=0).stop 2024-05-21T22:05:56.128 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.122+0000 7f74f0bbab80 1 -- 172.21.15.151:0/3649660431 shutdown_connections 2024-05-21T22:05:56.128 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.122+0000 7f74f0bbab80 1 --2- 172.21.15.151:0/3649660431 >> [v2:172.21.15.164:6836/1554609572,v1:172.21.15.164:6837/1554609572] conn(0x55d72accf510 0x55d72acd1930 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:56.128 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.122+0000 7f74f0bbab80 1 --2- 172.21.15.151:0/3649660431 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55d72ac43140 0x55d72ac5eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:56.128 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.122+0000 7f74f0bbab80 1 --2- 172.21.15.151:0/3649660431 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d72ac48f30 0x55d72acc73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:56.128 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.122+0000 7f74f0bbab80 1 --2- 172.21.15.151:0/3649660431 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d72ac48610 0x55d72ac5eff0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:56.128 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.122+0000 7f74f0bbab80 1 -- 172.21.15.151:0/3649660431 >> 172.21.15.151:0/3649660431 conn(0x55d72ab70be0 msgr2=0x55d72ac470a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:05:56.128 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.122+0000 7f74cd600700 0 client.17511 ms_handle_reset on v2:172.21.15.164:6836/1554609572 2024-05-21T22:05:56.128 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.122+0000 7f74f0bbab80 1 -- 172.21.15.151:0/3649660431 shutdown_connections 2024-05-21T22:05:56.129 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.122+0000 7f74f0bbab80 1 -- 172.21.15.151:0/3649660431 wait complete. 2024-05-21T22:05:56.129 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.122+0000 7f74f0bbab80 20 client.17511 trim_cache size 0 max 16384 2024-05-21T22:05:56.129 INFO:teuthology.orchestra.run.smithi151.stdout:[] 2024-05-21T22:05:56.130 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.123+0000 7f74e6200700 1 -- 172.21.15.151:0/2555483587 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f74c40b29f0 msgr2=0x7f74c40b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:56.130 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.123+0000 7f74e6200700 1 --2- 172.21.15.151:0/2555483587 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f74c40b29f0 0x7f74c40b4eb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f74d80117e0 tx=0x7f74d80047c0 comp rx=0 tx=0).stop 2024-05-21T22:05:56.130 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.123+0000 7f74e6200700 1 -- 172.21.15.151:0/2555483587 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74e01035c0 msgr2=0x7f74e0075840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:56.130 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.123+0000 7f74e6200700 1 --2- 172.21.15.151:0/2555483587 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74e01035c0 0x7f74e0075840 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f74d000ef10 tx=0x7f74d000c3a0 comp rx=0 tx=0).stop 2024-05-21T22:05:56.130 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.123+0000 7f74e6200700 1 -- 172.21.15.151:0/2555483587 shutdown_connections 2024-05-21T22:05:56.130 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.123+0000 7f74e6200700 1 --2- 172.21.15.151:0/2555483587 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f74c40b29f0 0x7f74c40b4eb0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:56.130 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.123+0000 7f74e6200700 1 --2- 172.21.15.151:0/2555483587 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74e01095d0 0x7f74e00762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:56.131 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.123+0000 7f74e6200700 1 --2- 172.21.15.151:0/2555483587 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74e01035c0 0x7f74e0075840 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:56.131 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.123+0000 7f74e6200700 1 --2- 172.21.15.151:0/2555483587 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74e0103f80 0x7f74e0075d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:56.131 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.123+0000 7f74e6200700 1 -- 172.21.15.151:0/2555483587 >> 172.21.15.151:0/2555483587 conn(0x7f74e007a610 msgr2=0x7f74e01022a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:05:56.131 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.123+0000 7f74e6200700 1 -- 172.21.15.151:0/2555483587 shutdown_connections 2024-05-21T22:05:56.131 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:56.123+0000 7f74e6200700 1 -- 172.21.15.151:0/2555483587 wait complete. 2024-05-21T22:05:56.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:05:56 smithi151 ceph-mon[31623]: from='client.27619 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T22:05:56.485 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:05:56 smithi184 ceph-mon[36942]: from='client.27619 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T22:05:56.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:05:56 smithi164 ceph-mon[36269]: from='client.27619 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T22:05:56.885 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-21T22:05:57.334 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:05:57.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:05:57 smithi151 ceph-mon[31623]: from='client.27625 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T22:05:57.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:05:57 smithi151 ceph-mon[31623]: pgmap v319: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-05-21T22:05:57.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:05:57 smithi184 ceph-mon[36942]: from='client.27625 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T22:05:57.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:05:57 smithi184 ceph-mon[36942]: pgmap v319: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-05-21T22:05:57.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:05:57 smithi164 ceph-mon[36269]: from='client.27625 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-21T22:05:57.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:05:57 smithi164 ceph-mon[36269]: pgmap v319: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-05-21T22:05:59.630 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.628+0000 7f7c7fe00700 1 -- 172.21.15.151:0/3042577480 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7c80106030 msgr2=0x7f7c80112540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:59.630 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.628+0000 7f7c7fe00700 1 --2- 172.21.15.151:0/3042577480 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7c80106030 0x7f7c80112540 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f7c7800b0a0 tx=0x7f7c7800b3b0 comp rx=0 tx=0).stop 2024-05-21T22:05:59.630 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.629+0000 7f7c7fe00700 1 -- 172.21.15.151:0/3042577480 shutdown_connections 2024-05-21T22:05:59.630 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.629+0000 7f7c7fe00700 1 --2- 172.21.15.151:0/3042577480 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7c80106030 0x7f7c80112540 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:59.630 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.629+0000 7f7c7fe00700 1 --2- 172.21.15.151:0/3042577480 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7c8010b670 0x7f7c8010ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:59.630 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.629+0000 7f7c7fe00700 1 --2- 172.21.15.151:0/3042577480 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7c80105670 0x7f7c80105af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:59.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.629+0000 7f7c7fe00700 1 -- 172.21.15.151:0/3042577480 >> 172.21.15.151:0/3042577480 conn(0x7f7c80101160 msgr2=0x7f7c80103580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:05:59.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.629+0000 7f7c7fe00700 1 -- 172.21.15.151:0/3042577480 shutdown_connections 2024-05-21T22:05:59.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.629+0000 7f7c7fe00700 1 -- 172.21.15.151:0/3042577480 wait complete. 2024-05-21T22:05:59.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.630+0000 7f7c7fe00700 1 Processor -- start 2024-05-21T22:05:59.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.630+0000 7f7c7fe00700 1 -- start start 2024-05-21T22:05:59.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.630+0000 7f7c7fe00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7c80105670 0x7f7c801a18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:59.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.630+0000 7f7c7fe00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7c80106030 0x7f7c801a1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:59.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.630+0000 7f7c7fe00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7c8010b670 0x7f7c8019bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:59.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.630+0000 7f7c7fe00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7c801158b0 con 0x7f7c80106030 2024-05-21T22:05:59.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.630+0000 7f7c7fe00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7c80115730 con 0x7f7c80105670 2024-05-21T22:05:59.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.630+0000 7f7c7fe00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7c80115a30 con 0x7f7c8010b670 2024-05-21T22:05:59.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.631+0000 7f7c7e000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7c80106030 0x7f7c801a1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:59.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.631+0000 7f7c7e000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7c80106030 0x7f7c801a1e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:53342/0 (socket says 172.21.15.151:53342) 2024-05-21T22:05:59.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.631+0000 7f7c7e000700 1 -- 172.21.15.151:0/3804567809 learned_addr learned my addr 172.21.15.151:0/3804567809 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:05:59.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.631+0000 7f7c7ea00700 1 --2- 172.21.15.151:0/3804567809 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7c80105670 0x7f7c801a18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:59.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.631+0000 7f7c7f400700 1 --2- 172.21.15.151:0/3804567809 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7c8010b670 0x7f7c8019bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:59.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.631+0000 7f7c7ea00700 1 -- 172.21.15.151:0/3804567809 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7c8010b670 msgr2=0x7f7c8019bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:59.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.631+0000 7f7c7ea00700 1 --2- 172.21.15.151:0/3804567809 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7c8010b670 0x7f7c8019bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:59.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.631+0000 7f7c7ea00700 1 -- 172.21.15.151:0/3804567809 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7c80106030 msgr2=0x7f7c801a1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:59.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.631+0000 7f7c7ea00700 1 --2- 172.21.15.151:0/3804567809 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7c80106030 0x7f7c801a1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:59.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.631+0000 7f7c7ea00700 1 -- 172.21.15.151:0/3804567809 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c8019c210 con 0x7f7c80105670 2024-05-21T22:05:59.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.631+0000 7f7c7e000700 1 --2- 172.21.15.151:0/3804567809 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7c80106030 0x7f7c801a1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:05:59.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.632+0000 7f7c7ea00700 1 --2- 172.21.15.151:0/3804567809 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7c80105670 0x7f7c801a18f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f7c7000e9f0 tx=0x7f7c7000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:05:59.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.632+0000 7f7c77400700 1 -- 172.21.15.151:0/3804567809 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c70004220 con 0x7f7c80105670 2024-05-21T22:05:59.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.632+0000 7f7c77400700 1 -- 172.21.15.151:0/3804567809 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c700043c0 con 0x7f7c80105670 2024-05-21T22:05:59.634 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.632+0000 7f7c77400700 1 -- 172.21.15.151:0/3804567809 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c70005020 con 0x7f7c80105670 2024-05-21T22:05:59.634 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.632+0000 7f7c7fe00700 1 -- 172.21.15.151:0/3804567809 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c8019c500 con 0x7f7c80105670 2024-05-21T22:05:59.634 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.632+0000 7f7c7fe00700 1 -- 172.21.15.151:0/3804567809 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c801a87c0 con 0x7f7c80105670 2024-05-21T22:05:59.637 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.637+0000 7f7c77400700 1 -- 172.21.15.151:0/3804567809 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7f7c70004750 con 0x7f7c80105670 2024-05-21T22:05:59.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.638+0000 7f7c8fe98b80 1 Processor -- start 2024-05-21T22:05:59.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.638+0000 7f7c8fe98b80 1 -- start start 2024-05-21T22:05:59.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.638+0000 7f7c8fe98b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5559356587c0 0x555935658ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:59.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.638+0000 7f7c8fe98b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5559356590e0 0x555935652d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:59.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.638+0000 7f7c8fe98b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555935653390 0x5559356537f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:59.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.638+0000 7f7c8fe98b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5559355b0e40 con 0x555935653390 2024-05-21T22:05:59.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.638+0000 7f7c8fe98b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5559355a09e0 con 0x5559356590e0 2024-05-21T22:05:59.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.638+0000 7f7c8fe98b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5559355a4a20 con 0x5559356587c0 2024-05-21T22:05:59.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.638+0000 7f7c77400700 1 --2- 172.21.15.151:0/3804567809 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f7c640b2a70 0x7f7c640b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:59.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.639+0000 7f7c77400700 1 -- 172.21.15.151:0/3804567809 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7f7c700e2520 con 0x7f7c80105670 2024-05-21T22:05:59.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.639+0000 7f7c7f400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5559356587c0 0x555935658ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:59.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.639+0000 7f7c7ea00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5559356590e0 0x555935652d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:59.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.639+0000 7f7c7f400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5559356587c0 0x555935658ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:52970/0 (socket says 172.21.15.151:52970) 2024-05-21T22:05:59.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.639+0000 7f7c7f400700 1 -- 172.21.15.151:0/3618192539 learned_addr learned my addr 172.21.15.151:0/3618192539 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:05:59.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.639+0000 7f7c7e000700 1 --2- 172.21.15.151:0/3618192539 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555935653390 0x5559356537f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:59.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.639+0000 7f7c7f400700 1 --2- 172.21.15.151:0/3804567809 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f7c640b2a70 0x7f7c640b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:59.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.639+0000 7f7c7f400700 1 -- 172.21.15.151:0/3618192539 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5559356590e0 msgr2=0x555935652d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:59.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.639+0000 7f7c7f400700 1 --2- 172.21.15.151:0/3618192539 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5559356590e0 0x555935652d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:59.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.639+0000 7f7c7f400700 1 -- 172.21.15.151:0/3618192539 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555935653390 msgr2=0x5559356537f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:59.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.639+0000 7f7c7f400700 1 --2- 172.21.15.151:0/3618192539 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555935653390 0x5559356537f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:59.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.639+0000 7f7c7f400700 1 -- 172.21.15.151:0/3618192539 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555935569420 con 0x5559356587c0 2024-05-21T22:05:59.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.639+0000 7f7c7ea00700 1 --2- 172.21.15.151:0/3618192539 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5559356590e0 0x555935652d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:05:59.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.640+0000 7f7c7f400700 1 --2- 172.21.15.151:0/3618192539 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5559356587c0 0x555935658ba0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f7c78015000 tx=0x7f7c780094e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b956728ccfd6fe server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:05:59.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.640+0000 7f7c6be00700 1 -- 172.21.15.151:0/3618192539 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c7802e070 con 0x5559356587c0 2024-05-21T22:05:59.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.640+0000 7f7c6be00700 1 -- 172.21.15.151:0/3618192539 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c78023070 con 0x5559356587c0 2024-05-21T22:05:59.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.640+0000 7f7c6be00700 1 -- 172.21.15.151:0/3618192539 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c7801f680 con 0x5559356587c0 2024-05-21T22:05:59.643 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.641+0000 7f7c8fe98b80 1 -- 172.21.15.151:0/3618192539 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5559356587c0 msgr2=0x555935658ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:59.643 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.641+0000 7f7c8fe98b80 1 --2- 172.21.15.151:0/3618192539 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5559356587c0 0x555935658ba0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f7c78015000 tx=0x7f7c780094e0 comp rx=0 tx=0).stop 2024-05-21T22:05:59.643 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.641+0000 7f7c7f400700 1 --2- 172.21.15.151:0/3804567809 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f7c640b2a70 0x7f7c640b4f30 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7c7802b000 tx=0x7f7c78024000 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:05:59.643 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.641+0000 7f7c8fe98b80 1 -- 172.21.15.151:0/3618192539 shutdown_connections 2024-05-21T22:05:59.643 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.641+0000 7f7c8fe98b80 1 --2- 172.21.15.151:0/3618192539 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555935653390 0x5559356537f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:59.643 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.641+0000 7f7c8fe98b80 1 --2- 172.21.15.151:0/3618192539 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5559356587c0 0x555935658ba0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:59.643 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.641+0000 7f7c8fe98b80 1 --2- 172.21.15.151:0/3618192539 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5559356590e0 0x555935652d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:59.643 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.641+0000 7f7c8fe98b80 1 -- 172.21.15.151:0/3618192539 >> 172.21.15.151:0/3618192539 conn(0x555935580be0 msgr2=0x55593564d810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:05:59.643 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.641+0000 7f7c8fe98b80 1 -- 172.21.15.151:0/3618192539 shutdown_connections 2024-05-21T22:05:59.644 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.641+0000 7f7c8fe98b80 1 -- 172.21.15.151:0/3618192539 wait complete. 2024-05-21T22:05:59.644 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.641+0000 7f7c8fe98b80 1 -- start start 2024-05-21T22:05:59.644 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.642+0000 7f7c8fe98b80 1 Processor -- start 2024-05-21T22:05:59.644 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.642+0000 7f7c8fe98b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555935653390 0x55593565ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:59.644 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.642+0000 7f7c8fe98b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5559356587c0 0x55593565c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:59.644 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.642+0000 7f7c8fe98b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5559356590e0 0x55593565c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:59.644 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.642+0000 7f7c8fe98b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5559354a6fa0 con 0x5559356590e0 2024-05-21T22:05:59.644 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.642+0000 7f7c8fe98b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x555935541760 con 0x555935653390 2024-05-21T22:05:59.644 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.642+0000 7f7c8fe98b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x55593565fc00 con 0x5559356587c0 2024-05-21T22:05:59.644 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.642+0000 7f7c8fe98b80 10 client.0 _ll_register_callbacks cb 0x55593554ac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:05:59.644 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.643+0000 7f7c7ea00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555935653390 0x55593565ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:59.644 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.643+0000 7f7c7f400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5559356590e0 0x55593565c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:59.644 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.643+0000 7f7c7ea00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555935653390 0x55593565ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:34924/0 (socket says 172.21.15.151:34924) 2024-05-21T22:05:59.644 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.643+0000 7f7c7e000700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5559356587c0 0x55593565c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:59.645 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.643+0000 7f7c7f400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5559356590e0 0x55593565c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:53372/0 (socket says 172.21.15.151:53372) 2024-05-21T22:05:59.645 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.643+0000 7f7c7ea00700 1 -- 172.21.15.151:0/1467911777 learned_addr learned my addr 172.21.15.151:0/1467911777 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:05:59.646 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.643+0000 7f7c7e000700 1 -- 172.21.15.151:0/1467911777 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555935653390 msgr2=0x55593565ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:59.646 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.643+0000 7f7c7e000700 1 --2- 172.21.15.151:0/1467911777 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555935653390 0x55593565ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:59.646 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.643+0000 7f7c7e000700 1 -- 172.21.15.151:0/1467911777 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5559356590e0 msgr2=0x55593565c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:59.646 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.643+0000 7f7c7e000700 1 --2- 172.21.15.151:0/1467911777 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5559356590e0 0x55593565c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:59.646 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.643+0000 7f7c7e000700 1 -- 172.21.15.151:0/1467911777 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559354f08d0 con 0x5559356587c0 2024-05-21T22:05:59.646 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.643+0000 7f7c7ea00700 1 --2- 172.21.15.151:0/1467911777 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555935653390 0x55593565ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:05:59.647 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.643+0000 7f7c7e000700 1 --2- 172.21.15.151:0/1467911777 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5559356587c0 0x55593565c1d0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c012cf0 tx=0x7f7c6c009e10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9019d642e01e97b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:05:59.647 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.643+0000 7f7c68c00700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-21T22:05:59.647 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.644+0000 7f7c68c00700 1 -- 172.21.15.151:0/1467911777 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c6c004050 con 0x5559356587c0 2024-05-21T22:05:59.647 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.644+0000 7f7c8fe98b80 1 -- 172.21.15.151:0/1467911777 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c78023070 con 0x5559356587c0 2024-05-21T22:05:59.647 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.644+0000 7f7c68c00700 1 -- 172.21.15.151:0/1467911777 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7f7c6c0041f0 con 0x5559356587c0 2024-05-21T22:05:59.647 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.644+0000 7f7c68c00700 1 -- 172.21.15.151:0/1467911777 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7c6c0043f0 con 0x5559356587c0 2024-05-21T22:05:59.647 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.644+0000 7f7c8fe98b80 10 client.27655 fetch_fsmap learned FSMap version 29 2024-05-21T22:05:59.647 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.644+0000 7f7c8fe98b80 1 -- 172.21.15.151:0/1467911777 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555935571060 con 0x5559356587c0 2024-05-21T22:05:59.647 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.645+0000 7f7c68c00700 1 -- 172.21.15.151:0/1467911777 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c78023070 con 0x5559356587c0 2024-05-21T22:05:59.647 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.645+0000 7f7c68c00700 1 -- 172.21.15.151:0/1467911777 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f7c6c003a70 con 0x5559356587c0 2024-05-21T22:05:59.647 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.645+0000 7f7c8fe98b80 10 client.27655 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:05:59.647 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.645+0000 7f7c8fe98b80 10 client.27655 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-21T22:05:59.647 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.645+0000 7f7c8fe98b80 20 client.27655 populate_metadata read hostname 'smithi151' 2024-05-21T22:05:59.647 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.645+0000 7f7c8fe98b80 1 --2- 172.21.15.151:0/1467911777 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x55593566c7a0 0x55593566eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:59.648 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.645+0000 7f7c8fe98b80 4 client.27655 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T22:05:59.648 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.645+0000 7f7c8fe98b80 1 -- 172.21.15.151:0/1467911777 --> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5559355b2d90 con 0x55593566c7a0 2024-05-21T22:05:59.648 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.645+0000 7f7c7ea00700 1 --2- 172.21.15.151:0/1467911777 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x55593566c7a0 0x55593566eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:59.648 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.646+0000 7f7c7ea00700 1 --2- 172.21.15.151:0/1467911777 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x55593566c7a0 0x55593566eb90 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=484ead5ba9753fb7 server_cookie=edd8a6621dc8046f in_seq=0 out_seq=0 2024-05-21T22:05:59.648 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.646+0000 7f7c68c00700 10 client.27655 ms_handle_connect on v2:172.21.15.184:6838/3059935143 2024-05-21T22:05:59.649 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.646+0000 7f7c68c00700 1 -- 172.21.15.151:0/1467911777 <== mds.1 v2:172.21.15.184:6838/3059935143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x7f7c70083e50 con 0x55593566c7a0 2024-05-21T22:05:59.649 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.646+0000 7f7c68c00700 10 client.27655 handle_command_reply: tid=0 2024-05-21T22:05:59.649 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.647+0000 7f7c8fe98b80 1 client.27655 shutdown 2024-05-21T22:05:59.649 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.647+0000 7f7c8fe98b80 1 -- 172.21.15.151:0/1467911777 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5559356587c0 msgr2=0x55593565c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:59.649 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.647+0000 7f7c8fe98b80 1 --2- 172.21.15.151:0/1467911777 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5559356587c0 0x55593565c1d0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c012cf0 tx=0x7f7c6c009e10 comp rx=0 tx=0).stop 2024-05-21T22:05:59.649 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.647+0000 7f7c8fe98b80 1 -- 172.21.15.151:0/1467911777 shutdown_connections 2024-05-21T22:05:59.649 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.647+0000 7f7c8fe98b80 1 --2- 172.21.15.151:0/1467911777 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x55593566c7a0 0x55593566eb90 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:59.649 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.647+0000 7f7c8fe98b80 1 --2- 172.21.15.151:0/1467911777 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x555935653390 0x55593565ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:59.649 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.647+0000 7f7c8fe98b80 1 --2- 172.21.15.151:0/1467911777 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5559356590e0 0x55593565c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:59.649 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.647+0000 7f7c8fe98b80 1 --2- 172.21.15.151:0/1467911777 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5559356587c0 0x55593565c1d0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:59.649 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.647+0000 7f7c68c00700 0 client.27655 ms_handle_reset on v2:172.21.15.184:6838/3059935143 2024-05-21T22:05:59.650 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.647+0000 7f7c8fe98b80 1 -- 172.21.15.151:0/1467911777 >> 172.21.15.151:0/1467911777 conn(0x555935580be0 msgr2=0x55593564cfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:05:59.650 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.647+0000 7f7c8fe98b80 1 -- 172.21.15.151:0/1467911777 shutdown_connections 2024-05-21T22:05:59.650 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.647+0000 7f7c8fe98b80 1 -- 172.21.15.151:0/1467911777 wait complete. 2024-05-21T22:05:59.650 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.647+0000 7f7c8fe98b80 20 client.27655 trim_cache size 0 max 16384 2024-05-21T22:05:59.664 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.664+0000 7f7c8fe98b80 1 Processor -- start 2024-05-21T22:05:59.664 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.664+0000 7f7c8fe98b80 1 -- start start 2024-05-21T22:05:59.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.664+0000 7f7c8fe98b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555935653390 0x555935652d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:59.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.664+0000 7f7c8fe98b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5559356587c0 0x55593565c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:59.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.664+0000 7f7c8fe98b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5559356590e0 0x555935653770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:59.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.664+0000 7f7c8fe98b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7c801afe90 con 0x5559356587c0 2024-05-21T22:05:59.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.664+0000 7f7c8fe98b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7c801b19d0 con 0x5559356590e0 2024-05-21T22:05:59.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.664+0000 7f7c8fe98b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7c801b0560 con 0x555935653390 2024-05-21T22:05:59.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.664+0000 7f7c7e000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5559356587c0 0x55593565c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:59.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.664+0000 7f7c7e000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5559356587c0 0x55593565c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:53374/0 (socket says 172.21.15.151:53374) 2024-05-21T22:05:59.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.664+0000 7f7c7e000700 1 -- 172.21.15.151:0/4037030751 learned_addr learned my addr 172.21.15.151:0/4037030751 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:05:59.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.664+0000 7f7c7f400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5559356590e0 0x555935653770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:59.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.665+0000 7f7c7ea00700 1 --2- 172.21.15.151:0/4037030751 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555935653390 0x555935652d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:59.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.665+0000 7f7c7e000700 1 -- 172.21.15.151:0/4037030751 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555935653390 msgr2=0x555935652d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:59.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.665+0000 7f7c7e000700 1 --2- 172.21.15.151:0/4037030751 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555935653390 0x555935652d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:59.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.665+0000 7f7c7e000700 1 -- 172.21.15.151:0/4037030751 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5559356590e0 msgr2=0x555935653770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:59.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.665+0000 7f7c7e000700 1 --2- 172.21.15.151:0/4037030751 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5559356590e0 0x555935653770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:59.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.665+0000 7f7c7e000700 1 -- 172.21.15.151:0/4037030751 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55593566ef00 con 0x5559356587c0 2024-05-21T22:05:59.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.665+0000 7f7c7f400700 1 --2- 172.21.15.151:0/4037030751 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5559356590e0 0x555935653770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:05:59.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.665+0000 7f7c7e000700 1 --2- 172.21.15.151:0/4037030751 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5559356587c0 0x55593565c6c0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c010480 tx=0x7f7c6c004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efdbce6f503082e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:05:59.668 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.665+0000 7f7c6b400700 1 -- 172.21.15.151:0/4037030751 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c6c029040 con 0x5559356587c0 2024-05-21T22:05:59.668 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.666+0000 7f7c6b400700 1 -- 172.21.15.151:0/4037030751 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c6c003b30 con 0x5559356587c0 2024-05-21T22:05:59.668 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.666+0000 7f7c6b400700 1 -- 172.21.15.151:0/4037030751 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c6c00fa70 con 0x5559356587c0 2024-05-21T22:05:59.668 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.666+0000 7f7c8fe98b80 1 -- 172.21.15.151:0/4037030751 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5559356587c0 msgr2=0x55593565c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:59.668 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.666+0000 7f7c8fe98b80 1 --2- 172.21.15.151:0/4037030751 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5559356587c0 0x55593565c6c0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c010480 tx=0x7f7c6c004610 comp rx=0 tx=0).stop 2024-05-21T22:05:59.668 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.666+0000 7f7c8fe98b80 1 -- 172.21.15.151:0/4037030751 shutdown_connections 2024-05-21T22:05:59.668 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.666+0000 7f7c8fe98b80 1 --2- 172.21.15.151:0/4037030751 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5559356590e0 0x555935653770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:59.668 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.666+0000 7f7c8fe98b80 1 --2- 172.21.15.151:0/4037030751 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555935653390 0x555935652d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:59.668 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.666+0000 7f7c8fe98b80 1 --2- 172.21.15.151:0/4037030751 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5559356587c0 0x55593565c6c0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:59.668 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.666+0000 7f7c8fe98b80 1 -- 172.21.15.151:0/4037030751 >> 172.21.15.151:0/4037030751 conn(0x555935580be0 msgr2=0x55593565bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:05:59.669 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.666+0000 7f7c8fe98b80 1 -- 172.21.15.151:0/4037030751 shutdown_connections 2024-05-21T22:05:59.669 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.667+0000 7f7c8fe98b80 1 -- 172.21.15.151:0/4037030751 wait complete. 2024-05-21T22:05:59.669 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.667+0000 7f7c8fe98b80 1 -- start start 2024-05-21T22:05:59.669 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.667+0000 7f7c8fe98b80 1 Processor -- start 2024-05-21T22:05:59.669 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.668+0000 7f7c8fe98b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555935653390 0x555935665190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:59.669 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.668+0000 7f7c8fe98b80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5559356587c0 0x5559356656d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:59.669 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.668+0000 7f7c8fe98b80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5559356590e0 0x555935673c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:59.669 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.668+0000 7f7c8fe98b80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7c801b1300 con 0x5559356590e0 2024-05-21T22:05:59.669 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.668+0000 7f7c8fe98b80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7c801b0c30 con 0x5559356587c0 2024-05-21T22:05:59.669 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.668+0000 7f7c8fe98b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7c801ade50 con 0x555935653390 2024-05-21T22:05:59.669 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.668+0000 7f7c8fe98b80 10 client.0 _ll_register_callbacks cb 0x55593554ac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:05:59.669 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.668+0000 7f7c7e000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5559356587c0 0x5559356656d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:59.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.668+0000 7f7c7ea00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555935653390 0x555935665190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:59.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.668+0000 7f7c7f400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5559356590e0 0x555935673c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:59.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.668+0000 7f7c7e000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5559356587c0 0x5559356656d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:34932/0 (socket says 172.21.15.151:34932) 2024-05-21T22:05:59.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.668+0000 7f7c7e000700 1 -- 172.21.15.151:0/3646008895 learned_addr learned my addr 172.21.15.151:0/3646008895 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:05:59.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.668+0000 7f7c7ea00700 1 -- 172.21.15.151:0/3646008895 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5559356587c0 msgr2=0x5559356656d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:59.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.668+0000 7f7c7ea00700 1 --2- 172.21.15.151:0/3646008895 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5559356587c0 0x5559356656d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:59.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.668+0000 7f7c7ea00700 1 -- 172.21.15.151:0/3646008895 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5559356590e0 msgr2=0x555935673c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:05:59.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.668+0000 7f7c7ea00700 1 --2- 172.21.15.151:0/3646008895 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5559356590e0 0x555935673c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:59.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.668+0000 7f7c7ea00700 1 -- 172.21.15.151:0/3646008895 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555935674400 con 0x555935653390 2024-05-21T22:05:59.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.669+0000 7f7c7ea00700 1 --2- 172.21.15.151:0/3646008895 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555935653390 0x555935665190 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f7c7000c5b0 tx=0x7f7c7001dcf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c84a467ce78db512 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:05:59.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.669+0000 7f7c68c00700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2024-05-21T22:05:59.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.669+0000 7f7c68c00700 1 -- 172.21.15.151:0/3646008895 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c700c2070 con 0x555935653390 2024-05-21T22:05:59.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.669+0000 7f7c68c00700 1 -- 172.21.15.151:0/3646008895 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7f7c700b9f60 con 0x555935653390 2024-05-21T22:05:59.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.669+0000 7f7c8fe98b80 1 -- 172.21.15.151:0/3646008895 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c6c003b30 con 0x555935653390 2024-05-21T22:05:59.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.669+0000 7f7c68c00700 1 -- 172.21.15.151:0/3646008895 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7c700c7050 con 0x555935653390 2024-05-21T22:05:59.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.669+0000 7f7c8fe98b80 10 client.27661 fetch_fsmap learned FSMap version 29 2024-05-21T22:05:59.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.669+0000 7f7c8fe98b80 1 -- 172.21.15.151:0/3646008895 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5559356df780 con 0x555935653390 2024-05-21T22:05:59.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.669+0000 7f7c68c00700 1 -- 172.21.15.151:0/3646008895 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c6c003b30 con 0x555935653390 2024-05-21T22:05:59.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.670+0000 7f7c68c00700 1 -- 172.21.15.151:0/3646008895 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f7c700bebe0 con 0x555935653390 2024-05-21T22:05:59.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.670+0000 7f7c8fe98b80 10 client.27661 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:05:59.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.670+0000 7f7c8fe98b80 10 client.27661 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-21T22:05:59.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.670+0000 7f7c8fe98b80 20 client.27661 populate_metadata read hostname 'smithi151' 2024-05-21T22:05:59.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.670+0000 7f7c8fe98b80 1 --2- 172.21.15.151:0/3646008895 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x5559356dfe30 0x5559356e2250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:05:59.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.670+0000 7f7c8fe98b80 4 client.27661 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T22:05:59.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.670+0000 7f7c8fe98b80 1 -- 172.21.15.151:0/3646008895 --> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5559355ac570 con 0x5559356dfe30 2024-05-21T22:05:59.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.670+0000 7f7c7e000700 1 --2- 172.21.15.151:0/3646008895 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x5559356dfe30 0x5559356e2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:05:59.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.671+0000 7f7c7e000700 1 --2- 172.21.15.151:0/3646008895 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x5559356dfe30 0x5559356e2250 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=28cbdad79bea35f0 server_cookie=e426e16fd89cf4e4 in_seq=0 out_seq=0 2024-05-21T22:05:59.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.671+0000 7f7c68c00700 10 client.27661 ms_handle_connect on v2:172.21.15.184:6838/3059935143 2024-05-21T22:05:59.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.671+0000 7f7c68c00700 1 -- 172.21.15.151:0/3646008895 <== mds.1 v2:172.21.15.184:6838/3059935143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5559355ac570 con 0x5559356dfe30 2024-05-21T22:05:59.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.671+0000 7f7c68c00700 10 client.27661 handle_command_reply: tid=0 2024-05-21T22:05:59.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.671+0000 7f7c8fe98b80 1 client.27661 shutdown 2024-05-21T22:05:59.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.671+0000 7f7c8fe98b80 1 -- 172.21.15.151:0/3646008895 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555935653390 msgr2=0x555935665190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:59.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.671+0000 7f7c8fe98b80 1 --2- 172.21.15.151:0/3646008895 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555935653390 0x555935665190 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f7c7000c5b0 tx=0x7f7c7001dcf0 comp rx=0 tx=0).stop 2024-05-21T22:05:59.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.671+0000 7f7c8fe98b80 1 -- 172.21.15.151:0/3646008895 shutdown_connections 2024-05-21T22:05:59.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.671+0000 7f7c8fe98b80 1 --2- 172.21.15.151:0/3646008895 >> [v2:172.21.15.184:6838/3059935143,v1:172.21.15.184:6839/3059935143] conn(0x5559356dfe30 0x5559356e2250 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:59.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.671+0000 7f7c8fe98b80 1 --2- 172.21.15.151:0/3646008895 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5559356590e0 0x555935673c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:59.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.671+0000 7f7c8fe98b80 1 --2- 172.21.15.151:0/3646008895 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555935653390 0x555935665190 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:59.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.671+0000 7f7c8fe98b80 1 --2- 172.21.15.151:0/3646008895 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5559356587c0 0x5559356656d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:59.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.671+0000 7f7c68c00700 0 client.27661 ms_handle_reset on v2:172.21.15.184:6838/3059935143 2024-05-21T22:05:59.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.671+0000 7f7c8fe98b80 1 -- 172.21.15.151:0/3646008895 >> 172.21.15.151:0/3646008895 conn(0x555935580be0 msgr2=0x5559356594c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:05:59.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.671+0000 7f7c8fe98b80 1 -- 172.21.15.151:0/3646008895 shutdown_connections 2024-05-21T22:05:59.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.671+0000 7f7c8fe98b80 1 -- 172.21.15.151:0/3646008895 wait complete. 2024-05-21T22:05:59.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.671+0000 7f7c8fe98b80 20 client.27661 trim_cache size 0 max 16384 2024-05-21T22:05:59.675 INFO:teuthology.orchestra.run.smithi151.stdout:[] 2024-05-21T22:05:59.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.672+0000 7f7c7fe00700 1 -- 172.21.15.151:0/3804567809 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f7c640b2a70 msgr2=0x7f7c640b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:59.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.672+0000 7f7c7fe00700 1 --2- 172.21.15.151:0/3804567809 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f7c640b2a70 0x7f7c640b4f30 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7c7802b000 tx=0x7f7c78024000 comp rx=0 tx=0).stop 2024-05-21T22:05:59.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.672+0000 7f7c7fe00700 1 -- 172.21.15.151:0/3804567809 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7c80105670 msgr2=0x7f7c801a18f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:05:59.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.672+0000 7f7c7fe00700 1 --2- 172.21.15.151:0/3804567809 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7c80105670 0x7f7c801a18f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f7c7000e9f0 tx=0x7f7c7000ed00 comp rx=0 tx=0).stop 2024-05-21T22:05:59.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.672+0000 7f7c7fe00700 1 -- 172.21.15.151:0/3804567809 shutdown_connections 2024-05-21T22:05:59.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.672+0000 7f7c7fe00700 1 --2- 172.21.15.151:0/3804567809 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f7c640b2a70 0x7f7c640b4f30 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:59.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.672+0000 7f7c7fe00700 1 --2- 172.21.15.151:0/3804567809 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7c80105670 0x7f7c801a18f0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:59.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.672+0000 7f7c7fe00700 1 --2- 172.21.15.151:0/3804567809 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7c8010b670 0x7f7c8019bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:59.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.672+0000 7f7c7fe00700 1 --2- 172.21.15.151:0/3804567809 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7c80106030 0x7f7c801a1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:05:59.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.672+0000 7f7c7fe00700 1 -- 172.21.15.151:0/3804567809 >> 172.21.15.151:0/3804567809 conn(0x7f7c80101160 msgr2=0x7f7c801016e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:05:59.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.673+0000 7f7c7fe00700 1 -- 172.21.15.151:0/3804567809 shutdown_connections 2024-05-21T22:05:59.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:05:59.673+0000 7f7c7fe00700 1 -- 172.21.15.151:0/3804567809 wait complete. 2024-05-21T22:05:59.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:05:59 smithi164 ceph-mon[36269]: pgmap v320: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 0 B/s wr, 14 op/s 2024-05-21T22:05:59.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:05:59 smithi151 ceph-mon[31623]: pgmap v320: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 0 B/s wr, 14 op/s 2024-05-21T22:05:59.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:05:59 smithi184 ceph-mon[36942]: pgmap v320: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 0 B/s wr, 14 op/s 2024-05-21T22:06:00.500 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-21T22:06:00.964 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:06:01.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:06:01 smithi151 ceph-mon[31623]: pgmap v321: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 0 B/s wr, 10 op/s 2024-05-21T22:06:01.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:06:01 smithi164 ceph-mon[36269]: pgmap v321: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 0 B/s wr, 10 op/s 2024-05-21T22:06:01.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:06:01 smithi184 ceph-mon[36942]: pgmap v321: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 0 B/s wr, 10 op/s 2024-05-21T22:06:03.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.099+0000 7fac74a00700 1 -- 172.21.15.151:0/1767124023 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac70103450 msgr2=0x7fac701038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:06:03.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.099+0000 7fac74a00700 1 --2- 172.21.15.151:0/1767124023 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac70103450 0x7fac701038d0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fac58009f90 tx=0x7fac5800b330 comp rx=0 tx=0).stop 2024-05-21T22:06:03.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.099+0000 7fac74a00700 1 -- 172.21.15.151:0/1767124023 shutdown_connections 2024-05-21T22:06:03.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.099+0000 7fac74a00700 1 --2- 172.21.15.151:0/1767124023 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fac701094e0 0x7fac701098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:03.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.099+0000 7fac74a00700 1 --2- 172.21.15.151:0/1767124023 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac70103e10 0x7fac70110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:03.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.099+0000 7fac74a00700 1 --2- 172.21.15.151:0/1767124023 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac70103450 0x7fac701038d0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:03.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.099+0000 7fac74a00700 1 -- 172.21.15.151:0/1767124023 >> 172.21.15.151:0/1767124023 conn(0x7fac700fef60 msgr2=0x7fac70101380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:06:03.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.099+0000 7fac74a00700 1 -- 172.21.15.151:0/1767124023 shutdown_connections 2024-05-21T22:06:03.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.099+0000 7fac74a00700 1 -- 172.21.15.151:0/1767124023 wait complete. 2024-05-21T22:06:03.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.100+0000 7fac74a00700 1 Processor -- start 2024-05-21T22:06:03.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.100+0000 7fac74a00700 1 -- start start 2024-05-21T22:06:03.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.101+0000 7fac74a00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac70103450 0x7fac7019f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:06:03.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.101+0000 7fac74a00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac70103e10 0x7fac7019fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:06:03.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.101+0000 7fac74a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fac701094e0 0x7fac70199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:06:03.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.101+0000 7fac74a00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fac70113640 con 0x7fac70103450 2024-05-21T22:06:03.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.101+0000 7fac74a00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fac701134c0 con 0x7fac701094e0 2024-05-21T22:06:03.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.101+0000 7fac74a00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fac701137c0 con 0x7fac70103e10 2024-05-21T22:06:03.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.101+0000 7fac6f400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac70103450 0x7fac7019f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:06:03.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.101+0000 7fac6f400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac70103450 0x7fac7019f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:53392/0 (socket says 172.21.15.151:53392) 2024-05-21T22:06:03.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.101+0000 7fac6f400700 1 -- 172.21.15.151:0/4235727879 learned_addr learned my addr 172.21.15.151:0/4235727879 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:06:03.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.101+0000 7fac66a00700 1 --2- 172.21.15.151:0/4235727879 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac70103e10 0x7fac7019fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:06:03.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.101+0000 7fac6fe00700 1 --2- 172.21.15.151:0/4235727879 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fac701094e0 0x7fac70199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:06:03.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.102+0000 7fac66a00700 1 -- 172.21.15.151:0/4235727879 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fac701094e0 msgr2=0x7fac70199a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:06:03.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.102+0000 7fac66a00700 1 --2- 172.21.15.151:0/4235727879 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fac701094e0 0x7fac70199a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:03.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.102+0000 7fac66a00700 1 -- 172.21.15.151:0/4235727879 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac70103450 msgr2=0x7fac7019f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:06:03.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.102+0000 7fac66a00700 1 --2- 172.21.15.151:0/4235727879 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac70103450 0x7fac7019f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:03.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.102+0000 7fac66a00700 1 -- 172.21.15.151:0/4235727879 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac7019a180 con 0x7fac70103e10 2024-05-21T22:06:03.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.102+0000 7fac66a00700 1 --2- 172.21.15.151:0/4235727879 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac70103e10 0x7fac7019fd50 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fac58000f80 tx=0x7fac5800ccb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:06:03.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.102+0000 7fac6cc00700 1 -- 172.21.15.151:0/4235727879 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac58021070 con 0x7fac70103e10 2024-05-21T22:06:03.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.102+0000 7fac6cc00700 1 -- 172.21.15.151:0/4235727879 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac58007ac0 con 0x7fac70103e10 2024-05-21T22:06:03.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.102+0000 7fac74a00700 1 -- 172.21.15.151:0/4235727879 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac7019a410 con 0x7fac70103e10 2024-05-21T22:06:03.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.102+0000 7fac6cc00700 1 -- 172.21.15.151:0/4235727879 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac58005040 con 0x7fac70103e10 2024-05-21T22:06:03.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.102+0000 7fac74a00700 1 -- 172.21.15.151:0/4235727879 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac701a66e0 con 0x7fac70103e10 2024-05-21T22:06:03.106 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.106+0000 7fac6cc00700 1 -- 172.21.15.151:0/4235727879 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7fac58024070 con 0x7fac70103e10 2024-05-21T22:06:03.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.107+0000 7fac8098ab80 1 Processor -- start 2024-05-21T22:06:03.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.107+0000 7fac8098ab80 1 -- start start 2024-05-21T22:06:03.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.108+0000 7fac8098ab80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5560f6a5b360 0x5560f6a5b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:06:03.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.108+0000 7fac8098ab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5560f6a5bc80 0x5560f6a5f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:06:03.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.108+0000 7fac8098ab80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5560f6a5fd90 0x5560f6a601f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:06:03.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.108+0000 7fac8098ab80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5560f69b4e40 con 0x5560f6a5fd90 2024-05-21T22:06:03.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.108+0000 7fac6cc00700 1 --2- 172.21.15.151:0/4235727879 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fac540b2a70 0x7fac540b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:06:03.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.108+0000 7fac8098ab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5560f69a49e0 con 0x5560f6a5bc80 2024-05-21T22:06:03.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.108+0000 7fac8098ab80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5560f69a8a20 con 0x5560f6a5b360 2024-05-21T22:06:03.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.108+0000 7fac6cc00700 1 -- 172.21.15.151:0/4235727879 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7fac5800f020 con 0x7fac70103e10 2024-05-21T22:06:03.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.108+0000 7fac6fe00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5560f6a5b360 0x5560f6a5b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:06:03.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.108+0000 7fac6fe00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5560f6a5b360 0x5560f6a5b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:53034/0 (socket says 172.21.15.151:53034) 2024-05-21T22:06:03.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.108+0000 7fac6f400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5560f6a5bc80 0x5560f6a5f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:06:03.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.108+0000 7fac6fe00700 1 -- 172.21.15.151:0/817583455 learned_addr learned my addr 172.21.15.151:0/817583455 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:06:03.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.108+0000 7fac6fe00700 1 --2- 172.21.15.151:0/817583455 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5560f6a5fd90 0x5560f6a601f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:06:03.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.108+0000 7fac66a00700 1 --2- 172.21.15.151:0/4235727879 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fac540b2a70 0x7fac540b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:06:03.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.109+0000 7fac6fe00700 1 -- 172.21.15.151:0/817583455 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5560f6a5bc80 msgr2=0x5560f6a5f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:06:03.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.109+0000 7fac6fe00700 1 --2- 172.21.15.151:0/817583455 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5560f6a5bc80 0x5560f6a5f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:03.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.109+0000 7fac6fe00700 1 -- 172.21.15.151:0/817583455 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5560f6a5fd90 msgr2=0x5560f6a601f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:06:03.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.109+0000 7fac6fe00700 1 --2- 172.21.15.151:0/817583455 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5560f6a5fd90 0x5560f6a601f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:03.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.109+0000 7fac6fe00700 1 -- 172.21.15.151:0/817583455 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560f696d420 con 0x5560f6a5b360 2024-05-21T22:06:03.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.109+0000 7fac6fe00700 1 --2- 172.21.15.151:0/817583455 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5560f6a5b360 0x5560f6a5b740 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fac68011840 tx=0x7fac680098a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94db6db2d092b2fa server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:06:03.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.109+0000 7fac66a00700 1 --2- 172.21.15.151:0/4235727879 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fac540b2a70 0x7fac540b4f30 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fac5801dfd0 tx=0x7fac5800cf50 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:06:03.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.109+0000 7fac64c00700 1 -- 172.21.15.151:0/817583455 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac68004030 con 0x5560f6a5b360 2024-05-21T22:06:03.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.109+0000 7fac64c00700 1 -- 172.21.15.151:0/817583455 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac68003680 con 0x5560f6a5b360 2024-05-21T22:06:03.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.109+0000 7fac64c00700 1 -- 172.21.15.151:0/817583455 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac6800e640 con 0x5560f6a5b360 2024-05-21T22:06:03.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.110+0000 7fac8098ab80 1 -- 172.21.15.151:0/817583455 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5560f6a5b360 msgr2=0x5560f6a5b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:06:03.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.110+0000 7fac8098ab80 1 --2- 172.21.15.151:0/817583455 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5560f6a5b360 0x5560f6a5b740 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fac68011840 tx=0x7fac680098a0 comp rx=0 tx=0).stop 2024-05-21T22:06:03.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.110+0000 7fac8098ab80 1 -- 172.21.15.151:0/817583455 shutdown_connections 2024-05-21T22:06:03.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.110+0000 7fac8098ab80 1 --2- 172.21.15.151:0/817583455 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5560f6a5fd90 0x5560f6a601f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:03.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.110+0000 7fac8098ab80 1 --2- 172.21.15.151:0/817583455 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5560f6a5b360 0x5560f6a5b740 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:03.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.110+0000 7fac8098ab80 1 --2- 172.21.15.151:0/817583455 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5560f6a5bc80 0x5560f6a5f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:03.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.110+0000 7fac8098ab80 1 -- 172.21.15.151:0/817583455 >> 172.21.15.151:0/817583455 conn(0x5560f6984be0 msgr2=0x5560f6a51810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:06:03.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.110+0000 7fac8098ab80 1 -- 172.21.15.151:0/817583455 shutdown_connections 2024-05-21T22:06:03.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.110+0000 7fac8098ab80 1 -- 172.21.15.151:0/817583455 wait complete. 2024-05-21T22:06:03.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.111+0000 7fac8098ab80 1 -- start start 2024-05-21T22:06:03.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.111+0000 7fac8098ab80 1 Processor -- start 2024-05-21T22:06:03.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.112+0000 7fac8098ab80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5560f6a5b360 0x5560f6a67260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:06:03.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.112+0000 7fac8098ab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5560f6a5bc80 0x5560f6a64650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:06:03.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.112+0000 7fac8098ab80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5560f6a5fd90 0x5560f6a64b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:06:03.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.112+0000 7fac8098ab80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5560f68aafa0 con 0x5560f6a5b360 2024-05-21T22:06:03.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.112+0000 7fac8098ab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x5560f6945760 con 0x5560f6a5bc80 2024-05-21T22:06:03.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.112+0000 7fac8098ab80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x5560f6a67fc0 con 0x5560f6a5fd90 2024-05-21T22:06:03.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.112+0000 7fac8098ab80 10 client.0 _ll_register_callbacks cb 0x5560f694ec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:06:03.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.112+0000 7fac6fe00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5560f6a5b360 0x5560f6a67260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:06:03.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.112+0000 7fac6f400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5560f6a5bc80 0x5560f6a64650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:06:03.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.112+0000 7fac6f400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5560f6a5bc80 0x5560f6a64650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:34962/0 (socket says 172.21.15.151:34962) 2024-05-21T22:06:03.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.112+0000 7fac6f400700 1 -- 172.21.15.151:0/817225392 learned_addr learned my addr 172.21.15.151:0/817225392 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:06:03.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.112+0000 7fac6fe00700 1 --2- 172.21.15.151:0/817225392 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5560f6a5fd90 0x5560f6a64b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:06:03.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.113+0000 7fac6f400700 1 -- 172.21.15.151:0/817225392 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5560f6a5fd90 msgr2=0x5560f6a64b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:06:03.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.113+0000 7fac6f400700 1 --2- 172.21.15.151:0/817225392 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5560f6a5fd90 0x5560f6a64b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:03.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.113+0000 7fac6f400700 1 -- 172.21.15.151:0/817225392 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5560f6a5b360 msgr2=0x5560f6a67260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:06:03.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.113+0000 7fac6f400700 1 --2- 172.21.15.151:0/817225392 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5560f6a5b360 0x5560f6a67260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:03.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.113+0000 7fac6f400700 1 -- 172.21.15.151:0/817225392 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560f68f48d0 con 0x5560f6a5bc80 2024-05-21T22:06:03.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.113+0000 7fac6fe00700 1 --2- 172.21.15.151:0/817225392 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5560f6a5b360 0x5560f6a67260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T22:06:03.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.113+0000 7fac6f400700 1 --2- 172.21.15.151:0/817225392 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5560f6a5bc80 0x5560f6a64650 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fac60012860 tx=0x7fac60012c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51d49142edc8a0e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:06:03.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.113+0000 7fac5d600700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-21T22:06:03.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.113+0000 7fac5d600700 1 -- 172.21.15.151:0/817225392 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac60012e60 con 0x5560f6a5bc80 2024-05-21T22:06:03.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.113+0000 7fac8098ab80 1 -- 172.21.15.151:0/817225392 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac68003680 con 0x5560f6a5bc80 2024-05-21T22:06:03.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.113+0000 7fac5d600700 1 -- 172.21.15.151:0/817225392 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7fac60004030 con 0x5560f6a5bc80 2024-05-21T22:06:03.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.113+0000 7fac5d600700 1 -- 172.21.15.151:0/817225392 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fac60004230 con 0x5560f6a5bc80 2024-05-21T22:06:03.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.114+0000 7fac8098ab80 10 client.27668 fetch_fsmap learned FSMap version 29 2024-05-21T22:06:03.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.114+0000 7fac8098ab80 1 -- 172.21.15.151:0/817225392 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5560f6975060 con 0x5560f6a5bc80 2024-05-21T22:06:03.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.114+0000 7fac5d600700 1 -- 172.21.15.151:0/817225392 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac68003680 con 0x5560f6a5bc80 2024-05-21T22:06:03.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.114+0000 7fac5d600700 1 -- 172.21.15.151:0/817225392 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fac600039c0 con 0x5560f6a5bc80 2024-05-21T22:06:03.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.114+0000 7fac8098ab80 10 client.27668 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:06:03.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.114+0000 7fac8098ab80 10 client.27668 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T22:06:03.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.114+0000 7fac8098ab80 20 client.27668 populate_metadata read hostname 'smithi151' 2024-05-21T22:06:03.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.114+0000 7fac8098ab80 1 --2- 172.21.15.151:0/817225392 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5560f6a74c20 0x5560f6a77050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:06:03.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.114+0000 7fac8098ab80 4 client.27668 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T22:06:03.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.114+0000 7fac8098ab80 1 -- 172.21.15.151:0/817225392 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5560f69b6d90 con 0x5560f6a74c20 2024-05-21T22:06:03.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.115+0000 7fac6fe00700 1 --2- 172.21.15.151:0/817225392 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5560f6a74c20 0x5560f6a77050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:06:03.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.115+0000 7fac6fe00700 1 --2- 172.21.15.151:0/817225392 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5560f6a74c20 0x5560f6a77050 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29a64ab5224ce557 server_cookie=dd2d5e79a67930a1 in_seq=0 out_seq=0 2024-05-21T22:06:03.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.115+0000 7fac5d600700 10 client.27668 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T22:06:03.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.116+0000 7fac5d600700 1 -- 172.21.15.151:0/817225392 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5560f69b6d90 con 0x5560f6a74c20 2024-05-21T22:06:03.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.116+0000 7fac5d600700 10 client.27668 handle_command_reply: tid=0 2024-05-21T22:06:03.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.116+0000 7fac8098ab80 1 client.27668 shutdown 2024-05-21T22:06:03.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.116+0000 7fac8098ab80 1 -- 172.21.15.151:0/817225392 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5560f6a5bc80 msgr2=0x5560f6a64650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:06:03.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.116+0000 7fac8098ab80 1 --2- 172.21.15.151:0/817225392 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5560f6a5bc80 0x5560f6a64650 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fac60012860 tx=0x7fac60012c20 comp rx=0 tx=0).stop 2024-05-21T22:06:03.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.116+0000 7fac8098ab80 1 -- 172.21.15.151:0/817225392 shutdown_connections 2024-05-21T22:06:03.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.116+0000 7fac8098ab80 1 --2- 172.21.15.151:0/817225392 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5560f6a5fd90 0x5560f6a64b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:03.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.116+0000 7fac8098ab80 1 --2- 172.21.15.151:0/817225392 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5560f6a5b360 0x5560f6a67260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:03.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.116+0000 7fac8098ab80 1 --2- 172.21.15.151:0/817225392 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5560f6a74c20 0x5560f6a77050 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:03.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.116+0000 7fac8098ab80 1 --2- 172.21.15.151:0/817225392 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5560f6a5bc80 0x5560f6a64650 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:03.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.116+0000 7fac8098ab80 1 -- 172.21.15.151:0/817225392 >> 172.21.15.151:0/817225392 conn(0x5560f6984be0 msgr2=0x5560f6a50fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:06:03.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.116+0000 7fac5d600700 0 client.27668 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T22:06:03.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.117+0000 7fac8098ab80 1 -- 172.21.15.151:0/817225392 shutdown_connections 2024-05-21T22:06:03.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.117+0000 7fac8098ab80 1 -- 172.21.15.151:0/817225392 wait complete. 2024-05-21T22:06:03.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.117+0000 7fac8098ab80 20 client.27668 trim_cache size 0 max 16384 2024-05-21T22:06:03.134 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.133+0000 7fac8098ab80 1 Processor -- start 2024-05-21T22:06:03.134 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.133+0000 7fac8098ab80 1 -- start start 2024-05-21T22:06:03.134 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.134+0000 7fac8098ab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5560f6a5b360 0x5560f6a60fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:06:03.134 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.134+0000 7fac8098ab80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5560f6a5bc80 0x5560f6a51740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:06:03.134 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.134+0000 7fac8098ab80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5560f6a5fd90 0x5560f6a7c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:06:03.134 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.134+0000 7fac8098ab80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fac701aba60 con 0x5560f6a5fd90 2024-05-21T22:06:03.134 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.134+0000 7fac8098ab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fac701ad5a0 con 0x5560f6a5b360 2024-05-21T22:06:03.135 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.134+0000 7fac8098ab80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fac701ac130 con 0x5560f6a5bc80 2024-05-21T22:06:03.136 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.134+0000 7fac6f400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5560f6a5bc80 0x5560f6a51740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:06:03.136 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.134+0000 7fac6f400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5560f6a5bc80 0x5560f6a51740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.151:53054/0 (socket says 172.21.15.151:53054) 2024-05-21T22:06:03.136 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.134+0000 7fac6f400700 1 -- 172.21.15.151:0/4021199668 learned_addr learned my addr 172.21.15.151:0/4021199668 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:06:03.136 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.134+0000 7fac6fe00700 1 --2- 172.21.15.151:0/4021199668 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5560f6a5b360 0x5560f6a60fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:06:03.136 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.134+0000 7fac6f400700 1 -- 172.21.15.151:0/4021199668 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5560f6a5b360 msgr2=0x5560f6a60fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:06:03.136 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.134+0000 7fac6f400700 1 --2- 172.21.15.151:0/4021199668 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5560f6a5b360 0x5560f6a60fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:03.136 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.134+0000 7fac6fe00700 1 --2- 172.21.15.151:0/4021199668 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5560f6a5fd90 0x5560f6a7c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:06:03.136 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.135+0000 7fac6f400700 1 -- 172.21.15.151:0/4021199668 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5560f6a5fd90 msgr2=0x5560f6a7c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:06:03.136 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.135+0000 7fac6f400700 1 --2- 172.21.15.151:0/4021199668 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5560f6a5fd90 0x5560f6a7c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:03.137 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.135+0000 7fac6f400700 1 -- 172.21.15.151:0/4021199668 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560f6a72640 con 0x5560f6a5bc80 2024-05-21T22:06:03.137 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.135+0000 7fac6fe00700 1 --2- 172.21.15.151:0/4021199668 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5560f6a5fd90 0x5560f6a7c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T22:06:03.137 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.135+0000 7fac6f400700 1 --2- 172.21.15.151:0/4021199668 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5560f6a5bc80 0x5560f6a51740 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fac600096b0 tx=0x7fac6000f6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c249cdb9ed9ebbe6 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:06:03.137 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.135+0000 7fac5fe00700 1 -- 172.21.15.151:0/4021199668 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac60028070 con 0x5560f6a5bc80 2024-05-21T22:06:03.137 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.135+0000 7fac5fe00700 1 -- 172.21.15.151:0/4021199668 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac6000fce0 con 0x5560f6a5bc80 2024-05-21T22:06:03.137 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.135+0000 7fac5fe00700 1 -- 172.21.15.151:0/4021199668 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac60003a40 con 0x5560f6a5bc80 2024-05-21T22:06:03.137 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.136+0000 7fac8098ab80 1 -- 172.21.15.151:0/4021199668 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5560f6a5bc80 msgr2=0x5560f6a51740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:06:03.137 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.136+0000 7fac8098ab80 1 --2- 172.21.15.151:0/4021199668 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5560f6a5bc80 0x5560f6a51740 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fac600096b0 tx=0x7fac6000f6a0 comp rx=0 tx=0).stop 2024-05-21T22:06:03.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.136+0000 7fac8098ab80 1 -- 172.21.15.151:0/4021199668 shutdown_connections 2024-05-21T22:06:03.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.136+0000 7fac8098ab80 1 --2- 172.21.15.151:0/4021199668 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5560f6a5b360 0x5560f6a60fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:03.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.136+0000 7fac8098ab80 1 --2- 172.21.15.151:0/4021199668 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5560f6a5fd90 0x5560f6a7c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:03.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.136+0000 7fac8098ab80 1 --2- 172.21.15.151:0/4021199668 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5560f6a5bc80 0x5560f6a51740 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:03.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.136+0000 7fac8098ab80 1 -- 172.21.15.151:0/4021199668 >> 172.21.15.151:0/4021199668 conn(0x5560f6984be0 msgr2=0x5560f6a608c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:06:03.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.136+0000 7fac8098ab80 1 -- 172.21.15.151:0/4021199668 shutdown_connections 2024-05-21T22:06:03.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.136+0000 7fac8098ab80 1 -- 172.21.15.151:0/4021199668 wait complete. 2024-05-21T22:06:03.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.137+0000 7fac8098ab80 1 -- start start 2024-05-21T22:06:03.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.137+0000 7fac8098ab80 1 Processor -- start 2024-05-21T22:06:03.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.137+0000 7fac8098ab80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5560f6a5b360 0x5560f6a6d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:06:03.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.137+0000 7fac8098ab80 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5560f6a5bc80 0x5560f6a6d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:06:03.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.137+0000 7fac8098ab80 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5560f6a5fd90 0x5560f6a57f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:06:03.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.137+0000 7fac8098ab80 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fac701aced0 con 0x5560f6a5fd90 2024-05-21T22:06:03.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.137+0000 7fac8098ab80 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fac701ac800 con 0x5560f6a5bc80 2024-05-21T22:06:03.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.137+0000 7fac8098ab80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fac701a9a40 con 0x5560f6a5b360 2024-05-21T22:06:03.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.137+0000 7fac8098ab80 10 client.0 _ll_register_callbacks cb 0x5560f694ec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T22:06:03.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.138+0000 7fac6f400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5560f6a5bc80 0x5560f6a6d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:06:03.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.138+0000 7fac6fe00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5560f6a5fd90 0x5560f6a57f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:06:03.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.138+0000 7fac6f400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5560f6a5bc80 0x5560f6a6d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:34976/0 (socket says 172.21.15.151:34976) 2024-05-21T22:06:03.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.138+0000 7fac6f400700 1 -- 172.21.15.151:0/1417372476 learned_addr learned my addr 172.21.15.151:0/1417372476 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:06:03.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.138+0000 7fac6fe00700 1 --2- 172.21.15.151:0/1417372476 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5560f6a5b360 0x5560f6a6d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:06:03.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.138+0000 7fac6f400700 1 -- 172.21.15.151:0/1417372476 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5560f6a5b360 msgr2=0x5560f6a6d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:06:03.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.138+0000 7fac6f400700 1 --2- 172.21.15.151:0/1417372476 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5560f6a5b360 0x5560f6a6d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:03.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.138+0000 7fac6f400700 1 -- 172.21.15.151:0/1417372476 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5560f6a5fd90 msgr2=0x5560f6a57f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T22:06:03.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.138+0000 7fac6f400700 1 --2- 172.21.15.151:0/1417372476 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5560f6a5fd90 0x5560f6a57f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:03.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.138+0000 7fac6f400700 1 -- 172.21.15.151:0/1417372476 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560f6a585e0 con 0x5560f6a5bc80 2024-05-21T22:06:03.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.138+0000 7fac6f400700 1 --2- 172.21.15.151:0/1417372476 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5560f6a5bc80 0x5560f6a6d840 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fac60012c60 tx=0x7fac6000fa60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75de825d6f0948ad server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:06:03.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.138+0000 7fac5d600700 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-05-21T22:06:03.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.139+0000 7fac5d600700 1 -- 172.21.15.151:0/1417372476 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac60031070 con 0x5560f6a5bc80 2024-05-21T22:06:03.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.139+0000 7fac8098ab80 1 -- 172.21.15.151:0/1417372476 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac6000fce0 con 0x5560f6a5bc80 2024-05-21T22:06:03.142 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.139+0000 7fac5d600700 1 -- 172.21.15.151:0/1417372476 <== mon.2 v2:172.21.15.164:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7fac60003ad0 con 0x5560f6a5bc80 2024-05-21T22:06:03.142 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.139+0000 7fac5d600700 1 -- 172.21.15.151:0/1417372476 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fac6003c050 con 0x5560f6a5bc80 2024-05-21T22:06:03.142 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.139+0000 7fac8098ab80 10 client.27674 fetch_fsmap learned FSMap version 29 2024-05-21T22:06:03.142 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.139+0000 7fac8098ab80 1 -- 172.21.15.151:0/1417372476 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5560f6a6acc0 con 0x5560f6a5bc80 2024-05-21T22:06:03.142 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.139+0000 7fac5d600700 1 -- 172.21.15.151:0/1417372476 <== mon.2 v2:172.21.15.164:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac6000fce0 con 0x5560f6a5bc80 2024-05-21T22:06:03.142 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.139+0000 7fac5d600700 1 -- 172.21.15.151:0/1417372476 <== mon.2 v2:172.21.15.164:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fac60025480 con 0x5560f6a5bc80 2024-05-21T22:06:03.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.139+0000 7fac8098ab80 10 client.27674 fetch_fsmap finished waiting for FSMap version 29 2024-05-21T22:06:03.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.139+0000 7fac8098ab80 10 client.27674 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-21T22:06:03.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.139+0000 7fac8098ab80 20 client.27674 populate_metadata read hostname 'smithi151' 2024-05-21T22:06:03.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.139+0000 7fac8098ab80 1 --2- 172.21.15.151:0/1417372476 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5560f6ae4170 0x5560f6ae6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:06:03.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.140+0000 7fac8098ab80 4 client.27674 mds_command: new command op to 24530 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-21T22:06:03.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.140+0000 7fac8098ab80 1 -- 172.21.15.151:0/1417372476 --> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5560f69b0570 con 0x5560f6ae4170 2024-05-21T22:06:03.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.140+0000 7fac6fe00700 1 --2- 172.21.15.151:0/1417372476 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5560f6ae4170 0x5560f6ae6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:06:03.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.140+0000 7fac6fe00700 1 --2- 172.21.15.151:0/1417372476 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5560f6ae4170 0x5560f6ae6590 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a38fb44bd0f5aaf server_cookie=adad6101ecc5d69 in_seq=0 out_seq=0 2024-05-21T22:06:03.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.140+0000 7fac5d600700 10 client.27674 ms_handle_connect on v2:172.21.15.164:6838/2646401247 2024-05-21T22:06:03.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.140+0000 7fac5d600700 1 -- 172.21.15.151:0/1417372476 <== mds.0 v2:172.21.15.164:6838/2646401247 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5560f69b0570 con 0x5560f6ae4170 2024-05-21T22:06:03.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.141+0000 7fac5d600700 10 client.27674 handle_command_reply: tid=0 2024-05-21T22:06:03.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.141+0000 7fac8098ab80 1 client.27674 shutdown 2024-05-21T22:06:03.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.141+0000 7fac8098ab80 1 -- 172.21.15.151:0/1417372476 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5560f6a5bc80 msgr2=0x5560f6a6d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:06:03.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.141+0000 7fac8098ab80 1 --2- 172.21.15.151:0/1417372476 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5560f6a5bc80 0x5560f6a6d840 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fac60012c60 tx=0x7fac6000fa60 comp rx=0 tx=0).stop 2024-05-21T22:06:03.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.141+0000 7fac8098ab80 1 -- 172.21.15.151:0/1417372476 shutdown_connections 2024-05-21T22:06:03.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.141+0000 7fac8098ab80 1 --2- 172.21.15.151:0/1417372476 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5560f6a5fd90 0x5560f6a57f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:03.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.141+0000 7fac8098ab80 1 --2- 172.21.15.151:0/1417372476 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5560f6a5b360 0x5560f6a6d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:03.145 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.141+0000 7fac8098ab80 1 --2- 172.21.15.151:0/1417372476 >> [v2:172.21.15.164:6838/2646401247,v1:172.21.15.164:6839/2646401247] conn(0x5560f6ae4170 0x5560f6ae6590 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:03.145 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.141+0000 7fac8098ab80 1 --2- 172.21.15.151:0/1417372476 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x5560f6a5bc80 0x5560f6a6d840 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:03.145 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.141+0000 7fac8098ab80 1 -- 172.21.15.151:0/1417372476 >> 172.21.15.151:0/1417372476 conn(0x5560f6984be0 msgr2=0x5560f6a5f470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:06:03.145 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.141+0000 7fac5d600700 0 client.27674 ms_handle_reset on v2:172.21.15.164:6838/2646401247 2024-05-21T22:06:03.145 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.141+0000 7fac8098ab80 1 -- 172.21.15.151:0/1417372476 shutdown_connections 2024-05-21T22:06:03.145 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.141+0000 7fac8098ab80 1 -- 172.21.15.151:0/1417372476 wait complete. 2024-05-21T22:06:03.145 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.141+0000 7fac8098ab80 20 client.27674 trim_cache size 0 max 16384 2024-05-21T22:06:03.146 INFO:teuthology.orchestra.run.smithi151.stdout:[] 2024-05-21T22:06:03.146 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.142+0000 7fac74a00700 1 -- 172.21.15.151:0/4235727879 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fac540b2a70 msgr2=0x7fac540b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:06:03.146 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.142+0000 7fac74a00700 1 --2- 172.21.15.151:0/4235727879 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fac540b2a70 0x7fac540b4f30 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fac5801dfd0 tx=0x7fac5800cf50 comp rx=0 tx=0).stop 2024-05-21T22:06:03.146 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.142+0000 7fac74a00700 1 -- 172.21.15.151:0/4235727879 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac70103e10 msgr2=0x7fac7019fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:06:03.146 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.142+0000 7fac74a00700 1 --2- 172.21.15.151:0/4235727879 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac70103e10 0x7fac7019fd50 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fac58000f80 tx=0x7fac5800ccb0 comp rx=0 tx=0).stop 2024-05-21T22:06:03.147 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.142+0000 7fac74a00700 1 -- 172.21.15.151:0/4235727879 shutdown_connections 2024-05-21T22:06:03.147 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.142+0000 7fac74a00700 1 --2- 172.21.15.151:0/4235727879 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fac540b2a70 0x7fac540b4f30 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:03.147 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.142+0000 7fac74a00700 1 --2- 172.21.15.151:0/4235727879 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fac701094e0 0x7fac70199a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:03.147 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.142+0000 7fac74a00700 1 --2- 172.21.15.151:0/4235727879 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac70103450 0x7fac7019f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:03.147 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.142+0000 7fac74a00700 1 --2- 172.21.15.151:0/4235727879 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac70103e10 0x7fac7019fd50 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:03.147 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.142+0000 7fac74a00700 1 -- 172.21.15.151:0/4235727879 >> 172.21.15.151:0/4235727879 conn(0x7fac700fef60 msgr2=0x7fac700ff780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:06:03.147 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.142+0000 7fac74a00700 1 -- 172.21.15.151:0/4235727879 shutdown_connections 2024-05-21T22:06:03.147 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:03.142+0000 7fac74a00700 1 -- 172.21.15.151:0/4235727879 wait complete. 2024-05-21T22:06:03.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:06:03 smithi164 ceph-mon[36269]: pgmap v322: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 44 KiB/s wr, 32 op/s 2024-05-21T22:06:03.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:06:03 smithi151 ceph-mon[31623]: pgmap v322: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 44 KiB/s wr, 32 op/s 2024-05-21T22:06:03.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:06:03 smithi184 ceph-mon[36942]: pgmap v322: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 44 KiB/s wr, 32 op/s 2024-05-21T22:06:04.059 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_7956b5e965f6ee5669688b5643c11d881b72442c/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_7956b5e965f6ee5669688b5643c11d881b72442c/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph-c_7956b5e965f6ee5669688b5643c11d881b72442c/qa/tasks/thrasher.py", line 47, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-05-21T22:06:04.060 INFO:tasks.fwd_scrub:done joining 2024-05-21T22:06:04.060 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-05-21T22:06:04.070 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-21T22:06:04.289 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.288+0000 7fc6f3200640 1 -- 172.21.15.151:0/2318791535 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc6f41136f0 msgr2=0x7fc6f4115ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:06:04.289 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.288+0000 7fc6f3200640 1 --2- 172.21.15.151:0/2318791535 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc6f41136f0 0x7fc6f4115ae0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec009cd0 tx=0x7fc6ec02a8e0 comp rx=0 tx=0).stop 2024-05-21T22:06:04.289 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.288+0000 7fc6f3200640 1 -- 172.21.15.151:0/2318791535 shutdown_connections 2024-05-21T22:06:04.289 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.288+0000 7fc6f3200640 1 --2- 172.21.15.151:0/2318791535 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc6f41136f0 0x7fc6f4115ae0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:04.289 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.288+0000 7fc6f3200640 1 --2- 172.21.15.151:0/2318791535 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc6f40778a0 0x7fc6f41131b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:04.289 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.288+0000 7fc6f3200640 1 --2- 172.21.15.151:0/2318791535 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc6f4076f80 0x7fc6f4077360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:04.290 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.288+0000 7fc6f3200640 1 -- 172.21.15.151:0/2318791535 >> 172.21.15.151:0/2318791535 conn(0x7fc6f4100520 msgr2=0x7fc6f4102940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:06:04.290 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.288+0000 7fc6f3200640 1 -- 172.21.15.151:0/2318791535 shutdown_connections 2024-05-21T22:06:04.290 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.289+0000 7fc6f3200640 1 -- 172.21.15.151:0/2318791535 wait complete. 2024-05-21T22:06:04.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.289+0000 7fc6f3200640 1 Processor -- start 2024-05-21T22:06:04.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.290+0000 7fc6f3200640 1 -- start start 2024-05-21T22:06:04.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.290+0000 7fc6f3200640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc6f4076f80 0x7fc6f419f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:06:04.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.290+0000 7fc6f3200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc6f40778a0 0x7fc6f419fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:06:04.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.290+0000 7fc6f3200640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc6f41136f0 0x7fc6f41a3f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:06:04.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.290+0000 7fc6f3200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f4119110 con 0x7fc6f40778a0 2024-05-21T22:06:04.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.290+0000 7fc6f3200640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f4118f90 con 0x7fc6f4076f80 2024-05-21T22:06:04.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.290+0000 7fc6f3200640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f4119290 con 0x7fc6f41136f0 2024-05-21T22:06:04.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.290+0000 7fc6f1400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc6f40778a0 0x7fc6f419fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:06:04.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.290+0000 7fc6f1400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc6f40778a0 0x7fc6f419fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:53438/0 (socket says 172.21.15.151:53438) 2024-05-21T22:06:04.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.290+0000 7fc6f1400640 1 -- 172.21.15.151:0/3425520162 learned_addr learned my addr 172.21.15.151:0/3425520162 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:06:04.293 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.290+0000 7fc6f1e00640 1 --2- 172.21.15.151:0/3425520162 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc6f4076f80 0x7fc6f419f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:06:04.293 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.290+0000 7fc6f2800640 1 --2- 172.21.15.151:0/3425520162 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc6f41136f0 0x7fc6f41a3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:06:04.293 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.291+0000 7fc6f1400640 1 -- 172.21.15.151:0/3425520162 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc6f41136f0 msgr2=0x7fc6f41a3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:06:04.293 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.291+0000 7fc6f1400640 1 --2- 172.21.15.151:0/3425520162 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc6f41136f0 0x7fc6f41a3f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:04.293 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.291+0000 7fc6f1400640 1 -- 172.21.15.151:0/3425520162 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc6f4076f80 msgr2=0x7fc6f419f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:06:04.293 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.291+0000 7fc6f1400640 1 --2- 172.21.15.151:0/3425520162 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc6f4076f80 0x7fc6f419f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:04.293 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.291+0000 7fc6f1400640 1 -- 172.21.15.151:0/3425520162 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6f41a4680 con 0x7fc6f40778a0 2024-05-21T22:06:04.293 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.291+0000 7fc6f1e00640 1 --2- 172.21.15.151:0/3425520162 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc6f4076f80 0x7fc6f419f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:06:04.293 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.291+0000 7fc6f1400640 1 --2- 172.21.15.151:0/3425520162 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc6f40778a0 0x7fc6f419fbd0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fc6e000e9f0 tx=0x7fc6e000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:06:04.293 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.291+0000 7fc6eaa00640 1 -- 172.21.15.151:0/3425520162 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e000cd80 con 0x7fc6f40778a0 2024-05-21T22:06:04.293 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.291+0000 7fc6eaa00640 1 -- 172.21.15.151:0/3425520162 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6e00108e0 con 0x7fc6f40778a0 2024-05-21T22:06:04.293 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.291+0000 7fc6f3200640 1 -- 172.21.15.151:0/3425520162 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6f41a4970 con 0x7fc6f40778a0 2024-05-21T22:06:04.293 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.291+0000 7fc6eaa00640 1 -- 172.21.15.151:0/3425520162 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e0018980 con 0x7fc6f40778a0 2024-05-21T22:06:04.294 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.292+0000 7fc6f3200640 1 -- 172.21.15.151:0/3425520162 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6f41ac200 con 0x7fc6f40778a0 2024-05-21T22:06:04.294 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.293+0000 7fc6f3200640 1 -- 172.21.15.151:0/3425520162 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6f4069be0 con 0x7fc6f40778a0 2024-05-21T22:06:04.296 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.296+0000 7fc6eaa00640 1 -- 172.21.15.151:0/3425520162 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7fc6e0010430 con 0x7fc6f40778a0 2024-05-21T22:06:04.297 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.297+0000 7fc6eaa00640 1 --2- 172.21.15.151:0/3425520162 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fc6d80b2960 0x7fc6d80b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:06:04.297 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.297+0000 7fc6eaa00640 1 -- 172.21.15.151:0/3425520162 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7fc6e00e31e0 con 0x7fc6f40778a0 2024-05-21T22:06:04.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.298+0000 7fc6f1e00640 1 --2- 172.21.15.151:0/3425520162 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fc6d80b2960 0x7fc6d80b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:06:04.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.298+0000 7fc6f1e00640 1 --2- 172.21.15.151:0/3425520162 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fc6d80b2960 0x7fc6d80b4e20 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4009fc0 tx=0x7fc6e4005f90 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:06:04.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.299+0000 7fc6eaa00640 1 -- 172.21.15.151:0/3425520162 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc6e00aab50 con 0x7fc6f40778a0 2024-05-21T22:06:04.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.513+0000 7fc6f3200640 1 -- 172.21.15.151:0/3425520162 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc6f41a09e0 con 0x7fc6f40778a0 2024-05-21T22:06:04.517 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.517+0000 7fc6eaa00640 1 -- 172.21.15.151:0/3425520162 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+23218 (secure 0 0 0) 0x7fc6f41a09e0 con 0x7fc6f40778a0 2024-05-21T22:06:04.518 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T22:06:04.519 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":89,"fsid":"c8c7a2e6-17ba-11ef-bc9a-c7b262605968","created":"2024-05-21T21:41:52.412833+0000","modified":"2024-05-21T22:04:14.141852+0000","last_up_change":"2024-05-21T21:50:05.401314+0000","last_in_change":"2024-05-21T21:49:45.684136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-21T21:45:55.196046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-21T21:51:11.447428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-21T21:51:15.618605+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":13,"snap_epoch":88,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"18db8e54-0cfc-4e12-9c84-6f72a99cc989","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":2420400473},{"type":"v1","addr":"172.21.15.151:6803","nonce":2420400473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":2420400473},{"type":"v1","addr":"172.21.15.151:6805","nonce":2420400473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":2420400473},{"type":"v1","addr":"172.21.15.151:6809","nonce":2420400473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":2420400473},{"type":"v1","addr":"172.21.15.151:6807","nonce":2420400473}]},"public_addr":"172.21.15.151:6803/2420400473","cluster_addr":"172.21.15.151:6805/2420400473","heartbeat_back_addr":"172.21.15.151:6809/2420400473","heartbeat_front_addr":"172.21.15.151:6807/2420400473","state":["exists","up"]},{"osd":1,"uuid":"bac384c7-1102-4815-81a8-10cc7661d230","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":3900240447},{"type":"v1","addr":"172.21.15.164:6801","nonce":3900240447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":3900240447},{"type":"v1","addr":"172.21.15.164:6803","nonce":3900240447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":3900240447},{"type":"v1","addr":"172.21.15.164:6807","nonce":3900240447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":3900240447},{"type":"v1","addr":"172.21.15.164:6805","nonce":3900240447}]},"public_addr":"172.21.15.164:6801/3900240447","cluster_addr":"172.21.15.164:6803/3900240447","heartbeat_back_addr":"172.21.15.164:6807/3900240447","heartbeat_front_addr":"172.21.15.164:6805/3900240447","state":["exists","up"]},{"osd":2,"uuid":"b7e4a782-1518-4d43-bee9-7fd768ee5cb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":3971150737},{"type":"v1","addr":"172.21.15.184:6801","nonce":3971150737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3971150737},{"type":"v1","addr":"172.21.15.184:6803","nonce":3971150737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3971150737},{"type":"v1","addr":"172.21.15.184:6807","nonce":3971150737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3971150737},{"type":"v1","addr":"172.21.15.184:6805","nonce":3971150737}]},"public_addr":"172.21.15.184:6801/3971150737","cluster_addr":"172.21.15.184:6803/3971150737","heartbeat_back_addr":"172.21.15.184:6807/3971150737","heartbeat_front_addr":"172.21.15.184:6805/3971150737","state":["exists","up"]},{"osd":3,"uuid":"ab3df826-39e3-4635-a49c-2b5ffdbf12f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":556594400},{"type":"v1","addr":"172.21.15.151:6811","nonce":556594400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":556594400},{"type":"v1","addr":"172.21.15.151:6813","nonce":556594400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":556594400},{"type":"v1","addr":"172.21.15.151:6817","nonce":556594400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":556594400},{"type":"v1","addr":"172.21.15.151:6815","nonce":556594400}]},"public_addr":"172.21.15.151:6811/556594400","cluster_addr":"172.21.15.151:6813/556594400","heartbeat_back_addr":"172.21.15.151:6817/556594400","heartbeat_front_addr":"172.21.15.151:6815/556594400","state":["exists","up"]},{"osd":4,"uuid":"00cbf6ae-0f42-4685-a77d-d803c6467aa0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":988881973},{"type":"v1","addr":"172.21.15.164:6809","nonce":988881973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":988881973},{"type":"v1","addr":"172.21.15.164:6811","nonce":988881973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":988881973},{"type":"v1","addr":"172.21.15.164:6815","nonce":988881973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":988881973},{"type":"v1","addr":"172.21.15.164:6813","nonce":988881973}]},"public_addr":"172.21.15.164:6809/988881973","cluster_addr":"172.21.15.164:6811/988881973","heartbeat_back_addr":"172.21.15.164:6815/988881973","heartbeat_front_addr":"172.21.15.164:6813/988881973","state":["exists","up"]},{"osd":5,"uuid":"fad14221-0ef0-4d40-b414-c6c9f1f9718c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3293311106},{"type":"v1","addr":"172.21.15.184:6809","nonce":3293311106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":3293311106},{"type":"v1","addr":"172.21.15.184:6811","nonce":3293311106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":3293311106},{"type":"v1","addr":"172.21.15.184:6815","nonce":3293311106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":3293311106},{"type":"v1","addr":"172.21.15.184:6813","nonce":3293311106}]},"public_addr":"172.21.15.184:6809/3293311106","cluster_addr":"172.21.15.184:6811/3293311106","heartbeat_back_addr":"172.21.15.184:6815/3293311106","heartbeat_front_addr":"172.21.15.184:6813/3293311106","state":["exists","up"]},{"osd":6,"uuid":"ab8a60f6-0dec-4c1c-89db-f2e2f638f1ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":3377550762},{"type":"v1","addr":"172.21.15.151:6819","nonce":3377550762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":3377550762},{"type":"v1","addr":"172.21.15.151:6821","nonce":3377550762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":3377550762},{"type":"v1","addr":"172.21.15.151:6825","nonce":3377550762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":3377550762},{"type":"v1","addr":"172.21.15.151:6823","nonce":3377550762}]},"public_addr":"172.21.15.151:6819/3377550762","cluster_addr":"172.21.15.151:6821/3377550762","heartbeat_back_addr":"172.21.15.151:6825/3377550762","heartbeat_front_addr":"172.21.15.151:6823/3377550762","state":["exists","up"]},{"osd":7,"uuid":"db332268-9972-4ed7-a307-822b83a0fbb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":3869696936},{"type":"v1","addr":"172.21.15.164:6817","nonce":3869696936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":3869696936},{"type":"v1","addr":"172.21.15.164:6819","nonce":3869696936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":3869696936},{"type":"v1","addr":"172.21.15.164:6823","nonce":3869696936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":3869696936},{"type":"v1","addr":"172.21.15.164:6821","nonce":3869696936}]},"public_addr":"172.21.15.164:6817/3869696936","cluster_addr":"172.21.15.164:6819/3869696936","heartbeat_back_addr":"172.21.15.164:6823/3869696936","heartbeat_front_addr":"172.21.15.164:6821/3869696936","state":["exists","up"]},{"osd":8,"uuid":"fe2fbe1c-549a-4947-a188-426dfca04d10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":575273541},{"type":"v1","addr":"172.21.15.184:6817","nonce":575273541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":575273541},{"type":"v1","addr":"172.21.15.184:6819","nonce":575273541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":575273541},{"type":"v1","addr":"172.21.15.184:6823","nonce":575273541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":575273541},{"type":"v1","addr":"172.21.15.184:6821","nonce":575273541}]},"public_addr":"172.21.15.184:6817/575273541","cluster_addr":"172.21.15.184:6819/575273541","heartbeat_back_addr":"172.21.15.184:6823/575273541","heartbeat_front_addr":"172.21.15.184:6821/575273541","state":["exists","up"]},{"osd":9,"uuid":"3a7b450f-8004-4adf-bae3-970918f98f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":337128533},{"type":"v1","addr":"172.21.15.151:6827","nonce":337128533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":337128533},{"type":"v1","addr":"172.21.15.151:6829","nonce":337128533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":337128533},{"type":"v1","addr":"172.21.15.151:6833","nonce":337128533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":337128533},{"type":"v1","addr":"172.21.15.151:6831","nonce":337128533}]},"public_addr":"172.21.15.151:6827/337128533","cluster_addr":"172.21.15.151:6829/337128533","heartbeat_back_addr":"172.21.15.151:6833/337128533","heartbeat_front_addr":"172.21.15.151:6831/337128533","state":["exists","up"]},{"osd":10,"uuid":"8fcdf271-90c4-4db3-be69-b0ca716dfd80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":1823448782},{"type":"v1","addr":"172.21.15.164:6825","nonce":1823448782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":1823448782},{"type":"v1","addr":"172.21.15.164:6827","nonce":1823448782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":1823448782},{"type":"v1","addr":"172.21.15.164:6831","nonce":1823448782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":1823448782},{"type":"v1","addr":"172.21.15.164:6829","nonce":1823448782}]},"public_addr":"172.21.15.164:6825/1823448782","cluster_addr":"172.21.15.164:6827/1823448782","heartbeat_back_addr":"172.21.15.164:6831/1823448782","heartbeat_front_addr":"172.21.15.164:6829/1823448782","state":["exists","up"]},{"osd":11,"uuid":"e661477b-0c09-409f-a2fe-6ebd34642a8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":1563340128},{"type":"v1","addr":"172.21.15.184:6825","nonce":1563340128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":1563340128},{"type":"v1","addr":"172.21.15.184:6827","nonce":1563340128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":1563340128},{"type":"v1","addr":"172.21.15.184:6831","nonce":1563340128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":1563340128},{"type":"v1","addr":"172.21.15.184:6829","nonce":1563340128}]},"public_addr":"172.21.15.184:6825/1563340128","cluster_addr":"172.21.15.184:6827/1563340128","heartbeat_back_addr":"172.21.15.184:6831/1563340128","heartbeat_front_addr":"172.21.15.184:6829/1563340128","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:45:02.821160+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:45:27.844564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:45:51.904981+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:46:24.376245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:46:50.036621+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:47:14.748422+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:47:47.396383+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:48:13.543174+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:48:38.213285+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:49:11.980042+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:49:38.519761+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:50:03.439898+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.151:0/3481112063":"2024-05-22T21:55:36.189531+0000","172.21.15.151:0/2415279581":"2024-05-22T21:55:36.189531+0000","172.21.15.151:0/1946920088":"2024-05-22T21:55:36.189531+0000","172.21.15.151:6800/2796622483":"2024-05-22T21:55:36.189531+0000","172.21.15.151:0/31963128":"2024-05-22T21:43:09.041455+0000","172.21.15.151:6801/2796622483":"2024-05-22T21:55:36.189531+0000","172.21.15.151:6800/3428827300":"2024-05-22T21:43:09.041455+0000","172.21.15.151:0/3232339048":"2024-05-22T21:55:36.189531+0000","172.21.15.151:6801/3428827300":"2024-05-22T21:43:09.041455+0000","172.21.15.151:0/131989141":"2024-05-22T21:43:09.041455+0000","172.21.15.151:6801/1128885292":"2024-05-22T21:42:35.440147+0000","172.21.15.151:0/2703664519":"2024-05-22T21:43:09.041455+0000","172.21.15.151:6800/1128885292":"2024-05-22T21:42:35.440147+0000","172.21.15.151:0/3052943699":"2024-05-22T21:42:35.440147+0000","172.21.15.151:0/3929452511":"2024-05-22T21:42:35.440147+0000","172.21.15.151:0/3762889700":"2024-05-22T21:42:35.440147+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":4,"length":1},{"begin":13,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-21T22:06:04.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.520+0000 7fc6f3200640 1 -- 172.21.15.151:0/3425520162 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fc6d80b2960 msgr2=0x7fc6d80b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:06:04.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.520+0000 7fc6f3200640 1 --2- 172.21.15.151:0/3425520162 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fc6d80b2960 0x7fc6d80b4e20 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4009fc0 tx=0x7fc6e4005f90 comp rx=0 tx=0).stop 2024-05-21T22:06:04.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.520+0000 7fc6f3200640 1 -- 172.21.15.151:0/3425520162 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc6f40778a0 msgr2=0x7fc6f419fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:06:04.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.520+0000 7fc6f3200640 1 --2- 172.21.15.151:0/3425520162 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc6f40778a0 0x7fc6f419fbd0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fc6e000e9f0 tx=0x7fc6e000eeb0 comp rx=0 tx=0).stop 2024-05-21T22:06:04.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.521+0000 7fc6f3200640 1 -- 172.21.15.151:0/3425520162 shutdown_connections 2024-05-21T22:06:04.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.521+0000 7fc6f3200640 1 --2- 172.21.15.151:0/3425520162 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fc6d80b2960 0x7fc6d80b4e20 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:04.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.521+0000 7fc6f3200640 1 --2- 172.21.15.151:0/3425520162 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc6f41136f0 0x7fc6f41a3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:04.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.521+0000 7fc6f3200640 1 --2- 172.21.15.151:0/3425520162 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc6f40778a0 0x7fc6f419fbd0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:04.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.521+0000 7fc6f3200640 1 --2- 172.21.15.151:0/3425520162 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc6f4076f80 0x7fc6f419f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:04.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.521+0000 7fc6f3200640 1 -- 172.21.15.151:0/3425520162 >> 172.21.15.151:0/3425520162 conn(0x7fc6f4100520 msgr2=0x7fc6f4114e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:06:04.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.521+0000 7fc6f3200640 1 -- 172.21.15.151:0/3425520162 shutdown_connections 2024-05-21T22:06:04.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.521+0000 7fc6f3200640 1 -- 172.21.15.151:0/3425520162 wait complete. 2024-05-21T22:06:04.541 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-21T21:45:55.196046+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-21T21:51:11.447428+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-21T21:51:15.618605+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 13, 'snap_epoch': 88, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-21T22:06:04.542 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-21T22:06:04.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.749+0000 7ff401600640 1 -- 172.21.15.151:0/635083169 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff3fc1043d0 msgr2=0x7ff3fc10c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:06:04.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.749+0000 7ff401600640 1 --2- 172.21.15.151:0/635083169 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff3fc1043d0 0x7ff3fc10c8b0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec009ca0 tx=0x7ff3ec02a940 comp rx=0 tx=0).stop 2024-05-21T22:06:04.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.749+0000 7ff401600640 1 -- 172.21.15.151:0/635083169 shutdown_connections 2024-05-21T22:06:04.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.749+0000 7ff401600640 1 --2- 172.21.15.151:0/635083169 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3fc10ced0 0x7ff3fc10f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:04.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.749+0000 7ff401600640 1 --2- 172.21.15.151:0/635083169 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff3fc1043d0 0x7ff3fc10c8b0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:04.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.749+0000 7ff401600640 1 --2- 172.21.15.151:0/635083169 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff3fc103ab0 0x7ff3fc103e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:04.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.749+0000 7ff401600640 1 -- 172.21.15.151:0/635083169 >> 172.21.15.151:0/635083169 conn(0x7ff3fc0fd2d0 msgr2=0x7ff3fc0ff6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:06:04.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.749+0000 7ff401600640 1 -- 172.21.15.151:0/635083169 shutdown_connections 2024-05-21T22:06:04.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.749+0000 7ff401600640 1 -- 172.21.15.151:0/635083169 wait complete. 2024-05-21T22:06:04.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.750+0000 7ff401600640 1 Processor -- start 2024-05-21T22:06:04.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.750+0000 7ff401600640 1 -- start start 2024-05-21T22:06:04.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.751+0000 7ff401600640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff3fc103ab0 0x7ff3fc101700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:06:04.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.751+0000 7ff401600640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3fc1043d0 0x7ff3fc101c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:06:04.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.751+0000 7ff401600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff3fc10ced0 0x7ff3fc102180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:06:04.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.751+0000 7ff401600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff3fc112b80 con 0x7ff3fc10ced0 2024-05-21T22:06:04.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.751+0000 7ff401600640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff3fc112a00 con 0x7ff3fc103ab0 2024-05-21T22:06:04.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.751+0000 7ff401600640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff3fc112d00 con 0x7ff3fc1043d0 2024-05-21T22:06:04.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.751+0000 7ff400c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff3fc10ced0 0x7ff3fc102180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:06:04.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.751+0000 7ff3fb400640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3fc1043d0 0x7ff3fc101c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:06:04.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.751+0000 7ff400c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff3fc10ced0 0x7ff3fc102180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:53458/0 (socket says 172.21.15.151:53458) 2024-05-21T22:06:04.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.751+0000 7ff400c00640 1 -- 172.21.15.151:0/4293369162 learned_addr learned my addr 172.21.15.151:0/4293369162 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:06:04.754 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.751+0000 7ff3fbe00640 1 --2- 172.21.15.151:0/4293369162 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff3fc103ab0 0x7ff3fc101700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:06:04.754 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.751+0000 7ff3fb400640 1 -- 172.21.15.151:0/4293369162 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff3fc103ab0 msgr2=0x7ff3fc101700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:06:04.754 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.751+0000 7ff3fb400640 1 --2- 172.21.15.151:0/4293369162 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff3fc103ab0 0x7ff3fc101700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:04.754 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.751+0000 7ff3fb400640 1 -- 172.21.15.151:0/4293369162 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff3fc10ced0 msgr2=0x7ff3fc102180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:06:04.754 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.751+0000 7ff3fb400640 1 --2- 172.21.15.151:0/4293369162 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff3fc10ced0 0x7ff3fc102180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:04.754 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.751+0000 7ff3fb400640 1 -- 172.21.15.151:0/4293369162 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3fc1a7620 con 0x7ff3fc1043d0 2024-05-21T22:06:04.754 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.751+0000 7ff400c00640 1 --2- 172.21.15.151:0/4293369162 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff3fc10ced0 0x7ff3fc102180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:06:04.754 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.752+0000 7ff3fb400640 1 --2- 172.21.15.151:0/4293369162 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3fc1043d0 0x7ff3fc101c40 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec006010 tx=0x7ff3ec030870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:06:04.754 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.752+0000 7ff3f8c00640 1 -- 172.21.15.151:0/4293369162 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3ec034040 con 0x7ff3fc1043d0 2024-05-21T22:06:04.754 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.752+0000 7ff3f8c00640 1 -- 172.21.15.151:0/4293369162 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3ec003e20 con 0x7ff3fc1043d0 2024-05-21T22:06:04.754 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.752+0000 7ff401600640 1 -- 172.21.15.151:0/4293369162 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3fc1a78b0 con 0x7ff3fc1043d0 2024-05-21T22:06:04.754 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.752+0000 7ff3f8c00640 1 -- 172.21.15.151:0/4293369162 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3ec004570 con 0x7ff3fc1043d0 2024-05-21T22:06:04.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.752+0000 7ff401600640 1 -- 172.21.15.151:0/4293369162 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3fc1a7da0 con 0x7ff3fc1043d0 2024-05-21T22:06:04.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.753+0000 7ff401600640 1 -- 172.21.15.151:0/4293369162 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3fc069790 con 0x7ff3fc1043d0 2024-05-21T22:06:04.756 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.756+0000 7ff3f8c00640 1 -- 172.21.15.151:0/4293369162 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7ff3ec037020 con 0x7ff3fc1043d0 2024-05-21T22:06:04.757 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.757+0000 7ff3f8c00640 1 --2- 172.21.15.151:0/4293369162 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7ff3e00b2960 0x7ff3e00b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:06:04.757 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.757+0000 7ff3f8c00640 1 -- 172.21.15.151:0/4293369162 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7ff3ec1032e0 con 0x7ff3fc1043d0 2024-05-21T22:06:04.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.758+0000 7ff3fbe00640 1 --2- 172.21.15.151:0/4293369162 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7ff3e00b2960 0x7ff3e00b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:06:04.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.758+0000 7ff3fbe00640 1 --2- 172.21.15.151:0/4293369162 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7ff3e00b2960 0x7ff3e00b4e20 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff3e4009bd0 tx=0x7ff3e4005f90 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:06:04.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.760+0000 7ff3f8c00640 1 -- 172.21.15.151:0/4293369162 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff3ec0cad60 con 0x7ff3fc1043d0 2024-05-21T22:06:04.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:06:04 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/3425520162' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T22:06:04.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.967+0000 7ff401600640 1 -- 172.21.15.151:0/4293369162 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff3fc107f70 con 0x7ff3fc1043d0 2024-05-21T22:06:04.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.969+0000 7ff3f8c00640 1 -- 172.21.15.151:0/4293369162 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7ff3ec0098b0 con 0x7ff3fc1043d0 2024-05-21T22:06:04.969 INFO:teuthology.orchestra.run.smithi151.stdout:pg_num: 1 2024-05-21T22:06:04.972 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.971+0000 7ff401600640 1 -- 172.21.15.151:0/4293369162 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7ff3e00b2960 msgr2=0x7ff3e00b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:06:04.972 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.971+0000 7ff401600640 1 --2- 172.21.15.151:0/4293369162 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7ff3e00b2960 0x7ff3e00b4e20 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff3e4009bd0 tx=0x7ff3e4005f90 comp rx=0 tx=0).stop 2024-05-21T22:06:04.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.972+0000 7ff401600640 1 -- 172.21.15.151:0/4293369162 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3fc1043d0 msgr2=0x7ff3fc101c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:06:04.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.972+0000 7ff401600640 1 --2- 172.21.15.151:0/4293369162 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3fc1043d0 0x7ff3fc101c40 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec006010 tx=0x7ff3ec030870 comp rx=0 tx=0).stop 2024-05-21T22:06:04.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.972+0000 7ff401600640 1 -- 172.21.15.151:0/4293369162 shutdown_connections 2024-05-21T22:06:04.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.972+0000 7ff401600640 1 --2- 172.21.15.151:0/4293369162 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7ff3e00b2960 0x7ff3e00b4e20 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:04.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.972+0000 7ff401600640 1 --2- 172.21.15.151:0/4293369162 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff3fc10ced0 0x7ff3fc102180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:04.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.972+0000 7ff401600640 1 --2- 172.21.15.151:0/4293369162 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3fc1043d0 0x7ff3fc101c40 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:04.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.972+0000 7ff401600640 1 --2- 172.21.15.151:0/4293369162 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff3fc103ab0 0x7ff3fc101700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:04.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.972+0000 7ff401600640 1 -- 172.21.15.151:0/4293369162 >> 172.21.15.151:0/4293369162 conn(0x7ff3fc0fd2d0 msgr2=0x7ff3fc0feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:06:04.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.972+0000 7ff401600640 1 -- 172.21.15.151:0/4293369162 shutdown_connections 2024-05-21T22:06:04.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:04.972+0000 7ff401600640 1 -- 172.21.15.151:0/4293369162 wait complete. 2024-05-21T22:06:04.985 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:06:04 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/3425520162' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T22:06:04.993 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-21T22:06:05.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:06:04 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/3425520162' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T22:06:05.193 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.192+0000 7f0dd3e00640 1 -- 172.21.15.151:0/4164173455 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0dd4106d00 msgr2=0x7f0dd4107190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:06:05.193 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.192+0000 7f0dd3e00640 1 --2- 172.21.15.151:0/4164173455 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0dd4106d00 0x7f0dd4107190 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc009ca0 tx=0x7f0dcc02a940 comp rx=0 tx=0).stop 2024-05-21T22:06:05.193 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.192+0000 7f0dd3e00640 1 -- 172.21.15.151:0/4164173455 shutdown_connections 2024-05-21T22:06:05.193 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.192+0000 7f0dd3e00640 1 --2- 172.21.15.151:0/4164173455 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0dd4106d00 0x7f0dd4107190 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:05.193 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.192+0000 7f0dd3e00640 1 --2- 172.21.15.151:0/4164173455 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0dd4102220 0x7f0dd41066e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:05.193 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.192+0000 7f0dd3e00640 1 --2- 172.21.15.151:0/4164173455 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0dd4101900 0x7f0dd4101ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:05.193 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.192+0000 7f0dd3e00640 1 -- 172.21.15.151:0/4164173455 >> 172.21.15.151:0/4164173455 conn(0x7f0dd40fd370 msgr2=0x7f0dd40ff790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:06:05.193 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.192+0000 7f0dd3e00640 1 -- 172.21.15.151:0/4164173455 shutdown_connections 2024-05-21T22:06:05.193 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.192+0000 7f0dd3e00640 1 -- 172.21.15.151:0/4164173455 wait complete. 2024-05-21T22:06:05.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.193+0000 7f0dd3e00640 1 Processor -- start 2024-05-21T22:06:05.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.193+0000 7f0dd3e00640 1 -- start start 2024-05-21T22:06:05.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.194+0000 7f0dd3e00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0dd4101900 0x7f0dd419f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:06:05.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.194+0000 7f0dd3e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0dd4102220 0x7f0dd419fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:06:05.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.194+0000 7f0dd3e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0dd4106d00 0x7f0dd41a3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:06:05.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.194+0000 7f0dd3e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd4119200 con 0x7f0dd4102220 2024-05-21T22:06:05.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.194+0000 7f0dd3e00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd4119080 con 0x7f0dd4106d00 2024-05-21T22:06:05.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.194+0000 7f0dd3e00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd4119380 con 0x7f0dd4101900 2024-05-21T22:06:05.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.194+0000 7f0dd2000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0dd4102220 0x7f0dd419fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:06:05.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.194+0000 7f0dd2000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0dd4102220 0x7f0dd419fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:53474/0 (socket says 172.21.15.151:53474) 2024-05-21T22:06:05.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.194+0000 7f0dd2000640 1 -- 172.21.15.151:0/207097148 learned_addr learned my addr 172.21.15.151:0/207097148 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:06:05.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.194+0000 7f0dd2a00640 1 --2- 172.21.15.151:0/207097148 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0dd4101900 0x7f0dd419f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:06:05.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.194+0000 7f0dd3400640 1 --2- 172.21.15.151:0/207097148 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0dd4106d00 0x7f0dd41a3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:06:05.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.194+0000 7f0dd2a00640 1 -- 172.21.15.151:0/207097148 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0dd4106d00 msgr2=0x7f0dd41a3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:06:05.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.194+0000 7f0dd2a00640 1 --2- 172.21.15.151:0/207097148 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0dd4106d00 0x7f0dd41a3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:05.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.194+0000 7f0dd2a00640 1 -- 172.21.15.151:0/207097148 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0dd4102220 msgr2=0x7f0dd419fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:06:05.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.194+0000 7f0dd2a00640 1 --2- 172.21.15.151:0/207097148 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0dd4102220 0x7f0dd419fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:05.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.195+0000 7f0dd2a00640 1 -- 172.21.15.151:0/207097148 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dd41a4710 con 0x7f0dd4101900 2024-05-21T22:06:05.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.195+0000 7f0dd2a00640 1 --2- 172.21.15.151:0/207097148 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0dd4101900 0x7f0dd419f720 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f0dc400e9f0 tx=0x7f0dc400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:06:05.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.195+0000 7f0dcb400640 1 -- 172.21.15.151:0/207097148 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc400ccc0 con 0x7f0dd4101900 2024-05-21T22:06:05.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.195+0000 7f0dd3e00640 1 -- 172.21.15.151:0/207097148 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dd41a4a00 con 0x7f0dd4101900 2024-05-21T22:06:05.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.195+0000 7f0dcb400640 1 -- 172.21.15.151:0/207097148 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0dc40108e0 con 0x7f0dd4101900 2024-05-21T22:06:05.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.195+0000 7f0dcb400640 1 -- 172.21.15.151:0/207097148 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc4018880 con 0x7f0dd4101900 2024-05-21T22:06:05.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.195+0000 7f0dd3e00640 1 -- 172.21.15.151:0/207097148 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dd41ac290 con 0x7f0dd4101900 2024-05-21T22:06:05.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.196+0000 7f0dd3e00640 1 -- 172.21.15.151:0/207097148 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0dd406b950 con 0x7f0dd4101900 2024-05-21T22:06:05.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.199+0000 7f0dcb400640 1 -- 172.21.15.151:0/207097148 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7f0dc4010430 con 0x7f0dd4101900 2024-05-21T22:06:05.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.200+0000 7f0dcb400640 1 --2- 172.21.15.151:0/207097148 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f0db80b2960 0x7f0db80b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:06:05.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.200+0000 7f0dcb400640 1 -- 172.21.15.151:0/207097148 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7f0dc40e2e30 con 0x7f0dd4101900 2024-05-21T22:06:05.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.200+0000 7f0dd2000640 1 --2- 172.21.15.151:0/207097148 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f0db80b2960 0x7f0db80b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:06:05.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.201+0000 7f0dd2000640 1 --2- 172.21.15.151:0/207097148 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f0db80b2960 0x7f0db80b4e20 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0dd4105d40 tx=0x7f0dc0005f90 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:06:05.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.203+0000 7f0dcb400640 1 -- 172.21.15.151:0/207097148 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0dc40aa150 con 0x7f0dd4101900 2024-05-21T22:06:05.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.409+0000 7f0dd3e00640 1 -- 172.21.15.151:0/207097148 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0dd41ac460 con 0x7f0dd4101900 2024-05-21T22:06:05.410 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.410+0000 7f0dcb400640 1 -- 172.21.15.151:0/207097148 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7f0dc40ae450 con 0x7f0dd4101900 2024-05-21T22:06:05.410 INFO:teuthology.orchestra.run.smithi151.stdout:pg_num: 32 2024-05-21T22:06:05.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.413+0000 7f0dd3e00640 1 -- 172.21.15.151:0/207097148 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f0db80b2960 msgr2=0x7f0db80b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:06:05.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.413+0000 7f0dd3e00640 1 --2- 172.21.15.151:0/207097148 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f0db80b2960 0x7f0db80b4e20 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0dd4105d40 tx=0x7f0dc0005f90 comp rx=0 tx=0).stop 2024-05-21T22:06:05.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.413+0000 7f0dd3e00640 1 -- 172.21.15.151:0/207097148 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0dd4101900 msgr2=0x7f0dd419f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:06:05.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.413+0000 7f0dd3e00640 1 --2- 172.21.15.151:0/207097148 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0dd4101900 0x7f0dd419f720 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f0dc400e9f0 tx=0x7f0dc400eeb0 comp rx=0 tx=0).stop 2024-05-21T22:06:05.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.413+0000 7f0dd3e00640 1 -- 172.21.15.151:0/207097148 shutdown_connections 2024-05-21T22:06:05.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.413+0000 7f0dd3e00640 1 --2- 172.21.15.151:0/207097148 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f0db80b2960 0x7f0db80b4e20 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:05.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.413+0000 7f0dd3e00640 1 --2- 172.21.15.151:0/207097148 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0dd4106d00 0x7f0dd41a3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:05.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.413+0000 7f0dd3e00640 1 --2- 172.21.15.151:0/207097148 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0dd4102220 0x7f0dd419fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:05.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.413+0000 7f0dd3e00640 1 --2- 172.21.15.151:0/207097148 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0dd4101900 0x7f0dd419f720 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:05.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.413+0000 7f0dd3e00640 1 -- 172.21.15.151:0/207097148 >> 172.21.15.151:0/207097148 conn(0x7f0dd40fd370 msgr2=0x7f0dd40fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:06:05.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.413+0000 7f0dd3e00640 1 -- 172.21.15.151:0/207097148 shutdown_connections 2024-05-21T22:06:05.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.413+0000 7f0dd3e00640 1 -- 172.21.15.151:0/207097148 wait complete. 2024-05-21T22:06:05.433 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-21T22:06:05.644 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.643+0000 7fa65ea00640 1 -- 172.21.15.151:0/3316556367 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa658102a20 msgr2=0x7fa658102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:06:05.644 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.643+0000 7fa65ea00640 1 --2- 172.21.15.151:0/3316556367 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa658102a20 0x7fa658102ea0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fa644009890 tx=0x7fa64402a360 comp rx=0 tx=0).stop 2024-05-21T22:06:05.644 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.643+0000 7fa65ea00640 1 -- 172.21.15.151:0/3316556367 shutdown_connections 2024-05-21T22:06:05.644 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.643+0000 7fa65ea00640 1 --2- 172.21.15.151:0/3316556367 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa6581033e0 0x7fa65810f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:05.644 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.643+0000 7fa65ea00640 1 --2- 172.21.15.151:0/3316556367 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa658102a20 0x7fa658102ea0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:05.645 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.643+0000 7fa65ea00640 1 --2- 172.21.15.151:0/3316556367 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa658108a30 0x7fa658108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:05.645 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.643+0000 7fa65ea00640 1 -- 172.21.15.151:0/3316556367 >> 172.21.15.151:0/3316556367 conn(0x7fa658079ea0 msgr2=0x7fa65807a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:06:05.645 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.643+0000 7fa65ea00640 1 -- 172.21.15.151:0/3316556367 shutdown_connections 2024-05-21T22:06:05.645 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.643+0000 7fa65ea00640 1 -- 172.21.15.151:0/3316556367 wait complete. 2024-05-21T22:06:05.649 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.644+0000 7fa65ea00640 1 Processor -- start 2024-05-21T22:06:05.649 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.644+0000 7fa65ea00640 1 -- start start 2024-05-21T22:06:05.649 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.645+0000 7fa65ea00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa658102a20 0x7fa658199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:06:05.649 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.645+0000 7fa65ea00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa6581033e0 0x7fa6581997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:06:05.649 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.645+0000 7fa65ea00640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa658108a30 0x7fa65819db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:06:05.649 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.645+0000 7fa65ea00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa658112cf0 con 0x7fa6581033e0 2024-05-21T22:06:05.649 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.645+0000 7fa65ea00640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa658112b70 con 0x7fa658102a20 2024-05-21T22:06:05.649 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.645+0000 7fa65ea00640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa658112e70 con 0x7fa658108a30 2024-05-21T22:06:05.649 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.645+0000 7fa657400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa6581033e0 0x7fa6581997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:06:05.649 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.645+0000 7fa657400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa6581033e0 0x7fa6581997d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:53500/0 (socket says 172.21.15.151:53500) 2024-05-21T22:06:05.650 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.645+0000 7fa657e00640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa658102a20 0x7fa658199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:06:05.650 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.645+0000 7fa657400640 1 -- 172.21.15.151:0/4100811385 learned_addr learned my addr 172.21.15.151:0/4100811385 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:06:05.650 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.645+0000 7fa65ca00640 1 --2- 172.21.15.151:0/4100811385 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa658108a30 0x7fa65819db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:06:05.650 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.646+0000 7fa657400640 1 -- 172.21.15.151:0/4100811385 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa658108a30 msgr2=0x7fa65819db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:06:05.650 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.646+0000 7fa657400640 1 --2- 172.21.15.151:0/4100811385 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa658108a30 0x7fa65819db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:05.650 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.646+0000 7fa657400640 1 -- 172.21.15.151:0/4100811385 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa658102a20 msgr2=0x7fa658199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:06:05.650 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.646+0000 7fa657400640 1 --2- 172.21.15.151:0/4100811385 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa658102a20 0x7fa658199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:05.650 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.646+0000 7fa657400640 1 -- 172.21.15.151:0/4100811385 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa65819e280 con 0x7fa6581033e0 2024-05-21T22:06:05.650 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.646+0000 7fa657e00640 1 --2- 172.21.15.151:0/4100811385 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa658102a20 0x7fa658199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:06:05.650 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.646+0000 7fa657400640 1 --2- 172.21.15.151:0/4100811385 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa6581033e0 0x7fa6581997d0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fa644009860 tx=0x7fa644032d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:06:05.650 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.646+0000 7fa654c00640 1 -- 172.21.15.151:0/4100811385 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa644040020 con 0x7fa6581033e0 2024-05-21T22:06:05.650 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.646+0000 7fa654c00640 1 -- 172.21.15.151:0/4100811385 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa644030d20 con 0x7fa6581033e0 2024-05-21T22:06:05.651 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.646+0000 7fa65ea00640 1 -- 172.21.15.151:0/4100811385 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa65819e510 con 0x7fa6581033e0 2024-05-21T22:06:05.651 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.646+0000 7fa654c00640 1 -- 172.21.15.151:0/4100811385 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa644037040 con 0x7fa6581033e0 2024-05-21T22:06:05.651 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.646+0000 7fa65ea00640 1 -- 172.21.15.151:0/4100811385 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6581a5e00 con 0x7fa6581033e0 2024-05-21T22:06:05.651 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.648+0000 7fa65ea00640 1 -- 172.21.15.151:0/4100811385 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa658069cc0 con 0x7fa6581033e0 2024-05-21T22:06:05.651 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.650+0000 7fa654c00640 1 -- 172.21.15.151:0/4100811385 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7fa644043020 con 0x7fa6581033e0 2024-05-21T22:06:05.651 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.651+0000 7fa654c00640 1 --2- 172.21.15.151:0/4100811385 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fa63c0b2960 0x7fa63c0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:06:05.652 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.652+0000 7fa654c00640 1 -- 172.21.15.151:0/4100811385 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7fa644102200 con 0x7fa6581033e0 2024-05-21T22:06:05.652 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.652+0000 7fa657e00640 1 --2- 172.21.15.151:0/4100811385 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fa63c0b2960 0x7fa63c0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:06:05.652 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.652+0000 7fa657e00640 1 --2- 172.21.15.151:0/4100811385 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fa63c0b2960 0x7fa63c0b4e20 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa6480096c0 tx=0x7fa648009260 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:06:05.656 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.656+0000 7fa654c00640 1 -- 172.21.15.151:0/4100811385 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa64404a020 con 0x7fa6581033e0 2024-05-21T22:06:05.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:06:05 smithi151 ceph-mon[31623]: pgmap v323: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 27 op/s 2024-05-21T22:06:05.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:06:05 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/4293369162' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-21T22:06:05.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:06:05 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/207097148' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-21T22:06:05.865 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.865+0000 7fa65ea00640 1 -- 172.21.15.151:0/4100811385 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa65810afc0 con 0x7fa6581033e0 2024-05-21T22:06:05.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.866+0000 7fa654c00640 1 -- 172.21.15.151:0/4100811385 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7fa6440c9c80 con 0x7fa6581033e0 2024-05-21T22:06:05.866 INFO:teuthology.orchestra.run.smithi151.stdout:pg_num: 64 2024-05-21T22:06:05.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.868+0000 7fa65ea00640 1 -- 172.21.15.151:0/4100811385 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fa63c0b2960 msgr2=0x7fa63c0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:06:05.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.868+0000 7fa65ea00640 1 --2- 172.21.15.151:0/4100811385 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fa63c0b2960 0x7fa63c0b4e20 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa6480096c0 tx=0x7fa648009260 comp rx=0 tx=0).stop 2024-05-21T22:06:05.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.868+0000 7fa65ea00640 1 -- 172.21.15.151:0/4100811385 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa6581033e0 msgr2=0x7fa6581997d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:06:05.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.868+0000 7fa65ea00640 1 --2- 172.21.15.151:0/4100811385 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa6581033e0 0x7fa6581997d0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fa644009860 tx=0x7fa644032d20 comp rx=0 tx=0).stop 2024-05-21T22:06:05.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.868+0000 7fa65ea00640 1 -- 172.21.15.151:0/4100811385 shutdown_connections 2024-05-21T22:06:05.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.868+0000 7fa65ea00640 1 --2- 172.21.15.151:0/4100811385 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fa63c0b2960 0x7fa63c0b4e20 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:05.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.868+0000 7fa65ea00640 1 --2- 172.21.15.151:0/4100811385 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa658108a30 0x7fa65819db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:05.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.868+0000 7fa65ea00640 1 --2- 172.21.15.151:0/4100811385 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa6581033e0 0x7fa6581997d0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:05.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.868+0000 7fa65ea00640 1 --2- 172.21.15.151:0/4100811385 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa658102a20 0x7fa658199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:05.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.868+0000 7fa65ea00640 1 -- 172.21.15.151:0/4100811385 >> 172.21.15.151:0/4100811385 conn(0x7fa658079ea0 msgr2=0x7fa6581005e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:06:05.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.869+0000 7fa65ea00640 1 -- 172.21.15.151:0/4100811385 shutdown_connections 2024-05-21T22:06:05.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:05.869+0000 7fa65ea00640 1 -- 172.21.15.151:0/4100811385 wait complete. 2024-05-21T22:06:05.889 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-05-21T22:06:05.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:06:05 smithi184 ceph-mon[36942]: pgmap v323: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 27 op/s 2024-05-21T22:06:05.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:06:05 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/4293369162' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-21T22:06:05.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:06:05 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/207097148' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-21T22:06:06.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:06:05 smithi164 ceph-mon[36269]: pgmap v323: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 27 op/s 2024-05-21T22:06:06.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:06:05 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/4293369162' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-21T22:06:06.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:06:05 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/207097148' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-21T22:06:06.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:06.091+0000 7fa817000640 1 -- 172.21.15.151:0/952796891 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa810075710 msgr2=0x7fa8101134f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:06:06.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:06.091+0000 7fa817000640 1 --2- 172.21.15.151:0/952796891 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa810075710 0x7fa8101134f0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc009900 tx=0x7fa7fc02a480 comp rx=0 tx=0).stop 2024-05-21T22:06:06.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:06.092+0000 7fa817000640 1 -- 172.21.15.151:0/952796891 shutdown_connections 2024-05-21T22:06:06.093 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:06.092+0000 7fa817000640 1 --2- 172.21.15.151:0/952796891 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa810113a30 0x7fa810115e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:06.093 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:06.092+0000 7fa817000640 1 --2- 172.21.15.151:0/952796891 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa810075710 0x7fa8101134f0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:06.093 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:06.092+0000 7fa817000640 1 --2- 172.21.15.151:0/952796891 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa810074df0 0x7fa8100751d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:06.093 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:06.092+0000 7fa817000640 1 -- 172.21.15.151:0/952796891 >> 172.21.15.151:0/952796891 conn(0x7fa810079e10 msgr2=0x7fa81007a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:06:06.093 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:06.092+0000 7fa817000640 1 -- 172.21.15.151:0/952796891 shutdown_connections 2024-05-21T22:06:06.093 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:06.092+0000 7fa817000640 1 -- 172.21.15.151:0/952796891 wait complete. 2024-05-21T22:06:06.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:06.093+0000 7fa817000640 1 Processor -- start 2024-05-21T22:06:06.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:06.093+0000 7fa817000640 1 -- start start 2024-05-21T22:06:06.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:06.093+0000 7fa817000640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa810074df0 0x7fa8101a55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:06:06.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:06.093+0000 7fa817000640 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa810075710 0x7fa8101a5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:06:06.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:06.093+0000 7fa817000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa810113a30 0x7fa81019f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:06:06.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:06.093+0000 7fa817000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa8101192b0 con 0x7fa810113a30 2024-05-21T22:06:06.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:06.093+0000 7fa817000640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa810119130 con 0x7fa810074df0 2024-05-21T22:06:06.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:06.093+0000 7fa817000640 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fa810119430 con 0x7fa810075710 2024-05-21T22:06:06.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:06.094+0000 7fa815000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa810113a30 0x7fa81019f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:06:06.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:06.094+0000 7fa815000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa810113a30 0x7fa81019f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:40482/0 (socket says 172.21.15.151:40482) 2024-05-21T22:06:06.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:06.094+0000 7fa815000640 1 -- 172.21.15.151:0/2217178833 learned_addr learned my addr 172.21.15.151:0/2217178833 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:06:06.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:06.094+0000 7fa80fe00640 1 --2- 172.21.15.151:0/2217178833 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa810074df0 0x7fa8101a55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:06:06.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:06.094+0000 7fa80f400640 1 --2- 172.21.15.151:0/2217178833 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa810075710 0x7fa8101a5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:06:06.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:06.094+0000 7fa815000640 1 -- 172.21.15.151:0/2217178833 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa810075710 msgr2=0x7fa8101a5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:06:06.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:06.094+0000 7fa815000640 1 --2- 172.21.15.151:0/2217178833 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa810075710 0x7fa8101a5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:06.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:06.094+0000 7fa815000640 1 -- 172.21.15.151:0/2217178833 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa810074df0 msgr2=0x7fa8101a55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:06:06.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:06.094+0000 7fa815000640 1 --2- 172.21.15.151:0/2217178833 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa810074df0 0x7fa8101a55b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:06.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:06.094+0000 7fa815000640 1 -- 172.21.15.151:0/2217178833 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa81019ff30 con 0x7fa810113a30 2024-05-21T22:06:06.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:06.094+0000 7fa80fe00640 1 --2- 172.21.15.151:0/2217178833 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa810074df0 0x7fa8101a55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:06:06.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:06.094+0000 7fa815000640 1 --2- 172.21.15.151:0/2217178833 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa810113a30 0x7fa81019f670 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fa80800eb40 tx=0x7fa80800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:06:06.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:06.095+0000 7fa80cc00640 1 -- 172.21.15.151:0/2217178833 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa808004b20 con 0x7fa810113a30 2024-05-21T22:06:06.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:06.095+0000 7fa80cc00640 1 -- 172.21.15.151:0/2217178833 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa808004cc0 con 0x7fa810113a30 2024-05-21T22:06:06.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:06.095+0000 7fa817000640 1 -- 172.21.15.151:0/2217178833 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8101a0220 con 0x7fa810113a30 2024-05-21T22:06:06.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:06.095+0000 7fa80cc00640 1 -- 172.21.15.151:0/2217178833 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa808010720 con 0x7fa810113a30 2024-05-21T22:06:06.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:06.095+0000 7fa817000640 1 -- 172.21.15.151:0/2217178833 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8101ac530 con 0x7fa810113a30 2024-05-21T22:06:06.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:06.096+0000 7fa817000640 1 -- 172.21.15.151:0/2217178833 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8101a0560 con 0x7fa810113a30 2024-05-21T22:06:06.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:06.098+0000 7fa80cc00640 1 -- 172.21.15.151:0/2217178833 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7fa808005560 con 0x7fa810113a30 2024-05-21T22:06:06.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:06.100+0000 7fa80cc00640 1 --2- 172.21.15.151:0/2217178833 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fa7f40b29e0 0x7fa7f40b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:06:06.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:06.100+0000 7fa80cc00640 1 -- 172.21.15.151:0/2217178833 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7fa8080e2ce0 con 0x7fa810113a30 2024-05-21T22:06:06.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:06.100+0000 7fa80fe00640 1 --2- 172.21.15.151:0/2217178833 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fa7f40b29e0 0x7fa7f40b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:06:06.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:06.101+0000 7fa80fe00640 1 --2- 172.21.15.151:0/2217178833 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fa7f40b29e0 0x7fa7f40b4ea0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa8000095f0 tx=0x7fa800005f90 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:06:06.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:06.104+0000 7fa80cc00640 1 -- 172.21.15.151:0/2217178833 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa8080aa650 con 0x7fa810113a30 2024-05-21T22:06:06.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:06.370+0000 7fa817000640 1 -- 172.21.15.151:0/2217178833 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7fa810069cc0 con 0x7fa810113a30 2024-05-21T22:06:06.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:06.404+0000 7fa80cc00640 1 -- 172.21.15.151:0/2217178833 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v27) ==== 81+0+816181 (secure 0 0 0) 0x7fa8080a9f50 con 0x7fa810113a30 2024-05-21T22:06:06.419 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T22:06:06.419 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-05-21T22:06:06.419 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 27, 2024-05-21T22:06:06.419 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.419 INFO:teuthology.orchestra.run.smithi151.stdout: "active_gid": 25138, 2024-05-21T22:06:06.419 INFO:teuthology.orchestra.run.smithi151.stdout: "active_name": "x", 2024-05-21T22:06:06.419 INFO:teuthology.orchestra.run.smithi151.stdout: "active_addrs": { 2024-05-21T22:06:06.419 INFO:teuthology.orchestra.run.smithi151.stdout: "addrvec": [ 2024-05-21T22:06:06.419 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:06.419 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "v2", 2024-05-21T22:06:06.420 INFO:teuthology.orchestra.run.smithi151.stdout: "addr": "172.21.15.151:6800", 2024-05-21T22:06:06.420 INFO:teuthology.orchestra.run.smithi151.stdout: "nonce": 958417056 2024-05-21T22:06:06.420 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.420 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:06.420 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "v1", 2024-05-21T22:06:06.420 INFO:teuthology.orchestra.run.smithi151.stdout: "addr": "172.21.15.151:6801", 2024-05-21T22:06:06.420 INFO:teuthology.orchestra.run.smithi151.stdout: "nonce": 958417056 2024-05-21T22:06:06.420 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:06.420 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-05-21T22:06:06.420 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.420 INFO:teuthology.orchestra.run.smithi151.stdout: "active_addr": "172.21.15.151:6801/958417056", 2024-05-21T22:06:06.420 INFO:teuthology.orchestra.run.smithi151.stdout: "active_change": "2024-05-21T21:55:36.189855+0000", 2024-05-21T22:06:06.420 INFO:teuthology.orchestra.run.smithi151.stdout: "active_mgr_features": 4540701547738038271, 2024-05-21T22:06:06.420 INFO:teuthology.orchestra.run.smithi151.stdout: "available": true, 2024-05-21T22:06:06.421 INFO:teuthology.orchestra.run.smithi151.stdout: "standbys": [ 2024-05-21T22:06:06.421 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:06.421 INFO:teuthology.orchestra.run.smithi151.stdout: "gid": 25129, 2024-05-21T22:06:06.421 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "z", 2024-05-21T22:06:06.421 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr_features": 4540701547738038271, 2024-05-21T22:06:06.421 INFO:teuthology.orchestra.run.smithi151.stdout: "available_modules": [ 2024-05-21T22:06:06.421 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:06.421 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "alerts", 2024-05-21T22:06:06.421 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:06.421 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:06.421 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:06.421 INFO:teuthology.orchestra.run.smithi151.stdout: "interval": { 2024-05-21T22:06:06.421 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "interval", 2024-05-21T22:06:06.422 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-05-21T22:06:06.422 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.422 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.422 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "60", 2024-05-21T22:06:06.422 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.422 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.422 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.422 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "How frequently to reexamine health status", 2024-05-21T22:06:06.422 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.422 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.422 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.422 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.423 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:06.423 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:06.423 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.423 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.423 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.423 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:06.423 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.423 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.423 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:06.423 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:06.423 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:06.423 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:06.423 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:06.423 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:06.424 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:06.424 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:06.424 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.424 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.424 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.424 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.424 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.424 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:06.424 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:06.424 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.425 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.425 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.425 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.425 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.425 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.425 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.425 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.425 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.425 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.425 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.425 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.425 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:06.425 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:06.425 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.425 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.426 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.426 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:06.426 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.426 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.426 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:06.426 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:06.426 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:06.426 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:06.426 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:06.426 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:06.426 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:06.426 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:06.426 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.426 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.427 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.427 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.427 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.427 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:06.427 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:06.427 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.427 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.428 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.428 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.428 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.428 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.428 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.428 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.429 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.429 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.429 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.429 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.429 INFO:teuthology.orchestra.run.smithi151.stdout: "smtp_destination": { 2024-05-21T22:06:06.429 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "smtp_destination", 2024-05-21T22:06:06.429 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.430 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.430 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.430 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:06.430 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.430 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.431 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.431 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Email address to send alerts to", 2024-05-21T22:06:06.431 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.431 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.431 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.431 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.431 INFO:teuthology.orchestra.run.smithi151.stdout: "smtp_from_name": { 2024-05-21T22:06:06.432 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "smtp_from_name", 2024-05-21T22:06:06.432 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.432 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.432 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.432 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "Ceph", 2024-05-21T22:06:06.432 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.432 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.433 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.433 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Email From: name", 2024-05-21T22:06:06.433 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.433 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.433 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.433 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.433 INFO:teuthology.orchestra.run.smithi151.stdout: "smtp_host": { 2024-05-21T22:06:06.434 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "smtp_host", 2024-05-21T22:06:06.434 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.434 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.434 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.434 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:06.434 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.435 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.435 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.435 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "SMTP server", 2024-05-21T22:06:06.435 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.435 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.435 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.436 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.436 INFO:teuthology.orchestra.run.smithi151.stdout: "smtp_password": { 2024-05-21T22:06:06.436 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "smtp_password", 2024-05-21T22:06:06.436 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.436 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.436 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.436 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:06.436 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.437 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.437 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.437 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Password to authenticate with", 2024-05-21T22:06:06.437 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.437 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.437 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.437 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.437 INFO:teuthology.orchestra.run.smithi151.stdout: "smtp_port": { 2024-05-21T22:06:06.438 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "smtp_port", 2024-05-21T22:06:06.438 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:06.438 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.438 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.438 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "465", 2024-05-21T22:06:06.438 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.439 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.439 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.439 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "SMTP port", 2024-05-21T22:06:06.439 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.439 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.439 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.440 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.440 INFO:teuthology.orchestra.run.smithi151.stdout: "smtp_sender": { 2024-05-21T22:06:06.440 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "smtp_sender", 2024-05-21T22:06:06.440 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.440 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.441 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.441 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:06.441 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.441 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.441 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.441 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "SMTP envelope sender", 2024-05-21T22:06:06.442 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.442 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.442 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.442 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.442 INFO:teuthology.orchestra.run.smithi151.stdout: "smtp_ssl": { 2024-05-21T22:06:06.443 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "smtp_ssl", 2024-05-21T22:06:06.443 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.443 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.443 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.443 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:06.443 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.443 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.443 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.444 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-21T22:06:06.444 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.444 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.444 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.444 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.444 INFO:teuthology.orchestra.run.smithi151.stdout: "smtp_user": { 2024-05-21T22:06:06.444 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "smtp_user", 2024-05-21T22:06:06.444 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.445 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.445 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.445 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:06.445 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.445 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.445 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.445 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "User to authenticate as", 2024-05-21T22:06:06.446 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.446 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.446 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.446 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.446 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:06.446 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:06.447 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:06.447 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:06.447 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.447 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:06.447 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.447 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.447 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.448 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.448 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.448 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.448 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.448 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:06.448 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:06.449 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.449 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:06.449 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "balancer", 2024-05-21T22:06:06.449 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:06.449 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:06.449 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:06.449 INFO:teuthology.orchestra.run.smithi151.stdout: "active": { 2024-05-21T22:06:06.449 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "active", 2024-05-21T22:06:06.449 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.450 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.450 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.450 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:06.450 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.450 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.450 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.450 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "automatically balance PGs across cluster", 2024-05-21T22:06:06.450 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.450 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.450 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.450 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.450 INFO:teuthology.orchestra.run.smithi151.stdout: "begin_time": { 2024-05-21T22:06:06.450 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "begin_time", 2024-05-21T22:06:06.451 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.451 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.451 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.451 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0000", 2024-05-21T22:06:06.451 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.451 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.451 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.451 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "beginning time of day to automatically balance", 2024-05-21T22:06:06.451 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-21T22:06:06.451 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.451 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.451 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.451 INFO:teuthology.orchestra.run.smithi151.stdout: "begin_weekday": { 2024-05-21T22:06:06.451 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "begin_weekday", 2024-05-21T22:06:06.452 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "uint", 2024-05-21T22:06:06.452 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.452 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.452 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:06.452 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "0", 2024-05-21T22:06:06.452 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "6", 2024-05-21T22:06:06.452 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.452 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-21T22:06:06.452 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-21T22:06:06.452 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.452 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.452 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.452 INFO:teuthology.orchestra.run.smithi151.stdout: "crush_compat_max_iterations": { 2024-05-21T22:06:06.452 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "crush_compat_max_iterations", 2024-05-21T22:06:06.453 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "uint", 2024-05-21T22:06:06.453 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.453 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.453 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "25", 2024-05-21T22:06:06.453 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "1", 2024-05-21T22:06:06.453 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "250", 2024-05-21T22:06:06.453 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.453 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-21T22:06:06.453 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.453 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.453 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.454 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.454 INFO:teuthology.orchestra.run.smithi151.stdout: "crush_compat_metrics": { 2024-05-21T22:06:06.454 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "crush_compat_metrics", 2024-05-21T22:06:06.454 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.454 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.454 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.454 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "pgs,objects,bytes", 2024-05-21T22:06:06.454 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.454 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.454 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.454 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-21T22:06:06.455 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-21T22:06:06.455 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.455 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.455 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.455 INFO:teuthology.orchestra.run.smithi151.stdout: "crush_compat_step": { 2024-05-21T22:06:06.455 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "crush_compat_step", 2024-05-21T22:06:06.455 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "float", 2024-05-21T22:06:06.455 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.455 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.455 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0.5", 2024-05-21T22:06:06.455 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "0.001", 2024-05-21T22:06:06.455 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "0.999", 2024-05-21T22:06:06.456 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.456 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "aggressiveness of optimization", 2024-05-21T22:06:06.456 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-21T22:06:06.456 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.456 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.456 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.456 INFO:teuthology.orchestra.run.smithi151.stdout: "end_time": { 2024-05-21T22:06:06.456 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "end_time", 2024-05-21T22:06:06.456 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.456 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.456 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.456 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "2359", 2024-05-21T22:06:06.457 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.457 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.457 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.457 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "ending time of day to automatically balance", 2024-05-21T22:06:06.457 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-21T22:06:06.457 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.457 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.457 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.457 INFO:teuthology.orchestra.run.smithi151.stdout: "end_weekday": { 2024-05-21T22:06:06.457 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "end_weekday", 2024-05-21T22:06:06.457 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "uint", 2024-05-21T22:06:06.457 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.458 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.458 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:06.458 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "0", 2024-05-21T22:06:06.458 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "6", 2024-05-21T22:06:06.458 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.458 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-21T22:06:06.458 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-21T22:06:06.458 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.458 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.458 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.458 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:06.458 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:06.458 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.459 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.459 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.459 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:06.459 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.459 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.459 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:06.459 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:06.459 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:06.459 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:06.460 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:06.460 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:06.460 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:06.460 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:06.460 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.460 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.461 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.461 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.461 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.461 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:06.461 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:06.461 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.462 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.462 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.462 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.462 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.462 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.462 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.462 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.462 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.462 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.462 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.463 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.463 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:06.463 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:06.463 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.463 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.463 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.463 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:06.463 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.463 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.463 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:06.463 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:06.463 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:06.463 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:06.464 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:06.464 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:06.464 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:06.464 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:06.464 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.464 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.464 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.464 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.464 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.464 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:06.464 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:06.465 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.465 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.465 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.465 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.465 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.465 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.465 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.465 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.465 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.465 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.465 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.465 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.465 INFO:teuthology.orchestra.run.smithi151.stdout: "min_score": { 2024-05-21T22:06:06.465 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "min_score", 2024-05-21T22:06:06.466 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "float", 2024-05-21T22:06:06.466 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.466 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.466 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:06.466 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.466 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.466 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.466 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-21T22:06:06.466 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.466 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.466 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.466 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.467 INFO:teuthology.orchestra.run.smithi151.stdout: "mode": { 2024-05-21T22:06:06.467 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "mode", 2024-05-21T22:06:06.467 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.467 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.467 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.467 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "upmap", 2024-05-21T22:06:06.467 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.467 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.467 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:06.467 INFO:teuthology.orchestra.run.smithi151.stdout: "crush-compat", 2024-05-21T22:06:06.467 INFO:teuthology.orchestra.run.smithi151.stdout: "none", 2024-05-21T22:06:06.467 INFO:teuthology.orchestra.run.smithi151.stdout: "read", 2024-05-21T22:06:06.467 INFO:teuthology.orchestra.run.smithi151.stdout: "upmap", 2024-05-21T22:06:06.467 INFO:teuthology.orchestra.run.smithi151.stdout: "upmap-read" 2024-05-21T22:06:06.468 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:06.468 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Balancer mode", 2024-05-21T22:06:06.468 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.468 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.468 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.468 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.468 INFO:teuthology.orchestra.run.smithi151.stdout: "pool_ids": { 2024-05-21T22:06:06.468 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "pool_ids", 2024-05-21T22:06:06.468 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.468 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.468 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.468 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:06.468 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.468 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.469 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.469 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-21T22:06:06.469 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.469 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.469 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.469 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.469 INFO:teuthology.orchestra.run.smithi151.stdout: "sleep_interval": { 2024-05-21T22:06:06.469 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sleep_interval", 2024-05-21T22:06:06.469 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-05-21T22:06:06.469 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.469 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.469 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "60", 2024-05-21T22:06:06.469 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.469 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.469 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.469 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-21T22:06:06.470 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.470 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.470 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.470 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.470 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:06.470 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:06.470 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:06.470 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:06.471 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.471 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:06.471 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.471 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.471 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.471 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.472 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.472 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.472 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.472 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.472 INFO:teuthology.orchestra.run.smithi151.stdout: "upmap_max_deviation": { 2024-05-21T22:06:06.472 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "upmap_max_deviation", 2024-05-21T22:06:06.473 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:06.473 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.473 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.473 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "5", 2024-05-21T22:06:06.473 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "1", 2024-05-21T22:06:06.473 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.473 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.473 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-21T22:06:06.473 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-21T22:06:06.473 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.473 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.473 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.474 INFO:teuthology.orchestra.run.smithi151.stdout: "upmap_max_optimizations": { 2024-05-21T22:06:06.474 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "upmap_max_optimizations", 2024-05-21T22:06:06.474 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "uint", 2024-05-21T22:06:06.474 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.474 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.474 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "10", 2024-05-21T22:06:06.474 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.474 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.474 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.474 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-21T22:06:06.474 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.474 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.475 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.475 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:06.475 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:06.475 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.475 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:06.475 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "cephadm", 2024-05-21T22:06:06.475 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:06.475 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:06.475 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:06.475 INFO:teuthology.orchestra.run.smithi151.stdout: "agent_down_multiplier": { 2024-05-21T22:06:06.475 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "agent_down_multiplier", 2024-05-21T22:06:06.475 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "float", 2024-05-21T22:06:06.475 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.476 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.476 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "3.0", 2024-05-21T22:06:06.476 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.476 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.476 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.476 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-21T22:06:06.476 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.476 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.477 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.477 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.477 INFO:teuthology.orchestra.run.smithi151.stdout: "agent_refresh_rate": { 2024-05-21T22:06:06.477 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "agent_refresh_rate", 2024-05-21T22:06:06.477 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-05-21T22:06:06.477 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.478 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.478 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "20", 2024-05-21T22:06:06.478 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.478 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.478 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.479 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-21T22:06:06.479 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.479 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.479 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.479 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.479 INFO:teuthology.orchestra.run.smithi151.stdout: "agent_starting_port": { 2024-05-21T22:06:06.480 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "agent_starting_port", 2024-05-21T22:06:06.480 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:06.480 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.480 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.480 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "4721", 2024-05-21T22:06:06.480 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.481 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.481 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.481 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-21T22:06:06.481 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.481 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.481 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.482 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.482 INFO:teuthology.orchestra.run.smithi151.stdout: "allow_ptrace": { 2024-05-21T22:06:06.482 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "allow_ptrace", 2024-05-21T22:06:06.482 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.482 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.483 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.483 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.483 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.483 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.483 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.483 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-21T22:06:06.484 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-21T22:06:06.484 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.484 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.484 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.484 INFO:teuthology.orchestra.run.smithi151.stdout: "autotune_interval": { 2024-05-21T22:06:06.484 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "autotune_interval", 2024-05-21T22:06:06.484 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-05-21T22:06:06.485 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.485 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.485 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "600", 2024-05-21T22:06:06.485 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.485 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.485 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.485 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-21T22:06:06.485 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.485 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.485 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.485 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.485 INFO:teuthology.orchestra.run.smithi151.stdout: "autotune_memory_target_ratio": { 2024-05-21T22:06:06.486 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "autotune_memory_target_ratio", 2024-05-21T22:06:06.486 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "float", 2024-05-21T22:06:06.486 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.486 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.486 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0.7", 2024-05-21T22:06:06.486 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.486 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.486 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.486 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-21T22:06:06.486 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.486 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.486 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.487 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.487 INFO:teuthology.orchestra.run.smithi151.stdout: "cephadm_log_destination": { 2024-05-21T22:06:06.487 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "cephadm_log_destination", 2024-05-21T22:06:06.487 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.487 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.488 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.488 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:06.488 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.488 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.488 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:06.488 INFO:teuthology.orchestra.run.smithi151.stdout: "file", 2024-05-21T22:06:06.489 INFO:teuthology.orchestra.run.smithi151.stdout: "file,syslog", 2024-05-21T22:06:06.489 INFO:teuthology.orchestra.run.smithi151.stdout: "syslog" 2024-05-21T22:06:06.489 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:06.489 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-21T22:06:06.489 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.489 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.489 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.489 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.489 INFO:teuthology.orchestra.run.smithi151.stdout: "cgroups_split": { 2024-05-21T22:06:06.490 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "cgroups_split", 2024-05-21T22:06:06.490 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.490 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.490 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.490 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:06.490 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.491 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.491 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.491 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-21T22:06:06.491 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.491 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.491 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.492 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.492 INFO:teuthology.orchestra.run.smithi151.stdout: "config_checks_enabled": { 2024-05-21T22:06:06.492 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "config_checks_enabled", 2024-05-21T22:06:06.492 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.492 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.492 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.492 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.492 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.492 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.492 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.492 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-21T22:06:06.493 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.493 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.493 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.493 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.493 INFO:teuthology.orchestra.run.smithi151.stdout: "config_dashboard": { 2024-05-21T22:06:06.493 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "config_dashboard", 2024-05-21T22:06:06.493 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.493 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.494 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.494 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:06.494 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.494 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.494 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.494 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-21T22:06:06.494 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.494 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.494 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.495 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.495 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_alertmanager": { 2024-05-21T22:06:06.495 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_alertmanager", 2024-05-21T22:06:06.495 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.495 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.495 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.495 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-21T22:06:06.495 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.495 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.495 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.495 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Prometheus container image", 2024-05-21T22:06:06.495 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.495 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.495 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.496 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.496 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_base": { 2024-05-21T22:06:06.496 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_base", 2024-05-21T22:06:06.496 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.496 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.496 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.496 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-21T22:06:06.496 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.496 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.496 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.496 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Container image name, without the tag", 2024-05-21T22:06:06.496 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.496 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.497 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.497 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.497 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_elasticsearch": { 2024-05-21T22:06:06.497 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_elasticsearch", 2024-05-21T22:06:06.497 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.497 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.497 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.497 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-21T22:06:06.497 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.497 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.497 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.498 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "elasticsearch container image", 2024-05-21T22:06:06.498 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.498 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.498 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.498 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.498 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_grafana": { 2024-05-21T22:06:06.498 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_grafana", 2024-05-21T22:06:06.498 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.498 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.498 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.498 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-21T22:06:06.498 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.499 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.499 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.499 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Prometheus container image", 2024-05-21T22:06:06.499 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.499 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.499 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.499 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.499 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_haproxy": { 2024-05-21T22:06:06.499 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_haproxy", 2024-05-21T22:06:06.499 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.499 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.499 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.499 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-21T22:06:06.500 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.500 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.500 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.500 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "HAproxy container image", 2024-05-21T22:06:06.500 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.500 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.500 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.500 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.500 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_jaeger_agent": { 2024-05-21T22:06:06.500 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_jaeger_agent", 2024-05-21T22:06:06.500 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.500 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.500 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.500 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-21T22:06:06.501 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.501 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.501 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.501 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Jaeger agent container image", 2024-05-21T22:06:06.501 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.501 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.501 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.501 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.501 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_jaeger_collector": { 2024-05-21T22:06:06.501 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_jaeger_collector", 2024-05-21T22:06:06.501 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.502 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.502 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.502 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-21T22:06:06.502 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.502 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.502 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.502 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Jaeger collector container image", 2024-05-21T22:06:06.502 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.502 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.503 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.503 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.503 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_jaeger_query": { 2024-05-21T22:06:06.503 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_jaeger_query", 2024-05-21T22:06:06.503 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.503 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.503 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.503 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-21T22:06:06.503 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.503 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.503 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.503 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Jaeger query container image", 2024-05-21T22:06:06.503 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.504 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.504 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.504 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.504 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_keepalived": { 2024-05-21T22:06:06.504 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_keepalived", 2024-05-21T22:06:06.504 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.505 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.505 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.505 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-21T22:06:06.505 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.505 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.505 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.505 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Keepalived container image", 2024-05-21T22:06:06.505 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.505 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.505 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.505 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.506 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_loki": { 2024-05-21T22:06:06.506 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_loki", 2024-05-21T22:06:06.506 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.506 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.506 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.506 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-05-21T22:06:06.506 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.506 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.506 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.506 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Loki container image", 2024-05-21T22:06:06.506 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.506 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.506 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.507 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.507 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_node_exporter": { 2024-05-21T22:06:06.507 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_node_exporter", 2024-05-21T22:06:06.507 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.507 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.507 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.508 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-21T22:06:06.508 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.508 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.508 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.508 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Prometheus container image", 2024-05-21T22:06:06.508 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.508 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.508 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.508 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.508 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_nvmeof": { 2024-05-21T22:06:06.508 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_nvmeof", 2024-05-21T22:06:06.509 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.509 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.509 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.509 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-05-21T22:06:06.509 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.509 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.509 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.509 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Nvme-of container image", 2024-05-21T22:06:06.509 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.509 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.509 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.509 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.510 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_prometheus": { 2024-05-21T22:06:06.510 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_prometheus", 2024-05-21T22:06:06.510 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.510 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.510 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.510 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-21T22:06:06.510 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.510 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.510 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.510 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Prometheus container image", 2024-05-21T22:06:06.510 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.510 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.511 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.511 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.511 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_promtail": { 2024-05-21T22:06:06.511 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_promtail", 2024-05-21T22:06:06.511 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.511 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.511 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.511 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-05-21T22:06:06.511 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.511 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.511 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.511 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Promtail container image", 2024-05-21T22:06:06.511 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.512 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.512 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.512 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.512 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_samba": { 2024-05-21T22:06:06.512 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_samba", 2024-05-21T22:06:06.512 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.513 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.513 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.513 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-21T22:06:06.513 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.513 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.513 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.514 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Samba/SMB container image", 2024-05-21T22:06:06.514 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.514 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.514 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.514 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.514 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_snmp_gateway": { 2024-05-21T22:06:06.514 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_snmp_gateway", 2024-05-21T22:06:06.515 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.515 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.515 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.515 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-21T22:06:06.515 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.515 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.515 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.515 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "SNMP Gateway container image", 2024-05-21T22:06:06.515 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.515 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.515 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.516 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.516 INFO:teuthology.orchestra.run.smithi151.stdout: "container_init": { 2024-05-21T22:06:06.516 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_init", 2024-05-21T22:06:06.516 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.516 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.516 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.516 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:06.516 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.516 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.516 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.516 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Run podman/docker with `--init`", 2024-05-21T22:06:06.517 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.517 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.517 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.517 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.517 INFO:teuthology.orchestra.run.smithi151.stdout: "daemon_cache_timeout": { 2024-05-21T22:06:06.517 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "daemon_cache_timeout", 2024-05-21T22:06:06.518 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-05-21T22:06:06.518 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.518 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.518 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "600", 2024-05-21T22:06:06.518 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.518 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.519 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.519 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-21T22:06:06.519 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.519 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.519 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.519 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.519 INFO:teuthology.orchestra.run.smithi151.stdout: "default_cephadm_command_timeout": { 2024-05-21T22:06:06.519 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "default_cephadm_command_timeout", 2024-05-21T22:06:06.519 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:06.519 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.519 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.519 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "900", 2024-05-21T22:06:06.520 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.520 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.520 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.520 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-21T22:06:06.520 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.520 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.520 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.520 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.520 INFO:teuthology.orchestra.run.smithi151.stdout: "default_registry": { 2024-05-21T22:06:06.520 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "default_registry", 2024-05-21T22:06:06.520 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.520 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.520 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.521 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "docker.io", 2024-05-21T22:06:06.521 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.521 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.521 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.521 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-21T22:06:06.521 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.521 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.521 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.521 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.521 INFO:teuthology.orchestra.run.smithi151.stdout: "device_cache_timeout": { 2024-05-21T22:06:06.521 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "device_cache_timeout", 2024-05-21T22:06:06.521 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-05-21T22:06:06.521 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.522 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.522 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "1800", 2024-05-21T22:06:06.522 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.522 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.522 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.522 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "seconds to cache device inventory", 2024-05-21T22:06:06.522 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.522 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.522 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.522 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.522 INFO:teuthology.orchestra.run.smithi151.stdout: "device_enhanced_scan": { 2024-05-21T22:06:06.522 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "device_enhanced_scan", 2024-05-21T22:06:06.523 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.523 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.523 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.523 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.523 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.523 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.523 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.523 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-21T22:06:06.523 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.523 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.523 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.523 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.523 INFO:teuthology.orchestra.run.smithi151.stdout: "facts_cache_timeout": { 2024-05-21T22:06:06.523 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "facts_cache_timeout", 2024-05-21T22:06:06.524 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-05-21T22:06:06.524 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.524 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.524 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "60", 2024-05-21T22:06:06.524 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.525 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.525 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.525 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "seconds to cache host facts data", 2024-05-21T22:06:06.525 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.525 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.525 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.526 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.526 INFO:teuthology.orchestra.run.smithi151.stdout: "grafana_dashboards_path": { 2024-05-21T22:06:06.526 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "grafana_dashboards_path", 2024-05-21T22:06:06.526 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.526 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.526 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.527 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-21T22:06:06.527 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.527 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.527 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.527 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-21T22:06:06.527 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.527 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.527 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.527 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.527 INFO:teuthology.orchestra.run.smithi151.stdout: "host_check_interval": { 2024-05-21T22:06:06.527 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "host_check_interval", 2024-05-21T22:06:06.527 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-05-21T22:06:06.527 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.528 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.528 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "600", 2024-05-21T22:06:06.528 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.528 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.528 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.528 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "how frequently to perform a host check", 2024-05-21T22:06:06.528 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.528 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.529 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.529 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.529 INFO:teuthology.orchestra.run.smithi151.stdout: "hw_monitoring": { 2024-05-21T22:06:06.529 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "hw_monitoring", 2024-05-21T22:06:06.529 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.529 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.529 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.529 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.529 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.529 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.529 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.529 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-21T22:06:06.529 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.529 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.530 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.530 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.530 INFO:teuthology.orchestra.run.smithi151.stdout: "inventory_list_all": { 2024-05-21T22:06:06.530 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "inventory_list_all", 2024-05-21T22:06:06.530 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.530 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.530 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.530 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.530 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.530 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.530 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.530 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-21T22:06:06.530 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.531 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.531 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.531 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.531 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:06.531 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:06.531 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.531 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.531 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.531 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:06.531 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.531 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.532 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:06.532 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:06.532 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:06.532 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:06.532 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:06.532 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:06.532 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:06.532 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:06.532 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.532 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.532 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.532 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.532 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.532 INFO:teuthology.orchestra.run.smithi151.stdout: "log_refresh_metadata": { 2024-05-21T22:06:06.533 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_refresh_metadata", 2024-05-21T22:06:06.533 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.533 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.533 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.533 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.533 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.533 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.533 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.533 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-21T22:06:06.533 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.533 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.533 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.533 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.534 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:06.534 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:06.534 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.534 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.534 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.534 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:06.534 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.534 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.534 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.534 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-21T22:06:06.534 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.534 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.534 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.535 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.535 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:06.535 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:06.535 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.535 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.535 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.535 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:06.535 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.535 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.535 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:06.535 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:06.535 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:06.535 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:06.536 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:06.536 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:06.536 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:06.536 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:06.536 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.536 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.536 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.536 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.536 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.536 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:06.536 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:06.536 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.536 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.536 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.537 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.537 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.537 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.537 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.537 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.537 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.537 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.537 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.537 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.537 INFO:teuthology.orchestra.run.smithi151.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-21T22:06:06.537 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-21T22:06:06.537 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.537 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.538 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.538 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.538 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.538 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.538 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.538 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-21T22:06:06.538 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.538 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.538 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.538 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.538 INFO:teuthology.orchestra.run.smithi151.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-21T22:06:06.538 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-21T22:06:06.539 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.539 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.539 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.539 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "*", 2024-05-21T22:06:06.539 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.539 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.539 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.539 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-21T22:06:06.539 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.539 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.540 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.540 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.540 INFO:teuthology.orchestra.run.smithi151.stdout: "max_count_per_host": { 2024-05-21T22:06:06.540 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "max_count_per_host", 2024-05-21T22:06:06.540 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:06.540 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.540 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.540 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "10", 2024-05-21T22:06:06.540 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.540 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.540 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.540 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "max number of daemons per service per host", 2024-05-21T22:06:06.540 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.540 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.541 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.541 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.541 INFO:teuthology.orchestra.run.smithi151.stdout: "max_osd_draining_count": { 2024-05-21T22:06:06.541 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "max_osd_draining_count", 2024-05-21T22:06:06.541 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:06.541 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.541 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.541 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "10", 2024-05-21T22:06:06.541 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.541 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.541 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.541 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-21T22:06:06.541 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.542 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.542 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.542 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.542 INFO:teuthology.orchestra.run.smithi151.stdout: "migration_current": { 2024-05-21T22:06:06.542 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "migration_current", 2024-05-21T22:06:06.542 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:06.542 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.542 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.542 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:06.542 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.542 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.542 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.542 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "internal - do not modify", 2024-05-21T22:06:06.543 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.543 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.543 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.543 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.543 INFO:teuthology.orchestra.run.smithi151.stdout: "mode": { 2024-05-21T22:06:06.543 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "mode", 2024-05-21T22:06:06.543 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.543 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.543 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.543 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "root", 2024-05-21T22:06:06.543 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.543 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.543 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:06.544 INFO:teuthology.orchestra.run.smithi151.stdout: "cephadm-package", 2024-05-21T22:06:06.544 INFO:teuthology.orchestra.run.smithi151.stdout: "root" 2024-05-21T22:06:06.544 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:06.544 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "mode for remote execution of cephadm", 2024-05-21T22:06:06.544 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.544 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.544 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.544 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.544 INFO:teuthology.orchestra.run.smithi151.stdout: "oob_default_addr": { 2024-05-21T22:06:06.544 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "oob_default_addr", 2024-05-21T22:06:06.544 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.544 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.544 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.545 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "169.254.1.1", 2024-05-21T22:06:06.545 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.545 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.545 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.545 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-21T22:06:06.545 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.545 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.545 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.545 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.545 INFO:teuthology.orchestra.run.smithi151.stdout: "prometheus_alerts_path": { 2024-05-21T22:06:06.545 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "prometheus_alerts_path", 2024-05-21T22:06:06.545 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.545 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.545 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.546 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-21T22:06:06.546 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.546 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.546 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.546 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-21T22:06:06.546 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.546 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.546 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.546 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.546 INFO:teuthology.orchestra.run.smithi151.stdout: "registry_insecure": { 2024-05-21T22:06:06.546 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "registry_insecure", 2024-05-21T22:06:06.546 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.547 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.547 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.547 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.547 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.547 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.547 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.547 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-21T22:06:06.547 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.547 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.547 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.547 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.547 INFO:teuthology.orchestra.run.smithi151.stdout: "registry_password": { 2024-05-21T22:06:06.548 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "registry_password", 2024-05-21T22:06:06.548 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.548 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.548 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.548 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:06.548 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.548 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.548 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.548 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-21T22:06:06.548 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.548 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.548 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.548 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.549 INFO:teuthology.orchestra.run.smithi151.stdout: "registry_url": { 2024-05-21T22:06:06.549 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "registry_url", 2024-05-21T22:06:06.549 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.549 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.549 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.549 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:06.549 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.549 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.549 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.550 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-21T22:06:06.550 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.550 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.550 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.550 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.550 INFO:teuthology.orchestra.run.smithi151.stdout: "registry_username": { 2024-05-21T22:06:06.550 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "registry_username", 2024-05-21T22:06:06.550 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.550 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.550 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.550 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:06.550 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.551 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.551 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.551 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-21T22:06:06.551 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.551 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.551 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.551 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.551 INFO:teuthology.orchestra.run.smithi151.stdout: "secure_monitoring_stack": { 2024-05-21T22:06:06.551 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "secure_monitoring_stack", 2024-05-21T22:06:06.551 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.551 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.551 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.551 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.552 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.552 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.552 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.552 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-21T22:06:06.552 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.552 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.552 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.552 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.552 INFO:teuthology.orchestra.run.smithi151.stdout: "service_discovery_port": { 2024-05-21T22:06:06.552 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "service_discovery_port", 2024-05-21T22:06:06.552 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:06.552 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.552 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.553 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "8765", 2024-05-21T22:06:06.553 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.553 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.553 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.553 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "cephadm service discovery port", 2024-05-21T22:06:06.553 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.553 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.553 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.553 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.553 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:06.553 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:06.553 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:06.553 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:06.554 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.554 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:06.554 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.554 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.554 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.554 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.554 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.554 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.554 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.554 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.554 INFO:teuthology.orchestra.run.smithi151.stdout: "ssh_config_file": { 2024-05-21T22:06:06.555 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "ssh_config_file", 2024-05-21T22:06:06.555 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.555 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.555 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.555 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:06.555 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.555 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.555 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.555 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-21T22:06:06.555 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.555 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.556 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.556 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.556 INFO:teuthology.orchestra.run.smithi151.stdout: "use_agent": { 2024-05-21T22:06:06.556 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "use_agent", 2024-05-21T22:06:06.556 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.556 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.556 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.556 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.556 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.556 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.556 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.556 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-21T22:06:06.557 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.557 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.557 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.557 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.557 INFO:teuthology.orchestra.run.smithi151.stdout: "use_repo_digest": { 2024-05-21T22:06:06.557 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "use_repo_digest", 2024-05-21T22:06:06.557 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.557 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.557 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.557 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:06.557 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.557 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.558 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.558 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-21T22:06:06.558 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.558 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.558 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.558 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.558 INFO:teuthology.orchestra.run.smithi151.stdout: "warn_on_failed_host_check": { 2024-05-21T22:06:06.558 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "warn_on_failed_host_check", 2024-05-21T22:06:06.558 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.558 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.558 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.558 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:06.559 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.559 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.559 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.559 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "raise a health warning if the host check fails", 2024-05-21T22:06:06.559 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.559 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.559 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.559 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.559 INFO:teuthology.orchestra.run.smithi151.stdout: "warn_on_stray_daemons": { 2024-05-21T22:06:06.559 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "warn_on_stray_daemons", 2024-05-21T22:06:06.560 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.560 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.560 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.560 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:06.560 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.560 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.560 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.560 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-21T22:06:06.560 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.560 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.560 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.561 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.561 INFO:teuthology.orchestra.run.smithi151.stdout: "warn_on_stray_hosts": { 2024-05-21T22:06:06.561 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "warn_on_stray_hosts", 2024-05-21T22:06:06.561 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.561 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.561 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.561 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:06.561 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.561 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.561 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.561 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-21T22:06:06.561 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.561 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.562 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.562 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:06.562 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:06.562 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.562 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:06.562 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "crash", 2024-05-21T22:06:06.562 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:06.562 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:06.562 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:06.562 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:06.562 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:06.562 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.562 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.563 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.563 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:06.563 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.563 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.563 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:06.563 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:06.563 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:06.563 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:06.563 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:06.563 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:06.563 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:06.563 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:06.564 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.564 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.564 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.564 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.564 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.564 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:06.564 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:06.565 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.565 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.565 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.565 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.565 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.565 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.565 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.566 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.566 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.566 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.566 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.566 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.566 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:06.567 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:06.567 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.567 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.567 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.567 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:06.567 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.567 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.568 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:06.568 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:06.568 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:06.568 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:06.568 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:06.568 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:06.569 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:06.569 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:06.569 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.569 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.569 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.569 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.570 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.570 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:06.570 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:06.570 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.570 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.570 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.571 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.571 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.571 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.571 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.571 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.571 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.572 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.572 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.572 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.572 INFO:teuthology.orchestra.run.smithi151.stdout: "retain_interval": { 2024-05-21T22:06:06.572 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "retain_interval", 2024-05-21T22:06:06.572 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-05-21T22:06:06.572 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.572 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.572 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "31536000", 2024-05-21T22:06:06.572 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.573 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.573 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.573 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-21T22:06:06.573 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.573 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.573 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.573 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.573 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:06.573 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:06.574 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:06.574 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:06.574 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.574 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:06.574 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.575 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.575 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.575 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.575 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.575 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.575 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.575 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.575 INFO:teuthology.orchestra.run.smithi151.stdout: "warn_recent_interval": { 2024-05-21T22:06:06.576 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "warn_recent_interval", 2024-05-21T22:06:06.576 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-05-21T22:06:06.576 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.576 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.576 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "1209600", 2024-05-21T22:06:06.576 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.576 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.576 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.576 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-21T22:06:06.577 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.577 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.577 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.577 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:06.577 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:06.578 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.578 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:06.578 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "dashboard", 2024-05-21T22:06:06.578 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:06.578 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:06.578 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:06.578 INFO:teuthology.orchestra.run.smithi151.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-21T22:06:06.578 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-21T22:06:06.578 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:06.579 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.579 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.579 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "10", 2024-05-21T22:06:06.579 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.579 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.579 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.579 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.579 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.579 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.580 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.580 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.580 INFO:teuthology.orchestra.run.smithi151.stdout: "ALERTMANAGER_API_HOST": { 2024-05-21T22:06:06.580 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-21T22:06:06.580 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.580 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.580 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.580 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:06.581 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.581 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.581 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.581 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.581 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.581 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.581 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.581 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.581 INFO:teuthology.orchestra.run.smithi151.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-21T22:06:06.581 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-21T22:06:06.581 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.582 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.582 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.582 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:06.582 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.582 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.582 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.582 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.582 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.582 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.582 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.582 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.582 INFO:teuthology.orchestra.run.smithi151.stdout: "AUDIT_API_ENABLED": { 2024-05-21T22:06:06.583 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "AUDIT_API_ENABLED", 2024-05-21T22:06:06.583 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.583 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.583 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.583 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.583 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.583 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.583 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.583 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.583 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.583 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.583 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.583 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.583 INFO:teuthology.orchestra.run.smithi151.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-21T22:06:06.584 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-21T22:06:06.584 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.584 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.584 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.584 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:06.584 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.584 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.584 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.584 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.584 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.584 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.584 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.584 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.584 INFO:teuthology.orchestra.run.smithi151.stdout: "ENABLE_BROWSABLE_API": { 2024-05-21T22:06:06.585 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-21T22:06:06.585 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.585 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.585 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.585 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:06.585 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.585 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.585 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.585 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.585 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.585 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.585 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.585 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.586 INFO:teuthology.orchestra.run.smithi151.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-21T22:06:06.586 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-21T22:06:06.586 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.586 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.586 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.586 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:06.586 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.586 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.586 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.586 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.586 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.586 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.586 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.587 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.587 INFO:teuthology.orchestra.run.smithi151.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-21T22:06:06.587 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-21T22:06:06.587 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.587 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.587 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.587 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:06.587 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.587 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.587 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.587 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.587 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.587 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.587 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.587 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.588 INFO:teuthology.orchestra.run.smithi151.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-21T22:06:06.588 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-21T22:06:06.588 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.588 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.588 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.588 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:06.588 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.588 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.588 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.588 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.588 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.588 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.588 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.588 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.588 INFO:teuthology.orchestra.run.smithi151.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-21T22:06:06.589 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-21T22:06:06.589 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.589 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.589 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.589 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:06.589 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.589 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.589 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.589 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.589 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.589 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.589 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.589 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.589 INFO:teuthology.orchestra.run.smithi151.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-21T22:06:06.590 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-21T22:06:06.590 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.590 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.590 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.590 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:06.590 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.591 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.591 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.591 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.591 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.591 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.591 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.591 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.592 INFO:teuthology.orchestra.run.smithi151.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-21T22:06:06.592 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-21T22:06:06.592 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.592 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.592 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.592 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:06.592 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.592 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.592 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.592 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.592 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.592 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.592 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.592 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.593 INFO:teuthology.orchestra.run.smithi151.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-21T22:06:06.593 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-21T22:06:06.593 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.593 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.593 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.593 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:06.593 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.593 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.593 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.593 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.593 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.593 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.593 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.593 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.594 INFO:teuthology.orchestra.run.smithi151.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-21T22:06:06.594 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-21T22:06:06.594 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.594 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.594 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.594 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:06.594 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.594 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.594 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.594 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.594 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.594 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.595 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.595 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.595 INFO:teuthology.orchestra.run.smithi151.stdout: "GRAFANA_API_PASSWORD": { 2024-05-21T22:06:06.595 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-21T22:06:06.595 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.595 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.595 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.595 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "admin", 2024-05-21T22:06:06.595 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.595 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.595 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.595 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.595 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.595 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.595 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.596 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.596 INFO:teuthology.orchestra.run.smithi151.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-21T22:06:06.596 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-21T22:06:06.596 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.596 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.596 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.596 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:06.596 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.596 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.596 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.596 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.596 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.597 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.597 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.597 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.597 INFO:teuthology.orchestra.run.smithi151.stdout: "GRAFANA_API_URL": { 2024-05-21T22:06:06.597 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "GRAFANA_API_URL", 2024-05-21T22:06:06.597 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.597 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.597 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.597 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:06.597 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.597 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.597 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.597 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.597 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.598 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.598 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.598 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.598 INFO:teuthology.orchestra.run.smithi151.stdout: "GRAFANA_API_USERNAME": { 2024-05-21T22:06:06.598 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-21T22:06:06.598 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.598 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.598 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.598 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "admin", 2024-05-21T22:06:06.598 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.598 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.598 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.598 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.599 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.599 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.599 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.599 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.599 INFO:teuthology.orchestra.run.smithi151.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-21T22:06:06.599 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-21T22:06:06.599 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.599 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.599 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.599 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:06.599 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.599 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.599 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.600 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.600 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.600 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.600 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.600 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.600 INFO:teuthology.orchestra.run.smithi151.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-21T22:06:06.600 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-21T22:06:06.600 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.600 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.600 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.600 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.600 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.600 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.600 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.600 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.601 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.601 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.601 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.601 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.601 INFO:teuthology.orchestra.run.smithi151.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-21T22:06:06.601 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-21T22:06:06.601 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.601 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.601 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.601 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:06.601 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.602 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.602 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.602 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.602 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.602 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.602 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.602 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.602 INFO:teuthology.orchestra.run.smithi151.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-21T22:06:06.602 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-21T22:06:06.602 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.603 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.603 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.603 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:06.603 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.603 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.603 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.603 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.603 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.603 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.603 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.603 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.603 INFO:teuthology.orchestra.run.smithi151.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-21T22:06:06.603 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-21T22:06:06.604 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.604 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.604 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.604 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "[]", 2024-05-21T22:06:06.604 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.604 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.604 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.604 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.605 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.605 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.605 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.605 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.605 INFO:teuthology.orchestra.run.smithi151.stdout: "MULTICLUSTER_CONFIG": { 2024-05-21T22:06:06.605 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-21T22:06:06.605 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.605 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.606 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.606 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "{}", 2024-05-21T22:06:06.606 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.606 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.606 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.606 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.606 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.606 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.606 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.606 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.606 INFO:teuthology.orchestra.run.smithi151.stdout: "PROMETHEUS_API_HOST": { 2024-05-21T22:06:06.606 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-21T22:06:06.609 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.610 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.610 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.610 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:06.610 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.610 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.610 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.610 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.611 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.611 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.611 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.611 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.611 INFO:teuthology.orchestra.run.smithi151.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-21T22:06:06.611 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-21T22:06:06.611 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.612 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.612 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.612 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:06.612 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.612 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.612 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.613 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.613 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.613 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.613 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.613 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.613 INFO:teuthology.orchestra.run.smithi151.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-21T22:06:06.613 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-21T22:06:06.614 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.614 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.614 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.614 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.614 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.614 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.615 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.615 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.615 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.615 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.615 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.615 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.616 INFO:teuthology.orchestra.run.smithi151.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-21T22:06:06.616 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-21T22:06:06.616 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.616 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.616 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.616 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.616 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.617 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.617 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.617 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.617 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.617 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.617 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.618 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.618 INFO:teuthology.orchestra.run.smithi151.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-21T22:06:06.618 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-21T22:06:06.618 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.618 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.618 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.619 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:06.619 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.619 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.619 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.619 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.620 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.620 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.620 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.620 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.620 INFO:teuthology.orchestra.run.smithi151.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-21T22:06:06.620 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-21T22:06:06.621 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.621 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.621 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.621 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:06.621 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.622 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.622 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.622 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.622 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.622 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.623 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.623 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.623 INFO:teuthology.orchestra.run.smithi151.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-21T22:06:06.623 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-21T22:06:06.623 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.623 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.623 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.624 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.624 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.624 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.624 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.624 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.624 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.624 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.624 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.624 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.624 INFO:teuthology.orchestra.run.smithi151.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-21T22:06:06.625 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-21T22:06:06.625 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.625 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.625 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.625 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.625 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.625 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.626 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.626 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.626 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.626 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.626 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.626 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.626 INFO:teuthology.orchestra.run.smithi151.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-21T22:06:06.626 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-21T22:06:06.626 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.626 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.626 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.627 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.627 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.627 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.627 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.627 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.627 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.627 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.627 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.627 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.627 INFO:teuthology.orchestra.run.smithi151.stdout: "PWD_POLICY_ENABLED": { 2024-05-21T22:06:06.627 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-21T22:06:06.627 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.627 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.627 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.628 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:06.628 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.628 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.628 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.628 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.628 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.628 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.628 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.628 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.628 INFO:teuthology.orchestra.run.smithi151.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-21T22:06:06.628 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-21T22:06:06.628 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.628 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.628 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.629 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-21T22:06:06.629 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.629 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.629 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.629 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.629 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.629 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.629 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.629 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.629 INFO:teuthology.orchestra.run.smithi151.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-21T22:06:06.629 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-21T22:06:06.629 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:06.629 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.629 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.629 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "10", 2024-05-21T22:06:06.630 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.630 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.630 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.630 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.630 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.630 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.630 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.630 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.630 INFO:teuthology.orchestra.run.smithi151.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-21T22:06:06.630 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-21T22:06:06.630 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:06.630 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.630 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.630 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "8", 2024-05-21T22:06:06.631 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.631 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.631 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.631 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.631 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.631 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.631 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.631 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.631 INFO:teuthology.orchestra.run.smithi151.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-21T22:06:06.631 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-21T22:06:06.631 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:06.631 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.631 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.631 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "45", 2024-05-21T22:06:06.631 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.631 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.632 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.632 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.632 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.632 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.632 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.632 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.632 INFO:teuthology.orchestra.run.smithi151.stdout: "RGW_API_ACCESS_KEY": { 2024-05-21T22:06:06.632 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-21T22:06:06.632 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.632 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.632 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.632 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:06.632 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.632 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.632 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.633 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.633 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.633 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.633 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.633 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.633 INFO:teuthology.orchestra.run.smithi151.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-21T22:06:06.633 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-21T22:06:06.633 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.633 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.633 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.633 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "admin", 2024-05-21T22:06:06.633 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.633 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.633 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.634 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.634 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.634 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.634 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.634 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.634 INFO:teuthology.orchestra.run.smithi151.stdout: "RGW_API_SECRET_KEY": { 2024-05-21T22:06:06.634 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-21T22:06:06.634 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.634 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.634 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.634 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:06.634 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.634 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.635 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.635 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.635 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.635 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.635 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.635 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.635 INFO:teuthology.orchestra.run.smithi151.stdout: "RGW_API_SSL_VERIFY": { 2024-05-21T22:06:06.635 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-21T22:06:06.635 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.635 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.635 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.635 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:06.635 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.636 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.636 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.636 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.636 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.636 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.636 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.636 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.636 INFO:teuthology.orchestra.run.smithi151.stdout: "UNSAFE_TLS_v1_2": { 2024-05-21T22:06:06.636 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-21T22:06:06.636 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.636 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.636 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.636 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.636 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.637 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.637 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.637 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.637 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.637 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.637 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.637 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.637 INFO:teuthology.orchestra.run.smithi151.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-21T22:06:06.637 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-21T22:06:06.637 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:06.637 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.637 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.637 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:06.637 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.638 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.638 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.638 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.638 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.638 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.638 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.638 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.638 INFO:teuthology.orchestra.run.smithi151.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-21T22:06:06.638 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-21T22:06:06.638 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:06.638 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.638 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.639 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "10", 2024-05-21T22:06:06.639 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.639 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.639 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.639 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.639 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.639 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.639 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.639 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.639 INFO:teuthology.orchestra.run.smithi151.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-21T22:06:06.639 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-21T22:06:06.639 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:06.639 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.639 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.640 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "5", 2024-05-21T22:06:06.640 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.640 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.640 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.640 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.640 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.640 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.640 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.640 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.640 INFO:teuthology.orchestra.run.smithi151.stdout: "cross_origin_url": { 2024-05-21T22:06:06.640 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "cross_origin_url", 2024-05-21T22:06:06.640 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.640 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.640 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.640 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:06.641 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.641 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.641 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.641 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.641 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.641 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.641 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.641 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.641 INFO:teuthology.orchestra.run.smithi151.stdout: "crt_file": { 2024-05-21T22:06:06.641 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "crt_file", 2024-05-21T22:06:06.641 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.641 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.641 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.641 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:06.642 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.642 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.642 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.642 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.642 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.642 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.642 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.642 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.642 INFO:teuthology.orchestra.run.smithi151.stdout: "debug": { 2024-05-21T22:06:06.642 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "debug", 2024-05-21T22:06:06.642 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.642 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.642 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.643 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.643 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.643 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.643 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.643 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Enable/disable debug options", 2024-05-21T22:06:06.643 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.643 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.643 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.643 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.643 INFO:teuthology.orchestra.run.smithi151.stdout: "jwt_token_ttl": { 2024-05-21T22:06:06.643 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "jwt_token_ttl", 2024-05-21T22:06:06.643 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:06.643 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.644 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.644 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "28800", 2024-05-21T22:06:06.644 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.644 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.644 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.644 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.644 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.644 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.644 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.644 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.644 INFO:teuthology.orchestra.run.smithi151.stdout: "key_file": { 2024-05-21T22:06:06.644 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "key_file", 2024-05-21T22:06:06.644 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.644 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.645 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.645 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:06.645 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.645 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.645 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.645 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.645 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.645 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.645 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.645 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.645 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:06.645 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:06.646 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.646 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.646 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.646 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:06.646 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.646 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.646 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:06.646 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:06.646 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:06.646 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:06.646 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:06.646 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:06.646 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:06.646 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:06.647 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.647 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.647 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.647 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.647 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.647 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:06.647 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:06.647 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.647 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.647 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.647 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.647 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.647 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.647 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.648 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.648 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.648 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.648 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.648 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.648 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:06.648 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:06.648 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.648 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.648 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.648 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:06.648 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.648 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.648 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:06.649 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:06.649 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:06.649 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:06.649 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:06.649 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:06.649 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:06.649 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:06.649 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.649 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.649 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.649 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.649 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.649 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:06.650 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:06.650 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.650 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.650 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.650 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.650 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.650 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.650 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.650 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.650 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.650 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.650 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.650 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.651 INFO:teuthology.orchestra.run.smithi151.stdout: "motd": { 2024-05-21T22:06:06.651 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "motd", 2024-05-21T22:06:06.651 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.651 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.651 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.651 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:06.651 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.651 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.651 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.651 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "The message of the day", 2024-05-21T22:06:06.651 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.652 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.652 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.652 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.652 INFO:teuthology.orchestra.run.smithi151.stdout: "redirect_resolve_ip_addr": { 2024-05-21T22:06:06.652 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "redirect_resolve_ip_addr", 2024-05-21T22:06:06.652 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.652 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.652 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.652 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.652 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.652 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.652 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.653 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.653 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.653 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.653 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.653 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.653 INFO:teuthology.orchestra.run.smithi151.stdout: "server_addr": { 2024-05-21T22:06:06.653 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "server_addr", 2024-05-21T22:06:06.653 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.653 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.653 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.653 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "::", 2024-05-21T22:06:06.653 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.653 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.653 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.654 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.654 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.654 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.654 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.654 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.654 INFO:teuthology.orchestra.run.smithi151.stdout: "server_port": { 2024-05-21T22:06:06.654 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "server_port", 2024-05-21T22:06:06.654 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:06.654 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.654 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.654 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "8080", 2024-05-21T22:06:06.654 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.655 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.655 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.655 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.655 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.655 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.655 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.655 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.655 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:06.655 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:06.655 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:06.655 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:06.655 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.655 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:06.655 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.655 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.656 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.656 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.656 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.656 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.656 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.656 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.656 INFO:teuthology.orchestra.run.smithi151.stdout: "ssl": { 2024-05-21T22:06:06.656 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "ssl", 2024-05-21T22:06:06.656 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.656 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.656 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.656 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:06.656 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.656 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.656 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.657 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.657 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.657 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.657 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.657 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.657 INFO:teuthology.orchestra.run.smithi151.stdout: "ssl_server_port": { 2024-05-21T22:06:06.657 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "ssl_server_port", 2024-05-21T22:06:06.657 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:06.657 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.657 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.657 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "8443", 2024-05-21T22:06:06.657 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.657 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.658 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.658 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.658 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.658 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.658 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.658 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.658 INFO:teuthology.orchestra.run.smithi151.stdout: "standby_behaviour": { 2024-05-21T22:06:06.658 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "standby_behaviour", 2024-05-21T22:06:06.658 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.658 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.658 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.658 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "redirect", 2024-05-21T22:06:06.658 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.659 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.659 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:06.659 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:06.659 INFO:teuthology.orchestra.run.smithi151.stdout: "redirect" 2024-05-21T22:06:06.659 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:06.659 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.659 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.659 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.659 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.659 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.659 INFO:teuthology.orchestra.run.smithi151.stdout: "standby_error_status_code": { 2024-05-21T22:06:06.659 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "standby_error_status_code", 2024-05-21T22:06:06.660 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:06.660 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.660 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.660 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "500", 2024-05-21T22:06:06.660 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "400", 2024-05-21T22:06:06.660 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "599", 2024-05-21T22:06:06.660 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.660 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.660 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.660 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.660 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.660 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.660 INFO:teuthology.orchestra.run.smithi151.stdout: "url_prefix": { 2024-05-21T22:06:06.660 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "url_prefix", 2024-05-21T22:06:06.661 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.661 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.661 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.661 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:06.661 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.661 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.661 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.661 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.661 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.661 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.661 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.661 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:06.661 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:06.661 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.661 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:06.662 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "devicehealth", 2024-05-21T22:06:06.662 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:06.662 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:06.662 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:06.662 INFO:teuthology.orchestra.run.smithi151.stdout: "enable_monitoring": { 2024-05-21T22:06:06.662 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "enable_monitoring", 2024-05-21T22:06:06.662 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.662 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.662 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.662 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:06.662 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.662 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.662 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.662 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "monitor device health metrics", 2024-05-21T22:06:06.663 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.663 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.663 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.663 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.663 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:06.663 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:06.663 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.663 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.663 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.663 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:06.663 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.663 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.663 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:06.663 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:06.664 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:06.664 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:06.664 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:06.664 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:06.664 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:06.664 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:06.664 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.664 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.664 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.664 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.665 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.665 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:06.665 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:06.665 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.665 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.665 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.665 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.665 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.665 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.665 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.665 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.665 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.665 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.666 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.666 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.666 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:06.666 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:06.666 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.666 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.666 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.666 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:06.666 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.666 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.666 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:06.666 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:06.666 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:06.666 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:06.667 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:06.667 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:06.667 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:06.667 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:06.667 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.667 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.667 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.667 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.667 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.667 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:06.667 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:06.667 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.667 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.668 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.668 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.668 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.668 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.668 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.668 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.668 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.668 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.668 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.668 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.668 INFO:teuthology.orchestra.run.smithi151.stdout: "mark_out_threshold": { 2024-05-21T22:06:06.668 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "mark_out_threshold", 2024-05-21T22:06:06.668 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-05-21T22:06:06.668 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.669 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.669 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "2419200", 2024-05-21T22:06:06.669 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.669 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.669 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.669 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-21T22:06:06.669 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.669 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.669 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.669 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.669 INFO:teuthology.orchestra.run.smithi151.stdout: "pool_name": { 2024-05-21T22:06:06.669 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "pool_name", 2024-05-21T22:06:06.669 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.669 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.670 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.670 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "device_health_metrics", 2024-05-21T22:06:06.670 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.670 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.670 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.670 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-21T22:06:06.670 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.670 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.670 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.670 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.670 INFO:teuthology.orchestra.run.smithi151.stdout: "retention_period": { 2024-05-21T22:06:06.670 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "retention_period", 2024-05-21T22:06:06.670 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-05-21T22:06:06.671 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.671 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.671 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "15552000", 2024-05-21T22:06:06.671 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.671 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.671 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.671 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "how long to retain device health metrics", 2024-05-21T22:06:06.671 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.671 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.671 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.671 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.671 INFO:teuthology.orchestra.run.smithi151.stdout: "scrape_frequency": { 2024-05-21T22:06:06.671 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "scrape_frequency", 2024-05-21T22:06:06.671 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-05-21T22:06:06.672 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.672 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.672 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "86400", 2024-05-21T22:06:06.672 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.672 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.672 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.672 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-21T22:06:06.672 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.672 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.672 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.672 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.672 INFO:teuthology.orchestra.run.smithi151.stdout: "self_heal": { 2024-05-21T22:06:06.672 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "self_heal", 2024-05-21T22:06:06.673 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.673 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.673 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.673 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:06.673 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.673 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.673 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.673 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-21T22:06:06.673 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.673 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.673 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.673 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.673 INFO:teuthology.orchestra.run.smithi151.stdout: "sleep_interval": { 2024-05-21T22:06:06.673 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sleep_interval", 2024-05-21T22:06:06.673 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-05-21T22:06:06.674 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.674 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.674 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "600", 2024-05-21T22:06:06.674 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.674 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.674 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.674 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "how frequently to wake up and check device health", 2024-05-21T22:06:06.674 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.674 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.674 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.674 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.674 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:06.675 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:06.675 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:06.675 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:06.675 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.675 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:06.675 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.675 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.675 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.675 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.675 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.675 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.675 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.676 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.676 INFO:teuthology.orchestra.run.smithi151.stdout: "warn_threshold": { 2024-05-21T22:06:06.676 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "warn_threshold", 2024-05-21T22:06:06.676 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-05-21T22:06:06.676 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.676 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.676 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "7257600", 2024-05-21T22:06:06.676 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.676 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.676 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.676 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-21T22:06:06.676 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.676 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.676 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.677 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:06.677 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:06.677 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.677 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:06.677 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "diskprediction_local", 2024-05-21T22:06:06.677 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:06.677 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:06.677 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:06.677 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:06.677 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:06.677 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.677 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.678 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.678 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:06.678 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.678 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.678 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:06.678 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:06.678 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:06.678 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:06.678 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:06.678 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:06.678 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:06.679 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:06.679 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.679 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.679 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.679 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.679 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.679 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:06.679 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:06.679 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.679 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.679 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.679 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.679 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.680 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.680 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.680 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.680 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.680 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.680 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.680 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.680 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:06.680 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:06.680 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.680 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.680 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.680 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:06.680 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.681 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.681 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:06.681 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:06.681 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:06.681 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:06.681 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:06.681 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:06.681 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:06.681 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:06.681 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.681 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.681 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.681 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.681 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.682 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:06.682 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:06.682 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.682 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.682 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.682 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.682 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.682 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.682 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.682 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.682 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.682 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.682 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.683 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.683 INFO:teuthology.orchestra.run.smithi151.stdout: "predict_interval": { 2024-05-21T22:06:06.683 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "predict_interval", 2024-05-21T22:06:06.683 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.683 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.683 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.683 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "86400", 2024-05-21T22:06:06.683 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.683 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.683 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.683 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.683 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.683 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.683 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.684 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.684 INFO:teuthology.orchestra.run.smithi151.stdout: "predictor_model": { 2024-05-21T22:06:06.684 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "predictor_model", 2024-05-21T22:06:06.684 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.684 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.684 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.684 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "prophetstor", 2024-05-21T22:06:06.684 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.684 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.684 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.684 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.684 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.684 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.685 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.685 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.685 INFO:teuthology.orchestra.run.smithi151.stdout: "sleep_interval": { 2024-05-21T22:06:06.685 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sleep_interval", 2024-05-21T22:06:06.685 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.685 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.685 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.685 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "600", 2024-05-21T22:06:06.685 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.685 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.685 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.685 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.685 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.686 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.686 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.686 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.686 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:06.686 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:06.686 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:06.686 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:06.686 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.686 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:06.686 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.686 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.686 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.687 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.687 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.687 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.687 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.687 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:06.687 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:06.687 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.687 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:06.687 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "influx", 2024-05-21T22:06:06.687 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": false, 2024-05-21T22:06:06.687 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "influxdb python module not found", 2024-05-21T22:06:06.687 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:06.687 INFO:teuthology.orchestra.run.smithi151.stdout: "batch_size": { 2024-05-21T22:06:06.688 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "batch_size", 2024-05-21T22:06:06.688 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:06.688 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.688 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.688 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "5000", 2024-05-21T22:06:06.688 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.688 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.688 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.688 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-21T22:06:06.688 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.688 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.688 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.688 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.688 INFO:teuthology.orchestra.run.smithi151.stdout: "database": { 2024-05-21T22:06:06.688 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "database", 2024-05-21T22:06:06.689 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.689 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.689 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.689 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "ceph", 2024-05-21T22:06:06.689 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.689 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.689 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.689 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-21T22:06:06.689 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.689 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.689 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.689 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.689 INFO:teuthology.orchestra.run.smithi151.stdout: "hostname": { 2024-05-21T22:06:06.689 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "hostname", 2024-05-21T22:06:06.690 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.690 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.690 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.690 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:06.690 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.690 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.690 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.690 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "InfluxDB server hostname", 2024-05-21T22:06:06.690 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.690 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.690 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.690 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.691 INFO:teuthology.orchestra.run.smithi151.stdout: "interval": { 2024-05-21T22:06:06.691 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "interval", 2024-05-21T22:06:06.691 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-05-21T22:06:06.691 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.691 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.691 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "30", 2024-05-21T22:06:06.691 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "5", 2024-05-21T22:06:06.691 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.691 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.691 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-21T22:06:06.691 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.691 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.692 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.692 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.692 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:06.692 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:06.692 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.692 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.692 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.692 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:06.692 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.692 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.692 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:06.692 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:06.692 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:06.692 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:06.693 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:06.693 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:06.693 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:06.693 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:06.693 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.693 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.693 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.693 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.693 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.693 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:06.693 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:06.693 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.694 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.694 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.694 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.694 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.694 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.694 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.694 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.694 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.694 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.694 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.694 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.694 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:06.694 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:06.694 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.695 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.695 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.695 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:06.695 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.695 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.695 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:06.695 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:06.695 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:06.695 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:06.695 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:06.695 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:06.695 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:06.695 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:06.695 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.695 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.696 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.696 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.696 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.696 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:06.696 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:06.696 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.696 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.696 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.696 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.696 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.696 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.696 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.696 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.696 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.697 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.697 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.697 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.697 INFO:teuthology.orchestra.run.smithi151.stdout: "password": { 2024-05-21T22:06:06.697 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "password", 2024-05-21T22:06:06.697 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.697 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.697 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.697 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:06.697 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.697 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.697 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.697 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "password of InfluxDB server user", 2024-05-21T22:06:06.698 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.698 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.698 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.698 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.698 INFO:teuthology.orchestra.run.smithi151.stdout: "port": { 2024-05-21T22:06:06.698 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "port", 2024-05-21T22:06:06.698 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:06.698 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.698 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.698 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "8086", 2024-05-21T22:06:06.699 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.699 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.699 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.699 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "InfluxDB server port", 2024-05-21T22:06:06.699 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.699 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.699 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.699 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.699 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:06.699 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:06.699 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:06.699 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:06.700 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.700 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:06.700 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.700 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.700 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.700 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.700 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.700 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.700 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.700 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.700 INFO:teuthology.orchestra.run.smithi151.stdout: "ssl": { 2024-05-21T22:06:06.700 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "ssl", 2024-05-21T22:06:06.700 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.701 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.701 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.701 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "false", 2024-05-21T22:06:06.701 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.701 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.701 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.701 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-21T22:06:06.701 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.701 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.701 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.701 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.701 INFO:teuthology.orchestra.run.smithi151.stdout: "threads": { 2024-05-21T22:06:06.702 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "threads", 2024-05-21T22:06:06.702 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:06.702 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.702 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.702 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "5", 2024-05-21T22:06:06.702 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "1", 2024-05-21T22:06:06.702 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "32", 2024-05-21T22:06:06.702 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.702 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-21T22:06:06.702 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.702 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.702 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.703 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.703 INFO:teuthology.orchestra.run.smithi151.stdout: "username": { 2024-05-21T22:06:06.703 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "username", 2024-05-21T22:06:06.703 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.703 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.703 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.703 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:06.703 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.703 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.703 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.703 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "username of InfluxDB server user", 2024-05-21T22:06:06.703 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.703 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.704 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.704 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.704 INFO:teuthology.orchestra.run.smithi151.stdout: "verify_ssl": { 2024-05-21T22:06:06.704 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "verify_ssl", 2024-05-21T22:06:06.704 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.704 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.704 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.704 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "true", 2024-05-21T22:06:06.704 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.704 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.704 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.704 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-21T22:06:06.704 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.705 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.705 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.705 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:06.705 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:06.705 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.705 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:06.705 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "insights", 2024-05-21T22:06:06.705 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:06.705 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:06.705 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:06.705 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:06.705 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:06.705 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.706 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.706 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.706 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:06.706 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.706 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.706 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:06.706 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:06.706 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:06.706 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:06.706 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:06.706 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:06.706 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:06.706 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:06.707 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.707 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.707 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.707 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.707 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.707 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:06.707 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:06.707 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.707 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.707 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.707 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.707 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.708 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.708 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.708 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.708 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.708 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.708 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.708 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.708 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:06.708 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:06.708 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.708 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.708 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.708 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:06.709 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.709 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.709 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:06.709 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:06.709 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:06.709 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:06.709 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:06.709 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:06.709 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:06.709 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:06.709 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.709 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.709 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.709 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.710 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.710 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:06.710 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:06.710 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.710 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.710 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.710 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.710 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.710 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.710 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.710 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.710 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.711 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.711 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.711 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.711 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:06.711 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:06.711 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:06.711 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:06.711 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.711 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:06.711 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.711 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.711 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.711 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.712 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.712 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.712 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.712 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:06.712 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:06.712 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.712 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:06.712 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "iostat", 2024-05-21T22:06:06.712 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:06.712 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:06.712 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:06.712 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:06.712 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:06.712 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.713 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.713 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.713 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:06.713 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.713 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.713 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:06.713 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:06.713 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:06.713 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:06.713 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:06.713 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:06.713 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:06.713 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:06.713 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.714 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.714 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.714 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.714 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.714 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:06.714 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:06.714 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.714 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.714 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.714 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.714 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.714 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.714 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.715 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.715 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.715 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.715 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.715 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.715 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:06.715 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:06.715 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.715 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.715 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.715 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:06.715 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.715 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.716 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:06.716 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:06.716 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:06.716 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:06.716 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:06.716 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:06.716 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:06.716 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:06.716 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.716 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.716 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.716 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.716 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.716 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:06.716 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:06.717 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.717 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.717 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.717 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.717 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.717 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.717 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.717 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.717 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.717 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.717 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.717 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.717 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:06.717 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:06.718 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:06.718 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:06.718 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.718 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:06.718 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.718 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.718 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.718 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.718 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.718 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.718 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.718 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:06.718 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:06.719 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.719 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:06.719 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "k8sevents", 2024-05-21T22:06:06.719 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:06.719 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:06.719 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:06.719 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_event_retention_days": { 2024-05-21T22:06:06.719 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "ceph_event_retention_days", 2024-05-21T22:06:06.719 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:06.719 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.719 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.719 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "7", 2024-05-21T22:06:06.720 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.720 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.720 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.720 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-21T22:06:06.720 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.720 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.720 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.720 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.720 INFO:teuthology.orchestra.run.smithi151.stdout: "config_check_secs": { 2024-05-21T22:06:06.720 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "config_check_secs", 2024-05-21T22:06:06.720 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:06.720 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.720 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.721 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "10", 2024-05-21T22:06:06.721 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "10", 2024-05-21T22:06:06.721 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.721 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.721 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-21T22:06:06.721 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.721 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.721 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.721 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.721 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:06.721 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:06.722 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.722 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.722 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.722 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:06.722 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.722 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.722 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:06.722 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:06.722 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:06.722 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:06.722 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:06.722 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:06.723 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:06.723 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:06.723 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.723 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.723 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.723 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.723 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.723 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:06.723 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:06.723 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.723 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.723 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.723 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.723 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.724 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.724 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.724 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.724 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.724 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.724 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.724 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.724 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:06.724 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:06.724 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.724 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.724 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.725 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:06.725 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.725 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.725 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:06.725 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:06.725 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:06.725 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:06.725 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:06.725 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:06.725 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:06.725 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:06.725 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.725 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.726 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.726 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.726 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.726 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:06.726 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:06.726 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.726 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.726 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.726 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.726 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.726 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.726 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.726 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.726 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.727 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.727 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.727 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.727 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:06.727 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:06.727 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:06.727 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:06.727 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.727 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:06.727 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.727 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.727 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.728 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.728 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.728 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.728 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.728 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:06.728 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:06.728 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.728 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:06.728 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "localpool", 2024-05-21T22:06:06.728 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:06.728 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:06.728 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:06.729 INFO:teuthology.orchestra.run.smithi151.stdout: "failure_domain": { 2024-05-21T22:06:06.729 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "failure_domain", 2024-05-21T22:06:06.729 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.729 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.729 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.729 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "host", 2024-05-21T22:06:06.729 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.729 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.729 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.729 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "failure domain for any created local pool", 2024-05-21T22:06:06.729 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-21T22:06:06.729 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.729 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.730 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.730 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:06.730 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:06.730 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.730 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.730 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.730 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:06.730 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.730 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.730 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:06.730 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:06.730 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:06.731 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:06.731 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:06.731 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:06.731 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:06.731 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:06.731 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.731 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.731 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.731 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.731 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.731 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:06.731 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:06.731 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.732 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.732 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.732 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.732 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.732 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.732 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.732 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.732 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.732 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.732 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.732 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.732 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:06.732 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:06.733 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.733 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.733 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.733 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:06.733 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.733 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.733 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:06.733 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:06.733 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:06.733 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:06.733 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:06.733 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:06.733 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:06.733 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:06.734 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.734 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.734 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.734 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.734 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.734 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:06.734 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:06.734 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.734 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.734 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.734 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.734 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.735 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.735 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.735 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.735 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.735 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.735 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.735 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.735 INFO:teuthology.orchestra.run.smithi151.stdout: "min_size": { 2024-05-21T22:06:06.735 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "min_size", 2024-05-21T22:06:06.735 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:06.735 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.735 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.735 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:06.735 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.736 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.736 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.736 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "default min_size for any created local pool", 2024-05-21T22:06:06.736 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-21T22:06:06.736 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.736 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.736 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.736 INFO:teuthology.orchestra.run.smithi151.stdout: "num_rep": { 2024-05-21T22:06:06.736 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "num_rep", 2024-05-21T22:06:06.736 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:06.736 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.736 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.737 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "3", 2024-05-21T22:06:06.737 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.737 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.737 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.737 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "default replica count for any created local pool", 2024-05-21T22:06:06.737 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.737 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.737 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.737 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.737 INFO:teuthology.orchestra.run.smithi151.stdout: "pg_num": { 2024-05-21T22:06:06.737 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "pg_num", 2024-05-21T22:06:06.737 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:06.737 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.737 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.737 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "128", 2024-05-21T22:06:06.738 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.738 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.738 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.738 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "default pg_num for any created local pool", 2024-05-21T22:06:06.738 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.738 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.738 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.738 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.738 INFO:teuthology.orchestra.run.smithi151.stdout: "prefix": { 2024-05-21T22:06:06.738 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "prefix", 2024-05-21T22:06:06.738 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.739 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.739 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.739 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:06.739 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.739 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.739 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.739 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "name prefix for any created local pool", 2024-05-21T22:06:06.739 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.739 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.739 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.739 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.739 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:06.739 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:06.740 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:06.740 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:06.740 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.740 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:06.740 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.740 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.740 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.740 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.740 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.740 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.740 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.740 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.740 INFO:teuthology.orchestra.run.smithi151.stdout: "subtree": { 2024-05-21T22:06:06.740 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "subtree", 2024-05-21T22:06:06.741 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.741 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.741 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.741 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "rack", 2024-05-21T22:06:06.741 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.741 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.741 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.741 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-21T22:06:06.741 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-21T22:06:06.741 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.741 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.741 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:06.741 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:06.741 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.742 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:06.742 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "mds_autoscaler", 2024-05-21T22:06:06.742 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:06.742 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:06.742 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:06.742 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:06.742 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:06.742 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.742 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.742 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.742 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:06.742 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.742 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.742 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:06.743 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:06.743 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:06.743 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:06.743 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:06.743 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:06.743 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:06.743 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:06.743 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.743 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.743 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.743 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.743 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.743 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:06.743 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:06.744 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.744 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.744 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.744 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.744 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.744 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.744 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.744 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.744 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.744 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.744 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.744 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.744 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:06.745 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:06.745 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.745 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.745 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.745 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:06.745 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.745 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.745 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:06.745 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:06.745 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:06.745 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:06.745 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:06.745 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:06.745 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:06.746 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:06.746 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.746 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.746 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.746 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.746 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.746 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:06.746 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:06.746 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.746 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.746 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.746 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.747 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.747 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.747 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.747 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.747 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.747 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.747 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.747 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.747 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:06.747 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:06.747 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:06.747 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:06.748 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.748 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:06.748 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.748 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.748 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.748 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.748 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.748 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.748 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.748 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:06.748 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:06.749 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.749 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:06.749 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "mirroring", 2024-05-21T22:06:06.749 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:06.749 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:06.749 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:06.749 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:06.749 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:06.749 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.749 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.749 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.749 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:06.749 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.750 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.750 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:06.750 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:06.750 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:06.750 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:06.750 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:06.750 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:06.750 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:06.750 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:06.750 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.750 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.751 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.751 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.751 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.751 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:06.751 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:06.751 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.751 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.751 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.751 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.751 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.751 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.751 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.752 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.752 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.752 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.752 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.752 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.752 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:06.752 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:06.752 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.752 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.752 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.752 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:06.752 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.753 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.753 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:06.753 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:06.753 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:06.753 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:06.753 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:06.753 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:06.753 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:06.753 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:06.753 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.753 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.754 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.754 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.754 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.754 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:06.754 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:06.754 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.754 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.754 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.754 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.754 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.754 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.754 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.755 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.755 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.755 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.755 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.755 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.755 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:06.755 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:06.755 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:06.755 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:06.755 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.755 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:06.755 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.756 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.756 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.756 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.756 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.756 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.756 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.756 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:06.756 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:06.756 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.756 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:06.756 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "nfs", 2024-05-21T22:06:06.756 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:06.756 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:06.757 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:06.757 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:06.757 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:06.757 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.757 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.757 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.757 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:06.757 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.757 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.757 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:06.757 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:06.757 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:06.757 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:06.758 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:06.758 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:06.758 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:06.758 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:06.758 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.758 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.758 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.758 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.758 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.758 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:06.758 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:06.758 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.759 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.759 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.759 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.759 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.759 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.759 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.759 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.759 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.759 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.759 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.759 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.759 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:06.759 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:06.760 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.760 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.760 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.760 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:06.760 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.760 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.760 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:06.760 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:06.760 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:06.760 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:06.760 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:06.760 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:06.760 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:06.761 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:06.761 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.761 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.761 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.761 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.761 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.761 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:06.761 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:06.761 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.761 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.761 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.761 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.761 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.762 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.762 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.762 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.762 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.762 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.762 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.762 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.762 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:06.762 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:06.762 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:06.762 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:06.762 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.763 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:06.763 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.763 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.763 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.763 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.763 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.763 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.763 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.763 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:06.763 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:06.763 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.763 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:06.764 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "orchestrator", 2024-05-21T22:06:06.764 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:06.764 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:06.764 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:06.764 INFO:teuthology.orchestra.run.smithi151.stdout: "fail_fs": { 2024-05-21T22:06:06.764 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "fail_fs", 2024-05-21T22:06:06.764 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.764 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.764 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.764 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.764 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.764 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.764 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.764 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-21T22:06:06.765 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.765 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.765 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.765 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.765 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:06.765 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:06.765 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.765 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.765 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.765 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:06.765 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.765 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.766 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:06.766 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:06.766 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:06.766 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:06.766 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:06.766 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:06.766 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:06.766 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:06.766 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.766 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.766 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.767 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.767 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.767 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:06.767 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:06.767 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.767 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.767 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.767 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.767 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.767 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.767 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.767 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.767 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.768 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.768 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.768 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.768 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:06.768 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:06.768 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.768 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.768 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.768 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:06.768 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.768 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.768 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:06.769 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:06.769 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:06.769 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:06.769 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:06.769 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:06.769 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:06.769 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:06.769 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.769 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.769 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.769 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.769 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.770 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:06.770 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:06.770 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.770 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.770 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.770 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.770 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.770 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.770 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.770 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.770 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.770 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.770 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.771 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.771 INFO:teuthology.orchestra.run.smithi151.stdout: "orchestrator": { 2024-05-21T22:06:06.771 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "orchestrator", 2024-05-21T22:06:06.771 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.771 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.771 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.771 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:06.771 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.771 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.771 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:06.771 INFO:teuthology.orchestra.run.smithi151.stdout: "cephadm", 2024-05-21T22:06:06.771 INFO:teuthology.orchestra.run.smithi151.stdout: "rook", 2024-05-21T22:06:06.771 INFO:teuthology.orchestra.run.smithi151.stdout: "test_orchestrator" 2024-05-21T22:06:06.771 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:06.772 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Orchestrator backend", 2024-05-21T22:06:06.772 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.772 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.772 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.772 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.772 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:06.772 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:06.772 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:06.772 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:06.772 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.772 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:06.772 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.773 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.773 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.773 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.773 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.773 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.773 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.773 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:06.773 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:06.773 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.773 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:06.773 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "osd_perf_query", 2024-05-21T22:06:06.773 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:06.773 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:06.774 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:06.774 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:06.774 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:06.774 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.774 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.774 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.774 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:06.774 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.774 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.774 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:06.774 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:06.774 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:06.774 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:06.774 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:06.775 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:06.775 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:06.775 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:06.775 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.775 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.775 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.775 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.775 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.775 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:06.775 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:06.775 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.775 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.775 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.775 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.776 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.776 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.776 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.776 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.776 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.776 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.776 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.776 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.776 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:06.776 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:06.776 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.776 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.776 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.776 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:06.777 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.777 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.777 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:06.777 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:06.777 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:06.777 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:06.777 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:06.777 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:06.777 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:06.777 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:06.777 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.777 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.777 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.778 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.778 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.778 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:06.778 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:06.778 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.778 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.778 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.778 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.778 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.778 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.778 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.778 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.779 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.779 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.779 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.779 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.779 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:06.779 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:06.779 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:06.779 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:06.779 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.779 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:06.779 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.779 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.779 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.780 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.780 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.780 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.780 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.780 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:06.780 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:06.780 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.780 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:06.780 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "osd_support", 2024-05-21T22:06:06.780 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:06.780 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:06.780 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:06.780 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:06.781 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:06.781 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.781 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.781 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.781 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:06.781 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.781 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.781 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:06.781 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:06.781 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:06.781 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:06.781 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:06.781 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:06.781 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:06.781 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:06.782 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.782 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.782 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.782 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.782 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.782 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:06.782 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:06.782 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.782 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.782 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.782 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.782 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.782 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.783 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.783 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.783 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.783 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.783 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.783 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.783 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:06.783 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:06.783 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.783 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.783 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.783 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:06.783 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.784 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.784 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:06.784 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:06.784 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:06.784 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:06.784 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:06.784 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:06.784 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:06.784 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:06.784 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.784 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.784 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.784 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.784 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.784 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:06.785 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:06.785 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.785 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.785 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.785 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.785 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.785 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.785 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.785 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.785 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.785 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.785 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.785 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.785 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:06.786 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:06.786 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:06.786 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:06.786 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.786 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:06.786 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.786 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.786 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.786 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.786 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.786 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.786 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.786 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:06.787 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:06.787 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.787 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:06.787 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "pg_autoscaler", 2024-05-21T22:06:06.787 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:06.787 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:06.787 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:06.787 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:06.787 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:06.787 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.787 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.787 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.787 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:06.787 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.788 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.788 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:06.788 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:06.788 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:06.788 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:06.788 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:06.788 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:06.788 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:06.788 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:06.788 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.788 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.788 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.788 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.788 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.789 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:06.789 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:06.789 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.789 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.789 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.789 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.789 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.789 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.789 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.789 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.789 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.789 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.789 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.790 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.790 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:06.790 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:06.790 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.790 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.790 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.790 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:06.790 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.790 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.790 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:06.790 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:06.790 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:06.791 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:06.791 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:06.791 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:06.791 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:06.791 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:06.791 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.791 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.791 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.791 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.791 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.791 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:06.791 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:06.791 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.792 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.792 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.792 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.792 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.792 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.792 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.792 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.792 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.792 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.792 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.792 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.792 INFO:teuthology.orchestra.run.smithi151.stdout: "sleep_interval": { 2024-05-21T22:06:06.792 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sleep_interval", 2024-05-21T22:06:06.793 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-05-21T22:06:06.793 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.793 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.793 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "60", 2024-05-21T22:06:06.793 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.793 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.793 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.793 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.793 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.793 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.793 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.793 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.793 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:06.794 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:06.794 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:06.794 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:06.794 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.794 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:06.794 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.794 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.794 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.794 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.794 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.794 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.794 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.795 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.795 INFO:teuthology.orchestra.run.smithi151.stdout: "threshold": { 2024-05-21T22:06:06.795 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "threshold", 2024-05-21T22:06:06.795 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "float", 2024-05-21T22:06:06.795 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.795 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.795 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "3.0", 2024-05-21T22:06:06.795 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "1.0", 2024-05-21T22:06:06.795 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.795 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.795 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "scaling threshold", 2024-05-21T22:06:06.796 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-21T22:06:06.796 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.796 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.796 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:06.796 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:06.796 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.796 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:06.796 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "progress", 2024-05-21T22:06:06.796 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:06.796 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:06.796 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:06.796 INFO:teuthology.orchestra.run.smithi151.stdout: "allow_pg_recovery_event": { 2024-05-21T22:06:06.796 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "allow_pg_recovery_event", 2024-05-21T22:06:06.797 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.797 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.797 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.797 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.797 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.797 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.797 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.797 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-21T22:06:06.797 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.797 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.797 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.797 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.798 INFO:teuthology.orchestra.run.smithi151.stdout: "enabled": { 2024-05-21T22:06:06.798 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "enabled", 2024-05-21T22:06:06.798 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.798 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.798 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.798 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:06.798 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.798 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.798 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.798 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.798 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.798 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.799 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.799 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.799 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:06.799 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:06.799 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.799 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.799 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.799 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:06.799 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.799 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.799 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:06.799 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:06.799 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:06.799 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:06.800 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:06.800 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:06.800 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:06.800 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:06.800 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.800 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.800 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.800 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.800 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.800 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:06.800 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:06.800 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.800 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.800 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.800 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.800 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.801 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.801 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.801 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.801 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.801 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.801 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.801 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.801 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:06.801 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:06.801 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.801 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.801 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.801 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:06.801 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.802 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.802 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:06.802 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:06.802 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:06.802 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:06.802 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:06.802 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:06.802 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:06.802 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:06.802 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.802 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.802 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.802 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.802 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.802 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:06.803 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:06.803 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.803 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.803 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.803 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.803 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.803 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.803 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.803 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.803 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.803 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.803 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.804 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.804 INFO:teuthology.orchestra.run.smithi151.stdout: "max_completed_events": { 2024-05-21T22:06:06.804 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "max_completed_events", 2024-05-21T22:06:06.804 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:06.804 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.804 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.804 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "50", 2024-05-21T22:06:06.804 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.804 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.804 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.804 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "number of past completed events to remember", 2024-05-21T22:06:06.804 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.804 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.804 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.805 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.805 INFO:teuthology.orchestra.run.smithi151.stdout: "sleep_interval": { 2024-05-21T22:06:06.805 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sleep_interval", 2024-05-21T22:06:06.805 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-05-21T22:06:06.805 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.805 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.805 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "5", 2024-05-21T22:06:06.805 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.805 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.805 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.805 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "how long the module is going to sleep", 2024-05-21T22:06:06.805 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.805 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.805 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.806 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.806 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:06.806 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:06.806 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:06.806 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:06.806 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.806 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:06.806 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.806 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.806 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.806 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.806 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.806 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.806 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.806 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:06.807 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:06.807 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.807 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:06.807 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "prometheus", 2024-05-21T22:06:06.807 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:06.807 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:06.807 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:06.807 INFO:teuthology.orchestra.run.smithi151.stdout: "cache": { 2024-05-21T22:06:06.807 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "cache", 2024-05-21T22:06:06.807 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.807 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.807 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.807 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:06.807 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.807 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.808 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.808 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.808 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.808 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.808 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.808 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.808 INFO:teuthology.orchestra.run.smithi151.stdout: "exclude_perf_counters": { 2024-05-21T22:06:06.808 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "exclude_perf_counters", 2024-05-21T22:06:06.808 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.808 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.808 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.808 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:06.808 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.809 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.809 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.809 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-21T22:06:06.809 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-21T22:06:06.809 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.809 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.809 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.809 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:06.809 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:06.809 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.809 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.809 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.809 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:06.809 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.810 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.810 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:06.810 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:06.810 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:06.810 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:06.810 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:06.813 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:06.813 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:06.813 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:06.814 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.814 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.814 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.814 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.814 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.814 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:06.814 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:06.814 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.814 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.814 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.814 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.814 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.814 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.815 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.815 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.815 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.815 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.815 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.815 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.815 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:06.815 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:06.815 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.815 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.815 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.815 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:06.815 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.815 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.816 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:06.816 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:06.816 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:06.816 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:06.816 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:06.816 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:06.816 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:06.816 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:06.816 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.816 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.816 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.816 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.816 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.816 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:06.817 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:06.817 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.817 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.817 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.817 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.817 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.817 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.817 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.817 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.817 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.817 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.817 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.817 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.818 INFO:teuthology.orchestra.run.smithi151.stdout: "rbd_stats_pools": { 2024-05-21T22:06:06.818 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "rbd_stats_pools", 2024-05-21T22:06:06.818 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.818 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.818 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.818 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:06.818 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.818 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.818 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.818 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.818 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.818 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.818 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.819 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.819 INFO:teuthology.orchestra.run.smithi151.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-21T22:06:06.819 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-21T22:06:06.819 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:06.819 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.819 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.819 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "300", 2024-05-21T22:06:06.819 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.819 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.819 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.819 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.819 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.819 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.819 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.820 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.820 INFO:teuthology.orchestra.run.smithi151.stdout: "scrape_interval": { 2024-05-21T22:06:06.820 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "scrape_interval", 2024-05-21T22:06:06.820 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "float", 2024-05-21T22:06:06.820 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.820 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.820 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "15.0", 2024-05-21T22:06:06.820 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.820 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.820 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.820 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.820 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.820 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.820 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.820 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.821 INFO:teuthology.orchestra.run.smithi151.stdout: "server_addr": { 2024-05-21T22:06:06.821 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "server_addr", 2024-05-21T22:06:06.821 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.821 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.821 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.821 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "::", 2024-05-21T22:06:06.821 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.821 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.821 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.821 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-21T22:06:06.821 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.821 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.821 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.821 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.822 INFO:teuthology.orchestra.run.smithi151.stdout: "server_port": { 2024-05-21T22:06:06.822 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "server_port", 2024-05-21T22:06:06.822 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:06.822 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.822 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.822 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "9283", 2024-05-21T22:06:06.822 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.822 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.822 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.822 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-21T22:06:06.822 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.822 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.822 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.823 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.823 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:06.823 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:06.823 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:06.823 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:06.823 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.823 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:06.823 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.823 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.823 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.823 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.823 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.823 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.823 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.823 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.824 INFO:teuthology.orchestra.run.smithi151.stdout: "stale_cache_strategy": { 2024-05-21T22:06:06.824 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "stale_cache_strategy", 2024-05-21T22:06:06.824 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.824 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.824 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.824 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "log", 2024-05-21T22:06:06.824 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.824 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.824 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.824 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.824 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.824 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.824 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.824 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.824 INFO:teuthology.orchestra.run.smithi151.stdout: "standby_behaviour": { 2024-05-21T22:06:06.825 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "standby_behaviour", 2024-05-21T22:06:06.825 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.825 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.825 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.825 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "default", 2024-05-21T22:06:06.825 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.825 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.825 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:06.825 INFO:teuthology.orchestra.run.smithi151.stdout: "default", 2024-05-21T22:06:06.825 INFO:teuthology.orchestra.run.smithi151.stdout: "error" 2024-05-21T22:06:06.825 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:06.826 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.826 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.826 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.826 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.826 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.826 INFO:teuthology.orchestra.run.smithi151.stdout: "standby_error_status_code": { 2024-05-21T22:06:06.826 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "standby_error_status_code", 2024-05-21T22:06:06.826 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:06.826 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.826 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.826 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "500", 2024-05-21T22:06:06.826 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "400", 2024-05-21T22:06:06.826 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "599", 2024-05-21T22:06:06.826 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.827 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.827 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.827 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.827 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.827 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:06.827 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:06.827 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.827 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:06.827 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "rbd_support", 2024-05-21T22:06:06.827 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:06.827 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:06.827 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:06.827 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:06.828 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:06.828 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.828 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.828 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.828 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:06.828 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.828 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.828 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:06.828 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:06.828 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:06.828 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:06.828 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:06.828 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:06.829 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:06.829 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:06.829 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.829 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.829 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.829 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.829 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.829 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:06.829 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:06.829 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.829 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.829 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.829 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.829 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.830 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.830 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.830 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.830 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.830 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.830 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.830 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.830 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:06.830 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:06.830 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.830 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.830 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.830 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:06.831 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.831 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.831 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:06.831 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:06.831 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:06.831 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:06.831 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:06.831 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:06.831 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:06.831 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:06.831 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.831 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.831 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.831 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.831 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.832 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:06.832 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:06.832 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.832 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.832 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.832 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.832 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.832 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.832 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.832 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.832 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.832 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.832 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.833 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.833 INFO:teuthology.orchestra.run.smithi151.stdout: "max_concurrent_snap_create": { 2024-05-21T22:06:06.833 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "max_concurrent_snap_create", 2024-05-21T22:06:06.833 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:06.833 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.833 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.833 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "10", 2024-05-21T22:06:06.833 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.833 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.833 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.833 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.833 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.834 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.834 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.834 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.834 INFO:teuthology.orchestra.run.smithi151.stdout: "mirror_snapshot_schedule": { 2024-05-21T22:06:06.834 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "mirror_snapshot_schedule", 2024-05-21T22:06:06.834 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.834 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.834 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.834 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:06.834 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.834 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.834 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.834 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.834 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.835 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.835 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.835 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.835 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:06.835 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:06.835 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:06.835 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:06.835 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.835 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:06.835 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.835 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.835 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.835 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.836 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.836 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.836 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.836 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.836 INFO:teuthology.orchestra.run.smithi151.stdout: "trash_purge_schedule": { 2024-05-21T22:06:06.836 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "trash_purge_schedule", 2024-05-21T22:06:06.836 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.836 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.836 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.836 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:06.836 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.836 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.836 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.837 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.837 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.837 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.837 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.837 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:06.837 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:06.837 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.837 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:06.837 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "restful", 2024-05-21T22:06:06.837 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:06.837 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:06.837 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:06.837 INFO:teuthology.orchestra.run.smithi151.stdout: "enable_auth": { 2024-05-21T22:06:06.837 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "enable_auth", 2024-05-21T22:06:06.838 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.838 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.838 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.838 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:06.838 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.838 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.838 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.838 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.838 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.838 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.838 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.838 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.838 INFO:teuthology.orchestra.run.smithi151.stdout: "key_file": { 2024-05-21T22:06:06.839 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "key_file", 2024-05-21T22:06:06.839 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.839 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.839 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.839 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:06.839 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.839 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.839 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.839 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.839 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.839 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.839 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.839 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.839 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:06.840 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:06.840 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.840 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.840 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.840 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:06.840 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.840 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.840 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:06.840 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:06.840 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:06.840 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:06.840 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:06.840 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:06.840 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:06.840 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:06.841 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.841 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.841 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.841 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.841 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.841 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:06.841 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:06.841 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.841 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.841 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.841 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.841 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.841 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.841 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.842 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.842 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.842 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.842 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.842 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.842 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:06.842 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:06.842 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.842 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.842 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.842 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:06.842 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.843 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.843 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:06.843 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:06.843 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:06.843 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:06.843 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:06.843 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:06.843 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:06.843 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:06.843 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.843 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.843 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.843 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.844 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.844 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:06.844 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:06.844 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.844 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.844 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.844 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.844 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.844 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.844 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.844 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.844 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.844 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.845 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.845 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.845 INFO:teuthology.orchestra.run.smithi151.stdout: "server_addr": { 2024-05-21T22:06:06.845 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "server_addr", 2024-05-21T22:06:06.845 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.845 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.845 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.845 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:06.845 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.845 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.845 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.845 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.845 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.845 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.846 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.846 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.846 INFO:teuthology.orchestra.run.smithi151.stdout: "server_port": { 2024-05-21T22:06:06.846 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "server_port", 2024-05-21T22:06:06.846 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.846 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.846 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.846 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:06.846 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.846 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.846 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.846 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.846 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.846 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.846 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.847 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.847 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:06.847 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:06.847 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:06.847 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:06.847 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.847 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:06.847 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.847 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.847 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.847 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.847 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.847 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.847 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.847 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:06.848 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:06.848 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.848 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:06.848 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "rgw", 2024-05-21T22:06:06.848 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:06.848 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:06.848 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:06.848 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:06.848 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:06.848 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.848 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.848 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.849 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:06.849 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.849 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.849 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:06.849 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:06.849 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:06.849 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:06.849 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:06.849 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:06.849 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:06.849 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:06.849 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.849 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.850 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.850 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.850 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.850 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:06.850 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:06.850 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.850 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.850 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.850 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.850 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.850 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.850 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.851 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.851 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.851 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.851 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.851 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.851 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:06.851 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:06.851 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.851 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.851 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.851 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:06.851 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.852 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.852 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:06.852 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:06.852 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:06.852 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:06.852 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:06.852 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:06.852 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:06.852 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:06.852 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.852 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.852 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.852 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.853 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.853 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:06.853 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:06.853 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.853 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.853 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.853 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.853 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.853 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.853 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.853 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.853 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.853 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.853 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.854 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.854 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:06.854 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:06.854 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:06.854 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:06.854 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.854 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:06.854 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.854 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.854 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.854 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.855 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.855 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.855 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.855 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:06.855 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:06.855 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.855 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:06.855 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "rook", 2024-05-21T22:06:06.855 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:06.855 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:06.855 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:06.855 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:06.855 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:06.855 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.856 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.856 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.856 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:06.856 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.856 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.856 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:06.856 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:06.856 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:06.856 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:06.856 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:06.856 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:06.856 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:06.856 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:06.857 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.857 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.857 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.857 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.857 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.857 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:06.857 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:06.857 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.857 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.857 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.857 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.857 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.858 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.858 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.858 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.858 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.858 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.858 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.858 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.858 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:06.858 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:06.858 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.858 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.858 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.859 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:06.859 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.859 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.859 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:06.859 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:06.859 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:06.859 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:06.859 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:06.859 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:06.859 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:06.859 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:06.859 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.859 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.859 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.860 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.860 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.860 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:06.860 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:06.860 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.860 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.860 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.860 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.860 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.860 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.860 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.860 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.860 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.860 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.861 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.861 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.861 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:06.861 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:06.861 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:06.861 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:06.861 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.861 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:06.861 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.861 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.861 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.861 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.861 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.862 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.862 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.862 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.862 INFO:teuthology.orchestra.run.smithi151.stdout: "storage_class": { 2024-05-21T22:06:06.862 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "storage_class", 2024-05-21T22:06:06.862 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.862 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.862 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.862 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "local", 2024-05-21T22:06:06.862 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.862 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.862 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.863 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-21T22:06:06.863 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.863 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.863 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.863 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:06.863 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:06.863 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.863 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:06.863 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "selftest", 2024-05-21T22:06:06.863 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:06.863 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:06.863 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:06.863 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:06.863 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:06.863 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.864 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.864 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.864 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:06.864 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.864 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.864 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:06.864 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:06.864 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:06.864 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:06.864 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:06.864 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:06.864 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:06.864 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:06.865 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.865 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.865 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.865 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.865 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.865 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:06.865 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:06.865 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.865 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.865 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.865 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.865 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.865 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.865 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.866 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.866 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.866 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.866 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.866 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.866 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:06.866 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:06.866 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.866 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.866 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.866 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:06.866 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.867 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.867 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:06.867 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:06.867 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:06.867 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:06.867 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:06.867 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:06.867 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:06.867 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:06.867 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.867 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.867 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.867 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.868 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.868 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:06.868 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:06.868 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.868 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.868 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.868 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.868 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.868 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.868 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.868 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.868 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.868 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.868 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.869 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.869 INFO:teuthology.orchestra.run.smithi151.stdout: "roption1": { 2024-05-21T22:06:06.869 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "roption1", 2024-05-21T22:06:06.869 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.869 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.869 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.869 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:06.869 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.869 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.869 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.869 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.869 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.869 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.870 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.870 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.870 INFO:teuthology.orchestra.run.smithi151.stdout: "roption2": { 2024-05-21T22:06:06.870 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "roption2", 2024-05-21T22:06:06.870 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.870 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.870 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.870 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "xyz", 2024-05-21T22:06:06.870 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.870 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.870 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.871 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.871 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.871 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.871 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.871 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.871 INFO:teuthology.orchestra.run.smithi151.stdout: "rwoption1": { 2024-05-21T22:06:06.871 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "rwoption1", 2024-05-21T22:06:06.871 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.871 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.871 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.871 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:06.871 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.871 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.872 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.872 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.872 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.872 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.872 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.872 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.872 INFO:teuthology.orchestra.run.smithi151.stdout: "rwoption2": { 2024-05-21T22:06:06.872 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "rwoption2", 2024-05-21T22:06:06.872 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:06.872 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.872 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.872 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:06.872 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.872 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.873 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.873 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.873 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.873 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.873 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.873 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.873 INFO:teuthology.orchestra.run.smithi151.stdout: "rwoption3": { 2024-05-21T22:06:06.873 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "rwoption3", 2024-05-21T22:06:06.873 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "float", 2024-05-21T22:06:06.873 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.873 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.873 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:06.873 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.873 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.874 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.874 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.874 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.874 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.874 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.874 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.874 INFO:teuthology.orchestra.run.smithi151.stdout: "rwoption4": { 2024-05-21T22:06:06.874 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "rwoption4", 2024-05-21T22:06:06.874 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.874 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.874 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.874 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:06.875 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.875 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.875 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.875 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.875 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.875 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.875 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.875 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.875 INFO:teuthology.orchestra.run.smithi151.stdout: "rwoption5": { 2024-05-21T22:06:06.875 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "rwoption5", 2024-05-21T22:06:06.875 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.875 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.875 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.876 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:06.876 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.876 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.876 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.876 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.876 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.876 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.876 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.876 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.876 INFO:teuthology.orchestra.run.smithi151.stdout: "rwoption6": { 2024-05-21T22:06:06.876 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "rwoption6", 2024-05-21T22:06:06.876 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.877 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.877 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.877 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:06.877 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.877 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.877 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.877 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.877 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.877 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.877 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.877 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.877 INFO:teuthology.orchestra.run.smithi151.stdout: "rwoption7": { 2024-05-21T22:06:06.877 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "rwoption7", 2024-05-21T22:06:06.877 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:06.878 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.878 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.878 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:06.878 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "1", 2024-05-21T22:06:06.878 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "42", 2024-05-21T22:06:06.878 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.878 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.878 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.878 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.878 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.878 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.878 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:06.879 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:06.879 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:06.879 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:06.879 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.879 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:06.879 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.879 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.879 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.879 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.879 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.879 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.879 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.879 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.880 INFO:teuthology.orchestra.run.smithi151.stdout: "testkey": { 2024-05-21T22:06:06.880 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "testkey", 2024-05-21T22:06:06.880 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.880 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.880 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.880 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:06.880 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.880 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.880 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.880 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.880 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.880 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.880 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.880 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.881 INFO:teuthology.orchestra.run.smithi151.stdout: "testlkey": { 2024-05-21T22:06:06.881 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "testlkey", 2024-05-21T22:06:06.881 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.881 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.881 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.881 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:06.881 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.881 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.881 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.881 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.881 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.881 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.881 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.881 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.881 INFO:teuthology.orchestra.run.smithi151.stdout: "testnewline": { 2024-05-21T22:06:06.882 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "testnewline", 2024-05-21T22:06:06.882 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.882 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.882 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.882 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:06.882 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.882 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.882 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.882 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.882 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.882 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.882 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.882 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:06.883 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:06.883 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.883 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:06.883 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "smb", 2024-05-21T22:06:06.883 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:06.883 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:06.883 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:06.883 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:06.883 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:06.883 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.883 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.883 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.883 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:06.883 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.883 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.884 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:06.884 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:06.884 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:06.884 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:06.884 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:06.884 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:06.884 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:06.884 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:06.884 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.884 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.884 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.884 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.884 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.884 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:06.884 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:06.885 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.885 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.885 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.885 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.885 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.885 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.885 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.885 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.885 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.885 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.885 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.885 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.885 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:06.885 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:06.885 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.886 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.886 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.886 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:06.886 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.886 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.886 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:06.886 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:06.886 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:06.886 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:06.886 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:06.886 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:06.886 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:06.886 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:06.887 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.887 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.887 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.887 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.887 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.887 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:06.887 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:06.887 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.887 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.887 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.887 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.887 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.888 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.888 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.888 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.888 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.888 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.888 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.888 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.888 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:06.888 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:06.888 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:06.888 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:06.888 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.888 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:06.888 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.889 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.889 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.889 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.889 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.889 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.889 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.889 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.889 INFO:teuthology.orchestra.run.smithi151.stdout: "update_orchestration": { 2024-05-21T22:06:06.889 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "update_orchestration", 2024-05-21T22:06:06.889 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.889 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.889 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.889 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:06.889 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.889 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.890 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.890 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-21T22:06:06.890 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.890 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.890 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.890 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:06.890 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:06.890 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.890 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:06.890 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "snap_schedule", 2024-05-21T22:06:06.890 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:06.890 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:06.890 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:06.891 INFO:teuthology.orchestra.run.smithi151.stdout: "allow_m_granularity": { 2024-05-21T22:06:06.891 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "allow_m_granularity", 2024-05-21T22:06:06.891 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.891 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.891 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.891 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.891 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.891 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.891 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.891 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "allow minute scheduled snapshots", 2024-05-21T22:06:06.891 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.891 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.891 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.891 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.892 INFO:teuthology.orchestra.run.smithi151.stdout: "dump_on_update": { 2024-05-21T22:06:06.892 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "dump_on_update", 2024-05-21T22:06:06.892 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.892 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.892 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.892 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.892 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.892 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.892 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.892 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "dump database to debug log on update", 2024-05-21T22:06:06.892 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.892 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.892 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.892 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.893 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:06.893 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:06.893 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.893 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.893 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.893 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:06.893 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.893 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.893 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:06.893 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:06.893 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:06.893 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:06.893 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:06.893 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:06.894 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:06.894 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:06.894 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.894 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.894 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.894 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.894 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.894 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:06.894 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:06.894 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.894 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.894 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.894 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.894 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.894 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.895 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.895 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.895 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.895 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.895 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.895 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.895 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:06.895 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:06.895 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.895 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.895 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.895 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:06.895 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.895 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.895 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:06.896 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:06.896 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:06.896 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:06.896 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:06.896 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:06.896 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:06.896 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:06.896 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.896 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.896 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.896 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.896 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.896 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:06.896 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:06.896 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.897 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.897 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.897 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.897 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.897 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.897 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.897 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.897 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.897 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.897 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.897 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.897 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:06.897 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:06.897 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:06.897 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:06.898 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.898 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:06.898 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.898 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.898 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.898 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.898 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.898 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.898 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.898 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:06.898 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:06.898 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.898 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:06.899 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "stats", 2024-05-21T22:06:06.899 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:06.899 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:06.899 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:06.899 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:06.899 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:06.899 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.899 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.899 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.899 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:06.899 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.899 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.899 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:06.899 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:06.900 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:06.900 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:06.900 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:06.900 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:06.900 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:06.900 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:06.900 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.900 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.900 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.900 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.901 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.901 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:06.901 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:06.901 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.901 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.901 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.901 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.901 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.901 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.901 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.901 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.901 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.901 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.901 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.902 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.902 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:06.902 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:06.902 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.902 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.902 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.902 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:06.902 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.902 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.902 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:06.902 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:06.902 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:06.903 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:06.903 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:06.903 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:06.903 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:06.903 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:06.903 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.903 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.903 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.903 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.903 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.903 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:06.903 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:06.903 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.903 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.904 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.904 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.904 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.904 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.904 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.904 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.904 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.904 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.904 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.904 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.904 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:06.904 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:06.904 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:06.905 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:06.905 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.905 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:06.905 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.905 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.905 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.905 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.905 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.905 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.905 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.905 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:06.905 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:06.905 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.905 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:06.905 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "status", 2024-05-21T22:06:06.906 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:06.906 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:06.906 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:06.906 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:06.906 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:06.906 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.906 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.906 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.906 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:06.906 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.906 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.906 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:06.907 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:06.907 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:06.907 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:06.907 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:06.907 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:06.907 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:06.907 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:06.907 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.907 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.907 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.907 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.907 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.907 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:06.907 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:06.908 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.908 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.908 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.908 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.908 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.908 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.908 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.908 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.908 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.908 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.908 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.908 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.908 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:06.909 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:06.909 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.909 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.909 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.909 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:06.909 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.909 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.909 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:06.909 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:06.909 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:06.909 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:06.909 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:06.909 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:06.910 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:06.910 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:06.910 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.910 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.910 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.910 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.910 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.910 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:06.910 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:06.910 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.910 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.910 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.910 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.911 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.911 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.911 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.911 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.911 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.911 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.911 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.911 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.911 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:06.911 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:06.911 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:06.911 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:06.911 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.911 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:06.912 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.912 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.912 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.912 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.912 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.912 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.912 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.912 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:06.912 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:06.912 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.912 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:06.912 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "telegraf", 2024-05-21T22:06:06.912 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:06.912 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:06.913 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:06.913 INFO:teuthology.orchestra.run.smithi151.stdout: "address": { 2024-05-21T22:06:06.913 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "address", 2024-05-21T22:06:06.913 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.913 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.913 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.913 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-21T22:06:06.913 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.913 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.913 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.913 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.913 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.913 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.913 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.913 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.914 INFO:teuthology.orchestra.run.smithi151.stdout: "interval": { 2024-05-21T22:06:06.914 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "interval", 2024-05-21T22:06:06.914 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-05-21T22:06:06.914 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.914 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.914 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "15", 2024-05-21T22:06:06.914 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.914 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.914 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.914 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.914 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.914 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.915 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.915 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.915 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:06.915 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:06.915 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.915 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.915 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.915 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:06.915 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.915 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.915 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:06.915 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:06.915 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:06.915 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:06.915 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:06.916 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:06.916 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:06.916 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:06.916 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.916 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.916 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.916 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.916 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.916 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:06.916 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:06.916 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.916 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.917 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.917 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.917 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.917 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.917 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.917 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.917 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.917 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.917 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.917 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.917 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:06.917 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:06.917 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.917 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.918 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.918 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:06.918 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.918 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.918 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:06.918 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:06.918 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:06.918 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:06.918 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:06.918 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:06.918 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:06.918 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:06.918 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.918 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.919 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.919 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.919 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.919 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:06.919 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:06.919 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.919 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.919 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.919 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.919 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.920 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.920 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.920 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.920 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.920 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.920 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.920 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.920 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:06.920 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:06.920 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:06.920 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:06.920 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.920 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:06.921 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.921 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.921 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.921 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.921 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.921 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.921 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.921 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:06.921 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:06.921 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.921 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:06.921 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "telemetry", 2024-05-21T22:06:06.922 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:06.922 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:06.922 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:06.922 INFO:teuthology.orchestra.run.smithi151.stdout: "channel_basic": { 2024-05-21T22:06:06.922 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "channel_basic", 2024-05-21T22:06:06.922 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.922 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.922 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.922 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:06.922 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.922 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.923 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.923 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-21T22:06:06.923 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.923 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.923 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.923 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.923 INFO:teuthology.orchestra.run.smithi151.stdout: "channel_crash": { 2024-05-21T22:06:06.923 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "channel_crash", 2024-05-21T22:06:06.923 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.923 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.923 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.923 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:06.924 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.924 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.924 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.924 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-21T22:06:06.924 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.924 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.924 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.924 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.924 INFO:teuthology.orchestra.run.smithi151.stdout: "channel_device": { 2024-05-21T22:06:06.924 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "channel_device", 2024-05-21T22:06:06.924 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.924 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.924 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.925 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:06.925 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.925 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.925 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.925 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-21T22:06:06.925 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.925 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.925 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.925 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.925 INFO:teuthology.orchestra.run.smithi151.stdout: "channel_ident": { 2024-05-21T22:06:06.925 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "channel_ident", 2024-05-21T22:06:06.925 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.926 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.926 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.926 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.926 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.926 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.926 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.926 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-21T22:06:06.926 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.926 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.926 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.926 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.926 INFO:teuthology.orchestra.run.smithi151.stdout: "channel_perf": { 2024-05-21T22:06:06.927 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "channel_perf", 2024-05-21T22:06:06.927 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.927 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.927 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.927 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.927 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.927 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.927 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.927 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-21T22:06:06.927 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.927 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.927 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.928 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.928 INFO:teuthology.orchestra.run.smithi151.stdout: "contact": { 2024-05-21T22:06:06.928 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "contact", 2024-05-21T22:06:06.928 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.928 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.928 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.928 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:06.928 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.928 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.928 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.928 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.928 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.928 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.928 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.929 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.929 INFO:teuthology.orchestra.run.smithi151.stdout: "description": { 2024-05-21T22:06:06.929 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "description", 2024-05-21T22:06:06.929 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.929 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.929 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.929 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:06.929 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.929 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.929 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.929 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.929 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.930 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.930 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.930 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.930 INFO:teuthology.orchestra.run.smithi151.stdout: "device_url": { 2024-05-21T22:06:06.930 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "device_url", 2024-05-21T22:06:06.930 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.930 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.930 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.930 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-21T22:06:06.930 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.930 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.930 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.930 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.931 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.931 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.931 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.931 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.931 INFO:teuthology.orchestra.run.smithi151.stdout: "enabled": { 2024-05-21T22:06:06.931 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "enabled", 2024-05-21T22:06:06.931 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.931 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.931 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.931 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.931 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.931 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.931 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.932 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.932 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.932 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.932 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.932 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.932 INFO:teuthology.orchestra.run.smithi151.stdout: "interval": { 2024-05-21T22:06:06.932 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "interval", 2024-05-21T22:06:06.932 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:06.932 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.932 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.932 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "24", 2024-05-21T22:06:06.932 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "8", 2024-05-21T22:06:06.932 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.932 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.932 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.933 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.933 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.933 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.933 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.933 INFO:teuthology.orchestra.run.smithi151.stdout: "last_opt_revision": { 2024-05-21T22:06:06.933 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "last_opt_revision", 2024-05-21T22:06:06.933 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:06.933 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.933 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.933 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "1", 2024-05-21T22:06:06.933 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.933 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.933 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.934 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.934 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.934 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.934 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.934 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.934 INFO:teuthology.orchestra.run.smithi151.stdout: "leaderboard": { 2024-05-21T22:06:06.934 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "leaderboard", 2024-05-21T22:06:06.934 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.934 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.934 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.934 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.934 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.934 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.935 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.935 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.935 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.935 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.935 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.935 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.935 INFO:teuthology.orchestra.run.smithi151.stdout: "leaderboard_description": { 2024-05-21T22:06:06.935 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "leaderboard_description", 2024-05-21T22:06:06.935 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.935 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.935 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.935 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:06.935 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.936 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.936 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.936 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.936 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.936 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.936 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.936 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.936 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:06.936 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:06.936 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.936 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.936 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.936 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:06.936 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.937 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.937 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:06.937 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:06.937 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:06.937 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:06.937 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:06.937 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:06.937 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:06.937 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:06.937 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.937 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.937 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.937 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.937 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.937 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:06.938 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:06.938 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.938 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.938 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.938 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.938 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.938 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.938 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.938 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.938 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.938 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.938 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.938 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.938 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:06.939 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:06.939 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.939 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.939 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.939 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:06.939 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.939 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.939 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:06.939 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:06.939 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:06.939 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:06.939 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:06.939 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:06.939 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:06.939 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:06.940 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.940 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.940 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.940 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.940 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.940 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:06.940 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:06.940 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.940 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.940 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.940 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.940 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.940 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.940 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.940 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.941 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.941 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.941 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.941 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.941 INFO:teuthology.orchestra.run.smithi151.stdout: "organization": { 2024-05-21T22:06:06.941 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "organization", 2024-05-21T22:06:06.941 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.941 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.941 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.941 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:06.941 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.941 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.941 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.942 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.942 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.942 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.942 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.942 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.942 INFO:teuthology.orchestra.run.smithi151.stdout: "proxy": { 2024-05-21T22:06:06.942 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "proxy", 2024-05-21T22:06:06.942 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.942 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.942 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.942 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:06.942 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.943 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.943 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.943 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.943 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.943 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.943 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.943 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.943 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:06.943 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:06.943 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:06.943 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:06.943 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.944 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:06.944 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.944 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.944 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.944 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.944 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.944 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.944 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.944 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.944 INFO:teuthology.orchestra.run.smithi151.stdout: "url": { 2024-05-21T22:06:06.944 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "url", 2024-05-21T22:06:06.944 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.945 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.945 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.945 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-21T22:06:06.945 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.945 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.945 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.945 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.945 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.945 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.945 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.945 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:06.945 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:06.946 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.946 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:06.946 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "test_orchestrator", 2024-05-21T22:06:06.946 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:06.946 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:06.946 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:06.946 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:06.946 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:06.946 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.946 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.946 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.947 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:06.947 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.947 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.947 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:06.947 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:06.947 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:06.947 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:06.947 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:06.947 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:06.947 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:06.947 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:06.947 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.948 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.948 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.948 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.948 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.948 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:06.948 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:06.948 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.948 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.948 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.948 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.948 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.948 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.949 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.949 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.949 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.949 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.949 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.949 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.949 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:06.949 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:06.949 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.949 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.949 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.949 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:06.949 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.950 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.950 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:06.950 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:06.950 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:06.950 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:06.950 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:06.950 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:06.950 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:06.950 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:06.950 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.950 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.951 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.951 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.951 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.951 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:06.951 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:06.951 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.951 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.951 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.951 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.951 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.951 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.951 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.952 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.952 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.952 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.952 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.952 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.952 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:06.952 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:06.952 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:06.952 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:06.952 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.952 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:06.952 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.953 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.953 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.953 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.953 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.953 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.953 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.953 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:06.953 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:06.953 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.953 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:06.953 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "volumes", 2024-05-21T22:06:06.953 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:06.953 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:06.953 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:06.954 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:06.954 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:06.954 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.954 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.954 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.954 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:06.954 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.954 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.954 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:06.954 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:06.954 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:06.954 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:06.955 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:06.955 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:06.955 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:06.955 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:06.955 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.955 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.955 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.955 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.955 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.955 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:06.955 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:06.956 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.956 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.956 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.956 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.956 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.956 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.956 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.956 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.956 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.956 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.956 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.956 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.956 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:06.956 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:06.957 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.957 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.957 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.957 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:06.957 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.957 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.957 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:06.957 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:06.957 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:06.957 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:06.957 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:06.957 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:06.957 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:06.957 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:06.958 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.958 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.958 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.958 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.958 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.958 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:06.958 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:06.958 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.958 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.958 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.958 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.958 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.959 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.959 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.959 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.959 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.959 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.959 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.959 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.959 INFO:teuthology.orchestra.run.smithi151.stdout: "max_concurrent_clones": { 2024-05-21T22:06:06.959 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "max_concurrent_clones", 2024-05-21T22:06:06.959 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:06.959 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.959 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.960 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "4", 2024-05-21T22:06:06.960 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.960 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.960 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.960 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-21T22:06:06.960 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.960 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.960 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.960 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.960 INFO:teuthology.orchestra.run.smithi151.stdout: "periodic_async_work": { 2024-05-21T22:06:06.960 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "periodic_async_work", 2024-05-21T22:06:06.960 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.961 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.961 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.961 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.961 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.961 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.961 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.961 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Periodically check for async work", 2024-05-21T22:06:06.961 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.961 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.961 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.961 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.961 INFO:teuthology.orchestra.run.smithi151.stdout: "snapshot_clone_delay": { 2024-05-21T22:06:06.961 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "snapshot_clone_delay", 2024-05-21T22:06:06.962 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:06.962 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.962 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.962 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:06.962 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.962 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.962 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.962 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-21T22:06:06.962 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.962 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.962 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.962 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.963 INFO:teuthology.orchestra.run.smithi151.stdout: "snapshot_clone_no_wait": { 2024-05-21T22:06:06.963 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "snapshot_clone_no_wait", 2024-05-21T22:06:06.963 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.963 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.963 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.963 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:06.963 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.963 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.963 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.963 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-21T22:06:06.963 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.963 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.963 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.963 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.964 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:06.964 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:06.964 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:06.964 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:06.964 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.964 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:06.964 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.964 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.964 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.964 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.964 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.964 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.964 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.965 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:06.965 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:06.965 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.965 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:06.965 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "zabbix", 2024-05-21T22:06:06.965 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:06.965 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:06.965 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:06.965 INFO:teuthology.orchestra.run.smithi151.stdout: "discovery_interval": { 2024-05-21T22:06:06.965 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "discovery_interval", 2024-05-21T22:06:06.965 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "uint", 2024-05-21T22:06:06.965 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.965 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.966 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "100", 2024-05-21T22:06:06.966 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.966 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.966 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.966 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.966 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.966 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.966 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.966 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.966 INFO:teuthology.orchestra.run.smithi151.stdout: "identifier": { 2024-05-21T22:06:06.966 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "identifier", 2024-05-21T22:06:06.966 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.966 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.966 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.967 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:06.967 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.967 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.967 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.967 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.967 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.967 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.967 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.967 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.967 INFO:teuthology.orchestra.run.smithi151.stdout: "interval": { 2024-05-21T22:06:06.967 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "interval", 2024-05-21T22:06:06.967 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-05-21T22:06:06.967 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.967 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.968 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "60", 2024-05-21T22:06:06.968 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.968 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.968 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.968 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.968 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.968 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.968 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.968 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.968 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:06.968 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:06.968 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.968 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.968 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.969 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:06.969 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.969 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.969 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:06.969 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:06.969 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:06.969 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:06.969 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:06.969 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:06.969 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:06.969 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:06.969 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.969 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.969 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.969 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.970 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.970 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:06.970 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:06.970 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.970 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.970 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.970 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.970 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.970 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.970 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.970 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.970 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.970 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.970 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.971 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.971 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:06.971 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:06.971 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.971 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.971 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.971 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:06.971 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.971 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.971 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:06.971 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:06.971 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:06.971 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:06.971 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:06.972 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:06.972 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:06.972 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:06.972 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.972 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.972 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.972 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.972 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.972 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:06.972 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:06.972 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.972 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.972 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.973 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.973 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.973 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.973 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.973 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.973 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.973 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.973 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.973 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.973 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:06.973 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:06.973 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:06.973 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:06.973 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.974 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:06.974 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.974 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.974 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.974 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.974 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.974 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.974 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.974 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.974 INFO:teuthology.orchestra.run.smithi151.stdout: "zabbix_host": { 2024-05-21T22:06:06.974 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "zabbix_host", 2024-05-21T22:06:06.974 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.975 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.975 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.975 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:06.975 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.975 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.975 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.975 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.975 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.975 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.975 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.975 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.975 INFO:teuthology.orchestra.run.smithi151.stdout: "zabbix_port": { 2024-05-21T22:06:06.975 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "zabbix_port", 2024-05-21T22:06:06.976 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:06.976 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.976 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.976 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "10051", 2024-05-21T22:06:06.976 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.976 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.976 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.976 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.976 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.976 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.976 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.976 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.977 INFO:teuthology.orchestra.run.smithi151.stdout: "zabbix_sender": { 2024-05-21T22:06:06.977 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "zabbix_sender", 2024-05-21T22:06:06.977 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.977 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.977 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:06.977 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-21T22:06:06.977 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.977 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.977 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.977 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.977 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.977 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.977 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.977 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:06.978 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:06.978 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:06.978 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-05-21T22:06:06.978 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.978 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:06.978 INFO:teuthology.orchestra.run.smithi151.stdout: "gid": 25142, 2024-05-21T22:06:06.978 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "y", 2024-05-21T22:06:06.978 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr_features": 4540701547738038271, 2024-05-21T22:06:06.978 INFO:teuthology.orchestra.run.smithi151.stdout: "available_modules": [ 2024-05-21T22:06:06.978 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:06.978 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "alerts", 2024-05-21T22:06:06.978 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:06.978 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:06.979 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:06.979 INFO:teuthology.orchestra.run.smithi151.stdout: "interval": { 2024-05-21T22:06:06.979 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "interval", 2024-05-21T22:06:06.979 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-05-21T22:06:06.979 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.979 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.979 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "60", 2024-05-21T22:06:06.979 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.979 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.979 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.979 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "How frequently to reexamine health status", 2024-05-21T22:06:06.979 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.979 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.980 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.980 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.980 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:06.980 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:06.980 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.980 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.980 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.980 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:06.980 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.980 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.980 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:06.980 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:06.980 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:06.980 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:06.981 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:06.981 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:06.981 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:06.981 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:06.981 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.981 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.981 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.981 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.981 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.981 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:06.981 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:06.981 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.981 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.981 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.982 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.982 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.982 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.982 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.982 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.982 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.982 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.982 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.982 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.982 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:06.982 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:06.982 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.982 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.983 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.983 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:06.983 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.983 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.983 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:06.983 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:06.983 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:06.983 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:06.983 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:06.983 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:06.983 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:06.983 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:06.984 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.984 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.984 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.984 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.984 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.984 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:06.984 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:06.984 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.984 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.984 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.984 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:06.984 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.984 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.984 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.984 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.984 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.985 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.985 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.985 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.985 INFO:teuthology.orchestra.run.smithi151.stdout: "smtp_destination": { 2024-05-21T22:06:06.985 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "smtp_destination", 2024-05-21T22:06:06.985 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.985 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.985 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.985 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:06.985 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.985 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.985 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.986 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Email address to send alerts to", 2024-05-21T22:06:06.986 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.986 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.986 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.986 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.986 INFO:teuthology.orchestra.run.smithi151.stdout: "smtp_from_name": { 2024-05-21T22:06:06.986 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "smtp_from_name", 2024-05-21T22:06:06.986 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.986 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.986 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.986 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "Ceph", 2024-05-21T22:06:06.986 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.987 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.987 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.987 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Email From: name", 2024-05-21T22:06:06.987 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.987 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.987 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.987 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.987 INFO:teuthology.orchestra.run.smithi151.stdout: "smtp_host": { 2024-05-21T22:06:06.987 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "smtp_host", 2024-05-21T22:06:06.987 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.987 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.987 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.988 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:06.988 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.988 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.988 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.988 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "SMTP server", 2024-05-21T22:06:06.988 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.988 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.988 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.988 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.988 INFO:teuthology.orchestra.run.smithi151.stdout: "smtp_password": { 2024-05-21T22:06:06.988 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "smtp_password", 2024-05-21T22:06:06.988 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.989 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.989 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.989 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:06.989 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.989 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.989 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.989 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Password to authenticate with", 2024-05-21T22:06:06.989 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.989 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.989 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.989 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.989 INFO:teuthology.orchestra.run.smithi151.stdout: "smtp_port": { 2024-05-21T22:06:06.990 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "smtp_port", 2024-05-21T22:06:06.990 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:06.990 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.990 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.990 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "465", 2024-05-21T22:06:06.990 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.990 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.990 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.990 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "SMTP port", 2024-05-21T22:06:06.990 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.990 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.990 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.990 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.991 INFO:teuthology.orchestra.run.smithi151.stdout: "smtp_sender": { 2024-05-21T22:06:06.991 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "smtp_sender", 2024-05-21T22:06:06.991 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.991 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.991 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.991 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:06.991 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.991 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.991 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.991 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "SMTP envelope sender", 2024-05-21T22:06:06.991 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.991 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.991 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.992 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.992 INFO:teuthology.orchestra.run.smithi151.stdout: "smtp_ssl": { 2024-05-21T22:06:06.992 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "smtp_ssl", 2024-05-21T22:06:06.992 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.992 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.992 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.992 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:06.992 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.992 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.992 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.992 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-21T22:06:06.992 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.992 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.993 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.993 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.993 INFO:teuthology.orchestra.run.smithi151.stdout: "smtp_user": { 2024-05-21T22:06:06.993 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "smtp_user", 2024-05-21T22:06:06.993 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.993 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.993 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.993 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:06.993 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.993 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.993 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.993 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "User to authenticate as", 2024-05-21T22:06:06.994 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.994 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.994 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.994 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.994 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:06.994 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:06.994 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:06.994 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:06.994 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.994 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:06.994 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.994 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.995 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.995 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:06.995 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.995 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.995 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.995 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:06.995 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:06.995 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.995 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:06.995 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "balancer", 2024-05-21T22:06:06.995 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:06.995 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:06.995 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:06.995 INFO:teuthology.orchestra.run.smithi151.stdout: "active": { 2024-05-21T22:06:06.995 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "active", 2024-05-21T22:06:06.996 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:06.996 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.996 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.996 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:06.996 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.996 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.996 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.996 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "automatically balance PGs across cluster", 2024-05-21T22:06:06.996 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.996 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.996 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.996 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.996 INFO:teuthology.orchestra.run.smithi151.stdout: "begin_time": { 2024-05-21T22:06:06.997 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "begin_time", 2024-05-21T22:06:06.997 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:06.997 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.997 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.997 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0000", 2024-05-21T22:06:06.997 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:06.997 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:06.997 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.997 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "beginning time of day to automatically balance", 2024-05-21T22:06:06.997 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-21T22:06:06.997 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.997 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.998 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.998 INFO:teuthology.orchestra.run.smithi151.stdout: "begin_weekday": { 2024-05-21T22:06:06.998 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "begin_weekday", 2024-05-21T22:06:06.998 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "uint", 2024-05-21T22:06:06.998 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.998 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.998 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:06.998 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "0", 2024-05-21T22:06:06.998 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "6", 2024-05-21T22:06:06.998 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.998 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-21T22:06:06.998 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-21T22:06:06.998 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.999 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:06.999 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:06.999 INFO:teuthology.orchestra.run.smithi151.stdout: "crush_compat_max_iterations": { 2024-05-21T22:06:06.999 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "crush_compat_max_iterations", 2024-05-21T22:06:06.999 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "uint", 2024-05-21T22:06:06.999 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:06.999 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:06.999 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "25", 2024-05-21T22:06:06.999 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "1", 2024-05-21T22:06:06.999 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "250", 2024-05-21T22:06:06.999 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:06.999 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-21T22:06:06.999 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:06.999 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:06.999 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.000 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.000 INFO:teuthology.orchestra.run.smithi151.stdout: "crush_compat_metrics": { 2024-05-21T22:06:07.000 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "crush_compat_metrics", 2024-05-21T22:06:07.000 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.000 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.000 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.000 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "pgs,objects,bytes", 2024-05-21T22:06:07.000 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.000 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.000 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.000 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-21T22:06:07.000 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-21T22:06:07.000 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.000 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.000 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.001 INFO:teuthology.orchestra.run.smithi151.stdout: "crush_compat_step": { 2024-05-21T22:06:07.001 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "crush_compat_step", 2024-05-21T22:06:07.001 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "float", 2024-05-21T22:06:07.001 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.001 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.001 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0.5", 2024-05-21T22:06:07.001 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "0.001", 2024-05-21T22:06:07.001 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "0.999", 2024-05-21T22:06:07.001 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.001 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "aggressiveness of optimization", 2024-05-21T22:06:07.001 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-21T22:06:07.001 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.001 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.001 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.001 INFO:teuthology.orchestra.run.smithi151.stdout: "end_time": { 2024-05-21T22:06:07.002 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "end_time", 2024-05-21T22:06:07.002 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.002 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.002 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.002 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "2359", 2024-05-21T22:06:07.002 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.002 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.002 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.002 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "ending time of day to automatically balance", 2024-05-21T22:06:07.002 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-21T22:06:07.002 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.002 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.003 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.003 INFO:teuthology.orchestra.run.smithi151.stdout: "end_weekday": { 2024-05-21T22:06:07.003 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "end_weekday", 2024-05-21T22:06:07.003 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "uint", 2024-05-21T22:06:07.003 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.003 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.003 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:07.003 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "0", 2024-05-21T22:06:07.003 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "6", 2024-05-21T22:06:07.003 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.003 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-21T22:06:07.003 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-21T22:06:07.003 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.003 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.003 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.004 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:07.004 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:07.004 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.004 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.004 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.004 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.004 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.004 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.004 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.004 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.004 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.004 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.004 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.004 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.005 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.005 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.005 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.005 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.005 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.005 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.005 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.005 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:07.005 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:07.005 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.005 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.005 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.005 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.005 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.006 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.006 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.006 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.006 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.006 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.006 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.006 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.006 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:07.006 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:07.006 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.006 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.006 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.006 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:07.007 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.007 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.007 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.007 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.007 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.007 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.007 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.007 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.007 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.007 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.007 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.007 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.008 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.008 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.008 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.008 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:07.008 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:07.008 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.008 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.008 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.008 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.008 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.008 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.008 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.008 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.009 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.009 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.009 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.009 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.009 INFO:teuthology.orchestra.run.smithi151.stdout: "min_score": { 2024-05-21T22:06:07.009 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "min_score", 2024-05-21T22:06:07.009 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "float", 2024-05-21T22:06:07.009 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.009 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.009 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:07.009 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.009 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.009 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.009 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-21T22:06:07.010 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.010 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.010 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.010 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.010 INFO:teuthology.orchestra.run.smithi151.stdout: "mode": { 2024-05-21T22:06:07.010 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "mode", 2024-05-21T22:06:07.010 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.010 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.010 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.010 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "upmap", 2024-05-21T22:06:07.010 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.010 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.010 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.011 INFO:teuthology.orchestra.run.smithi151.stdout: "crush-compat", 2024-05-21T22:06:07.011 INFO:teuthology.orchestra.run.smithi151.stdout: "none", 2024-05-21T22:06:07.011 INFO:teuthology.orchestra.run.smithi151.stdout: "read", 2024-05-21T22:06:07.011 INFO:teuthology.orchestra.run.smithi151.stdout: "upmap", 2024-05-21T22:06:07.011 INFO:teuthology.orchestra.run.smithi151.stdout: "upmap-read" 2024-05-21T22:06:07.011 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.011 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Balancer mode", 2024-05-21T22:06:07.011 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.011 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.011 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.011 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.011 INFO:teuthology.orchestra.run.smithi151.stdout: "pool_ids": { 2024-05-21T22:06:07.011 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "pool_ids", 2024-05-21T22:06:07.011 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.011 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.012 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.012 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.012 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.012 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.012 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.012 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-21T22:06:07.012 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.012 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.012 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.012 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.012 INFO:teuthology.orchestra.run.smithi151.stdout: "sleep_interval": { 2024-05-21T22:06:07.012 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sleep_interval", 2024-05-21T22:06:07.012 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-05-21T22:06:07.012 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.012 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.013 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "60", 2024-05-21T22:06:07.013 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.013 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.013 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.013 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-21T22:06:07.013 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.013 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.013 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.013 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.013 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:07.013 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:07.013 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.013 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:07.013 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.013 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:07.014 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.014 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.014 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.014 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.014 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.014 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.014 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.014 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.014 INFO:teuthology.orchestra.run.smithi151.stdout: "upmap_max_deviation": { 2024-05-21T22:06:07.014 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "upmap_max_deviation", 2024-05-21T22:06:07.014 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.014 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.014 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.017 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "5", 2024-05-21T22:06:07.017 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "1", 2024-05-21T22:06:07.017 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.017 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.017 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-21T22:06:07.018 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-21T22:06:07.018 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.018 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.018 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.018 INFO:teuthology.orchestra.run.smithi151.stdout: "upmap_max_optimizations": { 2024-05-21T22:06:07.018 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "upmap_max_optimizations", 2024-05-21T22:06:07.018 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "uint", 2024-05-21T22:06:07.018 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.018 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.018 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "10", 2024-05-21T22:06:07.018 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.018 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.019 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.019 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-21T22:06:07.019 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.019 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.019 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.019 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.019 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.019 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.019 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:07.019 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "cephadm", 2024-05-21T22:06:07.019 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:07.019 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:07.019 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:07.019 INFO:teuthology.orchestra.run.smithi151.stdout: "agent_down_multiplier": { 2024-05-21T22:06:07.019 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "agent_down_multiplier", 2024-05-21T22:06:07.020 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "float", 2024-05-21T22:06:07.020 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.020 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.020 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "3.0", 2024-05-21T22:06:07.020 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.020 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.020 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.020 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-21T22:06:07.020 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.020 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.020 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.020 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.020 INFO:teuthology.orchestra.run.smithi151.stdout: "agent_refresh_rate": { 2024-05-21T22:06:07.021 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "agent_refresh_rate", 2024-05-21T22:06:07.021 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-05-21T22:06:07.021 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.021 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.021 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "20", 2024-05-21T22:06:07.021 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.021 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.021 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.021 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-21T22:06:07.021 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.021 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.021 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.021 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.021 INFO:teuthology.orchestra.run.smithi151.stdout: "agent_starting_port": { 2024-05-21T22:06:07.022 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "agent_starting_port", 2024-05-21T22:06:07.022 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.022 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.022 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.022 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "4721", 2024-05-21T22:06:07.022 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.022 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.022 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.022 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-21T22:06:07.022 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.022 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.022 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.023 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.023 INFO:teuthology.orchestra.run.smithi151.stdout: "allow_ptrace": { 2024-05-21T22:06:07.023 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "allow_ptrace", 2024-05-21T22:06:07.023 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.023 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.023 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.023 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.023 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.023 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.023 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.023 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-21T22:06:07.023 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-21T22:06:07.023 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.023 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.023 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.024 INFO:teuthology.orchestra.run.smithi151.stdout: "autotune_interval": { 2024-05-21T22:06:07.024 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "autotune_interval", 2024-05-21T22:06:07.024 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-05-21T22:06:07.024 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.024 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.024 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "600", 2024-05-21T22:06:07.024 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.024 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.024 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.024 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-21T22:06:07.024 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.024 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.024 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.024 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.025 INFO:teuthology.orchestra.run.smithi151.stdout: "autotune_memory_target_ratio": { 2024-05-21T22:06:07.025 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "autotune_memory_target_ratio", 2024-05-21T22:06:07.025 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "float", 2024-05-21T22:06:07.025 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.025 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.025 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0.7", 2024-05-21T22:06:07.025 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.025 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.025 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.025 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-21T22:06:07.025 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.025 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.025 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.025 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.026 INFO:teuthology.orchestra.run.smithi151.stdout: "cephadm_log_destination": { 2024-05-21T22:06:07.026 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "cephadm_log_destination", 2024-05-21T22:06:07.026 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.026 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.026 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.026 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.026 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.026 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.026 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.026 INFO:teuthology.orchestra.run.smithi151.stdout: "file", 2024-05-21T22:06:07.026 INFO:teuthology.orchestra.run.smithi151.stdout: "file,syslog", 2024-05-21T22:06:07.026 INFO:teuthology.orchestra.run.smithi151.stdout: "syslog" 2024-05-21T22:06:07.027 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.027 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-21T22:06:07.027 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.027 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.027 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.027 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.027 INFO:teuthology.orchestra.run.smithi151.stdout: "cgroups_split": { 2024-05-21T22:06:07.027 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "cgroups_split", 2024-05-21T22:06:07.027 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.027 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.027 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.027 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:07.027 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.027 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.028 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.028 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-21T22:06:07.028 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.028 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.028 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.028 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.028 INFO:teuthology.orchestra.run.smithi151.stdout: "config_checks_enabled": { 2024-05-21T22:06:07.028 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "config_checks_enabled", 2024-05-21T22:06:07.028 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.028 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.028 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.028 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.028 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.028 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.029 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.029 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-21T22:06:07.029 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.029 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.029 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.029 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.029 INFO:teuthology.orchestra.run.smithi151.stdout: "config_dashboard": { 2024-05-21T22:06:07.029 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "config_dashboard", 2024-05-21T22:06:07.029 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.029 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.029 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.029 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:07.029 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.029 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.030 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.030 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-21T22:06:07.030 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.030 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.030 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.030 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.030 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_alertmanager": { 2024-05-21T22:06:07.030 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_alertmanager", 2024-05-21T22:06:07.030 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.030 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.030 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.030 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-21T22:06:07.031 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.031 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.031 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.031 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Prometheus container image", 2024-05-21T22:06:07.031 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.031 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.031 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.031 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.031 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_base": { 2024-05-21T22:06:07.031 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_base", 2024-05-21T22:06:07.031 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.031 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.032 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.032 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-21T22:06:07.032 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.032 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.032 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.032 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Container image name, without the tag", 2024-05-21T22:06:07.032 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.032 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.032 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.032 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.032 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_elasticsearch": { 2024-05-21T22:06:07.032 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_elasticsearch", 2024-05-21T22:06:07.032 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.032 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.033 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.033 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-21T22:06:07.033 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.033 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.033 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.033 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "elasticsearch container image", 2024-05-21T22:06:07.033 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.033 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.033 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.033 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.033 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_grafana": { 2024-05-21T22:06:07.033 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_grafana", 2024-05-21T22:06:07.033 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.033 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.033 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.034 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-21T22:06:07.034 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.034 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.034 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.034 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Prometheus container image", 2024-05-21T22:06:07.034 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.034 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.034 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.034 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.034 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_haproxy": { 2024-05-21T22:06:07.034 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_haproxy", 2024-05-21T22:06:07.034 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.034 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.035 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.035 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-21T22:06:07.035 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.035 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.035 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.035 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "HAproxy container image", 2024-05-21T22:06:07.035 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.035 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.035 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.035 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.035 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_jaeger_agent": { 2024-05-21T22:06:07.035 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_jaeger_agent", 2024-05-21T22:06:07.036 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.036 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.036 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.036 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-21T22:06:07.036 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.036 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.036 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.036 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Jaeger agent container image", 2024-05-21T22:06:07.036 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.036 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.036 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.036 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.036 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_jaeger_collector": { 2024-05-21T22:06:07.036 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_jaeger_collector", 2024-05-21T22:06:07.037 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.037 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.037 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.037 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-21T22:06:07.037 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.037 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.037 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.037 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Jaeger collector container image", 2024-05-21T22:06:07.037 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.037 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.037 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.037 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.037 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_jaeger_query": { 2024-05-21T22:06:07.037 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_jaeger_query", 2024-05-21T22:06:07.038 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.038 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.038 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.038 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-21T22:06:07.038 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.038 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.038 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.038 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Jaeger query container image", 2024-05-21T22:06:07.038 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.038 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.038 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.038 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.038 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_keepalived": { 2024-05-21T22:06:07.038 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_keepalived", 2024-05-21T22:06:07.039 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.039 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.039 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.039 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-21T22:06:07.039 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.039 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.039 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.039 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Keepalived container image", 2024-05-21T22:06:07.039 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.039 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.039 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.039 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.039 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_loki": { 2024-05-21T22:06:07.039 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_loki", 2024-05-21T22:06:07.039 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.040 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.040 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.040 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-05-21T22:06:07.040 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.040 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.040 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.040 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Loki container image", 2024-05-21T22:06:07.040 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.040 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.040 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.040 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.040 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_node_exporter": { 2024-05-21T22:06:07.040 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_node_exporter", 2024-05-21T22:06:07.040 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.040 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.041 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.041 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-21T22:06:07.041 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.041 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.041 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.041 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Prometheus container image", 2024-05-21T22:06:07.041 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.041 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.041 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.041 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.041 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_nvmeof": { 2024-05-21T22:06:07.041 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_nvmeof", 2024-05-21T22:06:07.041 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.041 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.041 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.042 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-05-21T22:06:07.042 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.042 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.042 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.042 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Nvme-of container image", 2024-05-21T22:06:07.042 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.042 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.042 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.042 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.042 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_prometheus": { 2024-05-21T22:06:07.042 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_prometheus", 2024-05-21T22:06:07.042 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.042 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.042 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.043 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-21T22:06:07.043 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.043 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.043 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.043 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Prometheus container image", 2024-05-21T22:06:07.043 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.043 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.043 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.043 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.043 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_promtail": { 2024-05-21T22:06:07.043 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_promtail", 2024-05-21T22:06:07.043 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.043 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.043 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.044 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-05-21T22:06:07.044 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.044 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.044 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.044 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Promtail container image", 2024-05-21T22:06:07.044 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.044 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.044 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.044 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.044 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_samba": { 2024-05-21T22:06:07.044 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_samba", 2024-05-21T22:06:07.044 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.044 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.044 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.045 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-21T22:06:07.045 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.045 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.045 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.045 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Samba/SMB container image", 2024-05-21T22:06:07.045 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.045 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.045 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.045 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.045 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_snmp_gateway": { 2024-05-21T22:06:07.045 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_snmp_gateway", 2024-05-21T22:06:07.045 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.045 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.045 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.046 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-21T22:06:07.046 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.046 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.046 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.046 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "SNMP Gateway container image", 2024-05-21T22:06:07.046 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.046 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.046 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.046 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.046 INFO:teuthology.orchestra.run.smithi151.stdout: "container_init": { 2024-05-21T22:06:07.046 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_init", 2024-05-21T22:06:07.046 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.046 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.046 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.046 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:07.047 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.047 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.047 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.047 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Run podman/docker with `--init`", 2024-05-21T22:06:07.047 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.047 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.047 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.047 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.047 INFO:teuthology.orchestra.run.smithi151.stdout: "daemon_cache_timeout": { 2024-05-21T22:06:07.047 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "daemon_cache_timeout", 2024-05-21T22:06:07.047 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-05-21T22:06:07.047 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.048 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.048 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "600", 2024-05-21T22:06:07.048 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.048 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.048 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.048 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-21T22:06:07.048 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.048 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.048 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.048 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.048 INFO:teuthology.orchestra.run.smithi151.stdout: "default_cephadm_command_timeout": { 2024-05-21T22:06:07.049 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "default_cephadm_command_timeout", 2024-05-21T22:06:07.049 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.049 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.049 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.049 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "900", 2024-05-21T22:06:07.049 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.049 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.049 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.049 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-21T22:06:07.049 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.049 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.049 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.049 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.049 INFO:teuthology.orchestra.run.smithi151.stdout: "default_registry": { 2024-05-21T22:06:07.050 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "default_registry", 2024-05-21T22:06:07.050 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.050 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.050 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.050 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "docker.io", 2024-05-21T22:06:07.050 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.050 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.050 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.050 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-21T22:06:07.050 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.050 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.050 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.051 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.051 INFO:teuthology.orchestra.run.smithi151.stdout: "device_cache_timeout": { 2024-05-21T22:06:07.051 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "device_cache_timeout", 2024-05-21T22:06:07.051 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-05-21T22:06:07.051 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.051 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.051 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "1800", 2024-05-21T22:06:07.051 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.051 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.051 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.051 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "seconds to cache device inventory", 2024-05-21T22:06:07.051 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.051 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.051 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.052 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.052 INFO:teuthology.orchestra.run.smithi151.stdout: "device_enhanced_scan": { 2024-05-21T22:06:07.052 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "device_enhanced_scan", 2024-05-21T22:06:07.052 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.052 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.052 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.052 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.052 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.052 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.052 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.052 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-21T22:06:07.052 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.052 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.052 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.052 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.053 INFO:teuthology.orchestra.run.smithi151.stdout: "facts_cache_timeout": { 2024-05-21T22:06:07.053 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "facts_cache_timeout", 2024-05-21T22:06:07.053 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-05-21T22:06:07.053 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.053 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.053 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "60", 2024-05-21T22:06:07.053 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.053 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.053 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.053 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "seconds to cache host facts data", 2024-05-21T22:06:07.053 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.053 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.053 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.053 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.054 INFO:teuthology.orchestra.run.smithi151.stdout: "grafana_dashboards_path": { 2024-05-21T22:06:07.054 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "grafana_dashboards_path", 2024-05-21T22:06:07.054 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.054 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.054 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.054 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-21T22:06:07.054 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.054 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.054 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.054 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-21T22:06:07.054 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.054 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.054 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.055 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.055 INFO:teuthology.orchestra.run.smithi151.stdout: "host_check_interval": { 2024-05-21T22:06:07.055 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "host_check_interval", 2024-05-21T22:06:07.055 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-05-21T22:06:07.055 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.055 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.055 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "600", 2024-05-21T22:06:07.055 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.055 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.055 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.055 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "how frequently to perform a host check", 2024-05-21T22:06:07.056 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.056 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.056 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.056 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.056 INFO:teuthology.orchestra.run.smithi151.stdout: "hw_monitoring": { 2024-05-21T22:06:07.056 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "hw_monitoring", 2024-05-21T22:06:07.056 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.056 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.056 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.056 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.056 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.056 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.056 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.057 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-21T22:06:07.057 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.057 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.057 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.057 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.057 INFO:teuthology.orchestra.run.smithi151.stdout: "inventory_list_all": { 2024-05-21T22:06:07.057 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "inventory_list_all", 2024-05-21T22:06:07.057 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.057 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.057 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.057 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.057 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.058 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.058 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.058 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-21T22:06:07.058 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.058 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.058 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.058 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.058 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:07.058 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:07.058 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.058 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.058 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.059 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.059 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.059 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.059 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.059 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.059 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.059 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.059 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.059 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.059 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.059 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.059 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.060 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.060 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.060 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.060 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.060 INFO:teuthology.orchestra.run.smithi151.stdout: "log_refresh_metadata": { 2024-05-21T22:06:07.060 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_refresh_metadata", 2024-05-21T22:06:07.060 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.060 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.060 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.060 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.060 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.060 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.060 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.060 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-21T22:06:07.061 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.061 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.061 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.061 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.061 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:07.061 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:07.061 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.061 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.061 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.061 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:07.061 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.061 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.061 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.061 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-21T22:06:07.062 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.062 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.062 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.062 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.062 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:07.062 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:07.062 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.062 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.062 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.062 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:07.062 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.062 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.062 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.063 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.063 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.063 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.063 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.063 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.063 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.063 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.063 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.063 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.063 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.063 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.063 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.063 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:07.064 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:07.064 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.064 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.064 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.064 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.064 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.064 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.064 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.064 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.064 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.064 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.064 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.064 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.064 INFO:teuthology.orchestra.run.smithi151.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-21T22:06:07.065 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-21T22:06:07.065 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.065 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.065 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.065 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.065 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.065 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.065 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.065 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-21T22:06:07.065 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.065 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.065 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.065 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.065 INFO:teuthology.orchestra.run.smithi151.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-21T22:06:07.066 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-21T22:06:07.066 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.066 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.066 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.066 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "*", 2024-05-21T22:06:07.066 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.066 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.066 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.066 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-21T22:06:07.066 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.066 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.066 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.066 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.066 INFO:teuthology.orchestra.run.smithi151.stdout: "max_count_per_host": { 2024-05-21T22:06:07.067 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "max_count_per_host", 2024-05-21T22:06:07.067 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.067 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.067 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.067 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "10", 2024-05-21T22:06:07.067 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.067 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.067 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.067 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "max number of daemons per service per host", 2024-05-21T22:06:07.067 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.067 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.067 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.068 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.068 INFO:teuthology.orchestra.run.smithi151.stdout: "max_osd_draining_count": { 2024-05-21T22:06:07.068 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "max_osd_draining_count", 2024-05-21T22:06:07.068 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.068 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.068 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.068 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "10", 2024-05-21T22:06:07.068 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.068 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.068 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.068 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-21T22:06:07.068 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.068 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.069 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.069 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.069 INFO:teuthology.orchestra.run.smithi151.stdout: "migration_current": { 2024-05-21T22:06:07.069 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "migration_current", 2024-05-21T22:06:07.069 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.069 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.069 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.069 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.069 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.069 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.069 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.069 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "internal - do not modify", 2024-05-21T22:06:07.070 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.070 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.070 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.070 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.070 INFO:teuthology.orchestra.run.smithi151.stdout: "mode": { 2024-05-21T22:06:07.070 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "mode", 2024-05-21T22:06:07.070 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.070 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.070 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.070 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "root", 2024-05-21T22:06:07.070 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.070 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.070 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.071 INFO:teuthology.orchestra.run.smithi151.stdout: "cephadm-package", 2024-05-21T22:06:07.071 INFO:teuthology.orchestra.run.smithi151.stdout: "root" 2024-05-21T22:06:07.071 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.071 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "mode for remote execution of cephadm", 2024-05-21T22:06:07.071 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.071 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.071 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.071 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.071 INFO:teuthology.orchestra.run.smithi151.stdout: "oob_default_addr": { 2024-05-21T22:06:07.071 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "oob_default_addr", 2024-05-21T22:06:07.071 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.071 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.072 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.072 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "169.254.1.1", 2024-05-21T22:06:07.072 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.072 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.072 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.072 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-21T22:06:07.072 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.072 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.072 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.072 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.072 INFO:teuthology.orchestra.run.smithi151.stdout: "prometheus_alerts_path": { 2024-05-21T22:06:07.072 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "prometheus_alerts_path", 2024-05-21T22:06:07.072 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.073 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.073 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.073 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-21T22:06:07.073 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.073 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.073 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.073 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-21T22:06:07.073 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.073 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.073 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.073 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.073 INFO:teuthology.orchestra.run.smithi151.stdout: "registry_insecure": { 2024-05-21T22:06:07.073 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "registry_insecure", 2024-05-21T22:06:07.074 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.074 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.074 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.074 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.074 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.074 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.074 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.074 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-21T22:06:07.074 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.074 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.075 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.075 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.075 INFO:teuthology.orchestra.run.smithi151.stdout: "registry_password": { 2024-05-21T22:06:07.075 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "registry_password", 2024-05-21T22:06:07.075 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.075 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.075 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.075 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.075 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.075 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.075 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.075 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-21T22:06:07.075 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.076 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.076 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.076 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.076 INFO:teuthology.orchestra.run.smithi151.stdout: "registry_url": { 2024-05-21T22:06:07.076 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "registry_url", 2024-05-21T22:06:07.076 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.076 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.076 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.076 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.076 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.076 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.076 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.076 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-21T22:06:07.077 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.077 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.077 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.077 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.077 INFO:teuthology.orchestra.run.smithi151.stdout: "registry_username": { 2024-05-21T22:06:07.077 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "registry_username", 2024-05-21T22:06:07.077 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.077 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.077 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.077 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.077 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.077 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.077 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.077 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-21T22:06:07.077 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.078 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.078 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.078 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.078 INFO:teuthology.orchestra.run.smithi151.stdout: "secure_monitoring_stack": { 2024-05-21T22:06:07.078 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "secure_monitoring_stack", 2024-05-21T22:06:07.078 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.078 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.078 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.078 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.078 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.078 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.078 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.078 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-21T22:06:07.079 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.079 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.079 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.079 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.079 INFO:teuthology.orchestra.run.smithi151.stdout: "service_discovery_port": { 2024-05-21T22:06:07.079 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "service_discovery_port", 2024-05-21T22:06:07.079 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.079 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.079 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.079 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "8765", 2024-05-21T22:06:07.079 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.079 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.079 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.079 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "cephadm service discovery port", 2024-05-21T22:06:07.080 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.080 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.080 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.080 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.080 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:07.080 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:07.080 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.080 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:07.080 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.080 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:07.080 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.080 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.080 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.080 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.080 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.081 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.081 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.081 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.081 INFO:teuthology.orchestra.run.smithi151.stdout: "ssh_config_file": { 2024-05-21T22:06:07.081 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "ssh_config_file", 2024-05-21T22:06:07.081 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.081 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.081 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.081 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.081 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.081 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.081 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.081 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-21T22:06:07.081 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.081 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.082 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.082 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.082 INFO:teuthology.orchestra.run.smithi151.stdout: "use_agent": { 2024-05-21T22:06:07.082 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "use_agent", 2024-05-21T22:06:07.082 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.082 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.082 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.082 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.082 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.082 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.082 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.082 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-21T22:06:07.082 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.083 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.083 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.083 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.083 INFO:teuthology.orchestra.run.smithi151.stdout: "use_repo_digest": { 2024-05-21T22:06:07.083 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "use_repo_digest", 2024-05-21T22:06:07.083 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.083 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.083 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.083 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:07.083 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.083 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.083 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.083 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-21T22:06:07.083 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.083 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.084 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.084 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.084 INFO:teuthology.orchestra.run.smithi151.stdout: "warn_on_failed_host_check": { 2024-05-21T22:06:07.084 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "warn_on_failed_host_check", 2024-05-21T22:06:07.084 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.084 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.084 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.084 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:07.084 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.084 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.084 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.084 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "raise a health warning if the host check fails", 2024-05-21T22:06:07.084 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.084 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.085 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.085 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.085 INFO:teuthology.orchestra.run.smithi151.stdout: "warn_on_stray_daemons": { 2024-05-21T22:06:07.085 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "warn_on_stray_daemons", 2024-05-21T22:06:07.085 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.085 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.085 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.085 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:07.085 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.085 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.085 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.085 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-21T22:06:07.085 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.085 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.085 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.086 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.086 INFO:teuthology.orchestra.run.smithi151.stdout: "warn_on_stray_hosts": { 2024-05-21T22:06:07.086 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "warn_on_stray_hosts", 2024-05-21T22:06:07.086 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.086 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.086 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.086 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:07.086 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.086 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.086 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.086 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-21T22:06:07.086 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.086 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.086 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.087 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.087 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.087 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.087 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:07.087 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "crash", 2024-05-21T22:06:07.087 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:07.087 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:07.087 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:07.087 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:07.087 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:07.087 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.087 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.087 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.087 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.088 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.088 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.088 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.088 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.088 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.088 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.088 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.088 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.088 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.088 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.088 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.088 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.088 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.088 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.089 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.089 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:07.089 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:07.089 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.089 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.089 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.089 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.089 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.089 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.089 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.089 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.089 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.089 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.089 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.090 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.090 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:07.090 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:07.090 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.090 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.090 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.090 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:07.090 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.090 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.090 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.090 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.090 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.090 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.090 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.091 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.091 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.091 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.091 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.091 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.091 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.091 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.091 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.091 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:07.091 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:07.091 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.091 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.091 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.091 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.091 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.092 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.092 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.092 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.092 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.092 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.092 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.092 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.092 INFO:teuthology.orchestra.run.smithi151.stdout: "retain_interval": { 2024-05-21T22:06:07.092 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "retain_interval", 2024-05-21T22:06:07.092 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-05-21T22:06:07.092 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.092 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.092 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "31536000", 2024-05-21T22:06:07.092 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.092 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.093 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.093 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-21T22:06:07.093 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.093 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.093 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.093 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.093 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:07.093 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:07.093 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.093 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:07.093 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.093 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:07.093 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.093 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.094 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.094 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.094 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.094 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.094 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.094 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.094 INFO:teuthology.orchestra.run.smithi151.stdout: "warn_recent_interval": { 2024-05-21T22:06:07.094 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "warn_recent_interval", 2024-05-21T22:06:07.094 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-05-21T22:06:07.094 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.094 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.094 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "1209600", 2024-05-21T22:06:07.095 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.095 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.095 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.095 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-21T22:06:07.095 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.095 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.095 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.095 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.095 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.095 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.095 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:07.095 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "dashboard", 2024-05-21T22:06:07.095 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:07.096 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:07.096 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:07.096 INFO:teuthology.orchestra.run.smithi151.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-21T22:06:07.096 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-21T22:06:07.096 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.096 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.096 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.096 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "10", 2024-05-21T22:06:07.096 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.096 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.096 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.096 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.096 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.097 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.097 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.097 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.097 INFO:teuthology.orchestra.run.smithi151.stdout: "ALERTMANAGER_API_HOST": { 2024-05-21T22:06:07.097 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-21T22:06:07.097 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.097 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.097 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.097 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.097 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.097 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.097 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.098 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.098 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.098 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.098 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.098 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.098 INFO:teuthology.orchestra.run.smithi151.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-21T22:06:07.098 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-21T22:06:07.098 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.098 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.098 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.098 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:07.099 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.099 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.099 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.099 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.099 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.099 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.099 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.099 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.099 INFO:teuthology.orchestra.run.smithi151.stdout: "AUDIT_API_ENABLED": { 2024-05-21T22:06:07.099 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "AUDIT_API_ENABLED", 2024-05-21T22:06:07.099 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.099 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.099 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.100 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.100 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.100 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.100 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.100 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.100 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.100 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.100 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.100 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.100 INFO:teuthology.orchestra.run.smithi151.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-21T22:06:07.100 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-21T22:06:07.100 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.100 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.101 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.101 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:07.101 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.101 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.101 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.101 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.101 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.101 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.101 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.101 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.101 INFO:teuthology.orchestra.run.smithi151.stdout: "ENABLE_BROWSABLE_API": { 2024-05-21T22:06:07.101 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-21T22:06:07.101 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.102 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.102 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.102 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:07.102 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.102 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.102 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.102 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.102 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.102 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.102 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.102 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.102 INFO:teuthology.orchestra.run.smithi151.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-21T22:06:07.103 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-21T22:06:07.103 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.103 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.103 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.103 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:07.103 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.103 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.103 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.103 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.103 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.103 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.103 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.103 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.104 INFO:teuthology.orchestra.run.smithi151.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-21T22:06:07.104 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-21T22:06:07.104 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.104 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.104 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.104 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:07.104 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.104 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.104 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.104 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.104 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.104 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.104 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.104 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.104 INFO:teuthology.orchestra.run.smithi151.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-21T22:06:07.104 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-21T22:06:07.105 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.105 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.105 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.105 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:07.105 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.105 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.105 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.105 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.105 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.105 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.105 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.105 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.105 INFO:teuthology.orchestra.run.smithi151.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-21T22:06:07.106 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-21T22:06:07.106 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.106 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.106 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.106 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:07.106 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.106 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.106 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.106 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.106 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.106 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.106 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.106 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.106 INFO:teuthology.orchestra.run.smithi151.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-21T22:06:07.107 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-21T22:06:07.107 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.107 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.107 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.107 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:07.107 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.107 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.107 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.107 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.107 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.107 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.107 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.107 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.107 INFO:teuthology.orchestra.run.smithi151.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-21T22:06:07.108 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-21T22:06:07.108 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.108 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.108 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.108 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:07.108 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.108 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.108 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.108 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.108 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.108 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.108 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.109 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.109 INFO:teuthology.orchestra.run.smithi151.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-21T22:06:07.109 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-21T22:06:07.109 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.109 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.109 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.109 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:07.109 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.109 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.109 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.109 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.109 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.110 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.110 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.110 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.110 INFO:teuthology.orchestra.run.smithi151.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-21T22:06:07.110 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-21T22:06:07.110 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.110 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.110 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.110 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.110 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.110 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.110 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.110 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.111 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.111 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.111 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.111 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.111 INFO:teuthology.orchestra.run.smithi151.stdout: "GRAFANA_API_PASSWORD": { 2024-05-21T22:06:07.111 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-21T22:06:07.111 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.111 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.111 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.111 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "admin", 2024-05-21T22:06:07.111 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.111 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.111 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.111 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.112 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.112 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.112 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.112 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.112 INFO:teuthology.orchestra.run.smithi151.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-21T22:06:07.112 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-21T22:06:07.112 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.112 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.112 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.112 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:07.112 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.112 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.112 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.112 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.112 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.113 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.113 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.113 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.113 INFO:teuthology.orchestra.run.smithi151.stdout: "GRAFANA_API_URL": { 2024-05-21T22:06:07.113 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "GRAFANA_API_URL", 2024-05-21T22:06:07.113 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.113 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.113 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.113 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.113 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.113 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.113 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.113 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.114 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.114 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.114 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.114 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.114 INFO:teuthology.orchestra.run.smithi151.stdout: "GRAFANA_API_USERNAME": { 2024-05-21T22:06:07.114 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-21T22:06:07.114 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.114 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.114 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.114 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "admin", 2024-05-21T22:06:07.114 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.115 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.115 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.115 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.115 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.115 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.115 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.115 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.115 INFO:teuthology.orchestra.run.smithi151.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-21T22:06:07.115 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-21T22:06:07.115 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.115 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.115 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.115 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.116 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.116 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.116 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.116 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.116 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.116 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.116 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.116 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.116 INFO:teuthology.orchestra.run.smithi151.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-21T22:06:07.116 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-21T22:06:07.116 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.116 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.116 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.116 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.116 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.117 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.117 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.117 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.117 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.117 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.117 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.117 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.117 INFO:teuthology.orchestra.run.smithi151.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-21T22:06:07.117 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-21T22:06:07.117 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.117 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.117 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.117 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:07.118 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.118 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.118 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.118 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.118 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.118 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.118 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.118 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.118 INFO:teuthology.orchestra.run.smithi151.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-21T22:06:07.118 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-21T22:06:07.118 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.118 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.118 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.119 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.119 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.119 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.119 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.119 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.119 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.119 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.119 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.119 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.119 INFO:teuthology.orchestra.run.smithi151.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-21T22:06:07.119 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-21T22:06:07.120 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.120 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.120 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.120 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "[]", 2024-05-21T22:06:07.120 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.120 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.120 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.120 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.120 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.120 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.120 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.120 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.121 INFO:teuthology.orchestra.run.smithi151.stdout: "MULTICLUSTER_CONFIG": { 2024-05-21T22:06:07.121 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-21T22:06:07.121 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.121 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.121 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.121 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "{}", 2024-05-21T22:06:07.121 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.121 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.121 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.121 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.121 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.122 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.122 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.122 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.122 INFO:teuthology.orchestra.run.smithi151.stdout: "PROMETHEUS_API_HOST": { 2024-05-21T22:06:07.122 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-21T22:06:07.122 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.122 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.122 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.122 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.122 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.122 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.122 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.122 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.123 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.123 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.123 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.123 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.123 INFO:teuthology.orchestra.run.smithi151.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-21T22:06:07.123 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-21T22:06:07.123 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.123 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.123 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.123 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:07.123 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.123 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.124 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.124 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.124 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.124 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.124 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.124 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.124 INFO:teuthology.orchestra.run.smithi151.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-21T22:06:07.124 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-21T22:06:07.124 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.124 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.124 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.124 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.124 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.125 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.125 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.125 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.125 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.125 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.125 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.125 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.125 INFO:teuthology.orchestra.run.smithi151.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-21T22:06:07.125 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-21T22:06:07.125 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.125 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.125 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.125 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.126 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.126 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.126 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.126 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.126 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.126 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.126 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.126 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.126 INFO:teuthology.orchestra.run.smithi151.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-21T22:06:07.126 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-21T22:06:07.126 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.126 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.126 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.127 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:07.127 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.127 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.127 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.127 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.127 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.127 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.127 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.127 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.127 INFO:teuthology.orchestra.run.smithi151.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-21T22:06:07.127 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-21T22:06:07.127 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.127 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.128 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.128 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:07.128 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.128 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.128 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.128 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.128 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.128 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.128 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.128 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.128 INFO:teuthology.orchestra.run.smithi151.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-21T22:06:07.128 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-21T22:06:07.128 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.129 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.129 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.129 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.129 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.129 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.129 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.129 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.129 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.129 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.129 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.129 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.129 INFO:teuthology.orchestra.run.smithi151.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-21T22:06:07.129 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-21T22:06:07.129 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.130 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.130 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.130 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.130 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.130 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.130 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.130 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.130 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.130 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.130 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.130 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.130 INFO:teuthology.orchestra.run.smithi151.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-21T22:06:07.130 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-21T22:06:07.131 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.131 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.131 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.131 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.131 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.131 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.131 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.131 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.131 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.131 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.131 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.131 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.131 INFO:teuthology.orchestra.run.smithi151.stdout: "PWD_POLICY_ENABLED": { 2024-05-21T22:06:07.131 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-21T22:06:07.132 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.132 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.132 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.132 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:07.132 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.132 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.132 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.132 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.132 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.132 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.132 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.132 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.132 INFO:teuthology.orchestra.run.smithi151.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-21T22:06:07.132 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-21T22:06:07.133 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.133 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.133 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.133 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-21T22:06:07.133 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.133 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.133 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.133 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.133 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.133 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.133 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.133 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.133 INFO:teuthology.orchestra.run.smithi151.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-21T22:06:07.133 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-21T22:06:07.134 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.134 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.134 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.134 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "10", 2024-05-21T22:06:07.134 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.134 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.134 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.134 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.134 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.134 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.134 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.134 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.135 INFO:teuthology.orchestra.run.smithi151.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-21T22:06:07.135 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-21T22:06:07.135 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.135 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.135 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.135 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "8", 2024-05-21T22:06:07.135 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.135 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.135 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.136 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.136 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.136 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.136 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.136 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.136 INFO:teuthology.orchestra.run.smithi151.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-21T22:06:07.136 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-21T22:06:07.136 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.136 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.136 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.136 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "45", 2024-05-21T22:06:07.136 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.137 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.137 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.137 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.137 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.137 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.137 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.137 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.137 INFO:teuthology.orchestra.run.smithi151.stdout: "RGW_API_ACCESS_KEY": { 2024-05-21T22:06:07.137 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-21T22:06:07.137 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.137 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.137 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.137 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.138 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.138 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.138 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.138 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.138 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.138 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.138 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.138 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.138 INFO:teuthology.orchestra.run.smithi151.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-21T22:06:07.138 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-21T22:06:07.138 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.138 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.139 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.139 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "admin", 2024-05-21T22:06:07.139 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.139 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.139 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.139 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.139 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.139 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.139 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.139 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.139 INFO:teuthology.orchestra.run.smithi151.stdout: "RGW_API_SECRET_KEY": { 2024-05-21T22:06:07.139 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-21T22:06:07.140 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.140 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.140 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.140 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.140 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.140 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.140 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.140 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.140 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.140 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.140 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.140 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.141 INFO:teuthology.orchestra.run.smithi151.stdout: "RGW_API_SSL_VERIFY": { 2024-05-21T22:06:07.141 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-21T22:06:07.141 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.141 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.141 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.141 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:07.141 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.141 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.141 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.141 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.141 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.141 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.142 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.142 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.142 INFO:teuthology.orchestra.run.smithi151.stdout: "UNSAFE_TLS_v1_2": { 2024-05-21T22:06:07.142 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-21T22:06:07.142 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.142 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.142 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.142 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.142 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.142 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.142 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.143 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.143 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.143 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.143 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.143 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.143 INFO:teuthology.orchestra.run.smithi151.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-21T22:06:07.143 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-21T22:06:07.143 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.143 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.143 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.143 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:07.144 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.144 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.144 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.144 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.144 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.144 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.144 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.144 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.144 INFO:teuthology.orchestra.run.smithi151.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-21T22:06:07.144 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-21T22:06:07.144 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.144 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.145 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.145 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "10", 2024-05-21T22:06:07.145 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.145 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.145 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.145 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.145 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.145 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.145 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.145 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.146 INFO:teuthology.orchestra.run.smithi151.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-21T22:06:07.146 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-21T22:06:07.146 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.146 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.146 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.146 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "5", 2024-05-21T22:06:07.146 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.146 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.146 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.146 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.146 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.146 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.147 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.147 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.147 INFO:teuthology.orchestra.run.smithi151.stdout: "cross_origin_url": { 2024-05-21T22:06:07.147 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "cross_origin_url", 2024-05-21T22:06:07.147 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.147 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.147 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.147 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.147 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.147 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.147 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.147 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.148 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.148 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.148 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.148 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.148 INFO:teuthology.orchestra.run.smithi151.stdout: "crt_file": { 2024-05-21T22:06:07.148 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "crt_file", 2024-05-21T22:06:07.148 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.148 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.148 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.148 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.148 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.148 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.148 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.149 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.149 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.149 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.149 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.149 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.149 INFO:teuthology.orchestra.run.smithi151.stdout: "debug": { 2024-05-21T22:06:07.149 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "debug", 2024-05-21T22:06:07.149 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.149 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.149 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.149 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.149 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.149 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.149 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.150 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Enable/disable debug options", 2024-05-21T22:06:07.150 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.150 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.150 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.150 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.150 INFO:teuthology.orchestra.run.smithi151.stdout: "jwt_token_ttl": { 2024-05-21T22:06:07.150 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "jwt_token_ttl", 2024-05-21T22:06:07.150 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.150 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.150 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.150 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "28800", 2024-05-21T22:06:07.150 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.150 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.151 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.151 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.151 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.151 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.151 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.151 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.151 INFO:teuthology.orchestra.run.smithi151.stdout: "key_file": { 2024-05-21T22:06:07.151 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "key_file", 2024-05-21T22:06:07.151 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.151 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.151 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.152 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.152 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.152 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.152 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.152 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.152 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.152 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.152 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.152 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.152 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:07.152 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:07.152 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.152 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.153 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.153 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.153 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.153 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.153 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.153 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.153 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.153 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.153 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.153 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.153 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.153 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.153 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.153 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.154 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.154 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.154 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.154 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:07.154 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:07.154 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.154 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.154 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.154 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.154 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.154 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.154 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.155 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.155 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.155 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.155 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.155 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.155 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:07.155 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:07.155 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.155 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.155 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.155 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:07.155 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.155 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.156 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.156 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.156 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.156 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.156 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.156 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.156 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.156 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.156 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.156 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.156 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.157 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.157 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.157 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:07.157 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:07.157 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.157 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.157 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.157 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.157 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.157 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.157 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.157 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.158 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.158 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.158 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.158 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.158 INFO:teuthology.orchestra.run.smithi151.stdout: "motd": { 2024-05-21T22:06:07.158 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "motd", 2024-05-21T22:06:07.158 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.158 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.158 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.158 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.158 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.158 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.158 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.159 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "The message of the day", 2024-05-21T22:06:07.159 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.159 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.159 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.159 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.159 INFO:teuthology.orchestra.run.smithi151.stdout: "redirect_resolve_ip_addr": { 2024-05-21T22:06:07.159 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "redirect_resolve_ip_addr", 2024-05-21T22:06:07.159 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.159 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.159 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.159 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.159 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.159 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.160 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.160 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.160 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.160 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.160 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.160 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.160 INFO:teuthology.orchestra.run.smithi151.stdout: "server_addr": { 2024-05-21T22:06:07.160 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "server_addr", 2024-05-21T22:06:07.160 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.160 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.160 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.160 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "::", 2024-05-21T22:06:07.160 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.161 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.161 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.161 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.161 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.161 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.161 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.161 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.161 INFO:teuthology.orchestra.run.smithi151.stdout: "server_port": { 2024-05-21T22:06:07.161 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "server_port", 2024-05-21T22:06:07.161 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.161 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.161 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.161 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "8080", 2024-05-21T22:06:07.162 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.162 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.162 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.162 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.162 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.162 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.162 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.162 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.162 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:07.162 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:07.162 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.163 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:07.163 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.163 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:07.163 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.163 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.163 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.163 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.163 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.163 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.163 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.163 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.163 INFO:teuthology.orchestra.run.smithi151.stdout: "ssl": { 2024-05-21T22:06:07.163 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "ssl", 2024-05-21T22:06:07.164 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.164 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.164 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.164 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:07.164 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.164 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.164 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.164 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.164 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.164 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.164 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.164 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.164 INFO:teuthology.orchestra.run.smithi151.stdout: "ssl_server_port": { 2024-05-21T22:06:07.165 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "ssl_server_port", 2024-05-21T22:06:07.165 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.165 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.165 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.165 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "8443", 2024-05-21T22:06:07.165 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.165 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.165 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.165 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.165 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.165 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.165 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.165 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.165 INFO:teuthology.orchestra.run.smithi151.stdout: "standby_behaviour": { 2024-05-21T22:06:07.166 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "standby_behaviour", 2024-05-21T22:06:07.166 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.166 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.166 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.166 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "redirect", 2024-05-21T22:06:07.166 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.166 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.166 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.166 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.166 INFO:teuthology.orchestra.run.smithi151.stdout: "redirect" 2024-05-21T22:06:07.166 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.166 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.166 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.167 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.167 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.167 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.167 INFO:teuthology.orchestra.run.smithi151.stdout: "standby_error_status_code": { 2024-05-21T22:06:07.167 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "standby_error_status_code", 2024-05-21T22:06:07.167 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.167 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.167 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.167 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "500", 2024-05-21T22:06:07.167 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "400", 2024-05-21T22:06:07.167 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "599", 2024-05-21T22:06:07.167 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.167 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.168 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.168 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.168 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.168 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.168 INFO:teuthology.orchestra.run.smithi151.stdout: "url_prefix": { 2024-05-21T22:06:07.168 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "url_prefix", 2024-05-21T22:06:07.168 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.168 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.168 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.168 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.168 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.168 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.169 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.169 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.169 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.169 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.169 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.169 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.169 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.169 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.169 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:07.169 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "devicehealth", 2024-05-21T22:06:07.169 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:07.169 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:07.169 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:07.169 INFO:teuthology.orchestra.run.smithi151.stdout: "enable_monitoring": { 2024-05-21T22:06:07.170 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "enable_monitoring", 2024-05-21T22:06:07.170 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.170 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.170 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.170 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:07.170 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.170 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.170 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.170 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "monitor device health metrics", 2024-05-21T22:06:07.170 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.170 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.170 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.170 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.171 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:07.171 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:07.171 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.171 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.171 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.171 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.171 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.171 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.171 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.171 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.171 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.172 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.172 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.172 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.172 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.172 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.172 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.172 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.172 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.172 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.172 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.172 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:07.172 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:07.172 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.172 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.172 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.173 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.173 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.173 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.173 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.173 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.173 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.173 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.173 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.173 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.173 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:07.173 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:07.173 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.173 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.173 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.174 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:07.174 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.174 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.174 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.174 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.174 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.174 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.174 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.174 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.174 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.174 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.174 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.174 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.175 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.175 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.175 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.175 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:07.175 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:07.175 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.175 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.175 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.175 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.175 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.175 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.175 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.175 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.175 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.176 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.176 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.176 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.176 INFO:teuthology.orchestra.run.smithi151.stdout: "mark_out_threshold": { 2024-05-21T22:06:07.176 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "mark_out_threshold", 2024-05-21T22:06:07.176 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-05-21T22:06:07.176 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.176 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.176 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "2419200", 2024-05-21T22:06:07.176 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.176 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.176 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.176 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-21T22:06:07.176 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.177 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.177 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.177 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.177 INFO:teuthology.orchestra.run.smithi151.stdout: "pool_name": { 2024-05-21T22:06:07.177 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "pool_name", 2024-05-21T22:06:07.177 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.177 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.177 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.177 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "device_health_metrics", 2024-05-21T22:06:07.177 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.177 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.177 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.177 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-21T22:06:07.177 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.177 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.178 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.178 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.178 INFO:teuthology.orchestra.run.smithi151.stdout: "retention_period": { 2024-05-21T22:06:07.178 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "retention_period", 2024-05-21T22:06:07.178 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-05-21T22:06:07.178 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.178 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.178 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "15552000", 2024-05-21T22:06:07.178 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.178 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.178 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.178 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "how long to retain device health metrics", 2024-05-21T22:06:07.178 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.179 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.179 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.179 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.179 INFO:teuthology.orchestra.run.smithi151.stdout: "scrape_frequency": { 2024-05-21T22:06:07.179 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "scrape_frequency", 2024-05-21T22:06:07.179 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-05-21T22:06:07.179 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.179 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.179 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "86400", 2024-05-21T22:06:07.179 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.179 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.179 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.179 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-21T22:06:07.179 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.180 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.180 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.180 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.180 INFO:teuthology.orchestra.run.smithi151.stdout: "self_heal": { 2024-05-21T22:06:07.180 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "self_heal", 2024-05-21T22:06:07.180 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.180 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.180 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.180 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:07.180 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.180 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.180 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.180 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-21T22:06:07.180 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.180 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.181 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.181 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.181 INFO:teuthology.orchestra.run.smithi151.stdout: "sleep_interval": { 2024-05-21T22:06:07.181 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sleep_interval", 2024-05-21T22:06:07.181 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-05-21T22:06:07.181 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.181 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.181 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "600", 2024-05-21T22:06:07.181 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.181 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.181 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.181 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "how frequently to wake up and check device health", 2024-05-21T22:06:07.181 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.182 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.182 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.182 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.182 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:07.182 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:07.182 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.182 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:07.182 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.182 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:07.183 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.183 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.183 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.183 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.183 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.183 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.183 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.183 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.183 INFO:teuthology.orchestra.run.smithi151.stdout: "warn_threshold": { 2024-05-21T22:06:07.183 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "warn_threshold", 2024-05-21T22:06:07.183 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-05-21T22:06:07.183 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.183 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.183 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "7257600", 2024-05-21T22:06:07.184 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.184 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.184 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.184 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-21T22:06:07.184 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.184 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.184 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.184 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.184 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.184 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.184 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:07.184 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "diskprediction_local", 2024-05-21T22:06:07.184 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:07.185 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:07.185 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:07.185 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:07.185 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:07.185 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.185 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.185 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.185 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.185 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.185 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.185 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.185 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.186 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.186 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.186 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.186 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.186 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.186 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.186 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.186 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.186 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.186 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.186 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.187 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:07.187 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:07.187 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.187 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.187 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.187 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.187 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.187 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.187 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.187 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.187 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.187 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.187 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.188 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.188 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:07.188 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:07.188 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.188 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.188 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.188 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:07.188 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.188 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.188 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.188 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.188 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.189 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.189 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.189 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.189 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.189 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.189 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.189 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.189 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.189 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.189 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.189 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:07.189 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:07.189 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.190 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.190 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.190 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.190 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.190 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.190 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.190 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.190 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.190 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.190 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.190 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.190 INFO:teuthology.orchestra.run.smithi151.stdout: "predict_interval": { 2024-05-21T22:06:07.191 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "predict_interval", 2024-05-21T22:06:07.191 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.191 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.191 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.191 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "86400", 2024-05-21T22:06:07.191 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.191 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.191 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.191 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.191 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.191 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.191 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.192 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.192 INFO:teuthology.orchestra.run.smithi151.stdout: "predictor_model": { 2024-05-21T22:06:07.192 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "predictor_model", 2024-05-21T22:06:07.192 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.192 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.192 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.192 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "prophetstor", 2024-05-21T22:06:07.192 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.192 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.192 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.192 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.192 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.192 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.193 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.193 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.193 INFO:teuthology.orchestra.run.smithi151.stdout: "sleep_interval": { 2024-05-21T22:06:07.193 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sleep_interval", 2024-05-21T22:06:07.193 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.193 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.193 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.193 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "600", 2024-05-21T22:06:07.193 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.193 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.193 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.193 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.193 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.193 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.194 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.194 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.194 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:07.194 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:07.194 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.194 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:07.194 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.194 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:07.194 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.194 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.194 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.194 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.194 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.195 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.195 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.195 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.195 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.195 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.195 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:07.195 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "influx", 2024-05-21T22:06:07.195 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": false, 2024-05-21T22:06:07.195 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "influxdb python module not found", 2024-05-21T22:06:07.195 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:07.195 INFO:teuthology.orchestra.run.smithi151.stdout: "batch_size": { 2024-05-21T22:06:07.195 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "batch_size", 2024-05-21T22:06:07.195 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.195 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.195 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.196 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "5000", 2024-05-21T22:06:07.196 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.196 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.196 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.196 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-21T22:06:07.196 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.196 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.196 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.196 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.196 INFO:teuthology.orchestra.run.smithi151.stdout: "database": { 2024-05-21T22:06:07.196 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "database", 2024-05-21T22:06:07.196 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.196 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.196 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.196 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "ceph", 2024-05-21T22:06:07.197 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.197 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.197 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.197 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-21T22:06:07.197 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.197 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.197 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.197 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.197 INFO:teuthology.orchestra.run.smithi151.stdout: "hostname": { 2024-05-21T22:06:07.197 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "hostname", 2024-05-21T22:06:07.197 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.197 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.197 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.197 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.198 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.198 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.198 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.198 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "InfluxDB server hostname", 2024-05-21T22:06:07.198 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.198 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.198 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.198 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.198 INFO:teuthology.orchestra.run.smithi151.stdout: "interval": { 2024-05-21T22:06:07.198 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "interval", 2024-05-21T22:06:07.198 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-05-21T22:06:07.198 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.198 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.199 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "30", 2024-05-21T22:06:07.199 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "5", 2024-05-21T22:06:07.199 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.199 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.199 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-21T22:06:07.199 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.199 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.199 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.199 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.199 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:07.199 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:07.199 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.199 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.199 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.200 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.200 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.200 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.200 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.200 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.200 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.200 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.200 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.200 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.200 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.200 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.200 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.201 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.201 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.201 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.201 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.201 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:07.201 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:07.201 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.201 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.201 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.201 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.201 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.201 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.201 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.201 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.202 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.202 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.202 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.202 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.202 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:07.202 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:07.202 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.202 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.202 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.202 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:07.202 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.202 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.203 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.203 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.203 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.203 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.203 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.203 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.203 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.203 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.203 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.203 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.203 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.203 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.204 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.204 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:07.204 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:07.204 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.204 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.204 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.204 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.204 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.204 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.204 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.204 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.204 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.204 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.205 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.205 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.205 INFO:teuthology.orchestra.run.smithi151.stdout: "password": { 2024-05-21T22:06:07.205 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "password", 2024-05-21T22:06:07.205 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.205 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.205 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.205 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.205 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.205 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.205 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.205 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "password of InfluxDB server user", 2024-05-21T22:06:07.205 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.206 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.206 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.206 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.206 INFO:teuthology.orchestra.run.smithi151.stdout: "port": { 2024-05-21T22:06:07.206 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "port", 2024-05-21T22:06:07.206 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.206 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.206 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.206 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "8086", 2024-05-21T22:06:07.206 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.206 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.206 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.207 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "InfluxDB server port", 2024-05-21T22:06:07.207 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.207 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.207 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.207 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.207 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:07.207 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:07.207 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.207 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:07.207 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.207 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:07.207 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.207 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.207 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.207 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.208 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.208 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.208 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.208 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.208 INFO:teuthology.orchestra.run.smithi151.stdout: "ssl": { 2024-05-21T22:06:07.208 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "ssl", 2024-05-21T22:06:07.208 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.208 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.208 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.208 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "false", 2024-05-21T22:06:07.208 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.208 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.208 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.208 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-21T22:06:07.209 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.209 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.209 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.209 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.209 INFO:teuthology.orchestra.run.smithi151.stdout: "threads": { 2024-05-21T22:06:07.209 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "threads", 2024-05-21T22:06:07.209 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.209 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.209 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.209 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "5", 2024-05-21T22:06:07.209 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "1", 2024-05-21T22:06:07.209 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "32", 2024-05-21T22:06:07.210 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.210 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-21T22:06:07.210 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.210 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.210 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.210 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.210 INFO:teuthology.orchestra.run.smithi151.stdout: "username": { 2024-05-21T22:06:07.210 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "username", 2024-05-21T22:06:07.210 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.210 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.210 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.210 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.210 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.211 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.211 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.211 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "username of InfluxDB server user", 2024-05-21T22:06:07.211 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.211 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.211 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.211 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.211 INFO:teuthology.orchestra.run.smithi151.stdout: "verify_ssl": { 2024-05-21T22:06:07.211 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "verify_ssl", 2024-05-21T22:06:07.211 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.211 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.211 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.211 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "true", 2024-05-21T22:06:07.211 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.212 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.212 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.212 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-21T22:06:07.212 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.212 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.212 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.212 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.212 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.212 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.212 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:07.212 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "insights", 2024-05-21T22:06:07.212 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:07.212 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:07.212 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:07.213 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:07.213 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:07.213 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.213 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.213 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.213 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.213 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.213 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.213 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.213 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.213 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.213 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.213 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.213 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.214 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.214 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.214 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.214 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.214 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.214 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.214 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.214 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:07.214 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:07.214 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.214 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.214 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.214 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.214 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.215 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.215 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.215 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.215 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.215 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.215 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.215 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.215 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:07.215 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:07.215 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.215 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.215 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.215 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:07.215 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.215 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.216 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.216 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.216 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.216 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.216 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.216 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.216 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.216 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.216 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.216 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.216 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.216 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.216 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.216 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:07.216 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:07.217 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.217 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.217 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.217 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.217 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.217 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.217 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.217 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.217 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.217 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.217 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.217 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.217 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:07.217 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:07.217 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.218 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:07.218 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.218 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:07.218 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.218 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.218 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.218 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.218 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.218 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.218 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.218 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.218 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.218 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.219 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:07.219 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "iostat", 2024-05-21T22:06:07.219 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:07.219 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:07.219 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:07.219 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:07.219 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:07.219 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.219 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.219 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.219 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.219 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.219 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.220 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.220 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.220 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.220 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.220 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.220 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.220 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.220 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.220 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.220 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.220 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.220 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.220 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.221 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:07.221 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:07.221 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.221 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.221 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.221 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.221 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.221 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.221 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.221 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.221 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.221 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.221 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.222 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.222 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:07.222 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:07.222 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.222 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.222 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.222 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:07.222 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.222 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.222 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.222 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.222 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.222 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.223 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.223 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.223 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.223 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.223 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.223 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.223 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.223 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.223 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.223 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:07.223 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:07.223 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.223 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.223 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.224 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.224 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.224 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.224 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.224 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.224 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.224 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.224 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.224 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.224 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:07.224 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:07.224 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.225 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:07.225 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.225 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:07.225 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.225 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.225 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.225 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.225 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.225 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.225 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.225 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.225 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.226 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.226 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:07.226 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "k8sevents", 2024-05-21T22:06:07.226 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:07.226 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:07.226 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:07.226 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_event_retention_days": { 2024-05-21T22:06:07.226 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "ceph_event_retention_days", 2024-05-21T22:06:07.226 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.226 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.226 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.226 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "7", 2024-05-21T22:06:07.226 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.227 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.227 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.227 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-21T22:06:07.227 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.227 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.227 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.227 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.227 INFO:teuthology.orchestra.run.smithi151.stdout: "config_check_secs": { 2024-05-21T22:06:07.227 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "config_check_secs", 2024-05-21T22:06:07.227 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.227 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.227 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.227 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "10", 2024-05-21T22:06:07.228 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "10", 2024-05-21T22:06:07.228 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.228 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.228 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-21T22:06:07.228 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.228 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.228 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.228 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.228 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:07.228 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:07.228 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.229 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.229 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.229 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.229 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.229 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.229 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.229 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.229 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.229 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.229 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.229 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.229 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.229 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.230 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.230 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.230 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.230 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.230 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.230 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:07.230 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:07.230 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.230 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.230 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.230 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.231 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.231 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.231 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.231 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.231 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.231 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.231 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.231 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.231 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:07.231 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:07.231 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.231 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.231 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.231 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:07.231 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.232 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.232 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.232 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.232 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.232 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.232 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.232 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.232 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.232 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.232 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.232 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.232 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.232 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.232 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.233 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:07.233 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:07.233 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.233 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.233 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.233 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.233 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.233 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.233 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.233 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.233 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.233 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.233 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.233 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.234 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:07.234 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:07.234 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.234 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:07.234 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.234 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:07.234 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.234 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.234 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.234 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.234 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.234 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.234 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.234 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.235 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.235 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.235 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:07.235 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "localpool", 2024-05-21T22:06:07.235 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:07.235 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:07.235 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:07.235 INFO:teuthology.orchestra.run.smithi151.stdout: "failure_domain": { 2024-05-21T22:06:07.235 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "failure_domain", 2024-05-21T22:06:07.235 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.235 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.235 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.235 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "host", 2024-05-21T22:06:07.235 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.235 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.236 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.236 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "failure domain for any created local pool", 2024-05-21T22:06:07.236 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-21T22:06:07.236 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.236 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.236 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.236 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:07.236 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:07.236 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.236 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.236 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.236 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.236 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.236 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.237 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.237 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.237 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.237 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.237 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.237 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.237 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.237 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.237 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.237 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.237 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.237 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.237 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.237 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:07.238 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:07.238 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.238 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.238 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.238 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.238 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.238 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.238 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.238 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.238 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.238 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.238 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.239 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.239 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:07.239 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:07.239 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.239 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.239 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.239 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:07.239 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.239 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.239 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.239 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.239 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.239 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.239 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.239 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.240 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.240 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.240 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.240 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.240 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.240 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.240 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.240 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:07.240 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:07.240 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.240 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.240 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.240 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.241 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.241 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.241 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.241 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.241 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.241 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.241 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.241 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.241 INFO:teuthology.orchestra.run.smithi151.stdout: "min_size": { 2024-05-21T22:06:07.241 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "min_size", 2024-05-21T22:06:07.241 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.241 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.242 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.242 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.242 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.242 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.242 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.242 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "default min_size for any created local pool", 2024-05-21T22:06:07.242 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-21T22:06:07.242 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.242 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.242 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.242 INFO:teuthology.orchestra.run.smithi151.stdout: "num_rep": { 2024-05-21T22:06:07.242 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "num_rep", 2024-05-21T22:06:07.242 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.243 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.243 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.243 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "3", 2024-05-21T22:06:07.243 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.243 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.243 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.243 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "default replica count for any created local pool", 2024-05-21T22:06:07.243 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.243 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.243 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.243 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.243 INFO:teuthology.orchestra.run.smithi151.stdout: "pg_num": { 2024-05-21T22:06:07.243 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "pg_num", 2024-05-21T22:06:07.244 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.244 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.244 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.244 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "128", 2024-05-21T22:06:07.244 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.244 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.244 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.244 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "default pg_num for any created local pool", 2024-05-21T22:06:07.244 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.244 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.244 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.244 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.244 INFO:teuthology.orchestra.run.smithi151.stdout: "prefix": { 2024-05-21T22:06:07.244 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "prefix", 2024-05-21T22:06:07.245 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.245 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.245 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.245 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.245 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.245 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.245 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.245 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "name prefix for any created local pool", 2024-05-21T22:06:07.245 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.245 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.245 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.245 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.245 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:07.245 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:07.245 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.246 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:07.246 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.246 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:07.246 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.246 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.246 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.246 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.246 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.246 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.246 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.246 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.246 INFO:teuthology.orchestra.run.smithi151.stdout: "subtree": { 2024-05-21T22:06:07.246 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "subtree", 2024-05-21T22:06:07.247 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.247 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.247 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.247 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "rack", 2024-05-21T22:06:07.247 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.247 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.247 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.247 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-21T22:06:07.247 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-21T22:06:07.247 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.247 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.247 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.247 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.248 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.248 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:07.248 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "mds_autoscaler", 2024-05-21T22:06:07.248 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:07.248 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:07.248 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:07.248 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:07.248 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:07.248 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.248 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.248 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.248 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.248 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.248 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.249 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.249 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.249 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.249 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.249 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.249 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.249 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.249 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.249 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.249 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.249 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.249 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.249 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.249 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:07.249 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:07.250 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.250 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.250 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.250 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.250 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.250 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.250 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.250 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.250 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.250 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.250 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.250 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.250 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:07.250 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:07.251 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.251 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.251 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.251 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:07.251 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.251 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.251 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.251 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.251 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.251 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.251 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.251 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.251 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.251 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.252 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.252 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.252 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.252 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.252 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.252 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:07.252 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:07.252 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.252 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.252 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.252 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.252 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.252 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.253 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.253 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.253 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.253 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.253 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.253 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.253 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:07.253 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:07.253 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.253 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:07.253 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.253 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:07.253 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.253 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.253 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.254 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.254 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.254 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.254 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.254 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.254 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.254 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.254 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:07.254 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "mirroring", 2024-05-21T22:06:07.254 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:07.254 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:07.254 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:07.254 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:07.254 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:07.255 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.255 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.255 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.255 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.255 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.255 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.255 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.255 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.255 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.255 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.255 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.255 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.255 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.255 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.256 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.256 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.256 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.256 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.256 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.256 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:07.256 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:07.256 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.256 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.256 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.256 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.256 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.256 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.256 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.257 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.257 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.257 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.257 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.257 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.257 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:07.257 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:07.257 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.257 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.257 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.257 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:07.257 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.257 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.258 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.258 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.258 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.258 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.258 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.258 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.258 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.258 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.258 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.258 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.258 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.258 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.258 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.259 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:07.259 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:07.259 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.259 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.259 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.259 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.259 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.259 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.259 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.259 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.259 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.259 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.260 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.260 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.260 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:07.260 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:07.260 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.260 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:07.260 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.260 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:07.260 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.260 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.260 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.260 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.260 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.260 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.260 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.261 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.261 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.261 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.261 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:07.261 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "nfs", 2024-05-21T22:06:07.261 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:07.261 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:07.261 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:07.261 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:07.261 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:07.261 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.261 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.261 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.261 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.262 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.262 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.262 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.262 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.262 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.262 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.262 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.262 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.262 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.262 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.262 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.262 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.262 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.262 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.263 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.263 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:07.263 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:07.263 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.263 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.263 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.263 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.263 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.263 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.263 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.263 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.263 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.263 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.263 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.263 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.263 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:07.264 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:07.264 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.264 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.264 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.264 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:07.264 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.264 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.264 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.264 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.264 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.264 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.264 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.264 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.264 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.265 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.265 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.265 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.265 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.265 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.265 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.265 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:07.265 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:07.265 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.265 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.265 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.265 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.265 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.265 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.266 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.266 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.266 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.266 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.266 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.266 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.266 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:07.266 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:07.266 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.266 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:07.266 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.266 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:07.266 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.267 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.267 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.267 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.267 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.267 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.267 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.267 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.267 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.267 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.267 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:07.267 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "orchestrator", 2024-05-21T22:06:07.267 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:07.267 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:07.267 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:07.267 INFO:teuthology.orchestra.run.smithi151.stdout: "fail_fs": { 2024-05-21T22:06:07.268 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "fail_fs", 2024-05-21T22:06:07.268 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.268 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.268 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.268 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.268 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.268 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.268 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.268 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-21T22:06:07.268 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.268 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.268 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.269 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.269 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:07.269 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:07.269 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.269 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.269 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.269 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.269 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.269 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.269 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.269 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.269 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.269 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.270 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.270 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.270 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.270 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.270 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.270 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.270 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.270 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.270 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.270 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:07.270 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:07.270 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.270 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.270 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.270 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.271 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.271 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.271 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.271 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.271 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.271 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.271 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.271 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.271 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:07.271 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:07.271 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.271 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.271 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.272 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:07.272 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.272 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.272 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.272 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.272 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.272 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.272 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.272 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.272 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.272 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.272 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.272 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.272 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.272 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.273 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.273 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:07.273 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:07.273 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.273 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.273 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.273 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.273 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.273 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.273 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.273 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.273 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.274 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.274 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.274 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.274 INFO:teuthology.orchestra.run.smithi151.stdout: "orchestrator": { 2024-05-21T22:06:07.274 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "orchestrator", 2024-05-21T22:06:07.274 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.274 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.274 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.274 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.274 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.274 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.274 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.274 INFO:teuthology.orchestra.run.smithi151.stdout: "cephadm", 2024-05-21T22:06:07.275 INFO:teuthology.orchestra.run.smithi151.stdout: "rook", 2024-05-21T22:06:07.275 INFO:teuthology.orchestra.run.smithi151.stdout: "test_orchestrator" 2024-05-21T22:06:07.275 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.275 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Orchestrator backend", 2024-05-21T22:06:07.275 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.275 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.275 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.275 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.275 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:07.275 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:07.275 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.275 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:07.275 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.276 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:07.276 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.276 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.276 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.276 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.276 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.276 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.276 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.276 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.276 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.276 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.276 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:07.276 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "osd_perf_query", 2024-05-21T22:06:07.276 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:07.277 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:07.277 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:07.277 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:07.277 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:07.277 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.277 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.277 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.277 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.277 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.277 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.277 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.277 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.277 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.277 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.278 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.278 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.278 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.278 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.278 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.278 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.278 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.278 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.278 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.278 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:07.278 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:07.278 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.278 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.279 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.279 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.279 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.279 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.279 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.279 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.279 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.279 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.279 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.279 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.279 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:07.279 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:07.279 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.280 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.280 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.280 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:07.280 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.280 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.280 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.280 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.280 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.280 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.280 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.280 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.280 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.280 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.280 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.280 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.280 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.281 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.281 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.281 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:07.281 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:07.281 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.281 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.281 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.281 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.281 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.281 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.281 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.281 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.281 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.281 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.281 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.282 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.282 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:07.282 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:07.282 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.282 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:07.282 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.282 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:07.282 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.282 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.282 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.282 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.282 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.282 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.283 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.283 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.283 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.283 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.283 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:07.283 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "osd_support", 2024-05-21T22:06:07.283 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:07.283 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:07.283 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:07.283 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:07.283 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:07.283 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.283 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.284 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.284 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.284 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.284 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.284 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.284 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.284 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.284 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.284 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.284 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.284 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.284 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.284 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.284 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.284 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.285 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.285 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.285 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:07.285 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:07.285 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.285 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.285 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.285 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.285 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.285 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.285 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.285 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.285 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.285 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.285 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.286 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.286 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:07.286 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:07.286 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.286 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.286 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.286 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:07.286 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.286 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.286 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.286 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.286 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.286 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.286 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.286 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.287 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.287 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.287 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.287 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.287 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.287 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.287 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.287 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:07.287 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:07.287 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.287 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.287 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.288 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.288 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.288 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.288 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.288 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.288 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.288 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.288 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.288 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.288 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:07.288 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:07.288 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.288 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:07.289 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.289 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:07.289 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.289 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.289 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.289 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.289 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.289 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.289 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.289 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.289 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.289 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.289 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:07.289 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "pg_autoscaler", 2024-05-21T22:06:07.290 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:07.290 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:07.290 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:07.290 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:07.290 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:07.290 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.290 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.290 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.290 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.290 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.290 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.290 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.290 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.290 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.291 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.291 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.291 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.291 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.291 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.291 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.291 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.291 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.291 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.291 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.291 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:07.291 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:07.291 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.291 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.292 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.292 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.292 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.292 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.292 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.292 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.292 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.292 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.292 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.292 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.292 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:07.292 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:07.292 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.292 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.292 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.293 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:07.293 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.293 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.293 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.293 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.293 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.293 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.293 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.293 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.293 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.293 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.293 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.293 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.293 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.294 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.294 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.294 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:07.294 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:07.294 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.294 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.294 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.294 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.294 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.294 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.294 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.294 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.294 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.295 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.295 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.295 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.295 INFO:teuthology.orchestra.run.smithi151.stdout: "sleep_interval": { 2024-05-21T22:06:07.295 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sleep_interval", 2024-05-21T22:06:07.295 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-05-21T22:06:07.295 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.295 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.295 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "60", 2024-05-21T22:06:07.295 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.295 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.295 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.295 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.296 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.296 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.296 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.296 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.296 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:07.296 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:07.296 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.296 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:07.296 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.296 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:07.296 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.296 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.297 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.297 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.297 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.297 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.297 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.297 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.297 INFO:teuthology.orchestra.run.smithi151.stdout: "threshold": { 2024-05-21T22:06:07.297 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "threshold", 2024-05-21T22:06:07.297 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "float", 2024-05-21T22:06:07.297 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.297 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.297 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "3.0", 2024-05-21T22:06:07.297 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "1.0", 2024-05-21T22:06:07.298 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.298 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.298 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "scaling threshold", 2024-05-21T22:06:07.298 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-21T22:06:07.298 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.298 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.298 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.298 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.298 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.298 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:07.298 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "progress", 2024-05-21T22:06:07.298 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:07.298 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:07.299 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:07.299 INFO:teuthology.orchestra.run.smithi151.stdout: "allow_pg_recovery_event": { 2024-05-21T22:06:07.299 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "allow_pg_recovery_event", 2024-05-21T22:06:07.299 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.299 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.299 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.299 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.299 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.299 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.299 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.299 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-21T22:06:07.299 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.300 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.300 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.300 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.300 INFO:teuthology.orchestra.run.smithi151.stdout: "enabled": { 2024-05-21T22:06:07.300 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "enabled", 2024-05-21T22:06:07.300 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.300 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.300 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.300 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:07.300 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.300 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.300 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.300 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.300 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.301 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.301 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.301 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.301 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:07.301 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:07.301 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.301 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.301 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.301 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.301 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.301 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.301 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.301 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.301 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.302 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.302 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.302 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.302 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.302 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.302 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.302 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.302 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.302 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.302 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.302 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:07.302 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:07.302 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.302 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.303 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.303 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.303 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.303 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.303 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.303 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.303 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.303 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.303 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.303 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.303 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:07.303 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:07.303 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.303 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.303 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.304 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:07.304 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.304 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.304 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.304 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.304 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.304 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.304 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.304 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.304 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.304 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.304 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.304 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.304 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.305 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.305 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.305 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:07.305 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:07.305 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.305 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.305 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.305 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.305 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.305 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.305 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.305 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.305 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.305 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.305 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.306 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.306 INFO:teuthology.orchestra.run.smithi151.stdout: "max_completed_events": { 2024-05-21T22:06:07.306 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "max_completed_events", 2024-05-21T22:06:07.306 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.306 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.306 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.306 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "50", 2024-05-21T22:06:07.306 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.306 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.306 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.306 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "number of past completed events to remember", 2024-05-21T22:06:07.306 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.306 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.306 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.307 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.307 INFO:teuthology.orchestra.run.smithi151.stdout: "sleep_interval": { 2024-05-21T22:06:07.307 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sleep_interval", 2024-05-21T22:06:07.307 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-05-21T22:06:07.307 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.307 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.307 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "5", 2024-05-21T22:06:07.307 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.307 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.307 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.307 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "how long the module is going to sleep", 2024-05-21T22:06:07.307 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.307 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.308 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.308 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.308 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:07.308 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:07.308 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.308 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:07.308 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.308 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:07.308 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.308 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.308 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.308 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.308 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.309 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.309 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.309 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.309 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.309 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.309 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:07.309 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "prometheus", 2024-05-21T22:06:07.309 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:07.309 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:07.309 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:07.309 INFO:teuthology.orchestra.run.smithi151.stdout: "cache": { 2024-05-21T22:06:07.309 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "cache", 2024-05-21T22:06:07.309 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.310 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.310 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.310 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:07.310 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.310 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.310 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.310 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.310 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.310 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.310 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.310 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.310 INFO:teuthology.orchestra.run.smithi151.stdout: "exclude_perf_counters": { 2024-05-21T22:06:07.311 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "exclude_perf_counters", 2024-05-21T22:06:07.311 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.311 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.311 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.311 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:07.311 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.311 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.311 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.311 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-21T22:06:07.311 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-21T22:06:07.311 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.311 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.312 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.312 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:07.312 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:07.312 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.312 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.312 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.312 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.312 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.312 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.312 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.312 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.312 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.312 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.312 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.313 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.313 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.313 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.313 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.313 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.313 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.313 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.313 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.313 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:07.313 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:07.313 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.313 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.313 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.313 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.313 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.313 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.314 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.314 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.314 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.314 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.314 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.314 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.314 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:07.314 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:07.314 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.314 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.314 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.314 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:07.314 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.314 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.315 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.315 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.315 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.315 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.315 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.315 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.315 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.315 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.315 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.315 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.315 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.315 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.315 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.315 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:07.315 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:07.316 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.316 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.316 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.316 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.316 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.316 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.316 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.316 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.316 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.316 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.316 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.316 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.316 INFO:teuthology.orchestra.run.smithi151.stdout: "rbd_stats_pools": { 2024-05-21T22:06:07.316 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "rbd_stats_pools", 2024-05-21T22:06:07.316 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.317 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.317 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.317 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.317 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.317 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.317 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.317 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.317 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.317 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.317 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.317 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.317 INFO:teuthology.orchestra.run.smithi151.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-21T22:06:07.317 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-21T22:06:07.317 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.318 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.318 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.318 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "300", 2024-05-21T22:06:07.318 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.318 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.318 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.318 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.318 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.318 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.318 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.318 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.318 INFO:teuthology.orchestra.run.smithi151.stdout: "scrape_interval": { 2024-05-21T22:06:07.319 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "scrape_interval", 2024-05-21T22:06:07.319 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "float", 2024-05-21T22:06:07.319 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.319 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.319 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "15.0", 2024-05-21T22:06:07.319 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.319 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.319 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.319 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.319 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.319 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.319 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.319 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.320 INFO:teuthology.orchestra.run.smithi151.stdout: "server_addr": { 2024-05-21T22:06:07.320 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "server_addr", 2024-05-21T22:06:07.320 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.320 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.320 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.320 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "::", 2024-05-21T22:06:07.320 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.320 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.320 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.320 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-21T22:06:07.320 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.320 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.320 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.320 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.320 INFO:teuthology.orchestra.run.smithi151.stdout: "server_port": { 2024-05-21T22:06:07.321 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "server_port", 2024-05-21T22:06:07.321 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.321 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.321 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.321 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "9283", 2024-05-21T22:06:07.321 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.321 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.321 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.321 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-21T22:06:07.321 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.321 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.321 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.322 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.322 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:07.322 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:07.322 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.322 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:07.322 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.322 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:07.322 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.322 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.322 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.322 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.322 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.322 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.323 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.323 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.323 INFO:teuthology.orchestra.run.smithi151.stdout: "stale_cache_strategy": { 2024-05-21T22:06:07.323 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "stale_cache_strategy", 2024-05-21T22:06:07.323 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.323 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.323 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.323 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "log", 2024-05-21T22:06:07.323 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.323 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.323 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.323 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.324 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.324 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.324 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.324 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.324 INFO:teuthology.orchestra.run.smithi151.stdout: "standby_behaviour": { 2024-05-21T22:06:07.324 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "standby_behaviour", 2024-05-21T22:06:07.324 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.324 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.324 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.324 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "default", 2024-05-21T22:06:07.324 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.324 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.325 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.325 INFO:teuthology.orchestra.run.smithi151.stdout: "default", 2024-05-21T22:06:07.325 INFO:teuthology.orchestra.run.smithi151.stdout: "error" 2024-05-21T22:06:07.325 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.325 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.325 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.325 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.325 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.325 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.325 INFO:teuthology.orchestra.run.smithi151.stdout: "standby_error_status_code": { 2024-05-21T22:06:07.325 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "standby_error_status_code", 2024-05-21T22:06:07.325 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.325 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.326 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.326 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "500", 2024-05-21T22:06:07.326 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "400", 2024-05-21T22:06:07.326 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "599", 2024-05-21T22:06:07.326 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.326 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.326 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.326 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.326 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.326 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.326 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.326 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.327 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:07.327 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "rbd_support", 2024-05-21T22:06:07.327 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:07.327 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:07.327 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:07.327 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:07.327 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:07.327 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.327 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.327 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.327 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.327 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.327 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.328 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.328 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.328 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.328 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.328 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.328 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.328 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.328 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.328 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.328 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.328 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.328 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.329 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.329 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:07.329 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:07.329 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.329 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.329 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.329 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.329 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.329 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.329 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.329 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.329 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.330 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.330 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.330 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.330 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:07.330 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:07.330 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.330 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.330 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.330 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:07.330 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.330 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.330 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.331 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.331 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.331 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.331 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.331 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.331 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.331 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.331 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.331 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.331 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.331 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.331 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.332 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:07.332 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:07.332 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.332 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.332 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.332 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.332 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.332 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.332 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.332 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.332 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.332 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.332 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.333 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.333 INFO:teuthology.orchestra.run.smithi151.stdout: "max_concurrent_snap_create": { 2024-05-21T22:06:07.333 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "max_concurrent_snap_create", 2024-05-21T22:06:07.333 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.333 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.333 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.333 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "10", 2024-05-21T22:06:07.333 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.333 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.333 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.333 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.333 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.334 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.334 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.334 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.334 INFO:teuthology.orchestra.run.smithi151.stdout: "mirror_snapshot_schedule": { 2024-05-21T22:06:07.334 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "mirror_snapshot_schedule", 2024-05-21T22:06:07.334 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.334 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.334 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.334 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.334 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.334 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.335 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.335 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.335 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.335 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.335 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.335 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.335 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:07.335 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:07.335 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.335 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:07.335 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.336 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:07.336 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.336 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.336 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.336 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.336 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.336 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.336 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.336 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.336 INFO:teuthology.orchestra.run.smithi151.stdout: "trash_purge_schedule": { 2024-05-21T22:06:07.336 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "trash_purge_schedule", 2024-05-21T22:06:07.336 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.337 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.337 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.337 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.337 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.337 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.337 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.337 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.337 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.337 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.337 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.337 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.338 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.338 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.338 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:07.338 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "restful", 2024-05-21T22:06:07.338 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:07.338 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:07.338 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:07.338 INFO:teuthology.orchestra.run.smithi151.stdout: "enable_auth": { 2024-05-21T22:06:07.338 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "enable_auth", 2024-05-21T22:06:07.338 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.338 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.338 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.339 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:07.339 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.339 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.339 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.339 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.339 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.339 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.339 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.339 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.339 INFO:teuthology.orchestra.run.smithi151.stdout: "key_file": { 2024-05-21T22:06:07.339 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "key_file", 2024-05-21T22:06:07.340 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.340 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.340 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.340 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.340 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.340 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.340 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.340 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.340 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.340 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.341 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.341 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.341 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:07.341 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:07.341 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.341 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.341 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.341 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.341 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.341 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.341 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.342 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.342 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.342 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.342 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.342 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.342 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.342 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.342 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.342 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.342 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.342 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.342 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.342 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:07.343 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:07.343 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.343 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.343 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.343 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.343 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.343 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.343 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.343 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.343 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.343 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.343 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.343 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.344 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:07.344 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:07.344 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.344 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.344 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.344 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:07.344 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.344 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.344 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.344 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.344 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.344 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.345 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.345 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.345 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.345 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.345 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.345 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.345 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.345 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.345 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.345 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:07.345 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:07.345 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.345 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.346 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.346 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.346 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.346 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.346 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.346 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.346 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.346 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.346 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.346 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.346 INFO:teuthology.orchestra.run.smithi151.stdout: "server_addr": { 2024-05-21T22:06:07.346 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "server_addr", 2024-05-21T22:06:07.346 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.347 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.347 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.347 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.347 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.347 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.347 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.347 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.347 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.347 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.347 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.347 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.347 INFO:teuthology.orchestra.run.smithi151.stdout: "server_port": { 2024-05-21T22:06:07.347 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "server_port", 2024-05-21T22:06:07.348 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.348 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.348 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.348 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.348 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.348 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.348 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.348 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.348 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.348 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.348 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.348 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.348 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:07.349 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:07.349 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.349 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:07.349 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.349 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:07.349 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.349 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.349 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.349 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.349 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.349 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.349 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.349 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.350 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.350 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.350 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:07.350 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "rgw", 2024-05-21T22:06:07.350 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:07.350 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:07.350 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:07.350 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:07.350 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:07.350 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.350 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.350 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.350 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.351 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.351 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.351 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.351 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.351 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.351 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.351 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.351 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.351 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.351 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.351 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.351 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.351 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.352 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.352 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.352 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:07.352 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:07.352 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.352 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.352 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.352 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.352 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.352 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.352 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.352 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.352 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.353 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.353 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.353 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.353 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:07.353 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:07.353 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.353 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.353 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.353 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:07.353 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.353 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.353 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.353 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.353 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.353 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.354 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.354 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.354 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.354 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.354 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.354 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.354 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.354 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.354 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.354 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:07.354 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:07.354 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.354 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.355 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.355 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.355 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.355 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.355 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.355 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.355 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.355 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.355 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.355 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.355 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:07.355 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:07.355 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.355 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:07.355 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.356 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:07.356 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.356 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.356 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.356 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.356 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.356 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.356 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.356 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.356 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.356 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.356 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:07.356 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "rook", 2024-05-21T22:06:07.357 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:07.357 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:07.357 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:07.357 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:07.357 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:07.357 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.357 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.357 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.357 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.357 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.357 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.357 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.357 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.358 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.358 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.358 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.358 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.358 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.358 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.358 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.358 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.358 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.358 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.358 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.358 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:07.359 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:07.359 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.359 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.359 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.359 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.359 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.359 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.359 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.359 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.359 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.359 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.359 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.359 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.359 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:07.360 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:07.360 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.360 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.360 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.360 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:07.360 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.360 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.360 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.360 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.360 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.360 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.360 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.360 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.360 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.360 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.361 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.361 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.361 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.361 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.361 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.361 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:07.361 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:07.361 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.361 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.361 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.361 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.361 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.361 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.361 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.361 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.362 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.362 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.362 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.362 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.362 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:07.362 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:07.362 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.362 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:07.362 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.362 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:07.362 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.362 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.362 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.363 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.363 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.363 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.363 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.363 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.363 INFO:teuthology.orchestra.run.smithi151.stdout: "storage_class": { 2024-05-21T22:06:07.363 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "storage_class", 2024-05-21T22:06:07.363 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.363 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.363 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.363 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "local", 2024-05-21T22:06:07.363 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.363 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.363 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.363 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-21T22:06:07.364 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.364 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.364 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.364 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.364 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.364 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.364 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:07.364 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "selftest", 2024-05-21T22:06:07.364 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:07.364 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:07.364 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:07.364 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:07.364 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:07.364 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.364 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.365 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.365 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.365 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.365 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.365 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.365 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.365 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.365 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.365 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.365 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.365 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.365 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.365 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.365 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.365 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.366 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.366 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.366 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:07.366 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:07.366 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.366 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.366 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.366 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.366 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.366 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.366 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.366 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.367 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.367 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.367 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.367 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.367 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:07.367 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:07.367 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.367 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.367 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.367 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:07.367 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.367 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.367 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.368 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.368 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.368 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.368 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.368 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.368 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.368 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.368 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.368 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.368 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.368 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.368 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.369 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:07.369 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:07.369 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.369 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.369 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.369 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.369 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.369 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.369 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.369 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.369 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.369 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.369 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.370 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.370 INFO:teuthology.orchestra.run.smithi151.stdout: "roption1": { 2024-05-21T22:06:07.370 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "roption1", 2024-05-21T22:06:07.370 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.370 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.370 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.370 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.370 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.370 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.370 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.370 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.370 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.371 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.371 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.371 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.371 INFO:teuthology.orchestra.run.smithi151.stdout: "roption2": { 2024-05-21T22:06:07.371 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "roption2", 2024-05-21T22:06:07.371 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.371 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.371 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.371 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "xyz", 2024-05-21T22:06:07.371 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.371 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.371 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.371 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.371 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.371 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.372 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.372 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.372 INFO:teuthology.orchestra.run.smithi151.stdout: "rwoption1": { 2024-05-21T22:06:07.372 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "rwoption1", 2024-05-21T22:06:07.372 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.372 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.372 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.372 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.372 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.372 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.372 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.372 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.372 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.372 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.372 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.373 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.373 INFO:teuthology.orchestra.run.smithi151.stdout: "rwoption2": { 2024-05-21T22:06:07.373 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "rwoption2", 2024-05-21T22:06:07.373 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.373 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.373 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.373 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.373 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.373 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.373 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.373 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.373 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.373 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.373 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.374 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.374 INFO:teuthology.orchestra.run.smithi151.stdout: "rwoption3": { 2024-05-21T22:06:07.374 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "rwoption3", 2024-05-21T22:06:07.374 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "float", 2024-05-21T22:06:07.374 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.374 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.374 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.374 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.374 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.374 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.374 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.374 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.374 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.374 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.375 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.375 INFO:teuthology.orchestra.run.smithi151.stdout: "rwoption4": { 2024-05-21T22:06:07.375 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "rwoption4", 2024-05-21T22:06:07.375 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.375 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.375 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.375 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.375 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.375 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.375 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.375 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.375 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.375 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.376 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.376 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.376 INFO:teuthology.orchestra.run.smithi151.stdout: "rwoption5": { 2024-05-21T22:06:07.376 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "rwoption5", 2024-05-21T22:06:07.376 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.376 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.376 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.376 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.376 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.376 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.376 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.376 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.376 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.376 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.376 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.377 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.377 INFO:teuthology.orchestra.run.smithi151.stdout: "rwoption6": { 2024-05-21T22:06:07.377 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "rwoption6", 2024-05-21T22:06:07.377 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.377 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.377 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.377 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:07.377 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.377 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.377 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.377 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.377 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.377 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.377 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.378 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.378 INFO:teuthology.orchestra.run.smithi151.stdout: "rwoption7": { 2024-05-21T22:06:07.378 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "rwoption7", 2024-05-21T22:06:07.378 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.378 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.378 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.378 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.378 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "1", 2024-05-21T22:06:07.378 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "42", 2024-05-21T22:06:07.378 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.378 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.378 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.379 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.379 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.379 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.379 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:07.379 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:07.379 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.379 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:07.379 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.379 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:07.379 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.379 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.379 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.379 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.379 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.380 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.380 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.380 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.380 INFO:teuthology.orchestra.run.smithi151.stdout: "testkey": { 2024-05-21T22:06:07.380 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "testkey", 2024-05-21T22:06:07.380 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.380 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.380 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.380 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.380 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.380 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.381 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.381 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.381 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.381 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.381 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.381 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.381 INFO:teuthology.orchestra.run.smithi151.stdout: "testlkey": { 2024-05-21T22:06:07.381 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "testlkey", 2024-05-21T22:06:07.381 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.381 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.381 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.381 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.381 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.381 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.382 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.382 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.382 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.382 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.382 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.382 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.382 INFO:teuthology.orchestra.run.smithi151.stdout: "testnewline": { 2024-05-21T22:06:07.382 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "testnewline", 2024-05-21T22:06:07.382 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.382 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.382 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.383 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.383 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.383 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.383 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.383 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.383 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.383 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.383 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.383 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.383 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.383 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.383 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:07.383 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "smb", 2024-05-21T22:06:07.384 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:07.384 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:07.384 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:07.384 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:07.384 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:07.384 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.384 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.384 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.384 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.384 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.384 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.384 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.385 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.385 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.385 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.385 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.385 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.385 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.385 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.385 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.385 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.385 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.385 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.385 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.386 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:07.386 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:07.386 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.386 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.386 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.386 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.386 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.386 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.386 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.386 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.386 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.386 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.387 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.387 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.387 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:07.387 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:07.387 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.387 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.387 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.387 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:07.387 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.387 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.387 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.387 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.387 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.388 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.388 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.388 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.388 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.388 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.388 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.388 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.388 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.388 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.388 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.388 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:07.388 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:07.388 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.389 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.389 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.389 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.389 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.389 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.389 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.389 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.389 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.389 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.389 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.389 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.389 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:07.389 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:07.390 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.390 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:07.390 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.390 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:07.390 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.390 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.390 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.390 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.390 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.390 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.390 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.390 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.390 INFO:teuthology.orchestra.run.smithi151.stdout: "update_orchestration": { 2024-05-21T22:06:07.391 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "update_orchestration", 2024-05-21T22:06:07.391 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.391 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.391 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.391 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:07.391 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.391 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.391 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.391 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-21T22:06:07.391 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.391 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.391 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.391 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.391 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.391 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.392 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:07.392 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "snap_schedule", 2024-05-21T22:06:07.392 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:07.392 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:07.392 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:07.392 INFO:teuthology.orchestra.run.smithi151.stdout: "allow_m_granularity": { 2024-05-21T22:06:07.392 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "allow_m_granularity", 2024-05-21T22:06:07.392 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.392 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.392 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.392 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.392 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.393 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.393 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.393 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "allow minute scheduled snapshots", 2024-05-21T22:06:07.393 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.393 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.393 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.393 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.393 INFO:teuthology.orchestra.run.smithi151.stdout: "dump_on_update": { 2024-05-21T22:06:07.393 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "dump_on_update", 2024-05-21T22:06:07.393 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.393 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.393 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.393 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.394 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.394 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.394 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.394 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "dump database to debug log on update", 2024-05-21T22:06:07.394 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.394 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.394 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.394 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.394 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:07.394 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:07.394 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.394 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.395 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.395 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.395 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.395 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.395 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.395 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.395 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.395 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.395 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.395 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.395 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.395 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.395 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.396 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.396 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.396 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.396 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.396 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:07.396 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:07.396 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.396 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.396 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.396 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.396 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.396 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.396 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.396 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.396 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.397 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.397 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.397 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.397 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:07.397 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:07.397 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.397 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.397 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.397 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:07.397 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.397 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.397 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.397 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.397 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.397 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.398 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.398 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.398 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.398 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.398 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.398 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.398 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.398 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.398 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.398 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:07.398 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:07.398 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.398 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.399 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.399 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.399 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.399 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.399 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.399 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.399 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.399 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.399 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.399 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.399 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:07.399 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:07.399 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.399 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:07.399 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.400 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:07.400 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.400 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.400 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.400 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.400 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.400 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.400 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.400 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.400 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.400 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.400 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:07.400 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "stats", 2024-05-21T22:06:07.400 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:07.401 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:07.401 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:07.401 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:07.401 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:07.401 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.401 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.401 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.401 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.401 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.401 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.401 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.401 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.401 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.401 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.401 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.402 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.402 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.402 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.402 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.402 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.402 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.402 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.402 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.402 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:07.402 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:07.402 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.402 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.402 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.402 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.403 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.403 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.403 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.403 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.403 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.403 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.403 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.403 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.403 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:07.403 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:07.403 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.403 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.403 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.404 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:07.404 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.404 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.404 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.404 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.404 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.404 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.404 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.404 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.404 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.404 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.404 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.404 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.404 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.404 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.405 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.405 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:07.405 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:07.405 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.405 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.405 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.405 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.405 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.405 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.405 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.405 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.405 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.405 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.405 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.406 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.406 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:07.406 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:07.406 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.406 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:07.406 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.406 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:07.406 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.406 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.406 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.406 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.407 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.407 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.407 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.407 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.407 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.407 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.407 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:07.407 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "status", 2024-05-21T22:06:07.407 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:07.407 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:07.407 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:07.407 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:07.407 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:07.407 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.408 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.408 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.408 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.408 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.408 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.408 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.408 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.408 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.408 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.408 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.408 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.408 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.408 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.408 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.409 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.409 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.409 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.409 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.409 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:07.409 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:07.409 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.409 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.409 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.409 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.409 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.409 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.409 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.409 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.409 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.410 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.410 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.410 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.410 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:07.410 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:07.410 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.410 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.410 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.410 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:07.410 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.410 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.410 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.410 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.410 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.411 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.411 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.411 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.411 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.411 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.411 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.411 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.411 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.411 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.411 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.411 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:07.411 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:07.411 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.411 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.412 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.412 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.412 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.412 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.412 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.412 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.412 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.412 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.412 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.412 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.412 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:07.412 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:07.412 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.412 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:07.412 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.413 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:07.413 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.413 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.413 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.413 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.413 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.413 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.413 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.413 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.413 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.413 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.413 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:07.413 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "telegraf", 2024-05-21T22:06:07.414 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:07.414 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:07.414 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:07.414 INFO:teuthology.orchestra.run.smithi151.stdout: "address": { 2024-05-21T22:06:07.414 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "address", 2024-05-21T22:06:07.414 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.414 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.414 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.414 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-21T22:06:07.414 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.414 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.414 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.414 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.415 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.415 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.415 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.415 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.415 INFO:teuthology.orchestra.run.smithi151.stdout: "interval": { 2024-05-21T22:06:07.415 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "interval", 2024-05-21T22:06:07.415 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-05-21T22:06:07.415 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.415 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.415 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "15", 2024-05-21T22:06:07.415 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.415 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.415 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.415 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.416 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.416 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.416 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.416 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.416 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:07.416 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:07.416 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.416 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.416 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.416 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.416 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.416 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.416 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.416 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.416 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.417 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.417 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.417 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.417 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.417 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.417 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.417 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.417 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.417 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.417 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.417 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:07.417 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:07.417 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.417 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.417 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.418 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.418 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.418 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.418 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.418 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.418 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.418 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.418 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.418 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.418 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:07.418 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:07.418 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.418 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.419 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.419 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:07.419 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.419 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.419 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.419 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.419 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.419 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.419 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.419 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.419 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.419 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.419 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.420 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.420 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.420 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.420 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.420 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:07.420 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:07.420 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.420 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.420 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.420 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.420 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.420 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.420 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.420 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.420 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.421 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.421 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.421 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.421 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:07.421 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:07.421 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.421 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:07.421 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.421 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:07.421 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.421 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.421 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.421 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.421 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.422 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.422 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.422 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.422 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.422 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.422 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:07.422 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "telemetry", 2024-05-21T22:06:07.422 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:07.422 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:07.422 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:07.422 INFO:teuthology.orchestra.run.smithi151.stdout: "channel_basic": { 2024-05-21T22:06:07.422 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "channel_basic", 2024-05-21T22:06:07.423 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.423 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.423 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.423 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:07.423 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.423 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.423 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.423 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-21T22:06:07.423 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.423 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.423 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.423 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.423 INFO:teuthology.orchestra.run.smithi151.stdout: "channel_crash": { 2024-05-21T22:06:07.423 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "channel_crash", 2024-05-21T22:06:07.423 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.424 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.424 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.424 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:07.424 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.424 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.424 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.424 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-21T22:06:07.424 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.424 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.424 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.424 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.424 INFO:teuthology.orchestra.run.smithi151.stdout: "channel_device": { 2024-05-21T22:06:07.425 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "channel_device", 2024-05-21T22:06:07.425 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.425 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.425 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.425 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:07.425 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.425 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.425 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.425 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-21T22:06:07.425 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.425 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.425 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.425 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.425 INFO:teuthology.orchestra.run.smithi151.stdout: "channel_ident": { 2024-05-21T22:06:07.425 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "channel_ident", 2024-05-21T22:06:07.426 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.426 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.426 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.426 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.426 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.426 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.426 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.426 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-21T22:06:07.426 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.426 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.426 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.426 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.426 INFO:teuthology.orchestra.run.smithi151.stdout: "channel_perf": { 2024-05-21T22:06:07.427 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "channel_perf", 2024-05-21T22:06:07.427 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.427 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.427 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.427 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.427 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.427 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.427 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.427 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-21T22:06:07.427 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.428 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.428 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.428 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.428 INFO:teuthology.orchestra.run.smithi151.stdout: "contact": { 2024-05-21T22:06:07.428 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "contact", 2024-05-21T22:06:07.428 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.428 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.428 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.428 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.428 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.428 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.428 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.428 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.429 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.429 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.429 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.429 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.429 INFO:teuthology.orchestra.run.smithi151.stdout: "description": { 2024-05-21T22:06:07.429 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "description", 2024-05-21T22:06:07.429 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.429 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.429 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.429 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.429 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.429 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.430 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.430 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.430 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.430 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.430 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.430 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.430 INFO:teuthology.orchestra.run.smithi151.stdout: "device_url": { 2024-05-21T22:06:07.430 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "device_url", 2024-05-21T22:06:07.430 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.430 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.430 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.431 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-21T22:06:07.431 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.431 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.431 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.431 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.431 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.431 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.431 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.431 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.431 INFO:teuthology.orchestra.run.smithi151.stdout: "enabled": { 2024-05-21T22:06:07.431 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "enabled", 2024-05-21T22:06:07.431 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.431 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.432 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.432 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.432 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.432 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.432 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.432 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.432 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.432 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.432 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.432 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.432 INFO:teuthology.orchestra.run.smithi151.stdout: "interval": { 2024-05-21T22:06:07.432 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "interval", 2024-05-21T22:06:07.432 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.433 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.433 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.433 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "24", 2024-05-21T22:06:07.433 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "8", 2024-05-21T22:06:07.433 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.433 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.433 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.433 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.433 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.433 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.433 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.433 INFO:teuthology.orchestra.run.smithi151.stdout: "last_opt_revision": { 2024-05-21T22:06:07.433 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "last_opt_revision", 2024-05-21T22:06:07.433 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.434 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.434 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.434 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "1", 2024-05-21T22:06:07.434 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.434 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.434 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.434 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.434 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.434 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.434 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.434 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.434 INFO:teuthology.orchestra.run.smithi151.stdout: "leaderboard": { 2024-05-21T22:06:07.434 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "leaderboard", 2024-05-21T22:06:07.434 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.435 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.435 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.435 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.435 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.435 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.435 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.435 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.435 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.435 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.435 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.435 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.435 INFO:teuthology.orchestra.run.smithi151.stdout: "leaderboard_description": { 2024-05-21T22:06:07.435 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "leaderboard_description", 2024-05-21T22:06:07.435 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.435 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.436 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.436 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.436 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.436 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.436 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.436 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.436 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.436 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.436 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.436 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.436 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:07.436 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:07.436 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.436 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.436 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.437 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.437 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.437 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.437 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.437 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.437 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.437 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.437 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.437 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.437 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.437 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.437 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.437 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.437 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.437 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.438 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.438 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:07.438 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:07.438 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.438 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.438 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.438 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.438 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.438 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.438 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.438 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.439 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.439 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.439 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.439 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.439 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:07.439 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:07.439 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.439 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.439 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.439 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:07.439 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.439 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.439 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.439 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.440 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.440 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.440 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.440 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.440 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.440 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.440 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.440 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.440 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.440 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.440 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.440 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:07.440 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:07.441 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.441 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.441 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.441 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.441 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.441 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.441 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.441 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.441 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.441 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.441 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.442 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.442 INFO:teuthology.orchestra.run.smithi151.stdout: "organization": { 2024-05-21T22:06:07.442 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "organization", 2024-05-21T22:06:07.442 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.442 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.442 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.442 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.442 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.442 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.442 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.442 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.442 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.443 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.443 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.443 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.443 INFO:teuthology.orchestra.run.smithi151.stdout: "proxy": { 2024-05-21T22:06:07.443 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "proxy", 2024-05-21T22:06:07.443 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.443 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.443 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.443 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.443 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.443 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.443 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.443 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.444 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.444 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.444 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.444 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.444 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:07.444 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:07.444 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.444 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:07.444 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.444 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:07.444 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.444 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.444 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.445 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.445 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.445 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.445 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.445 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.445 INFO:teuthology.orchestra.run.smithi151.stdout: "url": { 2024-05-21T22:06:07.445 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "url", 2024-05-21T22:06:07.445 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.445 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.445 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.445 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-21T22:06:07.445 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.446 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.446 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.446 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.446 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.446 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.446 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.446 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.446 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.446 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.446 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:07.446 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "test_orchestrator", 2024-05-21T22:06:07.447 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:07.447 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:07.447 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:07.447 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:07.447 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:07.447 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.447 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.447 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.447 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.447 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.447 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.447 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.447 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.448 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.448 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.448 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.448 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.448 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.448 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.448 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.448 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.448 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.448 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.448 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.448 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:07.448 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:07.449 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.449 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.449 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.449 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.449 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.449 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.449 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.449 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.449 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.449 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.449 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.449 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.449 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:07.449 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:07.450 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.450 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.450 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.450 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:07.450 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.450 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.450 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.450 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.450 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.450 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.450 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.450 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.450 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.451 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.451 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.451 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.451 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.451 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.451 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.451 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:07.451 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:07.451 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.451 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.451 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.451 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.451 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.451 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.451 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.452 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.452 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.452 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.452 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.452 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.452 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:07.452 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:07.452 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.452 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:07.452 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.452 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:07.452 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.452 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.452 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.452 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.453 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.453 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.453 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.453 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.453 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.453 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.453 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:07.453 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "volumes", 2024-05-21T22:06:07.453 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:07.453 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:07.453 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:07.453 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:07.453 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:07.453 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.453 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.454 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.454 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.454 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.454 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.454 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.454 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.454 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.454 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.454 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.454 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.454 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.454 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.454 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.454 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.455 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.455 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.455 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.455 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:07.455 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:07.455 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.455 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.455 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.455 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.455 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.455 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.455 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.455 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.455 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.455 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.456 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.456 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.456 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:07.456 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:07.456 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.456 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.456 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.456 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:07.456 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.456 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.456 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.456 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.456 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.456 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.457 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.457 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.457 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.457 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.457 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.457 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.457 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.457 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.457 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.457 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:07.457 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:07.457 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.458 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.458 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.458 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.458 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.458 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.458 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.458 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.458 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.458 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.458 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.458 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.458 INFO:teuthology.orchestra.run.smithi151.stdout: "max_concurrent_clones": { 2024-05-21T22:06:07.458 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "max_concurrent_clones", 2024-05-21T22:06:07.459 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.459 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.459 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.459 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "4", 2024-05-21T22:06:07.459 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.459 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.459 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.459 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-21T22:06:07.459 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.459 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.459 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.459 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.460 INFO:teuthology.orchestra.run.smithi151.stdout: "periodic_async_work": { 2024-05-21T22:06:07.460 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "periodic_async_work", 2024-05-21T22:06:07.460 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.460 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.460 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.460 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.460 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.460 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.460 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.460 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Periodically check for async work", 2024-05-21T22:06:07.460 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.460 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.460 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.460 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.461 INFO:teuthology.orchestra.run.smithi151.stdout: "snapshot_clone_delay": { 2024-05-21T22:06:07.461 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "snapshot_clone_delay", 2024-05-21T22:06:07.461 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.461 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.461 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.461 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:07.461 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.461 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.461 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.461 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-21T22:06:07.461 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.461 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.461 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.461 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.461 INFO:teuthology.orchestra.run.smithi151.stdout: "snapshot_clone_no_wait": { 2024-05-21T22:06:07.462 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "snapshot_clone_no_wait", 2024-05-21T22:06:07.462 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.462 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.462 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.462 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:07.462 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.462 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.462 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.462 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-21T22:06:07.462 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.462 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.462 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.462 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.462 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:07.463 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:07.463 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.463 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:07.463 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.463 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:07.463 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.463 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.463 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.463 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.463 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.463 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.463 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.463 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.463 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.464 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.464 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:07.464 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "zabbix", 2024-05-21T22:06:07.464 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:07.464 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:07.464 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:07.464 INFO:teuthology.orchestra.run.smithi151.stdout: "discovery_interval": { 2024-05-21T22:06:07.464 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "discovery_interval", 2024-05-21T22:06:07.464 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "uint", 2024-05-21T22:06:07.464 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.464 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.464 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "100", 2024-05-21T22:06:07.464 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.464 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.464 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.465 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.465 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.465 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.465 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.465 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.465 INFO:teuthology.orchestra.run.smithi151.stdout: "identifier": { 2024-05-21T22:06:07.465 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "identifier", 2024-05-21T22:06:07.465 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.465 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.465 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.465 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.465 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.465 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.465 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.465 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.466 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.466 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.466 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.466 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.466 INFO:teuthology.orchestra.run.smithi151.stdout: "interval": { 2024-05-21T22:06:07.466 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "interval", 2024-05-21T22:06:07.466 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-05-21T22:06:07.466 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.466 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.466 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "60", 2024-05-21T22:06:07.466 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.466 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.467 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.467 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.467 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.467 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.467 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.467 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.467 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:07.467 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:07.467 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.467 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.467 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.467 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.467 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.468 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.468 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.468 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.468 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.468 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.468 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.468 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.468 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.468 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.468 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.468 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.468 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.469 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.469 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.469 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:07.469 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:07.469 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.469 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.469 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.469 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.469 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.469 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.469 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.469 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.469 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.469 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.470 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.470 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.470 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:07.470 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:07.470 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.470 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.470 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.470 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:07.470 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.470 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.470 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.470 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.471 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.471 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.471 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.471 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.471 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.471 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.471 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.471 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.471 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.471 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.471 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.471 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:07.472 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:07.472 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.472 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.472 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.472 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.472 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.472 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.472 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.472 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.472 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.472 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.472 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.472 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.472 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:07.472 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:07.473 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.473 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:07.473 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.473 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:07.473 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.473 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.473 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.473 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.473 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.473 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.473 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.473 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.473 INFO:teuthology.orchestra.run.smithi151.stdout: "zabbix_host": { 2024-05-21T22:06:07.474 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "zabbix_host", 2024-05-21T22:06:07.474 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.474 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.474 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.474 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.474 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.474 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.474 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.474 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.474 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.474 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.474 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.474 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.475 INFO:teuthology.orchestra.run.smithi151.stdout: "zabbix_port": { 2024-05-21T22:06:07.475 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "zabbix_port", 2024-05-21T22:06:07.475 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.475 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.475 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.475 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "10051", 2024-05-21T22:06:07.475 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.475 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.475 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.475 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.475 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.475 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.475 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.475 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.476 INFO:teuthology.orchestra.run.smithi151.stdout: "zabbix_sender": { 2024-05-21T22:06:07.476 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "zabbix_sender", 2024-05-21T22:06:07.476 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.476 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.476 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.476 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-21T22:06:07.476 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.476 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.476 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.476 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.476 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.476 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.476 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.476 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.476 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.476 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.477 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-05-21T22:06:07.477 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.477 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.477 INFO:teuthology.orchestra.run.smithi151.stdout: "modules": [ 2024-05-21T22:06:07.477 INFO:teuthology.orchestra.run.smithi151.stdout: "cephadm", 2024-05-21T22:06:07.477 INFO:teuthology.orchestra.run.smithi151.stdout: "dashboard", 2024-05-21T22:06:07.477 INFO:teuthology.orchestra.run.smithi151.stdout: "iostat", 2024-05-21T22:06:07.477 INFO:teuthology.orchestra.run.smithi151.stdout: "nfs", 2024-05-21T22:06:07.477 INFO:teuthology.orchestra.run.smithi151.stdout: "restful", 2024-05-21T22:06:07.477 INFO:teuthology.orchestra.run.smithi151.stdout: "snap_schedule" 2024-05-21T22:06:07.477 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.477 INFO:teuthology.orchestra.run.smithi151.stdout: "available_modules": [ 2024-05-21T22:06:07.477 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:07.477 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "alerts", 2024-05-21T22:06:07.478 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:07.478 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:07.478 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:07.478 INFO:teuthology.orchestra.run.smithi151.stdout: "interval": { 2024-05-21T22:06:07.478 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "interval", 2024-05-21T22:06:07.478 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-05-21T22:06:07.478 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.478 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.478 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "60", 2024-05-21T22:06:07.478 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.478 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.478 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.478 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "How frequently to reexamine health status", 2024-05-21T22:06:07.479 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.479 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.479 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.479 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.479 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:07.479 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:07.479 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.479 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.479 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.479 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.479 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.479 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.479 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.479 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.480 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.480 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.480 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.480 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.480 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.480 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.480 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.480 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.480 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.480 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.480 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.480 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:07.480 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:07.480 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.480 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.481 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.481 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.481 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.481 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.481 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.481 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.481 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.481 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.481 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.481 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.481 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:07.481 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:07.481 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.481 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.481 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.482 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:07.482 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.482 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.482 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.482 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.482 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.482 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.482 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.482 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.482 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.482 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.482 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.482 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.482 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.482 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.483 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.483 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:07.483 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:07.483 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.483 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.483 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.483 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.483 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.483 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.483 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.483 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.483 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.483 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.484 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.484 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.484 INFO:teuthology.orchestra.run.smithi151.stdout: "smtp_destination": { 2024-05-21T22:06:07.484 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "smtp_destination", 2024-05-21T22:06:07.484 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.484 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.484 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.484 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.484 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.484 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.484 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.485 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Email address to send alerts to", 2024-05-21T22:06:07.485 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.485 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.485 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.485 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.485 INFO:teuthology.orchestra.run.smithi151.stdout: "smtp_from_name": { 2024-05-21T22:06:07.485 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "smtp_from_name", 2024-05-21T22:06:07.485 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.485 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.485 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.486 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "Ceph", 2024-05-21T22:06:07.486 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.486 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.486 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.486 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Email From: name", 2024-05-21T22:06:07.486 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.486 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.486 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.486 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.486 INFO:teuthology.orchestra.run.smithi151.stdout: "smtp_host": { 2024-05-21T22:06:07.486 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "smtp_host", 2024-05-21T22:06:07.486 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.487 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.487 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.487 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.487 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.487 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.487 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.487 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "SMTP server", 2024-05-21T22:06:07.487 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.487 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.487 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.487 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.487 INFO:teuthology.orchestra.run.smithi151.stdout: "smtp_password": { 2024-05-21T22:06:07.488 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "smtp_password", 2024-05-21T22:06:07.488 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.488 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.488 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.488 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.488 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.488 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.488 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.488 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Password to authenticate with", 2024-05-21T22:06:07.488 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.489 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.489 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.489 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.489 INFO:teuthology.orchestra.run.smithi151.stdout: "smtp_port": { 2024-05-21T22:06:07.489 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "smtp_port", 2024-05-21T22:06:07.489 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.489 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.489 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.489 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "465", 2024-05-21T22:06:07.489 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.489 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.489 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.489 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "SMTP port", 2024-05-21T22:06:07.490 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.490 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.490 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.490 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.490 INFO:teuthology.orchestra.run.smithi151.stdout: "smtp_sender": { 2024-05-21T22:06:07.490 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "smtp_sender", 2024-05-21T22:06:07.490 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.490 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.490 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.490 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.490 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.491 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.491 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.491 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "SMTP envelope sender", 2024-05-21T22:06:07.491 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.491 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.491 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.491 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.491 INFO:teuthology.orchestra.run.smithi151.stdout: "smtp_ssl": { 2024-05-21T22:06:07.491 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "smtp_ssl", 2024-05-21T22:06:07.491 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.491 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.491 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.491 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:07.491 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.492 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.492 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.492 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-21T22:06:07.492 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.492 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.492 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.492 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.492 INFO:teuthology.orchestra.run.smithi151.stdout: "smtp_user": { 2024-05-21T22:06:07.492 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "smtp_user", 2024-05-21T22:06:07.492 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.492 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.492 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.492 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.493 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.493 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.493 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.493 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "User to authenticate as", 2024-05-21T22:06:07.493 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.493 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.493 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.493 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.493 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:07.493 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:07.493 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.494 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:07.494 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.494 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:07.494 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.494 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.494 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.494 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.494 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.494 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.494 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.494 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.494 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.495 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.495 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:07.495 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "balancer", 2024-05-21T22:06:07.495 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:07.495 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:07.495 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:07.495 INFO:teuthology.orchestra.run.smithi151.stdout: "active": { 2024-05-21T22:06:07.495 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "active", 2024-05-21T22:06:07.495 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.495 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.495 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.496 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:07.496 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.496 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.496 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.496 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "automatically balance PGs across cluster", 2024-05-21T22:06:07.496 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.496 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.496 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.496 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.496 INFO:teuthology.orchestra.run.smithi151.stdout: "begin_time": { 2024-05-21T22:06:07.496 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "begin_time", 2024-05-21T22:06:07.496 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.496 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.496 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.497 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0000", 2024-05-21T22:06:07.497 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.497 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.497 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.497 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "beginning time of day to automatically balance", 2024-05-21T22:06:07.497 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-21T22:06:07.497 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.497 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.497 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.497 INFO:teuthology.orchestra.run.smithi151.stdout: "begin_weekday": { 2024-05-21T22:06:07.497 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "begin_weekday", 2024-05-21T22:06:07.497 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "uint", 2024-05-21T22:06:07.497 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.497 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.497 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:07.498 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "0", 2024-05-21T22:06:07.498 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "6", 2024-05-21T22:06:07.498 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.498 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-21T22:06:07.498 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-21T22:06:07.498 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.498 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.498 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.498 INFO:teuthology.orchestra.run.smithi151.stdout: "crush_compat_max_iterations": { 2024-05-21T22:06:07.498 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "crush_compat_max_iterations", 2024-05-21T22:06:07.498 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "uint", 2024-05-21T22:06:07.498 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.498 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.499 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "25", 2024-05-21T22:06:07.499 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "1", 2024-05-21T22:06:07.499 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "250", 2024-05-21T22:06:07.499 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.499 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-21T22:06:07.499 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.499 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.499 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.499 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.499 INFO:teuthology.orchestra.run.smithi151.stdout: "crush_compat_metrics": { 2024-05-21T22:06:07.499 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "crush_compat_metrics", 2024-05-21T22:06:07.499 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.499 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.499 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.500 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "pgs,objects,bytes", 2024-05-21T22:06:07.500 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.500 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.500 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.500 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-21T22:06:07.500 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-21T22:06:07.500 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.500 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.500 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.500 INFO:teuthology.orchestra.run.smithi151.stdout: "crush_compat_step": { 2024-05-21T22:06:07.500 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "crush_compat_step", 2024-05-21T22:06:07.500 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "float", 2024-05-21T22:06:07.500 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.500 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.500 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0.5", 2024-05-21T22:06:07.501 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "0.001", 2024-05-21T22:06:07.501 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "0.999", 2024-05-21T22:06:07.501 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.501 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "aggressiveness of optimization", 2024-05-21T22:06:07.501 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-21T22:06:07.501 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.501 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.501 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.501 INFO:teuthology.orchestra.run.smithi151.stdout: "end_time": { 2024-05-21T22:06:07.501 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "end_time", 2024-05-21T22:06:07.501 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.501 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.501 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.501 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "2359", 2024-05-21T22:06:07.502 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.502 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.502 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.502 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "ending time of day to automatically balance", 2024-05-21T22:06:07.502 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-21T22:06:07.502 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.502 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.502 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.502 INFO:teuthology.orchestra.run.smithi151.stdout: "end_weekday": { 2024-05-21T22:06:07.502 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "end_weekday", 2024-05-21T22:06:07.502 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "uint", 2024-05-21T22:06:07.502 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.502 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.503 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:07.503 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "0", 2024-05-21T22:06:07.503 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "6", 2024-05-21T22:06:07.503 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.503 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-21T22:06:07.503 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-21T22:06:07.503 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.503 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.503 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.503 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:07.503 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:07.503 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.503 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.504 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.504 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.504 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.504 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.504 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.504 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.504 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.504 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.504 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.504 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.504 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.504 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.504 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.504 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.505 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.505 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.505 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.505 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:07.505 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:07.505 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.505 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.505 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.505 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.505 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.505 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.505 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.505 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.506 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.506 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.506 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.506 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.506 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:07.506 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:07.506 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.506 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.506 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.506 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:07.506 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.506 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.507 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.507 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.507 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.507 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.507 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.507 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.507 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.507 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.507 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.507 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.507 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.507 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.508 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.508 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:07.508 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:07.508 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.508 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.508 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.508 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.508 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.508 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.508 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.508 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.508 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.509 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.509 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.509 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.509 INFO:teuthology.orchestra.run.smithi151.stdout: "min_score": { 2024-05-21T22:06:07.509 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "min_score", 2024-05-21T22:06:07.509 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "float", 2024-05-21T22:06:07.509 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.509 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.509 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:07.509 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.509 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.509 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.509 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-21T22:06:07.510 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.510 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.510 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.510 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.510 INFO:teuthology.orchestra.run.smithi151.stdout: "mode": { 2024-05-21T22:06:07.510 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "mode", 2024-05-21T22:06:07.510 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.510 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.510 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.510 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "upmap", 2024-05-21T22:06:07.510 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.511 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.511 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.511 INFO:teuthology.orchestra.run.smithi151.stdout: "crush-compat", 2024-05-21T22:06:07.511 INFO:teuthology.orchestra.run.smithi151.stdout: "none", 2024-05-21T22:06:07.511 INFO:teuthology.orchestra.run.smithi151.stdout: "read", 2024-05-21T22:06:07.511 INFO:teuthology.orchestra.run.smithi151.stdout: "upmap", 2024-05-21T22:06:07.511 INFO:teuthology.orchestra.run.smithi151.stdout: "upmap-read" 2024-05-21T22:06:07.511 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.511 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Balancer mode", 2024-05-21T22:06:07.511 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.511 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.511 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.511 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.511 INFO:teuthology.orchestra.run.smithi151.stdout: "pool_ids": { 2024-05-21T22:06:07.512 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "pool_ids", 2024-05-21T22:06:07.512 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.512 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.512 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.512 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.512 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.512 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.512 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.512 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-21T22:06:07.512 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.512 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.512 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.512 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.513 INFO:teuthology.orchestra.run.smithi151.stdout: "sleep_interval": { 2024-05-21T22:06:07.513 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sleep_interval", 2024-05-21T22:06:07.513 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-05-21T22:06:07.513 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.513 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.513 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "60", 2024-05-21T22:06:07.513 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.513 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.513 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.513 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-21T22:06:07.513 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.513 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.513 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.514 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.514 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:07.514 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:07.514 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.514 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:07.514 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.514 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:07.514 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.514 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.514 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.514 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.514 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.514 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.515 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.515 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.515 INFO:teuthology.orchestra.run.smithi151.stdout: "upmap_max_deviation": { 2024-05-21T22:06:07.515 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "upmap_max_deviation", 2024-05-21T22:06:07.515 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.515 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.515 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.515 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "5", 2024-05-21T22:06:07.515 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "1", 2024-05-21T22:06:07.515 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.515 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.515 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-21T22:06:07.515 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-21T22:06:07.516 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.516 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.516 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.516 INFO:teuthology.orchestra.run.smithi151.stdout: "upmap_max_optimizations": { 2024-05-21T22:06:07.516 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "upmap_max_optimizations", 2024-05-21T22:06:07.516 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "uint", 2024-05-21T22:06:07.516 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.516 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.516 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "10", 2024-05-21T22:06:07.516 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.516 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.516 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.516 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-21T22:06:07.517 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.517 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.517 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.517 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.517 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.517 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.517 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:07.517 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "cephadm", 2024-05-21T22:06:07.517 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:07.517 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:07.517 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:07.517 INFO:teuthology.orchestra.run.smithi151.stdout: "agent_down_multiplier": { 2024-05-21T22:06:07.517 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "agent_down_multiplier", 2024-05-21T22:06:07.518 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "float", 2024-05-21T22:06:07.518 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.518 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.518 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "3.0", 2024-05-21T22:06:07.518 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.518 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.518 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.518 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-21T22:06:07.518 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.518 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.518 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.518 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.518 INFO:teuthology.orchestra.run.smithi151.stdout: "agent_refresh_rate": { 2024-05-21T22:06:07.519 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "agent_refresh_rate", 2024-05-21T22:06:07.519 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-05-21T22:06:07.519 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.519 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.519 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "20", 2024-05-21T22:06:07.519 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.519 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.519 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.519 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-21T22:06:07.519 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.519 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.519 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.519 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.520 INFO:teuthology.orchestra.run.smithi151.stdout: "agent_starting_port": { 2024-05-21T22:06:07.520 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "agent_starting_port", 2024-05-21T22:06:07.520 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.520 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.520 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.520 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "4721", 2024-05-21T22:06:07.520 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.520 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.520 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.520 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-21T22:06:07.520 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.520 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.520 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.521 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.521 INFO:teuthology.orchestra.run.smithi151.stdout: "allow_ptrace": { 2024-05-21T22:06:07.521 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "allow_ptrace", 2024-05-21T22:06:07.521 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.521 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.521 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.521 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.521 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.521 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.521 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.521 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-21T22:06:07.521 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-21T22:06:07.521 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.522 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.522 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.522 INFO:teuthology.orchestra.run.smithi151.stdout: "autotune_interval": { 2024-05-21T22:06:07.522 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "autotune_interval", 2024-05-21T22:06:07.522 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-05-21T22:06:07.522 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.522 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.522 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "600", 2024-05-21T22:06:07.522 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.522 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.522 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.522 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-21T22:06:07.522 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.523 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.523 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.523 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.523 INFO:teuthology.orchestra.run.smithi151.stdout: "autotune_memory_target_ratio": { 2024-05-21T22:06:07.523 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "autotune_memory_target_ratio", 2024-05-21T22:06:07.523 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "float", 2024-05-21T22:06:07.523 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.523 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.523 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0.7", 2024-05-21T22:06:07.523 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.523 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.523 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.523 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-21T22:06:07.524 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.524 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.524 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.524 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.524 INFO:teuthology.orchestra.run.smithi151.stdout: "cephadm_log_destination": { 2024-05-21T22:06:07.524 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "cephadm_log_destination", 2024-05-21T22:06:07.524 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.524 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.524 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.524 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.524 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.524 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.524 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.524 INFO:teuthology.orchestra.run.smithi151.stdout: "file", 2024-05-21T22:06:07.525 INFO:teuthology.orchestra.run.smithi151.stdout: "file,syslog", 2024-05-21T22:06:07.525 INFO:teuthology.orchestra.run.smithi151.stdout: "syslog" 2024-05-21T22:06:07.525 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.525 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-21T22:06:07.525 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.525 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.525 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.525 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.525 INFO:teuthology.orchestra.run.smithi151.stdout: "cgroups_split": { 2024-05-21T22:06:07.525 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "cgroups_split", 2024-05-21T22:06:07.525 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.526 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.526 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.526 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:07.526 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.526 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.526 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.526 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-21T22:06:07.526 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.526 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.526 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.526 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.526 INFO:teuthology.orchestra.run.smithi151.stdout: "config_checks_enabled": { 2024-05-21T22:06:07.527 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "config_checks_enabled", 2024-05-21T22:06:07.527 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.527 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.527 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.527 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.527 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.527 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.527 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.527 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-21T22:06:07.527 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.527 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.527 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.527 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.528 INFO:teuthology.orchestra.run.smithi151.stdout: "config_dashboard": { 2024-05-21T22:06:07.528 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "config_dashboard", 2024-05-21T22:06:07.528 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.528 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.528 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.528 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:07.528 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.528 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.528 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.528 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-21T22:06:07.528 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.529 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.529 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.529 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.529 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_alertmanager": { 2024-05-21T22:06:07.529 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_alertmanager", 2024-05-21T22:06:07.529 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.529 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.529 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.529 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-21T22:06:07.529 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.529 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.529 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.529 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Prometheus container image", 2024-05-21T22:06:07.530 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.530 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.530 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.530 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.530 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_base": { 2024-05-21T22:06:07.530 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_base", 2024-05-21T22:06:07.530 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.530 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.530 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.530 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-21T22:06:07.530 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.530 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.530 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.531 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Container image name, without the tag", 2024-05-21T22:06:07.531 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.531 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.531 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.531 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.531 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_elasticsearch": { 2024-05-21T22:06:07.531 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_elasticsearch", 2024-05-21T22:06:07.531 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.531 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.531 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.531 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-21T22:06:07.532 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.532 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.532 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.532 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "elasticsearch container image", 2024-05-21T22:06:07.532 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.532 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.532 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.532 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.532 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_grafana": { 2024-05-21T22:06:07.532 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_grafana", 2024-05-21T22:06:07.532 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.532 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.533 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.533 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-21T22:06:07.533 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.533 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.533 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.533 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Prometheus container image", 2024-05-21T22:06:07.533 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.533 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.533 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.533 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.533 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_haproxy": { 2024-05-21T22:06:07.533 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_haproxy", 2024-05-21T22:06:07.533 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.534 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.534 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.534 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-21T22:06:07.534 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.534 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.534 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.534 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "HAproxy container image", 2024-05-21T22:06:07.534 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.534 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.534 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.534 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.534 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_jaeger_agent": { 2024-05-21T22:06:07.535 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_jaeger_agent", 2024-05-21T22:06:07.535 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.535 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.535 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.535 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-21T22:06:07.535 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.535 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.535 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.535 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Jaeger agent container image", 2024-05-21T22:06:07.535 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.535 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.535 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.535 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.536 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_jaeger_collector": { 2024-05-21T22:06:07.536 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_jaeger_collector", 2024-05-21T22:06:07.536 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.536 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.536 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.536 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-21T22:06:07.536 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.536 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.536 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.536 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Jaeger collector container image", 2024-05-21T22:06:07.536 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.537 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.537 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.537 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.537 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_jaeger_query": { 2024-05-21T22:06:07.537 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_jaeger_query", 2024-05-21T22:06:07.537 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.537 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.537 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.537 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-21T22:06:07.537 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.537 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.538 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.538 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Jaeger query container image", 2024-05-21T22:06:07.538 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.538 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.538 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.538 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.538 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_keepalived": { 2024-05-21T22:06:07.538 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_keepalived", 2024-05-21T22:06:07.538 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.538 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.538 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.538 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-21T22:06:07.538 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.539 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.539 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.539 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Keepalived container image", 2024-05-21T22:06:07.539 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.539 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.539 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.539 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.539 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_loki": { 2024-05-21T22:06:07.539 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_loki", 2024-05-21T22:06:07.539 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.539 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.539 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.539 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-05-21T22:06:07.539 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.540 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.540 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.540 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Loki container image", 2024-05-21T22:06:07.540 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.540 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.540 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.540 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.540 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_node_exporter": { 2024-05-21T22:06:07.540 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_node_exporter", 2024-05-21T22:06:07.540 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.540 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.541 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.541 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-21T22:06:07.541 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.541 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.541 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.541 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Prometheus container image", 2024-05-21T22:06:07.541 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.541 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.541 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.541 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.541 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_nvmeof": { 2024-05-21T22:06:07.541 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_nvmeof", 2024-05-21T22:06:07.542 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.542 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.542 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.542 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-05-21T22:06:07.542 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.542 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.542 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.542 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Nvme-of container image", 2024-05-21T22:06:07.542 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.542 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.542 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.542 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.542 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_prometheus": { 2024-05-21T22:06:07.543 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_prometheus", 2024-05-21T22:06:07.543 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.543 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.543 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.543 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-21T22:06:07.543 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.543 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.543 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.543 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Prometheus container image", 2024-05-21T22:06:07.543 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.543 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.543 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.544 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.544 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_promtail": { 2024-05-21T22:06:07.544 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_promtail", 2024-05-21T22:06:07.544 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.544 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.544 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.544 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-05-21T22:06:07.544 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.544 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.544 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.544 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Promtail container image", 2024-05-21T22:06:07.544 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.544 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.545 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.545 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.545 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_samba": { 2024-05-21T22:06:07.545 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_samba", 2024-05-21T22:06:07.545 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.545 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.545 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.545 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-21T22:06:07.545 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.545 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.545 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.545 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Samba/SMB container image", 2024-05-21T22:06:07.545 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.546 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.546 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.546 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.546 INFO:teuthology.orchestra.run.smithi151.stdout: "container_image_snmp_gateway": { 2024-05-21T22:06:07.546 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_image_snmp_gateway", 2024-05-21T22:06:07.546 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.546 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.546 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.546 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-21T22:06:07.546 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.546 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.546 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.547 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "SNMP Gateway container image", 2024-05-21T22:06:07.547 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.547 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.547 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.547 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.547 INFO:teuthology.orchestra.run.smithi151.stdout: "container_init": { 2024-05-21T22:06:07.547 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "container_init", 2024-05-21T22:06:07.547 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.547 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.547 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.547 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:07.547 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.547 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.548 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.548 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Run podman/docker with `--init`", 2024-05-21T22:06:07.548 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.548 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.548 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.548 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.548 INFO:teuthology.orchestra.run.smithi151.stdout: "daemon_cache_timeout": { 2024-05-21T22:06:07.548 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "daemon_cache_timeout", 2024-05-21T22:06:07.548 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-05-21T22:06:07.548 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.548 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.548 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "600", 2024-05-21T22:06:07.548 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.549 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.549 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.549 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-21T22:06:07.549 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.549 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.549 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.549 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.549 INFO:teuthology.orchestra.run.smithi151.stdout: "default_cephadm_command_timeout": { 2024-05-21T22:06:07.549 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "default_cephadm_command_timeout", 2024-05-21T22:06:07.549 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.549 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.549 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.549 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "900", 2024-05-21T22:06:07.549 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.550 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.550 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.550 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-21T22:06:07.550 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.550 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.550 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.550 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.550 INFO:teuthology.orchestra.run.smithi151.stdout: "default_registry": { 2024-05-21T22:06:07.550 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "default_registry", 2024-05-21T22:06:07.550 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.550 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.550 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.550 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "docker.io", 2024-05-21T22:06:07.550 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.551 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.551 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.551 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-21T22:06:07.551 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.551 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.551 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.551 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.551 INFO:teuthology.orchestra.run.smithi151.stdout: "device_cache_timeout": { 2024-05-21T22:06:07.551 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "device_cache_timeout", 2024-05-21T22:06:07.551 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-05-21T22:06:07.551 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.551 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.551 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "1800", 2024-05-21T22:06:07.551 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.552 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.552 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.552 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "seconds to cache device inventory", 2024-05-21T22:06:07.552 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.552 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.552 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.552 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.552 INFO:teuthology.orchestra.run.smithi151.stdout: "device_enhanced_scan": { 2024-05-21T22:06:07.552 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "device_enhanced_scan", 2024-05-21T22:06:07.552 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.552 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.552 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.552 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.552 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.553 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.553 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.553 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-21T22:06:07.553 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.553 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.553 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.553 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.553 INFO:teuthology.orchestra.run.smithi151.stdout: "facts_cache_timeout": { 2024-05-21T22:06:07.553 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "facts_cache_timeout", 2024-05-21T22:06:07.553 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-05-21T22:06:07.553 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.553 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.553 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "60", 2024-05-21T22:06:07.554 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.554 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.554 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.554 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "seconds to cache host facts data", 2024-05-21T22:06:07.554 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.554 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.554 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.554 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.554 INFO:teuthology.orchestra.run.smithi151.stdout: "grafana_dashboards_path": { 2024-05-21T22:06:07.554 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "grafana_dashboards_path", 2024-05-21T22:06:07.554 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.554 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.554 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.555 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-21T22:06:07.555 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.555 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.555 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.555 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-21T22:06:07.555 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.555 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.555 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.555 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.555 INFO:teuthology.orchestra.run.smithi151.stdout: "host_check_interval": { 2024-05-21T22:06:07.555 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "host_check_interval", 2024-05-21T22:06:07.555 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-05-21T22:06:07.555 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.555 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.555 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "600", 2024-05-21T22:06:07.556 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.556 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.556 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.556 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "how frequently to perform a host check", 2024-05-21T22:06:07.556 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.556 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.556 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.556 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.556 INFO:teuthology.orchestra.run.smithi151.stdout: "hw_monitoring": { 2024-05-21T22:06:07.556 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "hw_monitoring", 2024-05-21T22:06:07.556 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.556 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.556 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.557 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.557 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.557 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.557 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.557 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-21T22:06:07.558 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.558 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.558 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.558 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.558 INFO:teuthology.orchestra.run.smithi151.stdout: "inventory_list_all": { 2024-05-21T22:06:07.558 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "inventory_list_all", 2024-05-21T22:06:07.558 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.558 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.558 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.558 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.558 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.558 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.559 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.559 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-21T22:06:07.559 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.559 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.559 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.559 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.559 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:07.559 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:07.559 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.559 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.559 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.559 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.559 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.560 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.560 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.560 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.560 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.560 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.560 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.560 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.560 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.560 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.560 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.560 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.560 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.561 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.561 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.561 INFO:teuthology.orchestra.run.smithi151.stdout: "log_refresh_metadata": { 2024-05-21T22:06:07.561 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_refresh_metadata", 2024-05-21T22:06:07.561 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.561 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.561 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.561 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.561 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.561 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.561 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.561 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-21T22:06:07.561 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.561 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.562 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.562 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.562 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:07.562 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:07.562 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.562 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.562 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.562 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:07.562 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.562 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.562 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.562 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-21T22:06:07.563 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.563 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.563 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.563 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.563 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:07.563 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:07.563 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.563 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.563 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.563 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:07.563 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.563 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.563 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.563 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.564 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.564 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.564 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.564 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.564 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.564 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.564 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.564 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.564 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.564 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.564 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.564 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:07.564 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:07.564 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.565 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.565 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.565 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.565 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.565 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.565 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.565 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.565 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.565 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.565 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.565 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.565 INFO:teuthology.orchestra.run.smithi151.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-21T22:06:07.565 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-21T22:06:07.566 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.566 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.566 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.566 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.566 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.566 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.566 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.566 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-21T22:06:07.566 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.566 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.566 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.566 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.566 INFO:teuthology.orchestra.run.smithi151.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-21T22:06:07.567 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-21T22:06:07.567 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.567 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.567 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.567 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "*", 2024-05-21T22:06:07.567 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.567 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.567 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.567 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-21T22:06:07.567 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.567 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.567 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.567 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.567 INFO:teuthology.orchestra.run.smithi151.stdout: "max_count_per_host": { 2024-05-21T22:06:07.567 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "max_count_per_host", 2024-05-21T22:06:07.568 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.568 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.568 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.568 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "10", 2024-05-21T22:06:07.568 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.568 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.568 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.568 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "max number of daemons per service per host", 2024-05-21T22:06:07.568 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.568 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.568 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.568 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.568 INFO:teuthology.orchestra.run.smithi151.stdout: "max_osd_draining_count": { 2024-05-21T22:06:07.569 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "max_osd_draining_count", 2024-05-21T22:06:07.569 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.569 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.569 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.569 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "10", 2024-05-21T22:06:07.569 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.569 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.569 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.569 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-21T22:06:07.569 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.569 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.570 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.570 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.570 INFO:teuthology.orchestra.run.smithi151.stdout: "migration_current": { 2024-05-21T22:06:07.570 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "migration_current", 2024-05-21T22:06:07.570 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.570 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.570 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.570 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.570 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.570 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.570 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.570 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "internal - do not modify", 2024-05-21T22:06:07.571 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.571 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.571 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.571 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.571 INFO:teuthology.orchestra.run.smithi151.stdout: "mode": { 2024-05-21T22:06:07.571 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "mode", 2024-05-21T22:06:07.571 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.571 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.571 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.571 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "root", 2024-05-21T22:06:07.571 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.571 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.571 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.572 INFO:teuthology.orchestra.run.smithi151.stdout: "cephadm-package", 2024-05-21T22:06:07.572 INFO:teuthology.orchestra.run.smithi151.stdout: "root" 2024-05-21T22:06:07.572 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.572 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "mode for remote execution of cephadm", 2024-05-21T22:06:07.572 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.572 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.572 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.572 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.572 INFO:teuthology.orchestra.run.smithi151.stdout: "oob_default_addr": { 2024-05-21T22:06:07.572 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "oob_default_addr", 2024-05-21T22:06:07.572 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.572 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.572 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.572 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "169.254.1.1", 2024-05-21T22:06:07.572 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.573 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.573 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.573 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-21T22:06:07.573 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.573 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.573 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.573 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.573 INFO:teuthology.orchestra.run.smithi151.stdout: "prometheus_alerts_path": { 2024-05-21T22:06:07.573 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "prometheus_alerts_path", 2024-05-21T22:06:07.573 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.573 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.573 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.573 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-21T22:06:07.574 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.574 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.574 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.574 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-21T22:06:07.574 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.574 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.574 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.574 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.574 INFO:teuthology.orchestra.run.smithi151.stdout: "registry_insecure": { 2024-05-21T22:06:07.574 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "registry_insecure", 2024-05-21T22:06:07.574 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.574 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.574 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.575 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.575 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.575 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.575 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.575 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-21T22:06:07.575 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.575 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.575 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.575 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.575 INFO:teuthology.orchestra.run.smithi151.stdout: "registry_password": { 2024-05-21T22:06:07.575 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "registry_password", 2024-05-21T22:06:07.575 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.575 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.575 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.575 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.576 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.576 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.576 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.576 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-21T22:06:07.576 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.576 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.576 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.576 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.576 INFO:teuthology.orchestra.run.smithi151.stdout: "registry_url": { 2024-05-21T22:06:07.576 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "registry_url", 2024-05-21T22:06:07.576 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.576 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.576 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.577 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.577 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.577 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.577 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.577 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-21T22:06:07.577 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.577 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.577 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.577 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.577 INFO:teuthology.orchestra.run.smithi151.stdout: "registry_username": { 2024-05-21T22:06:07.577 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "registry_username", 2024-05-21T22:06:07.577 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.577 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.578 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.578 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.578 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.578 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.578 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.578 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-21T22:06:07.578 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.578 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.578 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.578 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.578 INFO:teuthology.orchestra.run.smithi151.stdout: "secure_monitoring_stack": { 2024-05-21T22:06:07.578 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "secure_monitoring_stack", 2024-05-21T22:06:07.579 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.579 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.579 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.579 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.579 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.579 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.579 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.579 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-21T22:06:07.579 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.579 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.579 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.579 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.579 INFO:teuthology.orchestra.run.smithi151.stdout: "service_discovery_port": { 2024-05-21T22:06:07.580 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "service_discovery_port", 2024-05-21T22:06:07.580 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.580 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.580 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.580 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "8765", 2024-05-21T22:06:07.580 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.580 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.580 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.580 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "cephadm service discovery port", 2024-05-21T22:06:07.580 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.580 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.580 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.581 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.581 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:07.581 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:07.581 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.581 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:07.581 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.581 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:07.581 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.581 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.581 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.581 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.581 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.581 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.582 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.582 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.582 INFO:teuthology.orchestra.run.smithi151.stdout: "ssh_config_file": { 2024-05-21T22:06:07.582 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "ssh_config_file", 2024-05-21T22:06:07.582 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.582 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.582 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.582 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.582 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.582 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.582 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.582 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-21T22:06:07.583 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.583 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.583 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.583 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.583 INFO:teuthology.orchestra.run.smithi151.stdout: "use_agent": { 2024-05-21T22:06:07.583 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "use_agent", 2024-05-21T22:06:07.583 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.583 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.583 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.583 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.583 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.583 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.584 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.584 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-21T22:06:07.584 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.584 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.584 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.584 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.584 INFO:teuthology.orchestra.run.smithi151.stdout: "use_repo_digest": { 2024-05-21T22:06:07.584 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "use_repo_digest", 2024-05-21T22:06:07.584 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.584 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.584 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.584 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:07.584 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.585 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.585 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.585 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-21T22:06:07.585 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.585 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.585 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.585 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.585 INFO:teuthology.orchestra.run.smithi151.stdout: "warn_on_failed_host_check": { 2024-05-21T22:06:07.585 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "warn_on_failed_host_check", 2024-05-21T22:06:07.585 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.585 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.585 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.586 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:07.586 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.586 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.586 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.586 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "raise a health warning if the host check fails", 2024-05-21T22:06:07.586 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.586 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.586 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.586 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.586 INFO:teuthology.orchestra.run.smithi151.stdout: "warn_on_stray_daemons": { 2024-05-21T22:06:07.586 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "warn_on_stray_daemons", 2024-05-21T22:06:07.586 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.587 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.587 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.587 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:07.587 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.587 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.587 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.587 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-21T22:06:07.587 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.587 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.587 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.587 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.587 INFO:teuthology.orchestra.run.smithi151.stdout: "warn_on_stray_hosts": { 2024-05-21T22:06:07.587 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "warn_on_stray_hosts", 2024-05-21T22:06:07.588 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.588 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.588 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.588 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:07.588 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.588 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.588 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.588 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-21T22:06:07.588 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.588 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.588 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.588 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.588 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.588 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.589 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:07.589 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "crash", 2024-05-21T22:06:07.589 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:07.589 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:07.589 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:07.589 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:07.589 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:07.589 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.589 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.589 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.589 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.589 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.589 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.589 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.589 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.590 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.590 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.590 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.590 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.590 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.590 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.590 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.590 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.590 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.590 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.590 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.590 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:07.590 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:07.591 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.591 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.591 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.591 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.591 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.591 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.591 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.591 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.591 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.591 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.591 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.591 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.591 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:07.591 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:07.591 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.592 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.592 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.592 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:07.592 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.592 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.592 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.592 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.592 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.592 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.592 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.592 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.592 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.592 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.592 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.592 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.593 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.593 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.593 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.593 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:07.593 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:07.593 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.593 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.593 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.593 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.593 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.593 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.593 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.593 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.593 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.593 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.594 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.594 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.594 INFO:teuthology.orchestra.run.smithi151.stdout: "retain_interval": { 2024-05-21T22:06:07.594 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "retain_interval", 2024-05-21T22:06:07.594 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-05-21T22:06:07.594 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.594 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.594 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "31536000", 2024-05-21T22:06:07.594 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.594 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.594 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.595 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-21T22:06:07.595 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.595 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.595 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.595 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.595 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:07.595 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:07.595 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.595 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:07.595 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.595 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:07.595 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.595 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.595 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.596 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.596 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.596 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.596 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.596 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.596 INFO:teuthology.orchestra.run.smithi151.stdout: "warn_recent_interval": { 2024-05-21T22:06:07.596 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "warn_recent_interval", 2024-05-21T22:06:07.596 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-05-21T22:06:07.596 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.596 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.596 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "1209600", 2024-05-21T22:06:07.596 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.597 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.597 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.597 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-21T22:06:07.597 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.597 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.597 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.597 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.597 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.597 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.597 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:07.597 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "dashboard", 2024-05-21T22:06:07.597 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:07.597 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:07.598 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:07.598 INFO:teuthology.orchestra.run.smithi151.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-21T22:06:07.598 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-21T22:06:07.598 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.598 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.598 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.598 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "10", 2024-05-21T22:06:07.598 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.598 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.598 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.598 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.598 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.598 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.599 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.599 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.599 INFO:teuthology.orchestra.run.smithi151.stdout: "ALERTMANAGER_API_HOST": { 2024-05-21T22:06:07.599 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-21T22:06:07.599 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.599 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.599 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.599 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.599 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.599 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.599 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.599 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.599 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.599 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.600 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.600 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.600 INFO:teuthology.orchestra.run.smithi151.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-21T22:06:07.600 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-21T22:06:07.600 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.600 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.600 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.600 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:07.600 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.600 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.600 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.600 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.600 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.601 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.601 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.601 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.601 INFO:teuthology.orchestra.run.smithi151.stdout: "AUDIT_API_ENABLED": { 2024-05-21T22:06:07.601 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "AUDIT_API_ENABLED", 2024-05-21T22:06:07.601 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.601 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.601 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.601 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.601 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.601 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.601 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.601 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.601 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.602 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.602 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.602 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.602 INFO:teuthology.orchestra.run.smithi151.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-21T22:06:07.602 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-21T22:06:07.602 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.602 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.602 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.602 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:07.602 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.602 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.603 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.603 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.603 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.603 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.603 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.603 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.603 INFO:teuthology.orchestra.run.smithi151.stdout: "ENABLE_BROWSABLE_API": { 2024-05-21T22:06:07.603 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-21T22:06:07.603 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.603 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.603 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.603 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:07.603 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.604 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.604 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.604 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.604 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.604 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.604 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.604 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.604 INFO:teuthology.orchestra.run.smithi151.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-21T22:06:07.604 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-21T22:06:07.604 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.604 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.604 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.604 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:07.604 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.604 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.605 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.605 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.605 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.605 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.605 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.605 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.605 INFO:teuthology.orchestra.run.smithi151.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-21T22:06:07.605 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-21T22:06:07.605 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.605 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.605 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.605 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:07.605 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.605 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.605 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.606 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.606 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.606 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.606 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.606 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.606 INFO:teuthology.orchestra.run.smithi151.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-21T22:06:07.606 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-21T22:06:07.606 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.606 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.606 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.606 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:07.606 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.606 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.606 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.607 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.607 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.607 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.607 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.607 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.607 INFO:teuthology.orchestra.run.smithi151.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-21T22:06:07.607 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-21T22:06:07.607 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.607 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.607 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.607 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:07.607 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.607 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.607 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.607 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.608 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.608 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.608 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.608 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.608 INFO:teuthology.orchestra.run.smithi151.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-21T22:06:07.608 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-21T22:06:07.608 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.608 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.608 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.608 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:07.608 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.609 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.609 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.609 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.609 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.609 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.609 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.609 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.609 INFO:teuthology.orchestra.run.smithi151.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-21T22:06:07.609 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-21T22:06:07.609 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.609 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.609 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.609 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:07.609 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.610 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.610 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.610 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.610 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.610 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.610 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.610 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.610 INFO:teuthology.orchestra.run.smithi151.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-21T22:06:07.610 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-21T22:06:07.610 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.610 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.610 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.610 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:07.610 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.611 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.611 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.611 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.611 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.611 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.611 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.611 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.611 INFO:teuthology.orchestra.run.smithi151.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-21T22:06:07.611 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-21T22:06:07.611 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.611 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.611 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.611 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.611 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.611 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.612 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.612 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.612 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.612 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.612 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.612 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.612 INFO:teuthology.orchestra.run.smithi151.stdout: "GRAFANA_API_PASSWORD": { 2024-05-21T22:06:07.612 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-21T22:06:07.612 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.612 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.612 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.612 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "admin", 2024-05-21T22:06:07.612 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.612 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.612 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.613 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.613 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.613 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.613 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.613 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.613 INFO:teuthology.orchestra.run.smithi151.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-21T22:06:07.613 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-21T22:06:07.613 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.613 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.613 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.613 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:07.613 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.613 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.613 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.614 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.614 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.614 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.614 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.614 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.614 INFO:teuthology.orchestra.run.smithi151.stdout: "GRAFANA_API_URL": { 2024-05-21T22:06:07.614 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "GRAFANA_API_URL", 2024-05-21T22:06:07.614 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.614 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.614 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.614 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.614 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.615 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.615 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.615 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.615 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.615 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.615 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.615 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.615 INFO:teuthology.orchestra.run.smithi151.stdout: "GRAFANA_API_USERNAME": { 2024-05-21T22:06:07.615 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-21T22:06:07.615 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.615 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.616 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.616 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "admin", 2024-05-21T22:06:07.616 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.616 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.616 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.616 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.616 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.616 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.616 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.616 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.616 INFO:teuthology.orchestra.run.smithi151.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-21T22:06:07.616 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-21T22:06:07.617 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.617 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.617 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.617 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.617 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.617 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.617 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.617 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.617 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.617 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.617 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.617 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.617 INFO:teuthology.orchestra.run.smithi151.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-21T22:06:07.617 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-21T22:06:07.617 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.618 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.618 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.618 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.618 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.618 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.618 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.618 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.618 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.618 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.618 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.618 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.618 INFO:teuthology.orchestra.run.smithi151.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-21T22:06:07.619 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-21T22:06:07.619 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.619 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.619 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.619 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:07.619 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.619 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.619 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.619 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.619 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.619 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.620 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.620 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.620 INFO:teuthology.orchestra.run.smithi151.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-21T22:06:07.620 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-21T22:06:07.620 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.620 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.620 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.620 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.620 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.620 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.620 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.620 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.621 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.621 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.621 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.621 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.621 INFO:teuthology.orchestra.run.smithi151.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-21T22:06:07.621 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-21T22:06:07.621 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.621 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.621 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.621 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "[]", 2024-05-21T22:06:07.621 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.621 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.621 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.622 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.622 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.622 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.622 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.622 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.622 INFO:teuthology.orchestra.run.smithi151.stdout: "MULTICLUSTER_CONFIG": { 2024-05-21T22:06:07.622 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-21T22:06:07.622 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.622 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.622 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.622 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "{}", 2024-05-21T22:06:07.622 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.622 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.623 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.623 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.623 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.623 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.623 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.623 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.623 INFO:teuthology.orchestra.run.smithi151.stdout: "PROMETHEUS_API_HOST": { 2024-05-21T22:06:07.623 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-21T22:06:07.623 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.623 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.623 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.623 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.623 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.624 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.624 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.624 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.624 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.624 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.624 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.624 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.624 INFO:teuthology.orchestra.run.smithi151.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-21T22:06:07.624 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-21T22:06:07.624 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.624 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.624 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.624 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:07.625 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.625 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.625 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.625 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.625 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.625 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.625 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.625 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.625 INFO:teuthology.orchestra.run.smithi151.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-21T22:06:07.625 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-21T22:06:07.625 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.625 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.626 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.626 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.626 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.626 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.626 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.626 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.626 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.626 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.626 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.626 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.626 INFO:teuthology.orchestra.run.smithi151.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-21T22:06:07.626 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-21T22:06:07.626 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.627 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.627 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.627 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.627 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.627 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.627 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.627 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.627 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.627 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.627 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.629 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.629 INFO:teuthology.orchestra.run.smithi151.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-21T22:06:07.629 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-21T22:06:07.630 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.630 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.630 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.630 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:07.630 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.630 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.630 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.630 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.630 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.630 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.630 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.630 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.630 INFO:teuthology.orchestra.run.smithi151.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-21T22:06:07.631 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-21T22:06:07.631 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.631 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.631 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.631 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:07.631 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.631 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.631 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.631 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.631 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.631 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.631 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.632 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.632 INFO:teuthology.orchestra.run.smithi151.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-21T22:06:07.632 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-21T22:06:07.632 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.632 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.632 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.632 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.632 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.632 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.632 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.632 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.632 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.632 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.632 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.633 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.633 INFO:teuthology.orchestra.run.smithi151.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-21T22:06:07.633 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-21T22:06:07.633 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.633 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.633 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.633 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.633 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.633 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.633 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.633 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.633 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.633 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.633 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.633 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.634 INFO:teuthology.orchestra.run.smithi151.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-21T22:06:07.634 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-21T22:06:07.634 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.634 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.634 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.634 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.634 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.634 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.634 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.634 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.634 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.634 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.634 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.635 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.635 INFO:teuthology.orchestra.run.smithi151.stdout: "PWD_POLICY_ENABLED": { 2024-05-21T22:06:07.635 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-21T22:06:07.635 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.635 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.635 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.635 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:07.635 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.635 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.635 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.635 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.635 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.635 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.635 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.636 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.636 INFO:teuthology.orchestra.run.smithi151.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-21T22:06:07.636 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-21T22:06:07.636 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.636 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.636 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.636 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-21T22:06:07.636 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.636 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.636 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.636 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.636 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.636 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.636 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.637 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.637 INFO:teuthology.orchestra.run.smithi151.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-21T22:06:07.637 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-21T22:06:07.637 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.637 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.637 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.637 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "10", 2024-05-21T22:06:07.637 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.637 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.637 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.637 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.637 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.637 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.638 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.638 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.638 INFO:teuthology.orchestra.run.smithi151.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-21T22:06:07.638 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-21T22:06:07.638 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.638 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.638 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.638 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "8", 2024-05-21T22:06:07.638 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.638 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.638 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.638 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.638 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.639 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.639 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.639 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.639 INFO:teuthology.orchestra.run.smithi151.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-21T22:06:07.639 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-21T22:06:07.639 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.639 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.639 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.639 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "45", 2024-05-21T22:06:07.639 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.639 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.639 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.639 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.639 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.640 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.640 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.640 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.640 INFO:teuthology.orchestra.run.smithi151.stdout: "RGW_API_ACCESS_KEY": { 2024-05-21T22:06:07.640 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-21T22:06:07.640 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.640 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.640 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.640 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.640 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.640 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.640 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.640 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.641 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.641 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.641 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.641 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.641 INFO:teuthology.orchestra.run.smithi151.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-21T22:06:07.641 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-21T22:06:07.641 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.641 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.641 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.641 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "admin", 2024-05-21T22:06:07.641 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.641 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.641 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.641 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.642 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.642 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.642 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.642 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.642 INFO:teuthology.orchestra.run.smithi151.stdout: "RGW_API_SECRET_KEY": { 2024-05-21T22:06:07.642 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-21T22:06:07.642 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.642 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.642 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.642 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.642 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.642 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.642 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.643 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.643 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.643 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.643 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.643 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.643 INFO:teuthology.orchestra.run.smithi151.stdout: "RGW_API_SSL_VERIFY": { 2024-05-21T22:06:07.643 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-21T22:06:07.643 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.643 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.643 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.643 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:07.643 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.643 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.643 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.644 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.644 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.644 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.644 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.644 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.644 INFO:teuthology.orchestra.run.smithi151.stdout: "UNSAFE_TLS_v1_2": { 2024-05-21T22:06:07.644 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-21T22:06:07.644 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.644 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.644 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.644 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.644 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.644 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.645 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.645 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.645 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.645 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.645 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.645 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.645 INFO:teuthology.orchestra.run.smithi151.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-21T22:06:07.645 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-21T22:06:07.645 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.645 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.645 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.645 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:07.645 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.646 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.646 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.646 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.646 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.646 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.646 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.646 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.646 INFO:teuthology.orchestra.run.smithi151.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-21T22:06:07.646 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-21T22:06:07.646 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.646 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.646 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.646 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "10", 2024-05-21T22:06:07.646 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.647 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.647 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.647 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.647 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.647 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.647 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.647 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.647 INFO:teuthology.orchestra.run.smithi151.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-21T22:06:07.647 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-21T22:06:07.647 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.647 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.647 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.647 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "5", 2024-05-21T22:06:07.647 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.648 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.648 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.648 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.648 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.648 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.648 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.648 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.648 INFO:teuthology.orchestra.run.smithi151.stdout: "cross_origin_url": { 2024-05-21T22:06:07.648 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "cross_origin_url", 2024-05-21T22:06:07.648 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.648 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.648 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.648 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.648 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.649 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.649 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.649 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.649 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.649 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.649 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.649 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.649 INFO:teuthology.orchestra.run.smithi151.stdout: "crt_file": { 2024-05-21T22:06:07.649 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "crt_file", 2024-05-21T22:06:07.649 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.649 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.649 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.649 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.649 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.650 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.650 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.650 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.650 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.650 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.650 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.650 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.650 INFO:teuthology.orchestra.run.smithi151.stdout: "debug": { 2024-05-21T22:06:07.650 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "debug", 2024-05-21T22:06:07.650 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.650 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.650 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.650 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.650 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.651 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.651 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.651 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Enable/disable debug options", 2024-05-21T22:06:07.651 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.651 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.651 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.651 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.651 INFO:teuthology.orchestra.run.smithi151.stdout: "jwt_token_ttl": { 2024-05-21T22:06:07.651 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "jwt_token_ttl", 2024-05-21T22:06:07.651 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.651 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.651 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.651 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "28800", 2024-05-21T22:06:07.651 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.652 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.652 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.652 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.652 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.652 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.652 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.652 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.652 INFO:teuthology.orchestra.run.smithi151.stdout: "key_file": { 2024-05-21T22:06:07.652 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "key_file", 2024-05-21T22:06:07.652 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.652 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.652 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.652 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.652 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.653 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.653 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.653 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.653 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.653 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.653 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.653 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.653 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:07.653 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:07.653 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.653 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.653 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.653 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.653 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.654 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.654 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.654 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.654 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.654 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.654 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.654 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.654 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.654 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.654 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.654 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.654 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.654 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.654 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.655 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:07.655 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:07.655 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.655 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.655 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.655 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.655 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.655 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.655 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.655 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.655 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.655 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.655 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.656 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.656 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:07.656 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:07.656 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.656 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.656 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.656 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:07.656 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.656 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.656 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.656 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.656 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.656 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.657 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.657 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.657 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.657 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.657 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.657 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.657 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.657 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.657 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.657 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:07.657 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:07.657 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.657 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.658 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.658 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.658 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.658 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.658 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.658 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.658 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.658 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.658 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.658 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.658 INFO:teuthology.orchestra.run.smithi151.stdout: "motd": { 2024-05-21T22:06:07.658 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "motd", 2024-05-21T22:06:07.658 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.659 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.659 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.659 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.659 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.659 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.659 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.659 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "The message of the day", 2024-05-21T22:06:07.659 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.659 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.659 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.659 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.659 INFO:teuthology.orchestra.run.smithi151.stdout: "redirect_resolve_ip_addr": { 2024-05-21T22:06:07.659 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "redirect_resolve_ip_addr", 2024-05-21T22:06:07.660 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.660 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.660 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.660 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.660 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.660 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.660 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.660 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.660 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.660 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.660 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.660 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.660 INFO:teuthology.orchestra.run.smithi151.stdout: "server_addr": { 2024-05-21T22:06:07.661 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "server_addr", 2024-05-21T22:06:07.661 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.661 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.661 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.661 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "::", 2024-05-21T22:06:07.661 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.661 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.661 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.661 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.661 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.661 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.661 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.661 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.661 INFO:teuthology.orchestra.run.smithi151.stdout: "server_port": { 2024-05-21T22:06:07.662 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "server_port", 2024-05-21T22:06:07.662 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.662 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.662 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.662 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "8080", 2024-05-21T22:06:07.662 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.662 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.662 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.662 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.662 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.662 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.662 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.662 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.663 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:07.663 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:07.663 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.663 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:07.663 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.663 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:07.663 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.663 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.663 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.663 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.663 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.663 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.663 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.663 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.664 INFO:teuthology.orchestra.run.smithi151.stdout: "ssl": { 2024-05-21T22:06:07.664 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "ssl", 2024-05-21T22:06:07.664 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.664 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.664 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.664 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:07.664 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.664 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.664 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.664 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.664 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.664 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.664 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.665 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.665 INFO:teuthology.orchestra.run.smithi151.stdout: "ssl_server_port": { 2024-05-21T22:06:07.665 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "ssl_server_port", 2024-05-21T22:06:07.665 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.665 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.665 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.665 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "8443", 2024-05-21T22:06:07.665 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.665 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.665 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.665 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.665 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.665 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.665 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.666 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.666 INFO:teuthology.orchestra.run.smithi151.stdout: "standby_behaviour": { 2024-05-21T22:06:07.666 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "standby_behaviour", 2024-05-21T22:06:07.666 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.666 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.666 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.666 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "redirect", 2024-05-21T22:06:07.666 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.666 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.666 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.666 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.666 INFO:teuthology.orchestra.run.smithi151.stdout: "redirect" 2024-05-21T22:06:07.666 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.667 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.667 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.667 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.667 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.667 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.667 INFO:teuthology.orchestra.run.smithi151.stdout: "standby_error_status_code": { 2024-05-21T22:06:07.667 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "standby_error_status_code", 2024-05-21T22:06:07.667 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.667 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.667 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.667 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "500", 2024-05-21T22:06:07.667 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "400", 2024-05-21T22:06:07.667 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "599", 2024-05-21T22:06:07.667 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.668 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.668 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.668 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.668 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.668 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.668 INFO:teuthology.orchestra.run.smithi151.stdout: "url_prefix": { 2024-05-21T22:06:07.668 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "url_prefix", 2024-05-21T22:06:07.668 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.668 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.668 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.668 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.668 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.668 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.669 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.669 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.669 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.669 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.669 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.669 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.669 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.669 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.669 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:07.669 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "devicehealth", 2024-05-21T22:06:07.669 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:07.669 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:07.669 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:07.670 INFO:teuthology.orchestra.run.smithi151.stdout: "enable_monitoring": { 2024-05-21T22:06:07.670 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "enable_monitoring", 2024-05-21T22:06:07.670 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.670 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.670 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.670 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:07.670 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.670 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.670 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.670 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "monitor device health metrics", 2024-05-21T22:06:07.670 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.670 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.670 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.671 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.671 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:07.671 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:07.671 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.671 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.671 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.671 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.671 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.671 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.671 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.671 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.671 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.671 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.671 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.672 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.672 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.672 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.672 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.672 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.672 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.672 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.672 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.672 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:07.672 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:07.672 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.672 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.672 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.673 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.673 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.673 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.673 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.673 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.673 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.673 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.673 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.673 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.673 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:07.673 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:07.673 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.673 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.674 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.674 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:07.674 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.674 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.674 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.674 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.674 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.674 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.674 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.674 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.674 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.674 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.674 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.675 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.675 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.675 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.675 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.675 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:07.675 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:07.675 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.675 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.675 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.675 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.675 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.675 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.675 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.675 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.676 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.676 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.676 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.676 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.676 INFO:teuthology.orchestra.run.smithi151.stdout: "mark_out_threshold": { 2024-05-21T22:06:07.676 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "mark_out_threshold", 2024-05-21T22:06:07.676 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-05-21T22:06:07.676 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.676 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.676 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "2419200", 2024-05-21T22:06:07.676 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.676 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.676 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.676 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-21T22:06:07.677 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.677 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.677 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.677 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.677 INFO:teuthology.orchestra.run.smithi151.stdout: "pool_name": { 2024-05-21T22:06:07.677 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "pool_name", 2024-05-21T22:06:07.677 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.677 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.677 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.677 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "device_health_metrics", 2024-05-21T22:06:07.677 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.677 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.677 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.678 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-21T22:06:07.678 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.678 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.678 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.678 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.678 INFO:teuthology.orchestra.run.smithi151.stdout: "retention_period": { 2024-05-21T22:06:07.678 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "retention_period", 2024-05-21T22:06:07.678 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-05-21T22:06:07.678 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.678 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.678 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "15552000", 2024-05-21T22:06:07.678 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.679 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.679 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.679 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "how long to retain device health metrics", 2024-05-21T22:06:07.679 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.679 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.679 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.679 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.679 INFO:teuthology.orchestra.run.smithi151.stdout: "scrape_frequency": { 2024-05-21T22:06:07.679 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "scrape_frequency", 2024-05-21T22:06:07.679 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-05-21T22:06:07.679 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.679 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.680 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "86400", 2024-05-21T22:06:07.680 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.680 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.680 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.680 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-21T22:06:07.680 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.680 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.680 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.680 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.680 INFO:teuthology.orchestra.run.smithi151.stdout: "self_heal": { 2024-05-21T22:06:07.680 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "self_heal", 2024-05-21T22:06:07.680 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.680 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.681 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.681 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:07.681 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.681 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.681 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.681 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-21T22:06:07.681 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.681 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.681 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.681 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.681 INFO:teuthology.orchestra.run.smithi151.stdout: "sleep_interval": { 2024-05-21T22:06:07.681 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sleep_interval", 2024-05-21T22:06:07.681 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-05-21T22:06:07.682 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.682 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.682 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "600", 2024-05-21T22:06:07.682 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.682 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.682 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.682 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "how frequently to wake up and check device health", 2024-05-21T22:06:07.682 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.682 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.682 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.682 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.682 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:07.683 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:07.683 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.683 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:07.683 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.683 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:07.683 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.683 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.683 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.683 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.683 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.683 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.683 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.683 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.684 INFO:teuthology.orchestra.run.smithi151.stdout: "warn_threshold": { 2024-05-21T22:06:07.684 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "warn_threshold", 2024-05-21T22:06:07.684 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-05-21T22:06:07.684 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.684 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.684 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "7257600", 2024-05-21T22:06:07.684 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.684 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.684 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.684 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-21T22:06:07.684 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.684 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.685 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.685 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.685 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.685 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.685 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:07.685 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "diskprediction_local", 2024-05-21T22:06:07.685 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:07.685 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:07.685 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:07.685 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:07.685 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:07.685 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.685 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.686 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.686 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.686 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.686 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.686 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.686 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.686 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.686 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.686 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.686 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.686 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.686 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.687 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.687 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.687 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.687 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.687 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.687 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:07.687 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:07.687 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.687 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.687 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.687 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.687 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.687 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.687 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.688 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.688 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.688 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.688 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.688 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.688 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:07.688 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:07.688 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.688 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.688 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.688 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:07.688 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.688 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.688 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.689 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.689 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.689 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.689 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.689 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.689 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.689 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.689 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.689 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.689 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.689 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.689 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.689 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:07.690 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:07.690 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.690 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.690 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.690 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.690 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.690 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.690 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.690 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.690 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.690 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.690 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.691 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.691 INFO:teuthology.orchestra.run.smithi151.stdout: "predict_interval": { 2024-05-21T22:06:07.691 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "predict_interval", 2024-05-21T22:06:07.691 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.691 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.691 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.691 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "86400", 2024-05-21T22:06:07.691 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.691 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.691 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.691 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.691 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.691 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.692 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.692 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.692 INFO:teuthology.orchestra.run.smithi151.stdout: "predictor_model": { 2024-05-21T22:06:07.692 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "predictor_model", 2024-05-21T22:06:07.692 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.692 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.692 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.692 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "prophetstor", 2024-05-21T22:06:07.692 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.692 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.692 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.692 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.692 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.693 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.693 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.693 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.693 INFO:teuthology.orchestra.run.smithi151.stdout: "sleep_interval": { 2024-05-21T22:06:07.693 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sleep_interval", 2024-05-21T22:06:07.693 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.693 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.693 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.693 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "600", 2024-05-21T22:06:07.693 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.693 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.693 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.693 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.693 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.694 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.694 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.694 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.694 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:07.694 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:07.694 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.694 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:07.694 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.694 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:07.694 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.694 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.694 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.694 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.695 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.695 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.695 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.695 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.695 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.695 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.695 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:07.695 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "influx", 2024-05-21T22:06:07.695 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": false, 2024-05-21T22:06:07.695 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "influxdb python module not found", 2024-05-21T22:06:07.695 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:07.695 INFO:teuthology.orchestra.run.smithi151.stdout: "batch_size": { 2024-05-21T22:06:07.695 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "batch_size", 2024-05-21T22:06:07.695 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.696 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.696 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.696 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "5000", 2024-05-21T22:06:07.696 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.696 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.696 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.696 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-21T22:06:07.696 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.696 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.696 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.696 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.696 INFO:teuthology.orchestra.run.smithi151.stdout: "database": { 2024-05-21T22:06:07.696 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "database", 2024-05-21T22:06:07.697 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.697 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.697 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.697 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "ceph", 2024-05-21T22:06:07.697 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.697 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.697 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.697 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-21T22:06:07.697 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.697 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.697 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.697 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.697 INFO:teuthology.orchestra.run.smithi151.stdout: "hostname": { 2024-05-21T22:06:07.698 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "hostname", 2024-05-21T22:06:07.698 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.698 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.698 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.698 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.698 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.698 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.698 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.698 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "InfluxDB server hostname", 2024-05-21T22:06:07.698 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.698 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.698 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.698 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.699 INFO:teuthology.orchestra.run.smithi151.stdout: "interval": { 2024-05-21T22:06:07.699 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "interval", 2024-05-21T22:06:07.699 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-05-21T22:06:07.699 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.699 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.699 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "30", 2024-05-21T22:06:07.699 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "5", 2024-05-21T22:06:07.699 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.699 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.699 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-21T22:06:07.699 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.699 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.699 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.699 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.700 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:07.700 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:07.700 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.700 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.700 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.700 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.700 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.700 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.700 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.700 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.700 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.700 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.700 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.700 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.701 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.701 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.701 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.701 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.701 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.701 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.701 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.701 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:07.701 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:07.701 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.701 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.701 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.701 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.701 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.702 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.702 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.702 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.702 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.702 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.702 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.702 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.702 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:07.702 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:07.702 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.702 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.702 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.702 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:07.703 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.703 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.703 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.703 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.703 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.703 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.703 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.703 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.703 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.703 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.703 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.703 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.703 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.704 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.704 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.704 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:07.704 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:07.704 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.704 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.704 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.704 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.704 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.704 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.704 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.704 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.704 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.705 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.705 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.705 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.705 INFO:teuthology.orchestra.run.smithi151.stdout: "password": { 2024-05-21T22:06:07.705 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "password", 2024-05-21T22:06:07.705 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.705 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.705 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.705 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.705 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.705 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.705 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.705 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "password of InfluxDB server user", 2024-05-21T22:06:07.706 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.706 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.706 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.706 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.706 INFO:teuthology.orchestra.run.smithi151.stdout: "port": { 2024-05-21T22:06:07.706 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "port", 2024-05-21T22:06:07.706 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.706 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.706 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.706 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "8086", 2024-05-21T22:06:07.706 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.706 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.706 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.707 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "InfluxDB server port", 2024-05-21T22:06:07.707 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.707 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.707 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.707 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.707 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:07.707 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:07.707 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.707 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:07.707 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.707 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:07.707 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.707 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.708 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.708 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.708 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.708 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.708 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.708 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.708 INFO:teuthology.orchestra.run.smithi151.stdout: "ssl": { 2024-05-21T22:06:07.708 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "ssl", 2024-05-21T22:06:07.708 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.708 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.708 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.708 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "false", 2024-05-21T22:06:07.708 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.709 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.709 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.709 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-21T22:06:07.709 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.709 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.709 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.709 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.709 INFO:teuthology.orchestra.run.smithi151.stdout: "threads": { 2024-05-21T22:06:07.709 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "threads", 2024-05-21T22:06:07.709 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.709 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.709 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.710 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "5", 2024-05-21T22:06:07.710 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "1", 2024-05-21T22:06:07.710 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "32", 2024-05-21T22:06:07.710 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.710 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-21T22:06:07.710 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.710 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.710 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.710 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.710 INFO:teuthology.orchestra.run.smithi151.stdout: "username": { 2024-05-21T22:06:07.710 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "username", 2024-05-21T22:06:07.710 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.710 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.711 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.711 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.711 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.711 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.711 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.711 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "username of InfluxDB server user", 2024-05-21T22:06:07.711 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.711 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.711 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.711 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.711 INFO:teuthology.orchestra.run.smithi151.stdout: "verify_ssl": { 2024-05-21T22:06:07.711 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "verify_ssl", 2024-05-21T22:06:07.712 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.712 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.712 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.712 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "true", 2024-05-21T22:06:07.712 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.712 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.712 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.712 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-21T22:06:07.712 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.712 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.712 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.712 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.712 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.713 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.713 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:07.713 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "insights", 2024-05-21T22:06:07.713 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:07.713 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:07.713 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:07.713 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:07.713 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:07.713 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.713 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.713 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.713 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.713 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.713 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.714 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.714 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.714 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.714 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.714 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.714 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.714 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.714 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.714 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.714 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.714 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.714 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.714 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.714 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:07.715 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:07.715 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.715 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.715 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.715 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.715 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.715 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.715 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.715 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.715 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.715 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.715 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.715 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.715 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:07.716 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:07.716 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.716 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.716 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.716 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:07.716 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.716 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.716 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.716 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.716 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.716 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.716 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.716 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.716 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.716 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.717 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.717 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.717 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.717 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.717 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.717 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:07.717 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:07.717 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.717 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.717 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.717 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.717 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.717 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.717 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.718 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.718 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.718 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.718 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.718 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.718 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:07.718 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:07.718 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.718 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:07.718 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.718 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:07.718 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.719 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.719 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.719 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.719 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.719 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.719 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.719 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.719 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.719 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.719 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:07.719 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "iostat", 2024-05-21T22:06:07.719 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:07.719 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:07.719 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:07.720 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:07.720 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:07.720 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.720 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.720 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.720 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.720 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.720 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.720 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.720 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.720 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.720 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.721 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.721 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.721 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.721 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.721 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.721 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.721 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.721 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.721 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.721 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:07.721 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:07.721 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.721 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.722 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.722 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.722 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.722 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.722 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.722 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.722 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.722 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.722 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.722 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.722 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:07.722 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:07.723 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.723 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.723 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.723 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:07.723 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.723 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.723 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.723 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.723 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.723 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.723 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.723 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.723 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.724 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.724 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.724 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.724 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.724 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.724 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.724 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:07.724 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:07.724 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.724 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.724 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.724 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.725 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.725 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.725 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.725 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.725 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.725 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.725 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.725 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.725 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:07.725 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:07.725 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.725 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:07.725 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.726 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:07.726 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.726 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.726 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.726 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.726 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.726 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.726 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.726 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.726 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.726 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.726 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:07.727 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "k8sevents", 2024-05-21T22:06:07.727 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:07.727 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:07.727 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:07.727 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_event_retention_days": { 2024-05-21T22:06:07.727 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "ceph_event_retention_days", 2024-05-21T22:06:07.727 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.727 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.727 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.727 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "7", 2024-05-21T22:06:07.727 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.728 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.728 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.728 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-21T22:06:07.728 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.728 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.728 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.728 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.728 INFO:teuthology.orchestra.run.smithi151.stdout: "config_check_secs": { 2024-05-21T22:06:07.728 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "config_check_secs", 2024-05-21T22:06:07.728 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.728 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.728 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.729 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "10", 2024-05-21T22:06:07.729 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "10", 2024-05-21T22:06:07.729 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.729 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.729 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-21T22:06:07.729 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.729 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.729 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.729 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.729 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:07.730 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:07.730 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.730 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.730 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.730 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.730 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.730 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.730 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.730 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.730 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.730 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.730 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.731 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.731 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.731 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.731 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.731 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.731 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.731 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.731 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.731 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:07.731 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:07.731 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.731 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.732 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.732 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.732 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.732 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.732 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.732 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.732 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.732 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.732 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.732 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.732 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:07.732 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:07.733 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.733 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.733 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.733 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:07.733 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.733 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.733 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.733 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.733 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.733 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.733 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.733 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.734 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.734 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.734 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.734 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.734 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.734 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.734 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.734 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:07.734 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:07.734 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.734 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.734 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.735 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.735 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.735 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.735 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.735 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.735 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.735 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.735 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.735 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.735 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:07.735 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:07.735 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.736 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:07.736 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.736 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:07.736 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.736 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.736 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.736 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.736 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.736 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.736 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.736 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.736 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.737 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.737 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:07.737 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "localpool", 2024-05-21T22:06:07.737 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:07.737 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:07.737 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:07.737 INFO:teuthology.orchestra.run.smithi151.stdout: "failure_domain": { 2024-05-21T22:06:07.737 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "failure_domain", 2024-05-21T22:06:07.737 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.737 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.737 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.737 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "host", 2024-05-21T22:06:07.737 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.737 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.738 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.738 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "failure domain for any created local pool", 2024-05-21T22:06:07.738 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-21T22:06:07.738 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.738 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.738 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.738 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:07.738 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:07.738 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.738 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.738 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.738 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.738 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.739 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.739 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.739 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.739 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.739 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.739 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.739 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.739 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.739 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.739 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.739 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.739 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.739 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.740 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.740 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:07.740 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:07.740 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.740 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.740 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.740 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.740 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.740 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.740 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.740 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.740 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.740 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.741 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.741 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.741 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:07.741 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:07.741 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.741 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.741 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.741 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:07.741 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.741 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.741 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.741 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.741 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.741 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.742 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.742 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.742 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.742 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.742 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.742 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.742 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.742 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.742 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.742 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:07.742 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:07.742 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.742 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.743 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.743 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.743 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.743 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.743 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.743 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.743 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.743 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.743 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.743 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.743 INFO:teuthology.orchestra.run.smithi151.stdout: "min_size": { 2024-05-21T22:06:07.743 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "min_size", 2024-05-21T22:06:07.743 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.743 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.744 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.744 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.744 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.744 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.744 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.744 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "default min_size for any created local pool", 2024-05-21T22:06:07.744 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-21T22:06:07.744 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.744 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.744 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.744 INFO:teuthology.orchestra.run.smithi151.stdout: "num_rep": { 2024-05-21T22:06:07.744 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "num_rep", 2024-05-21T22:06:07.744 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.745 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.745 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.745 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "3", 2024-05-21T22:06:07.745 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.745 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.745 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.745 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "default replica count for any created local pool", 2024-05-21T22:06:07.745 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.745 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.745 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.745 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.746 INFO:teuthology.orchestra.run.smithi151.stdout: "pg_num": { 2024-05-21T22:06:07.746 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "pg_num", 2024-05-21T22:06:07.746 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.746 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.746 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.746 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "128", 2024-05-21T22:06:07.746 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.746 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.746 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.746 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "default pg_num for any created local pool", 2024-05-21T22:06:07.746 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.746 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.747 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.747 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.747 INFO:teuthology.orchestra.run.smithi151.stdout: "prefix": { 2024-05-21T22:06:07.747 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "prefix", 2024-05-21T22:06:07.747 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.747 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.747 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.747 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.747 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.747 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.747 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.747 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "name prefix for any created local pool", 2024-05-21T22:06:07.747 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.747 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.748 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.748 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.748 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:07.748 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:07.748 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.748 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:07.748 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.748 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:07.748 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.748 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.748 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.748 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.749 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.749 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.749 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.749 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.749 INFO:teuthology.orchestra.run.smithi151.stdout: "subtree": { 2024-05-21T22:06:07.749 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "subtree", 2024-05-21T22:06:07.749 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.749 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.749 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.749 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "rack", 2024-05-21T22:06:07.749 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.749 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.750 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.750 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-21T22:06:07.750 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-21T22:06:07.750 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.750 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.750 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.750 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.750 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.750 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:07.750 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "mds_autoscaler", 2024-05-21T22:06:07.750 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:07.750 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:07.750 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:07.751 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:07.751 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:07.751 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.751 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.751 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.751 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.751 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.751 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.751 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.751 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.751 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.751 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.751 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.751 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.752 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.752 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.752 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.752 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.752 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.752 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.752 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.752 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:07.752 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:07.752 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.752 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.752 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.752 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.753 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.753 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.753 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.753 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.753 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.753 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.753 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.753 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.753 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:07.753 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:07.753 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.753 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.753 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.753 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:07.754 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.754 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.754 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.754 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.754 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.754 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.754 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.754 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.754 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.754 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.754 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.754 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.755 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.755 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.755 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.755 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:07.755 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:07.755 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.755 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.755 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.755 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.755 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.755 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.755 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.755 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.755 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.755 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.756 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.756 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.756 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:07.756 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:07.756 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.756 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:07.756 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.756 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:07.756 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.756 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.756 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.756 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.757 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.757 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.757 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.757 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.757 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.757 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.757 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:07.757 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "mirroring", 2024-05-21T22:06:07.757 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:07.757 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:07.757 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:07.757 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:07.757 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:07.758 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.758 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.758 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.758 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.758 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.758 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.758 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.758 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.758 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.758 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.758 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.758 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.758 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.759 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.759 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.759 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.759 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.759 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.759 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.759 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:07.759 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:07.759 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.759 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.759 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.759 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.759 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.760 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.760 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.760 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.760 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.760 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.760 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.760 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.760 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:07.760 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:07.760 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.760 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.760 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.761 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:07.761 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.761 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.761 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.761 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.761 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.761 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.761 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.761 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.761 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.761 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.761 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.762 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.762 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.762 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.762 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.762 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:07.762 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:07.762 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.762 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.762 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.762 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.762 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.762 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.762 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.763 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.763 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.763 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.763 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.763 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.763 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:07.763 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:07.763 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.763 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:07.763 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.763 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:07.763 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.763 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.764 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.764 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.764 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.764 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.764 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.764 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.764 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.764 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.764 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:07.764 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "nfs", 2024-05-21T22:06:07.764 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:07.764 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:07.764 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:07.765 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:07.765 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:07.765 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.765 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.765 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.765 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.765 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.765 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.765 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.765 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.765 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.765 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.765 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.766 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.766 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.766 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.766 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.766 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.766 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.766 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.766 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.766 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:07.766 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:07.766 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.766 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.767 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.767 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.767 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.767 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.767 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.767 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.767 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.767 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.767 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.767 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.767 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:07.767 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:07.768 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.768 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.768 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.768 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:07.768 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.768 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.768 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.768 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.768 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.769 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.769 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.769 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.769 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.769 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.769 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.769 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.769 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.769 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.769 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.769 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:07.769 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:07.770 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.770 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.770 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.770 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.770 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.770 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.770 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.770 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.770 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.770 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.770 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.770 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.771 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:07.771 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:07.771 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.771 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:07.771 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.771 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:07.771 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.771 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.771 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.771 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.771 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.771 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.771 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.771 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.772 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.772 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.772 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:07.772 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "orchestrator", 2024-05-21T22:06:07.772 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:07.772 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:07.772 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:07.772 INFO:teuthology.orchestra.run.smithi151.stdout: "fail_fs": { 2024-05-21T22:06:07.772 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "fail_fs", 2024-05-21T22:06:07.772 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.772 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.772 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.773 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.773 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.773 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.773 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.773 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-21T22:06:07.773 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.773 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.773 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.773 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.773 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:07.773 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:07.773 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.773 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.774 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.774 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.774 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.774 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.774 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.774 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.774 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.774 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.774 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.774 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.774 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.774 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.775 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.775 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.775 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.775 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.775 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.775 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:07.775 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:07.775 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.775 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.775 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.775 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.775 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.775 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.776 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.776 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.776 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.776 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.776 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.776 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.776 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:07.776 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:07.776 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.776 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.776 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.776 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:07.776 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.777 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.777 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.777 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.777 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.777 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.777 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.777 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.777 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.777 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.777 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.777 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.777 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.778 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.778 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.778 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:07.778 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:07.778 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.778 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.778 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.778 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.778 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.778 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.778 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.778 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.778 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.779 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.779 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.779 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.779 INFO:teuthology.orchestra.run.smithi151.stdout: "orchestrator": { 2024-05-21T22:06:07.779 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "orchestrator", 2024-05-21T22:06:07.779 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.779 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.779 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.779 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.779 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.779 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.779 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.780 INFO:teuthology.orchestra.run.smithi151.stdout: "cephadm", 2024-05-21T22:06:07.780 INFO:teuthology.orchestra.run.smithi151.stdout: "rook", 2024-05-21T22:06:07.780 INFO:teuthology.orchestra.run.smithi151.stdout: "test_orchestrator" 2024-05-21T22:06:07.780 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.780 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Orchestrator backend", 2024-05-21T22:06:07.780 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.780 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.780 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.780 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.780 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:07.780 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:07.780 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.781 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:07.781 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.781 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:07.781 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.781 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.781 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.781 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.781 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.781 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.781 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.781 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.781 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.781 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.782 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:07.782 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "osd_perf_query", 2024-05-21T22:06:07.782 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:07.782 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:07.782 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:07.782 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:07.782 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:07.782 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.782 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.782 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.782 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.782 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.783 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.783 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.783 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.783 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.783 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.783 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.783 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.783 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.783 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.783 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.783 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.783 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.783 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.784 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.784 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:07.784 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:07.784 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.784 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.784 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.784 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.784 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.784 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.784 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.784 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.784 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.784 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.784 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.785 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.785 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:07.785 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:07.785 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.785 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.785 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.785 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:07.785 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.785 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.785 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.785 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.785 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.785 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.786 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.786 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.786 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.786 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.786 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.786 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.786 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.786 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.786 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.786 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:07.786 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:07.787 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.787 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.787 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.787 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.787 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.787 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.787 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.787 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.787 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.787 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.787 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.787 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.787 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:07.788 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:07.788 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.788 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:07.788 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.788 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:07.788 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.788 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.788 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.788 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.788 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.788 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.788 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.788 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.789 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.789 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.789 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:07.789 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "osd_support", 2024-05-21T22:06:07.789 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:07.789 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:07.789 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:07.789 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:07.789 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:07.789 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.789 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.789 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.789 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.790 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.790 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.790 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.790 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.790 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.790 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.790 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.790 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.790 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.790 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.790 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.790 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.791 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.791 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.791 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.791 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:07.791 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:07.791 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.791 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.791 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.791 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.791 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.791 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.791 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.791 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.792 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.792 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.792 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.792 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.792 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:07.792 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:07.792 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.792 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.792 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.792 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:07.792 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.792 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.793 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.793 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.793 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.793 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.793 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.793 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.793 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.793 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.793 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.793 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.793 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.793 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.793 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.794 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:07.794 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:07.794 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.794 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.794 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.794 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.794 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.794 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.794 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.794 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.794 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.794 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.794 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.795 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.795 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:07.795 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:07.795 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.795 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:07.795 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.795 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:07.795 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.795 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.795 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.795 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.795 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.796 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.796 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.796 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.796 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.796 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.796 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:07.796 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "pg_autoscaler", 2024-05-21T22:06:07.796 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:07.796 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:07.796 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:07.796 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:07.796 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:07.796 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.796 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.797 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.797 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.797 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.797 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.797 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.797 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.797 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.797 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.797 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.797 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.797 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.797 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.797 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.797 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.798 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.798 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.798 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.798 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:07.798 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:07.798 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.798 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.798 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.798 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.798 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.798 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.798 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.798 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.799 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.799 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.799 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.799 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.799 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:07.799 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:07.799 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.799 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.799 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.799 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:07.799 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.799 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.799 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.799 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.800 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.800 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.800 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.800 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.800 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.800 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.800 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.800 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.800 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.800 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.800 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.800 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:07.800 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:07.800 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.801 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.801 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.801 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.801 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.801 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.801 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.801 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.801 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.801 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.801 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.801 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.801 INFO:teuthology.orchestra.run.smithi151.stdout: "sleep_interval": { 2024-05-21T22:06:07.801 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sleep_interval", 2024-05-21T22:06:07.801 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-05-21T22:06:07.802 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.802 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.802 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "60", 2024-05-21T22:06:07.802 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.802 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.802 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.802 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.802 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.802 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.802 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.802 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.802 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:07.802 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:07.803 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.803 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:07.803 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.803 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:07.803 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.803 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.803 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.803 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.803 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.803 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.803 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.804 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.804 INFO:teuthology.orchestra.run.smithi151.stdout: "threshold": { 2024-05-21T22:06:07.804 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "threshold", 2024-05-21T22:06:07.804 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "float", 2024-05-21T22:06:07.804 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.804 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.804 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "3.0", 2024-05-21T22:06:07.804 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "1.0", 2024-05-21T22:06:07.804 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.804 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.804 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "scaling threshold", 2024-05-21T22:06:07.804 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-21T22:06:07.804 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.804 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.805 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.805 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.805 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.805 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:07.805 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "progress", 2024-05-21T22:06:07.805 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:07.805 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:07.805 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:07.805 INFO:teuthology.orchestra.run.smithi151.stdout: "allow_pg_recovery_event": { 2024-05-21T22:06:07.805 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "allow_pg_recovery_event", 2024-05-21T22:06:07.805 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.805 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.806 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.806 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.806 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.806 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.806 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.806 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-21T22:06:07.806 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.806 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.806 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.806 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.806 INFO:teuthology.orchestra.run.smithi151.stdout: "enabled": { 2024-05-21T22:06:07.806 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "enabled", 2024-05-21T22:06:07.806 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.806 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.807 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.807 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:07.807 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.807 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.807 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.807 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.807 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.807 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.807 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.807 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.807 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:07.807 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:07.807 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.807 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.808 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.808 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.808 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.808 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.808 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.808 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.808 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.808 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.808 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.808 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.808 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.808 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.808 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.808 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.809 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.809 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.809 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.809 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:07.809 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:07.809 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.809 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.809 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.809 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.809 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.809 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.809 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.809 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.809 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.810 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.810 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.810 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.810 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:07.810 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:07.810 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.810 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.810 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.810 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:07.810 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.810 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.810 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.810 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.811 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.811 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.811 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.811 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.811 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.811 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.811 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.811 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.811 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.811 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.811 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.811 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:07.811 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:07.812 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.812 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.812 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.812 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.812 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.812 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.812 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.812 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.812 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.812 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.812 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.812 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.813 INFO:teuthology.orchestra.run.smithi151.stdout: "max_completed_events": { 2024-05-21T22:06:07.813 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "max_completed_events", 2024-05-21T22:06:07.813 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.813 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.813 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.813 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "50", 2024-05-21T22:06:07.813 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.813 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.813 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.813 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "number of past completed events to remember", 2024-05-21T22:06:07.813 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.813 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.813 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.813 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.814 INFO:teuthology.orchestra.run.smithi151.stdout: "sleep_interval": { 2024-05-21T22:06:07.814 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sleep_interval", 2024-05-21T22:06:07.814 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-05-21T22:06:07.814 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.814 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.814 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "5", 2024-05-21T22:06:07.814 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.814 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.814 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.814 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "how long the module is going to sleep", 2024-05-21T22:06:07.814 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.814 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.814 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.814 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.815 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:07.815 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:07.815 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.815 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:07.815 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.815 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:07.815 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.815 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.815 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.815 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.815 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.815 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.815 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.815 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.816 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.816 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.816 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:07.816 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "prometheus", 2024-05-21T22:06:07.816 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:07.816 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:07.816 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:07.816 INFO:teuthology.orchestra.run.smithi151.stdout: "cache": { 2024-05-21T22:06:07.816 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "cache", 2024-05-21T22:06:07.816 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.816 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.816 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.816 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:07.816 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.817 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.817 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.817 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.817 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.817 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.817 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.817 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.817 INFO:teuthology.orchestra.run.smithi151.stdout: "exclude_perf_counters": { 2024-05-21T22:06:07.817 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "exclude_perf_counters", 2024-05-21T22:06:07.817 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.817 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.817 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.817 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:07.817 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.817 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.818 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.818 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-21T22:06:07.818 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-21T22:06:07.818 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.818 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.818 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.818 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:07.818 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:07.818 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.818 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.818 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.818 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.818 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.819 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.819 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.819 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.819 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.819 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.819 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.819 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.819 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.819 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.819 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.819 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.819 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.819 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.820 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.820 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:07.820 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:07.820 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.820 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.820 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.820 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.820 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.820 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.820 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.820 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.820 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.820 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.820 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.821 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.821 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:07.821 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:07.821 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.821 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.821 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.821 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:07.821 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.821 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.821 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.821 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.821 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.821 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.822 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.822 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.822 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.822 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.822 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.822 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.822 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.822 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.822 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.822 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:07.822 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:07.822 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.823 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.823 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.823 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.823 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.823 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.823 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.823 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.823 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.823 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.823 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.823 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.823 INFO:teuthology.orchestra.run.smithi151.stdout: "rbd_stats_pools": { 2024-05-21T22:06:07.823 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "rbd_stats_pools", 2024-05-21T22:06:07.823 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.824 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.824 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.824 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.824 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.824 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.824 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.824 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.824 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.824 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.824 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.824 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.824 INFO:teuthology.orchestra.run.smithi151.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-21T22:06:07.824 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-21T22:06:07.824 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.825 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.825 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.825 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "300", 2024-05-21T22:06:07.825 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.825 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.825 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.825 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.825 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.825 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.825 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.825 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.825 INFO:teuthology.orchestra.run.smithi151.stdout: "scrape_interval": { 2024-05-21T22:06:07.825 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "scrape_interval", 2024-05-21T22:06:07.825 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "float", 2024-05-21T22:06:07.826 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.826 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.826 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "15.0", 2024-05-21T22:06:07.826 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.826 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.826 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.826 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.826 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.826 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.826 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.826 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.826 INFO:teuthology.orchestra.run.smithi151.stdout: "server_addr": { 2024-05-21T22:06:07.826 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "server_addr", 2024-05-21T22:06:07.827 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.827 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.827 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.827 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "::", 2024-05-21T22:06:07.827 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.827 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.827 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.827 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-21T22:06:07.827 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.827 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.827 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.827 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.827 INFO:teuthology.orchestra.run.smithi151.stdout: "server_port": { 2024-05-21T22:06:07.827 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "server_port", 2024-05-21T22:06:07.827 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.828 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.828 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.828 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "9283", 2024-05-21T22:06:07.828 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.828 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.828 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.828 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-21T22:06:07.828 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.828 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.828 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.828 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.828 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:07.828 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:07.829 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.829 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:07.829 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.829 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:07.829 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.829 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.829 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.829 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.829 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.829 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.829 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.829 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.829 INFO:teuthology.orchestra.run.smithi151.stdout: "stale_cache_strategy": { 2024-05-21T22:06:07.829 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "stale_cache_strategy", 2024-05-21T22:06:07.830 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.830 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.830 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.830 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "log", 2024-05-21T22:06:07.830 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.830 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.830 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.830 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.830 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.830 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.830 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.830 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.830 INFO:teuthology.orchestra.run.smithi151.stdout: "standby_behaviour": { 2024-05-21T22:06:07.830 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "standby_behaviour", 2024-05-21T22:06:07.831 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.831 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.831 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.831 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "default", 2024-05-21T22:06:07.831 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.831 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.831 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.831 INFO:teuthology.orchestra.run.smithi151.stdout: "default", 2024-05-21T22:06:07.831 INFO:teuthology.orchestra.run.smithi151.stdout: "error" 2024-05-21T22:06:07.831 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.831 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.831 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.831 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.832 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.832 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.832 INFO:teuthology.orchestra.run.smithi151.stdout: "standby_error_status_code": { 2024-05-21T22:06:07.832 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "standby_error_status_code", 2024-05-21T22:06:07.832 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.832 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.832 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.832 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "500", 2024-05-21T22:06:07.832 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "400", 2024-05-21T22:06:07.832 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "599", 2024-05-21T22:06:07.832 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.832 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.832 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.832 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.833 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.833 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.833 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.833 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.833 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:07.833 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "rbd_support", 2024-05-21T22:06:07.833 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:07.833 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:07.833 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:07.833 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:07.833 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:07.833 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.833 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.834 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.834 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.834 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.834 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.834 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.834 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.834 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.834 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.834 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.834 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.834 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.834 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.834 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.834 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.835 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.835 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.835 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.835 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:07.835 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:07.835 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.835 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.835 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.835 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.835 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.835 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.835 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.835 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.835 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.836 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.836 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.836 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.836 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:07.836 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:07.836 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.836 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.836 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.836 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:07.836 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.836 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.836 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.836 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.836 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.837 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.837 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.837 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.837 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.837 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.837 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.837 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.837 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.837 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.837 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.837 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:07.837 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:07.837 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.838 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.838 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.838 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.838 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.838 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.838 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.838 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.838 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.838 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.838 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.838 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.838 INFO:teuthology.orchestra.run.smithi151.stdout: "max_concurrent_snap_create": { 2024-05-21T22:06:07.838 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "max_concurrent_snap_create", 2024-05-21T22:06:07.839 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.839 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.839 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.839 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "10", 2024-05-21T22:06:07.839 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.839 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.839 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.839 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.839 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.839 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.839 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.839 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.839 INFO:teuthology.orchestra.run.smithi151.stdout: "mirror_snapshot_schedule": { 2024-05-21T22:06:07.839 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "mirror_snapshot_schedule", 2024-05-21T22:06:07.840 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.840 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.840 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.840 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.840 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.840 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.840 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.840 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.840 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.840 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.840 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.840 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.840 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:07.840 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:07.841 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.841 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:07.841 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.841 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:07.841 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.841 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.841 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.841 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.841 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.841 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.841 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.841 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.841 INFO:teuthology.orchestra.run.smithi151.stdout: "trash_purge_schedule": { 2024-05-21T22:06:07.841 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "trash_purge_schedule", 2024-05-21T22:06:07.842 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.842 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.842 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.842 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.842 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.842 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.842 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.842 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.842 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.842 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.842 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.842 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.843 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.843 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.843 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:07.843 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "restful", 2024-05-21T22:06:07.843 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:07.843 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:07.843 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:07.843 INFO:teuthology.orchestra.run.smithi151.stdout: "enable_auth": { 2024-05-21T22:06:07.843 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "enable_auth", 2024-05-21T22:06:07.843 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.843 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.843 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.843 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:07.843 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.844 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.844 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.844 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.844 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.844 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.844 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.844 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.844 INFO:teuthology.orchestra.run.smithi151.stdout: "key_file": { 2024-05-21T22:06:07.844 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "key_file", 2024-05-21T22:06:07.844 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.844 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.844 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.844 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.844 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.845 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.845 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.845 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.845 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.845 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.845 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.845 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.845 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:07.845 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:07.845 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.845 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.845 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.846 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.846 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.846 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.846 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.846 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.846 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.846 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.846 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.846 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.846 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.846 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.846 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.846 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.847 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.847 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.847 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.847 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:07.847 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:07.847 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.847 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.847 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.847 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.847 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.847 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.847 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.847 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.847 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.848 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.848 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.848 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.848 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:07.848 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:07.848 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.848 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.848 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.848 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:07.848 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.848 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.848 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.848 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.848 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.849 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.849 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.849 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.849 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.849 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.849 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.849 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.849 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.849 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.849 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.849 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:07.849 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:07.849 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.849 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.850 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.850 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.850 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.850 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.850 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.850 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.850 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.850 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.850 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.850 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.850 INFO:teuthology.orchestra.run.smithi151.stdout: "server_addr": { 2024-05-21T22:06:07.850 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "server_addr", 2024-05-21T22:06:07.850 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.850 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.851 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.851 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.851 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.851 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.851 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.851 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.851 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.851 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.851 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.851 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.851 INFO:teuthology.orchestra.run.smithi151.stdout: "server_port": { 2024-05-21T22:06:07.851 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "server_port", 2024-05-21T22:06:07.851 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.851 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.852 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.852 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.852 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.852 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.852 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.852 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.852 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.852 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.852 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.852 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.852 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:07.852 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:07.852 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.852 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:07.852 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.853 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:07.853 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.853 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.853 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.853 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.853 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.853 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.853 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.853 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.853 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.853 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.853 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:07.853 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "rgw", 2024-05-21T22:06:07.854 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:07.854 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:07.854 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:07.854 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:07.854 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:07.854 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.854 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.854 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.854 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.854 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.854 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.854 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.854 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.855 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.855 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.855 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.855 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.855 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.855 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.855 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.855 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.855 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.855 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.855 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.855 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:07.855 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:07.855 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.856 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.856 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.856 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.856 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.856 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.856 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.856 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.856 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.856 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.856 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.856 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.856 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:07.856 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:07.856 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.856 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.857 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.857 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:07.857 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.857 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.857 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.857 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.857 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.857 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.857 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.857 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.857 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.857 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.857 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.857 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.858 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.858 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.858 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.858 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:07.858 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:07.858 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.858 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.858 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.858 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.858 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.858 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.858 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.858 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.859 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.859 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.859 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.859 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.859 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:07.859 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:07.859 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.859 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:07.859 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.859 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:07.859 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.859 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.860 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.860 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.860 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.860 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.860 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.860 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.860 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.860 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.860 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:07.860 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "rook", 2024-05-21T22:06:07.860 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:07.860 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:07.861 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:07.861 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:07.861 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:07.861 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.861 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.861 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.861 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.861 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.861 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.861 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.861 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.861 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.861 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.861 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.862 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.862 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.862 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.862 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.862 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.862 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.862 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.862 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.862 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:07.862 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:07.862 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.862 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.863 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.863 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.863 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.863 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.863 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.863 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.863 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.863 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.863 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.863 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.864 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:07.864 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:07.864 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.864 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.864 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.864 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:07.864 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.864 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.864 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.864 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.864 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.865 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.865 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.865 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.865 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.865 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.865 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.865 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.865 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.865 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.865 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.865 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:07.865 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:07.865 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.866 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.866 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.866 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.866 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.866 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.866 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.866 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.866 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.866 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.866 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.867 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.867 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:07.867 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:07.867 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.867 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:07.867 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.867 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:07.867 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.867 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.867 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.868 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.868 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.868 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.868 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.868 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.868 INFO:teuthology.orchestra.run.smithi151.stdout: "storage_class": { 2024-05-21T22:06:07.868 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "storage_class", 2024-05-21T22:06:07.868 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.868 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.868 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.868 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "local", 2024-05-21T22:06:07.868 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.868 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.869 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.869 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-21T22:06:07.869 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.869 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.869 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.869 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.869 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.869 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.869 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:07.869 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "selftest", 2024-05-21T22:06:07.870 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:07.870 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:07.870 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:07.870 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:07.870 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:07.870 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.870 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.870 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.870 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.870 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.870 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.870 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.870 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.871 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.871 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.871 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.871 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.871 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.871 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.871 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.871 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.871 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.871 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.871 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.871 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:07.871 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:07.872 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.872 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.872 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.872 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.872 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.872 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.872 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.872 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.872 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.872 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.872 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.872 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.872 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:07.872 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:07.873 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.873 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.873 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.873 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:07.873 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.873 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.873 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.873 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.873 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.873 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.873 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.873 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.874 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.874 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.874 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.874 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.874 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.874 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.874 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.874 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:07.874 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:07.874 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.874 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.874 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.874 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.875 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.875 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.875 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.875 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.875 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.875 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.875 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.875 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.875 INFO:teuthology.orchestra.run.smithi151.stdout: "roption1": { 2024-05-21T22:06:07.875 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "roption1", 2024-05-21T22:06:07.875 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.875 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.875 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.876 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.876 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.876 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.876 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.876 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.876 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.876 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.876 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.876 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.876 INFO:teuthology.orchestra.run.smithi151.stdout: "roption2": { 2024-05-21T22:06:07.876 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "roption2", 2024-05-21T22:06:07.876 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.876 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.876 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.877 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "xyz", 2024-05-21T22:06:07.877 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.877 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.877 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.877 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.877 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.877 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.877 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.877 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.877 INFO:teuthology.orchestra.run.smithi151.stdout: "rwoption1": { 2024-05-21T22:06:07.877 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "rwoption1", 2024-05-21T22:06:07.877 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.878 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.878 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.878 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.878 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.878 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.878 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.878 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.878 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.878 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.878 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.878 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.878 INFO:teuthology.orchestra.run.smithi151.stdout: "rwoption2": { 2024-05-21T22:06:07.879 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "rwoption2", 2024-05-21T22:06:07.879 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.879 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.879 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.879 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.879 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.879 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.879 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.879 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.879 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.879 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.879 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.879 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.880 INFO:teuthology.orchestra.run.smithi151.stdout: "rwoption3": { 2024-05-21T22:06:07.880 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "rwoption3", 2024-05-21T22:06:07.880 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "float", 2024-05-21T22:06:07.880 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.880 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.880 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.880 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.880 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.880 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.880 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.880 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.880 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.881 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.881 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.881 INFO:teuthology.orchestra.run.smithi151.stdout: "rwoption4": { 2024-05-21T22:06:07.881 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "rwoption4", 2024-05-21T22:06:07.881 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.881 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.881 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.881 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.881 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.881 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.881 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.881 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.881 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.881 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.882 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.882 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.882 INFO:teuthology.orchestra.run.smithi151.stdout: "rwoption5": { 2024-05-21T22:06:07.882 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "rwoption5", 2024-05-21T22:06:07.882 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.882 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.882 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.882 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.882 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.882 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.882 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.882 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.883 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.883 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.883 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.883 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.883 INFO:teuthology.orchestra.run.smithi151.stdout: "rwoption6": { 2024-05-21T22:06:07.883 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "rwoption6", 2024-05-21T22:06:07.883 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.883 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.883 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.883 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:07.883 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.883 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.884 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.884 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.884 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.884 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.884 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.884 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.884 INFO:teuthology.orchestra.run.smithi151.stdout: "rwoption7": { 2024-05-21T22:06:07.884 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "rwoption7", 2024-05-21T22:06:07.884 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.884 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.884 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.884 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.884 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "1", 2024-05-21T22:06:07.885 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "42", 2024-05-21T22:06:07.885 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.885 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.885 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.885 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.885 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.885 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.885 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:07.885 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:07.885 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.886 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:07.886 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.886 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:07.886 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.886 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.886 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.886 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.886 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.886 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.886 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.886 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.887 INFO:teuthology.orchestra.run.smithi151.stdout: "testkey": { 2024-05-21T22:06:07.887 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "testkey", 2024-05-21T22:06:07.887 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.887 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.887 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.887 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.887 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.887 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.887 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.887 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.887 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.888 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.888 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.888 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.888 INFO:teuthology.orchestra.run.smithi151.stdout: "testlkey": { 2024-05-21T22:06:07.888 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "testlkey", 2024-05-21T22:06:07.888 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.888 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.888 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.888 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.889 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.889 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.889 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.889 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.889 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.889 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.889 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.889 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.889 INFO:teuthology.orchestra.run.smithi151.stdout: "testnewline": { 2024-05-21T22:06:07.889 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "testnewline", 2024-05-21T22:06:07.890 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.890 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.890 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.890 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.890 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.890 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.890 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.890 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.890 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.890 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.890 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.890 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.891 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.891 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.891 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:07.891 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "smb", 2024-05-21T22:06:07.891 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:07.891 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:07.891 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:07.891 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:07.891 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:07.891 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.891 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.891 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.891 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.891 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.891 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.892 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.892 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.892 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.892 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.892 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.892 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.892 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.892 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.892 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.892 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.892 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.892 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.892 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.892 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:07.893 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:07.893 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.893 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.893 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.893 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.893 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.893 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.893 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.893 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.893 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.893 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.893 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.893 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.893 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:07.894 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:07.894 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.894 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.894 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.894 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:07.894 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.894 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.894 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.894 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.894 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.894 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.894 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.894 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.895 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.895 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.895 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.895 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.895 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.895 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.895 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.895 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:07.895 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:07.895 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.895 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.895 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.895 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.895 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.895 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.896 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.896 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.896 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.896 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.896 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.896 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.896 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:07.896 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:07.896 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.896 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:07.896 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.896 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:07.896 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.896 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.896 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.897 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.897 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.897 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.897 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.897 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.897 INFO:teuthology.orchestra.run.smithi151.stdout: "update_orchestration": { 2024-05-21T22:06:07.897 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "update_orchestration", 2024-05-21T22:06:07.897 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.897 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.897 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.897 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:07.897 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.897 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.897 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.898 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-21T22:06:07.898 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.898 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.898 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.898 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.898 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.898 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.898 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:07.898 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "snap_schedule", 2024-05-21T22:06:07.898 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:07.898 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:07.898 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:07.898 INFO:teuthology.orchestra.run.smithi151.stdout: "allow_m_granularity": { 2024-05-21T22:06:07.899 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "allow_m_granularity", 2024-05-21T22:06:07.899 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.899 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.899 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.899 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.899 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.899 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.899 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.899 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "allow minute scheduled snapshots", 2024-05-21T22:06:07.899 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.899 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.899 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.899 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.900 INFO:teuthology.orchestra.run.smithi151.stdout: "dump_on_update": { 2024-05-21T22:06:07.900 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "dump_on_update", 2024-05-21T22:06:07.900 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.900 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.900 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.900 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.900 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.900 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.900 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.900 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "dump database to debug log on update", 2024-05-21T22:06:07.900 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.900 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.900 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.900 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.901 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:07.901 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:07.901 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.901 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.901 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.901 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.901 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.901 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.901 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.901 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.901 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.901 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.901 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.901 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.902 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.902 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.902 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.902 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.902 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.902 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.902 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.902 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:07.902 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:07.902 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.902 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.902 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.902 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.903 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.903 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.903 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.903 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.903 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.903 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.903 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.903 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.903 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:07.903 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:07.903 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.903 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.903 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.904 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:07.904 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.904 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.904 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.904 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.904 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.904 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.904 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.904 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.904 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.904 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.904 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.905 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.905 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.905 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.905 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.905 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:07.905 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:07.905 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.905 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.905 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.905 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.905 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.906 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.906 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.906 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.906 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.906 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.906 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.906 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.906 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:07.906 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:07.906 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.906 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:07.906 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.907 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:07.907 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.907 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.907 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.907 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.907 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.907 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.907 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.907 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.907 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.907 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.907 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:07.907 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "stats", 2024-05-21T22:06:07.907 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:07.908 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:07.908 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:07.908 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:07.908 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:07.908 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.908 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.908 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.908 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.908 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.908 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.908 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.908 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.908 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.908 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.909 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.909 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.909 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.909 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.909 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.909 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.909 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.909 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.909 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.909 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:07.909 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:07.909 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.909 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.909 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.910 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.910 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.910 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.910 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.910 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.910 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.910 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.910 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.910 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.910 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:07.910 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:07.910 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.910 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.911 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.911 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:07.911 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.911 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.911 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.911 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.911 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.911 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.911 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.911 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.911 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.911 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.911 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.911 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.911 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.912 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.912 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.912 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:07.912 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:07.912 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.912 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.912 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.912 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.912 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.912 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.912 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.912 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.912 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.912 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.912 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.913 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.913 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:07.913 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:07.913 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.913 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:07.913 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.913 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:07.913 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.913 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.913 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.913 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.913 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.913 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.913 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.913 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.914 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.914 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.914 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:07.914 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "status", 2024-05-21T22:06:07.914 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:07.914 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:07.914 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:07.914 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:07.914 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:07.914 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.914 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.914 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.914 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.914 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.915 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.915 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.915 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.915 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.915 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.915 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.915 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.915 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.915 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.915 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.915 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.915 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.915 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.915 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.915 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:07.916 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:07.916 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.916 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.916 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.916 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.916 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.916 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.916 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.916 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.916 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.916 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.916 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.917 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.917 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:07.917 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:07.917 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.917 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.917 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.917 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:07.917 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.917 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.917 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.917 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.917 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.917 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.917 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.918 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.918 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.918 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.918 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.918 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.918 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.918 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.918 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.918 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:07.918 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:07.918 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.918 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.918 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.918 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.919 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.919 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.919 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.919 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.919 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.919 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.919 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.919 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.919 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:07.919 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:07.919 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.919 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:07.919 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.919 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:07.920 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.920 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.920 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.920 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.920 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.920 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.920 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.920 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.920 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.920 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.920 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:07.920 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "telegraf", 2024-05-21T22:06:07.920 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:07.920 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:07.920 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:07.921 INFO:teuthology.orchestra.run.smithi151.stdout: "address": { 2024-05-21T22:06:07.921 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "address", 2024-05-21T22:06:07.921 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.921 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.921 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.921 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-21T22:06:07.921 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.921 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.921 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.921 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.921 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.921 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.921 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.921 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.922 INFO:teuthology.orchestra.run.smithi151.stdout: "interval": { 2024-05-21T22:06:07.922 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "interval", 2024-05-21T22:06:07.922 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-05-21T22:06:07.922 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.922 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.922 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "15", 2024-05-21T22:06:07.922 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.922 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.922 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.922 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.922 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.922 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.922 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.922 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.923 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:07.923 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:07.923 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.923 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.923 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.923 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.923 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.923 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.923 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.923 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.923 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.923 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.923 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.923 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.924 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.924 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.924 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.924 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.924 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.924 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.924 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.924 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:07.924 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:07.924 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.924 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.924 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.924 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.925 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.925 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.925 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.925 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.925 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.925 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.925 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.925 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.925 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:07.925 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:07.925 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.925 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.925 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.925 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:07.925 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.926 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.926 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.926 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.926 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.926 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.926 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.926 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.926 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.926 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.926 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.926 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.926 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.926 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.927 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.927 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:07.927 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:07.927 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.927 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.927 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.927 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.927 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.927 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.927 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.927 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.927 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.927 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.928 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.928 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.928 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:07.928 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:07.928 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.928 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:07.928 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.928 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:07.928 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.928 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.928 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.928 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.928 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.929 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.929 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.929 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.929 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.929 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.929 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:07.929 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "telemetry", 2024-05-21T22:06:07.929 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:07.929 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:07.929 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:07.929 INFO:teuthology.orchestra.run.smithi151.stdout: "channel_basic": { 2024-05-21T22:06:07.929 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "channel_basic", 2024-05-21T22:06:07.929 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.930 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.930 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.930 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:07.930 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.930 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.930 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.930 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-21T22:06:07.930 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.930 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.930 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.930 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.930 INFO:teuthology.orchestra.run.smithi151.stdout: "channel_crash": { 2024-05-21T22:06:07.930 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "channel_crash", 2024-05-21T22:06:07.931 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.931 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.931 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.931 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:07.931 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.931 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.931 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.931 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-21T22:06:07.931 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.931 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.931 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.931 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.931 INFO:teuthology.orchestra.run.smithi151.stdout: "channel_device": { 2024-05-21T22:06:07.932 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "channel_device", 2024-05-21T22:06:07.932 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.932 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.932 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.932 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:07.932 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.932 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.932 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.932 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-21T22:06:07.932 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.932 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.932 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.932 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.933 INFO:teuthology.orchestra.run.smithi151.stdout: "channel_ident": { 2024-05-21T22:06:07.933 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "channel_ident", 2024-05-21T22:06:07.933 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.933 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.933 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.933 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.933 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.933 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.933 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.933 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-21T22:06:07.933 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.933 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.933 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.934 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.934 INFO:teuthology.orchestra.run.smithi151.stdout: "channel_perf": { 2024-05-21T22:06:07.934 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "channel_perf", 2024-05-21T22:06:07.934 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.934 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.934 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.934 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.934 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.934 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.934 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.934 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-21T22:06:07.934 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.934 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.934 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.935 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.935 INFO:teuthology.orchestra.run.smithi151.stdout: "contact": { 2024-05-21T22:06:07.935 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "contact", 2024-05-21T22:06:07.935 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.935 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.935 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.935 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.935 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.935 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.935 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.935 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.935 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.935 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.936 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.936 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.936 INFO:teuthology.orchestra.run.smithi151.stdout: "description": { 2024-05-21T22:06:07.936 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "description", 2024-05-21T22:06:07.936 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.936 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.936 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.936 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.936 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.936 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.936 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.936 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.936 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.936 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.937 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.937 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.937 INFO:teuthology.orchestra.run.smithi151.stdout: "device_url": { 2024-05-21T22:06:07.937 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "device_url", 2024-05-21T22:06:07.937 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.937 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.937 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.937 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-21T22:06:07.937 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.937 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.937 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.937 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.937 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.938 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.938 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.938 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.938 INFO:teuthology.orchestra.run.smithi151.stdout: "enabled": { 2024-05-21T22:06:07.938 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "enabled", 2024-05-21T22:06:07.938 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.938 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.938 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.938 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.938 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.938 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.938 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.939 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.939 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.939 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.939 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.939 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.939 INFO:teuthology.orchestra.run.smithi151.stdout: "interval": { 2024-05-21T22:06:07.939 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "interval", 2024-05-21T22:06:07.939 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.939 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.939 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.939 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "24", 2024-05-21T22:06:07.939 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "8", 2024-05-21T22:06:07.939 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.940 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.940 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.940 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.940 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.940 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.940 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.940 INFO:teuthology.orchestra.run.smithi151.stdout: "last_opt_revision": { 2024-05-21T22:06:07.940 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "last_opt_revision", 2024-05-21T22:06:07.940 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.940 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.940 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.940 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "1", 2024-05-21T22:06:07.941 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.941 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.941 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.941 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.941 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.941 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.941 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.941 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.941 INFO:teuthology.orchestra.run.smithi151.stdout: "leaderboard": { 2024-05-21T22:06:07.941 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "leaderboard", 2024-05-21T22:06:07.941 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.941 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.941 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.941 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.941 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.942 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.942 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.942 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.942 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.942 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.942 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.942 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.942 INFO:teuthology.orchestra.run.smithi151.stdout: "leaderboard_description": { 2024-05-21T22:06:07.942 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "leaderboard_description", 2024-05-21T22:06:07.942 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.942 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.942 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.942 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.943 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.943 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.943 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.943 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.943 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.943 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.943 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.943 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.943 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:07.943 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:07.943 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.943 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.943 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.943 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.944 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.944 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.944 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.944 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.944 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.944 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.944 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.944 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.944 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.944 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.944 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.944 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.945 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.945 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.945 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.945 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:07.945 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:07.945 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.945 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.945 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.945 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.945 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.945 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.945 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.945 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.945 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.946 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.946 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.946 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.946 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:07.946 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:07.946 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.946 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.946 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.946 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:07.946 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.946 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.946 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.946 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.947 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.947 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.947 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.947 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.947 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.947 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.947 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.947 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.947 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.947 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.947 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.947 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:07.948 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:07.948 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.948 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.948 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.948 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.948 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.948 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.948 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.948 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.948 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.948 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.948 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.948 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.949 INFO:teuthology.orchestra.run.smithi151.stdout: "organization": { 2024-05-21T22:06:07.949 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "organization", 2024-05-21T22:06:07.949 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.949 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.949 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.949 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.949 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.949 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.949 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.949 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.949 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.949 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.949 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.950 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.950 INFO:teuthology.orchestra.run.smithi151.stdout: "proxy": { 2024-05-21T22:06:07.950 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "proxy", 2024-05-21T22:06:07.950 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.950 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.950 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.950 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.950 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.950 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.950 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.950 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.950 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.950 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.951 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.951 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.951 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:07.951 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:07.951 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.951 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:07.951 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.951 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:07.951 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.951 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.951 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.951 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.951 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.951 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.952 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.952 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.952 INFO:teuthology.orchestra.run.smithi151.stdout: "url": { 2024-05-21T22:06:07.952 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "url", 2024-05-21T22:06:07.952 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.952 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.952 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.952 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-21T22:06:07.952 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.952 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.952 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.952 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.952 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.952 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.952 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.953 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.953 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.953 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.953 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:07.953 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "test_orchestrator", 2024-05-21T22:06:07.953 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:07.953 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:07.953 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:07.953 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:07.953 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:07.953 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.953 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.953 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.954 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.954 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.954 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.954 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.954 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.954 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.954 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.954 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.954 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.954 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.954 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.954 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.955 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.955 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.955 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.955 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.955 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:07.955 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:07.955 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.955 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.955 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.955 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.955 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.955 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.955 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.955 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.956 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.956 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.956 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.956 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.956 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:07.956 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:07.956 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.956 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.956 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.956 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:07.956 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.956 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.956 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.956 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.956 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.957 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.957 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.957 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.957 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.957 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.957 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.957 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.957 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.957 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.957 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.957 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:07.957 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:07.957 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.957 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.957 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.958 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.958 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.958 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.958 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.958 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.958 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.958 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.958 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.958 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.958 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:07.958 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:07.958 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.958 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:07.959 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.959 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:07.959 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.959 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.959 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.959 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.959 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.959 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.959 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.959 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.959 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.959 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.959 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:07.960 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "volumes", 2024-05-21T22:06:07.960 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:07.960 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:07.960 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:07.960 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:07.960 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:07.960 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.960 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.960 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.960 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.960 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.960 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.960 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.961 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.961 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.961 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.961 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.961 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.961 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.961 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.961 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.961 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.961 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.961 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.961 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.961 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:07.961 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:07.962 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.962 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.962 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.962 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.962 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.962 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.962 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.962 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.962 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.962 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.962 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.962 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.963 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:07.963 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:07.963 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.963 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.963 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.963 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:07.963 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.963 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.963 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.963 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.963 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.963 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.963 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.964 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.964 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.964 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.964 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.964 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.964 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.964 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.964 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.964 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:07.964 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:07.964 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.964 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.964 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.964 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.965 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.965 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.965 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.965 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.965 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.965 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.965 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.965 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.965 INFO:teuthology.orchestra.run.smithi151.stdout: "max_concurrent_clones": { 2024-05-21T22:06:07.965 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "max_concurrent_clones", 2024-05-21T22:06:07.965 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.965 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.965 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.965 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "4", 2024-05-21T22:06:07.965 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.966 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.966 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.966 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-21T22:06:07.966 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.966 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.966 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.966 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.966 INFO:teuthology.orchestra.run.smithi151.stdout: "periodic_async_work": { 2024-05-21T22:06:07.966 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "periodic_async_work", 2024-05-21T22:06:07.966 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.966 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.966 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.966 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.967 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.967 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.967 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.967 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Periodically check for async work", 2024-05-21T22:06:07.967 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.967 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.967 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.967 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.967 INFO:teuthology.orchestra.run.smithi151.stdout: "snapshot_clone_delay": { 2024-05-21T22:06:07.967 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "snapshot_clone_delay", 2024-05-21T22:06:07.967 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.967 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.967 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.967 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:07.968 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.968 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.968 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.968 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-21T22:06:07.968 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.968 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.968 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.968 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.968 INFO:teuthology.orchestra.run.smithi151.stdout: "snapshot_clone_no_wait": { 2024-05-21T22:06:07.968 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "snapshot_clone_no_wait", 2024-05-21T22:06:07.968 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.968 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.969 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.969 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "True", 2024-05-21T22:06:07.969 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.969 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.969 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.969 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-21T22:06:07.969 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.969 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.969 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.969 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.969 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:07.969 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:07.969 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.970 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:07.970 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.970 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:07.970 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.970 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.970 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.970 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.970 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.970 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.970 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.970 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.970 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.971 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.971 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:07.971 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "zabbix", 2024-05-21T22:06:07.971 INFO:teuthology.orchestra.run.smithi151.stdout: "can_run": true, 2024-05-21T22:06:07.971 INFO:teuthology.orchestra.run.smithi151.stdout: "error_string": "", 2024-05-21T22:06:07.971 INFO:teuthology.orchestra.run.smithi151.stdout: "module_options": { 2024-05-21T22:06:07.971 INFO:teuthology.orchestra.run.smithi151.stdout: "discovery_interval": { 2024-05-21T22:06:07.971 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "discovery_interval", 2024-05-21T22:06:07.971 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "uint", 2024-05-21T22:06:07.971 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.971 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.971 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "100", 2024-05-21T22:06:07.972 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.972 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.972 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.972 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.972 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.972 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.972 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.972 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.972 INFO:teuthology.orchestra.run.smithi151.stdout: "identifier": { 2024-05-21T22:06:07.972 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "identifier", 2024-05-21T22:06:07.972 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.972 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.973 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.973 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.973 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.973 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.973 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.973 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.973 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.973 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.973 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.973 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.973 INFO:teuthology.orchestra.run.smithi151.stdout: "interval": { 2024-05-21T22:06:07.973 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "interval", 2024-05-21T22:06:07.973 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "secs", 2024-05-21T22:06:07.974 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.974 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.974 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "60", 2024-05-21T22:06:07.974 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.974 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.974 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.974 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.974 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.974 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.974 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.974 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.974 INFO:teuthology.orchestra.run.smithi151.stdout: "log_level": { 2024-05-21T22:06:07.974 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_level", 2024-05-21T22:06:07.975 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.975 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.975 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.975 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.975 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.975 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.975 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.975 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.975 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.975 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.975 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.975 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.975 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.976 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.976 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.976 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.976 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.976 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.976 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.976 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster": { 2024-05-21T22:06:07.976 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster", 2024-05-21T22:06:07.976 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.976 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.976 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.976 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.977 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.977 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.977 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.977 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.977 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.977 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.977 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.977 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.977 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_cluster_level": { 2024-05-21T22:06:07.977 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_cluster_level", 2024-05-21T22:06:07.977 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.977 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.977 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.977 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "info", 2024-05-21T22:06:07.977 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.978 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.978 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [ 2024-05-21T22:06:07.978 INFO:teuthology.orchestra.run.smithi151.stdout: "", 2024-05-21T22:06:07.978 INFO:teuthology.orchestra.run.smithi151.stdout: "critical", 2024-05-21T22:06:07.978 INFO:teuthology.orchestra.run.smithi151.stdout: "debug", 2024-05-21T22:06:07.978 INFO:teuthology.orchestra.run.smithi151.stdout: "error", 2024-05-21T22:06:07.978 INFO:teuthology.orchestra.run.smithi151.stdout: "info", 2024-05-21T22:06:07.978 INFO:teuthology.orchestra.run.smithi151.stdout: "warning" 2024-05-21T22:06:07.978 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.978 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.978 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.978 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.979 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.979 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.979 INFO:teuthology.orchestra.run.smithi151.stdout: "log_to_file": { 2024-05-21T22:06:07.979 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "log_to_file", 2024-05-21T22:06:07.979 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "bool", 2024-05-21T22:06:07.979 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.979 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.979 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "False", 2024-05-21T22:06:07.979 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.979 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.979 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.979 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.979 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.979 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.979 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.980 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.980 INFO:teuthology.orchestra.run.smithi151.stdout: "sqlite3_killpoint": { 2024-05-21T22:06:07.980 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sqlite3_killpoint", 2024-05-21T22:06:07.980 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.980 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "dev", 2024-05-21T22:06:07.980 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 1, 2024-05-21T22:06:07.980 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "0", 2024-05-21T22:06:07.980 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.980 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.980 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.980 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.980 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.980 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.980 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.980 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.981 INFO:teuthology.orchestra.run.smithi151.stdout: "zabbix_host": { 2024-05-21T22:06:07.981 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "zabbix_host", 2024-05-21T22:06:07.981 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.981 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.981 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.981 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "", 2024-05-21T22:06:07.981 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.981 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.981 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.981 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.981 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.981 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.981 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.981 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.982 INFO:teuthology.orchestra.run.smithi151.stdout: "zabbix_port": { 2024-05-21T22:06:07.982 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "zabbix_port", 2024-05-21T22:06:07.982 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "int", 2024-05-21T22:06:07.982 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.982 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.982 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "10051", 2024-05-21T22:06:07.982 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.982 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.982 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.982 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.982 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.982 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.983 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.983 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.983 INFO:teuthology.orchestra.run.smithi151.stdout: "zabbix_sender": { 2024-05-21T22:06:07.983 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "zabbix_sender", 2024-05-21T22:06:07.983 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "str", 2024-05-21T22:06:07.983 INFO:teuthology.orchestra.run.smithi151.stdout: "level": "advanced", 2024-05-21T22:06:07.983 INFO:teuthology.orchestra.run.smithi151.stdout: "flags": 0, 2024-05-21T22:06:07.983 INFO:teuthology.orchestra.run.smithi151.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-21T22:06:07.983 INFO:teuthology.orchestra.run.smithi151.stdout: "min": "", 2024-05-21T22:06:07.983 INFO:teuthology.orchestra.run.smithi151.stdout: "max": "", 2024-05-21T22:06:07.983 INFO:teuthology.orchestra.run.smithi151.stdout: "enum_allowed": [], 2024-05-21T22:06:07.983 INFO:teuthology.orchestra.run.smithi151.stdout: "desc": "", 2024-05-21T22:06:07.983 INFO:teuthology.orchestra.run.smithi151.stdout: "long_desc": "", 2024-05-21T22:06:07.984 INFO:teuthology.orchestra.run.smithi151.stdout: "tags": [], 2024-05-21T22:06:07.984 INFO:teuthology.orchestra.run.smithi151.stdout: "see_also": [] 2024-05-21T22:06:07.984 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.984 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.984 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.984 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.984 INFO:teuthology.orchestra.run.smithi151.stdout: "services": { 2024-05-21T22:06:07.984 INFO:teuthology.orchestra.run.smithi151.stdout: "dashboard": "https://172.21.15.151:8443/" 2024-05-21T22:06:07.984 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.984 INFO:teuthology.orchestra.run.smithi151.stdout: "always_on_modules": { 2024-05-21T22:06:07.984 INFO:teuthology.orchestra.run.smithi151.stdout: "octopus": [ 2024-05-21T22:06:07.984 INFO:teuthology.orchestra.run.smithi151.stdout: "balancer", 2024-05-21T22:06:07.984 INFO:teuthology.orchestra.run.smithi151.stdout: "crash", 2024-05-21T22:06:07.984 INFO:teuthology.orchestra.run.smithi151.stdout: "devicehealth", 2024-05-21T22:06:07.985 INFO:teuthology.orchestra.run.smithi151.stdout: "orchestrator", 2024-05-21T22:06:07.985 INFO:teuthology.orchestra.run.smithi151.stdout: "pg_autoscaler", 2024-05-21T22:06:07.985 INFO:teuthology.orchestra.run.smithi151.stdout: "progress", 2024-05-21T22:06:07.985 INFO:teuthology.orchestra.run.smithi151.stdout: "rbd_support", 2024-05-21T22:06:07.985 INFO:teuthology.orchestra.run.smithi151.stdout: "status", 2024-05-21T22:06:07.985 INFO:teuthology.orchestra.run.smithi151.stdout: "telemetry", 2024-05-21T22:06:07.985 INFO:teuthology.orchestra.run.smithi151.stdout: "volumes" 2024-05-21T22:06:07.985 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.985 INFO:teuthology.orchestra.run.smithi151.stdout: "pacific": [ 2024-05-21T22:06:07.985 INFO:teuthology.orchestra.run.smithi151.stdout: "balancer", 2024-05-21T22:06:07.985 INFO:teuthology.orchestra.run.smithi151.stdout: "crash", 2024-05-21T22:06:07.985 INFO:teuthology.orchestra.run.smithi151.stdout: "devicehealth", 2024-05-21T22:06:07.985 INFO:teuthology.orchestra.run.smithi151.stdout: "orchestrator", 2024-05-21T22:06:07.986 INFO:teuthology.orchestra.run.smithi151.stdout: "pg_autoscaler", 2024-05-21T22:06:07.986 INFO:teuthology.orchestra.run.smithi151.stdout: "progress", 2024-05-21T22:06:07.986 INFO:teuthology.orchestra.run.smithi151.stdout: "rbd_support", 2024-05-21T22:06:07.986 INFO:teuthology.orchestra.run.smithi151.stdout: "status", 2024-05-21T22:06:07.986 INFO:teuthology.orchestra.run.smithi151.stdout: "telemetry", 2024-05-21T22:06:07.986 INFO:teuthology.orchestra.run.smithi151.stdout: "volumes" 2024-05-21T22:06:07.986 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.986 INFO:teuthology.orchestra.run.smithi151.stdout: "quincy": [ 2024-05-21T22:06:07.986 INFO:teuthology.orchestra.run.smithi151.stdout: "balancer", 2024-05-21T22:06:07.986 INFO:teuthology.orchestra.run.smithi151.stdout: "crash", 2024-05-21T22:06:07.986 INFO:teuthology.orchestra.run.smithi151.stdout: "devicehealth", 2024-05-21T22:06:07.986 INFO:teuthology.orchestra.run.smithi151.stdout: "orchestrator", 2024-05-21T22:06:07.986 INFO:teuthology.orchestra.run.smithi151.stdout: "pg_autoscaler", 2024-05-21T22:06:07.987 INFO:teuthology.orchestra.run.smithi151.stdout: "progress", 2024-05-21T22:06:07.987 INFO:teuthology.orchestra.run.smithi151.stdout: "rbd_support", 2024-05-21T22:06:07.987 INFO:teuthology.orchestra.run.smithi151.stdout: "status", 2024-05-21T22:06:07.987 INFO:teuthology.orchestra.run.smithi151.stdout: "telemetry", 2024-05-21T22:06:07.987 INFO:teuthology.orchestra.run.smithi151.stdout: "volumes" 2024-05-21T22:06:07.987 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.987 INFO:teuthology.orchestra.run.smithi151.stdout: "reef": [ 2024-05-21T22:06:07.987 INFO:teuthology.orchestra.run.smithi151.stdout: "balancer", 2024-05-21T22:06:07.987 INFO:teuthology.orchestra.run.smithi151.stdout: "crash", 2024-05-21T22:06:07.987 INFO:teuthology.orchestra.run.smithi151.stdout: "devicehealth", 2024-05-21T22:06:07.987 INFO:teuthology.orchestra.run.smithi151.stdout: "orchestrator", 2024-05-21T22:06:07.987 INFO:teuthology.orchestra.run.smithi151.stdout: "pg_autoscaler", 2024-05-21T22:06:07.987 INFO:teuthology.orchestra.run.smithi151.stdout: "progress", 2024-05-21T22:06:07.987 INFO:teuthology.orchestra.run.smithi151.stdout: "rbd_support", 2024-05-21T22:06:07.988 INFO:teuthology.orchestra.run.smithi151.stdout: "status", 2024-05-21T22:06:07.988 INFO:teuthology.orchestra.run.smithi151.stdout: "telemetry", 2024-05-21T22:06:07.988 INFO:teuthology.orchestra.run.smithi151.stdout: "volumes" 2024-05-21T22:06:07.988 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-05-21T22:06:07.988 INFO:teuthology.orchestra.run.smithi151.stdout: "squid": [ 2024-05-21T22:06:07.988 INFO:teuthology.orchestra.run.smithi151.stdout: "balancer", 2024-05-21T22:06:07.988 INFO:teuthology.orchestra.run.smithi151.stdout: "crash", 2024-05-21T22:06:07.988 INFO:teuthology.orchestra.run.smithi151.stdout: "devicehealth", 2024-05-21T22:06:07.988 INFO:teuthology.orchestra.run.smithi151.stdout: "orchestrator", 2024-05-21T22:06:07.988 INFO:teuthology.orchestra.run.smithi151.stdout: "pg_autoscaler", 2024-05-21T22:06:07.988 INFO:teuthology.orchestra.run.smithi151.stdout: "progress", 2024-05-21T22:06:07.988 INFO:teuthology.orchestra.run.smithi151.stdout: "rbd_support", 2024-05-21T22:06:07.988 INFO:teuthology.orchestra.run.smithi151.stdout: "status", 2024-05-21T22:06:07.988 INFO:teuthology.orchestra.run.smithi151.stdout: "telemetry", 2024-05-21T22:06:07.989 INFO:teuthology.orchestra.run.smithi151.stdout: "volumes" 2024-05-21T22:06:07.989 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-05-21T22:06:07.989 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.989 INFO:teuthology.orchestra.run.smithi151.stdout: "last_failure_osd_epoch": 83, 2024-05-21T22:06:07.989 INFO:teuthology.orchestra.run.smithi151.stdout: "active_clients": [ 2024-05-21T22:06:07.989 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:07.989 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "devicehealth", 2024-05-21T22:06:07.989 INFO:teuthology.orchestra.run.smithi151.stdout: "addrvec": [ 2024-05-21T22:06:07.989 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:07.989 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "v2", 2024-05-21T22:06:07.989 INFO:teuthology.orchestra.run.smithi151.stdout: "addr": "172.21.15.151:0", 2024-05-21T22:06:07.989 INFO:teuthology.orchestra.run.smithi151.stdout: "nonce": 1797971414 2024-05-21T22:06:07.990 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.990 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-05-21T22:06:07.990 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.990 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:07.990 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "libcephsqlite", 2024-05-21T22:06:07.990 INFO:teuthology.orchestra.run.smithi151.stdout: "addrvec": [ 2024-05-21T22:06:07.990 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:07.990 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "v2", 2024-05-21T22:06:07.990 INFO:teuthology.orchestra.run.smithi151.stdout: "addr": "172.21.15.151:0", 2024-05-21T22:06:07.990 INFO:teuthology.orchestra.run.smithi151.stdout: "nonce": 3809831690 2024-05-21T22:06:07.990 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.990 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-05-21T22:06:07.990 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.991 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:07.991 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "rbd_support", 2024-05-21T22:06:07.991 INFO:teuthology.orchestra.run.smithi151.stdout: "addrvec": [ 2024-05-21T22:06:07.991 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:07.991 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "v2", 2024-05-21T22:06:07.991 INFO:teuthology.orchestra.run.smithi151.stdout: "addr": "172.21.15.151:0", 2024-05-21T22:06:07.991 INFO:teuthology.orchestra.run.smithi151.stdout: "nonce": 1032558035 2024-05-21T22:06:07.991 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.991 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-05-21T22:06:07.991 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.991 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:07.991 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "snap_schedule", 2024-05-21T22:06:07.991 INFO:teuthology.orchestra.run.smithi151.stdout: "addrvec": [ 2024-05-21T22:06:07.991 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:07.992 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "v2", 2024-05-21T22:06:07.992 INFO:teuthology.orchestra.run.smithi151.stdout: "addr": "172.21.15.151:0", 2024-05-21T22:06:07.992 INFO:teuthology.orchestra.run.smithi151.stdout: "nonce": 1923595685 2024-05-21T22:06:07.992 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.992 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-05-21T22:06:07.992 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.992 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:07.992 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "snap_schedule", 2024-05-21T22:06:07.992 INFO:teuthology.orchestra.run.smithi151.stdout: "addrvec": [ 2024-05-21T22:06:07.992 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:07.992 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "v2", 2024-05-21T22:06:07.992 INFO:teuthology.orchestra.run.smithi151.stdout: "addr": "172.21.15.151:0", 2024-05-21T22:06:07.992 INFO:teuthology.orchestra.run.smithi151.stdout: "nonce": 502761441 2024-05-21T22:06:07.992 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.993 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-05-21T22:06:07.993 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:07.993 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:07.993 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "volumes", 2024-05-21T22:06:07.993 INFO:teuthology.orchestra.run.smithi151.stdout: "addrvec": [ 2024-05-21T22:06:07.993 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-05-21T22:06:07.993 INFO:teuthology.orchestra.run.smithi151.stdout: "type": "v2", 2024-05-21T22:06:07.993 INFO:teuthology.orchestra.run.smithi151.stdout: "addr": "172.21.15.151:0", 2024-05-21T22:06:07.993 INFO:teuthology.orchestra.run.smithi151.stdout: "nonce": 3646578579 2024-05-21T22:06:07.993 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.993 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-05-21T22:06:07.993 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:07.993 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-05-21T22:06:07.993 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-05-21T22:06:07.994 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:06:06 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/4100811385' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-21T22:06:07.994 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:06:06 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/2217178833' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-21T22:06:07.995 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:06:06 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/4100811385' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-21T22:06:07.995 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:06:06 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/2217178833' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-21T22:06:07.995 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:06:07 smithi184 ceph-mon[36942]: pgmap v324: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 27 op/s 2024-05-21T22:06:07.996 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:06.414+0000 7fa817000640 1 -- 172.21.15.151:0/2217178833 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fa7f40b29e0 msgr2=0x7fa7f40b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:06:07.996 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:06.414+0000 7fa817000640 1 --2- 172.21.15.151:0/2217178833 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fa7f40b29e0 0x7fa7f40b4ea0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa8000095f0 tx=0x7fa800005f90 comp rx=0 tx=0).stop 2024-05-21T22:06:07.996 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:06.414+0000 7fa817000640 1 -- 172.21.15.151:0/2217178833 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa810113a30 msgr2=0x7fa81019f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:06:07.996 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:06.414+0000 7fa817000640 1 --2- 172.21.15.151:0/2217178833 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa810113a30 0x7fa81019f670 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fa80800eb40 tx=0x7fa80800c2d0 comp rx=0 tx=0).stop 2024-05-21T22:06:07.996 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:06.415+0000 7fa817000640 1 -- 172.21.15.151:0/2217178833 shutdown_connections 2024-05-21T22:06:07.996 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:06.415+0000 7fa817000640 1 --2- 172.21.15.151:0/2217178833 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fa7f40b29e0 0x7fa7f40b4ea0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:07.996 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:06.415+0000 7fa817000640 1 --2- 172.21.15.151:0/2217178833 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa810113a30 0x7fa81019f670 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:07.996 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:06.415+0000 7fa817000640 1 --2- 172.21.15.151:0/2217178833 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa810075710 0x7fa8101a5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:07.997 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:06.415+0000 7fa817000640 1 --2- 172.21.15.151:0/2217178833 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa810074df0 0x7fa8101a55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:07.997 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:06.415+0000 7fa817000640 1 -- 172.21.15.151:0/2217178833 >> 172.21.15.151:0/2217178833 conn(0x7fa810079e10 msgr2=0x7fa8100ff7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:06:07.997 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:06.415+0000 7fa817000640 1 -- 172.21.15.151:0/2217178833 shutdown_connections 2024-05-21T22:06:07.997 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:06.415+0000 7fa817000640 1 -- 172.21.15.151:0/2217178833 wait complete. 2024-05-21T22:06:08.051 DEBUG:tasks.check_counter:Getting stats from a 2024-05-21T22:06:08.051 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph daemon mds.a perf dump 2024-05-21T22:06:08.051 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:06:06 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/4100811385' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-21T22:06:08.052 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:06:06 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/2217178833' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-21T22:06:08.052 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:06:07 smithi151 ceph-mon[31623]: pgmap v324: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 27 op/s 2024-05-21T22:06:08.052 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:06:07 smithi164 ceph-mon[36269]: pgmap v324: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 27 op/s 2024-05-21T22:06:08.515 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:06:09.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:06:09 smithi151 ceph-mon[31623]: pgmap v325: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 44 KiB/s wr, 32 op/s 2024-05-21T22:06:09.704 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:06:09 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T22:06:09.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:06:09 smithi184 ceph-mon[36942]: pgmap v325: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 44 KiB/s wr, 32 op/s 2024-05-21T22:06:09.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:06:09 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T22:06:09.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:06:09 smithi164 ceph-mon[36269]: pgmap v325: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 44 KiB/s wr, 32 op/s 2024-05-21T22:06:09.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:06:09 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T22:06:10.735 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:06:10 smithi151 ceph-mon[31623]: osdmap e90: 12 total, 12 up, 12 in 2024-05-21T22:06:10.736 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:06:10 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T22:06:10.736 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:06:10 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T22:06:10.736 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:06:10 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T22:06:10.736 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:06:10 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T22:06:10.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:06:10 smithi184 ceph-mon[36942]: osdmap e90: 12 total, 12 up, 12 in 2024-05-21T22:06:10.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:06:10 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T22:06:10.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:06:10 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T22:06:10.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:06:10 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T22:06:10.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:06:10 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T22:06:11.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:06:10 smithi164 ceph-mon[36269]: osdmap e90: 12 total, 12 up, 12 in 2024-05-21T22:06:11.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:06:10 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T22:06:11.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:06:10 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T22:06:11.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:06:10 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T22:06:11.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:06:10 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T22:06:11.405 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-05-21T22:06:11.406 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-0": { 2024-05-21T22:06:11.406 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 1143, 2024-05-21T22:06:11.406 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 1129, 2024-05-21T22:06:11.406 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 2198016, 2024-05-21T22:06:11.406 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 429892, 2024-05-21T22:06:11.406 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 5, 2024-05-21T22:06:11.406 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 1, 2024-05-21T22:06:11.406 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.310445900, 2024-05-21T22:06:11.406 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.131388988, 2024-05-21T22:06:11.406 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.123290634, 2024-05-21T22:06:11.407 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.056162393, 2024-05-21T22:06:11.407 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T22:06:11.407 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 1128, 2024-05-21T22:06:11.407 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.106669836, 2024-05-21T22:06:11.407 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000094565 2024-05-21T22:06:11.407 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:11.407 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-05-21T22:06:11.407 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-21T22:06:11.407 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-21T22:06:11.407 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-21T22:06:11.407 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:11.407 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 1905696, 2024-05-21T22:06:11.407 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 153232 2024-05-21T22:06:11.408 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:11.408 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-1": { 2024-05-21T22:06:11.408 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 1513, 2024-05-21T22:06:11.408 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 1696, 2024-05-21T22:06:11.408 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 389138, 2024-05-21T22:06:11.408 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 968163, 2024-05-21T22:06:11.408 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 7, 2024-05-21T22:06:11.408 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 5, 2024-05-21T22:06:11.408 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.263038819, 2024-05-21T22:06:11.408 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.104816831, 2024-05-21T22:06:11.408 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.062794185, 2024-05-21T22:06:11.408 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.071063919, 2024-05-21T22:06:11.409 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T22:06:11.409 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 1694, 2024-05-21T22:06:11.409 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.147990116, 2024-05-21T22:06:11.409 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000087361 2024-05-21T22:06:11.409 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:11.409 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-05-21T22:06:11.409 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-21T22:06:11.409 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-21T22:06:11.409 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-21T22:06:11.409 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:11.409 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-05-21T22:06:11.409 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 488880 2024-05-21T22:06:11.410 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:11.410 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-2": { 2024-05-21T22:06:11.410 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 4, 2024-05-21T22:06:11.410 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 6, 2024-05-21T22:06:11.410 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 1629, 2024-05-21T22:06:11.410 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 1284, 2024-05-21T22:06:11.410 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 5, 2024-05-21T22:06:11.410 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 4, 2024-05-21T22:06:11.410 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.003745781, 2024-05-21T22:06:11.410 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.000197436, 2024-05-21T22:06:11.410 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.003089429, 2024-05-21T22:06:11.410 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.000199264, 2024-05-21T22:06:11.410 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T22:06:11.411 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 4, 2024-05-21T22:06:11.411 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.004671813, 2024-05-21T22:06:11.411 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.001167953 2024-05-21T22:06:11.411 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:11.411 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-05-21T22:06:11.411 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-21T22:06:11.411 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-21T22:06:11.411 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-21T22:06:11.411 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:11.411 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-21T22:06:11.411 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 0 2024-05-21T22:06:11.411 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:11.411 INFO:teuthology.orchestra.run.smithi151.stdout: "cct": { 2024-05-21T22:06:11.411 INFO:teuthology.orchestra.run.smithi151.stdout: "total_workers": 1, 2024-05-21T22:06:11.412 INFO:teuthology.orchestra.run.smithi151.stdout: "unhealthy_workers": 0 2024-05-21T22:06:11.412 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:11.412 INFO:teuthology.orchestra.run.smithi151.stdout: "finisher-MDSRank": { 2024-05-21T22:06:11.412 INFO:teuthology.orchestra.run.smithi151.stdout: "queue_len": 0, 2024-05-21T22:06:11.412 INFO:teuthology.orchestra.run.smithi151.stdout: "complete_latency": { 2024-05-21T22:06:11.412 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 4105, 2024-05-21T22:06:11.412 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.187203761, 2024-05-21T22:06:11.412 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000045603 2024-05-21T22:06:11.412 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:11.412 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:11.412 INFO:teuthology.orchestra.run.smithi151.stdout: "finisher-PurgeQueue": { 2024-05-21T22:06:11.412 INFO:teuthology.orchestra.run.smithi151.stdout: "queue_len": 0, 2024-05-21T22:06:11.412 INFO:teuthology.orchestra.run.smithi151.stdout: "complete_latency": { 2024-05-21T22:06:11.412 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-21T22:06:11.413 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-21T22:06:11.413 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-21T22:06:11.413 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:11.413 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:11.413 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": { 2024-05-21T22:06:11.413 INFO:teuthology.orchestra.run.smithi151.stdout: "request": 0, 2024-05-21T22:06:11.413 INFO:teuthology.orchestra.run.smithi151.stdout: "reply": 0, 2024-05-21T22:06:11.413 INFO:teuthology.orchestra.run.smithi151.stdout: "reply_latency": { 2024-05-21T22:06:11.413 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-21T22:06:11.413 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-21T22:06:11.413 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-21T22:06:11.413 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:11.413 INFO:teuthology.orchestra.run.smithi151.stdout: "slow_reply": 0, 2024-05-21T22:06:11.414 INFO:teuthology.orchestra.run.smithi151.stdout: "forward": 0, 2024-05-21T22:06:11.414 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_fetch_complete": 0, 2024-05-21T22:06:11.414 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_fetch_keys": 0, 2024-05-21T22:06:11.414 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_commit": 0, 2024-05-21T22:06:11.414 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_split": 0, 2024-05-21T22:06:11.414 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_merge": 0, 2024-05-21T22:06:11.414 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes": 0, 2024-05-21T22:06:11.414 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_top": 0, 2024-05-21T22:06:11.414 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_bottom": 0, 2024-05-21T22:06:11.414 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_pin_tail": 0, 2024-05-21T22:06:11.414 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_pinned": 0, 2024-05-21T22:06:11.414 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_expired": 0, 2024-05-21T22:06:11.414 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_with_caps": 0, 2024-05-21T22:06:11.414 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": 0, 2024-05-21T22:06:11.415 INFO:teuthology.orchestra.run.smithi151.stdout: "subtrees": 1, 2024-05-21T22:06:11.415 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse": 0, 2024-05-21T22:06:11.415 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_hit": 0, 2024-05-21T22:06:11.415 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_forward": 0, 2024-05-21T22:06:11.415 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_discover": 0, 2024-05-21T22:06:11.415 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_dir_fetch": 0, 2024-05-21T22:06:11.415 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_remote_ino": 0, 2024-05-21T22:06:11.415 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_lock": 0, 2024-05-21T22:06:11.415 INFO:teuthology.orchestra.run.smithi151.stdout: "load_cent": 0, 2024-05-21T22:06:11.415 INFO:teuthology.orchestra.run.smithi151.stdout: "q": 0, 2024-05-21T22:06:11.415 INFO:teuthology.orchestra.run.smithi151.stdout: "exported": 0, 2024-05-21T22:06:11.415 INFO:teuthology.orchestra.run.smithi151.stdout: "exported_inodes": 0, 2024-05-21T22:06:11.415 INFO:teuthology.orchestra.run.smithi151.stdout: "imported": 0, 2024-05-21T22:06:11.416 INFO:teuthology.orchestra.run.smithi151.stdout: "imported_inodes": 0, 2024-05-21T22:06:11.416 INFO:teuthology.orchestra.run.smithi151.stdout: "openino_dir_fetch": 0, 2024-05-21T22:06:11.416 INFO:teuthology.orchestra.run.smithi151.stdout: "openino_backtrace_fetch": 0, 2024-05-21T22:06:11.416 INFO:teuthology.orchestra.run.smithi151.stdout: "openino_peer_discover": 0, 2024-05-21T22:06:11.416 INFO:teuthology.orchestra.run.smithi151.stdout: "root_rfiles": 0, 2024-05-21T22:06:11.416 INFO:teuthology.orchestra.run.smithi151.stdout: "root_rbytes": 0, 2024-05-21T22:06:11.416 INFO:teuthology.orchestra.run.smithi151.stdout: "root_rsnaps": 0, 2024-05-21T22:06:11.416 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_backtrace_fetch": 0, 2024-05-21T22:06:11.416 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_set_tag": 0, 2024-05-21T22:06:11.416 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_backtrace_repaired": 0, 2024-05-21T22:06:11.416 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_inotable_repaired": 0, 2024-05-21T22:06:11.416 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_dir_inodes": 0, 2024-05-21T22:06:11.416 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_dir_base_inodes": 0, 2024-05-21T22:06:11.417 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_dirfrag_rstats": 0, 2024-05-21T22:06:11.417 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_file_inodes": 0, 2024-05-21T22:06:11.417 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_inode_file_caps": 0, 2024-05-21T22:06:11.417 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_revoke": 0, 2024-05-21T22:06:11.417 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_grant": 0, 2024-05-21T22:06:11.417 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_trunc": 0, 2024-05-21T22:06:11.417 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-21T22:06:11.417 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-21T22:06:11.417 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_caps": 0, 2024-05-21T22:06:11.417 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_caps_dirty": 0, 2024-05-21T22:06:11.417 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_cap_release": 0, 2024-05-21T22:06:11.417 INFO:teuthology.orchestra.run.smithi151.stdout: "process_request_cap_release": 0 2024-05-21T22:06:11.417 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:11.417 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_cache": { 2024-05-21T22:06:11.417 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_update": 0, 2024-05-21T22:06:11.418 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_update_receipt": 0, 2024-05-21T22:06:11.418 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_try_discover": 0, 2024-05-21T22:06:11.418 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_send_discover": 0, 2024-05-21T22:06:11.418 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_handle_discover": 0, 2024-05-21T22:06:11.418 INFO:teuthology.orchestra.run.smithi151.stdout: "num_strays": 0, 2024-05-21T22:06:11.418 INFO:teuthology.orchestra.run.smithi151.stdout: "num_strays_delayed": 0, 2024-05-21T22:06:11.418 INFO:teuthology.orchestra.run.smithi151.stdout: "num_strays_enqueuing": 0, 2024-05-21T22:06:11.418 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_created": 0, 2024-05-21T22:06:11.418 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_enqueued": 0, 2024-05-21T22:06:11.418 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_reintegrated": 0, 2024-05-21T22:06:11.418 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_migrated": 0, 2024-05-21T22:06:11.418 INFO:teuthology.orchestra.run.smithi151.stdout: "num_recovering_processing": 0, 2024-05-21T22:06:11.418 INFO:teuthology.orchestra.run.smithi151.stdout: "num_recovering_enqueued": 0, 2024-05-21T22:06:11.419 INFO:teuthology.orchestra.run.smithi151.stdout: "num_recovering_prioritized": 0, 2024-05-21T22:06:11.419 INFO:teuthology.orchestra.run.smithi151.stdout: "recovery_started": 0, 2024-05-21T22:06:11.419 INFO:teuthology.orchestra.run.smithi151.stdout: "recovery_completed": 0, 2024-05-21T22:06:11.419 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_quiesce_path": 0, 2024-05-21T22:06:11.419 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_quiesce_inode": 0, 2024-05-21T22:06:11.419 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_enqueue_scrub": 0, 2024-05-21T22:06:11.419 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_exportdir": 0, 2024-05-21T22:06:11.419 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_flush": 0, 2024-05-21T22:06:11.419 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_fragmentdir": 0, 2024-05-21T22:06:11.419 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_fragstats": 0, 2024-05-21T22:06:11.419 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_inodestats": 0 2024-05-21T22:06:11.419 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:11.419 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_log": { 2024-05-21T22:06:11.419 INFO:teuthology.orchestra.run.smithi151.stdout: "evlrg": 0, 2024-05-21T22:06:11.420 INFO:teuthology.orchestra.run.smithi151.stdout: "evadd": 0, 2024-05-21T22:06:11.420 INFO:teuthology.orchestra.run.smithi151.stdout: "evex": 0, 2024-05-21T22:06:11.420 INFO:teuthology.orchestra.run.smithi151.stdout: "evtrm": 0, 2024-05-21T22:06:11.420 INFO:teuthology.orchestra.run.smithi151.stdout: "ev": 4, 2024-05-21T22:06:11.420 INFO:teuthology.orchestra.run.smithi151.stdout: "evexg": 0, 2024-05-21T22:06:11.420 INFO:teuthology.orchestra.run.smithi151.stdout: "evexd": 0, 2024-05-21T22:06:11.420 INFO:teuthology.orchestra.run.smithi151.stdout: "segadd": 0, 2024-05-21T22:06:11.420 INFO:teuthology.orchestra.run.smithi151.stdout: "segex": 0, 2024-05-21T22:06:11.420 INFO:teuthology.orchestra.run.smithi151.stdout: "segtrm": 0, 2024-05-21T22:06:11.420 INFO:teuthology.orchestra.run.smithi151.stdout: "seg": 2, 2024-05-21T22:06:11.420 INFO:teuthology.orchestra.run.smithi151.stdout: "segmjr": 2, 2024-05-21T22:06:11.420 INFO:teuthology.orchestra.run.smithi151.stdout: "segexg": 0, 2024-05-21T22:06:11.420 INFO:teuthology.orchestra.run.smithi151.stdout: "segexd": 0, 2024-05-21T22:06:11.421 INFO:teuthology.orchestra.run.smithi151.stdout: "expos": 4194304, 2024-05-21T22:06:11.421 INFO:teuthology.orchestra.run.smithi151.stdout: "wrpos": 4195424, 2024-05-21T22:06:11.421 INFO:teuthology.orchestra.run.smithi151.stdout: "rdpos": 4195283, 2024-05-21T22:06:11.421 INFO:teuthology.orchestra.run.smithi151.stdout: "jlat": { 2024-05-21T22:06:11.421 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-21T22:06:11.421 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-21T22:06:11.421 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-21T22:06:11.421 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:11.421 INFO:teuthology.orchestra.run.smithi151.stdout: "replayed": 4 2024-05-21T22:06:11.421 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:11.421 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_mem": { 2024-05-21T22:06:11.421 INFO:teuthology.orchestra.run.smithi151.stdout: "ino": 3, 2024-05-21T22:06:11.421 INFO:teuthology.orchestra.run.smithi151.stdout: "ino+": 3, 2024-05-21T22:06:11.421 INFO:teuthology.orchestra.run.smithi151.stdout: "ino-": 0, 2024-05-21T22:06:11.422 INFO:teuthology.orchestra.run.smithi151.stdout: "dir": 1, 2024-05-21T22:06:11.422 INFO:teuthology.orchestra.run.smithi151.stdout: "dir+": 1, 2024-05-21T22:06:11.422 INFO:teuthology.orchestra.run.smithi151.stdout: "dir-": 0, 2024-05-21T22:06:11.422 INFO:teuthology.orchestra.run.smithi151.stdout: "dn": 0, 2024-05-21T22:06:11.422 INFO:teuthology.orchestra.run.smithi151.stdout: "dn+": 0, 2024-05-21T22:06:11.422 INFO:teuthology.orchestra.run.smithi151.stdout: "dn-": 0, 2024-05-21T22:06:11.422 INFO:teuthology.orchestra.run.smithi151.stdout: "cap": 0, 2024-05-21T22:06:11.422 INFO:teuthology.orchestra.run.smithi151.stdout: "cap+": 0, 2024-05-21T22:06:11.422 INFO:teuthology.orchestra.run.smithi151.stdout: "cap-": 0, 2024-05-21T22:06:11.422 INFO:teuthology.orchestra.run.smithi151.stdout: "rss": 49112, 2024-05-21T22:06:11.422 INFO:teuthology.orchestra.run.smithi151.stdout: "heap": 215416 2024-05-21T22:06:11.423 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:11.423 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_server": { 2024-05-21T22:06:11.423 INFO:teuthology.orchestra.run.smithi151.stdout: "dispatch_client_request": 0, 2024-05-21T22:06:11.423 INFO:teuthology.orchestra.run.smithi151.stdout: "dispatch_server_request": 0, 2024-05-21T22:06:11.423 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_request": 0, 2024-05-21T22:06:11.423 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_session": 0, 2024-05-21T22:06:11.423 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_peer_request": 0, 2024-05-21T22:06:11.423 INFO:teuthology.orchestra.run.smithi151.stdout: "req_create_latency": { 2024-05-21T22:06:11.423 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-21T22:06:11.423 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-21T22:06:11.423 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-21T22:06:11.423 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:11.423 INFO:teuthology.orchestra.run.smithi151.stdout: "req_getattr_latency": { 2024-05-21T22:06:11.423 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-21T22:06:11.424 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-21T22:06:11.424 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-21T22:06:11.424 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:11.424 INFO:teuthology.orchestra.run.smithi151.stdout: "req_getfilelock_latency": { 2024-05-21T22:06:11.424 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-21T22:06:11.424 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-21T22:06:11.424 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-21T22:06:11.424 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:11.424 INFO:teuthology.orchestra.run.smithi151.stdout: "req_link_latency": { 2024-05-21T22:06:11.424 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-21T22:06:11.424 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-21T22:06:11.424 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-21T22:06:11.424 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:11.424 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookup_latency": { 2024-05-21T22:06:11.425 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-21T22:06:11.425 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-21T22:06:11.425 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-21T22:06:11.425 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:11.425 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookuphash_latency": { 2024-05-21T22:06:11.425 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-21T22:06:11.425 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-21T22:06:11.425 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-21T22:06:11.425 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:11.425 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupino_latency": { 2024-05-21T22:06:11.425 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-21T22:06:11.425 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-21T22:06:11.425 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-21T22:06:11.425 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:11.426 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupname_latency": { 2024-05-21T22:06:11.426 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-21T22:06:11.426 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-21T22:06:11.426 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-21T22:06:11.426 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:11.426 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupparent_latency": { 2024-05-21T22:06:11.426 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-21T22:06:11.426 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-21T22:06:11.426 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-21T22:06:11.426 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:11.426 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupsnap_latency": { 2024-05-21T22:06:11.426 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-21T22:06:11.426 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-21T22:06:11.426 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-21T22:06:11.427 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:11.427 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lssnap_latency": { 2024-05-21T22:06:11.427 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-21T22:06:11.427 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-21T22:06:11.427 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-21T22:06:11.427 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:11.427 INFO:teuthology.orchestra.run.smithi151.stdout: "req_mkdir_latency": { 2024-05-21T22:06:11.427 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-21T22:06:11.427 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-21T22:06:11.427 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-21T22:06:11.427 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:11.427 INFO:teuthology.orchestra.run.smithi151.stdout: "req_mknod_latency": { 2024-05-21T22:06:11.427 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-21T22:06:11.427 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-21T22:06:11.428 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-21T22:06:11.428 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:11.428 INFO:teuthology.orchestra.run.smithi151.stdout: "req_mksnap_latency": { 2024-05-21T22:06:11.428 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-21T22:06:11.428 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-21T22:06:11.428 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-21T22:06:11.428 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:11.428 INFO:teuthology.orchestra.run.smithi151.stdout: "req_open_latency": { 2024-05-21T22:06:11.428 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-21T22:06:11.428 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-21T22:06:11.428 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-21T22:06:11.428 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:11.429 INFO:teuthology.orchestra.run.smithi151.stdout: "req_readdir_latency": { 2024-05-21T22:06:11.429 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-21T22:06:11.429 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-21T22:06:11.429 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-21T22:06:11.429 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:11.429 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rename_latency": { 2024-05-21T22:06:11.429 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-21T22:06:11.429 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-21T22:06:11.429 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-21T22:06:11.429 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:11.429 INFO:teuthology.orchestra.run.smithi151.stdout: "req_renamesnap_latency": { 2024-05-21T22:06:11.429 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-21T22:06:11.429 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-21T22:06:11.430 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-21T22:06:11.430 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:11.430 INFO:teuthology.orchestra.run.smithi151.stdout: "req_snapdiff_latency": { 2024-05-21T22:06:11.430 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-21T22:06:11.430 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-21T22:06:11.430 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-21T22:06:11.430 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:11.430 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rmdir_latency": { 2024-05-21T22:06:11.430 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-21T22:06:11.430 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-21T22:06:11.430 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-21T22:06:11.430 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:11.430 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rmsnap_latency": { 2024-05-21T22:06:11.431 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-21T22:06:11.431 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-21T22:06:11.431 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-21T22:06:11.431 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:11.431 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rmxattr_latency": { 2024-05-21T22:06:11.431 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-21T22:06:11.431 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-21T22:06:11.431 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-21T22:06:11.431 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:11.431 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setattr_latency": { 2024-05-21T22:06:11.431 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-21T22:06:11.431 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-21T22:06:11.431 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-21T22:06:11.432 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:11.432 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setdirlayout_latency": { 2024-05-21T22:06:11.432 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-21T22:06:11.432 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-21T22:06:11.432 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-21T22:06:11.432 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:11.432 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setfilelock_latency": { 2024-05-21T22:06:11.432 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-21T22:06:11.432 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-21T22:06:11.432 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-21T22:06:11.432 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:11.432 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setlayout_latency": { 2024-05-21T22:06:11.432 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-21T22:06:11.432 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-21T22:06:11.433 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-21T22:06:11.433 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:11.433 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setxattr_latency": { 2024-05-21T22:06:11.433 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-21T22:06:11.433 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-21T22:06:11.433 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-21T22:06:11.433 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:11.433 INFO:teuthology.orchestra.run.smithi151.stdout: "req_symlink_latency": { 2024-05-21T22:06:11.433 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-21T22:06:11.433 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-21T22:06:11.433 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-21T22:06:11.433 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:11.433 INFO:teuthology.orchestra.run.smithi151.stdout: "req_unlink_latency": { 2024-05-21T22:06:11.433 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-21T22:06:11.434 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-21T22:06:11.434 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-21T22:06:11.434 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:11.434 INFO:teuthology.orchestra.run.smithi151.stdout: "cap_revoke_eviction": 0, 2024-05-21T22:06:11.434 INFO:teuthology.orchestra.run.smithi151.stdout: "cap_acquisition_throttle": 0, 2024-05-21T22:06:11.434 INFO:teuthology.orchestra.run.smithi151.stdout: "req_getvxattr_latency": { 2024-05-21T22:06:11.434 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-21T22:06:11.434 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-21T22:06:11.434 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-21T22:06:11.434 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:11.434 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:11.434 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_sessions": { 2024-05-21T22:06:11.435 INFO:teuthology.orchestra.run.smithi151.stdout: "session_count": 0, 2024-05-21T22:06:11.435 INFO:teuthology.orchestra.run.smithi151.stdout: "session_add": 1, 2024-05-21T22:06:11.435 INFO:teuthology.orchestra.run.smithi151.stdout: "session_remove": 1, 2024-05-21T22:06:11.435 INFO:teuthology.orchestra.run.smithi151.stdout: "sessions_open": 1, 2024-05-21T22:06:11.435 INFO:teuthology.orchestra.run.smithi151.stdout: "sessions_stale": 0, 2024-05-21T22:06:11.435 INFO:teuthology.orchestra.run.smithi151.stdout: "total_load": 0, 2024-05-21T22:06:11.435 INFO:teuthology.orchestra.run.smithi151.stdout: "average_load": 0, 2024-05-21T22:06:11.435 INFO:teuthology.orchestra.run.smithi151.stdout: "avg_session_uptime": 1553, 2024-05-21T22:06:11.435 INFO:teuthology.orchestra.run.smithi151.stdout: "mdthresh_evicted": 0 2024-05-21T22:06:11.435 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:11.435 INFO:teuthology.orchestra.run.smithi151.stdout: "mempool": { 2024-05-21T22:06:11.435 INFO:teuthology.orchestra.run.smithi151.stdout: "bloom_filter_bytes": 0, 2024-05-21T22:06:11.435 INFO:teuthology.orchestra.run.smithi151.stdout: "bloom_filter_items": 0, 2024-05-21T22:06:11.435 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_alloc_bytes": 0, 2024-05-21T22:06:11.435 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_alloc_items": 0, 2024-05-21T22:06:11.436 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_data_bytes": 0, 2024-05-21T22:06:11.436 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_data_items": 0, 2024-05-21T22:06:11.436 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-21T22:06:11.436 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_onode_items": 0, 2024-05-21T22:06:11.436 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-21T22:06:11.436 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_meta_items": 0, 2024-05-21T22:06:11.436 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_other_bytes": 0, 2024-05-21T22:06:11.436 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_other_items": 0, 2024-05-21T22:06:11.436 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-21T22:06:11.436 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_buffer_items": 0, 2024-05-21T22:06:11.436 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_extent_bytes": 0, 2024-05-21T22:06:11.436 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_extent_items": 0, 2024-05-21T22:06:11.436 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_blob_bytes": 0, 2024-05-21T22:06:11.436 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_blob_items": 0, 2024-05-21T22:06:11.437 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-21T22:06:11.437 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_shared_blob_items": 0, 2024-05-21T22:06:11.437 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-21T22:06:11.437 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_inline_bl_items": 0, 2024-05-21T22:06:11.437 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_fsck_bytes": 0, 2024-05-21T22:06:11.437 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_fsck_items": 0, 2024-05-21T22:06:11.437 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_txc_bytes": 0, 2024-05-21T22:06:11.437 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_txc_items": 0, 2024-05-21T22:06:11.437 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-21T22:06:11.437 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_deferred_items": 0, 2024-05-21T22:06:11.437 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_bytes": 0, 2024-05-21T22:06:11.437 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_items": 0, 2024-05-21T22:06:11.437 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_bytes": 0, 2024-05-21T22:06:11.437 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_items": 0, 2024-05-21T22:06:11.438 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_reader_bytes": 0, 2024-05-21T22:06:11.438 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_reader_items": 0, 2024-05-21T22:06:11.438 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_writer_bytes": 0, 2024-05-21T22:06:11.438 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_writer_items": 0, 2024-05-21T22:06:11.438 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_anon_bytes": 200303, 2024-05-21T22:06:11.438 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_anon_items": 58, 2024-05-21T22:06:11.438 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_meta_bytes": 0, 2024-05-21T22:06:11.438 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_meta_items": 0, 2024-05-21T22:06:11.438 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_bytes": 0, 2024-05-21T22:06:11.438 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_items": 0, 2024-05-21T22:06:11.438 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_mapbl_bytes": 0, 2024-05-21T22:06:11.438 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_mapbl_items": 0, 2024-05-21T22:06:11.438 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_pglog_bytes": 0, 2024-05-21T22:06:11.439 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_pglog_items": 0, 2024-05-21T22:06:11.439 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_bytes": 8128, 2024-05-21T22:06:11.439 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_items": 169, 2024-05-21T22:06:11.439 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_mapping_bytes": 0, 2024-05-21T22:06:11.439 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_mapping_items": 0, 2024-05-21T22:06:11.439 INFO:teuthology.orchestra.run.smithi151.stdout: "pgmap_bytes": 0, 2024-05-21T22:06:11.439 INFO:teuthology.orchestra.run.smithi151.stdout: "pgmap_items": 0, 2024-05-21T22:06:11.439 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_co_bytes": 8512, 2024-05-21T22:06:11.439 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_co_items": 13, 2024-05-21T22:06:11.439 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_1_bytes": 0, 2024-05-21T22:06:11.439 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_1_items": 0, 2024-05-21T22:06:11.439 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_2_bytes": 0, 2024-05-21T22:06:11.439 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_2_items": 0 2024-05-21T22:06:11.439 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:11.439 INFO:teuthology.orchestra.run.smithi151.stdout: "objecter": { 2024-05-21T22:06:11.440 INFO:teuthology.orchestra.run.smithi151.stdout: "op_active": 0, 2024-05-21T22:06:11.440 INFO:teuthology.orchestra.run.smithi151.stdout: "op_laggy": 0, 2024-05-21T22:06:11.440 INFO:teuthology.orchestra.run.smithi151.stdout: "op_send": 2381, 2024-05-21T22:06:11.440 INFO:teuthology.orchestra.run.smithi151.stdout: "op_send_bytes": 21, 2024-05-21T22:06:11.440 INFO:teuthology.orchestra.run.smithi151.stdout: "op_resend": 3, 2024-05-21T22:06:11.440 INFO:teuthology.orchestra.run.smithi151.stdout: "op_reply": 2381, 2024-05-21T22:06:11.440 INFO:teuthology.orchestra.run.smithi151.stdout: "op_latency": { 2024-05-21T22:06:11.440 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 2381, 2024-05-21T22:06:11.440 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 2.086974435, 2024-05-21T22:06:11.440 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000876511 2024-05-21T22:06:11.440 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:11.440 INFO:teuthology.orchestra.run.smithi151.stdout: "op_inflight": 0, 2024-05-21T22:06:11.440 INFO:teuthology.orchestra.run.smithi151.stdout: "oplen_avg": { 2024-05-21T22:06:11.440 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 2381, 2024-05-21T22:06:11.441 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 2382 2024-05-21T22:06:11.441 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:11.441 INFO:teuthology.orchestra.run.smithi151.stdout: "op": 2381, 2024-05-21T22:06:11.441 INFO:teuthology.orchestra.run.smithi151.stdout: "op_r": 2381, 2024-05-21T22:06:11.441 INFO:teuthology.orchestra.run.smithi151.stdout: "op_w": 0, 2024-05-21T22:06:11.441 INFO:teuthology.orchestra.run.smithi151.stdout: "op_rmw": 0, 2024-05-21T22:06:11.441 INFO:teuthology.orchestra.run.smithi151.stdout: "op_pg": 0, 2024-05-21T22:06:11.441 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_stat": 1503, 2024-05-21T22:06:11.441 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_create": 0, 2024-05-21T22:06:11.441 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_read": 876, 2024-05-21T22:06:11.441 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_write": 0, 2024-05-21T22:06:11.441 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_writefull": 0, 2024-05-21T22:06:11.441 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_writesame": 0, 2024-05-21T22:06:11.441 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_append": 0, 2024-05-21T22:06:11.441 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_zero": 0, 2024-05-21T22:06:11.442 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_truncate": 0, 2024-05-21T22:06:11.442 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_delete": 0, 2024-05-21T22:06:11.442 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_mapext": 0, 2024-05-21T22:06:11.442 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_sparse_read": 0, 2024-05-21T22:06:11.442 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_clonerange": 0, 2024-05-21T22:06:11.442 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_getxattr": 1, 2024-05-21T22:06:11.442 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_setxattr": 0, 2024-05-21T22:06:11.442 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_cmpxattr": 0, 2024-05-21T22:06:11.442 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_rmxattr": 0, 2024-05-21T22:06:11.442 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_resetxattrs": 0, 2024-05-21T22:06:11.442 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_call": 0, 2024-05-21T22:06:11.442 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_watch": 0, 2024-05-21T22:06:11.442 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_notify": 0, 2024-05-21T22:06:11.442 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_src_cmpxattr": 0, 2024-05-21T22:06:11.443 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_pgls": 0, 2024-05-21T22:06:11.443 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_pgls_filter": 0, 2024-05-21T22:06:11.443 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_other": 0, 2024-05-21T22:06:11.443 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_active": 0, 2024-05-21T22:06:11.443 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_send": 0, 2024-05-21T22:06:11.443 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_resend": 0, 2024-05-21T22:06:11.443 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_ping": 0, 2024-05-21T22:06:11.443 INFO:teuthology.orchestra.run.smithi151.stdout: "poolop_active": 0, 2024-05-21T22:06:11.443 INFO:teuthology.orchestra.run.smithi151.stdout: "poolop_send": 0, 2024-05-21T22:06:11.443 INFO:teuthology.orchestra.run.smithi151.stdout: "poolop_resend": 0, 2024-05-21T22:06:11.443 INFO:teuthology.orchestra.run.smithi151.stdout: "poolstat_active": 0, 2024-05-21T22:06:11.443 INFO:teuthology.orchestra.run.smithi151.stdout: "poolstat_send": 0, 2024-05-21T22:06:11.443 INFO:teuthology.orchestra.run.smithi151.stdout: "poolstat_resend": 0, 2024-05-21T22:06:11.443 INFO:teuthology.orchestra.run.smithi151.stdout: "statfs_active": 0, 2024-05-21T22:06:11.444 INFO:teuthology.orchestra.run.smithi151.stdout: "statfs_send": 0, 2024-05-21T22:06:11.444 INFO:teuthology.orchestra.run.smithi151.stdout: "statfs_resend": 0, 2024-05-21T22:06:11.444 INFO:teuthology.orchestra.run.smithi151.stdout: "command_active": 0, 2024-05-21T22:06:11.444 INFO:teuthology.orchestra.run.smithi151.stdout: "command_send": 0, 2024-05-21T22:06:11.444 INFO:teuthology.orchestra.run.smithi151.stdout: "command_resend": 0, 2024-05-21T22:06:11.444 INFO:teuthology.orchestra.run.smithi151.stdout: "map_epoch": 90, 2024-05-21T22:06:11.444 INFO:teuthology.orchestra.run.smithi151.stdout: "map_full": 0, 2024-05-21T22:06:11.444 INFO:teuthology.orchestra.run.smithi151.stdout: "map_inc": 10, 2024-05-21T22:06:11.444 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_sessions": 6, 2024-05-21T22:06:11.444 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_session_open": 6, 2024-05-21T22:06:11.444 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_session_close": 0, 2024-05-21T22:06:11.444 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_laggy": 0, 2024-05-21T22:06:11.444 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_wr": 0, 2024-05-21T22:06:11.444 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_rd": 2, 2024-05-21T22:06:11.444 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_del": 0 2024-05-21T22:06:11.445 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:11.445 INFO:teuthology.orchestra.run.smithi151.stdout: "oft": { 2024-05-21T22:06:11.445 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_objs": 0, 2024-05-21T22:06:11.445 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_kv_pairs": 0, 2024-05-21T22:06:11.445 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_updates": 0, 2024-05-21T22:06:11.445 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_removes": 0 2024-05-21T22:06:11.445 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:11.445 INFO:teuthology.orchestra.run.smithi151.stdout: "osd-slow-ops": { 2024-05-21T22:06:11.445 INFO:teuthology.orchestra.run.smithi151.stdout: "slow_ops_count": 0 2024-05-21T22:06:11.445 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:11.446 INFO:teuthology.orchestra.run.smithi151.stdout: "purge_queue": { 2024-05-21T22:06:11.446 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing_ops": 0, 2024-05-21T22:06:11.446 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing_ops_high_water": 0, 2024-05-21T22:06:11.446 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing": 0, 2024-05-21T22:06:11.446 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing_high_water": 0, 2024-05-21T22:06:11.446 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executed_ops": 0, 2024-05-21T22:06:11.446 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executed": 0, 2024-05-21T22:06:11.446 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_item_in_journal": 0 2024-05-21T22:06:11.446 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:11.446 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-21T22:06:11.446 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-05-21T22:06:11.446 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 104857600, 2024-05-21T22:06:11.446 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-05-21T22:06:11.447 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 2656, 2024-05-21T22:06:11.447 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 2332729, 2024-05-21T22:06:11.447 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-05-21T22:06:11.447 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 2656, 2024-05-21T22:06:11.447 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 0, 2024-05-21T22:06:11.447 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 0, 2024-05-21T22:06:11.447 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 2656, 2024-05-21T22:06:11.447 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 2332729, 2024-05-21T22:06:11.447 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-05-21T22:06:11.447 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-21T22:06:11.447 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-21T22:06:11.447 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-21T22:06:11.447 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:11.447 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:11.448 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-objecter_bytes": { 2024-05-21T22:06:11.448 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-05-21T22:06:11.448 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 104857600, 2024-05-21T22:06:11.448 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-05-21T22:06:11.448 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 0, 2024-05-21T22:06:11.448 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 0, 2024-05-21T22:06:11.448 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-05-21T22:06:11.448 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 0, 2024-05-21T22:06:11.448 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 2381, 2024-05-21T22:06:11.448 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 1125, 2024-05-21T22:06:11.448 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 2381, 2024-05-21T22:06:11.448 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 1125, 2024-05-21T22:06:11.448 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-05-21T22:06:11.448 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-21T22:06:11.449 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-21T22:06:11.449 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-21T22:06:11.449 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:11.449 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:11.449 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-objecter_ops": { 2024-05-21T22:06:11.449 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-05-21T22:06:11.449 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 1024, 2024-05-21T22:06:11.449 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-05-21T22:06:11.449 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 0, 2024-05-21T22:06:11.449 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 0, 2024-05-21T22:06:11.449 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-05-21T22:06:11.449 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 0, 2024-05-21T22:06:11.450 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 2381, 2024-05-21T22:06:11.450 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 2381, 2024-05-21T22:06:11.450 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 2381, 2024-05-21T22:06:11.450 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 2381, 2024-05-21T22:06:11.450 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-05-21T22:06:11.450 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-21T22:06:11.450 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-21T22:06:11.450 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-21T22:06:11.450 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:11.450 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:11.450 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-write_buf_throttle": { 2024-05-21T22:06:11.450 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-05-21T22:06:11.450 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 3758096384, 2024-05-21T22:06:11.451 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-05-21T22:06:11.451 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 0, 2024-05-21T22:06:11.451 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 0, 2024-05-21T22:06:11.451 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-05-21T22:06:11.451 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 0, 2024-05-21T22:06:11.451 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 0, 2024-05-21T22:06:11.451 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 0, 2024-05-21T22:06:11.451 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 0, 2024-05-21T22:06:11.451 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 0, 2024-05-21T22:06:11.451 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-05-21T22:06:11.451 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-21T22:06:11.451 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-21T22:06:11.451 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-21T22:06:11.451 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:11.452 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:11.452 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-write_buf_throttle-0x564be6f68780": { 2024-05-21T22:06:11.452 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-05-21T22:06:11.452 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 3758096384, 2024-05-21T22:06:11.452 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-05-21T22:06:11.452 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 0, 2024-05-21T22:06:11.452 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 0, 2024-05-21T22:06:11.452 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-05-21T22:06:11.452 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 0, 2024-05-21T22:06:11.452 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 0, 2024-05-21T22:06:11.452 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 0, 2024-05-21T22:06:11.452 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 0, 2024-05-21T22:06:11.452 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 0, 2024-05-21T22:06:11.453 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-05-21T22:06:11.453 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-21T22:06:11.453 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-21T22:06:11.453 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-21T22:06:11.453 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:11.453 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:11.453 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-05-21T22:06:11.681 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:06:11 smithi151 ceph-mon[31623]: pgmap v327: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-05-21T22:06:11.681 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:06:11 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-21T22:06:11.681 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:06:11 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-21T22:06:11.681 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:06:11 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-21T22:06:11.682 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:06:11 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-21T22:06:11.682 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:06:11 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-21T22:06:11.682 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:06:11 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-21T22:06:11.682 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:06:11 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-21T22:06:11.682 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:06:11 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-21T22:06:11.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:06:11 smithi184 ceph-mon[36942]: pgmap v327: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-05-21T22:06:11.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:06:11 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-21T22:06:11.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:06:11 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-21T22:06:11.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:06:11 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-21T22:06:11.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:06:11 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-21T22:06:11.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:06:11 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-21T22:06:11.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:06:11 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-21T22:06:11.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:06:11 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-21T22:06:11.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:06:11 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-21T22:06:12.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:06:11 smithi164 ceph-mon[36269]: pgmap v327: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-05-21T22:06:12.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:06:11 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-21T22:06:12.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:06:11 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-21T22:06:12.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:06:11 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-21T22:06:12.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:06:11 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-21T22:06:12.037 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:06:11 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-21T22:06:12.037 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:06:11 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-21T22:06:12.037 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:06:11 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-21T22:06:12.037 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:06:11 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-21T22:06:12.170 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-05-21T22:06:12.170 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-05-21T22:06:12.170 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=0 2024-05-21T22:06:12.170 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=0 2024-05-21T22:06:12.170 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=0 2024-05-21T22:06:12.170 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-05-21T22:06:12.170 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-05-21T22:06:12.170 DEBUG:tasks.check_counter:Getting stats from d 2024-05-21T22:06:12.171 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph daemon mds.d perf dump 2024-05-21T22:06:12.644 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:06:12.930 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:06:12 smithi151 ceph-mon[31623]: Adjusting osd_memory_target on smithi184 to 176.2M 2024-05-21T22:06:12.930 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:06:12 smithi151 ceph-mon[31623]: Unable to set osd_memory_target on smithi184 to 184791040: error parsing value: Value '184791040' is below minimum 939524096 2024-05-21T22:06:12.930 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:06:12 smithi151 ceph-mon[31623]: Adjusting osd_memory_target on smithi164 to 176.2M 2024-05-21T22:06:12.930 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:06:12 smithi151 ceph-mon[31623]: Unable to set osd_memory_target on smithi164 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-05-21T22:06:12.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:06:12 smithi184 ceph-mon[36942]: Adjusting osd_memory_target on smithi184 to 176.2M 2024-05-21T22:06:12.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:06:12 smithi184 ceph-mon[36942]: Unable to set osd_memory_target on smithi184 to 184791040: error parsing value: Value '184791040' is below minimum 939524096 2024-05-21T22:06:12.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:06:12 smithi184 ceph-mon[36942]: Adjusting osd_memory_target on smithi164 to 176.2M 2024-05-21T22:06:12.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:06:12 smithi184 ceph-mon[36942]: Unable to set osd_memory_target on smithi164 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-05-21T22:06:13.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:06:12 smithi164 ceph-mon[36269]: Adjusting osd_memory_target on smithi184 to 176.2M 2024-05-21T22:06:13.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:06:12 smithi164 ceph-mon[36269]: Unable to set osd_memory_target on smithi184 to 184791040: error parsing value: Value '184791040' is below minimum 939524096 2024-05-21T22:06:13.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:06:12 smithi164 ceph-mon[36269]: Adjusting osd_memory_target on smithi164 to 176.2M 2024-05-21T22:06:13.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:06:12 smithi164 ceph-mon[36269]: Unable to set osd_memory_target on smithi164 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-05-21T22:06:13.896 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:06:13 smithi151 ceph-mon[31623]: pgmap v328: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-21T22:06:13.897 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:06:13 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T22:06:13.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:06:13 smithi184 ceph-mon[36942]: pgmap v328: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-21T22:06:13.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:06:13 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T22:06:14.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:06:13 smithi164 ceph-mon[36269]: pgmap v328: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-21T22:06:14.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:06:13 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T22:06:14.930 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:06:14 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T22:06:14.930 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:06:14 smithi151 ceph-mon[31623]: osdmap e91: 12 total, 12 up, 12 in 2024-05-21T22:06:14.935 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-05-21T22:06:14.935 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-0": { 2024-05-21T22:06:14.935 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 1159, 2024-05-21T22:06:14.935 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 1143, 2024-05-21T22:06:14.935 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 2204720, 2024-05-21T22:06:14.935 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 436268, 2024-05-21T22:06:14.935 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 5, 2024-05-21T22:06:14.935 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 1, 2024-05-21T22:06:14.935 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.303101705, 2024-05-21T22:06:14.935 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.098907592, 2024-05-21T22:06:14.935 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.159152157, 2024-05-21T22:06:14.935 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.060529746, 2024-05-21T22:06:14.935 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T22:06:14.936 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 1143, 2024-05-21T22:06:14.936 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.109233058, 2024-05-21T22:06:14.936 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000095566 2024-05-21T22:06:14.936 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:14.936 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-05-21T22:06:14.936 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-21T22:06:14.936 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-21T22:06:14.936 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-21T22:06:14.936 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:14.936 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 1911056, 2024-05-21T22:06:14.936 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 158336 2024-05-21T22:06:14.936 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:14.936 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-1": { 2024-05-21T22:06:14.936 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 4, 2024-05-21T22:06:14.937 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 194, 2024-05-21T22:06:14.937 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 876, 2024-05-21T22:06:14.937 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 490010, 2024-05-21T22:06:14.937 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 6, 2024-05-21T22:06:14.937 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 4, 2024-05-21T22:06:14.937 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.038658035, 2024-05-21T22:06:14.937 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.025570416, 2024-05-21T22:06:14.937 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.006479231, 2024-05-21T22:06:14.937 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.000083320, 2024-05-21T22:06:14.937 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T22:06:14.937 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 191, 2024-05-21T22:06:14.937 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.030173001, 2024-05-21T22:06:14.937 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000157973 2024-05-21T22:06:14.937 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:14.938 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-05-21T22:06:14.938 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-21T22:06:14.938 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-21T22:06:14.938 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-21T22:06:14.938 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:14.938 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-21T22:06:14.938 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 489312 2024-05-21T22:06:14.938 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:14.938 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-2": { 2024-05-21T22:06:14.938 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 1668, 2024-05-21T22:06:14.938 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 1670, 2024-05-21T22:06:14.938 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 427310, 2024-05-21T22:06:14.938 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 530431, 2024-05-21T22:06:14.939 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 4, 2024-05-21T22:06:14.939 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 3, 2024-05-21T22:06:14.939 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.226446180, 2024-05-21T22:06:14.939 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.070934214, 2024-05-21T22:06:14.939 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.060330730, 2024-05-21T22:06:14.939 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.077946446, 2024-05-21T22:06:14.939 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T22:06:14.939 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 1668, 2024-05-21T22:06:14.939 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.117600086, 2024-05-21T22:06:14.939 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000070503 2024-05-21T22:06:14.939 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:14.939 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-05-21T22:06:14.939 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-21T22:06:14.939 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-21T22:06:14.940 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-21T22:06:14.940 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:14.940 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-21T22:06:14.940 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 0 2024-05-21T22:06:14.940 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:14.940 INFO:teuthology.orchestra.run.smithi151.stdout: "cct": { 2024-05-21T22:06:14.940 INFO:teuthology.orchestra.run.smithi151.stdout: "total_workers": 1, 2024-05-21T22:06:14.940 INFO:teuthology.orchestra.run.smithi151.stdout: "unhealthy_workers": 0 2024-05-21T22:06:14.940 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:14.940 INFO:teuthology.orchestra.run.smithi151.stdout: "finisher-MDSRank": { 2024-05-21T22:06:14.940 INFO:teuthology.orchestra.run.smithi151.stdout: "queue_len": 0, 2024-05-21T22:06:14.940 INFO:teuthology.orchestra.run.smithi151.stdout: "complete_latency": { 2024-05-21T22:06:14.941 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 4233, 2024-05-21T22:06:14.941 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.199499242, 2024-05-21T22:06:14.941 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000047129 2024-05-21T22:06:14.941 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:14.941 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:14.941 INFO:teuthology.orchestra.run.smithi151.stdout: "finisher-PurgeQueue": { 2024-05-21T22:06:14.941 INFO:teuthology.orchestra.run.smithi151.stdout: "queue_len": 0, 2024-05-21T22:06:14.941 INFO:teuthology.orchestra.run.smithi151.stdout: "complete_latency": { 2024-05-21T22:06:14.941 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-21T22:06:14.941 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-21T22:06:14.941 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-21T22:06:14.941 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:14.941 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:14.942 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": { 2024-05-21T22:06:14.942 INFO:teuthology.orchestra.run.smithi151.stdout: "request": 0, 2024-05-21T22:06:14.942 INFO:teuthology.orchestra.run.smithi151.stdout: "reply": 0, 2024-05-21T22:06:14.942 INFO:teuthology.orchestra.run.smithi151.stdout: "reply_latency": { 2024-05-21T22:06:14.942 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-21T22:06:14.942 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-21T22:06:14.942 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-21T22:06:14.942 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:14.942 INFO:teuthology.orchestra.run.smithi151.stdout: "slow_reply": 0, 2024-05-21T22:06:14.942 INFO:teuthology.orchestra.run.smithi151.stdout: "forward": 0, 2024-05-21T22:06:14.942 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_fetch_complete": 0, 2024-05-21T22:06:14.942 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_fetch_keys": 0, 2024-05-21T22:06:14.943 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_commit": 0, 2024-05-21T22:06:14.943 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_split": 0, 2024-05-21T22:06:14.943 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_merge": 0, 2024-05-21T22:06:14.943 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes": 0, 2024-05-21T22:06:14.943 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_top": 0, 2024-05-21T22:06:14.943 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_bottom": 0, 2024-05-21T22:06:14.943 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_pin_tail": 0, 2024-05-21T22:06:14.943 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_pinned": 0, 2024-05-21T22:06:14.943 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_expired": 0, 2024-05-21T22:06:14.943 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_with_caps": 0, 2024-05-21T22:06:14.943 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": 0, 2024-05-21T22:06:14.943 INFO:teuthology.orchestra.run.smithi151.stdout: "subtrees": 1, 2024-05-21T22:06:14.944 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse": 0, 2024-05-21T22:06:14.944 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_hit": 0, 2024-05-21T22:06:14.944 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_forward": 0, 2024-05-21T22:06:14.944 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_discover": 0, 2024-05-21T22:06:14.944 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_dir_fetch": 0, 2024-05-21T22:06:14.944 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_remote_ino": 0, 2024-05-21T22:06:14.944 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_lock": 0, 2024-05-21T22:06:14.944 INFO:teuthology.orchestra.run.smithi151.stdout: "load_cent": 0, 2024-05-21T22:06:14.944 INFO:teuthology.orchestra.run.smithi151.stdout: "q": 0, 2024-05-21T22:06:14.944 INFO:teuthology.orchestra.run.smithi151.stdout: "exported": 0, 2024-05-21T22:06:14.944 INFO:teuthology.orchestra.run.smithi151.stdout: "exported_inodes": 0, 2024-05-21T22:06:14.944 INFO:teuthology.orchestra.run.smithi151.stdout: "imported": 0, 2024-05-21T22:06:14.944 INFO:teuthology.orchestra.run.smithi151.stdout: "imported_inodes": 0, 2024-05-21T22:06:14.944 INFO:teuthology.orchestra.run.smithi151.stdout: "openino_dir_fetch": 0, 2024-05-21T22:06:14.945 INFO:teuthology.orchestra.run.smithi151.stdout: "openino_backtrace_fetch": 0, 2024-05-21T22:06:14.945 INFO:teuthology.orchestra.run.smithi151.stdout: "openino_peer_discover": 0, 2024-05-21T22:06:14.945 INFO:teuthology.orchestra.run.smithi151.stdout: "root_rfiles": 0, 2024-05-21T22:06:14.945 INFO:teuthology.orchestra.run.smithi151.stdout: "root_rbytes": 0, 2024-05-21T22:06:14.945 INFO:teuthology.orchestra.run.smithi151.stdout: "root_rsnaps": 0, 2024-05-21T22:06:14.945 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_backtrace_fetch": 0, 2024-05-21T22:06:14.945 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_set_tag": 0, 2024-05-21T22:06:14.945 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_backtrace_repaired": 0, 2024-05-21T22:06:14.945 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_inotable_repaired": 0, 2024-05-21T22:06:14.945 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_dir_inodes": 0, 2024-05-21T22:06:14.945 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_dir_base_inodes": 0, 2024-05-21T22:06:14.945 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_dirfrag_rstats": 0, 2024-05-21T22:06:14.945 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_file_inodes": 0, 2024-05-21T22:06:14.946 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_inode_file_caps": 0, 2024-05-21T22:06:14.946 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_revoke": 0, 2024-05-21T22:06:14.946 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_grant": 0, 2024-05-21T22:06:14.946 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_trunc": 0, 2024-05-21T22:06:14.946 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-21T22:06:14.946 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-21T22:06:14.946 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_caps": 0, 2024-05-21T22:06:14.946 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_caps_dirty": 0, 2024-05-21T22:06:14.946 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_cap_release": 0, 2024-05-21T22:06:14.946 INFO:teuthology.orchestra.run.smithi151.stdout: "process_request_cap_release": 0 2024-05-21T22:06:14.946 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:14.946 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_cache": { 2024-05-21T22:06:14.946 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_update": 0, 2024-05-21T22:06:14.946 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_update_receipt": 0, 2024-05-21T22:06:14.947 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_try_discover": 0, 2024-05-21T22:06:14.947 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_send_discover": 0, 2024-05-21T22:06:14.947 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_handle_discover": 0, 2024-05-21T22:06:14.947 INFO:teuthology.orchestra.run.smithi151.stdout: "num_strays": 0, 2024-05-21T22:06:14.947 INFO:teuthology.orchestra.run.smithi151.stdout: "num_strays_delayed": 0, 2024-05-21T22:06:14.947 INFO:teuthology.orchestra.run.smithi151.stdout: "num_strays_enqueuing": 0, 2024-05-21T22:06:14.947 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_created": 0, 2024-05-21T22:06:14.947 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_enqueued": 0, 2024-05-21T22:06:14.947 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_reintegrated": 0, 2024-05-21T22:06:14.947 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_migrated": 0, 2024-05-21T22:06:14.947 INFO:teuthology.orchestra.run.smithi151.stdout: "num_recovering_processing": 0, 2024-05-21T22:06:14.947 INFO:teuthology.orchestra.run.smithi151.stdout: "num_recovering_enqueued": 0, 2024-05-21T22:06:14.947 INFO:teuthology.orchestra.run.smithi151.stdout: "num_recovering_prioritized": 0, 2024-05-21T22:06:14.948 INFO:teuthology.orchestra.run.smithi151.stdout: "recovery_started": 0, 2024-05-21T22:06:14.948 INFO:teuthology.orchestra.run.smithi151.stdout: "recovery_completed": 0, 2024-05-21T22:06:14.948 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_quiesce_path": 0, 2024-05-21T22:06:14.948 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_quiesce_inode": 0, 2024-05-21T22:06:14.948 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_enqueue_scrub": 0, 2024-05-21T22:06:14.948 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_exportdir": 0, 2024-05-21T22:06:14.948 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_flush": 0, 2024-05-21T22:06:14.948 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_fragmentdir": 0, 2024-05-21T22:06:14.948 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_fragstats": 0, 2024-05-21T22:06:14.948 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_inodestats": 0 2024-05-21T22:06:14.948 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:14.948 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_log": { 2024-05-21T22:06:14.948 INFO:teuthology.orchestra.run.smithi151.stdout: "evlrg": 0, 2024-05-21T22:06:14.948 INFO:teuthology.orchestra.run.smithi151.stdout: "evadd": 0, 2024-05-21T22:06:14.949 INFO:teuthology.orchestra.run.smithi151.stdout: "evex": 0, 2024-05-21T22:06:14.949 INFO:teuthology.orchestra.run.smithi151.stdout: "evtrm": 0, 2024-05-21T22:06:14.949 INFO:teuthology.orchestra.run.smithi151.stdout: "ev": 8, 2024-05-21T22:06:14.949 INFO:teuthology.orchestra.run.smithi151.stdout: "evexg": 0, 2024-05-21T22:06:14.949 INFO:teuthology.orchestra.run.smithi151.stdout: "evexd": 0, 2024-05-21T22:06:14.949 INFO:teuthology.orchestra.run.smithi151.stdout: "segadd": 0, 2024-05-21T22:06:14.949 INFO:teuthology.orchestra.run.smithi151.stdout: "segex": 0, 2024-05-21T22:06:14.949 INFO:teuthology.orchestra.run.smithi151.stdout: "segtrm": 0, 2024-05-21T22:06:14.949 INFO:teuthology.orchestra.run.smithi151.stdout: "seg": 2, 2024-05-21T22:06:14.949 INFO:teuthology.orchestra.run.smithi151.stdout: "segmjr": 2, 2024-05-21T22:06:14.949 INFO:teuthology.orchestra.run.smithi151.stdout: "segexg": 0, 2024-05-21T22:06:14.949 INFO:teuthology.orchestra.run.smithi151.stdout: "segexd": 0, 2024-05-21T22:06:14.949 INFO:teuthology.orchestra.run.smithi151.stdout: "expos": 4194304, 2024-05-21T22:06:14.949 INFO:teuthology.orchestra.run.smithi151.stdout: "wrpos": 4196657, 2024-05-21T22:06:14.950 INFO:teuthology.orchestra.run.smithi151.stdout: "rdpos": 4196516, 2024-05-21T22:06:14.950 INFO:teuthology.orchestra.run.smithi151.stdout: "jlat": { 2024-05-21T22:06:14.950 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-21T22:06:14.950 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-21T22:06:14.950 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-21T22:06:14.950 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:14.950 INFO:teuthology.orchestra.run.smithi151.stdout: "replayed": 8 2024-05-21T22:06:14.950 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:14.950 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_mem": { 2024-05-21T22:06:14.950 INFO:teuthology.orchestra.run.smithi151.stdout: "ino": 3, 2024-05-21T22:06:14.950 INFO:teuthology.orchestra.run.smithi151.stdout: "ino+": 3, 2024-05-21T22:06:14.950 INFO:teuthology.orchestra.run.smithi151.stdout: "ino-": 0, 2024-05-21T22:06:14.950 INFO:teuthology.orchestra.run.smithi151.stdout: "dir": 1, 2024-05-21T22:06:14.951 INFO:teuthology.orchestra.run.smithi151.stdout: "dir+": 1, 2024-05-21T22:06:14.951 INFO:teuthology.orchestra.run.smithi151.stdout: "dir-": 0, 2024-05-21T22:06:14.951 INFO:teuthology.orchestra.run.smithi151.stdout: "dn": 0, 2024-05-21T22:06:14.951 INFO:teuthology.orchestra.run.smithi151.stdout: "dn+": 0, 2024-05-21T22:06:14.951 INFO:teuthology.orchestra.run.smithi151.stdout: "dn-": 0, 2024-05-21T22:06:14.951 INFO:teuthology.orchestra.run.smithi151.stdout: "cap": 0, 2024-05-21T22:06:14.951 INFO:teuthology.orchestra.run.smithi151.stdout: "cap+": 0, 2024-05-21T22:06:14.951 INFO:teuthology.orchestra.run.smithi151.stdout: "cap-": 0, 2024-05-21T22:06:14.951 INFO:teuthology.orchestra.run.smithi151.stdout: "rss": 49376, 2024-05-21T22:06:14.951 INFO:teuthology.orchestra.run.smithi151.stdout: "heap": 215416 2024-05-21T22:06:14.951 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:14.951 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_server": { 2024-05-21T22:06:14.951 INFO:teuthology.orchestra.run.smithi151.stdout: "dispatch_client_request": 0, 2024-05-21T22:06:14.951 INFO:teuthology.orchestra.run.smithi151.stdout: "dispatch_server_request": 0, 2024-05-21T22:06:14.952 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_request": 0, 2024-05-21T22:06:14.952 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_session": 0, 2024-05-21T22:06:14.952 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_peer_request": 0, 2024-05-21T22:06:14.952 INFO:teuthology.orchestra.run.smithi151.stdout: "req_create_latency": { 2024-05-21T22:06:14.952 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-21T22:06:14.952 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-21T22:06:14.952 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-21T22:06:14.952 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:14.952 INFO:teuthology.orchestra.run.smithi151.stdout: "req_getattr_latency": { 2024-05-21T22:06:14.952 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-21T22:06:14.952 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-21T22:06:14.952 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-21T22:06:14.952 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:14.952 INFO:teuthology.orchestra.run.smithi151.stdout: "req_getfilelock_latency": { 2024-05-21T22:06:14.953 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-21T22:06:14.953 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-21T22:06:14.953 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-21T22:06:14.953 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:14.953 INFO:teuthology.orchestra.run.smithi151.stdout: "req_link_latency": { 2024-05-21T22:06:14.953 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-21T22:06:14.953 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-21T22:06:14.953 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-21T22:06:14.953 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:14.953 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookup_latency": { 2024-05-21T22:06:14.954 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-21T22:06:14.954 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-21T22:06:14.954 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-21T22:06:14.954 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:14.954 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookuphash_latency": { 2024-05-21T22:06:14.954 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-21T22:06:14.954 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-21T22:06:14.954 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-21T22:06:14.954 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:14.954 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupino_latency": { 2024-05-21T22:06:14.954 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-21T22:06:14.954 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-21T22:06:14.955 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-21T22:06:14.955 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:14.955 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupname_latency": { 2024-05-21T22:06:14.955 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-21T22:06:14.955 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-21T22:06:14.955 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-21T22:06:14.955 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:14.955 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupparent_latency": { 2024-05-21T22:06:14.955 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-21T22:06:14.955 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-21T22:06:14.955 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-21T22:06:14.955 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:14.956 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupsnap_latency": { 2024-05-21T22:06:14.956 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-21T22:06:14.956 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-21T22:06:14.956 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-21T22:06:14.956 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:14.956 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lssnap_latency": { 2024-05-21T22:06:14.956 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-21T22:06:14.956 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-21T22:06:14.956 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-21T22:06:14.956 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:14.956 INFO:teuthology.orchestra.run.smithi151.stdout: "req_mkdir_latency": { 2024-05-21T22:06:14.957 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-21T22:06:14.957 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-21T22:06:14.957 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-21T22:06:14.957 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:14.958 INFO:teuthology.orchestra.run.smithi151.stdout: "req_mknod_latency": { 2024-05-21T22:06:14.958 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-21T22:06:14.958 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-21T22:06:14.958 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-21T22:06:14.958 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:14.958 INFO:teuthology.orchestra.run.smithi151.stdout: "req_mksnap_latency": { 2024-05-21T22:06:14.958 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-21T22:06:14.958 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-21T22:06:14.958 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-21T22:06:14.958 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:14.958 INFO:teuthology.orchestra.run.smithi151.stdout: "req_open_latency": { 2024-05-21T22:06:14.958 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-21T22:06:14.959 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-21T22:06:14.959 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-21T22:06:14.959 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:14.959 INFO:teuthology.orchestra.run.smithi151.stdout: "req_readdir_latency": { 2024-05-21T22:06:14.959 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-21T22:06:14.959 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-21T22:06:14.959 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-21T22:06:14.959 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:14.959 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rename_latency": { 2024-05-21T22:06:14.959 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-21T22:06:14.959 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-21T22:06:14.959 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-21T22:06:14.960 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:14.960 INFO:teuthology.orchestra.run.smithi151.stdout: "req_renamesnap_latency": { 2024-05-21T22:06:14.960 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-21T22:06:14.960 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-21T22:06:14.960 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-21T22:06:14.960 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:14.960 INFO:teuthology.orchestra.run.smithi151.stdout: "req_snapdiff_latency": { 2024-05-21T22:06:14.960 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-21T22:06:14.960 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-21T22:06:14.960 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-21T22:06:14.960 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:14.960 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rmdir_latency": { 2024-05-21T22:06:14.960 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-21T22:06:14.960 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-21T22:06:14.961 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-21T22:06:14.961 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:14.961 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rmsnap_latency": { 2024-05-21T22:06:14.961 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-21T22:06:14.961 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-21T22:06:14.961 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-21T22:06:14.961 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:14.961 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rmxattr_latency": { 2024-05-21T22:06:14.961 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-21T22:06:14.961 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-21T22:06:14.961 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-21T22:06:14.961 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:14.961 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setattr_latency": { 2024-05-21T22:06:14.961 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-21T22:06:14.962 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-21T22:06:14.962 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-21T22:06:14.962 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:14.962 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setdirlayout_latency": { 2024-05-21T22:06:14.962 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-21T22:06:14.962 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-21T22:06:14.962 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-21T22:06:14.962 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:14.962 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setfilelock_latency": { 2024-05-21T22:06:14.962 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-21T22:06:14.962 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-21T22:06:14.962 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-21T22:06:14.962 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:14.962 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setlayout_latency": { 2024-05-21T22:06:14.963 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-21T22:06:14.963 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-21T22:06:14.963 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-21T22:06:14.963 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:14.963 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setxattr_latency": { 2024-05-21T22:06:14.963 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-21T22:06:14.963 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-21T22:06:14.963 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-21T22:06:14.963 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:14.963 INFO:teuthology.orchestra.run.smithi151.stdout: "req_symlink_latency": { 2024-05-21T22:06:14.963 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-21T22:06:14.963 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-21T22:06:14.963 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-21T22:06:14.963 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:14.963 INFO:teuthology.orchestra.run.smithi151.stdout: "req_unlink_latency": { 2024-05-21T22:06:14.964 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-21T22:06:14.964 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-21T22:06:14.964 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-21T22:06:14.964 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:14.964 INFO:teuthology.orchestra.run.smithi151.stdout: "cap_revoke_eviction": 0, 2024-05-21T22:06:14.964 INFO:teuthology.orchestra.run.smithi151.stdout: "cap_acquisition_throttle": 0, 2024-05-21T22:06:14.964 INFO:teuthology.orchestra.run.smithi151.stdout: "req_getvxattr_latency": { 2024-05-21T22:06:14.964 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-21T22:06:14.964 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-21T22:06:14.964 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-21T22:06:14.964 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:14.964 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:14.964 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_sessions": { 2024-05-21T22:06:14.964 INFO:teuthology.orchestra.run.smithi151.stdout: "session_count": 2, 2024-05-21T22:06:14.964 INFO:teuthology.orchestra.run.smithi151.stdout: "session_add": 4, 2024-05-21T22:06:14.965 INFO:teuthology.orchestra.run.smithi151.stdout: "session_remove": 2, 2024-05-21T22:06:14.965 INFO:teuthology.orchestra.run.smithi151.stdout: "sessions_open": 3, 2024-05-21T22:06:14.965 INFO:teuthology.orchestra.run.smithi151.stdout: "sessions_stale": 0, 2024-05-21T22:06:14.965 INFO:teuthology.orchestra.run.smithi151.stdout: "total_load": 0, 2024-05-21T22:06:14.965 INFO:teuthology.orchestra.run.smithi151.stdout: "average_load": 0, 2024-05-21T22:06:14.965 INFO:teuthology.orchestra.run.smithi151.stdout: "avg_session_uptime": 2831, 2024-05-21T22:06:14.965 INFO:teuthology.orchestra.run.smithi151.stdout: "mdthresh_evicted": 0 2024-05-21T22:06:14.965 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:14.965 INFO:teuthology.orchestra.run.smithi151.stdout: "mempool": { 2024-05-21T22:06:14.965 INFO:teuthology.orchestra.run.smithi151.stdout: "bloom_filter_bytes": 0, 2024-05-21T22:06:14.965 INFO:teuthology.orchestra.run.smithi151.stdout: "bloom_filter_items": 0, 2024-05-21T22:06:14.965 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_alloc_bytes": 0, 2024-05-21T22:06:14.965 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_alloc_items": 0, 2024-05-21T22:06:14.965 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_data_bytes": 0, 2024-05-21T22:06:14.966 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_data_items": 0, 2024-05-21T22:06:14.966 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-21T22:06:14.966 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_onode_items": 0, 2024-05-21T22:06:14.966 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-21T22:06:14.966 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_meta_items": 0, 2024-05-21T22:06:14.966 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_other_bytes": 0, 2024-05-21T22:06:14.966 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_other_items": 0, 2024-05-21T22:06:14.966 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-21T22:06:14.966 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_buffer_items": 0, 2024-05-21T22:06:14.966 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_extent_bytes": 0, 2024-05-21T22:06:14.966 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_extent_items": 0, 2024-05-21T22:06:14.966 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_blob_bytes": 0, 2024-05-21T22:06:14.966 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_blob_items": 0, 2024-05-21T22:06:14.966 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-21T22:06:14.966 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_shared_blob_items": 0, 2024-05-21T22:06:14.967 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-21T22:06:14.967 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_inline_bl_items": 0, 2024-05-21T22:06:14.967 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_fsck_bytes": 0, 2024-05-21T22:06:14.967 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_fsck_items": 0, 2024-05-21T22:06:14.967 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_txc_bytes": 0, 2024-05-21T22:06:14.967 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_txc_items": 0, 2024-05-21T22:06:14.967 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-21T22:06:14.967 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_deferred_items": 0, 2024-05-21T22:06:14.967 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_bytes": 0, 2024-05-21T22:06:14.967 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_items": 0, 2024-05-21T22:06:14.967 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_bytes": 0, 2024-05-21T22:06:14.967 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_items": 0, 2024-05-21T22:06:14.967 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_reader_bytes": 0, 2024-05-21T22:06:14.967 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_reader_items": 0, 2024-05-21T22:06:14.967 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_writer_bytes": 0, 2024-05-21T22:06:14.967 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_writer_items": 0, 2024-05-21T22:06:14.968 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_anon_bytes": 183112, 2024-05-21T22:06:14.968 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_anon_items": 49, 2024-05-21T22:06:14.968 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_meta_bytes": 0, 2024-05-21T22:06:14.968 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_meta_items": 0, 2024-05-21T22:06:14.968 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_bytes": 0, 2024-05-21T22:06:14.968 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_items": 0, 2024-05-21T22:06:14.968 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_mapbl_bytes": 0, 2024-05-21T22:06:14.968 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_mapbl_items": 0, 2024-05-21T22:06:14.968 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_pglog_bytes": 0, 2024-05-21T22:06:14.968 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_pglog_items": 0, 2024-05-21T22:06:14.968 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_bytes": 8248, 2024-05-21T22:06:14.968 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_items": 172, 2024-05-21T22:06:14.968 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_mapping_bytes": 0, 2024-05-21T22:06:14.968 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_mapping_items": 0, 2024-05-21T22:06:14.969 INFO:teuthology.orchestra.run.smithi151.stdout: "pgmap_bytes": 0, 2024-05-21T22:06:14.969 INFO:teuthology.orchestra.run.smithi151.stdout: "pgmap_items": 0, 2024-05-21T22:06:14.969 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_co_bytes": 8512, 2024-05-21T22:06:14.969 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_co_items": 13, 2024-05-21T22:06:14.969 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_1_bytes": 0, 2024-05-21T22:06:14.969 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_1_items": 0, 2024-05-21T22:06:14.969 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_2_bytes": 0, 2024-05-21T22:06:14.969 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_2_items": 0 2024-05-21T22:06:14.969 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:14.969 INFO:teuthology.orchestra.run.smithi151.stdout: "objecter": { 2024-05-21T22:06:14.969 INFO:teuthology.orchestra.run.smithi151.stdout: "op_active": 0, 2024-05-21T22:06:14.969 INFO:teuthology.orchestra.run.smithi151.stdout: "op_laggy": 0, 2024-05-21T22:06:14.969 INFO:teuthology.orchestra.run.smithi151.stdout: "op_send": 2544, 2024-05-21T22:06:14.969 INFO:teuthology.orchestra.run.smithi151.stdout: "op_send_bytes": 21, 2024-05-21T22:06:14.969 INFO:teuthology.orchestra.run.smithi151.stdout: "op_resend": 3, 2024-05-21T22:06:14.970 INFO:teuthology.orchestra.run.smithi151.stdout: "op_reply": 2544, 2024-05-21T22:06:14.970 INFO:teuthology.orchestra.run.smithi151.stdout: "op_latency": { 2024-05-21T22:06:14.970 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 2544, 2024-05-21T22:06:14.970 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 2.154973632, 2024-05-21T22:06:14.970 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000847080 2024-05-21T22:06:14.970 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:14.970 INFO:teuthology.orchestra.run.smithi151.stdout: "op_inflight": 0, 2024-05-21T22:06:14.970 INFO:teuthology.orchestra.run.smithi151.stdout: "oplen_avg": { 2024-05-21T22:06:14.970 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 2544, 2024-05-21T22:06:14.970 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 2545 2024-05-21T22:06:14.970 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:14.970 INFO:teuthology.orchestra.run.smithi151.stdout: "op": 2544, 2024-05-21T22:06:14.970 INFO:teuthology.orchestra.run.smithi151.stdout: "op_r": 2544, 2024-05-21T22:06:14.970 INFO:teuthology.orchestra.run.smithi151.stdout: "op_w": 0, 2024-05-21T22:06:14.971 INFO:teuthology.orchestra.run.smithi151.stdout: "op_rmw": 0, 2024-05-21T22:06:14.971 INFO:teuthology.orchestra.run.smithi151.stdout: "op_pg": 0, 2024-05-21T22:06:14.971 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_stat": 1658, 2024-05-21T22:06:14.971 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_create": 0, 2024-05-21T22:06:14.971 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_read": 884, 2024-05-21T22:06:14.971 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_write": 0, 2024-05-21T22:06:14.971 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_writefull": 0, 2024-05-21T22:06:14.971 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_writesame": 0, 2024-05-21T22:06:14.971 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_append": 0, 2024-05-21T22:06:14.971 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_zero": 0, 2024-05-21T22:06:14.971 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_truncate": 0, 2024-05-21T22:06:14.971 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_delete": 0, 2024-05-21T22:06:14.971 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_mapext": 0, 2024-05-21T22:06:14.971 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_sparse_read": 0, 2024-05-21T22:06:14.972 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_clonerange": 0, 2024-05-21T22:06:14.972 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_getxattr": 1, 2024-05-21T22:06:14.972 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_setxattr": 0, 2024-05-21T22:06:14.972 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_cmpxattr": 0, 2024-05-21T22:06:14.972 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_rmxattr": 0, 2024-05-21T22:06:14.972 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_resetxattrs": 0, 2024-05-21T22:06:14.972 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_call": 0, 2024-05-21T22:06:14.972 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_watch": 0, 2024-05-21T22:06:14.972 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_notify": 0, 2024-05-21T22:06:14.972 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_src_cmpxattr": 0, 2024-05-21T22:06:14.972 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_pgls": 0, 2024-05-21T22:06:14.972 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_pgls_filter": 0, 2024-05-21T22:06:14.972 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_other": 0, 2024-05-21T22:06:14.972 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_active": 0, 2024-05-21T22:06:14.973 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_send": 0, 2024-05-21T22:06:14.973 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_resend": 0, 2024-05-21T22:06:14.973 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_ping": 0, 2024-05-21T22:06:14.973 INFO:teuthology.orchestra.run.smithi151.stdout: "poolop_active": 0, 2024-05-21T22:06:14.973 INFO:teuthology.orchestra.run.smithi151.stdout: "poolop_send": 0, 2024-05-21T22:06:14.973 INFO:teuthology.orchestra.run.smithi151.stdout: "poolop_resend": 0, 2024-05-21T22:06:14.973 INFO:teuthology.orchestra.run.smithi151.stdout: "poolstat_active": 0, 2024-05-21T22:06:14.973 INFO:teuthology.orchestra.run.smithi151.stdout: "poolstat_send": 0, 2024-05-21T22:06:14.973 INFO:teuthology.orchestra.run.smithi151.stdout: "poolstat_resend": 0, 2024-05-21T22:06:14.973 INFO:teuthology.orchestra.run.smithi151.stdout: "statfs_active": 0, 2024-05-21T22:06:14.973 INFO:teuthology.orchestra.run.smithi151.stdout: "statfs_send": 0, 2024-05-21T22:06:14.973 INFO:teuthology.orchestra.run.smithi151.stdout: "statfs_resend": 0, 2024-05-21T22:06:14.973 INFO:teuthology.orchestra.run.smithi151.stdout: "command_active": 0, 2024-05-21T22:06:14.973 INFO:teuthology.orchestra.run.smithi151.stdout: "command_send": 0, 2024-05-21T22:06:14.973 INFO:teuthology.orchestra.run.smithi151.stdout: "command_resend": 0, 2024-05-21T22:06:14.974 INFO:teuthology.orchestra.run.smithi151.stdout: "map_epoch": 91, 2024-05-21T22:06:14.974 INFO:teuthology.orchestra.run.smithi151.stdout: "map_full": 0, 2024-05-21T22:06:14.974 INFO:teuthology.orchestra.run.smithi151.stdout: "map_inc": 11, 2024-05-21T22:06:14.974 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_sessions": 4, 2024-05-21T22:06:14.974 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_session_open": 4, 2024-05-21T22:06:14.974 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_session_close": 0, 2024-05-21T22:06:14.974 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_laggy": 0, 2024-05-21T22:06:14.974 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_wr": 0, 2024-05-21T22:06:14.974 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_rd": 2, 2024-05-21T22:06:14.974 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_del": 0 2024-05-21T22:06:14.974 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:14.974 INFO:teuthology.orchestra.run.smithi151.stdout: "oft": { 2024-05-21T22:06:14.974 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_objs": 0, 2024-05-21T22:06:14.975 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_kv_pairs": 0, 2024-05-21T22:06:14.975 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_updates": 0, 2024-05-21T22:06:14.975 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_removes": 0 2024-05-21T22:06:14.975 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:14.975 INFO:teuthology.orchestra.run.smithi151.stdout: "osd-slow-ops": { 2024-05-21T22:06:14.975 INFO:teuthology.orchestra.run.smithi151.stdout: "slow_ops_count": 0 2024-05-21T22:06:14.975 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:14.975 INFO:teuthology.orchestra.run.smithi151.stdout: "purge_queue": { 2024-05-21T22:06:14.975 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing_ops": 0, 2024-05-21T22:06:14.975 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing_ops_high_water": 0, 2024-05-21T22:06:14.975 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing": 0, 2024-05-21T22:06:14.975 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing_high_water": 0, 2024-05-21T22:06:14.975 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executed_ops": 0, 2024-05-21T22:06:14.976 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executed": 0, 2024-05-21T22:06:14.976 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_item_in_journal": 0 2024-05-21T22:06:14.976 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:14.976 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-21T22:06:14.976 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-05-21T22:06:14.976 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 104857600, 2024-05-21T22:06:14.976 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-05-21T22:06:14.976 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 2827, 2024-05-21T22:06:14.976 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 2361056, 2024-05-21T22:06:14.976 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-05-21T22:06:14.976 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 2827, 2024-05-21T22:06:14.976 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 0, 2024-05-21T22:06:14.976 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 0, 2024-05-21T22:06:14.977 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 2827, 2024-05-21T22:06:14.977 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 2361056, 2024-05-21T22:06:14.977 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-05-21T22:06:14.977 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-21T22:06:14.977 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-21T22:06:14.977 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-21T22:06:14.977 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:14.977 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:14.977 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-objecter_bytes": { 2024-05-21T22:06:14.977 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-05-21T22:06:14.977 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 104857600, 2024-05-21T22:06:14.977 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-05-21T22:06:14.977 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 0, 2024-05-21T22:06:14.978 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 0, 2024-05-21T22:06:14.978 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-05-21T22:06:14.978 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 0, 2024-05-21T22:06:14.978 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 2544, 2024-05-21T22:06:14.978 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 2358, 2024-05-21T22:06:14.978 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 2544, 2024-05-21T22:06:14.978 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 2358, 2024-05-21T22:06:14.978 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-05-21T22:06:14.978 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-21T22:06:14.978 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-21T22:06:14.978 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-21T22:06:14.978 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:14.979 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:14.979 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-objecter_ops": { 2024-05-21T22:06:14.979 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-05-21T22:06:14.979 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 1024, 2024-05-21T22:06:14.979 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-05-21T22:06:14.979 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 0, 2024-05-21T22:06:14.979 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 0, 2024-05-21T22:06:14.979 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-05-21T22:06:14.979 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 0, 2024-05-21T22:06:14.979 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 2544, 2024-05-21T22:06:14.979 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 2544, 2024-05-21T22:06:14.979 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 2544, 2024-05-21T22:06:14.979 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 2544, 2024-05-21T22:06:14.979 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-05-21T22:06:14.980 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-21T22:06:14.980 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-21T22:06:14.980 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-21T22:06:14.980 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:14.980 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:14.980 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-write_buf_throttle": { 2024-05-21T22:06:14.980 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-05-21T22:06:14.980 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 3758096384, 2024-05-21T22:06:14.980 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-05-21T22:06:14.980 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 0, 2024-05-21T22:06:14.980 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 0, 2024-05-21T22:06:14.980 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-05-21T22:06:14.980 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 0, 2024-05-21T22:06:14.981 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 0, 2024-05-21T22:06:14.981 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 0, 2024-05-21T22:06:14.981 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 0, 2024-05-21T22:06:14.981 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 0, 2024-05-21T22:06:14.981 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-05-21T22:06:14.981 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-21T22:06:14.981 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-21T22:06:14.981 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-21T22:06:14.981 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:14.981 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:14.981 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-write_buf_throttle-0x562148550780": { 2024-05-21T22:06:14.981 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-05-21T22:06:14.981 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 3758096384, 2024-05-21T22:06:14.981 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-05-21T22:06:14.982 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 0, 2024-05-21T22:06:14.982 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 0, 2024-05-21T22:06:14.982 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-05-21T22:06:14.982 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 0, 2024-05-21T22:06:14.982 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 0, 2024-05-21T22:06:14.982 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 0, 2024-05-21T22:06:14.982 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 0, 2024-05-21T22:06:14.982 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 0, 2024-05-21T22:06:14.982 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-05-21T22:06:14.982 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-21T22:06:14.982 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-21T22:06:14.982 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-21T22:06:14.982 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:14.983 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:14.983 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-05-21T22:06:14.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:06:14 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T22:06:14.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:06:14 smithi184 ceph-mon[36942]: osdmap e91: 12 total, 12 up, 12 in 2024-05-21T22:06:15.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:06:14 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T22:06:15.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:06:14 smithi164 ceph-mon[36269]: osdmap e91: 12 total, 12 up, 12 in 2024-05-21T22:06:15.828 INFO:tasks.check_counter:Daemon mds.d mds.exported=0 2024-05-21T22:06:15.828 INFO:tasks.check_counter:Daemon mds.d mds.imported=0 2024-05-21T22:06:15.828 INFO:tasks.check_counter:Daemon mds.d mds_cache.dir_update=0 2024-05-21T22:06:15.828 INFO:tasks.check_counter:Daemon mds.d mds_cache.dir_update_receipt=0 2024-05-21T22:06:15.828 INFO:tasks.check_counter:Daemon mds.d mds.root_rsnaps=0 2024-05-21T22:06:15.828 INFO:tasks.check_counter:Daemon mds.d mds_server.req_mksnap_latency.avgcount=0 2024-05-21T22:06:15.828 INFO:tasks.check_counter:Daemon mds.d mds_server.req_rmsnap_latency.avgcount=0 2024-05-21T22:06:15.828 DEBUG:tasks.check_counter:Getting stats from g 2024-05-21T22:06:15.829 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph daemon mds.g perf dump 2024-05-21T22:06:15.906 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:06:15 smithi151 ceph-mon[31623]: pgmap v330: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-05-21T22:06:15.906 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:06:15 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-21T22:06:15.906 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:06:15 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-21T22:06:15.906 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:06:15 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-21T22:06:15.907 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:06:15 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-21T22:06:15.907 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:06:15 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T22:06:15.907 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:06:15 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T22:06:15.907 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:06:15 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T22:06:15.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:06:15 smithi184 ceph-mon[36942]: pgmap v330: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-05-21T22:06:15.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:06:15 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-21T22:06:15.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:06:15 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-21T22:06:15.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:06:15 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-21T22:06:15.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:06:15 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-21T22:06:15.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:06:15 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T22:06:15.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:06:15 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T22:06:15.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:06:15 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T22:06:16.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:06:15 smithi164 ceph-mon[36269]: pgmap v330: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-05-21T22:06:16.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:06:15 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-21T22:06:16.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:06:15 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-21T22:06:16.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:06:15 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-21T22:06:16.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:06:15 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-21T22:06:16.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:06:15 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T22:06:16.037 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:06:15 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T22:06:16.037 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:06:15 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T22:06:16.287 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:06:16.691 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:06:16 smithi151 ceph-mon[31623]: Adjusting osd_memory_target on smithi151 to 176.4M 2024-05-21T22:06:16.692 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:06:16 smithi151 ceph-mon[31623]: Unable to set osd_memory_target on smithi151 to 185064857: error parsing value: Value '185064857' is below minimum 939524096 2024-05-21T22:06:16.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:06:16 smithi184 ceph-mon[36942]: Adjusting osd_memory_target on smithi151 to 176.4M 2024-05-21T22:06:16.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:06:16 smithi184 ceph-mon[36942]: Unable to set osd_memory_target on smithi151 to 185064857: error parsing value: Value '185064857' is below minimum 939524096 2024-05-21T22:06:17.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:06:16 smithi164 ceph-mon[36269]: Adjusting osd_memory_target on smithi151 to 176.4M 2024-05-21T22:06:17.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:06:16 smithi164 ceph-mon[36269]: Unable to set osd_memory_target on smithi151 to 185064857: error parsing value: Value '185064857' is below minimum 939524096 2024-05-21T22:06:17.924 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:06:17 smithi151 ceph-mon[31623]: pgmap v331: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-05-21T22:06:17.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:06:17 smithi184 ceph-mon[36942]: pgmap v331: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-05-21T22:06:18.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:06:17 smithi164 ceph-mon[36269]: pgmap v331: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-05-21T22:06:18.509 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-05-21T22:06:18.509 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-0": { 2024-05-21T22:06:18.509 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 0, 2024-05-21T22:06:18.509 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 2, 2024-05-21T22:06:18.509 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 0, 2024-05-21T22:06:18.509 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 0, 2024-05-21T22:06:18.510 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 4, 2024-05-21T22:06:18.510 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 0, 2024-05-21T22:06:18.510 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.004996291, 2024-05-21T22:06:18.510 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.000000000, 2024-05-21T22:06:18.510 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.000942818, 2024-05-21T22:06:18.510 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-21T22:06:18.510 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T22:06:18.510 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-21T22:06:18.510 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-21T22:06:18.510 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-21T22:06:18.510 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:18.510 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-05-21T22:06:18.511 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-21T22:06:18.511 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-21T22:06:18.511 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-21T22:06:18.511 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:18.511 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-21T22:06:18.511 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 0 2024-05-21T22:06:18.511 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:18.511 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-1": { 2024-05-21T22:06:18.511 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 263, 2024-05-21T22:06:18.511 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 249, 2024-05-21T22:06:18.511 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 1874000, 2024-05-21T22:06:18.511 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 153008, 2024-05-21T22:06:18.511 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 2, 2024-05-21T22:06:18.512 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 1, 2024-05-21T22:06:18.512 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.112087101, 2024-05-21T22:06:18.512 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.040089363, 2024-05-21T22:06:18.512 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.084754427, 2024-05-21T22:06:18.512 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.011065266, 2024-05-21T22:06:18.512 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T22:06:18.512 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 249, 2024-05-21T22:06:18.512 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.025191646, 2024-05-21T22:06:18.512 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000101171 2024-05-21T22:06:18.512 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:18.512 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-05-21T22:06:18.512 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-21T22:06:18.512 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-21T22:06:18.513 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-21T22:06:18.513 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:18.513 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 1874000, 2024-05-21T22:06:18.513 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 153008 2024-05-21T22:06:18.513 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:18.513 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-2": { 2024-05-21T22:06:18.513 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 2, 2024-05-21T22:06:18.513 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 189, 2024-05-21T22:06:18.513 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 320, 2024-05-21T22:06:18.513 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 190656, 2024-05-21T22:06:18.513 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 5, 2024-05-21T22:06:18.513 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 3, 2024-05-21T22:06:18.513 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.035288897, 2024-05-21T22:06:18.513 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.026294588, 2024-05-21T22:06:18.514 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.003563732, 2024-05-21T22:06:18.514 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-21T22:06:18.514 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T22:06:18.514 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 186, 2024-05-21T22:06:18.514 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.027829544, 2024-05-21T22:06:18.514 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000149621 2024-05-21T22:06:18.514 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:18.514 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-05-21T22:06:18.514 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-21T22:06:18.514 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-21T22:06:18.514 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-21T22:06:18.514 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:18.514 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-21T22:06:18.515 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 190656 2024-05-21T22:06:18.515 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:18.515 INFO:teuthology.orchestra.run.smithi151.stdout: "cct": { 2024-05-21T22:06:18.515 INFO:teuthology.orchestra.run.smithi151.stdout: "total_workers": 0, 2024-05-21T22:06:18.515 INFO:teuthology.orchestra.run.smithi151.stdout: "unhealthy_workers": 0 2024-05-21T22:06:18.515 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:18.515 INFO:teuthology.orchestra.run.smithi151.stdout: "mempool": { 2024-05-21T22:06:18.515 INFO:teuthology.orchestra.run.smithi151.stdout: "bloom_filter_bytes": 0, 2024-05-21T22:06:18.515 INFO:teuthology.orchestra.run.smithi151.stdout: "bloom_filter_items": 0, 2024-05-21T22:06:18.515 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_alloc_bytes": 0, 2024-05-21T22:06:18.515 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_alloc_items": 0, 2024-05-21T22:06:18.515 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_data_bytes": 0, 2024-05-21T22:06:18.515 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_data_items": 0, 2024-05-21T22:06:18.515 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-21T22:06:18.516 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_onode_items": 0, 2024-05-21T22:06:18.516 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-21T22:06:18.516 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_meta_items": 0, 2024-05-21T22:06:18.516 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_other_bytes": 0, 2024-05-21T22:06:18.516 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_other_items": 0, 2024-05-21T22:06:18.516 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-21T22:06:18.516 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_buffer_items": 0, 2024-05-21T22:06:18.516 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_extent_bytes": 0, 2024-05-21T22:06:18.516 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_extent_items": 0, 2024-05-21T22:06:18.516 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_blob_bytes": 0, 2024-05-21T22:06:18.516 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_blob_items": 0, 2024-05-21T22:06:18.516 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-21T22:06:18.516 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_shared_blob_items": 0, 2024-05-21T22:06:18.516 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-21T22:06:18.517 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_inline_bl_items": 0, 2024-05-21T22:06:18.517 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_fsck_bytes": 0, 2024-05-21T22:06:18.517 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_fsck_items": 0, 2024-05-21T22:06:18.517 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_txc_bytes": 0, 2024-05-21T22:06:18.517 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_txc_items": 0, 2024-05-21T22:06:18.517 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-21T22:06:18.517 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_deferred_items": 0, 2024-05-21T22:06:18.517 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_bytes": 0, 2024-05-21T22:06:18.517 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_items": 0, 2024-05-21T22:06:18.517 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_bytes": 0, 2024-05-21T22:06:18.517 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_items": 0, 2024-05-21T22:06:18.517 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_reader_bytes": 0, 2024-05-21T22:06:18.517 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_reader_items": 0, 2024-05-21T22:06:18.517 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_writer_bytes": 0, 2024-05-21T22:06:18.518 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_writer_items": 0, 2024-05-21T22:06:18.518 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_anon_bytes": 149745, 2024-05-21T22:06:18.518 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_anon_items": 31, 2024-05-21T22:06:18.518 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_meta_bytes": 0, 2024-05-21T22:06:18.518 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_meta_items": 0, 2024-05-21T22:06:18.518 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_bytes": 0, 2024-05-21T22:06:18.518 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_items": 0, 2024-05-21T22:06:18.518 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_mapbl_bytes": 0, 2024-05-21T22:06:18.518 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_mapbl_items": 0, 2024-05-21T22:06:18.518 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_pglog_bytes": 0, 2024-05-21T22:06:18.518 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_pglog_items": 0, 2024-05-21T22:06:18.518 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_bytes": 0, 2024-05-21T22:06:18.518 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_items": 0, 2024-05-21T22:06:18.519 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_mapping_bytes": 0, 2024-05-21T22:06:18.519 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_mapping_items": 0, 2024-05-21T22:06:18.519 INFO:teuthology.orchestra.run.smithi151.stdout: "pgmap_bytes": 0, 2024-05-21T22:06:18.519 INFO:teuthology.orchestra.run.smithi151.stdout: "pgmap_items": 0, 2024-05-21T22:06:18.519 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_co_bytes": 928, 2024-05-21T22:06:18.519 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_co_items": 2, 2024-05-21T22:06:18.519 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_1_bytes": 0, 2024-05-21T22:06:18.519 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_1_items": 0, 2024-05-21T22:06:18.519 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_2_bytes": 0, 2024-05-21T22:06:18.519 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_2_items": 0 2024-05-21T22:06:18.519 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:18.519 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-21T22:06:18.519 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-05-21T22:06:18.519 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 104857600, 2024-05-21T22:06:18.520 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-05-21T22:06:18.520 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 261, 2024-05-21T22:06:18.520 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 1834465, 2024-05-21T22:06:18.520 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-05-21T22:06:18.520 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 261, 2024-05-21T22:06:18.520 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 0, 2024-05-21T22:06:18.520 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 0, 2024-05-21T22:06:18.520 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 261, 2024-05-21T22:06:18.520 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 1834465, 2024-05-21T22:06:18.520 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-05-21T22:06:18.520 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-21T22:06:18.520 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-21T22:06:18.520 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-21T22:06:18.520 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:18.521 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:18.521 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-05-21T22:06:19.392 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-05-21T22:06:19.392 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-05-21T22:06:19.392 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.g 2024-05-21T22:06:19.392 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.g 2024-05-21T22:06:19.392 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-05-21T22:06:19.393 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-05-21T22:06:19.393 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-05-21T22:06:19.393 DEBUG:tasks.check_counter:Getting stats from j 2024-05-21T22:06:19.393 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph daemon mds.j perf dump 2024-05-21T22:06:19.843 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:06:19.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:06:19 smithi151 ceph-mon[31623]: pgmap v332: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 229 B/s wr, 11 op/s 2024-05-21T22:06:19.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:06:19 smithi184 ceph-mon[36942]: pgmap v332: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 229 B/s wr, 11 op/s 2024-05-21T22:06:20.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:06:19 smithi164 ceph-mon[36269]: pgmap v332: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 229 B/s wr, 11 op/s 2024-05-21T22:06:21.885 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:06:21 smithi151 ceph-mon[31623]: pgmap v333: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-21T22:06:21.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:06:21 smithi184 ceph-mon[36942]: pgmap v333: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-21T22:06:22.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:06:21 smithi164 ceph-mon[36269]: pgmap v333: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-21T22:06:22.195 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-05-21T22:06:22.195 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-0": { 2024-05-21T22:06:22.195 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 0, 2024-05-21T22:06:22.195 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 2, 2024-05-21T22:06:22.195 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 0, 2024-05-21T22:06:22.195 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 0, 2024-05-21T22:06:22.195 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 4, 2024-05-21T22:06:22.195 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 0, 2024-05-21T22:06:22.195 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.003690089, 2024-05-21T22:06:22.196 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.000000000, 2024-05-21T22:06:22.196 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.000701305, 2024-05-21T22:06:22.196 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-21T22:06:22.196 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T22:06:22.196 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-21T22:06:22.196 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-21T22:06:22.196 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-21T22:06:22.196 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:22.196 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-05-21T22:06:22.196 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-21T22:06:22.196 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-21T22:06:22.196 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-21T22:06:22.196 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:22.196 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-21T22:06:22.197 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 0 2024-05-21T22:06:22.197 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:22.197 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-1": { 2024-05-21T22:06:22.197 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 270, 2024-05-21T22:06:22.197 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 256, 2024-05-21T22:06:22.197 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 1875904, 2024-05-21T22:06:22.197 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 157488, 2024-05-21T22:06:22.197 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 2, 2024-05-21T22:06:22.197 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 1, 2024-05-21T22:06:22.197 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.109784661, 2024-05-21T22:06:22.197 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.039243793, 2024-05-21T22:06:22.197 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.082477871, 2024-05-21T22:06:22.197 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.010982965, 2024-05-21T22:06:22.197 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T22:06:22.198 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 256, 2024-05-21T22:06:22.198 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.023947701, 2024-05-21T22:06:22.198 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000093545 2024-05-21T22:06:22.198 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:22.198 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-05-21T22:06:22.198 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-21T22:06:22.198 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-21T22:06:22.198 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-21T22:06:22.198 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:22.198 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 1875904, 2024-05-21T22:06:22.198 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 157488 2024-05-21T22:06:22.198 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:22.198 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-2": { 2024-05-21T22:06:22.199 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 2, 2024-05-21T22:06:22.199 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 194, 2024-05-21T22:06:22.199 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 320, 2024-05-21T22:06:22.199 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 191536, 2024-05-21T22:06:22.199 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 5, 2024-05-21T22:06:22.199 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 3, 2024-05-21T22:06:22.199 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.038546824, 2024-05-21T22:06:22.199 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.028312826, 2024-05-21T22:06:22.199 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.003735777, 2024-05-21T22:06:22.199 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-21T22:06:22.199 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T22:06:22.199 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 191, 2024-05-21T22:06:22.200 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.028425029, 2024-05-21T22:06:22.200 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000148822 2024-05-21T22:06:22.200 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:22.200 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-05-21T22:06:22.200 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-21T22:06:22.200 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-21T22:06:22.200 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-21T22:06:22.200 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:22.200 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-21T22:06:22.200 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 191536 2024-05-21T22:06:22.200 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:22.200 INFO:teuthology.orchestra.run.smithi151.stdout: "cct": { 2024-05-21T22:06:22.200 INFO:teuthology.orchestra.run.smithi151.stdout: "total_workers": 0, 2024-05-21T22:06:22.200 INFO:teuthology.orchestra.run.smithi151.stdout: "unhealthy_workers": 0 2024-05-21T22:06:22.201 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:22.201 INFO:teuthology.orchestra.run.smithi151.stdout: "mempool": { 2024-05-21T22:06:22.201 INFO:teuthology.orchestra.run.smithi151.stdout: "bloom_filter_bytes": 0, 2024-05-21T22:06:22.201 INFO:teuthology.orchestra.run.smithi151.stdout: "bloom_filter_items": 0, 2024-05-21T22:06:22.201 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_alloc_bytes": 0, 2024-05-21T22:06:22.201 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_alloc_items": 0, 2024-05-21T22:06:22.201 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_data_bytes": 0, 2024-05-21T22:06:22.201 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_data_items": 0, 2024-05-21T22:06:22.201 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-21T22:06:22.201 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_onode_items": 0, 2024-05-21T22:06:22.201 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-21T22:06:22.201 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_meta_items": 0, 2024-05-21T22:06:22.201 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_other_bytes": 0, 2024-05-21T22:06:22.201 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_other_items": 0, 2024-05-21T22:06:22.201 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-21T22:06:22.202 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_buffer_items": 0, 2024-05-21T22:06:22.202 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_extent_bytes": 0, 2024-05-21T22:06:22.202 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_extent_items": 0, 2024-05-21T22:06:22.202 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_blob_bytes": 0, 2024-05-21T22:06:22.202 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_blob_items": 0, 2024-05-21T22:06:22.202 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-21T22:06:22.202 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_shared_blob_items": 0, 2024-05-21T22:06:22.202 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-21T22:06:22.202 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_inline_bl_items": 0, 2024-05-21T22:06:22.202 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_fsck_bytes": 0, 2024-05-21T22:06:22.202 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_fsck_items": 0, 2024-05-21T22:06:22.202 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_txc_bytes": 0, 2024-05-21T22:06:22.202 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_txc_items": 0, 2024-05-21T22:06:22.202 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-21T22:06:22.203 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_deferred_items": 0, 2024-05-21T22:06:22.203 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_bytes": 0, 2024-05-21T22:06:22.203 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_items": 0, 2024-05-21T22:06:22.203 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_bytes": 0, 2024-05-21T22:06:22.203 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_items": 0, 2024-05-21T22:06:22.203 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_reader_bytes": 0, 2024-05-21T22:06:22.203 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_reader_items": 0, 2024-05-21T22:06:22.203 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_writer_bytes": 0, 2024-05-21T22:06:22.203 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_writer_items": 0, 2024-05-21T22:06:22.203 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_anon_bytes": 148714, 2024-05-21T22:06:22.203 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_anon_items": 30, 2024-05-21T22:06:22.203 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_meta_bytes": 0, 2024-05-21T22:06:22.203 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_meta_items": 0, 2024-05-21T22:06:22.203 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_bytes": 0, 2024-05-21T22:06:22.203 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_items": 0, 2024-05-21T22:06:22.204 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_mapbl_bytes": 0, 2024-05-21T22:06:22.204 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_mapbl_items": 0, 2024-05-21T22:06:22.204 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_pglog_bytes": 0, 2024-05-21T22:06:22.204 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_pglog_items": 0, 2024-05-21T22:06:22.204 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_bytes": 0, 2024-05-21T22:06:22.204 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_items": 0, 2024-05-21T22:06:22.204 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_mapping_bytes": 0, 2024-05-21T22:06:22.204 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_mapping_items": 0, 2024-05-21T22:06:22.204 INFO:teuthology.orchestra.run.smithi151.stdout: "pgmap_bytes": 0, 2024-05-21T22:06:22.204 INFO:teuthology.orchestra.run.smithi151.stdout: "pgmap_items": 0, 2024-05-21T22:06:22.204 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_co_bytes": 928, 2024-05-21T22:06:22.204 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_co_items": 2, 2024-05-21T22:06:22.204 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_1_bytes": 0, 2024-05-21T22:06:22.204 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_1_items": 0, 2024-05-21T22:06:22.204 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_2_bytes": 0, 2024-05-21T22:06:22.205 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_2_items": 0 2024-05-21T22:06:22.205 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-05-21T22:06:22.205 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-21T22:06:22.205 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-05-21T22:06:22.205 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 104857600, 2024-05-21T22:06:22.205 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-05-21T22:06:22.205 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 268, 2024-05-21T22:06:22.205 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 1835375, 2024-05-21T22:06:22.205 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-05-21T22:06:22.205 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 268, 2024-05-21T22:06:22.205 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 0, 2024-05-21T22:06:22.205 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 0, 2024-05-21T22:06:22.205 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 268, 2024-05-21T22:06:22.205 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 1835375, 2024-05-21T22:06:22.206 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-05-21T22:06:22.206 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-05-21T22:06:22.206 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-05-21T22:06:22.206 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-05-21T22:06:22.206 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:22.206 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-05-21T22:06:22.206 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-05-21T22:06:23.025 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-05-21T22:06:23.025 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-05-21T22:06:23.025 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-05-21T22:06:23.025 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-05-21T22:06:23.025 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-05-21T22:06:23.025 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-05-21T22:06:23.025 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-05-21T22:06:23.025 DEBUG:tasks.check_counter:Getting stats from b 2024-05-21T22:06:23.026 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph daemon mds.b perf dump 2024-05-21T22:06:23.275 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.b/config 2024-05-21T22:06:23.652 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:06:23 smithi164 ceph-mon[36269]: pgmap v334: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 102 B/s wr, 10 op/s 2024-05-21T22:06:23.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:06:23 smithi151 ceph-mon[31623]: pgmap v334: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 102 B/s wr, 10 op/s 2024-05-21T22:06:23.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:06:23 smithi184 ceph-mon[36942]: pgmap v334: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 102 B/s wr, 10 op/s 2024-05-21T22:06:24.972 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-05-21T22:06:24.973 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-0": { 2024-05-21T22:06:24.973 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 1993, 2024-05-21T22:06:24.973 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 1835, 2024-05-21T22:06:24.973 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 2499030, 2024-05-21T22:06:24.973 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 915777, 2024-05-21T22:06:24.973 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 11, 2024-05-21T22:06:24.973 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 6, 2024-05-21T22:06:24.973 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 0.234196388, 2024-05-21T22:06:24.973 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 0.081329775, 2024-05-21T22:06:24.973 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 0.113255973, 2024-05-21T22:06:24.973 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 0.044828754, 2024-05-21T22:06:24.974 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T22:06:24.974 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 1835, 2024-05-21T22:06:24.974 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.508583543, 2024-05-21T22:06:24.974 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000277157 2024-05-21T22:06:24.974 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:24.974 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2024-05-21T22:06:24.974 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 2529, 2024-05-21T22:06:24.974 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000569993, 2024-05-21T22:06:24.974 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000225 2024-05-21T22:06:24.974 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:24.974 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 1950560, 2024-05-21T22:06:24.975 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 505472 2024-05-21T22:06:24.975 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:24.975 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-1": { 2024-05-21T22:06:24.975 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 2572, 2024-05-21T22:06:24.975 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 1322, 2024-05-21T22:06:24.975 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 854290, 2024-05-21T22:06:24.975 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 670676, 2024-05-21T22:06:24.975 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 15, 2024-05-21T22:06:24.975 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 7, 2024-05-21T22:06:24.975 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 0.316267557, 2024-05-21T22:06:24.975 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 0.088576252, 2024-05-21T22:06:24.975 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 0.120836813, 2024-05-21T22:06:24.975 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 0.100540131, 2024-05-21T22:06:24.976 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T22:06:24.976 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 1323, 2024-05-21T22:06:24.976 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.114915482, 2024-05-21T22:06:24.976 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000086859 2024-05-21T22:06:24.976 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:24.976 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2024-05-21T22:06:24.976 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 2358, 2024-05-21T22:06:24.976 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000586940, 2024-05-21T22:06:24.976 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000248 2024-05-21T22:06:24.976 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:24.976 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 329824, 2024-05-21T22:06:24.977 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 274880 2024-05-21T22:06:24.977 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:24.977 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-2": { 2024-05-21T22:06:24.977 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 1689, 2024-05-21T22:06:24.977 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 1320, 2024-05-21T22:06:24.977 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 663959, 2024-05-21T22:06:24.977 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 2932420, 2024-05-21T22:06:24.977 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 126, 2024-05-21T22:06:24.977 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 8, 2024-05-21T22:06:24.977 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 0.205739030, 2024-05-21T22:06:24.977 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 0.052060196, 2024-05-21T22:06:24.977 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 0.095926060, 2024-05-21T22:06:24.978 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 0.050232728, 2024-05-21T22:06:24.978 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T22:06:24.978 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 1314, 2024-05-21T22:06:24.978 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.179574212, 2024-05-21T22:06:24.978 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000136662 2024-05-21T22:06:24.978 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:24.978 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2024-05-21T22:06:24.978 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 1770, 2024-05-21T22:06:24.978 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000391899, 2024-05-21T22:06:24.978 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000221 2024-05-21T22:06:24.978 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:24.978 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 139520, 2024-05-21T22:06:24.979 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 32608 2024-05-21T22:06:24.979 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:24.979 INFO:teuthology.orchestra.run.smithi164.stdout: "cct": { 2024-05-21T22:06:24.979 INFO:teuthology.orchestra.run.smithi164.stdout: "total_workers": 1, 2024-05-21T22:06:24.979 INFO:teuthology.orchestra.run.smithi164.stdout: "unhealthy_workers": 0 2024-05-21T22:06:24.979 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:24.979 INFO:teuthology.orchestra.run.smithi164.stdout: "finisher-MDSRank": { 2024-05-21T22:06:24.979 INFO:teuthology.orchestra.run.smithi164.stdout: "queue_len": 0, 2024-05-21T22:06:24.979 INFO:teuthology.orchestra.run.smithi164.stdout: "complete_latency": { 2024-05-21T22:06:24.979 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 603, 2024-05-21T22:06:24.979 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.176280601, 2024-05-21T22:06:24.979 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000292339 2024-05-21T22:06:24.979 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-21T22:06:24.979 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:24.980 INFO:teuthology.orchestra.run.smithi164.stdout: "finisher-PurgeQueue": { 2024-05-21T22:06:24.980 INFO:teuthology.orchestra.run.smithi164.stdout: "queue_len": 0, 2024-05-21T22:06:24.980 INFO:teuthology.orchestra.run.smithi164.stdout: "complete_latency": { 2024-05-21T22:06:24.980 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 89, 2024-05-21T22:06:24.980 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.002899160, 2024-05-21T22:06:24.980 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000032574 2024-05-21T22:06:24.980 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-21T22:06:24.980 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:24.980 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": { 2024-05-21T22:06:24.980 INFO:teuthology.orchestra.run.smithi164.stdout: "request": 173, 2024-05-21T22:06:24.980 INFO:teuthology.orchestra.run.smithi164.stdout: "reply": 173, 2024-05-21T22:06:24.980 INFO:teuthology.orchestra.run.smithi164.stdout: "reply_latency": { 2024-05-21T22:06:24.980 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 173, 2024-05-21T22:06:24.981 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 23.890647890, 2024-05-21T22:06:24.981 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.138096230 2024-05-21T22:06:24.981 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:24.981 INFO:teuthology.orchestra.run.smithi164.stdout: "slow_reply": 0, 2024-05-21T22:06:24.981 INFO:teuthology.orchestra.run.smithi164.stdout: "forward": 0, 2024-05-21T22:06:24.981 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_fetch_complete": 0, 2024-05-21T22:06:24.981 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_fetch_keys": 0, 2024-05-21T22:06:24.981 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_commit": 12, 2024-05-21T22:06:24.981 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_split": 0, 2024-05-21T22:06:24.981 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_merge": 0, 2024-05-21T22:06:24.982 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes": 39, 2024-05-21T22:06:24.982 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_top": 2, 2024-05-21T22:06:24.982 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_bottom": 37, 2024-05-21T22:06:24.982 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_pin_tail": 0, 2024-05-21T22:06:24.982 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_pinned": 36, 2024-05-21T22:06:24.982 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_expired": 0, 2024-05-21T22:06:24.982 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_with_caps": 3, 2024-05-21T22:06:24.982 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": 3, 2024-05-21T22:06:24.982 INFO:teuthology.orchestra.run.smithi164.stdout: "subtrees": 2, 2024-05-21T22:06:24.982 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse": 236, 2024-05-21T22:06:24.982 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_hit": 220, 2024-05-21T22:06:24.983 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_forward": 0, 2024-05-21T22:06:24.983 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_discover": 0, 2024-05-21T22:06:24.983 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_dir_fetch": 0, 2024-05-21T22:06:24.983 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_remote_ino": 0, 2024-05-21T22:06:24.983 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_lock": 0, 2024-05-21T22:06:24.983 INFO:teuthology.orchestra.run.smithi164.stdout: "load_cent": 0, 2024-05-21T22:06:24.983 INFO:teuthology.orchestra.run.smithi164.stdout: "q": 0, 2024-05-21T22:06:24.983 INFO:teuthology.orchestra.run.smithi164.stdout: "exported": 0, 2024-05-21T22:06:24.983 INFO:teuthology.orchestra.run.smithi164.stdout: "exported_inodes": 0, 2024-05-21T22:06:24.983 INFO:teuthology.orchestra.run.smithi164.stdout: "imported": 0, 2024-05-21T22:06:24.983 INFO:teuthology.orchestra.run.smithi164.stdout: "imported_inodes": 0, 2024-05-21T22:06:24.983 INFO:teuthology.orchestra.run.smithi164.stdout: "openino_dir_fetch": 0, 2024-05-21T22:06:24.983 INFO:teuthology.orchestra.run.smithi164.stdout: "openino_backtrace_fetch": 185, 2024-05-21T22:06:24.983 INFO:teuthology.orchestra.run.smithi164.stdout: "openino_peer_discover": 0, 2024-05-21T22:06:24.984 INFO:teuthology.orchestra.run.smithi164.stdout: "root_rfiles": 4, 2024-05-21T22:06:24.984 INFO:teuthology.orchestra.run.smithi164.stdout: "root_rbytes": 234, 2024-05-21T22:06:24.984 INFO:teuthology.orchestra.run.smithi164.stdout: "root_rsnaps": 7, 2024-05-21T22:06:24.984 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_backtrace_fetch": 185, 2024-05-21T22:06:24.984 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_set_tag": 0, 2024-05-21T22:06:24.984 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_backtrace_repaired": 0, 2024-05-21T22:06:24.984 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_inotable_repaired": 0, 2024-05-21T22:06:24.984 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_dir_inodes": 117, 2024-05-21T22:06:24.984 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_dir_base_inodes": 16, 2024-05-21T22:06:24.984 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_dirfrag_rstats": 101, 2024-05-21T22:06:24.984 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_file_inodes": 68, 2024-05-21T22:06:24.984 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_inode_file_caps": 0, 2024-05-21T22:06:24.984 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_revoke": 21, 2024-05-21T22:06:24.984 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_grant": 38, 2024-05-21T22:06:24.985 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_trunc": 0, 2024-05-21T22:06:24.985 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-21T22:06:24.985 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-21T22:06:24.985 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_caps": 117, 2024-05-21T22:06:24.985 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_caps_dirty": 90, 2024-05-21T22:06:24.985 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_cap_release": 3, 2024-05-21T22:06:24.985 INFO:teuthology.orchestra.run.smithi164.stdout: "process_request_cap_release": 28 2024-05-21T22:06:24.985 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:24.985 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_cache": { 2024-05-21T22:06:24.985 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_update": 19, 2024-05-21T22:06:24.985 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_update_receipt": 0, 2024-05-21T22:06:24.985 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_try_discover": 0, 2024-05-21T22:06:24.985 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_send_discover": 0, 2024-05-21T22:06:24.985 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_handle_discover": 40, 2024-05-21T22:06:24.986 INFO:teuthology.orchestra.run.smithi164.stdout: "num_strays": 2, 2024-05-21T22:06:24.986 INFO:teuthology.orchestra.run.smithi164.stdout: "num_strays_delayed": 0, 2024-05-21T22:06:24.986 INFO:teuthology.orchestra.run.smithi164.stdout: "num_strays_enqueuing": 0, 2024-05-21T22:06:24.986 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_created": 6, 2024-05-21T22:06:24.986 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_enqueued": 4, 2024-05-21T22:06:24.986 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_reintegrated": 0, 2024-05-21T22:06:24.986 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_migrated": 0, 2024-05-21T22:06:24.986 INFO:teuthology.orchestra.run.smithi164.stdout: "num_recovering_processing": 0, 2024-05-21T22:06:24.986 INFO:teuthology.orchestra.run.smithi164.stdout: "num_recovering_enqueued": 0, 2024-05-21T22:06:24.986 INFO:teuthology.orchestra.run.smithi164.stdout: "num_recovering_prioritized": 0, 2024-05-21T22:06:24.986 INFO:teuthology.orchestra.run.smithi164.stdout: "recovery_started": 0, 2024-05-21T22:06:24.986 INFO:teuthology.orchestra.run.smithi164.stdout: "recovery_completed": 0, 2024-05-21T22:06:24.986 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_quiesce_path": 2, 2024-05-21T22:06:24.987 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_quiesce_inode": 24, 2024-05-21T22:06:24.987 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_enqueue_scrub": 16, 2024-05-21T22:06:24.987 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_exportdir": 0, 2024-05-21T22:06:24.987 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_flush": 0, 2024-05-21T22:06:24.987 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_fragmentdir": 0, 2024-05-21T22:06:24.987 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_fragstats": 0, 2024-05-21T22:06:24.987 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_inodestats": 0 2024-05-21T22:06:24.987 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:24.987 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_log": { 2024-05-21T22:06:24.987 INFO:teuthology.orchestra.run.smithi164.stdout: "evlrg": 0, 2024-05-21T22:06:24.987 INFO:teuthology.orchestra.run.smithi164.stdout: "evadd": 233, 2024-05-21T22:06:24.987 INFO:teuthology.orchestra.run.smithi164.stdout: "evex": 0, 2024-05-21T22:06:24.988 INFO:teuthology.orchestra.run.smithi164.stdout: "evtrm": 0, 2024-05-21T22:06:24.988 INFO:teuthology.orchestra.run.smithi164.stdout: "ev": 233, 2024-05-21T22:06:24.988 INFO:teuthology.orchestra.run.smithi164.stdout: "evexg": 0, 2024-05-21T22:06:24.988 INFO:teuthology.orchestra.run.smithi164.stdout: "evexd": 0, 2024-05-21T22:06:24.988 INFO:teuthology.orchestra.run.smithi164.stdout: "segadd": 2, 2024-05-21T22:06:24.988 INFO:teuthology.orchestra.run.smithi164.stdout: "segex": 0, 2024-05-21T22:06:24.988 INFO:teuthology.orchestra.run.smithi164.stdout: "segtrm": 0, 2024-05-21T22:06:24.988 INFO:teuthology.orchestra.run.smithi164.stdout: "seg": 2, 2024-05-21T22:06:24.988 INFO:teuthology.orchestra.run.smithi164.stdout: "segmjr": 2, 2024-05-21T22:06:24.988 INFO:teuthology.orchestra.run.smithi164.stdout: "segexg": 0, 2024-05-21T22:06:24.988 INFO:teuthology.orchestra.run.smithi164.stdout: "segexd": 0, 2024-05-21T22:06:24.988 INFO:teuthology.orchestra.run.smithi164.stdout: "expos": 4194304, 2024-05-21T22:06:24.989 INFO:teuthology.orchestra.run.smithi164.stdout: "wrpos": 5352015, 2024-05-21T22:06:24.989 INFO:teuthology.orchestra.run.smithi164.stdout: "rdpos": 0, 2024-05-21T22:06:24.989 INFO:teuthology.orchestra.run.smithi164.stdout: "jlat": { 2024-05-21T22:06:24.989 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 196, 2024-05-21T22:06:24.989 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.637209091, 2024-05-21T22:06:24.989 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.003251066 2024-05-21T22:06:24.989 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:24.989 INFO:teuthology.orchestra.run.smithi164.stdout: "replayed": 0 2024-05-21T22:06:24.989 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:24.989 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_mem": { 2024-05-21T22:06:24.989 INFO:teuthology.orchestra.run.smithi164.stdout: "ino": 29, 2024-05-21T22:06:24.989 INFO:teuthology.orchestra.run.smithi164.stdout: "ino+": 49, 2024-05-21T22:06:24.989 INFO:teuthology.orchestra.run.smithi164.stdout: "ino-": 20, 2024-05-21T22:06:24.990 INFO:teuthology.orchestra.run.smithi164.stdout: "dir": 20, 2024-05-21T22:06:24.990 INFO:teuthology.orchestra.run.smithi164.stdout: "dir+": 20, 2024-05-21T22:06:24.990 INFO:teuthology.orchestra.run.smithi164.stdout: "dir-": 0, 2024-05-21T22:06:24.990 INFO:teuthology.orchestra.run.smithi164.stdout: "dn": 39, 2024-05-21T22:06:24.990 INFO:teuthology.orchestra.run.smithi164.stdout: "dn+": 43, 2024-05-21T22:06:24.990 INFO:teuthology.orchestra.run.smithi164.stdout: "dn-": 4, 2024-05-21T22:06:24.990 INFO:teuthology.orchestra.run.smithi164.stdout: "cap": 3, 2024-05-21T22:06:24.990 INFO:teuthology.orchestra.run.smithi164.stdout: "cap+": 21, 2024-05-21T22:06:24.990 INFO:teuthology.orchestra.run.smithi164.stdout: "cap-": 18, 2024-05-21T22:06:24.990 INFO:teuthology.orchestra.run.smithi164.stdout: "rss": 55876, 2024-05-21T22:06:24.990 INFO:teuthology.orchestra.run.smithi164.stdout: "heap": 231800 2024-05-21T22:06:24.990 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:24.991 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_server": { 2024-05-21T22:06:24.991 INFO:teuthology.orchestra.run.smithi164.stdout: "dispatch_client_request": 230, 2024-05-21T22:06:24.991 INFO:teuthology.orchestra.run.smithi164.stdout: "dispatch_server_request": 45, 2024-05-21T22:06:24.991 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_request": 173, 2024-05-21T22:06:24.991 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_session": 33, 2024-05-21T22:06:24.991 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_peer_request": 90, 2024-05-21T22:06:24.991 INFO:teuthology.orchestra.run.smithi164.stdout: "req_create_latency": { 2024-05-21T22:06:24.991 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 8, 2024-05-21T22:06:24.991 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.007408650, 2024-05-21T22:06:24.991 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000926081 2024-05-21T22:06:24.991 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:24.991 INFO:teuthology.orchestra.run.smithi164.stdout: "req_getattr_latency": { 2024-05-21T22:06:24.991 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 108, 2024-05-21T22:06:24.992 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.024333763, 2024-05-21T22:06:24.992 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000225312 2024-05-21T22:06:24.992 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:24.992 INFO:teuthology.orchestra.run.smithi164.stdout: "req_getfilelock_latency": { 2024-05-21T22:06:24.992 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:24.992 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:24.992 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:24.992 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:24.992 INFO:teuthology.orchestra.run.smithi164.stdout: "req_link_latency": { 2024-05-21T22:06:24.992 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:24.992 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:24.992 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:24.992 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:24.993 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookup_latency": { 2024-05-21T22:06:24.993 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 3, 2024-05-21T22:06:24.993 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000707113, 2024-05-21T22:06:24.993 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000235704 2024-05-21T22:06:24.993 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:24.993 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookuphash_latency": { 2024-05-21T22:06:24.993 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:24.993 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:24.993 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:24.993 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:24.993 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupino_latency": { 2024-05-21T22:06:24.993 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 1, 2024-05-21T22:06:24.993 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000280850, 2024-05-21T22:06:24.994 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000280850 2024-05-21T22:06:24.994 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:24.994 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupname_latency": { 2024-05-21T22:06:24.994 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:24.994 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:24.994 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:24.994 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:24.994 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupparent_latency": { 2024-05-21T22:06:24.994 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:24.994 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:24.994 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:24.994 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:24.995 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupsnap_latency": { 2024-05-21T22:06:24.995 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:24.995 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:24.995 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:24.995 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:24.995 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lssnap_latency": { 2024-05-21T22:06:24.995 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 11, 2024-05-21T22:06:24.995 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.002464956, 2024-05-21T22:06:24.995 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000224086 2024-05-21T22:06:24.996 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:24.996 INFO:teuthology.orchestra.run.smithi164.stdout: "req_mkdir_latency": { 2024-05-21T22:06:24.996 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 8, 2024-05-21T22:06:24.996 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 23.723906504, 2024-05-21T22:06:24.996 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 2.965488313 2024-05-21T22:06:24.996 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:24.996 INFO:teuthology.orchestra.run.smithi164.stdout: "req_mknod_latency": { 2024-05-21T22:06:24.997 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:24.997 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:24.997 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:24.997 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:24.997 INFO:teuthology.orchestra.run.smithi164.stdout: "req_mksnap_latency": { 2024-05-21T22:06:24.997 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 11, 2024-05-21T22:06:24.997 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.057127883, 2024-05-21T22:06:24.997 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.005193443 2024-05-21T22:06:24.997 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:24.997 INFO:teuthology.orchestra.run.smithi164.stdout: "req_open_latency": { 2024-05-21T22:06:24.997 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 1, 2024-05-21T22:06:24.997 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000221789, 2024-05-21T22:06:24.998 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000221789 2024-05-21T22:06:24.998 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:24.998 INFO:teuthology.orchestra.run.smithi164.stdout: "req_readdir_latency": { 2024-05-21T22:06:24.998 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 2, 2024-05-21T22:06:24.998 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.005173561, 2024-05-21T22:06:24.998 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.002586780 2024-05-21T22:06:24.998 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:24.998 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rename_latency": { 2024-05-21T22:06:24.998 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 2, 2024-05-21T22:06:24.998 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.002318302, 2024-05-21T22:06:24.998 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.001159151 2024-05-21T22:06:24.999 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:24.999 INFO:teuthology.orchestra.run.smithi164.stdout: "req_renamesnap_latency": { 2024-05-21T22:06:24.999 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:24.999 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:24.999 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:24.999 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:24.999 INFO:teuthology.orchestra.run.smithi164.stdout: "req_snapdiff_latency": { 2024-05-21T22:06:24.999 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:24.999 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:24.999 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:24.999 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:24.999 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rmdir_latency": { 2024-05-21T22:06:24.999 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 2, 2024-05-21T22:06:24.999 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.018736215, 2024-05-21T22:06:25.000 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.009368107 2024-05-21T22:06:25.000 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:25.000 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rmsnap_latency": { 2024-05-21T22:06:25.000 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 4, 2024-05-21T22:06:25.000 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.018536630, 2024-05-21T22:06:25.000 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.004634157 2024-05-21T22:06:25.000 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:25.000 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rmxattr_latency": { 2024-05-21T22:06:25.000 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:25.000 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:25.000 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:25.000 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:25.000 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setattr_latency": { 2024-05-21T22:06:25.000 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 1, 2024-05-21T22:06:25.001 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.001530256, 2024-05-21T22:06:25.001 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.001530256 2024-05-21T22:06:25.001 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:25.001 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setdirlayout_latency": { 2024-05-21T22:06:25.001 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:25.001 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:25.001 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:25.001 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:25.001 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setfilelock_latency": { 2024-05-21T22:06:25.001 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:25.001 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:25.001 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:25.001 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:25.002 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setlayout_latency": { 2024-05-21T22:06:25.002 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:25.002 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:25.002 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:25.002 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:25.002 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setxattr_latency": { 2024-05-21T22:06:25.002 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 7, 2024-05-21T22:06:25.002 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.022453431, 2024-05-21T22:06:25.002 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.003207633 2024-05-21T22:06:25.002 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:25.002 INFO:teuthology.orchestra.run.smithi164.stdout: "req_symlink_latency": { 2024-05-21T22:06:25.002 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:25.003 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:25.003 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:25.003 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:25.003 INFO:teuthology.orchestra.run.smithi164.stdout: "req_unlink_latency": { 2024-05-21T22:06:25.003 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 4, 2024-05-21T22:06:25.003 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.005447987, 2024-05-21T22:06:25.003 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.001361996 2024-05-21T22:06:25.003 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:25.003 INFO:teuthology.orchestra.run.smithi164.stdout: "cap_revoke_eviction": 0, 2024-05-21T22:06:25.003 INFO:teuthology.orchestra.run.smithi164.stdout: "cap_acquisition_throttle": 0, 2024-05-21T22:06:25.003 INFO:teuthology.orchestra.run.smithi164.stdout: "req_getvxattr_latency": { 2024-05-21T22:06:25.003 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:25.004 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:25.004 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:25.004 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-21T22:06:25.004 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:25.004 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_sessions": { 2024-05-21T22:06:25.004 INFO:teuthology.orchestra.run.smithi164.stdout: "session_count": 2, 2024-05-21T22:06:25.004 INFO:teuthology.orchestra.run.smithi164.stdout: "session_add": 5, 2024-05-21T22:06:25.004 INFO:teuthology.orchestra.run.smithi164.stdout: "session_remove": 3, 2024-05-21T22:06:25.004 INFO:teuthology.orchestra.run.smithi164.stdout: "sessions_open": 2, 2024-05-21T22:06:25.004 INFO:teuthology.orchestra.run.smithi164.stdout: "sessions_stale": 0, 2024-05-21T22:06:25.005 INFO:teuthology.orchestra.run.smithi164.stdout: "total_load": 27, 2024-05-21T22:06:25.005 INFO:teuthology.orchestra.run.smithi164.stdout: "average_load": 13, 2024-05-21T22:06:25.005 INFO:teuthology.orchestra.run.smithi164.stdout: "avg_session_uptime": 637, 2024-05-21T22:06:25.005 INFO:teuthology.orchestra.run.smithi164.stdout: "mdthresh_evicted": 0 2024-05-21T22:06:25.005 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:25.005 INFO:teuthology.orchestra.run.smithi164.stdout: "mempool": { 2024-05-21T22:06:25.005 INFO:teuthology.orchestra.run.smithi164.stdout: "bloom_filter_bytes": 0, 2024-05-21T22:06:25.005 INFO:teuthology.orchestra.run.smithi164.stdout: "bloom_filter_items": 0, 2024-05-21T22:06:25.005 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_alloc_bytes": 0, 2024-05-21T22:06:25.005 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_alloc_items": 0, 2024-05-21T22:06:25.005 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_data_bytes": 0, 2024-05-21T22:06:25.006 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_data_items": 0, 2024-05-21T22:06:25.006 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-21T22:06:25.006 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_onode_items": 0, 2024-05-21T22:06:25.006 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-21T22:06:25.006 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_meta_items": 0, 2024-05-21T22:06:25.006 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_other_bytes": 0, 2024-05-21T22:06:25.006 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_other_items": 0, 2024-05-21T22:06:25.006 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-21T22:06:25.006 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_buffer_items": 0, 2024-05-21T22:06:25.006 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_extent_bytes": 0, 2024-05-21T22:06:25.007 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_extent_items": 0, 2024-05-21T22:06:25.007 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_blob_bytes": 0, 2024-05-21T22:06:25.007 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_blob_items": 0, 2024-05-21T22:06:25.007 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-21T22:06:25.007 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_shared_blob_items": 0, 2024-05-21T22:06:25.007 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-21T22:06:25.007 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_inline_bl_items": 0, 2024-05-21T22:06:25.007 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_fsck_bytes": 0, 2024-05-21T22:06:25.007 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_fsck_items": 0, 2024-05-21T22:06:25.007 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_txc_bytes": 0, 2024-05-21T22:06:25.007 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_txc_items": 0, 2024-05-21T22:06:25.007 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-21T22:06:25.007 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_deferred_items": 0, 2024-05-21T22:06:25.007 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_bytes": 0, 2024-05-21T22:06:25.008 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_items": 0, 2024-05-21T22:06:25.008 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_bytes": 0, 2024-05-21T22:06:25.008 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_items": 0, 2024-05-21T22:06:25.008 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_reader_bytes": 0, 2024-05-21T22:06:25.008 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_reader_items": 0, 2024-05-21T22:06:25.008 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_writer_bytes": 0, 2024-05-21T22:06:25.008 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_writer_items": 0, 2024-05-21T22:06:25.008 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_anon_bytes": 320443, 2024-05-21T22:06:25.008 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_anon_items": 121, 2024-05-21T22:06:25.008 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_meta_bytes": 88, 2024-05-21T22:06:25.008 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_meta_items": 1, 2024-05-21T22:06:25.008 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_bytes": 0, 2024-05-21T22:06:25.008 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_items": 0, 2024-05-21T22:06:25.008 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_mapbl_bytes": 0, 2024-05-21T22:06:25.008 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_mapbl_items": 0, 2024-05-21T22:06:25.008 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_pglog_bytes": 0, 2024-05-21T22:06:25.009 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_pglog_items": 0, 2024-05-21T22:06:25.009 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_bytes": 8128, 2024-05-21T22:06:25.009 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_items": 169, 2024-05-21T22:06:25.009 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_mapping_bytes": 0, 2024-05-21T22:06:25.009 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_mapping_items": 0, 2024-05-21T22:06:25.009 INFO:teuthology.orchestra.run.smithi164.stdout: "pgmap_bytes": 0, 2024-05-21T22:06:25.009 INFO:teuthology.orchestra.run.smithi164.stdout: "pgmap_items": 0, 2024-05-21T22:06:25.009 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_co_bytes": 140837, 2024-05-21T22:06:25.009 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_co_items": 1300, 2024-05-21T22:06:25.009 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_1_bytes": 0, 2024-05-21T22:06:25.009 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_1_items": 0, 2024-05-21T22:06:25.009 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_2_bytes": 0, 2024-05-21T22:06:25.009 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_2_items": 0 2024-05-21T22:06:25.009 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:25.009 INFO:teuthology.orchestra.run.smithi164.stdout: "objecter": { 2024-05-21T22:06:25.009 INFO:teuthology.orchestra.run.smithi164.stdout: "op_active": 0, 2024-05-21T22:06:25.010 INFO:teuthology.orchestra.run.smithi164.stdout: "op_laggy": 0, 2024-05-21T22:06:25.010 INFO:teuthology.orchestra.run.smithi164.stdout: "op_send": 1078, 2024-05-21T22:06:25.010 INFO:teuthology.orchestra.run.smithi164.stdout: "op_send_bytes": 1176872, 2024-05-21T22:06:25.010 INFO:teuthology.orchestra.run.smithi164.stdout: "op_resend": 37, 2024-05-21T22:06:25.010 INFO:teuthology.orchestra.run.smithi164.stdout: "op_reply": 1078, 2024-05-21T22:06:25.010 INFO:teuthology.orchestra.run.smithi164.stdout: "op_latency": { 2024-05-21T22:06:25.010 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 1078, 2024-05-21T22:06:25.010 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 31.212023754, 2024-05-21T22:06:25.010 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.028953639 2024-05-21T22:06:25.010 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:25.010 INFO:teuthology.orchestra.run.smithi164.stdout: "op_inflight": 0, 2024-05-21T22:06:25.010 INFO:teuthology.orchestra.run.smithi164.stdout: "oplen_avg": { 2024-05-21T22:06:25.010 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 1078, 2024-05-21T22:06:25.011 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 1129 2024-05-21T22:06:25.011 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:25.011 INFO:teuthology.orchestra.run.smithi164.stdout: "op": 1078, 2024-05-21T22:06:25.011 INFO:teuthology.orchestra.run.smithi164.stdout: "op_r": 219, 2024-05-21T22:06:25.011 INFO:teuthology.orchestra.run.smithi164.stdout: "op_w": 859, 2024-05-21T22:06:25.011 INFO:teuthology.orchestra.run.smithi164.stdout: "op_rmw": 0, 2024-05-21T22:06:25.011 INFO:teuthology.orchestra.run.smithi164.stdout: "op_pg": 0, 2024-05-21T22:06:25.011 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_stat": 0, 2024-05-21T22:06:25.011 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_create": 11, 2024-05-21T22:06:25.011 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_read": 18, 2024-05-21T22:06:25.011 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_write": 198, 2024-05-21T22:06:25.011 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_writefull": 31, 2024-05-21T22:06:25.011 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_writesame": 0, 2024-05-21T22:06:25.011 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_append": 0, 2024-05-21T22:06:25.011 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_zero": 0, 2024-05-21T22:06:25.012 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_truncate": 0, 2024-05-21T22:06:25.012 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_delete": 579, 2024-05-21T22:06:25.012 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_mapext": 0, 2024-05-21T22:06:25.012 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_sparse_read": 0, 2024-05-21T22:06:25.012 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_clonerange": 0, 2024-05-21T22:06:25.012 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_getxattr": 201, 2024-05-21T22:06:25.012 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_setxattr": 22, 2024-05-21T22:06:25.012 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_cmpxattr": 0, 2024-05-21T22:06:25.012 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_rmxattr": 0, 2024-05-21T22:06:25.012 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_resetxattrs": 0, 2024-05-21T22:06:25.012 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_call": 0, 2024-05-21T22:06:25.013 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_watch": 0, 2024-05-21T22:06:25.013 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_notify": 0, 2024-05-21T22:06:25.013 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_src_cmpxattr": 0, 2024-05-21T22:06:25.013 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_pgls": 0, 2024-05-21T22:06:25.013 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_pgls_filter": 0, 2024-05-21T22:06:25.013 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_other": 0, 2024-05-21T22:06:25.013 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_active": 0, 2024-05-21T22:06:25.013 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_send": 0, 2024-05-21T22:06:25.013 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_resend": 0, 2024-05-21T22:06:25.013 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_ping": 0, 2024-05-21T22:06:25.013 INFO:teuthology.orchestra.run.smithi164.stdout: "poolop_active": 0, 2024-05-21T22:06:25.014 INFO:teuthology.orchestra.run.smithi164.stdout: "poolop_send": 0, 2024-05-21T22:06:25.014 INFO:teuthology.orchestra.run.smithi164.stdout: "poolop_resend": 0, 2024-05-21T22:06:25.014 INFO:teuthology.orchestra.run.smithi164.stdout: "poolstat_active": 0, 2024-05-21T22:06:25.014 INFO:teuthology.orchestra.run.smithi164.stdout: "poolstat_send": 0, 2024-05-21T22:06:25.014 INFO:teuthology.orchestra.run.smithi164.stdout: "poolstat_resend": 0, 2024-05-21T22:06:25.014 INFO:teuthology.orchestra.run.smithi164.stdout: "statfs_active": 0, 2024-05-21T22:06:25.014 INFO:teuthology.orchestra.run.smithi164.stdout: "statfs_send": 0, 2024-05-21T22:06:25.014 INFO:teuthology.orchestra.run.smithi164.stdout: "statfs_resend": 0, 2024-05-21T22:06:25.014 INFO:teuthology.orchestra.run.smithi164.stdout: "command_active": 0, 2024-05-21T22:06:25.014 INFO:teuthology.orchestra.run.smithi164.stdout: "command_send": 0, 2024-05-21T22:06:25.014 INFO:teuthology.orchestra.run.smithi164.stdout: "command_resend": 0, 2024-05-21T22:06:25.014 INFO:teuthology.orchestra.run.smithi164.stdout: "map_epoch": 91, 2024-05-21T22:06:25.014 INFO:teuthology.orchestra.run.smithi164.stdout: "map_full": 0, 2024-05-21T22:06:25.015 INFO:teuthology.orchestra.run.smithi164.stdout: "map_inc": 11, 2024-05-21T22:06:25.015 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_sessions": 12, 2024-05-21T22:06:25.015 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_session_open": 12, 2024-05-21T22:06:25.015 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_session_close": 0, 2024-05-21T22:06:25.015 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_laggy": 0, 2024-05-21T22:06:25.015 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_wr": 56, 2024-05-21T22:06:25.015 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_rd": 0, 2024-05-21T22:06:25.015 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_del": 13 2024-05-21T22:06:25.015 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:25.015 INFO:teuthology.orchestra.run.smithi164.stdout: "oft": { 2024-05-21T22:06:25.015 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_objs": 1, 2024-05-21T22:06:25.015 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_kv_pairs": 0, 2024-05-21T22:06:25.015 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_updates": 0, 2024-05-21T22:06:25.015 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_removes": 0 2024-05-21T22:06:25.015 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:25.015 INFO:teuthology.orchestra.run.smithi164.stdout: "osd-slow-ops": { 2024-05-21T22:06:25.016 INFO:teuthology.orchestra.run.smithi164.stdout: "slow_ops_count": 71 2024-05-21T22:06:25.016 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:25.016 INFO:teuthology.orchestra.run.smithi164.stdout: "purge_queue": { 2024-05-21T22:06:25.016 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing_ops": 0, 2024-05-21T22:06:25.016 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing_ops_high_water": 64, 2024-05-21T22:06:25.016 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing": 0, 2024-05-21T22:06:25.016 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing_high_water": 3, 2024-05-21T22:06:25.016 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executed_ops": 67, 2024-05-21T22:06:25.016 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executed": 4, 2024-05-21T22:06:25.016 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_item_in_journal": 0 2024-05-21T22:06:25.016 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:25.016 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-21T22:06:25.016 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-05-21T22:06:25.016 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 104857600, 2024-05-21T22:06:25.016 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-05-21T22:06:25.017 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 6250, 2024-05-21T22:06:25.017 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 3397354, 2024-05-21T22:06:25.017 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-05-21T22:06:25.017 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 6250, 2024-05-21T22:06:25.017 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 0, 2024-05-21T22:06:25.017 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 0, 2024-05-21T22:06:25.017 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 6250, 2024-05-21T22:06:25.017 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 3397354, 2024-05-21T22:06:25.017 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-05-21T22:06:25.017 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:25.017 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:25.017 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:25.017 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-21T22:06:25.017 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:25.017 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-objecter_bytes": { 2024-05-21T22:06:25.017 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-05-21T22:06:25.018 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 104857600, 2024-05-21T22:06:25.018 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-05-21T22:06:25.018 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 0, 2024-05-21T22:06:25.018 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 0, 2024-05-21T22:06:25.018 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-05-21T22:06:25.018 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 0, 2024-05-21T22:06:25.018 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 1078, 2024-05-21T22:06:25.018 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 1177308, 2024-05-21T22:06:25.018 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 1078, 2024-05-21T22:06:25.018 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 1177308, 2024-05-21T22:06:25.018 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-05-21T22:06:25.018 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:25.018 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:25.019 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:25.019 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-21T22:06:25.019 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:25.019 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-objecter_ops": { 2024-05-21T22:06:25.019 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-05-21T22:06:25.019 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 1024, 2024-05-21T22:06:25.019 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-05-21T22:06:25.019 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 0, 2024-05-21T22:06:25.019 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 0, 2024-05-21T22:06:25.019 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-05-21T22:06:25.019 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 0, 2024-05-21T22:06:25.019 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 1078, 2024-05-21T22:06:25.019 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 1078, 2024-05-21T22:06:25.019 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 1078, 2024-05-21T22:06:25.019 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 1078, 2024-05-21T22:06:25.020 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-05-21T22:06:25.020 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:25.020 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:25.020 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:25.020 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-21T22:06:25.020 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:25.020 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-write_buf_throttle": { 2024-05-21T22:06:25.020 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-05-21T22:06:25.020 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 3758096384, 2024-05-21T22:06:25.020 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-05-21T22:06:25.020 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 4, 2024-05-21T22:06:25.020 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 436, 2024-05-21T22:06:25.020 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-05-21T22:06:25.020 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 4, 2024-05-21T22:06:25.021 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 0, 2024-05-21T22:06:25.021 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 0, 2024-05-21T22:06:25.021 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 2, 2024-05-21T22:06:25.021 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 436, 2024-05-21T22:06:25.021 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-05-21T22:06:25.021 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:25.021 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:25.021 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:25.021 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-21T22:06:25.021 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:25.021 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-write_buf_throttle-0x55b20ee80500": { 2024-05-21T22:06:25.021 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-05-21T22:06:25.021 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 3758096384, 2024-05-21T22:06:25.021 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-05-21T22:06:25.022 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 233, 2024-05-21T22:06:25.022 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 1157711, 2024-05-21T22:06:25.022 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-05-21T22:06:25.022 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 233, 2024-05-21T22:06:25.022 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 0, 2024-05-21T22:06:25.022 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 0, 2024-05-21T22:06:25.022 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 196, 2024-05-21T22:06:25.022 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 1157711, 2024-05-21T22:06:25.022 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-05-21T22:06:25.022 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:25.022 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:25.022 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:25.023 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-21T22:06:25.023 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-21T22:06:25.023 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-05-21T22:06:25.748 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:06:25 smithi164 ceph-mon[36269]: pgmap v335: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 99 B/s wr, 9 op/s 2024-05-21T22:06:25.775 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-05-21T22:06:25.775 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-05-21T22:06:25.775 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=19 2024-05-21T22:06:25.775 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=0 2024-05-21T22:06:25.775 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=7 2024-05-21T22:06:25.775 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=11 2024-05-21T22:06:25.775 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=4 2024-05-21T22:06:25.775 DEBUG:tasks.check_counter:Getting stats from e 2024-05-21T22:06:25.776 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph daemon mds.e perf dump 2024-05-21T22:06:25.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:06:25 smithi151 ceph-mon[31623]: pgmap v335: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 99 B/s wr, 9 op/s 2024-05-21T22:06:25.985 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:06:25 smithi184 ceph-mon[36942]: pgmap v335: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 99 B/s wr, 9 op/s 2024-05-21T22:06:26.012 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.b/config 2024-05-21T22:06:27.649 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-05-21T22:06:27.649 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-0": { 2024-05-21T22:06:27.649 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 661, 2024-05-21T22:06:27.649 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 793, 2024-05-21T22:06:27.649 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 200533, 2024-05-21T22:06:27.650 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 222323, 2024-05-21T22:06:27.650 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 14, 2024-05-21T22:06:27.650 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 4, 2024-05-21T22:06:27.650 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 0.090720682, 2024-05-21T22:06:27.650 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 0.041120968, 2024-05-21T22:06:27.650 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 0.041821790, 2024-05-21T22:06:27.650 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 0.004674169, 2024-05-21T22:06:27.650 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T22:06:27.650 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 792, 2024-05-21T22:06:27.650 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.053829442, 2024-05-21T22:06:27.651 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000067966 2024-05-21T22:06:27.651 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:27.651 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2024-05-21T22:06:27.651 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 1379, 2024-05-21T22:06:27.651 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000355153, 2024-05-21T22:06:27.651 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000257 2024-05-21T22:06:27.651 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:27.651 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-05-21T22:06:27.651 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 448 2024-05-21T22:06:27.651 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:27.651 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-1": { 2024-05-21T22:06:27.651 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 394, 2024-05-21T22:06:27.651 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 509, 2024-05-21T22:06:27.652 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 1949448, 2024-05-21T22:06:27.652 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 571398, 2024-05-21T22:06:27.652 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 13, 2024-05-21T22:06:27.652 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 5, 2024-05-21T22:06:27.652 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 0.074640002, 2024-05-21T22:06:27.652 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 0.025611414, 2024-05-21T22:06:27.652 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 0.053102673, 2024-05-21T22:06:27.652 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 0.006923989, 2024-05-21T22:06:27.652 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T22:06:27.652 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 508, 2024-05-21T22:06:27.652 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.061138372, 2024-05-21T22:06:27.653 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000120351 2024-05-21T22:06:27.653 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:27.653 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2024-05-21T22:06:27.653 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 112, 2024-05-21T22:06:27.653 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000026271, 2024-05-21T22:06:27.653 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000234 2024-05-21T22:06:27.653 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:27.653 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 1921824, 2024-05-21T22:06:27.653 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 487936 2024-05-21T22:06:27.653 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:27.653 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-2": { 2024-05-21T22:06:27.654 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 118, 2024-05-21T22:06:27.654 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 184, 2024-05-21T22:06:27.654 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 35559, 2024-05-21T22:06:27.654 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 572853, 2024-05-21T22:06:27.654 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 41, 2024-05-21T22:06:27.654 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 6, 2024-05-21T22:06:27.654 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 0.025275005, 2024-05-21T22:06:27.654 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 0.006342187, 2024-05-21T22:06:27.654 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 0.016796565, 2024-05-21T22:06:27.654 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 0.000551860, 2024-05-21T22:06:27.654 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T22:06:27.654 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 181, 2024-05-21T22:06:27.655 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.030362003, 2024-05-21T22:06:27.655 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000167745 2024-05-21T22:06:27.655 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:27.655 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2024-05-21T22:06:27.655 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 157, 2024-05-21T22:06:27.655 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000033847, 2024-05-21T22:06:27.655 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000215 2024-05-21T22:06:27.655 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:27.655 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 2016, 2024-05-21T22:06:27.656 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 176480 2024-05-21T22:06:27.656 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:27.656 INFO:teuthology.orchestra.run.smithi164.stdout: "cct": { 2024-05-21T22:06:27.656 INFO:teuthology.orchestra.run.smithi164.stdout: "total_workers": 1, 2024-05-21T22:06:27.656 INFO:teuthology.orchestra.run.smithi164.stdout: "unhealthy_workers": 0 2024-05-21T22:06:27.656 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:27.656 INFO:teuthology.orchestra.run.smithi164.stdout: "finisher-MDSRank": { 2024-05-21T22:06:27.656 INFO:teuthology.orchestra.run.smithi164.stdout: "queue_len": 0, 2024-05-21T22:06:27.656 INFO:teuthology.orchestra.run.smithi164.stdout: "complete_latency": { 2024-05-21T22:06:27.656 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 53, 2024-05-21T22:06:27.656 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.001695979, 2024-05-21T22:06:27.657 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000031999 2024-05-21T22:06:27.657 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-21T22:06:27.657 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:27.657 INFO:teuthology.orchestra.run.smithi164.stdout: "finisher-PurgeQueue": { 2024-05-21T22:06:27.657 INFO:teuthology.orchestra.run.smithi164.stdout: "queue_len": 0, 2024-05-21T22:06:27.657 INFO:teuthology.orchestra.run.smithi164.stdout: "complete_latency": { 2024-05-21T22:06:27.657 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 2, 2024-05-21T22:06:27.657 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000009230, 2024-05-21T22:06:27.657 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000004615 2024-05-21T22:06:27.657 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-21T22:06:27.657 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:27.658 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": { 2024-05-21T22:06:27.658 INFO:teuthology.orchestra.run.smithi164.stdout: "request": 3, 2024-05-21T22:06:27.658 INFO:teuthology.orchestra.run.smithi164.stdout: "reply": 2, 2024-05-21T22:06:27.658 INFO:teuthology.orchestra.run.smithi164.stdout: "reply_latency": { 2024-05-21T22:06:27.658 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 2, 2024-05-21T22:06:27.658 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000360003, 2024-05-21T22:06:27.658 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000180001 2024-05-21T22:06:27.658 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:27.658 INFO:teuthology.orchestra.run.smithi164.stdout: "slow_reply": 0, 2024-05-21T22:06:27.658 INFO:teuthology.orchestra.run.smithi164.stdout: "forward": 1, 2024-05-21T22:06:27.658 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_fetch_complete": 0, 2024-05-21T22:06:27.659 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_fetch_keys": 0, 2024-05-21T22:06:27.659 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_commit": 11, 2024-05-21T22:06:27.659 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_split": 0, 2024-05-21T22:06:27.659 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_merge": 0, 2024-05-21T22:06:27.659 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes": 16, 2024-05-21T22:06:27.659 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_top": 0, 2024-05-21T22:06:27.659 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_bottom": 16, 2024-05-21T22:06:27.659 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_pin_tail": 0, 2024-05-21T22:06:27.659 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_pinned": 13, 2024-05-21T22:06:27.659 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_expired": 4, 2024-05-21T22:06:27.659 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_with_caps": 0, 2024-05-21T22:06:27.659 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": 0, 2024-05-21T22:06:27.659 INFO:teuthology.orchestra.run.smithi164.stdout: "subtrees": 3, 2024-05-21T22:06:27.660 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse": 18, 2024-05-21T22:06:27.660 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_hit": 13, 2024-05-21T22:06:27.660 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_forward": 1, 2024-05-21T22:06:27.660 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_discover": 4, 2024-05-21T22:06:27.660 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_dir_fetch": 0, 2024-05-21T22:06:27.660 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_remote_ino": 0, 2024-05-21T22:06:27.660 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_lock": 0, 2024-05-21T22:06:27.660 INFO:teuthology.orchestra.run.smithi164.stdout: "load_cent": 0, 2024-05-21T22:06:27.660 INFO:teuthology.orchestra.run.smithi164.stdout: "q": 0, 2024-05-21T22:06:27.660 INFO:teuthology.orchestra.run.smithi164.stdout: "exported": 0, 2024-05-21T22:06:27.660 INFO:teuthology.orchestra.run.smithi164.stdout: "exported_inodes": 0, 2024-05-21T22:06:27.661 INFO:teuthology.orchestra.run.smithi164.stdout: "imported": 0, 2024-05-21T22:06:27.661 INFO:teuthology.orchestra.run.smithi164.stdout: "imported_inodes": 0, 2024-05-21T22:06:27.661 INFO:teuthology.orchestra.run.smithi164.stdout: "openino_dir_fetch": 0, 2024-05-21T22:06:27.661 INFO:teuthology.orchestra.run.smithi164.stdout: "openino_backtrace_fetch": 0, 2024-05-21T22:06:27.661 INFO:teuthology.orchestra.run.smithi164.stdout: "openino_peer_discover": 0, 2024-05-21T22:06:27.661 INFO:teuthology.orchestra.run.smithi164.stdout: "root_rfiles": 4, 2024-05-21T22:06:27.661 INFO:teuthology.orchestra.run.smithi164.stdout: "root_rbytes": 234, 2024-05-21T22:06:27.661 INFO:teuthology.orchestra.run.smithi164.stdout: "root_rsnaps": 7, 2024-05-21T22:06:27.661 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_backtrace_fetch": 0, 2024-05-21T22:06:27.661 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_set_tag": 0, 2024-05-21T22:06:27.661 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_backtrace_repaired": 0, 2024-05-21T22:06:27.661 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_inotable_repaired": 0, 2024-05-21T22:06:27.662 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_dir_inodes": 0, 2024-05-21T22:06:27.662 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_dir_base_inodes": 0, 2024-05-21T22:06:27.662 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_dirfrag_rstats": 0, 2024-05-21T22:06:27.662 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_file_inodes": 0, 2024-05-21T22:06:27.662 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_inode_file_caps": 0, 2024-05-21T22:06:27.662 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_revoke": 0, 2024-05-21T22:06:27.662 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_grant": 0, 2024-05-21T22:06:27.662 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_trunc": 0, 2024-05-21T22:06:27.662 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-21T22:06:27.662 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-21T22:06:27.663 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_caps": 0, 2024-05-21T22:06:27.663 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_caps_dirty": 0, 2024-05-21T22:06:27.663 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_cap_release": 0, 2024-05-21T22:06:27.663 INFO:teuthology.orchestra.run.smithi164.stdout: "process_request_cap_release": 0 2024-05-21T22:06:27.663 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:27.663 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_cache": { 2024-05-21T22:06:27.663 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_update": 0, 2024-05-21T22:06:27.663 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_update_receipt": 14, 2024-05-21T22:06:27.663 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_try_discover": 9, 2024-05-21T22:06:27.663 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_send_discover": 10, 2024-05-21T22:06:27.663 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_handle_discover": 0, 2024-05-21T22:06:27.664 INFO:teuthology.orchestra.run.smithi164.stdout: "num_strays": 0, 2024-05-21T22:06:27.664 INFO:teuthology.orchestra.run.smithi164.stdout: "num_strays_delayed": 0, 2024-05-21T22:06:27.664 INFO:teuthology.orchestra.run.smithi164.stdout: "num_strays_enqueuing": 0, 2024-05-21T22:06:27.664 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_created": 0, 2024-05-21T22:06:27.664 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_enqueued": 0, 2024-05-21T22:06:27.664 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_reintegrated": 0, 2024-05-21T22:06:27.664 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_migrated": 0, 2024-05-21T22:06:27.664 INFO:teuthology.orchestra.run.smithi164.stdout: "num_recovering_processing": 0, 2024-05-21T22:06:27.664 INFO:teuthology.orchestra.run.smithi164.stdout: "num_recovering_enqueued": 0, 2024-05-21T22:06:27.664 INFO:teuthology.orchestra.run.smithi164.stdout: "num_recovering_prioritized": 0, 2024-05-21T22:06:27.665 INFO:teuthology.orchestra.run.smithi164.stdout: "recovery_started": 0, 2024-05-21T22:06:27.665 INFO:teuthology.orchestra.run.smithi164.stdout: "recovery_completed": 0, 2024-05-21T22:06:27.665 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_quiesce_path": 2, 2024-05-21T22:06:27.665 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_quiesce_inode": 11, 2024-05-21T22:06:27.665 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_enqueue_scrub": 0, 2024-05-21T22:06:27.665 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_exportdir": 0, 2024-05-21T22:06:27.665 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_flush": 0, 2024-05-21T22:06:27.665 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_fragmentdir": 0, 2024-05-21T22:06:27.665 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_fragstats": 0, 2024-05-21T22:06:27.665 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_inodestats": 0 2024-05-21T22:06:27.665 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:27.665 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_log": { 2024-05-21T22:06:27.665 INFO:teuthology.orchestra.run.smithi164.stdout: "evlrg": 0, 2024-05-21T22:06:27.666 INFO:teuthology.orchestra.run.smithi164.stdout: "evadd": 4, 2024-05-21T22:06:27.666 INFO:teuthology.orchestra.run.smithi164.stdout: "evex": 0, 2024-05-21T22:06:27.666 INFO:teuthology.orchestra.run.smithi164.stdout: "evtrm": 0, 2024-05-21T22:06:27.666 INFO:teuthology.orchestra.run.smithi164.stdout: "ev": 4, 2024-05-21T22:06:27.666 INFO:teuthology.orchestra.run.smithi164.stdout: "evexg": 0, 2024-05-21T22:06:27.666 INFO:teuthology.orchestra.run.smithi164.stdout: "evexd": 0, 2024-05-21T22:06:27.666 INFO:teuthology.orchestra.run.smithi164.stdout: "segadd": 2, 2024-05-21T22:06:27.666 INFO:teuthology.orchestra.run.smithi164.stdout: "segex": 0, 2024-05-21T22:06:27.666 INFO:teuthology.orchestra.run.smithi164.stdout: "segtrm": 0, 2024-05-21T22:06:27.666 INFO:teuthology.orchestra.run.smithi164.stdout: "seg": 2, 2024-05-21T22:06:27.666 INFO:teuthology.orchestra.run.smithi164.stdout: "segmjr": 2, 2024-05-21T22:06:27.667 INFO:teuthology.orchestra.run.smithi164.stdout: "segexg": 0, 2024-05-21T22:06:27.667 INFO:teuthology.orchestra.run.smithi164.stdout: "segexd": 0, 2024-05-21T22:06:27.667 INFO:teuthology.orchestra.run.smithi164.stdout: "expos": 4194304, 2024-05-21T22:06:27.667 INFO:teuthology.orchestra.run.smithi164.stdout: "wrpos": 4195424, 2024-05-21T22:06:27.667 INFO:teuthology.orchestra.run.smithi164.stdout: "rdpos": 0, 2024-05-21T22:06:27.667 INFO:teuthology.orchestra.run.smithi164.stdout: "jlat": { 2024-05-21T22:06:27.667 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 3, 2024-05-21T22:06:27.669 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.009019000, 2024-05-21T22:06:27.669 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.003006333 2024-05-21T22:06:27.669 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:27.670 INFO:teuthology.orchestra.run.smithi164.stdout: "replayed": 0 2024-05-21T22:06:27.670 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:27.670 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_mem": { 2024-05-21T22:06:27.670 INFO:teuthology.orchestra.run.smithi164.stdout: "ino": 20, 2024-05-21T22:06:27.670 INFO:teuthology.orchestra.run.smithi164.stdout: "ino+": 22, 2024-05-21T22:06:27.670 INFO:teuthology.orchestra.run.smithi164.stdout: "ino-": 2, 2024-05-21T22:06:27.670 INFO:teuthology.orchestra.run.smithi164.stdout: "dir": 19, 2024-05-21T22:06:27.670 INFO:teuthology.orchestra.run.smithi164.stdout: "dir+": 23, 2024-05-21T22:06:27.670 INFO:teuthology.orchestra.run.smithi164.stdout: "dir-": 4, 2024-05-21T22:06:27.670 INFO:teuthology.orchestra.run.smithi164.stdout: "dn": 16, 2024-05-21T22:06:27.671 INFO:teuthology.orchestra.run.smithi164.stdout: "dn+": 20, 2024-05-21T22:06:27.671 INFO:teuthology.orchestra.run.smithi164.stdout: "dn-": 4, 2024-05-21T22:06:27.671 INFO:teuthology.orchestra.run.smithi164.stdout: "cap": 0, 2024-05-21T22:06:27.671 INFO:teuthology.orchestra.run.smithi164.stdout: "cap+": 2, 2024-05-21T22:06:27.671 INFO:teuthology.orchestra.run.smithi164.stdout: "cap-": 2, 2024-05-21T22:06:27.671 INFO:teuthology.orchestra.run.smithi164.stdout: "rss": 52808, 2024-05-21T22:06:27.671 INFO:teuthology.orchestra.run.smithi164.stdout: "heap": 223608 2024-05-21T22:06:27.671 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:27.671 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_server": { 2024-05-21T22:06:27.671 INFO:teuthology.orchestra.run.smithi164.stdout: "dispatch_client_request": 3, 2024-05-21T22:06:27.671 INFO:teuthology.orchestra.run.smithi164.stdout: "dispatch_server_request": 0, 2024-05-21T22:06:27.671 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_request": 3, 2024-05-21T22:06:27.672 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_session": 4, 2024-05-21T22:06:27.672 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_peer_request": 11, 2024-05-21T22:06:27.672 INFO:teuthology.orchestra.run.smithi164.stdout: "req_create_latency": { 2024-05-21T22:06:27.672 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:27.672 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:27.672 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:27.672 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:27.672 INFO:teuthology.orchestra.run.smithi164.stdout: "req_getattr_latency": { 2024-05-21T22:06:27.672 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 1, 2024-05-21T22:06:27.673 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000189734, 2024-05-21T22:06:27.673 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000189734 2024-05-21T22:06:27.673 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:27.673 INFO:teuthology.orchestra.run.smithi164.stdout: "req_getfilelock_latency": { 2024-05-21T22:06:27.673 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:27.673 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:27.673 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:27.673 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:27.674 INFO:teuthology.orchestra.run.smithi164.stdout: "req_link_latency": { 2024-05-21T22:06:27.674 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:27.674 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:27.674 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:27.674 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:27.674 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookup_latency": { 2024-05-21T22:06:27.674 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 1, 2024-05-21T22:06:27.674 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000170269, 2024-05-21T22:06:27.674 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000170269 2024-05-21T22:06:27.674 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:27.674 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookuphash_latency": { 2024-05-21T22:06:27.675 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:27.675 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:27.675 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:27.675 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:27.675 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupino_latency": { 2024-05-21T22:06:27.675 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:27.675 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:27.675 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:27.675 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:27.675 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupname_latency": { 2024-05-21T22:06:27.676 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:27.676 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:27.676 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:27.676 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:27.676 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupparent_latency": { 2024-05-21T22:06:27.676 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:27.676 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:27.676 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:27.676 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:27.676 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupsnap_latency": { 2024-05-21T22:06:27.676 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:27.676 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:27.677 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:27.677 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:27.677 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lssnap_latency": { 2024-05-21T22:06:27.677 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:27.677 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:27.677 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:27.677 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:27.677 INFO:teuthology.orchestra.run.smithi164.stdout: "req_mkdir_latency": { 2024-05-21T22:06:27.678 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:27.678 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:27.678 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:27.678 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:27.679 INFO:teuthology.orchestra.run.smithi164.stdout: "req_mknod_latency": { 2024-05-21T22:06:27.679 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:27.679 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:27.679 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:27.679 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:27.679 INFO:teuthology.orchestra.run.smithi164.stdout: "req_mksnap_latency": { 2024-05-21T22:06:27.679 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:27.679 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:27.679 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:27.679 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:27.679 INFO:teuthology.orchestra.run.smithi164.stdout: "req_open_latency": { 2024-05-21T22:06:27.679 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:27.680 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:27.680 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:27.680 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:27.680 INFO:teuthology.orchestra.run.smithi164.stdout: "req_readdir_latency": { 2024-05-21T22:06:27.680 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:27.680 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:27.680 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:27.680 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:27.680 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rename_latency": { 2024-05-21T22:06:27.680 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:27.680 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:27.681 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:27.681 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:27.681 INFO:teuthology.orchestra.run.smithi164.stdout: "req_renamesnap_latency": { 2024-05-21T22:06:27.681 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:27.681 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:27.681 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:27.681 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:27.681 INFO:teuthology.orchestra.run.smithi164.stdout: "req_snapdiff_latency": { 2024-05-21T22:06:27.681 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:27.681 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:27.681 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:27.681 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:27.682 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rmdir_latency": { 2024-05-21T22:06:27.682 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:27.682 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:27.682 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:27.682 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:27.682 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rmsnap_latency": { 2024-05-21T22:06:27.682 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:27.682 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:27.682 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:27.682 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:27.682 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rmxattr_latency": { 2024-05-21T22:06:27.683 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:27.683 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:27.683 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:27.683 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:27.683 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setattr_latency": { 2024-05-21T22:06:27.683 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:27.683 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:27.683 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:27.683 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:27.683 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setdirlayout_latency": { 2024-05-21T22:06:27.683 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:27.684 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:27.684 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:27.684 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:27.684 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setfilelock_latency": { 2024-05-21T22:06:27.684 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:27.684 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:27.684 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:27.684 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:27.684 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setlayout_latency": { 2024-05-21T22:06:27.684 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:27.684 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:27.684 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:27.684 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:27.685 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setxattr_latency": { 2024-05-21T22:06:27.685 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:27.685 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:27.685 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:27.685 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:27.685 INFO:teuthology.orchestra.run.smithi164.stdout: "req_symlink_latency": { 2024-05-21T22:06:27.685 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:27.685 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:27.685 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:27.685 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:27.685 INFO:teuthology.orchestra.run.smithi164.stdout: "req_unlink_latency": { 2024-05-21T22:06:27.686 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:27.686 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:27.686 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:27.686 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:27.686 INFO:teuthology.orchestra.run.smithi164.stdout: "cap_revoke_eviction": 0, 2024-05-21T22:06:27.686 INFO:teuthology.orchestra.run.smithi164.stdout: "cap_acquisition_throttle": 0, 2024-05-21T22:06:27.686 INFO:teuthology.orchestra.run.smithi164.stdout: "req_getvxattr_latency": { 2024-05-21T22:06:27.686 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:27.686 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:27.686 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:27.687 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-21T22:06:27.687 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:27.687 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_sessions": { 2024-05-21T22:06:27.687 INFO:teuthology.orchestra.run.smithi164.stdout: "session_count": 0, 2024-05-21T22:06:27.687 INFO:teuthology.orchestra.run.smithi164.stdout: "session_add": 1, 2024-05-21T22:06:27.687 INFO:teuthology.orchestra.run.smithi164.stdout: "session_remove": 1, 2024-05-21T22:06:27.687 INFO:teuthology.orchestra.run.smithi164.stdout: "sessions_open": 0, 2024-05-21T22:06:27.687 INFO:teuthology.orchestra.run.smithi164.stdout: "sessions_stale": 0, 2024-05-21T22:06:27.687 INFO:teuthology.orchestra.run.smithi164.stdout: "total_load": 1, 2024-05-21T22:06:27.687 INFO:teuthology.orchestra.run.smithi164.stdout: "average_load": 1, 2024-05-21T22:06:27.687 INFO:teuthology.orchestra.run.smithi164.stdout: "avg_session_uptime": 88, 2024-05-21T22:06:27.688 INFO:teuthology.orchestra.run.smithi164.stdout: "mdthresh_evicted": 0 2024-05-21T22:06:27.688 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:27.688 INFO:teuthology.orchestra.run.smithi164.stdout: "mempool": { 2024-05-21T22:06:27.688 INFO:teuthology.orchestra.run.smithi164.stdout: "bloom_filter_bytes": 0, 2024-05-21T22:06:27.688 INFO:teuthology.orchestra.run.smithi164.stdout: "bloom_filter_items": 0, 2024-05-21T22:06:27.688 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_alloc_bytes": 0, 2024-05-21T22:06:27.689 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_alloc_items": 0, 2024-05-21T22:06:27.689 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_data_bytes": 0, 2024-05-21T22:06:27.689 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_data_items": 0, 2024-05-21T22:06:27.689 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-21T22:06:27.689 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_onode_items": 0, 2024-05-21T22:06:27.690 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-21T22:06:27.690 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_meta_items": 0, 2024-05-21T22:06:27.690 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_other_bytes": 0, 2024-05-21T22:06:27.690 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_other_items": 0, 2024-05-21T22:06:27.690 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-21T22:06:27.691 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_buffer_items": 0, 2024-05-21T22:06:27.691 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_extent_bytes": 0, 2024-05-21T22:06:27.691 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_extent_items": 0, 2024-05-21T22:06:27.691 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_blob_bytes": 0, 2024-05-21T22:06:27.691 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_blob_items": 0, 2024-05-21T22:06:27.692 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-21T22:06:27.692 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_shared_blob_items": 0, 2024-05-21T22:06:27.692 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-21T22:06:27.692 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_inline_bl_items": 0, 2024-05-21T22:06:27.692 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_fsck_bytes": 0, 2024-05-21T22:06:27.692 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_fsck_items": 0, 2024-05-21T22:06:27.693 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_txc_bytes": 0, 2024-05-21T22:06:27.693 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_txc_items": 0, 2024-05-21T22:06:27.693 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-21T22:06:27.693 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_deferred_items": 0, 2024-05-21T22:06:27.693 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_bytes": 0, 2024-05-21T22:06:27.694 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_items": 0, 2024-05-21T22:06:27.694 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_bytes": 0, 2024-05-21T22:06:27.694 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_items": 0, 2024-05-21T22:06:27.694 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_reader_bytes": 0, 2024-05-21T22:06:27.695 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_reader_items": 0, 2024-05-21T22:06:27.695 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_writer_bytes": 0, 2024-05-21T22:06:27.695 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_writer_items": 0, 2024-05-21T22:06:27.695 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_anon_bytes": 263639, 2024-05-21T22:06:27.695 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_anon_items": 91, 2024-05-21T22:06:27.696 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_meta_bytes": 0, 2024-05-21T22:06:27.696 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_meta_items": 0, 2024-05-21T22:06:27.696 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_bytes": 0, 2024-05-21T22:06:27.696 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_items": 0, 2024-05-21T22:06:27.696 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_mapbl_bytes": 0, 2024-05-21T22:06:27.697 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_mapbl_items": 0, 2024-05-21T22:06:27.697 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_pglog_bytes": 0, 2024-05-21T22:06:27.697 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_pglog_items": 0, 2024-05-21T22:06:27.697 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_bytes": 8128, 2024-05-21T22:06:27.697 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_items": 169, 2024-05-21T22:06:27.697 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_mapping_bytes": 0, 2024-05-21T22:06:27.697 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_mapping_items": 0, 2024-05-21T22:06:27.697 INFO:teuthology.orchestra.run.smithi164.stdout: "pgmap_bytes": 0, 2024-05-21T22:06:27.697 INFO:teuthology.orchestra.run.smithi164.stdout: "pgmap_items": 0, 2024-05-21T22:06:27.697 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_co_bytes": 82717, 2024-05-21T22:06:27.698 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_co_items": 340, 2024-05-21T22:06:27.698 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_1_bytes": 0, 2024-05-21T22:06:27.698 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_1_items": 0, 2024-05-21T22:06:27.698 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_2_bytes": 0, 2024-05-21T22:06:27.698 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_2_items": 0 2024-05-21T22:06:27.698 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:27.698 INFO:teuthology.orchestra.run.smithi164.stdout: "objecter": { 2024-05-21T22:06:27.698 INFO:teuthology.orchestra.run.smithi164.stdout: "op_active": 0, 2024-05-21T22:06:27.698 INFO:teuthology.orchestra.run.smithi164.stdout: "op_laggy": 0, 2024-05-21T22:06:27.698 INFO:teuthology.orchestra.run.smithi164.stdout: "op_send": 41, 2024-05-21T22:06:27.698 INFO:teuthology.orchestra.run.smithi164.stdout: "op_send_bytes": 11567, 2024-05-21T22:06:27.699 INFO:teuthology.orchestra.run.smithi164.stdout: "op_resend": 33, 2024-05-21T22:06:27.699 INFO:teuthology.orchestra.run.smithi164.stdout: "op_reply": 41, 2024-05-21T22:06:27.699 INFO:teuthology.orchestra.run.smithi164.stdout: "op_latency": { 2024-05-21T22:06:27.699 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 41, 2024-05-21T22:06:27.699 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.293000150, 2024-05-21T22:06:27.699 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.007146345 2024-05-21T22:06:27.699 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:27.699 INFO:teuthology.orchestra.run.smithi164.stdout: "op_inflight": 0, 2024-05-21T22:06:27.699 INFO:teuthology.orchestra.run.smithi164.stdout: "oplen_avg": { 2024-05-21T22:06:27.699 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 41, 2024-05-21T22:06:27.699 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 65 2024-05-21T22:06:27.699 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:27.700 INFO:teuthology.orchestra.run.smithi164.stdout: "op": 41, 2024-05-21T22:06:27.700 INFO:teuthology.orchestra.run.smithi164.stdout: "op_r": 0, 2024-05-21T22:06:27.700 INFO:teuthology.orchestra.run.smithi164.stdout: "op_w": 41, 2024-05-21T22:06:27.700 INFO:teuthology.orchestra.run.smithi164.stdout: "op_rmw": 0, 2024-05-21T22:06:27.700 INFO:teuthology.orchestra.run.smithi164.stdout: "op_pg": 0, 2024-05-21T22:06:27.700 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_stat": 0, 2024-05-21T22:06:27.700 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_create": 10, 2024-05-21T22:06:27.700 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_read": 0, 2024-05-21T22:06:27.700 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_write": 3, 2024-05-21T22:06:27.700 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_writefull": 7, 2024-05-21T22:06:27.700 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_writesame": 0, 2024-05-21T22:06:27.700 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_append": 0, 2024-05-21T22:06:27.700 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_zero": 0, 2024-05-21T22:06:27.701 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_truncate": 0, 2024-05-21T22:06:27.701 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_delete": 6, 2024-05-21T22:06:27.701 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_mapext": 0, 2024-05-21T22:06:27.701 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_sparse_read": 0, 2024-05-21T22:06:27.701 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_clonerange": 0, 2024-05-21T22:06:27.701 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_getxattr": 0, 2024-05-21T22:06:27.701 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_setxattr": 20, 2024-05-21T22:06:27.701 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_cmpxattr": 0, 2024-05-21T22:06:27.701 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_rmxattr": 0, 2024-05-21T22:06:27.701 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_resetxattrs": 0, 2024-05-21T22:06:27.701 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_call": 0, 2024-05-21T22:06:27.701 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_watch": 0, 2024-05-21T22:06:27.701 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_notify": 0, 2024-05-21T22:06:27.702 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_src_cmpxattr": 0, 2024-05-21T22:06:27.702 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_pgls": 0, 2024-05-21T22:06:27.702 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_pgls_filter": 0, 2024-05-21T22:06:27.702 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_other": 0, 2024-05-21T22:06:27.702 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_active": 0, 2024-05-21T22:06:27.702 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_send": 0, 2024-05-21T22:06:27.702 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_resend": 0, 2024-05-21T22:06:27.702 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_ping": 0, 2024-05-21T22:06:27.702 INFO:teuthology.orchestra.run.smithi164.stdout: "poolop_active": 0, 2024-05-21T22:06:27.702 INFO:teuthology.orchestra.run.smithi164.stdout: "poolop_send": 0, 2024-05-21T22:06:27.702 INFO:teuthology.orchestra.run.smithi164.stdout: "poolop_resend": 0, 2024-05-21T22:06:27.703 INFO:teuthology.orchestra.run.smithi164.stdout: "poolstat_active": 0, 2024-05-21T22:06:27.703 INFO:teuthology.orchestra.run.smithi164.stdout: "poolstat_send": 0, 2024-05-21T22:06:27.703 INFO:teuthology.orchestra.run.smithi164.stdout: "poolstat_resend": 0, 2024-05-21T22:06:27.703 INFO:teuthology.orchestra.run.smithi164.stdout: "statfs_active": 0, 2024-05-21T22:06:27.703 INFO:teuthology.orchestra.run.smithi164.stdout: "statfs_send": 0, 2024-05-21T22:06:27.703 INFO:teuthology.orchestra.run.smithi164.stdout: "statfs_resend": 0, 2024-05-21T22:06:27.703 INFO:teuthology.orchestra.run.smithi164.stdout: "command_active": 0, 2024-05-21T22:06:27.703 INFO:teuthology.orchestra.run.smithi164.stdout: "command_send": 0, 2024-05-21T22:06:27.703 INFO:teuthology.orchestra.run.smithi164.stdout: "command_resend": 0, 2024-05-21T22:06:27.703 INFO:teuthology.orchestra.run.smithi164.stdout: "map_epoch": 91, 2024-05-21T22:06:27.703 INFO:teuthology.orchestra.run.smithi164.stdout: "map_full": 0, 2024-05-21T22:06:27.703 INFO:teuthology.orchestra.run.smithi164.stdout: "map_inc": 11, 2024-05-21T22:06:27.703 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_sessions": 10, 2024-05-21T22:06:27.703 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_session_open": 10, 2024-05-21T22:06:27.704 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_session_close": 0, 2024-05-21T22:06:27.704 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_laggy": 0, 2024-05-21T22:06:27.704 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_wr": 17, 2024-05-21T22:06:27.704 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_rd": 0, 2024-05-21T22:06:27.704 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_del": 2 2024-05-21T22:06:27.704 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:27.704 INFO:teuthology.orchestra.run.smithi164.stdout: "oft": { 2024-05-21T22:06:27.704 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_objs": 1, 2024-05-21T22:06:27.704 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_kv_pairs": 0, 2024-05-21T22:06:27.704 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_updates": 0, 2024-05-21T22:06:27.704 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_removes": 0 2024-05-21T22:06:27.704 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:27.704 INFO:teuthology.orchestra.run.smithi164.stdout: "osd-slow-ops": { 2024-05-21T22:06:27.705 INFO:teuthology.orchestra.run.smithi164.stdout: "slow_ops_count": 13 2024-05-21T22:06:27.705 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:27.705 INFO:teuthology.orchestra.run.smithi164.stdout: "purge_queue": { 2024-05-21T22:06:27.705 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing_ops": 0, 2024-05-21T22:06:27.705 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing_ops_high_water": 0, 2024-05-21T22:06:27.705 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing": 0, 2024-05-21T22:06:27.705 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing_high_water": 0, 2024-05-21T22:06:27.705 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executed_ops": 0, 2024-05-21T22:06:27.705 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executed": 0, 2024-05-21T22:06:27.705 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_item_in_journal": 0 2024-05-21T22:06:27.705 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:27.705 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-21T22:06:27.705 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-05-21T22:06:27.705 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 104857600, 2024-05-21T22:06:27.706 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-05-21T22:06:27.706 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 1169, 2024-05-21T22:06:27.706 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 2060920, 2024-05-21T22:06:27.706 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-05-21T22:06:27.706 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 1169, 2024-05-21T22:06:27.706 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 0, 2024-05-21T22:06:27.706 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 0, 2024-05-21T22:06:27.706 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 1169, 2024-05-21T22:06:27.706 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 2060920, 2024-05-21T22:06:27.706 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-05-21T22:06:27.706 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:27.706 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:27.707 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:27.707 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-21T22:06:27.707 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:27.707 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-objecter_bytes": { 2024-05-21T22:06:27.707 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-05-21T22:06:27.707 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 104857600, 2024-05-21T22:06:27.707 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-05-21T22:06:27.707 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 0, 2024-05-21T22:06:27.707 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 0, 2024-05-21T22:06:27.707 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-05-21T22:06:27.707 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 0, 2024-05-21T22:06:27.707 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 41, 2024-05-21T22:06:27.707 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 11567, 2024-05-21T22:06:27.707 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 41, 2024-05-21T22:06:27.708 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 11567, 2024-05-21T22:06:27.708 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-05-21T22:06:27.708 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:27.708 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:27.708 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:27.708 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-21T22:06:27.708 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:27.708 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-objecter_ops": { 2024-05-21T22:06:27.708 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-05-21T22:06:27.708 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 1024, 2024-05-21T22:06:27.708 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-05-21T22:06:27.708 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 0, 2024-05-21T22:06:27.708 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 0, 2024-05-21T22:06:27.709 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-05-21T22:06:27.709 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 0, 2024-05-21T22:06:27.709 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 41, 2024-05-21T22:06:27.709 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 41, 2024-05-21T22:06:27.709 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 41, 2024-05-21T22:06:27.709 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 41, 2024-05-21T22:06:27.709 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-05-21T22:06:27.709 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:27.709 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:27.709 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:27.709 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-21T22:06:27.709 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:27.709 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-write_buf_throttle": { 2024-05-21T22:06:27.709 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-05-21T22:06:27.710 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 3758096384, 2024-05-21T22:06:27.710 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-05-21T22:06:27.710 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 0, 2024-05-21T22:06:27.710 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 0, 2024-05-21T22:06:27.710 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-05-21T22:06:27.710 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 0, 2024-05-21T22:06:27.710 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 0, 2024-05-21T22:06:27.710 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 0, 2024-05-21T22:06:27.710 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 0, 2024-05-21T22:06:27.710 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 0, 2024-05-21T22:06:27.710 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-05-21T22:06:27.711 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:27.711 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:27.711 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:27.711 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-21T22:06:27.711 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:27.711 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-write_buf_throttle-0x55ca90c0e500": { 2024-05-21T22:06:27.711 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-05-21T22:06:27.711 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 3758096384, 2024-05-21T22:06:27.711 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-05-21T22:06:27.711 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 4, 2024-05-21T22:06:27.712 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 1120, 2024-05-21T22:06:27.712 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-05-21T22:06:27.712 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 4, 2024-05-21T22:06:27.712 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 0, 2024-05-21T22:06:27.712 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 0, 2024-05-21T22:06:27.712 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 3, 2024-05-21T22:06:27.712 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 1120, 2024-05-21T22:06:27.712 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-05-21T22:06:27.712 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:27.712 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:27.712 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:27.712 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-21T22:06:27.713 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-21T22:06:27.713 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-05-21T22:06:27.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:06:27 smithi164 ceph-mon[36269]: pgmap v336: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-05-21T22:06:27.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:06:27 smithi151 ceph-mon[31623]: pgmap v336: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-05-21T22:06:27.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:06:27 smithi184 ceph-mon[36942]: pgmap v336: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-05-21T22:06:28.332 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-05-21T22:06:28.332 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-05-21T22:06:28.332 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-05-21T22:06:28.332 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=14 2024-05-21T22:06:28.332 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=7 2024-05-21T22:06:28.332 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-05-21T22:06:28.332 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-05-21T22:06:28.333 DEBUG:tasks.check_counter:Getting stats from h 2024-05-21T22:06:28.333 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph daemon mds.h perf dump 2024-05-21T22:06:28.570 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.b/config 2024-05-21T22:06:29.897 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:06:29 smithi164 ceph-mon[36269]: pgmap v337: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2024-05-21T22:06:29.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:06:29 smithi151 ceph-mon[31623]: pgmap v337: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2024-05-21T22:06:29.985 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:06:29 smithi184 ceph-mon[36942]: pgmap v337: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2024-05-21T22:06:30.126 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-05-21T22:06:30.126 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-0": { 2024-05-21T22:06:30.126 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 402, 2024-05-21T22:06:30.126 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 382, 2024-05-21T22:06:30.126 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 1962521, 2024-05-21T22:06:30.126 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 245272, 2024-05-21T22:06:30.127 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 14, 2024-05-21T22:06:30.127 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 1, 2024-05-21T22:06:30.127 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 0.071849327, 2024-05-21T22:06:30.127 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 0.022160595, 2024-05-21T22:06:30.127 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 0.050601186, 2024-05-21T22:06:30.127 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 0.006818534, 2024-05-21T22:06:30.127 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T22:06:30.127 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 381, 2024-05-21T22:06:30.127 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.037509027, 2024-05-21T22:06:30.127 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000098448 2024-05-21T22:06:30.127 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:30.127 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2024-05-21T22:06:30.127 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 132, 2024-05-21T22:06:30.127 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000034268, 2024-05-21T22:06:30.128 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000259 2024-05-21T22:06:30.128 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:30.128 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 1933952, 2024-05-21T22:06:30.128 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 164928 2024-05-21T22:06:30.128 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:30.128 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-1": { 2024-05-21T22:06:30.128 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 51, 2024-05-21T22:06:30.128 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 245, 2024-05-21T22:06:30.128 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 9180, 2024-05-21T22:06:30.128 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 999777, 2024-05-21T22:06:30.128 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 45, 2024-05-21T22:06:30.128 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 5, 2024-05-21T22:06:30.129 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 0.028647985, 2024-05-21T22:06:30.129 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 0.010029437, 2024-05-21T22:06:30.129 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 0.014706721, 2024-05-21T22:06:30.129 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 0.000314195, 2024-05-21T22:06:30.129 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T22:06:30.129 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 240, 2024-05-21T22:06:30.129 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.036735819, 2024-05-21T22:06:30.129 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000153065 2024-05-21T22:06:30.129 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:30.129 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2024-05-21T22:06:30.129 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 34, 2024-05-21T22:06:30.129 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000003865, 2024-05-21T22:06:30.129 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000113 2024-05-21T22:06:30.129 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:30.130 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 480, 2024-05-21T22:06:30.130 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 575264 2024-05-21T22:06:30.130 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:30.130 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-2": { 2024-05-21T22:06:30.130 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 724, 2024-05-21T22:06:30.130 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 850, 2024-05-21T22:06:30.130 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 223660, 2024-05-21T22:06:30.130 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 190484, 2024-05-21T22:06:30.130 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 8, 2024-05-21T22:06:30.130 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 5, 2024-05-21T22:06:30.130 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 0.091411760, 2024-05-21T22:06:30.130 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 0.044492031, 2024-05-21T22:06:30.131 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 0.044126705, 2024-05-21T22:06:30.131 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 0.004544694, 2024-05-21T22:06:30.131 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T22:06:30.131 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 851, 2024-05-21T22:06:30.131 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.054248901, 2024-05-21T22:06:30.131 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000063747 2024-05-21T22:06:30.131 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:30.131 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2024-05-21T22:06:30.131 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 1478, 2024-05-21T22:06:30.131 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000464334, 2024-05-21T22:06:30.131 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000314 2024-05-21T22:06:30.131 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:30.131 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-05-21T22:06:30.131 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 448 2024-05-21T22:06:30.132 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:30.132 INFO:teuthology.orchestra.run.smithi164.stdout: "cct": { 2024-05-21T22:06:30.132 INFO:teuthology.orchestra.run.smithi164.stdout: "total_workers": 1, 2024-05-21T22:06:30.132 INFO:teuthology.orchestra.run.smithi164.stdout: "unhealthy_workers": 0 2024-05-21T22:06:30.132 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:30.132 INFO:teuthology.orchestra.run.smithi164.stdout: "finisher-MDSRank": { 2024-05-21T22:06:30.132 INFO:teuthology.orchestra.run.smithi164.stdout: "queue_len": 0, 2024-05-21T22:06:30.132 INFO:teuthology.orchestra.run.smithi164.stdout: "complete_latency": { 2024-05-21T22:06:30.132 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 52, 2024-05-21T22:06:30.132 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.001477658, 2024-05-21T22:06:30.132 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000028416 2024-05-21T22:06:30.132 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-21T22:06:30.133 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:30.133 INFO:teuthology.orchestra.run.smithi164.stdout: "finisher-PurgeQueue": { 2024-05-21T22:06:30.133 INFO:teuthology.orchestra.run.smithi164.stdout: "queue_len": 0, 2024-05-21T22:06:30.133 INFO:teuthology.orchestra.run.smithi164.stdout: "complete_latency": { 2024-05-21T22:06:30.133 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 2, 2024-05-21T22:06:30.133 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000007615, 2024-05-21T22:06:30.133 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000003807 2024-05-21T22:06:30.133 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-21T22:06:30.133 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:30.133 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": { 2024-05-21T22:06:30.133 INFO:teuthology.orchestra.run.smithi164.stdout: "request": 2, 2024-05-21T22:06:30.133 INFO:teuthology.orchestra.run.smithi164.stdout: "reply": 1, 2024-05-21T22:06:30.133 INFO:teuthology.orchestra.run.smithi164.stdout: "reply_latency": { 2024-05-21T22:06:30.134 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 1, 2024-05-21T22:06:30.134 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000192867, 2024-05-21T22:06:30.134 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000192867 2024-05-21T22:06:30.134 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:30.134 INFO:teuthology.orchestra.run.smithi164.stdout: "slow_reply": 0, 2024-05-21T22:06:30.134 INFO:teuthology.orchestra.run.smithi164.stdout: "forward": 1, 2024-05-21T22:06:30.134 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_fetch_complete": 0, 2024-05-21T22:06:30.134 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_fetch_keys": 0, 2024-05-21T22:06:30.134 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_commit": 11, 2024-05-21T22:06:30.134 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_split": 0, 2024-05-21T22:06:30.134 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_merge": 0, 2024-05-21T22:06:30.134 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes": 16, 2024-05-21T22:06:30.135 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_top": 0, 2024-05-21T22:06:30.135 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_bottom": 16, 2024-05-21T22:06:30.135 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_pin_tail": 0, 2024-05-21T22:06:30.135 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_pinned": 13, 2024-05-21T22:06:30.135 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_expired": 4, 2024-05-21T22:06:30.135 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_with_caps": 0, 2024-05-21T22:06:30.135 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": 0, 2024-05-21T22:06:30.135 INFO:teuthology.orchestra.run.smithi164.stdout: "subtrees": 3, 2024-05-21T22:06:30.135 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse": 16, 2024-05-21T22:06:30.135 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_hit": 10, 2024-05-21T22:06:30.135 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_forward": 0, 2024-05-21T22:06:30.135 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_discover": 5, 2024-05-21T22:06:30.135 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_dir_fetch": 0, 2024-05-21T22:06:30.136 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_remote_ino": 0, 2024-05-21T22:06:30.136 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_lock": 0, 2024-05-21T22:06:30.136 INFO:teuthology.orchestra.run.smithi164.stdout: "load_cent": 0, 2024-05-21T22:06:30.136 INFO:teuthology.orchestra.run.smithi164.stdout: "q": 0, 2024-05-21T22:06:30.136 INFO:teuthology.orchestra.run.smithi164.stdout: "exported": 0, 2024-05-21T22:06:30.136 INFO:teuthology.orchestra.run.smithi164.stdout: "exported_inodes": 0, 2024-05-21T22:06:30.136 INFO:teuthology.orchestra.run.smithi164.stdout: "imported": 0, 2024-05-21T22:06:30.136 INFO:teuthology.orchestra.run.smithi164.stdout: "imported_inodes": 0, 2024-05-21T22:06:30.136 INFO:teuthology.orchestra.run.smithi164.stdout: "openino_dir_fetch": 0, 2024-05-21T22:06:30.136 INFO:teuthology.orchestra.run.smithi164.stdout: "openino_backtrace_fetch": 0, 2024-05-21T22:06:30.136 INFO:teuthology.orchestra.run.smithi164.stdout: "openino_peer_discover": 0, 2024-05-21T22:06:30.136 INFO:teuthology.orchestra.run.smithi164.stdout: "root_rfiles": 4, 2024-05-21T22:06:30.136 INFO:teuthology.orchestra.run.smithi164.stdout: "root_rbytes": 234, 2024-05-21T22:06:30.136 INFO:teuthology.orchestra.run.smithi164.stdout: "root_rsnaps": 7, 2024-05-21T22:06:30.137 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_backtrace_fetch": 0, 2024-05-21T22:06:30.137 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_set_tag": 0, 2024-05-21T22:06:30.137 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_backtrace_repaired": 0, 2024-05-21T22:06:30.137 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_inotable_repaired": 0, 2024-05-21T22:06:30.137 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_dir_inodes": 0, 2024-05-21T22:06:30.137 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_dir_base_inodes": 0, 2024-05-21T22:06:30.137 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_dirfrag_rstats": 0, 2024-05-21T22:06:30.137 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_file_inodes": 0, 2024-05-21T22:06:30.137 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_inode_file_caps": 0, 2024-05-21T22:06:30.137 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_revoke": 0, 2024-05-21T22:06:30.137 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_grant": 0, 2024-05-21T22:06:30.137 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_trunc": 0, 2024-05-21T22:06:30.137 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-21T22:06:30.138 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-21T22:06:30.138 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_caps": 0, 2024-05-21T22:06:30.138 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_caps_dirty": 0, 2024-05-21T22:06:30.138 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_cap_release": 0, 2024-05-21T22:06:30.138 INFO:teuthology.orchestra.run.smithi164.stdout: "process_request_cap_release": 1 2024-05-21T22:06:30.138 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:30.138 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_cache": { 2024-05-21T22:06:30.138 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_update": 0, 2024-05-21T22:06:30.138 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_update_receipt": 12, 2024-05-21T22:06:30.138 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_try_discover": 8, 2024-05-21T22:06:30.138 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_send_discover": 10, 2024-05-21T22:06:30.138 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_handle_discover": 0, 2024-05-21T22:06:30.139 INFO:teuthology.orchestra.run.smithi164.stdout: "num_strays": 0, 2024-05-21T22:06:30.139 INFO:teuthology.orchestra.run.smithi164.stdout: "num_strays_delayed": 0, 2024-05-21T22:06:30.139 INFO:teuthology.orchestra.run.smithi164.stdout: "num_strays_enqueuing": 0, 2024-05-21T22:06:30.139 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_created": 0, 2024-05-21T22:06:30.139 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_enqueued": 0, 2024-05-21T22:06:30.139 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_reintegrated": 0, 2024-05-21T22:06:30.139 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_migrated": 0, 2024-05-21T22:06:30.139 INFO:teuthology.orchestra.run.smithi164.stdout: "num_recovering_processing": 0, 2024-05-21T22:06:30.139 INFO:teuthology.orchestra.run.smithi164.stdout: "num_recovering_enqueued": 0, 2024-05-21T22:06:30.139 INFO:teuthology.orchestra.run.smithi164.stdout: "num_recovering_prioritized": 0, 2024-05-21T22:06:30.139 INFO:teuthology.orchestra.run.smithi164.stdout: "recovery_started": 0, 2024-05-21T22:06:30.139 INFO:teuthology.orchestra.run.smithi164.stdout: "recovery_completed": 0, 2024-05-21T22:06:30.139 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_quiesce_path": 2, 2024-05-21T22:06:30.139 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_quiesce_inode": 11, 2024-05-21T22:06:30.140 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_enqueue_scrub": 0, 2024-05-21T22:06:30.140 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_exportdir": 0, 2024-05-21T22:06:30.140 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_flush": 0, 2024-05-21T22:06:30.140 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_fragmentdir": 0, 2024-05-21T22:06:30.140 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_fragstats": 0, 2024-05-21T22:06:30.140 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_inodestats": 0 2024-05-21T22:06:30.140 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:30.140 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_log": { 2024-05-21T22:06:30.140 INFO:teuthology.orchestra.run.smithi164.stdout: "evlrg": 0, 2024-05-21T22:06:30.140 INFO:teuthology.orchestra.run.smithi164.stdout: "evadd": 4, 2024-05-21T22:06:30.140 INFO:teuthology.orchestra.run.smithi164.stdout: "evex": 0, 2024-05-21T22:06:30.140 INFO:teuthology.orchestra.run.smithi164.stdout: "evtrm": 0, 2024-05-21T22:06:30.140 INFO:teuthology.orchestra.run.smithi164.stdout: "ev": 4, 2024-05-21T22:06:30.141 INFO:teuthology.orchestra.run.smithi164.stdout: "evexg": 0, 2024-05-21T22:06:30.141 INFO:teuthology.orchestra.run.smithi164.stdout: "evexd": 0, 2024-05-21T22:06:30.141 INFO:teuthology.orchestra.run.smithi164.stdout: "segadd": 2, 2024-05-21T22:06:30.141 INFO:teuthology.orchestra.run.smithi164.stdout: "segex": 0, 2024-05-21T22:06:30.141 INFO:teuthology.orchestra.run.smithi164.stdout: "segtrm": 0, 2024-05-21T22:06:30.141 INFO:teuthology.orchestra.run.smithi164.stdout: "seg": 2, 2024-05-21T22:06:30.141 INFO:teuthology.orchestra.run.smithi164.stdout: "segmjr": 2, 2024-05-21T22:06:30.141 INFO:teuthology.orchestra.run.smithi164.stdout: "segexg": 0, 2024-05-21T22:06:30.141 INFO:teuthology.orchestra.run.smithi164.stdout: "segexd": 0, 2024-05-21T22:06:30.141 INFO:teuthology.orchestra.run.smithi164.stdout: "expos": 4194304, 2024-05-21T22:06:30.141 INFO:teuthology.orchestra.run.smithi164.stdout: "wrpos": 4195424, 2024-05-21T22:06:30.141 INFO:teuthology.orchestra.run.smithi164.stdout: "rdpos": 0, 2024-05-21T22:06:30.141 INFO:teuthology.orchestra.run.smithi164.stdout: "jlat": { 2024-05-21T22:06:30.141 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 3, 2024-05-21T22:06:30.142 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.006012338, 2024-05-21T22:06:30.142 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.002004112 2024-05-21T22:06:30.142 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:30.142 INFO:teuthology.orchestra.run.smithi164.stdout: "replayed": 0 2024-05-21T22:06:30.142 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:30.142 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_mem": { 2024-05-21T22:06:30.142 INFO:teuthology.orchestra.run.smithi164.stdout: "ino": 20, 2024-05-21T22:06:30.142 INFO:teuthology.orchestra.run.smithi164.stdout: "ino+": 22, 2024-05-21T22:06:30.142 INFO:teuthology.orchestra.run.smithi164.stdout: "ino-": 2, 2024-05-21T22:06:30.142 INFO:teuthology.orchestra.run.smithi164.stdout: "dir": 19, 2024-05-21T22:06:30.142 INFO:teuthology.orchestra.run.smithi164.stdout: "dir+": 22, 2024-05-21T22:06:30.142 INFO:teuthology.orchestra.run.smithi164.stdout: "dir-": 3, 2024-05-21T22:06:30.142 INFO:teuthology.orchestra.run.smithi164.stdout: "dn": 16, 2024-05-21T22:06:30.142 INFO:teuthology.orchestra.run.smithi164.stdout: "dn+": 20, 2024-05-21T22:06:30.143 INFO:teuthology.orchestra.run.smithi164.stdout: "dn-": 4, 2024-05-21T22:06:30.143 INFO:teuthology.orchestra.run.smithi164.stdout: "cap": 0, 2024-05-21T22:06:30.143 INFO:teuthology.orchestra.run.smithi164.stdout: "cap+": 1, 2024-05-21T22:06:30.143 INFO:teuthology.orchestra.run.smithi164.stdout: "cap-": 1, 2024-05-21T22:06:30.143 INFO:teuthology.orchestra.run.smithi164.stdout: "rss": 55460, 2024-05-21T22:06:30.143 INFO:teuthology.orchestra.run.smithi164.stdout: "heap": 223608 2024-05-21T22:06:30.143 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:30.143 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_server": { 2024-05-21T22:06:30.143 INFO:teuthology.orchestra.run.smithi164.stdout: "dispatch_client_request": 3, 2024-05-21T22:06:30.143 INFO:teuthology.orchestra.run.smithi164.stdout: "dispatch_server_request": 0, 2024-05-21T22:06:30.143 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_request": 2, 2024-05-21T22:06:30.143 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_session": 4, 2024-05-21T22:06:30.143 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_peer_request": 11, 2024-05-21T22:06:30.143 INFO:teuthology.orchestra.run.smithi164.stdout: "req_create_latency": { 2024-05-21T22:06:30.144 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:30.144 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:30.144 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:30.144 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:30.144 INFO:teuthology.orchestra.run.smithi164.stdout: "req_getattr_latency": { 2024-05-21T22:06:30.144 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 1, 2024-05-21T22:06:30.144 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000192867, 2024-05-21T22:06:30.144 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000192867 2024-05-21T22:06:30.144 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:30.144 INFO:teuthology.orchestra.run.smithi164.stdout: "req_getfilelock_latency": { 2024-05-21T22:06:30.144 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:30.144 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:30.144 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:30.144 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:30.145 INFO:teuthology.orchestra.run.smithi164.stdout: "req_link_latency": { 2024-05-21T22:06:30.145 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:30.145 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:30.145 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:30.145 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:30.145 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookup_latency": { 2024-05-21T22:06:30.145 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:30.145 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:30.145 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:30.145 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:30.145 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookuphash_latency": { 2024-05-21T22:06:30.145 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:30.145 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:30.145 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:30.146 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:30.146 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupino_latency": { 2024-05-21T22:06:30.146 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:30.146 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:30.146 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:30.146 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:30.146 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupname_latency": { 2024-05-21T22:06:30.146 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:30.146 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:30.146 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:30.146 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:30.146 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupparent_latency": { 2024-05-21T22:06:30.147 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:30.147 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:30.147 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:30.147 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:30.147 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupsnap_latency": { 2024-05-21T22:06:30.147 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:30.147 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:30.147 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:30.147 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:30.147 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lssnap_latency": { 2024-05-21T22:06:30.147 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:30.147 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:30.147 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:30.147 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:30.148 INFO:teuthology.orchestra.run.smithi164.stdout: "req_mkdir_latency": { 2024-05-21T22:06:30.148 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:30.148 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:30.148 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:30.148 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:30.148 INFO:teuthology.orchestra.run.smithi164.stdout: "req_mknod_latency": { 2024-05-21T22:06:30.149 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:30.149 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:30.149 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:30.149 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:30.149 INFO:teuthology.orchestra.run.smithi164.stdout: "req_mksnap_latency": { 2024-05-21T22:06:30.149 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:30.149 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:30.149 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:30.149 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:30.149 INFO:teuthology.orchestra.run.smithi164.stdout: "req_open_latency": { 2024-05-21T22:06:30.149 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:30.149 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:30.149 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:30.150 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:30.150 INFO:teuthology.orchestra.run.smithi164.stdout: "req_readdir_latency": { 2024-05-21T22:06:30.150 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:30.150 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:30.150 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:30.150 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:30.150 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rename_latency": { 2024-05-21T22:06:30.150 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:30.150 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:30.150 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:30.150 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:30.150 INFO:teuthology.orchestra.run.smithi164.stdout: "req_renamesnap_latency": { 2024-05-21T22:06:30.150 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:30.151 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:30.151 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:30.151 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:30.151 INFO:teuthology.orchestra.run.smithi164.stdout: "req_snapdiff_latency": { 2024-05-21T22:06:30.151 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:30.151 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:30.151 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:30.151 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:30.151 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rmdir_latency": { 2024-05-21T22:06:30.151 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:30.151 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:30.151 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:30.151 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:30.152 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rmsnap_latency": { 2024-05-21T22:06:30.152 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:30.152 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:30.152 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:30.152 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:30.152 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rmxattr_latency": { 2024-05-21T22:06:30.152 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:30.152 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:30.152 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:30.152 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:30.152 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setattr_latency": { 2024-05-21T22:06:30.152 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:30.152 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:30.152 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:30.153 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:30.153 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setdirlayout_latency": { 2024-05-21T22:06:30.153 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:30.153 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:30.153 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:30.153 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:30.153 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setfilelock_latency": { 2024-05-21T22:06:30.153 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:30.153 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:30.153 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:30.153 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:30.153 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setlayout_latency": { 2024-05-21T22:06:30.153 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:30.153 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:30.154 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:30.154 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:30.154 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setxattr_latency": { 2024-05-21T22:06:30.154 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:30.154 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:30.154 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:30.154 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:30.154 INFO:teuthology.orchestra.run.smithi164.stdout: "req_symlink_latency": { 2024-05-21T22:06:30.154 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:30.154 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:30.154 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:30.154 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:30.154 INFO:teuthology.orchestra.run.smithi164.stdout: "req_unlink_latency": { 2024-05-21T22:06:30.155 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:30.155 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:30.155 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:30.155 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:30.155 INFO:teuthology.orchestra.run.smithi164.stdout: "cap_revoke_eviction": 0, 2024-05-21T22:06:30.155 INFO:teuthology.orchestra.run.smithi164.stdout: "cap_acquisition_throttle": 0, 2024-05-21T22:06:30.155 INFO:teuthology.orchestra.run.smithi164.stdout: "req_getvxattr_latency": { 2024-05-21T22:06:30.155 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:30.155 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:30.155 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:30.155 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-21T22:06:30.155 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:30.155 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_sessions": { 2024-05-21T22:06:30.155 INFO:teuthology.orchestra.run.smithi164.stdout: "session_count": 0, 2024-05-21T22:06:30.156 INFO:teuthology.orchestra.run.smithi164.stdout: "session_add": 1, 2024-05-21T22:06:30.156 INFO:teuthology.orchestra.run.smithi164.stdout: "session_remove": 1, 2024-05-21T22:06:30.156 INFO:teuthology.orchestra.run.smithi164.stdout: "sessions_open": 0, 2024-05-21T22:06:30.156 INFO:teuthology.orchestra.run.smithi164.stdout: "sessions_stale": 0, 2024-05-21T22:06:30.156 INFO:teuthology.orchestra.run.smithi164.stdout: "total_load": 1, 2024-05-21T22:06:30.156 INFO:teuthology.orchestra.run.smithi164.stdout: "average_load": 1, 2024-05-21T22:06:30.156 INFO:teuthology.orchestra.run.smithi164.stdout: "avg_session_uptime": 81, 2024-05-21T22:06:30.156 INFO:teuthology.orchestra.run.smithi164.stdout: "mdthresh_evicted": 0 2024-05-21T22:06:30.156 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:30.156 INFO:teuthology.orchestra.run.smithi164.stdout: "mempool": { 2024-05-21T22:06:30.156 INFO:teuthology.orchestra.run.smithi164.stdout: "bloom_filter_bytes": 0, 2024-05-21T22:06:30.156 INFO:teuthology.orchestra.run.smithi164.stdout: "bloom_filter_items": 0, 2024-05-21T22:06:30.156 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_alloc_bytes": 0, 2024-05-21T22:06:30.157 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_alloc_items": 0, 2024-05-21T22:06:30.157 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_data_bytes": 0, 2024-05-21T22:06:30.157 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_data_items": 0, 2024-05-21T22:06:30.157 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-21T22:06:30.157 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_onode_items": 0, 2024-05-21T22:06:30.157 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-21T22:06:30.157 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_meta_items": 0, 2024-05-21T22:06:30.157 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_other_bytes": 0, 2024-05-21T22:06:30.157 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_other_items": 0, 2024-05-21T22:06:30.157 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-21T22:06:30.157 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_buffer_items": 0, 2024-05-21T22:06:30.157 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_extent_bytes": 0, 2024-05-21T22:06:30.157 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_extent_items": 0, 2024-05-21T22:06:30.157 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_blob_bytes": 0, 2024-05-21T22:06:30.158 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_blob_items": 0, 2024-05-21T22:06:30.158 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-21T22:06:30.158 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_shared_blob_items": 0, 2024-05-21T22:06:30.158 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-21T22:06:30.158 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_inline_bl_items": 0, 2024-05-21T22:06:30.158 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_fsck_bytes": 0, 2024-05-21T22:06:30.158 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_fsck_items": 0, 2024-05-21T22:06:30.158 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_txc_bytes": 0, 2024-05-21T22:06:30.158 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_txc_items": 0, 2024-05-21T22:06:30.158 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-21T22:06:30.158 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_deferred_items": 0, 2024-05-21T22:06:30.158 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_bytes": 0, 2024-05-21T22:06:30.158 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_items": 0, 2024-05-21T22:06:30.159 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_bytes": 0, 2024-05-21T22:06:30.159 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_items": 0, 2024-05-21T22:06:30.159 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_reader_bytes": 0, 2024-05-21T22:06:30.159 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_reader_items": 0, 2024-05-21T22:06:30.159 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_writer_bytes": 0, 2024-05-21T22:06:30.159 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_writer_items": 0, 2024-05-21T22:06:30.159 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_anon_bytes": 255415, 2024-05-21T22:06:30.159 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_anon_items": 86, 2024-05-21T22:06:30.159 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_meta_bytes": 0, 2024-05-21T22:06:30.159 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_meta_items": 0, 2024-05-21T22:06:30.159 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_bytes": 0, 2024-05-21T22:06:30.159 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_items": 0, 2024-05-21T22:06:30.159 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_mapbl_bytes": 0, 2024-05-21T22:06:30.159 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_mapbl_items": 0, 2024-05-21T22:06:30.160 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_pglog_bytes": 0, 2024-05-21T22:06:30.160 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_pglog_items": 0, 2024-05-21T22:06:30.160 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_bytes": 8128, 2024-05-21T22:06:30.160 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_items": 169, 2024-05-21T22:06:30.160 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_mapping_bytes": 0, 2024-05-21T22:06:30.160 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_mapping_items": 0, 2024-05-21T22:06:30.160 INFO:teuthology.orchestra.run.smithi164.stdout: "pgmap_bytes": 0, 2024-05-21T22:06:30.160 INFO:teuthology.orchestra.run.smithi164.stdout: "pgmap_items": 0, 2024-05-21T22:06:30.160 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_co_bytes": 82717, 2024-05-21T22:06:30.160 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_co_items": 340, 2024-05-21T22:06:30.160 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_1_bytes": 0, 2024-05-21T22:06:30.160 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_1_items": 0, 2024-05-21T22:06:30.160 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_2_bytes": 0, 2024-05-21T22:06:30.160 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_2_items": 0 2024-05-21T22:06:30.160 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:30.161 INFO:teuthology.orchestra.run.smithi164.stdout: "objecter": { 2024-05-21T22:06:30.161 INFO:teuthology.orchestra.run.smithi164.stdout: "op_active": 0, 2024-05-21T22:06:30.161 INFO:teuthology.orchestra.run.smithi164.stdout: "op_laggy": 0, 2024-05-21T22:06:30.161 INFO:teuthology.orchestra.run.smithi164.stdout: "op_send": 38, 2024-05-21T22:06:30.161 INFO:teuthology.orchestra.run.smithi164.stdout: "op_send_bytes": 11332, 2024-05-21T22:06:30.161 INFO:teuthology.orchestra.run.smithi164.stdout: "op_resend": 29, 2024-05-21T22:06:30.161 INFO:teuthology.orchestra.run.smithi164.stdout: "op_reply": 38, 2024-05-21T22:06:30.161 INFO:teuthology.orchestra.run.smithi164.stdout: "op_latency": { 2024-05-21T22:06:30.161 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 38, 2024-05-21T22:06:30.161 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.262000131, 2024-05-21T22:06:30.161 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.006894740 2024-05-21T22:06:30.161 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:30.161 INFO:teuthology.orchestra.run.smithi164.stdout: "op_inflight": 0, 2024-05-21T22:06:30.161 INFO:teuthology.orchestra.run.smithi164.stdout: "oplen_avg": { 2024-05-21T22:06:30.161 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 38, 2024-05-21T22:06:30.162 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 60 2024-05-21T22:06:30.162 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:30.162 INFO:teuthology.orchestra.run.smithi164.stdout: "op": 38, 2024-05-21T22:06:30.162 INFO:teuthology.orchestra.run.smithi164.stdout: "op_r": 0, 2024-05-21T22:06:30.162 INFO:teuthology.orchestra.run.smithi164.stdout: "op_w": 38, 2024-05-21T22:06:30.162 INFO:teuthology.orchestra.run.smithi164.stdout: "op_rmw": 0, 2024-05-21T22:06:30.162 INFO:teuthology.orchestra.run.smithi164.stdout: "op_pg": 0, 2024-05-21T22:06:30.162 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_stat": 0, 2024-05-21T22:06:30.162 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_create": 10, 2024-05-21T22:06:30.162 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_read": 0, 2024-05-21T22:06:30.162 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_write": 3, 2024-05-21T22:06:30.162 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_writefull": 6, 2024-05-21T22:06:30.162 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_writesame": 0, 2024-05-21T22:06:30.163 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_append": 0, 2024-05-21T22:06:30.163 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_zero": 0, 2024-05-21T22:06:30.163 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_truncate": 0, 2024-05-21T22:06:30.163 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_delete": 6, 2024-05-21T22:06:30.163 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_mapext": 0, 2024-05-21T22:06:30.163 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_sparse_read": 0, 2024-05-21T22:06:30.163 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_clonerange": 0, 2024-05-21T22:06:30.163 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_getxattr": 0, 2024-05-21T22:06:30.163 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_setxattr": 20, 2024-05-21T22:06:30.163 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_cmpxattr": 0, 2024-05-21T22:06:30.163 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_rmxattr": 0, 2024-05-21T22:06:30.163 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_resetxattrs": 0, 2024-05-21T22:06:30.163 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_call": 0, 2024-05-21T22:06:30.163 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_watch": 0, 2024-05-21T22:06:30.164 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_notify": 0, 2024-05-21T22:06:30.164 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_src_cmpxattr": 0, 2024-05-21T22:06:30.164 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_pgls": 0, 2024-05-21T22:06:30.164 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_pgls_filter": 0, 2024-05-21T22:06:30.164 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_other": 0, 2024-05-21T22:06:30.164 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_active": 0, 2024-05-21T22:06:30.164 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_send": 0, 2024-05-21T22:06:30.164 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_resend": 0, 2024-05-21T22:06:30.164 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_ping": 0, 2024-05-21T22:06:30.164 INFO:teuthology.orchestra.run.smithi164.stdout: "poolop_active": 0, 2024-05-21T22:06:30.164 INFO:teuthology.orchestra.run.smithi164.stdout: "poolop_send": 0, 2024-05-21T22:06:30.164 INFO:teuthology.orchestra.run.smithi164.stdout: "poolop_resend": 0, 2024-05-21T22:06:30.164 INFO:teuthology.orchestra.run.smithi164.stdout: "poolstat_active": 0, 2024-05-21T22:06:30.164 INFO:teuthology.orchestra.run.smithi164.stdout: "poolstat_send": 0, 2024-05-21T22:06:30.165 INFO:teuthology.orchestra.run.smithi164.stdout: "poolstat_resend": 0, 2024-05-21T22:06:30.165 INFO:teuthology.orchestra.run.smithi164.stdout: "statfs_active": 0, 2024-05-21T22:06:30.165 INFO:teuthology.orchestra.run.smithi164.stdout: "statfs_send": 0, 2024-05-21T22:06:30.165 INFO:teuthology.orchestra.run.smithi164.stdout: "statfs_resend": 0, 2024-05-21T22:06:30.165 INFO:teuthology.orchestra.run.smithi164.stdout: "command_active": 0, 2024-05-21T22:06:30.165 INFO:teuthology.orchestra.run.smithi164.stdout: "command_send": 0, 2024-05-21T22:06:30.165 INFO:teuthology.orchestra.run.smithi164.stdout: "command_resend": 0, 2024-05-21T22:06:30.165 INFO:teuthology.orchestra.run.smithi164.stdout: "map_epoch": 91, 2024-05-21T22:06:30.165 INFO:teuthology.orchestra.run.smithi164.stdout: "map_full": 0, 2024-05-21T22:06:30.165 INFO:teuthology.orchestra.run.smithi164.stdout: "map_inc": 11, 2024-05-21T22:06:30.165 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_sessions": 8, 2024-05-21T22:06:30.165 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_session_open": 8, 2024-05-21T22:06:30.165 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_session_close": 0, 2024-05-21T22:06:30.165 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_laggy": 0, 2024-05-21T22:06:30.165 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_wr": 14, 2024-05-21T22:06:30.166 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_rd": 0, 2024-05-21T22:06:30.166 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_del": 1 2024-05-21T22:06:30.166 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:30.166 INFO:teuthology.orchestra.run.smithi164.stdout: "oft": { 2024-05-21T22:06:30.166 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_objs": 1, 2024-05-21T22:06:30.166 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_kv_pairs": 0, 2024-05-21T22:06:30.166 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_updates": 0, 2024-05-21T22:06:30.166 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_removes": 0 2024-05-21T22:06:30.166 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:30.166 INFO:teuthology.orchestra.run.smithi164.stdout: "osd-slow-ops": { 2024-05-21T22:06:30.166 INFO:teuthology.orchestra.run.smithi164.stdout: "slow_ops_count": 13 2024-05-21T22:06:30.166 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:30.166 INFO:teuthology.orchestra.run.smithi164.stdout: "purge_queue": { 2024-05-21T22:06:30.166 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing_ops": 0, 2024-05-21T22:06:30.167 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing_ops_high_water": 0, 2024-05-21T22:06:30.167 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing": 0, 2024-05-21T22:06:30.167 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing_high_water": 0, 2024-05-21T22:06:30.167 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executed_ops": 0, 2024-05-21T22:06:30.167 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executed": 0, 2024-05-21T22:06:30.167 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_item_in_journal": 0 2024-05-21T22:06:30.167 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:30.167 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-21T22:06:30.167 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-05-21T22:06:30.167 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 104857600, 2024-05-21T22:06:30.167 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-05-21T22:06:30.167 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 1173, 2024-05-21T22:06:30.167 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 2070441, 2024-05-21T22:06:30.168 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-05-21T22:06:30.168 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 1173, 2024-05-21T22:06:30.168 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 0, 2024-05-21T22:06:30.168 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 0, 2024-05-21T22:06:30.168 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 1173, 2024-05-21T22:06:30.168 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 2070441, 2024-05-21T22:06:30.168 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-05-21T22:06:30.168 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:30.168 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:30.168 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:30.168 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-21T22:06:30.168 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:30.168 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-objecter_bytes": { 2024-05-21T22:06:30.168 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-05-21T22:06:30.169 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 104857600, 2024-05-21T22:06:30.169 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-05-21T22:06:30.169 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 0, 2024-05-21T22:06:30.169 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 0, 2024-05-21T22:06:30.169 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-05-21T22:06:30.169 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 0, 2024-05-21T22:06:30.169 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 38, 2024-05-21T22:06:30.169 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 11332, 2024-05-21T22:06:30.169 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 38, 2024-05-21T22:06:30.169 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 11332, 2024-05-21T22:06:30.169 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-05-21T22:06:30.169 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:30.169 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:30.170 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:30.170 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-21T22:06:30.170 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:30.170 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-objecter_ops": { 2024-05-21T22:06:30.170 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-05-21T22:06:30.170 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 1024, 2024-05-21T22:06:30.170 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-05-21T22:06:30.170 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 0, 2024-05-21T22:06:30.170 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 0, 2024-05-21T22:06:30.170 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-05-21T22:06:30.170 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 0, 2024-05-21T22:06:30.170 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 38, 2024-05-21T22:06:30.170 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 38, 2024-05-21T22:06:30.170 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 38, 2024-05-21T22:06:30.171 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 38, 2024-05-21T22:06:30.171 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-05-21T22:06:30.171 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:30.171 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:30.171 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:30.171 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-21T22:06:30.171 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:30.171 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-write_buf_throttle": { 2024-05-21T22:06:30.171 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-05-21T22:06:30.171 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 3758096384, 2024-05-21T22:06:30.171 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-05-21T22:06:30.171 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 0, 2024-05-21T22:06:30.171 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 0, 2024-05-21T22:06:30.172 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-05-21T22:06:30.172 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 0, 2024-05-21T22:06:30.172 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 0, 2024-05-21T22:06:30.172 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 0, 2024-05-21T22:06:30.172 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 0, 2024-05-21T22:06:30.172 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 0, 2024-05-21T22:06:30.172 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-05-21T22:06:30.172 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:30.172 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:30.172 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:30.172 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-21T22:06:30.172 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:30.173 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-write_buf_throttle-0x5628f1160500": { 2024-05-21T22:06:30.173 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-05-21T22:06:30.173 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 3758096384, 2024-05-21T22:06:30.173 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-05-21T22:06:30.173 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 4, 2024-05-21T22:06:30.173 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 1120, 2024-05-21T22:06:30.173 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-05-21T22:06:30.173 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 4, 2024-05-21T22:06:30.173 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 0, 2024-05-21T22:06:30.173 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 0, 2024-05-21T22:06:30.173 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 3, 2024-05-21T22:06:30.173 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 1120, 2024-05-21T22:06:30.173 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-05-21T22:06:30.173 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:30.174 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:30.174 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:30.174 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-21T22:06:30.174 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-21T22:06:30.174 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-05-21T22:06:30.906 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-05-21T22:06:30.906 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-05-21T22:06:30.906 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2024-05-21T22:06:30.906 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=12 2024-05-21T22:06:30.907 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=7 2024-05-21T22:06:30.907 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-05-21T22:06:30.907 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-05-21T22:06:30.907 DEBUG:tasks.check_counter:Getting stats from k 2024-05-21T22:06:30.907 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph daemon mds.k perf dump 2024-05-21T22:06:31.145 INFO:teuthology.orchestra.run.smithi164.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.b/config 2024-05-21T22:06:31.936 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:06:31 smithi164 ceph-mon[36269]: pgmap v338: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:06:31.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:06:31 smithi151 ceph-mon[31623]: pgmap v338: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:06:31.985 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:06:31 smithi184 ceph-mon[36942]: pgmap v338: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:06:32.818 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-05-21T22:06:32.818 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-0": { 2024-05-21T22:06:32.818 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 294, 2024-05-21T22:06:32.818 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 278, 2024-05-21T22:06:32.818 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 1914130, 2024-05-21T22:06:32.819 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 163300, 2024-05-21T22:06:32.819 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 5, 2024-05-21T22:06:32.819 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 1, 2024-05-21T22:06:32.819 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 0.057026072, 2024-05-21T22:06:32.819 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 0.016858728, 2024-05-21T22:06:32.819 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 0.045106437, 2024-05-21T22:06:32.819 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 0.006486575, 2024-05-21T22:06:32.819 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T22:06:32.819 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 278, 2024-05-21T22:06:32.819 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.016757548, 2024-05-21T22:06:32.819 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000060278 2024-05-21T22:06:32.819 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:32.819 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2024-05-21T22:06:32.820 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:32.820 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:32.820 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:32.820 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:32.820 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 1913296, 2024-05-21T22:06:32.820 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 162976 2024-05-21T22:06:32.820 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:32.820 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-1": { 2024-05-21T22:06:32.820 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 806, 2024-05-21T22:06:32.820 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 1002, 2024-05-21T22:06:32.820 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 198126, 2024-05-21T22:06:32.820 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 752509, 2024-05-21T22:06:32.821 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 7, 2024-05-21T22:06:32.821 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 5, 2024-05-21T22:06:32.821 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 0.088289425, 2024-05-21T22:06:32.821 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 0.030911794, 2024-05-21T22:06:32.821 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 0.018944320, 2024-05-21T22:06:32.821 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 0.028586128, 2024-05-21T22:06:32.821 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T22:06:32.821 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 999, 2024-05-21T22:06:32.821 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.068591300, 2024-05-21T22:06:32.821 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000068659 2024-05-21T22:06:32.821 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:32.821 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2024-05-21T22:06:32.821 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:32.822 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:32.822 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:32.822 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:32.822 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-21T22:06:32.822 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 496832 2024-05-21T22:06:32.822 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:32.822 INFO:teuthology.orchestra.run.smithi164.stdout: "AsyncMessenger::Worker-2": { 2024-05-21T22:06:32.822 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_messages": 1789, 2024-05-21T22:06:32.822 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages": 1791, 2024-05-21T22:06:32.822 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_bytes": 535865, 2024-05-21T22:06:32.822 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_bytes": 568960, 2024-05-21T22:06:32.822 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_created_connections": 4, 2024-05-21T22:06:32.823 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_active_connections": 3, 2024-05-21T22:06:32.823 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_total_time": 0.185155482, 2024-05-21T22:06:32.823 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_send_time": 0.065778636, 2024-05-21T22:06:32.823 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_recv_time": 0.046165318, 2024-05-21T22:06:32.823 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_running_fast_dispatch_time": 0.055952167, 2024-05-21T22:06:32.823 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T22:06:32.823 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 1789, 2024-05-21T22:06:32.823 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.078078006, 2024-05-21T22:06:32.823 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000043643 2024-05-21T22:06:32.823 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:32.823 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_handle_ack_lat": { 2024-05-21T22:06:32.823 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:32.823 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:32.824 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:32.824 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:32.824 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-21T22:06:32.824 INFO:teuthology.orchestra.run.smithi164.stdout: "msgr_send_encrypted_bytes": 0 2024-05-21T22:06:32.824 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:32.824 INFO:teuthology.orchestra.run.smithi164.stdout: "cct": { 2024-05-21T22:06:32.824 INFO:teuthology.orchestra.run.smithi164.stdout: "total_workers": 1, 2024-05-21T22:06:32.825 INFO:teuthology.orchestra.run.smithi164.stdout: "unhealthy_workers": 0 2024-05-21T22:06:32.825 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:32.825 INFO:teuthology.orchestra.run.smithi164.stdout: "finisher-MDSRank": { 2024-05-21T22:06:32.825 INFO:teuthology.orchestra.run.smithi164.stdout: "queue_len": 0, 2024-05-21T22:06:32.825 INFO:teuthology.orchestra.run.smithi164.stdout: "complete_latency": { 2024-05-21T22:06:32.825 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 4377, 2024-05-21T22:06:32.825 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.129258054, 2024-05-21T22:06:32.825 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000029531 2024-05-21T22:06:32.825 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-21T22:06:32.825 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:32.825 INFO:teuthology.orchestra.run.smithi164.stdout: "finisher-PurgeQueue": { 2024-05-21T22:06:32.826 INFO:teuthology.orchestra.run.smithi164.stdout: "queue_len": 0, 2024-05-21T22:06:32.826 INFO:teuthology.orchestra.run.smithi164.stdout: "complete_latency": { 2024-05-21T22:06:32.826 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:32.826 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:32.826 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:32.826 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-21T22:06:32.826 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:32.826 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": { 2024-05-21T22:06:32.826 INFO:teuthology.orchestra.run.smithi164.stdout: "request": 0, 2024-05-21T22:06:32.826 INFO:teuthology.orchestra.run.smithi164.stdout: "reply": 0, 2024-05-21T22:06:32.826 INFO:teuthology.orchestra.run.smithi164.stdout: "reply_latency": { 2024-05-21T22:06:32.826 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:32.827 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:32.827 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:32.827 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:32.827 INFO:teuthology.orchestra.run.smithi164.stdout: "slow_reply": 0, 2024-05-21T22:06:32.827 INFO:teuthology.orchestra.run.smithi164.stdout: "forward": 0, 2024-05-21T22:06:32.827 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_fetch_complete": 0, 2024-05-21T22:06:32.827 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_fetch_keys": 0, 2024-05-21T22:06:32.827 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_commit": 0, 2024-05-21T22:06:32.827 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_split": 0, 2024-05-21T22:06:32.827 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_merge": 0, 2024-05-21T22:06:32.827 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes": 0, 2024-05-21T22:06:32.827 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_top": 0, 2024-05-21T22:06:32.827 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_bottom": 0, 2024-05-21T22:06:32.828 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_pin_tail": 0, 2024-05-21T22:06:32.828 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_pinned": 0, 2024-05-21T22:06:32.828 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_expired": 0, 2024-05-21T22:06:32.828 INFO:teuthology.orchestra.run.smithi164.stdout: "inodes_with_caps": 0, 2024-05-21T22:06:32.828 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": 0, 2024-05-21T22:06:32.828 INFO:teuthology.orchestra.run.smithi164.stdout: "subtrees": 1, 2024-05-21T22:06:32.828 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse": 0, 2024-05-21T22:06:32.828 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_hit": 0, 2024-05-21T22:06:32.828 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_forward": 0, 2024-05-21T22:06:32.828 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_discover": 0, 2024-05-21T22:06:32.828 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_dir_fetch": 0, 2024-05-21T22:06:32.828 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_remote_ino": 0, 2024-05-21T22:06:32.828 INFO:teuthology.orchestra.run.smithi164.stdout: "traverse_lock": 0, 2024-05-21T22:06:32.828 INFO:teuthology.orchestra.run.smithi164.stdout: "load_cent": 0, 2024-05-21T22:06:32.829 INFO:teuthology.orchestra.run.smithi164.stdout: "q": 0, 2024-05-21T22:06:32.829 INFO:teuthology.orchestra.run.smithi164.stdout: "exported": 0, 2024-05-21T22:06:32.829 INFO:teuthology.orchestra.run.smithi164.stdout: "exported_inodes": 0, 2024-05-21T22:06:32.829 INFO:teuthology.orchestra.run.smithi164.stdout: "imported": 0, 2024-05-21T22:06:32.829 INFO:teuthology.orchestra.run.smithi164.stdout: "imported_inodes": 0, 2024-05-21T22:06:32.829 INFO:teuthology.orchestra.run.smithi164.stdout: "openino_dir_fetch": 0, 2024-05-21T22:06:32.829 INFO:teuthology.orchestra.run.smithi164.stdout: "openino_backtrace_fetch": 0, 2024-05-21T22:06:32.829 INFO:teuthology.orchestra.run.smithi164.stdout: "openino_peer_discover": 0, 2024-05-21T22:06:32.829 INFO:teuthology.orchestra.run.smithi164.stdout: "root_rfiles": 0, 2024-05-21T22:06:32.829 INFO:teuthology.orchestra.run.smithi164.stdout: "root_rbytes": 0, 2024-05-21T22:06:32.829 INFO:teuthology.orchestra.run.smithi164.stdout: "root_rsnaps": 0, 2024-05-21T22:06:32.829 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_backtrace_fetch": 0, 2024-05-21T22:06:32.829 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_set_tag": 0, 2024-05-21T22:06:32.830 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_backtrace_repaired": 0, 2024-05-21T22:06:32.830 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_inotable_repaired": 0, 2024-05-21T22:06:32.830 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_dir_inodes": 0, 2024-05-21T22:06:32.830 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_dir_base_inodes": 0, 2024-05-21T22:06:32.830 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_dirfrag_rstats": 0, 2024-05-21T22:06:32.830 INFO:teuthology.orchestra.run.smithi164.stdout: "scrub_file_inodes": 0, 2024-05-21T22:06:32.830 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_inode_file_caps": 0, 2024-05-21T22:06:32.830 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_revoke": 0, 2024-05-21T22:06:32.830 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_grant": 0, 2024-05-21T22:06:32.830 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_trunc": 0, 2024-05-21T22:06:32.830 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-21T22:06:32.830 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-21T22:06:32.830 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_caps": 0, 2024-05-21T22:06:32.831 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_caps_dirty": 0, 2024-05-21T22:06:32.831 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_cap_release": 0, 2024-05-21T22:06:32.831 INFO:teuthology.orchestra.run.smithi164.stdout: "process_request_cap_release": 0 2024-05-21T22:06:32.831 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:32.831 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_cache": { 2024-05-21T22:06:32.831 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_update": 0, 2024-05-21T22:06:32.831 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_update_receipt": 0, 2024-05-21T22:06:32.831 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_try_discover": 0, 2024-05-21T22:06:32.831 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_send_discover": 0, 2024-05-21T22:06:32.831 INFO:teuthology.orchestra.run.smithi164.stdout: "dir_handle_discover": 0, 2024-05-21T22:06:32.832 INFO:teuthology.orchestra.run.smithi164.stdout: "num_strays": 0, 2024-05-21T22:06:32.832 INFO:teuthology.orchestra.run.smithi164.stdout: "num_strays_delayed": 0, 2024-05-21T22:06:32.832 INFO:teuthology.orchestra.run.smithi164.stdout: "num_strays_enqueuing": 0, 2024-05-21T22:06:32.832 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_created": 0, 2024-05-21T22:06:32.832 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_enqueued": 0, 2024-05-21T22:06:32.832 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_reintegrated": 0, 2024-05-21T22:06:32.832 INFO:teuthology.orchestra.run.smithi164.stdout: "strays_migrated": 0, 2024-05-21T22:06:32.832 INFO:teuthology.orchestra.run.smithi164.stdout: "num_recovering_processing": 0, 2024-05-21T22:06:32.832 INFO:teuthology.orchestra.run.smithi164.stdout: "num_recovering_enqueued": 0, 2024-05-21T22:06:32.832 INFO:teuthology.orchestra.run.smithi164.stdout: "num_recovering_prioritized": 0, 2024-05-21T22:06:32.832 INFO:teuthology.orchestra.run.smithi164.stdout: "recovery_started": 0, 2024-05-21T22:06:32.832 INFO:teuthology.orchestra.run.smithi164.stdout: "recovery_completed": 0, 2024-05-21T22:06:32.832 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_quiesce_path": 0, 2024-05-21T22:06:32.833 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_quiesce_inode": 0, 2024-05-21T22:06:32.833 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_enqueue_scrub": 0, 2024-05-21T22:06:32.833 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_exportdir": 0, 2024-05-21T22:06:32.833 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_flush": 0, 2024-05-21T22:06:32.833 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_fragmentdir": 0, 2024-05-21T22:06:32.833 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_fragstats": 0, 2024-05-21T22:06:32.833 INFO:teuthology.orchestra.run.smithi164.stdout: "ireq_inodestats": 0 2024-05-21T22:06:32.833 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:32.833 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_log": { 2024-05-21T22:06:32.833 INFO:teuthology.orchestra.run.smithi164.stdout: "evlrg": 0, 2024-05-21T22:06:32.833 INFO:teuthology.orchestra.run.smithi164.stdout: "evadd": 0, 2024-05-21T22:06:32.833 INFO:teuthology.orchestra.run.smithi164.stdout: "evex": 0, 2024-05-21T22:06:32.833 INFO:teuthology.orchestra.run.smithi164.stdout: "evtrm": 0, 2024-05-21T22:06:32.834 INFO:teuthology.orchestra.run.smithi164.stdout: "ev": 4, 2024-05-21T22:06:32.834 INFO:teuthology.orchestra.run.smithi164.stdout: "evexg": 0, 2024-05-21T22:06:32.834 INFO:teuthology.orchestra.run.smithi164.stdout: "evexd": 0, 2024-05-21T22:06:32.834 INFO:teuthology.orchestra.run.smithi164.stdout: "segadd": 0, 2024-05-21T22:06:32.834 INFO:teuthology.orchestra.run.smithi164.stdout: "segex": 0, 2024-05-21T22:06:32.834 INFO:teuthology.orchestra.run.smithi164.stdout: "segtrm": 0, 2024-05-21T22:06:32.834 INFO:teuthology.orchestra.run.smithi164.stdout: "seg": 2, 2024-05-21T22:06:32.834 INFO:teuthology.orchestra.run.smithi164.stdout: "segmjr": 2, 2024-05-21T22:06:32.834 INFO:teuthology.orchestra.run.smithi164.stdout: "segexg": 0, 2024-05-21T22:06:32.834 INFO:teuthology.orchestra.run.smithi164.stdout: "segexd": 0, 2024-05-21T22:06:32.834 INFO:teuthology.orchestra.run.smithi164.stdout: "expos": 4194304, 2024-05-21T22:06:32.834 INFO:teuthology.orchestra.run.smithi164.stdout: "wrpos": 4195424, 2024-05-21T22:06:32.834 INFO:teuthology.orchestra.run.smithi164.stdout: "rdpos": 4195283, 2024-05-21T22:06:32.835 INFO:teuthology.orchestra.run.smithi164.stdout: "jlat": { 2024-05-21T22:06:32.835 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:32.835 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:32.835 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:32.835 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:32.835 INFO:teuthology.orchestra.run.smithi164.stdout: "replayed": 4 2024-05-21T22:06:32.835 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:32.835 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_mem": { 2024-05-21T22:06:32.835 INFO:teuthology.orchestra.run.smithi164.stdout: "ino": 3, 2024-05-21T22:06:32.835 INFO:teuthology.orchestra.run.smithi164.stdout: "ino+": 3, 2024-05-21T22:06:32.835 INFO:teuthology.orchestra.run.smithi164.stdout: "ino-": 0, 2024-05-21T22:06:32.835 INFO:teuthology.orchestra.run.smithi164.stdout: "dir": 1, 2024-05-21T22:06:32.835 INFO:teuthology.orchestra.run.smithi164.stdout: "dir+": 1, 2024-05-21T22:06:32.835 INFO:teuthology.orchestra.run.smithi164.stdout: "dir-": 0, 2024-05-21T22:06:32.836 INFO:teuthology.orchestra.run.smithi164.stdout: "dn": 0, 2024-05-21T22:06:32.836 INFO:teuthology.orchestra.run.smithi164.stdout: "dn+": 0, 2024-05-21T22:06:32.836 INFO:teuthology.orchestra.run.smithi164.stdout: "dn-": 0, 2024-05-21T22:06:32.836 INFO:teuthology.orchestra.run.smithi164.stdout: "cap": 0, 2024-05-21T22:06:32.836 INFO:teuthology.orchestra.run.smithi164.stdout: "cap+": 0, 2024-05-21T22:06:32.836 INFO:teuthology.orchestra.run.smithi164.stdout: "cap-": 0, 2024-05-21T22:06:32.836 INFO:teuthology.orchestra.run.smithi164.stdout: "rss": 50808, 2024-05-21T22:06:32.836 INFO:teuthology.orchestra.run.smithi164.stdout: "heap": 215416 2024-05-21T22:06:32.836 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:32.836 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_server": { 2024-05-21T22:06:32.836 INFO:teuthology.orchestra.run.smithi164.stdout: "dispatch_client_request": 0, 2024-05-21T22:06:32.836 INFO:teuthology.orchestra.run.smithi164.stdout: "dispatch_server_request": 0, 2024-05-21T22:06:32.836 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_request": 0, 2024-05-21T22:06:32.836 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_client_session": 0, 2024-05-21T22:06:32.837 INFO:teuthology.orchestra.run.smithi164.stdout: "handle_peer_request": 0, 2024-05-21T22:06:32.837 INFO:teuthology.orchestra.run.smithi164.stdout: "req_create_latency": { 2024-05-21T22:06:32.837 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:32.837 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:32.837 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:32.837 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:32.838 INFO:teuthology.orchestra.run.smithi164.stdout: "req_getattr_latency": { 2024-05-21T22:06:32.838 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:32.838 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:32.838 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:32.838 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:32.838 INFO:teuthology.orchestra.run.smithi164.stdout: "req_getfilelock_latency": { 2024-05-21T22:06:32.838 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:32.838 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:32.838 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:32.838 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:32.838 INFO:teuthology.orchestra.run.smithi164.stdout: "req_link_latency": { 2024-05-21T22:06:32.839 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:32.839 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:32.839 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:32.839 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:32.839 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookup_latency": { 2024-05-21T22:06:32.839 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:32.839 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:32.839 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:32.839 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:32.839 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookuphash_latency": { 2024-05-21T22:06:32.840 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:32.840 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:32.840 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:32.840 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:32.840 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupino_latency": { 2024-05-21T22:06:32.840 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:32.840 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:32.840 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:32.840 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:32.840 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupname_latency": { 2024-05-21T22:06:32.840 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:32.840 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:32.840 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:32.841 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:32.841 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupparent_latency": { 2024-05-21T22:06:32.841 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:32.841 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:32.841 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:32.841 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:32.841 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lookupsnap_latency": { 2024-05-21T22:06:32.841 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:32.841 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:32.841 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:32.841 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:32.841 INFO:teuthology.orchestra.run.smithi164.stdout: "req_lssnap_latency": { 2024-05-21T22:06:32.841 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:32.841 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:32.842 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:32.842 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:32.842 INFO:teuthology.orchestra.run.smithi164.stdout: "req_mkdir_latency": { 2024-05-21T22:06:32.842 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:32.843 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:32.843 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:32.843 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:32.843 INFO:teuthology.orchestra.run.smithi164.stdout: "req_mknod_latency": { 2024-05-21T22:06:32.843 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:32.843 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:32.843 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:32.844 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:32.844 INFO:teuthology.orchestra.run.smithi164.stdout: "req_mksnap_latency": { 2024-05-21T22:06:32.844 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:32.844 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:32.844 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:32.844 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:32.844 INFO:teuthology.orchestra.run.smithi164.stdout: "req_open_latency": { 2024-05-21T22:06:32.844 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:32.844 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:32.845 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:32.845 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:32.845 INFO:teuthology.orchestra.run.smithi164.stdout: "req_readdir_latency": { 2024-05-21T22:06:32.845 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:32.845 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:32.845 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:32.845 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:32.845 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rename_latency": { 2024-05-21T22:06:32.845 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:32.845 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:32.845 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:32.845 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:32.845 INFO:teuthology.orchestra.run.smithi164.stdout: "req_renamesnap_latency": { 2024-05-21T22:06:32.846 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:32.846 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:32.846 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:32.846 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:32.846 INFO:teuthology.orchestra.run.smithi164.stdout: "req_snapdiff_latency": { 2024-05-21T22:06:32.846 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:32.846 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:32.846 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:32.846 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:32.846 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rmdir_latency": { 2024-05-21T22:06:32.846 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:32.846 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:32.846 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:32.847 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:32.847 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rmsnap_latency": { 2024-05-21T22:06:32.847 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:32.847 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:32.847 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:32.847 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:32.847 INFO:teuthology.orchestra.run.smithi164.stdout: "req_rmxattr_latency": { 2024-05-21T22:06:32.847 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:32.847 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:32.847 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:32.847 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:32.847 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setattr_latency": { 2024-05-21T22:06:32.847 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:32.847 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:32.848 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:32.848 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:32.848 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setdirlayout_latency": { 2024-05-21T22:06:32.848 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:32.848 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:32.848 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:32.848 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:32.848 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setfilelock_latency": { 2024-05-21T22:06:32.848 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:32.848 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:32.848 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:32.848 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:32.848 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setlayout_latency": { 2024-05-21T22:06:32.849 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:32.849 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:32.849 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:32.849 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:32.849 INFO:teuthology.orchestra.run.smithi164.stdout: "req_setxattr_latency": { 2024-05-21T22:06:32.849 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:32.849 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:32.849 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:32.849 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:32.849 INFO:teuthology.orchestra.run.smithi164.stdout: "req_symlink_latency": { 2024-05-21T22:06:32.849 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:32.849 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:32.849 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:32.850 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:32.850 INFO:teuthology.orchestra.run.smithi164.stdout: "req_unlink_latency": { 2024-05-21T22:06:32.850 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:32.850 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:32.850 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:32.850 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:32.850 INFO:teuthology.orchestra.run.smithi164.stdout: "cap_revoke_eviction": 0, 2024-05-21T22:06:32.850 INFO:teuthology.orchestra.run.smithi164.stdout: "cap_acquisition_throttle": 0, 2024-05-21T22:06:32.850 INFO:teuthology.orchestra.run.smithi164.stdout: "req_getvxattr_latency": { 2024-05-21T22:06:32.850 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:32.850 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:32.850 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:32.851 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-21T22:06:32.851 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:32.851 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_sessions": { 2024-05-21T22:06:32.851 INFO:teuthology.orchestra.run.smithi164.stdout: "session_count": 0, 2024-05-21T22:06:32.851 INFO:teuthology.orchestra.run.smithi164.stdout: "session_add": 1, 2024-05-21T22:06:32.851 INFO:teuthology.orchestra.run.smithi164.stdout: "session_remove": 1, 2024-05-21T22:06:32.851 INFO:teuthology.orchestra.run.smithi164.stdout: "sessions_open": 1, 2024-05-21T22:06:32.851 INFO:teuthology.orchestra.run.smithi164.stdout: "sessions_stale": 0, 2024-05-21T22:06:32.851 INFO:teuthology.orchestra.run.smithi164.stdout: "total_load": 0, 2024-05-21T22:06:32.851 INFO:teuthology.orchestra.run.smithi164.stdout: "average_load": 0, 2024-05-21T22:06:32.851 INFO:teuthology.orchestra.run.smithi164.stdout: "avg_session_uptime": 1550, 2024-05-21T22:06:32.851 INFO:teuthology.orchestra.run.smithi164.stdout: "mdthresh_evicted": 0 2024-05-21T22:06:32.851 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:32.851 INFO:teuthology.orchestra.run.smithi164.stdout: "mempool": { 2024-05-21T22:06:32.852 INFO:teuthology.orchestra.run.smithi164.stdout: "bloom_filter_bytes": 0, 2024-05-21T22:06:32.852 INFO:teuthology.orchestra.run.smithi164.stdout: "bloom_filter_items": 0, 2024-05-21T22:06:32.852 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_alloc_bytes": 0, 2024-05-21T22:06:32.852 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_alloc_items": 0, 2024-05-21T22:06:32.852 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_data_bytes": 0, 2024-05-21T22:06:32.852 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_data_items": 0, 2024-05-21T22:06:32.852 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-21T22:06:32.852 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_onode_items": 0, 2024-05-21T22:06:32.852 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-21T22:06:32.852 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_meta_items": 0, 2024-05-21T22:06:32.852 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_other_bytes": 0, 2024-05-21T22:06:32.852 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_other_items": 0, 2024-05-21T22:06:32.852 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-21T22:06:32.853 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_cache_buffer_items": 0, 2024-05-21T22:06:32.853 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_extent_bytes": 0, 2024-05-21T22:06:32.853 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_extent_items": 0, 2024-05-21T22:06:32.853 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_blob_bytes": 0, 2024-05-21T22:06:32.853 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_blob_items": 0, 2024-05-21T22:06:32.853 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-21T22:06:32.853 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_shared_blob_items": 0, 2024-05-21T22:06:32.853 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-21T22:06:32.853 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_inline_bl_items": 0, 2024-05-21T22:06:32.853 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_fsck_bytes": 0, 2024-05-21T22:06:32.853 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_fsck_items": 0, 2024-05-21T22:06:32.853 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_txc_bytes": 0, 2024-05-21T22:06:32.853 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_txc_items": 0, 2024-05-21T22:06:32.854 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-21T22:06:32.854 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_deferred_items": 0, 2024-05-21T22:06:32.854 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_bytes": 0, 2024-05-21T22:06:32.854 INFO:teuthology.orchestra.run.smithi164.stdout: "bluestore_writing_items": 0, 2024-05-21T22:06:32.854 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_bytes": 0, 2024-05-21T22:06:32.854 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_items": 0, 2024-05-21T22:06:32.854 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_reader_bytes": 0, 2024-05-21T22:06:32.854 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_reader_items": 0, 2024-05-21T22:06:32.854 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_writer_bytes": 0, 2024-05-21T22:06:32.854 INFO:teuthology.orchestra.run.smithi164.stdout: "bluefs_file_writer_items": 0, 2024-05-21T22:06:32.854 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_anon_bytes": 190179, 2024-05-21T22:06:32.854 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_anon_items": 53, 2024-05-21T22:06:32.854 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_meta_bytes": 0, 2024-05-21T22:06:32.854 INFO:teuthology.orchestra.run.smithi164.stdout: "buffer_meta_items": 0, 2024-05-21T22:06:32.855 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_bytes": 0, 2024-05-21T22:06:32.855 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_items": 0, 2024-05-21T22:06:32.855 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_mapbl_bytes": 0, 2024-05-21T22:06:32.855 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_mapbl_items": 0, 2024-05-21T22:06:32.855 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_pglog_bytes": 0, 2024-05-21T22:06:32.855 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_pglog_items": 0, 2024-05-21T22:06:32.855 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_bytes": 8128, 2024-05-21T22:06:32.855 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_items": 169, 2024-05-21T22:06:32.855 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_mapping_bytes": 0, 2024-05-21T22:06:32.855 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_mapping_items": 0, 2024-05-21T22:06:32.855 INFO:teuthology.orchestra.run.smithi164.stdout: "pgmap_bytes": 0, 2024-05-21T22:06:32.855 INFO:teuthology.orchestra.run.smithi164.stdout: "pgmap_items": 0, 2024-05-21T22:06:32.856 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_co_bytes": 8512, 2024-05-21T22:06:32.856 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_co_items": 13, 2024-05-21T22:06:32.856 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_1_bytes": 0, 2024-05-21T22:06:32.856 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_1_items": 0, 2024-05-21T22:06:32.856 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_2_bytes": 0, 2024-05-21T22:06:32.856 INFO:teuthology.orchestra.run.smithi164.stdout: "unittest_2_items": 0 2024-05-21T22:06:32.856 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:32.856 INFO:teuthology.orchestra.run.smithi164.stdout: "objecter": { 2024-05-21T22:06:32.856 INFO:teuthology.orchestra.run.smithi164.stdout: "op_active": 0, 2024-05-21T22:06:32.856 INFO:teuthology.orchestra.run.smithi164.stdout: "op_laggy": 0, 2024-05-21T22:06:32.856 INFO:teuthology.orchestra.run.smithi164.stdout: "op_send": 2594, 2024-05-21T22:06:32.857 INFO:teuthology.orchestra.run.smithi164.stdout: "op_send_bytes": 21, 2024-05-21T22:06:32.857 INFO:teuthology.orchestra.run.smithi164.stdout: "op_resend": 3, 2024-05-21T22:06:32.857 INFO:teuthology.orchestra.run.smithi164.stdout: "op_reply": 2594, 2024-05-21T22:06:32.857 INFO:teuthology.orchestra.run.smithi164.stdout: "op_latency": { 2024-05-21T22:06:32.857 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 2594, 2024-05-21T22:06:32.857 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 1.936001358, 2024-05-21T22:06:32.857 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000746338 2024-05-21T22:06:32.857 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:32.857 INFO:teuthology.orchestra.run.smithi164.stdout: "op_inflight": 0, 2024-05-21T22:06:32.857 INFO:teuthology.orchestra.run.smithi164.stdout: "oplen_avg": { 2024-05-21T22:06:32.857 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 2594, 2024-05-21T22:06:32.857 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 2595 2024-05-21T22:06:32.857 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:32.858 INFO:teuthology.orchestra.run.smithi164.stdout: "op": 2594, 2024-05-21T22:06:32.858 INFO:teuthology.orchestra.run.smithi164.stdout: "op_r": 2594, 2024-05-21T22:06:32.858 INFO:teuthology.orchestra.run.smithi164.stdout: "op_w": 0, 2024-05-21T22:06:32.858 INFO:teuthology.orchestra.run.smithi164.stdout: "op_rmw": 0, 2024-05-21T22:06:32.858 INFO:teuthology.orchestra.run.smithi164.stdout: "op_pg": 0, 2024-05-21T22:06:32.858 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_stat": 1694, 2024-05-21T22:06:32.858 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_create": 0, 2024-05-21T22:06:32.858 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_read": 898, 2024-05-21T22:06:32.858 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_write": 0, 2024-05-21T22:06:32.858 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_writefull": 0, 2024-05-21T22:06:32.858 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_writesame": 0, 2024-05-21T22:06:32.858 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_append": 0, 2024-05-21T22:06:32.858 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_zero": 0, 2024-05-21T22:06:32.858 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_truncate": 0, 2024-05-21T22:06:32.859 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_delete": 0, 2024-05-21T22:06:32.859 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_mapext": 0, 2024-05-21T22:06:32.859 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_sparse_read": 0, 2024-05-21T22:06:32.859 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_clonerange": 0, 2024-05-21T22:06:32.859 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_getxattr": 1, 2024-05-21T22:06:32.859 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_setxattr": 0, 2024-05-21T22:06:32.859 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_cmpxattr": 0, 2024-05-21T22:06:32.859 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_rmxattr": 0, 2024-05-21T22:06:32.859 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_resetxattrs": 0, 2024-05-21T22:06:32.859 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_call": 0, 2024-05-21T22:06:32.859 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_watch": 0, 2024-05-21T22:06:32.859 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_notify": 0, 2024-05-21T22:06:32.859 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_src_cmpxattr": 0, 2024-05-21T22:06:32.860 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_pgls": 0, 2024-05-21T22:06:32.860 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_pgls_filter": 0, 2024-05-21T22:06:32.860 INFO:teuthology.orchestra.run.smithi164.stdout: "osdop_other": 0, 2024-05-21T22:06:32.860 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_active": 0, 2024-05-21T22:06:32.860 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_send": 0, 2024-05-21T22:06:32.860 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_resend": 0, 2024-05-21T22:06:32.860 INFO:teuthology.orchestra.run.smithi164.stdout: "linger_ping": 0, 2024-05-21T22:06:32.860 INFO:teuthology.orchestra.run.smithi164.stdout: "poolop_active": 0, 2024-05-21T22:06:32.860 INFO:teuthology.orchestra.run.smithi164.stdout: "poolop_send": 0, 2024-05-21T22:06:32.860 INFO:teuthology.orchestra.run.smithi164.stdout: "poolop_resend": 0, 2024-05-21T22:06:32.860 INFO:teuthology.orchestra.run.smithi164.stdout: "poolstat_active": 0, 2024-05-21T22:06:32.860 INFO:teuthology.orchestra.run.smithi164.stdout: "poolstat_send": 0, 2024-05-21T22:06:32.860 INFO:teuthology.orchestra.run.smithi164.stdout: "poolstat_resend": 0, 2024-05-21T22:06:32.860 INFO:teuthology.orchestra.run.smithi164.stdout: "statfs_active": 0, 2024-05-21T22:06:32.861 INFO:teuthology.orchestra.run.smithi164.stdout: "statfs_send": 0, 2024-05-21T22:06:32.861 INFO:teuthology.orchestra.run.smithi164.stdout: "statfs_resend": 0, 2024-05-21T22:06:32.861 INFO:teuthology.orchestra.run.smithi164.stdout: "command_active": 0, 2024-05-21T22:06:32.861 INFO:teuthology.orchestra.run.smithi164.stdout: "command_send": 0, 2024-05-21T22:06:32.861 INFO:teuthology.orchestra.run.smithi164.stdout: "command_resend": 0, 2024-05-21T22:06:32.861 INFO:teuthology.orchestra.run.smithi164.stdout: "map_epoch": 91, 2024-05-21T22:06:32.861 INFO:teuthology.orchestra.run.smithi164.stdout: "map_full": 0, 2024-05-21T22:06:32.861 INFO:teuthology.orchestra.run.smithi164.stdout: "map_inc": 11, 2024-05-21T22:06:32.861 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_sessions": 5, 2024-05-21T22:06:32.861 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_session_open": 5, 2024-05-21T22:06:32.861 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_session_close": 0, 2024-05-21T22:06:32.861 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_laggy": 0, 2024-05-21T22:06:32.861 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_wr": 0, 2024-05-21T22:06:32.862 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_rd": 2, 2024-05-21T22:06:32.862 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_del": 0 2024-05-21T22:06:32.862 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:32.862 INFO:teuthology.orchestra.run.smithi164.stdout: "oft": { 2024-05-21T22:06:32.862 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_objs": 0, 2024-05-21T22:06:32.862 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_kv_pairs": 0, 2024-05-21T22:06:32.862 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_updates": 0, 2024-05-21T22:06:32.862 INFO:teuthology.orchestra.run.smithi164.stdout: "omap_total_removes": 0 2024-05-21T22:06:32.862 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:32.862 INFO:teuthology.orchestra.run.smithi164.stdout: "osd-slow-ops": { 2024-05-21T22:06:32.862 INFO:teuthology.orchestra.run.smithi164.stdout: "slow_ops_count": 0 2024-05-21T22:06:32.862 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:32.862 INFO:teuthology.orchestra.run.smithi164.stdout: "purge_queue": { 2024-05-21T22:06:32.863 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing_ops": 0, 2024-05-21T22:06:32.863 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing_ops_high_water": 0, 2024-05-21T22:06:32.863 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing": 0, 2024-05-21T22:06:32.863 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executing_high_water": 0, 2024-05-21T22:06:32.863 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executed_ops": 0, 2024-05-21T22:06:32.863 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_executed": 0, 2024-05-21T22:06:32.863 INFO:teuthology.orchestra.run.smithi164.stdout: "pq_item_in_journal": 0 2024-05-21T22:06:32.863 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:32.863 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-21T22:06:32.863 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-05-21T22:06:32.863 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 104857600, 2024-05-21T22:06:32.863 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-05-21T22:06:32.863 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 2885, 2024-05-21T22:06:32.863 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 2370635, 2024-05-21T22:06:32.864 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-05-21T22:06:32.864 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 2885, 2024-05-21T22:06:32.864 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 0, 2024-05-21T22:06:32.864 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 0, 2024-05-21T22:06:32.864 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 2885, 2024-05-21T22:06:32.864 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 2370635, 2024-05-21T22:06:32.864 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-05-21T22:06:32.864 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:32.864 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:32.864 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:32.864 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-21T22:06:32.864 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:32.864 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-objecter_bytes": { 2024-05-21T22:06:32.865 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-05-21T22:06:32.865 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 104857600, 2024-05-21T22:06:32.865 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-05-21T22:06:32.865 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 0, 2024-05-21T22:06:32.865 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 0, 2024-05-21T22:06:32.865 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-05-21T22:06:32.865 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 0, 2024-05-21T22:06:32.865 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 2594, 2024-05-21T22:06:32.865 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 1125, 2024-05-21T22:06:32.865 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 2594, 2024-05-21T22:06:32.865 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 1125, 2024-05-21T22:06:32.865 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-05-21T22:06:32.865 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:32.865 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:32.866 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:32.866 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-21T22:06:32.866 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:32.866 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-objecter_ops": { 2024-05-21T22:06:32.866 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-05-21T22:06:32.866 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 1024, 2024-05-21T22:06:32.866 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-05-21T22:06:32.866 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 0, 2024-05-21T22:06:32.866 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 0, 2024-05-21T22:06:32.866 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-05-21T22:06:32.866 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 0, 2024-05-21T22:06:32.866 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 2594, 2024-05-21T22:06:32.867 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 2594, 2024-05-21T22:06:32.867 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 2594, 2024-05-21T22:06:32.867 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 2594, 2024-05-21T22:06:32.867 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-05-21T22:06:32.867 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:32.867 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:32.867 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:32.867 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-21T22:06:32.867 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:32.867 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-write_buf_throttle": { 2024-05-21T22:06:32.867 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-05-21T22:06:32.867 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 3758096384, 2024-05-21T22:06:32.867 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-05-21T22:06:32.868 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 0, 2024-05-21T22:06:32.868 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 0, 2024-05-21T22:06:32.868 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-05-21T22:06:32.868 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 0, 2024-05-21T22:06:32.868 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 0, 2024-05-21T22:06:32.868 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 0, 2024-05-21T22:06:32.868 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 0, 2024-05-21T22:06:32.868 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 0, 2024-05-21T22:06:32.868 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-05-21T22:06:32.868 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:32.868 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:32.868 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:32.868 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-21T22:06:32.869 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-05-21T22:06:32.869 INFO:teuthology.orchestra.run.smithi164.stdout: "throttle-write_buf_throttle-0x564f9990c6e0": { 2024-05-21T22:06:32.869 INFO:teuthology.orchestra.run.smithi164.stdout: "val": 0, 2024-05-21T22:06:32.869 INFO:teuthology.orchestra.run.smithi164.stdout: "max": 3758096384, 2024-05-21T22:06:32.869 INFO:teuthology.orchestra.run.smithi164.stdout: "get_started": 0, 2024-05-21T22:06:32.869 INFO:teuthology.orchestra.run.smithi164.stdout: "get": 0, 2024-05-21T22:06:32.869 INFO:teuthology.orchestra.run.smithi164.stdout: "get_sum": 0, 2024-05-21T22:06:32.869 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_fail": 0, 2024-05-21T22:06:32.869 INFO:teuthology.orchestra.run.smithi164.stdout: "get_or_fail_success": 0, 2024-05-21T22:06:32.869 INFO:teuthology.orchestra.run.smithi164.stdout: "take": 0, 2024-05-21T22:06:32.869 INFO:teuthology.orchestra.run.smithi164.stdout: "take_sum": 0, 2024-05-21T22:06:32.869 INFO:teuthology.orchestra.run.smithi164.stdout: "put": 0, 2024-05-21T22:06:32.869 INFO:teuthology.orchestra.run.smithi164.stdout: "put_sum": 0, 2024-05-21T22:06:32.869 INFO:teuthology.orchestra.run.smithi164.stdout: "wait": { 2024-05-21T22:06:32.870 INFO:teuthology.orchestra.run.smithi164.stdout: "avgcount": 0, 2024-05-21T22:06:32.870 INFO:teuthology.orchestra.run.smithi164.stdout: "sum": 0.000000000, 2024-05-21T22:06:32.870 INFO:teuthology.orchestra.run.smithi164.stdout: "avgtime": 0.000000000 2024-05-21T22:06:32.870 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-21T22:06:32.870 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-05-21T22:06:32.870 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-05-21T22:06:33.655 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-05-21T22:06:33.655 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-05-21T22:06:33.656 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update=0 2024-05-21T22:06:33.656 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update_receipt=0 2024-05-21T22:06:33.656 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=0 2024-05-21T22:06:33.656 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2024-05-21T22:06:33.656 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2024-05-21T22:06:33.656 DEBUG:tasks.check_counter:Getting stats from c 2024-05-21T22:06:33.656 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph daemon mds.c perf dump 2024-05-21T22:06:33.907 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.c/config 2024-05-21T22:06:33.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:06:33 smithi151 ceph-mon[31623]: pgmap v339: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T22:06:33.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:06:33 smithi184 ceph-mon[36942]: pgmap v339: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T22:06:34.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:06:33 smithi164 ceph-mon[36269]: pgmap v339: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T22:06:35.704 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:06:35 smithi184 ceph-mon[36942]: pgmap v340: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:06:35.704 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-05-21T22:06:35.704 INFO:teuthology.orchestra.run.smithi184.stdout: "AsyncMessenger::Worker-0": { 2024-05-21T22:06:35.704 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_messages": 112, 2024-05-21T22:06:35.704 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages": 144, 2024-05-21T22:06:35.704 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_bytes": 37361, 2024-05-21T22:06:35.704 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_bytes": 97074, 2024-05-21T22:06:35.704 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_created_connections": 15, 2024-05-21T22:06:35.705 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_active_connections": 3, 2024-05-21T22:06:35.705 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_total_time": 0.023867547, 2024-05-21T22:06:35.705 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_send_time": 0.006479420, 2024-05-21T22:06:35.705 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_recv_time": 0.012754210, 2024-05-21T22:06:35.705 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_fast_dispatch_time": 0.000592507, 2024-05-21T22:06:35.705 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T22:06:35.705 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 142, 2024-05-21T22:06:35.705 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.056873821, 2024-05-21T22:06:35.705 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000400519 2024-05-21T22:06:35.705 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:35.705 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_handle_ack_lat": { 2024-05-21T22:06:35.705 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 167, 2024-05-21T22:06:35.705 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000072022, 2024-05-21T22:06:35.705 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000431 2024-05-21T22:06:35.706 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:35.706 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_encrypted_bytes": 928, 2024-05-21T22:06:35.706 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_encrypted_bytes": 1216 2024-05-21T22:06:35.706 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:35.706 INFO:teuthology.orchestra.run.smithi184.stdout: "AsyncMessenger::Worker-1": { 2024-05-21T22:06:35.706 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_messages": 398, 2024-05-21T22:06:35.706 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages": 512, 2024-05-21T22:06:35.706 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_bytes": 1956729, 2024-05-21T22:06:35.706 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_bytes": 569199, 2024-05-21T22:06:35.706 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_created_connections": 12, 2024-05-21T22:06:35.706 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_active_connections": 5, 2024-05-21T22:06:35.706 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_total_time": 0.073242366, 2024-05-21T22:06:35.706 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_send_time": 0.025624621, 2024-05-21T22:06:35.707 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_recv_time": 0.053243458, 2024-05-21T22:06:35.707 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_fast_dispatch_time": 0.006956231, 2024-05-21T22:06:35.707 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T22:06:35.707 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 511, 2024-05-21T22:06:35.707 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.056161077, 2024-05-21T22:06:35.707 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000109904 2024-05-21T22:06:35.707 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:35.707 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_handle_ack_lat": { 2024-05-21T22:06:35.707 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 130, 2024-05-21T22:06:35.707 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000031300, 2024-05-21T22:06:35.707 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000240 2024-05-21T22:06:35.707 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:35.707 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_encrypted_bytes": 1928032, 2024-05-21T22:06:35.707 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_encrypted_bytes": 489888 2024-05-21T22:06:35.707 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:35.708 INFO:teuthology.orchestra.run.smithi184.stdout: "AsyncMessenger::Worker-2": { 2024-05-21T22:06:35.708 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_messages": 705, 2024-05-21T22:06:35.708 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages": 934, 2024-05-21T22:06:35.708 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_bytes": 204954, 2024-05-21T22:06:35.708 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_bytes": 943002, 2024-05-21T22:06:35.708 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_created_connections": 41, 2024-05-21T22:06:35.708 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_active_connections": 5, 2024-05-21T22:06:35.708 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_total_time": 0.095856618, 2024-05-21T22:06:35.708 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_send_time": 0.037125907, 2024-05-21T22:06:35.708 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_recv_time": 0.051763206, 2024-05-21T22:06:35.708 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_fast_dispatch_time": 0.005464892, 2024-05-21T22:06:35.708 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T22:06:35.708 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 932, 2024-05-21T22:06:35.708 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.051111489, 2024-05-21T22:06:35.708 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000054840 2024-05-21T22:06:35.709 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:35.709 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_handle_ack_lat": { 2024-05-21T22:06:35.709 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 1472, 2024-05-21T22:06:35.709 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000404784, 2024-05-21T22:06:35.709 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000274 2024-05-21T22:06:35.709 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:35.709 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_encrypted_bytes": 8688, 2024-05-21T22:06:35.709 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_encrypted_bytes": 188832 2024-05-21T22:06:35.709 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:35.709 INFO:teuthology.orchestra.run.smithi184.stdout: "cct": { 2024-05-21T22:06:35.709 INFO:teuthology.orchestra.run.smithi184.stdout: "total_workers": 1, 2024-05-21T22:06:35.709 INFO:teuthology.orchestra.run.smithi184.stdout: "unhealthy_workers": 0 2024-05-21T22:06:35.709 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:35.709 INFO:teuthology.orchestra.run.smithi184.stdout: "finisher-MDSRank": { 2024-05-21T22:06:35.710 INFO:teuthology.orchestra.run.smithi184.stdout: "queue_len": 0, 2024-05-21T22:06:35.710 INFO:teuthology.orchestra.run.smithi184.stdout: "complete_latency": { 2024-05-21T22:06:35.710 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 66, 2024-05-21T22:06:35.710 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.001454951, 2024-05-21T22:06:35.710 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000022044 2024-05-21T22:06:35.710 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-05-21T22:06:35.710 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:35.710 INFO:teuthology.orchestra.run.smithi184.stdout: "finisher-PurgeQueue": { 2024-05-21T22:06:35.710 INFO:teuthology.orchestra.run.smithi184.stdout: "queue_len": 0, 2024-05-21T22:06:35.710 INFO:teuthology.orchestra.run.smithi184.stdout: "complete_latency": { 2024-05-21T22:06:35.710 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 2, 2024-05-21T22:06:35.711 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000008652, 2024-05-21T22:06:35.711 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000004326 2024-05-21T22:06:35.711 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-05-21T22:06:35.711 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:35.712 INFO:teuthology.orchestra.run.smithi184.stdout: "mds": { 2024-05-21T22:06:35.712 INFO:teuthology.orchestra.run.smithi184.stdout: "request": 9, 2024-05-21T22:06:35.712 INFO:teuthology.orchestra.run.smithi184.stdout: "reply": 6, 2024-05-21T22:06:35.712 INFO:teuthology.orchestra.run.smithi184.stdout: "reply_latency": { 2024-05-21T22:06:35.712 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 6, 2024-05-21T22:06:35.712 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.002467999, 2024-05-21T22:06:35.712 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000411333 2024-05-21T22:06:35.712 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:35.712 INFO:teuthology.orchestra.run.smithi184.stdout: "slow_reply": 0, 2024-05-21T22:06:35.712 INFO:teuthology.orchestra.run.smithi184.stdout: "forward": 3, 2024-05-21T22:06:35.712 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_fetch_complete": 0, 2024-05-21T22:06:35.712 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_fetch_keys": 0, 2024-05-21T22:06:35.712 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_commit": 11, 2024-05-21T22:06:35.712 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_split": 0, 2024-05-21T22:06:35.713 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_merge": 0, 2024-05-21T22:06:35.713 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes": 16, 2024-05-21T22:06:35.713 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_top": 0, 2024-05-21T22:06:35.713 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_bottom": 16, 2024-05-21T22:06:35.713 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_pin_tail": 0, 2024-05-21T22:06:35.713 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_pinned": 13, 2024-05-21T22:06:35.713 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_expired": 4, 2024-05-21T22:06:35.713 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_with_caps": 1, 2024-05-21T22:06:35.713 INFO:teuthology.orchestra.run.smithi184.stdout: "caps": 2, 2024-05-21T22:06:35.713 INFO:teuthology.orchestra.run.smithi184.stdout: "subtrees": 3, 2024-05-21T22:06:35.713 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse": 25, 2024-05-21T22:06:35.713 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_hit": 19, 2024-05-21T22:06:35.713 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_forward": 1, 2024-05-21T22:06:35.714 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_discover": 4, 2024-05-21T22:06:35.714 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_dir_fetch": 0, 2024-05-21T22:06:35.714 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_remote_ino": 0, 2024-05-21T22:06:35.714 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_lock": 0, 2024-05-21T22:06:35.714 INFO:teuthology.orchestra.run.smithi184.stdout: "load_cent": 1000, 2024-05-21T22:06:35.714 INFO:teuthology.orchestra.run.smithi184.stdout: "q": 1, 2024-05-21T22:06:35.714 INFO:teuthology.orchestra.run.smithi184.stdout: "exported": 0, 2024-05-21T22:06:35.714 INFO:teuthology.orchestra.run.smithi184.stdout: "exported_inodes": 0, 2024-05-21T22:06:35.714 INFO:teuthology.orchestra.run.smithi184.stdout: "imported": 0, 2024-05-21T22:06:35.714 INFO:teuthology.orchestra.run.smithi184.stdout: "imported_inodes": 0, 2024-05-21T22:06:35.715 INFO:teuthology.orchestra.run.smithi184.stdout: "openino_dir_fetch": 0, 2024-05-21T22:06:35.715 INFO:teuthology.orchestra.run.smithi184.stdout: "openino_backtrace_fetch": 0, 2024-05-21T22:06:35.715 INFO:teuthology.orchestra.run.smithi184.stdout: "openino_peer_discover": 0, 2024-05-21T22:06:35.715 INFO:teuthology.orchestra.run.smithi184.stdout: "root_rfiles": 4, 2024-05-21T22:06:35.715 INFO:teuthology.orchestra.run.smithi184.stdout: "root_rbytes": 234, 2024-05-21T22:06:35.715 INFO:teuthology.orchestra.run.smithi184.stdout: "root_rsnaps": 7, 2024-05-21T22:06:35.715 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_backtrace_fetch": 0, 2024-05-21T22:06:35.715 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_set_tag": 0, 2024-05-21T22:06:35.715 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_backtrace_repaired": 0, 2024-05-21T22:06:35.715 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_inotable_repaired": 0, 2024-05-21T22:06:35.715 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_dir_inodes": 0, 2024-05-21T22:06:35.715 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_dir_base_inodes": 0, 2024-05-21T22:06:35.715 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_dirfrag_rstats": 0, 2024-05-21T22:06:35.716 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_file_inodes": 0, 2024-05-21T22:06:35.716 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_inode_file_caps": 0, 2024-05-21T22:06:35.716 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_revoke": 2, 2024-05-21T22:06:35.716 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_grant": 0, 2024-05-21T22:06:35.716 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_trunc": 0, 2024-05-21T22:06:35.716 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-21T22:06:35.716 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-21T22:06:35.716 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_caps": 2, 2024-05-21T22:06:35.716 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_caps_dirty": 0, 2024-05-21T22:06:35.716 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_cap_release": 0, 2024-05-21T22:06:35.716 INFO:teuthology.orchestra.run.smithi184.stdout: "process_request_cap_release": 0 2024-05-21T22:06:35.716 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:35.716 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_cache": { 2024-05-21T22:06:35.717 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_update": 0, 2024-05-21T22:06:35.717 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_update_receipt": 14, 2024-05-21T22:06:35.717 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_try_discover": 9, 2024-05-21T22:06:35.717 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_send_discover": 10, 2024-05-21T22:06:35.717 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_handle_discover": 0, 2024-05-21T22:06:35.717 INFO:teuthology.orchestra.run.smithi184.stdout: "num_strays": 0, 2024-05-21T22:06:35.717 INFO:teuthology.orchestra.run.smithi184.stdout: "num_strays_delayed": 0, 2024-05-21T22:06:35.717 INFO:teuthology.orchestra.run.smithi184.stdout: "num_strays_enqueuing": 0, 2024-05-21T22:06:35.717 INFO:teuthology.orchestra.run.smithi184.stdout: "strays_created": 0, 2024-05-21T22:06:35.717 INFO:teuthology.orchestra.run.smithi184.stdout: "strays_enqueued": 0, 2024-05-21T22:06:35.717 INFO:teuthology.orchestra.run.smithi184.stdout: "strays_reintegrated": 0, 2024-05-21T22:06:35.717 INFO:teuthology.orchestra.run.smithi184.stdout: "strays_migrated": 0, 2024-05-21T22:06:35.717 INFO:teuthology.orchestra.run.smithi184.stdout: "num_recovering_processing": 0, 2024-05-21T22:06:35.717 INFO:teuthology.orchestra.run.smithi184.stdout: "num_recovering_enqueued": 0, 2024-05-21T22:06:35.718 INFO:teuthology.orchestra.run.smithi184.stdout: "num_recovering_prioritized": 0, 2024-05-21T22:06:35.718 INFO:teuthology.orchestra.run.smithi184.stdout: "recovery_started": 0, 2024-05-21T22:06:35.718 INFO:teuthology.orchestra.run.smithi184.stdout: "recovery_completed": 0, 2024-05-21T22:06:35.718 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_quiesce_path": 2, 2024-05-21T22:06:35.718 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_quiesce_inode": 11, 2024-05-21T22:06:35.718 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_enqueue_scrub": 0, 2024-05-21T22:06:35.718 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_exportdir": 0, 2024-05-21T22:06:35.718 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_flush": 0, 2024-05-21T22:06:35.718 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_fragmentdir": 0, 2024-05-21T22:06:35.718 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_fragstats": 0, 2024-05-21T22:06:35.718 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_inodestats": 0 2024-05-21T22:06:35.718 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:35.718 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_log": { 2024-05-21T22:06:35.718 INFO:teuthology.orchestra.run.smithi184.stdout: "evlrg": 0, 2024-05-21T22:06:35.719 INFO:teuthology.orchestra.run.smithi184.stdout: "evadd": 8, 2024-05-21T22:06:35.719 INFO:teuthology.orchestra.run.smithi184.stdout: "evex": 0, 2024-05-21T22:06:35.719 INFO:teuthology.orchestra.run.smithi184.stdout: "evtrm": 0, 2024-05-21T22:06:35.719 INFO:teuthology.orchestra.run.smithi184.stdout: "ev": 8, 2024-05-21T22:06:35.719 INFO:teuthology.orchestra.run.smithi184.stdout: "evexg": 0, 2024-05-21T22:06:35.719 INFO:teuthology.orchestra.run.smithi184.stdout: "evexd": 0, 2024-05-21T22:06:35.719 INFO:teuthology.orchestra.run.smithi184.stdout: "segadd": 2, 2024-05-21T22:06:35.719 INFO:teuthology.orchestra.run.smithi184.stdout: "segex": 0, 2024-05-21T22:06:35.719 INFO:teuthology.orchestra.run.smithi184.stdout: "segtrm": 0, 2024-05-21T22:06:35.719 INFO:teuthology.orchestra.run.smithi184.stdout: "seg": 2, 2024-05-21T22:06:35.719 INFO:teuthology.orchestra.run.smithi184.stdout: "segmjr": 2, 2024-05-21T22:06:35.719 INFO:teuthology.orchestra.run.smithi184.stdout: "segexg": 0, 2024-05-21T22:06:35.719 INFO:teuthology.orchestra.run.smithi184.stdout: "segexd": 0, 2024-05-21T22:06:35.719 INFO:teuthology.orchestra.run.smithi184.stdout: "expos": 4194304, 2024-05-21T22:06:35.720 INFO:teuthology.orchestra.run.smithi184.stdout: "wrpos": 4196657, 2024-05-21T22:06:35.720 INFO:teuthology.orchestra.run.smithi184.stdout: "rdpos": 0, 2024-05-21T22:06:35.720 INFO:teuthology.orchestra.run.smithi184.stdout: "jlat": { 2024-05-21T22:06:35.720 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 7, 2024-05-21T22:06:35.720 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.018505467, 2024-05-21T22:06:35.720 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.002643638 2024-05-21T22:06:35.720 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:35.720 INFO:teuthology.orchestra.run.smithi184.stdout: "replayed": 0 2024-05-21T22:06:35.720 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:35.720 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_mem": { 2024-05-21T22:06:35.720 INFO:teuthology.orchestra.run.smithi184.stdout: "ino": 20, 2024-05-21T22:06:35.720 INFO:teuthology.orchestra.run.smithi184.stdout: "ino+": 22, 2024-05-21T22:06:35.720 INFO:teuthology.orchestra.run.smithi184.stdout: "ino-": 2, 2024-05-21T22:06:35.721 INFO:teuthology.orchestra.run.smithi184.stdout: "dir": 19, 2024-05-21T22:06:35.721 INFO:teuthology.orchestra.run.smithi184.stdout: "dir+": 23, 2024-05-21T22:06:35.721 INFO:teuthology.orchestra.run.smithi184.stdout: "dir-": 4, 2024-05-21T22:06:35.721 INFO:teuthology.orchestra.run.smithi184.stdout: "dn": 16, 2024-05-21T22:06:35.721 INFO:teuthology.orchestra.run.smithi184.stdout: "dn+": 20, 2024-05-21T22:06:35.721 INFO:teuthology.orchestra.run.smithi184.stdout: "dn-": 4, 2024-05-21T22:06:35.721 INFO:teuthology.orchestra.run.smithi184.stdout: "cap": 2, 2024-05-21T22:06:35.721 INFO:teuthology.orchestra.run.smithi184.stdout: "cap+": 4, 2024-05-21T22:06:35.721 INFO:teuthology.orchestra.run.smithi184.stdout: "cap-": 2, 2024-05-21T22:06:35.721 INFO:teuthology.orchestra.run.smithi184.stdout: "rss": 53044, 2024-05-21T22:06:35.721 INFO:teuthology.orchestra.run.smithi184.stdout: "heap": 223608 2024-05-21T22:06:35.722 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:35.722 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_server": { 2024-05-21T22:06:35.722 INFO:teuthology.orchestra.run.smithi184.stdout: "dispatch_client_request": 12, 2024-05-21T22:06:35.722 INFO:teuthology.orchestra.run.smithi184.stdout: "dispatch_server_request": 0, 2024-05-21T22:06:35.722 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_request": 9, 2024-05-21T22:06:35.722 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_session": 26, 2024-05-21T22:06:35.722 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_peer_request": 12, 2024-05-21T22:06:35.722 INFO:teuthology.orchestra.run.smithi184.stdout: "req_create_latency": { 2024-05-21T22:06:35.722 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:35.722 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:35.722 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:35.722 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:35.722 INFO:teuthology.orchestra.run.smithi184.stdout: "req_getattr_latency": { 2024-05-21T22:06:35.723 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 4, 2024-05-21T22:06:35.723 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.002146824, 2024-05-21T22:06:35.723 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000536706 2024-05-21T22:06:35.723 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:35.723 INFO:teuthology.orchestra.run.smithi184.stdout: "req_getfilelock_latency": { 2024-05-21T22:06:35.723 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:35.723 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:35.723 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:35.723 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:35.723 INFO:teuthology.orchestra.run.smithi184.stdout: "req_link_latency": { 2024-05-21T22:06:35.723 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:35.723 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:35.723 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:35.723 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:35.724 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookup_latency": { 2024-05-21T22:06:35.724 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 1, 2024-05-21T22:06:35.724 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000207208, 2024-05-21T22:06:35.724 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000207208 2024-05-21T22:06:35.724 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:35.724 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookuphash_latency": { 2024-05-21T22:06:35.724 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:35.724 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:35.724 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:35.724 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:35.724 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookupino_latency": { 2024-05-21T22:06:35.724 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 1, 2024-05-21T22:06:35.724 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000113967, 2024-05-21T22:06:35.724 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000113967 2024-05-21T22:06:35.725 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:35.725 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookupname_latency": { 2024-05-21T22:06:35.725 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:35.725 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:35.725 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:35.725 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:35.725 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookupparent_latency": { 2024-05-21T22:06:35.725 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:35.725 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:35.725 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:35.725 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:35.726 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookupsnap_latency": { 2024-05-21T22:06:35.726 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:35.726 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:35.726 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:35.726 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:35.726 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lssnap_latency": { 2024-05-21T22:06:35.727 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:35.727 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:35.727 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:35.727 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:35.727 INFO:teuthology.orchestra.run.smithi184.stdout: "req_mkdir_latency": { 2024-05-21T22:06:35.727 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:35.727 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:35.728 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:35.728 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:35.728 INFO:teuthology.orchestra.run.smithi184.stdout: "req_mknod_latency": { 2024-05-21T22:06:35.728 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:35.728 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:35.728 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:35.728 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:35.728 INFO:teuthology.orchestra.run.smithi184.stdout: "req_mksnap_latency": { 2024-05-21T22:06:35.728 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:35.728 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:35.728 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:35.728 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:35.729 INFO:teuthology.orchestra.run.smithi184.stdout: "req_open_latency": { 2024-05-21T22:06:35.729 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:35.729 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:35.729 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:35.729 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:35.729 INFO:teuthology.orchestra.run.smithi184.stdout: "req_readdir_latency": { 2024-05-21T22:06:35.729 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:35.729 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:35.729 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:35.729 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:35.729 INFO:teuthology.orchestra.run.smithi184.stdout: "req_rename_latency": { 2024-05-21T22:06:35.729 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:35.729 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:35.730 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:35.730 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:35.730 INFO:teuthology.orchestra.run.smithi184.stdout: "req_renamesnap_latency": { 2024-05-21T22:06:35.730 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:35.730 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:35.730 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:35.730 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:35.730 INFO:teuthology.orchestra.run.smithi184.stdout: "req_snapdiff_latency": { 2024-05-21T22:06:35.731 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:35.731 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:35.731 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:35.731 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:35.731 INFO:teuthology.orchestra.run.smithi184.stdout: "req_rmdir_latency": { 2024-05-21T22:06:35.731 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:35.731 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:35.731 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:35.731 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:35.731 INFO:teuthology.orchestra.run.smithi184.stdout: "req_rmsnap_latency": { 2024-05-21T22:06:35.732 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:35.732 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:35.732 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:35.732 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:35.732 INFO:teuthology.orchestra.run.smithi184.stdout: "req_rmxattr_latency": { 2024-05-21T22:06:35.732 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:35.732 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:35.732 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:35.732 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:35.733 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setattr_latency": { 2024-05-21T22:06:35.733 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:35.733 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:35.733 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:35.733 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:35.733 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setdirlayout_latency": { 2024-05-21T22:06:35.733 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:35.733 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:35.733 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:35.733 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:35.733 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setfilelock_latency": { 2024-05-21T22:06:35.733 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:35.733 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:35.734 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:35.734 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:35.734 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setlayout_latency": { 2024-05-21T22:06:35.734 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:35.734 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:35.734 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:35.734 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:35.734 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setxattr_latency": { 2024-05-21T22:06:35.734 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:35.734 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:35.734 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:35.734 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:35.735 INFO:teuthology.orchestra.run.smithi184.stdout: "req_symlink_latency": { 2024-05-21T22:06:35.735 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:35.735 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:35.735 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:35.735 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:35.735 INFO:teuthology.orchestra.run.smithi184.stdout: "req_unlink_latency": { 2024-05-21T22:06:35.735 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:35.735 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:35.735 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:35.735 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:35.735 INFO:teuthology.orchestra.run.smithi184.stdout: "cap_revoke_eviction": 0, 2024-05-21T22:06:35.735 INFO:teuthology.orchestra.run.smithi184.stdout: "cap_acquisition_throttle": 0, 2024-05-21T22:06:35.735 INFO:teuthology.orchestra.run.smithi184.stdout: "req_getvxattr_latency": { 2024-05-21T22:06:35.735 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:35.736 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:35.736 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:35.736 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-05-21T22:06:35.736 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:35.736 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_sessions": { 2024-05-21T22:06:35.736 INFO:teuthology.orchestra.run.smithi184.stdout: "session_count": 2, 2024-05-21T22:06:35.736 INFO:teuthology.orchestra.run.smithi184.stdout: "session_add": 4, 2024-05-21T22:06:35.736 INFO:teuthology.orchestra.run.smithi184.stdout: "session_remove": 2, 2024-05-21T22:06:35.736 INFO:teuthology.orchestra.run.smithi184.stdout: "sessions_open": 2, 2024-05-21T22:06:35.736 INFO:teuthology.orchestra.run.smithi184.stdout: "sessions_stale": 0, 2024-05-21T22:06:35.736 INFO:teuthology.orchestra.run.smithi184.stdout: "total_load": 4, 2024-05-21T22:06:35.736 INFO:teuthology.orchestra.run.smithi184.stdout: "average_load": 1, 2024-05-21T22:06:35.736 INFO:teuthology.orchestra.run.smithi184.stdout: "avg_session_uptime": 620, 2024-05-21T22:06:35.737 INFO:teuthology.orchestra.run.smithi184.stdout: "mdthresh_evicted": 0 2024-05-21T22:06:35.737 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:35.737 INFO:teuthology.orchestra.run.smithi184.stdout: "mempool": { 2024-05-21T22:06:35.737 INFO:teuthology.orchestra.run.smithi184.stdout: "bloom_filter_bytes": 0, 2024-05-21T22:06:35.737 INFO:teuthology.orchestra.run.smithi184.stdout: "bloom_filter_items": 0, 2024-05-21T22:06:35.737 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_alloc_bytes": 0, 2024-05-21T22:06:35.737 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_alloc_items": 0, 2024-05-21T22:06:35.737 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_data_bytes": 0, 2024-05-21T22:06:35.737 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_data_items": 0, 2024-05-21T22:06:35.737 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-21T22:06:35.738 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_onode_items": 0, 2024-05-21T22:06:35.738 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-21T22:06:35.738 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_meta_items": 0, 2024-05-21T22:06:35.738 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_other_bytes": 0, 2024-05-21T22:06:35.738 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_other_items": 0, 2024-05-21T22:06:35.738 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-21T22:06:35.738 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_buffer_items": 0, 2024-05-21T22:06:35.738 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_extent_bytes": 0, 2024-05-21T22:06:35.738 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_extent_items": 0, 2024-05-21T22:06:35.738 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_blob_bytes": 0, 2024-05-21T22:06:35.738 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_blob_items": 0, 2024-05-21T22:06:35.739 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-21T22:06:35.739 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_shared_blob_items": 0, 2024-05-21T22:06:35.739 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-21T22:06:35.739 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_inline_bl_items": 0, 2024-05-21T22:06:35.739 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_fsck_bytes": 0, 2024-05-21T22:06:35.739 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_fsck_items": 0, 2024-05-21T22:06:35.739 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_txc_bytes": 0, 2024-05-21T22:06:35.739 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_txc_items": 0, 2024-05-21T22:06:35.739 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-21T22:06:35.739 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_deferred_items": 0, 2024-05-21T22:06:35.739 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_bytes": 0, 2024-05-21T22:06:35.739 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_items": 0, 2024-05-21T22:06:35.739 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_bytes": 0, 2024-05-21T22:06:35.740 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_items": 0, 2024-05-21T22:06:35.740 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_reader_bytes": 0, 2024-05-21T22:06:35.740 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_reader_items": 0, 2024-05-21T22:06:35.740 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_writer_bytes": 0, 2024-05-21T22:06:35.740 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_writer_items": 0, 2024-05-21T22:06:35.740 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_anon_bytes": 272837, 2024-05-21T22:06:35.740 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_anon_items": 102, 2024-05-21T22:06:35.740 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_meta_bytes": 0, 2024-05-21T22:06:35.740 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_meta_items": 0, 2024-05-21T22:06:35.740 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_bytes": 0, 2024-05-21T22:06:35.740 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_items": 0, 2024-05-21T22:06:35.740 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_mapbl_bytes": 0, 2024-05-21T22:06:35.740 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_mapbl_items": 0, 2024-05-21T22:06:35.741 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_pglog_bytes": 0, 2024-05-21T22:06:35.741 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_pglog_items": 0, 2024-05-21T22:06:35.741 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_bytes": 8128, 2024-05-21T22:06:35.741 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_items": 169, 2024-05-21T22:06:35.741 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_mapping_bytes": 0, 2024-05-21T22:06:35.741 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_mapping_items": 0, 2024-05-21T22:06:35.741 INFO:teuthology.orchestra.run.smithi184.stdout: "pgmap_bytes": 0, 2024-05-21T22:06:35.741 INFO:teuthology.orchestra.run.smithi184.stdout: "pgmap_items": 0, 2024-05-21T22:06:35.741 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_co_bytes": 83461, 2024-05-21T22:06:35.741 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_co_items": 345, 2024-05-21T22:06:35.741 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_1_bytes": 0, 2024-05-21T22:06:35.741 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_1_items": 0, 2024-05-21T22:06:35.742 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_2_bytes": 0, 2024-05-21T22:06:35.742 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_2_items": 0 2024-05-21T22:06:35.742 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:35.742 INFO:teuthology.orchestra.run.smithi184.stdout: "objecter": { 2024-05-21T22:06:35.742 INFO:teuthology.orchestra.run.smithi184.stdout: "op_active": 0, 2024-05-21T22:06:35.742 INFO:teuthology.orchestra.run.smithi184.stdout: "op_laggy": 0, 2024-05-21T22:06:35.742 INFO:teuthology.orchestra.run.smithi184.stdout: "op_send": 47, 2024-05-21T22:06:35.742 INFO:teuthology.orchestra.run.smithi184.stdout: "op_send_bytes": 12980, 2024-05-21T22:06:35.742 INFO:teuthology.orchestra.run.smithi184.stdout: "op_resend": 30, 2024-05-21T22:06:35.742 INFO:teuthology.orchestra.run.smithi184.stdout: "op_reply": 47, 2024-05-21T22:06:35.742 INFO:teuthology.orchestra.run.smithi184.stdout: "op_latency": { 2024-05-21T22:06:35.743 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 47, 2024-05-21T22:06:35.743 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.403991725, 2024-05-21T22:06:35.743 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.008595568 2024-05-21T22:06:35.743 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:35.743 INFO:teuthology.orchestra.run.smithi184.stdout: "op_inflight": 0, 2024-05-21T22:06:35.743 INFO:teuthology.orchestra.run.smithi184.stdout: "oplen_avg": { 2024-05-21T22:06:35.743 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 47, 2024-05-21T22:06:35.743 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 71 2024-05-21T22:06:35.743 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:35.743 INFO:teuthology.orchestra.run.smithi184.stdout: "op": 47, 2024-05-21T22:06:35.743 INFO:teuthology.orchestra.run.smithi184.stdout: "op_r": 0, 2024-05-21T22:06:35.743 INFO:teuthology.orchestra.run.smithi184.stdout: "op_w": 47, 2024-05-21T22:06:35.743 INFO:teuthology.orchestra.run.smithi184.stdout: "op_rmw": 0, 2024-05-21T22:06:35.744 INFO:teuthology.orchestra.run.smithi184.stdout: "op_pg": 0, 2024-05-21T22:06:35.744 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_stat": 0, 2024-05-21T22:06:35.744 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_create": 10, 2024-05-21T22:06:35.744 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_read": 0, 2024-05-21T22:06:35.744 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_write": 7, 2024-05-21T22:06:35.744 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_writefull": 9, 2024-05-21T22:06:35.744 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_writesame": 0, 2024-05-21T22:06:35.744 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_append": 0, 2024-05-21T22:06:35.744 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_zero": 0, 2024-05-21T22:06:35.744 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_truncate": 0, 2024-05-21T22:06:35.744 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_delete": 6, 2024-05-21T22:06:35.744 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_mapext": 0, 2024-05-21T22:06:35.744 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_sparse_read": 0, 2024-05-21T22:06:35.744 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_clonerange": 0, 2024-05-21T22:06:35.745 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_getxattr": 0, 2024-05-21T22:06:35.745 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_setxattr": 20, 2024-05-21T22:06:35.745 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_cmpxattr": 0, 2024-05-21T22:06:35.745 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_rmxattr": 0, 2024-05-21T22:06:35.745 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_resetxattrs": 0, 2024-05-21T22:06:35.745 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_call": 0, 2024-05-21T22:06:35.745 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_watch": 0, 2024-05-21T22:06:35.745 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_notify": 0, 2024-05-21T22:06:35.745 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_src_cmpxattr": 0, 2024-05-21T22:06:35.745 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_pgls": 0, 2024-05-21T22:06:35.745 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_pgls_filter": 0, 2024-05-21T22:06:35.745 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_other": 0, 2024-05-21T22:06:35.745 INFO:teuthology.orchestra.run.smithi184.stdout: "linger_active": 0, 2024-05-21T22:06:35.745 INFO:teuthology.orchestra.run.smithi184.stdout: "linger_send": 0, 2024-05-21T22:06:35.745 INFO:teuthology.orchestra.run.smithi184.stdout: "linger_resend": 0, 2024-05-21T22:06:35.746 INFO:teuthology.orchestra.run.smithi184.stdout: "linger_ping": 0, 2024-05-21T22:06:35.746 INFO:teuthology.orchestra.run.smithi184.stdout: "poolop_active": 0, 2024-05-21T22:06:35.746 INFO:teuthology.orchestra.run.smithi184.stdout: "poolop_send": 0, 2024-05-21T22:06:35.746 INFO:teuthology.orchestra.run.smithi184.stdout: "poolop_resend": 0, 2024-05-21T22:06:35.746 INFO:teuthology.orchestra.run.smithi184.stdout: "poolstat_active": 0, 2024-05-21T22:06:35.746 INFO:teuthology.orchestra.run.smithi184.stdout: "poolstat_send": 0, 2024-05-21T22:06:35.746 INFO:teuthology.orchestra.run.smithi184.stdout: "poolstat_resend": 0, 2024-05-21T22:06:35.746 INFO:teuthology.orchestra.run.smithi184.stdout: "statfs_active": 0, 2024-05-21T22:06:35.746 INFO:teuthology.orchestra.run.smithi184.stdout: "statfs_send": 0, 2024-05-21T22:06:35.746 INFO:teuthology.orchestra.run.smithi184.stdout: "statfs_resend": 0, 2024-05-21T22:06:35.746 INFO:teuthology.orchestra.run.smithi184.stdout: "command_active": 0, 2024-05-21T22:06:35.746 INFO:teuthology.orchestra.run.smithi184.stdout: "command_send": 0, 2024-05-21T22:06:35.746 INFO:teuthology.orchestra.run.smithi184.stdout: "command_resend": 0, 2024-05-21T22:06:35.747 INFO:teuthology.orchestra.run.smithi184.stdout: "map_epoch": 91, 2024-05-21T22:06:35.747 INFO:teuthology.orchestra.run.smithi184.stdout: "map_full": 0, 2024-05-21T22:06:35.747 INFO:teuthology.orchestra.run.smithi184.stdout: "map_inc": 11, 2024-05-21T22:06:35.747 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_sessions": 8, 2024-05-21T22:06:35.747 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_session_open": 8, 2024-05-21T22:06:35.747 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_session_close": 0, 2024-05-21T22:06:35.747 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_laggy": 0, 2024-05-21T22:06:35.747 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_wr": 17, 2024-05-21T22:06:35.747 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_rd": 0, 2024-05-21T22:06:35.747 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_del": 2 2024-05-21T22:06:35.747 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:35.747 INFO:teuthology.orchestra.run.smithi184.stdout: "oft": { 2024-05-21T22:06:35.747 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_total_objs": 1, 2024-05-21T22:06:35.748 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_total_kv_pairs": 0, 2024-05-21T22:06:35.748 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_total_updates": 0, 2024-05-21T22:06:35.748 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_total_removes": 0 2024-05-21T22:06:35.748 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:35.748 INFO:teuthology.orchestra.run.smithi184.stdout: "osd-slow-ops": { 2024-05-21T22:06:35.748 INFO:teuthology.orchestra.run.smithi184.stdout: "slow_ops_count": 13 2024-05-21T22:06:35.748 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:35.748 INFO:teuthology.orchestra.run.smithi184.stdout: "purge_queue": { 2024-05-21T22:06:35.748 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executing_ops": 0, 2024-05-21T22:06:35.748 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executing_ops_high_water": 0, 2024-05-21T22:06:35.748 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executing": 0, 2024-05-21T22:06:35.748 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executing_high_water": 0, 2024-05-21T22:06:35.748 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executed_ops": 0, 2024-05-21T22:06:35.748 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executed": 0, 2024-05-21T22:06:35.748 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_item_in_journal": 0 2024-05-21T22:06:35.749 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:35.749 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-21T22:06:35.749 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2024-05-21T22:06:35.749 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 104857600, 2024-05-21T22:06:35.749 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2024-05-21T22:06:35.749 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 1211, 2024-05-21T22:06:35.749 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 2069995, 2024-05-21T22:06:35.749 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2024-05-21T22:06:35.749 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 1211, 2024-05-21T22:06:35.749 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 0, 2024-05-21T22:06:35.749 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 0, 2024-05-21T22:06:35.749 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 1211, 2024-05-21T22:06:35.749 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 2069995, 2024-05-21T22:06:35.750 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2024-05-21T22:06:35.750 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:35.750 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:35.750 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:35.750 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-05-21T22:06:35.750 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:35.750 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-objecter_bytes": { 2024-05-21T22:06:35.750 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2024-05-21T22:06:35.750 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 104857600, 2024-05-21T22:06:35.750 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2024-05-21T22:06:35.750 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 0, 2024-05-21T22:06:35.750 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 0, 2024-05-21T22:06:35.750 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2024-05-21T22:06:35.750 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 0, 2024-05-21T22:06:35.751 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 47, 2024-05-21T22:06:35.751 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 12980, 2024-05-21T22:06:35.751 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 47, 2024-05-21T22:06:35.751 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 12980, 2024-05-21T22:06:35.751 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2024-05-21T22:06:35.751 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:35.751 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:35.751 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:35.751 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-05-21T22:06:35.751 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:35.751 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-objecter_ops": { 2024-05-21T22:06:35.751 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2024-05-21T22:06:35.751 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 1024, 2024-05-21T22:06:35.751 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2024-05-21T22:06:35.752 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 0, 2024-05-21T22:06:35.752 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 0, 2024-05-21T22:06:35.752 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2024-05-21T22:06:35.752 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 0, 2024-05-21T22:06:35.752 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 47, 2024-05-21T22:06:35.752 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 47, 2024-05-21T22:06:35.752 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 47, 2024-05-21T22:06:35.752 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 47, 2024-05-21T22:06:35.752 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2024-05-21T22:06:35.752 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:35.752 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:35.752 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:35.752 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-05-21T22:06:35.752 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:35.752 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-write_buf_throttle": { 2024-05-21T22:06:35.753 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2024-05-21T22:06:35.753 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 3758096384, 2024-05-21T22:06:35.753 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2024-05-21T22:06:35.753 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 0, 2024-05-21T22:06:35.753 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 0, 2024-05-21T22:06:35.753 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2024-05-21T22:06:35.753 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 0, 2024-05-21T22:06:35.753 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 0, 2024-05-21T22:06:35.753 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 0, 2024-05-21T22:06:35.753 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 0, 2024-05-21T22:06:35.753 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 0, 2024-05-21T22:06:35.753 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2024-05-21T22:06:35.753 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:35.753 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:35.753 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:35.754 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-05-21T22:06:35.754 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:35.754 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-write_buf_throttle-0x561f8fb2a500": { 2024-05-21T22:06:35.754 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2024-05-21T22:06:35.754 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 3758096384, 2024-05-21T22:06:35.754 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2024-05-21T22:06:35.754 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 8, 2024-05-21T22:06:35.754 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 2353, 2024-05-21T22:06:35.754 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2024-05-21T22:06:35.754 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 8, 2024-05-21T22:06:35.754 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 0, 2024-05-21T22:06:35.755 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 0, 2024-05-21T22:06:35.755 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 7, 2024-05-21T22:06:35.755 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 2353, 2024-05-21T22:06:35.755 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2024-05-21T22:06:35.755 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:35.755 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:35.755 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:35.755 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-05-21T22:06:35.755 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-05-21T22:06:35.755 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-05-21T22:06:35.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:06:35 smithi151 ceph-mon[31623]: pgmap v340: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:06:36.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:06:35 smithi164 ceph-mon[36269]: pgmap v340: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:06:36.449 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-05-21T22:06:36.449 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-05-21T22:06:36.449 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2024-05-21T22:06:36.449 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=14 2024-05-21T22:06:36.449 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=7 2024-05-21T22:06:36.449 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-05-21T22:06:36.449 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-05-21T22:06:36.449 DEBUG:tasks.check_counter:Getting stats from f 2024-05-21T22:06:36.449 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph daemon mds.f perf dump 2024-05-21T22:06:36.700 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.c/config 2024-05-21T22:06:37.847 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:06:37 smithi184 ceph-mon[36942]: pgmap v341: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:06:37.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:06:37 smithi151 ceph-mon[31623]: pgmap v341: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:06:38.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:06:37 smithi164 ceph-mon[36269]: pgmap v341: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:06:38.445 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-05-21T22:06:38.446 INFO:teuthology.orchestra.run.smithi184.stdout: "AsyncMessenger::Worker-0": { 2024-05-21T22:06:38.446 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_messages": 2203, 2024-05-21T22:06:38.446 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages": 2189, 2024-05-21T22:06:38.446 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_bytes": 3634495, 2024-05-21T22:06:38.446 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_bytes": 770382, 2024-05-21T22:06:38.446 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_created_connections": 5, 2024-05-21T22:06:38.446 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_active_connections": 1, 2024-05-21T22:06:38.446 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_total_time": 0.245721678, 2024-05-21T22:06:38.446 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_send_time": 0.079359649, 2024-05-21T22:06:38.446 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_recv_time": 0.092352794, 2024-05-21T22:06:38.447 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_fast_dispatch_time": 0.067457400, 2024-05-21T22:06:38.447 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T22:06:38.447 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 2188, 2024-05-21T22:06:38.447 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.089937671, 2024-05-21T22:06:38.447 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000041104 2024-05-21T22:06:38.447 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:38.447 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_handle_ack_lat": { 2024-05-21T22:06:38.447 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:38.447 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:38.447 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:38.447 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:38.448 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_encrypted_bytes": 1909920, 2024-05-21T22:06:38.448 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_encrypted_bytes": 161408 2024-05-21T22:06:38.448 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:38.448 INFO:teuthology.orchestra.run.smithi184.stdout: "AsyncMessenger::Worker-1": { 2024-05-21T22:06:38.448 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_messages": 7, 2024-05-21T22:06:38.448 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages": 202, 2024-05-21T22:06:38.448 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_bytes": 2136, 2024-05-21T22:06:38.448 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_bytes": 578746, 2024-05-21T22:06:38.448 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_created_connections": 8, 2024-05-21T22:06:38.448 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_active_connections": 5, 2024-05-21T22:06:38.448 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_total_time": 0.012722599, 2024-05-21T22:06:38.448 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_send_time": 0.008192220, 2024-05-21T22:06:38.448 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_recv_time": 0.002190029, 2024-05-21T22:06:38.449 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_fast_dispatch_time": 0.000153841, 2024-05-21T22:06:38.449 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T22:06:38.449 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 199, 2024-05-21T22:06:38.449 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.026878182, 2024-05-21T22:06:38.449 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000135066 2024-05-21T22:06:38.449 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:38.449 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_handle_ack_lat": { 2024-05-21T22:06:38.449 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:38.449 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:38.449 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:38.449 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:38.449 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_encrypted_bytes": 480, 2024-05-21T22:06:38.449 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_encrypted_bytes": 577408 2024-05-21T22:06:38.450 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:38.450 INFO:teuthology.orchestra.run.smithi184.stdout: "AsyncMessenger::Worker-2": { 2024-05-21T22:06:38.450 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_messages": 897, 2024-05-21T22:06:38.450 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages": 897, 2024-05-21T22:06:38.450 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_bytes": 222776, 2024-05-21T22:06:38.450 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_bytes": 284429, 2024-05-21T22:06:38.450 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_created_connections": 4, 2024-05-21T22:06:38.450 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_active_connections": 3, 2024-05-21T22:06:38.450 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_total_time": 0.091130316, 2024-05-21T22:06:38.450 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_send_time": 0.032804376, 2024-05-21T22:06:38.450 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_recv_time": 0.019273017, 2024-05-21T22:06:38.450 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_fast_dispatch_time": 0.030644213, 2024-05-21T22:06:38.451 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T22:06:38.451 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 896, 2024-05-21T22:06:38.451 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.035830637, 2024-05-21T22:06:38.451 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000039989 2024-05-21T22:06:38.451 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:38.451 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_handle_ack_lat": { 2024-05-21T22:06:38.451 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:38.451 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:38.451 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:38.451 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:38.451 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-05-21T22:06:38.451 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_encrypted_bytes": 448 2024-05-21T22:06:38.452 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:38.452 INFO:teuthology.orchestra.run.smithi184.stdout: "cct": { 2024-05-21T22:06:38.452 INFO:teuthology.orchestra.run.smithi184.stdout: "total_workers": 1, 2024-05-21T22:06:38.452 INFO:teuthology.orchestra.run.smithi184.stdout: "unhealthy_workers": 0 2024-05-21T22:06:38.452 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:38.452 INFO:teuthology.orchestra.run.smithi184.stdout: "finisher-MDSRank": { 2024-05-21T22:06:38.452 INFO:teuthology.orchestra.run.smithi184.stdout: "queue_len": 0, 2024-05-21T22:06:38.452 INFO:teuthology.orchestra.run.smithi184.stdout: "complete_latency": { 2024-05-21T22:06:38.452 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 4608, 2024-05-21T22:06:38.452 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.133983027, 2024-05-21T22:06:38.452 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000029076 2024-05-21T22:06:38.452 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-05-21T22:06:38.453 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:38.453 INFO:teuthology.orchestra.run.smithi184.stdout: "finisher-PurgeQueue": { 2024-05-21T22:06:38.453 INFO:teuthology.orchestra.run.smithi184.stdout: "queue_len": 0, 2024-05-21T22:06:38.453 INFO:teuthology.orchestra.run.smithi184.stdout: "complete_latency": { 2024-05-21T22:06:38.453 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:38.453 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:38.453 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:38.453 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-05-21T22:06:38.453 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:38.453 INFO:teuthology.orchestra.run.smithi184.stdout: "mds": { 2024-05-21T22:06:38.454 INFO:teuthology.orchestra.run.smithi184.stdout: "request": 0, 2024-05-21T22:06:38.454 INFO:teuthology.orchestra.run.smithi184.stdout: "reply": 0, 2024-05-21T22:06:38.454 INFO:teuthology.orchestra.run.smithi184.stdout: "reply_latency": { 2024-05-21T22:06:38.454 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:38.454 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:38.454 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:38.454 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:38.454 INFO:teuthology.orchestra.run.smithi184.stdout: "slow_reply": 0, 2024-05-21T22:06:38.454 INFO:teuthology.orchestra.run.smithi184.stdout: "forward": 0, 2024-05-21T22:06:38.454 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_fetch_complete": 0, 2024-05-21T22:06:38.455 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_fetch_keys": 0, 2024-05-21T22:06:38.455 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_commit": 0, 2024-05-21T22:06:38.455 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_split": 0, 2024-05-21T22:06:38.455 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_merge": 0, 2024-05-21T22:06:38.455 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes": 31, 2024-05-21T22:06:38.455 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_top": 0, 2024-05-21T22:06:38.455 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_bottom": 31, 2024-05-21T22:06:38.455 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_pin_tail": 0, 2024-05-21T22:06:38.455 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_pinned": 31, 2024-05-21T22:06:38.455 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_expired": 0, 2024-05-21T22:06:38.455 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_with_caps": 0, 2024-05-21T22:06:38.455 INFO:teuthology.orchestra.run.smithi184.stdout: "caps": 0, 2024-05-21T22:06:38.455 INFO:teuthology.orchestra.run.smithi184.stdout: "subtrees": 2, 2024-05-21T22:06:38.456 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse": 0, 2024-05-21T22:06:38.456 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_hit": 0, 2024-05-21T22:06:38.456 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_forward": 0, 2024-05-21T22:06:38.456 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_discover": 0, 2024-05-21T22:06:38.456 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_dir_fetch": 0, 2024-05-21T22:06:38.456 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_remote_ino": 0, 2024-05-21T22:06:38.456 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_lock": 0, 2024-05-21T22:06:38.456 INFO:teuthology.orchestra.run.smithi184.stdout: "load_cent": 0, 2024-05-21T22:06:38.456 INFO:teuthology.orchestra.run.smithi184.stdout: "q": 0, 2024-05-21T22:06:38.456 INFO:teuthology.orchestra.run.smithi184.stdout: "exported": 0, 2024-05-21T22:06:38.456 INFO:teuthology.orchestra.run.smithi184.stdout: "exported_inodes": 0, 2024-05-21T22:06:38.456 INFO:teuthology.orchestra.run.smithi184.stdout: "imported": 0, 2024-05-21T22:06:38.457 INFO:teuthology.orchestra.run.smithi184.stdout: "imported_inodes": 0, 2024-05-21T22:06:38.457 INFO:teuthology.orchestra.run.smithi184.stdout: "openino_dir_fetch": 0, 2024-05-21T22:06:38.457 INFO:teuthology.orchestra.run.smithi184.stdout: "openino_backtrace_fetch": 0, 2024-05-21T22:06:38.457 INFO:teuthology.orchestra.run.smithi184.stdout: "openino_peer_discover": 0, 2024-05-21T22:06:38.457 INFO:teuthology.orchestra.run.smithi184.stdout: "root_rfiles": 4, 2024-05-21T22:06:38.457 INFO:teuthology.orchestra.run.smithi184.stdout: "root_rbytes": 234, 2024-05-21T22:06:38.457 INFO:teuthology.orchestra.run.smithi184.stdout: "root_rsnaps": 7, 2024-05-21T22:06:38.457 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_backtrace_fetch": 0, 2024-05-21T22:06:38.457 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_set_tag": 0, 2024-05-21T22:06:38.457 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_backtrace_repaired": 0, 2024-05-21T22:06:38.457 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_inotable_repaired": 0, 2024-05-21T22:06:38.457 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_dir_inodes": 0, 2024-05-21T22:06:38.457 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_dir_base_inodes": 0, 2024-05-21T22:06:38.458 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_dirfrag_rstats": 0, 2024-05-21T22:06:38.458 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_file_inodes": 0, 2024-05-21T22:06:38.458 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_inode_file_caps": 0, 2024-05-21T22:06:38.458 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_revoke": 0, 2024-05-21T22:06:38.458 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_grant": 0, 2024-05-21T22:06:38.458 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_trunc": 0, 2024-05-21T22:06:38.458 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-21T22:06:38.458 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-21T22:06:38.458 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_caps": 0, 2024-05-21T22:06:38.458 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_caps_dirty": 0, 2024-05-21T22:06:38.458 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_cap_release": 0, 2024-05-21T22:06:38.459 INFO:teuthology.orchestra.run.smithi184.stdout: "process_request_cap_release": 0 2024-05-21T22:06:38.459 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:38.459 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_cache": { 2024-05-21T22:06:38.459 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_update": 0, 2024-05-21T22:06:38.459 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_update_receipt": 0, 2024-05-21T22:06:38.459 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_try_discover": 0, 2024-05-21T22:06:38.459 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_send_discover": 0, 2024-05-21T22:06:38.459 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_handle_discover": 0, 2024-05-21T22:06:38.459 INFO:teuthology.orchestra.run.smithi184.stdout: "num_strays": 0, 2024-05-21T22:06:38.459 INFO:teuthology.orchestra.run.smithi184.stdout: "num_strays_delayed": 0, 2024-05-21T22:06:38.459 INFO:teuthology.orchestra.run.smithi184.stdout: "num_strays_enqueuing": 0, 2024-05-21T22:06:38.459 INFO:teuthology.orchestra.run.smithi184.stdout: "strays_created": 0, 2024-05-21T22:06:38.460 INFO:teuthology.orchestra.run.smithi184.stdout: "strays_enqueued": 0, 2024-05-21T22:06:38.460 INFO:teuthology.orchestra.run.smithi184.stdout: "strays_reintegrated": 0, 2024-05-21T22:06:38.460 INFO:teuthology.orchestra.run.smithi184.stdout: "strays_migrated": 0, 2024-05-21T22:06:38.460 INFO:teuthology.orchestra.run.smithi184.stdout: "num_recovering_processing": 0, 2024-05-21T22:06:38.460 INFO:teuthology.orchestra.run.smithi184.stdout: "num_recovering_enqueued": 0, 2024-05-21T22:06:38.460 INFO:teuthology.orchestra.run.smithi184.stdout: "num_recovering_prioritized": 0, 2024-05-21T22:06:38.460 INFO:teuthology.orchestra.run.smithi184.stdout: "recovery_started": 0, 2024-05-21T22:06:38.460 INFO:teuthology.orchestra.run.smithi184.stdout: "recovery_completed": 0, 2024-05-21T22:06:38.460 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_quiesce_path": 0, 2024-05-21T22:06:38.460 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_quiesce_inode": 0, 2024-05-21T22:06:38.460 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_enqueue_scrub": 0, 2024-05-21T22:06:38.460 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_exportdir": 0, 2024-05-21T22:06:38.460 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_flush": 0, 2024-05-21T22:06:38.461 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_fragmentdir": 0, 2024-05-21T22:06:38.461 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_fragstats": 0, 2024-05-21T22:06:38.461 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_inodestats": 0 2024-05-21T22:06:38.461 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:38.461 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_log": { 2024-05-21T22:06:38.461 INFO:teuthology.orchestra.run.smithi184.stdout: "evlrg": 0, 2024-05-21T22:06:38.461 INFO:teuthology.orchestra.run.smithi184.stdout: "evadd": 0, 2024-05-21T22:06:38.461 INFO:teuthology.orchestra.run.smithi184.stdout: "evex": 0, 2024-05-21T22:06:38.461 INFO:teuthology.orchestra.run.smithi184.stdout: "evtrm": 0, 2024-05-21T22:06:38.461 INFO:teuthology.orchestra.run.smithi184.stdout: "ev": 233, 2024-05-21T22:06:38.461 INFO:teuthology.orchestra.run.smithi184.stdout: "evexg": 0, 2024-05-21T22:06:38.461 INFO:teuthology.orchestra.run.smithi184.stdout: "evexd": 0, 2024-05-21T22:06:38.462 INFO:teuthology.orchestra.run.smithi184.stdout: "segadd": 0, 2024-05-21T22:06:38.462 INFO:teuthology.orchestra.run.smithi184.stdout: "segex": 0, 2024-05-21T22:06:38.462 INFO:teuthology.orchestra.run.smithi184.stdout: "segtrm": 0, 2024-05-21T22:06:38.462 INFO:teuthology.orchestra.run.smithi184.stdout: "seg": 2, 2024-05-21T22:06:38.462 INFO:teuthology.orchestra.run.smithi184.stdout: "segmjr": 2, 2024-05-21T22:06:38.462 INFO:teuthology.orchestra.run.smithi184.stdout: "segexg": 0, 2024-05-21T22:06:38.462 INFO:teuthology.orchestra.run.smithi184.stdout: "segexd": 0, 2024-05-21T22:06:38.462 INFO:teuthology.orchestra.run.smithi184.stdout: "expos": 4194304, 2024-05-21T22:06:38.462 INFO:teuthology.orchestra.run.smithi184.stdout: "wrpos": 5352015, 2024-05-21T22:06:38.462 INFO:teuthology.orchestra.run.smithi184.stdout: "rdpos": 5351903, 2024-05-21T22:06:38.462 INFO:teuthology.orchestra.run.smithi184.stdout: "jlat": { 2024-05-21T22:06:38.462 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:38.463 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:38.463 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:38.463 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:38.463 INFO:teuthology.orchestra.run.smithi184.stdout: "replayed": 233 2024-05-21T22:06:38.463 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:38.463 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_mem": { 2024-05-21T22:06:38.463 INFO:teuthology.orchestra.run.smithi184.stdout: "ino": 20, 2024-05-21T22:06:38.463 INFO:teuthology.orchestra.run.smithi184.stdout: "ino+": 24, 2024-05-21T22:06:38.463 INFO:teuthology.orchestra.run.smithi184.stdout: "ino-": 4, 2024-05-21T22:06:38.463 INFO:teuthology.orchestra.run.smithi184.stdout: "dir": 11, 2024-05-21T22:06:38.463 INFO:teuthology.orchestra.run.smithi184.stdout: "dir+": 11, 2024-05-21T22:06:38.464 INFO:teuthology.orchestra.run.smithi184.stdout: "dir-": 0, 2024-05-21T22:06:38.464 INFO:teuthology.orchestra.run.smithi184.stdout: "dn": 31, 2024-05-21T22:06:38.464 INFO:teuthology.orchestra.run.smithi184.stdout: "dn+": 31, 2024-05-21T22:06:38.464 INFO:teuthology.orchestra.run.smithi184.stdout: "dn-": 0, 2024-05-21T22:06:38.464 INFO:teuthology.orchestra.run.smithi184.stdout: "cap": 0, 2024-05-21T22:06:38.464 INFO:teuthology.orchestra.run.smithi184.stdout: "cap+": 0, 2024-05-21T22:06:38.464 INFO:teuthology.orchestra.run.smithi184.stdout: "cap-": 0, 2024-05-21T22:06:38.464 INFO:teuthology.orchestra.run.smithi184.stdout: "rss": 51504, 2024-05-21T22:06:38.464 INFO:teuthology.orchestra.run.smithi184.stdout: "heap": 215416 2024-05-21T22:06:38.464 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:38.464 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_server": { 2024-05-21T22:06:38.464 INFO:teuthology.orchestra.run.smithi184.stdout: "dispatch_client_request": 0, 2024-05-21T22:06:38.465 INFO:teuthology.orchestra.run.smithi184.stdout: "dispatch_server_request": 0, 2024-05-21T22:06:38.465 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_request": 0, 2024-05-21T22:06:38.465 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_session": 0, 2024-05-21T22:06:38.465 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_peer_request": 0, 2024-05-21T22:06:38.465 INFO:teuthology.orchestra.run.smithi184.stdout: "req_create_latency": { 2024-05-21T22:06:38.465 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:38.465 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:38.465 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:38.465 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:38.465 INFO:teuthology.orchestra.run.smithi184.stdout: "req_getattr_latency": { 2024-05-21T22:06:38.465 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:38.465 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:38.465 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:38.465 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:38.466 INFO:teuthology.orchestra.run.smithi184.stdout: "req_getfilelock_latency": { 2024-05-21T22:06:38.466 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:38.466 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:38.466 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:38.466 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:38.466 INFO:teuthology.orchestra.run.smithi184.stdout: "req_link_latency": { 2024-05-21T22:06:38.466 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:38.466 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:38.466 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:38.466 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:38.466 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookup_latency": { 2024-05-21T22:06:38.466 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:38.466 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:38.466 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:38.467 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:38.467 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookuphash_latency": { 2024-05-21T22:06:38.467 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:38.467 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:38.467 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:38.467 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:38.467 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookupino_latency": { 2024-05-21T22:06:38.467 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:38.467 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:38.467 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:38.467 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:38.467 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookupname_latency": { 2024-05-21T22:06:38.467 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:38.467 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:38.467 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:38.468 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:38.468 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookupparent_latency": { 2024-05-21T22:06:38.468 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:38.468 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:38.468 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:38.468 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:38.468 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookupsnap_latency": { 2024-05-21T22:06:38.468 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:38.468 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:38.468 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:38.468 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:38.468 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lssnap_latency": { 2024-05-21T22:06:38.468 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:38.468 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:38.469 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:38.469 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:38.469 INFO:teuthology.orchestra.run.smithi184.stdout: "req_mkdir_latency": { 2024-05-21T22:06:38.469 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:38.469 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:38.469 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:38.469 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:38.469 INFO:teuthology.orchestra.run.smithi184.stdout: "req_mknod_latency": { 2024-05-21T22:06:38.470 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:38.470 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:38.470 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:38.470 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:38.470 INFO:teuthology.orchestra.run.smithi184.stdout: "req_mksnap_latency": { 2024-05-21T22:06:38.470 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:38.470 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:38.470 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:38.470 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:38.470 INFO:teuthology.orchestra.run.smithi184.stdout: "req_open_latency": { 2024-05-21T22:06:38.470 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:38.470 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:38.470 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:38.470 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:38.471 INFO:teuthology.orchestra.run.smithi184.stdout: "req_readdir_latency": { 2024-05-21T22:06:38.471 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:38.471 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:38.471 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:38.471 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:38.471 INFO:teuthology.orchestra.run.smithi184.stdout: "req_rename_latency": { 2024-05-21T22:06:38.471 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:38.471 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:38.471 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:38.471 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:38.471 INFO:teuthology.orchestra.run.smithi184.stdout: "req_renamesnap_latency": { 2024-05-21T22:06:38.471 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:38.471 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:38.472 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:38.472 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:38.472 INFO:teuthology.orchestra.run.smithi184.stdout: "req_snapdiff_latency": { 2024-05-21T22:06:38.472 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:38.472 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:38.472 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:38.472 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:38.472 INFO:teuthology.orchestra.run.smithi184.stdout: "req_rmdir_latency": { 2024-05-21T22:06:38.472 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:38.472 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:38.472 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:38.472 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:38.472 INFO:teuthology.orchestra.run.smithi184.stdout: "req_rmsnap_latency": { 2024-05-21T22:06:38.472 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:38.473 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:38.473 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:38.473 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:38.473 INFO:teuthology.orchestra.run.smithi184.stdout: "req_rmxattr_latency": { 2024-05-21T22:06:38.473 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:38.473 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:38.473 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:38.473 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:38.473 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setattr_latency": { 2024-05-21T22:06:38.473 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:38.473 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:38.473 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:38.473 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:38.473 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setdirlayout_latency": { 2024-05-21T22:06:38.473 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:38.474 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:38.474 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:38.474 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:38.474 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setfilelock_latency": { 2024-05-21T22:06:38.474 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:38.474 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:38.474 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:38.474 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:38.474 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setlayout_latency": { 2024-05-21T22:06:38.474 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:38.474 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:38.474 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:38.474 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:38.474 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setxattr_latency": { 2024-05-21T22:06:38.475 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:38.475 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:38.475 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:38.475 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:38.475 INFO:teuthology.orchestra.run.smithi184.stdout: "req_symlink_latency": { 2024-05-21T22:06:38.475 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:38.475 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:38.475 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:38.475 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:38.475 INFO:teuthology.orchestra.run.smithi184.stdout: "req_unlink_latency": { 2024-05-21T22:06:38.475 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:38.475 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:38.475 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:38.475 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:38.476 INFO:teuthology.orchestra.run.smithi184.stdout: "cap_revoke_eviction": 0, 2024-05-21T22:06:38.476 INFO:teuthology.orchestra.run.smithi184.stdout: "cap_acquisition_throttle": 0, 2024-05-21T22:06:38.476 INFO:teuthology.orchestra.run.smithi184.stdout: "req_getvxattr_latency": { 2024-05-21T22:06:38.476 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:38.476 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:38.476 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:38.476 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-05-21T22:06:38.476 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:38.476 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_sessions": { 2024-05-21T22:06:38.476 INFO:teuthology.orchestra.run.smithi184.stdout: "session_count": 2, 2024-05-21T22:06:38.476 INFO:teuthology.orchestra.run.smithi184.stdout: "session_add": 5, 2024-05-21T22:06:38.476 INFO:teuthology.orchestra.run.smithi184.stdout: "session_remove": 3, 2024-05-21T22:06:38.476 INFO:teuthology.orchestra.run.smithi184.stdout: "sessions_open": 4, 2024-05-21T22:06:38.476 INFO:teuthology.orchestra.run.smithi184.stdout: "sessions_stale": 0, 2024-05-21T22:06:38.476 INFO:teuthology.orchestra.run.smithi184.stdout: "total_load": 0, 2024-05-21T22:06:38.477 INFO:teuthology.orchestra.run.smithi184.stdout: "average_load": 0, 2024-05-21T22:06:38.477 INFO:teuthology.orchestra.run.smithi184.stdout: "avg_session_uptime": 4101, 2024-05-21T22:06:38.477 INFO:teuthology.orchestra.run.smithi184.stdout: "mdthresh_evicted": 0 2024-05-21T22:06:38.477 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:38.477 INFO:teuthology.orchestra.run.smithi184.stdout: "mempool": { 2024-05-21T22:06:38.477 INFO:teuthology.orchestra.run.smithi184.stdout: "bloom_filter_bytes": 0, 2024-05-21T22:06:38.477 INFO:teuthology.orchestra.run.smithi184.stdout: "bloom_filter_items": 0, 2024-05-21T22:06:38.477 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_alloc_bytes": 0, 2024-05-21T22:06:38.477 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_alloc_items": 0, 2024-05-21T22:06:38.477 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_data_bytes": 0, 2024-05-21T22:06:38.477 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_data_items": 0, 2024-05-21T22:06:38.477 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-21T22:06:38.477 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_onode_items": 0, 2024-05-21T22:06:38.477 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-21T22:06:38.477 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_meta_items": 0, 2024-05-21T22:06:38.478 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_other_bytes": 0, 2024-05-21T22:06:38.478 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_other_items": 0, 2024-05-21T22:06:38.478 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-21T22:06:38.478 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_buffer_items": 0, 2024-05-21T22:06:38.478 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_extent_bytes": 0, 2024-05-21T22:06:38.478 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_extent_items": 0, 2024-05-21T22:06:38.478 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_blob_bytes": 0, 2024-05-21T22:06:38.478 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_blob_items": 0, 2024-05-21T22:06:38.478 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-21T22:06:38.478 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_shared_blob_items": 0, 2024-05-21T22:06:38.478 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-21T22:06:38.478 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_inline_bl_items": 0, 2024-05-21T22:06:38.478 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_fsck_bytes": 0, 2024-05-21T22:06:38.478 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_fsck_items": 0, 2024-05-21T22:06:38.479 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_txc_bytes": 0, 2024-05-21T22:06:38.479 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_txc_items": 0, 2024-05-21T22:06:38.479 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-21T22:06:38.479 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_deferred_items": 0, 2024-05-21T22:06:38.479 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_bytes": 0, 2024-05-21T22:06:38.479 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_items": 0, 2024-05-21T22:06:38.479 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_bytes": 0, 2024-05-21T22:06:38.479 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_items": 0, 2024-05-21T22:06:38.479 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_reader_bytes": 0, 2024-05-21T22:06:38.479 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_reader_items": 0, 2024-05-21T22:06:38.479 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_writer_bytes": 0, 2024-05-21T22:06:38.479 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_writer_items": 0, 2024-05-21T22:06:38.479 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_anon_bytes": 190494, 2024-05-21T22:06:38.479 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_anon_items": 61, 2024-05-21T22:06:38.479 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_meta_bytes": 0, 2024-05-21T22:06:38.480 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_meta_items": 0, 2024-05-21T22:06:38.480 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_bytes": 0, 2024-05-21T22:06:38.480 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_items": 0, 2024-05-21T22:06:38.480 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_mapbl_bytes": 0, 2024-05-21T22:06:38.480 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_mapbl_items": 0, 2024-05-21T22:06:38.480 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_pglog_bytes": 0, 2024-05-21T22:06:38.480 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_pglog_items": 0, 2024-05-21T22:06:38.480 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_bytes": 8128, 2024-05-21T22:06:38.480 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_items": 169, 2024-05-21T22:06:38.480 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_mapping_bytes": 0, 2024-05-21T22:06:38.480 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_mapping_items": 0, 2024-05-21T22:06:38.480 INFO:teuthology.orchestra.run.smithi184.stdout: "pgmap_bytes": 0, 2024-05-21T22:06:38.480 INFO:teuthology.orchestra.run.smithi184.stdout: "pgmap_items": 0, 2024-05-21T22:06:38.480 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_co_bytes": 96037, 2024-05-21T22:06:38.480 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_co_items": 757, 2024-05-21T22:06:38.480 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_1_bytes": 0, 2024-05-21T22:06:38.481 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_1_items": 0, 2024-05-21T22:06:38.481 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_2_bytes": 0, 2024-05-21T22:06:38.481 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_2_items": 0 2024-05-21T22:06:38.481 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:38.481 INFO:teuthology.orchestra.run.smithi184.stdout: "objecter": { 2024-05-21T22:06:38.481 INFO:teuthology.orchestra.run.smithi184.stdout: "op_active": 0, 2024-05-21T22:06:38.481 INFO:teuthology.orchestra.run.smithi184.stdout: "op_laggy": 0, 2024-05-21T22:06:38.481 INFO:teuthology.orchestra.run.smithi184.stdout: "op_send": 2812, 2024-05-21T22:06:38.481 INFO:teuthology.orchestra.run.smithi184.stdout: "op_send_bytes": 26, 2024-05-21T22:06:38.481 INFO:teuthology.orchestra.run.smithi184.stdout: "op_resend": 4, 2024-05-21T22:06:38.481 INFO:teuthology.orchestra.run.smithi184.stdout: "op_reply": 2812, 2024-05-21T22:06:38.481 INFO:teuthology.orchestra.run.smithi184.stdout: "op_latency": { 2024-05-21T22:06:38.481 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 2812, 2024-05-21T22:06:38.481 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 1.429970927, 2024-05-21T22:06:38.482 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000508524 2024-05-21T22:06:38.482 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:38.482 INFO:teuthology.orchestra.run.smithi184.stdout: "op_inflight": 0, 2024-05-21T22:06:38.482 INFO:teuthology.orchestra.run.smithi184.stdout: "oplen_avg": { 2024-05-21T22:06:38.482 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 2812, 2024-05-21T22:06:38.482 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 2813 2024-05-21T22:06:38.482 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:38.482 INFO:teuthology.orchestra.run.smithi184.stdout: "op": 2812, 2024-05-21T22:06:38.482 INFO:teuthology.orchestra.run.smithi184.stdout: "op_r": 2812, 2024-05-21T22:06:38.482 INFO:teuthology.orchestra.run.smithi184.stdout: "op_w": 0, 2024-05-21T22:06:38.482 INFO:teuthology.orchestra.run.smithi184.stdout: "op_rmw": 0, 2024-05-21T22:06:38.482 INFO:teuthology.orchestra.run.smithi184.stdout: "op_pg": 0, 2024-05-21T22:06:38.482 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_stat": 1788, 2024-05-21T22:06:38.483 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_create": 0, 2024-05-21T22:06:38.483 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_read": 1021, 2024-05-21T22:06:38.483 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_write": 0, 2024-05-21T22:06:38.483 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_writefull": 0, 2024-05-21T22:06:38.483 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_writesame": 0, 2024-05-21T22:06:38.483 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_append": 0, 2024-05-21T22:06:38.483 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_zero": 0, 2024-05-21T22:06:38.483 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_truncate": 0, 2024-05-21T22:06:38.483 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_delete": 0, 2024-05-21T22:06:38.483 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_mapext": 0, 2024-05-21T22:06:38.483 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_sparse_read": 0, 2024-05-21T22:06:38.483 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_clonerange": 0, 2024-05-21T22:06:38.483 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_getxattr": 2, 2024-05-21T22:06:38.483 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_setxattr": 0, 2024-05-21T22:06:38.484 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_cmpxattr": 0, 2024-05-21T22:06:38.484 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_rmxattr": 0, 2024-05-21T22:06:38.484 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_resetxattrs": 0, 2024-05-21T22:06:38.484 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_call": 0, 2024-05-21T22:06:38.484 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_watch": 0, 2024-05-21T22:06:38.484 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_notify": 0, 2024-05-21T22:06:38.484 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_src_cmpxattr": 0, 2024-05-21T22:06:38.484 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_pgls": 0, 2024-05-21T22:06:38.484 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_pgls_filter": 0, 2024-05-21T22:06:38.484 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_other": 0, 2024-05-21T22:06:38.484 INFO:teuthology.orchestra.run.smithi184.stdout: "linger_active": 0, 2024-05-21T22:06:38.484 INFO:teuthology.orchestra.run.smithi184.stdout: "linger_send": 0, 2024-05-21T22:06:38.484 INFO:teuthology.orchestra.run.smithi184.stdout: "linger_resend": 0, 2024-05-21T22:06:38.484 INFO:teuthology.orchestra.run.smithi184.stdout: "linger_ping": 0, 2024-05-21T22:06:38.484 INFO:teuthology.orchestra.run.smithi184.stdout: "poolop_active": 0, 2024-05-21T22:06:38.484 INFO:teuthology.orchestra.run.smithi184.stdout: "poolop_send": 0, 2024-05-21T22:06:38.485 INFO:teuthology.orchestra.run.smithi184.stdout: "poolop_resend": 0, 2024-05-21T22:06:38.485 INFO:teuthology.orchestra.run.smithi184.stdout: "poolstat_active": 0, 2024-05-21T22:06:38.485 INFO:teuthology.orchestra.run.smithi184.stdout: "poolstat_send": 0, 2024-05-21T22:06:38.485 INFO:teuthology.orchestra.run.smithi184.stdout: "poolstat_resend": 0, 2024-05-21T22:06:38.485 INFO:teuthology.orchestra.run.smithi184.stdout: "statfs_active": 0, 2024-05-21T22:06:38.485 INFO:teuthology.orchestra.run.smithi184.stdout: "statfs_send": 0, 2024-05-21T22:06:38.485 INFO:teuthology.orchestra.run.smithi184.stdout: "statfs_resend": 0, 2024-05-21T22:06:38.485 INFO:teuthology.orchestra.run.smithi184.stdout: "command_active": 0, 2024-05-21T22:06:38.485 INFO:teuthology.orchestra.run.smithi184.stdout: "command_send": 0, 2024-05-21T22:06:38.485 INFO:teuthology.orchestra.run.smithi184.stdout: "command_resend": 0, 2024-05-21T22:06:38.485 INFO:teuthology.orchestra.run.smithi184.stdout: "map_epoch": 91, 2024-05-21T22:06:38.485 INFO:teuthology.orchestra.run.smithi184.stdout: "map_full": 0, 2024-05-21T22:06:38.485 INFO:teuthology.orchestra.run.smithi184.stdout: "map_inc": 11, 2024-05-21T22:06:38.485 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_sessions": 5, 2024-05-21T22:06:38.486 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_session_open": 5, 2024-05-21T22:06:38.486 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_session_close": 0, 2024-05-21T22:06:38.486 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_laggy": 0, 2024-05-21T22:06:38.486 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_wr": 0, 2024-05-21T22:06:38.486 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_rd": 2, 2024-05-21T22:06:38.486 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_del": 0 2024-05-21T22:06:38.486 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:38.486 INFO:teuthology.orchestra.run.smithi184.stdout: "oft": { 2024-05-21T22:06:38.486 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_total_objs": 0, 2024-05-21T22:06:38.486 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_total_kv_pairs": 0, 2024-05-21T22:06:38.486 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_total_updates": 0, 2024-05-21T22:06:38.486 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_total_removes": 0 2024-05-21T22:06:38.486 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:38.487 INFO:teuthology.orchestra.run.smithi184.stdout: "osd-slow-ops": { 2024-05-21T22:06:38.487 INFO:teuthology.orchestra.run.smithi184.stdout: "slow_ops_count": 0 2024-05-21T22:06:38.487 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:38.487 INFO:teuthology.orchestra.run.smithi184.stdout: "purge_queue": { 2024-05-21T22:06:38.487 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executing_ops": 0, 2024-05-21T22:06:38.487 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executing_ops_high_water": 0, 2024-05-21T22:06:38.487 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executing": 0, 2024-05-21T22:06:38.487 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executing_high_water": 0, 2024-05-21T22:06:38.487 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executed_ops": 0, 2024-05-21T22:06:38.487 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executed": 0, 2024-05-21T22:06:38.487 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_item_in_journal": 0 2024-05-21T22:06:38.487 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:38.487 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-21T22:06:38.488 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2024-05-21T22:06:38.488 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 104857600, 2024-05-21T22:06:38.488 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2024-05-21T22:06:38.488 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 3103, 2024-05-21T22:06:38.488 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 3562316, 2024-05-21T22:06:38.488 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2024-05-21T22:06:38.488 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 3103, 2024-05-21T22:06:38.488 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 0, 2024-05-21T22:06:38.488 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 0, 2024-05-21T22:06:38.488 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 3103, 2024-05-21T22:06:38.488 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 3562316, 2024-05-21T22:06:38.488 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2024-05-21T22:06:38.488 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:38.488 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:38.488 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:38.489 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-05-21T22:06:38.489 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:38.489 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-objecter_bytes": { 2024-05-21T22:06:38.489 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2024-05-21T22:06:38.489 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 104857600, 2024-05-21T22:06:38.489 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2024-05-21T22:06:38.489 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 0, 2024-05-21T22:06:38.489 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 0, 2024-05-21T22:06:38.489 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2024-05-21T22:06:38.489 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 0, 2024-05-21T22:06:38.489 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 2812, 2024-05-21T22:06:38.489 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 1157721, 2024-05-21T22:06:38.489 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 2812, 2024-05-21T22:06:38.489 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 1157721, 2024-05-21T22:06:38.490 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2024-05-21T22:06:38.490 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:38.490 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:38.490 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:38.490 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-05-21T22:06:38.490 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:38.490 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-objecter_ops": { 2024-05-21T22:06:38.490 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2024-05-21T22:06:38.490 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 1024, 2024-05-21T22:06:38.490 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2024-05-21T22:06:38.490 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 0, 2024-05-21T22:06:38.490 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 0, 2024-05-21T22:06:38.490 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2024-05-21T22:06:38.491 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 0, 2024-05-21T22:06:38.491 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 2812, 2024-05-21T22:06:38.491 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 2812, 2024-05-21T22:06:38.491 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 2812, 2024-05-21T22:06:38.491 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 2812, 2024-05-21T22:06:38.491 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2024-05-21T22:06:38.491 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:38.491 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:38.491 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:38.491 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-05-21T22:06:38.491 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:38.491 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-write_buf_throttle": { 2024-05-21T22:06:38.491 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2024-05-21T22:06:38.491 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 3758096384, 2024-05-21T22:06:38.491 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2024-05-21T22:06:38.492 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 0, 2024-05-21T22:06:38.492 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 0, 2024-05-21T22:06:38.492 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2024-05-21T22:06:38.492 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 0, 2024-05-21T22:06:38.492 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 0, 2024-05-21T22:06:38.492 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 0, 2024-05-21T22:06:38.492 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 0, 2024-05-21T22:06:38.492 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 0, 2024-05-21T22:06:38.492 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2024-05-21T22:06:38.492 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:38.492 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:38.492 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:38.492 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-05-21T22:06:38.492 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:38.492 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-write_buf_throttle-0x5597757026e0": { 2024-05-21T22:06:38.492 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2024-05-21T22:06:38.493 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 3758096384, 2024-05-21T22:06:38.493 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2024-05-21T22:06:38.493 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 0, 2024-05-21T22:06:38.493 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 0, 2024-05-21T22:06:38.493 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2024-05-21T22:06:38.493 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 0, 2024-05-21T22:06:38.493 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 0, 2024-05-21T22:06:38.493 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 0, 2024-05-21T22:06:38.493 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 0, 2024-05-21T22:06:38.493 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 0, 2024-05-21T22:06:38.493 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2024-05-21T22:06:38.493 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:38.493 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:38.493 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:38.494 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-05-21T22:06:38.494 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-05-21T22:06:38.494 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-05-21T22:06:39.234 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-05-21T22:06:39.235 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-05-21T22:06:39.235 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-05-21T22:06:39.235 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-05-21T22:06:39.235 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=7 2024-05-21T22:06:39.235 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-05-21T22:06:39.235 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-05-21T22:06:39.235 DEBUG:tasks.check_counter:Getting stats from i 2024-05-21T22:06:39.235 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph daemon mds.i perf dump 2024-05-21T22:06:39.476 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.c/config 2024-05-21T22:06:39.884 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:06:39 smithi184 ceph-mon[36942]: pgmap v342: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T22:06:39.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:06:39 smithi151 ceph-mon[31623]: pgmap v342: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T22:06:40.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:06:39 smithi164 ceph-mon[36269]: pgmap v342: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T22:06:41.279 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-05-21T22:06:41.279 INFO:teuthology.orchestra.run.smithi184.stdout: "AsyncMessenger::Worker-0": { 2024-05-21T22:06:41.280 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_messages": 739, 2024-05-21T22:06:41.280 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages": 872, 2024-05-21T22:06:41.280 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_bytes": 224571, 2024-05-21T22:06:41.280 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_bytes": 429111, 2024-05-21T22:06:41.280 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_created_connections": 10, 2024-05-21T22:06:41.280 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_active_connections": 2, 2024-05-21T22:06:41.280 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_total_time": 0.089374194, 2024-05-21T22:06:41.280 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_send_time": 0.034728900, 2024-05-21T22:06:41.280 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_recv_time": 0.045492470, 2024-05-21T22:06:41.280 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_fast_dispatch_time": 0.006556019, 2024-05-21T22:06:41.280 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T22:06:41.280 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 874, 2024-05-21T22:06:41.280 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.054854063, 2024-05-21T22:06:41.281 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000062762 2024-05-21T22:06:41.281 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:41.281 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_handle_ack_lat": { 2024-05-21T22:06:41.281 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 1532, 2024-05-21T22:06:41.281 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000398334, 2024-05-21T22:06:41.281 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000260 2024-05-21T22:06:41.281 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:41.281 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-21T22:06:41.281 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_encrypted_bytes": 0 2024-05-21T22:06:41.281 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:41.281 INFO:teuthology.orchestra.run.smithi184.stdout: "AsyncMessenger::Worker-1": { 2024-05-21T22:06:41.281 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_messages": 131, 2024-05-21T22:06:41.281 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages": 207, 2024-05-21T22:06:41.281 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_bytes": 38172, 2024-05-21T22:06:41.281 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_bytes": 669908, 2024-05-21T22:06:41.282 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_created_connections": 52, 2024-05-21T22:06:41.282 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_active_connections": 6, 2024-05-21T22:06:41.282 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_total_time": 0.034971926, 2024-05-21T22:06:41.282 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_send_time": 0.009373940, 2024-05-21T22:06:41.282 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_recv_time": 0.023419339, 2024-05-21T22:06:41.282 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_fast_dispatch_time": 0.000238546, 2024-05-21T22:06:41.282 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T22:06:41.282 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 204, 2024-05-21T22:06:41.282 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.018428506, 2024-05-21T22:06:41.282 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000090335 2024-05-21T22:06:41.282 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:41.282 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_handle_ack_lat": { 2024-05-21T22:06:41.283 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 205, 2024-05-21T22:06:41.283 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000061593, 2024-05-21T22:06:41.283 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000300 2024-05-21T22:06:41.283 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:41.283 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_encrypted_bytes": 2016, 2024-05-21T22:06:41.283 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_encrypted_bytes": 174560 2024-05-21T22:06:41.283 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:41.283 INFO:teuthology.orchestra.run.smithi184.stdout: "AsyncMessenger::Worker-2": { 2024-05-21T22:06:41.283 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_messages": 345, 2024-05-21T22:06:41.283 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages": 473, 2024-05-21T22:06:41.283 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_bytes": 1944467, 2024-05-21T22:06:41.283 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_bytes": 507123, 2024-05-21T22:06:41.283 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_created_connections": 10, 2024-05-21T22:06:41.283 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_active_connections": 8, 2024-05-21T22:06:41.284 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_total_time": 0.071431379, 2024-05-21T22:06:41.284 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_send_time": 0.024899641, 2024-05-21T22:06:41.284 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_recv_time": 0.051660004, 2024-05-21T22:06:41.284 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_fast_dispatch_time": 0.006666269, 2024-05-21T22:06:41.284 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T22:06:41.284 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 469, 2024-05-21T22:06:41.284 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.066556821, 2024-05-21T22:06:41.284 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000141912 2024-05-21T22:06:41.284 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:41.284 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_handle_ack_lat": { 2024-05-21T22:06:41.284 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 33, 2024-05-21T22:06:41.284 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000008016, 2024-05-21T22:06:41.284 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000242 2024-05-21T22:06:41.285 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:41.285 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_encrypted_bytes": 1942240, 2024-05-21T22:06:41.285 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_encrypted_bytes": 498064 2024-05-21T22:06:41.285 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:41.285 INFO:teuthology.orchestra.run.smithi184.stdout: "cct": { 2024-05-21T22:06:41.285 INFO:teuthology.orchestra.run.smithi184.stdout: "total_workers": 1, 2024-05-21T22:06:41.285 INFO:teuthology.orchestra.run.smithi184.stdout: "unhealthy_workers": 0 2024-05-21T22:06:41.285 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:41.285 INFO:teuthology.orchestra.run.smithi184.stdout: "finisher-MDSRank": { 2024-05-21T22:06:41.285 INFO:teuthology.orchestra.run.smithi184.stdout: "queue_len": 0, 2024-05-21T22:06:41.285 INFO:teuthology.orchestra.run.smithi184.stdout: "complete_latency": { 2024-05-21T22:06:41.285 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 58, 2024-05-21T22:06:41.285 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.001938606, 2024-05-21T22:06:41.285 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000033424 2024-05-21T22:06:41.286 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-05-21T22:06:41.286 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:41.286 INFO:teuthology.orchestra.run.smithi184.stdout: "finisher-PurgeQueue": { 2024-05-21T22:06:41.286 INFO:teuthology.orchestra.run.smithi184.stdout: "queue_len": 0, 2024-05-21T22:06:41.286 INFO:teuthology.orchestra.run.smithi184.stdout: "complete_latency": { 2024-05-21T22:06:41.286 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 2, 2024-05-21T22:06:41.286 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000007239, 2024-05-21T22:06:41.286 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000003619 2024-05-21T22:06:41.286 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-05-21T22:06:41.286 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:41.286 INFO:teuthology.orchestra.run.smithi184.stdout: "mds": { 2024-05-21T22:06:41.286 INFO:teuthology.orchestra.run.smithi184.stdout: "request": 1, 2024-05-21T22:06:41.286 INFO:teuthology.orchestra.run.smithi184.stdout: "reply": 0, 2024-05-21T22:06:41.286 INFO:teuthology.orchestra.run.smithi184.stdout: "reply_latency": { 2024-05-21T22:06:41.287 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:41.287 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:41.287 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:41.287 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:41.287 INFO:teuthology.orchestra.run.smithi184.stdout: "slow_reply": 0, 2024-05-21T22:06:41.287 INFO:teuthology.orchestra.run.smithi184.stdout: "forward": 1, 2024-05-21T22:06:41.287 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_fetch_complete": 0, 2024-05-21T22:06:41.287 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_fetch_keys": 0, 2024-05-21T22:06:41.287 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_commit": 11, 2024-05-21T22:06:41.287 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_split": 0, 2024-05-21T22:06:41.287 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_merge": 0, 2024-05-21T22:06:41.287 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes": 16, 2024-05-21T22:06:41.287 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_top": 0, 2024-05-21T22:06:41.287 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_bottom": 16, 2024-05-21T22:06:41.288 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_pin_tail": 0, 2024-05-21T22:06:41.288 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_pinned": 13, 2024-05-21T22:06:41.288 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_expired": 4, 2024-05-21T22:06:41.288 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_with_caps": 0, 2024-05-21T22:06:41.288 INFO:teuthology.orchestra.run.smithi184.stdout: "caps": 0, 2024-05-21T22:06:41.288 INFO:teuthology.orchestra.run.smithi184.stdout: "subtrees": 3, 2024-05-21T22:06:41.288 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse": 14, 2024-05-21T22:06:41.288 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_hit": 9, 2024-05-21T22:06:41.288 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_forward": 1, 2024-05-21T22:06:41.288 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_discover": 4, 2024-05-21T22:06:41.288 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_dir_fetch": 0, 2024-05-21T22:06:41.288 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_remote_ino": 0, 2024-05-21T22:06:41.288 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_lock": 0, 2024-05-21T22:06:41.289 INFO:teuthology.orchestra.run.smithi184.stdout: "load_cent": 1000, 2024-05-21T22:06:41.289 INFO:teuthology.orchestra.run.smithi184.stdout: "q": 1, 2024-05-21T22:06:41.289 INFO:teuthology.orchestra.run.smithi184.stdout: "exported": 0, 2024-05-21T22:06:41.289 INFO:teuthology.orchestra.run.smithi184.stdout: "exported_inodes": 0, 2024-05-21T22:06:41.289 INFO:teuthology.orchestra.run.smithi184.stdout: "imported": 0, 2024-05-21T22:06:41.289 INFO:teuthology.orchestra.run.smithi184.stdout: "imported_inodes": 0, 2024-05-21T22:06:41.289 INFO:teuthology.orchestra.run.smithi184.stdout: "openino_dir_fetch": 0, 2024-05-21T22:06:41.289 INFO:teuthology.orchestra.run.smithi184.stdout: "openino_backtrace_fetch": 0, 2024-05-21T22:06:41.289 INFO:teuthology.orchestra.run.smithi184.stdout: "openino_peer_discover": 0, 2024-05-21T22:06:41.289 INFO:teuthology.orchestra.run.smithi184.stdout: "root_rfiles": 4, 2024-05-21T22:06:41.289 INFO:teuthology.orchestra.run.smithi184.stdout: "root_rbytes": 234, 2024-05-21T22:06:41.289 INFO:teuthology.orchestra.run.smithi184.stdout: "root_rsnaps": 7, 2024-05-21T22:06:41.289 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_backtrace_fetch": 0, 2024-05-21T22:06:41.290 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_set_tag": 0, 2024-05-21T22:06:41.290 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_backtrace_repaired": 0, 2024-05-21T22:06:41.290 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_inotable_repaired": 0, 2024-05-21T22:06:41.290 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_dir_inodes": 0, 2024-05-21T22:06:41.290 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_dir_base_inodes": 0, 2024-05-21T22:06:41.290 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_dirfrag_rstats": 0, 2024-05-21T22:06:41.290 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_file_inodes": 0, 2024-05-21T22:06:41.290 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_inode_file_caps": 0, 2024-05-21T22:06:41.290 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_revoke": 0, 2024-05-21T22:06:41.290 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_grant": 0, 2024-05-21T22:06:41.290 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_trunc": 0, 2024-05-21T22:06:41.290 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-21T22:06:41.290 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-21T22:06:41.291 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_caps": 0, 2024-05-21T22:06:41.291 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_caps_dirty": 0, 2024-05-21T22:06:41.291 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_cap_release": 0, 2024-05-21T22:06:41.291 INFO:teuthology.orchestra.run.smithi184.stdout: "process_request_cap_release": 0 2024-05-21T22:06:41.291 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:41.291 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_cache": { 2024-05-21T22:06:41.291 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_update": 0, 2024-05-21T22:06:41.291 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_update_receipt": 14, 2024-05-21T22:06:41.291 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_try_discover": 9, 2024-05-21T22:06:41.291 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_send_discover": 10, 2024-05-21T22:06:41.291 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_handle_discover": 0, 2024-05-21T22:06:41.291 INFO:teuthology.orchestra.run.smithi184.stdout: "num_strays": 0, 2024-05-21T22:06:41.291 INFO:teuthology.orchestra.run.smithi184.stdout: "num_strays_delayed": 0, 2024-05-21T22:06:41.291 INFO:teuthology.orchestra.run.smithi184.stdout: "num_strays_enqueuing": 0, 2024-05-21T22:06:41.292 INFO:teuthology.orchestra.run.smithi184.stdout: "strays_created": 0, 2024-05-21T22:06:41.292 INFO:teuthology.orchestra.run.smithi184.stdout: "strays_enqueued": 0, 2024-05-21T22:06:41.292 INFO:teuthology.orchestra.run.smithi184.stdout: "strays_reintegrated": 0, 2024-05-21T22:06:41.292 INFO:teuthology.orchestra.run.smithi184.stdout: "strays_migrated": 0, 2024-05-21T22:06:41.292 INFO:teuthology.orchestra.run.smithi184.stdout: "num_recovering_processing": 0, 2024-05-21T22:06:41.292 INFO:teuthology.orchestra.run.smithi184.stdout: "num_recovering_enqueued": 0, 2024-05-21T22:06:41.292 INFO:teuthology.orchestra.run.smithi184.stdout: "num_recovering_prioritized": 0, 2024-05-21T22:06:41.292 INFO:teuthology.orchestra.run.smithi184.stdout: "recovery_started": 0, 2024-05-21T22:06:41.292 INFO:teuthology.orchestra.run.smithi184.stdout: "recovery_completed": 0, 2024-05-21T22:06:41.292 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_quiesce_path": 2, 2024-05-21T22:06:41.292 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_quiesce_inode": 11, 2024-05-21T22:06:41.292 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_enqueue_scrub": 0, 2024-05-21T22:06:41.292 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_exportdir": 0, 2024-05-21T22:06:41.293 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_flush": 0, 2024-05-21T22:06:41.293 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_fragmentdir": 0, 2024-05-21T22:06:41.293 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_fragstats": 0, 2024-05-21T22:06:41.293 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_inodestats": 0 2024-05-21T22:06:41.293 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:41.293 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_log": { 2024-05-21T22:06:41.293 INFO:teuthology.orchestra.run.smithi184.stdout: "evlrg": 0, 2024-05-21T22:06:41.293 INFO:teuthology.orchestra.run.smithi184.stdout: "evadd": 8, 2024-05-21T22:06:41.293 INFO:teuthology.orchestra.run.smithi184.stdout: "evex": 0, 2024-05-21T22:06:41.293 INFO:teuthology.orchestra.run.smithi184.stdout: "evtrm": 0, 2024-05-21T22:06:41.293 INFO:teuthology.orchestra.run.smithi184.stdout: "ev": 8, 2024-05-21T22:06:41.293 INFO:teuthology.orchestra.run.smithi184.stdout: "evexg": 0, 2024-05-21T22:06:41.293 INFO:teuthology.orchestra.run.smithi184.stdout: "evexd": 0, 2024-05-21T22:06:41.294 INFO:teuthology.orchestra.run.smithi184.stdout: "segadd": 2, 2024-05-21T22:06:41.294 INFO:teuthology.orchestra.run.smithi184.stdout: "segex": 0, 2024-05-21T22:06:41.294 INFO:teuthology.orchestra.run.smithi184.stdout: "segtrm": 0, 2024-05-21T22:06:41.294 INFO:teuthology.orchestra.run.smithi184.stdout: "seg": 2, 2024-05-21T22:06:41.294 INFO:teuthology.orchestra.run.smithi184.stdout: "segmjr": 2, 2024-05-21T22:06:41.294 INFO:teuthology.orchestra.run.smithi184.stdout: "segexg": 0, 2024-05-21T22:06:41.294 INFO:teuthology.orchestra.run.smithi184.stdout: "segexd": 0, 2024-05-21T22:06:41.294 INFO:teuthology.orchestra.run.smithi184.stdout: "expos": 4194304, 2024-05-21T22:06:41.294 INFO:teuthology.orchestra.run.smithi184.stdout: "wrpos": 4196657, 2024-05-21T22:06:41.294 INFO:teuthology.orchestra.run.smithi184.stdout: "rdpos": 0, 2024-05-21T22:06:41.294 INFO:teuthology.orchestra.run.smithi184.stdout: "jlat": { 2024-05-21T22:06:41.294 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 7, 2024-05-21T22:06:41.294 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.012850606, 2024-05-21T22:06:41.295 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.001835800 2024-05-21T22:06:41.295 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:41.295 INFO:teuthology.orchestra.run.smithi184.stdout: "replayed": 0 2024-05-21T22:06:41.295 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:41.295 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_mem": { 2024-05-21T22:06:41.295 INFO:teuthology.orchestra.run.smithi184.stdout: "ino": 20, 2024-05-21T22:06:41.295 INFO:teuthology.orchestra.run.smithi184.stdout: "ino+": 22, 2024-05-21T22:06:41.295 INFO:teuthology.orchestra.run.smithi184.stdout: "ino-": 2, 2024-05-21T22:06:41.295 INFO:teuthology.orchestra.run.smithi184.stdout: "dir": 19, 2024-05-21T22:06:41.295 INFO:teuthology.orchestra.run.smithi184.stdout: "dir+": 23, 2024-05-21T22:06:41.295 INFO:teuthology.orchestra.run.smithi184.stdout: "dir-": 4, 2024-05-21T22:06:41.295 INFO:teuthology.orchestra.run.smithi184.stdout: "dn": 16, 2024-05-21T22:06:41.295 INFO:teuthology.orchestra.run.smithi184.stdout: "dn+": 20, 2024-05-21T22:06:41.295 INFO:teuthology.orchestra.run.smithi184.stdout: "dn-": 4, 2024-05-21T22:06:41.296 INFO:teuthology.orchestra.run.smithi184.stdout: "cap": 0, 2024-05-21T22:06:41.296 INFO:teuthology.orchestra.run.smithi184.stdout: "cap+": 0, 2024-05-21T22:06:41.296 INFO:teuthology.orchestra.run.smithi184.stdout: "cap-": 0, 2024-05-21T22:06:41.296 INFO:teuthology.orchestra.run.smithi184.stdout: "rss": 53024, 2024-05-21T22:06:41.296 INFO:teuthology.orchestra.run.smithi184.stdout: "heap": 223608 2024-05-21T22:06:41.296 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:41.296 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_server": { 2024-05-21T22:06:41.296 INFO:teuthology.orchestra.run.smithi184.stdout: "dispatch_client_request": 1, 2024-05-21T22:06:41.296 INFO:teuthology.orchestra.run.smithi184.stdout: "dispatch_server_request": 0, 2024-05-21T22:06:41.296 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_request": 1, 2024-05-21T22:06:41.296 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_session": 26, 2024-05-21T22:06:41.296 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_peer_request": 11, 2024-05-21T22:06:41.296 INFO:teuthology.orchestra.run.smithi184.stdout: "req_create_latency": { 2024-05-21T22:06:41.297 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:41.297 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:41.297 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:41.297 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:41.297 INFO:teuthology.orchestra.run.smithi184.stdout: "req_getattr_latency": { 2024-05-21T22:06:41.297 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:41.297 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:41.297 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:41.297 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:41.297 INFO:teuthology.orchestra.run.smithi184.stdout: "req_getfilelock_latency": { 2024-05-21T22:06:41.297 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:41.297 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:41.297 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:41.297 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:41.297 INFO:teuthology.orchestra.run.smithi184.stdout: "req_link_latency": { 2024-05-21T22:06:41.298 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:41.298 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:41.298 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:41.298 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:41.298 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookup_latency": { 2024-05-21T22:06:41.298 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:41.298 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:41.298 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:41.298 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:41.298 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookuphash_latency": { 2024-05-21T22:06:41.298 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:41.298 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:41.298 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:41.299 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:41.299 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookupino_latency": { 2024-05-21T22:06:41.299 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:41.299 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:41.299 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:41.299 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:41.299 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookupname_latency": { 2024-05-21T22:06:41.299 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:41.299 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:41.299 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:41.299 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:41.299 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookupparent_latency": { 2024-05-21T22:06:41.299 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:41.299 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:41.300 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:41.300 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:41.300 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookupsnap_latency": { 2024-05-21T22:06:41.300 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:41.300 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:41.300 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:41.300 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:41.300 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lssnap_latency": { 2024-05-21T22:06:41.300 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:41.300 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:41.300 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:41.300 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:41.300 INFO:teuthology.orchestra.run.smithi184.stdout: "req_mkdir_latency": { 2024-05-21T22:06:41.301 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:41.301 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:41.301 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:41.301 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:41.301 INFO:teuthology.orchestra.run.smithi184.stdout: "req_mknod_latency": { 2024-05-21T22:06:41.301 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:41.302 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:41.302 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:41.302 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:41.302 INFO:teuthology.orchestra.run.smithi184.stdout: "req_mksnap_latency": { 2024-05-21T22:06:41.302 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:41.302 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:41.302 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:41.302 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:41.302 INFO:teuthology.orchestra.run.smithi184.stdout: "req_open_latency": { 2024-05-21T22:06:41.302 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:41.302 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:41.302 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:41.302 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:41.303 INFO:teuthology.orchestra.run.smithi184.stdout: "req_readdir_latency": { 2024-05-21T22:06:41.303 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:41.303 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:41.303 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:41.303 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:41.303 INFO:teuthology.orchestra.run.smithi184.stdout: "req_rename_latency": { 2024-05-21T22:06:41.303 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:41.303 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:41.303 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:41.303 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:41.303 INFO:teuthology.orchestra.run.smithi184.stdout: "req_renamesnap_latency": { 2024-05-21T22:06:41.303 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:41.303 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:41.303 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:41.303 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:41.304 INFO:teuthology.orchestra.run.smithi184.stdout: "req_snapdiff_latency": { 2024-05-21T22:06:41.304 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:41.304 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:41.304 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:41.304 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:41.304 INFO:teuthology.orchestra.run.smithi184.stdout: "req_rmdir_latency": { 2024-05-21T22:06:41.304 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:41.304 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:41.304 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:41.304 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:41.304 INFO:teuthology.orchestra.run.smithi184.stdout: "req_rmsnap_latency": { 2024-05-21T22:06:41.304 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:41.304 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:41.304 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:41.305 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:41.305 INFO:teuthology.orchestra.run.smithi184.stdout: "req_rmxattr_latency": { 2024-05-21T22:06:41.305 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:41.305 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:41.305 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:41.305 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:41.305 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setattr_latency": { 2024-05-21T22:06:41.305 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:41.305 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:41.305 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:41.305 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:41.305 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setdirlayout_latency": { 2024-05-21T22:06:41.305 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:41.305 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:41.306 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:41.306 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:41.306 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setfilelock_latency": { 2024-05-21T22:06:41.306 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:41.306 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:41.306 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:41.306 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:41.306 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setlayout_latency": { 2024-05-21T22:06:41.306 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:41.306 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:41.306 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:41.306 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:41.306 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setxattr_latency": { 2024-05-21T22:06:41.307 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:41.307 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:41.307 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:41.307 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:41.307 INFO:teuthology.orchestra.run.smithi184.stdout: "req_symlink_latency": { 2024-05-21T22:06:41.307 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:41.307 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:41.307 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:41.307 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:41.307 INFO:teuthology.orchestra.run.smithi184.stdout: "req_unlink_latency": { 2024-05-21T22:06:41.307 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:41.307 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:41.307 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:41.307 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:41.308 INFO:teuthology.orchestra.run.smithi184.stdout: "cap_revoke_eviction": 0, 2024-05-21T22:06:41.308 INFO:teuthology.orchestra.run.smithi184.stdout: "cap_acquisition_throttle": 0, 2024-05-21T22:06:41.308 INFO:teuthology.orchestra.run.smithi184.stdout: "req_getvxattr_latency": { 2024-05-21T22:06:41.308 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:41.308 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:41.308 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:41.308 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-05-21T22:06:41.308 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:41.308 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_sessions": { 2024-05-21T22:06:41.308 INFO:teuthology.orchestra.run.smithi184.stdout: "session_count": 2, 2024-05-21T22:06:41.308 INFO:teuthology.orchestra.run.smithi184.stdout: "session_add": 4, 2024-05-21T22:06:41.308 INFO:teuthology.orchestra.run.smithi184.stdout: "session_remove": 2, 2024-05-21T22:06:41.308 INFO:teuthology.orchestra.run.smithi184.stdout: "sessions_open": 2, 2024-05-21T22:06:41.308 INFO:teuthology.orchestra.run.smithi184.stdout: "sessions_stale": 0, 2024-05-21T22:06:41.308 INFO:teuthology.orchestra.run.smithi184.stdout: "total_load": 0, 2024-05-21T22:06:41.309 INFO:teuthology.orchestra.run.smithi184.stdout: "average_load": 0, 2024-05-21T22:06:41.309 INFO:teuthology.orchestra.run.smithi184.stdout: "avg_session_uptime": 650, 2024-05-21T22:06:41.309 INFO:teuthology.orchestra.run.smithi184.stdout: "mdthresh_evicted": 0 2024-05-21T22:06:41.309 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:41.309 INFO:teuthology.orchestra.run.smithi184.stdout: "mempool": { 2024-05-21T22:06:41.309 INFO:teuthology.orchestra.run.smithi184.stdout: "bloom_filter_bytes": 0, 2024-05-21T22:06:41.309 INFO:teuthology.orchestra.run.smithi184.stdout: "bloom_filter_items": 0, 2024-05-21T22:06:41.309 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_alloc_bytes": 0, 2024-05-21T22:06:41.309 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_alloc_items": 0, 2024-05-21T22:06:41.309 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_data_bytes": 0, 2024-05-21T22:06:41.309 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_data_items": 0, 2024-05-21T22:06:41.309 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-21T22:06:41.309 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_onode_items": 0, 2024-05-21T22:06:41.309 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-21T22:06:41.309 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_meta_items": 0, 2024-05-21T22:06:41.310 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_other_bytes": 0, 2024-05-21T22:06:41.310 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_other_items": 0, 2024-05-21T22:06:41.310 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-21T22:06:41.310 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_buffer_items": 0, 2024-05-21T22:06:41.310 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_extent_bytes": 0, 2024-05-21T22:06:41.310 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_extent_items": 0, 2024-05-21T22:06:41.310 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_blob_bytes": 0, 2024-05-21T22:06:41.310 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_blob_items": 0, 2024-05-21T22:06:41.310 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-21T22:06:41.310 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_shared_blob_items": 0, 2024-05-21T22:06:41.310 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-21T22:06:41.310 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_inline_bl_items": 0, 2024-05-21T22:06:41.310 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_fsck_bytes": 0, 2024-05-21T22:06:41.311 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_fsck_items": 0, 2024-05-21T22:06:41.311 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_txc_bytes": 0, 2024-05-21T22:06:41.311 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_txc_items": 0, 2024-05-21T22:06:41.311 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-21T22:06:41.311 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_deferred_items": 0, 2024-05-21T22:06:41.311 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_bytes": 0, 2024-05-21T22:06:41.311 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_items": 0, 2024-05-21T22:06:41.311 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_bytes": 0, 2024-05-21T22:06:41.311 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_items": 0, 2024-05-21T22:06:41.311 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_reader_bytes": 0, 2024-05-21T22:06:41.311 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_reader_items": 0, 2024-05-21T22:06:41.311 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_writer_bytes": 0, 2024-05-21T22:06:41.311 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_writer_items": 0, 2024-05-21T22:06:41.312 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_anon_bytes": 247657, 2024-05-21T22:06:41.312 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_anon_items": 88, 2024-05-21T22:06:41.312 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_meta_bytes": 0, 2024-05-21T22:06:41.312 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_meta_items": 0, 2024-05-21T22:06:41.312 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_bytes": 0, 2024-05-21T22:06:41.312 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_items": 0, 2024-05-21T22:06:41.312 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_mapbl_bytes": 0, 2024-05-21T22:06:41.312 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_mapbl_items": 0, 2024-05-21T22:06:41.312 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_pglog_bytes": 0, 2024-05-21T22:06:41.312 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_pglog_items": 0, 2024-05-21T22:06:41.312 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_bytes": 8128, 2024-05-21T22:06:41.312 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_items": 169, 2024-05-21T22:06:41.312 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_mapping_bytes": 0, 2024-05-21T22:06:41.312 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_mapping_items": 0, 2024-05-21T22:06:41.312 INFO:teuthology.orchestra.run.smithi184.stdout: "pgmap_bytes": 0, 2024-05-21T22:06:41.313 INFO:teuthology.orchestra.run.smithi184.stdout: "pgmap_items": 0, 2024-05-21T22:06:41.313 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_co_bytes": 82717, 2024-05-21T22:06:41.313 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_co_items": 340, 2024-05-21T22:06:41.313 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_1_bytes": 0, 2024-05-21T22:06:41.313 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_1_items": 0, 2024-05-21T22:06:41.313 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_2_bytes": 0, 2024-05-21T22:06:41.313 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_2_items": 0 2024-05-21T22:06:41.313 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:41.313 INFO:teuthology.orchestra.run.smithi184.stdout: "objecter": { 2024-05-21T22:06:41.313 INFO:teuthology.orchestra.run.smithi184.stdout: "op_active": 0, 2024-05-21T22:06:41.313 INFO:teuthology.orchestra.run.smithi184.stdout: "op_laggy": 0, 2024-05-21T22:06:41.313 INFO:teuthology.orchestra.run.smithi184.stdout: "op_send": 45, 2024-05-21T22:06:41.313 INFO:teuthology.orchestra.run.smithi184.stdout: "op_send_bytes": 12835, 2024-05-21T22:06:41.313 INFO:teuthology.orchestra.run.smithi184.stdout: "op_resend": 32, 2024-05-21T22:06:41.313 INFO:teuthology.orchestra.run.smithi184.stdout: "op_reply": 45, 2024-05-21T22:06:41.314 INFO:teuthology.orchestra.run.smithi184.stdout: "op_latency": { 2024-05-21T22:06:41.314 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 45, 2024-05-21T22:06:41.314 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.276994320, 2024-05-21T22:06:41.314 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.006155429 2024-05-21T22:06:41.314 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:41.314 INFO:teuthology.orchestra.run.smithi184.stdout: "op_inflight": 0, 2024-05-21T22:06:41.314 INFO:teuthology.orchestra.run.smithi184.stdout: "oplen_avg": { 2024-05-21T22:06:41.314 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 45, 2024-05-21T22:06:41.314 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 67 2024-05-21T22:06:41.314 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:41.314 INFO:teuthology.orchestra.run.smithi184.stdout: "op": 45, 2024-05-21T22:06:41.314 INFO:teuthology.orchestra.run.smithi184.stdout: "op_r": 0, 2024-05-21T22:06:41.314 INFO:teuthology.orchestra.run.smithi184.stdout: "op_w": 45, 2024-05-21T22:06:41.315 INFO:teuthology.orchestra.run.smithi184.stdout: "op_rmw": 0, 2024-05-21T22:06:41.315 INFO:teuthology.orchestra.run.smithi184.stdout: "op_pg": 0, 2024-05-21T22:06:41.315 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_stat": 0, 2024-05-21T22:06:41.315 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_create": 10, 2024-05-21T22:06:41.315 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_read": 0, 2024-05-21T22:06:41.315 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_write": 7, 2024-05-21T22:06:41.315 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_writefull": 9, 2024-05-21T22:06:41.315 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_writesame": 0, 2024-05-21T22:06:41.315 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_append": 0, 2024-05-21T22:06:41.315 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_zero": 0, 2024-05-21T22:06:41.315 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_truncate": 0, 2024-05-21T22:06:41.315 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_delete": 6, 2024-05-21T22:06:41.315 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_mapext": 0, 2024-05-21T22:06:41.315 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_sparse_read": 0, 2024-05-21T22:06:41.315 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_clonerange": 0, 2024-05-21T22:06:41.315 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_getxattr": 0, 2024-05-21T22:06:41.316 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_setxattr": 20, 2024-05-21T22:06:41.316 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_cmpxattr": 0, 2024-05-21T22:06:41.316 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_rmxattr": 0, 2024-05-21T22:06:41.316 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_resetxattrs": 0, 2024-05-21T22:06:41.316 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_call": 0, 2024-05-21T22:06:41.316 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_watch": 0, 2024-05-21T22:06:41.316 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_notify": 0, 2024-05-21T22:06:41.316 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_src_cmpxattr": 0, 2024-05-21T22:06:41.316 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_pgls": 0, 2024-05-21T22:06:41.316 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_pgls_filter": 0, 2024-05-21T22:06:41.316 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_other": 0, 2024-05-21T22:06:41.316 INFO:teuthology.orchestra.run.smithi184.stdout: "linger_active": 0, 2024-05-21T22:06:41.316 INFO:teuthology.orchestra.run.smithi184.stdout: "linger_send": 0, 2024-05-21T22:06:41.316 INFO:teuthology.orchestra.run.smithi184.stdout: "linger_resend": 0, 2024-05-21T22:06:41.316 INFO:teuthology.orchestra.run.smithi184.stdout: "linger_ping": 0, 2024-05-21T22:06:41.317 INFO:teuthology.orchestra.run.smithi184.stdout: "poolop_active": 0, 2024-05-21T22:06:41.317 INFO:teuthology.orchestra.run.smithi184.stdout: "poolop_send": 0, 2024-05-21T22:06:41.317 INFO:teuthology.orchestra.run.smithi184.stdout: "poolop_resend": 0, 2024-05-21T22:06:41.317 INFO:teuthology.orchestra.run.smithi184.stdout: "poolstat_active": 0, 2024-05-21T22:06:41.317 INFO:teuthology.orchestra.run.smithi184.stdout: "poolstat_send": 0, 2024-05-21T22:06:41.317 INFO:teuthology.orchestra.run.smithi184.stdout: "poolstat_resend": 0, 2024-05-21T22:06:41.317 INFO:teuthology.orchestra.run.smithi184.stdout: "statfs_active": 0, 2024-05-21T22:06:41.317 INFO:teuthology.orchestra.run.smithi184.stdout: "statfs_send": 0, 2024-05-21T22:06:41.317 INFO:teuthology.orchestra.run.smithi184.stdout: "statfs_resend": 0, 2024-05-21T22:06:41.317 INFO:teuthology.orchestra.run.smithi184.stdout: "command_active": 0, 2024-05-21T22:06:41.317 INFO:teuthology.orchestra.run.smithi184.stdout: "command_send": 0, 2024-05-21T22:06:41.317 INFO:teuthology.orchestra.run.smithi184.stdout: "command_resend": 0, 2024-05-21T22:06:41.317 INFO:teuthology.orchestra.run.smithi184.stdout: "map_epoch": 91, 2024-05-21T22:06:41.317 INFO:teuthology.orchestra.run.smithi184.stdout: "map_full": 0, 2024-05-21T22:06:41.318 INFO:teuthology.orchestra.run.smithi184.stdout: "map_inc": 11, 2024-05-21T22:06:41.318 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_sessions": 9, 2024-05-21T22:06:41.318 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_session_open": 12, 2024-05-21T22:06:41.318 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_session_close": 3, 2024-05-21T22:06:41.318 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_laggy": 0, 2024-05-21T22:06:41.318 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_wr": 14, 2024-05-21T22:06:41.318 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_rd": 0, 2024-05-21T22:06:41.318 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_del": 1 2024-05-21T22:06:41.318 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:41.318 INFO:teuthology.orchestra.run.smithi184.stdout: "oft": { 2024-05-21T22:06:41.318 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_total_objs": 1, 2024-05-21T22:06:41.318 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_total_kv_pairs": 0, 2024-05-21T22:06:41.318 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_total_updates": 0, 2024-05-21T22:06:41.318 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_total_removes": 0 2024-05-21T22:06:41.319 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:41.319 INFO:teuthology.orchestra.run.smithi184.stdout: "osd-slow-ops": { 2024-05-21T22:06:41.319 INFO:teuthology.orchestra.run.smithi184.stdout: "slow_ops_count": 13 2024-05-21T22:06:41.319 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:41.319 INFO:teuthology.orchestra.run.smithi184.stdout: "purge_queue": { 2024-05-21T22:06:41.319 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executing_ops": 0, 2024-05-21T22:06:41.319 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executing_ops_high_water": 0, 2024-05-21T22:06:41.319 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executing": 0, 2024-05-21T22:06:41.319 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executing_high_water": 0, 2024-05-21T22:06:41.319 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executed_ops": 0, 2024-05-21T22:06:41.319 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executed": 0, 2024-05-21T22:06:41.319 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_item_in_journal": 0 2024-05-21T22:06:41.319 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:41.319 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-21T22:06:41.320 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2024-05-21T22:06:41.320 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 104857600, 2024-05-21T22:06:41.320 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2024-05-21T22:06:41.320 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 1211, 2024-05-21T22:06:41.320 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 2078276, 2024-05-21T22:06:41.320 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2024-05-21T22:06:41.320 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 1211, 2024-05-21T22:06:41.320 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 0, 2024-05-21T22:06:41.320 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 0, 2024-05-21T22:06:41.320 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 1211, 2024-05-21T22:06:41.320 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 2078276, 2024-05-21T22:06:41.320 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2024-05-21T22:06:41.320 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:41.320 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:41.321 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:41.321 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-05-21T22:06:41.321 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:41.321 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-objecter_bytes": { 2024-05-21T22:06:41.321 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2024-05-21T22:06:41.321 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 104857600, 2024-05-21T22:06:41.321 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2024-05-21T22:06:41.321 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 0, 2024-05-21T22:06:41.321 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 0, 2024-05-21T22:06:41.321 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2024-05-21T22:06:41.321 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 0, 2024-05-21T22:06:41.321 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 45, 2024-05-21T22:06:41.321 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 12835, 2024-05-21T22:06:41.321 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 45, 2024-05-21T22:06:41.321 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 12835, 2024-05-21T22:06:41.322 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2024-05-21T22:06:41.322 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:41.322 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:41.322 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:41.322 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-05-21T22:06:41.322 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:41.322 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-objecter_ops": { 2024-05-21T22:06:41.322 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2024-05-21T22:06:41.322 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 1024, 2024-05-21T22:06:41.322 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2024-05-21T22:06:41.322 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 0, 2024-05-21T22:06:41.322 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 0, 2024-05-21T22:06:41.322 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2024-05-21T22:06:41.323 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 0, 2024-05-21T22:06:41.323 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 45, 2024-05-21T22:06:41.323 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 45, 2024-05-21T22:06:41.323 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 45, 2024-05-21T22:06:41.323 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 45, 2024-05-21T22:06:41.323 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2024-05-21T22:06:41.323 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:41.323 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:41.323 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:41.323 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-05-21T22:06:41.323 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:41.323 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-write_buf_throttle": { 2024-05-21T22:06:41.323 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2024-05-21T22:06:41.323 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 3758096384, 2024-05-21T22:06:41.324 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2024-05-21T22:06:41.324 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 0, 2024-05-21T22:06:41.324 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 0, 2024-05-21T22:06:41.324 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2024-05-21T22:06:41.324 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 0, 2024-05-21T22:06:41.324 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 0, 2024-05-21T22:06:41.324 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 0, 2024-05-21T22:06:41.324 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 0, 2024-05-21T22:06:41.324 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 0, 2024-05-21T22:06:41.324 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2024-05-21T22:06:41.324 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:41.324 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:41.324 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:41.325 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-05-21T22:06:41.325 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:41.325 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-write_buf_throttle-0x5560e0420500": { 2024-05-21T22:06:41.325 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2024-05-21T22:06:41.325 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 3758096384, 2024-05-21T22:06:41.325 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2024-05-21T22:06:41.325 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 8, 2024-05-21T22:06:41.325 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 2353, 2024-05-21T22:06:41.325 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2024-05-21T22:06:41.325 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 8, 2024-05-21T22:06:41.325 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 0, 2024-05-21T22:06:41.325 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 0, 2024-05-21T22:06:41.325 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 7, 2024-05-21T22:06:41.326 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 2353, 2024-05-21T22:06:41.326 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2024-05-21T22:06:41.326 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:41.326 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:41.326 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:41.326 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-05-21T22:06:41.326 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-05-21T22:06:41.326 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-05-21T22:06:41.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:06:41 smithi151 ceph-mon[31623]: pgmap v343: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:06:41.985 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:06:41 smithi184 ceph-mon[36942]: pgmap v343: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:06:42.035 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:06:41 smithi164 ceph-mon[36269]: pgmap v343: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:06:42.074 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-05-21T22:06:42.074 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-05-21T22:06:42.074 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=0 2024-05-21T22:06:42.074 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=14 2024-05-21T22:06:42.074 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=7 2024-05-21T22:06:42.074 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-05-21T22:06:42.074 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-05-21T22:06:42.074 DEBUG:tasks.check_counter:Getting stats from l 2024-05-21T22:06:42.075 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph daemon mds.l perf dump 2024-05-21T22:06:42.310 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.c/config 2024-05-21T22:06:43.736 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:06:43 smithi184 ceph-mon[36942]: pgmap v344: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T22:06:43.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:06:43 smithi151 ceph-mon[31623]: pgmap v344: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T22:06:44.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:06:43 smithi164 ceph-mon[36269]: pgmap v344: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T22:06:44.076 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2024-05-21T22:06:44.076 INFO:teuthology.orchestra.run.smithi184.stdout: "AsyncMessenger::Worker-0": { 2024-05-21T22:06:44.076 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_messages": 1814, 2024-05-21T22:06:44.076 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages": 1816, 2024-05-21T22:06:44.076 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_bytes": 544456, 2024-05-21T22:06:44.076 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_bytes": 576853, 2024-05-21T22:06:44.076 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_created_connections": 6, 2024-05-21T22:06:44.077 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_active_connections": 2, 2024-05-21T22:06:44.077 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_total_time": 0.198789881, 2024-05-21T22:06:44.077 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_send_time": 0.071036726, 2024-05-21T22:06:44.077 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_recv_time": 0.048955397, 2024-05-21T22:06:44.077 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_fast_dispatch_time": 0.060443094, 2024-05-21T22:06:44.077 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T22:06:44.077 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 1814, 2024-05-21T22:06:44.077 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.073877529, 2024-05-21T22:06:44.077 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000040726 2024-05-21T22:06:44.077 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:44.077 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_handle_ack_lat": { 2024-05-21T22:06:44.077 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:44.077 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:44.077 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:44.078 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:44.078 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-21T22:06:44.078 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_encrypted_bytes": 0 2024-05-21T22:06:44.078 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:44.078 INFO:teuthology.orchestra.run.smithi184.stdout: "AsyncMessenger::Worker-1": { 2024-05-21T22:06:44.078 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_messages": 963, 2024-05-21T22:06:44.078 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages": 949, 2024-05-21T22:06:44.078 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_bytes": 2076864, 2024-05-21T22:06:44.078 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_bytes": 378216, 2024-05-21T22:06:44.078 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_created_connections": 5, 2024-05-21T22:06:44.078 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_active_connections": 3, 2024-05-21T22:06:44.078 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_total_time": 0.129316689, 2024-05-21T22:06:44.079 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_send_time": 0.042711149, 2024-05-21T22:06:44.079 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_recv_time": 0.061106859, 2024-05-21T22:06:44.079 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_fast_dispatch_time": 0.030702993, 2024-05-21T22:06:44.079 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T22:06:44.079 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 948, 2024-05-21T22:06:44.079 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.043847047, 2024-05-21T22:06:44.079 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000046252 2024-05-21T22:06:44.079 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:44.079 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_handle_ack_lat": { 2024-05-21T22:06:44.079 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:44.079 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:44.079 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:44.079 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:44.079 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_encrypted_bytes": 1911392, 2024-05-21T22:06:44.080 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_encrypted_bytes": 165088 2024-05-21T22:06:44.080 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:44.080 INFO:teuthology.orchestra.run.smithi184.stdout: "AsyncMessenger::Worker-2": { 2024-05-21T22:06:44.080 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_messages": 8, 2024-05-21T22:06:44.080 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages": 206, 2024-05-21T22:06:44.080 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_bytes": 3740, 2024-05-21T22:06:44.080 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_bytes": 582110, 2024-05-21T22:06:44.080 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_created_connections": 8, 2024-05-21T22:06:44.080 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_active_connections": 4, 2024-05-21T22:06:44.080 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_total_time": 0.013618078, 2024-05-21T22:06:44.080 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_send_time": 0.008301179, 2024-05-21T22:06:44.080 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_recv_time": 0.002711643, 2024-05-21T22:06:44.080 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_fast_dispatch_time": 0.000024820, 2024-05-21T22:06:44.081 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages_queue_lat": { 2024-05-21T22:06:44.081 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 203, 2024-05-21T22:06:44.081 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.027993941, 2024-05-21T22:06:44.081 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000137901 2024-05-21T22:06:44.081 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:44.081 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_handle_ack_lat": { 2024-05-21T22:06:44.081 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:44.081 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:44.081 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:44.081 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:44.081 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_encrypted_bytes": 3472, 2024-05-21T22:06:44.081 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_encrypted_bytes": 581792 2024-05-21T22:06:44.081 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:44.081 INFO:teuthology.orchestra.run.smithi184.stdout: "cct": { 2024-05-21T22:06:44.082 INFO:teuthology.orchestra.run.smithi184.stdout: "total_workers": 1, 2024-05-21T22:06:44.082 INFO:teuthology.orchestra.run.smithi184.stdout: "unhealthy_workers": 0 2024-05-21T22:06:44.082 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:44.082 INFO:teuthology.orchestra.run.smithi184.stdout: "finisher-MDSRank": { 2024-05-21T22:06:44.082 INFO:teuthology.orchestra.run.smithi184.stdout: "queue_len": 0, 2024-05-21T22:06:44.082 INFO:teuthology.orchestra.run.smithi184.stdout: "complete_latency": { 2024-05-21T22:06:44.082 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 4229, 2024-05-21T22:06:44.082 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.129979623, 2024-05-21T22:06:44.082 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000030735 2024-05-21T22:06:44.082 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-05-21T22:06:44.082 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:44.082 INFO:teuthology.orchestra.run.smithi184.stdout: "finisher-PurgeQueue": { 2024-05-21T22:06:44.083 INFO:teuthology.orchestra.run.smithi184.stdout: "queue_len": 0, 2024-05-21T22:06:44.083 INFO:teuthology.orchestra.run.smithi184.stdout: "complete_latency": { 2024-05-21T22:06:44.083 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:44.083 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:44.083 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:44.083 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-05-21T22:06:44.083 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:44.083 INFO:teuthology.orchestra.run.smithi184.stdout: "mds": { 2024-05-21T22:06:44.083 INFO:teuthology.orchestra.run.smithi184.stdout: "request": 0, 2024-05-21T22:06:44.083 INFO:teuthology.orchestra.run.smithi184.stdout: "reply": 0, 2024-05-21T22:06:44.083 INFO:teuthology.orchestra.run.smithi184.stdout: "reply_latency": { 2024-05-21T22:06:44.083 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:44.083 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:44.083 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:44.083 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:44.083 INFO:teuthology.orchestra.run.smithi184.stdout: "slow_reply": 0, 2024-05-21T22:06:44.084 INFO:teuthology.orchestra.run.smithi184.stdout: "forward": 0, 2024-05-21T22:06:44.084 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_fetch_complete": 0, 2024-05-21T22:06:44.084 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_fetch_keys": 0, 2024-05-21T22:06:44.084 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_commit": 0, 2024-05-21T22:06:44.084 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_split": 0, 2024-05-21T22:06:44.084 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_merge": 0, 2024-05-21T22:06:44.084 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes": 0, 2024-05-21T22:06:44.084 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_top": 0, 2024-05-21T22:06:44.084 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_bottom": 0, 2024-05-21T22:06:44.084 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_pin_tail": 0, 2024-05-21T22:06:44.084 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_pinned": 0, 2024-05-21T22:06:44.084 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_expired": 0, 2024-05-21T22:06:44.084 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_with_caps": 0, 2024-05-21T22:06:44.084 INFO:teuthology.orchestra.run.smithi184.stdout: "caps": 0, 2024-05-21T22:06:44.084 INFO:teuthology.orchestra.run.smithi184.stdout: "subtrees": 1, 2024-05-21T22:06:44.085 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse": 0, 2024-05-21T22:06:44.085 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_hit": 0, 2024-05-21T22:06:44.085 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_forward": 0, 2024-05-21T22:06:44.085 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_discover": 0, 2024-05-21T22:06:44.085 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_dir_fetch": 0, 2024-05-21T22:06:44.085 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_remote_ino": 0, 2024-05-21T22:06:44.085 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_lock": 0, 2024-05-21T22:06:44.085 INFO:teuthology.orchestra.run.smithi184.stdout: "load_cent": 0, 2024-05-21T22:06:44.085 INFO:teuthology.orchestra.run.smithi184.stdout: "q": 0, 2024-05-21T22:06:44.085 INFO:teuthology.orchestra.run.smithi184.stdout: "exported": 0, 2024-05-21T22:06:44.085 INFO:teuthology.orchestra.run.smithi184.stdout: "exported_inodes": 0, 2024-05-21T22:06:44.085 INFO:teuthology.orchestra.run.smithi184.stdout: "imported": 0, 2024-05-21T22:06:44.085 INFO:teuthology.orchestra.run.smithi184.stdout: "imported_inodes": 0, 2024-05-21T22:06:44.086 INFO:teuthology.orchestra.run.smithi184.stdout: "openino_dir_fetch": 0, 2024-05-21T22:06:44.086 INFO:teuthology.orchestra.run.smithi184.stdout: "openino_backtrace_fetch": 0, 2024-05-21T22:06:44.086 INFO:teuthology.orchestra.run.smithi184.stdout: "openino_peer_discover": 0, 2024-05-21T22:06:44.086 INFO:teuthology.orchestra.run.smithi184.stdout: "root_rfiles": 0, 2024-05-21T22:06:44.086 INFO:teuthology.orchestra.run.smithi184.stdout: "root_rbytes": 0, 2024-05-21T22:06:44.086 INFO:teuthology.orchestra.run.smithi184.stdout: "root_rsnaps": 0, 2024-05-21T22:06:44.086 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_backtrace_fetch": 0, 2024-05-21T22:06:44.086 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_set_tag": 0, 2024-05-21T22:06:44.086 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_backtrace_repaired": 0, 2024-05-21T22:06:44.086 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_inotable_repaired": 0, 2024-05-21T22:06:44.086 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_dir_inodes": 0, 2024-05-21T22:06:44.086 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_dir_base_inodes": 0, 2024-05-21T22:06:44.086 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_dirfrag_rstats": 0, 2024-05-21T22:06:44.086 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_file_inodes": 0, 2024-05-21T22:06:44.087 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_inode_file_caps": 0, 2024-05-21T22:06:44.087 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_revoke": 0, 2024-05-21T22:06:44.087 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_grant": 0, 2024-05-21T22:06:44.087 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_trunc": 0, 2024-05-21T22:06:44.087 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-21T22:06:44.087 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-21T22:06:44.087 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_caps": 0, 2024-05-21T22:06:44.087 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_caps_dirty": 0, 2024-05-21T22:06:44.087 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_cap_release": 0, 2024-05-21T22:06:44.087 INFO:teuthology.orchestra.run.smithi184.stdout: "process_request_cap_release": 0 2024-05-21T22:06:44.087 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:44.087 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_cache": { 2024-05-21T22:06:44.087 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_update": 0, 2024-05-21T22:06:44.088 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_update_receipt": 0, 2024-05-21T22:06:44.088 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_try_discover": 0, 2024-05-21T22:06:44.088 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_send_discover": 0, 2024-05-21T22:06:44.088 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_handle_discover": 0, 2024-05-21T22:06:44.088 INFO:teuthology.orchestra.run.smithi184.stdout: "num_strays": 0, 2024-05-21T22:06:44.088 INFO:teuthology.orchestra.run.smithi184.stdout: "num_strays_delayed": 0, 2024-05-21T22:06:44.088 INFO:teuthology.orchestra.run.smithi184.stdout: "num_strays_enqueuing": 0, 2024-05-21T22:06:44.088 INFO:teuthology.orchestra.run.smithi184.stdout: "strays_created": 0, 2024-05-21T22:06:44.088 INFO:teuthology.orchestra.run.smithi184.stdout: "strays_enqueued": 0, 2024-05-21T22:06:44.088 INFO:teuthology.orchestra.run.smithi184.stdout: "strays_reintegrated": 0, 2024-05-21T22:06:44.088 INFO:teuthology.orchestra.run.smithi184.stdout: "strays_migrated": 0, 2024-05-21T22:06:44.088 INFO:teuthology.orchestra.run.smithi184.stdout: "num_recovering_processing": 0, 2024-05-21T22:06:44.088 INFO:teuthology.orchestra.run.smithi184.stdout: "num_recovering_enqueued": 0, 2024-05-21T22:06:44.088 INFO:teuthology.orchestra.run.smithi184.stdout: "num_recovering_prioritized": 0, 2024-05-21T22:06:44.088 INFO:teuthology.orchestra.run.smithi184.stdout: "recovery_started": 0, 2024-05-21T22:06:44.089 INFO:teuthology.orchestra.run.smithi184.stdout: "recovery_completed": 0, 2024-05-21T22:06:44.089 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_quiesce_path": 0, 2024-05-21T22:06:44.089 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_quiesce_inode": 0, 2024-05-21T22:06:44.089 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_enqueue_scrub": 0, 2024-05-21T22:06:44.089 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_exportdir": 0, 2024-05-21T22:06:44.089 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_flush": 0, 2024-05-21T22:06:44.089 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_fragmentdir": 0, 2024-05-21T22:06:44.089 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_fragstats": 0, 2024-05-21T22:06:44.089 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_inodestats": 0 2024-05-21T22:06:44.089 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:44.089 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_log": { 2024-05-21T22:06:44.089 INFO:teuthology.orchestra.run.smithi184.stdout: "evlrg": 0, 2024-05-21T22:06:44.089 INFO:teuthology.orchestra.run.smithi184.stdout: "evadd": 0, 2024-05-21T22:06:44.089 INFO:teuthology.orchestra.run.smithi184.stdout: "evex": 0, 2024-05-21T22:06:44.090 INFO:teuthology.orchestra.run.smithi184.stdout: "evtrm": 0, 2024-05-21T22:06:44.090 INFO:teuthology.orchestra.run.smithi184.stdout: "ev": 8, 2024-05-21T22:06:44.090 INFO:teuthology.orchestra.run.smithi184.stdout: "evexg": 0, 2024-05-21T22:06:44.090 INFO:teuthology.orchestra.run.smithi184.stdout: "evexd": 0, 2024-05-21T22:06:44.090 INFO:teuthology.orchestra.run.smithi184.stdout: "segadd": 0, 2024-05-21T22:06:44.090 INFO:teuthology.orchestra.run.smithi184.stdout: "segex": 0, 2024-05-21T22:06:44.090 INFO:teuthology.orchestra.run.smithi184.stdout: "segtrm": 0, 2024-05-21T22:06:44.090 INFO:teuthology.orchestra.run.smithi184.stdout: "seg": 2, 2024-05-21T22:06:44.090 INFO:teuthology.orchestra.run.smithi184.stdout: "segmjr": 2, 2024-05-21T22:06:44.090 INFO:teuthology.orchestra.run.smithi184.stdout: "segexg": 0, 2024-05-21T22:06:44.090 INFO:teuthology.orchestra.run.smithi184.stdout: "segexd": 0, 2024-05-21T22:06:44.090 INFO:teuthology.orchestra.run.smithi184.stdout: "expos": 4194304, 2024-05-21T22:06:44.090 INFO:teuthology.orchestra.run.smithi184.stdout: "wrpos": 4196657, 2024-05-21T22:06:44.091 INFO:teuthology.orchestra.run.smithi184.stdout: "rdpos": 4196516, 2024-05-21T22:06:44.091 INFO:teuthology.orchestra.run.smithi184.stdout: "jlat": { 2024-05-21T22:06:44.091 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:44.091 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:44.091 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:44.091 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:44.091 INFO:teuthology.orchestra.run.smithi184.stdout: "replayed": 8 2024-05-21T22:06:44.091 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:44.091 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_mem": { 2024-05-21T22:06:44.091 INFO:teuthology.orchestra.run.smithi184.stdout: "ino": 3, 2024-05-21T22:06:44.091 INFO:teuthology.orchestra.run.smithi184.stdout: "ino+": 3, 2024-05-21T22:06:44.091 INFO:teuthology.orchestra.run.smithi184.stdout: "ino-": 0, 2024-05-21T22:06:44.091 INFO:teuthology.orchestra.run.smithi184.stdout: "dir": 1, 2024-05-21T22:06:44.091 INFO:teuthology.orchestra.run.smithi184.stdout: "dir+": 1, 2024-05-21T22:06:44.092 INFO:teuthology.orchestra.run.smithi184.stdout: "dir-": 0, 2024-05-21T22:06:44.092 INFO:teuthology.orchestra.run.smithi184.stdout: "dn": 0, 2024-05-21T22:06:44.092 INFO:teuthology.orchestra.run.smithi184.stdout: "dn+": 0, 2024-05-21T22:06:44.092 INFO:teuthology.orchestra.run.smithi184.stdout: "dn-": 0, 2024-05-21T22:06:44.092 INFO:teuthology.orchestra.run.smithi184.stdout: "cap": 0, 2024-05-21T22:06:44.092 INFO:teuthology.orchestra.run.smithi184.stdout: "cap+": 0, 2024-05-21T22:06:44.092 INFO:teuthology.orchestra.run.smithi184.stdout: "cap-": 0, 2024-05-21T22:06:44.092 INFO:teuthology.orchestra.run.smithi184.stdout: "rss": 50280, 2024-05-21T22:06:44.092 INFO:teuthology.orchestra.run.smithi184.stdout: "heap": 215416 2024-05-21T22:06:44.092 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:44.092 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_server": { 2024-05-21T22:06:44.092 INFO:teuthology.orchestra.run.smithi184.stdout: "dispatch_client_request": 0, 2024-05-21T22:06:44.092 INFO:teuthology.orchestra.run.smithi184.stdout: "dispatch_server_request": 0, 2024-05-21T22:06:44.093 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_request": 0, 2024-05-21T22:06:44.093 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_session": 0, 2024-05-21T22:06:44.093 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_peer_request": 0, 2024-05-21T22:06:44.093 INFO:teuthology.orchestra.run.smithi184.stdout: "req_create_latency": { 2024-05-21T22:06:44.093 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:44.093 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:44.093 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:44.093 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:44.093 INFO:teuthology.orchestra.run.smithi184.stdout: "req_getattr_latency": { 2024-05-21T22:06:44.093 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:44.093 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:44.093 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:44.093 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:44.093 INFO:teuthology.orchestra.run.smithi184.stdout: "req_getfilelock_latency": { 2024-05-21T22:06:44.093 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:44.094 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:44.094 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:44.094 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:44.094 INFO:teuthology.orchestra.run.smithi184.stdout: "req_link_latency": { 2024-05-21T22:06:44.094 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:44.094 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:44.094 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:44.094 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:44.094 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookup_latency": { 2024-05-21T22:06:44.094 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:44.094 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:44.094 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:44.094 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:44.095 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookuphash_latency": { 2024-05-21T22:06:44.095 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:44.095 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:44.095 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:44.095 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:44.095 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookupino_latency": { 2024-05-21T22:06:44.095 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:44.095 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:44.095 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:44.095 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:44.095 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookupname_latency": { 2024-05-21T22:06:44.095 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:44.095 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:44.095 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:44.095 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:44.096 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookupparent_latency": { 2024-05-21T22:06:44.096 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:44.096 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:44.096 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:44.096 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:44.096 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookupsnap_latency": { 2024-05-21T22:06:44.096 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:44.096 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:44.096 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:44.096 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:44.096 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lssnap_latency": { 2024-05-21T22:06:44.096 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:44.096 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:44.096 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:44.096 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:44.097 INFO:teuthology.orchestra.run.smithi184.stdout: "req_mkdir_latency": { 2024-05-21T22:06:44.097 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:44.097 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:44.097 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:44.097 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:44.097 INFO:teuthology.orchestra.run.smithi184.stdout: "req_mknod_latency": { 2024-05-21T22:06:44.097 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:44.097 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:44.098 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:44.098 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:44.098 INFO:teuthology.orchestra.run.smithi184.stdout: "req_mksnap_latency": { 2024-05-21T22:06:44.098 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:44.098 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:44.098 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:44.098 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:44.098 INFO:teuthology.orchestra.run.smithi184.stdout: "req_open_latency": { 2024-05-21T22:06:44.098 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:44.098 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:44.098 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:44.098 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:44.098 INFO:teuthology.orchestra.run.smithi184.stdout: "req_readdir_latency": { 2024-05-21T22:06:44.099 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:44.099 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:44.099 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:44.099 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:44.099 INFO:teuthology.orchestra.run.smithi184.stdout: "req_rename_latency": { 2024-05-21T22:06:44.099 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:44.099 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:44.099 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:44.099 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:44.099 INFO:teuthology.orchestra.run.smithi184.stdout: "req_renamesnap_latency": { 2024-05-21T22:06:44.099 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:44.099 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:44.099 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:44.099 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:44.099 INFO:teuthology.orchestra.run.smithi184.stdout: "req_snapdiff_latency": { 2024-05-21T22:06:44.099 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:44.100 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:44.100 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:44.100 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:44.100 INFO:teuthology.orchestra.run.smithi184.stdout: "req_rmdir_latency": { 2024-05-21T22:06:44.100 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:44.100 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:44.100 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:44.100 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:44.100 INFO:teuthology.orchestra.run.smithi184.stdout: "req_rmsnap_latency": { 2024-05-21T22:06:44.100 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:44.100 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:44.100 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:44.100 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:44.100 INFO:teuthology.orchestra.run.smithi184.stdout: "req_rmxattr_latency": { 2024-05-21T22:06:44.101 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:44.101 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:44.101 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:44.101 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:44.101 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setattr_latency": { 2024-05-21T22:06:44.101 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:44.101 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:44.101 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:44.101 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:44.101 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setdirlayout_latency": { 2024-05-21T22:06:44.101 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:44.101 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:44.101 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:44.101 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:44.102 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setfilelock_latency": { 2024-05-21T22:06:44.102 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:44.102 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:44.102 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:44.102 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:44.102 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setlayout_latency": { 2024-05-21T22:06:44.102 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:44.102 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:44.102 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:44.102 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:44.102 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setxattr_latency": { 2024-05-21T22:06:44.102 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:44.102 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:44.102 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:44.102 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:44.103 INFO:teuthology.orchestra.run.smithi184.stdout: "req_symlink_latency": { 2024-05-21T22:06:44.103 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:44.103 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:44.103 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:44.103 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:44.103 INFO:teuthology.orchestra.run.smithi184.stdout: "req_unlink_latency": { 2024-05-21T22:06:44.103 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:44.103 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:44.103 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:44.103 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:44.103 INFO:teuthology.orchestra.run.smithi184.stdout: "cap_revoke_eviction": 0, 2024-05-21T22:06:44.103 INFO:teuthology.orchestra.run.smithi184.stdout: "cap_acquisition_throttle": 0, 2024-05-21T22:06:44.103 INFO:teuthology.orchestra.run.smithi184.stdout: "req_getvxattr_latency": { 2024-05-21T22:06:44.103 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:44.104 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:44.104 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:44.104 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-05-21T22:06:44.104 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:44.104 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_sessions": { 2024-05-21T22:06:44.104 INFO:teuthology.orchestra.run.smithi184.stdout: "session_count": 2, 2024-05-21T22:06:44.104 INFO:teuthology.orchestra.run.smithi184.stdout: "session_add": 4, 2024-05-21T22:06:44.104 INFO:teuthology.orchestra.run.smithi184.stdout: "session_remove": 2, 2024-05-21T22:06:44.104 INFO:teuthology.orchestra.run.smithi184.stdout: "sessions_open": 4, 2024-05-21T22:06:44.104 INFO:teuthology.orchestra.run.smithi184.stdout: "sessions_stale": 0, 2024-05-21T22:06:44.104 INFO:teuthology.orchestra.run.smithi184.stdout: "total_load": 0, 2024-05-21T22:06:44.104 INFO:teuthology.orchestra.run.smithi184.stdout: "average_load": 0, 2024-05-21T22:06:44.104 INFO:teuthology.orchestra.run.smithi184.stdout: "avg_session_uptime": 4102, 2024-05-21T22:06:44.104 INFO:teuthology.orchestra.run.smithi184.stdout: "mdthresh_evicted": 0 2024-05-21T22:06:44.104 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:44.105 INFO:teuthology.orchestra.run.smithi184.stdout: "mempool": { 2024-05-21T22:06:44.105 INFO:teuthology.orchestra.run.smithi184.stdout: "bloom_filter_bytes": 0, 2024-05-21T22:06:44.105 INFO:teuthology.orchestra.run.smithi184.stdout: "bloom_filter_items": 0, 2024-05-21T22:06:44.105 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_alloc_bytes": 0, 2024-05-21T22:06:44.105 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_alloc_items": 0, 2024-05-21T22:06:44.105 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_data_bytes": 0, 2024-05-21T22:06:44.105 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_data_items": 0, 2024-05-21T22:06:44.105 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-21T22:06:44.105 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_onode_items": 0, 2024-05-21T22:06:44.105 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-21T22:06:44.105 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_meta_items": 0, 2024-05-21T22:06:44.105 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_other_bytes": 0, 2024-05-21T22:06:44.105 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_other_items": 0, 2024-05-21T22:06:44.105 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-21T22:06:44.106 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_buffer_items": 0, 2024-05-21T22:06:44.106 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_extent_bytes": 0, 2024-05-21T22:06:44.106 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_extent_items": 0, 2024-05-21T22:06:44.106 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_blob_bytes": 0, 2024-05-21T22:06:44.106 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_blob_items": 0, 2024-05-21T22:06:44.106 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-21T22:06:44.106 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_shared_blob_items": 0, 2024-05-21T22:06:44.106 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-21T22:06:44.106 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_inline_bl_items": 0, 2024-05-21T22:06:44.106 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_fsck_bytes": 0, 2024-05-21T22:06:44.106 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_fsck_items": 0, 2024-05-21T22:06:44.106 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_txc_bytes": 0, 2024-05-21T22:06:44.106 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_txc_items": 0, 2024-05-21T22:06:44.106 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-21T22:06:44.107 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_deferred_items": 0, 2024-05-21T22:06:44.107 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_bytes": 0, 2024-05-21T22:06:44.107 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_items": 0, 2024-05-21T22:06:44.107 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_bytes": 0, 2024-05-21T22:06:44.107 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_items": 0, 2024-05-21T22:06:44.107 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_reader_bytes": 0, 2024-05-21T22:06:44.107 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_reader_items": 0, 2024-05-21T22:06:44.107 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_writer_bytes": 0, 2024-05-21T22:06:44.107 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_writer_items": 0, 2024-05-21T22:06:44.107 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_anon_bytes": 190139, 2024-05-21T22:06:44.107 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_anon_items": 52, 2024-05-21T22:06:44.107 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_meta_bytes": 0, 2024-05-21T22:06:44.107 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_meta_items": 0, 2024-05-21T22:06:44.107 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_bytes": 0, 2024-05-21T22:06:44.107 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_items": 0, 2024-05-21T22:06:44.107 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_mapbl_bytes": 0, 2024-05-21T22:06:44.108 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_mapbl_items": 0, 2024-05-21T22:06:44.108 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_pglog_bytes": 0, 2024-05-21T22:06:44.108 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_pglog_items": 0, 2024-05-21T22:06:44.108 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_bytes": 8128, 2024-05-21T22:06:44.108 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_items": 169, 2024-05-21T22:06:44.108 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_mapping_bytes": 0, 2024-05-21T22:06:44.108 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_mapping_items": 0, 2024-05-21T22:06:44.108 INFO:teuthology.orchestra.run.smithi184.stdout: "pgmap_bytes": 0, 2024-05-21T22:06:44.108 INFO:teuthology.orchestra.run.smithi184.stdout: "pgmap_items": 0, 2024-05-21T22:06:44.108 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_co_bytes": 8512, 2024-05-21T22:06:44.108 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_co_items": 13, 2024-05-21T22:06:44.108 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_1_bytes": 0, 2024-05-21T22:06:44.108 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_1_items": 0, 2024-05-21T22:06:44.108 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_2_bytes": 0, 2024-05-21T22:06:44.108 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_2_items": 0 2024-05-21T22:06:44.109 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:44.109 INFO:teuthology.orchestra.run.smithi184.stdout: "objecter": { 2024-05-21T22:06:44.109 INFO:teuthology.orchestra.run.smithi184.stdout: "op_active": 0, 2024-05-21T22:06:44.109 INFO:teuthology.orchestra.run.smithi184.stdout: "op_laggy": 0, 2024-05-21T22:06:44.109 INFO:teuthology.orchestra.run.smithi184.stdout: "op_send": 2485, 2024-05-21T22:06:44.109 INFO:teuthology.orchestra.run.smithi184.stdout: "op_send_bytes": 21, 2024-05-21T22:06:44.109 INFO:teuthology.orchestra.run.smithi184.stdout: "op_resend": 3, 2024-05-21T22:06:44.109 INFO:teuthology.orchestra.run.smithi184.stdout: "op_reply": 2485, 2024-05-21T22:06:44.109 INFO:teuthology.orchestra.run.smithi184.stdout: "op_latency": { 2024-05-21T22:06:44.109 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 2485, 2024-05-21T22:06:44.109 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 1.255974416, 2024-05-21T22:06:44.109 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000505422 2024-05-21T22:06:44.109 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:44.110 INFO:teuthology.orchestra.run.smithi184.stdout: "op_inflight": 0, 2024-05-21T22:06:44.110 INFO:teuthology.orchestra.run.smithi184.stdout: "oplen_avg": { 2024-05-21T22:06:44.110 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 2485, 2024-05-21T22:06:44.110 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 2486 2024-05-21T22:06:44.110 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:44.110 INFO:teuthology.orchestra.run.smithi184.stdout: "op": 2485, 2024-05-21T22:06:44.110 INFO:teuthology.orchestra.run.smithi184.stdout: "op_r": 2485, 2024-05-21T22:06:44.110 INFO:teuthology.orchestra.run.smithi184.stdout: "op_w": 0, 2024-05-21T22:06:44.110 INFO:teuthology.orchestra.run.smithi184.stdout: "op_rmw": 0, 2024-05-21T22:06:44.110 INFO:teuthology.orchestra.run.smithi184.stdout: "op_pg": 0, 2024-05-21T22:06:44.110 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_stat": 1571, 2024-05-21T22:06:44.110 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_create": 0, 2024-05-21T22:06:44.110 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_read": 912, 2024-05-21T22:06:44.110 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_write": 0, 2024-05-21T22:06:44.111 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_writefull": 0, 2024-05-21T22:06:44.111 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_writesame": 0, 2024-05-21T22:06:44.111 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_append": 0, 2024-05-21T22:06:44.111 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_zero": 0, 2024-05-21T22:06:44.111 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_truncate": 0, 2024-05-21T22:06:44.111 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_delete": 0, 2024-05-21T22:06:44.111 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_mapext": 0, 2024-05-21T22:06:44.111 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_sparse_read": 0, 2024-05-21T22:06:44.111 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_clonerange": 0, 2024-05-21T22:06:44.111 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_getxattr": 1, 2024-05-21T22:06:44.111 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_setxattr": 0, 2024-05-21T22:06:44.111 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_cmpxattr": 0, 2024-05-21T22:06:44.111 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_rmxattr": 0, 2024-05-21T22:06:44.111 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_resetxattrs": 0, 2024-05-21T22:06:44.111 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_call": 0, 2024-05-21T22:06:44.112 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_watch": 0, 2024-05-21T22:06:44.112 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_notify": 0, 2024-05-21T22:06:44.112 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_src_cmpxattr": 0, 2024-05-21T22:06:44.112 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_pgls": 0, 2024-05-21T22:06:44.112 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_pgls_filter": 0, 2024-05-21T22:06:44.112 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_other": 0, 2024-05-21T22:06:44.112 INFO:teuthology.orchestra.run.smithi184.stdout: "linger_active": 0, 2024-05-21T22:06:44.112 INFO:teuthology.orchestra.run.smithi184.stdout: "linger_send": 0, 2024-05-21T22:06:44.112 INFO:teuthology.orchestra.run.smithi184.stdout: "linger_resend": 0, 2024-05-21T22:06:44.112 INFO:teuthology.orchestra.run.smithi184.stdout: "linger_ping": 0, 2024-05-21T22:06:44.112 INFO:teuthology.orchestra.run.smithi184.stdout: "poolop_active": 0, 2024-05-21T22:06:44.112 INFO:teuthology.orchestra.run.smithi184.stdout: "poolop_send": 0, 2024-05-21T22:06:44.112 INFO:teuthology.orchestra.run.smithi184.stdout: "poolop_resend": 0, 2024-05-21T22:06:44.112 INFO:teuthology.orchestra.run.smithi184.stdout: "poolstat_active": 0, 2024-05-21T22:06:44.112 INFO:teuthology.orchestra.run.smithi184.stdout: "poolstat_send": 0, 2024-05-21T22:06:44.113 INFO:teuthology.orchestra.run.smithi184.stdout: "poolstat_resend": 0, 2024-05-21T22:06:44.113 INFO:teuthology.orchestra.run.smithi184.stdout: "statfs_active": 0, 2024-05-21T22:06:44.113 INFO:teuthology.orchestra.run.smithi184.stdout: "statfs_send": 0, 2024-05-21T22:06:44.113 INFO:teuthology.orchestra.run.smithi184.stdout: "statfs_resend": 0, 2024-05-21T22:06:44.113 INFO:teuthology.orchestra.run.smithi184.stdout: "command_active": 0, 2024-05-21T22:06:44.113 INFO:teuthology.orchestra.run.smithi184.stdout: "command_send": 0, 2024-05-21T22:06:44.113 INFO:teuthology.orchestra.run.smithi184.stdout: "command_resend": 0, 2024-05-21T22:06:44.113 INFO:teuthology.orchestra.run.smithi184.stdout: "map_epoch": 91, 2024-05-21T22:06:44.113 INFO:teuthology.orchestra.run.smithi184.stdout: "map_full": 0, 2024-05-21T22:06:44.113 INFO:teuthology.orchestra.run.smithi184.stdout: "map_inc": 11, 2024-05-21T22:06:44.113 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_sessions": 5, 2024-05-21T22:06:44.113 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_session_open": 7, 2024-05-21T22:06:44.113 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_session_close": 2, 2024-05-21T22:06:44.113 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_laggy": 0, 2024-05-21T22:06:44.113 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_wr": 0, 2024-05-21T22:06:44.113 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_rd": 2, 2024-05-21T22:06:44.114 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_del": 0 2024-05-21T22:06:44.114 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:44.114 INFO:teuthology.orchestra.run.smithi184.stdout: "oft": { 2024-05-21T22:06:44.114 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_total_objs": 0, 2024-05-21T22:06:44.114 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_total_kv_pairs": 0, 2024-05-21T22:06:44.114 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_total_updates": 0, 2024-05-21T22:06:44.114 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_total_removes": 0 2024-05-21T22:06:44.114 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:44.114 INFO:teuthology.orchestra.run.smithi184.stdout: "osd-slow-ops": { 2024-05-21T22:06:44.114 INFO:teuthology.orchestra.run.smithi184.stdout: "slow_ops_count": 0 2024-05-21T22:06:44.114 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:44.114 INFO:teuthology.orchestra.run.smithi184.stdout: "purge_queue": { 2024-05-21T22:06:44.114 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executing_ops": 0, 2024-05-21T22:06:44.114 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executing_ops_high_water": 0, 2024-05-21T22:06:44.115 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executing": 0, 2024-05-21T22:06:44.115 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executing_high_water": 0, 2024-05-21T22:06:44.115 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executed_ops": 0, 2024-05-21T22:06:44.115 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executed": 0, 2024-05-21T22:06:44.115 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_item_in_journal": 0 2024-05-21T22:06:44.115 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:44.115 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-21T22:06:44.115 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2024-05-21T22:06:44.115 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 104857600, 2024-05-21T22:06:44.115 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2024-05-21T22:06:44.115 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 2781, 2024-05-21T22:06:44.115 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 2356683, 2024-05-21T22:06:44.115 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2024-05-21T22:06:44.116 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 2781, 2024-05-21T22:06:44.116 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 0, 2024-05-21T22:06:44.116 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 0, 2024-05-21T22:06:44.116 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 2781, 2024-05-21T22:06:44.116 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 2356683, 2024-05-21T22:06:44.116 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2024-05-21T22:06:44.116 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:44.116 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:44.116 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:44.116 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-05-21T22:06:44.116 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:44.116 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-objecter_bytes": { 2024-05-21T22:06:44.116 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2024-05-21T22:06:44.116 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 104857600, 2024-05-21T22:06:44.116 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2024-05-21T22:06:44.116 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 0, 2024-05-21T22:06:44.117 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 0, 2024-05-21T22:06:44.117 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2024-05-21T22:06:44.117 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 0, 2024-05-21T22:06:44.117 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 2485, 2024-05-21T22:06:44.117 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 2358, 2024-05-21T22:06:44.117 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 2485, 2024-05-21T22:06:44.117 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 2358, 2024-05-21T22:06:44.117 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2024-05-21T22:06:44.117 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:44.117 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:44.117 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:44.117 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-05-21T22:06:44.117 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:44.117 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-objecter_ops": { 2024-05-21T22:06:44.117 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2024-05-21T22:06:44.118 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 1024, 2024-05-21T22:06:44.118 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2024-05-21T22:06:44.118 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 0, 2024-05-21T22:06:44.118 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 0, 2024-05-21T22:06:44.118 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2024-05-21T22:06:44.118 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 0, 2024-05-21T22:06:44.118 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 2485, 2024-05-21T22:06:44.118 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 2485, 2024-05-21T22:06:44.118 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 2485, 2024-05-21T22:06:44.118 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 2485, 2024-05-21T22:06:44.118 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2024-05-21T22:06:44.118 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:44.118 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:44.119 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:44.119 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-05-21T22:06:44.119 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:44.119 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-write_buf_throttle": { 2024-05-21T22:06:44.119 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2024-05-21T22:06:44.119 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 3758096384, 2024-05-21T22:06:44.119 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2024-05-21T22:06:44.119 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 0, 2024-05-21T22:06:44.119 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 0, 2024-05-21T22:06:44.119 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2024-05-21T22:06:44.119 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 0, 2024-05-21T22:06:44.119 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 0, 2024-05-21T22:06:44.119 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 0, 2024-05-21T22:06:44.119 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 0, 2024-05-21T22:06:44.120 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 0, 2024-05-21T22:06:44.120 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2024-05-21T22:06:44.120 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:44.120 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:44.120 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:44.120 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-05-21T22:06:44.120 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2024-05-21T22:06:44.120 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-write_buf_throttle-0x563c37e7e6e0": { 2024-05-21T22:06:44.120 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2024-05-21T22:06:44.120 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 3758096384, 2024-05-21T22:06:44.120 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2024-05-21T22:06:44.120 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 0, 2024-05-21T22:06:44.120 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 0, 2024-05-21T22:06:44.120 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2024-05-21T22:06:44.121 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 0, 2024-05-21T22:06:44.121 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 0, 2024-05-21T22:06:44.121 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 0, 2024-05-21T22:06:44.121 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 0, 2024-05-21T22:06:44.121 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 0, 2024-05-21T22:06:44.121 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2024-05-21T22:06:44.121 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2024-05-21T22:06:44.121 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2024-05-21T22:06:44.121 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2024-05-21T22:06:44.121 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-05-21T22:06:44.121 INFO:teuthology.orchestra.run.smithi184.stdout: } 2024-05-21T22:06:44.121 INFO:teuthology.orchestra.run.smithi184.stdout:} 2024-05-21T22:06:44.889 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-05-21T22:06:44.890 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-05-21T22:06:44.890 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update=0 2024-05-21T22:06:44.890 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update_receipt=0 2024-05-21T22:06:44.890 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=0 2024-05-21T22:06:44.890 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2024-05-21T22:06:44.890 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2024-05-21T22:06:44.890 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-05-21T22:06:44.901 INFO:tasks.kclient:Unmounting kernel clients... 2024-05-21T22:06:44.901 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-05-21T22:06:44.901 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-21T22:06:44.962 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-05-21T22:06:44.962 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-21T22:06:45.017 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph osd blocklist ls 2024-05-21T22:06:45.450 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:06:45.926 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:06:45 smithi151 ceph-mon[31623]: pgmap v345: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:06:45.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:06:45 smithi184 ceph-mon[36942]: pgmap v345: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:06:46.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:06:45 smithi164 ceph-mon[36269]: pgmap v345: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:06:47.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:47.719+0000 7f37d2400700 1 -- 172.21.15.151:0/4287767676 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f37cc105750 msgr2=0x7f37cc105bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:06:47.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:47.719+0000 7f37d2400700 1 --2- 172.21.15.151:0/4287767676 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f37cc105750 0x7f37cc105bd0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f37b4009990 tx=0x7f37b4009ca0 comp rx=0 tx=0).stop 2024-05-21T22:06:47.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:47.719+0000 7f37d2400700 1 -- 172.21.15.151:0/4287767676 shutdown_connections 2024-05-21T22:06:47.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:47.719+0000 7f37d2400700 1 --2- 172.21.15.151:0/4287767676 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37cc10b750 0x7f37cc10bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:47.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:47.719+0000 7f37d2400700 1 --2- 172.21.15.151:0/4287767676 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37cc106110 0x7f37cc112620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:47.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:47.719+0000 7f37d2400700 1 --2- 172.21.15.151:0/4287767676 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f37cc105750 0x7f37cc105bd0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:47.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:47.719+0000 7f37d2400700 1 -- 172.21.15.151:0/4287767676 >> 172.21.15.151:0/4287767676 conn(0x7f37cc101240 msgr2=0x7f37cc103660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:06:47.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:47.720+0000 7f37d2400700 1 -- 172.21.15.151:0/4287767676 shutdown_connections 2024-05-21T22:06:47.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:47.720+0000 7f37d2400700 1 -- 172.21.15.151:0/4287767676 wait complete. 2024-05-21T22:06:47.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:47.721+0000 7f37d2400700 1 Processor -- start 2024-05-21T22:06:47.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:47.721+0000 7f37d2400700 1 -- start start 2024-05-21T22:06:47.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:47.722+0000 7f37d2400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f37cc105750 0x7f37cc19bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:06:47.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:47.722+0000 7f37d2400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37cc106110 0x7f37cc19c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:06:47.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:47.722+0000 7f37d2400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37cc10b750 0x7f37cc1a0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:06:47.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:47.722+0000 7f37d2400700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f37cc1159a0 con 0x7f37cc105750 2024-05-21T22:06:47.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:47.722+0000 7f37d2400700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f37cc115820 con 0x7f37cc10b750 2024-05-21T22:06:47.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:47.722+0000 7f37d2400700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f37cc115b20 con 0x7f37cc106110 2024-05-21T22:06:47.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:47.722+0000 7f37cb400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f37cc105750 0x7f37cc19bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:06:47.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:47.722+0000 7f37cb400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f37cc105750 0x7f37cc19bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:37736/0 (socket says 172.21.15.151:37736) 2024-05-21T22:06:47.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:47.722+0000 7f37cb400700 1 -- 172.21.15.151:0/1175269406 learned_addr learned my addr 172.21.15.151:0/1175269406 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:06:47.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:47.722+0000 7f37c3e00700 1 --2- 172.21.15.151:0/1175269406 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37cc106110 0x7f37cc19c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:06:47.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:47.722+0000 7f37cbe00700 1 --2- 172.21.15.151:0/1175269406 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37cc10b750 0x7f37cc1a0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:06:47.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:47.723+0000 7f37cb400700 1 -- 172.21.15.151:0/1175269406 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37cc106110 msgr2=0x7f37cc19c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:06:47.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:47.723+0000 7f37cb400700 1 --2- 172.21.15.151:0/1175269406 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37cc106110 0x7f37cc19c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:47.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:47.723+0000 7f37cb400700 1 -- 172.21.15.151:0/1175269406 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37cc10b750 msgr2=0x7f37cc1a0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:06:47.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:47.723+0000 7f37cb400700 1 --2- 172.21.15.151:0/1175269406 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37cc10b750 0x7f37cc1a0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:47.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:47.723+0000 7f37cb400700 1 -- 172.21.15.151:0/1175269406 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37cc1a0e90 con 0x7f37cc105750 2024-05-21T22:06:47.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:47.723+0000 7f37c3e00700 1 --2- 172.21.15.151:0/1175269406 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37cc106110 0x7f37cc19c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:06:47.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:47.723+0000 7f37cbe00700 1 --2- 172.21.15.151:0/1175269406 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37cc10b750 0x7f37cc1a0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T22:06:47.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:47.723+0000 7f37cb400700 1 --2- 172.21.15.151:0/1175269406 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f37cc105750 0x7f37cc19bed0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f37bc00e9a0 tx=0x7f37bc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:06:47.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:47.723+0000 7f37c8c00700 1 -- 172.21.15.151:0/1175269406 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37bc00cbd0 con 0x7f37cc105750 2024-05-21T22:06:47.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:47.723+0000 7f37c8c00700 1 -- 172.21.15.151:0/1175269406 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37bc00cd70 con 0x7f37cc105750 2024-05-21T22:06:47.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:47.723+0000 7f37d2400700 1 -- 172.21.15.151:0/1175269406 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37cc1a1180 con 0x7f37cc105750 2024-05-21T22:06:47.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:47.724+0000 7f37d2400700 1 -- 172.21.15.151:0/1175269406 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37cc1a8a00 con 0x7f37cc105750 2024-05-21T22:06:47.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:47.724+0000 7f37c8c00700 1 -- 172.21.15.151:0/1175269406 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37bc018680 con 0x7f37cc105750 2024-05-21T22:06:47.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:47.725+0000 7f37d2400700 1 -- 172.21.15.151:0/1175269406 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37a8005330 con 0x7f37cc105750 2024-05-21T22:06:47.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:47.729+0000 7f37c8c00700 1 -- 172.21.15.151:0/1175269406 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7f37bc018820 con 0x7f37cc105750 2024-05-21T22:06:47.732 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:47.730+0000 7f37c8c00700 1 --2- 172.21.15.151:0/1175269406 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f37b00b29f0 0x7f37b00b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:06:47.732 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:47.730+0000 7f37c8c00700 1 -- 172.21.15.151:0/1175269406 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7f37bc0e2c60 con 0x7f37cc105750 2024-05-21T22:06:47.732 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:47.731+0000 7f37c3e00700 1 --2- 172.21.15.151:0/1175269406 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f37b00b29f0 0x7f37b00b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:06:47.733 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:47.731+0000 7f37c3e00700 1 --2- 172.21.15.151:0/1175269406 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f37b00b29f0 0x7f37b00b4eb0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f37cc1038d0 tx=0x7f37b400b260 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:06:47.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:47.733+0000 7f37c8c00700 1 -- 172.21.15.151:0/1175269406 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f37bc0aa5d0 con 0x7f37cc105750 2024-05-21T22:06:47.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:06:47 smithi151 ceph-mon[31623]: pgmap v346: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:06:47.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:06:47 smithi184 ceph-mon[36942]: pgmap v346: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:06:47.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:47.984+0000 7f37d2400700 1 -- 172.21.15.151:0/1175269406 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f37a8005620 con 0x7f37cc105750 2024-05-21T22:06:47.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:47.985+0000 7f37c8c00700 1 -- 172.21.15.151:0/1175269406 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v91) ==== 81+0+959 (secure 0 0 0) 0x7f37bc0a9ed0 con 0x7f37cc105750 2024-05-21T22:06:47.987 INFO:teuthology.orchestra.run.smithi151.stdout:172.21.15.151:0/3481112063 2024-05-22T21:55:36.189531+0000 2024-05-21T22:06:47.987 INFO:teuthology.orchestra.run.smithi151.stdout:172.21.15.151:0/2415279581 2024-05-22T21:55:36.189531+0000 2024-05-21T22:06:47.987 INFO:teuthology.orchestra.run.smithi151.stdout:172.21.15.151:0/1946920088 2024-05-22T21:55:36.189531+0000 2024-05-21T22:06:47.987 INFO:teuthology.orchestra.run.smithi151.stdout:172.21.15.151:6800/2796622483 2024-05-22T21:55:36.189531+0000 2024-05-21T22:06:47.987 INFO:teuthology.orchestra.run.smithi151.stdout:172.21.15.151:0/31963128 2024-05-22T21:43:09.041455+0000 2024-05-21T22:06:47.987 INFO:teuthology.orchestra.run.smithi151.stdout:172.21.15.151:6801/2796622483 2024-05-22T21:55:36.189531+0000 2024-05-21T22:06:47.987 INFO:teuthology.orchestra.run.smithi151.stdout:172.21.15.151:6800/3428827300 2024-05-22T21:43:09.041455+0000 2024-05-21T22:06:47.987 INFO:teuthology.orchestra.run.smithi151.stdout:172.21.15.151:0/3232339048 2024-05-22T21:55:36.189531+0000 2024-05-21T22:06:47.987 INFO:teuthology.orchestra.run.smithi151.stdout:172.21.15.151:6801/3428827300 2024-05-22T21:43:09.041455+0000 2024-05-21T22:06:47.987 INFO:teuthology.orchestra.run.smithi151.stdout:172.21.15.151:0/131989141 2024-05-22T21:43:09.041455+0000 2024-05-21T22:06:47.987 INFO:teuthology.orchestra.run.smithi151.stdout:172.21.15.151:6801/1128885292 2024-05-22T21:42:35.440147+0000 2024-05-21T22:06:47.987 INFO:teuthology.orchestra.run.smithi151.stdout:172.21.15.151:0/2703664519 2024-05-22T21:43:09.041455+0000 2024-05-21T22:06:47.988 INFO:teuthology.orchestra.run.smithi151.stdout:172.21.15.151:6800/1128885292 2024-05-22T21:42:35.440147+0000 2024-05-21T22:06:47.988 INFO:teuthology.orchestra.run.smithi151.stdout:172.21.15.151:0/3052943699 2024-05-22T21:42:35.440147+0000 2024-05-21T22:06:47.988 INFO:teuthology.orchestra.run.smithi151.stdout:172.21.15.151:0/3929452511 2024-05-22T21:42:35.440147+0000 2024-05-21T22:06:47.988 INFO:teuthology.orchestra.run.smithi151.stdout:172.21.15.151:0/3762889700 2024-05-22T21:42:35.440147+0000 2024-05-21T22:06:47.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:47.988+0000 7f37d2400700 1 -- 172.21.15.151:0/1175269406 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f37b00b29f0 msgr2=0x7f37b00b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:06:47.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:47.988+0000 7f37d2400700 1 --2- 172.21.15.151:0/1175269406 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f37b00b29f0 0x7f37b00b4eb0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f37cc1038d0 tx=0x7f37b400b260 comp rx=0 tx=0).stop 2024-05-21T22:06:47.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:47.988+0000 7f37d2400700 1 -- 172.21.15.151:0/1175269406 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f37cc105750 msgr2=0x7f37cc19bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:06:47.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:47.988+0000 7f37d2400700 1 --2- 172.21.15.151:0/1175269406 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f37cc105750 0x7f37cc19bed0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f37bc00e9a0 tx=0x7f37bc00ecb0 comp rx=0 tx=0).stop 2024-05-21T22:06:47.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:47.988+0000 7f37d2400700 1 -- 172.21.15.151:0/1175269406 shutdown_connections 2024-05-21T22:06:47.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:47.988+0000 7f37d2400700 1 --2- 172.21.15.151:0/1175269406 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f37b00b29f0 0x7f37b00b4eb0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:47.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:47.988+0000 7f37d2400700 1 --2- 172.21.15.151:0/1175269406 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37cc10b750 0x7f37cc1a0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:47.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:47.988+0000 7f37d2400700 1 --2- 172.21.15.151:0/1175269406 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f37cc105750 0x7f37cc19bed0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:47.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:47.988+0000 7f37d2400700 1 --2- 172.21.15.151:0/1175269406 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37cc106110 0x7f37cc19c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:47.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:47.988+0000 7f37d2400700 1 -- 172.21.15.151:0/1175269406 >> 172.21.15.151:0/1175269406 conn(0x7f37cc101240 msgr2=0x7f37cc1019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:06:47.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:47.988+0000 7f37d2400700 1 -- 172.21.15.151:0/1175269406 shutdown_connections 2024-05-21T22:06:47.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:47.988+0000 7f37d2400700 1 -- 172.21.15.151:0/1175269406 wait complete. 2024-05-21T22:06:47.991 INFO:teuthology.orchestra.run.smithi151.stderr:listed 16 entries 2024-05-21T22:06:48.035 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:06:47 smithi164 ceph-mon[36269]: pgmap v346: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:06:48.843 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-21T22:06:48.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-21T22:06:48.843 DEBUG:teuthology.orchestra.run.smithi151:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-21T22:06:48.952 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:06:48 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/1175269406' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-21T22:06:48.954 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi151.front.sepia.ceph.com 2024-05-21T22:06:48.954 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-21T22:06:48.954 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-21T22:06:48.985 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:06:48 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/1175269406' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-21T22:06:49.017 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-21T22:06:49.017 DEBUG:teuthology.orchestra.run.smithi151:> ip netns list 2024-05-21T22:06:49.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:06:48 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/1175269406' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-21T22:06:49.054 INFO:teuthology.orchestra.run.smithi151.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-21T22:06:49.054 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-21T22:06:49.055 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-21T22:06:49.157 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-21T22:06:49.157 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip link delete ceph-brx 2024-05-21T22:06:49.268 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-05-21T22:06:49.279 INFO:tasks.cephadm:Teardown begin 2024-05-21T22:06:49.279 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-21T22:06:49.279 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-21T22:06:49.683 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:06:49.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:06:49 smithi151 ceph-mon[31623]: pgmap v347: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T22:06:49.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:06:49 smithi184 ceph-mon[36942]: pgmap v347: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T22:06:50.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:06:49 smithi164 ceph-mon[36269]: pgmap v347: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-21T22:06:51.864 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:06:51 smithi151 ceph-mon[31623]: pgmap v348: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:06:51.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:06:51 smithi184 ceph-mon[36942]: pgmap v348: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:06:52.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:52.031+0000 7fae28e00700 1 -- 172.21.15.151:0/3604179883 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae24106030 msgr2=0x7fae24112540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:06:52.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:52.031+0000 7fae28e00700 1 --2- 172.21.15.151:0/3604179883 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae24106030 0x7fae24112540 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fae1c00b0a0 tx=0x7fae1c00b3b0 comp rx=0 tx=0).stop 2024-05-21T22:06:52.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:52.031+0000 7fae28e00700 1 -- 172.21.15.151:0/3604179883 shutdown_connections 2024-05-21T22:06:52.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:52.031+0000 7fae28e00700 1 --2- 172.21.15.151:0/3604179883 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae24106030 0x7fae24112540 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:52.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:52.031+0000 7fae28e00700 1 --2- 172.21.15.151:0/3604179883 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fae2410b670 0x7fae2410ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:52.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:52.031+0000 7fae28e00700 1 --2- 172.21.15.151:0/3604179883 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fae24105670 0x7fae24105af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:52.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:52.031+0000 7fae28e00700 1 -- 172.21.15.151:0/3604179883 >> 172.21.15.151:0/3604179883 conn(0x7fae24101160 msgr2=0x7fae24103580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:06:52.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:52.031+0000 7fae28e00700 1 -- 172.21.15.151:0/3604179883 shutdown_connections 2024-05-21T22:06:52.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:52.031+0000 7fae28e00700 1 -- 172.21.15.151:0/3604179883 wait complete. 2024-05-21T22:06:52.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:52.032+0000 7fae28e00700 1 Processor -- start 2024-05-21T22:06:52.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:52.033+0000 7fae28e00700 1 -- start start 2024-05-21T22:06:52.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:52.033+0000 7fae28e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fae24105670 0x7fae241a1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:06:52.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:52.033+0000 7fae28e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fae24106030 0x7fae241a1e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:06:52.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:52.033+0000 7fae28e00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae2410b670 0x7fae2419bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:06:52.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:52.033+0000 7fae28e00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fae24115900 con 0x7fae24105670 2024-05-21T22:06:52.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:52.033+0000 7fae28e00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fae24115780 con 0x7fae24106030 2024-05-21T22:06:52.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:52.033+0000 7fae28e00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fae24115a80 con 0x7fae2410b670 2024-05-21T22:06:52.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:52.033+0000 7fae23400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fae24105670 0x7fae241a1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:06:52.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:52.034+0000 7fae23400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fae24105670 0x7fae241a1910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:37756/0 (socket says 172.21.15.151:37756) 2024-05-21T22:06:52.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:52.034+0000 7fae23400700 1 -- 172.21.15.151:0/3025787996 learned_addr learned my addr 172.21.15.151:0/3025787996 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:06:52.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:52.034+0000 7fae22a00700 1 --2- 172.21.15.151:0/3025787996 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fae24106030 0x7fae241a1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:06:52.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:52.034+0000 7fae23e00700 1 --2- 172.21.15.151:0/3025787996 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae2410b670 0x7fae2419bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:06:52.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:52.034+0000 7fae22a00700 1 -- 172.21.15.151:0/3025787996 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae2410b670 msgr2=0x7fae2419bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:06:52.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:52.034+0000 7fae22a00700 1 --2- 172.21.15.151:0/3025787996 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae2410b670 0x7fae2419bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:52.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:52.034+0000 7fae22a00700 1 -- 172.21.15.151:0/3025787996 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fae24105670 msgr2=0x7fae241a1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:06:52.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:52.034+0000 7fae22a00700 1 --2- 172.21.15.151:0/3025787996 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fae24105670 0x7fae241a1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:52.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:52.034+0000 7fae22a00700 1 -- 172.21.15.151:0/3025787996 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae2419c280 con 0x7fae24106030 2024-05-21T22:06:52.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:52.034+0000 7fae22a00700 1 --2- 172.21.15.151:0/3025787996 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fae24106030 0x7fae241a1e50 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fae1000e9f0 tx=0x7fae1000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:06:52.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:52.035+0000 7fae1be00700 1 -- 172.21.15.151:0/3025787996 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae1000cb70 con 0x7fae24106030 2024-05-21T22:06:52.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:52.035+0000 7fae1be00700 1 -- 172.21.15.151:0/3025787996 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae1000ee90 con 0x7fae24106030 2024-05-21T22:06:52.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:52.035+0000 7fae1be00700 1 -- 172.21.15.151:0/3025787996 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae10003d40 con 0x7fae24106030 2024-05-21T22:06:52.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:52.035+0000 7fae28e00700 1 -- 172.21.15.151:0/3025787996 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae2419c570 con 0x7fae24106030 2024-05-21T22:06:52.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:52.035+0000 7fae28e00700 1 -- 172.21.15.151:0/3025787996 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae241a87e0 con 0x7fae24106030 2024-05-21T22:06:52.037 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:06:51 smithi164 ceph-mon[36269]: pgmap v348: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-21T22:06:52.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:52.036+0000 7fae28e00700 1 -- 172.21.15.151:0/3025787996 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae2406a230 con 0x7fae24106030 2024-05-21T22:06:52.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:52.039+0000 7fae1be00700 1 -- 172.21.15.151:0/3025787996 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7fae10003ee0 con 0x7fae24106030 2024-05-21T22:06:52.041 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:52.041+0000 7fae1be00700 1 --2- 172.21.15.151:0/3025787996 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fae080b2a70 0x7fae080b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:06:52.041 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:52.041+0000 7fae1be00700 1 -- 172.21.15.151:0/3025787996 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7fae100e2940 con 0x7fae24106030 2024-05-21T22:06:52.041 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:52.041+0000 7fae23400700 1 --2- 172.21.15.151:0/3025787996 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fae080b2a70 0x7fae080b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:06:52.042 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:52.042+0000 7fae23400700 1 --2- 172.21.15.151:0/3025787996 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fae080b2a70 0x7fae080b4f30 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fae14007c10 tx=0x7fae14007480 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:06:52.042 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:52.042+0000 7fae1be00700 1 -- 172.21.15.151:0/3025787996 <== mon.2 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fae10054df0 con 0x7fae24106030 2024-05-21T22:06:52.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:52.329+0000 7fae28e00700 1 -- 172.21.15.151:0/3025787996 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fae2419cfc0 con 0x7fae24106030 2024-05-21T22:06:52.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:52.330+0000 7fae1be00700 1 -- 172.21.15.151:0/3025787996 <== mon.2 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11958 (secure 0 0 0) 0x7fae100aa2b0 con 0x7fae24106030 2024-05-21T22:06:52.331 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T22:06:52.331 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14358,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/158010529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":158010529},{"type":"v1","addr":"172.21.15.151:6835","nonce":158010529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6839/2069058115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":2069058115},{"type":"v1","addr":"172.21.15.151:6839","nonce":2069058115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T21:51:19.552069+0000","modified":"2024-05-21T21:55:34.643755+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24530,"mds_1":24493,"mds_2":24491,"mds_3":24478,"mds_4":24473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14403":{"gid":14403,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6841/1749723794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6840","nonce":1749723794},{"type":"v1","addr":"172.21.15.151:6841","nonce":1749723794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/2026266114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":2026266114},{"type":"v1","addr":"172.21.15.184:6833","nonce":2026266114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/2467900844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":2467900844},{"type":"v1","addr":"172.21.15.164:6833","nonce":2467900844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3367640960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3367640960},{"type":"v1","addr":"172.21.15.151:6837","nonce":3367640960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/3264142621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3264142621},{"type":"v1","addr":"172.21.15.184:6835","nonce":3264142621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":17,"addr":"172.21.15.164:6835/2340428470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2340428470},{"type":"v1","addr":"172.21.15.164:6835","nonce":2340428470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":16,"addr":"172.21.15.184:6837/2988194888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":2988194888},{"type":"v1","addr":"172.21.15.184:6837","nonce":2988194888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.164:6837/1554609572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1554609572},{"type":"v1","addr":"172.21.15.164:6837","nonce":1554609572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.184:6839/3059935143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":3059935143},{"type":"v1","addr":"172.21.15.184:6839","nonce":3059935143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.164:6839/2646401247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":2646401247},{"type":"v1","addr":"172.21.15.164:6839","nonce":2646401247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24530,"qdb_cluster":[24530,24493,24491,24473,24478]},"id":1}]} 2024-05-21T22:06:52.334 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:52.333+0000 7fae28e00700 1 -- 172.21.15.151:0/3025787996 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fae080b2a70 msgr2=0x7fae080b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:06:52.334 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:52.333+0000 7fae28e00700 1 --2- 172.21.15.151:0/3025787996 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fae080b2a70 0x7fae080b4f30 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fae14007c10 tx=0x7fae14007480 comp rx=0 tx=0).stop 2024-05-21T22:06:52.334 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:52.333+0000 7fae28e00700 1 -- 172.21.15.151:0/3025787996 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fae24106030 msgr2=0x7fae241a1e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:06:52.334 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:52.333+0000 7fae28e00700 1 --2- 172.21.15.151:0/3025787996 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fae24106030 0x7fae241a1e50 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fae1000e9f0 tx=0x7fae1000ed00 comp rx=0 tx=0).stop 2024-05-21T22:06:52.334 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:52.333+0000 7fae28e00700 1 -- 172.21.15.151:0/3025787996 shutdown_connections 2024-05-21T22:06:52.334 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:52.333+0000 7fae28e00700 1 --2- 172.21.15.151:0/3025787996 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fae080b2a70 0x7fae080b4f30 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:52.334 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:52.333+0000 7fae28e00700 1 --2- 172.21.15.151:0/3025787996 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae2410b670 0x7fae2419bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:52.334 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:52.333+0000 7fae28e00700 1 --2- 172.21.15.151:0/3025787996 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fae24105670 0x7fae241a1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:52.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:52.333+0000 7fae28e00700 1 --2- 172.21.15.151:0/3025787996 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fae24106030 0x7fae241a1e50 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:52.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:52.333+0000 7fae28e00700 1 -- 172.21.15.151:0/3025787996 >> 172.21.15.151:0/3025787996 conn(0x7fae24101160 msgr2=0x7fae24101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:06:52.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:52.334+0000 7fae28e00700 1 -- 172.21.15.151:0/3025787996 shutdown_connections 2024-05-21T22:06:52.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:52.334+0000 7fae28e00700 1 -- 172.21.15.151:0/3025787996 wait complete. 2024-05-21T22:06:52.335 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 29 2024-05-21T22:06:52.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:06:52 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/3025787996' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T22:06:53.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:06:52 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/3025787996' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T22:06:53.197 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:06:52 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/3025787996' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T22:06:53.250 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph fs dump --format=json 2024-05-21T22:06:53.701 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:06:53.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:06:53 smithi184 ceph-mon[36942]: pgmap v349: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 597 B/s wr, 13 op/s 2024-05-21T22:06:54.020 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:06:53 smithi151 ceph-mon[31623]: pgmap v349: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 597 B/s wr, 13 op/s 2024-05-21T22:06:54.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:06:53 smithi164 ceph-mon[36269]: pgmap v349: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 597 B/s wr, 13 op/s 2024-05-21T22:06:55.890 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:55.889+0000 7ff567200700 1 -- 172.21.15.151:0/2306228897 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff560114410 msgr2=0x7ff560116800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:06:55.890 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:55.889+0000 7ff567200700 1 --2- 172.21.15.151:0/2306228897 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff560114410 0x7ff560116800 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7ff558009f90 tx=0x7ff55800b330 comp rx=0 tx=0).stop 2024-05-21T22:06:55.890 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:55.889+0000 7ff567200700 1 -- 172.21.15.151:0/2306228897 shutdown_connections 2024-05-21T22:06:55.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:55.889+0000 7ff567200700 1 --2- 172.21.15.151:0/2306228897 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff560114410 0x7ff560116800 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:55.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:55.889+0000 7ff567200700 1 --2- 172.21.15.151:0/2306228897 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff560077a70 0x7ff560077e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:55.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:55.889+0000 7ff567200700 1 --2- 172.21.15.151:0/2306228897 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff560078390 0x7ff560113ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:55.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:55.889+0000 7ff567200700 1 -- 172.21.15.151:0/2306228897 >> 172.21.15.151:0/2306228897 conn(0x7ff560101260 msgr2=0x7ff560103680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:06:55.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:55.890+0000 7ff567200700 1 -- 172.21.15.151:0/2306228897 shutdown_connections 2024-05-21T22:06:55.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:55.890+0000 7ff567200700 1 -- 172.21.15.151:0/2306228897 wait complete. 2024-05-21T22:06:55.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:55.891+0000 7ff567200700 1 Processor -- start 2024-05-21T22:06:55.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:55.891+0000 7ff567200700 1 -- start start 2024-05-21T22:06:55.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:55.891+0000 7ff567200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff560077a70 0x7ff5601a02e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:06:55.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:55.892+0000 7ff567200700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff560078390 0x7ff5601a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:06:55.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:55.892+0000 7ff567200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff560114410 0x7ff5601a4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:06:55.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:55.892+0000 7ff567200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff560119db0 con 0x7ff560114410 2024-05-21T22:06:55.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:55.892+0000 7ff567200700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff560119c30 con 0x7ff560077a70 2024-05-21T22:06:55.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:55.892+0000 7ff567200700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7ff560119f30 con 0x7ff560078390 2024-05-21T22:06:55.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:55.892+0000 7ff55fe00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff560077a70 0x7ff5601a02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:06:55.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:55.892+0000 7ff55fe00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff560077a70 0x7ff5601a02e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:38112/0 (socket says 172.21.15.151:38112) 2024-05-21T22:06:55.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:55.892+0000 7ff55fe00700 1 -- 172.21.15.151:0/418129214 learned_addr learned my addr 172.21.15.151:0/418129214 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:06:55.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:55.892+0000 7ff565200700 1 --2- 172.21.15.151:0/418129214 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff560114410 0x7ff5601a4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:06:55.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:55.892+0000 7ff55f400700 1 --2- 172.21.15.151:0/418129214 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff560078390 0x7ff5601a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:06:55.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:55.892+0000 7ff55fe00700 1 -- 172.21.15.151:0/418129214 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff560078390 msgr2=0x7ff5601a0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:06:55.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:55.892+0000 7ff55fe00700 1 --2- 172.21.15.151:0/418129214 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff560078390 0x7ff5601a0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:55.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:55.892+0000 7ff55fe00700 1 -- 172.21.15.151:0/418129214 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff560114410 msgr2=0x7ff5601a4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:06:55.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:55.892+0000 7ff55fe00700 1 --2- 172.21.15.151:0/418129214 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff560114410 0x7ff5601a4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:55.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:55.892+0000 7ff55fe00700 1 -- 172.21.15.151:0/418129214 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5601a52f0 con 0x7ff560077a70 2024-05-21T22:06:55.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:55.893+0000 7ff55fe00700 1 --2- 172.21.15.151:0/418129214 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff560077a70 0x7ff5601a02e0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7ff55000e9a0 tx=0x7ff55000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:06:55.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:55.893+0000 7ff55cc00700 1 -- 172.21.15.151:0/418129214 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff55000c9c0 con 0x7ff560077a70 2024-05-21T22:06:55.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:55.893+0000 7ff567200700 1 -- 172.21.15.151:0/418129214 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5601a55e0 con 0x7ff560077a70 2024-05-21T22:06:55.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:55.893+0000 7ff55cc00700 1 -- 172.21.15.151:0/418129214 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff55000cb60 con 0x7ff560077a70 2024-05-21T22:06:55.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:55.893+0000 7ff55cc00700 1 -- 172.21.15.151:0/418129214 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff550018680 con 0x7ff560077a70 2024-05-21T22:06:55.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:55.893+0000 7ff567200700 1 -- 172.21.15.151:0/418129214 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5601ace60 con 0x7ff560077a70 2024-05-21T22:06:55.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:55.895+0000 7ff567200700 1 -- 172.21.15.151:0/418129214 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff56006a230 con 0x7ff560077a70 2024-05-21T22:06:55.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:55.896+0000 7ff55cc00700 1 -- 172.21.15.151:0/418129214 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7ff550010430 con 0x7ff560077a70 2024-05-21T22:06:55.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:55.898+0000 7ff55cc00700 1 --2- 172.21.15.151:0/418129214 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7ff5440b29f0 0x7ff5440b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:06:55.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:55.898+0000 7ff55cc00700 1 -- 172.21.15.151:0/418129214 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7ff5500e2920 con 0x7ff560077a70 2024-05-21T22:06:55.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:55.898+0000 7ff55f400700 1 --2- 172.21.15.151:0/418129214 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7ff5440b29f0 0x7ff5440b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:06:55.899 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:55.899+0000 7ff55f400700 1 --2- 172.21.15.151:0/418129214 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7ff5440b29f0 0x7ff5440b4eb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff560104920 tx=0x7ff54c009310 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:06:55.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:55.902+0000 7ff55cc00700 1 -- 172.21.15.151:0/418129214 <== mon.2 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff5500aa290 con 0x7ff560077a70 2024-05-21T22:06:55.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:06:55 smithi184 ceph-mon[36942]: pgmap v350: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 9 op/s 2024-05-21T22:06:56.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:06:55 smithi164 ceph-mon[36269]: pgmap v350: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 9 op/s 2024-05-21T22:06:56.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:56.181+0000 7ff567200700 1 -- 172.21.15.151:0/418129214 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff56002e560 con 0x7ff560077a70 2024-05-21T22:06:56.181 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:06:55 smithi151 ceph-mon[31623]: pgmap v350: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 9 op/s 2024-05-21T22:06:56.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:56.182+0000 7ff55cc00700 1 -- 172.21.15.151:0/418129214 <== mon.2 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11958 (secure 0 0 0) 0x7ff5500b2310 con 0x7ff560077a70 2024-05-21T22:06:56.183 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T22:06:56.183 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14358,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/158010529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":158010529},{"type":"v1","addr":"172.21.15.151:6835","nonce":158010529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14397,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6839/2069058115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":2069058115},{"type":"v1","addr":"172.21.15.151:6839","nonce":2069058115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-21T21:51:19.552069+0000","modified":"2024-05-21T21:55:34.643755+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24530,"mds_1":24493,"mds_2":24491,"mds_3":24478,"mds_4":24473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14403":{"gid":14403,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6841/1749723794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6840","nonce":1749723794},{"type":"v1","addr":"172.21.15.151:6841","nonce":1749723794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6833/2026266114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":2026266114},{"type":"v1","addr":"172.21.15.184:6833","nonce":2026266114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.164:6833/2467900844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":2467900844},{"type":"v1","addr":"172.21.15.164:6833","nonce":2467900844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3367640960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3367640960},{"type":"v1","addr":"172.21.15.151:6837","nonce":3367640960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/3264142621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":3264142621},{"type":"v1","addr":"172.21.15.184:6835","nonce":3264142621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":17,"addr":"172.21.15.164:6835/2340428470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":2340428470},{"type":"v1","addr":"172.21.15.164:6835","nonce":2340428470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":16,"addr":"172.21.15.184:6837/2988194888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":2988194888},{"type":"v1","addr":"172.21.15.184:6837","nonce":2988194888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":15,"addr":"172.21.15.164:6837/1554609572","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6836","nonce":1554609572},{"type":"v1","addr":"172.21.15.164:6837","nonce":1554609572}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":14,"addr":"172.21.15.184:6839/3059935143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":3059935143},{"type":"v1","addr":"172.21.15.184:6839","nonce":3059935143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24530":{"gid":24530,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.164:6839/2646401247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6838","nonce":2646401247},{"type":"v1","addr":"172.21.15.164:6839","nonce":2646401247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24530,"qdb_cluster":[24530,24493,24491,24473,24478]},"id":1}]} 2024-05-21T22:06:56.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:56.185+0000 7ff567200700 1 -- 172.21.15.151:0/418129214 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7ff5440b29f0 msgr2=0x7ff5440b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:06:56.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:56.185+0000 7ff567200700 1 --2- 172.21.15.151:0/418129214 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7ff5440b29f0 0x7ff5440b4eb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff560104920 tx=0x7ff54c009310 comp rx=0 tx=0).stop 2024-05-21T22:06:56.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:56.185+0000 7ff567200700 1 -- 172.21.15.151:0/418129214 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff560077a70 msgr2=0x7ff5601a02e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:06:56.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:56.185+0000 7ff567200700 1 --2- 172.21.15.151:0/418129214 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff560077a70 0x7ff5601a02e0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7ff55000e9a0 tx=0x7ff55000ecb0 comp rx=0 tx=0).stop 2024-05-21T22:06:56.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:56.186+0000 7ff567200700 1 -- 172.21.15.151:0/418129214 shutdown_connections 2024-05-21T22:06:56.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:56.186+0000 7ff567200700 1 --2- 172.21.15.151:0/418129214 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7ff5440b29f0 0x7ff5440b4eb0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:56.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:56.186+0000 7ff567200700 1 --2- 172.21.15.151:0/418129214 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff560077a70 0x7ff5601a02e0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:56.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:56.186+0000 7ff567200700 1 --2- 172.21.15.151:0/418129214 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff560114410 0x7ff5601a4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:56.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:56.186+0000 7ff567200700 1 --2- 172.21.15.151:0/418129214 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff560078390 0x7ff5601a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:56.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:56.186+0000 7ff567200700 1 -- 172.21.15.151:0/418129214 >> 172.21.15.151:0/418129214 conn(0x7ff560101260 msgr2=0x7ff560103650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:06:56.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:56.186+0000 7ff567200700 1 -- 172.21.15.151:0/418129214 shutdown_connections 2024-05-21T22:06:56.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:56.186+0000 7ff567200700 1 -- 172.21.15.151:0/418129214 wait complete. 2024-05-21T22:06:56.188 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 29 2024-05-21T22:06:56.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:06:56 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/418129214' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T22:06:56.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:06:56 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/418129214' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T22:06:57.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:06:56 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/418129214' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-21T22:06:57.183 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph osd dump --format=json 2024-05-21T22:06:57.646 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:06:58.002 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:06:57 smithi151 ceph-mon[31623]: pgmap v351: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 9 op/s 2024-05-21T22:06:58.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:06:57 smithi164 ceph-mon[36269]: pgmap v351: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 9 op/s 2024-05-21T22:06:58.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:06:57 smithi184 ceph-mon[36942]: pgmap v351: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 9 op/s 2024-05-21T22:06:59.791 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:06:59 smithi151 ceph-mon[31623]: pgmap v352: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 597 B/s wr, 13 op/s 2024-05-21T22:06:59.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:59.861+0000 7f0f1f200700 1 -- 172.21.15.151:0/4107150938 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0f20075970 msgr2=0x7f0f20075d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:06:59.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:59.861+0000 7f0f1f200700 1 --2- 172.21.15.151:0/4107150938 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0f20075970 0x7f0f20075d50 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f0f10009990 tx=0x7f0f10009ca0 comp rx=0 tx=0).stop 2024-05-21T22:06:59.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:59.862+0000 7f0f1f200700 1 -- 172.21.15.151:0/4107150938 shutdown_connections 2024-05-21T22:06:59.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:59.862+0000 7f0f1f200700 1 --2- 172.21.15.151:0/4107150938 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f2010fcb0 0x7f0f20112120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:59.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:59.862+0000 7f0f1f200700 1 --2- 172.21.15.151:0/4107150938 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0f20075970 0x7f0f20075d50 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:59.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:59.862+0000 7f0f1f200700 1 --2- 172.21.15.151:0/4107150938 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f20076290 0x7f0f2010f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:59.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:59.862+0000 7f0f1f200700 1 -- 172.21.15.151:0/4107150938 >> 172.21.15.151:0/4107150938 conn(0x7f0f200fefc0 msgr2=0x7f0f201013e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:06:59.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:59.862+0000 7f0f1f200700 1 -- 172.21.15.151:0/4107150938 shutdown_connections 2024-05-21T22:06:59.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:59.862+0000 7f0f1f200700 1 -- 172.21.15.151:0/4107150938 wait complete. 2024-05-21T22:06:59.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:59.863+0000 7f0f1f200700 1 Processor -- start 2024-05-21T22:06:59.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:59.863+0000 7f0f1f200700 1 -- start start 2024-05-21T22:06:59.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:59.863+0000 7f0f1f200700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0f20075970 0x7f0f201033d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:06:59.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:59.863+0000 7f0f1f200700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f20076290 0x7f0f20103910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:06:59.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:59.863+0000 7f0f1f200700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f2010fcb0 0x7f0f20103e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:06:59.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:59.863+0000 7f0f1f200700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0f20115880 con 0x7f0f20075970 2024-05-21T22:06:59.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:59.863+0000 7f0f1f200700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0f20115700 con 0x7f0f2010fcb0 2024-05-21T22:06:59.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:59.863+0000 7f0f1f200700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f0f20115a00 con 0x7f0f20076290 2024-05-21T22:06:59.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:59.864+0000 7f0f1de00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0f20075970 0x7f0f201033d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:06:59.865 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:59.864+0000 7f0f1d400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f20076290 0x7f0f20103910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:06:59.865 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:59.864+0000 7f0f1de00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0f20075970 0x7f0f201033d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:42044/0 (socket says 172.21.15.151:42044) 2024-05-21T22:06:59.865 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:59.864+0000 7f0f1de00700 1 -- 172.21.15.151:0/1549275850 learned_addr learned my addr 172.21.15.151:0/1549275850 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:06:59.865 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:59.864+0000 7f0f1e800700 1 --2- 172.21.15.151:0/1549275850 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f2010fcb0 0x7f0f20103e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:06:59.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:59.864+0000 7f0f1e800700 1 -- 172.21.15.151:0/1549275850 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f20076290 msgr2=0x7f0f20103910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:06:59.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:59.864+0000 7f0f1e800700 1 --2- 172.21.15.151:0/1549275850 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f20076290 0x7f0f20103910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:59.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:59.864+0000 7f0f1e800700 1 -- 172.21.15.151:0/1549275850 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0f20075970 msgr2=0x7f0f201033d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:06:59.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:59.864+0000 7f0f1e800700 1 --2- 172.21.15.151:0/1549275850 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0f20075970 0x7f0f201033d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:06:59.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:59.864+0000 7f0f1e800700 1 -- 172.21.15.151:0/1549275850 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f201ac440 con 0x7f0f2010fcb0 2024-05-21T22:06:59.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:59.864+0000 7f0f1d400700 1 --2- 172.21.15.151:0/1549275850 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f20076290 0x7f0f20103910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:06:59.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:59.864+0000 7f0f1de00700 1 --2- 172.21.15.151:0/1549275850 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0f20075970 0x7f0f201033d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:06:59.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:59.865+0000 7f0f1e800700 1 --2- 172.21.15.151:0/1549275850 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f2010fcb0 0x7f0f20103e50 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f0f18009f60 tx=0x7f0f18009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:06:59.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:59.865+0000 7f0f16a00700 1 -- 172.21.15.151:0/1549275850 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f180092e0 con 0x7f0f2010fcb0 2024-05-21T22:06:59.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:59.865+0000 7f0f16a00700 1 -- 172.21.15.151:0/1549275850 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f18004bb0 con 0x7f0f2010fcb0 2024-05-21T22:06:59.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:59.865+0000 7f0f16a00700 1 -- 172.21.15.151:0/1549275850 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f18004020 con 0x7f0f2010fcb0 2024-05-21T22:06:59.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:59.865+0000 7f0f1f200700 1 -- 172.21.15.151:0/1549275850 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f201ac6d0 con 0x7f0f2010fcb0 2024-05-21T22:06:59.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:59.865+0000 7f0f1f200700 1 -- 172.21.15.151:0/1549275850 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f201acbc0 con 0x7f0f2010fcb0 2024-05-21T22:06:59.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:59.866+0000 7f0f1f200700 1 -- 172.21.15.151:0/1549275850 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f20077020 con 0x7f0f2010fcb0 2024-05-21T22:06:59.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:59.868+0000 7f0f16a00700 1 -- 172.21.15.151:0/1549275850 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7f0f1801b030 con 0x7f0f2010fcb0 2024-05-21T22:06:59.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:59.870+0000 7f0f16a00700 1 --2- 172.21.15.151:0/1549275850 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f0f040b2a70 0x7f0f040b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:06:59.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:59.870+0000 7f0f16a00700 1 -- 172.21.15.151:0/1549275850 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7f0f180e2100 con 0x7f0f2010fcb0 2024-05-21T22:06:59.871 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:59.870+0000 7f0f1de00700 1 --2- 172.21.15.151:0/1549275850 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f0f040b2a70 0x7f0f040b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:06:59.871 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:59.871+0000 7f0f1de00700 1 --2- 172.21.15.151:0/1549275850 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f0f040b2a70 0x7f0f040b4f30 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0f10006010 tx=0x7f0f1000b280 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:06:59.872 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:06:59.872+0000 7f0f16a00700 1 -- 172.21.15.151:0/1549275850 <== mon.2 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0f18022020 con 0x7f0f2010fcb0 2024-05-21T22:07:00.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:06:59 smithi164 ceph-mon[36269]: pgmap v352: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 597 B/s wr, 13 op/s 2024-05-21T22:07:00.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:00.104+0000 7f0f1f200700 1 -- 172.21.15.151:0/1549275850 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0f2002e720 con 0x7f0f2010fcb0 2024-05-21T22:07:00.106 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:00.106+0000 7f0f16a00700 1 -- 172.21.15.151:0/1549275850 <== mon.2 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+23218 (secure 0 0 0) 0x7f0f18010040 con 0x7f0f2010fcb0 2024-05-21T22:07:00.107 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-05-21T22:07:00.108 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":91,"fsid":"c8c7a2e6-17ba-11ef-bc9a-c7b262605968","created":"2024-05-21T21:41:52.412833+0000","modified":"2024-05-21T22:06:14.192622+0000","last_up_change":"2024-05-21T21:50:05.401314+0000","last_in_change":"2024-05-21T21:49:45.684136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-21T21:45:55.196046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-21T21:51:11.447428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-21T21:51:15.618605+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":16,"snap_epoch":90,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"18db8e54-0cfc-4e12-9c84-6f72a99cc989","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":2420400473},{"type":"v1","addr":"172.21.15.151:6803","nonce":2420400473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":2420400473},{"type":"v1","addr":"172.21.15.151:6805","nonce":2420400473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":2420400473},{"type":"v1","addr":"172.21.15.151:6809","nonce":2420400473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":2420400473},{"type":"v1","addr":"172.21.15.151:6807","nonce":2420400473}]},"public_addr":"172.21.15.151:6803/2420400473","cluster_addr":"172.21.15.151:6805/2420400473","heartbeat_back_addr":"172.21.15.151:6809/2420400473","heartbeat_front_addr":"172.21.15.151:6807/2420400473","state":["exists","up"]},{"osd":1,"uuid":"bac384c7-1102-4815-81a8-10cc7661d230","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":3900240447},{"type":"v1","addr":"172.21.15.164:6801","nonce":3900240447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":3900240447},{"type":"v1","addr":"172.21.15.164:6803","nonce":3900240447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":3900240447},{"type":"v1","addr":"172.21.15.164:6807","nonce":3900240447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":3900240447},{"type":"v1","addr":"172.21.15.164:6805","nonce":3900240447}]},"public_addr":"172.21.15.164:6801/3900240447","cluster_addr":"172.21.15.164:6803/3900240447","heartbeat_back_addr":"172.21.15.164:6807/3900240447","heartbeat_front_addr":"172.21.15.164:6805/3900240447","state":["exists","up"]},{"osd":2,"uuid":"b7e4a782-1518-4d43-bee9-7fd768ee5cb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":3971150737},{"type":"v1","addr":"172.21.15.184:6801","nonce":3971150737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":3971150737},{"type":"v1","addr":"172.21.15.184:6803","nonce":3971150737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":3971150737},{"type":"v1","addr":"172.21.15.184:6807","nonce":3971150737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":3971150737},{"type":"v1","addr":"172.21.15.184:6805","nonce":3971150737}]},"public_addr":"172.21.15.184:6801/3971150737","cluster_addr":"172.21.15.184:6803/3971150737","heartbeat_back_addr":"172.21.15.184:6807/3971150737","heartbeat_front_addr":"172.21.15.184:6805/3971150737","state":["exists","up"]},{"osd":3,"uuid":"ab3df826-39e3-4635-a49c-2b5ffdbf12f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":556594400},{"type":"v1","addr":"172.21.15.151:6811","nonce":556594400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":556594400},{"type":"v1","addr":"172.21.15.151:6813","nonce":556594400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":556594400},{"type":"v1","addr":"172.21.15.151:6817","nonce":556594400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":556594400},{"type":"v1","addr":"172.21.15.151:6815","nonce":556594400}]},"public_addr":"172.21.15.151:6811/556594400","cluster_addr":"172.21.15.151:6813/556594400","heartbeat_back_addr":"172.21.15.151:6817/556594400","heartbeat_front_addr":"172.21.15.151:6815/556594400","state":["exists","up"]},{"osd":4,"uuid":"00cbf6ae-0f42-4685-a77d-d803c6467aa0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":988881973},{"type":"v1","addr":"172.21.15.164:6809","nonce":988881973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":988881973},{"type":"v1","addr":"172.21.15.164:6811","nonce":988881973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":988881973},{"type":"v1","addr":"172.21.15.164:6815","nonce":988881973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":988881973},{"type":"v1","addr":"172.21.15.164:6813","nonce":988881973}]},"public_addr":"172.21.15.164:6809/988881973","cluster_addr":"172.21.15.164:6811/988881973","heartbeat_back_addr":"172.21.15.164:6815/988881973","heartbeat_front_addr":"172.21.15.164:6813/988881973","state":["exists","up"]},{"osd":5,"uuid":"fad14221-0ef0-4d40-b414-c6c9f1f9718c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3293311106},{"type":"v1","addr":"172.21.15.184:6809","nonce":3293311106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":3293311106},{"type":"v1","addr":"172.21.15.184:6811","nonce":3293311106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":3293311106},{"type":"v1","addr":"172.21.15.184:6815","nonce":3293311106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":3293311106},{"type":"v1","addr":"172.21.15.184:6813","nonce":3293311106}]},"public_addr":"172.21.15.184:6809/3293311106","cluster_addr":"172.21.15.184:6811/3293311106","heartbeat_back_addr":"172.21.15.184:6815/3293311106","heartbeat_front_addr":"172.21.15.184:6813/3293311106","state":["exists","up"]},{"osd":6,"uuid":"ab8a60f6-0dec-4c1c-89db-f2e2f638f1ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":3377550762},{"type":"v1","addr":"172.21.15.151:6819","nonce":3377550762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":3377550762},{"type":"v1","addr":"172.21.15.151:6821","nonce":3377550762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":3377550762},{"type":"v1","addr":"172.21.15.151:6825","nonce":3377550762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":3377550762},{"type":"v1","addr":"172.21.15.151:6823","nonce":3377550762}]},"public_addr":"172.21.15.151:6819/3377550762","cluster_addr":"172.21.15.151:6821/3377550762","heartbeat_back_addr":"172.21.15.151:6825/3377550762","heartbeat_front_addr":"172.21.15.151:6823/3377550762","state":["exists","up"]},{"osd":7,"uuid":"db332268-9972-4ed7-a307-822b83a0fbb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":3869696936},{"type":"v1","addr":"172.21.15.164:6817","nonce":3869696936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":3869696936},{"type":"v1","addr":"172.21.15.164:6819","nonce":3869696936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":3869696936},{"type":"v1","addr":"172.21.15.164:6823","nonce":3869696936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":3869696936},{"type":"v1","addr":"172.21.15.164:6821","nonce":3869696936}]},"public_addr":"172.21.15.164:6817/3869696936","cluster_addr":"172.21.15.164:6819/3869696936","heartbeat_back_addr":"172.21.15.164:6823/3869696936","heartbeat_front_addr":"172.21.15.164:6821/3869696936","state":["exists","up"]},{"osd":8,"uuid":"fe2fbe1c-549a-4947-a188-426dfca04d10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":575273541},{"type":"v1","addr":"172.21.15.184:6817","nonce":575273541}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":575273541},{"type":"v1","addr":"172.21.15.184:6819","nonce":575273541}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":575273541},{"type":"v1","addr":"172.21.15.184:6823","nonce":575273541}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":575273541},{"type":"v1","addr":"172.21.15.184:6821","nonce":575273541}]},"public_addr":"172.21.15.184:6817/575273541","cluster_addr":"172.21.15.184:6819/575273541","heartbeat_back_addr":"172.21.15.184:6823/575273541","heartbeat_front_addr":"172.21.15.184:6821/575273541","state":["exists","up"]},{"osd":9,"uuid":"3a7b450f-8004-4adf-bae3-970918f98f50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":337128533},{"type":"v1","addr":"172.21.15.151:6827","nonce":337128533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":337128533},{"type":"v1","addr":"172.21.15.151:6829","nonce":337128533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":337128533},{"type":"v1","addr":"172.21.15.151:6833","nonce":337128533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":337128533},{"type":"v1","addr":"172.21.15.151:6831","nonce":337128533}]},"public_addr":"172.21.15.151:6827/337128533","cluster_addr":"172.21.15.151:6829/337128533","heartbeat_back_addr":"172.21.15.151:6833/337128533","heartbeat_front_addr":"172.21.15.151:6831/337128533","state":["exists","up"]},{"osd":10,"uuid":"8fcdf271-90c4-4db3-be69-b0ca716dfd80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":1823448782},{"type":"v1","addr":"172.21.15.164:6825","nonce":1823448782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":1823448782},{"type":"v1","addr":"172.21.15.164:6827","nonce":1823448782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":1823448782},{"type":"v1","addr":"172.21.15.164:6831","nonce":1823448782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":1823448782},{"type":"v1","addr":"172.21.15.164:6829","nonce":1823448782}]},"public_addr":"172.21.15.164:6825/1823448782","cluster_addr":"172.21.15.164:6827/1823448782","heartbeat_back_addr":"172.21.15.164:6831/1823448782","heartbeat_front_addr":"172.21.15.164:6829/1823448782","state":["exists","up"]},{"osd":11,"uuid":"e661477b-0c09-409f-a2fe-6ebd34642a8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":1563340128},{"type":"v1","addr":"172.21.15.184:6825","nonce":1563340128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":1563340128},{"type":"v1","addr":"172.21.15.184:6827","nonce":1563340128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":1563340128},{"type":"v1","addr":"172.21.15.184:6831","nonce":1563340128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":1563340128},{"type":"v1","addr":"172.21.15.184:6829","nonce":1563340128}]},"public_addr":"172.21.15.184:6825/1563340128","cluster_addr":"172.21.15.184:6827/1563340128","heartbeat_back_addr":"172.21.15.184:6831/1563340128","heartbeat_front_addr":"172.21.15.184:6829/1563340128","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:45:02.821160+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:45:27.844564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:45:51.904981+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:46:24.376245+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:46:50.036621+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:47:14.748422+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:47:47.396383+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:48:13.543174+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:48:38.213285+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:49:11.980042+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:49:38.519761+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T21:50:03.439898+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.151:0/3481112063":"2024-05-22T21:55:36.189531+0000","172.21.15.151:0/2415279581":"2024-05-22T21:55:36.189531+0000","172.21.15.151:0/1946920088":"2024-05-22T21:55:36.189531+0000","172.21.15.151:6800/2796622483":"2024-05-22T21:55:36.189531+0000","172.21.15.151:0/31963128":"2024-05-22T21:43:09.041455+0000","172.21.15.151:6801/2796622483":"2024-05-22T21:55:36.189531+0000","172.21.15.151:6800/3428827300":"2024-05-22T21:43:09.041455+0000","172.21.15.151:0/3232339048":"2024-05-22T21:55:36.189531+0000","172.21.15.151:6801/3428827300":"2024-05-22T21:43:09.041455+0000","172.21.15.151:0/131989141":"2024-05-22T21:43:09.041455+0000","172.21.15.151:6801/1128885292":"2024-05-22T21:42:35.440147+0000","172.21.15.151:0/2703664519":"2024-05-22T21:43:09.041455+0000","172.21.15.151:6800/1128885292":"2024-05-22T21:42:35.440147+0000","172.21.15.151:0/3052943699":"2024-05-22T21:42:35.440147+0000","172.21.15.151:0/3929452511":"2024-05-22T21:42:35.440147+0000","172.21.15.151:0/3762889700":"2024-05-22T21:42:35.440147+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":16,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-21T22:07:00.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:00.110+0000 7f0f1f200700 1 -- 172.21.15.151:0/1549275850 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f0f040b2a70 msgr2=0x7f0f040b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:07:00.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:00.110+0000 7f0f1f200700 1 --2- 172.21.15.151:0/1549275850 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f0f040b2a70 0x7f0f040b4f30 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0f10006010 tx=0x7f0f1000b280 comp rx=0 tx=0).stop 2024-05-21T22:07:00.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:00.110+0000 7f0f1f200700 1 -- 172.21.15.151:0/1549275850 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f2010fcb0 msgr2=0x7f0f20103e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:07:00.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:00.110+0000 7f0f1f200700 1 --2- 172.21.15.151:0/1549275850 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f2010fcb0 0x7f0f20103e50 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f0f18009f60 tx=0x7f0f18009f90 comp rx=0 tx=0).stop 2024-05-21T22:07:00.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:00.110+0000 7f0f1f200700 1 -- 172.21.15.151:0/1549275850 shutdown_connections 2024-05-21T22:07:00.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:00.110+0000 7f0f1f200700 1 --2- 172.21.15.151:0/1549275850 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f0f040b2a70 0x7f0f040b4f30 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:07:00.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:00.110+0000 7f0f1f200700 1 --2- 172.21.15.151:0/1549275850 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f2010fcb0 0x7f0f20103e50 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:07:00.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:00.110+0000 7f0f1f200700 1 --2- 172.21.15.151:0/1549275850 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0f20075970 0x7f0f201033d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:07:00.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:00.110+0000 7f0f1f200700 1 --2- 172.21.15.151:0/1549275850 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f20076290 0x7f0f20103910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:07:00.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:00.110+0000 7f0f1f200700 1 -- 172.21.15.151:0/1549275850 >> 172.21.15.151:0/1549275850 conn(0x7f0f200fefc0 msgr2=0x7f0f200ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:07:00.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:00.110+0000 7f0f1f200700 1 -- 172.21.15.151:0/1549275850 shutdown_connections 2024-05-21T22:07:00.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:00.110+0000 7f0f1f200700 1 -- 172.21.15.151:0/1549275850 wait complete. 2024-05-21T22:07:00.238 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:06:59 smithi184 ceph-mon[36942]: pgmap v352: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 597 B/s wr, 13 op/s 2024-05-21T22:07:00.955 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph fs fail cephfs 2024-05-21T22:07:01.022 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:07:00 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/1549275850' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T22:07:01.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:07:00 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/1549275850' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T22:07:01.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:07:00 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/1549275850' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-21T22:07:01.384 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:07:01.850 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:07:01 smithi151 ceph-mon[31623]: pgmap v353: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 9 op/s 2024-05-21T22:07:02.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:07:01 smithi164 ceph-mon[36269]: pgmap v353: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 9 op/s 2024-05-21T22:07:02.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:07:01 smithi184 ceph-mon[36942]: pgmap v353: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 9 op/s 2024-05-21T22:07:03.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:03.595+0000 7f844b600700 1 -- 172.21.15.151:0/1210398837 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f844406c5d0 msgr2=0x7f84441103f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:07:03.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:03.595+0000 7f844b600700 1 --2- 172.21.15.151:0/1210398837 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f844406c5d0 0x7f84441103f0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f843c009f90 tx=0x7f843c00b330 comp rx=0 tx=0).stop 2024-05-21T22:07:03.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:03.596+0000 7f844b600700 1 -- 172.21.15.151:0/1210398837 shutdown_connections 2024-05-21T22:07:03.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:03.596+0000 7f844b600700 1 --2- 172.21.15.151:0/1210398837 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f84441095d0 0x7f84441099b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:07:03.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:03.596+0000 7f844b600700 1 --2- 172.21.15.151:0/1210398837 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f844406c5d0 0x7f84441103f0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:07:03.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:03.596+0000 7f844b600700 1 --2- 172.21.15.151:0/1210398837 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f844406bc10 0x7f844406c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:07:03.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:03.596+0000 7f844b600700 1 -- 172.21.15.151:0/1210398837 >> 172.21.15.151:0/1210398837 conn(0x7f84440793f0 msgr2=0x7f8444079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:07:03.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:03.596+0000 7f844b600700 1 -- 172.21.15.151:0/1210398837 shutdown_connections 2024-05-21T22:07:03.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:03.596+0000 7f844b600700 1 -- 172.21.15.151:0/1210398837 wait complete. 2024-05-21T22:07:03.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:03.597+0000 7f844b600700 1 Processor -- start 2024-05-21T22:07:03.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:03.598+0000 7f844b600700 1 -- start start 2024-05-21T22:07:03.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:03.598+0000 7f844b600700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f844406bc10 0x7f844419be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:07:03.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:03.598+0000 7f844b600700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f844406c5d0 0x7f844419c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:07:03.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:03.598+0000 7f844b600700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f84441095d0 0x7f84441a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:07:03.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:03.598+0000 7f844b600700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8444113730 con 0x7f844406c5d0 2024-05-21T22:07:03.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:03.598+0000 7f844b600700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f84441135b0 con 0x7f84441095d0 2024-05-21T22:07:03.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:03.598+0000 7f844b600700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f84441138b0 con 0x7f844406bc10 2024-05-21T22:07:03.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:03.598+0000 7f8443e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f844406c5d0 0x7f844419c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:07:03.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:03.598+0000 7f8443e00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f844406c5d0 0x7f844419c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:42062/0 (socket says 172.21.15.151:42062) 2024-05-21T22:07:03.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:03.599+0000 7f8443e00700 1 -- 172.21.15.151:0/1513262723 learned_addr learned my addr 172.21.15.151:0/1513262723 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:07:03.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:03.599+0000 7f8448c00700 1 --2- 172.21.15.151:0/1513262723 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f844406bc10 0x7f844419be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:07:03.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:03.599+0000 7f8449600700 1 --2- 172.21.15.151:0/1513262723 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f84441095d0 0x7f84441a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:07:03.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:03.599+0000 7f8443e00700 1 -- 172.21.15.151:0/1513262723 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f844406bc10 msgr2=0x7f844419be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:07:03.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:03.599+0000 7f8443e00700 1 --2- 172.21.15.151:0/1513262723 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f844406bc10 0x7f844419be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:07:03.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:03.599+0000 7f8443e00700 1 -- 172.21.15.151:0/1513262723 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f84441095d0 msgr2=0x7f84441a06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:07:03.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:03.599+0000 7f8443e00700 1 --2- 172.21.15.151:0/1513262723 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f84441095d0 0x7f84441a06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:07:03.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:03.599+0000 7f8443e00700 1 -- 172.21.15.151:0/1513262723 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84441a0e30 con 0x7f844406c5d0 2024-05-21T22:07:03.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:03.599+0000 7f8449600700 1 --2- 172.21.15.151:0/1513262723 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f84441095d0 0x7f84441a06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:07:03.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:03.599+0000 7f8443e00700 1 --2- 172.21.15.151:0/1513262723 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f844406c5d0 0x7f844419c360 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f843000ec10 tx=0x7f843000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:07:03.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:03.600+0000 7f8441600700 1 -- 172.21.15.151:0/1513262723 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f843000cc90 con 0x7f844406c5d0 2024-05-21T22:07:03.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:03.600+0000 7f8441600700 1 -- 172.21.15.151:0/1513262723 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f843000ce30 con 0x7f844406c5d0 2024-05-21T22:07:03.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:03.600+0000 7f844b600700 1 -- 172.21.15.151:0/1513262723 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84441a1120 con 0x7f844406c5d0 2024-05-21T22:07:03.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:03.600+0000 7f8441600700 1 -- 172.21.15.151:0/1513262723 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8430010430 con 0x7f844406c5d0 2024-05-21T22:07:03.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:03.600+0000 7f844b600700 1 -- 172.21.15.151:0/1513262723 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84441a89a0 con 0x7f844406c5d0 2024-05-21T22:07:03.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:03.601+0000 7f844b600700 1 -- 172.21.15.151:0/1513262723 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8444104c90 con 0x7f844406c5d0 2024-05-21T22:07:03.604 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:03.604+0000 7f8441600700 1 -- 172.21.15.151:0/1513262723 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7f8430004000 con 0x7f844406c5d0 2024-05-21T22:07:03.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:03.605+0000 7f8441600700 1 --2- 172.21.15.151:0/1513262723 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f842c0b29f0 0x7f842c0b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:07:03.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:03.605+0000 7f8441600700 1 -- 172.21.15.151:0/1513262723 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7f84300e2a00 con 0x7f844406c5d0 2024-05-21T22:07:03.606 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:03.605+0000 7f8448c00700 1 --2- 172.21.15.151:0/1513262723 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f842c0b29f0 0x7f842c0b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:07:03.606 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:03.606+0000 7f8448c00700 1 --2- 172.21.15.151:0/1513262723 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f842c0b29f0 0x7f842c0b4eb0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8438009be0 tx=0x7f8438009260 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:07:03.608 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:03.608+0000 7f8441600700 1 -- 172.21.15.151:0/1513262723 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f84300aa370 con 0x7f844406c5d0 2024-05-21T22:07:03.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:03.893+0000 7f844b600700 1 -- 172.21.15.151:0/1513262723 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f844406a230 con 0x7f844406c5d0 2024-05-21T22:07:03.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:03.898+0000 7f8441600700 1 -- 172.21.15.151:0/1513262723 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v30) ==== 161+0+0 (secure 0 0 0) 0x7f844406a230 con 0x7f844406c5d0 2024-05-21T22:07:03.903 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:03.902+0000 7f844b600700 1 -- 172.21.15.151:0/1513262723 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f842c0b29f0 msgr2=0x7f842c0b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:07:03.903 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:03.902+0000 7f844b600700 1 --2- 172.21.15.151:0/1513262723 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f842c0b29f0 0x7f842c0b4eb0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8438009be0 tx=0x7f8438009260 comp rx=0 tx=0).stop 2024-05-21T22:07:03.903 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:03.902+0000 7f844b600700 1 -- 172.21.15.151:0/1513262723 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f844406c5d0 msgr2=0x7f844419c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:07:03.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:03.902+0000 7f844b600700 1 --2- 172.21.15.151:0/1513262723 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f844406c5d0 0x7f844419c360 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f843000ec10 tx=0x7f843000ef20 comp rx=0 tx=0).stop 2024-05-21T22:07:03.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:03.903+0000 7f844b600700 1 -- 172.21.15.151:0/1513262723 shutdown_connections 2024-05-21T22:07:03.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:03.903+0000 7f844b600700 1 --2- 172.21.15.151:0/1513262723 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f842c0b29f0 0x7f842c0b4eb0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:07:03.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:03.903+0000 7f844b600700 1 --2- 172.21.15.151:0/1513262723 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f84441095d0 0x7f84441a06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:07:03.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:03.903+0000 7f844b600700 1 --2- 172.21.15.151:0/1513262723 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f844406bc10 0x7f844419be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:07:03.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:03.903+0000 7f844b600700 1 --2- 172.21.15.151:0/1513262723 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f844406c5d0 0x7f844419c360 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:07:03.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:03.903+0000 7f844b600700 1 -- 172.21.15.151:0/1513262723 >> 172.21.15.151:0/1513262723 conn(0x7f84440793f0 msgr2=0x7f844410e720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:07:03.907 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:03.907+0000 7f844b600700 1 -- 172.21.15.151:0/1513262723 shutdown_connections 2024-05-21T22:07:03.907 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:03.907+0000 7f844b600700 1 -- 172.21.15.151:0/1513262723 wait complete. 2024-05-21T22:07:03.910 INFO:teuthology.orchestra.run.smithi151.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-21T22:07:04.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:07:03 smithi164 ceph-mon[36269]: pgmap v354: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 30 op/s 2024-05-21T22:07:04.036 INFO:journalctl@ceph.mds.b.smithi164.stdout:May 21 22:07:03 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-b[66241]: ignoring --setuser ceph since I am not root 2024-05-21T22:07:04.036 INFO:journalctl@ceph.mds.b.smithi164.stdout:May 21 22:07:03 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-b[66241]: ignoring --setgroup ceph since I am not root 2024-05-21T22:07:04.036 INFO:journalctl@ceph.mds.b.smithi164.stdout:May 21 22:07:03 smithi164 ceph-mds[66245]: -- 172.21.15.164:0/3028743441 <== mon.2 v2:172.21.15.164:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56470e9e7a40 con 0x56470e9ba800 2024-05-21T22:07:04.036 INFO:journalctl@ceph.mds.b.smithi164.stdout:May 21 22:07:03 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-b[66241]: starting mds.b at 2024-05-21T22:07:04.037 INFO:journalctl@ceph.mds.e.smithi164.stdout:May 21 22:07:03 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-e[65623]: ignoring --setuser ceph since I am not root 2024-05-21T22:07:04.037 INFO:journalctl@ceph.mds.e.smithi164.stdout:May 21 22:07:03 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-e[65623]: ignoring --setgroup ceph since I am not root 2024-05-21T22:07:04.037 INFO:journalctl@ceph.mds.e.smithi164.stdout:May 21 22:07:03 smithi164 ceph-mds[65628]: -- 172.21.15.164:0/3902512155 <== mon.2 v2:172.21.15.164:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56206b38da40 con 0x56206b360c00 2024-05-21T22:07:04.037 INFO:journalctl@ceph.mds.e.smithi164.stdout:May 21 22:07:03 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-e[65623]: starting mds.e at 2024-05-21T22:07:04.037 INFO:journalctl@ceph.mds.k.smithi164.stdout:May 21 22:07:03 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-k[65312]: ignoring --setuser ceph since I am not root 2024-05-21T22:07:04.037 INFO:journalctl@ceph.mds.k.smithi164.stdout:May 21 22:07:03 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-k[65312]: ignoring --setgroup ceph since I am not root 2024-05-21T22:07:04.038 INFO:journalctl@ceph.mds.k.smithi164.stdout:May 21 22:07:03 smithi164 ceph-mds[65316]: -- 172.21.15.164:0/3753533293 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55cfd0faa4e0 con 0x55cfd0f80800 2024-05-21T22:07:04.038 INFO:journalctl@ceph.mds.k.smithi164.stdout:May 21 22:07:03 smithi164 ceph-mds[65316]: -- 172.21.15.164:0/3753533293 <== mon.2 v2:172.21.15.164:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cfd0fada40 con 0x55cfd0f80800 2024-05-21T22:07:04.038 INFO:journalctl@ceph.mds.k.smithi164.stdout:May 21 22:07:03 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-k[65312]: starting mds.k at 2024-05-21T22:07:04.038 INFO:journalctl@ceph.mds.h.smithi164.stdout:May 21 22:07:03 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-h[65939]: ignoring --setuser ceph since I am not root 2024-05-21T22:07:04.038 INFO:journalctl@ceph.mds.h.smithi164.stdout:May 21 22:07:03 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-h[65939]: ignoring --setgroup ceph since I am not root 2024-05-21T22:07:04.038 INFO:journalctl@ceph.mds.h.smithi164.stdout:May 21 22:07:03 smithi164 ceph-mds[65943]: -- 172.21.15.164:0/3547786646 <== mon.2 v2:172.21.15.164:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e154295a40 con 0x55e154268c00 2024-05-21T22:07:04.038 INFO:journalctl@ceph.mds.h.smithi164.stdout:May 21 22:07:03 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-h[65939]: starting mds.h at 2024-05-21T22:07:04.204 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:07:03 smithi151 ceph-mon[31623]: pgmap v354: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 30 op/s 2024-05-21T22:07:04.204 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:07:03 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mon-a[31619]: 2024-05-21T22:07:03.894+0000 7fb05c200700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-21T22:07:04.204 INFO:journalctl@ceph.mds.a.smithi151.stdout:May 21 22:07:03 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-a[71364]: ignoring --setuser ceph since I am not root 2024-05-21T22:07:04.204 INFO:journalctl@ceph.mds.a.smithi151.stdout:May 21 22:07:03 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-a[71364]: ignoring --setgroup ceph since I am not root 2024-05-21T22:07:04.204 INFO:journalctl@ceph.mds.a.smithi151.stdout:May 21 22:07:03 smithi151 ceph-mds[71371]: -- 172.21.15.151:0/1550081698 <== mon.1 v2:172.21.15.184:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55caa6b85a40 con 0x55caa6b58800 2024-05-21T22:07:04.205 INFO:journalctl@ceph.mds.a.smithi151.stdout:May 21 22:07:03 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-a[71364]: starting mds.a at 2024-05-21T22:07:04.205 INFO:journalctl@ceph.mds.d.smithi151.stdout:May 21 22:07:03 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-d[70133]: ignoring --setuser ceph since I am not root 2024-05-21T22:07:04.205 INFO:journalctl@ceph.mds.d.smithi151.stdout:May 21 22:07:03 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-d[70133]: ignoring --setgroup ceph since I am not root 2024-05-21T22:07:04.205 INFO:journalctl@ceph.mds.d.smithi151.stdout:May 21 22:07:03 smithi151 ceph-mds[70143]: -- 172.21.15.151:0/852389203 <== mon.0 v2:172.21.15.151:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56033e469a40 con 0x56033e43c800 2024-05-21T22:07:04.205 INFO:journalctl@ceph.mds.d.smithi151.stdout:May 21 22:07:03 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-d[70133]: starting mds.d at 2024-05-21T22:07:04.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:07:03 smithi184 ceph-mon[36942]: pgmap v354: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 30 op/s 2024-05-21T22:07:04.236 INFO:journalctl@ceph.mds.f.smithi184.stdout:May 21 22:07:03 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-f[67208]: ignoring --setuser ceph since I am not root 2024-05-21T22:07:04.236 INFO:journalctl@ceph.mds.f.smithi184.stdout:May 21 22:07:03 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-f[67208]: ignoring --setgroup ceph since I am not root 2024-05-21T22:07:04.236 INFO:journalctl@ceph.mds.f.smithi184.stdout:May 21 22:07:03 smithi184 ceph-mds[67212]: -- 172.21.15.184:0/2577400327 <== mon.1 v2:172.21.15.184:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560fee85da40 con 0x560fee831000 2024-05-21T22:07:04.236 INFO:journalctl@ceph.mds.f.smithi184.stdout:May 21 22:07:03 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-f[67208]: starting mds.f at 2024-05-21T22:07:04.237 INFO:journalctl@ceph.mds.i.smithi184.stdout:May 21 22:07:03 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-i[67811]: ignoring --setuser ceph since I am not root 2024-05-21T22:07:04.237 INFO:journalctl@ceph.mds.i.smithi184.stdout:May 21 22:07:03 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-i[67811]: ignoring --setgroup ceph since I am not root 2024-05-21T22:07:04.237 INFO:journalctl@ceph.mds.i.smithi184.stdout:May 21 22:07:03 smithi184 ceph-mds[67815]: -- 172.21.15.184:0/4083947141 <== mon.1 v2:172.21.15.184:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555862d99a40 con 0x555862d6d000 2024-05-21T22:07:04.237 INFO:journalctl@ceph.mds.i.smithi184.stdout:May 21 22:07:03 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-i[67811]: starting mds.i at 2024-05-21T22:07:04.237 INFO:journalctl@ceph.mds.l.smithi184.stdout:May 21 22:07:03 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-l[66907]: ignoring --setuser ceph since I am not root 2024-05-21T22:07:04.238 INFO:journalctl@ceph.mds.l.smithi184.stdout:May 21 22:07:03 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-l[66907]: ignoring --setgroup ceph since I am not root 2024-05-21T22:07:04.238 INFO:journalctl@ceph.mds.l.smithi184.stdout:May 21 22:07:03 smithi184 ceph-mds[66911]: -- 172.21.15.184:0/797448351 <== mon.2 v2:172.21.15.164:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5639f087ba40 con 0x5639f084f000 2024-05-21T22:07:04.238 INFO:journalctl@ceph.mds.l.smithi184.stdout:May 21 22:07:03 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-l[66907]: starting mds.l at 2024-05-21T22:07:04.238 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 21 22:07:03 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-c[67511]: ignoring --setuser ceph since I am not root 2024-05-21T22:07:04.238 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 21 22:07:03 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-c[67511]: ignoring --setgroup ceph since I am not root 2024-05-21T22:07:04.238 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 21 22:07:03 smithi184 ceph-mds[67515]: -- 172.21.15.184:0/3837575420 <== mon.1 v2:172.21.15.184:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f3800f3a40 con 0x55f3800c7000 2024-05-21T22:07:04.238 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 21 22:07:03 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-c[67511]: starting mds.c at 2024-05-21T22:07:04.762 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-05-21T22:07:04.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:07:04 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/1513262723' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-21T22:07:04.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:07:04 smithi184 ceph-mon[36942]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-21T22:07:04.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:07:04 smithi184 ceph-mon[36942]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-21T22:07:04.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:07:04 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/1513262723' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-21T22:07:04.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:07:04 smithi184 ceph-mon[36942]: osdmap e92: 12 total, 12 up, 12 in 2024-05-21T22:07:04.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:07:04 smithi184 ceph-mon[36942]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-05-21T22:07:05.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:07:04 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/1513262723' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-21T22:07:05.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:07:04 smithi164 ceph-mon[36269]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-21T22:07:05.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:07:04 smithi164 ceph-mon[36269]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-21T22:07:05.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:07:04 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/1513262723' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-21T22:07:05.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:07:04 smithi164 ceph-mon[36269]: osdmap e92: 12 total, 12 up, 12 in 2024-05-21T22:07:05.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:07:04 smithi164 ceph-mon[36269]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-05-21T22:07:05.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:07:04 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/1513262723' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-21T22:07:05.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:07:04 smithi151 ceph-mon[31623]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-21T22:07:05.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:07:04 smithi151 ceph-mon[31623]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-21T22:07:05.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:07:04 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/1513262723' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-21T22:07:05.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:07:04 smithi151 ceph-mon[31623]: osdmap e92: 12 total, 12 up, 12 in 2024-05-21T22:07:05.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:07:04 smithi151 ceph-mon[31623]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-05-21T22:07:05.218 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:07:05.954 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:07:05 smithi151 ceph-mon[31623]: pgmap v356: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-05-21T22:07:05.954 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:07:05 smithi151 ceph-mon[31623]: mds.? [v2:172.21.15.184:6832/1460379739,v1:172.21.15.184:6834/1460379739] up:boot 2024-05-21T22:07:05.954 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:07:05 smithi151 ceph-mon[31623]: mds.? [v2:172.21.15.164:6832/1929885476,v1:172.21.15.164:6835/1929885476] up:boot 2024-05-21T22:07:05.957 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:07:05 smithi151 ceph-mon[31623]: mds.? [v2:172.21.15.184:6833/422539861,v1:172.21.15.184:6835/422539861] up:boot 2024-05-21T22:07:05.957 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:07:05 smithi151 ceph-mon[31623]: mds.? [v2:172.21.15.164:6833/9500693,v1:172.21.15.164:6834/9500693] up:boot 2024-05-21T22:07:05.957 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:07:05 smithi151 ceph-mon[31623]: mds.? [v2:172.21.15.164:6836/1375208019,v1:172.21.15.164:6838/1375208019] up:boot 2024-05-21T22:07:05.957 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:07:05 smithi151 ceph-mon[31623]: mds.? [v2:172.21.15.164:6837/3145564146,v1:172.21.15.164:6839/3145564146] up:boot 2024-05-21T22:07:05.957 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:07:05 smithi151 ceph-mon[31623]: mds.? [v2:172.21.15.184:6836/1968872666,v1:172.21.15.184:6838/1968872666] up:boot 2024-05-21T22:07:05.957 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:07:05 smithi151 ceph-mon[31623]: mds.? [v2:172.21.15.184:6837/3504687694,v1:172.21.15.184:6839/3504687694] up:boot 2024-05-21T22:07:05.958 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:07:05 smithi151 ceph-mon[31623]: mds.? [v2:172.21.15.151:6836/3342938277,v1:172.21.15.151:6837/3342938277] up:boot 2024-05-21T22:07:05.958 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:07:05 smithi151 ceph-mon[31623]: mds.? [v2:172.21.15.151:6840/1267779799,v1:172.21.15.151:6841/1267779799] up:boot 2024-05-21T22:07:05.958 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:07:05 smithi151 ceph-mon[31623]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-05-21T22:07:05.958 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:07:05 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-21T22:07:05.958 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:07:05 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-21T22:07:05.958 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:07:05 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-21T22:07:05.958 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:07:05 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-21T22:07:05.958 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:07:05 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-21T22:07:05.958 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:07:05 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-21T22:07:05.958 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:07:05 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-21T22:07:05.958 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:07:05 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-21T22:07:05.959 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:07:05 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-21T22:07:05.959 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:07:05 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-21T22:07:06.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:07:05 smithi164 ceph-mon[36269]: pgmap v356: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-05-21T22:07:06.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:07:05 smithi164 ceph-mon[36269]: mds.? [v2:172.21.15.184:6832/1460379739,v1:172.21.15.184:6834/1460379739] up:boot 2024-05-21T22:07:06.037 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:07:05 smithi164 ceph-mon[36269]: mds.? [v2:172.21.15.164:6832/1929885476,v1:172.21.15.164:6835/1929885476] up:boot 2024-05-21T22:07:06.037 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:07:05 smithi164 ceph-mon[36269]: mds.? [v2:172.21.15.184:6833/422539861,v1:172.21.15.184:6835/422539861] up:boot 2024-05-21T22:07:06.037 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:07:05 smithi164 ceph-mon[36269]: mds.? [v2:172.21.15.164:6833/9500693,v1:172.21.15.164:6834/9500693] up:boot 2024-05-21T22:07:06.037 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:07:05 smithi164 ceph-mon[36269]: mds.? [v2:172.21.15.164:6836/1375208019,v1:172.21.15.164:6838/1375208019] up:boot 2024-05-21T22:07:06.037 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:07:05 smithi164 ceph-mon[36269]: mds.? [v2:172.21.15.164:6837/3145564146,v1:172.21.15.164:6839/3145564146] up:boot 2024-05-21T22:07:06.037 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:07:05 smithi164 ceph-mon[36269]: mds.? [v2:172.21.15.184:6836/1968872666,v1:172.21.15.184:6838/1968872666] up:boot 2024-05-21T22:07:06.037 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:07:05 smithi164 ceph-mon[36269]: mds.? [v2:172.21.15.184:6837/3504687694,v1:172.21.15.184:6839/3504687694] up:boot 2024-05-21T22:07:06.037 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:07:05 smithi164 ceph-mon[36269]: mds.? [v2:172.21.15.151:6836/3342938277,v1:172.21.15.151:6837/3342938277] up:boot 2024-05-21T22:07:06.037 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:07:05 smithi164 ceph-mon[36269]: mds.? [v2:172.21.15.151:6840/1267779799,v1:172.21.15.151:6841/1267779799] up:boot 2024-05-21T22:07:06.037 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:07:05 smithi164 ceph-mon[36269]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-05-21T22:07:06.037 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:07:05 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-21T22:07:06.037 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:07:05 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-21T22:07:06.037 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:07:05 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-21T22:07:06.038 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:07:05 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-21T22:07:06.038 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:07:05 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-21T22:07:06.038 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:07:05 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-21T22:07:06.038 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:07:05 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-21T22:07:06.038 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:07:05 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-21T22:07:06.038 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:07:05 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-21T22:07:06.038 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:07:05 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-21T22:07:06.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:07:05 smithi184 ceph-mon[36942]: pgmap v356: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-05-21T22:07:06.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:07:05 smithi184 ceph-mon[36942]: mds.? [v2:172.21.15.184:6832/1460379739,v1:172.21.15.184:6834/1460379739] up:boot 2024-05-21T22:07:06.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:07:05 smithi184 ceph-mon[36942]: mds.? [v2:172.21.15.164:6832/1929885476,v1:172.21.15.164:6835/1929885476] up:boot 2024-05-21T22:07:06.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:07:05 smithi184 ceph-mon[36942]: mds.? [v2:172.21.15.184:6833/422539861,v1:172.21.15.184:6835/422539861] up:boot 2024-05-21T22:07:06.237 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:07:05 smithi184 ceph-mon[36942]: mds.? [v2:172.21.15.164:6833/9500693,v1:172.21.15.164:6834/9500693] up:boot 2024-05-21T22:07:06.237 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:07:05 smithi184 ceph-mon[36942]: mds.? [v2:172.21.15.164:6836/1375208019,v1:172.21.15.164:6838/1375208019] up:boot 2024-05-21T22:07:06.237 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:07:05 smithi184 ceph-mon[36942]: mds.? [v2:172.21.15.164:6837/3145564146,v1:172.21.15.164:6839/3145564146] up:boot 2024-05-21T22:07:06.237 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:07:05 smithi184 ceph-mon[36942]: mds.? [v2:172.21.15.184:6836/1968872666,v1:172.21.15.184:6838/1968872666] up:boot 2024-05-21T22:07:06.237 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:07:05 smithi184 ceph-mon[36942]: mds.? [v2:172.21.15.184:6837/3504687694,v1:172.21.15.184:6839/3504687694] up:boot 2024-05-21T22:07:06.237 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:07:05 smithi184 ceph-mon[36942]: mds.? [v2:172.21.15.151:6836/3342938277,v1:172.21.15.151:6837/3342938277] up:boot 2024-05-21T22:07:06.237 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:07:05 smithi184 ceph-mon[36942]: mds.? [v2:172.21.15.151:6840/1267779799,v1:172.21.15.151:6841/1267779799] up:boot 2024-05-21T22:07:06.237 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:07:05 smithi184 ceph-mon[36942]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-05-21T22:07:06.237 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:07:05 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-21T22:07:06.237 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:07:05 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-21T22:07:06.237 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:07:05 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-21T22:07:06.237 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:07:05 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-21T22:07:06.237 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:07:05 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-21T22:07:06.238 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:07:05 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-21T22:07:06.238 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:07:05 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-21T22:07:06.238 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:07:05 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-21T22:07:06.238 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:07:05 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-21T22:07:06.238 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:07:05 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-21T22:07:07.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:07.471+0000 7f5195000700 1 -- 172.21.15.151:0/3427686243 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5190101b30 msgr2=0x7f5190101fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:07:07.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:07.471+0000 7f5195000700 1 --2- 172.21.15.151:0/3427686243 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5190101b30 0x7f5190101fb0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f517c0099c0 tx=0x7f517c009cd0 comp rx=0 tx=0).stop 2024-05-21T22:07:07.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:07.472+0000 7f5195000700 1 -- 172.21.15.151:0/3427686243 shutdown_connections 2024-05-21T22:07:07.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:07.472+0000 7f5195000700 1 --2- 172.21.15.151:0/3427686243 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f51901024f0 0x7f51901103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:07:07.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:07.472+0000 7f5195000700 1 --2- 172.21.15.151:0/3427686243 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5190109520 0x7f5190109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:07:07.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:07.472+0000 7f5195000700 1 --2- 172.21.15.151:0/3427686243 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5190101b30 0x7f5190101fb0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:07:07.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:07.472+0000 7f5195000700 1 -- 172.21.15.151:0/3427686243 >> 172.21.15.151:0/3427686243 conn(0x7f5190078ab0 msgr2=0x7f51900ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:07:07.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:07.472+0000 7f5195000700 1 -- 172.21.15.151:0/3427686243 shutdown_connections 2024-05-21T22:07:07.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:07.472+0000 7f5195000700 1 -- 172.21.15.151:0/3427686243 wait complete. 2024-05-21T22:07:07.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:07.473+0000 7f5195000700 1 Processor -- start 2024-05-21T22:07:07.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:07.473+0000 7f5195000700 1 -- start start 2024-05-21T22:07:07.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:07.473+0000 7f5195000700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5190101b30 0x7f519019be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:07:07.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:07.473+0000 7f5195000700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f51901024f0 0x7f519019c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:07:07.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:07.474+0000 7f5195000700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5190109520 0x7f51901a0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:07:07.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:07.474+0000 7f5195000700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5190113700 con 0x7f51901024f0 2024-05-21T22:07:07.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:07.474+0000 7f5195000700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5190113580 con 0x7f5190109520 2024-05-21T22:07:07.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:07.474+0000 7f5195000700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f5190113880 con 0x7f5190101b30 2024-05-21T22:07:07.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:07.474+0000 7f518d400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f51901024f0 0x7f519019c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:07:07.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:07.474+0000 7f518d400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f51901024f0 0x7f519019c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:53518/0 (socket says 172.21.15.151:53518) 2024-05-21T22:07:07.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:07.474+0000 7f518d400700 1 -- 172.21.15.151:0/3585088016 learned_addr learned my addr 172.21.15.151:0/3585088016 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:07:07.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:07.474+0000 7f518de00700 1 --2- 172.21.15.151:0/3585088016 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5190101b30 0x7f519019be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:07:07.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:07.474+0000 7f518e800700 1 --2- 172.21.15.151:0/3585088016 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5190109520 0x7f51901a0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:07:07.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:07.474+0000 7f518de00700 1 -- 172.21.15.151:0/3585088016 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5190109520 msgr2=0x7f51901a0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:07:07.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:07.474+0000 7f518de00700 1 --2- 172.21.15.151:0/3585088016 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5190109520 0x7f51901a0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:07:07.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:07.474+0000 7f518de00700 1 -- 172.21.15.151:0/3585088016 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f51901024f0 msgr2=0x7f519019c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:07:07.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:07.474+0000 7f518de00700 1 --2- 172.21.15.151:0/3585088016 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f51901024f0 0x7f519019c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:07:07.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:07.474+0000 7f518de00700 1 -- 172.21.15.151:0/3585088016 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51901a0e80 con 0x7f5190101b30 2024-05-21T22:07:07.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:07.474+0000 7f518d400700 1 --2- 172.21.15.151:0/3585088016 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f51901024f0 0x7f519019c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T22:07:07.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:07.475+0000 7f518de00700 1 --2- 172.21.15.151:0/3585088016 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5190101b30 0x7f519019be70 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f518000e9f0 tx=0x7f518000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:07:07.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:07.475+0000 7f5186a00700 1 -- 172.21.15.151:0/3585088016 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f518000c9c0 con 0x7f5190101b30 2024-05-21T22:07:07.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:07.475+0000 7f5195000700 1 -- 172.21.15.151:0/3585088016 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51901a1170 con 0x7f5190101b30 2024-05-21T22:07:07.477 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:07.475+0000 7f5186a00700 1 -- 172.21.15.151:0/3585088016 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f518000cb60 con 0x7f5190101b30 2024-05-21T22:07:07.477 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:07.475+0000 7f5186a00700 1 -- 172.21.15.151:0/3585088016 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5180018680 con 0x7f5190101b30 2024-05-21T22:07:07.477 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:07.475+0000 7f5195000700 1 -- 172.21.15.151:0/3585088016 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51901a89f0 con 0x7f5190101b30 2024-05-21T22:07:07.477 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:07.476+0000 7f5195000700 1 -- 172.21.15.151:0/3585088016 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5190104e50 con 0x7f5190101b30 2024-05-21T22:07:07.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:07.479+0000 7f5186a00700 1 -- 172.21.15.151:0/3585088016 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7f5180010430 con 0x7f5190101b30 2024-05-21T22:07:07.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:07.480+0000 7f5186a00700 1 --2- 172.21.15.151:0/3585088016 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f51740b29f0 0x7f51740b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:07:07.481 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:07.480+0000 7f5186a00700 1 -- 172.21.15.151:0/3585088016 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8928+0+0 (secure 0 0 0) 0x7f51800e2c50 con 0x7f5190101b30 2024-05-21T22:07:07.481 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:07.480+0000 7f518d400700 1 --2- 172.21.15.151:0/3585088016 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f51740b29f0 0x7f51740b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:07:07.481 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:07.481+0000 7f518d400700 1 --2- 172.21.15.151:0/3585088016 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f51740b29f0 0x7f51740b4eb0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5190104d30 tx=0x7f517c005960 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:07:07.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:07.482+0000 7f5186a00700 1 -- 172.21.15.151:0/3585088016 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f51800aa460 con 0x7f5190101b30 2024-05-21T22:07:07.766 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:07.766+0000 7f5195000700 1 -- 172.21.15.151:0/3585088016 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f51901a8bc0 con 0x7f5190101b30 2024-05-21T22:07:07.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:07.781+0000 7f5186a00700 1 -- 172.21.15.151:0/3585088016 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v32) ==== 104+0+0 (secure 0 0 0) 0x7f51800a9d60 con 0x7f5190101b30 2024-05-21T22:07:07.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:07.785+0000 7f5195000700 1 -- 172.21.15.151:0/3585088016 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f51740b29f0 msgr2=0x7f51740b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:07:07.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:07.785+0000 7f5195000700 1 --2- 172.21.15.151:0/3585088016 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f51740b29f0 0x7f51740b4eb0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5190104d30 tx=0x7f517c005960 comp rx=0 tx=0).stop 2024-05-21T22:07:07.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:07.785+0000 7f5195000700 1 -- 172.21.15.151:0/3585088016 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5190101b30 msgr2=0x7f519019be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:07:07.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:07.785+0000 7f5195000700 1 --2- 172.21.15.151:0/3585088016 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5190101b30 0x7f519019be70 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f518000e9f0 tx=0x7f518000ed00 comp rx=0 tx=0).stop 2024-05-21T22:07:07.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:07.785+0000 7f5195000700 1 -- 172.21.15.151:0/3585088016 shutdown_connections 2024-05-21T22:07:07.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:07.785+0000 7f5195000700 1 --2- 172.21.15.151:0/3585088016 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f51740b29f0 0x7f51740b4eb0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:07:07.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:07.785+0000 7f5195000700 1 --2- 172.21.15.151:0/3585088016 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5190109520 0x7f51901a0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:07:07.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:07.785+0000 7f5195000700 1 --2- 172.21.15.151:0/3585088016 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5190101b30 0x7f519019be70 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:07:07.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:07.785+0000 7f5195000700 1 --2- 172.21.15.151:0/3585088016 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f51901024f0 0x7f519019c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:07:07.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:07.785+0000 7f5195000700 1 -- 172.21.15.151:0/3585088016 >> 172.21.15.151:0/3585088016 conn(0x7f5190078ab0 msgr2=0x7f51900ffff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:07:07.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:07.785+0000 7f5195000700 1 -- 172.21.15.151:0/3585088016 shutdown_connections 2024-05-21T22:07:07.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:07.785+0000 7f5195000700 1 -- 172.21.15.151:0/3585088016 wait complete. 2024-05-21T22:07:08.099 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:07:07 smithi151 ceph-mon[31623]: pgmap v357: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-05-21T22:07:08.099 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:07:07 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/3585088016' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-21T22:07:08.099 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:07:07 smithi151 ceph-mon[31623]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-21T22:07:08.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:07:07 smithi184 ceph-mon[36942]: pgmap v357: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-05-21T22:07:08.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:07:07 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/3585088016' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-21T22:07:08.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:07:07 smithi184 ceph-mon[36942]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-21T22:07:08.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:07:07 smithi164 ceph-mon[36269]: pgmap v357: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-05-21T22:07:08.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:07:07 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/3585088016' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-21T22:07:08.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:07:07 smithi164 ceph-mon[36269]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-21T22:07:08.605 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-21T22:07:09.085 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:07:09.187 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:07:08 smithi164 ceph-mon[36269]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-21T22:07:09.187 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:07:08 smithi164 ceph-mon[36269]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-21T22:07:09.188 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:07:08 smithi164 ceph-mon[36269]: Cluster is now healthy 2024-05-21T22:07:09.188 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:07:08 smithi164 ceph-mon[36269]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-21T22:07:09.188 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:07:08 smithi164 ceph-mon[36269]: fsmap 12 up:standby 2024-05-21T22:07:09.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:07:08 smithi151 ceph-mon[31623]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-21T22:07:09.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:07:08 smithi151 ceph-mon[31623]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-21T22:07:09.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:07:08 smithi151 ceph-mon[31623]: Cluster is now healthy 2024-05-21T22:07:09.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:07:08 smithi151 ceph-mon[31623]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-21T22:07:09.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:07:08 smithi151 ceph-mon[31623]: fsmap 12 up:standby 2024-05-21T22:07:09.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:07:08 smithi184 ceph-mon[36942]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-21T22:07:09.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:07:08 smithi184 ceph-mon[36942]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-21T22:07:09.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:07:08 smithi184 ceph-mon[36942]: Cluster is now healthy 2024-05-21T22:07:09.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:07:08 smithi184 ceph-mon[36942]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-21T22:07:09.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:07:08 smithi184 ceph-mon[36942]: fsmap 12 up:standby 2024-05-21T22:07:10.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:07:09 smithi151 ceph-mon[31623]: pgmap v358: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-05-21T22:07:10.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:07:09 smithi184 ceph-mon[36942]: pgmap v358: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-05-21T22:07:10.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:07:09 smithi164 ceph-mon[36269]: pgmap v358: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-05-21T22:07:11.135 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:11.135+0000 7fcbd2c00700 1 -- 172.21.15.151:0/1365337504 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcbcc10fe20 msgr2=0x7fcbcc112210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:07:11.136 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:11.135+0000 7fcbd2c00700 1 --2- 172.21.15.151:0/1365337504 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcbcc10fe20 0x7fcbcc112210 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fcbc4009f90 tx=0x7fcbc400b330 comp rx=0 tx=0).stop 2024-05-21T22:07:11.136 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:11.135+0000 7fcbd2c00700 1 -- 172.21.15.151:0/1365337504 shutdown_connections 2024-05-21T22:07:11.136 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:11.135+0000 7fcbd2c00700 1 --2- 172.21.15.151:0/1365337504 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcbcc10fe20 0x7fcbcc112210 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:07:11.136 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:11.135+0000 7fcbd2c00700 1 --2- 172.21.15.151:0/1365337504 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcbcc102270 0x7fcbcc102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:07:11.136 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:11.135+0000 7fcbd2c00700 1 --2- 172.21.15.151:0/1365337504 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcbcc102c20 0x7fcbcc10f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:07:11.136 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:11.135+0000 7fcbd2c00700 1 -- 172.21.15.151:0/1365337504 >> 172.21.15.151:0/1365337504 conn(0x7fcbcc0fded0 msgr2=0x7fcbcc1002f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:07:11.136 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:11.135+0000 7fcbd2c00700 1 -- 172.21.15.151:0/1365337504 shutdown_connections 2024-05-21T22:07:11.136 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:11.135+0000 7fcbd2c00700 1 -- 172.21.15.151:0/1365337504 wait complete. 2024-05-21T22:07:11.138 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:11.136+0000 7fcbd2c00700 1 Processor -- start 2024-05-21T22:07:11.138 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:11.136+0000 7fcbd2c00700 1 -- start start 2024-05-21T22:07:11.138 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:11.137+0000 7fcbd2c00700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcbcc102270 0x7fcbcc19bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:07:11.138 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:11.137+0000 7fcbd2c00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcbcc102c20 0x7fcbcc19c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:07:11.138 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:11.137+0000 7fcbd2c00700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcbcc10fe20 0x7fcbcc1a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:07:11.138 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:11.137+0000 7fcbd2c00700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcbcc1158a0 con 0x7fcbcc102270 2024-05-21T22:07:11.138 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:11.137+0000 7fcbd2c00700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcbcc115720 con 0x7fcbcc102c20 2024-05-21T22:07:11.138 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:11.137+0000 7fcbd2c00700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7fcbcc115a20 con 0x7fcbcc10fe20 2024-05-21T22:07:11.138 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:11.137+0000 7fcbd0e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcbcc102c20 0x7fcbcc19c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:07:11.138 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:11.137+0000 7fcbd0e00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcbcc102c20 0x7fcbcc19c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:56754/0 (socket says 172.21.15.151:56754) 2024-05-21T22:07:11.138 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:11.137+0000 7fcbd0e00700 1 -- 172.21.15.151:0/174484199 learned_addr learned my addr 172.21.15.151:0/174484199 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:07:11.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:11.137+0000 7fcbd1800700 1 --2- 172.21.15.151:0/174484199 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcbcc102270 0x7fcbcc19bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:07:11.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:11.137+0000 7fcbd2200700 1 --2- 172.21.15.151:0/174484199 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcbcc10fe20 0x7fcbcc1a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:07:11.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:11.137+0000 7fcbd0e00700 1 -- 172.21.15.151:0/174484199 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcbcc10fe20 msgr2=0x7fcbcc1a0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:07:11.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:11.137+0000 7fcbd0e00700 1 --2- 172.21.15.151:0/174484199 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcbcc10fe20 0x7fcbcc1a0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:07:11.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:11.137+0000 7fcbd0e00700 1 -- 172.21.15.151:0/174484199 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcbcc102270 msgr2=0x7fcbcc19bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:07:11.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:11.137+0000 7fcbd0e00700 1 --2- 172.21.15.151:0/174484199 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcbcc102270 0x7fcbcc19bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:07:11.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:11.137+0000 7fcbd0e00700 1 -- 172.21.15.151:0/174484199 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbcc1a0d80 con 0x7fcbcc102c20 2024-05-21T22:07:11.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:11.138+0000 7fcbd0e00700 1 --2- 172.21.15.151:0/174484199 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcbcc102c20 0x7fcbcc19c2b0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fcbb800ec10 tx=0x7fcbb800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:07:11.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:11.138+0000 7fcbca000700 1 -- 172.21.15.151:0/174484199 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb800cc90 con 0x7fcbcc102c20 2024-05-21T22:07:11.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:11.138+0000 7fcbca000700 1 -- 172.21.15.151:0/174484199 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcbb800ce30 con 0x7fcbcc102c20 2024-05-21T22:07:11.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:11.138+0000 7fcbd2c00700 1 -- 172.21.15.151:0/174484199 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbcc1a1070 con 0x7fcbcc102c20 2024-05-21T22:07:11.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:11.138+0000 7fcbca000700 1 -- 172.21.15.151:0/174484199 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb8010430 con 0x7fcbcc102c20 2024-05-21T22:07:11.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:11.138+0000 7fcbd2c00700 1 -- 172.21.15.151:0/174484199 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbcc1a88f0 con 0x7fcbcc102c20 2024-05-21T22:07:11.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:11.139+0000 7fcbd2c00700 1 -- 172.21.15.151:0/174484199 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbcc06a230 con 0x7fcbcc102c20 2024-05-21T22:07:11.142 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:11.142+0000 7fcbca000700 1 -- 172.21.15.151:0/174484199 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7fcbb8004000 con 0x7fcbcc102c20 2024-05-21T22:07:11.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:11.143+0000 7fcbca000700 1 --2- 172.21.15.151:0/174484199 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fcbb40b29f0 0x7fcbb40b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:07:11.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:11.143+0000 7fcbca000700 1 -- 172.21.15.151:0/174484199 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8928+0+0 (secure 0 0 0) 0x7fcbb80e2b10 con 0x7fcbcc102c20 2024-05-21T22:07:11.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:11.144+0000 7fcbd1800700 1 --2- 172.21.15.151:0/174484199 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fcbb40b29f0 0x7fcbb40b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:07:11.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:11.144+0000 7fcbd1800700 1 --2- 172.21.15.151:0/174484199 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fcbb40b29f0 0x7fcbb40b4eb0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fcbc0009c60 tx=0x7fcbc0009260 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:07:11.145 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:11.145+0000 7fcbca000700 1 -- 172.21.15.151:0/174484199 <== mon.2 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcbb80aa320 con 0x7fcbcc102c20 2024-05-21T22:07:11.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:11.377+0000 7fcbd2c00700 1 -- 172.21.15.151:0/174484199 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fcbcc19d0d0 con 0x7fcbcc102c20 2024-05-21T22:07:11.826 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:11.823+0000 7fcbca000700 1 -- 172.21.15.151:0/174484199 <== mon.2 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v93) ==== 181+0+0 (secure 0 0 0) 0x7fcbb80a9c20 con 0x7fcbcc102c20 2024-05-21T22:07:11.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:11.831+0000 7fcbd2c00700 1 -- 172.21.15.151:0/174484199 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fcbb40b29f0 msgr2=0x7fcbb40b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:07:11.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:11.831+0000 7fcbd2c00700 1 --2- 172.21.15.151:0/174484199 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fcbb40b29f0 0x7fcbb40b4eb0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fcbc0009c60 tx=0x7fcbc0009260 comp rx=0 tx=0).stop 2024-05-21T22:07:11.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:11.831+0000 7fcbd2c00700 1 -- 172.21.15.151:0/174484199 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcbcc102c20 msgr2=0x7fcbcc19c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:07:11.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:11.831+0000 7fcbd2c00700 1 --2- 172.21.15.151:0/174484199 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcbcc102c20 0x7fcbcc19c2b0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fcbb800ec10 tx=0x7fcbb800ef20 comp rx=0 tx=0).stop 2024-05-21T22:07:11.833 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:11.832+0000 7fcbd2c00700 1 -- 172.21.15.151:0/174484199 shutdown_connections 2024-05-21T22:07:11.833 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:11.832+0000 7fcbd2c00700 1 --2- 172.21.15.151:0/174484199 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7fcbb40b29f0 0x7fcbb40b4eb0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:07:11.833 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:11.832+0000 7fcbd2c00700 1 --2- 172.21.15.151:0/174484199 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcbcc10fe20 0x7fcbcc1a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:07:11.833 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:11.832+0000 7fcbd2c00700 1 --2- 172.21.15.151:0/174484199 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcbcc102270 0x7fcbcc19bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:07:11.833 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:11.832+0000 7fcbd2c00700 1 --2- 172.21.15.151:0/174484199 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcbcc102c20 0x7fcbcc19c2b0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:07:11.833 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:11.832+0000 7fcbd2c00700 1 -- 172.21.15.151:0/174484199 >> 172.21.15.151:0/174484199 conn(0x7fcbcc0fded0 msgr2=0x7fcbcc1002c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:07:11.833 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:11.832+0000 7fcbd2c00700 1 -- 172.21.15.151:0/174484199 shutdown_connections 2024-05-21T22:07:11.833 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:11.832+0000 7fcbd2c00700 1 -- 172.21.15.151:0/174484199 wait complete. 2024-05-21T22:07:11.834 INFO:teuthology.orchestra.run.smithi151.stderr:pool 'cephfs_metadata' removed 2024-05-21T22:07:12.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:07:11 smithi151 ceph-mon[31623]: pgmap v359: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-05-21T22:07:12.203 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:07:11 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/174484199' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-21T22:07:12.204 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:07:11 smithi151 ceph-mon[31623]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-21T22:07:12.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:07:11 smithi184 ceph-mon[36942]: pgmap v359: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-05-21T22:07:12.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:07:11 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/174484199' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-21T22:07:12.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:07:11 smithi184 ceph-mon[36942]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-21T22:07:12.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:07:11 smithi164 ceph-mon[36269]: pgmap v359: 97 pgs: 97 active+clean; 259 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-05-21T22:07:12.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:07:11 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/174484199' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-21T22:07:12.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:07:11 smithi164 ceph-mon[36269]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-21T22:07:12.713 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995 shell --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-21T22:07:12.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:07:12 smithi151 ceph-mon[31623]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-21T22:07:12.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:07:12 smithi151 ceph-mon[31623]: osdmap e93: 12 total, 12 up, 12 in 2024-05-21T22:07:12.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:07:12 smithi151 ceph-mon[31623]: pgmap v361: 65 pgs: 65 active+clean; 258 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T22:07:13.163 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/mon.a/config 2024-05-21T22:07:13.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:07:12 smithi184 ceph-mon[36942]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-21T22:07:13.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:07:12 smithi184 ceph-mon[36942]: osdmap e93: 12 total, 12 up, 12 in 2024-05-21T22:07:13.236 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:07:12 smithi184 ceph-mon[36942]: pgmap v361: 65 pgs: 65 active+clean; 258 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T22:07:13.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:07:12 smithi164 ceph-mon[36269]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-21T22:07:13.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:07:12 smithi164 ceph-mon[36269]: osdmap e93: 12 total, 12 up, 12 in 2024-05-21T22:07:13.286 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:07:12 smithi164 ceph-mon[36269]: pgmap v361: 65 pgs: 65 active+clean; 258 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T22:07:15.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:15.324+0000 7f7d4c400700 1 -- 172.21.15.151:0/1066363131 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7d44075770 msgr2=0x7f7d44075b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:07:15.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:15.324+0000 7f7d4c400700 1 --2- 172.21.15.151:0/1066363131 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7d44075770 0x7f7d44075b50 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f7d380098f0 tx=0x7f7d38009c00 comp rx=0 tx=0).stop 2024-05-21T22:07:15.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:15.324+0000 7f7d4c400700 1 -- 172.21.15.151:0/1066363131 shutdown_connections 2024-05-21T22:07:15.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:15.324+0000 7f7d4c400700 1 --2- 172.21.15.151:0/1066363131 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d4410ff70 0x7f7d44112360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:07:15.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:15.324+0000 7f7d4c400700 1 --2- 172.21.15.151:0/1066363131 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7d44075770 0x7f7d44075b50 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:07:15.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:15.324+0000 7f7d4c400700 1 --2- 172.21.15.151:0/1066363131 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7d44076090 0x7f7d4410fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:07:15.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:15.324+0000 7f7d4c400700 1 -- 172.21.15.151:0/1066363131 >> 172.21.15.151:0/1066363131 conn(0x7f7d4407a6f0 msgr2=0x7f7d4407ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:07:15.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:15.325+0000 7f7d4c400700 1 -- 172.21.15.151:0/1066363131 shutdown_connections 2024-05-21T22:07:15.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:15.325+0000 7f7d4c400700 1 -- 172.21.15.151:0/1066363131 wait complete. 2024-05-21T22:07:15.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:15.326+0000 7f7d4c400700 1 Processor -- start 2024-05-21T22:07:15.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:15.326+0000 7f7d4c400700 1 -- start start 2024-05-21T22:07:15.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:15.326+0000 7f7d4c400700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7d44075770 0x7f7d4419be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:07:15.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:15.326+0000 7f7d4c400700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7d44076090 0x7f7d4419c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:07:15.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:15.327+0000 7f7d4c400700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d4410ff70 0x7f7d441a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:07:15.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:15.327+0000 7f7d4c400700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7d44115920 con 0x7f7d44076090 2024-05-21T22:07:15.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:15.327+0000 7f7d4c400700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7d441157a0 con 0x7f7d44075770 2024-05-21T22:07:15.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:15.327+0000 7f7d4c400700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 -- 0x7f7d44115aa0 con 0x7f7d4410ff70 2024-05-21T22:07:15.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:15.327+0000 7f7d49a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7d44075770 0x7f7d4419be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:07:15.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:15.327+0000 7f7d49a00700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7d44075770 0x7f7d4419be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.151:56776/0 (socket says 172.21.15.151:56776) 2024-05-21T22:07:15.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:15.327+0000 7f7d49a00700 1 -- 172.21.15.151:0/806834358 learned_addr learned my addr 172.21.15.151:0/806834358 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-05-21T22:07:15.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:15.327+0000 7f7d4a400700 1 --2- 172.21.15.151:0/806834358 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d4410ff70 0x7f7d441a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:07:15.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:15.327+0000 7f7d49000700 1 --2- 172.21.15.151:0/806834358 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7d44076090 0x7f7d4419c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:07:15.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:15.327+0000 7f7d49a00700 1 -- 172.21.15.151:0/806834358 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d4410ff70 msgr2=0x7f7d441a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:07:15.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:15.327+0000 7f7d49a00700 1 --2- 172.21.15.151:0/806834358 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d4410ff70 0x7f7d441a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:07:15.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:15.327+0000 7f7d49a00700 1 -- 172.21.15.151:0/806834358 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7d44076090 msgr2=0x7f7d4419c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:07:15.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:15.327+0000 7f7d49a00700 1 --2- 172.21.15.151:0/806834358 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7d44076090 0x7f7d4419c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:07:15.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:15.327+0000 7f7d49a00700 1 -- 172.21.15.151:0/806834358 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d441a0df0 con 0x7f7d44075770 2024-05-21T22:07:15.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:15.328+0000 7f7d49a00700 1 --2- 172.21.15.151:0/806834358 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7d44075770 0x7f7d4419be30 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f7d3800f990 tx=0x7f7d3800f9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:07:15.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:15.328+0000 7f7d42000700 1 -- 172.21.15.151:0/806834358 <== mon.2 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d38027070 con 0x7f7d44075770 2024-05-21T22:07:15.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:15.328+0000 7f7d42000700 1 -- 172.21.15.151:0/806834358 <== mon.2 v2:172.21.15.164:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d38005570 con 0x7f7d44075770 2024-05-21T22:07:15.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:15.328+0000 7f7d4c400700 1 -- 172.21.15.151:0/806834358 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d441a1080 con 0x7f7d44075770 2024-05-21T22:07:15.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:15.328+0000 7f7d42000700 1 -- 172.21.15.151:0/806834358 <== mon.2 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d38004930 con 0x7f7d44075770 2024-05-21T22:07:15.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:15.328+0000 7f7d4c400700 1 -- 172.21.15.151:0/806834358 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d441a8960 con 0x7f7d44075770 2024-05-21T22:07:15.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:15.329+0000 7f7d4c400700 1 -- 172.21.15.151:0/806834358 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d44076e60 con 0x7f7d44075770 2024-05-21T22:07:15.332 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:15.332+0000 7f7d42000700 1 -- 172.21.15.151:0/806834358 <== mon.2 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 27) ==== 149784+0+0 (secure 0 0 0) 0x7f7d3802a070 con 0x7f7d44075770 2024-05-21T22:07:15.333 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:15.333+0000 7f7d42000700 1 --2- 172.21.15.151:0/806834358 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f7d2c0b29f0 0x7f7d2c0b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T22:07:15.334 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:15.333+0000 7f7d42000700 1 -- 172.21.15.151:0/806834358 <== mon.2 v2:172.21.15.164:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8374+0+0 (secure 0 0 0) 0x7f7d380e36e0 con 0x7f7d44075770 2024-05-21T22:07:15.334 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:15.334+0000 7f7d49000700 1 --2- 172.21.15.151:0/806834358 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f7d2c0b29f0 0x7f7d2c0b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T22:07:15.334 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:15.334+0000 7f7d49000700 1 --2- 172.21.15.151:0/806834358 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f7d2c0b29f0 0x7f7d2c0b4eb0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7d30009750 tx=0x7f7d30009310 comp rx=0 tx=0).ready entity=mgr.25138 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T22:07:15.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:15.335+0000 7f7d42000700 1 -- 172.21.15.151:0/806834358 <== mon.2 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7d38031070 con 0x7f7d44075770 2024-05-21T22:07:15.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:15.569+0000 7f7d4c400700 1 -- 172.21.15.151:0/806834358 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f7d4419d140 con 0x7f7d44075770 2024-05-21T22:07:15.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:07:15 smithi151 ceph-mon[31623]: pgmap v362: 65 pgs: 65 active+clean; 258 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T22:07:15.703 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:07:15 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T22:07:15.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:07:15 smithi184 ceph-mon[36942]: pgmap v362: 65 pgs: 65 active+clean; 258 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T22:07:15.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:07:15 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T22:07:16.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:07:15 smithi164 ceph-mon[36269]: pgmap v362: 65 pgs: 65 active+clean; 258 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T22:07:16.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:07:15 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-21T22:07:16.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:16.572+0000 7f7d42000700 1 -- 172.21.15.151:0/806834358 <== mon.2 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v94) ==== 169+0+0 (secure 0 0 0) 0x7f7d38013030 con 0x7f7d44075770 2024-05-21T22:07:16.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:16.578+0000 7f7d4c400700 1 -- 172.21.15.151:0/806834358 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f7d2c0b29f0 msgr2=0x7f7d2c0b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:07:16.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:16.578+0000 7f7d4c400700 1 --2- 172.21.15.151:0/806834358 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f7d2c0b29f0 0x7f7d2c0b4eb0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7d30009750 tx=0x7f7d30009310 comp rx=0 tx=0).stop 2024-05-21T22:07:16.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:16.578+0000 7f7d4c400700 1 -- 172.21.15.151:0/806834358 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7d44075770 msgr2=0x7f7d4419be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T22:07:16.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:16.578+0000 7f7d4c400700 1 --2- 172.21.15.151:0/806834358 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7d44075770 0x7f7d4419be30 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f7d3800f990 tx=0x7f7d3800f9c0 comp rx=0 tx=0).stop 2024-05-21T22:07:16.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:16.579+0000 7f7d4c400700 1 -- 172.21.15.151:0/806834358 shutdown_connections 2024-05-21T22:07:16.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:16.579+0000 7f7d4c400700 1 --2- 172.21.15.151:0/806834358 >> [v2:172.21.15.151:6800/958417056,v1:172.21.15.151:6801/958417056] conn(0x7f7d2c0b29f0 0x7f7d2c0b4eb0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:07:16.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:16.579+0000 7f7d4c400700 1 --2- 172.21.15.151:0/806834358 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7d44075770 0x7f7d4419be30 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:07:16.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:16.579+0000 7f7d4c400700 1 --2- 172.21.15.151:0/806834358 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7d4410ff70 0x7f7d441a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:07:16.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:16.579+0000 7f7d4c400700 1 --2- 172.21.15.151:0/806834358 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7d44076090 0x7f7d4419c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T22:07:16.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:16.579+0000 7f7d4c400700 1 -- 172.21.15.151:0/806834358 >> 172.21.15.151:0/806834358 conn(0x7f7d4407a6f0 msgr2=0x7f7d44100820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T22:07:16.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:16.579+0000 7f7d4c400700 1 -- 172.21.15.151:0/806834358 shutdown_connections 2024-05-21T22:07:16.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-05-21T22:07:16.579+0000 7f7d4c400700 1 -- 172.21.15.151:0/806834358 wait complete. 2024-05-21T22:07:16.581 INFO:teuthology.orchestra.run.smithi151.stderr:pool 'cephfs_data' removed 2024-05-21T22:07:16.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:07:16 smithi151 ceph-mon[31623]: from='client.? 172.21.15.151:0/806834358' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-21T22:07:16.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:07:16 smithi151 ceph-mon[31623]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-21T22:07:16.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:07:16 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T22:07:16.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:07:16 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T22:07:16.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:07:16 smithi151 ceph-mon[31623]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T22:07:16.985 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:07:16 smithi184 ceph-mon[36942]: from='client.? 172.21.15.151:0/806834358' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-21T22:07:16.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:07:16 smithi184 ceph-mon[36942]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-21T22:07:16.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:07:16 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T22:07:16.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:07:16 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T22:07:16.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:07:16 smithi184 ceph-mon[36942]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T22:07:17.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:07:16 smithi164 ceph-mon[36269]: from='client.? 172.21.15.151:0/806834358' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-21T22:07:17.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:07:16 smithi164 ceph-mon[36269]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-21T22:07:17.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:07:16 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-21T22:07:17.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:07:16 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-21T22:07:17.036 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:07:16 smithi164 ceph-mon[36269]: from='mgr.25138 172.21.15.151:0/3519188646' entity='mgr.x' 2024-05-21T22:07:17.383 DEBUG:teuthology.orchestra.run.smithi151:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-21T22:07:17.456 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-21T22:07:17.484 DEBUG:teuthology.orchestra.run.smithi184:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-21T22:07:17.509 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-05-21T22:07:17.510 DEBUG:teuthology.orchestra.run.smithi151:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-21T22:07:17.554 DEBUG:teuthology.orchestra.run.smithi164:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-21T22:07:17.569 DEBUG:teuthology.orchestra.run.smithi184:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-21T22:07:17.584 INFO:tasks.cephadm:Stopping all daemons... 2024-05-21T22:07:17.584 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-05-21T22:07:17.584 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@mon.a 2024-05-21T22:07:17.659 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:07:17 smithi151 ceph-mon[31623]: pgmap v363: 65 pgs: 65 active+clean; 258 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T22:07:17.660 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:07:17 smithi151 ceph-mon[31623]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-21T22:07:17.660 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:07:17 smithi151 ceph-mon[31623]: osdmap e94: 12 total, 12 up, 12 in 2024-05-21T22:07:17.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:07:17 smithi164 ceph-mon[36269]: pgmap v363: 65 pgs: 65 active+clean; 258 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T22:07:17.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:07:17 smithi164 ceph-mon[36269]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-21T22:07:17.786 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:07:17 smithi164 ceph-mon[36269]: osdmap e94: 12 total, 12 up, 12 in 2024-05-21T22:07:17.953 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:07:17 smithi151 systemd[1]: Stopping Ceph mon.a for c8c7a2e6-17ba-11ef-bc9a-c7b262605968... 2024-05-21T22:07:17.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:07:17 smithi184 ceph-mon[36942]: pgmap v363: 65 pgs: 65 active+clean; 258 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-21T22:07:17.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:07:17 smithi184 ceph-mon[36942]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-21T22:07:17.986 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:07:17 smithi184 ceph-mon[36942]: osdmap e94: 12 total, 12 up, 12 in 2024-05-21T22:07:18.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:07:18 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mon-a[31619]: 2024-05-21T22:07:18.085+0000 7fb063000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T22:07:18.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:07:18 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mon-a[31619]: 2024-05-21T22:07:18.085+0000 7fb063000700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-05-21T22:07:18.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:07:18 smithi151 podman[121013]: 2024-05-21 22:07:18.231497471 +0000 UTC m=+0.271072371 container died 9cfe13f3b9612e0b654901b8e1252a003ced7683fbca5f515e236eedb8a1078e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mon-a, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-05-21T22:07:19.025 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:07:18 smithi151 podman[121013]: 2024-05-21 22:07:18.764508459 +0000 UTC m=+0.804083384 container cleanup 9cfe13f3b9612e0b654901b8e1252a003ced7683fbca5f515e236eedb8a1078e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mon-a, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-05-21T22:07:19.026 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:07:18 smithi151 bash[121013]: ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mon-a 2024-05-21T22:07:19.453 INFO:journalctl@ceph.mon.a.smithi151.stdout:May 21 22:07:19 smithi151 podman[121025]: 2024-05-21 22:07:19.024731111 +0000 UTC m=+0.784051533 container remove 9cfe13f3b9612e0b654901b8e1252a003ced7683fbca5f515e236eedb8a1078e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mon-a, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-05-21T22:07:19.511 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@mon.a.service' 2024-05-21T22:07:19.624 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T22:07:19.624 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-05-21T22:07:19.625 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-05-21T22:07:19.625 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@mon.b 2024-05-21T22:07:19.926 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:07:19 smithi164 systemd[1]: Stopping Ceph mon.b for c8c7a2e6-17ba-11ef-bc9a-c7b262605968... 2024-05-21T22:07:20.261 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:07:19 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mon-b[36264]: 2024-05-21T22:07:19.925+0000 7fa242191700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T22:07:20.261 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:07:19 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mon-b[36264]: 2024-05-21T22:07:19.925+0000 7fa242191700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-05-21T22:07:20.261 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:07:20 smithi164 podman[72500]: 2024-05-21 22:07:20.00711871 +0000 UTC m=+0.183552224 container died 921034200eb5664d9bfa4bbe36821e5483583cfc91526f16e6499d118ba2ac61 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-05-21T22:07:20.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:07:20 smithi164 podman[72500]: 2024-05-21 22:07:20.351575482 +0000 UTC m=+0.528008990 container cleanup 921034200eb5664d9bfa4bbe36821e5483583cfc91526f16e6499d118ba2ac61 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mon-b, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True) 2024-05-21T22:07:20.536 INFO:journalctl@ceph.mon.b.smithi164.stdout:May 21 22:07:20 smithi164 bash[72500]: ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mon-b 2024-05-21T22:07:20.832 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@mon.b.service' 2024-05-21T22:07:20.880 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T22:07:20.881 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-05-21T22:07:20.881 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-05-21T22:07:20.881 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@mon.c 2024-05-21T22:07:21.188 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:07:20 smithi184 systemd[1]: Stopping Ceph mon.c for c8c7a2e6-17ba-11ef-bc9a-c7b262605968... 2024-05-21T22:07:21.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:07:21 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mon-c[36938]: 2024-05-21T22:07:21.187+0000 7fd464941700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T22:07:21.486 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:07:21 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mon-c[36938]: 2024-05-21T22:07:21.187+0000 7fd464941700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-05-21T22:07:21.487 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:07:21 smithi184 podman[74210]: 2024-05-21 22:07:21.422940919 +0000 UTC m=+0.348502557 container died a586a91da0af8b7944a38d3aa1bb30f01bd3f6fbeee37b1953033c7587d92b02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mon-c, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-21T22:07:22.164 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:07:21 smithi184 podman[74210]: 2024-05-21 22:07:21.921834099 +0000 UTC m=+0.847395731 container cleanup a586a91da0af8b7944a38d3aa1bb30f01bd3f6fbeee37b1953033c7587d92b02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mon-c, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD) 2024-05-21T22:07:22.164 INFO:journalctl@ceph.mon.c.smithi184.stdout:May 21 22:07:21 smithi184 bash[74210]: ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mon-c 2024-05-21T22:07:22.310 DEBUG:teuthology.orchestra.run.smithi184:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@mon.c.service' 2024-05-21T22:07:22.363 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T22:07:22.364 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-05-21T22:07:22.364 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-05-21T22:07:22.364 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@mgr.x 2024-05-21T22:07:22.703 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 22:07:22 smithi151 systemd[1]: Stopping Ceph mgr.x for c8c7a2e6-17ba-11ef-bc9a-c7b262605968... 2024-05-21T22:07:23.204 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 22:07:22 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T22:07:22.925+0000 7fbf08400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T22:07:23.204 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 22:07:22 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x[31852]: 2024-05-21T22:07:22.925+0000 7fbf08400700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-05-21T22:07:23.204 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 22:07:23 smithi151 podman[121144]: 2024-05-21 22:07:23.018754496 +0000 UTC m=+0.234510657 container died 297d546f035d9044c7db4aa70103dc4b1b279122d03b50a982c43f618746edff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-05-21T22:07:23.953 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 22:07:23 smithi151 podman[121144]: 2024-05-21 22:07:23.620873485 +0000 UTC m=+0.836629689 container cleanup 297d546f035d9044c7db4aa70103dc4b1b279122d03b50a982c43f618746edff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-05-21T22:07:23.953 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 22:07:23 smithi151 bash[121144]: ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x 2024-05-21T22:07:23.953 INFO:journalctl@ceph.mgr.x.smithi151.stdout:May 21 22:07:23 smithi151 podman[121156]: 2024-05-21 22:07:23.798263273 +0000 UTC m=+0.761235506 container remove 297d546f035d9044c7db4aa70103dc4b1b279122d03b50a982c43f618746edff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3b748ea08891006c1152513d25ab4a03f7c9a995, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2) 2024-05-21T22:07:24.352 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@mgr.x.service' 2024-05-21T22:07:24.471 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T22:07:24.471 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-05-21T22:07:24.471 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-05-21T22:07:24.471 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@mgr.y 2024-05-21T22:07:24.786 INFO:journalctl@ceph.mgr.y.smithi164.stdout:May 21 22:07:24 smithi164 systemd[1]: Stopping Ceph mgr.y for c8c7a2e6-17ba-11ef-bc9a-c7b262605968... 2024-05-21T22:07:25.151 INFO:journalctl@ceph.mgr.y.smithi164.stdout:May 21 22:07:24 smithi164 podman[72624]: 2024-05-21 22:07:24.837150213 +0000 UTC m=+0.173315404 container died 64dc68d1e6d00c60eaf06d621145bc56f0fec52932d5668990a4db17e19998e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-y, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-05-21T22:07:25.468 INFO:journalctl@ceph.mgr.y.smithi164.stdout:May 21 22:07:25 smithi164 podman[72624]: 2024-05-21 22:07:25.250039013 +0000 UTC m=+0.586204193 container cleanup 64dc68d1e6d00c60eaf06d621145bc56f0fec52932d5668990a4db17e19998e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-y, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-21T22:07:25.468 INFO:journalctl@ceph.mgr.y.smithi164.stdout:May 21 22:07:25 smithi164 bash[72624]: ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-y 2024-05-21T22:07:25.590 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@mgr.y.service' 2024-05-21T22:07:25.624 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T22:07:25.624 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-05-21T22:07:25.624 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-05-21T22:07:25.624 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@mgr.z 2024-05-21T22:07:25.956 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 21 22:07:25 smithi184 systemd[1]: Stopping Ceph mgr.z for c8c7a2e6-17ba-11ef-bc9a-c7b262605968... 2024-05-21T22:07:26.238 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 21 22:07:25 smithi184 podman[74345]: 2024-05-21 22:07:25.956449454 +0000 UTC m=+0.148355839 container died ed05f69e3229a9e798932e71a5880eb38990e58ffe5f8a51e73f115c6d61355b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-z, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240514, GIT_CLEAN=True) 2024-05-21T22:07:26.486 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 21 22:07:26 smithi184 podman[74345]: 2024-05-21 22:07:26.317197528 +0000 UTC m=+0.509103906 container cleanup ed05f69e3229a9e798932e71a5880eb38990e58ffe5f8a51e73f115c6d61355b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-z, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS) 2024-05-21T22:07:26.486 INFO:journalctl@ceph.mgr.z.smithi184.stdout:May 21 22:07:26 smithi184 bash[74345]: ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mgr-z 2024-05-21T22:07:26.744 DEBUG:teuthology.orchestra.run.smithi184:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@mgr.z.service' 2024-05-21T22:07:26.778 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T22:07:26.778 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-05-21T22:07:26.779 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-05-21T22:07:26.779 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@osd.0 2024-05-21T22:07:27.203 INFO:journalctl@ceph.osd.0.smithi151.stdout:May 21 22:07:26 smithi151 systemd[1]: Stopping Ceph osd.0 for c8c7a2e6-17ba-11ef-bc9a-c7b262605968... 2024-05-21T22:07:27.703 INFO:journalctl@ceph.osd.0.smithi151.stdout:May 21 22:07:27 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-0[44060]: 2024-05-21T22:07:27.379+0000 7f8014c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T22:07:27.703 INFO:journalctl@ceph.osd.0.smithi151.stdout:May 21 22:07:27 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-0[44060]: 2024-05-21T22:07:27.379+0000 7f8014c00700 -1 osd.0 94 *** Got signal Terminated *** 2024-05-21T22:07:27.704 INFO:journalctl@ceph.osd.0.smithi151.stdout:May 21 22:07:27 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-0[44060]: 2024-05-21T22:07:27.379+0000 7f8014c00700 -1 osd.0 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-21T22:07:33.069 INFO:journalctl@ceph.osd.0.smithi151.stdout:May 21 22:07:32 smithi151 podman[121268]: 2024-05-21 22:07:32.765470599 +0000 UTC m=+5.557495520 container died c2162c11191257031a9efa0e01ac69471bf43ac1d73d884c3ac2fa464cd97038 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-05-21T22:07:33.650 INFO:journalctl@ceph.osd.0.smithi151.stdout:May 21 22:07:33 smithi151 podman[121268]: 2024-05-21 22:07:33.34703067 +0000 UTC m=+6.139055648 container cleanup c2162c11191257031a9efa0e01ac69471bf43ac1d73d884c3ac2fa464cd97038 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2) 2024-05-21T22:07:33.650 INFO:journalctl@ceph.osd.0.smithi151.stdout:May 21 22:07:33 smithi151 bash[121268]: ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-0 2024-05-21T22:07:33.953 INFO:journalctl@ceph.osd.0.smithi151.stdout:May 21 22:07:33 smithi151 podman[121284]: 2024-05-21 22:07:33.648972403 +0000 UTC m=+0.872610120 container remove c2162c11191257031a9efa0e01ac69471bf43ac1d73d884c3ac2fa464cd97038 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-0, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0) 2024-05-21T22:07:34.564 INFO:journalctl@ceph.osd.0.smithi151.stdout:May 21 22:07:34 smithi151 podman[121373]: 2024-05-21 22:07:34.167008387 +0000 UTC m=+0.032431087 image pull d6bc8f31cbc1a7165ca248fe9ba0b06ea14b30f46e1ea33ced1a5238e247af1f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28 2024-05-21T22:07:34.565 INFO:journalctl@ceph.osd.0.smithi151.stdout:May 21 22:07:34 smithi151 podman[121373]: 2024-05-21 22:07:34.266307913 +0000 UTC m=+0.131730619 container create 4f33999a87a317f303bbb45e87bdbfedf2980dd21dc4d16f89a21da32216e093 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-05-21T22:07:34.565 INFO:journalctl@ceph.osd.0.smithi151.stdout:May 21 22:07:34 smithi151 podman[121373]: 2024-05-21 22:07:34.563787576 +0000 UTC m=+0.429210321 container init 4f33999a87a317f303bbb45e87bdbfedf2980dd21dc4d16f89a21da32216e093 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-05-21T22:07:34.953 INFO:journalctl@ceph.osd.0.smithi151.stdout:May 21 22:07:34 smithi151 podman[121373]: 2024-05-21 22:07:34.575225426 +0000 UTC m=+0.440648195 container start 4f33999a87a317f303bbb45e87bdbfedf2980dd21dc4d16f89a21da32216e093 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-0-deactivate, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-05-21T22:07:34.953 INFO:journalctl@ceph.osd.0.smithi151.stdout:May 21 22:07:34 smithi151 podman[121373]: 2024-05-21 22:07:34.671091535 +0000 UTC m=+0.536514301 container attach 4f33999a87a317f303bbb45e87bdbfedf2980dd21dc4d16f89a21da32216e093 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-05-21T22:07:35.413 INFO:journalctl@ceph.osd.0.smithi151.stdout:May 21 22:07:35 smithi151 podman[121373]: 2024-05-21 22:07:35.034921569 +0000 UTC m=+0.900344336 container died 4f33999a87a317f303bbb45e87bdbfedf2980dd21dc4d16f89a21da32216e093 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-05-21T22:07:35.856 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@osd.0.service' 2024-05-21T22:07:35.965 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T22:07:35.966 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-05-21T22:07:35.966 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-05-21T22:07:35.966 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@osd.1 2024-05-21T22:07:36.286 INFO:journalctl@ceph.osd.1.smithi164.stdout:May 21 22:07:36 smithi164 systemd[1]: Stopping Ceph osd.1 for c8c7a2e6-17ba-11ef-bc9a-c7b262605968... 2024-05-21T22:07:36.786 INFO:journalctl@ceph.osd.1.smithi164.stdout:May 21 22:07:36 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-1[40856]: 2024-05-21T22:07:36.287+0000 7f5f54c75700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T22:07:36.786 INFO:journalctl@ceph.osd.1.smithi164.stdout:May 21 22:07:36 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-1[40856]: 2024-05-21T22:07:36.287+0000 7f5f54c75700 -1 osd.1 94 *** Got signal Terminated *** 2024-05-21T22:07:36.786 INFO:journalctl@ceph.osd.1.smithi164.stdout:May 21 22:07:36 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-1[40856]: 2024-05-21T22:07:36.287+0000 7f5f54c75700 -1 osd.1 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-21T22:07:42.089 INFO:journalctl@ceph.osd.1.smithi164.stdout:May 21 22:07:41 smithi164 podman[72750]: 2024-05-21 22:07:41.817465566 +0000 UTC m=+5.659964649 container died 55d1bf230b7a8a2c5a0ea18f39fe99823379a4de15556735f06dc1f58d9d6a9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-1, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-21T22:07:42.439 INFO:journalctl@ceph.osd.1.smithi164.stdout:May 21 22:07:42 smithi164 podman[72750]: 2024-05-21 22:07:42.188095838 +0000 UTC m=+6.030594912 container cleanup 55d1bf230b7a8a2c5a0ea18f39fe99823379a4de15556735f06dc1f58d9d6a9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-1, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-21T22:07:42.439 INFO:journalctl@ceph.osd.1.smithi164.stdout:May 21 22:07:42 smithi164 bash[72750]: ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-1 2024-05-21T22:07:42.786 INFO:journalctl@ceph.osd.1.smithi164.stdout:May 21 22:07:42 smithi164 podman[72767]: 2024-05-21 22:07:42.439620969 +0000 UTC m=+0.615985730 container remove 55d1bf230b7a8a2c5a0ea18f39fe99823379a4de15556735f06dc1f58d9d6a9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-1, org.label-schema.build-date=20240514, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-21T22:07:43.151 INFO:journalctl@ceph.osd.1.smithi164.stdout:May 21 22:07:42 smithi164 podman[72852]: 2024-05-21 22:07:42.708863743 +0000 UTC m=+0.016920089 image pull d6bc8f31cbc1a7165ca248fe9ba0b06ea14b30f46e1ea33ced1a5238e247af1f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28 2024-05-21T22:07:43.151 INFO:journalctl@ceph.osd.1.smithi164.stdout:May 21 22:07:42 smithi164 podman[72852]: 2024-05-21 22:07:42.807033167 +0000 UTC m=+0.115089493 container create 63d099d8e95a1f76bc87fd108eb53da97fbd1eff73292179d28e5c0ade530008 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-21T22:07:43.151 INFO:journalctl@ceph.osd.1.smithi164.stdout:May 21 22:07:43 smithi164 podman[72852]: 2024-05-21 22:07:43.034516801 +0000 UTC m=+0.342573134 container init 63d099d8e95a1f76bc87fd108eb53da97fbd1eff73292179d28e5c0ade530008 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea) 2024-05-21T22:07:43.151 INFO:journalctl@ceph.osd.1.smithi164.stdout:May 21 22:07:43 smithi164 podman[72852]: 2024-05-21 22:07:43.037675723 +0000 UTC m=+0.345732056 container start 63d099d8e95a1f76bc87fd108eb53da97fbd1eff73292179d28e5c0ade530008 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240514) 2024-05-21T22:07:43.151 INFO:journalctl@ceph.osd.1.smithi164.stdout:May 21 22:07:43 smithi164 podman[72852]: 2024-05-21 22:07:43.151142196 +0000 UTC m=+0.459198533 container attach 63d099d8e95a1f76bc87fd108eb53da97fbd1eff73292179d28e5c0ade530008 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-1-deactivate, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2) 2024-05-21T22:07:43.536 INFO:journalctl@ceph.osd.1.smithi164.stdout:May 21 22:07:43 smithi164 podman[72852]: 2024-05-21 22:07:43.332209959 +0000 UTC m=+0.640266293 container died 63d099d8e95a1f76bc87fd108eb53da97fbd1eff73292179d28e5c0ade530008 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-21T22:07:43.931 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@osd.1.service' 2024-05-21T22:07:43.968 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T22:07:43.969 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-05-21T22:07:43.969 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-05-21T22:07:43.969 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@osd.2 2024-05-21T22:07:44.279 INFO:journalctl@ceph.osd.2.smithi184.stdout:May 21 22:07:44 smithi184 systemd[1]: Stopping Ceph osd.2 for c8c7a2e6-17ba-11ef-bc9a-c7b262605968... 2024-05-21T22:07:44.736 INFO:journalctl@ceph.osd.2.smithi184.stdout:May 21 22:07:44 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-2[42060]: 2024-05-21T22:07:44.278+0000 7fe49951b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T22:07:44.736 INFO:journalctl@ceph.osd.2.smithi184.stdout:May 21 22:07:44 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-2[42060]: 2024-05-21T22:07:44.278+0000 7fe49951b700 -1 osd.2 94 *** Got signal Terminated *** 2024-05-21T22:07:44.736 INFO:journalctl@ceph.osd.2.smithi184.stdout:May 21 22:07:44 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-2[42060]: 2024-05-21T22:07:44.278+0000 7fe49951b700 -1 osd.2 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-21T22:07:50.054 INFO:journalctl@ceph.osd.2.smithi184.stdout:May 21 22:07:49 smithi184 podman[74467]: 2024-05-21 22:07:49.76347677 +0000 UTC m=+5.598274040 container died 209e64b2a5d60279e7e0c6e1687b2d76138d4cf70a957a59a46ceb704c26cc0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-2, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-05-21T22:07:50.382 INFO:journalctl@ceph.osd.2.smithi184.stdout:May 21 22:07:50 smithi184 podman[74467]: 2024-05-21 22:07:50.153170125 +0000 UTC m=+5.987967394 container cleanup 209e64b2a5d60279e7e0c6e1687b2d76138d4cf70a957a59a46ceb704c26cc0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-05-21T22:07:50.382 INFO:journalctl@ceph.osd.2.smithi184.stdout:May 21 22:07:50 smithi184 bash[74467]: ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-2 2024-05-21T22:07:50.692 INFO:journalctl@ceph.osd.2.smithi184.stdout:May 21 22:07:50 smithi184 podman[74482]: 2024-05-21 22:07:50.379391242 +0000 UTC m=+0.610460611 container remove 209e64b2a5d60279e7e0c6e1687b2d76138d4cf70a957a59a46ceb704c26cc0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True) 2024-05-21T22:07:50.944 INFO:journalctl@ceph.osd.2.smithi184.stdout:May 21 22:07:50 smithi184 podman[74571]: 2024-05-21 22:07:50.59501055 +0000 UTC m=+0.017276092 image pull d6bc8f31cbc1a7165ca248fe9ba0b06ea14b30f46e1ea33ced1a5238e247af1f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28 2024-05-21T22:07:50.944 INFO:journalctl@ceph.osd.2.smithi184.stdout:May 21 22:07:50 smithi184 podman[74571]: 2024-05-21 22:07:50.713616812 +0000 UTC m=+0.135882375 container create d3b6dde7d3b58945dc37a2163567d86155e20a4ec09db49ef1eb9df2a1a83e51 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-21T22:07:50.944 INFO:journalctl@ceph.osd.2.smithi184.stdout:May 21 22:07:50 smithi184 podman[74571]: 2024-05-21 22:07:50.94115897 +0000 UTC m=+0.363424503 container init d3b6dde7d3b58945dc37a2163567d86155e20a4ec09db49ef1eb9df2a1a83e51 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514) 2024-05-21T22:07:50.945 INFO:journalctl@ceph.osd.2.smithi184.stdout:May 21 22:07:50 smithi184 podman[74571]: 2024-05-21 22:07:50.94450706 +0000 UTC m=+0.366772603 container start d3b6dde7d3b58945dc37a2163567d86155e20a4ec09db49ef1eb9df2a1a83e51 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux ) 2024-05-21T22:07:51.235 INFO:journalctl@ceph.osd.2.smithi184.stdout:May 21 22:07:51 smithi184 podman[74571]: 2024-05-21 22:07:51.041387065 +0000 UTC m=+0.463652609 container attach d3b6dde7d3b58945dc37a2163567d86155e20a4ec09db49ef1eb9df2a1a83e51 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-05-21T22:07:51.486 INFO:journalctl@ceph.osd.2.smithi184.stdout:May 21 22:07:51 smithi184 podman[74571]: 2024-05-21 22:07:51.235904553 +0000 UTC m=+0.658170102 container died d3b6dde7d3b58945dc37a2163567d86155e20a4ec09db49ef1eb9df2a1a83e51 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-05-21T22:07:51.860 DEBUG:teuthology.orchestra.run.smithi184:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@osd.2.service' 2024-05-21T22:07:51.898 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T22:07:51.898 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-05-21T22:07:51.898 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-05-21T22:07:51.899 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@osd.3 2024-05-21T22:07:52.425 INFO:journalctl@ceph.osd.3.smithi151.stdout:May 21 22:07:52 smithi151 systemd[1]: Stopping Ceph osd.3 for c8c7a2e6-17ba-11ef-bc9a-c7b262605968... 2024-05-21T22:07:52.703 INFO:journalctl@ceph.osd.3.smithi151.stdout:May 21 22:07:52 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-3[50544]: 2024-05-21T22:07:52.454+0000 7f1893800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T22:07:52.703 INFO:journalctl@ceph.osd.3.smithi151.stdout:May 21 22:07:52 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-3[50544]: 2024-05-21T22:07:52.454+0000 7f1893800700 -1 osd.3 94 *** Got signal Terminated *** 2024-05-21T22:07:52.703 INFO:journalctl@ceph.osd.3.smithi151.stdout:May 21 22:07:52 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-3[50544]: 2024-05-21T22:07:52.454+0000 7f1893800700 -1 osd.3 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-21T22:07:54.036 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:07:53 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:07:53.676+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:30.508363+0000 front 2024-05-21T22:07:30.508149+0000 (oldest deadline 2024-05-21T22:07:52.808006+0000) 2024-05-21T22:07:54.153 INFO:journalctl@ceph.osd.3.smithi151.stdout:May 21 22:07:53 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-3[50544]: 2024-05-21T22:07:53.715+0000 7f188a000700 -1 osd.3 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:28.673487+0000 front 2024-05-21T22:07:28.673876+0000 (oldest deadline 2024-05-21T22:07:52.773210+0000) 2024-05-21T22:07:54.953 INFO:journalctl@ceph.osd.3.smithi151.stdout:May 21 22:07:54 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-3[50544]: 2024-05-21T22:07:54.676+0000 7f188a000700 -1 osd.3 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:28.673487+0000 front 2024-05-21T22:07:28.673876+0000 (oldest deadline 2024-05-21T22:07:52.773210+0000) 2024-05-21T22:07:54.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:07:54 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:07:54.485+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:27.976452+0000 front 2024-05-21T22:07:27.976969+0000 (oldest deadline 2024-05-21T22:07:53.876086+0000) 2024-05-21T22:07:55.036 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:07:54 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:07:54.641+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:30.508363+0000 front 2024-05-21T22:07:30.508149+0000 (oldest deadline 2024-05-21T22:07:52.808006+0000) 2024-05-21T22:07:55.736 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 21 22:07:55 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-5[48868]: 2024-05-21T22:07:55.336+0000 7f58eb9f2700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:29.310154+0000 front 2024-05-21T22:07:29.309983+0000 (oldest deadline 2024-05-21T22:07:54.609869+0000) 2024-05-21T22:07:55.954 INFO:journalctl@ceph.osd.3.smithi151.stdout:May 21 22:07:55 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-3[50544]: 2024-05-21T22:07:55.674+0000 7f188a000700 -1 osd.3 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:28.673487+0000 front 2024-05-21T22:07:28.673876+0000 (oldest deadline 2024-05-21T22:07:52.773210+0000) 2024-05-21T22:07:55.954 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:07:55 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:07:55.480+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:27.976452+0000 front 2024-05-21T22:07:27.976969+0000 (oldest deadline 2024-05-21T22:07:53.876086+0000) 2024-05-21T22:07:56.036 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:07:55 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:07:55.677+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:30.508363+0000 front 2024-05-21T22:07:30.508149+0000 (oldest deadline 2024-05-21T22:07:52.808006+0000) 2024-05-21T22:07:56.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:07:55 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:07:55.838+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:28.956267+0000 front 2024-05-21T22:07:28.956225+0000 (oldest deadline 2024-05-21T22:07:55.356623+0000) 2024-05-21T22:07:56.736 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 21 22:07:56 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-5[48868]: 2024-05-21T22:07:56.322+0000 7f58eb9f2700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:29.310154+0000 front 2024-05-21T22:07:29.309983+0000 (oldest deadline 2024-05-21T22:07:54.609869+0000) 2024-05-21T22:07:56.953 INFO:journalctl@ceph.osd.3.smithi151.stdout:May 21 22:07:56 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-3[50544]: 2024-05-21T22:07:56.693+0000 7f188a000700 -1 osd.3 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:28.673487+0000 front 2024-05-21T22:07:28.673876+0000 (oldest deadline 2024-05-21T22:07:52.773210+0000) 2024-05-21T22:07:56.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:07:56 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:07:56.529+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:27.976452+0000 front 2024-05-21T22:07:27.976969+0000 (oldest deadline 2024-05-21T22:07:53.876086+0000) 2024-05-21T22:07:57.036 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:07:56 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:07:56.633+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:30.508363+0000 front 2024-05-21T22:07:30.508149+0000 (oldest deadline 2024-05-21T22:07:52.808006+0000) 2024-05-21T22:07:57.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:07:56 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:07:56.853+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:28.956267+0000 front 2024-05-21T22:07:28.956225+0000 (oldest deadline 2024-05-21T22:07:55.356623+0000) 2024-05-21T22:07:57.735 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 21 22:07:57 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-5[48868]: 2024-05-21T22:07:57.290+0000 7f58eb9f2700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:29.310154+0000 front 2024-05-21T22:07:29.309983+0000 (oldest deadline 2024-05-21T22:07:54.609869+0000) 2024-05-21T22:07:57.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:07:57 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:07:57.539+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:27.976452+0000 front 2024-05-21T22:07:27.976969+0000 (oldest deadline 2024-05-21T22:07:53.876086+0000) 2024-05-21T22:07:58.036 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:07:57 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:07:57.646+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:30.508363+0000 front 2024-05-21T22:07:30.508149+0000 (oldest deadline 2024-05-21T22:07:52.808006+0000) 2024-05-21T22:07:58.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:07:57 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:07:57.852+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:28.956267+0000 front 2024-05-21T22:07:28.956225+0000 (oldest deadline 2024-05-21T22:07:55.356623+0000) 2024-05-21T22:07:58.357 INFO:journalctl@ceph.osd.3.smithi151.stdout:May 21 22:07:58 smithi151 podman[121478]: 2024-05-21 22:07:58.000067305 +0000 UTC m=+5.657148519 container died 90351da10fba22fb92eb7469bf4295a00d190778c8e9aa4bb12b355571e4c14d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-3, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea) 2024-05-21T22:07:58.359 INFO:journalctl@ceph.osd.6.smithi151.stdout:May 21 22:07:58 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-6[57087]: 2024-05-21T22:07:58.101+0000 7f0909400700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:31.896170+0000 front 2024-05-21T22:07:31.896361+0000 (oldest deadline 2024-05-21T22:07:57.795636+0000) 2024-05-21T22:07:58.609 INFO:journalctl@ceph.osd.3.smithi151.stdout:May 21 22:07:58 smithi151 podman[121478]: 2024-05-21 22:07:58.574761913 +0000 UTC m=+6.231843144 container cleanup 90351da10fba22fb92eb7469bf4295a00d190778c8e9aa4bb12b355571e4c14d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-3, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-05-21T22:07:58.609 INFO:journalctl@ceph.osd.3.smithi151.stdout:May 21 22:07:58 smithi151 bash[121478]: ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-3 2024-05-21T22:07:58.609 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:07:58 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:07:58.524+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:27.976452+0000 front 2024-05-21T22:07:27.976969+0000 (oldest deadline 2024-05-21T22:07:53.876086+0000) 2024-05-21T22:07:58.736 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 21 22:07:58 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-5[48868]: 2024-05-21T22:07:58.323+0000 7f58eb9f2700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:29.310154+0000 front 2024-05-21T22:07:29.309983+0000 (oldest deadline 2024-05-21T22:07:54.609869+0000) 2024-05-21T22:07:58.953 INFO:journalctl@ceph.osd.3.smithi151.stdout:May 21 22:07:58 smithi151 podman[121511]: 2024-05-21 22:07:58.751632471 +0000 UTC m=+0.736862278 container remove 90351da10fba22fb92eb7469bf4295a00d190778c8e9aa4bb12b355571e4c14d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-3, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, ceph=True) 2024-05-21T22:07:59.036 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:07:58 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:07:58.649+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:30.508363+0000 front 2024-05-21T22:07:30.508149+0000 (oldest deadline 2024-05-21T22:07:52.808006+0000) 2024-05-21T22:07:59.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:07:58 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:07:58.820+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:28.956267+0000 front 2024-05-21T22:07:28.956225+0000 (oldest deadline 2024-05-21T22:07:55.356623+0000) 2024-05-21T22:07:59.453 INFO:journalctl@ceph.osd.6.smithi151.stdout:May 21 22:07:59 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-6[57087]: 2024-05-21T22:07:59.056+0000 7f0909400700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:31.896170+0000 front 2024-05-21T22:07:31.896361+0000 (oldest deadline 2024-05-21T22:07:57.795636+0000) 2024-05-21T22:07:59.736 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 21 22:07:59 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-5[48868]: 2024-05-21T22:07:59.323+0000 7f58eb9f2700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:29.310154+0000 front 2024-05-21T22:07:29.309983+0000 (oldest deadline 2024-05-21T22:07:54.609869+0000) 2024-05-21T22:07:59.874 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:07:59 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:07:59.560+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:27.976452+0000 front 2024-05-21T22:07:27.976969+0000 (oldest deadline 2024-05-21T22:07:53.876086+0000) 2024-05-21T22:07:59.874 INFO:journalctl@ceph.osd.3.smithi151.stdout:May 21 22:07:59 smithi151 podman[121600]: 2024-05-21 22:07:59.491344004 +0000 UTC m=+0.033306177 image pull d6bc8f31cbc1a7165ca248fe9ba0b06ea14b30f46e1ea33ced1a5238e247af1f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28 2024-05-21T22:07:59.874 INFO:journalctl@ceph.osd.3.smithi151.stdout:May 21 22:07:59 smithi151 podman[121600]: 2024-05-21 22:07:59.602896822 +0000 UTC m=+0.144859040 container create 1e1c018f4050d7abb54c12ce29a4a871cabb5c1a2bed28d18c4a9ae475f2841c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514) 2024-05-21T22:08:00.036 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:07:59 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:07:59.615+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:30.508363+0000 front 2024-05-21T22:07:30.508149+0000 (oldest deadline 2024-05-21T22:07:52.808006+0000) 2024-05-21T22:08:00.203 INFO:journalctl@ceph.osd.6.smithi151.stdout:May 21 22:08:00 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-6[57087]: 2024-05-21T22:08:00.099+0000 7f0909400700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:31.896170+0000 front 2024-05-21T22:07:31.896361+0000 (oldest deadline 2024-05-21T22:07:57.795636+0000) 2024-05-21T22:08:00.203 INFO:journalctl@ceph.osd.3.smithi151.stdout:May 21 22:07:59 smithi151 podman[121600]: 2024-05-21 22:07:59.873462275 +0000 UTC m=+0.415424487 container init 1e1c018f4050d7abb54c12ce29a4a871cabb5c1a2bed28d18c4a9ae475f2841c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-05-21T22:08:00.203 INFO:journalctl@ceph.osd.3.smithi151.stdout:May 21 22:07:59 smithi151 podman[121600]: 2024-05-21 22:07:59.884980248 +0000 UTC m=+0.426942434 container start 1e1c018f4050d7abb54c12ce29a4a871cabb5c1a2bed28d18c4a9ae475f2841c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-21T22:08:00.203 INFO:journalctl@ceph.osd.3.smithi151.stdout:May 21 22:08:00 smithi151 podman[121600]: 2024-05-21 22:08:00.018394853 +0000 UTC m=+0.560357068 container attach 1e1c018f4050d7abb54c12ce29a4a871cabb5c1a2bed28d18c4a9ae475f2841c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-3-deactivate, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-05-21T22:08:00.235 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:07:59 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:07:59.836+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:28.956267+0000 front 2024-05-21T22:07:28.956225+0000 (oldest deadline 2024-05-21T22:07:55.356623+0000) 2024-05-21T22:08:00.598 INFO:journalctl@ceph.osd.3.smithi151.stdout:May 21 22:08:00 smithi151 podman[121600]: 2024-05-21 22:08:00.34836839 +0000 UTC m=+0.890330605 container died 1e1c018f4050d7abb54c12ce29a4a871cabb5c1a2bed28d18c4a9ae475f2841c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea) 2024-05-21T22:08:00.735 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 21 22:08:00 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-5[48868]: 2024-05-21T22:08:00.295+0000 7f58eb9f2700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:29.310154+0000 front 2024-05-21T22:07:29.309983+0000 (oldest deadline 2024-05-21T22:07:54.609869+0000) 2024-05-21T22:08:00.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:00 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:00.596+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:27.976452+0000 front 2024-05-21T22:07:27.976969+0000 (oldest deadline 2024-05-21T22:07:53.876086+0000) 2024-05-21T22:08:01.036 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:00 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:00.629+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:30.508363+0000 front 2024-05-21T22:07:30.508149+0000 (oldest deadline 2024-05-21T22:07:52.808006+0000) 2024-05-21T22:08:01.042 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@osd.3.service' 2024-05-21T22:08:01.153 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T22:08:01.154 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-05-21T22:08:01.154 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-05-21T22:08:01.154 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@osd.4 2024-05-21T22:08:01.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:00 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:00.801+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:28.956267+0000 front 2024-05-21T22:07:28.956225+0000 (oldest deadline 2024-05-21T22:07:55.356623+0000) 2024-05-21T22:08:01.449 INFO:journalctl@ceph.osd.4.smithi164.stdout:May 21 22:08:01 smithi164 systemd[1]: Stopping Ceph osd.4 for c8c7a2e6-17ba-11ef-bc9a-c7b262605968... 2024-05-21T22:08:01.453 INFO:journalctl@ceph.osd.6.smithi151.stdout:May 21 22:08:01 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-6[57087]: 2024-05-21T22:08:01.054+0000 7f0909400700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:31.896170+0000 front 2024-05-21T22:07:31.896361+0000 (oldest deadline 2024-05-21T22:07:57.795636+0000) 2024-05-21T22:08:01.736 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 21 22:08:01 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-5[48868]: 2024-05-21T22:08:01.345+0000 7f58eb9f2700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:29.310154+0000 front 2024-05-21T22:07:29.309983+0000 (oldest deadline 2024-05-21T22:07:54.609869+0000) 2024-05-21T22:08:01.786 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:01 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:01.664+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:30.508363+0000 front 2024-05-21T22:07:30.508149+0000 (oldest deadline 2024-05-21T22:07:52.808006+0000) 2024-05-21T22:08:01.786 INFO:journalctl@ceph.osd.4.smithi164.stdout:May 21 22:08:01 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-4[47510]: 2024-05-21T22:08:01.448+0000 7f4b9cea1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T22:08:01.786 INFO:journalctl@ceph.osd.4.smithi164.stdout:May 21 22:08:01 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-4[47510]: 2024-05-21T22:08:01.448+0000 7f4b9cea1700 -1 osd.4 94 *** Got signal Terminated *** 2024-05-21T22:08:01.786 INFO:journalctl@ceph.osd.4.smithi164.stdout:May 21 22:08:01 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-4[47510]: 2024-05-21T22:08:01.448+0000 7f4b9cea1700 -1 osd.4 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-21T22:08:01.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:01 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:01.646+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:27.976452+0000 front 2024-05-21T22:07:27.976969+0000 (oldest deadline 2024-05-21T22:07:53.876086+0000) 2024-05-21T22:08:02.235 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:01 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:01.805+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:28.956267+0000 front 2024-05-21T22:07:28.956225+0000 (oldest deadline 2024-05-21T22:07:55.356623+0000) 2024-05-21T22:08:02.453 INFO:journalctl@ceph.osd.6.smithi151.stdout:May 21 22:08:02 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-6[57087]: 2024-05-21T22:08:02.099+0000 7f0909400700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:31.896170+0000 front 2024-05-21T22:07:31.896361+0000 (oldest deadline 2024-05-21T22:07:57.795636+0000) 2024-05-21T22:08:02.735 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 21 22:08:02 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-5[48868]: 2024-05-21T22:08:02.324+0000 7f58eb9f2700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:29.310154+0000 front 2024-05-21T22:07:29.309983+0000 (oldest deadline 2024-05-21T22:07:54.609869+0000) 2024-05-21T22:08:02.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:02 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:02.673+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:27.976452+0000 front 2024-05-21T22:07:27.976969+0000 (oldest deadline 2024-05-21T22:07:53.876086+0000) 2024-05-21T22:08:03.036 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:02 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:02.619+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:30.508363+0000 front 2024-05-21T22:07:30.508149+0000 (oldest deadline 2024-05-21T22:07:52.808006+0000) 2024-05-21T22:08:03.036 INFO:journalctl@ceph.osd.7.smithi164.stdout:May 21 22:08:02 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-7[54180]: 2024-05-21T22:08:02.701+0000 7f8d4da11700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:37.480876+0000 front 2024-05-21T22:07:37.480856+0000 (oldest deadline 2024-05-21T22:08:02.181059+0000) 2024-05-21T22:08:03.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:02 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:02.770+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:28.956267+0000 front 2024-05-21T22:07:28.956225+0000 (oldest deadline 2024-05-21T22:07:55.356623+0000) 2024-05-21T22:08:03.453 INFO:journalctl@ceph.osd.6.smithi151.stdout:May 21 22:08:03 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-6[57087]: 2024-05-21T22:08:03.133+0000 7f0909400700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:31.896170+0000 front 2024-05-21T22:07:31.896361+0000 (oldest deadline 2024-05-21T22:07:57.795636+0000) 2024-05-21T22:08:03.536 INFO:journalctl@ceph.osd.4.smithi164.stdout:May 21 22:08:03 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-4[47510]: 2024-05-21T22:08:03.036+0000 7f4b99495700 -1 osd.4 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:39.046854+0000 front 2024-05-21T22:07:39.046881+0000 (oldest deadline 2024-05-21T22:08:02.546849+0000) 2024-05-21T22:08:03.735 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 21 22:08:03 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-5[48868]: 2024-05-21T22:08:03.290+0000 7f58eb9f2700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:29.310154+0000 front 2024-05-21T22:07:29.309983+0000 (oldest deadline 2024-05-21T22:07:54.609869+0000) 2024-05-21T22:08:03.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:03 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:03.658+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:27.976452+0000 front 2024-05-21T22:07:27.976969+0000 (oldest deadline 2024-05-21T22:07:53.876086+0000) 2024-05-21T22:08:04.036 INFO:journalctl@ceph.osd.7.smithi164.stdout:May 21 22:08:03 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-7[54180]: 2024-05-21T22:08:03.684+0000 7f8d4da11700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:37.480876+0000 front 2024-05-21T22:07:37.480856+0000 (oldest deadline 2024-05-21T22:08:02.181059+0000) 2024-05-21T22:08:04.036 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:03 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:03.574+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:30.508363+0000 front 2024-05-21T22:07:30.508149+0000 (oldest deadline 2024-05-21T22:07:52.808006+0000) 2024-05-21T22:08:04.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:03 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:03.813+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:28.956267+0000 front 2024-05-21T22:07:28.956225+0000 (oldest deadline 2024-05-21T22:07:55.356623+0000) 2024-05-21T22:08:04.453 INFO:journalctl@ceph.osd.6.smithi151.stdout:May 21 22:08:04 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-6[57087]: 2024-05-21T22:08:04.143+0000 7f0909400700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:31.896170+0000 front 2024-05-21T22:07:31.896361+0000 (oldest deadline 2024-05-21T22:07:57.795636+0000) 2024-05-21T22:08:04.536 INFO:journalctl@ceph.osd.4.smithi164.stdout:May 21 22:08:04 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-4[47510]: 2024-05-21T22:08:04.076+0000 7f4b99495700 -1 osd.4 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:39.046854+0000 front 2024-05-21T22:07:39.046881+0000 (oldest deadline 2024-05-21T22:08:02.546849+0000) 2024-05-21T22:08:04.736 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 21 22:08:04 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-5[48868]: 2024-05-21T22:08:04.283+0000 7f58eb9f2700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:29.310154+0000 front 2024-05-21T22:07:29.309983+0000 (oldest deadline 2024-05-21T22:07:54.609869+0000) 2024-05-21T22:08:04.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:04 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:04.615+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:27.976452+0000 front 2024-05-21T22:07:27.976969+0000 (oldest deadline 2024-05-21T22:07:53.876086+0000) 2024-05-21T22:08:04.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:04 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:04.615+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:39.580127+0000 front 2024-05-21T22:07:39.580305+0000 (oldest deadline 2024-05-21T22:08:04.280362+0000) 2024-05-21T22:08:05.036 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:04 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:04.613+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:30.508363+0000 front 2024-05-21T22:07:30.508149+0000 (oldest deadline 2024-05-21T22:07:52.808006+0000) 2024-05-21T22:08:05.036 INFO:journalctl@ceph.osd.7.smithi164.stdout:May 21 22:08:04 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-7[54180]: 2024-05-21T22:08:04.721+0000 7f8d4da11700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:37.480876+0000 front 2024-05-21T22:07:37.480856+0000 (oldest deadline 2024-05-21T22:08:02.181059+0000) 2024-05-21T22:08:05.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:04 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:04.811+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:28.956267+0000 front 2024-05-21T22:07:28.956225+0000 (oldest deadline 2024-05-21T22:07:55.356623+0000) 2024-05-21T22:08:05.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:04 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:04.811+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:38.857667+0000 front 2024-05-21T22:07:38.857566+0000 (oldest deadline 2024-05-21T22:08:04.757680+0000) 2024-05-21T22:08:05.453 INFO:journalctl@ceph.osd.6.smithi151.stdout:May 21 22:08:05 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-6[57087]: 2024-05-21T22:08:05.157+0000 7f0909400700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:31.896170+0000 front 2024-05-21T22:07:31.896361+0000 (oldest deadline 2024-05-21T22:07:57.795636+0000) 2024-05-21T22:08:05.453 INFO:journalctl@ceph.osd.6.smithi151.stdout:May 21 22:08:05 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-6[57087]: 2024-05-21T22:08:05.157+0000 7f0909400700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:38.298380+0000 front 2024-05-21T22:07:38.298169+0000 (oldest deadline 2024-05-21T22:08:04.198343+0000) 2024-05-21T22:08:05.536 INFO:journalctl@ceph.osd.4.smithi164.stdout:May 21 22:08:05 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-4[47510]: 2024-05-21T22:08:05.107+0000 7f4b99495700 -1 osd.4 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:39.046854+0000 front 2024-05-21T22:07:39.046881+0000 (oldest deadline 2024-05-21T22:08:02.546849+0000) 2024-05-21T22:08:05.736 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 21 22:08:05 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-5[48868]: 2024-05-21T22:08:05.240+0000 7f58eb9f2700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:29.310154+0000 front 2024-05-21T22:07:29.309983+0000 (oldest deadline 2024-05-21T22:07:54.609869+0000) 2024-05-21T22:08:05.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:05 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:05.588+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:27.976452+0000 front 2024-05-21T22:07:27.976969+0000 (oldest deadline 2024-05-21T22:07:53.876086+0000) 2024-05-21T22:08:05.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:05 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:05.588+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:39.580127+0000 front 2024-05-21T22:07:39.580305+0000 (oldest deadline 2024-05-21T22:08:04.280362+0000) 2024-05-21T22:08:06.036 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:05 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:05.595+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:30.508363+0000 front 2024-05-21T22:07:30.508149+0000 (oldest deadline 2024-05-21T22:07:52.808006+0000) 2024-05-21T22:08:06.036 INFO:journalctl@ceph.osd.7.smithi164.stdout:May 21 22:08:05 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-7[54180]: 2024-05-21T22:08:05.728+0000 7f8d4da11700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:37.480876+0000 front 2024-05-21T22:07:37.480856+0000 (oldest deadline 2024-05-21T22:08:02.181059+0000) 2024-05-21T22:08:06.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:05 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:05.823+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:28.956267+0000 front 2024-05-21T22:07:28.956225+0000 (oldest deadline 2024-05-21T22:07:55.356623+0000) 2024-05-21T22:08:06.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:05 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:05.823+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:38.857667+0000 front 2024-05-21T22:07:38.857566+0000 (oldest deadline 2024-05-21T22:08:04.757680+0000) 2024-05-21T22:08:06.453 INFO:journalctl@ceph.osd.6.smithi151.stdout:May 21 22:08:06 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-6[57087]: 2024-05-21T22:08:06.187+0000 7f0909400700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:31.896170+0000 front 2024-05-21T22:07:31.896361+0000 (oldest deadline 2024-05-21T22:07:57.795636+0000) 2024-05-21T22:08:06.453 INFO:journalctl@ceph.osd.6.smithi151.stdout:May 21 22:08:06 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-6[57087]: 2024-05-21T22:08:06.187+0000 7f0909400700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:38.298380+0000 front 2024-05-21T22:07:38.298169+0000 (oldest deadline 2024-05-21T22:08:04.198343+0000) 2024-05-21T22:08:06.498 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 21 22:08:06 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-5[48868]: 2024-05-21T22:08:06.238+0000 7f58eb9f2700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:29.310154+0000 front 2024-05-21T22:07:29.309983+0000 (oldest deadline 2024-05-21T22:07:54.609869+0000) 2024-05-21T22:08:06.536 INFO:journalctl@ceph.osd.4.smithi164.stdout:May 21 22:08:06 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-4[47510]: 2024-05-21T22:08:06.127+0000 7f4b99495700 -1 osd.4 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:39.046854+0000 front 2024-05-21T22:07:39.046881+0000 (oldest deadline 2024-05-21T22:08:02.546849+0000) 2024-05-21T22:08:06.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:06 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:06.572+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:27.976452+0000 front 2024-05-21T22:07:27.976969+0000 (oldest deadline 2024-05-21T22:07:53.876086+0000) 2024-05-21T22:08:06.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:06 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:06.572+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:39.580127+0000 front 2024-05-21T22:07:39.580305+0000 (oldest deadline 2024-05-21T22:08:04.280362+0000) 2024-05-21T22:08:07.036 INFO:journalctl@ceph.osd.4.smithi164.stdout:May 21 22:08:06 smithi164 podman[72970]: 2024-05-21 22:08:06.799176115 +0000 UTC m=+5.450035187 container died c937f40e2c94cb8acbead375f33e6ebd01d012709ae1b4e46d0cf844a0a7118a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-4, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea) 2024-05-21T22:08:07.036 INFO:journalctl@ceph.osd.7.smithi164.stdout:May 21 22:08:06 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-7[54180]: 2024-05-21T22:08:06.732+0000 7f8d4da11700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:37.480876+0000 front 2024-05-21T22:07:37.480856+0000 (oldest deadline 2024-05-21T22:08:02.181059+0000) 2024-05-21T22:08:07.037 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:06 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:06.612+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:30.508363+0000 front 2024-05-21T22:07:30.508149+0000 (oldest deadline 2024-05-21T22:07:52.808006+0000) 2024-05-21T22:08:07.037 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:06 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:06.612+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:41.009631+0000 front 2024-05-21T22:07:41.009565+0000 (oldest deadline 2024-05-21T22:08:05.709714+0000) 2024-05-21T22:08:07.236 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:06 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:06.867+0000 7f452bd16700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:40.689332+0000 front 2024-05-21T22:07:40.689184+0000 (oldest deadline 2024-05-21T22:08:06.589360+0000) 2024-05-21T22:08:07.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:06 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:06.842+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:28.956267+0000 front 2024-05-21T22:07:28.956225+0000 (oldest deadline 2024-05-21T22:07:55.356623+0000) 2024-05-21T22:08:07.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:06 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:06.842+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:38.857667+0000 front 2024-05-21T22:07:38.857566+0000 (oldest deadline 2024-05-21T22:08:04.757680+0000) 2024-05-21T22:08:07.441 INFO:journalctl@ceph.osd.4.smithi164.stdout:May 21 22:08:07 smithi164 podman[72970]: 2024-05-21 22:08:07.140240498 +0000 UTC m=+5.791099567 container cleanup c937f40e2c94cb8acbead375f33e6ebd01d012709ae1b4e46d0cf844a0a7118a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-4, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-21T22:08:07.441 INFO:journalctl@ceph.osd.4.smithi164.stdout:May 21 22:08:07 smithi164 bash[72970]: ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-4 2024-05-21T22:08:07.604 INFO:journalctl@ceph.osd.6.smithi151.stdout:May 21 22:08:07 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-6[57087]: 2024-05-21T22:08:07.205+0000 7f0909400700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:31.896170+0000 front 2024-05-21T22:07:31.896361+0000 (oldest deadline 2024-05-21T22:07:57.795636+0000) 2024-05-21T22:08:07.605 INFO:journalctl@ceph.osd.6.smithi151.stdout:May 21 22:08:07 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-6[57087]: 2024-05-21T22:08:07.205+0000 7f0909400700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:38.298380+0000 front 2024-05-21T22:07:38.298169+0000 (oldest deadline 2024-05-21T22:08:04.198343+0000) 2024-05-21T22:08:07.736 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 21 22:08:07 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-5[48868]: 2024-05-21T22:08:07.268+0000 7f58eb9f2700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:29.310154+0000 front 2024-05-21T22:07:29.309983+0000 (oldest deadline 2024-05-21T22:07:54.609869+0000) 2024-05-21T22:08:07.736 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 21 22:08:07 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-5[48868]: 2024-05-21T22:08:07.268+0000 7f58eb9f2700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:38.711280+0000 front 2024-05-21T22:07:38.711222+0000 (oldest deadline 2024-05-21T22:08:06.311633+0000) 2024-05-21T22:08:07.757 INFO:journalctl@ceph.osd.4.smithi164.stdout:May 21 22:08:07 smithi164 podman[72986]: 2024-05-21 22:08:07.441422678 +0000 UTC m=+0.644188725 container remove c937f40e2c94cb8acbead375f33e6ebd01d012709ae1b4e46d0cf844a0a7118a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-4, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-05-21T22:08:07.757 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:07 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:07.657+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:30.508363+0000 front 2024-05-21T22:07:30.508149+0000 (oldest deadline 2024-05-21T22:07:52.808006+0000) 2024-05-21T22:08:07.757 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:07 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:07.657+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:41.009631+0000 front 2024-05-21T22:07:41.009565+0000 (oldest deadline 2024-05-21T22:08:05.709714+0000) 2024-05-21T22:08:07.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:07 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:07.603+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:27.976452+0000 front 2024-05-21T22:07:27.976969+0000 (oldest deadline 2024-05-21T22:07:53.876086+0000) 2024-05-21T22:08:07.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:07 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:07.603+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:39.580127+0000 front 2024-05-21T22:07:39.580305+0000 (oldest deadline 2024-05-21T22:08:04.280362+0000) 2024-05-21T22:08:08.031 INFO:journalctl@ceph.osd.4.smithi164.stdout:May 21 22:08:07 smithi164 podman[73074]: 2024-05-21 22:08:07.809649123 +0000 UTC m=+0.017037306 image pull d6bc8f31cbc1a7165ca248fe9ba0b06ea14b30f46e1ea33ced1a5238e247af1f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28 2024-05-21T22:08:08.032 INFO:journalctl@ceph.osd.4.smithi164.stdout:May 21 22:08:07 smithi164 podman[73074]: 2024-05-21 22:08:07.917727603 +0000 UTC m=+0.125115808 container create 748c1e8ef7237a332232aa8267d30b5e945069bb892e5be7f04c131d65c7eaa7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-4-deactivate, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-05-21T22:08:08.032 INFO:journalctl@ceph.osd.7.smithi164.stdout:May 21 22:08:07 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-7[54180]: 2024-05-21T22:08:07.756+0000 7f8d4da11700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:37.480876+0000 front 2024-05-21T22:07:37.480856+0000 (oldest deadline 2024-05-21T22:08:02.181059+0000) 2024-05-21T22:08:08.235 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:07 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:07.876+0000 7f452bd16700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:40.689332+0000 front 2024-05-21T22:07:40.689184+0000 (oldest deadline 2024-05-21T22:08:06.589360+0000) 2024-05-21T22:08:08.235 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:07 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:07.810+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:28.956267+0000 front 2024-05-21T22:07:28.956225+0000 (oldest deadline 2024-05-21T22:07:55.356623+0000) 2024-05-21T22:08:08.235 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:07 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:07.810+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:38.857667+0000 front 2024-05-21T22:07:38.857566+0000 (oldest deadline 2024-05-21T22:08:04.757680+0000) 2024-05-21T22:08:08.286 INFO:journalctl@ceph.osd.4.smithi164.stdout:May 21 22:08:08 smithi164 podman[73074]: 2024-05-21 22:08:08.136997202 +0000 UTC m=+0.344385379 container init 748c1e8ef7237a332232aa8267d30b5e945069bb892e5be7f04c131d65c7eaa7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea) 2024-05-21T22:08:08.286 INFO:journalctl@ceph.osd.4.smithi164.stdout:May 21 22:08:08 smithi164 podman[73074]: 2024-05-21 22:08:08.139963092 +0000 UTC m=+0.347351282 container start 748c1e8ef7237a332232aa8267d30b5e945069bb892e5be7f04c131d65c7eaa7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-21T22:08:08.286 INFO:journalctl@ceph.osd.4.smithi164.stdout:May 21 22:08:08 smithi164 podman[73074]: 2024-05-21 22:08:08.211716839 +0000 UTC m=+0.419105037 container attach 748c1e8ef7237a332232aa8267d30b5e945069bb892e5be7f04c131d65c7eaa7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-21T22:08:08.486 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 21 22:08:08 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-5[48868]: 2024-05-21T22:08:08.234+0000 7f58eb9f2700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:29.310154+0000 front 2024-05-21T22:07:29.309983+0000 (oldest deadline 2024-05-21T22:07:54.609869+0000) 2024-05-21T22:08:08.486 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 21 22:08:08 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-5[48868]: 2024-05-21T22:08:08.234+0000 7f58eb9f2700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:38.711280+0000 front 2024-05-21T22:07:38.711222+0000 (oldest deadline 2024-05-21T22:08:06.311633+0000) 2024-05-21T22:08:08.650 INFO:journalctl@ceph.osd.6.smithi151.stdout:May 21 22:08:08 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-6[57087]: 2024-05-21T22:08:08.240+0000 7f0909400700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:31.896170+0000 front 2024-05-21T22:07:31.896361+0000 (oldest deadline 2024-05-21T22:07:57.795636+0000) 2024-05-21T22:08:08.650 INFO:journalctl@ceph.osd.6.smithi151.stdout:May 21 22:08:08 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-6[57087]: 2024-05-21T22:08:08.240+0000 7f0909400700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:38.298380+0000 front 2024-05-21T22:07:38.298169+0000 (oldest deadline 2024-05-21T22:08:04.198343+0000) 2024-05-21T22:08:08.731 INFO:journalctl@ceph.osd.4.smithi164.stdout:May 21 22:08:08 smithi164 podman[73074]: 2024-05-21 22:08:08.434418339 +0000 UTC m=+0.641806522 container died 748c1e8ef7237a332232aa8267d30b5e945069bb892e5be7f04c131d65c7eaa7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-4-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-21T22:08:08.731 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:08 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:08.661+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:30.508363+0000 front 2024-05-21T22:07:30.508149+0000 (oldest deadline 2024-05-21T22:07:52.808006+0000) 2024-05-21T22:08:08.732 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:08 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:08.661+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:41.009631+0000 front 2024-05-21T22:07:41.009565+0000 (oldest deadline 2024-05-21T22:08:05.709714+0000) 2024-05-21T22:08:08.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:08 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:08.648+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:27.976452+0000 front 2024-05-21T22:07:27.976969+0000 (oldest deadline 2024-05-21T22:07:53.876086+0000) 2024-05-21T22:08:08.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:08 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:08.648+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:39.580127+0000 front 2024-05-21T22:07:39.580305+0000 (oldest deadline 2024-05-21T22:08:04.280362+0000) 2024-05-21T22:08:08.998 INFO:journalctl@ceph.osd.7.smithi164.stdout:May 21 22:08:08 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-7[54180]: 2024-05-21T22:08:08.790+0000 7f8d4da11700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:37.480876+0000 front 2024-05-21T22:07:37.480856+0000 (oldest deadline 2024-05-21T22:08:02.181059+0000) 2024-05-21T22:08:09.021 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@osd.4.service' 2024-05-21T22:08:09.058 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T22:08:09.058 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-05-21T22:08:09.058 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-05-21T22:08:09.058 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@osd.5 2024-05-21T22:08:09.084 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:08 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:08.866+0000 7f452bd16700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:40.689332+0000 front 2024-05-21T22:07:40.689184+0000 (oldest deadline 2024-05-21T22:08:06.589360+0000) 2024-05-21T22:08:09.084 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:08 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:08.832+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:28.956267+0000 front 2024-05-21T22:07:28.956225+0000 (oldest deadline 2024-05-21T22:07:55.356623+0000) 2024-05-21T22:08:09.084 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:08 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:08.832+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:38.857667+0000 front 2024-05-21T22:07:38.857566+0000 (oldest deadline 2024-05-21T22:08:04.757680+0000) 2024-05-21T22:08:09.372 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 21 22:08:09 smithi184 systemd[1]: Stopping Ceph osd.5 for c8c7a2e6-17ba-11ef-bc9a-c7b262605968... 2024-05-21T22:08:09.372 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 21 22:08:09 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-5[48868]: 2024-05-21T22:08:09.249+0000 7f58eb9f2700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:29.310154+0000 front 2024-05-21T22:07:29.309983+0000 (oldest deadline 2024-05-21T22:07:54.609869+0000) 2024-05-21T22:08:09.373 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 21 22:08:09 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-5[48868]: 2024-05-21T22:08:09.249+0000 7f58eb9f2700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:38.711280+0000 front 2024-05-21T22:07:38.711222+0000 (oldest deadline 2024-05-21T22:08:06.311633+0000) 2024-05-21T22:08:09.373 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 21 22:08:09 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-5[48868]: 2024-05-21T22:08:09.372+0000 7f58f3406700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T22:08:09.373 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 21 22:08:09 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-5[48868]: 2024-05-21T22:08:09.372+0000 7f58f3406700 -1 osd.5 94 *** Got signal Terminated *** 2024-05-21T22:08:09.373 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 21 22:08:09 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-5[48868]: 2024-05-21T22:08:09.372+0000 7f58f3406700 -1 osd.5 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-21T22:08:09.610 INFO:journalctl@ceph.osd.6.smithi151.stdout:May 21 22:08:09 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-6[57087]: 2024-05-21T22:08:09.274+0000 7f0909400700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:31.896170+0000 front 2024-05-21T22:07:31.896361+0000 (oldest deadline 2024-05-21T22:07:57.795636+0000) 2024-05-21T22:08:09.610 INFO:journalctl@ceph.osd.6.smithi151.stdout:May 21 22:08:09 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-6[57087]: 2024-05-21T22:08:09.274+0000 7f0909400700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:38.298380+0000 front 2024-05-21T22:07:38.298169+0000 (oldest deadline 2024-05-21T22:08:04.198343+0000) 2024-05-21T22:08:09.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:09 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:09.609+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:27.976452+0000 front 2024-05-21T22:07:27.976969+0000 (oldest deadline 2024-05-21T22:07:53.876086+0000) 2024-05-21T22:08:09.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:09 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:09.609+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:39.580127+0000 front 2024-05-21T22:07:39.580305+0000 (oldest deadline 2024-05-21T22:08:04.280362+0000) 2024-05-21T22:08:10.036 INFO:journalctl@ceph.osd.7.smithi164.stdout:May 21 22:08:09 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-7[54180]: 2024-05-21T22:08:09.827+0000 7f8d4da11700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:37.480876+0000 front 2024-05-21T22:07:37.480856+0000 (oldest deadline 2024-05-21T22:08:02.181059+0000) 2024-05-21T22:08:10.036 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:09 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:09.619+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:30.508363+0000 front 2024-05-21T22:07:30.508149+0000 (oldest deadline 2024-05-21T22:07:52.808006+0000) 2024-05-21T22:08:10.036 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:09 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:09.619+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:41.009631+0000 front 2024-05-21T22:07:41.009565+0000 (oldest deadline 2024-05-21T22:08:05.709714+0000) 2024-05-21T22:08:10.236 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:09 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:09.861+0000 7f452bd16700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:40.689332+0000 front 2024-05-21T22:07:40.689184+0000 (oldest deadline 2024-05-21T22:08:06.589360+0000) 2024-05-21T22:08:10.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:09 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:09.845+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:28.956267+0000 front 2024-05-21T22:07:28.956225+0000 (oldest deadline 2024-05-21T22:07:55.356623+0000) 2024-05-21T22:08:10.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:09 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:09.845+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:38.857667+0000 front 2024-05-21T22:07:38.857566+0000 (oldest deadline 2024-05-21T22:08:04.757680+0000) 2024-05-21T22:08:10.640 INFO:journalctl@ceph.osd.6.smithi151.stdout:May 21 22:08:10 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-6[57087]: 2024-05-21T22:08:10.295+0000 7f0909400700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:31.896170+0000 front 2024-05-21T22:07:31.896361+0000 (oldest deadline 2024-05-21T22:07:57.795636+0000) 2024-05-21T22:08:10.640 INFO:journalctl@ceph.osd.6.smithi151.stdout:May 21 22:08:10 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-6[57087]: 2024-05-21T22:08:10.295+0000 7f0909400700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:38.298380+0000 front 2024-05-21T22:07:38.298169+0000 (oldest deadline 2024-05-21T22:08:04.198343+0000) 2024-05-21T22:08:10.736 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 21 22:08:10 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-5[48868]: 2024-05-21T22:08:10.270+0000 7f58eb9f2700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:29.310154+0000 front 2024-05-21T22:07:29.309983+0000 (oldest deadline 2024-05-21T22:07:54.609869+0000) 2024-05-21T22:08:10.736 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 21 22:08:10 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-5[48868]: 2024-05-21T22:08:10.270+0000 7f58eb9f2700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:38.711280+0000 front 2024-05-21T22:07:38.711222+0000 (oldest deadline 2024-05-21T22:08:06.311633+0000) 2024-05-21T22:08:10.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:10 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:10.639+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:27.976452+0000 front 2024-05-21T22:07:27.976969+0000 (oldest deadline 2024-05-21T22:07:53.876086+0000) 2024-05-21T22:08:10.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:10 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:10.639+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:39.580127+0000 front 2024-05-21T22:07:39.580305+0000 (oldest deadline 2024-05-21T22:08:04.280362+0000) 2024-05-21T22:08:11.036 INFO:journalctl@ceph.osd.7.smithi164.stdout:May 21 22:08:10 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-7[54180]: 2024-05-21T22:08:10.867+0000 7f8d4da11700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:37.480876+0000 front 2024-05-21T22:07:37.480856+0000 (oldest deadline 2024-05-21T22:08:02.181059+0000) 2024-05-21T22:08:11.036 INFO:journalctl@ceph.osd.7.smithi164.stdout:May 21 22:08:10 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-7[54180]: 2024-05-21T22:08:10.867+0000 7f8d4da11700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:49.083252+0000 front 2024-05-21T22:07:49.083231+0000 (oldest deadline 2024-05-21T22:08:10.183363+0000) 2024-05-21T22:08:11.036 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:10 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:10.639+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:30.508363+0000 front 2024-05-21T22:07:30.508149+0000 (oldest deadline 2024-05-21T22:07:52.808006+0000) 2024-05-21T22:08:11.036 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:10 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:10.639+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:41.009631+0000 front 2024-05-21T22:07:41.009565+0000 (oldest deadline 2024-05-21T22:08:05.709714+0000) 2024-05-21T22:08:11.236 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:10 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:10.814+0000 7f452bd16700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:40.689332+0000 front 2024-05-21T22:07:40.689184+0000 (oldest deadline 2024-05-21T22:08:06.589360+0000) 2024-05-21T22:08:11.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:10 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:10.835+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:28.956267+0000 front 2024-05-21T22:07:28.956225+0000 (oldest deadline 2024-05-21T22:07:55.356623+0000) 2024-05-21T22:08:11.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:10 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:10.835+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:38.857667+0000 front 2024-05-21T22:07:38.857566+0000 (oldest deadline 2024-05-21T22:08:04.757680+0000) 2024-05-21T22:08:11.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:10 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:10.835+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:44.758039+0000 front 2024-05-21T22:07:44.758082+0000 (oldest deadline 2024-05-21T22:08:10.658241+0000) 2024-05-21T22:08:11.687 INFO:journalctl@ceph.osd.6.smithi151.stdout:May 21 22:08:11 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-6[57087]: 2024-05-21T22:08:11.282+0000 7f0909400700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:31.896170+0000 front 2024-05-21T22:07:31.896361+0000 (oldest deadline 2024-05-21T22:07:57.795636+0000) 2024-05-21T22:08:11.687 INFO:journalctl@ceph.osd.6.smithi151.stdout:May 21 22:08:11 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-6[57087]: 2024-05-21T22:08:11.282+0000 7f0909400700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:38.298380+0000 front 2024-05-21T22:07:38.298169+0000 (oldest deadline 2024-05-21T22:08:04.198343+0000) 2024-05-21T22:08:11.688 INFO:journalctl@ceph.osd.6.smithi151.stdout:May 21 22:08:11 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-6[57087]: 2024-05-21T22:08:11.282+0000 7f0909400700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:48.702219+0000 front 2024-05-21T22:07:48.702298+0000 (oldest deadline 2024-05-21T22:08:10.402181+0000) 2024-05-21T22:08:11.736 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 21 22:08:11 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-5[48868]: 2024-05-21T22:08:11.300+0000 7f58eb9f2700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:29.310154+0000 front 2024-05-21T22:07:29.309983+0000 (oldest deadline 2024-05-21T22:07:54.609869+0000) 2024-05-21T22:08:11.736 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 21 22:08:11 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-5[48868]: 2024-05-21T22:08:11.300+0000 7f58eb9f2700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:38.711280+0000 front 2024-05-21T22:07:38.711222+0000 (oldest deadline 2024-05-21T22:08:06.311633+0000) 2024-05-21T22:08:11.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:11 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:11.686+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:27.976452+0000 front 2024-05-21T22:07:27.976969+0000 (oldest deadline 2024-05-21T22:07:53.876086+0000) 2024-05-21T22:08:11.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:11 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:11.686+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:39.580127+0000 front 2024-05-21T22:07:39.580305+0000 (oldest deadline 2024-05-21T22:08:04.280362+0000) 2024-05-21T22:08:11.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:11 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:11.686+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:47.782149+0000 front 2024-05-21T22:07:47.781979+0000 (oldest deadline 2024-05-21T22:08:10.682108+0000) 2024-05-21T22:08:12.036 INFO:journalctl@ceph.osd.7.smithi164.stdout:May 21 22:08:11 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-7[54180]: 2024-05-21T22:08:11.865+0000 7f8d4da11700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:37.480876+0000 front 2024-05-21T22:07:37.480856+0000 (oldest deadline 2024-05-21T22:08:02.181059+0000) 2024-05-21T22:08:12.036 INFO:journalctl@ceph.osd.7.smithi164.stdout:May 21 22:08:11 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-7[54180]: 2024-05-21T22:08:11.865+0000 7f8d4da11700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:49.083252+0000 front 2024-05-21T22:07:49.083231+0000 (oldest deadline 2024-05-21T22:08:10.183363+0000) 2024-05-21T22:08:12.036 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:11 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:11.622+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:30.508363+0000 front 2024-05-21T22:07:30.508149+0000 (oldest deadline 2024-05-21T22:07:52.808006+0000) 2024-05-21T22:08:12.036 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:11 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:11.622+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:41.009631+0000 front 2024-05-21T22:07:41.009565+0000 (oldest deadline 2024-05-21T22:08:05.709714+0000) 2024-05-21T22:08:12.236 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:11 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:11.795+0000 7f452bd16700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:40.689332+0000 front 2024-05-21T22:07:40.689184+0000 (oldest deadline 2024-05-21T22:08:06.589360+0000) 2024-05-21T22:08:12.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:11 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:11.851+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:28.956267+0000 front 2024-05-21T22:07:28.956225+0000 (oldest deadline 2024-05-21T22:07:55.356623+0000) 2024-05-21T22:08:12.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:11 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:11.851+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:38.857667+0000 front 2024-05-21T22:07:38.857566+0000 (oldest deadline 2024-05-21T22:08:04.757680+0000) 2024-05-21T22:08:12.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:11 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:11.851+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:44.758039+0000 front 2024-05-21T22:07:44.758082+0000 (oldest deadline 2024-05-21T22:08:10.658241+0000) 2024-05-21T22:08:12.677 INFO:journalctl@ceph.osd.6.smithi151.stdout:May 21 22:08:12 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-6[57087]: 2024-05-21T22:08:12.248+0000 7f0909400700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:31.896170+0000 front 2024-05-21T22:07:31.896361+0000 (oldest deadline 2024-05-21T22:07:57.795636+0000) 2024-05-21T22:08:12.677 INFO:journalctl@ceph.osd.6.smithi151.stdout:May 21 22:08:12 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-6[57087]: 2024-05-21T22:08:12.248+0000 7f0909400700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:38.298380+0000 front 2024-05-21T22:07:38.298169+0000 (oldest deadline 2024-05-21T22:08:04.198343+0000) 2024-05-21T22:08:12.677 INFO:journalctl@ceph.osd.6.smithi151.stdout:May 21 22:08:12 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-6[57087]: 2024-05-21T22:08:12.248+0000 7f0909400700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:48.702219+0000 front 2024-05-21T22:07:48.702298+0000 (oldest deadline 2024-05-21T22:08:10.402181+0000) 2024-05-21T22:08:12.736 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 21 22:08:12 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-5[48868]: 2024-05-21T22:08:12.284+0000 7f58eb9f2700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:29.310154+0000 front 2024-05-21T22:07:29.309983+0000 (oldest deadline 2024-05-21T22:07:54.609869+0000) 2024-05-21T22:08:12.736 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 21 22:08:12 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-5[48868]: 2024-05-21T22:08:12.284+0000 7f58eb9f2700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:38.711280+0000 front 2024-05-21T22:07:38.711222+0000 (oldest deadline 2024-05-21T22:08:06.311633+0000) 2024-05-21T22:08:12.736 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 21 22:08:12 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-5[48868]: 2024-05-21T22:08:12.284+0000 7f58eb9f2700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:46.812763+0000 front 2024-05-21T22:07:46.812699+0000 (oldest deadline 2024-05-21T22:08:12.112861+0000) 2024-05-21T22:08:12.937 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:12 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:12.660+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:30.508363+0000 front 2024-05-21T22:07:30.508149+0000 (oldest deadline 2024-05-21T22:07:52.808006+0000) 2024-05-21T22:08:12.937 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:12 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:12.660+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:41.009631+0000 front 2024-05-21T22:07:41.009565+0000 (oldest deadline 2024-05-21T22:08:05.709714+0000) 2024-05-21T22:08:12.938 INFO:journalctl@ceph.osd.7.smithi164.stdout:May 21 22:08:12 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-7[54180]: 2024-05-21T22:08:12.835+0000 7f8d4da11700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:37.480876+0000 front 2024-05-21T22:07:37.480856+0000 (oldest deadline 2024-05-21T22:08:02.181059+0000) 2024-05-21T22:08:12.938 INFO:journalctl@ceph.osd.7.smithi164.stdout:May 21 22:08:12 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-7[54180]: 2024-05-21T22:08:12.835+0000 7f8d4da11700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:49.083252+0000 front 2024-05-21T22:07:49.083231+0000 (oldest deadline 2024-05-21T22:08:10.183363+0000) 2024-05-21T22:08:12.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:12 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:12.676+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:27.976452+0000 front 2024-05-21T22:07:27.976969+0000 (oldest deadline 2024-05-21T22:07:53.876086+0000) 2024-05-21T22:08:12.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:12 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:12.676+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:39.580127+0000 front 2024-05-21T22:07:39.580305+0000 (oldest deadline 2024-05-21T22:08:04.280362+0000) 2024-05-21T22:08:12.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:12 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:12.676+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:47.782149+0000 front 2024-05-21T22:07:47.781979+0000 (oldest deadline 2024-05-21T22:08:10.682108+0000) 2024-05-21T22:08:13.236 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:12 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:12.751+0000 7f452bd16700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:40.689332+0000 front 2024-05-21T22:07:40.689184+0000 (oldest deadline 2024-05-21T22:08:06.589360+0000) 2024-05-21T22:08:13.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:12 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:12.850+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:28.956267+0000 front 2024-05-21T22:07:28.956225+0000 (oldest deadline 2024-05-21T22:07:55.356623+0000) 2024-05-21T22:08:13.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:12 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:12.850+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:38.857667+0000 front 2024-05-21T22:07:38.857566+0000 (oldest deadline 2024-05-21T22:08:04.757680+0000) 2024-05-21T22:08:13.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:12 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:12.850+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:44.758039+0000 front 2024-05-21T22:07:44.758082+0000 (oldest deadline 2024-05-21T22:08:10.658241+0000) 2024-05-21T22:08:13.695 INFO:journalctl@ceph.osd.6.smithi151.stdout:May 21 22:08:13 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-6[57087]: 2024-05-21T22:08:13.252+0000 7f0909400700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:31.896170+0000 front 2024-05-21T22:07:31.896361+0000 (oldest deadline 2024-05-21T22:07:57.795636+0000) 2024-05-21T22:08:13.695 INFO:journalctl@ceph.osd.6.smithi151.stdout:May 21 22:08:13 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-6[57087]: 2024-05-21T22:08:13.252+0000 7f0909400700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:38.298380+0000 front 2024-05-21T22:07:38.298169+0000 (oldest deadline 2024-05-21T22:08:04.198343+0000) 2024-05-21T22:08:13.695 INFO:journalctl@ceph.osd.6.smithi151.stdout:May 21 22:08:13 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-6[57087]: 2024-05-21T22:08:13.252+0000 7f0909400700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:48.702219+0000 front 2024-05-21T22:07:48.702298+0000 (oldest deadline 2024-05-21T22:08:10.402181+0000) 2024-05-21T22:08:13.736 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 21 22:08:13 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-5[48868]: 2024-05-21T22:08:13.313+0000 7f58eb9f2700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:29.310154+0000 front 2024-05-21T22:07:29.309983+0000 (oldest deadline 2024-05-21T22:07:54.609869+0000) 2024-05-21T22:08:13.736 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 21 22:08:13 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-5[48868]: 2024-05-21T22:08:13.313+0000 7f58eb9f2700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:38.711280+0000 front 2024-05-21T22:07:38.711222+0000 (oldest deadline 2024-05-21T22:08:06.311633+0000) 2024-05-21T22:08:13.736 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 21 22:08:13 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-5[48868]: 2024-05-21T22:08:13.313+0000 7f58eb9f2700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:46.812763+0000 front 2024-05-21T22:07:46.812699+0000 (oldest deadline 2024-05-21T22:08:12.112861+0000) 2024-05-21T22:08:13.950 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:13 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:13.678+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:30.508363+0000 front 2024-05-21T22:07:30.508149+0000 (oldest deadline 2024-05-21T22:07:52.808006+0000) 2024-05-21T22:08:13.951 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:13 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:13.678+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:41.009631+0000 front 2024-05-21T22:07:41.009565+0000 (oldest deadline 2024-05-21T22:08:05.709714+0000) 2024-05-21T22:08:13.951 INFO:journalctl@ceph.osd.7.smithi164.stdout:May 21 22:08:13 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-7[54180]: 2024-05-21T22:08:13.841+0000 7f8d4da11700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:37.480876+0000 front 2024-05-21T22:07:37.480856+0000 (oldest deadline 2024-05-21T22:08:02.181059+0000) 2024-05-21T22:08:13.951 INFO:journalctl@ceph.osd.7.smithi164.stdout:May 21 22:08:13 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-7[54180]: 2024-05-21T22:08:13.841+0000 7f8d4da11700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:49.083252+0000 front 2024-05-21T22:07:49.083231+0000 (oldest deadline 2024-05-21T22:08:10.183363+0000) 2024-05-21T22:08:13.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:13 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:13.694+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:27.976452+0000 front 2024-05-21T22:07:27.976969+0000 (oldest deadline 2024-05-21T22:07:53.876086+0000) 2024-05-21T22:08:13.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:13 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:13.694+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:39.580127+0000 front 2024-05-21T22:07:39.580305+0000 (oldest deadline 2024-05-21T22:08:04.280362+0000) 2024-05-21T22:08:13.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:13 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:13.694+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:47.782149+0000 front 2024-05-21T22:07:47.781979+0000 (oldest deadline 2024-05-21T22:08:10.682108+0000) 2024-05-21T22:08:14.236 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:13 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:13.741+0000 7f452bd16700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:40.689332+0000 front 2024-05-21T22:07:40.689184+0000 (oldest deadline 2024-05-21T22:08:06.589360+0000) 2024-05-21T22:08:14.236 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:13 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:13.741+0000 7f452bd16700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:48.890238+0000 front 2024-05-21T22:07:48.890266+0000 (oldest deadline 2024-05-21T22:08:12.990394+0000) 2024-05-21T22:08:14.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:13 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:13.897+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:28.956267+0000 front 2024-05-21T22:07:28.956225+0000 (oldest deadline 2024-05-21T22:07:55.356623+0000) 2024-05-21T22:08:14.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:13 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:13.897+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:38.857667+0000 front 2024-05-21T22:07:38.857566+0000 (oldest deadline 2024-05-21T22:08:04.757680+0000) 2024-05-21T22:08:14.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:13 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:13.897+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:44.758039+0000 front 2024-05-21T22:07:44.758082+0000 (oldest deadline 2024-05-21T22:08:10.658241+0000) 2024-05-21T22:08:14.700 INFO:journalctl@ceph.osd.6.smithi151.stdout:May 21 22:08:14 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-6[57087]: 2024-05-21T22:08:14.261+0000 7f0909400700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:31.896170+0000 front 2024-05-21T22:07:31.896361+0000 (oldest deadline 2024-05-21T22:07:57.795636+0000) 2024-05-21T22:08:14.700 INFO:journalctl@ceph.osd.6.smithi151.stdout:May 21 22:08:14 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-6[57087]: 2024-05-21T22:08:14.261+0000 7f0909400700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:38.298380+0000 front 2024-05-21T22:07:38.298169+0000 (oldest deadline 2024-05-21T22:08:04.198343+0000) 2024-05-21T22:08:14.700 INFO:journalctl@ceph.osd.6.smithi151.stdout:May 21 22:08:14 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-6[57087]: 2024-05-21T22:08:14.261+0000 7f0909400700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:48.702219+0000 front 2024-05-21T22:07:48.702298+0000 (oldest deadline 2024-05-21T22:08:10.402181+0000) 2024-05-21T22:08:14.713 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 21 22:08:14 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-5[48868]: 2024-05-21T22:08:14.358+0000 7f58eb9f2700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:29.310154+0000 front 2024-05-21T22:07:29.309983+0000 (oldest deadline 2024-05-21T22:07:54.609869+0000) 2024-05-21T22:08:14.713 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 21 22:08:14 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-5[48868]: 2024-05-21T22:08:14.358+0000 7f58eb9f2700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:38.711280+0000 front 2024-05-21T22:07:38.711222+0000 (oldest deadline 2024-05-21T22:08:06.311633+0000) 2024-05-21T22:08:14.714 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 21 22:08:14 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-5[48868]: 2024-05-21T22:08:14.358+0000 7f58eb9f2700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:46.812763+0000 front 2024-05-21T22:07:46.812699+0000 (oldest deadline 2024-05-21T22:08:12.112861+0000) 2024-05-21T22:08:14.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:14 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:14.699+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:27.976452+0000 front 2024-05-21T22:07:27.976969+0000 (oldest deadline 2024-05-21T22:07:53.876086+0000) 2024-05-21T22:08:14.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:14 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:14.699+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:39.580127+0000 front 2024-05-21T22:07:39.580305+0000 (oldest deadline 2024-05-21T22:08:04.280362+0000) 2024-05-21T22:08:14.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:14 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:14.699+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:47.782149+0000 front 2024-05-21T22:07:47.781979+0000 (oldest deadline 2024-05-21T22:08:10.682108+0000) 2024-05-21T22:08:14.986 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 21 22:08:14 smithi184 podman[74687]: 2024-05-21 22:08:14.827143216 +0000 UTC m=+5.578453011 container died 44a6b9505c018ba057d07da527746f6a075156ddd59e7fe93054aaef65d8b861 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, maintainer=Guillaume Abrioux ) 2024-05-21T22:08:14.986 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:14 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:14.713+0000 7f452bd16700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:40.689332+0000 front 2024-05-21T22:07:40.689184+0000 (oldest deadline 2024-05-21T22:08:06.589360+0000) 2024-05-21T22:08:14.986 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:14 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:14.713+0000 7f452bd16700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:48.890238+0000 front 2024-05-21T22:07:48.890266+0000 (oldest deadline 2024-05-21T22:08:12.990394+0000) 2024-05-21T22:08:14.986 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:14 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:14.873+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:28.956267+0000 front 2024-05-21T22:07:28.956225+0000 (oldest deadline 2024-05-21T22:07:55.356623+0000) 2024-05-21T22:08:14.986 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:14 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:14.873+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:38.857667+0000 front 2024-05-21T22:07:38.857566+0000 (oldest deadline 2024-05-21T22:08:04.757680+0000) 2024-05-21T22:08:14.986 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:14 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:14.873+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:44.758039+0000 front 2024-05-21T22:07:44.758082+0000 (oldest deadline 2024-05-21T22:08:10.658241+0000) 2024-05-21T22:08:15.036 INFO:journalctl@ceph.osd.7.smithi164.stdout:May 21 22:08:14 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-7[54180]: 2024-05-21T22:08:14.840+0000 7f8d4da11700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:37.480876+0000 front 2024-05-21T22:07:37.480856+0000 (oldest deadline 2024-05-21T22:08:02.181059+0000) 2024-05-21T22:08:15.036 INFO:journalctl@ceph.osd.7.smithi164.stdout:May 21 22:08:14 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-7[54180]: 2024-05-21T22:08:14.840+0000 7f8d4da11700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:49.083252+0000 front 2024-05-21T22:07:49.083231+0000 (oldest deadline 2024-05-21T22:08:10.183363+0000) 2024-05-21T22:08:15.036 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:14 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:14.710+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:30.508363+0000 front 2024-05-21T22:07:30.508149+0000 (oldest deadline 2024-05-21T22:07:52.808006+0000) 2024-05-21T22:08:15.036 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:14 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:14.710+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:41.009631+0000 front 2024-05-21T22:07:41.009565+0000 (oldest deadline 2024-05-21T22:08:05.709714+0000) 2024-05-21T22:08:15.036 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:14 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:14.710+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:49.011674+0000 front 2024-05-21T22:07:49.011868+0000 (oldest deadline 2024-05-21T22:08:13.711629+0000) 2024-05-21T22:08:15.372 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 21 22:08:15 smithi184 podman[74687]: 2024-05-21 22:08:15.163496514 +0000 UTC m=+5.914806309 container cleanup 44a6b9505c018ba057d07da527746f6a075156ddd59e7fe93054aaef65d8b861 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True) 2024-05-21T22:08:15.373 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 21 22:08:15 smithi184 bash[74687]: ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-5 2024-05-21T22:08:15.671 INFO:journalctl@ceph.osd.6.smithi151.stdout:May 21 22:08:15 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-6[57087]: 2024-05-21T22:08:15.232+0000 7f0909400700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:31.896170+0000 front 2024-05-21T22:07:31.896361+0000 (oldest deadline 2024-05-21T22:07:57.795636+0000) 2024-05-21T22:08:15.671 INFO:journalctl@ceph.osd.6.smithi151.stdout:May 21 22:08:15 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-6[57087]: 2024-05-21T22:08:15.232+0000 7f0909400700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:38.298380+0000 front 2024-05-21T22:07:38.298169+0000 (oldest deadline 2024-05-21T22:08:04.198343+0000) 2024-05-21T22:08:15.671 INFO:journalctl@ceph.osd.6.smithi151.stdout:May 21 22:08:15 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-6[57087]: 2024-05-21T22:08:15.232+0000 7f0909400700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:48.702219+0000 front 2024-05-21T22:07:48.702298+0000 (oldest deadline 2024-05-21T22:08:10.402181+0000) 2024-05-21T22:08:15.730 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 21 22:08:15 smithi184 podman[74703]: 2024-05-21 22:08:15.373177705 +0000 UTC m=+0.540218387 container remove 44a6b9505c018ba057d07da527746f6a075156ddd59e7fe93054aaef65d8b861 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-05-21T22:08:15.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:15 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:15.670+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:27.976452+0000 front 2024-05-21T22:07:27.976969+0000 (oldest deadline 2024-05-21T22:07:53.876086+0000) 2024-05-21T22:08:15.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:15 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:15.670+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:39.580127+0000 front 2024-05-21T22:07:39.580305+0000 (oldest deadline 2024-05-21T22:08:04.280362+0000) 2024-05-21T22:08:15.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:15 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:15.670+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:47.782149+0000 front 2024-05-21T22:07:47.781979+0000 (oldest deadline 2024-05-21T22:08:10.682108+0000) 2024-05-21T22:08:15.985 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:15 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:15.730+0000 7f452bd16700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:40.689332+0000 front 2024-05-21T22:07:40.689184+0000 (oldest deadline 2024-05-21T22:08:06.589360+0000) 2024-05-21T22:08:15.986 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:15 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:15.730+0000 7f452bd16700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:48.890238+0000 front 2024-05-21T22:07:48.890266+0000 (oldest deadline 2024-05-21T22:08:12.990394+0000) 2024-05-21T22:08:15.986 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 21 22:08:15 smithi184 podman[74791]: 2024-05-21 22:08:15.681751128 +0000 UTC m=+0.017239178 image pull d6bc8f31cbc1a7165ca248fe9ba0b06ea14b30f46e1ea33ced1a5238e247af1f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28 2024-05-21T22:08:15.986 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 21 22:08:15 smithi184 podman[74791]: 2024-05-21 22:08:15.79068013 +0000 UTC m=+0.126168169 container create 961e70660409ac088737b3163683144c23ac4df2cc08081ccd69f46aa2c56087 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, ceph=True) 2024-05-21T22:08:15.986 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:15 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:15.915+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:28.956267+0000 front 2024-05-21T22:07:28.956225+0000 (oldest deadline 2024-05-21T22:07:55.356623+0000) 2024-05-21T22:08:15.986 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:15 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:15.915+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:38.857667+0000 front 2024-05-21T22:07:38.857566+0000 (oldest deadline 2024-05-21T22:08:04.757680+0000) 2024-05-21T22:08:15.986 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:15 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:15.915+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:44.758039+0000 front 2024-05-21T22:07:44.758082+0000 (oldest deadline 2024-05-21T22:08:10.658241+0000) 2024-05-21T22:08:16.036 INFO:journalctl@ceph.osd.7.smithi164.stdout:May 21 22:08:15 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-7[54180]: 2024-05-21T22:08:15.859+0000 7f8d4da11700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:37.480876+0000 front 2024-05-21T22:07:37.480856+0000 (oldest deadline 2024-05-21T22:08:02.181059+0000) 2024-05-21T22:08:16.036 INFO:journalctl@ceph.osd.7.smithi164.stdout:May 21 22:08:15 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-7[54180]: 2024-05-21T22:08:15.859+0000 7f8d4da11700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:49.083252+0000 front 2024-05-21T22:07:49.083231+0000 (oldest deadline 2024-05-21T22:08:10.183363+0000) 2024-05-21T22:08:16.036 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:15 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:15.751+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:30.508363+0000 front 2024-05-21T22:07:30.508149+0000 (oldest deadline 2024-05-21T22:07:52.808006+0000) 2024-05-21T22:08:16.036 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:15 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:15.751+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:41.009631+0000 front 2024-05-21T22:07:41.009565+0000 (oldest deadline 2024-05-21T22:08:05.709714+0000) 2024-05-21T22:08:16.037 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:15 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:15.751+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:49.011674+0000 front 2024-05-21T22:07:49.011868+0000 (oldest deadline 2024-05-21T22:08:13.711629+0000) 2024-05-21T22:08:16.367 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 21 22:08:16 smithi184 podman[74791]: 2024-05-21 22:08:16.068414165 +0000 UTC m=+0.403902215 container init 961e70660409ac088737b3163683144c23ac4df2cc08081ccd69f46aa2c56087 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-05-21T22:08:16.367 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 21 22:08:16 smithi184 podman[74791]: 2024-05-21 22:08:16.071453862 +0000 UTC m=+0.406941903 container start 961e70660409ac088737b3163683144c23ac4df2cc08081ccd69f46aa2c56087 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-5-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240514, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-05-21T22:08:16.367 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 21 22:08:16 smithi184 podman[74791]: 2024-05-21 22:08:16.143121597 +0000 UTC m=+0.478609641 container attach 961e70660409ac088737b3163683144c23ac4df2cc08081ccd69f46aa2c56087 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-5-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-05-21T22:08:16.646 INFO:journalctl@ceph.osd.5.smithi184.stdout:May 21 22:08:16 smithi184 podman[74791]: 2024-05-21 22:08:16.367456237 +0000 UTC m=+0.702944283 container died 961e70660409ac088737b3163683144c23ac4df2cc08081ccd69f46aa2c56087 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-05-21T22:08:16.703 INFO:journalctl@ceph.osd.6.smithi151.stdout:May 21 22:08:16 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-6[57087]: 2024-05-21T22:08:16.239+0000 7f0909400700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:31.896170+0000 front 2024-05-21T22:07:31.896361+0000 (oldest deadline 2024-05-21T22:07:57.795636+0000) 2024-05-21T22:08:16.703 INFO:journalctl@ceph.osd.6.smithi151.stdout:May 21 22:08:16 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-6[57087]: 2024-05-21T22:08:16.239+0000 7f0909400700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:38.298380+0000 front 2024-05-21T22:07:38.298169+0000 (oldest deadline 2024-05-21T22:08:04.198343+0000) 2024-05-21T22:08:16.703 INFO:journalctl@ceph.osd.6.smithi151.stdout:May 21 22:08:16 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-6[57087]: 2024-05-21T22:08:16.239+0000 7f0909400700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:48.702219+0000 front 2024-05-21T22:07:48.702298+0000 (oldest deadline 2024-05-21T22:08:10.402181+0000) 2024-05-21T22:08:16.929 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:16 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:16.727+0000 7f452bd16700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:40.689332+0000 front 2024-05-21T22:07:40.689184+0000 (oldest deadline 2024-05-21T22:08:06.589360+0000) 2024-05-21T22:08:16.930 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:16 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:16.727+0000 7f452bd16700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:48.890238+0000 front 2024-05-21T22:07:48.890266+0000 (oldest deadline 2024-05-21T22:08:12.990394+0000) 2024-05-21T22:08:16.930 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:16 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:16.889+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:28.956267+0000 front 2024-05-21T22:07:28.956225+0000 (oldest deadline 2024-05-21T22:07:55.356623+0000) 2024-05-21T22:08:16.930 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:16 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:16.889+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:38.857667+0000 front 2024-05-21T22:07:38.857566+0000 (oldest deadline 2024-05-21T22:08:04.757680+0000) 2024-05-21T22:08:16.930 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:16 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:16.889+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:44.758039+0000 front 2024-05-21T22:07:44.758082+0000 (oldest deadline 2024-05-21T22:08:10.658241+0000) 2024-05-21T22:08:16.964 DEBUG:teuthology.orchestra.run.smithi184:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@osd.5.service' 2024-05-21T22:08:17.045 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T22:08:17.045 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-05-21T22:08:17.045 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-05-21T22:08:17.045 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@osd.6 2024-05-21T22:08:17.123 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:16 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:16.717+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:27.976452+0000 front 2024-05-21T22:07:27.976969+0000 (oldest deadline 2024-05-21T22:07:53.876086+0000) 2024-05-21T22:08:17.124 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:16 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:16.717+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:39.580127+0000 front 2024-05-21T22:07:39.580305+0000 (oldest deadline 2024-05-21T22:08:04.280362+0000) 2024-05-21T22:08:17.124 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:16 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:16.717+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:47.782149+0000 front 2024-05-21T22:07:47.781979+0000 (oldest deadline 2024-05-21T22:08:10.682108+0000) 2024-05-21T22:08:17.286 INFO:journalctl@ceph.osd.7.smithi164.stdout:May 21 22:08:16 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-7[54180]: 2024-05-21T22:08:16.813+0000 7f8d4da11700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:37.480876+0000 front 2024-05-21T22:07:37.480856+0000 (oldest deadline 2024-05-21T22:08:02.181059+0000) 2024-05-21T22:08:17.286 INFO:journalctl@ceph.osd.7.smithi164.stdout:May 21 22:08:16 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-7[54180]: 2024-05-21T22:08:16.813+0000 7f8d4da11700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:49.083252+0000 front 2024-05-21T22:07:49.083231+0000 (oldest deadline 2024-05-21T22:08:10.183363+0000) 2024-05-21T22:08:17.286 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:16 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:16.799+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:30.508363+0000 front 2024-05-21T22:07:30.508149+0000 (oldest deadline 2024-05-21T22:07:52.808006+0000) 2024-05-21T22:08:17.286 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:16 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:16.799+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:41.009631+0000 front 2024-05-21T22:07:41.009565+0000 (oldest deadline 2024-05-21T22:08:05.709714+0000) 2024-05-21T22:08:17.287 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:16 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:16.799+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:49.011674+0000 front 2024-05-21T22:07:49.011868+0000 (oldest deadline 2024-05-21T22:08:13.711629+0000) 2024-05-21T22:08:17.453 INFO:journalctl@ceph.osd.6.smithi151.stdout:May 21 22:08:17 smithi151 systemd[1]: Stopping Ceph osd.6 for c8c7a2e6-17ba-11ef-bc9a-c7b262605968... 2024-05-21T22:08:17.453 INFO:journalctl@ceph.osd.6.smithi151.stdout:May 21 22:08:17 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-6[57087]: 2024-05-21T22:08:17.263+0000 7f0909400700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:31.896170+0000 front 2024-05-21T22:07:31.896361+0000 (oldest deadline 2024-05-21T22:07:57.795636+0000) 2024-05-21T22:08:17.453 INFO:journalctl@ceph.osd.6.smithi151.stdout:May 21 22:08:17 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-6[57087]: 2024-05-21T22:08:17.263+0000 7f0909400700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:38.298380+0000 front 2024-05-21T22:07:38.298169+0000 (oldest deadline 2024-05-21T22:08:04.198343+0000) 2024-05-21T22:08:17.453 INFO:journalctl@ceph.osd.6.smithi151.stdout:May 21 22:08:17 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-6[57087]: 2024-05-21T22:08:17.263+0000 7f0909400700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:48.702219+0000 front 2024-05-21T22:07:48.702298+0000 (oldest deadline 2024-05-21T22:08:10.402181+0000) 2024-05-21T22:08:17.953 INFO:journalctl@ceph.osd.6.smithi151.stdout:May 21 22:08:17 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-6[57087]: 2024-05-21T22:08:17.634+0000 7f090dc00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T22:08:17.953 INFO:journalctl@ceph.osd.6.smithi151.stdout:May 21 22:08:17 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-6[57087]: 2024-05-21T22:08:17.634+0000 7f090dc00700 -1 osd.6 94 *** Got signal Terminated *** 2024-05-21T22:08:17.953 INFO:journalctl@ceph.osd.6.smithi151.stdout:May 21 22:08:17 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-6[57087]: 2024-05-21T22:08:17.634+0000 7f090dc00700 -1 osd.6 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-21T22:08:17.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:17 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:17.699+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:27.976452+0000 front 2024-05-21T22:07:27.976969+0000 (oldest deadline 2024-05-21T22:07:53.876086+0000) 2024-05-21T22:08:17.954 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:17 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:17.699+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:39.580127+0000 front 2024-05-21T22:07:39.580305+0000 (oldest deadline 2024-05-21T22:08:04.280362+0000) 2024-05-21T22:08:17.954 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:17 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:17.699+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:47.782149+0000 front 2024-05-21T22:07:47.781979+0000 (oldest deadline 2024-05-21T22:08:10.682108+0000) 2024-05-21T22:08:17.986 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:17 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:17.704+0000 7f452bd16700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:40.689332+0000 front 2024-05-21T22:07:40.689184+0000 (oldest deadline 2024-05-21T22:08:06.589360+0000) 2024-05-21T22:08:17.986 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:17 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:17.704+0000 7f452bd16700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:48.890238+0000 front 2024-05-21T22:07:48.890266+0000 (oldest deadline 2024-05-21T22:08:12.990394+0000) 2024-05-21T22:08:17.986 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:17 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:17.866+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:28.956267+0000 front 2024-05-21T22:07:28.956225+0000 (oldest deadline 2024-05-21T22:07:55.356623+0000) 2024-05-21T22:08:17.986 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:17 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:17.866+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:38.857667+0000 front 2024-05-21T22:07:38.857566+0000 (oldest deadline 2024-05-21T22:08:04.757680+0000) 2024-05-21T22:08:17.986 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:17 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:17.866+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:44.758039+0000 front 2024-05-21T22:07:44.758082+0000 (oldest deadline 2024-05-21T22:08:10.658241+0000) 2024-05-21T22:08:18.286 INFO:journalctl@ceph.osd.7.smithi164.stdout:May 21 22:08:17 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-7[54180]: 2024-05-21T22:08:17.806+0000 7f8d4da11700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:37.480876+0000 front 2024-05-21T22:07:37.480856+0000 (oldest deadline 2024-05-21T22:08:02.181059+0000) 2024-05-21T22:08:18.286 INFO:journalctl@ceph.osd.7.smithi164.stdout:May 21 22:08:17 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-7[54180]: 2024-05-21T22:08:17.806+0000 7f8d4da11700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:49.083252+0000 front 2024-05-21T22:07:49.083231+0000 (oldest deadline 2024-05-21T22:08:10.183363+0000) 2024-05-21T22:08:18.286 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:17 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:17.848+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:30.508363+0000 front 2024-05-21T22:07:30.508149+0000 (oldest deadline 2024-05-21T22:07:52.808006+0000) 2024-05-21T22:08:18.287 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:17 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:17.848+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:41.009631+0000 front 2024-05-21T22:07:41.009565+0000 (oldest deadline 2024-05-21T22:08:05.709714+0000) 2024-05-21T22:08:18.287 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:17 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:17.848+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:49.011674+0000 front 2024-05-21T22:07:49.011868+0000 (oldest deadline 2024-05-21T22:08:13.711629+0000) 2024-05-21T22:08:18.701 INFO:journalctl@ceph.osd.6.smithi151.stdout:May 21 22:08:18 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-6[57087]: 2024-05-21T22:08:18.273+0000 7f0909400700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:31.896170+0000 front 2024-05-21T22:07:31.896361+0000 (oldest deadline 2024-05-21T22:07:57.795636+0000) 2024-05-21T22:08:18.702 INFO:journalctl@ceph.osd.6.smithi151.stdout:May 21 22:08:18 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-6[57087]: 2024-05-21T22:08:18.273+0000 7f0909400700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:38.298380+0000 front 2024-05-21T22:07:38.298169+0000 (oldest deadline 2024-05-21T22:08:04.198343+0000) 2024-05-21T22:08:18.702 INFO:journalctl@ceph.osd.6.smithi151.stdout:May 21 22:08:18 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-6[57087]: 2024-05-21T22:08:18.273+0000 7f0909400700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:48.702219+0000 front 2024-05-21T22:07:48.702298+0000 (oldest deadline 2024-05-21T22:08:10.402181+0000) 2024-05-21T22:08:18.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:18 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:18.700+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:27.976452+0000 front 2024-05-21T22:07:27.976969+0000 (oldest deadline 2024-05-21T22:07:53.876086+0000) 2024-05-21T22:08:18.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:18 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:18.700+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:39.580127+0000 front 2024-05-21T22:07:39.580305+0000 (oldest deadline 2024-05-21T22:08:04.280362+0000) 2024-05-21T22:08:18.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:18 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:18.700+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:47.782149+0000 front 2024-05-21T22:07:47.781979+0000 (oldest deadline 2024-05-21T22:08:10.682108+0000) 2024-05-21T22:08:18.986 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:18 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:18.676+0000 7f452bd16700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:40.689332+0000 front 2024-05-21T22:07:40.689184+0000 (oldest deadline 2024-05-21T22:08:06.589360+0000) 2024-05-21T22:08:18.986 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:18 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:18.676+0000 7f452bd16700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:48.890238+0000 front 2024-05-21T22:07:48.890266+0000 (oldest deadline 2024-05-21T22:08:12.990394+0000) 2024-05-21T22:08:18.986 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:18 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:18.676+0000 7f452bd16700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:52.991143+0000 front 2024-05-21T22:07:52.991015+0000 (oldest deadline 2024-05-21T22:08:18.290930+0000) 2024-05-21T22:08:18.986 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:18 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:18.822+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:28.956267+0000 front 2024-05-21T22:07:28.956225+0000 (oldest deadline 2024-05-21T22:07:55.356623+0000) 2024-05-21T22:08:18.986 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:18 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:18.822+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:38.857667+0000 front 2024-05-21T22:07:38.857566+0000 (oldest deadline 2024-05-21T22:08:04.757680+0000) 2024-05-21T22:08:18.986 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:18 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:18.822+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:44.758039+0000 front 2024-05-21T22:07:44.758082+0000 (oldest deadline 2024-05-21T22:08:10.658241+0000) 2024-05-21T22:08:19.286 INFO:journalctl@ceph.osd.7.smithi164.stdout:May 21 22:08:18 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-7[54180]: 2024-05-21T22:08:18.855+0000 7f8d4da11700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:37.480876+0000 front 2024-05-21T22:07:37.480856+0000 (oldest deadline 2024-05-21T22:08:02.181059+0000) 2024-05-21T22:08:19.286 INFO:journalctl@ceph.osd.7.smithi164.stdout:May 21 22:08:18 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-7[54180]: 2024-05-21T22:08:18.855+0000 7f8d4da11700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:49.083252+0000 front 2024-05-21T22:07:49.083231+0000 (oldest deadline 2024-05-21T22:08:10.183363+0000) 2024-05-21T22:08:19.286 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:18 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:18.898+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:30.508363+0000 front 2024-05-21T22:07:30.508149+0000 (oldest deadline 2024-05-21T22:07:52.808006+0000) 2024-05-21T22:08:19.286 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:18 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:18.898+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:41.009631+0000 front 2024-05-21T22:07:41.009565+0000 (oldest deadline 2024-05-21T22:08:05.709714+0000) 2024-05-21T22:08:19.287 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:18 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:18.898+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:49.011674+0000 front 2024-05-21T22:07:49.011868+0000 (oldest deadline 2024-05-21T22:08:13.711629+0000) 2024-05-21T22:08:19.287 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:18 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:18.898+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:53.712167+0000 front 2024-05-21T22:07:53.712374+0000 (oldest deadline 2024-05-21T22:08:18.412083+0000) 2024-05-21T22:08:19.656 INFO:journalctl@ceph.osd.6.smithi151.stdout:May 21 22:08:19 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-6[57087]: 2024-05-21T22:08:19.311+0000 7f0909400700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:31.896170+0000 front 2024-05-21T22:07:31.896361+0000 (oldest deadline 2024-05-21T22:07:57.795636+0000) 2024-05-21T22:08:19.656 INFO:journalctl@ceph.osd.6.smithi151.stdout:May 21 22:08:19 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-6[57087]: 2024-05-21T22:08:19.311+0000 7f0909400700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:38.298380+0000 front 2024-05-21T22:07:38.298169+0000 (oldest deadline 2024-05-21T22:08:04.198343+0000) 2024-05-21T22:08:19.656 INFO:journalctl@ceph.osd.6.smithi151.stdout:May 21 22:08:19 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-6[57087]: 2024-05-21T22:08:19.311+0000 7f0909400700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:48.702219+0000 front 2024-05-21T22:07:48.702298+0000 (oldest deadline 2024-05-21T22:08:10.402181+0000) 2024-05-21T22:08:19.656 INFO:journalctl@ceph.osd.6.smithi151.stdout:May 21 22:08:19 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-6[57087]: 2024-05-21T22:08:19.311+0000 7f0909400700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:53.204620+0000 front 2024-05-21T22:07:53.204879+0000 (oldest deadline 2024-05-21T22:08:19.104832+0000) 2024-05-21T22:08:19.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:19 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:19.654+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:27.976452+0000 front 2024-05-21T22:07:27.976969+0000 (oldest deadline 2024-05-21T22:07:53.876086+0000) 2024-05-21T22:08:19.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:19 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:19.654+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:39.580127+0000 front 2024-05-21T22:07:39.580305+0000 (oldest deadline 2024-05-21T22:08:04.280362+0000) 2024-05-21T22:08:19.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:19 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:19.654+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:47.782149+0000 front 2024-05-21T22:07:47.781979+0000 (oldest deadline 2024-05-21T22:08:10.682108+0000) 2024-05-21T22:08:19.986 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:19 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:19.696+0000 7f452bd16700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:40.689332+0000 front 2024-05-21T22:07:40.689184+0000 (oldest deadline 2024-05-21T22:08:06.589360+0000) 2024-05-21T22:08:19.986 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:19 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:19.696+0000 7f452bd16700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:48.890238+0000 front 2024-05-21T22:07:48.890266+0000 (oldest deadline 2024-05-21T22:08:12.990394+0000) 2024-05-21T22:08:19.986 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:19 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:19.696+0000 7f452bd16700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:52.991143+0000 front 2024-05-21T22:07:52.991015+0000 (oldest deadline 2024-05-21T22:08:18.290930+0000) 2024-05-21T22:08:19.986 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:19 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:19.781+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:28.956267+0000 front 2024-05-21T22:07:28.956225+0000 (oldest deadline 2024-05-21T22:07:55.356623+0000) 2024-05-21T22:08:19.986 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:19 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:19.781+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:38.857667+0000 front 2024-05-21T22:07:38.857566+0000 (oldest deadline 2024-05-21T22:08:04.757680+0000) 2024-05-21T22:08:19.986 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:19 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:19.781+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:44.758039+0000 front 2024-05-21T22:07:44.758082+0000 (oldest deadline 2024-05-21T22:08:10.658241+0000) 2024-05-21T22:08:20.286 INFO:journalctl@ceph.osd.7.smithi164.stdout:May 21 22:08:19 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-7[54180]: 2024-05-21T22:08:19.859+0000 7f8d4da11700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:37.480876+0000 front 2024-05-21T22:07:37.480856+0000 (oldest deadline 2024-05-21T22:08:02.181059+0000) 2024-05-21T22:08:20.286 INFO:journalctl@ceph.osd.7.smithi164.stdout:May 21 22:08:19 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-7[54180]: 2024-05-21T22:08:19.859+0000 7f8d4da11700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:49.083252+0000 front 2024-05-21T22:07:49.083231+0000 (oldest deadline 2024-05-21T22:08:10.183363+0000) 2024-05-21T22:08:20.286 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:19 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:19.925+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:30.508363+0000 front 2024-05-21T22:07:30.508149+0000 (oldest deadline 2024-05-21T22:07:52.808006+0000) 2024-05-21T22:08:20.286 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:19 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:19.925+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:41.009631+0000 front 2024-05-21T22:07:41.009565+0000 (oldest deadline 2024-05-21T22:08:05.709714+0000) 2024-05-21T22:08:20.286 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:19 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:19.925+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:49.011674+0000 front 2024-05-21T22:07:49.011868+0000 (oldest deadline 2024-05-21T22:08:13.711629+0000) 2024-05-21T22:08:20.287 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:19 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:19.925+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:53.712167+0000 front 2024-05-21T22:07:53.712374+0000 (oldest deadline 2024-05-21T22:08:18.412083+0000) 2024-05-21T22:08:20.621 INFO:journalctl@ceph.osd.6.smithi151.stdout:May 21 22:08:20 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-6[57087]: 2024-05-21T22:08:20.328+0000 7f0909400700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:31.896170+0000 front 2024-05-21T22:07:31.896361+0000 (oldest deadline 2024-05-21T22:07:57.795636+0000) 2024-05-21T22:08:20.621 INFO:journalctl@ceph.osd.6.smithi151.stdout:May 21 22:08:20 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-6[57087]: 2024-05-21T22:08:20.328+0000 7f0909400700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:38.298380+0000 front 2024-05-21T22:07:38.298169+0000 (oldest deadline 2024-05-21T22:08:04.198343+0000) 2024-05-21T22:08:20.621 INFO:journalctl@ceph.osd.6.smithi151.stdout:May 21 22:08:20 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-6[57087]: 2024-05-21T22:08:20.328+0000 7f0909400700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:48.702219+0000 front 2024-05-21T22:07:48.702298+0000 (oldest deadline 2024-05-21T22:08:10.402181+0000) 2024-05-21T22:08:20.622 INFO:journalctl@ceph.osd.6.smithi151.stdout:May 21 22:08:20 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-6[57087]: 2024-05-21T22:08:20.328+0000 7f0909400700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:53.204620+0000 front 2024-05-21T22:07:53.204879+0000 (oldest deadline 2024-05-21T22:08:19.104832+0000) 2024-05-21T22:08:20.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:20 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:20.620+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:27.976452+0000 front 2024-05-21T22:07:27.976969+0000 (oldest deadline 2024-05-21T22:07:53.876086+0000) 2024-05-21T22:08:20.954 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:20 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:20.620+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:39.580127+0000 front 2024-05-21T22:07:39.580305+0000 (oldest deadline 2024-05-21T22:08:04.280362+0000) 2024-05-21T22:08:20.954 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:20 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:20.620+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:47.782149+0000 front 2024-05-21T22:07:47.781979+0000 (oldest deadline 2024-05-21T22:08:10.682108+0000) 2024-05-21T22:08:21.236 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:20 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:20.737+0000 7f452bd16700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:40.689332+0000 front 2024-05-21T22:07:40.689184+0000 (oldest deadline 2024-05-21T22:08:06.589360+0000) 2024-05-21T22:08:21.236 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:20 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:20.737+0000 7f452bd16700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:48.890238+0000 front 2024-05-21T22:07:48.890266+0000 (oldest deadline 2024-05-21T22:08:12.990394+0000) 2024-05-21T22:08:21.236 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:20 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:20.737+0000 7f452bd16700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:52.991143+0000 front 2024-05-21T22:07:52.991015+0000 (oldest deadline 2024-05-21T22:08:18.290930+0000) 2024-05-21T22:08:21.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:20 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:20.830+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:28.956267+0000 front 2024-05-21T22:07:28.956225+0000 (oldest deadline 2024-05-21T22:07:55.356623+0000) 2024-05-21T22:08:21.237 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:20 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:20.831+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:38.857667+0000 front 2024-05-21T22:07:38.857566+0000 (oldest deadline 2024-05-21T22:08:04.757680+0000) 2024-05-21T22:08:21.237 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:20 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:20.831+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:44.758039+0000 front 2024-05-21T22:07:44.758082+0000 (oldest deadline 2024-05-21T22:08:10.658241+0000) 2024-05-21T22:08:21.286 INFO:journalctl@ceph.osd.7.smithi164.stdout:May 21 22:08:20 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-7[54180]: 2024-05-21T22:08:20.870+0000 7f8d4da11700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:37.480876+0000 front 2024-05-21T22:07:37.480856+0000 (oldest deadline 2024-05-21T22:08:02.181059+0000) 2024-05-21T22:08:21.286 INFO:journalctl@ceph.osd.7.smithi164.stdout:May 21 22:08:20 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-7[54180]: 2024-05-21T22:08:20.870+0000 7f8d4da11700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:49.083252+0000 front 2024-05-21T22:07:49.083231+0000 (oldest deadline 2024-05-21T22:08:10.183363+0000) 2024-05-21T22:08:21.286 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:20 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:20.949+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:30.508363+0000 front 2024-05-21T22:07:30.508149+0000 (oldest deadline 2024-05-21T22:07:52.808006+0000) 2024-05-21T22:08:21.287 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:20 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:20.949+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:41.009631+0000 front 2024-05-21T22:07:41.009565+0000 (oldest deadline 2024-05-21T22:08:05.709714+0000) 2024-05-21T22:08:21.287 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:20 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:20.949+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:49.011674+0000 front 2024-05-21T22:07:49.011868+0000 (oldest deadline 2024-05-21T22:08:13.711629+0000) 2024-05-21T22:08:21.287 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:20 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:20.949+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:53.712167+0000 front 2024-05-21T22:07:53.712374+0000 (oldest deadline 2024-05-21T22:08:18.412083+0000) 2024-05-21T22:08:21.703 INFO:journalctl@ceph.osd.6.smithi151.stdout:May 21 22:08:21 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-6[57087]: 2024-05-21T22:08:21.375+0000 7f0909400700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:31.896170+0000 front 2024-05-21T22:07:31.896361+0000 (oldest deadline 2024-05-21T22:07:57.795636+0000) 2024-05-21T22:08:21.704 INFO:journalctl@ceph.osd.6.smithi151.stdout:May 21 22:08:21 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-6[57087]: 2024-05-21T22:08:21.375+0000 7f0909400700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:38.298380+0000 front 2024-05-21T22:07:38.298169+0000 (oldest deadline 2024-05-21T22:08:04.198343+0000) 2024-05-21T22:08:21.704 INFO:journalctl@ceph.osd.6.smithi151.stdout:May 21 22:08:21 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-6[57087]: 2024-05-21T22:08:21.375+0000 7f0909400700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:48.702219+0000 front 2024-05-21T22:07:48.702298+0000 (oldest deadline 2024-05-21T22:08:10.402181+0000) 2024-05-21T22:08:21.704 INFO:journalctl@ceph.osd.6.smithi151.stdout:May 21 22:08:21 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-6[57087]: 2024-05-21T22:08:21.375+0000 7f0909400700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:53.204620+0000 front 2024-05-21T22:07:53.204879+0000 (oldest deadline 2024-05-21T22:08:19.104832+0000) 2024-05-21T22:08:21.704 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:21 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:21.599+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:27.976452+0000 front 2024-05-21T22:07:27.976969+0000 (oldest deadline 2024-05-21T22:07:53.876086+0000) 2024-05-21T22:08:21.704 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:21 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:21.599+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:39.580127+0000 front 2024-05-21T22:07:39.580305+0000 (oldest deadline 2024-05-21T22:08:04.280362+0000) 2024-05-21T22:08:21.704 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:21 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:21.599+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:47.782149+0000 front 2024-05-21T22:07:47.781979+0000 (oldest deadline 2024-05-21T22:08:10.682108+0000) 2024-05-21T22:08:21.704 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:21 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:21.599+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:56.385736+0000 front 2024-05-21T22:07:56.386240+0000 (oldest deadline 2024-05-21T22:08:21.085525+0000) 2024-05-21T22:08:22.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:21 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:21.872+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:28.956267+0000 front 2024-05-21T22:07:28.956225+0000 (oldest deadline 2024-05-21T22:07:55.356623+0000) 2024-05-21T22:08:22.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:21 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:21.872+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:38.857667+0000 front 2024-05-21T22:07:38.857566+0000 (oldest deadline 2024-05-21T22:08:04.757680+0000) 2024-05-21T22:08:22.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:21 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:21.872+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:44.758039+0000 front 2024-05-21T22:07:44.758082+0000 (oldest deadline 2024-05-21T22:08:10.658241+0000) 2024-05-21T22:08:22.236 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:21 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:21.778+0000 7f452bd16700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:40.689332+0000 front 2024-05-21T22:07:40.689184+0000 (oldest deadline 2024-05-21T22:08:06.589360+0000) 2024-05-21T22:08:22.236 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:21 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:21.778+0000 7f452bd16700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:48.890238+0000 front 2024-05-21T22:07:48.890266+0000 (oldest deadline 2024-05-21T22:08:12.990394+0000) 2024-05-21T22:08:22.236 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:21 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:21.778+0000 7f452bd16700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:52.991143+0000 front 2024-05-21T22:07:52.991015+0000 (oldest deadline 2024-05-21T22:08:18.290930+0000) 2024-05-21T22:08:22.286 INFO:journalctl@ceph.osd.7.smithi164.stdout:May 21 22:08:21 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-7[54180]: 2024-05-21T22:08:21.919+0000 7f8d4da11700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:37.480876+0000 front 2024-05-21T22:07:37.480856+0000 (oldest deadline 2024-05-21T22:08:02.181059+0000) 2024-05-21T22:08:22.286 INFO:journalctl@ceph.osd.7.smithi164.stdout:May 21 22:08:21 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-7[54180]: 2024-05-21T22:08:21.919+0000 7f8d4da11700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:49.083252+0000 front 2024-05-21T22:07:49.083231+0000 (oldest deadline 2024-05-21T22:08:10.183363+0000) 2024-05-21T22:08:22.286 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:21 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:21.972+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:30.508363+0000 front 2024-05-21T22:07:30.508149+0000 (oldest deadline 2024-05-21T22:07:52.808006+0000) 2024-05-21T22:08:22.286 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:21 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:21.972+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:41.009631+0000 front 2024-05-21T22:07:41.009565+0000 (oldest deadline 2024-05-21T22:08:05.709714+0000) 2024-05-21T22:08:22.286 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:21 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:21.972+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:49.011674+0000 front 2024-05-21T22:07:49.011868+0000 (oldest deadline 2024-05-21T22:08:13.711629+0000) 2024-05-21T22:08:22.286 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:21 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:21.972+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:53.712167+0000 front 2024-05-21T22:07:53.712374+0000 (oldest deadline 2024-05-21T22:08:18.412083+0000) 2024-05-21T22:08:22.607 INFO:journalctl@ceph.osd.6.smithi151.stdout:May 21 22:08:22 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-6[57087]: 2024-05-21T22:08:22.353+0000 7f0909400700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:31.896170+0000 front 2024-05-21T22:07:31.896361+0000 (oldest deadline 2024-05-21T22:07:57.795636+0000) 2024-05-21T22:08:22.607 INFO:journalctl@ceph.osd.6.smithi151.stdout:May 21 22:08:22 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-6[57087]: 2024-05-21T22:08:22.353+0000 7f0909400700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:38.298380+0000 front 2024-05-21T22:07:38.298169+0000 (oldest deadline 2024-05-21T22:08:04.198343+0000) 2024-05-21T22:08:22.607 INFO:journalctl@ceph.osd.6.smithi151.stdout:May 21 22:08:22 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-6[57087]: 2024-05-21T22:08:22.353+0000 7f0909400700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:48.702219+0000 front 2024-05-21T22:07:48.702298+0000 (oldest deadline 2024-05-21T22:08:10.402181+0000) 2024-05-21T22:08:22.607 INFO:journalctl@ceph.osd.6.smithi151.stdout:May 21 22:08:22 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-6[57087]: 2024-05-21T22:08:22.353+0000 7f0909400700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:53.204620+0000 front 2024-05-21T22:07:53.204879+0000 (oldest deadline 2024-05-21T22:08:19.104832+0000) 2024-05-21T22:08:22.954 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:22 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:22.617+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:27.976452+0000 front 2024-05-21T22:07:27.976969+0000 (oldest deadline 2024-05-21T22:07:53.876086+0000) 2024-05-21T22:08:22.954 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:22 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:22.617+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:39.580127+0000 front 2024-05-21T22:07:39.580305+0000 (oldest deadline 2024-05-21T22:08:04.280362+0000) 2024-05-21T22:08:22.954 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:22 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:22.617+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:47.782149+0000 front 2024-05-21T22:07:47.781979+0000 (oldest deadline 2024-05-21T22:08:10.682108+0000) 2024-05-21T22:08:22.954 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:22 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:22.617+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:56.385736+0000 front 2024-05-21T22:07:56.386240+0000 (oldest deadline 2024-05-21T22:08:21.085525+0000) 2024-05-21T22:08:22.986 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:22 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:22.849+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:28.956267+0000 front 2024-05-21T22:07:28.956225+0000 (oldest deadline 2024-05-21T22:07:55.356623+0000) 2024-05-21T22:08:22.986 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:22 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:22.849+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:38.857667+0000 front 2024-05-21T22:07:38.857566+0000 (oldest deadline 2024-05-21T22:08:04.757680+0000) 2024-05-21T22:08:22.986 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:22 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:22.849+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:44.758039+0000 front 2024-05-21T22:07:44.758082+0000 (oldest deadline 2024-05-21T22:08:10.658241+0000) 2024-05-21T22:08:22.987 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:22 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:22.731+0000 7f452bd16700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:40.689332+0000 front 2024-05-21T22:07:40.689184+0000 (oldest deadline 2024-05-21T22:08:06.589360+0000) 2024-05-21T22:08:22.987 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:22 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:22.731+0000 7f452bd16700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:48.890238+0000 front 2024-05-21T22:07:48.890266+0000 (oldest deadline 2024-05-21T22:08:12.990394+0000) 2024-05-21T22:08:22.987 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:22 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:22.731+0000 7f452bd16700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:52.991143+0000 front 2024-05-21T22:07:52.991015+0000 (oldest deadline 2024-05-21T22:08:18.290930+0000) 2024-05-21T22:08:23.286 INFO:journalctl@ceph.osd.7.smithi164.stdout:May 21 22:08:22 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-7[54180]: 2024-05-21T22:08:22.959+0000 7f8d4da11700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:37.480876+0000 front 2024-05-21T22:07:37.480856+0000 (oldest deadline 2024-05-21T22:08:02.181059+0000) 2024-05-21T22:08:23.286 INFO:journalctl@ceph.osd.7.smithi164.stdout:May 21 22:08:22 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-7[54180]: 2024-05-21T22:08:22.959+0000 7f8d4da11700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:49.083252+0000 front 2024-05-21T22:07:49.083231+0000 (oldest deadline 2024-05-21T22:08:10.183363+0000) 2024-05-21T22:08:23.286 INFO:journalctl@ceph.osd.7.smithi164.stdout:May 21 22:08:22 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-7[54180]: 2024-05-21T22:08:22.959+0000 7f8d4da11700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:56.084870+0000 front 2024-05-21T22:07:56.084447+0000 (oldest deadline 2024-05-21T22:08:21.984293+0000) 2024-05-21T22:08:23.286 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:22 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:22.984+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:30.508363+0000 front 2024-05-21T22:07:30.508149+0000 (oldest deadline 2024-05-21T22:07:52.808006+0000) 2024-05-21T22:08:23.287 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:22 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:22.984+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:41.009631+0000 front 2024-05-21T22:07:41.009565+0000 (oldest deadline 2024-05-21T22:08:05.709714+0000) 2024-05-21T22:08:23.287 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:22 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:22.984+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:49.011674+0000 front 2024-05-21T22:07:49.011868+0000 (oldest deadline 2024-05-21T22:08:13.711629+0000) 2024-05-21T22:08:23.287 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:22 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:22.984+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:53.712167+0000 front 2024-05-21T22:07:53.712374+0000 (oldest deadline 2024-05-21T22:08:18.412083+0000) 2024-05-21T22:08:23.312 INFO:journalctl@ceph.osd.6.smithi151.stdout:May 21 22:08:23 smithi151 podman[121703]: 2024-05-21 22:08:23.052522604 +0000 UTC m=+5.541056809 container died 1283ff154bd165fc8b711664ae75c305434ae691d1db056c6527674321e50a90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-6, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, io.buildah.version=1.33.5, org.label-schema.build-date=20240514) 2024-05-21T22:08:23.640 INFO:journalctl@ceph.osd.6.smithi151.stdout:May 21 22:08:23 smithi151 podman[121703]: 2024-05-21 22:08:23.496873853 +0000 UTC m=+5.985408095 container cleanup 1283ff154bd165fc8b711664ae75c305434ae691d1db056c6527674321e50a90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-6, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-05-21T22:08:23.640 INFO:journalctl@ceph.osd.6.smithi151.stdout:May 21 22:08:23 smithi151 bash[121703]: ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-6 2024-05-21T22:08:23.953 INFO:journalctl@ceph.osd.6.smithi151.stdout:May 21 22:08:23 smithi151 podman[121721]: 2024-05-21 22:08:23.640120958 +0000 UTC m=+0.570025767 container remove 1283ff154bd165fc8b711664ae75c305434ae691d1db056c6527674321e50a90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-6, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514) 2024-05-21T22:08:23.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:23 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:23.645+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:27.976452+0000 front 2024-05-21T22:07:27.976969+0000 (oldest deadline 2024-05-21T22:07:53.876086+0000) 2024-05-21T22:08:23.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:23 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:23.645+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:39.580127+0000 front 2024-05-21T22:07:39.580305+0000 (oldest deadline 2024-05-21T22:08:04.280362+0000) 2024-05-21T22:08:23.954 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:23 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:23.645+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:47.782149+0000 front 2024-05-21T22:07:47.781979+0000 (oldest deadline 2024-05-21T22:08:10.682108+0000) 2024-05-21T22:08:23.954 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:23 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:23.645+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:56.385736+0000 front 2024-05-21T22:07:56.386240+0000 (oldest deadline 2024-05-21T22:08:21.085525+0000) 2024-05-21T22:08:24.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:23 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:23.862+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:28.956267+0000 front 2024-05-21T22:07:28.956225+0000 (oldest deadline 2024-05-21T22:07:55.356623+0000) 2024-05-21T22:08:24.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:23 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:23.862+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:38.857667+0000 front 2024-05-21T22:07:38.857566+0000 (oldest deadline 2024-05-21T22:08:04.757680+0000) 2024-05-21T22:08:24.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:23 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:23.862+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:44.758039+0000 front 2024-05-21T22:07:44.758082+0000 (oldest deadline 2024-05-21T22:08:10.658241+0000) 2024-05-21T22:08:24.236 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:23 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:23.776+0000 7f452bd16700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:40.689332+0000 front 2024-05-21T22:07:40.689184+0000 (oldest deadline 2024-05-21T22:08:06.589360+0000) 2024-05-21T22:08:24.237 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:23 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:23.776+0000 7f452bd16700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:48.890238+0000 front 2024-05-21T22:07:48.890266+0000 (oldest deadline 2024-05-21T22:08:12.990394+0000) 2024-05-21T22:08:24.237 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:23 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:23.776+0000 7f452bd16700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:52.991143+0000 front 2024-05-21T22:07:52.991015+0000 (oldest deadline 2024-05-21T22:08:18.290930+0000) 2024-05-21T22:08:24.286 INFO:journalctl@ceph.osd.7.smithi164.stdout:May 21 22:08:23 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-7[54180]: 2024-05-21T22:08:23.980+0000 7f8d4da11700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:37.480876+0000 front 2024-05-21T22:07:37.480856+0000 (oldest deadline 2024-05-21T22:08:02.181059+0000) 2024-05-21T22:08:24.286 INFO:journalctl@ceph.osd.7.smithi164.stdout:May 21 22:08:23 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-7[54180]: 2024-05-21T22:08:23.980+0000 7f8d4da11700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:49.083252+0000 front 2024-05-21T22:07:49.083231+0000 (oldest deadline 2024-05-21T22:08:10.183363+0000) 2024-05-21T22:08:24.286 INFO:journalctl@ceph.osd.7.smithi164.stdout:May 21 22:08:23 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-7[54180]: 2024-05-21T22:08:23.980+0000 7f8d4da11700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:56.084870+0000 front 2024-05-21T22:07:56.084447+0000 (oldest deadline 2024-05-21T22:08:21.984293+0000) 2024-05-21T22:08:24.286 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:23 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:23.986+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:30.508363+0000 front 2024-05-21T22:07:30.508149+0000 (oldest deadline 2024-05-21T22:07:52.808006+0000) 2024-05-21T22:08:24.286 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:23 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:23.986+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:41.009631+0000 front 2024-05-21T22:07:41.009565+0000 (oldest deadline 2024-05-21T22:08:05.709714+0000) 2024-05-21T22:08:24.287 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:23 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:23.986+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:49.011674+0000 front 2024-05-21T22:07:49.011868+0000 (oldest deadline 2024-05-21T22:08:13.711629+0000) 2024-05-21T22:08:24.287 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:23 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:23.987+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:53.712167+0000 front 2024-05-21T22:07:53.712374+0000 (oldest deadline 2024-05-21T22:08:18.412083+0000) 2024-05-21T22:08:24.618 INFO:journalctl@ceph.osd.6.smithi151.stdout:May 21 22:08:24 smithi151 podman[121819]: 2024-05-21 22:08:24.358085385 +0000 UTC m=+0.031134578 image pull d6bc8f31cbc1a7165ca248fe9ba0b06ea14b30f46e1ea33ced1a5238e247af1f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28 2024-05-21T22:08:24.618 INFO:journalctl@ceph.osd.6.smithi151.stdout:May 21 22:08:24 smithi151 podman[121819]: 2024-05-21 22:08:24.474701258 +0000 UTC m=+0.147750478 container create 4e9cf234a287d6af4806e26de7353c1a8710f58a720143d2b56423ab60f966fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-6-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0) 2024-05-21T22:08:24.954 INFO:journalctl@ceph.osd.6.smithi151.stdout:May 21 22:08:24 smithi151 podman[121819]: 2024-05-21 22:08:24.737823046 +0000 UTC m=+0.410872302 container init 4e9cf234a287d6af4806e26de7353c1a8710f58a720143d2b56423ab60f966fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-6-deactivate, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-05-21T22:08:24.954 INFO:journalctl@ceph.osd.6.smithi151.stdout:May 21 22:08:24 smithi151 podman[121819]: 2024-05-21 22:08:24.748989049 +0000 UTC m=+0.422038284 container start 4e9cf234a287d6af4806e26de7353c1a8710f58a720143d2b56423ab60f966fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-6-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, ceph=True, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-21T22:08:24.954 INFO:journalctl@ceph.osd.6.smithi151.stdout:May 21 22:08:24 smithi151 podman[121819]: 2024-05-21 22:08:24.819351735 +0000 UTC m=+0.492400968 container attach 4e9cf234a287d6af4806e26de7353c1a8710f58a720143d2b56423ab60f966fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=) 2024-05-21T22:08:24.954 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:24 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:24.616+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:27.976452+0000 front 2024-05-21T22:07:27.976969+0000 (oldest deadline 2024-05-21T22:07:53.876086+0000) 2024-05-21T22:08:24.954 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:24 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:24.616+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:39.580127+0000 front 2024-05-21T22:07:39.580305+0000 (oldest deadline 2024-05-21T22:08:04.280362+0000) 2024-05-21T22:08:24.954 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:24 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:24.616+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:47.782149+0000 front 2024-05-21T22:07:47.781979+0000 (oldest deadline 2024-05-21T22:08:10.682108+0000) 2024-05-21T22:08:24.954 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:24 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:24.616+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:56.385736+0000 front 2024-05-21T22:07:56.386240+0000 (oldest deadline 2024-05-21T22:08:21.085525+0000) 2024-05-21T22:08:25.137 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:24 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:24.888+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:28.956267+0000 front 2024-05-21T22:07:28.956225+0000 (oldest deadline 2024-05-21T22:07:55.356623+0000) 2024-05-21T22:08:25.137 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:24 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:24.888+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:38.857667+0000 front 2024-05-21T22:07:38.857566+0000 (oldest deadline 2024-05-21T22:08:04.757680+0000) 2024-05-21T22:08:25.137 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:24 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:24.888+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:44.758039+0000 front 2024-05-21T22:07:44.758082+0000 (oldest deadline 2024-05-21T22:08:10.658241+0000) 2024-05-21T22:08:25.138 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:24 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:24.759+0000 7f452bd16700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:40.689332+0000 front 2024-05-21T22:07:40.689184+0000 (oldest deadline 2024-05-21T22:08:06.589360+0000) 2024-05-21T22:08:25.138 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:24 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:24.759+0000 7f452bd16700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:48.890238+0000 front 2024-05-21T22:07:48.890266+0000 (oldest deadline 2024-05-21T22:08:12.990394+0000) 2024-05-21T22:08:25.138 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:24 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:24.759+0000 7f452bd16700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:52.991143+0000 front 2024-05-21T22:07:52.991015+0000 (oldest deadline 2024-05-21T22:08:18.290930+0000) 2024-05-21T22:08:25.286 INFO:journalctl@ceph.osd.7.smithi164.stdout:May 21 22:08:24 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-7[54180]: 2024-05-21T22:08:24.988+0000 7f8d4da11700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:37.480876+0000 front 2024-05-21T22:07:37.480856+0000 (oldest deadline 2024-05-21T22:08:02.181059+0000) 2024-05-21T22:08:25.286 INFO:journalctl@ceph.osd.7.smithi164.stdout:May 21 22:08:24 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-7[54180]: 2024-05-21T22:08:24.988+0000 7f8d4da11700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:49.083252+0000 front 2024-05-21T22:07:49.083231+0000 (oldest deadline 2024-05-21T22:08:10.183363+0000) 2024-05-21T22:08:25.286 INFO:journalctl@ceph.osd.7.smithi164.stdout:May 21 22:08:24 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-7[54180]: 2024-05-21T22:08:24.988+0000 7f8d4da11700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:56.084870+0000 front 2024-05-21T22:07:56.084447+0000 (oldest deadline 2024-05-21T22:08:21.984293+0000) 2024-05-21T22:08:25.286 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:24 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:24.981+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:30.508363+0000 front 2024-05-21T22:07:30.508149+0000 (oldest deadline 2024-05-21T22:07:52.808006+0000) 2024-05-21T22:08:25.286 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:24 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:24.981+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:41.009631+0000 front 2024-05-21T22:07:41.009565+0000 (oldest deadline 2024-05-21T22:08:05.709714+0000) 2024-05-21T22:08:25.286 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:24 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:24.981+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:49.011674+0000 front 2024-05-21T22:07:49.011868+0000 (oldest deadline 2024-05-21T22:08:13.711629+0000) 2024-05-21T22:08:25.286 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:24 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:24.981+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:53.712167+0000 front 2024-05-21T22:07:53.712374+0000 (oldest deadline 2024-05-21T22:08:18.412083+0000) 2024-05-21T22:08:25.453 INFO:journalctl@ceph.osd.6.smithi151.stdout:May 21 22:08:25 smithi151 podman[121819]: 2024-05-21 22:08:25.198496465 +0000 UTC m=+0.871545704 container died 4e9cf234a287d6af4806e26de7353c1a8710f58a720143d2b56423ab60f966fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-6-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-05-21T22:08:25.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:25 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:25.631+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:27.976452+0000 front 2024-05-21T22:07:27.976969+0000 (oldest deadline 2024-05-21T22:07:53.876086+0000) 2024-05-21T22:08:25.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:25 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:25.631+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:39.580127+0000 front 2024-05-21T22:07:39.580305+0000 (oldest deadline 2024-05-21T22:08:04.280362+0000) 2024-05-21T22:08:25.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:25 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:25.631+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:47.782149+0000 front 2024-05-21T22:07:47.781979+0000 (oldest deadline 2024-05-21T22:08:10.682108+0000) 2024-05-21T22:08:25.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:25 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:25.631+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:56.385736+0000 front 2024-05-21T22:07:56.386240+0000 (oldest deadline 2024-05-21T22:08:21.085525+0000) 2024-05-21T22:08:26.011 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@osd.6.service' 2024-05-21T22:08:26.123 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T22:08:26.123 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-05-21T22:08:26.123 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-05-21T22:08:26.123 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@osd.7 2024-05-21T22:08:26.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:25 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:25.895+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:28.956267+0000 front 2024-05-21T22:07:28.956225+0000 (oldest deadline 2024-05-21T22:07:55.356623+0000) 2024-05-21T22:08:26.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:25 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:25.895+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:38.857667+0000 front 2024-05-21T22:07:38.857566+0000 (oldest deadline 2024-05-21T22:08:04.757680+0000) 2024-05-21T22:08:26.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:25 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:25.895+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:44.758039+0000 front 2024-05-21T22:07:44.758082+0000 (oldest deadline 2024-05-21T22:08:10.658241+0000) 2024-05-21T22:08:26.236 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:25 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:25.779+0000 7f452bd16700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:40.689332+0000 front 2024-05-21T22:07:40.689184+0000 (oldest deadline 2024-05-21T22:08:06.589360+0000) 2024-05-21T22:08:26.236 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:25 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:25.779+0000 7f452bd16700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:48.890238+0000 front 2024-05-21T22:07:48.890266+0000 (oldest deadline 2024-05-21T22:08:12.990394+0000) 2024-05-21T22:08:26.236 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:25 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:25.779+0000 7f452bd16700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:52.991143+0000 front 2024-05-21T22:07:52.991015+0000 (oldest deadline 2024-05-21T22:08:18.290930+0000) 2024-05-21T22:08:26.286 INFO:journalctl@ceph.osd.7.smithi164.stdout:May 21 22:08:25 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-7[54180]: 2024-05-21T22:08:25.941+0000 7f8d4da11700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:37.480876+0000 front 2024-05-21T22:07:37.480856+0000 (oldest deadline 2024-05-21T22:08:02.181059+0000) 2024-05-21T22:08:26.286 INFO:journalctl@ceph.osd.7.smithi164.stdout:May 21 22:08:25 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-7[54180]: 2024-05-21T22:08:25.941+0000 7f8d4da11700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:49.083252+0000 front 2024-05-21T22:07:49.083231+0000 (oldest deadline 2024-05-21T22:08:10.183363+0000) 2024-05-21T22:08:26.286 INFO:journalctl@ceph.osd.7.smithi164.stdout:May 21 22:08:25 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-7[54180]: 2024-05-21T22:08:25.941+0000 7f8d4da11700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:56.084870+0000 front 2024-05-21T22:07:56.084447+0000 (oldest deadline 2024-05-21T22:08:21.984293+0000) 2024-05-21T22:08:26.286 INFO:journalctl@ceph.osd.7.smithi164.stdout:May 21 22:08:26 smithi164 systemd[1]: Stopping Ceph osd.7 for c8c7a2e6-17ba-11ef-bc9a-c7b262605968... 2024-05-21T22:08:26.287 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:25 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:25.987+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:30.508363+0000 front 2024-05-21T22:07:30.508149+0000 (oldest deadline 2024-05-21T22:07:52.808006+0000) 2024-05-21T22:08:26.287 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:25 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:25.987+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:41.009631+0000 front 2024-05-21T22:07:41.009565+0000 (oldest deadline 2024-05-21T22:08:05.709714+0000) 2024-05-21T22:08:26.287 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:25 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:25.987+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:49.011674+0000 front 2024-05-21T22:07:49.011868+0000 (oldest deadline 2024-05-21T22:08:13.711629+0000) 2024-05-21T22:08:26.287 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:25 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:25.987+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:53.712167+0000 front 2024-05-21T22:07:53.712374+0000 (oldest deadline 2024-05-21T22:08:18.412083+0000) 2024-05-21T22:08:26.786 INFO:journalctl@ceph.osd.7.smithi164.stdout:May 21 22:08:26 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-7[54180]: 2024-05-21T22:08:26.417+0000 7f8d5141d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T22:08:26.786 INFO:journalctl@ceph.osd.7.smithi164.stdout:May 21 22:08:26 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-7[54180]: 2024-05-21T22:08:26.417+0000 7f8d5141d700 -1 osd.7 94 *** Got signal Terminated *** 2024-05-21T22:08:26.786 INFO:journalctl@ceph.osd.7.smithi164.stdout:May 21 22:08:26 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-7[54180]: 2024-05-21T22:08:26.417+0000 7f8d5141d700 -1 osd.7 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-21T22:08:26.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:26 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:26.642+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:27.976452+0000 front 2024-05-21T22:07:27.976969+0000 (oldest deadline 2024-05-21T22:07:53.876086+0000) 2024-05-21T22:08:26.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:26 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:26.642+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:39.580127+0000 front 2024-05-21T22:07:39.580305+0000 (oldest deadline 2024-05-21T22:08:04.280362+0000) 2024-05-21T22:08:26.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:26 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:26.642+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:47.782149+0000 front 2024-05-21T22:07:47.781979+0000 (oldest deadline 2024-05-21T22:08:10.682108+0000) 2024-05-21T22:08:26.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:26 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:26.642+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:56.385736+0000 front 2024-05-21T22:07:56.386240+0000 (oldest deadline 2024-05-21T22:08:21.085525+0000) 2024-05-21T22:08:27.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:26 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:26.932+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:28.956267+0000 front 2024-05-21T22:07:28.956225+0000 (oldest deadline 2024-05-21T22:07:55.356623+0000) 2024-05-21T22:08:27.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:26 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:26.932+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:38.857667+0000 front 2024-05-21T22:07:38.857566+0000 (oldest deadline 2024-05-21T22:08:04.757680+0000) 2024-05-21T22:08:27.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:26 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:26.932+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:44.758039+0000 front 2024-05-21T22:07:44.758082+0000 (oldest deadline 2024-05-21T22:08:10.658241+0000) 2024-05-21T22:08:27.236 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:26 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:26.791+0000 7f452bd16700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:40.689332+0000 front 2024-05-21T22:07:40.689184+0000 (oldest deadline 2024-05-21T22:08:06.589360+0000) 2024-05-21T22:08:27.236 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:26 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:26.791+0000 7f452bd16700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:48.890238+0000 front 2024-05-21T22:07:48.890266+0000 (oldest deadline 2024-05-21T22:08:12.990394+0000) 2024-05-21T22:08:27.236 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:26 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:26.791+0000 7f452bd16700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:52.991143+0000 front 2024-05-21T22:07:52.991015+0000 (oldest deadline 2024-05-21T22:08:18.290930+0000) 2024-05-21T22:08:27.286 INFO:journalctl@ceph.osd.7.smithi164.stdout:May 21 22:08:26 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-7[54180]: 2024-05-21T22:08:26.904+0000 7f8d4da11700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:37.480876+0000 front 2024-05-21T22:07:37.480856+0000 (oldest deadline 2024-05-21T22:08:02.181059+0000) 2024-05-21T22:08:27.286 INFO:journalctl@ceph.osd.7.smithi164.stdout:May 21 22:08:26 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-7[54180]: 2024-05-21T22:08:26.904+0000 7f8d4da11700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:49.083252+0000 front 2024-05-21T22:07:49.083231+0000 (oldest deadline 2024-05-21T22:08:10.183363+0000) 2024-05-21T22:08:27.286 INFO:journalctl@ceph.osd.7.smithi164.stdout:May 21 22:08:26 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-7[54180]: 2024-05-21T22:08:26.904+0000 7f8d4da11700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:56.084870+0000 front 2024-05-21T22:07:56.084447+0000 (oldest deadline 2024-05-21T22:08:21.984293+0000) 2024-05-21T22:08:27.286 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:26 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:26.964+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:30.508363+0000 front 2024-05-21T22:07:30.508149+0000 (oldest deadline 2024-05-21T22:07:52.808006+0000) 2024-05-21T22:08:27.287 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:26 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:26.964+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:41.009631+0000 front 2024-05-21T22:07:41.009565+0000 (oldest deadline 2024-05-21T22:08:05.709714+0000) 2024-05-21T22:08:27.287 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:26 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:26.964+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:49.011674+0000 front 2024-05-21T22:07:49.011868+0000 (oldest deadline 2024-05-21T22:08:13.711629+0000) 2024-05-21T22:08:27.287 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:26 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:26.964+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:53.712167+0000 front 2024-05-21T22:07:53.712374+0000 (oldest deadline 2024-05-21T22:08:18.412083+0000) 2024-05-21T22:08:27.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:27 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:27.683+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:27.976452+0000 front 2024-05-21T22:07:27.976969+0000 (oldest deadline 2024-05-21T22:07:53.876086+0000) 2024-05-21T22:08:27.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:27 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:27.683+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:39.580127+0000 front 2024-05-21T22:07:39.580305+0000 (oldest deadline 2024-05-21T22:08:04.280362+0000) 2024-05-21T22:08:27.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:27 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:27.683+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:47.782149+0000 front 2024-05-21T22:07:47.781979+0000 (oldest deadline 2024-05-21T22:08:10.682108+0000) 2024-05-21T22:08:27.954 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:27 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:27.683+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:56.385736+0000 front 2024-05-21T22:07:56.386240+0000 (oldest deadline 2024-05-21T22:08:21.085525+0000) 2024-05-21T22:08:28.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:27 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:27.956+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:28.956267+0000 front 2024-05-21T22:07:28.956225+0000 (oldest deadline 2024-05-21T22:07:55.356623+0000) 2024-05-21T22:08:28.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:27 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:27.956+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:38.857667+0000 front 2024-05-21T22:07:38.857566+0000 (oldest deadline 2024-05-21T22:08:04.757680+0000) 2024-05-21T22:08:28.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:27 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:27.956+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:44.758039+0000 front 2024-05-21T22:07:44.758082+0000 (oldest deadline 2024-05-21T22:08:10.658241+0000) 2024-05-21T22:08:28.236 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:27 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:27.789+0000 7f452bd16700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:40.689332+0000 front 2024-05-21T22:07:40.689184+0000 (oldest deadline 2024-05-21T22:08:06.589360+0000) 2024-05-21T22:08:28.236 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:27 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:27.789+0000 7f452bd16700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:48.890238+0000 front 2024-05-21T22:07:48.890266+0000 (oldest deadline 2024-05-21T22:08:12.990394+0000) 2024-05-21T22:08:28.236 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:27 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:27.789+0000 7f452bd16700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:52.991143+0000 front 2024-05-21T22:07:52.991015+0000 (oldest deadline 2024-05-21T22:08:18.290930+0000) 2024-05-21T22:08:28.236 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:27 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:27.789+0000 7f452bd16700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-05-21T22:08:02.292736+0000 front 2024-05-21T22:08:02.292584+0000 (oldest deadline 2024-05-21T22:08:26.992601+0000) 2024-05-21T22:08:28.286 INFO:journalctl@ceph.osd.7.smithi164.stdout:May 21 22:08:27 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-7[54180]: 2024-05-21T22:08:27.873+0000 7f8d4da11700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:37.480876+0000 front 2024-05-21T22:07:37.480856+0000 (oldest deadline 2024-05-21T22:08:02.181059+0000) 2024-05-21T22:08:28.286 INFO:journalctl@ceph.osd.7.smithi164.stdout:May 21 22:08:27 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-7[54180]: 2024-05-21T22:08:27.873+0000 7f8d4da11700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:49.083252+0000 front 2024-05-21T22:07:49.083231+0000 (oldest deadline 2024-05-21T22:08:10.183363+0000) 2024-05-21T22:08:28.286 INFO:journalctl@ceph.osd.7.smithi164.stdout:May 21 22:08:27 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-7[54180]: 2024-05-21T22:08:27.873+0000 7f8d4da11700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:56.084870+0000 front 2024-05-21T22:07:56.084447+0000 (oldest deadline 2024-05-21T22:08:21.984293+0000) 2024-05-21T22:08:28.287 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:27 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:27.978+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:30.508363+0000 front 2024-05-21T22:07:30.508149+0000 (oldest deadline 2024-05-21T22:07:52.808006+0000) 2024-05-21T22:08:28.287 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:27 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:27.978+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:41.009631+0000 front 2024-05-21T22:07:41.009565+0000 (oldest deadline 2024-05-21T22:08:05.709714+0000) 2024-05-21T22:08:28.287 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:27 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:27.978+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:49.011674+0000 front 2024-05-21T22:07:49.011868+0000 (oldest deadline 2024-05-21T22:08:13.711629+0000) 2024-05-21T22:08:28.287 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:27 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:27.978+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:53.712167+0000 front 2024-05-21T22:07:53.712374+0000 (oldest deadline 2024-05-21T22:08:18.412083+0000) 2024-05-21T22:08:28.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:28 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:28.683+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:27.976452+0000 front 2024-05-21T22:07:27.976969+0000 (oldest deadline 2024-05-21T22:07:53.876086+0000) 2024-05-21T22:08:28.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:28 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:28.683+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:39.580127+0000 front 2024-05-21T22:07:39.580305+0000 (oldest deadline 2024-05-21T22:08:04.280362+0000) 2024-05-21T22:08:28.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:28 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:28.683+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:47.782149+0000 front 2024-05-21T22:07:47.781979+0000 (oldest deadline 2024-05-21T22:08:10.682108+0000) 2024-05-21T22:08:28.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:28 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:28.683+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:56.385736+0000 front 2024-05-21T22:07:56.386240+0000 (oldest deadline 2024-05-21T22:08:21.085525+0000) 2024-05-21T22:08:29.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:28 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:28.924+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:28.956267+0000 front 2024-05-21T22:07:28.956225+0000 (oldest deadline 2024-05-21T22:07:55.356623+0000) 2024-05-21T22:08:29.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:28 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:28.924+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:38.857667+0000 front 2024-05-21T22:07:38.857566+0000 (oldest deadline 2024-05-21T22:08:04.757680+0000) 2024-05-21T22:08:29.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:28 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:28.924+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:44.758039+0000 front 2024-05-21T22:07:44.758082+0000 (oldest deadline 2024-05-21T22:08:10.658241+0000) 2024-05-21T22:08:29.236 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:28 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:28.802+0000 7f452bd16700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:40.689332+0000 front 2024-05-21T22:07:40.689184+0000 (oldest deadline 2024-05-21T22:08:06.589360+0000) 2024-05-21T22:08:29.236 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:28 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:28.802+0000 7f452bd16700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:48.890238+0000 front 2024-05-21T22:07:48.890266+0000 (oldest deadline 2024-05-21T22:08:12.990394+0000) 2024-05-21T22:08:29.236 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:28 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:28.802+0000 7f452bd16700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:52.991143+0000 front 2024-05-21T22:07:52.991015+0000 (oldest deadline 2024-05-21T22:08:18.290930+0000) 2024-05-21T22:08:29.236 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:28 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:28.802+0000 7f452bd16700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-05-21T22:08:02.292736+0000 front 2024-05-21T22:08:02.292584+0000 (oldest deadline 2024-05-21T22:08:26.992601+0000) 2024-05-21T22:08:29.286 INFO:journalctl@ceph.osd.7.smithi164.stdout:May 21 22:08:28 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-7[54180]: 2024-05-21T22:08:28.920+0000 7f8d4da11700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:37.480876+0000 front 2024-05-21T22:07:37.480856+0000 (oldest deadline 2024-05-21T22:08:02.181059+0000) 2024-05-21T22:08:29.286 INFO:journalctl@ceph.osd.7.smithi164.stdout:May 21 22:08:28 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-7[54180]: 2024-05-21T22:08:28.920+0000 7f8d4da11700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:49.083252+0000 front 2024-05-21T22:07:49.083231+0000 (oldest deadline 2024-05-21T22:08:10.183363+0000) 2024-05-21T22:08:29.286 INFO:journalctl@ceph.osd.7.smithi164.stdout:May 21 22:08:28 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-7[54180]: 2024-05-21T22:08:28.920+0000 7f8d4da11700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:56.084870+0000 front 2024-05-21T22:07:56.084447+0000 (oldest deadline 2024-05-21T22:08:21.984293+0000) 2024-05-21T22:08:29.286 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:28 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:28.981+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:30.508363+0000 front 2024-05-21T22:07:30.508149+0000 (oldest deadline 2024-05-21T22:07:52.808006+0000) 2024-05-21T22:08:29.286 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:28 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:28.981+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:41.009631+0000 front 2024-05-21T22:07:41.009565+0000 (oldest deadline 2024-05-21T22:08:05.709714+0000) 2024-05-21T22:08:29.286 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:28 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:28.981+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:49.011674+0000 front 2024-05-21T22:07:49.011868+0000 (oldest deadline 2024-05-21T22:08:13.711629+0000) 2024-05-21T22:08:29.286 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:28 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:28.981+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:53.712167+0000 front 2024-05-21T22:07:53.712374+0000 (oldest deadline 2024-05-21T22:08:18.412083+0000) 2024-05-21T22:08:29.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:29 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:29.679+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:27.976452+0000 front 2024-05-21T22:07:27.976969+0000 (oldest deadline 2024-05-21T22:07:53.876086+0000) 2024-05-21T22:08:29.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:29 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:29.679+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:39.580127+0000 front 2024-05-21T22:07:39.580305+0000 (oldest deadline 2024-05-21T22:08:04.280362+0000) 2024-05-21T22:08:29.954 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:29 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:29.679+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:47.782149+0000 front 2024-05-21T22:07:47.781979+0000 (oldest deadline 2024-05-21T22:08:10.682108+0000) 2024-05-21T22:08:29.954 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:29 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:29.679+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:56.385736+0000 front 2024-05-21T22:07:56.386240+0000 (oldest deadline 2024-05-21T22:08:21.085525+0000) 2024-05-21T22:08:30.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:29 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:29.971+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:28.956267+0000 front 2024-05-21T22:07:28.956225+0000 (oldest deadline 2024-05-21T22:07:55.356623+0000) 2024-05-21T22:08:30.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:29 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:29.971+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:38.857667+0000 front 2024-05-21T22:07:38.857566+0000 (oldest deadline 2024-05-21T22:08:04.757680+0000) 2024-05-21T22:08:30.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:29 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:29.971+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:44.758039+0000 front 2024-05-21T22:07:44.758082+0000 (oldest deadline 2024-05-21T22:08:10.658241+0000) 2024-05-21T22:08:30.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:29 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:29.971+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-05-21T22:08:04.160535+0000 front 2024-05-21T22:08:04.160468+0000 (oldest deadline 2024-05-21T22:08:29.460490+0000) 2024-05-21T22:08:30.236 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:29 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:29.798+0000 7f452bd16700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:40.689332+0000 front 2024-05-21T22:07:40.689184+0000 (oldest deadline 2024-05-21T22:08:06.589360+0000) 2024-05-21T22:08:30.237 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:29 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:29.798+0000 7f452bd16700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:48.890238+0000 front 2024-05-21T22:07:48.890266+0000 (oldest deadline 2024-05-21T22:08:12.990394+0000) 2024-05-21T22:08:30.237 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:29 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:29.798+0000 7f452bd16700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:52.991143+0000 front 2024-05-21T22:07:52.991015+0000 (oldest deadline 2024-05-21T22:08:18.290930+0000) 2024-05-21T22:08:30.237 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:29 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:29.798+0000 7f452bd16700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-05-21T22:08:02.292736+0000 front 2024-05-21T22:08:02.292584+0000 (oldest deadline 2024-05-21T22:08:26.992601+0000) 2024-05-21T22:08:30.286 INFO:journalctl@ceph.osd.7.smithi164.stdout:May 21 22:08:29 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-7[54180]: 2024-05-21T22:08:29.904+0000 7f8d4da11700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:37.480876+0000 front 2024-05-21T22:07:37.480856+0000 (oldest deadline 2024-05-21T22:08:02.181059+0000) 2024-05-21T22:08:30.286 INFO:journalctl@ceph.osd.7.smithi164.stdout:May 21 22:08:29 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-7[54180]: 2024-05-21T22:08:29.904+0000 7f8d4da11700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:49.083252+0000 front 2024-05-21T22:07:49.083231+0000 (oldest deadline 2024-05-21T22:08:10.183363+0000) 2024-05-21T22:08:30.286 INFO:journalctl@ceph.osd.7.smithi164.stdout:May 21 22:08:29 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-7[54180]: 2024-05-21T22:08:29.904+0000 7f8d4da11700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:56.084870+0000 front 2024-05-21T22:07:56.084447+0000 (oldest deadline 2024-05-21T22:08:21.984293+0000) 2024-05-21T22:08:30.286 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:30 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:30.012+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:30.508363+0000 front 2024-05-21T22:07:30.508149+0000 (oldest deadline 2024-05-21T22:07:52.808006+0000) 2024-05-21T22:08:30.286 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:30 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:30.012+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:41.009631+0000 front 2024-05-21T22:07:41.009565+0000 (oldest deadline 2024-05-21T22:08:05.709714+0000) 2024-05-21T22:08:30.287 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:30 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:30.012+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:49.011674+0000 front 2024-05-21T22:07:49.011868+0000 (oldest deadline 2024-05-21T22:08:13.711629+0000) 2024-05-21T22:08:30.287 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:30 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:30.012+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:53.712167+0000 front 2024-05-21T22:07:53.712374+0000 (oldest deadline 2024-05-21T22:08:18.412083+0000) 2024-05-21T22:08:30.287 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:30 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:30.012+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-05-21T22:08:03.113068+0000 front 2024-05-21T22:08:03.113103+0000 (oldest deadline 2024-05-21T22:08:29.513902+0000) 2024-05-21T22:08:30.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:30 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:30.659+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:27.976452+0000 front 2024-05-21T22:07:27.976969+0000 (oldest deadline 2024-05-21T22:07:53.876086+0000) 2024-05-21T22:08:30.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:30 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:30.659+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:39.580127+0000 front 2024-05-21T22:07:39.580305+0000 (oldest deadline 2024-05-21T22:08:04.280362+0000) 2024-05-21T22:08:30.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:30 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:30.659+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:47.782149+0000 front 2024-05-21T22:07:47.781979+0000 (oldest deadline 2024-05-21T22:08:10.682108+0000) 2024-05-21T22:08:30.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:30 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:30.659+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:56.385736+0000 front 2024-05-21T22:07:56.386240+0000 (oldest deadline 2024-05-21T22:08:21.085525+0000) 2024-05-21T22:08:31.236 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:30 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:30.781+0000 7f452bd16700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:40.689332+0000 front 2024-05-21T22:07:40.689184+0000 (oldest deadline 2024-05-21T22:08:06.589360+0000) 2024-05-21T22:08:31.236 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:30 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:30.781+0000 7f452bd16700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:48.890238+0000 front 2024-05-21T22:07:48.890266+0000 (oldest deadline 2024-05-21T22:08:12.990394+0000) 2024-05-21T22:08:31.236 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:30 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:30.781+0000 7f452bd16700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:52.991143+0000 front 2024-05-21T22:07:52.991015+0000 (oldest deadline 2024-05-21T22:08:18.290930+0000) 2024-05-21T22:08:31.236 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:30 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:30.781+0000 7f452bd16700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-05-21T22:08:02.292736+0000 front 2024-05-21T22:08:02.292584+0000 (oldest deadline 2024-05-21T22:08:26.992601+0000) 2024-05-21T22:08:31.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:30 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:30.933+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:28.956267+0000 front 2024-05-21T22:07:28.956225+0000 (oldest deadline 2024-05-21T22:07:55.356623+0000) 2024-05-21T22:08:31.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:30 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:30.933+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:38.857667+0000 front 2024-05-21T22:07:38.857566+0000 (oldest deadline 2024-05-21T22:08:04.757680+0000) 2024-05-21T22:08:31.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:30 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:30.933+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:44.758039+0000 front 2024-05-21T22:07:44.758082+0000 (oldest deadline 2024-05-21T22:08:10.658241+0000) 2024-05-21T22:08:31.237 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:30 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:30.933+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-05-21T22:08:04.160535+0000 front 2024-05-21T22:08:04.160468+0000 (oldest deadline 2024-05-21T22:08:29.460490+0000) 2024-05-21T22:08:31.286 INFO:journalctl@ceph.osd.7.smithi164.stdout:May 21 22:08:30 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-7[54180]: 2024-05-21T22:08:30.913+0000 7f8d4da11700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:37.480876+0000 front 2024-05-21T22:07:37.480856+0000 (oldest deadline 2024-05-21T22:08:02.181059+0000) 2024-05-21T22:08:31.286 INFO:journalctl@ceph.osd.7.smithi164.stdout:May 21 22:08:30 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-7[54180]: 2024-05-21T22:08:30.913+0000 7f8d4da11700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:49.083252+0000 front 2024-05-21T22:07:49.083231+0000 (oldest deadline 2024-05-21T22:08:10.183363+0000) 2024-05-21T22:08:31.286 INFO:journalctl@ceph.osd.7.smithi164.stdout:May 21 22:08:30 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-7[54180]: 2024-05-21T22:08:30.913+0000 7f8d4da11700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:56.084870+0000 front 2024-05-21T22:07:56.084447+0000 (oldest deadline 2024-05-21T22:08:21.984293+0000) 2024-05-21T22:08:31.287 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:31 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:31.053+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:30.508363+0000 front 2024-05-21T22:07:30.508149+0000 (oldest deadline 2024-05-21T22:07:52.808006+0000) 2024-05-21T22:08:31.287 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:31 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:31.053+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:41.009631+0000 front 2024-05-21T22:07:41.009565+0000 (oldest deadline 2024-05-21T22:08:05.709714+0000) 2024-05-21T22:08:31.287 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:31 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:31.053+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:49.011674+0000 front 2024-05-21T22:07:49.011868+0000 (oldest deadline 2024-05-21T22:08:13.711629+0000) 2024-05-21T22:08:31.287 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:31 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:31.053+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:53.712167+0000 front 2024-05-21T22:07:53.712374+0000 (oldest deadline 2024-05-21T22:08:18.412083+0000) 2024-05-21T22:08:31.287 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:31 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:31.053+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-05-21T22:08:03.113068+0000 front 2024-05-21T22:08:03.113103+0000 (oldest deadline 2024-05-21T22:08:29.513902+0000) 2024-05-21T22:08:31.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:31 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:31.646+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:27.976452+0000 front 2024-05-21T22:07:27.976969+0000 (oldest deadline 2024-05-21T22:07:53.876086+0000) 2024-05-21T22:08:31.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:31 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:31.646+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:39.580127+0000 front 2024-05-21T22:07:39.580305+0000 (oldest deadline 2024-05-21T22:08:04.280362+0000) 2024-05-21T22:08:31.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:31 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:31.646+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:47.782149+0000 front 2024-05-21T22:07:47.781979+0000 (oldest deadline 2024-05-21T22:08:10.682108+0000) 2024-05-21T22:08:31.954 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:31 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:31.646+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:56.385736+0000 front 2024-05-21T22:07:56.386240+0000 (oldest deadline 2024-05-21T22:08:21.085525+0000) 2024-05-21T22:08:32.126 INFO:journalctl@ceph.osd.7.smithi164.stdout:May 21 22:08:31 smithi164 podman[73190]: 2024-05-21 22:08:31.837121512 +0000 UTC m=+5.520189085 container died a53d6eb75ee23bd5cca43b675bdc5697b3a04ce2274ea106b6356cf9b41c4d80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-7, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea) 2024-05-21T22:08:32.126 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:32 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:32.012+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:30.508363+0000 front 2024-05-21T22:07:30.508149+0000 (oldest deadline 2024-05-21T22:07:52.808006+0000) 2024-05-21T22:08:32.127 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:32 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:32.012+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:41.009631+0000 front 2024-05-21T22:07:41.009565+0000 (oldest deadline 2024-05-21T22:08:05.709714+0000) 2024-05-21T22:08:32.127 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:32 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:32.012+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:49.011674+0000 front 2024-05-21T22:07:49.011868+0000 (oldest deadline 2024-05-21T22:08:13.711629+0000) 2024-05-21T22:08:32.127 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:32 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:32.012+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:53.712167+0000 front 2024-05-21T22:07:53.712374+0000 (oldest deadline 2024-05-21T22:08:18.412083+0000) 2024-05-21T22:08:32.127 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:32 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:32.012+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-05-21T22:08:03.113068+0000 front 2024-05-21T22:08:03.113103+0000 (oldest deadline 2024-05-21T22:08:29.513902+0000) 2024-05-21T22:08:32.236 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:31 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:31.755+0000 7f452bd16700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:40.689332+0000 front 2024-05-21T22:07:40.689184+0000 (oldest deadline 2024-05-21T22:08:06.589360+0000) 2024-05-21T22:08:32.236 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:31 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:31.755+0000 7f452bd16700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:48.890238+0000 front 2024-05-21T22:07:48.890266+0000 (oldest deadline 2024-05-21T22:08:12.990394+0000) 2024-05-21T22:08:32.236 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:31 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:31.755+0000 7f452bd16700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:52.991143+0000 front 2024-05-21T22:07:52.991015+0000 (oldest deadline 2024-05-21T22:08:18.290930+0000) 2024-05-21T22:08:32.236 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:31 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:31.755+0000 7f452bd16700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-05-21T22:08:02.292736+0000 front 2024-05-21T22:08:02.292584+0000 (oldest deadline 2024-05-21T22:08:26.992601+0000) 2024-05-21T22:08:32.237 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:31 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:31.901+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:28.956267+0000 front 2024-05-21T22:07:28.956225+0000 (oldest deadline 2024-05-21T22:07:55.356623+0000) 2024-05-21T22:08:32.237 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:31 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:31.901+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:38.857667+0000 front 2024-05-21T22:07:38.857566+0000 (oldest deadline 2024-05-21T22:08:04.757680+0000) 2024-05-21T22:08:32.237 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:31 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:31.901+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:44.758039+0000 front 2024-05-21T22:07:44.758082+0000 (oldest deadline 2024-05-21T22:08:10.658241+0000) 2024-05-21T22:08:32.237 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:31 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:31.901+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-05-21T22:08:04.160535+0000 front 2024-05-21T22:08:04.160468+0000 (oldest deadline 2024-05-21T22:08:29.460490+0000) 2024-05-21T22:08:32.451 INFO:journalctl@ceph.osd.7.smithi164.stdout:May 21 22:08:32 smithi164 podman[73190]: 2024-05-21 22:08:32.208952439 +0000 UTC m=+5.892020008 container cleanup a53d6eb75ee23bd5cca43b675bdc5697b3a04ce2274ea106b6356cf9b41c4d80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-7, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-21T22:08:32.451 INFO:journalctl@ceph.osd.7.smithi164.stdout:May 21 22:08:32 smithi164 bash[73190]: ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-7 2024-05-21T22:08:32.769 INFO:journalctl@ceph.osd.7.smithi164.stdout:May 21 22:08:32 smithi164 podman[73207]: 2024-05-21 22:08:32.45191264 +0000 UTC m=+0.604835208 container remove a53d6eb75ee23bd5cca43b675bdc5697b3a04ce2274ea106b6356cf9b41c4d80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-21T22:08:32.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:32 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:32.616+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:27.976452+0000 front 2024-05-21T22:07:27.976969+0000 (oldest deadline 2024-05-21T22:07:53.876086+0000) 2024-05-21T22:08:32.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:32 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:32.616+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:39.580127+0000 front 2024-05-21T22:07:39.580305+0000 (oldest deadline 2024-05-21T22:08:04.280362+0000) 2024-05-21T22:08:32.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:32 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:32.616+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:47.782149+0000 front 2024-05-21T22:07:47.781979+0000 (oldest deadline 2024-05-21T22:08:10.682108+0000) 2024-05-21T22:08:32.954 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:32 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:32.616+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:56.385736+0000 front 2024-05-21T22:07:56.386240+0000 (oldest deadline 2024-05-21T22:08:21.085525+0000) 2024-05-21T22:08:33.036 INFO:journalctl@ceph.osd.7.smithi164.stdout:May 21 22:08:32 smithi164 podman[73294]: 2024-05-21 22:08:32.76937001 +0000 UTC m=+0.109404908 container create 6ebe50a824a62e85080dbabcbc62390d1345050ba92205e1e30d36a2992bac8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea) 2024-05-21T22:08:33.037 INFO:journalctl@ceph.osd.7.smithi164.stdout:May 21 22:08:32 smithi164 podman[73294]: 2024-05-21 22:08:32.67726433 +0000 UTC m=+0.017299245 image pull d6bc8f31cbc1a7165ca248fe9ba0b06ea14b30f46e1ea33ced1a5238e247af1f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28 2024-05-21T22:08:33.037 INFO:journalctl@ceph.osd.7.smithi164.stdout:May 21 22:08:32 smithi164 podman[73294]: 2024-05-21 22:08:32.98011965 +0000 UTC m=+0.320154556 container init 6ebe50a824a62e85080dbabcbc62390d1345050ba92205e1e30d36a2992bac8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux ) 2024-05-21T22:08:33.037 INFO:journalctl@ceph.osd.7.smithi164.stdout:May 21 22:08:32 smithi164 podman[73294]: 2024-05-21 22:08:32.983090493 +0000 UTC m=+0.323125404 container start 6ebe50a824a62e85080dbabcbc62390d1345050ba92205e1e30d36a2992bac8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-21T22:08:33.236 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:32 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:32.797+0000 7f452bd16700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:40.689332+0000 front 2024-05-21T22:07:40.689184+0000 (oldest deadline 2024-05-21T22:08:06.589360+0000) 2024-05-21T22:08:33.236 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:32 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:32.797+0000 7f452bd16700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:48.890238+0000 front 2024-05-21T22:07:48.890266+0000 (oldest deadline 2024-05-21T22:08:12.990394+0000) 2024-05-21T22:08:33.236 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:32 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:32.797+0000 7f452bd16700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:52.991143+0000 front 2024-05-21T22:07:52.991015+0000 (oldest deadline 2024-05-21T22:08:18.290930+0000) 2024-05-21T22:08:33.236 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:32 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:32.797+0000 7f452bd16700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-05-21T22:08:02.292736+0000 front 2024-05-21T22:08:02.292584+0000 (oldest deadline 2024-05-21T22:08:26.992601+0000) 2024-05-21T22:08:33.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:32 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:32.880+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:28.956267+0000 front 2024-05-21T22:07:28.956225+0000 (oldest deadline 2024-05-21T22:07:55.356623+0000) 2024-05-21T22:08:33.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:32 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:32.880+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:38.857667+0000 front 2024-05-21T22:07:38.857566+0000 (oldest deadline 2024-05-21T22:08:04.757680+0000) 2024-05-21T22:08:33.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:32 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:32.880+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:44.758039+0000 front 2024-05-21T22:07:44.758082+0000 (oldest deadline 2024-05-21T22:08:10.658241+0000) 2024-05-21T22:08:33.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:32 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:32.880+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-05-21T22:08:04.160535+0000 front 2024-05-21T22:08:04.160468+0000 (oldest deadline 2024-05-21T22:08:29.460490+0000) 2024-05-21T22:08:33.536 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:33 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:33.048+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:30.508363+0000 front 2024-05-21T22:07:30.508149+0000 (oldest deadline 2024-05-21T22:07:52.808006+0000) 2024-05-21T22:08:33.536 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:33 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:33.048+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:41.009631+0000 front 2024-05-21T22:07:41.009565+0000 (oldest deadline 2024-05-21T22:08:05.709714+0000) 2024-05-21T22:08:33.536 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:33 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:33.048+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:49.011674+0000 front 2024-05-21T22:07:49.011868+0000 (oldest deadline 2024-05-21T22:08:13.711629+0000) 2024-05-21T22:08:33.536 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:33 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:33.048+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:53.712167+0000 front 2024-05-21T22:07:53.712374+0000 (oldest deadline 2024-05-21T22:08:18.412083+0000) 2024-05-21T22:08:33.537 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:33 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:33.048+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-05-21T22:08:03.113068+0000 front 2024-05-21T22:08:03.113103+0000 (oldest deadline 2024-05-21T22:08:29.513902+0000) 2024-05-21T22:08:33.537 INFO:journalctl@ceph.osd.7.smithi164.stdout:May 21 22:08:33 smithi164 podman[73294]: 2024-05-21 22:08:33.080058127 +0000 UTC m=+0.420093025 container attach 6ebe50a824a62e85080dbabcbc62390d1345050ba92205e1e30d36a2992bac8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-7-deactivate, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-05-21T22:08:33.537 INFO:journalctl@ceph.osd.7.smithi164.stdout:May 21 22:08:33 smithi164 podman[73294]: 2024-05-21 22:08:33.281025117 +0000 UTC m=+0.621060025 container died 6ebe50a824a62e85080dbabcbc62390d1345050ba92205e1e30d36a2992bac8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-05-21T22:08:33.837 INFO:journalctl@ceph.osd.7.smithi164.stdout:May 21 22:08:33 smithi164 podman[73320]: 2024-05-21 22:08:33.817250194 +0000 UTC m=+0.530924392 container remove 6ebe50a824a62e85080dbabcbc62390d1345050ba92205e1e30d36a2992bac8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True) 2024-05-21T22:08:33.843 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@osd.7.service' 2024-05-21T22:08:33.882 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T22:08:33.882 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-05-21T22:08:33.883 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-05-21T22:08:33.883 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@osd.8 2024-05-21T22:08:33.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:33 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:33.569+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:27.976452+0000 front 2024-05-21T22:07:27.976969+0000 (oldest deadline 2024-05-21T22:07:53.876086+0000) 2024-05-21T22:08:33.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:33 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:33.569+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:39.580127+0000 front 2024-05-21T22:07:39.580305+0000 (oldest deadline 2024-05-21T22:08:04.280362+0000) 2024-05-21T22:08:33.954 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:33 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:33.569+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:47.782149+0000 front 2024-05-21T22:07:47.781979+0000 (oldest deadline 2024-05-21T22:08:10.682108+0000) 2024-05-21T22:08:33.954 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:33 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:33.569+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:56.385736+0000 front 2024-05-21T22:07:56.386240+0000 (oldest deadline 2024-05-21T22:08:21.085525+0000) 2024-05-21T22:08:34.198 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:33 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:33.850+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:28.956267+0000 front 2024-05-21T22:07:28.956225+0000 (oldest deadline 2024-05-21T22:07:55.356623+0000) 2024-05-21T22:08:34.199 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:33 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:33.850+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:38.857667+0000 front 2024-05-21T22:07:38.857566+0000 (oldest deadline 2024-05-21T22:08:04.757680+0000) 2024-05-21T22:08:34.199 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:33 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:33.850+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:44.758039+0000 front 2024-05-21T22:07:44.758082+0000 (oldest deadline 2024-05-21T22:08:10.658241+0000) 2024-05-21T22:08:34.199 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:33 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:33.850+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-05-21T22:08:04.160535+0000 front 2024-05-21T22:08:04.160468+0000 (oldest deadline 2024-05-21T22:08:29.460490+0000) 2024-05-21T22:08:34.199 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:33 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:33.819+0000 7f452bd16700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:40.689332+0000 front 2024-05-21T22:07:40.689184+0000 (oldest deadline 2024-05-21T22:08:06.589360+0000) 2024-05-21T22:08:34.199 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:33 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:33.819+0000 7f452bd16700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:48.890238+0000 front 2024-05-21T22:07:48.890266+0000 (oldest deadline 2024-05-21T22:08:12.990394+0000) 2024-05-21T22:08:34.199 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:33 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:33.819+0000 7f452bd16700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:52.991143+0000 front 2024-05-21T22:07:52.991015+0000 (oldest deadline 2024-05-21T22:08:18.290930+0000) 2024-05-21T22:08:34.199 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:33 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:33.819+0000 7f452bd16700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-05-21T22:08:02.292736+0000 front 2024-05-21T22:08:02.292584+0000 (oldest deadline 2024-05-21T22:08:26.992601+0000) 2024-05-21T22:08:34.199 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:33 smithi184 systemd[1]: Stopping Ceph osd.8 for c8c7a2e6-17ba-11ef-bc9a-c7b262605968... 2024-05-21T22:08:34.199 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:34 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:34.198+0000 7f452f722700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T22:08:34.200 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:34 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:34.198+0000 7f452f722700 -1 osd.8 94 *** Got signal Terminated *** 2024-05-21T22:08:34.200 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:34 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:34.198+0000 7f452f722700 -1 osd.8 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-21T22:08:34.286 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:34 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:34.005+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:30.508363+0000 front 2024-05-21T22:07:30.508149+0000 (oldest deadline 2024-05-21T22:07:52.808006+0000) 2024-05-21T22:08:34.286 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:34 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:34.005+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:41.009631+0000 front 2024-05-21T22:07:41.009565+0000 (oldest deadline 2024-05-21T22:08:05.709714+0000) 2024-05-21T22:08:34.286 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:34 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:34.005+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:49.011674+0000 front 2024-05-21T22:07:49.011868+0000 (oldest deadline 2024-05-21T22:08:13.711629+0000) 2024-05-21T22:08:34.286 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:34 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:34.005+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:53.712167+0000 front 2024-05-21T22:07:53.712374+0000 (oldest deadline 2024-05-21T22:08:18.412083+0000) 2024-05-21T22:08:34.286 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:34 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:34.005+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-05-21T22:08:03.113068+0000 front 2024-05-21T22:08:03.113103+0000 (oldest deadline 2024-05-21T22:08:29.513902+0000) 2024-05-21T22:08:34.703 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:34 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:34.532+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:27.976452+0000 front 2024-05-21T22:07:27.976969+0000 (oldest deadline 2024-05-21T22:07:53.876086+0000) 2024-05-21T22:08:34.703 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:34 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:34.532+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:39.580127+0000 front 2024-05-21T22:07:39.580305+0000 (oldest deadline 2024-05-21T22:08:04.280362+0000) 2024-05-21T22:08:34.703 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:34 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:34.533+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:47.782149+0000 front 2024-05-21T22:07:47.781979+0000 (oldest deadline 2024-05-21T22:08:10.682108+0000) 2024-05-21T22:08:34.703 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:34 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:34.533+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:56.385736+0000 front 2024-05-21T22:07:56.386240+0000 (oldest deadline 2024-05-21T22:08:21.085525+0000) 2024-05-21T22:08:35.199 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:34 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:34.997+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:30.508363+0000 front 2024-05-21T22:07:30.508149+0000 (oldest deadline 2024-05-21T22:07:52.808006+0000) 2024-05-21T22:08:35.199 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:34 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:34.997+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:41.009631+0000 front 2024-05-21T22:07:41.009565+0000 (oldest deadline 2024-05-21T22:08:05.709714+0000) 2024-05-21T22:08:35.199 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:34 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:34.997+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:49.011674+0000 front 2024-05-21T22:07:49.011868+0000 (oldest deadline 2024-05-21T22:08:13.711629+0000) 2024-05-21T22:08:35.199 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:34 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:34.997+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:53.712167+0000 front 2024-05-21T22:07:53.712374+0000 (oldest deadline 2024-05-21T22:08:18.412083+0000) 2024-05-21T22:08:35.199 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:34 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:34.997+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-05-21T22:08:03.113068+0000 front 2024-05-21T22:08:03.113103+0000 (oldest deadline 2024-05-21T22:08:29.513902+0000) 2024-05-21T22:08:35.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:34 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:34.855+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:28.956267+0000 front 2024-05-21T22:07:28.956225+0000 (oldest deadline 2024-05-21T22:07:55.356623+0000) 2024-05-21T22:08:35.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:34 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:34.856+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:38.857667+0000 front 2024-05-21T22:07:38.857566+0000 (oldest deadline 2024-05-21T22:08:04.757680+0000) 2024-05-21T22:08:35.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:34 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:34.856+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:44.758039+0000 front 2024-05-21T22:07:44.758082+0000 (oldest deadline 2024-05-21T22:08:10.658241+0000) 2024-05-21T22:08:35.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:34 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:34.856+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-05-21T22:08:04.160535+0000 front 2024-05-21T22:08:04.160468+0000 (oldest deadline 2024-05-21T22:08:29.460490+0000) 2024-05-21T22:08:35.236 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:34 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:34.844+0000 7f452bd16700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:40.689332+0000 front 2024-05-21T22:07:40.689184+0000 (oldest deadline 2024-05-21T22:08:06.589360+0000) 2024-05-21T22:08:35.236 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:34 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:34.844+0000 7f452bd16700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:48.890238+0000 front 2024-05-21T22:07:48.890266+0000 (oldest deadline 2024-05-21T22:08:12.990394+0000) 2024-05-21T22:08:35.236 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:34 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:34.844+0000 7f452bd16700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:52.991143+0000 front 2024-05-21T22:07:52.991015+0000 (oldest deadline 2024-05-21T22:08:18.290930+0000) 2024-05-21T22:08:35.236 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:34 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:34.844+0000 7f452bd16700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-05-21T22:08:02.292736+0000 front 2024-05-21T22:08:02.292584+0000 (oldest deadline 2024-05-21T22:08:26.992601+0000) 2024-05-21T22:08:35.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:35 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:35.530+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:27.976452+0000 front 2024-05-21T22:07:27.976969+0000 (oldest deadline 2024-05-21T22:07:53.876086+0000) 2024-05-21T22:08:35.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:35 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:35.530+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:39.580127+0000 front 2024-05-21T22:07:39.580305+0000 (oldest deadline 2024-05-21T22:08:04.280362+0000) 2024-05-21T22:08:35.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:35 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:35.530+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:47.782149+0000 front 2024-05-21T22:07:47.781979+0000 (oldest deadline 2024-05-21T22:08:10.682108+0000) 2024-05-21T22:08:35.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:35 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:35.530+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:56.385736+0000 front 2024-05-21T22:07:56.386240+0000 (oldest deadline 2024-05-21T22:08:21.085525+0000) 2024-05-21T22:08:36.236 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:35 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:35.893+0000 7f452bd16700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:40.689332+0000 front 2024-05-21T22:07:40.689184+0000 (oldest deadline 2024-05-21T22:08:06.589360+0000) 2024-05-21T22:08:36.236 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:35 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:35.893+0000 7f452bd16700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:48.890238+0000 front 2024-05-21T22:07:48.890266+0000 (oldest deadline 2024-05-21T22:08:12.990394+0000) 2024-05-21T22:08:36.236 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:35 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:35.893+0000 7f452bd16700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:52.991143+0000 front 2024-05-21T22:07:52.991015+0000 (oldest deadline 2024-05-21T22:08:18.290930+0000) 2024-05-21T22:08:36.236 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:35 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:35.893+0000 7f452bd16700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-05-21T22:08:02.292736+0000 front 2024-05-21T22:08:02.292584+0000 (oldest deadline 2024-05-21T22:08:26.992601+0000) 2024-05-21T22:08:36.237 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:35 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:35.869+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:28.956267+0000 front 2024-05-21T22:07:28.956225+0000 (oldest deadline 2024-05-21T22:07:55.356623+0000) 2024-05-21T22:08:36.237 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:35 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:35.869+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:38.857667+0000 front 2024-05-21T22:07:38.857566+0000 (oldest deadline 2024-05-21T22:08:04.757680+0000) 2024-05-21T22:08:36.237 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:35 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:35.869+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:44.758039+0000 front 2024-05-21T22:07:44.758082+0000 (oldest deadline 2024-05-21T22:08:10.658241+0000) 2024-05-21T22:08:36.237 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:35 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:35.869+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-05-21T22:08:04.160535+0000 front 2024-05-21T22:08:04.160468+0000 (oldest deadline 2024-05-21T22:08:29.460490+0000) 2024-05-21T22:08:36.237 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:35 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:35.869+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-21T22:08:10.561367+0000 front 2024-05-21T22:08:10.561307+0000 (oldest deadline 2024-05-21T22:08:35.861406+0000) 2024-05-21T22:08:36.536 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:36 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:36.047+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:30.508363+0000 front 2024-05-21T22:07:30.508149+0000 (oldest deadline 2024-05-21T22:07:52.808006+0000) 2024-05-21T22:08:36.536 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:36 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:36.047+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:41.009631+0000 front 2024-05-21T22:07:41.009565+0000 (oldest deadline 2024-05-21T22:08:05.709714+0000) 2024-05-21T22:08:36.536 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:36 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:36.047+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:49.011674+0000 front 2024-05-21T22:07:49.011868+0000 (oldest deadline 2024-05-21T22:08:13.711629+0000) 2024-05-21T22:08:36.536 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:36 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:36.047+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:53.712167+0000 front 2024-05-21T22:07:53.712374+0000 (oldest deadline 2024-05-21T22:08:18.412083+0000) 2024-05-21T22:08:36.536 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:36 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:36.047+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-05-21T22:08:03.113068+0000 front 2024-05-21T22:08:03.113103+0000 (oldest deadline 2024-05-21T22:08:29.513902+0000) 2024-05-21T22:08:36.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:36 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:36.579+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:27.976452+0000 front 2024-05-21T22:07:27.976969+0000 (oldest deadline 2024-05-21T22:07:53.876086+0000) 2024-05-21T22:08:36.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:36 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:36.579+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:39.580127+0000 front 2024-05-21T22:07:39.580305+0000 (oldest deadline 2024-05-21T22:08:04.280362+0000) 2024-05-21T22:08:36.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:36 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:36.579+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:47.782149+0000 front 2024-05-21T22:07:47.781979+0000 (oldest deadline 2024-05-21T22:08:10.682108+0000) 2024-05-21T22:08:36.954 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:36 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:36.579+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:56.385736+0000 front 2024-05-21T22:07:56.386240+0000 (oldest deadline 2024-05-21T22:08:21.085525+0000) 2024-05-21T22:08:37.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:36 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:36.908+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:28.956267+0000 front 2024-05-21T22:07:28.956225+0000 (oldest deadline 2024-05-21T22:07:55.356623+0000) 2024-05-21T22:08:37.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:36 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:36.908+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:38.857667+0000 front 2024-05-21T22:07:38.857566+0000 (oldest deadline 2024-05-21T22:08:04.757680+0000) 2024-05-21T22:08:37.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:36 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:36.908+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:44.758039+0000 front 2024-05-21T22:07:44.758082+0000 (oldest deadline 2024-05-21T22:08:10.658241+0000) 2024-05-21T22:08:37.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:36 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:36.908+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-05-21T22:08:04.160535+0000 front 2024-05-21T22:08:04.160468+0000 (oldest deadline 2024-05-21T22:08:29.460490+0000) 2024-05-21T22:08:37.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:36 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:36.908+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-21T22:08:10.561367+0000 front 2024-05-21T22:08:10.561307+0000 (oldest deadline 2024-05-21T22:08:35.861406+0000) 2024-05-21T22:08:37.236 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:36 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:36.924+0000 7f452bd16700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:40.689332+0000 front 2024-05-21T22:07:40.689184+0000 (oldest deadline 2024-05-21T22:08:06.589360+0000) 2024-05-21T22:08:37.236 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:36 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:36.924+0000 7f452bd16700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:48.890238+0000 front 2024-05-21T22:07:48.890266+0000 (oldest deadline 2024-05-21T22:08:12.990394+0000) 2024-05-21T22:08:37.237 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:36 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:36.924+0000 7f452bd16700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:52.991143+0000 front 2024-05-21T22:07:52.991015+0000 (oldest deadline 2024-05-21T22:08:18.290930+0000) 2024-05-21T22:08:37.237 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:36 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:36.924+0000 7f452bd16700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-05-21T22:08:02.292736+0000 front 2024-05-21T22:08:02.292584+0000 (oldest deadline 2024-05-21T22:08:26.992601+0000) 2024-05-21T22:08:37.536 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:37 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:37.062+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:30.508363+0000 front 2024-05-21T22:07:30.508149+0000 (oldest deadline 2024-05-21T22:07:52.808006+0000) 2024-05-21T22:08:37.536 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:37 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:37.062+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:41.009631+0000 front 2024-05-21T22:07:41.009565+0000 (oldest deadline 2024-05-21T22:08:05.709714+0000) 2024-05-21T22:08:37.536 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:37 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:37.062+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:49.011674+0000 front 2024-05-21T22:07:49.011868+0000 (oldest deadline 2024-05-21T22:08:13.711629+0000) 2024-05-21T22:08:37.536 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:37 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:37.062+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:53.712167+0000 front 2024-05-21T22:07:53.712374+0000 (oldest deadline 2024-05-21T22:08:18.412083+0000) 2024-05-21T22:08:37.536 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:37 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:37.062+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-05-21T22:08:03.113068+0000 front 2024-05-21T22:08:03.113103+0000 (oldest deadline 2024-05-21T22:08:29.513902+0000) 2024-05-21T22:08:37.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:37 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:37.539+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:27.976452+0000 front 2024-05-21T22:07:27.976969+0000 (oldest deadline 2024-05-21T22:07:53.876086+0000) 2024-05-21T22:08:37.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:37 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:37.539+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:39.580127+0000 front 2024-05-21T22:07:39.580305+0000 (oldest deadline 2024-05-21T22:08:04.280362+0000) 2024-05-21T22:08:37.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:37 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:37.539+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:47.782149+0000 front 2024-05-21T22:07:47.781979+0000 (oldest deadline 2024-05-21T22:08:10.682108+0000) 2024-05-21T22:08:37.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:37 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:37.539+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:56.385736+0000 front 2024-05-21T22:07:56.386240+0000 (oldest deadline 2024-05-21T22:08:21.085525+0000) 2024-05-21T22:08:38.236 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:37 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:37.918+0000 7f452bd16700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:40.689332+0000 front 2024-05-21T22:07:40.689184+0000 (oldest deadline 2024-05-21T22:08:06.589360+0000) 2024-05-21T22:08:38.236 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:37 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:37.918+0000 7f452bd16700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:48.890238+0000 front 2024-05-21T22:07:48.890266+0000 (oldest deadline 2024-05-21T22:08:12.990394+0000) 2024-05-21T22:08:38.236 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:37 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:37.918+0000 7f452bd16700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:52.991143+0000 front 2024-05-21T22:07:52.991015+0000 (oldest deadline 2024-05-21T22:08:18.290930+0000) 2024-05-21T22:08:38.236 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:37 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:37.918+0000 7f452bd16700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-05-21T22:08:02.292736+0000 front 2024-05-21T22:08:02.292584+0000 (oldest deadline 2024-05-21T22:08:26.992601+0000) 2024-05-21T22:08:38.236 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:37 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:37.918+0000 7f452bd16700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-21T22:08:12.595075+0000 front 2024-05-21T22:08:12.595118+0000 (oldest deadline 2024-05-21T22:08:37.895152+0000) 2024-05-21T22:08:38.237 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:37 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:37.927+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:28.956267+0000 front 2024-05-21T22:07:28.956225+0000 (oldest deadline 2024-05-21T22:07:55.356623+0000) 2024-05-21T22:08:38.237 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:37 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:37.927+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:38.857667+0000 front 2024-05-21T22:07:38.857566+0000 (oldest deadline 2024-05-21T22:08:04.757680+0000) 2024-05-21T22:08:38.237 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:37 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:37.927+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:44.758039+0000 front 2024-05-21T22:07:44.758082+0000 (oldest deadline 2024-05-21T22:08:10.658241+0000) 2024-05-21T22:08:38.237 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:37 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:37.927+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-05-21T22:08:04.160535+0000 front 2024-05-21T22:08:04.160468+0000 (oldest deadline 2024-05-21T22:08:29.460490+0000) 2024-05-21T22:08:38.237 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:37 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:37.927+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-21T22:08:10.561367+0000 front 2024-05-21T22:08:10.561307+0000 (oldest deadline 2024-05-21T22:08:35.861406+0000) 2024-05-21T22:08:38.536 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:38 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:38.044+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:30.508363+0000 front 2024-05-21T22:07:30.508149+0000 (oldest deadline 2024-05-21T22:07:52.808006+0000) 2024-05-21T22:08:38.536 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:38 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:38.044+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:41.009631+0000 front 2024-05-21T22:07:41.009565+0000 (oldest deadline 2024-05-21T22:08:05.709714+0000) 2024-05-21T22:08:38.536 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:38 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:38.044+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:49.011674+0000 front 2024-05-21T22:07:49.011868+0000 (oldest deadline 2024-05-21T22:08:13.711629+0000) 2024-05-21T22:08:38.536 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:38 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:38.044+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:53.712167+0000 front 2024-05-21T22:07:53.712374+0000 (oldest deadline 2024-05-21T22:08:18.412083+0000) 2024-05-21T22:08:38.536 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:38 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:38.044+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-05-21T22:08:03.113068+0000 front 2024-05-21T22:08:03.113103+0000 (oldest deadline 2024-05-21T22:08:29.513902+0000) 2024-05-21T22:08:38.536 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:38 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:38.044+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-21T22:08:13.615230+0000 front 2024-05-21T22:08:13.615079+0000 (oldest deadline 2024-05-21T22:08:37.714944+0000) 2024-05-21T22:08:38.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:38 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:38.503+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:27.976452+0000 front 2024-05-21T22:07:27.976969+0000 (oldest deadline 2024-05-21T22:07:53.876086+0000) 2024-05-21T22:08:38.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:38 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:38.503+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:39.580127+0000 front 2024-05-21T22:07:39.580305+0000 (oldest deadline 2024-05-21T22:08:04.280362+0000) 2024-05-21T22:08:38.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:38 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:38.503+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:47.782149+0000 front 2024-05-21T22:07:47.781979+0000 (oldest deadline 2024-05-21T22:08:10.682108+0000) 2024-05-21T22:08:38.954 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:38 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:38.503+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:56.385736+0000 front 2024-05-21T22:07:56.386240+0000 (oldest deadline 2024-05-21T22:08:21.085525+0000) 2024-05-21T22:08:39.236 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:38 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:38.880+0000 7f452bd16700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:40.689332+0000 front 2024-05-21T22:07:40.689184+0000 (oldest deadline 2024-05-21T22:08:06.589360+0000) 2024-05-21T22:08:39.236 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:38 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:38.880+0000 7f452bd16700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:48.890238+0000 front 2024-05-21T22:07:48.890266+0000 (oldest deadline 2024-05-21T22:08:12.990394+0000) 2024-05-21T22:08:39.236 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:38 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:38.880+0000 7f452bd16700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:52.991143+0000 front 2024-05-21T22:07:52.991015+0000 (oldest deadline 2024-05-21T22:08:18.290930+0000) 2024-05-21T22:08:39.236 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:38 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:38.880+0000 7f452bd16700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-05-21T22:08:02.292736+0000 front 2024-05-21T22:08:02.292584+0000 (oldest deadline 2024-05-21T22:08:26.992601+0000) 2024-05-21T22:08:39.236 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:38 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8[55823]: 2024-05-21T22:08:38.880+0000 7f452bd16700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-21T22:08:12.595075+0000 front 2024-05-21T22:08:12.595118+0000 (oldest deadline 2024-05-21T22:08:37.895152+0000) 2024-05-21T22:08:39.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:38 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:38.891+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:28.956267+0000 front 2024-05-21T22:07:28.956225+0000 (oldest deadline 2024-05-21T22:07:55.356623+0000) 2024-05-21T22:08:39.237 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:38 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:38.891+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:38.857667+0000 front 2024-05-21T22:07:38.857566+0000 (oldest deadline 2024-05-21T22:08:04.757680+0000) 2024-05-21T22:08:39.237 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:38 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:38.891+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:44.758039+0000 front 2024-05-21T22:07:44.758082+0000 (oldest deadline 2024-05-21T22:08:10.658241+0000) 2024-05-21T22:08:39.237 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:38 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:38.891+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-05-21T22:08:04.160535+0000 front 2024-05-21T22:08:04.160468+0000 (oldest deadline 2024-05-21T22:08:29.460490+0000) 2024-05-21T22:08:39.237 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:38 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:38.891+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-21T22:08:10.561367+0000 front 2024-05-21T22:08:10.561307+0000 (oldest deadline 2024-05-21T22:08:35.861406+0000) 2024-05-21T22:08:39.536 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:39 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:39.091+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:30.508363+0000 front 2024-05-21T22:07:30.508149+0000 (oldest deadline 2024-05-21T22:07:52.808006+0000) 2024-05-21T22:08:39.536 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:39 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:39.091+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:41.009631+0000 front 2024-05-21T22:07:41.009565+0000 (oldest deadline 2024-05-21T22:08:05.709714+0000) 2024-05-21T22:08:39.536 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:39 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:39.091+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:49.011674+0000 front 2024-05-21T22:07:49.011868+0000 (oldest deadline 2024-05-21T22:08:13.711629+0000) 2024-05-21T22:08:39.536 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:39 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:39.091+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:53.712167+0000 front 2024-05-21T22:07:53.712374+0000 (oldest deadline 2024-05-21T22:08:18.412083+0000) 2024-05-21T22:08:39.536 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:39 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:39.091+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-05-21T22:08:03.113068+0000 front 2024-05-21T22:08:03.113103+0000 (oldest deadline 2024-05-21T22:08:29.513902+0000) 2024-05-21T22:08:39.536 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:39 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:39.091+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-21T22:08:13.615230+0000 front 2024-05-21T22:08:13.615079+0000 (oldest deadline 2024-05-21T22:08:37.714944+0000) 2024-05-21T22:08:39.899 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:39 smithi184 podman[74906]: 2024-05-21 22:08:39.628469203 +0000 UTC m=+5.551963448 container died 4eade7a10472dc2796c7c1239a863e67a2793cc3cfb7ed96f454f9dd0d90fba1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD) 2024-05-21T22:08:39.899 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:39 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:39.859+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:28.956267+0000 front 2024-05-21T22:07:28.956225+0000 (oldest deadline 2024-05-21T22:07:55.356623+0000) 2024-05-21T22:08:39.900 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:39 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:39.859+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:38.857667+0000 front 2024-05-21T22:07:38.857566+0000 (oldest deadline 2024-05-21T22:08:04.757680+0000) 2024-05-21T22:08:39.900 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:39 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:39.859+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:44.758039+0000 front 2024-05-21T22:07:44.758082+0000 (oldest deadline 2024-05-21T22:08:10.658241+0000) 2024-05-21T22:08:39.900 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:39 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:39.859+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-05-21T22:08:04.160535+0000 front 2024-05-21T22:08:04.160468+0000 (oldest deadline 2024-05-21T22:08:29.460490+0000) 2024-05-21T22:08:39.900 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:39 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:39.859+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-21T22:08:10.561367+0000 front 2024-05-21T22:08:10.561307+0000 (oldest deadline 2024-05-21T22:08:35.861406+0000) 2024-05-21T22:08:39.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:39 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:39.500+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:27.976452+0000 front 2024-05-21T22:07:27.976969+0000 (oldest deadline 2024-05-21T22:07:53.876086+0000) 2024-05-21T22:08:39.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:39 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:39.500+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:39.580127+0000 front 2024-05-21T22:07:39.580305+0000 (oldest deadline 2024-05-21T22:08:04.280362+0000) 2024-05-21T22:08:39.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:39 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:39.500+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:47.782149+0000 front 2024-05-21T22:07:47.781979+0000 (oldest deadline 2024-05-21T22:08:10.682108+0000) 2024-05-21T22:08:39.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:39 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:39.500+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:56.385736+0000 front 2024-05-21T22:07:56.386240+0000 (oldest deadline 2024-05-21T22:08:21.085525+0000) 2024-05-21T22:08:39.954 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:39 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:39.500+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-21T22:08:10.487914+0000 front 2024-05-21T22:08:10.488029+0000 (oldest deadline 2024-05-21T22:08:39.288193+0000) 2024-05-21T22:08:40.236 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:39 smithi184 podman[74906]: 2024-05-21 22:08:39.989980122 +0000 UTC m=+5.913474360 container cleanup 4eade7a10472dc2796c7c1239a863e67a2793cc3cfb7ed96f454f9dd0d90fba1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-21T22:08:40.236 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:39 smithi184 bash[74906]: ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8 2024-05-21T22:08:40.536 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:40 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:40.078+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:30.508363+0000 front 2024-05-21T22:07:30.508149+0000 (oldest deadline 2024-05-21T22:07:52.808006+0000) 2024-05-21T22:08:40.536 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:40 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:40.078+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:41.009631+0000 front 2024-05-21T22:07:41.009565+0000 (oldest deadline 2024-05-21T22:08:05.709714+0000) 2024-05-21T22:08:40.536 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:40 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:40.078+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:49.011674+0000 front 2024-05-21T22:07:49.011868+0000 (oldest deadline 2024-05-21T22:08:13.711629+0000) 2024-05-21T22:08:40.536 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:40 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:40.078+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:53.712167+0000 front 2024-05-21T22:07:53.712374+0000 (oldest deadline 2024-05-21T22:08:18.412083+0000) 2024-05-21T22:08:40.537 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:40 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:40.078+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-05-21T22:08:03.113068+0000 front 2024-05-21T22:08:03.113103+0000 (oldest deadline 2024-05-21T22:08:29.513902+0000) 2024-05-21T22:08:40.537 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:40 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:40.078+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-21T22:08:13.615230+0000 front 2024-05-21T22:08:13.615079+0000 (oldest deadline 2024-05-21T22:08:37.714944+0000) 2024-05-21T22:08:40.605 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:40 smithi184 podman[74922]: 2024-05-21 22:08:40.241069933 +0000 UTC m=+0.608593162 container remove 4eade7a10472dc2796c7c1239a863e67a2793cc3cfb7ed96f454f9dd0d90fba1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0) 2024-05-21T22:08:40.900 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:40 smithi184 podman[75013]: 2024-05-21 22:08:40.508474568 +0000 UTC m=+0.017230675 image pull d6bc8f31cbc1a7165ca248fe9ba0b06ea14b30f46e1ea33ced1a5238e247af1f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28 2024-05-21T22:08:40.900 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:40 smithi184 podman[75013]: 2024-05-21 22:08:40.617322066 +0000 UTC m=+0.126078152 container create c51a95d20faf7732204f458cafc9a940bad36b490f8dc6cece4e1ff43648aad5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-21T22:08:40.900 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:40 smithi184 podman[75013]: 2024-05-21 22:08:40.844448314 +0000 UTC m=+0.353204402 container init c51a95d20faf7732204f458cafc9a940bad36b490f8dc6cece4e1ff43648aad5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, io.buildah.version=1.33.5) 2024-05-21T22:08:40.900 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:40 smithi184 podman[75013]: 2024-05-21 22:08:40.847514597 +0000 UTC m=+0.356270684 container start c51a95d20faf7732204f458cafc9a940bad36b490f8dc6cece4e1ff43648aad5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea) 2024-05-21T22:08:40.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:40 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:40.498+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:27.976452+0000 front 2024-05-21T22:07:27.976969+0000 (oldest deadline 2024-05-21T22:07:53.876086+0000) 2024-05-21T22:08:40.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:40 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:40.498+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:39.580127+0000 front 2024-05-21T22:07:39.580305+0000 (oldest deadline 2024-05-21T22:08:04.280362+0000) 2024-05-21T22:08:40.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:40 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:40.498+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:47.782149+0000 front 2024-05-21T22:07:47.781979+0000 (oldest deadline 2024-05-21T22:08:10.682108+0000) 2024-05-21T22:08:40.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:40 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:40.498+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:56.385736+0000 front 2024-05-21T22:07:56.386240+0000 (oldest deadline 2024-05-21T22:08:21.085525+0000) 2024-05-21T22:08:40.954 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:40 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:40.498+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-21T22:08:10.487914+0000 front 2024-05-21T22:08:10.488029+0000 (oldest deadline 2024-05-21T22:08:39.288193+0000) 2024-05-21T22:08:41.176 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:40 smithi184 podman[75013]: 2024-05-21 22:08:40.936034397 +0000 UTC m=+0.444790493 container attach c51a95d20faf7732204f458cafc9a940bad36b490f8dc6cece4e1ff43648aad5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-05-21T22:08:41.177 INFO:journalctl@ceph.osd.8.smithi184.stdout:May 21 22:08:41 smithi184 podman[75013]: 2024-05-21 22:08:41.145063098 +0000 UTC m=+0.653819199 container died c51a95d20faf7732204f458cafc9a940bad36b490f8dc6cece4e1ff43648aad5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, maintainer=Guillaume Abrioux ) 2024-05-21T22:08:41.177 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:40 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:40.899+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:28.956267+0000 front 2024-05-21T22:07:28.956225+0000 (oldest deadline 2024-05-21T22:07:55.356623+0000) 2024-05-21T22:08:41.177 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:40 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:40.899+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:38.857667+0000 front 2024-05-21T22:07:38.857566+0000 (oldest deadline 2024-05-21T22:08:04.757680+0000) 2024-05-21T22:08:41.177 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:40 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:40.899+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:44.758039+0000 front 2024-05-21T22:07:44.758082+0000 (oldest deadline 2024-05-21T22:08:10.658241+0000) 2024-05-21T22:08:41.177 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:40 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:40.899+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-05-21T22:08:04.160535+0000 front 2024-05-21T22:08:04.160468+0000 (oldest deadline 2024-05-21T22:08:29.460490+0000) 2024-05-21T22:08:41.177 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:40 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:40.899+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-21T22:08:10.561367+0000 front 2024-05-21T22:08:10.561307+0000 (oldest deadline 2024-05-21T22:08:35.861406+0000) 2024-05-21T22:08:41.536 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:41 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:41.058+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:30.508363+0000 front 2024-05-21T22:07:30.508149+0000 (oldest deadline 2024-05-21T22:07:52.808006+0000) 2024-05-21T22:08:41.536 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:41 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:41.058+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:41.009631+0000 front 2024-05-21T22:07:41.009565+0000 (oldest deadline 2024-05-21T22:08:05.709714+0000) 2024-05-21T22:08:41.536 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:41 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:41.058+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:49.011674+0000 front 2024-05-21T22:07:49.011868+0000 (oldest deadline 2024-05-21T22:08:13.711629+0000) 2024-05-21T22:08:41.536 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:41 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:41.058+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:53.712167+0000 front 2024-05-21T22:07:53.712374+0000 (oldest deadline 2024-05-21T22:08:18.412083+0000) 2024-05-21T22:08:41.536 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:41 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:41.058+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-05-21T22:08:03.113068+0000 front 2024-05-21T22:08:03.113103+0000 (oldest deadline 2024-05-21T22:08:29.513902+0000) 2024-05-21T22:08:41.536 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:41 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:41.058+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-21T22:08:13.615230+0000 front 2024-05-21T22:08:13.615079+0000 (oldest deadline 2024-05-21T22:08:37.714944+0000) 2024-05-21T22:08:41.764 DEBUG:teuthology.orchestra.run.smithi184:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@osd.8.service' 2024-05-21T22:08:41.802 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T22:08:41.802 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-05-21T22:08:41.802 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-05-21T22:08:41.802 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@osd.9 2024-05-21T22:08:41.882 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:41 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:41.473+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:27.976452+0000 front 2024-05-21T22:07:27.976969+0000 (oldest deadline 2024-05-21T22:07:53.876086+0000) 2024-05-21T22:08:41.882 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:41 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:41.473+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:39.580127+0000 front 2024-05-21T22:07:39.580305+0000 (oldest deadline 2024-05-21T22:08:04.280362+0000) 2024-05-21T22:08:41.882 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:41 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:41.473+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:47.782149+0000 front 2024-05-21T22:07:47.781979+0000 (oldest deadline 2024-05-21T22:08:10.682108+0000) 2024-05-21T22:08:41.882 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:41 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:41.473+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:56.385736+0000 front 2024-05-21T22:07:56.386240+0000 (oldest deadline 2024-05-21T22:08:21.085525+0000) 2024-05-21T22:08:41.882 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:41 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:41.473+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-21T22:08:10.487914+0000 front 2024-05-21T22:08:10.488029+0000 (oldest deadline 2024-05-21T22:08:39.288193+0000) 2024-05-21T22:08:42.203 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:41 smithi151 systemd[1]: Stopping Ceph osd.9 for c8c7a2e6-17ba-11ef-bc9a-c7b262605968... 2024-05-21T22:08:42.235 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:41 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:41.895+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:28.956267+0000 front 2024-05-21T22:07:28.956225+0000 (oldest deadline 2024-05-21T22:07:55.356623+0000) 2024-05-21T22:08:42.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:41 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:41.895+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:38.857667+0000 front 2024-05-21T22:07:38.857566+0000 (oldest deadline 2024-05-21T22:08:04.757680+0000) 2024-05-21T22:08:42.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:41 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:41.895+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:44.758039+0000 front 2024-05-21T22:07:44.758082+0000 (oldest deadline 2024-05-21T22:08:10.658241+0000) 2024-05-21T22:08:42.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:41 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:41.895+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-05-21T22:08:04.160535+0000 front 2024-05-21T22:08:04.160468+0000 (oldest deadline 2024-05-21T22:08:29.460490+0000) 2024-05-21T22:08:42.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:41 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:41.895+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-21T22:08:10.561367+0000 front 2024-05-21T22:08:10.561307+0000 (oldest deadline 2024-05-21T22:08:35.861406+0000) 2024-05-21T22:08:42.437 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:42 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:42.057+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:30.508363+0000 front 2024-05-21T22:07:30.508149+0000 (oldest deadline 2024-05-21T22:07:52.808006+0000) 2024-05-21T22:08:42.437 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:42 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:42.057+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:41.009631+0000 front 2024-05-21T22:07:41.009565+0000 (oldest deadline 2024-05-21T22:08:05.709714+0000) 2024-05-21T22:08:42.437 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:42 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:42.057+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:49.011674+0000 front 2024-05-21T22:07:49.011868+0000 (oldest deadline 2024-05-21T22:08:13.711629+0000) 2024-05-21T22:08:42.437 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:42 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:42.057+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:53.712167+0000 front 2024-05-21T22:07:53.712374+0000 (oldest deadline 2024-05-21T22:08:18.412083+0000) 2024-05-21T22:08:42.438 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:42 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:42.057+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-05-21T22:08:03.113068+0000 front 2024-05-21T22:08:03.113103+0000 (oldest deadline 2024-05-21T22:08:29.513902+0000) 2024-05-21T22:08:42.438 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:42 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:42.057+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-21T22:08:13.615230+0000 front 2024-05-21T22:08:13.615079+0000 (oldest deadline 2024-05-21T22:08:37.714944+0000) 2024-05-21T22:08:42.703 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:42 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:42.375+0000 7f5973000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T22:08:42.703 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:42 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:42.375+0000 7f5973000700 -1 osd.9 94 *** Got signal Terminated *** 2024-05-21T22:08:42.704 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:42 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:42.375+0000 7f5973000700 -1 osd.9 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-21T22:08:42.704 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:42 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:42.495+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:27.976452+0000 front 2024-05-21T22:07:27.976969+0000 (oldest deadline 2024-05-21T22:07:53.876086+0000) 2024-05-21T22:08:42.704 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:42 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:42.495+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:39.580127+0000 front 2024-05-21T22:07:39.580305+0000 (oldest deadline 2024-05-21T22:08:04.280362+0000) 2024-05-21T22:08:42.704 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:42 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:42.495+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:47.782149+0000 front 2024-05-21T22:07:47.781979+0000 (oldest deadline 2024-05-21T22:08:10.682108+0000) 2024-05-21T22:08:42.704 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:42 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:42.495+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:56.385736+0000 front 2024-05-21T22:07:56.386240+0000 (oldest deadline 2024-05-21T22:08:21.085525+0000) 2024-05-21T22:08:42.704 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:42 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:42.495+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-21T22:08:10.487914+0000 front 2024-05-21T22:08:10.488029+0000 (oldest deadline 2024-05-21T22:08:39.288193+0000) 2024-05-21T22:08:43.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:42 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:42.884+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:28.956267+0000 front 2024-05-21T22:07:28.956225+0000 (oldest deadline 2024-05-21T22:07:55.356623+0000) 2024-05-21T22:08:43.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:42 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:42.884+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:38.857667+0000 front 2024-05-21T22:07:38.857566+0000 (oldest deadline 2024-05-21T22:08:04.757680+0000) 2024-05-21T22:08:43.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:42 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:42.884+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:44.758039+0000 front 2024-05-21T22:07:44.758082+0000 (oldest deadline 2024-05-21T22:08:10.658241+0000) 2024-05-21T22:08:43.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:42 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:42.884+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-05-21T22:08:04.160535+0000 front 2024-05-21T22:08:04.160468+0000 (oldest deadline 2024-05-21T22:08:29.460490+0000) 2024-05-21T22:08:43.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:42 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:42.884+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-21T22:08:10.561367+0000 front 2024-05-21T22:08:10.561307+0000 (oldest deadline 2024-05-21T22:08:35.861406+0000) 2024-05-21T22:08:43.536 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:43 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:43.041+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:30.508363+0000 front 2024-05-21T22:07:30.508149+0000 (oldest deadline 2024-05-21T22:07:52.808006+0000) 2024-05-21T22:08:43.536 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:43 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:43.041+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:41.009631+0000 front 2024-05-21T22:07:41.009565+0000 (oldest deadline 2024-05-21T22:08:05.709714+0000) 2024-05-21T22:08:43.536 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:43 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:43.041+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:49.011674+0000 front 2024-05-21T22:07:49.011868+0000 (oldest deadline 2024-05-21T22:08:13.711629+0000) 2024-05-21T22:08:43.536 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:43 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:43.041+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:53.712167+0000 front 2024-05-21T22:07:53.712374+0000 (oldest deadline 2024-05-21T22:08:18.412083+0000) 2024-05-21T22:08:43.536 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:43 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:43.041+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-05-21T22:08:03.113068+0000 front 2024-05-21T22:08:03.113103+0000 (oldest deadline 2024-05-21T22:08:29.513902+0000) 2024-05-21T22:08:43.536 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:43 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:43.041+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-21T22:08:13.615230+0000 front 2024-05-21T22:08:13.615079+0000 (oldest deadline 2024-05-21T22:08:37.714944+0000) 2024-05-21T22:08:43.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:43 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:43.542+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:27.976452+0000 front 2024-05-21T22:07:27.976969+0000 (oldest deadline 2024-05-21T22:07:53.876086+0000) 2024-05-21T22:08:43.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:43 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:43.542+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:39.580127+0000 front 2024-05-21T22:07:39.580305+0000 (oldest deadline 2024-05-21T22:08:04.280362+0000) 2024-05-21T22:08:43.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:43 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:43.542+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:47.782149+0000 front 2024-05-21T22:07:47.781979+0000 (oldest deadline 2024-05-21T22:08:10.682108+0000) 2024-05-21T22:08:43.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:43 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:43.542+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:56.385736+0000 front 2024-05-21T22:07:56.386240+0000 (oldest deadline 2024-05-21T22:08:21.085525+0000) 2024-05-21T22:08:43.954 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:43 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:43.542+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-21T22:08:10.487914+0000 front 2024-05-21T22:08:10.488029+0000 (oldest deadline 2024-05-21T22:08:39.288193+0000) 2024-05-21T22:08:43.954 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:43 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:43.542+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.151:6822 osd.6 since back 2024-05-21T22:08:19.289514+0000 front 2024-05-21T22:08:19.289358+0000 (oldest deadline 2024-05-21T22:08:43.388963+0000) 2024-05-21T22:08:44.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:43 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:43.877+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:28.956267+0000 front 2024-05-21T22:07:28.956225+0000 (oldest deadline 2024-05-21T22:07:55.356623+0000) 2024-05-21T22:08:44.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:43 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:43.877+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:38.857667+0000 front 2024-05-21T22:07:38.857566+0000 (oldest deadline 2024-05-21T22:08:04.757680+0000) 2024-05-21T22:08:44.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:43 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:43.877+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:44.758039+0000 front 2024-05-21T22:07:44.758082+0000 (oldest deadline 2024-05-21T22:08:10.658241+0000) 2024-05-21T22:08:44.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:43 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:43.877+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-05-21T22:08:04.160535+0000 front 2024-05-21T22:08:04.160468+0000 (oldest deadline 2024-05-21T22:08:29.460490+0000) 2024-05-21T22:08:44.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:43 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:43.877+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-21T22:08:10.561367+0000 front 2024-05-21T22:08:10.561307+0000 (oldest deadline 2024-05-21T22:08:35.861406+0000) 2024-05-21T22:08:44.536 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:44 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:44.079+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:30.508363+0000 front 2024-05-21T22:07:30.508149+0000 (oldest deadline 2024-05-21T22:07:52.808006+0000) 2024-05-21T22:08:44.536 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:44 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:44.079+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:41.009631+0000 front 2024-05-21T22:07:41.009565+0000 (oldest deadline 2024-05-21T22:08:05.709714+0000) 2024-05-21T22:08:44.536 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:44 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:44.079+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:49.011674+0000 front 2024-05-21T22:07:49.011868+0000 (oldest deadline 2024-05-21T22:08:13.711629+0000) 2024-05-21T22:08:44.536 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:44 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:44.079+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:53.712167+0000 front 2024-05-21T22:07:53.712374+0000 (oldest deadline 2024-05-21T22:08:18.412083+0000) 2024-05-21T22:08:44.536 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:44 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:44.079+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-05-21T22:08:03.113068+0000 front 2024-05-21T22:08:03.113103+0000 (oldest deadline 2024-05-21T22:08:29.513902+0000) 2024-05-21T22:08:44.536 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:44 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:44.079+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-21T22:08:13.615230+0000 front 2024-05-21T22:08:13.615079+0000 (oldest deadline 2024-05-21T22:08:37.714944+0000) 2024-05-21T22:08:44.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:44 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:44.503+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:27.976452+0000 front 2024-05-21T22:07:27.976969+0000 (oldest deadline 2024-05-21T22:07:53.876086+0000) 2024-05-21T22:08:44.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:44 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:44.503+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:39.580127+0000 front 2024-05-21T22:07:39.580305+0000 (oldest deadline 2024-05-21T22:08:04.280362+0000) 2024-05-21T22:08:44.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:44 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:44.503+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:47.782149+0000 front 2024-05-21T22:07:47.781979+0000 (oldest deadline 2024-05-21T22:08:10.682108+0000) 2024-05-21T22:08:44.954 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:44 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:44.503+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:56.385736+0000 front 2024-05-21T22:07:56.386240+0000 (oldest deadline 2024-05-21T22:08:21.085525+0000) 2024-05-21T22:08:44.954 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:44 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:44.503+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-21T22:08:10.487914+0000 front 2024-05-21T22:08:10.488029+0000 (oldest deadline 2024-05-21T22:08:39.288193+0000) 2024-05-21T22:08:44.954 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:44 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:44.503+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.151:6822 osd.6 since back 2024-05-21T22:08:19.289514+0000 front 2024-05-21T22:08:19.289358+0000 (oldest deadline 2024-05-21T22:08:43.388963+0000) 2024-05-21T22:08:45.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:44 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:44.899+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:28.956267+0000 front 2024-05-21T22:07:28.956225+0000 (oldest deadline 2024-05-21T22:07:55.356623+0000) 2024-05-21T22:08:45.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:44 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:44.899+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:38.857667+0000 front 2024-05-21T22:07:38.857566+0000 (oldest deadline 2024-05-21T22:08:04.757680+0000) 2024-05-21T22:08:45.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:44 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:44.899+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:44.758039+0000 front 2024-05-21T22:07:44.758082+0000 (oldest deadline 2024-05-21T22:08:10.658241+0000) 2024-05-21T22:08:45.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:44 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:44.899+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-05-21T22:08:04.160535+0000 front 2024-05-21T22:08:04.160468+0000 (oldest deadline 2024-05-21T22:08:29.460490+0000) 2024-05-21T22:08:45.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:44 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:44.899+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-21T22:08:10.561367+0000 front 2024-05-21T22:08:10.561307+0000 (oldest deadline 2024-05-21T22:08:35.861406+0000) 2024-05-21T22:08:45.536 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:45 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:45.077+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:30.508363+0000 front 2024-05-21T22:07:30.508149+0000 (oldest deadline 2024-05-21T22:07:52.808006+0000) 2024-05-21T22:08:45.536 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:45 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:45.077+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:41.009631+0000 front 2024-05-21T22:07:41.009565+0000 (oldest deadline 2024-05-21T22:08:05.709714+0000) 2024-05-21T22:08:45.536 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:45 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:45.077+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:49.011674+0000 front 2024-05-21T22:07:49.011868+0000 (oldest deadline 2024-05-21T22:08:13.711629+0000) 2024-05-21T22:08:45.536 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:45 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:45.077+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:53.712167+0000 front 2024-05-21T22:07:53.712374+0000 (oldest deadline 2024-05-21T22:08:18.412083+0000) 2024-05-21T22:08:45.536 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:45 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:45.077+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-05-21T22:08:03.113068+0000 front 2024-05-21T22:08:03.113103+0000 (oldest deadline 2024-05-21T22:08:29.513902+0000) 2024-05-21T22:08:45.536 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:45 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:45.077+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-21T22:08:13.615230+0000 front 2024-05-21T22:08:13.615079+0000 (oldest deadline 2024-05-21T22:08:37.714944+0000) 2024-05-21T22:08:45.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:45 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:45.533+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:27.976452+0000 front 2024-05-21T22:07:27.976969+0000 (oldest deadline 2024-05-21T22:07:53.876086+0000) 2024-05-21T22:08:45.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:45 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:45.533+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:39.580127+0000 front 2024-05-21T22:07:39.580305+0000 (oldest deadline 2024-05-21T22:08:04.280362+0000) 2024-05-21T22:08:45.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:45 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:45.533+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:47.782149+0000 front 2024-05-21T22:07:47.781979+0000 (oldest deadline 2024-05-21T22:08:10.682108+0000) 2024-05-21T22:08:45.954 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:45 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:45.533+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:56.385736+0000 front 2024-05-21T22:07:56.386240+0000 (oldest deadline 2024-05-21T22:08:21.085525+0000) 2024-05-21T22:08:45.954 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:45 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:45.533+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-21T22:08:10.487914+0000 front 2024-05-21T22:08:10.488029+0000 (oldest deadline 2024-05-21T22:08:39.288193+0000) 2024-05-21T22:08:45.954 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:45 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:45.533+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.151:6822 osd.6 since back 2024-05-21T22:08:19.289514+0000 front 2024-05-21T22:08:19.289358+0000 (oldest deadline 2024-05-21T22:08:43.388963+0000) 2024-05-21T22:08:46.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:45 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:45.912+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:28.956267+0000 front 2024-05-21T22:07:28.956225+0000 (oldest deadline 2024-05-21T22:07:55.356623+0000) 2024-05-21T22:08:46.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:45 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:45.912+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:38.857667+0000 front 2024-05-21T22:07:38.857566+0000 (oldest deadline 2024-05-21T22:08:04.757680+0000) 2024-05-21T22:08:46.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:45 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:45.912+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:44.758039+0000 front 2024-05-21T22:07:44.758082+0000 (oldest deadline 2024-05-21T22:08:10.658241+0000) 2024-05-21T22:08:46.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:45 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:45.912+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-05-21T22:08:04.160535+0000 front 2024-05-21T22:08:04.160468+0000 (oldest deadline 2024-05-21T22:08:29.460490+0000) 2024-05-21T22:08:46.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:45 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:45.912+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-21T22:08:10.561367+0000 front 2024-05-21T22:08:10.561307+0000 (oldest deadline 2024-05-21T22:08:35.861406+0000) 2024-05-21T22:08:46.536 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:46 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:46.106+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:30.508363+0000 front 2024-05-21T22:07:30.508149+0000 (oldest deadline 2024-05-21T22:07:52.808006+0000) 2024-05-21T22:08:46.536 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:46 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:46.106+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:41.009631+0000 front 2024-05-21T22:07:41.009565+0000 (oldest deadline 2024-05-21T22:08:05.709714+0000) 2024-05-21T22:08:46.536 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:46 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:46.106+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:49.011674+0000 front 2024-05-21T22:07:49.011868+0000 (oldest deadline 2024-05-21T22:08:13.711629+0000) 2024-05-21T22:08:46.536 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:46 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:46.106+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:53.712167+0000 front 2024-05-21T22:07:53.712374+0000 (oldest deadline 2024-05-21T22:08:18.412083+0000) 2024-05-21T22:08:46.536 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:46 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:46.106+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-05-21T22:08:03.113068+0000 front 2024-05-21T22:08:03.113103+0000 (oldest deadline 2024-05-21T22:08:29.513902+0000) 2024-05-21T22:08:46.537 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:46 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:46.106+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-21T22:08:13.615230+0000 front 2024-05-21T22:08:13.615079+0000 (oldest deadline 2024-05-21T22:08:37.714944+0000) 2024-05-21T22:08:46.537 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:46 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:46.106+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6822 osd.6 since back 2024-05-21T22:08:20.016394+0000 front 2024-05-21T22:08:20.016252+0000 (oldest deadline 2024-05-21T22:08:45.915946+0000) 2024-05-21T22:08:46.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:46 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:46.554+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:27.976452+0000 front 2024-05-21T22:07:27.976969+0000 (oldest deadline 2024-05-21T22:07:53.876086+0000) 2024-05-21T22:08:46.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:46 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:46.554+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:39.580127+0000 front 2024-05-21T22:07:39.580305+0000 (oldest deadline 2024-05-21T22:08:04.280362+0000) 2024-05-21T22:08:46.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:46 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:46.554+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:47.782149+0000 front 2024-05-21T22:07:47.781979+0000 (oldest deadline 2024-05-21T22:08:10.682108+0000) 2024-05-21T22:08:46.954 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:46 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:46.554+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:56.385736+0000 front 2024-05-21T22:07:56.386240+0000 (oldest deadline 2024-05-21T22:08:21.085525+0000) 2024-05-21T22:08:46.954 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:46 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:46.554+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-21T22:08:10.487914+0000 front 2024-05-21T22:08:10.488029+0000 (oldest deadline 2024-05-21T22:08:39.288193+0000) 2024-05-21T22:08:46.954 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:46 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9[64086]: 2024-05-21T22:08:46.554+0000 7f596e800700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.151:6822 osd.6 since back 2024-05-21T22:08:19.289514+0000 front 2024-05-21T22:08:19.289358+0000 (oldest deadline 2024-05-21T22:08:43.388963+0000) 2024-05-21T22:08:47.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:46 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:46.927+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:28.956267+0000 front 2024-05-21T22:07:28.956225+0000 (oldest deadline 2024-05-21T22:07:55.356623+0000) 2024-05-21T22:08:47.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:46 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:46.927+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:38.857667+0000 front 2024-05-21T22:07:38.857566+0000 (oldest deadline 2024-05-21T22:08:04.757680+0000) 2024-05-21T22:08:47.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:46 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:46.927+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:44.758039+0000 front 2024-05-21T22:07:44.758082+0000 (oldest deadline 2024-05-21T22:08:10.658241+0000) 2024-05-21T22:08:47.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:46 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:46.927+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-05-21T22:08:04.160535+0000 front 2024-05-21T22:08:04.160468+0000 (oldest deadline 2024-05-21T22:08:29.460490+0000) 2024-05-21T22:08:47.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:46 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:46.927+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-21T22:08:10.561367+0000 front 2024-05-21T22:08:10.561307+0000 (oldest deadline 2024-05-21T22:08:35.861406+0000) 2024-05-21T22:08:47.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:46 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:46.927+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.151:6822 osd.6 since back 2024-05-21T22:08:20.463172+0000 front 2024-05-21T22:08:20.463034+0000 (oldest deadline 2024-05-21T22:08:46.362860+0000) 2024-05-21T22:08:47.536 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:47 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:47.077+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:30.508363+0000 front 2024-05-21T22:07:30.508149+0000 (oldest deadline 2024-05-21T22:07:52.808006+0000) 2024-05-21T22:08:47.536 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:47 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:47.077+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:41.009631+0000 front 2024-05-21T22:07:41.009565+0000 (oldest deadline 2024-05-21T22:08:05.709714+0000) 2024-05-21T22:08:47.536 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:47 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:47.077+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:49.011674+0000 front 2024-05-21T22:07:49.011868+0000 (oldest deadline 2024-05-21T22:08:13.711629+0000) 2024-05-21T22:08:47.536 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:47 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:47.077+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:53.712167+0000 front 2024-05-21T22:07:53.712374+0000 (oldest deadline 2024-05-21T22:08:18.412083+0000) 2024-05-21T22:08:47.536 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:47 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:47.077+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-05-21T22:08:03.113068+0000 front 2024-05-21T22:08:03.113103+0000 (oldest deadline 2024-05-21T22:08:29.513902+0000) 2024-05-21T22:08:47.537 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:47 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:47.077+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-21T22:08:13.615230+0000 front 2024-05-21T22:08:13.615079+0000 (oldest deadline 2024-05-21T22:08:37.714944+0000) 2024-05-21T22:08:47.537 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:47 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:47.077+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6822 osd.6 since back 2024-05-21T22:08:20.016394+0000 front 2024-05-21T22:08:20.016252+0000 (oldest deadline 2024-05-21T22:08:45.915946+0000) 2024-05-21T22:08:48.203 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:47 smithi151 podman[121923]: 2024-05-21 22:08:47.885439147 +0000 UTC m=+5.648235460 container died c0d77e7b9a7ab134537f6f25217e6bb28ea379ff39f7a4df973ee9df64bf06b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-21T22:08:48.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:47 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:47.889+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:28.956267+0000 front 2024-05-21T22:07:28.956225+0000 (oldest deadline 2024-05-21T22:07:55.356623+0000) 2024-05-21T22:08:48.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:47 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:47.889+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:38.857667+0000 front 2024-05-21T22:07:38.857566+0000 (oldest deadline 2024-05-21T22:08:04.757680+0000) 2024-05-21T22:08:48.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:47 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:47.889+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:44.758039+0000 front 2024-05-21T22:07:44.758082+0000 (oldest deadline 2024-05-21T22:08:10.658241+0000) 2024-05-21T22:08:48.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:47 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:47.889+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-05-21T22:08:04.160535+0000 front 2024-05-21T22:08:04.160468+0000 (oldest deadline 2024-05-21T22:08:29.460490+0000) 2024-05-21T22:08:48.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:47 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:47.889+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-21T22:08:10.561367+0000 front 2024-05-21T22:08:10.561307+0000 (oldest deadline 2024-05-21T22:08:35.861406+0000) 2024-05-21T22:08:48.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:47 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:47.889+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.151:6822 osd.6 since back 2024-05-21T22:08:20.463172+0000 front 2024-05-21T22:08:20.463034+0000 (oldest deadline 2024-05-21T22:08:46.362860+0000) 2024-05-21T22:08:48.469 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:48 smithi151 podman[121923]: 2024-05-21 22:08:48.452242122 +0000 UTC m=+6.215038402 container cleanup c0d77e7b9a7ab134537f6f25217e6bb28ea379ff39f7a4df973ee9df64bf06b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-05-21T22:08:48.469 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:48 smithi151 bash[121923]: ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9 2024-05-21T22:08:48.536 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:48 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:48.077+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:30.508363+0000 front 2024-05-21T22:07:30.508149+0000 (oldest deadline 2024-05-21T22:07:52.808006+0000) 2024-05-21T22:08:48.536 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:48 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:48.077+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:41.009631+0000 front 2024-05-21T22:07:41.009565+0000 (oldest deadline 2024-05-21T22:08:05.709714+0000) 2024-05-21T22:08:48.536 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:48 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:48.077+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:49.011674+0000 front 2024-05-21T22:07:49.011868+0000 (oldest deadline 2024-05-21T22:08:13.711629+0000) 2024-05-21T22:08:48.536 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:48 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:48.077+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:53.712167+0000 front 2024-05-21T22:07:53.712374+0000 (oldest deadline 2024-05-21T22:08:18.412083+0000) 2024-05-21T22:08:48.537 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:48 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:48.078+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-05-21T22:08:03.113068+0000 front 2024-05-21T22:08:03.113103+0000 (oldest deadline 2024-05-21T22:08:29.513902+0000) 2024-05-21T22:08:48.537 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:48 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:48.078+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-21T22:08:13.615230+0000 front 2024-05-21T22:08:13.615079+0000 (oldest deadline 2024-05-21T22:08:37.714944+0000) 2024-05-21T22:08:48.537 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:48 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:48.078+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6822 osd.6 since back 2024-05-21T22:08:20.016394+0000 front 2024-05-21T22:08:20.016252+0000 (oldest deadline 2024-05-21T22:08:45.915946+0000) 2024-05-21T22:08:48.953 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:48 smithi151 podman[121940]: 2024-05-21 22:08:48.662888784 +0000 UTC m=+0.766479082 container remove c0d77e7b9a7ab134537f6f25217e6bb28ea379ff39f7a4df973ee9df64bf06b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-05-21T22:08:49.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:48 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:48.889+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:28.956267+0000 front 2024-05-21T22:07:28.956225+0000 (oldest deadline 2024-05-21T22:07:55.356623+0000) 2024-05-21T22:08:49.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:48 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:48.889+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:38.857667+0000 front 2024-05-21T22:07:38.857566+0000 (oldest deadline 2024-05-21T22:08:04.757680+0000) 2024-05-21T22:08:49.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:48 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:48.889+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:44.758039+0000 front 2024-05-21T22:07:44.758082+0000 (oldest deadline 2024-05-21T22:08:10.658241+0000) 2024-05-21T22:08:49.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:48 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:48.889+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-05-21T22:08:04.160535+0000 front 2024-05-21T22:08:04.160468+0000 (oldest deadline 2024-05-21T22:08:29.460490+0000) 2024-05-21T22:08:49.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:48 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:48.889+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-21T22:08:10.561367+0000 front 2024-05-21T22:08:10.561307+0000 (oldest deadline 2024-05-21T22:08:35.861406+0000) 2024-05-21T22:08:49.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:48 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:48.889+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.151:6822 osd.6 since back 2024-05-21T22:08:20.463172+0000 front 2024-05-21T22:08:20.463034+0000 (oldest deadline 2024-05-21T22:08:46.362860+0000) 2024-05-21T22:08:49.536 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:49 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:49.050+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:30.508363+0000 front 2024-05-21T22:07:30.508149+0000 (oldest deadline 2024-05-21T22:07:52.808006+0000) 2024-05-21T22:08:49.536 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:49 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:49.050+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:41.009631+0000 front 2024-05-21T22:07:41.009565+0000 (oldest deadline 2024-05-21T22:08:05.709714+0000) 2024-05-21T22:08:49.536 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:49 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:49.050+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:49.011674+0000 front 2024-05-21T22:07:49.011868+0000 (oldest deadline 2024-05-21T22:08:13.711629+0000) 2024-05-21T22:08:49.536 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:49 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:49.050+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:53.712167+0000 front 2024-05-21T22:07:53.712374+0000 (oldest deadline 2024-05-21T22:08:18.412083+0000) 2024-05-21T22:08:49.536 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:49 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:49.050+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-05-21T22:08:03.113068+0000 front 2024-05-21T22:08:03.113103+0000 (oldest deadline 2024-05-21T22:08:29.513902+0000) 2024-05-21T22:08:49.536 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:49 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:49.050+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-21T22:08:13.615230+0000 front 2024-05-21T22:08:13.615079+0000 (oldest deadline 2024-05-21T22:08:37.714944+0000) 2024-05-21T22:08:49.536 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:49 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:49.050+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6822 osd.6 since back 2024-05-21T22:08:20.016394+0000 front 2024-05-21T22:08:20.016252+0000 (oldest deadline 2024-05-21T22:08:45.915946+0000) 2024-05-21T22:08:49.743 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:49 smithi151 podman[122026]: 2024-05-21 22:08:49.377458007 +0000 UTC m=+0.031678403 image pull d6bc8f31cbc1a7165ca248fe9ba0b06ea14b30f46e1ea33ced1a5238e247af1f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28 2024-05-21T22:08:49.743 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:49 smithi151 podman[122026]: 2024-05-21 22:08:49.488927273 +0000 UTC m=+0.143147693 container create 99b061789bd4d997602d98898ccc8e3b41cc99ded94b5e076c2645bc9498760f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-05-21T22:08:49.743 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:49 smithi151 podman[122026]: 2024-05-21 22:08:49.742881943 +0000 UTC m=+0.397102396 container init 99b061789bd4d997602d98898ccc8e3b41cc99ded94b5e076c2645bc9498760f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-21T22:08:50.204 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:49 smithi151 podman[122026]: 2024-05-21 22:08:49.753606633 +0000 UTC m=+0.407827022 container start 99b061789bd4d997602d98898ccc8e3b41cc99ded94b5e076c2645bc9498760f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9-deactivate, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-21T22:08:50.204 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:49 smithi151 podman[122026]: 2024-05-21 22:08:49.833376123 +0000 UTC m=+0.487596610 container attach 99b061789bd4d997602d98898ccc8e3b41cc99ded94b5e076c2645bc9498760f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-05-21T22:08:50.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:49 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:49.858+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:28.956267+0000 front 2024-05-21T22:07:28.956225+0000 (oldest deadline 2024-05-21T22:07:55.356623+0000) 2024-05-21T22:08:50.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:49 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:49.858+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:38.857667+0000 front 2024-05-21T22:07:38.857566+0000 (oldest deadline 2024-05-21T22:08:04.757680+0000) 2024-05-21T22:08:50.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:49 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:49.858+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:44.758039+0000 front 2024-05-21T22:07:44.758082+0000 (oldest deadline 2024-05-21T22:08:10.658241+0000) 2024-05-21T22:08:50.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:49 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:49.858+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-05-21T22:08:04.160535+0000 front 2024-05-21T22:08:04.160468+0000 (oldest deadline 2024-05-21T22:08:29.460490+0000) 2024-05-21T22:08:50.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:49 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:49.858+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-21T22:08:10.561367+0000 front 2024-05-21T22:08:10.561307+0000 (oldest deadline 2024-05-21T22:08:35.861406+0000) 2024-05-21T22:08:50.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:49 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:49.858+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.151:6822 osd.6 since back 2024-05-21T22:08:20.463172+0000 front 2024-05-21T22:08:20.463034+0000 (oldest deadline 2024-05-21T22:08:46.362860+0000) 2024-05-21T22:08:50.286 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:50 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:50.015+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:30.508363+0000 front 2024-05-21T22:07:30.508149+0000 (oldest deadline 2024-05-21T22:07:52.808006+0000) 2024-05-21T22:08:50.286 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:50 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:50.015+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:41.009631+0000 front 2024-05-21T22:07:41.009565+0000 (oldest deadline 2024-05-21T22:08:05.709714+0000) 2024-05-21T22:08:50.286 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:50 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:50.015+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:49.011674+0000 front 2024-05-21T22:07:49.011868+0000 (oldest deadline 2024-05-21T22:08:13.711629+0000) 2024-05-21T22:08:50.286 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:50 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:50.015+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:53.712167+0000 front 2024-05-21T22:07:53.712374+0000 (oldest deadline 2024-05-21T22:08:18.412083+0000) 2024-05-21T22:08:50.286 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:50 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:50.015+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-05-21T22:08:03.113068+0000 front 2024-05-21T22:08:03.113103+0000 (oldest deadline 2024-05-21T22:08:29.513902+0000) 2024-05-21T22:08:50.286 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:50 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:50.015+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-21T22:08:13.615230+0000 front 2024-05-21T22:08:13.615079+0000 (oldest deadline 2024-05-21T22:08:37.714944+0000) 2024-05-21T22:08:50.286 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:50 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:50.015+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6822 osd.6 since back 2024-05-21T22:08:20.016394+0000 front 2024-05-21T22:08:20.016252+0000 (oldest deadline 2024-05-21T22:08:45.915946+0000) 2024-05-21T22:08:50.551 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:50 smithi151 podman[122026]: 2024-05-21 22:08:50.207041794 +0000 UTC m=+0.861262177 container died 99b061789bd4d997602d98898ccc8e3b41cc99ded94b5e076c2645bc9498760f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514) 2024-05-21T22:08:51.067 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@osd.9.service' 2024-05-21T22:08:51.128 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:51 smithi151 podman[122051]: 2024-05-21 22:08:51.012843546 +0000 UTC m=+0.787807195 container remove 99b061789bd4d997602d98898ccc8e3b41cc99ded94b5e076c2645bc9498760f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-9-deactivate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-21T22:08:51.129 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:51 smithi151 systemd[1]: ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@osd.9.service: Deactivated successfully. 2024-05-21T22:08:51.129 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:51 smithi151 systemd[1]: Stopped Ceph osd.9 for c8c7a2e6-17ba-11ef-bc9a-c7b262605968. 2024-05-21T22:08:51.129 INFO:journalctl@ceph.osd.9.smithi151.stdout:May 21 22:08:51 smithi151 systemd[1]: ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@osd.9.service: Consumed 27.052s CPU time. 2024-05-21T22:08:51.161 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T22:08:51.161 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-05-21T22:08:51.161 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-05-21T22:08:51.161 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@osd.10 2024-05-21T22:08:51.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:50 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:50.873+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:28.956267+0000 front 2024-05-21T22:07:28.956225+0000 (oldest deadline 2024-05-21T22:07:55.356623+0000) 2024-05-21T22:08:51.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:50 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:50.873+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:38.857667+0000 front 2024-05-21T22:07:38.857566+0000 (oldest deadline 2024-05-21T22:08:04.757680+0000) 2024-05-21T22:08:51.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:50 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:50.873+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:44.758039+0000 front 2024-05-21T22:07:44.758082+0000 (oldest deadline 2024-05-21T22:08:10.658241+0000) 2024-05-21T22:08:51.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:50 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:50.873+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-05-21T22:08:04.160535+0000 front 2024-05-21T22:08:04.160468+0000 (oldest deadline 2024-05-21T22:08:29.460490+0000) 2024-05-21T22:08:51.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:50 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:50.873+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-21T22:08:10.561367+0000 front 2024-05-21T22:08:10.561307+0000 (oldest deadline 2024-05-21T22:08:35.861406+0000) 2024-05-21T22:08:51.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:50 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:50.873+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.151:6822 osd.6 since back 2024-05-21T22:08:20.463172+0000 front 2024-05-21T22:08:20.463034+0000 (oldest deadline 2024-05-21T22:08:46.362860+0000) 2024-05-21T22:08:51.286 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:51 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:50.999+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:30.508363+0000 front 2024-05-21T22:07:30.508149+0000 (oldest deadline 2024-05-21T22:07:52.808006+0000) 2024-05-21T22:08:51.287 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:51 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:50.999+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:41.009631+0000 front 2024-05-21T22:07:41.009565+0000 (oldest deadline 2024-05-21T22:08:05.709714+0000) 2024-05-21T22:08:51.287 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:51 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:50.999+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:49.011674+0000 front 2024-05-21T22:07:49.011868+0000 (oldest deadline 2024-05-21T22:08:13.711629+0000) 2024-05-21T22:08:51.287 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:51 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:50.999+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:53.712167+0000 front 2024-05-21T22:07:53.712374+0000 (oldest deadline 2024-05-21T22:08:18.412083+0000) 2024-05-21T22:08:51.287 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:51 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:50.999+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-05-21T22:08:03.113068+0000 front 2024-05-21T22:08:03.113103+0000 (oldest deadline 2024-05-21T22:08:29.513902+0000) 2024-05-21T22:08:51.287 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:51 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:50.999+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-21T22:08:13.615230+0000 front 2024-05-21T22:08:13.615079+0000 (oldest deadline 2024-05-21T22:08:37.714944+0000) 2024-05-21T22:08:51.287 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:51 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:50.999+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6822 osd.6 since back 2024-05-21T22:08:20.016394+0000 front 2024-05-21T22:08:20.016252+0000 (oldest deadline 2024-05-21T22:08:45.915946+0000) 2024-05-21T22:08:51.287 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:51 smithi164 systemd[1]: Stopping Ceph osd.10 for c8c7a2e6-17ba-11ef-bc9a-c7b262605968... 2024-05-21T22:08:51.786 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:51 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:51.468+0000 7f0b444ff700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T22:08:51.786 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:51 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:51.468+0000 7f0b444ff700 -1 osd.10 94 *** Got signal Terminated *** 2024-05-21T22:08:51.786 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:51 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:51.468+0000 7f0b444ff700 -1 osd.10 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-21T22:08:52.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:51 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:51.911+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:28.956267+0000 front 2024-05-21T22:07:28.956225+0000 (oldest deadline 2024-05-21T22:07:55.356623+0000) 2024-05-21T22:08:52.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:51 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:51.911+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:38.857667+0000 front 2024-05-21T22:07:38.857566+0000 (oldest deadline 2024-05-21T22:08:04.757680+0000) 2024-05-21T22:08:52.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:51 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:51.911+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:44.758039+0000 front 2024-05-21T22:07:44.758082+0000 (oldest deadline 2024-05-21T22:08:10.658241+0000) 2024-05-21T22:08:52.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:51 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:51.911+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-05-21T22:08:04.160535+0000 front 2024-05-21T22:08:04.160468+0000 (oldest deadline 2024-05-21T22:08:29.460490+0000) 2024-05-21T22:08:52.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:51 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:51.911+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-21T22:08:10.561367+0000 front 2024-05-21T22:08:10.561307+0000 (oldest deadline 2024-05-21T22:08:35.861406+0000) 2024-05-21T22:08:52.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:51 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:51.911+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.151:6822 osd.6 since back 2024-05-21T22:08:20.463172+0000 front 2024-05-21T22:08:20.463034+0000 (oldest deadline 2024-05-21T22:08:46.362860+0000) 2024-05-21T22:08:52.286 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:51 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:51.988+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:30.508363+0000 front 2024-05-21T22:07:30.508149+0000 (oldest deadline 2024-05-21T22:07:52.808006+0000) 2024-05-21T22:08:52.286 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:51 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:51.988+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:41.009631+0000 front 2024-05-21T22:07:41.009565+0000 (oldest deadline 2024-05-21T22:08:05.709714+0000) 2024-05-21T22:08:52.286 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:51 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:51.988+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:49.011674+0000 front 2024-05-21T22:07:49.011868+0000 (oldest deadline 2024-05-21T22:08:13.711629+0000) 2024-05-21T22:08:52.286 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:51 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:51.988+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:53.712167+0000 front 2024-05-21T22:07:53.712374+0000 (oldest deadline 2024-05-21T22:08:18.412083+0000) 2024-05-21T22:08:52.286 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:51 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:51.988+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-05-21T22:08:03.113068+0000 front 2024-05-21T22:08:03.113103+0000 (oldest deadline 2024-05-21T22:08:29.513902+0000) 2024-05-21T22:08:52.286 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:51 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:51.988+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-21T22:08:13.615230+0000 front 2024-05-21T22:08:13.615079+0000 (oldest deadline 2024-05-21T22:08:37.714944+0000) 2024-05-21T22:08:52.287 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:51 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:51.988+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6822 osd.6 since back 2024-05-21T22:08:20.016394+0000 front 2024-05-21T22:08:20.016252+0000 (oldest deadline 2024-05-21T22:08:45.915946+0000) 2024-05-21T22:08:53.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:52 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:52.873+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:28.956267+0000 front 2024-05-21T22:07:28.956225+0000 (oldest deadline 2024-05-21T22:07:55.356623+0000) 2024-05-21T22:08:53.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:52 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:52.873+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:38.857667+0000 front 2024-05-21T22:07:38.857566+0000 (oldest deadline 2024-05-21T22:08:04.757680+0000) 2024-05-21T22:08:53.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:52 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:52.873+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:44.758039+0000 front 2024-05-21T22:07:44.758082+0000 (oldest deadline 2024-05-21T22:08:10.658241+0000) 2024-05-21T22:08:53.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:52 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:52.873+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-05-21T22:08:04.160535+0000 front 2024-05-21T22:08:04.160468+0000 (oldest deadline 2024-05-21T22:08:29.460490+0000) 2024-05-21T22:08:53.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:52 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:52.873+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-21T22:08:10.561367+0000 front 2024-05-21T22:08:10.561307+0000 (oldest deadline 2024-05-21T22:08:35.861406+0000) 2024-05-21T22:08:53.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:52 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:52.873+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.151:6822 osd.6 since back 2024-05-21T22:08:20.463172+0000 front 2024-05-21T22:08:20.463034+0000 (oldest deadline 2024-05-21T22:08:46.362860+0000) 2024-05-21T22:08:53.286 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:52 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:52.945+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:30.508363+0000 front 2024-05-21T22:07:30.508149+0000 (oldest deadline 2024-05-21T22:07:52.808006+0000) 2024-05-21T22:08:53.286 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:52 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:52.945+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:41.009631+0000 front 2024-05-21T22:07:41.009565+0000 (oldest deadline 2024-05-21T22:08:05.709714+0000) 2024-05-21T22:08:53.286 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:52 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:52.945+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:49.011674+0000 front 2024-05-21T22:07:49.011868+0000 (oldest deadline 2024-05-21T22:08:13.711629+0000) 2024-05-21T22:08:53.286 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:52 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:52.945+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:53.712167+0000 front 2024-05-21T22:07:53.712374+0000 (oldest deadline 2024-05-21T22:08:18.412083+0000) 2024-05-21T22:08:53.286 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:52 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:52.945+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-05-21T22:08:03.113068+0000 front 2024-05-21T22:08:03.113103+0000 (oldest deadline 2024-05-21T22:08:29.513902+0000) 2024-05-21T22:08:53.287 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:52 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:52.945+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-21T22:08:13.615230+0000 front 2024-05-21T22:08:13.615079+0000 (oldest deadline 2024-05-21T22:08:37.714944+0000) 2024-05-21T22:08:53.287 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:52 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:52.945+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6822 osd.6 since back 2024-05-21T22:08:20.016394+0000 front 2024-05-21T22:08:20.016252+0000 (oldest deadline 2024-05-21T22:08:45.915946+0000) 2024-05-21T22:08:54.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:53 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:53.902+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:28.956267+0000 front 2024-05-21T22:07:28.956225+0000 (oldest deadline 2024-05-21T22:07:55.356623+0000) 2024-05-21T22:08:54.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:53 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:53.902+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:38.857667+0000 front 2024-05-21T22:07:38.857566+0000 (oldest deadline 2024-05-21T22:08:04.757680+0000) 2024-05-21T22:08:54.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:53 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:53.902+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:44.758039+0000 front 2024-05-21T22:07:44.758082+0000 (oldest deadline 2024-05-21T22:08:10.658241+0000) 2024-05-21T22:08:54.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:53 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:53.902+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-05-21T22:08:04.160535+0000 front 2024-05-21T22:08:04.160468+0000 (oldest deadline 2024-05-21T22:08:29.460490+0000) 2024-05-21T22:08:54.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:53 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:53.902+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-21T22:08:10.561367+0000 front 2024-05-21T22:08:10.561307+0000 (oldest deadline 2024-05-21T22:08:35.861406+0000) 2024-05-21T22:08:54.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:53 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:53.902+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.151:6822 osd.6 since back 2024-05-21T22:08:20.463172+0000 front 2024-05-21T22:08:20.463034+0000 (oldest deadline 2024-05-21T22:08:46.362860+0000) 2024-05-21T22:08:54.286 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:53 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:53.985+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:30.508363+0000 front 2024-05-21T22:07:30.508149+0000 (oldest deadline 2024-05-21T22:07:52.808006+0000) 2024-05-21T22:08:54.286 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:53 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:53.985+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:41.009631+0000 front 2024-05-21T22:07:41.009565+0000 (oldest deadline 2024-05-21T22:08:05.709714+0000) 2024-05-21T22:08:54.286 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:53 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:53.985+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:49.011674+0000 front 2024-05-21T22:07:49.011868+0000 (oldest deadline 2024-05-21T22:08:13.711629+0000) 2024-05-21T22:08:54.286 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:53 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:53.985+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:53.712167+0000 front 2024-05-21T22:07:53.712374+0000 (oldest deadline 2024-05-21T22:08:18.412083+0000) 2024-05-21T22:08:54.286 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:53 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:53.985+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-05-21T22:08:03.113068+0000 front 2024-05-21T22:08:03.113103+0000 (oldest deadline 2024-05-21T22:08:29.513902+0000) 2024-05-21T22:08:54.286 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:53 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:53.985+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-21T22:08:13.615230+0000 front 2024-05-21T22:08:13.615079+0000 (oldest deadline 2024-05-21T22:08:37.714944+0000) 2024-05-21T22:08:54.287 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:53 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:53.985+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6822 osd.6 since back 2024-05-21T22:08:20.016394+0000 front 2024-05-21T22:08:20.016252+0000 (oldest deadline 2024-05-21T22:08:45.915946+0000) 2024-05-21T22:08:55.138 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:54 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:54.860+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:28.956267+0000 front 2024-05-21T22:07:28.956225+0000 (oldest deadline 2024-05-21T22:07:55.356623+0000) 2024-05-21T22:08:55.138 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:54 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:54.860+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:38.857667+0000 front 2024-05-21T22:07:38.857566+0000 (oldest deadline 2024-05-21T22:08:04.757680+0000) 2024-05-21T22:08:55.138 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:54 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:54.860+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:44.758039+0000 front 2024-05-21T22:07:44.758082+0000 (oldest deadline 2024-05-21T22:08:10.658241+0000) 2024-05-21T22:08:55.138 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:54 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:54.860+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-05-21T22:08:04.160535+0000 front 2024-05-21T22:08:04.160468+0000 (oldest deadline 2024-05-21T22:08:29.460490+0000) 2024-05-21T22:08:55.138 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:54 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:54.860+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-21T22:08:10.561367+0000 front 2024-05-21T22:08:10.561307+0000 (oldest deadline 2024-05-21T22:08:35.861406+0000) 2024-05-21T22:08:55.138 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:54 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:54.860+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.151:6822 osd.6 since back 2024-05-21T22:08:20.463172+0000 front 2024-05-21T22:08:20.463034+0000 (oldest deadline 2024-05-21T22:08:46.362860+0000) 2024-05-21T22:08:55.138 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:54 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:54.860+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6820 osd.7 since back 2024-05-21T22:08:31.063841+0000 front 2024-05-21T22:08:31.063787+0000 (oldest deadline 2024-05-21T22:08:54.464219+0000) 2024-05-21T22:08:55.286 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:55 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:55.019+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:30.508363+0000 front 2024-05-21T22:07:30.508149+0000 (oldest deadline 2024-05-21T22:07:52.808006+0000) 2024-05-21T22:08:55.286 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:55 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:55.019+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:41.009631+0000 front 2024-05-21T22:07:41.009565+0000 (oldest deadline 2024-05-21T22:08:05.709714+0000) 2024-05-21T22:08:55.286 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:55 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:55.019+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:49.011674+0000 front 2024-05-21T22:07:49.011868+0000 (oldest deadline 2024-05-21T22:08:13.711629+0000) 2024-05-21T22:08:55.286 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:55 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:55.019+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:53.712167+0000 front 2024-05-21T22:07:53.712374+0000 (oldest deadline 2024-05-21T22:08:18.412083+0000) 2024-05-21T22:08:55.286 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:55 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:55.019+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-05-21T22:08:03.113068+0000 front 2024-05-21T22:08:03.113103+0000 (oldest deadline 2024-05-21T22:08:29.513902+0000) 2024-05-21T22:08:55.286 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:55 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:55.019+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-21T22:08:13.615230+0000 front 2024-05-21T22:08:13.615079+0000 (oldest deadline 2024-05-21T22:08:37.714944+0000) 2024-05-21T22:08:55.286 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:55 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:55.019+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6822 osd.6 since back 2024-05-21T22:08:20.016394+0000 front 2024-05-21T22:08:20.016252+0000 (oldest deadline 2024-05-21T22:08:45.915946+0000) 2024-05-21T22:08:56.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:55 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:55.878+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:28.956267+0000 front 2024-05-21T22:07:28.956225+0000 (oldest deadline 2024-05-21T22:07:55.356623+0000) 2024-05-21T22:08:56.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:55 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:55.878+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:38.857667+0000 front 2024-05-21T22:07:38.857566+0000 (oldest deadline 2024-05-21T22:08:04.757680+0000) 2024-05-21T22:08:56.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:55 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:55.878+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:44.758039+0000 front 2024-05-21T22:07:44.758082+0000 (oldest deadline 2024-05-21T22:08:10.658241+0000) 2024-05-21T22:08:56.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:55 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:55.878+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-05-21T22:08:04.160535+0000 front 2024-05-21T22:08:04.160468+0000 (oldest deadline 2024-05-21T22:08:29.460490+0000) 2024-05-21T22:08:56.237 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:55 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:55.878+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-21T22:08:10.561367+0000 front 2024-05-21T22:08:10.561307+0000 (oldest deadline 2024-05-21T22:08:35.861406+0000) 2024-05-21T22:08:56.237 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:55 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:55.878+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.151:6822 osd.6 since back 2024-05-21T22:08:20.463172+0000 front 2024-05-21T22:08:20.463034+0000 (oldest deadline 2024-05-21T22:08:46.362860+0000) 2024-05-21T22:08:56.237 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:55 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:55.878+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6820 osd.7 since back 2024-05-21T22:08:31.063841+0000 front 2024-05-21T22:08:31.063787+0000 (oldest deadline 2024-05-21T22:08:54.464219+0000) 2024-05-21T22:08:56.536 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:56 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:56.052+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:30.508363+0000 front 2024-05-21T22:07:30.508149+0000 (oldest deadline 2024-05-21T22:07:52.808006+0000) 2024-05-21T22:08:56.536 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:56 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:56.052+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:41.009631+0000 front 2024-05-21T22:07:41.009565+0000 (oldest deadline 2024-05-21T22:08:05.709714+0000) 2024-05-21T22:08:56.536 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:56 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:56.052+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:49.011674+0000 front 2024-05-21T22:07:49.011868+0000 (oldest deadline 2024-05-21T22:08:13.711629+0000) 2024-05-21T22:08:56.536 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:56 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:56.052+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6814 osd.3 since back 2024-05-21T22:07:53.712167+0000 front 2024-05-21T22:07:53.712374+0000 (oldest deadline 2024-05-21T22:08:18.412083+0000) 2024-05-21T22:08:56.537 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:56 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:56.052+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-05-21T22:08:03.113068+0000 front 2024-05-21T22:08:03.113103+0000 (oldest deadline 2024-05-21T22:08:29.513902+0000) 2024-05-21T22:08:56.537 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:56 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:56.052+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-21T22:08:13.615230+0000 front 2024-05-21T22:08:13.615079+0000 (oldest deadline 2024-05-21T22:08:37.714944+0000) 2024-05-21T22:08:56.537 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:56 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10[60754]: 2024-05-21T22:08:56.052+0000 7f0b40af3700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.151:6822 osd.6 since back 2024-05-21T22:08:20.016394+0000 front 2024-05-21T22:08:20.016252+0000 (oldest deadline 2024-05-21T22:08:45.915946+0000) 2024-05-21T22:08:57.086 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:56 smithi164 podman[73410]: 2024-05-21 22:08:56.806788437 +0000 UTC m=+5.446856165 container died 05e2afdedadce69e278866458870c12c16b882e81a45213fd7dee866cd8a765b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-05-21T22:08:57.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:56 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:56.878+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:28.956267+0000 front 2024-05-21T22:07:28.956225+0000 (oldest deadline 2024-05-21T22:07:55.356623+0000) 2024-05-21T22:08:57.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:56 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:56.878+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:38.857667+0000 front 2024-05-21T22:07:38.857566+0000 (oldest deadline 2024-05-21T22:08:04.757680+0000) 2024-05-21T22:08:57.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:56 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:56.878+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:44.758039+0000 front 2024-05-21T22:07:44.758082+0000 (oldest deadline 2024-05-21T22:08:10.658241+0000) 2024-05-21T22:08:57.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:56 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:56.878+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-05-21T22:08:04.160535+0000 front 2024-05-21T22:08:04.160468+0000 (oldest deadline 2024-05-21T22:08:29.460490+0000) 2024-05-21T22:08:57.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:56 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:56.878+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-21T22:08:10.561367+0000 front 2024-05-21T22:08:10.561307+0000 (oldest deadline 2024-05-21T22:08:35.861406+0000) 2024-05-21T22:08:57.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:56 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:56.878+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.151:6822 osd.6 since back 2024-05-21T22:08:20.463172+0000 front 2024-05-21T22:08:20.463034+0000 (oldest deadline 2024-05-21T22:08:46.362860+0000) 2024-05-21T22:08:57.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:56 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:56.878+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6820 osd.7 since back 2024-05-21T22:08:31.063841+0000 front 2024-05-21T22:08:31.063787+0000 (oldest deadline 2024-05-21T22:08:54.464219+0000) 2024-05-21T22:08:57.461 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:57 smithi164 podman[73410]: 2024-05-21 22:08:57.193855818 +0000 UTC m=+5.833923550 container cleanup 05e2afdedadce69e278866458870c12c16b882e81a45213fd7dee866cd8a765b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True) 2024-05-21T22:08:57.461 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:57 smithi164 bash[73410]: ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10 2024-05-21T22:08:57.753 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:57 smithi164 podman[73429]: 2024-05-21 22:08:57.461685052 +0000 UTC m=+0.652633407 container remove 05e2afdedadce69e278866458870c12c16b882e81a45213fd7dee866cd8a765b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-05-21T22:08:58.022 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:57 smithi164 podman[73522]: 2024-05-21 22:08:57.754121956 +0000 UTC m=+0.093611412 container create a40969555938b957367256b2c6de6f93fd4324cadc8a0bf1148c449c9427ef07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-21T22:08:58.023 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:57 smithi164 podman[73522]: 2024-05-21 22:08:57.677848017 +0000 UTC m=+0.017337476 image pull d6bc8f31cbc1a7165ca248fe9ba0b06ea14b30f46e1ea33ced1a5238e247af1f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28 2024-05-21T22:08:58.023 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:57 smithi164 podman[73522]: 2024-05-21 22:08:57.948642544 +0000 UTC m=+0.288132007 container init a40969555938b957367256b2c6de6f93fd4324cadc8a0bf1148c449c9427ef07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-21T22:08:58.023 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:57 smithi164 podman[73522]: 2024-05-21 22:08:57.952077494 +0000 UTC m=+0.291566951 container start a40969555938b957367256b2c6de6f93fd4324cadc8a0bf1148c449c9427ef07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10-deactivate, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.license=GPLv2) 2024-05-21T22:08:58.023 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:58 smithi164 podman[73522]: 2024-05-21 22:08:58.023083628 +0000 UTC m=+0.362573096 container attach a40969555938b957367256b2c6de6f93fd4324cadc8a0bf1148c449c9427ef07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-21T22:08:58.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:57 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:57.841+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:28.956267+0000 front 2024-05-21T22:07:28.956225+0000 (oldest deadline 2024-05-21T22:07:55.356623+0000) 2024-05-21T22:08:58.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:57 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:57.841+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:38.857667+0000 front 2024-05-21T22:07:38.857566+0000 (oldest deadline 2024-05-21T22:08:04.757680+0000) 2024-05-21T22:08:58.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:57 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:57.841+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:44.758039+0000 front 2024-05-21T22:07:44.758082+0000 (oldest deadline 2024-05-21T22:08:10.658241+0000) 2024-05-21T22:08:58.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:57 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:57.841+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-05-21T22:08:04.160535+0000 front 2024-05-21T22:08:04.160468+0000 (oldest deadline 2024-05-21T22:08:29.460490+0000) 2024-05-21T22:08:58.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:57 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:57.841+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-21T22:08:10.561367+0000 front 2024-05-21T22:08:10.561307+0000 (oldest deadline 2024-05-21T22:08:35.861406+0000) 2024-05-21T22:08:58.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:57 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:57.841+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.151:6822 osd.6 since back 2024-05-21T22:08:20.463172+0000 front 2024-05-21T22:08:20.463034+0000 (oldest deadline 2024-05-21T22:08:46.362860+0000) 2024-05-21T22:08:58.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:57 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:57.841+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6820 osd.7 since back 2024-05-21T22:08:31.063841+0000 front 2024-05-21T22:08:31.063787+0000 (oldest deadline 2024-05-21T22:08:54.464219+0000) 2024-05-21T22:08:58.286 INFO:journalctl@ceph.osd.10.smithi164.stdout:May 21 22:08:58 smithi164 podman[73522]: 2024-05-21 22:08:58.250876255 +0000 UTC m=+0.590365712 container died a40969555938b957367256b2c6de6f93fd4324cadc8a0bf1148c449c9427ef07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-10-deactivate, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True) 2024-05-21T22:08:58.835 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@osd.10.service' 2024-05-21T22:08:58.871 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T22:08:58.872 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-05-21T22:08:58.872 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-05-21T22:08:58.872 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@osd.11 2024-05-21T22:08:59.175 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:58 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:58.836+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:28.956267+0000 front 2024-05-21T22:07:28.956225+0000 (oldest deadline 2024-05-21T22:07:55.356623+0000) 2024-05-21T22:08:59.175 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:58 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:58.836+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:38.857667+0000 front 2024-05-21T22:07:38.857566+0000 (oldest deadline 2024-05-21T22:08:04.757680+0000) 2024-05-21T22:08:59.175 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:58 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:58.836+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:44.758039+0000 front 2024-05-21T22:07:44.758082+0000 (oldest deadline 2024-05-21T22:08:10.658241+0000) 2024-05-21T22:08:59.175 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:58 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:58.836+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-05-21T22:08:04.160535+0000 front 2024-05-21T22:08:04.160468+0000 (oldest deadline 2024-05-21T22:08:29.460490+0000) 2024-05-21T22:08:59.175 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:58 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:58.836+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-21T22:08:10.561367+0000 front 2024-05-21T22:08:10.561307+0000 (oldest deadline 2024-05-21T22:08:35.861406+0000) 2024-05-21T22:08:59.175 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:58 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:58.836+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.151:6822 osd.6 since back 2024-05-21T22:08:20.463172+0000 front 2024-05-21T22:08:20.463034+0000 (oldest deadline 2024-05-21T22:08:46.362860+0000) 2024-05-21T22:08:59.175 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:58 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:58.836+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6820 osd.7 since back 2024-05-21T22:08:31.063841+0000 front 2024-05-21T22:08:31.063787+0000 (oldest deadline 2024-05-21T22:08:54.464219+0000) 2024-05-21T22:08:59.175 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:58 smithi184 systemd[1]: Stopping Ceph osd.11 for c8c7a2e6-17ba-11ef-bc9a-c7b262605968... 2024-05-21T22:08:59.485 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:59 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:59.174+0000 7f61eb0ea700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T22:08:59.486 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:59 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:59.174+0000 7f61eb0ea700 -1 osd.11 94 *** Got signal Terminated *** 2024-05-21T22:08:59.486 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:59 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:59.174+0000 7f61eb0ea700 -1 osd.11 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-21T22:09:00.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:59 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:59.803+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:28.956267+0000 front 2024-05-21T22:07:28.956225+0000 (oldest deadline 2024-05-21T22:07:55.356623+0000) 2024-05-21T22:09:00.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:59 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:59.803+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:38.857667+0000 front 2024-05-21T22:07:38.857566+0000 (oldest deadline 2024-05-21T22:08:04.757680+0000) 2024-05-21T22:09:00.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:59 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:59.803+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:44.758039+0000 front 2024-05-21T22:07:44.758082+0000 (oldest deadline 2024-05-21T22:08:10.658241+0000) 2024-05-21T22:09:00.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:59 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:59.803+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-05-21T22:08:04.160535+0000 front 2024-05-21T22:08:04.160468+0000 (oldest deadline 2024-05-21T22:08:29.460490+0000) 2024-05-21T22:09:00.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:59 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:59.803+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-21T22:08:10.561367+0000 front 2024-05-21T22:08:10.561307+0000 (oldest deadline 2024-05-21T22:08:35.861406+0000) 2024-05-21T22:09:00.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:59 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:59.803+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.151:6822 osd.6 since back 2024-05-21T22:08:20.463172+0000 front 2024-05-21T22:08:20.463034+0000 (oldest deadline 2024-05-21T22:08:46.362860+0000) 2024-05-21T22:09:00.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:08:59 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:08:59.803+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6820 osd.7 since back 2024-05-21T22:08:31.063841+0000 front 2024-05-21T22:08:31.063787+0000 (oldest deadline 2024-05-21T22:08:54.464219+0000) 2024-05-21T22:09:01.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:09:00 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:09:00.834+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:28.956267+0000 front 2024-05-21T22:07:28.956225+0000 (oldest deadline 2024-05-21T22:07:55.356623+0000) 2024-05-21T22:09:01.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:09:00 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:09:00.834+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:38.857667+0000 front 2024-05-21T22:07:38.857566+0000 (oldest deadline 2024-05-21T22:08:04.757680+0000) 2024-05-21T22:09:01.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:09:00 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:09:00.834+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:44.758039+0000 front 2024-05-21T22:07:44.758082+0000 (oldest deadline 2024-05-21T22:08:10.658241+0000) 2024-05-21T22:09:01.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:09:00 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:09:00.834+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-05-21T22:08:04.160535+0000 front 2024-05-21T22:08:04.160468+0000 (oldest deadline 2024-05-21T22:08:29.460490+0000) 2024-05-21T22:09:01.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:09:00 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:09:00.834+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-21T22:08:10.561367+0000 front 2024-05-21T22:08:10.561307+0000 (oldest deadline 2024-05-21T22:08:35.861406+0000) 2024-05-21T22:09:01.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:09:00 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:09:00.834+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.151:6822 osd.6 since back 2024-05-21T22:08:20.463172+0000 front 2024-05-21T22:08:20.463034+0000 (oldest deadline 2024-05-21T22:08:46.362860+0000) 2024-05-21T22:09:01.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:09:00 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:09:00.834+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6820 osd.7 since back 2024-05-21T22:08:31.063841+0000 front 2024-05-21T22:08:31.063787+0000 (oldest deadline 2024-05-21T22:08:54.464219+0000) 2024-05-21T22:09:01.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:09:00 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:09:00.834+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6820 osd.8 since back 2024-05-21T22:08:34.464661+0000 front 2024-05-21T22:08:34.464640+0000 (oldest deadline 2024-05-21T22:09:00.364615+0000) 2024-05-21T22:09:02.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:09:01 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:09:01.881+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:28.956267+0000 front 2024-05-21T22:07:28.956225+0000 (oldest deadline 2024-05-21T22:07:55.356623+0000) 2024-05-21T22:09:02.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:09:01 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:09:01.881+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:38.857667+0000 front 2024-05-21T22:07:38.857566+0000 (oldest deadline 2024-05-21T22:08:04.757680+0000) 2024-05-21T22:09:02.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:09:01 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:09:01.881+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:44.758039+0000 front 2024-05-21T22:07:44.758082+0000 (oldest deadline 2024-05-21T22:08:10.658241+0000) 2024-05-21T22:09:02.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:09:01 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:09:01.881+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-05-21T22:08:04.160535+0000 front 2024-05-21T22:08:04.160468+0000 (oldest deadline 2024-05-21T22:08:29.460490+0000) 2024-05-21T22:09:02.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:09:01 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:09:01.881+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-21T22:08:10.561367+0000 front 2024-05-21T22:08:10.561307+0000 (oldest deadline 2024-05-21T22:08:35.861406+0000) 2024-05-21T22:09:02.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:09:01 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:09:01.881+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.151:6822 osd.6 since back 2024-05-21T22:08:20.463172+0000 front 2024-05-21T22:08:20.463034+0000 (oldest deadline 2024-05-21T22:08:46.362860+0000) 2024-05-21T22:09:02.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:09:01 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:09:01.881+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6820 osd.7 since back 2024-05-21T22:08:31.063841+0000 front 2024-05-21T22:08:31.063787+0000 (oldest deadline 2024-05-21T22:08:54.464219+0000) 2024-05-21T22:09:02.237 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:09:01 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:09:01.881+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6820 osd.8 since back 2024-05-21T22:08:34.464661+0000 front 2024-05-21T22:08:34.464640+0000 (oldest deadline 2024-05-21T22:09:00.364615+0000) 2024-05-21T22:09:03.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:09:02 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:09:02.896+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:28.956267+0000 front 2024-05-21T22:07:28.956225+0000 (oldest deadline 2024-05-21T22:07:55.356623+0000) 2024-05-21T22:09:03.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:09:02 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:09:02.896+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:38.857667+0000 front 2024-05-21T22:07:38.857566+0000 (oldest deadline 2024-05-21T22:08:04.757680+0000) 2024-05-21T22:09:03.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:09:02 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:09:02.896+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:44.758039+0000 front 2024-05-21T22:07:44.758082+0000 (oldest deadline 2024-05-21T22:08:10.658241+0000) 2024-05-21T22:09:03.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:09:02 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:09:02.896+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-05-21T22:08:04.160535+0000 front 2024-05-21T22:08:04.160468+0000 (oldest deadline 2024-05-21T22:08:29.460490+0000) 2024-05-21T22:09:03.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:09:02 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:09:02.896+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-21T22:08:10.561367+0000 front 2024-05-21T22:08:10.561307+0000 (oldest deadline 2024-05-21T22:08:35.861406+0000) 2024-05-21T22:09:03.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:09:02 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:09:02.896+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.151:6822 osd.6 since back 2024-05-21T22:08:20.463172+0000 front 2024-05-21T22:08:20.463034+0000 (oldest deadline 2024-05-21T22:08:46.362860+0000) 2024-05-21T22:09:03.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:09:02 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:09:02.896+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6820 osd.7 since back 2024-05-21T22:08:31.063841+0000 front 2024-05-21T22:08:31.063787+0000 (oldest deadline 2024-05-21T22:08:54.464219+0000) 2024-05-21T22:09:03.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:09:02 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:09:02.896+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6820 osd.8 since back 2024-05-21T22:08:34.464661+0000 front 2024-05-21T22:08:34.464640+0000 (oldest deadline 2024-05-21T22:09:00.364615+0000) 2024-05-21T22:09:04.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:09:03 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:09:03.940+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.151:6806 osd.0 since back 2024-05-21T22:07:28.956267+0000 front 2024-05-21T22:07:28.956225+0000 (oldest deadline 2024-05-21T22:07:55.356623+0000) 2024-05-21T22:09:04.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:09:03 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:09:03.940+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6804 osd.1 since back 2024-05-21T22:07:38.857667+0000 front 2024-05-21T22:07:38.857566+0000 (oldest deadline 2024-05-21T22:08:04.757680+0000) 2024-05-21T22:09:04.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:09:03 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:09:03.940+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2024-05-21T22:07:44.758039+0000 front 2024-05-21T22:07:44.758082+0000 (oldest deadline 2024-05-21T22:08:10.658241+0000) 2024-05-21T22:09:04.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:09:03 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:09:03.940+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6812 osd.4 since back 2024-05-21T22:08:04.160535+0000 front 2024-05-21T22:08:04.160468+0000 (oldest deadline 2024-05-21T22:08:29.460490+0000) 2024-05-21T22:09:04.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:09:03 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:09:03.940+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2024-05-21T22:08:10.561367+0000 front 2024-05-21T22:08:10.561307+0000 (oldest deadline 2024-05-21T22:08:35.861406+0000) 2024-05-21T22:09:04.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:09:03 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:09:03.940+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.151:6822 osd.6 since back 2024-05-21T22:08:20.463172+0000 front 2024-05-21T22:08:20.463034+0000 (oldest deadline 2024-05-21T22:08:46.362860+0000) 2024-05-21T22:09:04.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:09:03 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:09:03.940+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.164:6820 osd.7 since back 2024-05-21T22:08:31.063841+0000 front 2024-05-21T22:08:31.063787+0000 (oldest deadline 2024-05-21T22:08:54.464219+0000) 2024-05-21T22:09:04.237 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:09:03 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11[62548]: 2024-05-21T22:09:03.940+0000 7f61e76de700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6820 osd.8 since back 2024-05-21T22:08:34.464661+0000 front 2024-05-21T22:08:34.464640+0000 (oldest deadline 2024-05-21T22:09:00.364615+0000) 2024-05-21T22:09:04.941 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:09:04 smithi184 podman[75144]: 2024-05-21 22:09:04.687289327 +0000 UTC m=+5.621393334 container died 72df127bc3cf372688a3243979de49cc845928a001378a706aa5a6c2f63b28e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-05-21T22:09:05.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:09:05 smithi184 podman[75144]: 2024-05-21 22:09:05.032358666 +0000 UTC m=+5.966462665 container cleanup 72df127bc3cf372688a3243979de49cc845928a001378a706aa5a6c2f63b28e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-05-21T22:09:05.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:09:05 smithi184 bash[75144]: ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11 2024-05-21T22:09:05.651 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:09:05 smithi184 podman[75160]: 2024-05-21 22:09:05.333628908 +0000 UTC m=+0.646620629 container remove 72df127bc3cf372688a3243979de49cc845928a001378a706aa5a6c2f63b28e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5) 2024-05-21T22:09:05.919 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:09:05 smithi184 podman[75250]: 2024-05-21 22:09:05.554722255 +0000 UTC m=+0.017608950 image pull d6bc8f31cbc1a7165ca248fe9ba0b06ea14b30f46e1ea33ced1a5238e247af1f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28 2024-05-21T22:09:05.920 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:09:05 smithi184 podman[75250]: 2024-05-21 22:09:05.65944203 +0000 UTC m=+0.122328717 container create c8d17e6899d6095085171ac2e71bce54fa236e52db9606c98b46f39243a87bfc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea) 2024-05-21T22:09:05.920 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:09:05 smithi184 podman[75250]: 2024-05-21 22:09:05.861695476 +0000 UTC m=+0.324582165 container init c8d17e6899d6095085171ac2e71bce54fa236e52db9606c98b46f39243a87bfc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea) 2024-05-21T22:09:05.920 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:09:05 smithi184 podman[75250]: 2024-05-21 22:09:05.865576557 +0000 UTC m=+0.328463257 container start c8d17e6899d6095085171ac2e71bce54fa236e52db9606c98b46f39243a87bfc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11-deactivate, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-05-21T22:09:05.920 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:09:05 smithi184 podman[75250]: 2024-05-21 22:09:05.920007821 +0000 UTC m=+0.382894516 container attach c8d17e6899d6095085171ac2e71bce54fa236e52db9606c98b46f39243a87bfc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-05-21T22:09:06.236 INFO:journalctl@ceph.osd.11.smithi184.stdout:May 21 22:09:06 smithi184 podman[75250]: 2024-05-21 22:09:06.155335792 +0000 UTC m=+0.618222497 container died c8d17e6899d6095085171ac2e71bce54fa236e52db9606c98b46f39243a87bfc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-osd-11-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-05-21T22:09:06.751 DEBUG:teuthology.orchestra.run.smithi184:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@osd.11.service' 2024-05-21T22:09:06.787 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T22:09:06.788 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-05-21T22:09:06.788 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-05-21T22:09:06.788 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@mds.a 2024-05-21T22:09:07.203 INFO:journalctl@ceph.mds.a.smithi151.stdout:May 21 22:09:06 smithi151 systemd[1]: Stopping Ceph mds.a for c8c7a2e6-17ba-11ef-bc9a-c7b262605968... 2024-05-21T22:09:07.703 INFO:journalctl@ceph.mds.a.smithi151.stdout:May 21 22:09:07 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-a[71364]: 2024-05-21T22:09:07.389+0000 7fe36f800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T22:09:07.703 INFO:journalctl@ceph.mds.a.smithi151.stdout:May 21 22:09:07 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-a[71364]: 2024-05-21T22:09:07.389+0000 7fe36f800700 -1 mds.a *** got signal Terminated *** 2024-05-21T22:09:11.703 INFO:journalctl@ceph.mds.a.smithi151.stdout:May 21 22:09:11 smithi151 podman[122142]: 2024-05-21 22:09:11.401159604 +0000 UTC m=+4.140686964 container died 257d6d88cf359fe1af275777577778de79cd6c14ba68e0f7de4aa079c280fc5e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-a, ceph=True, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-21T22:09:12.201 INFO:journalctl@ceph.mds.a.smithi151.stdout:May 21 22:09:11 smithi151 podman[122142]: 2024-05-21 22:09:11.881890767 +0000 UTC m=+4.621418131 container cleanup 257d6d88cf359fe1af275777577778de79cd6c14ba68e0f7de4aa079c280fc5e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-a, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-05-21T22:09:12.201 INFO:journalctl@ceph.mds.a.smithi151.stdout:May 21 22:09:11 smithi151 bash[122142]: ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-a 2024-05-21T22:09:12.453 INFO:journalctl@ceph.mds.a.smithi151.stdout:May 21 22:09:12 smithi151 podman[122154]: 2024-05-21 22:09:12.200614069 +0000 UTC m=+0.776744322 container remove 257d6d88cf359fe1af275777577778de79cd6c14ba68e0f7de4aa079c280fc5e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-a, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-05-21T22:09:12.488 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@mds.a.service' 2024-05-21T22:09:12.595 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T22:09:12.596 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-05-21T22:09:12.596 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-05-21T22:09:12.596 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@mds.d 2024-05-21T22:09:13.184 INFO:journalctl@ceph.mds.d.smithi151.stdout:May 21 22:09:12 smithi151 systemd[1]: Stopping Ceph mds.d for c8c7a2e6-17ba-11ef-bc9a-c7b262605968... 2024-05-21T22:09:13.453 INFO:journalctl@ceph.mds.d.smithi151.stdout:May 21 22:09:13 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-d[70133]: 2024-05-21T22:09:13.182+0000 7f7257400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T22:09:13.453 INFO:journalctl@ceph.mds.d.smithi151.stdout:May 21 22:09:13 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-d[70133]: 2024-05-21T22:09:13.182+0000 7f7257400700 -1 mds.d *** got signal Terminated *** 2024-05-21T22:09:16.302 INFO:journalctl@ceph.mds.d.smithi151.stdout:May 21 22:09:16 smithi151 podman[122262]: 2024-05-21 22:09:16.027897494 +0000 UTC m=+2.961333168 container died 3dfebcc084a3ba55d4ad88c1d1bd9fcbfc2c3a700368fd851fa4160809e2ddb7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-d, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=) 2024-05-21T22:09:16.696 INFO:journalctl@ceph.mds.d.smithi151.stdout:May 21 22:09:16 smithi151 podman[122262]: 2024-05-21 22:09:16.412024943 +0000 UTC m=+3.345460595 container cleanup 3dfebcc084a3ba55d4ad88c1d1bd9fcbfc2c3a700368fd851fa4160809e2ddb7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-d, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514) 2024-05-21T22:09:16.696 INFO:journalctl@ceph.mds.d.smithi151.stdout:May 21 22:09:16 smithi151 bash[122262]: ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-d 2024-05-21T22:09:16.952 INFO:journalctl@ceph.mds.d.smithi151.stdout:May 21 22:09:16 smithi151 podman[122275]: 2024-05-21 22:09:16.695213565 +0000 UTC m=+0.650562282 container remove 3dfebcc084a3ba55d4ad88c1d1bd9fcbfc2c3a700368fd851fa4160809e2ddb7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-d, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-05-21T22:09:17.209 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@mds.d.service' 2024-05-21T22:09:17.308 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T22:09:17.308 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-05-21T22:09:17.308 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-05-21T22:09:17.309 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@mds.g 2024-05-21T22:09:17.856 INFO:journalctl@ceph.mds.g.smithi151.stdout:May 21 22:09:17 smithi151 systemd[1]: Stopping Ceph mds.g for c8c7a2e6-17ba-11ef-bc9a-c7b262605968... 2024-05-21T22:09:18.203 INFO:journalctl@ceph.mds.g.smithi151.stdout:May 21 22:09:17 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-g[70955]: 2024-05-21T22:09:17.854+0000 7f0c0bc00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T22:09:18.203 INFO:journalctl@ceph.mds.g.smithi151.stdout:May 21 22:09:17 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-g[70955]: 2024-05-21T22:09:17.854+0000 7f0c0bc00700 -1 mds.g *** got signal Terminated *** 2024-05-21T22:09:20.263 INFO:journalctl@ceph.mds.g.smithi151.stdout:May 21 22:09:20 smithi151 podman[122383]: 2024-05-21 22:09:20.010083843 +0000 UTC m=+2.275869505 container died d3e49631a0d408dd159d6e2d6f2f636fbaa457a01825abcb3f5c1db2ebc2abe8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-g, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-21T22:09:20.544 INFO:journalctl@ceph.mds.g.smithi151.stdout:May 21 22:09:20 smithi151 podman[122383]: 2024-05-21 22:09:20.367128659 +0000 UTC m=+2.632914315 container cleanup d3e49631a0d408dd159d6e2d6f2f636fbaa457a01825abcb3f5c1db2ebc2abe8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-g, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-05-21T22:09:20.544 INFO:journalctl@ceph.mds.g.smithi151.stdout:May 21 22:09:20 smithi151 bash[122383]: ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-g 2024-05-21T22:09:20.953 INFO:journalctl@ceph.mds.g.smithi151.stdout:May 21 22:09:20 smithi151 podman[122395]: 2024-05-21 22:09:20.543193449 +0000 UTC m=+0.517799348 container remove d3e49631a0d408dd159d6e2d6f2f636fbaa457a01825abcb3f5c1db2ebc2abe8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-g, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-05-21T22:09:21.101 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@mds.g.service' 2024-05-21T22:09:21.205 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T22:09:21.205 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-05-21T22:09:21.205 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-05-21T22:09:21.206 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@mds.j 2024-05-21T22:09:21.703 INFO:journalctl@ceph.mds.j.smithi151.stdout:May 21 22:09:21 smithi151 systemd[1]: Stopping Ceph mds.j for c8c7a2e6-17ba-11ef-bc9a-c7b262605968... 2024-05-21T22:09:22.203 INFO:journalctl@ceph.mds.j.smithi151.stdout:May 21 22:09:21 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-j[69722]: 2024-05-21T22:09:21.842+0000 7f6797800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T22:09:22.203 INFO:journalctl@ceph.mds.j.smithi151.stdout:May 21 22:09:21 smithi151 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-j[69722]: 2024-05-21T22:09:21.843+0000 7f6797800700 -1 mds.j *** got signal Terminated *** 2024-05-21T22:09:23.783 INFO:journalctl@ceph.mds.j.smithi151.stdout:May 21 22:09:23 smithi151 podman[122501]: 2024-05-21 22:09:23.519118696 +0000 UTC m=+1.849041883 container died d9eec75ffa5a3f1e595e2ba8321f3be8100eb7a90a9b6338f0b592ab524197a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-j, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-05-21T22:09:24.064 INFO:journalctl@ceph.mds.j.smithi151.stdout:May 21 22:09:23 smithi151 podman[122501]: 2024-05-21 22:09:23.896205047 +0000 UTC m=+2.226128251 container cleanup d9eec75ffa5a3f1e595e2ba8321f3be8100eb7a90a9b6338f0b592ab524197a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-j, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2) 2024-05-21T22:09:24.065 INFO:journalctl@ceph.mds.j.smithi151.stdout:May 21 22:09:23 smithi151 bash[122501]: ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-j 2024-05-21T22:09:24.453 INFO:journalctl@ceph.mds.j.smithi151.stdout:May 21 22:09:24 smithi151 podman[122525]: 2024-05-21 22:09:24.064088146 +0000 UTC m=+0.531820715 container remove d9eec75ffa5a3f1e595e2ba8321f3be8100eb7a90a9b6338f0b592ab524197a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-j, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-05-21T22:09:24.551 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@mds.j.service' 2024-05-21T22:09:24.655 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T22:09:24.656 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-05-21T22:09:24.656 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-05-21T22:09:24.656 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@mds.b 2024-05-21T22:09:25.019 INFO:journalctl@ceph.mds.b.smithi164.stdout:May 21 22:09:24 smithi164 systemd[1]: Stopping Ceph mds.b for c8c7a2e6-17ba-11ef-bc9a-c7b262605968... 2024-05-21T22:09:25.284 INFO:journalctl@ceph.mds.b.smithi164.stdout:May 21 22:09:25 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-b[66241]: 2024-05-21T22:09:25.018+0000 7fbe680d8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T22:09:25.284 INFO:journalctl@ceph.mds.b.smithi164.stdout:May 21 22:09:25 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-b[66241]: 2024-05-21T22:09:25.018+0000 7fbe680d8700 -1 mds.b *** got signal Terminated *** 2024-05-21T22:09:28.235 INFO:journalctl@ceph.mds.b.smithi164.stdout:May 21 22:09:27 smithi164 podman[73628]: 2024-05-21 22:09:27.970165239 +0000 UTC m=+3.120439868 container died 8598cc3b89fc922a00957b9961e3deb1bb83bcd9ff7cce7f4b4ca855d4223593 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-b, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-05-21T22:09:28.536 INFO:journalctl@ceph.mds.b.smithi164.stdout:May 21 22:09:28 smithi164 podman[73628]: 2024-05-21 22:09:28.32638624 +0000 UTC m=+3.476660867 container cleanup 8598cc3b89fc922a00957b9961e3deb1bb83bcd9ff7cce7f4b4ca855d4223593 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-b, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-21T22:09:28.536 INFO:journalctl@ceph.mds.b.smithi164.stdout:May 21 22:09:28 smithi164 bash[73628]: ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-b 2024-05-21T22:09:28.770 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@mds.b.service' 2024-05-21T22:09:28.806 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T22:09:28.806 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-05-21T22:09:28.806 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-05-21T22:09:28.806 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@mds.e 2024-05-21T22:09:29.286 INFO:journalctl@ceph.mds.e.smithi164.stdout:May 21 22:09:28 smithi164 systemd[1]: Stopping Ceph mds.e for c8c7a2e6-17ba-11ef-bc9a-c7b262605968... 2024-05-21T22:09:29.286 INFO:journalctl@ceph.mds.e.smithi164.stdout:May 21 22:09:29 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-e[65623]: 2024-05-21T22:09:29.104+0000 7f2948a0b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T22:09:29.286 INFO:journalctl@ceph.mds.e.smithi164.stdout:May 21 22:09:29 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-e[65623]: 2024-05-21T22:09:29.104+0000 7f2948a0b700 -1 mds.e *** got signal Terminated *** 2024-05-21T22:09:32.246 INFO:journalctl@ceph.mds.e.smithi164.stdout:May 21 22:09:31 smithi164 podman[73760]: 2024-05-21 22:09:31.96950206 +0000 UTC m=+2.948066992 container died 0bfde453384df9e536c07290e67ae58ce0063add83907a05e05c3039ac7b7b76 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-e, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-05-21T22:09:32.246 INFO:journalctl@ceph.mds.e.smithi164.stdout:May 21 22:09:32 smithi164 podman[73760]: 2024-05-21 22:09:32.246367551 +0000 UTC m=+3.224932463 container cleanup 0bfde453384df9e536c07290e67ae58ce0063add83907a05e05c3039ac7b7b76 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-e, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-21T22:09:32.536 INFO:journalctl@ceph.mds.e.smithi164.stdout:May 21 22:09:32 smithi164 bash[73760]: ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-e 2024-05-21T22:09:32.536 INFO:journalctl@ceph.mds.e.smithi164.stdout:May 21 22:09:32 smithi164 podman[73772]: 2024-05-21 22:09:32.439608177 +0000 UTC m=+0.468435626 container remove 0bfde453384df9e536c07290e67ae58ce0063add83907a05e05c3039ac7b7b76 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-e, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-05-21T22:09:32.677 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@mds.e.service' 2024-05-21T22:09:32.709 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T22:09:32.709 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-05-21T22:09:32.709 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-05-21T22:09:32.709 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@mds.h 2024-05-21T22:09:33.017 INFO:journalctl@ceph.mds.h.smithi164.stdout:May 21 22:09:32 smithi164 systemd[1]: Stopping Ceph mds.h for c8c7a2e6-17ba-11ef-bc9a-c7b262605968... 2024-05-21T22:09:33.286 INFO:journalctl@ceph.mds.h.smithi164.stdout:May 21 22:09:33 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-h[65939]: 2024-05-21T22:09:33.016+0000 7fe7dd375700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T22:09:33.286 INFO:journalctl@ceph.mds.h.smithi164.stdout:May 21 22:09:33 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-h[65939]: 2024-05-21T22:09:33.016+0000 7fe7dd375700 -1 mds.h *** got signal Terminated *** 2024-05-21T22:09:36.286 INFO:journalctl@ceph.mds.h.smithi164.stdout:May 21 22:09:35 smithi164 podman[73882]: 2024-05-21 22:09:35.975755607 +0000 UTC m=+3.040305631 container died 8738418e32525b423defbafc15f0a15c2bc9923b5e5a6a4134a880b2f7ce728e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-21T22:09:36.567 INFO:journalctl@ceph.mds.h.smithi164.stdout:May 21 22:09:36 smithi164 podman[73882]: 2024-05-21 22:09:36.383745891 +0000 UTC m=+3.448295913 container cleanup 8738418e32525b423defbafc15f0a15c2bc9923b5e5a6a4134a880b2f7ce728e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-h, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea) 2024-05-21T22:09:36.568 INFO:journalctl@ceph.mds.h.smithi164.stdout:May 21 22:09:36 smithi164 bash[73882]: ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-h 2024-05-21T22:09:36.708 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@mds.h.service' 2024-05-21T22:09:36.739 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T22:09:36.739 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-05-21T22:09:36.740 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-05-21T22:09:36.740 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@mds.k 2024-05-21T22:09:37.036 INFO:journalctl@ceph.mds.k.smithi164.stdout:May 21 22:09:36 smithi164 systemd[1]: Stopping Ceph mds.k for c8c7a2e6-17ba-11ef-bc9a-c7b262605968... 2024-05-21T22:09:37.536 INFO:journalctl@ceph.mds.k.smithi164.stdout:May 21 22:09:37 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-k[65312]: 2024-05-21T22:09:37.037+0000 7fac87520700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T22:09:37.536 INFO:journalctl@ceph.mds.k.smithi164.stdout:May 21 22:09:37 smithi164 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-k[65312]: 2024-05-21T22:09:37.037+0000 7fac87520700 -1 mds.k *** got signal Terminated *** 2024-05-21T22:09:40.286 INFO:journalctl@ceph.mds.k.smithi164.stdout:May 21 22:09:39 smithi164 podman[74004]: 2024-05-21 22:09:39.97480814 +0000 UTC m=+3.022322861 container died 7865d2590247b9bd0a85bcf77fc76cf86eff856c6fdd1e942dcbb03e8f93977c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-k, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-05-21T22:09:40.706 INFO:journalctl@ceph.mds.k.smithi164.stdout:May 21 22:09:40 smithi164 podman[74004]: 2024-05-21 22:09:40.295102555 +0000 UTC m=+3.342617274 container cleanup 7865d2590247b9bd0a85bcf77fc76cf86eff856c6fdd1e942dcbb03e8f93977c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-k, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-05-21T22:09:40.706 INFO:journalctl@ceph.mds.k.smithi164.stdout:May 21 22:09:40 smithi164 bash[74004]: ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-k 2024-05-21T22:09:40.706 INFO:journalctl@ceph.mds.k.smithi164.stdout:May 21 22:09:40 smithi164 podman[74015]: 2024-05-21 22:09:40.496501104 +0000 UTC m=+0.518114398 container remove 7865d2590247b9bd0a85bcf77fc76cf86eff856c6fdd1e942dcbb03e8f93977c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-k, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5) 2024-05-21T22:09:40.716 DEBUG:teuthology.orchestra.run.smithi164:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@mds.k.service' 2024-05-21T22:09:40.747 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T22:09:40.747 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-05-21T22:09:40.747 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-05-21T22:09:40.747 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@mds.c 2024-05-21T22:09:41.223 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 21 22:09:40 smithi184 systemd[1]: Stopping Ceph mds.c for c8c7a2e6-17ba-11ef-bc9a-c7b262605968... 2024-05-21T22:09:41.486 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 21 22:09:41 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-c[67511]: 2024-05-21T22:09:41.279+0000 7f0090522700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T22:09:41.486 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 21 22:09:41 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-c[67511]: 2024-05-21T22:09:41.279+0000 7f0090522700 -1 mds.c *** got signal Terminated *** 2024-05-21T22:09:44.236 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 21 22:09:43 smithi184 podman[75356]: 2024-05-21 22:09:43.981800371 +0000 UTC m=+3.045719715 container died 6364be5c643d85b7a0a3cc1969a2574ab74c8457d3bec2769f05cabbd13d84d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-c, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-05-21T22:09:44.888 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 21 22:09:44 smithi184 podman[75356]: 2024-05-21 22:09:44.704572452 +0000 UTC m=+3.768491787 container cleanup 6364be5c643d85b7a0a3cc1969a2574ab74c8457d3bec2769f05cabbd13d84d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-c, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, GIT_CLEAN=True) 2024-05-21T22:09:44.889 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 21 22:09:44 smithi184 bash[75356]: ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-c 2024-05-21T22:09:45.140 INFO:journalctl@ceph.mds.c.smithi184.stdout:May 21 22:09:44 smithi184 podman[75380]: 2024-05-21 22:09:44.888999265 +0000 UTC m=+0.906683274 container remove 6364be5c643d85b7a0a3cc1969a2574ab74c8457d3bec2769f05cabbd13d84d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-c, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=) 2024-05-21T22:09:45.145 DEBUG:teuthology.orchestra.run.smithi184:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@mds.c.service' 2024-05-21T22:09:45.181 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T22:09:45.181 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-05-21T22:09:45.181 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-05-21T22:09:45.181 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@mds.f 2024-05-21T22:09:45.486 INFO:journalctl@ceph.mds.f.smithi184.stdout:May 21 22:09:45 smithi184 systemd[1]: Stopping Ceph mds.f for c8c7a2e6-17ba-11ef-bc9a-c7b262605968... 2024-05-21T22:09:45.986 INFO:journalctl@ceph.mds.f.smithi184.stdout:May 21 22:09:45 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-f[67208]: 2024-05-21T22:09:45.508+0000 7fb1faaf9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T22:09:45.986 INFO:journalctl@ceph.mds.f.smithi184.stdout:May 21 22:09:45 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-f[67208]: 2024-05-21T22:09:45.508+0000 7fb1faaf9700 -1 mds.f *** got signal Terminated *** 2024-05-21T22:09:48.236 INFO:journalctl@ceph.mds.f.smithi184.stdout:May 21 22:09:47 smithi184 podman[75488]: 2024-05-21 22:09:47.966503793 +0000 UTC m=+2.569945138 container died e13fed9be1b512cee5c2dc5487666b6feb2ecf2193c40e0cc9bbc2e24309839e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-f, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.build-date=20240514, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-21T22:09:48.617 INFO:journalctl@ceph.mds.f.smithi184.stdout:May 21 22:09:48 smithi184 podman[75488]: 2024-05-21 22:09:48.341826047 +0000 UTC m=+2.945267382 container cleanup e13fed9be1b512cee5c2dc5487666b6feb2ecf2193c40e0cc9bbc2e24309839e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-f, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-05-21T22:09:48.617 INFO:journalctl@ceph.mds.f.smithi184.stdout:May 21 22:09:48 smithi184 bash[75488]: ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-f 2024-05-21T22:09:48.775 DEBUG:teuthology.orchestra.run.smithi184:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@mds.f.service' 2024-05-21T22:09:48.807 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T22:09:48.807 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-05-21T22:09:48.808 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-05-21T22:09:48.808 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@mds.i 2024-05-21T22:09:49.120 INFO:journalctl@ceph.mds.i.smithi184.stdout:May 21 22:09:48 smithi184 systemd[1]: Stopping Ceph mds.i for c8c7a2e6-17ba-11ef-bc9a-c7b262605968... 2024-05-21T22:09:49.486 INFO:journalctl@ceph.mds.i.smithi184.stdout:May 21 22:09:49 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-i[67811]: 2024-05-21T22:09:49.119+0000 7f84ad52e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T22:09:49.486 INFO:journalctl@ceph.mds.i.smithi184.stdout:May 21 22:09:49 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-i[67811]: 2024-05-21T22:09:49.119+0000 7f84ad52e700 -1 mds.i *** got signal Terminated *** 2024-05-21T22:09:52.236 INFO:journalctl@ceph.mds.i.smithi184.stdout:May 21 22:09:51 smithi184 podman[75608]: 2024-05-21 22:09:51.976146305 +0000 UTC m=+2.944499556 container died 1fa7a2f12571eda44f1771cbf7d4e32e7114a6b87a3f6ed9f7db2012a366f6e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-i, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-21T22:09:52.668 INFO:journalctl@ceph.mds.i.smithi184.stdout:May 21 22:09:52 smithi184 podman[75608]: 2024-05-21 22:09:52.311895323 +0000 UTC m=+3.280248595 container cleanup 1fa7a2f12571eda44f1771cbf7d4e32e7114a6b87a3f6ed9f7db2012a366f6e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-i, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-05-21T22:09:52.668 INFO:journalctl@ceph.mds.i.smithi184.stdout:May 21 22:09:52 smithi184 bash[75608]: ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-i 2024-05-21T22:09:52.669 INFO:journalctl@ceph.mds.i.smithi184.stdout:May 21 22:09:52 smithi184 podman[75622]: 2024-05-21 22:09:52.457938592 +0000 UTC m=+0.476627609 container remove 1fa7a2f12571eda44f1771cbf7d4e32e7114a6b87a3f6ed9f7db2012a366f6e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-i, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea) 2024-05-21T22:09:52.686 DEBUG:teuthology.orchestra.run.smithi184:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@mds.i.service' 2024-05-21T22:09:52.717 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T22:09:52.718 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-05-21T22:09:52.718 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-05-21T22:09:52.718 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@mds.l 2024-05-21T22:09:52.986 INFO:journalctl@ceph.mds.l.smithi184.stdout:May 21 22:09:52 smithi184 systemd[1]: Stopping Ceph mds.l for c8c7a2e6-17ba-11ef-bc9a-c7b262605968... 2024-05-21T22:09:53.486 INFO:journalctl@ceph.mds.l.smithi184.stdout:May 21 22:09:53 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-l[66907]: 2024-05-21T22:09:53.073+0000 7f38eefea700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-21T22:09:53.486 INFO:journalctl@ceph.mds.l.smithi184.stdout:May 21 22:09:53 smithi184 ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-l[66907]: 2024-05-21T22:09:53.073+0000 7f38eefea700 -1 mds.l *** got signal Terminated *** 2024-05-21T22:09:56.236 INFO:journalctl@ceph.mds.l.smithi184.stdout:May 21 22:09:55 smithi184 podman[75729]: 2024-05-21 22:09:55.973109991 +0000 UTC m=+3.035606551 container died 75cdbe32c0e3c375f95eec372eec98ad8d33d7530451d3986c3502918b941f42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-l, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-05-21T22:09:56.682 INFO:journalctl@ceph.mds.l.smithi184.stdout:May 21 22:09:56 smithi184 podman[75729]: 2024-05-21 22:09:56.490642143 +0000 UTC m=+3.553138725 container cleanup 75cdbe32c0e3c375f95eec372eec98ad8d33d7530451d3986c3502918b941f42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea8599a585f994c0ad8ea36c128b322754a11186a9b021a6d94f81f10cc23e28, name=ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-l, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-overdrive-export-3b748ea, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-05-21T22:09:56.683 INFO:journalctl@ceph.mds.l.smithi184.stdout:May 21 22:09:56 smithi184 bash[75729]: ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968-mds-l 2024-05-21T22:09:56.891 DEBUG:teuthology.orchestra.run.smithi184:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c8c7a2e6-17ba-11ef-bc9a-c7b262605968@mds.l.service' 2024-05-21T22:09:56.921 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-21T22:09:56.921 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-05-21T22:09:56.921 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 --force --keep-logs 2024-05-21T22:09:57.243 INFO:teuthology.orchestra.run.smithi151.stdout:Deleting cluster with fsid: c8c7a2e6-17ba-11ef-bc9a-c7b262605968 2024-05-21T22:10:02.144 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 --force --keep-logs 2024-05-21T22:10:02.338 INFO:teuthology.orchestra.run.smithi164.stdout:Deleting cluster with fsid: c8c7a2e6-17ba-11ef-bc9a-c7b262605968 2024-05-21T22:10:04.806 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 --force --keep-logs 2024-05-21T22:10:04.998 INFO:teuthology.orchestra.run.smithi184.stdout:Deleting cluster with fsid: c8c7a2e6-17ba-11ef-bc9a-c7b262605968 2024-05-21T22:10:07.294 DEBUG:teuthology.orchestra.run.smithi151:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-21T22:10:07.386 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-21T22:10:07.417 DEBUG:teuthology.orchestra.run.smithi184:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-21T22:10:07.442 INFO:tasks.cephadm:Archiving crash dumps... 2024-05-21T22:10:07.444 DEBUG:teuthology.misc:Transferring archived files from smithi151:/var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-05-21_21:13:35-fs-wip-lusov-quiesce-overdrive-export-distro-default-smithi/7719034/remote/smithi151/crash 2024-05-21T22:10:07.445 DEBUG:teuthology.orchestra.run.smithi151:> sudo tar c -f - -C /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/crash -- . 2024-05-21T22:10:07.543 INFO:teuthology.orchestra.run.smithi151.stderr:tar: /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/crash: Cannot open: No such file or directory 2024-05-21T22:10:07.543 INFO:teuthology.orchestra.run.smithi151.stderr:tar: Error is not recoverable: exiting now 2024-05-21T22:10:07.548 DEBUG:teuthology.misc:Transferring archived files from smithi164:/var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-05-21_21:13:35-fs-wip-lusov-quiesce-overdrive-export-distro-default-smithi/7719034/remote/smithi164/crash 2024-05-21T22:10:07.549 DEBUG:teuthology.orchestra.run.smithi164:> sudo tar c -f - -C /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/crash -- . 2024-05-21T22:10:07.586 INFO:teuthology.orchestra.run.smithi164.stderr:tar: /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/crash: Cannot open: No such file or directory 2024-05-21T22:10:07.586 INFO:teuthology.orchestra.run.smithi164.stderr:tar: Error is not recoverable: exiting now 2024-05-21T22:10:07.588 DEBUG:teuthology.misc:Transferring archived files from smithi184:/var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-05-21_21:13:35-fs-wip-lusov-quiesce-overdrive-export-distro-default-smithi/7719034/remote/smithi184/crash 2024-05-21T22:10:07.589 DEBUG:teuthology.orchestra.run.smithi184:> sudo tar c -f - -C /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/crash -- . 2024-05-21T22:10:07.631 INFO:teuthology.orchestra.run.smithi184.stderr:tar: /var/lib/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/crash: Cannot open: No such file or directory 2024-05-21T22:10:07.632 INFO:teuthology.orchestra.run.smithi184.stderr:tar: Error is not recoverable: exiting now 2024-05-21T22:10:07.633 INFO:tasks.cephadm:Checking cluster log for badness... 2024-05-21T22:10:07.633 DEBUG:teuthology.orchestra.run.smithi151:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-05-21T22:10:07.738 INFO:tasks.cephadm:Compressing logs... 2024-05-21T22:10:07.738 DEBUG:teuthology.orchestra.run.smithi151:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-21T22:10:07.742 DEBUG:teuthology.orchestra.run.smithi164:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-21T22:10:07.745 DEBUG:teuthology.orchestra.run.smithi184:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-21T22:10:07.768 INFO:teuthology.orchestra.run.smithi184.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph.log 2024-05-21T22:10:07.768 INFO:teuthology.orchestra.run.smithi184.stderr:: No such file or directory 2024-05-21T22:10:07.768 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-osd.2.log 2024-05-21T22:10:07.768 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-mds.f.log 2024-05-21T22:10:07.768 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph.log: /var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph.audit.log 2024-05-21T22:10:07.769 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-osd.5.log 2024-05-21T22:10:07.769 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-mds.c.log 2024-05-21T22:10:07.769 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-mds.f.log: /var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph.audit.log: /var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-mon.c.log 2024-05-21T22:10:07.771 INFO:teuthology.orchestra.run.smithi164.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph.log 2024-05-21T22:10:07.771 INFO:teuthology.orchestra.run.smithi164.stderr:: No such file or directory 2024-05-21T22:10:07.771 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph.audit.log 2024-05-21T22:10:07.771 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-osd.7.log 2024-05-21T22:10:07.771 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph.log: /var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-mds.b.log 2024-05-21T22:10:07.774 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-osd.4.log 2024-05-21T22:10:07.774 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-osd.1.log 2024-05-21T22:10:07.774 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-mgr.y.log 2024-05-21T22:10:07.774 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-mds.h.log 2024-05-21T22:10:07.774 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-osd.10.log 2024-05-21T22:10:07.774 INFO:teuthology.orchestra.run.smithi164.stderr: 87.9% -- replaced with /var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph.log.gz 2024-05-21T22:10:07.774 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-mds.h.log: 90.5% -- replaced with /var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph.audit.log.gz 2024-05-21T22:10:07.776 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-mgr.z.log 2024-05-21T22:10:07.776 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-mon.c.log: /var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-mgr.z.log: 87.9% -- replaced with /var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph.log.gz 2024-05-21T22:10:07.776 INFO:teuthology.orchestra.run.smithi184.stderr: 90.5% -- replaced with /var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph.audit.log.gz 2024-05-21T22:10:07.776 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-mds.i.log 2024-05-21T22:10:07.776 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph.cephadm.log 2024-05-21T22:10:07.777 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-osd.11.log 2024-05-21T22:10:07.777 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph.cephadm.log: 83.7% -- replaced with /var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph.cephadm.log.gz 2024-05-21T22:10:07.777 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-osd.8.log 2024-05-21T22:10:07.781 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-osd.11.log: 92.1% -- replaced with /var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-mgr.z.log.gz 2024-05-21T22:10:07.781 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-volume.log 2024-05-21T22:10:07.783 INFO:teuthology.orchestra.run.smithi164.stderr: 92.1% -- replaced with /var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-mgr.y.log.gz 2024-05-21T22:10:07.783 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph.cephadm.log 2024-05-21T22:10:07.784 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-mds.e.log 2024-05-21T22:10:07.784 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-volume.log 2024-05-21T22:10:07.785 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-mds.e.log: 83.8% -- replaced with /var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph.cephadm.log.gz 2024-05-21T22:10:07.793 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-mds.l.log 2024-05-21T22:10:07.798 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-mon.b.log 2024-05-21T22:10:07.799 INFO:teuthology.orchestra.run.smithi151.stderr:find: ‘/var/log/rbd-target-api’: No such file or directorygzip 2024-05-21T22:10:07.799 INFO:teuthology.orchestra.run.smithi151.stderr: -5 --verbose -- /var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph.log 2024-05-21T22:10:07.800 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-21T22:10:07.800 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph.audit.log 2024-05-21T22:10:07.801 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-mds.g.log 2024-05-21T22:10:07.802 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-mds.j.log 2024-05-21T22:10:07.803 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-osd.3.log 2024-05-21T22:10:07.804 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-mds.j.log: 87.9% -- replaced with /var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph.log.gz 2024-05-21T22:10:07.805 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-osd.0.log 2024-05-21T22:10:07.806 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-mgr.x.log 2024-05-21T22:10:07.807 INFO:teuthology.orchestra.run.smithi151.stderr: 90.4% -- replaced with /var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph.audit.log.gz 2024-05-21T22:10:07.808 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-mon.a.log 2024-05-21T22:10:07.809 INFO:teuthology.orchestra.run.smithi151.stderr: 94.4% -- replaced with /var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-mds.g.log.gz 2024-05-21T22:10:07.809 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-osd.6.log 2024-05-21T22:10:07.809 INFO:teuthology.orchestra.run.smithi151.stderr: 94.4% -- replaced with /var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-mds.j.log.gz 2024-05-21T22:10:07.810 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph.cephadm.log 2024-05-21T22:10:07.810 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-mds.k.log 2024-05-21T22:10:07.811 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-mds.a.log 2024-05-21T22:10:07.812 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-mon.b.log: 91.0% -- replaced with /var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-volume.log.gz 2024-05-21T22:10:07.812 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-21T22:10:07.813 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-volume.log 2024-05-21T22:10:07.813 INFO:teuthology.orchestra.run.smithi151.stderr: 84.1% -- replaced with /var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph.cephadm.log.gz 2024-05-21T22:10:07.813 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-mds.d.log 2024-05-21T22:10:07.816 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-volume.log: /var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-osd.9.log 2024-05-21T22:10:07.820 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-21T22:10:07.822 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-mds.l.log: /var/log/ceph/cephadm.log: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-21T22:10:07.822 INFO:teuthology.orchestra.run.smithi184.stderr: 91.0% -- replaced with /var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-volume.log.gz 2024-05-21T22:10:07.833 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-mds.k.log: /var/log/ceph/cephadm.log: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-21T22:10:07.846 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-osd.9.log: /var/log/ceph/cephadm.log: 90.9% -- replaced with /var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-volume.log.gz 2024-05-21T22:10:07.852 INFO:teuthology.orchestra.run.smithi184.stderr: 92.6% -- replaced with /var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-mds.c.log.gz 2024-05-21T22:10:07.852 INFO:teuthology.orchestra.run.smithi151.stderr: 94.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-21T22:10:07.866 INFO:teuthology.orchestra.run.smithi184.stderr: 94.0% -- replaced with /var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-mds.f.log.gz 2024-05-21T22:10:07.866 INFO:teuthology.orchestra.run.smithi164.stderr: 92.8% -- replaced with /var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-mds.e.log.gz 2024-05-21T22:10:07.877 INFO:teuthology.orchestra.run.smithi184.stderr: 92.6% -- replaced with /var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-mds.i.log.gz 2024-05-21T22:10:07.886 INFO:teuthology.orchestra.run.smithi164.stderr: 92.8% -- replaced with /var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-mds.h.log.gz 2024-05-21T22:10:07.888 INFO:teuthology.orchestra.run.smithi164.stderr: 94.5% -- replaced with /var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-mds.k.log.gz 2024-05-21T22:10:07.894 INFO:teuthology.orchestra.run.smithi151.stderr: 94.5% -- replaced with /var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-mds.a.log.gz 2024-05-21T22:10:07.899 INFO:teuthology.orchestra.run.smithi184.stderr: 94.4% -- replaced with /var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-mds.l.log.gz 2024-05-21T22:10:07.900 INFO:teuthology.orchestra.run.smithi151.stderr: 94.6% -- replaced with /var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-mds.d.log.gz 2024-05-21T22:10:08.124 INFO:teuthology.orchestra.run.smithi164.stderr: 93.1% -- replaced with /var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-mds.b.log.gz 2024-05-21T22:10:08.250 INFO:teuthology.orchestra.run.smithi151.stderr: 90.0% -- replaced with /var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-mgr.x.log.gz 2024-05-21T22:10:08.324 INFO:teuthology.orchestra.run.smithi164.stderr: 91.6% -- replaced with /var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-mon.b.log.gz 2024-05-21T22:10:08.341 INFO:teuthology.orchestra.run.smithi184.stderr: 91.6% -- replaced with /var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-mon.c.log.gz 2024-05-21T22:10:08.494 INFO:teuthology.orchestra.run.smithi184.stderr: 92.7% -- replaced with /var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-osd.8.log.gz 2024-05-21T22:10:08.526 INFO:teuthology.orchestra.run.smithi164.stderr: 92.8% -- replaced with /var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-osd.4.log.gz 2024-05-21T22:10:08.535 INFO:teuthology.orchestra.run.smithi184.stderr: 93.4% -- replaced with /var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-osd.11.log.gz 2024-05-21T22:10:08.630 INFO:teuthology.orchestra.run.smithi151.stderr: 92.6% -- replaced with /var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-osd.6.log.gz 2024-05-21T22:10:08.662 INFO:teuthology.orchestra.run.smithi184.stderr: 92.8% -- replaced with /var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-osd.2.log.gz 2024-05-21T22:10:08.679 INFO:teuthology.orchestra.run.smithi151.stderr: 92.7% -- replaced with /var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-osd.9.log.gz 2024-05-21T22:10:08.695 INFO:teuthology.orchestra.run.smithi164.stderr: 92.9% -- replaced with /var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-osd.1.log.gz 2024-05-21T22:10:08.821 INFO:teuthology.orchestra.run.smithi164.stderr: 93.1% -- replaced with /var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-osd.7.log.gz 2024-05-21T22:10:08.909 INFO:teuthology.orchestra.run.smithi151.stderr: 93.2% -- replaced with /var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-osd.0.log.gz 2024-05-21T22:10:09.000 INFO:teuthology.orchestra.run.smithi151.stderr: 92.9% -- replaced with /var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-osd.3.log.gz 2024-05-21T22:10:09.025 INFO:teuthology.orchestra.run.smithi164.stderr: 93.0% -- replaced with /var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-osd.10.log.gz 2024-05-21T22:10:09.027 INFO:teuthology.orchestra.run.smithi164.stderr: 2024-05-21T22:10:09.027 INFO:teuthology.orchestra.run.smithi164.stderr:real 0m1.268s 2024-05-21T22:10:09.027 INFO:teuthology.orchestra.run.smithi164.stderr:user 0m4.900s 2024-05-21T22:10:09.027 INFO:teuthology.orchestra.run.smithi164.stderr:sys 0m0.204s 2024-05-21T22:10:09.035 INFO:teuthology.orchestra.run.smithi184.stderr: 93.0% -- replaced with /var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-osd.5.log.gz 2024-05-21T22:10:09.037 INFO:teuthology.orchestra.run.smithi184.stderr: 2024-05-21T22:10:09.037 INFO:teuthology.orchestra.run.smithi184.stderr:real 0m1.279s 2024-05-21T22:10:09.037 INFO:teuthology.orchestra.run.smithi184.stderr:user 0m4.400s 2024-05-21T22:10:09.037 INFO:teuthology.orchestra.run.smithi184.stderr:sys 0m0.168s 2024-05-21T22:10:09.066 INFO:teuthology.orchestra.run.smithi151.stderr: 90.3% -- replaced with /var/log/ceph/c8c7a2e6-17ba-11ef-bc9a-c7b262605968/ceph-mon.a.log.gz 2024-05-21T22:10:09.069 INFO:teuthology.orchestra.run.smithi151.stderr: 2024-05-21T22:10:09.070 INFO:teuthology.orchestra.run.smithi151.stderr:real 0m1.295s 2024-05-21T22:10:09.070 INFO:teuthology.orchestra.run.smithi151.stderr:user 0m5.409s 2024-05-21T22:10:09.070 INFO:teuthology.orchestra.run.smithi151.stderr:sys 0m0.533s 2024-05-21T22:10:09.070 INFO:tasks.cephadm:Archiving logs... 2024-05-21T22:10:09.071 DEBUG:teuthology.misc:Transferring archived files from smithi151:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-05-21_21:13:35-fs-wip-lusov-quiesce-overdrive-export-distro-default-smithi/7719034/remote/smithi151/log 2024-05-21T22:10:09.072 DEBUG:teuthology.orchestra.run.smithi151:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-21T22:10:09.842 DEBUG:teuthology.misc:Transferring archived files from smithi164:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-05-21_21:13:35-fs-wip-lusov-quiesce-overdrive-export-distro-default-smithi/7719034/remote/smithi164/log 2024-05-21T22:10:09.844 DEBUG:teuthology.orchestra.run.smithi164:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-21T22:10:10.517 DEBUG:teuthology.misc:Transferring archived files from smithi184:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-05-21_21:13:35-fs-wip-lusov-quiesce-overdrive-export-distro-default-smithi/7719034/remote/smithi184/log 2024-05-21T22:10:10.517 DEBUG:teuthology.orchestra.run.smithi184:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-21T22:10:11.149 INFO:tasks.cephadm:Removing cluster... 2024-05-21T22:10:11.150 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 --force 2024-05-21T22:10:11.448 INFO:teuthology.orchestra.run.smithi151.stdout:Deleting cluster with fsid: c8c7a2e6-17ba-11ef-bc9a-c7b262605968 2024-05-21T22:10:12.058 DEBUG:teuthology.orchestra.run.smithi164:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 --force 2024-05-21T22:10:12.249 INFO:teuthology.orchestra.run.smithi164.stdout:Deleting cluster with fsid: c8c7a2e6-17ba-11ef-bc9a-c7b262605968 2024-05-21T22:10:12.518 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c8c7a2e6-17ba-11ef-bc9a-c7b262605968 --force 2024-05-21T22:10:12.699 INFO:teuthology.orchestra.run.smithi184.stdout:Deleting cluster with fsid: c8c7a2e6-17ba-11ef-bc9a-c7b262605968 2024-05-21T22:10:12.951 INFO:tasks.cephadm:Removing cephadm ... 2024-05-21T22:10:12.952 DEBUG:teuthology.orchestra.run.smithi151:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-21T22:10:13.008 DEBUG:teuthology.orchestra.run.smithi164:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-21T22:10:13.024 DEBUG:teuthology.orchestra.run.smithi184:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-21T22:10:13.041 INFO:tasks.cephadm:Teardown complete 2024-05-21T22:10:13.041 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-21T22:10:13.249 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-21T22:10:13.249 DEBUG:teuthology.orchestra.run.smithi151:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-21T22:10:13.257 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-21T22:10:13.259 DEBUG:teuthology.orchestra.run.smithi184:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-21T22:10:13.406 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-21T22:10:13.417 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-21T22:10:13.417 DEBUG:teuthology.orchestra.run.smithi151:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-21T22:10:13.425 DEBUG:teuthology.orchestra.run.smithi164:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-21T22:10:13.427 DEBUG:teuthology.orchestra.run.smithi184:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-21T22:10:13.440 INFO:teuthology.orchestra.run.smithi164.stderr:bash: line 1: ntpq: command not found 2024-05-21T22:10:13.442 INFO:teuthology.orchestra.run.smithi184.stderr:bash: line 1: ntpq: command not found 2024-05-21T22:10:13.444 INFO:teuthology.orchestra.run.smithi164.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-21T22:10:13.444 INFO:teuthology.orchestra.run.smithi164.stdout:=============================================================================== 2024-05-21T22:10:13.444 INFO:teuthology.orchestra.run.smithi164.stdout:^* hv01.front.sepia.ceph.com 3 6 377 25 +970us[ +979us] +/- 44ms 2024-05-21T22:10:13.444 INFO:teuthology.orchestra.run.smithi164.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 24 -1276us[-1276us] +/- 60ms 2024-05-21T22:10:13.444 INFO:teuthology.orchestra.run.smithi164.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 23 -8619ns[-8619ns] +/- 72ms 2024-05-21T22:10:13.444 INFO:teuthology.orchestra.run.smithi164.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-21T22:10:13.446 INFO:teuthology.orchestra.run.smithi184.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-21T22:10:13.446 INFO:teuthology.orchestra.run.smithi184.stdout:=============================================================================== 2024-05-21T22:10:13.446 INFO:teuthology.orchestra.run.smithi184.stdout:^* hv01.front.sepia.ceph.com 3 6 377 24 +974us[ +978us] +/- 44ms 2024-05-21T22:10:13.446 INFO:teuthology.orchestra.run.smithi184.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 24 -1275us[-1275us] +/- 60ms 2024-05-21T22:10:13.446 INFO:teuthology.orchestra.run.smithi184.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 25 +9537ns[ +13us] +/- 72ms 2024-05-21T22:10:13.446 INFO:teuthology.orchestra.run.smithi184.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-21T22:10:13.471 INFO:teuthology.orchestra.run.smithi151.stderr:bash: line 1: ntpq: command not found 2024-05-21T22:10:13.481 INFO:teuthology.orchestra.run.smithi151.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-21T22:10:13.481 INFO:teuthology.orchestra.run.smithi151.stdout:=============================================================================== 2024-05-21T22:10:13.481 INFO:teuthology.orchestra.run.smithi151.stdout:^* hv01.front.sepia.ceph.com 3 6 377 27 +1011us[+1016us] +/- 44ms 2024-05-21T22:10:13.481 INFO:teuthology.orchestra.run.smithi151.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 24 -1241us[-1241us] +/- 60ms 2024-05-21T22:10:13.481 INFO:teuthology.orchestra.run.smithi151.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 26 +10us[ +10us] +/- 72ms 2024-05-21T22:10:13.481 INFO:teuthology.orchestra.run.smithi151.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-21T22:10:13.482 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-21T22:10:13.493 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-21T22:10:13.494 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-21T22:10:13.516 DEBUG:teuthology.orchestra.run.smithi151:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-21T22:10:13.713 DEBUG:teuthology.orchestra.run.smithi164:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-21T22:10:13.780 DEBUG:teuthology.orchestra.run.smithi184:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-21T22:10:13.847 DEBUG:teuthology.orchestra.run.smithi151:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-21T22:10:13.937 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-21T22:10:13.938 DEBUG:teuthology.orchestra.run.smithi164:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-21T22:10:13.964 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-21T22:10:13.965 DEBUG:teuthology.orchestra.run.smithi184:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-21T22:10:13.991 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-21T22:10:13.992 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-21T22:10:14.004 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-21T22:10:14.025 INFO:teuthology.task.internal:Duration was 2279.438875 seconds 2024-05-21T22:10:14.026 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-21T22:10:14.057 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-21T22:10:14.058 DEBUG:teuthology.orchestra.run.smithi151:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-21T22:10:14.065 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-21T22:10:14.068 DEBUG:teuthology.orchestra.run.smithi184:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-21T22:10:14.102 INFO:teuthology.orchestra.run.smithi164.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-21T22:10:14.104 INFO:teuthology.orchestra.run.smithi184.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-21T22:10:14.208 INFO:teuthology.orchestra.run.smithi151.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-21T22:10:14.573 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-21T22:10:14.573 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi151.front.sepia.ceph.com 2024-05-21T22:10:14.573 DEBUG:teuthology.orchestra.run.smithi151:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-21T22:10:14.643 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi164.front.sepia.ceph.com 2024-05-21T22:10:14.644 DEBUG:teuthology.orchestra.run.smithi164:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-21T22:10:14.671 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi184.front.sepia.ceph.com 2024-05-21T22:10:14.671 DEBUG:teuthology.orchestra.run.smithi184:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-21T22:10:14.695 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-21T22:10:14.695 DEBUG:teuthology.orchestra.run.smithi151:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-21T22:10:14.702 DEBUG:teuthology.orchestra.run.smithi164:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-21T22:10:14.713 DEBUG:teuthology.orchestra.run.smithi184:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-21T22:10:14.782 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-21T22:10:14.783 DEBUG:teuthology.orchestra.run.smithi151:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-21T22:10:14.789 DEBUG:teuthology.orchestra.run.smithi164:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-21T22:10:14.791 DEBUG:teuthology.orchestra.run.smithi184:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-21T22:10:16.229 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-21T22:10:16.242 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-21T22:10:16.242 DEBUG:teuthology.orchestra.run.smithi151:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-21T22:10:16.319 DEBUG:teuthology.orchestra.run.smithi164:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-21T22:10:16.348 DEBUG:teuthology.orchestra.run.smithi184:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-21T22:10:16.376 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-21T22:10:16.387 DEBUG:teuthology.orchestra.run.smithi151:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-21T22:10:16.395 DEBUG:teuthology.orchestra.run.smithi164:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-21T22:10:16.397 DEBUG:teuthology.orchestra.run.smithi184:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-21T22:10:16.420 INFO:teuthology.orchestra.run.smithi164.stdout:kernel.core_pattern = core 2024-05-21T22:10:16.443 INFO:teuthology.orchestra.run.smithi184.stdout:kernel.core_pattern = core 2024-05-21T22:10:16.482 INFO:teuthology.orchestra.run.smithi151.stdout:kernel.core_pattern = core 2024-05-21T22:10:16.524 DEBUG:teuthology.orchestra.run.smithi151:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-21T22:10:16.572 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-21T22:10:16.572 DEBUG:teuthology.orchestra.run.smithi164:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-21T22:10:16.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-21T22:10:16.589 DEBUG:teuthology.orchestra.run.smithi184:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-21T22:10:16.605 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-21T22:10:16.605 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-21T22:10:16.617 INFO:teuthology.task.internal:Transferring archived files... 2024-05-21T22:10:16.618 DEBUG:teuthology.misc:Transferring archived files from smithi151:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-05-21_21:13:35-fs-wip-lusov-quiesce-overdrive-export-distro-default-smithi/7719034/remote/smithi151 2024-05-21T22:10:16.618 DEBUG:teuthology.orchestra.run.smithi151:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-21T22:10:16.742 DEBUG:teuthology.misc:Transferring archived files from smithi164:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-05-21_21:13:35-fs-wip-lusov-quiesce-overdrive-export-distro-default-smithi/7719034/remote/smithi164 2024-05-21T22:10:16.742 DEBUG:teuthology.orchestra.run.smithi164:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-21T22:10:16.779 DEBUG:teuthology.misc:Transferring archived files from smithi184:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-05-21_21:13:35-fs-wip-lusov-quiesce-overdrive-export-distro-default-smithi/7719034/remote/smithi184 2024-05-21T22:10:16.779 DEBUG:teuthology.orchestra.run.smithi184:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-21T22:10:16.817 INFO:teuthology.task.internal:Removing archive directory... 2024-05-21T22:10:16.817 DEBUG:teuthology.orchestra.run.smithi151:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-21T22:10:16.825 DEBUG:teuthology.orchestra.run.smithi164:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-21T22:10:16.827 DEBUG:teuthology.orchestra.run.smithi184:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-21T22:10:16.875 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-21T22:10:16.887 INFO:teuthology.task.internal:Not uploading archives. 2024-05-21T22:10:16.887 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-21T22:10:16.908 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-21T22:10:16.908 DEBUG:teuthology.orchestra.run.smithi151:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-21T22:10:16.914 DEBUG:teuthology.orchestra.run.smithi164:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-21T22:10:16.916 DEBUG:teuthology.orchestra.run.smithi184:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-21T22:10:16.929 INFO:teuthology.orchestra.run.smithi164.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 21 22:10 /home/ubuntu/cephtest 2024-05-21T22:10:16.933 INFO:teuthology.orchestra.run.smithi184.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 21 22:10 /home/ubuntu/cephtest 2024-05-21T22:10:16.963 INFO:teuthology.orchestra.run.smithi151.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 21 22:10 /home/ubuntu/cephtest 2024-05-21T22:10:16.966 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-21T22:10:16.978 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-21T22:10:17.062 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 4fed890e8ce3966dbe380e9eb4b0a796eddb841c description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-quiesce/with-quiesce 6-workunit/suites/fsync-tester}} duration: 2279.438875436783 flavor: default owner: scheduled_leonidus@teuthology success: true 2024-05-21T22:10:17.062 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-21T22:10:17.139 INFO:teuthology.run:pass